ServingServers

The threat model

GPU time is expensive and frequently idle. Matching idle hardware to people who need it is a straightforward marketplace idea — until you write down what you are actually shipping: a renter runs their own container image, with GPU access and a real shell, on hardware belonging to someone who is trusting you.

So the renter must get a genuinely usable environment while the host machine, the platform control plane, and every other renter stay protected from them. I treated the workload as hostile by default and designed outward from there.

The isolation model

The core decision was that the user's container runs no SSH daemon at all. Putting sshd inside the workload means shipping a privileged, network-facing service into the exact environment I assume is compromised.

Instead, two sidecar containers attach to the job's network and process namespaces — one providing SSH, one a reverse proxy for browser-based access. Logins are pinned to a forced command that enters the user's namespaces and drops privileges before handing over a shell. The renter gets a normal-feeling session; the platform never executes their code with more capability than intended.

Seven phases of platform work

• Image catalogue moved from a hardcoded list into an admin-curated database table, seeded with common ML images — so adding an image is an operation, not a deploy.
• Per-job SSH sidecar built minimal and single-purpose.
• Sidecar lifecycle wired into the agent behind a feature flag, so the new path could run alongside the old one and be rolled back instantly.
• Per-job egress firewall — a dedicated iptables chain dropping private address ranges and their IPv6 equivalents, with DNS restricted to the gateway. A rented GPU has no business reaching into the host's private network.
• Inter-rental cleanup — verified workspace wipe, GPU memory reset, and quarantine for any slot that fails to clean. A slot that cannot be proven clean does not get re-let.
• Multi-image support with browser-based notebook access through the proxy sidecar.
• Billing and abuse safeguards — a versioned terms gate, auto-suspension on abuse reports, and a per-job egress byte cap with soft and hard limits on a rolling window.

Attacking my own design

Building isolation is the easy half; believing in it is the hard half. I ran two adversarial pentests against my own platform — roughly a thousand lines of security tests across six files, executing real attacks over a real SSH session rather than asserting on configuration files.

The tests tried to:

• Attach a debugger to the SSH daemon from inside the workload.
• Read another process's environment out of /proc to steal secrets.
• Fill the host's disk from the rented workspace.
• Reach the reverse proxy's admin API and rewrite routing.
• Exfiltrate data over DNS.
• Escape the container via a cgroup release-agent.

Those tests produced 17 critical, 25 high, 38 medium and 5 low findings — 85 in total, all fixed, alongside a written threat model, vulnerability log and a repeatable pentest playbook. Hardening that came out of it included one-time single-use terminal tokens with a short TTL, request body limits, content-security-policy headers, correct handling of a password-hashing length limit, and locking the proxy's admin API down to loopback and gateway traffic only.

Infrastructure debugging highlights

• Notebooks served a blank page. I had set an environment variable belonging to a different ML platform's convention rather than the notebook server's own. Switching to the correct base-URL argument, and telling the proxy to preserve the path prefix instead of stripping it, fixed it.
• "Network unreachable" on one port turned out to be connection tracking behaving unreliably in a virtualised Docker networking stack. A stateless accept rule for non-SYN traffic worked around it — the kind of fix you only find by reading the packet path rather than the app logs.
• GPU access was blocked by an over-aggressive capability drop. Scoping the device flag properly restored access without handing the capability back — the difference between "make it work" and "make it work safely".

Outcome

• An isolation model where the untrusted workload runs no privileged services of its own.
• Per-job network egress control and verified cleanup between rentals.
• 85 security findings closed, with the attacks preserved as a repeatable playbook.
• Documentation good enough that someone else can re-run the attacks and check my answers — which is the only standard worth holding security work to.