Your AI endpoint doesn't have to be on the internet.
Every deployment guide assumes the endpoint is public and walks you through defending it. There is a stronger posture: absence. Because we own the network end to end, your inference endpoint can be reachable only over your own private transit, and the public internet stays out of the loop entirely.
In this article 4 sections
- Every AI deployment guide assumes the endpoint lives on the internet and walks you through defending it: auth, TLS, gateways, rate limiting, a WAF.
- There is a stronger posture available: absence. A single-tenant inference endpoint reachable only over your own private transit has no public surface to defend.
- We can offer it because we own the network end to end: our AS, our routing, an encrypted private mesh. Your prompts travel your private path to your machine and back. The public internet never carries them.
- For regulated workloads this shortens everything: the threat model, the pentest scope, the auditor's diagram, and the list of things that can go wrong at 3 a.m.
Every team deploying a private model draws the same diagram. App, then API gateway, then the endpoint, then the model, with a padlock icon standing in for the security review. The guides all agree on what comes next: authentication, TLS, rate limiting, a WAF, log monitoring. All of it sensible. All of it answering the second question while skipping the first.
The first question: why is this endpoint on the internet at all? Your documents are private. Your application is private. The model runs on a machine nobody else shares. The one public leg in the diagram exists because the diagram was copied from a world of public APIs, not because your workload needs it.
Defense is a budget. Absence is a property.
Everything internet-facing is a standing cost. The exposed surface needs patching on someone's schedule, the logs need watching, the WAF needs tuning, and the scans never stop arriving whether you look or not. All of that work is necessary, all of it is forever, and all of it is probabilistic: it lowers the odds, it cannot zero them. Removing the public leg converts the budget into a property: the oldest idea in network security, the management plane you cannot route to, applied to the newest workload in the rack.
The two architectures
Why we can build the second chain
The private path is not a product feature we toggled on. It is a consequence of what this company is. We run our own AS and our own routing, and an always-on encrypted mesh connects our datacenters, the same private fabric your replication traffic rides. For customers who want it, the inference endpoint is reachable over their own private transit into that network, and the public internet is never on the path: no public listener, no public DNS entry to find, no front door to knock on. A GPU marketplace reselling capacity in other people's facilities cannot offer this, because it is a network property, and they are not network operators.
- Public attack surface
- none: there is no internet-facing listener
- Credential stuffing and scan noise
- no public door to knock on
- Data in transit
- private path, encrypted, end to end
- The auditor's network diagram
- one private line, two endpoints you control
- What remains, honestly
- your own network's hygiene, access control on your side, and us
Where the boundary sits
Private reachability is precise about what it removes, and the claim only stays strong if it stays precise. Your own network's access control still matters: a compromised laptop on your VPN reaches whatever your VPN reaches, including this. The endpoint still authenticates, because absence removes the internet's share of the threat model, not your side of it. And workloads that genuinely want a public API, a customer-facing product, for instance, legitimately keep one and defend it properly. The claim is narrower and stronger: for internal and regulated workloads, the public leg is optional, and optional exposure should be declined. The pentest scope for a service that is not on the internet is a short document, and auditors notice.
Put the two halves of the private-AI argument together and the architecture completes itself. The pipeline that never leaves the machine (the knowledge assistant) answers what happens to your data at rest and in use. The private path answers how the data gets there and back. Between them, the question every regulated buyer starts with, who can see this, keeps its one-word answer at every layer: nobody.
“The strongest security posture for an endpoint is not a better firewall. It's not being on the internet at all.”
Straight answers
How do I secure a self-hosted LLM endpoint?+
The standard answers (auth, TLS, gateway, WAF) defend a public endpoint. The stronger question is whether it needs to be public at all: an endpoint reachable only over private transit has no internet-facing surface to defend, and the standard controls shrink to your side of the line.
What does "private transit" mean here?+
Your traffic reaches the machine over a private, encrypted path, your connectivity into our network, not across the open internet. We keep this concept-level on purpose: the design conversation is per customer, with your network team in the room.
Is this the same as air-gapped?+
It is the practical relative: not physically offline, but with no public internet path. True air-gap requirements are a different conversation we can also have. Most regulated workloads need "not on the internet," not "not on any network."
Does a private endpoint still need authentication?+
Yes. Absence of a public leg removes the internet's share of the threat model, not your own network's. Anything your private network reaches, a compromised machine on it reaches too. Auth stays; the scan noise goes.
Why can't a GPU cloud offer this?+
Because it is a network property, not a server feature. It requires owning the routing end to end: the AS, the paths, the encrypted fabric between sites. Marketplaces reselling GPUs in other people's facilities do not own a network to make private.
Your model, reachable only by you.
Bring us the model and your threat model. We'll design the private path, and the public internet stays out of the loop.
See GPU ServersYou’ve reached the end.
Put the reading to work
What are
you running?
Tell us about the workload, traffic and recovery requirements. Start with the server builder, or talk through the details with an engineer.
Written by the engineers who build and run the infrastructure it describes.Talk to an engineer