Owned vs rented GPU: utilisation, data locality, support

Where renting is the right answer

A card by the hour is the right tool for an experiment, a burst, a model you want to try for an afternoon, or a training run you will do once. The elasticity is real and worth paying for when the work is intermittent. The question is what happens when the work stops being intermittent: an inference endpoint that serves your application all day, a nightly fine-tune, an embedding pipeline that reprocesses a document store every week. At that point the comparison changes from price per hour to price per useful hour, and the two diverge.

Utilisation is a measurement, not an opinion

The crossover between renting and owning is a duty-cycle question, and it is answered by sampling rather than estimating. On whatever you run today, log the GPU's utilisation and memory for a representative week.

$ nvidia-smi --query-gpu=timestamp,name,utilization.gpu,memory.used,memory.total --format=csv -l 60 >> gpu-week.csv

A card that is busy for a few hours a day and idle otherwise is a rental. A card that is busy most of the day, or that must be available the instant a request arrives regardless of how busy it is, is a machine, because availability is being paid for whether or not the meter is running. The arithmetic for comparing the two is in stop comparing GPUs by the hour; the sampling above is the input it needs.

Data locality: the model is the small part

A model is a file that fits on one machine. The data around it does not: the document store it answers questions about, the embeddings built from it, the prompts and responses, the access logs an auditor will ask for. Each of those has to be where the GPU is. On a rented card, that means moving the data to the provider before the work and paying to move it out afterwards, or leaving it there and accepting that the data now lives wherever the provider's storage lives.

On an owned machine the data and the card share a rack. The storage underneath is all-flash, the network between them is ours, and nothing crosses a public cloud boundary to be processed. The example we build most often is a private inference endpoint for records that cannot leave the organisation: medical records analysed by a local model inside Digital Realty Athens, with the records, the model and the logs on the same machine and the endpoint reachable only from the customer's own network. The pattern is described in an AI endpoint that isn't on the internet.

Jurisdiction, settled by geography

For workloads that must stay under EU law, we place the machine in Digital Realty Athens or Digital Realty Amsterdam (AMS18). The hardware, the operator and the data are then under one jurisdiction, and the question an auditor asks first, who else is on this silicon and who can be compelled to reach it, has a short answer: nobody else, and the operator sitting in front of you. Shared and fractional GPU products place other tenants on the same die. A dedicated instance at a hyperscaler removes the neighbours on the card but keeps the shared hypervisor, network and storage layers, and the jurisdiction is whatever the provider's is.

Support is a person who built the box

A rented hour comes with an API and a status page. An owned machine comes with the engineers who racked it, installed the driver stack, tuned the storage path and wired the network. When a container cannot see the card after a kernel update, or the inference server's memory footprint changes between model releases, the ticket goes to someone who can read the box. The machine also changes when you decide: no reclaims, no fleet migrations, no retirement of the card you tuned for on someone else's schedule.

Which card, and how soon

Whatever the workload needs: A100, H100, L40, L40S and others, sized to the model and the batch size rather than to a catalogue tier. A 70B-class model with the right quantisation fits in 48 GB of GPU memory, which decides the card before the budget does. Machines are delivered with a 48-hour lead time into Athens, Amsterdam or Toronto, with the EU-jurisdiction placements in the first two.