Measure before you repatriate: sizing a cloud exit from real usage

The most expensive mistake in leaving public cloud is copying the estate as it is. Cloud environments grow by provisioning, and provisioned is not used: instance sizes chosen in a hurry, volumes sized for a peak that never came, services nobody remembers starting. Move that one to one and you carry the waste into a fixed contract. Here is how to measure what actually runs before anyone draws a new design.

The problem: the bill describes what you rent, not what you use

A cloud invoice lists instance types, volume sizes and service tiers. None of that tells you how much CPU your application consumed at its busiest hour, how many IOPS the database really performed, or how much memory the cache actually holds. Those are the numbers a physical or private cloud design needs, and the cloud provider has been recording most of them for you all along.

Step one: inventory everything, not just the servers

Start with a complete list of resources across every region you have ever used. Tags help, but untagged resources are exactly the ones you need to find:

for r in $(aws ec2 describe-regions --query 'Regions[].RegionName' --output text); do
  aws resourcegroupstaggingapi get-resources --region "$r" \
    --query 'ResourceTagMappingList[].ResourceARN' --output text
done | tr '\t' '\n' | sort > inventory.txt

Then group by service. The servers are usually the easy part. The work is in what surrounds them: managed databases, caches, load balancers, DNS zones, queues, object buckets, certificates, scheduled functions, and the IAM roles and security groups that tie them together. Each one needs an owner and a decision: move, replace, keep, or retire.

Step two: read percentiles, not averages

For each compute resource, pull at least a month of utilisation, and read the 95th percentile and the maximum rather than the average. An average of a quiet night and a busy evening describes neither.

aws cloudwatch get-metric-statistics \
  --namespace AWS/EC2 --metric-name CPUUtilization \
  --dimensions Name=InstanceId,Value=<instance-id> \
  --start-time "$(date -u -d '-30 days' +%FT%TZ)" --end-time "$(date -u +%FT%TZ)" \
  --period 3600 --extended-statistics p95 \
  --query 'Datapoints[].ExtendedStatistics.p95' --output text \
  | tr '\t' '\n' | sort -n | tail -1

Memory is the gap: hypervisor metrics do not see inside the guest. If the CloudWatch agent is installed, its memory metrics are in the CWAgent namespace. If not, collect from inside the instances for a couple of weeks with sar or your existing monitoring before you size anything. Guessing memory is how designs end up either starved or wasteful.

Do the same for storage: VolumeReadOps and VolumeWriteOps per volume give you real IOPS, and VolumeReadBytes and VolumeWriteBytes the throughput. For managed databases, read CPUUtilization, FreeableMemory, ReadIOPS and WriteIOPS. For caches, the memory actually used and the connection count. For traffic, NetworkOut per instance shows where egress really comes from, which matters for designing the network, not just for the invoice.

Step three: build the sizing sheet

Put the measurements into one sheet, one row per workload:

Then size each one to its measured peak with headroom you can justify, not to the instance it runs on today. Two rules we apply to every design:

• Never give a virtual machine more vCPU than one CPU socket has physical cores, nor more memory than one NUMA node serves. A VM spread across sockets pays a remote memory penalty on every access and often runs slower than a smaller one. If a workload needs more than a socket, it needs to scale out or move to bare metal.
• Right size, then stop. Cloud estates are routinely oversized because resizing was risky. On dedicated infrastructure the right size is cheaper and frequently faster.

Step four: map every managed service to something you own

For each managed service, write down what replaces it and what changes for the application:

Some services have no sensible equivalent in your new home, or are cheap and convenient where they are. Hybrid is a legitimate answer: keep them in the cloud and connect the two privately. A repatriation plan that tries to move everything is often the one that stalls.

Step five: design for the failure domains you are leaving behind

Availability zones did real work for you, so the new design must do the same work explicitly. Map each tier to separate hosts, separate storage arrays and, where it matters, separate sites. Quorum services (databases, caches, coordinators) need three members in three failure domains. Write it down tier by tier; "it was multi AZ" is not a design.

Step six: plan the DNS move like the critical path it is

DNS is usually the last thing to move and the one that can break everything at once. If the zone is signed with DNSSEC, sequence the key and DS record change so validating resolvers never see a broken chain. We describe one customer's full move off AWS DNS on our managed DNS page.

Your checklist

• A complete inventory across all regions, with an owner and a decision per resource.
• At least a month of p95 and peak utilisation for CPU, memory, IOPS and egress.
• Memory measured from inside the guests, not guessed.
• A sizing sheet that sizes to measured peaks, with VMs kept inside one socket.
• Every managed service mapped to a replacement, or deliberately kept.
• Failure domains designed explicitly, and the DNS move sequenced.

If you are weighing a move back to owned infrastructure, we are happy to start with the measuring.

Talk to an engineer

Related: Cloud repatriation and colocation exits · Managed datacenter migrations, rehearsed · High-availability platform builds