How we prepare a news site for a spike.
A news site's business is spikes. Here is the method we use to get one ready: read the right logs, know where the ceiling is, check every tier before spending, and do the arithmetic.
In this article 4 sections
- Traffic exposes ceilings; configuration sets the ceilings. Preparing for a spike means finding them before the audience does.
- First, confirm which log actually carries the site's traffic. A default server log can hold a small share while the site's own vhost log holds the rest.
- Check every tier before spending. The database is often idle-fast while the work queues one tier up.
- Then the arithmetic: workers divided by seconds-per-request is your real ceiling in requests per second. Size the pool to the peak you will actually hit, plus margin.
A news site's business is spikes: when the story breaks, the audience arrives at once. Getting ready for that is rarely a hardware question. It is a configuration question: the worker count, the timeouts, the logs, the cache. Here is how we prepare one.
Read the right log first
The first discipline is counting correctly. On a typical web server the default log and the site's own vhost log are different files, and the default can hold only a small share of the traffic. Size anything from the wrong log and you size it for the wrong load.
Know where the ceiling is
Under load, request times tell you where the limit is. Organic slowness scatters: some requests a bit slow, some very slow. Slow requests that cluster at exactly the gateway's timeout mean the tier behind the gateway is running out of workers inside its budget. That is a saturation signature, and it points straight at the setting to size.
Check every tier before you spend
Before anyone buys RAM or upgrades the data tier, check it. A database that is idle-fast, with no slow queries, answering everything the moment it is asked, is not the tier to size. The tier to size is the one whose workers are all busy. That pattern has its own post in size down, not up.
Your ceiling is arithmetic
WORKERS
SEC / REQUEST
REQ / SEC
The preparation is sizing, not shopping: the worker pool computed from the peak a news site will actually hit, plus margin, within a measured per-worker memory budget. Then a cache tier that serves stale content when the backend slows, so a busy backend degrades to slightly-old pages instead of error pages. The platform absorbs the spike.
“Traffic finds the ceiling you set. Set it on purpose.”
Straight answers
Why does a site throw 502 errors under heavy traffic?+
Usually the tier behind the gateway is out of workers and the gateway gives up at its timeout. The signature is slow requests pinned at exactly one number. Count the errors on the correct vhost log first, because the default log may hold only a fraction of your traffic.
How many workers should my app server have?+
Compute it: workers divided by average seconds per request is your ceiling in requests per second. Size for the peak you will actually hit plus margin, within a measured per-worker memory budget, instead of trusting the package default.
Is the database the bottleneck when traffic spikes?+
Verify before spending. Check whether it is idle or busy and whether slow queries exist. Often the database is idle-fast and the work is queuing one tier up. Check every tier first.
How do I prepare a site for a traffic spike?+
Do the worker math against your realistic peak, confirm which logs actually carry your traffic, load-test to the computed ceiling, and put stale-on-error caching in front so a slow backend serves old pages instead of errors.
Doesn't more traffic just require more servers?+
Eventually, but almost never first. First use the capacity your configuration is holding back: correctly sized pools, sane timeouts, caching that absorbs the read storm. Hardware bought to cover a configuration setting is the most expensive config change there is.
Expecting your best day soon?
Launch, press, season peak: bring us the date. We'll do the worker math, read the right logs, and make the spike just another graph.
Talk to an engineerYou’ve reached the end.
Put the reading to work
What are
you running?
Tell us about the workload, traffic and recovery requirements. Start with the server builder, or talk through the details with an engineer.
Written by the engineers who build and run the infrastructure it describes.Talk to an engineer