State table arithmetic: sizing a firewall for floods, not averages

Ask most people how big a firewall needs to be and they answer in bandwidth. Under attack, bandwidth is rarely what runs out first. The state table is: the list of connections the firewall is tracking, which has a fixed size, fills at the rate new connections arrive, and empties only as fast as the timeouts allow. Understand that arithmetic and you can size a firewall for the day it matters instead of the average day.

Why state runs out before bandwidth

When the table is full, new connections are dropped, legitimate or not, even if the links are nearly idle.

Two kinds of traffic fill a table quickly:

• SYN floods. Each spoofed SYN creates a half-open entry that will never complete.
• UDP to public services. DNS, NTP, game servers, VoIP and similar services see huge numbers of short exchanges from many sources, and each one can create its own entry.

The arithmetic

The steady-state number of entries in a table is, to a good approximation:

entries  =  new flows per second  x  seconds each entry lives

That is Little's law, and it is the whole game. Suppose a flood arrives at 50,000 new SYNs per second, and half-open TCP entries live for 30 seconds. The table needs 1,500,000 entries just for the attack, before any real user arrives. Cut that timeout to 5 seconds during the flood and the same attack needs 250,000. Same firewall, same memory, a sixfold difference made by one timer.

So there are only three levers: a bigger table (memory), shorter lifetimes (timeouts), and fewer entries per flow (not tracking state at all where you do not need it). Use all three.

Lever one: size the table on purpose

Work out what memory the platform allocates per state entry, decide how much of the firewall's RAM you are prepared to give to state, and set the maximum from that, rather than living with a default chosen for a small office. Then alarm well before the table is full.

On OPNsense and pf based firewalls the live numbers are one command away:

pfctl -si | grep -A4 "State Table"
pfctl -sm | grep states

Lever two: adaptive timeouts and SYN cookies

Static timeouts are a compromise between being kind to slow clients and surviving a flood. Adaptive timeouts remove the compromise: as the table fills past a start threshold, all timeouts scale down linearly, reaching zero at an end threshold. Normal days keep generous timers; under pressure entries expire fast. In pf terms:

set limit states 2000000
set timeout { adaptive.start 1200000, adaptive.end 2200000 }

Add SYN cookies, ideally in an adaptive mode that switches on only when the table crosses a threshold. With cookies, the firewall answers a SYN without creating state and only builds an entry once the client completes the handshake, which a spoofed source never does. That turns the most efficient attack on a stateful firewall into traffic the firewall barely notices.

Lever three: do not track what does not need tracking

For high-volume UDP services, tracking each exchange buys almost nothing: the service answers from a known port, and the rules are simple. A stateless rule pair (allow inbound to the service port, allow outbound from it) creates no entries at all, so a UDP flood cannot exhaust the table on behalf of a DNS or NTP server. Keep stateful handling for TCP and for anything where return traffic must be strictly tied to an outbound request.

Keep state through a failure

Sizing is half the job; the other half is not losing all of it when a node fails. In an active and standby pair, CARP moves the virtual addresses to the standby, and pfsync replicates the state table between the nodes continuously, over a dedicated link where possible. When the active node dies, the standby already knows every established connection, so sessions continue instead of every user reconnecting. Test it: start a long download or an SSH session through the pair, power off the active node, and watch it carry on.

Two practical habits help. Apply tuning changes to the standby first, verify, then fail over and change the other node; and spread packet processing across CPU cores with thread counts matched to each appliance, so one busy core does not become the real limit.

Order the ruleset for cheap decisions

A flood is cheaper to drop early. Put the coarse, cheap decisions first: a combined threat-intelligence block list, then country blocks, then known abusive hosting ranges, and only then the per-host allow rules. Name every alias and rule after the host it protects, so the ruleset documents itself and old rules can be removed with confidence.

Check your own firewall

• What is the state table limit, and what is today's peak as a percentage of it?
• Is there an alarm well below the limit?
• Are adaptive timeouts configured, and are SYN cookies available under load?
• Are high-volume UDP services handled statelessly?
• In an HA pair, is state synchronised, and have you tested a failover with live sessions?
• Do you know what an attack at ten times your normal connection rate would do to the table? Do the multiplication.

Our perimeter firewalls in Athens run as stateful pairs built and tuned this way, changed only through their API. If you want help doing the arithmetic for your own, we are glad to sit down with you.

Talk to an engineer

Related: Managed perimeter firewalls · How we change things: network change discipline · Network monitoring and evidence