Three guards that stop a customer BGP gateway leaking routes

A dedicated gateway per customer, which we build and operate, peering internally with our policy gateway over private AS numbers, is the right design: isolation, a policy that belongs to them, and routing that adapts on its own. The BGP is internal to our platform, not a session the customer runs. It also means you now run many small BGP speakers, and a single careless setting on any one of them can announce routes it should never announce. This article covers how that happens and the three guards we put on every customer gateway so it cannot.

Why per-customer gateways speak BGP at all

A shared VPN concentrator with static routes looks simpler, until you have to add a subnet, move a customer between firewalls, or survive a gateway failure. With a dedicated gateway per customer, each running a private AS inside our platform and peering with a central policy gateway, adding a network is an announcement, and losing a path is a withdrawal that converges by itself. The customer's policy stays at the customer's edge, and transit policy stays central.

The price is that every gateway is now able to influence routing for everyone. So it needs guardrails that do not depend on anybody being careful.

How leaks actually happen

In most published post-mortems, the cause of a route leak from an edge device is rarely an exotic BGP bug. It is redistribution. Somebody enables "redistribute connected" or "redistribute static" to save typing, and the gateway starts announcing its own WAN link, its VPN tunnel pool, a management subnet, or a route it learned from somewhere else. On an open source firewall with a routing daemon, that option is often one checkbox away.

Once a leaked route is accepted upstream, traffic for that prefix can be pulled toward the wrong gateway, or a customer's network can become reachable from places it should not be. Nothing crashes; traffic simply goes somewhere unexpected.

Guard one: announce with explicit network statements

A gateway should announce exactly the prefixes that belong to the customer, listed one by one, and nothing else. In FRR, which sits underneath many firewall routing plugins, that looks like:

router bgp 64601
 bgp router-id 10.0.0.2
 neighbor 10.0.0.1 remote-as 64600
 address-family ipv4 unicast
  network 198.51.100.0/27
  neighbor 10.0.0.1 prefix-list CUSTOMER-OUT out
 exit-address-family

The network statement only announces a prefix that is actually present in the routing table, so a typo announces nothing rather than something wrong.

Guard two: an outbound prefix list that matches

Belt and braces. Even with explicit network statements, an outbound prefix list permits only the customer's own prefixes and denies everything else:

ip prefix-list CUSTOMER-OUT seq 10 permit 198.51.100.0/27
ip prefix-list CUSTOMER-OUT seq 100 deny any

If something ever tries to announce more, this list drops it on the way out. Mirror it on the receiving side too: the central gateway accepts from each customer only the prefixes assigned to that customer.

Guard three: no redistribution, and check that it stays off

Redistribution is disabled, and it is part of the configuration you verify, not an assumption. On FRR you can check in one line:

vtysh -c 'show running-config' | grep -i redistribute

An empty result is the only acceptable result. Because some firewall interfaces expose redistribution as a convenient tick box, this check belongs in your build verification, not just in the original design.

Compare against a known good gateway before delivery

Three guards prevent leaks. They do not prevent the other common failure of per-customer gateways: one setting that differs from the gateways that work, causing a VPN that "does not connect" or connects without reaching anything. A missing pass rule for the VPN port on the WAN interface, outbound NAT left in automatic mode, an interface rule more restrictive than intended, a subtly different authentication option: each is easy to miss by eye.

The cure is mechanical. Configure every gateway through its API, never by hand, then pull the same settings from a proven reference gateway and the new one and compare them field by field:

• VPN instance settings: protocol, port, ciphers, authentication mode, pushed routes and DNS.
• WAN firewall rules for the VPN port and diagnostics.
• Rules on the VPN interface itself.
• Outbound NAT mode and rules.
• BGP configuration, including the redistribution field.

Any difference that is not explicitly expected for this customer blocks delivery. No credential leaves the building until the comparison is clean. It takes seconds and replaces an afternoon of remote troubleshooting with a user who just wants to work.

Per-person access, done properly

A client certificate per person, signed by a certificate authority that belongs to the customer, plus a credential, means a lost laptop is one revoked certificate and nobody else is affected. A fixed tunnel address per person lets firewall policy follow an individual across devices.

Checklist for your own gateways

• Is every customer prefix announced by an explicit network statement?
• Is there an outbound prefix list on every customer session, and a matching inbound one centrally?
• Does a grep for redistribute come back empty on every gateway?
• Are gateways configured through an API, with the result compared against a known good reference before delivery?
• Is remote access per person, with certificates you can revoke individually?

Every customer gateway we build has these guards, on its own segment and its own firewall. If you want private access to your platform built that way, we are happy to help.

Talk to an engineer

Related: Private connectivity: VPN gateways and leased lines · Managed perimeter firewalls · Multi-site encrypted backbone: the DMVPN mesh