16 October 2026, 11:17Виталий Лещенко

Networking for beginners. Part 22: BGP

All parts of the series “Networking for beginners”

OSPF from the previous part builds a map of one network and computes shortest paths over it. The internet is seventy-odd thousand independent networks, each with its own owner, its own money and its own interests; nobody will agree to hand a neighbor the map of their network, and "shortest path" is not the criterion there: a provider will carry traffic where it is cheaper for it, not where it is closer. This calls for a different kind of protocol — not about distances but about policy. That is BGP, Border Gateway Protocol, version 4, RFC 4271, 1994: the only routing protocol between networks, and the only one the entire internet rests on.

My blog already has a series on BGP in practice — there a Kubernetes cluster announces addresses to a router. Here — the protocol itself, from scratch.

The autonomous system

The unit of the internet in BGP terms is the autonomous system (AS): a set of networks under one administration with one routing policy. A provider, a cloud, a university, a large company with its own addresses. Each has an AS number: 16-bit historically, 32-bit since 2007, assigned by a regional registry together with a block of addresses from part five. The ranges 64512–65534 and 4200000000–4294967294 are private, for use inside your own network without registration; 64496–64511 are for documentation, and that's what we will use.

There are two kinds of relationships between ASes, and all policy grows out of them:

transit — a provider, for money, carries your traffic anywhere and announces your prefixes to the whole internet;
peering — two networks exchange the traffic of each other and their customers for free, usually at an exchange point (IXP), and pass each other's routes to nobody else.

Hence the basic rule: to customers and peers we give everything; to providers and peers we give only our own and our customers' routes. Breaking it means becoming transit for someone else's traffic at your own expense.

Where BGP lives

Three decisions distinguish BGP from all IGPs:

Over TCP, port 179. No reliability mechanism of its own: a session is an ordinary TCP connection, and everything we know about TCP from part ten applies. A packet is lost — TCP retransmits it; the firewall must allow port 179 between neighbors.

Neighbors are not discovered. No Hello on multicast: every neighbor is configured by hand with an address and an AS number. You cannot set up a BGP session by accident — only on purpose, and that is right for a protocol where one wrong neighbor can divert the traffic of the whole internet.

Path-vector. BGP does not pass a map and does not compute shortest paths. It passes prefixes with attributes, the main one being the list of ASes the prefix has passed through. It is the distance-vector of the previous part, with the whole path instead of a hop count: a loop is detected if your own AS number is already in the list.

Messages

Four types plus one additional one, with a common 19-byte header (a 16-byte marker of all ones, length, type):

Type	Message	What
1	OPEN	the first one after TCP is established: version 4, my AS number, hold time, Router ID, a list of capabilities (IPv6, 32-bit ASes, route refresh, …)
2	UPDATE	routes: a list of withdrawn prefixes, path attributes, a list of new prefixes with those attributes
3	NOTIFICATION	an error with a code; after it the session is closed
4	KEEPALIVE	"I'm alive", every 60 seconds; hold time 180 — three missed, and the neighbor is dead
5	ROUTE-REFRESH	"send me everything again" — to apply a new policy without dropping the session

Session states: Idle → Connect (trying TCP) → Active (TCP failed, trying again) → OpenSent → OpenConfirm → Established. A classic trap: the Active state sounds upbeat but means "no session, TCP is not being established" — most often a firewall or a wrong address. Only Established works.

Once established, neighbors send each other all routes allowed by policy in one stream of UPDATEs, and after that only changes. The full internet table today is more than a million IPv4 prefixes and about two hundred thousand IPv6 ones; the initial load is tens of megabytes and takes minutes.

UPDATE and path attributes

The most important message. Its structure: "these prefixes are no longer valid" (withdrawn), "here are the attributes" and "here are the prefixes they apply to" (NLRI). One UPDATE carries one set of attributes and any number of prefixes with it.

$ sudo tcpdump -i eth1 -n -v 'tcp port 179'
IP 198.51.100.1.179 > 198.51.100.2.41234: Flags [P.], ...: BGP
	Update Message (2), length: 55
	  Origin (1), length: 1, Flags [T]: IGP
	  AS Path (2), length: 6, Flags [T]: 64500 64510
	  Next Hop (3), length: 4, Flags [T]: 198.51.100.1
	  Multi Exit Discriminator (4), length: 4, Flags [O]: 0
	  Updated routes:
	    192.0.2.0/24

The provider with AS 64500 says: to reach 192.0.2.0/24, go through me (198.51.100.1); the path is through 64500 and then 64510, where the prefix originates. The attributes you need to know:

Attribute	What	Who sets it
AS_PATH	the list of ASes from us to the origin; every router passing a prefix into another AS prepends its own number	everyone
NEXT_HOP	the address to send traffic for this prefix to	whoever passes it into another AS
ORIGIN	how the prefix got into BGP: IGP (with the network command), EGP (historical), incomplete (redistribute)	the origin
LOCAL_PREF	preference within our own AS: higher is better; not passed outside	us, on input
MED	a hint to a neighboring AS about which of several links to use to enter us: lower is better	the neighbor, on output
COMMUNITY	`AS:number` tags that mean nothing by themselves but are used to build policy: "64500:100 — received from a customer", "64500:666 — do not announce to Europe"; there are well-known ones: `NO_EXPORT` — do not pass outside the AS	anyone
AGGREGATOR, ATOMIC_AGGREGATE	the prefix was obtained by merging smaller ones	whoever aggregated it

AS_PATH is at once loop protection, a crude distance measure and a policy tool: by lengthening its path by repeating its own number (prepending, 64496 64496 64496), a network makes that path less attractive to others and so shifts incoming traffic to another link.

eBGP and iBGP

The same BGP session behaves differently depending on whether the neighbors are in the same AS or in different ones. These are not two protocols but two sets of rules within one.

                      AS 64500 (provider A)          AS 64501 (provider B)
                            PA                              PB
                            │ eBGP                          │ eBGP
             198.51.100.1 ──┤                               ├── 198.51.100.5
                            │                               │
   ┌────────────────────────┼───────── AS 64496 ────────────┼────────────────────────┐
   │         198.51.100.2 ──┤                               ├── 198.51.100.6         │
   │                        R1 ───────── iBGP ──────────── R2                        │
   │                     10.0.0.1      (over OSPF)       10.0.0.2                    │
   │                         \───── 203.0.113.0/24 ──────/  our network              │
   └─────────────────────────────────────────────────────────────────────────────────┘

eBGP — between different ASes, R1 ↔ PA:

when passing on a prefix, a router adds its AS number to AS_PATH and sets NEXT_HOP = its own address on that link: a neighbor from another AS only knows that address;
the session is expected with a directly connected neighbor: packets are sent with TTL 1, and a neighbor behind a router will not answer (for sessions across several hops there is ebgp-multihop);
routes from an eBGP neighbor are passed to everyone: both other eBGP and iBGP neighbors, subject to policy;
administrative distance 20: an eBGP route beats OSPF (110) for the same prefix.

iBGP — within one AS, R1 ↔ R2:

AS_PATH does not change: within an AS the number is not added;
NEXT_HOP does not change: R2 will get the prefix 192.0.2.0/24 from R1 with next hop 198.51.100.1 — the address of provider A, which R2 does not know. A route without a reachable next hop is not used. Two solutions: either the internal routing (OSPF) knows the network 198.51.100.0/30 — for that the link to the provider is announced in OSPF as passive — or R1 replaces the next hop with itself before handing the route to iBGP: next-hop-self. The second is the standard;
a route received from an iBGP neighbor is not passed to other iBGP neighbors — the split horizon rule against loops, since AS_PATH does not change inside an AS and cannot catch a loop. The consequence: all BGP routers in an AS must be connected by iBGP sessions each to each, a full mesh, n(n−1)/2 sessions;
administrative distance 200: iBGP is weaker than OSPF, and if the same prefix is known from both, OSPF wins;
sessions are usually between loopback addresses (update-source lo) so as not to depend on a particular link, and the reachability of the loopbacks is provided by OSPF. That is why iBGP needs an IGP: BGP cannot deliver a packet to its own neighbor inside the AS; it only knows what to do with prefixes from outside.

A full mesh of 50 routers is 1225 sessions, and it is replaced by a route reflector: one or two routers with which all the others (clients) have iBGP, and which reflect routes between clients, breaking split horizon by rules (the ORIGINATOR_ID and CLUSTER_LIST attributes catch loops). Clients need one session instead of fifty. The alternative is confederations: split the AS into sub-ASes with eBGP between them, invisible from outside; less common.

Best path selection

A router usually has several paths to one prefix — from two providers, from iBGP neighbors. One goes into the routing table (or several equal ones with multipath), and it is chosen by a chain of rules, in order, until the first difference:

Higher LOCAL_PREF. Your own AS's policy beats everything: "provider A is preferred over B".
Locally originated (network) is better than received.
Shorter AS_PATH.
ORIGIN: IGP is better than EGP, which is better than incomplete.
Lower MED — only among paths from the same neighboring AS.
eBGP over iBGP.
Lower IGP metric to the NEXT_HOP — the nearest exit from the AS (hot potato: get rid of other people's traffic as quickly as possible).
The oldest path, then the lowest neighbor Router ID, then the lowest neighbor address — so the choice is deterministic.

$ vtysh -c 'show bgp ipv4 unicast 192.0.2.0/24'
BGP routing table entry for 192.0.2.0/24, version 12
Paths: (2 available, best #1, table default)
  64500 64510
    198.51.100.1 from 198.51.100.1 (10.0.1.1)
      Origin IGP, metric 0, localpref 200, valid, external, bestpath-from-AS 64500, best (Local Pref)
      Community: 64496:100
  64501 64520 64510
    10.0.0.2 from 10.0.0.2 (10.0.0.2)
      Origin IGP, localpref 100, valid, internal

Two paths: via provider A directly (eBGP) and via provider B behind R2 (iBGP, next hop 10.0.0.2 — R2 replaced it with itself). The first is chosen by Local Pref 200 versus 100 — even before comparing AS_PATH, which is also shorter for it.

Controlling this choice is what working with BGP is all about:

Outbound traffic (where we send) is easy to control: LOCAL_PREF on incoming routes. "Everything from provider A — localpref 200".
Inbound traffic (how others reach us) is hard to control: the decision is made by someone else's network. The tools are prepending (make the path via B longer), MED (a hint to one neighbor), communities the provider has promised to honor ("64500:80 — lower the localpref on your side"), and the last resort — announcing longer prefixes via A (/25 instead of /24), because the longest prefix beats everything (part six). The latter pollutes the internet table and is frowned upon.

Policy is expressed by filters on every session in both directions: prefix-list (which prefixes), as-path-list (which paths), route-map (condition → action: accept, drop, set an attribute). A session without an outbound filter is a classic outage: a network with two providers passes one provider's routes to the other and becomes transit for part of the internet over its hundred-megabit link.

What breaks at internet scale

Prefix hijacking. BGP trusts its neighbor. If AS 64520, by mistake or on purpose, announces 203.0.113.0/24, which belongs to us, 64520's providers will accept it, pass it on, and part of the internet will carry our traffic there. This is how, in 2008, a Pakistani provider "took" YouTube for an hour, and in 2018 cryptocurrency was stolen via a hijack. Protection comes in three layers:

Filters at providers: accept from a customer only the prefixes it has registered in IRR databases (whois databases of routing policy). Works if the provider filters.
RPKI: the address holder publishes, through the registry, a signed statement — a ROA — "the prefix 203.0.113.0/24 may only be announced by AS 64496, no longer than /24". Routers download the validated set of ROAs from a validator and mark routes: valid, invalid, unknown. Invalid ones are dropped. Since the 2020s most large networks do exactly this, and a hijack with someone else's AS as the origin stops propagating.
Maximum prefixes per session (maximum-prefix): a neighbor that sends a full table instead of its 10 prefixes (a classic leak) is disconnected automatically.

A route leak is not malice but a policy mistake: a network passed one provider's routes to another, and traffic between two giants went through someone's office router. Only the leaking side can fix it; RPKI does not protect against this, because the origin in AS_PATH is correct.

Convergence. BGP is slow by design: the MRAI timer delays sending UPDATEs by seconds so as not to send every intermediate version; a link failure somewhere in the world spreads across the internet in minutes, and during that time some networks see old paths. For fast detection of a neighbor failure there is BFD — the same fast-packet protocol as for OSPF — and it tells BGP about a failure within a fraction of a second instead of the 180-second hold time.

Session security. A TCP session can be reset with a forged RST if the sequence numbers are guessed. Protection: TCP MD5 (each segment signed with a shared key, TCP option 19) or the newer TCP-AO, and GTSM: send with TTL 255 and accept only packets with TTL 255, which guarantees the neighbor is one hop away.

BGP isn't only for the internet

The protocol turned out to be so flexible that it is used where the internet is not involved at all.

Data centers. In the spine-leaf fabrics from part eighteen, eBGP increasingly replaces OSPF: each switch gets its own private AS number, eBGP runs between all of them, and policy goes through AS_PATH (RFC 7938). It scales better than link-state, and it is one protocol for everything.
EVPN is a BGP extension for carrying MAC addresses and VNIs between VTEPs, the very control plane for VXLAN.
MPLS L3VPN — providers keep the routes of hundreds of customers in one BGP, telling them apart by labels (the VPNv4 family), and the customer gets "its own" network across the whole provider network. MP-BGP, multiprotocol, is the general name for BGP with several address families: IPv4, IPv6, VPNv4, EVPN in one session.
Kubernetes and load balancing. Cluster nodes announce service addresses to a router over eBGP, and the address becomes a route that exists as long as at least one node is alive. That is the very series: kube-vip and Calico with FRR on the router.
Anycast. One prefix is announced from dozens of points around the world, and BGP brings each client to the nearest one — this is how the root DNS servers from part twelve and all major CDNs work.

On FRR

R1 from the diagram: eBGP with provider A, iBGP with R2, our prefix announced outside, filters in both directions:

router bgp 64496
 bgp router-id 10.0.0.1
 neighbor 198.51.100.1 remote-as 64500
 neighbor 198.51.100.1 description provider-a
 neighbor 198.51.100.1 password s3cret
 neighbor 10.0.0.2 remote-as 64496
 neighbor 10.0.0.2 update-source lo
 address-family ipv4 unicast
  network 203.0.113.0/24
  neighbor 198.51.100.1 route-map from-a in
  neighbor 198.51.100.1 route-map to-a out
  neighbor 198.51.100.1 maximum-prefix 1300000
  neighbor 10.0.0.2 next-hop-self
 exit-address-family

ip prefix-list ours seq 5 permit 203.0.113.0/24
route-map to-a permit 10
 match ip address prefix-list ours
route-map from-a permit 10
 set local-preference 200

A remote-as equal to your own makes the session iBGP, a different one makes it eBGP; FRR applies the right rules itself. network 203.0.113.0/24 injects the prefix into BGP if it is in the routing table (usually a static route to null so the aggregate is always there). to-a lets out only our prefix, from-a accepts everything with localpref 200. On R2 it is the mirror image with provider B and localpref 100.

Session state:

$ vtysh -c 'show bgp summary'
Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt Desc
198.51.100.1    4      64500   1234567   1234560        0    0    0 3d02h15m      1012345        1 provider-a
10.0.0.2        4      64496      1234      1230        0    0    0 3d02h10m            3        3

The State/PfxRcd column shows either the number of accepted prefixes (the session is Established) or a state: Active, Connect, Idle (Admin). A million prefixes from the provider, one of ours going out, three inside — a normal picture for a network taking a full table.

To see from outside how the world sees your prefix: any provider's looking glass, bgp.tools, RIPE RIS — they show the AS_PATH to you from different vantage points, and from them you can see whether prepending works and whether someone has hijacked your prefix.

Key takeaways

The internet is autonomous systems with their own policies; BGP passes prefixes with attributes between them, not a map. Transit and peering determine who gets announced what.
Over TCP 179, neighbors are configured by hand, only Established works; Active means it doesn't work.
UPDATE carries AS_PATH (the path and loop protection), NEXT_HOP, LOCAL_PREF (your own policy, higher is better), MED (a hint to a neighbor, lower is better), community (tags for policy).
eBGP between ASes: adds its AS to the path, sets the next hop to itself, the neighbor is one hop away. iBGP within an AS: changes neither the path nor the next hop, doesn't pass iBGP routes to other iBGP neighbors, hence a full mesh or a route reflector, next-hop-self and an IGP for neighbor reachability.
Best path: LOCAL_PREF, locally originated, shorter AS_PATH, ORIGIN, MED, eBGP over iBGP, the nearest exit. Outbound is controlled with localpref, inbound with prepending, MED, communities and longer prefixes.
Filters in both directions on every session, maximum-prefix, RPKI against hijacks, TCP MD5 and GTSM for the session.

Next — Part 23: TLS and certificates, the last one: how a single TCP connection is protected, the TLS 1.3 handshake message by message, where the keys come from and why there is forward secrecy, what an X.509 certificate and a chain of trust are, who certificate authorities are and how they verify the owner, Let's Encrypt and the ACME protocol, revocation and Certificate Transparency, and what TLS does not hide.