Kubernetes for beginners. Part 15: How do Service and kube-proxy work?

In the previous part we followed a packet from pod to pod: every pod has its own address, routing between nodes goes without NAT. But reaching pods by their addresses is inconvenient and dangerous: the address changes every time a pod is recreated, there are several pods, and some of them may not be ready. That is what a Service is for — an object that gives a group of pods one stable address and name — and kube-proxy, which turns that address into netfilter rules on every node. Today we look at where the virtual ClusterIP comes from, how the EndpointSlice controller tracks ready pods by the readiness probes from part 6, what exactly kube-proxy writes into nftables on our lab, how the iptables and IPVS modes differ, and all kinds of services: NodePort, LoadBalancer, headless, ExternalName and a service without a selector. Pods are picked by the labels and selectors from part 11.

---

Why a Service

The hello Deployment in the demo namespace from part 11 has three pods. A client inside the cluster needs:

• one address that does not change, however many times the pods are recreated;
• distribution of connections over all pods;
• ready pods only: a pod that is still warming up or has stopped passing its readiness probe must not get traffic.

A Service describes this declaratively:

apiVersion: v1
kind: Service
metadata:
  name: hello
  namespace: demo
  labels:
    app.kubernetes.io/name: hello
    app.kubernetes.io/instance: hello
spec:
  selector:
    app.kubernetes.io/name: hello
    app.kubernetes.io/instance: hello
  ports:
    - name: http
      port: 80
      targetPort: http
      protocol: TCP

• metadata.labels — labels of the Service object itself, the same recommended app.kubernetes.io/* from part 11: other objects that need to select services rather than pods find the service by them.
• selector — which pods the service serves. Usually the same as the Deployment selector — here app.kubernetes.io/name and app.kubernetes.io/instance, as in part 11.
• port — the service port clients connect to.
• targetPort — the pod port to forward to. Here it is given by the name http declared in containerPort of the pod template. This is the best practice: if the application moves to another port, changing the Deployment is enough, the service stays as it is. A number is used when the pods have no port name.
• name of a port is required when there are several ports; Ingress and DNS records (part 16) find the port by it.

$ kubectl apply -f hello-service.yaml
service/hello created
$ kubectl -n demo get svc hello
NAME    TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
hello   ClusterIP   10.105.101.118   <none>        80/TCP    5s

10.105.101.118 is the ClusterIP: the service address from the service network 10.96.0.0/12. It does not change while the Service object exists. Clients inside the cluster go to http://10.105.101.118/ or, better, to the name hello.demo.svc.cluster.local, which the cluster DNS turns into this address (part 16).

---

ClusterIP: an address that does not exist

The address is assigned to the service by kube-apiserver when the object is created. The range is set by the --service-cluster-ip-range flag (kubeadm takes it from serviceSubnet) and is visible in the API as a ServiceCIDR object named kubernetes; every assigned address is an IPAddress object (both APIs are stable since 1.33), which rules out races during allocation and allows adding ranges without restarting kube-apiserver. You can ask for a specific address (spec.clusterIP), and for that the range is split into two bands: the lower one — the first min(max(16, size/16), 256) addresses, for our /12 these are the 256 addresses 10.96.0.1–10.96.1.0 — is used for automatic allocation only after the upper one runs out. So an explicitly requested address from the lower band is almost never taken, and 10.96.0.10 for DNS lives right there. The first address of the range, 10.96.0.1, always belongs to the kubernetes service in the default namespace — that is kube-apiserver. An address outside the range is refused:

$ kubectl -n demo create service clusterip bad --tcp=80:8080 --clusterip=10.200.0.10
error: failed to create ClusterIP service: Service "bad" is invalid: spec.clusterIPs: Invalid value: ["10.200.0.10"]: failed to allocate IP 10.200.0.10: the provided network does not match the current range

The main thing about a ClusterIP: this address is on no interface of any node. Nobody answers ARP for it, there is no listening socket, ping 10.105.101.118 gets no reply. The address exists only as a key in netfilter rules on every node: a packet sent to 10.105.101.118:80 gets a new destination address — the address of one of the pods — before routing, and then travels over the ordinary pod network from the previous part. This is done by kube-proxy.

 pod client 172.20.211.52 on k8s-node-1           k8s-node-1 (netfilter)
   │ connect 10.105.101.118:80
   └──────────────────────────────────────► prerouting, nat:
                                            10.105.101.118:80 is in service-ips →
                                            DNAT to 172.20.172.17:8080 (picked at random)
                                            conntrack remembers the connection
                                                  │
                                                  ▼ route 172.20.172.0/26 via k8s-node-2
                                            hello pod 172.20.172.17:8080
   ◄────────────── reply: conntrack puts 10.105.101.118:80 back as the source

The pod answers from 172.20.172.17:8080, and on the way back conntrack — the kernel's connection tracking table — puts 10.105.101.118:80 back as the source address: the client sees an answer from the one it talked to. The pod is chosen once per connection — for TCP on the first packet — and all further packets of that connection go to the same pod. So balancing in a Service is per connection, not per request: a client with a long-lived connection (HTTP/2 — the HTTP version in which many requests share one connection; gRPC — a remote procedure call protocol on top of HTTP/2; a database connection pool) talks to one pod all the time.

---

EndpointSlice: who is behind the service right now

The Service itself only knows its selector. The list of pod addresses is kept by the EndpointSlice controller in kube-controller-manager. It watches services and pods and creates EndpointSlice objects for every service with a selector:

$ kubectl -n demo get endpointslices -l kubernetes.io/service-name=hello
NAME          ADDRESSTYPE   PORTS   ENDPOINTS                                   AGE
hello-g6zpm   IPv4          8080    172.20.172.17,172.20.211.48,172.20.172.15   5s
$ kubectl -n demo get endpointslices -l kubernetes.io/service-name=hello -o yaml
apiVersion: v1
items:
- addressType: IPv4
  apiVersion: discovery.k8s.io/v1
  endpoints:
  - addresses:
    - 172.20.172.17
    conditions:
      ready: true
      serving: true
      terminating: false
    nodeName: k8s-node-2
    targetRef:
      kind: Pod
      name: hello-6d5c6db6b4-7k5d9
      namespace: demo
      uid: bde33654-1539-4c88-b0bf-3bf1a30c3d07
  - addresses:
    - 172.20.211.48
    conditions:
      ready: true
      serving: true
      terminating: false
    nodeName: k8s-node-1
    targetRef:
      kind: Pod
      name: hello-6d5c6db6b4-d7bpf
      namespace: demo
      uid: 3599259d-3f09-4d90-b841-6701367420ce
  # … the third entry: 172.20.172.15, hello-6d5c6db6b4-tbvhn on k8s-node-2
  kind: EndpointSlice
  metadata:
    annotations:
      endpoints.kubernetes.io/last-change-trigger-time: "2026-10-28T20:27:44Z"
    creationTimestamp: "2026-10-28T20:27:44Z"
    generateName: hello-
    generation: 1
    labels:
      app.kubernetes.io/instance: hello
      app.kubernetes.io/name: hello
      endpointslice.kubernetes.io/managed-by: endpointslice-controller.k8s.io
      kubernetes.io/service-name: hello
    name: hello-g6zpm
    namespace: demo
    ownerReferences:
    - apiVersion: v1
      blockOwnerDeletion: true
      controller: true
      kind: Service
      name: hello
      uid: 72008210-7b21-4ef0-b6e6-f85eae221dad
    resourceVersion: "819731"
    uid: 49ec3ae8-4404-46ca-8224-86a21e184814
  ports:
  - name: http
    port: 8080
    protocol: TCP
kind: List
metadata:
  resourceVersion: ""

The slice name is the service name plus a random suffix (generateName), the owner is the Service itself, and the controller copies the service labels into the slice and adds two of its own: kubernetes.io/service-name ties the slice to the service, endpointslice.kubernetes.io/managed-by says who manages the slice. How the controller fills the slice:

• selection — all pods of the namespace that match the service selector and have an address;
• port — the named targetPort: http is resolved to a number for every pod separately (the same name may mean different ports on different pods);
• conditions — serving repeats the pod's Ready condition, that is, the result of the readiness probes of all its containers; terminating means the pod has been told to stop (it got a deletionTimestamp); ready is shorthand for "serving and not terminating". kube-proxy sends new connections to ready endpoints, and to pods with serving: true, terminating: true only when no ready ones are left at all: this way connections are not lost while the last pods of a service are terminating (for example, behind an external load balancer with externalTrafficPolicy: Local that has not yet noticed the node has no pods left). The documentation describes this for the Local policy; on the lab kube-proxy does the same with Cluster;
• nodeName and the node's zone — for traffic policies and trafficDistribution (below);
• size — up to 100 addresses per slice (the --max-endpoints-per-slice flag of kube-controller-manager). A service with a thousand pods gets a dozen slices, and a change of one pod rewrites one slice, not a huge shared object. In the ENDPOINTS column kubectl shows the first three addresses and appends + N more....

A pod that stops passing its readiness probe gets ready: false within seconds, and kube-proxy on all nodes stops sending new connections to it. A pod that has never been ready does not get into balancing at all. That is why a readiness probe is not a formality: without it a pod counts as ready as soon as its process starts, and the first requests after a deployment go to an application that has not warmed up yet. Our hello in demo has no probes (the part 11 manifest does not set them) — in "Hands on" we will add a probe and see what happens to the slice.

If a client needs not-ready pods too (for example, members of a clustered system looking for each other before they are ready), the service sets publishNotReadyAddresses: true: then ready is true for all addresses.

The old Endpoints object — one per service, with all addresses in one list — is still created by the controller for compatibility, but this API has been deprecated since 1.33:

$ kubectl -n demo get endpoints hello
Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
NAME    ENDPOINTS                                                  AGE
hello   172.20.172.15:8080,172.20.172.17:8080,172.20.211.48:8080   5s

New code and your own tools should read EndpointSlice.

---

kube-proxy

kube-proxy is a DaemonSet in kube-system, one pod per node, in the node's network. It watches Service and EndpointSlice and turns them into kernel rules. It does not proxy traffic itself (the name is left from the very first implementation, which did accept connections in its own process): packets are rewritten by the kernel according to its rules, and if kube-proxy is stopped, the existing rules keep working — they just stop being updated.

The mode is set in its configuration:

$ kubectl -n kube-system get configmap kube-proxy -o jsonpath='{.data.config\.conf}' | grep -E '^(mode|clusterCIDR|nodePortAddresses):'
clusterCIDR: 172.20.0.0/16
mode: nftables
nodePortAddresses: null

On the lab it is nftables: kubeadm passed it a KubeProxyConfiguration at kubeadm init (part 4). The mode has been stable since 1.33 and requires kernel 5.13 or newer; in addition, the project advises against enabling it on nodes whose nft utility is older than 1.0.0 (the lab has kernel 7.0 and nft 1.1.6). The default on Linux is still iptables — for compatibility (kube-proxy --help says exactly that: 'iptables' (default)) — so the mode is worth always setting explicitly in the configuration, and for new clusters the documentation recommends nftables.

The ip kube-proxy table

kube-proxy creates its own table ip kube-proxy and touches nothing outside it. This is one of the advantages over iptables: other components (Calico, the node firewall) live in their own tables, each with its own transaction, and nobody overwrites others' rules or waits for a shared lock. Inside:

Object	Type	What it does
nat-prerouting	chain, hook prerouting, priority dstnat	packets arriving at the node (from pods and from outside) — to services
nat-output	chain, hook output, priority dstnat	packets of the node's own processes — to services
services	chain	mark for masquerading what goes to a ClusterIP from outside the pod network; find the service in service-ips and service-nodeports
service-ips	verdict map	key "address . protocol . port" → goto to the service chain
service-nodeports	verdict map	key "protocol . port" → the NodePort chain
nodeport-ips	set	node addresses that accept NodePort traffic
service-<hash>-<ns>/<name>/<proto>/<port>	chain per service port	DNAT to a random address from the list of pods
external-…, local-…	chains	entry for traffic from outside the cluster (NodePort, LoadBalancer): masquerading, or only local pods with Local
endpoint-…, affinity-…	chains and sets	only for services with sessionAffinity (below)
cluster-ips	set	all active ClusterIPs
nat-postrouting	chain, hook postrouting, priority srcnat	to masquerading
masquerading	chain	marked packets get the node address as the source
hairpin-connections	set	pods of this node that may reach their own service (below)
filter-*, *-check, no-endpoint-*, firewall-ips	filtering chains and maps	reject for services without pods, for non-existent ClusterIP ports and for unallocated addresses of 10.96.0.0/12; loadBalancerSourceRanges (the list of addresses allowed to reach a LoadBalancer)

The main difference from iptables is the verdict map service-ips. It is a hash table in the kernel: one rule ip daddr . meta l4proto . th dport vmap @service-ips finds the right service in one lookup, however many services there are. Updates are targeted too: a new service means one element added to the map and one chain created, not all rules rewritten.

Choosing a pod. The service chain has one rule:

meta l4proto tcp dnat ip to numgen random mod 3 map { 0 : 172.20.172.15 . 8080, 1 : 172.20.172.17 . 8080, 2 : 172.20.211.48 . 8080 }

numgen random mod 3 is a random number from 0 to 2 for every new connection; the anonymous map turns it into "address . port", and dnat rewrites the destination address and port. The distribution is uniform and stateless. If client affinity is enabled in the service (sessionAffinity, below), the rules are different: a chain per pod (endpoint-…) and a set of client addresses that recently went to it.

Masquerading. Packets from pods to a ClusterIP go through DNAT but keep their source address: the receiving pod sees the real client address, and that is good. But if the connection was started not by a pod but by the node itself (a process in the node network) or by a client outside the cluster, without rewriting the source the reply could take another path, bypassing the conntrack that remembers the DNAT. Therefore:

1. in services a rule marks packets to a ClusterIP whose source is not from 172.20.0.0/16 (clusterCIDR in the kube-proxy configuration): meta mark set meta mark | 0x00004000; traffic from outside to a NodePort is marked the same way in the external-… chain;
2. in masquerading (postrouting) marked packets are masqueraded with fully-random — a random choice of the source port, which lowers the chance of conntrack collisions with many simultaneous connections;
3. separately — hairpin: if a pod reached its own service and DNAT picked that very pod, the packet would come back to it with its own address as the source, and the reply would bypass conntrack. For this kube-proxy keeps the hairpin-connections set — for every pod of a service on this node the four-tuple "pod address . pod address . protocol . port" — and masquerades connections whose source and destination are the same after DNAT.

 nat-prerouting / nat-output (priority dstnat)
        │ jump services
        ▼
 services ── ClusterIP and source not from 172.20.0.0/16? ──► mark 0x4000
        │
        ├─ vmap @service-ips:       10.105.101.118 . tcp . 80 ──► service-VSI2YVQZ-demo/hello/tcp/http
        └─ vmap @service-nodeports: tcp . 30080 ──► external-BJ5L76QF-demo/hello-np/tcp/http
                                                    (mark 0x4000, goto service-BJ5L76QF-…)

 service-…-<service>
        dnat ip to numgen random mod N map { … }   pick a pod, rewrite the destination
        │
        ▼ routing to the pod
 nat-postrouting (priority srcnat)
        │ jump masquerading
        ▼
 masquerading: marked 0x4000 or hairpin → masquerade fully-random

iptables mode

In iptables mode kube-proxy writes the same into the iptables nat table. The lab does not run this mode, so below is not output but a scheme of the rules for the hello service with three pods, in the form kube-proxy writes them (hashes in chain names replaced with <hash>, -m comment comments shortened):

-A KUBE-SERVICES -d 10.105.101.118/32 -p tcp -m comment --comment "demo/hello:http cluster IP" -m tcp --dport 80 -j KUBE-SVC-<hash>
-A KUBE-SVC-<hash> ! -s 172.20.0.0/16 -d 10.105.101.118/32 -p tcp -m tcp --dport 80 -j KUBE-MARK-MASQ
-A KUBE-SVC-<hash> -m statistic --mode random --probability 0.33333333349 -j KUBE-SEP-<hash1>
-A KUBE-SVC-<hash> -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-<hash2>
-A KUBE-SVC-<hash> -j KUBE-SEP-<hash3>
-A KUBE-SEP-<hash1> -s 172.20.172.15/32 -j KUBE-MARK-MASQ
-A KUBE-SEP-<hash1> -p tcp -m tcp -j DNAT --to-destination 172.20.172.15:8080
-A KUBE-MARK-MASQ -j MARK --set-xmark 0x4000/0x4000
-A KUBE-POSTROUTING -m mark ! --mark 0x4000/0x4000 -j RETURN
-A KUBE-POSTROUTING -j MARK --set-xmark 0x4000/0x0
-A KUBE-POSTROUTING -j MASQUERADE --random-fully

The same three ideas — the 0x4000 mark in KUBE-MARK-MASQ, DNAT to a pod and masquerading of what is marked in KUBE-POSTROUTING — but built differently. The KUBE-SERVICES chain is a list, one rule per port of every service, and the first packet of a connection walks it top to bottom until it finds its address. Choosing a pod is a cascade of probabilities: the first rule fires with probability 1/3, otherwise the second with 1/2 of the rest, otherwise the third, which gives equal shares. The -s <pod address> -j KUBE-MARK-MASQ rule in every pod chain is the hairpin. This works and was the standard for many years, but with thousands of services the first packet of every connection walks thousands of rules, and every update goes through iptables-restore with a size proportional to the number of all services (since 1.26 kube-proxy skips unchanged rules, but not all of them), under the shared xtables lock that all programs on the node share. Hence second-long delays when syncing large clusters. For small clusters like ours the difference is invisible, and the mode remains fully functional.

There is something in iptables on the lab nodes after all — the kubelet's own service chains:

user@k8s-node-1:~$ sudo iptables-save | grep -- '-A KUBE'
-A KUBE-FIREWALL ! -s 127.0.0.0/8 -d 127.0.0.0/8 -m comment --comment "block incoming localnet connections" -m conntrack ! --ctstate RELATED,ESTABLISHED,DNAT -j DROP

There are no KUBE-SERVICES and KUBE-SVC-… chains: in nftables mode kube-proxy writes no iptables rules, and when the mode is switched it removes the rules of the previous one itself — nodes do not need a reboot.

IPVS mode

IPVS (IP Virtual Server) is a layer 4 load balancer in the Linux kernel, historically used for load balancing without Kubernetes. In this mode kube-proxy creates a dummy interface kube-ipvs0 on every node, assigns all ClusterIPs to it and describes every service as an IPVS virtual server with a balancing algorithm (round robin, least connections and so on). Lookup is a hash table, as in nftables. But the IPVS model fit the Service API poorly, some edge cases were never solved, and since 1.35 the ipvs mode is deprecated: the documentation recommends moving to nftables, and on old kernels to iptables. The documentation of the next version already names the dates: from 1.40 the mode will be disabled by default (it can be brought back with the KubeProxyIPVS feature gate), in 1.43 it will be removed. Switching, as between the other modes, does not require rebooting nodes.

eBPF instead of kube-proxy

There is a third way too — no kube-proxy at all. eBPF is a Linux kernel mechanism for loading verified programs into the kernel that process packets right at the network interface. Calico's eBPF dataplane watches services itself and picks a pod before the packet reaches netfilter; kube-proxy is turned off in such a cluster. This is covered in the series on highly available Kubernetes on BGP.

---

Service types

Type	What it adds	Who handles it
ClusterIP (default)	a virtual address inside the cluster	kube-proxy
NodePort	ClusterIP + a port 30000–32767 on the addresses of all nodes	kube-proxy
LoadBalancer	NodePort + an external address from a load balancer	external controller + kube-proxy
ExternalName	only a DNS alias	CoreDNS

NodePort

apiVersion: v1
kind: Service
metadata:
  name: hello-np
  namespace: demo
spec:
  type: NodePort
  selector:
    app.kubernetes.io/name: hello
    app.kubernetes.io/instance: hello
  ports:
    - name: http
      port: 80
      targetPort: http
      nodePort: 30080

$ kubectl apply -f hello-np.yaml
service/hello-np created
$ kubectl -n demo get svc hello-np
NAME       TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
hello-np   NodePort   10.105.48.156   <none>        80:30080/TCP   1s

kube-apiserver assigns the service a port from the --service-node-port-range range (30000–32767 by default); you can ask for a specific one — if it is free and inside the range. Copies of this manifest named hello-np2 with the same nodePort: 30080 and hello-np3 with nodePort: 20080 are refused by the API server:

The Service "hello-np2" is invalid: spec.ports[0].nodePort: Invalid value: 30080: provided port is already allocated
The Service "hello-np3" is invalid: spec.ports[0].nodePort: Invalid value: 20080: provided port is not in the valid range. The range of valid ports is 30000-32767

Now http://192.168.10.20:30080/ from the administrator's laptop reaches hello, and through any node — even through the master k8s-master-1, which has no hello pods: kube-proxy on every node knows all services.

In nftables mode a NodePort is accepted only on the node's primary addresses — those recorded in the Node object — not on all of them (nodePortAddresses: null in the configuration means exactly that here, the documentation calls this behaviour primary), and it does not answer on 127.0.0.1: curl http://127.0.0.1:30080/ on a node gets connection refused. In iptables mode without nodePortAddresses a NodePort is open on all node addresses, loopback included. Another difference: kube-proxy in nftables mode adds no allowing rules for NodePorts to the node firewall; if the nodes have their own nftables-based firewall, it has to let ports 30000–32767 through itself.

NodePort is a tool for debugging and for integration with an external load balancer that knows the node addresses. Exposing a service to users through non-standard ports on node addresses is bad practice: the client is tied to a specific node, and when that node goes down, the service disappears for that client.

externalTrafficPolicy: Cluster and Local

A client from outside came to 192.168.10.21:30080. By default (externalTrafficPolicy: Cluster) the node picks any pod of the service, including one on another node. So that the reply comes back through this same node (where conntrack remembers the DNAT), the packet is masqueraded with the node address:

 client 192.168.10.50 ──► k8s-node-2:30080 ── DNAT + SNAT ──► pod on k8s-node-1
                          the source becomes 192.168.10.21

The distribution is uniform, but the pod sees the node address, not the client address. hello logs the address a request came from, and it shows at once:

laptop:~$ for i in 1 2 3 4 5 6; do curl -s http://192.168.10.21:30080/; done
hello from hello-6d5c6db6b4-d7bpf
hello from hello-6d5c6db6b4-d7bpf
hello from hello-6d5c6db6b4-tbvhn
hello from hello-6d5c6db6b4-d7bpf
hello from hello-6d5c6db6b4-tbvhn
hello from hello-6d5c6db6b4-tbvhn
$ kubectl -n demo logs hello-6d5c6db6b4-d7bpf --since=1m
2026/10/28 20:29:11 192.168.10.21:8906 GET /
2026/10/28 20:29:11 192.168.10.21:35142 GET /
2026/10/28 20:29:11 192.168.10.21:10153 GET /

The d7bpf pod lives on k8s-node-1, and the requests came to it from 192.168.10.21 — the node the client entered through; traffic to the pods on k8s-node-2 itself is masqueraded too. For access logs, per-address rate limiting and allow lists this is unacceptable.

externalTrafficPolicy: Local changes the rule: the node sends external traffic only to pods on itself and without SNAT, so the pod sees the real client address. If the node has no pods of its own, the packet is dropped. So that an external load balancer does not send traffic to such nodes, for a type: LoadBalancer service with Local kube-apiserver allocates a healthCheckNodePort, and kube-proxy answers 200 on that port if the node has ready pods of the service and 503 if not. A type: NodePort service has no such field: a load balancer that uses NodePorts checks the nodes itself. The price of Local is imbalance: a node with one pod gets as much traffic as a node with three.

In nftables this shows in the external-… chain of a Local service: packets from the pod network and from the node itself go to the ordinary service chain (they may reach all pods), the rest go to the local-… chain, whose map holds only this node's pods. We will see all this in "Hands on".

How this is used with a real load balancer we will cover in part 17 with MetalLB; Traefik with externalTrafficPolicy: Local is covered in the series on BGP.

internalTrafficPolicy

The same for traffic from inside the cluster: internalTrafficPolicy: Local (stable since 1.26) — client pods and node processes go only to the service's pods on their own node, and if there are none, the connection is not established. It is used for node agents: for example, an application sends metrics or logs to the service of a DaemonSet agent and must reach the agent of its own node. For ordinary services it is Cluster (the default).

trafficDistribution

A soft way to prefer "nearby" pods is the spec.trafficDistribution field (stable since 1.33; the values below are stable since 1.35):

• PreferSameZone — prefer pods in the same zone (topology.kubernetes.io/zone of the node), and if there are none — any. Saves cross-zone traffic in clouds where it is paid;
• PreferSameNode — prefer pods on the same node, then in the same zone, then any;
• PreferClose — the former name of PreferSameZone, kept as a deprecated alias.

The EndpointSlice controller writes a hint hints.forZones (or forNodes) for every address, and kube-proxy builds the map on each node only from the matching addresses. Unlike internalTrafficPolicy: Local, traffic is not lost when there are no nearby pods. The risk is skew: if a zone has one pod and many clients, it gets all their traffic. The lab has no zones, so PreferSameZone has no effect, but PreferSameNode works — we will show that below as well.

sessionAffinity

sessionAffinity: ClientIP — connections from one client address go to the same pod for sessionAffinityConfig.clientIP.timeoutSeconds (10800 by default, three hours). This is affinity by address, not by application session: all clients behind one NAT end up on one pod, and with externalTrafficPolicy: Cluster the node address takes the place of the client address. For applications that need "sticky" sessions it is better to keep the session outside the pod (in Redis — an in-memory key-value store — or in a database) or to pin by cookie at the HTTP proxy level (part 18).

LoadBalancer

type: LoadBalancer is a NodePort plus a request to an external load balancer to provide an address and direct its traffic to the nodes. Kubernetes itself provides nothing: in a cloud this is done by the cloud controller manager, which creates the provider's load balancer and writes its address into status.loadBalancer.ingress. Our cluster has no such controller, and the service will wait forever — without events or errors:

$ kubectl apply -f hello-lb.yaml
service/hello-lb created
$ kubectl -n demo get svc hello-lb
NAME       TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
hello-lb   LoadBalancer   10.97.232.108   <pending>     80:31950/TCP   60s
$ kubectl -n demo get events --field-selector involvedObject.kind=Service
No resources found in demo namespace.

NodePort 31950 already works — kube-proxy has done its part. The loadBalancerClass field selects a specific controller when there are several, and allocateLoadBalancerNodePorts: false skips allocating a NodePort if the load balancer can send traffic straight to pods. How to get an address without a cloud — in part 17.

---

Headless, ExternalName and a service without a selector

A headless service has clusterIP: None. The service has no address, kube-proxy ignores it, but EndpointSlices are filled as usual, and DNS answers the service name with the list of pod addresses. The client chooses which pod to go to — this is how StatefulSets work (part 8), clustered databases and clients with their own balancing (for example, gRPC, which keeps connections to all pods at once).

ExternalName is an alias for an external name:

apiVersion: v1
kind: Service
metadata:
  name: registry
  namespace: demo
spec:
  type: ExternalName
  externalName: ghcr.io

A query for registry.demo.svc.cluster.local gets a CNAME record pointing to ghcr.io from the cluster DNS. No address, no kube-proxy rules, no ports. Handy for giving applications an internal name in their configuration that can be switched without touching them. The catch: the client connects to registry.demo.svc…, and the server receives this name in the Host header and in SNI — the extension of TLS (the encryption protocol HTTPS runs on; see part 23 of the networking series) in which the client names the server — and a certificate issued for ghcr.io does not match. So ExternalName fits HTTP and HTTPS poorly, which the documentation warns about too.

A service without a selector is for backends that are not cluster pods. We create a service without selector — the EndpointSlice controller leaves it alone — and an EndpointSlice by hand:

apiVersion: v1
kind: Service
metadata:
  name: legacy-api
  namespace: demo
spec:
  ports:
    - name: http
      port: 80
      targetPort: 8080
---
apiVersion: discovery.k8s.io/v1
kind: EndpointSlice
metadata:
  name: legacy-api-1
  namespace: demo
  labels:
    kubernetes.io/service-name: legacy-api
    endpointslice.kubernetes.io/managed-by: admins.example.com
addressType: IPv4
ports:
  - name: http
    port: 8080
    protocol: TCP
endpoints:
  - addresses: ["192.168.10.40"]
    conditions:
      ready: true
  - addresses: ["192.168.10.41"]
    conditions:
      ready: true

The link to the service is the kubernetes.io/service-name label. The managed-by label with its own value tells the EndpointSlice controller that the slice belongs to someone else and must not be touched. Pods go to legacy-api.demo and get balancing over the two external servers with all the kube-proxy rules. While there is no slice, the service without addresses answers with connection refused. Limitations: the addresses cannot be loopback or link-local — the API server refuses such a slice — and by the documentation must not be ClusterIPs of other services: kube-proxy does not support virtual addresses as a destination, and the API server does not reject such a slice, so watching for this is up to you. And there are no health checks — if 192.168.10.41 goes down, it has to be removed from the slice by hand or by an external script.

---

Hands on

First — which nftables tables a node has:

user@k8s-node-1:~$ sudo nft list tables | grep -E 'table (ip|arp) '
table ip mangle
table ip filter
table ip nat
table ip kube-proxy
table ip calico
table arp calico-arp

mangle, filter and nat are iptables-nft tables: the kubelet keeps its service chains there (KUBE-FIREWALL, KUBE-KUBELET-CANARY), there are no service rules in them. Next to the kube-proxy tables are the Calico tables from the previous part: Felix also runs in nftables mode and keeps policies and natOutgoing in its own ip calico table. They do not touch each other, and the order in which a packet is processed is set by the hooks and priorities of the base chains. kube-proxy's DNAT happens on the prerouting hook (nat-prerouting, priority dstnat), while Calico policies for traffic between pods are checked later, on the forward hook — so they see a packet to a service already with the destination pod address, and a rule "allow to the hello pods" works for connections through the ClusterIP too. On postrouting Calico's NAT chain nat-POSTROUTING (priority srcnat + 10) comes right after kube-proxy's nat-postrouting (srcnat).

The kube-proxy table on k8s-node-1 while demo has hello, hello-np and hello-lb, abridged (besides ours, the lab has the kubernetes and kube-dns services and two Calico services — calico-api and calico-typha):

user@k8s-node-1:~$ sudo nft list table ip kube-proxy
table ip kube-proxy {
	comment "rules for kube-proxy"
	set hairpin-connections {
		type ipv4_addr . ipv4_addr . inet_proto . inet_service
		comment "service hairpin connections"
		elements = { 192.168.10.20 . 192.168.10.20 . tcp . 5473,
			     172.20.211.48 . 172.20.211.48 . tcp . 8080 }
	}

	set cluster-ips {
		type ipv4_addr
		comment "Active ClusterIPs"
		elements = { 10.96.0.1, 10.96.0.10,
			     10.97.64.183, 10.97.232.108,
			     10.98.231.113, 10.105.48.156,
			     10.105.101.118 }
	}

	set nodeport-ips {
		type ipv4_addr
		comment "IPs that accept NodePort traffic"
		elements = { 192.168.10.20 }
	}
	…
	map service-ips {
		type ipv4_addr . inet_proto . inet_service : verdict
		comment "ClusterIP, ExternalIP and LoadBalancer IP traffic"
		elements = { 10.96.0.10 . tcp . 53 : goto service-NWBZK7IH-kube-system/kube-dns/tcp/dns-tcp,
			     10.96.0.10 . udp . 53 : goto service-FY5PMXPG-kube-system/kube-dns/udp/dns,
			     10.97.232.108 . tcp . 80 : goto service-WRGIWGFD-demo/hello-lb/tcp/http,
			     10.105.101.118 . tcp . 80 : goto service-VSI2YVQZ-demo/hello/tcp/http,
			     10.105.48.156 . tcp . 80 : goto service-BJ5L76QF-demo/hello-np/tcp/http,
			     10.98.231.113 . tcp . 5473 : goto service-EZ4UY2LR-calico-system/calico-typha/tcp/calico-typha,
			     10.96.0.1 . tcp . 443 : goto service-2QRHZV4L-default/kubernetes/tcp/https,
			     10.97.64.183 . tcp . 443 : goto service-W56QCMHD-calico-system/calico-api/tcp/apiserver,
			     10.96.0.10 . tcp . 9153 : goto service-AS2KJYAD-kube-system/kube-dns/tcp/metrics }
	}

	map service-nodeports {
		type inet_proto . inet_service : verdict
		comment "NodePort traffic"
		elements = { tcp . 30080 : goto external-BJ5L76QF-demo/hello-np/tcp/http,
			     tcp . 31950 : goto external-WRGIWGFD-demo/hello-lb/tcp/http }
	}
	…
	chain filter-forward {
		type filter hook forward priority filter; policy accept;
		ct state new jump service-endpoints-check
		ct state new jump cluster-ips-check
	}
	…
	chain nat-prerouting {
		type nat hook prerouting priority dstnat; policy accept;
		jump services
	}

	chain nat-output {
		type nat hook output priority dstnat; policy accept;
		jump services
	}

	chain nat-postrouting {
		type nat hook postrouting priority srcnat; policy accept;
		jump masquerading
	}
	…
	chain services {
		ip daddr @cluster-ips ip saddr != 172.20.0.0/16 meta mark set meta mark | 0x00004000 comment "masquerade clusterIP traffic from outside cluster"
		ip daddr . meta l4proto . th dport vmap @service-ips
		ip daddr @nodeport-ips meta l4proto . th dport vmap @service-nodeports
	}

	chain masquerading {
		meta mark & 0x00004000 != 0x00000000 meta mark set meta mark ^ 0x00004000 masquerade fully-random
		ct status dnat ip saddr . ip daddr . meta l4proto . th dport @hairpin-connections masquerade fully-random
	}

	chain cluster-ips-check {
		ip daddr @cluster-ips reject comment "Reject traffic to invalid ports of ClusterIPs"
		ip daddr 10.96.0.0/12 drop comment "Drop traffic to unallocated ClusterIPs"
	}
	…
	chain service-2QRHZV4L-default/kubernetes/tcp/https {
		meta l4proto tcp dnat ip to numgen random mod 1 map { 0 : 192.168.10.17 . 6443 }
	}

	chain service-VSI2YVQZ-demo/hello/tcp/http {
		meta l4proto tcp dnat ip to numgen random mod 3 map { 0 : 172.20.172.15 . 8080, 1 : 172.20.172.17 . 8080, 2 : 172.20.211.48 . 8080 }
	}
	…
	chain service-BJ5L76QF-demo/hello-np/tcp/http {
		meta l4proto tcp dnat ip to numgen random mod 3 map { 0 : 172.20.172.15 . 8080, 1 : 172.20.172.17 . 8080, 2 : 172.20.211.48 . 8080 }
	}

	chain external-BJ5L76QF-demo/hello-np/tcp/http {
		meta mark set meta mark | 0x00004000 comment "masquerade"
		goto service-BJ5L76QF-demo/hello-np/tcp/http
	}
}

Reading top to bottom. cluster-ips holds all seven ClusterIPs; a packet to an address of 10.96.0.0/12 that is not here is dropped in the cluster-ips-check chain, and one to an active ClusterIP but to a port the service does not have is rejected with an answer. nodeport-ips holds only 192.168.10.20, the node's primary address. service-ips has an element per port of every service, DNS has three of them (UDP and TCP on 53 and metrics on 9153). The hello service chain is one rule with a map of three pods; kubernetes has one address in its map — kube-apiserver on the master. The hello-np NodePort goes through external-…: traffic from outside is marked for masquerading and goes on to the ordinary service chain. hairpin-connections holds this node's hello pod (172.20.211.48) and the Typha pod, which runs in the node network.

The hash in a chain name is the first eight characters of SHA-256 of "namespace/name/protocol/port name" in base32, so that names are unique and fit the length limit. You can compute it yourself:

$ printf 'demo/hello/tcp/http' | sha256sum | cut -d' ' -f1 | xxd -r -p | base32 | cut -c1-8
VSI2YVQZ

How the map follows the pods. Scale hello down to two replicas and look at the chain:

$ kubectl -n demo scale deployment hello --replicas=2
deployment.apps/hello scaled
user@k8s-node-1:~$ sudo nft list chain ip kube-proxy 'service-VSI2YVQZ-demo/hello/tcp/http'
table ip kube-proxy {
	chain service-VSI2YVQZ-demo/hello/tcp/http {
		meta l4proto tcp dnat ip to numgen random mod 2 map { 0 : 172.20.172.17 . 8080, 1 : 172.20.211.48 . 8080 }
	}
}
$ kubectl -n demo scale deployment hello --replicas=3
deployment.apps/hello scaled

The rule is rewritten as a whole, but only this rule: the rest of the table is untouched. The third replica came back as a new pod hello-6d5c6db6b4-dj4mf with the address 172.20.172.18.

Distribution over the pods. For the experiments, start a client pod on k8s-node-1 — busybox, which has wget, nslookup and ping:

$ kubectl -n demo run client --image=busybox:1.37 --overrides='{"spec":{"nodeName":"k8s-node-1"}}' -- sleep 3600
pod/client created
$ kubectl -n demo get pod client -o wide
NAME     READY   STATUS    RESTARTS   AGE   IP              NODE         NOMINATED NODE   READINESS GATES
client   1/1     Running   0          3s    172.20.211.52   k8s-node-1   <none>           <none>
$ kubectl -n demo exec client -- sh -c 'for i in 1 2 3 4 5 6 7 8; do wget -qO- http://hello/; done'
hello from hello-6d5c6db6b4-dj4mf
hello from hello-6d5c6db6b4-d7bpf
hello from hello-6d5c6db6b4-d7bpf
hello from hello-6d5c6db6b4-dj4mf
hello from hello-6d5c6db6b4-d7bpf
hello from hello-6d5c6db6b4-dj4mf
hello from hello-6d5c6db6b4-dj4mf
hello from hello-6d5c6db6b4-7k5d9

Every wget is a new connection, and the random choice spreads them over the three pods — at random, not round robin, so over eight requests the shares need not be equal. The short name hello worked because the client is in the same namespace; how that works — in the next part.

A not-ready pod. Add a readiness probe on a non-existent path to the hello template and see what happens to the slice and the rules:

$ kubectl -n demo patch deployment hello --type=json -p='[{"op":"add","path":"/spec/template/spec/containers/0/readinessProbe","value":{"httpGet":{"path":"/nope","port":"http"},"periodSeconds":5}}]'
deployment.apps/hello patched
$ kubectl -n demo get pods -l app.kubernetes.io/name=hello -o wide
NAME                     READY   STATUS    RESTARTS   AGE     IP              NODE         NOMINATED NODE   READINESS GATES
hello-696bd4ff7-gf7wm    0/1     Running   0          30s     172.20.172.20   k8s-node-2   <none>           <none>
hello-6d5c6db6b4-7k5d9   1/1     Running   0          2m51s   172.20.172.17   k8s-node-2   <none>           <none>
hello-6d5c6db6b4-d7bpf   1/1     Running   0          2m51s   172.20.211.48   k8s-node-1   <none>           <none>
hello-6d5c6db6b4-dj4mf   1/1     Running   0          55s     172.20.172.18   k8s-node-2   <none>           <none>
$ kubectl -n demo get endpointslice -l kubernetes.io/service-name=hello -o jsonpath='{range .items[*].endpoints[*]}{.addresses[0]} ready={.conditions.ready} serving={.conditions.serving} terminating={.conditions.terminating}{"\n"}{end}'
172.20.172.17 ready=true serving=true terminating=false
172.20.211.48 ready=true serving=true terminating=false
172.20.172.18 ready=true serving=true terminating=false
172.20.172.20 ready=false serving=false terminating=false
$ kubectl -n demo get events --field-selector reason=Unhealthy
LAST SEEN   TYPE      REASON      OBJECT                      MESSAGE
0s          Warning   Unhealthy   pod/hello-696bd4ff7-gf7wm   Readiness probe failed: HTTP probe failed with statuscode: 404
user@k8s-node-1:~$ sudo nft list chain ip kube-proxy 'service-VSI2YVQZ-demo/hello/tcp/http'
table ip kube-proxy {
	chain service-VSI2YVQZ-demo/hello/tcp/http {
		meta l4proto tcp dnat ip to numgen random mod 3 map { 0 : 172.20.172.17 . 8080, 1 : 172.20.172.18 . 8080, 2 : 172.20.211.48 . 8080 }
	}
}

With three replicas the strategy defaults from part 7 — maxSurge 25 %, that is one pod rounded up, and maxUnavailable 25 %, that is zero rounded down — made the Deployment create one new pod on top of the three and leave the old ones alone. The new pod is not ready: it is in the slice, but with ready=false, and it did not get into the kube-proxy map. The rolling update is stuck, and the service keeps working on the three old pods. Roll back:

$ kubectl -n demo rollout undo deployment hello
Warning: resource deployments/hello was previously managed with 'kubectl apply'. Rolling back will not update the kubectl.kubernetes.io/last-applied-configuration annotation, which may cause unexpected behavior on future 'kubectl apply' operations. Consider using 'kubectl apply' with your previous configuration file instead.
deployment.apps/hello rolled back

A terminating pod. Delete one hello pod and look at the slice right away:

$ kubectl -n demo delete pod hello-6d5c6db6b4-dj4mf --wait=false
pod "hello-6d5c6db6b4-dj4mf" deleted from demo namespace
$ kubectl -n demo get endpointslice -l kubernetes.io/service-name=hello -o jsonpath='{range .items[*].endpoints[*]}{.addresses[0]} ready={.conditions.ready} serving={.conditions.serving} terminating={.conditions.terminating}{"\n"}{end}'
172.20.172.17 ready=true serving=true terminating=false
172.20.211.48 ready=true serving=true terminating=false
172.20.172.18 ready=false serving=true terminating=true

The pod still answers (serving=true) but is already terminating, so ready=false, and new connections do not go to it. Seconds later it disappeared from the slice, and the ReplicaSet created a replacement — hello-6d5c6db6b4-m6pbj with the address 172.20.172.19.

Conntrack on the node: DNAT at work. Ubuntu Server does not ship the conntrack utility by default, and installing packages on cluster nodes for debugging is pointless: it is easier to run it in a kubectl debug node debugging pod, which runs in the node's network and PID namespaces and, with the sysadmin profile, is privileged. The image is nicolaka/netshoot, a set of network debugging tools:

$ kubectl debug node/k8s-node-1 -n demo --profile=sysadmin --image=nicolaka/netshoot:v0.16 -- sleep 3600
Creating debugging pod node-debugger-k8s-node-1-2srlg with container debugger on node k8s-node-1.

Reach the ClusterIP from the node itself and from the client pod, then read the node's conntrack table:

user@k8s-node-1:~$ curl -s http://10.105.101.118/
hello from hello-6d5c6db6b4-d7bpf
$ kubectl -n demo exec client -- wget -qO- http://hello/
hello from hello-6d5c6db6b4-d7bpf
$ kubectl -n demo exec node-debugger-k8s-node-1-2srlg -- conntrack -L -d 10.105.101.118
conntrack v1.4.9 (conntrack-tools): 2 flow entries have been shown.
tcp      6 56 TIME_WAIT src=172.20.211.52 dst=10.105.101.118 sport=37466 dport=80 packets=5 bytes=336 src=172.20.211.48 dst=172.20.211.52 sport=8080 dport=37466 packets=5 bytes=458 [ASSURED] mark=0 use=1
tcp      6 55 TIME_WAIT src=192.168.10.20 dst=10.105.101.118 sport=52108 dport=80 packets=6 bytes=398 src=172.20.211.48 dst=192.168.10.20 sport=8080 dport=18188 packets=4 bytes=387 [ASSURED] mark=0 use=1

A conntrack entry reads in two halves: the original direction (who → 10.105.101.118:80) and the expected reply (from 172.20.211.48:8080). For the connection from the pod the reply is expected on the same port 37466 the pod came from: the source was not changed, and the hello pod sees the real address 172.20.211.52. For the connection from the node the reply port is 18188, not the original 52108: the source was outside the pod network, so the mark fired, and masquerading with fully-random picked a new port. The number after tcp 6 is how many seconds the entry has left: a closed connection stays in TIME_WAIT for two minutes.

Rejections. Create the legacy-api service from the section above — only the Service for now, without a slice. A service without addresses, a wrong ClusterIP port and an unallocated address of the service network behave differently:

$ kubectl -n demo exec client -- wget -qO- -T 3 http://legacy-api/
wget: can't connect to remote host (10.110.229.142): Connection refused
command terminated with exit code 1
user@k8s-node-1:~$ sudo nft list map ip kube-proxy no-endpoint-services
table ip kube-proxy {
	map no-endpoint-services {
		type ipv4_addr . inet_proto . inet_service : verdict
		comment "vmap to drop or reject packets to services with no endpoints"
		elements = { 10.110.229.142 . tcp . 80 comment "demo/legacy-api:http" : goto reject-chain }
	}
}
$ kubectl -n demo exec client -- wget -qO- -T 3 http://10.105.101.118:8080/
wget: can't connect to remote host (10.105.101.118): Connection refused
command terminated with exit code 1
$ kubectl -n demo exec client -- wget -qO- -T 3 http://10.100.0.99/
wget: download timed out
command terminated with exit code 1

legacy-api has neither a selector nor a slice: kube-proxy puts it into the no-endpoint-services map with a jump to reject-chain, and the client gets a refusal at once instead of waiting for a timeout. Port 8080 of the hello ClusterIP (the service has only port 80) is rejected by cluster-ips-check, and the address 10.100.0.99, which nobody owns, is silently dropped. The restrictions on slice addresses are checked by the API server — this is its answer to a slice legacy-bad with the address 0.0.0.0:

The EndpointSlice "legacy-bad" is invalid: endpoints[0].addresses[0]: Invalid value: "0.0.0.0": may not be unspecified (0.0.0.0)

externalTrafficPolicy: Local. Switch hello-np and go from the laptop to the nodes one by one:

$ kubectl -n demo patch svc hello-np -p '{"spec":{"externalTrafficPolicy":"Local"}}'
service/hello-np patched
laptop:~$ for i in 1 2 3; do curl -s http://192.168.10.20:30080/; done
hello from hello-6d5c6db6b4-d7bpf
hello from hello-6d5c6db6b4-d7bpf
hello from hello-6d5c6db6b4-d7bpf
laptop:~$ curl -s -m 3 http://192.168.10.17:30080/; echo "rc=$?"
rc=28
$ kubectl -n demo logs hello-6d5c6db6b4-d7bpf --since=20s
2026/10/28 20:31:18 192.168.10.50:56260 GET /
2026/10/28 20:31:18 192.168.10.50:56261 GET /
2026/10/28 20:31:18 192.168.10.50:56262 GET /

On k8s-node-1 all requests went to its only pod d7bpf, and it sees the real laptop address 192.168.10.50. The master has no pods, and the connection is not established (curl timed out, code 28). In the rules it looks like this:

user@k8s-node-1:~$ sudo nft list chain ip kube-proxy 'external-BJ5L76QF-demo/hello-np/tcp/http'
table ip kube-proxy {
	chain external-BJ5L76QF-demo/hello-np/tcp/http {
		ip saddr 172.20.0.0/16 goto service-BJ5L76QF-demo/hello-np/tcp/http comment "short-circuit pod traffic"
		fib saddr type local meta mark set meta mark | 0x00004000 comment "masquerade local traffic"
		fib saddr type local goto service-BJ5L76QF-demo/hello-np/tcp/http comment "short-circuit local traffic"
		goto local-BJ5L76QF-demo/hello-np/tcp/http
	}
}
user@k8s-node-1:~$ sudo nft list chain ip kube-proxy 'local-BJ5L76QF-demo/hello-np/tcp/http'
table ip kube-proxy {
	chain local-BJ5L76QF-demo/hello-np/tcp/http {
		meta l4proto tcp dnat ip to numgen random mod 1 map { 0 : 172.20.211.48 . 8080 }
	}
}
user@k8s-master-1:~$ sudo nft list map ip kube-proxy no-endpoint-nodeports
table ip kube-proxy {
	map no-endpoint-nodeports {
		type inet_proto . inet_service : verdict
		comment "vmap to drop or reject packets to service nodeports with no endpoints"
		elements = { tcp . 30080 comment "demo/hello-np:http" : drop }
	}
}

Traffic from the pod network and from the node itself (fib saddr type local — the source is an address of this node) goes to the common service chain, the rest to local-… with the node's only pod. The master has no pods of its own, and the NodePort lands in no-endpoint-nodeports with drop. hello-np has no healthCheckNodePort field — only a LoadBalancer has one. Switch hello-lb and ask the nodes on its health check port:

$ kubectl -n demo patch svc hello-lb -p '{"spec":{"externalTrafficPolicy":"Local"}}'
service/hello-lb patched
$ kubectl -n demo get svc hello-lb -o jsonpath='{.spec.healthCheckNodePort}{"\n"}'
31692
laptop:~$ curl -s -i http://192.168.10.21:31692/
HTTP/1.1 200 OK
Content-Type: application/json
X-Content-Type-Options: nosniff
X-Load-Balancing-Endpoint-Weight: 2
Date: Wed, 28 Oct 2026 20:31:36 GMT
Content-Length: 116

{
	"service": {
		"namespace": "demo",
		"name": "hello-lb"
	},
	"localEndpoints": 2,
	"serviceProxyHealthy": true
}
laptop:~$ curl -s -i http://192.168.10.17:31692/
HTTP/1.1 503 Service Unavailable
Content-Type: application/json
X-Content-Type-Options: nosniff
X-Load-Balancing-Endpoint-Weight: 0
Date: Wed, 28 Oct 2026 20:31:36 GMT
Content-Length: 116

{
	"service": {
		"namespace": "demo",
		"name": "hello-lb"
	},
	"localEndpoints": 0,
	"serviceProxyHealthy": true
}

kube-proxy answers on this port itself: k8s-node-2 has two local pods — 200, the master has none — 503. The documentation uses the /healthz path; kube-proxy answers the same on / too. The X-Load-Balancing-Endpoint-Weight header with the number of local pods is a hint for load balancers that can weight nodes and smooth out the imbalance of Local this way.

sessionAffinity and internalTrafficPolicy. Create two more services for the same pods — hello-sticky with sessionAffinity: ClientIP and hello-local with internalTrafficPolicy: Local (the manifests are like hello's, with one added field):

$ kubectl -n demo get svc hello-local hello-sticky
NAME           TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
hello-local    ClusterIP   10.102.119.174   <none>        80/TCP    3s
hello-sticky   ClusterIP   10.103.243.53    <none>        80/TCP    3s
$ kubectl -n demo get svc hello-sticky -o jsonpath='{.spec.sessionAffinityConfig}{"\n"}'
{"clientIP":{"timeoutSeconds":10800}}
$ kubectl -n demo exec client -- sh -c 'for i in 1 2 3 4 5 6; do wget -qO- http://hello-sticky/; done; echo ---; for i in 1 2 3 4 5 6; do wget -qO- http://hello-local/; done'
hello from hello-6d5c6db6b4-7k5d9
hello from hello-6d5c6db6b4-7k5d9
hello from hello-6d5c6db6b4-7k5d9
hello from hello-6d5c6db6b4-7k5d9
hello from hello-6d5c6db6b4-7k5d9
hello from hello-6d5c6db6b4-7k5d9
---
hello from hello-6d5c6db6b4-d7bpf
hello from hello-6d5c6db6b4-d7bpf
hello from hello-6d5c6db6b4-d7bpf
hello from hello-6d5c6db6b4-d7bpf
hello from hello-6d5c6db6b4-d7bpf
hello from hello-6d5c6db6b4-d7bpf

hello-sticky picked a pod at random once and sticks to it; hello-local sends the client only to the pod of its node k8s-node-1. Their rules are these (abridged):

user@k8s-node-1:~$ sudo nft list table ip kube-proxy
…
			     10.103.243.53 . tcp . 80 : goto service-4MXLGZVK-demo/hello-sticky/tcp/http,
…
			     10.102.119.174 . tcp . 80 : goto local-4H2GLPZP-demo/hello-local/tcp/http,
…
	set affinity-GWRCMAKN-demo/hello-sticky/tcp/http__172.20.172.17/8080 {
		type ipv4_addr
		size 65535	# count 1
		flags dynamic,timeout
		timeout 3h
		elements = { 172.20.211.52 expires 2h59m59s157ms }
	}
…
	chain endpoint-GWRCMAKN-demo/hello-sticky/tcp/http__172.20.172.17/8080 {
		update @affinity-GWRCMAKN-demo/hello-sticky/tcp/http__172.20.172.17/8080 { ip saddr }
		meta l4proto tcp dnat to 172.20.172.17:8080
	}
…
	chain service-4MXLGZVK-demo/hello-sticky/tcp/http {
		ip saddr @affinity-GWRCMAKN-demo/hello-sticky/tcp/http__172.20.172.17/8080 goto endpoint-GWRCMAKN-demo/hello-sticky/tcp/http__172.20.172.17/8080
		ip saddr @affinity-D7LANPZH-demo/hello-sticky/tcp/http__172.20.172.19/8080 goto endpoint-D7LANPZH-demo/hello-sticky/tcp/http__172.20.172.19/8080
		ip saddr @affinity-EXLX3BV3-demo/hello-sticky/tcp/http__172.20.211.48/8080 goto endpoint-EXLX3BV3-demo/hello-sticky/tcp/http__172.20.211.48/8080
		numgen random mod 3 vmap { 0 : goto endpoint-GWRCMAKN-demo/hello-sticky/tcp/http__172.20.172.17/8080, 1 : goto endpoint-D7LANPZH-demo/hello-sticky/tcp/http__172.20.172.19/8080, 2 : goto endpoint-EXLX3BV3-demo/hello-sticky/tcp/http__172.20.211.48/8080 }
	}

	chain local-4H2GLPZP-demo/hello-local/tcp/http {
		meta l4proto tcp dnat ip to numgen random mod 1 map { 0 : 172.20.211.48 . 8080 }
	}
}

For sessionAffinity every pod has its own affinity-… set with a three-hour timeout and its own endpoint-… chain: it records the client address in the set (update … { ip saddr }) and does the DNAT. The service chain first looks for the client in the sets and, if found, sends it to "its" pod, and only otherwise picks at random. Client 172.20.211.52 is already recorded for pod 172.20.172.17, and the entry expires three hours after the last connection. For hello-local the service-ips element leads straight to local-… with the node's pod. On the master, which has no pods, this service lands in no-endpoint-services with drop, and curl http://10.102.119.174/ from the master ends in a timeout.

trafficDistribution: PreferSameNode is checked on the lab the same way — a hello-near service with this field:

$ kubectl -n demo get endpointslice -l kubernetes.io/service-name=hello-near -o jsonpath='{range .items[*].endpoints[*]}{.addresses[0]} {.nodeName} hints={.hints}{"\n"}{end}'
172.20.172.17 k8s-node-2 hints={"forNodes":[{"name":"k8s-node-2"}]}
172.20.211.48 k8s-node-1 hints={"forNodes":[{"name":"k8s-node-1"}]}
172.20.172.19 k8s-node-2 hints={"forNodes":[{"name":"k8s-node-2"}]}
user@k8s-node-1:~$ sudo nft list table ip kube-proxy | grep -A2 'hello-near/tcp/http {'
	chain service-AKGLRN4Y-demo/hello-near/tcp/http {
		meta l4proto tcp dnat ip to numgen random mod 1 map { 0 : 172.20.211.48 . 8080 }
	}
user@k8s-master-1:~$ sudo nft list table ip kube-proxy | grep -A2 'hello-near/tcp/http {'
	chain service-AKGLRN4Y-demo/hello-near/tcp/http {
		meta l4proto tcp dnat ip to numgen random mod 3 map { 0 : 172.20.172.17 . 8080, 1 : 172.20.172.19 . 8080, 2 : 172.20.211.48 . 8080 }
	}
$ kubectl -n demo delete svc hello-near
service "hello-near" deleted from demo namespace

The EndpointSlice controller gave every address a hint with its node. On k8s-node-1 the map holds only its own pod, while the master has no pods of its own and its map holds all three — unlike internalTrafficPolicy: Local, traffic is not lost.

A headless service and ExternalName through the eyes of DNS. Create hello-headless with clusterIP: None (the same selector and port) and registry from the section above:

$ kubectl -n demo get svc hello-headless registry
NAME             TYPE           CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
hello-headless   ClusterIP      None         <none>        80/TCP    3s
registry         ExternalName   <none>       ghcr.io       <none>    3s
$ kubectl -n demo exec client -- nslookup hello-headless.demo.svc.cluster.local
Server:		10.96.0.10
Address:	10.96.0.10:53


Name:	hello-headless.demo.svc.cluster.local
Address: 172.20.172.17
Name:	hello-headless.demo.svc.cluster.local
Address: 172.20.211.48
Name:	hello-headless.demo.svc.cluster.local
Address: 172.20.172.19

$ kubectl -n demo exec client -- nslookup registry.demo.svc.cluster.local
Server:		10.96.0.10
Address:	10.96.0.10:53

registry.demo.svc.cluster.local	canonical name = ghcr.io

registry.demo.svc.cluster.local	canonical name = ghcr.io
Name:	ghcr.io
Address: 140.82.121.34

For the headless service name DNS returned the addresses of all three pods, while the kube-proxy table has not a single line about hello-headless, and it is not in cluster-ips. For registry a CNAME to ghcr.io came back (busybox prints it twice) followed by the real address of ghcr.io.

Finally, service addresses as objects:

$ kubectl get servicecidrs
NAME         CIDRS          AGE
kubernetes   10.96.0.0/12   4d10h
$ kubectl get ipaddresses
NAME             PARENTREF
10.102.119.174   services/demo/hello-local
10.103.243.53    services/demo/hello-sticky
10.105.101.118   services/demo/hello
10.105.48.156    services/demo/hello-np
10.110.229.142   services/demo/legacy-api
10.96.0.1        services/default/kubernetes
10.96.0.10       services/kube-system/kube-dns
10.97.232.108    services/demo/hello-lb
10.97.64.183     services/calico-system/calico-api
10.98.231.113    services/calico-system/calico-typha

The headless service and ExternalName have no IPAddress object — they have no address. All automatically assigned addresses are from the upper band of the range; the lower one holds only 10.96.0.1 and 10.96.0.10.

Clean up the training objects: kubectl -n demo delete svc hello-np hello-lb hello-local hello-sticky hello-headless legacy-api registry, kubectl -n demo delete pod client node-debugger-k8s-node-1-2srlg. If you added the legacy-api-1 slice to legacy-api, delete it too (kubectl -n demo delete endpointslice legacy-api-1): a slice created by hand has no owner reference and does not disappear together with the service. We keep the hello service — it is needed later.

---

Sources

• Service — service types, targetPort by name, NodePort and the port range, LoadBalancer, loadBalancerClass and allocateLoadBalancerNodePorts, headless, ExternalName and its problems with HTTP and HTTPS, a service without a selector and the address restrictions.
• Virtual IPs and Service Proxies — kube-proxy modes iptables, IPVS and nftables, nftables differences when migrating, IPVS deprecation, sessionAffinity, the ClusterIP range bands, traffic policies, traffic to terminating endpoints, trafficDistribution.
• EndpointSlices — the ready, serving, terminating conditions, the managed-by label, 100 addresses per slice.
• Service Internal Traffic Policy — internalTrafficPolicy: Local.
• Service ClusterIP allocation — how ClusterIPs are assigned, the lower and upper bands, worked examples.
• Extend Service IP Ranges — the ServiceCIDR and IPAddress objects.
• Create an External Load Balancer — externalTrafficPolicy and healthCheckNodePort.
• Using Source IP — the client address with Cluster and Local, checking healthCheckNodePort.
• kube-proxy — the --proxy-mode and --nodeport-addresses flags.
• NFTables mode for kube-proxy — why nftables, kernel and nft requirements, switching without rebooting nodes.
• Debugging Kubernetes Nodes With Kubectl — kubectl debug node and the sysadmin profile.
• nicolaka/netshoot — an image with network debugging tools.

---

Next — DNS in the cluster: how CoreDNS answers names of services and pods, which records exist for ordinary and headless services, what is in a pod's /etc/resolv.conf and why ndots:5 turns a query for api.example.com into several queries.