Vitaliy Leschenko’s blog
ru
Виталий Лещенко

How to Set Up ContainerD as the CRI for Windows Nodes in a Kubernetes Cluster

Previously on...

First, let's go over what has changed since I last wrote about setting up a cluster with Windows nodes:

  1. the current version of Kubernetes is 1.22;
  2. besides Flannel, Calico can now also be used as the CNI;
  3. Windows Server 2022 has been released;
  4. CRI-containerD can now be used, including in HostProcess mode.

A few words about Docker

We won't be using it in the context of Kubernetes. We are now only interested in ContainerD. That said, it's too early to uninstall Docker, because it's still needed to build images.

Now for the good part

I'll show how to set up a Kubernetes cluster with nodes based on Windows Server 2022 and CRI-ContainerD. Calico will be the CNI of the cluster.

Step one: installing a new cluster

Nothing has fundamentally changed here. Take a clean Ubuntu 20.04 LTS and download the packages for ContainerD, kubeadm, kubelet and kubectl. Then deploy the cluster with kubeadm. Since we didn't install Docker, as expected, the CRI on the master node will be the ContainerD we installed.

Step two: installing the CNI - Calico

Calico can work in 2 modes:

  • BGP - the default, but it doesn't suit us
  • VXLAN - this is the mode we'll use

Download calico.yaml ...

curl https://docs.projectcalico.org/manifests/calico.yaml -O

... and start modifying it:

  1. swap the values of the CALICO_IPV4POOL_IPIP and CALICO_IPV4POOL_VXLAN variables
env:
    # Enable IPIP
    - name: CALICO_IPV4POOL_IPIP
      value: "Never"
    # Enable or Disable VXLAN on the default IP pool.
    - name: CALICO_IPV4POOL_VXLAN
      value: "Always"
  1. replace calico_backend: "bird" with calico_backend: "vxlan"
kind: ConfigMap
apiVersion: v1
metadata:
     name: calico-config
     namespace: kube-system
data:
     calico_backend: "vxlan"
  1. disable the bird checks in the readiness/liveness probes.
livenessProbe:
     exec:
       command:
       - /bin/calico-node
       - -felix-live
       # - -bird-live
readinessProbe:
     exec:
       command:
       - /bin/calico-node
       # - -bird-ready
       - -felix-ready
  1. in the end we should get something like this. Apply the yaml
kubectl apply -f calico.yaml

A small digression here about the number of IP addresses per node. By default it's only 64 addresses. This can be changed, but more on that later.

Step three: preparing the cluster for Windows nodes

We're missing the DaemonSets for Windows nodes. Get them here: https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/hostprocess/calico/calico.yml https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/hostprocess/calico/kube-proxy/kube-proxy.yml

And for them (the DaemonSets) to work, make sure the cluster configuration contains all the required information:

#> kubectl -n kube-system get configmap/kubeadm-config -oyaml
apiVersion: v1
data:
  ClusterConfiguration: |
    apiServer:
      extraArgs:
        authorization-mode: Node,RBAC
      timeoutForControlPlane: 4m0s
    apiVersion: kubeadm.k8s.io/v1beta3
    certificatesDir: /etc/kubernetes/pki
    clusterName: kubernetes
    controllerManager: {}
    controlPlaneEndpoint: 192.168.99.150:6443
    dns: {}
    etcd:
      local:
        dataDir: /var/lib/etcd
    imageRepository: k8s.gcr.io
    kind: ClusterConfiguration
    kubernetesVersion: v1.22.2
    networking:
      dnsDomain: cluster.local
      podSubnet: 10.244.0.0/16
      serviceSubnet: 10.96.0.0/12
    scheduler: {}
kind: ConfigMap
metadata:
  creationTimestamp: "2021-10-24T08:02:20Z"
  name: kubeadm-config
  namespace: kube-system
  resourceVersion: "1168673"
  uid: 396b0bb8-e6ad-4a7a-9611-42538084e1ef

The important part here is the line controlPlaneEndpoint: 192.168.99.150:6443. If it's missing (and it may be), you need to add it.

Step four: adding a Windows node with CRI - ContainerD

Download 2 files:

  1. Install-Containerd.ps1
  2. PrepareNode.ps1

At the time of writing this little article, the working version of CRI-ContainerD was 1.6.0-beta.1. Run its installation:

Install-Containerd.ps1 -ContainerDVersion 1.6.0-beta.1

Install all the required software (kubelet, kubeadm, ...)

PrepareNode.ps1 -ContainerRuntime containerD -KubernetesVersion 1.22.2

Connect to the master over SSH and get a token for adding nodes to the cluster:

sudo kubeadm token create --print-join-command

You'll get something like this:

kubeadm join 192.168.99.150:6443 --token thmvc8.s7tfy38ej3dcawjt --discovery-token-ca-cert-hash sha256:0c6a4b9bc9f03b32dc133b79c491c6820bbb73f09b4edbeb77e6b55407a774eb

Recall that when installing ContainerD we were asked to add --cri-socket "npipe:////./pipe/containerd-containerd" to the kubeadm join command. That gives us the final command:

kubeadm join 192.168.99.150:6443 --token thmvc8.s7tfy38ej3dcawjt --discovery-token-ca-cert-hash sha256:0c6a4b9bc9f03b32dc133b79c491c6820bbb73f09b4edbeb77e6b55407a774eb --cri-socket "npipe:////./pipe/containerd-containerd"

Run it and enjoy the result. All done.

The result should look roughly like this:

PS C:\Users\v.leschenko> kubectl get nodes -owide
NAME             STATUS   ROLES                  AGE    VERSION   INTERNAL-IP      EXTERNAL-IP   OS-IMAGE                       KERNEL-VERSION     CONTAINER-RUNTIME
k8s-linux-1      Ready    control-plane,master   14d    v1.22.2   192.168.99.150   <none>        Ubuntu 20.04.3 LTS             5.4.0-89-generic   containerd://1.4.11
k8s-windows-1    Ready    <none>                 5d1h   v1.22.2   192.168.99.151   <none>        Windows Server 2022 Standard   10.0.20348.320     containerd://1.6.0-beta.1
k8s-windows-2    Ready    <none>                 2d2h   v1.22.2   192.168.99.152   <none>        Windows Server 2022 Standard   10.0.20348.320     containerd://1.6.0-beta.1

PS C:\Users\v.leschenko> kubectl -n kube-system get pods -owide
NAME                                       READY   STATUS    RESTARTS   AGE    IP               NODE             NOMINATED NODE   READINESS GATES
calico-kube-controllers-75f8f6cc59-6krlt   1/1     Running   0          162m   10.244.175.2     k8s-linux-1      <none>           <none>
calico-node-tggf4                          1/1     Running   0          162m   192.168.99.150   k8s-linux-1      <none>           <none>
calico-node-windows-6b8rc                  2/2     Running   0          158m   192.168.99.152   k8s-windows-2    <none>           <none>
calico-node-windows-ptx7m                  2/2     Running   0          136m   192.168.99.151   k8s-windows-1    <none>           <none>
coredns-78fcd69978-qsb9v                   1/1     Running   0          162m   10.244.175.3     k8s-linux-1      <none>           <none>
coredns-78fcd69978-wc5gd                   1/1     Running   0          162m   10.244.175.1     k8s-linux-1      <none>           <none>
etcd-k8s-linux-1                           1/1     Running   0          162m   192.168.99.150   k8s-linux-1      <none>           <none>
kube-apiserver-k8s-linux-1                 1/1     Running   0          162m   192.168.99.150   k8s-linux-1      <none>           <none>
kube-controller-manager-k8s-linux-1        1/1     Running   0          162m   192.168.99.150   k8s-linux-1      <none>           <none>
kube-proxy-windows-9p7xm                   1/1     Running   0          162m   192.168.99.151   k8s-windows-1    <none>           <none>
kube-proxy-windows-rnwtm                   1/1     Running   0          158m   192.168.99.152   k8s-windows-2    <none>           <none>
kube-proxy-x6h4q                           1/1     Running   0          162m   192.168.99.150   k8s-linux-1      <none>           <none>
kube-scheduler-k8s-linux-1                 1/1     Running   0          162m   192.168.99.150   k8s-linux-1      <none>           <none>
metrics-server-559f9dc594-c7vgv            1/1     Running   0          162m   10.244.175.4     k8s-linux-1      <none>           <none>

Only registered users can leave a comment.

Sign in or register to leave a comment.

© 2006-2026 Vitaliy Leschenko