This article is about using containerd 1.1 directly with Kubernetes instead of Docker.
Since 2008 with the first release of LXC and the release of Docker in 2013 a lot has happened in Linux container technology. With the launch of the Open Container Initiative (OCI) in June 2015 new standards evolved and were adapted by container runtimes. The runtime-spec was implemented by runc, and containerd using runc became an industry-standard for container management integrated into Docker starting with release 1.11.
In May 2018 Kubernetes announced general availabilty of the containerd integration. Since Kubernetes 1.5 container runtimes are integrated through the Container Runtime Interface (CRI). The CRI is basically a gRPC API which allows Kubelet to interface with a container runtime. This is often implemented through an adapter, the “shims”. The Kubelet CRI client would interact with a CRI shim (gRPC server) which translates from CRI to specific container runtime. With the release of containerd 1.1 the CRI became part of containerd so Kubelet and containerd can now interact directly. As improving performance was one of the major focus items of the containerd 1.1 release, benchmarks comparing containerd 1.1 and Docker Engine 18.03 CE (based on containerd 1.0) show container startup latency, CPU and memory usage improvements.
So, as Docker is based on containerd and runc what do I need Docker for? The following tutorial replaces Docker by directly using containerd.
Starting point is a vanilla Ubuntu >=16.04 installation.
sudo apt install -y unzip tar btrfs-tools libseccomp2 socat util-linux apt-transport-https curl
Setup containerd and runc
- containerd: v1.1.2
- runc: 1.0.0-rc5
sudo su - wget https://github.com/containerd/containerd/releases/download/v1.1.2/containerd-1.1.2.linux-amd64.tar.gz tar xf containerd-1.1.2.linux-amd64.tar.gz -C /usr/local wget -O /usr/local/sbin/runc https://github.com/opencontainers/runc/releases/download/v1.0.0-rc5/runc.amd64 chmod 755 /usr/local/sbin/runc containerd --version runc --version
To start the containerd daemon we need a systemd service file, which is provided by containerd/cri.
sudo su - curl -o /etc/systemd/system/containerd.service https://raw.githubusercontent.com/containerd/cri/master/contrib/systemd-units/containerd.service systemctl daemon-reload systemctl enable containerd systemctl start containerd systemctl status containerd
Setup Kubernetes with containerd
Now that the container runtime, the replacement for Docker, is ready, we can continue with the actual cluster setup.
sudo su - curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" > /etc/apt/sources.list.d/kubernetes.list apt update apt install -y cri-tools ebtables ethtool kubeadm kubectl kubelet kubernetes-cni
Notice the package cri-tools, it installs a tool called
crictl. It’s a CLI tool to connect to a CRI and manage the container runtime. It’s like the
docker command. To interact with containerd from
crictl without defining the CRI socket all the time, we create a configuration file to point crictl to the correct socket address.
sudo su - echo "runtime-endpoint: unix:///run/containerd/containerd.sock" > /etc/crictl.yaml
Now to test the containerd setup, we pull the first image.
crictl pull k8s.gcr.io/pause:3.1 crictl images IMAGE TAG IMAGE ID SIZE k8s.gcr.io/pause 3.1 da86e6ba6ca19 317kB
Before creating the cluster
kubelet needs to be pointed to the custom container runtime endpoint by setting
KUBELET_EXTRA_ARGS through a
systemd service overwrite.
cat <<EOF > /etc/systemd/system/kubelet.service.d/0-containerd.conf [Service] Environment="KUBELET_EXTRA_ARGS=--container-runtime=remote --runtime-request-timeout=15m --container-runtime-endpoint=unix:///run/containerd/containerd.sock" EOF systemctl daemon-reload
Next, we initialize the cluster using
kubeadm. Note: Swap should be disabled permanently. net.ipv4.ip_forward=1 should be persisted.
swapoff -a sysctl net.ipv4.ip_forward=1 kubeadm init --ignore-preflight-errors=all --cri-socket /run/containerd/containerd.sock --pod-network-cidr=192.168.0.0/16
kubelet already has the containerd socket configured,
kubeadm also needs to be pointed at the CRI socket of containerd to dispatch all runtime related operations to containerd (eg. pulling Kubernetes images).
mkdir -p $HOME/.kube cp -i /etc/kubernetes/admin.conf $HOME/.kube/config chown $(id -u):$(id -g) $HOME/.kube/config
After the cluster is initialized you should be able to query the K8s API.
kubectl get nodes NAME STATUS ROLES AGE VERSION ubuntu NotReady master 2m v1.11.0
To complete the setup, a network provider needs to be installed.
kubectl apply -f \ https://docs.projectcalico.org/v3.1/getting-started/kubernetes/installation/hosted/canal/rbac.yaml kubectl apply -f \ https://docs.projectcalico.org/v3.1/getting-started/kubernetes/installation/hosted/canal/canal.yaml
After the network provider setup, your cluster should be
kubectl get nodes NAME STATUS ROLES AGE VERSION ubuntu Ready master 9m v1.11.0
About the author
Simon is a hands on Kubernetes consultant helping customers to bring Kubernetes to production. If you’ve questions feel free to contact him on Twitter or drop him an Email.