install_micro_k8s.md

This section describes how to deploy the MicroK8s Kubernetes platform and configure it to be used with ETSI TeraFlowSDN controller. Besides, Docker is installed to build docker images for the ETSI TeraFlowSDN controller.

The steps described in this section might take some minutes depending on your internet connection speed and the resources assigned to your VM, or the specifications of your physical server.

To facilitate work, these steps are easier to be executed through an SSH connection, for instance using tools like PuTTY or MobaXterm.

Upgrade the Ubuntu distribution

Skip this step if you already did it during the creation of the VM.

sudo apt-get update -y
sudo apt-get dist-upgrade -y

Install prerequisites

sudo apt-get install -y ca-certificates curl gnupg lsb-release snapd jq

Install Docker CE

Install Docker CE and Docker BuildX plugin

sudo apt-get install -y docker.io docker-buildx

NOTE: Starting from Docker v23, Build architecture has been updated and docker build command entered into deprecation process in favor of the new docker buildx build command. Package docker-buildx provides the new docker buildx build command.

Add key "insecure-registries" with the private repository to the daemon configuration. It is done in two commands since sometimes read from and write to same file might cause trouble.

if [ -s /etc/docker/daemon.json ]; then cat /etc/docker/daemon.json; else echo '{}'; fi \
    | jq 'if has("insecure-registries") then . else .+ {"insecure-registries": []} end' -- \
    | jq '."insecure-registries" |= (.+ ["localhost:32000"] | unique)' -- \
    | tee tmp.daemon.json
sudo mv tmp.daemon.json /etc/docker/daemon.json
sudo chown root:root /etc/docker/daemon.json
sudo chmod 600 /etc/docker/daemon.json

Restart the Docker daemon

sudo systemctl restart docker

Install MicroK8s

Important: Some TeraFlowSDN dependencies need to be executed on top of MicroK8s/Kubernetes v1.24. It is not guaranteed (by now) to run on newer versions.

# Install MicroK8s
sudo snap install microk8s --classic --channel=1.24/stable

# Create alias for command "microk8s.kubectl" to be usable as "kubectl"
sudo snap alias microk8s.kubectl kubectl

It is important to make sure that ufw will not interfere with the internal pod-to-pod and pod-to-Internet traffic. To do so, first check the status. If ufw is active, use the following command to enable the communication.

# Verify status of ufw firewall
sudo ufw status

# If ufw is active, install following rules to enable access pod-to-pod and pod-to-internet
sudo ufw allow in on cni0 && sudo ufw allow out on cni0
sudo ufw default allow routed

NOTE: MicroK8s can be used to compose a Highly Available Kubernetes cluster enabling you to construct an environment combining the CPU, RAM and storage resources of multiple machines. If you are interested in this procedure, review the official instructions in How to build a highly available Kubernetes cluster with MicroK8s, in particular, the step Create a MicroK8s multi-node cluster.

References:

• The lightweight Kubernetes > Install MicroK8s
• Install a local Kubernetes with MicroK8s
• How to build a highly available Kubernetes cluster with MicroK8s

Add user to the docker and microk8s groups

It is important that your user has the permission to run docker and microk8s in the terminal. To allow this, you need to add your user to the docker and microk8s groups with the following commands:

sudo usermod -a -G docker $USER
sudo usermod -a -G microk8s $USER
sudo chown -f -R $USER $HOME/.kube
sudo reboot

In case that you get trouble executing the following commands, might due to the .kube folder is not automatically provisioned into your home folder, you may follow the steps below:

mkdir -p $HOME/.kube
sudo chown -f -R $USER $HOME/.kube
microk8s config > $HOME/.kube/config
sudo reboot

Check status of Kubernetes and addons

To retrieve the status of Kubernetes once, run the following command:

microk8s.status --wait-ready

To retrieve the status of Kubernetes periodically (e.g., every 1 second), run the following command:

watch -n 1 microk8s.status --wait-ready

Check all resources in Kubernetes

To retrieve the status of the Kubernetes resources once, run the following command:

kubectl get all --all-namespaces

To retrieve the status of the Kubernetes resources periodically (e.g., every 1 second), run the following command:

watch -n 1 kubectl get all --all-namespaces

Enable addons

First, we need to enable the community plugins (maintained by third parties):

microk8s.enable community

The Addons to be enabled are:

• dns: enables resolving the pods and services by name
• helm3: required to install NATS
• hostpath-storage: enables providing storage for the pods (required by registry)
• ingress: deploys an ingress controller to expose the microservices outside Kubernetes
• registry: deploys a private registry for the TFS controller images
• linkerd: deploys the linkerd service mesh used for load balancing among replicas
• prometheus: set of tools that enable TFS observability through per-component instrumentation
• metrics-server: deploys the Kubernetes metrics server for API access to service metrics

microk8s.enable dns helm3 hostpath-storage ingress registry prometheus metrics-server linkerd

Important: Enabling some of the addons might take few minutes. Do not proceed with next steps until the addons are ready. Otherwise, the deployment might fail. To confirm everything is up and running:

1. Periodically Check the status of Kubernetes until you see the addons [dns, ha-cluster, helm3, hostpath-storage, ingress, linkerd, metrics-server, prometheus, registry, storage] in the enabled block.
2. Periodically Check Kubernetes resources until all pods are Ready and Running.
3. If it takes too long for the Pods to be ready, we observed that rebooting the machine may help.

Then, create aliases to make the commands easier to access:

sudo snap alias microk8s.helm3 helm3
sudo snap alias microk8s.linkerd linkerd

To validate that linkerd is working correctly, run:

linkerd check

To validate that the metrics-server is working correctly, run:

kubectl top pods --all-namespaces

and you should see a screen similar to the top command in Linux, showing the columns namespace, pod name, CPU (cores), and MEMORY (bytes).

In case pods are not starting, check information from pods logs. For example, linkerd is sensitive for proper /etc/resolv.conf syntax.

kubectl logs <podname> --namespace <namespace>

If the command shows an error message, also restarting the machine might help.

Stop, Restart, and Redeploy

Find below some additional commands you might need while you work with MicroK8s:

microk8s.stop  # stop MicroK8s cluster (for instance, before power off your computer)
microk8s.start # start MicroK8s cluster
microk8s.reset # reset infrastructure to a clean state

If the following commands does not work to recover the MicroK8s cluster, you can redeploy it.

If you want to keep MicroK8s configuration, use:

sudo snap remove microk8s

If you need to completely drop MicroK8s and its complete configuration, use:

sudo snap remove microk8s --purge
sudo apt-get remove --purge docker.io docker-buildx

IMPORTANT: After uninstalling MicroK8s, it is convenient to reboot the computer (the VM if you work on a VM, or the physical computer if you use a physical computer). Otherwise, there are system configurations that are not correctly cleaned. Especially in what port forwarding and firewall rules matters.

After the reboot, redeploy as it is described in this section.