Rancher V2.3 Upgrade [HowTo]

Last week the new Rancher version was released – read about it here. In this Rancher V2.3 Upgrade HowTo, I present my approach. I couldn’t find a direct guide, which is why this article found its way onto my blog.

Aha, what’s new in V2.3?

Well, quite a bit. This was a feature release – not just fixes! Some of the new features seem very exciting. Here are a few keywords: Windows GA (only with Flannel), Enterprise Templating, ISTIO. Here is the link to the release notes.

Otherwise, the Kubernetes version was updated to 1.15.4 (stable). The menu under Projects has changed. The Workloads view is now hidden under “Resources”. The order of the Project navigation also changed significantly – that menu entry is now in first place. Additionally, there are more menu items including ISTIO.

In this Rancher V2.3 Upgrade HowTo, we’re not looking at the new features. I want to present the upgrade procedure here.

Rancher V2.3 Upgrade… Where? How?

Those who have followed the Kubernetes einfach mit Rancher Series should know what environment we’re dealing with. For everyone else, this article is recommended. In that article, I describe the environment that’s being updated in this article.

Before the update, Rancher was at version 2.2.8 and the cluster was running 1.14.x.

If Docker still needs to be updated on the nodes, you can follow this approach.

Rancher V2.3 Upgrade: Master

Updating Rancher itself is not difficult and took me less than 3 minutes. During the upgrade – the Docker image is being updated – the Rancher UI is not reachable.

To continue, we first need to gather some information. This is easily obtained via docker ps:

aytac@master:~$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
<containerid> rancher/rancher:<image_tag> "entrypoint.sh --acm…" 6 weeks ago Up 5 weeks 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp <container_name>

Important are the <image_tag> and <container_name> information – for me these were for example latest and silly_sinoussi.

Stop the Docker Container

After gathering the necessary information, we can proceed. First we need to stop the running Rancher container. We do this using the docker stop command with the container name:

docker stop <container_name>

For my example, this looks like:

docker stop silly_sinoussi

This command stops the current instance. Then we can create a backup – this step is optional but recommended, because we’ll use the created volume in the next-but-one step.

Back up the Current Environment

In this step, we back up the “volume” of our current environment. In case something goes wrong, we can restore the data.

First, we create a volume which we’ll then “tarball” and “gzip” – all with Docker:

docker create --volumes-from <container_name> --name rancher-data rancher/rancher:<image_tag>

For me, this command looked like:

docker create --volumes-from silly_sinoussi --name rancher-data rancher/rancher:latest

With this instruction, we create a new volume named “rancher-data”, based on the current container and its image. After the create instruction has completed, we now create the backup file:

docker run --volumes-from rancher-data -v $PWD:/backup busybox tar zcvf /backup/rancher-data-backup-<rancher-version>-<Date>.tar.gz /var/lib/rancher

So if I were to fire this command today – always with the premise that the Rancher version is 2.2.8 – it would look like this:

docker run --volumes-from rancher-data -v $PWD:/backup busybox tar zcvf /backup/rancher-data-backup-228-20191015.tar.gz /var/lib/rancher

With this command, we create a gzip package in the path where the command is executed, containing the contents of /var/lib/rancher. The package is then copied to the backup folder.

Pull New Image and Start Rancher

The upgrade is now completed by updating the image. With a docker pull command, we can pull the new image:

docker pull rancher/rancher:<image_tag>

I don’t use a specific image, so for my case the command looks like this:

docker pull rancher/rancher:latest

After pulling the current image, we can now restart our Rancher environment. There are various ways to do this. I use Let’s Encrypt as the certificate issuer, which is why I run:

docker run -d --volumes-from rancher-data \
 --restart=unless-stopped \
 -p 80:80 -p 443:443 \
 rancher/rancher:<image_tag> \
 --acme-domain <domain>

For my case, the command would look like this:

docker run -d --volumes-from rancher-data \
 --restart=unless-stopped \
 -p 80:80 -p 443:443 \
 rancher/rancher:latest \
 --acme-domain master.ak8s.de

We start a container with the new (pulled) image and attach the volume we created in the “backup” step. Additionally, we use the --acme-domain parameter to enable automatic certificate request and creation from Let’s Encrypt.

So, that completes this part. Our Rancher installation should now show version 2.3 (bottom left in the browser).

Rancher of course has a guide for this, covering among other things the restoration from backup and a few other topics. This guide can be accessed via this link.

Rancher V2.3 Upgrade: Cluster Nodes

This step is relatively simple and can be executed via the Rancher UI. To do this, we go into the Edit view of our cluster: Global → Cluster → 3 dots → Edit.

Under Kubernetes Options, we select the Kubernetes version. In this case, it’s 1.15.4.

We confirm our changes with the Save button. The upgrade process for the worker nodes then begins. We can monitor the status in the Cluster detail view.

After a successful upgrade, the worker nodes should display the new Kubernetes version in the Nodes view.

Conclusion

The upgrade process takes approximately half an hour. Whether that half hour is worth it is a personal decision. For my part – as an old infrastructure guy – I feel uncomfortable when I don’t install updates.

I couldn’t detect any significant changes after the update. What immediately caught my eye was the changed navigation. I can’t currently test new features like Enterprise Templates or Windows Container support. Maybe I’ll look into the simplified ISTIO commissioning, but currently I have no need for a micro-service mesh as I’m experimenting with OpenFaaS. And if the need for a service mesh environment ever arises, I’d lean more toward Rancher RIO.