Azure Container Services (AKS) - Upgrading your Kubernetes cluster

While working a lot with AKS and Kubernetes the last couple of months, I've found it to be a pretty slick experience using the CLI even if it still has some room for improvement.

The question of how to upgrade Kubernetes running in an Azure AKS cluster came up a few times both offline and online, thus I wanted to put a short post together on this topic.

I've redacted any details pointing to my subscription for obvious reasons, and in its place you will see "redacted" instead of the subscription-specific values of my cmds

Get current version of Kubernetes in your Azure AKS cluster

Get-versions: What version of Kubernetes are you on?

First we'll need to figure out what version you're currently on. This is easy:

az aks get-versions -g resourceGroupName -n aksClusterName  

Results will look similar to this, obviously with different version numbers if you aren't on exactly the same as mine. It is also likely that there's a later version than 1.8.7 when you're running this.

{
  "agentPoolProfiles": [
    {
      "kubernetesVersion": "1.7.7",
      "name": null,
      "osType": "Linux",
      "upgrades": [
        "1.8.6",
        "1.7.9",
        "1.8.2",
        "1.8.7",
        "1.7.12",
        "1.8.1"
      ]
    }
  ],
  "controlPlaneProfile": {
    "kubernetesVersion": "1.7.7",
    "name": null,
    "osType": "Linux",
    "upgrades": [
      "1.8.6",
      "1.7.9",
      "1.8.2",
      "1.8.7",
      "1.7.12",
      "1.8.1"
    ]
  },
  "id": "/subscriptions/REDACTED/resourcegroups/resourceGroupName/providers/Microsoft.ContainerService/managedClusters/aksClusterName/upgradeprofiles/default",
  "name": "default",
  "resourceGroup": "resourceGroupName",
  "type": "Microsoft.ContainerService/managedClusters/upgradeprofiles"
}

We can see that we are running 1.7.7 but there are several upgrades available that we can upgrade to, for example the latest one as of this writing being 1.8.7.

Upgrade Azure AKS kubernetes control and agents to the latest version

First a word of caution. You should -always- ready the release notes, and also do a test-upgrade on a dev/test/pre-prod cluster before doing it with your critical system. But that's just common sense.

(Good place to start: https://github.com/kubernetes/kubernetes/releases)

Right, now that that's out of the way, we can start the upgrade:

az aks upgrade -g resourceGroupName -n aksClusterName --kubernetes-version "1.8.7"  

This will give you a prompt to confirm this choice, living on the edge!

Kubernetes may be unavailable during cluster upgrades.  
Are you sure you want to perform this operation? (y/n): y  
 - Running ...

You can optionally use the --no-wait param to not have to sit and wait for it in the console. If you did that, or you want to spin up a new console to check the status, you can do that too.

Verify upgrade

Once the upgrade has finished, you are able to see it directly after your previously issued command, and hopefully it'll look something like this when your upgrade-command has succeeded. (Please note the version numbers now saying 1.8.7 instead of 1.7.7)

Verify Kubernetes upgrade by checking the output of the upgrade command

If you waited in the console for the command to finish, the results will be similar to this:

{
  "agentPoolProfiles": [
    {
      "count": 2,
      "dnsPrefix": null,
      "fqdn": null,
      "name": "nodepool1",
      "osDiskSizeGb": null,
      "osType": "Linux",
      "ports": null,
      "storageProfile": "ManagedDisks",
      "vmSize": "Standard_A4",
      "vnetSubnetId": null
    }
  ],
  "dnsPrefix": "redacted",
  "fqdn": "redacted.hcp.westeurope.azmk8s.io",
  "id": "/subscriptions/redacted/resourcegroups/resourceGroupName/providers/Microsoft.ContainerService/managedClusters/aksClusterName",
  "kubernetesVersion": "1.8.7",
  "linuxProfile": {
    "adminUsername": "azureuser",
    "ssh": {
      "publicKeys": [
        {
          "keyData": "redacted"
        }
      ]
    }
  },
  "location": "westeurope",
  "name": "aksClusterName",
  "provisioningState": "Succeeded",
  "resourceGroup": "resourceGroupName",
  "servicePrincipalProfile": {
    "clientId": "redacted",
    "keyVaultSecretRef": null,
    "secret": null
  },
  "tags": null,
  "type": "Microsoft.ContainerService/ManagedClusters"
}

Verify Kubernetes upgrade with get-version command

If you didn't stick around in the console or perhaps used the --no-wait flag, then you can use the get-versions command again as detailed above. It will hopefully now say that there's no upgrades (indicated by "upgrades": null).

{
  "agentPoolProfiles": [
    {
      "kubernetesVersion": "1.8.7",
      "name": null,
      "osType": "Linux",
      "upgrades": null
    }
  ],
  "controlPlaneProfile": {
    "kubernetesVersion": "1.8.7",
    "name": null,
    "osType": "Linux",
    "upgrades": null
  },
  "id": "/subscriptions/redacted/resourcegroups/resourceGroupName/providers/Microsoft.ContainerService/managedClusters/aksClusterName/upgradeprofiles/default",
  "name": "default",
  "resourceGroup": "resourceGroupName",
  "type": "Microsoft.ContainerService/managedClusters/upgradeprofiles"
}

Enjoy & thanks for reading.

by Tobias Zimmergren via Zimmergren's thoughts on tech