Tutorial: Using Azure DevOps to setup a CI/CD pipeline and deploy to Kubernetes

Note: for its 1-year anniversary, I refreshed this blog article in November 2019 to leverage new features with Helm 3 and Azure Pipelines (mainly YAML for both Build/CI and Release/CD), as well as to incorporate great feedback we’ve been receiving from our readers. Thank you! 

Introduction

In this blog article, we will show you how to set up a CI/CD pipeline to deploy your apps on a Kubernetes cluster with Azure DevOps by leveraging a Linux agent, Docker, and Helm. The combination of these technologies will illustrate how you can easily set up a CI/CD pipeline, leverage Configuration-as-Code, and Infrastructure-as-Code, and accelerate your DevOps journey with containers.

Here are the technologies we will walkthrough below:

Azure DevOps helps to implement your CI/CD pipelines for any platform, any languages.

Docker adds more consistency and quality for your apps, their deployment, and management. Docker allows also to be programming languages agnostic, all your apps packaged as Docker images could be in different languages: .NET Core, Java, Node.js, Go, Python, etc.

Helm, as the package manager for Kubernetes, simplifies and automates more your apps deployments in Kubernetes. We will use the new Helm 3 throughout this tutorial.

Kubernetes is a container orchestrator that makes the collaboration between developers and IT pros easy and will for sure help you in the orchestration, management, and monitoring of all your apps containerized, in a consistent way.

Azure Kubernetes Service (AKS) is a fully managed Kubernetes container orchestration service, simplifying the process around creating, scaling, and upgrading your Kubernetes cluster. You are not paying for the master nodes since that’s part of the managed offer.

Azure Container Registry (ACR) is a dedicated enterprise container registry with advanced features like Helm chart repository, geo-replication of your registry across the globe, container build capabilities, security scanning for your images, etc.

Prerequisites & Setup

• Azure DevOps account: we will use an Azure DevOps project for a Git repo and build/release pipelines. Create your free account and a new project here.
• Azure Subscription: needed to provision the Azure services for this demonstration. If you don’t have one, you can get a free trial one here. Make sure the Multi-stage pipelines in the Preview experience is turned on.
• Bash Shell: we will leverage Azure Cloud Shell. Once your Azure Subscription is set up you can enable and use your associated Azure Cloud Shell session. In this article, Bash, which you can access here. Notes: you could use any local bash terminal. Make sure you have installed the Azure CLI version .0.73 or later and you have installed the Helm version 3.0.0 or later.
• Container registry: we will use Azure Container Registry (ACR) to store both your Docker images and your Helm charts. For that you can run the commands below from Azure Cloud Shell:

# Create a resource group $rg on a specific location $location (for example eastus) which will contain the Azure services we need 
$ az group create -l $location -n $rg
# Create an ACR registry $acr
$ az acr create -n $acr -g $rg -l $location --sku Basic

• Kubernetes cluster: we will leverage Azure Kubernetes Service (AKS). For that, you can run the commands below from Azure Cloud Shell:

# Setup of the AKS cluster
$ latestK8sVersion=$(az aks get-versions -l $location --query 'orchestrators[-1].orchestratorVersion' -o tsv)
$ az aks create -l $location -n $name -g $rg --generate-ssh-keys -k $latestK8sVersion
# Once created (the creation could take ~10 min), get the credentials to interact with your AKS cluster
$ az aks get-credentials -n $name -g $rg
# Setup the phippyandfriends namespace, you will deploy later some apps into it
$ kubectl create namespace phippyandfriends
$ kubectl create clusterrolebinding default-view --clusterrole=view --serviceaccount=phippyandfriends:default

• Azure Roles and Permissions: we need to assign 3 specific service principals with specific Azure Roles that need to interact with our ACR and our AKS

# 1. Assign acrpull role on our ACR to the AKS-generated service principal, the AKS cluster will then be able to pull images from our ACR
$ ACR_ID=$(az acr show -n $acr -g $rg --query id -o tsv)
$ az aks update -g $rg -n $aks --attach-acr $ACR_ID
# 2. Create a specific Service Principal for our Azure DevOps pipelines to be able to push and pull images and charts of our ACR
$ registryPassword=$(az ad sp create-for-rbac -n $acr-push --scopes $ACR_ID --role acrpush --query password -o tsv)
# Important note: you will need this registryPassword value later in this blog article in the Create a Build pipeline and Create a Release pipeline sections
$ echo $registryPassword
# 3. Create a specific Service Principal for our Azure DevOps pipelines to be able to deploy our application in our AKS
$ AKS_ID=$(az aks show -n $aks -g $rg --query id -o tsv)
$ aksSpPassword=$(az ad sp create-for-rbac -n $aks-deploy --scopes $AKS_ID --role "Azure Kubernetes Service Cluster User Role" --query password -o tsv)
# Important note: you will need this aksSpPassword value later in this blog article in the Create a Release pipeline section
$ echo $aksSpPassword

You now have an ACR registry and AKS cluster ready to be used throughout this blog article.

The DevOps workflow with containers

This is the DevOps workflow with containers illustrated in this blog article:

• Devs and Ops commit code change (apps, infrastructure-as-code, etc.) to Azure repos
• Azure build pipeline will build and push both the app as a Docker image and the Helm chart in an Azure Container Registry (ACR)
• Azure release pipeline will deploy the specific Helm chart to an Azure Kubernetes Service (AKS) cluster

Source control

We will use the existing Azure/phippyandfriends GitHub repo. This repo has a few services, each of them represents an individual character of the Children’s Guide to Kubernetes and their new friend NodeBrady. Each service is written in a different language, showing how an AKS cluster can run anything you can bring it. Parrot is in .NET Core, CaptainKube is in Go, Phippy in PHP and NodeBrady in Node.js.

First, you will need to import this repo https://github.com/Azure/phippyandfriends into your own Azure repos (from the Azure DevOps project you just created earlier):

Once imported you will be able to browse the source code of the four different apps: CaptainKube, NodeBrady, Parrot and Phippy:

Each app has its own folder and the same structure within it:

• Files of the app itself, depending on the programming languages: Parrot is in .NET Core, CaptainKube is in Go, Phippy in PHP and NodeBrady in Node.js.
• Dockerfile file is a script leveraged by Docker, composed of various commands (instructions) and arguments listed successively to automatically perform actions on a base image in order to create a new Docker image by packaging the app.
• charts/* folder contains the files defining Helm Chart of the app. Helm Charts helps you define, install, and upgrade your app in Kubernetes.

With the next sections, we will also leverage the ci-pipeline.yml and cd-pipeline.yml files, which play an important role as Configuration-as-Code for the Azure Pipeline definitions.

Create a Build / Continuous Integration (CI) pipeline

We will now create an Azure build pipeline for the Parrot app (parrot-ci) to be able to both build/push its Docker image and package/push its Helm chart. For that, we will need to create a build pipeline definition using the parrot/ci-pipeline.yml file. From the menu Pipelines > Pipelines, follow the steps illustrated below:

Here is how you could retrieve the values of the associated Variables by running the following commands:

• registryLogin: az ad sp show --id http://$acr-push --query appId -o tsv
• registryName: echo $acr
• registryPassword:You got this value in the Prerequisites & Setup section by setting up the associated service principal.

You could now Run Pipeline which will push both the Docker image and the Helm chart in your ACR. You could look at both the parrot/ci-pipeline.yml and the common/ci-steps-template.yml files to see the details for those steps.

Important note: you need to rename this Build definition by parrot-ci. This name is important as it will be reused as an input for the next CD/Release phase.

From there you can run some Azure CLI commands to check what you pushed in ACR:

# List Docker images from your ACR
$ az acr repository list -n $acr
# List Helm charts from your ACR
$ az acr helm list -n $acr
# Show details of a specific Helm chart from your ACR
$ az acr helm show chart-name -n $acr

Now both the Docker image and the Helm chart could be used for any Kubernetes cluster from anywhere: locally, etc. You will see in the next section how to use them with a release pipeline.

Create a Release / Continuous Delivery (CD) pipeline

We will now create an Azure release pipeline for the Parrot app (parrot-cd) to be able to deploy it via its associated Helm chart. For that, we will need to create a release pipeline definition using the parrot/cd-pipeline.yml file. From the menu Pipelines > Pipelines, follow the steps illustrated below:

Here is how you could retrieve the values of the associated Variables by running the following commands:

• rg: echo $rg
• aks: echo $aks
• aksSpId: az ad sp show --id http://$aks-deploy --query appId -o tsv
• aksSpSecret: You got this value in the Prerequisites & Setup section by setting up the associated service principal
• aksSpTenantId: az account show --query tenantId -o tsv
• registryLogin: az ad sp show --id http://$acr-push --query appId -o tsv
• registryName: echo $acr
• registryPassword: You got this value in the Prerequisites & Setup section by setting up the associated service principal

You could now Run pipeline which will deploy the Parrot’s Helm chart into our AKS cluster. You could look at both the parrot/cd-pipeline.yml and the common/cd-steps-template.yml files to see the details for those steps.

Once deployed successfully, you can now run this command from your Azure Cloud Shell to get the EXTERNAL-IP value of the service/parrot-parrot:

$ kubectl get all -n phippyandfriends

Then, open a web browser to navigate to this URL:

Congratulations! You have deployed the Parrot app on your Kubernetes cluster via a Helm Chart. With the next section, you will see how to deploy the other apps/friends.

Where are Phippy and the other friends?

You are right, we just have the Parrot app so far! You could now repeat with both Parrot’s Build and Release pipelines definitions you created and adapt them for CaptainKube, Phippy, and NodeBrady.

You should now have four build definitions and four release definitions that you can now run to eventually see the Parrot app showing its friends:

Further considerations

You may want to complete this tutorial with other features and concepts, here are a few examples:

• Approvals and Checks to pause your CI/CD pipeline with Manual Approvals.
• Variable groups and Azure KeyVault to reuse, store, and better protect your secrets.
• Azure DevOps CLI to automate all your Azure DevOps actions.
• Clone the development stage to add more environments/stages like production, etc.

Conclusion

Congratulations! You have deployed four different apps in a Kubernetes cluster by leveraging Azure DevOps, Azure Container Registry, Azure Kubernetes Service, Docker, and Helm! You have also used Azure Cloud Shell to run some Azure, Helm, and Kubernetes commands.

As illustrated throughout this blog post, all together the technologies used will help you create a generic, strong and robust DevOps pipeline to deploy your apps from any language to any platform. There are three central pieces simplifying the collaboration between the developers and the IT Pros – it’s about Configuration-as-Code and Infrastructure-as-Code:

• Dockerfile as the description of the packaging of your app
• Helm chart as the description of the deployment of your app on Kubernetes
• YAML Build and Release pipelines as the description of the CI/CD pipeline process in Azure DevOps

Hope you enjoyed this blog article and hopefully you will be able to leverage this for your own apps and needs. Questions or feedback? Let us know in the comments below. Cheers!

Resources:

• GitHub repository used to illustrate this blog article: Azure/phippyandfriends
• Kubernetes Learning Path – 50 days from zero to hero with Kubernetes
• Phippy and friends join the Cloud Native Computing Foundation
• Helm 3: Simpler and more secure

Mathieu Benoit

Cloud Solution Architect

See more articles from this author