{"id":77394,"date":"2019-06-11T09:00:11","date_gmt":"2019-06-11T16:00:11","guid":{"rendered":""},"modified":"2025-06-27T06:39:11","modified_gmt":"2025-06-27T13:39:11","slug":"five-steps-add-automated-performance-quality-gates-azure-devops-pipelines","status":"publish","type":"post","link":"https:\/\/opensource.microsoft.com\/blog\/2019\/06\/11\/five-steps-add-automated-performance-quality-gates-azure-devops-pipelines\/","title":{"rendered":"Five steps to add\u00a0automated performance quality gates to Azure DevOps pipelines"},"content":{"rendered":"\n

In our last post<\/a>, Daniel Semedo and I provided an overview of how to add automated performance quality gates using a performance specification file, as defined in the open source project Keptn Pitometer<\/a>.<\/p>\n\n\n\n

In this post, I\u2019ll explain the steps required to add a performance quality gate to your Azure DevOps <\/a>pipelines for both DevOps \u201cMulti-Stage\u201d and \u201cClassic\u201d pipelines<\/a> using Keptn Pitometer.<\/p>\n\n\n\n

Step 1: Setup Keptn Pitometer<\/h2>\n\n\n\n

As a quick refresher, Keptn Pitometer is not an application \u2013 it\u2019s a set of open source Node.js modules used to create your own Pitometer client. Pitometer provides the processing of a performance specification (PerfSpec). This PerfSpec file defines the metrics, evaluation thresholds, and scoring objectives that constitute the resulting pass, warning or fail status calculated by Pitometer.<\/p>\n\n\n\n

The quickest way to try Pitometer is to use the Pitometer webservice<\/a>. In Azure, this is as simple as starting up an Azure container instance<\/a> that uses the Pitometer webservice Docker image. This way any pipeline can add a quality gate by just calling webservice with their PerfSpec file. Follow these instructions<\/a> to create a new container instance through the Azure portal or Azure CLI.<\/p>\n\n\n\n

After the container instance is running, go to the overview page (shown below) and copy the IP address. Use this IP within the Pitometer webservice URL: http:\/\/<Your IP>:8080\/api\/pitometer.<\/p>\n\n\n\n

\"overview<\/figure>\n\n\n\n

Step 2: Create your PerfSpec file<\/h2>\n\n\n\n

The PerfSpec file is tailored to your quality objectives and application. You\u2019ll need to specify the indicator metrics to capture, and the scoring for the \u201cPitometer\u201d service to evaluate. See example PerfSpec file here.<\/a><\/p>\n\n\n\n

Once you have your own PerfSpec file, check it into your Azure DevOps or GitHub repository. This allows you to manage artifacts the same way and keep them in-sync with code changes.<\/p>\n\n\n\n

Step 3: Create a script to call Pitometer<\/h2>\n\n\n\n

A PowerShell script is an easy way to call and parse results from the Pitometer webservice. The example below expects the following parameters and will first read the \u201cPerfSpec\u201d file contents and construct the request to the Pitometer webservice.<\/p>\n\n\n\n

    \n
  1. Start and stop times in UTC milliseconds for gathering metrics<\/li>\n\n\n\n
  2. Pitometer webservice URL<\/li>\n\n\n\n
  3. PerfSpec file location within the pipeline working directory<\/li>\n<\/ol>\n\n\n\n

    After the Pitometer webservice is called, the script will parse the response and pull out the status attribute. The status attribute will have a value of \u201cpass,\u201d \u201cwarning,\u201d \u201cfail,\u201d or \u201cerror.\u201d   If there is a failure or an \u201cerror,\u201d the script will return \u201cexit 1\u201d thus stopping the pipeline.<\/p>\n\n\n

    \n# input variables \n$START_TIME=$env:startTime\n$END_TIME=$env:endTime\n$PITOMETER_WEBSERVICE_URL=$env:pitometerUrl\nSet-Variable -Name \"PERFSPEC_FILE\" -Value \"$(System.DefaultWorkingDirectory)\/_Demo-App-Build\/app\/perfspec\/perfspec.json\"\n\n# read perfspec file contents and compose the Pitometer service POST body content \nSet-Variable -Name \"PERFSPEC_DIR\" -Value \"$($PERFSPEC_FILE)\" \n$PERFSPEC_CONTENT = Get-Content -Path $PERFSPEC_DIR \n$PERFSPEC_REQUEST_BODY=\"{\"\"timeStart\"\": $START_TIME,\"\"timeEnd\"\": $END_TIME,\"\"perfSpec\"\": $($PERFSPEC_CONTENT)}\" \n \n# some debug output\nWrite-Host \"===============================================================\"\nWrite-Host \"startTime: \"$START_TIME\nWrite-Host \"endTime: \"$END_TIME\nWrite-Host \"url: \"$PITOMETER_WEBSERVICE_URL\nWrite-Host \"request body: \"$PERFSPEC_REQUEST_BODY\nWrite-Host \"===============================================================\"\n\n# calling Pitometer Service \n$PERFSPEC_RESULT_BODY = Invoke-RestMethod -Uri $PITOMETER_WEBSERVICE_URL -Method Post -Body $PERFSPEC_REQUEST_BODY -ContentType \"application\/json\"  \n\n# more debug output\nWrite-Host \"response body: \"$PERFSPEC_RESULT_BODY\nWrite-Host \"===============================================================\"\n \n# save the Pitometer service response and pull out the result status \n$PERFSPEC_JSON = $PERFSPEC_RESULT_BODY | ConvertTo-Json -Depth 5 \n$PERFSPEC_RESULT = $PERFSPEC_RESULT_BODY.result \n \n# evaluate the result and pass or fail the pipeline \nif (\"$PERFSPEC_RESULT\" -eq \"fail\" -Or \"$PERFSPEC_RESULT\" -eq \"error\") { \n    Write-Host \"Failed the quality gate\"  \n    exit 1 \n} else { \n    Write-Host \"Passed the quality gate\"  \n} \n<\/pre><\/div>\n\n\n
    Here is how to setup up the PowerShell script inputs.<\/p>\n\n\n\n