Tagging Azure resources with Git metadata using Terraform & Azure DevOps

Introduction

When dealing with enterprise-grade Azure deployments, tracking a specific resource down to the Terraform code it stems from can be difficult...

Also, finding the person responsible who can be asked questions regarding the configuration used is not always obvious.

This is especially true in setups where multiple teams maintain Azure deployments with Terraform.

So, when looking at an Azure resource within the Azure portal, I find it helpful to get a quick answer to the following questions:

• Who can I contact if you have questions regarding the resource configuration?
• Where can I find the Terraform code that describes the resource?
• At which specific commit hash do I need to look?
• When was the last time a change was made to that resource?

You might be aware of a tool called yor an extensible auto-tagger for IaC files written in Go. Unfortunately, this tool didn't work for me and produced many git blame warnings I couldn't solve.

Instead, we will leverage a Terraform provider called metio/git to tag Azure resources for the purpose described and let an Azure DevOps pipeline run terraform apply for us.

GitHub - metio/terraform-provider-git: Terraform provider for local Git operations

Terraform provider for local Git operations. Contribute to metio/terraform-provider-git development by creating an account on GitHub.

GitHubmetio

Let's get to it! 😎

Discussion

These are the tags we will add to our Azure resources

• git_repo, git_branch and git_commit_hash holding the shortened 8-digit version of the latest commit. So we know where to find the code defining the resources.
• git_commit_message and git_commit_timestamp so we get a hint about when the code got committed and what has changed
• Tags called git_author_name and git_author_email so we know who wrote the deployed code.

We can pull all of this information by using three data sources, which are git_commit, git_repository, and git_remote.

data "git_commit" "head" {
  directory = var.git_directory
  revision  = "@"
}

data "git_repository" "repo" {
  directory = var.git_directory
}

data "git_remote" "remote" {
  directory = var.git_directory
  name      = "origin"
}

It's worth noting that I am using the head shortcut @ , which refers to the latest commit in the branch and represents the current state of the working directory.

Further, I feed the repository folder by a variable called var.git_directory. This will be required later when we execute terraform apply by an Azure DevOps pipeline.

For ease of use, we can create a map like this and put it into a locals block. See the documentation for more details regarding the data source schema.

locals {
  git_tags = {
    git_commit_hash      = substr(data.git_commit.head.sha1, 0, 8)
    git_commit_message   = data.git_commit.head.message
    git_commit_timestamp = data.git_commit.head.author.timestamp
    git_author_name      = data.git_commit.head.author.name
    git_author_email     = data.git_commit.head.author.email
    git_repo             = data.git_remote.remote.urls[0]
    git_branch           = data.git_repository.repo.branch
  }
}

Later, we can add the gathered information to a resource like this:

resource "azurerm_resource_group" "foobar" {
  name     = "rg-tagging-demo-1"
  location = "switzerlandnorth"

  tags = local.git_tags
}

So far, so good. Let's move on to the Azure DevOps pipeline. I use the Azure Pipelines Terraform Tasks extension, which can be installed via Marketplace.

Azure Pipelines Terraform Tasks - Visual Studio Marketplace

Extension for Azure DevOps - Tasks to install and execute terraform with Azure Pipelines for Azure and AWS.

Visual Studio Marketplace

Here is the pipeline.

trigger: none

pool:
  vmImage: ubuntu-latest

steps:
  - checkout: self

  - task: TerraformInstaller@2
    displayName: "Terraform install"
    inputs:
      terraformVersion: "latest"

  - task: TerraformCLI@2
    displayName: "Terraform init"
    inputs:
      command: 'init'
      backendType: 'azurerm'
      backendServiceArm: 'service-connection-1'
      backendAzureRmResourceGroupName: 'rg-automation'
      backendAzureRmStorageAccountName: '<hidden>'
      backendAzureRmContainerName: 'tfstate'
      backendAzureRmKey: 'terraform.tfstate'
      allowTelemetryCollection: false

  - task: TerraformCLI@2
    displayName: "Terraform apply"
    inputs:
      command: 'apply'
      environmentServiceName: 'service-connection-1'
      commandOptions: '-var="git_directory=$(Build.SourcesDirectory)"'
      allowTelemetryCollection: false

In the first step, I check out the repository with checkout: self. It's worth noting that Azure DevOps does a checkout on a commit, not a branch, which results in a detached head. Next, I install the latest version of the terraform binary and initialize the backend.

The last step applies the terraform code. Please note that I passed the ADO variable Build.SourcesDirectory on to the terraform configuration. This way, the terraform git provider can pick up the proper directory from the build agent.

After a successful pipeline run, the resources carry the expected Azure tags, as depicted below. Cool, isn't it 😎

💡Please note that commit hashes can get lost when executing terraform apply from a feature branch that later gets merged back to main

Therefore, in the described situation, you should always apply your terraform code from the main branch (or another stable branch). Otherwise, you might be surprised not to find a matching commit in your repository🤯

Conclusion

• Azure DevOps does a checkout on a commit, not a branch, resulting in a detached HEAD
• Commit hashes can get lost when applying from feature branches
• We need to pass the git repository of the build agents to the terraform configuration by using a variable git_directory=$(Build.SourcesDirectory)

A proper Azure tagging strategy is crucial when navigating & managing your Azure deployment. But especially in large enterprise setups where resources are being deployed utilizing IaC, adding git metadata makes everyone's life easier.

I hope you enjoyed reading my article and would gladly receive feedback! What does your Azure tagging strategy look like?

Happy hacking! 😎

Further reading

Terraform Registry

Azure Pipelines Terraform Tasks - Visual Studio Marketplace

Extension for Azure DevOps - Tasks to install and execute terraform with Azure Pipelines for Azure and AWS.

Visual Studio Marketplace