About a month ago, Microsoft announced the release of the Azure DevOps Provider for Terraform. Using this provider, we’re able to create Azure DevOps (ADO) projects, repositories, pipelines, variable groups, etc.

Recently, I delivered the 3rd part in my 3-part presentation series on “Infrastructure-as-Code (IaC) Using Terraform”. This “advanced edition” covered deploying infrastructure (ie. a hub-and-spoke VNet in Azure, inclusive of VNet Peering) using Terraform, but specifically in a DevOps methodology. In my presentation, I compared deploying this Terraform code using Azure DevOps, GitHub Actions, and Terraform Cloud.

Note: The recordings of this 3-part series is available on my YouTube channel here: AdinErmie

As a follow-up to that 3-part series, people have been asking for a “Part 4” on more real-world scenarios including governance and deploying applications/services into an existing environment. And so, I started to think about what would make a good real-world example. I came to the conclusion that, well, if I’m going to use a full DevOps process for this real-world example, I might as well start with deploying/managing my Azure DevOps project via Terraform itself!

So this is where we will begin.

Note: The Azure DevOps Terraform provider (at the time of this writing) is in version 0.0.1; so very basic/starting functionality.

UPDATE: This article now has a video recoding version available on my YouTube channel as well!

Documentation

I first want to share a few observations (both good and bad) about the currently available documentation. Firstly, as generally seems to be the case, the existing documentation and code samples leave a lot to be desired. Usually, they only include a simple example and do not provide enough detail on all the properties available, values that can be used, etc. More on this in a minute.

One thing that I did appreciate about the documentation though, is the inclusion of the permissions required. Take a look, for example, at the azuredevops_project resource documentation. Aside from the standard usage example, arguments list, etc. at the bottom of the page it also includes the PAT Permissions Required. This lists the permission(s) required from your Azure DevOps Organization’s Personal Access Token (PAT) to be able to create this specific resource. This is really helpful for organizations that want to follow a least-privilege access approach.

Code Breakdown

I’m not going to simply provide a screenshot of all the code, file-by-file, as the official documentation does provide usage examples. And besides, I will have my code examples available in a GitHub repo for reference. But, I am going to talk about how I organized my code (so that I could keep everything straight).

If you take a look at how I’ve organized my code, you’ll notice that I have the standard “provider”, “variable”, “tfvars”, and “output” files. But notice how I have a separate file for each of the Azure DevOps resource types, including project, repo, pipeline, variables (specific to ADO, not Terraform), and service endpoints.

This allows me to only focus/code that specific resource type in that .TF file instead of getting lost in one large/giant Terraform template. This will still work because, all .TF files within the same directory are compiled together; as-in, Terraform merges them all together as if they were one big Terraform file. I find this method easier to work/organize my code.

Azure DevOps Project

The first thing we need to start with is an Azure DevOps project. Now, this assumes that you already have an Azure DevOps Organization already created and available. Notice in the following screenshot that you have 2 different ways of declaring the features you want enabled for your Azure DevOps project. It’s completely personal preference, as both ways will work.

This is an example where the documentation is not as complete as it should/could be. If you look at the azuredevops_project documentation again, notice that even though it mentions the work_item_template attribute, it only tells you that ‘agile’ is the default; it does not list all potential options available. This may be minor in this specific example (as most will probably use agile), but I personally still expect the documentation to be complete and show all options available to fully educate the end-user.

Azure DevOps Repository (Repo)

The next example I am going to specifically share is the Azure DevOps Git repository. If you look at the azuredevops_git_repository documentation, it provides a usage example for a new repo, as well as forking an existing repo. But, it does not provide an example for importing a repo. I’m not sure why that is not provided since the documentation clearly shows ‘import’ as an option.

Through my testing, I was able to very easily/simply create a brand new repository. However, even though the documentation lists ‘import’ as a valid value for “init_type”, it didn’t work for me.

While trying to figure out why the import wasn’t working, I came across this issue on the Azure DevOps Provider’s GitHub repo: Add support for imported git repos to Git Repository resource. Apparently, “currently azuredevops_git_repository does not support importing a private git repo (one that requires authentication). ”

This is another example of the documentation not being completely accurate (or helpful). Yes “import” in a valid option (I have yet to try it against a public repo), but there are some apparent limitations/functionality gaps. It would be good to mention those up-front, instead of allowing people to try to use the provider in a way that seems to be acceptable, but not yet supported.

Azure DevOps Variable Groups

In my “Infrastructure-as-Code (IaC) Using Terraform — Advanced Edition” presentation, I walk through the code breakdown for deploying my Terraform template(s) using Azure DevOps and ADO’s pipelines. As part of that demonstration and example, I also include the use of Variable Groups, and in particular, integrate it with Azure Key Vault where sensitive values (like credentials, access keys, etc.) are stored.

When attempting to replicate the same setup and configuration using Terraform, I ran into some additional limitations/issues.

For example, in the azuredevops_variable_group documentation, it doesn’t even provide a usage example for integrating with Azure Key Vault. So initially I thought it wasn’t even an option/supported. When I looked at the open Issues in the Provider GitHub repo, I found the following issue: [Documentation Update] Variable Groups Sourced from Azure Key Vault. In it, it states “This is actually the documentation issue, variable groups sourced from Key Vault are supported” and provides a code example.

However, I find the sample code to be limited. In this case, it implies that Terraform is also creating the Key Vault within the existing module, and does not have an example of referencing a pre-existing Key Vault. You could simply accommodate that by using a data resource to reference an existing object outside of Terraform’s management.

When I attempted to deploy the Variable Group (with its integration with Azure Key Vault), aside from some errors I encountered (probably from multiple code runs), when the Variable Group did deploy, there was one key part missing.

Notice in my screenshot below how I have the Variable Group associated to my Azure Service Connection (which is an azuredevops_serviceendpoint_azurerm resource type), and also semi-enabled for my Azure Key Vault. However, notice the error message that states:

The specified Azure service connection needs to have “Get, List” secret management permissions on the selected key vault. Click “Authorize” to enable Azure Pipelines to set these permissions or manage secret permissions in the Azure portal.

This is something that I have yet to find a resolution for. Of course, we could easily click the ‘Authorize’ button, but we want to be able to deploy the Azure DevOps project, along with all it’s elements, programmatically via Terraform.

If you look at the Azure DevOps Variable Groups documentation, in particular the Link secrets from an Azure Key Vault sub-section, it says basically the same thing:

Ensure the Azure service connection has at least Get and List management permissions on the vault for secrets.

And so, in my Azure Key Vault Terraform code (since I create a new Key Vault as part of this deployment), I attempt to grant the Service Endpoint ID the required permissions; but unfortunately, this also did not work for me.

Azure DevOps Pipelines

The final example I want to share with everyone is about the Azure DevOps pipelines, also referred to as “build definitions” (from a Terraform code perspective).

Again, in reference to my “Infrastructure-as-Code (IaC) Using Terraform — Advanced Edition” presentation, I demonstrate the YAML files that define the pipelines for deploying my Hub-and-Spoke Virtual Network, and complete the peering between them.

In attempting to reproduce that using the Azure DevOps Terraform provider, I have defined an azuredevops_build_definition resource (depicted below). This is one of the more complicated resources, as there are many attributes available to be set (depending on what you’re trying to accomplish).

The key parts of this code are the CI Trigger (which we have set to use YAML, since we have separate YAML files as part of our Pipeline-as-Code (PaC) process), the linkage to the Variable Group we have created (which should itself be linked to an Azure Key Vault), and a repository.

In this specific setup, I am using a new repo that I create as part of the Azure DevOps project deployment, on account that I cannot import an existing one. But this then leads to a chicken-and-egg problem.

Because as part of this code we are creating a new repo, the YAML files will not be present, and so even though the Pipeline will be created, you will encounter the following error:

An error occurred while fetching the YAML file ‘PATH_TO_YAML_FILE’ in the repository’s default branch and latest build branches: master. Error on last attempt: File PATH_TO_YAML_FILE not found in repository REPO_NAME at version/branch master.

I will assume this can be resolved once we can import an existing GitHub repo. But for now, it means that if you are creating a repository as part of your deployment code, you then have to somehow upload/import your YAML files into that new repo, before the pipeline code is executed.

Conclusion

All in all, it is interesting to see what we can accomplish through the Azure DevOps Terraform provider, especially since it is only in version 0.0.1! It is obviously not quite ready for full production use, but no doubt the gaps and limitations will be filled/corrected.

If you are going to try it out for yourself, I would suggest bookmarking the GitHub repo, and checking the Issues regularly, not only for issues/limitations you might encounter, but also to track the progress/updates to the provider as they are released.