What is ARM and what are ARM Templates?
Azure is managed using an API: Originally it was managed using the Azure Service Management API or ASM which control deployments of what is termed “Classic”. This was replaced by the Azure Resource Manager or ARM API. The resources that the ARM API manages are objects in Azure such as network cards, virtual machines, hosted databases.
The main benefits of the ARM API are that you can deploy several resources together in a single unit and that the deployments are idempotent, in that the user declares the type of resource, what name to use and which properties it should have; the ARM API will then either create a new object that matches those details or change an existing object which has the same name and type to have the same properties.
ARM Templates are a way to declare the objects you want, the types, names and properties in a JSON file which can be checked into source control and managed like any other code file. ARM Templates are what really gives us the ability to roll out Azure “Infrastructure as code”.
What can ARM templates do ?
An ARM template can either contain the contents of an entire resource group or it can contain one or more resources from a resource group. When a template is deployed, you have the option of either using ‘complete’ or ‘incremental’ mode.
The ‘complete’ mode deletes any objects that do not appear in the template and the resource group you are deploying to. In this scenario, what you get is the ability to know that whenever you deploy you will be in exactly the same state.
The ‘incremental’ deployment uses the template to add additional resources to an existing resource group. The benefit of this is that you don’t lose any infrastructure that is missing from the template but the downside is that you will have to clear up any old resources some other way.
The ideal deployment is ‘complete’ but it does mean that you need to have a good automated deployment pipeline with at least one test environment where you can validate that the template doesn’t rip the heart out of your beautiful production environment.
What don’t they do?
The ARM API deploys resources to Azure, but doesn’t deploy code onto those resources. For example you can use ARM to deploy a virtual machine with SQL Server already installed but you can’t use ARM to deploy a database from an SSDT DacPac.
To save time when designing solutions, it is important to understand that ARM API is used simply for resources and we need to use some other technology such as DSC or PowerShell to manage the deployments onto the infrastructure once it is deployed.
How are they tested
Functional testing
The actual ARM templates are JSON, which is essentially a block of text that is designed to be read by a machine rather than being simple to read for a human. Formatting does help but it is still a single block of text that doesn’t have an actual testing framework, so testing really comes down to performing a deployment and seeing what it creates. If used as part of a wider suite of tests, then the test process should be:
- Deploy to a test environment, possibly a dev/test subscription in Azure
- Deploy code and application tests
- Execute tests
- Report results
If all the tests return ‘success’ then the template is, by definition, valid.
Basic testing
Apart from the functional testing, it is possible to validate whether a template is actually deployable to the resource group you want to deploy. If you use the PowerShell function “Test-AzureRmResourceGroupDeployment”, it will take your template, parse it and check that it is syntactically correct as well as validating that you meet requirements such as not hitting quote limits before you deploy.
This is really useful, because the ARM template is not compiled. This prevents you from using a “build” step in the build process that you could otherwise use to make sure that the ARM template is even deployable. Furthermore, it wouldn’t be ideal in the “complete” type of deployments if you deleted various resources and then failed to create all the new resources that you wanted because you had hit your quota limits in Azure.
ARM template execution
There are two important concepts to understand when using ARM templates. The first is that the ARM REST API is the part that actually does the “heavy lifting”. It is this that provides the idempotency of the whole process. The REST API is well documented, updated regularly and available on either the Microsoft docs site or on GitHub if you felt like contributing:
Azure REST API Reference – https://docs.microsoft.com/en-us/rest/api/
Azure/azure-rest-api-specs – https://github.com/Azure/azure-rest-api-specs
There is one set of REST APIs called “Resource Management” which is where you send an ARM template. The REST API takes the template and:
- Parses the JSON
- Fills in any parameters that are passed in
- Executes any ARM template functions
- Calls the REST API of whatever type of resource that needs to be created to create it
If we look at this simple example of an ARM template which will deploy a single storage account:
The process that the REST API goes through is to read the parameter “storageAccountType” from the parameters that are also passed in at the same time as the deployment. Because the “storageAccountType” parameter also specifies what the “allowedValues” are the parameter is validated against the list of allowedValues. If the parameter passed in has a typo or other mistake then the deployment is cancelled at this point. If you omit the “allowedValues” then this check will not happen.
The REST API then creates the variable diagStorageAccountName and the value of the variable is the result of the ARM template functions concat which concatenates the string ‘diags’ with the output of uniqueString(resourcegroup().id) which will create a unique string for that resource group. The function uniqueString is passed the id of the resource group to use as a base string for uniqueString so it will be unique per resource group. If you need a second string to be unique you would need to pass another string to hash or omit the base string. The function uniqueString is best used with a base string so subsequent deployments do not create new objects each time.
The REST API then uses the resources section to call the resource specific APIs. There is only one resource to create in this example and it is of type “Microsoft.Storage/storageAccounts”. The type in the ARM template maps directly to another of the REST APIs types which are documented:
The apiVersion tells the resource manager which version of the API to call, the properties you can set and also the behaviour of the API can and does change between versions so it is important you get the correct API version. The API version is specified for each resource so you can deploy two things of the same type (storage account for example) but deploy them using different versions of the REST API.
The location is determined from the function resourceGroup() which returns a list of properties, one of which is location.
The value that is set for “sku” is resolved from the parameter “storageAccountName”. Any value in the JSON template that is surrounded by [ and ] is evaluated as code. The full list of template functions that can be used is documented https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-template-functions
The resource manager then creates the JSON which is going to be sent to the storageAccount REST API which will look like:
This JSON is then sent to the following URI as a PUT request:
/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/Microsoft.Storage/storageAccounts/diags{someUniqueStringCreatedAtRuntime}?api-version=2016-01-01
The storageAccounts REST API is then responsible for checking whether there is a storage account with the name we have. If there is not then it creates a new one and makes sure it has the same properties as the ones we pass in. If it already exists then the storage REST API will just ensure the properties are set correctly.
When the storageAccount PUT request happens, if there is something it cannot do such as create an account with the same name as an existing one in another resource group or subscription (maybe even created a different person or organisation) then it will fail. There are also some changes that it finds impossible such as the compute API changing the base image a virtual machine was created for. You will sometimes need to delete resources and start again for some changes.
When the PUT request finishes, the API returns a JSON document which contains the definition of the object that it just created so some things that can’t be known at build time are available. This is useful for cases where you create something like a storage account which creates the access keys when it is created. You could, in your ARM template, reference the key and use that in other objects that need a storage account name and key.
The properties are returned whether or not the object already existed, so in that way you know that whether it is the first time that the resource was created, or the hundredth time it was checked and already existed, you can still deploy downstream dependent resources.
What else can we do in ARM templates?
Defining dependencies
In some cases, you need to deploy resources in a specific order: For instance, in order to create a virtual machine, you will need to deploy a NIC and for a NIC you might want to deploy a public ip address. If you try to create the NIC and reference a public ip address that doesn’t exist, then the deployment will fail. The answer to this is that dependencies, as well as each resource, can have a “dependsOn” section that lists all the dependencies that need to be created before the resource can be created. Dependencies are really useful, but should be used with caution because they limit the number of simultaneous things the resource manager can do. The more dependencies you have and the longer the dependency chains, then the longer that deployments will take.
It is also possible to define dependencies by nesting resources, and in that case child dependencies are created after the parent resource is created.
Finally there is a third way to create a dependency which is to use the “reference” template function. What this does is to get the properties of another resource, the same as in the output to the PUT request above. This enables you to do things such as to use an account key from a storage account which may not be known at build time. If you use a reference to access another resource in the template then an implicit dependency is created for you.
The dependencies are documented by Microsoft:
https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-define-dependencies
Copy Sections
When we deploy resources, we may want just one, but more likely we will want a number of similar resources such as virtual machines to be created. To help with this, there is the “copy” section which specified how many copies of a resource we want. This is great, because it means that, if we want 100 virtual machines, we don’t have to define 100 resources in the ARM template. The way it works is that you add a copy block to your resource such as:
What this says is that we should have 3 storage accounts created. There are two things to note here: the first is that, because we are saying ‘make 3 copies of this’, we need to make the name unique amongst the three objects. We do this so that, instead of just using the account name variable, we append the copyIndex which is the id of the copy operation. We do this because, if we pass 1 into the copyIndex function, it starts at 1 rather than 0 so we get accountName1, accountName2 and accountName3.
The second thing is that the copy block has a name. This is really useful because we can use it in a dependsOn block and the dependent resources will not be created until all three of the storage accounts have been created. This is useful in such cases as creating virtual machines where you could get the first machine taking longer to deploy than the last. If you didn’t have the ability to reference by copy name you would need to add a dependsOn to all the resources individually which would get messy quickly.
Conditionals
Conditionals are a recent addition to ARM templates. They specify whether or not to deploy a specific resource. This can be useful when handling different environments where, in development, you might only want one virtual machine without a load balancer but in production you might want five machines with a load balancer. To use the same template you could pass the number in as a parameter but to choose whether or not to deploy the load balancer you can return false in the condition property.
In this example we have a “condition” property that checks whether a parameter is set to “yes” or not.
Conditionals should definitely be used with caution and I would suggest having at least one environment before production where the entire template is deployed. To be fair, they fulfil a need that was missing and which previously involved passing whole blobs of JSON in as parameters.
How do you get started?
There are a couple of approaches, the easiest of which is to use the quick start templates created by Microsoft which give examples of how to use most resources. The quick start templates are available here:
https://azure.microsoft.com/en-gb/resources/templates/
The second approach is to deploy some resources: In the azure portal, there is a button called “Automation Script” which will generate the ARM template to deploy the entire resource group. This always creates a script for the entire resource group even if you click the “Automation Script” on one particular resource.
Not every type of resource can be generated today, although it seems as if Microsoft keep increasing the scope of the resources that they generate ARM templates for. It might be that you just need to wait a few weeks.
The automation script doesn’t, in my opinion, generate the ideal templates. The names are odd and even though they put names behind parameters they use the name of the resource in the parameter names so it is hard to read and edit the scripts. I would say that they are best used to show how a resource is configured and just take the parts from the script that you need to re-use and manually create the other parts of the template or use a company-shared template to create the ARM template.
The automation also generates the scripts that can be used by different clients such as PowerShell, Bash and C#. This gives you the code you need to send your ARM template to the ARM REST API for processing. Again, these are quite verbose and you don’t need everything. For example, if you are using PowerShell, then all you really need to do is to create the resource group itself if it doesn’t exist, and then call “New-AzureRmResourceGroupDeployment” from the AzureRM module.
Documentation
The ARM templates capabilities themselves as opposed to the actual resources are documented:
https://docs.microsoft.com/en-us/azure/azure-resource-manager/
The resources and their properties are documented:
https://docs.microsoft.com/en-us/azure/templates/
This only documents the latest version of an API. For the exact details of what can and cannot be used in an ARM template for a resource the JSON schemas are available:
https://github.com/Azure/azure-resource-manager-schemas
Although these are not as easy to follow as the documentation page, they do give the exact details of what can and cannot be deployed.
Sometimes the REST APIs are updated before the documentation so, if you are using something that is bleeding edge, then you can be lucky if you look at the documentation for the REST API, see what properties you can set and then just put them in the ARM template.
Comments
Post a Comment