Keep your Azure Subscription for Development tidy

A good practice to keep things together in your Azure Subscription is using resource groups for specific purposes. Maybe a development team is working on a new feature and wants to setup some services to work on it. Or for a development spike you are playing around a bit with various services and trying things out.

The problem that can occur is that resources and resource groups are not cleaned up when not needed anymore. For some services that is not a problem — you only pay for the use. But other services have different pricing and will cost money even if you are not using it anymore. So you want to make sure your subscription is maintained to prevent unnecessary spending.

For a large development team (100+ people) we worked on, we implemented a simple mechanism to clean the development subscription, using Tags, Azure Policies, scripts and a pipeline.

The tags

For our solution we came up with these 4 tags:

• CreatedOnDate — creation date of the resource. This is automatically set by the policy as you will see (some resources like resource groups don’t have this property themselves).
• OwnerEmail — the e-mail address of the owner for notifications. This is set manually in our case.
• IsPermanent — indication whether this resource group must be kept at all times. This should only be used for carefully selected resource groups, e.g. the main deployment from your CI/CD pipelines. Set the value to ‘true’ to mark as permanent.
• MaxExpirationDays — number of days this resource group needs to be kept active after the CreatedOnDate date. The default is set to 14 days, but this can be configured of course. And it can be modified manually to extend the expiration.

Setting up Azure Policies

Now we know which tags we want to use, we can setup the automatic creation of them for new Azure resources using Azure Policies. To do this using Azure Cli you can use this command to define the policy:

az policy definition create --name "Tag-CreatedOnDate" --display-name "Tag-CreatedOnDate" --mode 'All' --metadata "category=Tags" --rules "{ \"if\": {\"allOf\": [ { \"field\": \"tags['CreatedOnDate']\", \"exists\": \"false\" } ] }, \"then\": { \"effect\": \"append\", \"details\": [ { \"field\": \"tags['CreatedOnDate']\", \"value\": \"[utcNow()]\" } ] } }" --param "{}"

To demystify this a bit — the rules part is a kind of if-statement to add the CreatedOnDate tag if it doesn’t exist yet and set the value to [utcNow()] being the timestamp at time of creation.

Next step is to assign this policy to a subscription:

az policy assignment create --name "Tag-CreatedOnDate" --display-name "Tag-CreatedOnDate" --scope "/subscriptions/[subscription ID]" --policy "/subscriptions/[subscription ID]/providers/Microsoft.Authorization/policyDefinitions/Tag-CreatedOnDate"

To list all custom policies you can use this Azure Cli command:

az policy definition list --query "[?policyType=='Custom'&&contains()]"

This activates the definition for the provided subscription immediately. All resources created after these two commands will get the tag automatically. Of course we want this for all the tags we defined. This is the complete script that takes the subscription ID as parameter OR takes the default subscription ID of the environment.

The weekly process

In our project we created a pipeline running a script once a week to do a few things:

• Delete all expired resource groups
• Check all resource groups to expire in a week and notify owners
• Validate the settings of the resource groups

The validation step is to check the MaxExpirationDays tags of all resource groups against a configured maximum. If the value is above that maximum the configured administrators are notified. In our configuration we have set the maximum to 60 days.

For the first two steps the configured administators are notified as well what happened or will happen in a week.

The logic for deleting a resource group is simple. If the CreatedOnDate + MaxEpirationDays is before Today and IsPermanent is not equal to ‘true’, the resource group is deleted. For the check for next week, replace the Today value by Today + 7 days.

The implementation of the script to check and delete resource groups

Using Azure Cli again you can use this command to get a list of all resource groups that match our conditions:

az group list --query "[?tags.IsPermanent!='true'].{Name:name,CreatedOn:tags.CreatedOnDate,MaxDays:tags.MaxExpirationDays,Owner:tags.OwnerEmail,IsPermanent:tags.IsPermanent}" -o tsv

To handle this list in the bash script line by line we use this mechanism:

az group list --query "[?tags.IsPermanent!='true'].{Name:name,CreatedOn:tags.CreatedOnDate,MaxDays:tags.MaxExpirationDays,Owner:tags.OwnerEmail,IsPermanent:tags.IsPermanent}" -o tsv |
while read -r Name CreatedOn MaxDays Owner IsPermanent; do
    # handling per row here
    # Use e.g. $Name for the resource group name
done

NOTE: One problem that occurs with using pipes, is that it spawns a sub-shell. This makes it impossible to fill variables in the loop and use it afterwards. For this purpose we have a call to `shopt -s lastpipe` in the beginning of the bash script to enable using variables in the loop and outside the loop.

In the processing of the resource groups, we create a new tsv table using printf. It is essential that the fields are separated with a tab (\t) and the line is closed with linefeed and carriage return (\n\r). We use the variable expired_list to add resource groups that need to be deleted or marked as 'to be expired'. We add each to the variable like this:

if [[ "$expirationDate" < "$checkdate" ]] || [[ "$expirationDate" == "$checkdate" ]]
then
    # resource group expired
    expired_list+=$(printf '%s\t%s\t%s\t%s\t%s\n\r' "$Name" "$CreatedOn" "$MaxDays" "$Owner" "$print_checkdate")
fi

When we want to navigate through this list, we use the read command again but this time we pipe the expired_list variable:

while read -r Name CreatedOn MaxDays Owner Expires; do
  # handling per row here
  # Use e.g. $Name for the resource group name
done <<<$expired_list

To retrieve all resource groups that have a MaxExpirationDays which is equal or higher than the configured threshold (default 60):

az group list --query "[?tags.MaxExpirationDays>='$thresholdExpirationDays'].{Name:name,CreatedOn:tags.CreatedOnDate,MaxDays:tags.MaxExpirationDays,Owner:tags.OwnerEmail}" -o tsv

The complete logic of the script is the as follows:

To run the script to delete resource group with the described logic, you can enter something like this:

./CleanupSubscription.sh -d -a "admin1@domain.com,admin2@domain.com" -m "[subscription key]"

The -m “[subscription key]” flag is to have a subscription key to connect to a logic app we created to send the e-mail. We use curl to post JSON to that endpoint.

To run the script do check for resource groups to be deleted in a week, you can enter something like this:

./CleanupSubscription.sh -c "$(date -d "+7 days")" -a "admin1@domain.com,admin2@domain.com" -m "[subscription key]"

The pipeline

Now we have these scripts, we can setup an Azure DevOps Pipeline which runs on a schedule. We chose every Sunday evening at 10pm GMT. The secrets are coming from a key vault. The settings are coming from a template file:

This is the pipeline:

Conclusion

Having this in place is saving us money and making people aware of what’s happening. Hope this will benefit you as well.