Configuring Azure Traffic Manager, Application Gateway and App Services with Terraform

Azure App Service is a great choice for a Platform As A Service (PaaS) option to host Web and Api applications.

The service is fully managed, scales vertically, horizontally and runs platforms such as:

• .NET
• .NET Core
• Java
• Ruby
• Node.JS
• PHP
• Python
• Docker

When the service is created it provides a publicly accessible Virtual IP Address and a *.azurewebites.net URL.

For a production deployment you are going to want to protect your App Services from potential attacks such as Denial of Service(DDOS), SQL Injection and Cross Side Scripting(XSS). To do this you can use Azure Application Gateway with a Web Application Firewall SKU.

Typically you are also going to want to support a
multi-region deployment for high availability, redundancy or fail over reasons. To do this you can use Azure Traffic Manager to route traffic as needed.

Terraform

I like using Terraform to create my cloud infrastructure. Most of the GA Azure services are supported but there are always limitations which will need to be supplemented using some post deployment Powershell scripts.

Installation instructions can be found here

Creating a Terraform Azure Principal

Follow the instructions on the Terraform Website to create the principal account. This account will have Contributor access to your Azure Subscription so that it can be used to create / modify / remove resources.

The instructions can be found here.

Terraform Script

mkdir /c/code/contoso.com/infrastructure  
cd /c/code/contoso.com/infrastructure  

Create an environment.tf file it should look like the sample below with the Azure Principal configuration and the creation of 3 resource groups.

Create a config.dev01.tfvars file it should look like the sample below.

NOTE: Please set the environment variable to something unique to avoid resource naming conflicts.

subscription_id - navigate to Subscriptions in Azure Portal, copy/paste Subscription ID.

clientid - navigate to your Azure Active Directory, App Registrations and search for your App. Application ID = clientid

tenantid - navigate to the Azure Active Directory blade and select Properties. Directory ID = tenantid.

secret_key - this would have been created while following the Terraform Principal creation guidance mentioned above. Keep it in a safe place.

Initialize a terraform environment. For now we will keep the state file local. Normally I'd centralize the state file using a blob storage container or something similar.

terraform init  
terraform workspace new dev01  
terraform plan -var-file=config.dev01.tfvars  

If everything has been configured correctly you should have the following plan output.

terraform apply -var-file=config.dev01.tfvars -auto-approve  

If everything is working as planned your 3 Resource Groups will now be visible for your Subscription in the Azure Portal.

Now that we have our infrastructure as code automation working, our next step is to create the App Services.

Web App Services

Add this terraform code to your environment.tf file to create a web app service in the East and West resource groups. We will create an app service plan for each too using the Free tier.

terraform workspace select dev01  
terraform plan -var-file=config.dev01.tfvars  
terraform apply -var-file=config.dev01.tfvars -auto-approve  

Navigate to you App Service endpoints to verify they are "up and running".

• https://as-eastus-contoso-dev01.azurewebsites.net/
• https://as-westus-contoso-dev01.azurewebsites.net/

Application Gateway

Next we will add the following terraform code to create the Azure Application Gateway. We will be adding the Web Application Firewall (OWASP 3.0) and we will be enabling HTTP2 which it now supports.

Application Gateway requires several other services namely:

• Virtual Network (VNET)
• Subnet
• Dynamic Public IP

Use the following code to create these resources:

Notice that the IP and DNS have NOT been assign to the Dynamic Public IP Address. This is because there is nothing attached to it. Once we create the gateway and attache the Public IP it will generate an address and DNS for us.

Use the following terraform code to create the Application Gateways:

Unfortunately, at the time of writing, App Services and Application Gateway are not fully supported. There is an additional boolean flag that needs to be set. This can only be done via powershell or resource explorer (api), it is not possible to set this flag through terraform or the azure portal. If you know an alternative please correct me. I've spent hours trying to figure this out.

For my example I used Azure Resource Explorer which can be used to edit specific values on your resources. Navigate to your subscription --> resource group --> providers --> Microsoft.Network --> your gateway

Look for the pickHostNameFromBackendAddress click Edit and set the value to true, click PUT to update. Browse back to application gateway on the portal and wait for the update to complete.

Select Backend Health on the Application Gateway blade, you should now see a healthy status.

Navigate back to the Dynamic IP address that we were looking at earlier, the one that is attached to the Frontend off your Application Gateway. You will now notice it has an IP address and a generated DNS name. This is because it is not attached to the gateway.

Browse to the DNS name provided and your web site should be displayed.

Go ahead and test both the west and east gateway DNS to ensure both are working.

The next step would be to setup Traffic Manager to route traffic to the gateways.

Traffic Manager

Use the following Terraform code to create the Traffic Manager Profile and an Endpoint for each Gateway.

Traffic Manager Endpoints supports different types. I tried using azureEndpoints and then specifying the public IP resource but Terraform timed out with a 500 for some reason.

I ended up using externalEndpoints and then specifying the FQDN of the public IP resources which worked just fine.

Traffic Manager will monitor the health of your endpoints:

Browsing to the DNS name for the Traffic Manager should now display your site relative to your location:

Enabling HTTP2

Enabling HTTP2 on the Application Gateway is currently not support by Terraform.

You can use the following PowerShell script to enable HTTP2:

Configure using the same Principal credentials as Terraform and execute for both East and West gateway.

This will only work once you have configured HTTPS since that is a requirement for HTTP2.

Limitations

I thought I'd talk about some of the limitations that I found during this build out as well as some points of interest.

pickHostNameFromBackendAddress Flag

This is an obvious limitation, I'm hoping that soon there will be a selection in the Azure Portal for an App Service when adding to your backend pool instead of using the FQDN. Selecting this option should then set the pickHostNameFromBackendAddress flag for you automatically.

Ideally Terraform could also provide this setting to make automation easier and more consistent.

The only recommendation Microsoft documentation currently has for doing this is to use powershell.

You can read about it here.

Http vs Https

In my example I have configured only for HTTP. In reality you are going to want to use HTTPS. This is easy enough to configure on the DNS and App Service side using a CName and adding a Custom Domain. Best practice then is to terminate SSL at the Application Gateway level by attaching your SSL Certificate to a HTTPS listener on port 443. You can then route traffic to http (port 80) on your App Service and change your Application Gateway HTTP listener to route to the HTTPS listener.

One limitation is that Terraform currently does not support the option to route from one listener to another. So in terms of automation you may need to write some powershell for that.

IP Restrictions

One of the reasons for using the Application Gateway is to prevent users from directly accessing your App Services. You can use IP Restriction on the Networking blade for this, providing the IP address of your Application Gateway.

A few things to note here is that if you have any CI/CD build servers deploying code you may need to allow that IP address access to your App Service too.

Currently neither Terraform nor Powershell support the addition of IP Restrictions for App Services. There is custom powershell to do this which I submitted as a PR to MSDN documentation. The response I received was that a Powershell module was coming out soon for this.

You can use the following PowerShell script to configure IP restrictions:

WAF Detection vs Prevention

In this example the WAF is set to Detection which will only log attacks (if configured) but not prevent access to the gateway. Ideally you want to set the WAF to Prevention mode. Doing this however prevents all traffic for me with my current setup and I still need to figure out why. There is most likely one or 2 of the OWASP 3.0 rules that are being broken. My plan is to configure logging which should identify the rules being broken. These rules could either be disabled (if that makes sense) or the root cause could be addressed.

Traffic Manager

When I was initially testing this setup I tried using Traffic Manager directly to my App Services (bypassing the Application Gateway). An interesting learning is that Traffic Manager will only support the Standard and above tiers. Using the Free tier will allow you to add the App Services as endpoints but traffic will not route correctly.

Another friendly tip is that during testing dns caching is a factor. Clear your cache and flush your DNS, use Chrome Incognito mode and test with multiple browsers. Often I had actually configured correctly but cache from previous requests were letting me down.

Conclusion

Traffic Manager, Application Gateway (WAF) with App Services can be used to provide a secure and efficient multi-region deployment option for your PaaS applications.

Using Terraform to help deploy your infrastructure is powerful but will only get you 80% of the way there. Be prepared to write post deployment Powershell scripts to supplement the limitation of Terraform.

Configure for HTTPS and take advantage of HTTP2 which can be enabled on the Application Gateway. Consider terminating HTTPS at the gateway level that way you wont need to upload a certificate for each App Service.

Set the WAF to Prevention instead of Detection to protect you from DDOS, XSS and SQL Injection attacks.

Restrict direct access to your App Services so that users cannot bypass the Application Gateway.