One of the most critical parts in software architecture is designing a release process for upgrading the software in your applications. The
release plan is crucial and affects several factors such as:
- Website’s availability or downtime
- Time required to release new features
- Ability to rollback your upgrades and return to a previous stable version (in case you find out that the new version fails)
The most common scenarios for traditional web applications are the following:
Entirely shutdown the website, display a friendly message to users that the site is under maintenance and finally upgrade code and SQL database. It’s easy, it works but it also has a huge disadvantage: During the release you prevent users from accessing your application which is unacceptable.
A widely used release plan with zero downtime requires to deploy your web app at two servers at least and put a load balancer on the front. When you decide to release new software, you disable one server at a time using the load balancer and you upgrade the code. SQL updates mustn’t have any breaking changes since both of the servers accessing the same database. While one of the servers are down the load balancer directs the requests to the server that is up and users don’t experience any downtime.
Things get really complicated when your application is deployed in more than one locations and also uses SQL Geo-Replication meaning that there is one primary
READ-WRITE SQL database that continuously sends committed transactions asynchronously to its secondary databases so that they are up to date. Secondary databases are
READ-ONLY and available for querying and failover as well. The following picture describes this type of architecture.
What makes this scenario difficult to release new software is the SQL Geo-Replication feature. In case you wish to release breaking SQL schema changes you have to apply them in the primary READ-WRITE database. When you do that, changes will be transferred to secondary databases as well which means that all web apps should also be already updated otherwise will crash (due to the breaking changes). The solution to the problem is to provision new web apps and databases and apply the changes on them instead to those that are currently in production. When you finish with the upgrades you disable the previous endpoints from your load balancer, add and enable new endpoints that point to your new web apps which run the latest version of your software. Still, it’s not that simple as it sounds and several steps needs to be taken to accomplish such type of a release so let’s break it down.
Step 1 – Provision new Web Apps and SQL Databases
In this step you provision new web apps and SQL server/databases. You provision the same number of web apps and databases that are currently in production. Let’s assume that you have deployed your application in West Europe, West Central US and South East Asia and that the primary
READ-WRITE database is in West Europe.
Step 2 – Setup chaining SQL Geo-Replication
At this step you start replicating your SQL data at your new databases by setting them as secondaries of your primary
READ-WRITE database. In Microsoft Azure you can set up to 4 secondary databases using the SQL Active Geo-Replication feature. If you wish to add more secondaries you use a process known as chaining where you create secondary of a secondary.
Step 3 – Switch primary READ-WRITE database
At this step, you stop replicating data and also change the primary
READ-WRITE database to one of the newly created. The purpose of this step is to apply the SQL schema changes to the new databases while they contain the latest snapshot of your data. Following are the sub-tasks needed in order to complete this step:
READ-ONLYthe first primary database. This will ensure that the new secondaries will contain the latest snapshot of your data. During this point your application should be able to operate without a
READ-WRITEdatabase (e.g. using queues)
- Disconnect the main new secondary database
- Make the disconnected secondary database
Step 4 – Deploy code and SQL shcema changes
You can safely deploy your code and SQL schema changes to the new web apps and databases. SQL schema changes are applied only in the new
READ-WRITE database and replicated to the secondaries.
Notice that the web app is still served by the old version of your software and you haven’t removed SQL Geo-Replication from the old databases even they are all in
Step 5 – Switch active web apps
It is high time you serve the users the new version of your software. At this step, you add and enable endpoints to your load balancer for your new web apps and disable the old ones.
In case you decide to switch back to the old version, revert again the endpoints in your load balancer. Mind though that before doing so, you have to switch again the primary
READ-WRITE database. After a successful release cycle you can safely delete the resources you don’t actually use (web apps and sql server/databases).
Microsoft Azure App Services
In case you use App Services for your web apps you have the option to create staging slots instead of provision new web apps, something that will certainly reduce the costs for your infrastructure. At this scenario you don’t add new endpoints to your Azure Traffic Manager but instead you swap the production and staging slots and the switch is done internally for you by the traffic manager.
About this post
This post is part of the new e-book Globally-Distributed Applications with Microsoft Azure that describes in detail how to build and deploy modern, highly available and planet-scale web applications with Microsoft Azure. The source code associated with the book contains a cross-platform Web App built with .NET Core and Angular and uses the following Azure Services:
- Azure CDN
- Azure Traffic Manager
- Azure Storage
- Azure Cosmos DB
- Azure Search
- Azure Redis Cache
- Service Bus
- App Services & WebJobs
- Azure SQL Databases
- SQL Active Geo-Replication
- Azure Active Directory B2C
Users authenticate using either ASP.NET Core Identity or Azure Active Directory B2C
The first 4 parts of the book describe the features for each
Azure Service used and their role in the design of the web application. It provides step by step instructions to configure application settings and secret keys so that you can deploy it all over the globe. You will also find plenty of code for accessing Azure Services programmatically. The final chapter explains the
PowerShell scripts that you can use to automate processes in a globally distributed application (resource provisioning, releases or rolling back updates). This means that all of the steps described in this post will be automated by scripts such as the following:
.\init-geo-replication.ps1 ` -Database "<primary name>" ` -PrimaryResourceGroupName "<primary-resource-group>" ` -PrimaryServerName "<primary-server>" ` -SecondaryResourceGroupName "<secondary-resource-group>" ` -SecondaryServerName "<sercondary-server>" .\deploy-webjob.ps1 ` -PrimaryDatabaseServer "<primary-database-server>" ` -Database "<database>" ` -SqlServerLogin "<login>" ` -SqlServerPassword "<password>" ` -WebappParentResourceGroup "<parent-resource-group>"` -WebappResourceGroup "<webapp-resource-group>" ` -WebjobAppLocation "<path-to-webjob>" ` -DeploymentDestinationFolder "<destination-folder>" ` -slot "upgrade" .\start-deployment.ps1 ` -token "<token>" -accountName "<account-name>" ` -projectSlug "<project-slug>" ` -webappName "<webapp-name>" ` -resourceGroupName "<resource-group>" ` -deploymentEnvironment "<environment>" ` -slot "upgrade"
In case you find my blog’s content interesting, register your email to receive notifications of new posts and follow chsakell’s Blog on its Facebook or Twitter accounts.
|.NET Web Application Development by Chris S.|
Categories: Angular, asp.net core, Azure, Best practices
Thank you for another amazing post Chris, keep up the good work!
I have read the book and I must say that I have learnt a lot for building large scale applications. I hope you create a book about microservices architecture in the feature.
Glad to hear that Noah. Your suggestion sounds interesting so I ‘ll definitely consider it