Java, NoSQL, SQL, REST API and other scary words, QA (eng)

Blue Green Deployment, Canary Run, A/B testing difference

I put the word “difference” to the header just for fun because all these things are different and could work together. Here are definitions:

  • Blue Green Deployment  is about how to deploy a new version of your software
  • Canary Run is about testing your software on a small amount of random participants who got no ideas that they are testers now
  • A/B Testing is about testing your ideas about how to improve  usability, popularity, noticeability, etc on a different focus groups.

Let’s start with the most complicated of these 3 items –

Blue Green Deployment

I believe, Martin Fowler already did all work for me and this is a great description he made at his article about Blue Green Deployment:

One of the challenges with automating deployment is the cut-over itself, taking software from the final stage of testing to live production. You usually need to do this quickly in order to minimize downtime. The blue-green deployment approach does this by ensuring you have two production environments, as identical as possible. At any time one of them, let’s say blue for the example, is live. As you prepare a new release of your software you do your final stage of testing in the green environment. Once the software is working in the green environment, you switch the router so that all incoming requests go to the green environment – the blue one is now idle.

Blue-green deployment also gives you a rapid way to rollback – if anything goes wrong you switch the router back to your blue environment. There’s still the issue of dealing with missed transactions while the green environment was live, but depending on your design you may be able to feed transactions to both environments in such a way as to keep the blue environment as a backup when the green is live. Or you may be able to put the application in read-only mode before cut-over, run it for a while in read-only mode, and then switch it to read-write mode. That may be enough to flush out many outstanding issues.

The two environments need to be different but as identical as possible. In some situations they can be different pieces of hardware, or they can be different virtual machines running on the same (or different) hardware. They can also be a single operating environment partitioned into separate zones with separate IP addresses for the two slices.

Once you’ve put your green environment live and you’re happy with its stability, you then use the blue environment as your staging environment for the final testing step for your next deployment. When you are ready for your next release, you switch from green to blue in the same way that you did from blue to green earlier. That way both green and blue environments are regularly cycling between live, previous version (for rollback) and staging the next version.

Canary Run

Canary Run

Now let’s have a look at Canary Run, also known as  Canary Release: this is a technique to reduce the risk of introducing a new software version in production by slowly rolling out the change to a small subset of users before rolling it out to the entire infrastructure and making it available to everybody.

Similar to a Blue Green Deployment, you start by deploying the new version of your software to a subset of your infrastructure, to which no users are routed. Whenever it is ready to be displayed for your users – show it for 2-5% of your users to check if everything is ok and ready to be introduced to all other users you got.

A/B testing

green deployment

And the simplest one – A/B testing. Imagine that you would like to test – if users  find your new idea of “hot items” (like – hot articles or hot items to sell) with a new algorithm better then previous or not? You basically need to check conversion here as a main indicator of your success. So you need 2 versions of your system/web-site to be active and tested (based on previous 2 topics, for example) and all you need to do – route 50% of users to Variation A and the rest 50% – to variation B and compare conversion.



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s