Analytics & Testing

How to Run an A/B Test with ChangeAgain

The team from ChangeAgain, a tool for a/b testing, provided us with this walkthrough of how to set up a workflow for a/b testing experiments that is accurate and reliable.

What is A/B Testing?

Also known as split testing, a/b testing refers to two versions of a web page or application – version A and version B. A/B testing platforms allow marketers to insert code into their page and then develop the two versions in the A/B testing platform. The A/B testing platform ensures each variant is displayed to the visitor and analytics is provided on which one performed better. Typically, the performance is tied to a click-through on a call-to-action.

The Process of setting up A/B Test

  1. Generate a hypotheses – Brainstorm a list of 15 hypotheses of what is not convenient on your web site, what value prepositions are not clear, and which call-to-actions are not obvious. Prioritize them by the influence on your conversions and time needed to implement it. Choose the experiment that will mostly effect the conversion and needs less time to implement.
  2. Set the goals for the experiment – Every experiment should increase the certain metric of your web-site. For example, if you have landing page – the changes should affect sign in/order button.
  3. Create variations – When you have chosen the hypothesis you would like to change and establish traceable goal – implement the variation. The most important step for that step is to make only one change per variation. If you have changed title of the web page, do not change color of the button, because it will be rather difficult to interpret the results of the test. Give the designer and developer task to prepare variation.
  4. Launch the experiment – Typically, this is accomplished by pasting the code from your A/B test into your content management system and enabling the experiment. Be sure to test your page to ensure the test is published as tested.
  5. Observe the experiment over a period of time or number of visits where you’re assured that the final analytics will be statistically sound. Two weeks is pretty standard for a site with 100 conversions a day. If you receive less conversions, you’ll want to wait longer.
  6. Choose the winner based on statistically valid results. Don’t know what statistically valid is? Utilize the A/B Significance Test from KISSmetrics.
  7. Apply the winning changes to your site. Remove the A/B Testing code and replace it with the winning variant of the A/B Test.
  8. Start over at #1 to further clarify the results or start another test.

A/B testing is infinite process; you should be able to increase your conversion rates 3 to 5 times through different tests. Not all the experiments will be successful but when they are, it’s a great way to maximize the performance of your site.

About ChangeAgain’s A/B Testing Platform

ChangeAgain offers a platform that is priced out by the number of experiments you have and not based on the impressions of your site – very helpful since large volume sites can get very expensive to test. They also have some distinguishing features, like the ability to synchronize goals with Google Analytics and a visual editor that requires no coding experience.

Leave a Reply