How To Set Up an A/B Test Using Google Optimize

If you find our step-by-step guide for setting up an A/B test useful, you might also like our article detailing when you should consider running an A/B test! 

Running A/B tests on your website is a great way to identify how to improve your conversion rates without having to undergo an entire site redesign. In fact, it’s becoming more common for brands to use A/B testing to guide decisions typically made during redesign discussions. For example, your design team might be struggling with the decision of what image to place within the homepage header or how to create the most effective calls-to-action (CTA). Running an A/B test can provide data-driven insights that may help inform design decisions. 

Related Content:

• Website Analytics Agency
• Google Analytics 4 Agency
• Conversion Rate Optimization Services

But before you can start running A/B tests, you’ll need to successfully set up a test. At Go Fish Digital, we use Google Optimize as our testing tool and we’ve created an easy step-by-step guide for getting your first A/B test quickly up and running in the tool.

Identify a Clear Hypothesis & Test Objectives

Identifying Clear Test Objectives

A key part of setting up a test in Google Optimize is to identify the test’s primary and secondary objectives. Objectives are the metrics you want to monitor while you’re testing. For example, if my objective is to increase revenue by redesigning a portion of my website’s homepage, I will want to set transactions as my primary objective. If I just want to increase the number of clicks a CTA receives, then I would want to set “Add to Cart Button Clicks” as my primary objective. 

Secondary objectives are optional and less essential to define, but can provide helpful insights that you would not have gotten otherwise. For example, if you want to improve transactions coming from a product page, you would want to set transactions as the primary objective and “Add to Cart” clicks as the secondary objective. During the test, if you notice your audience is clicking on the “Add to Cart” button, but there’s an overall decrease in transactions, this could be a sign of an issue with your shopping cart user experience. With that information in mind, you can make positive changes to your website that can lead to more conversions.

Writing a Clear Test Hypothesis

Once you’ve identified what your primary and secondary objectives are, you’ll want to generate a clear hypothesis statement. Your hypothesis will help keep your team aligned on what change you want to make and how it will impact the website’s performance.

In order to write a clear and effective hypothesis statement, you’ll want to think through three major questions: 

1. What element are you modifying on the site?
2. What do you think the outcome will be?
3. Why do you think this will be the outcome?

Once you’ve answered these three questions, you’ll be able to write your hypothesis statement using an “If, Then” format.

If (we change element A), then (expected outcome will happen), because (reason behind your hypothesis). 

Identify Optimal Testing Page

Whether you want to run an A/B test to inform design decisions or improve your site’s conversion rate, it’s important to run the test on a page that will deliver results. For example, pages that receive a lot of traffic, such as the homepage or a popular product page or blog post, are great options for A/B testing because you’ll receive a significant amount of data, which will lead to a more accurate result in the end.

One way to identify good test pages is to look at your top landing pages over the last 6 months in Google Analytics. You can find this information by navigating to Behavior -> Site Content -> Landing Pages. Then you’ll want to configure the date range in the top-right corner to display data from the last 6 months. 

Consider seasonality when you’re choosing your test pages. If you’re running a test in the off-season, it may take longer to collect data and you might not get data from the most qualified audience, as it’s likely regular users are no longer searching for a specific product or information. Selecting the right landing page can have a major impact on your overall timeline and test results. It’s important to consider these factors carefully when making your selection. 

Calculate Test Duration

Calculating how long it will take for a test to deliver desired results is important as it will help you set clear expectations with internal and external teams. There are many calculators online that help you do this. Our favorite tool is VWO’s a/b test duration calculator.  It takes into consideration how many visitors your site receives and how much you want to increase your current conversion rate to calculate the expected test duration.You can easily use the calculator to manipulate different factors to try to predict an optimal timeline. 

Setting Up & Monitoring Your Test

Once you’ve figured out the specifics of what you’re testing, why and how long, you’re ready to get the test up and running in Google Optimize. Google Optimize has a user-friendly interface that makes it easy for people with no prior design/dev experience to make changes to test landing pages. 

10 Steps to Set Up Your A/B Test

1. After logging in to Google Optimize, click “Create experience” in the top right corner.
2. Enter the experiment name, target URL and the type of test you plan to run (i.e A/B test, multivariate, redirect test). 
   1. A/B – tests two or more variants of a page.
   2. Multivariate – tests variants with two or more different sections.
   3. Redirect – tests separate web pages identified by different URLs or paths. 
3. Next, select “Add variant” and name the variant you plan to change for your test. For example, if you’re testing a different CTA color, you would want to name your variant “CTA Color – Blue”. 
4. Click edit and begin editing the page using Google Optimize’s visual editor. 
5. The best way to make changes to a specific element is by clicking on what you want to edit and using the visual editor in the bottom right corner. The visual editor enables you to intuitively change the background color, font size, font type, etc. 
6. If you select “Edit element” on the pop-up, you also have the ability to edit the existing HTML, insert new HTML or run javascript. You also have the option to remove the element entirely. The optimize resource hub has a great overview of using the visual editor.
7. Once you’ve made the desired change, click “Save”, then “Done” and return to the main dashboard. 
8. Next, you’ll need to add targeting rules. Targeting rules are necessary in cases where you might be testing a templated change. Regular expressions can help effectively target traffic from all URLs that utilize a specific template. By clicking “+Add rule”, you can see different options for customizing URL targets.
9. Now you’re ready to set your test objectives. In the “Measurement and objectives” tab, click “Add experiment objective”. Here you can either choose from a list or create a custom objective. This is where you input your objectives identified earlier. Choosing from the list is a great option when you want to track the performance of an existing goal configured in Google Analytics. Creating a custom objective is another good option when you might want to measure the performance of an entirely new or more broad goal. For example, do you just want to see if a different page design gets more clicks overall? You can use Google Optimize to create a custom test objective. For more information, Google has a detailed guide for creating test objectives.  
10.  Before launching your test, be sure to run a diagnostics check to make sure Google Optimize is running smoothly on your site. You can do so by clicking “Run diagnostics” in the “Settings” section. 

Now that you’ve implemented your test change using the visual editor, defined targeting rules, selected test objectives and ensured Google Optimized is running smoothly, you’re ready to launch! 

Pro tip: Create a daily reminder to ensure you frequently check in on your test’s progress. It’s important to keep an eye on daily sessions and statistical significance in case there’s an opportunity to end the test early. 

Reporting

Google Optimize will indicate statistically significant results by highlighting whether a change is performing better or worse in red or green. You can view ongoing test results by clicking on the reporting tab in the top left corner. Here’s a great resource for interpreting Optimize’s report data. 

Conclusion

A/B testing can yield incredibly powerful results and now you have the information to begin setting up your own tests. Whether you decide to use Google Optimize or another platform, you’ll still need to make sure you have fundamental questions answered about why you’re running your test and what results you expect to see. We hope our guide has provided you with valuable information and has shown how achievable setting up an A/B test is. If you still have unanswered questions about A/B testing, feel free to reach out to us today!