Are you diving into the world of A/B testing? Running an A/B test can provide online business owners with valuable insights into what designs, headlines, placement, colors, and other variables their customers respond to the most. However, an A/B test is only helpful if it is run correctly—and strategically. Here are a few testing best practices to keep in mind that will ensure your A/B test gives you the most accurate and actionable information possible.
1. Choose the right A/B testing tool for your business.
There are a variety of tools at your disposal for running A/B tests, with two of the most popular being Optimizely (best suited for enterprise-level customers) and Visual Website Optimizer (VWO). This article provides a pretty comprehensive breakdown of some of the most well-known options (including Google Analytics, which is free).
When choosing the right testing tool for your business, make sure you carefully consider what your needs are. What is your budget? How skilled are you/your team in terms of similar software? Do you need a whole gamut of features, or just simple implementation and reporting capabilities? You have a plethora of options, so don’t just go with the first tool you find.
2. Set your expectations to “moderate.”
Test results aren’t always crystal clear, and while the results will show you how certain site changes will perform, they will not show you why that happens. This is an important distinction, and you should be careful not to assume you know the answer. To keep testing in perspective, it’s wise to remember an observation by Ron DeLeggee II, author of Gents With No Cents: “99% of all statistics only tell 49% of the story.”
3. Be open to test ideas from any source.
One way to determine your testing variables is by asking people who have never visited your website to give you feedback on what they like and don’t like. If there’s one thing everyone has, it’s an opinion, and you’ll be surprised at how often others can easily recognize problems and opportunities that you yourself never notice.
You can also employ inexpensive consumer feedback tools that lend a “fresh pair of eyes” to your site. For example, User Testing offers an inexpensive alternative to traditional marketing research methods and can be a gold mine for insights. You’ll likely be surprised by what you can learn in a very short period of time.
4. Make note of your typical metrics beforehand.
Of course, A/B testing has its own built-in metrics—the end of your test should indicate a winning variable by observing which variation performs better than the other. But if you are testing two new variations against one another (rather than a new variation against your current version), noting your metrics before the test is a vital step that will give you even more valuable data regarding what actions to take after your test.
Say you are changing the CTA buttons on your site from their usual blue color and have decided to test a beveled red button against a sharp-cornered red button. If a red, beveled button wins the test with metrics much higher than your blue button did in the same amount of time, then making that change on your site is a great idea. However, if the red, sharp-cornered button wins the test but performs on par with your blue button, then making that change on your site may not be necessary.
5. Run your tests for at least a week.
Resist the urge to celebrate if your tests show a clear winner and loser on Day One. Early results—especially where you’ve observed fewer than 100 conversions—are often misleading. Testing timeframes can vary dramatically; however, if it’s been a few weeks, you’ve got a good volume of traffic in each variation, and your testing tool is displaying 90% or higher confidence, then you’re probably good to celebrate your conversion-boosting winners.
6. Don’t let testing replace your own convictions—but do respect the test.
At the end of the day, your ecommerce site is yours, so you have the ultimate say. But don’t be afraid to try something totally different—even if you don’t particularly like the idea. Amazon founder Jeff Bezos once said, “The amount of useful invention you do is directly proportional to the number of experiments you can run per week per month per year. So if you’re going to increase the number of experiments, you’re also going to increase the number of failures.” It’s easy to see how this applies to website optimization. The more you test different variables, the faster you’ll find what works best—and what doesn’t.
In Conclusion
When you take the time to run A/B tests in a strategic manner, your tests can yield helpful insights that you can incorporate into future plans for your business. Check out our posts on the benefits of A/B testing and how to run an A/B test for more information.