Home › Guides › A/B Testing › Case-Studies
A Complete A/B Testing Guide With Examples: What & How It Works
GraphicSprings is one of the popular logo-maker software companies that helps businesses design a logo for free.
The company used survey feedback software Qualaroo to create online surveys to help collect information required before running an A/B test.
The survey feedback provided valuable insights for their A/B tests, which produced astonishing results, as GraphicSprings Logo Maker saw a 30 to 50% boost in sales conversion rate. Moreover, there was an increase of 41% in the company’s revenue.
According to GraphicSprings, gathering insightful data before running A/B tests was the key to their success.
Atlassian caters to one of the biggest companies in the world like Coca-Cola, and BMW, providing a suite of collaboration software to streamline teamwork. At Atlassian, experimentation is a crucial part of every customer journey stage.
They start with qualitative insights and use the data to develop multiple hypotheses for solving the customers' issues and pain points. Then, they test various solutions to come up with the one that provides the desired results.
Example: Product bundling
They hypothesized that bundling the products together can improve customer convenience and encourage more people to try their products. However, after running the A/B tests, they found that the customer got confused with bundles including three or more products and could not fully engage with any single product. So, they stopped creating bundles of more than two products.
Example: Payment information
They decided to remove the credit card details field from the free product trial registration form. It would allow users to try the product without providing any payment information. With this hypothesis, they ran the A/B tests and noticed a significant increase in the traffic without loss in conversion rate.
Atlassian runs over 25 experiments per month. Experimentation is used at various departmental levels to take calculated risks, design better products, and make data-backed decisions for improving customer experience.
Groovehq, a company that offers customer service tools, faced conversion issues on its landing page. Though the traffic was significantly higher, it had a conversion rate of 2.3%.
The team started by collecting the required data about the areas of improvement. They talked to CRO experts and marketers to find their current messaging and page design problems. They also got in touch with their customers to collect their insights about the product and brand.
They collated the entire data to develop a hypothesis and design the new webpage using a basic wireframe.
Choosing from over 50 mock headlines and numerous design elements, the team took over four days to create a design prototype for A/B testing. They ran the test for over ten days to achieve the required statistical significance and sample size for the test data.
Result: The new page lifted Groovehq's conversion rate from 2.3% to 4.3%. The testing also saved months of work and thousands of dollars that would have been spent in blindly overhauling the entire page.
When Belron, one of the world's leading automotive glass repair and replacement companies, wanted to enhance customer experience, they opted for Qualaroo.
They targeted the visitors’ exit intent using focused surveys to collect information about their needs and experience on the website.
They used Google Analytics and customer feedback from test variations during A/B testing to discover friction points that the users might have experienced with new features or functionality.
It helped the Belron team identify several segments and experiences of people leaving the website using the exit-intent research data and testing.
They learned about three distinct visitor segments that helped them modify the website for each segment type to cater to the immediate customers and possible prospects.
Bonus Read: Best NPS Software: Top Net Promoter Score Survey Tools
Watch: How Companies Used Feedback to Drive Tests that Drove Conversions
These are only a few examples of the impacts that testing and experimentation can produce in different departments. Whether it is A/B testing, multivariate testing, or any other technique, testing gives you the advantage of following data points rather than intuition to build better experiences for your customers. Even when the test fails, it gives insights on what not to do in the next iteration.
Qualaroo is the world’s most versatile survey tool
The Beginner’s Guide to Conversion Rate Optimization (CRO) is an in-depth tutorial designed to help you convert more passive website visitors into active users that engage with your content or purchase your products.
With a 30% or higher response rate, every product owner should be asking their customers these questions.
Whether you are developing a new product or have been selling the same one for years, you need user feedback.