At this point in time, the field is flooded with solutions. Finding a proper A/B testing tool isn’t the problem anymore. Now, the problem is choosing the right one.
If you work in conversion optimization – whether at an agency, in-house, or as a consultant – you almost certainly run A/B tests.
Though you could hire someone full time to sift through and analyze the pros and cons of each tool, it’s easier to learn from the experience of others and make a decision based on that.
To help you with that, we’ve assembled a list of the most popular a/b testing tools and corresponding reviews from A/B testing experts to help you in your decision making process.
Without further ado, here are the top A/B testing tools used by experts (click below to navigate to a specific tool):
- AB Tasty
- Adobe Target
- Dynamic Yield
- Google Optimize
- Sentient Ascend
AB Tasty is a reasonably priced and simple to use tool that serves as a good starting point for companies just starting in conversion optimization.
AB Tasty offers A/B testing, split testing, multivariate testing, and funnel testing capabilities. Using AB Tasty’s visual editor, you are able to quickly put together variants and tests and receive real-time reports with confidence levels on your goals. At the same time, you have full control of testing through advanced targeted scenarios, allowing you to trigger tests based on URL, geolocation, demographics, etc. You are also able to enrich the platform’s data by integrating existing data from CRMs and DMPs.
Chloe O’ Keeffe, Associate Conversion Consultant at Conversion.com, shares some pros and cons:
The basic starter plan rounds out to $249/month and includes 50,000 unique tested users and carryover of unused credit.
Adobe Target is a popular enterprise tool that combines targeted testing and personalization.
Adobe Target walks you through a three-step workflow where you first create the variant, then target the variant based on visitor segmentation, and lastly customize your goals and settings for the test. Based on this emphasis of targeting, it should come as no surprise that Adobe Target’s most exciting feature is automated personalization.
Target’s self-learning algorithm continually and automatically adjusts itself to make associations between visitor behavior and content in order to give you the best results possible. Real-time adaptation of content can help you better meet the needs of diverse visitors.
Here’s a review from Sean Patterson, a Digital Analytics and Optimization Consultant at Conversion.com:
Pricing for Adobe Target is available on request.
Apptimize is a testing tool focused on mobile optimization and is a perfect choice if mobile is an important platform for your business.
Apptimize offers you a full control visual editor where you’ll be able to rapidly create new variants and target them at different user segments. You’ll also be able to add dynamic variations and code blocks that support Objective-C, Swift, Java, HTML5, Xamarin, and React Native. Their randomized experiment methodology also ensures your experiments will return statistically significant results.
We sourced reviews for Apptimize from G2Crowd. Here’s an anonymous review talking about some things they liked about Apptimize:
It’s easy to set up and track tests. The code needed to implement the tests is straightforward. It integrates with Localytics.
And here’s a review discussing some disappointments:
“The real-time changes that we can deploy are acutally limited in the scope – so, not all of our original goals can be met with this tool. Also, we have to plan in advance what sections of parts of our app we would like to change, and only after that we can start using this feature, which means that there is a lag/delay between the time that we want to change something and the time when it actually becomes a reality. For example, with our sprint structure this delay can be as long as 3 – 4 weeks.”
Apptimize starts at $995/month and supports up to 100,000 monthly active users. You can also request a custom plan based on your needs.
ChangeAgain.me provides a visual interface to Google Experiments for A/B Testing.
A good solution for those companies that already have Google Analytics and want to try split testing.
They bill based on the number of tests you run, not the number of visitors as other split testing solutions do.
No coding skills necessary. You can launch new tests with a point-and-click visual editor
Conductrics is an advanced tool that offers methods from AB Testing blended with Machine Learning to deliver optimal experiences, for each user, automatically.
You can use it as either a server-side or client-side tool, and you can pretty much customize it in anyway you’re trying to use it.
Here’s a review from Experimentation and Optimization Expert Andrew Anderson:
Pricing for Conductrics is available on request.
Convert is perfect for work for small to medium size clients and businesses doing in-house optimization. It is well loved for its exceptional customer support over live chat.
Convert operates using an intuitive drag and drop system. You can create A/B, multivariate, and split tests using the provided visual editor and break out more the more advanced style sheet editor for dynamic pages, giving you full control of your content. It also boosts a smooth onboarding process of your CMS and e-commerce platform and seamless integration with Google Analytics. Personalization is also an option with over 35 different elements you can use to build customer profiles.
Here’s Julien Le Nestour, an applied behavioral scientist and CRO consultant based in Sydney, giving his opinion:
Convert’s most basic plan, Convert Plus starts at $599/month and gives you access to 400,000 unique users per month and 5 active domains for testing.
Dynamic Yield is a powerful suite of personalization and engagement solutions.
With Dynamic Yield you’ll have access to a wide range of tools. Most importantly, you’ll be able to continuously A/B test and optimize across platforms (desktop, mobile web, apps). These tests are predictive in nature and leverage the capabilities of machine learning algorithms to continuously improve ROI.
Dynamic Yield, however, specializes in advanced omnichannel personalization solutions. You’ll be able to segment and quantify every user interaction and response and dynamically adjust your content to best suit each individual. Combine your segments with personalized notifications to get the most out of this particular tool.
According to a review by Martin Nicholson on TrustRadius, some pros of the tool are:
- “The out of the box features are impressive and quick to implement. Overlays, messaging and content changes can be made instantly enabling tests to be carried out quickly.
- Creation of custom actions enables us to bridge the gap between our online and offline tests by injecting code changes directly onto the page.
- The test segmenting features are very robust, actions can be triggered by device, ip, cookie, audience type and at specific times of day allowing us to test whatever we want, when we want.’
And some cons:
- “We’re not using all the features of the tool, there’s so much to choose from it’s sometimes overwhelming. The support team however are superb and have always been able to help.
- More integration with GA would be useful.
- Some of the features do require some technical knowledge to implement, again support are always on hand to help.”
Pricing for Dynamic Yield is available on request.
Google Optimize is a Google tool that lets you test and tailor different variations of your website.
The main selling point of Google Optimize is its native integration with Google Analytics, allowing you to gain automatic access to rich behavioral insights. You’ll also be able to target the valuable segments you’ve already discovered using Analytics.
Try out Google Optimize for free and upgrade to Google Optimize 360 for enterprise-level website testing and personalization.
Kameleoon is an advanced optimization platform that offers a variety of features.
Using Kameleoon, you’ll be able to run advanced A/B, split, and multivariate tests quickly and easily. You’ll also be able to understand key user insights with their navigation analysis tool and test different user segments with over 40 targeting criteria. Dynamic traffic allocation algorithms also make sure your traffic is optimally divided in order to shorten your decision cycle and improve return on investment.
Julien Nouet, Global Frontend & UX Manager at Lacoste, reviews it here:
Get started on Kameleoon by creating a free account and then upgrade to an enterprise plan if the tool suits your needs.
Maxymiser is a powerful testing tool that helps you design and run sophisticated campaigns.
Like other tools, Maxymiser offers a visual editor that will help you set up variants and tests as well as an extensible coding environment for more complex projects.
However, Maxymiser shines through most in terms of personalization and targeting. This tool provides you with the ability to build rules directly using the visual editor to target profitable customer segments and display personalized content. Most impressively, Maxymiser automatically builds a customer profile that aggregates data on historic and in-session behaviors and visitor attributes for each unique visitor to your page.
Pricing for Maxymiser is available on request.
Monetate is another well-loved testing tool that has a reputation for providing fantastic client services. They provide both a testing and segmentation tool as well as a personalization tool.
Using Monetate Test & Segment, you are able to perform advanced A/B tests and multivariate tests. They also have built-in measures of statistical significance and dynamic testing which adjusts traffic allocation to optimize for ROI. Monetate’s Intelligent Personalization Engine is an advanced machine learning engine that determines which actions will help you achieve a given goal. This means you can automate and optimize personalization decisions.
Pricing for Monetate is available on request.
Omniconvert is another new player in the testing tool market. They position themselves as a sort of all in one optimization tool: pop-ups, on-page surveys, A/B testing, personalization, segmentation.
They’re pretty affordable for all they offer too, even offering a free version for under 10k visitors a month.
They’re an especially good tool for beginners, affordable with lots of features included.
Omniconvert offers a variety of plans to suite any need, from a forever free introductory plan all the way up to custom solutions.
Optimizely is one of the most popular, WYSIWYG experimentation tools and a go-to choice when working with any size company.
At the root of it, Optimizely is focused on web experimentation and personalization. They offer easy-to-use A/B and multi-page experimentation and supports experimentation across platforms (web, mobile web, iOS, Android, etc.) and shows you results in organized formats using their user-friendly Stats Engine.
More importantly, Optimizely allows you to target your experiment variations based on ad campaign, geography, cookies, and a variety of other dimensions to help you design and optimized personalized content based on your audience segments.
Diane Leung, Associate Consultant at Conversion.com, shares her thoughts:
Optimizely’s Pay As You Go is a good way to go if you’re just starting out and costs $49 for every 1,000 monthly unique visitors you test on. Even if the same visitor sees many experiments, the visitor is still only counted once.
Qubit is a testing platform focused around personalization, making it one of the strongest segmentation tools on the market.
With Qubit, you’ll be able to run A/B tests and multivariate tests in order to measure progress and effectiveness of various personalization techniques. You’ll also get access to abandonment recovery, product recommendations, and social proof tools, making this software a great choice for e-commerce businesses.
- “Great A/B test ideas based on both quantitative and qualitative evidence.
- The Qubit consultants and engineers we have worked with have an excellent ‘can do’ attitude.
- Excellent communication and clarity on test results.”
From the same review, the cons are:
- “There is room for improvement on the velocity of getting new tests developed and QA’d.
- I would like to see improvements to the exit survey interface to make it easier to flag, categorise and disseminate feedback to different areas in the business.
- Depending on the test/experience it can add extra weight to page load times and cause ‘flicker’.”
Pricing for Qubit is available on request.
Sentient Ascend is a new generation testing tool that merges machine learning algorithms with conversion optimization.
The concept is simple, genetic algorithms can find optimal combinations of design choices faster and with greater accuracy than humans can. By isolating the best local maximum, the machine assembles the highest performing segment and pushes forth optimal personalization solutions. Of course, this also means you can test all of your ideas at once and see evolution in action.
Here’s Guido Jansen from Euroflorist explaining some pros and cons of the tool:
Viljo Vabrit, Managing Director at CXL Agency, wanted to point out a few issues with the tool as well:
- “You can’t edit active tests. If there’s a small bug or something breaks due to a change in the control you either stop the experiment and create a new one or contact the support and hope they can make the change for you as soon as possible. Same also goes for the changes related to audience and URL targeting.
- QA-ing everything is a pain. And I am not talking about going through the thousands or even millions of possible combinations. Just previewing a specific combination requires you to understand the complex query parameter logic in place.
- The visual editor can be quite slow and crashes occasionally, sometimes with all of your changes.”
Pricing for Sentient Ascend is available on request.
SiteGainer is a company that offers a full suite of conversion optimization tools, including A/B testing, multivariate testing, personalization, heat maps, popups, and surveys.
The tool is quite easy to set up and use, and creating everything from simple a/b tests to surveys, popups, and more is very intuitive.
SiteGainer offers a free trial and then pricing plans are based on monthly traffic, payment schedule, and additional services.
SiteSpect offers a server-side solution and is a great choice for more technically advanced testers. By editing HTML before it leaves the server, it is popular with self-hosting companies and security-conscious clients.
Of course, SiteSpect does offer a visual editor tool to help make setting up optimization and testing processes convenient and efficient. What makes it different is its ability to give you access to both client and server side, allowing for a comprehensive view of the entire customer experience and high performance for speed and scaling.
Choose from a variety of packages that vary in integrations, features, and accounts depending on your scale and needs.
Visual Website Optimizer (VWO) is another big player in the A/B testing software market. It is a top choice for businesses with slightly smaller budgets and popular amongst many marketers.
One of the reasons it’s so popular is because of the variety in tools it gives you access to, which VWO separates into four categories: testing & experimentation, research & user feedback, analytics & reporting, and targeting & personalization.
With VWO you are able to create multiple variations of web pages and track various conversion goals. Like Optimizely, VWO also provides statistical analysis of your results. A favorite feature of VWO is its built-in heatmap tool that tracks visitor click behavior and tracking habits.
Notably, if personalization is important to your work, VWO offers easy tools for segmenting and delivering targeted offers.
Here’s a review of Alex Harris, Conversion Optimization, Director at Perficient:
The standard plan for VWO starts at $49/month and accommodates up to 10,000 monthly users.
There you have it: the top a/b testing tools, reviewed by the experts.
While each has their own pros and cons, it’s likely you’ll have to some some strategic thinking and exploration to get the solution that is right for your business and specific situation. Hopefully these reviews help you in the process, though.