This is the methodology that I have developed over 12 years in the industry and working with over 300 organizations. It is also the methodology that has been used to have a near perfect test streak (6 test failures in 5.5 years), even if most others do not believe that stat.
While testing is a critical part of conversion optimization to make sure we actually made things better and by how much, it’s also the tip of the iceberg of the full CRO picture. Testing tools are affordable (even free), and increasingly easier to use – so pretty much any idiot can set up and run A/B tests. This is not where the difficulty lies. The hard part is testing the right things, and having the right treatment.
The success of your testing program is a sum of these two: number of tests run (volume) and percentage of tests that provide a win. Those two add up to indicate execution velocity. Add average sample size and impact per successful experiment, and you get an idea of total business impact.
So in a nutshell, this is how you succeed:
Executing point #1 obvious, but how to do well for points #2 and #3? This comes down to the most important thing about conversion optimization – the discovery of what matters.
What is an irresistible offer?
Simply put … an irresistible offer is one that your audience cannot help but respond to. It’s compelling, it’s powerful, and it drives action.
So, how do you craft an irresistible offer?
Before you get out your pitchforks, I want to stress that this article does not represent Peep’s views.
The easiest lies to believe are the ones we want to be true, and nothing speaks to us more than validation of the work we are doing or what we already believe. Due to this we become naturally defensive when someone challenges that world view.
The “truth” is that there is no single state of truth and that all actions, disciplines, and behaviors can and should be evaluated for growth opportunities. It doesn’t matter if we are designers, optimizers, product managers, marketers, executives, or engineers, we all come from our own disciplines and will naturally defend to the death if we feel threatened even in the face of overwhelming evidence.
I’ve been to a lot of conferences. A lot. Most of them are mediocre. I made it my personal goal to deliver an amazing conference.
A/B testing is great and very easy to do these days. Tools are getting better and better. As a result, people rely more and more on the tools. As a result, critical thinking is much less common.
It’s not fair to just blame the tools of course. It’s very human to try to (over)simplify everything. Now the internet is flooded with A/B testing posts and case studies full of bullshit data, imaginary wins. Be wary when you read any testing case study, or whenever you hear someone say “we tested that”.
While mediocre people are dime a dozen, good people are always in demand. But good optimizers are in very high demand, everyone and their mother is trying to hire one. Alas they can be hard to find. If you’re looking to hire one, what should you look for? What makes an optimizer a great optimizer?
You should split test the headline first! Bollocks. The images! What a load of crap.
That’s the kind of advice most articles out there would give you. The truth is that answering “what to test first” is like answering “how much does a car cost?” – it depends.
AB testing is supposed to be straightforward and extremely transparent. It should be so easy to see the ROI – especially when compared to opaque stuff like SEO. But is it really so transparent as we’d like to think?