CRO & Testing

The world’s best archive of content on conversion optimization and online experimentation (A/B, MVT, bandit, AI etc).

fishing net

Years ago, when I first started split-testing, I thought every test was worth running. It didn’t matter if it was changing a button color or a headline—I wanted to run that test.

My enthusiastic, yet misguided, belief was that I simply needed to find aspects to optimize, set up the tool, and start the test. After that, I thought, it was just a matter of awaiting the infamous 95% statistical significance.

I was wrong.

Keep reading »
Hold-Out Groups: Gold Standard for Testing—or False Idol?

You regularly run A/B tests on the design of a pop-up. You have a process, implement it correctly, find statistically significant winners, and roll out winning versions sitewide.

Your tests answer every question except one: Is the winning version still better than never having shown a pop-up? Keep reading »

9 Skills Marketers Need in the Age of AI

The robots are taking over. They’re going to be better than us, and then steal our jobs, and eventually turn on us and conquer the world. That’s the kind of hyperbolic language you’re probably hearing everywhere today. But some of it is true.

Will the skills you’re learning today be obsolete by tomorrow? Is there a point to learning new things if artificial intelligence (AI) is going to do it better? Is marketing going to become completely automated, and no longer require the human touch?

Keep reading »