CRO & Testing

The world’s best archive of content on conversion optimization and online experimentation (A/B, MVT, bandit, AI etc).

fishing net

Years ago, when I first started split-testing, I thought every test was worth running. It didn’t matter if it was changing a button color or a headline—I wanted to run that test.

My enthusiastic, yet misguided, belief was that I simply needed to find aspects to optimize, set up the tool, and start the test. After that, I thought, it was just a matter of awaiting the infamous 95% statistical significance.

I was wrong.

Keep reading »
Hold-Out Groups: Gold Standard for Testing—or False Idol?

You regularly run A/B tests on the design of a pop-up. You have a process, implement it correctly, find statistically significant winners, and roll out winning versions sitewide.

Your tests answer every question except one: Is the winning version still better than never having shown a pop-up? Keep reading »

9 Skills Marketers Need in the Age of AI

The robots are taking over. They’re going to be better than us, and then steal our jobs, and eventually turn on us and conquer the world. That’s the kind of hyperbolic language you’re probably hearing everywhere today. But some of it is true.

Will the skills you’re learning today be obsolete by tomorrow? Is there a point to learning new things if artificial intelligence (AI) is going to do it better? Is marketing going to become completely automated, and no longer require the human touch?

Keep reading »

How to Get Started with Machine Learning and AI for Marketing

Artificial Intelligence is powerful and will open doors we don’t even know exist at the moment. But there are some steps you can take today to ensure that you are making the most of this new capability and are not left behind.

As this is such a nascent area, it serves to begin with a brief description of Artificial Intelligence and Machine Learning. My assumption is that you have read a dozen different definitions and should be curious about which applies to this particular essay.

Keep reading »