Join the Conversation Add Your Comment

  1. Steven Macdonald

    This is a fantastic resource for AB testing!
    Great job Jaan-Matti. I’ll be bookmarking this page for many years to come.

    1. Jaan-Matti Lillevälja

      Thanks, Steve! Glad you liked it

  2. Britt McCrimmon

    Great post. I am going to have to come back to this one and chew on it for a bit. You have got it packed with info!

  3. Derek Slater

    Wow – truly epic. The details are really useful (e.g. 302 redirects). Thank you.

    My own background is in B2B media organizations, which are mostly in the dark ages in regards to testing. A fallacy I encounter often is the notion of “I don’t care what results site X got, because their business isn’t the same as ours.” Of course their business isn’t exactly the same – but considering others’ results is still a great way to create a good, informed hypothesis for testing.

  4. sheriffbradshaw

    as a Conversion Optimization Consultant, I find this to be a huge resource for A/B Testing. Tank you!

  5. Farvede kontaktlinser

    Really a good post, which comes around the most, but you say, that we should use 302 and use rel=canonical, but I thought if you use Google new tool or payed like visual website optimizer, that such things was not needed?
    best regards

  6. Awesome tips, Jaan-Matti! I definitely have not done testing nearly as much as I should. I’ve bookmarked this page so I can reference it and all your great insight. Thanks!

  7. This is fantastic – just what I needed…

  8. This is awesome! Thanks for sharing your knowledge! I totally agree with “Never stop testing – No matter how well your landing pages or e-mails may be doing, they can always do better.”

  9. Srihari Thalla

    I thought the Guide is a pdf and surprisingly this is a webpage!! I subscribed mainly because I’d download it and read it offline, NOT ONLINE.

  10. david

    It would be really nice if the print button worked and this could be printed out

  11. Georgi

    “The importance of testing to statistical relevance” sounds like a serious misnomer to me. There is no such concept as “statistical relevance”. Statistical significance is in no way a measure of “relevance”. Interpreting a statistically significant result as a “relevant” or “significant” result is a grave mistake.

    Also, testing “to” statistical significance is another commonly made error. Sadly most tools encourage you to make it…

    I explain those issues in much more detail here:

  12. Keith

    Really, an impressive article that provides insight about the various areas of A/B testing. Apart from this, other importantb areas of considerations are having a clear knowledge about your key performance indicators, analyzing the effectiveness of your A/B testing tools and running your tests for the scheduled time duration for achieving the desired test goals.
    Further, there are various A/B testing tools like Optimizely, Mocking Fish, Maxymiser that can provide accurate results and have relatively less testing errors so as to ensure an effective split testing results for your business organization.

  13. Really great article. So many great point!

Comments are closed.

Current article:

How to Build a Strong A/B Testing Plan That Gets Results