intermediate

A/B Testing Statistics: An Easy-to-Understand Guide

Testing tools are getting more sophisticated. Blogs are brimming with “inspiring” case studies. Experimentation is becoming more and more common for marketers. Statistical know-how, however, lags behind.

This post is filled with clear explanations of A/B testing statistics from top CRO experts. A/B testing statistics aren’t that complicated—but they are that essential to running tests correctly.

Here’s what we’ll cover (feel free to jump ahead):

  1. Mean, variance, and sampling;
  2. Statistical significance;
  3. P-values;
  4. Statistical power;
  5. Confidence intervals and margin of errors;
  6. Regression to the mean;
  7. Segmenting;
  8. Confounding variables and external factors.

And just in case you’re uncertain about why A/B testing statistics are so essential…

Keep reading »
Product Analytics: A Comprehensive Guide to Using Data for Better Product Decisions

I remember the first time that a client told me how much analytics had helped their business.

They were able to increase their sign up rate for their product by 22% while reducing their marketing costs. It wasn’t magic or fancy tactics. They simply used their analytics data to make informed decisions.

They didn’t have to guess or take huge bets. They knew exactly what was working and what they needed to do more of.

This is power of analytics and more specifically, product analytics.

Keep reading »

Agile Marketing: How to Implement Scrum for Digital Marketing

Agile marketing may not be a phrase you hear often, but it’s becoming increasingly popular and important.

Traditionally associated with development and product management, agile is a lightweight and, well, agile framework for software development and bringing features and products to market.

It stands in opposition to “waterfall” production methods that treat analysis, design, coding, and testing as discrete phases – where in agile they are treated as continuous.

As marketing becomes more data-driven, quantitative, and iterative, we can use many of these same management practices to hone our marketing campaigns, mitigate risk, and ultimately ship more effective marketing campaigns.

Keep reading »

Personalization: How to Build a Revenue Boosting Program from Scratch

Did you know that Netflix has only 90 seconds to find a show that suits a user before she gets frustrated and quits? According to a recent academic study, “a typical Netflix member loses interest after perhaps 60 to 90 seconds of choosing, having reviewed 10 to 20 titles (perhaps 3 in detail) on one or two screens.”

How does Netflix manage to find the right show for the right user so quickly?

According to the same study, 80% of its customers’ video plays comes from its personalized recommendation engine. Netflix estimates the value brought by this personalized recommendation system at a billion dollars per year.

That’s a serious win achieved through personalization.

Keep reading »