SaaS CRO: What You’re Not Testing (But Should)

SaaS CRO: What You're Not Testing (But Should)

Think back to the last experiment you ran for your SaaS company. What were you trying to learn or improve? Maybe you wanted to increase email captures or free-trial leads

Now try to think of the last time you experimented with something other than your acquisition strategy. If you’re struggling, you’re not alone. 

Article after article, course after course, conference talk after conference talk addresses acquisition experimentation—getting more conversions at the top of the funnel. 

Exhibit A: This is the table of contents from the article that currently ranks first for the keyword “conversion rate optimization tips.” Every tip is acquisition-centric.

example of cro guide that focuses only on acquisition.

Exhibit B: This is the best-selling course in the “Conversion Rate Optimization” category on Udemy. It doesn’t even mention post-acquisition experimentation.

example of cro course that focuses only on acquisition tactics.

Now, the exact terms are a little different at every company, but the SaaS customer lifecycle generally looks something like this:

  1. Awareness. They become aware of your company and/or product.
  2. Acquisition. They make a purchase.
  3. Activation. They begin to use your product and discover value.
  4. Adoption. They experience value and regularly use your product.
  5. Expansion. They upgrade, become more engaged with your ecosystem, etc.
  6. Referral. They advocate for your company and/or product, actively referring new business to you.
  7. Reactivation. They’re at-risk or churned, and need to be re-engaged.

We write and talk almost exclusively about the first two stages—how to capture more emails, how to get more people to sign up for a free trial, how to get more people to enter their credit card information, etc.

You see the point, right? There’s a gaping hole in the way that SaaS companies talk about conversion rate optimization and experimentation.

What about, you know, all the important stuff that happens after those initial, top-of-funnel conversion points? What about stages three through seven, which can generally be grouped under the “retention” umbrella?

Why lifecycle optimization matters for SaaS

We know how important and valuable retention is. We’re all familiar with the classic retention stats:

  1. Acquiring new customers is 5–25 times more expensive than retaining existing customers.
  2. The Pareto Principle states that you get 80% of your revenue from 20% of your customers.

But as the years pass, more and more data piles up to support the value of retention:

  • It only takes one or two bad experiences to lose a customer forever. The 2017 Customer Service Barometer found that 33% of U.S. customers consider switching companies immediately following a single instance of poor service. That number jumps to 60% after a second instance. In fact, an unhappy customer will share their negative experience with about 15 people. A positive experience, similarly, is shared with about 11 people.
  • Customers trust each other much more than they trust us. According to the Q1 2018 HubSpot Research Trust Survey, 81% of customers trust recommendations from family and friends over business advice. Some 65% don’t trust ads, and 71% don’t trust sponsored social media ads, in particular.
hubspot statistics on report of consumer trust.
  • The ROI of an existing customer is higher than the ROI of a net new customer. According to Bain & Company, a 5% increase in customer retention correlates with at least a 25% increase in profit. In a two-year study, Motista found that emotionally invested customers will spend $699 with a company annually while regular, satisfied customers will spend only about $275.
  • Customer loyalty is very, very lucrative for businesses. InMoment found that 60% of loyal customers purchase more frequently from their preferred companies, and 50% of loyal customers will purchase more products from those preferred companies.

Pretty compelling case, right? 

Yet according to Reply.io, SaaS executives continue to throw money at acquisition. Why? Because it’s more straightforward and, frankly, usually more exciting to talk about. 

Nearly 90% of respondents listed new customer acquisition as a “high priority.” Only 59% and 46% said the same of existing customer renewals and upsells, respectively.

Let’s break the cycle.

The 5 neglected stages of the customer lifecycle

Before we dive into these customer lifecycle stages, a word of caution: Any reference to a linear funnel or customer lifecycle is an oversimplification. Funnels and lifecycles are rarely linear.

Leads and customers often move through them in new and unexpected ways. However, funnels and lifecycles are helpful for comprehension and communication, so roll with me here, okay?

1. Activation

During the activation stage, customers are beginning to use your product and discover the value. These are the first 30, 60, 90, 120 days using your product. It really depends on the nature of your product and business. The goal at this stage is to get the customer to experience your product’s value as quickly and then as frequently as possible.

Metric examples:

  • 30-day retention, 60-day retention, 90-day retention, 120-day retention, etc.;
  • Product or onboarding milestone completion rates;
  • Speed to first value experience.

2. Adoption

During the adoption stage, customers have experienced the value and are regularly using your product. At this point, your onboarding email series and in-product cues have cycled through. The goal at this stage is to get the customer to form habits around the use of your product. 

I’ve written an entire article on this topic, but here’s what you need to know about habit-forming products: We want to get from trigger to reward as quickly as possible. Humans develop habits based on what we believe is the fastest way to get from trigger to reward.

Hook Model for habit-forming products
Habit-forming products shorten the time between trigger and reward. (Image source)

Metric examples:

  • Login frequency and consistency;
  • Frequency of value experience;
  • Product usage (e.g., number of HubSpot user accounts, number of Trello cards, number of WordPress articles, etc.);
  • Renewal rate.

3. Expansion

During the expansion stage, customers have upgraded, become more engaged with your ecosystem, etc.

Take HubSpot, for example. At this stage, a HubSpot customer might have hired a HubSpot partner to boost their marketing. Or upgraded from the Professional plan to the Enterprise plan. Or even just joined a HubSpot user group.

The goal at this stage is to deepen engagement and loyalty, whether that results directly in monetary gain (e.g., upgrading plans) or not (e.g., joining a user group).

Metric examples:

  • Monthly recurring revenue (MRR);
  • Average revenue per account (ARPA);
  • Engagement;
  • Customer lifetime value (LTV);
  • Upsell/cross-sell conversion rates.

4. Referral

During the referral stage, customers have become advocates for your company and/or product, actively referring new business to you.

Perhaps they’re enrolled in a formal affiliate program. Or maybe they’re actively adding collaborators to their Trello boards. Or maybe they’re earning Uber Cash by referring friends to the app. Or maybe they’re volunteering their time in your community forums, helping new customers grow.

uber cash.

Whether they’re part of a formal or informal referral loop, the goal is to get the customer to identify with your company and/or product so heavily that they become a marketing and sales vehicle.

Metric examples:

  • Product affinity;
  • Referral or affiliate revenue;
  • Loyalty rewards redemption rate.

5. Reactivation

During the reactivation stage, customers are at-risk or churned, and need to be re-engaged. Maybe they haven’t logged in for a couple weeks. Or maybe they recently cancelled their subscription, which ends in a few days. Or maybe their subscription is up and their account is completely dormant.

The goal at this stage is to re-engage and reactivate those who are demonstrating at-risk behavior patterns or who have completely churned.

Metric examples:

  • Customer save rate;
  • Customer churn rate;
  • Re-engagement rate.

How to master CRO for SaaS

1. Clearly define customer lifetime value (LTV).

LTV is a complex, advanced metric. Depending on your business model, it can be tricky to define. 

First, the model has to assume the definition of a “lifetime.” For example, do you use a true lifetime? Or something more finite, like three years?

Second, the model has to define revenue and cost items, which can be complicated in large SaaS ecosystems. For example, do you include staff salaries? Do you include revenue sharing with other parties?

The list of complexities goes on. When you start considering LTV forecasting, segmentation, cohorts, etc., the waters get muddy quickly.

Tommy Walker wrote an entire article on the topic, but here’s an example of a basic SaaS LTV formula:

LTV = (Average Revenue Per Account x The Difference Between Revenue and Cost of Goods Sold) / Customer Churn Rate

Again, this is an oversimplified version of LTV. But it’s a good starting point if you have imperfect or missing data. Starting with something is better than ignoring retention altogether.

As your retention experimentation program matures, you’ll adjust the formula to account for MRR fluctuations, non-linear churn, and enterprise customers, for example.

If LTV isn’t defined for all possible routes a customer can take, you’ll find it difficult to meaningfully experiment at the bottom of the funnel. When it is defined, you can use the data to forecast and set high-level retention objectives, which will fuel your experimentation program.

Tactical resources

2. Choose the right metrics.

When you’re experimenting at the acquisition level, you have a simple focus. For example: increase the number of visitors converting to email captures, increase the number of email captures converting to leads, increase the number of leads converting to customers, etc. 

It’s as close to linear optimization as you can get.

Sure, you need balance metrics (e.g., gross customer adds and net customer adds) to ensure you’re not unconsciously gaming the proverbial system. But, generally speaking, you know that more email captures, more leads, and more customers are good for business.

Retention metrics are more complex for a few reasons. 

  1. There’s a whole lot of options for SaaS customers to choose from. Deciding what the next most valuable action for a specific customer is at any given time is complicated and contextual.
  2. As we discovered above with LTV, retention metrics are more complicated to benchmark and forecast.
  3. Third, cohorts and segments make experimentation more difficult. Say you want to isolate a segment of customers and run an experiment to increase adoption for a new product. You have to shift your thinking from something as simple as session-to-lead conversion rate to something as complicated as a specific cohort’s adoption and retention rate for a specific product, for which you may or may not have a reliable benchmark.

Again, the list goes on.

It’s perhaps not surprising, then, that marketers who do venture into the world of retention experimentation tend to get distracted by less-than-useful retention metrics, like net promoter score (NPS).

chart showing which saas metrics are most common.
The most common SaaS metrics, according to a Totango report. (Image source)

Choosing the right retention metrics comes down to one thing: understanding what best predicts the long-term success of your customers. (Remember when Facebook found that users who add seven friends in 10 days are more likely to remain engaged long-term?)

Simple, but not easy. These metrics will change as your data and retention program matures. For example, you might begin to rely on propensity modeling

Use the customer lifecycle to loosely define “conversion points” (e.g., adoption to expansion).

Tactical resources

3. Choose the right audiences.

You have two landing pages. One might be for enterprise customers; the other for freelancers, for example. Or maybe one gets mostly organic search traffic while the other is a non-indexed destination for paid social campaigns. You’d optimize those pages differently, right?

Similarly, at the bottom of the funnel, very rarely should you run a single experiment on all of your customers. Instead, you choose pockets of customers.

For example:

  • Customers who use Product X.
  • Customers who have completed milestone Y, but not milestone Z.
  • Customers who have hired a partner (think back to that HubSpot example).
  • Customers who have created more than 5 Trello cards in 24 hours.
  • Customers who have sent more than 10 FreshBooks invoices this month.
  • Customers who are demonstrating at-risk behavioral patterns.
  • Customers who pay $XX per month in third-party fees.

Your imagination is the limit. 

Another common division is high-value vs. low-value customers. How do you get more out of your high-value customers, and how can you turn more low-value customers into high-value customers? 

These definitions are different for every company, of course. For some, a high-value customer is in the 80th percentile of MRR. For others, a high-value customer is simply on the most expensive plan. 

The sophistication of the division isn’t important; it will evolve as you grow. What’s important is that:

  1. The entire company agrees upon the definitions of these customer states, including the at-risk state.
  2. You can clearly track movement between these customer states.

Tactical resources

4. Be aware of interference and degrading results.

testing-example-with-interference
Guillaume Saint-Jacques, an experimentation research scientist at LinkedIn, explains interference: “When treatment leaks into control, we can no longer rely on computing mean(B) – mean(A).” (Image source)

Running experiments properly and with rigor is important, whether it’s at the acquisition or retention level. A well-designed and well-implemented experiment is well-designed and well-implemented whether at the top or bottom of the funnel.

But here are a few rules and concepts that are especially important to keep in mind for retention experimentation:

  • It’s easy for a customer to end up in multiple experiments at the same time. This isn’t inherently bad. For example, at any given time, you’re likely in multiple Netflix experiments. The problem emerges when experiments conflict with one another, skewing the results. How are you monitoring experiment interference and preventing this? (LinkedIn has a thorough article on detecting interference.)
  • Are you reliably recording which acquisition-level experiments customers were assigned to at the top of the funnel? This information will be valuable to know when assigning them to bottom-of-the-funnel experiments.
  • Experiment results degrade. Are you rerunning experiments to verify initial findings? Are you measuring down the funnel (in addition to your primary metric) to identify this degradation, as well as false negatives and false positives?
  • Are you using balance metrics (think back to gross customer adds vs. net customer adds) to ensure you’re not gaming retention metrics? For example, are you increasing 30-day retention to the detriment of MRR? It’s especially easy to unknowingly game retention metrics at large companies where different departments have different key performance indicators (KPIs).
  • Are you acknowledging the importance of incrementality and recording it in your experiment results? Your customers will take a lot of actions, with or without your intervention. You need to understand the true value of your intervention.

Tactical resources

Conclusion

Listen, I get it. Acquisition experimentation is fun and arguably easier to talk about. 

Adding urgency to a landing page and seeing a 23% increase in the session-to-lead conversion rate makes for a more compelling headline than adding a time-based, in-product cue and seeing a 23% increase product milestone completion rate for Segment A.

But we’re doing our discipline (and SaaS businesses) a disservice by ignoring five of the most valuable stages of the customer lifecycle: activation, adoption, expansion, referral, and reactivation.

So, here’s what you need to do to master conversion optimization for SaaS:

  1. Clearly define your customer lifetime value (LTV), even if the initial formula you use is oversimplified.
  2. Understand what best predicts the long-term success of your customers so that you can choose the right retention metrics for your experiments.
  3. Get a company-wide consensus on the definitions of key customer states (e.g., high-value customers, low-value customers, at-risk customers, etc.) and ensure you can track movement between those customer states accurately.
  4. Be aware of common experimentation pitfalls and traps that are especially prevalent when experimenting at the bottom of the funnel, like interference and degrading results. Once you know the risks, you can take meaningful steps to prevent or, at least, limit them.

Related Posts

Join 95,000+ analysts, optimizers, digital marketers, and UX practitioners on our list

Emails once or twice a week on growth and optimization.

  • This field is for validation purposes and should be left unchanged.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Current article:

SaaS CRO: What You’re Not Testing (But Should)