How to Do Conversion Attribution Modeling

How to Do Conversion Attribution Modeling

Customers don’t usually see one ad and then click over to purchase.

In reality, the path is much more complex, and usually includes various marketing channels – organic and paid search, referral, social media, television.

But if you’re a rigorous and data-driven marketer, the question has to cross your mind: how much credit can I give each channel for this conversion?

That is the question answered by the nearly universal (and challenging) problem of attribution.

Note: we launched a course in ConversionXL Institute taught by Kelly Gray, Program Manager – Attribution, at Google. It goes deeper into the topics brought up in this article.

What is Attribution?

Marketing attribution, according to Wikipedia, is “the process of identifying a set of user actions (“events” or “touchpoints”) that contribute in some manner to a desired outcome, and then assigning a value to each of these events.”

In other words, it’s a way of remedying that old advertising quote: “Half the money I spend on advertising is wasted; the trouble is I don’t know which half.”

Image Source
Image Source

Analytics informs you, generally, of customer behavior; attribution informs you on the effectiveness of your marketing mix. As a Google blog post put it, “Think of attribution as the peanut butter to analytics’ jelly. Yes, each is great on its own, but for many, they’re even better together.”

There are many ways to do this, many methodologies that will give you slightly different answers and with different accuracies that reflect the goals and complexities of your own organization.

And as marketing channels continue to proliferate, the problem of attribution is not getting any easier. In fact, with multiple devices and increasing channels, including many forms of dark traffic, it’s getting harder and harder to pinpoint the value of a touchpoint with much accuracy.

Let’s talk about your goals

If you’ve heard about attribution before, just for a minute, forget about everything you know (model types, accuracy, etc.). Instead, just for a minute, think about your goals. What will the outputs on your attribution model mean? What will you do with the information?

If, for instance, it seems that your display ads aren’t adding much of a conversion value according to a particular model, will you stop using display ads? Or do you believe that, given your place in the market (let’s say a newer competitor that is struggling for awareness), they give you a certain level of intangible awareness that the model isn’t accurately expressing?

At least start the conversation with your team. Answer the question: what will I do different given [X] scenario that shows up in this data?

In addition, your goals will determine how complex your attribution model actually is.

ytlain5yj-8-glen-carrie

If your marketing mix is limited to a few channels and you’re not spending much on paid acquisition, you can probably just suffice with your basic analytics setup and last click attribution (assuming you’re actually measuring everything accurately). If you’re working with a large number of marketing channels, media platforms, and offline channels, things get more complicated.

A Quick Run-through of Some Common Attribution Models

These are all “business-logic” models, or based on business rules that you set from the top down based on what you believe is the credit distributiont. Calibrating them with real-world accuracy remains a challenge. How do you know a given model is actually representing the true values of a marketing channel? It’s tough.

There are many different business-logic models. They all have some pros and some cons, so you’ll just have to weigh them in relation to your company’s abilities and needs in choosing the appropriate model.

1. Last Click Attribution

Image Source
Image Source

Last click attribution is likely the most common and the most lambasted of the attribution models built into your analytics tool.

As you can see from the image above, last click attribution is a transparently simple and inherently inaccurate model. It weighs all of the credit to the absolute last interaction. So if that last interaction was direct traffic, it ignores any efforts you’ve put into social, email, etc.

As Avinash Kaushik put it, “The only use for last click attribution now is to get you fired. Avoid it.”

2. Last Non-Direct Click Attribution Model.

Image Source
Image Source

Same deal with this model – it gives all the credit to the interaction before the last click.

Pros: it’s simple. Cons: it tends not to reflect the reality of marketing very well.

It will also almost always undervalue brand awareness because it will avoid crediting direct more often than not. As Avinash put it, “Why undervalue Direct? Why undervalue a marketer’s efforts to create brand recognition and brand value?”

3. Last Adwords Click Attribution

Image Source
Image Source

This model gives all the credit to your Adwords campaigns. It should be easy to see the inherent problems/conflicts of interest with this model.

4. First Click Attribution

Image Source
Image Source

First click is the opposite of last click attribution; it weighs brand awareness interactions much more heavily than intent-based or action-based interactions.

Of course, it comes with all the problems usually inherent with these narrow-focused models. Just one touchpoint getting all the credit just feels wrong, unless your business has an incredibly simple marketing approach. As Avinash wrote, “First click attribution is akin to giving my first girlfriend 100% of the credit for me marrying my wife.”

5. Linear Attribution

Image Source
Image Source

Finally, here’s a model that distributes credit among more than one channel. The linear attribution model simply gives the same amount of credit to every touch point.

Of course, this is a highly idealistic model. Do you really think email weighs the exact same significance as paid, as organic, as social, etc.?

6. Time Decay Attribution

Image Source
Image Source

Time decay makes a bit more sense, at least to me. The closer to the conversion, the more weight you give the channel in terms of credit.

As Avinash wrote, “After all, if the [earlier] touch points were magnificent, why did they not convert?”

7. Position-based Attribution

Image Source
Image Source

This model, sometimes known as the “bathtub model,” assigns greater weights to the interactions that occurs first and last.

While you can tweak this according to your beliefs and assumptions in your own data, the most common iteration of this divides 80% of the conversion value between the first and last interaction, and then gives 20% for everything in between.

Custom Attribution Modeling

The above models are baseline attribution models – simple heuristic-based and out-of-the-box models found in Google Analytics. They all give you an answer, but the accuracy of these models is questioned.

You can also build out custom models on top of the out-of-the-box rule-based models in Google Analytics, or you can use something like R to build more complicated models.

When attempting the former, you’re generally going to be dealing with the same problems inherent in all the above models: they’re based on gut-feeling and essentially arbitrary assumptions as to how your purchase path works.

For instance, in free Google Analytics, I can go in and shift the channel credits around, change time-decay and the lookback window, and create custom credit rules, but it’s still all based on business-rule assumptions and thus subject to discrepancies in non-stationary data (data that changes over time).

screen-shot-2016-11-29-at-10-04-03-am

For a really good article on building your own attribution models, check out OptimizeSmart’s guide.

I’ve also seen some interesting articles on doing attribution using Markov Models in Google Analytics and R. This model can be used, as LunaMetrics put it, when you’re not satisfied with the simple business heuristic models I listed above, but you don’t have access to data-driven attribution modeling (talked about below).

To really simplify a Markov Model in this use case, it looks at the likelihood of the next steps in a given conversion path, and then tries to calculate the relative importance of a given touchpoint based on its removal:

Image Source
Image Source

Doing attribution with Markov Models has the following benefits, outlined by LunaMetrics (and before that, in this paper):

  • Objectivity – No gut feelings.
  • Predictive Accuracy – Predicts conversion events.
  • Robustness – Valid and reliable results.
  • Interpretability – Transparent and relatively easy to interpret.
  • Versatility – Not dataset dependent. Able to adapt to new data.
  • Algorithmic Efficiency – Provides timely results.

Here’s a good post explaining how to do that. Here’s another. If you’ve got a good analyst, give them a go at it.

Image Source
Image Source

Don’t Forget to Look at Your Cohorts

Sometimes, time-based cohorts hold the key to finding the effectiveness of changes or channels (or at least hints at it – combine cohort analysis with controlled experiments for greater validity).

Specifically, looking at cohorts can help you determine how effective a certain marketing action was, at least correlatively. Jim Novo, founder of The Drilling Down Project, put it well on a Digital Analytics Podcast episode:

Jim Novo:

“I think the people in SaaS working with cohort analysis are doing this well.

So you look at the people that signed up in January, and by the time we got to March this percentage dropped out. We can associate that with a certain promotional effort we did in that time frame.

But then we look at the cohort that started in March, we had a different kind of promotional effort going on, and we had much better conversion from freemium to paid, or whatever, in that model.”

Related to this is a type of  “existence testing,” where you infer the effectiveness of a certain channel by dropping it out of the mix for a bit (this is what I believe the data-driven models below are based on, by the way, but at scale). Jim Novo from the Digital Analytics Power Hour podcast again:

Jim Novo:

“If you think [display] is so valuable in terms of helping other campaigns, why don’t you just kill it for a week or two weeks and see what happens? And then add it back.

Can’t you do that kind of testing? Is it worth it to you to invest that money elsewhere? How serious are you about figuring out the value of display?”

…or if You’ve Got the Cash, Buy a Data-Driven Solution

So technology has improved in the past few years. We’re able to user larger data-sets to yield clearer insights at a level of greater accuracy. Therefore, attribution solutions that offer algorithmic (or “data-driven”) models are becoming more popular. As Bill Macaitis, former CMO of Slack, said in an interview:

Bill Macaitis:

“Today, algorithmic attribution has become the best practice for data-driven marketers and companies.

We can now utilize all the available data collection, tools and models to take in all different touch points and make predictive, algorithmic attributions. When set up properly, we can track each touch point and all downstream funnel metrics. And by weighting proportionally across a very large data set, we can determine with much more accuracy and precision what should get the credit–including both online, offline, performance-based and brand advertising.

It’s not perfect, and it’s not easy. It gets difficult with word of mouth referrals, dark social, and other “hidden touches.” But, it drives a much deeper understanding of the buyer’s journey and which of your marketing efforts are paying off.”

Google Analytics 360 offers algorithmic attribution in both the online and the offline arenas, meaning you can, to a certain degree of accuracy, gauge the effectiveness of your television campaigns.

Image Source
Image Source

Of course, there are other products than the ones Google offers (Convertro is one, Impact Radius another). There’s a whole ecosystem of products or suites that are trying to solve this problem, and of course the sales pitch is compelling.

On one level – and not to short change this – the solutions really are impressive and provide tons of value. While I can’t tell you what goes on inside the big box, I believe that it’s something on the level of mass-scale experimentation where the products consistently run those “removal experiments.” This gives you a level of accuracy and granularity not remotely possible in rule-based attribution.

But on the other side, buyers are both skeptical and awed at these products because they don’t know what’s going on inside the black-box. As a SAS article put it, “digital marketing is an overwhelming ecosystem, and who has the time to discuss analytical model diagnostics, misclassification rates, ROC plots, lift curves, and that silly confusion matrix…”

In any case, you need to be spending a lot of money on marketing, and making a lot more selling products and services, before these models cross your mind.

Work on your organization, goals, and simpler/actionable models first…

Attribution is an Organizational Challenge

The more complex and large your organization is, the more difficult the problem of attribution becomes – from a management standpoint and usually a technical one as well. But even at the beginning levels of measuring, attributing, and optimizing your metrics, attribution is still not an easily understood concept.

And getting the company on board and aligned is the first step to success in this area (and any other…).

As Joanna Lord, ‎Chief Marketing Officer at ClassPass, wrote:

Joanna Lord:

“Attribution isn’t just about models or the latest approaches to funnel mapping. Attribution is about being able to assign the right value or credit to each marketing touchpoint. It’s no bigger than that. Let’s not get caught up fighting paper tigers on this one.

The real barrier to doing it well comes in this assumption that the CMO can do it well alone and in a silo. It takes development resources, financial modeling collaboration, and product leadership support. It takes a company-wide familiarity with all the touchpoints available to us and how/when to use them. This is what I call “getting intimate with your customer’s experience.”

If my product team builds a new on-site acquisition and onboarding flow, with little consideration to how that might affect my existing attribution tracking, and all of a sudden our LTVs drop like whoa, it could go unnoticed for a year…unless we are doing this together.

Attribution is not a marketing problem, it’s a company problem.”

This point is complementary to the one I made at the beginning of the article. You need to define what your goals are – what will you do in the event of [X] data?

You need to get the company on board.

Finally, you and your boss need to be able to trust the numbers that your model spits back to you…

Can You Actually Trust Your Numbers?

At its core, the purpose of attribution is not to help a marketing executive feel good, but rather to inform decision making. So the most important thing is that you can trust the numbers you’re getting back.

If, for instance, you get back results that say social advertising isn’t performing well, will you throw up your hands and say, “I don’t believe it!” Or will you take action?

fzfh41iuciy-paul-bergmeir

Mistrust in attribution tends to fall on one of two sides:

  • On one side, people don’t trust last click attribution because it’s easily understandable and widely recognized as bad.
  • On the other side, people don’t trust expensive “black-box” solutions that use complicated algorithms, because, well, they don’t know what the hell is going on inside.

There’s a whole grey area in-between these two dichotomies, by the way, in which marketing and analytics teams should refine measurement, define attribution models, and begin trusting their numbers at a level where it makes it possible to make organization-level decisions.

But it begins with a conversation – with your team, boss, any stakeholders. What can you agree on to reach an actionable model, one on which you can inform decisions?

Conclusion

Marketing attribution is hard.

This post explored many models, from the basics that are default in GA, to Markov Models and the relative blackbox/robust model that premium Google Analytics and Attribution 360 use.

While it can be daunting, especially when faced with the prospect of having to use sophisticated statistical models in R to build out accurate models, remember this: attribution uses past information to help you make future optimization decisions.

For some businesses, really nailing last click attribution might help inform future decisions. For large multi-channel brands that have a large media spend – well, they might need a sophisticated data-driven model.

It’s all relative to your goals, abilities, resources, and achievable ROI. Use this as another tool in your analytical/optimization arsenal.

Note: we launched a course in ConversionXL Institute taught by Kelly Gray, Program Manager – Attribution, at Google. It goes deeper into the topics brought up in this article. Send us a message if you want to learn more about it, and the Institute in general.

Related Posts

Join 95,000+ analysts, optimizers, digital marketers, and UX practitioners on our mailing list

Emails once or twice a week on growth and optimization.

  • This field is for validation purposes and should be left unchanged.