Should Your Opt-in Offer Be Video, PDF or Web Content? [Original Research]

Should Your Opt-in Offer Be Video, PDF or Web Content? [Original Research]

Have you ever wondered which type of content, say a ‘pdf’, web content (a web page) or video is best to use as a lead magnet on your landing pages?

We conducted an experiment to explore this.  The idea is that by testing offer prototypes for opt-in probability before the campaign launch marketers can gain insight as to what media type to use.

Does the medium of your lead magnet affect email opt-in rate conversions?

To answer that question the CXL Institute research team set the goal of developing a testing strategy to help marketers test to see if different content formats affect opt-in rates.

But how do you measure or know if any given media type is the best fit for any said product? A guiding principle to keep in mind while exploring this concept with us is that when all is said and done, product satisfaction and preference is contextual.

While we cannot guarantee a one-size fits all answer to this problem, the results presented in this study are intended to suggest ways teams can test their content early on for marketing optimization.

Study Design: Data Collection Methods and Operations

Hypothesis

We hypothesized that if we compare the probability of users opting-in across content marketing formats then we can make an informed decision about which media type is best-suited to release.

Scenario & Methodology

To keep the focus general, we chose a scenario that asked survey participants to imagine they were looking for more information on email marketing and were then directed to a landing page offer for “Email Marketing 101.” Of all the various content marketing formats that exist we chose to test the following three: a pdf, a website, and video.

After exposing participants to the given scenario, they were then directed to a single variation of the landing page of product offers (seen below) and asked to select the probability they would submit their email address using an 11-point Juster Scale. Juster scales are used to measure purchase probability.  In our case, we are using email opt-ins as a form of purchase behavior.

(Variations tested)

Analysis

To achieve statistical significance in our results we tested a total of 1206 participants (n=1206). The distribution of participants, randomly assigned, were 419 for Video, 398 for Website, and 390 for PDF respectively.

The semantic responses captured in the survey were translated into probability percentages according to the Juster Scale and were then transformed to allow for an Analysis of Variance (ANOVA).

Looking at the early percentages for the probability a visitor would opt-in for the email marketing product offer would be misleading if not tested for statistical significance.

Result: web content and pdf were more attractive offers than video content. PDFs content was 34% more probable to convert than video and regular web content (content served on a web page) was 24% more probable to convert than video.

The numbers were then crunched and the following graph (intended for visualization purposes only) illustrates the effect of the ANOVA results

  • We are 100% confident that the means are different with Video across the three variables. Therefore, Video is less probable to convert than web content or PDF.
  • There is no significant difference for opt-in between pdf and website as they fall within the same normal distribution.
    • This is a great place where qualitative information can supplement teams for further evaluation.

Conclusions we can draw from this type of study is if media type is a determining factor for conversions or not. In our particular study, we can derive from the data that video is not the optimal method of product delivery. As we saw with Website and PDF, it is not the media type that will affect conversion since they fall within the same normal distribution.

Limitations

The intent of the study is to help provide readers a framework in which they can inform their own content marketing strategies. Things still left to consider in our tested scenario of training for an Email Marketing 101 offer is frequency of use, opportunity costs, actuals of true opt-in conversion rates to benchmark against and so on.

Not all media types were tested. Typical content users find behind an email gate can be but are not limited to the following: e-books, videos, websites, coupon codes or discounts, pdfs, trials, podcasts, templates and other downloadable assets. To keep a focus on the study we chose to test three media types as our dependent variables: a website, pdf, and video in the scenario of a user learning more about the topic of email marketing.

Probability percentages do not reflect expected conversion rates. It is a known bias that study participants over report intended purchase behavior. The idea of measuring for probability in this study is to give indicators about future opt-in behavior.

Conclusions

In our study design, the data showed that

  • PDFs are 34% more probable to convert than video.
  • Websites are 24% more probable to convert than video.
  • No difference between PDF and web content.

Join 95,000+ analysts, optimizers, digital marketers, and UX practitioners on our mailing list

Emails once or twice a week on growth and optimization.

  • This field is for validation purposes and should be left unchanged.

Join the Conversation Add Your Comment

  1. This really surprises me with the dominance of video on the Internet. Very good to know, but I was almost sure that PDF (ebook) was old school and not effective. Very interesting.

    Reply
    1. Diego Chacon

      Hey Mike,
      The internet is a complex, interesting space. Video has its place out there too.
      Thank you for reading,
      Diego

  2. While web site visitors are more likely to follow a link to a pdf or web site than a video, it doesn’t mean potential customers follow that same behavior.

    As “useful” content has become a primary component of any modern marketing effort, readers are hit with a deluge of useful white papers, PDF reports like the one used here.

    So many in fact that for the past few years I routinely look for them while reading marketing trade publications, email newsletters, ads, etc. And I download them by the boatload.

    I download so many of them that I rarely read them all.
    While my information hoarding isn’t necessarily mainstream behavior, I suspect that there are enough readers like me to skew your survey results.

    The bigger question is which vehicle most appeals to potential customers actually looking for a solution TODAY?

    In a purchasing decision tree, is the pdf more or less effective in moving a potential buyer down the marketing funnel, or is it more likely to be a dead end?

    A potential buyer browsing deeper into your web site, or viewing a video may be much more engaged in both the decision process and with your offering.
    The buyer who browses solutions for 30 minutes, disengages from the subject, and comes back to it after lunch and starts with your pdf is in essence, starting over, and most of the good will you attempted to develop before he (or she) left your site has evaporated.

    I would like to see a study like this which follows potential customers through to an eventual purchase, at my sight or a competitor.

    Knowing which info format most satisfies the instant need of my web visitors doesn’t tell me what behavior is followed by actual buyers

    Reply
    1. Diego Chacon

      Hi Dave,

      You and I share the same information hoarding tendency with pdfs. With that said, the rate at which we opt-in is in line with the study results which found that visitors are more probable to opt-in if the offer is pdf.

      The results in this study are unable to your question about which media type would have the highest impact in a purchase decision tree as that was not measured. This study illustrates a method that detects the probability visitors would opt-in in relation to different media types offered. A study analyzing a purchase decision tree for your site or a competitor is something our agency is able to help you with.

      Thank you for reading,
      Diego

  3. Very interesting and not surprising.

    Like the comment above, I would love to take this research one step further and find out what percentage of people consumed each piece of content.

    If your goal is branding these stats would be very interesting.

    Reply
    1. Diego Chacon

      Hey Dave,
      Agreed. Information on customer next-steps is very enticing. Figuring that out is something we can do. Let us know if you need help there.
      Thank you for reading,
      Diego

  4. I was really surprised that video was so low.

    To be honest, I think the topic in question “people looking for information on email marketing” influences this study and skews it.

    The fact that the topic is about writing emails would lead to a visual text reading and scanning format being more appropriate.

    What if the subject was “looking for information on losing weight – Weight loss 101” or on “how to run faster”. Would a pdf or web text, on running or losing weight be better than a video format? Not sure.

    I would love for you to test it and let us know.

    Point is, do this test on a more neutral topic and then let us see the results.
    Or do 3 tests one with a more movement and visual related topic, and see if pdf’s, and websites still dominate?

    Let us know.

    Reply
    1. Diego Chacon

      Hi Jason,

      The results of the study suggest that video is more likely to yield a higher opt-in rate over pdf and website in the case of a user looking for more information on email marketing. The results could indeed be different dependent on whichever type of content marketing strategy is being tested. In some cases, a video may very well be the winner. The scenarios you’ve introduced, weight-loss programs and running regimens, are part of the health and fitness industry with users who may need a different media type in order to achieve their goals and expectations with any given product. The upside is that you can test for that. I like your idea of comparing this test to 3 or more scenarios to see what differences (if any) can be detected and will add that to our idea bank here.

      Thank you for reading,
      Diego

  5. Really this is an Excellent blog thank you so much for sharing it.

    Reply
    1. Diego Chacon

      Hey Ashwin,
      Always looking for ways to push our content here and appreciate you reading.
      Thank you!
      Diego

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.