Internal Promotions vs Third-Party Banner Advertisements [Original Research]

Internal Promotions vs Third-Party Banner Advertisements [Original Research]

We were asked recently about the effects of using internal promotions (e.g., a discounted product sold within the site) vs. third-party (from an outside business) banner advertising on web site clarity and visitor perceptions.

Our first study used the five-second test to examine whether ads on website homepages distract visitors from understanding a site’s purpose. This follow-up study looks for differences in user perceptions between ad types: internal promotions versus third-party ads.

Results summary

The five-second tests yielded some interesting results:

  • Respondents recalled seeing third-party ads more often than internal promotions.
  • The webpage with internal promotions was rated as higher quality and more attractive than the web page with third-party ads.

The effects seen here are limited to the treatments used. We’re looking soon to expand this test across hundreds of ads to normalize for ad type and presentation.

Background

If a website homepage is cluttered with irrelevant information, first-time visitors may become distracted or overwhelmed. More time is spent trying to simply figure out the purpose of a site and, if the page is confusing enough, visitors will abandon a website when this task becomes too difficult. This issue can lead to short and long-term decreases in sales.

While we can now safely say that third-party advertisements are considered “clutter”, we wonder whether internal promotions are also seen as advertisements.

Internal promotions, like a storewide 30% discount, are advertisements for the website the visitor is already on. They don’t intend to take users to a different site, but to encourage their current exploration.

However, internal promotions often look identical to third-party ads. Our question is this: Do users distinguish between internal promotions and third-party ads, or do they look at internal promotions and see an ad?

Study Report

Data Collection Methods and Operations:

We used the homepage of Colonial Candle as we thought it provided the perfect test subject in terms of not being well-known to a broad audience and with a relatively vague homepage.

The thought here was that we didn’t want it to be too easy to gather what the website was selling with just 5 seconds to view the page.

Here are the two variations tested where you can see the addition of large, easily seen banner ads in the right sidebars:

Variation with 2 ‘internal’ ads, promoting a product within the site.
Variation with 2 ‘external’ ads, promoting services or products external to the website.

Participants – Using UsabilityHub.com, we ran five-second tests on 800 participants (400/variation).

We asked three questions:

  1. What is this website selling?
  2. How many advertisements did you see?
  3. Rate the quality and attractiveness of this page 1 (worst) and 5 (best).

These questions were intended to gauge the difference in perceptions of third-party ads versus internal promotions, and to see if either had an effect on perceptions of the entire web page.

Findings

1. “What is this website selling?”

A Chi-square goodness of fit test told us that there was no difference between variations in the percentage of people able to identify ‘candles’ as the primary product class being sold by the website: 59% for the internal promotion variation and 60% for the third-party ad variation.

2. “How many advertisements did you see?”

We compared mean responses between the variations using a t-test:

number of ads results
Visualization of the respondent data for the survey question, “How many advertisements did you see?”

The two third-party ads increased the number of ads that respondents recalled seeing after the 5-second test.

The results for this survey question reveal that participants considered the internal promotions to be advertisements. Out of the two internal website promotions displayed, participants reported seeing an average of 1.76 ads.

3. “Rate the quality and attractiveness of this page 1 (worst) and 5 (best).”

We compared mean response between the variations using a t-test:

quality_results
Visualization of the respondent data for the survey question, “Rate the quality and attractiveness of this page 1 (worst) and 5 (best)”

Participants who saw the homepage with third-party ads found the page to be less attractive than the homepage with internal promotions.

Limitations

This test is very limited in its ability to extrapolate findings, as we tested just two ads on one website. Survey question three on the quality and attractiveness of the web page would surely fluctuate with different websites and ads.

Maybe people really don’t like American Express. Perhaps they found the third-party ads unattractive or tacky. There’s a possibility that different ads —or different products being advertised— would elicit different results.

We’re looking forward to community ideas on how we can develop this study for a follow-up.

While the majority of participants considered internal promotions to be advertisements, it’s possible that they answered this way simply because the survey question asked about advertisements. If the question had asked about promotions, perhaps participants would’ve responded that they saw two promotions.

Conclusion

While there could be many confounding variables, we found that, in this case, people recalled viewing more ads when they were third party vs. internal promotions. However, they rated the variations with internal promotions as more attractive.

Join 95,000+ analysts, optimizers, digital marketers, and UX practitioners on our list

Emails once or twice a week on growth and optimization.

  • This field is for validation purposes and should be left unchanged.

Join the Conversation Add Your Comment

  1. I think another limitation is not only the vernacular variability between “promotions” and “advertisements”, but the bias inherent in how question #2 was asked.

    When you ask a respondent how many ads they remember seeing, their biased to assume that there were indeed ads on the page, but you’re just asking how many there were.

    It’s entirely plausible that they initially assumed the internal promotions weren’t ads at all, but when subtly “convinced” after the fact that there were ads (but how many were there is the question), they immediately recall the two boxes …. “ahh, those must have been the ads”.

    I’d be very interested to see the results if question #2 was something like, “Do you recall seeing any ads on the page?”.

    Good stuff though … keep on truckin’

    Reply
    1. Ben Labay

      Hey Isac, good point. Your question is indeed better but would have to be tweaked to say “…, if so how many?” so we could get the data we were after. That said, the only defense of the original phrasing would be that the bias was uniform across participants, therefore any differences in numbers couldn’t be attributed to some having that bias and others not. Thanks for the comment! Ben

  2. Interesting post Ben, thanks for sharing the useful research/data!

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.