Which Types of Social Proof Work Best? [Original Research]

Which Types of Social Proof Work Best? [Original Research]

When shopping online, you can’t hold the product, test it out, or talk to a salesperson about how different brands compare to one another. For these scenarios, social proof is frequently used to guide shoppers towards the best product choice.

Which brings us to the real question: Which social proof techniques are most effective? Are some of them totally ineffective?

This study from CXL Institute explores how different forms of social proof are perceived (with eye-tracking), and then how they are recalled (with post-task survey questions).

Results summary

Perception of Social Proof Variations (Eye-Tracking)

  • The different social proof types didn’t differ significantly in how quickly they attracted user’s attention (average of 8.3 seconds to first fixation).
  • There were differences among in how long people paid attention to different types of social proofs. The differences were driven by the three social proof versions with text (the two testimonials and the press mentions with short quotes) that resulted in the longest mean times fixating (average of 1.06 seconds). This was likely due to the large blocks of text causing increased cognitive processing (longer time to think about).

Short-Term Memory (Survey Questions)

  • Out of the 8 treatments (plus control), viewers were significantly more likely to remember high profile client logos (not low profile), testimonials with photos (not testimonials without photos) and press mentions. Here are a few takeaways:
    • People recalled high profile client logos more than low profile logos (p-value = 0.009).
    • Press mentions with featured logos resulted in moderately higher recall rates than press mentions including logos and short quotes (close to significance at alpha of 0.05; p-value = 0.0953).
    • Testimonials with photos were significantly more effective (p-value = 0.0035) at generating viewer recall than testimonials with no photos.
    • When it comes to recognizing variants among treatments, photos are memorable, logos and numbers are not.
    • In most treatments, participants were more or less guessing which variation they saw (they got it right half the time). The only memorable treatment (61% accuracy) was for testimonials with or without photos, probably because the photos in one treatment were highly memorable and made it easy to distinguish between the two.

How do I apply this research?

From the eye-tracking component of this study, we know that the testimonials, especially with photos, keep attention the longest. This information can help in different ways for this particular type of social proof:

  • If you have great testimonials, ideally from well-known authorities, use them. They’re your primary selling tool.
  • However, keep in mind that all testimonials draw in user attention, so if yours are mediocre, that may be all people will remember about your page.

Testing viewer recall rates provided great supplemental insights into how people perceived, then remembered, the social proof treatments:

  • High profile client logos, testimonials with photos, and press mentions are the most memorable. If you have good versions of these, use them. But use them wisely
    • Testimonials should have photos….always. More photos! It might be worth testing a reduction in the testimonial copy per photo as much as possible.
    • High profile client logos are likely the best social proof to have, as they balance high recall with low cognitive load. But they’re not easy to get. If you don’t have any, testimonials with photos might be worth testing over low profile logos.

Full Study Report: Which Social Proof Works Best?

Screen Shot 2016-04-05 at 4.22.48 PM

Study Setup:

This consisted of 2 study tasks:

Study Task 1. We used eye-tracking to quantify user perception of the web page. We manipulated web hosting site Host Gator so it contained different social proofs.

A total of eight different types of social proof were tested along with a control variation, which had no social proof.

Social proofs were consistently placed in the same location on the web page so that all proofs were equally visible (see red box on the right image).

Participants were given 15 seconds to browse the page after being briefed by a task prompt: 

“Imagine your friend has a new business and needs help finding a web hosting company. Please review the following web hosting web page as you normally would to see if you would recommend this service to your friend.”

Number of participants per treatment with usable eye-tracking data
Number of participants per treatment with usable eye-tracking data.

 

Social proof variations tested:

Pres Mens w Logos
Press mentions with logos only.

 

Press Mens w Short Quote
Press mentions with logos and short quotes.

 

Test w Phots
Testimonials with photos.

 

Test wo Phots
Testimonials without photos.

 

High Prof
High profile client logos.

 

Low Prof
Low profile client logos.

 

High Soc Follow
High social following.

 

Low Soc Follow
Low social following.

 

Cntrl, no proof
Control variation, no social proof.

The statistics we were primarily concerned with were time to first fixation (TTFF) and total time fixating.

Study Task 2. A 2-question survey determined whether each social proof got into the viewer’s short-term memory.

After viewing the web page, we asked users the following two questions:

  1. “Remembering the web page you were just on, were there any elements or parts that led to an increase in trust/value to you?”
  2. “Select which of the two webpages below you recall seeing:” (Here we showed two variations that were similar to each other e.g. high & low social following.)

If participants mentioned the social proof they saw in the first question or selected the correct treatment variation in the second, then that provided evidence of the social proof’s impression on the viewer’s short-term memory.

Findings

1. Eye-Tracking

Though heat maps are more common, we’re going to illustrate our findings with fixation dot maps.

Here’s a gif to show the difference between the dot map and heat map, where we can see that the dot map lets us see individual viewers fixating on the social proof area and the heat map doesn’t.

Gif of dotmap (left) and heatmap (right) showing user fixation
Gif of dot map (left) and heat map (right) showing user fixation.

And here are the treatment dot map figures, paired accordingly:

Website treatment with high profile client logos (left) and low profile client logos (right)
Website treatment with high profile client logos (left) and low profile client logos (right).

 

Website treatment with high social following (left) and low social following (right)
Website treatment with high social following (left) and low social following (right).

 

Website treatment with testimonial with photos (left) and without photos (right)
Website treatment with testimonial with photos (left) and without photos (right).

 

Website treatment with press mention logos (left) and logos with short quote (right)
Website treatment with press mention logos (left) and logos with short quotes (right).

As far as the stats are concerned, remember we’re interested primarily in two metrics, time to first fixation (TTFF) and total time fixating. This helps us understand 1) how quickly the social proof attracted attention and 2) how long it held attention.

Here are the raw summary stats for time to first fixation, for which an ANOVA yielded no significant differences between treatment groups F(8,191) = 0.6338, p = 0.7487.

Mean and standard deviation total time to first fixation
Mean and standard deviation total time to first fixation

Here are the raw summary stats for total time fixating, for which an ANOVA yielded significant differences between treatment groups F(8,191) = 2.7528, p = 0.0068.

This result was driven by the long time paid to the testimonial treatments, and was likely the effect of the large block of text in the testimonials causing increased cognitive processing, thus resulting in a longer time to process. Note that the three versions with text (the two testimonials and the press mentions with short quotes) are on top of the list with the longest mean times fixating.

Mean and standard deviation total time fixating per social proof treatment 
Mean and standard deviation total time fixating per social proof treatment.

2. Survey Questions

Q1: Were there any elements or parts that led to an increase in trust/value to you? 

We ran a Chi-squared goodness of fit test to find differences among all our treatments and the relevant pairs. Viewers were significantly [X2 (5, N = 1153) = 34.54, p < 0.001] more likely to remember high profile client logos (not low profile), testimonials with photos (not testimonials without) and press mentions.

Here are the stats, test results and individual Chi-square results for the related pairs of treatments, and the overall Chi-square test among all treatments reported below the table.

stats for comparing recall rates of the social proof treatment variations, with chi-square results
Stats for comparing recall rates of the social proof treatment variations, with Chi-square results.

A few specific takeaways:

  • High profile client logos did better than low profile at getting into people’s short-term memory.
  • High social following didn’t result in significantly more recall compared to low social following (p-value = 0.1177), but the p-value is low enough to suggest that there could be a difference.
  • Same for press mentions with logos only, it showed a moderate pattern for more recall that was close to significance (at alpha of 0.05; p-value = 0.0953).
  • Testimonials with photos social proof were significantly more effective (p-value = 0.0035) at generating viewer recall.

Q2: Select which of the two web pages below you recall seeing:

For this question, we didn’t want to initially perform an overall goodness-of-fit test (or otherwise) as the differences among the 4 main proof types (client logos, social following, testimonials, press mentions) could be driven by the apparent visual differences within them (e.g. the photo in the testimonial treatments).

Therefore, to test differences between mean recall rates for each paired treatment, we performed an N-1 two proportion test for each. No pair test resulted in significant differences between the recall rates. Here are the numbers:

Recall rates for each treatment when viewers were presented two variations to pick from after performing the task.
Recall rates for each treatment when viewers were presented two variations to pick from after performing the task.

The higher rates of recall for the testimonial treatments did result in a near significant difference among main proof types (p-value = 0.0661) when running a Chi-square goodness-of-fit test (see below). But as mentioned above, this is likely due to the noticeable difference the picture on the testimonial makes when comparing the two versions.

Recall rates for each main proof type and result of Chi-square goodness-of-fit test.
Recall rates for each main proof type and result of Chi-square goodness-of-fit test.

Takeaway: Photos are memorable, logos and numbers are not.

Limitations

  • There are tons of forms of social proof. We had to narrow it down to some of the most commonly seen types. Perhaps there are some less commonly used variations that are more effective than the ones we tested.
  • We only tested these social proofs on a service website, specifically a web hosting site. Results would vary on a different type of website where trust would be a bigger issue. There’s surely an interaction between the type of business/business goals and the type of social proof that makes sense considering this.
  • Our treatments were placed in a similar area, with the same blue bar background, but the height of the treatments varied for a couple, namely the testimonials, which required a bit more real estate. There is a known correlation that elements that take up more pixels draw more attention. However, the pattern of increased fixation time held with the press mentions with short quotes variation, and the recall patterns among all and among related pairs provided support and additional insights.
  • The second recall-related question was uninformative, most likely due to a poorly designed test of recall. Rates were little better than random overall, and were highest only when using a drastic change between the two variants presented. This effectively tested inclusion/exclusion of photos, rather than social proof effectiveness. If the social proof presentation in general were more pronounced, a larger part of the page, this question might have produced more useful insights.

Conclusion

Though there are an infinite amount of ways you could implement social proof, this study yielded some interesting insights.

First, it seems that testimonials that included text garnered more attention that simple logo social proofs.

Next, testimonials with photos social proof were significantly more effective at generating recalls. Similarly, high profile client logos were better than low profile logos at generating recall.

Finally, we found that, in general, photos are memorable, logos and numbers are not.

One big takeaway is that if you have great testimonials from known brands, use them. Social proof draws attention in any form, and if you can bolster your brand with these positive associations, it’s worth doing.

cxli-logo

Join the Conversation Add Your Comment

  1. That was an intense post. So essentially we need to integrate any form of social proof, but the forms themselves don’t differ that much?

    Reply
    1. Ben Labay

      Hey Tom, I admin this study is a but more combersome that most of our others. The main takeaways are in our ‘how to apply’ section up top, and that is that the high profile logos are the best all around…for attention and recall. After that, or rather if you don’t have high profile client logos, then testimonials with photos and a bit of text is best for balancing recall and attention.

  2. Hi Ben. This is a great practical study. Lately, I’ve been focused on the role of social drives in web design. For the testimonial and photo condition, did your numbers suggest the photo had any impact on the testimonial credibility? Or rather do you think they served to lure in people, in a way that boosts the odds that users read the testimonials? Or some other explanation?

    Reply
    1. Ben Labay

      Hey Brian, the data wasn’t collected in a way to answer anything about credibility, and we didn’t see any differences in how it ‘lures’ people….but admittedly we probably made a mistake using such a ‘busy’ site, our time to fixation numbers could have been less variable with a simpler design. However, we did see that recall increased with the photos, so that combo, with the excerpt, was more effective compared to the excerpt alone. And in terms of bang-for-buck (within the bounds of our data), the logos were the best when they were recognizable (low processing times and still memorable). Thanks for the question, and if you want to see any variations of this or have any other test ideas let me know. Cheers, Ben

Leave a Reply

Your email address will not be published. Required fields are marked *