How Users Read the NYTimes Website – 2004 vs. 2016 [Original Research]

How Users Read the NYTimes Website – 2004 vs. 2016 [Original Research]

We found an interesting, and rather old, eye-tracking study from 2004 and decided to try to replicate a part of it to see how it works today.

This study, conducted through CXL Institute, involved eye-tracking a couple homepages of the New York Times, one from this year, 2016, and one from 2004. Our primary goal wasn’t the comparison to the old study, rather it was to see what were the ‘priority viewing areas’ for how people process a news site and to see if ‘today’s users’ process the contemporary design differently than one from more than a decade ago.

Results summary

  • The large banner ads in the 2016 variation made users jump around the page, causing way more variability in how people read compared to the 2004 version.
    • Study participants started staring at the ad in the 2016 version, but quickly went elsewhere, although where they went was much more variable compared to the ‘seen path’ of the 2004 design.
  • The priority viewing areas did not differ much at all between the two version of the New York Times website, and also agreed with the 2004 study.
  • Given the designs tested, the regions that have more immediate prominence were the center and upper left of the page.

 

How do I apply this research?

This study was primarily exploratory, as we have follow-up plans looking at reading patterns, however, it does provide us an example of viewer perception path variability.

Website owners usually want visitors to quickly process a homepage, get the point, and move further down the sales funnel….whatever that might look like. So if one design over another causes viewers to process things in a more variable way, this can have consequences on visitor flow through a site.

So not only do these results show banner blindness, but that banner blindness has consequences on collective user focus and perception.

 

Background

So the original study that this little test was inspired by was performed by a seemingly out of business research branch of Poynter.org. This study doesn’t necessarily reproduce any part of the old study explicitly, rather, we take their approach and see how it works today.

Study Report

Data Collection Methods and Operations:

Here is the ‘priority viewing area’ aggregate map of 5 news sites analyzed in the 2004 eyetracking study by Eyetrack III. This study only considered user viewing patterns of the news site http://www.nytimes.com/.

 

Priority viewing areas of news websites, from a 2004 Eyetrack III study
Priority viewing areas of news websites, from a 2004 Eyetrack III study

Our ‘priority viewing area’ grid had to adjust according to the precision of our eye-tracking tool. We used a grid of 3X5 rather than 4X4.

The 2004 version of the New York Times homepage was obtained through the internet archive ‘way back machine’.

Participants

The eye-tracking survey was completed by 200 participants, though only 132 resulted in data with enough accuracy to be useful, 68 for the 2016 version and 64 for the 2004 version.

Findings

Heatmaps of the New York Times online homepages.
Heatmap for the two versions. 2016 on left and 2004 on right.
seen order comparison
Priority viewing areas and seen order for the New York Times homepage in 2016 (left) and 2004 (right).

Conclusion

The large banner ads in the 2016 variation made users jump around the page, causing way more variability in what people read compared to the 2004 version. Study participants started staring at the ad in the 2016 version, but quickly went elsewhere, although where they went was much more variable compared to the ‘seen path’ of the 2004 design.

The priority viewing areas did not differ much at all between the two version of the New York Times website, and also generally agreed with viewing patterns from the 2004 study.  Given the designs tested, the regions that have more immediate prominence were the center and upper left of the page. The variation in the 2016 design was seemingly a result of the banner advertisement from IBM, which caused viewers to essentially skip around it to find and fixate on text content.

Join 95,000+ analysts, optimizers, digital marketers, and UX practitioners on our list

Emails once or twice a week on growth and optimization.

  • This field is for validation purposes and should be left unchanged.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.