Mozilla · PERFORMANCE

Can we use messaging to improve perceptions of Firefox?

As a contracted researcher at Mozilla, I was the lead in benchmarking perceived performance (user perceptions of speed and responsiveness) of Firefox. Seeing that UI and technical changes were not having a large effect on user perceived performance, we wondered how brand messaging is related to perceived performance and whether we could utilize it to improve perceived performance.

BY THE NUMBERS

  • 1,500users in a remote experiment
  • 6control or experimental article combinations
  • 4counterbalanced videos rated per user
3D Firefox browser icon on a rounded white square
DATE
Summer 2017
ORGANIZATION
Mozilla, Firefox Team
ROLE
ResearchContent Design
DELIVERABLES
LINKS
Paper (pdf)
PRESS
New ScientistPC GamerThe Register
Photo of a billboard advertising Google Chrome as "fast"
Even though no one had specifically studied the role of targeted messaging on user perception of speed and performance in web browsers, Google Chrome already seemed to be using these tactics in their advertising ecosystem.

Research Process

  1. 01

    Benchmarking Firefox Perceived Performance

    We designed and worked with an external vendor to execute in-lab experiments and surveys to benchmark user perceived performance of Firefox so that we could quantify the effects of technical improvements and UI changes. Performance improvements were not what we had expected, so we wondered whether performance messaging, like we had seen used by Google, would have a greater effect.

  2. 02

    Unmoderated Online Experiment

    We designed an online experiment where 1,500 users were divided into branches and read one of 6 control or experimental article combinations (tech/non-tech source X control/UI/performance content). They were then shown 4 counterbalanced videos of Chrome and Firefox opening windows and tabs and rated the interactions on speed, smoothness, and design.

  3. 03

    Sharing Results with Marketing & External Community

    After analyzing the data, we created a deck of our findings and worked with marketing to put our results to work in a national ad campaign. We also wrote a paper sharing our findings with the external research community, as we saw this as an opportunity for smaller companies to compete against large companies with an established brand.

Fabricated The Verge (tech source) article on Firefox Quantum performance presented to users in one experimental branch
Fabricated The Verge (tech source) article on Firefox Quantum performance presented to users in one experimental branch
Fabricated USA Today (non-tech source) article on Firefox Quantum performance presented to users in one experimental branch
Fabricated USA Today (non-tech source) article on Firefox Quantum performance presented to users in one experimental branch

Key Insights & Opportunities

  1. 01

    Messaging about performance affects performance

    Media stories matter and can influence users’ perception of performance. Theme of the story is also important. Engaging with the media to present Firefox as a more performant browser can influence users’ beliefs about the performance of Firefox.

  2. 02

    The message source isn't important

    The source of the story does not appear to affect the effect of messaging, so it’s just as important to engage with non-technology media. For the Photon launch, we should invite media to test drive the new Firefox.

  3. 03

    We need to keep studying perceived performance

    Performance (and its perception in general) is an important factor in users' overall assessment of a browser. In addition to continuing to study media influence on perception of performance, an on-going, full time user research program to study and improve the perceived performance of Firefox is essential.

Marketing Content Created as a Result

An ad saying Firefox is "ridiculously fast" showing a person with their hair being blown back and other user quotes about speed
A banner ad for Firefox saying "more fast more good" showing an excited person
Two ads with different color schemes saying "Firefox Fast for good."