Mozilla · PERFORMANCE
Can we use messaging to improve perceptions of Firefox?
As a contracted researcher at Mozilla, I was the lead in benchmarking perceived performance (user perceptions of speed and responsiveness) of Firefox. Seeing that UI and technical changes were not having a large effect on user perceived performance, we wondered how brand messaging is related to perceived performance and whether we could utilize it to improve perceived performance.
BY THE NUMBERS
- 1,500users in a remote experiment
- 6control or experimental article combinations
- 4counterbalanced videos rated per user
- DATE
- Summer 2017
- ORGANIZATION
- Mozilla, Firefox Team
- ROLE
- ResearchContent Design
- DELIVERABLES
- LINKS
- Paper (pdf)
Research Process
- 01
Benchmarking Firefox Perceived Performance
We designed and worked with an external vendor to execute in-lab experiments and surveys to benchmark user perceived performance of Firefox so that we could quantify the effects of technical improvements and UI changes. Performance improvements were not what we had expected, so we wondered whether performance messaging, like we had seen used by Google, would have a greater effect.
- 02
Unmoderated Online Experiment
We designed an online experiment where 1,500 users were divided into branches and read one of 6 control or experimental article combinations (tech/non-tech source X control/UI/performance content). They were then shown 4 counterbalanced videos of Chrome and Firefox opening windows and tabs and rated the interactions on speed, smoothness, and design.
- 03
Sharing Results with Marketing & External Community
After analyzing the data, we created a deck of our findings and worked with marketing to put our results to work in a national ad campaign. We also wrote a paper sharing our findings with the external research community, as we saw this as an opportunity for smaller companies to compete against large companies with an established brand.
Key Insights & Opportunities
- 01
Messaging about performance affects performance
Media stories matter and can influence users’ perception of performance. Theme of the story is also important. Engaging with the media to present Firefox as a more performant browser can influence users’ beliefs about the performance of Firefox.
- 02
The message source isn't important
The source of the story does not appear to affect the effect of messaging, so it’s just as important to engage with non-technology media. For the Photon launch, we should invite media to test drive the new Firefox.
- 03
We need to keep studying perceived performance
Performance (and its perception in general) is an important factor in users' overall assessment of a browser. In addition to continuing to study media influence on perception of performance, an on-going, full time user research program to study and improve the perceived performance of Firefox is essential.
Marketing Content Created as a Result