One goal, many methods

Editor's note: Maria Petersen is vice president of customer intelligence at Dow Jones.

News media and print journalism have faced considerable challenges in recent years. The general decline in circulation has been hotly debated and a slew of solutions have been applied across titles to ensure that news media can continue its vital role in society. For some that means pay walls; for others we have seen a pivoting to video as a more engaging medium to deliver journalism. In the midst of this, The Wall Street Journal has become a fully-fledged membership business. However, as Shelly Seale of the International News Media Association has said, “The paywall models of metered and freemium all have one thing in common: Their access rules are determined by content, rather than customer data. This has led to a one-size-fits-all approach.”

At The Wall Street Journal we understood that as the business pivoted to a membership model, we needed to make our offering as dynamic and customer-centric as possible. We needed to better understand what customers really want from their media, in order to provide a world-class membership experience for our members and to deliver clear direction for product development – as well as future-proofing the brand. We were curious as to what the emergent needs and behaviors of our audience were and how these needs and behaviors interact with the content we provide. 

We therefore engaged the team at global research firm Kadence International to undertake a significant mixed-method research study aimed at providing the necessary intelligence to drive change and growth at The Wall Street Journal.

A thoroughly integrated approach

The research was designed as a thoroughly integrated project – not only in terms of qual and quant but also of different qual methodologies. The study integrated qualitative research throughout the process, including 60 seven-day diaries that were completed by customers to detail their behavior; depth telephone interviews to drill into motivations; extensive user experience labs; and three focus groups. An additional 94 people installed software on their phones so we could track their mobile behavior without influencing them directly. All this served to frame over 5,000 quantitative interviews.

Our approach was to use three core qualitative methods to triangulate behavior. The diaries gave us stated behavior but they had the weakness of not being comprehensive in isolation, as consumers of course select what they report. So we used the passive measurement of online behavior using the phone software to get a more holistic picture of consumer behavior. Finally, overlaying the passive measurement data with demographics provided a rich seam of data. In a sense, given the numbers who took part in the passive measurement phase it evolved into a quasi-quantitative methodology (and had originally been envisaged as such). However, to derive real value from the data collected, we had to interpret it and make sense of it using qualitative frameworks. 

The passive measurement data told us what people had done, comprehensively, but it didn’t give their opinions on why they’d done it or the emotional context. So the third step to complete our triangulation was using the UX labs to observe – seeing how people flipped through the newspaper, how they interacted with the app and how they engaged with our content. 

Comprehensive picture

By understanding the strengths and weaknesses of those three approaches we were able to paint a really comprehensive, holistic picture. We still elected to run focus groups at the end of the project, using these as a sandbox to explore hypotheses, push boundaries and test more left-field ideas. Even though the groups were at the end of the project they provided significant insight to help us hone our ideas – and most importantly, validated what we’d learned to date.

None of these techniques would have been enough in isolation but each gave us varying degrees of context, reported or actual behavior and emotionally motivated insights. The research identified three key reading modes: priority, discovery and exploratory. We found that customers transition between these three modes, depending on the time of day and their news needs. Understanding these modes and their consequent actions helped us pinpoint where we could enhance specific features to improve the overall user experience.

Aligned experience

The results have provided us with a unique understanding of: our customers; the implications of using multiple devices in news media; how appetites for content are changing; and how we can adapt content to deliver better value in future. The insights we gathered helped us create a familiar, aligned experience across all our products. We have been able to build new tools based on our findings, drive emotional connection with our brand and attract new audiences.

For example, the findings from the project form the cornerstone of our digital product strategy. The first product to benefit has been the WSJ mobile app, which was completely redesigned using the findings from the research and the resulting understanding of the different reading modes the app needed to cater to. Through the research, we’d also come to understand the role the products played within the larger product portfolio, so we also brought this aspect to bear on the redesign. We worked with the product, design and engineering team on translating the key principles identified in the research into practical steps for the redesign. We also rethought the definition of engagement, broadening the scope.

The product, design and engineering team adopted an “objectives and key results” approach to defining and measuring success, so we worked with that as well. As a result we’ve seen performance improve across all the aspects we were targeting. For example, the app is more relevant – more members use it. The app is habit-building – people use it more frequently. The app facilitates learning and discovery – when our members are using it, they go deeper and broader within our portfolio, engaging with more pieces of content and for longer. In fact, engagement with the app has increased fourfold and the user base has seen double-digit growth percentages.

Attracting new audiences

Going forward, The Wall Street Journal will follow six core principles based on the study, which work cross-platform, to guide its product and content development. These principles focus on: preserving and strengthening our product ecosystem; designing for our different reading modes; driving membership across all we do; elevating our insight and analysis; applying rich content where appropriate; and delivering content with real utility for our audiences.

Having started with our mobile app, we – alongside our colleagues in marketing, product and the newsroom – have now turned our attention to the other products in the portfolio. The research has also served as a springboard for a whole host of other projects. By aligning the business around universal principles, it’s generated an appetite for more focused research to help with tactical developments across our product suite. 

For The Wall Street Journal, the study is a strong example of how established media organizations can revitalize their offerings by putting the needs of their customers at the heart of product strategy. It was only by using an extensive and rigorous combination of qualitative and quantitative research that we could do this – enabling us to both have evidence to convince internal stakeholders of the right direction and to develop the right strategy for success. 

Wonderful to see 

It has been wonderful to see how readily the different teams across The Wall Street Journal have taken the research to heart, adopting it to use in their day-to-day jobs. The findings underpin the digital product strategy and roadmaps and the audience insights are alive and kicking in the newsrooms, informing thinking on content, storytelling and new ways of connecting our members with our journalists. 

Aspects of the methodology, the multi-disciplinary approach, the rigor of the research, and the collaborative manner of working both with our research partner Kadence International and with our colleagues internally at Dow Jones, is something we in the customer intelligence team see as the north star for all the other research projects in the future.