New medium, new needs?
Editor’s note: Rachel Swanson is director of consumer research and insights at Condé Nast, New York. Scott McDonald is senior vice president, research and insights at Condé Nast.
Without question, the iPad and other touchscreen tablet and smart mobile devices have rapidly changed the consumer tech landscape. These new market entries are impacting how we communicate, spend downtime and consume media. For the publishing industry, the iPad and other devices like it offer an opportunity to experiment with delivering highly-produced content in a new way, seemingly unbounded by the confines of ink and paper.
However, as with any new product launch, understanding both physical use examples and the mental modes of users is critical to assess their needs and deliver a desirable and profitable product. The common research inquiries still apply: Who are the users? What are their expectations? Do they like the product? But research professionals may wonder if the shift in consumer landscape changes the way we ask these questions.
Based on our multifaceted exploration of iPad usage among early adopters we recommend that researchers consider a new five-point checklist, which we will explore in this article, when studying consumers within this environment.
Measurement was underdeveloped
Condé Nast was the first magazine publisher to bring to market iPad digital magazine “replicas” of the monthly print issue via applications available on Apple’s iTunes store. At the time the iPad launched, app measurement was underdeveloped and inadequate for determining the success of our digital publications - making primary research necessary. Our study included both qualitative and quantitative work to generate insight on the audience reading Condé Nast digital magazines available on the iPad; to understand how users are incorporating the device and available content into their lifestyle; and to uncover perceptions of the editorial content and advertising in this new environment. Four Condé Nast digital magazines were studied over a series of months immediately post-device launch: GQ, Vanity Fair, Wired and Glamour.
The qualitative research phase ran from May through August 2010 and comprised 70 in-depth interviews (IDIs) with iPad owners in New York City, Los Angeles and Boston who had some previous familiarity with the magazine titles being studied. Interview participants were instructed to download various magazine apps on their devices prior to coming to their interview session. In their one-on-one chats, users were led through usability tasks, probed on their media consumption habits and asked to compare and contrast articles, photo spreads and advertising on the iPad versus the relevant print edition.
For the quantitative phase, an in-app survey was included in the digital issues of the aforementioned titles over several months during the second half of 2010. A creative ad unit conforming to each title’s branding style and tone was developed and inserted at various positions in the digital magazine experience - to be perceived by consumers as a “page” of the magazine. Embedded within the ad unit was a redirect link to a 10-question Web-based survey. Tapping the creative ad unit called this survey into the magazine app’s in-app browser, providing users with a seamless experience of getting from edit and advertising content to the survey. Questions included basic user demographics, relationship to the print publication being evaluated and satisfaction with the digital issue experience. After answering the survey, users returned to the same place in the issue. Over 6,000 responses were collected.
Encountered some challenges
As a result of using both qualitative and quantitative techniques over a series of months, our research findings were rich and able to direct various internal stakeholders on how to continue optimizing our digital magazine products. However, we encountered some challenges during execution which may be worth considering when determining your own company’s in-tablet research approach.
1. Don’t miss out on an opportunity to determine the audience.
It would have been easy for Condé Nast to assume that anyone reading our publications on the iPad fit the same demographic profile as a print reader and subsequently omit these questions from our in-app survey. But research indicated this is not the case. Our survey data showed that Wired’s iPad reader gender skew, age and income was quite similar to the print audience. However for GQ, Vanity Fair and Glamour, iPad readers tend to be more male and more affluent than the print reader base.
Similarly, in the IDI recruit, we asked demographics without using those questions as screening criteria for qualified recruits in order to get a clean read on the early adopter iPad audience for each brand. Keeping an open recruit and having survey data which matched the profile of participants we’d seen from “behind the mirror” during the interview gave our team more confidence in the audience assessment of who reads each digital title. These data are now considered the benchmark in lieu of underdeveloped syndicated solutions.
2. Spend some time with your tech team to learn about your company’s app development process and their relationship with Apple.
Every company has a unique process and guidelines to follow when developing or working with apps before anything gets sent to Apple. Apple then reviews and quality-checks all app content and functionality in-depth prior to pushing apps live on the iTunes app store. In our case, it was critical that not only the creative ad unit invitation be finalized but the actual survey program be fully functional before beginning the Apple review process. The last thing our team wanted was for the survey to prevent the digital magazine from launching on time. If your company is interested in executing an in-app intercept of some sort, start out by asking what’s possible and what’s not. Also be clear on timelines for both internal and external approval processes, especially if you have an “app live date” in mind. Armed with this information, you’ll be more effective in selecting a research vendor who can provide the right solution.
3. Consider any in-app intercepting as a test of the methodology and set expectations accordingly.
The iPad is too new to make generalizations about consumer responsiveness to survey research, so both internal and client stakeholders should be aware that benchmarks or standards don’t quite exist. Since the Condé Nast digital magazine apps do not use an ad server and push notifications were not approved for a survey invitation, our interception technique of hard-coding the invite as a page of the digital magazine was the only option - albeit one that had never been executed before. Thus, we were careful to avoid providing stakeholders with a guarantee of number of completes expected. As it turned out, survey completion rates across three of the four magazines were tested were comparable, thus giving us a baseline range of expectation for number of respondents as a function of average monthly app downloads.
This completion rate is better than what we sometimes see from our print subscriber databases and Web site intercepts but not as strong as on our opt-in panel lists. Time will tell whether this benchmark holds.
4. Don’t assume a mobile survey solution works best on the iPad.
As found in our IDIs, consumers tend to approach accessing content on the iPad in a “lean-back” fashion. That is, the iPad’s larger size allows for more considered engagement with what’s being presented on-screen than on a phone’s smaller screen. When requesting RFPs for our in-app survey, most research vendors were adamant about using their “mobile” survey platform and drastically limiting the number of questions we could ask (usually to around 10). This was especially recommended as our magazine apps on the iPhone also included the survey invitation. During field, we found tap-through to our survey on the iPhone to be virtually non-existent and behavioral metrics for time spent per issue on the iPad during the survey period to average two to three times higher than time spent on the iPhone. This indicated that it was unlikely digital magazine readers on the iPad would have found fault with a slightly longer survey after all. While mobile-specific survey solutions might be buzzworthy and appropriate in some instances, make sure to work with a research vendor that offers great survey software and user experience regardless of access device.
5. Invite research participants to be a part of your ongoing app development and optimization.
Including a respondent information form with an opt-in to be contacted for more research on the iPad is critical. Since Apple does not currently share transaction data or device identification numbers on who purchases Condé Nast publications’ digital issues, we are unable to match readers of a digital magazine on the iPad to their subscription of the same magazine in our traditional consumer database. It’s something the industry’s working on furiously but in the meantime including an opt-in for re-contact is a win-win: It allows us to manually update our database files with digital purchasers and execute marketing plans and it makes users feel like they’re part of the development process, helping to define the new media landscape. Having brand advocates weigh in on product extensions ensures you won’t alienate the core, and collecting this opt-in within the new environment adds a layer of relevance to your sample when studying tech products in the future.
A good foundation
The tablet’s place in the lives of consumers is still in its infancy. Companies like Condé Nast who are studying this fascinating time are staying flexible and learning as they go, not only about the consumers and their relationship to the device but about how the research industry may change as a result. These guidelines we’ve compiled as a by-product of achieving our learning goals will hopefully provide others with a good foundation from which to start testing and developing new methodologies in this rapidly evolving time as well.