Editor's note: Tim Macer is managing director and Sheila Wilson is an associate at meaning ltd., the U.K.-based research software consultancy that carried out the study on which this article is based. The FocusVision 2014 Annual MR Technology Report can be downloaded by visiting meaning.uk.com/technology-survey.
Now in its eleventh year, the annual marketing research technology survey has a new sponsor, FocusVision, though it is still carried out by meaning ltd, so it draws on the previous decade for the tracking data it presents, while continuing to explore new and topical areas of interest on the technological side of marketing research.
Among the trends revealed this year are that research worldwide now embraces more different interviewing channels than ever before and some of the more high-tech – but until recently, lower-volume – modes are now on the rise.
The influence of mobile is also starting to affect most firms. We made this one of the three topics we explored more deeply 2014, along with how firms are coping with rising demands for more data visualization and dashboards on the delivery side and another intriguing area – the rise of shadow IT and, more generally, IT that research companies find they are using, rather than planned to use.
Fieldwork was carried out using FocusVision’s Decipher platform in December 2014 and January 2015, among 230 research companies worldwide. It is a 15-minute online survey that speaks to those responsible for or influential in technology decisions within research companies. For 2014 it was again translated into French, German, Chinese and Japanese.
We are most grateful to our sponsor, FocusVision, for their generous financial and technical support and to all of our survey participants, who have also been generous with their time and candor.
Research modes offered
One of our key trackers follows the interview technologies or modes that research companies are utilizing. Figure 1 clearly shows how the sands are shifting – the less high-tech modes, paper and CATI, are on the wane, whereas newer and more high-tech modes are either holding their ground or growing.
Web is dominant around the world or at least in the parts we survey. Looked at regionally (not shown in the charts), CATI now a distant second, at 57 percent, while it is still mobilized by 72 percent of companies in Europe. Web in both regions enjoys 95 percent utilization.
The downward trend in CATI is strongest in North America. In 2006, 74 percent of North American companies were doing CATI but by 2014 this was down to 57 percent. Europe, however, shows little change: 68 percent in 2006 and 72 percent in 2014.
We also have data that shows the volumes of business that each mode generates within each company. While the utilization chart tracks a rapid rise in CAPI, volumes are still low, although they have doubled from 5 percent in 2006 to 10 percent in 2014.
CAPI is making inroads at a time that those two industry mainstays of CATI and paper are in decline. One-half of companies still utilize paper and nearly two-thirds (65 percent) CATI. With the development of tablets, smartphones, improved data handling on mobile networks and increasing availability of Wi-Fi, CAPI is becoming a more realistic option.
CAPI and paper are important in Europe – both are used by over half of companies in Europe but under a third in North America. This difference is probably due to greater diversity in market conditions and culture in Europe, although we do expect to see a drop in pen-and-paper research in Europe.
Every year, we ask our participants to consider their quant work and estimate the proportion of it that each mode represents. In Figure 2 we show the North American and European data for Web, CATI and paper. The results show that the North American marketplace is still, after all these years, gravitating towards the Web.
The volume of Web research in North America in 2014 (68 percent) has grown steadily since we first measured it in 2006 (45 percent). Furthermore, the picture in North America can now be described as Web first, CATI a poor second (13 percent) and then half a dozen other modes that account for the other 18 percent, with none taking more than a 5 percent share. Paper is no longer a major mode in North America because the amount of research conducted with pen and paper is now close to eliminated – as recently as 2006 it still accounted for 16 percent but by 2014 it is down to 3 percent.
The trends are similar in Europe but not as pronounced, with Web and CATI accounting for 67 percent together and paper still clinging on to 11 percent.
Trends in the minority modes
Focusing on the more rarely-practiced modes (Figure 3), we have noticed that there is lots of recent growth in all things mobile – and this appears to be across the world, not just in North America. When we first looked at the proportion of companies offering “mobile self complete” (i.e. online research specifically designed for smartphones or other mobile devices) in 2009, 6 percent of companies provided this service. Just five years later, this has surged to 27 percent.
Volumes for mobile are growing too, albeit from a tiny base – 1 percent of total quantitative work in 2009, 3.6 percent in 2014.
We also think CAPI is showing signs of a renaissance in the form of mobile CAPI (mCAPI). With the widespread availability of cheap consumer tablets and greatly improved support for data by the mobile networks, mCAPI makes perfect sense for face-to-face. The proportion of companies including this in their portfolios has increased slightly in 2014.
Even SMS research is showing signs of life. But we question why interactive voice response (IVR) is languishing at the bottom of our chart.
Many other industries use IVR widely, including our close cousin, customer service feedback management, so why not market research? It would seem ideal in combination with other technologies such as CATI, on many topics where smart voice prompts can finish off the interview once rapport and trust have been established – avoiding the cost of and away from the moderating effects of the interview. Is the research industry missing a trick?
Mobile participation in Web surveys
For the last four years, we have been asking companies how many of their survey participants are attempting to take Web or online surveys on smartphones and other mobile devices. It has risen progressively each year from 6.7 percent in 2011 to 21.4 percent in 2014 (Figure 4). There are more accurate measures of volume, including one recently published by the survey’s sponsor, FocusVision, which put the number of survey starts across the 74 million surveys fielded on its Decipher platform in 2014 at 7 percent for panel surveys and 22 percent for surveys using client-provided sample.1 However, the upward trajectory it reports is closely mirrored in what research firms report to us.
What is revealing is that many companies found the question too difficult to answer, with 29 percent answering “don’t know” – which is slightly down from 2013, when 37 percent gave no answer. To us, this is essential information anyone involved in supporting online research should know.
Go to a conference or read any of the research discussion groups and you will find people are increasingly talking about the need to take a “mobile first” approach to online survey design. This is because, given current trends, in a few years’ time a majority of survey starts will be on a mobile device. So this year, we delved further to see how proactive research companies are in designing for mobile in their online studies.
We asked companies “How often would you say that the question of dealing with survey participants on mobile devices is discussed when designing online research projects?”
Overall, as shown in Figure 5, companies split almost 50/50 between those who mostly do (48 percent “always” or “often” discuss this) and those who mostly don’t talk about mobile when designing their online studies (52 percent report “sometimes” or “rarely/never”). Only one in five routinely take it into account.
There was no difference between larger and smaller companies in terms of those actively talking about it, but more smaller companies were likely to ignore it altogether. It’s talked about most in Asia-Pacific (59 percent “always” or “often”), yet is less likely to get consideration in Europe (43 percent).
Other questions (which you can read about in detail in the full report) probed how the extent to which their online surveys are now mobile-friendly and how good the survey platforms they used were in creating mobile-friendly versions of online surveys. A similar division emerged there too.
Two-thirds of firms report they support mobile in many or all of their surveys, with support more prevalent in North America and among larger firms. However, the smaller firms have the edge on taking a mobile-first approach: 36 percent of smaller companies say all of their surveys are now mobile-friendly, against 22 percent of larger companies and 31 percent overall.
Technology does not seem to be the barrier here: 29 percent of companies reported the software they used made it “very easy” and 42 percent said it was “moderately easy” to create online surveys that were also mobile-friendly. Interestingly, 10 percent did not know – a high number compared to other questions in this study. It again reveals the extent to which the topic is still not high on everyone’s agenda.
Technology by stealth
Another emergent trend in the wider field of corporate IT is the rise of what is sometimes called “shadow IT” or “stealth technology” – something that IT managers are increasingly worried about. This is where individuals or departments bring in their own technology devices and connect them to the network or buy their own software, which they either download from the Internet or simply subscribe to as a cloud-based service paid for on the company credit card.
For research companies, the pressure to use technology not chosen centrally by the company is even greater, as both research suppliers and in some cases clients too are twisting the arms of researchers to use their own systems for sampling, fieldwork administration, data collection or, in the case of clients, to distribute and report results.
We asked what “additional” technology resources the company made use of and listed 10 specific kinds of usage, plus “something else,” which we have grouped in four categories.
This reveals that using other people’s technology is very common across research companies of all sizes. In total, 82 percent of firms report they use software that is owned or controlled by a research supplier – data collection platforms being the most common – and 48 percent use software controlled by their clients, with reporting software most frequently referenced (Figure 6).
Actual shadow IT is mentioned by 28 percent of firms. It comes ahead of the more official equivalent: software selected and purchased from a departmental budget, which 24 percent report.
We also asked about policies and attitudes towards using external technology and shadow IT and found a very mixed bag of responses, with many firms nailing it down through policies and procedures but a minority leaving it to good fortune.
This is not just about challenging the status of the CTO or IT manager. It is also about knowing where all your data is and ensuring that both commercial confidentiality and personally identifiable data are properly protected, which is hard to do unless good controls and policies have been put in place. It is a fundamental requirement of every research firm to know where its data is and who has access to it. We know this is not always achievable but there is no excuse for unintended release of confidential data and we are not confident that research companies are doing all they need to to manage this risk.
Dashboards and data visualization
The appetite for dashboards and more graphical ways of presenting data among research buyers shows no signs of diminishing. But we know from when we asked about this in 2011 that companies were under strain to deliver these more exotic kinds of output. Three years on, we wanted to see if things had changed and hoped to isolate where the problems lie.
For 2014, we ascertained that graphical reporting is very common among companies as a delivery method. Each firm was asked the percentage of projects they deliver that require each of four different kinds of visualization methods – we have simple-averaged those percentage values here:
- Static charts or graphs presented online: 37%
- Online dashboards that include some data visualization: 27%
- Interactive charts presented online: 18%
- Custom infographics in reports or online: 15%
We then probed to see how difficult companies were finding it to satisfy client demands for each and discovered that there is still quite a lot of pain involved (Figure 7).
With the exception of static charts, more firms found each of the other methods difficult rather than easy. The question was a 10-point scale, calibrated from 1 for easy to 10 for difficult.
We probed into the specific difficulties with dashboards. Top of the list was not enough budget, followed closely by the need to customize the software to handle dashboards (which is another cost consideration) and lacking the in-house skills.
However, skills is one area in which there is a noticeable shift in the last three years.
We asked who was involved in the production of data visualizations in both years. Not surprisingly, researchers are involved in most cases, but beyond that, the picture has changed dramatically between 2011 and 2014 (Figure 8).
In 2011, after researchers, it was research assistants, DP and IT specialists and charting specialists that followed. Come 2014, the second place now goes to graphic artists. However, DP and IT specialists are in third place – and this is not surprising given that much of the preparatory work involved in getting data ready for visualization depends on getting the data into an amenable format to work with. In 2011, very few firms mentioned visual designers either but the number has doubled since that time.
Figure 8 illustrates another point: that data viz is a team effort, as it calls on a lot of different people with a lot of different skills. The ideal combination is probably the researcher (to identify the story), the IT specialist (to get to the data) and the designer (to lift both to the next level). Overall, the balance of skills reported in 2014 looks a lot more healthy than it did in 2011 and more conducive to producing a higher quality product too.
In another question, we probed into the technology being used and this again shows there is unmet need in the technology being used. However, this is not surprising since few firms report that they actually use any specialist software and a worryingly large number seem to expect a combination of PowerPoint, Excel and a crosstab package to be up to the task. In the area of data visualization, the industry seems to be upping its game on the skills front but not with its technology. We are hoping that’s what comes next.
REFERENCES
1Trend Report: Mobile Participation in Online Surveys, FocusVision. Download available from http://bit.ly/1Eiabl4.