Editor’s note: Tim Macer, managing director of U.K. consulting firm meaning ltd., writes as an independent software analyst and advisor.
Q is a new data analysis program from Australia-based Numbers International that is designed to allow researchers to reveal hidden depths in their survey data using the power of statistical testing and modeling but without expecting researchers to become advanced statisticians. It’s perhaps fitting for software from Down Under that this tool can turn the process of analyzing market research data on its head. If it borrows from any school of data analysis, it is probably from the SPSS Base statistics approach, but in a way that is much more market research-savvy than the more general-purpose SPSS.
Q deals intelligently with every kind of survey question - single-coded, multi-coded, numeric value, even grids - in a consistent and even-handed way. Unlike the majority of conventional market research tabulation tools, it is not afraid of letting researchers - the primary audience for this software - get eyeball-close to the data: All the case data is only a mouse click away, on a dedicated data tab.
Q is offered as a desktop tool that works under Windows. You start by opening your Q file (with the file type “.Q”, which is the study’s database containing both case data and survey metadata) just as you would open a Word document or PowerPoint deck. The expectation is that your data provider or data processing department would set this up for you and even create some reports ready to work on. If you want to do it yourself, you can also create your own Q database by importing directly from SPSS SAV or SPS files or from Triple-S files. These will load in all the variables from your study and give them the right designations (single-coded, numeric, etc.), which are important to ensure Q knows the most appropriate models or tests to apply to each question. CSV import is also there as a fallback, though to get the best from the program you will then need to spend some time setting up appropriate question and category labels and ensuring the right question types are set.
Easy to learn and use
This software is very easy to learn and to use, though it is not necessarily intuitive at first sight - probably because there’s some unlearning to do for most experienced researchers. To make the point, Numbers provides not only a quick-start guide to take you through basic tables to choice modeling and latent class analysis in 60 pages, but also an instant-start guide which distills the basics into a single sheet. There is also integrated help and online training with show-me features that take over the software, select the right menu options and then undo it again, ready for you to do the work yourself.
What really differentiates Q from other survey data analysis tools is that it offers the researcher a blended approach to data analysis, combining straight crosstabs for primary reporting with advanced multivariate approaches to reveal hidden trends and connections in the data. So often, these connections remain undetected in most survey datasets simply because the researcher lacks either the tools or the time and budget to dig any deeper. Q can help move the task of analysis from superficial reporting of the numbers to telling the client something he or she really hadn’t realized - based on evidence and backed up with confidence scores.
For the more involved operations, such as multivariate mapping or latent class analysis, you always start from basic tables and analysis. It help keeps you grounded, letting you approach more advanced and possibly less familiar analytical techniques in a stepwise process, building on what you have already seen and verified.
One of the design principles Numbers applied to Q was to put users in front of the actual numbers as early as possible in the process. You always start in table view, looking at some of the data, but this view is highly dynamic and many of the options that you find tucked away in menus, pick-lists and property sheets in other analysis tools are achieved simply and elegantly by dragging and dropping. For example, just clicking, dragging and dropping will let you merge categories; create nets; and rename, reorder or even remove categories. Most functions or options are no more than a single context-sensitive click away.
Another difference is that the program works out the best way to analyze the questions you have selected - you use the same table option whether your question is numeric, categorical or grid. There are a lot of different options that help Q understand the kind of data it is dealing with, and from this it will also select the most appropriate significance tests to apply to the question. It makes appropriate adjustments according to whether the data are weighted or not, and also takes into account the effect of applying multiple significance tests that can otherwise lead to false positives.
In the tables, arrows and color-coding show not only which values are statistically significant but highlight the direction of the difference. As you generate tables and other outputs, these appear in a tree on the left. You can keep them or discard them and you can also organize them into subfolders. From this, you can create a package of a subset of the tables or models you have created. This creates a small e-mailable file which others can then view by downloading the free Q Reader.
The Reader can provide a very simple and inexpensive route to distributing interactive tables to clients. Numbers limits the options in the Q Reader version but clients and co-workers can still slice and dice the data in different ways that are relevant to them but have no direct recourse to the raw data.
Another impressive feature is its handling of conjoint analysis. Q lets you roll up an entire multilevel choice model into a single composite question, which you can then crosstabulate and filter with the same ease as a simple yes/no question. And with all of the built-in significance tests and other analytical techniques at your disposal, you can very quickly determine the real drivers in any choice-based model.
A little lackluster
Where the software is perhaps a little lackluster is in the quality and range of the options to finesse the outputs it provides. It makes little attempt to represent data graphically in histograms or pie charts. Charts are restricted to those associated with correspondence mapping or other such models. There is no integrated support for Excel or PowerPoint, either. If your point of reference is SPSS, then you may find its outputs a step up, but if you are coming to it from other market research data analysis tools, you may well be disappointed.
The full version of Q will also let you import and refresh your data, which provides some rudimentary support for trackers. However, the current version, although it contains very good support for time-series analysis, is poor at version control and reconciling differences in data formats between waves of the study. Perhaps surprisingly in these days of data integration, you can only have one study open at once, though Windows will let you have more than one instance of Q open.
Overall, these are relatively minor weaknesses in a highly-intelligent software product. They are largely indicative of a developer whose priorities lay in simplifying the most challenging problems, and in doing so allowing substance to triumph perhaps a little too much over style.
Gradual introduction
One company making extensive use of Q is Sweeney Research, also in Australia. Erik Heller is general manager of Sweeney’s Sydney office and an experienced researcher with a background in advanced quantitative methods. He has overseen the gradual introduction of Q as an analysis tool for researchers to use on data collected largely in-house from a broad range of telephone, online and in-person surveys. “One of the advantages of Q is that it is very easy for someone who is not that involved in the data analysis to go into the data and run some additional crosstabs,” Heller says.
“There is a huge efficiency gain if someone works up a hypothesis when writing his report and does not have to stop what he is doing, run downstairs or write an e-mail to the data analyst. It may not sound like much, but it is really quite disruptive and therefore desirable to streamline this from the business perspective. That is not something that is especially novel to Q, but where Q differentiates itself from other tools like SPSS is the extent to which it is intuitive and easy for people to immerse themselves in the data.”
Asked how long it might take a novice user to become familiar with the software, Heller says, “That really depends on the individual, but most people seem quite capable of using Q within a couple of hours: checking the data make sense, interrogating the tables and producing some additional basic tables.”
However, as Heller points out, this is just the start for most users in understanding what Q is capable of. “The statistical abilities of this program are the biggest reason I’ve been driving this internally to get people to use it. What’s really good about it is the sophistication of the tests and the foolproof way that they are applied. If you think about the purpose for which most of the traditional tests were originally designed, it is very different from the ways we analyze commercial research studies today.
“In a way, at the 95 percent confidence level, every twentieth test will be a false positive. And on a typical study, we run hundreds of these tests. Q uses testing approaches that aim to account for this and are therefore more appropriate for the huge number of tests that are done on a typical study. It means it is easy to cut across big data sets and see the important, statistically-significant effects. To do that in a software like SPSS is infinitely more cumbersome.”
Heller had also used Q to analyze conjoint studies, both before and after the conjoint analysis module was added to Q, where an entire choice-based experiment is simply treated as a single composite question. “I’ve use it on one project, so far” he says. “It is brilliant in its simplicity. Most people are probably used to having a bit more control over the data and this way you are putting a bit more trust into the software. Clearly, the approach they have taken is that they want to make it as simple as possible. It is ideal for someone who does not have the time to get involved with all the statistics behind it. I think it is a great feature and one I will use a lot more going forward.”
Another plus for Sweeney Research is the ability to export tables and charts in Q for clients to view in the free Q Reader. “For them to be able to look at the tables and merge categories without manipulating the original data is very beneficial and we find they are very keen to use it. It does not overload them because the free version has fewer options; it just allows them to check the little queries they might have. It is all about simplifying things. There is so much information out there these days and there is no shortage of data - the aim is to provide it in the most usable format you can.”