Editor’s note: In conjunction with his BOSS Academy Radio podcast, Paul Kirch, CEO of Actus Sales Intelligence, a Fort Worth, Texas, business and sales consulting agency, is interviewing authors, marketers and marketing researchers on a wide range of topics. By special arrangement, we’ll periodically feature edited recaps in the e-newsletter including portions of the conversations that touch on research-related topics.
Debates surrounding survey research and questionnaire design are intense and many of the questions long-standing. According to David Harris, head of marketing research and competitive intelligence at Merz North America, these debates stem from one problem: In marketing research, questionnaire design is not treated as a professional discipline and as such, many people writing questionnaires are not trained.
In an interview with Paul Kirch for BOSS Academy Radio, Harris discussed common misconceptions around survey design and his personal dive into questionnaire research while writing his book, The Complete Guide to Writing Questionnaires: How to Get Better Information for Better Decisions. Harris is a proponent of considering qualitative methods before quantitative and organizing research to support decisions instead of always relying on a go-to form of research regardless of the target audience and information needed to make decisions.
Paul Kirch: The topic of long surveys has been such a heated debate…
David Harris: Well, the first thing people have to realize is that you only have so much time and energy from respondents. The moment you start asking questions that don’t make any sense to them or use a bunch of jargon or you ask them to select from a list and their item isn’t on the list … they are going to just tune out, get through your survey or drop out. (Consumers) are doing it a ton now when they are taking surveys on their smartphones.
One of the things that I learned – it started about 15 years ago – was this whole philosophy of organizing research to support decisions. Most decisions are made based on three, five, seven pieces of information. There are some exceptions like segmentation studies but most of them are key pieces of information that you need.
You are able to work with clients to define what’s on the table – what are we going to decide? – and then you go through to what information do we need to make that decision. I’m not talking about how to write questions (but rather) what information do we need? So often we need qualitative information, so we do interviews. So often the information we need we already have the data for. But when you do that process (of defining the desired outcomes), you end up with questionnaires that are so much shorter and so much more direct and you get the information you need for what you’re going to do.
Are shorter questionnaires detracting from the types of results or enhancing? What are you finding?
I think it enhances (results) enormously. Number one, we write a questionnaire that gets the information that’s needed for the decision that’s on the table and we also write it in such a way that it makes perfect sense to a respondent.
I’m not the first person to say this – the idea of writing your questionnaire plan where you really think through the information we need and how we’re going to analyze the data. It becomes so much easier to analyze because you’re dealing with a much better data set. There is less error in there, there is less non-response error.
Are you a fan of any particular form of research? Online, telephone, paper-pencil?
That issue has been around forever. It all depends on the respondent group.
That’s why when you do the census, they use all forms – they have to (in order) to get to the audience. If I’m doing a survey with gastroenterologists (the form) may be very different than if I’m doing a survey with restaurant workers. You really have to match the method to the audience.
There is such a line between qualitative and quantitative. Talk a little bit about organizing research to support decisions and how that should really be facilitated.
The truth is, 90 percent of the time when people come to me and say, Dave, I want to do a survey, I try and talk them out of jumping straight into a survey because the information they need is qualitative in nature. This can’t be replaced by a survey. There is nothing more tortured then a survey that is being asked to get information that is qualitative in nature.
I think of qualitative research like a physical exam, when you go to the doctor. And quantitative research is maybe like an x-ray or an MRI. They are serving different purposes and they are so complementary.
How often do you really see that (qualitative before quantitative research) being leveraged? Obviously budgets and pricing are sometimes the big barriers.
It’s a huge problem. I don’t know why people so often want to jump straight to the survey. I think they undervalue what they can get from really good qualitative work. Maybe they have not seen it executed properly.
I think there is another issue: planning. If this is such an important decision, why didn’t we start a few months ago?
Since your book is on writing questionnaires, do you have any easy takeaways or fun stories?
One was a question that was administered to women that were discharged from the hospital with a healthy baby. They were asked, “How long was your baby in the hospital?” Well, some answered in inches and some answered in days! So that goes to the guideline of: always state the unit of measurement. How many days was your baby in the hospital? How many inches was your baby in the hospital?
You can listen to the entire interview at www.bossacademy.com.