Integrating data through mixed and multi-method research

Editor’s note: Julia Maier is an analytics associate at research firm KJT Group, Honeoye Falls, N.Y. 

Research using a single method is quick and cost effective. A short survey with a few direct questions provides descriptive data to summarize people’s attitudes and behaviors. Unfortunately, a single-method design is often unable to capture the full picture and limits the insights that can be derived. 

The parable of the blind men and the elephant demonstrates this shortcoming. In the story, the men, looking to determine what it is, touch a different part of the elephant and describe it based on their limited experience. Depending on their vantage point, the men decide the object is a rope, a tree or a fan. It is only from the integration of multiple pieces of data that they can deduce the object is, in fact, an elephant. 

High-quality research uses mixed methods and multi-method approaches to obtain data via multiple sources to provide more reliable insight narratives.

Mixed method research

Qualitative and quantitative approaches to research are the fundamental approaches to collecting data. Qualitative research allows us to take an idiographic approach to understanding respondents’ experiences: each person is an individual with a unique story and perspective. Through in-depth conversations, we learn about aspects of the topic we might never have considered or would have missed using a standard questionnaire. Alternatively, quantitative research allows us to take a nomothetic approach and make inferences about people in general. From these inferences we can attempt to predict how different people would respond in the same situation.

Mixed method research uses qualitative and quantitative research as different phases during which information is gathered about the same research question. From the qualitative phase, we learn about the nuances of experience as unconstrained responses allow for unique information to be shared. From the quantitative phase, we capture specific information about a greater number of people – statistical methods summarize data and allow for inferences about the general population. Conducting both qualitative and quantitative research on a topic allows for the synergy of information, yielding more insights than could be garnered from either method in isolation.

The greatest aspect of mixed method research is that qualitative and quantitative methods complement each other in terms of strengths and limitations.

Use qualitative research as phase one of a study to enhance the design of a quantitative phase two. 

Phase one: Capture the variety of experiences in the landscape and how they relate

Phase two: Develop expectations for these experiences, such as what they generally look like and
how common they are

Multi-method research

Another approach often used to maximize insights is conducting multi-method research. Rather than combining qualitative and quantitative research, a multi-method approach refers to the use of different tools within a single paradigm. Many different techniques are available for qualitative research: focus groups, structured interviews, unstructured interviews, etc. A multi-method qualitative study would employ more than one of these tools to examine the same question. As stated above, the goal of qualitative research is to capture unique, individual-level experiences. These methods work in different ways to elicit this information. People may be more comfortable sharing private things in a one-on-one unstructured interview, but the responses of others in a focus group may encourage participants to share things they wouldn’t have thought of otherwise. Without the use of both, vital information may be missed.

Similarly, multi-method research in a quantitative study allows for research to ensure information is not being missed by utilizing different operational definitions. We talk about concepts like “satisfaction” or “brand loyalty” on a regular basis, but what exactly do these terms mean? Let’s assume we all have a shared high-level definition that satisfaction reflects the fulfillment of one’s expectations (we can have the discussion about how accurate this assumption may or may not be at another time). With this definition, how do you measure it? We can use a single method, a single item question we are all familiar with: On a scale from 1-5, how satisfied are you with X?

How valid is that operational definition (i.e., working definition of how the concept is measured in the given study)? The best way to know is to employ a second operational definition to see if that method yields the same results. Perhaps we could also ask people to indicate what their expectations are (unaided, aided or both) and then have them select which of these expectations were met. Does the percent of expectations met correspond with the 1-5 single-item measure? If so, then we have some good validation of our measures.

It is not only beneficial to demonstrate method validation for ourselves; being able to report consistent findings increases the confidence of our clients or internal stakeholders toward our recommendations. The single-item satisfaction question, reported out on an infographic or top-line report, gives the quick-and-dirty look at a KPI. The second method provides added insight by incorporating information about how each person may be determining their satisfaction and which expectations are not being met.

Multi-method tips

Every data collection method available has its own set of benefits and drawbacks; there is no single best method to answer a research question. I’ll close this article with four tips you can turn to when selecting a multi-method approach:

• Ask directly and indirectly to get at explicit and implicit attitudes
• Use a single-item and an activity to capture different levels of information
• Expand qualitative studies to include group and solo settings
• Use mixed methods research and develop ideas about different quantitative operational definitions from the qualitative phase

By employing mixed method and multi-method research, we can maximize how much information is being collected. From these different sources, we can examine where the information converges and diverges. It is by gathering information at both points that we can recognize the elephant.