There are many questions to ask before the questioning begins
Editor's note: Andrew D. Cutler is principal at Integrated Marketing Associates, a Bryn Mawr, Pa., research firm.
Global qualitative research is not for the faint of heart. Unlike conducting focus groups or one-on-one interviews in a handful of U.S. cities, conducting research abroad requires a heightened sensitivity to the nuances and unique challenges that await. Experienced researchers know that it is crucially important to follow certain rules if a global qual project is to be successful. What follows is a summary of many, but by no means all, of the steps that will help to ensure that your qualitative research has a successful outcome on the global stage.
Allow plenty of time for each step in the project. While rushing market research projects is never a good idea, it is especially important to allot sufficient time when you are dealing with the complexities of a global qual project. Case in point: the process of approving study materials. For each document, you need more than just approval of the English-language version of the document. Chances are, your client’s partners in the local markets will want to review it and provide input. A product profile, for example, may have specific nuances that apply only to certain markets. With many stakeholders involved, reaching consensus on the exact wording of each document may take weeks; there may be internal disagreements at your client’s firm as to what the precise wording should be. Once all of this is sorted out and the translated versions arrive, the process begins again, with approval needed from your direct client as well as his/her colleagues in the local market. If any of the stakeholders ask for changes to the translated documents, this of course adds further time to the process, as the translations then need to be implemented and approved by all parties.
Work with reliable recruiting partners. There are a lot of recruiting agencies out there that claim to be experts at finding respondents for global qual research. The safest approach is to work with recruiting partners whom you know from personal experience to be excellent. If you don’t have these kinds of relationships, you need to ask some tough questions of any company that you are considering for the recruit: How do you find potential respondents? What channels do you use? In which countries do you have “troops on the ground”? For each country, how long have you had a presence in that country? What types of audiences are you recruiting most often? How large are your databases of each respondent type, by country? What steps do you take if you run out of people to contact from your database?
Work with experienced moderators. For overseas research, when moderators who speak the local language are needed, it is vital to select those who are truly excellent at what they do. But how do you find the best of the best? Again, the recommended approach is to select moderators whom you have found to be outstanding in prior research projects. If you don’t have firsthand experience with any moderators in a specific market, or if those with whom you have worked have been less than impressive, then compile a set of possible candidates who are recommended by colleagues in that market. Study their CVs carefully, paying particular attention to the number of years they’ve been moderating and the level of experience they have in the relevant industries you are researching. To assess their intellectual acumen, talk with them on the phone and obtain (blinded) writing samples. And ask for references from other clients with whom they have worked. Keep in mind, too, that it is preferable to work with moderators who are fluent in English (as well as their local language) so that you and the moderator can freely communicate about the research at every stage of the process.
Conduct the interviews in local languages. Occasionally, for research in a market where English is not the first language, clients will ask if it is possible to just recruit participants who happen to be fluent in English and thereby enable the interviews to be conducted in English. Unfortunately, as enticing as this approach may sound, it is not advisable, for a number of reasons.
First, being “fluent” in English is a subjective matter and someone whom the recruiters consider fluent may not be fluent in the eyes (and ears) of others involved in the project, such as yourself and the end client. Attempting to conduct an English-language interview with someone who has little grasp of English can provide little meaningful feedback (at best) and may lead to miscommunications that actually distort what the respondent is trying to say.
Second, finding participants who do speak English is likely to be challenging, especially in countries that de-emphasize English as a second language, and will slow or even halt the recruiting process. You do not want to be in a position where the recruiters say “I’m sorry, but we have reached out to all the eye surgeons in Italy and only have found two who are (a) fluent in English and (b) willing to participate in the research.”
Third, even if the recruiters are able to find a large enough sample of English-speakers, limiting the research to English-speakers inevitably reduces the degree to which the respondent sample represents the larger universe from which you are drawing.
Attend the research. In an attempt to save money, and particularly with the advent of videostreaming technology, it may be tempting to consider monitoring the research from the comfort of your home office in the United States. Resist this temptation! If your research is to go off without a hitch, you (or a colleague) need to attend in-person. What if the videostreaming doesn’t work properly? What if respondents are late or don’t show up at all? What if someone from your client’s organization decides to attend at the last minute? What if the moderator, despite your careful coaching, has a question or problem in the middle of an interview? In any of these circumstances, your physical presence at the research can make a big difference.
Talk to the moderators before the research begins. Preparation is the primary ingredient in a successful research project. I like to prepare by placing myself, mentally speaking, in my client’s shoes and asking myself: What are my key objectives? What are my challenges? Who is my competition? What issues am I facing? What questions do I need answers to? What else do I want to explore? What will I be doing with the research findings? How can marketing research help me?
Asking questions like these forces me, the researcher/moderator, to truly see things through the eyes of my client. If you internalize your client’s situation, you will inevitably create research tools that will directly address your client’s needs. Not only that, when you meet with the ex-U.S. moderators you will be able to convey to them exactly what the objectives are and thoroughly inculcate them on the specific areas on inquiry. Note that you should meet with each of the moderators at least two – and ideally three – times before the fieldwork: One meeting alone will not provide the reinforcement necessary for the moderator to internalize the project objectives.
Talk to the moderators during the research. While you are watching ex-U.S. research, it can be tempting to assume that the simultaneous translator is capturing exactly what the respondent (and the moderator) are saying. Resist this temptation. Because of the challenges of real-time translation, even a top-notch simultaneous translator will only be able to capture a portion of what is actually being said in the interview. And the non-verbal components of the interview, such as the respondent’s tone of voice and the modulations in volume and intensity, are even harder to capture.
For all of these reasons, it is vital that you talk with the moderator for regular debriefs throughout the day of research. Make it a habit to spend three-to-four minutes with the moderator after each interview for a download. You may be surprised at how much additional information you capture by doing this. If nothing else, having these systematic debriefs and dialogues will strengthen your grasp of the research findings and will likely enable you to pick up patterns in the responses that you might otherwise have missed.
Talk to the moderators after the research is over. Just as it is vital to talk with the ex-U.S. moderators throughout the day of research, it is equally important to hold debriefs with them at the conclusion of the fieldwork in that market. It is at this point that the findings from the research in the market can be synthesized and the person who was literally closest to the respondents – the moderator – is best equipped to do this. Set aside an hour for this debriefing session, during which time you and the moderator (and the simultaneous translator, if they are available) can talk undisturbed and uninterrupted. Make certain that you go through all the questions in the discussion guide. Ask the moderator to provide his or her thoughts on the patterns and variances that emerged over the course of the research in that market. Pay attention to commonalities and to differences in how you and the moderator assess the research findings and challenge the moderator if s/he has perspectives that differ from yours, keeping in mind that s/he was the one who actually conducted the interviews and heard the respondents firsthand.
Get transcriptions. English-language transcripts are expensive – very expensive. But they are worth it. By communicating to you and your client, word-for-word, exactly what was said in each interview, transcriptions can provide invaluable insights into the research content. Keep in mind that translating is arduous work, mentally. The brain has to process two languages almost simultaneously. A simultaneous translator has to verbalize the translations instantly, while at the same time listening to the next snippet of conversation. This is enormously taxing work and, given the pressures of real-time translation, even the best simultaneous translators cannot translate every single question-and-answer with 100 percent accuracy. Thus, what the simultaneous translator says should be seen as merely a rough draft of the actual interview content.
A transcriptionist, on the other hand, has the luxury of time to get the translation right: time to pause the recording and decide on the best way to translate a word or phrase; time to look a word up in the dictionary if it (or its translation) is unfamiliar or unknown; time to correct any errors in his/her translation that become evident before the transcript is issued; and, most importantly, time to take breaks in between chunks of translation.
Besides providing an accurate encapsulation of the interview content, transcripts also benefit you in other ways. Including quotations from the interviews adds considerable depth to the reports that you write and, for the aforementioned reasons, you should not quote what the simultaneous translator said. Transcripts provide a clean record, for posterity, of every interview’s content, in English. And, if you or your client wish to delve further into what specific respondents said, it is much faster and more efficient to use transcripts than audio recordings for this purpose. All this being said, if your client’s budget simply does not allow for transcripts of every interview, then post-fieldwork you should choose the interviews that you and the moderator feel were the most insightful and obtain transcripts of those.
Hire the best darn simultaneous translators you can find. The value of simultaneous translators is sorely underestimated by much of the market research community. Keep in mind that, for the research in each ex-U.S. country, you and your client are probably going to be listening to the simultaneous translator throughout the course of the interviews. If the translator is hard to understand or has difficulty keeping up with the interviews, you and your client are going to derive very little value from the research, at least until you get the transcripts of the actual interviews. The solution, again, is to select simultaneous translators whom you have found to be outstanding in prior research projects. If you do not have experience with translators in that market or have not had positive experiences with the translators you’ve used so far, then you should conduct a careful screening of the translators who are available and recommended by colleagues in that market.
Just as with the moderators, study the translators’ CVs to assess the number of years they’ve worked in their field and the level of experience they have in the relevant industries. Talk with the translators on the phone and ask for references from other clients with whom they have worked. In my experience, the most useful simultaneous translators are those whose first language is English but who have lived abroad for years and thus have become highly conversant in the native language. When such translators translate what the interviewer and interviewee are saying, it is almost as if you are watching the interview in English.
One more tip: Brief the simultaneous translator(s) thoroughly on the interview purpose and content, including every question in the discussion guide, before the research begins, and make sure they understand any technical terms that may arise during the interviews.
Come prepared
Global research is a complex and tricky undertaking but if you come prepared with the right strategies and tactics, you will emerge with a bevy of insights that are both meaningful and accurate and that capture the commonalities and differences in the perspectives of respondents from different parts of the world. This knowledge will make you invaluable in the eyes of your client, as the insights you bring will ultimately provide solutions to their global marketing issues and challenges.