Why Web-assisted TDIs are a cost-effective qualitative methodology

Marrying phone and Web

Editor’s note: Tom Donnelly is an associate director at Double Helix, a Fort Washington, Pa., research firm.

Market researchers are frequently asked to provide more value for less cost in a shorter time frame. The Internet offers a variety of suitable methodologies to conduct qualitative research, such as online bulletin boards, virtual focus groups and video chat interviews. One cost-effective methodology is the Web-assisted tele-depth interview (Web-TDI). While the technique may be old news for many, its pros and cons may not be well understood. This article presents an overview of: how Web-TDIs differ from traditional methods; two Web-TDI example studies; advantages of Web-TDIs; limitations of Web-TDIs; technological choices and avoiding pitfalls; when to use Web-TDIs.

How Web-TDIs differ from traditional methods

Web-TDIs are one-on-one telephone interviews conducted while also showing respondents information over the Internet. With standard TDIs, respondents either are not shown stimuli or are sent stimuli via mail/e-mail. Sending stimuli to respondents is not ideal because this information can be easily sent to competitors. Unlike in-depth in-person interviews (IDIs), moderators and clients cannot observe facial expression or body language with standard Web-TDIs. Also, it is more logistically challenging to manipulate materials with Web-TDIs vs. conducting IDIs. With these two exceptions, Web-TDIs are very similar to IDIs. The discussion guide may be identical for Web-TDIs and IDIs except for the moderator’s stage directions.

Two Web-TDI example studies

Two common research studies in which Web-TDIs have been used successfully are concept testing and message testing. One example is concept testing for physician journal ads where the manufacturer is trying to decide the best way to convey product messages to its key targets. Often three concepts are produced, with two different executions of each concept. To measure stopping power (i.e., the likelihood a respondent will stop and read the ad), one execution of each concept is shown briefly and then respondents are asked what they recall and how likely they would be to stop and read further. After this exercise, each execution is examined in detail and respondents are asked about the imagery, headline, body text and tagline. Conducting Web-TDIs for this concept test involves showing the concept executions in specific shuffled orders for a set amount of time.

Another example is message testing for a pharmaceutical brand where the manufacturer is interested in which messages resonate with the target physicians and how best to convey the brand’s story. Typically, there are sets of messages (aka message buckets), such as efficacy, safety, etc. One way to conduct Web-TDIs is to use a spreadsheet with a tab for each message bucket. For each tab, messages are listed as rows and various attributes (e.g., credible, unique, motivate to Rx) for which messages are to be rated are listed as columns. After all message sets have been reviewed, messages that were selected to be included in the story can be shown on a new tab using spreadsheet programming. On this tab, messages can be put into the best order for a conversation between a drug representative and a physician.

Advantages of Web-TDIs

There are several advantages to using Web-TDIs.

• Research objectives can often be met at lower study cost by saving on travel, facility and lower honoraria.
• Research projects can be completed in less time because the recruitment and fielding is faster and travel days are eliminated.
• Stimuli can be adjusted easier and quicker. Sometimes a new set of materials needs to be produced for IDIs and shipped to the next facility location, whereas using Web-TDIs, the stimuli can be adjusted and sent electronically to the moderator in between interviews.
• Respondents can be recruited from a broad geographic area, whereas IDIs are often limited to a handful of cities.
• Scheduling is more convenient for respondents due to a broader range of interview times and elimination of respondents’ travel to the facility.
• Replacement of cancelled interviews is easier because the Web-TDI schedule is flexible and the pool of potential respondents is much larger.
• More detailed notes can be taken by the moderator compared to an IDI, in which the moderator must focus on respondent eye contact and body language as well.

Limitations of Web-TDIs

There are also several limitations to using Web-TDIs.

• Respondent IT issues can potentially derail interviews, such as if respondents do not have high-speed Internet access at the time of the interview or respondents’ firewalls prevent access to the Web portal. It is prudent to verify conditions before the interview with a brief trial run.
• Manipulating and/or rotating the stimuli across interviews might interfere with moderating. It is recommended to practice the process to mitigate this issue.
• Keeping documents confidential remains somewhat an issue, although Web-presentations are safer vs. standard TDIs. For example, Web-TDI respondents might be able to take screen shots or photos of the screen. Most Web portals have built-in safeguards but this remains a potential risk.
• Client guidance for the moderator during the interview may be limited. One option is to use instant messaging or chat on the Web portal out of the view of the respondent. Another option is to use the lobby feature of the teleconference call to confer with the client privately.
• Inability to observe a respondent’s body language or facial expressions with standard Web-TDIs can reduce interpretations. Video chat can be added if seeing respondents’ reactions is vital.
• Lack of commitment by the client market research team can be an issue. Often the team is willing to travel to view IDIs but do not listen to research live for Web-TDIs due to unexpected impromptu meetings that may take priority. Clients would need to closely manage their team, perhaps using calendar meeting requests or asking for feedback from the team at the end of each interview day.
• Sorting, rankings and ratings can be logistically more challenging for respondents. A brief online survey or spreadsheets could be adopted to capture respondents’ answers.
• Some stimuli are not suitable for Web-TDIs, such as medical devices or oversized materials.

Technological choices and avoiding pitfalls

It is important to choose the best technology to fit the research objectives and the moderator’s style and experience. There are many factors to consider.

• Some Web portals allow the moderator to pre-load stimuli onto the Web interface. This can be useful because it allows the moderator to focus on the interaction with the respondent rather than having to think about what/how stimuli are shown.
• A Web survey can be created to click through. The survey can be built with skip patterns and the respondent can enter responses (as with standard online surveys) for later data analyses.
• Some Web interfaces allow screen-sharing to facilitate viewing of the stimuli by the respondent. In addition, these interfaces allow the moderator to pass control to the respondent for self-paced reading and data entry (e.g., ranking and ratings). The moderator needs to take steps to avoid non-research information being displayed to respondents (e.g., e-mail “ghosts,” IM chat).
• Some Web portals provide teleconferencing as an add-on service, such as VOIP and/or standard landline. However, a separate teleconferencing line may be considered based on factors relating to cost, reliability and simplicity.
• Interview recording is often a requirement. Record the interview by using an audio recorder connected to the telephone line or computer. Or, utilize the conferencing recording feature, either audio-only or capture the visual portion as well.
  

Potential pitfalls using Internet technology should be anticipated and steps should be taken to avoid them.

• Design screeners to ask respondents if they will have high-speed Internet access during the interview.
• Provide simple instructions to respondents to test the Web portal with the computer they anticipate using during the interview.
• Moderators should rehearse the interview process thoroughly.
• Moderators should have a checklist for themselves that includes all steps involved during the interview process. This will avoid forgetting simple commands (hit *1; dial respondent; turning on the audio recording; mute client phone lines, etc.).
• Moderators should prepare materials for data collection processes (e.g., rankings and ratings) and rotation of stimuli across interviews.
• Anticipate and rehearse client and moderator communication during the interview.
• Build in extra time between the first few interviews to allow for delays and changes to stimuli and procedures.

When to use Web-TDIs

In some cases, Web-TDIs can replace IDIs and save both time and study costs while still meeting research objectives. However, some studies are best conducted in-person. Making this decision involves considerations beyond time and budget issues. If the answer is yes to any of the following, traditional IDIs should be contemplated.

• Does the respondent need extensive moderator explanations during the interview because the interview exercises are complex?
• Are the body language and facial expression of respondents crucial to the research analysis?
• Is there an extremely low tolerance for the risk of the stimuli being obtained by a competitor?
• Are materials too large or 3-D or do they need to be physically handled (e.g., a medical device)?
• Does the moderator need detailed client guidance during most interviews?
• Will team members be unlikely to dedicate the needed time to listen in live to Web-TDIs?

Web-TDIs are most useful when:

• Time and budget are limited.
• Research stimuli are likely to change often during the research.
• Respondents are difficult to recruit or a wide geographic sample is requested.
• Stimuli can be easily displayed on a computer screen.

Tackle the challenge

The Web-TDI is a proven market research tool for tackling the challenge of providing more for less. Market research clients need to evaluate the pros and cons of using Web-TDIs versus traditional IDIs. Moderators need to carefully choose their interview Web-interface platform and spend time practicing to avoid potential pitfalls. When used appropriately and judiciously, Web-TDIs can be a convenient and cost-effective way to achieve research objectives.