Tips for selecting a real-time virtual focus group facility

Smart shopping

Editor’s note: Jeff Walkowski is the principal of QualCore.com Inc., a Minneapolis research firm.

In the current climate of tightened marketing budgets and heightened concerns about air travel, companies that engage in qualitative research are beginning to consider alternatives to traditional face-to-face groups.

One of the alternatives available to researchers is the text-based online focus group. There are two general types of text-based online focus groups - real-time groups (all participants are online at the same time, typically for 90 minutes or so) and message board discussions (participants “drop in” at their leisure several times a day over the course of several days). The focus of this article is on real-time groups, which are the more frequently used type of online focus groups today.

Over the past several years a number of companies have emerged, each offering competing real-time focus group facilities to the research community. A researcher who is new to online groups might think that all virtual facilities are the same. However, just as the experienced researcher notices important differences in brick-and-mortar facilities across the country and around the world, there are important differences in virtual facilities as well.

It behooves the first-timer to know what to look for when evaluating potential real-time facilities. This article reviews some of the basic questions that the first-time buyer should consider as he/she goes about selecting a virtual facility. This is not an exhaustive set of questions. However, they will help the buyer open up a constructive dialog with virtual facility representatives. By asking these questions, the buyer will gain a better understanding of the differences between the options available. By being a smart shopper, surprises in the execution of a project can be avoided, which will maximize the success of the project.

1. How does the moderator communicate with respondents?

The most basic method for the moderator to communicate with respondents is “freestyle” or “on the fly.” That is, the moderator types in all questions, instructions, and probes into the text-entry area of the chat room in real-time (live, on-the-spot). A variation on this method is for the moderator to copy and paste questions from an electronic version of the guide into the text entry area. Here, the moderator will toggle back and forth between the document and the chat room.

An advantage of the freestyle method is that it forces the moderator to truly adapt to the needs of the group rather than use a series of canned questions. A disadvantage of this method is that typing everything freestyle (or even copying and pasting from a separate document) takes time; the extra time required can slow the pace of the group and can take the moderator’s attention away from the group, perhaps leading to missed opportunities to probe. Thus, this method is recommended only for moderators who can type very fast and can do so without looking at the keyboard, or for moderators who can dictate what they want to say to an assistant/typist.

Some facilities enable moderators’ guides to be pre-loaded. Here, the guide is readily available in the chat room for the moderator to use. Depending on the facility, the moderator may see the entire question, or he/she may see a summary or title of the item. Also, some systems have the capability of allowing the moderator to modify the pre-loaded item without having to re-type the entire question.

An advantage of pre-loaded guides is that they allow the moderator to spend more time paying attention to what respondents are saying, enabling him/her to do a better job of leading the group. A risk associated with pre-loaded guides is that the moderator might become a slave to them, not taking the time to adjust the guide to the needs of the particular group.

2. How do respondents see stimuli?

At the most basic level, a moderator can give respondents a URL where a stimulus (a concept statement, a mock-up of a print ad, a short product demo on video, etc.) is located. Respondents then copy the URL from the chat stream, open another browser window, paste in the URL, and view it. An advantage of this approach is its simplicity. However, there are several disadvantages. First, if the respondent does not copy the URL correctly, he/she will not see it. Another disadvantage is that once respondents open another browser, they have “left the room” and the moderator has lost their attention; researchers must hope that respondents will return within the specified amount of time.

More advanced virtual facilities reserve a frame (section) of the screen for stimuli to be shown. Here, the moderator has control over what is shown in the stimulus area. The advantage of this approach is that the respondent does not have to do any work to see the stimuli. A disadvantage is that the frame may not be big enough for the respondent to get a full view of the stimuli without scrolling left-right and/or up-down within the frame. Another potential disadvantage stems from the frame occupying space on the screen that would otherwise be used to show more of the chat stream (the running text of the discussion); respondents see less of the discussion on their screens and therefore some interaction opportunities among participants may be lost.

3. What does technical support consist of?

No online technology is foolproof. Even the most Internet-savvy participants (online moderators, respondents, and observers) may run into problems requiring immediate resolution for a group to run smoothly. All virtual facilities offer technical support, but that support varies from facility to facility. Be sure to understand what the technical support capabilities are and whether particular technical support features are standard or optional (available for an additional fee). When shopping for a virtual facility, ask the following questions about the technical support that will be provided.

• What are the qualifications of the technical support staff?
• Can tech support staff be contacted by toll-free phone number?
• Since most virtual facilities can run multiple groups at once, how many groups is a tech support person responsible for (e.g., one staffer per group? Two per group? One for every two groups?).
• Does tech support include host(ess) responsibilities (greeting respondents as they arrive in the waiting room, dealing with those who show but are not accepted into the focus group discussion)?

4. Is there a separate virtual waiting room?

Most virtual facilities have a waiting room, but some do not. Respondents come to the facility’s waiting room and are directed from there to the virtual focus group room. Just as at a brick-and-mortar facility, a virtual waiting room offers many benefits.

First, a virtual waiting room serves as a buffer zone between the outside world and the actual focus group discussion. In an overflow situation (when more respondents show up than expected) the number of recruits who are permitted to move on to the virtual focus group room can be controlled.

Second, a virtual waiting room allows participants to get to know each other and become comfortable with the technology.

Third, the waiting room can provide a place where additional re-screening can take place. (Re-screening in the waiting room should only happen if it can be done privately. See the next section.)

5. How can the facility assure that respondents are really the people they claim to be?

Because we cannot see respondents and ask them to show their IDs, there is no 100-percent guarantee that there are no “cheaters” (respondents pretending to be someone else) in a text-based online focus group. While guarantees cannot be given, there are some ways that cheaters can be identified. Each facility has its own capabilities.

One way is to ask some questions at the very beginning of the group to reassess the qualifications of the participants in the group. These questions could include some of the questions asked on the screener as well as additional questions relevant to the topic. Those who give different answers from the screener or who do not appear to be fully qualified can be excused. An advantage of this approach is that it is simple and does not require additional technical capabilities on the part of the virtual facility. Some disadvantages are that this re-screening takes place in a public arena (giving true cheaters an opportunity to answer based on what others say) and that it takes time away from the actual focus group session.

A better option is to conduct the re-screening in the waiting room (if the facility has one) before the group begins. This has the advantage of not using designated group time to take care of re-screening. The risks associated with re-screening publicly and en masse remain, unless it is handled via private message. However, using private messaging with all recruits in a waiting room will be an arduous task since it is the equivalent of conducting a set of brief in-depth interviews simultaneously.

An even better option is to have the respondent complete a self-administered questionnaire online immediately before the beginning of the group. This questionnaire would include items from the original screener that are unlikely to be correctly answered by a cheater. Each respondent’s answers should be instantly available to the research team (moderator and observers) for comparison against other information available about the participant. If cheating is suspected, the participant may be denied access to the facility, or once he/she enters the waiting room additional conversation (via private messaging or even the phone) could be used to investigate further. If this option is available, be sure to find out about costs.

6. How can “problem” respondents be dealt with?

One of the biggest challenges an online moderator faces is maintaining control of the discussion - keeping it focused on the topic. Sometimes even the most skilled online moderator encounters a respondent exhibiting counterproductive behavior (refusing to stay on topic, constantly joking, using offensive language, etc.). Since most virtual facilities offer instant messaging capabilities between the moderator and individual respondents, the moderator can try to work with the respondent, asking him/her to curb the disruptive behavior. Usually this will work. In rare instances, however, the only alternative is to remove the problem respondent from the room to salvage the group. How this is done depends on the virtual facility.

In some virtual facilities, the moderator must phone or send a message to technical support, asking them to electronically remove the person from the room. (Check to see if this procedure permanently blocks the respondent from returning.) An advantage of this option is that it is a straightforward process for the moderator. A disadvantage of this process is that it takes time for the moderator to contact technical support staff.

Other real-time virtual focus group facilities equip the moderator with an “eject button” for each respondent. When all else fails, the moderator clicks on a button on his/her screen and the respondent is gone and permanently barred from the room. The advantage of this is that it can be done quickly and without having to bring a third person into the process. The only disadvantage is that if the moderator accidentally ejects someone, it requires more work (and an apology) to bring the respondent back to the group.