Online focus group FAQs

Abstract

Acceptance of online focus groups by qualitative researchers has increased with advances in technique and technology. This article discusses how to successfully execute an online focus group, including application issues, recruitment and respondents, mechanics, and moderation and analysis.

You've got questions, we've got answers

Editor’s note: Monica Zinchiak is owner of Z. Research Services, San Diego. Much of the following information has been generated from a collaborative effort by the members of the QRCA Online Qualitative Research Task Force (Phase II). Members include Kalpana Biswas, Mary Cowden, Lina Di Blasio, Brenda Johnson, Lisa Kindig, Abby Leafe, Casey Sweet, David Van Nuys, Jeff Walkowski, Foster Winter, and Monica Zinchiak. The perspectives offered reflect those of the author.

As techniques and technology have advanced, the acceptance of online focus groups by qualitative researchers has grown. What began as modified chat rooms have morphed into sophisticated online research environments. Online moderators have developed distinctive styles that they feel produce meaningful results. Professional organizations have sprung up to address the needs of researchers using the online medium.

All of this has led to more questions than ever regarding how to successfully execute an online focus group project. Following is a list of frequently asked questions collected from both experienced and inexperienced online moderators, clients using their services, and vendors that support online focus groups.

Application issues

Q: Can online focus groups be used as a substitute for face-to-face groups?

A moderator must evaluate the objectives of the study and each available methodology to determine whether online focus groups will be appropriate. Online focus groups are another tool available to moderators, not an equivalent replacement, just as telephone focus groups are not a replacement for face-to-face groups.

Q: What kinds of topics or respondents are best suited for online qualitative research?

Online qualitative research is especially good for (but not limited to) some of the following situations:

studies with low-incidence respondents where face-to-face groups are not feasible;
studies where respondents are widely dispersed, including those in rural areas;
studies on sensitive topics, where anonymity is desired;
Web site evaluation or feedback sessions or other topics related to the Internet or computer technology;
B2B professionals with limited time.

Q: When is it appropriate to use the online focus group method?

The general belief among well-seasoned online qualitative researchers is that in some topic areas (high-tech, Internet usage/Web site evaluation, or highly personal/sensitive topics) online focus groups can perform as well as or better than traditional groups. Bulletin-board focus groups (BBFGs) may be the best example of this, as respondents can respond at their leisure and take more time to compose their answers - often generating lengthy transcripts. Probing by a moderator can also be longer and more in-depth.

Situations where online focus groups may not be the best application are:

when capturing body language or facial expressions is vital;
when you need to show prototypes or 3D models;
when products need to be handled, when hands-on usage is critical or touch/feel experience is mandatory;
if conducting taste-testing, commercial testing, or testing ads with heavy copy;
if client material or the topic is highly confidential.

Recruitment and respondents

Q: How do I recruit for online focus groups?

Recruitment method principles are the same as recruiting for traditional focus groups - contact the correct individual, screen for qualifications, invite, and confirm attendance before the group. The main differences will be how the respondent is contacted.

1) Convenient and popular are e-mail invitations and/or screening. While this may seem to be an efficient and appropriate method there are issues to be aware of.

You must know with certainty that the potential respondent has given permission to be contacted via e-mail (opt-in lists).
Many people do not check their e-mail every day and you may lose valuable recruitment time.
Without clear identification or familiarity with the sender’s name, people will not always open unsolicited e-mails.
E-mail addresses change often and people do not take the time to update everyone.

2) Telephone recruitment, identical to traditional recruitment, is usually reliable. Be aware of the sophistication of the respondent base. It is advisable to make some e-mail contact with them prior to the group. E-mail addresses need to be recorded correctly and confirmed as this will be often used to send the URL, instructions, and individual passwords.

3) A combination of both e-mail and telephone works well. Confirmation phone calls used with e-mail contact will elicit a real commitment from the respondent that otherwise may be lost via e-mail confirmation.

Q: Which companies recruit for online focus groups?

Many of the online focus group rental facilities have panels they maintain for recruitment. These panels vary in their scope and reliability. Be sure to ask how panel participants were collected to assess your comfort with their database. There are very few services that cater strictly to online market research recruitment. Some of the online recruiting services are new to recruiting so it is important to ask them to describe their recruitment steps, review their correspondence with respondents, and establish the timelines they will adhere to.

A few of the bonus-point vendors, such as My Points or Flooz, have programs that you can utilize to recruit respondents. Keep in mind that these vendors generally are looking for large participant commitment and the cost to the project can be high. The trade-off here is that their databases are enormous and the participants are cooperative.

Lastly, your most trusted database may come directly from your client. Client resources are typically reliable and also cost-efficient. However, if you are using a client’s database make certain the people have given their permission to be contacted via e-mail. You can increase the effectiveness of this method (response rate) if the originating correspondence comes directly from your client.

Q: What show rate can I expect with online focus groups?

The show rate is highly dependent on the type of respondents being recruited and the recruitment method employed. For typical consumer groups, recruited solely online and confirmed online, show rates are generally lower than face-to-face groups; for eight participants, it is recommended that at least 16 be recruited. Response rates tend to be higher when recruiting is done via telephone or at least confirmed via telephone with comparable over-recruitment. Problems with respondents’ ISPs and/or traffic on the Internet can also affect your show rate.

Q: What are the issues to keep in mind when screening for online projects?

If you are using a Web-based screener you will want to keep in mind how the question will be read or interpreted. Some moderators feel it is possible to have more extensive screening questions using a Web-based screener because they feel respondents can answer at their leisure. Other researchers disagree and prefer to keep Web-based screening simple and to-the-point. You may want to ask at least one open-ended screening question to give you a feel for how well the respondents express themselves online. Telephone recruitment follows the same guidelines as with traditional groups.

Q: How representative is the online population?

Some would argue that the online user is more affluent than the average consumer, but every day the online population grows and becomes increasingly more reflective of the general population. According to an August 2000 study by the U.S. Department of Commerce, approximately 42 percent of homes are presently connected to the Internet. Online focus group topics tend to be better directed towards users of the Internet or projects that surround Internet issues. A possible advantage to using the online population is the ability to reach lower-incidence respondents more efficiently.

Q: What are appropriate incentives for an online focus group?

Incentives tend to be in line with traditional focus group incentives. At this time, $40 seems appropriate for a typical consumer real-time group running 90 minutes. As with traditional groups, the lower the incidence generally the higher the incentive to motivate those individuals to commit. Similarly, B2B professionals will require a higher incentive.

BBFG incentive practices are not well defined yet. The incentive amount will greatly depend on the time commitment you ask from the respondent, and in some cases, the affinity they have for the topic. For example, doctors may have great interest in sharing with peers their experiences with a new pharmaceutical and may not require large incentives. Keep in mind there is a trade-off with BBFGs: Respondents can participate at their convenience but they are asked to dedicate a significant amount of time during the run of the study.

Q: What is the right size for an online focus group?

Just as with traditional focus groups, if you have too many respondents you sacrifice time and depth of research. In the end, your project will determine its own right size but generally eight respondents for a real-time group are easier for the moderator to manage, observers to follow, and participants to build a relationship. The size of the transcript generated must also be factored into your analysis time.

Larger groups are recruited for a BBFG because the attrition rate is high and respondents have more time to read and post. Again, the complexity of the discussion and the length of the board will determine the appropriate size. Some projects may require as many as 25 participants; a more common number is 15-20. Keep in mind, too many participants in BBFG can lead to a large number of postings and an increased likelihood that not every participant will take the time needed to read them all.

Q: Is there a certain type of individual that is best suited for online focus groups?

Individuals need not be articulate, but they do need to feel comfortable with the online environment and be able to express themselves using words. They should also have adequate keyboard skills. BBFGs work well for individuals who have less availability for real-time groups, like B2B professionals. Real-time groups work well for consumers, but are not limited to only this type of online group. These are generalizations and the complexity of the subject and commitment level of the participants may determine which methodology will work best.

Q: How do I make sure the person sitting on the other end of the computer is who they say they are?

Telephone confirmation and re-screening help to some extent. If a respondent does not seem to be qualified the moderator can send a private message to the respondent and ask them qualifying questions, and it is generally simple to remove them without being obvious to the others in the group. You may find it prudent not to send the URL or password to the respondent too early - no more than one to two days prior to the group, giving them less time to pass this information along.

The mechanics of the online focus group process

Q: How secure is the online environment?

Most facilities have designed their software to provide password protection for participants and observers. Some facilities claim to have the ability to protect visual concepts through print-screen security measures. However, if stimuli (visual or descriptive) are highly confidential, you may not wish to incur the risk that such intellectual property might be compromised.

Q: What kind of visual stimuli can be presented in online groups?

Any image that can be transferred into a graphic file or Web page can be displayed during an online focus group. Online facilities that have this capability offer a couple of different options for presentation: a separated frame, known as a whiteboard (similar to Microsoft Netmeeting); or a new browser window that opens on the respondent’s desktop. Active Web pages can be presented in the whiteboard area and static images are most often required to take the .jpg format. Be aware that the whiteboard is not a full frame image and respondents may need to scroll up and down or sideways to view the entire image.

Spawning a new browser window is a nice way to show true form Web pages, but there are concerns with this approach. One problem is that your respondents have now been taken away from the discussion. This browser page is live, and the moderator has no control over the respondent’s actions when viewing. Some respondents may get distracted by this new window and may not return to the discussion in a timely manner. It can be difficult for the moderator to regain control of the discussion in this situation.

Q: Can clients view or observe the group? To what extent can clients be involved with the moderator?

In most virtual facilities, clients can view the groups without the participants being aware of their presence. In some facilities, clients can chat with other clients, clients can chat with the moderator, and tech support people can communicate with the moderator, all without the respondents knowing. Beware: Clients can overuse this feature, taking too much of the moderator’s attention away from the session. The moderator must establish protocols with the client regarding private messaging during the group.

Q: How do you schedule groups to accommodate time zones?

With real-time groups, evenings are still the most popular for consumer discussions, but depending on the targeted population, daytime groups can work. The timing of online B2B groups varies just as it does with traditional B2B groups. Timing doesn’t have to be a huge hurdle if all your respondents are in close time zones (i.e., North America), but what happens with cross-continent participants? Whenever possible, it’s best to group them within a few time zones of each other. In any case, make sure that your respondent knows the correct group start time in his/her area.

With BBFGs you do not always have to consider time zones as the respondent checks the bulletin board at his or her convenience. For close time zones, new topic threads are generally posted in the early morning to accommodate the early birds who log-on.

Moderation and analysis

Q: What moderator skills will I have to call upon in an online focus group?

The same skills are required from the moderator whether it be online or offline. Extra thought should be put into writing the discussion guide to eliminate misinterpretation of the questions. Probing questions should be phrased to welcome responses from all respondents. Clarification will be even more important in the text environment as you will not be able to use non-verbal clues to determine the meaning of the respondent’s answer. You may find analysis more or less demanding depending on how much you rely on the transcripts. Familiarity with chat room culture/slang and competent typing skills are also helpful.

Q: What projective techniques are possible in online focus groups?

Some projective exercises are easily adaptable to the online environment, while others are difficult to transfer. Obviously, the successful techniques are those which are mostly verbal - “What kind of animal does this brand bring to mind?” - something respondents can think about and then put into words. Unique projective techniques are likely to be developed as this methodology matures.

Q: How is the moderator’s guide handled in online focus groups?

Careful thought must be given to how the question is composed and the language in which it is written. How the respondent interprets the question, particularly when the discussion pursues a path not anticipated, should be thought out and tested. Many online focus group facilities allow you to pre-load your discussion guide, eliminating the need to cut and paste questions during the real-time discussion. With these preloaded questions editing is possible, but this takes your attention away from the respondent’s answers.

BBFG discussion guides can also be preloaded (even automatically posted by the server without you manually entering questions) with editing being seamless because the group is not running in real time. This gives the moderator much more editing flexibility and accuracy. In addition, the client can be consulted about the path of the discussion to address issues that arise, making it possible to re-develop the discussion guide on the fly.

Q: Are individuals more or less candid because of online anonymity?

Most researchers who have used online groups agree that online respondents tend to be more candid and direct. Generally, respondents compose their answers before reading others’ postings. Therefore they express only their opinions, not a reflection of the peer pressure sometimes experienced in traditional groups. Respondents also seem to express their opinions without the tempering sometimes seen in traditional groups. For example, respondents feel more comfortable giving you negative or controversial feedback (“This ad sucks!”) if they are not sitting at a table where people might verbally challenge them.

Q: Do dominators exist online as with in-person groups? What techniques can be utilized to control respondents?

It is difficult for a person to dominate an online focus group due to the text environment; everyone is an equal with no body language, facial expressions or perceived social status. A real-time group may contain someone who types a longer response, but since responses are generally posted in groupings there is no sense of dominance by one respondent. In addition, most respondents type their replies before reading the postings of others and therefore are not influenced by their fellow participants. However, it is not impossible for a dominant respondent to surface online. Some of the online facilities have blocking and private messaging features that will help you deal with the problem respondent without alerting the other participants.

Q: Do you wait until all respondents answer one question before moving to the next?

Respondent typing skills, server refresh rates, and Internet lag time are some of the factors that delay by a few seconds the posting of responses. Give the respondents time to read the other postings and the moderator to probe before moving on to the next question. The group will move smoothly despite the lag time. You will develop a rhythm in the group that will dictate when to move on. It is not always necessary to have all postings before doing so. Some moderators use a general rule of waiting for at least two-thirds of the responses to appear before jumping to the next question.

Q: How does the absence of body language affect the results?

Face-to-face focus groups and online focus groups should not be compared. Instead, the objectives of the project should be evaluated to determine which methodology will best serve the project’s needs. If you apply the online methodology appropriately, the absence of body language will not be an issue. The learning achieved using online focus groups can often be more in-depth, as with BBFGs that generate a great amount of data. Currently there is not much information surrounding how the lack of body language impacts qualitative findings through this methodology.

Q: What is the attention span of a respondent and how does a moderator maintain the enthusiasm and interest level of the group?

The enthusiasm of the group can be affected by a number of variables, including:

the interest level of the participants;
the time of day - participants can be tired late in the evening;
the changing of visual stimuli;
the use of humor, html language, and punctuation;
warm-up exercises;
spirited conversation in the waiting room;
the pace and tone established by the moderator.

Ninety-minute online focus groups are the most common, but this does not mean that longer groups will be less successful. Keep in mind that respondents are reading from a computer screen continuously and tend to get burned out if the group is too long. However, as with traditional focus groups, if respondents are highly engaged by the topic you will find it difficult to get them to leave the room. Scheduling groups with a 30-minute break in between will help if the group runs overtime. This also gives the moderator a short break.

Q: What techniques help make BBFGs less like serial questioning?

Due to the large amount of text, respondents may not take the time to read everything written in a BBFG. Instead they may choose to only answer the moderator’s daily posted questions, resulting in something that resembles simultaneous one-on-one interviews. Encouraging respondents to read previous postings on a daily basis and respond to them can reduce this tendency. It is important that respondents understand that interaction with other participants is one of their responsibilities. Some helpful suggestions:

Send out e-mails to inactive participants directing their attention to a particular posting.
Try not to add any verbal clues to your postings that might be interpreted as closing a discussion thread.
Let respondents know that you will keep the BBFG open for an additional couple of days so that they may take time to add or review anything they may have missed.
Formulate questions that require participants to give you a timeline of their behavior. (This is what I used to do… This is what I do now… I plan on doing this in the future…)
Stay away from closed-ended or brief questions that only require brief answers.

Q: What are the best ways to keep attrition rates down for a BBFG?

BBFGs require a high commitment from respondents and they will lose interest if the discussion is not engaging. Keeping the topic lively and including probing several times a day helps keep participants engaged. Respondents will need to understand their visit requirements at the outset. E-mails and/or a tickler on “Today’s Topic” serve to remind participants of their promise to log on daily. Some BBFG software offers a feature that sends respondents an auto-notification that their posting has received a reply. This is very effective in getting them to return and read the other postings.

A daily incentive, such as paying them $20 for each day they visit twice and leave a thread in the discussion, is another approach that has been effective for some BBFG moderators.

Q: What is the best way to handle lengthy transcripts when doing analysis?

Analysis techniques are a highly individual process. Transcript use in analysis will vary from moderator to moderator. Online focus group transcripts may be easier to work with because of their formatting; most are in a CSV (comma separated value) format which can be easily manipulated or merged with other data. Ask the online facility if the transcripts can be exported to Microsoft Excel or Word or whatever format you feel most comfortable with. In addition, transcripts are generated immediately and can be reviewed while the group is fresh in the moderator’s mind. Keep in mind BBFGs generate an enormous amount of text and this requires additional reading and analysis time.

Q: How important are the keyboard skills of the moderator?

The keyboard skills of the moderator can be a limiting factor to probing on-the-fly. The availability of facilities that offer pre-loaded discussion guides and generalized probing questions make a moderator’s typing speed less of a factor. However, communication with clients via private message will be difficult if the moderator is a slow typist or is acting solo (without a typing assistant) during real-time groups. Some first-time online moderators choose to use a typist in order to familiarize themselves with the dynamics of an online group and not become overwhelmed.