Editor's note: Paul Jacobson is senior vice president of Greenfield Consulting Group, Inc., and Greenfield Online, Westport, Conn.

We've watched with great interest the swirling debate over on-line focus groups. Some decry on-line groups as blasphemy, the ruination of our craft! Others debate nomenclature or engage in semantics - whether to call a moderated on-line discussion an on-line focus group, or something else, given the absence of that telltale body language.

We recognize that on-line marketing research is in its relative infancy, though growing at Bunyanesque rates. And since the only rule of this game thus far is constant change, we're loath to state any hard and fast rules. That said, we have an unequivocal opinion on the subject of on-line focus groups (or whatever one wishes to call them) - they work!

As a company that has conducted "off-line" focus groups since the early 1980s - and in any given week has up to 10 moderators roaming the country, checking out the body language - it might be understandable for us to be naysayers. After all, why would we get behind a methodology that might cannibalize an off-line business base that has grown for the past 14 years? The truth is, on-line focus groups are not a substitute for the face-to-face thing, and were never meant to be. This is simply an additional tool in the box, meant to productively coexist with "the real thing." And under the right circumstances, clients get substantial bang for their buck (and they get to sleep in their own beds at night, too!).

There are circumstances when the on-line approach is inappropriate. Highly emotive subject matter, in which the skilled moderator peels away layers of "skin" to get to deeply held attitudes and emotions is one. Circumstances in which panelists are asked to work with photographs and icons to build collages are another. (We admittedly haven't yet figured out a way to dump 50 images on a cyber table and have people sort through and pin the right ones on the virtual wall.) Working in certain low-end or commodity product or service categories, where the prospect profile just doesn't sync with the profile of Internet users, is another obvious example.

So what does work? Here are four successful designs to consider, provided as mini-cases.

  • Trend monitoring - when you recruit people who represent leading-edge users of a certain product or service to see what they're up to, what new category language exists and where the category is heading.

A leading manufacturer of office equipment sought to understand meaningful physical and cultural shifts in the office environment, and the effect of high technology on the work space. Greenfield recruited full-time employees who work in open spaces and who have access to the Internet from their offices. The discussion centered on changes in the office environment, encompassing issues such as noise levels, visual and acoustical privacy, and housing and using high-tech equipment. Findings from these sessions confirmed short-term product and marketing priorities and helped generate new product directions and nomenclature, for the longer term.

  • Screening a large pool of concepts to a smaller, more manageable set, for further evaluation - be it qualitative and/or quantitative evaluation.

A worldwide provider of disposable and rechargeable batteries needed to evaluate five new print campaign concepts, ultimately looking to screen down to two campaigns for final evaluation. Typeset color comps from each campaign were shown to owners of cellular telephones in the on-line setting. Panelists were able to spend ample time with each of five print executions, with the moderator subsequently probing on overall impact, main point, likes and dislikes, clarity and credibility. This qualitative work was critical in reducing the number of contending campaigns from five to two, for final decision-making. 

  •  Evaluating the effectiveness of Web sites. This evaluation can be progressive, identifying concept appeal, gaining critical content and navigation direction at the beta stage, and understanding how the final site delivers against the company's Web marketing/brand marketing objectives.

A regional communications company with an aggressive approach to developing on-line businesses had a vision for a new travel-based service for both business and leisure travelers. Prior to embarking on expensive developmental work, the company wanted feedback from heavy business travelers (who also combine leisure travel), to gauge the level of interest and to see what core benefits bubbled to the top. This information was instrumental in the decision to go to the beta design stage, and what specific areas to focus on in site design.

The next phase of on-line qualitative work required prospects to visit the test site, conduct a number of tasks, and share their experiences in a moderated group setting. This activity provided fundamental learning from which the final, live (and now highly successful) version was created.

  • Generating additional diagnostics from recent participants in a quantitative research study. Frequently on-line surveys are conducted revealing areas that are worthy of further exploration. Working in the on-line medium allows researchers to quickly convene an on-line discussion among a group of people who responded in a like manner. This can be a time- and cost-efficient way to get additional input, float some new ideas, or even brainstorm new approaches.

Another global marketer was interested in transferring a broad cross section of their customer service functions to the Web. On-line quantitative research revealed that certain of their market segments found this a highly appealing and practical approach, while certain others were concerned about losing "personalized service." On-line focus groups were conducted among those same dissenters who participated in the quantitative exercise to determine which activities could be handled on-line and which should be done in a more traditional fashion. Importantly, brainstorming among this group resulted in new components being added to the service mix.

Could these groups have been conducted in the time-honored, face-to-face fashion? No doubt about it. The fact is, however, all this work took place on-line, in many cases resulting in substantial time and cost savings - to say nothing of obviating the sometimes disruptive effect of key executives being on the road for a week at a time - who then have to work the weekend to catch up when they return home.

Managing expectations

We have found that the key to successful use of on-line focus groups lies in managing expectations. The first expectation to manage is that on-line focus groups are not the same as "regular" focus groups, and there are limitations to what can be accomplished. Spelling out the pros and cons of the methodology up front is essential to the process. The second step is to give clients a feel for what the experience and deliverables will be, in advance of the session. We make it a requirement that clients participate in a dress rehearsal before the actual sessions take place, so they can decide whether the vehicle makes sense for their research needs.

Fortunately, Greenfield practices both on-line and off-line focus groups, every week. This allows us to be pretty objective about when to use which tool, without having to sell or defend either. Our interest is in creating the best design and working with satisfied clients. Alas, we still haven't solved that pressing semantic issue.