Seven ways to fail a consumer label comprehension study

Editor's note: Morris S. Whitcup is founder and president of Whitcup Associates, a New York research firm, where Keith LaMattina is director of research services.

The Food and Drug Administration (FDA) is charged with ensuring the safe use of prescription (Rx) and over-the-counter (OTC) drugs in the United States. The FDA is vitally concerned that consumers understand the correct use of OTC medications and has issued detailed Guidance about proper drug facts labeling. The Guidance requires that the outside packaging and/or the container itself have information about: the name of the medication; its ingredients; uses; warnings and contraindications; dosing; and other information that may be deemed pertinent by the manufacturer¹.

The FDA usually requires that a consumer label comprehension test be conducted:

• when a product switches from Rx to OTC status;
• is newly introduced;
• when there is a new indication or target market;
• if changes are made in directions or warnings; or
• if there are other substantial changes in formulation or dosing.

Guidance for this type of research was issued in August 2010² and forms the basis for this article. Commentary is also included based upon actual experience in conducting such studies. Although the manufacturer’s following of the FDA Guidance is not mandatory, the Guidance is evidence of the FDA viewpoint on these matters and any deviance from the Guidance should be carefully considered.

Potential failure #1: Not (adequately) specifying primary and secondary communication objectives

The Guidance specifies that both primary and secondary communication objectives (if any) be detailed before conducting label comprehension research. Endpoints, based on comprehension levels in specific questions, are also included in the protocol that companies prepare before conducting a label comprehension study. Sounds easy, right? Actually, setting these objectives is a complex task:

• The endpoints should be based on clinical criteria and pre-study discussions with the FDA. Correct communication is critical for clinically relevant items (such as warnings and contraindications) and less important for “nice to know” information.
• Including too many objectives may lead to failure, inasmuch as success or failure will be judged based upon achieving each of the endpoints.
• Having too many questions or items addressing each endpoint may result in only a partial success; with lack of correct consumer comprehension for some and higher levels of correct understanding for others.
• Setting the bar too high for desired levels of correct communication for endpoints can lead to failure. Not only must the specified correct levels of response be achieved, they must also be statistically reached at the 95 percent level of confidence. Thus, for example, achieving 85 percent correct comprehension may necessitate a 90 percent correct level of comprehension because of the range for the statistical confidence associated with each response.

Potential failure #2: Conducting Internet interviewing

For many business decisions of consequence, Internet interviewing predominates. Such interviewing is cost- and time-efficient. However, it is inappropriate for consumer label comprehension studies:

• Internet interviewing has no built-in visual verification that a real respondent exists and may have serious population skews.
• The Internet population excludes respondents who have no access to the Internet and is thus a biased population.
• Consumer reading and attention to actual packaging during an in-person interview is far different than perusal of packaging shown on a screen in Internet interviewing.

Potential failure #3: Interviewing only the target market

Marketers are traditionally taught to focus marketing efforts on the target market for a product.³ The target is often based upon prior category or brand use, age or other demographics, attitudinal or psychographic segments, family life-cycle, etc.⁴ FDA Guidance, however, stipulates a broader audience than the medication’s target: “Because the non-prescription drug products are available for purchase without a learned intermediary, and since no drug product is administered in the study, exclusion factors should be minimal (e.g., inability to read and understand English) and should be justified in the study protocol.”

At first blush a broader sample may seem to place an undue burden on companies. A priori, one may assume that target consumers and actual users of the category are more familiar with the medication. However, actual experience conducting label comprehension studies also indicates that target consumers may be less attentive in the reading of drug facts labeling, thus canceling out any familiarity advantages in understanding that target consumers may have.

Potential failure #4: Conducting a true “all comers” study

The Guidance states that the study should include all subjects who can potentially use the drug product, regardless of age, sex, underlying medical conditions and use of concomitant medications. Logically this implies what is known as an “all comers” study without any restrictions.

However, such a “random” sample would not be truly random!

• Shopping malls, where consumer label comprehension studies are typically conducted at interviewing facilities, do not yield a random population. The mall population skews female and younger. Depending upon the specific malls involved, the mall population may have little representation of various minority ethnic populations (e.g., African-Americans, those of Hispanic heritage, etc.) and lean towards a middle-class population.
• Furthermore, the mall population may have few respondents with lower levels of literacy. Guidance specifically states that label comprehension studies should have adequate inclusion of such low-literacy respondents (who score at the 8th grade reading level or lower based upon reading tests such as the Rapid Estimate of Adult Low Literacy in Medicine5). Although augment (“supplemental”) interviewing is often needed to enroll an adequate number of low-literacy respondents, prudent mall site selection can boost the number of low-literacy respondents who naturally would be screened and interviewed.

Study population controls are therefore necessary for a supposedly random sample to be more representative.

Potential failure #5: Not pre-testing the questionnaire and study procedures

As marketers and social scientists, many of us could construct good questionnaires in our sleep. Pre-tests thus seem an unnecessary cost and slow down study implementation. However, this is not the case.

• The Guidance assumes that study questionnaires will be pretested.
• Pretests may uncover lack of understanding of the issues due to questionnaire wording or question/item placement (and not what is to be elicited from reading the medication package). Improving correct understanding even by a few percentage points can mean the difference between obtaining an acceptable level of comprehension or not.
• Label design can also be improved through pre-testing, even by something as simple as highlighting a label facts item that is inadequately comprehended.
• Pre-testing may discover questionnaire items whose levels of comprehension may be increased through use of techniques such as flash cards showing response alternatives, patient scenarios and other items.

Potential failure #6: Accepting a correct response as a correct response

A correct response is a correct response, isn’t it? Actually, Guidance states that correct responses should be probed (“Why do you say that?”) to ascertain that consumers correctly understand the drug facts labeling.

This process does lead to classification of “correct” responses as, in fact, incorrect when consumers give invalid reasons why they answered a question seemingly correctly. However, conversely, some apparently incorrect answers may be uncovered to be, in fact, correct, whereby the respondent evidenced correct understanding but for idiosyncratic reasons answered incorrectly.

Potential failure #7: Assuming that a PowerPoint presentation is the final product

The typical marketing research study is not normally a full report but a not-overly-detailed presentation. Label comprehension studies, however, require a detailed report in Word or other similar software with many appendices, including the protocol, questionnaires, study procedures, drug facts labeling tested, etc., as well as detailed statistically-tested tables of question responses.

Other avenues

While not typical, other avenues of potential label comprehension studies do exist, in particular:

• Arguably, the greatest addition pitfall is changing packaging wording/graphics after the label comprehension study is conducted. This may necessitate redoing the label comprehension study, adding to project cost and potentially delaying the submission to the FDA. The best way of guarding against changes is to fully pre-test the package before the label comp study is conducted.
• Using a comparator in assessing your own label communication is dangerous. There is sometimes the temptation to test an additional label (a competitor) to serve as a benchmark against which to measure the results for your own label. If your label is superior to the competitor, perhaps, to a limited degree, the FDA would be more inclined to judge your own label’s correct communication to be “successful.” However, the ultimate onus is on the manufacturer to demonstrate the success of its own label. Moreover, any ways in which the competitive label is superior would be potentially damaging to the acceptability of your own label’s results.

Forethought and adherence

Label comprehension studies are, in many ways, radically different than many primary marketing research studies. Much forethought and adherence to Guidance are the best ways to help assure successful results.

REFERENCES
¹ “Guidance for industry: label comprehension studies for nonprescription drug products,” Food and Drug Administration, Center for Drug Evaluation and Research, August 2010.
² “Guidance for industry: labeling OTC human drug products - questions and answers,” Food and Drug Administration, Center for Drug Evaluation and Research, December 2008.
³ Small Business Encyclopedia, “target market definition,” ND.
⁴ Wikipedia, “target market,” December 14, 2014.
⁵ Davis, T.C., et al, “Rapid estimate of adult literacy in medicine: a shortened screening instrument,” Family Medicine, 1993, Vol. 25: 391-395.