To field successful medical interviews, run them through these tests

A thorough examination

Editor’s note: John Voda is a moderator and marketing research consultant based in St. Louis.

Medical marketing research is challenging and rewarding. Most respondents are extremely intelligent and engaging, the topics are intriguing and the research results can have life-or-death consequences. But research is expensive and medical interviews run at the high end of the spectrum. The honorarium alone typically costs several hundred dollars and you also have to add to that the costs for survey design, recruiting, interviewing, project management and data analysis.

With stakes - and budgets - this high, it is essential to maximize the quality of information captured. All research is adversely affected by unqualified interviewers, unqualified respondents, poor survey design and many other factors - some beyond our control. Our goal is to eliminate sources of error we can control (the research chain is only as strong as the weakest link). The effort and expense invested to carefully recruit and interview is wasted if doctors misunderstand the questions.

It is critical, therefore, that interview guides be reviewed and pre-tested by an experienced interviewer. A series of pilot interviews can quickly and inexpensively identify and eliminate potential sources of error before the study goes to the field. The small investment will help ensure that questions don’t surface only after hundreds of interviews have been completed around the world.

Following is a checklist of suggestions and considerations regarding how to improve the design and execution of in-depth interviewing. Most apply to all types of research, and several of the suggestions may appear obvious or simplistic. But based on my experience interviewing thousands of doctors and patients, I know the problems they address still occur - and far too often.

Pre-testing

• Invest the time and effort to have an experienced interviewer pre-test the guide with a variety of doctors and specialties. Interviewers have a different perspective than the team that designed the instrument and they bring real-world experience regarding what works in the field. Encourage the pilot interviewer to ask questions and make suggestions. This will help ensure that potential problems are found and fixed before the study goes in the field.
• During pilot testing, ask respondents to provide a disaster check at the end to ensure questions are clear, the survey order makes sense and all relevant topics are addressed.
• For multinational surveys, ensure the survey is carefully checked (preferably pre-tested) in each country after being translated.
• If pilot testing cannot be conducted, encourage the client to monitor the first few interviews.

Briefing

• Provide adequate background information. The more the interviewer knows about the topic and research objectives, the better he/she will perform.
• Ensure field agencies receive and review all materials early enough to obtain client clarifications (if needed) before interviewing begins.
• Conduct a briefing for every study to ensure details are clear and all materials have been reviewed. Encourage field agencies to ask questions and suggest improvements (take advantage of their experience).
• For paper surveys, indicate if answers and interview notes should be captured in pencil or pen, and in a particular color.

Recruiting and fieldwork

• If you want all screeners saved (even doctors that DQ) inform the field offices before the project begins.
• Provide an accurate assessment of the interview length (including any self-completion assignments or patient diaries). Lengthy interviews may negatively impact recruiting. An unrealistically short estimate can improve recruiting but upset respondents and impact the research quality. For longer surveys, compensate respondents accordingly.
• Avoid changing quotas or changing the screener after fieldwork begins unless essential.
• Don’t sacrifice in-field time to make up for other delays. Provide field staff with adequate time to recruit and interview. If the time frame is unrealistic, recruiting will suffer and field offices may be tempted to cut corners.
• Provide a contact that interviewers can call if they encounter a problem or need clarification during an interview.
• Encourage interviewers to provide feedback so you can quickly identify and resolve issues that arise. Promptly review and share with all field offices relevant new questions/clarifications so everyone can benefit. Ensure interviewers make no changes without the client’s explicit approval.
• Respect the doctor’s decision to not participate in marketing research. Provide an option to be removed from future study invitations.
• For taped interviews, encourage interviewers to take notes on the paper survey as a backup to the recording equipment and to aid in probing when referring to prior answers.
• If the survey is administered electronically, ensure there is a way to capture comments/clarifications on the computer during the interview. The doctor/interviewer may forget them by the interview’s end.
• When feasible, allow backing up (in the event an answer is captured incorrectly). If not possible, provide a log for capturing comments or edits. Ensure the form clearly captures the question(s), the required edits and the unique respondent ID.
• If the interview will generate medical waste, provide the field offices with sharps boxes for proper disposal.
• Assign respondent numbers when the study begins (if possible) to ensure respondent confidentiality and reduce confusion. Ensure all paperwork and forms include space for recording the number.
• Ensure all paperwork has the date and/or version number displayed so that if edits are made the field offices can distinguish the old from new paperwork. Ensure the client/product/ brand does not appear anywhere.

Formatting

• Before printing materials, fix page breaks to ensure questions and/or answer options don’t wrap onto two pages (which could result in the question or the answer options being misread, misunderstood or missed altogether).
• Use bolding and underlining to highlight differences between questions that might otherwise seem identical (such as conjoint profiles).
• Indicate words that should be emphasized when read aloud by using bolding, underlining or uppercasing.
• Ensure there is adequate space to capture a complete answer to every question. If you provide a small space for a question that warrants a long answer, the interviewer/respondent will likely summarize or eliminate valuable comments.
• For long lists, use lines, shading or borders to ensure doctors can easily (and correctly) indicate their answer (such as when matching attributes with drugs). If several options are similar, underline or bold the key differences.
• Unless a list must be randomized, sort items alphabetically or group them logically (such as side effects, then contraindications, etc.) to make it easier for the interviewer to record responses.
• Do not combine a series of questions into one long block of text. Break each question into a separate item. Similarly, if a probe or instruction is actually a unique question, format it accordingly to ensure the interviewer sees and asks it.
• For taped interviews, when asking several questions about a list of items, create a grid so the interviewer can keep track of what has been asked and can record any explanatory comments from the doctor.
• Ensure questions with precoded answer options accept an “Other - specify” option (unless absolutely essential not to) and capture the verbatim comment. Don’t force doctors to select the wrong answer because the right one isn’t allowed.

Instructions

• Make instructions easy to read, complete and precise.
• Clearly indicate when answers must sum to a specific amount or equal other numbers (such as 100 percent, a subtotal or a previous answer).
• Many people are confused by the symbols < (less than) and > (more than). Spell them out and review them in the briefing to ensure questions are asked correctly.
• Use formatting to clearly distinguish interviewer instructions (which are not read aloud) from questions (which are read aloud). Use bolding, italicizing and brackets for clarification. Do not embed questions within interviewer instructions.
• Clearly indicate if probes and answer options are to be read aloud. Do not leave interpretation up to each interviewer.
• Don’t ask the interviewer to refer to prior answers unless it will clarify the question, enhance the accuracy and/or expedite the interview.
• Ensure skip patterns are clearly marked and keep them to a minimum. Ensure skips are correct - especially if question numbers are changed due to additions/deletions.
• When answer options are to be read aloud by the interviewer, have them first say: “Please do not answer until you have heard the entire list.”
• Provide instructions for the interviewer to read aloud before giving out paperwork/self-completion forms. Once the paperwork is handed over, the respondent may begin reading it and not pay attention to the interviewer. When using a show card or self-completion form ensure the doctor reads the entire text before answering.
• If questions or forms are to be rotated or randomized, ensure instructions are clearly marked (and prepared in advance when possible).
• When asking a series of questions about a series of items, clearly indicate if the questions are to be asked vertically (ask all attributes for one item, then the next item, etc.) or horizontally (ask one attribute for all items then ask the next attribute). Answers can be impacted by the order asked.
• If two attitude questions are opposites of the same issue, instruct the interviewer to challenge and clarify inconsistent answers.
• Clearly instruct whether to rank (1 = most, 2 = second, etc.) or rate (1 = low and 10 = high) items.
• Clearly indicate whether only one answer is allowed or multiple answers are acceptable.

Question wording and answer options

• Abbreviations can have multiple meanings. Ensure all abbreviations are spelled out in the survey (or at least provided in the briefing information).
• Spell out phonetically the pronunciation of each drug or condition that will be read aloud. If conducting a multinational study, ensure lists of drug brands are customized for each country.
• Ensure questions are clear and provide all necessary information. (The brand-name drug or a generic? What dose? What strength? What mode? What usage frequency? What time period? What co-morbidities?)
• Ensure answer options are exhaustive and mutually exclusive. For example, consider a drug where one prescription covers three injections per week, for four weeks. The survey asks: “How many times per month do you use Drug X?” What does “use” refer to? Should the interviewer record one prescription or 12 injections? Office injections only or also self-administered?
• Provide adequate information for the doctor to give a definitive answer rather than “Well, that depends...”
• Order questions to enhance clarity and minimize redundancy. For example, ask “Which drugs have you used?” prior to “Which drugs are you aware of?” and simply transfer answers from the first question to the second. Reversing the order forces doctors to answer twice.
• When using a list, capture yes/no responses for each item rather than “check all that apply” to ensure each option was asked.
• When asking “the average number of patients” or “the average length of use” provide instructions to capture bimodal distributions - such as when half the patients use it for 10 years and half use it only a few days. The “average” of five years does not convey the true length!
• When asking use of drugs or procedures, don’t ask yes/no questions - which don’t capture nuances. Provide options such as “I used to but recently stopped” and “I typically don’t, but would if the need arose.” Similarly, it is usually better to capture intended use on a likelihood scale rather than as a “yes/no” question, since the answer often depends on numerous criteria not explicitly provided.
• Capture an exact number rather than a range (when possible). The values can be collapsed into categories later. If ranges are necessary, ensure the number of categories is adequate (for example, don’t break a 0-100 scale into just two ranges).
• Ensure attributes/statements are meaningfully different, so doctors don’t ask “Didn’t I just answer that question?” If the difference between phrases is subtle, underline, bold or uppercase the key words that are different.

Show cards, handouts and self-completion forms

• When using a show card with a long list of answer options, number the options to make it easier/faster for the interviewer to find and record the answer. Ensure the numbers on the show card match the survey.
• When creating charts/graphs, use shading/shapes that will remain apparent if copied/printed as black and white.
• Use show cards when there is a lot of text for the doctor to evaluate, to ensure comprehension.
• Only use separate handouts or show cards when necessary to enhance understanding or to expedite the interview.
• If paperwork must be faxed, ensure it is formatted on standard 8.5 x 11 paper (rather than legal).
• Ensure the question matches the handout or self-completion forms. For example, don’t ask “importance” while the handout reads “performance.”
• Ensure instructions clearly indicate when each self-completion sheet and show card is to be used. Ensure each is clearly marked and the label matches the survey. If possible, name the show card as the question(s) it is used for. Ensure question numbers are updated if questions are added or deleted.
• When pre-interview assignments are given (such as patient), ensure the doctor has adequate time to complete the materials prior to the interview. Call prior to the interview to ensure the doctor has completed all assignments. If necessary, reschedule the interview rather than have the doctor rush through the materials.
• For paperwork sent to the doctor prior to a phone interview, ensure all forms are clearly marked in large text on each page whether they are to be read/completed in advance or during the interview.
• Have field offices submit the original paperwork the doctor completed rather than transferring answers onto a new sheet, which could result in errors.
• E-mail or ship paperwork rather than faxing (when possible) to maintain legibility. If paperwork must be faxed, ensure shading isn’t too dark, the type isn’t too small and items aren’t too close to the page margins (where they will be cut off). If an image must be faxed, ensure it is a line drawing (versus a picture) so the fax will be clear (shading can make faxes unviewable).
• If paperwork must be mailed back by the doctor, provide a pre-addressed, postage-paid return envelope.
• Ensure contact information (including your phone and fax number) and the project deadline is clearly displayed on materials sent to the doctor.
• Ensure the respondent number is recorded on all materials before sending it to the doctor.

Considerations specific to medical marketing research

• Indicate if samples (i.e., not prescribed) should be included or excluded when reporting drug usage.
• Price/insurance/coverage/formulary frequently has a major impact on prescribing. Provide relevant instructions to ensure consistent answers.
• When asking “What percentage of your patients discontinue treatment with Drug X?” clarify whether patients who die while on it should be included or excluded.
• When asking if a doctor would switch a patient from their current drug to a new drug, the instructions should specify if the patient is currently performing well or poorly.
• When asking about workload (“patients typically treated per month” etc.) it may be more accurate to capture subtotals as percentages rather than numbers. The percentages can be converted to numbers during analysis and vice versa. The interviewer can use percentages to calculate a number and verify it with the doctor: “So, that would be about X patients per month?” (This provides more accurate data, especially when splitting small bases.)
• For multidoctor practices, it is probably unrealistic to have a doctor estimate how many patients the entire practice treated in three-month or six-month period. If asked, capture how many doctors are in the practice.
• Eliminate ambiguity. What does “problematic” mean in the following question: “Do you experience any adverse reactions in patients on Drug X that are problematic to manage?” Does it mean “required titrating” or “required stopping that drug” or “required adding another drug” or “caused side effects”? One doctor may consider a side effect that is treatable (though uncomfortable) not to be problematic. Another may consider patient inconvenience to be problematic.
• Does “it limited your treatment” mean “it caused you to discontinue treatment” or “it caused you to change the dosing” or something else? Don’t leave questions open to varying interpretations.
• Don’t ask “Why did you prescribe Drug X to this patient?” (which may result in a discussion about drug classes) when you really want to know “Why did you prescribe Drug X to this patient rather than some other drug?” (which captures how this specific drug differs from other drugs). Similarly, don’t ask “What do you seek from a drug rep?” when you want to know “What personality traits do you want in a drug sales rep?”
• When creating drug profiles, be as exact as possible. Reduces by how much? How quickly? Sustained for how long? Don’t leave it to each doctor to speculate what is meant. The more precise the profile, the more accurate and comparable answers will be.

Enhance the value

If you follow all the suggestions above you will greatly enhance the value of your research. If you remember only one thing it should be this: leave nothing open to interpretation. While it is easier to rely on the experience of the interviewer, your trust may be unfounded and your data could be jeopardized. The effort you spend refining your survey instrument directly impacts the quality of the research results. And with medical research, your findings may have lifesaving implications.