What was lost, what was found

Editor's note: Bj Kirschner is director, business development at New York research firm Just Worldwide.

Tap below all that apply:

• In the past week, I have made a purchase using my credit card on a mobile device.
• In the past month, I provided medical details to a physician, physician’s office or other health care practitioner on a mobile device. 
• In the past day, I have posted pictures or videos of myself on social media.
• In the past second, I realized I was being asked to tap on a magazine article.

Whether you wondered at the initial instruction or at the fourth choice if the writer of this article was batty, I am guessing you were fully prepared to tap. We all do it all day long on our mobile devices. Even I do, and I am, at best, an un-early adopter of technology.

Fossil that I am, my original reaction to mobile applications was negative, though not because I am a technophobe who blames every change in eyesight on small devices but because they were laughably rudimentary. The applications looked ugly, did not take into account different types of devices, often required downloading of other apps just to run the one needed and, of course, your fingers ached just looking at the amount of tapping required. I kept expecting DOS windows to pop up. Oh, and then companies wanted their own bespoke platforms, meaning that respondents would confront unforeseen and different challenges for which recruiters could not prepare them because they were new to everyone. We smoothed that all out and now health care market research is filled with technologically sound and unique mobile applications used across the world, even in countries where technology is less welcome. There is no bigger fan of them for market research use than I.

As we know, 2020 and 2021 have changed the way we use technology and market research has not been exempt. Those of us on the health care side were fortunate that COVID-19 did not stop our work but rather forced it to change dramatically and quickly. The lifeline of course turned out to be mobile applications, either alone or partnered with web-assisted telephone interviews. A natural fit, it made for a fairly smooth pivot. But two issues that have always existed regarding mobile apps suddenly took on giant importance: over-worrying and overworking. Those terms may sound scary but they are actually positives.

Except when we forget how they all started.

In this article, I will combine what was learned from three rare-patient recruits in 2020 and 2021. Two studies – one with patients who had recent procedures for rare heart conditions and one with late-stage cancer patients – started as traditional in-home interviews but had to change to mobile ethnographies. The third, focusing on patients with a very rare genetic condition, was always a mobile ethnography but the realities of COVID-19 forced changes upon it.

Affects everyone in the process 

Let’s start with over-worrying, as it affects everyone in the process from the companies sponsoring the research to the companies managing it and of course the respondents. I am using the term “over-worrying” as a fear of the process before it actually begins. Market research has always done that. We spend longer writing and approving screeners than we do conducting fieldwork. 

COVID-19 came and we in qual moved everything to mobile apps and video chats. The over-worrying on the client side started with the wondering of how much time would be needed by the respondents to adapt to the new methodologies. In the cardiology and oncology studies, the original study design had a moderator spending a half day with the respondents at home. Experienced moderators seamlessly sweep up answers in free-flow conversations where the information sought comes up as organically as possible. If we switch to a series of online questions and requests for photos and videos, will we lose the natural ease in favor of checklists that will be going on for a few weeks? Will respondents hit a wall of burnout if we ask too much work of them in a short period of time? And, of course, will the patients even feel motivated, given what is going on around them? 

The patients, and their caregivers, worried too. Could they figure out the technology? How personal would the questions be? Would the video requests require them to openly discuss intimate subjects and require them to be seen on days they may not be at their best? One cancer patient declined to appear on camera simply upon being told there would be video clips required yet she had been okay with people recording her for five full hours. Perception is everything and she was okay with someone else using a camera as opposed to her doing it.

Remember, the original methodology required a stranger or two hanging out with patients for a few hours in their homes. While the clients were worrying about how to move from the idyll of comfortable chat for a few hours with a video camera that goes unnoticed after a while, the respondents were worried about an increased depth of required information.

Confronted with a brand new situation, everyone was so focused on getting or giving the same information that they broke up the original study design into puzzle pieces and then tried to refit them for the same outcome. Some pieces had to be shaved to fit but ultimately the pieces went back together, mostly. All of this is natural, healthy over-worrying. 

But one step was skipped, one we in the qualitative world see as our strength: communication. Perhaps the mobile apps could be used first to “speak” to the respondents and ask them some basic questions about how they felt moving forward with the process. This was instead accomplished with two phone calls (one to set a time, another to speak) but doing it through the mobile app itself would have been not only very fast but also a small technology test for everyone. We over-worried ourselves into complexities when simplicity was staring us in the face.

While the mobile app exercises were being programmed, the overall study designs were decided. Two studies follow the same pattern: a recruiter booking interview times, an introductory phone call from the client, a few e-mails with instructions, exercises released at the same time each week for four weeks with video chats after the exercises. Five phone calls, four weeks of exercises, three (dozen) e-mails – and a partridge in a pear tree.

Did the new design actually address the worries? The clients, wanting the respondents to feel comfortable and not burdened, relied on the benefit of time to take care of their worries – the less required at a time, the less chance of fatigue. The respondents’ issues of what would be asked and how comfortable they would be had to wait until it all started, to find out if their fears were sound, but the chance to talk to a person over the phone was a worthy attempt at keeping it more personal. The clients had the respondents’ best interests at heart; they were understanding and solicitous. But, could they have gone one step further and asked this in a few short mobile app questions, once again a technology check, but this time more personal because it openly asked for specific opinions in shaping the process individually?

No one was wrong 

I should say that that no one was wrong in how they made decisions. The world was in chaos, we were over-worrying 24/7 about life-and-death issues, no one had the answers. My only point is that over-worrying by groups of people who seek solutions by trade was so intense that we may have made things harder for ourselves, the step when worrying becomes over-worrying.

When the new design was presented to the respondents, their first thoughts were ones of intimidation – in other words, overworking. Yes, they were being paid hearty incentives, their schedules were being respected and no force was ever exerted but they saw the process as long and technical, personal in the fact that they were sending videos of themselves but at the same time impersonal because it was all happening over little devices. 

For 75% of the respondents, everything went pretty much as planned. The respondents sailed through the exercises, tossing aside fears. Some found themselves so deeply invested, so proud of their work, that success created more diligence and more success. 

Most of the exercises required maybe 10 minutes a day. But the respondents, aiming to please, often spent triple that much time. A video request to answer a “how are you feeling today” netted videos of 20, 30, even 60 minutes – the step when working becomes overworking. The clients were thrilled with the information. They did not want to risk anyone losing steam, so at times they asked that the recruiters perhaps gently remind the respondents that no one was trying to take up so much valuable time – but not in a way that made it seem like a bad thing. The respondents didn’t even realize how long they were spending.

Did we lose a chance right there? Why have the recruiters broach the topic of too much time? Why not program a question on the mobile app that is a self-assessment? You get an honest answer and it takes one less phone call. Like the other missed opportunities, a small one, but missed nonetheless. 

Not everything went so well. In the end, mobile apps are still technology-based, which means total control is not in the hands of humans. I have not mentioned until now that not all of this work was in the United States. It’s a logical assumption, as where else do respondents go on at such length? One of the studies was in the U.S., Europe and South America. South America proved challenging because some of the respondents were in very remote locations where internet conditions were not strong enough to handle video uploads of 10 minutes (or lots more). We went through a raft of solutions. One was: Let’s send cameras! No. Turns out postal employees were not showing up to work for fear of getting COVID and mail was at a crawl. How about friends and family, anyone with better internet speeds? No. That would be breaking quarantine. It was tough finding solutions but eventually we worked it out with each respondent individually. As soon as we did, we realized we skipped the obvious step: use the mobile app to ask the respondents their suggestions.

Filled with valuable insights 

Did over-worrying lead to overworking? Yes, absolutely. Did over-worrying make things more complicated? Yes, but in the context of a global pandemic, not with harm. Did the respondents overwork? Yes, but every second of it was filled with valuable insights and nary a complaint. Was client over-worrying valid? Absolutely. Was respondent overworking burdensome? For a few respondents, yes, but overall, no.

Did mobile apps live up to their full potential during the worst of the pandemic? No, but only because we were all so focused on over-worrying and overworking that we missed the small details: we missed making use of the mobile apps for tiny scraps of feedback. Did that hurt or hinder market research, the use of mobile apps, the willingness of respondents? Of course not. But, as we face a world where not everyone is going back to the old way of doing things, is it time to look at this methodology again and see how it can benefit us in ways we overlooked amidst that hazy mix of great intentions and general uncertainty?

Tap below all that apply:

• I will always over-worry; all market researchers do.
• I will always allow respondents to overwork if they are not complaining.
• I will try to look for simplicity among complexities.
• I will once again fall into the trap of you trying to make me press a button in a magazine article.