Editor's note: If you’re an end-client researcher and interested in participating in a Q&A with Quirk’s, please e-mail me at emilyk@quirks.com.
Some of your research has involved spending extended time with families outside of the U.S. Could you describe one of those experiences?
Many years ago I had an opportunity to partner with a mobile phone maker’s design team to research product needs for a device targeted to the 4 billion people who didn’t have access to the category. At the time, CK Prahalad’s “Fortune at the Bottom of the Pyramid” was driving strategy with global companies as they worked to expand the total addressable markets for their product portfolios. Instead of trying to force existing solutions into those markets, we decided to get firsthand knowledge of the lives of people who were in this market and we chose India as the place to invest our time.
We utilized an ethnographic methodology and immersed ourselves into the life of a family that lived in a village about four hours outside of New Delhi. And when I say immersed, I mean it. For four days, we lived under the same roof as this family. We slept in the same house. We ate with them. We ran errands and worked with them. The village wasn’t modern by our standards. The home was made of wood and cement bricks. There were a few rooms with hard clay as the floor. There was no electricity or running water in the home – in fact, one of my contributions was carrying water twice a day to the home.
Our task was to live the life and assess how to create a product that would work for this family. The company I worked for made products that had sensitive and fragile electronics and through this immersion it became very clear that our traditional mobile phone design wouldn’t work in the conditions in the village. Dust, dirt and moisture would be a problem. Temperature fluctuations would be a problem. Battery life would be a problem – it’s hard to charge a phone daily when there is no electrical outlet to plug it into. The things we learned about what a phone had to be couldn’t have been learned in a survey. We had to experience them and live them. We then married these experiences with the connectivity needs of the family we were living with – who, as a side note, were simply the most gracious, kind and forthright people I have ever met. It was one of the best research experiences of my life.
Do you have a favorite research technique?
For the past few years, we’ve been experimenting with how to use organic social data to answer business questions. Our program started looking at the basics – social volume, sentiment and key themes surrounding topics important to Microsoft, such as Surface in an NFL game or response to stories around JEDI. But then we started expanding our capabilities to try to answer specific business questions.
One recent method I’m excited about is our social satisfaction analysis, which we recently deployed to look at Microsoft Teams. Using advanced query systems, we’re able to scan terabytes of raw, unstructured social data to pick up how Teams users are talking about their experience with the product. What is driving a positive experience? A negative experience? It’s sort of a cousin to a traditional usage and satisfaction quantitative survey. However, what I like about this social technique is that it can be done at the speed of technology. It’s fast – literally takes three-to-five days. It’s cost-effective – normally about 85-90% cheaper than a similar quantitative study. It’s significant – the analysis is built on thousands of raw data points. And it’s organic – there is no survey bias in the results.
When I talk to research purists about the method I get arguments about representation – social represents certain kinds of people and leaves others out – and a lack of an ability to get KPIs, such as a satisfaction metric. They are right in these arguments. However, when you are making product and marketing decisions daily, social offers a continuous data stream that points you in the right direction. If we used traditional research techniques for some of these business questions, by the time we received the data, the market opportunity would’ve passed. I remain incredibly excited about what the future holds for using social data to answer business questions.
What are some of the steps you and your team take to ensure your research is inclusive?
Diversity and inclusion remain in the top three business priorities for Microsoft marketing. I’m lucky to work with a team of researchers [80+] who all share passion, curiosity and a growth mind-set to ensure the team we work on keeps D&I a key priority in how we come together, and stay together. At the same time, we have a team of experts within the overall research organization whose job it is to look at our quantitative research practices and challenge how we’ve done things. They are constantly interrogating on the inclusivity of question wording. Are we asking gender in a way that is representative of society? Are we keeping the number of words in our questions and response rates low enough, so they are easy to take on a mobile device? The team asks these questions, conducts parallel tests and comes up with standards for the organization to use. Additionally, they’ve spent hours working with our research partners to ensure that our sampling frames allow accurate representation of the audiences we are interviewing. The dedication and passion this team puts into their work is amazing and allows Microsoft marketing research, and thus Microsoft marketing, to keep D&I at the forefront of the programs that go to market.
You’ve volunteered as a youth baseball coach for several years. How does this experience impact your work as a researcher?
I’ve been a coach on my son’s baseball team since he started T-ball at the age of four. Even though we live in a large city, the baseball community is small, so I’ve had the privilege of seeing many of the same kids through the years. We are now beginning our third year of kid pitch and it’s been fun to watch these kids excel on the mound, at the plate or in the field – and to be clear, the majority of these kids ran up the third base line in tee ball! This progress has come through two things – repetition and mental toughness. These kids didn’t just become great ball players, they’ve fielded thousands of grounders and swung at thousands of pitches. They’ve slowly been taught the fundamentals of how their bodies work when they throw and hit, and they’ve honed these skills through repetition. At the same time, they’ve developed mental toughness. Playing 20 innings error free and then bobbling three easier grounders in a row. Walking to the batter’s box and nervously realizing that the pitcher doesn’t just throw hard, but is also wild. These situations are frustrating for 10-year-old boys, and they will show that frustration. A key part of being a coach is to help the kids get through this. Most of the boys will tell you that one of the things I say most to them when I can see they’re struggling is “the best part about baseball is that every pitch is a new game.” Helping them reset, focus and progress is one of the most important things I do as a coach.
Now the parallel to research. Repetition is a key element to doing great research. It hones your hard skills, such as questionnaire development, sampling and analytics. It also hones the soft skills, such as immediately understanding that the question a marketer is asking you to research isn’t really the question they are after. You develop these skills through repetition, much like fielding grounders in baseball. At the same time, we’ve all delivered research results that were counter to what the marketer or engineer wanted to hear. The concept the engineer was so excited about was a dog. The insight that the message written by the copywriter wasn’t understood. There is a certain degree of objectivity in representing the customer that we have to bring into the room that requires a mental toughness, especially when the results aren’t going to jive with what the room initially wanted. It’s no different than walking to the batter’s box and knowing the pitcher is the fastest in the league. It’s where mental toughness comes into play.