Machine learning and journey mapping results in 133 customer needs

Q&A with the 2023 winners of the Health Care/Pharmaceutical Research Project award 

Editor’s note: Applied Marketing Science and Boson Children’s Hospital are the winners of the 2023 Health Care/Pharmaceutical Research Project Award, a category in the Marketing Research and Insight Excellence Awards. The award winners were announced during a virtual celebration on November 14, 2023. To learn more about the awards go to https://www.quirksawards.com/ 

Boston Children’s Hospital and Applied Marketing Science (AMS) teamed up to find out what the hospital’s patient journey for pediatric specialty and complex care was like. This project resulted in the two winning the 2023 Health Care/Pharmaceutical Research Project award, a category in the Marketing Research and Insight Excellence Awards.  

The team utilized a novel application of machine learning (AI) to understand the journeys of complex care patients and their families. Through this research, AMS demonstrated machine learning’s ability to produce rich journey maps detailing the phases and stages of patient journeys as well as the underlying customer needs along the journey.  

This research incorporated insights from thousands of individuals in an organized and comprehensive way and provided Boston Children’s Hospital with rich insights to further improve the patient experience.  

Carmel Dibner, principal at AMS, commented on the company's plans.  

“We are continuing to develop our machine learning methodology and identify new ways to use Large Language Models (LLMs) to gather and prioritize customer needs for competitive advantage,” Dibner said. “Our capabilities will continue to expand as technology improves and researchers become increasingly comfortable working with machine learning data to answer critical research questions.”  

Dibner also answered some questions about this innovative project with Boston Children’s Hospital.  

What were the driving factors in doing this project?  

Boston Children’s Hospital came to Applied Marketing Science (AMS) seeking to understand the patient journey for pediatric specialty and complex care. Specifically, Boston Children’s was interested in mapping the detailed stages of the journey and the customer needs associated with each stage. To accomplish this objective and identify unmet needs, Boston Children’s and AMS used machine learning to analyze rich data sources.  

The final journey map detailed 133 customer needs associated with more than a dozen stages of the journey including patient-provider communication, procedures and in-hospital experience and quality of care.  

As part of the initiative, the team also evaluated perceptions of Boston Children’s compared to peer hospitals. 

What were some of the methods you used for this project?  

AMS and Boston Children’s utilized machine learning in the form of the Automated Content Evaluator (ACE™), an algorithm developed by researchers at the MIT Sloan School of Management and Northwestern University Kellogg School of Management in collaboration with AMS researchers. The machine is trained to process massive amounts of readily available user-generated content to effectively and efficiently identify consumer needs and hidden gems of insights. 

For this study, we began with 66,208 sentences of patient and caregiver feedback. We trained the algorithm to distinguish between information that is informative – that contains a want or need – and information that isn’t. The trained algorithm returned 2,000 sentences that were representative of all sentences in the data set. From those 2,000 sentences, AMS extracted 133 customer needs.  

As part of the project, AMS utilized the principles of journey mapping and used the machine to help identify and detail the steps along the patient journey. This was our first attempt to apply machine learning to journey mapping, and the end result was as detailed as the output of a study where traditional market research methods would have been used. 

What is being done for patients as a result of this study?  

As a result of the machine learning study, Boston Children’s has an even deeper sense of the experiences of patients and their families. Throughout the journey, they have identified specific areas where Boston Children’s can further improve the patient experience.  

The sentiment analysis, along with the more detailed patient journey map, identified specific areas where additional family support may be needed.  

As patient family support and services evolve, the hospital now has another avenue, following the learnings from this study, to monitor and analyze changes to consumer sentiment as well as to benchmark sentiment for Boston Children’s against peer institutions.