Going beyond the tactical

Editor's note: Michael Carlon is founder and managing storyteller at Vertigo Partners, a Stamford, Conn., research firm. 

I first started conducting user experience (UX) research back in 1997 while working for Modem Media, a digital ad agency. At that time, we were building some of the world’s first consumer-facing websites (mobile was still far off) for clients such as AT&T, Delta Air Lines, Citibank and JCPenney. We also built and ran the first banner ads; yes, you can blame Modem Media for those. It was a great time to start a career and since a lot of what we were doing had never been done before, we learned a lot through trial and error.

Modem was also the first agency of its kind to have a full-service research department and in those days our clients entrusted us to do everything from needs assessment (what features should be included in a website) to concept development (which designs are most appealing and why) to usability testing (how well a website meets consumer needs and where opportunities are for optimization). 

While I’ve worn many hats over the past two decades including client-side researcher and qualitative moderator, UX research remains at the center of my wheelhouse. These days, though, as a principal researcher, most of the UX projects I take on are very tactical, meaning a client has already identified a number of ways an existing site or mobile app can be changed and wants to test solutions that have already been built. While this tactical work is very rewarding, clients who only engage UX researchers to test solutions they’ve already developed are missing an opportunity to tap into our diverse skill sets and reap more value out of our relationship. Specifically, UX researchers can go beyond standard UX testing to help clients with needs identification, ideation and concept development.

Needs identification. Many UX researchers are trained in other aspects of consumer research and can apply both qualitative and quantitative methodologies to identify and prioritize needs around any given topic. For example, prior to beginning a development initiative on a new product, a client can choose to invest in some exploratory research to uncover the needs that end users would want met by the new tool (i.e., website or app) as well as how big those needs are. In the case of a redesign, we can explore how well an existing solution is meeting identified needs and whether any needs are not being met. This could provide important direction and prioritization for any design initiative – and some solutions may even be closer and less expensive than you think (see sidebar).

Ideation. Many UX researchers I know, myself included, are also trained in facilitating ideation sessions that can be used to craft early-stage solutions rooted in user needs and insights. Importantly, these are hands-on sessions where insights are presented and potential solutions are sketched/conceived by developers and designers using a structured approach to ideation. An external facilitator will bring an independent perspective to the session, allowing the presentation of alternate viewpoints amongst workshop members. Additionally, an outsider can help manage conflict that can arise when a diverse, passionate and opinionated team is assembled. Lastly, an independent facilitator will ensure that everyone has a say in the process and that no single viewpoint dominates the session.

Concept development. Once solutions are generated, UX researchers can employ a number of different qualitative and quantitative methods to explore the drivers of and barriers to appeal. Importantly, this type of development research should never be considered a beauty pageant where the goal is to pick a winner. To be most useful, this phase of work should not only explore which concepts are most desired by users but also dig into why. This research should also explore how each tested solution can be optimized so those lower performing ideas aren’t thrown out with the bathwater.

Digital wallet case study

In the late 1990s this very process was employed for a large New York-based bank that had the idea of digitizing its credit cards as a security measure. It sounds like a no-brainer now, when apps like Apple Pay and Samsung Pay are built right into our smartphones, but in the days before smartphones even existed we had a client that wanted to create a branded digital wallet for its customers. The research team at Modem Media was hired to conduct early-stage exploratory research to understand the viability of our client’s idea and to understand what else consumers would like to see built into this service.

Our first step was to explore the idea in traditional focus groups – at the end of the 1990s, online focus groups were all text-based and limited in their usefulness. The session began with a discussion around online payments in general and barriers consumers faced with e-commerce. Security and fraud were the biggest concerns and we learned that a tool that would alleviate fears around these two issues would be welcomed. We also ran an early-stage concept by group participants to see what they thought of the idea – it was a bit ahead of its time so it didn’t score all 10s but there was enough enthusiasm and intrigue to pursue it further. We also explored what else could be built into the product to make it more valuable and learned that the ability to store multiple payment cards – not just our client’s branded cards – was of significant interest.

Given our client was calling this a digital wallet, they also wanted to test the idea of letting users store other items found in physical wallets including insurance cards, a driver’s license, Social Security card, etc. When we brought these features up in the discussion, we learned that consumers preferred that the client stick to their core competence (i.e., payment cards) and not, for lack of a better term, over-egg the omelet. These findings were confirmed in a follow-up round of quantitative research and our guidance helped prioritize development focus and prevent our client from building in features that weren’t desired. 

Once the proposition was finalized, my agency built a prototype that we brought to consumers in a first round of user testing to get a read on how the tool might look and work. We were early enough in development that we could iterate designs between sessions so that the prototype was constantly being improved with user input. At the end of this phase, we had clear guidance on how to proceed with a build-out for the bank’s digital wallet product. 

Unfortunately, while we had a strong proposition and a solid path forward for development, our client could not secure the funding needed to build and launch the tool and the project was shelved. The dot-com bubble burst not long after and it was a long time before any of us would hear of a digital wallet. Today, though, whenever I use Apple Pay to make a purchase at a vending machine or retailer I think of that project and wonder how, if at all, that digital wallet idea that we tested in the 1990s may have impacted where digital wallets are today.

Already rooted in user needs

In summary, clients who engage UX researchers in needs identification, ideation and concept development will see that, once they eventually test the usability/acceptance of a new tool (i.e., website, mobile app, feature, etc.), they will be testing something that is already rooted in user needs and that has been optimized in exploratory rounds of research. While this adds both time and money to the process, investing in these learnings upfront will lead to more efficient post-launch development and may even save the organization some money in the process, as fewer changes may be required post-launch as users will be more satisfied with the solution. 

Using customer service data to uncover unmet needs

After my tenure at Modem Media I went to work for MasterCard. The team I was on was responsible for MasterCard.com and it was my responsibility to uncover areas of website optimization and to test the appeal of potential features as well as the user experience of those we brought to the site.

It was at MasterCard in 2000 where I learned how rich customer service data can be as a source for uncovering unmet needs and exploring satisfaction with a site feature. Working with our customer service group, I was able to organize customer service output from both our call centers and online servicing tool to see which areas of our site were leading to the most questions/complaints and what the nature of those questions/complaints were. This was an invaluable (and inexpensive) source of insight directing us to which areas of the site needed immediate attention. Further, as changes were implemented, we were able to track how the number of questions (or complaints) about different site features changed over time and use this as a key performance indicator of not only how successful a site update was but as an important metric for overall site satisfaction.

Through advances in text analytics, clients can inexpensively tap into customer service data as a means of uncovering user needs. Furthermore, the prevalence of review sites as well as ratings and reviews left in app stores and on blogs are additional sources of inexpensive insight that should serve as input into the needs identification process.