Understanding the product development process and market research

Editor’s note: Stephen Cribbett is the founder and CEO of U.K.-based market research firm Further. This is an edited version of a post that originally appeared under the title, “Using market research for product innovation.”

Explaining product innovation

Product innovation is about creating products and solutions that solve problems customers have.

Innovative products are not those that are just new or different, they are themselves a product of incremental testing and refinement to see what works and what doesn't, with failure along the way leading to changes and improvement that guarantee market success and growth.

The innovation process embraces rapid failure in the pursuit of finding out what customers truly need and want. It is an iterative process, where the outputs of one subtask necessarily act as inputs into another. With the goal being one of minimizing risk by arriving at the most viable product concepts quickly.

Product innovation, therefore, isn't a nice-to-have, but an integral part of every successful business today. Failure to innovate puts an organization regardless of size at risk.

The value market research adds to product development 

Research gets you closer to the customer.

Differentiation of new products and services is not the only goal. In today’s modern world, consumers have way more power than in the past. For products to resonate with their target audience, the innovation process itself needs to be consumer centric.

So what does consumer centricity really mean?

Put simply consumer centricity can be defined as being aligned with and responding to customer's current and future needs.

It's a way of operating from the inside out, and, when done well, positively affects overall business performance over other operating models. Meeting the customer’s needs is the heart and soul of any sale, of any business. New product development should always keep this as its north star.

And, market research is a key tool in tailoring innovation to the needs of a particular target audience. Done well, it bridges the knowledge gap between your team and what they think they understand about a market and the actual needs, wants and desires of your prospects and customers. It's the process of uncovering their pains and problems with current solutions (or lack thereof).

By integrating customers into every stage of the product innovation and development process, from ideation to testing, packaging and pricing, you're able to solve very real pain points by uncovering and incorporating preferences, habits and buying behaviors

There are classic approaches to asking your customers what they want. This includes surveys, focus groups and analyzing behavior. But market research does not need to be cold and clinical. For example, creating communities around product development is a relatively new approach made possible by high-speed Internet access and specialized online platforms.

Gaining insights from the product life cycle

There are four main stages of the product life cycle: introduction, growth, maturity and decline. At each stage consumer market research will be required to support key decision-making – below are some of the most common areas where market research can support a product strategy:

  • Idea ideation. 
  • Setting initial prices and adjusting prices in later stages.
  • Inspiring ideas for related products or services.
  • Finding ways to breathe life back into older products.
  • Uncovering new markets or relevant audiences.
  • The new product’s packaging and delivery considerations.
  • How company reputation intersects with the product or service.

An experienced research team knows how to balance these potential arenas for insight.

A blackboard with the product life cycle process. There is a graph showing a tree life cycle starting with the intro, followed by growth, maturity and decline.

The stages of the new product development process

New product development can essentially be broken down into seven stages: ideation, research, planning, prototyping, sourcing, costing and commercialization.

The steps below outline a basic approach to developing new products or services. Market research can and should be used at every stage:

  • Idea generation. Generate new product ideas through focus groups, online research, customer interviews etc.
  • Concept testing. Check whether you’re selling something people want and will pay for. Ensure you can gather feedback from a substantial and unbiased audience.
  • Product testing. A minimum viable product (MVP) is an early version of your product with just enough functionality for early customers to use. 
  • Packaging and branding – Research helps guide packaging design, promotions and service.

Product development research phases

Phase 1: Generating a new product idea (exploratory research)

Exploratory research happens at the earliest stage of the new product development process. This kind of research is about establishing a deep understanding of target market sentiment, needs or problems. It serves to give you insights into the state of the market, and where you might be able to seize opportunity or avoid risks.

Areas that exploratory research might assess include:

  • Market demographics and psychographics. What kind of customers are most interested in a new product: who are they and where are they? How do customer preferences change between markets?
  • Customer journey mapping. What choices, decisions or paths might a potential customer take when considering buying from you?
  • Product improvement opportunities. Is there a need for something brand new or could existing products be improved upon?
  • Awareness. Who knows about you and your products? Are you top of mind for anyone? 

There are many research tools available for getting answers to these questions. Some consumer-centric examples include:

  • Social listening analysis. There are millions of publicly available opinions posted online. Automated systems can aggregate these and detect broad patterns. Qualitative research can dig deeper and mine insights.
  • Community stewardship. Maintain an active and meaningful role within online communities that attract your target market. This means backing off pushing your own agenda and doing more listening than talking.
  • Open-ended, in-depth interviews. Recruit passionate members of your audience and invest in expert interviewers. The goal: unearth pain points and discover gaps in the market.

Phase 2: New product concept testing

Assuming the ideation stage bears fruit, the next stage is prototyping. Again, this needs to involve potential customers. The prototype could be a physical product, or it could be a digital mock-up or digital product. 

The outcome of the prototyping stage is an MVP – a product that has the features necessary to go to launch without desirable features, which can create delays to the launch.

This phase may go through several iterative subphases before the final product is ready. That is: you may have to have several rounds of revision as you incorporate feedback and test again. Products with higher complexity tend to require more of these cycles.

Areas that prototype testing research might assess include:

  • Pricing. How much would people pay for this? Are there tiers? Is it a subscription basis? Will there be a little or a lot of price elasticity?
  • Distribution. How and where do people pay? Is this something that is shipped, sold in stores, available online etc.?
  • Feature priority. Which specifications, features or key benefits seem to be the most sought after or valuable?
  • User experience. How exactly do people use the product once they have it – as expected or are folks finding novel ways to interact with it? Or are people confused?
  • Marketing insight. What’s the best way to talk about this product to a broader audience? How do testers talk about it to others?

Consumer-centric tools that you might use in this phase of market research include:

  • Targeted surveys. Depending on the concept, a simple multiple-choice survey with as little as one question can provide valuable insight. There are many programs that compensate people for taking the time to quickly lend their opinions.
  • Digital property testing. Standing up a simple landing page and building in a mechanism to measure intent is how some product ideas get their start. An ad campaign and page can be created and deployed in hours, often with multiple variations.
  • Online research communities. Similar to (or even an extension of) an online focus group – users are given access to early versions of the product or service, packaging ideas etc. and are incentivized to participate in tasks and activities that provide in-the-moment feedback

Starting the product development research process

1. Set goals and objectives

There are many questions that could be asked and many ways to do it. We rarely have the resources or luxury to do it all, so it’s important to define and prioritize the key outputs.

Think about:

  • Expectations. Yours, your team’s, your organization’s.
  • Benchmarks. What measures or targets have been established in the past?
  • Timeline. Work backward from when the new product needs to be in market.

The well-known SMART method for goal setting is useful in this scenario too. Goals should be:

  • Specific.
  • Measurable.
  • Attainable.
  • Relevant.
  • Time-bound.

 2. Define a scope of work

The more specific you can be about this, the better. Having SMART goals is a huge help in laying out a scope for a market research study, which can include tasking details such as:

  • Who is the target audience?
  • What is our budget?
  • How detailed do any research questions need to be?
  • How long will the study or studies last?
  • How many participants are needed to generate results?
  • What research methodologies will we use?

3. Forming a partnership and finding market research

Finding the right participants for your studies is typically the most challenging part of any consumer market research project. It doesn’t matter if you're doing qualitative or quantitative research – it takes a huge amount of time and energy.

A full-service partner will be able to take on recruiting, designing and running the studies, if that is what you need.

As part of the goal-setting and scoping process, you should have a good sense of what aspects of product development research you can do in-house, and which need to be outsourced.

When looking for a research agency partner, there are a few important traits to look for:

  • Flexibility and agility. Can this potential partner adapt quickly to your needs or to sudden changes in the market? Today's world moves fast – you’ll want to avoid vendors with rigid processes.
  • Responsiveness. Recruiting can be an especially lengthy endeavor. You’ll need a partner that can keep you and your team looped in on any problems and progress.
  • Industry experience. A partner who knows your space can usually
    provide cost and speed efficiencies.

Achieving success

The heart of any product’s success is the consumer. Good product development research gets you closer to the customer and their needs. Customer-centric market research can add value to practically every aspect of the product life cycle.

To retain a competitive advantage and create relevant products, you need to:

  • Set solid research goals. 
  • Scope research projects properly.
  • Partner with flexible and responsible research partners.