Making research a renewable resource

Editor's note: James Wycherley is CEO of the Insight Management Academy. He can be reached at james.wycherley@insight-management.org. This article is adapted with permission from an episode of the IMA’s ”Transforming Insight“ podcast.

The evolution from hunter-gathering to farming the land was a critical step for human society. The drive to become insight farmers is just as critical for researchers and analysts who want their department to have a sustainable impact in their organization.

In 2005 I was given the opportunity to lead the first combined insight team at banking and financial services firm Barclays, incorporating market research, market analysis, customer analysis and competitor intelligence. Keen to improve the department’s visibility and impact, I worked hard to get onto senior executives’ appointment calendars and talk to them about our projects. At that time Barclays was recruiting a lot of new senior managers from the U.S. and after a while I found myself invited to their induction meetings, marched into the room as part of these new executives’ introduction to the bank. 

But what I discovered was that the more senior the audience, the less they really wanted to know about individual insight projects. They weren’t after the findings from particular pieces of analysis or research. What interested them was the big-picture understanding that we had developed off the back of hundreds of different projects. They wanted to know how and why consumers in our markets became customers of our organization and how they then created or destroyed value for it.

The problem for me then, and I think the problem for many insight leaders today, is that while we will have usually an instinctive understanding of part of that big picture, it’s a bit hit-and-miss, for two reasons. 

First, insight projects tend to be planned bottom-up: We look back on our work each year and see a collection of projects undertaken to support different decisions made by different stakeholders, not one comprehensive piece of work to which the individual pieces of research and analysis have contributed. 

Second, the vast majority of our insight specialists’ time is spent working on new investigations, so when senior people ask us to explain the big picture, we often have to think on our feet and there’s every chance that our colleagues would come up with a different answer.

If insight teams are responsible for insight as well as insights, we need to transform the way we plan our time, allocate resources and measure success. We need to move away from a world where we spend all our days hunting for new insights and recognize that wise insight teams know that insight farming is more valuable to their organizations and more sustainable for the team itself.

Explain something fundamental

So, do you hunt insights or farm insight? It’s worth exploring that farming analogy, because it helps explain something fundamental about the evolution of insight teams and what sets the most effective teams apart.

  • Farmers balance the short-term need to make an income with the long-term stewardship of their land.
  • Farmers value experience and use it to develop a big-picture view of how all the elements of nature combine to shape their environment.
  • Farmers prepare the ground carefully, tilling the soil and sowing their seeds.
  • Farmers have the patience to cultivate their crops, investing in irrigation systems that will sustain the land for many years.
  • Farmers can then harvest from their fields time and again and they have the satisfaction of knowing that they are feeding the nation, not just hunting to satisfy an immediate need.

What has all this got to do with insight teams? I think that too often insight teams act like hunters: We get drawn into the thrill of the chase as we pursue new findings; then, having reached our target, we move onto the next one. I’m not saying that we don’t need to run new insight investigations but putting all our focus into new projects means that we never have time to reflect on all the things that we already know, make connections we hadn’t spotted, investigate contradictions and come up with a comprehensive view of our customers, our markets and our organizations. 

Our organization, the Insight Management Academy (IMA), would argue that there is always going to be more value in accumulated knowledge than in a single new project. Therefore, it stands to reason that our companies can make a better return on their insight investment if we devote a significant part of our resources to farming our existing findings and seeing each new project as one more piece of a far bigger jigsaw puzzle. 

5 key takeaways

Progressive insight teams recognize their responsibility to produce big-picture insight as well as new insights.

  1. Progressive insight teams recognize their responsibility to produce big-picture insight as well as new insights.
  2. This takes a different mind-set, more akin to insight farming than insight hunting.
  3. Insight farming takes time and the patience to plow through previous research and cultivate knowledge.
  4. Insight teams can really reap the reward as they cut out duplication and produce better, more contextualized insights.
  5. Insight farming is ultimately more worthwhile for our colleagues, more sustainable for our insight teams and supports more agile decision-making in our organizations.

Paid back many times over

Driven by a desire to realign what the senior executives at Barclays needed and what our insight team was actually working on, I directed a lot of my energy for the next 10 years on rebalancing our focus on new research and analysis with the broader imperative to develop a knowledge asset for our organization. And we discovered that the apparent contradiction between the two activities was often quite superficial and that time invested in managing the collective customer and market knowledge base paid back many times over. 

You would be forgiven for thinking that time spent plowing through previous projects rather than quickly moving onto the next one is unlikely to pay back very quickly. But what you find is that developing this big-picture knowledge means that you can both cut out duplication and respond in a far more agile way when someone next asks you a question. 

You can also offer decision makers a choice: take a broad understanding based on previous work straight away or spend money and wait several weeks for the results of new research. 

And you can produce better insights every day, because if insights are contextualized observations, it’s your customer and market knowledge base that provides a lot of the context that we need to interpret the new things we see. It’s the knowledge base that provides perspective and those really big numbers that give us a sense of scale when looking at new numbers. 

In my book, “Transforming Insight,” I said that the seventh secret of successful insight teams is that they aspire to be insight farmers, cultivating and harvesting accumulated knowledge. Out of all the topics discussed in over 17 years of our organization’s insight forums, this is perhaps the one I feel the most passionately about. When the IMA’s co-founder, Steve Wills, first raised it as an important aspect of insight management, it immediately resonated with me, and it was at insight forum discussions that I had chance to compare my experience with that of other leaders.

It was from one of those other insight leaders, Caroline van den Bos, who holds a record for having represented four major organizations at the forums, that I first heard reference to hunting and farming, an analogy which we think is very pertinent and has been worth spending a lot more time developing.

Often hidden

Now, it’s very rare to find any progressive head of market research, analysis or insight who doesn’t recognize that all the understanding hidden within previous reports, presentations, dashboards, workshop notes, personal notebooks – and insight specialists’ heads – wouldn’t be worth far more to their organization than doing one further piece of research or analysis. But the problem is that it is often hidden, it’s totally fragmented, unusable in its current form and frequently contradictory if it was based on different sources at different moments in time. I know some insurance companies that have had projects running for years simply trying to define “a customer” and hence how many customers an organization actually had, so try getting everyone in an insight team to agree on anything more sophisticated, like the top five reasons why customers leave your organization and the top 10 interventions that would have biggest return on investment!

Just like real farming, insight farming requires patience, hard work and the ability to change insight professionals’ ways of working and then stick to a plan. Effecting this sort of behavior change in our teams is not easy, which is probably why the IMA’s benchmarking scores reveal that most insight leaders consider their teams to be considerably better at generating new insights than at cultivating and harvesting accumulated insight.

There is always a short-term need that seems more pressing, a stakeholder jumping up and down demanding some analysis that will have a more tangible, immediate benefit. But stitch many of these individual requests together at the end of the year and it often becomes apparent that our insight teams have actually produced very little new big-picture understanding and, further, that they would have been much better off splitting their resources between the obvious short-term need to find out new things and the less obvious but equally important need to build a customer and market knowledge asset that can then underpin every subsequent piece of work and conversation.

The disconnect between the importance of this big-picture work – and the difficulty in actually making it happen – has never been more apparent than during the lockdowns. At several insight forums IMA members have described the difficulties faced by less-experienced analysts and researchers when they’ve tried to produce new insights while working remotely. It would have been so much easier for them if their first few weeks had included a comprehensive introduction to all the key knowledge held on customer segments, key markets, categories, products, channels and brands and if they had been encouraged to see their new projects as answering an immediate business question and adding to that big-picture view.

Without established processes in place, however, it has been very difficult to start building a dynamic knowledge base without the benefit of conversations across the desk and the group pressure to adopt new ways of working.

But the long-term benefits remain. In fact, in a world where many organizations have had to fundamentally rethink their propositions, customer journeys and business models, it has been even more important to draw on big-picture understanding, to differentiate between short-term changes and long-term, fundamental customer truths.

Act as stewards

So, however busy it is in your insight team, with every researcher and analyst hunting for the next individual insight, please don’t forget that insight farming is ultimately more valuable for our organizations. It’s a more satisfying and worthwhile activity for our colleagues and it’s more sustainable for our insight teams. It helps us to act as stewards of customer and market understanding, developing an asset that can provide our organizations with sustainable competitive advantage.