Editor’s note: Matthew Hellon is a research executive at global insights agency Northstar Research, London. 

In Parts 1 and 2 of this series exploring behavioral science frameworks and randomized control trials we covered:

This final article will look at the most effective and robust way of measuring intervention (developed using behavioral science frameworks) efficacy using randomized control trials (RCTs). Marketing is – at its simplest – about changing behavior. Marketing actions are similar to behavioral science interventions and thus should also take advantage of RCTs.

What are RCTs? 

The ingenuity of RCTs lies in its simplicity. A homogenous group of individuals – the same on a number of factors such as demographics and attitudes – are randomly assigned to different groups: 

  • A control group – Shown/provided with the status quo (e.g., the existing communications around home energy usage).
  • A number of experimental groups – Shown/provided with an intervention (e.g., the modified communications around home energy usage, utilizing social proof).

Results are then compared across the groups to assess if the intervention influences the target behavior.

Typically, the differences between the groups are assessed with a statistical test, such as a t-test. The specific test depends on the number of groups (control and experimental) and the number of dependent variables (outcome measures such as energy usage or click-through rates). 

Controlling variables

The key here is to keep all variables – besides the intervention – consistent between the control and experimental group. For example, if you’re testing which message framing is going to lead to the largest change in behavior, the message framing needs to be the only thing that changes between the control and experimental groups. This ensures that any potential differences in the results are due to the intervention and not other factors.

This is also why the starting group of individuals, randomly assigned to the groups, needs to be homogeneous. Any large differences in the starting population make it difficult to determine if differences in the results are due to the intervention or the different makeup of the groups. Things that may influence the results, outside of the intervention being tested, are known as confounding factors. 

What are the outputs?

The outputs depend on your dependent variable(s). Behavioral science interventions seek to change behavior so the outcome is often measuring that change. For example, if the intervention was designed to reduce energy usage, the main dependent variable (what we’re trying to impact) would be energy usage. 

It can also be valuable to measure other variables that may change due to the intervention. For example, measuring any well-being changes because of a change in energy usage. An intervention may be so effective that other unintended consequences occur and, therefore, these need to be measured.

Why RCTs are important

Behavioral science is evidence-based. That means before deploying an intervention to the target population, its effectiveness should be tested. RCTs are the basis of the scientific method. 

  • They are simple to set up and run with the right knowledge.
  • They are the gold standard of research, as evidenced by their use in medical research.
  • They are credible and have a strong reputation.
  • They can prove your work’s effectiveness in changing behavior.
  • They allow hypotheses to be proven or disproven with experimentation and are the perfect methodology for assessing effectiveness.

As referenced above, placebo groups in medical research are simply control groups. They’ve been instrumental in testing therapeutics and vaccinations to treat COVID-19. The stakes of medical research are high, potentially even life and death. This tells you the robustness and power of RCTs. Part of behavioral science’s success, especially in public policy, has been its ability to prove the success of its interventions. RCTs have been central to this. 

However, RCTs are not exclusively used by academics. They are used commercially as well and can be remarkably cheap. Richard Shotton conducted an experiment into gender perceptions using dating website Hot or Not for just £5. 

Three frameworks

And with that, we conclude this series on behavioral science frameworks and RCTs. I hope you’ve found value in what we’ve covered: three prominent frameworks – MINDSPACE, EAST and the behavior change wheel – how they can be used to generate behavior change interventions and how best to test those interventions with RCTs.