Peter Henderson

Stephanie Wardell at the EIF has a great piece summarising the state of the art on supporting evidence use. Anyone working at a What Works Centre or trying to have an impact through research should read it.

My notes are below but it’s short enough that you should really read the whole thing.

Stephanie has five recommendations for WWCs thinking about promoting evidence use. These are based on a literature review, interviews with experts and EIF’s own experience.

1. Understand your role in the evidence ecosystem

• The ‘evidence ecosystem’ describes the complex networks of research producers (e.g. universities), research users (e.g. policy makers) and intermediaries (e.g. academic journals). The paper gives a simple overview of the evidence ecosystem (see diagram below).
• Not everyone can or should do everything within the system. If you want to have an impact, identify your role and how that relates to others in the ecosytem.
• This recommendation is a really useful steer. I’d emphasise that anyone thinking about their role in the evidence ecosystem should get very specific. Users, producers and intermediaries are broad categories and it’s not enough to simply identify as one of them – you’ll need to be more specific to really understand your place in the ecosystem. Each of the three roles contains a large number of potential sub-roles:
  • Producers could include people doing early-stage development, later-stage effectiveness testing, evidence synthesis, or theoretical develoment.
  • Intermediaries could include publishers of research, creators of evidence tools, people sharing research on social media, CPD deliverers, or programme designers.
  • Users could include national policy makers, local commissioners, practitioners interacting with service users, or people identifying gaps in evidence and setting research agendas.
• The evidence ecosystem framing is useful because important actors might not always be found in obvious places. E.g. a particular social media account could be an influential part of the evidence ecosystem but not usually considered an important element of the policy/practice system more broadly defined.

2. Understand the wider context

• The evidence ecosystem exists in wider context of complex interactions between different people and organisations with their own motivations, incentives, resources, worldviews, timescales, processes etc.
• The better you can understand this context, the better you can support evidence use.
• Start by asking: ‘what is our audience trying to do?’ and ‘what influences them?’ Then ask ‘how could our evidence feature in this?’ and ‘what incentivises or disincentivises evidence use?’

3. Think about what is meant by evidence

• The starting point for getting research used is understanding the demand for research. What do people want to know? How can you help them find the answer?
• Decision makers might want answers to a range of questions, not just ‘what works?’ E.g they may also want to know: ‘How prevalent is a problem? What are its root causes? How do people experience our services?’
• This will require flexibility in approach and choice of methods. Different questions should be addressed with different forms of evidence. E.g qual research to better understand experiences of services, RCTs to understand the impact of interventions.

4. Identify what meaningful evidence use would look like

• Understanding context and how to help audiences reach their goals (recommendations 2 and 3 above) are the foundation for thinking about what good evidence use looks like.
• You can use this knowledge to create a framework for describing what good evidence use means. You can plot the types of evidence use behaviour that you’d like to see and the influences on this behaviour.
• There are existing frameworks that can help with this. EIF developed the following framework to think about evidence use in the early intervention system.
• This framework could be used to pinpoint specific sets of system components under each heading that could help define clear goals for evidence use. E.g. ‘endorsing evidence-use through school planning and policy documents’ and ‘supporting specific research-informed professional learning’ could be components of ‘leadership.’ These become outcomes you can target in a theory of change and inform selection of activity to promote evidence use.  

5. Develop context-informed plans to support evidence use

• The final recommendation explains how to bring all this information together in a plan to promote evidence use.
• This recommendation emphasised the importance of working out where you are most likely to have influence. This sounded similar to the Effective Altruist importance/tractability/neglected framework which is a very useful tool for prioritising opportunities.

The paper has some practical recommendations for planning:

• Actively engage end-user audiences at an early stage
  • User involvement is important to ensuring outputs are relevant and actionable. The first step is to work out what your users want to know.
  • What Works Centres will often consult users in this way, but its rare that users are given power over what is researched or how research is interpreted. The paper suggests this might lead to more relevant and actionable evidence.
• Select and sequence knowledge mobilisation activities in response to an understanding of context
  • The key thing is that the approach needs to be good fit for context. How will your approach align with existing priorities, timelines and processes? How can you avoid duplication and confusing research users?
  • There’s no magic bullet for effective research translation. One of the key insights is that knowledge mobilisation plans should be multi-faceted and not just rely on simple communication of evidence.
  • Some other useful tools and insights:
    • Langer et al’s (2016) evidence review on knowledge mobilisation approaches found promise for:
      • Communication that motivates audiences
      • Building evidence use skills through training
      • Embedding evidence within existing tools and structures – e.g. building evidence into existing tools or training. (EEF’s work on early career framework good example of this).  
    • Michie’s behaviour change wheel useful for selecting approaches for addressing different behavioural barriers
    • EAST framework – make it easy, attractive, social and timely.
• Align efforts with others in the evidence ecosystem
  • Important to understand other actors in ecosystem and attempt to align efforts.
  • A nice quote from David Gough: ‘the effectiveness of What Works Centres is a function of how well they integrate with external organisations and the systems in which they operate.’
  • E.g. EEF research schools network, WT Grant Foundation research-practice partnerships, EIF’s work with staff college
• Test and learn
  • Most efforts to promote evidence use are not well-documented or evaluated. There’s so much more to learn!
  • What Works Centres have a role to play in developing the evidence on effective evidence use. Some good examples include EEF’s ‘literacy octopus’ trial and EIF’s user testing of its guidebook.