Standards of Evidence in UK social policy – what are they and how are they used?
Over the past decade, Standards of Evidence have become increasingly ubiquitous in the UK, and around the world. Our recent study of the 18 frameworks used in the UK examines what they are and how they can be used, and tries to address the ongoing question of whether there should be a single set of standards of evidence.
When we started this research, I assumed that standards of evidence would all look relatively similar. I could not have been more wrong. Although there are common principles underpinning them, particularly the shared goal of improving decision making, they often ask different questions, are engaging different audiences, generate different content, and have varying uses.
Our analysis segmented the standards of evidence into three groups based on how they are used by their host organisation:
The first group is standards of evidence that can be used to inform the host organisation’s strategy and funding decisions. An example, is the National Institute of Health and Care Excellence’s (NICE) GRADE framework which is used to help develop the clinical guidelines used in the NHS.
The second group is standards of evidence that to make recommendations to the wider field. An example in this category is the What Works Centre for Crime Reduction’s EMMIE Framework. The EMMIE framework summarises the best available research on what works to reduce crime. The What Works Centre for Crime Reduction uses EMMIE with the primary goal of making recommendations to the wider field on how effective different interventions are at reducing crime. It does this by presenting findings in an easy to access and understandable format.
The third group are those standards of evidence which provide a resource for organisations developing interventions. An example is Project Oracle’s Standards of Evidence. Project Oracle supports organisations to understand and assess the evidence behind their interventions. This review, or “validation” as Project Oracle term it, doesn’t endorse the organisation or their work, but instead uses the standards of evidence to assess the plans and evidence in place and to understand whether the evidence submitted is of a sufficiently high quality.
As these categories demonstrate, the standards of evidence used in UK social policy are very different. It is understandable that differences may reflect the practical goal of the host organisation. However, there is a need to consider more philosophical and theoretical tensions about what constitutes good evidence particularly as differences in opinion can foster confusion. In our research, we came across examples of different organisations reaching different conclusions about the same intervention; one thought it worked well, and the other was less confident. Who is right? Does the intervention work, or not?
One suggested response to minimise confusion is to develop a single set of standards of evidence. Although this sounds inherently sensible, our research has identified several major challenges which would need to be overcome to achieve this.
We have mapped the landscape of standards of evidence after in-depth discussions with providers - the ‘supply side’. Our next steps are to facilitate knowledge sharing and to explore the feasibility of creating a single set of standards. As part of this, we are keen to understand the ‘demand side’, engaging the wider field in a conversation about how to make standards of evidence as useable and useful as possible. We are forging partnerships internationally, such as with the OECD, to share best practice more widely.
This blog was originally published by Nesta