The evaluation of Project Oracle

We published the independent evaluation of Project Oracle. It shows that Project Oracle is a much needed initiative, helping to grow the evidence behind youth services, and then helping link this into the funding and commissioning process.

Project Oracle is the only city–wide evidence generation campaign of its kind anywhere in the world. Led by the Greater London Authority with a consortium of co–funders, it strives to bring evaluations of youth programmes – many of which are delivered by small or charitable organisations – in line with academically rigorous and internationally recognised Standards of Evidence, improving consistency and quality in understanding what does – and does not – work.

Project Oracle has several delivery strands. Alongside the Standards of Evidence framework is a validation process to help assess the strength of evidence behind projects. There is training, with a seminar series delivered by academics to teach public sector commissioners and other funders about social research methods and evidence–based commissioning, as well as a training programme for providers of youth services to learn more about evaluation techniques. There are also research placements to ‘match’ academic researchers with projects that need support, helping broker a relationship between the world class research community in London with youth services delivering programmes on the ground.

The results are encouraging. Our study shows that Project Oracle is a much needed initiative, with demand for its services high amongst charities. The Project Oracle team’s personal approach and support of individual initiatives is widely praised by those involved, with Standards of Evidence seen as a particularly useful tool to guide evaluation design, and the training is particularly popular.

Nesta has been a long standing champion and supporter of Project Oracle, and have adapted the Standards of Evidence for use in our own work. We were keen to lead the evaluation for a number of reasons. Firstly, we wanted to learn about the development and expansion in real time to help the Project Oracle delivery team refine and improve the model. We also hope that the recommendations on what has worked, and areas for improvement in future phases will help other evidence initiatives around the world, such as the What Works network here in the UK, to learn from the effective mechanisms used.

The evaluation of Project Oracle also raises some important questions about the wider context, for instance, how open providers want to be about discussing what is not working, how willing are charities to share their practice with others, and who defines what ‘good evidence’ looks like, and how to effectively engage with more academic researchers.  These issues are going to be discussed in a future blog.

The independent evaluation shows that despite being at an early stage of development, Project Oracle is well placed to make a real impact on decision making practice surrounding youth services in London. We look forward to watching how it grows and develops.

This blog was originally posted on Nesta.org.uk. Read the original blog http://www.nesta.org.uk/blog/evaluation-project-oracle