Generating evidence on using evidence: does the EIF Guidebook work to support vital decisions?
The EIF Guidebook is one of our flagship resources for supporting evidence-based decision-making about support for children and young people. EIF senior adviser Hannah Wilson highlights our recent work to better understand how the Guidebook fits into its users’ decisions.
As a What Works centre, we focus much of our effort day-to-day encouraging others to use the best available evidence in their decision-making. Where there is existing evidence, we advocate for this being used to inform decisions; where there isn’t, we provide guidance on how more evidence might be collected. We believe that using this evidence gives the work of local services and other organisations the best chance of making a difference.
Since EIF’s conception as an organisation, we have been bringing together evidence on early intervention programmes, and our assessments of how confident we can be about their ability to improve the lives of children, in the EIF Guidebook, our free, online, searchable database. The Guidebook is designed to support decision-makers within early intervention services to access and use evidence. It aims to encourage the implementation of evidence-based programmes that are more likely to be effective, and so improve outcomes for children, young people and families. Many other What Works centres also have similar evidence repositories; however, there has been limited testing across the network to understand how the evidence presented within these repositories is used.
Recognising that we knew very little about how the EIF Guidebook was being used, and standing by our principle that we should use evidence to give us the best chance of having an impact, last autumn we commissioned Dartington Service Design Lab to undertake mixed-methods research into the impact and user experience of the Guidebook. This work focused on usability and reach: who is using the Guidebook, how are they using it, and what are the barriers and enablers to its use.
Over the last few months, Dartington has reached out the early intervention community through a survey and a series of interviews, to understand how commissioners, service managers, practitioners, intervention designers and academics use the EIF Guidebook in their work and decision-making.
We know that the impact of the Guidebook within decision-making is hard to pin down. So, in undertaking this work, we wanted to delve deeper into how individuals weigh the evidence within the EIF Guidebook against other factors, including their colleagues’ recommendations, hard budget decisions, and a multitude of other variables that play into the availability of early intervention support available for children. We know that navigating this complexity is extremely challenging and want to ensure we are responding, in real time, to our audiences needs.
The impact of Covid on local and national commissioning environment is unavoidable. With purse strings tightening, we believe it becomes increasingly important that decisions (both ours and our audiences) are informed by evidence. But we also understand that constraints on funding means there may be fewer opportunities to commission new interventions. This is an important time for the What Works network to respond to the rapidly changing needs of those we are hoping to support.
With this work now drawing to close, we are excited to listen, reflect and consider how we can respond to the findings. While we’re still analysing and absorbing the lessons from the data that has been collected, some key themes are already emerging. The good news is that our stakeholders appear to find the Guidebook well-structured and accessible. However, it’s clear that there are areas where our users would like more support, with a clear appetite to understand more about the implementation of programmes, and also – beyond the world of programmes – about the effectiveness of broader approaches to supporting children’s outcomes.
Undoubtedly, there will be learning for the wider network of organisations working to promote evidence in decision-making, and we are keen to share this. Over the next year, we will be thinking about how to further develop the Guidebook to better support the needs of its users and to ensure that the best available evidence is used to support children at risk of poor outcomes.