100 not out: how the EIF Guidebook remains essential to our work and your evidence-based decision-making
Nearly five years on from the launch of EIF’s flagship ‘what works’ online portal, EIF researcher Jack Martin celebrates the 100 programmes milestone, and looks ahead to new challenges and opportunities.
Given everything else that’s in the news right now, the EIF Guidebook reaching 100 programmes may not be making national headlines. Nonetheless, for us – and our community of Guidebook users – it signals an important milestone. It represents five years of hard work to play our small but essential part in helping to improve outcomes for children, young people and families, and in showing off the range of incredible early intervention services available and the dedicated army of professionals who design, support and deliver them.
For those unfamiliar with our work, the purpose of the Guidebook is simple: to support commissioners and those designing services to make the best possible decisions by providing access to the best available information about what works. Evidence isn’t everything, but we believe it should be an integral part – alongside other factors like cost and existing local resources – of all decisions about commissioning or designing early intervention support. The simple fact is that, on balance, children and families who receive services that have been shown through robust studies to improve outcomes are more likely to benefit, and to a greater degree than those who don’t. There are a vast array of interventions out there, so it is really important to be clear about which approaches have been shown to improve child outcomes and which ones have not.
The Guidebook is how we help people understand which is which. It is a fully searchable online resource which provides information about early intervention programmes, including EIF’s assessment of the strength of their evidence from past trials and evaluations, as well as information on the cost of programmes, who programmes are designed to help and how, and the implementation requirements – which form such a vital part of any early intervention’s ultimate success.
For my part, I have witnessed the progress of the past five years up close, having worked on the Guidebook in a variety of ways since its inception; today, I’m lucky enough to be leading our work on the programme assessment process which underpins and feeds into the online tool itself. Having reached the 100 programmes mark, it feels fitting to take this opportunity to draw some attention to our newest work, but also to look back over the last five years and to see how far this work has come.
The Guidebook was launched in July 2014, and we’ve been busy ever since trying to provide the best resource possible.
Originally, the EIF Guidebook was based entirely upon evidence assessments conducted and published by other similar organisations, such as Blueprints in the United States. But over time, to give ourselves the flexibility to provide information about the evidence for programmes of particular interest to our audience of UK commissioners – including smaller, local programmes which are not often included on large international websites – we have developed the methods, expertise and database technology to produce programme reports based on EIF’s own assessments.
Growing the Guidebook: We have completed a number of major ‘what works’ reviews over the last five years, each of which has provided new programmes to assess and add to the Guidebook.
- In 2016, we added 35 programmes relating to the early years, attachment, child behaviour and cognitive development, through our major Foundations for Life review.
- That same year, we added 19 programmes focused on improving young people’s social and emotional skills in 2016, building on our review of early intervention in this area.
- We added 28 further programmes in 2017/18, including new ratings and information for programmes that had been part of the original Guidebook.
- In early 2018, we kicked off an annual cycle of updating existing Guidebook programmes to ensure that the information is current, and that our evidence ratings incorporate newly published and relevant evidence.
- New look: In early 2017, the Guidebook was given a new lick of paint, with a substantially overhauled interface designed to arrange our content in a way which is simple, clear and digestible, and to help our users focus on what really matters.
- Capturing complexity: In 2018, we added new ways to help you keep on top of a constantly changing evidence landscape. An increasing number of evaluations of early intervention programmes are revealing mixed findings – especially when interventions are being transported to the UK from other countries, such as the United States. To reflect this, we have added further clarification to evidence ratings where the evidence is mixed, to draw greater attention to these findings and help you unpick the complexity (as EIF director of evidence Tom McBride explained in a blog).
That’s a lot to accomplish in just under five years, and the extensive and detailed work that lies hidden behind each and every programme described on the Guidebook has required an inspiring amount of skill, determination and hard graft from our team of researchers (see Lucy Brims’ excellent description of what it’s like to be one of the ‘evidence detectives’) and our panel of outside experts and moderators.
If the last few years have been busy, this year has been no exception. We have worked hard to bring our users:
- New programmes: In 2018, for the first time, we launched an open call inviting providers and developers to submit their programmes for assessment and potential inclusion in the Guidebook. Almost 20 of these programmes have just been added to the Guidebook – bringing us to 101, at the time of writing. At the same time, we have continued to update existing Guidebook programmes and to make sure that our assessments of the evidence are up to date.
New features: To help our users get to the most relevant content quickly and easily, we have added two handy new features:
- Spotlight sets bring together programmes that are unified by a common theme, rather than a specific set of programme characteristics. Currently, we provide Spotlights on programmes intending to improve relationships between parents, and on school-based social and emotional learning programmes. And we will be adding more over time.
- Extra filter options: In response to user feedback, we have made it possible to search or filter programmes based on settings, delivery model (individual, group etc) and classification (whether they are universal or targeted). This information has always been part of the Guidebook, and now you can incorporate it directly into your research and decision-making.
We don’t plan on resting on our laurels just because we’ve hit the 100 programmes mark.
We know that there are many programme providers out there who are keen to have their programme assessed. We know that the value of the Guidebook will only be improved as we capture programmes across all the domains that our work covers – and there are hundreds more programmes to go. We also know that programmes already on the Guidebook are constantly publishing new evidence that we need to take into account. So, we will continue to add new programmes to the Guidebook, and we will be launching our second open call for submissions in the coming year.
It’s not all about adding programmes, though, and it is our ambition to constantly seek out new ways to make our Guidebook as useful as possible to those who rely on it. In particular, we want to:
- Add information on the impact of programmes, so that users can compare programmes not simply in terms of cost and strength of evidence, but also by how much they have been shown to improve the lives of children, young people and families.
- Improve the ways in which we collect data on our how people are using the Guidebook, and how it is supporting both their work and our own.
- Support our audiences to use the Guidebook to its full potential, by developing a set of materials and masterclasses for commissioners and local decision-makers.
So, we’ve come a long way in the last five years, and it’s great to have grown the Guidebook to 100 programmes – but this work is never really finished. Still, it is endlessly inspiring to develop a resource that we think has such massive potential to help our users inject evidence into their vital decisions about early intervention, and consequently to improve life outcomes for vulnerable children, young people and families up and down the country. We’d like to thank our community of users and contributors, and look forward to continuing our work together over the next five years, on the next 100 programmes.