Early intervention into action: Innovation and evaluation

One of the questions we get asked most often concerns where early intervention has been done well, and how other places can learn from those examples.

The UK early intervention sector is young and maturing quickly, so knowledge sharing is key. The evidence base for early intervention continues to grow, and lots of places are innovating and experimenting with new approaches. All of these innovations and reforms are, in turn, fantastic opportunities to generate more evidence about what works – what does it take to achieve positive change within local communities and existing local systems?

This ‘Early intervention into action’ section provides case studies of places that have made changes in how they work and put in place an evaluation to help them capture and learn from the impact of those changes.

The evaluation challenge

For EIF, effective early intervention is early intervention that has been evaluated to gauge its impact. We all want to make a difference to children and young people’s lives, but resources are always finite. This is why it is vital that local commissioners and service leaders are able to monitor, test and adapt the programmes and systems that they put in place.

We know that evaluating the impact of decisions and actions in a complex environment can be challenging. The scientific trials and experiments that can be used to evaluate the impact of individual programmes and interventions in isolation, for example, can be difficult to recreate on a wider scale.

But this doesn’t mean that evaluation is impossible, nor that it is merely a nice-to-have. Evaluation of what you are doing today provides vital information for improved decision-making and more effective services in the future.

Putting the right kind of evaluation in place

We have a set of ‘rules of thumb’ that we believe underpin evaluation at the local level. By commissioning or planning evaluations that adhere to these principles, local areas can be more confident that they have put in place a good process, and that their findings are more likely to be reliable, relevant – and in turn, useful.

The same set of principles also underpins the EIF evidence standards, which we use to assess the strength of evidence for individual programmes. While complex evaluations at the local level may not yet qualify for a formal evidence rating – of the kind we provide via the EIF Guidebook – we believe that sticking to these guidelines means that local changes may at least be seen as on the path towards being ‘evidence-based’.

By focusing on examples of evaluation that adhere to these rules of thumb, we seek to provide good examples of local authorities and other organisations that are monitoring the effects of their decisions and actions, and contributing to the wider evidence base on effective early intervention at the same time.

Our ‘Early intervention into action’ case studies

In this section, we tell the stories of local places who have made changes to bolster or expand early intervention in their areas, and who have put in place plans to evaluate the effects of those changes. Local changes may be aimed at improving outcomes for children, young people and families, or at improving the effectiveness or efficiency of local services.

Stories are based on short interviews with individuals connected with the local changes, and on documents relating to their evaluation (evaluation plans and/or reports). The content of case studies has not been independently verified by EIF.

Effective early intervention must be responsive and appropriate to local circumstances, and so our case studies are not templates or models to be replicated. We believe they are interesting, useful, illustrative stories that shine light on the difficult but essential business of evaluating early intervention at the local level.

iStock-513542280-400x400

Multisystemic Therapy, Manchester

Manchester MST was implemented in 2014 in response to growing concern from Manchester council about the number of children going into care. The number was growing – and the direction of travel suggested this would continue. The council was also concerned about the poor outcomes for children who were in the care system. This was further reinforced by a particularly critical Ofsted report.  The Council considered a number of evidence-based interventions and found that MST fit their needs most closely. While selecting a programme, the Council also wanted to ensure that the programme was having the anticipated and desired impact and so commissioned the internal Public Intelligence team to conduct an evaluation. While the provider (Action for Children) and the MST programme themselves both collected information it was felt that this was too qualitative and that it would easily be biased given the incentives of both parties.

What is the scale of the change?

New programme across the local authority area

Who does it target?

Children on the edge of care, children in care

What issues does it address?

Severe behavioural problems, risk of child removal

Who is involved?

  • Local authority commissioning teams
  • Local authority delivery teams
  • Community health services
  • Other health services
  • Police
  • Local political leaders

When did it start?

April 2014

Is it still ongoing?

Yes, contracted to 2018

Evaluation model

A ‘before and after’ method is used to compare clients’ outcomes during a period immediately before their MST intervention with an equivalent period of the same length, post-intervention period. The interim evaluation is therefore designed to provide some initial insights into how clients’ outcomes have changed over the two year period, for some this will be (approximately) a 16 month period (six months pre-intervention, four months intervention and six months post intervention), others is will be 28 months. A cost benefit analysis is also provided in the interim evaluation. .

Evaluation status

Completed; to be expanded and repeated

Read more

school-corridor-worried-teacher-400x400

School Wellbeing Service, York

The School Wellbeing Service is a school based early intervention mental health support service. It is jointly funded by Health, Local Authority and Schools in York. The School Wellbeing Workers (SWWs) are managed by the Local Authority, clinically supervised by Child Adolescent Mental Health Service (CAMHS) and based across a cluster of schools. Their focus is to work with children and young people and school staff around emerging and developing mental health need. In particular at children and young people who are presenting mental health issues and concerns that are below an intervention from specialist CAMHS and above what school pastoral structures can support The SWWs provide schools with consultation and advice, training, direct work in partnership with school staff and increased communication with specialist CAMHS services.

What is the scale of the change?

Piloted in two school clusters, roll-out to six clusters

Who does it target?

Children where schools have emerging concerns about mental wellbeing

What issues does it address?

Mental wellbeing, including confidence, depression, behaviour

Who is involved?

  • Local authority commissioning teams
  • Local authority delivery team
  • CAMHS
  • Schools

When did it start?

2015

Is it still ongoing?

Yes, as a mainstream service

Evaluation model

Baseline data was collected through staff surveys and semi structured interviews with Head teachers. In all direct work with children and young people a locally developed questionnaire, the Social, Emotional and Behavioural Competencies (SEB) questionnaire, was completed. Questionnaires were completed in group work and training workshops and case studies have been developed to demonstrate ‘what has worked’. Two measuring tools were used to record any changes in children and young people’s emotional wellbeing and mental health following involvement in group or individual work.

Evaluation status

Pilot evaluation completed; new design being developed for full roll-out.

Read more

father-children-400x400

Big Manchester (Improving Futures)

Big Manchester, part of the national Improving Futures scheme, is a holistic family support service for parents and children aged 5–11 years who are experiencing domestic abuse, to meet need, mitigate risk and build family strength and resilience. This service is delivered by a partnership, led by Barnardo’s, together with Manchester Women’s Aid, Manchester Mind, and Lifeline/Eclypse and Homestart. The model is a mixture of intensive one-to-one support and spot-purchased services.

What is the scale of the change?

One locality: north Manchester

Who does it target?

Families experiencing domestic abuse, substance misuse and parental mental ill health with children who are age 5–11. While referrals often focus on one particular child in a family, the project’s ethos is centred on whole-family working, ensuring each child in the target age-group is involved.

What issues does it address?

Domestic abuse, substance misuse and poor mental wellbeing, and the various factors that contribute to and result from these challenges

Who is involved?

  • Voluntary sector providers Barnardo’s, together with Manchester Women’s Aid, Manchester Mind, Lifeline/Eclypse and Homestart North
  • Strategic partnership with Manchester council

When did it start?

April 2012

Is it still ongoing?

Yes, funded through to September 2019 through Reaching Communities together with some additional funding from Barnardo’s and Manchester City Council

Evaluation model

A multi-method evaluation combining qualitative and quantitative approaches including data on changes in child, adult and family outcomes between entry to and exit from the programme.

Evaluation status

Local and national evaluation completed

Read more

child-chalk-house-400x400

Pembury Children's Community, East London

Following the English riots of 2011, the Pembury Estate in Hackney, east London, received significant amounts of bad press, focusing on ‘at risk’ young people who were painted as participating in the riots locally, exacerbating the stigma associated with the estate. In response, the chief executives of the Peabody Trust (the landlord) and Hackney council began to develop the Pembury Children’s Community, modelled on the Harlem Children’s Zone in New York, with the aim of radically transforming outcomes for every child and young person living on the estate. The 10-year programme has three strategic areas of focus: the early years and primary school, secondary school and into adulthood, and support for parents. The Children’s Community has been up and running since 2014 and is now one of three Children’s Communities championed nationally by Save the Children.

What is the scale of the change?

One estate

Who does it target?

All children and families

What issues does it address?

All issues, by a holistic approach

Who is involved?

  • Housing association (Peabody Trust)
  • Local council (Hackney)
  • Growing engagement from voluntary sector, schools and health

When did it start?

2014

Is it still ongoing?

Yes

Evaluation model

The evaluation mixes quantitative and qualitative approaches, including interviews with stakeholders, participatory methods, observation, documentary and social media analysis; data dashboards bringing together local administrative and other data; preliminary analysis of costs and savings; development of local evaluation capacity.

Evaluation status

Underway

Read more

mother-baby-beach-400x400

The Gateway, Fife (Improving Futures)

The Gateway project, part of the national Improving Futures scheme, began in August 2012 as a response to an identified gap in services in Levenmouth, where families were being affected by unemployment, poverty, substance misuse or domestic violence but were not eligible for support from core statutory services. The project works alongside families with the oldest child in primary school and where the families have multiple and complex needs with parents who often have little or poor self-identity, self-esteem or confidence. The project aims to reduce the risk of these multiple and complex disadvantages from reaching crisis point.

What is the scale of the change?

A small area: Levenmouth and five or six nearby neighbourhoods in a deprived area of Fife

Who does it target?

All children in the catchment area

What issues does it address?

Multiple and complex needs that are not yet at the level of statutory intervention

Who is involved?

  • Fife Gingerbread
  • Barnardo’s
  • Fife College
  • Fife Voluntary Action
  • Family and Community Support Team

When did it start?

August 2012

Is it still ongoing?

Yes, funded through 2017 with funding coming from the local area committee and the Fife Council Community Investment Team

Evaluation model

A multi-method evaluation combining qualitative and quantitative approaches including data on changes in child, adult and family outcomes between entry to and exit from the programme.

Evaluation status

Local and national evaluation completed

Read more

father-son-soccer-park-400x400

Neighbourhood Alliance, Sunderland (Improving Futures)

The Neighbourhood Alliance project, part of the Big Lottery Fund’s Improving Futures programme, brings together local services to deliver intensive and tailored support to families with complex needs through referrals from schools. It included more than 130 organisations, who then responded to the needs of the children and their families. The project was led by the Foundation of Light, the official charity of Sunderland Football Club. The project started in 2012 with funding through 2016. The project was awarded additional funding in 2013 to operate through 2017.

What is the scale of the change?

Sunderland (the five geographical areas of Sunderland: North, East, West, Washington and The Coalfields)

Who does it target?

Families with eldest child aged 5–10 years

What issues does it address?

Self-esteem and wellbeing; children’s school behaviour, attendance and attainment; family situations; and families’ strength and resilience

Who is involved?

  • The Foundation of Light
  • Sunderland City Council (Children’s Services)
  • Sunderland Children’s Trust
  • Gentoo Group (housing)
  • Sunderland North Community Business Centre (employment)
  • Sunderland Voluntary and Community Network (voluntary organisations)
  • Salvation Army
  • Southwick Community Forum
  • primary and secondary schools

When did it start?

June 2012

Is it still ongoing?

Yes, funded through to July 2017

Evaluation model

A multi-method evaluation combining qualitative and quantitative approaches including data on changes in child, adult and family outcomes between entry to and exit from the programme.

Evaluation status

Local and national evaluation completed

Read more

Share this page