The results are, of course, disappointing. The evaluation found no evidence that the Troubled Families programme had had any significant impact on the wide range of outcomes that it was intended to address, from employment to child welfare to school attendance. This is not to say that there were no improvements in the lives of the families who went through the programme, but that these improvements could not be attributed to the programme itself.

Evaluating the Troubled Families programme cannot have been easy or straightforward. Unlike the evaluation of a single, specific self-contained intervention with consistent features, Troubled Families involved major systemic changes, as services were re-geared around the whole family, with a lead or key worker in place, and a focus on specific outcomes through payment by results. The programme was set up at a time when localism and decentralisation were gathering pace, and while the programme had essential common elements, there was a lack of standardisation in how Troubled Families was implemented at the local level. There was a wide variation in practice, as local authorities chose how to interpret the programme for their populations. This high level of local variation made national evaluation more difficult (and may also have resulted in patchy or reduced quality across the programme).

Nevertheless, however difficult it might be to conduct this kind of evaluation, if we take the headline findings at face value then there are a number of changes to the Troubled Families programme that should be considered. In particular, there is an important question about the fit between the high and complex needs of some of the families in the programme and the support that was made available. This group of families is very likely to need intensive and persistent support provided by a range of appropriately qualified professionals. This is not to say that key workers do not play an important role in improving the quality of services to families, and in building a relationship with them, so that root causes can be understood and addressed by other agencies, but this approach is unlikely on its own to be sufficient to make a significant difference to these families’ outcomes. In its next stage, Troubled Families needs to use evidence-based programmes and practice that are directly relevant to families with complex and sometimes intergenerational difficulties to guide decisions on commissioning, workforce training and supervision. It is encouraging that plans to do so appear to be in place already.

There is an important question about the fit between the high and complex needs of some of the families in the programme and the support that was made available.

Finding a new and sustainable way forward for this programme is critical. Troubled Families has been and remains a significant element of the government’s social policy for some of the most disadvantaged. It has been central to local governments’ ability to protect and reshape early help services for families since 2011. Significant funds have been invested and entire services have been reconfigured, while committed family support workers have grappled with an array of complex and persistent difficulties in thousands of families. Given these deep and, frankly, expensive efforts, it is critical that we learn the right lessons from the programme’s evaluation.

First, evaluation remains vital; it is an essential component of effective and transparent policymaking and public investment. We need to know what works, for whom and in what context, and if it is not working. The disappointing outcome of this first evaluation of the Troubled Families programme is not a strike against the value of evaluation in public policy, and it is positive that DCLG has committed to further evaluation of the programme in the future.

Second, data and evidence (positive and negative) need to be shared openly and efficiently, and used to inform policy. The What Works network and other evidence-based bodies, parliamentary committees and the best journalists are all part of this endeavour. But politicians may be impatient for change, may over-claim the benefits, and may shy away from difficult results.

Third, and relatedly, the purpose of evaluation is to help us improve, not to prove. We need to create a more open learning culture inside and outside government, one which would allow us to have a more thoughtful and transparent dialogue about evidence, what it tells us about policy and how it informs decisions. In the case of Troubled Families, the combination of ministerial targets and payment by results left little space for this sort of reflective approach.

The purpose of evaluation is to help us improve, not to prove. We need to create a more open learning culture inside and outside government, one which would allow us to have a more thoughtful and transparent dialogue about evidence, what it tells us about policy and how it informs decisions.

Fourth, while outcome-based payments may have some attractive features, they should not be seen as a guarantee of service efficiency. They are not a measure of effectiveness and should not be used as such. The premise of payment by results is to devolve all design and delivery of services to local agents, who — unlike the central decision-maker — are thought to know ‘what works’. Unfortunately, even if this approach achieves better outcomes, it makes learning about what works very difficult, because services become fragmented or seem to emerge from an unknowable ‘black box’ of activities and outcomes.

Fifth, ultimately we need a broader approach, where equal effort is given to spotting risk early and making early interventions designed to prevent problems from worsening and the number of families becoming troubled from multiplying. The overall objective, to improve the lives of families facing complex and deeply entrenched problems, is too important for us all to not turn our best endeavours — and our best evidence — to achieving it.