Most of us are familiar with flowchart-style theory of change diagrams. But in practice, getting a range of local stakeholders to agree on what an intervention, service or other approach is trying to achieve, and how it will achieve those outcomes, can be challenging.

One popular and useful way of guiding the development of a theory of change is through a workshop, bringing together different voices and perspectives from across a local system. Based on a series of workshops EIF has recently run in four different local areas, this blog records some of our key recommendations and reflections on how to make these workshops work.

Background

We are working with the with the Department for Levelling Up, Housing & Communities (DLUHC) to understand, through robust evaluation, what specific approaches funded through their Supporting Families programme are most effective in improving outcomes for families. The ultimate aim is to provide implementable evidence to support local areas to improve early help family support services.

Through this work, we are supporting four local areas from different parts of England to evaluate approaches which have been identified as promising. In three of these areas, clinical psychologists work in their teams to provide training and reflective practice sessions to key workers. In the other, housing officers are being given access to early help data to improve joint working and early identification of families in need. 

As is often the case in early help, the approaches being delivered in many local areas are complex, with multiple components working across services and professionals. Because of this complexity, the core components of an area's approach have often not been clearly defined by those involved in commissioning and delivering them. This makes evaluation difficult, because without knowing what you are evaluating in detail, you cannot know how to evaluate it. 

A theory of change is an essential tool to help clarify this complex picture. Following our 10 steps for evaluation success model and guidance on our Evaluation Hub, as the first step in the evaluation, we looked to develop a theory of change with each of the four local areas, to map out the different components of each approach. This would help to answer key questions about why the specific approach is needed, what it involves and who it is for, as well as what outcomes it aims to achieve.

This lays the foundations for building a better understanding of how an approach works, supporting its continuous improvement, and ultimately evaluating its impact – in addition to the ongoing process of refining the theory of change as new evidence emerges.

Find out more about theory of change: what is it and why does it matter?
• Introductory video: Developing a good theory of change
• Creating a theory of change (EIF Evaluation Hub, step 1)

Running a theory of change workshop

We co-created a theory of change with each of the local areas to ensure it accurately captured the views of key stakeholders and to increase buy-in to the evaluation process.

We did this through a series of half-day workshops, designed to encourage discussion and create consensus on the objectives and intended outcomes of each approach.

Find out more about running a theory of change workshop
We’ve summarised our experiences and recommendations in a two-page cheatsheet: Running a theory of change workshop: A quick reference for workshop facilitators

Preparing for the workshops

We conducted scoping interviews with key stakeholders in each area. This helped to give us an initial understanding of the approach, which we used to plan the four half-day workshops. Three workshops took place in person and the last one took place online due to the pandemic. Each local area identified a group of around 14 relevant stakeholders to invite, making sure to include people involved in commissioning and delivering the approach.

As expected, participants had mixed experiences of theory of change. Being sent an information sheet and EIF’s short introductory video, which explains what a theory of change is and why it is useful, meant participants could familiarise themselves with the basic principles and key steps involved.

“The pre-session materials were helpful and provided a backdrop so we knew what to expect on the day.” – workshop participant

Running an effective and engaging workshop

The workshop consisted of six sessions, lasting around half a day altogether. We were conscious that workshops can be fatiguing, so the timetable included several breaks throughout (with refreshments for the in-person sessions).

Each session was facilitated by a lead facilitator and had a dedicated notetaker. Switching roles helped to mix up the dynamics of each session and keep participants engaged.

To make sure everything was accurately captured, we got participants’ consent to record the sessions.

Six sessions to develop a theory of change

Session 1: Introducing a theory of change, facilitators and participants

At the start of the workshop, we explained what we mean by theory of change and why it was important. We also asked several warm-up questions tailored to each local area, to help participants feel confident contributing to the workshop. For example, we asked about people’s previous experiences with theory of change, and what they wanted to get out of the workshop. This information also helped facilitators to pitch the workshop content at the right level.

Session 2: Why is the approach needed?

The next session was focused on understanding why the intervention is needed in the local context. We asked participants what issue the approach aims to tackle, and how it adds value over other approaches used currently and previously.

“I really enjoyed revisiting the work that we do and why we do it, as sometimes we can become complacent in regards to ‘this is just how we work’ and not take into consideration how differently we do work. Really good for reflection.” – workshop participant

Session 3: Who is the approach for?

Participants were then asked who the intervention is for, based on their articulation of why the approach is needed. We wanted to find out the characteristics and level of need of the families the approach was intended for. In some workshops, this was done with all participants together, where we felt the group had a shared perspective; in others, we divided participants into groups to draw out different perspectives.

Session 4: What is achieved by the approach?

This is really the main session of the workshop. It involves mapping out the intended outcomes of the approach, and clarifying its intended ultimate goal.

During the in-person workshops, participants worked in pairs and used colour-coded post-it notes to set out the key outcomes for children, families, practitioners and wider services in just a few words. During the online workshop, we divided participants into small groups of three or four, who used an online whiteboard (we used Jamboard) to create their own virtual post-it notes. In both cases, the process helped participants to map out the outcomes for different beneficiaries and think about how to sequence them.

During the session, facilitators helped participants to distinguish between outcomes, which are a key part of a theory of change, and activity outputs, which are not usually included. At the end of the session, participants were invited to share their outcomes with the rest of the group, and facilitators clustered similar outcomes together around emerging themes.

“Thinking about outcomes from different perspectives and over different timescales was interesting.” – workshop participant

Session 5: What are the key activities and how do they link to outcomes?

The next step involves mapping out the core components and activities which make up the approach (or intervention or service), and reflecting on how they work in their local context. Participants were then asked to detail which particular aspects they thought led to the intended outcomes, and how. That is, we’re interested in identifying the ‘mechanisms’ by which they believe the activities cause outcomes to change. This is one of the hardest parts of the process, but also one of the most valuable.

It is useful to allow plenty of time for participants to understand the sequencing between activities, mechanisms and outcomes. To encourage participants to think carefully about mechanisms, facilitators probed on what they wanted families and practitioners to be thinking, feeling or doing while engaging with the approach. Common mechanisms that came up were families trusting practitioners and feeling listened to, and practitioners having the space to think and reflect.     

“The workshop helped me reflect and consider what kinds of mechanisms of change might be relevant to this area of practice.” – workshop participant

Session 6: What are the barriers, enablers and unintended consequences?

The final session explores the barriers and enablers to achieving outcomes. Participants were prompted to consider factors both internal to the intervention, such as skill of practitioners, and outside it, such as the impact of the pandemic or availability of funding. 

Finally, we discussed what participants thought could go wrong, and what the risks or unintended consequences of the approach might be.

We closed the workshop by getting participants to reflect on the theory of change they had developed and explaining the next steps for developing it further. We also asked for feedback about the sessions themselves, in addition to a sending out a short survey to gather anonymous feedback after the workshop was over.

Local area reflections on the theory of change workshops

In the post-workshop feedback survey, a large majority of participants agreed or strongly agreed that the workshop increased their knowledge of theories of change and that the workshop was relevant and useful to their role.

In particular, participants valued hearing from others with different roles and expertise, who brought with them different perspectives on the approach. The workshop provided an opportunity to bring a diverse group together to discuss the approach in depth, which has been a rarity, especially during the pandemic.

Using breakout rooms for the online workshop facilitated discussion and the sharing of ideas among smaller groups of participants. It was useful to mix up the groups for each session to bring new perspectives to the discussions.

There were differing views on the length of the workshops. In one area, participants said they would have liked more time to explore some of the issues; but in another, participants felt they were fatigued by the end of the workshop.

“I found the workshop very useful and meaningful and a good way to create a theory of change that we understand.” – workshop participant

After the workshop: writing up a narrative theory of change

For each local area we collated the information from the scoping interviews and workshops to produce a ‘narrative’ theory of change document. This describes each element of the theory of change, capturing details such as the context and nuances of the approaches which cannot be distilled in higher-level theory of change diagrams.

We found that these narrative documents were a helpful way to play back to local areas what we have understood about their approach and to allow them to shape the theory of change, which is often more difficult when commenting on a high-level visual diagram. Local areas also found it helpful to have all the details captured in one place.

More broadly, areas have reported that it has been very beneficial to be able to share the theory of change with local partners, as an articulation of approaches which, in some cases, had previously been ill-defined. It helped them to move away from a focus on processes to describing why the approach is important within their local context and how they believe it leads to positive change in people’s lives.

What comes next? Through this theory of change work, we have developed a thorough understanding of the interventions being delivered in each area. The theory of change lays the foundation for deciding what to evaluate and how. Following the model set out in 10 steps for evaluation success and our Evaluation Hub, we will carry out feasibility studies to refine the theory of change in each area, which will be used to inform possible future impact evaluations.

Find out more about running a theory of change workshop
We’ve summarised our experiences and recommendations in a two-page cheatsheet: Running a theory of change workshop: A quick reference for workshop facilitators