Matt Stevenson-Dodd was CEO of Street League before founding Trust Impact in 2019 to help social impact organisations find the best ways to measure and report their impact. As part of our ‘Monitoring and Evaluation’ month, Matt imparts some wisdom for the Levelling the Playing Field network on measuring impact how help achieve our common goals.

After three successive roles as a charity chief executive, what made you branch out on your own and focus on impact measurement?

I got really excited about impact in the last two years at Street League. We created an online dashboard to show where we were successful but also understand where we weren’t. That was the bit that really intrigued me. It’s dead easy to pick the biggest number out of your statistics, find a glossy photo of some disadvantaged beneficiaries and write a nice case study, but you learn most from what you don’t get right.

I wanted to talk about the young people who dropped out of Street League and find out why. That transparency drove better conversations internally: how do we help these kids that need more support? What can our success in Birmingham teach Glasgow? etc.

I started having conversations with other CEOs about impact. I became more pragmatic and focused, asking pertinent questions such as, what is the purpose of our organisation? What do we need to measure? All the time trying to boil it down to under five things.

At Street League we could have measured all sorts of things – teamwork, communication, confidence, physical health etc – but really, only two things mattered: did the young person come from a disadvantaged background? (i.e. were we working with the right person, or would they have got a job anyway?); and at the end of the process, did they get a job and stay in it for six months? That was all we really needed to know to measure our success.

So that led to you starting Trust Impact?

Yes. It has really taken off. We have worked with 56 organisations from small ones to major charities like YMCA, Help for Heroes and Teenage Cancer Trust. We now employ 30 people including experts in data, strategy, programming and research.

What’s the first stage when you work with an organisation?

Our first question is, what is your purpose? We often find that key people in the organisation are not aligned on that core purpose. Imagine five people in a team with everyone having a different view of the purpose, and thus different views on what success looks like, and the organisation trying to measure its impact? It gets very confused.

We have a free Purpose Alignment tool. It asks colleagues to describe the organisation’s purpose in eight words, then compares the responses using Natural Language Processing. The average alignment across the sector is 45%. Sometimes the Chair and CEO will give totally different answers and sometimes it’s more nuanced.

What do you do in that situation?

We have to get that core purpose clear. Some organisations’ mission statements are very broad – e.g. ‘We support young people’ – but what do you actually do? For example, there are hundreds of cancer charities, so if you’re one of them what do you do (that others don’t) that means you add real value?

That leads on to the next stage which is nailing down three to five things you really want to know, in order to understand how well you’re achieving that core purpose. That’s how you achieve good impact measurement.

What happens once you’ve agreed those measures?

We do something called ‘Measure what matters.’ Our research team apply tested methodologies to gain measurable outcomes. For example, in the case of wellbeing, we use the Warwick Edinburgh Mental Wellbeing Scale or the Office of National Statistics (ONS) Wellbeing Survey. You can then compare your participants’ wellbeing results against a national data set.

One charity we work with has an aim to get under-represented groups into sport. So we needed to assess, ‘Are your local sports groups representative of the local community?’ We boiled it down to a set of dials which showed gender, race and disability of local sports clubs on one side, and the ONS census data from that area on the other. We got a snapshot of their successes and priorities straight away.

Funders love it when you can clearly identify ways they can help you! If your data identifies a specific area where you need more support, you’ll be amazed how positive a discussion can be with funders about whether they can help you fill the gap. That’s why it’s best to present the whole picture; so you can learn from it. Data shouldn’t be used just to convince people of your success.

So what advice would you give to the LtPF network?

The mistake many organisations make is to talk about too many different things in their annual report. One charity’s report had 64 identical dials all showing different outcomes. It looked beautiful but what was the story? It was so difficult to process. It’s much more important to be great at measuring three to five things rather than being mediocre at measuring 80 things.

Also, set up your database in the right way. One charity we worked with was sending out surveys but putting all the answers into one text field. It was stored, but you couldn’t query it or compare it. It was a lot of work collecting data with no benefit.

Mandate certain fields in your surveys makes your data useful, rather than patchy. It’s no good if only 27% of people answered certain questions and they are the important ones.

Don’t overdo it. Respondents will generally answer five questions, but sending them 40 questions is pointless. Just because you can measure something, doesn’t mean you should!

Also, a personal request to fill in a survey gets a response rate above 65% as opposed to around 15% if you just send out something en masse. That could be a phone call or a text message to say, ‘It will really help us if you can fill this in.’ Use electronic systems to send stuff out, but that personal appeal achieves far better results.

Any other wisdom to add on M&E?

Trust Impact helps charities of all sizes develop a pragmatic approach to impact and data, including developing data visualisation and a data-driven culture. We use Power BI which is a great tool which charities can use themselves or we can help set up. You can see an example of what it produces from our work with YMCA here.

We’ve also just launched a platform called Connect Mix Share which allows you to connect your database and stream that data live to Power BI. Rather than spending hours crunching the same data each month for reporting, Connect Mix Share does it automatically. It’s low cost and saves days of staff processing time.

Many thanks for your time and insights, Matt!