Camilla Nevill, New Philanthropy Capital
Keep it simple: Charities that want to measure their impact should start simple and build from there. Evaluation and monitoring needs to be a learning process. I also think charities need to invest in training. We recommend courses by the CES and CASS for more advanced research methods. Also, as @MattLampLights says, start by measuring one or two key outcomes first and do it well.
I also agree @Tim Goodspeed that the first step is to understand and articulate a theory of change - showing how activities link to desired outcomes.
Use the right tools: There are many tools that you could use to measure clinical outcomes. The Improving Access to Psychological Therapies (IAPT) programme has a list of recommended tools (including the General Anxiety Disorder Questionnaire which is good for measuring reduction in symptoms of anxiety and stress, and the Penn State Worry Questionnaire). Check the IAPT data handbook. CORE is also a good tool for measuring global clinical distress.
These clinical tools are designed to measure the absence of negative symptoms. There are also tools to measure more positive well-being outcomes, such as improvements in self-esteem, resilience or relationships.
This is a good website on soft outcomes tools and includes links to Rosenberg's self-esteem scale and Wagnild and Young's resilience scales. The Warwick Edinburgh Mental Well-being scale is a good tool designed to measure overall well-being in adults. NPC has also developed its own Well-being Measure that measures seven aspects of 11- to 16-year-olds well-being, common to charities work with this age group. We eventually plan to develop well-being tools like this for other demographic groups.
The most important thing is to check the tools and assess whether the questions they ask are really relevant to the work you do.
Tim Goodspeed, SROI Network
When it comes to measuring 'soft' outcomes I would start with these questions:
1. Who changes as a result of our activities?
2. How do they change?
3. How do we prove it has changed?
4. Which changes are (most) important?
5. Is it all down to our activities?
A good understanding of impact needs to address these questions.
The issue is often how to address them adequately enough. How far you go depends on your objectives for measuring the impact in the first place. I would agree with evanstgiles that this includes thinking about who the audience is. For some charities, the resource and expertise requirements to gather good outcomes data can be a barrier to answering these questions in a meaningful way. Demos found that 'most [charities] did not have the data or the expertise to measure their outcome'.
Overcoming barriers can be complex: When we talk about impact, we usually mean change as a result of the activities we deliver, in the world outside of those activities and the time outside that which we spend with users. So, by definition, it requires time and resource to go beyond our activities and the time we have with users, to find out the impact in their world. This can become infinitely complex, if not managed.
How do we overcome this barrier? Identify one or two outcomes (changes) to begin with and work at gathering data and measuring these. Go as far as you can sensibly until you learn something new that will make you better at what you do. Apply the Pareto principle (80/20 rule). When you have applied what you have learnt from this, then go further and try some more. Be proportionate to your objectives.
Roland Marden, RM Insight
Use hard data to make your argument: I saw a report recently that highlighted how few charity annual reports have any account of social impact. Very telling. I think charities have traditionally relied on sentiment rather than hard facts to communicate their value. Moreover, providing data on impact can expose an organisation to hard scrutiny and sometimes criticism. Thankfully, things are changing with the widespread call to evidence outcomes.
What simple (practical) steps can a charity that wants to measure its impact take?
1. Survey/interview service users to identify what outcomes of value are produced (both intended and unintended)
2. Identify indicators of these changes, ie well-being, improved health, employment, etc, and identify a straightforward means of measuring the indicators, ie well-being/health index, jobs gained, etc
3. Set up quarterly data gathering
4. At yearly intervals collect and analyse data.
5. Disseminate findings along with case studies demonstrating human impact
You still need to communicate the human aspect: Increasingly the call is for 'hard evidence' and this usually means some form of cost-benefit analysis. Value for Money (VfM) calculations can be done to measure outputs. But usually an estimate of the value of outcomes compared to cost of inputs is also required. SROI is a rigorous approach and increasingly popular in the sector. It provides a single ratio that tells you the value of outcomes for every pound invested in the project.
I would also stress the need for a qualitative dimension that communicates the human impact of a project; case studies, interviews, diaries, etc.
If the project is government funded, it is important to link impact findings to relevant government policy agendas (Every Child Matters for children and young people). Projects need to demonstrate that they acheive objectives but that these are widely understood as desirable. They can also link impact findings to academic research.
Tris Lumley, New Philanthropy Capital
You can't pigeonhole commissioners: While it's tempting to say that commissioners are looking for cost-effectiveness, value for money and specific outcomes, and donors are looking for something more personal and emotional, I think the reality is that both groups are very diverse.
Therefore, the best starting point is to communicate from the charity's own perspective - mission, specific objectives, activities, outcomes, overall progress against mission.
To work out how to engage people in both groups, I think a fair assumption is that they want a mixture of the quantitative data that gives a comprehensive picture of impact, and the qualitative, case studies that give a personal sense of impact.
But something too few charities actually do is start a dialogue with funders, and say 'What is it that you really want to know about us?' A great resource for thinking about segmenting your donors based on their approach to giving is Hope Consulting's Money For Good.
Hannah Lownsbrough, 38degrees
Remember that there is no universal approach: Evaluation works differently for campaigning organisations, especially where a substantial element of the work is carried out online. Obviously, most importantly, we can look at the extent to which decision-makers sign-up to the demands that our members make during the campaigns that we run.
On a more day-to-day monitoring level, however, 38 Degrees is able to draw on information from the open / click rate of e-mails and we're able to see how many people have taken an action, too. We also use Facebook, Twitter and our blog to hear more from our members about what they like about what 38 Degrees is doing, and what they'd like to be different. Polls are another way to hear from members, and we make sure we're regularly asking them questions about what issues they care most about, etc.
It's not only campaigning organisations that use social media to communicate with members or supporters, however. I've chatted to groups who use Facebook for fundraising about ways in which they could make the most of the data that's provided as standard to people who administrate a page. Similarly, I think polling can be really useful and with tools like SurveyMonkey, it can be free or cheap.
This content is brought to you by Guardian Professional.To join the Voluntary Sector Network, click here
