Avoidable pain is a hard sell. For anybody, looking back at our own work to see what could have been better is painful: it requires admitting mistakes and our own fallibility.
It's also unavoidable for most organisations: companies have auditors, ministers have Parliamentary select committees, schools have inspections, and so on.
Almost uniquely, philanthropic donors have no external scrutiny. Facing this pain is essentially optional.
It can also be hard. For a donor to 'measure their impact' is normally impossible. A precise measure of impact involves finding what happened which would not have happened otherwise. But there is no 'otherwise': there is no usable counterfactual. Add in the fact many funders support work too diverse for its effect to be meaningfully be aggregated, and no wonder many donors give up. A recent study found that one in five funders does not measurement impact at all.
Five easy tools
Happily, there are various simple tools which help donors understand important aspects of their performance. Giving Evidence has collated them in a new white paper, published this week. We draw on Voltaire's observation that 'the best is the enemy of the good': these tools don't give the best conceivable 'measure of impact', they are good in each giving a line of sight into a dimension of performance which funders manage and influence. They are:
Monitoring the 'success rate'
The proportion of grants which do well, do alright and which fail. Though clearly the definition of success varies between grants, presumably each one is made with some purpose: this tool simply asks how many succeed in their own terms. It's unashamedly a basic measure, but then it's hard to argue that a funder is succeeding if barely any of its grants succeed. We're not saying that every grant should succeed: many funders sensibly support experimental or exploratory work, but, like venture capitalists, donors should expect some failures, though should have some system for noticing which grants those are to enable learning from the patterns.
Tracking whether 'the patient is getting better'
Whether biodiversity is increasing around the lake, or whether malaria is becoming less prevalent. This of course indicates nothing about why it's happening nor the donor's contribution. Nonetheless, it's imperative to know if the problem is worsening – in which case, we might re-double our efforts or invite other funders in – or if it's gone away. Often data from public or commercial sources shows progress on a funder's goals.
Measure the costs created for charities by the funder's application and reporting processes
These can be huge: sometimes consuming most of the grant given or even more. A physicist at Colombia University calculates that some grants leave his lab worse off, and we've heard stories of application processes which cost twice the amount eventually given. Grantees may make great progress despite a meddlesome funder. The avoidable costs from application and reporting processes in the UK alone are estimated at about £400m every single year. It's worth examining the process to see where savings lie.
Hearing what your grantees think of you
As a charity chief executive, I often saw ways that funders could be more helpful (and less unhelpful) but never told them because the stakes are too high: charities can't risk offending people whose help they may need in future. So the learning is lost. Yet listening to grantees and beneficiaries has brought great benefits in philanthropy, medicine and social services.
Lastly, clarify what you're learning, and tell others
Funders do publish, but mainly about their successes. 'Publication bias' in medicine in which positive stories are disproportionately likely to be shared means that 'the true effects of loads of prescribed medicines are essentially unknown', according to epidemiologist Dr Ben Goldacre. Philanthropy can avoid the same fate. Tales of failure and challenges, however inglorious, teach us a great deal.
Perhaps 'measuring impact' is too hard and too off-putting, and we should all instead talk about 'understanding performance'. The tools in this white paper help with that. Giving Evidence is working with donors on several of them, and will happily talk to anybody about them.
Caroline Fiennes is director of Giving Evidence, a consultancy and campaign promoting charitable giving based on evidence. She is author of It Ain't What You Give, It's The Way That You Give It, and serves on boards of The Cochrane Collaboration, the US Center for Effective Philanthropy, and Charity Navigator, the world's largest charity ratings agency. Follow @carolinefiennes on twitter.
Content on this page is produced and controlled by Emirates Foundation
