Posted on 20th July 2008
The House of Commons public administration select committee (PASC) on the third sector has said that it has been unable to corroborate the claim that third sector organisations can deliver public services in a unique way, due to a lack of evidence.
‘Public Services and the Third Sector: Rhetoric and Reality’ report said “The central claim made by the Government … is that third sector organisation can deliver services in distinctive ways which will improve outcomes for service users. We were unable to corroborate that claim. Too much of the discussion is still hypothetical or anecdotal.”
The report concluded “The general direction of government policy appears to us to be positive. Nonetheless, further steps along this path should be accompanied by the collection of much firmer evidence on the impact that third sector delivery is having.”
The report said that there was not enough support for claims that the third sector distinctly “adds value” or offers, as it is frequently referred to, social return. They used the example of an organisation in Liverpool: Bulky Bobs provides recycling services to the public and also employs ex-offenders, thereby “adding value”. The report said that “The type of added value provided by Bulky Bob’s could theoretically at least be provided by other types of provider.”
The big picture
Both the PASC report and many third sector observers argue that better information on ‘added’ value, or social return, is important to all funders, from a government procuring third sector services to private givers. Indeed, Tony Wright MP, chair of the committee, told Third Sector magazine: “The Government needs to put its money where its mouth is. It needs to develop an evidence base of where the sector adds value and where it doesn’t”.
For its part, the Office of the Third Sector (OTS) has already committed to investing £5m in a new Research Centre for the Third Sector, and also has started a three-year programme to create a standard tool to measure social return on investment (SROI ) . For instance, some approaches to measuring SROI, such as that developed by the New Economics Foundation (NEF), incorporate a consideration for ‘what would have happened anyway’, without the intervention of the charity.
Jeremy Nicholls, Chair of SROI-UK and a NEF Fellow, says “Organisations tend to overlook analysing social impacts because most funders currently request information on financial measures and specific outcomes, rather than the ‘big picture’ of social impact.”
In a recent debate on charity scrutiny hosted by Philanthropy UK, NPC’s Martin Brookes argued that charity performance is largely unscrutinised, whilst the limited evidence suggests that charity performance varies enormously.
NCVO chief executive Stuart Etherington, agreed that that improving effectiveness is important, but added, “There is much to do to help organisations improve their effectiveness, not least of which is educating donors that it costs money to measure performance.”
In the June 2007 issue of our quarterly Newsletter, a panel of donors and charities talked about how charities can best demonstrate their effectiveness and the challenges they face in measuring ‘softer’ outcomes.
Daniela Barone-Soares, chief executive of the award-winning venture philanthropy Impetus Trust, explained: “Impact is super hard to measure. For example, charities dealing with 'prevention' or targeting policy changes may have a huge impact, but it is hard to quantify. Reality in the charity world is that the numbers alone do not tell the whole story, so charities have to invest in creating a picture of the whole that is credible.”
And Jeremy Swain, chief executive of the homelessness charity Thames Reach, gave an example of how simple output measures do not capture the true impact his charity is having: “We work with extremely chaotic, rough sleepers, and [in 2006] one of our teams had great success in encouraging a homeless woman to come in off the streets after 15 years of rough sleeping. However, it took another two months before she was prepared to use a bed and a further three months before she was willing to use the dining area with other residents of the hostel. However, she is alive and the quality of her life is improving slowly through the patient efforts of staff.”
In summary, the dearth of information on charities’ impact – rather than just their outputs – is due in no small part to funders asking for information on outputs, not impact. As funders of all types begin to realise that they require better information on social return, they must first understand what they seek to measure, and second, be prepared to support the costs of measuring it.
