Case studies

In the third sector’s quest to find meaningful ways to define the social, environmental and economic impact of its work, a plethora of approaches has emerged. The non-profit universe is broad and diverse, and there has been a multifarious response to demonstrating impact. Here we focus on a selection currently being used by non-profit organisations and their funders.

• The Outcomes Star – St Mungo’s
• Bespoke tools - Acumen Fund
• Social Return on Investment (SROI) – FRC Group
• Balanced Score Card – Impetus Trust and NPL
• Survey – The Funding Network (TFN)
• Social Measures – The Fledgling Fund

St. Mungo’s is London’s largest homelessness agency, running hostels, care homes, complex needs housing and supported housing that accommodates 1,400 people each night.

Its interest in outcomes was kindled in the mid-nineties. Services had previously been viewed in terms of inputs and outputs. St Mungo’s decided to develop an independent system, tailor made for the homeless client group which incorporated the best elements of existing systems – it would actively include the client in their assessment, have a holistic approach and also address some of the clear weaknesses of existing fledgling systems. The aim was to increase objectivity, have very clear guidelines and have a system that was able to visually represent progress. With a grant from the London Housing Foundation it developed a system in conjunction with Sara Burns of Triangle Consulting that is now known as the Outcomes Star.

The Outcomes Star aims to provide a visual representation of the whole person. Each of the ten axes on the Star represents a key area against which a client may be assessed. The Star places the client firmly at the heart of the work done.

Plotting Stars at intervals allows both clients and staff to track changes over time, to look at the interplay between different areas of their recovery journey and to understand the journey of change. It is not simply the measuring of client outcomes that is critical but what this measure means for the service that is being delivered. St Mungo’s has linked the Star with the Cycle of Change so that its staff can easily recognise the supportive actions and behaviours that correlate with a client outcomes measure.

Elizabeth Harper, Head of Performance at St Mungo’s, acknowledges that there are risks in the world of outcomes measurement. “It is essential that the Outcomes Star  fits with your existing way of working and that this way of working is client focussed and has clarity of process and purpose.”

“St Mungo’s is currently using the Outcomes Star to identify patterns in different clients groups’ recovery journey so that it can learn how to orient resources to offer ‘the right support at the right time”. The preliminary findings will be published this summer.

Back to top >>

Acumen Fund is a non-profit global venture fund that uses entrepreneurial approaches to solve the problems of global poverty. It seeks to prove that small amounts of philanthropic capital, combined with large doses of business acumen, can build thriving enterprises that serve vast numbers of the poor. Its investments focus on delivering affordable, critical goods and services – like health, water, housing and energy – through innovative, market-oriented approaches.
 
It assesses its investments along four criteria: financial sustainability, social impact, scale, and cost effectiveness.

Before it makes an investment, it seeks to understand whether there is another group in the charitable marketplace that can deliver the same product or service more cost-effectively. To do so, it has developed the Best Available Charitable Option (BACO) method.

By comparing each of its investments with a real or hypothetical charitable option, Acumen Fund has begun to quantify, at least to an order of magnitude, how its social impact, for each net dollar invested, compares with that of other philanthropic options.

With support from Google and several foundations, it has built a web-based tool called Pulse to capture enterprise performance data using social, financial and operational metrics. The tool was originally designed to share portfolio performance data across Acumen Fund’s four international offices, but has evolved into a platform that will be shared with social investment organizations more broadly, and over 50 groups have already tested the beta version.

Pulse allows Acumen to keep metrics data and insights both reliable and readily available, cataloguing by geography, portfolio, and industry. It can then analyze this aggregate pool of data to identify cross-cutting principles—and to explore and communicate breakthrough insights into how to reach base-of-the-pyramid markets. As this tool is more widely adopted, it will allow for a shared taxonomy of social metrics and increased understanding of the overall impact of social enterprises.

Brian Trelstad, CIO of Acumen Fund, says: “Though it has limitations, this calculation [of BACO] reaches to the heart of the market-based approach, spelling out the cost saving, the financial leverage, and enterprise and technology efficiencies that drive financial sustainability and social impact. This allows us to understand the opportunity cost of our charitable capital, and make informed investment decisions that deliver the greatest social impact.

“We expect quarterly metrics from each enterprise on the corresponding financial operational and social impact metrics. We then complete our evaluation framework with a Capability Assessment by our portfolio team that gives a broader qualitative snapshot of the health of the enterprise’s management team, its organizational systems, its continuing fit with Acumen Fund’s mission and its potential for financial sustainability and scale.”

Back to top >>

Founded in 1988, FRC Group has grown from a small community based response to poverty in inner city Liverpool to being one of Britain's best known social businesses turning over millions of pounds.

Verity Timmins, FRC’s Impact Manager, says: “As a social business with charitable status, we feel a sense of moral obligation to prove the added value we create,” and the group has won many awards for its reporting.

The business had used a number of approaches to measurement over the years and turned to the Social Return on Investment (SROI) framework in 2005. Among other aspects of its business, it used SROI to measure the impact of its Driving Change programme, which helps the long-term unemployed into employment.

The SROI framework maps the added value of an organisation’s work, expressing it in financial proxies.

The first step for FRC was to conduct a small focus group with Driving Change trainees and identify the impacts the programme could achieve by thinking around how improving skills, qualifications and training could benefit trainees and wider society, economically and socially.

“Stakeholder engagement and context is key in identifying these impacts,” says Timmins. “As well as the tangible impacts of getting people back into work, such as increases in tax contributions, there were some surprising intangibles.

“A man who had been unemployed for twenty years said being employed on a 12-month ‘Driving Change’ programme working within Bulky Bob’s, our waste management and recycling business operated on behalf of local authorities, had changed the way his children viewed their own futures – he had become a role model offering them hope that they too would be employed.”

Timmins says applying values to intangibles is still a developmental area within SROI, “how do you start to put a price on renewed confidence?” she says, but the more the SROI network grows and information is shared the easier it is becoming.

“The map helps in putting some theory into the practice and identifying factors  such as how long it takes for results to be felt, for example. One thing it clearly identified for us is the tipping points. It showed us the levers of change and how to use them to bring about greater impact. For example it showed us that training programmes with larger cohorts had a marked effect on confidence for those taking part as it stretched and developed their team-building skills,” explains Timmins.

The analysis showed that for the Oldham Driving Change programme in Greater Manchester (2006) the SROI was £2.22 for every £1 invested.

“What SROI allows you to do is translate a very long story into something that’s short and speaks in a meaningful language to many different people,” says Timmins.

Back to top >>

Impetus Trust provides charities with strategic funding, expertise and capacity building support over a defined period of time, usually between three and five years, with the aim that the charities can focus on transforming more lives.

Throughout the relationship Impetus maintains a high level of engagement including monthly meetings with each charity chief executive, and careful monitoring to ensure progress against agreed objectives – a distinguishing characteristic that underpins the trust’s successful track record on impact.

Among the tools Impetus uses to assess impact is the Balanced Scorecard – a strategic planning and management system that is used extensively across industries, government and the non-profit world to align business activities to the vision and strategy of each organisation.

The approach provides a clear prescription as to what organisations should measure in order to 'balance' the financial perspective.

The Balanced Scorecard was used to help the Naz Project London (NPL), which provides sexual health and HIV prevention and support services to Black and Minority Ethnic (BME) communities in London, to “make sure the individual projects delivered on the ground tied in with higher level performance indicators,” explains Impetus vice-chair Nat Sloane.
“Like any system, it has to be tailored to the organisation using it and it evolves over time.”

The challenge in using the Balanced Scorecard is in setting meaningful indicators and this is done in partnership with the charity and by consultation with external stakeholders. “The stronger the alignment between indicators and the strategic goals of an organisation the better,” says Sloane.

“The benefits for NPL are emerging in terms of specific output and outcome measures as it rolls out impact measurement for particular programmes with specific communities. More broadly, NPL is leading the way in developing an innovative ‘impact architecture’. NPL is increasingly regarded as a ‘go-to’ organisation in the sexual health sector inasmuch as leading statutory players acknowledge the thought that NPL has put into aligning its strategic vision and operational performance.”

Since working with Impetus, NPL has won a major statutory contract with primary care trusts in three London boroughs as lead contractor.

It has seen a 114% increase in the number of people helped through support groups, and grown income 31% since 2004/5. In addition NPL has launched a peer advice programme and a programme to develop community advocates.

Back to top >>

The Funding Network (TFN), founded in 2002, enables individuals to join together to fund social change projects. It’s been described as a marketplace for donors and charities, as the UK’s first public, open giving circle, and as the ‘Dragon’s Den' for charities, who are to attend a funding event where they pitch to members for support.

It has raised over £2 million for more than 350 diverse local, national and international projects, through TFN Groups in London, the South West, Leeds, Oxford, Johannesburg and Toronto and through Youth TFN (YTFN) in London.

TFN sponsors small and medium-size organisations for whom £5,000 would be a useful contribution.

For this small, rather informal organisation, measuring the outcomes of a project is particularly difficult – the capacity barely exists at TFN, which employs two full-time members of staff, or the charities it funds.

However TFN is now evaluating its own impact using a survey approach and committed volunteers. The main criterion was that the process should not be onerous or time-consuming for fundees.  This pilot study, run by volunteer Dr Claire Cohen, surveyed a random sample of 10% (23) of its funded organisations. It received 18 responses.

The main objectives of the pilot were:

• To help understand the typology and size of charitable organisations funded by TFN at the point of funding and today
• To measure the impact that TFN funded projects have had on society
• To understand the nature of social change and its effect.

The 25 questions asked covered the type and size of organisations funded, when they were established, their geographical reach and funding impact.

TFN used the New Philanthropy Capital (NPC) four tier model to analyse the level at which the charities were working within society.¹

“The results of this research have shown that TFN funding has had a highly positive impact. However because of the diversity of the organisations TFN has funded, the quantitative measurement of this impact is difficult to scale and even harder to compare between organisations,” says Sonal Shenai, executive Director of TFN.

As a result of the survey, TFN has reviewed its Project Application template and issued a revised version of its sponsor and charity guidelines. 

“The Pilot Impact Measurement Survey 2009: methodology, results and initial findings,” by Dr Claire Rebekah Cohen and TFN’s Impact Group: Shuna Kennedy, Helen Knight, Iona Joy, Theresa Burton and Sonal Shenai is now published and available on request from info@thefundingnetwork.org.uk.

The approved questionnaire will be used to survey all of TFN’s funded projects on an annual basis.

¹ Lumley T, Langerman C and Brookes M 2005, Funding Success NPC’s approach to analysing charities.

Back to top >>

The Fledgling Fund disburses more than $1.5m in funding annually, supporting media for social impact. Executive Director Sheila Leddy explains that they are interested in strong stories and in how filmmakers plan to use these stories to improve the lives of individuals and communities, change policy and raise awareness. The Fund does not support the production stage of media projects, however. Instead they focus almost exclusively on post-production and outreach.

The Fund is developing new ways to successfully measure the social impact of media projects, as funders need to be able to point to outcomes. Aside from considering more traditional measurements such as awards, broadcasts and critical reviews, they also consider public awareness of the project. They try to gauge whether the general public is learning about the project, for example, by considering if it was mentioned in non-entertainment news. For example, Ghosts of Abu Ghraib, made with the support of the Fund, brought together a host of torture organisations, and this was another measure of impact.

Back to top >>