EXECUTIVE SUMMARY

At the beginning of the twenty-first century, inequality has reached levels not seen in the UK for over 40 years¹. Despite decades of economic regeneration programmes in low-income communities, our place of birth continues to be a major predictor of the jobs we do, our health and life expectancy, and the income we earn².

Why this research?

This report from nef (the new economics foundation) asks why inequality is increasing when investment in deprived areas is growing. It is not only an important question to ask but also a timely one as the Government is pumping significant funding into regeneration initiatives in the build-up to the 2012 Olympic Games.

Focusing on the experience of St Helens metropolitan borough on Merseyside, our research suggests that the methods used by policy makers to measure the success of Local Economic Growth Initiatives (LEGIs) , although an improvement on what went before, are still inadequate. A more sophisticated approach is needed for the shortcomings of successive regeneration programmes to be understood and overcome.

Prior to LEGI, the evaluations attached to regeneration programmes tended to focus mainly on quantitative economic outputs, such as the numbers of jobs and enterprises created and people trained³. But projects can be successful on these terms without changing underlying inequalities. This report argues that the approach taken to evaluations has relied too heavily on two flawed assumptions – that outputs provide a true measurement of change, and that there is necessarily a direct cause-and-effect relationship between investment and the achievement of policy objectives.

Insufficiently robust evaluations have led to a perpetuation of similar types of initiatives. The outcomes required to address inequality have rarely been achieved, and governments have potentially exaggerated the improvements secured by their investments.

Although LEGI is more outcomes-focussed, the research raised further questions about the extent to which improvements were likely to be attributable to the investment from this programme. Given the scale of the task that a programme like LEGI has undertaken, a more sophisticated approach is needed for the shortcomings of successive regeneration programmes to be understood and overcome.

The research upon which this report is based was conducted under nef’s Measuring What Matters programme. Measuring What Matters was established to investigate how government policy making might be improved by measuring and valuing what matters most to people, communities, the environment and local economies. nef has a track record in working with communities to develop radical solutions to the problems created by the shortcomings of the macro-economic system. For example, its Clone Town work looked at the impact on communities of a reduction in the diversity of shops and services; and its Plugging the Leaks initiative shows how money spent locally can continue to benefit people through a local multiplier effect⁴.

It was through projects such as these that the need for a new approach to measuring change became apparent. The objectives of nef’s research in St Helen’s have been to evaluate current methods of measurement, to suggest alternatives, and to try to achieve a more rounded measurement of social return on investment (SROI) that will be applicable not only to local enterprise investment but also to other areas of public policy.

The approach

Measuring What Matters advocates a long-term, transparent approach to measurement in which a central role is played by local communities – the people on the receiving end of the investment. Policy decisions are often driven by considerations of where governments can make financial savings. In contrast SROI advocates a triple bottom line approach in which environmental and social impacts are evaluated alongside economic benefits. In SROI analysis an attempt is made to measure and value the things that really matter to people as well as those things that are easy to count, or that are traded in the market place.

SROI analysis culminates in a ratio, or ‘social value’, generated for investment. More importantly, however, SROI tells the story of how value is created. It is designed to inspire those delivering and using services to engage with the change to which their work contributes.

The policy

Until recently the Local Enterprise Growth initiative (LEGI) was the government’s flagship economic development programme. It aimed to transform deprived communities by raising levels of enterprise and employment, and by fostering entrepreneurial and work-oriented aspirations. It was different from its predecessors in that its programmes were designed to operate over a longer period (10 years) and were more focused on outcomes. For each programme central government specified the outcomes it wanted to see and it was up to local authorities to determine how those outcomes would be achieved.
While this research was taking place a restructure at the Department for Communities and Local Government saw the creation of a new regeneration programme – the Working Neighbourhoods Fund (WNF) – which means that no additional rounds of LEGI will be announced. It is intended, however, that the WNF will have a strong enterprise focus. Lessons from this research may offer much to inform the design and monitoring of the new fund.

The scope

St Helens is the 47th most deprived borough in the UK⁵. This is only the first year of a long-term study in the borough. After one year we can measure changes in enterprise and employment in St Helens but it is not possible to take a view on whether these changes are the result of activities funded by the LEGI. Instead, we have made some projections for the ‘social return’ we could expect to see at the end of three years. At this stage, therefore the findings focus on what we have learned from the process as well as qualitative findings from conversations with a range of stakeholders.

The findings

• Our research has found that if St Helens LEGI were to meet its goals in the first three years, this would represent a social return of 14:1, or £14 worth of social value for every £1 invested. This is based on achievement of all the expected outcomes and projecting these over eight years⁶. Where the indicator of the outcome does not already have a market value, financial proxies have been used. In future years the intention is to compare the actual return against this projection.
• As well as the headline outcomes of increasing local enterprise, employment and self-employment, our research modelled a range of other benefits that local people believed would flow from getting more people into work – such as improvements in health, crime and quality of life. However, only those things for which there were sufficient data on which to base calculations were included in the social return projections. According to these calculations, the main recipients of the value generated by the LEGI will be people who are economically inactive.
• The SROI projections are at present based on absolute rather than relative numbers, for example the value of jobs to people, rather than the value of reducing inequality, irrespective of how many jobs are created⁷. The next update will be based on a more sophisticated approach that measures and monetises the value of the improved social outcomes that a more equal society brings about, and uses that to derive the calculations.
• We also looked at the issue of valuing equity in relation to those experiencing more acute exclusion, such as women and people on incapacity benefit (IB). When we weighted the benefits to these groups more heavily we found that the social return ratio for these groups was similar, at 15:1, to the overall 14:1 ratio. In the next phase of the research it should be easier to assess whether the emphasis on these groups is sufficient in the overall mix of projects

The recommendations

This research highlights the need for new approaches to regeneration policy and how it is measured. Our recommendations are as follows:

Measurement

• Don’t assume that public investment alone has created change. SROI offers a useful framework within which to examine the information required, in order to assess change arising from an investment. Deadweight and attribution are central considerations for an SROI analysis, ensuring that influential factors additional to the investment itself are taken on board.
• Targets can create perverse incentives and do not necessarily reflect the reality of people’s lives. LEGI targets – across the programme – focus on full employment, yet experience in St Helens suggests that this is often unrealistic for people who have been excluded for a long time from the labour market.
• Measure and value distance travelled. Stepping stones into employment need to be created, and the length of the journey into employment needs to be recognised. The pre-employment phase – intensive coaching and work on attitudinal change with the unemployed, as well as awareness-raising among employers – is especially important. For example, it may take up to a year to build someone’s confidence to the point where serious employment discussions can begin.
• Have a vision for how change will happen. A theory of change is needed to set out the logical flow from the investment programme through to how change will be created. This is an essential building block for planning any initiative of this kind. Theories of change should be explicit and the process through which they are developed should be transparent.
• Measure the things that matter. If not addressed, gaps in the data will hinder the ability to carry out future evaluations. Timely and appropriate data are missing at a local and national level for a number of indicators. Data tend to be skewed towards bigger business and larger areas, although the problems of deprivation are often concentrated in smaller localities and the many smaller enterprises that will never grow to be eligible for VAT registration.
• Involve stakeholders in setting and measuring indicators. One of the biggest challenges in moving towards outcomes-focused measurement is engaging staff that have been trained in a target-driven culture. It is important to involve them fully in developing systems in which they take more responsibility for understanding and managing the change that their service is bringing about.
• Take a long-term view. Some outcomes from a programme such as the LEGI are not likely to accrue for a generation. Timescales within which measurement takes place need to reflect the actual pace of change, rather than policy or political cycles.
• Economic development policy should be underpinned by an awareness of its impact on inequality. New research from nef has found that growth in share of enterprise within a local authority does not necessarily lead to a similar increase in the most deprived neighbourhoods (report forthcoming). Apart from employment there are other impacts of a decline in enterprise that need to be considered, such as a decline in local shops and services that reduce people’s access to these amenities.
• Take risks. Innovation in economic development should not be limited to projects themselves but should also apply to the theory of change developed to help implement them. An acceptance of risk is essential to achieve innovation. Consultation with local people in St Helens highlighted the importance of the following factors in the development of a theory of change for economic development:
1. Formal and informal networks and their role in getting people into the labour market (see the Bizz Fizz approach to building on networks in deprived communities)⁸.
2. The informal economy and its importance to those who are on low incomes and/or intending to move into self-employment.
• Any job is not enough. While paid employment was very important to those seeking work, meaningful work was also considered important. Negative experiences of casual and insecure employment can be discouraging for those with already low levels of confidence. Future plans for the WNF should also take account of the quality of jobs created in terms of sustainability, conditions, satisfaction and so on. These factors matter to stakeholders.
• Develop a joined-up approach. WNF needs to dovetail policy in this area with initiatives by other departments. For example, the recent Green Paper on employment made no mention of enterprise, despite its importance to the Department of Communities and Local Government and the Department for Business, Enterprise and Regulatory Reform. Neither did the Green Paper refer to sustainable development, a significant policy concern both for the Department for Environment, Food and Rural Affairs and in the 2006 Local Government White Paper.
• Take the politics out of regeneration. There is still a significant gap in the evidence about what works in relation to economic development policy. It is essential that policy becomes more informed and less political if we are to build the evidence base and use public investment as effectively as possible to combat inequality.

Conclusions
This research has set out to explore the use of SROI as a way of evaluating an economic development programme. Our report makes the case for using a SROI approach to help build an evidence base and to enable the more effective targeting of public policy and investment. Assuming the right data systems are in place, the next steps will be to compare the SROI projections with actual outcomes in the second year. There are strong grounds for optimism that the SROI process and its results will provide better measures of effectiveness both for the LEGI and for future WNF programmes on local economic development.

The moral and economic arguments for reducing inequality and raising the incomes of the poor have been made by others. The UK economy has maintained average growth rates of 2.5 per cent since the 1950s but the benefits of this growth have not been equally distributed, which has led to more concentrated and entrenched poverty and social exclusion. It is essential that policy and investment should be effectively focused on those areas where it will have the most impact.

Finally, future economic development programmes need to be underpinned by a coherent and innovative theory of change. Evaluations need to be focused on the achievement of sustainable development outcomes, which directly relate to addressing the causes of inequality.

The whole report is available here.

Notes:

1. See Dorling D et al (2007) Poverty, Wealth and Place in Britain, 1968 to 2005 (York: Joseph Rowntree Foundation); cf. Orton M and Rowlingson K (2007) Public Attitudes to Economic Inequality (York: Joseph Rowntree Foundation).
2. Dorling D and Thomas B (2007) Identity in Britain: A Cradle-To-Grave Atlas (Policy Press, Bristol).
3. See for example, evaluations from Enterprise Zones, Regional Selective Assistance and Phoenix Development Fund: Kornblatt T and Troni L (2006) City Markets: business location in deprived areas (London: ippr); ODPM (2003) Business-led Regeneration of Deprived Areas: A Review of the Evidence Base, Research Report 5 (London: Office of the Deputy Prime Minister); and Ramsden P (2005) Phoenix Fund: final evaluation for SBS, availabke from www.sbs.gov.uk [26 February 2008].
4. See Simms A et al (2005) Clone Town Britain: The survey results on the bland state of the nation (London: nef) and www.pluggingtheleaks.org [26 February 2008].
5. http://www.communities.gov.uk/communities/neighbourhoodrenewal/deprivation/deprivation07/
6. Five years is a standard period over which to project the returns, and so for a three-year programme the projection is over eight years.
7. This was close to the one-year LIBOR rate as of January 2007 when the calculations were made. It would also be possible to take a longer-term average but it is unlikely that this would have an impact on the overall SROI ratio.
8. Issues of equity are only considered in so far as a closing of the St Helens enterprise gap makes the achievement of the employment outcomes more likely.