Guest Editors

–    Marie J. Bouchard, Full Professor, Université du Québec à Montréal, Presidentof the International Scientific Commission on Social and Cooperative Economy of CIRIEC

–   Damien Rousselière, Full Professor, AGROCAMPUS OUEST, France

 

Is the proof in the pudding? Measuring the impacts of social and solidarity economy. Why? When? How?

The interest in impact measurement of the social and solidarity economy (SSE) is reaffirmed year upon year. Each organization, at some time or another in its development, will come to ask itself about the effects that it generates in order to improve its efficiency and the efficacy of its actions, but also to enhance its recognition among the external stakeholders. Moreover, the financial  backers – whether public, philanthropic or private – also express their desire to better know how their financial support contributes to solving or resorbing problems. This renewed interest manifests itself in requirements of demonstration for which SSE organizations often feel poorly of inadequately equipped. This special edition of the journal wants to focus on the issues of impact measurement in the SSE.

 

Evaluation of the SSE is not a new affair

Evaluation in the SSE is not a new affair. In fact, organisations of the SSE are well accustomed to it (Bouchard, Bourque & Lévesque 2000). A CIRIEC working group reviewed the evaluation practices of organizations and sectors of the SSE from an international perspective (Bouchard 2009). Results showed that evaluation modalities vary considerably, serving various purposes, either to meet funder’s expectations, improve practices through formative evaluation, or produce social balance sheets to mobilize stakeholders and increase legitimacy. Practices range from accountability centred on objectives of results determined a priori (as is the case for government programmes), to voluntary disclosure centred on responsible social or environmental practices (cooperative balance, societal balance, social audit). Certain sectors of the SSE have measured their economic impact (cost-benefit analysis, intersectoral input-output model, avoided social costs, expanded value added statement), their social impact (health and welfare of the population), or environmental impact (carbon footprint, product life-cycles). Important observations are that evaluation has become an mandatory function of management, but more crucially that it plays an implicit role in the governance of organizations.

 

New societal and environmental challenges, new financial players

The concern about severe and persistent social and environmental problems (growing inequalities, unsustainable development, climate change, etc.) has increased and become global, as shown by the 2030 United Nations Sustainable Development Agenda. Calls are being heard for a better  assessment of the solutions brought to help curbing them. In this context, evaluation becomes a strategic issue.

 

Now, the rise of the current of social investment over the past decade has changed the picture. The term “measurement” has become more and more frequently used, alongside terms such as “evidence”, “scaling” and “impact”. At the heart of the model is the need felt by the financial backers, whether public or charitable, beyond mere accountability, to « see » the effects of their

 

action. In particular, we see a new generation of initiatives and impact measurement indicators focused on the needs of the financial world with regards to social or environemental responsibility: Impact Reporting and Investments Standards (IRIS), Social Return on Investment (SROI), Global Impact Investing Ratings System (GIIRS) supported by the B-Lab, Measurement and monitoring of the social impact of investments in France (MESIS). The current of social finance goes farther, in seeking to mobilise venture capital to generate a return both social and financial. This model presumes a better effectiveness of social services through the injection of private capital and by the obligation of demonstration of results, as can be seen with the contracts of social impact bonds  (SIB). While the demonstration has not been made (Maier & Meyer 2017; McHugh et al. 2013), the craze for this type of approach nevertheless spreads very quickly.

 

Consequences for the social an solidarity economy

Around one hundred different approaches, tools and methods can be identified for social impact measurement as conceived for social investment (Salathé-Beaulieu et al. 2019). The diversity and fragmentation of these initiatives is seen as a common problem for the organisations of the SSE and for their financial backers. Various views on the role of the SSE offer many different angles from which to appreciate its performance (Carman 2011, Gadrey 2005). As a consequence, various perspectives can lead to contradictory views about what should be evaluated, who should evaluate, when, what for and for whom (Nicholls 2018).

 

This new context puts pressure on the SSE. On the one hand, the institutional setting (new public management, evidence-based policies, pay-for-performance schemes) and the accentuation of market regulation (competition for niches but also for subsidies and public contracts) increase the importance of demonstrating efficiency and accounting for responsible management of resources. On the other hand, philanthropic capitalism requires to demonstrate impacts as it is at the heart of its remuneration scheme. With this, we are moving away from a culture of redistribution and towards that of social finance. This has important consequences for the practices and public policies regarding SSE.

 

The SSE can be seen as an important player in finding lasting solutions on a number of the sustainable development goals at different scales (Brown et al. 2015; Chaves & Monzon 2012, Duguid 2017; Hudon & Huybrechts 2017; Noya & Clarence 2007; Seyfang & Smith 2007; Vézina, Malo & Ben Selma 2017). Many case studies have exposed the contribution of SSE organizations in various national settings (see COPAC, ILO and UNRISD¹). It remains to be seen how to move from promises to measurable evidence. The craze for impact measurement can however prove to be demanding for SSE organizations, as it requires additional resources to manage evaluation expectations. The ability of organizations to cope with the demands of evaluation varies greatly (Carman & Fredericks 2010). Approaches are more or less well adapted to the SSE (Besançon & Chochoy 2019, Liket et al. 2014), some can be subject to controversy (Cupitt 2015, Mertens, Xhauflair & Marée 2015). For example, standardized assessment methods and indicators are said to be poorly suited to the SSE, given the variety of its forms and the numerous sectors of activity where it is present (GECES 2014), which may even lead stakeholders to refuse to submit to it (TIESS 2019).

 

This vogue for evaluation can nevertheless represent an opportunity for the SSE to « demonstrate » its value (Vo & Christie 2018) and develop stronger relationships with its stakeholders (Greiling & Stötzer 2015). The specificity of the mode of production of social economy organizations suggests however to go beyond traditional measurement tools (Mertens & Marée 2015; Mook, Richmond & Quarter 2003).

 

About this special issue:

The question that arises is the effect – not to say the impact! – on the SSE of the will to measure its impact. What are the ramifications of impact measurement and what consquences do they have on SSE activities? What are the similarities and differences between evaluation and impact measurement as conceived from a SSE or from a social finance perspective? What are the effects of different evaluation and impact measurement frameworks on the practices and development of SSE and its ecosystem? Knowing the particularities of the SSE, should we consider using specific methodologies and indicators so that its impacts can be recognized at their true value? Given the complexity of social and environmental problems, does the current trend for impact measurement oversimplify the issues? How should the SSE actors position themselves in this context?

 

Among the themes that can be the subject of contributions:

• Overview of different approaches to measuring social, economic and environmental impacts of the
• Analysis of emerging trends in the SSE financing (new public management, social finance, impact investment, social impact bonds …) and their impact on SSE evaluation and impact assessment.
• Representations of the SSE in different methods or approaches to impact measurement
• Methodological challenges associated with measuring economic, social and environmental impacts of the
• Critical views about impact measurement of the

 

Submitting a proposal

Timeline:

• Call for papers opens: September 30, 2020
• Call for papers closes: December 30, 2020 Publication: Vol. 93, I, 2022

Submissions with indication              “Special              Issue” to http://Mc.manuscriptcentral.com/apce

 

¹ See: COPAC: http://www.copac.coop/transforming-our-world-a-cooperative-2030-cooperative-contributions- to-sdg-3/; ILO : https://sustainabledevelopment.un.org/content/documents/1247ilo.pdf; UN: https://www.un.org/development/desa/cooperatives/; UNRISD: http://www.unrisd.org/80256B3C005BD6AB/(httpEvents)/ABE0A432A9D42782C12583AE004AF31C?OpenDocument.

References

Besançon, E. & N. Chochoy (2019) « Mesurer l’impact de l’innovation sociale: quelles perspectives en dehors de la théorie du changement? » RECMA, (2), 42–57.

Bouchard, M. J. (ed.) (2009), The Worth of the Social Economy. An International Perspective,  Brussels, PIE Peter Lang, Ciriec collection Social Economy and Public Economy

Bouchard, M. J., G. L. Bourque & B. Lévesque, «L’évaluation de l’économie sociale dans la perspective des nouvelles formes de régulation socio-économique de l’intérêt général», Cahiers de recherche sociologique, mars 2001, p. 31-53

Bouchard, M. J. & D. Rousselière (Eds) (2015), The Weight of the Social Economy. An International Perspective, Bruxelles, Berne, Berlin, Frankfurt am Main, New York, Oxford, Wien, PIE Peter Lang, 360 p.

Brown, L., C. Carini, J. Gordon Nembhard, L. H. Ketilson, E. Hicks, J. McNamara, R. Simmons (Eds.) (2015), Co-operatives for sustainable communities – Tools to Measure Co-operative Impact and Performance, Canada, Co-operatives and Mutuals Canada – Centre for the Study of Co-operatives.

Carman, J. G. (2011), “Understanding evaluation in nonprofit organizations”, Public performance and management review, 34/3, p. 350-377.

Carman, J. G. & K. A. Fredericks (2010), “Evaluation capacity and nonprofit organizations: is the glass half-empty or half-full?”, American Journal of Evaluation, 31/1, 84-104.

Chaves, R. & J. L. Monzon (2012), “Beyond the crisis : the social economy, prop of a new model of sustainable economic development”, Service Business, 6/5, 5-26, DOI 10.1007/s11628-011-0125- 7

Cupitt, S. (2015, May), Randomised controlled trials – gold standard or fool’s gold? The role of experimental methods in voluntary sector impact assessment, London (UK), The National Council for Volountary Organisations (NCVO).

Duguid, F. (2017), “Non-Financial Tools and Indicators for Measuring the Impact of Co-operatives”,

Journal of Co-Operative Accounting and Reporting, 5(1), 40–54.

Gadrey, J. (2005), « L’invention de l’utilité sociale des associations en France: à la recherche de conventions, de régulations, de critères et de méthodes d’évaluation », Économie et Solidarités, 36/1, 7–26.

GECES Sub-group on Impact Measurement (2014), Proposed Approaches to Social Impact Measurement in European Commission legislation and in practice relating to: EuSEFs and the EaSI. Brussels, GECES.

Greiling, D. & S. Stötzer (2015), “Performance Accountability as a Driver for Changes in Nonprofit– Government Relationships: An Empirical Insight from Austria”, Voluntas: International Journal of Voluntary and Nonprofit Organizations, 26/5. 1690-1717. DOI 10.1007/s11266-015-9609-8

Hudon, M. & B. Huybrechts (2017), “From distant neighbours to bedmates: exploring the synergies between the social economy and sustainable development”, Annals of Public and Cooperative Economics, 88/2, 141-154.

Liket, K. C., M. Rey-Garcia, K. E.H. Maas (2014), “Whay aren’t evaluations working and what do we do about it? A framework for negociating meaningful evaluations in nonprofits”, American Journal of Evaluation, 35/2, 171-188.

Maier, F., & Meyer, M. (2017), “Social impact bonds and the perils of aligned interests”,

Administrative Sciences, 7/3, 24.

McHugh, N., S. Sinclair, M. Roy, L. Huckfield & C. Donaldson (2013), “Social impact bonds. A wolf in sheep’s clothing?”, Journal of Poverty and Social Justice, 21/3, 247-257.

Mertens, S., & Marée, M. (2015), “So, What Does a Social Economy Enterprise Produce?” 141-160 In

1. J. Bouchard & D. Rousselière (Eds.), The Weight of the Social Economy, An International Perspective, Liège/Brussels, CIRIEC/ Peter Lang.

Mertens, S., V. Xhauflair & M. Marée, M. (2015), Questioning the social return on investment (SROI), Liège, SOCENT Publication, no 2015-1. iap- socent.be/sites/default/files/SOCENT%20Pub%202015-01%20Mertens%20et%20al%20NEW.pdf

 

Mook, L., B. J. Richmond, & J. Quarter (2003), “Integrated social accounting for nonprofits: A case from Canada”, Voluntas: International Journal of Voluntary and Nonprofit Organizations, 14/3, 283–297.

Nicholls, A. (2018). “A General Theory of Social Impact Accounting: Materiality, Uncertainty and Empowerment”, Journal of Social Entrepreneurship, 9/2, 132–153. https://doi.org/10.1080/19420676.2018.1452785

Noya, A. & E. Clarence (2007), The social economy: building inclusive economies, Paris, OECD. Salathé-Beaulieu, G., M. J. Bouchard & M. Mendell (2019), Sustainable Development Impact

Indicators for Social and Solidarity Economy. State of the Art, Geneva, UNRISD. https://www.unrisd.org/80256B3C005BCCF9/(LookupAllDocumentsByUNID)/5A72131687B4AE4 78025848D003847E4?OpenDocument

Seyfang, G. & A, Smith (2007), “Grassroots innovations for sustainable development: Towards a new research and policy agenda”, Environmental Politics, 16/4, 584-603, DOI: 10.1080/09644010701419121

Vézina, M., M.-C. Malo & Mé Ben Selma (2017), “Mature social economy enteprise and social innovation : the case of Dejardins environmental fund”, Annals of Public and Cooperative Economics, 88/2, 257-278.

Vo, A. T., & C. A. Christie (2018), “Where Impact Measurement Meets Evaluation: Tensions, Challenges, and Opportunities”, American Journal of Evaluation, 39/3, 383–388. https://doi.org/10.1177/1098214018778813

TIESS (2019), Déclaration de Montréa sur l’évaluation et la mesure d’impact social, Montréal, Territoires innovants en économie sociale et solidaire, http://www.tiess.ca/declaration/