Evaluation of Sistema Scotland

Sistema Scotland is a charity on a mission to transform lives through music. 

Through its Big Noise programme, Sistema Scotland believes that children from disadvantaged backgrounds can gain significant social and wellbeing benefits and acquire a range of life skills by being part of an intensive and immersive symphony orchestra programme. Based on the Venezuelan El Sistema model, Sistema Scotland aims to use music making to foster wellbeing, confidence, self-esteem, pride and aspiration among the children and young people taking part. 

Currently there are four Big Noise centres in Scotland delivering a long-term orchestral programme for preschool and school-age children and young people; currently totalling approximately 2,500 participants. Big Noise Raploch, in Stirling, was the first site in Scotland – established in 2008, with Big Noise Govanhill, in Glasgow, following in 2013. Big Noise Torry, in Aberdeen, opened in 2015. With the fourth Big Noise centre in Douglas, Dundee opening in autumn 2017. 

Our evaluation work

Since 2013 we have been leading the evaluation of Sistema Scotland’s Big Noise programme. The evaluation aims are to make clear key features of Big Noise delivery and describe the impacts of the programme.

What sets our evaluation of Big Noise apart from most other studies in this field is the depth of evaluative approaches and measures used as well as the range of expertise contributing to the study. In the short-term the evaluation uses primarily qualitative methodologies to understand the delivery of Big Noise and the nature of its impacts. In the longer term we will examine quantitative life-course outcomes of participants in comparison with an appropriate control group. This will include analysis of education, health, social care, welfare system and justice system records in the decades to come.

Main findings

So far we have found that there are seven main areas of a child or young person’s life that can be positively affected by attending Big Noise:

Educational: Concentration, listening, co-ordination, language development, school attendance, school outcomes.

Life Skills: Problem solving, decision-making, creativity, determination, self-discipline, leadership.

Emotional: Happiness, security, pride, self-esteem, emotional intelligence, an emotional outlet, resilience.

Social: Social mixing, social skills, cultural awareness, diverse friendships, strong friendships, support networks.

Musical: Strong instrument skills, reading music, performance skills, music career options, access to other music organisations.

Physical: Healthy snacks, opportunities for games/ exercise, creating healthy habits for adulthood.

Protection: Someone to confide in, calm environment, safe environment, reduced stress. 

The evaluation is led by Chris Harkins, assisted by Katie Moore. The evaluation is governed by a senior advisory group.

Publications

Evaluation Plan

Published in 2014, this evaluation plan sets out the innovative life course study devised by the GCPH. Chris Harkins has also written a blog outlining his vision for the evaluation

Arts and health: systematic reviews

To inform the evaluation of Big Noise the GCPH commissioned three distinct systematic evidence reviews to assess the current evidence on the role the arts can play in regeneration, grassroots positive social change and in improving the health and wellbeing of disadvantaged communities. A concise synthesis of all three reviews is also available. 

Initial findings

The initial findings were published in 2015 and relate to the evaluation of Big Noise centres in Govanhill, Glasgow and Raploch in Stirling. The report, summary report (also produced in Romanian, Slovakian, Urdu and Polish) and technical appendices are also available. 

As part of the GCPH evaluation of Sistema Scotland, we commissioned a filmmaker to work with six orchestra members from Big Noise Raploch - view the film. 

Dr Lisa Garnham (formerly of the evaluation team) also wrote two blogs, reflecting on the different methods used to engage young people with the research, including drawing and filmmaking.  A further blog looks at the challenges of measuring and evaluating the impact of arts on health.

Principles for Social Regeneration

Based on the 2015 initial findings this paper proposes and discusses seven principles for effective social interventions and social regeneration. 

Big Noise Torry - initial findings report

Published in 2017, this evaluation concerns the Big Noise centre in Torry, Aberdeen. An infographic is also available summarising some key information within the report. 

Journal Publications 

Harkins C, Garnham L, Campbell A, Tannahill C. Hitting the right note for child and adolescent mental and emotional wellbeing: a formative qualitative evaluation of Sistema Scotland’s “Big Noise” orchestral programme. Journal of Public Mental Health 2016;15:125-36. Open Access 

Garnham LM, Campbell A. ‘It makes me feel happy and joyful’: the evaluation of arts-based social interventions in public health. Journal of Public Health 2016;38(4);589-e591. 

Garnham LM, Harkins C. ‘Transforming lives through music’ as a public health intervention: further reflections on our evaluation of Sistema Scotland. Journal of Public Health 2017;39(4):793–795

Related topic: Places and regeneration