Evaluation of Sistema Scotland
Sistema Scotland is a charity on a mission to transform lives through music.
Through its Big Noise programme, Sistema Scotland believes that children from disadvantaged backgrounds can gain significant social and wellbeing benefits and acquire a range of life skills by being part of an intensive and immersive symphony orchestra programme. Based on the Venezuelan El Sistema model, Sistema Scotland aims to use music making to foster wellbeing, confidence, self-esteem, pride and aspiration among the children and young people taking part.
Currently there are four Big Noise centres in Scotland delivering a long-term orchestral programme for preschool and school-age children and young people; currently totalling approximately 2,500 participants. Big Noise Raploch, in Stirling, was the first site in Scotland – established in 2008, with Big Noise Govanhill, in Glasgow, following in 2013. Big Noise Torry, in Aberdeen, opened in 2015. The fourth Big Noise centre in Douglas, Dundee opened in autumn 2017.
Our evaluation work
Since 2013 we have been leading the evaluation of Sistema Scotland’s Big Noise programme. The evaluation aims are to make clear key features of Big Noise delivery and describe the impacts of the programme.
What sets our evaluation of Big Noise apart from most other studies in this field is the depth of evaluative approaches and measures used as well as the range of expertise contributing to the study. In the short-term the evaluation uses primarily qualitative methodologies to understand the delivery of Big Noise and the nature of its impacts. In the longer term we will examine quantitative life-course outcomes of participants in comparison with an appropriate control group. This will include analysis of education, health, social care, welfare system and justice system records in the decades to come.
Main findings
So far we have found that there are seven main areas of a child or young person’s life that can be positively affected by attending Big Noise:
Educational: Concentration, listening, co-ordination, language development, school attendance, school outcomes.
Life Skills: Problem solving, decision-making, creativity, determination, self-discipline, leadership.
Emotional: Happiness, security, pride, self-esteem, emotional intelligence, an emotional outlet, resilience.
Social: Social mixing, social skills, cultural awareness, diverse friendships, strong friendships, support networks.
Musical: Strong instrument skills, reading music, performance skills, music career options, access to other music organisations.
Physical: Healthy snacks, opportunities for games/ exercise, creating healthy habits for adulthood.
Protection: Someone to confide in, calm environment, safe environment, reduced stress.
The evaluation is led by Chris Harkins and governed by a senior advisory group.
Publications
Published in 2014, this evaluation plan sets out our innovative life course study. Chris Harkins has also written a blog outlining his vision for the evaluation.
Arts and health: systematic reviews
To inform the evaluation of Big Noise we commissioned three distinct systematic evidence reviews to assess the current evidence on the role the arts can play in regeneration, grassroots positive social change and in improving the health and wellbeing of disadvantaged communities.
The initial findings were published in 2015 and relate to the evaluation of Big Noise centres in Govanhill, Glasgow and Raploch in Stirling. The report, summary report (also produced in Romanian, Slovakian, Urdu and Polish) and technical appendices are also available.
As part of our evaluation we commissioned a filmmaker to work with six orchestra members from Big Noise Raploch - view the film.
Dr Lisa Garnham also wrote two blogs, reflecting on the different methods used to engage young people with the research, including drawing and filmmaking. A further blog looks at the challenges of measuring and evaluating the impact of arts on health.
Principles for Social Regeneration
Based on the 2015 initial findings this paper proposes and discusses seven principles for effective social interventions and social regeneration.
Big Noise Torry - initial findings report
Published in 2017, this evaluation concerns the Big Noise centre in Torry, Aberdeen. An infographic is also available summarising some key information within the report.
People change lives: Evaluation of Sistema Scotland’s Big Noise programme
This report, published in 2019, consolidated and summarised the learning from the first five years of a ‘life-course evaluation’ of Sistema Scotland’s Big Noise programme.
Statistical analysis of educational outcomes among Big Noise Raploch participants
Published in November 2022, this report focuses on the educational outcomes of Big Noise Raploch participants of school leaving age and uses quantitative statistical analysis of life-course outcomes to assess the long-term impact of the Big Noise programme.
Journal Publications
Harkins C, Garnham L, Campbell A, Tannahill C. Hitting the right note for child and adolescent mental and emotional wellbeing: a formative qualitative evaluation of Sistema Scotland’s “Big Noise” orchestral programme. Journal of Public Mental Health 2016;15:125-36.
Garnham LM, Campbell A. ‘It makes me feel happy and joyful’: the evaluation of arts-based social interventions in public health. Journal of Public Health 2016;38(4);589-e591.
Garnham LM, Harkins C. ‘Transforming lives through music’ as a public health intervention: further reflections on our evaluation of Sistema Scotland. Journal of Public Health 2017;39(4):793–795