Evaluation of Sistema Scotland

We worked with Sistema Scotland to evaluate the newly initiated Big Noise Govanhill programme in Glasgow and support the ongoing monitoring and evaluation of Big Noise Raploch near Stirling.

Within the evaluation, we led in the collection of reliable data that will describe key outcomes of Sistema’s work on programme participants, their families, their community and the wider benefits of this work at a regional or societal level. The evaluation was longitudinal and incorporated a mixed-method design.

The data was analysed in a way that will help inform how the respective Sistema programmes develop, as well as making clear the characteristics of Sistema’s approach that are important to achieving positive outcomes on a range of agreed indicators.

The evaluation was led by Chris Harkins, Senior Public Health Research Specialist and Dr Lisa Garnham, Public Health Research Specialist.

Aims

The two overarching aims for the evaluation in the initial stage:

  1. To assess, over the long term, the outcomes of the work of Sistema Scotland in terms of social development and impacts on health and wellbeing at the individual, family, community and regional or societal levels within each programme site.
  2. To gain insight into Sistema Scotland’s approaches to programme delivery and the characteristics of implementation which are critical to enhancing inclusion, engagement and retention and achieving positive outcomes at the individual, family, community and regional and societal levels.

The vision for the evaluation was to capture important learning from the implementation and impact of Sistema’s work in Govanhill and Raploch as a means of furthering understanding, within Scotland and beyond, of effective social interventions/regeneration within disadvantaged areas.

The evaluation ethos was inclusive, reciprocal and supportive. We held responsibility for leading the evaluation process, from planning through to completion and dissemination, and this meant working closely with Sistema and a range of local and national partners.

Advisory group

A senior, multidisciplinary evaluation advisory group was established to ensure sustained working relationships with other researchers, practitioners and policy-makers within Glasgow, Raploch and nationally.

The function of the evaluation advisory group was to provide guidance in relation to the approach, content and implementation of the evaluation and dissemination processes; and to contribute additional hands-on support for particular aspects of the process, as required.

Publications

Access the initial findings report. 

summary of the findings in this report is also available, as is a report for communities involved in the research, available in English as well as Romanian, Slovak, Urdu and Polish.

Big Noise study: report for communities 
Romanian translation
Slovak translation
Urdu translation
Polish translation

Download a summary handout of the research (PDF).

As part of the GCPH evaluation of Sistema Scotland, we commissioned a filmmaker to work with six orchestra members from Big Noise Raploch - view the film.

Dr Lisa Garnham also wrote two blogs, reflecting on the different methods used to engage young people with the research, including drawing and filmmaking.

Access further information on the Sistema projects.