I recently blogged about using drawing as a method to talk with young children, as part of the GCPH evaluation of Sistema Scotland’s ‘Big Noise’ programmes in Govanhill, Glasgow and Raploch, Stirling.  Sistema Scotland aim to ‘transform lives through music’ by building neighbourhood-based children’s orchestras, and we’ve been looking at the impacts this might be having on health.  I wanted to now also cover my experiences of using filmmaking as a method of data collection.  

Although we knew we could just ask young people to tell us what they thought (unlike the younger children in Govanhill), we wanted to somehow stimulate deeper discussion and capture the experience of being part of a Big Noise orchestra for our evaluation audience.

Making the film

We commissioned a filmmaker to work with us over six hour-long sessions, during which a group of Big Noise participants (aged 12-15) discussed, designed and refined their ideas for their film.  Although they were restricted to a ‘documentary-style’ short film, they had control over what we filmed and who appeared in the film.  

They decided to mix footage of various parts of the programme together with interviews from Big Noise staff, participants (of all ages) and their parents.  They drew up the lists of questions for interviewees and helped to film most of the footage, as well as interviewing many of those who appeared in the film.  Not only did they pick up some film-making and interviewing skills, but this also made for great chemistry on camera! 

Because of this, we heard parents’, children’s, young people’s and musicians’ responses to a range of questions designed by our young filmmakers that we would not normally have asked.  This included questions for Big Noise staff like, “why are some musicians strict and others not?”, and “do you do this for the money or the banter?”.  The fact that our young filmmakers wanted to ask these questions told us as much about their relationships with staff as the answers they gave in return.  

Making the film sparked reminiscence, conversation and heated debate.  We were privileged to be part of our young filmmakers’ discussions about what they felt was most (and least) important about Big Noise, in a way that we could not have been in traditional interviews.  Not only did we gain special insights into their experiences, this process produced a powerful piece of media that communicated those insights in a way that our text-based outputs could not.

Learning from the process

We also learned a lot from the film-making process.  Firstly, it took significantly more organisation than we had anticipated.  The six planning sessions, which were originally intended to be loosely guided discussions, in reality needed to be much more structured to maintain our young filmmakers’ interest, enthusiasm and momentum.

Moreover, the eight interview sessions required us to: convince potential interviewees to appear on camera; have multiple consent forms signed (often by parents of children taking part); find a quiet but visually stimulating environment (with power sockets!) in which to film; and put ourselves, the filmmaker, the interviewees and, often, our young filmmakers, all in the same place at the same time.  

We are heavily indebted to both Big Noise and school staff, who helped us every step of the way.Secondly, we underestimated the scope for audiences’ interpretations of the film to be different from our own.  For example, in the first rough cut of the film we had included a clip of an interviewee making a joke and this was misinterpreted as a serious comment by some of those we screen tested.  It was only at this point that we appreciated how limited our ability to control or moderate the meaning of interviewees’ words was, in comparison with our text-based outputs. 

Moving on

Making the film was one of the most exciting and challenging parts of our evaluation.  It provided us with really useful insights into the way young people see the Big Noise programme and what they get from it.  It is definitely a format I would use again.  Next time though, I’ll have my eyes open to three things: how much commitment is needed to carry it off; the risks and rewards presented by the power of film as a communication tool; and how effective the creative process can be in getting young people to talk about what they think and how they feel. View the film below:

View the film.

Find out more about our evaluation work with Sistema Scotland.

Lisa's colleague Aileen, who works for Audit Scotland, has blogged about her experience working with us on the evaluation and Anthony Clark, Audit Scotland Assistant Director, has also written a blog reflecting on the process of auditing complex community projects.