Research and Evaluation

Evaluation

From its earliest days, Roundabout recognised the importance of monitoring and evaluating its work, and developed various approaches to ensure this happened. Initially, this was through the feed-back between service users and dramatherapists and between staff, parents and carers and Dramatherapists, through the supervision of all our work by the Project Directors, and through written reports.

By reflecting on the information we collect, we ensure our projects are meeting the needs of the service users and our own high standards. Because feed-back and supervision happens every week, we can respond quickly to changing situations, areas of difficulty and any other challenges. And because we work as a team, we can draw on the skills of everyone in that team to make sure we respond effectively.

Over the years, monitoring and evaluation has become increasingly important throughout the Voluntary Sector, and the demand from funders of all types for ‘evidence based need’ has increased. The nature of our work – the diversity of our client base, the sensitive nature of the therapeutic process, and the fact that our work generates ‘soft’ rather than ‘hard’ outcomes – has presented Roundabout with some interesting challenges in finding or creating evaluation methods that are sensitive to the needs of our service users and to the dramatherapy process.

Thanks to generous support from the Terpsichore Trust, the Wates Foundation and the City Bridge Trust we have been able to invest the necessary time over the last few years in order to meet that challenge. In work co-ordinated by Susan Crockford, we undertook an audit of all the methods our dramatherapists currently employed, and what was being used in the wider community of arts therapists, and amongst practitioners working with relevant client groups.

It became apparent that there was no single methodology that could be applied across all our projects, and even after sub-dividing our work into service user categories, it was still proving difficult to find suitable methods. Until we heard about Psychlops (see below for a full description). We also began to use, when appropriate, recognised systems such as Strength and Difficulties Questionnaire and, for working with adults with learning disabilities, CORE LD.

As well as working in partnership with King’s College, London, to adapt Psychlops to the school setting, we have experimented with systems we created ourselves. Finding the right Monitoring and Evaluation methodology for Roundabout is still a work in progress. We are currently embarking on a trial of the Goal Attainment Scoring (GAS) methodology.

To summarise, we monitor and evaluate our projects and outcomes through:

1. Project evaluation methods:

Psychlops

PsychlopsKids

PsychlopsYP

Hope and Wishes

Behavioural Summarised Evaluation

CORE questionnaire specially adapted for people with learning disabilities

Roundabout’s own evaluation tool using ‘faces’ and tick boxes, for adults with more profound learning disabilities

Verbal and written feed-back from family, teachers, carers, support workers etc

Observations of the Dramatherapists

Written reports

2.Statistical monitoring methods:

Collecting and collating statistics on the projects we have facilitated, and the numbers of people we have worked with, as well as their diagnosis, ethnicity and sex.

Research

Psychlops Kids

During 2010/11 Roundabout completed the development and piloting of ‘Psychlops Kids’. This was a major undertaking for Roundabout and resulted in the development of a new way to evaluate the impact of dramatherapy. ‘Psychlops Kids’ is a self-report methodology of evaluating therapy designed specifically for children aged between 7 and 13 years old, offering them the opportunity to say in their own words what their worries are and how dramatherapy might have helped. ‘Psychlops Kids’ is based on ‘Psychlops’, a mental health outcome measure for adults developed by primary health-care professionals and therapists based at King’s College London. Between 2008 and 2011 Roundabout worked with this team to develop a version suitable for young service users as research had discovered no other self-report evaluation for under 13 year olds. Roundabout has been in a unique position to develop and pilot such a measure because of the large number of children and young people we work with each year.

The ‘Psychlops Kids’ forms identify, at the beginning of therapy, issues that are bothering the child or young person and also how they are feeling about going to dramatherapy. Through the pilot process the Roundabout team were able to add creative forms of evaluation to the format established by the original team. ‘Psychlops Kids’ was piloted 3 times over 2 years and we were able to elicit feed-back from children, young people, the adults who supported them to complete the forms and the therapists. This led us to the final version which was used during the academic year September 2010 to July 2011. Our work with ‘Psychlops Kids’ has shown that this form of evaluation really does give the children and young people ‘a voice’. The pre-therapy forms provide a helpful starting point for the therapeutic work, the mid-therapy forms provide an opportunity to reflect on change and the impact of therapy, and the forms completed after therapy has ended offer both qualitative and quantitative results.

Results

The results have been very positive indicating that dramatherapy is really making a difference in children and young people’s lives. The results of 66 completed before and after Psychlops Kids forms showed an effect size of 0.9. The effect size calculation is based on the section in Psychlops Kids that can be given a numerical scoring and shows that for those who completed the forms they felt significantly better after the dramatherapy programme. Other data from the forms indicated that the issues children were most worried about were: school problems, peer problems, self-esteem issues, anxiety about school transfer and family problems.

Accessing Psychlops Kids

This book edited by Deborah Haythorne, Lauraine Leigh, Irvine Gersch and Ann Dix is the first book to specifically evaluate the unique value of drama therapy in the education environment. A variety of highly experienced drama therapists, educational psychologists and childhood experts discuss the benefits of dramatherapy to children and young people and also in relation to the involvement of teachers, the multi-disciplinary team and families. See here for further information and to order this book: