community servicesevaluationperformance measurementchild and family servicesservice usersaccountabilitysocial justice

Issue Date:

Mar-2006

Publisher:

University of SydneyFaculty of Economics and BusinessPolitical Economy

Abstract:

After two decades of public management reform, the ‘new accountability’ of performance measurement is a routine feature in the relationships between Australian government agencies and the non-profit organisations they fund to provide child and family services. While performance measurement offers to resolve tensions about how governments manage the quality and productivity of contracted services, the indicators they commonly adopt raise well-documented practical, political and epistemological challenges in social services. Left unresolved, these challenges risk biasing representations of service performance, by emphasising the most tangible dimensions of service activities (such as measures of client throughput) over relationship building and care. Capturing only part of service activity compromises the usefulness of performance data for managing quality and outcomes, and denies policy makers critical information about the value and meaning of care in users’ lives.
This thesis identifies and critically explores one set of challenges for performance measurement: the role of service users. Uniquely, I explore how user involvement in social service evaluation can make visible how these services enhance the quality of family and personal life. Using a case study of family support services in New South Wales, the research makes a series of empirical and theoretical contributions to problems of user involvement in social service evaluation.
Firstly, the research examines the performance indicators currently used by government to monitor the efficiency and effectiveness of family support services in NSW. This shows that performance indicators in family support capture output more thoroughly than outcome, and confirms the minimal role that service users play in assessing service quality and outcomes. But while service users are largely excluded from participation in performance measurement, theoretical perspectives as diverse as managerialism and feminism treat service users as well placed to capture and report otherwise elusive information about care quality and outcomes. Further, participation in evaluation facilitates the exercise of users’ rights to self-expression and self-determination in the social service delivery and policy process.
After identifying the widespread exclusion of service users’ perspectives from performance measurement in NSW family support, the thesis makes its more substantial contribution, in documenting findings from a detailed study involving adult family support service users (parents) and their workers (the ‘Burnside Study’). This qualitative study was conducted in four socio-economically disadvantaged service delivery sites located around New South Wales. Using focus group, interview and observational methods and a modified grounded theory approach, the study contributes exploratory evidence of what these service users think of, and how they think about service quality, outcomes, and evaluation in family support.
The parents’ accounts of using family support capture their unfulfilled social ideals and the broader visions of the justice they hoped these social services would help them achieve. Their criteria for measuring service outcomes and service quality, and their views on evaluation methods embody core themes that social theorists have struggled to analyse, about the purpose of social services and the nature of ‘a good life’. The theoretical framework I develop highlights the role of family support in the context of service users’ struggles for social justice, and in particular, their struggles for self-realisation, recognition and respect (Honneth, 1995). The research extends theories of recognition beyond publicly articulated social movements to those struggles in social life and social politics that exist in what Axel Honneth terms the ‘shadows’ of the political-public sphere (2003a: 122).
After establishing a conceptual framework that facilitates deeper interpretation of users’ perspectives, I present the findings in three categories: users’ perspectives on service outcomes; users’ perspectives on service quality; and users’ perspectives on evaluation methods.
The findings show how service users define ‘service outcomes’ in the context of their struggles for recognition and respect, highlighting the contribution welfare services and welfare professionals make beyond the managerial ‘Three E’s’ of economy, efficiency, and effectiveness.
Further, the findings confirm the importance of ‘helping relationships’ to the quality of service delivery in family support, despite the invisibility of service relationships in existing performance indicators. The complexity of worker-client bonds highlights the difficulty of evaluating social services using simple numerical counts of client or service episodes, and plays into broader debates about strategies for revaluing care work, and the role of care recipients.
Finally, the findings show the role performance measurement processes and methods might play in facilitating users’ struggles for recognition. Users identified a role for evaluation in making visible the contribution of family support in pursuing their social justice goals, and saw evaluation as an opportunity in itself to facilitate recognition and respect.
Overall, the thesis offers concrete evidence about how family support service users experience and define service quality and outcomes, and how they see their own role in evaluating the services they use. The research shows how users’ perspectives both contest and confirm the ‘new accountability’ of performance measurement, pointing to new directions, and further challenges, for conceptualising – and evaluating – social services.

This work is protected by Copyright. All rights reserved. Access to this work is provided for the purposes of personal research and study. Except where permitted under the Copyright Act 1968, this work must not be copied or communicated to others without the express permission of the copyright owner. Use the persistent URI in this record to enable others to access this work.