You are here

Collecting and Analyzing Outcomes Data

The basic procedures and systems programs use to collect and analyze outcomes data about their client services can be relatively simple conceptually. Once a program identifies the specific types of data it wants to use (e.g., financial and non-financial benefits provided clients in a case), it must implement systems with the functionalities to:

Enable case handlers to easily enter these data into their program’s case management systems (CMS's) at the time they close a case

Aggregate these data and generate a range of reports that present data in formats that enable the program to effectively analyze and present data for a variety of purposes (e.g., assessing success in accomplishing mission and goals, developing strategies and allocating resources, and fundraising).

Limited service cases make up approximately three-quarters of the cases that Legal Services Corporation (LSC) grantees close in a given year. Such cases are closed with Case Service Report (CSR) Handbook closure codes A (“counsel and advice”) and B (“limited action”).

Despite their prevalence, little is currently known about the outcomes of these types of cases. In extended service cases, grantees usually know what the outcome of the case is because they are involved in the case and have contact with the client for an extended period of time. By contrast, in limited service cases, the interaction with clients is, by definition, limited, involving the provision of legal advice or taking some limited action on behalf of a client in a single interaction. Thus, grantees do not have the opportunity to learn about what happens with the client’s case.

LSC is committed to helping grantees think through ways they can address the challenge of collecting data on the outcomes of limited service cases. This document provides a high-level discussion of some potential methods for collecting outcomes data based on a scan of strategies being employed in the legal services community and in other industries.

Four methods are discussed:

Follow-up phone calls

Text message surveys

Online surveys

Tracking administrative records

Procedures and Data Management Systems for Collecting Outcomes Data

Several steps are required for programs to effectively and efficiently collect outcomes data.

Programs must first identify the types of outcomes data they want to use and therefore collect.

Case handlers must identify and record in the case file the results of individual extended services cases when they close the case. For example, the case handlers identify whether the client obtained a civil protection order, avoided eviction, obtained (or retained) SSI benefits, and so on. Since this is considered a case handling “best practice,” many programs require their case handlers to document this information in the case file, even if the program does not use a systematic outcomes data system. Learn more about the types of outcome measures used by many legal services programs.

Programs typically collect these data only for extended services cases (LSC CSR case closing codes F-L), not for limited services cases (LSC CSR case closing codes A and B). It is relatively easy and inexpensive to capture outcomes data for extended services cases because case handlers know when they close a case, what the outcome(s) of a particular case is or are and can record this information on simple forms or enter it into specified fields in the CMS. In contrast, it is costly to identify the results of limited services cases—for example, follow-up services or similar means are required to obtain this information—and the ultimate outcomes of these cases may be determined by factors unrelated to the program’s representation (e.g., to what extent and in what ways did the client act on the advice provided by the program).

BRLS case handlers do not directly enter case outcomes data into the CMS. Instead, they complete a case closing memorandum which capturesthis information; the information is transferred into the CMS later. See the case closing memorandum form that identifies the outcomes data BRLS case handlers document at case closing.

Capture that data in fields related to other CMS fields, so that the outcomes can be linked to type of case, level of service, and other key fields.

Provide the efficient transfer of these data into the system(s) that will produce outcomes data reports.

Some programs use—or adapt—their vendor-provided case management system to collect and store outcomes data. The vendors of the different CMSs used by the large majority of LSC grantees (e.g., LegalServer, Kemps Case Works, Legal Files, Pika) have informed LSC that their systems have the necessary functionalities for collecting outcomes data or that these capacities can be incorporated into systems. Contact the vendor of your program’s CMS for more information.

Systems for Developing Analytical Reports with Outcomes Data

Once they have the CMS's—or other database systems—for collecting outcomes data, programs need to develop and implement system(s) with the capacity for aggregating these data and generating the types of outcomes data reports that will meet their particular needs.

Programs do not need sophisticated statistical and qualitative data analysis packages to generate useful and effective outcomes data reports. (Click here and here for information about some of those packages.) Instead, they can use database systems that are already used by many legal services programs.

For example, BRLS uses Excel to generate a variety of outcomes data reports, while CLAS uses Crystal Reports to compile data and generate a variety of reports with outcomes data. Learn more about reports that programs generate to analyze the results of their client services on a variety of dimensions.