Beginning in 1994, the Evaluation of Family Preservation and Reunification
Programs was undertaken by Westat, the Chapin Hall Center for Children at
the University of Chicago, and James Bell Associates under a contract to
the Office of the Assistant Secretary for Planning and Evaluation, U.S.
Department of Health and Human Services (DHHS). The evaluation's purpose
was to determine the extent to which family preservation and reunification
programs are effective in meeting the basic needs of children, in promoting
improved family functioning, and in reducing unnecessary foster care placements.
The study was to explore the extent to which these programs have varying
degrees of success with different target populations and to compare the costs
of treatment cases that received family preservation services to the costs
of control cases that received "regular" child welfare or other closely related
services.

The cost study design was developed as part of the original evaluation plan
in 1995. However, analyses of survey and administrative data gathered through
the evaluation found there were no significant differences between the treatment
and control groups on foster care placement and other outcome measures of
interest. Because of the evaluation findings, a consensus among advisory
panel members was that the field would benefit more from a report on how
a cost study can be conducted in the child welfare field than pursuing a
comparison of costs of services for treatment and control cases in this study.

This document presents our refined methodology for conducting a cost study
of family preservation services based on the lessons we learned from the
four study sites for which plans were developed. The methodology is designed
for this specific evaluation; however, it can be adapted for other research
studies requiring cost analyses or for ongoing budgetary and cost analyses
efforts in child welfare programs.

The goals of this report are to provide a description of what may be encountered
in the child welfare cost data environment and to develop a methodology that
includes a comprehensive framework for developing and obtaining service units
and unit costs in order to assess the cost of services to families involved
in the child welfare system. Although the cost methodology was not applied
in this evaluation, within a larger child welfare context, the lessons learned
through this work seem more relevant than ever as efforts to hold social
service programs accountable are gaining popularity.

The field has acknowledged the need for a cost methodology that can quantify
more accurately both the levels of effort and costs expended in the provision
of services. The most recent laws passed and initiatives developed in the
administration of child welfare focus heavily on accountability and cost
effectiveness. These are briefly described in the next section to provide
a background for why new methods for tracking and monitoring the true costs
of service delivery are now so critical.

In the past decade, child welfare programs all across the country have placed
increased emphasis on permanency, safety, and well-being for children and
families involved in the child welfare system. Both state and federal laws
were enacted that mandate timeframes for permanency decisions for children
in foster care.

The Adoption and Safe Families Act (ASFA) included a requirement for DHHS,
in consultation with states and experts in the field, to identify a national
set of outcome measures that can be used to gauge state and national progress
in reaching the goals of child safety, permanency, and well-being for child
and family. DHHS finalized rules by which state child welfare programs can
be monitored by the established outcome measures. The federal review process
has been updated so that DHHS conducts state-by-state reviews for rating
the performance of states in operating child protection and child welfare
programs.(1)

In addition to the legislation emphasizing outcomes, the federal government
has approved waivers of Title IV-E funding limitations in several states.
The waivers allow IV-E funding of service programs directed at permanency
for children beyond those directly related to a foster care placement. The
waivers require evaluations that address success of programs and cost
containment. It is hoped that the waiver demonstration projects achieve better
outcomes for children with the same or less money. If cost savings are realized
then the savings can be reinvested in other child welfare activities.

In concert with the focus on outcomes and results, federal and state governments
continue to seek viable mechanisms by which performance measurement based
on client outcomes can be tied to cost effectiveness. The goal is to develop
a cost-effective service delivery system that produces the best possible
outcomes for children and families. In the current legislative environment,
incentives in the form of bonuses are given to states with improved performance
in the number of finalized adoptions, child support collections, and TANF
successes.

In addition to federal bonus incentives, new federal regulations call for
the design of performance-based budgeting systems. Some states have already
implemented performance-based budgeting in the planning, developing, and
monitoring of state budgets. Performance-based budgeting ties performance-outcome
measurement to the funding of programs. This budgetary process first requires
the development of a strategic plan that specifies the goals and objectives
of service. In a service agency, the goals and objectives should relate directly
to successful outcomes for its clientele. Services are defined and developed
that are believed to contribute to obtaining those goals and objectives as
measured through outcomes. Outcome measures must be quantifiable. The defined
client outcomes should be obtainable and reasonable considering available
resources.

An agency then develops strategies by which it can deliver the service
successfully within its resources and produce good outcomes for its clients.
As in any business providing a service, the characteristics of the clients
receiving the services are analyzed, the cost of providing services is tracked,
and client outcome data are gathered and analyzed.

A complete description of performance-based budgeting requirements and methods
is beyond the scope of this report; however, the capability to perform accurate
cost analyses of the services provided and an accounting of to whom they
were provided is one of its
requirements.(2)

Accurate cost analyses require the measurement of how much service each client
received (the number of units of service per client) and applying unit costs
to obtain accurate service costs that are based on the level of effort expended
per client. True service cost cannot be computed for a client until service
components (units of service) are discreetly identified and tracked, and
the price of each component (unit cost or unit rate) is determined and assessed.
Without cost analysis capability based on the units of service completed
by individual clients, the performance measurement or the cost effectiveness
of delivering services based on the outcomes of a varied and complex client
population cannot be accurately determined.

Increasingly, state and local child welfare systems are attempting to measure
progress toward specified outcomes. Beyond this, members of state legislatures,
facing tightening budgets, are demanding to know the success of the services
they are financing. Similarly, program managers undertaking strategic planning
efforts require information on the relative costs of services and programs
and the tradeoffs involved with financing alternative approaches. In short,
the demand for cost data and analysis is growing; yet, those seeking this
information quickly run up against a system that remains largely impervious
to their efforts. Other than foster care maintenance payments, financial
information is not consistently maintained on either a case-specific or service
unit-specific basis.(3) As a consequence,
little information is available on effective approaches for measuring cost
within a child welfare services environment.

Though conceptually simple and widely applied in some fields of health and
human service, case-level cost analyses are extremely rare in the child welfare
field. This is because historically, child welfare agencies have not been
required to manage their systems from the perspective of financial
accountability. In addition (and perhaps as a consequence), only a limited
number of child welfare service costs are claimed or tied directly to a specific
child or family. Often, even if the payment for service can be tied to a
child or family, the level of effort expended in terms of time and activities
performed in the delivery of the service is not captured.

A unit rate structure, if designed correctly, is the best method for compiling
accurate service costs. The tracking of the amount of service each client
receives in terms of the number of units of service provided allows for a
good estimation of the level of effort expended per client. The cost of service
is calculated by multiplying the unit rate times the number of units of service
that were received. Some child welfare services provided through contractual
agreements are billed to the referring agency using a unit rate structure.
Foster care board payments are a good example of this type of contractual
agreement. The unit of service is a day of board and room for a child. The
facility or type of facility has a contractual rate per day based on the
services available at the facility.

Often, however, agencies will use a simple administrative contract, similar
to a block grant agreement, that does not require the accounting of service
delivery to a specific client. The provider of service is given a lump sum
amount to serve either a set number of families or to provide the service
until funds are expended. Bills are submitted to the referring agency but
do not include a record of the amount of service provided or the identification
of the clients who received the services. The results of the services delivered
are almost never reported.

In addition, child welfare agencies refer clients to other state and locally
funded service agencies for mental health, medical, food, and housing services
and often do not ask for verification and documentation of service receipt.
Data retrieval is also complicated by the fact that needed information resides
in more than one agency's records and case files (e.g., child welfare, social
service, mental health, etc.), and it is normal for each agency to maintain
its own separate payment system.

Furthermore, child welfare caseworkers do not record their own time 
a major cost element  by case or discrete activity type. For many
services, agency records do not show if the service delivered to a family
involved 10 minutes or 10 hours of child welfare caseworker time. In the
sites studied, case documentation did not include agency staff level of effort.

Due to these complexities, most prior cost analysis in this field compared
only average costs; for example, the average cost of providing family
preservation with the average costs of providing foster care. These two services
are among the few in child welfare where cost and service delivery come close
to being well documented for a family and child. But even for these services,
there are problems in calculating complete service costs and the level of
effort per client.

Family preservation services are generally charged to the referring agency
using a fixed rate per family using an average treatment period (e.g., six
weeks or six months). A family of three is charged the same as a family of
six. The actual number of family members who get service and the amount of
service hours vary depending on the cooperation of family members. These
variations are not tracked in the accounting system and may or may not be
documented in the agency's case records.

The cost of foster care room and board as stated above is tracked using a
unit rate structure in most states. The child welfare agency is charged for
the cost of care using a daily rate based on the level of services provided
by the facility providing care to the child. The type of placement and level
of care provided range from a foster family home with room and board only
to intensive inpatient residential treatment centers that provide therapy,
mental health, and medical services as well as room and board. However, there
are other undocumented costs in foster care. A child is placed in a facility
based on the level of care needed by the child at the time of placement.
Over time, the child's needs may change. The child may need more services
or may not need the services available at the facility. Reassessment of the
child's needs is an ongoing effort in order to match the child to the best
placement and rate. Some of the undocumented costs are the worker's time
in assessing, arranging, and monitoring the placement; developing foster
homes; and court services.

However, to provide a more complete accounting of cost, it is necessary to
expand on existing cost analyses and improve accuracy by including a more
comprehensive array of services and casework activities into the cost framework.
Accompanying the framework, a cost-analysis plan should provide methodologies
and systems for capturing not only service costs by case but also the level
of effort expended per case in delivering the service. One method, defining
a service by its discreet activities and developing units of service and
unit rates will be discussed further in the next section.

The following section provides a cost methodology based on the lessons learned
at the four study sites and applying cost methodologies from previous cost
studies. The primary focus of the study was to assess the impact of family
preservation in four sites and reunification services in one site. The family
preservation study involved randomly assigning children to treatment and
control groups in four sites: Kentucky, New Jersey, Philadelphia, and
Tennessee.(4)

In conjunction with survey and administrative data collection, between December
1996 and January 2000, the study team completed a series of telephone interviews
and site visits to each of the four family preservation study sites. Information
from this work was used to develop site-specific cost data collection plans
that both recorded information availability and documented site-specific
factors that would have to be taken into account.

One of the greatest challenges was that at two sites, both administrative
and financial payment systems changed over the course of the study. The
descriptions of each of the four study site's cost data, data sources, and
other site-specific issues are discussed more fully in Appendices A-D.

A variety of cost analyses models are available that have been developed
for human services. The methodology used for this report was developed under
consultation with Ernst and Young LLP. The methodology relies heavily on
an ingredients-based approach, formally known as the Resource Cost Model,
originally developed by Hartman (1979) and Chambers and Parrish (1983). Other
sources for this cost study methodology are from Harkavy and Bond, (1992),
Wolff and Helminiak, (1993), and Olds, Henderson, Phelps, Kitzman, and Hanks
(1993).

The methodology requires a list of the ingredients and the unit cost of each
ingredient. The ingredients that make up costs in child welfare are service
components or units of service. A quantity of an activity performed in providing
a service defines the unit of service. For example, an hour of counseling,
a trip to and from treatment, or a day in foster care is considered a unit
of service. A rate or unit cost is established for each component. In the
best circumstance, the costs of all service components can be gathered and
computed for an individual client.

The sum of those costs produces an estimate of the total cost of serving
that client. The advantage of calculating the costs of services by tracking
the number of units of service delivered to each client and applying the
appropriate unit rate is that the costs are easily summed by type of service
or for any particular group of clients included in the cost analysis. Summing
the costs of all the service components that apply to that service can produce
the total cost of a service, such as family preservation or foster care.
Additionally, the tracking of the units of service provided to individual
clients allows for the estimation of the level of effort required to provide
services to special-interest client groups (e.g., child welfare clients who
are substance abusers) when these characteristics of the individual clients
are also tracked.

This methodology is organized into six suggested steps that include designing
a data collection framework; defining targeted services by service components
called units of service; determining how service activity is recorded and
tracked; identifying cost data sources such as accounting systems, classification
and salary schedules, and unit rate tables; establishing data retrieval methods;
and lastly performing cost data analysis.

Develop a cost data collection framework that contains all relevant services
of the client population of interest and provides the formulary for calculating
the cost of providing those services. For clarity, the relevant services
are called the targeted services.

Define the targeted services by identifying each of the activities involved
in delivering the service. This includes both contracted and directly provided
activities. Define the unit of service for each activity identified. When
performing cost analyses at multiple sites, even within the same state, it
is important to define services at each site to reveal any differences in
terminology or in the actual activities performed in service delivery. Service
definitions based on terminology may appear to be the same but often are
not. The activities performed in the provision of a service may vary from
site to site.

Determine how the targeted services are recorded, tracked, and funded, and
how payment for these services is made. Include the following:

Determine how services are recorded and tracked by the agency responsible
for completion of service delivery and client outcomes.

Determine all possible funding streams for each of the services, how the
funds are applied to the payment for services, and which agency provides
the services and manages the funds.

Determine how payments for services are recorded and how systems are structured
(on a case-specific basis, using a unit rate structure, unit cost measure,
cost reimbursement, flat fee for services, administrative block grant contracts,
or budget line item).

Determine whether plans are being made to change the systems for recording
data about clients, services, costs, and payments over the course of the
cost study. If there are plans for new systems or system upgrades, plan and
take the necessary steps to ensure data integrity from one system to another.

Review all existing cost data sources and supplemental documentation:

Financial systems used in tracking payments for service and salaries for
agency staff.

Contract administration systems.

Administrative data systems.

Rate setting methodologies, structures, and schedules.

Voucher and invoice paper trails.

Methodologies for budgeting and cost allocation.

Agency policies regarding service delivery, both those provided directly
by the agency and those provided through contractual agreements with another
agency.

Develop data retrieval methods for each targeted service, including estimation
procedures for cost information unavailable on a unit of service, unit cost
measure, and/or case-specific basis. If cost analysis is being conducted
in conjunction with an evaluation of service outcomes, make sure the cost
data records are maintained along with the collection of client and service
tracking data so that they cover the same time period and are assessed based
on the unit rates, salaries, and costs in effect during the course of the
study.

Once data become available and are being collected, the cost analysis can
be conducted and it is important to get feedback on the findings from agency
officials for quality-control purposes.

Each of these steps requires considerable and ongoing input from the study
site. For this reason, if data are being collected from more than one site
within a state, it is recommended that most of the steps be repeated at each
site to verify the cost data elements and data retrieval methods. This exposes
any unexpected differences in policies and procedures that affect data collection
and analysis. Moreover, it is advisable to complete steps 1 - 5 prior to
initiating the collection of administrative data. This is because, depending
on the quality of existing data sources, it may be necessary to implement
new data collection forms for the study population.

Be aware of any existing plans for changes to service reporting and financial
systems at each site. Map out alternative plans of action for data collection
in the event these systems do change during the study timeframe. It is necessary
to map old systems to new ones so that data remain consistent throughout
the study. This section addresses in detail each of the six steps of the
cost methodology.

In order to design a cost data collection framework, it is necessary to define
clearly the research questions to be addressed. A cost study can be used
for several purposes. For instance, it might be conducted for the purpose
of:

Comparing the costs of serving different populations (example: families that
have different presenting problems such as substance abuse versus
emotional/behavioral problems);

Assessing the cost of implementing a new service or policy;

Examining the cost effectiveness of an entire service delivery system based
on client outcomes; or for,

Developing a performance-based budgeting and management system.

Clearly defining the understanding to be developed and selecting those services
and activities believed to affect the analysis are necessary to refine the
data elements to be included in the framework. For example, if cost effectiveness
based on outcomes is the goal of the cost analysis, then the relevant services
included in the study are those that are most likely to influence
the outcomes.

Table 1 lists typical child welfare services and
functions that were included in the cost study data collection plan designed
for the Evaluation of Family Preservation and Reunification Services. The
framework design was based on three expectations: 1) foster care and family
preservation services account for the largest percentage of the cost for
each case; 2) caseworker labor is a significant component of cost and accounts
for differences in the average cost per case; 3) several other key services,
other than those associated with family preservation and foster care, are
likely to contribute significantly to the average cost per case.

Though applied somewhat differently across sites, three major classes of
cost are used to account for public expenditures on child welfare. These
are child welfare services provided and paid for by the public child welfare
agency, services provided and paid for by an external agency, and indirect
costs incurred by the public child welfare agency that support the general
operation of the agency such as fringe benefits, administration, and other
overhead costs.

The framework and all decisions regarding cost inclusion and exclusion criteria
must be consistent with the research questions that define what it is that
is to be learned. Due to variations in both the structure of child welfare
systems and the availability of community services, it is necessary to review
the list of targeted services with site staff to determine service availability
and the service delivery system. For example, while all four of our study
sites used contracted family preservation services, other states deliver
this service directly through public child welfare agency staff. In addition,
some sites that use contracted family preservation services will have designated
family preservation contract monitors where others might use ongoing protective
workers to monitor family preservation cases. In either situation, both sets
of costs (contract and agency personnel) need to be included in the cost
analysis.

Whether the objective of the cost study is to answer a research question
involving a comparison of cost between a treatment and a control group, to
evaluate cost effectiveness, or simply to determine the cost of service delivery,
it is essential to record all service costs consistently. If a service is
identified as relevant, it should be included for all cases even if multiple
sources must be queried to determine cost. For instance, if psychological
assessments are selected as a cost component, all incidents of this service
should be included, regardless of whether the assessment is paid for by the
child welfare agency, Medicaid, or through private insurance. To collect
this information on some cases but not others will bias the information
collected. Again, the analysis should be an objective account of all service
expenditures included in the cost framework.

There must be a consistent internal logic that drives all decisions about
the breadth of the cost components. The public child welfare agency, an external
agency, or a combination of the public child welfare agency and external
agencies may incur the service costs included in the framework provided in
this document. A decision was made in the cost study design for the Evaluation
of Family Preservation and Reunification Programs to limit external services
to those that are directly related to child welfare outcomes or that duplicate
services provided by the public child welfare agency. Excluded are physical
health care, income maintenance, and other services and assistance for at-risk
families and children that fall outside the public child welfare agency domain.

Following discussions with site staff on the structure of the child welfare
agency and data availability, it will be necessary to refine the framework.
Final decisions about the framework will also be influenced by three factors:

The extent to which a given service is used and will affect study results,

The resources available to the study team to support the data collection
effort, and

The ease with which sites can provide or generate the targeted data.

The foremost consideration should be the extent to which families in the
study population use the service. If a majority of the study population is
receiving a service  especially if those clients are highly skewed
to the treatment or control groups  inclusion of this service
cost is essential. This might include alternative home-based services available
to the control group as an alternative to family preservation.

A second factor, the amount of resources available to the study team, may
also affect data collection efforts. For example, some of the data needed
to estimate service costs might have to be collected manually from case records.
Needless to say, this is a labor-intensive activity and a significant cost
to the study team. The issue that needs to be addressed when resources are
scarce is whether to cut the number of sites or the scope of the study. If
it is necessary to reduce the number of cost variables due to prohibitive
cost of data collection, it is important to do this without introducing bias.

One way to reduce the cost of data collection in a comparative study is to
test whether certain services have similar costs for each group of clients.
Service variables that are found to have similar costs regardless of the
client group or client characteristics can be eliminated from the cost analysis.

Third, due to variations in financial accounting systems, producing case-level
and unit-based cost information will be easier for some sites than it will
be for others. For example, among our study sites, Tennessee's child welfare
agency operated multiple payment systems, each of which would be used to
provide a portion of the necessary data. On the other hand, Philadelphia
operated a single payment system that was already tied to its administrative
database and was able to give us case-specific data (in certain areas) with
relative ease.

In all cost studies, decisions need to be made about how to use available
resources to produce accurate results. The objective of the cost study can
help in these decisions; however, accurate results to meet the objective
should not be jeopardized by the limitations in scope or the number of sites.

Finally, a decision must be made about the time period to cover for cost
data collection. In comparative cost analysis between treatment and control
groups, the cost data collection period should mirror the time period for
collecting service and outcome data. If services and outcomes are tracked
for 12 months from case selection, then cost data are needed for the same
period in order to capture the correct costs paid during that time for the
services provided to clients in the study.

It is also important to capture effective dates of all applicable rates during
the period covered. In child welfare, as in other fields, payment rates fluctuate
over time, even if based only on inflationary rate increases.

In tracking cost of service delivery and in performance-based budgeting,
the period covered by cost data collection is often dependent on the budget
cycles of the state or federal governments, and these are normally based
on fiscal year. States will often require reporting on performance and cost
measures for monitoring the budget on a monthly, quarterly, and yearly basis.

Learning how a particular state or site defines its services is a necessary
step in analyzing cost. Often similar terminology is used in different sites,
but the actual activity components of a service are delivered in different
ways and can contribute to a higher or lower cost of service. This becomes
important in determining what service activity components are included in
staff salaries, in the agency's budget by object codes, or in a rate, fee,
or payment for contracted services. As an example, foster care services in
one site may include only foster family home and foster group home placements
and not residential and institutional care. Other sites may include all
out-of-home placement activity, including institutional care, in its definition
of foster care services but may also have some placements in institutional
care paid for by Medicaid. Foster home development, monitoring, and licensing
may be included in the per diem rate by type of facility. For other sites,
this function may be administered by a separate agency or by a group of
specialized workers with separate funding. The level of care determination
and the job of finding the most appropriate home for a child may be handled
through a managed care contract and, therefore, be excluded from a public
agency's worker activity.

Costs for direct service worker activity are almost always captured as an
average cost per client based on an aggregate of workers' total cost to the
agency that includes salary, time, benefits, and supporting staff. Knowing
whether the cost of service is recorded for an individual client at a unit
of service level or only as an average cost per client is critical in comparative
cost studies of special-needs client populations or of treatment and control
groups. In this cost study, as in most child welfare cost analyses, there
would be a mix of average costs per client and those costs that can be obtained
at the individual client level.

Knowing what is actually performed in the delivery of a service is also important
when developing a list of service activities that are reported in direct
service workload time studies or if case-specific service time logs are used
in the study. The intake process for handling calls reporting abuse and neglect
is an example of the need to define activities by discrete categories. Some
sites include all phone calls made to the protective service agency, not
distinguishing information and referral calls from calls about a possible
abuse and/or neglect incident. Even in an investigation, the activities involved
may become indistinguishable from in-home service provision. Many states
allow and encourage investigation workers to initiate in-home and support
services to the family as quickly as possible to avoid placement.

Variations in the definitions of terminology used in child welfare can often
create misconceptions about the data stored in the administrative systems
as well. For instance, a case opening for service was defined and recorded
in the case management systems differently at all four of the study sites.
It is extremely beneficial to ask for systems' documentation. Services may
be defined in policy a certain way, but the automated system may process
the data using a different definition. For example, in Kentucky a case was
opened when an abuse/neglect incident was accepted for investigation and
the opening date for service on the system was the report date. However,
New Jersey used a multi-tracking system that allowed cases to be opened for
service due to family problems or due to abuse and neglect after the
investigation. The date opened for ongoing services was discrete from that
for the investigation services.

Defining a service by the activities performed by each study site in the
provision of that service becomes important in the interpretation of the
data and assessing the cost of services. Since child welfare cases have inherent
variations in the amount and duration of even the commonly defined services,
it is crucial in a comparative cost analysis that the service definitions
be clear.

After the cost study services are chosen and defined, it must be determined
how the agency records the data necessary to track clients, services, service
costs, and outcomes. An important step is to learn and understand the systems
the agency uses, either automated or manual, for record keeping and where
data on client demographics and characteristics, service provision, and service
costs are collected and stored. Along with learning the record-keeping
procedures, it is just as important in a cost study to determine how the
agency makes payments for services, what mechanisms and systems are used
to record payments to external providers and salaries to agency staff.

Understanding how services are funded facilitates efforts to retrieve cost
information. For instance, several services, such as substance abuse treatment
and mental health services are usually delivered and/or funded by outside
agencies. These should be considered for inclusion into the cost analysis
because they directly affect child welfare outcomes. However, substance abuse
services can be funded by the child welfare agency, through programs delivered
by state health or public health departments, Medicaid, private insurance,
or through independent grants to community providers. Obviously, data collection
for this type of service presents certain challenges that should be considered
carefully and resolved as early as possible. This may involve developing
cost-estimation procedures that can be applied to hard-to-get data. Cost-
estimation procedures for externally delivered services are discussed more
fully in section E-3 below.

Due to the fact that a significant amount of cost information may reside
outside of the child welfare system, it is necessary to determine the extent
to which other agencies will share cost information with the study team.
Getting buy-in at the early stages of study design is critical to conducting
the cost analysis. For example, New Jersey operates an automated accounting
system, called the "child ledger" that tracks a wide range of child welfare
services and ties the expenditures directly to individual children. The state's
Department of Human Services (DHS), the umbrella agency that includes the
child welfare agency, administers the child ledger system. The child ledger
is not an online system. It is the product of matches between multiple payment
systems and the state's administrative service information system. Child
welfare agency staff receives printed reports each month from the system
to verify the accuracy of claims. Our child welfare contacts recommended
that we extract cost information on study families from the child ledger
printouts because they did not have access to the automated matches. Data
collection from the monthly child ledger reports is labor intensive.

Had the cost study actually been conducted, it might have been possible that
through up-front pre-study arrangements, a cooperative plan between DHS and
the Department of Youth and Family Services (DYFS) could have been established
whereby DHS could provide the automated payment match files to DYFS routinely.
This arrangement could have been made to last only for the study period or
on a permanent basis beneficial to both agencies in tracking expenditures
and tying service delivery to client outcomes.

Once the range of services has been selected and the source of cost data
identified, it is necessary to understand how the local payment systems are
structured. States use a variety of financial systems to track service contracts
and other child welfare expenditures. From our experience, there are two
basic types of automated systems used for tracking contracted services that
can supply valuable information for a cost study:

Contract Information Systems that track all contracts in a jurisdiction
or state, and contain basic provider information as well as total contract
amounts. For each contract, service unit costs may be available or can be
calculated (e.g., the cost of a psychological assessment or a day in child
day care).

Automated payment systems are the ideal source of cost data because they
track case-level expenditures by the number of units of service, the unit
rate, and the dates of service. This gives the study team actual case-level
data with the quantity of service and the amount spent and, ideally, a data
tape from which costs can be sorted and analyzed without having to construct
cost estimates as well as a new database to store the estimated data.

In addition to these two automated financial systems, states use a variety
of other accounting systems, including manual records or ledgers that can
be used to gather case-specific data. For example, Kentucky tracks a variety
of contract costs using case-specific vouchers and invoices. In Philadelphia,
records for the use of flexible funds are maintained manually on a ledger
that identifies case-specific expenditures. Similarly, New Jersey district
offices use local bank accounts from which they can draw down flexible funds,
and case-specific records are maintained on all expenditures.

In order to gather cost data from any of the financial systems, it is necessary
to understand the identifying information used by agencies. Generally, sites
employ one of the following: case name, a combination of case name and birth
date, the child welfare case number, or the child or parent's Social Security
number. It is important to understand the identifying information needed
by the finance office in order to retrieve the identifiers during the collection
of administrative data.

Child welfare agency personnel budgets are generally maintained by line items
that represent the worker's service area or unit (e.g., investigation, foster
care). Budgets for each service area  including costs for front
line, supervisory, and support staff  are generally based on caseload
projections multiplied by a weighted service unit cost based on the unit's
staffing structure. The level of detail maintained on agency personnel costs
is dependent on the number of staffing functions used by the finance office.
During the study period, none of the sites had a concrete methodology for
constructing case-specific expenditures of public agency workers. Kentucky
for many years depended on an elaborate client-specific service activity
reporting system that was tied to the state's cost allocation formula. At
the time of the study, this system was no longer being used or maintained.
The old social services systems were being dismantled and replaced with the
newer case management systems.

Depending on the period of interest to the study, it may also be necessary
to learn about the availability of historic data and how cost information
is archived. At a minimum, it is important to understand what data are archived
and how they are maintained  on an automated file or in hard
copy  and the complexity of accessing the information.

Finally, it is also important to determine if the site plans to modify (or
fully replace) any of its financial or administrative systems during the
course of the study. This occurred in both Tennessee and Kentucky and clearly
has several potential ramifications ranging from the use of new case identifiers
to changes in the types of services claimed and the claiming system itself.
In order to analyze outcomes using administrative data in Kentucky, data
elements from the old and the new systems were mapped to each other to maintain
consistent definitions of services, and service dates were cross-referenced
for an accurate account of the service history for each client. This was
no small task, and, therefore, changes in data collection and payment systems
during a study period can affect the accuracy of the analysis as well as
increase the cost of the study.

In Kentucky, one aspect of the new systems should prove beneficial for later
cost studies. That aspect is the tying of the new case management system
to the financial payment systems. Whenever the administrative case management
system is used as the source for the record of service by the financial payment
system, reporting of the provision and cost of services is likely to be more
accurate. This is explained in more detail in the next section.

Once the targeted services have been clearly defined and service recording,
funding, and payment systems are identified, it is time to review the data
sources. This step is important in the understanding of not only what data
can be retrieved, but also their accuracy. There may be many possible sources
but only a few reliable ones. Additionally, completeness of record is important
to any cost analysis.

When case management systems are tied to payment systems, the data are generally
more complete and accurate. This is true of foster care payment systems in
many states where workers record each placement in the case management system,
and these records are used by the payment system to create invoices for payment
to a facility or agency by child. These systems are edited against each other
daily, weekly, or monthly depending on the billing cycle. Kentucky's new
MARS builds its placement records for payment from the case management system,
TWIST; Kentucky's SACWIS is described in more detail in Appendix D. This
constant editing and the need to get accurate payments to foster parents
make them more accurate and complete than systems that are not used for paying
bills.

The new State Automated Child Welfare Information System (SACWIS) being developed
in many states are designed to be useful for workers to manage their cases
in an automated form, thereby making the record keeping less cumbersome.
The verdict on the success of this endeavor is still not in. Under this premise,
these systems should prove more complete in tracking services to clients
by activity components. The older child welfare administrative data systems,
while able to provide the total number of clients served in global service
categories such as investigation, in-home, and foster care, did not provide
the necessary discrete service activities required for unit costs analysis.

In all of the study sites, records on cost and services were available for
review in financial payment, administrative data, and contract administration
systems. Other records were available on paper reports such as ledgers, invoice
and voucher files, case records, and contract files. Asking questions of
knowledgeable staff is critical to the understanding of the data. Data sources
will either complement, overlap, or duplicate each other. In some sources
that duplicate or overlap, the data may not match record for record. It is
important to find out why they do not match. The aim is to find the most
complete and accurate source of needed data on clients, services, and cost.

Other types of records that are important to review in a cost analysis are
rate setting methods, structures, and schedules; cost allocation and budgetary
formulation; policies and procedures regarding contractual agreements; staff
salary levels and functions; and workload analysis and time studies already
performed by the site. The information provided by these records is helpful
in understanding the data that are eventually retrieved and used in the cost
study.

Once all possible sources have been reviewed for completeness and accuracy,
a determination of what sources to use in data retrieval for the cost analysis
is made. A helpful tool is a data matrix listing all the cost data sources
by the services being studied. The data matrix can include details about
the source such as accuracy, completeness, and utilization dates (e.g., the
period of operational use of a form or system by a site). It can include
important information about how the cost data are stored, in hard copy or
on an automated data file. For each source, the unit of recorded cost can
be defined by whether the data are recorded by units of service per client,
by client, by case, by staff person, by units of service by contract, or
by contract amount. A data matrix was prepared for each of the study sites,
and these can be found in the descriptions of available cost data in Appendices
A - D.

Using the data collection framework as well as information on the agency
financial and contracting systems, it is then necessary to review each targeted
service with the site to develop a cost-retrieval system. As described above,
the child welfare agency will probably be able to produce either case-level
expenditures or unit cost information on its own contracted services. Some
targeted services may have more than one source of funding (e.g., through
contract with the child welfare agency, Medicaid, other public agencies or
through community-based organizations). Consequently, all sources must be
identified and contacted to determine the best way to identify payments made
on behalf of study families.

Turning first to child welfare services contracted by the agency and other
externally provided services (categories 1-4 of the data collection framework
presented earlier in Table 1), we have listed a series of overriding questions
that should be reviewed with site financial and administrative personnel.

For each contracted service determine:

Is there an automated financial system that tracks the costs on a case-specific
basis?

If so, how can the information be accessed from this system?

For those service costs that cannot be collected on a case-specific basis:

Is service-unit cost available? If yes, how can the information be accessed?

What is the most thorough and accurate source of service utilization data
(e.g., automated case management systems, case records)? What procedures
need to be developed to access this information?

How labor intensive is each data collection option for the agency and/or
for the study team?

Is the site willing to implement a new data collection procedure for study
cases to track service utilization such as a service utilization tracking
form or periodic worker telephone interviews? (More on this option below.)

If not, is the site willing to collect this information on a random subset
of cases to lessen the data collection burden?

Do agency administrators have other recommendations about gathering and
estimating costs?

Which of the data collection options do site officials prefer? Why?

The following tables present the service categories from the original data
collection framework developed for the Evaluation of Family Preservation
and Reunification Programs (contracted family preservation, foster care,
other contracted services, directly delivered agency services, and indirect
costs). These tables present the relevant issues to review with agency personnel
for each service or cost component and may be adapted to other service models.

The following table summarizes issues to be explored with regard to contracted
family preservation services. If public agency staff directly provide family
preservation services, refer to the section "Directly Provided Services"
below that offers several options to estimate these costs.

Are there additional contract costs (e.g., payments for "booster" or follow-up
visits)? If so, how can those costs be retrieved?

For the Evaluation of Family Preservation and Reunification Programs, family
preservation was a contracted service at all four of the study sites. At
each site, service unit rates included both therapy and administrative costs.
The contracts were based on a set cost per slot, regardless of the actual
length or intensity of the intervention. The service unit costs for the
intervention were $2,600 in Tennessee, $4,170 in Kentucky, $4,300 in New
Jersey, and $4,500 in Philadelphia (1997 rates). In addition, some family
preservation models include the potential use of "booster" or follow-up
interventions. These interventions may be reimbursed at a lower rate, as
the average service duration is designed to be shorter.

None of the sites maintained family preservation expenditures on a client-level
payment system. While unit cost information was available from contract
management systems, service utilization information would need to be gathered
from the study's administrative database or from one of the sources discussed
in section 3 below.

Determine source of miscellaneous payments for concrete support services
such as clothing, school supplies, transportation, food, etc.

Determine if there are separate Independent Living payments.

Determine if there are separate administrative costs for foster home development
and licensing.

Determine if there are additional contract costs, and if so, data source
and retrieval methods.

As discussed above, case-level foster care maintenance payments are nearly
always available from child welfare agencies. However, it is important to
review the availability of cost information for each type of placement as
a state may use more than one payment system to track maintenance payments.
For example, Tennessee maintains case-level expenditure data on all types
of placements except state-operated group homes. These facilities do not
claim on a case-level basis and would require the matching of client-level
service utilization information with per diem maintenance rates.

States use a variety of standardized payment rates for each type of placement
and, in some cases, negotiate individual rates with group and residential
facilities. For instance, New Jersey has developed a payment matrix to reimburse
foster homes that correspond to both the child's age and special physical
and emotional needs, see Table 4 below.

These monthly payments do not include clothing or other special expenditures.
In addition, New Jersey does not use standardized rates for group homes or
residential or shelter care. Each provider's contract is negotiated separately.
For the purpose of the Evaluation of Family Preservation and Reunification
Programs, New Jersey estimated the average annual cost per child in each
type of placement (Table 5) including the congregate
placements by dividing its total annual expenditure by type of placement
by the total number of children served in the placement.

The foster care board rates for the Department of Children's Services (DCS)
in Tennessee is presented in Table 6 below. Each
placement is made with an Individual Child Placement Contract signed by the
child's case manager and the DCS-approved family foster home. The special
circumstances monthly board rates are available upon approval of the Residential
Case Management Team for the child with unique needs due to medical problems,
behavioral problems, or a child who requires extraordinary supervision or
requires extra care due to physical, emotional, or mental handicaps. Foster
homes that care for children who are moderately, severely, or profoundly
mentally retarded receives special rates beginning at $600. Additionally,
Tennessee has separate daily rates for foster group homes, residential
facilities, camps, and therapeutic and contract foster homes.

* Added to these rates is $25 per child per month for
the care of sibling groups of three or more in the same foster home. Sibling
groups include half-siblings and adoptive siblings.

Ideally, payment records for each child in the study can be reviewed to determine
the amount of money paid to each home or facility that cared for a child
during the study period. If these records are not available, it may be possible
to develop a reasonably good estimate of maintenance costs by using
administrative data on the child's placement history. In all four sites,
at least one automated system tracked placements by entry and exit dates
by type of facility. To develop child-level estimates, the number of days
in care by facility type could be calculated using the entry and exit dates
and then matched with agency rate schedules to calculate the cost for each
placement. Payment rate variations based on the child's age and special needs
will need to be factored in as well. It is also important to ask for the
agency's rate schedules for the appropriate periods in order to apply the
rate in effect at the time of each placement. If rates vary both over time
and by facility based on negotiations that factor in facility services or
the child's needs, as in New Jersey, average rates by type of facility and
needs addressed can be calculated over the time period of the study. The
average rate over time is not as exact as established standard rates, but
it provides a good estimation of the cost of foster care by type of care.

How service is funded (e.g., child welfare agency, community provider, Medicaid).

For each payment source, how each payment system is structured (client-level,
unit rate, flat-fee for service, cost reimbursement, block grant).

Data retrieval methods.

For service contracts that do not use case identifiers, best source of service
utilization information.

If necessary, the average service units and average service costs.

In general, the study team found that it was easier to locate and isolate
service-unit costs for contracted services than it was to determine client
utilization of those services. Where payment for contracted services was
made on a client-specific level through an automated financial management
system, services utilization and payments were more easily obtainable by
case. Problems arise because the payment mechanisms for the various contracted
services are not consistent and are generally contract or agency specific.
As an example, day care services in all four sites studied were tracked and
payment made from multiple systems depending on which state agency or funding
stream paid for the service. Protective day care was generally paid for by
the child welfare agency, but if a family was TANF eligible, the day care
would be paid by that separate agency using a different payment system. Another
example is mental health services, which most often are paid for by Medicaid
through the health services agency. However, for those not Title XIX eligible,
child welfare may pay using a different payment method. In Kentucky, substance
abuse treatment services for children are paid through the child welfare
system and are tracked through the automated financial payment system, on
a manual payment ledger, or through paper invoices. Substance abuse treatment
for adults is paid for by the mental health agency through the Medicaid system.
Even in the same agency, payment for services can be made and tracked differently
depending on the contract negotiated. As in the Kentucky example above, payment
for services can be tracked on an automated system, in a manual ledger, or
on invoices and vouchers depending on which contract provides the service.

In none of the sites were workers required to enter a comprehensive list
of services received from outside providers in either the automated case
management systems or case records. Workers may have noted a service referral,
but rarely was there a clear indication of service receipt. Sources for
determining client utilization, include one (or a combination) of the following:

Agency automated files,

Vouchers or invoices,

Agency case records,

Case-specific service logs, or

Contract agency service reports.

The strengths and weaknesses of each of these options are discussed below.

The most clear and concise source of contracted services information is the
automated financial payment systems that record service utilization and cost
at the client/case-specific level. When these systems were used for payment,
the client was identified; the service was identified; and the number of
units of service was recorded along with the service start and end dates,
the unit rate, and the total amount paid. Philadelphia's child welfare payment
system and the Kentucky STARS and MARS were examples of this type of payment
system. As previously discussed, however, these systems do not contain the
full complement of services provided to the case. Because of the incompleteness,
other sources have to be identified. Philadelphia's child welfare system
is very reliant on community referrals, and thus nearly all cost data for
support services must be retrieved from outside providers.

The first source of data to be reviewed for service utilization information
should be the child welfare agency's own automated case management system.
Most child welfare case management systems have a service screen where workers
can enter information on service referrals. It should be noted that at our
study sites, workers used these screens for service referral information,
that is, as a record of service recommendations. Other states may use similar
screens for information on service receipt information, and they might include
information on service dates. System administrators should be queried about
the purpose, consistent use, and accuracy of the data entered by workers
using the automated case management system. In addition, the data
comprehensiveness and validity should be reviewed, as front-line workers
may not be required to enter certain data fields.

Provided the automated case management systems are found to contain reliable
and useful information on case-level service utilization, there are various
ways of using the information to estimate service costs. Two alternative
methods are described below.

The first method would involve retrieving case-level cost data on each service
from the service-providing agencies listed by the worker. The major drawback
to this approach is that it will necessitate submitting client names and
approximate dates of service to a range of service providers who would then
supply the requested cost information. The process could be costly and time
consuming with additional concerns about client confidentiality.

An alternative approach would involve applying average or estimated service
costs to the service information contained in the automated case management
system. Average service costs would be based on information on actual service
duration (if available) or on average service duration collected from agency
contracting records, case worker interviews, and/or information gained from
the caseworker service logs collected on a sample of study families (discussed
below). This utilization data would be applied to service unit costs provided
by the child welfare agency and all participating community service providers.

If information on actual service duration is not available from case records
or automated files, estimates of average service duration can be developed.
This can be done by reviewing agency contracting records and generating an
average number of service units received per child or family. If there are
wide variations in service duration among cases, identifying client
characteristics that correlate to different lengths of service duration could
provide a more accurate average number of units of service per type of case.
For example, home visiting services may, on average, be delivered longer
to first-time mothers than to mothers with other children in the home. Again,
the use of information gathered from the caseworker service logs conducted
on a sample of families can be used to construct profiles of different groups.
This approach, however, should be thoroughly tested for accuracy and may
be less than accurate when applied to subgroups such as family preservation
since these type of cases are unlikely to be average.

To develop average unit cost information, it would be necessary to know the
range of typical expenditures made for a service and the proportion of payments
made at various rates. For instance, an hour of counseling may range from
$60 paid for by Medicaid to $100 paid for by the child welfare agency. If
80 percent of the study population is found to be Medicaid eligible, a reasonable
average cost for 1 hour of counseling is $68  ($60 X 80%) + ($100
X 20%) = $68. Unit rates for certain services are sometimes standardized
and those are readily available, but if they are not, a rate can often be
found in the signed contract for each provider. Therefore, if unit rates
vary by provider and the provider of the service is known, the rate normally
charged may be found in the contract administration files.

If neither agency-automated records nor agency case records provide a reliable
account of service utilization, then cost estimates become less reliable.
If this is the case, it may be possible to verify service utilization by
contacting study families or, by submitting information on service referrals
to the service-providing agencies for them to verify service receipt. Otherwise,
other sources of information on service utilization should be explored.

When voucher and invoice systems were used, they were often client specific
and provided a good accounting of the services provided to a client with
the cost of the service. The number of units of service provided is sometimes
available on the invoice. The drawback was that they are manually stored
by payment date, not service date, making it difficult to match back to a
referral recorded in the case folder. The client is often identified not
by a client number used in an automated case management system, but with
a Social Security number. Additionally, leafing through files of paper vouchers
and invoices would be very labor intensive. Computer-generated or manual
payment ledgers contain a good record of service utilization and cost; however,
these sources were also stored by payment month, not service month, and are
also very labor intensive to find data for a specific child or family member.

Case records contain handwritten contact sheets listing the worker's ongoing
interaction with the case, computer printouts from the agency's administrative
database, and several forms that provide demographic and case history
information. Information on service referrals is noted in a variety of forms
as well as in the case notes. Site contacts can help to limit the time involved
with case record reviews by identifying the most relevant forms in the records.
An example of such a form from New Jersey is available in Appendix E. Even
with this assistance, case-record reviews can still be labor intensive and
costly to undertake. (It took the evaluation study staff an average of 1
to 2 hours to review each record at two of the study sites.)

While labor intensive to review, case records were found to contain more
information about service receipt than the automated administrative data
files, including some supporting documentation from community agencies. However,
like the administrative data files, there was virtually no information on
the length of a service intervention or the number of service units used.
If case-record review becomes a serious option, researchers are strongly
encouraged to first review a sample of records to assess the extent to which
they can supply the needed information.

Estimating service costs from information contained in case records would
involve applying the same methods discussed under agency automated administrative
data files.

If existing documentation is found to be inadequate for estimation purposes,
then the best way to gather a comprehensive and accurate record of service
receipt would be to have caseworkers complete service logs that record the
full range of services received by study families. At a minimum, the following
information should be included on a data collection form: type and description
of contracted (or community-based and funded) service received, date(s) of
service provision, and length of intervention and/or number of service units
received. The forms would be placed in the family's case record upon selection
into the study. Because this would be an additional record-keeping requirement
for frontline staff, case records should be flagged and have explanatory
notes attached to assist with data collection. If the sample size is adequate,
the information could also be used to develop profiles of families that use
different services in different amounts.

Another source of service information might come from the service contractors
themselves. There may be instances where the public agency relies on a contract
provider to deliver intensive case management services (such as family
preservation services) and requires that the provider submit detailed reports
about service receipt. This was the case in Philadelphia. Two types of service
providers, family preservation workers and the city's less intensive in-home
case management providers (Services to Children in their Own Homes - SCOH)
are required to submit detailed service histories for all families served.
While the study team did not review case records in Philadelphia, the city's
Family Preservation Coordinator strongly urged the study team to use these
service reports as the primary source of service utilization data.

In Philadelphia, family preservation and SCOH provider reports supply a variety
of information including names and often addresses of service providers with
whom the client worked during the family preservation or SCOH intervention.
Family preservation reports also include provider telephone numbers and dates
of service receipt. While the reports do not supply any cost information
and would require follow-up contact with each agency, they supply the most
detailed account of actual service history found by the study team at any
of the study sites. The main drawback to this source is that the reports
will only cover the period that the case was in treatment. Therefore, these
reports cannot be relied upon as the sole source for service information.
If this source of information is available and selected for review, it would
be important to determine how consistently these reports are maintained across
service providers.

At all of the study sites, Medicaid emerged as a key source of information,
as this funding stream was used to fund a range of mental health and substance
abuse services that the study team had originally identified as being central
to the cost-analysis framework. State and local Medicaid systems may be able
to provide case-level expenditures made on behalf of study families.

It would be necessary to gather Medicaid identifiers on all study cases.
It may be possible to retrieve Medicaid identifiers directly from the child
welfare agency. For example, Kentucky's child welfare administrative files
interface with the Medicaid and public assistance files. Otherwise, case
names and birth dates would be submitted to the Medicaid office so that system
administrators can conduct a special data run to match study names with Medicaid
numbers.

One factor that needs to be taken into account is that case-level expenditures
may not always be available from state agencies. With the conversion of state
Medicaid systems to managed care, some states are now paying providers capitated
rates that prevent an analysis of individual costs. If this is the case,
service and service cost data may need to be gathered directly from the managed
care provider.

States, and often local jurisdictions within states, establish individual
policies for the expenditure of flexible funds or emergency outlays made
to families. As described earlier, in general, the study team found that
states maintain records of these expenditures on a case-level basis.
Table 8 lists common expenditures made from these
funds and provides some issues to explore to devise a data collection system.

States use a variety of payment tracking systems for emergency outlays and
other flexible funds. For instance, sites may use handwritten ledgers or
automated payment tracking systems. There may be payment receipts or vouchers
in case files. It would be necessary to review data sources for each expenditure
because sites may separate some costs into separate accounts such as a
transportation voucher system. In addition, many states maintain separate
accounts for emergency expenditures for child protective cases versus children
in foster care.

Producing case-specific cost estimates of agency casework is perhaps the
greatest challenge of a cost study in child welfare. Traditionally, state
and local finance offices do not track worker time by family served. Although
one of the study sites, Kentucky, designed a new case-specific reporting
system for its administrative files, the system was not fully functional
during the study period. Without such a tracking system in place, our four
study sites were reluctant to conduct cost analyses of directly provided
services beyond dividing total annual expenditures for service units (e.g.,
investigation, foster care case management), by total caseload served. This
method generates an annual, average cost-per-case that cannot be used to
distinguish differences between the treatment and control groups in service
needs and intensity of service activity.

In order to calculate case-specific expenditures for directly provided casework
services, it is necessary to first determine both the amount of time workers
dedicate to individual case activities and the value of worker labor that
would be applied to these activities. To establish labor costs, the study
team must understand the range of budget lines corresponding to casework
activities. The team must also determine whether sites can generate an hourly
rate of labor for each service unit. If so, there is a question of whether
the unit includes the labor costs of frontline, supervisory and support staff.
In many cases, service unit budgets for directly provided services are based
on average annual caseloads (e.g., 20 cases per caseworker and established
staffing patterns (e.g., one supervisor and administrative assistant to five
caseworkers). These issues will need to be understood prior to data collection
in order to determine appropriate units of service information to collect.
Alternatively, if personnel costs within different service units do not differ
significantly from one another, a standard hourly rate of casework labor
(that includes costs of supervisory and support staff) could be generated
and applied to all casework activity.

Once the availability of labor unit costs is established, it is necessary
to collect or estimate casework time devoted to study families. Several pieces
of information are needed to calculate worker time spent on a case. First,
standard service units must be developed that can be tracked (e.g., casework
activities related to in-home service models or activities related to foster
care placement). Second, one must estimate how much time, at least on average,
it takes workers to provide the service. Third, the number of times that
the service was delivered must be determined. A discussion of data sources
follows.

One method to collect actual, case-specific information would be for caseworkers
to complete case-specific service or activity logs for each study family.
The service logs could be designed to capture both the type of service delivered
and the time spent on each activity. Activities involved with each designated
service could be developed from worker focus groups or from any workload
analysis previously done by the site. Relatively simple data collection forms
could be inserted into the family's case record after case selection into
the study and completed by caseworkers each time they work on the case. (A
sample data collection form for investigative workers is shown in
Exhibit 1.) If a case is in more than one child
welfare unit simultaneously (e.g., the family is within the investigation
unit and one of the children is in foster care), than separate time sheets
would need to be maintained in each unit and compiled at the time of data
analysis. As with contract services logs, because this is a new record-keeping
system requirement, case records should be flagged and have explanatory notes
to assist with data collection. If this method is used, it is critical that
a validation system be implemented such as random supervisory reviews and/or
conducting cross checks of worker logs with case records.

An alternative approach would be to have workers complete a checklist of
activities delivered to families each month. Through the use of worker focus
groups described below, the study team could generate average timeframes
for each activity or service unit.

If sites are unwilling to have workers complete service logs on all study
families, forms could be completed on a representative sample of cases.
Information on actual service delivery could be used to develop average service
duration and applied to actual casework activities identified from a full
case record review described below.

One source of information on casework activity that should be considered
is a state's own analysis of workload standards. These projects produce
information on both standard service units and service durations, based on
casework observation. For example, New Jersey had conducted such an analysis
in 1991 for the purpose of determining appropriate caseloads for its case
managers. Using a random sample of 425 cases selected from field offices
across the state, case workers were asked to record the amount of time required
for various activities conducted on behalf of families at various risk levels
and at various stages of the child welfare continuum (pre-custody,
ongoing/in-home services, custody cases). Once baseline data were gathered,
a panel of local and national experts led by the Child Welfare League of
America worked to establish case practice standards and duration of individual
casework activities.

The casework standards resulting from the project include a description of
activities and frequency and duration of activities by case type (see Appendix
F for examples of service descriptions). Information generated by a workload
standards project can be applied to other sources of cost information to
generate estimates of directly provided services. Specifically, the service
descriptions and information on service duration can be used in conjunction
with other data sources that supply information on the frequency of service
delivery (e.g., the automated case management system or case records.)

Information from the workload standards project can be useful, even if the
specific findings need to be updated. Components of the analysis can be used
for discussions with caseworkers and supervisors as a basis on which to build
new estimates of direct service costs. For example, New Jersey's workload
analysis specified all of the activities that make up foster care "placement"
and also recommended that the bundle of activities that make up placement
should take 270 minutes. This information could be used in caseworker focus
groups (described below) to devise discrete casework activities and establish
updated service duration.

Another source of administrative data that should be considered is the site's
automated case management system. These systems generally supply information
on the number of CPS referrals, investigations of abuse and neglect, the
number and length of foster care placements, and the length of time the case
was opened in the agency with case opening and closing dates. In short,
administrative data files can provide information on the frequency of many
of the major bundles of direct service activities provided to a case (e.g.,
number of investigations, number of foster care placements). However, because
this data source does not supply information on the number of worker hours
dedicated to each service, information from the case management system would
have to be combined with other sources of information discussed in this section.

The major benefits of this data source are its widescale availability and
the fact that it is easy to access. Like other sources of data, however,
its reliability and thoroughness should be reviewed with site personnel.

In addition, as a component of the case management system, some states have
begun to operate automated visitation systems designed to ensure compliance
with mandated visitation timeframes. Philadelphia operates such a system
that requires workers to record the date and type of home or office visits
conducted for each child welfare case that has been opened for service. These
systems do not record the time involved with each visit (or any other activities
associated with the case), but they offer an automated record of actual casework
activity that can be combined with other sources of information to generate
case-specific cost information. If the study team chooses to create profiles
of families based on risk factors and service needs, information from the
automated visitation system could also be used to help classify families
based on the number of visits or amount of casework contact recorded by the
visitation system. The availability and accuracy of this and any other automated
tracking systems of actual casework activity should be explored with site
personnel.

Client case records are another source of actual case-specific data. Casework
entries on contact sheets should be identified as they can provide detailed
information about the type and purpose of casework contact. For example,
they describe home visits and phone calls to families and other concerned
parties, service referrals and court activity.

Again, reviewing case records is very labor intensive, typically requiring
an average of 1 to 2 hours per record. Consequently, prior to choosing to
conduct a case record review, it is important to both review a sample of
records to verify the amount of information on casework activities they include
and determine the best way to abstract needed data. This is important because
at one of the study sites, where investigative and ongoing protective service
caseloads were very high, case record contact sheets contained relatively
few detailed entries. Whether the lack of information was due to limited
time spent with each case or a lack of emphasis on recording casework activities
was unclear.

The most efficient way to conduct case record reviews would be to develop
standardized data collection forms. These forms would include a list of casework
activities by service function (e.g., activities involved with investigation)
that are developed by an agency's own workload standards project or by worker
focus groups. Reviewers would then fill in the relevant cells of the table
with the dates of these activities. If collecting data at multiple sites,
the casework activities developed at each site should be comparable across
sites.

If any of the above data sources are unavailable, intensive interviews with
worker focus groups can be used to create standardized service units and
generate estimates of service duration. Information gathered from focus groups
can be used to supplement other data sources described above. For example,
while an automated case management system will provide a discrete list of
services or case management functions and can supply information on how many
times each service occurred, it does not provide information on the length
of time it took to deliver each service. Worker focus groups can be used
to develop estimates of service duration.

There are several issues to keep in mind when using focus groups to define
discrete casework services. First, as discussed in the initial steps, in
order to facilitate data collection, it is important to target a discrete
number of casework activities based on their relevance to the study design.
Second, assuming that hourly labor rates differ across child welfare service
units, it is important to keep the service categories within a casework unit
to facilitate establishing a labor cost (or identify the difference in service
duration if it is conducted in different units). To do this, researchers
must become familiar with the agency's service delivery system 
from screening to case closure. Using a list of casework activities, determine
which agency worker is responsible for each task. Third, if combining this
activity with other data sources, services must be defined in such a way
that they can be matched with information contained in those other sources.
For example, if case records are to be used as the source for information
on actual case activity, it will be necessary to discuss how each service
is defined and recorded in the records. Finally, focus on the development
of service units that are likely to be provided in different amounts to the
treatment and control groups. For example, casework activity involved with
monitoring family preservation contracts or delivering the service directly.

Establishing discrete services within ongoing case management activities,
including ongoing protective services and out-of-home case management, will
require detailed discussions with supervisory and line staff. Every effort
should be made to create categories that account for the majority of casework
time.

Worker focus groups can also be used to help estimate the average amount
of time spent on providing each service unit. If time estimates vary widely,
it may be necessary to subdivide the service unit. For example, the counseling
needs will likely differ significantly between a sex abuse case and a neglect
case. Site officials and front line staff should be able to suggest
classification systems. The result of this effort would be a list of key
casework service units and the average number worker hours associated with
each unit. Using data on the cost of an hour of worker time, a unit cost
for each service unit would be established.

Indirect costs include an agency's overhead expenses based on costs such
as general administration, facilities and equipment, consumable supplies,
and staff training. It also includes fringe benefit costs for medical insurance,
pension, paid holidays, and paid leave. We found that site finance staff
can readily provide researchers with the agency's indirect rate. The rate
would then be applied to all appropriate cost elements.

Once data become available and are collected, the cost methodology formulas
for calculating the cost of serving clients in the study population can be
applied. The costs of each service component, activity, and unit of service
can be summed by client, by service, or by any particular group of clients
or services. The summation of costs is dependent on the purpose of the cost
analysis and the level of aggregation required in answering the questions
posed by the study. Due to the range of estimation procedures necessary to
conduct a cost analysis in child welfare, it is strongly recommended that
procedures for validating information be developed. For example, worker logs
and case records could be cross-checked to validate activities listed in
the logs. Preliminary findings on the costs of both the treatment and control
cases should be shared with site officials to discuss the reliability of
the findings. If costs for particular cases appear to be out of line, workers
could be interviewed to verify data collection procedures. In addition, meetings
between the research team and program staff should take place on a periodic
basis relevant to the study period to re-examine the list of targeted services
and make any necessary changes if new services have been introduced to the
area.

The purpose of conducting a cost analysis in an experimental design study
is to determine the differences in the costs of services provided to the
treatment and control groups. The validity and breadth of the data, and the
accuracy with which the cost information can address the research questions
of interest determines the strength of the analysis.

This paper described a series of useful sources of case-level data as well
as several estimation methods that can be used to develop other service costs.
The paper outlined six interrelated steps that can act as a guide to conducting
a cost analysis of child welfare interventions of interest.

Develop a cost data collection framework that contains all relevant services
and casework activities that will most likely influence outcomes between
the comparison groups.

Define the targeted services that have been selected for the cost study
analysis by identifying each of the components and casework activities involved
in delivering the service.

Learn how services are recorded, tracked, and funded and how payments
for services are made.

Review all existing cost data sources and the supplemental
documentation.

Establish data retrieval systems for each of the targeted services.

Conduct cost analyses and get feedback on results from site administrators
and knowledgeable staff for quality-control purposes.

Historically, child welfare agencies have not been required to manage their
systems from the perspective of financial accountability. To a large extent,
child welfare agencies have operated program management and financial systems
separately. Only recently have states and local jurisdictions attempted to
integrate these systems through, for example, introducing managed care concepts
into the financing and delivery of child welfare services. Moreover, many
other states and localities are interested in applying these methods within
their child welfare systems. Yet these efforts can be stymied by the lack
of case-specific and service-specific cost data from which to build capitated
rate structures. There is a growing realization that with a goal of improving
child welfare systems, program and financial management must become better
integrated. Means of deriving accurate cost information on services delivered
to clients is an essential element of this reform. The reforms must include
detailed systems of accountability for both public agency direct services
and contract-provided services.

As local, state, and federal child welfare laws, initiatives, waivers, budgets,
and clients demand more accountability for client outcomes and cost effectiveness
of services to children and their families, the service providers, both public
and private, are moved to reform the systems for accountability. Methods
for improvements in recording the data necessary for documenting performance,
client outcomes, cost of services, cost containment and effectiveness of
services will be valuable. System reform efforts to better integrate child
welfare management and financial systems will rely on improved means of deriving
accurate costs of services and the outcomes to families served. This paper
offers a variety of methods to establish such costs for a wide range of child
welfare services and thus can help inform these efforts.

Chambers, Jay G. and William T. Hartman. (April 1981). "A Cost-Based Approach
to the Funding of Educational Programs: An Application to Special Education."
Institute for Research on Educational Finance and Governance, School of
Education, Stanford University.

Chambers, Jay G. and Thomas B. Parrish. (December 1983). "The Development
of a Resource Cost Model Funding Base for Education Finance in Illinois"
for the Illinois State Board of Education. Final Report of the AEFP School
Finance Study for the State of Illinois.

Moore, Mary T., E. William Strang, Myron Schwartz, and Mark Braddock. (1988).
"Patterns in Special Education Service Delivery and Cost." For the Department
of Education. Decision Resources Corporation.

2. The Guide to Performance Measurement for State
Agencies by the Texas State Auditor's Office, Legislative Budget board,
and the Governor's Office of Budget and Planning was the resource used in
describing the performance based budgeting model concepts.

4. The fifth site, the
New York City HomeRebuilders
project, delivered family reunification and permanency services rather
than family preservation. In addition to significant differences in service
design, another difference between New York and the other study sites was
that the HomeRebuilders evaluation was conducted retrospectively 
assignment to experimental and control groups and the intensive phase of
the demonstration were complete when data collection began under this evaluation.