3 ACKNOWLEDGEMENTS In developing the design for evaluating the State Partnership Initiative, we have benefited substantially from support provided by staff from the Social Security Administration. The project s initial task manager, Paul O Leary, guided the initial development process and helped to coordinate the critical early data processing tasks done at SSA. Subsequently, Jim Sears, the project s final task manager, helped to shape the ultimate design and to organize all the SSA-based data processing required to test the design. Thuy Ho with assistance from Jeff Shapiro very ably conducted that data processing, particularly the numerous computer jobs required to test the iterative beneficiary matching process. This report literally could not have been completed without her efforts. In addition, Minh Huynh, Mary Barbour, Mike Abramo, and Joel Packman helped immensely by providing data extracts and running computer programs. The project team also benefited substantially from the ongoing technical advice of our colleague Peter Schochet. Even more important was the work done by our great team of programmers who processed the more than 800 gigabytes of data used to assess the core net-outcomes evaluation design: Kate Bartkus, Miriam Loewenberg, Nora Paxton, and Rachel Sullivan. Finally, Vinita Jethwani helped to compile information about State Project activities and the policy context within which the initiative operates. We have also benefited from the advice of the project s Technical Evaluation Support Group. This group reviewed our design and provided comments on earlier versions of this report. The group included the following individuals: Natalie Funk, Office of Employment Support Programs, Social Security Administration Lex Frieden, The Institute for Rehabilitation and Research Alan Krueger, Department of Economics and Woodrow Wilson School of Public Policy, Princeton University Robert Moffitt, Department of Economics, The Johns Hopkins University Kalman Rupp, Office of Research, Evaluation, and Statistics, Social Security Administration Charles Scott, Office of Research, Evaluation, and Statistics, Social Security Administration Mark Shroder, Office of Policy Development, U.S. Department of Housing and Urban Development Jack Worrall, Department of Economics, Rutgers University Anthony Young, NISH, formerly the National Industries for the Severely Handicapped The report was edited by Walt Brower and Roy Grisham and produced by William Garrett and Sharon Clark.

4

5 CONTENTS Chapter Page EXECUTIVE SUMMARY...xiii I THE STATE PARTNERSHIP INITIATIVE AND ITS EVALUATION... 1 A. THE STATE PARTNERSHIP INITIATIVE Characteristics of the State Project Interventions Characteristics of State Project Participants... 5 B. POLICY CONTEXT... 7 C. THE FOUR-PART EVALUATION STRATEGY The Core Net Outcomes Evaluation The Supplemental Evaluation Component State Projects Evaluations Implementation and Synthesis Analyses II SELECTION OF COMPARISON AREAS FOR THE CORE EVALUATION A. DEFINITION OF POTENTIAL COMPARISON AREAS B. METHODS AND DATA USED TO SELECT COMPARISON AREAS Population Density Population Growth Unemployment Rate Total County Employment Percentage of County Land in Farming Presence of Substantial Manufacturing Public Transportation Use Poverty Rate Percentage of County Population in Racial/Ethnic Minorities SSI Beneficiary Employment Rate Other Area Characteristics Considered C. ESTIMATING WEIGHTS FOR THE COUNTY CHARACTERISTICS D. PRELIMINARY LIST OF COMPARISON COUNTIES... 36

6 vi Chapter Page E. FINAL SELECTION OF COMPARISON AREAS California Illinois Iowa (SSA) Minnesota New Hampshire New Mexico New York North Carolina Ohio Oklahoma Utah Vermont Wisconsin F. CONCLUSIONS AND ROBUSTNESS OF THE SELECTIONS III COMPARISON BENEFICIARY SELECTION PROCESS A. HOW COMPARISON GROUPS ARE SELECTED Propensity Score Matching Potential Comparison Group Members When Characteristics of Potential Comparison Groups Are Measured Tests Used to Assess the Similarity of Participants and the Comparison Groups Characteristics Used in the Matching Process B. PRELIMINARY ASSESSMENT OF THE MATCHING PROCESS C. TIPS FOR IMPLEMENTING THE MATCHING PROCESS IN THE FUTURE IV COMPUTING NET OUTCOMES AND ASSESSING THE VALIDITY OF THE RESULTS Contents A. METHOD USED TO COMPUTE NET OUTCOMES B. ANALYSES USED TO ASSESS THE VALIDITY OF THE RESULTS Matching Several Periods Before Enrollment Comparing Our Comparison Group Results to Experimental Results Alternative Validity Analyses... 76

7 vii Chapter Page C. EXAMINING PRE-ENROLLMENT NET OUTCOMES D. COMPARISON OF PRELIMINARY FINDINGS FROM THE CORE AND RANDOM ASSIGNMENT EVALUATIONS FOR NEW YORK S SPI PROJECT The New York Experiment Preliminary Estimated Effects: Experimental Results The Similarity of Employment Changes Estimated Using the Core Evaluation and Experimental Designs Using the Full Treatment Group V PROTOCOL FOR IMPLEMENTING THE CORE EVALUATION IN THE FUTURE A. CREATING THE MATCHING FILE Verifying Participant SSNs Creating the Finder File Obtaining and Processing the SSA Extracts Merging the Processed Extracts to Create Two Matching Files B. SELECTING COMPARISON GROUPS C. PRODUCING NET OUTCOME ESTIMATES D. STATISTICAL POWER OF THE CORE EVALUATION E. SUGGESTED FUTURE SCHEDULE VI SUPPLEMENTAL EVALUATION POSSIBILITIES A. PRELIMINARY SUPPLEMENTAL EVALUATION DESIGN Analyze Service Delivery and Participation Using State Project Data Estimate the Impact of Services Using State Project and Process Data Refine Core Earnings Estimates Using State Unemployment Insurance Data for Key State Projects Estimating Intervention Costs and Effects on Tax Payments Methodological Analysis B. SUMMARY OF STATE PROJECT DATA Overview of the Current Status of the Data Collection System Overview of the State Project Data File Structure Contents

9 TABLES Table Page I.1 SUMMARY OF INTERVENTIONS BEING TESTED BY THE STATE PROJECTS... 4 I.2 CHARACTERISTICS OF BENEFICIARIES ENROLLED IN SSA-FUNDED AND RSA-FUNDED STTE PROJECTS (THROUGH DECEMBER 2001)... 6 I.3 STATE PROJECT ENROLLMENTS... 8 I.4 ISSUES ADDRESSED IN THE STATE PARTNERSHIP INITIATIVE EVALUATION II.1 STATES FROM WHICH COMPARISON COUNTIES WERE SELECTED II.2 CHARACTERISTICS USED IN THE COUNTY MATCHING PROCESS II.3 II.4 II.5 II.6 II.7 MARGINAL EFFECTS AND WEIGHTS FOR 11 COUNTY CHARACTERISTICS RELATED TO SSI RECIPIENT EMPLOYMENT SSI BENEFICIARIES EMPLOYMENT STATUS IN JUNE 1999 AND IN THE PREVIOUS THREE MONTHS ADEQUACY OF THE BENEFICIARY POPULATION IN THE MATCHED COMPARISON AREAS FOR SSI AND SSDI BENEFICIARIES PERCENT OF COUNTIES SELECTED IN THE REGRESSION-BASED METHOD ALSO SELECTED BY ALTERNATIVE SELECTION METHODS PERCENT OF COUNTIES SELECTED BY STATE PROJECTS ALSO SELECTED BY THE EQUAL-WEIGHT METHOD III.1 CHARACTERISTICS INITIALLY USED IN THE BENEFICIARY MATCHING PROCESS III.2 SIMILARITY OF PARTICIPANT AND COMPARISON GROUPS III.3 IOWA PROJECT NUMBER OF SSI PARTICIPANTS AND COMPARISON MEMBERS IN POPULOUS AREAS BY THE ESTIMATED PROPENSITY SCORE.. 62 III.4 IOWA PROJECT SSI PARTICIPANTS AND COMPARISON GROUP MEMBERS IN POPULOUS AREAS CHARACTERISTICS USED IN THE MATCHING PROCESS IV.1 SAMPLE SIZES IN THE NEW YORK PROJECT EXPERIMENT... 82

10 x Table Page V.1 SUMMARY OF THE PROTOCOL FOR IMPLEMENTING THE CORE EVALUATION V.2 EXPECTED NUMBER OF PARTICIPANTS AND THE MINIMUM DETECTABLE DIFFERENCES (MDD) FOR THE CORE EVALUATION Tables

11 FIGURES Figure Page III.1 PROCESS USED TO SELECT COMPARISON GROUPS IV.1 NEW YORK PROJECT EMPLOYMENT RATES IV.2 NEW YORK CITY DEMONSTRATION AND COMPARISON AREA UNEMPLOYMENT RATES IV.3 BUFFALO DEMONSTRATION AND COMPARISON AREA UNEMPLOYMENT RATES IV.4 NEW YORK PROJECT EMPLOYMENT RATES FOR TREATMENT, CONTROL, AND COMPARISON GROUPS IV.5 NEW YORK PROJECT ALTERNATIVE ESTIMATES OF THE NET CHANGE IN EMPLOYMENT RATES FOR THE FULL TREATMENT GROUP V.1 PROCESS USED TO CREATE THE MATCHING FILES... 90

12

13 EXECUTIVE SUMMARY T he State Partnership Initiative (SPI) funded 18 projects in 17 states to investigate new strategies that promote employment among people with disabilities, particularly those who receive disability benefits from either the Supplemental Security Income (SSI) or the Social Security Disability Insurance (SSDI) programs. The projects share a common goal but differ considerably in their methods. Their approaches include (1) helping beneficiaries understand the ways in which Social Security Administration (SSA) programs encourage work, (2) providing waivers that make work more attractive to beneficiaries, (3) delivering training to improve people s abilities, (4) persuading employers to hire people with disabilities, and (5) trying to improve the coordination and efficiency of the overall service system. To understand fully the effects of SPI, SSA developed a four-part evaluation that combines multiple data sets and methods to produce several estimates of the effects the State Project interventions generate. It then uses qualitative information about project implementation to synthesize those estimates and to develop an understanding of the relative performance of the projects. A major goal of this evaluation is to compare the projects to gain a better understanding of which mix of services is the most effective and learn whether services are especially useful for any particular participant groups. Specifically, the four components of the overall SPI evaluation design are as follows: Core Evaluation. This evaluation component will compare key outcomes of the beneficiaries who participated in the State Projects (participants) with outcomes of a comparison group that is selected to match the participants in terms of the characteristics of the areas in which they live, as well as their demographics, prior labor market experiences, and prior benefit receipt. The core evaluation will use only SSA administrative data and can be applied consistently in the 13 State Projects that have provided participants Social Security numbers (SSNs) to the evaluation. Supplemental Evaluation. The core evaluation can be supplemented with data that the State Projects plan to provide to SSA. These include detailed data about participants characteristics, their receipt of project services, and their employment and benefitreceipt outcomes. These data are not yet available for analysis, but enough is known about them to provide several designs for their use in the SPI evaluation. The most important use will be to study the nature, intensity, and duration of services that participants receive. State Project Internal Evaluations. Each State Project is conducting its own evaluation. These evaluations can often use state data that will provide more detail than is possible using only the SSA data available in the core evaluation. However, variation among the

14 xiv projects in these data sources and in their evaluation methods will make it hard to compare findings across the projects. Implementation and Synthesis Analysis. The SPI Project Office at Virginia Commonwealth University (VCU) is documenting and analyzing the interventions fielded by State Projects. In addition, each project will submit a final report about its interventions. This information will help evaluators understand the ways in which State Projects changed the services available to participants, as well as the context within which projects operated. It will also facilitate comparisons among the projects and an overall synthesis of the findings from the other SPI evaluation components. This report focuses on our efforts to develop and test a design for the core evaluation. Our design begins by selecting comparison areas that match the areas in which State Projects offered services and then selects matched comparison beneficiaries from those areas. It uses statistical methods to correct for any residual differences between the participants and the matched comparison group. Finally, it includes a validity test that compares results from the core evaluation with those derived from the random assignment experiment the New York State Project is using to test its intervention. We also include a summary of the procedures required to implement the core evaluation. In addition, the report describes designs for two other evaluation components: the supplemental evaluation and the implementation and synthesis evaluation. While the core evaluation design was thoroughly tested, it is still too early to use the design to estimate net outcomes for the projects. Our tests used an early sample of participants, and the available follow-up data for that sample cover only the first three months after they enrolled in a project. It would be extremely unlikely that the projects would have had any substantial effects in that short a time. The projects offered services whose effects are likely to be seen only after six or more months. Unlike programs that place participants directly in jobs, the State Projects generally offered services that affect employment more indirectly. For example, the benefits counseling services offered by all projects help beneficiaries understand the available work incentives. That understanding should help employed beneficiaries maintain their jobs and encourage unemployed beneficiaries to seek jobs. But those effects will take a while to materialize, since they rely on the actions participants take after receiving project services. We recommend that the core evaluation design be implemented in early At that time, there would be enough Summary Earnings Record (SER) data to select comparison beneficiaries for all participants. Also, 12 months of follow-up data would be available to measure benefit receipt for all participants and to measure employment and earnings for SSI beneficiaries. In addition, the implementation evaluation should be complete by the end of 2003, so that it can provide the qualitative information required to interpret quantitative findings from the core evaluation. A more complete evaluation that uses the SER data to measure effects on employment and earnings for SSDI beneficiaries and SSI beneficiaries who leave the rolls will have to wait until early 2005, when the full SER data for calendar 2003 becomes available. Executive Summary

15 xv OVERVIEW OF THE STATE PARTNERSHIP INITIATIVE The 18 State Projects are addressing the persistently low rates of employment among people with disabilities, especially disabled SSA beneficiaries. For example, only 6.7 percent of disabled SSI recipients were employed in December 2000 (Pickett 2001). While this rate has risen slowly over the past 25 years (from 3.4 percent in 1976), it has been largely stable since In addition, almost two-thirds of working SSI recipients earn less than the amount SSA designates as substantial gainful activity (currently $780 a month). Furthermore, among disabled SSI recipients who work, use of SSA s current work-incentive programs is low. The overall goal of the SPI evaluation is to determine the extent to which State Projects interventions improve economic independence among SSA s disabled beneficiaries. Specifically, how much do these interventions increase average earnings and decrease dependence on SSI and SSDI benefits? A key feature of the initiative is the variation among State Projects with respect to the services and policies they are implementing. This variation will enable the evaluation to study which combinations of services appear to be most effective and which groups of participants seem to benefit the most. SSA and the Rehabilitation Services Administration (RSA) have provided most of the funding for SPI. SSA funded 12 projects through cooperative agreements and RSA funded 6 as systems change grants. Supplementary funding and support have also been provided by the U.S. Department of Labor (DOL) and the Substance Abuse and Mental Health Services Administration (SAMHSA). Finally, many states have provided services that supplemented those funded through SPI. These 18 State Projects seek to increase employment among people with disabilities by improving information about current work incentives, instituting new work incentives, providing better access to vocational supports, working with employers, and changing service systems so as to place greater emphasis on employment outcomes. Most State Projects are targeting beneficiaries with severe mental illness, although many projects target other groups, including disabled students and people with HIV/AIDS. The State Projects began in fall 1998, initiated participant enrollment in 1999, and, with one exception, are expected to continue serving participants through fall Through December 2001, the State Projects have enrolled more than 6,500 participants (4,284 in the SSA-funded projects and 2,300 in the RSA-funded ones). Projects have sent the SPI Project Office intake data for all these participants. Further, all 12 SSA-funded projects and the RSA-funded project in Utah have reported their participants SSNs to the national evaluation and can therefore be included in the core evaluation. Enrollment is expected to continue at least through September 2002, so the total should exceed 9,000 with current rates of enrollment (with over 6,000 expected in the 13 projects that have provided SSNs). OVERVIEW OF THE SPI EVALUATION DESIGN The main conclusion of this report is that the core evaluation design is feasible, that it can produce accurate estimates of the extent to which State Projects change the employment, earnings, benefit receipt, and total income of participants. Furthermore, the process of identifying comparison counties and comparison beneficiaries has been thoroughly tested and Executive Summary

16 xvi appears to produce valid comparison groups for the core evaluation. However, this selection process requires the extraction and processing of substantial data from SSA s administrative system, as well as a great deal of researcher input. Comparison Counties Are Well Matched to the Demonstration Counties We selected comparison counties on a county-by-county basis. For each demonstration county, we ranked, in terms of their similarity with the demonstration county, nearby counties where project services were not offered. To match counties, we used 13 county-level characteristics that affect employment, including population density and growth, unemployment rate, prominence of farming and manufacturing, use of public transportation, poverty rate, county demographics, and pre-demonstration employment rate of SSI beneficiaries. In ranking counties, we weighed these factors to reflect their relative importance in explaining employment among SSI beneficiaries. Using data on the employment and characteristics of a total of 750,000 SSI beneficiaries, we estimated (for each State Project) separate sets of weights for these characteristics. As a final step, we asked State Project staff to review our initial selection of comparison counties and delete those that had service environments substantially different from those in the demonstration counties. Overall, the basic elements of this approach appear to work. We were able to select comparison counties similar to all the demonstration counties included in the core evaluation. Furthermore, the comparison counties contain enough beneficiaries to form a good pool from which evaluators can select a comparison group that matches the beneficiaries in the demonstration. Comparison Beneficiaries Can Be Well Matched to Participants Our process for selecting comparison beneficiaries uses SSA data to find people who are similar to participants along many important characteristics. For SSI beneficiaries, the characteristics include 7 demographics, 24 months of pre-enrollment information for 9 outcomes, 5 calendar year measures of pre-enrollment employment, 5 calendar year measures of pre-enrollment earnings, and 13 area characteristics a total of nearly 250 variables. The list includes most of the characteristics that the literature has found to be related to the outcomes that will be analyzed as part of the evaluation. For beneficiaries who only receive SSDI, the number of characteristics is smaller, but it still includes the most important variables, such as preenrollment employment and earnings. Statistical matching using propensity scores is used to select comparison groups. Our preliminary assessment of the matching process for four of the State Projects indicates that it will produce comparison groups that are well matched to participants; however, this process must be implemented on SSA s mainframe computer. Creating the files used by the process for matching beneficiaries and by the method for producing net outcomes requires dozens of different computer programs to extract and process several million records. Some data processing took days to complete. The evaluation had to use SSA s mainframe because (1) only SSA staff can use the calendar year employment and earnings information that the design needs, and (2) the mainframe processes the data much faster than personal computers can. Executive Summary

17 xvii Preliminary Validity Tests Suggest That the Core Evaluation Will Produce Reliable Findings In the interest of providing SSA policymakers with accurate evidence of the effect of the State Projects, we have designed two analyses to assess whether the core evaluation design will produce valid results. The first validity analysis selects comparison groups for participants several periods before enrollment and then examines net outcomes during the periods after that earlier date, but prior to enrollment. If our matching process is accurate, net outcomes during this period should equal zero, because neither participants nor comparison group members received any project services. The second validity analysis compares the comparison group selected using the core evaluation design with the control group selected using random assignment in the New York project s experiment. Because experimental methods are widely regarded as the benchmark for estimating the effect of an intervention, the results of this comparison will provide strong evidence of the validity of the matching process developed for the core evaluation. We conducted a preliminary implementation of these validity analyses based on an early cohort of participants in the New York project. The analysis was based on the New York project, because it is the only one that is conducting a randomized experiment and has enrolled enough beneficiaries to support the analysis. The results suggest that our design will provide accurate evidence of the effect of the State Projects. However, the preliminary nature of the currently available data means that the tests should be conducted again with the full set of participants and a longer follow-up period. Implementing the Core Evaluation Will Require Substantial Decision-Making by Researchers Based on our experience of testing the evaluation design, updating the short-term net outcomes requires a great deal of data processing and researcher input. For example, if the core evaluation is run in early 2004, it will require more than a month of programmer time to obtain data from the more than 30 SSA data sets needed for updating the evaluation data files. It would then take two to three months of time from a researcher and programmer team to select the matched comparison groups. This last step involves examining results from hundreds of statistical tests and modifying the selection process if those tests indicate that the initially selected comparison group is not suitable. While we have developed computer programs to implement the core evaluation design, evaluators will have to review a few aspects of those programs. In particular, they will have to ensure that the formats and content of SSA data extracts have not changed, and make judgments about the specification of the process used to select comparison groups. All Components of the Four-Part Evaluation Design Have Been Implemented In addition to the core evaluation described in this report, there has been substantial progress on the other evaluation components. The major activity supporting the supplemental evaluation has been data collection. All but one State Project have developed participant-tracking systems Executive Summary

18 xviii and are providing the evaluation with data about participant characteristics, use of project services, reliance on benefit programs, and employment. Data about participant characteristics are already available from these systems and provide a clear description of the people who have enrolled. The projects and Project Office are still refining the systems that collect data on service use, benefit receipt, and employment, but they appear confident that these systems will produce reliable data in 2003 for most projects. All the SSA-funded State Projects have developed evaluation designs for their internal evaluations. They have begun to build the required data sets, and several states have already used data from their state unemployment insurance systems to produce preliminary analyses of net effects on employment and earnings. Initially, the RSA-funded states had not been required to conduct evaluations, so their evaluation plans lag behind those of the SSA-funded states. Nevertheless, they have begun to develop plans to document their interventions and conduct at least qualitative assessments of their results. Finally, the states and the Project Office have begun the implementation analysis. All the SSA-funded projects have provided initial descriptions of their interventions and plan to update those descriptions in The Project Office has conducted cross-project analyses of three key components: the provision of benefits counseling, the coordination of services through the DOL One-Stop Centers, and the state s provision of Medicaid Buy-In programs that allow people with disabilities to purchase health insurance through the Medicaid program. A final round of data collection and analysis, scheduled for 2003, should provide enough information to conduct the planned implementation and synthesis evaluations. Most Projects Will Enroll Enough Participants to Support Implementing the Core Evaluation It appears that most projects will enroll enough participants that the methods developed for the core evaluation will be able to detect policy-relevant impacts. For example, the evaluation should be able to detect a change in annual employment rates of 4.4 percentage points for the 942 SSI beneficiaries the New York project expects to enroll. Overall, it should be possible to detect increases in employment of 14 percentage points or more in most projects. Such increases are similar to those observed for SSA s Transitional Employment Training Demonstration (TETD), which increased employment rates by 9 to 14 percentage points over the 6 years following participants enrollment (an approximate doubling of employment rates relative to that project s control group). The power of the evaluation is greater when states are pooled. For example, if the evaluation combines the samples from the three states that implemented SSI waivers (California, New York, and Wisconsin), the evaluation should be able to detect increases in employment of just 3 percentage points. Similarly, if the samples for all states were combined, the resulting sample of approximately 6,000 beneficiaries should be able to detect increases of less than 2 percentage points. Such a combined analysis would indicate the overall extent to which a policy that gave states the general mandate and funding provided by SPI would increase beneficiary employment and effect other key outcomes. Executive Summary

19 I THE STATE PARTNERSHIP INITIATIVE AND ITS EVALUATION T he State Partnership Initiative (SPI) includes 18 projects in 17 states that are investigating new strategies to promote employment among people with disabilities, primarily those who receive disability benefits from either the Supplemental Security Income (SSI) or the Social Security Disability Insurance (SSDI) programs. Most State Projects began enrolling participants in 1999 and will continue to provide services into The Initiative also included a project office to provide technical assistance to the State Projects and collect data about implementation. Finally, SPI included the development of the cross-state net outcomes evaluation described in this report. The overall goal of SPI is to determine the extent to which the services provided by the State Projects improve economic independence among people with disabilities, particularly those receiving SSI or SSDI benefits. Specifically, how much do the services increase earnings and decrease dependence on benefits? A key feature of SPI is the variation among the State Projects with respect to the services and policies they are implementing. This variation provides an opportunity to study which combinations of services appear to be most effective and which groups of participants seem to benefit the most. SSA developed a four-part evaluation strategy for SPI. This strategy will use multiple data sets and methods to produce several estimates of the effects generated by the State Project interventions. It will then use qualitative information about project implementation to synthesize those estimates and to develop an understanding of the relative performance of the projects. Specifically, the four components are as follows: Core Evaluation. This evaluation component will compare outcomes for the beneficiaries who participated in the State Projects (participants) with outcomes for a comparison group that is selected to match the participants in terms of demographics, prior employment and earnings, and prior benefit receipt. The design uses only Social Security Administration (SSA) administrative data and can be applied consistently in all the State Projects that provide participants Social Security numbers (SSNs) to the evaluation. The design begins by selecting comparison areas that match the areas in which State Projects offered services (Chapter II). It then selects matched comparison beneficiaries from those selected areas (Chapter III). This design appears to produce valid results based on several tests, including comparison of findings with the randomassignment experiment fielded by the New York State Project (Chapter IV). The procedures, data files, and computer programs required to implement the design are summarized in Chapter V with full details provided in a separate report (Khan et al. 2002).

20 2 Supplemental Evaluation. The core evaluation can be supplemented using data that the State Projects plan to provide to SSA. These include detailed data about participants characteristics, their receipt of project services, and their employment and benefit-receipt outcomes. These data are not yet available for analysis, but enough is known about them to provide several designs for their use in the SPI evaluation (Chapter VI). State Project Internal Evaluations. Each State Project is conducting its own evaluation. These evaluations can often use state data that will provide more detail than is possible using only the SSA data used in the core evaluation. However, variation among the projects in these data sources and in their evaluation methods will make it hard to compare findings across the projects. Most of the projects have submitted evaluation designs (which are summarized later in this chapter) and expect to produce findings late in Implementation and Synthesis Analysis. The SPI Project Office at Virginia Commonwealth University (VCU) is documenting and analyzing the interventions fielded by the State Projects. In addition, each project will submit a final report about its interventions. This information will help evaluators understand the ways in which State Projects changed the services available to participants and the context within which projects operated (Chapter VII). It will also facilitate comparisons among the projects and an overall synthesis of the findings from the other SPI evaluation components. A. THE STATE PARTNERSHIP INITIATIVE SPI represents the first multi-agency effort under Executive Order 13078, which in March 1998 established the President s Task Force on Employment of Adults with Disabilities. This task force brought together federal departments and agencies to support the goals of the Americans with Disabilities Act. Its mission is to evaluate existing federal programs in order to determine what changes, modifications, and innovations may be necessary to remove barriers that inhibit adults with disabilities from becoming gainfully employed. SSA and the Rehabilitation Services Administration (RSA) have taken the lead in funding and directing SPI, which includes 18 projects in 17 states. 1 The U.S. Department of Labor (DOL) and the Substance Abuse and Mental Health Services Administration (SAMHSA) have also provided supplementary funding and support for SPI. These 18 State Projects (12 cooperative agreements funded by SSA, 6 systems change grants by RSA) seek to increase employment among people with disabilities by improving information about current work incentives, instituting new work incentives, providing better access to vocational supports, working with employers, and changing service systems so as to place greater emphasis on employment outcomes. Most State Projects are targeting beneficiaries with severe mental 1 The RSA initiative focused on activities aimed at changing the overall system that helps people with disabilities obtain employment and live independently. As a result, the overall SSA/RSA effort is sometimes referred to as the State Partnership and Systems Change Initiative, although the acronym SPI is still used. I. The State Partnership Initiative and Its Evaluation

11 Disability Services: Structure and Funding Section Purpose Provide One-Stop staff a basic understanding of the disability service structure, and various disability funding agencies, to assist them in

Office of Policy Office of Research, Evaluation, and Statistics Research and Statistics Note No. 2001-02 October 2001 Follow-up of Former Drug Addict and Alcoholic Beneficiaries * Introduction In 1996,

WORKING PAPER NUMBER: 2014-06 Youth with Disabilities at the Crossroads: The Intersection of Vocational Rehabilitation and Disability Benefits for Youth with Disabilities July 2014 Todd Honeycutt * Allison

The American Council of Life Insurers Written Statement for the Record for Maintaining the Disability Insurance Trust Fund s Solvency Before the United States House Committee on Ways and Means Subcommittee

Order Code RS22501 September 6, 2006 CRS Report for Congress Received through the CRS Web Summary Child Welfare: The Chafee Foster Care Independence Program (CFCIP) Adrienne L. Fernandes Analyst in Social

CHILD CARE Building Pathways, Creating Roadblocks: State Child Care Assistance Policies for Parents in School May 2015 Education and training opportunities help parents trying to gain more stable employment

Medicare Advantage Cuts in the Affordable Care Act: March 2013 Update Robert A. Book l March 2013 The Centers for Medicare and Medicaid Services (CMS) recently announced proposed rules that would cut payments

Health Care Policy Cost Index 2012: Ranking the States According to Policies Affecting the Cost of Health Coverage by Raymond J. Keating Chief Economist Small Business & Entrepreneurship Council February

Social Security Disability Insurance and Supplemental Security Income for Undergraduates with Disabilities: An Analysis of the National Postsecondary Student Aid Survey (NPSAS 2000) By Hugh Berry & Megan

Social Security Administration Benefits Planning Query Handbook (BPQY) Distributed by Office of Program Development and Research Office of Retirement and Disability Policy September 2009 Version 2.0.3C

New Federal Funding to Serve Unemployed Entrepreneurs in Your Community Small businesses not only drive the economy and job growth, they also provide a pathway for business owners particularly minorities,

Service Area Background Information Service Area Plan Department of Rehabilitation Services Vocational Rehabilitation Services Service Area Description This service area provides comprehensive services

EMPLOYERS: 9 Ways to Earn Income Tax Credits for Your Company The Work Opportunity Tax Credit (WOTC): An Employer-Friendly Benefit for Hiring Job Seekers Most in Need of Employment Background and Updates:

IcI Institute for Co m m u n it y In c l u s i o n Issue 17 2009 Update Tools for Inclusion family and consumer series Making it Easier to Go to Work: What the Changes at Social Security Mean to You By

THE CHARACTERISTICS OF PERSONS REPORTING STATE CHILDREN S HEALTH INSURANCE PROGRAM COVERAGE IN THE MARCH 2001 CURRENT POPULATION SURVEY 1 Charles Nelson and Robert Mills HHES Division, U.S. Bureau of the

REPORT April 2009 States Act to Help People Laid Off from Small Firms: More Needs to Be Done In the past two months, several states have taken action to make sure state residents who lose their jobs in

The assessment of Survey of Income and Program Participation (SIPP) benefit data using longitudinal administrative records * Minh Huynh, Kalman Rupp, and James Sears Office of Research, Evaluation, and

9 Social Security Disability Benefits: The Impact of Employment Section Purpose Provide information on social security disability benefits, so that One-Stop staff have an understanding of how these benefits

Issue Brief November 2007 Changes in the Cost of Medicare Prescription Drug Plans, 2007-2008 BY JOSHUA LANIER AND DEAN BAKER* The average premium for Medicare Part D prescription drug plans rose by 24.5

State Assistance Programs for SSI Recipients, January 2005 Social Security Administration Office of Policy Office of Research, Evaluation, and Statistics Selected Features of State Assistance Programs,

EMPLOYERS: 12 Ways to Earn Income Tax Credits for Your Company The Work Opportunity Tax Credit (WOTC): An Employer-Friendly Benefit for Hiring Job Seekers Most in Need of Employment Background and Updates:

This document is scheduled to be published in the Federal Register on 10/01/2015 and available online at http://federalregister.gov/a/2015-24648, and on FDsys.gov DEPARTMENT OF THE TREASURY Internal Revenue

Employment after Traumatic Brain Injury Living with Brain Injury This brochure was developed for friends, family members, and caregivers of persons with brain injury. It also may be used in discussions

Employment After Spinal Cord Injury (SCI) A resource for individuals with SCI and their supporters This presentation is based on SCI Model Systems research and was developed with support from the National

Summary of the State Elder Abuse Questionnaire for West Virginia A Final Report to: Department of Health and Human Services February 2002 Prepared by Researchers at The University of Iowa Department of

Summary of the State Elder Abuse Questionnaire for Ohio A Final Report to: Department of Human Services February 2002 Prepared by Researchers at The University of Iowa Department of Family Medicine 2 Grant,

Introduction to Social Security Work Incentives 3.19.14 Presented by the Disability Employment Initiative (DEI) a Federal grant operated by the Alaska Dept. of Labor and Workforce Development (DOL&WD),

Summary of the State Elder Abuse Questionnaire for Vermont A Final Report to: Vermont Adult Protective Services February 2002 Prepared by Researchers at The University of Iowa Department of Family Medicine

1 Summary of the State Elder Abuse Questionnaire for Georgia A Final Report to: Department of Human Resources Office of Adult Services February 2002 Prepared by Researchers at The University of Iowa Department

THE CENTER FOR STATE AND LOCAL FINANCE JUNE 2015 Georgia s Ranking Among the States: Budget, Taxes, and Other Indicators IN COLLABORATION WITH ABOUT THE CENTER FOR STATE AND LOCAL FINANCE The (CSLF) mission

At SuperStar Careers, we think our Military veterans are the true stars. It takes a lot of bravery and leadership to serve and protect the United States of America. As a token of our appreciation, we have

A Monthly Public Assistance Update from the Illinois Department of Human Services January 2014 Summary Total cases receiving Public Assistance in Illinois decreased by 10,008 (20,138 persons) in January

Unemployment Insurance and Social Security Retirement Offsets By National Employment Law Project December 2003 Introduction The policy of denying or reducing unemployment insurance benefits to Social Security

WORKING AND KEEPING YOUR MEDICAL ASSISTANCE (MEDICAID) BENEFITS: A FACT SHEET FOR SOCIAL SECURITY BENEFICIARIES Prepared by the Disability Rights Network of Pennsylvania Are you a Social Security beneficiary

Summary of the State Elder Abuse Questionnaire for Washington A Final Report to: Department of Social and Health Services February 2002 Prepared by Researchers at The University of Iowa Department of Family

BUSINESS DEVELOPMENT OUTCOMES Small Business Ownership Description Total number of employer firms and self-employment in the state per 100 people in the labor force, 2003. Explanation Business ownership

Measuring the Performance of North Carolina s Workforce Development System January 2015 Introduction Session Law 2012-131 required the N.C. Commission on Workforce Development (recently renamed NCWorks

New York's workers' comp: High benefits, higher costs New York s workers' comp benefits have risen to enter the mainstream but they cannot explain why employers costs remain so high By Paul Jahn Executive

What Is College and Career Readiness? A Summary of State Definitions Peter A. Conforti On March 13, 2010, President Barack Obama issued a blueprint for reauthorizing the Elementary and Secondary Education

Summary of the State Elder Abuse Questionnaire for Illinois A Final Report to: Illinois Department on Aging February 2002 Prepared by Researchers at The University of Iowa Department of Family Medicine

Summary of the State Elder Abuse Questionnaire for Alabama A Final Report to: Department of Human Resources Office of Adult Services February 2002 Prepared by Researchers at The University of Iowa Department

Summary of the State Elder Abuse Questionnaire for Kentucky A Final Report to: Department for Social Services February 2002 Prepared by Researchers at The University of Iowa Department of Family Medicine

Summary of the State Elder Abuse Questionnaire for Connecticut A Final Report to: Department of Social Services February 2002 Prepared by Researchers at The University of Iowa Department of Family Medicine

Special Education Reform: Basics for SLTs Introduction This overview is designed to explain the New York City Department of Education s special education reform and how it relates to your school. We will

Sources of Health Insurance Coverage in Georgia 2007-2008 Tabulations of the March 2008 Annual Social and Economic Supplement to the Current Population Survey and The 2008 Georgia Population Survey William

Work Incentives For Individuals Receiving SSDI or SSI Payments SSDI Work Incentives * Continuation of Benefits While in a Rehabilitation Program * Continuation of Medicare Coverage * Exemption of Scholarships

National Employment Law Project Concise Guide to Assistance for Jobless Workers in the American Recovery and Reinvestment Act Introduction Congress recently passed the American Recovery and Reinvestment

Order Code RS22440 Updated August 5, 2008 Unemployment Compensation (Insurance) and Military Service Summary Julie M. Whittaker Specialist in Income Security Domestic Social Policy Division The Unemployment

Health Insurance Options for Children with Autism Spectrum Disorders Meg Comeau, MHA Boston University School of Public Health Who we are Funded by the Maternal and Child Health Bureau A project of the

Page 1 of 7 (https://www.insidehighered.com) Data show key role for community colleges in 4-year degree production Submitted by Doug Lederman on September 10, 2012-3:00am The notion that community colleges

FEDERAL PROGRAMS FOR TRANSITION-AGE YOUTH WITH SERIOUS MENTAL HEALTH CONDITIONS 7-1 7. Independent Living for People with Disabilities and Other Special Populations Vocational Rehabilitation Title I Formula

College Completion in : The Impact on the Workforce and the Economy February 2012 Investing in Individuals and Society through College Completion 30 25 20 15 10 5 0 Higher Levels of Education and Skills

Health Care Policy Cost Index 2011: Ranking the States According to Policies Affecting the Cost of Health Care by Raymond J. Keating Chief Economist Small Business & Entrepreneurship Council 2944 Hunter

Following the Patient Protection and Affordable Care Act s emphasis on Accountable Care Organizations (ACOs) and the announcement of the Medicare Shared Savings Program, an increased interest has emerged

Is Applying for Disability the Right Step for Me? 300 S. Adams, Green Bay, WI 54301 920-448-4300 www.adrcofbrowncounty.org In This Handout Is applying for disability the right step for me? 2 Considering