The New Hampshire Study of Supported Employment

for People With Severe Mental Illness

Robert E. DrakeDepartment of Psychiatry and Department of Community
and Family Medicine Dartmouth Medical SchoolGregory J. McHugoDepartment of Psychiatry and Department of Community
and Family Medicine Dartmouth Medical SchoolDeborah R. BeckerDepartment of Psychiatry and Department of Community
and Family Medicine Dartmouth Medical SchoolWilliam A. AnthonyDepartment of Rehabilitation Counseling Boston UniversityRobin E. ClarkDepartment of Community and Family Medicine Dartmouth
Medical School

ABSTRACTThis study compared supported employment services in 2 contrasting
programs: (a) Group Skills Training, a professional rehabilitation agency
outside of the mental health center that provided preemployment skills
training and support in obtaining and maintaining jobs, or (b) the Individual
Placement and Support (IPS) model, which integrated clinical and vocational
services within the mental health center. People with severe mental disorders
who expressed interest in competitive employment ( N = 143) were
randomly assigned to 1 of these 2 programs. Results showed that clients
in the IPS program were more likely to be competitively employed throughout
most of the 18-month follow-up. Among those who obtained jobs, there were
few group differences, although workers in the IPS program did work more
total hours and earn more total wages during the 18-month follow-up. There
were no group differences on nonvocational outcomes.
This work was supported by U.S. Public Health Services
Grant MH-00839 from the National Institute of Mental Health and Grant MH-47650
from the National Institute of Mental Health and the Substance Abuse and
Mental Health Services Administration and by the New Hampshire Divisions
of Mental Health and Vocational Rehabilitation, the Mental Health Center
of Greater Manchester, the Central New Hampshire Community Mental Health
Services, and the Employment Connection Specialists. We thank Walter Wilcox
for his contributions.Correspondence may be addressed to Robert E. Drake, New
Hampshire-Dartmouth Psychiatric Research Center, Main Building, 105 Pleasant
Street, Concord, New Hampshire, 03301.Electronic mail may be sent to Robert.E.Drake@Dartmouth.edu

Competitive employment is a primary goal for a large majority
of people with severe mental disorders (SMD). Approximately three fourths
of these individuals, including more than two thirds of those without jobs,
desire paid employment ( Rogers, Walsh, Masotta, & Danley, 1991 ).
Vocational agencies ( Hursh, Rogers, & Anthony, 1988 ), mental health
providers ( Bond & McDonel, 1991 ), families of people with mental
illness ( Kasper, Steinwachs, & Skinner, 1992 ), and mental health
services researchers ( Attkisson et al., 1992 ) also advocate vocational
opportunities. Employment is considered important, not only because of
the direct improvements in activity, social contacts, and remuneration
but also because work may promote gains in related areas such as self-esteem,
illness self-management, community tenure, integration into the community,
and quality of life ( Black, 1988 ; Lehman, 1983 ; Mathews, 1979 ; Palmer,
1989 ; Strauss, Harding, Silverman, Eichler, & Lieberman, 1988) . Work
may also decrease use of mental health services and reliance on the mental
health system ( Clark & Bond, in press ; Rogers, Sciarappa, McDonald-Wilson,
& Danley, 1995) .

Despite the interests of consumers, families, and professional groups,
the actual rates of vocational assistance and of competitive employment
for people with SMD are quite meager. People with SMD have consistently
had the worst employment outcomes of the various disability groups served
by state and federal rehabilitation programs ( Marshak, Bostick, &
Turton, 1990 ). Many studies show that less than 15% of people with SMD
are employed at any one time ( Anthony & Blanch, 1987 ). Even successful
programs show low levels of competitive employment for people with the
most severe disorders, such as schizophrenia and bipolar disorder (e.g.,
Jacobs, Kardashian, Kreinbring, Ponder, & Simpson, 1984 ).

The barriers to employment for people with SMD include societal stigma;
attitudes of professionals, family members, consumers, and employers; economic
incentives of social insurance programs (e.g., Supplemental Security Income,
Social Security Disability Insurance, Medicaid, and Medicare); lack of
access to vocational services; and services that emphasize assessment and
prevocational goals rather than competitive employment and follow-along
supports ( Bond & McDonel, 1991 ; Rutman, 1994 ). How the Americans
with Disabilities Act will affect these barriers is not yet clear.

Little consensus exists, however, on how to extend SE to people with
SMD. Specific approaches to SE have not been compared in experimental studies.
One issue that particularly warrants study is the organizational linkage
between mental health and vocational services ( National Institute of Mental
Health, 1991 ). In the traditional service system (the brokered model),
clients are linked with vocational agencies, or vendors, with the Department
of Vocational Rehabilitation serving as a broker. Categorical funding,
organizational differences, and differences in training perpetuate a philosophical
and practical separation of mental health and vocational services. This
structure of parallel organizations has the potential advantages of specialization,
avoiding the stigma of mental illness treatment in vocational services
and developing a different culture of higher expectations ( Cook, 1992
). In practice, collaboration between clinical and vocational programs
often proves problematic because the service systems become rigid at the
interface and have difficulty cooperating to individualize services for
a particular client ( Harding, Strauss, Hafez, & Liberman, 1987 ; Rutman,
1994 ). Some providers have, therefore, advocated for integration of clinical
and vocational services so that the burden of combining clinical and rehabilitative
perspectives in a coherent way rests entirely on the providers rather than
on the clients. Most prominent among the integrated clinical and vocational
models is the Assertive Community Treatment (ACT) model, which has continued
to evolve to emphasize competitive employment ( Russert & Frey, 1991
). Earlier controlled studies of ACT failed to demonstrate clear advantages
in terms of competitive employment ( Hoult, Reynolds, Charbonneau-Powis,
Weekes, & Briggs, 1983 ; Mulder, 1982 ; Stein & Test, 1980 ), and
most ACT replications have dropped or deemphasized the vocational component
( Bond, 1992 ). Two recent unpublished studies do, however, show advantages
for ACT programs over the usual brokered system in terms of competitive
employment ( McFarlane et al., 1993 ; Test, 1992 ).

A second issue involves the role of preemployment skills training (
Bond, 1992 ). Many experts advocate prevocational skills training before
vocational placement ( Mueser & Liberman, 1988 ). Others have argued
that clients need job experiences rather than skills training before vocational
placement ( Bond, 1992 ). Rapid placement has some empirical support. Bond
and Dincin (1986) randomly assigned 107 clients in a psychiatric rehabilitation
program to accelerated or gradual placement in transitional employment.
After 15 months, 20% of those in the accelerated group were in competitive
jobs, compared with 7% of those in gradual placement. In a second study,
Bond et al. (1995) randomly assigned 86 psychiatric clients to accelerated
versus gradual entry into supported employment. After 3 years, 59% of those
who received accelerated placement, compared with only 6% who received
gradual placement, were competitively employed.

The purpose of the present study was to compare two models of supported
employment for people with SMD. One program (Group Skills Training; GST)
was an exemplar of the brokered service model (i.e., vocational and mental
health services in separate agencies) and emphasized preemployment skills
training. The second program (Individual Placement and Support; IPS) used
an integrated service model (vocational and mental health services combined
within the same program) and did not include preemployment skills training.
Thus, the models offered a clear contrast on these two key dimensions.
In this article, vocational and nonvocational outcomes for 18 months are
examined.

Study Hypotheses

Because the study focused on clients who expressed interest in competitive
employment and on two programs that were explicitly aimed at competitive
employment, the primary hypotheses focused exclusively on competitive employment.
Vocational outcomes for GST participants were expected to lag behind those
for IPS participants for the initial 2-3 months of the study because IPS
clients began to look for jobs immediately, whereas GST clients began with
skills training before searching for employment. Following this initial
advantage for IPS, it was hypothesized that GST clients would obtain jobs
at a higher rate, would obtain better jobs, would work more hours, would
keep their jobs longer, and would be more satisfied with their jobs. All
hypotheses were examined as two-tailed tests to allow for the possibility
that IPS would continue to outperform GST after the first 3 months.

Method

Participants

One hundred forty-three adults with SMD from the two mental health centers
in which the study was conducted served as voluntary participants in this
experiment. For inclusion in this study, a participant was required to
have (a) a major mental illness with major role dysfunction of at least
2 years duration, (b) clinical stability (i.e., out of the hospital) for
at least 1 month, (c) local residence for at least 6 months, (d) an age
between 20 and 65, (e) unemployment for at least 1 month but interest in
competitive employment, (f) informed consent, and (g) absence of significant
memory impairment, medical illness, or substance dependence that would
preclude participating in a training program.

As Table 1 shows, clients in the sample were predominantly young, Caucasian,
and single. They had relatively good employment histories but were not
currently working. Their primary diagnoses were heterogeneous: schizophrenia
and related psychotic disorders, 46.9%; bipolar disorder and other severe
mood disorders, 42.7%; and other disorders (primarily severe personality
disorders), 10.5%. Nearly all were on prescribed medications. Their current
levels of psychiatric symptoms and of alcohol and drug use were low, but
many had histories of hospitalization, homelessness, or incarceration during
the previous year.

To examine representativeness, we compared the sample with two groups.
The first comparison group involved a statewide survey of 1,536 patients
who were certified as having chronic mental illness and were being served
in community mental health programs in New Hampshire during the time of
the study. Results of this comparison, shown in Table 1 , indicate that
the clients in the study were demographically similar to clients in the
community throughout the state, although those in the study were slightly
younger and more educated. Clients in the study were also less likely to
be working and less likely to be abusing alcohol and other drugs, which
was consistent with inclusion criteria for the study. They were also slightly
more likely to be in independent living situations.

The second comparison was with a sample of 29 nonstudy clients selected
at random in the two mental health centers in which the study took place.
This comparison group showed that study clients were younger, better educated,
less likely to be in school or working, more likely to have a history of
competitive employment, and more likely to desire employment. On the other
hand, they had lower Global Assessment Scale scores. 1Few
other differences were statistically significant, even before adjustment
for the Bonferroni inequality. Thus, consistent with study criteria, clients
had good vocational histories and motivation to work but were otherwise
similar to other clients with SMD in the same community mental health centers.

Complete 18-month vocational outcome data were obtained on 140 of the
143 clients (97.9%). One client dropped out of the study after completing
the baseline interview, a second client died of cancer 7 months after beginning
the study, and a third dropped out after 12 months in the study.
Programs

The first program was identified as the outstanding exemplar of the
brokered model in New Hampshire. GST was a supported employment program
for individuals with SMD that began in Massachusetts in 1985 ( Harrison
& Perelson, 1989 ; Trotter, Minkoff, Harrison, & Hoops, 1988 ).
The program offered individualized intake, preemployment training in a
group format, individualized placement and support on the job, liaison
with mental health providers, and follow-along supports. The preemployment
training was designed to develop awareness and skills in the three areas
of choosing, getting, and keeping a job. In addition to discussing and
practicing the skills needed for these tasks, clients were encouraged to
explore work-related values and to understand realistically their strengths
and weaknesses as workers. Following the initial skills training, clients
met with staff in a group twice each week to continue building interview
skills and to discuss potential job leads and interviews. Once employed,
clients continued to receive individual support services from GST staff.
Empirical support for GST came from an open clinical trial in Boston in
which 35% of 156 clients with SMD completed the program and obtained competitive
employment ( Harrison & Perelson, 1989 ).

The IPS model was also developed to provide supported employment services
for people with SMD ( Becker & Drake, 1994 ). IPS used a team approach
to integrate mental health and vocational services. Employment specialists
were hired by mental health centers and attached directly to clinical teams
to ensure coordinated services. Rather than providing preemployment assessment
and training in job-related activities, IPS employment specialists began
helping clients to find jobs immediately and, after securing employment,
provided training and follow-along supports as needed. The IPS employment
specialists assumed that clients would learn about the job world, and about
their skills and preferences, on the job rather than through preemployment
training. The IPS model was derived primarily from the ACT program. In
contrast to the ACT model, however, employment specialists in IPS handle
all of the vocational activities, link with more than one clinical team,
and concentrate on clients who express interest in competitive employment
rather than on all clients.

Empirical support for IPS came from a quasi-experimental study in New
Hampshire in which one of two comparable rehabilitative day treatment centers
was closed and replaced by the IPS program ( Drake, Becker, Biesanz, et
al., 1994 ; Torrey, Becker, & Drake, 1995 ). In the program that converted,
rates of competitive employment among 71 clients with SMD increased during
the first year in the IPS program from 25% to 39%, and among 27 regular
day treatment attenders from 33% to 56%, while employment remained constant
in the comparison day treatment program. No negative outcomes were associated
with the program conversion. A replication in the site that served as a
comparison for the initial study showed similar outcomes ( Drake, Becker,
Biesanz, & Wyzik, in press ). Competitive employment increased from
13% to 23% among 112 clients with SMD, and from 9% to 40% among 35 clients
who had been regular attenders of day treatment.
Measures

All clients were assessed at baseline, 6, 12, and 18 months using a
combined interview that incorporated sections of several standardized instruments:
the Employment and Income Review ( Center for Psychiatric Rehabilitation,
1989 ), the Global Assessment Scale (GAS; Endicott, Spitzer, Fleiss, &
Cohen, 1976 ), the expanded Brief Psychiatric Rating Scale ( Lukoff, Liberman,
& Nuechterlein, 1986 ), the Rosenberg Self-Esteem Scale ( Rosenberg,
1969 ), and the Quality of Life Interview (QOLI; Lehman, 1983 ). The interview
assessed demographics, vocational history, psychiatric history, entitlements,
financial status, several domains of functioning, symptoms, self-esteem,
global functioning, vocational goals, and quality of life. Subjects also
received a diagnostic interview using the Structured Clinical Interview
for the Diagnostic and Statistical Manual of Mental Disorders (3rd
ed., revised; Spitzer, Williams, Gibbon, & First, 1989 ) during the
course of the study. The primary outcome, competitive employment, was defined
as work in the competitive job market at prevailing wages supervised by
personnel employed by the business. Employment was assessed weekly by employment
specialists in both programs and by direct interviews with clients.
Design and Procedure

The basic study design was a two-site, controlled, clinical trial with
random assignment to GST or IPS. Both the GST and IPS programs were implemented
in two New Hampshire cities with populations of 166,000 and 119,000 respectively.
The same private, nonprofit vocational agency implemented the GST model
in both cities. This agency conducted the 8-week skills training classes
in a single setting and used a common staff for job development in the
two cities, but separate staff members provided the direct support in finding
and maintaining jobs in the two cities. The IPS model was implemented in
two separate community mental health centers that served their respective
cities. Each mental health center had its own staff and preexisting vocational
program. A team leader within each mental health center supervised the
IPS employment specialists in that center.

Implementation was monitored through observations of team meetings in
both programs, through site visits from individual members of the research
team and through daily logs of use of services within both programs. The
research project director observed 1-hr team meetings with GST staff involved
in the project weekly throughout the 3 years of the project; she also attended
a 2-hr staff meeting every other week with the combined IPS staff from
the two mental health centers. The project director also reviewed computerized
implementation data and gave feedback to program leaders whenever evidence
of model drift was detected. Other members of the research team made periodic
visits to both programs to interview staff and administrators regarding
implementation.

Implementation data generally supported the fidelity of both interventions.
The two programs had approximately equivalent personnel-three full-time
staff members-throughout most of the project but deployed the staff differently.
The three IPS staff worked directly with clients in all phases of supported
employment, whereas the three GST staff divided functions into job training,
job development, and job support roles. Clients in the two programs received
approximately the same amount of direct contact hours, (61.6 ± 37.1
for IPS versus 74.1 ± 59.2 for GST, t (141) = 1.49, ns
),
but the types of service units received were different and consistent with
their respective models. Implementation of GST across the two cities was
similar, although the program focused attention on the smaller city during
the last year of the study. Consistent with the model, the program exclusively
targeted competitive jobs. Implementation of IPS differed in the two cities.
Both IPS programs assisted some clients in obtaining volunteer work and
sheltered jobs. In one site, these jobs were used as a means of transitioning
clients to competitive work, which was consistent with the IPS model. In
the second site, however, employment specialists placed more emphasis on
sheltered jobs and used them for assessment and long-term placement, contrary
to the IPS model. Despite feedback to supervisors from the project director,
this pattern persisted throughout the study, and this site was considered
to have a weaker implementation of IPS.

Informational meetings with clients, families, and mental health providers
were used to recruit clients with SMD from two community mental health
centers in New Hampshire. Interested clients were required to attend at
least four sessions of a weekly research induction group, in which the
research project director and staff from the two vocational programs described
all research and program procedures, answered questions, and screened clients
for inclusion criteria ( Drake, Becker, & Anthony, 1994 ). The purpose
of the prerandomization group experience was to ensure that potential participants
met inclusion criteria, were fully informed, were motivated to participate
in the project, and were able to give informed consent.

After giving informed consent for all research procedures, clients were
stratified on the extent of previous employment and randomly assigned within
site to GST or IPS. All clients completed baseline assessment procedures
and were linked successfully with their assigned vocational program. They
were assessed at baseline with a composite interview of approximately 1
hr duration by a research interviewer who was independent of the clinical
or vocational programs. Clients were reevaluated using a similar interview
at 6, 12, and 18 months. They were also interviewed about each job after
1 month of employment, after every 6 months of employment, and at the termination
of the job. Employment specialists and clients were interviewed separately
about all jobs to assess hours worked and wages earned. Discrepancies,
which usually involved jobs that clients obtained on their own, were clarified
by further investigation (e.g., reports from third parties).

Interviewers were trained using survey research interview techniques
from the University of Michigan ( Guenzel, Berckmans, & Cannell, 1983
) and checked throughout the study for reliability. Both inter-rater and
test-retest reliability were maintained at high levels (kappas and intraclass
correlations consistently above .7, with the exception of recent drug use,
which was infrequent in this sample).

Standard statistics were used to test group differences throughout the
analyses. All reported effect sizes (ES) are variants of the d statistic.
Hedges' g, which divides the mean difference by the pooled standard
deviation, was used for the difference between independent group means
( Rosenthal, 1984 ). Cohen's w, which is equal to the phi coefficient
in a 2 × 2 contingency table, was converted to d ( Rosenthal,
1984 ). Within-group effect sizes were computed as g, which divides
the difference between the posttest and the pretest by the standard deviation
of the pretest ( Becker, 1988 ).

Results

Group Comparability

The entire sample ( N = 143) at baseline was used to assess group
equivalence after random assignment to treatment conditions. The IPS group
had 74 clients, and the GST group had 69 clients. Univariate tests of significance
were conducted for each variable, using chi-square tests of significance
for discrete data and t tests of the difference between independent
group means for continuous data. All statistical tests were two-tailed.
In some cases, in which there was high skewness or underused categories,
the data were recoded into fewer categories.

For the purpose of assessing group equivalence, 78 variables in six
domains (demographic, psychiatric, employment history, current employment
status, quality of life, and self-esteem) were examined. The groups differed
at p < .05 on six of these variables (see Footnote 1). Since
four differences would be expected on the basis of chance, the groups were
largely equivalent at the outset. The groups did not differ on any measures
of demographic characteristics, psychiatric diagnosis, employment history,
education and training, global assessment scale, quality of life, alcohol
and drug use, living situation, psychiatric history, or vocational supports.
Of 24 variables relating to current work status, three self-report items
regarding reasons for current unemployment were significant at p <
.05. GST clients were more likely to report that they were not currently
working due to mental disability, fear of losing benefits, and being a
homemaker. After Bonferroni correction within this category, only one variable
(not working because of mental disability) remained significant at p
=
.05. One subscale of the BPRS (anergia) showed a significant difference
favoring IPS clients, but this difference was not significant after Bonferroni
correction. A group difference on the Rosenberg Self-Esteem Scale also
favored IPS clients: 24.0 versus 21.5; t (141) = 2.67, p =
.008. Finally, IPS clients were more likely to take lithium as a medication,
but this difference failed to attain significance after Bonferroni correction
within the category of medications.

Vocational Outcomes for All Clients

For the analysis of vocational outcomes, the three clients with incomplete
follow-up data (one in IPS and two in GST) were dropped. Because there
is no consensus in the field about the most important vocational outcome
variable, several measures of vocational attainment were analyzed. The
analysis proceeded in two steps. First, overall group differences were
examined. Second, because differences in the overall employment rate might
account for further differences in quality or quantity of work, work variables
were examined with the analysis restricted to clients who obtained employment.

IPS clients were approximately twice as likely to obtain a competitive
job during the study: 78.1% versus 40.3%; &chi;21,
N
=
140= 20.78, p
< .001; ES = 84. They were also
more than twice as likely to work at a job for 20 hr or more per week at
some time during the study: 46.6% versus 22.4%; &chi;21,
N
=
140= 8.98, p
= .003; ES =52. In addition, IPS
clients averaged more total hours in competitives jobs: 607.03 ±
842.59 versus 205.13 ± 400.09; t (104.8) = 3.65, p <
.0001; ES = .60. They also earned more total wages in competitive jobs:
$3,394.01 ± $5,446.25 versus $1,077.82 ± $2,237.84; t
(97.33)
= 3.34, p = .001; ES = .55. Because total hourse and wages were
variables with censored distribution because of many zeros, Mann-Whitney
nonparametric tests were used to confirm the differences in hours and wages.
For hours, U = 1446.5, W = 3724.5, Z
= 4.31, and p
< .0001. For wages, U = 1451.5, W
= 3729.5, Z =4.49,
and p < .0001. Hours and wages were also highly correlated (r
= .96). As expected, results using hours or wages were in every case similar.

To examine the temporal pattern of competitive employment in the two
programs, we studied monthly employment rates throughout the 18-month follow-up
(see Figure 1 ). As expected, IPS subjects began to become employed in
the first month of the study. Their rate of competitive employment stabilized
close to 40% by the fourth month of the study and was maintained at around
that level through the 18 months. On the other hand, GST clients worked
very little during the first 3 months of the study while they were engaged
in skills training. Their rate of competitive employment increased gradually
and stabilized at around 20% in the 10th month of the study. The monthly
employment rates show a significant difference in favor of IPS in 14 of
the 18 months.

Figure 1

No interactions of Program × Site were found, and this study was
not designed to support separate analyses in the two sites, because the
samples were small in each setting ( n = 91 in the larger city and
n
= 49 in the smaller city) and the sites were not independent (GST staff
overlapped in the two sites). The process analysis indicated, however,
that IPS was poorly implemented in the smaller city and that, conversely,
GST staff concentrated their efforts in the smaller city. These observations
led to exploratory analyses by site. The overall rates of competitive employment
strongly favored IPS over GST in the larger city 85.4%
vs. 37.2%, &chi;21,
N
=
91= 22.53, p
< .001; ES = 1.15, whereas the
difference was not significant in the smaller city 64.0%
vs. 45.8%, &chi;21,
N
=
49= 1.63, ns
; ES = .37.

To examine the influence of other variables on the overall rate of employment,
we conducted logistic regression analyses that included previous work,
treatment group, site, gender, age, and psychiatric diagnosis as predictors.
As shown in Table 2 , only previous work and group were significant predictors
of competitive employment in the main-effects model; no interaction term
was statistically significant.

Vocational Outcomes for Workers Only

Among clients who obtained jobs ( n = 57 for IPS, and n =
27 for GST), few treatment group differences emerged. IPS and GST workers
were similar in terms of proportion working at least 20 hr per week, weeks
in the longest job, and work status in the 18th month. However, IPS workers
averaged more total hours (777.4 ± 882.0 vs. 509.0 ± 495.7)
and earned more total wages ($4,346.71 ± $5,824.20 vs. $2,674.58
± $2,877.07) than GST clients, and these differences were statistically
significant when previous work was used as a covariate, F s(1, 81)
= 5.35 and 5.57, respectively, p s < .05; ES = .34 and .33, respectively.
Previous work did not interact with treatment group, and there were no
significant main effects or interactions for age, gender, diagnosis, or
site. Among those who were working at the 18-month interview, there were
also no group differences in satisfaction with job, F (1, 29) =
1.40, ns .
Nonvocational Outcomes for All Clients

Changes over time in global functioning, quality of life, self-esteem,
and psychiatric symptoms were also examined. These analyses were considered
exploratory, leading to downward adjustment of the per-comparison alpha
level to account for multiple tests. The multivariate approach to repeated
measures analysis of variance was used to examine the main effect for group
(IPS vs. GST), the main effect for time (baseline and 6, 12, and 18 months),
and the Group × Time interaction. Effect sizes for the main effect
for time are reported as the standardized mean difference between the endpoint
(18 months) and the pretreatment baseline point.

A second cluster of variables that were not directly related to the
employment outcomes in this study included measures of overall symptom
severity (BPRS total score; sum of 24 seven-point Likert-type symptom items;
range, 24-168), overall life satisfaction (QOLI; average of two 7-point
Likert-type items), satisfaction with both housing and town (QOLI; average
of five 7-point Likert-type items), satisfaction with mental health care
(QOLI; one 7-point Likert-type item), and self-esteem (Rosenberg 10-item
scale; range, 10-50). The results showed significant main effects for time
on overall symptom severity and self-esteem. For symptom severity, the
full sample showed a slight increase in symptoms (still in the nonclinical
range) over the course of the study mpre
38.90 8.41, mpost
=
42.35 11.98, F3,
113= 4.52, p
= .005; ES = .41. For self-esteem,
examination of the main effect for time indicated that the positive change
from baseline was significant in the middle months of the study but had
eroded to nonsignificant levels by the endpoint. Once again, there were
no statistically significant Group × Time interactions.

Discussion

The main finding of this study was that the IPS program was more successful
at helping people with SMD to obtain competitive employment. Clients in
IPS got jobs faster and maintained their advantage throughout the 18 months
of the study. Further group differences, in terms of time or amount of
work, can be explained most parsimoniously by the differential rates of
employment. The success of the IPS model in helping people to obtain competitive
jobs can be attributed to several aspects of the program. In IPS, vocational
services were integrated within the mental health program, so that clients
did not have difficulties making the transition to another program and
did not experience problems that were due to miscommunication between mental
health and vocational staff. These interagency barriers are characteristic
of the brokered model ( Harding et al., 1987 ; Katz, Geckle, Goldstein,
& Eichenmuller, 1990 ; Marrone, Horgan, Scripture, & Grossman,
1984 ; Rutman, 1994 ) and were documented repeatedly in the process evaluation
of the GST program in this study. In addition, clients in IPS did not become
discouraged during preemployment tasks, because IPS specifies a direct
approach to finding jobs, to supporting people in jobs, and to helping
people move on to new jobs if they are dissatisfied or lose their jobs
( Becker & Drake, 1994 ).

The lack of interactions between program type and individual characteristics
indicates that IPS was the preferred program for all of the groups that
were considered: clients with good vocational histories versus those with
poor histories, men versus women, older versus younger clients, and those
with schizophrenia versus those with affective disorders. Employment history
also predicted attaining competitive jobs but did not interact with the
influence of vocational program assignment.

One study hypothesis was that skills training might lead to greater
quality of employment: more satisfying, longer lasting, or higher paying
jobs. Once clients became employed, however, job satisfaction and job tenure
were quite similar in the two programs, whereas total earnings, which were
strongly correlated with total hours of employment, actually favored IPS.
In other words, the types of jobs that clients obtained were similar between
the two programs. Consistent with studies of rapid placement ( Bond &
Dincin, 1986 ; Bond, Dietzen, McGrew, & Miller, 1995 ), there was no
evidence that preemployment skills training yielded any advantages in terms
of quantity or quality of employment.

Because the brokered model in this study (GST) differed from the integrated
model (IPS) on two dimensions-offering vocational services in a program
that was separate from the mental health program and offering preemployment
skills training-it is difficult to say whether one of these dimensions
alone would have made a difference. The process analysis clearly indicated
that interagency difficulties were common in the GST model. Clients often
had difficulty transitioning between one agency and another, and miscommunications
were normative. Explaining how skills training could have long-term adverse
consequences rather than merely result in a delay in finding employment
is difficult on the surface, but the explanation may be related to limited
resources. In other words, devoting time and energy to skills training
means not devoting resources to helping clients find and keep jobs.

Quality of implementation of the IPS model appeared to be related to
differential effectiveness, and the outcome difference in the larger city,
which had a better implementation of IPS, appeared to be carrying the overall
program effect. However, these site-specific findings are difficult to
interpret because of the small sample sizes, the nonindependence of sites,
and the observations that many factors related to clients and work environments,
in addition to implementation, differed between the two cities.

The vocational results reported here could have been due to other factors.
One possibility that cannot be ruled out involves the confound between
program and site. The IPS program was delivered only in mental health centers
and the GST program only in the rehabilitation agency. Unmeasured influences,
such as staff conflict, attitudes, skills, or demoralization related to
these sites, could have influenced the results.

Another possibility is an overall poor implementation of GST. Process
data showed, however, that the GST program was a faithful implementation
of the intended model and that the GST staff were well educated, well trained,
and well supervised. Moreover, the vocational outcomes in a previous implementation
of the GST model in Boston found a 35% rate of competitive employment (
Harrison & Perelson, 1989 ), as compared with 39% in this study. Nevertheless,
other external factors, such as local economic conditions, the nature of
the New Hampshire mental health system, or the restrictions imposed on
the program by participation in a research study, may have limited the
effectiveness of GST. On the other side, independent studies of IPS and
of other integrated models support the results of the present study ( Drake,
Becker, Biesanz, Torrey, McHugo, & Wyzik, 1994 ; Drake, Becker, Biesanz,
& Wyzik, 1994 ; McFarlane et al., 1993 ; Test, 1992 ). Preliminary
findings from a study of IPS versus the brokered model that is ongoing
in Washington, DC, with an urban, formerly homeless sample are also consistent
with the results reported here.

Despite assumptions in the literature, there was no evidence in this
study that program differences produced outcome differences in domains
of functioning other than employment. Clients in both programs improved
over time in areas that were proximally related to employment, such as
global functioning and satisfaction with finances, but showed little change
in other areas. The mild increase in symptom levels is difficult to interpret
but probably means little because symptoms remained in the subclinical
range. It seems likely that nonvocational domains of outcome are only weakly
related to vocational function and that program effects are specific to
the content of the program. Another possibility is that clients need more
time in jobs before vocational gains generalize to other domains of functioning.

Kasper,J. A., Steinwachs,D. M. & Skinner,E. A. (1992).
Family perspectives on the service needs of people with serious and persistent
mental illness: Part II: Needs for assistance and needs that go unmet.
Innovations
& Research, 1, 21-33.

McFarlane,W. R., Stastny,P., Deakins,S., Dushay,R., Lukens,E.
& Link,B. (1993). Family psychoeducation, assertive community treatment
and vocational rehabilitation for persons with schizophrenia. Paper
presented at the fourth annual conference of the National Association of
Mental Health Program Directors Research Institute on state mental health
agency services research and program evaluation: Proceedings. Alexandria,
VA: National Association of Mental Health Program Directors Research Institute.

Mulder,R. (1982). Final evaluation of the Harbinger
Program as a demonstration project. Unpublished manuscript.

National Institute of Mental Health., (1991). Caring
for people with severe mental disorders: A national plan of research to
improve services (DHHS Publication No. ADM 91-1762). Washington, DC:
U.S. Government Printing Office.

Nichols,M. (1989). Demonstration study of a supported
employment program for persons with severe mental illness: Benefits, costs,
and outcomes. Unpublished master's thesis, Department of Psychology,
Indiana University-Purdue University at Indianapolis.

Palmer,F. (1989). The place of work in psychiatric rehabilitation.
Hospital
and Community Psychiatry, 40, 222-224.

Test,M. A. (1992, April). Programs of assertive community
treatment: Twenty years of research. Paper presented at the National
Institute of Mental Health Conference on Building Social Work Knowledge
for Effective Mental Health Services and Policies Conference, Bethesda,
MD.