Introduction

The National Youth at Risk Programs Sustainability Study was designed to examine the sustainability of Youth at Risk projects initially funded through the USDA/CSREES CYFAR (Children, Youth, and Families at risk) Initiative. The current report focuses on 94 Youth at Risk (YAR) projects four years after their initial grant ended and represents the latest in a series of reports focused on the sustainability of these projects. For more information on this research, please see the following reports (Mancini & Marek, 1998; Marek, Mancini, & Brock, 1999; Mancini & Marek, 2000; Marek, Mancini, & Brock, 2000). [To read the 1998 and 1999 reports go to]:

Findings from the past YAR reports suggest that there are a number of questions that are important for understanding project sustainability. These include:

Is the project meeting its original goals?

Is the project maintaining or expanding rather than reducing efforts to meet its goals?

Does the project have secure future funding?

There are a number of factors that facilitate program sustainability and are incorporated into this report. These factors include: Program flexibility, Adequate and qualified staff, Collaboration within communities and involvement by the communities, Adequate and long-term funding, and the Ability to demonstrate impact. The central questions addressed in this report are:

What is the current status of the 94 Youth at Risk projects four years after funding ended?

What are the present roles of Cooperative Extension and community collaborators in supporting these community-based projects?

What has facilitated or impeded the sustainability of these projects?

The results of the study are organized into seven sections:

Descriptions of project status and goals,

Project flexibility in terms of programming modifications, and changes in programming, project sites and participants;

Staffing patterns of both paid staff and volunteers;

Collaborations including community involvement, Cooperative Extension's current role in projects, and project leadership;

Funding plans and current sources of funding;

Current evaluation efforts and uses of evaluation results; and

Obstacles to project survival.

Methods

Procedures

The present survey was constructed based on information obtained from data collected by qualitative telephone interviews of program leaders at 1 1/2 years and 2 1/2 years post-USDA/CSREES CYFAR funding. Some survey questions were further revised following survey data collection from the 67 YAR projects funded between 1991-1996 at four years post USDA/CSREES CYFAR funding. However, survey items remain focused on project status, goals, programs, sites, participants, staffing, collaboration, Cooperative Extension involvement, project leadership, community involvement, funding, evaluation, and obstacles to sustainability. Current project functioning as well as change over the past year were assessed in each of these areas.

Surveys were mailed to project leaders at each of the 84 projects that were active 2 1/2 years post-USDA/CSREES CYFAR funding. Surveys were sent to two contacts at nine projects because two project leaders were identified-these data were combined where applicable. Therefore, a total of 93 surveys were sent to the projects. In order to maximize the response rate, follow-up emails, post-cards, telephone calls, and follow-up surveys were sent to non-respondents. In all, at least one completed survey was returned from each of the 84 active projects, yielding a project response rate of 100%. [Some level of data was collected from all projects, including those that became inactive since the last assessment.]

Data Analysis

Data were entered in the Statistical Package for the Social Sciences (SPSS) and frequencies were run for each survey item. The frequencies reported in the Results section vary depending on the number of responses for any item that were provided. With the exception of reporting on project status, all analyses were conducted only on active projects and valid percentages are provided.

In addition to descriptive statistics, a variety of bivariate analyses were conducted including chi-square (*) and one-way analysis of variance (**). In addition, linear regression (***) analysis was also conducted. Throughout the report the asterisk designation indicates which analysis was used. Most of the analyses focused on the impact of various project characteristics on perceived sustainability and the extent of current programming for at risk youth.

Results

I. Project Status

The following section provides descriptions of project activity, perceived sustainability of projects, goals, and programming efforts. In addition, relevant changes and the reasons for these changes are presented.

Project ActivityThe majority of projects (76%) still remain active four years post USDA/CSREES CYFAR funding (see Table 1). The current status of each of the 94 projects, including leadership patterns and collaborators involved, is shown in Appendix A.

Perceived Sustainability of ProjectsRespondents from 65 of the active projects indicated how much they perceived their project to be sustained since the end of USDA/CSREES CYFAR funding. As shown in Figure 1, nearly all respondents (97%) reported that their projects were at least partially sustained, with 26% (n=17) identifying their projects as „Completely¾ sustained. Twenty-seven respondents (42%) perceived their projects as „Mostly¾ sustained, 19 projects (29%) perceived their projects as "Partially" sustained, and two respondents (3%) perceived their projects as "Not at All" sustained.

Project GoalsRespondents were asked about their current goals and whether these goals had changed over the past year. For the majority of projects (73%), goals remain unchanged from the previous year. Eighteen projects (26%) experienced a change in their goals. The most common reasons for goals changing were changes in community needs (n=8) or changes in funding requirements (n=6).

Goals of meeting needs of at risk youth and familiesOne of the original goals of all CYFAR funded projects was to address the needs of at risk youth and families. Respondents were asked to indicate the extent that their project currently meets this goal. As shown in Table 2, only 28% of respondents believe their projects are fully meeting the needs of at risk youth and families. It is noteworthy that respondents reporting that their projects were more fully meeting the needs of at risk youth were also more likely to perceive their projects as sustained***.

Table 1: Project activity four years post USDA/CSREES funding

1 1/2 years post funding

2 1/2 years post funding

4 years post funding

Project Status

Number

Percent

Number

Percent

Number

Percent

Active

81

86%

84

89%

71

76%

Inactive

13

14%

10

11%

23

24%

Table 2. Extent project meets needs of at risk youth and families

Extent project meets needs of at risk youth

Number

Percent

Not at all

0

0%

Somewhat

14

20%

Moderately

35

52%

Fully

19

28%

II. Program Flexibility

This section reports on program flexibility evidenced by programming additions and modifications as well as changes in the number of sites and/or participants.

Program modificationsOverall, 52 respondents (73%) reported adding at least one new program over the past year. The reasons for adding new programs are presented in Table 3.

Table 3: Reasons for the establishment of new programming

Reasons

Number

Percent

Change in community needs

24

46%

Changing participant needs

21

40%

New sites/communities added

15

29%

Required by funding source

15

29%

Change in program goals

5

10%

Among active projects, 52 (73%) of respondents reported modifying their existing programming (expanding some programs, reducing some programs, or both). Reasons for program expansion and/or reduction are reported in Table 4. Changes in community needs, funding, participant needs, and insufficient staff were factors most commonly mentioned as catalysts for modifications in programming efforts

Table 4: Reasons for expansion and reduction

Reasons

Number

Percent

Changing Community Needs

24

43%

Change in Funding

21

38%

Change in Participant Needs

18

32%

Insufficient Staff

16

29%

Changing Program Goals

5

9%

Stable and Adequate Staffing

4

7%

In addition to program expansion and reduction, it was sometimes necessary to eliminate programs. Overall, 32 (45%) active projects reported that at least one program had been eliminated. The most common reasons for program elimination (see Table 5) were insufficient staffing and lack of funding.

Table 5: Reasons for program elimination

Reasons

Number

Percent

Insufficient staff

16

50%

Lack of funding

13

41%

Change in program goals

7

22%

Too few participants

4

13%

Change in community needs

4

13%

Lack of receptivity/support

3

9%

Number and Changes in Sites and ParticipantsRespondents were asked to indicate what kind of changes had taken place in the number of sites and participants of their projects. Most respondents (57%) reported that the projects maintained the same number of sites over the previous year, 32% reported an increase in their number of sites, and 11% reported a decrease in their number of sites. The majority of respondents also reported the number of participants served by their projects were either the same as (45%) or greater than (45%) the number of participants served in the previous year. Only 10% reported a decrease in the number of participants. Increased number of sites and/or participants was significantly related to greater perceived project sustainability*.

Respondents were also asked to indicate why changes in sites or participants occurred. Obtaining new or additional funding (n=13), strong community receptivity and support (n=12), changing community needs (n=7), and program replication (n=6) were identified as reasons for increased sites. The most commonly cited reasons for increased participants was an increase in the number of programs to offer (n=21) or having an increase in the number of sites offering programs (n=12).

III. Staffing Patterns

Respondents were asked to indicate the status of their paid staff and volunteers, whether this had changed from the past year, and if so, the reason(s) for these changes. They were also asked to report on how qualified their staff and volunteers were for the positions they hold within their projects and how much staff and volunteers were involved in suggesting programming ideas.

Paid Staff

Over half (52%) of the respondents reported that they have the same number of paid staff as last year; and 28% reported having more and 20% reported having less than last year.

For those indicating an increase in the number of staff, the most common reasons were increased funding (n=15), increased resources for training, supervision and/or recruiting (n=6), and increased number of collaborators (n=5).

For those indicating a decrease in the number of staff, the most common reason was decreased funding (n=10).

A large proportion of respondents (76%) reported that paid staff were very involved in suggesting new and creative programming ideas.

Volunteers

A majority (68%) of the respondents indicated that the number of volunteer staff for their projects remained the same as last year. Of those experiencing a change in the number of volunteers, 21% reported an increase and 11% reported a decrease.

For those indicating an increase in the number of volunteers, the most common reasons were successful marketing (n=9), increased resources (n=8), increased number of collaborators (n=8), and increased visibility in the community (n=5).

For those indicating a decrease in the number of volunteers, the most common reasons were lack of resources (n=6), lack of marketing (n=3), and decreased number of collaborators (n=2).

The majority of respondents reported that volunteers where either "Very Involved" (39%) or "Somewhat Involved" (49%) in suggesting new and creative programming ideas.

Over two thirds of respondents (69%) reported that, to some degree, their projects used volunteers instead of paid staff for program delivery.

IV. CollaborationThis section provides information on who is collaborating with these Youth at Risk projects, how the community is involved in these projects, the resources collaborators provide, the role of Cooperative Extension in these projects, and leadership patterns for these projects.

When asked to identify the various collaborators involved in their projects, respondents indicated a broad spectrum of participating organizations. These collaborators are listed in Table 6. More than three-quarters of the projects reported schools and Cooperative Extension as collaborators.

Table 6: Types of collaborators

Collaborators

Percent

Schools

89%

Cooperative Extension

80%

Community Service Agencies

64%

Land Grant Universities

59%

Government Agencies

59%

Civic Organizations

44%

Local Colleges

38%

Local Businesses

35%

Religious Institutions

33%

YMCA/YWCA

16%

Boys and Girls Club

13%

Respondents also indicated that over half (52%) of project collaborators remained the same as last year while 40% of respondents reported an increase and 8% reported a decrease in their number of collaborators. For those reporting an increase in the number of collaborators, the most frequently cited reasons were increased community awareness of the project (n=17) and increased emphasis on recruitment (n=15). Those projects that had an increase in the number of collaborators reported significantly greater levels of perceived sustainability*.

Community Involvement and ParticipationAs previously indicated in Table 6, a variety of different collaborators continue to be involved in these projects. Respondents were asked to indicate the degree of involvement and resource provision by these collaborators. Community collaborators took on a leadership role for the majority (79%) of the projects while serving in an advisory or resource capacity for 19% of the projects (see Figure 2). All but one project had at least some involvement by the community (outside of CES). Resources provided by these community collaborators are shown in Table 7. Overall, when community collaborators had a more prominent role in project leadership, they were also significantly more likely to be involved in the provision of training and workshops, the provision of personnel, supervision of personnel, and program implementation*.

Table 7: Resources provided by collaborators

Resources provided

Projects receiving resources from community collaborators

Provision of Materials/Equipment

97%

Program Implementation

93%

Provision of Space

92%

Provision of Personnel

86%

In-Kind Support

86%

Involved in a Coalition

85%

Evaluation of Programs

85%

Curriculum Development

82%

Provides Training/Workshops

81%

Advisory Board Membership

78%

Direct Cash Funding

77%

Supervision of Staff/Volunteers

75%

Fiscal Management

64%

Grant Proposal Writing

63%

In addition to community collaborators, program participants and the parents of program participants have also been involved in planning, implementation, and to a lesser extent, administration of these projects. Table 8 demonstrates the involvement of youth participants and their parents in these projects.

Cooperative Extension

Cooperative Extension, whether at the state, county, or local level, continues to be actively involved in most projects. Cooperative Extension provides leadership to over half (58%) of the active projects while serving in an advisory or resource capacity to 25% of the projects (see Figure 3). The remaining 17% of respondents indicated that Cooperative Extension was no longer involved in their project. As indicated in Table 9, Cooperative Extension is involved in a wide range of project management and technical assistance activities. Overall, when Cooperative Extension provides leadership to projects, they are significantly more likely to provide all of those resources listed in Table 9*.

Table 8: Community role in projects

Community group

Program Planning

Program Implementation

Program Administration

Youth Participants

84%

87%

17%

Parents of Participants

73%

70%

28%

Table 9: Resources Provided by Cooperative Extension

Resources provided

Projects receiving resources from Cooperative Extension

Provides Training/Workshops

75%

Provision of Materials/Equipment

72%

Evaluation of Programs

68%

Curriculum Development

67%

In-Kind Support

65%

Involved in a Coalition

63%

Grant Proposal Writing

63%

Program Implementation

62%

Advisory Board Membership

57%

Supervision of Staff/Volunteers

51%

Provision of Personnel

51%

Fiscal Management

50%

Provision of Space

40%

Direct Cash Funding

37%

For those respondents that identified themselves as Cooperative Extension agents (n=44), the vast majority (92%) agreed that working on their Youth at Risk project was a part of their ongoing work responsibilities. Furthermore, 60% of these respondents "Strongly Agreed" and 32% "Agreed" that serving at risk youth is an expectation of Cooperative Extension in their county and state.

Project Leadership

Projects were almost evenly divided between being led by a single agency or organization (52%), or having shared leadership by two or more entities (48%). It is important to remember, however, that although an agency, coalition, or Cooperative Extension may have taken primary responsibility for project leadership, they did not manage the project alone. In all but one, project community collaborators appeared to contribute to project longevity.

V. Funding

Funding PlansRespondents were asked to determine whether or not they have enough funding to maintain their projects over the next one, two, and three years. The majority (81%) of projects stated they had enough funding to sustain programming through the upcoming year while 52% had enough funding for the next two years, and 37% had enough funding in place for the next three years. Furthermore, 77% of respondents reported that were actively working on obtaining new funding. Projects that reported they had enough funding to sustain their projects up to three years were significantly more likely to perceive their project as sustained ***.

Funding SourcesWhen respondents were asked to identify the sources of cash funding they received (see Table 10), the majority (77%) reported receiving grants to fund their projects, while almost half (46%) reported receiving user fees. In addition, Cooperative Extension, and local businesses were prominent sources of funding for these projects.

Table 10: Sources of Hard Cash Funding

Source of Funding Support

Number

Percent

Grants

52

77%

User Fees

31

46%

Cooperative Extension

26

41%

Local Businesses

19

30%

Schools

18

28%

Foundation Funding

18

27%

Fundraising

18

27%

Contracts

14

21%

United Way

10

15%

Religious Institutions

10

15%

VI. Demonstrating Impact

This section provides information on current evaluation efforts of the projects, who conducts these evaluations, and how evaluation results are used.

Current Evaluation EffortsThe majority (88%) of active projects have been conducting some type of evaluation, including participant/parent/staff satisfaction, informal assessments, and/or change in knowledge or personal indicators (see Table 11). These efforts were most often conducted by project staff, including directors and coordinators, and Cooperative Extension personnel (see Table 12). When Cooperative Extension was involved in evaluation, the projects were significantly more likely to assess knowledge gained through pre and post- testing*. Even though the majority of projects reported conducting some type of evaluation, only 8 projects (12%) indicated that they had funding specifically targeted towards these efforts.

Informal assessment through testimonials or other forms of communication

41

62%

Assessments of knowledge gained by participants through pre- and post-testing

34

52%

Assessments of personal indicators, such as well-being, self esteem, or school achievement

31

47%

Table 12: Who Conducts Project Evaluations

Evaluators

Number

Percent

Project Staff

33

56%

Project Coordinator/Director

29

49%

Cooperative Extension Personnel

29

49%

Collaborators

17

29%

Outside Consultants

12

20%

Uses for Project EvaluationAs indicated in Table 13, evaluation results were used in a variety of ways. Over three-fourths of respondents indicated that evaluation results were used to inform programming and report writing.

Table 13: Evaluation Uses

Uses for Evaluation

Number

Percent

To assess program effectiveness

53

88%

Program modification

48

80%

Program planning

47

78%

Reports to collaborators and funders

47

78%

Documentation for grant proposals

37

62%

Market to community

35

58%

Market to potential funders

32

53%

Support program replication

25

42%

Additionally, 45 projects (63%) reported using the results of needs assessments for documentation of program efforts for grant proposal writing (n=30), modification of current programming (n=29), and replication of programs (n=14).

VII. Obstacles to Project Survival

Respondents were asked to identify obstacles faced by their projects. Only 7 (11%) of the 64 projects responding to this item reported no obstacles; the remaining 57 projects (89%) identified one or more obstacles. Obstacles reported by at least a third of the projects are listed in Table 14. Projects that reported they had enough funding to sustain their projects up to three years were less likely to report funding as an a obstacle to their project's sustainability***.

Table 14: Commonly Reported Obstacles

Funding Obstacles

Number

Percent

Lack of funding for adequate staffing

34

50%

Lack of funding for program expansion

34

50%

Lack of predictable funding

33

49%

Lack of funding for current programming

29

43%

Lack of funding for qualified staff

24

35%

Community Obstacles

Lack of parent involvement

30

44%

Community's economy won't support project funding needs

28

41%

Competition for local resources

26

38%

Staffing Obstacles

Staff turnover

27

40%

Cooperative Extension staff overload

24

35%

Program staff overload

23

34%

Summary and Next Steps

The three general research questions focused on project status, the role of Cooperative Extension and community collaborators, and facilitators and obstacles to project sustainability. Overall, the findings from this report revealed that the vast majority of projects continue to provide services to at risk youth and their families, that the support of Cooperative Extension and community collaborators is substantial, and that there are a number of means to facilitate sustainability. In addition, the major obstacles to sustaining projects revolve around funding, community resources, and staffing.

Current Status

Over three-fourths (76%) of the originally funded YAR projects remain active four years post USDA funding. The vast majority of respondents from these active projects believed they were at least "Somewhat" able to meet the needs of at risk youth. The majority of the projects experienced some program, participant, or staffing change over the past year, with program expansion and program reduction being the most common forms of change. Furthermore, the establishment of new programming was more commonly reported than the elimination of programs. Due in part to program expansion, almost half of the respondents reported an increase in the number of participants they serve.

Overall, the remaining active YAR projects appear to be either maintaining or expanding their level of activity. The most prominent reasons for program expansion were changes in community needs, participants' needs, and increases in funding. In addition, community receptivity and support were cited as reasons for increasing the number of sites and participants. For the respondents that reported program reduction and program elimination, the most prominent reasons were lack of funding and consequently insufficient staff.

The vast majority of respondents reported that they are conducting some type of program evaluation, with informal assessment and the evaluation of participant satisfaction being more commonly utilized than formal pre- and post-testing. Evaluation data was most commonly used to assess program effectiveness, to modify current programming, to plan future programs, and to report to funders and collaborators.

Collaborator Involvement

The data suggests that collaboration has been essential to the continuation of these YAR projects. The most prominent collaborators were Cooperative Extension, schools, community service agencies, land grant universities, and government agencies. Cooperative Extension continues to serve in a leadership capacity for over half of the active projects. When Cooperative Extension has a leadership role in the project, they tend to be involved in a wide range of program activities (e.g., providing training and workshops, providing materials and equipment, program evaluation, and curriculum development). In addition to Cooperative Extension, other community agencies also retain a leadership role in the projects. In these cases, community collaborators tended to be more involved in providing training and workshops, supervision and provision of personnel, and program implementation. Overall, both Cooperative Extension and community collaborators continue to contribute to these projects, making collaboration a likely facilitator of project success and continuance.

Sustainability

The primary goal of this research was to examine the sustainability of YAR projects after the ending of USDA/CSREES CYFAR funding. While a number of obstacles (lack of funding, staffing difficulties, lack of parent involvement, and limited community resources) remain problematic for many of these YAR projects, it is impressive that so many have been able to continue and expand. Several factors were significantly related to respondents' greater perceptions of program sustainability, including: 1) increasing program sites and participants, 2) increasing collaborator involvement, and 3) securing funding for a known period of time. Interestingly, respondents reporting greater perceived sustainability were more likely to report that their projects were fully meeting the needs of at risk youth and families. This is an encouraging finding in that the primary purpose of the YAR project was to create and sustain programming for at risk youth and families.

Next Steps in the Sustainability Research

This is the latest report in a series of analyses focused on program sustainability as it pertains to community-based Youth at Risk projects funded from 1991-1998 by the CYFAR Initiative. Over the course of this research program we have monitored the ebb and flow of projects and their program components. Our understanding of these projects has been incremental, and each data collection period has provided greater depth in the nuances of program sustainability. There are several avenues that our future work will take in this area. First, there is the matter of developing an instrument or set of measures that can be easily and efficiently used by community-based programs to assess their own sustainability. We have field tested a first version of such an instrument and will continue to refine it. Second, a parallel task is to continue to refine our sustainability framework. With the completion of this most recent data collection period we are at a place where the data can better inform the framework (at earlier phases in our research program, the conceptual framework tended to inform the research); at this point the framework and the data provide an important dialectical setting for improving both concept and measurement. Third, consistent with our applied orientation, we will translate the theory and the research that has emerged from this research program into program planning tools for community-based programs. These will complement the sustainability assessment measure. Data will be collected from these same YAR projects at the point six years post USDA/CSREES funding and will be reported at a future date. In addition, sustainability research is in progress on those projects funded through the USDA/CSREES State Strengthening funding.

The Authors

Lydia I. Marek is a Research Scientist in the Department of Human Development at Virginia Tech. She received her Ph.D. in Marriage and Family Therapy from Virginia Tech and is a Certified Family Life Educator (National Council on Family Relations).

Jay A. Mancini is a Professor in the Department of Human Development at Virginia Tech and a Human Development Extension Specialist with Virginia Cooperative Extension. He received his Ph.D. from the University of North Carolina at Greensboro.

G. Erik Earthman was a Research Associate in the Department of Human Development at Virginia Tech. He is a doctoral candidate in Human Development and Family Studies at the University of Arizona.

Donna-Jean P. Brock is a Research Associate in the Department of Human Development at Virginia Tech. She received her M.A. degree in psychology from Wake Forest University.

Appendix A: 94 Youth At Risk Programs Funded Between 1991-1998

4 years post funding

State

Project

Leadership

Collaborators

Alabama

Assess and Address

Inactive

Inactive

Alaska

Kuskokwim 4-H Fisheries

Shared

Land Grant Universities, Local Colleges, CES, Community Service Agencies, Schools, Civic Organizations, Government Agencies, Local Businesses

Land Grant Universities, Religious Institutions, Community Service Agencies, Schools, Civic Organizations, Government Agencies

Texas

Making the Grade

Single

Local Colleges, YMCA/YWCA, Community Service Agencies, Schools, Civic Organization, Government Agencies, Local Businesses

Texas

Rutabaga

Inactive

Inactive

Utah

CARES

Shared

Local Colleges, CES, Religious Institutions, Community Service Agencies, Schools, Civic Organizations, Local Businesses

Vermont

ECA

Unknown

Land Grant Universities, Local Colleges, CES, Community Service Agencies, Schools, Government Agencies, Civic Organizations, Local Businesses

Virginia

Making the Grade

Shared

Land Grant Universities, Local Colleges, CES, Community Service Agencies, Schools, Civic Organizations, Local Businesses

Virginia

Partnership for Excellence in Education

Shared

Local Colleges, CES, Religious Institutions, Community Service Agencies, Schools, Civic Organizations, Government Agencies, Local Businesses

Virginia

Strong Families

Inactive

Inactive

Washington

ONTU

Shared

CES, Religious Institutions, Community Service Agencies, Schools, Civic Organizations, Government Agencies, Local Businesses

Washington

Spokane Family Focus

Shared

Land Grant Universities, Local Colleges, CES, BGCA, Religious Institutions, Community Service Agencies, Schools, Civic Organizations, Government Agencies, Local Businesses

Washington

STAR Youth

Shared

Land Grant Universities, Local Colleges, CES, BGCA, Religious Institutions, Community Service Agencies, Schools, Government Agencies

West Virginia

Charleston After School Program

Shared

Land Grant Universities, Local Colleges, CES, Community Service Agencies, Schools, Government Agencies

Wisconsin

Youth Futures

Single

Unknown

Wyoming

HICAP

Inactive

Inactive

Wyoming

Wind River

Shared

Land Grant Universities, CES, BGCA, Community Service Agencies, Schools, Government Agencies

Reviewed by Novella Ruffin, Extension Specialist, Virginia State University

Virginia Cooperative Extension materials are available for public use, reprint, or citation without further permission, provided the use includes credit to the author and to Virginia Cooperative Extension, Virginia Tech, and Virginia State University.