The Role Of Evaluation In Prevention Of College Student Drinking Problems

I. That Sounds Like a Lot of Trouble… Why Do It?

A. Increase likelihood of effectivenessboth local and in general

The first, if obvious, answer is that successful evaluation will increase the likelihood of
program effectiveness, both in a specific setting and more generally. Some have argued that the
exercise of merely defining “effectiveness” would improve many interventions that are now too
diffuse to be effective. Apart from having specific evaluation results on one or another
prevention strategy, having access to multiple evaluations would also improve the more general
prevention approach, as instances of effectiveness and ineffectiveness begin to accumulate
evidence of program generalizability and sensitivity to the wider context of application.

B. Cycle of improvement blurs distinction between outcome evaluation and formative research

Ideally, evaluation should be a continuous “feedback” mechanism to program managers. Too often,
programs that have shown promise under highly-controlled experimental conditions fail to achieve
that same effect when implemented in the “real world.” This loss of impact may be due to the
absence of a halo effect that arose in the experimental setting, or the loss of a dynamic leader
from the original implementation, or perhaps from the difficulty inherent in mounting a scaled up
version of the program. Even modest evaluation data and analysis may show when and where a program
is failing to meet its objectives, and focus on ways to reshape the intervention. Thus, evaluation
evolves from a “one-shot” effort to a continuous cycle of improvement.

C. Encourages strategy over activity

Unfortunately, many prevention programs are adopted from a desire to do “something” about a
problem. If the design or selection of an intervention is closely tied to an evaluation design,
there is a greater chance that overt strategies for prevention will be articulated and discussed
rather than the prevention activities per se. This provides a good counter-balance to the impulse
to choose a program based on more superficial qualities (e.g., slick brochures, “fun” activities,
etc.).

D. Counterbalance to adopting “high visibility” or popular programs or desire to blindly “do
something”

The absence of good empirical data on program effectiveness leaves program managers and
administrators with little basis for selecting an intervention other than what some other campus
might be doing. Once a program reaches critical mass, there is even greater temptation to assume
that it must be effective.

E. Discourages complacency

A recurring theme in discussions of evaluation is that it can sharpen the focus on program or
intervention strategy, goals, and objectives. Novice evaluators are often surprised to discover
how often programs, even long-established ones, have poorly articulated objectives and are vague
about how the desired goals are supposed to be achieved. In addition, even well-articulated
programs may not have been put to the “test” of evaluation. A thorough evaluation design will
address not only the expected outcomes of the intervention, but each link of the chain of
intermediate effects that are hypothesized to influence the endpoints.

F. Maximizes resources (almost an ethical responsibility to make best use of funds)

In an era when public funding of any kind is becoming increasingly scarce, it behooves anyone in
the field of prevention to demonstrate the effectiveness of their interventions. Going further,
more funding sources are expecting to see a benefit-cost analysis that can put a dollar value on
the program activities.

G. Enhances overall program credibility

Many have complained that prevention specialists are given little resources and have low
visibility within the institutions they are expected to serve. This marginalization of prevention
will only get worse until and unless those specialists can provide evidence of their benefit to
the organization. Solid evaluation enhances not only the credibility of a specific intervention
but the credibility of prevention efforts at large. More resources will be committed to prevention
when it can be shown to work.