Reasons for built-in evaluation

In the interest of brevity, we will concentrate on delineating just four of
the major reasons for incorporating evaluation activities as a regular activity
in a nutrition intervention.

Programme Operation Improved

The primary reason for incorporating evaluation activities into a nutrition
programme is that the knowledge derived will help programme managers improve the
quality of their intervention. At the most basic level, the monitoring of the
service delivery system will help sharpen the implementation of the
intervention. For example, careful monitoring of the stocks and flows of
programme inputs may facilitate rationalization of the flow of supplies from
warehouses to project sites. This will help avoid losses due to spoilage,
contamination, or deterioration that result when commodities and supplies are
overstocked at the community level.

At a higher level, evaluation results provide a basis and incentive for
programme redesign. It is our belief that the long-term duration of most
interventions guarantees that the environment surrounding each intervention must
change. Therefore, the "best" programme design must change too. For
example, a poor harvest might reduce food availability, raise prices, and
decrease food intake by the poor. The resulting deterioration in nutritional
status would signal the need for a temporary increase in ration size in a
supplementary feeding programme. Furthermore, components of an intervention may
meet with success and, as a consequence, no longer be needed. For example, a
nutrition education programme, geared at inducing mothers to use oral
rehydration techniques at the first sign of diarrhoeal disease may work so well
that attention in nutrition education classes should shift to some other
activity, for example, boiling water or home gardening. Responding to changing
circumstances, as described in these examples, is contingent upon: (a) noting
that a change in nutritional status has taken place, and (b) involving
functionaries in the determination of why the change occurred and in the
formulation of an appropriate response.

Built-in evaluation further facilitates the use of data to improve programme
operations for two good reasons: (a) the data will be available in a more timely
fashion than data collected in any other way, and (b) the aura of participation
surrounding built-in evaluation at all levels will increase the receptivity of
site managers, their supervisors, and even the participants themselves, to
modifications in programme activities suggested by the findings of the system.
It is far less likely that signals generated by a built-in, self-run evaluation
system will be discarded or dismissed as incorrect.

Data Quality Improved

By building evaluation functions explicitly into an intervention, programme
designers provide a genuine incentive for field workers to collect and record
accurate data. All too often, programmes are initiated with a set of forms to be
filled out in the field and, in some cases, transmitted to some central office.
Field workers rarely understand the necessity and purpose of the forms. More
often than not, forms, especially those with anthropometric data, simply clutter
the health or feeding center and remain unused. Even when forms are shipped to
the central office, field workers soon come to realize that no response is
forthcoming. Data collection appears to them to be a futile and cumbersome
activity, and all motivation for filling out forms accurately is lost. We have
encountered field personnel who merely copy last month's form rather than
prepare a new one because they perceive that the forms are useless.

However, if a programme is initiated with a set of forms for collecting
limited quantities of data and these are used actively for management at the
local level, field workers can perceive an immediate purpose in their efforts.
When the data are aggregated at higher levels, with feedback given to the
field-level functionaries, there is an even stronger motivation for collecting
data properly.

A by-product of generating feedback at higher levels of management is the
rapid identification of poorly collected and/or falsified data. The review of
trends emanating from a single location by a skilled manager is the best
protection against spurious or incorrect data. It is immediately apparent, upon
review of longitudinal trends, where the data system is breaking down: any place
with inordinately large or surprisingly little change has, in all probability, a
worker not processing the data correctly.

Quantity of Data Increased

If one hundred people spend fifteen minutes each day collecting data, four
people would have to spend over six hours each to collect the same amount of
data, assuming that all of the data can be collected at the same location. By
incorporating data collection as a routine part of an intervention, the
collection procedure is spread out over a far larger number of people and
places. Thus, the opportunity for generating additional data, more variables, or
more cases on the same variables, goes up dramatically.

Also, by sharing the burden of data collection, the cost goes down. Initial
costs, the training of so many workers, are high, but recurring costs are
minimal. The task becomes another function performed in the field by people
already there, often at no additional cost.

The opportunity to generate longitudinal sequences of data on individuals
through a built-in evaluation system offers another substantial benefit. The
measurement techniques available in the field for ascertaining nutritional
status are inherently weak. Also, nutritional status is a dynamic condition that
can change rapidly in the face of adversity or improved circumstances. The
chance to review the velocity of growth in individuals, in reference to a
standard, is of considerable importance and becomes possible through an internal
data system.

Contextual Information Provided

Having played the role of outside evaluator several times, we have become
highly sensitive to our inability to interpret locally-generated data because of
our lack of knowledge of the local context. To illustrate: during a recent
evaluation of a supplementary feeding programme in Sri Lanka, we encountered an
unanticipated result. The nutritional status of preschoolers on the tea
plantations, generally assumed to suffer the most severe malnutrition, was
better than in the urban, rural, or suburban areas canvassed during our
activities. A visit to the tea plantation from which over half the cases in the
sample were drawn revealed that this plantation had the model health facility
for plantations in the country. (The plantation manager was highly talented and
truly concerned with health issues.) Moreover, the medical practitioner in
charge noted that the infant mortality rate, even at this model clinic, was
abnormally high, because the most malnourished infants were not surviving. Had
we not learned of these quirks in our sample, we would have had to challenge the
accepted notion that malnutrition was more prevalent on the tea estates. We
would have been wrong.

Understanding the local context is critical for proper interpretation of
locally-generated data. Even when statistically valid sampling techniques are
used to generate programme-wide samples, contextual information knowledge of
programme selection procedures, economic trends, impact of parallel programmes,
and so forth is needed for proper interpretation. Observation of the local
context is heightened when analysis begins at the local level, as it must be
with a built-in evaluation
system.