Part 3 of a Commentary
on the CIA's Remote Viewing Report
by Paul H. Smith

[Note: This is the third part of my review of the CIA- sponsored report
by the American Institutes of Research of its evaluation of the
Government's twenty-four year long remote viewing program. Part One,
"Bologna on Wry," covered the operational intelligence portion of
the program. Part Two, "A Second Helping of Bologna on Wry,"
found that the research reviewed by the AIR was inadequate as a
basis for a fair assessment of remote viewing. Part III examines the
AIR's faulty evaluation of that research.]

PART III

If one is limited only to the information contained in the AIR
report, one forms the impression that the evaluators did a reasonably
thorough job in assessing the SAIC/SRI experiments and analyzing
the results. The ambiguous conclusions (that there is an anomaly,
but after 20+ years of research it is still a tentative one, and
no cause and effect has yet been demonstrated) leads surely to the AIR
conclusion-of-choice that it really doesn't make sense for the
government to waste further money on it. But we would be misled.
The AIR examination was neither in depth, nor conclusive. AIR
employees themselves focused mostly on their rather cursory evaluation
of the intelligence operations part of the STAR GATE program.
Though some of them were involved as well with evaluating the
remote viewing research program, they contributed little but a brief
concluding summation to the final AIR report. Drs. Utts and Hyman,
specially engaged by AIR to review the research program, produced
by far the bulk of that assessment. Utts' is first sequentially
in the report. She starts with a general discussion of the
statistical theory used to gauge experimental success in parapsychology
research. She follows this with an instructive discussion about
RV experimental design, some history of RV research, and an
exploration of the SAIC experiments, augmented by more detailed
information in an appendix. She also discusses briefly how the results
correlate with earlier work done at SRI (they are consistent with these
earlier statistically-significant experiments), and also lists
the results of a number of related remote viewing and ganzfeld (a
form of remote viewing) experiments conducted at various labs
around the world. According to Utts, the effects of these
strongly correlate with those achieved in the SAIC remote viewing
experiments.

In the course of her remarks she anticipates and answers many of
the objections Hyman later brings up in his portion of the review. Even
allowing for my own personal bias in favor of her conclusions, I
find her assessment to be more rational, well- reasoned, and
soundly supported than is that of Hyman.

On the other hand, so general are Hyman's comments that he could
handily have written most of his evaluation without ever once having to
refer to the remote viewing experiments themselves. Ultimately,
he acknowledges that there are significant effects demonstrated,
but then spends a good deal of time discussing why in principle
he rejects these effects. He admits that he can find no flaws in
the experiments, yet says we must wait indefinitely to decide whether
they have or have not proved a psi effect so as to allow a
lengthy interval for thus- far unidentified flaws to be ferreted
out. He warns that given enough time, methodological flaws might
turn up that had not yet dawned on anyone. He then cites as his only
examples of such methodological flaws two cases that are
decades-old and unrelated to remote viewing, where the only
"flaws" uncovered were instances of fraud. Meanwhile, Utts has
already pointed out that fraud as an explanation is untenable because
of the numbers of institutions in diverse locations around the
globe that have produced results equally significant as those of
the SAIC experiments.

Utts later addresses and disposes of a number of Hyman's other
arguments and errors in her rebuttal that follows Hyman's comments in
the report. However, there were several other "literary offenses"
that Hyman or AIR or both commit that are not discussed. Since
Hyman's evaluation is at the heart of the AIR case against the
remote viewing research program, I will focus my attention there.
In the interests of space--which I consume ever more of as this review
progresses--I will only consider a few of the more egregious
errors and misjudgments the good doctor makes.

THE BABY OUT WITH THE BATH

To begin with, Hyman and AIR ignored twenty years of research conducted
prior to the SAIC experiments. Despite the AIR's express assignment to
thoroughly review "all laboratory experiments and meta-analytic
reviews conducted as part of the research program," ultimately
only ten experiments were actually reviewed--all of them
performed at SAIC in just the last three or four years of the
government's program. One reason for this was likely due, as
Hyman says, to the "limited time frame [that was] allotted for
this evaluation" [p. 3-43, 3-44]. The AIR reviewers were given
only a month and a half--from mid-July to the end of August--to
conduct a supposedly "exhaustive" review.

Ed May asserts in his own rebuttal to the AIR report (Journal of Scientific Exploration,
vol. 10, no. 1, Spring 1996) that in recognition of this
unrealistically short time allotment, someone at AIR requested May
provide only the reports from his ten best experiments for
evaluation. Quite properly he demurred, since for sound
scientific reasons this would skew the results-- in so doing,
only successful results would be considered, when to form a fair
picture required that poor results should be evaluated as well
(selecting only experimental results that show positive effects
is known as the "file drawer" effect). As an alternative, May
proposed a different procedure that would have allowed examination of
all the materials within the time constraints, resulting in a
much more thorough and reliable assessment. His suggestion was
ignored. Instead, in a conference call between the AIR
evaluators, Hyman got agreement that only the ten latest experiments
would be evaluated. It was tacitly recognized that there were
both relevant and irrelevant experiments among these ten, but it
made for a more manageable evaluation pool, and it avoided the
"file drawer" problem.

This is where it gets interesting. As earlier noted, Hyman
explains that a limited number of experiments were selected
because of lack of time to consider all of those available, and
these ten were the most recent. But he also cavalierly dismisses
the need to examine the other two decades worth of experiments by
alleging that the handful of SAIC experiments selected were "the
only ones for which we have adequate documentation" (p. 3-43).
Earlier research was discounted as suffering "from methodological
inadequacies" upon which he chooses not to elaborate further in
his report. Hyman makes this amazing assertion despite the fact
that he had never even looked at the documents of which he is being so
dismissive. Sometime back in the mid 1980s, he reportedly saw
some of the results from the first few years of SRI experiments
when he participated in another flawed "scientific" evaluation of
enhanced human performance programs [i.e., the National Research
Council's somewhat infamous "Enhancing Human Performance" report].

Still, there remained perhaps ten years' worth of subsequent
remote viewing research conducted at SRI and elsewhere to which
Hyman had never previously had access. It, along with the ten SAIC
experiments, had been classified Secret or higher until the CIA
decided to make it all available in support of the AIR study.

Because of the CIA's declassification action, Hyman finally WAS
authorized access to the majority of the research, had he chosen to
examine it. However, he himself admits he never bothered, since most of
the experiments prior to the SAIC era were in the "three large
cartons of documents" he was given at the outset of the study but which
he freely admits in a recent article he "didn't have time" to
look into (Skeptical Inquirer,
March/April 1996, p. 22). In short, he couldn't possibly have
known whether those experiments really did suffer from
"methodological inadequacies."

Still, Dr. Hyman couches his remarks in such a way as to make an
unsuspecting reader suppose that the ten experiments reviewed were the
best examples available. Though he clearly knew better, he
nevertheless claims in the Skeptical Inquirer article that the
ten experiments he and Dr. Utts evaluated were the "ten best
studies," and "the best [RV] laboratory studies" (p. 22),
implying by assumption that they must therefore be sufficient on
which to base an adequate assessment of remote viewing. This
despite the fact previously explored in Part II of this review
that a number of the SAIC experiments had little or nothing to do
with remote viewing, and that the remainder were generally not fully
state-of-the-art RV experiments.

Nonetheless, a mere two pages after telling us that he and his
AIR fellows themselves arbitrarily decided that only ten experiments
would be reviewed, he proceeds to deplore the entire
two-and-a-half decades of research for producing "only ten adequate
experiments for consideration." Hyman writes:

"Unfortunately, ten experiments. . .is far too few to establish
reliable relationships in almost any area of inquiry. In the
traditionally elusive quest for psi, ten experiments from one
laboratory promise very little in the way of useful conclusions."
(3-46)

He is, of course, absolutely right in the process of being
altogether wrong.

PRIMA FACIE EVIDENCE

The arbitrarily limited data base is not the only difficulty with
AIR's study. Perhaps more problematic is Hyman's arbitrary exclusion of
so-called "prima facie" evidence (3-71). This is introduced in
the section where Hyman (without, I might add, any qualifications
whatsoever in the field of intelligence) considers whether RV has
potential for use in operational intelligence settings. Though in this
part of his discussion he is concerned with practical
applications, he seems to have carried over this bias against
prima facie evidence from his treatment of the research program
itself. Hyman says that he relies on a definition of prima facie
evidence that originated with May and Utts. In her remarks (3- 11),
Utts describes prima facie RV evidence as a remote viewing result
that is so spectacularly accurate that it virtually proves the
existence of the phenomenon, though it is beyond the ability of
statistics to describe. This meaning is derived from jurisprudence
definitions of prima facie evidence as that evidence which
clearly proves a fact, if there can be no other explanations for
what has occurred. Prima facie evidence of remote viewing would be
unambiguous information produced by a viewer about a target that could
not have been obtained in any other way (i.e., fraud, leaky
methodology, etc.). This might be in the form of sketches or
verbal responses or both. If the target were, for example, the
Eiffel Tower, the sketches and/or verbal descriptions would
strikingly match the Eiffel Tower.

There was apparently no specific "prima facie" proof in the ten
SAIC experiments (though a couple of the RV sessions appear to have
come close), so Hyman's embargo of such evidence would seem not
to matter much. However, despite his remarks to the contrary, he
doesn't seem to be working from the same definition of prima
facie evidence to which Utts and May subscribe. Hyman doesn't
elaborate further as to what his personal understanding of the term is,
but from the context it seems apparent that he means to exclude
all evidence that cannot be statistically evaluated. If someone
designated as judge must look at an RV result, compare it to a
target, then come to a conclusion based on his/her own opinion as
to whether or not it matches, that evidence is unacceptable because it
is based on a subjective judgement.

One of the most time-honored evaluation methods in remote viewing
research is to provide the judge with the same set of targets used to
task the remote viewers, then allow the judge to "blind match" the
remote viewer's results against all the possible targets in that pool.
Since the judge thus has no idea what the original target was
except that it had been selected from the available target pool,
the belief is that the better the RV session, the more likely is the
judge to correctly match the viewer's results to the actual
target. How many times the judge successfully matches a session
to its correct target is then quantified with statistics. It's
obvious that this is only one step removed from subjective
judgement. But it allows the RV data to be turned into numbers,
which can then be more easily manipulated.

This procedure works so long as there is a reasonably limited
target pool. However, if the target pool is infinite-- i.e.,
could be any site, person, object, or event in the entire world
(as is the case in intelligence operations)--it is virtually
impossible for a judge to be able to match an RV session
transcript to a given target based only on internal information.
If the viewer says the site is the Eiffel Tower, the judge must
evaluate the session data, and if it matches the Eiffel Tower,
he/she must go with that conclusion. Success or failure cannot be
statistically determined in such a situation. Either the viewer
accurately and unmistakably describes the site, or he/she doesn't.
Let's say in the case of the "Eiffel Tower" session that the site was
actually a missile launch gantry at Vandenberg AFB. Let's say
further that the viewer's data was all extremely accurate in
describing the gantry, but that the girder lattice- work, the strong
vertical orientation, and the metallic construction caused the
viewer to subjectively interpret the site as the Eiffel Tower. In
a blind-judging situation with an infinite target pool, this
session would be judged as a miss.

Obviously, it was not a miss. The data was accurate, but the
viewer's subjective interpretation was wrong. It is clear that
another option for judging the accuracy of such a session is
necessary. The only alternative that I know of would allow the judge to
concurrently compare the actual target information with the
session data the remote viewer produced to see how close the RV data
matches the actual site. Of course, the judge is no longer
"blind," so this becomes an exercise in subjective judgement, and
would therefore be rejected out of hand according to Hyman's
criteria. Certainly, there are potential problems with subjective
evaluations of this nature. If the data is somewhat ambiguous-- that
is, the elements contained in the feedback potentially match
several targets--then the human tendency might be for the judge
to think he/she sees the target in the data, even though the data
itself isn't accurate enough for a truly objective match. But with
"prima facie" evidence, we are not talking about these ambiguous
cases, but rather a target and transcript that match
unambiguously. Any competent person would recognize that the
target folder and the remote viewing data describe the same target.
Ray Hyman would, unfortunately, exclude this as evidence.

As justification for this rejection Hyman cites a study done by
David Marks and Richard Kamman in 1981 that purports to prove that a
psychological phenomenon they call "subjective validation" was
responsible for good results shown by early SRI remote viewing
experiments. Essentially, Marks and Kamman maintain that a judge
may see what s/he wants to see in evaluating any given remote
viewing session, since viewers often describe a variety of
elements that might be found in more than one target. However,
this study centered around blind judging of targets from a
limited target pool, some targets of which shared characteristics with
other targets in the series.

This does not hold water in relation to the definition that Utts
and May had in mind when referring to prima facie evidence. A true
"prima facie" session is not ambiguous. There is NO DOUBT that
the correct target has been addressed and described, and any reasonable
person would be able to make that same judgement.

In effect, Hyman rejects the use of any sketches or other visual
data that must be subjectively compared to the target to determine
whether there is correspondence or not. If the viewer is targeted
(in the blind, of course) against the Eiffel Tower, and during the
course of the session draws unmistakably the Eiffel Tower, it is
by Hyman's standards still inadmissible as evidence of remote
viewing. What Hyman and his colleagues seem to be saying is that
even if it looks like a duck, walks like a duck, quacks like a
duck, and floats like a duck, we must assume that it's NOT a duck
until we have something more convincing.

The irony is that if Hyman's strictures were applied to
conventional science, numerous branches of study that rely on
subjective comparisons between one thing and another would dry up and
blow away--among these, plant and animal taxonomy, paleontology,
and comparative biology.

LOST IN THE NUMBERS, OR "STATISTICS AIN'T EVERYTHING!"

Early in his remarks Hyman alleges that "Parapsycholo[gy] is
unique among the sciences in relying solely on significant departures
from a chance baseline to establish the presence of its alleged
phenomenon" (p. 3-51). In other words, parapsychology is the only
science that has to prove itself by showing that something
consistently happens more often than you would expect by accident.

Hyman is generally right in saying this about statistical proof
as far as psychokinesis (PK) research is concerned--no one has yet
demonstrated under scientific conditions the moving of lamps or pianos
through the air using "mental" power alone. Indeed, most PK
research involves microeffects that only manifest themselves as
statistical deviations from the chance baseline to which Hyman refers.
One of SAIC's experiments--the computer- driven binary-choice
experiment--falls into this "deviation from chance" category.

Hyman is wrong, however, in claiming that remote viewing
(obviously a parapsychological effect) is provable only by a
statistical deviation from chance. Valid remote viewing produces
true "macro" effects in the form of word descriptions, drawings,
sketches, etc., that provide information directly applicable to
the real world. The statistics involved in evaluating RV research are
really only an imperfect, after-the-fact attempt to measure how
well remote viewing works in a given experiment. The statistical
analysis also serves the goal of limiting the subjective judging
mistakes to which humans are vulnerable in ambiguous situations.

But the statistical evaluations are not the proof. The proof is
the information provided during the session that could not possibly
have been obtained through any other known means of
communication. Statistics can be extremely useful as an evaluative
tool, but relying too much on them can also be dangerous. It is too
easy to get lost in the numbers and lose sight of what they represent.

In theoretical terms, it only takes a single successful remote
viewing session to prove once and for all the existence of the
phenomenon. If a viewer in isolation provides accurate data about
a target, and if ALL other means by which the information could
have been obtained can be ruled out--to include both fraud and chance,
no matter how unlikely--the only possible conclusion left must be
something beyond our current understanding of the physical
universe: in other words "paranormal." We do not, however, live
in a perfect world. First, there is always a possibility that
through some incredible hiccup of fate the viewer might by accident hit
on the correct target. Second, in the real world theoretical
perfection in experimental design is approachable but ultimately
unreachable; we often cannot conclusively rule out every
explanation besides psi for the effects of a given experiment, the
first (or even second or third) time around. Therefore, science
insists on replication of successful experiments before the
phenomenon the experiments were meant to confirm may be accepted
as being real.

Let us assume, now, that after much thought, trial, and error, a
proposed set of remote viewing experiments have been "hermetically
sealed" against external contamination, mistaken analysis,
erroneous conclusions, etc. Let us further suppose that the
experimental design is excellent, with a virtually unlimited target
pool, and constructed such that clear distinctions between accurate and
inaccurate data can be made when it comes time to judge results.
Let us finally suppose that there is adequate oversight to
guarantee against fraud.

Now, what if after one or two experimental sessions, a RV
researcher produces an excellent match with the chosen target?
This could of course be just wild, hole-in-one luck. Let's say
further that after two or three more sessions there is another
unmistakable, if uncanny match. Still chance? Yes, but considerably
less likely. But what if the viewer continues to have these
explicit matches every few sessions--indeed has runs where maybe
two or three sessions in a row match significantly-- or even
precisely--with the respective targets? At what point do we give up on
chance and acknowledge that something is going on that can't be
explained in standard physical terms?

These results could not be evaluated statistically--at best one
could say 50% of the time the viewer was accurate, or 30% or 72%, or
whatever. But these statistics would be completely meaningless.
According to Hyman's interpretation of the rules of empirical
science, barring a very rare accident of probability the viewer
should not be able to describe the target accurately even ONCE. If the
viewer is successful in describing the target not just once but a
number of times on an ongoing basis the fact is that it doesn't
matter if he or she fails most of the rest of the time. In the
paradigm of the physical universe under which Hyman and his AIR
friends operate, the viewer should ALWAYS be wrong. This is not proof
obtained by statistical "deviation from a chance baseline." Those
terms make no sense here. Yet this is indeed proof, though proof
that is unacceptable to the skeptics.

Ironically, the requirement for statistical proof that Hyman
deplores was imposed on RV research by the skeptics themselves
when they rejected evidence that required subjective evaluation
of any sort, no matter how obvious. Now, based on Jessica Utts'
thorough discussion in the AIR report, it seems clear that the
statistical evidence Hyman and his fellows demanded has now been
provided. Yet Hyman states that it is premature to accept these
figures as proof. We must wait to see if anyone can come up with
some way of showing that the data does not say what it obviously
does say. In other words, now that we can no longer dispute that
it looks, walks, and quacks like a duck, we must now carry out
exhaustive genetic tests to prove its ducky heritage. When THOSE
tests confirm that it is a duck, then we must wait through a few
more generations of technical development in genetic testing to see if
we can create a test that WILL prove that it is not a duck.

But this attitude is no surprise. Skeptical evaluation of psi
research has often resembled an archery match where during the
contest the judges keep moving the target of one competitor while
leaving those of all other contestants in place. By refusing to
acknowledge that there is now adequate proof that psi exists; by
insisting that we cannot make any judgement about the existence
of psi based on SAIC's experiments (as well as the others
mentioned by Utts); by declining to examine ALL the newly
available experimental evidence; and by failing altogether to consider
the historical track record of the intelligence operations
portion of STAR GATE's predecessors, Hyman and his cohorts have
effectively "moved the target" once more. In so doing, he has not
preserved the purity of science. He has only demonstrated his
apparent intention never to accept ANY proof, no matter how compelling,
for the effectiveness of remote viewing or the existence of psi.

SUMMATION

Since at the conclusion of all three parts of this review the
discussion is now quite long and convoluted, I shall summarize
the general points below:

1. AIR narrowed the scope of its evaluation to focus on only a
few years and a few experiments out of more than two decades of RV
research and many experiments. As a result, the AIR assessment is
useless as a comprehensive and meaningful evaluation of remote
viewing and its practical applications.

2. The SAIC experiments that AIR reviewed were not themselves a
fair test of the remote viewing phenomenon. Yet despite their
shortcomings, the experiments still demonstrated a persistent
positive result that it seems can only be ascribed to a
paranormal cause.

3. Though Hyman admits the data shows an effect, he wants to keep
the door open indefinitely--never admitting that psi may be
involved--in hopes that eventually an alternative explanation to psi
can be discovered to account for these effects (by inference, he
seems to imply fraud).

4. Ultimately, though Utts makes a far stronger case for the
existence of some sort of psi phenomenon being evidenced by SAIC
results, AIR throws the debate to Hyman, without satisfactorily
explaining why his case was deemed more compelling. Based on his
flawed evaluation Hyman decides that he has sufficient data and
personal expertise to extend his evaluation into the operational
arena--and concludes that remote viewing is of no use in
intelligence collection.

Of course, the purported motivation for the AIR evaluation that
produced in the flawed report for the CIA was to determine whether
remote viewing was useful as an intelligence collection tool. By
the manner in which the study was conducted and in the way the negative
conclusions were reached in the report, it should be clear by now
that the evaluation not only failed to honestly determine whether
remote viewing was of any intelligence use: It also showed conclusively
that there was an unacknowledged, predetermined agenda to produce
negative findings as the conclusion to the report. Presumably,
the AIR itself had no particular prior bias against remote
viewing. This leaves the contracting agency as the culprit. It
would seem that the Central Intelligence Agency gave the AIR its
marching orders: To find no merit in the program no matter what
the evidence itself showed. In Part One I suggested reasons for
this, but at this point that all still remains only speculative.
Nonetheless, there does appear to be a smoking gun here; and, as has so
often been the case recently, it seems to be lying at the feet of
the CIA.