GAO: DHS overstating reliability of nuclear weapons detectors

Space & Missile Defense Report On Nov 5, 2008
Source: XOD

The Department of Homeland Security (DHS) has used figures that may
suggest a new type of nuclear weapons detector is more reliable than warranted,
the Government Accountability Office (GAO) stated in a report to Congress.

To be sure, the GAO report didn't find that screening detectors installed
at shipping ports are unable to detect nuclear weapons or materials secreted
within cargo containers.
Rather, the watchdog agency found that test results may overstate the
competence and reliability of those sensor systems in ferreting out threats.

The DHS Domestic Nuclear Detection Office is tasked with preventing
smuggled nuclear weapons from entering the nation, and using the radiation
detection portal monitors as one means of achieving that goal.

Specifically, the GAO focused on a new type of detector called advanced
spectroscopic portal (ASP) monitors. The competing versions of the ASP are made
by Raytheon Co. [RTN], Canberra and Thermo to screen trucks carrying cargo
containers as they arrive at ports to be loaded on ships. The three contracts
totaled $1.2 billion over five years.
So-called Phase 3 tests of how well the ASP systems work were interpreted
by DHS in a way that the GAO asserted was flawed. For example, while DHS might
say test results show ASP would identify a nuclear source in a container about
half the time, actually it might be in a range of 15 to 85 percent of the time,
the GAO found.

The DHS "reporting of the test results ... makes them appear more
conclusive and precise than they really are," the GAO report stated.
Congress mandated that ASPs cannot be procured at full production unless
and until tests show the ASPs would be a major improvement over existing
detectors.

The government review agency determined that "it is not appropriate to
use the Phase 3 test report in determining whether the ASPs represent a
significant improvement over currently deployed radiation [detection] equipment
because the limited number of test runs do not support many of the comparisons
of ASP performance made in the Phase 3 report."

These results, the GAO asserted, "could be misleading."
If an ASP correctly spots nuclear material every time, but the test is
run just five times, then the most that can be concluded with a high level of
statistical confidence is that "the probability of identification is no less
than about 60 percent," the GAO stated.

DHS strongly disagreed with the report and some conclusions, and DHS
refused to make two changes that GAO recommended, while agreeing to make a third
change. The GAO, meanwhile, stands behind its report and its recommendations.

To read the GAO report titled "Combating Nuclear Smuggling: DHS's Phase 3
Test Report on Advanced Portal Monitors Does Not Fully Disclose the Limitations
of the Test Results" in entirety, please go to www.gao.gov on the Web.