The Department of Homeland Security is assessing a new radiation screening system to improve detection of nuclear or radiological material in cargo containers entering the United States. Shortcomings in the design and evaluation of tests conducted in 2008 reduce the Department's ability to compare the performance of the proposed system -- advanced spectroscopic portals -- to the handheld devices and portal monitors currently in use, according to the National Research Council. Further, the U.S. Congress has required cost-benefit analysis to assure that funds are spent on the new system only if the improvement in security justifies the additional investment. This report finds that the Department's analysis needs to examine more alternatives and better evaluate how they improve security before it can support such decision making.

Key Messages

• Physical tests of the advanced spectroscopic portals were not joined with computer modeling to build an understanding of the system's performance against different threats over a wide range of configurations and operating environments, as suggested in an interim National Research Council report in 2009.
• The draft cost-benefit analysis uses methods the National Research Council committee recommended to account for security benefits from advanced spectroscopic portals. However, the committee found that the Department lacked a strategic framework for justifying expenditures on portals, the set of alternatives examined was too narrow, and quantitative modeling was applied inappropriately, where qualitative reasoning could effectively support the analysis.
• The principal metric used for summarizing detector performance in the draft cost-benefit analysis is technically flawed and could be misleading. The report recommends that the Department of Homeland Security use a different metric.