Expert Report

Evaluation of the Updated Site-Specific Risk Assessment for the National Bio- and Agro-Defense Facility in Manhattan, Kansas (2012)

Each report is produced by a committee of experts selected by the Academy to address a particular statement of task and is subject to a rigorous, independent peer review; while the reports represent views of the committee, they also are endorsed by the Academy. Learn more on our expert consensus reports.

The updated site-specific risk assessment for the National Bio- and Agro-Defense Facility is a substantial improvement over the original 2010 site-specific risk assessment, but is still inadequate in fully characterizing the risks associated with operating a high biocontainment facility in Manhattan, Kansas. Many of the shortcomings identified in a 2010 National Research Council report have been addressed in this updated assessment and more conventional risk analysis methods were used. However, many risk analysis methods were misinterpreted and misapplied when executed, and questionable and inappropriate assumptions were used throughout the updated site-specific risk assessment which led to artificially low estimates of the probabilities and amounts of pathogen that might be released.

Key Messages

  • The updated site-specific risk assessment is a substantial improvement over the original 2010 site-specific risk assessment, addressing many shortcomings identified in the previous National Research Council report. Despite these improvements, the report's authoring committee finds that the updated site-specific risk assessment underestimates the risks of pathogen release and infection and inadequately characterizes the uncertainties of these risks.
  • The latest designs for the National Bio- and Agro-Defense Facility are 65 percent complete and appear sound, however, it was beyond the committee's task to formally review or pass judgment on the actual engineering or safety of the facility.
  • The 2010 site-specific risk assessment showed that for the two greatest release scenarios, there is nearly a 70 percent probability that a release of foot-and-mouth disease virus could result in an infection outside the laboratory over the 50-year lifespan of the National Bio- and Agro-Defense Facility in Manhattan, Kansas. In contrast, the updated site-specific risk assessment concludes that for 142 possible release events, the cumulative probability of such a release is just 0.11 percent (or 1 in 46,000 chance per year).
  • The updated site-specific risk assessment uses a quantitative modeling framework that includes the identification of risk scenarios, total calculation of risk of all events, and uncertainty analysis—an important advance over the 2010 risk assessment. However, the committee identified concerns about how the framework was implemented and found that methods were inconsistently applied across the various sections of the updated site-specific risk assessment and that probabilistic dependencies in calculating risk scenarios were ignored, resulting in underestimations of total risk and incorrect ranking of risk contributors.
  • Questionable and inappropriate assumptions used throughout the updated site-specific risk assessment lead to artificially low estimates of the probabilities and amounts of pathogen that might be released. Because the updated site-specific risk assessment contains inconsistent information, it has been difficult to interpret data or to reconstruct risk scenarios in order to determine the degree to which risks were underestimated.
  • The updated site-specific risk concludes that a release caused by a natural hazard, such as a tornado or earthquake, is 20 times more likely than a release caused by an error or failure during normal facility operations. The committee questions this conclusion and finds that the updated site-specific risk assessment overestimates the likelihood of a release caused by a natural hazard.
  • The updated site-specific risk assessment does not consider overall risk for biosafety level 4 work and presents a limited qualitative assessment of impact. It focused exclusively on the unique risks associated with handling large animals in biosafety level 4 containment and provides only a minimal risk estimate by considering other activities in the biosafety level 4 suites, such as the handling of Nipah virus and Hendra virus, suggesting these activities are risk-free. The updated site-specific risk assessment does not adequately consider the totality of risks, and likely underestimates the overall risk related to the biosafety level 4 suite.