FBI Response to the report Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods
Federal Bureau of Investigation
September 20, 2016
Below is the Federal Bureau of Investigations's response to the President's Council of Advisors on Science and Technology report entitled Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods.
Comments on:
President’s Council of Advisors on Science and Technology
REPORT TO THE PRESIDENT
Forensic Science in Federal Criminal Courts: Ensuring Scientific
Validity of Pattern Comparison Methods
REPORT TO THE PRESIDENT
Forensic Science in Federal Criminal Courts: Ensuring Scientific
Validity of Pattern Comparison Methods
The FBI agrees with the authors of the President’s Council of Advisors on Science and Technology (PCAST) report that forensic science plays a critical role in the criminal justice system, and therefore needs to be held to high standards. Further, the FBI agrees with the PCAST report as well as the 2009 National Research Council report (2009 NAS report) that significant funding is needed to develop stronger ties between the academic research community and the forensic science community. It is inherent within science that over time, our knowledge of a subject evolves. It is critical that continued research be pursued in order to ensure that forensic science meets the high standards necessary to be used in a court of law.
However, the FBI disagrees with many of the scientific assertions and conclusions of the report. The report makes broad, unsupported assertions regarding science and forensic science practice. For example, the report states that “the only way” to establish “validity as applied” is through proficiency testing, and requires a measurement of how often the examiner gets the correct answer, which is fundamentally at odds with a report of the National Academy of Sciences.1
The report also creates its own criteria for scientific validity and then proceeds to apply these tests to seven forensic science disciplines, failing to provide scientific support that these criteria are well accepted within the scientific community. In fact, PCAST defines their internally developed criteria as “scientific criteria” by which forensic feature-comparison methods must be supported by. However, PCAST does not apply its own criteria consistently or transparently. The PCAST criteria define “black box” studies as the benchmark to demonstrate foundational validity, but provide no clarification on how many studies are needed or why some studies that have been conducted do not meet their criteria. These criteria seem to be subjectively derived and are therefore inconsistent and unreliable.
The report does not mention numerous published research studies which seem to meet PCAST’s criteria for appropriately designed studies providing support for foundational validity. That omission discredits the PCAST report as a thorough evaluation of scientific validity.
The report proposes federal government criminal justice related-databases should be made available to researchers for independent studies while consistently overlooking the legal authorization and limitations set out in statutes and regulations regarding the use of such databases.
Finally, the report ignores important differences between forensic science disciplines, conflating fundamental differences between class-level and identification-level evidence, leading to troubling generalized conclusions about all forensic science disciplines.
1National Academy of Sciences,STRENGTHENING FORENSIC SCIENCE IN THE UNITED STATES, 9 (2009); see also id. At 207 (“[b]lind proficiency testing is recommended ... not as a way to determine error rates, but as amore precise test of a worker’s accuracy.”)
Read the document
Article posted September 21, 2016