Validation involves performing laboratory tests to verify that a particular instrument, software program, or measurement technique is working properly.

Confidence in forensic DNA results is gained through validation studies, which provide objective evidence that a DNA testing method is robust, reliable and reproducible. Validation experiments define procedural limitations, identify critical components of the procedure that require quality control and monitoring, and establish standard operating procedures and interpretation guidelines for laboratories to follow while processing samples.

There are a variety of challenges a forensic DNA laboratory faces when implementing a new methodology. A common challenge identified by laboratories is a lack of resources available to perform validation experiments. Laboratories also point to the existence of diverse opinions with respect to validation protocols, sample numbers and definition of appropriate and effective experiments as notable challenges. These variables have been shown to contribute to extensive validation studies that include unnecessary or excessive tests without the benefit of additional confidence. In addition, data management and analysis are cumbersome processes that are often manual operations or utilize a series of tools which analysts have developed on their own.

The Validation and Verification Atlas CD organizes information on forensic DNA analysis methods and provides direct access to Applied Biosystems developmental validation studies, reference materials, quality assurance and control activities guidelines, user bulletins, user manuals, and other supporting documentation. It is an extension of the validation support tools available from Applied Biosystems and our commitment to continue working with the forensic community to help scientists meet vital quality assurance and control activities guidelines.

Validation: What Is It, Why Does It Matter, and How Should It Be Done?
By John M. Butler, National Institute of Standards and Technology

Validation involves performing laboratory tests to verify that a particular instrument, software program, or measurement technique is working properly. These validation experiments typically examine precision, accuracy, and sensitivity, which all play a factor on the 3 R's of measurements: reliability, reproducibility, and robustness.1

Without validation studies, laboratories cannot be confident in results produced by a new genetic test, instrument or software program. These studies help define range and relevance of measurements made with a method. For example, are reproducible results expected when one or only a few cells are used to amplify degraded DNA templates that may be found in casework samples? A dilution series of a well characterized DNA sample to measure sensitivity can help answer the question of what level of input DNA with a new test is expected to produce a full DNA profile. Validation studies will also verify if a new instrument performs as well or better than a previous one in terms of sensitivity or precision of results. Since reliable analytical data are highly desirable in courts of law debating the innocence or guilt of a defendant, validation information underpinning DNA typing measurements is often scrutinized by the court in order to assess admissibility of evidence submitted by forensic laboratories. Thus, validation builds confidence for the court as well as aiding quality assurance and control activities in the lab.

Although there is not yet a standardized validation strategy that is generally accepted or utilized across forensic DNA laboratories,2 bringing a procedure (assay, instrument, or software) "on-line" in a forensic lab typically includes the following steps: (a) installation of the instrumentation or software and purchase of assay reagents, (b) learning about the technique and how to perform it properly, (c) validation of the analytical procedure to define its range and reliability, (d) creation of the standard operating procedures with interpretation guidelines based on the validation studies, (e) training of other personnel on the technique, and (f) each trained analyst passing a qualification test for initial use in forensic casework. After a procedure has been successfully implemented into use with forensic casework, proficiency tests are performed on a regular basis to demonstrate successful application of the technique over time by qualified analysts.

Over the years the forensic DNA community has perpetuated a number of misconceptions regarding validation — many of which are addressed in a recent article.3 A common perception is that validation can (or should) take many months to perform. Unfortunately, forensic labs often embark on their validation voyage without a map or a clear idea of their intended destination. Without a validation plan, these labs become weary and woeful wanderers that lose valuable time and expend unnecessary labor and reagent costs when driven off course by the winds of worry. Fear of auditors rather than scientific reasoning governs the collection of large numbers of data points in some cases. Thus, the application of a new technology for solving cases more quickly can be delayed because an overzealous number of validation experiments are performed.

Resources available to aid in formulating an effective validation plan include Section 8 of the FBI's DNA Advisory Board Quality Assurance Standards,4 which describes the primary aspects of forensic DNA validation studies. The SWGDAM Revised Validation Guidelines5 provide further detail and recommend that a total of at least 50 samples be examined as part of a careful validation study. In addition, the NIST STRBase website contains a validation section with helpful information and links to workshop materials on validation: http://www.cstl.nist.gov/biotech/strbase/validation.htm

The author, John M. Butler, is funded by the National Institute of Justice through interagency agreement 2003-IJ-R-029 with the NIST Office of Law Enforcement Standards. Points of view in this document are those of the author and do not necessarily represent the official position or policies of the US Department of Justice. Certain commercial equipment, instruments and materials are identified in order to specify experimental procedures as completely as possible. In no case does such identification imply a recommendation or endorsement by the National Institute of Standards and Technology nor does it imply that any of the materials, instruments or equipment identified are necessarily the best available for the purpose.