Tools

Digital forensic evidence is
usually latent, in that it can only be seen by the trier of fact at
the desired level of detail through the use of tools. In order for
tools to be properly applied to a legal standard, it is normally
required that the people who use these tools properly apply their
scientific knowledge, skill, experience, training, and/or education to
use a methodology that is reliable to within defined standards, to
show the history, pedigree, and reliability of the tools, proper
testing and calibration of those tools, and their application to
functions they are reliable at performing within the limitations of
their reliable application.

Tools are used in all phases of evidence
processing. In order for tools used in forensic processes to be
accepted by the legal system, the tools have to be properly applied by
people who know how to use them properly following a methodology that
meets the legal requirements associated with the particular
jurisdiction. (FRE 701-706) [3]

Methodology:
One of the key things that experts need to know about is the tools
that they use. This is because tools are used in almost all tasks
associated with DFE processing and tool failures that yield wrong
results or tool output that is not properly interpreted leads to
opinions and conclusions that may be wrong. One of the main tasks of
the DFE expert witness is to identify a meaningful methodology for
applying tools to address the legal issues and use that methodology
and tools that implement it with known accuracy and precision by
examining the evidence and the claims made with regard to the
evidence. While some of the claims may be understood with only the
experts knowledge, such as assertions that are inconsistent with each
other or that fly in the face of current scientific thinking in the
field of expertise, most claims in legal matters that involve DFE
involve the application of scientific methodologies to evidence
through tools.

Pedigree:
and History: Tools have
history and pedigree that helps to indicate their
reliability. Depending on the extent to which the tool provides
scientific results that are not obviously verifiable by independent
means by others, these factors are more important or less
important. For example, if a tool, such as the Unix command "wc"
counts the number of words, lines, and characters in a file, and the
result is used to draw a conclusion about the evidence in the matter,
it is something that can be readily confirmed or refuted by any party
by simply counting, or in the case of files with many lines, using an
independent tool. In this case, the history and pedigree are less
important than that the tool has shown reliability at the task it is
being relied upon to carry out, that it has been adequately tested,
and that it be properly calibrated for its intended use.

Reliability:
and Testing: While testing of tools may
be reasonably done by those who have background in testing of digital
systems or by independent bodies, such as NIST, which performs select
test of forensic tools in the United
States [12], calibration must be done by
the digital forensics expert prior to and after the use of the tool,
assuming that that is required for validation of the tool's accuracy
and precision to the level being used for presentation of the results
of its use. Very little testing has been formalized in this field for
the specific needs of digital forensics, so examiners wishing to be
prudent should undertake their own testing programs, and this should
be a normal part of the process used in preparing for legal matters
where such tools are used. There is a substantial body of well defined
knowledge in testing of digital systems, including refereed
professional journals, books, conferences, and classes at the
undergraduate and graduate level. As an example, the IEEE has had a
refereed journal on the subject since
1984. [13]

Testing of tools is fundamental to their use, and in
the field of DFE, an individual brought forth as an expert who has not
tested their tools and does not know their function and limitations in
adequate detail, is unlikely to be able to withstand cross-examination
with regard to those tools or the things those tools are being applied
to. This may, ultimately, lead to their disqualification as an expert,
or the disregarding of their testimony as not meeting the standards
required for credible expert testimony.

Calibration:
The notion of calibration is foreign to many in the digital computer
arena, largely because, unlike analog devices which have minor
variances due to temperature, pressure, and other physical conditions,
digital systems, when working within normal operating ranges, produce
either 1s or 0s and do so with very high reliability. Nevertheless,
there are calibrations that can and should be done prior to and after
the use of DFE tools to validate that what was done did not introduce
inaccuracies into the process. As an example, when doing a forensic
image of digital media to a different media, the destination media
should be pre-configured to a known state so that process failures can
be detected. Otherwise, residual data from previous events or from the
manufacturing process might be mistakenly intermixed with the new DFE
to produce corrupted results. This sort of spoliation has the
potential to create enormous problems if the tools and media are not
properly calibrated, if error messages are not carefully preserved and
taken into account, if contemporaneous logs of the forensic activities
are not produced and retained, and if evidence isn't created to verify
that the image taken is a true copy of the original evidence. This is
similar to the process of cleaning a pipet for a chemical analysis,
testing the cleaned pipet to verify that it is free of contaminants,
processing the sample, getting the result, then verifying that the
pipet is free of contaminants after the sample is analyzed. Failure to
undertake such a process would violate standard procedure in chemical
testing that has been shown to produce faulty chemical
analysis. Similarly, failure to undertake measures to calibrate and
verify digital forensic processing of evidence can introduce
contaminants or produce faulty digital analysis.

Digital forensic analysis processes often include
the creation of special purpose filters, the development of search
criteria, and the authoring of small computer programs, sometimes
including combinations of scripts written in languages such as the
command language of the Unix shell, the Perl language, and other
programs written in other languages, and pre-packaged utility programs
that come with systems, such as the stream editor "sed", the regular
expression string search program "grep", and many other similar sorts
of elements. These are commonly combined with tools that retrieve data
from Internet sites and process them in various ways to produce
outputs that show some analytical result.

Function:
and Limitations:
When such tools produce results that are readily verified by
inspection, such as counts of how many lines of particular types were
at particular locations within particular files, the conclusions
themselves constitute a testable result that the opposition can
challenge and verify. As such, the tools and techniques need not be
shown; however, when introducing such evidence, it is incumbent on the
producing party to make certain that the results are accurate and
precise. To the extent that they are in error and the opposition can
demonstrate this, the court will often levy sanctions and potentially
exclude the expert and the results from use in court under the
admissibility restriction that the results are less probative than
prejudicial, the expert witness is not reliably applying a scientific
method to the evidence, and that the expert is not in fact adequately
knowledgeable or skilled to express scientific opinions to the trier
of fact. It is incumbent on experts to provide details of the limits
of their results in terms of the limits of accuracy and precision and
to not overstate results. For example, when analyzing text files
against a format specification, the expert had better understand the
extent to which the formal specification is reflected in actual use,
and examine results produced for anomalies before declaring the
results of the program to be precise and accurate. To the extent that
anomalies are detected, they should be explained and the precision and
accuracy of results properly characterized.