The purpose of this document is to provide recommendations and guidelines for the use of digital image processing in the criminal justice system. The objective is to ensure the successful introduction of forensic imagery as evidence in a court of law. This document includes brief descriptions of advantages, disadvantages, and potential limitations of each major process.

Background

Digital image processing is an accepted practice in forensic science. It is the position of the Scientific Working Group on Imaging Technologies (SWGIT) that any changes to an image made through digital image processing are acceptable in forensic applications provided the following criteria are met:

The original image is preserved

The processing steps are logged when they include techniques other than those used in a traditional photographic darkroom

The end result is presented as an enhanced image, which may be reproduced by applying the logged steps to the original image

The recommendations of this document are followed.

Introduction

This document addresses digital image processing and related legal considerations in the following four categories:

Image enhancement

Image restoration

Image compression

Image analysis

When using digital image processing techniques, use caution to avoid the introduction of unexplainable artifacts that add misleading information to the image and the loss of image detail that could lead to an erroneous interpretation. Any processing techniques should be applied only to the working image.

The successful introduction of forensic imagery as evidence in a court of law is dependant upon the following four legal tests:

Artifact: Any visible feature or distortion in a recorded image or output image that is not present in the corresponding imaged object or input image. Image artifacts can be introduced inadvertently by hardware or software or intentionally by an operator. The latter type includes annotation or other direct alteration of an image in order to clarify or call attention to some particular image content. Artifacts introduced by hardware and software generally degrade an image and, if severe enough, can impair interpretation.

Image analysis: The extraction of quantitative information from an image beyond which is readily apparent through visual examination.

Image averaging: The process of averaging together similar images, such as sequential video frames, to reduce noise in stationary scenes.

Image compression: The process of reducing the size of a data file.

Image enhancement: Any process intended to improve the visual appearance of an image.

Image processing: Any activity that transforms an input image into an output image. Note: Image processing does not mean that the input image is overwritten during the process. Forensic image processing should be performed only on working images.

Image restoration: Any process applied to an image that has been degraded by a known cause, such as defocus or motion blur, so the effects of that degradation are partially or totally removed.

Image synthesis: Any process that renders an image through the use of computer graphics techniques for illustrative purposes (e.g., age progression, facial reconstruction, accident/crime scene reconstruction). This subject is beyond the scope of this document.

Interlaced scan: Video image format. The video frame consists of two fields. The first field contains all the odd-numbered horizontal lines and the second field all the even-numbered lines. All standard TV video signals used in North America and elsewhere under the NTSC (RS-170, CCIR) standard are in the interlaced format.

Interpolation: A process by which the apparent resolution of an image is increased. In most cases the software mathematically averages adjacent pixel densities and places a pixel of that density between the two.

JPEG (Joint Photographic Experts Group): A lossy image compression process. Users can set their own quality settings on a sliding scale within the application software.

JPEG 2000: An image compression process currently under development.

LZW (Lempel-Ziv-Welch): A lossless compression process used by the TIFF and GIF file formats.

MPEG (Motion Pictures Experts Group): Similar to JPEG, a standard compression algorithm used to compress video and audio sequences.

Original image: An accurate and complete replica of the primary image, regardless of media. For film and analog video, the primary image is the original image.

Primary image: Refers to the first instance in which an image is recorded onto any media that is a separate, identifiable object or objects. Examples include a digital image recorded on a flash card or a digital image downloaded from the Internet.

Progressive scan: (Noninterlace) Video in which each image frame contains information from every horizontal scan line of the imaging sensor.

Reliability: The extent to which information can be depended upon.

Reproducibility: The extent to which a process yields the same results on repeated trials.

Security: The extent to which the evidence has been preserved and safeguarded.

TIFF (Tagged Image File Format): A standardized image file exchange format. It is widely supported by both hardware and software manufacturers and is platform independent. Can be lossless or lossy.

Working image: Any image subjected to processing.

Image Enhancement

Image enhancement is any process intended to improve the visual appearance of an image. This includes processes that have a direct counterpart in the conventional silver-based photographic laboratory and those that can be accomplished only by using a computer.

Traditional Enhancement Techniques

Traditional enhancement techniques are techniques that have direct counterparts in traditional darkrooms. They include brightness and contrast adjustment, color balancing, cropping, and dodging and burning. These traditional and acceptable forensic techniques are used to achieve an accurate recording of an event or object.

Brightness adjustment is used when the image is too bright or too dark. If the image is made too bright, there is a risk of loss of detail in light areas. If the image is made too dark, there is a risk of loss of detail in the dark areas.

Color balancing is the adjustment of the color components of an image. The purpose of color balancing is to render the colors in the scene faithfully. Improper color balance adjustment can render colors inaccurately, and objects will appear to have the wrong color when compared to the actual subject.

Contrast adjustment is used when the image lacks sufficient contrast. If the image contrast is increased too much, there is a risk of loss of detail in both light and dark areas.

Cropping is used to remove that portion of the image that is outside the area of interest.

Dodging and burning have the same effect as brightness adjustment but are used in localized areas.

Spotting traditionally has been used to remove artifacts due to dust and scratches on the negatives, but it is not considered to be an acceptable practice on any forensic image.

Note: The use of spotting and cropping techniques may come under scrutiny in a court of law. Specific agency policies should address the use of these techniques.

Nontraditional Enhancement Techniques

Some nontraditional image enhancement processes are used and accepted by a variety of scientific fields such as medicine, aerospace, and cartography. These processes have no direct counterpart within traditional silver-based photography. In fact, only recently have they been applied within the forensic environment; therefore, their general acceptance may be subject to challenge. Examples of nontraditional processes discussed here are color processing, linear filtering, nonlinear contrast adjustments, pattern noise reduction, and random noise reduction.

Color processing includes color space transformations, pseudocoloring, and hue and saturation adjustments. These techniques can be used to modify the color characteristics of objects within an image. Caution: Application of these techniques can compromise the color fidelity of the image.

Linear filtering techniques include sharpening, deblurring, edge enhancement, and deconvolution. They are used to increase the contrast of small detail in an image. If a low degree of enhancement is used, the image will remain an accurate representation of the scene. If a high degree of enhancement is used, the image may no longer be an accurate representation of the overall scene, though still may be useful as an adjunct for interpretation of small details. Caution: A high degree of enhancement can also increase the visibility of existing noise and artifacts. Examples of noise include film grain, snow appearing on a TV screen, or random color dots. (Figure 1)

Nonlinear contrast adjustments include gamma correction, grayscale transformation, curves, and look-up tables. They are an extension of traditional photographic sensitometric techniques and are used to adjust the contrast in selected brightness ranges within the image. For example, details may be brought out in the shadow areas without affecting the highlight areas. Caution: A severe adjustment can cause loss of detail, color reversal, and the introduction of artifacts. (Figure 2)

Figure 2 This example shows nonlinear contrast adjustments. Left: original image; Middle: enhancement of shadow and highlight areas, at the expense of midrange tones; Right: enhancement of midrange tones, at the expense of shadow and highlight areas. Click to enlarge image.

Pattern noise reduction filters identify repeating patterns in the image and allow the user to selectively remove them. This type of filter can be used to remove patterns such as fabric weaves, window screens, security patterns, and halftone dots. Caution: Overuse of this technique can cause selective removal of relevant image detail.

Random noise reduction techniques include such filters as low pass, blurring, median, and despeckling. They are used to reduce the contrast of small detail in the image in order to suppress random noise. Caution: Overuse of this technique can cause loss of relevant detail.

Considerations for the Application of Image Enhancement Techniques

Question: What type of image must not be enhanced? Answer: A primary or original image.Discussion: Because a primary or original image represents the first instance where the image is recorded onto any media, or it is an accurate and complete replica of the primary image, it must not be altered or modified. Enhancements are performed only on working images.

Question: Is it necessary to document the enhancement process used to produce an enhanced image?Answer: The need to document the enhancement process is determined by the process used.Discussion: Documentation of enhancement steps is not necessary when using traditional darkroom techniques. When using nontraditional image enhancement techniques such as unsharp masking or random noise reduction, enhancement steps should be documented in the case notes in sufficient detail to enable another comparatively trained individual to repeat the steps and produce the same output when the image is subjected to image analysis.

Question: In a legal setting, are enhanced images discoverable?Answer: Yes.Discussion: All images may be discoverable. In cases where images are enhanced, both the original and the enhanced image, along with associated documentation, may be discoverable.

Question: Who is responsible for testifying about an enhanced image?Answer: The person doing the enhancement or a person skilled in and knowledgeable about the enhancement process that was used.Discussion: The person who performed the enhancement is best qualified to testify about the enhancement techniques used. However, there may be occasions where the court will require the assistance of additional subject-matter experts.

Question: Are there legal ramifications associated with the software used specifically for image enhancement?Answer: Yes. Discussion: Some considerations may include:

Have the particular functions within the software been accepted by the scientific community?

Does the software perform as the manufacturer purports?

Has the use of this software been reviewed by the judicial system?

Does the software have “-ins” that are produced by another manufacturer?

Is the enhancement process repeatable and reliable?

Image Restoration

Image restoration is any process applied to an image that has been degraded by a known cause (e.g., defocus or motion blur) to partially or totally remove the effects of that degradation.

Limitations are imposed on this technique by any noise in the image and by the fact that information that has been totally lost cannot be replaced. Often partial restoration can be successful even when total restoration is impossible.

Restoration Techniques

Blur removal is a filtering technique designed to partially or completely remove an image blur imposed by a known cause. It differs from the image enhancement filtering processes because the blur removal filter is designed specifically for the process that blurred the particular image under examination. Examples include defocus and motion blur, since these blurring phenomena can be described mathematically. Thus, a specific filter can be designed to compensate for each blur. The degree to which a blur can be successfully removed is limited by noise in the image, the accuracy with which the actual blurring process can be described mathematically, and the fact that information has been totally lost and cannot be replaced. Often partial deblurring can be successful even when total deblurring is impossible.

Color balancing is the extension of grayscale linearization to a color image. It is the adjustment of the color components of an image. The purpose of color balancing is to render the colors in the scene faithfully. For example, a color test target having known colors can be placed in the scene prior to recording the image. Then a grayscale transformation (nonlinear contrast stretch) can be designed for each color channel (red, green, and blue) to place the different colors on the test target in their proper relationship. It is commonly assumed that the color of other objects in the scene will be rendered accurately as well. Improper color balance can render colors inaccurately, causing objects to appear to have the wrong color.

Geometric restoration is the removal of geometric distortion from an image. Its purpose is to restore the proper spatial relationships among the objects in the scene. It can be used for the removal of geometric distortion, such as that introduced by a curved mirror or a fish-eye lens. It differs from image warping in that the geometric transformation is designed specifically for the process that distorted the particular image under examination. The degree to which geometric distortion can be successfully restored is limited by the accuracy with which the actual distortion process can be described mathematically and the fact that information that has been totally lost (e.g., hidden behind another object or obscured from the camera) cannot be replaced. Often partial geometric restoration can be successful even when exact geometric restoration is impossible.

Grayscale linearization is the adjustment of brightness relationships among the objects in a scene. The purpose of grayscale linearization is to render faithfully the different brightness values in the scene. For example, a monochrome test target having known gray values can be placed in the scene prior to recording the image. Then a grayscale transformation (nonlinear contrast stretch) can be designed to place the different gray values on the test target in their proper relationship. It is commonly assumed that the other objects in the scene will be put in their proper brightness relationship as well. Improper grayscale linearization can render brightness values inaccurately so that objects may appear brighter or darker than they actually appeared when the image was recorded.

Warping, unlike other image restoration processes, changes the spatial relationships among the objects in an image. It is analogous to printing a photograph on a rubber sheet, then stretching the sheet in different directions and then tacking it down. Warping can be used, for example, to remove perspective from an image or to “unroll” a poster that was wrapped around a pole. Used improperly, it can distort the natural appearance of the objects in a scene.

Considerations for the Application of Image Restoration Techniques

Question: What type of image must not be restored? Answer: A primary or original image.Discussion: Because a primary or original image represents the first instance where the image is recorded onto any media, or it is an accurate and complete replica of the primary image, it must not be altered or modified.

Question: Is it necessary to document the restoration process? Answer: Yes. Discussion: Documentation of restoration steps is always required.

Question: Are restored images discoverable in legal proceedings?Answer: Yes.Discussion: All images may be discoverable. In cases where images are restored, both the original and the restored image, along with associated documentation, may be discoverable.

Question: Who is responsible for testifying about a restored image?Answer: The person doing the restoration or a person skilled in and knowledgeable about the restoration process that was used.Discussion: The person who performed the restoration is best qualified to testify about the restoration techniques used. However, there may be occasions when the court will require the assistance of additional subject-matter experts.

Question: Are there legal ramifications associated with the software used specifically for image restoration?Answer: Yes. Discussion: Some considerations may include:

Have the particular functions within the software been accepted by the scientific community?

Does the software perform as the manufacturer purports?

Has the use of this software been reviewed by the judicial system?

Does the software have “plug-ins” that are produced by another manufacturer?

Is the restoration process repeatable and reliable?

Has the degradation process been accurately modeled?

Image Compression

Digital images produce a large amount of data to be stored. Image compression techniques reduce the storage requirements by making image data files smaller.

Compression Processes

Lossless compression reduces file size by removing redundant information. Because the redundant information can be replaced in order to display the image, lossless compression results in no loss of information. Lossless compression does not alter the content of an image when it is decompressed. An example of a file format that uses lossless compression is the graphical interchange format (GIF).

Lossy compression achieves greater reduction in file size by removing both redundant and irrelevant information. Because the irrelevant information (as determined by the compression algorithm) cannot be replaced upon reconstruction of an image for display, lossy compression results in some loss of image content as well as the introduction of artifacts. The degradation occurs each time the image is saved in a lossy file format. Higher compression ratios result in the loss of more information. Normally the degree of compression can be specified. Depending upon the application, lossy compression may render an image less useful. Caution: Lossy compression should be used with care to avoid material degradation of the image. Additionally, the compression settings used by one camera or software program may not be the same as the compression settings used by another camera or software program.

The commonly used joint photographic experts group (JPEG) image storage format employs lossy image compression. It is applied to the image in 8-pixel by 8-pixel blocks. Normally, the degree of compression can be specified prior to storing the image. At high compression ratios, JPEG could remove important image detail and introduce blocking artifacts as the block boundaries become visible. (Figure 3) Digital cameras often create digital images in JPEG format, so that some lossy compression is unavoidable. The degree of compression should be set low enough that important image content is not lost or obscured by artifacts.

Figure 3 Left: original image; Middle: the result of JPEG compression (compression ratio = 15:1); Right: the result of edge enhancement after compression. Click to enlarge image.

Considerations for the Application of Image Compression Techniques

Question: What type of image must not be compressed? Answer: It depends on the end use and need.Discussion: In instances where the primary or original image is already compressed, it should not be further compressed using lossy compression processes; additional data will be lost. Sources of compressed primary images may include electronic booking photographs, some types of digital camera images, and images downloaded from the Internet or E-mail. The file format is not an indicator of the compression history for an image. For example, a TIFF file may have been previously compressed in a lossy file format (JPEG). Be aware that the end use of any image may change over time, and the use of compression may become problematic. Images intended for laboratory analysis should not be compressed using a lossy process.

Question: Is it necessary to document the compression history of an image?Answer: It depends on the intended use of the image. Discussion: The type and degree of compression may become an issue in a judicial proceeding. Documentation may be necessary in a court of law when argued that compression might have introduced artifacts or relevant information was lost.

Question: Who is responsible for testifying about a compressed image? Answer: The person doing the compression can testify about the settings used to compress an image. Discussion: Questions concerning the actual compression process should be referred to individuals who possess sufficient technical expertise to explain the specific process.

Quantitative Image Analysis

Quantitative image analysis is the process used to extract quantitative (measurable) data from an image, whereas cognitive image analysis is the process used to extract visual information from an image. This section discusses quantitative analysis only.

Quantitative image analysis requires proper calibration of the image. In a digital image the pixel spacing must be known in order to extract accurate size measurements. Objects that are different distances from the camera will have different pixel spacing. The accuracy of the extracted measurements will depend upon the accuracy of calibration. Caution: The use of image compression can degrade the accuracy of subsequent quantitative image analysis.

An example of a quantitative image analysis might be if a circular object in an image includes 314 pixels, and the area covered by a single pixel is one square millimeter, then one can conclude that the area of the object is 314 square millimeters. Similarly, if the distance between the adjacent pixels in an image of a document is 0.02 inches, and the length of the document is 340 pixels, then it must be 340 times 0.02, or 6.8 inches long. These examples do not consider perspective distortion.

Quantitative Image Analysis Techniques

Colorimetry is the quantification of the color of an object.

Image authentication verifies that the original image has not been altered.

Photogrammetry is the science involving methods, techniques, and analytical procedures used to make accurate measurements of distances and/or sizes of objects from photographic images.

Photometry is the measurement of light values of objects in an image.

Considerations for the Application of Image Analysis Techniques

Question: Which types of image should be subjected to quantitative image analysis? Answer: A working image.Discussion: Because a primary or original image represents the first instance where the image is recorded onto any media, or it is an accurate and complete replica of the primary image, it must not be altered or modified.

Question: Is it necessary to document quantitative image analysis?Answer: Yes.Discussion: Documentation of quantitative image analysis steps is required in sufficient detail to enable another comparably trained individual to repeat the steps and produce the same conclusions.

Question: Are analyzed images discoverable?Answer: Yes.Discussion: All analyzed images, documentation, and conclusions may be discoverable.

Question: Who is responsible for testifying about an analyzed image?Answer: The person doing the analysis or a person skilled in and knowledgeable about the analysis performed.Discussion: The person who performed the analysis is best qualified to testify concerning the techniques used. However, there may be occasions where the court will require the assistance of additional subject-matter experts.

Question: Are there legal ramifications associated with the software used specifically for image analysis?Answer: Yes. Discussion: Some considerations may include:

Have the particular functions within the software been accepted by the scientific community?

Does the software perform as the manufacturer purports?

Has the use of this software been reviewed by the judicial system?

Does the software have “plug-ins” that are produced by another manufacturer?

Is the analysis repeatable and reliable?

Additional Imaging Considerations

Question: Where does image processing take place, in the field or in a controlled environment?Answer: Both. Discussion: Whereas most image processing takes place in a controlled environment, some image processing, such as photogrammetry and image compression, may take place in the field.

Question: Who performs image processing?Answer: Photographers, analysts, and technicians.Discussion: The person performing the processing must be properly trained.

Question: What are file management processes?Answer: File management processes are the capture, storage, indexing, retrieval, and archiving of image files. Discussion: Agencies and organizations should establish file management procedures for managing image files for use at a later date.

Question: Does image processing alter images?Answer: Yes.Discussion: The purpose of image processing is to alter the images in a controlled, predictable, and repeatable manner. Image processing does not mean that the input image is overwritten during the process. Forensic image processing should only be performed on working images.

Guidelines for Digital Image Processing Standard Operating Procedures

The purpose of image processing procedures is to apply processing techniques intended to enhance, restore, compress, and/or analyze digital images. The success of the processing of digital images is measured against the four legal tests: reliability, reproducibility, security, and discovery. To achieve success, standard operating procedures should be followed. Appendix A is a sample standard operating procedure.

Guidelines for Equipment

The agency should address the following minimum hardware and software equipment requirements.

Hardware:

Input/capture device

Image processing systems

Output devices

Storage/archive

Software:

Image management

Image processing

Guidelines for Procedures

Agencies should establish specific step-by-step procedures for image processing according to agency requirements using SWGIT guidelines. These procedures should address the following as a minimum:

Capture

Processing

Storage/archive

Image management

Security

Output

Guidelines for Calibration

If necessary, agencies should develop calibration procedures specific to their needs.

Guidelines for Calculations

If necessary, agencies should develop calculation procedures specific to their needs.

Guidelines for Limitations

Agencies should take into consideration agency-specific budget, equipment, management, and accrediting agency requirements.

Agencies should document procedures to ensure sufficient training to afford competence and proficiency with applicable image processing. Refer to the Guidelines and Recommendations for Training in Imaging Technologies in the Criminal Justice System at the following: