The Bureau of Alcohol, Tobacco, Firearms and Explosives’ and Federal Bureau of Investigation’s Arson and Explosives Intelligence Databases

Audit Report 05-01
October 2004
Office of the Inspector General

Findings and Recommendations

DUPLICATE SYSTEMS ARE MAINTAINED BY THE ATF AND FBI FOR REPORTING ARSON AND EXPLOSIVES INCIDENTS

We found that the ATF and the FBI systems for collecting and disseminating arson and explosives intelligence are duplicative. Although the respective systems have some unique features, their overall purpose is the same and many of the data fields are identical. This condition occurred because both the ATF and the FBI are charged with similar responsibilities for collecting and disseminating such information, and the two agencies have not collaborated to develop a single system that serves the needs of their customers. As a result, customers do not have a single source to which they report incidents or a consolidated source for obtaining information to assist in their investigations.

In addition, we found significant differences in the ATF’s and FBI’s performance in managing their databases. We found minimal errors among the data fields tested in the ATF’s AEXIS and BATS databases. However, we found a significantly higher error rate among the data fields tested in the FBI’s AIRS database. Errors in the ATF and FBI databases occurred because of system limitations and because of inadequate polices and procedures for receiving and entering data. Customers of both the ATF and FBI provided favorable responses regarding the timeliness of services. However, in August 2003, the FBI’s BDC had a backlog of 5,745 Activity Reports and 770 Incident Reports that had not been entered into AIRS by contract Louisiana State University (LSU) staff, resulting in intelligence information being unavailable to customers in a timely manner. According to LSU staff, as of July 2004 the backlog has been eliminated.

Duplication in Reporting

The overall purpose of the ATF and FBI database systems is the same: to collect, and make available to the law enforcement community, information related to incidents of bombing or arson. We found duplication of effort and a lack of consistency in reporting practices among the agencies that report arson or explosives intelligence information to the ATF and the FBI.

In the course of our audit, we interviewed a judgmental sample of 48 federal, state, and local law enforcement agency officials by telephone (See Appendix III). Our sample included bomb squads, sheriff’s departments, police departments, ATF and FBI field offices, and the United States Fire Administration. The agencies we contacted served 11 of the largest cities in the United States, as follows:

New York City
Los Angeles
Chicago
Houston
Philadelphia
Phoenix
San Diego
Dallas
San Antonio
Detroit
Atlanta

We asked representatives of the agencies, “To which federal agencies do you normally report incidents?” We also asked, “How do you decide to which agency you report arson or explosives incidents?” As the following chart shows, we found little consistency to whom the 48 agencies reported data.

Agency Interviewed

Agency Reported Data To

ATF Only

FBI Only

ATF & FBI

Neither

ATF Field Offices

6

FBI Field Offices

7

1

US Fire Administration

1

Non-Federal Agencies

5

10

15

3

Four of the 15 non-federal agencies that reported to both the ATF and the FBI indicated they report all incidents to both agencies. One of the 15 indicated it reported “most” incidents to both the ATF and the FBI. Three of the remaining ten indicated they report “some” incidents to both the ATF and the FBI. Seven of the FBI field offices were not reporting incidents to the ATF as required by Title 18, United States Code (U.S.C.), Chapter 40, section 846(b).

The reporting policies among the non-federal agencies were inconsistent, and they differed widely regarding the criteria they used for deciding to whom they report incidents. Examples of responses to our questions regarding reporting policies include:

Agency “prefers” reporting to the ATF

Agency will “report arson to ATF if the sheriff decides it can
be used by ATF”

Agency believes it is “mandated to send to FBI”

Agency believes terrorism must be reported to both ATF and FBI

Agency believes “domestic crackpot” incidents should be reported to the ATF but “domestic terrorist” incidents should be reported to the FBI

The lack of consistent policies and practices among reporting agencies on where to report arson and explosives incidents can create confusion on the part of the reporting agencies and duplication in reporting.

Legislative History Resulting in Duplication

ATF

On September 30, 1996, Public Law 104-208 (110 Stat. 3009), the Omnibus Consolidated Appropriations Act of 1997 (Act), was enacted. The Act amended the Federal Explosives laws in Title 18, U.S.C., Chapter 40. As amended, section 846(b) states, in part:

The Secretary (of the Treasury) is authorized to establish a national repository of information on incidents involving arson and the suspected criminal misuse of explosives. All Federal agencies having information concerning such incidents shall report the information to the Secretary pursuant to such regulations as deemed necessary to carry out the provisions of this subsection. The repository shall also contain information on incidents voluntarily reported to the Secretary by State and local authorities.

This section clearly requires all federal agencies having information concerning incidents involving arson and suspected criminal use of explosives to report the information to the Secretary. This includes information regarding arson and explosives incidents investigated by a federal agency, as well as by other entities, such as state and local agencies. The Secretary gave the ATF the responsibility to establish the repository and collect such information.

The ATF established its primary database (AEXIS) for collecting this information. At the time of our review, the ATF reported that AEXIS contained over 100,000 detailed records of bombings, thefts of explosives, recovered explosives and devices, and ATF fire investigations dating back to 1975.

FBI

In 1988, Congress passed Public Law 100-690 entitled “The Uniform Federal Crime Reporting Act of 1988,” which required that standardized reporting of crime from law enforcement agencies with law enforcement programs be reported automatically on a monthly basis, under prescribed specifications, to the FBI. The Uniform Crime Reports, as authorized under Title 28, United States Code (U.S.C), Section 534, were to be administered by the FBI. This section states, in part:

The Attorney General shall acquire, collect, classify, and preserve identification, criminal identification, crime, and other records…
and … exchange such records and information with, and for the official use of, authorized individuals of the Federal government, the States, cities, and penal and other institutions.

The reports submitted under this requirement compiled nationwide criminal statistics for use in law enforcement administration, operation, and management and assess the nature and type of crime in the United States. In addition, under Title 28, Code of Federal Regulations (C.F.R), Section 85(f), the FBI was designated to operate a central clearinghouse for police statistics under the Uniform Crime Reporting Program, and a computerized nationwide index of law enforcement under the National Crime Information Center (NCIC).

In addition, the National Incident Based Reporting System (NIBRS)6 was formulated under the Uniform Crime Reporting Program. Law enforcement agencies report crimes within their designated regions and the data is compiled into the UCR. This data collection responsibility is handled by the Criminal Justice Information Services Division (CJIS) within the FBI.

The legislative mandate for the ATF’s responsibility to collect and maintain information on arson and explosives incidents is specific and clear. The legislation affecting the FBI is less specific, and the authorization for the BDC to collect and report such data is not as clear and is only implied by the Uniform Crime Reporting Act.

Comparison of ATF and FBI Database Systems

As noted earlier, the purpose of the ATF’s and FBI’s databases is to collect, and make available to the law enforcement community, information related to incidents of bombing or arson. To determine how the ATF and FBI achieve this purpose, we examined the ATF’s AEXIS and BATS systems and the FBI’s AIRS system and compared the accessibility, data collection features, user interfaces, data fields, types of outputs, and data sharing capabilities of each. In doing so, we identified overlapping and unique features, and where possible, we noted system advantages and weaknesses. We also tested system accuracy of the ATF’s and FBI’s databases, and assessed the timeliness of data entry and response time to inquiries by system customers. A summary of the features of each database is found in Appendix IV.

Accessibility

AEXIS. The ATF’s AEXIS system was developed for arson and explosives specialists to assist in the collection, maintenance, evaluation, and dissemination of information for determining criminal activity patterns, trends, and motives. The AEXIS system is an enhancement of the ATF’s Explosives Incident System, which was developed in 1974 and implemented in 1975. AEXIS contains information on over 70,000 arson cases and 40,000 bombings dating back to the early 1920s.

The AEXIS system is available primarily to ATF Repository staff. Some Repository staff members have administrator privileges, which enable them to change information within the system. In each of the ATF field divisions, one person is authorized to have limited access to AEXIS, and that person is designated as the contact person for the whole field division. A web-based access system is available for authorized state and local agencies to have read-only rights to parts of the AEXIS system, whereby queries can be performed to obtain statistical information. Also, state and local law enforcement agencies may contact the Repository to obtain information, but direct access to AEXIS is not available to individuals outside of the Repository.

Comments we received from system users regarding accessibility indicated they would like to have greater online access and a secure system to download information via the Internet.

BATS. The ATF’s BATS system is a web-based, Oracle database that provides ATF and state and local law enforcement agencies with access to real-time nationwide information on fire investigations, arson, bombings, and other criminal misuse of explosives. The BATS system was developed by ATF, with participation by law enforcement agencies responsible for fire, arson, and post-blast investigations. BATS has been operational since October 2003, contains about 1,280 records, and has about 140 users at two sites. Law enforcement agencies can use BATS in one of two ways: as a supplement to an existing records management system or as a new records management system.

BATS is made available to participating agencies through a software application that uses a secure Internet connection. Mandatory provisions for connectivity to the BATS application require that: 1) only government owned or leased computers are used to establish communication with the BATS software application, 2) all connections established with the BATS software application are made through a physical wire connection, and 3) under no conditions can connections be established through any type of wireless communication appliance or medium without written approval from ATF.

BATS is made available to law enforcement agencies that meet the following requirements:

The agency must be a duly constituted, empowered, and regulated law enforcement agency, and the agency must have responsibility for the investigation of incidents involving arson and the suspected criminal misuse of explosives.

The agency and the members of that agency that access the BATS system must have current National Crime Information Center (NCIC) access and user privileges.

The agency must have an originating agency identifier (ORI) number.

In addition, agencies requesting access to BATS must:

Submit a letter of interest from the Department Chief on departmental letterhead requesting BATS access,

Complete and submit to ATF the required access forms,

Participate in BATS security and operational training, and

Sign a memorandum of understanding (MOU) acknowledging participation in the BATS system.7

AIRS. The FBI’s AIRS system is a web-based, Oracle database that tracks bombing incidents and activities in the United States and was designed by the BDC to meet the needs of explosives reporting and data warehousing. AIRS has been in operation since January 1998 and is now fully operational, with the exception of scheduled upgrades. AIRS is considered by the FBI to be the main method of communicating intelligence between FBI Special Agent bomb technicians and state and local accredited bomb squads nationwide.

AIRS is available nationwide via the secure Law Enforcement Online (LEO) network,8 which is accessed through a virtual private network.9 Within LEO, Special Interest Groups (SIG)10 are established by the FBI for users who meet certain criteria. Users may access the AIRS website by using a valid LEO username and password and be a member of an appropriate SIG.

SIG membership requirements for the bomb technician level are:

Users must be graduates of the FBI Hazardous Devices School at Redstone Arsenal in Huntsville, Alabama.

Users must be graduates of the United States Naval School Explosives Ordinance Disposal (NAVSCOLEOD).

Users must be active bomb technicians that have been verified by the BDC.

SIG membership requirements for investigator access are:

Users must be assigned by an agency to conduct bombing and/or arson investigations.

Users must submit a letter to the BDC on agency stationary from a superior officer identifying the user as being responsible for bombing and/or arson investigations.

Users must be active bomb technicians, but need not be graduates of the FBI Hazardous Devices School or NAVSCOLEOD. In addition, users must submit on departmental letterhead identifying themselves as an assigned bomb technician for the department.

Criteria for members who have investigator access must be verified annually by the BDC.

SIG membership requirements for current LEO members are:

Users must fill out forms identifying their name, agency, and date of graduation from the Hazardous Device School.

Users must send a letter to the BDC identifying their name, agency, date of graduation from NAVSCOLEOD (for bomb technician access), or a letter from their department verifying they have bombing investigation responsibilities.

Comments we received from some system users indicated that they would like to be able to access data without having to go through a “middle man,” and that it is very cumbersome to get into LEO.

The ATF’s BATS and FBI’s AIRS overlap significantly. They perform similar functions and the same types of agencies are eligible to participate in either system. In our judgment, the ATF’s BATS database has the potential to be the easiest database to access by eligible users. AEXIS is very restrictive, and access to AIRS via LEO, which is less restrictive than AEXIS, can be difficult. However, as of the time of our review, BATS was in use at only two agencies, and only two more were expected to be using BATS by the end of 2004.

Data Collection

AEXIS. Data is collected for the ATF’s AEXIS system electronically or in hardcopy format. Repository staff perform an electronic transfer into AEXIS by using N-Force,11 a case management system used by ATF agents. Data transfer is performed via the tools menu located on the task bar within AEXIS. Prior to acceptance of data, Repository staff have the option of reviewing the transferred data and determining whether to include the data in AEXIS.

State and local agencies provide data via fax, mail, e-mail, and reports of investigations. In addition, data is collected from such sources as forensic lab reports from ATF chemists, bomb tech reports, photograph devices from crime scenes, evidence from the ATF laboratory in Maryland, and telephone calls from ATF forensic laboratory agents.

Comments we received from system users regarding data collection were largely positive, such as “no room for improvement,” “no complaints,” and “timely, efficient.” Some responders indicated, however, that ATF staff should be more readily available by telephone and that better communication is needed.

BATS. Data is collected for the ATF’s BATS system from investigators at the federal, state, and local level. There is no requirement to submit data in a specific format, as BATS accommodates virtually any type of data. Investigators may enter data from printed reports from their respective agency’s case management system, or data may be entered into BATS directly, while being used simultaneously as a case management system. In addition, investigators can enter data from notes taken during the investigation of an incident. Passwords and user identifications are necessary to access BATS. No data is entered by Repository staff. All data is entered by law enforcement agencies outside the Repository.

AIRS. Data is collected for the FBI’s AIRS system electronically or in hardcopy format via mail or fax. Authorized members of the BDC/SIG may enter bombing incidents and bomb squad activities directly into the system. Also, authorized LEO members may enter data on their incident and activities directly into AIRS. Further, electronic data from other database platforms such as Microsoft Access or Oracle can either be imported or exported into AIRS by BDC personnel. Future plans are for an “email to fax service” within AIRS, which will give reporting agencies the ability to electronically transmit their completed incident and activity report forms to the BDC.

All hard copy submissions of data must be reported on either an incident reporting form (FD-873) or the activity report form (FD-873a). Incident reports contain information on arson and explosives incidents that are investigated by the reporting agency and forwarded to the BDC. They provide significant details about specific occurrences. Activity reports contain information reported by bomb squads that may not be related directly to actual arson and explosives incidents but provide details of investigations of suspicious activities, devices, or threats. Only Special Agent Bombing Technicians or members of a bomb squad are given the choice to create activity reports within AIRS.

Comments we received from system users were mixed. Some were satisfied with the job the BDC was doing. Others were somewhat negative, indicating that the BDC could use more staff, information was not always current, and statistics were helpful but not timely.

In our judgment, a system in which data can be entered on-line directly by the reporting agency, such as the BATS system, has the advantage of furnishing real-time data. Such a system can also minimize errors that occur when a second party transcribes information from a source document.

User Interface

AEXIS. Data in the ATF’s AEXIS database can be entered or obtained through two forms of user interfaces. One interface addresses data elements that can be entered or obtained only by Repository staff. Outside agencies must contact Repository staff regarding such data. The second form addresses data elements that are available to all agencies that have read-only rights to use the web-based querying and reporting feature of AEXIS.

The user interface for Repository staff is patterned after Windows Explorer: it uses an expandable file structure for data storage. The file structure allows the user to browse all records that are part of a particular incident and view the contents of the record. The elements of this portion of the user interface for the AEXIS system provides numerous ways that Repository staff can interact with the system for productive use. Examples of elements follow:

Incident selection browser that lists all incidents within a given period, with querying options

The web-based querying and reporting interface offers users the capability to query AEXIS and view incident-related photographs, and provides data administration functions for reporting. The elements of the imaging and data administration are as follows:

Four querying tools that pull data and provide views of data from five data tables

Querying tools that pull data from seven tables within the National Fire Incident Reporting System (NFIRS)12

Query reporting systems that incorporate data from different databases: AEXIS, NFIRS, and the Uniform Crime Report13

BATS. The ATF’s BATS system uses a Java based web browser designed to allow the user to navigate the system easily. Examples of user interface elements are:

Incident maintenance to create and update records

Incident search by incident, subject, or device

Export agency information to other users

Restricted/Unrestricted option for sharing data

Geographic information system

Edit features

AIRS. The FBI’s AIRS user interface is built in a web format, whereby the features of AIRS can be accessed through easy to understand web-style controls. The elements of the user interface for the AIRS system provide numerous ways that the user can interact with the system to use it productively. Examples of interface elements are:

Navigational buttons for moving from one section to another in a particular report

Validation indicators per work section prior to completion of the entry of a report

Search modes to perform simple and advanced searches

Required field denotations for an agency’s name, city, state, zip code, and bomb squad identification number

Duplicate entry detection

Computer generated report numbers

Each of the database systems has unique user interface features. Based on our review of the systems and feedback from the users, we did not conclude that the user interface features of any one system to be better than the others.

Data Fields

The most significant instances of duplication of effort were disclosed by our review of the data fields for each of the database systems. We found that the ATF’s AEXIS and BATS systems, and the FBI’s AIRS system collected 11 principal types of information:

Date of incident

Time of incident

Type of incident

Address of incident

Target type

Motive

Method of delivery

Incident summary

Incident involvement

Evidence components

Reporting agency name, address, telephone number

Charts outlining each of the data fields for the ATF’s AEXIS and BATS database systems, and for the FBI’s AIRS database system are found in Appendix V.

In our judgment, a single, consolidated database that includes each of the data fields currently provided by AEXIS, BATS, and AIRS would effectively capture all necessary information concerning incidents of bombings or arson. In addition, we noted that AEXIS also includes an explosives tracing feature and a theft of explosives feature, as described earlier in this report, which can provide useful information for linking thefts of explosive materials with criminal misuse of the explosives. Such data should also be captured by the consolidated database system.

Types of Outputs

AEXIS. The ATF’s AEXIS system has the capability of generating automated statistical summary reports from a given data range, standard reports per record number, tracing reports, and canned reports based on data extractions from AEXIS, NFIRS, and the UCR.

BATS. The ATF’s BATS system provides output via the following reports:

Incident Reports that capture all data entered on an incident;

Audit reports that allow agency managers to see what the users at a particular agency have entered at any given time; and

Management reports that provide a manager of an agency with a variety of measures by which an assessment can be made of an agency’s performance.

All BATS reports include both Portable Document Format (PDF) and HyperText Markup Language (HTML) formats and can be generated from the main menu within the BATS system.

AIRS. The FBI’s AIRS system is capable of generating multiple output formats in PDF, HTML, or Excel. Reports can be generated using various scenarios, as well as simple or customized reports upon request.

Our review of the systems and feedback from the users did not show a discernable difference among the outputs of the three systems.

Data Sharing Capabilities

AEXIS. Data sharing capability within the ATF’s AEXIS system is limited to authorized members of the Repository staff and the designated contact person at each ATF field division office. Requests for information from state and local agencies can be received by fax, telephone, and emails. Repository staff respond to such requests to law enforcement agencies either by mail or telephone. In addition, approved law enforcement agencies may query the AEXIS system in order to obtain information from AEXIS, NFIRS, and the UCR, individually or collectively.

BATS. The ATF’s BATS system contains a search feature that allows law enforcement agencies that have access to BATS to share real time data nationwide. Agencies can export their own agency’s data to other agencies. In addition, if an agency is using BATS as an inter-agency case management system, information can be designated as restricted or unrestricted within BATS.

AIRS. Data can be shared from the FBI’s AIRS system with authorized SIG members. Requests for information are received by mail, fax, and e-mail from non-SIG law enforcement agencies. Such requests are reviewed by BDC staff, which determines what information is available at BDC. The BDC staff then forward information to the requesting agency.

In our judgment, the accessibility of BATS users to one another’s data is clearly an advantage in investigating incidents of bombing and arson. We believe that to be effective, a database should have this data sharing capability.

Accuracy

We found significant differences in the accuracy of the ATF and FBI databases when we compared system output to information obtained from source documents.

AEXIS. We tested the accuracy of the: 1) intelligence information in the ATF’s AEXIS database, which included the theft and loss feature within AEXIS; and 2) tracing feature within AEXIS. For both of these tests, we compared a sample of database output to source documents. We drew our sample from a computer-generated list of documents for October 2001 - October 2003 furnished by the ATF.

We identified from the ATF report a universe of 1,361 input documents for AEXIS for the review period. We judgmentally selected a sample of 10 percent of the documents for verification, or a total of 136 documents. We were able to locate 64 of the documents,14 which contained a total of 1,562 data entry fields. We found errors in 10 data entry fields, or 0.6 percent. The following summarizes the incidence of errors in AEXIS, by category:

AEXIS

Number of Errors

Incident Time

5

Incident Data

1

Incident Address

1

Agency Address

2

Motive

1

Total Number of Errors

10

The error rate we observed for the AEXIS database was very low and ATF officials were able to correct each of the above errors during the time of our review.

In addition, we examined the Explosives Tracing feature of AEXIS. We identified from an ATF computer-generated report a universe of 261 records within our review period. We judgmentally selected a sample of 10 percent, or 26 total records, and found
16 errors.15

An ATF official stated that the ATF checks about 10 percent of the data entries for accuracy. The ATF officials stated that some of the errors we found resulted because of the limited selection of options available within the AEXIS drop-down menus. This system weakness limits the options that staff can choose to enter data from source documents. Thus, the options do not always adequately describe the details included on the source documents.

BATS. The ATF’s BATS system began operating in calendar year 2003. The Maine State Fire Marshals was the first law enforcement agency to gain access to BATS information sharing. During our review, the Glendale, Arizona Fire and Police Department also deployed BATS.

We were able to test the accuracy of information in the BATS database by comparing a sample of system output to case files at the Glendale location, where we drew our sample from a computer-generated management report furnished by the Glendale Police Department for the period January 2001- February 2004. Staff of the Maine State Fire Marshals Office did not prepare or maintain source documents for entering data into BATS regarding bombing and arson incidents. Instead, field staff entered all information directly into laptop computers used on-site. After review by supervisors, the Maine data was then uploaded directly to the BATS server. Therefore, we were unable to verify the accuracy of system output at the Maine location by comparing it to source data.

At Glendale, we found that entries in 3 of the 244 data fields we tested, or 1.2 percent, contained errors because data was entered into the system incorrectly. Errors occurred in transcribing the zip code, the age of the subject, and the estimated damage caused by fire.

In addition, we noted at Glendale that not all data entered into the system was related to bombing and arson incidents. Contrary to the provisions of the MOU with the ATF, 5 of the 18 incidents were unrelated to bombing and arson, such as fires caused by malfunctioning household appliances. The Glendale’s MOU with ATF states, “This information system will be used by the parties for the collection, analysis, and dissemination of incidents involving arson and the suspected criminal misuse of explosives through ATF’s Arson and Explosives National Repository Bomb Arson Tracking System (BATS) software application.”

Glendale officials told us their procedures had been discussed and cleared with ATF staff. Further, one Glendale official believed that all incidents should be included in BATS because a case initially identified as an accident could later become a case of suspected bombing or arson. Additionally, ATF staff had agreed that an incident may become an arson or explosives incident with criminal intent, and that the use of BATS should be more inclusive, rather than exclusive. In our view, however, populating the BATS system with data that is clearly unrelated to bombing and arson incidents is contrary to the intent of BATS and may result in time wasted by investigators who needlessly access and review this data.

AIRS. Authorized federal, state, and local law enforcement agencies that have access to LEO entered bombing incidents and bomb squad Activity Reports directly into LEO. This data is then used to populate the BDC’s AIRS database. Incident and Activity Reports submitted by federal, state, and local law enforcement agencies to the FBI’s BDC are put into the system by BDC staff. Such reports are received by the BDC electronically, by mail, and by facsimile.

The FBI has a contract with LSU to enter data in LEO to assist the BDC to reduce its data backlog. According to LSU officials, contract staff have not received any formal policies or procedures for receiving data. LSU officials stated that whenever they need documents to enter, they simply call BDC officials to request that additional documents be sent to them. Also, there are no performance standards for timely entry. Moreover, we found that LSU had data entry instructions that had not been updated to reflect system upgrades.

The 351 FBI reports contained a total of 7,558 data fields. Of these data fields, 730 (9.7 percent) contained errors. A summary of the incidence of errors follows:

Incident Reports

Incident Reports contain information on arson and explosives incidents that are investigated by the reporting agency and forwarded to the BDC or entered directly into AIRS. They provide significant details about specific occurrences.

Reporting Agency Data: (397 errors) These errors are significant because a law enforcement agency seeking additional information relating to an investigation might receive incomplete or inaccurate information and might be unable to easily contact the reporting agency.

Reporting Agency Data

Number of Errors

Missing Data

31

Inaccurate Data

Agency name

11

Agency city

24

Agency street

50

Agency state

5

Agency telephone number

132

Agency fax number

117

Bomb Squad ID

16

Responding agency number

5

Investigating agency number

2

Agency zip code

4

Total Number of Errors

397

Incident Type: (2 errors) This data field describes the type of incident reported, such as “recovered explosives,” “bombing,” and “hoax.” We found two instances in which the type of incident was listed incorrectly. Although the error rate for this category was low, it is significant because an investigating agency seeking assistance on all incidents of a particular type would be unable to obtain complete information. In addition, statistical reports on the categories of incidents may be over or understated.

Incident Data

Number of Errors

Bombing Data

1

Recovery of IED

1

Total Number of Errors

2

Incident Details: (61 errors) As stated above, the errors in this category are significant because a law enforcement agency seeking data relating to an incident might not receive complete and accurate information to assist with its investigation.

Incident Details

Number of Errors

Missing Data

41

Inaccurate Data

Incident start date/time

5

Incident end date/time

2

Incident street address

4

Incident county

2

Incident zip code

1

Target

1

Apparent Involvement

2

Narrative

2

Total Damage

1

Total Number of Errors

61

Device Data: (52 errors) This data field describes the type of device used in the incident, such as detonators, pipe bombs, and hoax device. The most significant number of errors we found in this category pertained to missing data. In 45 instances, we noted that information on the type of device used was included on the Incident Report but not entered in the system. In our view, this information is critical to investigations and its absence could adversely affect the progress of investigations.

Device Data

Number of Errors

Missing Data

45

Inaccurate Data

Category

4

Device Type

1

Subtype

1

External Container

1

Total Number of Errors

52

Activity Reports

Activity Reports contain information reported by bomb squads that may not be related directly to arson and explosives incidents but that provides details of investigations of suspicious activities, devices, or threats. Because of their investigative value, errors in data entry could have adverse effects similar to those described for Incident Reports above. The following tables cite the incidence of errors we noted in the sampled Activity Reports we reviewed:

Reporting Agency Data: (133 errors)

Reporting Agency Data

Number of Errors

Agency name

3

Agency street

15

Agency city

10

Agency state

5

Agency zip

4

Agency telephone number

44

Agency fax number

52

Total Number of Errors

133

Activity Type: (5 errors) This data field describes the type of activity performed by reporting agency staff, such as attending training, examining equipment, and disposing of explosives.

Activity Type

Number of Errors

Protective Detail

5

Activity Data: (80 errors)

Activity Data

Number of Errors

Activity start date/time

40

Activity end date/time

40

Total Number of Errors

80

The BDC personnel stated that errors occurred because there is a feature within the telephone and fax number fields that automatically completes information based on the law enforcement bomb squad’s identification number. They said those errors could have occurred for the following reasons: 1) agents included personal contact information on the source documents, and 2) information for the specific identification numbers changed but was not updated in the system. The BDC did not have an explanation for the remaining errors.

The LSU personnel believed that missing data occurred when the BDC upgraded the system and data was not transferred to the new system. Additionally, errors found in the incident time category were caused because the LSU staff did not enter data included on source documents.

Data entry errors occurred even though BDC technical staff stated they edit and perform quality assurance measures on all data submitted, and that every report received via the mail or data fax was reviewed and edited prior to data entry.17

Our tests showed that the FBI’s error rates were significantly higher than the ATF’s. We attribute this condition to system limitations and a lack of an effective second-party review system. However, the FBI is moving toward increased data entry on-line by reporting agencies. In our judgment, direct entry of data on-line should improve database accuracy.

Timeliness of Data Entry and Responsiveness to Customers

Based on our examination of available data and interviews with ATF and FBI customers, we did not find any significant problems concerning timeliness of responses. However, we found that the FBI had a significant backlog of Incident Reports and Activity reports that had not been entered into its AIRS database.

AEXIS. We reviewed a judgmental sample of ATF AEXIS report data to determine the length of time from receipt of data to the date of entry into the ATF database. The ATF staff stated that information is entered as soon as it is received. However, this could not be verified because the ATF did not date stamp documents. Further, the system did not have automated indicators to show when the data is entered into AEXIS.

We relied upon our interviews with agency officials who contacted the ATF to determine whether ATF responses were timely. None of the agency officials we interviewed complained about the ATF’s timeliness. Specifically, the agency officials who contacted ATF stated they received the information requested as follows:

How long did it take you to receive the requested information?

Agency Response

Immediately

7

Within 1 day

2

Within a week

13

A week or more

1

Based on these responses, it appeared that the ATF is responding to customer requests quickly. However, the ATF had not established controls, such as recording the date requests are received and responses are sent out, to monitor whether requests received were promptly answered.

BATS. Because users of ATF’s BATS system enter data directly on-line and have access to one another’s data, there is no delay in either data entry or retrieval.

AIRS. We found that the FBI’s BDC had a large backlog of source documents containing data that had not been entered into the AIRS. In August 2003, data from at least 5,745 Activity Reports and 770 Incident Reports, a total of 6,515 documents, was more than 4 years old and still had not been entered into the system. We noted that
reports sent to LSU were kept at the BDC until they had a full box of forms and that the BDC kept no record of the forms sent to LSU. According to LSU staff, however, as of July 2004 the backlog had been eliminated. On August 19, 2004, FBI officials indicated the contract with LSU was terminated.

We were unable to determine how quickly the FBI responded to requests for information because requests were not date stamped on receipt. However, the requesting agency officials we interviewed told us the FBI responded timely to their requests.

How long did it take you to receive the requested information?

Agency Response

Immediately

3

Within 1 day

2

A week or less

4

A week or more

1

Neither ATF nor FBI customers complained about the timeliness of the current systems. If the ATF and FBI consolidate their databases, and system users do data entry and retrieval from the database system directly on-line, we believe that timeliness should continue to be satisfactory.

ATF, FBI, and Departmental Efforts to Define Responsibility for Data Collection and Management

On November 15, 2001, the ATF issued Notice of Proposed Rulemaking No. 933, which proposed to amend 27 CFR Part 55, a federal regulation governing the activities of the ATF. The ATF proposed to require all agencies having information concerning incidents involving arson and the suspected criminal misuse of explosives, from whatever source received, to the ATF. The term “agency” was defined in the proposed regulations as each of the executive agencies and military departments, and the United States Postal Service. The proposal also specified the minimum types of data to be furnished to the ATF, which included “general information about the …incident.”

The FBI disagreed with the proposed rules on several points. In its February 12, 2002, response, the FBI stated that: 1) it also had responsibilities for compiling data, and 2) that it retained the authority and responsibility for obtaining data concerning bombing incidents and other crimes involving explosives, including collection from other federal agencies. Additionally, the FBI stated that since its data compilation responsibilities may encompass the same data addressed by the ATF’s proposed rule, the ATF should clarify in the regulation that the ATF did not supplant concurrent data compilation by the FBI. The FBI was also concerned about the proposed rules’ definition of “from whatever source.” Specifically, the FBI recommended three changes:

Adding “Nothing in this regulation shall be construed as modifying or otherwise affecting in any way the authority of any other federal agency, including the FBI’s National Bomb Data Center.”

Changing “having information on incidents” to “having information on incidents derived from operations for which the agency exercises primary responsibility,” and changing “must report the information” to “must report appropriate information.”

In our view, the FBI did not want to relinquish its responsibilities for collecting data and countered the ATF’s proposal by amending the wording of the proposed rule so that it, in effect, retained those responsibilities.

At the time of our review, the proposed rules were still pending. In July 2003, the ATF drafted a proposal to merge the common functions of the Repository and the BDC. Specifically, the ATF proposed to: 1) eliminate the cost of the ongoing maintenance and support of two reporting systems, 2) provide more efficient reporting procedures for state and local law enforcement agencies, and 3) maximize customer and stakeholder benefits by making full use of the Repository’s capabilities. To achieve these objectives, the ATF recommended that the information and publication functions of the BDC be merged with the Repository as a single entity within the Department, and managed by the ATF.

Subsequent to the ATF’s proposal, on March 4, 2004, the Attorney General created the Explosives Review Group (ERG) to identify options and make recommendations to ensure effective coordination of the Department’s efforts concerning explosives. The Attorney General directed the ERG to be chaired by a senior level representative from ATF, and to include senior level representatives from the Office of the Deputy Attorney General (ODAG), the FBI, the Criminal Division, and the Office of Legal Policy. Among the topics to be addressed by the ERG was the coordination and possible merging of ATF and FBI databases relating to explosives investigations. The ERG was to submit options and recommendations to the ODAG by May 4, 2004, and the ODAG was to present options and recommendations to the Attorney General by June 4, 2004. As of July 14, 2004, the ODAG had not submitted its recommendations to the AG, but indicated to us that the ATF and FBI agreed in principle that the databases should be consolidated. On August 11, 2004, the Attorney General directed: 1) the ATF and the FBI to consolidate all of the Department’s arson and explosives incidents databases, including, but not limited to, the ATF’s BATS, and the FBI’s AIRS, into a single database; and 2) that all consolidated arson and explosives incident databases be maintained by the ATF (See Appendix VI for details of the Attorney General’s August 11, 2004 directive).

Conclusion

Both the ATF and the FBI compile and disseminate data related to arson and the illegal use of explosives. This has resulted in duplication of effort by the ATF and the FBI and duplicate reporting of incidents by non-federal agencies. It also has resulted in confusion and a lack of consistency in the reporting process. This condition stems from statutory and regulatory overlaps concerning the responsibility to compile data, and continues to exist because of a lack of concurrence between the ATF and the FBI on how to ensure that duplication is avoided and uniformity is achieved.

The FBI’s error rate for entering data into AIRS far exceeded the ATF’s error rate for entering data into AEXIS. However, as the FBI migrates from data entry by BDC and LSU staff to data entry by reporting agencies, the error rate should decrease. The accuracy of the ATF’s BATS system could not be fully assessed at the time of the review because it was still in the pilot stage and implemented in only two locations. The error rate for the BATS data we were able to review was very low. However, we found that much of the data entered was unrelated to bombing and arson incidents, and therefore contrary to the purpose of BATS.

We were unable to fully assess the timeliness of the ATF and FBI systems because neither was recording the dates that information was received from reporting agencies. Most agencies, however, responded favorably regarding the ATF’s and FBI’s timeliness on their requests for information. The most significant condition we observed regarding timeliness was the FBI’s large backlog of Incident Reports and Activity Reports that had not been entered into AIRS, and the lack of controls for tracking the reports that were sent to LSU. The BDC simply sent boxes of Incident and Activity Reports periodically to LSU for data entry. This condition, according to FBI staff, was improving. Again, migration to direct entry by reporting agencies would alleviate this condition and prevent its recurrence.

To provide the law enforcement community with the most efficient and effective means of collecting arson and explosives intelligence, we believe that one reporting system should be established, which would eliminate duplication of effort and the confusion as to which system to use and rely on for such information. Ensuring accessibility to the system by appropriate users would enhance the ability of federal, state, and local law enforcement agencies to share valuable intelligence information about arson and explosives incidents and devices.

Based on our review of the databases currently managed by the ATF’s Repository and the FBI’s BDC, we compiled a list of suggested features, which we list below, that a consolidated system should have for reporting and sharing arson and explosives information within the law enforcement community. No one system we examined had all of these features.

We also found that both the ATF and FBI have encountered problems in maintaining their respective databases. The management of any consolidated system plays an important role in the effectiveness of data collection and sharing. We believe that the management of such a consolidated system should include the establishment of policies and procedures for collecting and disseminating data, responding timely to requests, ensuring the information is current and accurate, setting minimum access requirements for potential end users, and advertising the system to the law enforcement community. In our judgment, a single database having the features listed above with established policies and procedures would result in the most efficient and effective means of collecting and disseminating arson and explosives information.

Recommendations

We recommend the Department:

Consolidate the Repository and BDC databases under ATF
management in accordance with the Attorney General’s August 11,
2004, directive, in order to eliminate duplication of effort, ensure
consistency in reporting, and facilitate sharing of intelligence among
eligible law enforcement agencies.

Pending the implementation of Recommendation 1, we recommend the ATF:

Ensure that BATS users adhere to the MOU and require that data used to populate the BATS database involve only arson and the suspected misuse of explosives.

The NIBRS is an incident-based reporting system through which data is collected on each single crime occurrence. The data is designed to be generated as a by-product of local, state, and federal automated records systems. The NIBRS collects data on each single incident and arrest within 22 offense categories made up of 46 specific crimes.

Law enforcement agencies interested in acquiring access to BATS have to enter into an MOU which reads as follows: “This MOU is entered into by the United States Department of Justice, Bureau of Alcohol, Tobacco, Firearms, and Explosives (ATF), and the_____________, hereinafter collectively referred to as “the parties.” This MOU establishes and defines a partnership between the parties that will result in the operation and administration of an online secure Internet based intelligence and information system. This information system will be used by the parties for the collection, analysis, and dissemination of incidents involving arson and the suspected criminal misuse of explosives through ATF’s Arson and Explosives National Repository Bomb Arson Tracking System (BATS) software application.”

LEO is a national interactive computer communications system and information service exclusively for the law enforcement community and can be accessed by an approved employee of a duly constitutional local, state, or federal law enforcement agency, or an approved member of an authorized law enforcement special interest group. The system provides a vehicle for these communities to exchange information, conduct on-line education programs, and participate in professional special interest and topically focused dialog. The information contained within LEO is for the sole use of members and may contain confidential and privileged information.

A virtual private network is a security structure that utilizes a program to provide secure, encrypted channels between LEO users and the applications and data contained with the LEO network.

SIGs are segmented areas with controlled access for specialized law enforcement groups. SIGs represent a special discipline area within the law enforcement profession and have their own members.

N-Force is a case management system that supports the law enforcement operations of ATF; information within N-Force is entered once and can be used in multiple areas throughout the system.

The National Fire Incident Reporting System, maintained by the United States Fire Administration, represents the world’s largest national, annual database of fire incident information, whereby local and state agencies submit incident and casualty reports as fires occur in their respective areas.

The Uniform Crime Report is a system of collecting and analyzing crime statistics gathered on selected crimes by participating law enforcement agencies throughout the United States.

We were able to locate only 64 documents because ATF staff had destroyed the remaining 72 documents. According to ATF staff, they destroy documents at the end of each calendar year because of storage problems.

The Explosive Tracing feature of AEXIS does not generate a report; therefore we were unable to determine the number of applicable fields to calculate an error rate.

Only LSU staff entered data into AIRS from Activity Reports.

Quality assurance examples include: reviewing reports for spelling, variances in types of devices used with description included in the narratives of reports, and determining whether the correct forms were used for reporting data to the BDC.

On August 19, 2004, FBI officials indicated they did not want to include subject and suspect data, or data related to juveniles due to Privacy Act and other concerns.