Follow-Up Audit of the Bureau of Alcohol, Tobacco, Firearms and Explosives
Forensic Science Laboratories Workload Management
Audit Report 06-15
Office of the Inspector General
The Bureau of Alcohol, Tobacco, Firearms and Explosives’ (ATF) forensic science laboratories support the ATF’s mission of enforcing federal criminal laws relating to firearms, explosives, arson, alcohol, tobacco, and the regulatory functions associated with the firearms and explosives industry. In the course of their investigations, ATF Special Agents submit to the laboratories for the purpose of forensic analysis evidence that includes firearms, explosives, fire accelerants, fire devices, and debris from explosives and fire scenes.
The ATF’s three regional forensic laboratories are located near Washington, D.C., and San Francisco, and in Atlanta. The laboratories are units of the Office of Laboratory Services, which is part of the ATF’s Office of Science and Technology. Each of the regional laboratories primarily serves an assigned group of states. In fiscal year (FY) 2005, the ATF laboratories performed over 2,600 forensic examinations on evidence from crimes involving arson, explosives, and firearms, with an authorized staff of 106 positions and a budget of approximately $16 million.
The ATF laboratories’ forensic chemists, firearm and toolmark examiners, fingerprint specialists, and document examiners (who we collectively refer to as examiners) support the ATF’s 23 field divisions by performing and reporting the results of forensic examinations in the disciplines of:
In addition to providing forensic services, examiners perform duties outside the laboratories, such as providing training, crime scene assistance, and court testimony.
This audit was performed by the Department of Justice, Office of the Inspector General (DOJ OIG) as a follow-up to the Department of the Treasury Office of Inspector General (Treasury OIG) report OIG-01-068, CRIMINAL ENFORCEMENT: ATF Forensic Science Laboratories Need to Improve Workload Management, issued in April 2001. The DOJ OIG is following up on an audit performed by the Treasury OIG because the ATF was transferred to the Department of Justice in January 2003 as a result of the Homeland Security Act.
The Treasury OIG audit found that the ATF laboratories did not always provide timely service, did not properly prioritize workloads, and did not consistently follow established case file controls. The Treasury OIG recommended that the ATF ensure that laboratories are adequately staffed, work performed outside the laboratory is coordinated, incoming evidence is better prioritized, and management controls are followed. In its response to the findings and recommendations, ATF management substantially agreed with the Treasury OIG and identified specific corrective actions it would take to address the issues raised.
For the DOJ OIG audit, our objectives were to determine whether the forensic laboratories were providing timely service, effectively prioritizing workloads, and following two specific case file management controls that were identified in the Treasury OIG audit as weaknesses. The weaknesses identified by the Treasury OIG were that not all closed case files contained evidence control cards as required and that laboratory analysts were not always recording the number of hours spent performing examinations and writing reports on the evidence control cards. In conducting this audit, we focused on the regional forensic laboratories that analyze and maintain custody of evidence submitted by the ATF field divisions. We interviewed officials from the ATF’s Office of Laboratory Services, managers and staff at each of the three laboratories, and Special Agents who had requested laboratory services. We also reviewed records at each of the three forensic laboratories and analyzed workload data published by the Office of Laboratory Services.
Timeliness of Completed Exams
The Treasury OIG used a 30-day standard to define timely service in its audit report in 2001. The formal performance standard for the Office of Laboratory Services for 2005 calls for requested examinations to be completed within 30 days in 30 percent of firearms cases, 35 percent of explosives cases, and 40 percent of arson cases during FY 2005. The Director of Laboratory Services told the DOJ OIG that he has adopted a long-term goal to complete all examinations within 30 days of receiving the evidence. We found that a 30-day turnaround time is commonly identified by the forensic community as a standard that forensic laboratories should try to achieve, but that it is generally not met for 100 percent of examinations because of resource constraints.
We used a 30-day standard for this audit to compare our findings with those of the Treasury OIG and to determine how far the ATF is from meeting this long-term goal. We also consider 30 days a reasonable goal toward which laboratories should work because of the potentially serious consequences of delayed forensic examinations. These effects include the costs of wasted investigative time and delayed trials, and the more serious possibility that additional crimes may be committed by offenders who are not identified and arrested quickly.
To evaluate timeliness, we obtained and analyzed data from the Office of Laboratory Services’ Forensic Automated Case and Exam Tracking System (FACETS), a database system used by the ATF’s regional laboratories to track and manage submitted forensic evidence.1 The data included all forensic examinations that were completed from October 1, 2003, through May 13, 2005, or were pending at that time. As shown in the following chart, we found that 2,871 examinations (63 percent) were not completed within the 30-day standard. Of the 2,871 examinations not completed within 30 days, more than half took more than 90 days to complete. The average turnaround time for all examinations was 95 days.
Examination Processing Time
In its 2001 report, the Treasury OIG attributed the laboratories’ processing delays to the large backlogs of examination requests, an inability to hire sufficient staff, and the staff’s other duties outside the laboratories that competed for their time, such as providing instruction and crime scene assistance. In general, the DOJ OIG found that these same factors contributed to the laboratories’ average processing times and that the ATF had not implemented some of the corrective actions it proposed in response to the Treasury OIG’s report.
The DOJ OIG found that the number of backlogged examinations had decreased since the Treasury OIG audit, but the backlog still remained significant. The DOJ OIG audit compared the backlog reported in the Treasury audit to FY 2005 backlog figures, and found that the total backlog of 1,289 examinations had been reduced to 983 examinations. The DOJ OIG determined that 64 percent of pending examinations represented in the recent data were more than 90 days old, 47 percent were more than 180 days old, and 23 percent were a year or more old. We further analyzed the backlog by type of examination to determine if the number of cases in specific disciplines had increased or decreased, and how long it would take to eliminate it. We found that the backlog varied by examination type from approximately 2 to 8 months in terms of workload. The backlog was increasing for arson and questioned documents and was decreasing for explosives, firearms, fingerprints, and trace examinations. For examinations decreasing backlogs, it would take between 6 and 10 years to eliminate the backlog if the laboratories continued to complete examinations at the FY 2005 monthly rate.
Following the Treasury OIG audit, ATF management planned to provide each field division with a list of cases with pending examinations so that field divisions could identify cases that could be deleted from the backlog. According to the Treasury OIG report, the ATF management had provided each field division with a list of cases that were at least a year old as of May 1999, which resulted in 94 inactive cases being removed from the laboratories’ backlogs. The ATF repeated this process in November 2000, and removed an additional 85 cases. The DOJ OIG found that ATF management had not provided these reports to the field divisions since November 2000 and therefore had not continued this effort to clear the backlog. According to the Director of Laboratory Services, FACETS does not produce such reports in a usable format; however, the Director anticipated that electronic communications between field divisions and the laboratories using the new information management system would facilitate the timely removal of backlogged examinations associated with inactive cases. At the time of our audit, the ATF had not implemented the new information management system, although its acquisition had been planned since at least 2001.
Inability to Hire Sufficient Staff
In response to the Treasury OIG’s recommendation that the ATF ensure the laboratories are adequately staffed, in 2001 the ATF stated it would increase the total number of positions within the Office of Laboratory Services by 19. However, the DOJ OIG found that, rather than increasing, the number of positions available within the three regional forensic science laboratories in FY 2005 had instead decreased by two from the number of positions that existed in FY 2000. The number of vacancies noted at the laboratories in 2001 had not changed significantly as of September 2005.
In addition, the ATF proposed in 2001 to centralize its laboratory personnel functions, and to attract, hire, and retain experienced personnel. The DOJ OIG found evidence these actions had been taken, but we concluded that they had no effect on reducing the time it took the ATF to fill vacant/open positions. Moreover, the average of 9 months to 1 year that vacancies remained unoccupied in 2001 increased to 14 months during FYs 2004 and 2005. According to the Director of Laboratory Services, the extended time it took to hire replacement personnel was partially due to the fact that the ATF had implemented a new hiring process. Three of the replacement actions we reviewed were in San Francisco, and laboratory officials there stated that because of the high cost of living, it is difficult to locate qualified personnel at a salary commensurate with their qualifications.
Given the examiner vacancies, the DOJ OIG found that the staffing level in FY 2005 was not sufficient to both manage the incoming workload and reduce the existing backlog. Consequently, backlogged requests continued to interfere with the timely analysis of incoming examinations requests.
Other Duties Outside the Laboratories
In response to the Treasury OIG’s recommendation that the ATF ensure that laboratory managers coordinate the amount of outside work performed by laboratory personnel to limit the negative effect on the laboratory’s workload, the ATF stated it would centralize and coordinate all training requests, and evaluate all requests for laboratory assistance at crime scenes to ensure that a valid need existed. The DOJ OIG found evidence that these actions had taken place and concluded that laboratory officials were reasonably managing the demands for these services.
Effect on ATF Investigations
To determine how the Office of Laboratory Services’ management of its workload affected ATF investigations, the DOJ OIG reviewed ratings and comments from Special Agents on customer satisfaction cards that were returned with laboratory reports. The cards ask respondents to rate the laboratory’s performance in three areas: service, timeliness, and reports/statements. Only 23 percent of the recipients of 4,576 laboratory reports included in our universe returned the customer satisfaction cards, and those respondents were overwhelmingly satisfied with the service and laboratory reports provided by the regional forensic laboratories. However, a minority of respondents (10 percent) expressed at least some dissatisfaction with the time it took the laboratories to complete examinations and issue reports.
We also obtained supplemental information through 22 telephone interviews from a judgmental sample of Special Agents regarding their satisfaction with the service provided by the regional forensic laboratories. We asked questions about the effect processing delays had on investigations, and whether the agents ever used state and local laboratories instead of the ATF laboratories.
Based on those interviews, it appears that delayed receipt of laboratory reports did not adversely affect investigations in most instances, primarily because of the absence of a suspect or because of a confession. However, about 30 percent of the Special Agents we interviewed stated that they were less than satisfied with the timeliness of the service provided by the laboratories. Additionally, because of delays at ATF laboratories, most of the Special Agents we contacted told us that they had at least some laboratory examinations (primarily latent prints) performed at state and local laboratories.
We concluded that the questions on the customer satisfaction cards should be revised to ask more specific information about the effect of the analysis on investigations, such as whether the analysis was received in time to be of assistance on a case, or if it was not, what the negative effect was on the progress of any investigation or the outcome of a case.
The Treasury OIG recommended the ATF develop a priority system for incoming evidence submissions to support its investigative priorities. In response, the ATF stated that a new priority system was under development. The DOJ OIG found, however, that the new priority system had not been implemented by FY 2005, 4 years after it was proposed as a corrective action. Additionally, the ATF had not established a methodology for classifying its non-expedited work, which accounts for about two-thirds of all submissions. Non-expedited examinations were completed on a first-in, first-out basis, which did not account for the varying degrees of importance of these examinations.
As shown in the following chart, examinations completed on submissions for which expedited service was requested were generally completed in a more timely manner than other examinations. However, 46 percent of examinations for which expedited service was requested were not completed within 30 days.
Under the current priority system, requests for expedited service have to be justified according to informal criteria and have to include a copy of the ATF’s Report of Investigation to assist laboratory personnel in performing examinations. We found that these management controls were not always followed. While almost all expedited submissions we reviewed included justifications, 19 percent did not meet the ATF’s informal criteria. Additionally, 42 percent of case files reviewed were missing the required Report of Investigation.
It is crucial that forensic results be of high quality to ensure reliable results. While the issue of quality was not addressed in the Treasury OIG’s 2001 audit, the DOJ OIG performed a limited assessment to ensure that the ATF had a quality assurance process in place.
We found the ATF had a quality assurance process that the Office of Laboratory Services followed to ensure the quality of its services. The three regional forensic laboratories are accredited and internal quality reviews are performed periodically by the Office of Laboratory Services.
OIG Conclusions and Recommendations
The OIG’s follow-up review to the Treasury OIG’s 2001 audit report found that forensic science examination processing times had not significantly improved in the past 4 years. Two-thirds of completed forensic examinations continued to take more than 30 days to complete, and about one-third of examinations took more than 90 days. While customers of the ATF laboratories appreciated the quality of work produced, interviews with Special Agents in field offices continued to reflect dissatisfaction with the processing times, and more than half of the agents we spoke with used other laboratories at times to obtain more timely results. We also found that the ATF laboratories follow a quality assurance program that ensures its laboratories are accredited by the American Society of Crime Laboratory Directors/Laboratory Accreditation Board and the laboratories comply with the ATF’s Office of Laboratory Services Policy and Procedures Guidelines.
Although the ATF had implemented several corrective actions as a result of the Treasury audit, other corrective actions that could have had a significant impact on laboratory workload management had not been implemented.
The ATF did not increase the number of examiner positions in the forensic laboratories, and the time it took to fill examiner vacancies had increased. For example, the DOJ OIG found that the staffing level in FY 2005 could not manage the incoming workload in combination with the existing backlog in light of examiner vacancies.
The ATF also did not implement a new priority system as it had stated it would. The priority system currently in place does not adequately identify evidence submissions that are appropriate for expedited service. No standard process is in place to classify all examination requests so that they are processed in a timely manner. Although after the 2001 Treasury OIG audit the ATF said a new priority system was under development, the new system had not been implemented when the DOJ OIG conducted its audit in FY 2005.
Finally, the ATF did not implement a new information management system, so it did not realize the anticipated benefits associated with the new system. The ATF also did not continue its initial efforts to clear the backlog of requests for examinations that were no longer needed. We found that staffing was adequate to manage the incoming workload but, as a result of never clearing the backlog, backlogged requests continued to interfere with the timely analysis of incoming examination requests.
The DOJ OIG recommends seven improvements that focus on managing the incoming workload and existing backlog, developing and implementing a revised priority system, and developing strategies to reduce the time it takes to fill examiner vacancies. Without taking these or similar actions, the backlog, inadequate priority system, and vacant examiner positions will continue to interfere with the laboratories’ ability to handle the incoming workload of evidence on a timely basis, and more serious consequences may occur if delays in performing the work allows offenders to commit additional crimes.
We recommend that the ATF:
|« Previous||Table of Contents||Next »|