OIG Audit Report 07-20

Our objective was to determine if Department components and the Department as a whole properly gather, classify, and report accurate terrorism-related statistics. We conducted our audit in accordance with the Government Auditing Standards and included such tests as were considered necessary to accomplish our objectives. Our audit concentrated on, but was not limited to, the period October 1, 2000, through September 30, 2005.

We performed audit work at the Department’s Office of Public Affairs, the Federal Bureau of Investigation (FBI) headquarters, the Criminal Division headquarters, the Executive Office for United States Attorneys (EOUSA) headquarters, and the Justice Management Division (JMD) headquarters, all in Washington, D.C. We also conducted audit work at the following FBI field offices and United States Attorney’s Offices (USAO).

FBI Field Offices

Atlanta, Georgia
Baltimore, Maryland
Charlotte, North Carolina
Dallas, Texas
El Paso, Texas
Philadelphia, Pennsylvania
Salt Lake City, Utah

USAOs

Los Angeles, California
Denver, Colorado
Washington, D.C.
Tampa, Florida
Detroit, Michigan
Newark, New Jersey
Houston, Texas
Alexandria, Virginia

We performed the following work at the Department’s Office of Public Affairs, FBI headquarters, EOUSA headquarters, Criminal Division headquarters, and JMD headquarters:

We developed a universe of 209 unique terrorism-related statistics reported by the Department and its components by:

interviewing key FBI, EOUSA, Criminal Division, and JMD personnel regarding internal and external documents in which terrorism-related statistics are reported;
searching the Department’s websites for congressional testimony and speeches containing terrorism-related statistics; and
reviewing the documents identified through interviews and searches for terrorism-related statistics.

We reviewed documentation and interviewed Department officials to identify each of the 209 terrorism-related statistics reported for:

the period covered by the statistic;
the key FBI, EOUSA, Criminal Division, and Department personnel responsible (source) for tracking and reporting the statistic and what mechanisms (automated systems, case files, spreadsheets, and databases) were used to track the statistics;
the processes used to report the statistics and the internal controls established to ensure the accuracy of the reported statistics; and
how the statistic is used (operational decisions or informational purposes).

We excluded 17 of the 209 terrorism-related statistics from audit testing because we determined the statistics were reported by the Department but sourced to agencies outside the Department. T he remaining 192 terrorism-related statistics were reported by and sourced to the FBI (121), EOUSA (20), the Criminal Division (14), and the Department as a whole (37).
We divided the 192 terrorism-related statistics into two categories consisting of 123 operational statistics and 69 informational statistics. The operational statistics represent terrorism-related statistics reported in various documents including annual budgets, annual financial statements, performance plans and assessments, statistical reports, and other Department reports. The informational statistics represent statistics used in various forms such as speeches, press releases, and unclassified publications.
We excluded the 69 informational statistics from our audit because we judged them to be less significant to the war on terror than the 123 operational statistics.
We excluded 16 of the 123 operational statistics from testing because Department officials could not identify the source of the statistic (See Appendix II). Without the source, we could not determine whether internal controls had been established to ensure the accuracy of the statistics reported.
We excluded 48 of the 123 operational statistics from testing because the statistics, or the systems supporting them, had been previously reviewed by the OIG and recommendations made to correct deficiencies identified.
We used a risk level to decide which statistics to test by developing a method to weigh the level of risk that the remaining 59 operational statistics were reported accurately. We came up with the following six categories of risk and used a risk rating scale of one to three within each category with one being relatively low risk and three being relatively high risk.

OIG’s Methodology for Measuring
Risk of Terrorism-Related Statistics

Risk Category/ Rating Elements	Risk Rating
1. Frequency of use
Reported 1 to 3 times	1
Reported 4 to 10 times	2
Reported 11 to 35 times	3
2. Internal controls
Established and documented	1
Established but not documented	2
Not established	3
3. Magnitude of the statistic reported⁶⁰
From 0 to 1,000	1
From 1,001 to 10,000	2
Greater than 10,000, to include any dollar-related numbers, percentages, and hours	3
4. Inconsistencies in statistics reported
No inconsistencies	1
One inconsistency	2
Multiple inconsistencies	3
5. Reported in informational formats by others
Reported by other than the Attorney General, Deputy Attorney General, FBI Director, or EOUSA	1
Reported by EOUSA	2
Reported by the Attorney General, Deputy Attorney General, or FBI Director	3
6. Preciseness of the statistic reported
No operational statistics in group are precisely stated	1
Some operational statistics in group are precisely stated	2
All operational statistics in group are precisely stated	3

We calculated an overall average risk score for each of the 59 operational statistics by adding the risk rating assigned for each of the six risk categories and then dividing this sum by six.

We selected the following 26 of the 59 operational statistics (FBI – 10, EOUSA – 11, Criminal Division – 5) for audit testing based on the risk rating we calculated and on our judgment of the significance of the statistic to the war on terror.

Terrorism-Related Statistics Selected for Detailed Testing

Description of Statistic	Primary Reporting Agency	System Used to Track Statistic
1. Number of Intelligence Information Reports issued	FBI	Terrorism Reports and Requirements Section’s database
2. Number of threats racked	FBI	Threat Monitoring Unit’s spreadsheet database and Guardian Threat Tracking System
3. Number of threats to transportation and facilities	FBI	Counterterrorism Watch’s Operational Support Log and spreadsheet summary
4. Number of threats to people and cities	FBI	Counterterrorism Watch’s Operational Support Log and spreadsheet summary
5. Intelligence Assessments issued	FBI	Global Terrorism Analysis Unit’s database
6. Number of intelligence products produced/disseminated to the U.S. Intelligence Community and appropriate federal elements	FBI	Intelligence Requirements and Collection Management Unit 1’s spreadsheet
7. Number of Intelligence Bulletins issued	FBI	Global Terrorism Analysis Unit’s database and the Threat Analysis Unit’s word processing file
8. Number of Counterterrorism threat assessments produced	FBI	Threat Analysis Unit’s word processing file
9. Number of Presidential Terrorist Threat Reports produced	FBI	Global Terrorism Analysis Unit’s database
10. Number of convictions	FBI	Integrated Statistical Reporting and Analysis System
11. Number of individuals charged as a result of terrorism investigations	Criminal Division	Counterterrorism Section’s database
12. Number of individuals who have been convicted or pleaded guilty as a result of terrorism investigations	Criminal Division	Counterterrorism Section’s database
13. Number of people charged with material support of terrorism or similar crimes	Criminal Division	Counterterrorism Section’s database
14. Number of people convicted of material support of terrorism or similar crimes	Criminal Division	Counterterrorism Section’s database
15. Number of material support to terrorism cases and matters the Criminal Division participated in or coordinated	Criminal Division	Deputy Chief of Counterterrorism Section’s manual count based on e-mails received from Counterterrorism Section attorneys and the Deputy for Terrorist Financing
16. Number cases filed – defendants for program activity anti-terrorism	EOUSA	EOUSA LIONS
17. Number of terrorism convictions	EOUSA	EOUSA LIONS
18. Number of terrorism-related convictions	EOUSA	EOUSA LIONS
19. Number sentenced to prison – defendants for program activity anti-terrorism	EOUSA	EOUSA LIONS
20. Number prosecuted – defendants for program activity anti-terrorism	EOUSA	EOUSA LIONS
21. Number guilty – defendants for program activity anti-terrorism	EOUSA	EOUSA LIONS
22. Number of terrorism cases tried	EOUSA	EOUSA LIONS
23. Number of terrorism and anti-terrorism cases against defendants that were terminated	EOUSA	EOUSA LIONS
24. Number of terrorism and anti-terrorism cases pending	EOUSA	EOUSA LIONS
25. Number of defendants that the U.S. Attorneys terminated terrorism and anti-terrorism cases against	EOUSA	EOUSA LIONS
26. Percentage of defendants whose cases were terminated that were convicted	EOUSA	EOUSA LIONS

We interviewed Department officials and reviewed supporting documentation to determine whether the 26 operational statistics were accurately reported.

At the FBI field offices and USAOs visited, we interviewed FBI and USAO officials and reviewed case file documentation to determine whether the following FBI and USAO statistics were accurately reported.

For the Criminal Division statistics we reviewed, we initially reviewed reconstructed data provided to us in March 2006 by the Criminal Division to support the numbers reported. In August 2006 we discussed with Criminal Division officials our audit results that showed the statistics reported by the Criminal Division were not accurately reported. The Criminal Division believed it could support all the numbers reported so we agreed to review additional support that the Criminal Division said it could provide. In August 2006, the Criminal Division provided another set of reconstructed lists to support the statistics we tested. We met with Criminal Division officials and reconciled the March 2006 data to the August 2006 data. We then selected a sample of transactions for each statistic and analyzed supporting documentation provided by the Criminal Division to ensure the cases reported were linked to terrorism and occurred during the reporting period.

Methodology for Sampling the FBI Statistic on Number of Convictions

The FBI’s FY 2006 - FY 2011 Performance Plan includes a performance goal to “Prevent, disrupt, and defeat terrorist operations before they occur.” In its FY 2006 – FY 2011 Performance Plan, the FBI reported that its investigations resulted in 206 convictions during FY 2004 towards the achievement of this performance goal. To determine the accuracy of the reported convictions, we:

obtained a listing from the FBI showing details of the 206 convictions reported for FY 2004;
selected a judgmental sample of 107 of the 206 convictions by selecting all the convictions reported by the 7 FBI field offices with the highest number of reported convictions (El Paso, Texas - 32; Dallas, Texas - 22; Salt Lake City, Utah - 17; Atlanta, Georgia – 12; Richmond, Virginia – 10; Philadelphia, Pennsylvania – 8; and Charlotte, North Carolina - 6); and
reviewed the case files and held discussions with the case agents regarding whether the individuals were convicted of or pleaded guilty to crimes under a terrorism statute, and how the convictions met the FBI’s Performance Goal of “Prevent, disrupt, and defeat terrorist operations before they occur.”

Methodology for Sampling EOUSA Statistics

For the 11 EOUSA operational statistics that we selected for detailed testing that were reported 20 times, we obtained listings from the LIONS system that is used to record data for the statistics reported. We then compared the LIONS data to the number reported for each statistic to determine if the LIONS data agreed with the reported number. Our comparison showed the following.

Comparison of LIONS Data to Reported EOUSA Statistics

Statistic Description	Number Reported	LIONS Data Agreed With Number Reported
1. Number cases filed – Defendants under the anti-terrorism program activity	365 in FY 2002 786 in FY 2003 725 in FY 2004⁶¹	No Yes Yes⁶¹
2. Number of terrorism convictions	103 in FY 2003 118 in FY 2004	Yes Yes
3. Number of terrorism-related convictions	558 in FY 2003 379 in FY 2004	Yes Yes
4. Number of defendants sentenced to prison under the anti-terrorism program activity	264 in FY 2003 256 in FY 2004	Yes Yes
5. Number of defendants prosecuted under the anti-terrorism program activity	245 in FY 2002	No
6. Number of defendants found guilty under the anti-terrorism program activity	153 in FY 2002	No
7. Number of terrorism cases tried	3 in FY 2001	No
8. Number of terrorism and anti-terrorism cases against defendants that were terminated	682 in FY 2003 504 in FY 2004	Yes Yes
9. Number of terrorism and anti-terrorism cases pending	737 in FY 2002 637 in FY 2003 719 in FY 2004	Yes Yes Yes
10. Number of defendants the U.S. Attorneys terminated terrorism and anti-terrorism cases against	643 in FY 2004	Yes
11. Percentage of defendants whose cases were terminated that were convicted	87% in FY 2003	Yes

For the 16 times where numbers reported for the 8 statistics agreed with the LIONS data, we performed additional testing to determine if the data recorded in LIONS was accurate. We selected a judgmental sample for each of the 16 times the 8 statistics were reported by:

obtaining a spreadsheet showing the number of transactions reported for each statistic by a United States Attorney’s District Office,
calculating the total number of transactions reported for each district office for all 16 times the statistics were reported,
selecting the 8 district offices that reported more than 200 total transactions for the 16 times the statistics were reported, and
reviewing all the transactions reported by these 8 district offices for each of the 16 times the statistics were reported.

The following table shows the number of transactions selected for each of the 16 times the 8 statistics were reported.

Samples Selected for EOUSA Statistics Where the
LIONS Data Matched the Statistic Reported

Statistic Description	Number Reported	Transactions Sampled
1. Number cases filed – Defendants under the anti-terrorism program activity	786 in FY 2003 725 in FY 2004⁶²	268 338
2. Number of terrorism convictions	103 in FY 2003 118 in FY 2004	30 40
3. Number of terrorism-related convictions	558 in FY 2003 379 in FY 2004	301 166
4. Number of defendants sentenced to prison under the anti-terrorism program activity	264 in FY 2003 256 in FY 2004	81 91
5. Number of terrorism and anti-terrorism cases against defendants that were terminated	682 in FY 2003 504 in FY 2004	364 240
6. Number of terrorism and anti-terrorism cases pending	737 in FY 2002 637 in FY 2003 719 in FY 2004	487 335 362
7. Number of defendants the U.S. Attorneys terminated terrorism and anti-terrorism cases against	643 in FY 2004	294
8. Percentage of defendants whose cases were terminated that were convicted	87% in FY 2003	364

Footnotes

The magnitude of the statistic reported relates to the quantitative number associated with the statistic. For example, a statistic reported as being 2 is probably less likely to be inaccurate than a statistic reported as being 10,247.
The 725 reported for FY 2004 was reported twice – once in the U.S. Attorneys FY 2004 Annual Statistical Report and once in the FY 2006 President’s Budget Request to Congress.
The 725 reported for FY 2004 was reported twice – once in the U.S. Attorneys FY 2004 Annual Statistical Report and once in the FY 2006 President’s Budget Request to Congress.