That was the general observation of a report out this week from the Department of Justice’s Office of the Inspector General which found that while the FBI has an annual process, known as Threat Review andPrioritization (TRP), to identify the most severe and substantial threats and direct resources to them, the process employs subjective terminology that is open to interpretation, and as such does not prioritize cyber threats in an objective, data-driven, reproducible, and auditable manner.

Also, because TRP is conducted annually, it may not be agile enough to identify emerging cyber threats in a timely manner, the OIG stated.

The OIG said that the FBI Cyber Division has a relatively new tool that it says could greatly help the FBI in its cyber threat evaluations. Developed in June 2014, Threat Examination and Scoping (TExAS) software uses a weighted algorithm to prioritize cyber threats based on specific data, rather than on subjective determinations as used in the TRP process, the OIG stated. The data visualization tool also allows decision makers to prioritize or otherwise allocate resources toward new intrusions sets or towards ones where better intelligence is needed.

A Cyber Division official told the OIG that it intends to have Sentinel, the FBI's case management system, automatically update TExAS with available data once a day In FY 2017 and to have the applicable field offices manually enter the data that Sentinel cannot transfer every 30 days.

“Real-time updates represent a useful augmentation to the TRP because It allows for transparency-intelligence analysts and decision-makers can clearly visualize the threats - and it also indicates new [emerging] and/or adapting threats. The 9/11 Review Commission also noted that, under the current system, once Cyber Division resources are allocated under the annual TRP process, the division had to scramble to reallocate existing resources to address any newly-identified threats,” the OIG stated.

If integrated with Sentinel, we believe that the TExAS tool has the potential to provide a current picture of the threat landscape. According to an FBI Sentinel official, interfacing TExAS with Sentinel would not be difficult because the interface design already exists.

However, the OIG found that TExAS lacks written policies and procedures outlining data entry and how the data should be used in prioritizing threats. If emerging threats are not identified or addressed in a timely manner, the FBI may well not be allocating appropriate resources to significant emerging cyber national security matter, the OIG stated.

“While we recognize that any system is only as good as the data entered into it, we believe an application like TExAS, is a best practice that could streamline the prioritization within the Cyber Division and potentially across other FBI operational divisions. Additionally, we found that the FBI is not able to adequately track agent resource utilization by threat. As a result, the FBI cannot be sure that it is aligning its cyber resources to the highest priority threats. We believe the FBI should develop and implement a record keeping system that tracks agent time utilization by threat.

We believe that greater reliance on objective and auditable information in the threat ranking process will enhance the FBI's ability to accurately and efficiently prioritize cyber threats and direct resources accordingly. A key requirement for a threat driven organization is the ability to track resources according to the threat, and we find that the FBI can improve in this area.

For its part the FBI agreed with the OIG’s report though offered no timetable to implement its suggestions