Fbi Virtual Case File System Case Study

The main stakeholders in the VCF system were the FBI, principally the system's users, the contractors, principally SAIC, and the federal government.

After reviewing and recognising the limitations of the web front end for the ACS system, two special agents convinced the FBI director in 2001 that the FBI needed a new system with an improved user interface and database management system that could improve storage and analysis capabilities of “relationships on everything from witnesses, suspects, and informants, to evidence such as documents, photos, and audio recordings”(Goldstein, 2005). In December 2001, SAIC was told to stop building a web front end to the FBI’s existing systems and instead begin development of a new application to replace the old ACS.”[10]

The FBI refined the VCF concept through Joint Application Development (JAD) sessions held with SAIC engineers and other experts between January and June 2002. “The JAD sessions brought together FBI representatives to determine what applications were needed to support the case management and information requirements of FBI agents, analysts, and support personnel; [user application component] contractor representatives to determine what applications could be created; and infrastructure contractor representatives to ensure that the applications could be supported by the groundwork that was being developed.”[11]

Political Commitment Good

The FBI executes critical and confidential operations that require the management of great amounts of information, and therefore the implementation of a modern and efficient system to do this was widely recognised by the agency and Congress as a necessary investment.

Congress provided the financial support requested for the project – and more funding later on as it began to overrun. “In September 2000, Congress approved USD379.8 million for a three-year project that was called the FBI Information Technology Upgrade project under the direction of FBI Director Louis Freeh.”[12]

As the VCF project spend exceeded its estimates, its deadline was brought forward. “In January 2002, the FBI requested an additional USD70 million from Congress to accelerate the Trilogy project, but received USD78 million. SAIC agreed to deliver the VCF system in December 2003 rather than June 2004.”[13]

The FBI director at the time, Robert Mueller, also expressed his support for the project, even after it had "gone wrong". “With regard to the funding of VCF, this Committee has been supportive of our efforts and has generously provided the funding we have needed to overcome obstacles and attempt to move forward... I am disheartened by this result but remain confident in our ability to deliver a case management system to our employees' desktops in the future."[14]

Public Confidence Good

It is difficult to establish the level of public confidence in such a project, as its details at the time were confidential, so that knowledge of its details was probably limited to the few individuals managing the institution and budget. There is, however, evidence of public pressure on the FBI as a whole to improve its efficiency after the September 11 attacks in 2001.

The development of Trilogy and VCF was started shortly before the attacks in 2001, and this situation had a major impact on the pressure on the FBI to improve its administrative efficiency. “Due to intense public and congressional pressure, Mueller instructed that the three-year schedule be put into overdrive so that the Trilogy project could be completed 'as soon as technically possible'.”[15]

Policy

Clear Objectives Fair

The objectives of the VCF project were very high level at its inception, and there is no evidence that measurable targets were set to accompany the project’s goals.

The FBI did not have a set of defined VCF requirements when the original contract was signed in 2001. “The development of the VCF application started with a very simple concept – the FBI's need for a modern case management system. As the FBI's mission evolved over the past several years, so did our technological needs. As a result of these changes and other issues, the FBI faced obstacles in a number of key areas relating to the VCF programme.”[16]

“Enable agents anywhere in the US quickly to search various documents and allow them to connect possible leads from different sources

“Provide a case management system, an evidence management system, and a records management system to eliminate the need for FBI employees to scan hard-copy documents into computer files.”[17]

The overall Trilogy project was in a similar situation. “During the initial years of the project, the FBI had no firm design baseline or road map for Trilogy. According to one FBI Trilogy project manager, Trilogy’s scope grew by about 80 percent [from the] initiation of the project. Such large changes in the requirements meant that the specific detailed guidance for the project was not established, and as a result a final cost and schedule was not established.”[18]

Evidence Fair

This project was meant to replace an outdated document management system. However, although there had been previous IT projects implemented within the FBI, there is no evidence that experience from them was used in the implementation of the VCF project. The FBI relied on pilots (after the first stage was delivered) to test the project. However, this seems to have happened too late into the project to lead towards a successful implementation.

The VCF contractor, SAIC, delivered the first of three planned system deliveries for the VCF in December 2003, which was not fully functional and was rejected. The delivery of the project was then split into two parallel tracks, IOC and FOC, and by 31 December 2004, SAIC provided the FBI with a "proof-of-concept" or prototype VCF in line with the JAD approach to software development. “This prototype is being used by agents and analysts in a test field office to demonstrate an electronic workflow process in ongoing investigations. Over a 6-week pilot test period scheduled between January and March 2005, the FBI’s New Orleans Field Office and the Baton Rouge Resident Agency are entering actual investigative lead and case data into the VCF. This information will be uploaded into the ACS to create paper case files upon electronic approval of the electronic information in the VCF.”[19]

Feasibility Weak

The project faced several feasibility difficulties, including lack of qualified staff, unrealistic timelines, and inappropriate infrastructure. Although sufficient funding had been approved for the project, these difficulties resulted in significant overruns.

Some of the issues acknowledged by the FBI after the project was cancelled included the limited engineering capabilities of the FBI and SAIC in creating the VCF system. “We lacked skill sets in our personnel such as qualified software engineering, programme management, and contract management. We also experienced a high turnover in Trilogy programme managers and chief information officers (CIOs).”[20] The agency also stated that it underestimated the complexity of interfacing with its legacy systems, of addressing its security needs, and of establishing an enterprise architecture.

There were also statements that the IT infrastructure of the FBI at the time was not appropriate for supporting such a project. According to a 2002 report from the Department of Justice's Office of the Inspector General, 13,000 of its computers could not run modern software. “Most of the 400 resident agency offices were connected to the FBI intranet with links about the speed of a 56-kilobit-per-second modem. Many of the bureau's network components were no longer manufactured or supported. And agents couldn't email US Attorney offices, federal agencies, local law enforcement, or each other; instead, they typically faxed case-related information.”[21]

The schedules and budget were greatly underestimated. Although enough funds were allocated to the VCF project, the estimated budget was exceeded by around USD50 million, ultimately reaching USD170 million.[22] Additionally, the timeline commitments from developers were considered unrealistic. “Not getting the right resources when they are needed may create a risk that the schedule will not be met. Unrealistic schedules may doom the project before it even starts.”[23]

Action

Management Weak

The FBI and SAIC, the contractor executing the project, were the organisations principally responsible for the outcome of the VCF system. However, there were several weaknesses in the planning and monitoring of the programme as a whole. Both sides blamed each other for the mismanagement.

For the FBI, the head of the agency was the top person accountable for the project, and he designated several people to manage it more directly. Larry Depew, who was one of the first proponents of the project – although he had no IT project management experience – was named the VCF project manager. His team communicated the FBI’s investigative and administrative processes to the SAIC engineers. C. Z. (Sherry) Higgins, a seasoned IT professional, was hired to create the Office of Programme Management, and was in charge of centralising IT management and overseeing "the FBI’s most expensive, complex, and risky projects", as well as managing Trilogy.[24]

When SAIC’S work was first evaluated after the delivery in 2003, it was concluded that “lack of effective engineering discipline has led to inadequate specification, design and development of VCF... There was ‘no assurance‘ that the requirements were satisfied, nor that the architecture, Concept of Operations, and requirements were correct and complete."[25]

On the other hand, SAIC criticised the FBI’s management. Duane Andrews, SAIC’s chief operating officer, said that: “The FBI modernisation effort involved a massive technological and cultural change, agency-wide. To add to that complexity, in the time that SAIC has been working on the Trilogy project, the FBI has had four different CIOs and 14 different managers. Establishing and setting system requirements in this environment has been incredibly challenging.”[26]

Finally, an audit report from the Office of the Inspector General found that: “Despite the use of two contractors to provide three major project components, the FBI did not hire a professional project integrator to manage contractor interfaces and take responsibility for the overall integrity of the final product until the end of 2003. According to FBI IT managers, FBI officials performed the project integrator function even though they had no experience performing such a role.”[27]

Measurement Weak

There were no monitoring guidelines or tracking indicators included in the project’s plan at the outset, and the project was first tested and measured only a few years after inception, which was arguably too late to recover from the initial mistakes.

According to FBI IT and contract managers, the cost-plus-award-fee type of contract used for Trilogy did not specify key items to guide the implementation, which also led to some of the challenges mentioned. “It did not require specific completion milestones, did not include critical decision review points, and did not provide for penalties if the milestones were not met.”[28]

The Office of the Inspector General also noted in its audit report that had monitoring taken place from the outset, many of the problems might have been avoided “If monitoring the scheduling of the project had been a priority, the FBI could have taken more timely action to effectively address Trilogy’s problems. At the end of 2003 — well over two years into the project — the FBI hired a contractor to perform these project integration duties when it became apparent that a professional project integrator was needed to effectively complete the project.”[29]

Alignment Weak

A lack of clarity regarding the project’s specific objectives and timelines led to several disagreements between the FBI personnel leading the project and the contractors. This eventually led to both parties blaming each other for the issues with the project.

Initially, the FBI did not have a clear vision of what the FBI's Trilogy project should achieve or of specific design requirements, and the contractors were not held to a firm series of achievable milestones. “The FBI's investment management process also left it ill-equipped to ensure that all three components of Trilogy were developed in an integrated fashion. Moreover, at the outset, the FBI and others did not provide consistent or effective management of Trilogy, leading to technical and scheduling problems.”[30]

The tensions between SAIC developers and the FBI increased over the winter and spring of 2003, as the first part of the VCF project was being delivered. A number of problems were blamed on the contractors but were also compounded by the FBI’s "sloppy inventories" of existing networks and an underestimation of the difficulties of such a complex implementation.

The acting CIO for the FBI made the decision to reject the deliverable, and the FBI documented 17 functional deficiencies that SAIC would have to fix before the system could be accepted. SAIC argued that many of these deficiencies were caused by requirements changes from the FBI team, so an arbitrator had to be called in. In March 2004, the arbitrator released a report that found fault with both the FBI and SAIC. “According to the report, 59 issues and sub-issues were derived from the original 17 deficiencies – 19 were due to requirements changes and the FBI’s fault, while the remaining 40 were SAIC problems. SAIC offered to make all the changes if given one more year and USD56 million. [The FBI’s CIO] rejected the offer."[31]