Agile approach ‘reduces risk’ but offers no delivery guarantee for FBI as Sentinel project slips again

A recent story by the US magazine InformationWeek Government has cast some doubt over whether a move to Agile development will necessarily bring IT projects in on time and to budget, despite the best intentions. Sometimes other technology factors, such as legacy hardware, come into play.

The report says an FBI project to develop a digital case-management system to replace outdated, paper-based processes has been further delayed, despite a move to use Agile development to hasten the project’s completion.

The system, dubbed Sentinel, is now due to go live in May, eight months later than the FBI planned when it embarked on the Agile development plan.

It follows a number of delays going back to 2006 in the plans to build a replacement for the FBI’s 17-year-old Automated Case Support system, which is used by agents and analysts to manage their cases.

In 2006, Information Week Government reports, the FBI awarded Lockheed Martin a $305 million contract to lead development of Sentinel, but took back control of the project in September 2010 amid delays and cost overruns.

Then, the FBI said it planned to finish Sentinel within 12 months using Agile development. But that worked has slipped (the FBI had earlier pushed Sentinel’s deployment from September 2011 to January 2012), and a four-hour test of the system in October resulted in two outages, according to a report by the Inspector General released in December which contains details of the Agile development’s sprints’ progress.

The FBI said the glitches were down to overburdened legacy computer hardware and said the hardware will need to be upgraded to support Sentinel’s use across the agency, according to the Inspector General.

The report’s conclusion says:

“The FBI’s transition to an Agile development approach has reduced the risk that Sentinel will either exceed its budget or fail to deliver the expected functionality by reducing the rate at which the FBI is spending money on Sentinel and by instituting a more direct approach to the FBI’s monitoring of the development of Sentinel.

!When we provided our initial draft of this report to the FBI in October 2011, we expressed concern that the rate at which the FBI was developing Sentinel’s functionality indicated the project was at risk of falling behind the FBI’s then planned January 2012 deployment date.

“In December 2011, after we completed our fieldwork for this report and after we provided the FBI with a revised draft report, FBI officials told us that the FBI extended the Sentinel deployment date to May 2012. While we have not had the opportunity to fully review the FBI’s plan to meet these revised completion dates, we continue to believe it will be challenging for the FBI to meet this latest goal for deploying Sentinel to all FBI users in this timeframe.

“It is too early to judge whether the FBI’s Agile development of Sentinel will meet its newly revised budget and completion goals and the needs of FBI agents and analysts. While the Sentinel Advisory Group responded positively to the version of Sentinel it tested, results from wider testing were not as positive. Also, none of the Agile-developed Sentinel has been deployed to all users to give them the ability to enter actual case data and assist FBI agents and analysts in more efficiently performing their jobs.

“Despite the FBI’s self-reported progress in developing Sentinel, we are concerned that the FBI is not documenting that the functionality developed during each sprint has met the FBI’s acceptance criteria.

“Our concerns about the lack of transparency of Sentinel’s progress are magnified by the apparent lack of comprehensive and timely system testing. Our concerns about the lack of transparency also extend to Sentinel’s cooperation with internal and external oversight entities, to which Sentinel did not provide the necessary system documentation for them to perform their critical oversight and reporting functions. We believe that this issue could be resolved, at least in part, with a revision to the FBI’s Life Cycle Management Directive to include standards for Agile development methodologies.”

Responding to the report the FBI said:

“The Federal Bureau of Investigation (FBI) appreciates the opportunity to review and respond to your draft report entitled, “Status of the Federal Bureau of Investigation’s Implementation of the Sentinel Project.”

“We are pleased with your conclusion that, by adopting an Agile development approach, the FBI bas “reduced its rate of spending on Sentinel” and instituted a more “direct approach to monitoring the development of the system’s functionality.”

“Indeed, as the FBI’s figures included in this Report demonstrate, while we have expended only 52% oftbe Agile development budget of$32.6 million. as of December 6 we had completed 88% of the required system functionality. The percentage of functionality completed has further increased during the time that has passed since your report was last updated.

“This accomplishment is significant. In mid-2010, the FBI charted a new course for completing the remaining two phases oftbe Sentinel program using an Agile development approach, which represented a substantial departure from its prior development activities. As a result, you concluded in this Report that the FBI is “expending significantly fewer dollars per month than it had in Phases 1 and 2 for the project.”

“In sum, we agree with your conclusion that the FBI’s transition to an Agile development approach has “reduced the risk that Sentinel will either exceed its budget or fail to deliver the expected functionality.” As you note, “at this point in time, the FBI does not foresee exceeding the $451 million budget to complete the Sentinel project.”

“With that in mind, we are mindful of the short delay we have recently encountered under our new” Agile” approach. The Sentinel development schedule has recently been extended by two months (from December 2011 to February 2012), and the FBI-wide deployment is now scheduled for May 2012, as described in this Report.

“This modest extension is due primarily to the need to implement a standard five-year “refresh” of computer hardware, so the Sentinel software will provide the required functionality as intended. Indeed, you have determined that, given the pace at which the program has proceeded under the Agile approach over the time period you reviewed, your estimate for completion is essentially the same – June 2012.”

I wasn’t sure I understood the “take-away” from this. Would it be an unfair precis to say that agile is not a magic bullet (who knew…) and that all projects are about the organisation not the IT (see above)

The only thing I did learn was that in this instance was that agile slowed the burn rate but more interesting would have been a critical assessment of the original budget. Previous magic bullets include PRINCE wherein, IIRC, PRINCE2 increased the contingency which to my mind is ex-ante cheating in order to claim success or worse “coming in under budget”