Posted
by
samzenpus
on Thursday November 15, 2012 @12:27AM
from the on-second-thought dept.

angry tapir writes "The U.S. Air Force has decided to scrap a major ERP (enterprise resource planning) software project after spending $1 billion, concluding that finishing it would cost far too much more money for too little gain. Dubbed the Expeditionary Combat Support System (ECSS), the project has racked up $1.03 billion in costs since 2005, 'and has not yielded any significant military capability,' an Air Force spokesman said in a statement. 'We estimate it would require an additional $1.1B for about a quarter of the original scope to continue and fielding would not be until 2020. The Air Force has concluded the ECSS program is no longer a viable option for meeting the FY17 Financial Improvement and Audit Readiness (FIAR) statutory requirement. Therefore, we are canceling the program and moving forward with other options in order to meet both requirements.'"

If they are, they should: if a drone fired at them can't take 'em out, something's not working correctly./* Note: I'm not for drone strikes, necessarily, but, dammit, if you're targeting something, you'd better take it TF out. */

Are the victims of drone attacks complaining much about the quality of service?

Most drone attacks are done by the CIA, not the Air Force. If the Air Force launched the attacks, the results could be second guessed by CIA analysts evaluating satellite photos. But if the CIA both launches the attacks and evaluates the results, it is all wrapped up in a neat little package with no loose strings of accountability.

Obama is in charge... the buck stops with him; he's the one who brags about his "kill list"... Oh, wait, this is Slashdot... Obama has a GREAT smile and a cool attitude and nobody is to blame for the drone strikes. Move along, nothing to see here. Dick Cheney is retired so there is no evil to be denounced.

And our "ally" Israel has launched terrorist attacks against Americans on US soil, attempted to bomb the offices of a Muslim US senator, has repeated been caught carrying out espionage against our intel agencies and businesses, deliberately feeds false intel to the State department, sells weapons to our enemies, sells our military technology to China, attempted to bomb the Mexican parliament, and boasts about carrying out false flag terrorist attacks so that the US will retaliate against their enemies for them. Hopefully Netenyahu shot himself in the foot with his open endorsement of Romney.

Oh to be young an idealistic again. Someday you'll understand that the world just isn't so simple. For example, while I do not condone Obama's actions in the Middle East conflicts, Romney would have been far more agressive and openly so. I actually believe he would start a war with Iran. So who am I to vote for in such a tightly contested race? Gary Johnson? We all know that he won't win because America won't even vote Libertarians into low level local offices. So it would be irresponsible for me to not vote for and support Obama.

I love how each branch of the DoD gets to pick it's own ERP solution. It says Oracle won it over SAP, not that i have a preference but SAP has a showing of being successful in the market via is use in the Navy. With all ERP solutions there are going to be issues, but overall the Navy has been very successful with their SAP deployment.

Again, why isn't this pushed from the top of the DoD vs. every branch figuring it out and reinventing the wheel each time?

One has to wonder if the Navy was all that successful or just willing to handle a portion of the job, or willing to settle for half the result.

You will never know, because those who do have too much ass to cover, and they will be slipping in fixes and upgrades for decades, before deciding the whole thing is too top heavy.

Systems of this size are grandiose and seldom successful. Not only government fails at systems this big, private industry does as well. But private industry learns from their costly mistakes faster. Google is a good example. They hold a house cleaning each spring and just arbitrarily kill off projects that have no chance of a ROI.

Its amazing that two world wars were fought with this kind of stuff being handled by people.

I fully understand where you are coming from, and i can't answer for the Navy on the system as a whole, but i will say their implementation PM (Plant Maintenance) portion of SAP is a very good example of a very functional implementation that is very effective at doing it's job.

This comment makes me think that you've worked in neither government nor industry. Or you've been very, very lucky with your employers. Or never worked at a very large company.

Part of the reason (possibly the main one) they fail is due to people. That is the same for both sectors.

With a project that large, it's a big embarressment if it fails, so it's in the interest of the people in charge of the project to force it through at all costs no matter what. Because they don't care about their host organisation (be it government or industry), they care about their own career. Having a big failure like that is a blot. So, instead some half-asses expensive, buggy and minimally functional heap of shit is usually foisted onto the hapless minions of the organisation, usually with a large loss of productivity.

Oracle is usually the cause, and the event should be known as getting "Oracled".

It happens in the public, private and education sector. Oracle knows no limits. They will screw anyone they can get their hands on with crap products. There is no escape.

At least the USAF pulled the plug. After $1bn and a 10x overrun, there is not a single change in hell that the system would every be a net gain. It was a huge fuckup. But given where it was at that time, this was the only sane solution.

The problem is inherent to large organisations. It's not a public versus private problem. It's a big versus small one. That means that the public sector experiences the problems more often due to its size. But basically, large companies suffer exactly the same problems too.

A a small college I worked at (3500 students, 200 staff/faculty) I watched a 1 year People soft transition roll into a 7 year project, with both consultant and wife hired on as full time db administrators (with complimentary condo on the beach).

I wouldn't be surprised that the DoD is encouraging this. In this way, each branch picks their own solution because they need to satisfy so many domestic "interests". (Yes, SAP America contributes to political campaigns and PACs, just like every other large ERP company in the US). Besides, the only reason that anyone has been successful is probably because they are sipping more Kool-Aid and sitting in a circle "reassuring" one another.

This is perfect - fucking perfect! They got that far into it, all that mad bank, and no long-term responsibility for actual delivery, support, upgrades, roll-out, etc. Steve Miller: Go on, take the money and run. Day-um, someone gonna get a bonus at Oracle!

Did anyone also do a double-take on this story? $1B spent on software, and nothing to show for it? Let's say you pay a developer $100K/year, and the project lasted 10 years. That's 1000 developers working on this for 10 years! And after this, nothing to show for it? That's probably the most ridiculous thing I've heard in a while... I wonder if a big chunk of this money went to crony suppliers like Halliburton.

Let's say you pay a developer $100K/year, and the project lasted 10 years.

Your numbers are way off. Each cleared contractor probably costs the government 200+$/hr. Then there's hardware. If the project has been going on for 10 years, they probably went through at least one refresh cycle. Then there are other costs like admin overhead, facilities, travel etc.

Here in Melbourne Australia we have a train ticket system called Myki (the name was the first red flag) which cost 1.3 Bn last I heard. At that point it wasn't working at all. It is now and presumably they've spent a few more bucket loads of money, so it's sure to be over 1.5 Bn now.

They couldn't just buy a working system from Singapore, noooooo. They had to piss taxpayer's money away developing their own from scratch. 1.5 Bn buys a lot of paper tickets and probably several pre-developed systems. It ta

Here in Seattle the transit system paid to write their own DVR software for video recordings on the trains, and then proceeded to write a different DVR package for the buses. Why? Because the project specs, as edited by the political flunkies near the end of the process, prohibited COTS solutions. Actually specified proprietary hardware and proprietary software in both cases. Of course delivered it cost twice as much for half the utility of existing products on the market, but the consultants and contra

ERP is dead--especially for very large, agile institutions. The only people that don't think so are companies, like Oracle, that are pretending that it can scale to large institutions with some sort of economy of scale, let alone ones that probably make many changes. The fact that it took the Air Force an extra $900+ million to realize this is shameful. Especially since institutions like the Air Force are probably better off looking at agile and adaptive front-end software (it's not just the Marines that

In what sense do you think ERP is dead? The functions are all required and if you buy best of breed individual packages, you still need to integrate them, so either you do it yourself or you buy the ERP package that is already integrated.

I agree that some decisions can be made to break it up into manageable pieces and accept less efficiency, but with an organization of that size you still have a problem of complexity whether using an ERP package or creating point solutions and integrating them.

ERP is dead because word is on the street: Too many failed or seriously delayed implementations.I have seen (first hand) too many institutions decide to implement ERP, pay a tremendous amount of cash, and watch it fail. If it ever does get fully implemented (in a way that was originally envisioned) the institutions have spent so much time and effort to get it running that the institutions have lost their focus because senior management was distracted or the cost of full implementation has affected the bottom line. In some cases, the institution was irreparably damaged or failed.(often surpassed by their competition).

Bullshit. I have been involved in dozens of ERP implementations over the years. The software works. When implementations fail it is always, in my experience, because of the people (i.e. management) making the decisions on how to implement the product.

Me: "Let me show you how Product X handles Accounts Payable"Client: "That's not how we do it"Me: "This might be a good opportunity to take a look at your current business practices and see if they can be done in a more efficient way"Client: "But we've always done it this way"Me: "Why?"Client "Dunno...just always have. And I doubt that the team is willing to change"Me: "Ok, we can customize the product to make it work the way you want but it's going to take more time and money. And when you do an upgrade later on there will be implications as well"Client: "Fine. Just make it work the way we do it now"

And so it goes. Time and again I see clients go out and buy an expensive ERP system only to customize the bejezus out of it to make it look exactly like the systems they are retiring. They are not open to better business practices. Too many political headwinds.

What does this say about these clowns in the Air Force? It takes them 10 years and $1.03B to realize that the project is going to fail? On an original budget of $88M? One of the big problems with trying to shoehorn a best practice ERP system into a large government institution is that often they employ worst practices. They won't, or can't, change them so you have to end up rewriting the product to fit their ass backwards ways. The whole purpose of implementing an ERP system is to replace aging, stove-piped systems with modern integrated systems. It can work well if it's implemented properly and the right decisions are made along the way. But it's not a magic pill.

BING! BING! BING!We have a winner. I am seeing this very poli-drama being played out right now at my institution. The multi-decade tenured staff will not change from business processes implemented to fit a bad system bought 3 decades ago; and will not listen because they don't have to.

While I hear what you're saying, government entities, and especially the military, are also subject to legal requirements that they not do things in certain ways, or have unique requirements not accounted for in a 'best practices' system.

Just as often it's the contractors trying to cut corners, over promising, getting lawyers to weasel them out of contract agreements.

You're assumption the ERP = better business process is wrong. Sometime entrenched process are there for a reason, often a legal reason. Sadly the people who knew that reason have left and no one wanted to spend the money to hire someone to properly record it so they don't know. And they continue to not know until the begin to replace it. Once the agree to replace it they start

I used to be a programmer for the Air Force, and I can tell you that the development process is not agile at all. The Air Force, because it's primarily lead by people who used to fly planes, treats every product development as though they were developing an airframe or weapon. Until people who understand software are put in charge of such matters, we'll continue to see stupid stuff like this happen. This type of thing happens more often than you may expect, just not quite to this incredible scale (by whi

Probably like all large projects, no one wants to pull the plug as at that point they have to admit the whole thing is a failure. The more it blows out the more determined they become to deliver as the fallout for the failure only gets worse the further over budget they become. I have been involved in similar projects (though not to this scale) and it is always easier to beg for more funds or bleed the funds of other projects than it is to admit the project is a failure and simply stop the waste.

To my dismay, I worked on this project. The project started with controversy -- the Oracle bid that beat out SAP like seven years ago was surrounded by complaints. The article skips some details. CSC (Computer Sciences Corp, who is quoted) was the main driver of about $800-million of that spending. It is accurate to say that this change didn't affect them, but that's because hundreds of people had already been laid off or moved of the project between last September and last March.

There's enough blame to go all over the place. Years spent in requirements that weren't turned into code; time spent passing blame back and forth across development teams who were so large and segregated that they rarely communicated properly, both within the Air Force and within CSC and between the other teams. At it's peak I believe the project had roughly 800 people on it. I don't know what the maximum size a development project should have, but it's got to be smaller than that. That number includes everyone, trainers, managers, and some key initial users and testers, but still it's a very high number.

The Air Force tried several times to realign the project, but there were contractual disputes or, once that was over, difficulty deciding what to keep and what to scrap, which lead to a death spiral where everything went back on the drawing board and I think ultimately leadership just lost hope.

It wasn't a complete loss, though. A few small teams, including the one I was previously on, have survived. We built a robust data quality system and are working on some enterprise data dictionary and master data tools, which will help the systems that are left behind. With hundreds of systems supporting a half million users, $1billion probably isn't off the chart -- at least not had this been a successful project, but the worst part is that there's still much work that needs to be done, and now someone will have to start over... again.

Wow. I guess that this is a new record, eclipsing even the FBI's failure from a couple of years ago. Have to say, I am impressed. Leave it to the Pentagram to do things bigger and worse than anyone else on the planet.

Wow. I guess that this is a new record, eclipsing even the FBI's failure from a couple of years ago. Have to say, I am impressed. Leave it to the Pentagram to do things bigger and worse than anyone else on the planet.

In fairness, in terms of employees the US Air Force is 10X the size of the FBI, and bigger than any but a small handful of corporations. In terms of assets and materiel, military forces have one or two orders of magnitude more than a comparable corporation. It's pretty much guaranteed that their expensive failed efforts are going to be bigger and worse than just about anyone's.

Seriously, has there EVER been an ERP implementation that was anything other than a colossal fuckup? Way behind schedule, overbudget, and not functioning properly are the general themes of ERP. And businesses continue to fall for this scam.

Most of the ERP implementations I have been part of have been successful. Every major corporation has a working ERP system, how do you think that happened?

There are big failures, typically in situations where the size of the project exceeds the experience and capabilities of the people managing them. With something as big as the DoD, there just aren't too many opportunities for anyone to gain the proper experience to know how to make it successful. Something like that needs to be broken into much smaller pieces and you just have to forego some of the efficiencies of a completely integrated non-redundant system in favor of more manageable pieces.

Don't confuse the "Resource Planning" part of ERP with actually meaning anything. ERP used to be MRP when it was just focused on the manufacturing aspects of a business and specifically planning raw material requirements to meet the deman for the finished goods. But as companies added modules to the suite to encompass the entire enterprise, they decided to change the name to Enterprise Resource Planning because it sounded more "Enterprisey". Sure there is some planning in there and a whole bunch of transaction execution and tracking that really isn't related to planning.

All organizations should streamline their operation before even considering to introduce ERP. They all end up with a massive disaster because their procedures are inconsistent and there are a lot of differences how departments handle the same processes.

Once you have modelled the 5th separate way to order stionary and the umpteenth vacation policy for a department of 5 you know that you are screwed. I wouldn't speak of a system as such but rather a set of specific exceptions.

It is always the same pattern. And since you never start small and you never start flexible you will end up with a bloated, slow hairball that approximately does was the customer wants. Not what he needs.

Consulting is the worst part of a consulting gig. If we had the cojones to tell them what they have to do or we are out our jobs would be much easier. And riskier. Instead they "take our suggestions under advisement". Meaning they'll return to internal politicking and do jack-diddly-doo.

The Air Force propably has a lot more red tape than that. Red, white and blue tape tangled up in a fine mess.

I remember when my country(Germany) wanted to introduce a unified system for our state police(we've got 16 stat

No, if you get the contract it wont get done either, since you'll spend the next ten years in meetings for 120h per week, while your flunkies try to figure out what the hell they are supposed to do and then it will get cancelled at a cost of 10 billion. The Pentagon never learns and their never ending meetings management process never changes.

Those billions could have put a man on mars, or housed many,many homeless people, or any of a bunch of other uses. When will we realize that most of out debt is crime useless military spending, not social programs?

Useless? Do you know what a "contested sea zone" is and how it affects commerce? No? Yeah, that's what I thought, and the reason why is overwhelming dominance. Assuming, of course, you like imported coffee at the hip indie coffeeshop and hipster fruits like the Durian instead of that crap domestically made junk.

The US government spends 19% on defense, and refunds 19% on social security and 20% on healthcare to recipients among whom many have past contributions in excess of benefits received in total. (This is the nature of insurance, you know. Insurance is a communist plot.)

Wrong. Social Security and friends almost invariably gives back much more than the recipient paid in contributions. Which is why it is not insurance, but a Ponzi scheme backed by general tax revenues (I remember the uproar a while back when s

The only thing you proved is you don't know what a Ponzi scheme is.It's missing 3 crucial elements1) A 'undisclosed' way of making money2) A handful of people collecting the majority of the money.3) Unsustainable under any condition. Meaning no adjustment can be made without collapse the whole thing.

In a Ponzi scheme, the people 'late' the the investment(which is everyone who didn't start it) won't get anything out of it.Social Security is running really well, it has minimal overhead, it's accountable, has money set aside, and is designed to allow for adjustments along the way.

It should be held op up as one of Americans crowning achievements. Right there with the Hoover Dam, Golden Gate Bridge, the Interstate highway and putting a men on the moon.But republicans don't like it so they keep lying about it.

With at least eight full-lifecycle development projects under my belt as both a Software Engineer and a Development Team Lead I cannot even wrap my mind around the amount of irresponsible waste that would be required to throw away that much money.

The real news here is that a branch of the military actually avoided the sunk cost fallacy. I know it's probably not the first time. Nevertheless, I can't help but wonder if they will use the money they save for porcine pilot training.

If you're ever near Washington DC, take a stroll through the Air and Space Museum in Dulles. Its much bigger than the one on the Mall. Of particular interest are the many early rocket projects that were cancelled. The plaques all start off telling how awesome the project was and end with "canceled due to cost overruns". There is most certainly a precedent.

I think I must have spent too much time idly hanging out in RP areas like Pocket D in City of Heroes and The Busted Flagon in Guild Wars 2. Shamefully, I first saw the headline "US Air Force Scraps ERP Project..." as "US Air Force Scraps Erotic Role Play Project..."

Since this thread is just going to be a bunch of "zomg wasted muney!" why don't you educate academics like me about what exactly "ERP" systems are and what you do with it and why its so great?

The university I work at gets new crazy "enterprise" software sometimes and usually it ends up offloading some of the work the bureaucrats used to do on me (purchasing paperwork) meanwhile they take 51% of my grant money.

It manages stuff. People, salaries, suppliers, inventories, clients, payments and whatever else you can think of. In the company I work for we just did a whole school management system (students, teachers, evaluations, etc) on top of OpenERP [openerp.com] (python based, AGPL licensed ERP).

ERP is basically all of the typical office functions put together in an integrated package. For example, an Inventory module to track items, an Order Entry module to allow customers to order those items, an Accounts Receivable system to track the invoices generated when you shipped items to the customer, etc. etc. It's all integrated so, for example, the inventory transactions and the invoicing transactions all feed to the General Ledger for financial reporting.

ERP is a bunch of disparate functions mashed together then held in place with a metric assload of duck tape. It's only natural that if you try to tacle the whole thing at once the result will be a sort of dynamic paralysis where you run back and forth in a nearly random pattern burning money all the way.

Just as well, if you ever manage to build the thing, you'll create paralysis across the entire company if you suddenly drop this chimera on people's desks.

Note, I am NOT claiming that the individual functions aren't necessary nor am I claiming that they shouldn't support common data formats.I am claiming that trying to build the whole thing at once and as a single 'solution' is wrong headed and doomed to failure.

You shouldn't need to build the whole thing at once. A decent ERP system is modular, and can be easily upgraded in place. And while there's always some duck tape, it's still much better than an assorted collection of programs, often times written in different languages and running on different machines (e.g. client vs web based). One of our clients was doing "IPC" by manually adapting files in Excel!

(Disclosure: the company I work for does projects based on a free AGPL licensed ERP system)

Correct, you shouldn't do the whole thing at once. Unfortunately, too many (including the USAF apparently) try anyway. The ability to run distributed is a good thing. That doesn't necessarily (and shouldn't) mean a mis-mash written in different languages and to different coding standards.

I've seen this happen repeatedly. Trying to implement a Grand Plan usually results in nothing.

Almost every large project I've dealt with and was delivered without any major catastrophes was rolled out piecemeal. We picked the core/representative functionality, nailed the scope down with a sledgehammer and built it as "phase 1". All scope creep got pushed to "phase 2". Once phase 1 was done we just pick stuff out of the phase 2 bucket that can be done in the allotted time and just repeated the process. Obvio

Perfect application of Hanlon's Razor [wikipedia.org]: Not so much a conspiracy to waste money as the worst combination of both world (defense acquisition and enterprise software development). Both fields are very prone to overruns, scope creep, and repeated waste of funds as manager after manager--or contractor after contractor--throws away work to start over again. Another great example is the FAA's version of enterprise software [wikipedia.org], which is currently at $63.4 BILLION and counting (though, to be fair, it's quite possible the most complicated software project in the world).

Still, there are worse examples--specifically, when these kinds of overruns, violations, and program restarts are done deliberately to ensure continued funding to entrenched players in a limited field and / or to pursue minor permutations on someone's pet dream of a project. This can occur at the cost of throwing away many years and billions of dollars of decent work while never really getting closer to a functioning system. Space Launch System [wikipedia.org], anyone? (Not a software example, but the line between software and aerospace engineering is a lot thinner than most people realize.)

1 Billion wasted, but remember people, those entitlement programs need to be cut! They're a waste of money! Private insurance companies have overheads of 20% while the government insurance has around 4%, but lets gut entitlements anyway....
I wish we lived in a data driven world....

And Obama's bailout has been more harmful that all recent wars put together. It ensured no financial companies not connected to the main mafia can thrive: they were either bankrupted, bought out or marginalized, while investors received a clear message that

In this country? Whoever led that project within Oracle will probably get a public rebuke, but then a huge-ass bonus for exemplary work. I mean, think about it: Oracle got all the money and has to deliver... what? Nothing. Support? Nope. Roll out? None. Project scrapped, money in da bank. Oracle's all good. Probably get an uptick in the financial magazines for their unbelievably profitable project execution. On to the next project. Booyah!