Mission Failure

As many of you are aware, the launch of the Glory satellite was a failure. The mission would have studied solar irradiance, aerosols, and clouds — all of which are important data for climate studies. Alas, the satellite failed to deploy and the mission — if it happens at all — will have to wait. It’s surely a demoralizing blow to the Glory team, and a blow to climate science since it follows hard upon the launch failure of the OCO satellite two years ago. RealClimate has a post on the subject.

Reader comments at RealClimate have included some conspiracy theory speculation (but just a little, thank goodness!), spurred by two consecutive mission failures of important climate science satellites — that those who don’t want to know the truth about global warming may have undertaken sabotage. I think such speculation is completely unjustified, best left to those who thrive on conspiracy theories. The fact is that launching a satellite into orbit is hard. Lots of things can go wrong, and even in this day and age the chance of mission failure is uncomfortably high.

Still, the question arises naturally, what’s the chance of two consecutive mission failures for a given type of science mission?

I’ll consider the probability of two consecutive failures for a launch vehicle with only a small number of flights on which to base estimation. This is not meant to be an analysis of the probability of the actual observed failures — I don’t even know whether OCO and Glory used the same launch vehicles — it’s just meant to illustrate the nontrivial probability of consecutive launch failures for the extremely difficult task of successful deployment of satellites into earth orbit.

… the worldwide flight history of ELVs from 1980 to 2002 reveals that launch operators who have never launched vehicles successfully before had 8 failures in 11 launch attempts. Worldwide flight history for “experienced launch vehicle developers” over the same period indicates 5 failures in 18 launch attempts. Many factors influence the level of experience of a launch vehicle developer. However, in the results of the recent CSWG investigation, the term “experienced launch vehicle developer” corresponded to developers who had produced at least one launch vehicle with a demonstrated probability of failure less than or equal to 33 percent. The probability of failure was based on the reference values in table A.

Table A (in the FAA guide) gives baseline failure probabilities based on the number of launch failures in (up to 10) preceding launches. They’re based on confidence limits for the binomial distribution, although they don’t specify exactly how the confidence limits are estimated (there are many methods, including Wald, Clopper-Pearson, Willson). My natural instinct would be to use a Bayesian estimate, which is also sanctioned in the FAA guide:

The FAA may also consider other approaches. Once a launch vehicle completes at least two flights, the FAA will accept a Bayesian estimate based on a uniform prior distribution of one hypothetical failure in two hypothetical flights updated with the outcomes of all previous flights of the subject vehicle. The reference probability estimate will be the final estimate input to any launch risk analysis unless the FAA has a reason to make an adjustment away from the reference value.

But when estimating the probability of two consecutive failures, a proper Bayesian analysis doesn’t rely on any single failure probability estimate — it should incorporate all the information contained in the posterior distribution for the failure rate. If we’ve observed failures in missions, the posterior distribution for the failure rate using a uniform prior is

.

The probability of failure on the next launch can be estimated as , as already mentioned. Ordinarily the probability of failure for the next two launches would be that quantity squared. But a full Bayesian estimate, using the full distribution, turns out to be a little different, namely

.

Now let’s plug in some numbers. Suppose the launch vehicle behaves as the FAA expects for a new system from an “experienced launch vehicle developer.” Suppose further that the system has been launched times, with failures. Then the probability of two consecutive failures is estimated as

In other words, there’s a 10% chance of such a run of bad luck!

That’s pretty substantial. Even though this is not a direct analysis of the OCO and Glory systems, I’m guessing it gets us in the right ballpark. So: although the results of OCO and Glory are certainly lamentable, they shouldn’t be regarded as implausible. And they certainly shouldn’t be regarded as so implausible that we start entertaining conspiracy theories about sabotage.

Here’s hoping that both missions are tried again soon, with complete success.

65 responses to “Mission Failure”

Not consecutive (although I have no idea how you’re defining “consecutive”, anyway.

Also, Glory and OCO used exactly the same launch vehicle. They failed in exactly the same way, after an expensive investigation into the first failure. Now that, I believe, is rare – the same failure after all that work?

I’m not a conspiracy theorist, but I’m convinced that the contractors involved aren’t exactly doing their best work.

But some Republicans, who hold a majority in the House of Representatives, want to see NASA give up climate science so it can focus on returning astronauts to space once the 30-year-old shuttle program ends later this year.

So: although the results of OCO and Glory are certainly lamentable, they shouldn’t be regarded as implausible. And they certainly shouldn’t be regarded as so implausible that we start entertaining conspiracy theories about sabotage.

I certainly agree with this part, though – the private industry Taurus system currently sucks, as did early NASA and ESA efforts.

I think it’s teething problems, which the commercial supplier felt they’d addressed after the first failure, which I’m sure they’re going to re-evaluate.

I’m pretty sure there are those who are happy about these failures, so I have to admit sabotage did cross my mind.
But then I reminded myself about the World Trade Centre, contrails and tin foil and the simple fact I do not in any shape nor form wish to be associated with those of that ilk.
I’ll let the experts try to sort this out.
There are far too many armchair experts out there already and I have no desire to add myself to that number.

Bearing in mind the probability of such mission failures, is it possible to compute the advisability of always building two identical satellites, rather than just one?
Presumably building two satellites simultaneously costs considerably less than twice the cost of building one and also less than the cost of building one satellite after the first if the first fails.

While the launch systems are the same, the system that failed was different. In fact, the system that failed on OCO was replaced with the system that failed on Glory.

I do wonder how much testing they did on those farings. Did they do separation tests? How many? Were they relying on computer simulations without validating these simulations with tests? I know that the aerospace industry is relying on modeling of the designs to a much, much greater extent to lower costs at the expense of testing. Maybe this is an example of trusting too much on computer modeling.

Since satellite launch is a high risk, high value business, using public money in this case, wouldn’t it make sense to buy a satellite insurance policy and let that pay for the rebuilding of the lost spacecraft?
And another thing: wouldn’t it also make sense to use a more expensive, but more reliable launch vehicle than the Taurus XL?
Global climate research is more important than ever and we’re counting on a rookie launch vehicle with a 30% launch failure rate?

So it was the same rocket as the OCO, and the same rocket as the OCO-2 (a replacement for the OCO) will launch on in 2013.

This is only the 9th launch attempt for the Taurus XL rocket, and so far the rocket has had 6 successes. It’s really not surprising there a series of launches using the same rocket. In order to cut down costs, its often better to order a bunch of rockets rather than one at a time. The “conspiracy” here is likely one of economics and and issue with the rocket’s fairing (it was the same type of failure as OCO), nothing more.

IMHO, the relevant probability is even higher. If someone is going to suggest a conspiracy as soon as two consecutive failures to launch a climate satelite ocurrs, then the probability of interest is the probability of observing two successive failures in the n attempted launches of climate satelites (whatever n may be) that there have been by now, rather than only the two most recent. That would be the probability of someone making an accusation of conspiracy by now, even though the failures are independent and occurring at the usual rate.

For the people who are calling this sabotage, please answer the following questions:

*Who* performed the sabotage? How many people have access to the rocket or the payload? It’s not like *anyone* can walk up to the payload and futz with the fairing separation system.

*When* did they perform the sabotage? During payload integration, in full view of all the other engineers and technicians? On the pad, after closeout, when the rocket was vertical?

*How* did the sabotage get past the final closeout inspection? Obviously, the people performing the final closeout would have to have been in on the plan.

There are a limited number of people with access to the rocket; you’re claiming one of them deliberately sabotaged it, putting their careers and the careers of their colleagues at risk for…what, exactly?

First, let me say that any speculation of conspiracy or sabotage places one well into the tin-hat crew.

Second, the fact that they had such a similar failure even after the post-OCO FRB redesign may point to a systematic problem with Taurus.

Third, I think the Taurus now has a reliability record worse than Long March–just more evidence that a faith-based space program is probably not advisable.

Fourth, the Earth Sciences program at NASA is woefully underfunded. The satellites are built on a shoestring, and the funding limits the technologies that can be chosen. One of the criteria that requires such simple satellites is that the satellites are launched on wimpy systems like Taurus. Launch systems continues to be a prime vulnerability of the US Space Program.

Fifth, as Gulf-State congressional delegations–populated mostly by the science-hating GOP–continue to raid Earth sciences to prop up the pathetic manned program, expect less and less help from orbit in understanding how we’re screwing up the planet. Flying blind is a policy advocated by one of the two main parties in this country.

Whether you are a “true-believer” or a “skeptic”, the loss of Glory is a travesty. If you are really looking for the truth of what’s going on, the only way to understand it is to gather data. And this program would have done just that… what a shame!

And ditto about the sacrificing of the space missions to the high altar of STS/CX/JWST…

From what I have seen of the various conspiracy theories I’ve noticed that the people who push them usually have a profit motive. This is usually them selling their book detailing the “Truth” (WTC/Pentagon Conspiratists), but can also include things like getting hits on a web site (EQ Pegasi Hoax) or even the gain of followers (UFO Nazis of Ernest Zundel). So one not only has to ask themselves who profits from the conspiracy, but who profits from the belief in the conspiracy.

That said, it should be pointed out that hard, firsthand evidence trumps all. If you want to find out if there is a conspiracy or not, talk to the people who physically worked on the rockets.

Very unfortunate indeed. In contrast to conspiracy theories, this has “lowest bidder” written all over it. I can’t imagine multiple consecutive failures (in 2011) for billion-dollar Air Force projects.

“… Orbital Sciences Corp. subsequently modified the fairing design, based on analyses by a NASA panel that reviewed the OCO launch failure.
The original version of the Taurus rocket used hot, pressurized gas to break frangible joints that hold the fairing in place, beginning a process that ends when pistons push the fairing pieces away and the satellite moves into orbit.
The revised version used in today’s Glory launch used cold, compressed nitrogen gas to break those frangible joints….”

I’m happy to hear you reject these conspiracy theories. I wouldn’t consider myself to be completely convinced by the consensus, however conspiracy theories wouldn’t help anyone either way. You pegged it when you state that it’s hard to get to space.

Andrew Dodd (at 11:52am) While chemical rockets might not be ideal, they are infinitely better than all other options since none exist. There has been a lot of money dumped into trying to find cheaper access to space. From an engineering perspective, the most feasible so far is mass production of these chemical systems (flaws and all).

I wonder what the feasibility is of incorporating parachutes and flotation devices that deploy if separation fails. Apparently the trajectory is entirely over the ocean until that critical point. We cannot afford to lose OCO2 if this happen again.

Also, anyone know why they do not use the system that will be used to launch, for example, the Landsat Data Continuity Mission?

Jim,
In the event of a launch failure, the vehicle is typically on a ballistic trajectory. These are of necessity very aerodynamic vehicles, so terminal velocity is somewhere around “freakin’ fast” . Even if you could separate the payload from the vehicle, a parachute wouldn’t slow it down sufficiently before it was torn to shreds. Shedding momentum is one of the toughest portions getting a payload down safely.

LDCM is a big bird. A Taurus wouldn’t work to launch it even if the Taurus worked to begin with. OCO and Glory were relatively small, cheap satellites. The decision to use the Taurus was probably predicated entirely on cost. In fact, the launch capability of the Taurus was likely one of the driving factors in the design of both birds.

Vendicar Decarian,
It is simply flat irresponsible to speculate about conspiracy before there is any evidence. Nor is conspiracy necessary. Taurus is a crap launch system. Its marginal lift capability often drives mass requirements and thereby limits capability. Unfortunately it is also rather cheap, so bean counters like it. If you want someone to blame, blame gummint procurement regulations, not the engineers.

It is worth noting that the Taurus rocket is built by a private company (OSC if I recall). Usually conspiracy theorists like to have the government screwing the people. In this case, it’s rather the opposite, with a private company’s subpar product costing 700 millions to taxpayers in 2 repeated failures. I’m sorry but 6 sucesses out of 9 launches is nothing to brag about. That vehicle should not be used again.

Tamino, thanks for airing this. It does remind us of how dicey rockets are. To put a statistical foot on the floor, we obviously don’t know how many military launches routinely fail and land in the Pacific, because we in the US live in the equivalent of the USSR in 1971. It might behoove us smart folks to histogram the secret v. non-secret launches over the past 40 years. Oops, can’t do that. It’s all secret. Puzzle this. We, the people of the US, have absolutely no idea what is careening over our heads in orbit. It’s all top secret classified.

OT: Tamino, doubtless you will have heard about the new Rignot et al. ice sheet mass loss study that was announced. I haven’t read the paper yet, but the graph reproduced by Tom Yulsman causes me to wonder intensely if the trend really is linear, especially given that 2010 was a big mass loss year (in Greenland at least). It seems like a great topic for you to post on.

OK, had a first read of the paper (thanks to Eric Rignot for a prompt response to my request) and it does indeed look like a good topic for this blog. Tamino, I’m thinking of this in the context of Hansen and Sato (2011), in which it is proposed based on close examination of the two warmest interglacials for which SLR proxies exist that a fast collapse of the ice sheets may already be starting, and that an ice loss doubling time of as little as ten years (anything close to that would mean fast collapse) can’t be excluded based on existing observations. It seems unlikely from the timing that they would have seen the Rignot et al. analysis.

I’m asking where and with whom you think the opportunities for sabotage lie. Not who did it, but who could have done it. Not how they did it, but how they could have cut some wiring without anyone else noticing at the time or after the fact.

A limited number of people have access to the spacecraft. Systems are constantly checked and rechecked. Windows of opportunity for what you’re describing are vanishingly small.

“I’m asking where and with whom you think the opportunities for sabotage lie. Not who did it, but who could have done it. Not how they did it, but how they could have cut some wiring without anyone else noticing at the time or after the fact.” – Jfb

All good questions. Do you have any answers?

[Response: You’ve had your say, more than once. Now drop it. That goes for everyone.]

“First, let me say that any speculation of conspiracy or sabotage places one well into the tin-hat crew.” – Ladbury

Speculation is an integral part of scientific investigation.

You know, like Feynman speculating that the challenger o-rings were to blame for it’s failure.

“Fifth, as Gulf-State congressional delegations–populated mostly by the science-hating GOP–continue to raid Earth sciences to prop up the pathetic manned program, expect less and less help from orbit in understanding how we’re screwing up the planet.” – Ray Ladbury

Correct. Starving the beast has long been the Libertarian/Randite Plan.

I’ve quite honestly been watching that plan unfold for literally the last 35 years.

“We need to manufacture an crisis in order to assure that there are no alternatives to a smaller government.” – Jeb Bush, Imprimus magazine 1995.

The problems with your speculations are:
1) It is not necessary to sabotage the Taurus for it to fail. It’s a crappy launch system.
2) Launch vehicles are crawling with sensors and telemetry. A “snip, snip” would be very difficult to hide.
3) If one were trying to systematically sabotage a program, having the same system fail would be a stupid way to do it. This is much more likely to be a fundamental unreliability of the system that failed.
4)Speculation in the absence of reliable information can cause you to ignore clues that could tip you off as to what is actually going on.
5)If you want to look for skulduggery, you’d do better to look at the whole procurement system. Why did Orbital win the contracts for OCO and Glory to begin with? Why was Glory launched on a Taurus, despite the recent loss of OCO on the same vehicle?

Now as to your allusion to Feynman’s O-ring trick…that was more theater than science. The O-ring problem was well known long before Challenger. Several other missions had nearly burnt trough the O-rings previously. Morton-Thiokol’s engineers unanimously recommended against launch in the sub-freezing weather at the cape–to be overruled by management under pressure from Marshall, who was in turn under pressure from the administration (Reagan’s State of the Union, scheduled for that night had mention of a teacher in space).
Feynman was briefed on the problem by those who already knew about it. He performed his role in the theater piece masterfully, but the cause of the disaster was well known even as it happened.

The real conspiracy here concerns why a program critical to the future well being not just of the country, but of the species is being run on a shoestring–and why pieces of that shoestring are being appropriated to keep a huge manned pork barrel in orbit.

“1) It is not necessary to sabotage the Taurus for it to fail. It’s a crappy launch system.” – RL

Certainly the fairing design seems inadequate provided that there was no sabotage. But there is no reason to decry the entire launch vehicle.

“2) Launch vehicles are crawling with sensors and telemetry. A “snip, snip” would be very difficult to hide.” – RL

“The more they overtake the works, the easier it is to stop up the drain.” – Scotty – Chief Engineer USS Enterprise.

Ya, it’s scifi.. But the sentiment is exactly correct.

“3) If one were trying to systematically sabotage a program, having the same system fail would be a stupid way to do it.” – RL

Your point fails on two grounds. First it neglects the fact that most crimes are crimes of opportunity, and there are probably limited opportunities for undetected sabotage as you will undoubtedly agree.

Second, you presume it is directed at a program rather than a mission.

I agree with you that if there is a conspiracy it is the Republican Conspiracy to defund the science that disagrees with their political liedeology.

But I haven’t seen anyone claim conspiracy. The suspicion is sabotage, and sabotage does not require or imply conspiracy.

“4)Speculation in the absence of reliable information can cause you to ignore clues that could tip you off as to what is actually going on.” – RL

And without speculation you may miss clues that can tip you off as to what is really going on.

It works both ways.

“Why did Orbital win the contracts for OCO and Glory to begin with?” – RL

That is simple to answer. It is because NASA for political reasons is cow-towing to the Republican ideal that every aspect of government should be privatized, so this company was chosen because they are outside the typical government contractors who are used for launches.

As to the particular vehicle used for the launch, I can not say. I presume that NASA was looking to shave as many pennies as possible off of the cost of launching in order to save funds for it’s other missions which are chronically underfunded.

“The real conspiracy here concerns why a program critical to the future well being not just of the country, but of the species is being run on a shoestring–and why pieces of that shoestring are being appropriated to keep a huge manned pork barrel in orbit.” – RL

I agree entirely. And we all know the answer. Starving the beast provides the Republican party to preferentially starve those programs that produce results that are counter to their political liedeology.

I mention this often on RealClimate, but somehow they aren’t interested in such things and invariably delete those messages.

The American people deserve the government they elect, and the world deserves the American state that it tolerates.

I prefer honesty.

[Response: Now that you’ve fully had your say about sabotage, let’s put that subject to rest.]

Dhogaza,
I don’t think there was any way the cause of the failure would have been swept under the rug. In addition to Feynman, you had Morton-Thiokol engineers highlighting the problem with the O-rings. You had astronauts on that panel who lost friends in that disaster. And in the end, what really happened? Well, Morton-Thiokol never paid a dime of the fine that should have been levied against them, and nobody bothered to investigate where the pressure to launch was coming from.

In the end, it was the Agency that fell on its sword to protect political interests above it. That was probably the beginning of the end of NASA.

“I was also surprised that the weight of the fairings could keep the thing from achieving orbit.” – JB

I was a bit surprised as well, but then it has to be a robust structure to withstand the stress of launch.

Fuel needed for launch is computed based on the orbit desired. Keeping extra fuel in the tanks at orbit means exponentially higher volumes are needed for launch. So they don’t want to keep much in reserve.