You don't mean the kind that looks like jillions of tiny tires (or black donuts) intersecting with the wires of a chain-link fence, are you?

Yes, he does mean Core Memory, and yes, the AP-101 as flown in the Shuttle from mid-70s through to mid-90s did indeed use Core memory.

Indeed, the upgrade to the AP-101s with (I think) static-column RAM took so long because Core memory has the lovely property of retaining information even when the power dies - a key factor, sadly, in the ability to retrieve information from Challenger's onboard computers after the 1986 crash. Another key factor is that Core memory is remarkably resilient to bit-flipping caused by cosmic rays and other radiation (events known as "SEUs" or "Single Event Upsets").

All of which meant that it was a major project just to replace that memory with more modern RAM. And it's not just a couple' sticks of SDRAM either - most of the space-savings you'd expect from replacing bulky core with nice compact RAM chips is taken up with additional hardware to a) provide sufficient power support to retain memory in the event of main power failure b) continually scan through memory doing parity checks to detect and correct for SEUs...

the upgrade to the AP-101s with (I think) static-column RAM took so long because Core memory has the lovely property of retaining information even when the power dies.....Another key factor is that Core memory is remarkably resilient to bit-flipping caused by cosmic rays and other radiation

70's computers seem much more radiation friendly than the newer stuff. It is harder to take advantage of Moores Law on spacecraft due to radiation out there, especially when visiting gas giants like Jupiter that are swarming with radiation. (Not likely the shuttles will be visiting Jupiter anytime soon, though.)

The Checkout and Launch Control System (CLCS) was created differently than LPS. It provides more safety and more operator visibility into launch problems before they occur. It is expandable and capable of being upgraded. It is designed to accomplish not only today's Shuttle launches, but also provides a launch capability for any future vehicle.[emphasis added]

CLCS would apparently be flexible enough to control anything NASA needs for several decades.

I would think that requires rebuilding the whole thing anyway. So it might not actually improve it.

Also you have to ensure that the simulator has zero bugs, which means simulating the bugs in the original equiptment which their code depends on.

Writing a perfect software simulation of hardware is IMO a job as equally hard as just rewriting the original code.

It's not like they have millions and millions of lines of code, the original rom must to have been less then 64k or so. They just have to rewrite the code in a language that is more maintainable which machine code is not.

Auditing the emulator and the host OS would be a problem - the code they've currently got has a very low rate of bugs, and has been extensively audited. NASA knows everything from the hardware up, exactly what the failure rate is and so forth.

Now, imagine you take modern commodity hardware (which changes periodically - look at how often Intel silently release new steppings of their CPUs). You're not going to have a guarantee of consistency there. You're going to have to boot an OS off it - and even the simplest RTOSes are still much, much bigger than the whole platform currently. Then you need an emulator. Then you need the system. And the only problem you've solved with all that work is the unavailablility of the old hardware - you still have a old machine language on a tiny platform which can't be easily extended for new functionality.

This is a very pertinent point that appears to have been lost on the initiators (and now burger flippers) of the replacement-launch-thingy project.

What they have, right there, is one spectacularly reliable piece of software. I suspect it's significantly more bug free than even the microcode in a modern processor, let alone the companion chips, bios, operating system, and virtual machine for some god awful p-code language (not that I'm naming names here).

The question that should have been asked is "how can we make a sustainable process for making extremely reliable control computers?". How to go about cutting custom silicon, tiny os's etc. How to save the happy tax payer hundreds of millions of dollars by reselling these services to people making nuclear power stations, heart pace makers etc. instead of going shopping for big sun boxes.

"Now, imagine you take modern commodity hardware (which changes periodically - look at how often Intel silently release new steppings of their CPUs). You're not going to have a guarantee of consistency there. You're going to have to boot an OS off it - and even the simplest RTOSes are still much, much bigger than the whole platform currently. Then you need an emulator. Then you need the system. And the only problem you've solved with all that work is the unavailablility of the old hardware - you still have a old machine language on a tiny platform which can't be easily extended for new functionality."

Might I suggest using FPGAs [vcc.com] to emulate the hardware old system so the software doesn't have to be thrown out?

Assuming that circuit layouts are available for these old chips, it would be a piece of cake to emulate them in VHDL (a hardware description language) because they are comparatively simple to today's integrated circuits. Once the chip descriptions are written in VHDL, it would be relatively easy to 'port' the hardware over to a new FPGA if the old one dies or whatever. Then it would not be necessary to truly port or re-code any of the currently working code, and it would be much easier to fix bugs and extend it because you don't have the memory and speed limitations of the old system.

Replacing it can be harder. I used to work in newspaper publishing; the core editorial systems of one employer were old ATEX J11 systems with a proprietary, tightly integrated OS and application suite. Over time, various aspects of the system were offloaded to more modern systems (eg, PostScript output and integration with graphics from desktop systems had dedicated AIX systems, imagesetters driven by PostScript RIPs, dumb terminals run from dedicated I/O boards replaced with terminal emulators on the desktop).

Despite all this tweaking, the crufty old systems stayed in place. Why? Well, on each of these old boxes, we could support 25-30 journos and the systems just worked, grinding out newspapers day after day.

People kept talking about replacing them, not least because we had to train up operators and engineers on them every time new staff came in, parts were hard to come by (the standards-not-compatible SCSI and ethernet interfaces were picky about what they talked to, and the filesystem could only address 600 MB of disk per system), and they used huge amounts of power and floor space.

For the three years I worked there and in the three years hence no-one has been able to deliver an editorial system that just works. When vendors rolled their rigged demos in, they crash. The major vendors like CyberGraphics and ATEX couldn't point to successful implementations of their new systems producing a decent number of newspapers on the basis of more than one edition per day.

Would it have been nice to have a Unix or Windows based system? Sure. Reduced overheads and training burdens, able to buy the latest and greatest hardware, and so on. But no-one could actually deliver something that worked better than the crufty old J11 systems.

NASA are probably in a similar bind; it's a very familiar problem: old systems developed by tight, focused, skilled teams and developed over the years are very, very hard to replace.

Surely with slightly more advanced, but far more available, and stable and simple technologies (programmable gate arrays, etc), they could do a next generation (well,.5 generations ahead) version of this.

The 7400's and such are just interface hardware; that logic, as well as replacement for the 64k ram, etc., could all be put on a single reliable chip (no I386, no heavy OS, no emulator).

There's got to be an only-slightly-more-complicated and nearly-as-reliable by not being too ambitious with the latest tech.

Or use old, but solidly reliable, thoroughly-tested and completely documented CPUs from Ye Olde Days, like the 6809. Or even the 6502.

When dealing with shuttles and the like, K.I.S.S. should be the mandate. Ain't nuttin' complicated about a 6809 mobo, yet it can be coupled with an MMU to provide upwards of 1Mb of memory, and there are excellent realtime, multitasking OSes available... with source, IIRC (OS-9, from Microware; and I think there's a QNX for it, too. Plus [ooooh, he stretches his memory...] Flex-09?).

There comes a time in every products lifetime when its time to start over, and I believe this is the time. In the 70s, we knew significantly less about good coding practices as today. GOTO was first becoming considered harmful. Procedural programming was on the rise. Object-oriented as a paradigm was beginning to take root.

With these considerations in mind, clearly simulating an old computer is a very backwards idea. bug-for-bug compatibility [tuxedo.org] is not a positive effect.

Same as bug-compatible, with the additional implication that much tedious effort went into ensuring that each (known) bug was replicated.

There comes a time in every products lifetime when its time to start over,.

Exactly. And that includes the shuttle. It has never lived up to what it was envisioned to be and it is only going to become more costly and more failure prone in the future as every bit of hardware on that pig is already showing signs of fatigue.

There are many launch systems that cost far less per pound to throw things into orbit. The reasons we still have those monstrosities flying are political only, not technological or scientific.

Sure this is flamebate. (Gosh, getting rid of the old karma system is so LIBERATING!) But if we can discuss how some little bits of hardware in the shuttle are past their time, why can't we discuss the big bit?

As a proponent of an aggressive manned space program, I'd agree that the shuttle has not lived up to its initial objectives. This is due, in large measure, to the shuttle's inability to launch with the frequency NASA intended. The actual design of the shuttle was permanently compromised in political and budget battles during the Nixon administration.

Few, if any, expendable boosters have the lift capacity of the shuttle, so I'm skeptical abut your claim that "anything would be cheaper. But that's a quibble. More importantly, in the two decades that the shuttle has been flying, no one -- government or private -- has made a serious effort to design, build and launch boosters large enough to dramatically reduce cost-to-orbit. The technology is there; it has been there since the 1960's. What's keeping this from happening? Timidity and lack of political will. The "Final Frontier" is in the hands of bureaucrats and corporate execs who can't see beyond the next commo satellite launch. It's as if the only reason to move into space is to make our damn cell phones work.

how do you fix it? Oh, that's right, you send up a space shuttle full of people to fix it.

Not for long. Robotics is getting better and cheaper faster than space tech + expensive humans. It's only a few millisecs of latency for telepresence, until AI can do the job locally (much later).

Do you really think that private enterprise is going to terraform other planets?

Terraforming? How quaint. Extrapolating tech into the future indicates we'll sooner be taking the planets apart for raw molecular building material than wasting space & material & escape velocity energy, by living on the tiny surface area of another gravity well.

Can you come up with a better way to get humans into space (and bring them back if needed)?

Yeah. Divert a larger chunk of the money that would be wasted on chemical rockets and deadweight NASA beaurocrats into nanotech research so we can bootstrap ourselves off this planet much much earlier.

The HST was a PR thing anyway, for the same money they could of build several ground based telescopes that made the same nice pictures

Huh ? at what frequency regimes ? what about X-rays ? IR ? UV ?

besides, the atmosphere does distort image even for visible-light imagery. It is true that advances in image-fixing algorithms made, AFAIK, in the last decade attenuate the problem to a large degree, but, AFAIK, there was nothing like that in the seventies, and nothing is better than eliminating the problem altogether anyway.

perhaps today it will better to build bigger telescopes on earth than launch them (again, for the visible light regime. I very much doubt this is true for X, UV or IR imagery... ) but I don't think this was even remotely true when the HST was designed and built.

saying such a large project, with published scientific results, is "just PR" with no references to back up your claim seems like slander to me.

I wouldn't be amazed if somebody told me they fitted the broken mirror on purpose so they could go and fix it with the shuttle...

I wouldn't be amazed by a lot of things, but I don't ususally go slandering hard-working people just based on what I suspect they are capable of doing.

The US space program and NASA deserve (and get) a lot of criticism, much of it is quite pejorative, much of it is technically sound. I haven't seen any such thing in your post, which is IMHO just nasty unbased negative PR.

Remember that whatever machine you use to simulate it in needs the special connections and circuitry to interact with other parts of the ship. You can implement the software fine but how do you connect the engine, flappers, and sensors to it?

You can't just buy a system from Dell and put it into the Space Shuttle. You can't use a Pentium, a modern hard drive, Linux, Windows, or Open Source anything.

As far as the hardware goes, everything mission-critical that goes aboard the Shuttle has to be ruggedize against incredible vibration, tested a thousand different ways to make sure that it can't be affected by exposure to vacuum/heat/cold/radiation/cosmic rays/etc., tested another thousand ways to make sure it doesn't interfere with other critical Shuttle systems... and on and on.

And a bug in the newly written software could cause not only the death of several astronauts, but potentially the loss of a Shuttle, a launch facility, and the ISS. Would you, under any circumstances, put your life, five other lives, and billions of dollars in the hands of software that you found in an Open Source project?

I believe the discussion is about a computer that is based on the ground.. even if not, I think you fail to realize that NASA has been using Linux in space for a while.. PC104 boards with flash (solid state) memory running Linux.

I don't, honestly, think much of the nation, save the immidiate families and friends of the astronauts, would mourn.

There hasn't been a front page story of a shuttle launch for as long as I can remember, and even TV stations don't broadcast it like they used to do in the 80s.

Besides, the space program is not, at this point, existing in a vital capacity for our nation. There are no weapons platforms orbiting the earth. There are no survillance photographs. Hell, there isn't even a space tourism project run by the United States or a U.S. corporation -- the Russian government does that!

The Russian equipment is good enough to get people up and back safely -- and they're not above taking millions of dollars to let someone else experience it. The U.S. seems to have a hissy fit when they suggest it -- NASA is about "science" not "entertainment."

The nation's (and the law makers and budget appropriators) backwards views on the space program in general need to change, and before any type of a hardware change. It needs to be retasked to be under the Department of Defense, or outsourced to the public sector. Private companies build satellites that work right. Why can't they build the launch platform and send people up, too?

This is a common problem in big projects. The time it takes to design a system and then actually implement that system is so great, that by the time the sytem is complete, the hardware used to make that system is 'obsolete.' You can't just add more memory and speed, because then you'd have to go through and make sure that everything still works perfectly, and that would take so long as to make the current hardware 'obsolete.' The real problem here is public hype. You don't need 4 GHz and 40GB of memory to program the space shuttle, but if the public finds out that NASA only uses 64k, they will think NASA is behind the times, even though 64K is enough for the system. Of course, the space shuttle is already considered obsolete by some, and new sytems are being created, so don't fret much over this.

Actually, the old AP101 computers may have had 64k of memory (I don't recall). We upgraded those bad boys a long time ago to AP101S which have a whopping 256k. Who could ask for anything more.

FYI: That extra bump in memory allowed us to store the entire Entry program in upper memory so that in the event of a Trans-Atlantic abort, we wouldn't have to wait 20 seconds for it to load from the mass memory.

The time it takes to design a system and then actually implement that system is so great, that by the time the sytem is complete, the hardware used to make that system is 'obsolete.'

Which is why serious software engineering is done on platforms like the SPARC, where you can guarantee that later CPUs can run earlier code, or on IBM operating systems where everything is a virtual amchine anyway.

It's not the software that's a problem, it's the hardware. What if we want to update the hardware one day (which is what the article is talking about)? Well, let's assume we used a language that SHOULD be platform dependant in the first place. Do we assume the infalliability of the language implementors or do we test every possible thing and make sure it works with the new hardware? Well, this is space buisness, we're dealing in lives, of course we check everything, and that takes time, which was my point.

I don't know about everyone else, but when I was a kid I got a Radio Shack 300-in-1 electronic project kit for my birthday which came with a dozen or so 7400 chips. When I plugged one in backwards I just went down to my local Radio Shack [radioshack.com] and picked up a new 74LS00, which they had plenty of in stock all the time.

Certainly the 7400 series as a whole is still widespread and used in hobbyists kits, I'm not that old. Maybe the original 7400 is becoming obsolete, being replaced with the 74LS (low-power Schottkey) or CMOS chips? If then it shouldn't be too difficult to replace the TTL logic with CMOS logic, given a few adjustment levels in voltage, or they could use the TTL-logic and CMOS-logic in one compatible chips [cjb.net].

Of course, the 5400 series SSIs (small-scale integrated circuits) are preferred over the 7400s for industrial purposes, and as a plus they are completely backwards compatible. Why isn't NASA using those?

You can cause a lot of problems by replacing a part in a working system with the manufacturer's new and improved part. Often the new part has faster outputs, which can change your PCB layouts from working to marginal.

Given the above, I still don't see why they would not reimplement the whole thing in a slightly newer logic family and requalify it.

The 54 series parts were like the 74 series, but in a hermitically sealed case, 100% tested over a wider temperature range, and burned in to remove infant failures. For this application they used space qualified components. The same as 54 series parts, more stringent tests, and now the chips are also evaluated for radiation resistance. Any change in the design or production process and the 54 & space qualified chips must be requalified. What can happen is that a chip is produced to be fuctionally the same, but using smaller geometries, and now is more suseptiple to ESD and radiation.

CMOS chips, because of their high impedances, are notorious for ESD and rad sensitivity so they won't do.

With the reduction in military, aerospace, and space spending many manufacturers have dropped the 54 series and space qualified components. They haven't made any attempts to add replacements in their product lines.

When a part is dropped, the manufacturer usually informs the industry of their intent. You're given a date & price for a final order. the theory is that you can buy a lifetime supply of these parts. Industry isn't likely to but any more than they need to complete existing contracts plus a few spares, there's no guarenty that you'll get any more contracts to build items requiring these parts so these purchases will cut into your profits. Government procurment may buy additional components, but lack funding to really buy large quantities.

An opportunity is presented, and they will be taken advantage of. A distributer might buy some additional parts -- since the distribributer has several customers buying a particular part from him, his risk of being stuck with an unseable component is small.

After the final production run, the chip manufactorers will sell the documentation, tooling, and rights to make a chip. There are small manufacturers who buy these, all well as the out of date machinery to produce these parts. They can then make small production runs, sometimes under a hundred components, for a price. In addition, they might buy untested dice or wafers from the last production run. The untested & unpackaged componets are very cheap, so it's more affordable & less risky to buy and store these than the completed components.

When I plugged one in backwards I just went down to my local Radio Shack and picked up a new 74LS00

Dunno about the Shuttle, but I assume my experience applies. I used to write autopilot & autostabilser software for helicopters. They used 80286 & 68000 CPUs, which have started to become more difficult to find. Not because there are no 286's or 68K's out there, but because there aren't so many 286's and 68K's available that are certified for flight.

Remind me not to fly on any plane that you worked on the avionics for. There are times when overwhelming paranoia is an asset. When your Playstation crashes you can always start a new game. When your main engines get told to hardover on launch because of a failed chip, you die.

its a good question. not sure, to be honest. i was told that they were supposed survive more severe conditions - heat, humidity, radiation etc, no idea if that's true though. i seem to remember that their entire history is documented so they can guarantee they've not been mishandled by some intern who hadn't been on the anti-static course. So much of it could be paperwork, but its still a burden that has a cost, and most definately rules out the use of second hand CPUs.

I once asked the electronics guru at the university satellite lab where I used to work why were were launching a 386 when we could have stuck a more modern processor (like a StrongARM or something) on the main board. He pointed out that older style chips were preferrable because the gates and interconnects were all bigger. A bigger gate was less likely to get triggered or flipped (if it were in a register or something) by a stray particle of cosmic radiation. Low tech chips were easier to certify for space use because of this.

I'm sure we could have gotten faster chips rated for space, but we were on a tight budget, so the 386 was it.:)

(2) Break down the old mainframes until you have roughly 50,000 pieces...

(3) Sell it on eBay (or other auction sites) as space memorabilia, mention that the computer the parts came from were responsible for guiding the Apollo missions to the moon, etc and so on... The machines are SO obsolete now that the only way they could pose a security risk is by sending them back in time...

(4) Profit!

(5) Buy a nice little beowulf cluster, hire 20 Linux geeks and feed each of them $50 in dew and pizza in exchange for setting up the system...

(6) Use remaining funds to pay the Russian space agency to have a little "airlock accident" for that Nsync guy...

In re: point number 6, I know you'll be sad to hear that 'N Sync guy's flight is no longer on. There was an article in the NY Times last Wednesday that made me laugh.

...
[Lance] Bass, of the pop group 'N Sync, had been training at the Star City cosmonaut complex outside Moscow; he was told today to pack his gear and leave after "failing to fulfill the conditions of his contract," a spokesman for the space agency told Reuters.
Adding insult to injury, the space agency said Mr. Bass, 23, would be replaced on the October mission by a cargo container.

Let's face it; the Space Shuttle is obsolete- it's 30 year old technology barely warmed over. It's completely failed all of the main design goals; NASA told congress that they were aiming for costs as low as $500/kg and 5 nines reliability- they're currently at about $20,000/kg and only 2 nines reliability. These are not small issues. Missing the target price by forty times is an enormous gap.

In fact, the Saturn V was able to launch 4x as much for about the same cost. It could probably have launched most of an ISS in a single launch, and tacked on more sections in 2 or 3 more launches.

In fact, the Saturn V was able to launch 4x as much for about the same cost. It could probably have launched most of an ISS in a single launch, and tacked on more sections in 2 or 3 more launches.

Here's where you lose on your argument. The manned vehicle compartments that were available to the Saturn V made extra-vehicular activities very difficult. The Shuttle has a much better design for short-term laboratory work, for crane work, and for extra-vehicular work. You could dump all the parts for an ISS into orbit with a few Saturn V trips, but you couldn't work with those parts to assemble them. In fact, you'd have to drop the parts a fair distance away and then find some way to bring them closer to the building orbit location. You wouldn't want to chance using NASA's equivalent of a Greyhound Bus to maneuver anywhere near the multi-billion dollar parts (and stationed lives) that may already be on-site.

The manned vehicle compartments that were available to the Saturn V made extra-vehicular activities very difficult.

It seems there are much easier ways around that issue than to design something as costly as the shuttle.

You wouldn't want to chance using NASA's equivalent of a Greyhound Bus to maneuver anywhere near the multi-billion dollar parts (and stationed lives) that may already be on-site.

To use your analogy, then you have the Greyhound bus tow a little delivery vehicle, rather than trying to turn the Greyhound bus into something that can both carry lots of cargo and maneuver like a forklift.

The Russians have a shuttle and have had one for at least 8 years (I saw it in person about the same time of the failed coup, Sept 1993, in Moscow at MAKS (the Russian Air Show.)) I have a great picture of an IL-76 EWACS version with their shuttle in the background.

Buran -- Russian for snowstorm -- is the name of the Russian shuttle that made one unmanned spaceflight in November 1988.

It circled Earth twice, landed automatically and since then has sat in storage at the Baikonur Cosmodrome in Kazakhstan. On May 12 the vehicle was damaged by falling debris when portions of the roof of the building the Buran was in collapsed.

Several other copies of the Russian shuttle were built as part of a test program and through the years have all become known by the name Buran.

Of those, one Buran was turned into a space-themed restaurant at Gorky Park in Moscow and another was given a fresh coat of paint before going on display for more than a year in Sydney, Australia during the time of the 2000 Summer Olympic Games.

Russian hardware makes lots of odd choices. Much of it can be traced to saving money; and the Russians are very good at this, compared to NASA.

The Space Shuttle main engines are taken out, torn down and rebuilt after every flight. I don't believe this to be very cost effective.

One point that is missed is that it costs much less to mass-produce engines, the Space Shuttle main engines are one-offs, and hence are very expensive to start with. The Russian approach is to reduce costs via mass production. I believe that this is a sensible approach right now. Only if the launch rate is high enough is reusability going to win.

The code in the Shuttle's launch system is old? The entire Space Shuttle is old. I'll bet a lot of slashdotters don't even remember the Columbia's maiden voyage.

I'm not one to replace things that are working fine, but as I understand it, newer designs could be a whole lot cheaper to operate. So I wonder if pouring more into the Space Shuttle program is the best thing to do.

I'm not saying "let's throw out the space shuttle" but it bothers me that there's apparently nothing in the works with a decent shot at replacing it any time soon. It seems the field of space exploration is becoming antiquated.

John Carmack might be a kick-arse game programmer and a very smart guy, but he is not an expert compiler designer, complexity theorist, or, as is most relevant here, embedded systems programmer for safety-critical systems (though I'm sure he's rapidly learning about it with his rocketry hobby).

They need a cheap replacement for a 7400? No problem! I have an old 7800 they can have for free. I'll throw in some 2600 games that it can play - StarMaster & Missle Command, that should get them back into orbit in no time, right?

Only 58 centimetres square and weighing 50 kilograms, the tiny FedSat satellite is packed with five scientific experiments and all of the instruments required to communicate with Earth during its anticipated three-year life. At the heart of the satellite is a 10MHz ERC-32 processor - a SPARC-based 32-bit RISC processor developed for high-reliability space applications.

The ERC-32 sacrifices processing power for durability and reliability. It uses three chips to process a modest 10 million instructions per second and two million floating-point operations per second - less than 1 per cent of a Pentium 4's capabilities.

The pay-off is reliability: the ERC-32 uses concurrent error-detection to correct more than 95 per cent of errors.

Power-hungry microprocessors such as the Pentium 4, which runs a standard office PC bought off the shelf today, would be an intolerable burden on the solar-powered satellite. The ERC-32 consumes less than 2.25 watts at 5.5 volts.

Designed to survive extreme radiation bursts from solar flares, the ERC-32 can tolerate radiation doses up to 50,000 rad. This is 100 times the lethal dose for humans.

...A team of Australian programmers developed FedSat's onboard software, building on work done in Britain. It is written in Ada-95, a programming language designed for embedded systems and safety-critical software. All it has to work with is 16MB of RAM, 2MB of flash memory for storing the program, a 128K boot prompt and 320MB of DRAM in place of a hard disk that would never survive the launch process. All essential data is stored in three physically different locations.

The software is built in a similar way - lots of internal checks, tell-me-thrice memory, soft-failure-bit-flip-correcting daemons etc. In this case, lives aren't at stake, but the people doing the programming are used to situations where they are.

The software is built in a similar way - lots of internal checks, tell-me-thrice memory, soft-failure-bit-flip-correcting daemons etc. In this case, lives aren't at stake, but the people doing the programming are used to situations where they are.

Not only that, a single space launch of even a fairly small satellite still costs over a billion dollars. If there's a software glitch, it could render the satellite totally inoperable, and I doubt that these engineers want to tell their source of funding that a glitch they're responsible for just wasted the whole launch...

Which is also why Microsoft doesn't do aerospace embedded systems.:) Whoops, Satellite Redmond I just had a BSOD...

We're getting a free ride along with the ADEOS II [nasda.go.jp] megasat (the Japanese get access to some of the data in return), but we're still talking significant money for development. And you're right re funding: it's no exaggeration to say that the future of Australia's space programme is at stake.

As regards Microsoft doing space/embedded systems, another quote from the original article [smh.com.au]:

"The system must be ductile - bending, not breaking - when things go wrong. In space no one can press Control/Alt/ Delete."

Note that they are most likely using GNU software. Here [atmel-wm.com] is a list of the software development environments for these chips, and Here [estec.esa.nl] is the European Space Agency's web page for the tools and emulator.

The context was that of software for an unmanned microsatellite, not the shuttle.

Crewed spacecraft have an even more strict set of rules attached to the software development process. Have a look at some of the articles [af.mil] on DO-178B [lynuxworks.com], the software development standard for avionics. Similar issues apply, but even more so.

Look, people - not Geniuses - just normal, everyday programmers - have been making software you can bet your life on [adaic.com] for a long time now. We know how to do it even more cheaply [af.mil] than the normal buggy commercial work (though testing is radically expensive and blows out the total cost). There's no need, and no excuse, for BSDs and security problems. None. You just have to have the right tools, the right training, and the right attitude. If you like, the Right Stuff [fastcompany.com]. Here's a quote from that article:

It's strictly an 8-to-5 kind of place -- there are late nights, but they're the exception. The programmers are intense, but low-key. Many of them have put in years of work either for IBM ( which owned the shuttle group until 1994 ), or directly on the shuttle software. They're adults, with spouses and kids and lives beyond their remarkable software program.

That's the culture: the on-board shuttle group produces grown-up software, and the way they do it is by being grown-ups. It may not be sexy, it may not be a coding ego-trip -- but it is the future of software. When you're ready to take the next step -- when you have to write perfect software instead of software that's just good enough -- then it's time to grow up.

People like myself look upon any work over about 7 hours a day more than twice a month as signs that "I personally screwed up", because I'm the guy who sets the schedule, not some PHB. We have lives. We have kids. We have hobbies. And the stuff we do is hard, the systems do a lot more than most commercial apps, and with far fewer memory and CPU resources. It's both incredible fun "boldly going.." and all that, but also a crushing responsibility when we do safety-critical work. People's lives depend on us doing the best possible job we can.

One area I disagree with in the "Right Stuff" article is that the work doesn't involve creativity. This is balderdash - we're doing stuff no-one has ever done before under really tight resource constraints. To get a reliable architecture often requires significant smarts, lateral thinking. Anyone can make a complex solution to a complex problem, the really good guys and gals make solutions so drop-dead simple, obviously-correct and efficient that it's miraculous how much such simple, obvious and readable code actually accomplishes.

Looking at the general world of InfoTech, we see that most programmers out there would rather write the winning entry for the "Obfuscated C" contest than make some software that gets us around the solar system. And that people who make reliable software hit the unemployment queue on project completion, while those making buggy stuff have jobs-for-life in maintenance. Of course, they often have 80-hour weeks too, and are driven by PHBs who know b* all, and can't even take pride in the product, so there is some justice.

Get a TI-89 and write an assembly program to control the space shuttle. The TI-89 runs off a MC68000 chip, and has (almost) a meg of space. That's about the programming power of the Apollo computers in a pocket-sized object--plenty of power to calculate the orbital trajectory/angle of entry/etc. It even has built-in calculus functions in case the astronauts forget the Fundamental Theorem of Calculus:) .

Some interesting information in the article... like the main reasons for cancelling the project are a lack of significant improvements in safety, reliability, or cost savings over the shuttle program's remaining lifetime. I'm no fan of keeping obsolete systems hobbling along beyond their years, but this reasoning doesn't seem outrageous to me. The outrageous thing is that it took 400 contractors to develop something that won't outperform a 30-year-old system that runs in 64k.

At the time of the Challenger inquiry, the late physicist Richard Feynman was part of the investigation committee. He found that most of NASA at the time was in full delusional mode about how reliable the Shuttle really was.

The only exception was the computer systems group, in particular the software side. They had metrics, procedures and rigour.At the time of the enquiry the hardware was already old.

It's the attitude that counts, not the hardware, not the methodology of the month. OO is not going to solve NASA's problem, it's going to be difficult. Myself I'd just make sure that the hardware would always be available, and not change a thing.

Trying to write such a system in C/C++ strikes me as rather stupid. It is extremely hard to write reliable software in C/C++. That may not matter much for desktop applications, but it matters when billions of dollars are in the balance.

They obviously don't need very high performance, since it runs on 1970s hardware, but they do need high reliability and low development costs.

That means that they should be using a safe, secure high-level language. Something with a virtual machine might be a good idea so that it will be easy to adapt to new hardware platforms: you verify the virtual machine on the new machine and then have reasonable confidence that your code runs.

If they want something in widespread use, a home-built Java byte-code interpreter (not a JIT--they are too buggy) might be a reasonable choice--it's well specified and there are lots of people who know how to program it. They should probably avoid JNI like the plague and instead add new bytecodes for I/O and communications and verify them the same way that they do the virtual machine itself.. VLISP [nec.com] might be another good choice--or at least a source of ideas for how to implement a verified Java interpreter--DARPA already has paid for its development.

And they should hire someone who doesn't recommed COTS with C++, lest we see the next shuttle go up in flames again.

That means that they should be using a safe, secure high-level language.

Indeed. So where does this rubbish about Java bytecode come from? You're already going to have to verify the processor and the compiler output. Why introduce a third level (a VM) where things can go wrong?

And they should hire someone who doesn't recommed COTS with C++, lest we see the next shuttle go up in flames again.

That was just a gratuitous and offensive swipe. The Challenger shuttle went up due to mechanical problems, not software bugs.

You seem to be one of those people who doesn't like C++, and therefore lumps it together with C and/or has a dig at it whenever possible. It's up to you whether you like a language or not, but please spare the rest of us the ill-informed language wars, OK?

I know the Java JVM is alreasy stack based , but is is far too complex to for the generated code to be verified. Stick with a very simple FORTH based stack with three data stack, long (64) int, Floating point ( 80/128? ). Note, no strings at all, all object/Array access via int syscalls.

"They obviously don't need very high performance, since it runs on 1970s hardware, but they do need high reliability and low development costs. "

Don't confuse "not needing a processor that wastes millions of cycles a second" with "low" performance.

Then need to work with very percse units of time, and have a exceptionally high success rate, fault tolerant and correcting, minimal suseptability to SUEs. The is high performance, as far as industry is concerned.

I've been programming for many years and can now happily churn out C/C++ applications (many of which are multi-threaded) which (post-testing) never suffer unrecoverable errors.

And if you stay in the business for many more years, you'll realize the hubris and folly of that statement.

The myth that C/C++ is difficult

I made no statement to the effect of whether C/C++ is hard or not. I actually think C is a pretty easy language to learn. But it can be hard to write reliable software in otherwise easy languages, and it can be hard to verify compilers for easy languages.

In fact, C++ is one of the most difficult languages to verify a compiler for, and I have personally tracked down a number of bugs in both commercial and open source compilers. The fact that C++ compilers are hard to verify alone would be a reason not to use the language on critical tasks, even all other issues aside.

No: it is easy to write UNreliable software if your methodology is not up to scratch. Given proper techniques and rigour, writing reliable software in C++ is *easy*.

Given that you are so brilliant, go work for Microsoft or Netscape or any of a number of big companies, who evidently lack the "proper technique and rigour" to produce reliable software. If it's so easy, you should be able to fix their numerous bugs and problems in no time.

Given proper techniques and rigour, writing reliable software in C++ is *easy*.

That really depends on your definition of reliability. If you're talking about things like buffer overflows and memory leaks, then yes, good basic programming technique in C++ makes it a far more powerful tool than most give it credit for. OTOH, C++ compilers are complex and buggy, and the imperative rather than declarative nature of the language makes verification of algorithms much harder than it might be. There is still plenty of scope for unreliable C++, on the scales we're talking about here.

In a case like this, you could well afford to go for a more advanced language and make sure you've got a well-trained development team and a verified compiler. The programming world has better tools than C++ available, but pragmatism puts them beyond mainstream use for the time being, which is why C++ remains such a useful tool. In this case, though, you have both the resoures and the motivation to use better.

Just my.05 cents (if that)
I used to work for GSFC (Goddard Space Flight Center). It was wonderful... many years ago.

Anywho... they had *shitloads of unbelievable equipment... ages old... *name that piece of hardware*. We could wander from building to building, and look/view/see the equipment.

Lots were there because they were running projects that took many many years to see results, thus they could not upgrade *in-the-field* because it would stop the project.

Indeed, part of GSFC when I was there was to backup Houston on launches. When they upgraded they built a totally new floor above the existing backup, and on a *grand* day they transfered power, with one big switch, from one floor to the next - why? because they had to. It had to be well tested and well checked before it could be put in live production, yet the existing systems had to be on-line to backup Houston.

It was fantastic walking through the various buildings and rooms... I've seen equipment I've no idea what it did. For example, one room had these rather large, circular platforms with clear plastic or glass domes. Inside the domes where flat plates - think silicon... but BIG.. 1 1/2 ft octogon. Stacked with about 2 inches spacing, about 10 of them. I'd say, looking at the room, some very old old old type of RAM.

"State of the Art" is a good way to run your pocket book into the ground. Jumping on the newest, fanciest programming language doesn't usually make a business successful.

Here's yet another example: My company's (former) largest competitor invested *millions* into Sun hardware and development in Java. Why? "State of the Art". And guess what! With all of their "state of the art" infrastructure, their system was still slow as molasses.

What did we do? We spent less than a tenth of what they did to develop with Perl on x86 servers. Our site handles huge traffic loads pretty effectively, and we did it without running ourselves to the bankruptcy court.

The Space Shuttle's flight computers (not the ground checkout system mentioned in the article) have already been upgraded once, and there's another $405 million upgrade in progress, planned for completion in 2006.

NASA is currently struggling with obtaining a reasonably modern rad-hard CPU. The market is so dinky that nobody wants to bother with it. But they have been able to retrofit flat panel displays, at least.

If NASA's budget is hurting so badly, why not swallow a bit of pride and recruit help from fans of the space program who may also happen to be hardware and software engineers?

Perhaps the crew at, say, ham radio organizations like AMSAT, [amsat.org] or other groups that already combine volunteer engineering effort with an interest in space exploration, would be happy to help out with modernizing the systems. I wonder if anyone's asked them?

NASA would, of course, keep enough engineering staff around to check the improvements out, but why limit themselves to paid labor if the resource to pay is drying up?

Caveat: I am not a rocket scientist, nor an architect of safety-critical systems, just someone who has put in a lot of time over the years on low-level code where reliability and performance (in that order) were essentials.

It strikes me that this is exactly the sort of project where you don't want to attempt to construct an ambitious, all-singing, all-dancing, state of the art, eighth wonder of the world. This misses the point about what is actually needed. Instead, you go for something as simple and straightforward as you can design which will have the capacity to do the job and continue doing the job for the forseeable future. It needs to be simple so that you can analyse its behaviour and failure modes with a high degree of confidence. You can push the sexy bells and whistles out to helper boxes, but the core systems must 'just work'. And technology that's far enough behind the bleeding edge for its characteristics to be well understood is definitely a Good Thing in these situations.

Remember the old engineering rule of thumb: "when in doubt, make it stout, out of things you know about".

Nothing to do with the government. Pretty much every replacement of a so-called legacy system I've ever seen blows out the same way. Anyone who's seen replacement banking systems, SAP rollouts, you name it, will have the same experience.

I was thinking this. Why don't they open some of the current code and some of the requirements they need to "the community".

Think of who space enthusiasts are and what a lot of them do; software and hardware development. In a budget crunch a good strategy would be to allow interested hobbyists to write some of the code, and then have NASA's boys peer review it.

I love the idea and would love to see this. It also makes sense, since NASA, like most national space agencies are for the good of human kind.

On the other hand trying to see it from you average government official's point of view, there would be paranoia since, I believe, NASA shares technology with the military and even without the mililtary ties, there would still be fear mungering over 'national security'. Also, your average programmer probably wouldn't have access to the hardware to run and test the stuff on.

C _is_ the state of the art of procedural languages for over 30 years. It's not like such a simple thing has a lot of room for improvement, there are other areas where new languages can be created, but honestly how many ways are there to do things like preprocessor, functions, variables, etc.?

OO languages' authors may feel that they are doing something more "advanced" but in fact they are working on a completely ortogonal area of development. And most of them are far from C elegance (ex: Stroustrup doesn't even understand C design properly, so C++ is even more inconsistent than what its origin would suggest, and I don't even consider a rotting pile of shit that its "standard" libraty is, to be a part of language), or are simply badly designed (ex: Java), or are not languages but eclectic messes made by including specific librariers' and object models design into the language itself (ex: C#).

This is the area where we can use a lot of progress until it will reach the state where we can keep call the same thing "state of the art" for 30 years, but I won't hold my breath -- OO language design is dead, everyone is just making "OO" languages as various vehicles to promote their narrow-minded ideas. So I won't be surprised if Stroustrup's mess will remain the most useful semi-OO language for the next 30 years, too (but those libraries HAVE TO GO, and so should the attitude that students should learn that atrocity without studying C first).

It's really not myopic planning. It is planning within too many budgetary constraints. After the Apollo program America's focus on space exploration was greatly diminished. There were no more grand goals and manned space exploration was confined to earth orbit. NASA had a huge reduction in force(RIF) in the mid 70's. My former boss described it as a very dismal and depressing time from which the agency has never really recovered.

NASA falls under the classification of "independent agency" within the Federal government. The budget is hooked up with other agencies such as the Vetran's Administration if that tell you anything about how things are considered.

I think it's important to realize that the Shuttle also represents the pinnacle of 1970's computing and that the whole of computing has changed significantly in the last ~25 years. In the 1970's, you didn't worry about things like GUIs (and all the "bloat" that they entail), TCP/IP stacks, extensive amounts of code to deal with the wide variety of hardware configuration, etc.

It's not so much an issue of bloated code as it is an attempt to cover all the bases. The shuttle software was designed with one purpose in mind -- get that shit-heap into orbit. You can't compare it to a modern Linux distro without invoking an apples-to-oranges counter-argument.

Furthermore, the launch of the shuttle isn't handled by a single onboard computer. It's handled by several. Please reference The Space Shuttle Operator's Manual [amazon.com] for more on the systems aboard the shuttle. It's a general, non-technical overview, but a great reference, nonetheless.

You ask "where will it stop?" Here's a hint: it won't. And this same argument probably came up in the 1970's when they started writing the spec for the shuttle. The computer aboard the shuttle is more capable than Apollo for a mission profile that isn't significantly more difficult in any regard (generally speaking). Hell, the PDA you have sitting on your desktop right now has far more computing power than all the computers involved in the Apollo program put together, and it certainly doesn't do anything like putting men on the moon.

I used to work with military electronics and found that the best gear was always from the 80s. The stuff from the 60s and 70s (yes, some of that is still in service) was too primitive. The 90s hardware was too complicated and suffered from unreliable software.

In the 80s the microcontroller technology was just good enough to embed a processor with 64k of ROM full of finely crafted code written by a single programmer and it always just worked, perfectly, every time.

What was "state of the art" in the '80s is now ubiquitous and hidden from the end user.

Ever found a bug in your portable Nomad MP3 player (The flash-based ones, not any of the disk-based ones - Although the disk ones are still pretty strong)

Has your car ever shut down because its computer crashed? (Note: Hardware failure doesn't count, although automotive ECU failure is RARE unless you've done something to screw with its cooling.)

What about your VCR?

These are all cases of coding like you described - Fitting as much as possible into as little space as possible. In Lucent's (now Avaya) business communications division, there was (maybe still is) a raging debate on whether the usability benefits of using two LEDs rather than one justified the *pennies* of extra cost on an item that sold for a few hundred dollars. In a cost-cutting environment that intensive, you're not going to spec a processor with 8k of flash and 2k RAM when a processor with 2k/256 bytes will do. (Note - Popular microcontroller such as the Atmel AVR, Microchip PIC, Motorola 68HC11, etc. all are in this range.)

And it's quite easy for a single programmer to do all of this. I've seen CD-based MP3 players developed in a few weeks by a team of two college students for their Microcontrollers course taking 3-4 other classes at the same time.

What I see very well is products that don't satisfy their intended audience because their embedded software sucks. This is happening more often than 12 years ago.

Why is this happening? My guess is that since the hardware is much more powerful engineers are tempted to try to do much more in each product. The problem is that quality software development management hasn't caught up with the increase in hardware capacity.

In other words, engineers have lost the ability to think simple.

What was "state of the art" in the '80s is now ubiquitous and hidden from the end user.

It's apparently so well hidden that it does not affect overall customer satisfaction.

Has your car ever shut down because its computer crashed?

No, but it has been doing really wierd things with the idle RPMs on cold start and in five visits to the garage the mechanic has never been able to pinpoint the problem. Resetting the computer helps for while, though.

These are all cases of coding like you described - Fitting as much as possible into as little space as possible.

Sure there are, but there are also many cases where an embedded device gets a 32 bit RISC cpu with a few megabytes of flash ROM, a few hundred ks of RAM, a realtime operating system, multiple threads, multiple programmers, bloatware, bugs, endlessly stretching schedules, etc when a 8 bit micro with 32k of of ROM would have been enough..

No that article was based on a stable situation (i.e. same technology, stable requirements and same methodology for each project). Because of stringent control and heaps of experience, they had perfected the process. With this project they picked new technologies and new methodology, hired a bunch of contracters and ran into the inevitable problem of shifting requirements, more new technology and methodological issues (off the shelf methods always need to be fine tuned and tailored).

The worrying thing is that with their expertise, they could have known that this would happen. The appropriate thing to do would have been pilot projects followed by more ambitious projects. Now they've bet everything on one horse and are left empty handed. What they had before they started was a piece of shit (presumably that was why they wanted to get rid of it), and now they have to face maintaining that piece of shit again.

It may have been an elegant system when it was first designed. However by now it has probably seen countless adaptations, comes with thousands of pages of documented changes (they are control freaks) and probably is very hard to understand and maintain. Likely many of its original designers are deceased or retired by now.

The decision to cancel the project appears to be a panic reaction by management. What they will soon find out is that their old stuff no longer can be modified cost effectively to new requirements. Replacing it will easily take half a decade if you start from scratch and they have just thrown away the efforts of half a decade of development.