Posted
by
timothyon Wednesday January 13, 2010 @04:04PM
from the how-rms-uses-outlook dept.

Pigskin-Referee writes with this excerpt from Office Watch: "On the Space Shuttle and the International Space Station they use Microsoft Outlook 2003, but not quite in the same way that us earthbound Earthlings do. The space shuttle Atlantis is orbiting the earth right now and the crew exchange emails with the ground a few times each day. Bandwidth is a constraint and you don't want the busy crewmembers bothered with spam or unnecessary messages so NASA has a special system in place. The crew use fairly standard laptops running Microsoft Outlook (currently Outlook 2003) with Exchange Server as the email host, but they don't link to the server using any of the standard methods."

When NASA first started sending up astronauts, they quickly discovered that Mozilla thunbird would not work in zero gravity.To combat the problem, NASA scientists spent a drunken weekend and $12 billion on Microsoft Outlook and Exchange licensing to develop a mail server that works in zero gravity, upside down, covered in stale beer, and old pizza boxes, and at temperatures ranging from below 10 to 25 degrees Celsius.

They're doing it to save bandwidth. Yet they probably spend more on bandwidth dealing with human error issues in the process than they would if the system was engineered properly in the first place.

You don't see an issue because you aren't an engineer trying to save every drop of energy/bandwidth/processing time possible.

Basically, you're a java or C# developer when then need C and assembly developers with a clue.

Custom hacks when there are already systems (even build into EXCHANGE!) to do EXACTLY what they need to do are beyond stupid. Its one thing to use a custom hack so you don't get tied into a vendor, but their hack is entirely tied to their vendors so that rules that reason out.

Next you do it because you have a requirement that no existing solution fills in properly, which is certainly not the case here. As I already said, even Exchange will be happy to do store and forward batching on a schedule. A tiny exchange server (or a more efficient/less resource intensive alternative) on the space station could be designed to consume pretty much no energy unless it was actually in use.

In short, this is clearly something thrown together by engineers who knew nothing about the tools they were working with. Not their fault (probably), some douche bag manager probably didn't ask the IT guys.

The problem is, they went through effort and resources to make a system that is clearly less efficient than any of the possibly alternatives I can come up with.

I'm just surprised that they have to do this at all. They're in LEO, not on Mars. Shouldn't they be able to manage communications at least as well as even the simplest communications satellite (which can do a LOT more bandwidth that little 4MB bursts)?

It's not a matter of signals being impossible. Rather, adding additional communications infrastructure to an existing space installation requires power, mass, and a rocket to take it up there. If they can use their existing (poor) connection speed, they can avoid all of that.

Why not just run a normal mailserver with a simple script to deliver any messages in the files uploaded? No need for the astronauts to mess with weird outlook files, just hit "check mail" on whatever client they prefer.

seriously, their method is on crack. SMTP supports queue of mail, use the god-damn feature and us a compressed link for the exchange.

put quotes on the uplink as necessary to prevent flooding (size, or number of messages) if it's an issue, but otherwise, where;s the problem to solve? SMTP worked when people used 1200bps modems for internet links.

I suspect that NASA doesn't run servers on the ISS though. Their computer model seems to be ancient, proprietary, space-hardened embedded stuff for mission critical needs, and a pile of disposable laptops for crew needs. That's probably crippling their network infrastructure in many ways.

IMs - Email for those who don't realize nothing they do is so important that someone else has to know/answer RIGHT NOW!@%!@%!@%!@%@#^@#^!I'm sure you could find being more wrong very hard.

IM is, on the one hand, quite instant in delivery, like voice. But, and here's where you are phenomenally wrong, it doesn't need immediate attention. It will be there when you feel like answering. Or not answering - because it's for things that aren't so important or so complex as the ones discussed in e-mails. Just one qu

Is it actually cheaper to upload all the e-mails in a burst instead of using a webmail system where only the mail the receiver wants to receive would be opened? Wouldn't work if they want to read offline I guess but the concern mentioned is bandwidth not connectivity.

Is it actually cheaper to upload all the e-mails in a burst instead of using a webmail system where only the mail the receiver wants to receive would be opened? Wouldn't work if they want to read offline I guess but the concern mentioned is bandwidth not connectivity.

Any mail experts comment?

If you read the article you notice that mails sent there are pre-filtered so everything is critical to read and contains no spam.

So yes, uploading all mails neatly packaged together ONCE takes way less bandwidth than a webmail interface, even if it's a very lightweight one (think about it, Hotmail or Gmail probably transfers the same amount of data than 20 or more email messages just to display their fancy interface).

In fact, you should be surprised that Windows is _STILL_ running after a Virus has hit the ISS orbiting the planet.

No kidding, Google it.

It is particularly sad that NASA IT guys aren't obviously that pathetic to license Outlook from MS. Something really going on there, a lot of open source software/operating systems has NASA contributed excellent code in them.

PS: I remember they also had Norton Utilities with "rescue diskettes" back in 1990s, it leaked while I was trying to find a way to manually uninstall

It's too bad the article didn't address the architecture behind all this. I would be curious to hear what kind of network they use, and what sort of relays (satellite?). If it is satellites, why is the bandwidth so low? (Hmmm... maybe they really should have made that ethernet cable just a little longer after all...)

I am a Martian prince from the Splugorthian region of the Xylerom. I have inherited a bountiful estate worth 1.8345E8 drow'xlian that I must hide from the ruthless Prxyzzilic crime family. I am willing to share 20% of my fortune with you will allow me to deposit fund in your account. Please send me your account information if you wish to do business.
Live long and prosper!
Prince Ryzzriwz

I am a Martian prince from the Splugorthian region of the Xylerom. I have inherited a bountiful estate worth 1.8345E8 drow'xlian that I must hide from the ruthless Prxyzzilic crime family. I am willing to share 20% of my fortune with you will allow me to deposit fund in your account. Please send me your account information if you wish to do business. Live long and prosper! Prince Ryzzriwz

UUCP worked quite nicely in the days when links were ephemeral, slow, or generally unreliable. This seems like a lot of effort to solve a problem that existed 30 years ago, solved, and even adapted for RFC821 and its successors. There's a reason that Sendmail knows how to rewrite addresses!

No, the bandwidth needed is really quite high, because the signaling system is really inefficient. You could send a lot more data over the same radio link if it was encoded more efficiently.

The problem is they don't want to give up their old reliable voice comm system. That's probably a reasonable choice. But that choice eats a non-trivial portion of the bandwidth available for other communications systems.

So, once a day they bundle a bunch of emails into a single.OST file and upload it to the shuttle. The astronauts then open that.OST file in their local copy of Outlook. And they have to shut down Outlook while the upload is in progress because of Outlook file locking.

In addition, communication with the ground isn't always possible (you'll hear warnings of LOS - Loss of Signal during mission communications) so standard methods of email transfer like POP/SMTP, IMAP etc might not be reliable.

If a 'Loss of Signal' can interrupt a POP session, wouldn't it also interrupt a file upload? Couldn't they just POP into the server on Earth once a day to grab their emails to be stored in a simple mbox or some such? Wouldn't this also eliminate the file locking issue as mboxes and Maildirs are pretty old and stable solutions that don't have this problem? This just sounds like someone wanted to use Microsoft Outlook no matter what and hacked together a procedure to use it even though there are way better approaches. And isn't the whole point of Outlook that it has a built in calendar and meeting request system and network folders? They're not even using those more advanced parts of it, they just need email.

I gather that the idea is that the OST stores the messages more efficiently than if you were trying to transfer individual messages. That's an important issue if you're operating with limited bandwidth. Also, the article implies that the messages are basically being hand-sorted into the OST file to prevent any waste in bandwidth.

Beyond that, there's the issue of what protocols are best for this purpose. The article doesn't go into detail about how the OST files are being uploaded, but the protocol may

If they have minimal bandwidth, then pop probably isn't ideal because of the back and forth communication. Also, I would suspect that to minimize transfer time, their file transfer mechanism uses compression (email is HIGHLY compressible). As far as I know, there's no way to do pop compression (though if the compression were implemented at the connection/tunneling level, then I suppose that would be transparent)

As for your other question, there are resumable file upload/download methods.

'The crew use fairly standard laptops running Microsoft Outlook (currently Outlook 2003) with Exchange Server as the email host...Because there's limited bandwidth up to the shuttle it's important to keep the OST fairly small so occasionally you'll hear NASA controllers ask the crew to clean out their Outlook files'

I say we take off and re-install the entire OS from orbit. It's the only way to be sure.

As there is a new President in the Office and he doesn't really like (it seems) fantasy and unrealistic plans, he should also order his IT guys to start an investigation why standard, documented protocols like IMAP, XMPP aren't used. A visit from a Internet2 academic could be enough...

In fact, it is an International issue. ISS doesn't "belong" to USA, there are several billions of dollars of other countries out there.

While on it, they should also ask NASA about why on Earth "NASA TV is best viewed fullscreen with Windows Media Player", why there isn't a standard MP4 based live broadcast, why it defaults to Windows Media regardless of your setup...

Something really happening over there, trust me on that... These are the guys who had a genius idea of using Kermit as a protocol for communication before these "Outlook", "Windows Media Player" guys took over the job.

If there are people thinking "Oh but MS is an American company", let me remind, Red Hat, Sun Micro, IBM and lots of standards bodies are American too... That is in case the multi hundred billion dollar project should be a billboard for pathetic software setups.

You don't need anything fancy. I was using UUCP to do bulk and batched transfers of email, Usenet feeds and even files back in the early 1990s. It's become obsolete in a lot of cases, because everyone went from low-bandwidth limited connection modems to always-on broadband connections, but back in the day, I got all my email, newsfeed and even the odd file a few times a day via a scheduled UUCP transfers (which also sent any emails and posts I might have). Ah, the good ol' days of bang paths! Still, UUCP has its purposes, and it strikes me that it is a well-established protocol designed just for this sort of environment.

It just goes to show you how much damage has been done to tech by Microsoft, and this pervasive psychological need to use its shitty software, its shitty file formats and its shitty protocols, even with an organization populated by people who should be intimately familiar with Unix and its own much more rigorous and time-tested protocols. I mean, this is nothing more than FTPing mbox files back and forth, which, twenty years ago, would have had anybody with a moderate knowledge of mail systems and communications protocols rolling on the floor laughing.

If only that headline used "Nuking" instead of "Using" Outlook from Orbit.

My company recently switched from a really screwball lotus notes install to msexchange and thereby screwed every unix and mac user -- which is to say, 95% of the technical staff. Some of that I can't blame MSFT for, we do have some real chimpanzees on our email team, but the experience does have me shaking my fist in Redmond's direction even more than usual of late.

I noticed you pushed a button on your console. Are you trying to steer your spacecraft? Please wait whilst Clippy ShuttleBuddy extensions for.NET 3.0 SP6 is installed, then after a reboot we'll get right on with that.

Push queued mail on demand to the orbiting mail server. Cron up the EXTN trigger or setup sendmail (which its happy to do) to handle the queuing whever you want.

Guess what, it works with exchange too!

I guess NASA spends its money on aeronautical engineers and not computer system admins. I'd be willing to bet that I could do it cheaper and more reliably even with exchange than there method, in their constraints of bandwidth and available connection time.

These OST files are tiny by ground-based standards – around at most 4MB for shuttle crew.

Amazing. A just over a half-dozen people and yet they manage to keep their email communications down to just 2,000 pages of text a day! How do they manage.

The OST file, now with outgoing emails, is copied back to NASA on the ground where the messages are sent, copied to the Sent Items folder and any new email is placed in the OST ready for the next upload.

Well, that makes sense. They reply with the same type of file that they receive with. If it's good for bandwidth one way, it's good for bandwidth the other I'd guess.

Because there’s limited bandwidth up to the shuttle it’s important to keep the OST fairly small so occasionally you’ll hear NASA controllers ask the crew to clean out their Outlook files (the OST).

Whajah? They're sending the *entire* mailbox both ways and just bouncing the same messages back and forth every time? How does that save bandwidth? How do these guys send pictures to each other, zip up an image of the entire hard drive?

I guess that explains why they need to transfer 2,000+ pages of text every day.

This sounds cumbersome and messy

True. Because it is cumbersome and messy.

it’s certainly not the way you’d do it here on the Green Hills of Earth.

It's also not the way I'd do it in space either, because of the bandwidth constraints.

However it makes sense

No it doesn't. Not under any circumstance does "send the whole thing back and forth every time" make sense if the thing you're trying to conserve is bandwidth.

You might also hear ‘CapCom’ asking the crew to shut down their copies of Outlook so that an OST transfer can occur. Outlook puts a file lock on any PST/OST file which prevents any copying (a problem anyone trying to do an Outlook backup might be familiar with).

Ahh, so that's it. They're not trying to conserve bandwidth. They're trying to conserve "thinking about it." Otherwise, they'd only have to shut down outlook when renaming "file.ost.xfer" to "c:\...\outlookdir\file.ost"

In addition, communication with the ground isn’t always possible (you’ll hear warnings of LOS – Loss of Signal during mission communications) so standard methods of email transfer like POP/SMTP, IMAP etc might not be reliable.

True. Why does it need to be email, though. Why can't they just send a psk-31 HF radiogram? or the even more fault tolerant HF packet radio? You only need a transmit station somewhere in the same hemisphere for that to work.

Hell, with a directional antenna (and a doppler-compensating transmitter), there's no reason why they couldn't use 3G cell service when over a country which has it. 300 miles up gets you a window of up to 11 minutes which would let you download quite a bit.

But I don't think bandwidth is really the issue. There's enough bandwidth to transmit live video for pete's sake, but email is somehow a problem? The issue is that "outlook is email." It clearly has simply never occurred to anyone in the chain that there might even be any other way to handle email-type communications.

"Why can't they just send a psk-31 HF radiogram? or the even more fault tolerant HF packet radio?"The Ionosphere maybe. It will tend to block HF and one of the reasons HF is so good for long range on earth is that it can bounch and skip off the Ionosphere.At best you would still just get line of site and the antenna would have to be very large.You would be better off using VHF,UHF, or even Microwave since you could get more bandwidth.

I use a 2400 modem to call google voice and record the e-mail. Then my computer uses dragon to transcribe them back to digital, then print. I use fed-ex (Hey, not too many donkeys around anymore ya know?) to send them to my mail box. I retreive them from the mailbox by walking up hill in the snow both ways! (but replies go back to the mail box on a belt system).

BEAT THAT!

**note above challenge was tongue in cheek, nothing was real, and if you so much as are reading this post today, you've already won.**

You may jest, but you can do a lot more, more easily, from a command line than a GUI on any platform. Even Windows. Try to

ren antique???.jpg desk???.jpg

in File Manager

In Windows: CTRL+A, F2, "desk", ENTER.

Admittedly that gives you names like "desk (01).jpg" and not "desk01.jpg" but it's close enough. If you want a significantly higher level of control, try something like Flexible Renamer [vector.co.jp] (somewhat prone to crashing, but the most versatile and powerful I've found).

It would be insightful if it were talking about something else.
Using the command line to read email is hardly a 'good' way to go about it.
It works and is usable for some, but even most shell users use 'a gui' like Pine or the like.

OK, so mailx(1) isn't the perfect mail client (didn't support MIME, last time I checked).
But Nasa using Outlook, a sucky client tied with a proprietary interface to MS Exchange -- what's up with that?

1. You received the mail with my family pictures. I need the one that has a picture of my house in it. Find it and send it back to me.

2. You receive a ton of new albums in mp3 format, but since your mp3 player is short on space, you need to weed out the "interludes", which are each less than 1 megabyte. Save the songs, but delete the interludes.

3. Take a link from the email, go to that website, and post the albums and pictures to the "cloud" for backup.

They have to be very careful in a close environment such as the shuttle or the space station to keep the air healthy. Using mailx like you do would give off too much smug for their filters and cleaners to handle.

Whys that? Because the terminal window on your monitor takes less energy to display? Nope, thats not true without using an LED display (not LED backlight, the entire thing has to be LED or it doesn't make a difference).

You think that just displaying a GUI consumes energy? Please provide a citation. Any GUI on a modern OS doesn't require any processing power for displaying something on the screen that is static. You update the screen to current and it sits there.

Direct TV's satellites are geosynchronous the space station is not- so I would think it is feasible in part of the orbit where there is line of sight-
Otherwise, just upgrade the worldwide systems that already talk to the station for a thicker connection.

Honestly, it sounds like a pretty fun grad level dynamics problem. The orbital mechanics to track an object in orbit from the ground are calculated all the time, tracking an object in geostationary orbit from an orbiting object, i'm not sure if that has been done before- probably but not sure. Just because it is moving "fast" doesn't mean is isn't doable.

1) Due to the fact that the ISS orbits Earth very quickly, with an orbital period of 91 minutes, you'd barely get your receiver up and running and locked onto the signal before it disappeared again. DirecTV doesn't work on the opposite side of the planet to the USA, and the planet definitely makes a better door than a window in this case.2) TV birds are generally spot beamed. Yes the ISS orbits much closer to Earth than satellites in geostationary orbit, but regardless, if you're outside