2010-01-25

It's January 24th. Opportunity has been on Mars one year today. Unlike Spirit's birthday, there's little fanfare, just people getting on with the job. And occasionally reminding each other, with the usual subdued wonder: we've been at this a year.[1]

So it's the beginning of Opportunity's second year on Mars, and the end of something else: namely, it's the end of our exploration of the heat shield. ("And there was much rejoicing ....")

I know, I said that last time. But this time, I really mean it. We planned the post-heat-shield drive, but there's one more thing they want us to do before we go. The MI images we got over the weekend looked great (I'm a genius!), but shooting them through the dust cover, as Erickson mandated to protect the instrument in case I screwed up, reduced the contrast. So the EDL team wants us to retake a few of them with the dust cover open. (To my surprise, Erickson agrees to this. I think that's to his surprise as well: "I said 'no' originally" he tells me, "but Squyres argued me around." Which is yet another surprise: the heat shield being primarily of engineering interest, the project has been generally unwilling to subject the rover to any risk over it -- for example, we can't use any of the instruments except the MI.) After that, we'll stow the IDD and drive away, as planned last week.

We're actually damn lucky we got anything at all. Last week, while building the sequences, there was a mix-up about which MI image sequences we were supposed to call. The first version of our IDD sequence used the wrong MI sequence IDs, but we redelivered a newer IDD sequence with the right MI sequence IDs in it. So far, nothing particularly unusual. But somehow, they managed to send the wrong version of the IDD sequence to the spacecraft! (We have software to warn about this, but they somehow managed to overlook the warning. Yeesh.) Happily, the "wrong" MI sequences were already on board, having been left over from some long-ago sol, so the spacecraft was happy -- it took images with a somewhat suboptimal exposure, but at least it took them.

But that was just dumb luck. It could easily have been otherwise: we could have moved the IDD to all the right places, and taken no pictures. Or, if the difference between the two IDD sequences had been something else, arbitrarily bad things could have happened. Lucky, lucky, lucky ....

At least thisol's sequencing is relatively easy. The drive's already in the bag, and the IDD stuff is just a subset of what we did over the weekend, so it's mostly a matter of copying and pasting. It doesn't take me very long to get it done. Which is great, because it leaves time for the Iowa Space Girl.

The Iowa Space Girl is Susan Kurtik's seven-year-old grandniece -- "she's bonkers about space," Susan says.[2] The Iowa Space Girl is also known as "Abigail," and I spend about half an hour showing her and her family around the MER area, letting her look through the 3-D glasses at Mars, letting her play with RSVP, and so on. Kids love RSVP, because it's like a video game -- which was the idea. "Scott got to grow up and get paid to write video games, and then he gets paid to play them," Susan says.[3] Abigail seems to like this idea.

Abigail seems like a great kid -- polite, smart, genuinely interested in space. But Steve Squyres is a big kid, too. On his way past he picks up a pair of the 3-D goggles and peers through them at the heat shield. "Gosh, what a weird-looking thing!" he exclaims. "A crashed spaceship on another planet!"

"Shame it's ours," I say.

He laughs. "Now that's a circumstance under which I'd be willing to use the APXS on it. APXS, MB, everything!"

[Next post: sol 387 (Opportunity sol 366), February 3.]

Courtesy NASA/JPL-Caltech. One of the MI images of Opportunity's heat shield. So, this robot goes to another planet, studies the planet a while, and then wanders over to squint at parts of its own wrecked spaceship. OK, so I don't have a flying car; I'm still living in the future.

Footnotes:

[1] I'd forgotten about this. For a while there, Opportunity was hogging the birthday glory; the project would wait until Opportunity's landing day to make a big deal out of the anniversary. But the first one was reversed: the big press conferences and such happened around Spirit's birthday. It wasn't until year 5 (2009) that they simply split the difference.

2010-01-23

It's Leo's last sol on MER; after today, he moves on to MSL. He picked a heck of a sol for his last: we're in on a Saturday, for one thing; for another, we're planning three sols today.

So we've got a lot of work to do, but we can't get started on it. The auto-generated terrain meshes are useless; they used the horizon mask, which clips off any features above a certain height. Normally, this is a sensible thing to do, which is why it's the default for the automated process -- the stereo correlation will occasionally be confused by some distant feature and try to place it in the mesh near the rover and a meter or so off the ground, and usually we just clip that junk out.

But in this case, the heat shield shoulder piece projects above the limit. Naturally, the horizon mask clips it off just below the area we're interested in. I came in last night to pick a few spots in the imagery that are possible candidates for our MI survey, but we can't do the real work until the meshes come along.

So right off the bat, we're delayed by an hour. If I believed in omens, I'd call this a bad one.

Not only is it Leo's last sol, it's also Ray Arvidson's birthday. And Jim Erickson's.

At least we've got some free time to celebrate ....

But when the meshes show up, I'm able to get to work. We had an aggressive plan for MIing a long strip, but the three-dimensional view shows that this will be impossible: we want to avoid touching the material, and we're constrained by a fluffy piece on the left and a jagged finger on the right. The entire reachable range is maybe 5cm, much less than we'd planned for. What's more, the surface we're trying to MI is barely visible from our current position: the HAZCAMs almost can't see it, since it lies mostly along their line of sight, and in the NAVCAM we took to compensate for that (since we knew it might happen), the strip they want to image is obscured by the upthrust fluffy piece.

One way to respond to this situation is to take more images, closer together. This helps compensate for the uncertainty we're facing, and the total number of images (and corresponding downlink volume) will still be less than we'd planned for when we thought we could get a large range.

To add to the pressure, everyone really wants these MIs to turn out well. Once we get them, we're done at the heat shield and can move on to the etched terrain. We've been at the heat shield about a month, and except for our SpongeBob observations, it's been a purely engineering-oriented endeavor -- no real science at all. So the scientists are champing at the bit to move on, and the management team wants to cut loose as well.

Luckily for us, we have an ace up our sleeve. An ace named Eric Baumgartner -- Andy asked him to come in today (on a Saturday!) and help out as needed. Eric suggests a different direction for the stacks, one that might track the surface of the strip better, but ends up mostly doing analysis. Which is just as good as far as I'm concerned: if Eric says the sequence is okay, you can bet your house on it. And he says it's okay.

When I'm done, I spend some time looking over the sequence. It's about a hundred commands or so, only moderately complex by IDD-sequence standards. I find myself thinking that for all the trouble I had building the damn thing, it ought to be longer.

Meanwhile, Cooper's been working on the drive. We're going to wait until we see the MI results, and if they look good, we'll uplink the drive sequence and be gone as of sol 358. Otherwise, Cooper's just wasting his time -- yet another reason I really want to get my part right.[1]

Despite a mishap that requires Cooper to rewrite the entire drive sequence, and despite all the time I take, we're not the ones holding up the process thisol. For some reason, it's just one of those days on Mars -- both Spirit and Opportunity have a series of random ground software problems that delay the process, seemingly endlessly, and they're still holding things up when I have to leave for an radio interview.

Lucky me; since Cooper's staying, I can leave. Leo's not so lucky -- he's got nobody to backstop him, and he'd promised his wife he'd leave by 6:30 so she could take him out to dinner.

[1] This overstates the case a little: Cooper's drive sequence could have simply been used on a later sol. It still would have meant he was spending his Saturday doing work that could have been done on a normal weekday, but it wouldn't have been altogether wasted.

2010-01-21

Leo cocks an eye at Khaled and grins wickedly. "Oh, good -- I'll be sure to ask lots of questions ...."

While I was gone, they IDDd the heck out of SpongeBob, finding pretty much what the scientists expected: it's a meteorite. Amazing stuff, and I'm sorry I missed it.

But we're back to the heat shield now. Yestersol's drive brought Opportunity back to an approach position, and thisol we're just bumping 70cm or so to get the piece of interest into the IDD work volume. Khaled's going to do the actual drive, but we have a lot of work to do to decide precisely where it should take us.

To do this, we have to study previous images of this wacky object -- all taken from different perspectives, with different cameras, under different lighting conditions -- and figure out where the strip of TPS material is now, with respect to the rover. It's while doing this that I get really, really annoyed about something.

One of the EDL team members who's been working on this with us, shows me an image from our previous position. He runs his finger along part of it, a part that looks like a bumpy railroad track -- "This is the part we want to MI, right here."

The part he's pointing to is exactly the part I worked out a way to get to the last time we were here, right before he claimed he'd need images at an up-angle -- the angle we couldn't get to from that position, which led to our temporarily leaving the heat shield.

But the part he's running his fingers along is visible from a NAVCAM image, an image where we were looking down at the part.

Which means the part is facing up.

Which means shooting at an up-angle is useless -- we have to shoot at a cross-angle, or down.

Which is what I had before.

MotherFUCKer.

Making matters even worse, our reapproach angle is constrained -- we won't be able to get back to the spot we were in before. Our new position will likely be worse.

I need John Wright to come by and say, "This is so cool." But he's off thisol.

Ah, fuck it. We'll do our best. And tomorrow, we'll see how good that was.

2010-01-12

When we start planning, we don't have the downlink. "So we don't know whether Khaled broke the rover?" I ask Frank. He looks at me, fakes a worried face, and shakes his head no.

Of course, when the data arrives, it transpires that Khaled has done a perfect job; the rover's right where we wanted it.

We're ready to reapproach. But in the middle of the discussion, the science team decides to completely change the plan. Again.

The issue is the long MLK holiday weekend: if we start the heat shield work now, we'll have one sol of IDD work and mostly waste the vehicle's time over the weekend. But if we drive to SpongeBob now, we'll have a ton of IDD work to do there throughout the weekend, using the vehicle much better. So we're going to just drive to SpongeBob thisol, and return to the heat shield on a future sol.

Despite Khaled's success on his first drive, Frank and I decide to sequence this one. Frank slaps together a candidate drive, then goes off to hack RSVP while I modify and refine it. And a nice job I do of it, too: we reapproach part of the way to the heat shield, taking PANCAM images of the part we'll eventually want to come back and IDD, then we're off to SpongeBob in a long, lazy "Z"-drive that avoids nearby debris scraps.

The scientists' interest in SpongeBob is easy to understand: remote sensing suggests it's an iron meteorite! Like our own spacecraft, it likely fell from outer space to the surface of Mars. The weird part is that it's so damn close to our heat shield, just meters away. Either these meteorites are all over Mars, or this is one damn freaky coincidence.

Or, as I jokingly suggest, we hit it during the cruise stage and knocked it into Mars ourself. If that had really happened, of course, there'd be a more recent, hence more prominent, crater around it. The idea might be highly implausible, but it's fun.

I enjoy working on thisol's drive, despite having the usual RP-1 experience of continual interruptions and plan changes. "You sure you want this job?" I tease Khaled.

He answers wryly, "You haven't done data management."

[Next post: sol 374 (Opportunity sol 355), January 21.]

Courtesy NASA/JPL-Caltech. Our next destination, SpongeBob, is in the upper left of this image. The science team called "SpongeBob" by another name, "Heat Shield Rock," apparently because they couldn't think of anything more boring. But it'll always be "SpongeBob" (or sometimes "SpongeRock") to me.

2010-01-11

The plan for thisol is a touch-and-go: we'll MI the shoulder piece of the heat shield, then drive to SpongeBob. Reaching the exact spot the TPS team wants to see is going to be tricky -- indeed, it initially looks impossible -- but I keep hacking away at it, and together Frank and I figure out a way. Then we show it to the main guy, Ben.

Ben isn't satisfied. He wants the pictures from a different angle.

Okay, then. The angle he wants can't be reached from our current position; the arm interferes with itself when trying to get into the right spot. So after all that work, we blow off the IDD stuff for the day, and change the drive as well. Instead, we'll have to do a two-sol "reapproach" -- stow the IDD, then back off from the heat shield and, nextersol, come back in to a spot where we can scratch the TPS team's itch. ("Well, today went from really hard to really easy," Frank mutters.)

While I was immersed in the IDD stuff we ended up scrapping, Frank suggested we let Khaled[1] handle the drive, which sounded fine to me. I start to regret acquiescing when we decide that the IDD "work" thisol will amount to little more than a stow. But as it turns out, the IDD work is more interesting than expected. We just need to retract the MB from the magnets and stow, but with the heat shield in the way, our maneuvering room is tightly constrained; the usual procedure would whack the arm into the heat shield. So my strategy is, roughly, to reverse the moves that they used to get the IDD up there in the first place, so that we're going through known-working positions on the way out. In doing this, I discover that we had less than a millimeter of clearance at one point! Yikes.

With that stuff done, we turn to discussing nextersol's drive, the reapproach part. One of our decisions is to take images of the underside of the broken-off piece during the reapproach. When we were right next to the heat shield, we were at the right distance but the wrong angle to see the area we were interested in. So as we drive in, we'll take images from one or two standoff positions, trading resolution for viewing angle. If it turns out the underside's shape will keep us from being able to place the IDD at all, we'll know we needn't bother to deploy. Then it occurs to us that we could have taken those pictures on this drive, as we backed away; that would have potentially saved us a sol. By the time we think of this, it's too late to fix it, so we just shrug collectively. We'll get the pictures tomorrow, and that's that.

The exciting discovery of the day is that Khaled is on a terrorist watch list. Well, not our Khaled Ali, but there is a Khaled Ali on the list, so our guy gets special treatment every time he flies. He found out about this when airline security personnel started asking him if he has a tattoo. He does -- on his back; the terrorist suspect has one on his left arm.

"Next time they ask you if you have a tattoo, you should say, 'Not on my left arm,'" I suggest. "Blow their minds."

2010-01-06

"Thank God the spacecraft is smarter than we are," Art says wryly. Our manual data product deletes failed yesterday, so Opportunity deleted some for us to keep flash from getting too full. It was so busy doing this that the sun-find timed out, which kept Opportunity from updating its attitude knowledge. So Opportunity decided it didn't know its current attitude, which causes it to mark itself unsafe for driving. Luckily, this problem was discovered last night, and Opportunity's already been manually reset to a better state of attitude knowledge.

But the autodeletes are a problem. You'll remember a few days back, when Jim Bell wanted to take some sky-flat images to help calibrate the PANCAM data, and he and Justin Maki got into a rather acrimonious argument about it? Well, the sky flats weren't taken that sol, but they were taken on a later sol. But they hadn't been downloaded yet. And guess what the autodelete logic just deleted?

So they're going to have to take those again.

Justin has something else to worry about today -- namely, me. There's something that's been annoying me through much of the extended mission. During the nominal mission, we built the post-drive NAVCAM sequences from scratch each sol; as a result, we could assign a sequence ID up front and worry about the pointing later, when we had the drive done and knew what we needed. But nowadays we're trying to use a "sequence library," which means we try to pick an already-built sequence. We still have to choose a sequence ID up front, but since sequences in the library are identified by sequence ID, this implies that we're choosing the post-drive imaging before the drive is built.

Obviously, this is screwed up. That decision was driven by another screwed-up decision made somewhere else, and, well, long story short, it's not going to get fixed today. Most of the time it doesn't matter; we image such a wide area that we have plenty of room for error. (Which is another gripe: why are we so wasteful? Maybe if we weren't so routinely wasteful, we wouldn't have needed autodelete, which killed the sky flats and ... ah, never mind.) But on this sol, we narrowed the post-drive imaging to save on downlink, and we're going to have to play with the drive to ensure we get onto the right approach vector for our final approach to the heat shield.

This combination of things -- an incompletely specified drive and a narrow field of view for the post-drive imaging -- is exactly the case where the sequence library approach is at its worst. Justin originally chooses a post-drive NAVCAM centered at 90 degrees, but after talking with the TPS folks, it becomes clear we should shoot for a very different orientation to the heat shield. (Which is bad news for Jeff, too, in a way. He couldn't sleep, so he came in early and built the drive. Now I have to rebuild it. That's fun for me, but he must feel he wasted a bit of his time.) So he has to redeliver with a new mosaic centered around 72 degrees.

Then I start looking at the situation more carefully in RSVP. Turns out that the new azimuth is fine, but I'm not as sanguine about the elevation. RSVP shows the NAVCAM pointing as being marginal -- we should get the whole heat shield, but about a third of the image is above the heat shield, and it's doubtful we'll see its bottom edge. Since the bottom edge is nearer to what we want to IDD, and more importantly what we'd smack the rover into in the event of a misjudgment, I'd feel safer about lowering the pointing a notch.

Justin argues against this change, and his is not a judgment to dismiss lightly. It's not just that he doesn't want to re-redeliver; he's sure we're going to image the heat shield whether we lower the elevation or not, and if we keep it the way it is, we're more likely to get distant features (namely, our own tracks) in the distance, which makes the images look cooler when MIPL converts them to overhead mosaics.

Justin knows this stuff like the back of his hand; it's hard to escape the feeling that arguing with him about it makes me a jerk. But in the end I decide to ask him to change it. Getting the tracks in the NAVCAMs would be cool, but getting the entire heat shield is critical. If we're aimed lower than necessary, then we'll just image more of the terrain leading up to the heat shield, which is perfectly fine with me; whatever we can't see well in the front HAZCAM view of this area, we'll pick up in the NAVCAMs.

So Justin redelivers again.

Note to self: if Justin was right and I was wrong, apologize profusely.

Second note to self: prepare to apologize profusely.

Worse yet, most of this work might be moot -- or, at least, delayed. Despite the space cleared out by autodelete, we won't have enough room in flash to do the entire planned sol unless we get confirmation of some additional manual deletes that were uplinked at the last second last night. The plan for the sol is, first, to drive up next to the heat shield, about 2m away, and take a whole bunch of pictures. Then we back off and drive around to its west side, where we take some more pictures. (Assuming this all goes well, we'll be able to approach to IDD range on the following sol.) But if we don't have room in flash, we won't be able to take all the pictures, which means we'll cut the sol in half. The work won't be entirely lost, but we'll have to defer most of it by a sol.

So we go ahead, hoping for the best. And our optimism is rewarded when Roger Klemm strolls into the room with a big grin on his face. "The deletes succeeded," he announces. "We have 361 megabits available." So we're good to go.

And "go" we do. If anything, we go too fast. When we finish the CAM, Emily complains about exactly that. "That went too fast," she said. "Did we miss something?" She's serious enough about it to bring up her checklist and go over it. But we didn't miss anything. Apparently, we're just getting better at this. Maybe we're smarter than Art thinks.

"It's a fuck-up -- no data until 15:20. We're treating it as a restricted sol."

Which means no driving, no IDDing. That blows our schedule out of the water. We were supposed to do our final approach to the heat shield thisol, but now we'll have to put that off for a day. "I'll stay late."

"Tell Art."

I do that, but Art shakes his head. "You're one of many."

That's what I expected. "Gotcha," I say, "but I'm casting my vote. I'm happy to stay late and get this done. After all, I hate to blow off a million-dollar sol."

"Well, in Euros, it's only a couple hundred thousand."

So that doesn't happen. I don't know what caused the delay, except that it resulted from some kind of DSN problem, and it's expected to continue for the near future.

Before leaving, I sign a 50-Tolarjev note -- Slovenian currency. This is something like 29 cents, in US currency. (What did thisol cost in Slovenian Tolarjev? I don't want to think about it.) This kind of thing happens now and then; a space fan from somewhere in the world sends in something for the ops team to sign. I'm always happy to do this, as indeed is most of the ops team. We're just engineers with unusually cool jobs, but it's nice when somebody makes you feel like a rock star.

Justin's the only one who refuses to sign it. "Isn't that illegal?" he frets. "I'm sure I'd be breaking some international money law, or something."

Justin is a nice guy, and a heck of a bright guy; I like him a lot. But for the love of God, Justin! Interpol is not going to drag you off to some UN dungeon for scrawling your name on a Slovenian banknote!

2010-01-04

As of today, Spirit has been on Mars for one full (Earth) year -- four times her design lifetime. I've come in early because today promises to be a full day. Not only are we likely to have reasonably complex IDD work as well as a reasonably complex drive, but we'll also be interrupted frequently for Spirit's birthday celebrations.

The TPS team is much happier with the MIs than I expected. When I took a look at the images from home, three of the five center-position MIs appeared to be badly exposed and possibly also out of focus. But when I look at them again, I see they're a lot better than I thought. Still, the TPS team wants some additional coverage, so we've got more MIing to do -- and in a challenging region, near the limit of what the IDD can reach.

On top of that, we're driving to East Point -- a spot about 10m east of the heat shield -- stopping to take images, and then continuing to the far side of the heat shield. And somewhere in there, we have to make time for anniversary cake.

Because I'm not missing the cake. I've missed out on a lot of things on this mission, but I'm not missing the cake.

The IDD sequence is going to be a fair bit of work, and since Jeff's RP-2 thisol, I hand him the drive, which makes him happier than a pig in slop. The challenging part of this IDD sequence is not simply reaching the desired target, but reaching it at the correct angle. Just to reach it, we have to extend the elbow joint almost to its limit. And to get what the TPS team most wants to see, we have to angle the MI back toward the rover, which means the IDD needs to reach even farther out. I try a dozen different tricks, but in the end I reluctantly conclude that we simply can't get the angle they want and they'll have to settle for something else, though I don't like it.

All this consumes a great deal of time. Meanwhile, the TV is playing the press conference, in which Elachi and O'Keefe are thanking all the team members who made this possible, etc. Happily, they have the good sense to thank not only "all the people in this room" but also all those who can't be in the room to get our media-whoring faces on TV because we're actually, you know, driving the rovers. Ahem.

I do plan to get my picture taken in the team picture at 14:00, which is why I'm surprised just after the press conference, about 13:00, when John Wright comes in and says, "They're taking the team picture now." (Most people missed whatever announcement he heard, so lots of folks weren't included in the picture. That's going to be good for morale.) So he and I head over to von Karman and turn out to be the last two arrivals. There's no more room on the stage, so they seat us in front, right in front of Elachi and O'Keefe. Cool!

Afterward, I have John take my picture with O'Keefe, who's very nice about this even though he has no idea who the hell I am. He's just as nice the second time, after John screws up the picture and we have to do it over. Sheesh, the guy can drive a rover on Mars, but ... well, do I really need to say it?

As I'm leaving, I run into a woman who tells me I was great in "Mission: Space." Which was this Disney interview I did months ago, that sort of seemed to vanish into the aether. Apparently it turned into a special that aired on the PAX network, hosted by Levar Burton, promoting a Disney (EPCOT) ride of the same name. Frank had told me his wife saw me in this show, but they didn't know what it was except that it was on PAX and hosted by Levar Burton. I missed it, but the nice lady says she's trying to get a copy for her own files -- she helped rewrite the show, or something like that -- and she'll try to get me one, too. She says repeatedly that I "presented well," which is a term I'm not familiar with, but the way she says it, it sounds like high praise.[1]

They showed the cake at the press conference, but they pulled a switch and served some other cake to the assembled guests. The real cake was brought back to Building 264 so it could be shared with the ops team. Jim Erickson gives a little speech -- "Let's make this an annual event," he says. (Earlier, Elachi called this the fourth anniversary, not the first -- because, after all, this represents four times the warranty period.) He invites Squyres to say a few words, and he says this:

"With the anniversary coming up, I've been doing a lot of interviews, and one of the best questions I got asked recently was, 'What would you say to the engineering teams who helped you get to this point?' And all I could think of to say was, 'Thank you.' I've been working on this one way or another since 1987, and for the science team and for me personally, this is absolutely, literally, a dream come true. So --" he starts to get choked up -- "thank you."

The cake is good.

The anniversary prompts a great deal of reminiscing, as anniversaries will. In the sequencing room, Art Thompson and Steve Squyres can't stop talking about how close we came to disaster, and how many times. They have more stories than time to tell them -- miswired thruster valves, software problems, important last-minute tests gone awry. "I wonder what are the things that almost got us but didn't, you know?" Steve says. "The five-meter sharp pointy rock right next to our first bounce ...."

"We were lucky," Art says. "And I'd rather be lucky than good. But actually, we were probably a little of both."

"A little of both," Steve nods.

This doesn't fit the thread of the narrative, but I have to relate it:

We can see a bunch of springs in the vicinity of the heat shield, and I remember to ask one of the metric-challenged TPS folks something I've been wondering about: "How big are those springs?"

"Cucumber," she says.

I look at her thoughtfully. "Would that be a metric cucumber?"

As is his wont, Steve asks some pointed, detailed technical questions at the CAM. Then he seems to feel the need to apologize for them. "My secret desire, after you've all gone off to work on other projects, is to become a rover planner," he explains.