When an Extra Second Matters

As the new year of 2006 approached, many news stories noted that an extra second would be added to the world's clocks at the stroke of midnight, December 31st. For most, this was a quirky factoid to mention during the night's festivities. But, for many people at Chandra's Operation Control Center (OCC) and other places, it added up to a lot of time.

A "leap second" is a one-second adjustment that is used to keep civil time aligned with the Earth's rotation. Such an adjustment is necessary because atomic clocks, which are based on a frequency of cesium atoms, are incredibly stable and are used to define the duration of a second. The Earth's rotation, on the other hand, is not as stable and has been shown to slow down by tiny amounts.

When the difference between civil time and the Earth's rotation approaches 0.9 seconds, a leap second is considered. It is the responsibility of the International Earth and Reference Systems Services (IERS) to evaluate this difference and every six months determine if a leap second is necessary.

Waiting for the crowds to clear after the 2004 Independence Day fireworks, the OCC's Bill Davis stopped by his Cambridge office and checked his email. There, he found a bulletin waiting for him from the IERS announcing the intention to add a leap second to 2005. After forwarding the bulletin to the various Chandra operations teams, Bill Davis entered a request to respond to the upcoming leap second just 10 days later.

Why go through all of this effort for just one second? The answer is that each X-ray photon is given a detection time by the spacecraft and subsequent ground processing. Some observations have stringent time accuracy requirements, 20 microseconds (0.00002 seconds) in some cases.

Without addressing the issue of the leap second, the accurate timing required for certain observations could have been lost. Also, any errors could have caused considerable delay in the delivery of X-ray data to the world-wide community of science observers who use Chandra. Finally, recovery of telemetry files (information about where the satellite is in space) from the Deep Space Network is time-dependent. Therefore, any end-of-year data requests had to be crafted to make sure the leap second could not cause any data loss.

Previous leap seconds had been added every year or two, sometime twice a year. The unusual trait of the end-of-2005 leap second was that the previous leap second had been 7 year before, in 1998, the year before Chandra's launch. The 2005 leap second would be the first of Chandra's mission. To share information with other space-based missions, which would also be dealing with the issue, Bill Davis joined the Leap Second Working Group at NASA's Goddard Space Flight Center (GSFC) in Greenbelt, Md. Having worked on several space missions at GSFC before joining the Chandra project, Bill had an idea which aspects of Chandra's software systems would be most vulnerable to a leap second: the software that handles the calibration of Chandra's clock and the software that determines its position.

To make sure there were no surprises as the New Year passed, OCC's system engineer Jeff Holmes convened a meeting for the Chandra operations teams. During this meeting, Jeff asked all of the groups to evaluate their systems to accomplish one of two things: either determine that Chandra's systems would be immune to the leap second, or find solutions to deal with any potential problems that might arise (known as "work-arounds" to engineers and scientists).

Eventually, eight different Chandra systems were evaluated, both those on the ground at the OCC and others aboard the spacecraft itself. Ultimately, a simple change to the telemetry data removed the effect of the leap second for the calibration clock on Chandra. This allowed the clock on-board Chandra to sync up with the existing software that was not built to take leap seconds into account.

For Chandra position and velocity data, the Chandra team needed to adjust the start time of the file containing these data. This work-around let Chandra's software obtain the proper time for each data point. The final solution was to allow the OCC telemetry archive to contain essentially twice the amount of data for the last "second" of 2005 than would be stored for a normal second, thus adjusting the software the extra second.

All of the efforts undertaken by the staff at the OCC were entirely successful. Their planning and execution prevented any precious data loss for NASA's premier X-ray telescope. Thanks to them, the 2005 leap second passed as quietly for Chandra as it did for the rest of the New Year's revelers.

Many thanks go to George Leussis of the OCC, Ian and Janet Evans of the CXCDS, and Ken Gage and Sabina Bucher of the Flight Operations Team (FOT). This article was written by Bill Davis and Jeff Holmes of the CXC OCC and edited by the CXC EPO group.