On 14 November 2023, NASA’s interstellar space probe Voyager 1 began sending gibberish back to Earth. For five months, the spacecraft transmitted unusable data equivalent to a dial tone.
In March, engineers discovered the cause of the communication snafu: a stuck bit in one of the chips comprising part of Voyager’s onboard memory. The chip contained lines of code used by the flight data subsystem (FDS), one of three computers aboard the spacecraft and the one that is responsible for collecting and packaging data before sending it back to Earth.
JPL engineers sent a command through the Deep Space Networkon 18 April to relocate the affected section of code to another part of the spacecraft’s memory, hoping to fix the glitch in the archaic computer system. Roughly 22.5 hours later, the radio signalreached Voyager in interstellar space, and by the following day it was clear the command had worked. Voyager began returning useful data again on 20 April.
NASA engineers managed to diagnose and repair Voyager 1 from 24 billion kilometers away—all while working within the constraints of the vintage technology. “We had some people left that we could rely on [who] could remember working on bits of the hardware,” says project scientist Linda Spilker. “But a lot of it was going back through old memos, like an archeological dig to try and find information on the best way to proceed.”
Minuscule Memory
Voyager 1 and its twin, Voyager 2—which also remains operational—were launched nearly 50 years ago, in 1977, to tour the solar system. Both spacecraft far surpassed their original missions of visiting Jupiter and Saturn, and in 2012, entered interstellar space.
“That mission literally rewrote the textbooks on the solar system,” says Jim Bell, a planetary scientist at Arizona State University and author of a book recounting 40 years of the mission. “We’ve never sent anything out that far, so every bit of data they send back is new.” The 1960s and 1970s technology, on the other hand, is now ancient.
Decades after the tech went out of vogue, the FDS still uses assembly language and 16-bit words. “These are two positively geriatric spacecraft,” says Todd Barber, a propulsion engineer for Voyager. Working to fix the issues, he says, is “like palliative care.”
Voyager’s flight data subsystem (FDS) and other computers were custom built for the mission by General Electric.JPL PHOTO 360-751AC
To first diagnose the issue, NASA’s engineers first tried turning on and off different instruments, says Spilker. When that proved unsuccessful, they initiated a full memory readout of the FDS. “That’s what led to us finding that piece of hardware that had failed and that 256-bit chunk of memory,” she says. In one chip, the engineers found a stuck bit, fixed at the same binary value. It became clear that the chip was irreparable, so the team had to identify and relocate the affected code.
However, no single location was large enough to accommodate the extra 256 bits. “The size of the memory was the biggest challenge in this anomaly,” says Spilker. Voyager’s computers each have a mere 69.63 kilobytes of memory.
To begin fixing the issue, the team searched for corners of Voyager’s memory to place segments of code that would allow for the return of engineering data, which includes information about the status of science instruments and the spacecraft itself. One way the engineers freed up extra space was by identifying processes no longer used. For example, Voyager was programmed with several data modes—the rate at which data is sent back to Earth—because the spacecraft could transmit data much faster when it was closer to Earth. At Jupiter, the spacecraft transmitted data at 115.2 kilobits per second; now, that rate has slowed to 40 bits per second, and faster modes can be overwritten. However, the engineers have to be careful to ensure they don’t delete code that is used by multiple data modes.
Having successfully returned engineering data, the team is working to relocate the rest of the affected code in the coming weeks. “We’re having to look a little harder to find the space and make some key decisions about what to overwrite,” says Spilker. When their work is completed, the Voyager team hopes to return new science data, though unfortunately, all data from the anomaly period was lost.
Built to Last
The cause of the stuck bit is a mystery, but it’s likely the chip either wore out with age or was hit by a highly energetic particle from a cosmic ray. Having entered interstellar space, “Voyager is out bathed in the cosmic rays,” Spilker says. Luckily, the spacecraft was built to take it, with its electronic components shielded from the large amount of radiation present at Jupiter. “That’s serving us quite well now in the interstellar medium.”
The 8-track tape aboard Voyager stores data about planets and plasma, not pop music. NASA/JPL-CALTECH
When Voyager was built, the 12-year trip to Uranus and Neptune alone was a “seemingly impossible goal for a 1977 launch,” says Barber. The longevity of Voyager is a testament of its engineering, which accounted for many contingencies and added redundancy. The mission also included several firsts, for example, as the first spacecraft with computers able to hold data temporarily using volatile CMOS memory. (An 8-track digital tape recorder onboard stores data when collected at a high rate.)
Importantly, it was also the first mission with a reprogrammable computer. “We take it for granted now,” Bell says, but before Voyager, it wasn’t possible to adjust software in-flight. This capability proved essential when the mission was extended, as well as when issues arise.
Going forward, the Voyager team expects to encounter additional problems in the aging spacecraft—though they hope to make it to the 50-year anniversary before the next one. “With each anomaly, we just learn more about how to work with the spacecraft and are just amazed at the capabilities that the engineers built into it using that 1960s and ’70s technology,” Spilker says. “It’s just amazing.”
Quelle: IEEE Spectrum