When Science Goes Wrong (34 page)
With TCM-3 out of the way, most of the team members turned their attention to solving the problem of the balky gimbal drive while the navigators prepared for TCM-4 – the last scheduled course correction. This was planned for September 15, just eight days before arrival at Mars. TCM-4 was the really crucial manoeuvre: it had to leave the spacecraft aiming for a point of closest approach to Mars (the point known as first periapse) that was about 150 to 200 kilometres above the planet’s surface. Much higher than that, and the ensuing aerobraking procedure would take too long for the Orbiter to be in place by the time the Lander arrived on December 3. Much lower, and the spacecraft might be damaged by frictional heating in the planet’s outer atmosphere; the lowest survivable altitude was thought to be about 80 kilometres. Previous missions had achieved their preset altitudes with extraordinary precision, sometimes missing by as little as four kilometres – not bad marksmanship after a 400 million kilometre voyage. But the fuzzy navigational solutions made such a precise result unlikely with Mars Climate Orbiter. In fact, the solutions obtained by use of Doppler measurements and those obtained by range measurements were predicting fly-by altitudes that differed by tens of kilometres. No one knew which set of calculations was more accurate, so it wasn’t clear exactly how large the TCM-4 correction should be. The team decided to aim for an altitude of 226 kilometres, a height that left considerable leeway in case the spacecraft was coming in lower than the navigators realised.
By September 15, the spacecraft team had the gimbal-drive problem fixed, and TCM-4 went smoothly: the solar panels stowed themselves and redeployed without incident, and the spacecraft did not go into safe mode. Now, with arrival at Mars just a week away, most of the spacecraft team turned their attention to making preparations for the aerobraking phase that would follow insertion into Mars orbit. These preparations were far behind schedule on account of the gimbal problem.
Meanwhile the navigators worked almost non-stop to refine their trajectory calculations. As the days passed, the predicted altitude of the first periapse gradually decreased to about 150 kilometres – a safe altitude, but only if their prediction was correct to within a few tens of kilometres. During the final two or three days before arrival, the navigators got a serious case of cold feet, and they brought up the possibility of making yet another, fifth course correction to raise the spacecraft’s fly-by to a safer altitude.
This was the last thing that Sam Thurman, the flight operations manager, wanted to hear. The possibility of conducting a TCM-5 was written into the mission’s contingency plans. However, the optional TCM-5 was mainly planned for a different circumstance, namely for a situation in which the second periapse (on the first aerobraking orbit) would be at the wrong altitude. Reprogramming a TCM-5 to alter the first periapse altitude would be a major task for a relatively small spacecraft team, especially when it was so far behind on its preparations for the aerobraking phase.
‘I remember that the nav team chief was nervous,’ Thurman told me. ‘He said, “I think we ought to bump up [the altitude],” and other people [were] saying, “If we do that, it’s going to screw up our aerobraking preparations.”’ Thurman himself sided with the latter group. “[Doing a TCM-5] would have put at risk our ability to get into the correct science mapping orbit a few weeks later,’ he said. ‘It would have jeopardised our ability to transition people over to preparations for the Lander’s arrival. We needed the same people at Lockheed Martin to do that as well as to get aerobraking started.’
Given the lack of consensus, it was left to Thurman (perhaps in conjunction with other leaders of the mission) to make the decision, and they decided against a course correction. ‘I think that was an error of judgment,’ John Casani told me, ‘but that’s easy to say.’
Up until about noon on the day before orbit insertion, it looked as if Thurman had made the right judgment call, because the navigational solutions began to cluster more closely together, suggesting that the 150-kilometre periapse prediction was correct. Unfortunately, the solutions were clustering tightly around an incorrect value. At about 1am the following morning, which was an hour before the spacecraft’s arrival at Mars, a new set of solutions became available, refined by observation of the ever-increasing pull of Mars’s gravity. These solutions now predicted that the spacecraft would fly by the planet at an altitude of 110 kilometres – a distance that was very much on the low side, though still probably survivable. Nothing could be done now except wait and pray.
Two groups of scientists and engineers had gathered at JPL and Lockheed Martin for the critical orbital insertion event. NASA’s cable channel began broadcasting the event live. Probably only a small group of insomniac space buffs watched the live broadcast, but several other TV channels sent cameramen and reporters to tape the event. Thurman and seven other JPL team members worked at computer terminals in a glass-lined control room, while the media and other curious onlookers peered in through the glass. A similar scene took place at Lockheed Martin’s mission support centre in Denver, though the working group there was much larger and the media representatives fewer. The dress code was shirtsleeves and jeans at the science lab, ties and slacks at the contractor.
At 1:41am in ‘Earth receive time’ – 11 minutes after the corresponding events at Mars – signals arrived indicating that the Mars Climate Orbiter had begun retracting and stowing its solar panels in preparation for the orbital insertion burn. The process took eight minutes. Then the spacecraft began turning 180 degrees so as to convert its main rocket engine into a retrorocket. This took six minutes. At this point, the spacecraft was heading over Mars’s north pole, and NASA-TV showed an animation of the gracefully gyrating spacecraft as it cruised over the white expanses of frozen carbon dioxide that marked the pole. At 1:56, pyros (explosive devices) fired to pressurise the fuel and oxidiser tanks. As planned, the spacecraft stopped transmitting data: the only signal still being received at JPL was the single-frequency ‘carrier’ signal. The tension was visible in the faces of the eight men in the control room.
Then at 2:01am came the voice of Lockheed Martin systems engineer Kelly Irish: ‘Real-time Doppler indicates main engine burn.’ In other words, engineers had seen that the frequency of the carrier signal was beginning to rise as the spacecraft decelerated under the influence of the retrorocket. It was a moment of exuberant relief in the JPL control room.
The group could follow the slowing of the spacecraft for the first four minutes of the planned 16-minute burn, but at 2:04 and 56 seconds the spacecraft’s carrier signal began to break up, and six seconds later it disappeared completely. This was the expected effect of the spacecraft passing behind the planet into its radio shadow. ‘At this time we are in our occultation period,’ announced Irish.
The spacecraft actually went into occultation 52 seconds before the event was expected. Less than a minute early – surely that tiny error could be of no significance after a nine-month voyage? Irish’s voice didn’t betray any surprise, and the NASA-TV commentators continued their chatter about the details of aerobraking. But to Thurman, it was a bad omen. He started staring intensely at a sheet of paper that he was holding in his left hand, then glancing up at the screen in front of him.
‘I remember I had a plot next to me that our mission engineer had made that allowed us to correlate the time that the spacecraft headed behind the planet. He’d come up with a clever scheme where, by looking at the time of loss of signal, we could get a guesstimate of the actual altitude. The lower the altitude, the earlier the loss of signal. I had this very quick way of watching data on the screen, seeing the loss-of-signal time, and then looking at the data sheet for the guesstimate. I remember we had the signal – it was very early, and I remember thinking, Uh-oh, this is not good. That really – yeah, my anxiety level shot up, I tried to hold my composure since there were seven guys with cameras in front of me.’
The NASA-TV commentator eventually seemed to pick up on the early occultation, because without mentioning the fact that it was early he attempted to explain it away. ‘The signal can be refracted by the atmosphere,’ he said. ‘The accuracy is not as deterministic as we’d like.’ Meanwhile, the eight men in the glass booth sat or stood there, fidgeting, looking at one another’s screens, well aware that they could do nothing but wait 21 minutes for the scheduled re-emergence of the spacecraft from occultation. By that time the spacecraft should have terminated its burn and turned so that its signals could be picked up by the 70-metre Tidbinbilla deep space antenna near Canberra, Australia.
At 2:26am, the predicted time for the spacecraft to re-emerge from occultation, there was dead silence in the control room, and even the television commentator stopped talking. The silence went on for minutes, while the men in the glass booth became increasingly fidgety, staring at their watches or at the computer screens, standing up and sitting down again for no apparent reason, or glancing into one another’s tired, anxious faces. Sam Thurman kept adjusting his wedding ring, as if its precise alignment was crucial for the spacecraft’s destiny. ‘Waiting for acquisition of signal,’ said the disembodied voice of Kelly Irish. And a minute later: ‘Still haven’t seen anything; stand by.’
After a few minutes, Thurman stood up and began to talk with the other members of the JPL team, including the Mars ’98 project manager, Richard Cook. His words weren’t audible on the TV broadcast, so I asked him whether he had been telling them that the spacecraft was probably lost. ‘I don’t say things like that,’ he said. ‘That can be devastating to a team’s morale. You always have to hope for the best and do everything you can to make it come about. And there’s a little bit of lore called the 24-hour rule, which is, when something seemingly bad or scary happens, don’t overreact for a minimum of 24 hours, because more often than not what actually happened and what you need to do about it might be different from what you think at first. So I tried to reach inside and gather the presence to say – getting emotional doesn’t serve any purpose, so you have to say, “Here are the options. The vehicle survived and may be in safe mode; it might be tumbling; it might have ended up in a much lower or higher orbit than expected because of an overburn or an underburn.” Remember, the ground stations need fairly accurate predictions of what the transmitter frequency is going to be in order to tune their receivers properly to hear a spacecraft 120 million miles away, orbiting another planet. So it could be the spacecraft was there and transmitting fine, and the set of predicts we used to drive the antennas were off.’
About 30 minutes later, Richard Cook came out of the glass booth and spoke with NASA-TV. He outlined the possible factors that might have caused the spacecraft to go into safe mode. ‘At this point, we’re still very confident that we’re in orbit at Mars,’ he said, ‘and we’re going to see the spacecraft signal sometime in the next few hours.’ With that, NASA wrapped up its live television coverage for the night.
Very quickly, however, devastating news came in. During the occultation period, the navigators had been working on a revised prediction of the spacecraft’s periapse altitude, based on data received shortly before orbital insertion. The results weren’t good. ‘They came back in and said, “Oh my God, this thing was 60 or 70 kilometres lower than we thought it was going to be,”’ said Thurman. ‘And that’s when we thought, “Oh boy, if it went that deep, it must have fried.”’
Engineers continued searching for a signal from the Mars Climate Orbiter for 48 hours, but it was largely a formality. Navigational errors, it now seemed clear, had led to the spacecraft dipping too deep into the Martian atmosphere during orbital insertion. Its exact fate was a matter for speculation. It may have broken up or exploded, scattering debris over the Martian surface – in which case there is some concern that terrestrial germs may have survived the heat of the re-entry and contaminated the surface. Alternatively, frictional heating may have caused the retrorocket to cut out prematurely – in which case the spacecraft may not have been captured into orbit around the planet at all and would have continued forever on its lonely solar orbit, perhaps to be seen again by some distant generation of Earthlings or Martians.
JPL’s John Casani was quickly appointed to head an internal investigation of the mishap. There was considerable time pressure, because the Orbiter’s companion spacecraft, Mars Polar Lander, was fast approaching the planet and was due to arrive on December 3. There were many similarities between the two spacecraft, and investigators wanted to ensure that whatever doomed the Orbiter would not also affect the Lander.
All the focus was on the navigational problems. Every piece of navigational software was scrutinised line by line, and on September 29 an engineer identified the crucial error: the lack of a conversion factor to change pounds of force to newtons in the Small Forces software. Once that error was identified, all the problems with navigation were readily explained.
To my knowledge, Casani’s report only circulated internally and was never published. ‘In my subjective opinion, it focused too much on all the technical details of what didn’t get done, or didn’t get done well enough,’ commented Thurman, ‘and too little on how the lab got itself in that position to begin with.’ That deficiency was quickly made up for by a second investigation, headed by Art Stephenson, Director of NASA’s Marshall Space Flight Center in Alabama. While agreeing with Casani that the units problem was the root cause of the mishap, Stephenson’s report put far more emphasis on the numerous contributing factors – inadequate training, testing and communication; the failure to resolve the anomalous navigational solutions or to report the problem through the proper channels; the failure to execute TCM-5, and so on – that together amounted to a systems-engineering or even programmatic failure. While not exactly criticising Goldin’s ‘Faster, Better, Cheaper’ philosophy, Stephenson urged that it be practiced under a set of guidelines that he summarised as ‘Mission Success First.’