Battle’s Lawsby Dwayne A. Day
|
| Out at the pad several of the workers noticed smoke emerging about 60 feet up, from the spot where the Hustler and the Thor were connected. This was bad, they knew. Really bad. |
By this time, 29 Thor missiles had been launched from Cape Canaveral, with 22 partial or complete successes. Six of these had been Thor Able launches, and three had lifted the Lunar Probe and Pioneers 1 and 2 into space. This launch was only about a month or so late from the original launch date, which was pretty good considering that the program did not even exist a year before. On December 16, 1958, a little over a month before, Vandenberg Air Force Base was christened as the Air Force’s west coast launch facility with its first missile shot—a Combat Training Launch of a Thor IRBM, which successfully flew 1,300 miles downrange into the Pacific. The little missile had essentially proven itself by this time in a number of test flights and was ready to become operational overseas. Thor Able had also proven itself as a space launcher. Although the Air Force still wanted to increase Thor reliability to eighty or ninety percent, most of the problems with the rocket had been ironed out by early 1959.
The Hustler was a different story. In fact, officially it did not even have a name. Some people called it a Hustler after its Bell engine, which had been destined for the now-canceled “Pilotless Propulsion Pod” (i.e. a missile) for the B-58 Hustler. Some called it the WS-117L. But most of those who worked on it at Lockheed Building 104 at Sunnyvale referred to it simply as “the vehicle.” This particular vehicle was called by its serial number, 1019.
On this cold day in January the pad workers performing the final checkout of the rocket and spacecraft were oblivious to the scenery, however. There were only a few of them—less than a handful, all enlisted men—working at the base of the vehicle on its concrete launch pad. They were encased in heavy acid suits, designed to protect them from the Hustler’s dangerous propulsion fuel. The rocket used Inhibited Red Fuming Nitric Acid as an oxidizer. It was every bit as nasty as its name implied. IRFNA (pronounced “urfna”) was combined with JP-4 jet fuel on the first missions. Lockheed engineers already had plans to substitute a different fuel for the JP-4, but it was the IRFNA that everyone worried about and that was why they wore the acid suits. If an acid suit got a hole in it, technicians would slap a patch over it. The suit with the most patches over it went to the highest-ranking member of the pad crew, based on the fact that the extra level of fabric from all the patches offered greater protection to the wearer.
The Thor was unfueled—its kerosene and liquid oxygen would be loaded just before launch. But the Hustler was receiving its supply of IRFNA when someone decided to conduct a test of the Hustler Guidance and Control Auto Sequencer. This device, known as the “D” timer, controlled various events during the vehicle’s ascent into orbit. Within the blockhouse, about half a mile away, an alarm horn suddenly went off. Controllers frantically looked over their instruments, trying to discern what was wrong.
Out at the pad several of the workers who were farther away from the vehicle noticed smoke emerging about 60 feet up, from the spot where the Hustler and the Thor were connected. This was bad, they knew. Really bad. They ran, scattering as fast as they could to hide behind concrete blast walls on the pad.
One person who did not see this was the crew chief, who was standing right next to the vehicle in his patched acid suit when smoke started pouring out of the rocket high above his head. He was apparently checking his clipboard when a foot square aluminum plate fell off the Hustler and clanged at his feet. He too decided that the last place he wanted to be was next to thousands of pounds of Red Fuming Nitric Acid and he beat a hasty retreat for safer ground.
Inside the blockhouse the controllers had realized what was happening. Somehow the Hustler’s internal timer had been activated. The vehicle behaved as if the Thor had burned out after boosting it high into the atmosphere. First it fired the explosively activated collar that held the two vehicles together so that they could separate. Then it had fired its small solid-propellant ullage rockets used to push the Hustler away from its spent booster and push the propellant in its tanks to the rear so that the engine could fire. These rockets were located on the aft end of the Hustler, between it and the Thor’s upper liquid oxygen tank. A few seconds later, the sequencer had ejected covers over the spacecraft’s antennas and horizon sensors—the aluminum plate that landed in front of the Crew Chief was one of these covers.
Fortunately, someone in the blockhouse reacted quickly. He immediately cut power to the rocket and yanked the fuel levers so that the Hustler’s toxic fuel began draining out of the rocket and back into its storage tanks. The rocket sat there for a long time as everyone waited in horror to see if the IRFNA might explode from the heat of the ullage rockets, or if the unsecured Hustler—which now sat on top of the Thor without anything other than gravity holding it there—might tip over in the wind, fall to the ground, and burst into a fireball.
| The failure was really a failure of systems engineering—the interaction of complex systems all hooked together. |
The whole experience was an embarrassment. The Air Force had flown several space missions during the preceding year. How come this one had not even gotten off the ground? Although the launch attempt had not been named beforehand, after the vehicle was secured and hauled back down to the horizontal and everything was made safe, those who knew about it began calling it “Discoverer Zero.” The trade magazine Aviation Week, which had by now changed its name to Aviation Week & Space Technology, dutifully reported the failure a week later.
In the investigation that followed, Air Force and Lockheed officials quickly determined what had gone wrong. Somehow there was a “sneak circuit” between the Hustler, Thor, and the blockhouse. This sneak circuit had activated the event sequencer, which commanded certain things to happen on the spacecraft at certain times and in specific order. This sneak circuit had crept into the system because nobody had been assigned the task of overseeing the interaction of all of the vehicle’s separate systems with the Thor and the systems in the blockhouse. There were people who ran full checks of the Hustler on the ground, but not in concert with the other systems.
According to Frank Buzard, a young Air Force officer who was in charge of the overall launch program, the quick fix to the problem was easy. Lockheed engineers developed a system for testing the sequencer without it actually activating any of the Hustler equipment. They also reviewed all the wiring diagrams to find just exactly where the sneak circuit was and eliminate it. The review of the wiring diagrams had to be done by hand, laboriously poring over blueprints and other information, figuring out how one system connected to the others.
But the real failure was not because of equipment, it was a problem with planning and procedures. The failure was really a failure of systems engineering—the interaction of complex systems all hooked together.
Dan DeBra was one of the Lockheed engineers working on the vehicle in the company’s Sunnyvale facility in Building 104, a self-described “peon” who worked on proposing other missions for the rocket. He was one of the people who helped define how a Sentry Hustler would differ from a Discoverer Hustler, and how a Discoverer Hustler could be adapted to other space applications. He was not at the pad for the failure of 1019, and the first test was not his responsibility, but he knew a lot about how the vehicle was supposed to work because it was his job to try and convince other users that their vehicle could do amazing things, assuming that it got into orbit.
As DeBra pointed out, the discovery that systems engineering had failed on this launch was not a startling revelation in retrospect. It is the type of lesson that all designers of complex machines eventually learn, and often have to relearn. Beginning around the turn of the 19th century, manufacturers recognized that as the products they produced became more complicated, designing and building them became more chaotic. Many people worked on a set of engineering blueprints, modifying their own individual parts. When the blueprints reached the workmen building the machine, parts did not fit or work as designed. The solution was to have a master designer sign the blueprints. No change could be made to them without his agreement and signature. Clear lines of responsibility and accountability had to be established or things would go wrong on the manufacturing floor, or worse, the machine would break down completely. Nobody had done that with the Hustler and the designers all smacked their foreheads when they realized it. It was so obvious, and yet they had missed it.
Every organization eventually learns the lesson of systems engineering. They learn that one part of a complex machine can affect another part in unforeseen ways. They learn that someone has to be responsible for the entire system and aware of its individual components so that they do not interfere with each other. And they learn that communication, and procedures for communicating, are important for success. The earlier they learn this lesson the better. Skip it and rockets blow up.
The ballistic missile program had learned the lesson of systems engineering. The military space program had not. Discoverer Zero was the result.
As Buzard said, “the primary lesson from 1019 remains—you must conduct an all-up end-to-end test in the final configuration of the spacecraft before it leaves the factory and again on the launch pad when it is mated to the booster and the launch facility and blockhouse.”
Colonel Lee Battle was in charge of the Air Force’s Discoverer program office in Los Angeles. He felt that Discoverer Zero had been a warning. They needed to implement better checks and procedures. He made this a major component of his management of Discoverer, insisting that Lockheed designate certain people to be responsible for the overall integration of the vehicle. He insisted that the program office and contractors clearly designate the procedures to be followed for each step of a launch and operation of a space vehicle. He made specific officials accountable for the success of the system. Battle was tireless, even obsessive, about this. They had to fix these problems before they started flying rockets.
| Battle believed that all failures were preventable. “There is no such thing as a random failure!” Battle would declare. It would become his mantra. |
Within the program office in Los Angeles, this became known as the “Lesson of 1019,” often shortened to “1019.” All Battle or anyone had to do was say “1019” and everyone knew what they meant—check to make sure that nothing has been missed, that doing one thing will not cause something else to go wrong. Emphasize “systems integration” so that all of the complex parts of a spacecraft will work properly together. Quality control was vital and required constant vigilance. When the Lockheed engineers took the vehicle apart back in the factory, they found a number of flaws that would have prevented it from ever reaching orbit.
Battle later incorporated this lesson into an overall list of his rules for successful military space operations (Battle’s Laws can be downloaded here). Among rules like “Don’t over-communicate with higher headquarters,” there was “System integration is very important,” and below it the entry “1019.” These rules became known as “Battle’s Laws.” They were later simplified and shortened and posted in the Discoverer Program Office. Young officers were instructed on their importance for successful programs. “1019” was dropped from the shortened version, but by that time everyone in the office had the lesson of systems integration drummed into their skulls.
“It was a common remark back in those days, ‘Well, there was a random failure of a two cent component that caused this $5-million thing to blow up,’” Frank Buzard recalled. “Well, you never used the words ‘random failure’ around Lee Battle, because the result was very predictable. I mean there was an explosion.” Battle believed that all failures were preventable. “There is no such thing as a random failure!” Battle would declare. It would become his mantra.
Discoverer Zero never got another chance to fly. The Thor rocket was damaged. The upper hemisphere of the kerosene tank had been compressed inwards by the thrust from the Hustler’s ullage rockets. But it was salvageable and the Air Force sent it back to Douglas for repair.
Vehicle 1019 was another matter. Dan DeBra remembered seeing it after it had been returned to Lockheed’s assembly facility in Sunnyvale. It had been cleaned up and he did not see any obvious signs of damage. But nobody trusted the vehicle. IRFNA had spilled over 1019’s aft equipment rack. IRFNA was acid and acid did nasty things to metal, rubber, copper, and plastic. Nobody wanted to sign off on the spacecraft, certifying that vehicle for flight ever again. And because it was one of the very first vehicles, it lacked improvements that were being incorporated into newer models. 1019 became a hangar queen, used for tests and fit checks in the factory and destined never to make it into space. But its legacy lived on in the military space program.