Of hydrogen and humilityby Jeff Foust
|
| “We feel really good that, once we get through that,” NASA’s Cobb said of the hydrogen kickstart test, “we’ll know everything we need to know about the vehicle.” |
Agency officials, in briefings leading up to the launch, were also confident. “We’re not going to promise that we’re going to get off on Monday,” Mike Sarafin, NASA Artemis mission manager, said at a briefing two days before that first launch attempt, citing not just potential technical problems but also weather and range violations. “But we do feel good about our attempt on Monday.”
Others echoed that confidence. “The vehicle is in great shape,” Sharon Cobb, NASA SLS associate program manager, said in an interview the same day as the briefing. “We have really tested everything we can test up until launch and we believe our systems are in really good shape, except for that one activity of chilling down the engines.”
Ah, that exception. During the four wet dress rehearsals of the SLS performed in April and June, NASA never tested what is sometimes called the “hydrogen kickstart,” where liquid hydrogen flows into the core stage’s RS-25 engines to cool them so they are not thermally shocked when that fuel starts coursing through them just before launch. A leak in a hydrogen bleed line kept the agency from testing it in the final dress rehearsal in June, so they decided to move up the kickstart earlier in the countdown so they would have time to troubleshoot any issues that come up.
“If we do not successfully demonstrate that,” Sarafin said of the hydrogen kickstart at a briefing after the flight readiness review August 22, “we are not going to launch that day.”
Cobb was confident that, once that was successfully tested, there would be few obstacles for launch. “We feel really good that, once we get through that, we’ll know everything we need to know about the vehicle.”
Was it any surprise, then, that the hydrogen kickstart became a problem during the first launch attempt? By the time they were ready to start flowing liquid hydrogen into the engines, they were already behind schedule, having to deal with a delayed start of tanking because of weather and a liquid hydrogen leak in a quick-disconnect fitting of a line leading into the core stage. Three of the engines cooled to near the planned temperature, but the fourth, designated engine #3, failed to do so.
Engineers assumed the engine was not cooling because it was not getting enough liquid hydrogen, so they took steps to try and increase the flow into the engine, including shutting off the liquid hydrogen flowing into the other three engines. That did not work, and problems with a vent valve complicated those troubleshooting efforts. After an extended, unplanned hold at T-40 minutes, mission managers scrubbed the launch around the time the launch window opened at 8:33 am EDT—just after Vice President Harris landed at the Kennedy Space Center to watch the launch. (Harris stayed for a tour, but did not give a speech before returning to Washington.)
By the next day, NASA officials concluded the problem with not with the much-maligned engine #3 at all, but rather with a temperature sensor. The hydrogen kickstart is designed to cool the engines to about –250 degrees Celsius. Three of the engines got down to about –245 degrees Celsius, but engine #3 was only at about –230 degrees Celsius, according to temperature sensors in the engines.
John Honeycutt, NASA SLS program manager, said other data showed plenty of liquid hydrogen was flowing into the engine. “We understand the physics about hydrogen performs,” he said. “The way the sensor is behaving doesn’t line up with the physics of the situation.”
Thus, they concluded the sensor was at fault. “Over the course of my career I have seen many sensors on launch vehicles be erratic,” Honeycutt said, “and go out of calibration.”
| “We understand the physics about hydrogen performs,” Honeycutt said. “The way the sensor is behaving doesn’t line up with the physics of the situation.” |
Honeycutt said that, on the next launch attempt, NASA would move up the start of the hydrogen kickstart to just after the start of “fast fill” of the core stage’s liquid hydrogen tank, following the procedure it used for Green Run tests at the Stennis Space Center. “The only thing I know to change to replicate the success we had at Stennis is moving the test earlier in the timeline,” he said. Controllers would also essentially ignore the temperature sensor, relying on other data to confirm the engines were properly chilled.
At another briefing, Sarafin, the Artemis mission manager, suggested that the launch team might have come to the same conclusion during the launch attempt, but were already behind schedule because of weather and other technical problems. “We were off the script in terms of the normal tanking operation,” he said. “One of the worst things you can do when you find yourself in a hazardous condition is just go even further off-script.” (Even without any technical problems, weather would have been no-go at the start of the two-hour launch window, although meteorologists later said it would have turned go in the last half-hour of the window.)
NASA managers felt that, with that understanding of the hydrogen kickstart issue and some other work, including adjusting a liquid hydrogen connection at the pad that suffered a minor leak in the first launch attempt, they were ready to try again September 3. “We’re comfortable with our risk posture,” Sarafin said September 1 as the countdown started for the next launch attempt. “That said, there’s no guarantee we’re going to get off on Saturday, but we’re going to try.”
 SLS stands on the pad at Launch Complex 39B between the first and second launch attempts last week. (credit: NASA/Bill Ingalls) |
Leaking hydrogen
As it turned out, NASA would never find out if the revised procedures would change anything with the hydrogen kickstart. The second countdown on the morning of September 3 never got to that point in the countdown.
As controllers prepared to begin loading liquid oxygen and liquid hydrogen into the core stage of the SLS, they ran into two issues. One was a brief issue with the liquid oxygen line, which was quickly resolved. Around the same time, an overpressure warning stopped the chilling of the liquid hydrogen line, but that, too, appeared to be quickly resolved, allowing liquid hydrogen to start flowing into the core stage.
But then another liquid hydrogen leak was detected. There was initially not much concern about the leak, as one had been seen on the first attempt and dealt with by allowing the quick-disconnect, or QD, fitting warm up, then flowing liquid hydrogen back through it, allowing the change in temperature to reseat the connection—the equivalent of turning it off and back on again. So, engineers tried that and then resumed liquid hydrogen flow.
| “In terms of the leak that we saw on Monday, it was a manageable leak. This was not a manageable leak,” Sarafin said. |
This time, though, it didn’t work and the leak continued. It was on to Plan B: close the valve and hit the QD with a burst of helium from ground pressurization systems to try and reseat it. The leak, though, continued. One final effort involved going back to Plan A, warming and then cooling the fitting again. And, once again, the leak continued.
After some extended deliberations among mission managers, NASA scrubbed the launch exactly three hours before the window was set to open. The agency had taken a step backwards in this launch attempt, never getting far in the loading of liquid hydrogen in the core stage and not even beginning propellant loading of the upper stage.
At a briefing later September 3, Sarafin said the leak on the latest attempt was much bigger than the one on the first attempt, exceeding a limit of 4% concentration of hydrogen by a factor of two to three. “In terms of the leak that we saw on Monday, it was a manageable leak. This was not a manageable leak,” he said.
He suggested there may be a link between the liquid hydrogen leak and the “inadvertent overpressurization” of the liquid hydrogen leak that triggered an alarm earlier in the countdown. A valve was opened incorrectly—possibly because of human error—that briefly caused pressure in the line to reach three times the limit of 20 psi, enough, potentially, to damage the seal for the QD fitting and cause the leak. He added, though, it was still too early to confirm any cause and effect.
At that September 3 briefing, the most recent update from NASA, it was not clear when the agency would be ready for a launch, beyond ruling out the current launch period, designated launch period 25 by NASA, that ended September 6. “Launch period 25 is definitely off the table,” said Jim Free, NASA associate administrator for exploration systems development.
The next two opportunities are launch period 26, from September 20 to October 4, and launch period 27, from October 17 to 31. In comments immediately after the scrub on NASA TV, NASA administrator Bill Nelson suggested a launch no earlier than mid-October, or the beginning of launch period 27, saying that a higher priority was the early October launch of the Crew-5 mission to the International Space Station from neighboring Launch Complex 39A.
At the briefing, though, NASA officials suggested both launch periods were in play, depending on the path forward. Free and Sarafin said they were looking at options to either roll the vehicle back to the VAB and repair the QD fitting there or attempt a repair on the pad.
“We need to look at what work exactly we need to do,” Sarafin said. Attempting the repairs at the pad has pros and cons, “and the cons happen pretty much every afternoon around here when you get a shower or thunderstorm rolling through.” The pros, though, include the ability to then test the repairs at cryogenic temperatures using liquid hydrogen supplies at the pad, something that can’t be done at the VAB.
Another pro would be avoiding rolling back the vehicle to the VAB, saving time as well as wear and tear on the vehicle. (Agency officials would not say how many times the SLS is rated for rolling out to the pad and back, but it is far less than the number of tanking cycles on the core stage.) That could preserve an opportunity of a next attempt in launch period 26, something that is unlikely if the vehicle has to go back to the VAB.
| “I would simply say to you that space is hard,” Nelson said. “We are developing new systems and new technologies, and it takes money and it takes time.” |
A further complication in those plans is the SLS flight termination system (FTS), whose certification by the Eastern Range runs out at the end of the current launch period. Because the system’s batteries cannot be serviced at the pad, NASA would have to roll back to the VAB regardless of the decision on where to do the repairs unless the US Space Force’s Space Launch Delta 45, which runs the range, granted another extension.
“We don’t have an FTS waiver right now beyond 25 days,” or the end of the current launch window, said Free. “Until we have that, we have to roll back.”
He added NASA would consider asking the Eastern Range for an extension, but first needed to determine how long of an extension it wanted. “That negotiation hasn’t happened, and so as far as I’m concerned, we have to roll back.”
 NASA administrator Bill Nelson delivers the bad news about the launch scrub September 3 to VIPs at the Kennedy Space Center. He defended the design choices behind the rocket, noting that “space is hard.” (credit: NASA/Bill Ingalls) |
Shuttle’s revenge
While NASA touts the SLS as a cutting-edge vehicle, the latest scrub is the result of design choices made more than a half-century ago. NASA elected to use liquid hydrogen and liquid oxygen propellants for the Space Shuttle’s main engines for the performance they offered, taking with them the headaches of dealing with liquid hydrogen. At the briefing after the second scrub, officials invoked the “summer of hydrogen” in 1990 that grounded the shuttle fleet for months because of leaks. Hydrogen leaks would crop up time and again into the final years of the shuttle program.
When Congress directed NASA, in the 2010 authorization act, to develop the SLS using shuttle components, it meant taking with it problems like liquid hydrogen leaks. “We deferred to people who best knew the systems,” Nelson said after the second scrub when asked if had any concerns about using liquid hydrogen. “We did not have any question about hydrogen. We deferred to the experts.”
Nelson, of course, helped author that 2010 act as a senator. That bill directed NASA to complete the SLS and enter “operational capability” by the end of 2016. NASA has missed that goal by five and a half years, and counting.
“I would simply say to you that space is hard,” he said at an August 27 briefing when asked what lessons NASA could take from the extended delays in SLS’s development. “We are developing new systems and new technologies, and it takes money and it takes time.”
The SLS will take a little more time and maybe a little more money as well. (NASA officials declined to estimate the cost of the two scrubs or the repairs needed to be ready for flight, beyond Nelson stating that “the cost of two scrubs is a lot less than a failure.”) But when NASA is ready to try again to launch the SLS, be it later this month or late next month or some time after that, it may be wise to temper that confidence seen earlier with the recognition that even tried-and-true technologies pose challenges—with more, perhaps, yet to come.
Note: we are using a new commenting system, which may require you to create a new account.