Reports of technical issues, as well as a lack of a steady flow of progress reports, make some wonder whether the problems with the development of the Ares 1 are greater than first thought. (credit: NASA)

How to know when an engineering project is failing

by Eric R. Hedman
Monday, June 30, 2008

The media has, over the last several months, devoted some coverage to the thrust oscillation problems and other issues with the Ares 1. That brings to mind two questions about projects like these. How does an outside observer know if a project like this is succeeding or failing? How does an engineering team communicate with interested and influential observers the status of a project effectively?

Before I begin, I want to emphasize that I have no personal knowledge of what is going on inside of the Ares 1 development program. I have worked on a number of development projects that have had their challenges during their development. There have been times when customers have had doubts about the successful outcome. The first one of these was on a project at company I worked for shortly out of college. I was on a team developing software to control automated material handling cranes and shuttle cars for a project at two large parts stamping plants for an auto company. It was a strange lesson in how some projects in industry are managed.

Right after I started on the job I was assigned the task of developing a critical component of the project, communications software to run on a serial ports card for a DEC minicomputer. It was an assembly language project to translate command packets into formats specific to different brands of programmable controllers spread across a factory floor. The software had to drive sixteen 9600-baud serial channels with an eight-bit processor. I had never worked on anything similar before—or since. The first sign that I had that there were problems in this project was that no project manager or coworker asked me a thing about how I was doing for the first three months. The first question I was asked was, “Are you ready to take this out in the field and install it?” The answer was yes, but I was amazed at this lack of interest in a component of a multimillion-dollar project that was absolutely critical to the success of the project. I later found out that there was the same low level of interest by project managers in the other components of the project. Needless to say the project went way over budget and finished more than a year behind schedule. The chief lesson I learned from this struggling project was the need to effectively communicate within the development team and with the customer.

NASA has a communications problem. We, the interested public, are some of the key customers paying for this project in the sense that we push for the existence of NASA and mission that it has. We’re the ones that remind our members of Congress that we want NASA to carry on with its mission. We do have a habit of trying to believe and make others believe that we are a bigger presence than we really are. The interested public is definitely a minority of the American public. NASA doesn’t seem to understand how communications with this sector is important to maintaining their support with Congress and whichever administration is currently occupying the White House.

Several years ago, when Lockheed was developing the X-33 VentureStar, they had a website that detailed the progress of the project. I regularly checked it out to see how it was coming along (I can imagine a Lockheed veteran thinking while reading this, “So that was the guy visiting our site.”). There were pictures of the test firing of the linear aerospike engines. There were pictures of the fuel tanks and spacecraft body as they were fabricated. It was not a comprehensive site but it was interesting and made me think that the project was steadily progressing. I was a bit surprised when the project was canceled in part because the website made little mention of the challenges the project was facing. Information was not leaking out to the public nearly as much then as it is now with the Constellation project development. The big difference now is the number of users and sites on the Internet today focused on space.

I work on projects where my customers often do not understand the details and scope of the information the software we write is handling for them. It makes it an interesting challenge to communicate along the way the problems that are being encountered. I find that if you explain to the customer any significant unexpected challenge you find soon after you run into it, and explain what you are going do to find a solution, the customer will have greater confidence in the final outcome of a project. Even if the customer doesn’t fully understand the scope of the problem, I find that they do appreciate the attempt at an explanation.

There is one surefire way to know that the Ares 1 development has problems: look at the size of the budget. Virtually every project anywhere near this size has major challenges along the way. That’s one of the reasons it’s worth being part of a project like this. Without tough challenges to overcome there wouldn’t be a great sense of satisfaction when success finally comes. These are the types of projects that could have a number of major critical problems to overcome. The fact that it may have a number of major problems does not mean that they won’t all be over come. On the flip side of a project like this is that it’s possible to solve all the major problems but one—and fail.

A number of outside observers have expressed their doubts about NASA successfully overcoming the possible vibration problems in the five-segment first stage of the Ares 1. The issue that has been discussed is that there is a thrust oscillation in the first stage as there typically are in solid rocket motors. One problem with a vibration is that when the frequency of it comes close to or matches the natural frequency or a harmonic of the natural frequency of the part or assembly it is driving it can get greatly magnified in amplitude as each cycle reinforces the previous one. Mechanical engineers are taught methods to change the natural frequency of a part or assembly. There are in many situations ways to change the frequency of the vibration affecting the system or dampen them out. Unfortunately there is usually some weight penalty involved.

In NASA’s report to Congress they say they have a number of possible solutions to the vibration problem. They still have a problem with payload capacity. That is why the engineers at NASA are looking at ways to lighten the Orion capsule. Two problems in combination seem to have created the impetus for developing an innovative solution for a better and lighter airbag landing system.

I don’t know how bad the vibration problem on the Ares 1 is. I don’t know if there are other problems that might be worse. I don’t know if NASA has a handle on what they are trying to do. In the private sector where I work, if I don’t regularly communicate with the customer how a project is coming along, it isn’t acceptable. NASA needs to understand that they need to regularly communicate with the public how these projects are coming along. My customers assume the worst if I don’t regularly communicate with them. If information is coming out about problems in the form of rumors and innuendos people are going to start to assume the worst. It is often a sign that there are problems.

Boeing has recently announced delays on the first flight and initial customer shipments of the new 787 Dreamliner. This follows the delays Airbus has stumbled with on the A380 superjumbo. Both of these projects are pushing technology just as the Constellation project is. That is why I believe that schedule delays and cost overruns will be inevitable on the Orion/Ares 1 stack. The gap between the retirement of the shuttle and the start of Orion flights to the ISS will only increase. I am hoping, and think that NASA is betting on, the COTS program to fill this gap. I do hope that they are right.

On the 787 project Boeing’s goal has been to get more complete subassemblies from suppliers creating a quick final assembly process. The initial goals of having suppliers deliver complete subassemblies have not been met. Boeing has had to do work on these assemblies on the final assembly line. In addition they have discovered that they need to strengthen the center wing box. They are doing this by adding brackets and other design changes. They have had to make these changes when they were scheduled to do wiring in this area. The wing box problem delayed the installation of another system. The Boeing 787 is being designed using the latest in PLM (Product Lifecycle Management) software that was supposed to minimize or prevent these types of delays in development. I have no doubt that the Constellation project will run into similar projects. Boeing has added padding in their revised development schedule so that the delivery of this new product that they are betting the company on will not slip. This is something that I recommend NASA does so that there is no bad schedule news at a politically inopportune time.

On NASA’s Constellation website they show some interesting animations of how portions of the Constellation program will work. They have some interesting photos of some definite signs of progress such as the Orion Pad Abort Mock-Up in different stages of construction. What is missing is a regularly updated project status section. I know that project managers would rather work on the project than write status reports targeted at the public. NASA needs to realize that the next administration may not support the Vision for Space Exploration and could consider NASA a target for funding cuts. Regular postings on what is happening, including the problems that are cropping up and how they are being dealt with, would do a great deal in building the public’s confidence that the project can succeed. The site could also use a feature where the public could post questions or provide suggestions that could be addressed in the updates.

I don’t know if the vibration issue or other issues that we haven’t heard about are threatening the success of the Constellation program. When NASA doesn’t effectively communicate with its stakeholders it may be needlessly raising the question of the chances of the project’s success. When NASA isn’t effectively communicating on a regular basis how the Constellation project is coming along it’s time to ask if it has any significant problems that they’re not telling us about.

I’ve worked on projects where solutions to the problems aren’t apparent up front. While talking to the customer about these problems I have often gotten ideas on how to solve these problems. In a Florida Today article earlier this year, NASA administrator Michael Griffin said, “We can’t change our minds every three years and get anything done.” If NASA doesn’t do what it takes to maintain public confidence in what they are doing, the next administration may in a year change the country’s mind on NASA’s direction. If the next president changes NASA’s path, the Constellation project will just be the latest to fail, and for a reason that has nothing to do with technical issues.

Eric R. Hedman (ehedman@ldcglobal.com) is the chief technology officer of Logic Design Corporation.

Home