Posted by Sten Westgard, MS
An interesting report came out a few days ago. A retrospective internal study of the risks found that "NASA seriously underestimated the dangers astronauts faced when the shuttle fleet began flying in the early 1980s."
"At the time, managers thought there was only a 1-in-100,000 chance of losing a shuttle and its crew. Engineers thought the probability was closer to 1 in 100. But in reality, the odds of a disaster were much higher.
"On each of the shuttle's first nine missions, there was a 1 in 9 chance of a catastrophic accident, according to the new risk analysis. On the next 16 flights that led up to and included the January 1986 Challenger disaster, the odds were 1 in 10."
The full study won't be released until later this year, but even these highlights are sobering. These are rocket scientists applying a very quantitative, rigorous risk analysis, but they still got it wrong. As the difference between the risk perceptions of the managers and engineers illustrates, often the assessment of risk can be influenced by different skills, roles and even motivation. Whenever risk relies on qualitative assessment, there's room for error.
So it's safe to assume that if NASA's best and brightest couldn't get their risk analyses to be completely accurate, we shouldn't assume that we're going to be able to assess risk perfectly, either. Instead, we should assume that there will always be more residual risk than we realize. That's another sign it's premature to abandon "traditional QC" and reduce our quality control checks to once a week or once a month check.
-----