General > General Technical Chat
The half a millimeter error that nearly cost 469 lives
ejeffrey:
--- Quote from: EPAIII on December 15, 2023, 08:56:37 am --- i was able to easily make parts with a tolerance of a thousandth of an inch or 0.025mm. And can/could measure them to a ten thousandth of an inch or 0.0025mm. An error of half a mm may sound small but, in the reality of a modern machine shop, is actually enormous. And RR was using the very best available machines to make these assemblies and then to measure/inspect them. And that error was made by some of what are supposedly the best engineers and machinists in the business.
I am not trying to sound like a pompous ass here. I too make mistakes, but I usually work alone with no back-up. It is just difficult to understand how so much can go wrong with so many experts involved.
--- End quote ---
Because the issue wasn't with the accuracy of the machines per se, but the ability to locate a new machining pass relative to a previous reference surface which is buried inside a partly assembled subassembly. Ultimately it was a failure of communication between the design enginerrs the manufacturing engineers, the tool manufacturers, and the factory supervisors. It's exactly the type of problem that doesn't happen with single person operations.
Presumably 99% of the original design was easily compatible with the available machining technology, but in a project this large 1% of hard to manufacture parts lead to either implementing extremely inefficient and potentially error prone manufacturing work arounds (as happened here) or seemingly endless redesign cycles with cascading impact.
To phrase the issue mathematically, the native communications overhead in a project scales as O(N^2). Left to itself, a project with more than about 10-20 people will result in output decreasing, eventually to near zero. Bureaucracy and project management are the attempt to fix that and, at great cost, make communication overhead O(N). But it always has failures where the right person doesn't get the right information and have the authority to act on it.
coppice:
--- Quote from: johansen on December 15, 2023, 10:30:06 pm ---
--- Quote from: coppice on December 15, 2023, 10:14:39 pm ---
--- Quote from: SiliconWizard on December 15, 2023, 10:03:08 pm ---
--- Quote from: johansen on December 15, 2023, 09:55:13 pm ---
--- Quote from: porter on December 15, 2023, 10:08:11 am ---Here is an interesting article about the safety of the 737 max. Another look into design decisions and quality control.
HOW THE BOEING 737 MAX DISASTER LOOKS TO A SOFTWARE DEVELOPER
--- Quote ---The flight management computer is a computer. What that means is that it's not full of aluminum bits, cables, fuel lines,
or all the other accoutrements of aviation. It's full of lines of code. And that's where things get dangerous.
Those lines of code were no doubt created by people at the direction of managers. Neither such coders nor their managers are as in touch with the particular culture
and mores of the aviation world as much as the people who are down on the factory floor, riveting wings on, designing control yokes, and fitting landing gears.
Those people have decades of institutional memory about what has worked in the past and what has not worked. Software people do not.
--- End quote ---
https://spectrum.ieee.org/how-the-boeing-737-max-disaster-looks-to-a-software-developer
--- End quote ---
simple fix for this. hold the software developers liable for their criminal negligence.
--- End quote ---
I don't disagree with that idea in theory, but in practice, as long as software developers will have subordinate relationships with their boss/employer (which is a majority of cases these days, especially in regulated fields), it's a Pandora's box. Any management screw-up will point fingers at some poor chap that may end up in jail, and the managers free and with a pay raise. Yes, I know it already happens occasionally, but without the "go to jail" part for the employee. So yeah, it would be much too easy to use this kind of law to twist liabilities.
--- End quote ---
Yep. The negligence is seldom attributable to one person. Some developers want to be thorough, and some don't, but very few are allowed the resources to be thorough. If management actually allows more resources, a lot of people relax and soak them up in an unproductive way. The more years you spend observing the dynamics of projects, the more amazing it is that anything bigger than a one person project ever gets finished and works well.
--- End quote ---
the negligence doesn't need to be one person.
the epa iirc is now up to 12 million per person, that's what the gov thinks your life is worth. that's the number they use to determine if pollution is worth cleaning up. multiply by 469 deaths and you get a 2 billion dollar settlement. given that 12 mill per person is more than the cost to jail a person for the rest of their life... no need for the legal system to get involved. the company can pay the fine and keep working, its probably a drop in the bucket anyways.
just make the companies pay for their negligence. if the cost of airplane flights go up.. so be it.
oh and prosecute the FAA too. they signed off on it and allowed it to fly. oh my bad they are a gov agency. sorry, do not pass go do not collect 200$.
--- End quote ---
Maybe we should just hold the entire human race responsible for creating the culture in which these things happen. Come on. You know you're all to blame, wanting cheap transport, and putting convenience above everything else. Just cauterise the planet and be done with it.
harerod:
--- Quote from: NiHaoMike on December 11, 2023, 11:59:32 pm ---
... The most unbelievable part was how several loosening of tolerances were just accepted in the aerospace industry. ...
--- End quote ---
Thanks for sharing this article. I may have flown on that very aircraft several times. The author is a bit too cheerful for my taste. Kudos to the crew and the overall aircraft design.
While it is important to look at the whole chain of fuck-ups that led to this accident, it is important to note that we have a component manufacturer who altered design documents and failed to comply with mandatory procedure in well over a hundred different cases.
Another wtf is the bit about a guy trying to figure out with a statistics tool (which he can't operate), how many defective items may be in service.
My clients sometimes feel that I act too friendly towards manufacturers. However, any manufacturer that changes my design without my approval, will lose the contract. He may get away with one stunt like this, but only if he convinces me that he will make sure this will not repeat.
To make that crystal clear: a manufacturer does not change designs. They give feedback to the designer, who may change the design.
Edit: typo
watchmaker:
Eventually, the design comes off the computer and is executed. Most times this involves a fallible human being.
Designs must be robust. The Ford Explorer/tire failure fiasco of 30 years ago made this crystal clear. Fatal rollovers because Firestone tires failed. Ford tried to blame it on the tires when the finding was that the vehicle was not designed to contend with a predictable (known) failure point. Tire failures.
I had tires rotated at my dealership. Driving at 70 MPH there was a thump like a piston about to go through the block. Felt nothing in the car. Stopped, looked it over and found nothing. Continued on. Started going through a diagnosis with my wife. Engine mount? Nothing I could do about that.
Speed related. Brake issue? No obvious change in braking. Gets worse on right hand turns? Bearing, axle? Get real loud!
Pull over and start looking for nearby tows. Wife looks at wheels and notices two missing lugnuts on front left.
The other three were hanging on by less than 25% thread. Tightened them down and went on my way.
Robust design. In fact, it would hold with one lug nut.
What scares me is I felt NOTHING thru the wheel. It is essentially steer by wire these days.
There are several parables here. It took us about 15 minutes to identify the problem and while that was too long, we are reasonably intelligent. In my defense, we were in a remote area (White Mountain National Forest) and the weather was getting lousy. An hour from home. Nothing was obviously explainable so we decided to limp along.
For completeness, in the US there is no liability for this unless we had actually rolled over. Courts have ruled that properly tightened lugnuts are not guaranteed. So I have a socket and wrench in the cars for checking them from now on.
I calmly asked the dealer to replace the lugnuts and suggested he turn this into a teachable moment for the mechanic (young). How would he feel when a fatal accident was traced back to him? Very cheap life lesson for the young mechanic.
Dewey
RJSV:
Ok but while reading this I see a logical disconnect:
One post starts by presenting aircraft safety but from a software developer's perspective. That could maybe involve some (natural) degree of ignorance, but that could be often remedied...at least in the traditional engineering office setting, meaning that the software folks get daily exposure to aeroscience issues.
That's fine, but then another post professes the remedy to be a 'get tough' blanket policy...that don't work, usually, you can't legislate or 'rule-make' to get rid of ignorance.
Navigation
[0] Message Index
[#] Next page
[*] Previous page
Go to full version