Author Topic: don't forget to reboot your boeing, or else  (Read 3540 times)

0 Members and 1 Guest are viewing this topic.

Offline peter-h

  • Super Contributor
  • ***
  • Posts: 3694
  • Country: gb
  • Doing electronics since the 1960s...
Re: don't forget to reboot your boeing, or else
« Reply #25 on: April 07, 2020, 05:44:35 am »
Airbus is not going to gloat too much because



All this shows is that those who write safety critical software are not supermen, and are probably no better than your nearest PHP hacker who has left SQL injection back doors all over the place :)

Probably the biggest thing that protects us is the fact that most of the systems are really old, perform very narrow functions (basically do the same thing every day; airline pilots tend to press the same buttons in the same order every time), and a lot of them (like jet engine ECUs) are carefully designed self contained boxes and then don’t get changed for many years.

Defects discovered by crew are reported internally, covered up immediately because the Daily Trash sharks are always circling, while the mfg sh**s itself and issues a rapid fix under the guise of an innocent sounding “feature update” so the certification agency never finds out ;)

One might think the reason Airbus are not making political capital out of the B737 saga is because they are upright and decent people, but in fact they are keeping quiet because they know their cupboard has just as many skeletons in it as all the others.

Many years ago, 1983, I designed a multizone heating controller which had self-learning optimisation on the advance turn-on. It was all written in Z80 assembler. Many many hundreds were sold and installed (c. £500 each) and no bugs were ever found. Well, all of them will have stopped working in 2012. The reason is that it computed the day of the week from the date (which came out of a clock chip) and the algorithm I used was just a lookup table which took advantage of the fact (?) that a calendar repeats every 29 years, and I never provided for any subsequent 29 year period. I mean, it’s obvious, in 1983 I was 26, and when you are 26 it is unthinkable that you will ever be 55 because nobody who is of any interest to a 26 year old is that old. I left the company (my first business) in 1991, and it went bust in 1993. Go figure, as they say…

I recently spoke to someone who worked on the Honeywell KFC225 autopilot, introduced into general aviation c. 2000 and dropped c. 2003 due to widespread servo burnout issues caused by defective autopilot computer software (believed to be integer maths underflow / dramatic loss of precision). The software had one file per function, because the FAA would push for recertification if more than x functions were changed. If you had 100 functions in one file then any edit of that file changed all 100 functions. The result was an almost unreadable source code…
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Online 2N3055

  • Super Contributor
  • ***
  • Posts: 6600
  • Country: hr
Re: don't forget to reboot your boeing, or else
« Reply #26 on: April 07, 2020, 06:39:59 am »
Boeing, what an enigma. They make bombs to kill 3rd world people, to help make rich people richer, and they make nice planes, overall.

Boeing has never made a bomb.
GBU-39 Small Diameter Bomb
 

Offline I wanted a rude username

  • Frequent Contributor
  • **
  • Posts: 627
  • Country: au
  • ... but this username is also acceptable.
Re: don't forget to reboot your boeing, or else
« Reply #27 on: April 07, 2020, 10:32:21 am »
 

Offline donotdespisethesnake

  • Super Contributor
  • ***
  • Posts: 1093
  • Country: gb
  • Embedded stuff
Re: don't forget to reboot your boeing, or else
« Reply #28 on: April 07, 2020, 12:22:02 pm »
Doesn't surprise me at all. Boeing have also screwed up with Starliner, but I see the same issues at other similar companies.

Many projects I have ended up asking "what happens when this timer counter wraps? Have we tested it?". Frequent answers are "don't worry it will not matter". Even if that is true at the time, later coders don't realise, and use it for a purpose where it does matter. We had a bug where random "EEPROM timeouts" were reported, turned out to be a false error due to a timer wrap.

Other frequent answer is "it's too difficult to test". Obviously testing a product for it's lifetime before release is impractical, so one has to be smart about identifying the timer limits and devising tests with special code if necessary.

We were recently requested to add a so-called "safety layer" for an aviation product, it adds a timestamp to each message to identify stale data. Obviously I immediately asked "what happens when...". The answer in this case, "it's not our problem, we've been told to implement according to the spec".

Y2K was a specific instance of a class of bugs, but those same bugs are everywhere in embedded software, even software supposed to be "safety critical". There is an assumption that "implementing to the spec" automagically means good code. Quality becomes a box ticking exercise, it's like a cargo cult. Coders and managers alike continue to operate in blissful ignorance, while the project slips further behind schedule.
Bob
"All you said is just a bunch of opinions."
 

Offline Tepe

  • Frequent Contributor
  • **
  • Posts: 572
  • Country: dk
Re: don't forget to reboot your boeing, or else
« Reply #29 on: April 07, 2020, 02:05:59 pm »
Boeing has never made a bomb.
Maybe not the part that actually goes bang but https://www.boeing.com/defense/weapons/
 

Offline eugenenine

  • Frequent Contributor
  • **
  • Posts: 865
  • Country: us
Re: don't forget to reboot your boeing, or else
« Reply #30 on: April 07, 2020, 02:35:40 pm »
"The FAA has received a report indicating that the stale-data monitoring function of CCS {common core system (CCS) – a Wind River VxWorks realtime OS product} may be lost when continuously powered on for 51 days"

Sounds like its technically not Boeing's fault here, it is the (VxWorks) OS they used.  What else uses VxWorks that could also be affected?  Airplanes, Rockets, medical devices?, nuclear launch controllers?
 

Offline coppercone2Topic starter

  • Super Contributor
  • ***
  • Posts: 9420
  • Country: us
  • $
Re: don't forget to reboot your boeing, or else
« Reply #31 on: April 07, 2020, 07:35:42 pm »
you can't blame a subassembly for a failure unless it happens after it passes testing and is released, otherwise you did not integrate it correctly IMO
 

Offline eugenenine

  • Frequent Contributor
  • **
  • Posts: 865
  • Country: us
Re: don't forget to reboot your boeing, or else
« Reply #32 on: April 07, 2020, 08:46:23 pm »
My point was more that its not just a Boeing subassembly, its used in other places so other things could have issues as well.
 

Offline coppercone2Topic starter

  • Super Contributor
  • ***
  • Posts: 9420
  • Country: us
  • $
Re: don't forget to reboot your boeing, or else
« Reply #33 on: April 07, 2020, 09:06:57 pm »
My point was more that its not just a Boeing subassembly, its used in other places so other things could have issues as well.

scary to think there is a missile silo or detection radar system some where that happens to do a generator test monthly and its not been found.. god forbid someone gets lazy. I wonder if the military is more prone to this (perhaps a UPS can be considered a reliability problem and they would rather tolerate a small power down for seconds on a generator, or budget cuts)? Also mobile things that setup temporarily but connect to a complex sensor grid might have this undetected, used for training frequently, etc (SAM stuff maybe?)
« Last Edit: April 07, 2020, 09:11:21 pm by coppercone2 »
 

Offline Psi

  • Super Contributor
  • ***
  • Posts: 9930
  • Country: nz
Re: don't forget to reboot your boeing, or else
« Reply #34 on: April 07, 2020, 09:14:30 pm »
Boeing, what an enigma. They make bombs to kill 3rd world people, to help make rich people richer, and they make nice planes, overall.

Boeing has never made a bomb.

Unless Wikipedia is wrong, they make rockets and missiles.
I'm not sure if they make the explosive payloads themselves, but even if they don't i think it's fair to say they make weapons of war.
Greek letter 'Psi' (not Pounds per Square Inch)
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf