Author Topic: Toyota firmware fail  (Read 24612 times)

0 Members and 1 Guest are viewing this topic.

Online mikeselectricstuff

  • Super Contributor
  • ***
  • Posts: 12135
  • Country: gb
    • Mike's Electric Stuff
Youtube channel:Taking wierd stuff apart. Very apart.
Mike's Electric Stuff: High voltage, vintage electronics etc.
Day Job: Mostly LEDs
 

Offline dr.diesel

  • Super Contributor
  • ***
  • Posts: 2189
  • Country: us
  • Cramming the magic smoke back in...
Re: Toyota firmware fail
« Reply #1 on: October 29, 2013, 09:32:24 pm »
Their piss poor debugging is the worst part, they didn't even try, pathetic..

Offline AwArD_RzD

  • Regular Contributor
  • *
  • Posts: 90
  • Country: ca
Re: Toyota firmware fail
« Reply #2 on: October 29, 2013, 09:32:52 pm »
I wonder how many bug are around me, it's a little scary to see big company skipping testing when human life is at risk. 
 

Offline AlfBaz

  • Super Contributor
  • ***
  • Posts: 2030
  • Country: au
Re: Toyota firmware fail
« Reply #3 on: October 29, 2013, 10:35:48 pm »
I've never understood why the convention to grow stacks down toward data areas and to top it off no simple mechanism to check stack pointer incursions to out of bound areas, whilst placing the stack at the top of ram and growing it up can give you a mem exception when you hit the wall <--- EDIT: oops, sorry for the pun  :)
 

Online edavid

  • Super Contributor
  • ***
  • Posts: 2988
  • Country: us
Re: Toyota firmware fail
« Reply #4 on: October 29, 2013, 10:53:52 pm »

http://www.edn.com/design/automotive/4423428/1/Toyota-s-killer-firmware--Bad-design-and-its-consequences

This article gives a laundry list of reasons why the code could have failed, but apparently no one knows if or how it did, so what's the point?  It's a legal rather than technical discussion.

 

Offline Bored@Work

  • Super Contributor
  • ***
  • Posts: 3932
  • Country: 00
Re: Toyota firmware fail
« Reply #5 on: October 29, 2013, 11:17:33 pm »
The article is a bit too sensationalistic for my liking. And it has some factual errors, like claiming OSEK is an RTOS. OSEK is a consortium specifying an API.

Another one is the rather stupid claim of
Quote
Anyone working with safe systems knows that single points of failure are to be avoided at almost any cost,

I was about to shout "Dude, anyone knows cars just have one accelerator pedal and one break pedal. Both are mechanical parts and single points of failure and are part of a safety system. Yet no one is avoiding them at almost any cost".
« Last Edit: October 29, 2013, 11:19:39 pm by Bored@Work »
I delete PMs unread. If you have something to say, say it in public.
For all else: Profile->[Modify Profile]Buddies/Ignore List->Edit Ignore List
 

Offline apelly

  • Supporter
  • ****
  • Posts: 1039
  • Country: nz
Re: Toyota firmware fail
« Reply #6 on: October 29, 2013, 11:19:32 pm »
It's a legal rather than technical discussion.

Not here!

That article points and appalling cascading failure chain. There is clearly a systemic problem in Toyota's R&D.

Given how shit it all turns out to be, why not just bloody open source it all. Is it really that secret? At least that will open the developers to public scrutiny; who wants to be publicly embarrassed by their shit design?
I'd rather a Google clue, link, or some theory than "do this" (generally)
 

Offline David_AVD

  • Super Contributor
  • ***
  • Posts: 2607
  • Country: au
Re: Toyota firmware fail
« Reply #7 on: October 30, 2013, 12:21:17 am »
We used to have a Holden Combo van (Opel / Vauxhall overseas) with an electronic throttle sensor.

One day I accidentally left the parking lights on overnight.  When I went back to the van I could hear a relay chattering due to the really low battery.

So, I jump started it and took it for a short drive to give it a bit of charge.  Part way down the street, I turned the A/C off so it has less load.

Well, that proved interesting.  The second the A/C was disengaged, the van started accelerating hard.  Foot off the pedal and it's still speeding up.   ???

After a few seconds I turned the A/C back on and it went back to normal operation.  I'm thinking WTF!!!

So, I take it back home, disconnect the battery for 30 minutes, connect it back up and no sign of the fault.
 

Offline Stonent

  • Super Contributor
  • ***
  • Posts: 3824
  • Country: us
Re: Toyota firmware fail
« Reply #8 on: October 30, 2013, 12:32:24 am »
I wonder if this will result in a recall and/or firmware rewrite for those vehicles?
The larger the government, the smaller the citizen.
 

Offline HackedFridgeMagnet

  • Super Contributor
  • ***
  • Posts: 1970
  • Country: au
Re: Toyota firmware fail
« Reply #9 on: October 30, 2013, 12:36:14 am »
I agree with Bored, it is very one sided and
Quote
single point of failure
wow, lets go back to biplanes.

The other thing is they have the source code but they haven't said they can reproduce the bug.
I am not saying the code was good or anything but if acceleration was caused by the firmware or the hardware or some combination of both then why cant they reproduce it?

And just because an jury of one nation found against a car manufacturer of another nation doesn't prove it in my eyes, especially something in such a technical niche such as automotive firmware.

 

Offline Dave

  • Super Contributor
  • ***
  • Posts: 1261
  • Country: si
  • I like to measure things.
Re: Toyota firmware fail
« Reply #10 on: October 30, 2013, 12:48:54 am »
But seriously, how difficult is it to stop a car with a "jammed" accelerator pedal? It's not like the brakes died (and even that shouldn't be a problem, if you are not on a steep slope or just about to turn into a bend).
If a driver is that incompetent, he/she shouldn't be driving in the first place.
<fellbuendel> it's arduino, you're not supposed to know anything about what you're doing
<fellbuendel> if you knew, you wouldn't be using it
 

Offline c4757p

  • Super Contributor
  • ***
  • Posts: 7805
  • Country: us
  • adieu
Re: Toyota firmware fail
« Reply #11 on: October 30, 2013, 12:51:13 am »
If a driver is that incompetent, he/she shouldn't be driving in the first place.

Welcome to America, where you get a driver's license for knowing what a stop sign is.

I seriously think that most of the people on the road here wouldn't be able to tell you what to do in that situation if you gave them time to think. While driving? Forget about it.
No longer active here - try the IRC channel if you just can't be without me :)
 

Offline Stonent

  • Super Contributor
  • ***
  • Posts: 3824
  • Country: us
Re: Toyota firmware fail
« Reply #12 on: October 30, 2013, 01:05:45 am »
If a driver is that incompetent, he/she shouldn't be driving in the first place.

Welcome to America, where you get a driver's license for knowing what a stop sign is.

I seriously think that most of the people on the road here wouldn't be able to tell you what to do in that situation if you gave them time to think. While driving? Forget about it.

What is this sign? Apparently I'm the only one left in this world that knows.



And also the purpose of those mysterious orange lights on the back of your car.
The larger the government, the smaller the citizen.
 

Offline Corporate666

  • Supporter
  • ****
  • Posts: 2007
  • Country: us
  • Remember, you are unique, just like everybody else
Re: Toyota firmware fail
« Reply #13 on: October 30, 2013, 01:23:40 am »
I agree with Bored, it is very one sided and
Quote
single point of failure
wow, lets go back to biplanes.

The other thing is they have the source code but they haven't said they can reproduce the bug.
I am not saying the code was good or anything but if acceleration was caused by the firmware or the hardware or some combination of both then why cant they reproduce it?

And just because an jury of one nation found against a car manufacturer of another nation doesn't prove it in my eyes, especially something in such a technical niche such as automotive firmware.

I agree with this.  Very one sided article and a bit sensationalist for my tastes... obviously looking to create a grand claim like Toyota is incompetent and made a big mess of the ECU.

The proof is in the pudding however - as far as I know, nobody has been able to reproduce the "unintended acceleration" problem.  Ever.  Nor has anyone been able to find a manner in which the hardware or software can malfunction and cause it.

IIRC, all vehicles brakes can overcome the acceleration of the engine, not to mention one can shut off the engine, go into neutral, etc, so it seems blaming these incidents on Toyota is sort of like blaming a shooting on the forge that made the metal that was used on the firing pin.
It's not always the most popular person who gets the job done.
 

Offline c4757p

  • Super Contributor
  • ***
  • Posts: 7805
  • Country: us
  • adieu
Re: Toyota firmware fail
« Reply #14 on: October 30, 2013, 01:31:30 am »
I was about to shout "Dude, anyone knows cars just have one accelerator pedal and one break pedal. Both are mechanical parts and single points of failure and are part of a safety system. Yet no one is avoiding them at almost any cost".

*cough* handbrake *cough*
*cough* shifter *cough*

Stopping and slowing are redundant. Remember that the handbrake usually actuates a completely separate set of brakes through a completely separate actuator.
No longer active here - try the IRC channel if you just can't be without me :)
 

Online nctnico

  • Super Contributor
  • ***
  • Posts: 19665
  • Country: nl
    • NCT Developments
Re: Toyota firmware fail
« Reply #15 on: October 30, 2013, 01:51:03 am »
I was about to shout "Dude, anyone knows cars just have one accelerator pedal and one break pedal. Both are mechanical parts and single points of failure and are part of a safety system. Yet no one is avoiding them at almost any cost".
Only the brakes and clutch are an all-mechanical system. The accelerator is just a pedal on a potmeter with a spring. It gets hairy on cars with an automatic gearbox. In those the software decides what happens if you brake. I once drove a (company) VW with an automatic gearbox in which there was a short between the brake lights and the normal lights. It made the car run like the fuel tank was as good as empty if/when I turned on the head lamps.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline orion242

  • Supporter
  • ****
  • Posts: 745
  • Country: us
Re: Toyota firmware fail
« Reply #16 on: October 30, 2013, 01:58:15 am »
How about....

1 Turn the ignition off!!!
2 Put it in neutral
3 Brakes
4 E brake

I think there is a PR hit job currently on Toyota.  Consumer reports just slammed them on crash safety also.  Funny many other cars also failed the same tests, yet only Toyota is all over the media.

Apparently they have not bought enough air time for commercials and this is the reminder that they may want to rethink that move.
 

Offline Bored@Work

  • Super Contributor
  • ***
  • Posts: 3932
  • Country: 00
Re: Toyota firmware fail
« Reply #17 on: October 30, 2013, 02:04:19 am »
I was about to shout "Dude, anyone knows cars just have one accelerator pedal and one break pedal. Both are mechanical parts and single points of failure and are part of a safety system. Yet no one is avoiding them at almost any cost".

*cough* handbrake *cough*
*cough* shifter *cough*

Stopping and slowing are redundant. Remember that the handbrake usually actuates a completely separate set of brakes through a completely separate actuator.

Nop kid. The handbrake is not designed, and can not, stop the car when it is driven at some speed. Easy to test. Keep the handbrake on and start driving. You can. Shifter? Engine breaking? No substitute for a real break. Your break pedal is a single point of failure in your single full-force break system in your car. All the other stuff is no real substitute.

And when an article like that EDN piece makes a big fuss about the qualification of the dude
Quote
As a primary expert witness for the plaintiffs, the in-depth analysis conducted by Barr and his colleagues illuminates a shameful example of software design and development, and provides a cautionary tale to all involved in safety-critical development, whether that be for automotive, medical, aerospace, or anywhere else where failure is not tolerable. Barr is an experienced developer, consultant, former professor, editor, blogger, and author.
then they should get their facts right, instead of blaring absolute bullshit around.
I delete PMs unread. If you have something to say, say it in public.
For all else: Profile->[Modify Profile]Buddies/Ignore List->Edit Ignore List
 

Online nctnico

  • Super Contributor
  • ***
  • Posts: 19665
  • Country: nl
    • NCT Developments
Re: Toyota firmware fail
« Reply #18 on: October 30, 2013, 02:08:08 am »
@orion242:
If you know anything about crashes then you'd probably know that most people barely manage to just press the brake pedal. That is only one out of four actions you suggest.

@bored@work:
Actually (almost) every car has a dual brake system. The master brake cylinder consists of two seperate cylinders which feed the brake calipers on different wheels. So worst case you'll have half the braking capacity. I wouldn't call that a single point of failure.

Braking on the engine is not to be underestimated. When driving downhill its the only viable option.
« Last Edit: October 30, 2013, 02:12:08 am by nctnico »
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline orion242

  • Supporter
  • ****
  • Posts: 745
  • Country: us
Re: Toyota firmware fail
« Reply #19 on: October 30, 2013, 02:10:05 am »
Its instinct to hit the brakes.

If the car kept moving the key would be next.
 

Offline c4757p

  • Super Contributor
  • ***
  • Posts: 7805
  • Country: us
  • adieu
Re: Toyota firmware fail
« Reply #20 on: October 30, 2013, 02:11:26 am »
Downshift to slow down, then apply the brake, of course. I agree, there is a single point of failure for full power braking, but you still always have an alternate way to stop the car.
No longer active here - try the IRC channel if you just can't be without me :)
 

Online nctnico

  • Super Contributor
  • ***
  • Posts: 19665
  • Country: nl
    • NCT Developments
Re: Toyota firmware fail
« Reply #21 on: October 30, 2013, 02:23:47 am »
Its instinct to hit the brakes.
It most certainly is not. Lots of people press the accellerator.
Quote
If the car kept moving the key would be next.
You'll crash before you get to that point. There simply isn't enough time. I strongly suggest to take a crash course. You'll see your reaction time is way worse than what you'd expect.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline Bored@Work

  • Super Contributor
  • ***
  • Posts: 3932
  • Country: 00
Re: Toyota firmware fail
« Reply #22 on: October 30, 2013, 02:33:27 am »
@bored@work:
Actually (almost) every car has a dual brake system. The master brake cylinder consists of two seperate cylinders which feed the brake calipers on different wheels. So worst case you'll have half the braking capacity. I wouldn't call that a single point of failure.

Braking on the engine is not to be underestimated. When driving downhill its the only viable option.

A single point of failure is that, a point, not the whole system. When a system has a single point of failure it means that when that single point fails the whole system fails. Redundancy of other system components don't help if the system depends on that single point.

The pedal as such is a single point of failure. You have no alternative to invoke and control full breaking power. If the pedal breaks, if it got stuck, if it fails in any way, you are screwed. You can no longer control your full breaking system.

The accelerator is just a pedal on a potmeter with a spring.

Like with the break pedal we can agree there is only one, and its proper function is relevant for safety and it is a single point of failure. Something this Barr dude claims
Quote
Anyone working with safe systems knows that single points of failure are to be avoided at almost any cost,

So now show me where in a car this single point of failure "is avoided at almost any cost"? Where is the redundant accelerator pedal or redundant break pedal?

What I want to point out is that this Barr dude was hired by the plaintiffs to support their case. He is not neutral, and of course he spun everything he legally could in the way the plaintiffs needed it. This "Anyone working with safety systems ..." is a statement where he spins the truth. And he manged to convince the judge/jury more than the defense managed. Not everything he says is absolutely true, and the article in EDN is public relations work to drum up more business for him (or, if you believe in a conspiracy against Toyota, to further damage Toyota).

You should also take into account that US courts love to sit with the plaintiffs when they are ordinary citizens and the defendant is a big corporation. Especially a foreign corporation.

Of course, the not so great, maybe even sad state of the Toyota code and their omissions helped that Barr dude to argue in favor of those who paid him. It doesn't mean he found out the absolute truth.
I delete PMs unread. If you have something to say, say it in public.
For all else: Profile->[Modify Profile]Buddies/Ignore List->Edit Ignore List
 

Offline orion242

  • Supporter
  • ****
  • Posts: 745
  • Country: us
Re: Toyota firmware fail
« Reply #23 on: October 30, 2013, 02:33:59 am »
Its instinct to hit the brakes.
It most certainly is not. Lots of people press the accellerator.

Sounds like mother nature sorting things out then.  How can "lots" of people even handle normal traffic signals...

Quote
If the car kept moving the key would be next.
You'll crash before you get to that point. There simply isn't enough time. I strongly suggest to take a crash course. You'll see your reaction time is way worse than what you'd expect.
I put on over 30K miles a year for work and have done so for 16 years.  Plenty of close calls, one crash, even watched fatal a crash right next to me over that time.  NEVER have I hit the gas in a panic.  Its always been black trails leading to my tires and the smell of burning rubber with dust settles.
 

Offline Dave

  • Super Contributor
  • ***
  • Posts: 1261
  • Country: si
  • I like to measure things.
Re: Toyota firmware fail
« Reply #24 on: October 30, 2013, 03:32:49 am »
Only the brakes and clutch are an all-mechanical system. The accelerator is just a pedal on a potmeter with a spring.
Actually, no.
The modern accelerator pedal uses a hall effect sensor and a metal slope that moves in front of the sensor to detect the position of the pedal. Well, not one, but two individual sensors that feed the pedal position information to the ECU. One signal is twice as large as the other (the amplitudes), so in case they are not in correct proportion, the ECU knows something went wrong.
It also has two springs, a large main spring and a smaller backup, that's located inside the large spring.

In case one of the sensors dies, the dashboard will let you know. In case one of the springs break, you will definitely feel it under your foot. In both cases you will still have a fully operational pedal, and you can safely drive to a mechanic.

Shifter? Engine breaking? No substitute for a real break.
You can actually bring a car to a very low speed (walking pace) with downshifting and engine braking alone. Stopping the car at that point isn't too difficult.
But you do need the distance to be able to do that. If your whole brake system fails just as you are approaching a bend at speed, you are well and truly f*cked.
<fellbuendel> it's arduino, you're not supposed to know anything about what you're doing
<fellbuendel> if you knew, you wouldn't be using it
 

Offline Harvs

  • Super Contributor
  • ***
  • Posts: 1184
  • Country: au
Re: Toyota firmware fail
« Reply #25 on: October 30, 2013, 04:12:59 am »
As a motorcyclist, a failure in a system like this makes hairs stand on end.  It's one of the reasons bikes have (generally) always had dual cables for the throttle, so it wouldn't just return under spring tension.

But as with cars, the trend is slowly to move over to drive by wire throttle systems.  This has allowed some even more stupidly high cam tuning, as it allows the ECU to make the bike (engine) somewhat drivable at low rpm.

However, imagine being on a new R1 et al and the throttle suddenly slams open uncommanded.  Yes you've got a clutch right by your left hand, but if you're in first or second you'll probably have the front wheel in the air and be at >200kph in a few sec time.  Chances of you hitting something hard before getting it under control are very good.
 

Offline David_AVD

  • Super Contributor
  • ***
  • Posts: 2607
  • Country: au
Re: Toyota firmware fail
« Reply #26 on: October 30, 2013, 04:21:36 am »
Speaking as someone who's had that "UA" experience, my first instinct was to go for the brake.

The very next thought was to turn the ignition off (to accessories position, not lock!), but then I recognised that the UA was linked to switching the A/C off, so switching it back on averted more drastic immediate action.

In hindsight I could have also popped it into neutral and let the rev limiter kick in while I braked to a stop.  Luckily there was nobody close in front of me at the time.  If it had happened in heavy traffic there might have been a small collision.
 

Offline walshms

  • Regular Contributor
  • *
  • Posts: 183
  • Country: us
Re: Toyota firmware fail
« Reply #27 on: October 30, 2013, 04:30:37 am »
As a motorcyclist, a failure in a system like this makes hairs stand on end.

As a fellow motorcyclist, and former MSF Ridercoach, I can tell you that training is the only way to actually protect yourself.  Developing good habits, and experience, would save you in that case.  TCLOCS was something I stressed in classes, and understanding the machine thoroughly arms you for the unexpected.

You can downshift without a clutch, though it'll be a rough ride, and if you've got one of the more recent bikes, your brakes will function down to the point where they're nearly empty of fluid.  If you've done your pre-ride inspection properly, you're probably okay.
 

Offline walshms

  • Regular Contributor
  • *
  • Posts: 183
  • Country: us
Re: Toyota firmware fail
« Reply #28 on: October 30, 2013, 04:39:01 am »
Oh, and of course, that handy-dandy engine cutoff switch right there by your right thumb.  ;)
 

Offline SeanB

  • Super Contributor
  • ***
  • Posts: 15387
  • Country: za
Re: Toyota firmware fail
« Reply #29 on: October 30, 2013, 04:59:00 am »
Not helped by the "Oh we made this thing keyless so you do not have the inconvenience of needing to find the slot just press this button" who decided that you need to hold the button for 5 seconds to switch off, and also decided that if you press the button while in motion it will be disregarded as an accidental press.
 

Offline andtfoot

  • Supporter
  • ****
  • Posts: 352
  • Country: au
Re: Toyota firmware fail
« Reply #30 on: October 30, 2013, 05:13:51 am »
Not helped by the "Oh we made this thing keyless so you do not have the inconvenience of needing to find the slot just press this button" who decided that you need to hold the button for 5 seconds to switch off, and also decided that if you press the button while in motion it will be disregarded as an accidental press.

I wonder what would happen if you threw the wireless key fob thingie out the window while driving...
 

Offline Stonent

  • Super Contributor
  • ***
  • Posts: 3824
  • Country: us
Re: Toyota firmware fail
« Reply #31 on: October 30, 2013, 05:18:13 am »
Not helped by the "Oh we made this thing keyless so you do not have the inconvenience of needing to find the slot just press this button" who decided that you need to hold the button for 5 seconds to switch off, and also decided that if you press the button while in motion it will be disregarded as an accidental press.

I wonder what would happen if you threw the wireless key fob thingie out the window while driving...

That would be interesting to try if you were on a street with no traffic, toss it out the window into the grass and see if it will keep going. I suspect it might because it would just waste the battery to keep in contact with it.
The larger the government, the smaller the citizen.
 

Offline Harvs

  • Super Contributor
  • ***
  • Posts: 1184
  • Country: au
Re: Toyota firmware fail
« Reply #32 on: October 30, 2013, 06:38:13 am »
That would be interesting to try if you were on a street with no traffic, toss it out the window into the grass and see if it will keep going. I suspect it might because it would just waste the battery to keep in contact with it.

Or if it was your car, you could just pass it out the window to a friend :)
 

Offline chickenHeadKnob

  • Frequent Contributor
  • **
  • Posts: 874
  • Country: ca
  • doofus programus semi-retiredae
Re: Toyota firmware fail
« Reply #33 on: October 30, 2013, 06:46:57 am »
 I can recall when the Prius runaway problem hit the news Steve Wozniak chimed in and claimed his prius was affected, Here is a youtube clip:
http://youtu.be/u44XjkWgFac    Read some of the comments, it seems Woz was confused by the cruise control.
 I partially remember reading there was a strong driver age bias (over 60 years old) to the cohort of un-commanded accelleration complainers.

In the early 80's when GM was a ghung-ho  early adopter of microprocessors  everywhere I heard about a software bug in one of the Cadillac models. When accelerating uphill or under load and transitioning through the middle gears - auto trans, if the driver then honked the horn the engine would suddenly lose power. :-DD

Diesel engines are reluctant to shut down once they are running, I think the standard design is to cut the fuel supply to the common fuel rail, but sometimes even that isn't sufficient. VW's first generation 4 cylinder engine installed in Rabbits in the mid 70's replicated a noob design flaw also exhibited by a few early american truck engines in 50's. They would, on rare occasion, cannilbalize  their own crankcase oil as fuel if the cylinders or rings were worn. When that happened the engine would race to 6000 rpm and seize in short order, but not before leaving the driver with an intense and bewildering experience, no software required.
 

Offline Stonent

  • Super Contributor
  • ***
  • Posts: 3824
  • Country: us
Re: Toyota firmware fail
« Reply #34 on: October 30, 2013, 08:19:28 am »
2 stroke Detroit diesel engines would eat their own oil sometimes. They used to make a kit you could install where if the suction on the intake got high enough due to overspeed or runaway, it would suck a trap door closed over the intake pipe and cut off all air.
The larger the government, the smaller the citizen.
 

Offline andersm

  • Super Contributor
  • ***
  • Posts: 1148
  • Country: fi
Re: Toyota firmware fail
« Reply #35 on: October 30, 2013, 08:35:25 am »
Normally people here are ready to jump on anything and call it terrible, but now that there's evidence of bad design the board suddenly turns into the Toyota defense force?

Offline Stonent

  • Super Contributor
  • ***
  • Posts: 3824
  • Country: us
Re: Toyota firmware fail
« Reply #36 on: October 30, 2013, 08:41:19 am »
That would be interesting to try if you were on a street with no traffic, toss it out the window into the grass and see if it will keep going. I suspect it might because it would just waste the battery to keep in contact with it.

Or if it was your car, you could just pass it out the window to a friend :)

Doh! Much easier!
The larger the government, the smaller the citizen.
 

Offline dr.diesel

  • Super Contributor
  • ***
  • Posts: 2189
  • Country: us
  • Cramming the magic smoke back in...
Re: Toyota firmware fail
« Reply #37 on: October 30, 2013, 11:00:45 am »
Normally people here are ready to jump on anything and call it terrible, but now that there's evidence of bad design the board suddenly turns into the Toyota defense force?

Agreed.

While there is always two sides of every story, it's likely that Toyota dropped the ball on this one.  I imagine Toyota is similar to most other huge corporate slugs, cost cutting till it starts to hurt, or failing to invest where necessary.  Current gen ECMs are a crucial part of the vehicle and probably haven't received the time/money/development necessary.

Offline grumpydoc

  • Super Contributor
  • ***
  • Posts: 2740
  • Country: gb
Re: Toyota firmware fail
« Reply #38 on: October 30, 2013, 11:02:07 am »
Quote
By the time memory exceptions came in we had MMUs anyway, which could do the same thing with a downward growing stack.
The difficulty here is that to handle the exception typically the first thing you need to do is extend the stack......

I have a problem with safety critical code and "exceptions" anyway - do you really want safety critical code to be saying "oops, didn't expect that"
 

Offline AlfBaz

  • Super Contributor
  • ***
  • Posts: 2030
  • Country: au
Re: Toyota firmware fail
« Reply #39 on: October 30, 2013, 11:29:12 am »
I have a problem with safety critical code and "exceptions" anyway - do you really want safety critical code to be saying "oops, didn't expect that"
Code: [Select]
if(oops_didnt_expect_that)
ShutDownEngine();
Surely better than having application data obliterated obliviously
 

Offline dr.diesel

  • Super Contributor
  • ***
  • Posts: 2189
  • Country: us
  • Cramming the magic smoke back in...
Re: Toyota firmware fail
« Reply #40 on: October 30, 2013, 11:33:37 am »
I have a problem with safety critical code and "exceptions" anyway - do you really want safety critical code to be saying "oops, didn't expect that"
Code: [Select]
if(oops_didnt_expect_that)
ShutDownEngine();
Surely better than having application data obliterated obliviously

What if this happens when your attempting to get off the train tracks, or get out of the way of a semi?  Eventually this condition will happen and the ECM must be prepared for it, exception or not.

Offline amyk

  • Super Contributor
  • ***
  • Posts: 6808
Re: Toyota firmware fail
« Reply #41 on: October 30, 2013, 11:35:21 am »
I agree with those saying this is sensationalistic -- nowhere in the article does it say they actually found the cause, just some questionable marginal design (which I am almost willing to bet would be the same with various other manufacturers.)
Quote
Toyota claimed only 41% of the allocated stack space was being used. Barr's investigation showed that 94% was closer to the truth.
94% is still <= 100%. If they could prove that it never got over 100%, then that's not a point of failure to me.

My car's ECU only runs the fuel injection and the throttle is mechanical. I once drove a friend's recent "drive-by-wire" car and the sensation was a bit disturbing. At times the accelerator pedal felt like it was completely disconnected and the car had a mind of its own (which it does...)
 

Offline Dr. Frank

  • Super Contributor
  • ***
  • Posts: 1874
  • Country: de
Re: Toyota firmware fail
« Reply #42 on: October 30, 2013, 11:45:34 am »
The report in EDN is very strange in several  aspects:

1. Toyota is a car manufacturer, they do not (to my knowledge) design and manufacture car electronics.
The real supplier of the ECU is not named - strange.

2. The NASA had already analysed the software of this ECU, in a previous trial, and found no such bugs.
The driver was blamed of maloperation in first instance.

3. Such a relatively small and perhaps unexperienced  (in terms of automotive business or saftey design) SW engineer group should really find bugs, which the NASA team did not see??

4. All Tier 1 suppliers for OEMs have to follow strictly on design processes for Hardware and Software, which are standard in the automotive industry, e.g. according to CMMI and other models. A supplier, which does not conform to such requirements, would not work for OEMs, especially not for Toyota, which is a super-critical customer for electronic manufacturers.

Toyota normally audits those suppliers very strictly, especially on saftey electronics.

I can hardly believe, that such a safety component should not have been designed with extra effort,  and care in the usual validation tests.

And I also doubt, that such severe software design errors, or the lack of solid design rules and a lack of a sophisticated software system should be present in this case.


Frank
« Last Edit: October 30, 2013, 11:48:00 am by Dr. Frank »
 

Online 0xdeadbeef

  • Super Contributor
  • ***
  • Posts: 1459
  • Country: de
Re: Toyota firmware fail
« Reply #43 on: October 30, 2013, 12:37:11 pm »
Yeah, the EDN article is weird in many ways. Claiming that ECC RAM (never heard the term "EDAC" anywhere before) was state of the art in an ECU ("ECM") from 2005 (if I figure this right) seems a bit weird. E.g. Tricore controllers had only simple parity check until recently and some years ago even this had to be disabled for some models due to according errata.
I refuse to believe that no stack measurement was done, as this is so stupidly easy to do (initialize with stack pattern, check where from the top the first pattern was overwritten) that it was done in every project I ever saw in the last 15 years. And static stack analysis is kinda futile as it will always result in catastrophically high stack utilization. Besides, it's kinda hard to believe that a stack overflow resulted in acceleration instead of a reset.

Other stuff like the paradigm of "mirroring all important data" is questionable at least. Apart from the runtime hit when mirroring all important variables and consistency issues: what should you do if the main value and the mirrored value differ? Which one would you trust? Stuff like this was used for NVRAM or reset safe values in the past (instead of proper CRC), but just to detect inconsistency and return to a default value. For inconsistent runtime variables, you could only cause a reset from my point of view. Anyway, monitoring variables instead of monitoring functions seems like a bad idea. At least I'm not aware that this is part of any automotive safety concept. And probably for a good reason.

MISRA-C rule violations don't mean anything without further details. MISRA is very strict and a lot of rules are questionable at best. Anyway, a MISRA violation doesn't mean that the code is not working correctly. It can be just a "wrong" C++ comment, a missing (needless) cast, a goto (even if it's the best solution in some cases, e.g. leaving multiple loops) or other formal stuff.

Indeed it's not clearly described what exactly caused the task to stop. My impression is that this was neither caused by a stack overflow nor by a bit fault in SRAM, nor due to MISRA violations.

Last but not least, wrong behavior due to SW bugs or HW issues (e.g. CPU errata) can never be ruled out, even in the most safety relevant systems  and when following all possible processes and regulations. Therefore there has to be a safety concept on the system side which avoids critical behavior - and unintended acceleration is the top candidate there. In every ECU I ever saw there was a secondary controller checking the safety relevant paths. E.g. monitoring the pedal sensor values, comparing them to the ones reported by the CPU and shutting off the engine (or performing a reset) in case of severe misbehavior.
So it seems that Toyota (or the according supplier) hasn't implement a valid safety concept. This is the main issue, not bugs in the software or whatever.
« Last Edit: October 30, 2013, 03:44:35 pm by 0xdeadbeef »
Trying is the first step towards failure - Homer J. Simpson
 

Online nctnico

  • Super Contributor
  • ***
  • Posts: 19665
  • Country: nl
    • NCT Developments
Re: Toyota firmware fail
« Reply #44 on: October 30, 2013, 04:12:43 pm »
I have a problem with safety critical code and "exceptions" anyway - do you really want safety critical code to be saying "oops, didn't expect that"
Code: [Select]
if(oops_didnt_expect_that)
ShutDownEngine();
Surely better than having application data obliterated obliviously

What if this happens when your attempting to get off the train tracks, or get out of the way of a semi?  Eventually this condition will happen and the ECM must be prepared for it, exception or not.
I agree. The last thing you want is the engine to shutdown completely. My car has a protection against loading the engine too much when its cold. I had it kick in once when I wanted to overtake some cars. Very annoying and besides that the way it kicks in produces an enormous cloud of black smoke behind the car scaring the shit out of people driving behind.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline Rufus

  • Super Contributor
  • ***
  • Posts: 2094
Re: Toyota firmware fail
« Reply #45 on: October 30, 2013, 05:32:09 pm »
I partially remember reading there was a strong driver age bias (over 60 years old) to the cohort of un-commanded accelleration complainers.

I had an old lady drive into my garden a few years ago, though a brick wall. She set off from about 40 feet away and I heard a racing engine, screech of tires followed shortly by a loud crunch.

If you think you are pressing the brake and the car doesn't slow down a natural reaction is to press it harder.
 

Offline SeanB

  • Super Contributor
  • ***
  • Posts: 15387
  • Country: za
Re: Toyota firmware fail
« Reply #46 on: October 30, 2013, 06:51:39 pm »
IIRC an exam of a TPS sensor did show the growth of tin whiskers that led to a shorting of the wiper to the one side of the pot assembly.
 

Offline Neilm

  • Super Contributor
  • ***
  • Posts: 1455
  • Country: gb
Re: Toyota firmware fail
« Reply #47 on: October 30, 2013, 06:55:01 pm »
Not helped by the "Oh we made this thing keyless so you do not have the inconvenience of needing to find the slot just press this button" who decided that you need to hold the button for 5 seconds to switch off, and also decided that if you press the button while in motion it will be disregarded as an accidental press.

I wonder what would happen if you threw the wireless key fob thingie out the window while driving...

I've not thrown one out a window, but I was driving my parents Prius which has one. The key was in Mums handbag which she picked up when she got out the car. When she was about 3 foot away the car started complaining that it could not detect the key.
Two things are infinite: the universe and human stupidity; and I'm not sure about the the universe. - Albert Einstein
Tesla referral code https://ts.la/neil53539
 

Offline andersm

  • Super Contributor
  • ***
  • Posts: 1148
  • Country: fi
Re: Toyota firmware fail
« Reply #48 on: October 30, 2013, 07:36:10 pm »
There was an interesting link posted in the article's comments: http://www.japanfocus.org/-David-McNeill/3993

Offline HackedFridgeMagnet

  • Super Contributor
  • ***
  • Posts: 1970
  • Country: au
Re: Toyota firmware fail
« Reply #49 on: October 31, 2013, 12:31:22 am »
Quote
The Camry ETCS code was found to have 11,000 global variables. Barr described the code as “spaghetti.” Using the Cyclomatic Complexity metric, 67 functions were rated untestable (meaning they scored more than 50). The throttle angle function scored more than 100 (unmaintainable).

11,000 global variables WTF, that cant be true, I wonder if they are counting every byte in an array as a global?

Was it writen in C anyone know?

I looked up Cyclomatic Complexity, it seems an odd way to measure complexity as a switch statement with 50 cases would seem to rate as untestable. Tell me if I am wrong.

Quote
On top of that, stack-killing, MISRA-C rule-violating recursion was found in the code, and the CPU doesn't incorporate memory protection to guard against stack overflow.
Recursion doesn't seem like a good idea in embedded software. But the stack was shown to only use 94% of available, the recursion must have been limited somehow.



 

Online 0xdeadbeef

  • Super Contributor
  • ***
  • Posts: 1459
  • Country: de
Re: Toyota firmware fail
« Reply #50 on: October 31, 2013, 02:13:59 am »
11,000 global variables WTF, that cant be true, I wonder if they are counting every byte in an array as a global?
This is typical in automotive software. The calibration guys want to look at every internal variable for validation and calibration. To be able to export variables to the calibration tool, they have to be defined global as statics are not listed in the map file (and might be decorated in the debug info). So global (in C sense) doesn't necessarily mean that these variables were really used globally. Then again, using global variables as interfaces between functionalities is unfortunately also pretty common in automotive software - at least in the higher levels. Partly because it's very effective and partly because all the global variables exist anyway (for calibration).

Was it writen in C anyone know?
I would think so. Automotive software is usually either written in C or autocoded (resulting in C code again).

Quote
Recursion doesn't seem like a good idea in embedded software. But the stack was shown to only use 94% of available, the recursion must have been limited somehow.
Yeah, recursion would be pretty bad. Even though recursion should not mean "endless recursion". Anyway, you'd need to see the exact code to judge how bad this really was. MISRA violation says nothing at all.
Trying is the first step towards failure - Homer J. Simpson
 

Offline N TYPE

  • Regular Contributor
  • *
  • Posts: 57
  • Country: au
Re: Toyota firmware fail
« Reply #51 on: October 31, 2013, 03:52:57 am »
I'm pretty sure it's a Japanese company DENSO who handles all of the ecu's and electronics for Toyota. they have done so for decades.
 

Offline N TYPE

  • Regular Contributor
  • *
  • Posts: 57
  • Country: au
Re: Toyota firmware fail
« Reply #52 on: October 31, 2013, 03:59:33 am »
I'm pretty sure it's a Japanese company DENSO who handles all of the ecu's and electronics for Toyota. they have done so for decades.
Having said that, DENSO are responsible for the Japanese Domestic market. They normally use DENSO branded chips which are typically just customized brand-name micro's, restamped with denso's logo.
 I notice this particular case its from an American Prius which the ECU could have been developed by the americans to meet their own emissions standards and safety laws etc etc, considering they're using NEC parts rather than DENSO, it wouldnt surprise me if this were the case.
 

Offline G0HZU

  • Super Contributor
  • ***
  • Posts: 2576
  • Country: gb
Re: Toyota firmware fail
« Reply #53 on: November 01, 2013, 05:13:44 pm »
Quote
1. Toyota is a car manufacturer, they do not (to my knowledge) design and manufacture car electronics.
The real supplier of the ECU is not named - strange.

Google for NipponDenso or just Denso. They are owned by Toyota and have been making engine management systems for Toyota vehicles for decades.

I can't say if the ECU in question was made by ND but I can say that I have a LOT of experience in reverse engineering Toyota ECUs of the 1980s and 1990s and they were all made by Denso. Note: The image of the ECU in the article looks like it has a Denso 'D' logo on one of the chips but this logo is newer than the one I'm used to seeing so I can't be certain.

One thing that impressed me a lot about Denso ECUs of the 80s and 90s was the attention to detail in terms of safety and also the quality and efficiency of the coding.

You could also access memory addresses via a simple serial interface which allowed the user to log all aspects of the ECU whilst it was being developed. So looking at the memory used /not used by the stack was possible whilst it was being driven. Also all of the variables and status flags could be logged with several hundred variables logged a second.

I learned an awful lot about efficient coding from analysing Denso's code. I guess that modern ECUs will be far more complex and will be written in a high level language so bugs are far more likely. I looked at dozens of Denso ECUs and only ever found a couple of minor bugs.



« Last Edit: November 01, 2013, 05:26:34 pm by G0HZU »
 

Online nctnico

  • Super Contributor
  • ***
  • Posts: 19665
  • Country: nl
    • NCT Developments
Re: Toyota firmware fail
« Reply #54 on: November 01, 2013, 07:16:05 pm »
Until the early 2000's Toyota's cars where rock solid. After that it went downhill. For example they have had huge problems with their diesel engines which got so bad that Toyota buys diesel engines from BMW nowadays. Picture this: the biggest car manufacturer in the world suddenly can't design a reliable diesel engine. A reliable/durable diesel engine was one reason to buy a Toyota for many people. I guess at some point the good people in the engineering department left or retired.
« Last Edit: November 01, 2013, 07:17:42 pm by nctnico »
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline Stonent

  • Super Contributor
  • ***
  • Posts: 3824
  • Country: us
Re: Toyota firmware fail
« Reply #55 on: November 01, 2013, 08:13:11 pm »
I think Continental and Siemens are some ecu brands I've seen in vehicles that were labeled.  Many just have the car brand.
« Last Edit: November 02, 2013, 02:37:24 am by Stonent »
The larger the government, the smaller the citizen.
 

Online 0xdeadbeef

  • Super Contributor
  • ***
  • Posts: 1459
  • Country: de
Re: Toyota firmware fail
« Reply #56 on: November 01, 2013, 10:05:10 pm »
Siemens Automotive was bought by Continental some years ago (which didn't make ECUs before). So they're essentially one brand regarding ECUs.
Other big players are Bosch, Delphi and Magneti Marelli.
« Last Edit: November 01, 2013, 10:07:02 pm by 0xdeadbeef »
Trying is the first step towards failure - Homer J. Simpson
 

Offline Stonent

  • Super Contributor
  • ***
  • Posts: 3824
  • Country: us
Re: Toyota firmware fail
« Reply #57 on: November 02, 2013, 02:38:11 am »
Siemens Automotive was bought by Continental some years ago (which didn't make ECUs before). So they're essentially one brand regarding ECUs.

The first time I saw that I thought "Don't they make tires?"
The larger the government, the smaller the citizen.
 

Online 0xdeadbeef

  • Super Contributor
  • ***
  • Posts: 1459
  • Country: de
Re: Toyota firmware fail
« Reply #58 on: November 02, 2013, 10:38:31 am »
The first time I saw that I thought "Don't they make tires?"
They do ;) But they had a much larger product range even before they bought Siemens VDO Automotive.
Trying is the first step towards failure - Homer J. Simpson
 

Offline Andreas

  • Super Contributor
  • ***
  • Posts: 2545
  • Country: de
Re: Toyota firmware fail
« Reply #59 on: November 02, 2013, 03:00:43 pm »
I once drove a (company) VW with an automatic gearbox in which there was a short between the brake lights and the normal lights. It made the car run like the fuel tank was as good as empty if/when I turned on the head lamps.

Congratulations: you have successfully tested one of the features of the monitoring concept.
If brake is pressed (brake light switch active) while the accelerator is pressed it is assumed
that the accelerator spring is broken and the pedal stucks.
In this case engine torque is reduced by the brake signal.

4. All Tier 1 suppliers for OEMs have to follow strictly on design processes for Hardware and Software, which are standard in the automotive industry,

The processes are the one, but the opinions on how a car has to work may be different in other countries.
Do not forget: here in Germany (and maybe Europe) due to the VDA guidelines ("EGAS Lastenheft") we have
a relative good standard regarding UA in hardware and in software.

http://www.iav.com/sites/default/files/attachments/seite/ak-egas-v5-5-en-130705.pdf


But often hardware restrictions and driver behaviour may compromize the standards.
Just 2 (of course freely invented) examples:
1.  a car manufacturer wants to have implemented a cruise control software in a elder generation ECU for a model year upgrade. But the car has no redundant brake switch which is mandatory (according to VDA) for this. But how to convince the overseas manufacturer to implement a better break pedal (and of course a new cable harness).

 I don´t know how this would end. But probably the manufacturer signs a weaver and takes the responsibility so the Tier1 will deliver a ECU with cruise control for only 1 brake switch.

2. There may be different opinions between american car manufacturers and european deliverers regarding "two footed driving"

At least if I believe a EE of a american car manufacturer the normal way to drive a american car is to put the right foot on the accelerator and the left on the brake pedal (with automated gear). The brake light switch might be always on during this procedure. And maybe the redundant brake switch is still unactivated due to switch tolerances leading to plausibility error between the switches.

From monitoring side engine torque and cruise control have to be switched off when only one switch is activated. (Because the other could be defective). Further depending on debouncing strategy after some debouncing time or some of those implausibility events a fault code will be entered in fault memory switching all comfort functions off.
From OEM side this behavior is unwanted regarding two footed driving.
So will the OEM spend more money for a additional brake pressure sensor?
Or even a better pedal with analog sensors like in brake assistant?

Other stuff like the paradigm of "mirroring all important data" is questionable at least. Apart from the runtime hit when mirroring all important variables and consistency issues: what should you do if the main value and the mirrored value differ? Which one would you trust?

According to the VDA rules there is much more than just only mirroring important data.
You have a 3-Level concept:

Level 1 is the standard application software with the normal torque path. Usually no mirrroring of data.

Level 2 is the functional monitoring software with a simplified model of the torque path.
If the torque in level 2 is lower than in level 1  (with some percent offset) then the torque in level 1 is limited.
In Level 2 each variable has a cyclical RAM check and either a complementary storage or a checksum.
The question which one is to trust is simple: neither. The ECU does the same what you would do with a hanging computer: simply press the reset button. If the error persists either ECU or external watchdog will shut off all power stages finally.

Level 2' is the same as Level 2 but in this case fixed sensor values out of arrays are used for calculation (giving predefined final results) to prove that the processors ALU is still calculating in a right manner. The result is not compared directly but fed together with other tests to the final response for the external watchdog.

Level 3 monitors the hardware and the external watchdog. The watchdog asks questions to the CPU and shuts down the power stages if  the response is either wrong or outside of the time window. The response is calculated as result which depends on the execution and order of the level 2 routines.

With best regards

Andreas


 
 

Offline Dr. Frank

  • Super Contributor
  • ***
  • Posts: 1874
  • Country: de
Re: Toyota firmware fail
« Reply #60 on: November 02, 2013, 03:28:27 pm »
The first time I saw that I thought "Don't they make tires?"
They do ;) But they had a much larger product range even before they bought Siemens VDO Automotive.

That's right. We have undergone many steps of metamorphosis in the past 15 years:

It started with VDO (instrument clusters /*/ and other electronics).
Then we became Mannesmann - (seamless steel pipes)  VDO + Kienzle (electronics / mechanics for vans + mobile license) + Philips Car Systems (radio + navigation).

Then the ww. stock bubble was initiated in 1999, when a much smaller company (for the first time ever) bought Mannesmann: Vodafone wanted to acquire the mobile license.

So we belonged a short time to vodafone, being named ATECS.

Shortly after that, we were sold to Siemens Automotive.

VDO and Siemens both made ECUs already.

Then, Continental (tyres) bought SiemensVDO in 2007, but they already had : Teves (brake systems), Motorola Automotive (telematics) and ContiTech (rubber belts).

Then in 2008 Schaeffler KG (ball-bearings) tried to take over Conti, but in the end, they suffered greatly from the finance crisis in 2009/2010 and had to sell again parts of their shares.

So, without changing job, in the last 16 years at VDO we had at least 6 different owners or company names, and the number of people increased from about 15.000 to now over 175.000. Some times, it was quite an uncomfortable situation for the people, due to many changes, but it was also very exciting to experience / take part in all those business migrations.


Well, from the photograph in the EDN article, the Toyota engine control electronics perhaps is produced by Delphi, as they call theirs ECM also, and it's called ECU otherwise.

http://delphi.com/manufacturers/auto/powertrain/gas/ecm/
Those units also feature ETC.

And this company was already mentioned in connection with this Toyota incident.

Also, I did not see the typical Conti imprint or code numbers on the PCB.

So, I feel relieved that it's obviously not our company to blame, as Toyota will for sure charge the supplier for the 3M$, if the SW really is as badly engineered as described (which I really doubt at the moment).
For sure there will also be a big mess for the complete series production, as this judgement will cause further trials in US, further compensation payments, and worst case a massive call-back.

If latter event happens, this might ruin the supplier.

Frank


/*/ 111 years of Tachometer
« Last Edit: November 02, 2013, 03:43:47 pm by Dr. Frank »
 

Online 0xdeadbeef

  • Super Contributor
  • ***
  • Posts: 1459
  • Country: de
Re: Toyota firmware fail
« Reply #61 on: November 02, 2013, 05:41:26 pm »
Quote
According to the VDA rules there is much more than just only mirroring important data.
You have a 3-Level concept:
Yeah, but the article didn't talk of a safety concept at all but claimed that every "important" data - including RTOS task activation bits - should be mirrored.
This is nonsense in my point of view as it completely misses the point.
Trying is the first step towards failure - Homer J. Simpson
 

Offline Andreas

  • Super Contributor
  • ***
  • Posts: 2545
  • Country: de
Re: Toyota firmware fail
« Reply #62 on: November 02, 2013, 06:28:36 pm »
Yeah, but the article didn't talk of a safety concept at all but claimed that every "important" data - including RTOS task activation bits - should be mirrored.
Ok in this case you are right. You will have to prove that the task (or at least the safety relevant routine) is actually activated. Usually this is done by cryptographic methods when entering the safety relevant code and exiting it.
If something goes wrong then the answer to the external watchdog will be wrong. The mirroring of the activation bits is useless if the task is activated but terminated on its way.

But I fear that in the according ECU no external challenge response windowed watchdog is used.

With best regards

Andreas
 

Online 0xdeadbeef

  • Super Contributor
  • ***
  • Posts: 1459
  • Country: de
Re: Toyota firmware fail
« Reply #63 on: November 02, 2013, 07:12:26 pm »
That's why I said that if there is a flaw in the ECU, it would be mainly a flaw in the safety concept.
Letting the CPU monitor itself has only limited use IMHO. It's important to have a bulletproof safety concept on the system side. Starting from mechanics (e.g. throttle blade going to limp home position if not controlled, dual sensors for pedal and throttle position) over hardware (e.g. shutdown paths in HW, secondary CPU monitoring the main CPU) up to the application software. Anyway, the software can be only a part of the concept, not the only place to perform safety checks. After all: if the CPU stopped to work correctly, skips tasks, has corrupted stack etc: who could be sure that the monitoring functions in the software would work at all?
Trying is the first step towards failure - Homer J. Simpson
 

Offline qno

  • Frequent Contributor
  • **
  • Posts: 422
  • Country: nl
Re: Toyota firmware fail
« Reply #64 on: November 02, 2013, 07:50:29 pm »
  This is the US.
If you ask a company that sells software analysis to analyse your software the will find something.
Fortunately the ECU does not run on Windows. Else you will get an update every 2 weeks.
Why spend money I don't have on things I don't need to impress people I don't like?
 

Offline G0HZU

  • Super Contributor
  • ***
  • Posts: 2576
  • Country: gb
Re: Toyota firmware fail
« Reply #65 on: November 02, 2013, 08:45:15 pm »
Quote
Well, from the photograph in the EDN article, the Toyota engine control electronics perhaps is produced by Delphi, as they call theirs ECM also, and it's called ECU otherwise.

http://delphi.com/manufacturers/auto/powertrain/gas/ecm/
Those units also feature ETC.

And this company was already mentioned in connection with this Toyota incident.

Also, I did not see the typical Conti imprint or code numbers on the PCB.


Hi Dr Frank
Did you miss my earlier post?
I'm virtually certain that it's a Denso ECU. Denso are owned by Toyota and they have been making ECUs for Toyota for many, many years.
 
Two of the chips on it have the newer Denso logo on them and if you do what I suggested earlier and google "Denso" then you will see a very similar ECU circuit board on their European website.

I spent several years taking various 1980s and 1990s Denso ECUs apart and reverse engineering their program code and I was always impressed with the quality of their ECUs.

For example, the ECUs for the UK market were designed to be as failsafe as possible. Even if the main MCU chip fails the ECU can fall back into analogue mode and keep sending basic ignition pulses and signals to the fuel injectors to try and keep the engine running well enough to allow the driver to have some control as this buys vital time to look for somewhere safe to stop.

All the time this is happening the ECU also tries to restart the MCU chip via a hard reset loop.

They were masters of efficient coding as well and only had a few kb of ROM to play with. So one trick to save space was to program at hex level allowing the MCU to jump into the middle byte of two byte instructions as a way of changing the way the program runs.

This gives a disassembler a tough time because this is illegal to most disassemblers. I wrote my own module for IDA Pro to cope with the 1990s ECUs as they also had a unique instruction set that took a fair bit of reverse engineering! (no datasheet or instruction set is available for the MCU)
« Last Edit: November 02, 2013, 08:48:27 pm by G0HZU »
 

Offline G0HZU

  • Super Contributor
  • ***
  • Posts: 2576
  • Country: gb
Re: Toyota firmware fail
« Reply #66 on: November 02, 2013, 09:11:01 pm »


Quotes from the EDN article:

Quote
Toyota claimed the 2005 Camry's main CPU had error detecting and correcting (EDAC) RAM. It didn't. EDAC, or at least parity RAM, is relatively easy and low-cost insurance for safety-critical systems.

I can only speak for the (fairly basic) 1980s and 1990s ECUs from Denso and these had basic error detection on the battery backed up RAM.

Every few milliseconds during ECU runtime the ECU runs an error/health check on these variables to see if they ever get corrupted as it stores this data in more than one way. I think the ECU alerts the driver with a dash lamp if there is a problem detected.

Most of the ECUs also ran checksums on the code and also basic read/write health checks on the entire system RAM during the initial few milliseconds after the ignition was turned on at the key.


Quote
They also failed to perform run-time stack monitoring.
This could be (crudely) done on the 80s and 90s Denso ECUs because I did it myself via their inbuilt (hidden) diagnostics port inside the ECU.



Quote
Two key items were not mirrored: The RTOS' critical internal data structures; and—the most important bytes of all, the final result of all this firmware—the TargetThrottleAngle global variable.

The Denso ECUs I looked at were able to read the TPS or throttle position many, many times per second so I'm not sure 'why' anyone would want to mirror this data.

If it read it or stored it wrongly in one instance then it would get refreshed in the blink of an eye anyway. The old ECUs also had the ability to detect 'untypical' results from the TPS and reject them (and store an error code and alert the driver instantly via the dashboard)
« Last Edit: November 02, 2013, 10:03:41 pm by G0HZU »
 

Offline G0HZU

  • Super Contributor
  • ***
  • Posts: 2576
  • Country: gb
Re: Toyota firmware fail
« Reply #67 on: November 02, 2013, 09:58:41 pm »
I looked on the web for more info on the affected ECUs and the general thought is that some 'task' carried out by the controller gets corrupted and doesn't finish and this corrupts the stack causing mayhem.

On the older Denso units the ECU had various healthchecks to guard against a system crash. Obviously this was a far simpler system but the ECU (and the MCU) was able to detect a system crash and reboot the MCU within a fraction of a second. It's a few years since I looked at this stuff but the ECU and the MCU worked together to produce a kind of 'analogue + MCU' heartbeat and if this missed a beat then the MCU was rebooted and the failsafe system kicks in to keep the injectors and ignition running in the very crude analogue override mode. Hopefully, the MCU then reboots in a fraction of a second and the MCU then regains control and runs OK for the rest of the journey.

Denso even went as far as filling all unused bytes in the ROM with a special instruction opcode that rapidly forced a reboot if the MCU ran off the rails into uncharted/unused parts of its ROM memory.

The ECU boot up code also flushes the RAM and writes sensible values into RAM for the initial lap around the main program loop until the ECU has read all the sensors and found its feet.

I suppose one potential bug on a fly by wire throttle could be if the ECU MCU kept crashing/rebooting due to a stack or RAM failure at some point in the overall program. It would therefore keep reusing a fixed initial default throttle setting during each attempted reboot up until the program crashed again (before getting to respond to the throttle position set by the driver). However, I would seriously hope that Denso would not make such a mistake in the reboot code.


« Last Edit: November 02, 2013, 10:20:59 pm by G0HZU »
 

Offline dannyf

  • Super Contributor
  • ***
  • Posts: 8229
  • Country: 00
Re: Toyota firmware fail
« Reply #68 on: November 03, 2013, 12:27:40 am »
Quote
This article gives a laundry list of reasons why the code could have failed, but apparently no one knows if or how it did, so what's the point?  It's a legal rather than technical discussion.

You nailed it right there.

This is a war funded by lawyers, fought by lawyers and will be won / lost by lawyers too.

All it does is is to add costs to future vehicles that innocent consumers pay.
================================
https://dannyelectronics.wordpress.com/
 

Offline G0HZU

  • Super Contributor
  • ***
  • Posts: 2576
  • Country: gb
Re: Toyota firmware fail
« Reply #69 on: November 03, 2013, 01:34:45 am »
Quote
Having said that, DENSO are responsible for the Japanese Domestic market. They normally use DENSO branded chips which are typically just customized brand-name micro's, restamped with denso's logo.
 I notice this particular case its from an American Prius which the ECU could have been developed by the americans to meet their own emissions standards and safety laws etc etc, considering they're using NEC parts rather than DENSO, it wouldnt surprise me if this were the case.

I looked at dozens of 80s and 90s Toyota ECUs from UKDM cars, USDM cars and JapDM. All were made by Denso :)

The MCU chips they used were usually special versions of mainstream MCU chips that had extra features that weren't on the mainstream equivalent MCUs.

This was probably for copy protection as well as for custom requirements. Also Denso made their own custom IC 'modules' that had thick film technology inside. I think this saved space on the PCB and also provided a degree of copy protection.

I worked out how the MCU operated and was able to make the ECU run my own copy of the factory ROM so I could modify it at will. eg I partly rewrote their code to support real time map retuning by uploading a version of factory code with the maps running in (external) RAM rather than internal ROM and I upgraded their diagnostics to include write access rather than just read access.

So it could still datalog but it could also show where it was on its maps as well as allow retuning via a laptop on a rolling road :)
 

Offline Teemo

  • Regular Contributor
  • *
  • Posts: 58
  • Country: ee
Re: Toyota firmware fail
« Reply #70 on: November 03, 2013, 01:56:32 am »
The complete design should be made public to make the evaluation possible  >:(  It should be basic right of everybody to examine the safety related design of the appatatus they are using.

If the design is not made public or is presented in unverifyable (uncomplete , unsystematic) manner, the one should assume that it is unadequate, and not to use the apparatus.
 

Offline Dr. Frank

  • Super Contributor
  • ***
  • Posts: 1874
  • Country: de
Re: Toyota firmware fail
« Reply #71 on: November 03, 2013, 10:15:31 am »
Hi Dr Frank
Did you miss my earlier post?
I'm virtually certain that it's a Denso ECU. Denso are owned by Toyota and they have been making ECUs for Toyota for many, many years.

Hi,

well yes, I just read your previous post again: "I can't say if the ECU in question was made by ND..." and you were not certain about the logo...

Anyhow, if it's really Toyotas internal supplier, or anybody else, I still can't believe that Toyota should have accepted bad SW engineering (that's the key argument of the so called embedded experts).

And anyhow, that judgement opened the door for further costly trouble for Toyota in US, and that's right, it's all about legal arguments now, not technically anymore.

Frank
 

Offline G0HZU

  • Super Contributor
  • ***
  • Posts: 2576
  • Country: gb
Re: Toyota firmware fail
« Reply #72 on: November 03, 2013, 02:03:31 pm »
Quote
it's all about legal arguments now, not technically anymore.

Yes, it's certainly getting scary for Toyota...

However, here's a link to the Denso website that shows a similar ECU

http://denso-europe.com/products/electronics/

It still doesn't conclusively prove that Denso make the 'whole' ECU but I think it's safe to assume they are heavily involved in this case.

I also managed to find a listing of the court case proceedings online and you can see the embedded expert (Mr Barr) getting cross examined by both sides. There's also reference to Denso being involved in the ECU design and production.

I only skimmed through the court proceedings but it seems like Barr tried to invoke the UA (unintended acceleration) by deliberately flipping bits in registers/memory during roadtests. Toyota gave him help in setting up the test with a system that let him corrupt the memory in real time in an attempt to invoke the UA.

It appears that in every attempt he failed...

However, Toyota still lost the case and I hope I never have to be involved in a legal case in a US court.
« Last Edit: November 03, 2013, 02:06:55 pm by G0HZU »
 

Offline dannyf

  • Super Contributor
  • ***
  • Posts: 8229
  • Country: 00
Re: Toyota firmware fail
« Reply #73 on: November 03, 2013, 02:16:04 pm »
Quote
It still doesn't conclusively prove that Denso make the 'whole' ECU but I think it's safe to assume they are heavily involved in this case.

I would be surprised if Denso is a major player in the ecu (chip) business: Renesas and Freescale are two names I would put on top in that market.

Quote
However, Toyota still lost the case

Our legal system relies heavily on who has the best lawyers (who can spin) than who has the facts on his side.
================================
https://dannyelectronics.wordpress.com/
 

Offline G0HZU

  • Super Contributor
  • ***
  • Posts: 2576
  • Country: gb
Re: Toyota firmware fail
« Reply #74 on: November 03, 2013, 02:37:17 pm »
Quote
Our legal system relies heavily on who has the best lawyers (who can spin) than who has the facts on his side.

Here in the UK I think the legal system is slightly less extreme but still very unimpressive... :(

According to the court notes Barr earned about $500 per hour over thousands of hours on the case so he definitely 'won' from a financial point of view. So I see him as being competent at making money and writing books about coding standards and saying how bad Toyota/Denso are at coding in a court.

But maybe a lot less impressive at breaking their product in the manner he says they can be broken :-DD

Why did he fail to invoke the UA if Toyota's poor coding standards code broke so many violations?

He was even allowed to spend thousands of hours studying the code (with Toyota to help him) and could flip bits illegally to try to make the ECU go into the UA mode. But from my initial reading of the court notes he appears to have failed.
« Last Edit: November 03, 2013, 02:39:25 pm by G0HZU »
 

Offline dannyf

  • Super Contributor
  • ***
  • Posts: 8229
  • Country: 00
Re: Toyota firmware fail
« Reply #75 on: November 03, 2013, 02:54:38 pm »
Quote
could flip bits illegally

I don't know how you could flip bits "illegally" or "legally" for that matter.

The person in question seems to be an expert at selectively presenting his facts and selling his stories to the legal system.

Whether he is an expert on identifying the issues remain to be seen.

Unfortunately, our system favors people like that.
================================
https://dannyelectronics.wordpress.com/
 

Offline G0HZU

  • Super Contributor
  • ***
  • Posts: 2576
  • Country: gb
Re: Toyota firmware fail
« Reply #76 on: November 03, 2013, 03:17:48 pm »
Quote
I don't know how you could flip bits "illegally" or "legally" for that matter.

They are just the words I chose to describe the process... what I mean is that I'm assuming they allowed him to tap into the system to overwrite bits in various registers/memory at will.

I could do this with Denso ECUs once I had modified their code to allow write access. Their ECUs in the 80s/90s had  a diagnostic window where you could ask the ECU for the contents of registers or memory on each lap of the main runtime. It isn't difficult to mod this to allow write access direct to the registers and RAM.

Obviously, this could cause all kinds of chaos if you started modifying the RAM with the intent to corrupt the system. It would be possible to write code that flipped the bits during other parts of the run time as well. What I find amusing is that Barr still failed to cause the UA even with the ability to modify the memory contents.

 Obviously I have never seen the program code for the ECU in question but I remain really impressed with the quality of Densos ECUs (and coding) for the 80s and 90s ECUs that I did look at. However, I'm not a professional ECU designer or reviewer so I guess my opinion carries little weight :)
 

Offline pinkysbrein

  • Contributor
  • Posts: 33
Re: Toyota firmware fail
« Reply #77 on: November 03, 2013, 03:46:24 pm »
One day I accidentally left the parking lights on overnight.  When I went back to the van I could hear a relay chattering due to the really low battery.

Naive question to which I assume there is an answer which I should simply realize, but I don't and google isn't helping. Why do cars generally not have low voltage lockouts which just turn stuff off until you start the engine?
« Last Edit: November 03, 2013, 03:49:13 pm by pinkysbrein »
 

Offline dannyf

  • Super Contributor
  • ***
  • Posts: 8229
  • Country: 00
Re: Toyota firmware fail
« Reply #78 on: November 03, 2013, 04:26:33 pm »
Quote
They are just the words I chose to describe the process...

That's fine as long as you don't use those words to communicate with other people - otherwise, people get confused by your arbitrary meanings of words.

Quote
I could do this with Denso ECUs once I had modified their code to allow write access.

It is a typical fairy tale of people modifying modern ECUs (without access to OEM documentation).

Quote
Why do cars generally not have low voltage lockouts which just turn stuff off until you start the engine?

If the battery has gotten so low it wouldn't have started the car anyway. Then, what's the point of such a feature?
================================
https://dannyelectronics.wordpress.com/
 

Offline G0HZU

  • Super Contributor
  • ***
  • Posts: 2576
  • Country: gb
Re: Toyota firmware fail
« Reply #79 on: November 03, 2013, 04:57:32 pm »
Quote
It is a typical fairy tale of people modifying modern ECUs (without access to OEM documentation).

I'm the one getting confused by your meaning of words now...  What do you mean by fairy tale? Are you implying that I am making this up?

 

Offline dannyf

  • Super Contributor
  • ***
  • Posts: 8229
  • Country: 00
Re: Toyota firmware fail
« Reply #80 on: November 03, 2013, 05:32:32 pm »
Quote
Are you implying that I am making this up?

I wasn't implying at all.
================================
https://dannyelectronics.wordpress.com/
 

Offline G0HZU

  • Super Contributor
  • ***
  • Posts: 2576
  • Country: gb
Re: Toyota firmware fail
« Reply #81 on: November 03, 2013, 06:54:09 pm »
Whatever... :)


One day I ought to revisit my work on this stuff and do a youtube teardown of a hacked Denso ECU.

I did do this for a Toyota club magazine about 10 years ago and explained in several magazine articles how all the mapping worked and printed out the maps and explained how it was able to control the fuel and ignition.

I managed to find a screenshot of an early tuning interface of mine. I think this was for a MR2 turbo and this would have been the interface used by Ryan (and Charlie) at Surrey rolling road maybe 7 years ago.

The interface allowed the stock 'read only' diagnostics to run but it also supported my extra code that allowed write access (via a packet system over RS-232) to the relevant map address in RAM memory. It used various forms of error checking to minimise bugs.

The idea was to then burn these tuned maps into the factory program code (in external FLASH) once the mapping was complete. So my modded code was only temporary and wasn't used for daily driving. So only the map values were changed in the version uploaded to FLASH memory after tuning.

To get the factory ECU to support remapping required the MCU in the ECU to be reconfigured to support external memory and this required a piggy PCB to hold the factory MCU plus a CPLD and a datalogging/tuning interface that I designed. The piggy PCB is below and shows the factory MCU plus the CPLD and external Flash and SRAM. The ribbon cables fit into the 64 holes vacated by the original 64 pin MCU.
« Last Edit: November 03, 2013, 07:02:33 pm by G0HZU »
 

Offline Dr. Frank

  • Super Contributor
  • ***
  • Posts: 1874
  • Country: de
Re: Toyota firmware fail
« Reply #82 on: November 03, 2013, 07:10:17 pm »
I would be surprised if Denso is a major player in the ecu (chip) business: Renesas and Freescale are two names I would put on top in that market.

You have misunderstood something:

Denso, as Delphi, Bosch, Conti and so on are suppliers of the complete modules / electronics for Toyota.

Renesas and Freescale are sub-Suppliers, they produce the microcontrollers only, and deliver them to Denso et.al., but do not design ECUs or other modules on their own.

Frank
« Last Edit: November 03, 2013, 07:18:17 pm by Dr. Frank »
 

Offline G0HZU

  • Super Contributor
  • ***
  • Posts: 2576
  • Country: gb
Re: Toyota firmware fail
« Reply #83 on: November 03, 2013, 07:20:52 pm »
I found the court case PDF here:


http://cybergibbons.com/wp-content/uploads/2013/10/Bookout_v_Toyota_Barr_REDACTED.pdf

It's not exactly riveting reading but it does show that Toyota were keen to hide various technical details about the ECU because lots of info about the affected 'tasks' in the program code is masked using the classic black block shapes to hide certain words...

 

Offline Dr. Frank

  • Super Contributor
  • ***
  • Posts: 1874
  • Country: de
Re: Toyota firmware fail
« Reply #84 on: November 03, 2013, 07:26:25 pm »
However, here's a link to the Denso website that shows a similar ECU

http://denso-europe.com/products/electronics/

Yes, you are right.

The unit in question seems to be a successor, or from the same family as the one on the DENSO site.

Same processors, same kind of components (esp. XTALs), very similar design / layout.
That's more than similar, definitely.

On the DENSO site, P/N is 275036-1150-2, the unit in question is 275036-2290.

It is very improbable, that a 2nd source supplier of the ECU is allowed to use the same processors and such a similar design.

So, if Denso still belongs to Toyota (I don't know anything about that connection), then Toyota keeps the blame in house.

Frank

 

Offline Dr. Frank

  • Super Contributor
  • ***
  • Posts: 1874
  • Country: de
Re: Toyota firmware fail
« Reply #85 on: November 03, 2013, 07:40:28 pm »
I found the court case PDF here:


http://cybergibbons.com/wp-content/uploads/2013/10/Bookout_v_Toyota_Barr_REDACTED.pdf

It's not exactly riveting reading but it does show that Toyota were keen to hide various technical details about the ECU because lots of info about the affected 'tasks' in the program code is masked using the classic black block shapes to hide certain words...

286 pages - very interesting, how they argue.
Will take my time for that reading.

Everybody in the engineering is informed / trained about financial regress due to faulty design...

But SW flaws are really very hard to mitigate, such systems are mostly too complex, and test methods can never dig 100% deep.

Frank
« Last Edit: November 03, 2013, 07:44:13 pm by Dr. Frank »
 

Offline David_AVD

  • Super Contributor
  • ***
  • Posts: 2607
  • Country: au
Re: Toyota firmware fail
« Reply #86 on: November 03, 2013, 09:29:58 pm »
One day I accidentally left the parking lights on overnight.  When I went back to the van I could hear a relay chattering due to the really low battery.

Naive question to which I assume there is an answer which I should simply realize, but I don't and google isn't helping. Why do cars generally not have low voltage lockouts which just turn stuff off until you start the engine?

I'd expect that the lighting circuits on most (certainly lower end) cars are simple switches or switches directly controlling relays.

Maybe I managed to get the van into a state where the ECU was repeatedly booting?

There was definitely a relay chattering but I don't know what that relay was controlling.  I don't recall now (it was 4 or 5 years ago ) if the parking lights were flickering while in that state.
 

Offline Stonent

  • Super Contributor
  • ***
  • Posts: 3824
  • Country: us
Re: Toyota firmware fail
« Reply #87 on: November 03, 2013, 09:50:40 pm »
Cars like my Jeep that like to turn the headlights on when the doors are unlocked can be annoying when the battery is a bit weak and you're trying to start it with the extra load.
The larger the government, the smaller the citizen.
 

Offline Marco

  • Super Contributor
  • ***
  • Posts: 4752
  • Country: nl
Re: Toyota firmware fail
« Reply #88 on: November 03, 2013, 10:34:52 pm »
I'd expect that the lighting circuits on most (certainly lower end) cars are simple switches or switches directly controlling relays.
You could still just put a latching relay in front of all the non essential stuff and switch it off when the battery gets too low.
 

Offline David_AVD

  • Super Contributor
  • ***
  • Posts: 2607
  • Country: au
Re: Toyota firmware fail
« Reply #89 on: November 03, 2013, 10:54:06 pm »
I'd expect that the lighting circuits on most (certainly lower end) cars are simple switches or switches directly controlling relays.
You could still just put a latching relay in front of all the non essential stuff and switch it off when the battery gets too low.

Ah, but that would cost money.   ;)
 

Offline N TYPE

  • Regular Contributor
  • *
  • Posts: 57
  • Country: au
Re: Toyota firmware fail
« Reply #90 on: November 04, 2013, 11:47:26 am »
Whatever... :)


One day I ought to revisit my work on this stuff and do a youtube teardown of a hacked Denso ECU.

I did do this for a Toyota club magazine about 10 years ago and explained in several magazine articles how all the mapping worked and printed out the maps and explained how it was able to control the fuel and ignition.

I managed to find a screenshot of an early tuning interface of mine. I think this was for a MR2 turbo and this would have been the interface used by Ryan (and Charlie) at Surrey rolling road maybe 7 years ago.

The interface allowed the stock 'read only' diagnostics to run but it also supported my extra code that allowed write access (via a packet system over RS-232) to the relevant map address in RAM memory. It used various forms of error checking to minimise bugs.

The idea was to then burn these tuned maps into the factory program code (in external FLASH) once the mapping was complete. So my modded code was only temporary and wasn't used for daily driving. So only the map values were changed in the version uploaded to FLASH memory after tuning.

To get the factory ECU to support remapping required the MCU in the ECU to be reconfigured to support external memory and this required a piggy PCB to hold the factory MCU plus a CPLD and a datalogging/tuning interface that I designed. The piggy PCB is below and shows the factory MCU plus the CPLD and external Flash and SRAM. The ribbon cables fit into the 64 holes vacated by the original 64 pin MCU.

http://toyota.kgbconsulting.ca/wiki/Main_Page
 

Offline nerdyHippy

  • Contributor
  • Posts: 37
  • Country: us
Re: Toyota firmware fail
« Reply #91 on: November 04, 2013, 03:20:34 pm »
Quote
Why do cars generally not have low voltage lockouts which just turn stuff off until you start the engine?
If the battery has gotten so low it wouldn't have started the car anyway. Then, what's the point of such a feature?
It would prevent the battery from sustaining damage from discharging too far.
 

Online nctnico

  • Super Contributor
  • ***
  • Posts: 19665
  • Country: nl
    • NCT Developments
Re: Toyota firmware fail
« Reply #92 on: November 04, 2013, 04:06:21 pm »
One day I accidentally left the parking lights on overnight.  When I went back to the van I could hear a relay chattering due to the really low battery.
Naive question to which I assume there is an answer which I should simply realize, but I don't and google isn't helping. Why do cars generally not have low voltage lockouts which just turn stuff off until you start the engine?
For most stuff to get power you have to turn the ignition key to 'ON'. During starting itself most cars turn everything off (including the lights) so all power is available for the starter motor. OTOH you'll want parking lights. I once visited a customer who lived in the middle of nowhere. When I stepped out the door it was already dark and without any lighting outside it was completely dark. Fortunately I left the parking lights on so I could see my car.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline bcx

  • Contributor
  • Posts: 7
  • Country: au
Re: Toyota firmware fail
« Reply #93 on: November 04, 2013, 04:24:37 pm »
I worked out how the MCU operated and was able to make the ECU run my own copy of the factory ROM so I could modify it at will. eg I partly rewrote their code to support real time map retuning by uploading a version of factory code with the maps running in (external) RAM rather than internal ROM and I upgraded their diagnostics to include write access rather than just read access.

So it could still datalog but it could also show where it was on its maps as well as allow retuning via a laptop on a rolling road :)

hehe, I do this now, but with Renesas/Hitachi H8/539 or SuperH - specifically for either Mitsubishi Magna/Verada or Mitsubishi Galant/Legnum VR4. Except no need for any hardware mods, plenty of onboard RAM & flash to use.

Infact, there is a big market for modding the stock firmware. Mitsubishi Evolution guys have added heaps of features to their roms, like speed density, launch control, antilag, live tuning and more recently FuelFlex for ethanol fuels. Same for the Subaru guys too - their ecus are manufactured by Mitsubishi Electric and use Renesas SuperH too.  Slightly off topic from the Toyota discussion.
« Last Edit: November 04, 2013, 04:27:57 pm by bcx »
 

Offline SeanB

  • Super Contributor
  • ***
  • Posts: 15387
  • Country: za
Re: Toyota firmware fail
« Reply #94 on: November 04, 2013, 07:18:17 pm »
Chattering relays on the Toyota with low battery will be the 2 DC bus relays that reduce the load on the ignition switch. Made by Denso, and have 2 6.3mm coil spades and some 10mm high current connections that switch all the DC bus loads. One bus runs engine management and ECU, and the other runs all other loads that are turned off during starting. There is a 3rd relay for the accessory bus, but this is a 4 6.3mm spade type rated at 25A. The main relays are IIRC rated at 50A. Interesting way they have to reduce back EMF by using a resistor in parallel with the relay coil instead of a diode.
 

Offline trackman44

  • Regular Contributor
  • *
  • Posts: 67
  • Country: ca
Re: Toyota firmware fail
« Reply #95 on: November 09, 2013, 04:33:52 am »
When the government is breathing down your neck and mandating for better fuel economy, as a car manufacturer what are you supposed to do? Take away more control from the driver! So that's why we have drive-by-wire throttle control. Like variable displacement and cam shaft timing wasn't good enough?!?!  :wtf: . Why would you design something more complex than a cheap as nails throttle cable? Next thing you know will have drive-by-wire clutch, brakes and steering or worse, autonomous driven cars!  " Open the pod bay doors HAL.... HAL open the pod bay doors!!.... HAL!!!"

Will
How 'bout them Maple Leafs?
 

Offline Teemo

  • Regular Contributor
  • *
  • Posts: 58
  • Country: ee
Re: Toyota firmware fail
« Reply #96 on: November 09, 2013, 09:43:41 am »
“Everything should be made as simple as possible, but not simpler.” (Albert Einstein)

It seems to me that for last 20 years the automotive industy only made cars more complex -- unneccesarily. The Otto cycle gasoline engine already reached to its best designs 20 years ago. There is very slight improvement in fuel economy and emissions, but at the cost of safety, serviceability and most importantly loss of simplicity and elegant design.

Only way out of this, to get things simple again, is some completely new engine design. This may very well come from some old invention whose time has come (like Bourke engine with modern electronics). Techincally it is all possible but we must overcome more difficult obstructions of user habits(a la engine sound) and legal issues.
 

Offline Nerull

  • Frequent Contributor
  • **
  • Posts: 681
Re: Toyota firmware fail
« Reply #97 on: November 09, 2013, 09:01:47 pm »
I can recall when the Prius runaway problem hit the news Steve Wozniak chimed in and claimed his prius was affected, Here is a youtube clip:
http://youtu.be/u44XjkWgFac    Read some of the comments, it seems Woz was confused by the cruise control.
 I partially remember reading there was a strong driver age bias (over 60 years old) to the cohort of un-commanded accelleration complainers.

In the early 80's when GM was a ghung-ho  early adopter of microprocessors  everywhere I heard about a software bug in one of the Cadillac models. When accelerating uphill or under load and transitioning through the middle gears - auto trans, if the driver then honked the horn the engine would suddenly lose power. :-DD

Diesel engines are reluctant to shut down once they are running, I think the standard design is to cut the fuel supply to the common fuel rail, but sometimes even that isn't sufficient. VW's first generation 4 cylinder engine installed in Rabbits in the mid 70's replicated a noob design flaw also exhibited by a few early american truck engines in 50's. They would, on rare occasion, cannilbalize  their own crankcase oil as fuel if the cylinders or rings were worn. When that happened the engine would race to 6000 rpm and seize in short order, but not before leaving the driver with an intense and bewildering experience, no software required.

I just helped replace a 40 year old Volvo diesel engine in a sailboat. The only way to shut it off if the throttle control failed was to reach into the engine compartment and throw the compression release levers that vented the cylinders.
 

Online wraper

  • Supporter
  • ****
  • Posts: 11506
  • Country: lv
Re: Toyota firmware fail
« Reply #98 on: November 09, 2013, 09:25:30 pm »
Chattering relays on the Toyota with low battery will be the 2 DC bus relays that reduce the load on the ignition switch. Made by Denso, and have 2 6.3mm coil spades and some 10mm high current connections that switch all the DC bus loads. One bus runs engine management and ECU, and the other runs all other loads that are turned off during starting. There is a 3rd relay for the accessory bus, but this is a 4 6.3mm spade type rated at 25A. The main relays are IIRC rated at 50A. Interesting way they have to reduce back EMF by using a resistor in parallel with the relay coil instead of a diode.
Diode increases relay release time significantly, so contacts start arching. I personally used bidirectional TVS diodes with 2x of the coil voltage in my latest design which included high power relays.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf