EEVblog #1066 - Uber Autonomous Car Fatality - How?

[coppice]

But the fact is that the current tech is massively overrated and often ascribed almost magical capabilities.

Wouldn't some of the "lower tech" collision avoidance technology (the type you can already get on production cars) have noticed the bicyclist at all, and at least have started to brake?

If you look at the performance specs of current collision avoidance systems fitted to cars, you'll find they are far from generic "stop before we hit the object in front" systems. Their crude designs (needed to avoid the cost exploding) means they will only properly react when the car AND the object being approached fit certain speed profiles. They tend to have big problems when driving rapidly towards a broken down vehicle on a motorway, but will slow down nicely to follow a car which is rapidly decelerating for some reason. Travelling around the city at 50kph, the same car would probably slow down and stop smoothly behind a broken down vehicle.

[Wollvieh]

Wouldn't some of the "lower tech" collision avoidance technology (the type you can already get on production cars) have noticed the bicyclist at all, and at least have started to brake?

According to this article Mobileye would have detected the pedestrian even from the shitty dashcam-footage 1 second before the impact.
https://techcrunch.com/2018/03/26/mobileye-chastises-uber-by-detecting-struck-pedestrian-in-footage-well-before-impact/

[Rasz]

Next time read with comprehension, mate. He explicitly talks about the AI algorithms doing the "thinking" and that LIDAR only feeds the data in!

does he now? whats this then:  "now one of them thinks it’s a dog" as if sensor did image classification, or as if someone used object classification on lidar output alone.

Most of the text is about how the system needs to balance data from multiple sensors.

First paragraphs I read seemed to be from distant past, not current state of the art AI, but 30 year old expert systems level of thinking. I guess in a sense author is right - AI couldnt save this pedestrian if HE was the one who programmed it.

I would say he has more clue about what he is talking about than most, given that he is working directly in the field.  What is your background to judge the article as "nonsense"?

1 he is not working in the field, he is a researcher at an university
2 its not my sole opinion, many people on hackernews had same sentiment.
3 Btw did Uber recruit in Talin?

btw "Intel Corp.’s Mobileye, which makes chips and sensors used in collision-avoidance systems and is a supplier to Aptiv, said Monday that it tested its own software after the crash by playing a video of the Uber incident on a television monitor. Mobileye said it was able to detect Herzberg one second before impact in its internal tests, despite the poor second-hand quality of the video relative to a direct connection to cameras equipped to the car."

Even Mobileye, famous Tesla decapitation platform, would be able to at least initiate braking with shitty chinese dashcam as a data source.

Wouldn't some of the "lower tech" collision avoidance technology (the type you can already get on production cars) have noticed the bicyclist at all, and at least have started to brake?

Dont worry, Uber was on top of things: https://www.sfgate.com/business/article/Uber-Disabled-Volvo-SUV-s-Standard-Safety-System-12782878.php

[coppice]

Not sure whether this has been posted already, but it is a pretty good analysis of how these technologies work and what can realistically be expected (e.g. LIDAR may not have saved the day even if it was installed and was working):

https://medium.com/@rebane/could-ai-have-saved-the-cyclist-had-i-programmed-the-uber-car-6e899067fefe

Dont know what background author has, but I stopped reading after few paragraphs of nonsense.
Lidar doesnt "think" and doesnt do classification, it just provides additional data points to classification algorithm, plus this dude keeps using dashcap footage as a proof :/

Next time read with comprehension, mate. He explicitly talks about the AI algorithms doing the "thinking" and that LIDAR only feeds the data in! Most of the text is about how the system needs to balance data from multiple sensors.

And re his background - if I am not mistaken, it is this dude:
https://www.researchgate.net/profile/Martin_Rebane

I would say he has more clue about what he is talking about than most, given that he is working directly in the field.  What is your background to judge the article as "nonsense"?

The article reads like a series of excuses, rather than an analysis. The car wasn't slowing, even as it hit the bicycle. It looks like the car's awareness of its situation was so weak that it would have continued on its journey with the cyclist and cycle stuck to the front grill, as if nothing had happened. It seems only the "driver" caused the car to react. Situational awareness issues are one of the major reasons people fail driving tests, and there are sounds reasons for that.  A vehicle with such poor situational awareness doesn't belong on public roads.

[janoc]

But the fact is that the current tech is massively overrated and often ascribed almost magical capabilities.

Wouldn't some of the "lower tech" collision avoidance technology (the type you can already get on production cars) have noticed the bicyclist at all, and at least have started to brake?

It possibly could - except Uber has disabled it so that it doesn't interfere with their testing. Volvo's subcontractor which supplies their automatic braking systems has said that today.

https://www.sfgate.com/business/article/Uber-Disabled-Volvo-SUV-s-Standard-Safety-System-12782878.php#item-85307-tbla-5

Now these systems aren't infallible neither - e.g. Volvo's pedestrian detection works only at low speed and not 60km/h the car was doing. And even then it is not fool-proof:

https://tv.sme.sk/v/16594/test-zastavi-volvo-s60-bez-zasahu-vodica-pred-chodcom.html

(Video in Slovak and shows an S60 instead of the XC90 SUV Uber has, but should give you an idea).

[janoc]

The article reads like a series of excuses, rather than an analysis. The car wasn't slowing, even as it hit the bicycle. It looks like the car's awareness of its situation was so weak that it would have continued on its journey with the cyclist and cycle stuck to the front grill, as if nothing had happened. It seems only the "driver" caused the car to react. Situational awareness issues are one of the major reasons people fail driving tests, and there are sounds reasons for that.  A vehicle with such poor situational awareness doesn't belong on public roads.

You are reading waay too much into it. Rebane's article is only meant to demystify for general public how these systems work and make decisions, nothing else. I don't see why an Estonian machine learning PhD student would want to "make excuses" for the poor performance of the Uber's car they are not involved with - and which is not in dispute, btw, and Arizona has banned further tests already for that reason).

Also the safety driver was evidently distracted (looking at a phone?) and stopped the car only after the impact has occurred - if that (the car could have detected the impact and stopped by itself too).

[Marco]

He has a commercial interest that a field which doesn't really deliver all that much commercial value stays as hot as it is.

The market only needs so many PhD's to datamine for advertising and security services.

[janoc]

Next time read with comprehension, mate. He explicitly talks about the AI algorithms doing the "thinking" and that LIDAR only feeds the data in!

does he now? whats this then:  "now one of them thinks it’s a dog" as if sensor did image classification, or as if someone used object classification on lidar output alone.

Selective quoting without understanding, yay.

Here is the full quote:

Modern CA systems are operating with memory, they have maps and registers of what they have seen. They keep track of recorded objects from image to image. If two seconds ago both sensors (or more precisely, the algorithms that interprets the sensor readings) agreed the object to be a lorry and now one of them thinks it’s a dog,

(emphasis mine).

Most of the text is about how the system needs to balance data from multiple sensors.

First paragraphs I read seemed to be from distant past, not current state of the art AI, but 30 year old expert systems level of thinking. I guess in a sense author is right - AI couldnt save this pedestrian if HE was the one who programmed it.

Because the first paragraph does talk about an deterministic, expert system kind of thing! Namely the collision avoidance you may have in your car today - aka automatic braking. The immediately following paragraph speaks about the difference of modern AI from this.

Jeeze, did you actually even make a token effort to understand the text?

1 he is not working in the field, he is a researcher at an university

So working in machine learning research and directly with the tech involved (e.g. LIDAR mapping) is not in the field.  Where do you think companies like Waymo or Uber got their machine learning algorithms from?

2 its not my sole opinion, many people on hackernews had same sentiment.
3 Btw did Uber recruit in Talin?

How is that relevant?

btw "Intel Corp.’s Mobileye, which makes chips and sensors used in collision-avoidance systems and is a supplier to Aptiv, said Monday that it tested its own software after the crash by playing a video of the Uber incident on a television monitor. Mobileye said it was able to detect Herzberg one second before impact in its internal tests, despite the poor second-hand quality of the video relative to a direct connection to cameras equipped to the car."

Even Mobileye, famous Tesla decapitation platform, would be able to at least initiate braking with shitty chinese dashcam as a data source.

Yes and Aptiv has also explicitly said that Uber has disabled their stuff on the Volvo as to not interfere with their own systems (Aptiv doesn't want to be tainted by the scandal). The poor performance of the Uber's car is not in dispute, get off your high horse.

Reban only explains how these systems work so that even mere mortals can understand it, he is not defending the poor performance of the Uber's tech (which is apparently not news and was known even before this accident).

But given that you didn't understand nor read his article, I don't think you are in position to judge it neither.

[janoc]

He has a commercial interest that a field which doesn't really deliver all that much commercial value stays as hot as it is.

The market only needs so many PhD's to datamine for advertising and security services.

  Seriously?

So someone with "a commercial interest in the field" is going to spend 5 3 years**  in Tallinn in Estonia, slaving away for next to nothing (and likely accruing debt in the process) doing a PhD at uni instead of joining some startup today, right? You don't need a PhD to do apply a bunch of existing tools like Keras or Tensorflow to a problem, such as that data mining, you know.

Holy crap that logic ...

** corrected before someone attacks me on it, missing the point - but 5y PhDs are quite common, especially when done in external form, i.e. along having a regular job outside of the uni.

[coppice]

I don't see why an Estonian machine learning PhD student would want to "make excuses" for the poor performance of the Uber's car they are not involved with

That's human nature, and you'll see it when flaws are pointed out with any technology being developed. You generally get two types of response:

• "Their system is flawed, but we have a robust fix for that flaw in our version."
• Replies like the one from the PhD student, because an attack on one of us is an attack on all of us.

[janoc]

That's human nature, and you'll see it when flaws are pointed out with any technology being developed. You generally get two types of response:

• "Their system is flawed, but we have a robust fix for that flaw in our version."
• Replies like the one from the PhD student, because an attack on one of us is an attack on all of us.

You forgot the third type of response - anyone posting something trying to provide context and not right away joining the mob calling for lynching is obviously defending the indefensible and needs to be tarred and feathered. Ideally by folks who didn't even bother to read (and understand) what they have said. The good old - "if you aren't with us, you are against us" mentality.

I fail to see where is he providing any "defense" or "responding to an attack". But I rest my case, I have better things to do than to argue with people who are not even attempting to understand what others have to say before attacking them.

[ez24]

Wouldn't some of the "lower tech" collision avoidance technology (the type you can already get on production cars) have noticed the bicyclist at all, and at least have started to brake?

Toyota is coming out with what they call TSS-P 2.0 next year and in the specs is "bicycle avoidance".

[Rasz]

Selective quoting without understanding, yay.

Here is the full quote:

Modern CA systems are operating with memory, they have maps and registers of what they have seen. They keep track of recorded objects from image to image. If two seconds ago both sensors (or more precisely, the algorithms that interprets the sensor readings) agreed the object to be a lorry and now one of them thinks it’s a dog,

(emphasis mine).

directly suggests running object classification on lidar output alone, this is where I stopped reading

So working in machine learning research and directly with the tech involved (e.g. LIDAR mapping) is not in the field.  Where do you think companies like Waymo or Uber got their machine learning algorithms from?

by this measure I could claim to be working in the field as well, after all I do have a depth camera on my desk and did 3 semesters of machine learning.
For me working in the field actually means deploying the tech, not learning about it.

2 its not my sole opinion, many people on hackernews had same sentiment.
3 Btw did Uber recruit in Talin?

How is that relevant?

-other ML people also didnt quite agree with his conclusions
-self explanatory

Yes and Aptiv has also explicitly said that Uber has disabled their stuff on the Volvo as to not interfere with their own systems (Aptiv doesn't want to be tainted by the scandal). The poor performance of the Uber's car is not in dispute, get off your high horse.

The whole thing is a list of excuses with a conclusion its so hard it was basically unavoidable, meanwhile pretty crappy mobileye has no problem even with shitty copy of a copy of a dashcam footage, same footage author deemed too bad for image classification (and implied this was the picture actual system would use!?!).

So someone with "a commercial interest in the field" is going to spend 5 3 years**  in Tallinn in Estonia, slaving away for next to nothing (and likely accruing debt in the process) doing a PhD

Higher ed is mostly tuition free in Estonia.

Basically people who actually do work with technology all agreed this was a big fat WTF from Uber, and then this dude pops out with "its really hard guys, mmmkay". This was the PERFECT setup for autonomous car: slow moving person, dragging metal reflector, crossing 3-4 lanes of road at 90 angle, directly under two street lamps, on an empty road with good visibility (actual one, not the lol dashcam).


Edit:

https://www.theverge.com/2018/3/27/17168606/nvidia-suspends-self-driving-test-uber

"NVIDIA Titan V Reportedly Producing Errors in Scientific Simulations" https://wccftech.com/nvidia-titan-v-error/

[alho]

Producer of No Agenda Show had this to say.

 
  Boots on the ground from Brian Kaufman
       
        I live right by where the Uber self-driving accident
        occurred, and the Uber released video showing that apparently no driver could
        have avoided hitting that woman in a million years is totally misleading.
        See my YouTube video below where I drive the
        same path as the Uber autonomous car was on. The attached photos were
        taken from the point of the accident.

Link to video in question
https://youtu.be/CRW0q8i3u6E?t=22s
What kind of camera are these Uber folks using? Cheapest that can be found from shenzhen?

Never heard about No Agenda Show, The Best Podcast In The Universe? Take few minutes to listen, maybe you'll find it interesting.
http://naplay.it/1018/2-13-31

[ez24]

https://www.theverge.com/2018/3/27/17168606/nvidia-suspends-self-driving-test-uber

From this link is the attached picture of the "Brain" used by AVs.   Does anyone know anything about the connectors?

thanks

[apis]

And re his background - if I am not mistaken, it is this dude:
https://www.researchgate.net/profile/Martin_Rebane

I would say he has more clue about what he is talking about than most, given that he is working directly in the field.  What is your background to judge the article as "nonsense"?

He probably knows a bit about AI but he doesn't seem to know anything about how these cars work in practice.

His conclusion is nonsense at least: "The internals of modern AI system are far too complex to assess without having exactly the same data as was available for the Uber car". While that is technically correct, without all the data it is hard to tell what happened, that is true for all and every system, AI or not. He seems to think these programs are black boxes though, which they most definitely are not (at least not google's, I don't know much about Uber's cars). It's not like you take all sensor data, feed it to a neural net and then feed the output to the car's controls.

The car's cameras would have had a large dynamic range, which means it shouldn't have had any problem with lighting conditions (which he doesn't seem to realise either). There should also have been lidar and radar which are independent of lighting. At the very least the lidar is certain to have detected there was a large object directly in front of the car and the program should have noticed that and activated the brakes in time. There really is no excuse for the software not to handle this situation, it's such a common and predictable scenario. So unless the breaks suddenly failed for some other reason this really shouldn't have happened. Hopefully a proper investigation will tell exactly what went wrong eventually.

[maginnovision]

https://www.theverge.com/2018/3/27/17168606/nvidia-suspends-self-driving-test-uber

From this link is the attached picture of the "Brain" used by AVs.   Does anyone know anything about the connectors?

thanks

Not sure on exact series but pretty basic automotive stuff. Some LVDS, coax, and a big rugged connector. Probably for dlc/dme. Hard to say for sure though. A TCU looks pretty similar but fewer connectors. Could actually be fiber optic for the blue? Just can't be sure. Also I don't think that board is out yet.

[G7PSK]

There is one thing that has not been brought up yet. Suppose it was not an accident but some one was hacking uber for whatever reason and deliberately either made the car blind to the pedestrian or just overrode the controls and drove the car over her.

[Brumby]

Conspiracy theories are such a fascinating waste of time.    JMHO

[apis]

If it was hacked that would be relatively easy to tell afterwards. Doubt it's the case here but if autonomous cars are networked they will likely be susceptible to hacking, which is a big problem imho. You can build unhackable systems in theory but it seems very hard to do in practice.

[Decoman]

Somehow, I suspect that autonomous vehicles with mediocre hardware/software (whatever is current really), might be as "impractical" (as in being directly hazardous, because of poor operational security) and as dangerous as having people in boxes go ballistic through pipes just to go as fast as one can that way.

[TheDane]

If it was hacked that would be relatively easy to tell afterwards. Doubt it's the case here but if autonomous cars are networked they will likely be susceptible to hacking, which is a big problem imho. You can build unhackable systems in theory but it seems very hard to do in practice.

You could say it was 'hacked' - literally - if removing almost all of the LIDAR's is equal to 'hacking a system into defective pieces' 
https://www.reuters.com/article/us-uber-selfdriving-sensors-insight/ubers-use-of-fewer-safety-sensors-prompts-questions-after-arizona-crash-idUSKBN1H337Q

[CatalinaWOW]

Maybe this is a case of someone "Muntzing" themselves to the lowest cost solution.  And going at least one step too far.

[Marco]

Everything is build down to a price. I don't necessarily see the problem with using LIDAR + stereo.

Stereo cameras in IR should be able to get a good depth picture near the car.

[CatalinaWOW]

Everything is build down to a price. I don't necessarily see the problem with using LIDAR + stereo.

Stereo cameras in IR should be able to get a good depth picture near the car.

The car had an adequate set of sensors installed.  But various reports indicate one or more of the sensor systems was disabled.  No comments have been made on the processors, but there is always the possibility that some processing capacity was also disabled.

I agree, everything is built to a price.  But not everyone agrees on the minimum required performance, and there are certainly mistakes made on the route to whatever performance level a particular group is trying to achieve.