EEVblog #1066 - Uber Autonomous Car Fatality - How?

[EEVblog]

How did the Uber autonomous car fatality in Tempe Arizona happen?
It basically shouldn't have.
A look at the newly released camera footage of the accident, the location, and the car LIDAR, RADAR, and camera sensor suites available to prevent such an accident.
Video footage: https://twitter.com/TempePolice/status/976585098542833664
Location of accident: https://www.google.com.au/maps/@33.4369934,-111.9429875,3a,75y,115.16h,92.53t/data=!3m6!1e1!3m4!1scUyILaxFs5z63AL2SupCJw!2e0!7i13312!8i6656
Inside Uber’s self-driving car mess:
https://www.recode.net/2017/3/24/14737438/uber-self-driving-turmoil-otto-travis-kalanick-civil-war

[BravoV]

Trust the "Car" Autonomous System with your life ? 

Even the "Train" Autonomous System thats still evolving for century, still keep crashing (includes each other) almost every week all over the world including in so called advanced developed countries, oh btw, its still running on a "fixed" rail, not roams freely like car on road.

 -> List of rail accidents (2010–present)

[EEVblog]

LIDAR footage form Google self driving car:

https://www.youtube.com/watch?v=tiwVMrTLUWg&feature=youtu.be&t=9m5s

[bitseeker]

Although we still need more data, specifically what the car did or didn't know about its surroundings during the time in question, after scrubbing through the exterior and interior footage, it seems highly unusual that the car behaved as though it picked up nothing at all.

The driver wasn't jostled by any change in G-forces during the moments prior to impact, there were no other distracting objects on the roadway, and the orientation of the pedestrian with bicycle made for a very large obstruction. If a fence had been placed across the lanes, it's as if the car would've just driven right through it as well.

[EEVblog]

I personally do not think autonomous vehicles should hit the road too early instead look at creating or dividing roads into separate driving territories. This will be very difficult to implement though due to numbers of cars in proportion. I would hate to be an authority to decide this.

This is another reason why I don't think fully atonymous cars will be put into mainstream use any time soon. Some politician has to sign off on that, and they like covering their arse. Look at how the virtually never want the raise the speed limits or remove speed humps etc once they have lowered them, no one wants to take responsibility.

[EEVblog]

Although we still need more data, specifically what the car did or didn't know about its surroundings during the time in question, after scrubbing through the exterior and interior footage, it seems highly unusual that the car behaved as though it picked up nothing at all.
The driver wasn't jostled by any change in G-forces during the moments prior to impact, there were no other distracting objects on the roadway, and the orientation of the pedestrian with bicycle made for a very large obstruction. If a fence had been placed across the lanes, it's as if the car would've just driven right through it as well.

Yep, it's almost a perfect test case. Uber may be in trouble over this.

[Brumby]

Looking at it as a visual human there is still conjecture. I know that my dash-cam cannot pick up as much detail as my eyes, so the video footage is not entirely final.

Indeed.  I tried pulling a still from that footage, just before the pedestrian was visible.  Unsurprisingly, there was not enough information to get any sense of there being something there.  Human vision is far superior than this example.

If you will notice - even though the safety driver was obviously distracted by something in their hands - probably a mobile - they DID, in fact, glance up just before impact - as the direction of their attention was slightly to the left of the vehicle.  It was as though something caught their eye.  I might go so far as to say if they had been driving normally, with full attention on the road, they might have seen enough to have been able to have taken some sort of evasive action.  I doubt it would have avoided the accident completely, but it may have turned out to be non-fatal.  Just my thoughts.

BUT, as Dave has said - that is not the point.  The point is the technology's ability to detect (and respond to) the hazard.

In this case, it is hard to see how the technology would have found the environment difficult.  It was open road with ample clearance and unobstructed view.  It's a scenario I would have expected in the early days of machine vision where simple recognition scenarios were being explored at varying velocities.


I am at a loss.


I can only think something really dumb has happened - like someone forgot to plug in a cable or there was a bad solder joint that gave way.

[tpowell1830]

I have said many, many times that the technology is too premature to be viable. I live in a very high traffic area and as I drive down the freeway with 5 lanes full of traffic at road speeds (60-70 mph) and marvel at how all of the vehicles merge into other lanes and the varying speeds of different vehicles do not crash into each other because humans can anticipate and adjust. Even when there are traffic cones and construction, the number of fender benders, compared to how many vehicles on a given street is minimal. The statistics tell the story of vehicle fatalities, and, given the sheer number of vehicles on the road, the fatalities, IMHO are extremely low.

I dare say that many may not grasp this concept because in city centers like where I live, the traffic is horrifying for the unskilled driver in such conditions. The roads, streets and highways in the US are huge compared to most countries, so I can't even imagine how autonomous driving could even work in countries where the roads and streets are tiny.

Although this film has been around a long time, when an autonomous car can do this, then it might be ready for public release.

If you are a bit squimish about fast driving, do not watch this film.



Admittedly, the streets were fairly empty in this film.

[Monkeh]

Even if a human driver could've seen her from further away, how much further?

As far as I'm aware, the US white lines are 10 feet long with 30 feet gaps - so a very generous estimation of range at the moment of visibility on the camera is about 75 feet - let's call it 25 metres. How far do you think the driver could've seen her? 30? 35?

It's a 45mph road. UK guidance on thinking + braking time for 40mph is 36 metres. Okay, a lot of cars will brake better, and some drivers are faster. But this is also both dark and at night, which will slow reactions further.

I'm pretty sure I'd have hit her.

I'm absolutely sure the car shouldn't have.

given the sheer number of vehicles on the road, the fatalities, IMHO are extremely low.

IMHO the number of fatalities on your roads are distressingly high. You realise your roads kill more than three times as many people as ours, right? I saw more dangerous driving in three weeks in NC than I've seen in the last few years here. Hell, I saw three crashes in as many miles immediately after leaving the airport..

[EEVblog]

If you will notice - even though the safety driver was obviously distracted by something in their hands - probably a mobile - they DID, in fact, glance up just before impact - as the direction of their attention was slightly to the left of the vehicle. 

I'd just like to re-affirm that I was the one who synced up these videos, and really didn't have a solid reference for that. So there is no real evidence the driver noticed anything before the actual impact.

It was as though something caught their eye.  I might go so far as to say if they had been driving normally, with full attention on the road, they might have seen enough to have been able to have taken some sort of evasive action.  I doubt it would have avoided the accident completely, but it may have turned out to be non-fatal.  Just my thoughts.

Possibly.

BUT, as Dave has said - that is not the point.  The point is the technology's ability to detect (and respond to) the hazard.

Indeed, and I don't think that any discussion should lead down that path.

I can only think something really dumb has happened - like someone forgot to plug in a cable or there was a bad solder joint that gave way.

Yes, that bike and person should have been a really good target for the Lidar, and as a researcher on the video comments pointed out, they can get 2cm detail at 100m with such systems.

[ataradov]

I'm absolutely sure the car shouldn't have.

And they won't as they get better. If this was a human accident, we would not even known about it. And in the same time span, human drivers killed way more people than self driving cars.

Those cars are everywhere in Phoenix area, by many companies. It is not like some one-off test that went wrong, they have been driving there for along time. And this rate of accident is kind of surprising.

[pb2k]

The person crossing the road wins a Darwin award, full stop. The saving grace of this situation is we get to examine the logs to figure out what went wrong and tweak the algorithm and hardware to make it better tomorrow. I suspect there was still some sort of serious failure here, be it code or hardware, but we'll have to wait on the official report.

[taemun]

Would a LIDAR be able to 'see' the pedestrian's black jumper? I'm not sure if it would be reflective to IR, or if the diffuse/soft surface would reflect enough energy anyway. Regardless, I suppose you'd be able to 'see' a hole - a lack of other reflections from beyond that surface.

This does pose an interesting question - what standard should we be holding autonomous car systems to? Human level? The best possible, with the sensors available? No fatalities at all?

Clearly the Tempe police chief was holding it to human standard, when he said that Uber wasn't at fault.

How would you confirm "the best possible, with the sensors available"? That sort of firmware/software qualification sounds verging on impossible.

It won't happen, but I'd like to see a full sensor dump from Uber for this event.

[EEVblog]

I'm absolutely sure the car shouldn't have.

And they won't as they get better. If this was a human accident, we would not even known about it. And in the same time span, human drivers killed way more people than self driving cars.

Accident statistics are an entirely different argument to what we are having here.

Those cars are everywhere in Phoenix area, by many companies. It is not like some one-off test that went wrong, they have been driving there for along time. And this rate of accident is kind of surprising.

Because most humans pedestrians are pretty good at avoiding these accidents to begin with. So given the relatively low percentage of these autonomous cars on the road, the amount of time the driver uses that mode and is not paying attention, and humans doing the avoiding, it not surprising that it hasn't happened much. But now it's happened and it looks to be that the tech failed in an almost ideal practical test case, it raises serious questions.

[tpowell1830]

LIDAR footage form Google self driving car:

https://www.youtube.com/watch?v=tiwVMrTLUWg&feature=youtu.be&t=9m5s

The little Google cars are driving in city street conditions, driving <30mph, but highway/freeway speed conditions, no. In my mind, cars can safely drive at high speeds (100mph or more) in open highways but under normal large city traffic conditions, again, no. Vehicles will need to be able to communicate with each other from a distance in order to be safe and, unless we put communication devices on foot/bicycle/skateboard traffic, autonomous vehicles will not work.

The examples given in the TED video are a tiny tip of the iceberg for problems to overcome on roadways.

With all of that said, when all of this autonomous driverless vehicles was first proposed/discussed, my thoughts at the time was, again, too many challenges to overcome to be safe. The only way that I thought that this could become a reality is to create a train type of roadway that had redundant communications posts setup every few hundred meters on a roadway and all vehicle communicated with each other so that the onboard computer could fall in line with other vehicles moving in the same direction, at the same speed, like a train. The 'train' of vehicles could then achieve high speeds, and the onboard, pre-programmed destination computer could coordinate with other vehicles in the 'train' to allow the vehicle to navigate to it's destination. As one commenter above said, this roadway would be strictly for autonomous use, and once the vehicle breaks out of this autonomous roadway, the human driver would need to take over. The logistics for this type of roadway/vehicle are also a very large problem. The vehicle maintenance for this type of vehicle would have to be regulated to a cost and would be so expensive for all situations that it becomes non-viable. The roadway itself would need a computer every few hundred meters as a traffic regulator to communicate with all the vehicle on its' roads. This means tax dollars to accomplish this, which escalates the cost of the system. There would need to be certified safety technicians/engineers to maintain all of this equipment, which escalates the cost.

In my humble opinion, the autonomous vehicle is too costly and premature. In fact, unless there are tremendous breakthroughs in technology that will lower costs and raise the AI to much higher levels, autonomous vehicles are not ready for public use.

The airplane industry back in the beginnings struggled with this problem, and eventually airplanes became safer to use for transporting people. That took 50 years to accomplish before planes were carrying large numbers of travelers. Planes today are much safer, statistically, for travel than any ground vehicle (not sure about trains). The problem with airplanes, when one falls out of the sky, hundreds of people die in one felled swoop, which makes flying seem unsafe to some, but when you look at the numbers, planes win the safety race, hands down. In 50 years, some may look back to now and marvel at the crude methods that we are currently using to navigate our roads, but much more work will have to be done in order to get to that point.

[EEVblog]

This does pose an interesting question - what standard should we be holding autonomous car systems to? Human level? The best possible, with the sensors available? No fatalities at all?

We expect better than humans when we take humans out of the loop. All this extra tech means that it should perform better than a human in the areas that the sensor excels against a human. e.g. at night with LIDAR.

It won't happen, but I'd like to see a full sensor dump from Uber for this event.

I suspect that may never be released, unless there is some legal requirement to do so. Uber will probably claim it's proprietary data and will only release to coronial inquest etc.

[ataradov]

What is the point you want to make by saying this?

That it is not a huge deal that self driving car had an accident. It will be investigated and resolved.

[ataradov]

We expect better than humans when we take humans out of the loop. All this extra tech means that it should perform better than a human in the areas that the sensor excels against a human. e.g. at night with LIDAR.

Even if accident rate is exactly the same, but we have gained ability to do other stuff while commuting, it is a win overall.

And things will only get better from there.

[Xenon Photon]

So sad to hear about this accident.
A huge research and industry efforts are going in autonomous vehicles with a lot of hype everywhere!

AI research is great but unfortunately, not perfect (yet!). State of the art object detection and semantic segmentation networks still misses some obstacles even in consecutive frames. Relying on tracking does fail in some of these cases too. The network might get confused because of the cyclist and pedestrians instances with this unusual orientation. Probably not enough training data - if any - having a similar situation.
Let alone existing hardware limitations. For example, the latency during data buffering and encoding from camera/LIDAR to GPUs.

However, the safety and behavior planning algorithms should have detected that and at the very least issued a warning to the driver. Did this happen?

[tpowell1830]

Even if a human driver could've seen her from further away, how much further?

As far as I'm aware, the US white lines are 10 feet long with 30 feet gaps - so a very generous estimation of range at the moment of visibility on the camera is about 75 feet - let's call it 25 metres. How far do you think the driver could've seen her? 30? 35?

It's a 45mph road. UK guidance on thinking + braking time for 40mph is 36 metres. Okay, a lot of cars will brake better, and some drivers are faster. But this is also both dark and at night, which will slow reactions further.

I'm pretty sure I'd have hit her.

I'm absolutely sure the car shouldn't have.

given the sheer number of vehicles on the road, the fatalities, IMHO are extremely low.

IMHO the number of fatalities on your roads are distressingly high. You realise your roads kill more than three times as many people as ours, right? I saw more dangerous driving in three weeks in NC than I've seen in the last few years here. Hell, I saw three crashes in as many miles immediately after leaving the airport..

I am not sure that you can compare our roads and traffic to your little country, and I am not sure about your point. I was talking about in my experience on my roads in my area. If you want to make this a country comparison, get your facts together and make your case. As I commented, my area is a very high traffic area and there are fender benders all over, rarely do I ever see a fatal accident, for that matter, I have seen but a few fatal vehicle accidents in my 65 years, but I encounter thousands of vehicles on my short 7 mile drive to my office every day and that is why I say that it is in my experience. Please don't make this about country comparisons.

[lgarbo]

I've been lucky enough to hear a in-depth talk about the physics of the specific Waymo/Velodyne LIDAR. I also had an opportunity to talk to an Uber ATC (their AV wing) engineer at a career fair. Going from memory, I think some of the lidar and radar sensors fused had a minimum range of ~50m in poor conditions (i.e. fog or rain).

I'd wager this to be a software problem, not a sensing/hardware problem. As someone in the video comments mentioned, it's likely Uber has fewer layers of failsafes than other companies and instead relies on a small number of high level programs running various SLAM and signal processing algorithms that handle all situations, emergency or not. This is in contrast to having a dedicated emergency breaking system that is looking for a precise but limited set of circumstances. Based on my (very) limited knowledge of what Uber has been doing, it would not surprise me one bit to find out that in a rush to market they had put all their eggs in a single control system.

Obviously this is all speculation, but I will be looking out for the NTSB (US federal investigators known mostly for excellent air crash investigations) reports on it. It's very strongly my belief that we have 100% of the tech needed for safe self driving, but there is a huge rush to get it to market and I think that is leaving cut corners.

[iaso]

I work at a large T1 automotive supplier and as expected this is a hot topic at the water cooler. We are quite experienced with self-driving cars, most of our management drives a Tesla and our pool cars are also Tesla's.

We have some very anti-self driving car employees here in the office and this was the thing they were waiting for to push their archaic agenda. "A human would not have made this error!",  "If a person was driving that poor person would be alive now!".

I took the video that was released and just asked them before playing the video to say "BRAKE" whenever they saw the person. I might have converted a few people.

Sad as it is, this incident will help shape legal conundrums that we have been talking about since self-driving cars were first envisioned. Hopefully, this starts shaping a legal framework for liability opening the way for more innovation.

[orin]

That view of stepping out from the shadows may be misleading. A human may have had better vision than the vision shown by the camera footage.

I really doubt it.

To put things in perspective, I rode my bicycle to/from work today; 13 miles each way.  When the weather is good, I ride maybe two days out of five.  I see more poor driving than you'd believe, both driving and bicycling, but I really cannot fault the driving here.  There are idiots in cars not paying attention and there are idiots on bicycles.

The big problem here for a human driver is the crossing from deep shadow to the area illuminated by the street light to another area of deep shadow.  The same happens with dappled shadows from trees in the summer and I've witnessed a bicycle rider go head on into another on a trail in such lighting conditions... at a speed much lower than we have here.  You should really slow down when there are such abrupt changes in lighting, but no-one ever does and I'd get harassed if I did.

Certainly in WA, even if a crosswalk was involved, by law, you have to give traffic time to stop before crossing.  You can't just cross the street and say to hell with the cars.  For sure, as a pedestrian, you have to be aggressive at stepping out at times or they'll never stop for you, but you always have to be prepared to stop and let a car go by once you do enter a crosswalk.  But in this case, there was no crosswalk so even that doesn't apply.

Now Dave's argument is that the sensors on the car should have done better than the average driver's vision and I think that has to be true before we let autonomous cars loose on their own.

[Brumby]

If you will notice - even though the safety driver was obviously distracted by something in their hands - probably a mobile - they DID, in fact, glance up just before impact - as the direction of their attention was slightly to the left of the vehicle. 

I'd just like to re-affirm that I was the one who synced up these videos, and really didn't have a solid reference for that. So there is no real evidence the driver noticed anything before the actual impact.

My comment had nothing to do with your syncing of the two videos.  It is purely on my observations of the face of the "safety driver".  They look to the left of the vehicle and there is only something of interest in that direction a second before the impact.

[iaso]

If you will notice - even though the safety driver was obviously distracted by something in their hands - probably a mobile - they DID, in fact, glance up just before impact - as the direction of their attention was slightly to the left of the vehicle. 

I'd just like to re-affirm that I was the one who synced up these videos, and really didn't have a solid reference for that. So there is no real evidence the driver noticed anything before the actual impact.

My comment had nothing to do with your syncing of the two videos.  It is purely on my observations of the face of the "safety driver".  They look to the left of the vehicle and there is only something of interest in that direction a second before the impact.

Image this is the picture of you that will be seen by millions. Yowza!