EEVblog #1068 - Autonomous Uber Incident Update

[G7PSK]

If they can't perform better than a human then they're useless.

Define "better".

I am being quite serious about that.

One could argue using a hammer is "better" - until an improperly programmed hammer misses the nail and hits your thumb.  Is it still "better" in that case?

Even if the hammer hits your thumb it is still a better tool for putting nails in, try knocking six inch nails in with your bare hands, any tool is better even a rock compared to a human alone in this case.

As for comparing human drivers to machine drivers, the difference here is just because one human in ten thousand drives dangerously does not mean they all do, mass produce a machine that drives dangerously and every single one will do the same so a machine has to be of a higher standard as the results will always be the same or they should unless you have the engineering wrong, is that not what this discussion is all about engineering standards.

[apis]

No, not something wrong, just not 100% correct. You can never get 100% accuracy here, humans can not either. How good it has to be depends on the application. If that collision avoidance system manages to prevent even 5% of all accidents that means it's saving many lives every year (i.e. even if it fails to trigger 95% of the time it's still beneficial).

For autonomous cars where the system replaces human drivers, you want the new system to perform better than humans (still doesn't have to be perfect though). As long as robot cars are safer than human operated cars it means lives will be saved.

If you're one of the 5%, I think you'd have a fair claim that it's not beneficial. And you can't compare accident rates between human drivers and autonomous vehicles. People aren't going to accept a machine with a firmware error or a sensor error making life and death decisions. People who don't understand or don't care about the underlying engineering won't cut it any slack.

People can relate to crossing the road in front of an oncoming car driven by a human, sometimes it is done by making eye contact and confirming the driver has seen you. You cannot do that with an AV. That mistrust will be the Achilles heel of AV acceptance.

We are talking about different things I believe, you may have to read the previous posts I made as well to understand it.

The 5% refer to people that are saved by a collision avoidance system if it was only 5% accurate. The 95% that are not saved by the collision avoidance system will probably not care if it's installed or not since they would have been hit by the car either way. I.e. a collision avoidance system does not have to be perfect to be beneficial.

A collision avoidance system is not the same thing as an autonomous car. The standard for autonomous cars would have to be better than that of a human driver (in terms of accident statistics).

I'm convinced you can achieve much better performance from an autonomous car with today's technology, but it remains to be proven. Waymo have driven over 4 million miles on public roads without any fatalities so far though.

[apis]

I agree that people think that way. It's very unfortunate. The intervening period between "self driving cars have a lower accident rate" and "people trust self driving cars" could last for many, many years.

I suspect if you asked the average person "do you prefer 500 people dying to software bugs or 1000 people dying to drink driving", they would respond "I prefer 0 people dying", and continue to support the status quo.

Very good point. The question if people will accept autonomous cars has more to do with human psychology, politics and economics than the real benefits or performance of them.

There is always resistance to new technology, especially when it is perceived as a threat to peoples livelihood. E.g. https://en.wikipedia.org/wiki/Luddite. But whatever you think about people being replaced by robots, it's not a problem with the technology but rather with how society is organised.

No doubt there will be people living today who never accept them, but people are programmed by evolution to be lazy and penny-pinching so most people will probably use AV taxis if they are cheaper and more comfortable than the human operated taxis. The generations who grow up with autonomous vehicles will think it's natural and not give it a second thought.

Even more so when it comes to logistic AVs since then it will be some bean counter who makes the decision what company to use and not the people who have to deal with the actual autonomous trucks.

[nctnico]

People can relate to crossing the road in front of an oncoming car driven by a human, sometimes it is done by making eye contact and confirming the driver has seen you. You cannot do that with an AV. That mistrust will be the Achilles heel of AV acceptance.

That is a good point which has crossed my mind as well. However the fact is that if you can't make eye contact with the driver you shouldn't walk/drive in front of a car. This is something I have told my kids over and over.

[Fungus]

Very good point. The question if people will accept autonomous cars has more to do with human psychology, politics and economics than the real benefits or performance of them.

I suspect there's large segments of the population who see driving as a chore/problem and they'll all buy self-driving cars overnight if it lets them do other stuff instead.

I don't just mean people who'll sit there texting; Imagine being able to send the kids to school automatically, etc. Plenty of people would want that.

Then there's all the people who'll see it as a good way to drink more alcohol. There won't be any shortage of those.

etc.

None of those people will give a rats ass about pedestrians making eye contact.

[apis]

As for comparing human drivers to machine drivers, the difference here is just because one human in ten thousand drives dangerously does not mean they all do, mass produce a machine that drives dangerously and every single one will do the same so a machine has to be of a higher standard as the results will always be the same or they should unless you have the engineering wrong, is that not what this discussion is all about engineering standards.

Conversely, once you have proven that the autonomous car drives (hopefully much) safer than the average human, that means all cars will drive safer than the average human.

Another benefit from this is that if an accident do happen it will be added to the database of test cases and future revisions of the cars/software will (should) never make the same mistake again. I.e. the robot cars will just keep getting better and better (more experienced) with time.

And a robot is never tired, bored, drunk or playing with it's smartphone. They always perform at their highest capacity.

[apis]

I suspect there's large segments of the population who see driving as a chore/problem and they'll all buy self-driving cars overnight if it lets them do other stuff instead.

I don't just mean people who'll sit there texting; Imagine being able to send the kids to school automatically, etc. Plenty of people would want that.

Then there's all the people who'll see it as a good way to drink more alcohol. There won't be any shortage of those.

etc.

None of those people will give a rats ass about pedestrians making eye contact.

Hmm, probably true, so then it will be up to policymakers to make sure the autonomous cars actually are safer than human drivers.

[apis]

The real solution is to not design cities where 1hour plus commutes are necessary. Cars have taken us from living the dream of freedom and mobility to a nightmare of congestion and distance.

If you're in heavy AV traffic with all the cars locked together in some algorithmic synchronicity, you may as well be in a bus.

I agree with the need for better city planing. However you could also make busses autonomous for example, it doesn't have to be only cars.

There are those who claim that PRT systems (https://en.wikipedia.org/wiki/Personal_rapid_transit) are the most efficient form of mass transit available. A city with mainly autonomous taxis would be very similar to a PRT system.

And then there's the need for transportation outside of cities as well.

Also, in general, having robots take over boring and dangerous jobs means those people will be able to work with other (hopefully) safer and more stimulating things, so that's another benefit. If nothing else it means a more productive economy.

[SilverSolder]

You cannot do that [make eye contact] with an AV.

"cannot" is a strong word...   These days,  most cameras recognize eye contact in order to focus more accurately.

Is it impossible to imagine an AI driver noticing it is being looked at, and perhaps signaling back to the pedestrian somehow (a flash of the lights, a little blip of the horn, or some other way we haven't thought of yet)?

[ez24]

I suspect there's large segments of the population who see driving as a chore/problem and they'll all buy self-driving cars overnight if it lets them do other stuff instead.

You forgot old people.  Also I never heard of them being a "market" for the AV market.

[Bud]

"cannot" is a strong word...   These days,  most cameras recognize eye contact in order to focus more accurately.

....if you sit right in front of it. Useless for the purpose being discussed.

[t_i_t_o]

Volvo system fails too:

[nctnico]

If you're in heavy AV traffic with all the cars locked together in some algorithmic synchronicity, you may as well be in a bus.

That would be excellent because AVs will talk to eachother and not cause so much traffic jams. Traffic with human drivers has many similarities to turbulence of liquids or gasses flowing through a tube. If you take some time to sit next to a highway you'll notice traffic moves in pulses / groups. There is no continuous flow.

[Fungus]

There's a good point made in the comments in that Volvo video:

What if a gang of bad people* attacks your autonomous car?

You won't be able to drive away - the sensors will refuse to run them over! 


(*) Or zombies, or whatever...

[coppice]

There's a good point made in the comments in that Volvo video:

What if a gang of bad people* attacks your autonomous car?

You won't be able to drive away - the sensors will refuse to run them over! 


(*) Or zombies, or whatever...

I believe you can turn off AEB on all current production cars. There are special cases where you want your car to be able to touch or push on something.

[apis]

Volvo system fails too: (...)

The people in that video failed and deserve honorary mentions at the Darwin awards. As I've been saying, those collision avoidance system aren't perfect, but they do prevent many accidents. (And it is NOT the same as a self driving car which should not fail in that situation.)

Here are some (supposedly) independent statistics showing the general performance of collision avoidance systems:
http://www.iihs.org/media/3b08af57-8257-4630-ba14-3d92d554c2de/68O-Nw/QAs/Automation%20and%20crash%20avoidance/IIHS-real-world-CA-benefits-0218.pdf
For example: 56% less front-to-rear crashes with injuries. That is pretty impressive imo. Would be nice if that figure was 100% of course, but even so it's still a very good safety system that saves lives.

[Brumby]

I see a natural extension of the AV to be one of "co-operative" traffic flow.

From simple things like ensuring freeway entrance lane traffic merges into the main stream, without the odd "nervous Nellie" who slows everything down and risks accidents - to lane balancing and eliminating the slow down from "rubber neckers" gawking at an incident on the opposite side of the road.

Eventually, I can easily imagine swarm technology getting involved with route optimisation and traffic balancing across a whole city road network.  The existing roads may, in fact, be quite capable of handling a lot more traffic - if we can just get those pesky humans out from behind the wheel.

[SilverSolder]

"cannot" is a strong word...   These days,  most cameras recognize eye contact in order to focus more accurately.

....if you sit right in front of it. Useless for the purpose being discussed.

Here is a video that shows where facial recognition is heading...   hint, achieving eye contact is probably the least of its capabilities...

https://videos.posttv.com/washpost-production/The_Washington_Post/20171222/5a3d5cd1e4b0330b121d744b/5a4fd674e4b0cb2c842a15d3_1439412153584-wn5qra_t_1515181698375_640_360_600.mp4

[G7PSK]

"cannot" is a strong word...   These days,  most cameras recognize eye contact in order to focus more accurately.

....if you sit right in front of it. Useless for the purpose being discussed.

Here is a video that shows where facial recognition is heading...   hint, achieving eye contact is probably the least of its capabilities...

https://videos.posttv.com/washpost-production/The_Washington_Post/20171222/5a3d5cd1e4b0330b121d744b/5a4fd674e4b0cb2c842a15d3_1439412153584-wn5qra_t_1515181698375_640_360_600.mp4

Now that really is Big Brother 1984. Does any one really want to go that far although I am sure it will get foisted on us all eventually.

[apis]

Now that really is Big Brother 1984. Does any one really want to go that far although I am sure it will get foisted on us all eventually.

Hmm, isn't the UK pioneering that kind of system. It's 20 years since I was in London but already then I couldn't help to notice there were cameras everywhere. Since then I've learned they are networked and used by the UK government to track people, and they use facial recognition software as well. At that time they said the facial recognition wasn't good enough, it caused too many false positives to be truly useful, but that was many years ago and I couldn't help think that maybe they wanted to downplay the effectiveness a bit. I'm sure the system have and will continue to 'improve' though.

Of course it's more terrifying if used by a totalitarian regime, but as history shows liberal democracies can turn totalitarian rather quickly and then you already have all that nifty surveillance gear in place. But this is getting far from the topic of the tread.

Anyway, detecting eye contact is probably well within the realm of possibility for a neural network, but maybe you just need some way for the car to signal that it believes you are about to cross the road and that it gives you the right of way. That should not be any more complicated than adding the signalling device since the software already keeps track of that.

[Marco]

Terrorism threats are increasing, from multiculturalism, to ease of information (internet) and accessibility of internet technology (RC/UAV).

It's inevitable that surveillance will increase too.

[apis]

A Tesla X with the autopilot engaged crashed last week causing another fatality...

https://www.tesla.com/blog/update-last-week%E2%80%99s-accident

[Marco]

Either Apple is really scraping the bottom of the barrel with their engineers or this sounds a bit like a suicide.

Before getting into the details of Tesla's statement, what happened? On March 23  a Tesla Model X crashed on a South Bay freeway -- at the Highway 101 and Highway 85 connector in Mountain View -- causing the death of 38-year-old San Mateo resident, Apple software engineer Walter Huang.

Huang's family reported that he alerted Tesla's service department to a big problem with his Model X's Autopilot. As the Mercury News reported, Huang's family asserted that he "had taken the car to a dealer several times and complained that the function kept steering the car toward the highway divider into which he crashed."

"Hey this autopilot keeps trying to murder me, lets keep using it."?

PS. autopilot is a deceptive name and I hope Tesla gets taken to the cleaners for their continued use of the term.

PPS. taking a look at the stock price this was a 8 billion dollar accident, Tesla isn't kidding when they say the level of damage is unprecedented.

[nctnico]

Huang's family reported that he alerted Tesla's service department to a big problem with his Model X's Autopilot. As the Mercury News reported, Huang's family asserted that he "had taken the car to a dealer several times and complained that the function kept steering the car toward the highway divider into which he crashed."

"Hey this autopilot keeps trying to murder me, lets keep using it."?

Agreed. Tesla's 'auto pilot' is nothing more than a fancy cruise control. If you don't understand that then Darwin has an award for you. Especially if the driver complained a few times about it and KNEW that the auto pilot had problems with this specific section of road.  If the latter is really true then the Darwin award is well deserved.

[Marco]

I wonder what's going to happen when an handsfree driver on "autopilot" finally kills someone other than the occupants. If it happens in the wrong place and there is some enthusiastic prosecutor I could see things going very badly for Tesla.

PS. if it's a roadworker very very badly, if it's a cop ... nice knowing you Tesla.

PPS. thinking about it, I suspect Tesla is playing this dangerous game out of necessity. Autopilot is probably what mostly sells their cars at this point, there's only so many rich hippies and value signallers. If it backfires it could really be the end of Tesla.