EEVblog #1068 - Autonomous Uber Incident Update

[nctnico]

I'm wondering why Tesla's autopilot doesn't steer the car onto the hard shoulder (or the slowest lane) and then just stops (or drives very slow with the alarm lights blinking) when the driver isn't driving the car. IMHO that would be the best (safest) way for the car to deal with such a situation.

edit: One more additional remark: what I describe above would also help in case the driver becomes so ill he/she can no longer control the car. We shouldn't rule out that is a possible cause of the Tesla crash last week.

[Stray Electron]

Well isn't that the point of any tool? To do things BETTER than a human can?  What else is a wrench for? Or a volt meter? Or even a common hammer?   If they can't perform better than a human then they're useless.

Not necessarily. It's like any form on automation IMO. Even if it only works as well on average, it can still free up people to do other things, so it's not useless.

   "Better" would include things like freeing up human time.  For example, we could use a hand saw to cut wood but a hand held, power driven circular saw does it quicker.  That's why we use power saws.  The same can apply to a power driven drill vs a hand drill, and dozens of other examples including things like electric mixers that we use in the kitchen.  They don't necessary do a better job, mechanically speaking, than a manual power unit but they do it faster.

   Using a power saw is a apt comparison to the driverless cars.  Thousands of people are injured every year with power saws but we continue to use them because we consider the potential danger to be worth the risk because of the time savings.  The question regarding driverless cars is how much risk and losses are we willing to accept as a society in return for the time saved by not having to manually drive a car?

[G7PSK]

Tesla was told to stop using "Autopilot" By the German authorities in 2016.

http://www.latimes.com/business/autos/la-fi-tesla-germany-autopilot-20161017-snap-story.html

[Marco]

Agreed. Tesla's 'auto pilot' is nothing more than a fancy cruise control.

It's an intentionally dangerous fancy cruise control. It's build to offer comfort in exchange for risk with Tesla being intensely disingenuous about that fact. When Tesla compares it's safety it doesn't compare it to cars with modern adaptive cruise control, lane drift warning and brake assist ... it only compares it to ye average car.

When that risk finally kills or maims an innocent bystander it's going to be Uber v2 and a mere 8 billion in stock valuation vanishing will be a drop in the pond.

[Fungus]

I'm wondering why Tesla's autopilot doesn't steer the car onto the hard shoulder (or the slowest lane)

Can it actually change lanes? I thought it was just a fancy lane-follower.

[Fungus]

PS. autopilot is a deceptive name and I hope Tesla gets taken to the cleaners for their continued use of the term.

It's actually a very accurate name. The problem is that joe public doesn't know what an "autopilot" is.

PPS. taking a look at the stock price this was a 8 billion dollar accident, Tesla isn't kidding when they say the level of damage is unprecedented.

Stock price != real money.

[Fungus]

Another thing I haven't seen discussed anywhere is how easy these cars are to fool.

eg. What happens if a bunch of youtube idiots make some fake "Stop" signs and goes out to the autobahn to hold them up?

A human driver will easily ignore them but what will a robot car do? Emergency braking?

[Marco]

The problem is that joe public doesn't know what an "autopilot" is.

They are selling to joe public, they know they are selling to joe public, they know they are being deceptive.

[NiHaoMike]

Another thing I haven't seen discussed anywhere is how easy these cars are to fool.

eg. What happens if a bunch of youtube idiots make some fake "Stop" signs and goes out to the autobahn to hold them up?

A human driver will easily ignore them but what will a robot car do? Emergency braking?

Rumor says that if you place a beautiful naked woman beside a busy street, it won't be long before there are multiple crashes. (How will the Mythbusters test that in a safe and ethical way?) The self driving cars will probably recognize her as just another person.

[coppice]

Another thing I haven't seen discussed anywhere is how easy these cars are to fool.

eg. What happens if a bunch of youtube idiots make some fake "Stop" signs and goes out to the autobahn to hold them up?

A human driver will easily ignore them but what will a robot car do? Emergency braking?

Rumor says that if you place a beautiful naked woman beside a busy street, it won't be long before there are multiple crashes. (How will the Mythbusters test that in a safe and ethical way?) The self driving cars will probably recognize her as just another person.

On a more practical level, autonomous cars should largely eliminate the regular pileups on motorways caused by people looking at an earlier pileup on the opposite carriageway. 

[apis]

Another thing I haven't seen discussed anywhere is how easy these cars are to fool.

eg. What happens if a bunch of youtube idiots make some fake "Stop" signs and goes out to the autobahn to hold them up?

A human driver will easily ignore them but what will a robot car do? Emergency braking?

Rumor says that if you place a beautiful naked woman beside a busy street, it won't be long before there are multiple crashes. (How will the Mythbusters test that in a safe and ethical way?) The self driving cars will probably recognize her as just another person.

I've also heard that billboards with ads for underwear increase the probability for crashes!

How easy it is to fool an autonomous car is an interesting question. I don't think anyone can answer that except people who actually work with it since I assume it's not something the companies divulge freely.

I think someone mentioned that in the early days the google cars would think a life-size cutout of a person (i.e. ads) next to the road were real pedestrians, and if they were standing next to a zebra crossing, it would cause the car to think they were about to cross and slow down/stop. But these days they can tell the difference apparently.

There was also a demo of a facial recognition program in the tv-segment about Chinese surveillance technology that was posted above. In that example the software was able to tell that a life size printed image of a face was not a real face of a living person. What it takes to fool that system wasn't clear though. But if it was not a problem that some facial recognition systems are easy to fool with just a picture they would not have featured that technology I guess.

It will probably be possible to fool robot cars to some extent, but I assume they prioritise safety over traffic rules at least. So if following a law would cause a crash, or conversely if breaking a law means it can avoid an almost certain crash, the car would choose to break the law. But to program such behaviour into the software sounds a lot like the developers are planning to break the law. That kind of issues are probably giving them and their lawyers a headache.

I've heard that they have to bend the rules a little just to be able to drive normally. For example if a traffic sign just turned yellow you should stop if possible, but when they did that other cars behind them often did not keep enough distance and would sometimes crash into them because they wouldn't expect anyone to stop suddenly in that situation. Another problem was getting onto a busy freeway or traversing a busy intersection, if they followed the rules perfectly they would have to wait forever  which is not what human drivers do.

If it meant avoiding a serious accident I think most would agree it is acceptable to break the law (sometimes even the law itself says so). But what about less serious accidents, or when both actions could cause an accident but breaking the law is less likely to do so. etc. It's not an easy problem to solve.

[thm_w]

Another thing I haven't seen discussed anywhere is how easy these cars are to fool.

eg. What happens if a bunch of youtube idiots make some fake "Stop" signs and goes out to the autobahn to hold them up?

A human driver will easily ignore them but what will a robot car do? Emergency braking?

Seriously?

[mikerj]

Another thing I haven't seen discussed anywhere is how easy these cars are to fool.

eg. What happens if a bunch of youtube idiots make some fake "Stop" signs and goes out to the autobahn to hold them up?

A human driver will easily ignore them but what will a robot car do? Emergency braking?

Seriously?

That doesn't actually address the question does it?

[fsedano]

Hi, Tesla owner here so I can offer some insights.

Yes, system is a a glorified fancy cruise control, but it uses two sources of info: Edges/lines to learn about the road and tracking other cars via cameras/radar. I.e. it tracks cars around you and moves inside your lane or brakes if it detects any of them is moving into your way.

For cars you're following, it does bounce radar signal underneath the car next to you so it can detect when the car which is 2 position ahead of you starts to brake.



I've experienced this on my own car, and it's a big saver. My personal experience using this is I'm way less tired when doing a long trip. Yes I still pay attention, always have hands on the wheel, but it removes you the need to do all small corrections you're doing in the background, and after a long trip you feel more rested.

Regarding what the car does when you don't pay attention, it brakes slowly, put the warning on and then stops.



If instead of this case (not reacting at all) you react after some time, after the 2nd time you do that, it does not allow you to engage autopilot until you fully reset the car.

Huang's family reported that he alerted Tesla's service department to a big problem with his Model X's Autopilot. As the Mercury News reported, Huang's family asserted that he "had taken the car to a dealer several times and complained that the function kept steering the car toward the highway divider into which he crashed."

"Hey this autopilot keeps trying to murder me, lets keep using it."?

Agreed. Tesla's 'auto pilot' is nothing more than a fancy cruise control. If you don't understand that then Darwin has an award for you. Especially if the driver complained a few times about it and KNEW that the auto pilot had problems with this specific section of road.  If the latter is really true then the Darwin award is well deserved.

[Fungus]

Seriously?

Yes, seriously.

I can't imagine how anybody might think that throwing stuff off bridges might be fun but they still do it.

I can imagine people messing around to see how the dumbass computers react, eg. walking around town in a stop-sign t-shirt and filming it, etc. ("Not my fault, officer, I was just wearing a t-shirt")

[thm_w]

Yes, seriously.

I can't imagine how anybody might think that throwing stuff off bridges might be fun but they still do it.

I can imagine people messing around to see how the dumbass computers react, eg. walking around town in a stop-sign t-shirt and filming it, etc. ("Not my fault, officer, I was just wearing a t-shirt")

If someone is going to wave around a fake stop sign, they are breaking the law as it is, for example:

California Vehicle Code section 21465 states, "No person shall place, maintain, or display upon, or in view of, any highway any unofficial sign, signal, device, or marking, or any sign, signal, device or marking which purports to be or is an imitation of, or resembles, an official traffic control device or which attempts to direct the movement of traffic or which hides from view any official traffic control device."

[james_s]

I wonder if that would apply to a t-shirt with a picture of a stop sign on it? What if it didn't actually say "Stop" but some clever word that resembles it at a glance? The law is obviously intended to prevent people from putting up fake traffic control signs.

[metrologist]

Haven't you seen those little plastic kid figures placed at the edge of neighborhood roads that may be holding a flag and say SLOW! on the side?

Not sure how school crossing guards are viewed.

roadtrafficsigns.com

Private drive no entery...

fartkontrol

[Fungus]

I wonder if that would apply to a t-shirt with a picture of a stop sign on it? What if it didn't actually say "Stop" but some clever word that resembles it at a glance? The law is obviously intended to prevent people from putting up fake traffic control signs.

I get 127 million hits for "stop sign t-shirt" so that means there must already be people walking around wearing them:

https://encrypted.google.com/search?q=stop+sign+t-shirt

[apis]

A person wearing a stop t-shirt wouldn't fool neither human nor any half decent computer program. And I'm not sure the google cars care about that at all or if they just use their internal map data? There might still be ways to fool autonomous cars though, but the important questions are these:

* Would it be a big problem?
Probably not. Certainly not something that significantly compromise safety.

* Is it better than humans? How easy is it to fool humans by placing fake signs and pedestrians. Are the autonomous cars worse than humans?
This is a little trickier to answer because humans and computers will have somewhat different strengths and weaknesses here.

* Should you not adopt autonomous vehicles because of it?
If autonomous vehicles are much safer (as in causes significantly less accidents) the benefits far outweigh any minor annoyances caused by such pranks.

[apis]

Haven't you seen those little plastic kid figures placed at the edge of neighborhood roads that may be holding a flag and say SLOW! on the side?

You mean something like this:

I guess that could be difficult to handle for an autonomous car if it was placed on the road or at a zebra crossing. If it's a common phenomena like this particular sign they could teach the cars to recognise them and treat them as a warning sign though. I'm pretty sure it would be possible for malicious people to create dolls that would fool the cars, but such dolls could probably fool people as well. What would happen is the cars would stop or slow down needlessly, whereas a less cautious human driver would not.

[f4eru]

There was a case where they updated the software to identify speed limitations on the back of trucks in EU that replicate road signs to not trigger any more after an update.
We can imagine that the t-shirt problem can be handled the same once identified.

[orion242]

They might want to focus on identifying and avoiding people and concrete barriers first...

[apis]

They might want to focus on identifying and avoiding people and concrete barriers first...

I'm sure Tesla and Uber are doing just that. But there are many other companies/teams, including those who have driven longer than Uber, who haven't had any fatal accidents yet. It is counterproductive to hold the other teams responsible for Uber's failure.

If this tech means a significantly reduction in the number of traffic related accidents and deaths (as all evidence indicate it will) everyone is better off the faster we can have autonomous cars replace human drivers on the streets. Even the Tesla autopilot (which is just an advanced cruise control) is preventing more accidents than it is causing. According to Tesla even the drivers who misuse the autopilot (like the drivers involved in the fatal accidents) are better off if they continue to use it compared to if they turn it off.

[james_s]

Personally if I don't get to drive the car myself I'd rather just take the bus or a cab though. The primary reason I own a car is that I enjoy driving it, otherwise I couldn't justify the expense.