Author Topic: EEVblog #1068 - Autonomous Uber Incident Update  (Read 177095 times)

0 Members and 1 Guest are viewing this topic.

Offline oldway

  • Super Contributor
  • ***
  • !
  • Posts: 2172
Re: EEVblog #1068 - Autonomous Uber Incident Update
« Reply #25 on: March 28, 2018, 05:32:35 pm »
Here is what I posted on EEVblog #1066:

Quote
Re: EEVblog #1066 - Uber Autonomous Car Fatality - How?
« Reply #134 on: March 23, 2018, 10:08:13 PM »
This police inquery and its conclusions are staggering .... I wonder how is it possible that people do not notice that ....

The video presented does not have the scope of formal proof because that it is provided by the main interested party who can very well have manipulated it for its own interest ... (reduction of luminosity, cuts to make believe the impossibility of reaction and braking, etc ...)

Remember that this is not a surveillance camera video.

Several evidences seem to show that there is a problem with this video.

1) the road was lit, there is no appearance of lighting on the video.

2) The efficiency and the range of the headlights do not correspond to the headlights of a modern car. (Looks like headlights of a Ford Model T ... !!!)

3) There is no progressivity in the appearance of the pedestrian with his bike, it seems that portions of video are missing.

4) The sensitivity of the camera seems totally abnormal. Digital cameras are usually more sensitive to light than the human eyes .... There are even cameras capable of shooting in very low light levels.
Here, the video seems to have been made with the sensitivity of an old super 8 camera

Under such conditions, a reconstruction was essential to be done and it would have been necessary to publish the video made during the reconstitution so that one can compare.

Publish only the video of which there is no proof of authenticity, is biased and abnormal.

The result of the police investigation implies that the pedestrian would have committed suicide by knowingly crossing the road in a dark place in front of a vehicle that could not see and avoid it ..... this is no sense.

The reality would be rather that the pedestrian thought it was perfectly visible and that a motorist would certainly have braked to avoid it .... Of course, he committed an imprudence, but not a suicide.
Unfortunately, there was no driver, but an automatic system that failed to detect her and did not brake .....

UBER wants to pass this for a fatality (and succeeded), but for me, it is clear that it was not inevitable.

A pedestrian crossing a lighted road is visible from a distance enought for the driver to brake and at least to reduce the speed enough not to kill the pedestrian.
 

Offline tzok

  • Contributor
  • Posts: 48
  • Country: pl
Re: EEVblog #1068 - Autonomous Uber Incident Update
« Reply #26 on: March 28, 2018, 06:22:13 pm »
It totally failed to detect the cyclist at 6:17 in the video.
Same at 6:47... but this is even worse as the cyclist drives towards the car on a collision course.

« Last Edit: March 28, 2018, 06:27:22 pm by tzok »
 

Offline coppice

  • Super Contributor
  • ***
  • Posts: 8605
  • Country: gb
Re: EEVblog #1068 - Autonomous Uber Incident Update
« Reply #27 on: March 28, 2018, 07:15:37 pm »
Seem like Waymo are still going full steam ahead. They've ordered 20k Jaguar iPace EVs for delivery over the next 2 years.
 

Offline mattinson

  • Newbie
  • Posts: 1
  • Country: gr
Re: EEVblog #1068 - Autonomous Uber Incident Update
« Reply #28 on: March 28, 2018, 07:26:48 pm »
Maybe the programmer hated bikes  :o
 
The following users thanked this post: Koen

Offline thm_w

  • Super Contributor
  • ***
  • Posts: 6272
  • Country: ca
  • Non-expert
Re: EEVblog #1068 - Autonomous Uber Incident Update
« Reply #29 on: March 28, 2018, 07:30:06 pm »
It totally failed to detect the cyclist at 6:17 in the video.
Same at 6:47... but this is even worse as the cyclist drives towards the car on a collision course.



I only found a short newer video, but it detects the cyclist ok here: youtube.com/watch?v=jKfwHsHUdVc
Profile -> Modify profile -> Look and Layout ->  Don't show users' signatures
 

Offline Fungus

  • Super Contributor
  • ***
  • Posts: 16560
  • Country: 00
Re: EEVblog #1068 - Autonomous Uber Incident Update
« Reply #30 on: March 28, 2018, 08:25:07 pm »
Maybe the programmer hated bikes  :o

"It's not a bug, it's a feature!"   8)

 

Online Brumby

  • Supporter
  • ****
  • Posts: 12287
  • Country: au
Re: EEVblog #1068 - Autonomous Uber Incident Update
« Reply #31 on: March 28, 2018, 11:01:51 pm »
When I first noticed the cyclist I tried to find where he came from. As far as I can see he just teleported into the road because I can't see him ride from further back. It is wierd when you look close.

I noticed that too.

I suspect that the bike didn't get detected because it did not appear in the field of view for long enough - and when it did, it was off to the side and heading away from the area of interest.
 

Offline apis

  • Super Contributor
  • ***
  • Posts: 1667
  • Country: se
  • Hobbyist
Re: EEVblog #1068 - Autonomous Uber Incident Update
« Reply #32 on: March 28, 2018, 11:48:54 pm »
In this case it wasn't important though, since that cyclist wasn't anywhere near that car.
I completely disagree. The cyclist is much closer than some of the other pedestrians that the system did detect and highlight.  By the color of the boxes it appears that the system priorities each pedestrian but it appears to have completely overlooked the cyclist.  The only reason that he was "unimportant" was that he didn't pull in front of the automated car and get run over.  I don't know who's car this was but if someone had looked at this video sooner and realized that there was a problem with the system not recognizing cyclists then perhaps the accident in Arizona would have been prevented.  (And hind sight is always 20/20 as they say!)

It totally failed to detect the cyclist at 6:17 in the video.
In this case it wasn't important
Right, and the cyclist hit by the Uber car wasn't important either, not until the very last second...

I really can't understand your answer. How is the cyclist not worthy of a blue rectangle?
This video is NOT from a self-driving car, it's from a collision avoidance system only intended to act as a backup system if the human driver fails to act in time. There is a big difference.

This system is supposed to break in the last second if an object is directly in front of the car, when the human driver does not react in time. This system will not cause any accident if it misinterprets the video. (Actually, it might cause an accident if it believed there was something in front of the car when there really was not, and suddenly break. So in this case it's better it detects too few than too many.) Every time it breaks where the human driver failed to react in time it prevents an accident that would otherwise have happened. So even if it fails to detect some things in front of the car, the net result is that this system saves many lives. Naturally, the more the better, but failing to detect someone doesn't cause an accident here. If you don't use that system, the accident would have happened anyway. There isn't any downside to using it even if it's much less than 100% accurate (as long as it doesn't cause false positives). No one is 100% accurate though, certainly not humans. And in this case that bicycle was never in imminent danger of being hit by the car so it made no difference if the system saw it in this case.

The situation is very different for a self-driving car though. If a self-driving car does not realise something is in front of it, it will very likely crash into it. It's not the backup system, it is the primary driver (and the human "safety driver" is the backup). A self driving car not only tracks things in front of the car but everywhere around it, it's much more advanced. So my point is that such a system, using video only, while impressive, isn't good enough for a self-driving car (by itself). That's why e.g. Waymo also uses LIDAR since it's much easier to interpret the data from such a sensor than it is to interpret 2D images.

The fact that this much simpler collision avoidance system detected the victim in the Uber accident by only using the very bad video that was released by the police shows that the Uber car really should have had no problem to detect that person in time.
 

Offline ez24

  • Super Contributor
  • ***
  • Posts: 3082
  • Country: us
  • L.D.A.
Re: EEVblog #1068 - Autonomous Uber Incident Update
« Reply #33 on: March 29, 2018, 01:00:10 am »
  IMO the Sheriff in Arizona was much too quick to say that it was NOT Uber's fault.
Or told so by the governor
YouTube and Website Electronic Resources ------>  https://www.eevblog.com/forum/other-blog-specific/a/msg1341166/#msg1341166
 

Offline Fungus

  • Super Contributor
  • ***
  • Posts: 16560
  • Country: 00
Re: EEVblog #1068 - Autonomous Uber Incident Update
« Reply #34 on: March 29, 2018, 08:17:12 am »
This system is supposed to break in the last second if an object is directly in front of the car

nitpick: the word is "brake"

This system will not cause any accident if it misinterprets the video.

Red herring.

There's something wrong in the software if it can't see that bike.

 

Offline apis

  • Super Contributor
  • ***
  • Posts: 1667
  • Country: se
  • Hobbyist
Re: EEVblog #1068 - Autonomous Uber Incident Update
« Reply #35 on: March 29, 2018, 10:56:46 am »
This system will not cause any accident if it misinterprets the video.
Red herring.

There's something wrong in the software if it can't see that bike.
No, not something wrong, just not 100% correct. You can never get 100% accuracy here, humans can not either. How good it has to be depends on the application. If that collision avoidance system manages to prevent even 5% of all accidents that means it's saving many lives every year (i.e. even if it fails to trigger 95% of the time it's still beneficial).

For autonomous cars where the system replaces human drivers, you want the new system to perform better than humans (still doesn't have to be perfect though). As long as robot cars are safer than human operated cars it means lives will be saved.
 

Offline apis

  • Super Contributor
  • ***
  • Posts: 1667
  • Country: se
  • Hobbyist
 

Offline mikerj

  • Super Contributor
  • ***
  • Posts: 3233
  • Country: gb
Re: EEVblog #1068 - Autonomous Uber Incident Update
« Reply #37 on: March 29, 2018, 11:31:11 am »
By this logic, human drivers are not fit for purpose and should also be banned from roads, because this sort of accident happens all the time.

Some human drivers are not fit to drive, and should be banned from the roads.  This technology is claimed to improve safety - if it fails under what should have been a relatively trivial test case then it's not living up to its claims.
 

Offline G7PSK

  • Super Contributor
  • ***
  • Posts: 3859
  • Country: gb
  • It is hot until proved not.
Re: EEVblog #1068 - Autonomous Uber Incident Update
« Reply #38 on: March 29, 2018, 02:27:08 pm »
Uber have settled with the victims family:
https://www.theguardian.com/technology/2018/mar/29/uber-settles-with-family-of-woman-killed-by-self-driving-car

The Guardian article seems to have bought the story that the road was unlit and that was the reason for the collision. So it looks like ubers media management is largely working.
 

Offline orion242

  • Supporter
  • ****
  • Posts: 746
  • Country: us
Re: EEVblog #1068 - Autonomous Uber Incident Update
« Reply #39 on: March 29, 2018, 03:27:04 pm »
So it looks like ubers media management is largely working.

Not sure about that.  Two states have tossed them out and they quietly settle with the family who are likely under NDA now.  Sure seems like the blame is pointing to Uber, which is surprising.  I would have thought they tossed the safety driver under the bus for being distracted. 

Still waiting to see if AZ is going to press charges.  Paying off the family might make the state less interested in charging anyone and dealing with a case that will have tons of publicity and set precedents.
« Last Edit: March 29, 2018, 03:30:00 pm by orion242 »
 

Offline SilverSolder

  • Super Contributor
  • ***
  • Posts: 6126
  • Country: 00
Re: EEVblog #1068 - Autonomous Uber Incident Update
« Reply #40 on: March 29, 2018, 11:06:55 pm »
Perhaps it is a good design decision to have multiple systems on the car - made by different teams, so it doesn't repeat the same mistaken assumptions and broken implementations as the primary system?

Having a separate system to detect "Oh, crap, things are going out of control" is just as good a feature for an AI driver as it is for a human one.

Instead of catching human failings like getting distracted or getting drowsy, it would catch AI problems like design flaws or hard to detect sensor errors...

Certainly this present case is a very strong argument in favour of that approach.   It should be a requirement to have at least one working secondary collision avoidance system (made by a different team!) in order to be allowed to test on public roads...
 

Offline sakujo7

  • Contributor
  • Posts: 39
  • Country: au
Re: EEVblog #1068 - Autonomous Uber Incident Update
« Reply #41 on: March 29, 2018, 11:37:37 pm »
By this logic, human drivers are not fit for purpose and should also be banned from roads, because this sort of accident happens all the time.

Some human drivers are not fit to drive, and should be banned from the roads.  This technology is claimed to improve safety - if it fails under what should have been a relatively trivial test case then it's not living up to its claims.

Okay, so it should be banned for failing to live up to its claims then? In that case, Uber could simply stop claiming it's safer than humans and get back on the road.

Again, I personally think they failed miserably here and need to fix what went wrong. But I'm interested in the logic of what should/shouldn't be allowed. As Dave pointed out, people tend to hold machines (ie. something that was designed) to a different standard than humans. Get that standard wrong and it's either exploited by manufacturers, dangerous stuff goes out on the road (or in homes etc. this isn't specific to cars) and people are hurt, or it gets held back from a mass rollout even when it's slightly safer and could save lives but isn't yet perfect.
 

Offline Stray Electron

  • Super Contributor
  • ***
  • Posts: 2039
Re: EEVblog #1068 - Autonomous Uber Incident Update
« Reply #42 on: March 30, 2018, 12:58:15 am »

. But I'm interested in the logic of what should/shouldn't be allowed. As Dave pointed out, people tend to hold machines (ie. something that was designed) to a different standard than humans.

   Well isn't that the point of any tool? To do things BETTER than a human can?  What else is a wrench for? Or a volt meter? Or even a common hammer?   If they can't perform better than a human then they're useless.

   That's really the unlying argument regarding self driving cars.
 

Online Brumby

  • Supporter
  • ****
  • Posts: 12287
  • Country: au
Re: EEVblog #1068 - Autonomous Uber Incident Update
« Reply #43 on: March 30, 2018, 01:04:00 am »
If they can't perform better than a human then they're useless.

Define "better".

I am being quite serious about that.

One could argue using a hammer is "better" - until an improperly programmed hammer misses the nail and hits your thumb.  Is it still "better" in that case?
 

Offline Stray Electron

  • Super Contributor
  • ***
  • Posts: 2039
Re: EEVblog #1068 - Autonomous Uber Incident Update
« Reply #44 on: March 30, 2018, 01:11:03 am »
[

Still waiting to see if AZ is going to press charges.

   That will NEVER happen IMO. If Az presses charges it would have to be based on the argument  that the self driving car was inherently dangerous or were reckless.  A machine can't be reckless so that leaves dangerous. But if Az goes with that argument then the State and the governor could/should be considered complicit (ie involved with others in an illegal activity or wrongdoing) and as failing to do their sworn DUTY by allowing the cars on the public roads in the first place.  Remember the governor of Arizona issued an executive order allowing Uber to drive the cars on Arizona's roads.  A public trial is the last thing that the Governor and state officials want.
 

Offline Fungus

  • Super Contributor
  • ***
  • Posts: 16560
  • Country: 00
Re: EEVblog #1068 - Autonomous Uber Incident Update
« Reply #45 on: March 30, 2018, 01:22:18 am »
People can relate to crossing the road in front of an oncoming car driven by a human, sometimes it is done by making eye contact and confirming the driver has seen you. You cannot do that with an AV. That mistrust will be the Achilles heel of AV acceptance.

That's an interesting point.
 

Offline sakujo7

  • Contributor
  • Posts: 39
  • Country: au
Re: EEVblog #1068 - Autonomous Uber Incident Update
« Reply #46 on: March 30, 2018, 02:32:05 am »
Well isn't that the point of any tool? To do things BETTER than a human can?  What else is a wrench for? Or a volt meter? Or even a common hammer?   If they can't perform better than a human then they're useless.

Not necessarily. It's like any form on automation IMO. Even if it only works as well on average, it can still free up people to do other things, so it's not useless.
 
The following users thanked this post: NiHaoMike

Offline Stray Electron

  • Super Contributor
  • ***
  • Posts: 2039
Re: EEVblog #1068 - Autonomous Uber Incident Update
« Reply #47 on: March 30, 2018, 02:34:54 am »
If they can't perform better than a human then they're useless.

Define "better".

I am being quite serious about that.

One could argue using a hammer is "better" - until an improperly programmed hammer misses the nail and hits your thumb.  Is it still "better" in that case?

  So do you think that it's better for you to drive nails with your bare hands instead of using a hammer and hitting your thumb once in every, umm, let's say 1000 strikes?  Oh and yeah, "serious" because that's what your argument is.

 
Define "better".

   In this case; cheaper, smaller, faster, more accurately, more reliably, sooner and without constant human monitoring.  There are  other arguments in favor of an ideal automated driverless car but those will do for a start.   In Uber's case in Arizona, the system failed on ALL of those counts. 

    For crying out loud, they (Uber) even disabled the built-in safety system that came standard on the car!   That's about like taking the brakes out of a car and then allowing a test driver to drive it on the public highway "just to see what might happen".   That's not a test, it's deliberate recklessness.  Ask yourself, what would happen to you as a private citizen, in your own country, if you took the safety system out of your own car and then took it on the highway and ran over and killed a pedestrian because the built in system had been disabled?  In this country and I think most countries, you would be facing felony manslaughter charges and likely jail time; at the very minimum.   Bad enough that Uber's system failed miserably but there was NO excuse for them disabling the standard built in system.
 

Offline sakujo7

  • Contributor
  • Posts: 39
  • Country: au
Re: EEVblog #1068 - Autonomous Uber Incident Update
« Reply #48 on: March 30, 2018, 02:48:33 am »
If you're one of the 5%, I think you'd have a fair claim that it's not beneficial. And you can't compare accident rates between human drivers and autonomous vehicles. People aren't going to accept a machine with a firmware error or a sensor error making life and death decisions. People who don't understand or don't care about the underlying engineering won't cut it any slack.

I agree that people think that way. It's very unfortunate. The intervening period between "self driving cars have a lower accident rate" and "people trust self driving cars" could last for many, many years.

I suspect if you asked the average person "do you prefer 500 people dying to software bugs or 1000 people dying to drink driving", they would respond "I prefer 0 people dying", and continue to support the status quo.
 

Offline sakujo7

  • Contributor
  • Posts: 39
  • Country: au
Re: EEVblog #1068 - Autonomous Uber Incident Update
« Reply #49 on: March 30, 2018, 03:12:20 am »
For crying out loud, they (Uber) even disabled the built-in safety system that came standard on the car!   That's about like taking the brakes out of a car...
Bad enough that Uber's system failed miserably but there was NO excuse for them disabling the standard built in system.

I don't think that's entirely fair. It's more like removing the brakes then replacing them with your own supposedly-more-sophisticated brakes....and then not releasing the data when something goes wrong.

I expect their excuse is that the existing system would have interfered with their own.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf