Man trapped inside driverless car as it spins in circles

[magic]

You can have two kinds of stop button.
No, I don't trust AI blob, that nobody understands how it works. There is regular reviewed and certified software to stop things. But no, let's hand over all control.

OK, a different question. What do you trust more: AI, or a random idiot driving in an autonomous taxi right in front of you and being able to stop it rapidly at any time?

[Siwastaja]

OK, a different question. What do you trust more: AI, or a random idiot driving in an autonomous taxi right in front of you and being able to stop it rapidly at any time?

The whole discussion is ridiculous (usual "tszaboo entered infinite loop"). Even currently, passenger taking controls of a car (e.g. to stop it) is a criminal offense at least here, and probably in most countries. The same would obviously extend to self-driving cars, and any sensible regulation would directly forbid such "immediate e-stop" buttons for reasons presented here by many.

But it appears that the fact we are discussing a taxi passenger seems to go unnoticed by tszaboo no matter how many times it's mentioned.

[jonovid]

OK, a different question. What do you trust more: AI, or a random idiot driving in an autonomous taxi right in front of you and being able to stop it rapidly at any time?

The whole discussion is ridiculous (usual "tszaboo entered infinite loop"). Even currently, passenger taking controls of a car (e.g. to stop it) is a criminal offense at least here, and probably in most countries. The same would obviously extend to self-driving cars, and any sensible regulation would directly forbid such "immediate e-stop" buttons for reasons presented here by many.

But it appears that the fact we are discussing a taxi passenger seems to go unnoticed by tszaboo no matter how many times it's mentioned.

taxi passenger using the emergency stop button would be a criminal offense in a Non emergency. same as on a passenger train
 an intercom button would be a way to get help in most unfortunate circumstances in which one finds oneself , including dead phone or taxi app not working or vehicle is going in the wrong direction.

a map with large text on the front centre console screen would reassure back seat passengers in the taxi that the vehicle is functioning properly & under full control of the circumstances.

[tszaboo]

OK, a different question. What do you trust more: AI, or a random idiot driving in an autonomous taxi right in front of you and being able to stop it rapidly at any time?

The whole discussion is ridiculous (usual "tszaboo entered infinite loop"). Even currently, passenger taking controls of a car (e.g. to stop it) is a criminal offense at least here, and probably in most countries. The same would obviously extend to self-driving cars, and any sensible regulation would directly forbid such "immediate e-stop" buttons for reasons presented here by many.

But it appears that the fact we are discussing a taxi passenger seems to go unnoticed by tszaboo no matter how many times it's mentioned.

taxi passenger using the emergency stop button would be a criminal offense in a Non emergency. same as on a passenger train
 an intercom button would be a way to get help in most unfortunate circumstances in which one finds oneself , including dead phone or taxi app not working or vehicle is going in the wrong direction.

a map with large text on the front centre console screen would reassure back seat passengers in the taxi that the vehicle is functioning properly & under full control of the circumstances.

"This is too logical. We must remove the button. What if someone presses it?"

[Analog Kid]

Such an e-stop button is only possible with a driver who can immediately take over, what can you not understand about this being inappropriate in a vehicle where the computer is the only driver?

What an utterly ridiculous statement.

OK, the goddamn car has a computer, correct? After all, that's what's driving it.
So let's say there's a giant "STOP" button accessible to the poor rider in case the thing goes haywire.
That button will be connected to said computer.

Who says that that means that when the rider pushes it, the car comes to a screeching halt with 70 MPH traffic right behind it?
The goddamn COMPUTER can make a reasonable, moderate brake application, which will of course turn the brake lights on, and the car can come to a safe stop, even if it's in the middle of high-speed traffic.

These things can be programmed into the car's computer, no?

[tom66]

Such an e-stop button is only possible with a driver who can immediately take over, what can you not understand about this being inappropriate in a vehicle where the computer is the only driver?

What an utterly ridiculous statement.

OK, the goddamn car has a computer, correct? After all, that's what's driving it.
So let's say there's a giant "STOP" button accessible to the poor rider in case the thing goes haywire.
That button will be connected to said computer.

Who says that that means that when the rider pushes it, the car comes to a screeching halt with 70 MPH traffic right behind it?
The goddamn COMPUTER can make a reasonable, moderate brake application, which will of course turn the brake lights on, and the car can come to a safe stop, even if it's in the middle of high-speed traffic.

These things can be programmed into the car's computer, no?

Please go back and read the context again.

Such a safe stop button DOES exist on every SDC I'm aware of... but that wasn't tszaboo's concern.  They wanted a system that would immediately disengage the SDC and apply brakes, bringing the car to a halt quickly regardless of the driving context.    That was what was linked to (when button is pressed, actuators for the computer are disabled.)  For a highway, the car should pull over to the shoulder/emergency bay, or leave at the next junction, not stop in an active lane.

[Analog Kid]

Such a safe stop button DOES exist on every SDC I'm aware of... but that wasn't tszaboo's concern.  They wanted a system that would immediately disengage the SDC and apply brakes, bringing the car to a halt quickly regardless of the driving context.    That was what was linked to (when button is pressed, actuators for the computer are disabled.)  For a highway, the car should pull over to the shoulder/emergency bay, or leave at the next junction, not stop in an active lane.

Obviously the STOP button will be connected to the car's computer.
Meaning that at least initially the computer will be handling the situation.
The computer will be the one applying the brakes. OBVIOUSLY. As I'm sure @tszaboo is well aware.
The computer can apply the brakes, gently, bringing the car to a halt quickly. Regardless of the driving context, yes, because at this time there exists a fault situation in which the computer's judgment may be (likely is) impaired.
So the prudent thing to do, if you're in a car barreling down the road at 70 MPH towards an obstacle, is to get the fucking thing stopped ASAP.
If it can also pull over to the shoulder or otherwise in a safe place, fine.
But at this point it should drop out of "self-driving" mode.

[Siwastaja]

Who says that that means that when the rider pushes it, the car comes to a screeching halt with 70 MPH traffic right behind it?

That is what tzaboo is literally saying.

The goddamn COMPUTER can make a reasonable, moderate brake application, which will of course turn the brake lights on, and the car can come to a safe stop, even if it's in the middle of high-speed traffic.

These things can be programmed into the car's computer, no?

Obviously, yes. This is what everybody else is saying.

[SiliconWizard]

Who says that that means that when the rider pushes it, the car comes to a screeching halt with 70 MPH traffic right behind it?

That is what tzaboo is literally saying.

The goddamn COMPUTER can make a reasonable, moderate brake application, which will of course turn the brake lights on, and the car can come to a safe stop, even if it's in the middle of high-speed traffic.

These things can be programmed into the car's computer, no?

Obviously, yes. This is what everybody else is saying.

Yep. In which case, it's merely a "hint" from the user - the machine is "free" not to do it if it determines that it's "unsafe" to do so for long enough. If a car's software can make it spin in circles, what makes anyone think that pressing a "stop" button would be honored at all?

And, there's always bugs that would make it ignore entirely, or precisely honor it at the wrong moment.

I am overall baffled by how much trust some people have in software. It's like they have never used any? Autopilots in airplaines are a fallacious equivalence: they are very different from what's at the core of autonomous cars. Plane autopilots merely maintain a given set of parameters (speed, altitude, climb rate), they can also follow a route (but with much more leeway than what's allowed on the road obviously, where just 50 cm of error in some cases could lead to a catastrophe). That's it. There's no obstacle avoidance of any kind, and they'll disconnect themselves automatically at the first inconsistent conditions or readings (unless there's a severe bug). All that under the constant surveillance of one or two highly-trained pilots. Almost nothing to do with what happens in an "autonomous" car with no driver. And, they are designed under very strict processes (although even that can occasionally malfunction, see Boeing, but that's still an exception rather than the norm). I have not really seen the same level in car companies, and autonomous vehicles so far have too little regulation anyway to be able to enforce as much as we can enforce for airplanes. That can't go well.

[magic]

Yep. In which case, maybe you are opposed to FSD cars as such, rather than to the way their stop button works or doesn't work.

If a car next to you is already driven by buggy software, adding a button to stop it immediately (and nothing more than the button) makes the car even more likely to do something dangerous.

[tszaboo]

Yep. In which case, it's merely a "hint" from the user - the machine is "free" not to do it if it determines that it's "unsafe" to do so for long enough. If a car's software can make it spin in circles, what makes anyone think that pressing a "stop" button would be honored at all?

And, there's always bugs that would make it ignore entirely, or precisely honor it at the wrong moment.

I am overall baffled by how much trust some people have in software. It's like they have never used any? Autopilots in airplaines are a fallacious equivalence: they are very different from what's at the core of autonomous cars. Plane autopilots merely maintain a given set of parameters (speed, altitude, climb rate), they can also follow a route (but with much more leeway than what's allowed on the road obviously, where just 50 cm of error in some cases could lead to a catastrophe). That's it. There's no obstacle avoidance of any kind, and they'll disconnect themselves automatically at the first inconsistent conditions or readings (unless there's a severe bug). All that under the constant surveillance of one or two highly-trained pilots. Almost nothing to do with what happens in an "autonomous" car with no driver. And, they are designed under very strict processes (although even that can occasionally malfunction, see Boeing, but that's still an exception rather than the norm). I have not really seen the same level in car companies, and autonomous vehicles so far have too little regulation anyway to be able to enforce as much as we can enforce for airplanes. That can't go well.

They really really don't get the difference between machine learning models and regular software.
That leads to these pointless discussions where they assume what a car is supposed to do when you disconnect the "AI" part. Good thing that the standards and regulations are written by people who actually understand software.
For example the EU regulations prevented Waymo to enter this market.

[magic]

Your EU car can still stop randomly in the middle of a highway on its own initiative, and EUSSR actually requires it to be so in all new cars.

edit
Besides, Waymo is Murrican and Murrica is Bad. As soon as Volksvagen comes up with self driving cars, it will turn out that they perfectly meet standards and regulations written by sensible EUrocrats who understand software, their cars will hit the roads, and then it will be same shitshow as in the US.

[tszaboo]

Your EU car can still stop randomly in the middle of a highway on its own initiative, and EUSSR actually requires it to be so in all new cars.

edit
Besides, Waymo is Murrican and Murrica is Bad. As soon as Volksvagen comes up with self driving cars, it will turn out that they perfectly meet standards and regulations written by sensible EUrocrats who understand software, their cars will hit the roads, and then it will be same shitshow as in the US.

https://en.wikipedia.org/wiki/Straw_man

[Zero999]

Trains also (usually) operate in environments that use block signalling, which prohibits other trains from entering their 'block' if they are still present.  Applying the brakes in an emergency is therefore usually safe, though there are environments where it is not ideal to stop in, such as inside tunnels.

The same cannot be said for cars, stopping on a highway is extremely dangerous, and even stopping suddenly on a normal road carries dangers if people are not expecting it.

In a world where only SDCs exist, an e-stop button may be appropriate because that car can signal to other vehicles of its need to suddenly stop... But right now SDCs have to cope with human drivers.

So what if a car stops if a big red button is pressed? You will be passed drunk out on the back seat of your driverless car, saying nothing so that the police doesn't arrest you for saying something offensive. Why do you care?
You will be perfectly safe there. Machines don't make mistakes according to you anyway.

So what? Let me know next time how well things go for you when you press the brakes to zero on the motorway, let's see how long you like being stopped in a live lane with traffic hurtling around you at 120 kph.  Because that's exactly the kind of situation you'd create if your e-stop was just making the car suddenly stop.

That's not what a emergency stop should do in a driverless car. It should pass control over to another emergency control system, which would stop the car safely, preferably pull over to the edge of the road and come to a gradual stop.

[Analog Kid]

That's not what a emergency stop should do in a driverless car. It should pass control over to another emergency control system, which would stop the car safely, preferably pull over to the edge of the road and come to a gradual stop.

That's exactly what I was trying to get across in my post above.
Obviously the STOP button press is going to have to be handled by some kind of electronic system:
it's not going to be connected to a big honking solenoid that engages the brakes fully and immediately, coming to a screeching halt.
It will be another computer system, separate from the "self-driving" one, that manages the emergency braking.
In some situations it may be OK to stop dead on the roadway after a rapid but gradual deceleration, rather than hit an obstacle.

[Njk]

If the procedure mandates a pre-ride briefing for customers? In the US, I was usually provided with cheapest tickets for regional flights. Sometimes, when the plane was almost all set for take off, a cabin lady approached me and started small talks for some time (to make sure I'm a sane person and can understand her). Then she said that on this flight I'm not just a passenger but actually a crew member because of the very important responsibility. Since my seat is located close to the emergency exit hatch, I'll have to open it in the case of necessity. So I have to know how to do that. Then she'd indoctrinated me in that matter and checked that my understanding is correct. That approach to safety seems very reasonable.

[Zero999]

That's not what a emergency stop should do in a driverless car. It should pass control over to another emergency control system, which would stop the car safely, preferably pull over to the edge of the road and come to a gradual stop.

That's exactly what I was trying to get across in my post above.
Obviously the STOP button press is going to have to be handled by some kind of electronic system:
it's not going to be connected to a big honking solenoid that engages the brakes fully and immediately, coming to a screeching halt.
It will be another computer system, separate from the "self-driving" one, that manages the emergency braking.
In some situations it may be OK to stop dead on the roadway after a rapid but gradual deceleration, rather than hit an obstacle.

That's how emergency stop systems on many machines work. For example it might have conveyor belt which suddenly stopping could cause items to flip and fall off. When someone hits the emergency stop, the safety system sends the variable speed drive controlling the motor to decelerate at a controlled rate, before cutting the power. The safety controller has a separate processor to the main control system, so it will continue to function, if the main controller fails.

[Analog Kid]

If the procedure mandates a pre-ride briefing for customers? In the US, I was usually provided with cheapest tickets for regional flights. Sometimes, when the plane was almost all set for take off, a cabin lady approached me and started small talks for some time (to make sure I'm a sane person and can understand her). Then she said that on this flight I'm not just a passenger but actually a crew member because of the very important responsibility. Since my seat is located close to the emergency exit hatch, I'll have to open it in the case of necessity. So I have to know how to do that. Then she'd indoctrinated me in that matter and checked that my understanding is correct. That approach to safety seems very reasonable.

Very reasonable, yes.
That just ain't gonna happen with any kind of ride in a car, self-driving or no.
It would have to be mandated by a federal agency, and that's not in the cards.

[Njk]

It would have to be mandated by a federal agency, and that's not in the cards.

Perhaps it can be mandated by the driverless car operator's insurance company

[Siwastaja]

For example it might have conveyor belt which suddenly stopping could cause items to flip and fall off. When someone hits the emergency stop, the safety system sends the variable speed drive controlling the motor to decelerate at a controlled rate, before cutting the power.

Yeah. But a passenger stop button for a self-driving car is unfortunately going to be more complicated than that. There are areas where stopping is explicitly forbidden; then there is the real-world traffic culture where if you stop in the middle of the highway you are going to cause a massive chain disaster, even if you slow down gradually and blink whatever amount of brake lights, because large part (possibly over 90% or so!) of human drivers are incapable of following traffic rules related to safe driving distances and society has given up trying to rectify it, instead accepting disaster every time someone stops in the middle of a busy highway, and sanctioning the one who stops even though everybody else did commit even more serious crimes. Unless we replace all cars with self-driving cars overnight, we can't just change this culture; so gradual stop in the middle of the road is out of question as well.

Just like a human driver needs to, self-driving solution also needs to understand at least four different operating modes:
* Sudden emergency brake,
* Gradual stop,
* Pull over on the side of the road and gradual stop there,
* Keep on driving to find a place where one can stop or pull over,

and choosing between them is not very simple. Intelligence, human or artificial, is needed for that.

[magic]

To put it shortly, stopping safely isn't significantly easier than driving safely, and any system for it won't be significantly smaller and more auditable.

I expect every vendor to simply put this task on the main self-driving system, ever time. Sorry to disappoint

[Siwastaja]

Really, the best way forward with self-driving cars is to mimic human intelligence and take advantage of two features humans do not have but are very simple (low hanging fruits) on machines, namely (1) lack of fatigue and random erroneous actions (against the algorithm) and (2) supernatural* physical skills like having ten eyes instead of two, lidars (sensing distance with light), radars (sensing metal objects with radio frequencies), and so on.

*) from human perspective

One thing which Tesla is clearly doing wrong IMHO is they insist on neural network processing of input data not much different from humans (camera-only). I mean, that can be made work, but it misses the opportunity of doing something better and more easily. But it could also be just a public decoy, it is well possible Tesla is seriously working e.g. on low-cost solid state mass-manufacturable LIDAR. I would if I were Elon Musk, and I might also be quiet about it, and make all the copycats put all their money into training NNs.

[AVGresponding]

Musk seems physically and psychologically incapable of keeping his mouth shut about anything, so unless Tesla engineers haven't told him what they're doing, I suspect not.

[tom66]

Elon and Tesla bet on camera only solution because it's cost effective, but it cost them the chance of being the first to launch a driverless robotaxi.  It remains to be seen if it is possible with cameras alone with a sufficiently large safety margin - I'd want to see in the range of 5x safer per mile than a human driver before riding in one. Waymo achieve 10x, so it should be possible!

[AVGresponding]

Waymo achieve 10x, so it should be possible!

-cough- geofenced system -cough-