Tesla Full Self Driving (FSD) info - interesting stuff!

[wraper]

Might well be the wrong place,probabley the wrong page as well. There are 272.1 million cars on the road in the US last year of which 171,500 caught fire Tesla has built 300,00 cars and from what I can find online there are at least 2 fires a week relating to them. I will leave the math.

There were 2 fires in one week not too long time ago. But 2 fires a week is not true at all. Usually there are no fires for several months in a row, then each fire is a big news. No publicly available info of model 3 catching fire so far.

[apis]

Google has already demonstrated they are able to map the parts of the world where they would want to drive in the foreseeable future, so the need for maps is not much of a limitation. Using maps is safer so why not use them if you can? Clearly they are able to deal with some changes in the map environment or else they wouldn't be able to drive as much as they do.

Yow fresh they generally are? 1 year, 2 years? I've seen plenty of situations when map came out yesterday, but does not match actual road which was rebuilt differently a few months ago. Not to say, it's not just satellite view that is required in this case.

It's impossible to discuss this when the data isn't public. The details are still treated as secret sauce so we will basically just have to wait and see. But I would be amazed if they didn't think about this very obvious problem long before Google (and then others) invested billions in the technology, and the engineers and computer scientists who have created this technology aren't idiots either.

(The real world changes continuously so the map is outdated immediately when it's made in some sense, so the cars can obviously deal with some amount of change.)

[extide]

There were 2 fires in one week not too long time ago. But 2 fires a week is not true at all. Usually there are no fires for several months in a row, then each fire is a big news. No publicly available info of model 3 catching fire so far.

Funnily enough, even if they were catching on fire 2x a week, it would still be way below average.

According to his numbers, the national average is 1 fire/year per 1587 cars, and Tesla is at 1 fire/year per 2885 cars. (For Tesla, that's calculating 2 fires/week or 104 fires/year, which is definitely more than reality.)

[wraper]

Funnily enough, even if they were catching on fire 2x a week, it would still be way below average.

According to his numbers, the national average is 1 fire/year per 1587 cars, and Tesla is at 1 fire/year per 2885 cars. (For Tesla, that's calculating 2 fires/week or 104 fires/year, which is definitely more than reality.)

According to this it's more like 1 fire in 4 months in average https://www.autoblog.com/2018/05/11/a-list-of-tesla-car-fires-since-2013/?guccounter=1&guce_referrer=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&guce_referrer_sig=AQAAAIAWmKnMzLZc1kifeclBxdibttYoe8QYTp9tZNQIGQYz1ien_c4gZdjnv8hd9WyUhQFvCGY1gLsz8dyN_OpXQ7S8X3CFHw-XSrn8trVVabPIBm3Ur70nMsGj3gd1D1UB7B---pxR4mX9fo9luFDUhmDENShOPflotJSmf4PE40wA
BTW Tesla made 350k cars in 2018 alone, 100k in 2017, 84k in 2016.

[nctnico]

Google has already demonstrated they are able to map the parts of the world where they would want to drive in the foreseeable future, so the need for maps is not much of a limitation. Using maps is safer so why not use them if you can? Clearly they are able to deal with some changes in the map environment or else they wouldn't be able to drive as much as they do.

Yow fresh they generally are? 1 year, 2 years? I've seen plenty of situations when map came out yesterday, but does not match actual road which was rebuilt differently a few months ago. Not to say, it's not just satellite view that is required in this case.

But how much of the total amount of roads is that? 0.0001%? You also have to understand that the map data + location is used to augment what the sensors bring in.

IMHO it is pretty useless to discuss all kinds of edge cases. People also drive off the road for many different reasons. Why should a self driving car be error free? In the end what counts is that less people get injured or killed in car accidents and that mobility improves.

[wraper]

Google has already demonstrated they are able to map the parts of the world where they would want to drive in the foreseeable future, so the need for maps is not much of a limitation. Using maps is safer so why not use them if you can? Clearly they are able to deal with some changes in the map environment or else they wouldn't be able to drive as much as they do.

Yow fresh they generally are? 1 year, 2 years? I've seen plenty of situations when map came out yesterday, but does not match actual road which was rebuilt differently a few months ago. Not to say, it's not just satellite view that is required in this case.

But how much of the total amount of roads is that? 0.0001%? You also have to understand that the map data + location is used to augment what the sensors bring in.

IMHO it is pretty useless to discuss all kinds of edge cases. People also drive off the road for many different reasons. Why should a self driving car be error free? In the end what counts is that less people get injured or killed in car accidents and that mobility improves.

I don't say it should be error free but relying on precision maps is a kind of dead end approach. You can get stellar results when in perfect environment but when it comes to real life, it will kinda suck. They report low number of disengagements but IMO it's a misleading number since they drive in thoroughly investigated areas.

But how much of the total amount of roads is that? 0.0001%

Like 1k-10k times more than that. If it was 0.0001% I doubt you would ever stumble on that in your life, yet I've seen it several times in one 300 km trip.

IMHO it is pretty useless to discuss all kinds of edge cases.

Edge cases is the most important part since it's what breaks proper operation.

[nctnico]

But how does a changed road suddenly becomes an edge case? Why would that be? Also I never wrote that a system has to rely on precision maps however precises maps are a good start to find out which road a car is driving on. Current navigation systems can easely get confused. Accurate maps and cm precision (terrestrial) positioning will help to let a self driving car know where it actually is.

[wilfred]

IMHO it is pretty useless to discuss all kinds of edge cases. People also drive off the road for many different reasons. Why should a self driving car be error free? In the end what counts is that less people get injured or killed in car accidents and that mobility improves.

An auto should be error free because when it fails it is necessary to look at who, how and why it had the error and if necessary look to seek justice for the victims (if any) of the error. I think it will be good if fewer people die or get injured overall. But since that doesn't matter to the victim or their family and friends who will want to hold someone responsible it can't work to justify the introduction of autos.

If people don't trust and feel safe in or near autos then they will not succeed. People are shocked when someone mistakenly presses the accelerator instead of the brake and drives into a crowd waiting for a bus or whatever. However, because they are also human and fallible they can and have to accept that such things will happen. There will be an investigation to find out what factors led to the incident like alchohol or drugs, mobile phone distraction and so on.

Those factors may not be what causes an auto to fail but people will also have a harder time getting their heads around what a firmware error or sensor failure is and how it might impact them in their daily interactions with an auto. People just will not accept a corporation (or a government) doing the math and putting profit ahead of lives, their lives.

The real problem I see with autos is not getting them to drive from A to B but dealing with the issues when they don't.

[Rick Law]

Well...  We shall see if he is right:

CNBC, May 2, 2019: "Elon Musk to investors: Self-driving will make Tesla a $500 billion company"
"... ... Musk confidently told investors on the call that autonomous driving will transform Tesla into a company with a $500 billion market cap, these people said. Its current market cap stands around $42 billion. He also said that existing Teslas will increase in value as self-driving capabilities are added via software, and will be worth up to $250,000 within three years. ... ..."

Full article:
^{https://www.cnbc.com/2019/05/02/elon-musk-on-investor-call-autonomy-will-make-tesla-a-500b-company.html}

[madires]

Emphasis on "up to". It could be also just an upgrade package for US$10k since all car manufacturers will offer self-driving capabilities in the future.

[apis]

I don't say it should be error free but relying on precision maps is a kind of dead end approach. You can get stellar results when in perfect environment but when it comes to real life, it will kinda suck. They report low number of disengagements but IMO it's a misleading number since they drive in thoroughly investigated areas.

Thoroughly investigated as in having detailed maps? All companies including Tesla (as far as I know) intend to launch taxi services in limited areas in the beginning. I have no doubt that Alphabet has the ability to provide detailed maps of whatever dense cities and interconnecting highways where they wan't to launch first. If outdated maps will be a problem it will be in remote areas so it won't be an issue anytime soon.

(While Waymo's cars can drive anywhere, including in complex city traffic, Tesla autopilot only works on highways, which is the easiest type of road.)

I believe Waymo use the maps to guarantee the car knows where it is on the road within a few cm, even without GPS, and also to help discern dynamic objects (the car needs to pay closer attention too) from static objects. That means they can dedicate more processing power to other problems, and the cars are more or less guaranteed to not confuse the road with an object on the road (as we have seen both Tesla and Uber had problems with sadly).

Waymo's cars have all the sensors that a Tesla car does, and on top of that lidar and better radar. Waymo make their own sensors in house.
https://medium.com/waymo/introducing-waymos-suite-of-custom-built-self-driving-hardware-c47d1714563

[apis]

This is something that's a piece of cake for Waymo and similar cars, but the Tesla autopilot thingie gets confused just because the lane marking isn't perfectly visible:


Tesla doesn't officially say it should either, they say it's only cruise control (unlike Waymo), but if you listen to Elon you'd think it's ready for full autonomous driving within a year.

[wraper]

This is something that's a piece of cake for Waymo and similar cars, but the Tesla autopilot thingie gets confused just because the lane marking isn't perfectly visible:

How about Waymo not working when raining but Tesla doing just fine?

https://youtu.be/KQo0KzbZNmc

[apis]

How about Waymo not working when raining but Tesla doing just fine?

It can only do that since it's not pretending to be fully autonomous. Showing a few minutes of driving in heavy rain with a driver prepared to take over any second doesn't prove it can do that over millions of miles without any mistakes. As far as I can tell, Tesla haven't shown they are anywhere close to the level of autonomy that Waymo and others who use Lidar and detailed maps are.

"To poke holes in Musk’s comments, you need look no further than the Society of Automotive Engineers (SAE) taxonomy for autonomous vehicles, commonly referred to as the SAE levels, which have become the global standard for defining self-driving. Most experts would categorize Navigate on Autopilot as Level 2 autonomy, meaning the vehicle can handle basic tasks, like acceleration, braking, and lane changes, but the human driver needs to maintain full attention of the road and be prepared to take control of the vehicle in a moment’s notice.
...
Not even highly automated vehicles like those operated by Alphabet’s Waymo, Ford’s Argo AI, or GM’s Cruise Automation would really qualify as “full self-driving” because they are only able to operate within a specific geographic location and under specific conditions, like good weather. Those companies typically keep safety drivers behind the wheel, with the understanding that their vehicles would be considered Level 4 capable under the SAE’s taxonomy."
https://www.theverge.com/2019/1/30/18204427/tesla-autopilot-elon-musk-full-self-driving-confusion

I like Elon, and he's doing cool stuff, but you have to take everything he says with a truckload of salt. Tesla is giving self driving cars a bad reputation imho. The one who deserve credit for being first and who have gotten the furthest with this technology is still Waymo.

The SAE classification isn't very good and maybe Tesla have a secret prototype they are developing in Singapore or somewhere else, who knows, but so far Tesla autonomous self driving is "all hat, no cattle" as far as I can tell.

[Dubbie]

Does anyone know the accidents per mile driven of Tesla Vs Waymo?

This to me would seem like a much more relevant statistic.

If tesla has driven 300K miles with 3 accidents, and Waymo has driven 50K with zero, this doesn't put Waymo ahead in safety in my mind.

[apis]

If Tesla's autopilot can't even handle a highway interchange reliably, which is a very basic task, they are clearly not anywhere near full autonomy, so trying to compare the number of fatalities is like comparing apples and oranges anyway. The best public data are number of disengagements in California which the companies are required to publish by law. According to that data Waymo are twice as good as GM who are second.

Tesla haven't even published any data because they aren't testing in California.
'Tesla said it “did not test any vehicles on public roads in California in autonomous mode or operate any autonomous vehicles, as defined by California law.”'

You can see the full table here:
https://www.therobotreport.com/waymo-autonomous-vehicles-apple/

[Dubbie]

Interesting.

Thanks for the data.

[CatalinaWOW]

The disengagements data is interesting on many levels.  For example Nissan managed to make very little improvement in the last year.  And Uber, who averaged a problem every half mile never had any business on public roads.  Not even close to ready for prime time.  It is really hard to assign any credibility to those who are going single digit distances without disengaging.  Plenty of fuel here for those who say that self driving is impossible, or at least impossibly far in the future.

On the other hand Waymo and GM Cruise are well into the danger zone.  Good enough to inspire complacency on the part of the human operator, but maybe not good enough to be fully in charge.

Finally, plotting the data shows that in general performance is well correlated with experience.  When experience reaches about ten times what Waymo has now you should expect about one disengagement in the life of an automobile.  Somewhere in that neighborhood of performance should be getting close to satisfying most folks.  Certainly in the same class as human operators.

The plot also clearly shows Apple and Uber as not good learners well out of family on the bad side, and Telenav and BMW on the other side of the scale.  All are low on the experience scale.  If Telenav and BMW can keep up that level of performance through a significant body of experience they could easily arrive at the finish line first.

[apis]

Interesting plot that.

Maybe it's worth pointing out again that disengagement doesn't mean the car would have caused an accident.

For example, as wraper has pointed out heavy rain is problematic because it blocks the lidar sensors. So heavy rain is likely one cause for disengagements in those statistics. But at the same time we see that Tesla autopilot drives in heavy rain without lidar. I.e. a car would likely be able to continue driving without any accidents relying only on the cameras and radars even in heavy rain. Heavy rain would mean an increased risk of accidents (the same is true for human drivers though), but an accident would still be unlikely, and the car might handle it by simply pull over and stop in a suitable location until the weather changes.

Similarly if the map is out of date, I would assume the cars will fail gracefully.

On the other hand, a disengagement might also occur because the car was about to run over a pedestrian and the safety driver had to step on the brakes to avoid it.

The latter is obviously a bigger problem than the former. Disengagements are a crude metric, but it's the best that's available since it's something the companies are required to report by law.

[wilfred]

Wouldn't a disengagement require a licenced driver at the wheel? I'm starting to see a problem with training the next generation of licenced drivers. If you're going to take over in a crisis you sort of need experience to be able to make the decisions. Hopefully the car will be better than an inexperienced driver.

I'm still unconvinced a non-driving driver will have any hope of remaining engaged with the traffic. They have to sit there concentrating as if they were driving and babysit a machine that they implicitly don't trust completely. I'd rather take a bus.

[maginnovision]

Also the idea that your car is just going to stop until weather gets better... Who would want that? I don't think the market for that would be very large.

[Dubbie]

Or it could stop while the passenger got themselves into a position to drive the rest of the journey manually.
That would make sense.

Obviously if there was no steering wheel this couldn't happen, but i imagine that by the time the steering wheel goes, the cars would be better drivers than any human.

[Marco]

AFAICS "disengagement" does not include when the vehicle just comes to a safe stop because it has no clue what to do next.

[wilfred]

What will an auto do when one of these bastards crawls across the lidar sensor. They love crawling across windscreens and freaking out drivers. But in the words of Douglas Adams they are mostly harmless. F'ing big though.

(short video, warning it is vertical)

[apis]

What will an auto do when one of these bastards crawls across the lidar sensor. They love crawling across windscreens and freaking out drivers. But in the words of Douglas Adams they are mostly harmless. F'ing big though.

It's like a tiny eight-legged cat, cute.

The cars have several lidars and several cameras and radars, they don't depend on just one sensor, so it would probably handle it much better than an arachnophobic human driver.