Tesla Full Self Driving (FSD) info - interesting stuff!

[max_torque]

Whatever your personal opinion of Tesla, you have to admit they employ some good engineers an let them do good engineering, see the long talk on the new FSD system, including details of the ASIC developed to handle the data streams at an incredible rate:

http://youtu.be/Ucp0TTmvqOE?t=4331

[SilverSolder]

Very impressive presentation, far more detailed overview of the kind of AI that goes into making a self driving car - and very interesting hardware!

[Dubbie]

Some of those numbers are extremely impressive!

They really are much further along than I ever appreciated.

[maginnovision]

I think the actual metric is the cars driving themselves of which they don't currently do. GM is further along than them in production. You are actually allowed to let go of the wheel. Waymo seems ahead but as of now it's not exactly a public operation.

[Dubbie]

Did you watch the video?

Your comments suggest that you didn't.

[maginnovision]

I'm not interested in the bits that aren't relevant, just the end result. I've been reading all about it but I did not watch any videos so you are correct.

[wraper]

I'm not interested in the bits that aren't relevant, just the end result. I've been reading all about it but I did not watch any videos so you are correct.

All your post is not relevant to actual situation. They even made full self driving demo rides (signs, traffic lights, junctions) for attendees. By the end of the year, it is supposed to be released for actual customers.

[wilfred]

I think the actual metric is the cars driving themselves of which they don't currently do. GM is further along than them in production. You are actually allowed to let go of the wheel. Waymo seems ahead but as of now it's not exactly a public operation.

Someone wake me up when they don't have a steering wheel or controls of any sort. According to family folklore my grandmother was confident if my grandfather stayed too long after the council meeting at least his horse would know the way home. That's the car I want.

[Dubbie]

Watch the video Wilfred, your point is discussed at length. ~3h:25m

[extide]

Wow, only watched about 30 mins so far, will have to catch the rest later but I love love all the details on the SoC itself, die size, fab node, floor plan, etc. This must be the project that Jim Keller* was working on. I absolutely love the details of chip designs and stuff, and am surprised they have come forth with so much info, definitely very cool! They obviously designed this chip for running a very specific workload and optimized the absolute crap out of it. The NN accelerator is very lean. Does exactly what they need and nothing else. I wonder what GPU IP they used, based on the floorplan it looks like a MALI or maybe PowerVR design, (and maybe they reveal it later in the video, I haven't seen it all yet), but it would be very interesting if they designed their own GPU IP. I kinda doubt they did because that's not really the focus here, the neural network accelerator obviously is and that is definitely all custom, no point in designing a GPU IP when an off the shelf one is fine for what they need. I like the the quad-core clusters of A72's here, they look to have quite a bit of cache as well. I wonder if they are using an off the shelf ARM interconnect there or if they have rolled their own for that. I'd guess it's a CCN-500 series although I'm not sure. I wonder if they will move to one of the newer designs in the future allowing fewer larger clusters (perhaps 2x 8core A76 clusters in next gen).

*Jim Keller was the brains behind some very successful chips like some of the Alpha designs in the 90's, AMD K7 Athlon, AMD K8 Athlon 64, Apple A4 and A5, and AMD Zen. He is now at Intel, purportedly working on their push into discrete graphics.

[maginnovision]

I'm not interested in the bits that aren't relevant, just the end result. I've been reading all about it but I did not watch any videos so you are correct.

All your post is not relevant to actual situation. They even made full self driving demo rides (signs, traffic lights, junctions) for attendees. By the end of the year, it is supposed to be released for actual customers.

If they release full self driving I'd be amazed. They had hiccups even with their fully setup mock drive. I hope for all tesla drivers that FSD does come out one day because they don't seem to be thinking. Seriously they've been saying FSD next year for 3 years. What I don't know is what's different this time? Why was the FSD hype train event the day before a terrible quarter financial document release. Obviously I don't need the answer for that.

In all seriousness though I could never get behind tesla and their FSD program when the CEO says LIDAR is a crutch, developers using it are doomed and at the same time they don't seem to be doing anything the others aren't. I definitely do not think FSD will come out this year or next but maybe they'll upgrade all the cars they're obligated to by the end of next year. Then the next computer will come out and it'll start again. From my POV the hardware itself is about as interesting as a crypto miner ASIC. It's cool but I don't want it. From what I recall it's faster than the old Nvidia system they had but still slower than the newer Nvidia systems. Time will tell if that pays off but the reduced energy consumption probably is required for their cars.

[Dubbie]

and at the same time they don't seem to be doing anything the others aren't.

They went over at length all the things they are doing that others aren't.
Training NN with a constant stream of real automatically annotated data is a pretty huge thing. nobody else has that. Like they said, it's the long tail of rare events that cause problems.  Look at the lady crossing the road carrying her bike that was killed. Having a NN that is trained well enough to deal with odd scenarios like that requires absolute torrents of real world data.

I'm not addressing your question of if this is the time that level 4-5 self driving will really happen, time will tell on that count. But you can't deny that they are doing some extremely cutting edge stuff here.

[maginnovision]

and at the same time they don't seem to be doing anything the others aren't.

They went over at length all the things they are doing that others aren't.
Training NN with a constant stream of real automatically annotated data is a pretty huge thing. nobody else has that. Like they said, it's the long tail of rare events that cause problems.  Look at the lady crossing the road carrying her bike that was killed. Having a NN that is trained well enough to deal with odd scenarios like that requires absolute torrents of real world data.

I'm not addressing your question of if this is the time that level 4-5 self driving will really happen, time will tell on that count. But you can't deny that they are doing some extremely cutting edge stuff here.

I think that's a poor example though, Ubers car never should have hit that lady. It simply wasn't setup to stop. It knew she was there and it knew it was going to collide it just wasn't allowed to do anything. Tesla may also come into the scenario where they overfit leading to extreme examples of WTF(I'd argue it already has happened many times but you don't have to) or underfit and also fail to react.

I don't really know if I trust Tesla to be honest about everything either, it's not their strong suit. For anything they're truly doing that nobody else does they deserve credit, assuming it's helpful. I also like that Waymo has their simulator where they can cook up a scenario, run it and train against it. You actually get to build a sort of similar thing in the autonomous vehicle program at Udacity that culminates in control of a real car. For such a complicated problem where a single death could literally end the Tesla story I think they are being very stubborn assuming they know better than everyone else. The whole autonomous day was nothing more than a publicity event and I really hope they don't try and push out some half cooked solution. Every company working on this problem is doing cutting edge work though, Tesla is literally no different in that respect and nobody likely has a really good idea of where everyone else is at. Regardless of what happens I think it's really interesting to follow and look forward to hearing more from every company involved in solving this problem but I don't expect Tesla is years ahead like they think they do. They may just not be aware of what they don't know.

Just to add... If anyone has a link to an overview of everything they say they're doing I'd like to read it but I have no interest in a long video.

[Dubbie]

You may have missed it but they talked at length about the reasons why simulations are a poor substitute for real data. When you see the two compared, it’s very obvious. The real world images have so much more variety and complexity.

You also mentioned that they may not know what they don’t know yet. I think they are past this stage. Their system can run in shadow mode where a human driver is driving the car and the computer is constantly making predictions about what it would do next if it was driving. It can self check simply by waiting a minute and seeing what actually happened. This is vastly different to running things in a simulation as there is a vast depth and richness of scenarios and behaviours that could never possibly be replicated in a simulation.

I think at one point they mentioned that to have realistic enough other cars in the simulator, you’d almost have to solve the self driving problem!

[maginnovision]

You may have missed it but they talked at length about the reasons why simulations are a poor substitute for real data. When you see the two compared, it’s very obvious. The real world images have so much more variety and complexity.

You also mentioned that they may not know what they don’t know yet. I think they are past this stage. Their system can run in shadow mode where a human driver is driving the car and the computer is constantly making predictions about what it would do next if it was driving. It can self check simply by waiting a minute and seeing what actually happened. This is vastly different to running things in a simulation as there is a vast depth and richness of scenarios and behaviours that could never possibly be replicated in a simulation.

I think at one point they mentioned that to have realistic enough other cars in the simulator, you’d almost have to solve the self driving problem!

If they release FSD this year I'll believe what they said if not I'm going to say they're just trying to tell everyone they're right and everyone else is wrong. The fact is Tesla's all vision system has already resulted in 2 decapitations and many fatal accidents. The only saving grace they have now is they get to pass the blame to drivers. "Should have been paying attention, should have managed it themselves". They won't have that to fall back on if they call it FSD. I guess we'll see what happens in the future. Just don't forget to question them when they say they are the experts and their demo is no more impressive than a years old Google demo.

[wraper]

If they release FSD this year I'll believe what they said if not I'm going to say they're just trying to tell everyone they're right and everyone else is wrong. The fact is Tesla's all vision system has already resulted in 2 decapitations and many fatal accidents.

Moot point. Still less accidents than with human drivers in average. Not to say those were negligent drivers abusing the system. Nothing will be 100% perfect ever, and Tesla makes the safest cars you can currently buy. Basically all naysayers like to call upon anecdotal evidence and avoid hard statistics.
One death in Tesla somewhere in China - big news. 100 people die in car crashes in US every day - who cares.

[extide]

Quote from: maginnovision on Today at 07:29:53 am

If they release FSD this year I'll believe what they said if not I'm going to say they're just trying to tell everyone they're right and everyone else is wrong. The fact is Tesla's all vision system has already resulted in 2 decapitations and many fatal accidents. The only saving grace they have now is they get to pass the blame to drivers. "Should have been paying attention, should have managed it themselves". They won't have that to fall back on if they call it FSD. I guess we'll see what happens in the future. Just don't forget to question them when they say they are the experts and their demo is no more impressive than a years old Google demo.

So, I think you need to be a little bit realistic as far as the safety incidents go -- I mean it will never be zero, or at least not for a long time. It will however be much better than a human driver and that's an improvement we will see right away. All of this same sentiment will apply to any other manufacturer, too.

As far as what Tesla is doing that nobody else is ... you have to talk about that ASIC. I don't know if anyone else is building a fully custom ASIC for this purpose, but but I can't imagine that very many if anyone else is. The ASIC is extremely efficient, although not that fast ultimately. It has 2 neural network processors that do 36TOPs apice in each chip, for a total of 72TOPs per chip. For example the TU102 chip used in the GTX 2080 Ti can do 228 TOPs. However TU102 does that with 754mm^2, 18.6B transistors, and 250W. The Tesla ASIC is only 260mm^2, 6B transistors, and like 30W. That Tesla ASIC also has 12 fairly large ARM cores AND an entire GPU on that same die as well.

(All TOP numbers are for INT8)

[wraper]

For example the TU102 chip used in the GTX 2080 Ti can do 228 TOPs. However TU102 does that with 754mm^2, 18.6B transistors, and 250W. The Tesla ASIC is only 260mm^2, 6B transistors, and like 30W. That Tesla ASIC also has 12 fairly large ARM cores AND an entire GPU on that same die as well.

(All TOP numbers are for INT8)

Those figures are completely incomparable FWIW.

[extide]

How so, they are FMA dot products, just like the tesla chip is doing. Smaller matrices, though, but I don't believe that makes them not comparable as those numbers are including each add and each multiply for each position in the matrix. (16x16 vs 96x96)

[Bud]

. By the end of the year, it is supposed to be released for actual customers.

By the end of Last year Musk was supposed to release space tourism. I am still looking for a space tickets sales office. The problem is this idiot smokes too much.

[SilverSolder]

There was a recent thread hereabouts on why software projects are prone to be late.  It seemed the "fudge factor" ended up at about 1.8 ...  so if Tesla promises fully automated driving within one year, they might actually make it in two!

[wraper]

. By the end of the year, it is supposed to be released for actual customers.

By the end of Last year Musk was supposed to release space tourism. I am still looking for a space tickets sales office. The problem is this idiot smokes too much.

Please stop posting this nonsense misinformation in every thread remotely related to Musk  . Please provide any reference to your words. Oh you can't, because there isn't.

[james_s]

I wish they would focus on making innovative high performance and practical EVs. I could not care less about self driving, the whole point of owning a personal car is that I get to drive it, when I don't feel like driving, I take the bus which is a much more efficient way of moving people.

I think we are still at least 10 years away from fullly autonomous cars, possibly longer. Driving a car outside of carefully controlled conditions is just too complex of a task. Computers are good at the mechanical aspect but way back when I took drivers ed I remember the instructor emphasizing that driving is primarily a social activity and computers still utterly fail when it comes to nuanced social interactions. It will be trivial for people to troll autonomous cars and exploit their predictable behavior and I suspect a lot of people will make a game of it.

[Dubbie]

It will be trivial for people to troll autonomous cars and exploit their predictable behavior and I suspect a lot of people will make a game of it.

You could do that now!
You could put a brick in a paper bag and trick people into running over it. You could turn around signs to point the wrong way on dangerous corners. There are many ways to “troll” human drivers.

But guess what? People on the whole aren’t psychopaths and aren’t running around trying to cause car accidents by exploiting weaknesses in whatever way cars are currently driven.

[CatalinaWOW]

I wish they would focus on making innovative high performance and practical EVs. I could not care less about self driving, the whole point of owning a personal car is that I get to drive it, when I don't feel like driving, I take the bus which is a much more efficient way of moving people.

I think we are still at least 10 years away from fullly autonomous cars, possibly longer. Driving a car outside of carefully controlled conditions is just too complex of a task. Computers are good at the mechanical aspect but way back when I took drivers ed I remember the instructor emphasizing that driving is primarily a social activity and computers still utterly fail when it comes to nuanced social interactions. It will be trivial for people to troll autonomous cars and exploit their predictable behavior and I suspect a lot of people will make a game of it.

I actually like the self driving car concept.  It has been more than forty years since I lived and worked somewhere where riding the bus was an easy option.  The closest it has come in the recent past involved several miles of walking, and currently I have a ten mile walk to the nearest bus stop.  While I enjoy driving sometimes, most of the time it is a necessary evil.  Particularly on long vacation trips I would enjoy the ability to sit back and enjoy the scenery.  But even on the daily trips it would be nice to use that time for research, pleasure reading, thinking about the problem du jour or whatever.