Share this story

On Tuesday, Nvidia announced a new version of its automotive-grade compute platforms, Drive PX Pegasus. It's recognition that the computational needs of fully autonomous (also known as level 5) vehicles are going to be demanding. Such vehicles will have to fuse inputs from multiple sensors and sensor-types, then make sense of it all with no fuss to get us from A to B. "The reality is we need more horsepower to get to level 5," said Danny Shapiro, Nvidia's senior director of automotive.

Drive PX Pegasus puts its predecessors in the shade. It's built around a pair of Xavier SoCs, plus another pair of discrete, as-yet unnamed GPUs which will do the heavy lifting when it comes to machine learning and computer vision. There are 16 inputs for sensors like lidar, radar, and cameras, and it can connect to controller area networks (CAN), Flexray, and 10Gbit ethernet. And because humans will be trusting their lives to Drive PX Pegasus, it's been designed for ASIL D certification—the most stringent safety level for automotive applications. At 320 trillion operations per seconds, the latest platform should be more than an order of magnitude more capable than Drive PX Xavier.

"Everyone developing robo-taxis has a trunk full of servers," Shapiro said in a briefing, referring to autonomous research vehicles like the ones from Bosch or Audi. "This is the path to production for level 5 vehicles." Drive PX Pegasus is due in the second half of 2018.

Nvidia also revealed that it is working with ZF and Deutsche Post DHL in Germany to add autonomous driving capabilities to the latter's delivery vehicles. Deutsche Post commissioned a fleet of bespoke EVs called Street Scooters, and that fleet now numbers 3,400. Starting next year, these will be fitted with the ZF ProAI self-driving system, which is based on the Drive PX platform.

"ZF ProAI is the brain between our autonomous driving sensor set to detect and understand the environment and our motion control based on outstanding mechanical competence—the entire system follows our 'see—think—act’ approach. In supply logistics and on the last mile where autonomous driving has tremendous benefits, goods can be delivered independent of the time of the day and delivery staff, with minimal noise and emissions, thus significantly reducing traffic congestion in city centers," said Stefan Sommer, CEO of ZF.

So the good news: your new shiny electric car has eliminated a lot of the old unreliable mechanical gubbins, however the bad news is we've added a ton of complicated electronics stuff. It really shouldn't go wrong very often, but boy when it does, are you going to get a bill. Oh, and on that note, let me introduce you to Tony, he handles our extended warranties, in case you're interested.

In supply logistics and on the last mile where autonomous driving has tremendous benefits, goods can be delivered independent of the time of the day and delivery staff...

If it is not dependent on the delivery staff available will be dependent on the recipient being available? Even for an envelope you still need someone to poke it into the mailbox. Getting a bot/drone to do that in the real-world is a very different task to doing that in a warehouse that has been designed for autonomous efficiency.

Setting up the requisites for Level 5 is smart. The biggest risk to a future of autonomous driving is Level 3 and Level 4 which will fail in spectacular ways, and may result in innovation-stifling regulation or just dry up the demand.

Once we have a lot of freely available self-driving taxis, do you think a lot of people will be willing to dump their personal vehicle? Unless that happens I can't see there being much improvement in traffic congestion. There is only so far you can go, even with a theoretically perfect self-driving system. Even if human drivers are only 50% efficient (which is unlikely) all you can do is double throughput. In major urban centers, mass transit makes a lot more sense. I'm not saying I want to force anyone to do anything, but if there is a better alternative, I can see people going for it.

Rural doesn't have this problem, people can just keep driving their pickups as is.

Just wait until you have to install Nvidia Experience in order to optimize your ride for different road types... ;-)

People joke about this. The running joke with Microsoft sync in ford vehicles is that you would have to reboot the car in the middle of driving. Well funny thing is bluetooth will stop working while driving and there was a recent update to sync that added a reboot option, I'm not lying! I sometimes have to reboot my Microsoft sync radio while driving because it quits working, lol.

I hope they install these things in redundant pairs or have redundancy built in.

Edit, I know this isn't at the same level but my blind spot warning system will sometimes stop working if it's raining too hard.

Once we have a lot of freely available self-driving taxis, do you think a lot of people will be willing to dump their personal vehicle? Unless that happens I can't see there being much improvement in traffic congestion. There is only so far you can go, even with a theoretically perfect self-driving system. Even if human drivers are only 50% efficient (which is unlikely) all you can do is double throughput. In major urban centers, mass transit makes a lot more sense. I'm not saying I want to force anyone to do anything, but if there is a better alternative, I can see people going for it.

Rural doesn't have this problem, people can just keep driving their pickups as is.

I've actually had this conversation with my wife. We are currently a two-car household and I could really see us going to a 1-car or a 1-vehicle family if fully autonomous taxis become available and are affordable.

I use mass transit when it makes sense, but it doesn't always make sense. I can't easily get to work via mass transit. Also, I live in place that snows, and getting to the train stop in the snow might be easier if you could take a self-driving vehicle to the train stop. In other words, I really feel like fully-autonomous vehicles are just another piece in the transit puzzle.

Also, reduction in traffic isn't just about less vehicles on the road. Once we get critical mass of autonomous vehicles, we can eliminate human-error generated issues like traffic jams. Hell, imagine how quickly you could get through a 4-way intersection if the vehicles could communicate with each other?

Just wait until you have to install Nvidia Experience in order to optimize your ride for different road types... ;-)

People joke about this. The running joke with Microsoft sync in ford vehicles is that you would have to reboot the car in the middle of driving. Well funny thing is bluetooth will stop working wile driving and there was a recent update to sync that added a reboot option, I'm not lying! I sometimes have to reboot my Microsoft sync radio while driving because it quits working, lol.

I hope they install these thing in redundant pairs or have redundancy built in.

Self driving seems to be a prime example of things that are much harder than you thought - even when you thought they were hard in the first place.

preface for the next bit, I'm well aware I'm storming into Dunning-Kruger

I'm clearly wrong, but in my head I don't understand the need for these ultra high res maps. I know this isn't easy, but surely you take two spatially separated images perform a depth comparison to produce a crude 3d map. Sure, use LIDAR. Then you repeat this temporally to identify things that are moving, then you forecast those movements - initially assuming straight lines, then crudely classifying it to add a forecast confidence and from there deriving direction and speed.

It feels that's how I drive. I only use a map for large scale navigation.

look, I'm just rambling here - and I like learning things. Why are those maps needed?

Just wait until you have to install Nvidia Experience in order to optimize your ride for different road types... ;-)

People joke about this. The running joke with Microsoft sync in ford vehicles is that you would have to reboot the car in the middle of driving. Well funny thing is bluetooth will stop working wile driving and there was a recent update to sync that added a reboot option, I'm not lying! I sometimes have to reboot my Microsoft sync radio while driving because it quits working, lol.

I hope they install these thing in redundant pairs or have redundancy built in.

I would expect that the software running on these devices will be much more well vetted and debugged than the software running on an entertainment system...

It's going to be exciting to see the first circuit races set up to showcase strictly driverless cars racing each other. It'll all be about who has the best algorithm for turning, braking and acceleration curves, who has the best processor speed so there's less lag from sensing the environment to acting on it.

Here's a situation I can see the car software running through:The map says that a road is coming up on the right that I need to turn down, but the lidar doesn't see it. The car starts to spend a little more processing power trying to see why it doesn't see the road. Oh, it's because there's something blocking the road. Well, let me stop and wait for the road to clear.

Obviously that's a very simple example, but more data is rarely bad in these situations.

Just wait until you have to install Nvidia Experience in order to optimize your ride for different road types... ;-)

People joke about this. The running joke with Microsoft sync in ford vehicles is that you would have to reboot the car in the middle of driving. Well funny thing is bluetooth will stop working while driving and there was a recent update to sync that added a reboot option, I'm not lying! I sometimes have to reboot my Microsoft sync radio while driving because it quits working, lol.

I hope they install these thing in redundant pairs or have redundancy built in.

Edit, I know this isn't at the same level but my blind spot warning system will sometimes stop working if it's raining too hard.

Are these systems being built to be 100% autonomous 100% of the time?

I'm still convinced that self-driving cars are the next fusion reactor. All the software in the world wont defeat physics, things like rain, snow, and dirt will easily defeat the smartest sensors on the planet, and we still cant make infotainment systems that can go 10+ years without developing kinks in their operation.

A full self driving car, with our current technology, still isnt feasible IMO. Level 3 might be as far as we can realistically get.

It's going to be exciting to see the first circuit races set up to showcase strictly driverless cars racing each other. It'll all be about who has the best algorithm for turning, braking and acceleration curves, who has the best processor speed so there's less lag from sensing the environment to acting on it.

Just wait until you have to install Nvidia Experience in order to optimize your ride for different road types... ;-)

People joke about this. The running joke with Microsoft sync in ford vehicles is that you would have to reboot the car in the middle of driving. Well funny thing is bluetooth will stop working wile driving and there was a recent update to sync that added a reboot option, I'm not lying! I sometimes have to reboot my Microsoft sync radio while driving because it quits working, lol.

I hope they install these thing in redundant pairs or have redundancy built in.

I would expect that the software running on these devices will be much more well vetted and debugged than the software running on an entertainment system...

This is the same industry that has had multiple major compromises in their "4G" connected cars, see the jeep fiasco.

Once we have a lot of freely available self-driving taxis, do you think a lot of people will be willing to dump their personal vehicle? Unless that happens I can't see there being much improvement in traffic congestion. There is only so far you can go, even with a theoretically perfect self-driving system. Even if human drivers are only 50% efficient (which is unlikely) all you can do is double throughput. In major urban centers, mass transit makes a lot more sense. I'm not saying I want to force anyone to do anything, but if there is a better alternative, I can see people going for it.

Rural doesn't have this problem, people can just keep driving their pickups as is.

I've actually had this conversation with my wife. We are currently a two-car household and I could really see us going to a 1-car or a 1-vehicle family if fully autonomous taxis become available and are affordable.

I use mass transit when it makes sense, but it doesn't always make sense. I can't easily get to work via mass transit. Also, I live in place that snows, and getting to the train stop in the snow might be easier if you could take a self-driving vehicle to the train stop. In other words, I really feel like fully-autonomous vehicles are just another piece in the transit puzzle.

Also, reduction in traffic isn't just about less vehicles on the road. Once we get critical mass of autonomous vehicles, we can eliminate human-error generated issues like traffic jams. Hell, imagine how quickly you could get through a 4-way intersection if the vehicles could communicate with each other?

4 way intersections are called roundabouts in the rest of the civilized world.

Snark aside, autonomous vehicles will reduce traffic congestion as they come online. The fewer people tailgate, slam on their brakes, or otherwise drive without any regard to other traffic, the better the traffic will get.

Just wait until you have to install Nvidia Experience in order to optimize your ride for different road types... ;-)

People joke about this. The running joke with Microsoft sync in ford vehicles is that you would have to reboot the car in the middle of driving. Well funny thing is bluetooth will stop working while driving and there was a recent update to sync that added a reboot option, I'm not lying! I sometimes have to reboot my Microsoft sync radio while driving because it quits working, lol.

I hope they install these things in redundant pairs or have redundancy built in.

Edit, I know this isn't at the same level but my blind spot warning system will sometimes stop working if it's raining too hard.

Are these systems being built to be 100% autonomous 100% of the time?

One thing, there's not much similarity between an infotainment system and an autonomous vehicle hardware platform.

Secondly, there's no such thing as "Microsoft Sync". There's Ford SYNC, which was built on top of Microsoft's Auto platform until the recent generation, which is now build on Blackberry's QNX (since 2015). The distinction I'm trying to make here is that Ford engineers were responsible for implementation.

It's going to be exciting to see the first circuit races set up to showcase strictly driverless cars racing each other. It'll all be about who has the best algorithm for turning, braking and acceleration curves, who has the best processor speed so there's less lag from sensing the environment to acting on it.

I'm still convinced that self-driving cars are the next fusion reactor. All the software in the world wont defeat physics, things like rain, snow, and dirt will easily defeat the smartest sensors on the planet, and we still cant make infotainment systems that can go 10+ years without developing kinks in their operation.

A full self driving car, with our current technology, still isnt feasible IMO. Level 3 might be as far as we can realistically get.

I'm still convinced that self-driving cars are the next fusion reactor. All the software in the world wont defeat physics, things like rain, snow, and dirt will easily defeat the smartest sensors on the planet, and we still cant make infotainment systems that can go 10+ years without developing kinks in their operation.

A full self driving car, with our current technology, still isnt feasible IMO. Level 3 might be as far as we can realistically get.

Don't Teslas work pretty well rain, snow and dirt ?

Pretty well, as long as they are kept clean.

Put some snow or dirt on the sensors though, and watch the dashboard light up like a christmas tree. This is also an issue with other manufacturers.

Clever software cant defeat physics. Much of the self driving car testing has been restricted to flatter, warm climates for a reason.

Once we have a lot of freely available self-driving taxis, do you think a lot of people will be willing to dump their personal vehicle? Unless that happens I can't see there being much improvement in traffic congestion. There is only so far you can go, even with a theoretically perfect self-driving system. Even if human drivers are only 50% efficient (which is unlikely) all you can do is double throughput. In major urban centers, mass transit makes a lot more sense. I'm not saying I want to force anyone to do anything, but if there is a better alternative, I can see people going for it.

Rural doesn't have this problem, people can just keep driving their pickups as is.

It is going to be a generational change. In the current day and age, a large portion of the population "needs" a vehicle to get to (work / school / play / shopping / etc). We have accepted the costs of purchasing / leasing and maintaining a vehicle as normal.

Once autonomous, on demand vehicles become available, more and more people will opt to avoid the costs of vehicle ownership. I drive for Lyft every once in a while and I am already seeing these situations. People living in apartments and working in average paying jobs can get back and forth to work for ~$8 a day / $160 a month. Compare that with my $400 / month car loan, $125 / month insurance, + $gas money that I have to pay and it quickly becomes clear why people will opt for "shared vehicles", be they autonomous or otherwise.

I foresee a very near future where car ownership is a luxury for people with large amounts of discretionary income. For everyone else, there will be on demand vehicles. It will be interesting to see where all of current spending on vehicles will shift to.

I'm still convinced that self-driving cars are the next fusion reactor. All the software in the world wont defeat physics, things like rain, snow, and dirt will easily defeat the smartest sensors on the planet, and we still cant make infotainment systems that can go 10+ years without developing kinks in their operation.

A full self driving car, with our current technology, still isnt feasible IMO. Level 3 might be as far as we can realistically get.

Don't Teslas work pretty well rain, snow and dirt ?

Pretty well, as long as they are kept clean.

Put some snow or dirt on the sensors though, and watch the dashboard light up like a christmas tree. This is also an issue with other manufacturers.

Clever software cant defeat physics. Much of the self driving car testing has been restricted to flatter, warm climates for a reason.

Cars are currently driven to a socially acceptable level of safety with a pair of cameras, a pair of microphones, some tilt sensors and very, very, very clever software.

Just wait until you have to install Nvidia Experience in order to optimize your ride for different road types... ;-)

People joke about this. The running joke with Microsoft sync in ford vehicles is that you would have to reboot the car in the middle of driving. Well funny thing is bluetooth will stop working while driving and there was a recent update to sync that added a reboot option, I'm not lying! I sometimes have to reboot my Microsoft sync radio while driving because it quits working, lol.

I hope they install these things in redundant pairs or have redundancy built in.

Edit, I know this isn't at the same level but my blind spot warning system will sometimes stop working if it's raining too hard.

Are these systems being built to be 100% autonomous 100% of the time?

One thing, there's not much similarity between an infotainment system and an autonomous vehicle hardware platform.

Secondly, there's no such thing as "Microsoft Sync". There's Ford SYNC, which was built on top of Microsoft's Auto platform until the recent generation, which is now build on Blackberry's QNX (since 2015). The distinction I'm trying to make here is that Ford engineers were responsible for implementation.

Ok, Ford sync powered by Microsoft. I had to walk out and look at my car... If I had corrected my wife like this I would have been in a world of shit. Nice correction though, lol.

I'm still convinced that self-driving cars are the next fusion reactor. All the software in the world wont defeat physics, things like rain, snow, and dirt will easily defeat the smartest sensors on the planet, and we still cant make infotainment systems that can go 10+ years without developing kinks in their operation.

A full self driving car, with our current technology, still isnt feasible IMO. Level 3 might be as far as we can realistically get.

Don't Teslas work pretty well rain, snow and dirt ?

Pretty well, as long as they are kept clean.

Put some snow or dirt on the sensors though, and watch the dashboard light up like a christmas tree. This is also an issue with other manufacturers.

Clever software cant defeat physics. Much of the self driving car testing has been restricted to flatter, warm climates for a reason.

Cars are currently driven to a socially acceptable level of safety with a pair of cameras, a pair of microphones, some tilt sensors and very, very, very clever software.

Ultimately the sensors are not the problem.

Current "sensors" as you call them, are also capable of cleaning and maintaining themselves, and thinking about a solution, as opposed to operating on pre-programmed responses or basic AI.

Also, my "sensors" are not blinded because a dime of snow falls on the windshield. They are not confused by a teenager spray painting a sign or the road. They can see for miles ahead of them. They can perceive depth and angles quicker then computers, and posses situational awareness. They do not require software updates, patches, or OS upgrades to work properly. They work well in a variety of conditions. Minor impairments can be fixed with corrective lenses, which can be acquired for literal decades from many, many different shops, as opposed to expensive parts and repairs, only a few vendors and being seceptable to a lack of supply in 15-20 years.

Seems to me that current techy sensors have a very long way to go until they can match what human sensors can do.

bit of a tangent, but I think there's something that gets covered over slightly.

1) there's a general expectation that people won't own self driving cars, but subscribe to fleets.2) there's a general expectation that cars are going electric3) there's some talk of using electric cars to buffer national grids

There's something of a knot in there to be cut through. The fleet owners will want to be paid to help the grid, but they can only do that when they're not moving. Again the owners will want the minimum number of vehicles for efficiency purposes. They'll act like Uber/Lyft and try to use people heading in the correct direction again for efficiency.

We're likely to see less cars, but those that exist will probably do more miles. I think we'll need to get used to seeing entirely empty vehicles as they head off to their next user.

So given the cars are unlikely to be still much of the time except overnight - and that's unhelpful to point 3.

I'm still convinced that self-driving cars are the next fusion reactor. All the software in the world wont defeat physics, things like rain, snow, and dirt will easily defeat the smartest sensors on the planet, and we still cant make infotainment systems that can go 10+ years without developing kinks in their operation.

A full self driving car, with our current technology, still isnt feasible IMO. Level 3 might be as far as we can realistically get.

Don't Teslas work pretty well rain, snow and dirt ?

Pretty well, as long as they are kept clean.

Put some snow or dirt on the sensors though, and watch the dashboard light up like a christmas tree. This is also an issue with other manufacturers.

Clever software cant defeat physics. Much of the self driving car testing has been restricted to flatter, warm climates for a reason.

Cars are currently driven to a socially acceptable level of safety with a pair of cameras, a pair of microphones, some tilt sensors and very, very, very clever software.

Ultimately the sensors are not the problem.

Current "sensors" as you call them, are also capable of cleaning and maintaining themselves, and thinking about a solution, as opposed to operating on pre-programmed responses or basic AI.

Also, my "sensors" are not blinded because a dime of snow falls on the windshield. They are not confused by a teenager spray painting a sign or the road. They can see for miles ahead of them. They can perceive depth and angles quicker then computers, and posses situational awareness. They do not require software updates, patches, or OS upgrades to work properly. They work well in a variety of conditions. Seems to me that current techy sensors have a very long way to go until they can match what human sensors can do.

windscreen wipers. The rest of your points are software issues.

Plus there's only a handful of humans that are really level 5 - the rest of us pull over and wait or call for help if things get hairy.

I'm still convinced that self-driving cars are the next fusion reactor. All the software in the world wont defeat physics, things like rain, snow, and dirt will easily defeat the smartest sensors on the planet, and we still cant make infotainment systems that can go 10+ years without developing kinks in their operation.

A full self driving car, with our current technology, still isnt feasible IMO. Level 3 might be as far as we can realistically get.

Don't Teslas work pretty well rain, snow and dirt ?

Pretty well, as long as they are kept clean.

Put some snow or dirt on the sensors though, and watch the dashboard light up like a christmas tree. This is also an issue with other manufacturers.

Clever software cant defeat physics. Much of the self driving car testing has been restricted to flatter, warm climates for a reason.

Cars are currently driven to a socially acceptable level of safety with a pair of cameras, a pair of microphones, some tilt sensors and very, very, very clever software.

Ultimately the sensors are not the problem.

Current "sensors" as you call them, are also capable of cleaning and maintaining themselves, and thinking about a solution, as opposed to operating on pre-programmed responses or basic AI.

Also, my "sensors" are not blinded because a dime of snow falls on the windshield. They are not confused by a teenager spray painting a sign or the road. They can see for miles ahead of them. They can perceive depth and angles quicker then computers, and posses situational awareness. They do not require software updates, patches, or OS upgrades to work properly. They work well in a variety of conditions. Seems to me that current techy sensors have a very long way to go until they can match what human sensors can do.

windscreen wipers. The rest of your points are software issues.

Plus there's only a handful of humans that are really level 5 - the rest of us pull over and wait or call for help if things get hairy.

Software will not defeat physics. Unless you are going to put windscreen wipers on every single sensor on your car, and adapt your software to be able to run without your sensors all having perfect vision, you will have problems.If a level 4-5 car goes blind because its sensors have snow in them, no software in the world is going to allow it to run.

Proponents of self driving cars love to pretend that software will fix all of self driving's woes, and refuse to admit that software cant fix everything.