Tesla shows a self-driving car with all its eyes on the road

November 22, 2016
by Nancy Owano

(Tech Xplore)—A self-driving car from Tesla: what does it see? Finding answers could be quite important because there are people who, although treated to regular reports of technology advancements in self-driving cars braking, changing lanes, and arriving safely, cannot get their raw nerves to catch up to their brains.

Alone in a self-driving car? A recent video could settle a lot of nerves. A recent Tesla demo will show you what a fully autonomous car sees, fully autonomous meaning as in no human input to touch the driver's wheel or gas pedals or brakes—so how did it do?

The Tesla demo video grabbed the attention of a number of tech watchers on various sites. It's one thing seeing any self-driving car on a near-empty highway but this video showed a self-driving Tesla on the move through an urban-looking area with traffic and turns.

Darrell Etherington in TechCrunch noted the three separate views that were shown of what the Tesla optical cameras see at all times during the drive. These include "colored overlays showing what the onboard computer is detecting in terms of objects and lane markers."

"We can see the system picking up different objects as the car makes it way, objects like signs, road lines, cars, and even pedestrians. It captures the flow of the environment so to speak in order to get a sense of where it's headed," said Adnan Farooqui in Ubergizmo.

Steve Hanley in Gas2 discussed this video of "a Model X with the second-generation hardware package driving along the roads surrounding company headquarters in Silicon Valley. The video offers us insight into what the Autopilot system 'sees' as it drives. A series of colorful boxes and lines focus on real-world objects detected by the sensors."

Andrew Liptak in The Verge made note of the colored boxes for "in path objects," e.g., road lights, objects and roadsigns. "These boxes highlight people, other vehicles, and other potential hazards that a human driver would ordinarily take into consideration while driving."

Hanley noted that "The refresh rate of the system is an other worldly 2,100 frames per second."

Teslarati, which is a site focused on all things Tesla, said that "Using what the company describes as Tesla Neural Net for vision – combining signals captured through eight cameras and twelve advanced ultrasonic sensors – Tesla's fleet of vehicles with "hardware 2" will have a 360-degree view and see the world in precision beyond what humans are capable of."

But wait. Isn't there a human in the car? Tesla said at the intro of the video, "the person in the driver's seat is only there for legal reasons. He is not doing anything. The car is driving itself."