Autonomous Cars, Vision, and Protocols

Remember how, last May, I wrote about the five levels of autonomy of the autonomous car? I’ve been thinking about cars a lot because I just got a new-to-me car, a 2015 model (I won’t tell you what it is since I don’t want to be seen endorsing a brand, but let's just say that it rhymes with “VW Schmetta”).

A quick refresher; click for higher resolution

As I drove back to the dealership a week later to get some of the features explained to me again (because I couldn’t, for the life of me, figure out how to pair my phone with the Bluetooth), I was reminded of a presentation I attended at the Embedded Vision Summit last May, by Mark Bünger of Lux Research. His main premise was that in the future, cars will do lots of amazing things, while at the same time, car makers are terrible at adopting new technology. For example, he said that Bluetooth was introduced in 1994 or so, but because car makers didn’t want to “lose control of the dashboard”, it took forever became standard. In 2008-9, smartphones came into being, but we are still having trouble ten years later synching our phones to our cars. Hence my trip back to the dealership.

As another example, my new car has navigation, great! But it’s clunky, reminds me of the old Garmin I used in the mid-2000s and doesn’t take traffic into consideration. I get far better results using Waze on my phone than whatever it is they have loaded into the car’s software. People prefer to use their devices hooked up to a charger or an aux cable, rather than rely on what is embedded in the automobile. And the problem isn’t with the technology. Cars don’t know how to do telematics, pure and simple.

So what makes us all think that the current auto manufacturers can keep up with the demands of true level-5 autonomy? What is it going to take, to truly make cars surpass the designs of the most recent iThing or Galaxy S-whatever? Tesla and Google are dragging the automotive industry kicking and screaming into the electric and autonomous automobile into disruption.

I want to quote a passage from Vivek Wadhwa’s book, The Driver in the Driverless Car:

Let me paint a picture of what streets will look like in an age of driverless cars. We will no longer need traffic lights: robot cars will synchronize wirelessly to time mass movements across city intersections and entries onto freeways or balletic dances around four-way stop signs. Having no human eyes behind the wheel will obviate much of the need for signaling and signage. When all the driverless cars are talking to each other, there will be no need for them to ever come to a complete halt and waste all their kinetic energy.

This is all very well and is something to look forward to, no doubt. But my goodness so much must happen between now and then. Just as the transition from Level Three to Level Four will involve the most growing pains (seamlessly integrating human drivers and autonomous systems), the most challenging time will likely be when a significant percentage of the cars on the road will have no one behind the wheel, trying to predict what those moody and distracted human drivers of the cars with steering wheels are going to do next. When all cars are well-connected into their environment, with each other, and with the infrastructure, the world will look much like Wadhwa suggests. But when half of them are autonomous, and half of them are 1979 Datsun 280ZXes (yes, this is the car my son drives), I predict a messy situation on the roads.

Until then, cars are going to need specialized sensors or arrays of simple sensors performing multiple functions coming out of our ears and eyeballs and every inch of the car, to be able to sense that Datsun that may actually only come to a rolling stop at that stop sign. When all the cars on the road have V2X communication, sensors aren’t going to be as necessary, because everything will be communicated via some vision protocol. (I understand that work is already underway for this, building intelligent transportation systems, or ITS. But since that’s been handled by the car industry, I hesitate to celebrate just yet; see how successful previous technological releases have been in the car industry.)

But with vision protocols, there are many more reasons that this technology will succeed, as summed up in this slide:

Click for higher resolution, and check out the original presentation in total at the Embedded Vision Alliance website.

Fun fact: the Cadence Tensilica Vision DSPs, especially the Vision C5 DSP, is meant specifically for automotive applications. They offer a breakthrough in energy efficiency and performance for neural network and digital signal processing. When you have multiple vision sensors around your car, and they are collecting images faster than your favorite high-definition movie can project it, the sheer volume of processing that data requires some specialty processing, and that quandary is answered directly by the Tensilica Vision family of DSPs.

Where does this processing happen? It can’t be in the trunk of your car, because it may take the power of a rack of servers to make split-second decisions. It can’t be in the “cloud” because the last thing you need when a child’s ball bounces into the street is the message “buffering…”. Chances are, it will be on the “edge” of the cloud — the part with the silver lining? — that is, so much of the processing power required to make autonomous cars go will be on localized servers. For a truly autonomous automotive society to make the transition from driver to driverless, a network of servers along those frequently-used routes must be in place, creating this protocol that Bünger suggested in his presentation last May.

From Mark Bunger of Lux Research; for more from this presentation, see the complete presentation on the Embedded Vision Alliance.

It may not be practical to have a fully autonomous car share the road with my son’s Datsun, but in the meantime, I suggest working on this protocol.

—Meera

P. S. Edited to add some breaking news about California allowing testing of self-driving cars without a driver present! Check out this article!