Automotive Evolution – 1 Step at a Time

While at an event, I learned Google and Tesla - both tech companies with no traditional automotive ties - are very much admired but also viewed as a thorn in the side of Detroit's automakers.

DEARBORN, Mich. — As I follow the automotive industry, where safety demands are paramount, a long product development cycle is a given, and margins are low, I'm slowly learning that revolutionary changes are hard to come by.

In contrast, in the electronics industry where I grew up as a beat reporter, changes -- I mean, big, constant changes -- are the key to survival.

In that context, when I attended the Integrated Electrical Solutions Forum (IESF), an automotive industry event held here last week sponsored by Mentor Graphics, I was fascinated to note names like Google and Tesla coming up so often -- during the keynote speech, presentations, even at cocktail hour and dinner. Google and Tesla, both tech companies with no traditional automotive ties, appear to be very much admired but also viewed as a thorn in the side of Detroit's automakers.

I also learned that an idea like bringing Ethernet into cars, which might have been once viewed as a "revolutionary" step, is slowly but surely gaining a foothold even in the conservative automotive industry.

During a speech entitled "In-Vehicle Networking Simulation: Both Sides of the Story," Vincent Bidault, telecom and network specialist at Renault, showed slides that illustrate how applications of CE technology are becoming an integral part of the in-car networking roadmap.

The key point of his presentation was how system-wide vision and electrical simulation are critical in the design of in-car networking. Removing or adding one ECU in an existing CAN network, for example, can profoundly affect the electrical integrity of the signal. The new global length of the network wires and differences in electrostatic discharge (ESD) on the CAN bus can influence harness diversity, resulting in extra cost and logistic issues for automotive system designers, Bidault explained.

When asked, after the presentation, where Renault stands in terms of embracing Ethernet in its cars, Bidault replied, "We've started to look into [the transition from FlexRay to] Ethernet."

Renault's Vincent Bidault shows a roadmap for in-car networking.

Of course, such a transition is no overnight affair. Any changes in in-car networking require good system vision and a lot of electrical simulations. But also, a growing number of carmakers, including Renault, are beginning to see Ethernet as a fault-tolerant bus for fast and reliable applications.

Hans-Juergen Mantsch, technical marketing engineer, responsible for Mentor Graphics' automotive networking design, identified "diagnostics" as the first place Ethernet is already making inroads inside the car. When a car is brought to a shop for diagnostics, it's well known that tracking down the source of an ECU warning signal can end up being a day's work. The Ethernet, by replacing the bottleneck CAN bus, has powers to speed up the process of ECU flashing.

Mantsch went even further. "The end game envisioned by some is to do ECU flashing wirelessly -- by having cars [brought into the shop for diagnostics] parked in a parking lot and do software updating simultaneously."

I'm not sure how it is currently used. Initially the customer was using it to encode the input of a camera transport it to a decoder and display it on a screen. The total encoder+buffer+decoder latency was much less than a frame because they also transported the clock.

This was a few years ago, the design has now been qualified for automotive and it is in prduction as an ASIC.

In general, if you have high bandwidth, the latency can be reduced and, in any case, at least for our encoder, it would make no difference with a JPEG core (from the latency point of view).

Wether it's good enough after decoding for vision algorithms, I'm not sure. Generally H.264 will compress better than JPEG but our core is Baseline, so it's only YUV 4:2:0 8 bits/sample. We have been asked by automotive customers if the sample precision could be increased to 10 or 12 bits (probably for vision algorithms) but never got enough commitment to actually do it.

If you are happy with luminance only at 8 bits/sample, then it should be better than equivalent JPEG. We have also an I frame only version that uses no external DRAM.

Anyway, basic datasheet of the cores are in the download section of our web page :

"H.264 is too lossy and suffers from frame latency as @pmundhenk pointe"

Latency is not an issue for H.264 : our H.264 encoder core has a latency of only 16 video lines (~50 microseconds for 1080p@30) and only because it takes at least 16 video lines to form a 16x16 macroblock, otherwise the real latency for our encoder would be ~3000 clock cycles .

This with P frames and bitrate control. Our core is used in automotive.

Exactly! A carefully adjusted MPEG compression should reach even better results, as higher image quality can be coupled with lower bandwidth. This optimization needs also take into account the end devices that decode the stream. If these have limited processing power, and no additional decoding hardware, information might actually be lost, as the decoding is not fast enough. This is in most cases a minor concern, especially when going to smaller buffers (i.e. less calculation), but one should keep that in mind as well.I think when the overall architecture is well known, one should be able to parametrize MPEG in a way that the delay introduced by the buffer is below 120ms, so not noticeable by humans. This should be a reasonable value for in-vehicle cameras, which are only shown to the user. If these camera images are used for computations, one probably needs to consider an even smaller delay.

And you are right, MJPEG is similar to MPEG without P and I frames. It encodes every frame and can use JPEG decoders which have been heavily optimized for processors. This gives a big advantage, as it does not make sense to replay a delayed frame anyway. Similar to VoIP transmissions.

thank you very much for your offer! I forwarded this also to my colleagues and we are evaluating your portfolio. I will let you know if we need something.

Yes, I agree. Though the speed of the automotive industry is picking up, regarding electronics and software, it is overall still a business with a rather traditional mindset. I see this as a chance for new players in the market though. Tesla has shown what can be done, if you tackle the current problems starting from another perspective. I myself haven't sit in one of their cars, but from images, videos, etc, as well as the data sheets, it looks very impressive. I think the automotive industry needs this push to bring their electronics up and beyong the level of current consumer electronics.

Good point about the delay intrioduced by MPEG, pmundhenk. But even without going to motion JPEG, the MPEG delays are adjustable.

The problem is that in order to get the most possible compression, MPEG likes to create as few full interframes as possible. It introduces interpolated and predictive frames between the big interframes, which of course means you need to intyroduce a long buffer between two interframes. The longer the time, the better the coding efficiency. However, this can be tuned down, reducing the delays and reducing the compression efficiency.

I think your approach of going to MJPEG is essentially the same thing as doing away with the predictive and interplotaed frames of MPEG altogether, and it should still result in at least 10:1 compression. Or thereabouts.

We have had to deal with this issue several times at CogniVue when developing products with our customers. While you are right that the machine vision aspects do not need the high resolution some applications such as Enhanced Backup Camera do still need the "pretty pictures" sent to the infotainment console for display. The result of the vision processing for object detection and distance estimation etc. needs to be overlayed on top of the image displayed.

H.264 is too lossy and suffers from frame latency as @pmundhenk pointed out, MJPEG is still preferred in the Automotive applications. But any amount of compression (ie loss) is problematic if you need to make safety decisions on the camera sensor data. That is why we have our existing CV220X processor family focused on processing close to the sensor and not requiring high bandwidth data transmission across the car chassis. We had to design with the size and power limitations this approach requires.