Apple has explored building displays with dynamic backlight adaptation for better picture quality, particularly when watching letterboxed widescreen movies on a high-definition screen.

The concept was revealed this week in a new Apple patent application discovered by AppleInsider. The filing, entitled "Dynamic Backlight Adaptation for Black Bars with Subtitles," focuses on improving picture quality when watching letterboxed content, like Hollywood movies, on an LCD display.

The application was filed just months ago, in September of 2011, and comes as rumors of a full-fledged Apple television set continue to build. One report this week claimed that Apple design chief Jonathan Ive has a 50-inch prototype set located in his secure work studio at the company's corporate headquarters.

A common problem with LCD displays is the ability to show "true" black colors on the screen. This becomes amplified when black bars are included in a video, such as when watching a letterboxed film. The difficulty of adjusting the backlight properly when the black bars are present can result in poorer quality of the remainder of the video.

Even though modern television sets are built with widescreen 16-by-9 aspect ratios, Blu-ray and DVD films, as well as those sold on iTunes, often show movies in an even wider format, leaving black bars at the top and bottom. For example, many movies are shot in Panavision's 2.35:1 ratio.

"Many video images are encoded with black bars, e.g., non-picture portions of the video images," the filing reads. "These non-picture portions complicate the analysis of the brightness of the video images, and therefore can create problems when determining the trade-off between the brightness of the video signals and the intensity setting of the light source. Moreover, these non-picture portions can also produce visual artifacts, which can degrade the overall user experience."

Further complicating picture quality and brightness is the fact that users can often view subtitles in the black bars located at the top and bottom of a widescreen film. This makes it even more difficult for the system to dynamically adjust and ensure the highest level of quality.

Apple's solution is a complex processing system that could "spatially vary visual information" on a display. This would dynamically adjust the backlight source on a screen, like a high-definition television set, in a way that would improve the picture quality.

The display would have multiple brightness settings for its backlight based on the processing of the image. For example, the "picture portion" of the screen would be illuminated by LED backlighting to an appropriate level, whereas the "non-picture portion," which would include the black bars, would have a different backlight setting.

The application describes an "extraction circuit" included in the display, which would calculate a brightness metric associated with the video signal. Then an "analysis circuit" would analyze and identify specific subsets of a video, like black bars that are shown when watching a movie.

The display would also include an "intensity circuit" that would determine the ideal intensity of the light source that illuminates the LCD display. The system could also employ a mapping function to determine optimum quality by using features like a "distortion metric" to limit image distortion.

Apple's system could also process the video signal in advance and synchronize the intensity of the light source based on the image currently being displayed.

"The system determines the intensity setting of the light source on an image-by-image basis for the sequence of video images, where the intensity of the given video image is based on the brightness setting and/or brightness information contained in the video signals associated with the given image," the filing reads. "Then, the system synchronizes the intensity of the setting of the light source with the current video image to be displayed."

The filing, made public this week by the U.S. Patent and Trademark Office, is credited to inventors Ulrich T. Barnhoefer, Wei H. Yao, Wei Chen, Barry J. Corlett, and Jean-didier Allegrucci.

Though rumors of a full-fledged Apple television set have persisted for years, they picked up once again late last year, when it was revealed that Apple co-founder Steve Jobs told his biographer that he had "cracked' the secret to building an integrated, easy-to-use television set.

"It will have the simplest user interface you could imagine," Jobs said, prompting speculation that an Apple-branded television would use Siri, Apple's voice control software featured on the iPhone 4S, as its primary input method.

Apple can deliver content across the Internet, they are not limited by broadcast standards like other TV providers. If they want the best TV out there, do a deal with the studios for higher definition content than everybody else. An iTV powered by iTunes, with 4k x 4k content, when the next best competitor has 1080p.

Apple can deliver content across the Internet, they are not limited by broadcast standards like other TV providers. If they want the best TV out there, do a deal with the studios for higher definition content than everybody else. An iTV powered by iTunes, with 4k x 4k content, when the next best competitor has 1080p.

How many users actually have the bandwidth to access 4k content? I don't think it's as simple as just offering higher HD content.

Apple can deliver content across the Internet, they are not limited by broadcast standards like other TV providers. If they want the best TV out there, do a deal with the studios for higher definition content than everybody else. An iTV powered by iTunes, with 4k x 4k content, when the next best competitor has 1080p.

Sounds great but do you really think Apple would do this? I mean their Apple TV isn;t even 1080. If anything they're usually behind in everyone else and charge more. ODN;t get me wrong I love Apple and would love for something like this. But I just don't see it, literally...sorry bad pun...

if this method is going to be included in their upcoming (rumoured) TV set, I think it's bad news rather than good. If they use this technique, then it seems to me that they can't at the same time use the other technique they recently patented whereby a layer of OLED acts as a sort of dynamic backlight/mask which would seem on the surface to be a much more interesting and truly new approach.

Or they could just do the sensible thing and use plasma. LCDs still aren't a patch on plasmas for black level, colour reproduction, and refresh rates. All LCDs can do that plasmas can't is burn eyes out with stupid amounts of brightness.

The SoC in your DTV already does all of the above in this article if it has 2D local dimming of the LED backlight. Most mid -> high end now have this as standard.

Aspect ratio info is ( depending on the codec ) coded into the elementary stream in the picture headers or via the embedded userdata. Additionally many systems have either SW or HW inbuilt BBD ( black bar detection ) when this info is available.

In short, nothing innovative here. The PQ ( picture quality ) processing in most TVs is where the innovation lies and is highly complex. Apple does not have this IP, something I know to be true as they tried to licence the IP for reasons known only to them.

The SoC in your DTV already does all of the above in this article if it has 2D local dimming of the LED backlight. Most mid -> high end now have this as standard.

Aspect ratio info is ( depending on the codec ) coded into the elementary stream in the picture headers or via the embedded userdata. Additionally many systems have either SW or HW inbuilt BBD ( black bar detection ) when this info is available.

In short, nothing innovative here. The PQ ( picture quality ) processing in most TVs is where the innovation lies and is highly complex. Apple does not have this IP, something I know to be true as they tried to licence the IP for reasons known only to them.

That's what I love about this place. There's members with expertise in a lot of different areas. That's so much better than just guessing.

The display would have multiple brightness settings for its backlight based on the processing of the image. For example, the "picture portion" of the screen would be illuminated by LED backlighting to an appropriate level, whereas the "non-picture portion," which would include the black bars, would have a different backlight setting.

I've often wondered why no manufacturer has had a television where the letterboxed portion of the screen simply turns off. This sounds like Apple may be the first to actually do that. Still though, this would do nothing to improve picture quality (despite what the patent suggests); it would just give the illusion that the LCD has better black levels than it actually does. Another trick in a deep bag of tricks used to compensate for the technology's inherent shortcomings.

Apple can deliver content across the Internet, they are not limited by broadcast standards like other TV providers. If they want the best TV out there, do a deal with the studios for higher definition content than everybody else. An iTV powered by iTunes, with 4k x 4k content, when the next best competitor has 1080p.

Before posting, I wish that people Googled existing HDTV manufacturers or visited the TV department in their nearest Best Buy to bone-up on the current state of product offerings.

Two things:

Video with black bars at the top and bottom is called "letterbox." I have never seen video broadcast OTA in letterbox format. However, many movies on cable is shown in letterbox format. In the case of DVD and Blu-ray, however, letterbox is the rule rather than the exception. The takeaway message is that Apple breaks no new ground by distributing programming in letterbox format.

We now go from the ridiculous to the sublime. In the most optimistic scenario this side of outright fantasy, Apple's new HDTV sets will compete with the likes of Pioneer Elite LED HDTVs and not home theaters. A 4k display is sufficient for digital commercial cinema but is massive overkill for the home living room or den.

The Pioneer Elite is worth examination because the family of HDTVs feature Intelligent Variable Contrast. This technique sounds at the very least like an early iteration of the patented Apple process. Also, the line appears to be based on panels supplied by Sharp, the rumored supplier of Apple's panels.

Current TV's know when the aspect ratio is different than the TV itself and add the black bars, which are illuminated in, well, black. Can't the black borders be eliminated from illumination?

Yes, that simple. Each pixel can be turned on or off, no? Or isn't that the case?

No, that's not the case. Each pixel does not have its own light source. Each pixel attempts to block out as much light as it needs to from the main source. You haven't paid very much attention to TVs in the past 10 years have you?

No, that's not the case. Each pixel does not have its own light source. Each pixel attempts to block out as much light as it needs to from the main source. You haven't paid very much attention to TVs in the past 10 years have you?

Nope, but you knew that after reading (and before posting)

Quote:

Originally Posted by Cory Bauer

That is unfortunately not the case with LCD or Plasma, but that is the capability that OLED will bring to televisions when it becomes more affordable in the near future.

if this method is going to be included in their upcoming (rumoured) TV set, I think it's bad news rather than good. If they use this technique, then it seems to me that they can't at the same time use the other technique they recently patented whereby a layer of OLED acts as a sort of dynamic backlight/mask which would seem on the surface to be a much more interesting and truly new approach.

It's a thing with tech research an patents. During R&D companies stumble over hundreds and thousends of interesting ideas with are patentable. In the end they will use just some dozens for actual products. Just saying, but I guess it's nothing new to you.

Sounds great but do you really think Apple would do this? I mean their Apple TV isn;t even 1080. If anything they're usually behind in everyone else and charge more. ODN;t get me wrong I love Apple and would love for something like this. But I just don't see it, literally...sorry bad pun...

Another good question would be: why would the content providers produce content in that format for a limited subset of sets

As I suspect you're aware, most if not all of the "technologies" used in LCDs to make up for their shortcomings end up introducing all new problems. "Intelligently" dimming parts of the screen can result in odd shifts in brightness mid-scene, dark scenes being dimmed to the point of being unwatchable, haloing effects, and from what I remember far worse viewing angles. All of the sake of trying to make LCDs reach black levels they can't actually achieve. This is much like the motion interpolation they introduced a few years ago to try and compensate for LCDs poor motion handling; that "intelligent" feature resulted in a lot of artifacting and tearing as the tv tried to make up frames in-between real frames. Not to mention the side-effect of making every movie look like a cheap daytime soap opera.

As Apple has waited this long to get into the TV business, if they're really going to do it I don't know why they wouldn't just wait another year or two for OLED to become affordable. LCD is a crap technology for television.

I see the charm, and this is exciting in that in offers a glympse of Apple's continuing tech innovations. But I still think the key to "cracking" TV is in Steve's comments regarding the integrated system of content providers and set top boxes and TV units is accurate: it isn't the tech that is hard - it is the business model. IOW: content content content. AppleTV remains a "hobby" because without user controled content (read: a la carte channels or programming) there isn't anything new for apple to offer beyond it.

Once they get a license deal in place, like they did for music on iTunes, this will be "cracked." To get that, they need to show studios (like the music labels) that they have a platform that can Kill It. Like iPods, and perhaps iBooks is becoming.

As an aside, the next thing apple could totally Kill It with is a iOS capable nano/iPhone on your wrist. The tech challenge is significant, but a wearable, portable computer that has a cell chip, BT, GPS and wifi? w.o.w.

And a prediction for 2012: You'll see the eBook readers (nook, kindle, etc) being offered for free. Yeah, that's right. Free. Maybe bundled with a "buy 10 ebooks, get the reader free" kinda thing, but what retailer won't go berzerk to put a device that at the touch of a button, makes a purchase in the hands of as many people as they can?

My $.02. Yeah, I'm a heretic. And I'm wrong about a lot of things. Oh well.

A very fair observation. The ISPs would do their own innovation - as in innovative tiered data plans.

And its not just a matter of dataplan, ISP networks cant support everyone streaming video at the same time. They slowdown peer2peer to avoid there network getting crush by the load. If too many people start streaming HD, they will have to slow it down.

Forget massive video streaming over the net, its not possible with current ISP networks. Don't forget malicious slowdowns done on purpose with the intention of screwing up Apple streams. I guaranty you that IPS that also offer live TV (like cable) will do this.

I see the charm, and this is exciting in that in offers a glympse of Apple's continuing tech innovations. But I still think the key to "cracking" TV is in Steve's comments regarding the integrated system of content providers and set top boxes and TV units is accurate: it isn't the tech that is hard - it is the business model. IOW: content content content. AppleTV remains a "hobby" because without user controled content (read: a la carte channels or programming) there isn't anything new for apple to offer beyond it.

Once they get a license deal in place, like they did for music on iTunes, this will be "cracked." To get that, they need to show studios (like the music labels) that they have a platform that can Kill It. Like iPods, and perhaps iBooks is becoming.

You are of course right. What I'd expect Apple to do, if the content providers would only allow them, is offer a subscription plan to Apple TV and iTV owners that grants them X number of shows per month for X number of dollars, plus X number of movies. Plans would be tiered, with prices like $29/mo, $59/mo and $99/mo. You'd subscribe to shows, not channels, and you'd get a number of allotted movie rentals per month as part of your subscription.

A consistent and continuous bill from customers each month is the only way I could see Apple offering something more affordable than the current $3-per-episode structure, which would become absurdly expensive if you watch 4 or more shows a night, every night.

The LCDs can't display proper blacks thing seems like such a dated issue. Newer ones provide relatively deep blacks and contrast ratios at least in line with most crts if not higher. Displaying an absolutely neutral black is a separate issue.

As I suspect you're aware, most if not all of the "technologies" used in LCDs to make up for their shortcomings end up introducing all new problems. "Intelligently" dimming parts of the screen can result in odd shifts in brightness mid-scene, dark scenes being dimmed to the point of being unwatchable, haloing effects, and from what I remember far worse viewing angles. All of the sake of trying to make LCDs reach black levels they can't actually achieve. This is much like the motion interpolation they introduced a few years ago to try and compensate for LCDs poor motion handling; that "intelligent" feature resulted in a lot of artifacting and tearing as the tv tried to make up frames in-between real frames. Not to mention the side-effect of making every movie look like a cheap daytime soap opera.

As Apple has waited this long to get into the TV business, if they're really going to do it I don't know why they wouldn't just wait another year or two for OLED to become affordable. LCD is a crap technology for television.

You are entitled to your opinion of course but I would argue this exact same thing but reversing the positions of OLED and LCD technology.

LCD tech is far superior to OLED and (opposite to your argument), all that OLED had going for it initially was superior contrast and cheap manufacturing. All subsequent "fixes" for all the many many things wrong with OLED (poor colour reproduction and poor performance in sunlight being the biggest two), have introduced more problems and more cost.

Here we are today years and years after the introduction of OLED and LCD panels *still* have better colour reproduction, better viewing angles, and better performance in daylight. They have also closed the contrast ratio gap substantially.

OLED on the other hand has made only minor improvements (if at all) in fixing the problems associated with that display technology. They still have that infinite contrast ratio, but they still can't reproduce colours accurately and still don't work very well in anything but a dimly lit room. On top of all that, every attempt to "fix" OLED's problems (along with some price gouging), has added substantial cost to the panels themselves.

No, that's not the case. Each pixel does not have its own light source. Each pixel attempts to block out as much light as it needs to from the main source. You haven't paid very much attention to TVs in the past 10 years have you?

Backlit LED TVs (and even some edge lit) do not have a main light source. There is an array of LEDs behind the LCD screen. Whilst it isn't a light per pixel, the LEDs are small enough they can vary the light across the screen to get deeper blacks. Watch a widescreen movie and the LEDs behind the bars of the letterbox will be off.

I'm not certain what is new with Apple's proposal that isn't already being done on TVs wih local dimming.

The SoC in your DTV already does all of the above in this article if it has 2D local dimming of the LED backlight. Most mid -> high end now have this as standard.

Aspect ratio info is ( depending on the codec ) coded into the elementary stream in the picture headers or via the embedded userdata. Additionally many systems have either SW or HW inbuilt BBD ( black bar detection ) when this info is available.

In short, nothing innovative here. The PQ ( picture quality ) processing in most TVs is where the innovation lies and is highly complex. Apple does not have this IP, something I know to be true as they tried to licence the IP for reasons known only to them.

Well, it might be innovative if the existing technology can only do BBD or simple aspect-ratio calcs. What Apple appears to be thinking of is a display where the black areas may be anywhere on the screen.

Imagine a TV that can have video, images, content guides, etc. in one or more resizable/relocatable windows (like we have now on our PCs). You would need some pretty sophisticated software to know where to dim the backlight.

You are entitled to your opinion of course but I would argue this exact same thing but reversing the positions of OLED and LCD technology.

LCD tech is far superior to OLED and (opposite to your argument), all that OLED had going for it initially was superior contrast and cheap manufacturing. All subsequent "fixes" for all the many many things wrong with OLED (poor colour reproduction and poor performance in sunlight being the biggest two), have introduced more problems and more cost.

Here we are today years and years after the introduction of OLED and LCD panels *still* have better colour reproduction, better viewing angles, and better performance in daylight. They have also closed the contrast ratio gap substantially.

OLED on the other hand has made only minor improvements (if at all) in fixing the problems associated with that display technology. They still have that infinite contrast ratio, but they still can't reproduce colours accurately and still don't work very well in anything but a dimly lit room. On top of all that, every attempt to "fix" OLED's problems (along with some price gouging), has added substantial cost to the panels themselves.

Your marks against OLED are the exact opposite of what I know to be true. From one of Sony's professional OLED field monitor descriptions:

Quote:

...offers picture contrast greater than a CRT display and is less affected by ambient light, allowing images to be viewed even in strong sunlight.

And regarding color accuracy, the description of another one of Sony's professional OLED monitors:

EDIT: I wonder if you're not thinking of AMOLED, for which the flaws you mentioned are most certainly true. That is not the tech being used in broadcast displays and soon TVs.

Quote:

Originally Posted by gwmac

2) No content to take advantage of the enhanced screen. Not even Blu Ray can show the detail or color range. So without content no real advantage. Similar to the problem 3D Tv's have.

That...doesn't make a lot of sense. A better screen makes everything look better. Few if any tvs today can show the full contrast ratio of movies, because they can't reach infinite blacks and have rather poor color accuracy out-of-the-box. Seeing a wider color gamut would not be necessary to see the improvements an OLED screen offers over LCD; infinite contrast, more accurate color and great motion resolution.

One of Apple's big strengths is mobile devices, but they can't really bring that to bear in this fight.

One thing they could potentially leverage is iTunes Extras. Most of the time, with movies, it's a big yawn. But for TV shows, you only return to them week after week if you actually like the characters, or want to be one of them. In that sense people might actually watch iTunes Extras for a TV show.

The SoC in your DTV already does all of the above in this article if it has 2D local dimming of the LED backlight. Most mid -> high end now have this as standard.

Aspect ratio info is ( depending on the codec ) coded into the elementary stream in the picture headers or via the embedded userdata. Additionally many systems have either SW or HW inbuilt BBD ( black bar detection ) when this info is available.

In short, nothing innovative here. The PQ ( picture quality ) processing in most TVs is where the innovation lies and is highly complex. Apple does not have this IP, something I know to be true as they tried to licence the IP for reasons known only to them.