Will copyright hamper IP closed captioning?

The FCC is setting up rules for closed captioning on broadband television, but …

The Federal Communications Commission is in the process of implementing the Twenty-First Century Communications and Video Accessibility Act. Passed by Congress last year, the CCVA requires the FCC revise its rules to mandate closed captioning on IP television, and the Commission is running a proceeding on how to implement the law. Not surprisingly, copyright questions are coming up fast in discussions between the agency and video providers, especially when it comes to enhancing captions in various ways.

Under the FCC's proposal, video programming distributors and providers "would not be required to improve caption quality," the agency's Notice of Proposed Rulemaking explains. "[R]ather, they would be required to ensure that the quality of captions does not decline when delivered via IP as compared to when shown on television. To the extent that VPDs/VPPs have permission to alter captions on the programming so that they improve the viewing experience, we propose that they be permitted to do so."

But some IP video distributors and their representative trade associations insist that intellectual property restrictions require the FCC to tread gingerly on this requirement. Expressions of caution have been submitted by AT&T and Microsoft. Here is the National Association of Broadcaster's commentary on the matter:

"'[E]ncouraging' captioners to provide high-quality captions (a goal fully supported by NAB and its members) should not be confused with a rule that would impose a quality mandate," the NAB notes. "To the extent the Commission takes any action regarding performance objectives, it should establish a safe harbor by which a covered entity that uses the same or substantially the same captioning used on television will be deemed in compliance."

In various contexts, VPDs would be unable to provide captions better than those available on television "because any alteration to the captions would violate the VPOs copyright to those captions."

The Public Knowledge advocacy group pushes back on this assertion. Public Knowledge argues that copyright should not be seen as an impediment to improving captioning on IP television.

Thanks to the fair use limitations in US copyright law, enhancing captions either in the form of corrections or better graphic quality "would not cause VPPs/VPDs to infringe the copyright in the underlying video programming," according to the group's filing with the FCC. "Indeed, despite the claims of several commentors, VPPs/VPDs would even be able to add captions to programming without violating copyright law."

Lost in transcoding

Captioning on television for the hard-of-hearing has been around since the early 1970s. At first it appeared on select programs in an "open" form—everyone could see it. In 1977, the FCC reserved a specific line on TV screens for "closed captioning"—text derived from the programming speech that could be turned on or off at the viewer's discretion.

The Communications and Video Accessibility Act applies this requirement broadly to distributors and providers of IP video programming. It also tells the FCC to establish a Video Programming Accessibility Advisory Committee, which submitted the following recommendations on July 12:

that the Commission require IP-delivered captions to be complete, such that "[n]othing must be lost in transcoding when converting captions between conventional broadcast captioning formats and Internet;"

that "[f]or Internet-delivered caption content, the positioning information as originally authored shall be made available to the consumer device;"

that the accuracy of IP-delivered video programming must be "equal to or greater than the accuracy of captions shown on television;"

that the Commission require IP-delivered captions to possess sufficient timing, such that "[a]ll processing through the distribution chain, including transcoding, must provide a timing experience that is equal to or an improvement to the timing of captions provided in the captioning shown on television;"

that a user’s Internet-connected media players should support the ability to change character color, opacity, size, font, background color and opacity, character edge attributes, window color, and language.

Don't preempt fair use

None of this is impeded by copyright, Public Knowledge's staff attorney Jodie Griffin argues. Here's a recap of fair use's four factors for whether the limitation is at play.

the purpose and character of the use, including whether such use is of a commercial nature or is for nonprofit educational purposes;

the nature of the copyrighted work;

the amount and substantiality of the portion used in relation to the copyrighted work as a whole; and

the effect of the use upon the potential market for or value of the copyrighted work.

Griffin notes that the point of captioning is not to duplicate something in order to resell it, but to make it easier to understand. "When adding or altering captions," she writes, "the purpose and character of the use is somewhat unique in that the captions are ultimately used alongside the original copyrighted work":

Essentially, video captioning is best understood, not as creating a new work or creating new copies of a work, but as enabling audiences to access the original work in a meaningful way. Even if the VPP/VPD distributes video programming commercially—which may not always be the case—the purpose of the captions is to make the video programming accessible and to comply with the statutory obligations of the CVAA, not to distribute the captions themselves for a profit.

In fact, better captioning increases the value of a work, thus improving its market value. "The Commission should not preemptively discourage such activity through its implementation of the CVAA, which is meant to improve access to communications for persons with disabilities," PK advises.

Along the same lines, allowing users to alter captions by changing or enlarging the font should be understood as a non-infringing activity. Such an adjustment isn't a public performance—it's a private interaction between the user and the content.

"If the actual user is the one modifying the presentation of the captions to make them more effective, the fair use factors weigh even more heavily toward a finding of fair use," Griffin adds:

It may be helpful for the Commission to consider user-controlled captioning options not as the creation of a new or altered work, but rather as the process by which a person with a hearing disability experiences an audio-visual work. Improving the content or presentation of captions simply helps the individual view the programming, much like adjusting the volume or brightness on a television set.

PK got in just under the wire on this proceeding. In late October, the FCC extended the reply comment date to November 1, the day the public interest group submitted its commentary.

26 Reader Comments

In various contexts, VPDs would be unable to provide captions better than those available on television "because any alteration to the captions would violate the VPOs [video programming owners] copyright to those captions."

Call me naive, but to think that anyone would equate closed captioning to copyright violation is mind-boggling. Considering most captions on broadcast TV are full of typos and fully omitted words, lag several seconds behind, and are generally poor quality, I'd say improving that would only help things for both the provider and the content owners.

However, if the closed captions are embedded in the video (as opposed to an overlay stream), and encoded in the video package... then altering that video (improving graphics quality, etc) would require "cracking" the video codec to re-overlay better captioning...

And, if the FCC "allows" broadcasters to correct the video, but it is encoded with a proprietary DRM... then the broadcasters would have a "mandate" to be able to force the content owners to disclose their DRM so the broadcaster can "correct" the captioning...

And... that means the DRM key is "out"; not in the wild yet, but 1-foot-out-the-door as far as the DRM-owners are concerned.

goglen: You are probably right. But I think that any DRM that prevents people with disabilities from accessing something that they normally would be able to should fall outside legal protection. Isn't it about time that hard of hearing citizens get the amount of attention on this issue they should have gotten years ago?

I think the reason is they want the exemption is so that they can advance the control of copyrighted materiel.If they grant this then they can use it in court against other types of fair use by saying "Hey you gave this an exemption but this is close enough that it doesn't have one so we can can sue you for breaking the law"

As a broadcaster, there is no DRM on any of the feeds or codecs used for video delivery to us, or cable headends for that matter, just encrypted satellite delivery so you can't tune it yourself with any off the shelf sat receiver. The DRM exists only in your tv and blueray players as well rokus or other such devices.

the biggest issue with this will be defining some sort of ip delivery standard. Broadcast has standards defined in ATSC but nothing exists like that for iptv. Also does this apply to hulu or YouTube? If so where does the FCC boundaries end? Broadcast makes sense in that its part of our public service commitments, but hulu and YouTube aren't bound by such requirements legally so how do you enforce it? Uverse is also treated differently than cable or broadcast, so my guess is, in court the FCC will be on shaky ground.

I'm profoundly deaf. So all I can say is, wherever possible, the goal needs to be improvement of closed captioning, both its veracity and the options available to users of it.

I am sensitive to the needs of the content owners, though. I understand that captioning is an expense, and that expense should be minimized where possible. So I hope that a clear standard can be created that will allow content creators to caption their work product once, and then not have to worry about it again.

These copyright challenges seem silly to me. I fully defend the rights of copyright holders, but these guys need to show an actual use-case demonstrating that captioning devalues their product in some way if they want such concerns to be taken seriously.

i think they should use a better font for closed captioning and make it so it doesnt need the black background

I don't know how different this is with digital TV, but with the old analog system, the presentation was a function of the television set. The CC data (along with other data) is carried in the "vertical blanking interval" which is the black "bar" one could see if the picture rolled when playing with the "vertical hold" control (this will be complete gibberish to you if you're under 30 or so I imagine).

The US Copyright code (http://copyright.gov/title17/92chap1.html) defines a derivative work as follows:A “derivative work” is a work based upon one or more preexisting works, such as a translation, musical arrangement, dramatization, fictionalization, motion picture version, sound recording, art reproduction, abridgment, condensation, or any other form in which a work may be recast, transformed, or adapted. A work consisting of editorial revisions, annotations, elaborations, or other modifications, which, as a whole, represent an original work of authorship, is a “derivative work”.

Note the condition in the last sentence that in order for the modified work to be a derivative work, the modifications must add up to "an original work of authorship".

If the transcription of a court proceeding does not constitute authorship, then I don't see why the correction of captions would.

Simply because there is no exception in the law defined for them. There does exist a specific exception for court proceeding transcriptions, however. It really is that simple. And yes, it is really that stupid.

In various contexts, VPDs would be unable to provide captions better than those available on television "because any alteration to the captions would violate the VPOs [video programming owners] copyright to those captions."

Call me naive, but to think that anyone would equate closed captioning to copyright violation is mind-boggling. Considering most captions on broadcast TV are full of typos and fully omitted words, lag several seconds behind, and are generally poor quality, I'd say improving that would only help things for both the provider and the content owners.

Hopefully this gets judged as fair use, as it should be.

Here in Sweden we sometimes have *very* ambitious "captioneers". For some shows you could think there's been +10 people spending over a month on captioning all the right things and timing it just right. But of course, we have our fair share of stupid captions too.

My first thought was that they're trying to avoid the obligation to provide captions at all.

Second thought is that they're trying to make damned sure they don't wind up with any obligation to do a better job of it than is typical on broadcast television.

I don't know how this stuff works out in the US, but here in Australia it's become a lowest-bidder job and live captions in particular are dreadful. Even many recorded programmes have quite bad captions. The whole thing is a profit centre for Macquarie Bank, though probably not a very large one compared to their other scams.

The sad (?) thing is that generally you do better caption-wise to download a TV show via BitTorrent and snarf a community-produced subtitle file from one of the websites that host such data. Sometimes you can get proper captions, sometimes all you can get are English subtitles, but either way they're usually a lot better than the commercial product.

However, if the closed captions are embedded in the video (as opposed to an overlay stream), and encoded in the video package... then altering that video (improving graphics quality, etc) would require "cracking" the video codec to re-overlay better captioning...

And, if the FCC "allows" broadcasters to correct the video, but it is encoded with a proprietary DRM... then the broadcasters would have a "mandate" to be able to force the content owners to disclose their DRM so the broadcaster can "correct" the captioning...

And... that means the DRM key is "out"; not in the wild yet, but 1-foot-out-the-door as far as the DRM-owners are concerned.

I suspect that is the real concern...

Which is why, once again we come back to the stupidity of IP law, the impossibly of having any perfect DRM system, and the world being forced to stop on behalf of Big Media.

And yet another example of where current IP law HINDERS the progress of science and useful arts. Which is in direct violation of the Constitution, but it isn't like any of the US politicians know what the Constitution is anymore.

In various contexts, VPDs would be unable to provide captions better than those available on television "because any alteration to the captions would violate the VPOs [video programming owners] copyright to those captions."

Call me naive, but to think that anyone would equate closed captioning to copyright violation is mind-boggling. Considering most captions on broadcast TV are full of typos and fully omitted words, lag several seconds behind, and are generally poor quality, I'd say improving that would only help things for both the provider and the content owners.

Hopefully this gets judged as fair use, as it should be.

They are just trying to make a problem where a problem doesn't exist. It happens pretty often these days.

In various contexts, VPDs would be unable to provide captions better than those available on television "because any alteration to the captions would violate the VPOs [video programming owners] copyright to those captions."

Call me naive, but to think that anyone would equate closed captioning to copyright violation is mind-boggling. Considering most captions on broadcast TV are full of typos and fully omitted words, lag several seconds behind, and are generally poor quality, I'd say improving that would only help things for both the provider and the content owners.

Hopefully this gets judged as fair use, as it should be.

They are just trying to make a problem where a problem doesn't exist. It happens pretty often these days.

The only problem is with (1). However, if the head-end stream is not encrypted, as stated by the MVPD earlier in the comments, there's no problem. The MVPD is already authorized for (3) by their upstream customer.

I'm profoundly deaf. So all I can say is, wherever possible, the goal needs to be improvement of closed captioning, both its veracity and the options available to users of it.

I am sensitive to the needs of the content owners, though. I understand that captioning is an expense, and that expense should be minimized where possible. So I hope that a clear standard can be created that will allow content creators to caption their work product once, and then not have to worry about it again.

These copyright challenges seem silly to me. I fully defend the rights of copyright holders, but these guys need to show an actual use-case demonstrating that captioning devalues their product in some way if they want such concerns to be taken seriously.

There are no challenges. The vast majority of the content we are talking about here is already subject to the regulations that govern TV broadcasts. Simply use the work that has already been done and exploit the formats and standards that already exist. Pirates are doing it better and they are doing it for free. There's really no good reason for this issue to have been allowed to slide for so long.

I don't know anything about the transmission format used by DTV, but in the case of MP4 the subtitles are a separate, time marked track. Also the rendering of the text and the font is decided by the TV and not the transmitter.

BTW in the UK subtitles for transmitted TV has been handled by teletext feature. I don't know how much this varies from the solution used in the USA.

Ok, so I deal with this shit all the time, and it's a pain in the ass. That said, I also agree that the IP infringement claim is BS - but the real issue is where the liability for violations sits.

Let's be clear about the state of the tech to begin with - there is NO editing application (FCP, Avid, Premiere, Vegas, Edius, etc) that preserves CC data when the codec changes. Extraction of CC from an encode and transfer to a separate track, like the track that QT and MPEG4 supports, is not built into any encoder or editor I have seen, doing this requires running all files thru a separate application first, and even then, there is little chance that the data will be preserved and passed to the editorial output file.

Ok, so - that said, this is not about supporting day to day broadcasts - for 90% of the shows and workflows, folks are correct that it's not that big of a deal, you simply extract your CC, and upload the required file (for youtube it's .srt) and long with the movie and you are good to go.

The issue is in the 'edge cases.'Take something liek the Daily Show for instance, where they make extensive use of footage captured from other stations. Because the CC is NOT PRESERVED when they cut in these previously, the Daily Show basically makes new captions for that content. Let's say that FOX & Friends uses the CC placement options to display the captions on the left and right to match the speaker positions. One could argue that when the Daily Show CC'd files are uploaded to the Comedy Central website and the F&F clip has 'lost' it's positional info. So, if a complaint is filed, who is responsible? Daily Show? Fox? The 3rd party service that hosts and encodes files for Comedy Central? All of Viacom? Adobe, whose Flash Player is used?

Another example - HBO shows 'Inglorious Bastards' which includes subtitles of the german language. HBO then adds CC data for the English parts of the film. When they make this available via HBO go, the English CC data will meet the requirements for font, color and size changes, but the subtitle in the movie will not. Is HBO now libale for this?

Sure, these seem like outside cases, but as I read this, how the less straight forward cases work out is a serious concern...

This has been tried many times, but in truth there is just too much background noise to get consistant enough results.

What is interesting is that BBC does a lot of this using voice to text software, but they have speakers with trained diction repeat what is being said line by line, and use that as the input instead of the actual sound from the program.

Chaoswarrior is correct, there are currently no editors that preserve caption data through codec changes, or through renders generally either. And it can be a real pain in the ass. We live caption our newscasts, and we pay to have live captioning done for a couple religious half hour shows we air on sunday mornings, simply because they are too small to contract with someone to do the captioning for them. A typical captioning box to insert live captions or captions streamed from a prompter system runs about $5k for standard def, and about $10k for HD. So alot of small production done by churches don't have them.

Live captioning will never be as good especially if the people speaking have heavy accents, bad pronunciation, or use alot of words that are hard to spell as they are literally typing them captions as they hear it. They dial into a box that lets them listen to the audio live and type it into the caption encoder.

DTV has standards for caption data that is spelled out in the ATSC specifications, and the files used in play to air equipment preserve that data. However those files are much to large to stream easily as they are typically MPEG-2 Long gop record at 10-12megabit for SD and 35 megabit for HD. IPTV providers typically encode the transmissions in MPEG-4 to cut the data rates down. As to the transcode equipment they use for that, as long as it preserves the original data in a format the home users equipment can recognize and let you turn it on, then it should be no problem. As I said earlier the tricky part is building that into a web player to be consistant. Secondly the only way a web player could improve it would be to recaption the program, which isn't cheap and is time consuming.

ATSC spec also defines the ability to change the color, or caption additional data streams so you can have both english and spanish captions as well, but I have no idea if the IPTV gear supports those additional data streams.

Matthew Lasar / Matt writes for Ars Technica about media/technology history, intellectual property, the FCC, or the Internet in general. He teaches United States history and politics at the University of California at Santa Cruz.