Whenever someone proposes EPUB as a solution, ask yourself a question: what’s the problem they’re trying to solve? As a standard drafted by the IDPF, a self-proclaimed “organization for the Digital Publishing Industry”, EPUB is built squarely to address the industry’s biggest headache: ensuring that, in the digital age, they retain the ability to charge money for distributing content. The best interests of authors or readers simply do not figure in the equation. Read more…

Accessibility is a feature every publisher needs to own and implement

Since adding the accessible DAISY format to our ebook bundles in late 2010, we’ve seen a slow but steady uptick in customers downloading and using these files. In looking at downloads month to month in 2012, we find that downloads are routinely into the high hundreds and often over the thousand mark (for more data on which formats O’Reilly customers are downloading, check out Joe’s post from earlier this year).

The Android ecosystem shares some of the same obstacles

Ebook vendors enjoy a closed loop ecosystem. They have millions of reader/customers who are satisfied with EPUB 2 display capabilities and devices. Amazon readers, for example, are largely content with the offerings in the proprietary Kindle store; they’re not lining up with torches and pitchforks to push for improvements. While publishers wait for eReader device manufacturers to add new features and EPUB 3 support, eBooksellers are just as happy to wait.

A user experience plea for more consistency across platforms

The “best price” phase of TOC NY 2013 registration is about to end. Don’t wait or you’ll end up paying more than you would today. To save even more on your registration, sign up here and use the discount code JOE20 to get an additional 20% off the current price on the conference package of your choice.

Ebook publishing is full of problem areas, most of which cannot be addressed through standardisation but can only come about via a sea-change in the behaviour and nature of the various participants in the ebook industry.

There are, however, several issues that could be addressed, at least partially, via standardisation, that would make everybody’s life easier if implemented.

Overrides

One of the major issues facing publishers today is the spiralling complexity of dealing with vendor rendering overrides.

Each vendor applies different CSS overrides with differing behaviours, sometimes even only enabling features through server-side manipulation, which means that proper testing of an ebook is not only difficult, but impossible.

If vendors cannot be talked out of requiring these overrides then they need to be standardised and normalised. Any reading system that implements a CSS override is in violation of how the CSS standard defines the cascade and so is in violation of the EPUB 3 standard.

CSS overrides come in four broad types:

Vendor styles only — The publisher’s styles are completely ignored in favour of the vendor’s.

Aggressive vendor styles, but publisher styles enabled — Very little is seen of the publisher styles in this scenario. They mainly surface in edge cases that weren’t accounted for in the vendor’s stylesheet.

Minimal overrides — The vendor only really enforces control over margins, backgrounds, and possibly font styles.

Publisher styles — The mode that the reading app goes into when the reader deliberately selects ‘publisher styles’. Under ordinary circumstances this would simply disable the overrides but in most reading apps this mode has a unique behaviour.

Our two experts wrap up their debate on the best path towards richer content

Over the past week I’ve posted excerpts from a very insightful email exchange between Bill McCoy of the IDPF and Sanders Kleinfeld of O’Reilly (first piece here and second piece here). This is the final installment of that thread and we pick it up where Bill suggests a tighter connection between the simple ebooks of today and the richer ones we’re likely to see in the future:

Distinguishing ebooks from apps

Bill: There’s a continuum between non-enhanced text-centric publications and enhanced interactive publications, not a dichotomy.

Sanders: Well, I think you always need to draw a line somewhere between “ebook” and “app”. That’s true right now, in that “Finding Nemo: My Puzzle Book” is an app and “Does the Noise in My Head Bother You (Enhanced Edition)” is an ebook. My current prediction is that in the future, the line is going to be drawn such that more and more content slides over to the “app” side, and more specifically that more of these will be web apps, as opposed to the native iOS/Android book apps that predominate now. It sounds like you believe the reverse will be true, and you may very well be right.

Honestly, I just want publishers to put out more and more awesome, innovative digital content, preferably crafted using open Web standards. If it’s easier for them to do it using EPUB 3, then I’m all for it. But if circumstances militate against this route, as they do now, I’d rather see them explore web apps than throw in the towel.

Bill: Your argument would seem to imply that if you need to add a video or an equation visualization widget to an eBook you should move to the “other side” and build a web app instead. That’s not realistic for publishers. A lot of content will bring limited enhancements to largely text-centric linear documents.

Your own “HTML5 for Publishers” may even be a good example. It obviously has a lot of enhanced capabilities. It can live as a web app but as far as I know it doesn’t, or if it does it isn’t the normal way it’s consumed. And if you did make it into a standalone web app you’d have to code up a whole skeleton of supporting infrastructure around it, using non-standard tools. Do you really expect every publisher to do this for every title?

Sanders: I disagree with this point, in that it implies that in circumstances where you want to add a video or another widget, it’s necessarily a lot of additional work to do a web app than an EPUB 3. There’s no reason why you can’t take the very same content that you zip up in an EPUB 3 and instead post it wholesale on the Internet as a website. And if you did want to add additional supporting infrastructure for a title, it could likely be reused for other titles; I don’t think it’s true that publishers would have to start over from scratch for every book.

Bill: And a whole lot of content is going to be less enhanced than “HTML5 for Publishers” (since after all it’s subject is interactivity). I think any title being created today with iBooks Author and Habitat will be doable with off-the-shelf EPUB 3 based tools and deliverable to multiple off-the-shelf EPUB 3 reading systems within the year.

Sanders: That’s great. I really do hope that proves to be true.

Closing the gap between HTML5 and EPUB 3 support

Bill: We are still in the initial rollout stage of EPUB 3 support. As EPUB 3 proliferates, there’s no reason for EPUB 3 support to continue substantially lag browser support.

If I thought reading system implementations would remain well behind browser stack level capabilities forever, that it wouldn’t become normal to expect to deploy the same wide variety of JavaScript libraries in an interactive ebook as in a web app, I’d be less positive. But I see things converging to a model where everyone will be using WebKit, in many cases the same one that the platform’s web browser is using, so no reason that it should be any less feature-rich. iBooks is already there, and things that work in browser-based web apps that are disabled there are only for business or security considerations, and I view the former as temporary. We’re catching up more than 10 years in one transition – it’s painful that it’s taking over a year to accomplish but that’s no reason to presume it won’t be successful. In fact I’d much rather have your help getting us there than having you arguing that we shouldn’t bother. But maybe I’m misunderstanding your POV and would welcome your thoughts and feedback.

Sanders: It’s great to hear you say that. I do acknowledge that my perspective on this debate may be overly biased by my experiences as an ebook developer for the past year, who has been greatly frustrated by the current state of the ereader landscape’s support for HTML5 features and how much it’s limiting innovation in the EPUB (and Mobi) space. But I know you’re traveling all over, fiercely advocating for EPUB 3, HTML5, and open Web standards, and have a much better perspective than I do where things are headed, and how fast we’ll get there. I’m not sure what you’re advocating publishers do in the meantime, though. Should they just wait? Should they predicate their digital publishing initiatives on whether NOOK will support Canvas in the next year?

I have a great deal of admiration for the IDPF and their mission to achieve broad-based support for EPUB3, and I most certainly regret that anything I said may have diminished the value of this goal or argued against it. At the same time, I hope it’s understandable why publishers might view putting all their eggs in the EPUB 3 basket to be undesirable at this time.

I believe this statement by Bill was one of the most important ones in the whole discussion:

That lag is one of my biggest concerns as well and why I feel publishers should focus on HTML5 itself and not something else that’s built upon HTML5 (and therefore requires more time for development and adoption).

What’s your opinion? Will the gap between HTML functionality and EPUB reader support for those features shrink or will there always be a lengthy delay in the latter’s ability to keep up with the former?

Why ebook publishing will look more like software development than print production

In an article posted a few days ago I shared the first part of an email exchange between Bill McCoy of the IDPF and Sanders Kleinfeld of O’Reilly. They were debating the merits of HTML5 and EPUB 3. In the second of this three-part series they dig deeper into the capabilities of EPUB 3 and what the future of this format might look like:

Why isn’t EPUB 3 leading to more innovative products?

Bill: It seems to me that your argument amounts to (to paraphrase): “EPUB 2 is good enough for plain text, everything else can and should be a web app, so let’s not bother with EPUB 3″. That’s not an entirely new argument, but I’m a bit surprised to hear it from you.

Sanders: I definitely did imply this, and I really wish I hadn’t, because at its heart, EPUB 3 *is a web app*; it’s just packaged according to a specific standard. I have absolutely no qualms about the feature set EPUB 3 was designed to support, and I think those features are exactly what is needed to do Category 2 digital content.

My complaint is that there’s really no sane way to do full-fledged Category 2 [more sophisticated, feature-rich] emedia *right now* in EPUB 3 given today’s landscape of ereaders and the features they support. But that’s really a knock against the tablet vendors, not EPUB 3, as I feel they’re the ones stifling innovation.

Anyway, if you are a publisher who wants to press forward today, you really have to pursue an app, in the sense of something that can either be served via a Web browser or compiled and distributed in an app store. I don’t really feel I can advocate that publishers do an EPUB 3 at this time, unless they’re content with limiting its marketability largely to the iPad, and even then, Apple does not make it especially easy to do a standard EPUB 3 for iBooks (because presumably they’d much rather you use iBooks Author).

But I do regret sounding so dismissive of EPUB 3 in my remarks. I also wish I had specifically mentioned Readium as an example of an app that holds a lot of promise in terms of furthering the goals of an open HTML5-based pathway for modern “ebook” content. I hope it helps crack the stranglehold that tablet vendors have over feature support.

Bill: [My first issue with what you noted earlier is that] publishers can’t afford to create custom web apps for each title at scale. A declarative content model and associated toolset will be required. Unless we want that model and toolset to be closed and proprietary, EPUB 3 or something very much like it will be needed.

As you know, creating compelling engaging web apps isn’t easy. Most publishers don’t have O’Reilly’s technical talent and resources and I doubt even O’Reilly has enough JavaScript software developers to make web apps for every title. There’s a reason there’s only one Financial Times and a reason there’s only one Principles of Biology, and that’s in large part that they are extremely expensive endeavors. At Google I/O the Chrome team spent a lot of time talking about how offline in the browser didn’t deliver a great user experience for things like Google Reader, and that’s why they are rolling out new packaged apps.

Publishers will demand tooling and that tooling will require underlying content representation that’s not just JavaScript, and will largely be assembling and configuring widgets, not writing them and the apps around them from scratch. HTML5-based tools like Habitat and iBooks Author have underlying representations but they are entirely closed and proprietary. And, I don’t think you are implicitly arguing that either of these toolsets is the path forward rather than a free and open content representation that enables multiple tools from different vendors as well as multiple distribution options.

Sanders: Agree with all your points above, and heck no, I am definitely not arguing for iBooks Author or Habitat over EPUB 3

Open source development will lead to additional EPUB 3 growth

Bill: If you get to the more micro level of features, EPUB 3’s “extra layer” (as you called it) has features like Media Overlays and declarative triggers. Do you really think that publishers who need to synch audio with text or wire up buttons to media controls should have to hand-code home-grown solutions every time they need to do it? How about other accessibility features?

Sanders: Excellent points. I agree, and I do not think I gave credit where it was due here to the packaging layer of EPUB 3. However, much of the hard work of creating an interactive web app exists whether or not it’s packaged in EPUB 3 format or not. Composing interactive games in <canvas> is basically the same challenge either way. I think that in the future, ebook publishing is going to look a lot more like software development than print production, so if content producers want to stay competitive, I think they will need to hire more software engineers.

I do think, though, that just as there’s been a huge wave of open source innovation that’s sprouted in the past several years around doing web apps for mobile and tablet devices (I’m thinking of software like jQuery Mobile and bootstrap.js), there will be a similar wave of development around tools and widgets facilitating the creation of interactive ebook content. It seems likely that this will facilitate and complement the growth of EPUB 3, which is great. The bummer right now is that EPUB 3 feels like more of a hindrance than a help, because I can’t get it to render the way I want in Nook Tablet’s ereader, but I can just unzip that same EPUB file and put it up as a web site, and come much closer to the results I want.

Bill: [My second issue with your earlier points is that] EPUB 2 isn’t good enough even for text-centric non-enhanced content.

For one thing we have fixed layout. For another, vertical writing and other global language features: even for a novel, EPUB 2 isn’t enough in Japan. Thirdly, accessibility. These are all defined in terms of modern HTML5/CSS features that aren’t part of EPUB 2. Fourthly, we have a much tighter spec for content and reading system conformance in EPUB 3 than in EPUB 2. So even if we accepted the proposition that publishers should create web apps to deliver highly interactive content, there’s enough reason to move non-interactive content to EPUB 3 to get these other benefits.

Sanders: Yes, I agree with this point. In retrospect I feel I was wrong to suggest that EPUB 2.1 was good enough as is for text-centric content. I do see the need for broader accessibility and foreign-language support.

Sanders makes a great point about EPUB 3 not being fully supported in most ereader apps. I’d like to think once that happens we’ll see much more innovation and less of a need for publishers to create platform-specific apps. I’m skeptical though. Ours tends to be an industry of followers where we wait for others to take the risk, prove the investment worthy and then jump in. I figure it will be the small, lean startups that will do more exciting things with EPUB 3 and the rest of the industry will follow. What’s your opinion?

Two experts help us navigate the ebook format jungle

One of the benefits of working on TOC is that I get to see some of the behind-the-scenes industry debates that take place via email. Since it’s “formats” month here in TOC-land I thought it would be fun to share a thread about HTML5 vs. EPUB 3 featuring O’Reilly’s Sanders Kleinfeld and the IDPF’s Bill McCoy. They’ve both agreed to share this thread with the TOC community since it helps clarify the state of both EPUB 3 and HTML5.

Your mileage may vary, especially on the Nook

Bill: Your interview with Joe for TOC was interesting. As you know from prior discussion I totally agree with you that the current situation is sub-optimal. It’s disappointing that nearly a year after EPUB 3 is out, B&N’s flagship tablet device doesn’t support EPUB 3 yet, even though it ships with a reasonably modern browser that does support HTML5. This is due to delays by their current vendor for eBook rendering software (Adobe) rather than a lack of desire on B&N’s part, but it’s still a problem. I also agree with you that where EPUB 3 is supported like Apple iBooks and Kobo the consistency in terms of being able to safely use leading-edge JavaScript libraries and other browser-stack features in disparate reading systems is not where it needs to be. While it’s true that EPUB 3 needs to be able to be supported on limited capability devices I think we didn’t help by making so many things optional and being silent on reading system support for various APIs that are de facto part of the browser stack but not officially part of the HTML5 spec normatively referenced by EPUB 3 spec (e.g. geolocation, local storage, XMLHttpRequest).

I also agree with you that Web-technologies-based apps are the future for experience delivery, both in browser and increasingly for native-class apps that are liberated from the browser (whether wrapped in PhoneGap or CEF, W8 Metro apps or the new Chrome Packaged App model Google rolled out this summer at I/O).

Sanders: Yes, I think what I find so frustrating is that we’ve got tablets on the market like the Nook, which use two different engines to render Web content–one for the Web browser and one for the ereader–and the ereader is lagging so far behind the browser in HTML5 support. The ereader is being treated like a second-class citizen, even though it’s ostensibly the primary feature of the tablet (or at least the primary feature by which the tablet is being marketed; I concede that there are many people buying the Nook who just want a low-cost Android tablet, and have no intention of reading ebooks on it).

One of the things I appreciate most about iBooks is that it uses the same Webkit engine as Mobile Safari, and if you test the same HTML5 content in Safari and in iBooks, you usually get the same results.

Distinguishing apps from ebooks

Bill: But… despite all this violent agreement… I’m having a hard time with your contention in the interview that “a lot less is going to fall in the eBook side than it does now for enhanced text and graphics… anything that’s more enhanced is going to drift over to the app side… we’ll have our standard EPUBs for fiction/nonfiction and then biology textbooks will be on the other side”. That’s what I want to probe on in this email.

Sanders: I’ve thought about this quite a bit after my initial conversation with Joe, and I think the debate of “ebook” vs. “app” can be pretty facile if you’re not careful about how you define those somewhat loaded terms. And I think I failed to do a good job of that in my interview. So let me step back and try to do better now.

If you define “ebook” as “something that can be represented in EPUB 2/3 or Mobi format and rendered in an ereader app”, and you define “web app” as “something that can be rendered in a Web browser and/or sold in an app store”, then you’re really just arguing about packaging, not the underlying content, because an EPUB 3 file is composed of HTML5, CSS, and JavaScript, just like a web app. In the quote you cite above, I wasn’t really trying to make an argument about packaging, as much as about the kinds of electronic media that publishers are creating right now, and will be creating in the future.

I think Phase 1 of ebook creation for publishers was basically, “Let’s take all our print books and digitize them so they can be read on a Kindle or iPad”, without much in the way of innovation in terms of interactivity, customized rendering for a reflowable context, or even hyperlinking.

I think we’ve now graduated to Phase 2, where publishers are thinking, “How can we make customized digital content for tablet devices, instead of plain-old text-and-graphics ebooks? Do I make an ‘app’ or do I make an ‘enhanced ebook?'” I think publishers are generally taking two approaches to this:

Approach #1: Hire on software developers to make full-fledged native apps they can sell in the app stores for iPad/iPhone/iPad and Android (lots of these are children’s titles, e.g. “Finding Nemo: My Puzzle Book“)

I’m rather against Approach #2, because I don’t think it’s especially innovative, and I don’t think it’s what customers really want out of next-generation e-content. I feel like the whole notion of “enhanced ebooks” is somewhat of a transitional concept, as publishers start making baby steps in rethinking how they produce content for a Digital First world. In the long term (next 5 years or so), I think a large part of the “enhanced-ebook” middle ground is going to go away, and ebook content is going to fall more neatly into one of two categories:

Category 1: The standard text-and-graphic content that you can read on even the lowest-end eInk ereader (because I don’t think all-text fiction/nonfiction is ever going to go out of favor)

Category 2: Everything else, which will be more and more app-like in the sense that it will be highly interactive, to-some-degree social (commenting, linked to Facebook/Twitter, etc.), and conceived from the start for a Web context (densely hyperlinked, with sophisticated mechanisms to search content and navigate it in a nonlinear fashion)

So, that’s what I was getting at above; I just wish I had articulated it better in my interview.

That’s just part one, folks. Stay tuned later this week for excerpts from the rest of the thread where Bill and Sanders talk further about the subtleties of EPUB 3, HTML5 and web apps.

EPUB 3: The future of digital publications

The first two parts of this three-part series covered the enduring need for portable documents and why PDF’s fundamental architecture is too dated and too limited to fill this need. In this final part, we’ll take a look at EPUB, the format that has rapidly emerged as the open standard for eBooks. It’s my contention that EPUB, not PDF, represents the future of portable documents in our increasingly Web-based world. Why? In short, EPUB addresses all the key limitations of PDF. EPUB is reflowable, accessible, modular (with packaging and content cleanly separated), and based on HTML5 and related Web Standards. It’s a truly open format, developed in a collaborative process to meet global requirements rather than by a single vendor to support its proprietary products. Let’s take a harder look at these points.

Sanders Kleinfeld on obstacles to a unified ebook format.

In a recent video interview, O'Reilly's Sanders Kleinfeld addressed a number issues surrounding ebook formats. He also talked about how vendors are among the biggest obstacles to an open, universal ebook standard and the end of DRM.

Bill McCoy on EPUB 3 and keeping pace with innovation.

In this video interview, Bill McCoy, executive director of the IDPF, says it's important to emphasize and encourage the innovative aspects of building upon EPUB 3, as long as that innovation doesn't lock consumers in to one closed silo.