Tuesday, April 01, 2008

What Is the Open Web and Why Is It Important?

What is the Open Web?

Folks toss the term "Open Web" around a bunch, but what exactly is it? Is the Open Web HTTP, HTML, JavaScript, etc., or is it something deeper? Rather than a laundry list of technologies the Open Web is a set of philosophies. These philosophies include:

Decentralization - Rather than controlled by one entity or centralized, the web is decentralized -- anyone can create a web site or web service. Browsers can work with millions of entities, rather than tying into one location. It's not the Google or Microsoft Web, but rather simply the web, an open system that anyone can plug into and create information at the end-points.

Transparency - An Open Web should have transparency at all levels. This includes being able to view the source of web pages; having human-readable network identifiers, such as URLs; and having clear network entry points, such as HTTP and REST exposes.

Hackability - It should be easy to lash together and script the different portions of this web. MySpace, for example, allows users to embed components from all over the web; Google's AdSense, another example, allows ads to be integrated onto arbitrary web pages. What would you like to hack together, using the web as a base?

Openness - Whether the protocols used are de facto or de-jure, they should either be documented with open specifications or open code. Any entity should be able to implement these standards or use this code to hook into the system, without penalty of patents, copyright of standards, etc.

From Gift Economies to Free Markets - The Open Web should support extreme gift economies, such as open source and Wikis, all the way to traditional free market entities, such as Amazon.com and Google. I call this Freedom of Social Forms; the tent is big enough to support many forms of social and economic organization, including ones we haven't imagined yet.

Third-Party Integration - At all layers of the system third-parties should be able to hook into the system, whether creating web browsers, web servers, web services, etc.

Third-Party Innovation - Parties should be able to innovate and create without asking the powers-that-be for permission.

Civil Society and Discourse - An open web promotes both many-to-many and one-to-many communication, allowing for millions of conversations by millions of people, across a range of conversation modalities.

Two-Way Communication - An Open Web should allow anyone to assume three different roles: Readers, Writers, and Code Hackers. Readers read content, Writers write content, and Code Hackers hack new network services that empower the first two roles.

End-User Usability and Integration - One of the original insights of the web was to bind all of this together with an easy to use web browser that was integrated for ease of use, despite the highly decentralized nature of the web. The Open Web should continue to empower the mainstream rather than the tech elite with easy to use next generation browsers that are highly usable and integrated despite having an open infrastructure. Open should not mean hard to use. Why can't we have the design brilliance of Steve Jobs coupled with the geek openness of Steve Wozniak? Making them an either/or is a false dichotomy.

Notice something very important; at no time above did I bind the Open Web to a particular set of technologies. Today the above philosophy is instantiated using a particular set of technologies, including URLs, HTTP, HTML, CSS, JavaScript, etc. However, if we define the Open Web in terms of these technologies, then we risk losing sight of what makes the web special and being able to have the intellectual nimbleness to evolve the infrastructure of the web. For example, we can and should evolve better layout languages than CSS, better document formats than HTML, etc., especially if we want the web to survive as a long-term social institution and public good, similar to the electricity grid, public-water systems, etc. We will be fighting yesterdays battle while allowing new, proprietary technologies to take over if we focus on technologies rather than philosophy. If we take the long term view, how can we give the web an open enough infrastructure to evolve over time and meet each generations needs, while maintaining its structure enough to actually mean something and stay true to its promise, similar to the U.S. Constitution?

Why does the Open Web matter?

The Open Web is like something from an Arthur C. Clarke science fiction story: its a globe spanning, hypertext network containing billions of documents, conversations, and applications, used by a huge cross section of society. Who would have thought it ever would have been successful or stayed as open as it has? It's not controlled by any one government or company. Our historical closeness to the web creates a kind of myopia, where we can't see how amazing it is. It's a billion Library of Alexandria's dropped into our laps.

Douglas Engelbart, father of much of what we have in the computer industry, including the mouse, hypertext, and groupware, thought that computers would become as fundamental to humanity as the development of writing and language have been. Language is probably only about forty-thousand years old, while writing is only about eight-thousand years old. If you pull back and take the larger view, the web and computers are part of a grand development playing out over decades around new tools for communication. Writing and language have fundamentally changed our sense of self, with positive and negative ramifications; computers and the web hold the same promise, though it will take decades for this to play out.

If the web and computers hold this promise, its important to keep the resulting system as open and accessible as possible. Do we want a system that devolves into something like Ancient Egypt, with an authoritarian force controlling and centralizing the water supply? Many archaeologists believe that deep control of access to water, literally something required for life, lead to the longest known authoritarian civilization in history, lasting for thousands of un-broken years. For example, what if the pencil and paper had never escaped the grasp of the Church? If we can keep the open web nimble and open, it can set the stage to fuel further innovations and inventions, just as writing and language gave rise to books, social polities, etc.

How Can We Support the Open Web?

If we agree that the Open Web is important, how do we create a way to update the web and keep it relevant? The U.S. Constitution, for example, includes special provisions to evolve itself and stay relevant. Even with its warts, the U.S. is now the world's oldest and continuous republic.

The web's existing update mechanisms just don't work. It takes years for new features to go from proposal to show up across enough browsers to be used consistently; this is a recipe for fail if we want the web to exist as a long-term entity, rather than a one-hit wonder.

I joined Google to help with a project known as Gears. Gears is an open source plug-in that teaches current web browsers new tricks. Gears is a clever way to raise the bar cross-browser and cross-platform, today, running inside of Firefox and Internet Explorer on Windows, Mac OS X, and Linux. No more waiting years for features to show up across all browsers and platforms.

I want to look back five years from now and say that I worked with the community to build an open source update mechanism for the web. Why can't we rev the base infrastructure of the web much quicker, plus create more robust, open extension points along the entire web stack, ala Greasemonkey? Is Gears the answer to this? I'm not sure, but its the best answer we have today. Gears is a great way to get the conversation started, plus get HTML 5 out to today's browsers.

On a more prosaic level, Gears gets the following features into today's browsers without waiting years, ready to use by web devs right now:

Great post, Brad. I like the idea of a "transparent" web a lot. The web and companies that build for it should ideally be as transparent with their technology as they are forced to be with their finances.

This relates directly to the "open" question. I'm a strong believer in open protocols and open standards in particular, as you clearly are too. In the p2p world, open protocols are at least as important and open source. In terms of transparency, I think they're more important. The reason is that it's so much easier to read one protocol spec than to familiarize yourself with an entire code base. That gets in to the "usability" of the web and companies that build for it. There should be just as much care going into usability for programmers as goes into usability for the end user. This is something Apple largely misses and that Amazon nails.

Ideally companies would write open source code and use open protocols, but I think open protocols give a bigger bang for the buck from a usability perspective. That's again not to discount open source by any means, but rather to level the disparity between emphasis on open source versus open protocols.

There are concerns about open web, especially given the relative slow evolution of browsers in comparison to the fast advancement of other technologies (Flash, Silverlight etc). I did a post on this a couple days ago Will Rich Internet Become Rich Man's Internet?.

Also, it would be really helpful if you take a look at OpenAjax Runtime wish list, and give your oppinions on it:

@CoachWei "There are concerns about open web, especially given the relative slow evolution of browsers in comparison to the fast advancement of other technologies (Flash, Silverlight etc)."

Slow evolution of browsers? You must be kidding!!! Flash has been around for 13 years now, and I'm not impressed by the performance of the Adobe virtual machine. If you see the performance explosion of JavaScript engines in browsers like Webkit and Chrome, that's amazing!

Dear Brad, while I agree with your view that the Open Web should be a philosophy, and not technologies, I have a difference of opinion where you compare it to writing and language. While the terms 'language' and 'writing' have given a different meaning to the human society, they have also created so many different societies--English, French, German, Chinese, Hindi, and so on. When it comes to the interactions between these societies, communication still remains a challenge. I'm afraid that the Open Web is also panning out in a similar way, with different 'societies' being formed online--Google, Microsoft, Java, and so on. While everyone might be promising to promote the Open Web, everyone has their own 'proprietary' architectures. So you can't migrate easily from one technology to another. You can't enjoy the same service on multiple browsers, and so on. If the web is really to become open, then why is everyone building up forts around their technologies? With you all the best in making the web an open place!