As Web traffic has come to dominate the Internet, remedying the weaknesses
of the HTTP protocol has become critical. W3C has worked with the Internet Engineering Task Force (IETF) to
develop a number of refinements to HTTP, work that has culminated in a new
specification for the protocol. This new specification, called HTTP/1.1, has
just recently become an IETF Draft Standard.

IETF specifications proceed through three main steps: Proposed
Standard; Draft Standard; and finally Internet Standard. A
Draft Standard is considered to be the final specification, apart from only
small changes of a very specific nature. Only a short step away is the
Internet Standard - a specification that, to quote the IETF, is "stable and
well-understood, is technically competent, has multiple, independent, and
interoperable implementations with substantial operational experience, enjoys
significant public support, and is recognizably useful in some or all parts of
the Internet."

The standardization of HTTP was started within the IETF in late 1994, and
W3C has strongly supported these efforts. Significant W3C effort went into
HTTP/1.1, particularly from Jim Gettys, visiting scientist at W3C from Compaq,
HTTP/1.1 editor and co-author, and from Henrik Frystyk Nielsen, formerly W3C
staff, and co-author of HTTP/1.1.

W3C has several HTTP/1.1 implementations and a large community of
developers have already demonstrated their interest. Experimental
implementations include the libwww client
library, and Jigsaw, W3C's Web
server. These were among the very first HTTP/1.1 implementations, and they
played a key role in discovering errors in the HTTP/1.1 Proposed Standard, and
show the full potential of the protocol.

Features of HTTP/1.1

The focus of the work behind the HTTP/1.1 specification has been to
alleviate the most prominent problems in HTTP/1.0 which has led to serious
bottlenecks on the Internet as the Web continues to grow. The result has been
the specification of a protocol which will make the Web faster and more
efficient. The three main features of interest are as follows:

1) Support for Virtual Hosting

The rapid growth of the Web has produced a frenzy for domain names like
mycompany.example.com, often as important for corporate recognition as
a logo. Domain names may be infinite in number, but the IP addresses they
translate into are not, and IP address depletion has become a serious concern.
The HTTP/1.1 Host header field allows Web service providers to assign
multiple domain names to a single IP address in such a way that a Web server
can distinguish the home page for mycompany.example.com from
yourcompany.example.net without using more than one IP address.

2) Requests for information handled more efficiently

HTTP uses the Internet TCP/IP protocol stack. All information you
read or write on the Web is sent accross the Net in TCP/IP packets. A
TCP connection is really like a responsible courier getting around in a big
city (the Net) - it makes sure the data you send and receive reaches the final
destination realiably while avoiding traffic jams and allowing other people to
get through as well. The funny thing is that TCP drives an old car - it takes
time for it to warm up, and as soon as it is done, it cools off again very
quickly.

To function efficiently, HTTP must take advantage of TCP/IP's strengths and
avoid its weaknesses, something that HTTP/1.0 does not do very well. Whenever
a client accesses a document, an image, a sound bite etc. HTTP/1.0 creates a
new TCP connection and as soon as it is done, it is immediately dismissed and
never reused. As a result, TCP rarely has time to get warm leaving lots of
"cold cars" with little data creating a lot of traffic jams.

HTTP/1.1 fixes this in two ways. First, it allows the client to reuse the
same TCP connection (persistent connections) again and again when
talking to the same server. Second, it makes sure that the courier carries as
much information as possible (pipelining) so that it doesn't have to
run back and forth as much. That is, not only does HTTP/1.1 use less TCP
connections, it also makes sure that they are better used. The result is less
traffic jam and faster delivery.

3) Efficient Caching

Documents you read on the Web are often read by thousands and even millions
of other people at the same time. This of course keeps servers very busy.
Imagine that instead of having everybody talking to the same server people
could get the same information much closer to where they are. This is what
caching allows us to do.

Whereas HTTP/1.0 merely enabled caching, it did not specify any
well-defined rules describing how a cache should interact with clients or with
origin servers. The lack of control resulted in that most content providers
and users did not trust the HTTP/1.0 caching model and instead tried to
short-circuit it. The result was that many busy parts of the Internet were
bogged down even more. A major part of the HTTP/1.1 specification is devoted
to providing a well-defined caching model which allows both servers and
clients to control the level of cachability and the conditions under which the
cache should update its contents.

Digest Authentication

Another important part of HTTP/1.1 is the Digest Authentication
Specification. Digest authentication allows users to authenticate themselves
to a server without sending their passwords in clear text which can be sniffed
by anybody listening on the network. In HTTP/1.0, passwords are sent without
being encrypted using so-called basic authentication. Although not providing
real security, Digest Authentication is an important step in a making the Web
a more secure place to live.

The HTTP Extension Framework

A continuing area of interest is how HTTP can be extended according to the
needs of specific applications. HTTP has been extended locally, as well as
globally, in ways that few could have predicted. Current efforts span an
enormous range, including distributed authoring, collaboration, printing, and
remote procedure call mechanisms. The usual practice is to add new header
fields to the protocol, and rely on the software at the other end to recognize
the header and process it accordingly. This, however, is the equivalent of
relying on magic! A standard framework for defining extensions has for some
time been badly needed.

The HTTP Extension Framework provides a
simple yet powerful mechanism for extending HTTP. The Framework enables
authors to introduce extensions in a systematic manner: programmers will be
able to specify which extensions are introduced along with information
about who the recipient is, and how the recipient should deal
with them.

The specification has been accepted as an experimental RFC as of February
2000 (RFC 2774). Here are
some W3C specifications that are already using it:

The P3P specification will enable Web sites to express their
privacy practices and users to to exercise preferences over those
practices. P3P products will allow users to be informed of site
practices, to delegate decisions to their computer when possible, and
allow users to tailor their relationship to specific sites.

Jigsaw has already an experimental implementation of
the extension framework and implementation is ongoing in the libwww.

Libwww and HTTP/1.1

Libwww is a highly modular, general-purpose client-side Web API written in
C for Unix and Windows. It is well suited for both small and large
applications, such as browser/editors, robots, batch tools, etc. Pluggable
modules provided with libwww include complete HTTP/1.1 (with caching,
pipelining, PUT, POST, Digest Authentication, deflate, etc). As for all W3C
OpenSource code, the purpose of libwww is to provide an environment for
experimenting with extensions and new features. The focus of libwww is
performance, modularity, and extensibility. Libwww now supports a large
community of authors and boasts a number of new features: HTML 4, XML, RDF,
SSL and much more.

Jigsaw and HTTP/1.1

Jigsaw is not only a server, it also provide a reusable HTTP/1.1 stack,
with a RFC 2616 compliant cache in its latest 2.1.1 release. It allows every
Java program to use a HTTP/1.1 compliant stack without modification.