Java Beyond the Browser: The Channel MetaphorMarimba's Castanet Addresses Several Constraints of Java Appletsby Bill VennersFirst Published in JavaWorld, December 1996

Summary
Although Java applets demonstrated a new way to think about
software delivery and use, they brought with them some new
constraints on software size, structure, and security. This
article takes a look at Marimba Castanet, one company's approach
to dealing with the constraints imposed by applets.

When
Java landed on the world stage in the
spring of 1995, it represented a paradigm shift for the software
industry. It demonstrated a new way to administer and deliver
software in the dawning network age. Java technology was well
suited for networked environments because of its platform
independence, built-in security mechanisms, and robustness. Java
applets showed how software could be administered on a single
server and downloaded to multiple clients via a Web browser. When a
new version of a program was ready, only the server needed
updating. Clients would automatically download the new version
the next time they accessed the Web page.

The trouble with applets
Although implementing programs as applets
embedded in Web pages greatly simplifies some aspects of software administration
and delivery, it also places some constraints on the form and
size of the programs. For example, developers are compelled to
use the browser metaphor for the user interface of their
programs. In this metaphor, the various functions of a program
are divided among several applets, which are placed on different
Web pages. Applets are downloaded when the user accesses the
containing Web page. Although the browser metaphor can be a
simple, easy-to-understand user interface for some programs, it
can be cumbersome for others.

On the other hand, it is possible to write an
entire program as a single applet that opens up a separate window
from the browser, thus avoiding the browser metaphor. However, a
different problem manifests itself as an applet grows larger, because
applets usually are downloaded across the network each time they are
used. When people want to use an applet, they go to the
appropriate Web page in their browser, wait until the applet is
downloaded across the network, and then use it. Once an applet
exceeds a certain size, it becomes painful to use because it
takes too long to download.

The size limit of applets hinges on the time
it takes the applets to download. Various factors, such as
network bandwidth, network traffic, and user patience, influence
an applet's maximum practical size. Over time network performance
can be expected to improve, thus bringing some relief to the
applet size limitation. However, just as user expectations for
software have increased to coincide with advances in available disk
space, RAM, and processing power, so too will user expectations for
network-delivered software likely increase as network bandwidth improves.

Popular browsers have also imposed one other
restriction on applets in the interest of security: Applets
downloaded across the network aren't allowed to read or
write files on the local disk. This restriction may be relaxed in
the future, by allowing the user to specify varying degrees of
access to the local disk depending upon the source of the applet.
Until that day comes, however, this security restriction is one of
the barriers preventing applets from becoming a viable solution
for many types of software problems.

A new channel for network-delivered Java
A few weeks ago a new technology was unveiled
that addresses some of the problems with delivering software as
applets. Marimba Inc., founded last February by four former
members of the Java team at Sun, has released a beta version of a product called
Castanet. Castanet enables the network delivery of
bundles of software and content, which they call channels.
In the first version of Castanet, a channel is either a Java
program or a set of HTML pages. Clients use what Marimba calls a tuner,
which allows users to subscribe to channels. The server
uses what Marimba calls a transmitter to manage a
channel. Here's a graphical depiction of Marimba's
model and terminology:

A channel is, at its most fundamental level, a
directory hierarchy filled with files. Castanet provides a way to
distribute the channel's files and directory hierarchy from
a server to clients. When you subscribe to a channel, you
get a complete copy of the channel's files and directory
hierarchy on your local disk.

Java programs (applets and applications) are
composed of class files and, possibly, data files. With little or
no tweaking to source code, Java programs can be delivered as
Castanet channels. Castanet, therefore, allows Java programs to
be delivered across networks without a browser. In this scheme,
the Castanet tuner takes the place of the browser. This eliminates
the requirement that network-delivered Java programs adopt the
browser metaphor, because the program has jumped out of the
browser.

The channel metaphor
The Castanet tuner uses the desktop metaphor
rather than the browser metaphor. Channels look like regular
applications on the client's desktop. Therefore, the
developer has free reign in user-interface design. The channel
metaphor, then, is one in which network-delivered
applications, which are mirrored in entirety on the user's local
disk, appear as desktop applications to the user. The main
difference from the user's perspective between a Castanet channel
and a traditional desktop application is that the channel
automatically installs and evolves over time. In this regard
channels behave less like traditional desktop applications and
more like applets. The channel metaphor represents a new way of
using the network, but looks and feels like the old way of using
computers.

Java programs delivered as channels are free
from the size constraint of applets. The larger an applet is, the
longer it takes to download. Once an applet becomes too large,
the program must be partitioned into several applets and spread
across several Web pages. Java applets have this problem because
they are delivered on demand. The browser does not attempt
to retrieve the applet until the user requests the Web page that
hosts the applet. At that point, the user is ready to go, but
must wait for the applet to download. Java programs delivered as
channels do not have this problem, because channels are not delivered
on demand. They are, in effect, delivered before demand.

When you first subscribe to a channel, you must
wait while the channel's entire directory hierarchy is
downloaded to your local disk. This may take quite some time,
depending on the size of the channel. However, after the initial
download, only changes are delivered. Rather than downloading the
channel in entirety each time, as is done with applets, channels
are updated based on any files and directories that were added,
deleted, or changed since the last download. Channels are not updated
upon each use, as are applets, but rather on a regular frequency that
is specified by the channel developer. You don't have to
wait for a download when you want to use a channel. Because the
channel is already on your local disk, you can start using
it immediately.

Although channels break through the size
barrier of applets, there is still a size limitation to Java
channels gated by both disk space and download time. However, by
abandoning the "download on demand" model of applets, a
channel's maximum practical size is much greater than that
of an applet.

Differential updating
At the frequency specified for each channel by
the developer, the tuner sends a request to the appropriate
transmitter for an update. The request sent by the tuner contains
an "index" that identifies the version of the files and
directories current on the local disk. No state is kept on the
server that tells it anything about a client's current channel
version. The transmitter determines whether any changes are
necessary on the client side based upon the uploaded index, and
replies with the changes required to make current the
client's copy of the channel. Updates occur in units of
files. If only one byte is changed in a file on the server, the
entire file must be downloaded again to the client. This process
of sending only the changes is called differential updating,
a far more efficient use of bandwidth than the model used by
applets, in which the entire applet is sent every time.

Marimba has applied for a patent for their
solution to the problem of differential updating. Their solution
uses a "pull" model, in which the clients periodically
poll for updates, instead of a "push" model, in which
the updates are broadcast from the server. The pull model allows
for correct operation through firewalls, because the transaction
is initiated at the client rather than the server. The pull model
also works better with intermittently connected machines such as
notebook computers.

Because servers do not keep track of any
client's current version, the index sent by the tuner must
contain enough information for the transmitter to determine the
required changes on the client side, if any. Marimba designed a
format for an index that indicates the version of each file and
directory in the channel's hierarchy of files and directories.
The index data structure is a tree structure where each node of
the tree corresponds to one file or directory in the channel's
hierarchy. Each node in the tree contains a 128-bit
"fingerprint" that is generated by running the content of the
file or directory through the MD5 algorithm. Regardless of a
file's length, a 128-bit fingerprint is generated. The
fingerprint is used as an indicator of the file's content, because
the odds are low that two files, or two different versions of the
same file, will have the same fingerprint. A fingerprint is
generated for each directory as well, all the way up to the
top-level directory in the channel.

When a transmitter receives an index, it
compares the hierarchy of fingerprints from the client to the
hierarchy of fingerprints representing the most current state of
the channel. Comparison is aided by the fact that the
directories, and not just the files, are fingerprinted. For
example, if the fingerprint of the top-level directory matches,
the transmitter can assume the client is already up to date, with
no need to compare individual files. By comparing fingerprints,
the transmitter can determine if any changes are needed on the
client side, and if so, what changes are required.

The transmitter performs one other function to
ease bandwidth requirements: It caches the most recent indexes.
Each time a channel changes, a new index is generated that
represents the new state of the channel. When a channel changes
on the transmitter, the previous index is not thrown away. A
certain number of previous indexes representing the most recent channel
states is cached by the transmitter. When a client requests an
update, it doesn't initially send its entire index. Instead, it
just sends the 128-bit fingerprint for its top level directory.
The transmitter compares this top-level fingerprint with those
in its cache, and if a match is found, uses the cached index for its
compare. If the transmitter does not find a match for the top-level
fingerprint in its cache, the transmitter requests the complete index data
structure from the client, which replies with the complete index. Marimba's
mechanism for channel updating ensures that each client will
always be at one of the states downloaded from the server, and
never in an in-between state. Because most requests for updates
from clients will take place when the client is in one of the
most recent channel states, most update requests will not require
the client to send a complete index, thus saving bandwidth.

Persistent storage
Java channels have the same level of security
as applets, with one exception. Java channels are assigned one
directory on the local disk under which the channel can read and
write files. Java channels effectively have access to their own
persistent storage sandbox on the local disk. This addresses
another problem that applets currently have in the popular
browsers: safe access to persistent storage. Because each channel
is assigned its own unique area on the local disk, one channel
can't meddle with another channel's files, or with the
user's non-channel files. It provides security while
allowing persistent storage.

Offline use
The channel model also ensures that a program
can be used offline. With simple caching, such as is commonly
employed by browsers, you don't get a piece of data in your
cache unless you actually access that piece of data while
connected to a network. For example, if you visit a site about
camel grooming, which contains 20 Web pages packed full of useful information,
but you only access 18 of them in a frenzy of information
gathering, you'll only have 18 of those pages in your cache.
If, later, you wish to review the information offline, you can
call up the 18 pages in your cache, but won't be able to get
at the two pages that were not cached.

A Castanet channel is downloaded to the
client's disk in totality. Because one type of channel is a
Web site, an entire Web site can be mirrored on a client's
disk. If you subscribed to the camel-grooming Web site channel,
you'd get all 20 Web pages on your disk. The same holds true
for Java programs delivered as channels. They are always
delivered as complete packages. Therefore, if you are offline,
you may not have the latest version of the channel, but you will
definitely have a complete version.

Channel issues
One consequence of the channel scheme is that
it will normally use more disk space than a browser. With a
browser the network becomes an extension of your computer's
persistent storage. Applets are stored at servers, and you
download them across the network each time you want to use them.
This model implies that you are using the network to store your programs
and data, instead of your local disk space, but requires that you
wait for code and data to come across the network when you want
to use them. For programs that take a long time to download, you
may prefer to give up some local disk space in exchange for a
reduction in time spent waiting for the download. The channel
model consumes your local disk space to save your time.

One feature currently not available to Java
channel developers is a mechanism for inter-channel
communication, although Marimba may add one in the future.
Currently all channels started by a tuner share the same Java
Virtual Machine, a scheme Marimba may change in the future.
Therefore, they also have left the issue of inter-channel
communication to the future.

Other delivery metaphors
There are many companies currently promoting
alternatives to Web browsing, the dominant model for information transmission
on the Internet today. Web browsers "pull" information
down from servers. Information is delivered across the network at
the initiation of the client. Within this framework it is
difficult for Web sites to attract visitors, and to ensure their return
once they have visited. It is difficult for individuals to find
Web sites of interest to them and to determine when a Web site
may have changed enough to justify a return visit. New models,
such as that provided by NETdelivery, PointCast, or Netscape
Inbox-Direct, "push" information. Individuals subscribe
to various sources of information, which are delivered across the
network to their local disk at the initiation of the source.

Although the solutions provided by the delivery
schemes overlap somewhat with Castanet, they are primarily aimed
at solving a different problem. Most network delivery systems
attempt to make it easier for sources of information to connect
with the willing consumers of the information across a network.
Castanet, on the other hand, is trying to make good on the
original promise of Java by enabling automatic installation and
updating of secure, robust, platform-independent software and
content across a network.

WordSurfer: From Java applet to Castanet channel
It is easy to convert a Java applet into a Castanet channel. All you
need to do, once you have a Castanet Transmitter running somewhere, is run a tool called the Castanet Putback. Putback will install your
applet onto the transmitter you specify. (You must know the password
to the transmitter, and your host needs to be on a charmed list of
hosts allowed access to the transmitter.) You simply enter information
about your channel into various edit boxes of the Putback user
interface, and push the "Apply" and "Putback" buttons. The Putback
program will send your channel's directory hierarchy up to the
transmitter, and you're done.

The WordSurfer applet currently is being transmitted as a Castanet
channel from JavaWorld. (It also appears in this HTML
document for demonstration purposes.) This applet is a simple English
vocabulary flash-card drill. I plan to change the words on the channel
every Sunday, so each week we'll present another set of ten words to
learn.

To subscribe to the WordSurfer channel, first download Marimba's
tuner. (There is a link to the tuner download page at the bottom of
this article.) Run the tuner, then click on the link below. You must
wait for the initial download to complete before the channel will
start. Look at the tuner to monitor the progress of the subscription
and download.

You can no longer subscribe to and start the Artima WordSurfer channel

If the one-click method doesn't work, you can also subscribe via the
several-click, multiple-keypress method. First, run the tuner. Select
the tuner menubar item, then select New Channel from the drop down
menu. Type nigeria.wpi.com:2483 in the edit box
labeled Transmitter. Type Artima WordSurfer in
the edit box labeled Channel. Then click Subscribe.

The WordSurfer applet demonstrates one kind of
program that fits the channel metaphor. WordSurfer combines code
with content, all of which can be delivered as files. The content
will change more frequently than the code, but over time, the
code will also change. The subscriber gets a service, in this
case, a Java gizmo that exercises and hopefully boosts the
subscriber's vocabulary. The subscriber doesn't worry
about pulling down the latest version of the software or content.
The program doesn't necessarily even have discrete version
numbers that are known to the user. It just evolves over time.

Because the WordSurfer applet is small, it is
practical to deliver it as an applet as well as a channel. It is
shown below in applet form as a demonstration. Only the channel
(not this applet) will be updated weekly, so if you want to
become a weekly word surfer, you'll have to subscribe to the
channel.

To use this applet, simply type in a word in
the edit box in response to the prompt. If you answer correctly,
the next card will be displayed. If you answer incorrectly, your
answer will be highlighted so you can try again. To see the
correct answer just hit Return in the edit box without typing
anything. Clicking Random will cause the
cards to be rearranged into a random order. Clicking Ordered will put
them back in order. After all ten cards have been presented, the first card
is shown again and the same drill repeats. Happy typing.

About the author
Bill Venners provides custom software development and
consulting services in Silicon Valley under the name Artima Software Company. He has
been object oriented for 5 years, primarily working in C++ on MS
Windows. Before that he did a lot of C on UNIX and assembly
language on various microprocessors. He is currently attempting
to write the Great American Java Book. Reach Bill at bv@artima.com.