Introduction

One of the most exciting promises of digital libraries is access
to a great variety of information and services that transcend
what is available today through on-line services, such as the
World-Wide Web (WWW). A library is more than just stacks of materials
on shelves; it is also highly trained people that provide valuable
services. These services include such things as organization
and cataloging, research, notification of new publications,
and so forth. Indeed, one of the greatest assets of libraries
are these high-valued services. The WWW, while it probably contains
more information than any single traditional library, is arguably
not as useful as a traditional library because it lacks these
services (particularly organization and sophisticated search support).
No one is dismantling their libraries because of the WWW yet.
The University of Michigan Digital Library Project (UMDL) [1,2]
believes that a successful digital library needs to provide both
access to a wide variety of valuable content and services.

Because the range of both content and services that are possible
for a digital library are potentially large (we cannot even imagine
what will be available or needed in the future), there will be
no single, complete digital-library solution. Rather, we expect
that as editing tools become better and access to networks becomes
easier and cheaper, there will be millions of content suppliers;
"everyman" can become a vanity press on the information
superhighway. We believe that the days of centralized suppliers
of information (e.g., large publishing houses and traditional
libraries) are numbered, and that the traditional notion of a
"collection" will span multiple databases, each residing
in a different place in cyberspace.

Furthermore, the creativity of users of digital libraries will
spawn thousands of different, specialized services (e.g., notification
and translation, even special collections of information). Perhaps
most importantly, methods of organizing information will transcend
a single "digital library," in that it is unlikely that
a single indexing or naming scheme (e.g., the Dewey Decimal System)
will be used across the multiplicity of digital libraries that
are sure to emerge. Thus, we must create flexible software architectures
that can federate as many content suppliers, information-organizational
schemes, and service providers as possible, and yet scale to the
extremely large size needed to support the digital libraries of
the future.

Considering this view of digital libraries, we have developed
some guidelines and objectives for our system. First, the guidelines:

Given that many digital libraries will emerge, we want to
make ours as attractive to users and content and service providers
as possible. Thus, we intend to make the fewest and least-restrictive
standards.

The only way to ensure intellectual property in the future
is to provide incentives for its creation. Thus, we intend to
make support for economic incentives an integral part of the UMDL
architecture. This covers a wide range of issues, from definition
and protection of intellectual property rights through payment
for the use of intellectual property. Please note: we are not
establishing policies related to rates or payment, rights of users
to access the contents of the library, or other related issues
such as "fair use". We simply plan to have the machinery
in place to support whatever policies may arise in the future.

The elements of the library (services and collections of information)
are autonomous in that these elements will make decisions based
on their own perspective. In other words, there is no central
authority that can press an element into service. All elements
are considered peers, and thus interaction is achieved entirely
through negotiation processes.

Broadly speaking, the objectives of the architecture are to provide
services that fall under the following categories:

Registration: maintaining a comprehensive list of all the
agents (collections, user interface, and others) in the UMDL.

Commerce support: providing mechanisms to support commerce
for information goods, and protecting intellectual property and
privacy.

Furthermore, we require that the architecture have the following
properties:

Modular, in that new elements can be added or removed without
effecting the operation of other elements;

Scaleable, to allow for the potential of millions of constituent
elements;

Extensible, to allow new elements (collections, data types,
services, etc.) to be easily added to the digital library.

In the remainder of this paper, an overview of the UMDL architecture
is given. We describe the notion of software agents and types
of agents in UMDL, and then describe how agents interact to provide
service.

Agents

The architecture is based on the notion of a software agent. An
agent represents an element of the digital library (collection
or service), and is a highly encapsulated piece of software that
has the following special properties:

Autonomy: the agent represents both the capabilities (ability
to compute something) and the preferences over how that capability
is used. Thus, agents have the ability to reason about how they
use their resources. In other words, an agent not have to fulfill
every request for service, only those consistent with its preferences.
A traditional computer program does not have this reasoning ability.

Negotiation: since the agents are autonomous, they must negotiate
with other agents to gain access to other resources or capabilities.
The process of negotiation can be, but is not required to be,
stateful and will often consist of a "conversation sequence",
where multiple messages are exchanged according to some prescribed
protocol, which itself can be negotiated.

Autonomy is critical to scaling UMDL to a large size because autonomy
implies local or decentralized control. As a result, we do not
have to update some "master" program everytime a new
agent is added to UMDL. The effects of adding or removing an agent
are propagated locally using a set of protocols. Thus, there is
no need for global coordination among all agents [4]. The notion
of decentralized control of autonomous agents is similar to the
way our economy works. Each of us is similar to an agent in that
the decision about how money is spent is done individually. These
spending decisions do not require communication across the entire
economy (e.g., when ones buy a car, she do not need to tell the
whole country or even the car manufacturer, just the car dealer),
nor does one need to get permission from a central authority.
Similarly, UMDL agents can make decisions and form teams at a
local level, without requiring interaction with all agents in
the system or with a central authority.

Negotiation is complementary to autonomy, in that autonomous agents
must be capable of making binding commitments for the system to
work. Thus, when agents negotiate and strike a deal (i.e., something
of value is exchanged for something else of value), the agents
are bound to fulfill that deal. It is possible, and even likely,
that some deals will allow agents to back out. This "feature",
however, must be explicitly negotiated in our system.

The UMDL is populated by three classes of agents:

UIAs (User Interface Agents) provide a communication wrapper
around a user interface. This wrapper performs two functions.
First, it encapsulates user queries in the proper form for the
UMDL protocols. Second, it publishes a profile of the user
to appropriate agents, which is used by mediator agents to guide
the search process.

Mediator agents [8], of which there are many types, perform
a variety of functions: essentially, all tasks that are required
to refer a query from a UIA to a collection, monitor the progress
of the query, transmit the results of a query, and perform all
manner of translation and bookkeeping. Presently, two types of
mediators populate the UMDL. Registry agents capture the address
and contents of each collection. Query-planning agents [5] receive
queries and route them to collections, possibly consulting other
sources of information to establish the route. Another special
class of mediators currently being developed, called facilitators
[7], mediate negotiation among agents [3].

CIAs (Collection Interface Agents) provide a communication
wrapper for a collection of information. While performing translation
tasks similar to those performed by the UIA for a user interface,
the CIA also publishes the contents and capabilities of a collection
in the conspectus language (described in the next section,
"What the architecture provides").

Figure 1: UMDL agent types

As the architecture is developed, the broad classes of agents
depicted in Figure 1 will be continually refined; specialized
agents will be added to the system as needed (the modularity property).
For example, we can create user interfaces that are customized
to a particular class of users, rather than to a particular collection
or access mechanism (e.g., Boolean search over controlled vocabulary).
In addition, the ability to team agents (as described in
the next section, "What the architecture provides")
dynamically creates new services with new agents, which is especially
important since we anticipate the agent population will be constantly
changing.

What
the architecture provides

From a user's perspective, the types of high-level support that
make a digital library worth using, such as searching, will be
performed by a team of agents. For example, consider Figure 2,
where a user (through the UIA) is searching for all articles by
"Joan Q. Publique". Assuming that all agents have registered
with the registry agent, the UIA contacts a query planner by first
requesting the registry for a query planner that knows about author
searching. The query planner then goes to the registry to get
the addresses of a name authority (meta data that gives variations
of Joan Q. Publique) and a name index (a partial listing of collections
that contain works sorted by author). The planner then interrogates
the authority, and then the index, finally determining the address
of a particular collection. The collection is then accessed by
the UIA using a protocol specific to the CIA.

It is easy to image how this process can be extended for different
types of search by adding new types of agents (e.g., subject indexes
and new kinds of query planners). The teaming methods gives the
architecture a dynamic planning ability[5] that is critical for
finding the best way to perform some service, as well as easily
incorporate new types of search methods. There is, however, a
cost.

This cost is coordinating the agents, which includes communication
and negotiation of which negotiation may prove the more expensive.
Communicating among agents certainly takes more time than would
be required by a monolithic system. We believe, however, that
improved network technology will ameliorate these costs, making
them insignificant. The major overhead may come from negotiation.

Recall that agents are autonomous and cannot be coerced into responding
to a request for service. What is not shown in Figure 2 is interaction
with facilitators. It is possible that the UIA-query planner,
query planner-name authority, query planner-name index, and UIA-CIA
interactions will all require some type of negotiation (we assume
that registration is "free"). Striking deals among all
these agents could require significant time and computation resource.
This overhead, however, can be minimized by prearranged deals
among agents. For example, a UIA could buy a token that
specifies certain access privileges for a cartel of CIAs; similar
arrangements can be made among other agents.

We separate the activities of agents UMDL into two types: that
used to organize agents to perform the team building (called
architectural), and that used to perform the actual task
(called task), such as actually querying a database. Strictly
separating these activities allows us to reduce the commitments
that an agent must make to operate in our system (i.e., a CIA
is not required to support all query languages used in UMDL, only
those it chooses to support.). Thus, we require only that agents
use a language, called the conspectus language, designed
to support architectural activities(see the next section,
"The Conspectus and the conspectus language"); the decision
to support any particular task language is left up to individual
agents.

The distinction between architecture and task has advantages and
disadvantages. The advantages include minimal standards, and therefore
increased flexibility in creating agents. Furthermore, the agents
themselves are smaller, and therefore easier to build and maintain.
A disadvantage is that not all agents will have access to all
other agents. For example, if a CIA supports only Z39.50, but
a UIA uses some other language X (and no mediator exists that
can translate X to Z39.50), then that UIA cannot access the CIA.
We see, however, no practical solution to this problem at this
time.

Since it is impossible to create an architecture that has everything,
we prefer flexibility over guaranteed interoperability among all
agents. Task languages that will undoubtedly evolve over time,
as we learn more about digital libraries. By being neutral on
which languages are supported, we avoid having to rewrite significant
portions of our software as the languages change.

The Conspectus
and the conspectus language

The space of information in UMDL is potentially enormous, as is
the possibility of bringing the system to its knees with rogue
query processes. To limit queries to potentially applicable CIAs,
we reason about the contents of each collection to derive an estimate
of their likely usefulness. This leads us to a two-level partition
of the information space:

Conspectus: includes, among other things, the content
of the collection, the search capabilities of the search engine(s)
associated with the collection, and the structure of the material
(documents) in the collection.

Collection: the set of actual documents in a collection.
These documents are in native formats, and the search engines
are engaged through native query languages.

The conspectus is an abstracted description of the aggregate of
collections populating the UMDL. Additionally, the conspectus
is a normalized description of content. This is important,
as various collections will have different methods for describing
the same thing (e.g., title as TI or TL). To help normalize terms,
we are using a variety of thesauri developed by various researchers
around the world.

The conspectus is written in a language that we have defined (the
UMDL conspectus language, UCL). Although we retain complete control
over the UCL, the actual conspectus expressed in UCL will be specified
by the separate collections. Our aim is that UCL (and its associated
resources, such as various thesauri and cataloging systems) provide
sufficient structure for developing compatible representations
of collections. Thus, the conspectus provides interoperability
for various search and retrieval methods through a common representation
over collections.

Since the conspectus will be large both in scope and in size,
it will be distributed and hierarchically organized. We expect
to create special mediator agents whose sole responsibility is
to maintain the integrity of the conspectus.

Agents communicate using patterns of messages, where the content
of the message is specified by UCL and sets of performatives
describing the purpose of the communication (e.g., to ASK
or TELL something) [6]. The messages transmitted between the agents
describe capabilities, services, and other primitives. For example,
all agents use the ASK performative to make requests to the registry
for notification about classes of agents with certain capabilities.
The registry agent continues sending information about these agents,
as they come on-line, until the UNASK performative is received.

Another example performative set is TELL, which is typically used
in response to an ASK. The registry agent uses TELL to send the
names of agents that correspond to some capability specification.
The registry agent uses the UNTELL performative to express that
an agent is no longer available, or that its capabilities have
changed.

Protocols specify communication patterns among agents. In order
to participate in UMDL, an agent must use our protocols. Since
these protocols are minimally restrictive in how a task is accomplished,
we believe they are not a significant impediment to the development
of agents by third parties. Standardizing the protocols, but not
the task languages, strikes a balance between flexibility and
ease of integration into the UMDL environment.

The agent-identification protocol (used by both the CIA and query
planner in the example depicted in Figure 2) provides a way for
agents to locate other agents with specific capabilities (Figure
3). The requesting agent (R) uses the ASK performative
to describe the specific capabilities to the registry agent. The
registry agent executes a lookup operation to match the specifications
to the agents it knows about. Any matches are sent to the requesting
agent via the TELL performative. The ASK performative implies
a standing request for information about agents, so that the registry
agent continues to send R information about other agents
as they advertise their capabilities. When R receives information
about an agent (A) from the registry agent, it has the
option of storing that information in its local knowledge base
for future use.

If R no longer wants to receive information about an agent,
then it uses the UNASK performative to communicate this desire
to the registry agent. Upon receipt of the UNASK performative,
the registry agent stops sending information to R. If
A is no longer available, or has a change in capabilities,
then the registry agent sends the UNTELL performative to all agents
who received a TELL performative about A. Thus, the registry
agent must keep track of the agents to which it sent the TELL
performative.

Figure 3: Agent-identification protocol.

The performative and protocol features of the UMDL architecture
are general enough to accommodate a variety of actions within
the library. As illustrated here, the same protocol can be used
by several different agents to achieve their objectives. We expect
that once we have established a basic set of protocols, including
those for negotiations about intellectual property, they will
become relative stable even though the variety of information
and services in the library will grow enormously. In fact, the
stability of these protocols is the foundation for growth of the
system.

Status and summary

The UMDL is operational, and can be accessed through http://www.sils.umich.edu/Catalog/UMDL.html
. The current system has about 50 CIAs and basic search support.
We expect to have subscription, notification, and known-item search
running by the end of the calendar year. Two task languages are
supported: Z39.50 and FTL (a locally created query language).

The current system demonstrates that the agent architecture approach
outlined in this article is viable, and paves the way for more
interesting experiments with scaling both the total number of
agents as well as the types of services and collections available.
It is interesting to note that the architecture was able to handle
the addition of new services (new collections and a notification
service) without modifications to existing agents and protocols,
thus demonstrating properties of scaleability, extensibility,
and modularity.

Acknowledgments

The members of the UMDL architecture group contributed many of
ideas presented here. In particular, Fritz Freiheit provided helpful
suggestions to drafts of this paper.

The UMDL project is funded under a joint initiative of NSF/ARPA/NASA;
we are grateful for their financial support, and their enthusiasm
for the initiative. The views expressed in this paper are those
of the author only, and do not necessarily represent the views
of the funding agencies (nor of the UMDL project).