Publication as a Working Draft does not imply endorsement by the
W3C Membership. This is a draft document and may be updated, replaced
or obsoleted by other documents at any time. It is inappropriate to
cite this document as other than work in progress.

This specification describes a general
SOAP-based protocol for enumerating a sequence of XML elements that
is suitable for traversing logs, message queues, or other linear
information models.

English

Last Modified: $Date: 2009/04/01 10:35:33 $

Introduction

There are numerous applications for which a
simple single-request/single-reply metaphor is insufficient for
transferring large data sets over SOAP. Applications that do not
fit into this simple paradigm include streaming, traversal, query,
and enumeration.

This specification defines a simple SOAP-based
protocol for enumeration that allows the data source to provide a
session abstraction, called an enumeration context, to a consumer
that represents a logical cursor through a sequence of data items.
The consumer can then request XML element information items using
this enumeration context over the span of one or more SOAP
messages.

Somewhere, state must be maintained regarding
the progress of the iteration. This state may be maintained between
requests by the data source being enumerated or by the data
consumer. WS-Enumeration allows the data source to decide, on a
request-by-request basis, which party will be responsible for
maintaining this state for the next request.

In its simplest form, WS-Enumeration defines a
single operation, Pull, which allows a data source, in the context
of a specific enumeration, to produce a sequence of XML elements in
the body of a SOAP message. Each subsequent Pull operation returns
the next N elements in the aggregate sequence.

A data source may provide a custom mechanism for
starting a new enumeration. For instance, a data source that
provides access to a SQL database may support a SELECT operation
that performs a database query and uses an explicit database cursor
to iterate through the returned rows. In general, however, it is
simpler if all data sources support a single, standard operation to
start an enumeration. This specification defines such an operation,
Enumerate, that data sources may implement for starting a new
enumeration of a data source. The Enumerate operation is used to
create new enumeration contexts for subsequent traversal/retrieval.
Each Enumerate operation results in a distinct enumeration context,
each with its own logical cursor/position.

It should be emphasized that different
enumerations of the same data source may produce different results;
this may happen even for two enumeration contexts created
concurrently by a single consumer using identical Enumerate
requests. In general, the consumer of an enumeration should not
make any assumptions about the ordering or completeness of the
enumeration; the returned data items represent a selection by the
data source of items it wishes to present to that consumer at that
time in that order, with no guarantee that every available item is
returned or that the order in which items is returned has any
semantic meaning whatsoever (of course, any specific data source
may provide strong guarantees, if so desired). In particular, it
should be noted that the very act of enumerating the contents of a
data source may modify the contents of the data source; for
instance, a queue might be represented as a data source such that
items that are returned in a Pull response are removed from the
queue.

Requirements

This specification intends to meet the following requirements:

Support enumeration of data sources
that cannot practically fit into a single SOAP
message.

Support both server-side and
client-side enumeration state maintenance.

Minimize additional mechanism
beyond the current web service architecture.

Notations and Terminology

This section specifies the notations,
namespaces, and terminology used in this
specification.

Notational Conventions

The keywords "MUST", "MUST NOT", "REQUIRED",
"SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY",
and "OPTIONAL" in this document are to be interpreted as described
in RFC 2119 .

This specification uses the following syntax to
define normative outlines for messages:

The syntax appears as an XML
instance, but values in italics indicate data types instead of
values.

Characters are appended to elements
and attributes to indicate cardinality:

"?" (0 or 1)

"*" (0 or more)

"+" (1 or more)

The character "|" is used to
indicate a choice between alternatives.

The characters "(" and ")" are used to indicate that contained items
are to be treated as a group with respect to cardinality or choice.

The characters "[" and "]" are used to call out references and
property names.

Ellipsis (i.e. "...") indicate points of extensibility.

XML namespace prefixes (see ) are used
to indicate the namespace of the element being
defined.

In addition to Message Information Header properties
,
this specification uses the following properties to define messages:

The elements defined in this specification MAY be extended at the
points indicated by their outlines and schema. Implementations MAY
add child elements and/or attributes at the indicated extension
points but MUST NOT contradict the semantics of the parent and/or
owner, respectively. If a receiver does not recognize an extension,
the receiver SHOULD ignore that extension. Senders MAY indicate
the presence of an extension that has to be understood through the use
of a corresponding SOAP Header with a soap:mustUnderstand attribute
with the value "1".

XML Namespaces

The XML namespace URI that MUST be used by
implementations of this specification is:

http://www.w3.org/2009/02/ws-enu

lists XML namespaces that are used in
this specification. The choice of any namespace prefix is arbitrary
and not semantically significant.

Prefixes and XML namespaces used in this specification

Prefix

XML Namespace

Specification(s)

wsen

http://www.w3.org/2009/02/ws-enu

This specification

s

http://www.w3.org/2003/05/soap-envelope

SOAP 1.2

s11

http://schemas.xmlsoap.org/soap/envelope/

SOAP 1.1

wsa

http://www.w3.org/2005/08/addressing

WS-Addressing

wsam

http://www.w3.org/2007/05/addressing/metadata

WS-Addressing

xs

http://www.w3.org/2001/XMLSchema

XML Schema ,

wsdl

http://schemas.xmlsoap.org/wsdl

WSDL/1.1

The working group intends to update the value of the Web Services
Enumeration namespace URI each time a new version of this document is
published until such time that the document reaches Candidate
Recommendation status. Once it has reached Candidate Recommendation
status, the working group intends to maintain the value of the
Web Services Enumeration namespace URI that was assigned in the
Candidate Recommendation unless significant changes are made that
impact the implementation or break post-CR implementations of the
specification. Also see
http://www.w3.org/2001/tag/doc/namespaceState.html
and
http://www.w3.org/2005/07/13-nsuri
.

Terminology
Consumer

The Web service that is requesting the data
enumeration from the data source

Data source

A Web service that supports traversal using
enumeration contexts via the Enumerate operation defined in this
specification

Enumeration context

A session context that represents a specific
traversal through a logical sequence of XML element information
items using the Pull operation defined in this
specification

Compliance

An implementation is not compliant with this
specification if it fails to satisfy one or more of the MUST or
REQUIRED level requirements defined herein. A SOAP Node MUST NOT
use the XML namespace identifier for this specification (listed in
) within SOAP Envelopes unless
it is compliant with this specification.

Normative text within this specification takes
precedence over normative outlines, which in turn takes precedence
over the XML Schema and WSDL descriptions.

Enumeration Messages

Enumeration contexts represent a specific
traversal through a sequence of XML information items. An Enumerate
operation may be used to establish an enumeration context from a
data source. A Pull operation is used to fetch information items
from a data source according to a specific enumeration context. A
Release operation is used to tell a data source that the consumer
is abandoning an enumeration context before it has completed the
enumeration.

Enumeration contexts are represented as XML data
that is opaque to the consumer. Initially, the consumer gets an
enumeration context from the data source by means of an Enumerate
operation. The consumer then passes that XML data back to the data
source in the Pull request. Optionally, the data source may return
an updated enumeration context in the Pull response; when present,
this new enumeration context should replace the old one on the
consumer, and should be passed to the data source in all future
responses until and unless the data source again returns an updated
enumeration context.

Consumers should not reuse old enumeration
contexts that have been replaced by the data source. Using a
replaced enumeration context in a Pull response MAY yield undefined
results, including being ignored or generating a
fault.

Once the last element in a sequence has been
returned, or the enumeration context has expired, the enumeration
context is considered invalid and the result of subsequent
operations referencing that context is undefined.

Callers MAY issue a Release operation against a
valid enumeration context at any time, which causes the enumeration
context to become invalid and allows the data source to free up any
resources it may have allocated to the enumeration. Issuing a
Release operation prior to reaching the end of the sequence of
elements is explicitly allowed; however, no further operations
should be issued after a Release.

In addition, the data source MAY invalidate an
enumeration context at any time, as necessary.

All messages defined by this specification MUST be sent
to a Web service that is addressable by an EPR
.

Enumerate

All data sources MUST support some operation
that allows an enumeration to be started. A data source MAY support
the Enumerate operation, or it may provide some other mechanism for
starting an enumeration and receiving an enumeration
context.

The Enumerate operation is initiated by sending
an Enumerate request message to the data source. The Enumerate
request message MUST be of the following form:

The following describes additional, normative
constraints on the outline listed above:

[Body]/wsen:Enumerate/wsen:EndTo

This optional element denotes where to send
an EnumerationEnd message if the enumeration is terminated
unexpectedly
(see ).
If present, this element MUST
be of type wsa:EndpointReferenceType. Default is to not send this
message. The endpoint to which the EndTo EPR
refers MUST support the EnumerationEndPortType portType.

[Body]/wsen:Enumerate/wsen:Expires

Requested expiration time for the
enumeration. (No implied value.) The data source defines the actual
expiration and is not constrained to use a time less or greater
than the requested expiration. The expiration time may be a
specific time or a duration from the enumeration's creation time.
Both specific times and durations are interpreted based on the data
source's clock.

If this element does not appear, then the
request is for an enumeration that will not expire. That is, the
consumer is requesting the data source to create an enumeration
with an indefinite lifetime. If the data source grants such an
enumeration, it will terminate when the end of the enumeration is
reached, or if the consumer sends a Release request, or by the data
source at any time for reasons such as connection termination,
resource constraints, or system shut-down.

If the expiration time is either a zero
duration or a specific time that occurs in the past according to
the data source, then the request MUST fail, and the data source
MAY generate a fault indicating that an invalid expiration time was
requested.

The SOAP 1.1 binding of this fault
is:

faultcode = s11:Client

faultstring = e.g., "invalid expiration time"

The SOAP 1.2 binding of this fault is:

s12:Code/s12:Value = s12:Sender

s12:Code/s12:Subcode/s12:Value = wsen:InvalidExpirationTime

s12:Reason/s12:Text = e.g., "invalid expiration time"

Some data sources may not have a "wall time"
clock available, and so are able only to accept durations as
expirations. If such a source receives an Enumerate request
containing a specific time expiration, then the request MUST fail;
if so, the data source SHOULD generate a fault indicating that an
unsupported expiration type was requested.

The SOAP 1.1 binding of this fault is:

faultcode = s11:Client

faultstring = e.g., "unsupported expiration type"

The SOAP 1.2 binding of this fault is:

s12:Code/s12:Value = s12:Sender

s12:Code/s12:Subcode/s12:Value = wsen:UnsupportedExpirationType

s12:Reason/s12:Text = e.g., "unsupported expiration type"

[Body]/wsen:Enumerate/wsen:Filter

This optional element contains a Boolean
predicate in some dialect (see )
that all elements of
interest must satisfy. The resultant enumeration context MUST NOT
return elements for which this predicate expression evaluates to
the value false. If this element is absent, then the implied value
is the expression true(), indicating that no filtering is
desired.

If the data source does not support
filtering, the request MUST fail, and the data source MAY generate
a SOAP fault as follows:

SOAP 1.1:

faultcode = s11:Client

faultstring = e.g., "filtering not supported"

SOAP 1.2:

s12:Code/s12:Value = s12:Sender

s12:Code/s12:Subcode/s12:Value = wsen:FilteringNotSupported

s12:Reason/s12:Text = e.g., "filtering not supported"

If the data source supports filtering but
cannot honor the requested filter dialect, the request MUST fail,
and the data source MAY generate a SOAP fault as
follows:

SOAP 1.1:

faultcode = s11:Client

faultstring = e.g., "filter dialect requested unavailable"

[optional]
detail/wsen:SupportedDialect = repeating; one per filter dialect
supported by the container

Other components of the outline above are not
further constrained by this specification.

Upon successful processing of an Enumerate
request message, a data source is expected to create an enumeration
context and return that context in an Enumerate response message,
which MUST adhere to the following form:

The following describes additional, normative
constraints on the outline listed above:

[Body]/wsen:EnumerateResponse/wsen:Expires

The expiration time assigned by the data
source. The expiration time MAY be either an absolute time or a
duration but SHOULD be of the same type as the requested expiration
(if any).

If this element does not appear, then the
enumeration will not expire. That is, the enumeration has an
indefinite lifetime. It will terminate when the end of the
enumeration is reached, or if the consumer sends a Release request,
or by the data source at any time for reasons such as connection
termination, resource constraints, or system
shut-down.

[body]/wsen:EnumerateResponse/wsen:EnumerationContext

The required EnumerationContext element
contains the XML representation of the new enumeration context. The
consumer is required to pass this XML data in Pull requests for
this enumeration context, until and unless a PullResponse message
updates the enumeration context.

Lines (05-07) in indicate this message
is an Enumerate request and that the data source is expected to
respond with an Enumerate response message. The wsen:Expires
element on line (20) indicates that the consumer would like an
enumeration context that will be good for at least 10 minutes; that
is, it expects to complete its enumeration within a 10 minute
period. No wsen:Filter element is present, so the resultant
enumeration context is expected to return all available
elements.

Lines (05-07) in indicate this message
is an Enumerate response message. Line (17) indicates that the data
source has actually created an enumeration context with a lifetime
of 15 minutes. Lines (18-20) are the XML representation of the
enumeration context that supports the Pull operation defined
below.

Pull

The Pull operation is initiated by sending a
Pull request message to the data source. The Pull request message
MUST be of the following form:

The following describes additional, normative
constraints on the outline listed above:

[Body]/wsen:Pull/wsen:EnumerationContext

This required element contains the XML data
that represents the current enumeration context.

If the enumeration context is not valid,
either because it has been replaced in the response to another Pull
request, or because it has completed (EndOfSequence has been
returned in a Pull response), or because it has been Released, or
because it has expired, or because the data source has had to
invalidate the context, then the data source SHOULD fail the
request, and MAY generate the following fault:

SOAP 1.1:

faultcode = s11:Server

faultstring = text explaining why the enumeration context is
no longer valid, or "Invalid enumeration context"
if no additional information is available

SOAP 1.2:

s:Code/s:Value = s:Receiver

s:Code/s:Subcode/s:Value = wsen:InvalidEnumerationContext

s:Reason/s:Text = text explaining why the enumeration context
is no longer valid, or "Invalid enumeration context"
if no additional information is available

Note that the data source may not be able to
determine that an enumeration context is not valid, especially if
all of the state associated with the enumeration is kept in the
enumeration context and refreshed on every
PullResponse.

[Body]/wsen:Pull/wsen:MaxTime

This optional element (of type xs:duration)
indicates the maximum amount of time the initiator is willing to
allow the data source to assemble the Pull response. When this
element is absent, the data source is not required to limit the
amount of time it takes to assemble the Pull
response.

This is useful with data sources that
accumulate elements over time and package them into a single Pull
response.

[Body]/wsen:Pull/wsen:MaxElements

This optional element (of type xs:long)
indicates the number of items (child elements of wsen:Items in the
Pull response) the consumer is willing to accept. When this element
is absent, its implied value is 1. Implementations MUST NOT return
more than this number of elements in the Pull response message.
Implementations MAY return fewer than this number based on either
the wsen:MaxTime timeout, the wsen:MaxCharacters size limit, or
implementation-specific constraints.

[Body]/wsen:Pull/wsen:MaxCharacters

This optional element (of type xs:long)
indicates the maximum size of the returned elements, in Unicode
characters, that the initiator is willing to accept. When this
element is absent, the data source is not required to limit the
number of characters in the Pull response. Implementations MUST NOT
return a Pull response message whose wsen:Items element is larger
than MaxCharacters. Implementations MAY return a smaller message
based on the wsen:MaxTime timeout, the wsen:MaxElements limit, or
implementation-specific constraints.

Even if a Pull request contains a
MaxCharacters element, the consumer MUST be prepared to receive a
Pull response that contains more data characters than specified, as
XML canonicalization or alternate XML serialization algorithms may
change the size of the representation.

It may happen that the next item the data
source would return to the consumer is larger than MaxCharacters.
In this case, the data source MAY skip the item, or MAY return an
abbreviated representation of the item that fits inside
MaxCharacters. If the data source skips the item, it MAY return it
as part of the response to a future Pull request with a larger
value of MaxCharacters, or it MAY omit it entirely from the
enumeration. If the oversize item is the last item to be returned
for this enumeration context and the data source skips it, it MUST
include the wsen:EndOfSequence item in the Pull response and
invalidate the enumeration context; that is, it may not return zero
items but not consider the enumeration completed. See the
discussion of wsen:EndOfSequence below.

Other components of the outline above are not
further constrained by this specification.

Upon receipt of a Pull request message, the
data source may wait as long as it deems necessary (but not longer
than the value of the wsen:MaxTime element, if present) to produce
a message for delivery to the consumer. The data source MUST
recognize the wsen:MaxTime element and return the following fault
if no elements are available prior to the request message's
deadline:

SOAP 1.1:

faultcode = s11:Server

faultstring = e.g., "timeout"

SOAP 1.2:

s:Code/s:Value = s:Receiver

s:Code/s:Subcode/s:Value = wsen:TimedOut

s:Reason/s:Text = e.g., "timeout"

Note, however, that this fault SHOULD NOT cause
the enumeration context to become invalid (of course, the data
source may invalidate the enumeration context for other reasons).
That is, the requestor should be able to issue additional Pull
requests using this enumeration context after receiving this fault.

Upon successful processing of a Pull request
message, a data source is expected to return a Pull response
message, which MUST adhere to the following form:

The following describes additional, normative
constraints on the outline listed above:

[Body]/wsen:PullResponse/wsen:EnumerationContext

The optional EnumerationContext element, if
present, contains a new XML representation of the current
enumeration context. The consumer is required to replace the prior
representation with the contents of this element.

[Body]/wsen:PullResponse/wsen:Items/xs:any

The optional Items element contains one or
more enumeration-specific elements, one for each element being
returned.

[Body]/wsen:PullResponse/wsen:EndOfSequence

This optional element indicates that no more
elements are available from this enumeration. Additionally, once
this element is returned in a Pull response message, subsequent
Pull requests using that enumeration context SHOULD generate a
wsen:InvalidEnumerationContext fault message; in any case, they
MUST NOT return a valid PullResponse.

Note that at least one of wsen:Items or
wsen:EndOfSequence MUST appear. It is possible for both to appear
if items are returned and the sequence is exhausted. Similarly,
wsen:EnumerationContext and wsen:EndOfSequence MUST NOT both
appear; neither may appear, or one without the other, but not both
in the same PullResponse.

Lines (05-07) in indicate this message
is a Pull request and that the data source is expected to respond
with a Pull response message. Line (21) indicates that the response
message should be generated no more than 30 seconds after receipt
of the Pull request message. Line (22) indicates that no more than
10 elements should be returned in the body of the Pull response
message.

Lines (05-07) in indicate this message
is a Pull response message. Lines (18-22) are the five elements
returned by this Pull request. The presence of a wsen:EndOfSequence
element (line (24)) indicates that no more elements are available
and that the enumeration context is now invalid.

The consumer SHOULD NOT issue additional Pull
request messages after a Pull response containing a
wsen:EndOfSequence element has been returned. Similarly, upon
receipt of a Pull response containing a wsen:EndOfSequence element,
the consumer SHOULD NOT issue a Release operation to signal that
the enumeration context is no longer needed.

If the consumer does issue a Pull or Release on
an invalid enumeration context, the result is undefined: the data
source MAY ignore the request or MAY return a
wsen:InvalidEnumerationContext fault, as described above, or MAY
take some other action.

Renew

Data sources MUST support requests to renew
enumerations.

To renew an enumeration, the consumer sends a
request of the following form to the data source:

Components of the outline listed above are
additionally constrained as for a request to create an enumeration
(see ) with the following
addition(s):

[Body]/wsen:Renew/wsen:EnumerationContext

This required element contains the XML data
that represents the current enumeration context.

If the enumeration context is not valid,
either because it has been replaced in the response to another Pull
request, or because it has completed (EndOfSequence has been
returned in a Pull response), or because it has been Released, or
because it has expired, or because the data source has had to
invalidate the context, then the data source SHOULD fail the
request, and MAY generate the following fault:

SOAP 1.1:

faultcode = s11:Server

faultstring = text explaining why the enumeration context is
no longer valid, or "Invalid enumeration context"
if no additional information is available

SOAP 1.2:

s:Code/s:Value = s:Receiver

s:Code/s:Subcode/s:Value = wsen:InvalidEnumerationContext

s:Reason/s:Text = text explaining why the enumeration
context is no longer valid, or "Invalid enumeration
context" if no additional information is available

Note that the data source may not be able to
determine that an enumeration context is not valid, especially if
all of the state associated with the enumeration is kept in the
enumeration context and refreshed on every
PullResponse.

Other components of the outline above are not
further constrained by this specification.

If the data source accepts a request to renew an
enumeration, it MUST reply with a response of the following
form:

Components of the outline listed above are
additionally constrained as for a request to renew an enumeration
(see ). Other components of the outline above are
not further constrained by this specification.

If the enumeration is valid and has not expired,
the data source MUST reply with a response of the following
form:

Lines (05-07) in indicate this message
is a Release request and that the data source is expected to
respond with a Release response message. Line (20) identifies the
enumeration context to be released.

If the data source terminates an enumeration
unexpectedly, the data source SHOULD send an EnumerationEnd SOAP
message to the endpoint reference indicated when the enumeration
was created (see ). The message MUST be of the
following form:

This value MUST be used if the data source
terminated the enumeration because the source is being shut down in
a controlled manner; that is, if the data source is being shut down
but has the opportunity to send an EnumerationEnd message before it
exits.

This value MUST be used if the data source
terminated the enumeration for some other reason before it
expired.

[Body]/wsen:EnumerationEnd/wsen:Reason

This optional element contains text, in the
language specified by the @xml:lang attribute, describing the
reason for the unexpected enumeration termination.

Other components of the outline above are not
further constrained by this specification.

Security Considerations

It is strongly recommended that the
communication between services be secured using the mechanisms
described in .

In order to properly secure messages, the body
(even if empty) and all relevant headers need to be included in the
signature. Specifically, the WS-Addressing header
blocks, WS-Security timestamp, and any header blocks resulting from
a <wsa:ReferenceParameters>
in references need to be signed along with the body in order to
"bind" them together and prevent certain types of attacks.

If a requestor is issuing multiple messages to a
Web service, such as when a consumer is enumerating a data source,
it is recommended that a security context be established using the
mechanisms described in . It is often
appropriate to establish a security context that is used both for
the initiation of enumeration (i.e., the Enumerate request or an
equivalent service-specific request) and the actual enumeration
itself (i.e., the Pull requests). It is further recommended that if
shared secrets are used, message-specific derived keys should be
used to protect the secret from crypto attacks.

The access control semantics of data sources is
out-of-scope of this specification and are specific to each data
source. Similarly, any protection mechanisms on data
source independent of their transfer (e.g. embedded signatures and
encryption) are also out-of-scope.

It is recommended that the security
considerations of WS-Security also be considered.

While a comprehensive set of attacks is not
feasible, the following list summarizes common classes of attacks
that apply to this protocol and identifies the mechanism(s) to
prevent/mitigate the attacks.

Replay
- Messages, or portions
of messages, can be replayed in an attempt to gain access or
disrupt services. Freshness checks such as timestamps,
digests, and sequences can be used to detect duplicate
messages.

Invalid tokens
- There are a number of token attacks including certificate
authorities, false signatures, and PKI attacks. Care should be taken
to ensure each token is valid (usage window, digest, signing
authority, revocation, ...), and that the appropriate delegation
policies are in compliance.

Man-in-the-middle
- The message exchanges in this
specification could be subject to man-in-the-middle attacks so care
should be taken to reduce possibilities here such as establishing a
secure channel and verifying that the security tokens user
represent identities authorized to speak for, or on behalf of, the
desired resource reference.

Message alteration
- Alteration is prevented by
including signatures of the message information using WS-Security.
Care should be taken to review message part references
to ensure they haven't been forged (e.g. ID duplication).

Key integrity
- Key integrity is maintained
by using the strongest algorithms possible (by comparing secured
policies - see and
) and by using derived
keys ().

Authentication
- Authentication is
established using the mechanisms described in WS-Security and
WS-Trust. Each message is authenticated using the
mechanisms described in WS-Security.

Accountability
- Accountability is
a function of the type of and string of the key and algorithms
being used. In many cases, a strong symmetric key
provides sufficient accountability. However, in some
environments, strong PKI signatures are required.

Availability
- All reliable
messaging services are subject to a variety of availability
attacks. Replay detection is a common attack and it is
recommended that this be addressed by the mechanisms described in
WS-Security. Other attacks, such as network-level
denial of service attacks are harder to avoid and are outside the
scope of this specification. That said, care should be
taken to ensure that minimal state is saved prior to any
authenticating sequences.