Abstract

Status of this Document

Editors DRAFT

This document has been developed for discussion by the W3C Technical Architecture Group. This
finding addresses the TAG issue metadataInURI-31.
The content of this document is intended for discussion and does NOT necessarily represent a consensus position of the TAG.

1 Introduction

Web-based software uses URIs to identify resources.
The authority who assigns a URI is responsible for assuring that it is associated with the intended resource,
and that operations targeted to the URI manipulate or return the appropriate data.
Many URI schemes offer a flexible structure that can also be used to carry additional information,
called metadata, about the resource.
Such metadata might include
the title of a document,
the creation date of the resource, the MIME media type that is likely
to be returned by an HTTP GET, a digital signature usable to
verify the integrity or authorship of the resource content, or hints
about URI assignment policies that would allow one to guess the URIs
for related resources.

This finding addresses several questions regarding such metadata in URIs:

What information about a resource can or should be embedded
in its URI?

What metadata can be reliably determined
from a URI, and in what circumstances is it appropriate to rely on the correctness of such information?

In what circumstances is it appropriate to use information from a URI as a hint as to the nature of a resource or its representations?

The first question is primarily of concern to URI assignment authorities, who must choose a suitable URI for each resource they control.
The other questions are focused on people and software making use of
URIs, whether at the resource authority or elsewhere.
Of course, the questions are related insofar as one reason for an authority to encode metadata is for the benefit of resource users.

The TAG has earlier published a finding Authoritative Metadata[AUTHMETA], which
explains how to determine correct metadata in cases where conflicting information has been provided.
This finding is concerned with just one possible means of
determining resource metadata, i.e. from the URI itself.
The TAG publication [AWWW] discusses related issues under the heading of URI Opacity; this finding provides additional detail and guidance on the encoding of metadata into URIs, and on when it is or isn't appropriate to attempt to infer metadata from a URI.

2 Encoding and using metadata in URIs

This section uses simple examples to illustrate some
issues that arise when encoding metadata in URIs, or
when relying on information gleaned from such URIs.
Good Practice Notes are provided to explain how to use the
Web effectively, and Constraints are given
where necessary for using
the Web correctly.
As these examples show, encoding or not encoding metadata in a URI
or deciding whether to rely on such metadata
is often a tradeoff, involving some benefits and some costs.
In such cases, choices should be made that best meet the needs of particular
resource providers and users.

2.1 Reliability of URI metadata

Consider Martin, who is using a Web-based bug tracking system to
investigate some software problems. He sees a bug report which says:

"See http://example.org/bugdata/brokenfile.xml for an example of XML that is not well-formed."

The bug tracking system is built to show examples just as they are entered
into the system, so for http://example.org/bugdata/brokenfile.xml it returns
a stream of (poorly formed) XML with Content-Type text/plain.
That Content-Type should cause a properly configured browser
to show Martin the erroneous text just as it was recorded:

Unfortunately, Martin uses a browser that incorrectly attempts to infer the
format of the returned data from the URI suffix.
Keying on the ".xml" in the URI, it launches
an XML renderer for what should have been plain text.
When Martin attempts to view the faulty file,
he sees instead a browser error saying that the
erroneous XML could not be displayed.

Constraint

Constraint:
Web software MUST NOT depend on the correctness of metadata inferred
from a URI, except when the encoding of such metadata is documented
by applicable standards and specifications.

Such standards and specifications include pertinent Web and Internet
RFCs and Recommendations such as [URI],
as well as documentation provided by the URI assignment authority.

Martin's browser is in error because its inference that the URI suffix provides file type metadata is not provided for by normative Web specifications or, we may assume, in documentation from the assignment authority.
A correctly written browser would have shown the faulty XML
as text, or might conceivably have shown a
warning about the apparent mismatch between the type inferred from
the URI and
the returned Content-Type.
(Martin's browser is also ignoring TAG finding
"Authoritative Metadata" [AUTHMETA], which mandates
that the Content-Type HTTP header takes precedence even if
type information had somehow been reliably encoded in the URI.)

Note that the constraint refers to conclusions drawn by software,
which must
be trustworthy, as opposed to guesses made by people.
As discussed in 2.2 Guessing information from a URI, guessing is
something that people using the Web do quite
often and for good reason.
Software tends to be long lived and widely distributed.
Thus software dependencies on undocumented URI metadata result not only
in buggy systems, but in inappropriate expectations that
authorities will constrain their URI assignment policies
and representation types to match
dependencies in the clients. For both of these
reasons, the constraint above requires that software must not have
such dependencies.

There is certain metadata that Martin or his browser
can reliably determine from the URI.
For example, the URI conveys that the http scheme has been used,
and that attempts to access the resource should be directed to the IP address
returned from the DNS resolution of the string "example.org".
These conclusions are supported by normative specifications such as [URI] and [HTTP].

2.2 Guessing information from a URI

Bob is walking down a street, and he sees an advertisement
on the side of a bus:

"For the best Chicago Weather information on the Web, visit http://example.org/weather/Chicago."

Bob goes home and types the URI into his browser, which does indeed
display for him a Chicago weather forecast.
Bob then realizes that he'll be visiting Boston, and he guesses that a Boston weather page might be available at a similar URI:
http://example.org/weather/Boston.
He types that into his browser and reads the response that comes back.

Bob is using the original URI for more than its intended purpose,
which is to identify the Chicago weather page.
Instead, he's inferring from it information about the structure of a Web site
that, he guesses, might use a uniform naming convention for the
weather in lots of cities.
So, when Bob tries the Boston URI, he has to be prepared
for the possibility that his
guess will prove wrong:
Web architecture does not guarantee that the retrieved page, if there is one,
has the weather for Boston, or indeed that it contains
any weather report at all.
Even if it does,
there is no assurance that it is current weather, that it
is intended for reliable use by consumers, etc.
Bob has seen an advertisement listing just the Chicago URI, and that is the only one that the URI authority has
warranted will be a useful weather report.

Still, the ability to explore the Web informally and experimentally is very
valuable, and Web users act on such guesses about URIs all the time.
Many authorities
facilitate
such flexible use of the Web by assigning URIs in an orderly and predictable
manner.
Nonetheless, in the example above,
Bob is responsible for determining whether
the information returned is indeed what he needs.

2.3 HTML Forms, and Documenting Metadata Assignment Policies

Bob would not have had to guess the Boston weather URI if the
authority had documented its URI assignment policy.
Assignment authorities have no obligation to provide such documentation,
but it can be a useful way of advertising in bulk the URIs for
a collection of related resources. For example, an advertisement might
read:

"For the best weather information for your city, visit http://example.org/weather/your-city-name-here."

Reading that advertisement, Bob can reasonably assume that
weather reports are available by substituting specific
city names into the URI pattern http://example.org/weather/your-city-name-here.
Moreover, the advertisement claims that the weather information
obtainable at those URIs is "the best", so Bob can
assume that the weather reports are trustworthy and current.

HTML forms [HTMLForms] and now XForms [XFORMS]
each provide a means by which an authority can
assert its support
for a class of parameterized URIs,
while simultaneously
programming Web clients to prompt for the necessary parameters.
For example, a Web site http://example.org/weatherfinder
might offer
a city lookup page containing the following HTML form fragment:

<FORM ACTION="http://example.org/cityweather" METHOD="GET">
For what city would you like a weather report:
<INPUT TYPE="TEXT" NAME="city"/>?
<INPUT TYPE="SUBMIT" VALUE="Get the weather"/>
</FORM>

A browser receiving this form, or Bob if he
views the source of the form, is assured that
the assigning authority is supporting an entire class of URIs of the form:

http://example.org/cityweather?city=CityName

The same HTML Form is also a computer program, executable by the
browser, that prompts for and retrieves representations for all such URIs,
and the English text in the form assures Bob that these are indeed
for weather reports.
Bob is not guessing the encoding of the URI
or the nature of the resources referenced —
he is acting on authoritative information provided by
the assignor of the URIs.
He can assume not just that he will get weather reports for certain cities,
but that no URIs in the class correspond to anything other than
weather reports (though some may correspond to no resource at all).
Bob could, with this assurance, write his own software to construct
and use such URIs to retrieve weather reports.
Of course, the typical Web user would neither directly inspect
the URIs nor write software to build them,
but would instead type in city names and push the handy "Get the weather"
button on his or her browser screen.

Note that the example carefully specifies that the
HTML form is sourced from the same authority as the
individual weather URIs that the form queries.
In fact, it is also common for
the ACTION attributes in HTML forms to refer to URIs
from other authorities.
In such cases, it is the provider of the form rather than the
assigning authority for the queried URIs who is responsible for the claims
made in the form.
In particular, users (and software)
should check the origin of HTML forms before depending on the URI assignment
patterns that they appear to imply.
Of course, you can always use such a form to perform a query and see
what comes back;
what you can't do is blame the assignment authority if the generated
URIs either don't
resolve (status code 404)
or return representations that don't
match the expectations established when reading the
form (you got a football score instead of a weather report).

2.4 Authority use of URI metadata

In the examples in 2.3 HTML Forms, and Documenting Metadata Assignment Policies above, resource metadata
(I.e. the city associated with each resource) was encoded into URIs primarily for the benefit of users such as Bob,
or to facilitate use of the HTML Forms or XForms acting on those users' behalf.

Often, metadata is encoded into a URI not primarily for the benefit of users,
but to facilitate management of the resources themselves. For example,
assume that the administrators at example.org have established
a policy of assigning URIs based on the media types
of representations:
all GIF images are named with URIs ending in ".gif", and all JPEG images
are named with URIs ending in ".jpeg", and so on.
Although 2.1 Reliability of URI metadata warned that users of a resource cannot
rely on undocumented naming conventions to determine media types and
other information about a resource, the owner of a resource
controls such naming and can depend on it.
Example.org may therefore rely on their
policy in an Apache Web Server .htaccess file, which causes
the correct media type to be served automatically for each resource:

Even if it does not document this policy publicly, example.org's
own Web servers
can safely depend on it.

Good Practice

Good Practice:
URI assignment authorities and the Web servers deployed for them
may benefit from
an orderly mapping from resource metadata into URIs.

In addition to filename-based conventions, authorities may
choose to base URIs on database keys, customer identifiers, or other
information that makes it easy to associate a URI with information
pertinent to the corresponding resource.
Such encodings are both useful and common on the Web, but there can also
be drawbacks to including such information in URIs.
Some of those problems are discussed in the three sections immediately below.

2.5 URIs that are convenient for people to use

URIs optimized for use by the assignment authority may sometimes be
inconvenient for resource users.
Consider Mary who is walking down the street,
and who sees the same weather advertisement as Bob:

"For the best Chicago Weather information on the Web, visit http://example.org/weather/Chicago."

Like Bob,
Mary is pleased to learn about a valuable Web site, and she finds
that the URI itself is quite easy both to remember and to
type into her browser.
This is because,
in addition to the required scheme and authority components, the URI
is based on the word weather and the city name Chicago, both of which fit her expectations for this resource.

The next day, Mary sees another advertisement reading:

"For the best Atlanta Weather information on the Web, visit http://example.org/123Hx67v4gZ5234Bq5rZ."

Mary is annoyed, because the URI is both
difficult to remember and hard to transcribe accurately.
She guesses that the authority has assigned this URI for its own
convenience (see 2.4 Authority use of URI metadata) rather than for hers.
Although Web architecture does not require that URIs be easy
to understand or suggestive of the resource named, it's handy if those
intended for direct use by people are.

Good Practice

Good Practice:
URIs intended for direct use by people should be
easy to understand, and should be suggestive of the resource
actually named.

Note that the second
URI might be based on a database key
that facilitates efficient access to the weather data at the server
(see 2.4 Authority use of URI metadata);
such a URI might have been a good choice if it were intended only for use in
HTML hyperlinks,
rather than in an advertisement on the side of a bus.

2.6 Changing metadata

URIs should generally not encode metadata that will change,
regardless of whether the encoding policy is established
to benefit URI assignment authorities, resource users, or both.
Consider a Web site that organizes document URIs according
to the documents' lead author or editor.
Thus, the documents:

are named for their editor, Bob Smith.
Bob retires, and Mary Jones takes over as editor for document1.
If the URI is changed to encode her name,
then existing links break, but
if the URI is not changed, the naming policy is violated.
By encoding into the URI metadata that will change, the
authority has put itself in a difficult position.

Good Practice

Good Practice:
Resource metadata that will change SHOULD NOT be encoded in a URI.

Indeed, RDF statements about the resource, headers returned with representations (e.g. Content-Type) or metadata embedded in the representations themselves (e.g. HTML <META> tags) are all better alternatives
for conveying such volatile metadata about the resource.

2.7 Hiding metadata for security reasons

A bank establishes a URI assignment policy in which account numbers
are encoded directly in the URI. For example, the URI http://example.org/customeraccounts/456123 accesses information for account number 456123.
A malicious worker at an Internet Service Provider notices these URIs
in his traffic logs, and determines the bank account numbers for
his Internet customers.
Furthermore, if access controls are not properly in place,
he might be able to guess the URIs for other accounts, and to attempt
to access them.

Good Practice

Good Practice:
URI assignment authorities SHOULD NOT put
into URIs metadata that is to be kept confidential.

2.8 Confusing or malicious metadata

Although a URI suffix such as .jpeg or .exe plays no role in establishing the media type of a Web resource,
such suffixes are often significant in operating system filenames.
This inconsistency can be confusing to users, and may in some cases be exploited by malicious Web sites to cause harm.
Consider Ed, who browses to an HTML page that includes an image of his favorite movie star.
Underneath the picture is a suggestion that Ed "Right click on the picture and select 'Save as' to save a copy of this picture on your local disk".
The HTML sent to the browser is:

<img src="./moviestar.exe" />
<p>
Right click on the picture and select 'Save as' to save
a copy of this picture on your local disk.
</p>

Unfortunately, the Web site is attempting to trick Ed's browser into saving
the retrieved data not as an image file, but as an executable.
Specifically, the site is gambling on the possibility that his browser
will preserve the .exe extension when saving the file, and that
such an extension will cause his operating system to
treat the file as executable.

When saving information from the Web,
browsers must preserve to the extent practical the authoritative typing information provided with the representation.
As discussed in 2.1 Reliability of URI metadata, the Content-Type is the authoritative source of type information in this example. If the local operating system considers filename extensions to be significant, then either .jpeg or .jpg is more likely
to be the appropriate choice for a resource of media type image/jpeg, regardless of what suffix
may appear in the URI.

Good Practice

Good Practice:
When saving to filesystems that use extensions to represent media types,
user agents MUST choose an extension that is constistent with the media
type of the representation.

Indeed, many modern browsers suggest a name such as moviestar.exe.jpeg when saving the example file above.
Nonetheless, it is inappropriate for Web sites to intentionally mislead
users.
Although naming an image/jpeg file with a URI ending in
.exe is not prohibited by Web architecture, doing so with
the intention to deceive users or to compromise their systems is
of course not acceptable.

Note that the example above is contrived in at least one respect:
to achieve its malicious goals,
the Web site must serve a file that displays as an image in the
browser, but that also runs as an executable after being saved
to the local filesystem.
Whether this is possible in practice is likely to depend on the exact
image format and operating system involved.
A slightly more complex approach to achieving a similar
deception
involves sending Ed an image that serves as a link to a separate executable.
For example:

<a href="./malicious.exe">
<img src="./moviestar.jpg"/>
</a>
<p>
Click on the picture to see a larger copy of the picture.
</p>

The executable served for malicious.exe may in fact render a larger image
of the movie star, but it could also be programmed to damage Ed's computer.
For the reasons described above, the correct way for Ed's browser
to determine the type of the linked representation, which in this case
may indeed be executable, is from the media-type.
Of course, a well-written agent will warn users before executing
any code retrieved from the Web, regardless of whether
the determination of its type was made in the appropriate
manner or
inappropriately from the URI suffix.

3 Conclusions

The principle conclusions of this finding are:

It is legitimate for assignment authorities to encode static identifying properties of a resource, e.g. author, version, or creation date, within the URIs they assign. This may contribute to the unique assignment of URIs. It may also contribute to the use of efficient mechanisms for dereferencing resources within origin servers e.g. use of database keys within URIs.

Assignment authorities may publish specifications detailing the structure and semantics of the URIs they assign. Other users of those URIs may use such specifications to infer information about resources identified by URI assigned by that authority.

The ability to explore and experiment is important to Web users. Users therefore benefit from the ability to infer either the nature of the named resource, or the likely URI of other resources, from inspection of a URI.
Such inferences are reliable only when supported by normative specifications
or by documentation from the assignment authorities.
In other cases, users should be aware that their inferences
may be incorrect and the effect could be malicious.

People and software using URIs assigned outside of their own authority should make as few inferences as possible about a resource based on its URI. The more dependencies a piece of software has on particular constraints and inferences, the more fragile it becomes to change and the lower its generic utility.