ETags are “opaque validator[s] for differentiating between
multiple representations of the same resource”. They are used in a
variety of ways in HTTP to determine the outcome of conditional
requests as described in RFC 7232. Understanding the full breadth
of ETags requires a very complete understanding of HTTP and the
nuances of resources and their representations. This document does
not attempt to address all applications of ETags at once, instead it
addresses specific use cases that have arisen in response to other
guidelines. It will evolve over time.

HTTP is fundamentally a system for sending representations of
resources back and forth across a network connection. A common
interaction is to GET/some/resource, modify the representation,
and then PUT/some/resource to update the resource on the server.
This is an extremely useful and superfically simple pattern that drives
many APIs in OpenStack and beyond.

That apparently simplicity is misleading: If there are two or more
clients performing operations on /some/resource in the same time
frame they can experience the lost update problem:

Client A and client B both GET/some/resource and make changes to
their local representation.

Client B does a PUT/some/resource at time 1.

Client A does a PUT/some/resource at time 2.

Client B’s changes have been lost. Neither client is made aware of this.
This is a problem.

HTTP/1.1 and beyond has a solution for this problem called ETags.
These provide a validator for different representations of a
resource that make it straightforward to determine if the
representation provided by a request or a response is the same as
one already in hand. This is very useful when validating cached GET
requests (the ETag answers the question “is what I have in my cache
the same as what the server would give me?”) but is also useful for
avoiding the lost update problem.

If the scenario described above is modified to use ETags it would
work like this:

Client A and client B both GET/some/resource, including a
response header named ETag that is the same for both clients
(let’s make the ETag ‘red57’). Details on ETag generation can be
found below.

They both make changes to their local representation.

Client B does a PUT/some/resource and includes a header
named If-Match with a value of red57. The request is
successful because the ETag sent in the request is the same as the
ETag generated by the server of its current state of the resource.

Client A does a PUT/some/resource and includes the If-Match
header with value red57. This request fails (with a 412
response code) because red57 no longer matches the ETag
generated by the server: Its current state has been updated by the
request from client B.

Client B’s changes have not been lost and client A has not
inadvertently changed something that is not in the form they
expected. Client A is made aware of this by the response code.
At this stage, client A can choose to GET the resource again and compare
their local representation with that just retrieved and choose a course
of action.

If a service accepts PUT requests and needs to avoid lost updates it
can do so by:

Sending responses to GET requests with an ETag header (see
below for some discussion on ETag-like attributes in
representations).

Requiring clients to send an If-Match header with a valid ETag when
processing PUT requests.

Processing the If-Match header on the server side to compare the
ETag provided in the request with the generated ETag of the
currently stored representation. If there is a match, carry on
with the request action, if not, respond with a 412 status code.

Note

An ETag value is a double-quoted string: "theetag".

Note

The If-Match header may contain multiple ETags (separated
by commas). If it does, at least one must match for the
request to proceed.

Note

What section of a codebase takes the responsibility of
managing the ETag and If-Match headers is greatly dependent on
the architecture of the service. In general the handler or
controller for each resource should be the locus of
responsibility. It may be there are decorators or libraries
that can be shared but such things are beyond the scope of
this document. Early implementors are encouraged to write code
that is transparent and easy to inspect, allowing easier
future extraction.

Note

ETags can be either strong or weak. see RFC 7232 for
discussion on how weak ETags may be used. They are not
addressed in this document as their import is primarily
related to cache handling. Strong ETags signify
byte-for-byte equivalence between representations of the
same resource. Weak ETags indicate only semantic equivalence.

Each of the steps listed above require functionality to generate ETags
for representations. Whenever the representation is different the ETag
should be different. RFC 7232 has advice on how to
generate good ETags. In practice they should be:

Different for different forms of the same resource. For example, the
XML and JSON representations of the same version of a resource
should have different ETags.

Different from version to version.

Not based on something that will change when the system restarts.
For example not be based on inodes or database keys that are ints
or other non-universal identifiers.

Not be based on hashes of strings that do not have reliable
ordering. For example it can be tempting to make md5 or sha hashes
of the JSON string that represents a resource. If the ordering in
that JSON is not guaranteed, the ETag is not useful.

Ideally they should be fast to calculate or if not fast then easy
to store (when the representation is written). A hash of a last
udpated timestamp and the content-type can work, but only if updates
are less frequent than clock updates.

Note

Many details of how ETags can be useful are left out of this
document. It is worth reading RFC 7232 in its entirety to
understand their purpose, how they work, edge cases and
how they interact with other modes of conditional request
handling.

For simple resources that represent a single unified entity the
above handling works well. For more complex resources the situation
becomes more complicated. Some scenarios worth considering:

When there is a resource which represents a collection of
resources (e.g. GET/resources versus GET/resources/some-id) the strict process for updating one of the
resources in that collection when using ETags would be:

GET/resources to get the list of resources.

Do some client side processing to choose a singe resource’s id.

GET/resources/that-id to get the resource and its ETag
header.

Modify the local representation.

PUT/resources/that-id with an If-Match header
containing the ETag.

This may be considered cumbersome. One way to optimize this is to
include an attribute whose value is the ETag in the individual
representations of the singular resources in the collection
resource. Then the second GET above can be skipped as the ETag is
already available.

When a resource has sub resources (e.g. an /image/id resource
contains a metadata attribute whose content is also available at
/image/id/metadata) it can be desirable to retrieve the image
resource and then PUT to the metadata resource. Strictly speaking
this would require a GET of the metadata resource to determine the
ETag.

If this is a problem, an optimization to work around this is to
allow the ETag of the image resource to be an acceptable ETag of
the metadata resource when provided in an If-Match header.
If this is done, then it is important that the reverse not be
true: The ETag sent with the metadata resource should not be valid
in an If-Match header sent to the image resource.

Note

In both of the above scenarios the semantics of ETags are being
violated. An ETag is not a magic key to unlock a resource and
make it writable. It is a value used to determine if two
representations of the same resource are in fact the same. In
the situations above they are comparing different resources.
Services should only do so if they must. Either because the
performance benefit is huge (in which case consider fixing the
performance of the API) or the user experience improvement is
significant. The latter is far more important and legitimate
than the former