Note:">
MAY">
MUST">
MUST NOT">
OPTIONAL">
RECOMMENDED">
REQUIRED">
SHALL">
SHALL NOT">
SHOULD">
SHOULD NOT">
]>
'Out-Of-Band' Content Coding for HTTPgreenbytes GmbHHafenweg 16MuensterNW48155Germanyjulian.reschke@greenbytes.dehttp://greenbytes.de/tech/webdav/EricssonTorshamnsgatan 2116483StochholmSwedensalvatore.loreto@ericsson.comApplications and Real-Time
HTTPcontent codingouf-of-band
This document describes an Hypertext Transfer Protocol (HTTP) content
coding that can be used to describe the location of a secondary resource
that contains the payload.
Distribution of this document is unlimited. Although this is not a work
item of the HTTPbis Working Group, comments should be sent to the
Hypertext Transfer Protocol (HTTP) mailing list at ietf-http-wg@w3.org,
which may be joined by sending a message with subject
"subscribe" to ietf-http-wg-request@w3.org.
Discussions of the HTTPbis Working Group are archived at
.
XML versions, latest edits, and issue tracking for this document
are available from and
.
The changes in this draft are summarized in .
This document describes an Hypertext Transfer Protocol (HTTP) content
coding () that can be used
to describe the location of a secondary resource that contains the payload.
The primary use case for this content coding is to enable origin servers
to securely delegate the delivery of content to a secondary server that might
be "closer" to the client (with respect to network topology) and/or
able to cache content (), leveraging content encryption
().
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in .
This document reuses terminology used in the base HTTP specifications,
namely and
.
The 'Out-Of-Band' content coding is used to direct the recipient to retrieve the
actual message representation ()
from a secondary resource, such as a public cache:

Client performs a request

Received response specifies the 'out-of-band' content coding; the payload
of the response contains additional meta data, plus the location of the secondary
resource

A &REQUIRED; array of JSON objects.
Objects having a member named 'r'
describe a secondary resource, with the member's string value containing
a URI reference () of the secondary
resource (URI references that are relative references are resolved against
the URI of the primary resource).
An &OPTIONAL; member 'crypto-key'
carries an array of strings, each of which specifying keying material
for use in encryption encodings such as the 'aes128gcm' encoding
defined in . Values consist of the
name of the content coding, a "=", and the base64url encoded keying
material (see ).

The payload format uses an array so that the origin server can specify
multiple secondary resources. The ordering within the array reflects the
origin server's preference (if any), with the most preferred secondary
resource location being first. Clients receiving a response containing
multiple entries are free to choose which of these to use.
In some cases, the origin server might want to specify a "fallback URI"; identifying
a secondary resource served by the origin server itself, but otherwise
equivalent "regular" secondary resources. Any secondary resource hosted
by the origin server can be considered to be a "fallback"; origin servers
will usually list them last in the "sr" array so that they only will be
used by clients when there is no other choice.
New specifications can define new &OPTIONAL; member fields, thus clients &MUST;
ignore unknown fields. Furthermore, new specifications can define new object
formats for the 'sr' array; however, they &MUST-NOT; use a member named 'r'
unless the semantics are compatible with those defined above.
Extension specifications will have to update this specification.
Upon receipt of an 'out-of-band' encoded response, a client first needs to
obtain the secondary resource's presentation. This is done using
an HTTP GET request (independently of the original request method).
In order to prevent any leakage of information, the GET request for
the secondary resource &MUST; only contain information provided by
the origin server or the secondary server itself, namely HTTP authentication
credentials () and cookies ().
Furthermore, the request &MUST; include an "Origin" header field indicating
the origin of the original resource ().
The secondary server &MUST; verify that the specified origin is
authorized to retrieve the given payload (or otherwise return an
appropriate 4xx status code).
In addition to that, the secondary server's response &MUST; include a
"Content-Type" header field indicating an Internet media type
of "application/oob-stream". Clients &MUST; check for this media type
and abort out-of-band processing if no media type is specified, or if it
doesn't match this value.
After receipt of the secondary resource's payload, the client then
reconstructs the original message by:

Unwrapping the encapsulated HTTP message by removing any transfer and content codings.

Replacing/setting any response header fields from the primary
response except for framing-related information such as
Content-Length, Transfer-Encoding and Content-Encoding.

If the client is unable to retrieve the secondary resource's representation
(host can't be reached, non 2xx response status code, payload failing
integrity check, etc.), it can choose
an alternate secondary resource (if specified), try the fallback URI (if
given), or simply retry the
request to the origin server without including 'out-of-band' in the
Accept-Encoding request header field. In the latter case, it can be useful
to inform the origin server about what problems were encountered
when trying to access the secondary resource; see
for details.
Note that although this mechanism causes the inclusion of external
content, it will not affect the application-level security properties
of the reconstructed message, such as its web origin ().
The cacheability of the response for the secondary resource does not affect
the cacheability of the reconstructed response message, which is the same as
for the origin server's response.
Use of the 'out-of-band' coding is similar to HTTP redirects ()
in that it can lead to cycles. Unless with HTTP redirects, the client however
is in full control: it does not need to advertise support for the 'out-of-band'
coding in requests for secondary resources. Alternatively, it can protect itself
just like for HTTP redirects -- by limiting the number of indirections it supports.
Note that because the server's response depends on the request's Accept-Encoding
header field, the response usually will need to be declared to vary on that. See
and
for details.
Client request of primary resource at https://www.example.com/test:
GET /test HTTP/1.1
Host: www.example.com
Accept-Encoding: gzip, out-of-band
Response:
HTTP/1.1 200 OK
Date: Thu, 14 May 2015 18:52:00 GMT
Content-Type: text/plain
Cache-Control: max-age=10, public
Content-Encoding: out-of-band
Content-Length:
Vary: Accept-Encoding
{
"sr": [
{ "r" :
"http://example.net/bae27c36-fa6a-11e4-ae5d-00059a3c7a00"},
{ "r" :
"/c/bae27c36-fa6a-11e4-ae5d-00059a3c7a00"}
]
}
(note that the Content-Type header field describes the media type of the
secondary's resource representation, and the origin server supplied
a fallback URI)
Client request for secondary resource:
GET /bae27c36-fa6a-11e4-ae5d-00059a3c7a00 HTTP/1.1
Host: example.net
Origin: https://www.example.com
Response:
HTTP/1.1 200 OK
Date: Thu, 14 May 2015 18:52:10 GMT
Cache-Control: private
Content-Type: application/oob-stream
Content-Length: Hello, world.
Final message after recombining header fields:
HTTP/1.1 200 OK
Date: Thu, 14 May 2015 18:52:00 GMT
Content-Length:
Cache-Control: max-age=10, public
Content-Type: text/plain
Hello, world.
requires the client to include an "Origin"
header field in the request to a secondary server. The example below
shows how the server for the secondary resource would respond to a request
which contains an "Origin" header field identifying an unauthorized origin.
Continuing with the example from ,
and a secondary server that is configured to allow only access for requests
initiated by "https://www.example.org":
Client request for secondary resource:
GET /bae27c36-fa6a-11e4-ae5d-00059a3c7a00 HTTP/1.1
Host: example.net
Origin: https://www.example.com
Response:
HTTP/1.1 403 Forbidden
Date: Thu, 14 May 2015 18:52:10 GMT
Note that a request missing the "Origin" header field would be
treated the same way.
Any reason why to *mandate* a specific 4xx code?
Given the example HTTP message from ,
a primary resource could use the 'out-of-band' coding to specify just
the location of the secondary resource plus the keying material
needed to decrypt the payload:
Response:
HTTP/1.1 200 OK
Date: Thu, 14 May 2015 18:52:00 GMT
Content-Encoding: aes128gcm, out-of-band
Content-Type: text/plain
Content-Length:
Vary: Accept-Encoding
{
"sr": [
{ "r" :
"http://example.net/bae27c36-fa6a-11e4-ae5d-00059a3c7a00",
"crypto-key" :
[ "aes128gcm=yqdlZ-tYemfogSmv7Ws5PQ" ] }
]
}
(note that the Content-Type header field describes the media type of the
secondary's resource representation)
Response for secondary resource:
HTTP/1.1 200 OK
Date: Thu, 14 May 2015 18:52:10 GMT
Content-Type: application/oob-stream
Content-Length: 54
I1BsxtFttlv3u_Oo94xnmwAAEAAA-NAVub2qFgBEuQKRapoZu-IxkIva3MEB1PD-
ly8Thjg(payload body shown in base64 for presentation purposes)Final message undoing all content codings:
HTTP/1.1 200 OK
Date: Thu, 14 May 2015 18:52:00 GMT
Content-Length:
Content-Type: text/plain
I am the walrusNote: in this case, the ability to undo the 'aes128gcm' is needed
to process the response. If 'aes128gcm' wasn't listed as acceptable content coding
in the request, the origin server wouldn't be able to use the 'out-of-band'
mechanism.
Use of the 'out-of-band' encoding is a case of "proactive content negotiation",
as defined in .
This however does not rule out combining it with other content codings. As an example, the
possible iteractions with the 'gzip' content coding ()
are described below:
Case 1: Primary resource does not support 'gzip' encoding
In this case, the response for the primary resource will never include
'gzip' in the Content-Encoding header field. The secondary resource
however might support it, in which case the client could negotiate
compression by including "Accept-Encoding: gzip" in the request to the
secondary resource.
Case 2: Primary resource does support 'gzip' encoding
Here, the origin server would actually use two different secondary resources,
one of them being gzip-compressed. For instance — going back to the first
example in — it might reply with:
HTTP/1.1 200 OK
Date: Thu, 14 May 2015 18:52:00 GMT
Content-Type: text/plain
Cache-Control: max-age=10, public
Content-Encoding: gzip, out-of-band
Content-Length:
Vary: Accept-Encoding
{
"sr": [
{ "r" :
"http://example.net/bae27c36-fa6a-11e4-ae5d-00059a3c7a01"},
{ "r" :
"/c/bae27c36-fa6a-11e4-ae5d-00059a3c7a01"}
]
}
which would mean that the payload for the secondary resource already is
gzip-compressed.
Note: The origin server could also apply gzip
compression to the out-of-band payload, in which case the Content-Encoding
field value would become: "gzip, out-of-band, gzip".
The combination of content codings ( with
range requests () can lead to surprising results, as
applying the range request happens after applying content codings.
Thus, for a request for the bytes starting at position 100000 of a video:
GET /test.mp4 HTTP/1.1
Host: www.example.com
Range: bytes=100000-
Accept-Encoding: identity
...a successful response would use status code 206 (Partial Content) and
have a payload containing the octets starting at position 100000.
HTTP/1.1 206 Partial Content
Date: Thu, 08 September 2015 16:49:00 GMT
Content-Type: video/mp4
Content-Length: 134567
Content-Range: bytes 100000-234566/234567
(binary data)
However, if the request would have allowed the use of 'out-of-band' coding:
GET /test.mp4 HTTP/1.1
Host: www.example.com
Range: bytes=100000-
Accept-Encoding: out-of-band
...a server might return an empty payload (if the out-of-band
coded response body would be shorter than 100000 bytes, as would be usually the case).
Thus, in order to avoid unnecessary network traffic, servers &SHOULD-NOT;
apply range request processing to responses using ouf-of-band content coding
(or, in other words: ignore "Range" request header fields in this case).
New content codings can be deployed easily, as the client can use
the "Accept-Encoding" header field ()
to signal which content codings are supported.
This specification does not define means to verify that the payload
obtained from the secondary resource really is what the origin server
expects it to be. Content signatures can address this concern
(see and ).
The 'out-of-band' content coding could be used to circumvent the same-origin
policy () of user agents: an
attacking site which knows the URI of a secondary resource would use the
'out-of-band' coding to trick the user agent to read the contents of the secondary resource,
which then, due to the security properties of this coding, would be
handled as if it originated from the origin's resource.
This scenario is addressed by the client requirement to include
the "Origin" request header field and the server requirement to verify
that the request was initiated by an authorized origin. In addition,
the restriction of the secondary server response's media type
to "application/oob-stream" protects existing content on "regular"
servers not implementing this specification.
Note: similarities with the "Cross-Origin Resource Sharing"
protocol () are intentional.
Requiring the secondary resource's payload to be encrypted ()
is an additional mitigation.
In general, content codings can be used in both requests and responses. This particular
content coding has been designed for responses. When supported in requests, it
creates a new attack vector where the receiving server can be tricked into
including content that the client might not have access to otherwise
(such as HTTP resources behind a firewall).
The IANA "HTTP Content Coding Registry", located at ,
needs to be updated with the registration below:

Name:

out-of-band

Description:

Payload needs to be retrieved from a secondary resource

Reference:

of this document

IANA maintains the registry of Internet media types at
.
This document serves as the specification for the Internet media type
"application/oob-stream". The following is to be registered with
IANA.
The "application/oob-stream" media type represents a sequence of octets sent as part
of the "out-of-band" content coding protocol exchange. The sender does
not have any further information about the type of the enclosed data.
This type is different from "application/octet-stream" as it is known
not to be in use for pre-existing content.

Type name:

application

Subtype name:

oob-stream

Required parameters:

N/A

Optional parameters:

N/A

Encoding considerations:

always "binary"

Security considerations:

see

Interoperability considerations:

N/A

Published specification:

This specification (see ).

Applications that use this media type:

HTTP servers for secondary resources as defined by this specification.

Fragment identifier considerations:

N/A

Additional information:

Magic number(s):

N/A

Deprecated alias names for this type:

N/A

File extension(s):

N/A

Macintosh file type code(s):

N/A

Person and email address to contact for further information:

See Authors' Addresses section.

Intended usage:

COMMON

Restrictions on usage:

N/A

Author:

See Authors' Addresses section.

Change controller:

IESG

Key words for use in RFCs to Indicate Requirement LevelsUniform Resource Identifier (URI): Generic SyntaxWeb LinkingHTTP State Management MechanismThe JavaScript Object Notation (JSON) Data Interchange FormatHypertext Transfer Protocol (HTTP/1.1): Message Syntax and RoutingHypertext Transfer Protocol (HTTP/1.1): Semantics and ContentHypertext Transfer Protocol (HTTP/1.1): AuthenticationMedia Type Specifications and Registration ProceduresDefinition of the URL MIME External-Body Access-TypeA Mechanism for Content Indirection in Session Initiation Protocol (SIP) MessagesThe Base16, Base32, and Base64 Data EncodingsThe Transport Layer Security (TLS) Protocol Version 1.2The Web Origin ConceptHypertext Transfer Protocol (HTTP/1.1): Conditional RequestsHypertext Transfer Protocol (HTTP/1.1): Range RequestsEncrypted Content-Encoding for HTTPContent-Signature Header Field for HTTPMerkle Integrity Content EncodingResource MapsAn Architecture for Secure Content Delegation using HTTPCross-Origin Resource Sharing
Latest version available at
.
This is a rough proposal for an error reporting mechanism. Is it good enough? Is it needed at all?
Note that Alt-Svc doesn't have anything like this.
When the client fails to obtain the secondary resource, it can be useful
to inform the origin server about the condition. This can be accomplished
by adding a "Link" header field () to a subsequent request to the origin server,
detailing the URI of the secondary resource and the failure reason.
The following link extension relations are defined:
Used in case the server was not reachable.
Link relation:
http://purl.org/linkrel/not-reachable
Used in case the server responded, but the object could not be obtained.
Link relation:
http://purl.org/linkrel/resource-not-found
Used in case the payload could be obtained, but wasn't usable
(for instance, because integrity checks failed).
Link relation:
http://purl.org/linkrel/payload-unusable
Used in case of a TLS handshare failure ().
Link relation:
http://purl.org/linkrel/tls-handshake-failure
Client requests primary resource as in , but the
attempt to access the secondary resource fails.
Response:
HTTP/1.1 404 Not Found
Date: Thu, 08 September 2015 16:49:00 GMT
Content-Type: text/plain
Content-Length: Resource Not Found
Client retries with the origin server and includes Link
header field reporting the problem:
GET /test HTTP/1.1
Host: www.example.com
Accept-Encoding: gzip, out-of-band
Link: <http://example.net/bae27c36-fa6a-11e4-ae5d-00059a3c7a00>;
rel="http://purl.org/linkrel/resource-not-found"
A plausible alternative approach would be to implement this functionality one level
up, using a new redirect status code (). However,
this would have several drawbacks:

Servers will need to know whether a client understands the new status code;
thus some additional signal to opt into this protocol would always be needed.

Another alternative would be to implement the indirection on the level
of the media type using something similar to the type "message/external-body",
defined in and refined for use in the
Session Initiation Protocol (SIP) in . This approach
though would share most of the drawbacks of the status code approach mentioned
above.
One use-case for this protocol is to enable a system of "blind caches",
which would serve the secondary resources. These caches might only be populated
on demand, thus it could happen that whatever mechanism is used to populate
the cache hasn't finished when the client hits it (maybe due to race
conditions, or because the cache is behind a middlebox which doesn't allow
the origin server to push content to it).
In this particular case, it can be useful if the client was able to
"piggyback" the URI of the fallback for the primary resource, giving the secondary server
a means by which it could obtain the payload itself. This information could
be provided in yet another Link header field:
GET /bae27c36-fa6a-11e4-ae5d-00059a3c7a00 HTTP/1.1
Host: example.net
Link: <http://example.com/c/bae27c36-fa6a-11e4-ae5d-00059a3c7a00>;
rel="http://purl.org/linkrel/fallback-resource"
(continuing the example from )
When 'out-of-band' coding is used as part of a caching solution, the additional
round trips to the origin server can be a significant performance problem;
in particular, when many small resources need to be loaded (such as
scripts, images, or video fragments). In cases like these, it could be
useful for the origin server to provide a "resource map", allowing
to skip the round trips to the origin server for these mapped resources.
Plausible ways to transmit the resource map could be:

as extension in the 'out-of-band' coding JSON payload, or

as separate resource identified by a "Link" response header field.

See for further information.
It might be interesting to divide the original resource's payload into fragments,
each of which being mapped to a distinct secondary resource. This would
allow to not store the full payload of a resource in a single cache, thus

distribute load,

caching different parts of the resource with different characteristics (such as only distribute the first minutes of a long video), or

fetching specific parts of a resource (similar to byte range requests), or

hiding information from the secondary server.

Another benefit might be that it would allow the origin server to only serve the first
part of a resource itself (reducing time to play of a media resource), while
delegating the remainder to a cache (however, this might require further adjustments
of the 'out-of-band' payload format).
Right now this specification is orthogonal to /; that is, it could be
used for public content such as software downloads. However, the lack of mandatory encryption
affects the security considerations (which currently try to rule attack vectors
caused by ambient authority ().
We need to decide whether we need this level of independence.
This specification already defines hooks through which a client can report
failures when accessing secondary resources (see ).
However, it would be useful if there were also ways to report on statistics such as:

Success (Cache Hit) rates, and

Bandwidth to secondary servers.

This could be implemented using a new service endpoint and a (JSON?) payload
format.
Similarly, a reporting facility for use by the secondary servers
could be useful.
Clients by default might include request header fields such as "User-Agent"
(or some of the newly defined "Client Hints") into their requests to the
secondary server. If the secondary server does not perform any
content negotiation, none of these header fields is actually useful,
so suppressing them by default might be a good idea to reduce fingerprinting.
In this case, we could allow the origin server to opt into sending some of
them though.
Mention media type approach.
Explain that clients can always fall back not to use oob when the secondary
resource isn't available.
Add Vary response header field to examples and mention that it'll
usually be needed
().
Experimentally add problem reporting using piggy-backed Link header fields
().
Updated ENCRYPTENC reference.
Add MICE reference.
Remove the ability of the secondary resource to contain anything but the
payload ().
Changed JSON payload to be an object containing an array of URIs plus
additional members. Specify "fallback" as one of these additional members,
and update accordingly).
Discuss extensibility a bit.
Mention "Content Stealing" thread.
Mention padding.
Reduce information leakage by disallowing ambient authority information
being sent to the secondary resource. Require "Origin" to be included
in request to secondary resource, and require secondary server to check it.
Mention "Origin" + server check on secondary resource as defense to content stealing.
Update ENCRYPTENC reference, add SCD reference.
Mention fragmentation feature.
Discuss relation with range requests.
Remove redundant Cache-Control: private from one example response (the response payload is encrypted anyway).
Mention looping.
Remove 'metadata' payload element.
Align with changes in ENCRYPTENC spec.
Fix incorrect statement about what kind of cookies/credentials can be used in the request to the secondary resource.
Rename "URIs" to "sr" ("secondary resources") and treat the fallback URI like a regular secondary resource.
Mention reporting protocol ideas.
Changed the link relation name to the fallback resource from "primary" to "fallback".
Added link relation for reporting TLS handshake failures.
Added an example about the interaction with 'gzip' coding.
Update ENCRYPTENC, MICE, and SCD references.
Restrict the valid media types for the response of the secondary server to "application/oob-stream".
Changed JSON format to allow annotation (optional flags) and entirely new types of entries.
Moved error reporting into appendix (because it's optional and we're not sure about the utility of it). See .
Updated references for ENCRYPTENC, MICE, and SCD.
Mention that we could suppress certain request header fields in the request to the secondary server.
Updated reference for ENCRYPTENC. Added RMAP reference.
Use all-lowercase PURLs and remove "/net" in them.
Updated reference for ENCRYPTENC: instead of using Crypto-Key response
header field move the key material into the OOB payload.
Removed note about registration of purl URIs for link relations.
Updated reference for ENCRYPTENC (now RFC 8188).
Thanks to Christer Holmberg, Daniel Lindstrom, Erik Nygren, Goran Eriksson, John Mattsson, Kevin Smith, Magnus Westerlund, Mark Nottingham, Martin Thomson,
and Roland Zink for feedback on this document.