RFC 6295

RTP Payload Format for MIDI

4. The Recovery Journal System
The recovery journal is the default resiliency tool for unreliable
transport. In this section, we normatively define the roles that
senders and receivers play in the recovery journal system.
MIDI is a fragile code. A single lost command in a MIDI command
stream may produce an artifact in the rendered performance. We
normatively classify rendering artifacts into two categories:
o Transient artifacts. Transient artifacts produce immediate but
short-term glitches in the performance. For example, a lost
NoteOn (0x9) command produces a transient artifact: one note fails
to play, but the artifact does not extend beyond the end of that
note.
o Indefinite artifacts. Indefinite artifacts produce long-lasting
errors in the rendered performance. For example, a lost NoteOff
(0x8) command may produce an indefinite artifact: the note that
should have been ended by the lost NoteOff command may sustain
indefinitely. As a second example, the loss of a Control Change
(0xB) command for controller number 7 (Channel Volume) may produce
an indefinite artifact: after the loss, all notes on the channel
may play too softly or too loudly.
The purpose of the recovery journal system is to satisfy the recovery
journal mandate: the MIDI performance rendered from an RTP MIDI
stream sent over unreliable transport MUST NOT contain indefinite
artifacts.
The recovery journal system does not use packet retransmission to
satisfy this mandate. Instead, each packet includes a special
section called the recovery journal.

The recovery journal codes the history of the stream back to an
earlier packet called the checkpoint packet. The range of coverage
for the journal is called the checkpoint history. The recovery
journal codes the information necessary to recover from the loss of
an arbitrary number of packets in the checkpoint history. Appendix
A.1 normatively defines the checkpoint history.
When a receiver detects a packet loss, it compares its own knowledge
about the history of the stream with the history information coded in
the recovery journal of the packet that ends the loss event. By
noting the differences in these two versions of the past, a receiver
is able to transform all indefinite artifacts in the rendered
performance into transient artifacts by executing MIDI commands to
repair the stream.
We now state the normative role for senders in the recovery journal
system.
Senders prepare a recovery journal for every packet in the stream.
In doing so, senders choose the checkpoint packet identity for the
journal. Senders make this choice by applying a sending policy.
Appendix C.2.2 normatively defines three sending policies: "closed-
loop", "open-loop", and "anchor".
By default, senders MUST use the closed-loop sending policy. If the
session description overrides this default policy by using the
parameter j_update defined in Appendix C.2.2, senders MUST use the
specified policy.
After choosing the checkpoint packet identity for a packet, the
sender creates the recovery journal. By default, this journal MUST
conform to the normative semantics in Section 5 and Appendices A and
B in this memo. In Appendix C.2.3, we define parameters that modify
the normative semantics for recovery journals. If the session
description uses these parameters, the journal created by the sender
MUST conform to the modified semantics.
Next, we state the normative role for receivers in the recovery
journal system.
A receiver MUST detect each RTP sequence number break in a stream.
If the sequence number break is due to a packet loss event (as
defined in [RFC3550]), the receiver MUST repair all indefinite
artifacts in the rendered MIDI performance caused by the loss. If
the sequence number break is due to an out-of-order packet (as
defined in [RFC3550]), the receiver MUST NOT take actions that
introduce indefinite artifacts (ignoring the out-of-order packet is a
safe option).

Receivers take special precautions when entering or exiting a
session. A receiver MUST process the first received packet in a
stream as if it were a packet that ends a loss event. Upon exiting a
session, a receiver MUST ensure that the rendered MIDI performance
does not end with indefinite artifacts.
Receivers are under no obligation to perform indefinite artifact
repairs at the moment a packet arrives. A receiver that uses a
playout buffer may choose to wait until the moment of rendering
before processing the recovery journal, as the "lost" packet may be a
late packet that arrives in time to use.
Next, we state the normative role for the creator of the session
description in the recovery journal system. The sender, the
receivers, and other parties may take part in creating or approving
the session description, depending on the application.
A session description that specifies the default closed-loop sending
policy and the default recovery journal semantics satisfies the
recovery journal mandate. However, these default behaviors may not
be appropriate for all sessions. If the creators of a session
description use the parameters defined in Appendix C.2 to override
these defaults, the creators MUST ensure that the parameters define a
system that satisfies the recovery journal mandate.
Finally, we note that this memo does not specify sender or receiver
recovery journal algorithms. Implementations are free to use any
algorithm that conforms to the requirements in this section. The
non-normative [RFC4696] discusses sender and receiver algorithm
design.
5. Recovery Journal Format
This section introduces the structure of the recovery journal and
defines the bitfields of recovery journal headers. Appendices A and
B complete the bitfield definition of the recovery journal.
The recovery journal has a three-level structure:
o Top-level header.
o Channel and system journal headers. These headers encode recovery
information for a single voice channel (channel journal) or for
all system commands (system journal).
o Chapters. Chapters describe recovery information for a single
MIDI command type.

By default, the payload format does not use enhanced Chapter C
encoding. In this default case, the H bit MUST be set to 0 for all
packets in the stream.
If the stream has been configured so that controller numbers for one
or more MIDI channels use enhanced Chapter C encoding, the H bit MUST
be set to 1 in all packets in the stream. In Appendix C.2.3, we show
how to configure a stream to use enhanced Chapter C encoding.
The 16-bit Checkpoint Packet Seqnum header field codes the sequence
number of the checkpoint packet for this journal, in network byte
order (big-endian). The choice of the checkpoint packet sets the
depth of the checkpoint history for the journal (defined in Appendix
A.1).
Receivers may use the Checkpoint Packet Seqnum field of the packet
that ends a loss event to verify that the journal checkpoint history
covers the entire loss event. The checkpoint history covers the loss
event if the Checkpoint Packet Seqnum field is less than or equal to
one plus the highest RTP sequence number previously received on the
stream (modulo 2^16).
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S| CHAN |H| LENGTH |P|C|M|W|N|E|T|A| Chapters ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 9 -- Channel Journal FormatFigure 9 shows the structure of a channel journal: a 3-octet header
followed by a list of leaf elements called channel chapters. A
channel journal encodes information about MIDI commands on the MIDI
channel coded by the 4-bit CHAN header field. Note that CHAN uses
the same bit encoding as the channel nibble in MIDI Channel Messages
(the cccc field in Figure E.1 of Appendix E).
The 10-bit LENGTH field codes the length of the channel journal. The
semantics for LENGTH fields are uniform throughout the recovery
journal and are defined in Appendix A.1.
The third octet of the channel journal header is the Table of
Contents (TOC) of the channel journal. The TOC is a set of bits that
encode the presence of a chapter in the journal. Each chapter
contains information about a certain class of MIDI channel command:
o Chapter P: MIDI Program Change (0xC)
o Chapter C: MIDI Control Change (0xB)

o Chapter M: MIDI Parameter System (part of 0xB)
o Chapter W: MIDI Pitch Wheel (0xE)
o Chapter N: MIDI NoteOff (0x8), NoteOn (0x9)
o Chapter E: MIDI Note Command Extras (0x8, 0x9)
o Chapter T: MIDI Channel Aftertouch (0xD)
o Chapter A: MIDI Poly Aftertouch (0xA)
Chapters appear in a list following the header, in order of their
appearance in the TOC. Appendices A.2-A.9 describe the bitfield
format for each chapter and define the conditions under which a
chapter type MUST appear in the recovery journal. If any chapter
types are required for a channel, an associated channel journal MUST
appear in the recovery journal.
The H bit indicates if controller numbers on a MIDI channel have been
configured to use the enhanced Chapter C encoding (Appendix A.3.3).
By default, controller numbers on a MIDI channel do not use enhanced
Chapter C encoding. In this default case, the H bit MUST be set to 0
for all channel journal headers for the channel in the recovery
journal, for all packets in the stream.
However, if at least one controller number for a MIDI channel has
been configured to use the enhanced Chapter C encoding, the H bit for
its channel journal MUST be set to 1, for all packets in the stream.
In Appendix C.2.3, we show how to configure a controller number to
use enhanced Chapter C encoding.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S|D|V|Q|F|X| LENGTH | System chapters ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 10 -- System Journal FormatFigure 10 shows the structure of the system journal: a 2-octet header
followed by a list of system chapters. Each chapter codes
information about a specific class of MIDI system commands:
o Chapter D: Song Select (0xF3), Tune Request (0xF6), Reset (0xFF),
undefined system commands (0xF4, 0xF5, 0xF9, 0xFD)
o Chapter V: Active Sense (0xFE)
o Chapter Q: Sequencer State (0xF2, 0xF8, 0xF9, 0xFA, 0xFB, 0xFC)
o Chapter F: MIDI Time Code (MTC) Tape Position (0xF1, 0xF0 0x7F
0xcc 0x01 0x01)
o Chapter X: System Exclusive (all other 0xF0)

The 10-bit LENGTH field codes the size of the system journal and
conforms to semantics described in Appendix A.1.
The D, V, Q, F, and X header bits form a Table of Contents (TOC) for
the system journal. A TOC bit that is set to 1 codes the presence of
a chapter in the journal. Chapters appear in a list following the
header, in the order of their appearance in the TOC.
Appendix B describes the bitfield format for the system chapters and
defines the conditions under which a chapter type MUST appear in the
recovery journal. If any system chapter type is required to appear
in the recovery journal, the system journal MUST appear in the
recovery journal.
6. Session Description Protocol
RTP does not perform session management. Instead, RTP works together
with session management tools, such as the Session Initiation
Protocol (SIP, [RFC3261]) and the Real Time Streaming Protocol (RTSP,
[RFC2326]).
RTP payload formats define media type parameters for use in session
management (for example, this memo defines rtp-midi as the media type
for native RTP MIDI streams).
In most cases, session management tools use the media type parameters
via another standard, the Session Description Protocol (SDP,
[RFC4566]).
SDP is a textual format for specifying session descriptions. Session
descriptions specify the network transport and media encoding for RTP
sessions. Session management tools coordinate the exchange of
session descriptions between participants ("parties").
Some session management tools use SDP to negotiate details of media
transport (network addresses, ports, etc.). We refer to this use of
SDP as "negotiated usage". One example of negotiated usage is the
Offer/Answer protocol ([RFC3264] and Appendix C.7.2 in this memo) as
used by SIP.
Other session management tools use SDP to declare the media encoding
for the session but use other techniques to negotiate network
transport. We refer to this use of SDP as "declarative usage". One
example of declarative usage is RTSP ([RFC2326] and Appendix C.7.1 in
this memo).

Below, we show session description examples for native (Section 6.1)
and mpeg4-generic (Section 6.2) streams. In Section 6.3, we
introduce session configuration tools that may be used to customize
streams.
6.1. Session Descriptions for Native Streams
The session description below defines a unicast UDP RTP session (via
a media ("m=") line) whose sole payload type (96) is mapped to a
minimal native RTP MIDI stream.
v=0
o=lazzaro 2520644554 2838152170 IN IP4 first.example.net
s=Example
t=0 0
m=audio 5004 RTP/AVP 96
c=IN IP4 192.0.2.94
a=rtpmap:96 rtp-midi/44100
The rtpmap attribute line uses the rtp-midi media type to specify an
RTP MIDI native stream. The clock rate specified on the rtpmap line
(in the example above, 44100 Hz) sets the scaling for the RTP
timestamp header field (see Section 2.1 and also [RFC3550]).
Note that this document does not specify a default clock rate value
for RTP MIDI. When RTP MIDI is used with SDP, parties MUST use the
rtpmap line to communicate the clock rate. Guidance for selecting
the RTP MIDI clock rate value appears in Section 2.1.
We consider the RTP MIDI stream shown above to be "minimal" because
the session description does not customize the stream with
parameters. Without such customization, a native RTP MIDI stream has
these characteristics:
1. If the stream uses unreliable transport (unicast UDP, multicast
UDP, etc.), the recovery journal system is in use, and the RTP
payload contains both the MIDI command section and the journal
section. If the stream uses reliable transport (such as TCP),
the stream does not use journalling, and the payload contains
only the MIDI command section (Section 2.2).
2. If the stream uses the recovery journal system, the recovery
journal system uses the default sending policy and the default
journal semantics (Section 4).
3. In the MIDI command section of the payload, command timestamps
use the default comex semantics (Section 3).

4. The recommended temporal duration ("media time") of an RTP packet
ranges from 0 to 200 ms, and the RTP timestamp difference between
sequential packets in the stream may be arbitrarily large
(Section 2.1).
5. If more than one minimal rtp-midi stream appears in a session,
the MIDI name spaces for these streams are independent: channel 1
in the first stream does not reference the same MIDI channel as
channel 1 in the second stream (see Appendix C.5 for a discussion
of the independence of minimal rtp-midi streams).
6. The rendering method for the stream is not specified. What the
receiver "does" with a minimal native MIDI stream is out of the
scope of this memo. For example, in content creation
environments, a user may manually configure client software to
render the stream with a specific software package.
As is standard in RTP, RTP sessions managed by SIP are sendrecv by
default (parties send and receive MIDI), and RTP sessions managed by
RTSP are recvonly by default (server sends and client receives).
In sendrecv RTP MIDI sessions for the session description shown
above, the 16 voice channel + systems MIDI name space is unique for
each sender. Thus, in a two-party session, the voice channel 0 sent
by one party is distinct from the voice channel 0 sent by the other
party.
This behavior corresponds to what occurs when two MIDI 1.0 DIN
devices are cross-connected with two MIDI cables (one cable routing
MIDI Out from the first device into MIDI In of the second device and
a second cable routing MIDI In from the first device into MIDI Out of
the second device). We define this "association" formally in Section
2.1.
MIDI 1.0 DIN networks may be configured in a "party-line" multicast
topology. For these networks, the MIDI protocol itself provides
tools for addressing specific devices in transactions on a multicast
network and for device discovery. Thus, apart from providing a 1-to-
many forward path and a many-to-1 reverse path, IETF protocols do not
need to provide any special support for MIDI multicast networking.
6.2. Session Descriptions for mpeg4-generic Streams
An mpeg4-generic [RFC3640] RTP MIDI stream uses an MPEG 4 Audio
Object Type to render MIDI into audio. Three Audio Object Types
accept MIDI input:

o General MIDI (Audio Object Type ID 15), based on the General MIDI
rendering standard [MIDI].
o Wavetable Synthesis (Audio Object Type ID 14), based on the
Downloadable Sounds Level 2 (DLS 2) rendering standard [DLS2].
o Main Synthetic (Audio Object Type ID 13), based on Structured
Audio and the programming language SAOL [MPEGSA]. The name of the
language (SAOL) is an acronym that expands to Structured Audio
Orechestra Language.
The primary service of an mpeg4-generic stream is to code Access
Units (AUs). We define the mpeg4-generic RTP MIDI AU as the MIDI
payload shown in Figure 1 of Section 2.1 of this memo: a MIDI command
section optionally followed by a journal section.
Exactly one RTP MIDI AU MUST be mapped to one mpeg4-generic RTP MIDI
packet. The mpeg4-generic options for placing several AUs in an RTP
packet MUST NOT be used with RTP MIDI. The mpeg4-generic options for
fragmenting and interleaving AUs MUST NOT be used with RTP MIDI. The
mpeg4-generic RTP packet payload (Figure 1 in [RFC3640]) MUST contain
empty AU Header and Auxiliary sections. These rules yield
mpeg4-generic packets that are structurally identical to native RTP
MIDI packets, an essential property for the correct operation of the
payload format.
The session description that follows defines a unicast UDP RTP
session (via a media ("m=") line) whose sole payload type (96) is
mapped to a minimal mpeg4-generic RTP MIDI stream. This example uses
the General MIDI Audio Object Type under Synthesis Profile @ Level 2.
v=0
o=lazzaro 2520644554 2838152170 IN IP6 first.example.net
s=Example
t=0 0
m=audio 5004 RTP/AVP 96
c=IN IP6 2001:DB8::7F2E:172A:1E24
a=rtpmap:96 mpeg4-generic/44100
a=fmtp:96 streamtype=5; mode=rtp-midi; profile-level-id=12;
config=7A0A0000001A4D546864000000060000000100604D54726B0000
000600FF2F000
(The a=fmtp line has been wrapped to fit the page to accommodate memo
formatting restrictions; it comprises a single line in SDP.)
The fmtp attribute line codes the four parameters (streamtype, mode,
profile-level-id, and config) that are required in all mpeg4-generic
session descriptions [RFC3640]. For RTP MIDI streams, the streamtype

parameter MUST be set to 5, the mode parameter MUST be set to rtp-
midi, and the profile-level-id parameter MUST be set to the MPEG-4
Profile Level for the stream. For the Synthesis Profile, legal
profile-level-id values are 11, 12, and 13, coding low (11), medium
(12), or high (13) decoder computational complexity, as defined by
MPEG conformance tests.
In a minimal RTP MIDI session description, the config value MUST be a
hexadecimal encoding [RFC3640] of the AudioSpecificConfig data block
[MPEGAUDIO] for the stream. AudioSpecificConfig encodes the Audio
Object Type for the stream and also encodes initialization data (SAOL
programs, DLS 2 wave tables, etc.). Standard MIDI Files encoded in
AudioSpecificConfig in a minimal session description MUST be ignored
by the receiver.
Receivers determine the rendering algorithm for the session by
interpreting the first 5 bits of AudioSpecificConfig as an unsigned
integer that codes the Audio Object Type. In our example above, the
5 bits are coded within the first two nibbles ("7A") of the config
string. The Audio Object Type coded within "7A" is Audio Object Type
15 (General MIDI). In Appendix E.4, we derive the config string
value in the session description shown above; the starting point of
the derivation is the MPEG bitstreams defined in [MPEGSA] and
[MPEGAUDIO].
We consider the stream to be "minimal" because the session
description does not customize the stream through the use of
parameters, other than the 4 required mpeg4-generic parameters
described above. In Section 6.1, we describe the behavior of a
minimal native stream as a numbered list of characteristics. Items
1-4 on that list also describe the minimal mpeg4-generic stream, but
items 5 and 6 require restatements, as listed below:
5. If more than one minimal mpeg4-generic stream appears in a
session, each stream uses an independent instance of the Audio
Object Type coded in the config parameter value.
6. A minimal mpeg4-generic stream encodes the AudioSpecificConfig as
an inline hexadecimal constant. If a session description is sent
over UDP, it may be impossible to transport large
AudioSpecificConfig blocks within the Maximum Transmission Unit
(MTU) of the underlying network (for Ethernet, the MTU is 1500
octets). In some cases, the AudioSpecificConfig block may exceed
the maximum size of the UDP packet itself.
The comments in Section 6.1 on SIP and RTSP stream directional
defaults, sendrecv MIDI channel usage, and MIDI 1.0 DIN multicast
networks also apply to mpeg4-generic RTP MIDI sessions.

In sendrecv sessions, each party's session description MUST use
identical values for the mpeg4-generic parameters (including the
required streamtype, mode, profile-level-id, and config parameters).
As a consequence, each party uses an identically configured MPEG 4
Audio Object Type to render MIDI commands into audio. The preamble
to Appendix C discusses a way to create "virtual sendrecv" sessions
that do not have this restriction.
6.3. Parameters
This section introduces parameters for session configuration for RTP
MIDI streams. In session descriptions, parameters modify the
semantics of a payload type. Parameters are specified on an fmtp
attribute line. See the session description example in Section 6.2
for an example of a fmtp attribute line.
The parameters add features to the minimal streams described in
Sections 6.1 and 6.2 and support several types of services:
o Stream subsetting. By default, all MIDI commands that are legal
to appear on a MIDI 1.0 DIN cable may appear in an RTP MIDI
stream. The cm_unused parameter overrides this default by
prohibiting certain commands from appearing in the stream. The
cm_used parameter is used in conjunction with cm_unused to
simplify the specification of complex exclusion rules. We
describe cm_unused and cm_used in Appendix C.1.
o Journal customization. The j_sec and j_update parameters
configure the use of the journal section. The ch_default,
ch_never, and ch_anchor parameters configure the semantics of the
recovery journal chapters. These parameters are described in
Appendix C.2 and override the default stream behaviors 1 and 2
(listed in Section 6.1 and referenced in Section 6.2).
o MIDI command timestamp semantics. The tsmode, octpos, mperiod,
and linerate parameters customize the semantics of timestamps in
the MIDI command section. These parameters let RTP MIDI
accurately encode the implicit time coding of MIDI 1.0 DIN cables.
These parameters are described in Appendix C.3 and override
default stream behavior 3 (listed in Section 6.1 and referenced in
Section 6.2).
o Media time. The rtp_ptime and rtp_maxptime parameters define the
temporal duration ("media time") of an RTP MIDI packet. The
guardtime parameter sets the minimum sending rate of stream
packets. These parameters are described in Appendix C.4 and
override default stream behavior 4 (listed in Section 6.1 and
referenced in Section 6.2).

o Stream description. The musicport parameter labels the MIDI name
space of RTP streams in a multimedia session. Musicport is
described in Appendix C.5. The musicport parameter overrides
default stream behavior 5 (in Sections 6.1 and 6.2).
o MIDI rendering. Several parameters specify the MIDI rendering
method of a stream. These parameters are described in Appendix
C.6 and override default stream behavior 6 (in Sections 6.1 and
6.2).
In Appendix C.7, we specify interoperability guidelines for two RTP
MIDI application areas: content streaming using RTSP (Appendix C.7.1)
and network musical performance using SIP (Appendix C.7.2).
7. Extensibility
The payload format defined in this memo exclusively encodes all
commands that may legally appear on a MIDI 1.0 DIN cable.
Many worthy uses of MIDI over RTP do not fall within the narrow scope
of the payload format. For example, the payload format does not
support the direct transport of Standard MIDI File (SMF) meta-event
and metric timing data. As a second example, the payload format does
not define transport tools for user-defined commands (apart from
tools to support System Exclusive commands [MIDI]).
The payload format does not provide an extension mechanism to support
new features of this nature, by design. Instead, we encourage the
development of new payload formats for specialized musical
applications. The IETF session management tools [RFC3264] [RFC2326]
support codec negotiation, to facilitate the use of new payload
formats in a backward-compatible way.
However, the payload format does provide several extensibility tools,
which we list below:
o Journalling. As described in Appendix C.2, new token values for
the j_sec and j_update parameters may be defined in IETF
Standards-Track documents. This mechanism supports the design of
new journal formats and the definition of new journal sending
policies.
o Rendering. The payload format may be extended to support new MIDI
renderers (Appendix C.6.2). Certain general aspects of the RTP
MIDI rendering process may also be extended, via the definition of
new token values for the render (Appendix C.6) and smf_info
(Appendix C.6.4.1) parameters.

o Undefined commands. [MIDI] reserves 4 MIDI system commands for
future use (0xF4, 0xF5, 0xF9, 0xFD). If updates to [MIDI] define
the reserved commands, IETF Standards-Track documents may be
defined to provide resiliency support for the commands. Opaque
LEGAL fields appear in System Chapter D for this purpose (Appendix
B.1.1).
A final form of extensibility involves the inclusion of the payload
format in framework documents. Framework documents describe how to
combine protocols to form a platform for interoperable applications.
For example, a stage and studio framework might define how to use SIP
[RFC3261], RTSP [RFC2326], SDP [RFC4566], and RTP [RFC3550] to
support media networking for professional audio equipment and
electronic musical instruments.
8. Congestion Control
The RTP congestion control requirements defined in [RFC3550] apply to
RTP MIDI sessions, and implementors should carefully read the
congestion control section in [RFC3550]. As noted in [RFC3550], all
transport protocols used on the Internet need to address congestion
control in some way, and RTP is not an exception.
In addition, the congestion control requirements defined in [RFC3551]
apply to RTP MIDI sessions run under applicable profiles. The basic
congestion control requirement defined in [RFC3551] is that RTP
sessions that use UDP transport should monitor packet loss (via RTCP
or other means) to ensure that the RTP stream competes fairly with
TCP flows that share the network.
Finally, RTP MIDI has congestion control issues that are unique for
an audio RTP payload format. In applications such as network musical
performance [NMP], the packet rate is linked to the gestural rate of
a human performer. Senders MUST monitor the MIDI command source for
patterns that result in excessive packet rates and take actions
during RTP transcoding to reduce the RTP packet rate. [RFC4696]
offers implementation guidance on this issue.
9. Security Considerations
Implementors should carefully read the Security Considerations
sections of the RTP [RFC3550], AVP [RFC3551], and other RTP profile
documents, as the issues discussed in these sections directly apply
to RTP MIDI streams. Implementors should also review the Secure
Real-time Transport Protocol (SRTP, [RFC3711]), an RTP profile that
addresses the security issues discussed in [RFC3550] and [RFC3551].

Here, we discuss security issues that are unique to the RTP MIDI
payload format.
When using RTP MIDI, authentication of incoming RTP and RTCP packets
is RECOMMENDED. Per-packet authentication may be provided by SRTP or
by other means. Without the use of authentication, attackers could
forge MIDI commands into an ongoing stream, damaging speakers and
eardrums. An attacker could also craft RTP and RTCP packets to
exploit known bugs in the client and take effective control of a
client machine.
Session management tools (such as SIP [RFC3261]) SHOULD use
authentication during the transport of all session descriptions
containing RTP MIDI media streams. For SIP, the Security
Considerations section in [RFC3261] provides an overview of possible
authentication mechanisms. RTP MIDI session descriptions should use
authentication because the session descriptions may code
initialization data using the parameters described in Appendix C. If
an attacker inserts bogus initialization data into a session
description, he can corrupt the session or forge an client attack.
Session descriptions may also code renderer initialization data by
reference, via the url (Appendix C.6.3) and smf_url (AppendixC.6.4.2) parameters. If the coded URL is spoofed, both session and
client are open to attack, even if the session description itself is
authenticated. Therefore, URLs specified in url and smf_url
parameters SHOULD use [RFC2818].
Section 2.1 allows streams sent by a party in two RTP sessions to
have the same SSRC value and the same RTP timestamp initialization
value, under certain circumstances. Normally, these values are
randomly chosen for each stream in a session, to make plaintext
guessing harder to do if the payloads are encrypted. Thus, Section
2.1 weakens this aspect of RTP security.
10. Acknowledgements
We thank the networking, media compression, and computer music
community members who have commented or contributed to the effort,
including Kurt B, Cynthia Bruyns, Steve Casner, Paul Davis, Robin
Davies, Joanne Dow, Tobias Erichsen, Roni Even, Nicolas Falquet,
Adrian Farrel, Dominique Fober, Philippe Gentric, Michael Godfrey,
Chris Grigg, Todd Hager, Alfred Hoenes, Russ Housley, Michel Jullian,
Phil Kerr, Young-Kwon Lim, Jessica Little, Jan van der Meer, Alexey
Melnikov, Colin Perkins, Charlie Richmond, Herbie Robinson, Dan
Romascanu, Larry Rowe, Eric Scheirer, Dave Singer, Martijn Sipkema,
Robert Sparks, William Stewart, Kent Terry, Sean Turner, Magnus
Westerlund, Tom White, Jim Wright, Doug Wyatt, and Giorgio Zoia. We

also thank the members of the San Francisco Bay Area music and audio
community for creating the context for the work, including Don
Buchla, Chris Chafe, Richard Duda, Dan Ellis, Adrian Freed, Ben Gold,
Jaron Lanier, Roger Linn, Richard Lyon, Dana Massie, Max Mathews,
Keith McMillen, Carver Mead, Nelson Morgan, Tom Oberheim, Malcolm
Slaney, Dave Smith, Julius Smith, David Wessel, and Matt Wright.
11. IANA Considerations
The bulk of this section is a verbatim reproduction of the IANA
considerations that appear in Section 11 of [RFC4695]. Preceding
this reproduction, we list several issues concerning this memo that
are related to the IANA considerations, as follows:
o All existing IANA references to [RFC4695] have been deleted, and
replaced with references to this memo. In addition, a reference
to this memo has been added to the audio/mpeg4-generic MIME type
registration.
o In Section 11.3, a sentence has been added to the Encoding
Considerations asc Media Type Registration: "Disk files that store
this data object use the file extension ".acn"".
The reproduction of the [RFC4695] IANA considerations section appears
directly below.
This section makes a series of requests to IANA. The IANA has
completed registration/assignments of the below requests.
The subsections that follow hold the actual, detailed requests. All
registrations in this section are in the IETF tree and follow the
rules of [RFC4288] and [RFC4855], as appropriate.
In Section 11.1, we request the registration of a new media type:
audio/rtp-midi. Paired with this request is a request for a
repository for new values for several parameters associated with
audio/rtp-midi. We request this repository in Section 11.1.1.
In Section 11.2, we request the registration of a new value (rtp-
midi) for the mode parameter of the mpeg4-generic media type. The
mpeg4-generic media type is defined in [RFC3640], and [RFC3640]
defines a repository for the mode parameter. However, we believe we
are the first to request the registration of a mode value, so we
believe the registry for mode has not yet been created by IANA.

Paired with our mode parameter value request for mpeg4-generic is a
request for a repository for new values for several parameters we
have defined for use with the rtp-midi mode value. We request this
repository in Section 11.2.1.
In Section 11.3, we request the registration of a new media type:
audio/asc. No repository request is associated with this request.
11.1. rtp-midi Media Type Registration
This section requests the registration of the rtp-midi subtype for
the audio media type. We request the registration of the parameters
listed in the "optional parameters" section below (both the "non-
extensible parameters" and the "extensible parameters" lists). We
also request the creation of repositories for the "extensible
parameters"; the details of this request appear in Section 11.1.1.
Media type name:
audio
Subtype name:
rtp-midi
Required parameters:
rate: The RTP timestamp clock rate. See Sections 2.1 and 6.1
for usage details.
Optional parameters:
Non-extensible parameters:
ch_anchor: See Appendix C.2.3 for usage details.
ch_default: See Appendix C.2.3 for usage details.
ch_never: See Appendix C.2.3 for usage details.
cm_unused: See Appendix C.1 for usage details.
cm_used: See Appendix C.1 for usage details.
chanmask: See Appendix C.6.4.3 for usage details.
cid: See Appendix C.6.3 for usage details.
guardtime: See Appendix C.4.2 for usage details.
inline: See Appendix C.6.3 for usage details.
linerate: See Appendix C.3 for usage details.
mperiod: See Appendix C.3 for usage details.
multimode: See Appendix C.6.1 for usage details.
musicport: See Appendix C.5 for usage details.
octpos: See Appendix C.3 for usage details.

Applications that use this media type:
Audio content-creation hardware, such as MIDI controller piano
keyboards and MIDI audio synthesizers. Audio content-creation
software, such as music sequencers, digital audio workstations,
and soft synthesizers. Computer operating systems, for network
support of MIDI Application Programmer Interfaces. Content
distribution servers and terminals may use this media type for
low bitrate music coding.
Additional information:
None.
Person & email address to contact for further information:
John Lazzaro <lazzaro@cs.berkeley.edu>
Intended usage:
COMMON.
Author:
John Lazzaro <lazzaro@cs.berkeley.edu>
Change controller:
IETF Audio/Video Transport Working Group delegated
from the IESG.
11.1.1. Repository Request for audio/rtp-midi
For the rtp-midi subtype, we request the creation of repositories for
extensions to the following parameters (which are those listed as
"extensible parameters" in Section 11.1).
j_sec:
Registrations for this repository may only occur
via an IETF Standards-Track document. Appendix C.2.1
of this memo describes appropriate registrations for this
repository.
Initial values for this repository appear below:
"none": Defined in Appendix C.2.1 of this memo.
"recj": Defined in Appendix C.2.1 of this memo.

j_update:
Registrations for this repository may only occur
via an IETF Standards-Track document. Appendix C.2.2
of this memo describes appropriate registrations for this
repository.
Initial values for this repository appear below:
"anchor": Defined in Appendix C.2.2 of this memo.
"open-loop": Defined in Appendix C.2.2 of this memo.
"closed-loop": Defined in Appendix C.2.2 of this memo.
render:
Registrations for this repository MUST include a
specification of the usage of the proposed value.
See the preamble of Appendix C.6 for details
(the paragraph that begins "Other render token ...").
Initial values for this repository appear below:
"unknown": Defined in Appendix C.6 of this memo.
"synthetic": Defined in Appendix C.6 of this memo.
"api": Defined in Appendix C.6 of this memo.
"null": Defined in Appendix C.6 of this memo.
subrender:
Registrations for this repository MUST include a
specification of the usage of the proposed value.
See Appendix C.6.2 for details (the paragraph
that begins "Other subrender token ...").
Initial values for this repository appear below:
"default": Defined in Appendix C.6.2 of this memo.
smf_info:
Registrations for this repository MUST include a
specification of the usage of the proposed value.
See Appendix C.6.4.1 for details (the paragraph
that begins "Other smf_info token ...").

Initial values for this repository appear below:
"ignore": Defined in Appendix C.6.4.1 of this memo.
"sdp_start": Defined in Appendix C.6.4.1 of this memo.
"identity": Defined in Appendix C.6.4.1 of this memo.
11.2. mpeg4-generic Media Type Registration
This section requests the registration of the rtp-midi value for the
mode parameter of the mpeg4-generic media type. The mpeg4-generic
media type is defined in [RFC3640], and [RFC3640] defines a
repository for the mode parameter. We are registering mode rtp-midi
to support the MPEG Audio codecs [MPEGSA] that use MIDI.
In conjunction with this registration request, we request the
registration of the parameters listed in the "optional parameters"
section below (both the "non-extensible parameters" and the
"extensible parameters" lists). We also request the creation of
repositories for the "extensible parameters"; the details of this
request appear in Appendix 11.2.1.
Media type name:
audio
Subtype name:
mpeg4-generic
Required parameters:
The mode parameter is required by [RFC3640]. [RFC3640]
requests a repository for mode so that new values for mode
may be added. We request that the value rtp-midi be
added to the mode repository.
In mode rtp-midi, the mpeg4-generic parameter rate is
a required parameter. Rate specifies the RTP timestamp
clock rate. See Sections 2.1 and 6.2 for usage details
of rate in mode rtp-midi.
Optional parameters:
We request registration of the following parameters
for use in mode rtp-midi for mpeg4-generic.

Security considerations:
See Section 9 of this memo.
Interoperability considerations:
Except for the marker bit (Section 2.1), the packet formats
for audio/rtp-midi and audio/mpeg4-generic (mode rtp-midi)
are identical. The formats differ in use: audio/mpeg4-generic
is for MPEG work, and audio/rtp-midi is for all other work.
Published specification:
This memo, [MIDI], and [MPEGSA] are the normative references.
In addition, [NMP], [GRAME], and [RFC4696] provide
non-normative implementation guidance.
Applications that use this media type:
MPEG 4 servers and terminals that support [MPEGSA].
Additional information:
None.
Person & email address to contact for further information:
John Lazzaro <lazzaro@cs.berkeley.edu>
Intended usage:
COMMON.
Author:
John Lazzaro <lazzaro@cs.berkeley.edu>
Change controller:
IETF Audio/Video Transport Working Group delegated
from the IESG.
11.2.1. Repository Request for Mode rtp-midi for mpeg4-generic
For mode rtp-midi of the mpeg4-generic subtype, we request the
creation of repositories for extensions to the following parameters
(which are those listed as "extensible parameters" in Section 11.2).

j_sec:
Registrations for this repository may only occur
via an IETF Standards-Track document. Appendix C.2.1
of this memo describes appropriate registrations for this
repository.
Initial values for this repository appear below:
"none": Defined in Appendix C.2.1 of this memo.
"recj": Defined in Appendix C.2.1 of this memo.
j_update:
Registrations for this repository may only occur
via an IETF Standards-Track document. Appendix C.2.2
of this memo describes appropriate registrations for this
repository.
Initial values for this repository appear below:
"anchor": Defined in Appendix C.2.2 of this memo.
"open-loop": Defined in Appendix C.2.2 of this memo.
"closed-loop": Defined in Appendix C.2.2 of this memo.
render:
Registrations for this repository MUST include a
specification of the usage of the proposed value.
See the preamble of Appendix C.6 for details
(the paragraph that begins "Other render token ...").
Initial values for this repository appear below:
"unknown": Defined in Appendix C.6 of this memo.
"synthetic": Defined in Appendix C.6 of this memo.
"null": Defined in Appendix C.6 of this memo.
subrender:
Registrations for this repository MUST include a
specification of the usage of the proposed value.
See Appendix C.6.2 for details (the paragraph
that begins "Other subrender token ..." and
subsequent paragraphs). Note that the text in
Appendix C.6.2 contains restrictions on subrender
registrations for mpeg4-generic (the sentence that

begins "Registrations for mpeg4-generic subrender
values ...").
Initial values for this repository appear below:
"default": Defined in Appendix C.6.2 of this memo.
smf_info:
Registrations for this repository MUST include a
specification of the usage of the proposed value.
See Appendix C.6.4.1 for details (the paragraph
that begins "Other smf_info token ...").
Initial values for this repository appear below:
"ignore": Defined in Appendix C.6.4.1 of this memo.
"sdp_start": Defined in Appendix C.6.4.1 of this memo.
"identity": Defined in Appendix C.6.4.1 of this memo.
11.3. asc Media Type Registration
This section registers asc as a subtype for the audio media type. We
register this subtype to support the remote transfer of the "config"
parameter of the mpeg4-generic media type [RFC3640] when it is used
with mpeg4-generic mode rtp-midi (registered in Appendix 11.2 above).
We explain the mechanics of using audio/asc to set the config
parameter in Section 6.2 and Appendix C.6.5 of this document.
Note that this registration is a new subtype registration and is not
an addition to a repository defined by MPEG-related memos (such as
[RFC3640]). Also, note that this request for audio/asc does not
register parameters and does not request the creation of a
repository.
Media type name:
audio
Subtype name:
asc
Required parameters:
None.

Optional parameters:
None.
Encoding considerations:
The native form of the data object is binary data,
zero-padded to an octet boundary. Disk files that
store this data object use the file extension ".acn".
Restrictions on usage:
This type is only defined for data object (stored file)
transfer. The most common transports for the type are
HTTP and SMTP.
Security considerations:
See Section 9 of this memo.
Interoperability considerations:
None.
Published specification:
The audio/asc data object is the AudioSpecificConfig
binary data structure, which is normatively defined in
[MPEGAUDIO].
Applications that use this media type:
MPEG 4 Audio servers and terminals that support
audio/mpeg4-generic RTP streams for mode rtp-midi.
Additional information:
None.
Person & email address to contact for further information:
John Lazzaro <lazzaro@cs.berkeley.edu>
Intended usage:
COMMON.

Author:
John Lazzaro <lazzaro@cs.berkeley.edu>
Change controller:
IETF Audio/Video Transport Working Group delegated
from the IESG.
12. Changes from RFC 4695
This document fixes errors found in RFC 4695 by reviewers. We thank
Alfred Hoenes, Roni Even, and Alexey Melnikov for reporting the
errors. To our knowledge, there are no interoperability issues
associated with the errors that are fixed by this document. In this
section, we list the error fixes.
In Section 3 of RFC 4695, the bitfield format shown in Figure 3 is
inconsistent with the normative text that (correctly) describes the
bitfield. Specifically, Figure 3 in RFC 4695 incorrectly states the
dependence of the Delta Time 0 field on the Z header bit. Figure 3
in this document corrects this error. To our knowledge, this error
did not result in incorrect implementations of RFC 4695.
The remaining errors are in Appendices C and D and concern session
configuration parameters. The numbered list ((1) through (11)) below
describes these errors in detail, in order of appearance in the
document. To our knowledge, there are no interoperability issues
associated with these errors, as implementation activity has so far
focused on an application domain that does not use SDP for session
management.
(1) In Appendices C.1 and C.2.3 of RFC 4695, an ABNF rule related to
System Chapter X is incorrectly defined as:
<parameter> = "__" <h-list> ["_" <h-list>] "__"
The correct version of this rule is:
<parameter> = "__" <h-list> *( "_" <h-list> ) "__"
(2) In Appendix C.6.3 of RFC 4695, the URIs permitted to be assigned
to the url parameter are not stated clearly. URIs assigned to url
MUST specify either HTTP or HTTP over TLS transport protocols.
In Appendix C.7.1 and C.7.2 of RFC 4695, the transport
interoperability requirements for the url parameter are not stated

clearly. For both C.7.1 and C.7.2, HTTP is REQUIRED and HTTP over
TLS is OPTIONAL.
(3) In Appendix C.6.5, the filename extension ".acn" has been defined
for use with AudioSpecificConfig.
(4) Both fmtp lines in both session description examples in Appendix
C.7.2 of RFC 4695 contain instances of the same syntax error (a
missing ";" at a line wrap after a cm_used assignment).
(5) In the session description examples in Appendix C.5, C.6, and C.7
of RFC 4695, the parameter assignment
rinit="audio/asc";
is incorrect. The correct parameter assignment appears below:
rinit=audio/asc;
Note that this error also appears in the session descriptions shown
in Figures 1 and 2 of the informative RFC 4696. We are not aware of
existing implementations that use the rinit parameter, and so the
incorrect examples in RFC 4695 and RFC 4696 should not cause
interoperability problems.
(6) In Appendix D of RFC 4695, all uses of "*ietf-extension" in rules
are in error and should be replaced with "ietf-extension". Likewise,
all uses of "*extension" are in error and should be replaced with
"extension". This bug incorrectly lets the null token be assigned to
the j_sec, j_update, render, smf_info, and subrender parameters.
(7) In Appendix D of RFC 4695, the definitions of <command-type> and
<chapter-rules> incorrectly allow lowercase letters to appear in
these strings. The correct definitions of these rules appear below:
command-type = [A] [B] [C] [F] [G] [H] [J] [K] [M]
[N] [P] [Q] [T] [V] [W] [X] [Y] [Z]
chapter-list = [A] [B] [C] [D] [E] [F] [G] [H] [J] [K]
[M] [N] [P] [Q] [T] [V] [W] [X] [Y] [Z]
A = %x41
B = %x42
C = %x43
D = %x44
E = %x45
F = %x46
G = %x47