Received changes through RFC Editor sync (created alias RFC 8451, changed title to 'Considerations for Selecting RTP Control Protocol (RTCP) Extended Report (XR) Metrics for the WebRTC Statistics API', changed abstract to 'This document describes monitoring features related to media streams in Web real-time communication (WebRTC). It provides a list of RTP Control Protocol (RTCP) Sender Report (SR), Receiver Report (RR), and Extended Report (XR) metrics, which may need to be supported by RTP implementations in some diverse environments. It lists a set of identifiers for the WebRTC's statistics API. These identifiers are a set of RTCP SR, RR, and XR metrics related to the transport of multimedia flows.', changed pages to 18, changed standardization level to Informational, changed state to RFC, added RFC published event at 2018-09-19, changed IESG state to RFC Published)

> Error-resilience mechanisms, like RTP retransmission or FEC, are> optional in RTCWEB because the overhead of the repair bits adding to> the original streams.

This is a little misleading. draft-ietf-rtcweb-rtp-usage says:

WebRTC endpoints MUST follow the recommendations for FEC use given in [I-D.ietf-rtcweb-fec].

And, in turn, draft-ietf-rtcweb-fec says:

To support the functionality recommended above, implementations MUST be able to receive and make use of the relevant FEC formats for their supported audio codecs, and MUST indicate this support, as described in Section 4. Use of these formats when sending, as mentioned above, is RECOMMENDED.

Rather than trying to reiterate this somewhat nuanced situation in *this*document, I recommend removing the entire sentence and fixing the rest of theparagraph. If you choose to keep it, please make sure it more clearly explainsthe level of FEC support required of RTCWEB implementations.

> For example, it may be useful to distinguish between> packets lost and packets discarded due to late arrival, even though> they have the same impact on the multimedia quality, it helps in> identifying and diagnosing issues.

This is a run-on sentence. Suggest: "...due to late arrival. Even though..."

> The WebRTC application extracts the statistic from the browser by> querying the getStats() API [W3C.WD-webrtc-20161124], but the browser> currently only reports the local variables i.e., the statistics> related to the outgoing RTP media streams and the incoming RTP media> streams.

I don't think the "currently" in this sentence is actually true any longer; andI expect it to get increasingly less true as time goes on. Consider rephrasing.

> [I-D.ietf-rtcweb-rtp-usage] does not mandate the use of> any RTCP XRs and since their usage is optional.

> For example, an application may choose> to poll the stack for statistics every 1 second, in this case the> underlying stack local will return the current snapshot of the local> statistics (for incoming and outgoing media streams).

This is a comma splice. Consider: "...every second. In this case..."

> However it may> return the same remote statistics as before for the remote> statistics

> Error-resilience mechanisms, like RTP retransmission or FEC, are> optional in RTCWEB because the overhead of the repair bits adding to> the original streams.

This is a little misleading. draft-ietf-rtcweb-rtp-usage says:

WebRTC endpoints MUST follow the recommendations for FEC use given in [I-D.ietf-rtcweb-fec].

And, in turn, draft-ietf-rtcweb-fec says:

To support the functionality recommended above, implementations MUST be able to receive and make use of the relevant FEC formats for their supported audio codecs, and MUST indicate this support, as described in Section 4. Use of these formats when sending, as mentioned above, is RECOMMENDED.

Rather than trying to reiterate this somewhat nuanced situation in *this*document, I recommend removing the entire sentence and fixing the rest of theparagraph. If you choose to keep it, please make sure it more clearly explainsthe level of FEC support required of RTCWEB implementations.

> For example, it may be useful to distinguish between> packets lost and packets discarded due to late arrival, even though> they have the same impact on the multimedia quality, it helps in> identifying and diagnosing issues.

This is a run-on sentence. Suggest: "...due to late arrival. Even though..."

> The WebRTC application extracts the statistic from the browser by> querying the getStats() API [W3C.WD-webrtc-20161124], but the browser> currently only reports the local variables i.e., the statistics> related to the outgoing RTP media streams and the incoming RTP media> streams.

I don't think the "currently" in this sentence is actually true any longer; andI expect it to get increasingly less true as time goes on. Consider rephrasing.

> [I-D.ietf-rtcweb-rtp-usage] does not mandate the use of> any RTCP XRs and since their usage is optional.

> For example, an application may choose> to poll the stack for statistics every 1 second, in this case the> underlying stack local will return the current snapshot of the local> statistics (for incoming and outgoing media streams).

This is a comma splice. Consider: "...every second. In this case..."

> However it may> return the same remote statistics as before for the remote> statistics

[Ballot Position Update] New position, No Objection, has been recorded for Adam Roach

2018-05-21

09

Deborah Brungard

[Ballot Position Update] New position, No Objection, has been recorded for Deborah Brungard

2018-05-21

09

Spencer Dawkins

[Ballot comment]Thanks for producing this document.

I did have one comment (and I apologize in advance that this is a particular hot button for ...

[Ballot comment]Thanks for producing this document.

I did have one comment (and I apologize in advance that this is a particular hot button for me).

I understand that you can do things with these metrics that you can't do with previous metrics, but I wish there was a clearer description of "if you want to understand X, you should implement this metric" earlier in the document. The impression the reader is left with, is "if you're doing RTCWeb, you should implement this metric", but I'm guessing that at least some of these metrics would be useful for people who weren't doing RTCWeb, but need to measure something that people who are doing RTCWeb also need to measure, and if that's true, a short section that helped them figure that out would be helpful.

2018-05-21

09

Spencer Dawkins

[Ballot Position Update] New position, No Objection, has been recorded for Spencer Dawkins

2018-05-21

09

Ben Campbell

Intended Status changed to Informational from Proposed Standard

2018-05-18

09

Alissa Cooper

[Ballot Position Update] New position, No Objection, has been recorded for Alissa Cooper

2018-05-18

09

Mirja Kühlewind

[Ballot discuss]For the AD: Please note that the intended status in the datatracker is "Proposed Standard" while the doc itself says "Informational". After reading ...

[Ballot discuss]For the AD: Please note that the intended status in the datatracker is "Proposed Standard" while the doc itself says "Informational". After reading the doc, I would find informational correct. However, please clarify what the intended status is supposed to be!

2018-05-18

09

Mirja Kühlewind

[Ballot comment]Thanks for the well-written document. A few comments mostly on references:

- Maybe also provide an (informative) reference to draft-ietf-rtcweb-overview-19 on the first occurrence ...

[Ballot comment]Thanks for the well-written document. A few comments mostly on references:

- Maybe also provide an (informative) reference to draft-ietf-rtcweb-overview-19 on the first occurrence of WebRTC...?

- Is ReportGroup defined in another RFC? If so, please provide reference in ther terminology section.

The IESG has received a request from the Metric Blocks for use with RTCP'sExtended Report Framework WG (xrblock) to consider the following document: -'Considerations for Selecting RTCP Extended Report (XR) Metrics for the WebRTC Statistics API' <draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-08.txt> as Proposed Standard

The IESG plans to make a decision in the next few weeks, and solicits finalcomments on this action. Please send substantive comments to theietf@ietf.org mailing lists by 2018-02-23. Exceptionally, comments may besent to iesg@ietf.org instead. In either case, please retain the beginning ofthe Subject line to allow automated sorting.

Abstract

This document describes monitoring features related to media streams in Web real-time communication (WebRTC). It provides a list of RTCP Sender Report, Receiver Report and Extended Report metrics, which may need to be supported by RTP implementations in some diverse environments. It lists a set of identifiers for the WebRTC's statistics API. These identifiers are a set of RTCP SR, RR, and XR metrics related to the transport of multimedia flows.

This is my AD Evaluation of draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-06. I would like to resolve at least the substantive comments prior to IETF LC.—————————Substantive:

- General: If I ...

This is my AD Evaluation of draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-06. I would like to resolve at least the substantive comments prior to IETF LC.—————————Substantive:

- General: If I understand correctly, this document lists and describes XR metrics that a WebRTC application might choose to negotiate and expose via the WebRTC API. It does not define any new XR metrics. That brings up two questions:

1) Why is this specific to WebRTC? For the most parts, all the arguments here would apply to any sort of RTP endpoint.

2) Why is this standards track? This sort of material is typically informational, or occasionally a BCP. There are a small number of 2119 keywords, but I’m not sure they are needed or appropriate. (Specifics below.) My initial instinct is that this should be informational.

- General: This needs an IANA considerations section, even if it just contains the statement to the effect of “This document makes no requests for IANA”.

- 1, last paragraph: “ The document also creates a registry containing identifiers from the metrics reported in the RTCP Sender, Receiver, and Extended Reports.”

It doesn’t actually do that. I understand from the shepherd’s report that this was intentionally removed.

— “All identifiers proposed in this document are RECOMMENDED to be implemented by an endpoint. An endpoint MAY choose not to expose an identifier if it does not implement the corresponding RTCP Report. “

Does the “RECOMMENDED” apply to all endpoints or just WebRTC endpoints? If the latter, doesn’t that requirement belong (or need to update) some requirement from an RTCWEB (maybe rtp-usage) document, or even the API itself? (I suspect that this draft does not have standing to state this normatively.) What is meant by “MAY choose not to expose”. Is that talking about via the WebRTC API? If so, isn’t that up to the API definition? (That is, it shouldn’t be normative here.)

-2: If this document keeps the normative keywords, please use the updated boilerplate from RFC 8174. (I note that there are at least some uncapitalized “may”s that do not seem normative.)

- 5.2.2, last paragraph: “The following metrics can also be considered…”Be considered by whom? Implementors? API designers?

- 6, first paragraph: “In practice the application MUST be able to query the statistic identifiers on both an incoming (remote) and outgoing (local) media stream.”What does the MUST requirement apply to? The WebRTC API? This seems more like a statement of fact that “ the application needs to be able…”

-8: "Therefore encryption procedures, such as the ones suggested for a Secure RTCP (SRTCP), need to be used.”The text should describe the reasons encryption is needed. Also, is this a new normative requirement, or a restatement of an existing requirement?

Editorial:

-1, paragraph 1:" If sufficient information (metrics or statistics) are provided to the applications, it can attempt to improve the media quality. “

s / “are provided” / “is provided”s / applications / application

-3, 2nd paragraph: It would help to clarify that the references to section 5 and 6 are to this document, not RFC 3611.

— 3rd paragraph: "At the moment…”Please clarify that is at the time of writing (not reading)

The document shepherd is Shida Schubert. The responsible Area Director is Ben Campbell.

(1) What type of RFC is being requested (BCP, Proposed ...

1. Summary

The document shepherd is Shida Schubert. The responsible Area Director is Ben Campbell.

(1) What type of RFC is being requested (BCP, Proposed Standard, Internet Standard, Informational, Experimental, or Historic)? Why is this the proper type of RFC? Is this type of RFC indicated in the title page header?

- Proposed Standard- The title page header indicates that it is a standard-track.- The document defines a process/mechanism for standard defined in XRBLOCK WG to be referenced/used by WebRTC WG in W3C.

(2) The IESG approval announcement includes a Document Announcement Write-Up. Please provide such a Document Announcement Write-Up. Recent examples can be found in the "Action" announcements for approved documents. The approval announcement contains the following sections:

Technical Summary:This document describes monitoring features related to media streams in Web real-time communication (WebRTC). It provides a list of RTCP Sender Report, Receiver Report and Extended Report metrics, which may need to be supported by RTP implementations in some diverse environments.For each of the metrics proposed, the document provides rational along with consideration when using them.

Working Group Summary:A lot of effort went into how to ensure proposed metrics are referenced and used by WebRTC WG in W3C. There were no objection or questions about the usefulness of the metrics proposed but more so about ensuringimplementors are aware and take full advantage of the metrics.

Initially a creating a registry was agreed to be the right way forward but after much debates, in order for better exposure to the implementing community, it was agreed that metrics defined in IETF are to be added to thespecification within W3C and for this document to define initial proposed metrics along with how to add/expose future metrics useful to the WebRTC WG and surrounding community.

The defined metrics here are already part of the W3C documents and the metrics proposed to be used by WebRTC are implemented and used in the wild.

Personnel:The document shepherd is Shida Schubert.

(3) Briefly describe the review of this document that was performed by the Document Shepherd. If this version of the document is not ready for publication, please explain why the document is being forwarded to the IESG.

The document shepherd has read the draft and is confident that this document is ready for publication.

(4) Does the document Shepherd have any concerns about the depth or breadth of the reviews that have been performed?

No.

(5) Do portions of the document need review from a particular or from broader perspective, e.g., security, operational complexity, AAA, DNS, DHCP, XML, or internationalization? If so, describe the review that took place.

We had contributors from OPS area and W3C folks beside the usual reviewers in XRBLOCK contributing to development of this document.

(6) Describe any specific concerns or issues that the Document Shepherd has with this document that the Responsible Area Director and/or the IESG should be aware of? For example, perhaps he or she is uncomfortable with certain parts of the document, or has concerns whether there really is a need for it. In any event, if the WG has discussed those issues and has indicated that it still wishes to advance the document, detail those concerns here.

None.

(7) Has each author confirmed that any and all appropriate IPR disclosures required for full conformance with the provisions of BCP 78 and BCP 79 have already been filed. If not, explain why?

Yes.

(8) Has an IPR disclosure been filed that references this document? If so, summarize any WG discussion and conclusion regarding the IPR disclosures.

Irrelevant as no IPR disclosures have been filed.

(9) How solid is the WG consensus behind this document? Does it represent the strong concurrence of a few individuals, with others being silent, or does the WG as a whole understand and agree with it?

The WG as a whole understand and agree with the current standing of specification.

(10) Has anyone threatened an appeal or otherwise indicated extreme discontent? If so, please summaries the areas of conflict in separate email messages to the Responsible Area Director. (It should be in a separate email because this questionnaire is publicly available.)

There was nothing except for the format of the document that raised extensive discussions. Current form of this document is the format that the Work group agreed will provide the most value to the impelemtors.

(12) Describe how the document meets any required formal review criteria, such as the MIB Doctor, media type, and URI type reviews.

N/A

(13) Have all references within this document been identified as either normative or informative?

YES

(14) Are there normative references to documents that are not ready for advancement or are otherwise in an unclear state? If such normative references exist, what is the plan for their completion?

No problem here.

(15) Are there downward normative references references (see RFC 3967)? If so, list these downward references to support the Area Director in the Last Call procedure.

No.

(16) Will publication of this document change the status of any existing RFCs? Are those RFCs listed on the title page header, listed in the abstract, and discussed in the introduction? If the RFCs are not listed in the Abstract and Introduction, explain why, and point to the part of the document where the relationship of this document to the other RFCs is discussed. If this information is not in the document, explain why the WG considers it unnecessary.

No.

(17) Describe the Document Shepherd's review of the IANA considerations section, especially with regard to its consistency with the body of the document. Confirm that all protocol extensions that the document makes are associated with the appropriate reservations in IANA registries. Confirm that any referenced IANA registries have been clearly identified. Confirm that newly created IANA registries include a detailed specification of the initial contents for the registry, that allocations procedures for future registrations are defined, and a reasonable name for the new registry has been suggested (see RFC 5226).

N/A

(18) List any new IANA registries that require Expert Review for future allocations. Provide any public guidance that the IESG would find useful in selecting the IANA Experts for these new registries.

N/A

(19) Describe reviews and automated checks performed by the Document Shepherd to validate sections of the document written in a formal language, such as XML code, BNF rules, MIB definitions, etc.

No nits error

2017-09-21

06

Cindy Morgan

Notification list changed to Shida Schubert <shida.at.ietf@gmail.com>

2017-09-21

06

Cindy Morgan

Document shepherd changed to Shida Schubert

2017-09-21

06

Cindy Morgan

Responsible AD changed to Ben Campbell

2017-09-21

06

Cindy Morgan

Intended Status changed to Proposed Standard

2017-09-21

06

Cindy Morgan

IESG process started in state Publication Requested

2017-09-21

06

Cindy Morgan

Working group state set to Submitted to IESG for Publication

2017-07-20

06

Varun Singh

New version available: draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-06.txt