Subject: [payload] Adam Roach's Discuss on draft-ietf-payload-rtp-ancillary-10: (with DISCUSS and COMMENT)

From: Adam Roach <[email-masked]>
Date: Tue, 15 Aug 2017 19:24:14 -0700

Adam Roach has entered the following ballot position for
draft-ietf-payload-rtp-ancillary-10: Discuss

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.

The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-payload-rtp-ancillary/

----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------

The grouping semantics here are problematic. Section 4.1 talks about LS to
associate ANC streams with other streams. LS is strictly a synchronization
construct, not used to otherwise state relationships between streams. More
importantly, there's nothing that prevents it being used to synchronize an
arbitrary number of video streams with each other. The problem that arises is
best illustrated by this example, where I'm going to transmit two video
streams, each with their own ANC data, but need them to be synchronized (e.g.,
perhaps I'm going to have two border-less screens physically right next to each
other for a double-wide image, and synchronization is important to avoid
tearing):

        v=0
        o=Al 123456 11 IN IP4 host.example.com
        s=Professional Networked Media Test
        i=A test of synchronized video and ANC data
        t=0 0
        a=group:LS V1 V2 M1 M2
        m=video 50000 RTP/AVP 96
        c=IN IP4 233.252.0.1/255
        a=rtpmap:96 raw/90000
        a=fmtp:96 sampling=YCbCr-4:2:2; width=1280; height=720; depth=10
        a=mid:V1
        m=video 50002 RTP/AVP 96
        c=IN IP4 233.252.0.1/255
        a=rtpmap:96 raw/90000
        a=fmtp:96 sampling=YCbCr-4:2:2; width=1280; height=720; depth=10
        a=mid:V2
        m=video 50010 RTP/AVP 97
        c=IN IP4 233.252.0.2/255
        a=rtpmap:97 smpte291/90000
        a=fmtp:97 DID_SDID={0x61,0x02};DID_SDID={0x41,0x05}
        a=mid:M1
        m=video 50012 RTP/AVP 97
        c=IN IP4 233.252.0.2/255
        a=rtpmap:97 smpte291/90000
        a=fmtp:97 DID_SDID={0x61,0x02};DID_SDID={0x41,0x05}
        a=mid:M2

If the recipient of these mutually synchronized streams needs to reintegrate
the ANC data into the video for offloading to an HDMI connection, how does it
know which ANC stream to insert into which video stream?

There needs to be some additional association mechanism here. You can either
add an attribute to the smpte291 media sections that clearly indicates which
video stream they correspond to, or you can include the smpte291 packets in the
same stream as the corresponding video, like so:

        v=0
        o=Al 123456 11 IN IP4 host.example.com
        s=Professional Networked Media Test
        i=A test of synchronized video and ANC data
        t=0 0
        m=video 50000 RTP/AVP 96 97
        c=IN IP4 233.252.0.1/255
        a=rtpmap:96 raw/90000
        a=rtpmap:97 smpte291/90000
        a=fmtp:96 sampling=YCbCr-4:2:2; width=1280; height=720; depth=10
        a=fmtp:97 DID_SDID={0x61,0x02};DID_SDID={0x41,0x05}
        m=video 50002 RTP/AVP 96 97
        c=IN IP4 233.252.0.1/255
        a=rtpmap:96 raw/90000
        a=rtpmap:97 smpte291/90000
        a=fmtp:96 sampling=YCbCr-4:2:2; width=1280; height=720; depth=10
        a=fmtp:97 DID_SDID={0x61,0x02};DID_SDID={0x41,0x05}

(See RFC4733 for an example of a payload that performs stream association in
this fashion)

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

It is common in examples of RTP payloads (such as Figure 1) to include values
for those fields whose values are illustrative of the format itself. The "V=2"
in Figure 1 is an example of this. I would suggest that the example would be
slightly improved by including a similar notation for (e.g.) "Length = 32",
"ANC_Count = 2", and the two Data_Count fields.

Section 2.1 defines "b01" as "invalid" for the F field. It does not, however,
prescribe what recipients of such a value should do; reasonable options would
include (a) treat it as 0b00; (b) ignore the ANC but process any others that
might be present; or (c) ignore the entire packet.  I'll note that (b) allows
you to come back and add semantics for "0b01" at a later date, if you find a
reason to do so. In any case, to minimize variations among recipients, I would
suggest defining concrete behavior here.

[Warning: I'm not terribly familiar with SMPTE video formats, and with the
UHD-and-higher specs in particular, so it may be that the following comment can
be answered with "that can never happen no matter how high-res streams get in
the future."] As far as I know, SMPTE defines 4k as having 4 streams (each
efectively a normal 1080 stream), 8k as having 16, and the future-looking 16k
(presumably) having 64. Given that the Horizontal_Offset can communicate widths
only up to 4093 samples (well within striking distance of the 2640 samples used
by a single stream), and that the StreamNum field can communicate at most 128
streams, it seems that any sizes larger than 16k run the risk of exceeding the
ability for these fields to place ancillary data packets at the correct
horizontal position. I would recommend defining another reserved value for
Horizontal_Offset (0xFFD), that can be used to mean "longer field follows" in a
future specification; and a reserved value (0x7F) for StreamNum for the same
purpose. Line_Number, with a maximum value of 2045, is similarly within
striking distance of the 1125 lines used by a single stream, and would likely
need the same accommodations.

It is implied, but not stated, that implementations should use different
sessions (or at least different media sections) to convey smpte291 payloads
than the video streams they correspond to. If this is the intention, please
include a normative statement to that effect. If it is okay to include both
payload types in the same media section, please add a statement explicitly
saying so, and indicate that this can be used as an alternative to using LS
groups.

Section 5.1 indicates that the answerer can respond with "all or a subset" of
the DID_SDID parameters, or that it "MAY reject the offer."  Of course, an
answerer is always able to reject an offer, but this text seems to imply that
it might do so if it doesn't want to receive any of the DID_SDID parameters
indicated in the offer. That seems like massive overkill. Since the model here
appears to be using multiple sessions (or, at least, separate media sections)
to convey smpte291 data, I would offer that this should instead say something
like "MAY set the corresponding port number to 0 to decline the smpte291
stream". (If you opt to allow the smpte291 payloads to be in the same media
section as their corresponding video streams, this would instead involve
removing the corresponding PT).

Nit: The final line in the final example of section 4 is too long. Consider
outdenting by two characters.