Subject: Re: [payload] Adam Roach's Discuss on draft-ietf-payload-rtp-ancillary-10: (with DISCUSS and COMMENT)

From: Thomas Edwards <[email-masked]>
Date: Wed, 20 Sep 2017 13:31:59 +0000

Adam,

Thanks for your DISCUSS on the Ancillary RTP I-D.  I understand your concern about the difference between RTP streams that are simply synchronized using “LS” versus RTP streams that also have an intimate relationship (such as being part of the same “m=” media section).

The challenge is that the use case of the Ancillary I-D within the SMPTE ST 2110 architecture would generally require separate network destination addresses for the different elements (video, audio, ancillary data) so they can be flexibly composed within broadcast plants at the network level.  Your alternative SDP example appears to place both the video and the ancillary data onto the same destination address (233.252.0.1).  Is there an appropriate SDP mechanism to communicate the intimate relationship between the RTP streams, but also have them transmitted on separate destination IP addresses (and synchronized)?

(In production practice, this may not be a significant issue as the RTP streams to be combined by IP receivers for conversion to SDI/HDMI or for input to a distribution encoder may be specified by a control system directly to the receiver using an API that isn’t just an SDP file.)

-Thomas Edwards
 FOX

On 8/15/17, 7:24 PM, "Adam Roach" <[email-masked]> wrote:

    ----------------------------------------------------------------------
    DISCUSS:
    ----------------------------------------------------------------------
    
    The grouping semantics here are problematic. Section 4.1 talks about LS to
    associate ANC streams with other streams. LS is strictly a synchronization
    construct, not used to otherwise state relationships between streams. More
    importantly, there's nothing that prevents it being used to synchronize an
    arbitrary number of video streams with each other. The problem that arises is
    best illustrated by this example, where I'm going to transmit two video
    streams, each with their own ANC data, but need them to be synchronized (e.g.,
    perhaps I'm going to have two border-less screens physically right next to each
    other for a double-wide image, and synchronization is important to avoid
    tearing):
    
            v=0
            o=Al 123456 11 IN IP4 https://urldefense.proofpoint.com/v2/url?u=http-3A__host.example.com&d=DwICaQ&c=uw6TLu4hwhHdiGJOgwcWD4AjKQx6zvFcGEsbfiY9-EI&r=lekNOOM5noV61zrPH3rwPyhtNnLLWoLEHgd0quQxly8&m=uqtx2oHwdMjoXvsVmB3c2yiP1jUG970YiY9D6u40vC4&s=HQ1haZcCzEc4KXzHJA54Qk3c-u2bkzVIMwUSCtAnmkU&e= 
            s=Professional Networked Media Test
            i=A test of synchronized video and ANC data
            t=0 0
            a=group:LS V1 V2 M1 M2
            m=video 50000 RTP/AVP 96
            c=IN IP4 https://urldefense.proofpoint.com/v2/url?u=http-3A__233.252.0.1_255&d=DwICaQ&c=uw6TLu4hwhHdiGJOgwcWD4AjKQx6zvFcGEsbfiY9-EI&r=lekNOOM5noV61zrPH3rwPyhtNnLLWoLEHgd0quQxly8&m=uqtx2oHwdMjoXvsVmB3c2yiP1jUG970YiY9D6u40vC4&s=PhRxFiNrvX2Ek5_CdsixJxYiH3alearJ9KKI-agaFlQ&e= 
            a=rtpmap:96 raw/90000
            a=fmtp:96 sampling=YCbCr-4:2:2; width=1280; height=720; depth=10
            a=mid:V1
            m=video 50002 RTP/AVP 96
            c=IN IP4 https://urldefense.proofpoint.com/v2/url?u=http-3A__233.252.0.1_255&d=DwICaQ&c=uw6TLu4hwhHdiGJOgwcWD4AjKQx6zvFcGEsbfiY9-EI&r=lekNOOM5noV61zrPH3rwPyhtNnLLWoLEHgd0quQxly8&m=uqtx2oHwdMjoXvsVmB3c2yiP1jUG970YiY9D6u40vC4&s=PhRxFiNrvX2Ek5_CdsixJxYiH3alearJ9KKI-agaFlQ&e= 
            a=rtpmap:96 raw/90000
            a=fmtp:96 sampling=YCbCr-4:2:2; width=1280; height=720; depth=10
            a=mid:V2
            m=video 50010 RTP/AVP 97
            c=IN IP4 https://urldefense.proofpoint.com/v2/url?u=http-3A__233.252.0.2_255&d=DwICaQ&c=uw6TLu4hwhHdiGJOgwcWD4AjKQx6zvFcGEsbfiY9-EI&r=lekNOOM5noV61zrPH3rwPyhtNnLLWoLEHgd0quQxly8&m=uqtx2oHwdMjoXvsVmB3c2yiP1jUG970YiY9D6u40vC4&s=J4gusMHuMl8NrBgaiMZeu7a64DxgoTQHenMNItNU8cw&e= 
            a=rtpmap:97 smpte291/90000
            a=fmtp:97 DID_SDID={0x61,0x02};DID_SDID={0x41,0x05}
            a=mid:M1
            m=video 50012 RTP/AVP 97
            c=IN IP4 https://urldefense.proofpoint.com/v2/url?u=http-3A__233.252.0.2_255&d=DwICaQ&c=uw6TLu4hwhHdiGJOgwcWD4AjKQx6zvFcGEsbfiY9-EI&r=lekNOOM5noV61zrPH3rwPyhtNnLLWoLEHgd0quQxly8&m=uqtx2oHwdMjoXvsVmB3c2yiP1jUG970YiY9D6u40vC4&s=J4gusMHuMl8NrBgaiMZeu7a64DxgoTQHenMNItNU8cw&e= 
            a=rtpmap:97 smpte291/90000
            a=fmtp:97 DID_SDID={0x61,0x02};DID_SDID={0x41,0x05}
            a=mid:M2
    
    If the recipient of these mutually synchronized streams needs to reintegrate
    the ANC data into the video for offloading to an HDMI connection, how does it
    know which ANC stream to insert into which video stream?
    
    There needs to be some additional association mechanism here. You can either
    add an attribute to the smpte291 media sections that clearly indicates which
    video stream they correspond to, or you can include the smpte291 packets in the
    same stream as the corresponding video, like so:
    
            v=0
            o=Al 123456 11 IN IP4 https://urldefense.proofpoint.com/v2/url?u=http-3A__host.example.com&d=DwICaQ&c=uw6TLu4hwhHdiGJOgwcWD4AjKQx6zvFcGEsbfiY9-EI&r=lekNOOM5noV61zrPH3rwPyhtNnLLWoLEHgd0quQxly8&m=uqtx2oHwdMjoXvsVmB3c2yiP1jUG970YiY9D6u40vC4&s=HQ1haZcCzEc4KXzHJA54Qk3c-u2bkzVIMwUSCtAnmkU&e= 
            s=Professional Networked Media Test
            i=A test of synchronized video and ANC data
            t=0 0
            m=video 50000 RTP/AVP 96 97
            c=IN IP4 https://urldefense.proofpoint.com/v2/url?u=http-3A__233.252.0.1_255&d=DwICaQ&c=uw6TLu4hwhHdiGJOgwcWD4AjKQx6zvFcGEsbfiY9-EI&r=lekNOOM5noV61zrPH3rwPyhtNnLLWoLEHgd0quQxly8&m=uqtx2oHwdMjoXvsVmB3c2yiP1jUG970YiY9D6u40vC4&s=PhRxFiNrvX2Ek5_CdsixJxYiH3alearJ9KKI-agaFlQ&e= 
            a=rtpmap:96 raw/90000
            a=rtpmap:97 smpte291/90000
            a=fmtp:96 sampling=YCbCr-4:2:2; width=1280; height=720; depth=10
            a=fmtp:97 DID_SDID={0x61,0x02};DID_SDID={0x41,0x05}
            m=video 50002 RTP/AVP 96 97
            c=IN IP4 https://urldefense.proofpoint.com/v2/url?u=http-3A__233.252.0.1_255&d=DwICaQ&c=uw6TLu4hwhHdiGJOgwcWD4AjKQx6zvFcGEsbfiY9-EI&r=lekNOOM5noV61zrPH3rwPyhtNnLLWoLEHgd0quQxly8&m=uqtx2oHwdMjoXvsVmB3c2yiP1jUG970YiY9D6u40vC4&s=PhRxFiNrvX2Ek5_CdsixJxYiH3alearJ9KKI-agaFlQ&e= 
            a=rtpmap:96 raw/90000
            a=rtpmap:97 smpte291/90000
            a=fmtp:96 sampling=YCbCr-4:2:2; width=1280; height=720; depth=10
            a=fmtp:97 DID_SDID={0x61,0x02};DID_SDID={0x41,0x05}
    
    (See RFC4733 for an example of a payload that performs stream association in
    this fashion)

---

From: Thomas Edwards <[email-masked]>
Date: Thu, 21 Sep 2017 14:37:15 +0000

I am still trying to determine a good response to Adam Roach’s DISCUSS item regarding draft-ietf-payload-rtp-ancillary about relationship of streams in SDP.  But I can address the other comments now:

1) I can add some additional field values in the RTP payload example

2) I will add normative behavior for F field 0b01.

3) All SDI formats that I am currently aware of are based on multiple streams of 1080p.  However, as you suggest, I will add codes in Line_Number and Horizontal_Offset for “larger than can be represented” for any future formats that might be larger than 1080p.  We did find a corner case of 720p24 that has more luma sample numbers than 4096, but the consensus within knowledgeable broadcast engineers is that no one would use up all the 2840 words of HANC space available on a line in these formats, although it is worth a note that I will add.  Easy conversion to/from SMPTE ST 2038 is considered important, thus I am not in favor of enlarging the bit size of these fields as SMPTE ST 2038 has the same bit sizes.

4) Although I suspect typical use would be ANC data on its own multicast IP address, I see no reason to stop it from being on the same multicast IP address as related video.   I can add a statement explicitly saying they are both OK, and indicate that being in the same media section this can be used as an alternative to using LS groups.  However, the response to this may be affected by the solution to the DISCUSS item on stream relationships.

5) I can add offer rejections of setting the corresponding port number to 0 or removing the PT.

-Thomas

-- 
Thomas Edwards
FOX Networks Engineering & Operations
VP Engineering & Development
[email-masked]
+1-310-369-6696
10201 W Pico Blvd
Los Angeles, CA 90035

    COMMENT:
    
    It is common in examples of RTP payloads (such as Figure 1) to include values
    for those fields whose values are illustrative of the format itself. The "V=2"
    in Figure 1 is an example of this. I would suggest that the example would be
    slightly improved by including a similar notation for (e.g.) "Length = 32",
    "ANC_Count = 2", and the two Data_Count fields.
    
    Section 2.1 defines "b01" as "invalid" for the F field. It does not, however,
    prescribe what recipients of such a value should do; reasonable options would
    include (a) treat it as 0b00; (b) ignore the ANC but process any others that
    might be present; or (c) ignore the entire packet.  I'll note that (b) allows
    you to come back and add semantics for "0b01" at a later date, if you find a
    reason to do so. In any case, to minimize variations among recipients, I would
    suggest defining concrete behavior here.
    
    [Warning: I'm not terribly familiar with SMPTE video formats, and with the
    UHD-and-higher specs in particular, so it may be that the following comment can
    be answered with "that can never happen no matter how high-res streams get in
    the future."] As far as I know, SMPTE defines 4k as having 4 streams (each
    efectively a normal 1080 stream), 8k as having 16, and the future-looking 16k
    (presumably) having 64. Given that the Horizontal_Offset can communicate widths
    only up to 4093 samples (well within striking distance of the 2640 samples used
    by a single stream), and that the StreamNum field can communicate at most 128
    streams, it seems that any sizes larger than 16k run the risk of exceeding the
    ability for these fields to place ancillary data packets at the correct
    horizontal position. I would recommend defining another reserved value for
    Horizontal_Offset (0xFFD), that can be used to mean "longer field follows" in a
    future specification; and a reserved value (0x7F) for StreamNum for the same
    purpose. Line_Number, with a maximum value of 2045, is similarly within
    striking distance of the 1125 lines used by a single stream, and would likely
    need the same accommodations.
    
    It is implied, but not stated, that implementations should use different
    sessions (or at least different media sections) to convey smpte291 payloads
    than the video streams they correspond to. If this is the intention, please
    include a normative statement to that effect. If it is okay to include both
    payload types in the same media section, please add a statement explicitly
    saying so, and indicate that this can be used as an alternative to using LS
    groups.
    
    Section 5.1 indicates that the answerer can respond with "all or a subset" of
    the DID_SDID parameters, or that it "MAY reject the offer."  Of course, an
    answerer is always able to reject an offer, but this text seems to imply that
    it might do so if it doesn't want to receive any of the DID_SDID parameters
    indicated in the offer. That seems like massive overkill. Since the model here
    appears to be using multiple sessions (or, at least, separate media sections)
    to convey smpte291 data, I would offer that this should instead say something
    like "MAY set the corresponding port number to 0 to decline the smpte291
    stream". (If you opt to allow the smpte291 payloads to be in the same media
    section as their corresponding video streams, this would instead involve
    removing the corresponding PT).
    
    Nit: The final line in the final example of section 4 is too long. Consider
    outdenting by two characters.

---

From: "Ali C. Begen" <[email-masked]>
Date: Thu, 21 Sep 2017 18:00:30 +0300

Regarding the grouping, I understand Adam's concern on using LS semantics.
Maybe a good compromise would be to use FID semantics?
https://tools.ietf.org/html/rfc5888#section-8.4

It would be an overkill to propose a new grouping semantics for ANC. In
Adam's example there are two video (mcast 1) and two ANC flows (mcast 2).
How typical is this scenario is I am not sure (I'd expect each video and
ANC flow to be on a separate mcast session). Thomas may provide further
insight here. And if something is not (or should not be) supported, we can
just say that in the draft.

Managing complex grouping relations is a not very easy. RFC 4756 had to be
revised rather quickly once we had FEC framework in place.

-acbegen

On Thu, Sep 21, 2017 at 5:37 PM, Thomas Edwards <[email-masked]>
wrote:

> I am still trying to determine a good response to Adam Roach’s DISCUSS
> item regarding draft-ietf-payload-rtp-ancillary about relationship of
> streams in SDP.  But I can address the other comments now:
>
> 3) All SDI formats that I am currently aware of are based on multiple
> streams of 1080p.  However, as you suggest, I will add codes in Line_Number
> and Horizontal_Offset for “larger than can be represented” for any future
> formats that might be larger than 1080p.  We did find a corner case of
> 720p24 that has more luma sample numbers than 4096, but the consensus
> within knowledgeable broadcast engineers is that no one would use up all
> the 2840 words of HANC space available on a line in these formats, although
> it is worth a note that I will add.  Easy conversion to/from SMPTE ST 2038
> is considered important, thus I am not in favor of enlarging the bit size
> of these fields as SMPTE ST 2038 has the same bit sizes.
>
> 4) Although I suspect typical use would be ANC data on its own multicast
> IP address, I see no reason to stop it from being on the same multicast IP
> address as related video.   I can add a statement explicitly saying they
> are both OK, and indicate that being in the same media section this can be
> used as an alternative to using LS groups.  However, the response to this
> may be affected by the solution to the DISCUSS item on stream relationships.
>

---

From: Adam Roach <[email-masked]>
Date: Thu, 21 Sep 2017 10:22:09 -0500

> The challenge is that the use case of the Ancillary I-D within the SMPTE ST 2110 architecture would generally require separate network destination addresses for the different elements (video, audio…

With that requirement, I think Ali's suggestion is the right one: the 
grouping you want to use to bind these together is FID. I believe that 
simply replacing "LS" with "FID" in the document would be adequate 
(i.e., I don't think this document needs a treatment of LS). It is 
probably also worth mentioning in the document that the other approach 
(i.e., including both the smpte291 format and the main video codec 
format in the same session) is _valid_, but that using different IP 
addresses is frequently necessary, and FID is the mechanism used to do so.

Thanks!

/a

---

From: Adam Roach <[email-masked]>
Date: Thu, 21 Sep 2017 10:25:30 -0500

> 5) I can add offer rejections of setting the corresponding port number to 0 or removing the PT.

Thanks! This all looks good.

/a

---

From: Thomas Edwards <[email-masked]>
Date: Fri, 22 Sep 2017 14:48:27 +0000

Is there an understanding that FID grouped objects are to be presented in time synchronized fashion?

(It might not matter - implementations are likely to synchronize presentation of the FID grouped streamed anyway based on RTP timestamps and RFC 7273, RTP Clock Source Signalling).

Thanks!

-Thomas

On 9/21/17, 8:22 AM, "Adam Roach" <[email-masked]> wrote:

    On 9/20/17 08:31, Thomas Edwards wrote:
    > The challenge is that the use case of the Ancillary I-D within the SMPTE ST 2110 architecture would generally require separate network destination addresses for the different elements (video, audio, ancillary data) so they can be flexibly composed within broadcast plants at the network level.  Your alternative SDP example appears to place both the video and the ancillary data onto the same destination address (https://urldefense.proofpoint.com/v2/url?u=http-3A__233.252.0.1&d=DwIDaQ&c=uw6TLu4hwhHdiGJOgwcWD4AjKQx6zvFcGEsbfiY9-EI&r=lekNOOM5noV61zrPH3rwPyhtNnLLWoLEHgd0quQxly8&m=HESr2t9GphJ1DG-VC5BBE5bqCuFq085olInzqKtqp4k&s=AmiuURi1XkubNbBE2DmLed7zdEsZV6utcHjhHSXP6ls&e= ).  Is there an appropriate SDP mechanism to communicate the intimate relationship between the RTP streams, but also have them transmitted on separate destination IP addresses (and synchronized)?

    Thanks!
    
    /a

---

From: "Ali C. Begen" <[email-masked]>
Date: Fri, 22 Sep 2017 17:52:25 +0300

FID grouping does not imply synchronization. But obviously the media
sink for example can use RTP timestamps, sender reports, etc. to
achieve synchronization if needed.

---

From: Adam Roach <[email-masked]>
Date: Fri, 22 Sep 2017 10:47:18 -0500

> (It might not matter - implementations are likely to synchronize presentation of the FID grouped streamed anyway based on RTP timestamps and RFC 7273, RTP Clock Source Signalling).

Given the relationship between the kinds of data, I would think that 
synchronization would be implicit. I'm not opposed to saying that you 
put things in FID groups and LS groups, but it seems kind of like 
unnecessary noise.

/a

---

From: Thomas Edwards <[email-masked]>
Date: Wed, 27 Sep 2017 01:13:29 +0000

Regarding draft-ietf-payload-rtp-ancillary-10, on further thought I believe that FID grouping is NOT appropriate for Section 4.1 “Grouping ANC Streams with other Media Streams”.

RFC 5888 Section 8.4. (“FID Semantics”) states:

       “It is assumed that the application uses only one codec at a time to encode the media produced.  This codec MAY change dynamically during the session, but at any particular moment, only one codec is in use.”

RFC 5888 Section 8.5.1 states:

	“FID semantics are useful when the application only uses one codec at a time.  An application that encodes the same media using different codecs simultaneously MUST NOT use FID to group those media lines.”

When we have one video stream and one ancillary data stream, senders are not switching between codecs.  FID really sounds to me like a situation where you have one media stream, but senders could send that stream one of multiple codecs, but not at the same time.

Compare with RFC 5888 Section 7:

	“Note that LS semantics apply not only to a video stream that has to be synchronized with an audio stream; the playout of two streams of the same type can be synchronized as well.”

Synchronized presentation to the user of an audio, video, and of course ancillary data flow is generally what we need for production television workflows.

draft-ietf-payload-rtp-ancillary-10 Section 4.1 has the example:

        v=0
        o=Al 123456 11 IN IP4 host.example.com
        a=group:LS V1 M1
        c=IN IP4 233.252.0.1/255
        c=IN IP4 233.252.0.2/255

As some have pointed out, RTP timestamps and RTCP provide inherent synchronization between “m” lines, but of course as RFC 5888 Section 1 says without grouping: “When a session description contains more than one "m" line, SDP does not provide any means to express a particular relationship between two or more of them.”  “LS” grouping makes it pretty clear that these streams are intended for synchronized presentation.

An alternative to “LS” grouping is to drop all grouping in the example, and be unclear about what these “m” lines are doing together in the same file:

        v=0
        o=Al 123456 11 IN IP4 host.example.com
        c=IN IP4 233.252.0.1/255
        c=IN IP4 233.252.0.2/255

But I’m still more comfortable with keeping the “LS” grouping in the example unless there are objections.
 
This brings us to the main point of Adam Roach’s DISCUSS item, how do you provide multi-level hierarchical grouping in SDP.  Imagine a group of (a group of video and ancillary data) and (a group of video and ancillary data).  There actually is a case of this in production television, the multiviewer (e.g. see https://www.blackmagicdesign.com/products/multiview).

Unfortunately, I really can’t find any SDP mechanism to describe multi-level hierarchy.  RFC 4796 “The Session Description Protocol (SDP) Content Attribute” is about the best I can find, where synchronized multimedia (such as “slides”, “speaker”, “main”, etc.) is presented.  But this is a very limited set of values.  RFC 5583 “Signaling Media Decoding Dependency in the Session Description Protocol (SDP)” has various dependencies of streams, but this is for layered or multiple descriptive media coding.  Neither of these seems appropriate.

Perhaps this issue needs to be brought up in Multiparty Multimedia Session Control (mmusic) WG, but I’m not sure that it is wise to hold up the ANC RTP Payload over this.

Thanks everyone!

-Thomas

---

From: Roni Even <[email-masked]>
Date: Wed, 27 Sep 2017 11:07:01 +0000

Hi,
I agree that LS is better fit for ANC

As for Adam's example it is allowed by RFC5888 to specify
        a=group:LS V1 V2 M1 M2
        a=group:LS V1 M1 
        a=group:LS V2  M2
this is similar to RFC5956  which replaced RFC4756

this is not a clean way but  is allowed otherwise there is a need to specify some SDP attribute for specifying dependencies.

Roni Even

> -----Original Message-----
> From: Thomas Edwards [mailto:[email-masked]]
> Sent: יום ד 27 ספטמבר 2017 04:13
> To: Adam Roach; The IESG
> Cc: [email-masked]; [email-masked];
> [email-masked]; [email-masked]
> Subject: Re: Adam Roach's Discuss on draft-ietf-payload-rtp-ancillary-10:
> (with DISCUSS and COMMENT)
> 
> Regarding draft-ietf-payload-rtp-ancillary-10, on further thought I believe
> that FID grouping is NOT appropriate for Section 4.1 “Grouping ANC Streams
> with other Media Streams”.
> 
>        “It is assumed that the application uses only one codec at a time to
> encode the media produced.  This codec MAY change dynamically during the
> session, but at any particular moment, only one codec is in use.”
> 
> 	“FID semantics are useful when the application only uses one codec
> at a time.  An application that encodes the same media using different
> codecs simultaneously MUST NOT use FID to group those media lines.”
> 
> When we have one video stream and one ancillary data stream, senders are
> not switching between codecs.  FID really sounds to me like a situation where
> you have one media stream, but senders could send that stream one of
> multiple codecs, but not at the same time.
> 
> 	“Note that LS semantics apply not only to a video stream that has to
> be synchronized with an audio stream; the playout of two streams of the
> same type can be synchronized as well.”
> 
> Synchronized presentation to the user of an audio, video, and of course
> ancillary data flow is generally what we need for production television
> workflows.
> 
> As some have pointed out, RTP timestamps and RTCP provide inherent
> synchronization between “m” lines, but of course as RFC 5888 Section 1 says
> without grouping: “When a session description contains more than one "m"
> line, SDP does not provide any means to express a particular relationship
> between two or more of them.”  “LS” grouping makes it pretty clear that
> these streams are intended for synchronized presentation.
> 
> An alternative to “LS” grouping is to drop all grouping in the example, and be
> unclear about what these “m” lines are doing together in the same file:
> 
>         v=0
> This brings us to the main point of Adam Roach’s DISCUSS item, how do you
> provide multi-level hierarchical grouping in SDP.  Imagine a group of (a group
> of video and ancillary data) and (a group of video and ancillary data).  There
> actually is a case of this in production television, the multiviewer (e.g. see
> https://www.blackmagicdesign.com/products/multiview).
> 
> Unfortunately, I really can’t find any SDP mechanism to describe multi-level
> hierarchy.  RFC 4796 “The Session Description Protocol (SDP) Content
> Attribute” is about the best I can find, where synchronized multimedia (such
> as “slides”, “speaker”, “main”, etc.) is presented.  But this is a very limited set
> of values.  RFC 5583 “Signaling Media Decoding Dependency in the Session
> Description Protocol (SDP)” has various dependencies of streams, but this is
> for layered or multiple descriptive media coding.  Neither of these seems
> appropriate.
> 
> Perhaps this issue needs to be brought up in Multiparty Multimedia Session
> Control (mmusic) WG, but I’m not sure that it is wise to hold up the ANC RTP
> Payload over this.
> 
> Thanks everyone!
> 
> -Thomas
> 
> --
> Thomas Edwards

---

From: Thomas Edwards <[email-masked]>
Date: Wed, 27 Sep 2017 15:47:09 +0000

Roni, thanks for the feedback on LS!

I’m still not sure if it is appropriate to include the hierarchical grouping SDP example in draft-ietf-payload-rtp-ancillary.  I suspect it could be confusing.

However, I will note that the Alliance for IP Media Solutions (http://aimsalliance.org) Technical Committee is considering work on documenting different SDP groupings for live television production scenarios, so it is possible that a future I-D will be submitted that could include a range of SDP applications in this area, perhaps including documentation of Adam Roach’s scenario.

I can go ahead and upload a draft-ietf-payload-rtp-ancillary-11 based on the IESG feedback.

-Thomas

On 9/27/17, 4:07 AM, "Roni Even" <[email-masked]> wrote:

    Hi,

    
    
            a=group:LS V1 V2 M1 M2
    
            a=group:LS V1 M1 
    
            a=group:LS V2  M2

    

    Roni Even

    

    

    

    

    

    

    

    

    

    

    

    

    

    

    
    >         o=Al 123456 11 IN IP4 https://urldefense.proofpoint.com/v2/url?u=http-3A__host.example.com&d=DwIGaQ&c=uw6TLu4hwhHdiGJOgwcWD4AjKQx6zvFcGEsbfiY9-EI&r=lekNOOM5noV61zrPH3rwPyhtNnLLWoLEHgd0quQxly8&m=mmtyLXtyaWHEg3JYcq3xSVLXzRqkIVgWAJDgqyIgreU&s=UuOwClXoGCvGP2qksiV3h85HNgQRo_Z7b0HgU2qVwzQ&e=

    

    
    >         c=IN IP4 https://urldefense.proofpoint.com/v2/url?u=http-3A__233.252.0.1_255&d=DwIGaQ&c=uw6TLu4hwhHdiGJOgwcWD4AjKQx6zvFcGEsbfiY9-EI&r=lekNOOM5noV61zrPH3rwPyhtNnLLWoLEHgd0quQxly8&m=mmtyLXtyaWHEg3JYcq3xSVLXzRqkIVgWAJDgqyIgreU&s=mDjccdFR8MWnfQcP_xiMhb-bh-Wef_2tKhRDyRilYUM&e=

    

    >         c=IN IP4 https://urldefense.proofpoint.com/v2/url?u=http-3A__233.252.0.2_255&d=DwIGaQ&c=uw6TLu4hwhHdiGJOgwcWD4AjKQx6zvFcGEsbfiY9-EI&r=lekNOOM5noV61zrPH3rwPyhtNnLLWoLEHgd0quQxly8&m=mmtyLXtyaWHEg3JYcq3xSVLXzRqkIVgWAJDgqyIgreU&s=7e66ewQVAuL1YUq2FzASyEJpqw7Tc5p8fO8lZbZ4tUo&e=

    

    

    

    

    
    
    >         o=Al 123456 11 IN IP4 https://urldefense.proofpoint.com/v2/url?u=http-3A__host.example.com&d=DwIGaQ&c=uw6TLu4hwhHdiGJOgwcWD4AjKQx6zvFcGEsbfiY9-EI&r=lekNOOM5noV61zrPH3rwPyhtNnLLWoLEHgd0quQxly8&m=mmtyLXtyaWHEg3JYcq3xSVLXzRqkIVgWAJDgqyIgreU&s=UuOwClXoGCvGP2qksiV3h85HNgQRo_Z7b0HgU2qVwzQ&e=

    

    >         c=IN IP4 https://urldefense.proofpoint.com/v2/url?u=http-3A__233.252.0.1_255&d=DwIGaQ&c=uw6TLu4hwhHdiGJOgwcWD4AjKQx6zvFcGEsbfiY9-EI&r=lekNOOM5noV61zrPH3rwPyhtNnLLWoLEHgd0quQxly8&m=mmtyLXtyaWHEg3JYcq3xSVLXzRqkIVgWAJDgqyIgreU&s=mDjccdFR8MWnfQcP_xiMhb-bh-Wef_2tKhRDyRilYUM&e=

    
    
    >         c=IN IP4 https://urldefense.proofpoint.com/v2/url?u=http-3A__233.252.0.2_255&d=DwIGaQ&c=uw6TLu4hwhHdiGJOgwcWD4AjKQx6zvFcGEsbfiY9-EI&r=lekNOOM5noV61zrPH3rwPyhtNnLLWoLEHgd0quQxly8&m=mmtyLXtyaWHEg3JYcq3xSVLXzRqkIVgWAJDgqyIgreU&s=7e66ewQVAuL1YUq2FzASyEJpqw7Tc5p8fO8lZbZ4tUo&e=

    
    
    > But I’m still more comfortable with keeping the “LS” grouping in the example
    
    > unless there are objections.

    

    
    > https://urldefense.proofpoint.com/v2/url?u=https-3A__www.blackmagicdesign.com_products_multiview&d=DwIGaQ&c=uw6TLu4hwhHdiGJOgwcWD4AjKQx6zvFcGEsbfiY9-EI&r=lekNOOM5noV61zrPH3rwPyhtNnLLWoLEHgd0quQxly8&m=mmtyLXtyaWHEg3JYcq3xSVLXzRqkIVgWAJDgqyIgreU&s=IdKozSZWbohK0Fg_CvZcmeNO6JxjGZvwknrCRLaQVX8&e=).

---

From: Adam Roach <[email-masked]>
Date: Fri, 29 Sep 2017 16:00:02 -0500

> When we have one video stream and one ancillary data stream, senders are not switching between codecs.  FID really sounds to me like a situation where you have one media stream, but senders could s…

In this case, where you have data and meta-data, it's arguably two 
different codecs; and, since they don't represent the same data, you're 
encoding with one codec or the other, based on whether you're sending 
video data or meta-data. From that perspective, I don't see the use of 
FID as being outside the spirit of what is intended.

That said, if the WG agrees with your evaluation that FID isn't quite 
the right fit, I'm happy for y'all to come up with some alternate 
solution...

> But I’m still more comfortable with keeping the “LS” grouping in the example unless there are objections.

...as long as that solution isn't misusing LS. there's really no way to 
construe LS to mean the kind of grouping you intend here. If you don't 
want to use FID, I'd propose that you could add a new metadata group 
type (a=group:META video ancillary) that means exactly what you intend here.

Importantly, I think that "dropping all grouping" would not be okay.

> This brings us to the main point of Adam Roach’s DISCUSS item, how do you provide multi-level hierarchical grouping in SDP.  Imagine a group of (a group of video and ancillary data) and (a group of…

Under what I propose, this would be:

a=group:LS video1 video2
a=group:FID video1 ancillary1
a=group:FID video2 ancillary2

Or, if you want to use a new META instead of re-using FID:

a=group:LS video1 video2
a=group:META video1 ancillary1
a=group:META video2 ancillary2

> Perhaps this issue needs to be brought up in Multiparty Multimedia Session Control (mmusic) WG, but I’m not sure that it is wise to hold up the ANC RTP Payload over this.

Let's see if we can find a straightforward path to completion based on 
what I say above.

/a

---

From: Roni Even <[email-masked]>
Date: Sun, 1 Oct 2017 06:39:11 +0000

Hi Adam,

This is the definition of LS from RFC5888 section 7

Lip Synchronization (LS)

   An application that receives a session description that contains "m"
   lines that are grouped together using LS semantics MUST synchronize
   the playout of the corresponding media streams.  Note that LS
   semantics apply not only to a video stream that has to be
   synchronized with an audio stream; the playout of two streams of the
   same type can be synchronized as well.

   For RTP streams, synchronization is typically performed using the RTP
   Control Protocol (RTCP), which provides enough information to map
   time stamps from the different streams into a local absolute time
   value.  However, the concept of media stream synchronization MAY also
   apply to media streams that do not make use of RTP.  If this is the
   case, the application MUST recover the original timing relationship
   between the streams using whatever mechanism is available.

So I am not sure why LS is not right for the ANC video data which that must be synchronized with the main video?

Roni

> From: payload [mailto:[email-masked]] On Behalf Of Adam Roach
> Sent: שבת 30 ספטמבר 2017 00:00
> To: Thomas Edwards; The IESG
> Subject: Re: [payload] Adam Roach's Discuss on draft-ietf-payload-rtp-
> ancillary-10: (with DISCUSS and COMMENT)
> 
> > When we have one video stream and one ancillary data stream, senders
> are not switching between codecs.  FID really sounds to me like a situation
> where you have one media stream, but senders could send that stream one
> of multiple codecs, but not at the same time.
> 
> In this case, where you have data and meta-data, it's arguably two different
> codecs; and, since they don't represent the same data, you're encoding with
> one codec or the other, based on whether you're sending video data or
> meta-data. From that perspective, I don't see the use of FID as being outside
> the spirit of what is intended.
> 
> That said, if the WG agrees with your evaluation that FID isn't quite the right
> fit, I'm happy for y'all to come up with some alternate solution...
> 
> 
> > An alternative to “LS” grouping is to drop all grouping in the example, and
> be unclear about what these “m” lines are doing together in the same file:
> >
> >          v=0
> ...as long as that solution isn't misusing LS. there's really no way to construe
> LS to mean the kind of grouping you intend here. If you don't want to use
> FID, I'd propose that you could add a new metadata group type
> (a=group:META video ancillary) that means exactly what you intend here.
> 
> Let's see if we can find a straightforward path to completion based on what I
> say above.
> 
> /a
> 
> _______________________________________________
> payload mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/payload