Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter

From: "Wang, Ye-Kui" <[email-masked]>
Date: Fri, 11 Apr 2014 20:17:26 +0000

We should also try to conclude herein.

Tom, do you have any response to Jonatan's comment below?

BR, YK

From: payload [mailto:[email-masked]] On Behalf Of Jonatan Samuelsson
Sent: Friday, March 07, 2014 12:20 AM
To: Tom Kristensen; Wang, Ye-Kui
Cc: Tom Kristensen ([email-masked]); [email-masked]
Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter

Thanks Tom for your feedback,

For me to get a better understanding of if it is reasonable to make an exception from the design principle of not re-defining level definitions from the video specification, it would be very valuable with answers to my earlier question:

"If the motivation for max-fps is only for SDP O/A, why isn't the SDP attribute a=framerate sufficient?"

and to Magnus' earlier question:

"The issues with the media pipeline has they been with the reception of the high framerate of packets and forwarding them to the decoder or the playback part, i.e. handling of the decoded picture?"

Best Regards Jonatan

From: Tom Kristensen [mailto:[email-masked]]
Sent: den 7 mars 2014 09:00
To: Wang, Ye-Kui
Cc: Jonatan Samuelsson; Stephan Wenger; Magnus Westerlund; Geir Sandbakken (geirsand); [email-masked]<mailto:[email-masked]>; Tom Kristensen ([email-masked]<mailto:[email-masked]>)
Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter

Well, I cannot see a reason for using SHOULD in offer/answer scenarios - as Stephan argued for earlier here. If there is a limit and a reason for signalling max-fps due, there is little use waiving the flag with a SHOULD (which is often ignored, as we know).

I think we should rather remove max-fps for declarative usage (or relax the MUST to SHOULD there).

-- Tom

On 4 March 2014 18:16, Wang, Ye-Kui <[email-masked]<mailto:[email-masked]>> wrote:
Let me pull Tom into the discussion :-) Personally I am OK with the SHOULD language here too as I don't have a good counter argument against Jonatan's comment.

Tom, would the SHOULD language be fine with you as well (as the proponent of this parameter)? Do you see any technical problem with this?

BR, YK

-----Original Message-----
From: Jonatan Samuelsson [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Tuesday, March 04, 2014 2:43 AM
To: Stephan Wenger; Wang, Ye-Kui; Magnus Westerlund; Geir Sandbakken (geirsand); [email-masked]<mailto:[email-masked]>
Subject: RE: [payload] #12 (rtp-h265): The max-fps parameter

As far as I know, all other similar parameters in H.264 RTP PL and H.265 RTP PL extends on the levels defined in the video specification in the sense that you are capable of expressing for example "I can decode level 4.1 plus resolutions up to 4K" which can never put a level 4.1 compliant encoder in a difficult situation (i.e. everything in level 4.1 is still allowed).

If the motivation for max-fps is only for SDP O/A, why isn't the SDP attribute a=framerate sufficient?

Best Regards Jonatan

-----Original Message-----
From: Stephan Wenger [mailto:[email-masked]<mailto:[email-masked]>]
Sent: den 4 mars 2014 10:48
To: Jonatan Samuelsson; Wang, Ye-Kui; Magnus Westerlund; Geir Sandbakken (geirsand); [email-masked]<mailto:[email-masked]>
Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter

Let me push back once more.  Overwriting the limits of the level definition is common practice in the video conferencing industry, and has been done since staid industry changed to H.264 (which first introduced the level concepts in video compression standards widely deployed in video conferencing).  That SHOULD will be ignored, or overwritten by system standards.
Is there any technical reason for not allowing overwriting level limitations in scenarios where there is two way negotiation?
(I would be fine with a SHOULD, or even the removal of max_fps entirely, in SDP declarative use scenarios.  I only care about O/A scenarios.) Stephan

On 4.3.14, 9:41, "Jonatan Samuelsson" <[email-masked]<mailto:[email-masked]>>
wrote:

>Thanks Ye-Kui for the suggested changes. I think they improve the
>situation to a large extent, but there is one more thing that I would
>like change; replacing the "MUST" with a "SHOULD" so that it becomes:
>
>"The encoder SHOULD use a picture rate equal to or less than this value.
>In cases where the max-fps parameter is absent the encoder is free to
>choose any picture rate according to the highest level and any signaled
>optional parameters."
>
>I think this change would be needed in order to not redefine the level
>definitions of the HEVC specification. A sender that for some reason is
>not capable of producing lower picture rate (e.g. since the material is
>pre-encoded) should be allowed to use a picture rate higher than the
>value of max-fps, but it must then be aware that the receiver might not
>be capable of adequately present all decoded pictures.
>
>Best Regards Jonatan
>
>-----Original Message-----
>From: payload [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Wang,
>Ye-Kui
>Sent: den 27 februari 2014 19:20
>To: Magnus Westerlund; Geir Sandbakken (geirsand); [email-masked]<mailto:[email-masked]>
>Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter
>
>I am with Magnus here. If a decoder cannot DECODE a picture rate
>allowed by a level, then the decoder is not even a confirming decoder.
>We usually don't standardize mechanisms for non-confirming decoders,
>unless there is a very strong reason.
>
>I think making the following changes (4 places in the first paragraph
>of the semantics) would resolve the issue, with conceptually good
>semantics, but the use of the parameter is still practically the same.
>
>-----------------Start of the semantics, with suggested
>changes--------------------------
>max-fps:
>The value of max-fps is an integer indicating the maximum picture rate
>in units of hundreds of pictures per second that can be efficiently
>received (***CHANGE TO "effectively processed by the receiver"***).
>The max-fps parameter MAY be used to signal that the receiver has a
>constraint in that it is not capable of decoding (***CHANGE TO
>"processing"***) video efficiently (***CHANGE TO  "effectively"***) at
>the full picture rate that is implied by the highest level and, when
>present, one or more of the parameters max-lsr, max-lps, and max-br.
>
>The value of max-fps is not necessarily the picture rate at which the
>maximum picture size can be sent, it constitutes a constraint on
>maximum picture rate for all resolutions.
>
>Informative note: The max-fps parameter is semantically different from
>max-lsr, max-lps, max-cpb, max-dpb, max-br, max-tr, and max-tc in that
>max-fps is used to signal a constraint, lowering the maximum picture
>rate from what is implied by other parameters.
>
>The encoder MUST use a picture rate equal to or less than this value.
>In cases where the max-fps parameter is absent the encoder is free to
>choose any picture rate according to the highest level and any signaled
>optional parameters.
>-----------------End of the semantics, with suggested
>changes--------------------------
>
>Please express your opinion if this is not acceptable to you.
>
>BR, YK
>
>
>-----Original Message-----
>From: payload [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Magnus
>Westerlund
>Sent: Thursday, February 27, 2014 7:32 AM
>To: Geir Sandbakken (geirsand); [email-masked]<mailto:[email-masked]>
>Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter
>
>On 2014-02-27 11:18, Geir Sandbakken (geirsand) wrote:
>> #12: The max-fps parameter
>>
>> Lessons learned from video conferencing using H.264 showed that a lot
>>of equipment designed for 30 fps could not handle 60 fps even at
>>sufficiently low resolution. We do anticipate similar problems when
>>moving towards 120 fps in H.265.  It might not be caused by
>>limitations in the decoder itself,  but that the surrounding media
>>pipeline will not have been properly tested for 120 fps before being
>>put into the wild.
>> If we don't allow for a code point to be used by the "restricted"
>> equipment, it will be required by the "flexible/proper" to include one.
>> This is what happened with max-fps in H.264 enabling higher frame
>>rates compared to lowering them.
>>
>
>I do have concerns with defining a parameter that allow dynamically
>sub-setting the profile and levels that has been defined for HEVC. This
>can also create interoperability issues with any pre-encoded content to
>a particular profile and level.
>
>The issues with the media pipeline has they been with the reception of
>the high framerate of packets and forwarding them to the decoder or the
>playback part, i.e. handling of the decoded picture?
>
>If it is predominately the later it appears very reasonable to have
>this as an informative parameter. I will not make use of higher FPS
>than X, if you send me higher than X I will need to reduce that to X or
>lower before displaying it. That way we are not re-defining the
>profiles, we are ensuring that we can handle content encoded within the
>profile and still by taking care of the this in the end of your decoder
>chain you can protect the rest of the media framework from not yet
>supported frame-rates.
>
>Cheers
>
>Magnus Westerlund
>
>----------------------------------------------------------------------
>Services, Media and Network features, Ericsson Research EAB/TXM
>----------------------------------------------------------------------
>Ericsson AB                 | Phone  +46 10 7148287
>Färögatan 6                 | Mobile +46 73 0949079
>SE-164 80 Stockholm, Sweden | mailto: [email-masked]<mailto:[email-masked]>
>----------------------------------------------------------------------
>
>_______________________________________________
>payload mailing list
>[email-masked]<mailto:[email-masked]>
>https://www.ietf.org/mailman/listinfo/payload
>
>_______________________________________________
>payload mailing list
>[email-masked]<mailto:[email-masked]>
>https://www.ietf.org/mailman/listinfo/payload
>
>_______________________________________________
>payload mailing list
>[email-masked]<mailto:[email-masked]>
>https://www.ietf.org/mailman/listinfo/payload

_______________________________________________
payload mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/payload

--
# Cisco                         |  http://www.cisco.com/telepresence/
## [email-masked]<mailto:[email-masked]>  |  http://www.tandberg.com
###                               |  http://folk.uio.no/tomkri/

---

From: "Wang, Ye-Kui" <[email-masked]>
Date: Wed, 23 Apr 2014 21:00:42 +0000

We need to proceed. In case there is no more input, I suggest that we go ahead to change the semantics as follows:

max-fps:
The value of max-fps is an integer indicating the maximum picture rate in units of hundreds of pictures per second that can be efficiently received effectively processed by the receiver.  The max-fps parameter MAY be used to signal that the receiver has a constraint in that it is not capable of decoding video efficiently processing video effectively at the full picture rate that is implied by the highest level and, when present, one or more of the parameters max-lsr, max-lps, and max-br.

The value of max-fps is not necessarily the picture rate at which the maximum picture size can be sent, it constitutes a constraint on maximum picture rate for all resolutions.

Informative note: The max-fps parameter is semantically different from max-lsr, max-lps, max-cpb, max-dpb, max-br, max-tr, and max-tc in that max-fps is used to signal a constraint, lowering the maximum picture rate from what is implied by other parameters.

The encoder MUST SHOULD use a picture rate equal to or less than this value.  An exception is when sending a pre-encoded bitstream the picture rate may be greater than the value of max-fps.  In cases where the max-fps parameter is absent the encoder is free to choose any picture rate according to the highest level and any signaled optional parameters.

The value of max-fps MUST be smaller than or equal to the full picture rate that is implied by the highest level and, when present, one or more of the parameters max-lsr, max-lps, and max-br. [Note that addition of this sentence is to address the Magnus' comment on a value range for each parameter.]

Jonatan, please suggest if you have a better exception case in mind (note that nowadays generally an exception case needs to be clarified for each SHOULD wording).

BR, YK

From: payload [mailto:[email-masked]] On Behalf Of Wang, Ye-Kui
Sent: Friday, April 11, 2014 1:17 PM
To: Jonatan Samuelsson; Tom Kristensen
Cc: [email-masked]; Tom Kristensen ([email-masked])
Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter

BR, YK

Cc: Tom Kristensen ([email-masked]<mailto:[email-masked]>); [email-masked]<mailto:[email-masked]>

Best Regards Jonatan

-- Tom

BR, YK

-----Original Message-----
From: Jonatan Samuelsson [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Tuesday, March 04, 2014 2:43 AM
To: Stephan Wenger; Wang, Ye-Kui; Magnus Westerlund; Geir Sandbakken (geirsand); [email-masked]<mailto:[email-masked]>
Subject: RE: [payload] #12 (rtp-h265): The max-fps parameter

Best Regards Jonatan

---

From: Jonatan Samuelsson <[email-masked]>
Date: Thu, 24 Apr 2014 06:41:15 +0000

Thanks Ye-Kui,

Your proposed text looks good. Perhaps the exception sentence can be modified slightly: An exception is when sending a pre-encoded bitstream, in which case the picture rate may be greater than the value of max-fps.

When reading this section again another question comes to my mind regarding "The value of max-fps is an integer indicating the maximum picture rate in units of hundreds of pictures per second". It seems to me that this could be interpreted as the value 1 means "one hundred pictures per second", the value 2 means "two hundred pictures per second" etc. I'm sure this is not the intention. Wouldn't it be better to just write: "The value of max-fps is an integer indicating the maximum picture rate in units of pictures per second" so that the value 1 means "one picture per second", the value 2 means "two pictures per second" etc.

(Or maybe the intention was to say "in units of pictures per hundred seconds" in order to be able to indicate fractional frame rates such as 29.97 fps by sending the value 2997?)

Best Regards Jonatan

From: Wang, Ye-Kui [mailto:[email-masked]]
Sent: den 23 april 2014 23:01
To: Jonatan Samuelsson; Tom Kristensen
Cc: [email-masked]; Tom Kristensen ([email-masked])
Subject: RE: [payload] #12 (rtp-h265): The max-fps parameter

max-fps:

BR, YK

Cc: [email-masked]<mailto:[email-masked]>; Tom Kristensen ([email-masked]<mailto:[email-masked]>)

BR, YK

Best Regards Jonatan

-- Tom

BR, YK

-----Original Message-----
From: Jonatan Samuelsson [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Tuesday, March 04, 2014 2:43 AM
To: Stephan Wenger; Wang, Ye-Kui; Magnus Westerlund; Geir Sandbakken (geirsand); [email-masked]<mailto:[email-masked]>
Subject: RE: [payload] #12 (rtp-h265): The max-fps parameter

Best Regards Jonatan

---

From: "Wang, Ye-Kui" <[email-masked]>
Date: Thu, 24 Apr 2014 22:13:00 +0000

Thanks Jonatan. The change you suggested is good to me.

I believe the intention of the unit is "in units of pictures per hundred seconds", exactly to enable the representation of 29.97 fps by sending the value 2997, and so on. Good catch. I will make this change too.

BR, YK

From: Jonatan Samuelsson [mailto:[email-masked]]
Sent: Wednesday, April 23, 2014 11:41 PM
To: Wang, Ye-Kui; Tom Kristensen
Cc: [email-masked]; Tom Kristensen ([email-masked])
Subject: RE: [payload] #12 (rtp-h265): The max-fps parameter

Thanks Ye-Kui,

Best Regards Jonatan

max-fps:

BR, YK

BR, YK

Best Regards Jonatan

-- Tom

BR, YK

-----Original Message-----
From: Jonatan Samuelsson [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Tuesday, March 04, 2014 2:43 AM
To: Stephan Wenger; Wang, Ye-Kui; Magnus Westerlund; Geir Sandbakken (geirsand); [email-masked]<mailto:[email-masked]>
Subject: RE: [payload] #12 (rtp-h265): The max-fps parameter

Best Regards Jonatan