Subject: Re: [rtcweb] Performance of H.264...

From: Bernard Aboba <[email-masked]>
Date: Wed, 20 Nov 2013 09:52:04 -0800

Sorry to interrupt the ongoing codec rant-o-rama, but some may find the following pre-print paper of interest: http://iphome.hhi.de/marpe/download/Performance_HEVC_VP9_X264_PCS_2013_preprint.pdf

---

From: Maik Merten <[email-masked]>
Date: Wed, 20 Nov 2013 20:08:12 +0100

Am 20.11.2013 18:52, schrieb Bernard Aboba:
> Sorry to interrupt the ongoing codec rant-o-rama, but some may find the
> following pre-print paper of interest:
> http://iphome.hhi.de/marpe/download/Performance_HEVC_VP9_X264_PCS_2013_preprint.pdf

There's much to be said about PSNR as means for comparing image quality. 
There's also much to be said about PSNR as means for conducting 
cross-codec quality assessments.

And little of it is pretty.

I always chuckle when I see x264 being given the "--tune psnr" 
parameter. Even core developers of x264 have strong opinions on this: 
http://x264dev.multimedia.cx/archives/458

Maik

---

From: Bernard Aboba <[email-masked]>
Date: Wed, 20 Nov 2013 11:23:33 -0800

> There's much to be said about PSNR as means for comparing image quality. 
[BA] Section III A states that the comparison really doesn't relate to interactive/low delay uses (e.g. use of B frames, etc.) which is what we'd care about on this mailing list.   But it's the first study of its kind so it seemed worth posting a link to (in hopes that someone else might have a link to something more relevant).

---

From: John Leslie <[email-masked]>
Date: Wed, 20 Nov 2013 14:25:50 -0500

> And little of it is pretty.

   I don't think we need to pick on the authors here -- they were quite
clear about what they were doing.

   IMHO, the point to take away from the paper is that _neither_ H.264
nor VP8 are considered "current" first choices.

   The paper basically showed _by_how_much_ H.264 falls short of current
state-of-the-art.

> http://x264dev.multimedia.cx/archives/458

   Fun reading...

   But we _don't_ need any more lengthy criticisms of comparisons right
now. We _aren't_ going to drop everything and switch to H.265 as MTI --
or any other state-of-the-art codec.

   I fully agree that both H.264 and VP8 deserve SHOULD status; and
I agree H.261, being good enough for sign-language reading, looks like
the right fallback.

   H.261 is certainly easy enough to "implement" and deploy; and I'll
bet 80%  of us are ready for the question...

--
John Leslie <[email-masked]>

---

From: Stephan Wenger <[email-masked]>
Date: Wed, 20 Nov 2013 19:40:14 +0000

From: Bernard Aboba <[email-masked]<mailto:[email-masked]>>
Date: Wednesday, 20 November, 2013 at 14:23
To: Maik Merten <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [rtcweb] Performance of H.264...

>[BA] Section III A states that the comparison really doesn't relate to interactive/low delay uses (e.g. use of B frames, etc.) which is what we'd care about on this mailing list.   But it's the firs…

Ask, and it shall be given you; seek, and ye shall find...

See here: https://skydrive.live.com/view.aspx?resid=A6F8E452A49F023D!769&app=WordPdf&authkey=!AP15CciU4zPcTHc

This paper also compares codec operation points relevant for entertainment, and does not focus on video conferencing (without I pictures).  However, since it contains data for both random access GOP structures and pure intra, one could extrapolate pure predicted picture performance and arrive some conclusion (which is left as an exercise for the reader :-)
Note that the use of B pictures is uncorrelated with the use in low delay environments, and has been since at least 2003.  (In H.264 and later standards, both predictors of B pictures can point to the past.)  The use of B pictures nowadays is almost exclusively a complexity question.

Stephan

---

From: Monty Montgomery <[email-masked]>
Date: Wed, 20 Nov 2013 14:42:23 -0500

>  I don't think we need to pick on the authors here --

Yeah, actually we do.

> they were quite

They were clear about what, 'why they think that's meaningful' is
considerably less clear.

>    The paper basically showed _by_how_much_ H.264 falls short of current

No, PSNR is by itself a completely worthless metric for comparing two
different codecs.  Not 'lacking' but 'worthless'.

Monty

---

From: Thomas Reisinger <[email-masked]>
Date: Wed, 20 Nov 2013 23:53:21 +0200

From my opinion we should stick with h.263 even the minimal risk or licences for the implementer. H261 is a no go for me.

Thomas Reisinger

> On 20.11.2013, at 21:25, John Leslie <[email-masked]> wrote:
>

---

From: Basil Mohamed Gohar <[email-masked]>
Date: Wed, 20 Nov 2013 16:57:19 -0500

> From my opinion we should stick with h.263 even the minimal risk or licences for the implementer. H261 is a no go for me.

Thomas,

The point of choosing something aside from VP8 and H.264, which are both
superior to H.263, is to *completely* avoid all raised and known
licensing concerns in the mandatory part of the spec.

This is where H.261 has any value at all in this discussion, even more
so than any alleged quality tweaks and performance.  It is because any
rtcweb implementation can include a standards-compliant H.261 en/decoder
without worrying at all about patent risks.

If we're willing to accept even a small degree of patent risk, then that
opens up the door to formats far superior to H.261 and even H.263,
several of which have already been presented.

-- 
Libre Video
http://librevideo.org

---

From: Stephan Wenger <[email-masked]>
Date: Wed, 20 Nov 2013 22:14:07 +0000

Hi,
I believe you can indeed include an H.261 *de*coder without running
unmanageable risk, for pretty much every value of unmanageable.  The
*en*coder is much trickier, as you would have to ensure that your
non-standardized mechanisms are also not covered by enforceable patents.
H.261 *En*coders have never been standardized.
You could in theory implement, without deviation, RM8 (the final H.261
test model), published around the same time as H.261.
Stephan

On 11.20.2013, 16:57 , "Basil Mohamed Gohar" <[email-masked]>
wrote:

>> From my opinion we should stick with h.263 even the minimal risk or
>>licences for the implementer. H261 is a no go for me.
>> 
>> Thomas Reisinger
>
>Thomas,
>
>_______________________________________________
>rtcweb mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Harald Alvestrand <[email-masked]>
Date: Thu, 21 Nov 2013 11:19:47 +0100

On 11/20/2013 06:52 PM, Bernard Aboba wrote:
> Sorry to interrupt the ongoing codec rant-o-rama, but some may find 
> the following pre-print paper of interest:

Yes, locking the QP on VP9 (and VP8) harms performance.