Subject: [rtcweb] Cross-check of Google VP8 vs H.264 comparison

From: Bo Burman <[email-masked]>
Date: Thu, 14 Mar 2013 18:13:52 +0000

Hi,

As said on the microphone today, we have cross-checked Google's VP8 vs H.264 comparison. Google reported bit rate gains for VP8 of 19%. However, the H.264 was at an unfair advantage due to the fact that the --tune psnr flag was not used in x264. This should of course be used when doing psnr-measurements. For vp8, this flag is set to psnr by default. 

When re-running the tests with --tune psnr for x264, and using version 130 of x264 instead of version 128 that was used in the Google test, the difference disappeared (1% difference). 

Cheers,
Bo

---

From: Sergio Garcia Murillo <[email-masked]>
Date: Thu, 14 Mar 2013 21:43:41 +0100

Hi,

I also cross-posted the x264 encoding parameters used in the comparison 
in the x264-devel list:

http://mailman.videolan.org/pipermail/x264-devel/2013-March/009920.html

>/  What would be the best encoding parameters for x264 in these scenario?
/
1.  Set the slowest preset you're willing to tolerate.
2.  Set --tune to psnr or ssim (based on the one you're testing).
3.  Since they're doing CBR streaming for WebRTC, they need to set
appropriate ratecontrol settings.  Some examples:
1 second of latency: --vbv-maxrate $b --vbv-bufsize $b
0.25 seconds of latency: --vbv-maxrate $b --vbv-bufsize $b/4
low latency streaming with capped frame size: --vbv-maxrate $b
--vbv-bufsize $b/fps

In addition, if they care about latency, they'd want to make sure both
encoders are set with appropriate low-latency settings.  They'd also
need to verify how well both encoders are abiding by these

Best regards
Sergio

El 14/03/2013 19:13, Bo Burman escribió:
> _______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Harald Alvestrand <[email-masked]>
Date: Fri, 15 Mar 2013 11:19:11 +0100

Bo, did you run tests on our clips or your own clips, and with the rest of the settings our way or your way? Clip selection seems to matter a lot - which is why we published ours.

Bo Burman <[email-masked]> wrote:

>
>Hi,
>
>As said on the microphone today, we have cross-checked Google's VP8 vs
>H.264 comparison. Google reported bit rate gains for VP8 of 19%.
>However, the H.264 was at an unfair advantage due to the fact that the
>--tune psnr flag was not used in x264. This should of course be used
>when doing psnr-measurements. For vp8, this flag is set to psnr by
>default. 
>
>When re-running the tests with --tune psnr for x264, and using version
>130 of x264 instead of version 128 that was used in the Google test,
>the difference disappeared (1% difference). 
>
>Cheers,
>Bo

-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.

---

From: 邓灵莉/Lingli Deng <[email-masked]>
Date: Fri, 15 Mar 2013 20:55:17 +0800

Hi Bo and Harald,

To be honest, from the perspective of a user, I really don’t see real difference between these two or the reason to do further quality tests. (PSNR is not the best choice to do objective tests, let alone objective tests are not to be replaced by subjective tests.)

I am not a codec expert either. But I have both seen cool face-time and bad voip apps using software H.264, as well as really cool RCS app demos using VP8 (not as bad as yesterday’s show). And I am sure Google would cover it up for mobile devices should VP8 was chose to be MTI.

I voted for H.264 yesterday. And here are my thoughts:

First of all, I would definitely not go with no MTI option. Since we have seen real-time transcoding simply for audio calls can eat up to 75% throughput of an interop gateway in our network. No one would expect anything better for high-quality video transcoding, I suppose. That’s expensive~

On the top of my list in considering the suitable MTI codec would be: 

1)       To VP8, my concern is whether or not the other patent holders would hold me responsible if I provide the service?

2)       To H.264, my concern is  whether or not the device platforms would open sufficient API for browsers to allow good-quality video communication? And of course if the patent holders would hold me responsible if I provide the service?

3)       To both, a further concern is whether or not the cost for interop with legacy systems is easy and cheap?

Since no clear answers are provided for questions 1) or 2), I chose to go with the answer to 3). Because the most existing systems and end-points supports H.264. At this moment, I have no choice but to live with it, at least for some time unless someone resolves questions 1) or 2) for me^^

BR

Lingli

China Mobile

发件人: [email-masked] [mailto:[email-masked]] 代表 Harald Alvestrand
发送时间: 2013年3月15日 18:19
收件人: Bo Burman; [email-masked]
主题: Re: [rtcweb] Cross-check of Google VP8 vs H.264 comparison

Hi,

Cheers,
Bo

  _____  

rtcweb mailing list
[email-masked]

---

From: Bo Burman <[email-masked]>
Date: Fri, 15 Mar 2013 16:08:28 +0000

We used the Google set of clips:
desktop_640_360_30
gipsrecmotion_1280_720_50
gipsrecstat_1280_720_50
kirland_640_480_30
macmarcomoving_640_480_30
macmarcostationary_640_480_30
niklas_1280_720_30
niklas_640_480_30
tacomanarrows_640_480_30
tacomasmallcameramovement_640_480_30
thaloundeskmtg_640_480_30
I also attach two files showing more detailed results, with and without the --tune psnr option for x264

Cheers,
Bo.

________________________________
From: Harald Alvestrand [mailto:[email-masked]]
Sent: Friday, March 15, 2013 6:19 AM
To: Bo Burman; [email-masked]
Subject: Re: [rtcweb] Cross-check of Google VP8 vs H.264 comparison

Hi,

Cheers,
Bo
________________________________

rtcweb mailing list
[email-masked]