Subject: [rtcweb] H.261 encoding samples at typical bitrates - sign language example

From: Basil Mohamed Gohar <[email-masked]>
Date: Mon, 02 Dec 2013 22:03:34 -0500

Let's let any further discussions about the usability of H.261, or any
other codec for that matter, use actual examples going forward.

The following is a VERY quick test of ffmpeg's h261 encoder in the
context of the IETF's rtcweb working group's discussion of an MTI
(mandatory-to-implement) video codec.

sine_irene_cif.y4m taken from derf's collection:

http://media.xiph.org/video/derf/y4m/sign_irene_cif.y4m

ffmpeg version N-58565-gc122e69

bitrate=64k,128k,256k,512k

ffmpeg -i sign_irene_cif.y4m -codec:v h261 -b:v $bitrate -g 30
sign_irene_cif.y4m-$bitrate.h261

http://media.basilgohar.com/rtcweb/sign_irene_cif.y4m-64k.h261
(real rate: 157.8kbits/s)

http://media.basilgohar.com/rtcweb/sign_irene_cif.y4m-128k.h261
(real rate: 165.6kbits/s)

http://media.basilgohar.com/rtcweb/sign_irene_cif.y4m-256k.h261
(real rate: 289.5kbits/s)

http://media.basilgohar.com/rtcweb/sign_irene_cif.y4m-512k.h261
(real rate: 541.8kbits/s)

I apologize for the bitrate inflation, but if I had more time I can
tweak the settings for a more accurate number.  These are simply the
rates that ffmpeg produced with such a short clip at the given requested
rates.
-- 
Libre Video
http://librevideo.org

---

From: Basil Mohamed Gohar <[email-masked]>
Date: Mon, 02 Dec 2013 23:23:41 -0500

> rates that ffmpeg produced with such a short clip at the given requested

I've updated the encoding settings as follows to get more accurate
resulting bitrates, but ffmpeg's h261 encoder seems to bottom-out at
around ~140kbps, so the only examples from above that are close (after
using the new settings) are 256k and 512k.

for bitrate in {1..512}k; do ffmpeg -i ../sign_irene_cif.y4m -codec:v
h261 -b:v $bitrate -minrate $bitrate -maxrate $bitrate -bufsize $bitrate
-qmax 1024 -g 30 -y sign_irene_cif.y4m-$bitrate.h261; done;

All the above posted examples can be viewed with mplayer and a bash
command line using the following, if you're interested:

mplayer
http://media.basilgohar.com/rtcweb/sign_irene_cif.y4m-{64,128,256,512}k.h261

The full integer range of bitrates from 1 to 512 can be found here:

http://media.basilgohar.com/rtcweb/h261/

Target bitrate and actual bitrate start to match around 150kbps with
these new settings.

I am currently exploring other codecs with the same methodology and will
share the results accordingly.

-- 
Libre Video
http://librevideo.org

---

From: Justin Uberti <[email-masked]>
Date: Thu, 5 Dec 2013 13:05:57 -0800

Ow, my eyes...

The 256 kbps and lower clips are unusable. The 512 kbps clip is borderline,
but might be usable if the framerate was cut in half.

Remember also that these test clips are far better than what would be
obtained from consumer webcams (i.e. good lighting, no shake, no temporal
noise), so real-world performance is likely to be worse than what you see
here.

On Mon, Dec 2, 2013 at 8:23 PM, Basil Mohamed Gohar <
[email-masked]> wrote:

> _______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb
>

---

From: Justin Uberti <[email-masked]>
Date: Thu, 5 Dec 2013 13:21:47 -0800

A frame comparison for the various encodings:

On Thu, Dec 5, 2013 at 1:05 PM, Justin Uberti <[email-masked]> wrote:

> Ow, my eyes...
>
> The 256 kbps and lower clips are unusable. The 512 kbps clip is
> borderline, but might be usable if the framerate was cut in half.
>

---

From: Justin Uberti <[email-masked]>
Date: Thu, 5 Dec 2013 13:24:27 -0800

oops - the h261-541kbps and h261-289kbps labels are swapped. Sorry about
that...

---

From: Gunnar Hellstrom <[email-masked]>
Date: Thu, 05 Dec 2013 22:56:47 +0100

During the times when H.261 was used in H.320 products, around 360 
kbit/s provided good performance for sign language with 25 fps and CIF 
resolution. ( and powerful videoconferencing units)

It was also used at about 105 kbit/s with sign language but then in fact 
far below really usable performance both with QCIF and CIF. 
Communication is a necessity...

/Gunnar
------------------------------------------------------------------------
On 2013-12-05 22:05, Justin Uberti wrote:
> obtained from consumer webcams (i.e. good lighting, no shake, no 
> temporal noise), so real-world performance is likely to be worse than 
> what you see here.
>
>
> On Mon, Dec 2, 2013 at 8:23 PM, Basil Mohamed Gohar 
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>     <http://media.basilgohar.com/rtcweb/sign_irene_cif.y4m-%7B64,128,256,512%7Dk.h261>
>
>     [email-masked] <mailto:[email-masked]>

---

From: Gunnar Hellstrom <[email-masked]>
Date: Thu, 05 Dec 2013 23:05:49 +0100

The 289 kbit/s is clearly usable for sign language without problems. The 
fingers are clearly visible, the mouth shape and the eye-gaze direction.
If that was done with 25 fps it is good and sufficient. Never lower 
frame-rate below 20 fps!

With the size of the person, two persons could be fit into the same 
picture and still be good.

That matches the old observations from mid-90-s with the best conference 
units.

( the 541 kbit/s looks a bit overdone in artificial sharpening or 
something. Not as good as the 289. )

Gunnar

On 2013-12-05 22:21, Justin Uberti wrote:
> On Thu, Dec 5, 2013 at 1:05 PM, Justin Uberti <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     Ow, my eyes...
>
>     Remember also that these test clips are far better than what would
>     be obtained from consumer webcams (i.e. good lighting, no shake,
>     no temporal noise), so real-world performance is likely to be
>     worse than what you see here.
>
>

---

From: Gunnar Hellstrom <[email-masked]>
Date: Thu, 05 Dec 2013 23:10:28 +0100

On 2013-12-05 22:24, Justin Uberti wrote:
> oops - the h261-541kbps and h261-289kbps labels are swapped. Sorry 
> about that...
Ah, that explains my evaluation.
So, 289 kbit/s is still usable for sign language. You can perceive the 
important characteristics, but it is unpleasant and probably a bit tiring.
And 541 good.
So, if they were done at 25 fps then 541 is good and 289 usable.

Gunnar
>
>
> On Thu, Dec 5, 2013 at 1:21 PM, Justin Uberti <[email-masked] 
>         The 256 kbps and lower clips are unusable. The 512 kbps clip
>         is borderline, but might be usable if the framerate was cut in
>         half.
>
>         Remember also that these test clips are far better than what
>         would be obtained from consumer webcams (i.e. good lighting,
>         no shake, no temporal noise), so real-world performance is
>         likely to be worse than what you see here.
>
>

---

From: cowwoc <[email-masked]>
Date: Thu, 05 Dec 2013 17:31:35 -0500

Agreed. Both are usable, but 289 is tiring/annoying on the eyes.

Gili

On 05/12/2013 5:10 PM, Gunnar Hellstrom wrote:
>>         is borderline, but might be usable if the framerate was cut
>>         in half.
>>
>>         <[email-masked]
>>         <mailto:[email-masked]>> wrote:
>>

---

From: Gunnar Hellstrom <[email-masked]>
Date: Thu, 05 Dec 2013 23:50:17 +0100

> Agreed. Both are usable, but 289 is tiring/annoying on the eyes.
That makes the old experience very likely still true: around 360 kbit/s 
is needed for good usability and then CIF and 25 (or 30) fps should be used.

How about robustness against packet loss and jitter?

Well there are at least a couple of methods to indicate packet loss and 
need for refresh mentioned in RFC 4587.

Gunnar
>> So, 289 kbit/s is still usable for sign language. You can perceive 
>> the important characteristics, but it is unpleasant and probably a 
>> bit tiring.
>> And 541 good.
>>>     On Thu, Dec 5, 2013 at 1:05 PM, Justin Uberti
>>>     <[email-masked] <mailto:[email-masked]>> wrote:
>>>
>>>         Ow, my eyes...
>>>

---

From: David Singer <[email-masked]>
Date: Fri, 06 Dec 2013 16:21:41 -0800

On Dec 5, 2013, at 13:05 , Justin Uberti <[email-masked]> wrote:

> Ow, my eyes...
> 
> The 256 kbps and lower clips are unusable. The 512 kbps clip is borderline, but might be usable if the framerate was cut in half.

Be careful, in a motion-prediction codec, moving the frames further apart in time increases the likelihood that motion prediction will fail, and thus may actually increase, or plateau, the bit-rate.

(many years ago I did experiments with 261 and 263 on this; these codecs have limited MV ‘reach’ and so really can exhibit this behavior)

David Singer
Multimedia and Software Standards, Apple Inc.

---

From: Gunnar Hellstrom <[email-masked]>
Date: Sat, 07 Dec 2013 06:46:56 +0100

> (many years ago I did experiments with 261 and 263 on this; these codecs have limited MV ‘reach’ and so really can exhibit this behavior)
And cutting frame rate is also the worst you can do to a sign language 
communication. So please don't.
Reducing resolution below CIF, or increasing quantization are much more 
acceptable actions than skipping frames. (to some limit of course)
But I think we already stated that there is no need to improve picture 
quality over the 512 kbit/s sample. Instead somewhere around 360 kbit/s 
would be sufficient for CIF 25 fps and sign language of two persons.
Which is in fact very close to the recommendation we use today for H.263 
and H.264 for the same application.  Around 300 kbit/s for video is a 
commonly used figure with H.263 and H.264 with CIF and 25 or 30 fps for 
sign language usage.

/Gunnar  (still hoping that this is a hypothetical discussion )

---

From: Maik Merten <[email-masked]>
Date: Sat, 07 Dec 2013 21:41:46 +0100

> Ow, my eyes...

Indeed, but old codecs such as H.261, MPEG-1 Part2, H.263 etc. somewhat 
rely on postprocessing at the receiving end for acceptable perceived 
quality in low-bitrate scenarios.

I attached a decoded frame of the ~256 kbps clip: One without 
postprocessing, one with mplayer's "pp" video postprocessing filter, and 
one with mplayer's "pp7" postprocessing filter. Clearly the filters 
cannot recover information lost in the low-bitrate encoding, but make 
things much less hurtful to look at, especially in motion.

Real-world performance thus also depends on the receiving end. This 
unfortunately is hard to assess and completely depends on implementation 
details of the receiver.

Maik