Subject: [rtcweb] Encoding with FFMPEG - controls?

From: Harald Alvestrand <[email-masked]>
Date: Mon, 09 Dec 2013 16:22:21 +0100

Totally off track for this discussion, but I wonder if someone here can 
help, or point me to the right forum for asking?

I've been playing with some of ffmpeg's codecs, just to see what the 
alternatives proposed for "royalty-free alternatives" would look like.

So far, one of my results has been that either ffmpeg's rate controller 
is extremely strange, or I haven't figured out how to control it.

Some results of running ffmpeg -i <yuvfile> -b:v <rate> -codec:v <codec> 
<outfile>:

File                          Rate       H.261      H.263p MJPEG         VP8
Kimono1_1920x1080_24          1600    1535 33.9   3028 34.2  38355 
30.0   1498 37.0
Kimono1_1920x1080_24          2500    2369 34.7   3030 34.2  45589 
30.1   2300 38.5
Kimono1_1920x1080_24          4000    3260 35.2   3936 35.8  61791 
30.2   3721 39.9
Kimono1_1920x1080_24          6000    3444 35.3   5889 38.3  63107 
30.2   5613 41.1
ParkScene_1920x1080_24        1600    1584 30.0   2965 31.5  56733 
30.6   1562 34.0
ParkScene_1920x1080_24        2500    2423 30.4   2980 31.5  66163 
30.7   2424 35.4
ParkScene_1920x1080_24        4000    3462 30.6   3953 32.7  81212 
30.9   3912 37.0
ParkScene_1920x1080_24        6000    3462 30.6   5856 34.6  98638 
31.0   5878 38.4
BasketballDrill_832x480_50     512    1130 30.0   1460 30.3  17681 
31.9    712 32.6
BasketballDrill_832x480_50     768    1643 30.8   1732 31.3  18941 
32.2    818 33.4
BasketballDrill_832x480_50    1200    2512 31.5   2607 33.4  25211 
33.5   1202 34.9
BasketballDrill_832x480_50    2000    4116 32.1   4215 35.6  44846 
34.6   1980 36.9

(H.261 was shoehorned into performing for higher resolutions by 
rescaling to CIF and then rescaling back up before computing PSNR - a 
most abhorrent tactic. Also, the VP8 command lines were different, of 
course).

As you can see, I got some bitrates that were pretty closely matched for 
H.261, but other bitrates that were 2x the expected bitrate - for H.263p 
the bitrates were always at least double the ones I asked for, and for 
MJPEG - well, the numbers speak for themselves.

The rate controller is also having *some* effect - it's the only thing 
differing between the encodes - but it's not hitting its target.

Any advice?

Harald

---

From: Basil Mohamed Gohar <[email-masked]>
Date: Mon, 09 Dec 2013 10:45:24 -0500

> Kimono1_1920x1080_24          1600    1535 33.9   3028 34.2  38355

Harald,

Please see the text files I posted with my H.261 samples last week.
There are some additional options that help ffmpeg track the target
bitrate more accurately.  For example:

ffmpeg -i ../sign_irene_cif.y4m -codec:v h261 -b:v $bitrate -minrate
$bitrate -maxrate $bitrate -bufsize $bitrate -qmax 1024 -g 30 -y
sign_irene_cif.y4m-$bitrate.h261

The key options are -b:v, -minrate, -maxrate, and -bufsize.

You'd need to set all of these to roughly the same size to have as close
to CBR as possible, across the 1 second window.

Also, make sure the -qmin and -qmax options follow the whole range of
possible quantizer values.

It doesn't help, either, that different codecs use different options.
For example, -qscale works for libtheora, but not libvpx.

Finally, since ffmpeg implements many of their own codecs, while this is
beneficial in some cases, in others, they just simply cannot reach
certain rates.  For example, mpeg1video and h261 both really cannot
reach certain rates.  The h261 encoder could not get below ~140kbps with
sign_irene.y4m no matter what I fiddled.  I don't know if the spec
allows VFR or dropping frames.  But libtheora could, but it did so by
dropping a large number of frames (basically every delta frame was dropped).

-- 
Libre Video
http://librevideo.org

---

From: Stephan Wenger <[email-masked]>
Date: Mon, 9 Dec 2013 16:20:46 +0000

Hi,

On 12.9.2013, 07:45 , "Basil Mohamed Gohar" <[email-masked]>
wrote:

>[...]
>  The h261 encoder could not get below ~140kbps with
>allows VFR or dropping frames.
>[...].

All video coding specs I know (with the exception of H.120, IIRC) do allow
frame drops, and dropping frames is the key strategy to get out of a
bandwidth starvation situation by pretty much all commercial video
conferencing products I know.  Dialing down quality may be appropriate for
some uses (Gunnar indicated that the sign language people prefer that) but
it¹s not what products commonly do, at least not in their default
configuration.

Stephan

>
>-- 
>Libre Video
>http://librevideo.org
>_______________________________________________
>rtcweb mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 09 Dec 2013 17:31:28 +0100

> to CBR as possible, across the 1 second window.

Thanks! I don't care that much about undershoots - presumably the rate 
controller would use as many bits as it's allowed to; in other threads 
I've seen, people are saying that -minrate will cause filler to be 
inserted in the bitstream, which doesn't really make sense for the 
Internet case (and does not improve quality).

> possible quantizer values.

Is there a way short of code inspection to figure out what these params 
are set to by default?

> dropping a large number of frames (basically every delta frame was dropped).

It's those limits (and what happens when you hit them) that are part of 
my reason for experimentation...

---

From: Basil Mohamed Gohar <[email-masked]>
Date: Mon, 09 Dec 2013 11:38:19 -0500

> Internet case (and does not improve quality).

I think for our cases, namely, older codecs and lower rates, this is not
much of a concern.  It's probably rare that it's using the
highest-quality (lowest) quantizers.  I think this would be the case
only if you don't see much increase in filesize as you bump up the
allowable target rate, meaning, it can't use any higher-quality images,
and so THEN resorts to stuffing the bitstream.

In truth, I find ffmpeg will be gated at both the low bitrate end as
well as the higher one, meaning, I don't think they normally use filler.
 I've seen MJPEG, for example, hitting a floor as well as a ceiling.
But this can be different depending on codecs, no doubt.

> Is there a way short of code inspection to figure out what these params

Usually ffmpeg outputs what the range of q-values are it uses to the
terminal by default.  Look for something like [q=2 - 31] or somesuch.

> my reason for experimentation...

I think talking to the ones that implemented those codecs in ffmpeg
would be more fruitful, because ffmpeg is complex enough to make drawing
clear conclusions from it for a format difficult, depending on different
types of content.

One bit we're overlooking, but that caught me by surprise, is that MJPEG
is (or was, a while ago) using full video level signals (i.e., 0 - 255)
vs., say, libtheora, which uses the gated ones (something like 16 -
230), which makes direct comparisons unfair and incomplete.

You get around this by passing -strict -2 or something to ffmpeg, which
forces the yuv420p pixel format, instead of yuvj420p, which has the
higher range.

Fair comparisons are tough!

-- 
Libre Video
http://librevideo.org

---

From: Sergio Garcia Murillo <[email-masked]>
Date: Mon, 09 Dec 2013 18:01:02 +0100

> Internet case (and does not improve quality). 

My 2 cents:

I have been using ffmpeg's h263 codec in my MCU (via libavcodec) and 
struggled quite a lot to find the correct parameters that worked for me 
in a RTC enviroment. The ones I am using are:

     ctx->bit_rate  = bitrate;
     ctx->bit_rate_tolerance = bitrate/fps+1;
     ctx->rc_max_rate = bitrate;
     ctx->rc_buffer_size = bitrate/fps+1;
     ctx->rc_initial_buffer_occupancy = 0;
     ctx->rc_qsquish = 1;

Best regards
Sergio

---

From: Leon Geyser <[email-masked]>
Date: Mon, 9 Dec 2013 20:34:30 +0200

For MJPEG to hit the target bitrate you can use a higher '-qscale x' value,
lower the framerate or resolution. x can be from 2 to 32.

H.261 usually works in multiples of 64Kbit/s like (128Kbps, 192Kbps,
256Kbps etc.) as far as I know. You can easily hit 64Kbps if you scale down
to QCIF.

---

From: Stephan Wenger <[email-masked]>
Date: Mon, 9 Dec 2013 22:05:24 +0000

Hi,
H.261 doesn't care about bitrate incrementals; not the standard, nor product implementations.  Historically, the p x 64 stems from the application scenario H.261 was designed for (transport over ISDN B channels), but even over ISDN the majority of the systems used G.728 audio (at least towards the end of the H.261 lifespan as far as real-world products go), which needs 16 kbit/s.  Add a (very) few kbit/s for H.230 call control, and you end up with ~110 kbit/s over two B channels.  That was a very typical video bitrate at the time.  The other common one was around 360 kbit/s (using 6 B channels).  (In 7-bit ISDN scenarios like the US at the time, multiply above bitrates by 7/8).
Stephan

From: Leon Geyser <[email-masked]<mailto:[email-masked]>>
Date: Monday, 9 December, 2013 at 10:34
To: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [rtcweb] Encoding with FFMPEG - controls?

For MJPEG to hit the target bitrate you can use a higher '-qscale x' value, lower the framerate or resolution. x can be from 2 to 32.

H.261 usually works in multiples of 64Kbit/s like (128Kbps, 192Kbps, 256Kbps etc.) as far as I know. You can easily hit 64Kbps if you scale down to QCIF.