Subject: Re: [Cellar] Lacing for video frames

From: Steve Lhomme <[email-masked]>
Date: Sun, 5 Nov 2017 19:18:48 +0100

Hi,

I see the subject is not very popular but I still want Matroska to be
the best, so I had a deeper quick look.

As I don't do any adaptive streaming encoding for the web I tried to
do it with ffmpeg but I failed miserably. Then I found these samples
from Apple [1].

So I took the lowest bitrate video [2] which is 530kbps according to
the manifest and 369 kbps according to MediaInfo. Then I remuxed it
with mkvmerge. Then go through mkclean and here are the results:
- 27 672 619 original fMP4 with H264
- 27 449 794 mkvmerge with default options (we win already)
- 27 447 068 mkclean with default options
- 27 439 197 mkclean with --live
- 27 357 090 mkclean with --remux and --optimize
- 27 349 220 mkclean with --remux and --optimize and --live

Same thing with the advanced HEVC sample that has 165 kbps variants
for HEVC and H264:
- 10 663 861 original fMP4 with H264
- 10 558 115 mkvmerge with default options (we win already)
- 10 554 002 mkclean with default options
- 10 498 886 mkclean with --remux and --optimize

- 11 492 052 original fMP4 with HEVC
- 11 410 786 mkvmerge with default options (we win already)
- 11 407 257 mkclean with default options
- 11 371 266 mkclean with --remux and --optimize

So in short, even with the 4KB padding that mkvmerge uses, the
Matroska variants are still smaller. With boundaries set at every
keyframe, that should do proper segments for adaptive streaming just
like with fMP4. Except with have a proper seeking cue, checksums on
vital parts and the possible extreme low latency described in my
earlier post. And all that without using lacing at all.

At 1.54% smaller in the second H264 sample, that's quite a difference
with no data lost at all.

[1]: https://developer.apple.com/streaming/examples/
[2]: https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v2/main.mp4

2017-10-29 16:53 GMT+01:00 Steve Lhomme <[email-masked]>:
> As some of you may know I was in San Fancisco a few weeks ago for 2
> conferences (FOMS and Demuxed) about video, mostly on the web. I did a
> blog about it here
> http://robux4.blogspot.fr/2017/10/foms-and-demuxed.html
>
> Among other interresting things I found out about a new video
> container called CMAF. It seems to be a set of guidelines around
> fragmented MP4 which I didn't know existed either. It's targeted
> mainly at web video distribution with adaptive streaming in mind. CMAF
> seems even more specified in distribution of encrypted (DRM) content.
>
> WebM is used the same way and can achieve the same level of features
> (and even more with Matroska). But while I dug a little more, I
> noticed that fragmented MP4 tend to be much smaller than the same
> content in Matroska/WebM (see blog post for some details). It's
> because with fragmented MP4 (or the files I analyzed) the video frames
> are packed contiguously, the boundaries and timestamp being stored in
> a separate atom (equivalent of an EBML Master) which itself doesn't
> contain more info that that.
>
> To compare a fragmented MP4 frame with Matroska it would be something like this:
> fMP4 = [encoded frame] + [size of encoded frame] + [key frame flag]
> MKV = [SimpleBlock ID] + [SimpleBlock Length] + [SimpleBlock Header] +
> [encodec frame]
>
> I think we could reduce our overhead to reach something more in line
> with fMP4. It would be mean allowing lacing for video frames
> explicitly. Right now there's nothing that prevent it from being used.
> It may lack some descriptions though on how to interpret the
> timestamps and the SimpleBlock flags. I think for video it should
> required a DefautlDuration being present for the track and also that
> all flags are the same in the (Simple)Block, especially the keyframe.
> I'm also not sure if mkvmerge blocks it for video or not.
>
> Because the same flag has to be present and because we store frames in
> coding order, there might be cases where this doesn't work. When B
> frames are present you cannot tell the frame timestamp within a lace.
> So that may limit the use with codecs like H264 or HEVC.
>
> --
> Steve Lhomme
> Matroska association Chairman

-- 
Steve Lhomme
Matroska association Chairman

---

From: Moritz Bunkus <[email-masked]>
Date: Sun, 05 Nov 2017 19:33:55 +0100

Hey,

I did read your mail, but to be honest I don't care all that much about
having the smallest possible files in all cases. In all comparisons I did
in the past Matroska came out looking very competitive (only Nut was
consistently smaller). On top of that my gut feeling was that lacing video
frames wouldn't give that much of an advantage to warrant a large investment
of time and effort.

Your measurements seem corroborate my gut feeling.

Kind regards,
mosu

---

From: Steve Lhomme <[email-masked]>
Date: Sun, 5 Nov 2017 19:37:50 +0100

I agree. The only case where lacing is useful is when it actually
saves bytes. But lacing itself is not space efficient. So it's only
worth for very small frames and possibly similar frame sizes. That's a
rare/edge case for video.

On the other hand, this overhead thing is one of the key feature of
Matroska. It's why EBML and default values and lacing exist. That was
one of the main design goal and it would be side to have that
competitive feature lost.

2017-11-05 19:33 GMT+01:00 Moritz Bunkus <[email-masked]>:
> _______________________________________________
> Cellar mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/cellar

---

From: Tobias Rapp <[email-masked]>
Date: Mon, 6 Nov 2017 09:41:28 +0100

> frames wouldn't give that much of an advantage to warrant a large investment

Similar experience on my side. I looked at MP4 some time ago and found 
that this way of putting all index data into one box and all encoded 
data into another box has a negative impact on playback initialization 
time: A player has to load all the index data to be able to start 
playing. When file duration is some hours and network throughput is slow 
(web) the effect really hurts much more than what is gained by the 
possibly lower container overhead.

Fragmented MP4 compensates for the playback initialization time and 
enables DASH. As fast player initialization and DASH already seem to be 
supported by Matroska/WebM I see no big gain in further file size 
optimization -- the current numbers look comparable enough.

Just my personal thoughts.

Tobias