Subject: [Cellar] Lacing for video frames

From: Steve Lhomme <[email-masked]>
Date: Sun, 29 Oct 2017 16:53:57 +0100

As some of you may know I was in San Fancisco a few weeks ago for 2
conferences (FOMS and Demuxed) about video, mostly on the web. I did a
blog about it here
http://robux4.blogspot.fr/2017/10/foms-and-demuxed.html

Among other interresting things I found out about a new video
container called CMAF. It seems to be a set of guidelines around
fragmented MP4 which I didn't know existed either. It's targeted
mainly at web video distribution with adaptive streaming in mind. CMAF
seems even more specified in distribution of encrypted (DRM) content.

WebM is used the same way and can achieve the same level of features
(and even more with Matroska). But while I dug a little more, I
noticed that fragmented MP4 tend to be much smaller than the same
content in Matroska/WebM (see blog post for some details). It's
because with fragmented MP4 (or the files I analyzed) the video frames
are packed contiguously, the boundaries and timestamp being stored in
a separate atom (equivalent of an EBML Master) which itself doesn't
contain more info that that.

To compare a fragmented MP4 frame with Matroska it would be something like this:
fMP4 = [encoded frame] + [size of encoded frame] + [key frame flag]
MKV = [SimpleBlock ID] + [SimpleBlock Length] + [SimpleBlock Header] +
[encodec frame]

I think we could reduce our overhead to reach something more in line
with fMP4. It would be mean allowing lacing for video frames
explicitly. Right now there's nothing that prevent it from being used.
It may lack some descriptions though on how to interpret the
timestamps and the SimpleBlock flags. I think for video it should
required a DefautlDuration being present for the track and also that
all flags are the same in the (Simple)Block, especially the keyframe.
I'm also not sure if mkvmerge blocks it for video or not.

Because the same flag has to be present and because we store frames in
coding order, there might be cases where this doesn't work. When B
frames are present you cannot tell the frame timestamp within a lace.
So that may limit the use with codecs like H264 or HEVC.

-- 
Steve Lhomme
Matroska association Chairman