Subject: [Cellar] TrackTypes for non-standard data formats

From: Matt Gruenke <[email-masked]>
Date: Mon, 23 Jan 2017 20:32:29 -0500

Hello,

I have a use case for Matroska files/streams I'd like to discuss, and was
referred to this list.

I need to multiplex a video stream, possibly an audio stream, and one or
more streams of time-stamped, non-standard binary data.  In considering
which TrackType to use, I am concerned that using audio or video might
break compatibility with standard players (i.e. for the *actual*
audio/video tracks contained in these files).

I saw recent discussion of adding a track type specifically for ancillary
data, but it seems to me that it might also be worth considering reserving
a small block of TrackType values for non-standard, application-specific
uses.  As I try to imagine the different kinds of structures & uses tracks
might have, within a family of applications, I think this block should
consist of somewhere between 4 and 8 values.

Thank you for considering my suggestion.

Matt

---

From: Dave Rice <[email-masked]>
Date: Tue, 24 Jan 2017 10:24:26 -0500

> On Jan 23, 2017, at 8:32 PM, Matt Gruenke <[email-masked]> wrote:
> 
> Hello,
> 
> I have a use case for Matroska files/streams I'd like to discuss, and was referred to this list.
> 
> I need to multiplex a video stream, possibly an audio stream, and one or more streams of time-stamped, non-standard binary data.  In considering which TrackType to use, I am concerned that using audio or video might break compatibility with standard players (i.e. for the actual audio/video tracks contained in these files).
> 
> I saw recent discussion of adding a track type specifically for ancillary data, but it seems to me that it might also be worth considering reserving a small block of TrackType values for non-standard, application-specific uses.  As I try to imagine the different kinds of structures & uses tracks might have, within a family of applications, I think this block should consist of somewhere between 4 and 8 values.

IIRC the proposal for ancillary tracks was to have Codec_IDs prefixed with "N_". If your binary data stream is wholly private with contents that are not intended to be known by the Matroska specification then perhaps a private Codec ID prefix (such a "X_") would be worthwhile to define. On the other hand, if you could be more transparent you could also consider defining your codec openly, perhaps the application specific use could have wider interest.

Or could OggKate store the values you need? Perhaps it's worthwhile to define storage for a time-based metadata track such as this.

Dave Rice

---

From: Matt Gruenke <[email-masked]>
Date: Tue, 24 Jan 2017 23:39:11 -0500

If it'd help to deal with specifics, some examples I have in mind are scene
geometry (e.g. point clouds, polygonal meshes, Z-buffer), camera pose (e.g.
position & orientation as vector & quaternion pairs, GPS coordinates,
compass readings), and various telemetry data (plenty of examples at
https://en.wikipedia.org/wiki/Telemetry ).

Although nothing specific comes to mind that wouldn't have a stream-like
structure, one might allow for more such things as menus and button
tracks.  Perhaps a game streaming application might use a track for player
information and statistics.

In considering the utility & possible necessity of multiple,
application-defined TrackTypes, perhaps it's worth reviewing the purposes
they serve.  One function of TrackType is certainly to simplify semantic
mapping of the available tracks.  To this end, it can prove more reliable
than CodecID, since it's certainly possible to use the same encoding for
tracks with different semantics.

Of course, TrackType also dictates the content model of TrackEntry.
Though, I don't know enough about EBML to have a sense of whether it's
practical and reasonable for applications to extend the schema to allow for
custom track metadata.

I guess I'd ask why *not* allow ancillary data to have one of ~8
TrackTypes?  At the very least, I'd suggest using somewhere near the top of
the range.  Then, more values for ancillary data could be added,
subsequently, while keeping the range contiguous.

Matt

On Tue, Jan 24, 2017 at 10:24 AM, Dave Rice <[email-masked]> wrote:

>
> IIRC the proposal for ancillary tracks was to have Codec_IDs prefixed with
> "N_". If your binary data stream is wholly private with contents that are
> not intended to be known by the Matroska specification then perhaps a
> private Codec ID prefix (such a "X_") would be worthwhile to define. On the
> other hand, if you could be more transparent you could also consider
> defining your codec openly, perhaps the application specific use could have
> wider interest.
>
> Or could OggKate store the values you need? Perhaps it's worthwhile to
> define storage for a time-based metadata track such as this.
>
> Dave Rice
>