Subject: [payload] H.265 payload format: new proposed codepoint sprop-sei

From: Stephan Wenger <[email-masked]>
Date: Wed, 23 Apr 2014 21:40:10 +0000

Hi all,

I want to propose an additional code point sprop-sei for the H.265 payload format, which is a WG draft.  There is agreement between four of the five authors (#5 is on vacation or traveling or something) that this is useful.  The ISO file format spec for H.265 has a similar code point fro similar purposes.

The following would be added after the sprop-* parameter sets into section 7.1

"
sprop-sei: This parameter MAY be used to convey one or more SEI messages that describe bit stream characteristics.  When present, a decoder can rely on the sub-stream or bitstream characteristics, as the case may be, that are described in the SEI message for the entire duration of the session, independently from the persistence rules as defined in HEVC.
The value of the parameter is a comma-separated (',') list of base64 [RFC4648] representations of SEI message NAL units as specified in Section 7.3.2.4 of [HEVC].

Informative Note: Intentionally, no list of applicable or inapplicable SEI messages is specified here.  Conveying certain SEI messages in sprop-sei may be sensible in some application scenarios and meaningless in others.  However, a few examples can be provided:
In an environment where the encoded stream was created from film-based source material, and no splicing is going to occur during the lifetime of the session, the film grain characteristics SEI message or the tone mapping information SEI message are likely meaningful, and sending them in sprop-sei rather than in the video bitstream at each entry point may help saving bits and allows to configure the renderer only once, avoiding unwanted artifacts.
The structure of pictures information SEI message in sprop-sei can be used to inform a decoder of the use of a certain structure of pictures (also known as Group of Pictures, GOP).  Having such knowledge can be helpful for error recovery.
Examples for SEI messages that would be meaningless to be conveyed in sprop-sei include the Decoded picture hash SEI message (it is close to impossible that all decoded pictures have the same hash-tag), on a handheld device the Display orientation SEI message (as the display orientation may change when the handheld device is turned aournd), or the filler payload SEI message (as there is no point in wasting bits in SDP).
"

As a design choice, we could forego the comma-separated list, and just allow one such NAL unit, because SEI messages have their own encapsulation mechanism (in contrast to parameter sets, multiple SEI messages can be conveyed in a single NAL unit).  OTOH, there is probably value in keeping design and implementation orthogonality between the various sprops, and that is why the comma-separated list is proposed.

I don't think there is much to say about this in the O/A section.   With sprop-sei, we ONLY want to express stream characteristics as a subset of the possible negotiated features (as indicated by profile, level, tier) and not as an additional negotiation mechanism.  Similarly, there is nothing to add in the declarative use section.

One wanted side effect of this proposal is the possible inclusion of user specified SEI messages into the SDP.  This would allow users, including downstream standardization groups, to "label" bitstreams conforming to their specs (which they can already do today in the HEVC bitstream), and expose such labeling in the SDP for use during session setup.  One example would be to "label" a bitstream offered as conforming to an application layer specification.

We plan to include the language above into the next revision of the draft, unless there is noisy opposition here.  So if you don't like what you have read, please yell.

Thanks,
Stephan