Subject: Re: [Cellar] Rationale Numbers for timestamps

From: Steve Lhomme <[email-masked]>
Date: Sat, 4 Feb 2017 15:28:32 +0100

2017-01-15 18:12 GMT+01:00 Reto Kromer <[email-masked]>:
> Dave Rice wrote:
>
>>For hand cranked silent film, the film prints would usually
>>be distributed with recommended frame rates, so that the
>>projectionist is ‘trying’ to crank in order to present at
>>18 fps (or as directed). If that film is digitization to a
>>Matroska file then I think that the Matroska should also be
>>labelled for an 18 fps projection just as the film print
>>would have been. If there’s desire to replicate the
>>unperfect cranking of a human operator than that could be
>>accomplished by the player and not the file itself. Should
>>I file a ticket for VLC to accept input from a USB-attached
>>hand crank?

I'm sure we have had weirder requests. You may want to give an example
of such a device we can test with, though :)

> +1
>
> There is not only the operator's "imperfect hand-cranking",
> but also the films themselves that may have different
> projection speeds between reels and also inside a single
> reel. Many instruction sheets for the operator as well as
> for the conductor are known.

Given that's an edge case we would like to handle can you tell a bit
more about the limits we would have to reach ? The fact it's in the
same reel or different reel doesn't really matter if it's all in one
Segment. But the different possible values or if some say things like
'just roll it as fast as humanly possible'. Of course in such case we
still have the old system that still works fine when you don't need
pure digital accuracy.

> Best regards, Reto
>
> _______________________________________________
> Cellar mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/cellar

-- 
Steve Lhomme
Matroska association Chairman

---

From: Steve Lhomme <[email-masked]>
Date: Sat, 4 Feb 2017 15:39:37 +0100

2017-01-15 17:24 GMT+01:00 Dave Rice <[email-masked]>:
> Hi Jerome, Steve,
>
>> On Jul 27, 2016, at 3:28 PM, Steve Lhomme <[email-masked]> wrote:
>>
>> 2016-07-27 6:13 GMT+02:00 Jerome Martinez <[email-masked]>:
>>> On 27/07/2016 04:35, Steve Lhomme wrote:
>>>>
>>>> As the topic often resurfaces maybe it's time we discuss it in the
>>>> context of CELLAR.
>>>>
>>>> There have been talks about ways to do it in the Matroska mailing
>>>> before in this thread:
>>>>
>>>> https://lists.matroska.org/pipermail/matroska-devel/2012-October/004294.html
>>>>
>>>> https://lists.matroska.org/pipermail/matroska-devel/2012-October/004295.html
>>>>
>>>> https://lists.matroska.org/pipermail/matroska-devel/2012-October/004296.html
>>>>
>>>> https://lists.matroska.org/pipermail/matroska-devel/2012-October/004299.html
>>>>
>>>> An element to handle it was added (still in specdata.xml but commented)
>>>>
>>>> https://lists.matroska.org/pipermail/matroska-devel/2012-September/004289.html
>>>> and later removed
>>>>
>>>> https://lists.matroska.org/pipermail/matroska-devel/2012-December/004317.html
>>>>
>>>> So far there was nothing conclusive but it seems it's not possible to
>>>> do it properly in a backward compatible way.
>>>>
>>>> The constraints are simple:
>>>> - backward compatible, so the 16 bits timestamp value in the
>>>> (Simple)Block must remain unchanged. This value is multiplied by the
>>>> scale (global and/or per track) to get the actual timestamp.
>>>> - There could be some acceptable rounding between the two coexisting
>>>> systems.
>>>> - The rounding should happen in the current system, the rationale
>>>> should always be exact (provided the source is exact)
>>>> - A lot of Matroska elements are based on the timestamp scales. So
>>>> they may be adjusted accordingly
>>>> - We must minimize the elements where the same information is found
>>>> twice. Basically only scale elements should be added. Maybe some
>>>> elements for the Cluster timestamp might be added as well if
>>>> necessary.
>>>> - because of the 16 bits range per Cluster differences between a
>>>> rationale system and the ns system must be tight.
>>>> - we still want Clusters to be 4-5s long to minimize the scope of
>>>> errors and buffering before playback. It may also correspond to some
>>>> GOP values.
>>>> - sample accuracy for audio would be good.
>>>>
>>>> Let the math begin.
>>>
>>>
>>> Rationale Numbers for timestamps are usually for US style fixed framerate,
>>> so I think my answer in a previous email still applies.
>>> https://mailarchive.ietf.org/arch/msg/cellar/WQCExNvAZy4goibE3_chnFB1w0c
>>> Please comment about which need would not be fulfilled with this proposal.
>>>
>>> Copy/paste:
>>>
>>> I don't like the idea to have an incompatible change with such feature,
>>> because this is not a "must have" for 99% of people, and Matroska "v5"
>>> will not be adopted with that.
>>> I think that we can have a workaround for having additional 0.99% happy
>>> without the need of a break in compatibility:
>>> - lot of people want to know the exact frame rate of a fixed frame rate
>>> stream, people muxing from tapes know that the frame rate is fixed
>>> (hardware limitation) so can set this element in the header.
>>
>> Let's rewind a decade or more. The timestamps were originally using
>> floating points on purpose. At the time all sources were analog (film
>> or video tape). And in the analog world the clock is never perfect.
>> You cannot expect to have 30000/1001 frames per second and 48000
>> samples a second for hours without some drift. And I'd rather have the
>> timing as accurate as the analog source rather than clean it for the
>> digital world.
>
> It’s true that analog format frame rates are more analogous to floating points. A video tape may stretch unevenly from use so that frames don’t start with exact precision. However these discrepancies complicate digitization and most videotape digitization will use a hardware time base corrector to buffer analog video in order to transmit it accordingly to a more precise clock, so that the resulting digital file will have already had it’s frame rate ‘cleaned’.
>
> To Jerome, by "set this element in the header” do you mean within the Info Element similar to how it contains a nominal Duration Element? Or do you mean to set this within the associated Track Element? I’m assuming per track.
>
>> In the context of digitizing for archivists I don't know if that's
>> something that's taken in account. Dave ? For example how is that
>> solved for movies from the early days where a guy would have to turn a
>> handle to advance the film. It's definitely not a very precise/clean
>> clock.
>
> For videotapes that are stretched or shrunken to affect the presentation timing, these deviations are not the intent of the recording, so using a time base corrector fixes the frame rate. Also in the case of film shrinkage the projector should still present at 24 fps although damage is more likely. So I think following the intent of the format whether 30000/1001 or 24 fps is correct.
>
> For hand cranked silent film, the film prints would usually be distributed with recommended frame rates, so that the projectionist is ‘trying’ to crank in order to present at 18 fps (or as directed). If that film is digitization to a Matroska file then I think that the Matroska should also be labelled for an 18 fps projection just as the film print would have been. If there’s desire to replicate the unperfect cranking of a human operator than that could be accomplished by the player and not the file itself. Should I file a ticket for VLC to accept input from a USB-attached hand crank?
>
>>> putting
>>> frame rate info in the header with numerator and denominator is enough
>>> for doing the difference between 30000/1001 and 29970/1000, and the
>>> current millisecond timestamp is enough (a demuxer aware of the
>>> framerate element can "correct" the timecode from e.g. 0.033s to
>>> 1001/30000 second; an old player just play the file with 1 ms
>>> precision). We could add some rules in case a timecode does not fit the
>>> frame rate (e.g. if timecode is 0.031, frame rate info is considered not
>>> applicable)
>>
>> In video the issue is not too big. But if we want sample precision for
>> audio, the fraction is a lot bigger (up to 192000 fps with modern pro
>> hardware). And with a timestamp scale shared among tracks that could
>> be problematic. If we want sample precision for audio and video we
>> could end up we'd lose a big range of the 16 bits available per
>> Cluster. Each of the 65536 Block timestamp would be 1/192,000s
>> (0.0052ms) giving a max Cluster duration of 0.341333s (65536/192000).
>> If we want to precision for a 1001/30000 track and a 1/192000 track we
>> would need a fraction of 1001/(30000*192000) (0.000173ms) resulting in
>> a Cluster max duration of 0.011s.
>
> Yes, this approach is not scalable. One could have 24000/1001 video, 30000/1001 video, 44100 hz audio, and 192000 hz audio in the same file.
>
>> If the muxer knows that a track will always pack samples by an amount
>> X (1152 in MP3 for example) we can regain some duration. But that's
>> not always the case. Some codecs like Vorbis pack only a small amount
>> of samples per frame and the amount can vary between frames (In Vorbis
>> I, legal frame sizes are powers of two from 64 to 8192 samples). The
>> worst case scenario (I know of) of Vorbis at 192 KHz, frame precision
>> of the timestamp means we can only have a Cluster duration of 21.84s
>> (65536*64/192000). Mixed with NTSC video that's
>> 65536*64*1001/30000*192000 = 0.72s max duration for sample precision
>> per frame (not sample but it's close enough). That's still 21 video
>> frames in that Cluster. Not too bad.
>>
>>> - people having variable frame rate with very precise timestamp need to
>>> know it before starting the muxing and can use a Timecodescale element
>>> of 1 (so precision of 1 nanosecond, usually enough when there is no
>>> fixed frame rate).
>>
>> I can think of 3 different variable frame rate use cases:
>>
>> - mixing video (25/29fps) and film sources (24fps) into one final video
>> - a camera dropping some frames or only storing images when they
>> change. In this case there's likely a stable main clock like in
>> constant frame rate.
>> - a camera that would work like the eye. When there's
>> adrenaline/danger you capture images faster. In this case there's no
>> real clock. The timestamps can happen at anytime.
>
> I understand the issues of adding a frame rate numerator and denominator to the track header as multiple timestamps could round to the same increment of the frame rate. However what if the rounding to the nearest increment of the frame rate is enabled by one of the reserved flag bits of the Block Header (such as bit 1 of flags = Enable TimeScale Alignment)?
>
> Thus if the frame rate of the track header is 120000/1001, then
>
> If Matroska timecode is 4 and Enable TimeScale Alignment is 0, than it is at 4 / (1000000000 / TimecodeScale ).
> If Matroska timecode is 4 and Enable TimeScale Alignment is 1, than it is at 0 / 1200000 (nearest increment of the rationale frame rate).
>
> If Matroska timecode is 17 and Enable TimeScale Alignment is 0, than it is at 17 / (1000000000 / TimecodeScale ).
> If Matroska timecode is 17 and Enable TimeScale Alignment is 1, than it is at 2002 / 1200000 (nearest increment of the rationale frame rate).

I'm not sure I follow the math but using a flag in the
SimpleBlock/Block sounds interresting. But in effect wouldn't all
blocks have either 1 or 0 from a muxer version ? In that case it would
just mean the rationale timecode system should be used, if the demuxer
can deal with it. And basically that would mean it should be used if
present. Do we need to extra flag for that ?

> Ignoring the rationale frame rate and the Enable TimeScale Alignment bit would still decode to the accuracy supported by the TimecodeScale; however a player that uses the rationale frame rate and Enable TimeScale Alignment could round the timecode to the nearest increment of the frame rate.
>
> Such a feature would not be useful for mixed timebase content (documentary that mixes 24000/1001 and 30000/1001 framerates together).

As said by Jerome. In most cases there's an easy common factor. The
problem is really the example described early with a more variable
analog source. If it doesn't have a stable clock (why not?) then the
old system should be "good enough".

>>> Which real life scenario can not deal with such workaround?
>>
>> The question is: why do we want rationale number if we're going to
>> allow some form of rounding between the tracks that match the
>> timestamp scale precisely and the one that don't. IMO if there's
>> rounding let's just keep what we have.
>>
>>> Compared to the concerns in:
>>> https://lists.matroska.org/pipermail/matroska-devel/2012-October/004294.html
>>> It is per track so it resolves most concern
>>> For PCM, the corresponding video framerate is sometimes used and it would
>>> not be resolved correctly, right (i.e. at 30000/1001 fps, there are
>>> sometimes 1601 samples and sometimes 1602 samples and it is not possible to
>>> do the difference with the 16-bit timestamp having the millisecond
>>> precision)
>>> Note that my proposal is similar to the one in the mail having criticism
>>> about TimecodeScaleDenominator (proposal of TrackTimeBaseNumerator and
>>> TrackTimeBaseDenominator)
>
> […]
>
> Best Regards,
> Dave Rice
>

---

From: Jerome Martinez <[email-masked]>
Date: Sat, 4 Feb 2017 16:34:23 +0100

On 04/02/2017 15:39, Steve Lhomme wrote:
> [...]

In previous discussions there were some concerns about when a muxer 
starts to mux with constant frame rate config but the "real time" 
feeding changes during muxing and it is no more with this frame rate, 
the "rouding" would be still there from a demuxer point of view without 
the flag;
This flag is an answer to this concern, the algo can be deactivated 
during muxing even if it is not possible to change the Matroska header 
e.g. live stream.

---

From: Steve Lhomme <[email-masked]>
Date: Sat, 4 Feb 2017 16:36:07 +0100

Ok, I remember now. That sounds usable then.

Le 4 févr. 2017 16:34, "Jerome Martinez" <[email-masked]> a écrit :

> In previous discussions there were some concerns about when a muxer starts
> to mux with constant frame rate config but the "real time" feeding changes
> during muxing and it is no more with this frame rate, the "rouding" would
> be still there from a demuxer point of view without the flag;
> This flag is an answer to this concern, the algo can be deactivated during
> muxing even if it is not possible to change the Matroska header e.g. live
> stream.
>
>
>

---

From: Steve Lhomme <[email-masked]>
Date: Sun, 5 Feb 2017 14:59:10 +0100

I'll try to summarize the proposition as I understood it and how it
applies to real world example.

- We would keep the current TimecodeScale (TimestampScale) as it is to
scale from 1ns to something that gives enough range in a Cluster
(65536 values of 1ns * TimestampScale). The current default for
TimestampScale is 1.000.000 so that each tick (Block sample start) has
a 1ms precision (65s per Cluster with that amount).

- the worst case (from previous emails) seem to be sample precision
for 192kHz audio. That would need 5.2 µs precision. To achieve that
the TimestampScale would be around 5.000, giving a maximum Cluster
duration to 0.341ms which is probably not desirable. Given these data
are usually packed a factor of 8 sample per block seem reasonable
enough. That gives a 2.7s duration for a Cluster which is reasonable
for streaming and resilience.

- There are cases where using the same tick with both clocks might
create some rounding issues. Because the tick basis is the nanosecond
one, and it's usually not in a multiple of each channels' clock, and
we need a reasonable range for the Cluster, the precision is not a
nanoseconds but usually in 1ms (default values). Even 1ms is not good
enough for videos that have 30000/1001 timestamps (29.970029.. ms per
frame).

- The new system would work in parallel, reusing the same ticks but
using a rationale clock specific to each track.

- The chapter times are stored in non scaled format, ie in
nanoseconds. So we can be sample precise (with some rounding) for
192kHz sources easily.

Given that I made a short simulation of timestamps for a 30000/1001
video stream. https://docs.google.com/a/matroska.org/spreadsheets/d/1az-uc2Niklvm7Cy9fSpXGMIlWISUxlmwMboUFpY96sE/edit?usp=sharing
It turns out that the rounding we apply between the original timestamp
and the 1ms precision shifts the tick by minus one every ~34 frames.
So you increase the tick by 30 every frame but every 34 frame you
increase the tick by 29. Because of that the difference between the
Matroska clock and the pure rationale clock keeps drifting. This drift
cannot be represented by a bit. We do have at least 3 bits available
in SimpleBlock and Block. Given us 7 possible drifts allowed in a
single Cluster. From my example that allows 250 frames or 7.5s. That's
ample enough.

We still have to study the effect of reducing the TimestampScale to be
closer to the duration per frame and for audio.

One thing we need to take in account though is if it's going to create
problems when remuxing and cutting. Since the rounding compensation is
done on one basis for 0.0. If you cut a few frames the rounding will
probably need to be adjusted by the same number of frames. On the
other hand a muxer that is not aware of this system will drop it and
only the old value remains. A muxer that is aware will not that the
correct timestamp is the one from the rationale value and should
compute the tick value from that.

2017-02-04 16:36 GMT+01:00 Steve Lhomme <[email-masked]>:
>> during muxing and it is no more with this frame rate, the "rouding" would be
>> still there from a demuxer point of view without the flag;

---

From: hubblec4 <[email-masked]>
Date: Sun, 19 Feb 2017 14:17:03 +0100

Hi all

I hope this topic is not too boring and we can improve/implement 
important "use" notes.

In my chapterEditor thread was a topic "chapter.xml from makemkv" which 
generate non-ordered chapters with end time stamps. So far known to me 
there is no splitter which read the end times of non-ordered chapters.
In Mosu's new Chapter Editor is it not more possible to set the start 
times equal to the end times.
The DVDMenuXtractor generate ordered chapters without an end time stamp.

What is the correct behavior?

My suggestions are:
Data for non-ordered chapters can be set, but with a warning/info that 
this data will not be used.
Ordered-chapter end times can be set to "empty", equal or greater as the 
start time but never smaller as the start time.

The normal way:
End times are greater as the start times. A new virtual timeline use 
this chapter and takes the play duration of this chapter into account.

End time equal to start time:
Lav splitter ignores such ordered chapters, cause the play duration is 0.
Furthermore LAV-splitter recognized only Level 1 ordered chapters.
For this I have three possible ways in my mind.
1. Only as chapter mark
The chapter is shown to the user interface and you can jump to this 
point in the timeline. This is the same behaviour like ordered chapters 
from Level 2 and deeper. LAV-splitter ignores the "ordered" state and 
reads the start time only.

2. Use the ordered times from the nested ordered chapter (I prefer this 
option)
Nested-ordered-endtimes is a topic where I can't find infos. My theorie 
is that such chapters must have then
nested chapters and the first nested chapter start time must be the 
start/end time of the parent chapter.
A splitter should read than the nested-ordered play duration an takes 
into acount.

3. Endless loop
The frame which correspond to the start time of such a chapter is shown 
endless.
We could simulate a behaviour for menu's like in DVD's/Bluray's. Some 
user option in the player could be prohibited.

Empty end times:
This can be found in DVDMenuXtractor menu.xml output.
In principle we could use the same 3 possibilities here like for "End 
time equal to start time"

Another EBML element which makes not right sense to me, is 
ChapterSegmentEditionUID[6E][BC].
When such an UID is specified, the splitter should play this edition 
from the linked segment.
But an edition have not a start nor an end time!
What is with the specified chapter start and end time? This chapter 
duration must than match as a sum of all chapter play durations of the 
linked edition. What is when in the linked edition has chapters with 
linking to other editions?
Or is there an other method in use?

Thanks for reading and sorry for my bad english.

Best reagrds
Martin

_______________________________________________
Matroska-devel mailing list
[email-masked]
https://lists.matroska.org/cgi-bin/mailman/listinfo/matroska-devel
Read Matroska-Devel on GMane:http://dir.gmane.org/gmane.comp.multimedia.matroska.devel

---

From: Dave Rice <[email-masked]>
Date: Tue, 21 Feb 2017 10:37:45 -0500

Hi Martin,

> On Feb 19, 2017, at 8:17 AM, hubblec4 <[email-masked]> wrote:
> 
> Hi all
> 
> I hope this topic is not too boring and we can improve/implement important "use" notes.
> 
> In my chapterEditor thread was a topic "chapter.xml from makemkv" which generate non-ordered chapters with end time stamps. So far known to me there is no splitter which read the end times of non-ordered chapters.
> In Mosu's new Chapter Editor is it not more possible to set the start times equal to the end times.

Good point. Is there any purpose to ChapterTimeEnd if the chapters are non-ordered? Would it be fair to describe ChapterTimeEnd as only relevant to playback if EditionFlagOrdered=1.

> My suggestions are:
> Data for non-ordered chapters can be set, but with a warning/info that this data will not be used.

This could be said within the specification. It's a bit much to request that all Matroska editors also say it.

> Ordered-chapter end times can be set to "empty", equal or greater as the start time but never smaller as the start time.

That makes sense to me, though is there any meaning of ChapterTimeStart = ChapterTimeEnd? Is an empty ChapterTimeEnd semantically equivalent to ChapterTimeStart = ChapterTimeEnd or equivalent to ChapterTimeEnd of the last timestamp (or infinity)?

> The normal way: 
> End times are greater as the start times. A new virtual timeline use this chapter and takes the play duration of this chapter into account.
> 

+1, Having an ordered chapter where ChapterTimeStart = ChapterTimeEnd should be ignored.

> Furthermore LAV-splitter recognized only Level 1 ordered chapters.

Is there a use case for hierarchical ordered chapters? I'm not sure what a level 2 of an ordered chapter would do? Should an ordered chapter be permitted to have an ordered chapter as a child?

> The chapter is shown to the user interface and you can jump to this point in the timeline. This is the same behaviour like ordered chapters from Level 2 and deeper. LAV-splitter ignores the "ordered" state and reads the start time only.
> 
> 2. Use the ordered times from the nested ordered chapter (I prefer this option)
> Nested-ordered-endtimes is a topic where I can't find infos. My theorie is that such chapters must have then
> nested chapters and the first nested chapter start time must be the start/end time of the parent chapter.
> A splitter should read than the nested-ordered play duration an takes into acount.
> 
> 3. Endless loop
> The frame which correspond to the start time of such a chapter is shown endless.
> We could simulate a behaviour for menu's like in DVD's/Bluray's. Some user option in the player could be prohibited.
> 
> Empty end times:
> In principle we could use the same 3 possibilities here like for "End time equal to start time"
> 
> 
> Another EBML element which makes not right sense to me, is ChapterSegmentEditionUID[6E][BC].
> When such an UID is specified, the splitter should play this edition from the linked segment.
> What is with the specified chapter start and end time? This chapter duration must than match as a sum of all chapter play durations of the linked edition. What is when in the linked edition has chapters with linking to other editions?

I've never used ChapterSegmentEditionUID but it does seem valuable. The specification is not so clear, but my presumption would be that you'd add the ChapterTimeStart of the chapter using ChapterSegmentEditionUID to the starting time of the related Segment's Edition.

> Thanks for reading and sorry for my bad english.

No worries. I think there is a lot to discuss regarding chapters. Honestly I often go to http://mod16.org/hurfdurf/?p=8 <http://mod16.org/hurfdurf/?p=8> for information about Matroska chapters rather than the more official but sparse documentation on matroska.org <http://matroska.org/>. Perhaps first we could clarify the documentation of the chapter elements in the Matroska Schema <https://github.com/Matroska-Org/matroska-specification/blob/master/ebml_matroska.xml> and then extend the chapter documentation <https://github.com/Matroska-Org/matroska-specification/blob/master/chapters.md> to give descriptive sections on:
- the use of Non-Ordered Chapters
- the use of Ordered Chapters
- how to handle Chapters that reference external Segments and/or Editions
- also a definition for the xml expression of chapters

Dave Rice

> [email-masked] <mailto:[email-masked]>
> https://lists.matroska.org/cgi-bin/mailman/listinfo/matroska-devel <https://lists.matroska.org/cgi-bin/mailman/listinfo/matroska-devel>
> Read Matroska-Devel on GMane: http://dir.gmane.org/gmane.comp.multimedia.matroska.devel <http://dir.gmane.org/gmane.comp.multimedia.matroska.devel>_______________________________________________
> Cellar mailing list
> [email-masked]

---

From: hubblec4 <[email-masked]>
Date: Wed, 22 Feb 2017 14:06:25 +0100

Hi all

Am 21.02.2017 um 16:37 schrieb Dave Rice:
> Hi Martin,
>
>> On Feb 19, 2017, at 8:17 AM, hubblec4 <[email-masked] 
>> <mailto:[email-masked]>> wrote:
>>
>> Hi all
>>
>> In my chapterEditor thread was a topic "chapter.xml from makemkv" 
>> which generate non-ordered chapters with end time stamps. So far 
>> known to me there is no splitter which read the end times of 
>> non-ordered chapters.
> Good point. Is there any purpose to ChapterTimeEnd if the chapters are 
> non-ordered?

No there is no purpose! But as info it is maybe important.
A short description:
When you create mkv's with makemkv and you need ordered 
editions/chapters, so it is easy to build an ordered editon.
In practice:
You load such an mkv to Mosu's Chapter Editor(CE), then enable the 
EditionFlagOrdered=1 and all fine, cause the end times was allready 
exists. If there are no end times you must enter all end times manually 
(a lot of work).
Mosu's CE has no function to create/calculate ordered end times, but my 
chaperEditor(cE) has such a function.
An other problem of ordered chapter end times, is the last chapter. The 
end time of the last chapter must be the duration of the video(in normal 
situations), but this info is nowhere strored in a chapter.xml (only in 
an mkv where it is possible to extract the play duration info).

> Would it be fair to describe ChapterTimeEnd as only relevant to 
> playback if EditionFlagOrdered=1.

Yes it would be fair. Every splitter will ignore this EBML elements.
But there are more EBML elements which affects that:
- ChapterTimeEnd
- SegmentUID
- SegmentEditionUID
- the whole ChapProcess
(ChapProcess: I have found it only in DVDMenuXtractor(DMX) output.xml 
files. Without ordered chapters I think it makes not sense, but I'm not 
100% sure)

> This could be said within the specification. It's a bit much to 
> request that all Matroska editors also say it.

+1 Such an info is very important(my opinion). All "new" Matroska 
editors can use this info an old one could be updated(or not).

>
>> Ordered-chapter end times can be set to "empty", equal or greater as 
>> the start time but never smaller as the start time.
>
> That makes sense to me, though is there any meaning of 
> ChapterTimeStart = ChapterTimeEnd? Is an empty ChapterTimeEnd 
> semantically equivalent to ChapterTimeStart = ChapterTimeEnd or 
> equivalent to ChapterTimeEnd of the last timestamp (or infinity)?
>

For the moment is no meaning defined for ChapterTimeStart(CTS) = 
ChapterTimeEnd(CTE) but we could/should.
In pratice such a chapter will be ignored (this makes the 
parsing/splitting process easier).
But I say that is not good and we lose flexibility.

> +1, Having an ordered chapter where ChapterTimeStart = ChapterTimeEnd 

Why should such a chapter be ignored? Ok, there is no duration and 
irrelevant for the new virtual timeline, but you can have other info in 
such chapters:
1. a deeper ordered chapter where you can find a duration
2. ChapProcess can be used
3. only use as chapter mark

> Is there a use case for hierarchical ordered chapters? I'm not sure 
> what a level 2 of an ordered chapter would do? Should an ordered 
> chapter be permitted to have an ordered chapter as a child?

Yes there are use case's (only in an early VLC version, but not really 
for practice).
DVDMenuXtractor output.xml for VMG or VTS or the VTSM.
A hugh topic and to much to discribe here.
When it is ok please download my cE 
http://forum.gleitz.info/attachment.php?attachmentid=99288&d=1486654181
You find in this email a 7zip with DMX.xml files from the Aliens DVD.

My cE reads all the XML comments und have a very nice overview about the 
hierarchical Matroska-DVD-Menu structure.
Then you will see that the level 5 ordered chapters are used for the new 
timeline.
Cause, all levels between 1 and 4 are reserved for the DVD-menu-structure.
(I have just noticed that DMX.xml files can have a start time stamp and 
an end time stamp with set to 00:00:00.000000000)

There are one pratice at the moment. My Matroska Menu Editor:
When I combine many files of an episode in many editions in a 
menu-start.mkv (all with Chapter-Segment-Linking)
and I will naviagte through the editions/chapters it is not really handy.
For LAV splitter: In the tray icon you see many editions(thats fine). 
But such an edition(with 20 or more files) has a lot of chapters.
To get a better overview which chapters belong to an episode.mkv I use 
nested chapters.
And therfore is a complex algorithm necessary cause all chapters have to 
be defined twice.
First chapter (level 1) for the ordering and the new timeline duration 
and only the first chapter of an episode.mkv is visible, all others must 
be set to hidden=true.
Second chapter (level 2) with the same time stamps(at the moment it is 
enough when the start time is defined), which is used as chapter mark/jump.

Here a link to a small samle with 3 cutted epsiode.mkv files
http://forum.videohelp.com/attachments/40602-1487505284/MatroskaMenu_Chapter-Segment-Linking.7z
Here an other menu.xml sample.
http://forum.gleitz.info/attachment.php?attachmentid=99279&d=1485686941

I reported a playback bug of such menu.xml files to VLC forum and it was 
fixed.
https://forum.videolan.org/viewtopic.php?f=2&t=137611

In the videohelp forum I have discussed this with a member and he his 
the same opinion like me.

http://forum.videohelp.com/threads/368560-chapterEditor%28OGG-XML-TTXT-m-AVCHD-m-editions-mkv-Matroska-Menu%29-names-CLI?p=2425793&viewfull=1#post2425793

A ordered chapter with a valid duration(greater then 0) should not have 
nested-ordered chapters, cause it makes not sense. The duration of the 
nested-ordered chapter is not defined in the parent chapter-duration.

But for ordered chapters that have a 0 duration or no end time stamp a 
nested-ordered chapter can/should be used.

>> point in the timeline. This is the same behaviour like ordered 
>> chapters from Level 2 and deeper. LAV-splitter ignores the "ordered" 
>> state and reads the start time only.
>>
>> Nested-ordered-endtimes is a topic where I can't find infos. My 
>> theorie is that such chapters must have then
>> duration must than match as a sum of all chapter play durations of 
>> the linked edition. What is when in the linked edition has chapters 
>> with linking to other editions?
> I've never used ChapterSegmentEditionUID but it does seem valuable. 
> The specification is not so clear, but my presumption would be that 
> you'd add the ChapterTimeStart of the chapter using 
> ChapterSegmentEditionUID to the starting time of the related Segment's 
> Edition.

(Sorry I don't right understand)

The specs say's
"The EditionUID to play from the segment linked in ChapterSegmentUID."

A splitter generate the virtual time line but the duration info is not 
available/wrong, or the splitter must calculate it first.

> No worries. I think there is a lot to discuss regarding chapters. 
> Honestly I often go to http://mod16.org/hurfdurf/?p=8 for information 
> about Matroska chapters rather than the more official but sparse 
> documentation on matroska.org <http://matroska.org>.
All my Matroska features in cE comes through very often reading this 
wonderful page.

> Perhaps first we could clarify the documentation of the chapter 
> elements in the Matroska Schema 
> <https://github.com/Matroska-Org/matroska-specification/blob/master/ebml_matroska.xml> and 
> then extend the chapter documentation 
> <https://github.com/Matroska-Org/matroska-specification/blob/master/chapters.md> to 
> give descriptive sections on:

I will help where I can.

Best regards
Martin Below

>> [email-masked] <mailto:[email-masked]>

---

From: Dave Rice <[email-masked]>
Date: Wed, 22 Feb 2017 09:34:35 -0500

Hi,

> On Feb 22, 2017, at 8:06 AM, hubblec4 <[email-masked]> wrote:
> 
> Hi all
> 
>>> On Feb 19, 2017, at 8:17 AM, hubblec4 <[email-masked] <mailto:[email-masked]>> wrote:
>>> 
>>> Hi all
>>> 
>> Good point. Is there any purpose to ChapterTimeEnd if the chapters are non-ordered?
> 

I thought about this more later. When the playback point is greater than or equal to the chapter start time and less than the chapter end time, a player could display the title of that chapter. So for instance

chapter1 00:01-00:03
chapter2 00:05-00:07

At 00:02, the player could display that the playback is within chapter1.
At 00:04, the player could display that it it outside a chapter.

If the end times are omitted, then 00:04 would be inferred to be part of chapter1.
Perhaps we could define that chapters are ranges rather than points in time and that if omitted then the end time is whatever comes first, the next sibling chapter start in time or the end of the segment.

> A short description:
> When you create mkv's with makemkv and you need ordered editions/chapters, so it is easy to build an ordered editon.
> In practice: 
> You load such an mkv to Mosu's Chapter Editor(CE), then enable the EditionFlagOrdered=1 and all fine, cause the end times was allready exists. If there are no end times you must enter all end times manually (a lot of work). 
> Mosu's CE has no function to create/calculate ordered end times, but my chaperEditor(cE) has such a function.
> An other problem of ordered chapter end times, is the last chapter. The end time of the last chapter must be the duration of the video(in normal situations), but this info is nowhere strored in a chapter.xml (only in an mkv where it is possible to extract the play duration info).

mkvpropedit supports chapter xml with an ordered edition without an end time. I use this to create one chapter edition which refers to the entire segment, whereas another chapter edition references only part of it. So it is like:

<EditionEntry>
    <EditionFlagOrdered>1</EditionFlagOrdered>
    <EditionFlagHidden>0</EditionFlagHidden>
    <EditionFlagDefault>0</EditionFlagDefault>
    <EditionUID>12338659363134957115</EditionUID>
    <ChapterAtom>
      <ChapterTimeStart>00:00:00.000000000</ChapterTimeStart>
      <ChapterFlagHidden>0</ChapterFlagHidden>
      <ChapterFlagEnabled>1</ChapterFlagEnabled>
      <ChapterUID>14022955415078936670</ChapterUID>
      <ChapterDisplay>
        <ChapterString>Full</ChapterString>
        <ChapterLanguage>eng</ChapterLanguage>
      </ChapterDisplay>
    </ChapterAtom>
  </EditionEntry>

Any issue with no ChapterTimeEnd in an ordered edition?

>> Would it be fair to describe ChapterTimeEnd as only relevant to playback if EditionFlagOrdered=1.
> 
> (ChapProcess: I have found it only in DVDMenuXtractor(DMX) output.xml files. Without ordered chapters I think it makes not sense, but I'm not 100% sure)

See above use case: to allow a player to say what chapter the player is 'within'.

> +1 Such an info is very important(my opinion). All "new" Matroska editors can use this info an old one could be updated(or not).
> 
>> 
> For the moment is no meaning defined for ChapterTimeStart(CTS) = ChapterTimeEnd(CTE) but we could/should.
> In pratice such a chapter will be ignored (this makes the parsing/splitting process easier).
> Why should such a chapter be ignored? Ok, there is no duration and irrelevant for the new virtual timeline, but you can have other info in such chapters:
> Yes there are use case's (only in an early VLC version, but not really for practice).
> When it is ok please download my cE http://forum.gleitz.info/attachment.php?attachmentid=99288&d=1486654181 <http://forum.gleitz.info/attachment.php?attachmentid=99288&d=1486654181>
> My cE reads all the XML comments und have a very nice overview about the hierarchical Matroska-DVD-Menu structure.
> Then you will see that the level 5 ordered chapters are used for the new timeline.
> (I have just noticed that DMX.xml files can have a start time stamp and an end time stamp with set to 00:00:00.000000000)
> 
> When I combine many files of an episode in many editions in a menu-start.mkv (all with Chapter-Segment-Linking)
> For LAV splitter: In the tray icon you see many editions(thats fine). But such an edition(with 20 or more files) has a lot of chapters. 
> To get a better overview which chapters belong to an episode.mkv I use nested chapters. 
> And therfore is a complex algorithm necessary cause all chapters have to be defined twice.
> First chapter (level 1) for the ordering and the new timeline duration and only the first chapter of an episode.mkv is visible, all others must be set to hidden=true.
> Second chapter (level 2) with the same time stamps(at the moment it is enough when the start time is defined), which is used as chapter mark/jump.
> 
> http://forum.videohelp.com/attachments/40602-1487505284/MatroskaMenu_Chapter-Segment-Linking.7z <http://forum.videohelp.com/attachments/40602-1487505284/MatroskaMenu_Chapter-Segment-Linking.7z>
> http://forum.gleitz.info/attachment.php?attachmentid=99279&d=1485686941 <http://forum.gleitz.info/attachment.php?attachmentid=99279&d=1485686941>
> 
> I reported a playback bug of such menu.xml files to VLC forum and it was fixed.
> https://forum.videolan.org/viewtopic.php?f=2&t=137611 <https://forum.videolan.org/viewtopic.php?f=2&t=137611>  
> 
> 
> In the videohelp forum I have discussed this with a member and he his the same opinion like me.
> 
> http://forum.videohelp.com/threads/368560-chapterEditor%28OGG-XML-TTXT-m-AVCHD-m-editions-mkv-Matroska-Menu%29-names-CLI?p=2425793&viewfull=1#post2425793 <http://forum.videohelp.com/threads/368560-chapterEditor%28OGG-XML-TTXT-m-AVCHD-m-editions-mkv-Matroska-Menu%29-names-CLI?p=2425793&viewfull=1#post2425793>
> 
> 
> A ordered chapter with a valid duration(greater then 0) should not have nested-ordered chapters, cause it makes not sense. The duration of the nested-ordered chapter is not defined in the parent chapter-duration.
> 
> But for ordered chapters that have a 0 duration or no end time stamp a nested-ordered chapter can/should be used.   
> 
> 
>> 
> A splitter generate the virtual time line but the duration info is not available/wrong, or the splitter must calculate it first.
> 
> 
>> 
>> No worries. I think there is a lot to discuss regarding chapters. Honestly I often go to http://mod16.org/hurfdurf/?p=8 <http://mod16.org/hurfdurf/?p=8> for information about Matroska chapters rather than the more official but sparse documentation on matroska.org <http://matroska.org/>.
> All my Matroska features in cE comes through very often reading this wonderful page.

Then perhaps some of the statements of that page should be included within the specification. ;)

>> Perhaps first we could clarify the documentation of the chapter elements in the Matroska Schema <https://github.com/Matroska-Org/matroska-specification/blob/master/ebml_matroska.xml> and then extend the chapter documentation <https://github.com/Matroska-Org/matroska-specification/blob/master/chapters.md> to give descriptive sections on:

Dave Rice

>>> https://www.ietf.org/mailman/listinfo/cellar <https://www.ietf.org/mailman/listinfo/cellar>
>> 
>> 
>> 
>> https://www.ietf.org/mailman/listinfo/cellar <https://www.ietf.org/mailman/listinfo/cellar>
> 
> <DVDMenuXtractor.7z>_______________________________________________
> Read Matroska-Devel on GMane: http://dir.gmane.org/gmane.comp.multimedia.matroska.devel

---

From: hubblec4 <[email-masked]>
Date: Wed, 22 Feb 2017 16:14:56 +0100

Hi Dave

Am 22.02.2017 um 15:34 schrieb Dave Rice:
> Hi,
>
>> On Feb 22, 2017, at 8:06 AM, hubblec4 <[email-masked] 
> I thought about this more later. When the playback point is greater 
> than or equal to the chapter start time and less than the chapter end 
> time, a player could display the title of that chapter. So for instance
>
> chapter1 00:01-00:03
> chapter2 00:05-00:07
>

Your 2 chapters (use in an ordered edition) will interpereted to a new 
virtual timeline as follows:

chapter1: start time 00:00 - 2minutes duration - end time 00:02(not 
included)
chapter2: start time 00:02 - 2minutes duration - end time 00:04(not 
included)

At 00:02, the player could display that the playback is within chapter2 
ot chapter1.
At 00:04, the chapter point is behind the new virtual timeline.

ChapterTime stamps in ordered editions are not the defined time stamps, 
all time stamps must be recalculated to virtual time stamps.
Only for simple ordered editions (1 edition with continuous ordered 
chapters) are defined time stamps equal to virtual time stamps.

> Perhaps we could define that chapters are ranges rather than points in 
> time and that if omitted then the end time is whatever comes first, 
> the next sibling chapter start in time or the end of the segment.
>

No 100% clear what you mean, but it sound not really good to me.

>> exists. If there are no end times you must enter all end times 
>> manually (a lot of work).
>> An other problem of ordered chapter end times, is the last chapter. 
>> The end time of the last chapter must be the duration of the video(in 
>> normal situations), but this info is nowhere strored in a chapter.xml 
>> (only in an mkv where it is possible to extract the play duration info).
>
> mkvpropedit supports chapter xml with an ordered edition without an 
> end time. I use this to create one chapter edition which refers to the 
> entire segment, whereas another chapter edition references only part 
> of it. So it is like:
>
> <EditionEntry>

This case is only working with a "normal mkv", like a single episode.
But what is with multi-Edition-Title mkv's created from multi-Edition 
Blurays?
The end of a segment is different to the end of an edition.
In such mkv's you have one first edition with the ordered m2ts contents 
and at the end are all other m2ts files.
In this case your chapter-sample plays to long.

Martin Below

---

From: hubblec4 <[email-masked]>
Date: Thu, 23 Feb 2017 12:24:58 +0100

Hi Dave

Am 22.02.2017 um 15:34 schrieb Dave Rice:
>
>>>
>>>> Ordered-chapter end times can be set to "empty", equal or greater 
>>>> as the start time but never smaller as the start time.
>>>
>>>> End times are greater as the start times. A new virtual timeline 
>>>> use this chapter and takes the play duration of this chapter into 
>>>> account.
>>>>
>> irrelevant for the new virtual timeline, but you can have other info 
>> in such chapters:

In the Matroska specs(Chapters) section "DVD menu", the following is 
written:
"Each level of a chapter corresponds to a logical level in the DVD 
system that is stored in the first octet of the ChapProcessPrivate."

Like in DMX output.xml files are the start time stamps equal to the end 
time stamps.
When we forbidden such ordered chapters we break the Matroska-DVD-Menu 
system.
It is recommend to say ordered chapters with start time equal to the end 
time, must have a nested-ordered chapter.
Otherwise such a chapter is used for chapter mark only (or for menu 
navigation commands stored in the ChapProcess section).

I'm not really interessted in Matroska-DVD-Menu system, it is to hugh, 
to complex.
But I will bring to life the Matroska-Native-Menu system. I'm sure I 
will need nested-ordered-chapters for this.

Best regards
Martin Below

---

From: hubblec4 <[email-masked]>
Date: Fri, 24 Feb 2017 12:31:57 +0100

Hi Dave

I found this on github:
VLC supports Matroska-DVD-Menu system, uses "SegmentFamily" and read all 
the Chapter-Process commands.
Here a link to the source code of the mkv-demuxer.
https://github.com/videolan/vlc/tree/master/modules/demux/mkv

Martin Below

Am 22.02.2017 um 16:14 schrieb hubblec4:
> At 00:02, the player could display that the playback is within 
> chapter2 ot chapter1.
> ChapterTime stamps in ordered editions are not the defined time 
> stamps, all time stamps must be recalculated to virtual time stamps.
>> Perhaps we could define that chapters are ranges rather than points 
>> in time and that if omitted then the end time is whatever comes 
>> first, the next sibling chapter start in time or the end of the segment.
>>
>
>>> The end time of the last chapter must be the duration of the 
>>> video(in normal situations), but this info is nowhere strored in a 
>>> chapter.xml (only in an mkv where it is possible to extract the play 
>>> duration info).
>>
>> end time. I use this to create one chapter edition which refers to 
>> the entire segment, whereas another chapter edition references only 
>> part of it. So it is like:
>>
>> <EditionEntry>
> In such mkv's you have one first edition with the ordered m2ts 
> contents and at the end are all other m2ts files.