Subject: Re: [Cellar] Language codes in Matroska and EBML

From: Dave Rice <[email-masked]>
Date: Sun, 26 Feb 2017 10:59:22 -0500

Hi Doug and others,

I created a pull request to add an BCP47 contextualized Language Element everywhere there was an existing ISO 639-2 Language Element, and updated documentation to give precedence to the BCP47 version when available. https://github.com/Matroska-Org/matroska-specification/pull/103/files <https://github.com/Matroska-Org/matroska-specification/pull/103>.

For the old ISO 639-2 Elements I added a disclaimer like this:

> This Element MUST be ignored if the LanguageIETF Element is used in the same TrackEntry.

The new BCP 47 Language Elements are defined like this:

> Specifies the language of the track according to <a href="https://tools.ietf.org/html/bcp47">BCP 47</a> and using the <a href="http://www.iana.com/assignments/language-subtag-registry/language-subtag-registry">IANA Language Subtag Registry</a>.

For reference there are three Language Elements in Matroska: Language (to provide the Language of a related Track), TagLanguage (to give the name used in a piece of metadata), ChapLanguage (to give the language of the label of a Chapter).

Comments?

Dave Rice

> On Aug 2, 2016, at 1:56 PM, Doug Ewell <[email-masked]> wrote:
> 
> Thanks to Jérôme and Moritz for pointing to the January discussion
> about language coding.
> 
> It looks like the discussion originally focused on expanding the
> existing data element in Matroska to incorporate ISO 639-3. Later
> discussion was on truly adopting BCP 47, and maintaining compatibility
> with existing Matroska implementations by creating a new, BCP
> 47-compliant language element that would take precedence over the
> existing one. This seems sensible and would allow data producers to
> migrate to the BCP 47 approach over time.
> 
> An important detail about BCP 47 is that compliant applications use the
> IANA Language Subtag Registry [1] as their canonical source for language
> subtags (codes) and other subtags, and NOT the ISO standards directly.
> The Registry is maintained carefully through IANA to be compatible with
> the ISO standards, except when doing so would cause conflicts or
> incompatibility.
> 
> For example, the chaos that ensued in 2003 when ISO 3166/MA reassigned
> the retired code element [CS] to mean "Serbia and Montenegro," when
> numerous applications were still using it to mean "Czechoslovakia,"
> would have caused very little trouble if BCP 47 had existed, because a
> numeric replacement code would have been chosen for Serbia and
> Montenegro to resolve the conflict.
> 
> So it's important to steer clear of statements like "RFC 5646 recommends
> to use ISO 639 when it is possible" and "RFC 5646 recommends 2-letter
> codes when 3-letters codes are not necessary," because what RFC 5646
> actually says is to use the subtags that are in the Registry.
> 
> Another important point is that parsing a BCP 47 tag to remove all but
> the primary language subtag is dead simple: just stop right before the
> first hyphen, if any. So "sr-Latin-ME" becomes "sr". There is no need to
> worry about whether the primary language subtag is 2 or 3 letters long.
> The only exceptions are for the grandfathered tags "i-default" and
> "i-mingo" and for private-use tags starting with "x-", all of which are
> unlikely to occur in multimedia data. There are well-defined matching
> mechanisms for more complex scenarios; see RFC 4647.
> 
> BCP 47 (sometimes as "RFC 5646" or its predecessors) is referenced
> normatively in more than 60 RFCs, and informatively in another 20, plus
> numerous drafts. So there is no question that it is widely supported!
> 
> Please feel free to contact the ietf-languages mailing list [2] with any
> questions concerning BCP 47.
> 
> [1] http://www.iana.com/assignments/language-subtag-registry
> [2] http://www.alvestrand.no/mailman/listinfo/ietf-languages
> 
> 
> --
> Doug Ewell | Thornton, CO, US | ewellic.org
> 
> _______________________________________________
> Cellar mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/cellar