From nobody Sun Sep 4 05:46:05 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CDB6A12B204 for ; Sun, 4 Sep 2016 05:46:03 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1BtHy8c-5h4h for ; Sun, 4 Sep 2016 05:46:02 -0700 (PDT) Received: from mail-yb0-x229.google.com (mail-yb0-x229.google.com [IPv6:2607:f8b0:4002:c09::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2DE1912B203 for ; Sun, 4 Sep 2016 05:46:02 -0700 (PDT) Received: by mail-yb0-x229.google.com with SMTP id x93so53159665ybh.1 for ; Sun, 04 Sep 2016 05:46:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=mmO1pq6q+VPT8Do60DFl0ES+80IHRzLVkDx3k+wOOcs=; b=tRTGR8qm2ZMNLd7WyK3+fWwCEZgFj3oAuwBYQ6bmHMrTsTYEAL1fQACiIesQm/91fY fkSlGWOcvFE2ED2l2Ez91hAiJI8O/C57GIA0U2wfzj/OoJ51a1qjwArBG+lJL5t/VPHN 7/sJ4oMYwcNv3uW7pela3UeEmgH3XD4lk5iOu+o98lk5X8mJFMnnTgXuy/wKkniMBD5N 1a3FZ2nm61tLcSqli6uzIzFV7lTcLKdg1GBXPrCt5oZavkjNHsPtGJ3C1/cpGs8vbqFi bnpxShCMsmt5WUscXJGmDqu76jiYYBtTNv0qn92BBTIM8Ru72YLiKOeXc9hkxjB3tnAM mjLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=mmO1pq6q+VPT8Do60DFl0ES+80IHRzLVkDx3k+wOOcs=; b=jiRcPYpjD0YreDwmRkQvA65QNsHZFWF48r1+rnQUEqYw2KdARQvbJclKsD+JZqtcEw XY8ZHLcCOygeUczB0nLcXnszHI20c/eEbr2rNgrhvL+bGLv9bxB/i/X6wjEDJy6d+t60 TSTlHTenqLg16SXuI4n53d8J51qAvusZ72qrjY/+hTnB0vTvBlCkphliD+8RqvmLa14S 5ay5u0bGCeM+uPw5fWjcP6pcO6waSLfWtJBwHbliR7Ch+43R+4B/FAkjrIrbAq7DWhel o60Pu5OyYE8WvOI8GmJ5skbjsoiBS9Zp39CMJUSQNh86i0B182GEDubByjWJLHbnZFiY ZKpw== X-Gm-Message-State: AE9vXwNHIRuhkHYvOxKsFynr38vOSAy6+PoZ+wP/x4Xvplw506NmZmoMHeuj8mG+Azi1XYhx4sv4nxaCpJO8FQ== X-Received: by 10.37.223.149 with SMTP id w143mr24411320ybg.22.1472993161226; Sun, 04 Sep 2016 05:46:01 -0700 (PDT) MIME-Version: 1.0 Received: by 10.83.20.135 with HTTP; Sun, 4 Sep 2016 05:46:00 -0700 (PDT) In-Reply-To: References: From: Steve Lhomme Date: Sun, 4 Sep 2016 14:46:00 +0200 Message-ID: To: Michael Bradshaw Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: Cc: Codec Encoding for LossLess Archiving and Realtime transmission Subject: Re: [Cellar] 360/VR videos in Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Sep 2016 12:46:04 -0000 2016-08-30 18:40 GMT+02:00 Michael Bradshaw : > Matroska currently doesn't have a way to specify projection information f= or > 360/VR videos. Google's Spherical Video V2 RFC draft was just updated a f= ew > days ago to include some new elements specifying projection and viewer po= se > information[1]. > > These proposed elements haven't made their way over to the WebM > specification (and I haven't asked the WebM team if they intend to > incorporate them), but I think it raises a good question about Matroska > support for these kinds of things. > > I don't know of any past work in specifying projection information in > Matroska. Assuming there hasn't been any, is there interest in adding it > (I'm assuming the answer is still yes, from those I asked in Berlin)? Do = the Since I'm currently adding 360=C2=B0 support in VLC the answer is a definite YES. So far we only have MP4 sources. For now we only support some XML embedded in an MP4 UUID XML360 box. The XML is similar to the one defined here: https://github.com/google/spatial-media/blob/master/docs/spherical-video-rf= c.md#user-content-appendix-1---global-metadata-sample There's already hardware that saves directly to this format. > elements from the Spherical Video V2 RFC draft look like good candidates? It's a good start ;) I think the 0.0 default value should be 0x0p+1. Also I think the ProjectionType value 0 should be None corresponding to the current situation when the field doesn't exist. Are the Yaw/Pitch/Roll used in practice ?; > (For the record, I'm just breaking the ice here and getting the conversat= ion > started; this isn't a proposal to adopt the new elements just quite yet) > > [1]: > https://github.com/google/spatial-media/blob/master/docs/spherical-video-= v2-rfc.md#webm-matroska > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > --=20 Steve Lhomme Matroska association Chairman From nobody Sun Sep 4 06:09:54 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 74C7112B208 for ; Sun, 4 Sep 2016 06:09:53 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.6 X-Spam-Level: X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ze5SLme0ko2q for ; Sun, 4 Sep 2016 06:09:50 -0700 (PDT) Received: from mail-yw0-x234.google.com (mail-yw0-x234.google.com [IPv6:2607:f8b0:4002:c05::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BA21312B20B for ; Sun, 4 Sep 2016 06:09:50 -0700 (PDT) Received: by mail-yw0-x234.google.com with SMTP id g192so36103027ywh.1 for ; Sun, 04 Sep 2016 06:09:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=OPno2+8c0MgUpqihr+SFJizDi855OO1m74sxm09H/h8=; b=jIoP7g7Fc10Lmj9APEBTDugsNrIMxcZp3nLox3OOcdWK7g6VDCAk7p5P3ErLQaXnRv m0EKPoRFqk8QS3LRMTAczJsO2WxCogEcYXZBpZNiEh+l29vERy694GTSzJD2Ce2dqCbe pLZO2y/56j8Y33lrnJmMiAco7nqP3w3tdq1YDsZfjp/xAlmByH0LDBpDuUscNdBrLtLL VvZNzywvYhnBmSo/S1UVDX2JBGKDkoaP3PtKKljYtnLSPGxPfAaIVf8J2tORU6EvhBI4 a7FiGMO/nFticrIyE9G1m0SOI2GIuQYpVI/34xHPR4IRB57lkHYEof3AZ0URsaf+wdAf pyVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=OPno2+8c0MgUpqihr+SFJizDi855OO1m74sxm09H/h8=; b=HFwWk8d23re5SFGv4yRHnoxo7oQLArkECMcs4cnHCv8sqka9MrsiNABVIQgYZOB5hZ o1aGPs9fybtXMZ7PMpzuVednBDpMICKnu4ewy+0OWSxXPlL9qEfK6xHTZ68JbPVjWMuk InkAz5LIyQMPlWNmUvj7oqQx++bXRtZiT4spc//jMi+BD1nt722CD7o8+Wv0D4oaf2Ay cUYyhRiDaar6jTrYX1J3dks51fYAGZf9rMHrXWJ4yPQvfU4fPGAJdqPuowcsMU6cTBvd NfhGQHrNY2BHDlPLpNNltI/3tFKnnEgJm2PwxiGlr8pzbNX6yQsA61Lamay+Rp/MzNZ9 maww== X-Gm-Message-State: AE9vXwMGo6igRtX+u924UZVC7T/NbjcwVk2wf9TyHeA0nfWjEbW9TGEMmeHqvFoSlTrng9XJFDZ7MeASYuikbw== X-Received: by 10.13.205.70 with SMTP id p67mr24290823ywd.112.1472994589870; Sun, 04 Sep 2016 06:09:49 -0700 (PDT) MIME-Version: 1.0 Received: by 10.83.20.135 with HTTP; Sun, 4 Sep 2016 06:09:49 -0700 (PDT) In-Reply-To: References: From: Steve Lhomme Date: Sun, 4 Sep 2016 15:09:49 +0200 Message-ID: To: Michael Bradshaw Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: Cc: Codec Encoding for LossLess Archiving and Realtime transmission Subject: Re: [Cellar] 360/VR videos in Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Sep 2016 13:09:53 -0000 2016-09-04 14:46 GMT+02:00 Steve Lhomme : > 2016-08-30 18:40 GMT+02:00 Michael Bradshaw : >> Matroska currently doesn't have a way to specify projection information = for >> 360/VR videos. Google's Spherical Video V2 RFC draft was just updated a = few >> days ago to include some new elements specifying projection and viewer p= ose >> information[1]. >> >> These proposed elements haven't made their way over to the WebM >> specification (and I haven't asked the WebM team if they intend to >> incorporate them), but I think it raises a good question about Matroska >> support for these kinds of things. >> >> I don't know of any past work in specifying projection information in >> Matroska. Assuming there hasn't been any, is there interest in adding it >> (I'm assuming the answer is still yes, from those I asked in Berlin)? Do= the > > Since I'm currently adding 360=C2=B0 support in VLC the answer is a > definite YES. So far we only have MP4 sources. For now we only support > some XML embedded in an MP4 UUID XML360 box. The XML is similar to the > one defined here: > https://github.com/google/spatial-media/blob/master/docs/spherical-video-= rfc.md#user-content-appendix-1---global-metadata-sample > > There's already hardware that saves directly to this format. > >> elements from the Spherical Video V2 RFC draft look like good candidates= ? > > It's a good start ;) > > I think the 0.0 default value should be 0x0p+1. > > Also I think the ProjectionType value 0 should be None corresponding > to the current situation when the field doesn't exist. Never mind, Rectangular (0) is actually the regular/flat projection. > Are the Yaw/Pitch/Roll used in practice ?; > >> (For the record, I'm just breaking the ice here and getting the conversa= tion >> started; this isn't a proposal to adopt the new elements just quite yet) >> >> [1]: >> https://github.com/google/spatial-media/blob/master/docs/spherical-video= -v2-rfc.md#webm-matroska >> >> _______________________________________________ >> Cellar mailing list >> Cellar@ietf.org >> https://www.ietf.org/mailman/listinfo/cellar >> > > > > -- > Steve Lhomme > Matroska association Chairman --=20 Steve Lhomme Matroska association Chairman From nobody Sun Sep 4 06:35:38 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E458A12B21E for ; Sun, 4 Sep 2016 06:35:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.6 X-Spam-Level: X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wtgdrrRajmhK for ; Sun, 4 Sep 2016 06:35:35 -0700 (PDT) Received: from mail-yw0-x236.google.com (mail-yw0-x236.google.com [IPv6:2607:f8b0:4002:c05::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0F77212B21C for ; Sun, 4 Sep 2016 06:35:35 -0700 (PDT) Received: by mail-yw0-x236.google.com with SMTP id j1so5892978ywb.2 for ; Sun, 04 Sep 2016 06:35:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=WtwKYRF/VXVyM8bwOffglVFhYSSP07jbPpVvi/USM7o=; b=gsFzrsG+4/zqtjStGabTqIEey1kUwH07Ln10boSo20dtvxArJnpFZZ0i4ylAwSHewn K4iXHqfLK0l1KoIvCMygK530F0gRcVoU0To9hakbGr0QPXCPMq8ukrkZhoVZvdEAmmVR mIMON7u9H2PO0hYxuP/4yDarDZnrj4fJ174eoXH66ob8e0eA0JOY95leGTymwfPue++v a/X9pwj69j4UrfSDh1hGq3wz8hkY4nomPnBUgRXUZ8e/LjGaEj9O1433jLn8IzexASEd xtpP61Zaxk00Uashdz/ILn0+d7NDd6DucEh3Bl+mgIO740kUfSuhtVDzMoQz00aaWhin OJ8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=WtwKYRF/VXVyM8bwOffglVFhYSSP07jbPpVvi/USM7o=; b=KY0LcZnf88FbigO8cSChWFX/6AcCbIoCNroWUyCufPPP4ze2LxPM0dHKc9FrYaiiuq GH2dpX4/lT/059qEuJCT1fX7/QHQ4+mTqo3EvnahaGykokTocixIJqP7Jkhx+7ZTJCwi MgvMJThbcggpqndn5YEFjEpFODJrb5wnoQeKlwv9o2pZdMBlO7qbnnI75nZ6yIK0jUk9 8g35Y1VAawyM1yMDZVINk7EPv3NbBogLc6d+Mn7OHgh3VDKmBtJiM3ph7rgR5AuscmSA fuZK7xT9m4t34segKovDPT9a3LPnzmzGUZXC6SCkX7nVz8td2tnAhApUi7uR5PbM8+vD 4Wjg== X-Gm-Message-State: AE9vXwMBBJm95hRd5vEqQCZQQNVkzewnl2bloAxBUb17NtLdHQnzMMfzrSyAMCeFD4YTXt86DDoFCabqC4lIgA== X-Received: by 10.13.241.66 with SMTP id a63mr24911534ywf.162.1472996134332; Sun, 04 Sep 2016 06:35:34 -0700 (PDT) MIME-Version: 1.0 Received: by 10.83.20.135 with HTTP; Sun, 4 Sep 2016 06:35:33 -0700 (PDT) In-Reply-To: <947bf331-de0c-144e-d33b-1243f629997f@gmx.de> References: <947bf331-de0c-144e-d33b-1243f629997f@gmx.de> From: Steve Lhomme Date: Sun, 4 Sep 2016 15:35:33 +0200 Message-ID: To: "Sebastian G. " Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: Cc: Codec Encoding for LossLess Archiving and Realtime transmission Subject: Re: [Cellar] Encryption in Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Sep 2016 13:35:37 -0000 I agree with everything that's been said. We should remove the elements that are not used and especially the ones better handled in other layers. I created an issue so we don't forget to do it https://github.com/Matroska-Org/ebml-specification/issues/85 Not being an expert enough I will trust that the WebM encryption elements are sufficient and good enough and we should adopt these. 2016-08-30 19:48 GMT+02:00 Sebastian G. : > 21.07.2016, 15:12 Jerome Martinez: >> Le 20/07/2016 =C3=A0 20:44, Michael Bradshaw a =C3=A9crit : >>> [...] >>> >>> Personally, I'm in favor of just adopting WebM's encryption >>> specification and >>> removing the other encryption features of Matroska (assuming no one is >>> using them). >> >> With the same assumption (do we have real file with Matroska >> encryption?), I am also in favour of removing old encryption features of >> Matroska from the draft for IETF and for only integration of WebM >> encryption elements. >> >> > > +1 > > For interoperability and maybe because encryption can still be useful, > even though it often has negative effects if DRM is so anti-consumer > that its restrictions make legitimate use harder, I an in favor of > adopting WebM encryption elements. > > Best Regards, > Sebastian > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar --=20 Steve Lhomme Matroska association Chairman From nobody Sun Sep 4 07:40:29 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D2A8612B241 for ; Sun, 4 Sep 2016 07:40:27 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.679 X-Spam-Level: X-Spam-Status: No, score=-1.679 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, MISSING_HEADERS=1.021, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3Wt6miTI0erD for ; Sun, 4 Sep 2016 07:40:26 -0700 (PDT) Received: from mail-wm0-x236.google.com (mail-wm0-x236.google.com [IPv6:2a00:1450:400c:c09::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A565B12B068 for ; Sun, 4 Sep 2016 07:40:26 -0700 (PDT) Received: by mail-wm0-x236.google.com with SMTP id w2so90539181wmd.0 for ; Sun, 04 Sep 2016 07:40:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:cc; bh=8zxVE72hNRYMD312efoeGwrNWNTVNIvpg1k5qhajovA=; b=S0QunwLprFWHmIZ/xx5OxAmzFa+CBGK+F2gE/2GAGpueNLRMmNoIryP3MLCUUHckCM De0aVWvNyyATSL/fkX82T9gfUudmJTCssbYctdiBeW/F+mVfswgVv4Rvjd4cJcwXJZJU dALc9oO65fn+JM+2LtW0r9wbgPcqAnqa4c1/rMj8OPaQZKlrn86mOWCASAPE1UubqATD O0s+lN1xaV4HNA1PyZkuonswqi9TNTTqSp9jeZKlURyLhnyMBCRGSYvIhMbVmrs7ziKI VodK+boQDAlHhO4aq1J0Z4YMztUjiJfmCbefF7sp4hfIU+IsBm7auDpGSVJXgu6hk4/h GMrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:cc; bh=8zxVE72hNRYMD312efoeGwrNWNTVNIvpg1k5qhajovA=; b=glA1wtBPzvP9bfsQ2WIfzZdh1XT4J5cNnjXBuiOzAcDvh8VesUv83TwyGX0xkiO+Op WBB8N7MtMF8JQ9PSFRfttEpiX1zHJPPHCOXf72ghICZPD7c7ttJYrHLCz8sSlnM94Pet pMgyaTCD6ZAGXlyuxr17hyxfVfQqEdFj2JklLttvHPrSD2UJ1gC13BJ96H6IYm3XQJS6 Fqs0fpKoUiY23juiISXlI+aTEoeL9WSH/QvsRGN4baCZPW98xNOyxhN56Gn57d+lmwSQ Jvj7CUFjOeaVF4Au1ChzbC87gtH8Nt/ZGfjPGlUhdyVeP+Z6TXx8tjIenPS7jzqy9E5K YesQ== X-Gm-Message-State: AE9vXwM+5M9kVSzGl5C5aOGn5dQ9GXSpo2oEsLrI3duCtTT9fJa8mXhycc0fBH8UvbhtNAc0ODvqGt+4qYTMXw== X-Received: by 10.194.172.133 with SMTP id bc5mr27512896wjc.54.1473000024787; Sun, 04 Sep 2016 07:40:24 -0700 (PDT) MIME-Version: 1.0 Received: by 10.194.242.72 with HTTP; Sun, 4 Sep 2016 07:40:23 -0700 (PDT) In-Reply-To: References: From: Kieran O Leary Date: Sun, 4 Sep 2016 15:40:23 +0100 Message-ID: Cc: cellar@ietf.org Content-Type: text/plain; charset=UTF-8 Archived-At: Subject: Re: [Cellar] Colour Format proposal X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Sep 2016 14:40:28 -0000 Hi everyone, I'd like to revisit this topic as I've been thinking about it a lot lately. On Tue, Feb 16, 2016 at 7:27 PM, Reto Kromer wrote: > Peter B. wrote: > >>I wouldn't want to rule out the option of transcoding from >>DPX to FFV1/MKV. >>But on the other hand, I'm very much in favor of keeping >>things simple - and not overdoing it. >> >>I think being able to handle non-linear color values (as >>often found in DPX scans) is important, but I wouldn't go >>as far as putting something like "printing density" into >>the standard. > I think keeping it simple is important, especially as I doubt that 'Printing Density' is ever used outside of DPX film scanning. Speaking anecdotally , I have not seen many 'Log' values in the color primaries/transfer characteristics fields in DPX files. I'm basing this on seeing the output of DPX from about 5 different scanners and from files that have passed through Da Vinci Resolve. It's often 'Printing Density', 'Linear', or just no value at all, which shows up as '0' for the 801/802 values. I think that there is a value in adding 'Printing Density' to the list of Matroska values for Transfer Characteristic, as I'm sure that it appears frequently in film archives. While it would be ideal if FFV1 could hold these values on a codec level, having Matroska support would be great. There is a good chance (I will write another post to CELLAR about this next week) that the ffmpeg DPX encoder should be able to read a color primary/transfer characteristic value from a source file (DPX or FFV1.MKV), and then encode the correct colour value into the output DPX file. From what I understand, this would be done via AvCodecContext. I do not know enough about the inner workings of ffmpeg (does it look at container values?) to know if these kinds of color values could be passed from a container -> AvCodecContext ->DPX encoder. If so, then it would be good to know that Printing Density values could be passed correctly, not just Log and Linear. Best, Kieran. From nobody Sun Sep 4 08:02:16 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 17C7E12B028 for ; Sun, 4 Sep 2016 08:02:15 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.699 X-Spam-Level: X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YGGa45NXaxej for ; Sun, 4 Sep 2016 08:02:14 -0700 (PDT) Received: from mail-wm0-x22a.google.com (mail-wm0-x22a.google.com [IPv6:2a00:1450:400c:c09::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id ABEA412B00F for ; Sun, 4 Sep 2016 08:02:13 -0700 (PDT) Received: by mail-wm0-x22a.google.com with SMTP id c133so90780691wmd.1 for ; Sun, 04 Sep 2016 08:02:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=+axi5TKgE8RkfB7TnkhIyXlNGmsdxgDH+fx2lhwdofI=; b=tQWmJNcQ8WK2+yUrFC0O+z5ErbXZuNxjcjnXHQQRzxFX9RA2TijrBIQn5kIQuz6q6e 05QQDXydUJQCV7rT/9+5xCJDQ1s4KJvMzHT9i4vPXElccde43D1bQmlkV61/2BgKYHtM ORUq2ghlfKHc19/28d3fgq4oQpemGHSZ3l2WdCamfmwFyWxssDPIUHVVWtA+52UFqDCE S4oDwjn87aNoBXRTb+8vEWJSE39G+FSpr4+/fMU2fE817M0hNeFlfSXOyZxdamW6BgrZ 1KZHKxfkg80B+8WIe0T0QFbQsSdlmA/SKfjzGDI16+AeOfaku1quVxhIzH8+LqetpkdB TeHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=+axi5TKgE8RkfB7TnkhIyXlNGmsdxgDH+fx2lhwdofI=; b=hJ5fAD692GLpQNzloa1J9VzXBudrcuwFIhp8FAz4qaG3vZy5I8wnj9mJdgbwNtt4Ed cml1AXNrAfObVaJIDXpzyjNE6wGIqd4eQJuZX+zJ8vusCvmvnNByxfG1zi/OlseDkVDJ +5vCPUZFMRZe6/wPxIBLYHSP5RKWoaqr8MdbXR6cFFzZHq5dkz76fXWKYAKV7nMizbDc uNei1F1u6Hltfjr9NCAgRj++hC/aivffGZDyAG0Zra9SyNU8Oqf9chtZ20jkoYTEyqho BvyWFiT0YE88UBZObxiCBPI4hWYUmZkkui3plrUpUoYQinH1SAMxM2O1aXXxc+r784g8 NXfw== X-Gm-Message-State: AE9vXwOn5r0BrQkuZX4ZjwV89wZ4rt8foTFaBO3FjzV1z98/Lj63jiIeGvAwX7GmjcwnlFWsWrGmwUlED6sVhg== X-Received: by 10.194.110.102 with SMTP id hz6mr26940630wjb.5.1473001331938; Sun, 04 Sep 2016 08:02:11 -0700 (PDT) MIME-Version: 1.0 Received: by 10.194.242.72 with HTTP; Sun, 4 Sep 2016 08:02:10 -0700 (PDT) Received: by 10.194.242.72 with HTTP; Sun, 4 Sep 2016 08:02:10 -0700 (PDT) In-Reply-To: References: From: Kieran O Leary Date: Sun, 4 Sep 2016 16:02:10 +0100 Message-ID: To: cellar@ietf.org Content-Type: multipart/alternative; boundary=047d7bf10ad2251ec5053bafd901 Archived-At: Subject: Re: [Cellar] Colour Format proposal X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Sep 2016 15:02:15 -0000 --047d7bf10ad2251ec5053bafd901 Content-Type: text/plain; charset=UTF-8 Hi On 4 Sep 2016 3:40 p.m., "Kieran O Leary" wrote: > > > From what I understand, this would be done via > AvCodecContext. I do not know enough about the inner workings of > ffmpeg (does it look at container values?) This was confusing when I re-read. I meant, does AvCodecContext look at container values? Probably a stupid question, but the 'codec' in the name has put some doubt in my head. --047d7bf10ad2251ec5053bafd901 Content-Type: text/html; charset=UTF-8

Hi

On 4 Sep 2016 3:40 p.m., "Kieran O Leary" <kieran.o.leary@gmail.com> wrote:
>
>
> From what I understand, this would be done via
> AvCodecContext. I do not know enough about the inner workings of
> ffmpeg (does it look at container values?)

This was confusing when I re-read. I meant, does AvCodecContext look at container values? Probably a stupid question, but the 'codec' in the name has put some doubt in my head.

--047d7bf10ad2251ec5053bafd901-- From nobody Sun Sep 4 12:48:48 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EF092127076 for ; Sun, 4 Sep 2016 12:48:46 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.121 X-Spam-Level: X-Spam-Status: No, score=-1.121 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8FgRaGDlhWbu for ; Sun, 4 Sep 2016 12:48:45 -0700 (PDT) Received: from s172.web-hosting.com (s172.web-hosting.com [68.65.122.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0713012B281 for ; Sun, 4 Sep 2016 12:48:45 -0700 (PDT) Received: from cpe-184-152-63-84.nyc.res.rr.com ([184.152.63.84]:41284 helo=[10.0.1.3]) by server172.web-hosting.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.86_1) (envelope-from ) id 1bgdOp-003z5X-TJ; Sun, 04 Sep 2016 15:48:17 -0400 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) From: Dave Rice In-Reply-To: Date: Sun, 4 Sep 2016 15:48:13 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <328CC515-3F09-4618-8E59-2B91ECF3EDE6@dericed.com> References: To: Kieran O Leary X-Mailer: Apple Mail (2.3124) X-OutGoing-Spam-Status: No, score=-2.9 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Cc: cellar@ietf.org Subject: Re: [Cellar] Colour Format proposal X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Sep 2016 19:48:47 -0000 > On Sep 4, 2016, at 10:40 AM, Kieran O Leary = wrote: >=20 > Hi everyone, >=20 > I'd like to revisit this topic as I've been thinking about it a lot = lately. >=20 > On Tue, Feb 16, 2016 at 7:27 PM, Reto Kromer wrote: >> Peter B. wrote: >>=20 >>> I wouldn't want to rule out the option of transcoding from >>> DPX to FFV1/MKV. >>> But on the other hand, I'm very much in favor of keeping >>> things simple - and not overdoing it. >>>=20 >>> I think being able to handle non-linear color values (as >>> often found in DPX scans) is important, but I wouldn't go >>> as far as putting something like "printing density" into >>> the standard. >>=20 >=20 > I think keeping it simple is important, especially as I doubt that > 'Printing Density' is ever used outside of DPX film scanning. Speaking > anecdotally , I have not seen many 'Log' values in the color > primaries/transfer characteristics fields in DPX files. I'm basing > this on seeing the output of DPX from about 5 different scanners and > from files that have passed through Da Vinci Resolve. It's often > 'Printing Density', 'Linear', or just no value at all, which shows up > as '0' for the 801/802 values. I think that there is a value in adding > 'Printing Density' to the list of Matroska values for Transfer > Characteristic, as I'm sure that it appears frequently in film > archives. While it would be ideal if FFV1 could hold these values on a > codec level, having Matroska support would be great. >=20 > There is a good chance (I will write another post to CELLAR about this > next week) that the ffmpeg DPX encoder should be able to read a color > primary/transfer characteristic value from a source file (DPX or > FFV1.MKV), and then encode the correct colour value into the output > DPX file. =46rom what I understand, this would be done via > AvCodecContext. I do not know enough about the inner workings of > ffmpeg (does it look at container values?) to know if these kinds of > color values could be passed from a container -> AvCodecContext ->DPX > encoder. If so, then it would be good to know that Printing Density > values could be passed correctly, not just Log and Linear. For reference the color format data of DPX is defined in S268M, copy = archived at = http://web.archive.org/web/20060927044735/http://www.smpte.org/smpte_store= /standards/pdf/s268m.pdf. See page 14 in particular. These lists only = partially overlap with what is already proposed for Matroska. = Interestingly at the end of page 23 it states that "The term = 'logarithmic' is considered to be inadequately defined to achieve = interchange between different implementations." Dave Rice= From nobody Sun Sep 4 13:13:23 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 33ECE12B292 for ; Sun, 4 Sep 2016 13:13:22 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.121 X-Spam-Level: X-Spam-Status: No, score=-1.121 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7Tj7dEd3BasX for ; Sun, 4 Sep 2016 13:13:20 -0700 (PDT) Received: from s172.web-hosting.com (s172.web-hosting.com [68.65.122.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BA85C12B291 for ; Sun, 4 Sep 2016 13:13:20 -0700 (PDT) Received: from cpe-184-152-63-84.nyc.res.rr.com ([184.152.63.84]:48431 helo=[10.0.1.3]) by server172.web-hosting.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.86_1) (envelope-from ) id 1bgdmj-0048Vg-KU; Sun, 04 Sep 2016 16:12:59 -0400 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) From: Dave Rice In-Reply-To: Date: Sun, 4 Sep 2016 16:12:54 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <728DBBAB-68D5-45DE-9C7E-E3EFF34D9B25@dericed.com> References: <947bf331-de0c-144e-d33b-1243f629997f@gmx.de> To: Steve Lhomme X-Mailer: Apple Mail (2.3124) X-OutGoing-Spam-Status: No, score=-2.1 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Cc: Codec Encoding for LossLess Archiving and Realtime transmission , "Sebastian G. " Subject: Re: [Cellar] Encryption in Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Sep 2016 20:13:22 -0000 I agree as well: 1. deprecate abandoned Matroska encryption elements with a maxver value 2. add webM encryption elements: ContentEncAESSettings and = AESSettingsCipherMode 3. add webM encryption documentation at = https://www.webmproject.org/docs/webm-encryption/ as a new markdown file = in https://github.com/Matroska-Org/matroska-specification I could possibly draft PR for #1 and #2, but am not well enough familiar = with the documentation of #3. Also note that the element definitions and the related documentation is = Copyright The WebM Project and I=E2=80=99m a little unclear of the = precise license of the current specification work. Dave Rice > On Sep 4, 2016, at 9:35 AM, Steve Lhomme wrote: >=20 > I agree with everything that's been said. We should remove the > elements that are not used and especially the ones better handled in > other layers. >=20 > I created an issue so we don't forget to do it > https://github.com/Matroska-Org/ebml-specification/issues/85 >=20 > Not being an expert enough I will trust that the WebM encryption > elements are sufficient and good enough and we should adopt these. >=20 > 2016-08-30 19:48 GMT+02:00 Sebastian G. > : >> 21.07.2016, 15:12 Jerome Martinez: >>> Le 20/07/2016 =C3=A0 20:44, Michael Bradshaw a =C3=A9crit : >>>> [...] >>>>=20 >>>> Personally, I'm in favor of just adopting WebM's encryption >>>> specification = and >>>> removing the other encryption features of Matroska (assuming no one = is >>>> using them). >>>=20 >>> With the same assumption (do we have real file with Matroska >>> encryption?), I am also in favour of removing old encryption = features of >>> Matroska from the draft for IETF and for only integration of WebM >>> encryption elements. >>>=20 >>>=20 >>=20 >> +1 >>=20 >> For interoperability and maybe because encryption can still be = useful, >> even though it often has negative effects if DRM is so anti-consumer >> that its restrictions make legitimate use harder, I an in favor of >> adopting WebM encryption elements. >>=20 >> Best Regards, >> Sebastian >>=20 >> _______________________________________________ >> Cellar mailing list >> Cellar@ietf.org >> https://www.ietf.org/mailman/listinfo/cellar >=20 >=20 >=20 > --=20 > Steve Lhomme > Matroska association Chairman >=20 > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar From nobody Sun Sep 4 15:15:36 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2A52912B0EF for ; Sun, 4 Sep 2016 15:15:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.121 X-Spam-Level: X-Spam-Status: No, score=-1.121 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yINNwfLgYkLH for ; Sun, 4 Sep 2016 15:15:32 -0700 (PDT) Received: from s172.web-hosting.com (s172.web-hosting.com [68.65.122.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7862012B0E9 for ; Sun, 4 Sep 2016 15:15:32 -0700 (PDT) Received: from cpe-184-152-63-84.nyc.res.rr.com ([184.152.63.84]:40913 helo=[10.0.1.3]) by server172.web-hosting.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.86_1) (envelope-from ) id 1bgfhH-000f6Q-Uj for cellar@ietf.org; Sun, 04 Sep 2016 18:15:31 -0400 From: Dave Rice Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Message-Id: <1718BFCC-DB28-4E17-BDCB-AD7D74BD5F0F@dericed.com> Date: Sun, 4 Sep 2016 18:15:22 -0400 To: Codec Encoding for LossLess Archiving and Realtime transmission Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) X-Mailer: Apple Mail (2.3124) X-OutGoing-Spam-Status: No, score=-2.9 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Subject: [Cellar] EBML Terminology X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Sep 2016 22:15:36 -0000 Hi all, Based on this comment from Tim=E2=80=99s review: > On Jul 16, 2016, at 12:10 AM, Timothy B. Terriberry = wrote: [=E2=80=A6] > In general, this document uses a lot of terms and concepts before = defining or explaining them, without providing cross-references, = requiring a good deal of flipping back-and-forth to understand = everything. While adding cross-references is one useful step, a more = logical organization of the document to explain concepts before = introducing constructs which rely on them would reduce the amount of = back-and-forth required and make things much easier to understand. I've = included some specific suggestions of what should move where below. I added this section of the EBML Spec. Also viewable as a PR here: = https://github.com/Matroska-Org/ebml-specification/pull/87 This document defines specific terms in order to define the format and = application of EBML. Specific terms are defined below: `Child Element`: A Child Element is a relative term to describe the EBML = Elements immediately contained within a Master Element. `EBML`: Extensible Binary Meta Language `Element Data`: The value(s) of the `EBML Element` which is identified = by its `Element ID` and `Element Data Size`. The form of the `Element = Data` is defined by this document and the corresponding `EBML Schema` of = the Element's `EBML Document Type`. `Element Data Size`: An expression, encoded as a `Variable Size = Integer`, of the length in octets of `Element Data`. `EBML Body`: All data of an `EBML Document` following the `EBML Header` = may be considered the `EBML Body`. `EBML Class`: An representation of the octet length of an `Element ID`. `EBML Document`: An `EBML Document` is a datastream comprised of only = two components, an `EBML Header` and an `EBML Body`. `EBML Document Type`: An `EBML Document Type` is a name provided by an = `EBML Schema` for a particular implementation of `EBML` for a data = format (examples: matroska and webm). `EBML Element`: A foundation block of data that contains three parts: an = `Element ID`, an `Element Data Size`, and `Element Data`. `EBML Header`: The `EBML Header` is a declaration that provides = processing instructions and identification of the `EBML Body`. The `EBML = Header` may be considered as analogous to an XML Declaration. `EBML Schema`: A standardized definition for the structure of an `EBML = Document Type`. `EBML Stream`: An `EBML Stream` is a file that consists of one or many = `EBML Documents` that are concatenated together. `Element ID`: The `Element ID` is a binary value, encoded as a `Variable = Size Integer`, used to uniquely identify a defined `EBML Element` within = a specific `EBML Schema`. `Element Name`: The official human-readable name of the `EBML Element`. `Empty Element`: An `Empty Element` is an `EBML Element` that has an = `Element Data Size` with all VINT\_DATA bits set to zero which indicates = that the `Element Data` of the Element is zero octets in length. `Level`: `Level` notes at what hierarchical depth the `EBML Element` may = occur within an `EBML Document`, starting from zero. `Master Element`: The `Master Element` contains zero, one, or many other = `EBML Elements`. `Parent Element`: A relative term to describe the `Master Element` which = contains a specified element. `Root Element`: A mandatory, non-repeating EBML Element which occurs = within an `EBML Document` at `Level 0` and contains all other `EBML = Elements` of the `EBML=20 Document`, excepting the `EBML Header` and optional Void Elements. `Top-Level Element`: An elements defined to only occur at `Level 1` = within the `Root Element`. `Unknown-Sized Element`: An Element with an unknown `Element Data Size`. `Variable Size Integer`: A compact variable-length binary value which = defines its own length. `VINT`: Also known as `Variable Size Integer`. `VINTMAX`: The maximum possible value that can be stored as `Element = Data Size`. Some questions/ comments: Should we add more terms or remove some of these? This increases redundancy in the overall spec, so some sentences may be = removed from later in the doc, but I=E2=80=99d like to finalize this = section first before doing the trimming. Should we backtick all defined terms? (lots of backticking). Best Regards, Dave Rice From nobody Mon Sep 5 13:05:39 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0289412B430 for ; Mon, 5 Sep 2016 13:05:37 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.12 X-Spam-Level: X-Spam-Status: No, score=-1.12 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JZCZgYiPfqg5 for ; Mon, 5 Sep 2016 13:05:35 -0700 (PDT) Received: from s172.web-hosting.com (s172.web-hosting.com [68.65.122.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 353C7128B44 for ; Mon, 5 Sep 2016 13:05:35 -0700 (PDT) Received: from cpe-184-152-63-84.nyc.res.rr.com ([184.152.63.84]:38367 helo=[10.0.1.3]) by server172.web-hosting.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.86_1) (envelope-from ) id 1bh096-002vgY-MU; Mon, 05 Sep 2016 16:05:34 -0400 Content-Type: multipart/alternative; boundary="Apple-Mail=_DE3D113A-DCC5-42AE-BF2D-C34DADDA44BE" Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) From: Dave Rice In-Reply-To: <67AE0413-6C6D-4B4E-B90F-45085CBF12B0@iovu.io> Date: Mon, 5 Sep 2016 16:05:25 -0400 Message-Id: References: <67AE0413-6C6D-4B4E-B90F-45085CBF12B0@iovu.io> To: "Questions, help, instructions, talk about Matroska" X-Mailer: Apple Mail (2.3124) X-OutGoing-Spam-Status: No, score=-2.9 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Cc: Codec Encoding for LossLess Archiving and Realtime transmission Subject: Re: [Cellar] [Matroska-users] Max value for VINT X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Sep 2016 20:05:37 -0000 --Apple-Mail=_DE3D113A-DCC5-42AE-BF2D-C34DADDA44BE Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi Iovu, I=E2=80=99m cc=E2=80=99ing the CELLAR mailing list since the topic is = relevant to current work there. > On Sep 5, 2016, at 3:46 PM, Ion Iovu wrote: >=20 > Hi guys.=20 >=20 > I'd like to know what the maximum value for the VINT is. The RFC draft = page mention examples of VINT = values with up to a tail of 3 bytes, however no mention of any upper = limit for VINT is mentioned. The 2004 RFC draft = on matroska.org is superseded by the current work of the IETF CELLAR = working group. The current RFC = for EBML is a work in progress at = https://github.com/Matroska-Org/ebml-specification/blob/master/specificati= on.markdown. > It can be inferred from the following text: >=20 > =E2=80=9CSince the highest value is used for unknown size the = effective maximum data size is 2^56-2, using variable size integer width = 8.=E2=80=9D >=20 > that particularly for element sizes you can only go to at most 8 bytes = (a tail of 7 bytes),=20 > 2^56-1 is a reserved one, and valid values are in the range 0..2^56-2 That=E2=80=99s right. The spec we=E2=80=99re working on includes a = clarification on this: "An Element Data Size with an octet length of 8 is able to express a = size of 2^56-2 or 72,057,594,037,927,934 octets (or about 72 petabytes). = The maximum possible value that can be stored as Element Data Size is = referred to as VINTMAX." > However, that does not mean the VINT as a data type is limited by that = value. Within the EBML Body the max length of the Element Data Size is declared = in the EBML Header in EBMLMaxSizeLength. For Matroska that is limited to = 8, which provides a VINTMAX of 72,057,594,037,927,934; however another = EBML format could permit the EBMLMaxSizeLength to be values higher than = 8, if you find the 72,057,594,037,927,934 bytes too constraining. > Is it possible to go with a VINT_ALIGNMENT of more than 8 bits? Hmm, the term VINT_ALIGNMENT is in the superseded 2004 RFC Draft = but in the new RFC = work it is just referred to as zero-padding. VINT_ALIGNMENT of greater than 8 bits is not allowed in Element IDs = since they are required to use the shortest valid encoding, but an = Element Data Size may use more than 8 bits of zero-padding (aka = VINT_ALIGNMENT). For instance, both 0b10000001 and = 0b0000000100000000000000000000000000000000000000000000000000000001 (a = VINT_ALIGNMENT of 55 bits) are valid methods to express a VINT of = Element Data Size of 1. [=E2=80=A6] Best Regards, Dave Rice --Apple-Mail=_DE3D113A-DCC5-42AE-BF2D-C34DADDA44BE Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
Hi Iovu,

I=E2=80=99m cc=E2=80=99ing the CELLAR = mailing list since the topic is relevant to current work there.

On = Sep 5, 2016, at 3:46 PM, Ion Iovu <ion@iovu.io> wrote:

Hi = guys. 

I'd like = to know what the maximum value for the VINT is. The RFC = draft page mention examples of VINT values with up to a tail of = 3 bytes, however no mention of any upper limit for VINT is = mentioned.

The 2004 RFC draft on matroska.org is superseded by the current work of the = IETF CELLAR working group. The current RFC for EBML is a = work in progress at https://github.com/Matroska-Org/ebml-specification/blob/master/= specification.markdown.

It can be inferred from the = following text:

=E2=80=9CSince the highest value is used for = unknown size the effective maximum data size is 2^56-2, using variable = size integer width 8.=E2=80=9D

that particularly for element sizes you can only go to at = most 8 bytes (a tail of 7 bytes), 
2^56-1 is a = reserved one, and valid values are in the range = 0..2^56-2

That=E2=80=99s right. The spec we=E2=80=99re = working on includes a clarification on this:

"An Element Data Size with an octet length of 8 is = able to express a size of 2^56-2 or 72,057,594,037,927,934 octets = (or about 72 petabytes). The maximum possible value that can = be stored as Element Data Size is referred to = as VINTMAX."

However, that does not mean = the VINT as a data type is limited by that = value.

Within the EBML Body the max length of the Element = Data Size is declared in the EBML Header in EBMLMaxSizeLength. For = Matroska that is limited to 8, which provides a VINTMAX of = 72,057,594,037,927,934; however another EBML format could permit the = EBMLMaxSizeLength to be values higher than 8, if you find the = 72,057,594,037,927,934 bytes too constraining.

Is it = possible to go with a VINT_ALIGNMENT of more than 8 = bits?

Hmm, = the term VINT_ALIGNMENT is in the superseded 2004 RFC Draft but in the new RFC work it is just = referred to as zero-padding.
VINT_ALIGNMENT of greater than 8 = bits is not allowed in Element IDs since they are required to use the = shortest valid encoding, but an Element Data Size may use more than 8 = bits of zero-padding (aka VINT_ALIGNMENT).

For instance, both 0b10000001 and = 0b0000000100000000000000000000000000000000000000000000000000000001 (a = VINT_ALIGNMENT of 55 bits) are valid methods to express a VINT of = Element Data Size of 1.

[=E2=80=A6]

Best= Regards,
Dave Rice

= --Apple-Mail=_DE3D113A-DCC5-42AE-BF2D-C34DADDA44BE-- From nobody Tue Sep 6 07:43:31 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7471612B018 for ; Tue, 6 Sep 2016 07:43:29 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.208 X-Spam-Level: X-Spam-Status: No, score=-4.208 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-1.508, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XyHUc1g4iS_J for ; Tue, 6 Sep 2016 07:43:24 -0700 (PDT) Received: from mail-qk0-x231.google.com (mail-qk0-x231.google.com [IPv6:2607:f8b0:400d:c09::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BAC5F12B1F3 for ; Tue, 6 Sep 2016 07:43:24 -0700 (PDT) Received: by mail-qk0-x231.google.com with SMTP id v123so219414859qkh.2 for ; Tue, 06 Sep 2016 07:43:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=SiUKeZpl0qTJ+Dvd9jaMT0x9AWAEiwcNC6B/yQWaAsQ=; b=kYQGCNVNZlSdx7QbV4RyspRDYabw0rgfcbwu6xYA50YbkBbp9nenTz99wAiYfNrOjb RcWnvVHPgqrJAOSC28g7eGavfp4rYaRnbvKrQGj8ONxMXkEOJkCPXvSJG263Wr0cPfTm xzYTv7unEbD+YycFfqLVQrwG5LwLzx9Sa470K/MwVT6mYnCD7ELwPC/7fAGd6O/ndhtn YTZC9voIArRSTfoKaAl5hqzh3Qm1OP93O+dmYvz4fvoJy+qYdnd1eckiBrZYt9HlqBl4 v1rpeL0mDNUBR/w2QHOIlbY/KsUCpzF2p6LftQaM/BOVPGpR3xNy/AafUuNJGS3qH0eJ vJ2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=SiUKeZpl0qTJ+Dvd9jaMT0x9AWAEiwcNC6B/yQWaAsQ=; b=Mlwu66JCq23fLEUF0MFfUFNOUcBqfKNfnvZEGwLl4u47kiAEQdM0J/VHE0McJuenV+ IFGQrGgG0c2mQIKAmi0q8zxUolZsexB0djPFZEViRCrJdooGjCbDN4+8LWouJ/8pWOzG KLoQUiZ6ofWnjc5/8pP0UX/XK+hup7PxrwR+oWSUVPCuEwfxnXigusNieoAdlcyV7tQf Rk3Z50kYlqRHXTF8tMFxLDOszZiyJ8PWyHNteIp87rJHNaX3Qr3JdH7GWdjjeqRy2kK+ Er7t4cBRrBAkyzjAVUzeDrJKAzKq8tQGvKteroinK6GQeDV1BBxYbjaj1EIwf7dvOKkr o4Ug== X-Gm-Message-State: AE9vXwMjA9lMoKoa3Q5s0wUbIkl7DduRVCLYMLm0oNG0o61ZJaMu3s+/76tiDlpZN1gcuNvtO1KoMOxgAfJ48RmX X-Received: by 10.55.75.79 with SMTP id y76mr47309713qka.9.1473173003778; Tue, 06 Sep 2016 07:43:23 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Aaron Colwell Date: Tue, 06 Sep 2016 14:43:12 +0000 Message-ID: To: Steve Lhomme , Michael Bradshaw Content-Type: multipart/alternative; boundary=001a114a685095b4f8053bd7d1da Archived-At: Cc: Codec Encoding for LossLess Archiving and Realtime transmission Subject: Re: [Cellar] 360/VR videos in Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Sep 2016 14:43:29 -0000 --001a114a685095b4f8053bd7d1da Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Comments inline... On Sun, Sep 4, 2016 at 6:09 AM Steve Lhomme wrote: > 2016-09-04 14:46 GMT+02:00 Steve Lhomme : > > 2016-08-30 18:40 GMT+02:00 Michael Bradshaw : > >> Matroska currently doesn't have a way to specify projection informatio= n > for > >> 360/VR videos. Google's Spherical Video V2 RFC draft was just updated = a > few > >> days ago to include some new elements specifying projection and viewer > pose > >> information[1]. > >> > >> These proposed elements haven't made their way over to the WebM > >> specification (and I haven't asked the WebM team if they intend to > >> incorporate them), but I think it raises a good question about Matrosk= a > >> support for these kinds of things. > >> > >> I don't know of any past work in specifying projection information in > >> Matroska. Assuming there hasn't been any, is there interest in adding = it > >> (I'm assuming the answer is still yes, from those I asked in Berlin)? > Do the > > > > Since I'm currently adding 360=C2=B0 support in VLC the answer is a > > definite YES. So far we only have MP4 sources. For now we only support > > some XML embedded in an MP4 UUID XML360 box. The XML is similar to the > > one defined here: > > > https://github.com/google/spatial-media/blob/master/docs/spherical-video-= rfc.md#user-content-appendix-1---global-metadata-sample > > > > There's already hardware that saves directly to this format. > > > >> elements from the Spherical Video V2 RFC draft look like good > candidates? > > > > It's a good start ;) > > > > I think the 0.0 default value should be 0x0p+1. > I don't really understand this suggestion. Could you please clarify. Sorry if this may be obvious to others, but I'm new to this list. :) > > > > Also I think the ProjectionType value 0 should be None corresponding > > to the current situation when the field doesn't exist. > > Never mind, Rectangular (0) is actually the regular/flat projection. > > > Are the Yaw/Pitch/Roll used in practice ?; > Yes. They are particularly helpful when you want to use multiple directionally-biased projections of the same scene. A simple, but not ideal, example is using the increased pixel density at the equirect poles as a "directional projection". You could create multiple transcodes of a scene where different yaw/pitch/roll combinations move different parts of the sphere to the poles so you could have increased resolution in that direction. Then a player could switch between these in a DASH-like manner based on where the viewer is looking. Equirect is not a particularly great projection for this, but it gets the idea across. The yaw/pitch/roll fields help the client map these different transcodes into a single coordinate space. Aaron --001a114a685095b4f8053bd7d1da Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Comments inline...

On Sun, Sep 4, 2016 at 6:09 AM Steve Lhomme <slhomme@matroska.org> wrote:
2016-09-04 14:46 GMT+02:00 Steve Lhomme <slhom= me@matroska.org>:
> 2016-08-30 18:40 GMT+02:00 Michael Bradshaw <mjbshaw@google.com&g= t;:
>> Matroska currently doesn't have a way to specify projection in= formation for
>> 360/VR videos. Google's Spherical Video V2 RFC draft was just = updated a few
>> days ago to include some new elements specifying projection and vi= ewer pose
>> information[1].
>>
>> These proposed elements haven't made their way over to the Web= M
>> specification (and I haven't asked the WebM team if they inten= d to
>> incorporate them), but I think it raises a good question about Mat= roska
>> support for these kinds of things.
>>
>> I don't know of any past work in specifying projection informa= tion in
>> Matroska. Assuming there hasn't been any, is there interest in= adding it
>> (I'm assuming the answer is still yes, from those I asked in B= erlin)? Do the
>
> Since I'm currently adding 360=C2=B0 support in VLC the answer is = a
> definite YES. So far we only have MP4 sources. For now we only support=
> some XML embedded in an MP4 UUID XML360 box. The XML is similar to the=
> one defined here:
> https://github.com/googl= e/spatial-media/blob/master/docs/spherical-video-rfc.md#user-content-append= ix-1---global-metadata-sample
>
> There's already hardware that saves directly to this format.
>
>> elements from the Spherical Video V2 RFC draft look like good cand= idates?
>
> It's a good start ;)
>
> I think the 0.0 default value should be 0x0p+1.
<= br>
I don't really understand this suggestion. Could you plea= se clarify. Sorry if this may be obvious to others, but I'm new to this= list. :)
=C2=A0
>
> Also I think the ProjectionType value 0 should be None corresponding > to the current situation when the field doesn't exist.

Never mind, Rectangular (0) is actually the regular/flat projection.

> Are the Yaw/Pitch/Roll used in practice ?;

Yes. They are particularly helpful when you want to use multiple d= irectionally-biased projections of the same scene. A simple, but not ideal,= example is using the increased pixel density at the equirect poles as a &q= uot;directional projection". You could create multiple transcodes of a= scene where different yaw/pitch/roll combinations move different parts of = the sphere to the poles so you could have increased resolution in that dire= ction. Then a player could switch between these in a DASH-like manner based= on where the viewer is looking. Equirect is not a particularly great proje= ction for this, but it gets the idea across. The yaw/pitch/roll fields help= the client map these different transcodes into a single coordinate space.<= /div>

Aaron
--001a114a685095b4f8053bd7d1da-- From nobody Tue Sep 6 09:11:31 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 11DA312B05F for ; Tue, 6 Sep 2016 09:11:30 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.6 X-Spam-Level: X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 29fyTEnEnp0P for ; Tue, 6 Sep 2016 09:11:27 -0700 (PDT) Received: from mail-yw0-x233.google.com (mail-yw0-x233.google.com [IPv6:2607:f8b0:4002:c05::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 87D6812B233 for ; Tue, 6 Sep 2016 09:10:00 -0700 (PDT) Received: by mail-yw0-x233.google.com with SMTP id l8so61529626ywb.0 for ; Tue, 06 Sep 2016 09:10:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=ziuM29NgKnjvqVuhRW4r2K8csTpdHaweNhJnP3IK2ZI=; b=VrRbPKaY+g+KjomQ+7vBLEoYD6wU+G5MEJOPh+7PcCa9+9vMe0Dx68ZpYJlvIB2zpD qEwapPB65iq+9+/jzXBT8qLfeb4jawkxKaasRY+qAWDHo+B+V3jMeDThIxvzqTId3+NR ceQzTwcAOcoTq52/4Jns7oJLepxVFQ7v3NYNpYjYCTVEI6bGIZPDTsZHiNpfhdugQW9v rBdIFI1zLmHsN3Nt2+8O/4GCIf30gCkJTgi4U4QY3BKPFIYbtITwKyPDBaVWIB+Ln1YT jf/dklJVkODcQUT0q+4vxWdOPE2eJM94YpFc7uMb00H+FrNmj2Y5PdFtpUxDLCcxTpxN QyEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=ziuM29NgKnjvqVuhRW4r2K8csTpdHaweNhJnP3IK2ZI=; b=ZLrbtshPCbBmr3PrJEKaYPidkapRd0Mp1rl/f20CEOHjxej1/U6KBKpQ34uzFkjs7A TQYcbH+uILVxWa2faOFkBLcQ9bYEIUtXooCJW7gsDysEgnWrTjWBuWokQZkASh8dcjfI lkVSJnUBrVxlV6gNJcIKdtjc8VWWT2HBUtwcStBpw4diBzr3c5nXQ5wH/7xwAqJTlwnK EmwTOW5oGZx7lApSb4S8nRy2hTUx7sDkfGItOPMJICaU48Hjtq3E9AIVKbJ74fz4WEoO +lt1HvObT7P8VjC1F7xaMI6kxp8vlnzz5gNBHzJfnm9Mm6Kn76cgTBcK8aXSw3BPhw9k eKYg== X-Gm-Message-State: AE9vXwPUK+vAzUxwxzi8/nLd94K7WLjuxBmsPqy7/uEb++iomc38okEQNuvKnUxowzj8t5fab5xko6KRjDdNjA== X-Received: by 10.13.193.196 with SMTP id c187mr33231827ywd.301.1473178199680; Tue, 06 Sep 2016 09:09:59 -0700 (PDT) MIME-Version: 1.0 Received: by 10.83.20.135 with HTTP; Tue, 6 Sep 2016 09:09:59 -0700 (PDT) In-Reply-To: References: From: Steve Lhomme Date: Tue, 6 Sep 2016 18:09:59 +0200 Message-ID: To: Aaron Colwell Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: Cc: Michael Bradshaw , Codec Encoding for LossLess Archiving and Realtime transmission Subject: Re: [Cellar] 360/VR videos in Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Sep 2016 16:11:30 -0000 2016-09-06 16:43 GMT+02:00 Aaron Colwell : > Comments inline... > > On Sun, Sep 4, 2016 at 6:09 AM Steve Lhomme wrote: >> >> 2016-09-04 14:46 GMT+02:00 Steve Lhomme : >> > 2016-08-30 18:40 GMT+02:00 Michael Bradshaw : >> >> Matroska currently doesn't have a way to specify projection informati= on >> >> for >> >> 360/VR videos. Google's Spherical Video V2 RFC draft was just updated= a >> >> few >> >> days ago to include some new elements specifying projection and viewe= r >> >> pose >> >> information[1]. >> >> >> >> These proposed elements haven't made their way over to the WebM >> >> specification (and I haven't asked the WebM team if they intend to >> >> incorporate them), but I think it raises a good question about Matros= ka >> >> support for these kinds of things. >> >> >> >> I don't know of any past work in specifying projection information in >> >> Matroska. Assuming there hasn't been any, is there interest in adding >> >> it >> >> (I'm assuming the answer is still yes, from those I asked in Berlin)? >> >> Do the >> > >> > Since I'm currently adding 360=C2=B0 support in VLC the answer is a >> > definite YES. So far we only have MP4 sources. For now we only support >> > some XML embedded in an MP4 UUID XML360 box. The XML is similar to the >> > one defined here: >> > >> > https://github.com/google/spatial-media/blob/master/docs/spherical-vid= eo-rfc.md#user-content-appendix-1---global-metadata-sample >> > >> > There's already hardware that saves directly to this format. >> > >> >> elements from the Spherical Video V2 RFC draft look like good >> >> candidates? >> > >> > It's a good start ;) >> > >> > I think the 0.0 default value should be 0x0p+1. > > > I don't really understand this suggestion. Could you please clarify. Sorr= y > if this may be obvious to others, but I'm new to this list. :) We try to use a more standardized way to express floats. Obviously for 0.0 there's not problem but for other values it's better expressed in a non ambiguous hexademical format. See this section https://github.com/Matroska-Org/ebml-specification/blob/master/specificatio= n.markdown#textual-expression-of-floats >> >> > >> > Also I think the ProjectionType value 0 should be None corresponding >> > to the current situation when the field doesn't exist. >> >> Never mind, Rectangular (0) is actually the regular/flat projection. >> >> > Are the Yaw/Pitch/Roll used in practice ?; > > > Yes. They are particularly helpful when you want to use multiple > directionally-biased projections of the same scene. A simple, but not ide= al, > example is using the increased pixel density at the equirect poles as a > "directional projection". You could create multiple transcodes of a scene > where different yaw/pitch/roll combinations move different parts of the > sphere to the poles so you could have increased resolution in that > direction. Then a player could switch between these in a DASH-like manner > based on where the viewer is looking. Equirect is not a particularly grea= t > projection for this, but it gets the idea across. The yaw/pitch/roll fiel= ds > help the client map these different transcodes into a single coordinate > space. OK, got it. --=20 Steve Lhomme Matroska association Chairman From nobody Tue Sep 6 09:12:29 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 716C212B220 for ; Tue, 6 Sep 2016 09:12:27 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tpByZEs-iRmH for ; Tue, 6 Sep 2016 09:12:24 -0700 (PDT) Received: from mail-yb0-x236.google.com (mail-yb0-x236.google.com [IPv6:2607:f8b0:4002:c09::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2800D12B05F for ; Tue, 6 Sep 2016 09:11:39 -0700 (PDT) Received: by mail-yb0-x236.google.com with SMTP id d205so64918326ybh.0 for ; Tue, 06 Sep 2016 09:11:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=nb58Gbmxepcje7CmOD24QaG1gdwM3m6YZvYP2nbGP30=; b=iLcCpnQIS5fpwg8lyH2yUPNHMcrUWLaLlNSiNjgMNlIxD0hKUteaAIo9SHBaVhZALq xMo5O47gpdc+GpLJKgJ7YVlFqh1JJ7K7t9+Rj75qoxQYRVh87NpeN1Arybar1Ogn84I0 /m56o4mnuY1veB23BOzZ9L3pbYSpMk+ORtvlh45dYAv2OXXfLDAUhObPobxuFwkmt4oS 3+nflSsazOBv5jgwrqLmTsVGRBBSW6LCNJjUKkA5yfwq8ITFh+iqxWbloGylr+DM9h4p bRde+9jsdvXANv40wXit+92XJ2Yx5mFlGJugMvL4ZlrbFM2+yDlD3jFC5OsfBIEj4sbF S6LQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=nb58Gbmxepcje7CmOD24QaG1gdwM3m6YZvYP2nbGP30=; b=SVdE+kiacGEH80TEVuuT7Hmk1DgQBjWwlA6moxsT2G1fsN/Q8Mq9svFnJCBQroxgtW weItMyGsxVT/K7opGNP02XdKffYQ7qryBcPpKmcWqLwZKsg8Yygnc7mfnrXNLdUpuf0O Vto4R/6rc3LOrumOUzuNHfTJ6r1sh5JXZm7EtHgZsaelz2K8TM6IJOGXpw5ZEdVymfd+ i1JZ65VwVKHGorToWiQDFbFlFTXCsnKP7fosx1ojPhqwDo1oArxZTLXawKg5EbR1bLGk 0CVrMU6Vko+Fxgg3/9z96WLkvBVRIrJU/zP/v67ZDyvF/auw2vl/IxJXqbAqD5dUs/tA rKaw== X-Gm-Message-State: AE9vXwOG7DIwKhrm3hoPK2JLzB+wHyHzPhefEqBNkbiZYgDJYYWdXrQ29LgezEZvM4H/q/9kzL5VARxaaQXtfA== X-Received: by 10.37.231.68 with SMTP id e65mr27864690ybh.137.1473178298425; Tue, 06 Sep 2016 09:11:38 -0700 (PDT) MIME-Version: 1.0 Received: by 10.129.116.3 with HTTP; Tue, 6 Sep 2016 09:11:38 -0700 (PDT) In-Reply-To: References: From: Nithin Mathew Kurien Date: Tue, 6 Sep 2016 21:41:38 +0530 Message-ID: To: Aaron Colwell Content-Type: multipart/alternative; boundary=94eb2c0b14482b30a8053bd90d95 Archived-At: Cc: Codec Encoding for LossLess Archiving and Realtime transmission Subject: Re: [Cellar] 360/VR videos in Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Sep 2016 16:12:27 -0000 --94eb2c0b14482b30a8053bd90d95 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Tue, Sep 6, 2016 at 8:13 PM, Aaron Colwell wrote: > Comments inline... > > On Sun, Sep 4, 2016 at 6:09 AM Steve Lhomme wrote: > >> 2016-09-04 14:46 GMT+02:00 Steve Lhomme : >> > 2016-08-30 18:40 GMT+02:00 Michael Bradshaw : >> >> Matroska currently doesn't have a way to specify projection >> information for >> >> 360/VR videos. Google's Spherical Video V2 RFC draft was just updated >> a few >> >> days ago to include some new elements specifying projection and viewe= r >> pose >> >> information[1]. >> >> >> >> These proposed elements haven't made their way over to the WebM >> >> specification (and I haven't asked the WebM team if they intend to >> >> incorporate them), but I think it raises a good question about Matros= ka >> >> support for these kinds of things. >> >> >> >> I don't know of any past work in specifying projection information in >> >> Matroska. Assuming there hasn't been any, is there interest in adding >> it >> >> (I'm assuming the answer is still yes, from those I asked in Berlin)? >> Do the >> > >> > Since I'm currently adding 360=C2=B0 support in VLC the answer is a >> > definite YES. So far we only have MP4 sources. For now we only support >> > some XML embedded in an MP4 UUID XML360 box. The XML is similar to the >> > one defined here: >> > https://github.com/google/spatial-media/blob/master/ >> docs/spherical-video-rfc.md#user-content-appendix-1--- >> global-metadata-sample >> > >> > There's already hardware that saves directly to this format. >> > >> >> elements from the Spherical Video V2 RFC draft look like good >> candidates? >> > >> > It's a good start ;) >> > >> > I think the 0.0 default value should be 0x0p+1. >> > > I don't really understand this suggestion. Could you please clarify. Sorr= y > if this may be obvious to others, but I'm new to this list. :) > This refers to the representation of floats in EBML as hexadecimal constants for better precision, according to the C11 notation ( https://github.com/Matroska-Org/ebml-specification/blob/master/specificatio= n.markdown#textual-expression-of-floats). This is a very recent feature, adopted a few weeks ago by CELLAR. :) > >> > >> > Also I think the ProjectionType value 0 should be None corresponding >> > to the current situation when the field doesn't exist. >> >> Never mind, Rectangular (0) is actually the regular/flat projection. >> >> > Are the Yaw/Pitch/Roll used in practice ?; >> > > Yes. They are particularly helpful when you want to use multiple > directionally-biased projections of the same scene. A simple, but not > ideal, example is using the increased pixel density at the equirect poles > as a "directional projection". You could create multiple transcodes of a > scene where different yaw/pitch/roll combinations move different parts of > the sphere to the poles so you could have increased resolution in that > direction. Then a player could switch between these in a DASH-like manner > based on where the viewer is looking. Equirect is not a particularly grea= t > projection for this, but it gets the idea across. The yaw/pitch/roll fiel= ds > help the client map these different transcodes into a single coordinate > space. > > Aaron > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > > --94eb2c0b14482b30a8053bd90d95 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable


On Tue, Sep 6, 2016 at 8:13 PM, Aaron Colwell <acolwell@google.com> wrote:
<= div dir=3D"ltr">Comments inline...

2016-09-04 14:46 GMT+02:00 Steve Lhomme <slhomme@matroska.org>:
> 2016-08-30 18:40 GMT+02:00 Michael Bradshaw <mjbshaw@google.com>:
>> Matroska currently doesn't have a way to specify projection in= formation for
>> 360/VR videos. Google's Spherical Video V2 RFC draft was just = updated a few
>> days ago to include some new elements specifying projection and vi= ewer pose
>> information[1].
>>
>> These proposed elements haven't made their way over to the Web= M
>> specification (and I haven't asked the WebM team if they inten= d to
>> incorporate them), but I think it raises a good question about Mat= roska
>> support for these kinds of things.
>>
>> I don't know of any past work in specifying projection informa= tion in
>> Matroska. Assuming there hasn't been any, is there interest in= adding it
>> (I'm assuming the answer is still yes, from those I asked in B= erlin)? Do the
>
> Since I'm currently adding 360=C2=B0 support in VLC the answer is = a
> definite YES. So far we only have MP4 sources. For now we only support=
> some XML embedded in an MP4 UUID XML360 box. The XML is similar to the=
> one defined here:
> https://github.com/google/spatial-me= dia/blob/master/docs/spherical-video-rfc.md#user-content-appendix= -1---global-metadata-sample
>
> There's already hardware that saves directly to this format.
>
>> elements from the Spherical Video V2 RFC draft look like good cand= idates?
>
> It's a good start ;)
>
> I think the 0.0 default value should be 0x0p+1.
<= br>
I don't really understand this suggestion. Could y= ou please clarify. Sorry if this may be obvious to others, but I'm new = to this list. :)

This ref= ers to the representation of floats in EBML as hexadecimal constants for be= tter precision, according to the C11 notation (https://github.com/Matroska-Org/ebml-specification/= blob/master/specification.markdown#textual-expression-of-floats). This = is a very recent feature, adopted a few weeks ago by CELLAR. :)
<= br>
=C2=A0
>
> Also I think the ProjectionType value 0 should be None corresponding > to the current situation when the field doesn't exist.

Never mind, Rectangular (0) is actually the regular/flat projection.

> Are the Yaw/Pitch/Roll used in practice ?;

Yes. They are particularly helpful when you want to use mul= tiple directionally-biased projections of the same scene. A simple, but not= ideal, example is using the increased pixel density at the equirect poles = as a "directional projection". You could create multiple transcod= es of a scene where different yaw/pitch/roll combinations move different pa= rts of the sphere to the poles so you could have increased resolution in th= at direction. Then a player could switch between these in a DASH-like manne= r based on where the viewer is looking. Equirect is not a particularly grea= t projection for this, but it gets the idea across. The yaw/pitch/roll fiel= ds help the client map these different transcodes into a single coordinate = space.

<= /div>
Aaron

_______________________________________________
Cellar mailing list
Cellar@ietf.org
https://www.ietf.org/mailman/listinfo/cellar

--94eb2c0b14482b30a8053bd90d95-- From nobody Tue Sep 6 09:56:36 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F002E12B4F1 for ; Tue, 6 Sep 2016 09:56:27 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.921 X-Spam-Level: X-Spam-Status: No, score=-1.921 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2SVcZ8w735bO for ; Tue, 6 Sep 2016 09:56:22 -0700 (PDT) Received: from 4.mo173.mail-out.ovh.net (4.mo173.mail-out.ovh.net [46.105.34.219]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CB2CD12B1AA for ; Tue, 6 Sep 2016 09:56:19 -0700 (PDT) Received: from player157.ha.ovh.net (b9.ovh.net [213.186.33.59]) by mo173.mail-out.ovh.net (Postfix) with ESMTP id A55B5100C311 for ; Tue, 6 Sep 2016 18:56:17 +0200 (CEST) Received: from [192.168.2.101] (p5DDB5F23.dip0.t-ipconnect.de [93.219.95.35]) (Authenticated sender: jerome@francoallemand.eu) by player157.ha.ovh.net (Postfix) with ESMTPSA id 68472500099 for ; Tue, 6 Sep 2016 18:56:17 +0200 (CEST) To: cellar@ietf.org From: Jerome Martinez Message-ID: <9146e646-0afb-7dfb-911b-b3f7c901ca71@mediaarea.net> Date: Tue, 6 Sep 2016 18:56:17 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="------------70C4B4EC542473DFDC4D5E4B" X-Ovh-Tracer-Id: 17667902815198515345 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 0 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrfeeluddrheelgddutdekucdltddurdefledtrddttddmucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfenuceurghilhhouhhtmecufedttdenuc Archived-At: Subject: [Cellar] [PATCH FFV1] Median Predictor formula update X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Sep 2016 16:56:28 -0000 This is a multi-part message in MIME format. --------------70C4B4EC542473DFDC4D5E4B Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Some streams in the wild were found not conforming to current specification. This patch describes what a FFV1 decoder should do in such case, and also what an encoder should do in order to permit correct decoding by decoders in the wild (the current goal is to describe what exists, so we will change the expected behavior only in the next version of the bitstream). Impacted streams are FFV1 configured with 16-bit YCbCr color space and Range Coder coder (this includes 16-bit gray because it is YCbCr color space with chroma deactivated). --------------70C4B4EC542473DFDC4D5E4B Content-Type: text/plain; charset=UTF-8; name="0001-Median-Predictor-formula-update.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="0001-Median-Predictor-formula-update.patch" RnJvbSA1Y2JiNmFkNDhkMDgwYjQxNjlhNDM4Njg1Mzg0ZWY5YmMwYTQzZmRmIE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiA9P1VURi04P3E/Sj1DMz1BOXI9QzM9QjRtZT0yME1h cnRpbmV6Pz0gPGplcm9tZUBtZWRpYWFyZWEubmV0PgpEYXRlOiBUdWUsIDYgU2VwIDIwMTYg MTg6NDM6MjEgKzAyMDAKU3ViamVjdDogW1BBVENIXSBNZWRpYW4gUHJlZGljdG9yIGZvcm11 bGEgdXBkYXRlCgpJbiBvcmRlciB0byBiZSBjb21wYXRpYmxlIHdpdGggZmlsZXMgaW4gdGhl IHdpbGQKLS0tCiBmZnYxLm1kIHwgMTIgKysrKysrKysrKysrCiAxIGZpbGUgY2hhbmdlZCwg MTIgaW5zZXJ0aW9ucygrKQoKZGlmZiAtLWdpdCBhL2ZmdjEubWQgYi9mZnYxLm1kCmluZGV4 IGY5OTY1OTUuLjI0ZDVlMWMgMTAwNjQ0Ci0tLSBhL2ZmdjEubWQKKysrIGIvZmZ2MS5tZApA QCAtMTY3LDYgKzE2NywxOCBAQCBsZWZ0LCB0b3AsIGRpYWcgYXJlIHRoZSBsZWZ0LCB0b3Ag YW5kIGxlZnQtdG9wIHNhbXBsZXMKIAogTm90ZSwgdGhpcyBpcyBhbHNvIHVzZWQgaW4gW0pQ RUctTFMgYW5kIEh1ZmZZdXZdKCNyZWZlcmVuY2VzKS4KIAorRXhjZXB0aW9uIGZvciB0aGUg bWVkaWEgcHJlZGljdG9yOgoraWYgY29sb3JzcGFjZV90eXBlID09IDAgJiYgYml0c19wZXJf cmF3X3NhbXBsZSA9PSAxNiAmJiAoIGNvZGVyX3R5cGUgPT0gMSB8fCBjb2Rlcl90eXBlID09 IDIgKSwgdGhlIGZvbGxvd2luZyBtZWRpYSBwcmVkaWN0b3IgTVVTVCBiZSB1c2VkOgorCitt ZWRpYW4obGVmdDE2cywgdG9wMTZzLCBsZWZ0MTZzICsgdG9wMTZzIC0gZGlhZzE2cykKKwor d2l0aDogIAorLSBsZWZ0MTZzID0gbGVmdCA+PSAzMjc2OCA/ICggbGVmdCAtIDY1NTM2ICkg OiBsZWZ0ICAKKy0gdG9wMTZzID0gdG9wID49IDMyNzY4ID8gKCB0b3AgLSA2NTUzNiApIDog dG9wICAKKy0gZGlhZzE2cyA9IGRpYWcgPj0gMzI3NjggPyAoIGRpYWcgLSA2NTUzNiApIDog ZGlhZyAgCisKK0JhY2tncm91bmQ6IGEgdHdvJ3MgY29tcGxlbWVudCBzaWduZWQgMTYtYml0 IHNpZ25lZCBpbnRlZ2VyIHdhcyB1c2VkIGZvciBzdG9yaW5nIHBpeGVsIHZhbHVlcyBpbiBh bGwga25vd24gaW1wbGVtZW50YXRpb25zIG9mIEZGVjEgYml0c3RyZWFtLiBTbyBpbiBzb21l IGNpcmN1bXN0YW5jZXMsIHRoZSBtb3N0IHNpZ25pZmljYW50IGJpdCB3YXMgd3JvbmdseSBp bnRlcnByZXRlZCAodXNlZCBhcyBhIHNpZ24gYml0IGluc3RlYWQgb2YgdGhlIDE2dGggYml0 IG9mIGFuIHVuc2lnbmVkIGludGVnZXIpLiBOb3RlIHRoYXQgd2hlbiB0aGUgaXNzdWUgaXMg ZGlzY292ZXJlZCwgdGhlIG9ubHkgY29uZmlndXJhdGlvbiBvZiBhbGwga25vd24gaW1wbGVt ZW50YXRpb25zIGJlaW5nIGltcGFjdGVkIGlzIDE2LWJpdCBZQ2JDciBjb2xvciBzcGFjZSB3 aXRoIFJhbmdlIENvZGVyIGNvZGVyLCBhcyBvdGhlciBwb3RlbnRpYWxseSBpbXBhY3RlZCBj b25maWd1cmF0aW9ucyAoZS5nLiAxNS8xNi1iaXQgSlBFRyAyMDAwIFJDVCBjb2xvciBzcGFj ZSB3aXRoIFJhbmdlIENvZGVyIGNvZGVyLCBvciAxNi1iaXQgYW55IGNvbG9yIHNwYWNlIHdp dGggR29sb21iIFJpY2UgY29kZXIpIHdlcmUgaW1wbGVtZW50ZWQgbm93aGVyZS4gSW4gdGhl IG1lYW53aGlsZSwgMTYtYml0IEpQRUcgMjAwMCBSQ1QgY29sb3Igc3BhY2Ugd2l0aCBSYW5n ZSBDb2RlciBjb2RlciB3YXMgaW1wbGVtZW50ZWQgd2l0aG91dCB0aGlzIGlzc3VlIGluIG9u ZSBpbXBsZW1lbnRhdGlvbiBhbmQgdmFsaWRhdGVkIGJ5IG9uZSBjb25mb3JtYW5jZSBjaGVj a2VyLiBJdCBpcyBleHBlY3RlZCAodG8gYmUgY29uZmlybWVkKSB0byByZW1vdmUgdGhpcyBl eGNlcHRpb24gZm9yIHRoZSBtZWRpYSBwcmVkaWN0b3IgaW4gdGhlIG5leHQgdmVyc2lvbiBv ZiB0aGUgYml0c3RyZWFtLgorCiAjIyBDb250ZXh0CiAKIHwgICB8ICAgfCAgIHwgICB8Ci0t IAoyLjcuMC53aW5kb3dzLjEKCg== --------------70C4B4EC542473DFDC4D5E4B-- From nobody Tue Sep 6 12:50:12 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D6ED712B438 for ; Tue, 6 Sep 2016 12:50:10 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.001 X-Spam-Level: X-Spam-Status: No, score=-5.001 tagged_above=-999 required=5 tests=[BAYES_20=-0.001, RCVD_IN_DNSWL_HI=-5] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TX8zba_hb6J7 for ; Tue, 6 Sep 2016 12:50:09 -0700 (PDT) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5016C12B435 for ; Tue, 6 Sep 2016 12:50:08 -0700 (PDT) Received: from eris.local (dynamic-adsl-84-221-242-241.clienti.tiscali.it [84.221.242.241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: lu_zero) by smtp.gentoo.org (Postfix) with ESMTPSA id 83E38340961 for ; Tue, 6 Sep 2016 19:50:07 +0000 (UTC) To: cellar@ietf.org References: <9146e646-0afb-7dfb-911b-b3f7c901ca71@mediaarea.net> From: Luca Barbato Message-ID: <4472f91d-da83-3eba-04ab-7edba8b065e0@libav.org> Date: Tue, 6 Sep 2016 21:50:03 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.2 MIME-Version: 1.0 In-Reply-To: <9146e646-0afb-7dfb-911b-b3f7c901ca71@mediaarea.net> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [Cellar] [PATCH FFV1] Median Predictor formula update X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Sep 2016 19:50:11 -0000 On 06/09/16 18:56, Jerome Martinez wrote: > Impacted streams are FFV1 configured with 16-bit YCbCr color space and > Range Coder coder (this includes 16-bit gray because it is YCbCr color > space with chroma deactivated). Maybe we can make that generic in case somebody is creative enough to try to use even higher bitdepth ? lu From nobody Tue Sep 6 13:09:16 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AE6DC12B0C6 for ; Tue, 6 Sep 2016 13:09:15 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.921 X-Spam-Level: X-Spam-Status: No, score=-1.921 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id i2qAsz3JaQ_2 for ; Tue, 6 Sep 2016 13:09:13 -0700 (PDT) Received: from 6.mo173.mail-out.ovh.net (6.mo173.mail-out.ovh.net [46.105.43.93]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 65C2412B043 for ; Tue, 6 Sep 2016 13:09:13 -0700 (PDT) Received: from player157.ha.ovh.net (b9.ovh.net [213.186.33.59]) by mo173.mail-out.ovh.net (Postfix) with ESMTP id 934671008908 for ; Tue, 6 Sep 2016 22:09:11 +0200 (CEST) Received: from [192.168.2.101] (p5DDB5F23.dip0.t-ipconnect.de [93.219.95.35]) (Authenticated sender: jerome@francoallemand.eu) by player157.ha.ovh.net (Postfix) with ESMTPSA id 43A83500072 for ; Tue, 6 Sep 2016 22:09:10 +0200 (CEST) To: cellar@ietf.org References: <9146e646-0afb-7dfb-911b-b3f7c901ca71@mediaarea.net> <4472f91d-da83-3eba-04ab-7edba8b065e0@libav.org> From: Jerome Martinez Message-ID: <802b986a-e6ee-7cfe-e255-d3b7cf0e46fd@mediaarea.net> Date: Tue, 6 Sep 2016 22:09:10 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <4472f91d-da83-3eba-04ab-7edba8b065e0@libav.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-Ovh-Tracer-Id: 2478950121503199377 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 0 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrfeeluddrheelgddugeekucdltddurdefledtrddttddmucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfenuceurghilhhouhhtmecufedttdenuc Archived-At: Subject: Re: [Cellar] [PATCH FFV1] Median Predictor formula update X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Sep 2016 20:09:16 -0000 Le 06/09/2016 21:50, Luca Barbato a crit : > On 06/09/16 18:56, Jerome Martinez wrote: >> Impacted streams are FFV1 configured with 16-bit YCbCr color space and >> Range Coder coder (this includes 16-bit gray because it is YCbCr color >> space with chroma deactivated). > Maybe we can make that generic in case somebody is creative enough to > try to use even higher bitdepth ? I don't follow: the specification supports any bit depth, so people wanting to use e.g. bits_per_raw_sample = 1000 can do it (if they have code for handling maths with 1000-bit or 1001-bit integers for YUV or RGB, and the corresponding CPU :) ). the exception is only for files already in the wild, and this "bug" is limited to streams with 16-bit YCbCr color space because the "bug" was discover before 16-bit RGB is implemented in an encoder (and the bug has no impact on streams with bitdepth <16). The goal is actually to permit such creative people to use 32-bit integers (for example: newly 16-bit RGB support needs 17 bits for intermediate storage, so a 32-bit integer is used). Long stories short, the patch is for making that generic in case somebody is creative enough to try to use even higher bitdepth, with support of files already in the wild (for example, the support of FFV1 in MediaConch is being upgraded to maximum 31-bit per component thanks to 32-bit integers, we could expand it to 63-bit per component if need with an upgrade to 64-bit integers, and we added an exception for 16-bit YCbCr color space only because the files in the wild are not compatible with the generic method) Note that we propose to remove the exception in the next version of FFV1 so it will be 100% generic in the future. From nobody Tue Sep 6 13:17:02 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 00C7A12B0C6 for ; Tue, 6 Sep 2016 13:17:01 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.9 X-Spam-Level: X-Spam-Status: No, score=-6.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eAKAIZ8M_bMf for ; Tue, 6 Sep 2016 13:17:00 -0700 (PDT) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1E0F212B069 for ; Tue, 6 Sep 2016 13:17:00 -0700 (PDT) Received: from eris.local (dynamic-adsl-84-221-242-241.clienti.tiscali.it [84.221.242.241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: lu_zero) by smtp.gentoo.org (Postfix) with ESMTPSA id D2F7B3408B8 for ; Tue, 6 Sep 2016 20:16:58 +0000 (UTC) To: cellar@ietf.org References: <9146e646-0afb-7dfb-911b-b3f7c901ca71@mediaarea.net> <4472f91d-da83-3eba-04ab-7edba8b065e0@libav.org> <802b986a-e6ee-7cfe-e255-d3b7cf0e46fd@mediaarea.net> From: Luca Barbato Message-ID: Date: Tue, 6 Sep 2016 22:16:54 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.2 MIME-Version: 1.0 In-Reply-To: <802b986a-e6ee-7cfe-e255-d3b7cf0e46fd@mediaarea.net> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit Archived-At: Subject: Re: [Cellar] [PATCH FFV1] Median Predictor formula update X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Sep 2016 20:17:01 -0000 On 06/09/16 22:09, Jerome Martinez wrote: > Le 06/09/2016 21:50, Luca Barbato a crit : >> On 06/09/16 18:56, Jerome Martinez wrote: >>> Impacted streams are FFV1 configured with 16-bit YCbCr color space and >>> Range Coder coder (this includes 16-bit gray because it is YCbCr color >>> space with chroma deactivated). >> Maybe we can make that generic in case somebody is creative enough to >> try to use even higher bitdepth ? > Note that we propose to remove the exception in the next version of FFV1 > so it will be 100% generic in the future. Ah, good as proposed then =) lu From nobody Wed Sep 7 12:09:31 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C40B712B28A for ; Wed, 7 Sep 2016 12:09:29 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.12 X-Spam-Level: X-Spam-Status: No, score=-1.12 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JVZKLfqVn9gb for ; Wed, 7 Sep 2016 12:09:28 -0700 (PDT) Received: from s172.web-hosting.com (s172.web-hosting.com [68.65.122.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4DB7B12B24E for ; Wed, 7 Sep 2016 12:09:28 -0700 (PDT) Received: from [146.96.19.240] (port=31252 helo=[10.10.202.53]) by server172.web-hosting.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.86_1) (envelope-from ) id 1bhiDs-00447K-0C for cellar@ietf.org; Wed, 07 Sep 2016 15:09:27 -0400 From: Dave Rice Content-Type: multipart/alternative; boundary="Apple-Mail=_D944A4C6-88F5-4B28-8994-047B4FE11FD7" Message-Id: <900FE24A-0670-4A99-9CE0-300ED91103DB@dericed.com> Date: Wed, 7 Sep 2016 15:09:22 -0400 To: cellar@ietf.org Mime-Version: 1.0 (Mac OS X Mail 9.2 \(3112\)) X-Mailer: Apple Mail (2.3112) X-OutGoing-Spam-Status: No, score=-2.9 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Subject: [Cellar] Matroska metadata for digitization and process history X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Sep 2016 19:09:29 -0000 --Apple-Mail=_D944A4C6-88F5-4B28-8994-047B4FE11FD7 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hi all, I wanted to bring a discussion here which happened a bit at the No Time = to Wait symposium a few months ago regarding metadata in Matroska about = digitization, transfer, and source. This was a discussion on how to = store data in Matroska about the history of that stored audiovisual = data, which would cover the source of that data and the historical = process used to create the current Matroska file, possibly itemizes out = that process generation by generation. For precedence a few other av standards cover this type of data, for = instance in WAV files the bext = chunk contains = 'codinghistory' which is a list of historical processes used to create = the stored audio. An example of bext coding history: A=3DANALOG,M=3Dmono,T=3DStuder A810; SN3690; 15 ips; open reel tape, A=3DPCM,F=3D96000,W=3D24,M=3Dmono,T=3DBenchmark; ADC1; SN00252; A/D, A=3DPCM,F=3D96000,W=3D24,M=3Dmono,T=3DLynx; AES16; DIO, Also etree has a database where md5 and ffps values from SHN and FLAC = files can be related to historical processes data: see = http://db.etree.org/shnlist.php?artist=3D2&year=3D1995 (in this case the = process data is stored in a separate database rather than the file = itself but the scope is similar). Some use cases I'm thinking of: - A set of Matroska files are created using various digitization = processes and hardware and those processes should be document (example: = 1.mkv created from the output of a film scanner, 2.mkv from running a = VHS through component video to a blackmagic card and 3.mkv from running = a Betamax through cosposite video to an Elgato capture card). In some = cases the data could be collected as introduced into the mkv file as it = is written and in other cases it could be added after via mkvpropedit. - Another case, an mkv is created from a film scanner, then another mkv = is created from the first to make some correct corrections, and then a = third mkv is created as an access copy with a lossy compressed codec. = The process data of each mkv file could be added to with each = generational move, so that with the third file the history of its = creation could be clear. - One mkv file is used to create another at a different set of = characteristics, such data could communicate that the data used to be in = a different frame rate, interlacement pattern, or colorspace to give = some context to why it looks the way it does. There is some overlap with this concept and the PREMIS = metadata standard which documents = an XML document that defines and relates concepts such as Agents = (hardware, software, people), Events (digitization, transcoding, = processing), and Objects (source mkv files, source videotapes, source = film, etc). It could be feasible to simply determine a method of = documenting this data in PREMIS XML and then store that as an = attachment, though I think it makes it more accessible as a set of = Matroska metadata tags. I tried working on this subject a few years ago and created this sample = which attaches a photo of the = source tape and adds process metadata. Here's an output of that metadata = from FFmpeg. Metadata: CREATION_TIME : 2011-05-03 02:12:03 COMPATIBLE_BRANDS: qt =20 ORIGINAL_MEDIA_TYPE: {Physical format of the source tape. = Vocabulary?} ORIGINAL_MEDIA_TYPE/BARCODE: {Barcode or other identifier from the = tape.} ORIGINAL_MEDIA_TYPE/BARCODE/source: {name of barcode authority or = creator; example "XYZ Archive Barcode"} ORIGINAL_MEDIA_TYPE/LABEL: {The record label or imprint on the = source media object.} ORIGINAL_MEDIA_TYPE/DATE_RECORDED: {The time that the recording = began. This is akin to the TDRC tag in ID3.} ORIGINAL_MEDIA_TYPE/condition: {Comments on the condition of the = source, any preparation for playback of the source tape (if relevant).} ORIGINAL_MEDIA_TYPE/initial_source_timecode: 01:02:15;10 ORIGINAL_MEDIA_TYPE/initial_source_timecode/settings: DropFrame=3DYes = / 24HourMax=3DNo / IsVisual=3DNo DATE_DIGITIZED : {Date and time of the file creation process. Enter = in ISO 8601 format.} ENCODED_BY : {Name of the technician or organization = responsible for the encoding and file creation process} ENCODED_BY/URL : {URL for the technician or organization listed in = ENCODED_BY.} ENCODED_BY/processing_actions: {Description of any actions performed = during processing, such as trimming silence.} ENCODED_BY/capture_software: Blackmagic Media Express ENCODED_BY/capture_software/version: 2.0.3 ENCODED_BY/capture_operating_system: Ubuntu ENCODED_BY/capture_operating_system/version: 10.10 ENCODED_BY/capture_device: Decklink Studio SDI ENCODED_BY/capture_device/manufacturer: Blackmagic-Design ENCODED_BY/capture_device/serial_no: xyz6789 ENCODED_BY/capture_device/settings: {notes on adjustments or = settings} ENCODED_BY/playback_device: SVO-5800 ENCODED_BY/playback_device/manufacturer: Sony ENCODED_BY/playback_device/serial_no: abc1234 ENCODED_BY/playback_device/settings: {notes on adjustments or = settings} ENCODED_BY/playback_device/playback_signal_path: {Protocols used to = transfer audiovisual data between the playback deck and the capture = device. example: Component} MAJOR_BRAND : qt =20 MINOR_VERSION : 537199360 ENCODER : Lavf53.2.0 Official tags are in ALL CAPS and my own local tags are in lowercase. I propose a discussion on: - Is defining process metadata (aka digitization metadata, coding = history) worthwhile to include in CELLAR work? - Should such metadata be added to Matroska tags or should we use = existing metadata standards as attachments? - Or (building off the prior question) should we add namespaces to = Matroska metadata tags so that externally defined standards may be used = within Matroska metadata directly? - And .... what metadata values are needed? Best Regards, Dave Rice= --Apple-Mail=_D944A4C6-88F5-4B28-8994-047B4FE11FD7 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii Hi all,

I = wanted to bring a discussion here which happened a bit at the No Time to = Wait symposium a few months ago regarding metadata in Matroska about = digitization, transfer, and source. This was a discussion on how to = store data in Matroska about the history of that stored audiovisual = data, which would cover the source of that data and the historical = process used to create the current Matroska file, possibly itemizes out = that process generation by generation.

For precedence a few other av standards = cover this type of data, for instance in WAV files = the bext chunk contains 'codinghistory' which is a list of = historical processes used to create the stored audio. An example of bext = coding history:

A=3DANALOG,M=3Dmono,T=3DStuder A810; SN3690; 15 ips; open = reel tape,
A=3DPCM,F=3D96000,W=3D24,M=3Dmono,T=3DBenchmark; = ADC1; SN00252; A/D,
A=3DPCM,F=3D96000,W=3D24,M=3Dmono,T=3DLy= nx; AES16; DIO,

Also etree has a database where md5 and ffps values from SHN = and FLAC files can be related to historical processes data: see http://db.etree.org/shnlist.php?artist=3D2&year=3D1995 = (in this case the process data is stored in a separate database rather = than the file itself but the scope is similar).

Some use cases I'm thinking = of:

- A set of = Matroska files are created using various digitization processes and = hardware and those processes should be document (example: 1.mkv created = from the output of a film scanner, 2.mkv from running a VHS through = component video to a blackmagic card  and 3.mkv from running a = Betamax through cosposite video to an Elgato capture card). In some = cases the data could be collected as introduced into the mkv file as it = is written and in other cases it could be added after via = mkvpropedit.

- = Another case, an mkv is created from a film scanner, then another mkv is = created from the first to make some correct corrections, and then a = third mkv is created as an access copy with a lossy compressed codec. = The process data of each mkv file could be added to with each = generational move, so that with the third file the history of its = creation could be clear.

- One mkv file is used to create another at a different set = of characteristics, such data could communicate that the data used to be = in a different frame rate, interlacement pattern, or colorspace to give = some context to why it looks the way it does.

There is some overlap with this concept = and the PREMIS metadata standard which documents an XML = document that defines and relates concepts such as Agents (hardware, = software, people), Events (digitization, transcoding, processing), and = Objects (source mkv files, source videotapes, source film, etc). It = could be feasible to simply determine a method of documenting this data = in PREMIS XML and then store that as an attachment, though I think it = makes it more accessible as a set of Matroska metadata tags.

I tried working on this = subject a few years ago and created this sample which attaches a photo of the source tape and = adds process metadata. Here's an output of that metadata from = FFmpeg.

  Metadata:
  =   CREATION_TIME   : 2011-05-03 02:12:03
    COMPATIBLE_BRANDS: qt  
    ORIGINAL_MEDIA_TYPE: {Physical format of = the source tape. Vocabulary?}
  =   ORIGINAL_MEDIA_TYPE/BARCODE: {Barcode or other identifier = from the tape.}
  =   ORIGINAL_MEDIA_TYPE/BARCODE/source: {name of barcode = authority or creator; example "XYZ Archive Barcode"}
    ORIGINAL_MEDIA_TYPE/LABEL: {The record = label or imprint on the source media object.}
  =   ORIGINAL_MEDIA_TYPE/DATE_RECORDED: {The time that the = recording began. This is akin to the TDRC tag in ID3.}
    ORIGINAL_MEDIA_TYPE/condition: {Comments = on the condition of the source, any preparation for playback of the = source tape (if relevant).}
  =   ORIGINAL_MEDIA_TYPE/initial_source_timecode: 01:02:15;10
  =   ORIGINAL_MEDIA_TYPE/initial_source_timecode/settings: = DropFrame=3DYes / 24HourMax=3DNo / IsVisual=3DNo
 =   DATE_DIGITIZED  : {Date and time of the file = creation process. Enter in ISO 8601 format.}
  =   ENCODED_BY      : {Name of the = technician or organization responsible for the encoding and file = creation process}
  =   ENCODED_BY/URL  : {URL for the technician or = organization listed in ENCODED_BY.}
  =   ENCODED_BY/processing_actions: {Description of any actions = performed during processing, such as trimming silence.}
    ENCODED_BY/capture_software: Blackmagic = Media Express
  =   ENCODED_BY/capture_software/version: 2.0.3
    ENCODED_BY/capture_operating_system: = Ubuntu
  =   ENCODED_BY/capture_operating_system/version: 10.10
    ENCODED_BY/capture_device: Decklink Studio = SDI
  =   ENCODED_BY/capture_device/manufacturer: Blackmagic-Design
    ENCODED_BY/capture_device/serial_no: = xyz6789
  =   ENCODED_BY/capture_device/settings: {notes on adjustments or = settings}
    ENCODED_BY/playback_device: = SVO-5800
  =   ENCODED_BY/playback_device/manufacturer: Sony
    ENCODED_BY/playback_device/serial_no: = abc1234
  =   ENCODED_BY/playback_device/settings: {notes on adjustments = or settings}
  =   ENCODED_BY/playback_device/playback_signal_path: {Protocols = used to transfer audiovisual data between the playback deck and the = capture device. example: Component}
  =   MAJOR_BRAND     : qt  
    MINOR_VERSION   : = 537199360
    ENCODER     =     : Lavf53.2.0

Official tags are in ALL CAPS and my own local tags are in = lowercase.

I = propose a discussion on:

- Is defining process metadata (aka digitization metadata, = coding history) worthwhile to include in CELLAR work?

- Should such metadata = be added to Matroska tags or should we use existing metadata standards = as attachments?

- Or (building off the prior question) should we add = namespaces to Matroska metadata tags so that externally defined = standards may be used within Matroska metadata directly?

- And .... what metadata = values are needed?

Best Regards,
Dave = Rice
= --Apple-Mail=_D944A4C6-88F5-4B28-8994-047B4FE11FD7-- From nobody Wed Sep 7 17:53:43 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DA1E212B318 for ; Wed, 7 Sep 2016 17:53:41 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.699 X-Spam-Level: X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rfIeAUFhZYuz for ; Wed, 7 Sep 2016 17:53:39 -0700 (PDT) Received: from mail-wm0-x22b.google.com (mail-wm0-x22b.google.com [IPv6:2a00:1450:400c:c09::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1134712B01F for ; Wed, 7 Sep 2016 17:53:39 -0700 (PDT) Received: by mail-wm0-x22b.google.com with SMTP id w12so8670076wmf.0 for ; Wed, 07 Sep 2016 17:53:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=x2xITUZLjMeLfjfvQ+ySk7DNSv0vys+bY6Mrw3EHJZI=; b=Pzl3wASYGL2ZQr9i+28HXLCuaj6PzqeaSSYp75MtwwpqVavFoRCRbWD1Xl7pclbXr1 kZgd6aYw1dD6R6hGdOxnn8ZOnlZaeuJeUbsq/+hhL7BuEmbYhLtWrGlWrgNmcQcq2znt buyyfZ6VNIQSDl2Fq2zCcs/t8hleibKgirqYTDuG1kzp4Qn5liQXbxxd5iyz2dVQjCEX 49W9TCTibke4uLF+l4LSJL1sVH+xmJH7NSa2H0F4lzUToD9cw96wArxvGA3EoxyyMmgi pscU4e0MRVIWxJsuH+xGz6hslb8yWHeM2BAgXCLo/Sk2aaLOnk7WHpuziWCx4fnIBQ4F WosQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=x2xITUZLjMeLfjfvQ+ySk7DNSv0vys+bY6Mrw3EHJZI=; b=GdYtHF7+55GQJZe1+dGyUbPxttKtwvPsCVf0ADYf8Q83s6XXLsavy3kpUTqyLRyGoN JVfNXwDmPprMsvXeUpAE3/OKXEy/JEdUh12y8867HdvEdc3lmL/JWDaHdBtH/ZJLGlOE /onHQooODNQNTJskEwlDuEBnielL4TyPqWIt+OCXGo+DJJFzNKw7hgCWRzgDRYQsqUGM +gojh3g78FnSlSgt7Fg/X7mo1k1FaFGbV2zUvEgCkb6Fi/FshazymLuQuoutDVFSHfnn A0yDmm+aSmAs4DIJGuSm+2OL3B+a5bGwL7X8tjQoVJchc9cizII4ZqbSaqWjv2d4L62J 6PIQ== X-Gm-Message-State: AE9vXwNLF4h6mMliR/ksxQkjkx5p5mvcFHaHYTULFDG2Ei5TkCecGGCr4nAMpinYlXQyquHgnCqkMKDlhMTAbA== X-Received: by 10.28.185.202 with SMTP id j193mr6047217wmf.78.1473296017265; Wed, 07 Sep 2016 17:53:37 -0700 (PDT) MIME-Version: 1.0 Received: by 10.194.85.37 with HTTP; Wed, 7 Sep 2016 17:53:36 -0700 (PDT) Received: by 10.194.85.37 with HTTP; Wed, 7 Sep 2016 17:53:36 -0700 (PDT) In-Reply-To: <900FE24A-0670-4A99-9CE0-300ED91103DB@dericed.com> References: <900FE24A-0670-4A99-9CE0-300ED91103DB@dericed.com> From: Kieran O Leary Date: Thu, 8 Sep 2016 01:53:36 +0100 Message-ID: To: cellar@ietf.org Content-Type: multipart/alternative; boundary=001a1148ef26c21c55053bf4755e Archived-At: Subject: Re: [Cellar] Matroska metadata for digitization and process history X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Sep 2016 00:53:42 -0000 --001a1148ef26c21c55053bf4755e Content-Type: text/plain; charset=UTF-8 Hi, I'm very happy that this topic has come up for discussion. On Wed, Sep 7, 2016 at 8:09 PM, Dave Rice wrote: > > - A set of Matroska files are created using various digitization processes > and hardware and those processes should be document (example: 1.mkv created > from the output of a film scanner, 2.mkv from running a VHS through > component video to a blackmagic card and 3.mkv from running a Betamax > through cosposite video to an Elgato capture card). In some cases the data > could be collected as introduced into the mkv file as it is written and in > other cases it could be added after via mkvpropedit. > Inserted as it is written sounds very interesting, as the capture file does not end up with an altered whole file checksum, which is what would happen with post capture metadata insertion. I'm not sure of how mkvpropedit works, and I wonder if a framemd5 check of the streams would be paranoid or reasonable after such an update? Perhaps the embedding of stream level checksums is a good idea at this stage? I saw that Andrew Weaver was experimenting with this on vrecord. > - Another case, an mkv is created from a film scanner, then another mkv is > created from the first to make some correct corrections, and then a third > mkv is created as an access copy with a lossy compressed codec. The process > data of each mkv file could be added to with each generational move, so that > with the third file the history of its creation could be clear. > This is pretty much an exact scenario that the Irish Film Archive is dealing with right now. We're investigating PREMIS as a means of recording this information and linking all the different assets to each other via UUIDs. Does/could mkvpropedit generate UUIDs? It would be interesting if it did, as the identifier could then be harvested and also inserted into a PREMIS xml description afterwards.. Digital Cinema Packages use UUIDs in their xml files (also embedded in the mxfs) in order to create relationships between single/multi reel audio, visual and subtitle files. Perhaps they could also help create this provenance description among this chain of film assets as well? I don't think that there would be an automated way to make the link between assets though. > - One mkv file is used to create another at a different set of > characteristics, such data could communicate that the data used to be in a > different frame rate, interlacement pattern, or colorspace to give some > context to why it looks the way it does. > I think that this is a very valid use case, especially for access copies. I can see the potential for something like vimeo or other streaming platforms being able to possibly harvest and display this metadata if they bought into this idea? > There is some overlap with this concept and the PREMIS metadata standard > which documents an XML document that defines and relates concepts such as > Agents (hardware, software, people), Events (digitization, transcoding, > processing), and Objects (source mkv files, source videotapes, source film, > etc). It could be feasible to simply determine a method of documenting this > data in PREMIS XML and then store that as an attachment, though I think it > makes it more accessible as a set of Matroska metadata tags. > I'm not very familiar with Matroska attachments. I think this is a nice idea for distributing files publicly, in that a user could gain additional context if required. For long term preservation, I can't help but think that the PREMIS xml would function just as well in a sidecar in an AIP, with the added benefit of not altering the checksum of the Matroska file. > I tried working on this subject a few years ago and created this sample > which attaches a photo of the source tape and adds process metadata. Here's > an output of that metadata from FFmpeg. > > Metadata: > CREATION_TIME : 2011-05-03 02:12:03 > COMPATIBLE_BRANDS: qt > ORIGINAL_MEDIA_TYPE: {Physical format of the source tape. Vocabulary?} > ORIGINAL_MEDIA_TYPE/BARCODE: {Barcode or other identifier from the > tape.} > ORIGINAL_MEDIA_TYPE/BARCODE/source: {name of barcode authority or > creator; example "XYZ Archive Barcode"} > ORIGINAL_MEDIA_TYPE/LABEL: {The record label or imprint on the source > media object.} > ORIGINAL_MEDIA_TYPE/DATE_RECORDED: {The time that the recording began. > This is akin to the TDRC tag in ID3.} > ORIGINAL_MEDIA_TYPE/condition: {Comments on the condition of the source, > any preparation for playback of the source tape (if relevant).} > ORIGINAL_MEDIA_TYPE/initial_source_timecode: 01:02:15;10 > ORIGINAL_MEDIA_TYPE/initial_source_timecode/settings: DropFrame=Yes / > 24HourMax=No / IsVisual=No > DATE_DIGITIZED : {Date and time of the file creation process. Enter in > ISO 8601 format.} > ENCODED_BY : {Name of the technician or organization responsible > for the encoding and file creation process} > ENCODED_BY/URL : {URL for the technician or organization listed in > ENCODED_BY.} > ENCODED_BY/processing_actions: {Description of any actions performed > during processing, such as trimming silence.} > ENCODED_BY/capture_software: Blackmagic Media Express > ENCODED_BY/capture_software/version: 2.0.3 > ENCODED_BY/capture_operating_system: Ubuntu > ENCODED_BY/capture_operating_system/version: 10.10 > ENCODED_BY/capture_device: Decklink Studio SDI > ENCODED_BY/capture_device/manufacturer: Blackmagic-Design > ENCODED_BY/capture_device/serial_no: xyz6789 > ENCODED_BY/capture_device/settings: {notes on adjustments or settings} > ENCODED_BY/playback_device: SVO-5800 > ENCODED_BY/playback_device/manufacturer: Sony > ENCODED_BY/playback_device/serial_no: abc1234 > ENCODED_BY/playback_device/settings: {notes on adjustments or settings} > ENCODED_BY/playback_device/playback_signal_path: {Protocols used to > transfer audiovisual data between the playback deck and the capture device. > example: Component} > MAJOR_BRAND : qt > MINOR_VERSION : 537199360 > ENCODER : Lavf53.2.0 > > Official tags are in ALL CAPS and my own local tags are in lowercase. I think that looks great! Were you local tags based on a controlled vocabulary? > > I propose a discussion on: > > - Is defining process metadata (aka digitization metadata, coding history) > worthwhile to include in CELLAR work? > Just to be 100% clear, do you mean defining process metadata for insertion into a Matroska container, or a general vocabulary/schema? Perhaps the latter is out of the scope of CELLAR, but selfishly I'd like it included:)) I'm very interested in this topic and think that it is essential metadata that should be captured. I'm on the fence with regards to embedding the metadata versus storing it in a sidecar file or database. > - Should such metadata be added to Matroska tags or should we use existing > metadata standards as attachments? > I'm in favour of increasing the number of Matroska's official tags. Would using existing metadata standards as attachments require any extra work or discussion as attaching metadata xml files could probably be done right now? I took a look at the Matroska tags recently and they do not map well to process history for digitisation. As Matroska is increasingly used as a format for digitisation and preservation, I think it would be very useful if the tag vocabulary could support this kind of metadata. > - Or (building off the prior question) should we add namespaces to Matroska > metadata tags so that externally defined standards may be used within > Matroska metadata directly? This is interesting, but can you give an example? Would a tag look something like PREMIS:AgentType=Software? I wonder if this is actually the best of both worlds, as it allows reference to other standards through embedding rather than attachments. I'm not sure how this would be any different to your example of using a customised local vocabulary, unless there was some kind of reference to these standards built into the Matroska spec, which could even be validated? > > - And .... what metadata values are needed? I think that this is a great discussion topic that would have benefit beyond the scope of Matroska. PREMIS is ultimately quite vague in terms of what is actually recorded. It would be very helpful if archives shared what kind of information they recorded pre/during/post capture. I'm sure that we would be happy to share the outcomes of what we record in PREMIS for our film and tape workflows, once we actually get that far. Best, Kieran. --001a1148ef26c21c55053bf4755e Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

Hi,

I'm very happy that this topic has come up for discussio= n.

On Wed, Sep 7, 2016 at 8:09 PM, Dave Rice <dave@dericed.com> wrote:
>
> - A set of Matroska files are created using various digitization proce= sses
> and hardware and those processes should be document (example: 1.mkv cr= eated
> from the output of a film scanner, 2.mkv from running a VHS through > component video to a blackmagic card and 3.mkv from running a Betamax<= br> > through cosposite video to an Elgato capture card). In some cases the = data
> could be collected as introduced into the mkv file as it is written an= d in
> other cases it could be added after via mkvpropedit.
>

Inserted as it is written sounds very interesting, as the ca= pture file does not end up with an altered whole file checksum, which is wh= at would happen with post capture metadata insertion. I'm not sure of h= ow mkvpropedit works, and I wonder if a framemd5 check of the streams would= be paranoid or reasonable after such an update? Perhaps the embedding of s= tream level checksums is a good idea at this stage? I saw that Andrew Weave= r was experimenting with this on vrecord.

> - Another case, an mkv is created from a film scanner, = then another mkv is
> created from the first to make some correct corrections, and then a th= ird
> mkv is created as an access copy with a lossy compressed codec. The pr= ocess
> data of each mkv file could be added to with each generational move, s= o that
> with the third file the history of its creation could be clear.
>

This is pretty much an exact scenario that the Irish Film Ar= chive is dealing with right now. We're investigating PREMIS as a means = of recording this information and linking all the different assets to each = other via UUIDs. Does/could mkvpropedit generate UUIDs? It would be interes= ting if it did, as the identifier could then be harvested and also inserted= into a PREMIS xml description afterwards.. Digital Cinema Packages use UUI= Ds in their xml files (also embedded in the mxfs) in order to create relati= onships between single/multi reel audio, visual and subtitle files. Perhaps= they could also help create this provenance description among this chain o= f film assets as well? I don't think that there would be an automated w= ay to make the link between assets though.

> - One mkv file is used to create another at a different= set of
> characteristics, such data could communicate that the data used to be = in a
> different frame rate, interlacement pattern, or colorspace to give som= e
> context to why it looks the way it does.
>

I think that this is a very valid use case, especially for a= ccess copies. I can see the potential for something like vimeo or other str= eaming platforms being able to possibly harvest and display this metadata i= f they bought into this idea?

> There is some overlap with this concept and the PREMIS = metadata standard
> which documents an XML document that defines and relates concepts such= as
> Agents (hardware, software, people), Events (digitization, transcoding= ,
> processing), and Objects (source mkv files, source videotapes, source = film,
> etc). It could be feasible to simply determine a method of documenting= this
> data in PREMIS XML and then store that as an attachment, though I thin= k it
> makes it more accessible as a set of Matroska metadata tags.
>

I'm not very familiar with Matroska attachments. I think= this is a nice idea for distributing files publicly, in that a user could = gain additional context if required. For long term preservation, I can'= t help but think that the PREMIS xml would function just as well in a sidec= ar in an AIP, with the added benefit of not altering the checksum of the Ma= troska file.

> I tried working on this subject a few years ago and cre= ated this sample
> which attaches a photo of the source tape and adds process metadata. H= ere's
> an output of that metadata from FFmpeg.
>
> Metadata:
> CREATION_TIME : 2011-05-03 02:12:03
> COMPATIBLE_BRANDS: qt
> ORIGINAL_MEDIA_TYPE: {Physical format of the source tape. Vocabulary?}=
> ORIGINAL_MEDIA_TYPE/BARCODE: {Barcode or other identifier from the
> tape.}
> ORIGINAL_MEDIA_TYPE/BARCODE/source: {name of barcode authority or
> creator; example "XYZ Archive Barcode"}
> ORIGINAL_MEDIA_TYPE/LABEL: {The record label or imprint on the source<= br> > media object.}
> ORIGINAL_MEDIA_TYPE/DATE_RECORDED: {The time that the recording began.=
> This is akin to the TDRC tag in ID3.}
> ORIGINAL_MEDIA_TYPE/condition: {Comments on the condition of the sourc= e,
> any preparation for playback of the source tape (if relevant).}
> ORIGINAL_MEDIA_TYPE/initial_source_timecode: 01:02:15;10
> ORIGINAL_MEDIA_TYPE/initial_source_timecode/settings: DropFrame=3DYes = /
> 24HourMax=3DNo / IsVisual=3DNo
> DATE_DIGITIZED : {Date and time of the file creation process. Enter in=
> ISO 8601 format.}
> ENCODED_BY : {Name of the technician or organization responsible
> for the encoding and file creation process}
> ENCODED_BY/URL : {URL for the technician or organization listed in
> ENCODED_BY.}
> ENCODED_BY/processing_actions: {Description of any actions performed > during processing, such as trimming silence.}
> ENCODED_BY/capture_software: Blackmagic Media Express
> ENCODED_BY/capture_software/version: 2.0.3
> ENCODED_BY/capture_operating_system: Ubuntu
> ENCODED_BY/capture_operating_system/version: 10.10
> ENCODED_BY/capture_device: Decklink Studio SDI
> ENCODED_BY/capture_device/manufacturer: Blackmagic-Design
> ENCODED_BY/capture_device/serial_no: xyz6789
> ENCODED_BY/capture_device/settings: {notes on adjustments or settings}=
> ENCODED_BY/playback_device: SVO-5800
> ENCODED_BY/playback_device/manufacturer: Sony
> ENCODED_BY/playback_device/serial_no: abc1234
> ENCODED_BY/playback_device/settings: {notes on adjustments or settings= }
> ENCODED_BY/playback_device/playback_signal_path: {Protocols used to > transfer audiovisual data between the playback deck and the capture de= vice.
> example: Component}
> MAJOR_BRAND : qt
> MINOR_VERSION : 537199360
> ENCODER : Lavf53.2.0
>
> Official tags are in ALL CAPS and my own local tags are in lowercase.<= /p>

I think that looks great! Were you local tags based on a con= trolled vocabulary?

>
> I propose a discussion on:
>
> - Is defining process metadata (aka digitization metadata, coding hist= ory)
> worthwhile to include in CELLAR work?
>

Just to be 100% clear, do you mean defining process metadata= for insertion into a Matroska container, or a general vocabulary/schema? P= erhaps the latter is out of the scope of CELLAR, but selfishly I'd like= it included:)) I'm very interested in this topic and think that it is = essential metadata that should be captured. I'm on the fence with regar= ds to embedding the metadata versus storing it in a sidecar file or databas= e.

> - Should such metadata be added to Matroska tags or sho= uld we use existing
> metadata standards as attachments?
>

I'm in favour of increasing the number of Matroska's= official tags. Would using existing metadata standards as attachments requ= ire any extra work or discussion as attaching metadata xml files could prob= ably be done right now? I took a look at the Matroska tags recently and the= y do not map well to process history for digitisation. As Matroska is incre= asingly used as a format for digitisation and preservation, I think it woul= d be very useful if the tag vocabulary could support this kind of metadata.=

> - Or (building off the prior question) should we add na= mespaces to Matroska
> metadata tags so that externally defined standards may be used within<= br> > Matroska metadata directly?

This is interesting, but can you give an example? Would a ta= g look something like PREMIS:AgentType=3DSoftware? I wonder if this is actu= ally the best of both worlds, as it allows reference to other standards thr= ough embedding rather than attachments. I'm not sure how this would be = any different to your example of using a customised local vocabulary, unles= s there was some kind of reference to these standards built into the Matros= ka spec, which could even be validated?

>
> - And .... what metadata values are needed?

I think that this is a great discussion topic that would hav= e benefit beyond the scope of Matroska. PREMIS is ultimately quite vague in= terms of what is actually recorded. It would be very helpful if archives s= hared what kind of information they recorded pre/during/post capture. I'= ;m sure that we would be happy to share the outcomes of what we record in P= REMIS for our film and tape workflows, once we actually get that far.

Best,

Kieran.

--001a1148ef26c21c55053bf4755e-- From nobody Thu Sep 8 08:37:09 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B54B512B4D3 for ; Thu, 8 Sep 2016 08:37:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.43 X-Spam-Level: X-Spam-Status: No, score=-2.43 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, HTML_OBFUSCATE_05_10=0.26, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_REMOTE_IMAGE=0.01] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=viaa.be Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bzklb_v1dy66 for ; Thu, 8 Sep 2016 08:37:04 -0700 (PDT) Received: from mail-vk0-x22e.google.com (mail-vk0-x22e.google.com [IPv6:2607:f8b0:400c:c05::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4F08112B4F8 for ; Thu, 8 Sep 2016 08:33:37 -0700 (PDT) Received: by mail-vk0-x22e.google.com with SMTP id v189so44263096vkv.1 for ; Thu, 08 Sep 2016 08:33:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=viaa.be; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=bM3gmRs8o5NctJDYpgkfJQID/EFbZgYqjqsI3uqRWHQ=; b=U+qFyrZtvdNco1A4Id1VKg1JwqFXjUvGZxtws0wG/gIWg6xTHnZshk08F+IwmY9XLv NyRUkYT0HOMOQcHXivQAz1NrKIjDsNBLYpkz9Cl7B13ho6P+3KEkYLd6lBYeeEeB5FAZ CeJYRatCjD35RvVB8nLbgr3v3Bm3Mj3rzI9asnROC1QvYNuQSkmLoaJGUPiJbsTEX7Q4 xRJkKm23RMLX3m2zTwEYJRC63soF36nwIIW50z+10kkZGVCN4gdmWiJiPYdwjPip6omV 5ztcILDrOEykIaUO25Sj884x+2IpMbvW9n0WhNL18+RDR+wXaBehy84A7DaNVJMyQWv4 Uixg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=bM3gmRs8o5NctJDYpgkfJQID/EFbZgYqjqsI3uqRWHQ=; b=Xt+BKj5WoObPlXzbj9eRe71YZyun4sgsiiXor9hsVo3xojopOhKUdScwxkxXVTq5sk iKuR2fmmIpo5Xu5rq1jchSSoNoY0U9+0TS6OpLR8AloMNdEugLLpND/om/L9qflfCTTn ojX6EiVm8wQvqEijafF3RliaRPJmhsFHU5luFdC3ttBkOpjtxjx2p3Q3KM1H/tLi4MHo OTNJQ8T+smQJdwAFYkLdhQAh3D38WiqfEoqtWW45ydH2RVurD/OVY7uMPWPtgR3F72tW ghnY2G4zK/BlG3AAZ3hMtLYn8sPiTs9SvV07/sOJOq7+lmMnrhift0bFJ/DQEksymv2L I8ZA== X-Gm-Message-State: AE9vXwPEFdpejGqzkWTUebCYWtShsQr9uLhY/CR5bJ70Vni80+NKVe7VNHx/mZ+1szHVc6nmQ3/cT/qmbanaotXi X-Received: by 10.31.14.133 with SMTP id 127mr206594vko.50.1473348816175; Thu, 08 Sep 2016 08:33:36 -0700 (PDT) MIME-Version: 1.0 Received: by 10.103.36.133 with HTTP; Thu, 8 Sep 2016 08:33:35 -0700 (PDT) In-Reply-To: References: <900FE24A-0670-4A99-9CE0-300ED91103DB@dericed.com> From: Brecht Declercq Date: Thu, 8 Sep 2016 17:33:35 +0200 Message-ID: To: Kieran O Leary Content-Type: multipart/alternative; boundary=001a114422aed1b07f053c00c082 Archived-At: Cc: cellar@ietf.org Subject: Re: [Cellar] Matroska metadata for digitization and process history X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Sep 2016 15:37:08 -0000 --001a114422aed1b07f053c00c082 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Dear all, My first contribution to this discussion group! In principle I endorse Kieran stating that PREMIS-based info on the digitization (preservation in general) history of the essence may as well (even preferably?) be stored in a side-car file. The logic is that metadata in general, and preservation metadata in particular are by nature more evolutive than the essence. Altering the whole file when only the metadata should be altered might cause an unnecessary workload for the archive's system. I'd also like to point to the work done on this issue for the MP-AF under MPEG-A, by a working group lead by Walter Allasia (however, I don't know what the current status of this work is): http://david-preservation.eu/wp-content/uploads/2014/10/ipres2014_mpaf_cr_v= 4.pdf In the FIAT/IFTA MMC we have contributed to this work by pointing to the fact that many (broadcasting?) archives don't stick to a 1-to-1 relation between the original physical carrier and the new digital master file. This is because they consider the limits of the physical carrier format largely (but not completely, cfr. e.g. the max size of 4GB size of a .wav) irrelevant in the digital world. Basically there are 3 situations possible: 1 carrier =3D multiple files (we've always called this "div") 1 carrier =3D 1 file ("equal") multiple carriers =3D 1 file ("append") It would be interesting if this could be documented as preservation metadata too... Best, 2016-09-08 2:53 GMT+02:00 Kieran O Leary : > Hi, > > I'm very happy that this topic has come up for discussion. > > On Wed, Sep 7, 2016 at 8:09 PM, Dave Rice wrote: > > > > - A set of Matroska files are created using various digitization > processes > > and hardware and those processes should be document (example: 1.mkv > created > > from the output of a film scanner, 2.mkv from running a VHS through > > component video to a blackmagic card and 3.mkv from running a Betamax > > through cosposite video to an Elgato capture card). In some cases the > data > > could be collected as introduced into the mkv file as it is written and > in > > other cases it could be added after via mkvpropedit. > > > > Inserted as it is written sounds very interesting, as the capture file > does not end up with an altered whole file checksum, which is what would > happen with post capture metadata insertion. I'm not sure of how > mkvpropedit works, and I wonder if a framemd5 check of the streams would = be > paranoid or reasonable after such an update? Perhaps the embedding of > stream level checksums is a good idea at this stage? I saw that Andrew > Weaver was experimenting with this on vrecord. > > > - Another case, an mkv is created from a film scanner, then another mkv > is > > created from the first to make some correct corrections, and then a thi= rd > > mkv is created as an access copy with a lossy compressed codec. The > process > > data of each mkv file could be added to with each generational move, so > that > > with the third file the history of its creation could be clear. > > > > This is pretty much an exact scenario that the Irish Film Archive is > dealing with right now. We're investigating PREMIS as a means of recordin= g > this information and linking all the different assets to each other via > UUIDs. Does/could mkvpropedit generate UUIDs? It would be interesting if = it > did, as the identifier could then be harvested and also inserted into a > PREMIS xml description afterwards.. Digital Cinema Packages use UUIDs in > their xml files (also embedded in the mxfs) in order to create > relationships between single/multi reel audio, visual and subtitle files. > Perhaps they could also help create this provenance description among thi= s > chain of film assets as well? I don't think that there would be an > automated way to make the link between assets though. > > > - One mkv file is used to create another at a different set of > > characteristics, such data could communicate that the data used to be i= n > a > > different frame rate, interlacement pattern, or colorspace to give some > > context to why it looks the way it does. > > > > I think that this is a very valid use case, especially for access copies. > I can see the potential for something like vimeo or other streaming > platforms being able to possibly harvest and display this metadata if the= y > bought into this idea? > > > There is some overlap with this concept and the PREMIS metadata standar= d > > which documents an XML document that defines and relates concepts such = as > > Agents (hardware, software, people), Events (digitization, transcoding, > > processing), and Objects (source mkv files, source videotapes, source > film, > > etc). It could be feasible to simply determine a method of documenting > this > > data in PREMIS XML and then store that as an attachment, though I think > it > > makes it more accessible as a set of Matroska metadata tags. > > > > I'm not very familiar with Matroska attachments. I think this is a nice > idea for distributing files publicly, in that a user could gain additiona= l > context if required. For long term preservation, I can't help but think > that the PREMIS xml would function just as well in a sidecar in an AIP, > with the added benefit of not altering the checksum of the Matroska file. > > > I tried working on this subject a few years ago and created this sample > > which attaches a photo of the source tape and adds process metadata. > Here's > > an output of that metadata from FFmpeg. > > > > Metadata: > > CREATION_TIME : 2011-05-03 02:12:03 > > COMPATIBLE_BRANDS: qt > > ORIGINAL_MEDIA_TYPE: {Physical format of the source tape. Vocabulary?} > > ORIGINAL_MEDIA_TYPE/BARCODE: {Barcode or other identifier from the > > tape.} > > ORIGINAL_MEDIA_TYPE/BARCODE/source: {name of barcode authority or > > creator; example "XYZ Archive Barcode"} > > ORIGINAL_MEDIA_TYPE/LABEL: {The record label or imprint on the source > > media object.} > > ORIGINAL_MEDIA_TYPE/DATE_RECORDED: {The time that the recording began. > > This is akin to the TDRC tag in ID3.} > > ORIGINAL_MEDIA_TYPE/condition: {Comments on the condition of the source= , > > any preparation for playback of the source tape (if relevant).} > > ORIGINAL_MEDIA_TYPE/initial_source_timecode: 01:02:15;10 > > ORIGINAL_MEDIA_TYPE/initial_source_timecode/settings: DropFrame=3DYes / > > 24HourMax=3DNo / IsVisual=3DNo > > DATE_DIGITIZED : {Date and time of the file creation process. Enter in > > ISO 8601 format.} > > ENCODED_BY : {Name of the technician or organization responsible > > for the encoding and file creation process} > > ENCODED_BY/URL : {URL for the technician or organization listed in > > ENCODED_BY.} > > ENCODED_BY/processing_actions: {Description of any actions performed > > during processing, such as trimming silence.} > > ENCODED_BY/capture_software: Blackmagic Media Express > > ENCODED_BY/capture_software/version: 2.0.3 > > ENCODED_BY/capture_operating_system: Ubuntu > > ENCODED_BY/capture_operating_system/version: 10.10 > > ENCODED_BY/capture_device: Decklink Studio SDI > > ENCODED_BY/capture_device/manufacturer: Blackmagic-Design > > ENCODED_BY/capture_device/serial_no: xyz6789 > > ENCODED_BY/capture_device/settings: {notes on adjustments or settings} > > ENCODED_BY/playback_device: SVO-5800 > > ENCODED_BY/playback_device/manufacturer: Sony > > ENCODED_BY/playback_device/serial_no: abc1234 > > ENCODED_BY/playback_device/settings: {notes on adjustments or settings} > > ENCODED_BY/playback_device/playback_signal_path: {Protocols used to > > transfer audiovisual data between the playback deck and the capture > device. > > example: Component} > > MAJOR_BRAND : qt > > MINOR_VERSION : 537199360 > > ENCODER : Lavf53.2.0 > > > > Official tags are in ALL CAPS and my own local tags are in lowercase. > > I think that looks great! Were you local tags based on a controlled > vocabulary? > > > > > I propose a discussion on: > > > > - Is defining process metadata (aka digitization metadata, coding > history) > > worthwhile to include in CELLAR work? > > > > Just to be 100% clear, do you mean defining process metadata for insertio= n > into a Matroska container, or a general vocabulary/schema? Perhaps the > latter is out of the scope of CELLAR, but selfishly I'd like it included:= )) > I'm very interested in this topic and think that it is essential metadata > that should be captured. I'm on the fence with regards to embedding the > metadata versus storing it in a sidecar file or database. > > > - Should such metadata be added to Matroska tags or should we use > existing > > metadata standards as attachments? > > > > I'm in favour of increasing the number of Matroska's official tags. Would > using existing metadata standards as attachments require any extra work o= r > discussion as attaching metadata xml files could probably be done right > now? I took a look at the Matroska tags recently and they do not map well > to process history for digitisation. As Matroska is increasingly used as = a > format for digitisation and preservation, I think it would be very useful > if the tag vocabulary could support this kind of metadata. > > > - Or (building off the prior question) should we add namespaces to > Matroska > > metadata tags so that externally defined standards may be used within > > Matroska metadata directly? > > This is interesting, but can you give an example? Would a tag look > something like PREMIS:AgentType=3DSoftware? I wonder if this is actually = the > best of both worlds, as it allows reference to other standards through > embedding rather than attachments. I'm not sure how this would be any > different to your example of using a customised local vocabulary, unless > there was some kind of reference to these standards built into the Matros= ka > spec, which could even be validated? > > > > > - And .... what metadata values are needed? > > I think that this is a great discussion topic that would have benefit > beyond the scope of Matroska. PREMIS is ultimately quite vague in terms o= f > what is actually recorded. It would be very helpful if archives shared wh= at > kind of information they recorded pre/during/post capture. I'm sure that = we > would be happy to share the outcomes of what we record in PREMIS for our > film and tape workflows, once we actually get that far. > > Best, > > Kieran. > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > > --=20 Brecht Declercq Manager Digitalisering & Acquisitie VIAA vzw* | Sassevaartstraat 46/209 | 9000 Gent | Belgi=C3=AB | www.vi= aa.be * T: +32 9 298 05 01 *|* M: +32 474 25 04 67 --001a114422aed1b07f053c00c082 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Dear all,=C2=A0

My first contribution t= o this discussion group!
In principle I endorse Kieran stating th= at PREMIS-based info on the digitization (preservation in general) history = of the essence may as well (even preferably?) be stored in a side-car file.=
The logic is that metadata in general, and preservation metadata= in particular are by nature more evolutive than the essence. Altering the = whole file when only the metadata should be altered might cause an unnecess= ary workload for the archive's system.=C2=A0

I= 'd also like to point to the work done on this issue for the MP-AF unde= r MPEG-A, by a working group lead by Walter Allasia (however, I don't k= now what the current status of this work is):

In the FIAT/IFTA MMC we have contributed = to this work by pointing to the fact that many (broadcasting?) archives don= 't stick to a 1-to-1 relation between the original physical carrier and= the new digital master file. This is because they consider the limits of t= he physical carrier format largely (but not completely, cfr. e.g. the max s= ize of 4GB size of a .wav) irrelevant in the digital world.
Basic= ally there are 3 situations possible:

1 carrier = =3D multiple files (we've always called this "div")
1 carrier =3D 1 file ("equal")
multiple carriers =3D 1= file ("append")

It would be interesting= if this could be documented as preservation metadata too...

=
Best, =C2=A0=C2=A0



2016-09-08 2:53 GMT+02:00 Kieran = O Leary <kieran.o.leary@gmail.com>:

Hi,

I'm very happy that this topic has come up for discussio= n.

On Wed, Sep 7, 2016 at 8:09 PM, Dave Rice <dave@dericed.com> wrote:
>
> - A set of Matroska files are created using various digitization proce= sses
> and hardware and those processes should be document (example: 1.mkv cr= eated
> from the output of a film scanner, 2.mkv from running a VHS through > component video to a blackmagic card and 3.mkv from running a Betamax<= br> > through cosposite video to an Elgato capture card). In some cases the = data
> could be collected as introduced into the mkv file as it is written an= d in
> other cases it could be added after via mkvpropedit.
>

Inserted as it is written sounds very interesting, as the ca= pture file does not end up with an altered whole file checksum, which is wh= at would happen with post capture metadata insertion. I'm not sure of h= ow mkvpropedit works, and I wonder if a framemd5 check of the streams would= be paranoid or reasonable after such an update? Perhaps the embedding of s= tream level checksums is a good idea at this stage? I saw that Andrew Weave= r was experimenting with this on vrecord.

> - Another case, an mkv is created from a film scanner, = then another mkv is
> created from the first to make some correct corrections, and then a th= ird
> mkv is created as an access copy with a lossy compressed codec. The pr= ocess
> data of each mkv file could be added to with each generational move, s= o that
> with the third file the history of its creation could be clear.
>

This is pretty much an exact scenario that the Irish Film Ar= chive is dealing with right now. We're investigating PREMIS as a means = of recording this information and linking all the different assets to each = other via UUIDs. Does/could mkvpropedit generate UUIDs? It would be interes= ting if it did, as the identifier could then be harvested and also inserted= into a PREMIS xml description afterwards.. Digital Cinema Packages use UUI= Ds in their xml files (also embedded in the mxfs) in order to create relati= onships between single/multi reel audio, visual and subtitle files. Perhaps= they could also help create this provenance description among this chain o= f film assets as well? I don't think that there would be an automated w= ay to make the link between assets though.

> - One mkv file is used to create another at a different= set of
> characteristics, such data could communicate that the data used to be = in a
> different frame rate, interlacement pattern, or colorspace to give som= e
> context to why it looks the way it does.
>

I think that this is a very valid use case, especially for a= ccess copies. I can see the potential for something like vimeo or other str= eaming platforms being able to possibly harvest and display this metadata i= f they bought into this idea?

> There is some overlap with this concept and the PREMIS = metadata standard
> which documents an XML document that defines and relates concepts such= as
> Agents (hardware, software, people), Events (digitization, transcoding= ,
> processing), and Objects (source mkv files, source videotapes, source = film,
> etc). It could be feasible to simply determine a method of documenting= this
> data in PREMIS XML and then store that as an attachment, though I thin= k it
> makes it more accessible as a set of Matroska metadata tags.
>

I'm not very familiar with Matroska attachments. I think= this is a nice idea for distributing files publicly, in that a user could = gain additional context if required. For long term preservation, I can'= t help but think that the PREMIS xml would function just as well in a sidec= ar in an AIP, with the added benefit of not altering the checksum of the Ma= troska file.

> I tried working on this subject a few years ago and cre= ated this sample
> which attaches a photo of the source tape and adds process metadata. H= ere's
> an output of that metadata from FFmpeg.
>
> Metadata:
> CREATION_TIME : 2011-05-03 02:12:03
> COMPATIBLE_BRANDS: qt
> ORIGINAL_MEDIA_TYPE: {Physical format of the source tape. Vocabulary?}=
> ORIGINAL_MEDIA_TYPE/BARCODE: {Barcode or other identifier from the
> tape.}
> ORIGINAL_MEDIA_TYPE/BARCODE/source: {name of barcode authority or=
> creator; example "XYZ Archive Barcode"}
> ORIGINAL_MEDIA_TYPE/LABEL: {The record label or imprint on the source<= br> > media object.}
> ORIGINAL_MEDIA_TYPE/DATE_RECORDED: {The time that the recording b= egan.
> This is akin to the TDRC tag in ID3.}
> ORIGINAL_MEDIA_TYPE/condition: {Comments on the condition of the sourc= e,
> any preparation for playback of the source tape (if relevant).}
> ORIGINAL_MEDIA_TYPE/initial_source_timecode: 01:02:15;10
> ORIGINAL_MEDIA_TYPE/initial_source_timecode/settings: DropFrame= =3DYes /
> 24HourMax=3DNo / IsVisual=3DNo
> DATE_DIGITIZED : {Date and time of the file creation process. Enter in=
> ISO 8601 format.}
> ENCODED_BY : {Name of the technician or organization responsible
> for the encoding and file creation process}
> ENCODED_BY/URL : {URL for the technician or organization listed in
> ENCODED_BY.}
> ENCODED_BY/processing_actions: {Description of any actions performed > during processing, such as trimming silence.}
> ENCODED_BY/capture_software: Blackmagic Media Express
> ENCODED_BY/capture_software/version: 2.0.3
> ENCODED_BY/capture_operating_system: Ubuntu
> ENCODED_BY/capture_operating_system/version: 10.10
> ENCODED_BY/capture_device: Decklink Studio SDI
> ENCODED_BY/capture_device/manufacturer: Blackmagic-Design
> ENCODED_BY/capture_device/serial_no: xyz6789
> ENCODED_BY/capture_device/settings: {notes on adjustments or sett= ings}
> ENCODED_BY/playback_device: SVO-5800
> ENCODED_BY/playback_device/manufacturer: Sony
> ENCODED_BY/playback_device/serial_no: abc1234
> ENCODED_BY/playback_device/settings: {notes on adjustments or set= tings}
> ENCODED_BY/playback_device/playback_signal_path: {Protocols used = to
> transfer audiovisual data between the playback deck and the capture de= vice.
> example: Component}
> MAJOR_BRAND : qt
> MINOR_VERSION : 537199360
> ENCODER : Lavf53.2.0
>
> Official tags are in ALL CAPS and my own local tags are in lowercase.<= /p>

I think that looks great! Were you local tags based on a con= trolled vocabulary?

>
> I propose a discussion on:
>
> - Is defining process metadata (aka digitization metadata, coding hist= ory)
> worthwhile to include in CELLAR work?
>

Just to be 100% clear, do you mean defining process metadata= for insertion into a Matroska container, or a general vocabulary/schema? P= erhaps the latter is out of the scope of CELLAR, but selfishly I'd like= it included:)) I'm very interested in this topic and think that it is = essential metadata that should be captured. I'm on the fence with regar= ds to embedding the metadata versus storing it in a sidecar file or databas= e.

> - Should such metadata be added to Matroska tags or sho= uld we use existing
> metadata standards as attachments?
>

I'm in favour of increasing the number of Matroska's= official tags. Would using existing metadata standards as attachments requ= ire any extra work or discussion as attaching metadata xml files could prob= ably be done right now? I took a look at the Matroska tags recently and the= y do not map well to process history for digitisation. As Matroska is incre= asingly used as a format for digitisation and preservation, I think it woul= d be very useful if the tag vocabulary could support this kind of metadata.=

> - Or (building off the prior question) should we add na= mespaces to Matroska
> metadata tags so that externally defined standards may be used within<= br> > Matroska metadata directly?

This is interesting, but can you give an example? Would a ta= g look something like PREMIS:AgentType=3DSoftware? I wonder if this is actu= ally the best of both worlds, as it allows reference to other standards thr= ough embedding rather than attachments. I'm not sure how this would be = any different to your example of using a customised local vocabulary, unles= s there was some kind of reference to these standards built into the Matros= ka spec, which could even be validated?

>
> - And .... what metadata values are needed?

I think that this is a great discussion topic that would hav= e benefit beyond the scope of Matroska. PREMIS is ultimately quite vague in= terms of what is actually recorded. It would be very helpful if archives s= hared what kind of information they recorded pre/during/post capture. I'= ;m sure that we would be happy to share the outcomes of what we record in P= REMIS for our film and tape workflows, once we actually get that far.

Best,

Kieran.


_______________________________________________
Cellar mailing list
Cellar@ietf.org https://www.ietf.org/mailman/listinfo/cellar



--

=C2=A0 =C2=A0 =C2=A0Brecht Declercq
=C2=A0 =C2=A0 =C2=A0Manag= er Digitalisering & Acquisitie

=C2=A0 =C2=A0 =C2=A0VIAA vzw=C2=A0|= =C2=A0Sassevaartstraat 46/209=C2=A0| 9000 Gent | Belgi=C3=AB |=C2=A0= www.viaa.be
=C2=A0 =C2=A0 =C2=A0T: +32 9 298 05 01=C2=A0|=C2=A0M: +32 474 25 04 67=C2=A0




--001a114422aed1b07f053c00c082-- From nobody Thu Sep 8 09:32:50 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C0D4912B1E0 for ; Thu, 8 Sep 2016 09:32:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.12 X-Spam-Level: X-Spam-Status: No, score=-1.12 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nYJp9JzOKr2B for ; Thu, 8 Sep 2016 09:32:47 -0700 (PDT) Received: from s172.web-hosting.com (s172.web-hosting.com [68.65.122.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2E0BA12B1FE for ; Thu, 8 Sep 2016 09:32:44 -0700 (PDT) Received: from [146.96.19.240] (port=22341 helo=[10.10.202.53]) by server172.web-hosting.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.86_1) (envelope-from ) id 1bi2Fg-001RIh-1f; Thu, 08 Sep 2016 12:32:43 -0400 Content-Type: multipart/alternative; boundary="Apple-Mail=_385B9B98-D3DA-4053-8050-8C3CC3813171" Mime-Version: 1.0 (Mac OS X Mail 9.2 \(3112\)) From: Dave Rice In-Reply-To: Date: Thu, 8 Sep 2016 12:32:14 -0400 Message-Id: <4D92811C-EE62-437E-BC3C-55F0E0C9C87B@dericed.com> References: <900FE24A-0670-4A99-9CE0-300ED91103DB@dericed.com> To: Brecht Declercq X-Mailer: Apple Mail (2.3112) X-OutGoing-Spam-Status: No, score=-2.4 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Cc: cellar@ietf.org, Kieran O Leary Subject: Re: [Cellar] Matroska metadata for digitization and process history X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Sep 2016 16:32:50 -0000 --Apple-Mail=_385B9B98-D3DA-4053-8050-8C3CC3813171 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hi Brecht and Kieran, Replying to both posts inline. > On Sep 8, 2016, at 11:33 AM, Brecht Declercq = wrote: >=20 > Dear all,=20 >=20 > My first contribution to this discussion group! Welcome Brecht! > In principle I endorse Kieran stating that PREMIS-based info on the = digitization (preservation in general) history of the essence may as = well (even preferably?) be stored in a side-car file. There are reasons for both and I'm not sure it's in the scope of CELLAR = to defined which tactic (embedded or sidecar) is preferred in a given = situation, but to define (and/or consider defining) the option to store = preservation metadata. I can think of several AV formats where transfer metadata is embedded = when there is no standardized method to store the equivalent data in a = side car file. For example in DV transferred from tapes each DIF block = contains a STA value which documents the status (success or error) of = the playback device reading the data from tape. Similar a recovered = stream from DAT tape stores data on the verification of parity data in = tape reads when the DAT stream is moved from tape to file. Many containers generally stores some context about the creation of the = file such as encoding timestamp and in the case of Matroska it also = stores the WritingApp and MuxingApp. When archives or Matroska creators = need to manage further context about the creation and/or provenance of = that file we'd have to decide if that additional data should be stored = within the file (such as WritingApp and MuxingApp are) or stored in a = separate sidecar file. If using a sidecar file then we have a division = of some creation data being within Matroska and some without. ... Just noticed that the AS-07 project with MXF addresses this in their = specification. See in particular pages 90-91 of = http://www.amwa.tv/downloads/specifications/AS-07_Proposed_Application_Spe= cification.pdf where it defines fields for AS_07_Core_DMS_Devices to = document capture history. Page 91 defines metadata to document capture = devices such as device type, manufacturer, model, serial number, etc. > The logic is that metadata in general, and preservation metadata in = particular are by nature more evolutive than the essence. Altering the = whole file when only the metadata should be altered might cause an = unnecessary workload for the archive's system. I think Matroska's design well considers the scenario of such = alteration. The EBML specification recommends that all Top-Level = Elements have a CRC-32 Element. With this the fixity of the components = of the Matroska file are managed independently, so the metadata in the = Tags element may be edited without a compromise to the fixity of the AV = data in Clusters or other sections. In other containers without internal = fixity, a sidecar checksum must be used, and any metadata edit impacts = the fixity of the entire file. Additionally Matroska's main metadata = wranglers such as mkvpropedit do well to edit the file efficiently so = that metadata may be manipulated while impacting only a minimal amount = of the file. > I'd also like to point to the work done on this issue for the MP-AF = under MPEG-A, by a working group lead by Walter Allasia (however, I = don't know what the current status of this work is): > = http://david-preservation.eu/wp-content/uploads/2014/10/ipres2014_mpaf_cr_= v4.pdf = Thanks for the reference. > In the FIAT/IFTA MMC we have contributed to this work by pointing to = the fact that many (broadcasting?) archives don't stick to a 1-to-1 = relation between the original physical carrier and the new digital = master file. This is because they consider the limits of the physical = carrier format largely (but not completely, cfr. e.g. the max size of = 4GB size of a .wav) irrelevant in the digital world. > Basically there are 3 situations possible: >=20 > 1 carrier =3D multiple files (we've always called this "div") > 1 carrier =3D 1 file ("equal") > multiple carriers =3D 1 file ("append") >=20 > It would be interesting if this could be documented as preservation = metadata too... Matroska does have a linking mechacism via PrevUID and NextUID where = multiple Matroska files may be linked in order to emulate the linking of = multiple physical carriers. Additionally Matroska has Ordered Chapters = so that one Matroska file may represent one physical carrier which has = multiple presentations. I've been testing with Ordered Chapters and = Matroska so that we can digitize videotapes in their entirety with one = Edition for the presentation of the whole captured tape and subsequent = editions for each presentation within that content. [...] > 2016-09-08 2:53 GMT+02:00 Kieran O Leary >: > Hi, >=20 > I'm very happy that this topic has come up for discussion. >=20 > On Wed, Sep 7, 2016 at 8:09 PM, Dave Rice > wrote: > > > > - A set of Matroska files are created using various digitization = processes > > and hardware and those processes should be document (example: 1.mkv = created > > from the output of a film scanner, 2.mkv from running a VHS through > > component video to a blackmagic card and 3.mkv from running a = Betamax > > through cosposite video to an Elgato capture card). In some cases = the data > > could be collected as introduced into the mkv file as it is written = and in > > other cases it could be added after via mkvpropedit. > > >=20 > Inserted as it is written sounds very interesting, as the capture file = does not end up with an altered whole file checksum, which is what would = happen with post capture metadata insertion. I'm not sure of how = mkvpropedit works, and I wonder if a framemd5 check of the streams would = be paranoid or reasonable after such an update? >=20 Moritz may have some comments, but IIUC mkvpropedit wouldn't affect the = Clusters at all while editing the metadata. It would just edit the Tags = in place and adding a Void element if needed to cover space or extend = the Tags if at the end of the file, or Void the existing tags and append = a new larger Tags element to the end of the file. > Perhaps the embedding of stream level checksums is a good idea at this = stage? I saw that Andrew Weaver was experimenting with this on vrecord. >=20 Yes, this is a good use case. vrecord already manages variable to store = which connection is going into the Blackmagic card and other context = about the creation of files in video digitization. These are currently = written out to a sidecar file. > > - Another case, an mkv is created from a film scanner, then another = mkv is > > created from the first to make some correct corrections, and then a = third > > mkv is created as an access copy with a lossy compressed codec. The = process > > data of each mkv file could be added to with each generational move, = so that > > with the third file the history of its creation could be clear. > > >=20 > This is pretty much an exact scenario that the Irish Film Archive is = dealing with right now. We're investigating PREMIS as a means of = recording this information and linking all the different assets to each = other via UUIDs. Does/could mkvpropedit generate UUIDs? >=20 I think most Matroska muxers generate a UUID in the Info Element. Though = technically the UUID Element is not required so there many be muxers = that opt not too. IIRC webm muxers do not add the UUID Element. > It would be interesting if it did, as the identifier could then be = harvested and also inserted into a PREMIS xml description afterwards.. = Digital Cinema Packages use UUIDs in their xml files (also embedded in = the mxfs) in order to create relationships between single/multi reel = audio, visual and subtitle files. >=20 That sounds reasonable. The UUIDs are easy to harvest via mkvinfo, = ffmpeg, mediainfo, etc. > Perhaps they could also help create this provenance description among = this chain of film assets as well? I don't think that there would be an = automated way to make the link between assets though. >=20 > > - One mkv file is used to create another at a different set of > > characteristics, such data could communicate that the data used to = be in a > > different frame rate, interlacement pattern, or colorspace to give = some > > context to why it looks the way it does. > > >=20 > I think that this is a very valid use case, especially for access = copies. I can see the potential for something like vimeo or other = streaming platforms being able to possibly harvest and display this = metadata if they bought into this idea? >=20 > > There is some overlap with this concept and the PREMIS metadata = standard > > which documents an XML document that defines and relates concepts = such as > > Agents (hardware, software, people), Events (digitization, = transcoding, > > processing), and Objects (source mkv files, source videotapes, = source film, > > etc). It could be feasible to simply determine a method of = documenting this > > data in PREMIS XML and then store that as an attachment, though I = think it > > makes it more accessible as a set of Matroska metadata tags. > > >=20 > I'm not very familiar with Matroska attachments. I think this is a = nice idea for distributing files publicly, in that a user could gain = additional context if required. >=20 Most historical documentation of Matroska attachments specifically = documents Attachments for storage of cover art and publicity materials, = but any file could be stored and described in there. > For long term preservation, I can't help but think that the PREMIS xml = would function just as well in a sidecar in an AIP, with the added = benefit of not altering the checksum of the Matroska file. >=20 > > I tried working on this subject a few years ago and created this = sample > > which attaches a photo of the source tape and adds process metadata. = Here's > > an output of that metadata from FFmpeg. > > > > Metadata: > > CREATION_TIME : 2011-05-03 02:12:03 > > COMPATIBLE_BRANDS: qt=20 > > ORIGINAL_MEDIA_TYPE: {Physical format of the source tape. = Vocabulary?} > > ORIGINAL_MEDIA_TYPE/BARCODE: {Barcode or other identifier from the > > tape.} > > ORIGINAL_MEDIA_TYPE/BARCODE/source: {name of barcode authority or > > creator; example "XYZ Archive Barcode"} > > ORIGINAL_MEDIA_TYPE/LABEL: {The record label or imprint on the = source > > media object.} > > ORIGINAL_MEDIA_TYPE/DATE_RECORDED: {The time that the recording = began. > > This is akin to the TDRC tag in ID3.} > > ORIGINAL_MEDIA_TYPE/condition: {Comments on the condition of the = source, > > any preparation for playback of the source tape (if relevant).} > > ORIGINAL_MEDIA_TYPE/initial_source_timecode: 01:02:15;10 > > ORIGINAL_MEDIA_TYPE/initial_source_timecode/settings: DropFrame=3DYes = / > > 24HourMax=3DNo / IsVisual=3DNo > > DATE_DIGITIZED : {Date and time of the file creation process. Enter = in > > ISO 8601 format.} > > ENCODED_BY : {Name of the technician or organization responsible > > for the encoding and file creation process} > > ENCODED_BY/URL : {URL for the technician or organization listed in > > ENCODED_BY.} > > ENCODED_BY/processing_actions: {Description of any actions performed > > during processing, such as trimming silence.} > > ENCODED_BY/capture_software: Blackmagic Media Express > > ENCODED_BY/capture_software/version: 2.0.3 > > ENCODED_BY/capture_operating_system: Ubuntu > > ENCODED_BY/capture_operating_system/version: 10.10 > > ENCODED_BY/capture_device: Decklink Studio SDI > > ENCODED_BY/capture_device/manufacturer: Blackmagic-Design > > ENCODED_BY/capture_device/serial_no: xyz6789 > > ENCODED_BY/capture_device/settings: {notes on adjustments or = settings} > > ENCODED_BY/playback_device: SVO-5800 > > ENCODED_BY/playback_device/manufacturer: Sony > > ENCODED_BY/playback_device/serial_no: abc1234 > > ENCODED_BY/playback_device/settings: {notes on adjustments or = settings} > > ENCODED_BY/playback_device/playback_signal_path: {Protocols used to > > transfer audiovisual data between the playback deck and the capture = device. > > example: Component} > > MAJOR_BRAND : qt=20 > > MINOR_VERSION : 537199360 > > ENCODER : Lavf53.2.0 > > > > Official tags are in ALL CAPS and my own local tags are in = lowercase. >=20 > I think that looks great! Were you local tags based on a controlled = vocabulary? >=20 I don't think so, I was just making things up. > > I propose a discussion on: > > > > - Is defining process metadata (aka digitization metadata, coding = history) > > worthwhile to include in CELLAR work? > > >=20 > Just to be 100% clear, do you mean defining process metadata for = insertion into a Matroska container, or a general vocabulary/schema? >=20 Both (though the second one may just be an adoption). Currently = Matroska's metadata documentation primarily covers descriptive metadata = and a little technical metadata. I was considering whether this should = be expanded to cover digitization / process history / provenance = metadata as well, as discussed at the No Time to Wait symposium. > Perhaps the latter is out of the scope of CELLAR, but selfishly I'd = like it included:)) I'm very interested in this topic and think that it = is essential metadata that should be captured. I'm on the fence with = regards to embedding the metadata versus storing it in a sidecar file or = database. >=20 > > - Should such metadata be added to Matroska tags or should we use = existing > > metadata standards as attachments? > > >=20 > I'm in favour of increasing the number of Matroska's official tags. = Would using existing metadata standards as attachments require any extra = work or discussion as attaching metadata xml files could probably be = done right now? >=20 Yes. mkvpropedit cellar.mkv -add-attachment premis.xml > I took a look at the Matroska tags recently and they do not map well = to process history for digitisation. As Matroska is increasingly used as = a format for digitisation and preservation, I think it would be very = useful if the tag vocabulary could support this kind of metadata. >=20 I found the same in the sample I referenced above where = ORIGINAL_MEDIA_TYPE and ENCODED_BY are awkwardly used to cover = digitization context. > > - Or (building off the prior question) should we add namespaces to = Matroska > > metadata tags so that externally defined standards may be used = within > > Matroska metadata directly? >=20 > This is interesting, but can you give an example? Would a tag look = something like PREMIS:AgentType=3DSoftware? I wonder if this is actually = the best of both worlds, as it allows reference to other standards = through embedding rather than attachments. I'm not sure how this would = be any different to your example of using a customised local vocabulary, = unless there was some kind of reference to these standards built into = the Matroska spec, which could even be validated? >=20 Currently Matroska tags are either local-defined or official Matroska = tags which are defined, but there's not a way to use an official tag = from another standard. So it would something like the example below, = which is the XML representation of Matroska tags but adds a new element = for which references the authority of Tag Names for that = value and all nested SimpleTags below it. 50 event = http://www.loc.gov/standards/premis/premis.xsd eventIdentifier eventIdentifierType local eventIdentifierValue CELLAR.123.Event-4321 eventType digitization eventDateTime 2016-09-08T14:32:11-05:00 eventDetail umatic vo9850 -> dps575 -> ultrastudio express -> = vrecord > > > > - And .... what metadata values are needed? >=20 > I think that this is a great discussion topic that would have benefit = beyond the scope of Matroska. PREMIS is ultimately quite vague in terms = of what is actually recorded. >=20 I agree. It crams a lot of detail into a few vague fields. > It would be very helpful if archives shared what kind of information = they recorded pre/during/post capture. I'm sure that we would be happy = to share the outcomes of what we record in PREMIS for our film and tape = workflows, once we actually get that far. >=20 Thanks, Dave Rice --Apple-Mail=_385B9B98-D3DA-4053-8050-8C3CC3813171 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii Hi Brecht and Kieran,
Replying to both posts = inline.

On Sep 8, 2016, at 11:33 AM, Brecht Declercq = <brecht.declercq@viaa.be> wrote:

Dear all, 

My first contribution to this discussion = group!

Welcome Brecht!

In principle I endorse Kieran stating that PREMIS-based info = on the digitization (preservation in general) history of the essence may = as well (even preferably?) be stored in a side-car = file.

There = are reasons for both and I'm not sure it's in the scope of CELLAR to = defined which tactic (embedded or sidecar) is preferred in a given = situation, but to define (and/or consider defining) the option to store = preservation metadata.

I can think = of several AV formats where transfer metadata is embedded when there is = no standardized method to store the equivalent data in a side car file. = For example in DV transferred from tapes each DIF block contains a STA = value which documents the status (success or error) of the playback = device reading the data from tape. Similar a recovered stream from DAT = tape stores data on the verification of parity data in tape reads when = the DAT stream is moved from tape to file.

Many containers generally stores some context = about the creation of the file such as encoding timestamp and in the = case of Matroska it also stores the WritingApp and MuxingApp. When = archives or Matroska creators need to manage further context about the = creation and/or provenance of that file we'd have to decide if that = additional data should be stored within the file (such as WritingApp and = MuxingApp are) or stored in a separate sidecar file. If using a sidecar = file then we have a division of some creation data being within Matroska = and some without.

... Just noticed = that the AS-07 project with MXF addresses this in their specification. = See in particular pages 90-91 of http://www.amwa.tv/downloads/specifications/AS-07_Proposed_Appl= ication_Specification.pdf where it defines fields for = AS_07_Core_DMS_Devices to document capture history. Page 91 defines = metadata to document capture devices such as device type, manufacturer, = model, serial number, etc.

The= logic is that metadata in general, and preservation metadata in = particular are by nature more evolutive than the essence. Altering the = whole file when only the metadata should be altered might cause an = unnecessary workload for the archive's = system.

I = think Matroska's design well considers the scenario of such alteration. = The EBML specification recommends that all Top-Level Elements have a = CRC-32 Element. With this the fixity of the components of the Matroska = file are managed independently, so the metadata in the Tags element may = be edited without a compromise to the fixity of the AV data in Clusters = or other sections. In other containers without internal fixity, a = sidecar checksum must be used, and any metadata edit impacts the fixity = of the entire file. Additionally Matroska's main metadata wranglers such = as mkvpropedit do well to edit the file efficiently so that metadata may = be manipulated while impacting only a minimal amount of the = file.

I'd also like to = point to the work done on this issue for the MP-AF under MPEG-A, by a = working group lead by Walter Allasia (however, I don't know what the = current status of this work is):
http://david-preservation.eu/wp-content/uploads/2014/10/ipres20= 14_mpaf_cr_v4.pdf

Thanks for the reference.

In the FIAT/IFTA MMC we have = contributed to this work by pointing to the fact that many = (broadcasting?) archives don't stick to a 1-to-1 relation between the = original physical carrier and the new digital master file. This is = because they consider the limits of the physical carrier format largely = (but not completely, cfr. e.g. the max size of 4GB size of a .wav) = irrelevant in the digital world.
Basically there = are 3 situations possible:

1 carrier =3D multiple files (we've always called this = "div")
1 carrier =3D 1 file ("equal")
multiple carriers =3D 1 file ("append")

It would be interesting = if this could be documented as preservation metadata = too...

Matroska does have a linking mechacism via PrevUID = and NextUID where multiple Matroska files may be linked in order to = emulate the linking of multiple physical carriers. Additionally Matroska = has Ordered Chapters so that one Matroska file may represent one = physical carrier which has multiple presentations. I've been testing = with Ordered Chapters and Matroska so that we can digitize videotapes in = their entirety with one Edition for the presentation of the whole = captured tape and subsequent editions for each presentation within that = content.

[...]

2016-09-08 2:53 GMT+02:00 Kieran O Leary <kieran.o.leary@gmail.com>:

Hi,

I'm very happy = that this topic has come up for discussion.

On Wed, Sep 7, 2016 at 8:09 PM, Dave Rice <dave@dericed.com> wrote:
>
> - A set of Matroska files are created using various digitization = processes
> and hardware and those processes should be document (example: 1.mkv = created
> from the output of a film scanner, 2.mkv from running a VHS = through
> component video to a blackmagic card and 3.mkv from running a = Betamax
> through cosposite video to an Elgato capture card). In some cases = the data
> could be collected as introduced into the mkv file as it is written = and in
> other cases it could be added after via mkvpropedit.
>

Inserted as it is written sounds very = interesting, as the capture file does not end up with an altered whole = file checksum, which is what would happen with post capture metadata = insertion. I'm not sure of how mkvpropedit works, and I wonder if a = framemd5 check of the streams would be paranoid or reasonable after such = an = update?

Mo= ritz may have some comments, but IIUC mkvpropedit wouldn't affect the = Clusters at all while editing the metadata. It would just edit the Tags = in place and adding a Void element if needed to cover space or extend = the Tags if at the end of the file, or Void the existing tags and append = a new larger Tags element to the end of the file.

Perhaps = the embedding of stream level checksums is a good idea at this stage? I = saw that Andrew Weaver was experimenting with this on = vrecord.

Y= es, this is a good use case. vrecord already manages variable to store = which connection is going into the Blackmagic card and other context = about the creation of files in video digitization. These are currently = written out to a sidecar file.

> - = Another case, an mkv is created from a film scanner, then another mkv = is
> created from the first to make some correct corrections, and then a = third
> mkv is created as an access copy with a lossy compressed codec. The = process
> data of each mkv file could be added to with each generational = move, so that
> with the third file the history of its creation could be clear.
>

This is pretty much an exact scenario = that the Irish Film Archive is dealing with right now. We're = investigating PREMIS as a means of recording this information and = linking all the different assets to each other via UUIDs. Does/could = mkvpropedit generate = UUIDs?

I = think most Matroska muxers generate a UUID in the Info Element. Though = technically the UUID Element is not required so there many be muxers = that opt not too. IIRC webm muxers do not add the UUID = Element.

It would be = interesting if it did, as the identifier could then be harvested and = also inserted into a PREMIS xml description afterwards.. Digital Cinema = Packages use UUIDs in their xml files (also embedded in the mxfs) in = order to create relationships between single/multi reel audio, visual = and subtitle = files.

Tha= t sounds reasonable. The UUIDs are easy to harvest via mkvinfo, ffmpeg, = mediainfo, etc.

Perhaps= they could also help create this provenance description among this = chain of film assets as well? I don't think that there would be an = automated way to make the link between assets though.

> - One mkv file is used to create another at a different = set of
> characteristics, such data could communicate that the data used to = be in a
> different frame rate, interlacement pattern, or colorspace to give = some
> context to why it looks the way it does.
>

I think that this is a very valid use = case, especially for access copies. I can see the potential for = something like vimeo or other streaming platforms being able to possibly = harvest and display this metadata if they bought into this idea?

> There is some overlap with this concept and = the PREMIS metadata standard
> which documents an XML document that defines and relates concepts = such as
> Agents (hardware, software, people), Events (digitization, = transcoding,
> processing), and Objects (source mkv files, source videotapes, = source film,
> etc). It could be feasible to simply determine a method of = documenting this
> data in PREMIS XML and then store that as an attachment, though I = think it
> makes it more accessible as a set of Matroska metadata tags.
>

I'm not very familiar with Matroska = attachments. I think this is a nice idea for distributing files = publicly, in that a user could gain additional context if = required.

= Most historical documentation of Matroska attachments specifically = documents Attachments for storage of cover art and publicity materials, = but any file could be stored and described in there.

For long = term preservation, I can't help but think that the PREMIS xml would = function just as well in a sidecar in an AIP, with the added benefit of = not altering the checksum of the Matroska file.

> I tried working on this subject a few years ago and = created this sample
> which attaches a photo of the source tape and adds process = metadata. Here's
> an output of that metadata from FFmpeg.
>
> Metadata:
> CREATION_TIME : 2011-05-03 02:12:03
> COMPATIBLE_BRANDS: qt
> ORIGINAL_MEDIA_TYPE: {Physical format of the source tape. = Vocabulary?}
> ORIGINAL_MEDIA_TYPE/BARCODE: {Barcode or other identifier from = the
> tape.}
> ORIGINAL_MEDIA_TYPE/BARCODE/source: {name of = barcode authority or
> creator; example "XYZ Archive Barcode"}
> ORIGINAL_MEDIA_TYPE/LABEL: {The record label or imprint on the = source
> media object.}
> ORIGINAL_MEDIA_TYPE/DATE_RECORDED: {The time that = the recording began.
> This is akin to the TDRC tag in ID3.}
> ORIGINAL_MEDIA_TYPE/condition: {Comments on the condition of the = source,
> any preparation for playback of the source tape (if relevant).}
> ORIGINAL_MEDIA_TYPE/initial_source_timecode: = 01:02:15;10
> ORIGINAL_MEDIA_TYPE/initial_source_timecode/settings:= DropFrame=3DYes /
> 24HourMax=3DNo / IsVisual=3DNo
> DATE_DIGITIZED : {Date and time of the file creation process. Enter = in
> ISO 8601 format.}
> ENCODED_BY : {Name of the technician or organization responsible
> for the encoding and file creation process}
> ENCODED_BY/URL : {URL for the technician or organization listed = in
> ENCODED_BY.}
> ENCODED_BY/processing_actions: {Description of any actions = performed
> during processing, such as trimming silence.}
> ENCODED_BY/capture_software: Blackmagic Media Express
> ENCODED_BY/capture_software/version: 2.0.3
> ENCODED_BY/capture_operating_system: Ubuntu
> ENCODED_BY/capture_operating_system/version: = 10.10
> ENCODED_BY/capture_device: Decklink Studio SDI
> ENCODED_BY/capture_device/manufacturer: = Blackmagic-Design
> ENCODED_BY/capture_device/serial_no: xyz6789
> ENCODED_BY/capture_device/settings: {notes on = adjustments or settings}
> ENCODED_BY/playback_device: SVO-5800
> ENCODED_BY/playback_device/manufacturer: Sony
> ENCODED_BY/playback_device/serial_no: abc1234
> ENCODED_BY/playback_device/settings: {notes on = adjustments or settings}
> ENCODED_BY/playback_device/playback_signal_path: = {Protocols used to
> transfer audiovisual data between the playback deck and the capture = device.
> example: Component}
> MAJOR_BRAND : qt
> MINOR_VERSION : 537199360
> ENCODER : Lavf53.2.0
>
> Official tags are in ALL CAPS and my own local tags are in = lowercase.

I think that looks great! Were = you local tags based on a controlled = vocabulary?

I don't think so, I was just making things up.

> I propose a discussion on:
>
> - Is defining process metadata (aka digitization metadata, coding = history)
> worthwhile to include in CELLAR work?
>

Just to be 100% clear, do you mean = defining process metadata for insertion into a Matroska container, or a = general = vocabulary/schema?

Both (though the second one may just be an adoption). Currently = Matroska's metadata documentation primarily covers descriptive metadata = and a little technical metadata. I was considering whether this should = be expanded to cover digitization / process history / provenance = metadata as well, as discussed at the No Time to Wait = symposium.

Perhaps the = latter is out of the scope of CELLAR, but selfishly I'd like it = included:)) I'm very interested in this topic and think that it is = essential metadata that should be captured. I'm on the fence with = regards to embedding the metadata versus storing it in a sidecar file or = database.

> - Should such metadata be = added to Matroska tags or should we use existing
> metadata standards as attachments?
>

I'm in favour of increasing the number = of Matroska's official tags. Would using existing metadata standards as = attachments require any extra work or discussion as attaching metadata = xml files could probably be done right = now?

Yes.<= /div>
mkvpropedit cellar.mkv -add-attachment = premis.xml

I took a = look at the Matroska tags recently and they do not map well to process = history for digitisation. As Matroska is increasingly used as a format = for digitisation and preservation, I think it would be very useful if = the tag vocabulary could support this kind of metadata. =

I found = the same in the sample I referenced above where ORIGINAL_MEDIA_TYPE = and ENCODED_BY are awkwardly used to cover digitization = context.

> - Or = (building off the prior question) should we add namespaces to = Matroska
> metadata tags so that externally defined standards may be used = within
> Matroska metadata directly?

This is = interesting, but can you give an example? Would a tag look something = like PREMIS:AgentType=3DSoftware? I wonder if this is actually the best = of both worlds, as it allows reference to other standards through = embedding rather than attachments. I'm not sure how this would be any = different to your example of using a customised local vocabulary, unless = there was some kind of reference to these standards built into the = Matroska spec, which could even be = validated?

Currently Matroska tags are either local-defined or official Matroska = tags which are defined, but there's not a way to use an official tag = from another standard. So it would something like the example below, = which is the XML representation of Matroska tags but adds a new element = for <NameSpace> which references the authority of Tag Names for = that value and all nested SimpleTags below it.

<?xml = version=3D"1.0"?>
<Tags>
  = <Tag>
    <Targets>
  =     = <TargetTypeValue>50</TargetTypeValue>
  =   </Targets>
    = <Simple>
      = <Name>event</Name>
      = <NameSpace>http://www.loc.gov/standards/premis/premis.xsd</NameSpac= e >
      = <Simple>
        = <Name>eventIdentifier</Name>
      =   <Simple>
          = <Name>eventIdentifierType</Name>
    =       <String>local</String>
  =       </Simple>
      =   <Simple>
          = <Name>eventIdentifierValue</Name>
    =       = <String>CELLAR.123.Event-4321</String>
  =       </Simple>
      = </Simple>
      = <Simple>
        = <Name>eventType</Name>
        = <String>digitization</String>
      = </Simple>
      = <Simple>
        = <Name>eventDateTime</Name>
      =   = <String>2016-09-08T14:32:11-05:00</String>
  =     </Simple>
      = <Simple>
        = <Name>eventDetail</Name>
      =   <String>umatic vo9850 -> dps575 -> ultrastudio = express -> vrecord</String>
      = </Simple>
    </Simple>
  = </Tag>
</Tags>

>
> - And .... what metadata values are needed?

I think that this is a great discussion topic that would have = benefit beyond the scope of Matroska. PREMIS is ultimately quite vague = in terms of what is actually = recorded.

= I agree. It crams a lot of detail into a few vague = fields.

It would be = very helpful if archives shared what kind of information they recorded = pre/during/post capture. I'm sure that we would be happy to share the = outcomes of what we record in PREMIS for our film and tape workflows, = once we actually get that = far.

Thanks,
Dave Rice

= --Apple-Mail=_385B9B98-D3DA-4053-8050-8C3CC3813171-- From nobody Thu Sep 8 09:58:18 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2158412B2AD for ; Thu, 8 Sep 2016 09:58:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.339 X-Spam-Level: X-Spam-Status: No, score=-2.339 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, HTML_OBFUSCATE_05_10=0.26, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=avpreserve-com.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GqxLxg0UOoeT for ; Thu, 8 Sep 2016 09:58:13 -0700 (PDT) Received: from mail-wm0-x230.google.com (mail-wm0-x230.google.com [IPv6:2a00:1450:400c:c09::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5239112B305 for ; Thu, 8 Sep 2016 09:57:44 -0700 (PDT) Received: by mail-wm0-x230.google.com with SMTP id w12so100748085wmf.0 for ; Thu, 08 Sep 2016 09:57:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=avpreserve-com.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=/gS/yfw10/efk5nZwB+JEG+yVrMen7SfpYAREjpFlzQ=; b=tVSFTcLa5VZDfYWmnGkgl93wFjecmiB7bsxBD7YPsxEDMxe6k2HBTu8lLxBg2EynZB 4/NvCVbr/1OeuqBbiwjGwy0Z+6j9z9agqngiTB8suKq8Yrp6uXT7VRuBNlaRd8fX3iq/ nWOKy745Bo37Lh9SxY1u7htZxwC8ejXmulnEWau58JK0Zeuf2MVvxzw3kK5b4phnCho+ FtMyVEpUXuE0KhjSN8cAcfenHTwc1VrWHQQE/jlmbet7kfXcAzepn2d9Q1AO3yYBy/LS uaX0SO/UvLuWCNW4DaqZTpFn+BlAMnWCL7LWzPR5dhfrc+z7Fh7QzDAHpwJz5ZzvPNGs 93TQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=/gS/yfw10/efk5nZwB+JEG+yVrMen7SfpYAREjpFlzQ=; b=cTPox2Id5rBEMhKjLjvAMztSPvWGtJGfTNEGple9zUA9VGdgWiwTNyma3xibkppnqE O15yUry4OPYoZCkld5m8LA0U2Bzj+dTuaYZx3SwxR2tId/xnF19YyKQYHCA3Xze9Ez8/ NQnGTKzjk83qNEW1Gaxqz4+z3aEex92podPGiwVB6T8ePaMPMNvxmJDX1blomw7x1xzf 3dlwZHTY7n+tdQO2x5PyIvpFNziTBr78ur2Xg6JE1IJ53+q5ioNBIkcIfI9altjJh3Bp ubTiGu9Ov/FXor1tXIeAkkZBm64fMuThmslLMLX0Ia8GEJv8mnY4r29YiupsdSnlB+nt bN9Q== X-Gm-Message-State: AE9vXwNNHHRJyLcwDYIrhtSonSC9U/kqvf+arYXOnHVBzgJ25DPp1gihoP+WxEQMvcMep/gyAG0xothaL4ixouFn X-Received: by 10.28.151.72 with SMTP id z69mr630485wmd.61.1473353862156; Thu, 08 Sep 2016 09:57:42 -0700 (PDT) MIME-Version: 1.0 Received: by 10.28.216.206 with HTTP; Thu, 8 Sep 2016 09:57:41 -0700 (PDT) From: Bertram Lyons Date: Thu, 8 Sep 2016 11:57:41 -0500 Message-ID: To: cellar@ietf.org Content-Type: multipart/alternative; boundary=001a11452c10954179053c01ed45 Archived-At: Subject: Re: [Cellar] Matroska metadata for digitization and process history X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Sep 2016 16:58:16 -0000 --001a11452c10954179053c01ed45 Content-Type: text/plain; charset=UTF-8 Hi all, one question inline below. > ---------- Forwarded message ---------- > From: Dave Rice > To: Brecht Declercq > Cc: cellar@ietf.org, Kieran O Leary > Date: Thu, 8 Sep 2016 12:32:14 -0400 > Subject: Re: [Cellar] Matroska metadata for digitization and process > history > Hi Brecht and Kieran, > Replying to both posts inline. > > 2016-09-08 2:53 GMT+02:00 Kieran O Leary : > >> >> For long term preservation, I can't help but think that the PREMIS xml >> would function just as well in a sidecar in an AIP, with the added benefit >> of not altering the checksum of the Matroska file. >> >> > I tried working on this subject a few years ago and created this sample >> > which attaches a photo of the source tape and adds process metadata. >> Here's >> > an output of that metadata from FFmpeg. >> > >> > Metadata: >> > CREATION_TIME : 2011-05-03 02:12:03 >> > COMPATIBLE_BRANDS: qt >> > ORIGINAL_MEDIA_TYPE: {Physical format of the source tape. Vocabulary?} >> > ORIGINAL_MEDIA_TYPE/BARCODE: {Barcode or other identifier from the >> > tape.} >> > ORIGINAL_MEDIA_TYPE/BARCODE/source: {name of barcode authority or >> > creator; example "XYZ Archive Barcode"} >> > ORIGINAL_MEDIA_TYPE/LABEL: {The record label or imprint on the source >> > media object.} >> > ORIGINAL_MEDIA_TYPE/DATE_RECORDED: {The time that the recording began. >> > This is akin to the TDRC tag in ID3.} >> > ORIGINAL_MEDIA_TYPE/condition: {Comments on the condition of the source, >> > any preparation for playback of the source tape (if relevant).} >> > ORIGINAL_MEDIA_TYPE/initial_source_timecode: 01:02:15;10 >> > ORIGINAL_MEDIA_TYPE/initial_source_timecode/settings: DropFrame=Yes / >> > 24HourMax=No / IsVisual=No >> > DATE_DIGITIZED : {Date and time of the file creation process. Enter in >> > ISO 8601 format.} >> > ENCODED_BY : {Name of the technician or organization responsible >> > for the encoding and file creation process} >> > ENCODED_BY/URL : {URL for the technician or organization listed in >> > ENCODED_BY.} >> > ENCODED_BY/processing_actions: {Description of any actions performed >> > during processing, such as trimming silence.} >> > ENCODED_BY/capture_software: Blackmagic Media Express >> > ENCODED_BY/capture_software/version: 2.0.3 >> > ENCODED_BY/capture_operating_system: Ubuntu >> > ENCODED_BY/capture_operating_system/version: 10.10 >> > ENCODED_BY/capture_device: Decklink Studio SDI >> > ENCODED_BY/capture_device/manufacturer: Blackmagic-Design >> > ENCODED_BY/capture_device/serial_no: xyz6789 >> > ENCODED_BY/capture_device/settings: {notes on adjustments or settings} >> > ENCODED_BY/playback_device: SVO-5800 >> > ENCODED_BY/playback_device/manufacturer: Sony >> > ENCODED_BY/playback_device/serial_no: abc1234 >> > ENCODED_BY/playback_device/settings: {notes on adjustments or settings} >> > ENCODED_BY/playback_device/playback_signal_path: {Protocols used to >> > transfer audiovisual data between the playback deck and the capture >> device. >> > example: Component} >> > MAJOR_BRAND : qt >> > MINOR_VERSION : 537199360 >> > ENCODER : Lavf53.2.0 >> > >> > Official tags are in ALL CAPS and my own local tags are in lowercase. >> >> I think that looks great! Were you local tags based on a controlled >> vocabulary? >> > I don't think so, I was just making things up. > I apologize if this has already been covered, but has there been discussion of including reVTMD as a starting point for thinking about process metadata elements that may be of interest? https://www.archives.gov/preservation/products/reVTMD.xsd Additionally, the UTVideoSrc Schema (hosted by Harvard) focuses on the source of video digitization processes: http://hul.harvard.edu/ois/xml/xsd/drs/UTVIDEOSrc-2-0.xsd Perhaps these are useful to add to the conversation about source and process metadata in Matroska if not already in the mix. Best regards -- Bert --001a11452c10954179053c01ed45 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi all, one question inline below.
=C2=A0
---------- Forwarded message ----------From:=C2=A0Dave Rice <dave@dericed.= com>
To:=C2=A0Brecht Declercq <brecht.declercq@viaa.be>
Cc:=C2=A0cellar@ietf.org, Kieran O Leary <kieran.o.leary@gmail.com>
Date:=C2=A0Thu= , 8 Sep 2016 12:32:14 -0400
Subject:=C2=A0Re: [Cellar] Matroska metadata= for digitization and process history
Hi Brecht and Kieran,
Replying to both posts inline.
<= br>
2016-09-08 2:53 GMT+02:00 Kieran O Leary <= kieran.o.leary@gmail.com>:


For long term preservation, I can't h= elp but think that the PREMIS xml would function just as well in a sidecar = in an AIP, with the added benefit of not altering the checksum of the Matro= ska file.

> I tried working on this subject a few year= s ago and created this sample
> which attaches a photo of the source tape and adds process metadata. H= ere's
> an output of that metadata from FFmpeg.
>
> Metadata:
> CREATION_TIME : 2011-05-03 02:12:03
> COMPATIBLE_BRANDS: qt
> ORIGINAL_MEDIA_TYPE: {Physical format of the source tape. Vocabulary?}=
> ORIGINAL_MEDIA_TYPE/BARCODE: {Barcode or other identifier from the
> tape.}
> ORIGINAL_MEDIA_TYPE/BARCODE/source: {name of barcode authority or=
> creator; example "XYZ Archive Barcode"}
> ORIGINAL_MEDIA_TYPE/LABEL: {The record label or imprint on the source<= br> > media object.}
> ORIGINAL_MEDIA_TYPE/DATE_RECORDED: {The time that the recording b= egan.
> This is akin to the TDRC tag in ID3.}
> ORIGINAL_MEDIA_TYPE/condition: {Comments on the condition of the sourc= e,
> any preparation for playback of the source tape (if relevant).}
> ORIGINAL_MEDIA_TYPE/initial_source_timecode: 01:02:15;10
> ORIGINAL_MEDIA_TYPE/initial_source_timecode/settings: DropFrame= =3DYes /
> 24HourMax=3DNo / IsVisual=3DNo
> DATE_DIGITIZED : {Date and time of the file creation process. Enter in=
> ISO 8601 format.}
> ENCODED_BY : {Name of the technician or organization responsible
> for the encoding and file creation process}
> ENCODED_BY/URL : {URL for the technician or organization listed in
> ENCODED_BY.}
> ENCODED_BY/processing_actions: {Description of any actions performed > during processing, such as trimming silence.}
> ENCODED_BY/capture_software: Blackmagic Media Express
> ENCODED_BY/capture_software/version: 2.0.3
> ENCODED_BY/capture_operating_system: Ubuntu
> ENCODED_BY/capture_operating_system/version: 10.10
> ENCODED_BY/capture_device: Decklink Studio SDI
> ENCODED_BY/capture_device/manufacturer: Blackmagic-Design
> ENCODED_BY/capture_device/serial_no: xyz6789
> ENCODED_BY/capture_device/settings: {notes on adjustments or sett= ings}
> ENCODED_BY/playback_device: SVO-5800
> ENCODED_BY/playback_device/manufacturer: Sony
> ENCODED_BY/playback_device/serial_no: abc1234
> ENCODED_BY/playback_device/settings: {notes on adjustments or set= tings}
> ENCODED_BY/playback_device/playback_signal_path: {Protocols used = to
> transfer audiovisual data between the playback deck and the capture de= vice.
> example: Component}
> MAJOR_BRAND : qt
> MINOR_VERSION : 537199360
> ENCODER : Lavf53.2.0
>
> Official tags are in ALL CAPS and my own local tags are in lowercase.<= /p>

I think that looks great! Were you local tags based on a = controlled vocabulary?

I don't think so, I was just making things up.
<= /div>

I apologize if this has already= been covered, but has there been discussion of including reVTMD as a start= ing point for thinking about process metadata elements that may be of inter= est?=C2=A0
=C2=A0

Additionally, the UTVideoSrc S= chema (hosted by Harvard) focuses on the source of video digitization proce= sses:=C2=A0


Perhaps these are useful to add= to the conversation about source and process metadata in Matroska if not a= lready in the mix.

Best regards --

<= /div>
Bert

--001a11452c10954179053c01ed45-- From nobody Fri Sep 9 01:02:57 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3081512B05E for ; Fri, 9 Sep 2016 01:02:56 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.501 X-Spam-Level: X-Spam-Status: No, score=-0.501 tagged_above=-999 required=5 tests=[BAYES_05=-0.5, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dfsk9gZozgC7 for ; Fri, 9 Sep 2016 01:02:52 -0700 (PDT) Received: from p1002.netstorage.at (p1002.netstorage.at [89.207.146.186]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1DE6F12B09B for ; Fri, 9 Sep 2016 01:02:50 -0700 (PDT) Received: from mailix (noaport.de [46.237.252.213]) by p1002.netstorage.at (Postfix) with ESMTPA id AD6AD82DF7; Fri, 9 Sep 2016 10:02:48 +0200 (CEST) Received: from [127.0.0.1] (HSI-KBW-46-237-252-214.hsi.kabel-badenwuerttemberg.de [46.237.252.214]) by mailix ; Fri, 9 Sep 2016 10:02:48 +0200 To: cellar@ietf.org References: From: Tobias Rapp Organization: NOA GmbH Message-ID: Date: Fri, 9 Sep 2016 10:02:48 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-PPP-Message-ID: <20160909080248.10781.22569@p1002.netstorage.at> X-PPP-Vhost: noa-archive.com Archived-At: Cc: Bertram Lyons Subject: Re: [Cellar] Matroska metadata for digitization and process history X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Sep 2016 08:02:56 -0000 On 08.09.2016 18:57, Bertram Lyons wrote: > Hi all, one question inline below. > > [...] > > I apologize if this has already been covered, but has there been > discussion of including reVTMD as a starting point for thinking about > process metadata elements that may be of interest? > > https://www.archives.gov/preservation/products/reVTMD.xsd Now that you mention it I remember that Kate Murray talked about reVTMD on day 2 of the No Time to Wait session in Berlin. I had problems to transfer the heard acronym into letters so thanks for the link :-). The structure of reVTMD seems to be simpler than PREMIS and it contains a lot of useful fields. Do you have some example file for illustration how multiple coding/process history entries are put together? From a first look it seems that there is only one place for date and operator/engineer info, it that correct? > Additionally, the UTVideoSrc Schema (hosted by Harvard) focuses on the > source of video digitization processes: > > http://hul.harvard.edu/ois/xml/xsd/drs/UTVIDEOSrc-2-0.xsd > > Perhaps these are useful to add to the conversation about source and > process metadata in Matroska if not already in the mix. Best regards, Tobias From nobody Fri Sep 9 01:30:25 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E93E612B1F3 for ; Fri, 9 Sep 2016 01:30:23 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.7 X-Spam-Level: X-Spam-Status: No, score=-2.7 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WJhuw6s5zthf for ; Fri, 9 Sep 2016 01:30:22 -0700 (PDT) Received: from mail-wm0-x22e.google.com (mail-wm0-x22e.google.com [IPv6:2a00:1450:400c:c09::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7855212B18B for ; Fri, 9 Sep 2016 01:30:21 -0700 (PDT) Received: by mail-wm0-x22e.google.com with SMTP id 1so19283709wmz.1 for ; Fri, 09 Sep 2016 01:30:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=Q6rISii/FkpeAyapaRdnH59Cmv1Bu/ezQhfNXvNhuno=; b=j7YieBRmOiBYX2rOEPvYUmtQ0HYanj0DwyfYhyfLvsuroPXIC/+q1pNVuIHrB8OKe6 MsMGIQmtKOknDZfRvoujEf3VnNMaKEngyYUIx5FoskGAhNPf/iCRO+Uq9BoIorCV9sc4 zEv8KYoR6RTY0k0nvUgUdsuEzhWLa8VYauy2FWV7Q1YKVYs/J7E2zNGKLk2c/NS5WWal OFMZ5KkETOQHi3RZUVSO7JUOiNj9RoYhFlpTqFh9Jopxcr3D/jhUDn9arRuSWf/3mlKF zJKuD+cD1krSVheUh87acHakZoTBq0MyxJqGTLHPWk1Ggcp8dEND+rgc65HA33u+9umq zB+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=Q6rISii/FkpeAyapaRdnH59Cmv1Bu/ezQhfNXvNhuno=; b=h1pUk45z+Ynd6OnCindOjPRj6SdB4l9jy/WHZvY/67qDvbcnMfVVztghzDxKzcfehs aoynjvsZkjsHVPLqDoaO/AridlT2B/xa3LrOI0hYZbrydz6XWzjVX9JzBcYGULcm1bZ7 yX5C+rAHfx/Wm7hKQx8zykvqIor0wBH0qtak+A+qyWqb8NZwbcJIcjl9vrkrqg8hns1U SHRovKiLg5LVQbDKN9PbBHKkXUs7zZLRiBSg4dpmI8Tc5qzIUpwYKX9jd4MNMtfgZCkG AzCKDAvj7MgKFbcn/6v6p5rIzXYoyQCSlErjr//kIEC/ryqelFbEFroxFMJE4Xi0tB2b WLUw== X-Gm-Message-State: AE9vXwNu4jODkKXG+0Luj0M0W59AjVqLZoRWInRAhHcTLfv8G43safQYHp79zk3IibjHxvACDFnAgm5KsnIRlA== X-Received: by 10.194.107.69 with SMTP id ha5mr1807443wjb.100.1473409819746; Fri, 09 Sep 2016 01:30:19 -0700 (PDT) MIME-Version: 1.0 Received: by 10.194.85.37 with HTTP; Fri, 9 Sep 2016 01:30:18 -0700 (PDT) In-Reply-To: References: From: Kieran O Leary Date: Fri, 9 Sep 2016 09:30:18 +0100 Message-ID: To: cellar@ietf.org Content-Type: text/plain; charset=UTF-8 Archived-At: Subject: Re: [Cellar] Matroska metadata for digitization and process history X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Sep 2016 08:30:24 -0000 Hi Tobias, On Fri, Sep 9, 2016 at 9:02 AM, Tobias Rapp wrote: > On 08.09.2016 18:57, Bertram Lyons wrote: >> >> Hi all, one question inline below. >> >> [...] >> >> I apologize if this has already been covered, but has there been >> discussion of including reVTMD as a starting point for thinking about >> process metadata elements that may be of interest? >> >> https://www.archives.gov/preservation/products/reVTMD.xsd > > > Now that you mention it I remember that Kate Murray talked about reVTMD on > day 2 of the No Time to Wait session in Berlin. I had problems to transfer > the heard acronym into letters so thanks for the link :-). > > The structure of reVTMD seems to be simpler than PREMIS and it contains a > lot of useful fields. Do you have some example file for illustration how > multiple coding/process history entries are put together? From a first look > it seems that there is only one place for date and operator/engineer info, > it that correct? > This is from an old AMIA hack day which features both standards mentioned in the previous mails. https://github.com/amiaopensource/amazingmets/blob/master/digitized_video_mets2.xml I believe that reVTMD is in the process of being actively updated as well. I think that you're correct in saying that there is only one place for date/operator info, so if multiple operators did different things in the coding process history, you can't link them to a specific task, or give more detail about what they specifically did. Best, Kieran. From nobody Fri Sep 9 02:08:22 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5456B12B204 for ; Fri, 9 Sep 2016 02:08:21 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.901 X-Spam-Level: X-Spam-Status: No, score=-1.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OEqidPqOHFOR for ; Fri, 9 Sep 2016 02:08:17 -0700 (PDT) Received: from p1002.netstorage.at (p1002.netstorage.at [89.207.146.186]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8A63312B1A7 for ; Fri, 9 Sep 2016 02:08:14 -0700 (PDT) Received: from mailix (noaport.de [46.237.252.213]) by p1002.netstorage.at (Postfix) with ESMTPA id 1AF0F8252D for ; Fri, 9 Sep 2016 11:08:12 +0200 (CEST) Received: from [127.0.0.1] (HSI-KBW-46-237-252-214.hsi.kabel-badenwuerttemberg.de [46.237.252.214]) by mailix ; Fri, 9 Sep 2016 11:08:12 +0200 To: cellar@ietf.org References: <900FE24A-0670-4A99-9CE0-300ED91103DB@dericed.com> From: Tobias Rapp Organization: NOA GmbH Message-ID: Date: Fri, 9 Sep 2016 11:08:11 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <900FE24A-0670-4A99-9CE0-300ED91103DB@dericed.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-PPP-Message-ID: <20160909090812.19065.70260@p1002.netstorage.at> X-PPP-Vhost: noa-archive.com Archived-At: Subject: Re: [Cellar] Matroska metadata for digitization and process history X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Sep 2016 09:08:21 -0000 On 07.09.2016 21:09, Dave Rice wrote: > Hi all, > > I wanted to bring a discussion here which happened a bit at the No Time > to Wait symposium a few months ago regarding metadata in Matroska about > digitization, transfer, and source. This was a discussion on how to > store data in Matroska about the history of that stored audiovisual > data, which would cover the source of that data and the historical > process used to create the current Matroska file, possibly itemizes out > that process generation by generation. > > For precedence a few other av standards cover this type of data, for > instance in WAV files the bext > chunk contains > 'codinghistory' which is a list of historical processes used to create > the stored audio. An example of bext coding history: > > A=ANALOG,M=mono,T=Studer A810; SN3690; 15 ips; open reel tape, > A=PCM,F=96000,W=24,M=mono,T=Benchmark; ADC1; SN00252; A/D, > A=PCM,F=96000,W=24,M=mono,T=Lynx; AES16; DIO, > > Also etree has a database where md5 and ffps values from SHN and FLAC > files can be related to historical processes data: > see http://db.etree.org/shnlist.php?artist=2&year=1995 (in this case the > process data is stored in a separate database rather than the file > itself but the scope is similar). > > Some use cases I'm thinking of: > > - A set of Matroska files are created using various digitization > processes and hardware and those processes should be document (example: > 1.mkv created from the output of a film scanner, 2.mkv from running a > VHS through component video to a blackmagic card and 3.mkv from running > a Betamax through cosposite video to an Elgato capture card). In some > cases the data could be collected as introduced into the mkv file as it > is written and in other cases it could be added after via mkvpropedit. > > - Another case, an mkv is created from a film scanner, then another mkv > is created from the first to make some correct corrections, and then a > third mkv is created as an access copy with a lossy compressed codec. > The process data of each mkv file could be added to with each > generational move, so that with the third file the history of its > creation could be clear. > > - One mkv file is used to create another at a different set of > characteristics, such data could communicate that the data used to be in > a different frame rate, interlacement pattern, or colorspace to give > some context to why it looks the way it does. > > There is some overlap with this concept and the PREMIS > metadata standard which documents > an XML document that defines and relates concepts such as Agents > (hardware, software, people), Events (digitization, transcoding, > processing), and Objects (source mkv files, source videotapes, source > film, etc). It could be feasible to simply determine a method of > documenting this data in PREMIS XML and then store that as an > attachment, though I think it makes it more accessible as a set of > Matroska metadata tags. The good thing about PREMIS (as far as I understand it) would be that it has a very flexible structure. In the film scanner example the video frames may be digitized on machine A, the audio part may be digitized by some different hardware B and both is later joined into a single MKV file by software C. The structure of PREMIS allows to describe this relationship between A/B/C. The bad thing about PREMIS is that while its structure is quite elaborated the specific details to be stored are not (IMHO). > I tried working on this subject a few years ago and created this sample > which > attaches a photo of the source tape and adds process metadata. Here's an > output of that metadata from FFmpeg. > > Metadata: > CREATION_TIME : 2011-05-03 02:12:03 > COMPATIBLE_BRANDS: qt > ORIGINAL_MEDIA_TYPE: {Physical format of the source tape. Vocabulary?} > ORIGINAL_MEDIA_TYPE/BARCODE: {Barcode or other identifier from > the tape.} > ORIGINAL_MEDIA_TYPE/BARCODE/source: {name of barcode authority > or creator; example "XYZ Archive Barcode"} > ORIGINAL_MEDIA_TYPE/LABEL: {The record label or imprint on the > source media object.} > ORIGINAL_MEDIA_TYPE/DATE_RECORDED: {The time that the recording > began. This is akin to the TDRC tag in ID3.} > ORIGINAL_MEDIA_TYPE/condition: {Comments on the condition of > the source, any preparation for playback of the source tape (if relevant).} > ORIGINAL_MEDIA_TYPE/initial_source_timecode: 01:02:15;10 > ORIGINAL_MEDIA_TYPE/initial_source_timecode/settings: DropFrame=Yes > / 24HourMax=No / IsVisual=No > DATE_DIGITIZED : {Date and time of the file creation process. > Enter in ISO 8601 format.} > ENCODED_BY : {Name of the technician or organization > responsible for the encoding and file creation process} > ENCODED_BY/URL : {URL for the technician or organization listed > in ENCODED_BY.} > ENCODED_BY/processing_actions: {Description of any actions > performed during processing, such as trimming silence.} > ENCODED_BY/capture_software: Blackmagic Media Express > ENCODED_BY/capture_software/version: 2.0.3 > ENCODED_BY/capture_operating_system: Ubuntu > ENCODED_BY/capture_operating_system/version: 10.10 > ENCODED_BY/capture_device: Decklink Studio SDI > ENCODED_BY/capture_device/manufacturer: Blackmagic-Design > ENCODED_BY/capture_device/serial_no: xyz6789 > ENCODED_BY/capture_device/settings: {notes on adjustments or settings} > ENCODED_BY/playback_device: SVO-5800 > ENCODED_BY/playback_device/manufacturer: Sony > ENCODED_BY/playback_device/serial_no: abc1234 > ENCODED_BY/playback_device/settings: {notes on adjustments or settings} > ENCODED_BY/playback_device/playback_signal_path: {Protocols used > to transfer audiovisual data between the playback deck and the > capture device. example: Component} > MAJOR_BRAND : qt > MINOR_VERSION : 537199360 > ENCODER : Lavf53.2.0 > > Official tags are in ALL CAPS and my own local tags are in lowercase. > > I propose a discussion on: > > - Is defining process metadata (aka digitization metadata, coding > history) worthwhile to include in CELLAR work? I think that finding/building the one-and-only process metadata standard for audio/video archiving might take some time and it can be separated from the work on the file format itself. > - Should such metadata be added to Matroska tags or should we use > existing metadata standards as attachments? Because of the above argument I'd propose to focus on adding such metadata as attachments. > - Or (building off the prior question) should we add namespaces to > Matroska metadata tags so that externally defined standards may be used > within Matroska metadata directly? Not sure. What would be the benefit compared to directly prefixing the tag name (e.g. "noa:archive_signature" or "noa_archive_signature")? BTW: I like the uppercase => official tags, lowercase => local tags convention. > - And .... what metadata values are needed? Yeah, that would take the most discussion time. First the list of entities that are part of the process needs to be defined ("actors" or "roles") like hardware (computer, replayer, A/D converter), software (OS, ingest application, transcoder, quality check) and users (ingest operator, visual inspector). Then there needs to be some way to describe how each step of the process builds on other steps. This can be linear (like in BWF or reVTMD), tree-like or even freely linked (like in PREMIS). Best regards, Tobias From nobody Sun Sep 11 19:47:12 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5C5CE12B044 for ; Sun, 11 Sep 2016 19:47:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.121 X-Spam-Level: X-Spam-Status: No, score=-1.121 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GewKaPKrR-fW for ; Sun, 11 Sep 2016 19:47:06 -0700 (PDT) Received: from s172.web-hosting.com (s172.web-hosting.com [68.65.122.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id F40D5128E19 for ; Sun, 11 Sep 2016 19:47:05 -0700 (PDT) Received: from cpe-184-152-63-84.nyc.res.rr.com ([184.152.63.84]:38661 helo=[10.0.1.77]) by server172.web-hosting.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.86_1) (envelope-from ) id 1bjHGl-002IZ3-8S; Sun, 11 Sep 2016 22:47:04 -0400 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) From: Dave Rice In-Reply-To: <5789B3D2.3060308@xiph.org> Date: Sun, 11 Sep 2016 22:46:49 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <768596D2-493B-4462-9947-C7B796F60AB1@dericed.com> References: <8E087D93-C4A4-419C-85F3-475A988E804D@dericed.com> <5789B3D2.3060308@xiph.org> To: "Timothy B. Terriberry" X-Mailer: Apple Mail (2.3124) X-OutGoing-Spam-Status: No, score=-2.6 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Cc: cellar@ietf.org Subject: Re: [Cellar] Review of draft-lhomme-cellar-ebml-00 (Was: recent CELLAR documents) X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Sep 2016 02:47:11 -0000 Hi, Last response to this, but I=E2=80=99ve worked through many of these = suggestions. I think everything is responded to with links to the = corresponding commit, else it is moved to the issue tracker in the ebml = repo. TL;DR A pull request responding to this thread can be reviewed at = https://github.com/Matroska-Org/ebml-specification/pull/99. Non-resolved = issues were moved to the tickets 88-98, see = https://github.com/Matroska-Org/ebml-specification/issues. There=E2=80=99s= a few questions to Tim, Moritz, and Steve mixed in below prefixed with = their name. Thanks much to Marion for taking the first pass at this. > On Jul 16, 2016, at 12:10 AM, Timothy B. Terriberry = wrote: >=20 > Dave Rice wrote: >> Hi all, >>=20 >> I wanted to point out the posting of three documents in = https://datatracker.ietf.org/wg/cellar/documents/. >>=20 >> EBML: https://datatracker.ietf.org/doc/draft-lhomme-cellar-ebml/ >=20 > I thought it might be helpful to have some detailed feedback early in = the process so I reviewed the EBML document. These are just my opinions = as an individual contributor. This should not be taken as a formal chair = review in any way, and is by no means a complete review, as I am sure = I've missed things. At a minimum, while I noted some typos and = grammatical faults, that wasn't my focus, and there are still plenty of = places the language could be improved. >=20 > In general, this document uses a lot of terms and concepts before = defining or explaining them, without providing cross-references, = requiring a good deal of flipping back-and-forth to understand = everything. While adding cross-references is one useful step, a more = logical organization of the document to explain concepts before = introducing constructs which rely on them would reduce the amount of = back-and-forth required and make things much easier to understand. I've = included some specific suggestions of what should move where below. I did some drafting to expand the introduction to terminology. A draft = is in this PR https://github.com/Matroska-Org/ebml-specification/pull/87 = and discussion moved to a separate thread at = https://mailarchive.ietf.org/arch/msg/cellar/J7JwDeHQQswIlIRaqAnIpcyQvHI. > Detailed comments by section follow. >=20 > =3D=3D Abstract =3D=3D >=20 >> "...this document defines an EBML Schema to convey the semantics..." > It defines a single schema, or it defines HOW schemas convey = semantics? Updated to say: "this document defines how EBML Schemas are created to = convey the semantics of an EBML Document=E2=80=9D at = https://github.com/Matroska-Org/ebml-specification/commit/ae2915c69e7eb161= ade2b1d63da7505a3fdfc6eb. >> "EBML is designed as a binary equivalent to XML and utilizes a = storage-efficient approach..." > I would replace every instance of the word "utilize" (or its = conjugations) in this document with "use=E2=80=9D. Re: =E2=80=98utilize=E2=80=99 to =E2=80=98use=E2=80=99: Marion Jaks = fixed most of this = https://github.com/Matroska-Org/ebml-specification/commit/54952268951129a6= 63aa959006a4950b45c7e012. I found one more in the front_matter and fixed it here: = https://github.com/Matroska-Org/ebml-specification/commit/c15c275460b19bde= ed530f25a31c47dab422959a. > =3D=3D=3D EBML specifications =3D=3D=3D >=20 > I would just remove the entire top-level section here, so that = "Introduction" is Section 1 instead of 1.1, "Notations and Conventions" = is Section 2 instead of 1.2, etc. I dropped all section levels by 1 here: = https://github.com/Matroska-Org/ebml-specification/commit/d246912bdd44cbcd= babb483b7006ea8a7a00acba > =3D=3D Introduction =3D=3D >=20 > Should expand/define XML. Added a parenthetical "(a framework for structuring data)=E2=80=9D. = Deserve more? = https://github.com/Matroska-Org/ebml-specification/commit/6d408daf4298b9f8= 338ad972afec393656bb476c > =3D=3D Notation and Conventions =3D=3D >=20 > The reference to RFC2119 is sufficient, you don't also need the URL. Removed. = https://github.com/Matroska-Org/ebml-specification/commit/6b3cbcedaa6373ed= b94f3864e4ff11ba1a9dfb8b > =3D=3D Security Considerations =3D=3D >=20 >> "...only marginally useful and effective data integrity options..." > This sentence fragment should be turned into an independent sentence. Paragraph re-ordered and rewritten to: "EBML itself does not offer any kind of security and does not provide = confidentiality. EBML does not provide any kind of authorization. EBML = only offers marginally useful and effective data integrity options, such = as CRC elements.=E2=80=9D = https://github.com/Matroska-Org/ebml-specification/commit/c43c113bee8d4f8a= e0be0204d2b175d8256a8fe0 >> "Attacks on an EBML reader may include:" > List formatting. Formatting error. Fixed at = https://github.com/Matroska-Org/ebml-specification/commit/ef1dd12599cb2ce5= e75630976dbac9b51a484881. > This section refers to a number of things defined elsewhere in the = document. You should probably either a) provide forward references to = the places where these things are defined or b) move this section to the = end of the document, after those things are defined. With the terminology PR at = https://github.com/Matroska-Org/ebml-specification/pull/87, this would = define Element ID, Element Data Size, Parent Element, EBML Header, and = EBML Element before the Security section. In markdown it=E2=80=99s been hard to make forward references to things = that are defined in tables. So I=E2=80=99m thinking after the current = pull requests are accepted to work on this issue = https://github.com/Matroska-Org/ebml-specification/issues/89 to move = some definitions from tables to hierarchical sections so that forward = references can work a bit easier. > A number of the things listed as "attacks" are simply things you said = someone writing an EBML Document MUST NOT do (or not doing things you = said they MUST do). But in none of those cases do you say what is = expected of someone reading a document containing these attacks. This is true. There=E2=80=99s a few places where the behavior of a = =E2=80=98reader=E2=80=99 is mentioned, but an EBML Reader is not yet = defined nor is its behavior or expectations described in detail. This is = a larger issue so I moved it to an issue in the EBML repo. = https://github.com/Matroska-Org/ebml-specification/issues/88 > =3D=3D Variable Size Integer =3D=3D >=20 >> "...shall be referred to..." > "SHALL" is an RFC 2119 keyword, but appears here in lower case. = Nothing in 2119 explicitly says that the case matters, so the careful = document author will refrain from using these words in non-2119 contexts = (i.e., unless they intentionally wish to impose a normative = requirement). This is the subject of some debate = (), = but if you do the careful thing, you will avoid getting your document = drawn into the argument and getting held up until it is resolved (which = may never happen). I didn=E2=80=99t think these =E2=80=98shall=E2=80=99s were too close to = the intent of the rfc2119:shall, so I reworded the cases where it was = used. = https://github.com/Matroska-Org/ebml-specification/commit/49ab2b6adcd2142b= 33ff15f974051b6c020b34f5 > =3D VINT_WIDTH =3D >=20 > How are these bits packed into octets? In VINT_DATA we already have "If the number of bits required for = VINT\_DATA are less than the bit size of VINT\_DATA, then VINT\_DATA may = be zero-padded to the left to a size that fits.=E2=80=9D In the Variable Size Integer section I added: "Variable Size Integers = SHALL left-pad the VINT\_DATA value with zero bits so that the whole = Variable Size Integer is octet-aligned.=E2=80=9D = https://github.com/Matroska-Org/ebml-specification/commit/5b3afb36d11d96e8= 85b44ca0362931b7651b1095 > (the "VINT Examples" section makes this clear, but you shouldn't just = rely on examples) Should there be a limit on the number of zero-value = bits? The limit would be inferred from EBMLMaxIDLength or EBMLMaxSizeLength = depending on if the VINT is used as an Element ID or an Element Data = Size. Could use help making the language less awkward but I drafted = these sentences to add to the end of the VINT_WIDTH section. "Within the EBML Header the VINT\_WIDTH MUST NOT exceed three bits in = length (meaning that the Variable Size Integer MUST NOT exceed four = octets in length). Within the EBML Body, when VINTs are used to express = an Element ID, the maximum length allowed for the VINT\_WIDTH is one = less than the value set in EBMLMaxIDLength. Within the EBML Body, when = VINTs are used to express an Element Data Size, the maximum length = allowed for the VINT\_WIDTH is one less than the value set in = EBMLMaxSizeLength.=E2=80=9D = https://github.com/Matroska-Org/ebml-specification/commit/e8a1c82ecaee8258= e4882adf6ed85f22135a56c5 > If so, what happens if this limit is exceeded? Not sure, but moved the question to the EBML Reader issue: = https://github.com/Matroska-Org/ebml-specification/issues/88#issuecomment-= 244650162. > What are you supposed to do if attempting to read a length (e.g., of a = String, which has no defined maximum length) and the entire rest of the = file consists of zero bytes? same. > The document later defines some context-specific limits, but never = answers the question about what you're supposed to do if these limits = are violated for any context. >=20 > =3D VINT_DATA =3D >=20 >> "...one bit per octet..." > It would be clearer to say "one out of eight bits", since the bit is = not, in fact, taken out of each octet. Thanks. = https://github.com/Matroska-Org/ebml-specification/commit/866a3ddb00cf5d9a= 1e80bc7a0a4c0326446b9da0 >> "...then VINT_DATA may be zero-padded..." > Another RFC 2119 keyword ("MAY"). But why is this just a MAY? What = other things are you allowed to do if "the number of bits required is = less than the bit size of VINT_DATA [sic]=E2=80=9D? Corrected to change the =E2=80=98may=E2=80=99 to =E2=80=98SHOULD=E2=80=99.= = https://github.com/Matroska-Org/ebml-specification/commit/698acf450fd87ea1= 865e3b9bc76a0af93f11b4e7 >> "...a big-endian unsigned integer." > It might be useful to state up-front that these Variable Size Integers = are only unsigned (so, e.g., the reader is not left wondering if instead = of zero-padding above they are expected to sign-extend, or something = else silly). The last sentence of the VINT_DATA section is "The VINT\_DATA value MUST = be expressed as a big-endian unsigned integer=E2=80=9D. Since the = VINT_MARKER is simply a single bit and VINT_WIDTH is a tally of = sequential zero bits, the concept of signedness seems odd to me to apply = to these as well, so I prefer the reference to unsignedness only in the = VINT_DATA section rather than in the VINT introduction. Suggestions? > =3D VINT Examples =3D >=20 > This switches from talking about "octet lengths" as is done in the = "VINT_WIDTH" section to "octet widths". You should be consistent. Normalized to =E2=80=98octet length=E2=80=99. = https://github.com/Matroska-Org/ebml-specification/commit/6d7cb99912263692= 5d0aec40877aba0c3575fe32 > The "Size" column should make it clear that this is just the VINT_DATA = size (in bits). Added sentence to clarify this, "The Size column refers to the size of = the VINT\_DATA in bits.": = https://github.com/Matroska-Org/ebml-specification/commit/4181a7cdaedce433= e38b386dd597639ddb85c9d3. > =3D=3D Element ID =3D=3D >=20 >> "By default, EBML Element IDs may be encoded... although Element IDs = of greater lengths MAY be used..." > MAY? Reworded =E2=80=9Cmay/MAY be=E2=80=9D to =E2=80=9Care=E2=80=9D. = https://github.com/Matroska-Org/ebml-specification/commit/c1a894ef053e056a= 70301af8054e106d9c61af11 > You should include a forward reference to EBMLMaxIDLength. Forward references will be easier to handle if some of the definitions = which are currently in tables are moved into sections with their own = section headers. I filed an issue about this and will work on it after = the current PRs are resolved. = https://github.com/Matroska-Org/ebml-specification/issues/89 >> "The VINT_DATA component of the Element ID MUST NOT be set to either = all zero values or all one values." > What is an implementation supposed to do if it encounters a file that = violates this restriction? What is the purpose of reserving the various = all-ones Element IDs? Or the zero Element ID, for that matter? Not sure. Steve? Moritz? I filed an issue on this question at = https://github.com/Matroska-Org/ebml-specification/issues/90. >> "The VINT_DATA component of the Element ID MUST be encoded at the = shortest valid length." > What is the definition of "valid=E2=80=9D? I added comments to the issue of defining EBML Reader and valid elements = at = https://github.com/Matroska-Org/ebml-specification/issues/88#issuecomment-= 244797740. > Is the Element ID 0100 0000 0111 1111 allowed? Although the shortest = length required to encode the integer 127 is one octet, that encoding = has VINT_DATA set to all ones, which you just said I MUST NOT do. So is = one the shortest valid length or two? I can guess what you meant by = looking at the Number of Possible Element IDs in the table below, but it = would be better to say so explicitly. We have an example sentence following this which states: "For example, = an Element ID with binary encoding of 1011 1111 is valid, whereas an = Element ID with binary encoding of 0100 0000 0011 1111 stores a = semantically equal VINT\_DATA but is invalid because a shorter VINT = encoding is possible.=E2=80=9D I added one additional example sentence to cover the case you mention: = "Additionally, an Element ID with binary encoding of 1111 1111 is = invalid since the VINT\_DATA section is set to all one values, whereas = an Element ID with binary encoding of 0100 0000 0111 1111 stores a = semantically equal VINT\_DATA and is the shortest VINT encoding is = possible.=E2=80=9D I expanded the example table to cover the all-0 and all-1 VINT_DATA = scenarios like: VINT\_WIDTH | VINT\_MARKER | VINT\_DATA | Element ID Status -----------:|-------------:|---------------:|:----------------- | 1 | 0000000 | Invalid: VINT\_DATA MUST = NOT be set to all 0 0 | 1 | 00000000000000 | Invalid: VINT\_DATA MUST = NOT be set to all 0 | 1 | 0000001 | Valid 0 | 1 | 00000000000001 | Invalid: A shorter = VINT\_DATA encoding is available. | 1 | 0111111 | Valid 0 | 1 | 00000000111111 | Invalid: A shorter = VINT\_DATA encoding is available. | 1 | 1111111 | Invalid: VINT\_DATA MUST = NOT be set to all 1 0 | 1 | 00000001111111 | Valid commit at = https://github.com/Matroska-Org/ebml-specification/commit/aa3606a2cae29525= 4040567f2c08dd1c35664e1f >> "The octet length of an Element ID determines its EBML Class." > I'd add, "...as defined in Table XX" (and give the table a number and = a caption). >=20 > Should you say something about the classes for Element IDs of longer = lengths, since you explicitly allow them above with an appropriate = EBMLMaxIDLength declaration? >=20 > =3D=3D Element Data Size =3D=3D >=20 > You should include a forward reference to EBMLMaxSizeLength. Waiting on https://github.com/Matroska-Org/ebml-specification/issues/89. >> "...the VINT_DATA component of the Element Data Size is not required = to be encoded at the shortest valid length." > "REQUIRED" is a 2119 keyword, but "NOT REQUIRED" is not, so you may = wish to rephrase this. You may also wish to mention the redundant, = equivalent encodings as a potential side channel in your Security = Considerations section. Reworded =E2=80=9Cnot required=E2=80=9D to =E2=80=9Cnot mandated=E2=80=9D.= = https://github.com/Matroska-Org/ebml-specification/commit/c1a894ef053e056a= 70301af8054e106d9c61af11 Side channel section started here: = https://github.com/Matroska-Org/ebml-specification/commit/a3850f30c51c1de6= 5e359ce4a3e4501e8e4e5217 (please review). >> "...both store a semantically equal value." > It might be helpful to the reader to explicitly say what that value = is. Added "(both 0b00000000111111 and 0b0111111, the VINT\_DATA sections of = the examples, represent the integer 63)=E2=80=9D to the end of the = sentence. = https://github.com/Matroska-Org/ebml-specification/commit/ae9371f622c5203f= 7bc3f83d775edd519edfa6c5 > You should define or forward reference what an "EBML Data Type" is. = This is the only mention of it in the document. Did you mean EBML = Element Type? >=20 >> "...definition of the EBML Element Types." > Why is this plural? Typo. Fixed in = https://github.com/Matroska-Org/ebml-specification/commit/5e496766e446558f= 1387a5add206eb51b4fdacb7?short_path=3D01f3b0f#diff-01f3b0f2fbf3e44327e6072= f64ddc8a0. >> "If an Empty Element has a 'default' value declared then that default = value MUST be interpreted as the value of the Empty Element. If an Empty = Element has no 'default' value declared then the semantic meaning of = Empty Element is defined as part of the definition of the EBML Element = Types." > This part is confusingly worded. The normative requirement combined = with passive voice makes it unclear on whom this requirement is being = imposed. I think what you are trying to say is that first, you check to = see if the definition of the Element ID corresponding to this element = has a declared default value, and, if not, you then take the default = from the Element Type associated with that Element ID. But since neither = sentence mentions Element ID, this is pretty unclear. Reworded to "If an Empty Element has a `default` value declared then the = EBML Reader MUST interpret the value of the Empty Element as the = `default` value. If an Empty Element has no `default` value declared = then the EBML Reader MUST interpret the value of the Empty Element as = defined as part of the definition of the corresponding EBML Element Type = associated with the Element ID.=E2=80=9D I reference the requirement = imposed onto the EBML Reader to make it less passive, though EBML Reader = still needs a definition. = https://github.com/Matroska-Org/ebml-specification/commit/197765732acfdbeb= f0432ef872f041594e2e4bc9?short_path=3D01f3b0f#diff-01f3b0f2fbf3e44327e6072= f64ddc8a0 >> "...SHOULD NOT be used unnecessarily." > Why not MUST? Can you describe reasons someone might reasonably = violate this? What qualifies as "necessary=E2=80=9D? I re-ordered and rewrote the paragraph about Unknown Sized Elements. "An Element Data Size with all VINT\_DATA bits set to one is reserved as = an indicator that the size of the Element is unknown. The only reserved = value for the VINT\_DATA of Element Data Size is all bits set to one. An = Element with an unknown Element Data Size is referred to as an = `Unknown-Sized Element`. Only Master Elements SHALL be Unknown-Sized = Elements. Master Elements MUST NOT use an unknown size unless the = `unknownsizeallowed` attribute of their EBML Schema is set to true. The = use of Unknown-Sized Elements allows for an Element to be written and = read before the size of the Element is known. Unknown-Sized Element MUST = NOT be used or defined unnecessarily; however if the Element Data Size = is not known before the Element Data is written, such as in some cases = of data streaming, then Unknown-Sized Elements MAY be used. The end of = an Unknown-Sized Element is determined by the beginning of the next = element that is not a valid sub-element of the Unknown-Sized Element.=E2=80= =9D = https://github.com/Matroska-Org/ebml-specification/commit/95bedcc76d3586b1= e9778196729651ec61951508 >> "...MUST be a Master-element..." > I don't think hyphenation is what you want here (hyphenation is used = in phrases that function as adjectives or adverbs, not phrases that = function as nouns). Might it be clearer to say that unknown Element Data = Size values MUST NOT be used unless the element is a Master Element? Historically this EBML data type had been consistently referred to as = =E2=80=9CMaster-element=E2=80=9D. Marion updated it to =E2=80=9CMaster = Element=E2=80=9D in = https://github.com/Matroska-Org/ebml-specification/commit/54952268951129a6= 63aa959006a4950b45c7e012. >> "...MAY only..." > This is not how 2119 MAY is meant to work. I'd rephrase with a = different keyword, e.g., again, "...MUST NOT use an unknown size unless = the 'unknownsizeallowed' attribute... is set to true." You should = forward reference the definition of the "unknownsizeallowed" attribute. Marion beat me to it: = https://github.com/Matroska-Org/ebml-specification/commit/54952268951129a6= 63aa959006a4950b45c7e012. > How does someone know if an element is a valid sub-element of a Master = Element? Can you forward reference some text that defines this? Will add forward references when working on = https://github.com/Matroska-Org/ebml-specification/issues/89. >> "The end of a Master-element with unknown size is determined by the = beginning of the next element that is not a valid sub-element of that = Master-element." > Is an invalid element ID (the all-zeros or all-ones element IDs) = considered the beginning of an element that is not a valid sub-elmeent = of a Master Element? Hmmmm. Later in the definition of Master Element it speaks to this: = "When EBML is used in transmission or streaming, data that is not part = of an EBML Element is permitted to be present within a Master Element if = `unknownsizeallowed` is enabled within that Master Element's definition. = In this case, the reader should skip data until a valid Element ID of = the same level or the next greater level of the Master Element is = found.=E2=80=9D I think an all-1 or all-0 Element ID would be considered = =E2=80=9Cnot part of an EBML Element=E2=80=9D. In this section I changed the sentence you referenced to indicate that = the =E2=80=98next element=E2=80=99 must be a =E2=80=98defined = element=E2=80=99: "The end of an Unknown-Sized Element is determined by = the beginning of the next element, defined by this document or the = corresponding EBML Schema, that is not a valid sub-element of the = Unknown-Sized Element.=E2=80=9D = https://github.com/Matroska-Org/ebml-specification/commit/edcf6659a7b3f378= fd1cfc72ced21321e6a15c5a?short_path=3D01f3b0f#diff-01f3b0f2fbf3e44327e6072= f64ddc8a0 > =3D=3D EBML Element Types =3D=3D >=20 > The section header says "EBML Element Type", but earlier text used = both "EBML Data Type" and "EBML Element Type", and the tables in this = section use "Element Data Type". You should be consistent. Normalized to "EBML Element Type". = https://github.com/Matroska-Org/ebml-specification/commit/c76d630f5b7d16dd= a4ed4478f97b0d9d5378b363 >> "Each defined EBML Element MUST have a declared Element Type." > What would it mean for an EBML Element to be "defined" but not have a = declared Element Type? The EBML Schema requires a type declared for each element. I changed the = introductory sentence of this section to: "EBML Element are defined by = an EBML Schema which MUST declare one of the follow EBML Element Types = for each EBML Element.=E2=80=9D = https://github.com/Matroska-Org/ebml-specification/commit/309898127c64f2bc= aea040b8f2fdfb0e521eb1ef > What does it mean to "have" that declaration? I.e., what normative = requirement are you actually attempting to impose here? How does someone = know that they have met it? Good question. Since the declaration are all performed with an XML = Document (an EBML Schema), I=E2=80=99ve been thinking that it may make = sense to write an XML Schema as an alternate machine-readible definition = of that XML Document. I see a few other RFC docs such as = https://tools.ietf.org/html/rfc5168 and = https://tools.ietf.org/html/rfc5935 so perhaps this is worthwhile here = as well. I filed an issue about this at = https://github.com/Matroska-Org/ebml-specification/issues/91. >> "...may be constrained..." > MAY? I rewrote that sentence as "An EBML Element Type defines a concept of = storing data within an EBML Element that describes such characteristics = as length, endianness, and definition.=E2=80=9D = https://github.com/Matroska-Org/ebml-specification/commit/d031d83aca030765= 4720b0d76674db5ecc55e6c8 >> "An Signed [sic] Integer Element with a zero-octet length represents = an integer value of zero." > This is only true for Element IDs that do not have a declared default = value, right? It would be a good idea to explicitly say that, otherwise = it appears to contradict the earlier text in the "Element Data Size" = section. The same applies for the defaults declared for every other = Element Type. Added an opening clause about this: "If the EBML Element is not defined = to have a `default` value, then a Signed Integer Element with a = zero-octet length represents an integer value of zero.=E2=80=9D Added = the same to other EBML Element Types where needed. = https://github.com/Matroska-Org/ebml-specification/commit/80b522b1a0e14fcf= d005f1f5cfbfff5ead097279 > You may also wish to mention how the signed integers are stored. I = would assume two's complement, but that's not explicitly stated. Added a clarification: "Signed Integers MUST be stored with two's = complement notation with the leftmost bit being the sign bit.=E2=80=9D = https://github.com/Matroska-Org/ebml-specification/commit/d0d30380d65e23a9= 5090e75941c7a2aff534aae6 >> "A Float Element MUST declare of length of..." > You meant "...a length of=E2=80=A6" Marion fixed this earlier. >> "A Float Elements stores a floating-point number as defined in IEEE = 754." > You meant "A Float Element stores=E2=80=A6." Marion fixed this earlier. > You should include a normative reference to IEEE 754. Added in = https://github.com/Matroska-Org/ebml-specification/commit/1d32d654e2af3782= 4462daefb621ce435f496912 >> "A String Element may declare any length from zero to 'VINTMAX'." > MAY? VINTMAX is not a single value. Its value depends on the octet = width (length?) of the Element Data Size. Is there any restriction on = this octet width? If not, I would just phrase this as, "A String Element = can have any length, including zero." However, it is obvious that no = implementation will be able to support strings of completely arbitrarily = length. Is there some required minimum that decoders are required to = handle? When is a 32-bit system with no more than 4 GB of RAM justified = in rejecting a string as too large? How about an embedded media player = in someone's television? How would someone writing an EBML document know = when writing a string if all decoders will be able to handle it? Good question. Currently and historically there hasn=E2=80=99t been a = conveyed limit on string lengths. I moved a comment to = https://github.com/Matroska-Org/ebml-specification/issues/88#issuecomment-= 244822687 as I think this needs some discussion. >> "A String Element may either be empty (zero-length)..." > MAY? The previous Element Types all define what it means when the = length is zero in the section on "Length". Should you do the same for = the string types? Added "If the EBML Element is not defined to have a `default` value, = then a String Element with a zero-octet length represents an empty = string.=E2=80=9D and an equivalent for the UTF-8 definition. Is there a = better way to say this? = https://github.com/Matroska-Org/ebml-specification/commit/87205a7f03082b5f= 8ac16ec089d8f910a058c7cd > Reference for ASCII? Added normative reference for RFC0020. = https://github.com/Matroska-Org/ebml-specification/commit/75b2b73bcbcb2073= 0a08903e08e686e4749d7d68 > What is a decoder supposed to do when it encounters a string with = octets outside of the Printable ASCII range (excluding zero octets at = the end)? Moved question to = https://github.com/Matroska-Org/ebml-specification/issues/88#issuecomment-= 244824292. >> "Octets with all bits set to zero may follow the string value when = needed." > MAY? When is this "needed=E2=80=9D? For example, when the string value of a file is edited to reduce the = size of the string by one octet. There is not enough room for a Void = Element (needs two octets) so the string could be null padded to prevent = having to rewrite the file. Or honestly the length of the Element Data = Size could be extended by using an extra byte to write the length. I extended the sentence to "Octets with all bits set to zero may follow = the string value when needed, such as reducing the length of a stored = string while maintaining the same Element Data Size.=E2=80=9D = https://github.com/Matroska-Org/ebml-specification/commit/1c37cba202442ed9= 90a6fa2aeab08cd03ebcd029 > Are these zero bytes meant to be part of the string, or is a string = with one or more zero bytes appended to it semantically equivalent to a = string without any zero bytes appended to it? If think so (Steve, Moritz, confirm?). I added a sentence to say "A = string with one or more octets with all bits set to zero and a string = without one or more octets with all bits set to zero are semantically = equal.=E2=80=9D = https://github.com/Matroska-Org/ebml-specification/commit/1c37cba202442ed9= 90a6fa2aeab08cd03ebcd029 > If so, in your Security Considerations section, you may wish to = mention these equivalent values as a potential side channel. I have a very basic understanding of side channel attacks and don=E2=80=99= t fully understand how this equivalence could be used in one, but = started a draft section on security considerations here: = https://github.com/Matroska-Org/ebml-specification/commit/a3850f30c51c1de6= 5e359ce4a3e4501e8e4e5217. Please review, suggest. > All of the above questions and comments for the String Element Type = also apply to the UTF-8 Element Type. Applied in = https://github.com/Matroska-Org/ebml-specification/commit/6ffb2e87e1c45b8f= 896d2a149bbdf525a1a0b6cd. > Again, the reference to RFC2279 is sufficient without the extra URL = (which does not even point to the same place as the URL for 2119). Cleaned up in = https://github.com/Matroska-Org/ebml-specification/commit/95cf760bbed36867= 4a3083ebdb5192ab896141a. > Should the reference to RF2279 be normative? Can the element data be = interpreted or validated without understanding the referenced document? Changed RFC2779 to normative in = https://github.com/Matroska-Org/ebml-specification/commit/477e1765603ae9ba= 980ab0d60a12bda0c897a824. > The "Date" Element Type says "Endianness: None", but Signed Integers = were specified to be big endian. Oops, changed to big endian in = https://github.com/Matroska-Org/ebml-specification/commit/7d4004b5548c6942= c09fc8f64de728f7c56fe2ea. >> "The Data Element MUST contain a Signed Integer..." > What does it meant to "contain a Signed Integer" here, given that this = is the definition of an Element Type, and a Signed Integer is also an = Element Type. Maybe it would make more sense to say it stores an integer = in the same format as a Signed Integer? Updated to "The Date Element stores an integer in the same format as the = Signed Integer Element that expresses a point in time referenced in = nanoseconds from the precise beginning of the third millennium of the = Gregorian Calendar in Coordinated Universal Time (also known as = 2001-01-01T00:00:00.000000000 UTC).=E2=80=9D = https://github.com/Matroska-Org/ebml-specification/commit/2b4d4a0692b1d97c= 5d06a935df03483af46facee > You should provide a normative reference for UTC. Added in = https://github.com/Matroska-Org/ebml-specification/commit/92b0a07f89a89697= 1490a6adfb19266a6cde7584. >> "A Master-element may declare any length from zero to 'VINTMAX'." > Same comments as for strings. Updated comment at = https://github.com/Matroska-Org/ebml-specification/issues/88#issuecomment-= 244822761. >> "The Master-element may also use an unknown length. See the section = on Element Data Size for rules that apply to elements of unknown = length." > MAY? Many =E2=80=98may=E2=80=99s reworded or capitalized in = https://github.com/Matroska-Org/ebml-specification/commit/d9a04b98ee0d8cf1= 7ece67c4a0843d9f4a4064b6. > This should include an explicit section cross-reference. I=E2=80=99ve having trouble getting cross-references working properly = through mmark and xml2rfc. I opened a ticket here; = https://github.com/miekg/mmark/issues/94. >> "Elements contained within a Master-element must be defined for use = at levels greater than the level of the Master-element." > MUST? What is a level? What does it mean to be defined for use at a = level? What is the level of the Master Element? What does it mean to be = valid at a level? You should forward reference text that explains all of = this. Will do cross-references to level after = https://github.com/Matroska-Org/ebml-specification/issues/89 and = https://github.com/miekg/mmark/issues/94. >> "...all contained Elements must be valid at level 3." > MUST? Fixed in = https://github.com/Matroska-Org/ebml-specification/commit/4a6d0939f9a8f6c6= bed6cf13715ac93bb57a6820. >> "For instance is a Master-element..." > You meant, "For instance, if a Master Element=E2=80=A6" Marion fixed this one. >> "Element Data stored within Master-elements SHOULD only consist of = EBML Elements and SHOULD NOT contain any data that is not part of an = EBML Element." > Why not MUST? Fixed in = https://github.com/Matroska-Org/ebml-specification/commit/4a6d0939f9a8f6c6= bed6cf13715ac93bb57a6820 > Can you describe reasons someone might reasonably violate these = SHOULDs? You've specified _when_ they are allowed to, but not _why_. Is = the "is permitted" case the only case in which it is permitted? If so, = you should structure the normative language to make that clear. What = does it mean for EBML to be "used in transmission or streaming"? If = someone sends you an EBML document over HTTP, and you save it to disk, = are you required to strip out any non-element data from Master Elements? Moved to an issue for now: = https://github.com/Matroska-Org/ebml-specification/issues/92 >> "Any data contained with a Master-element that is not part of an = Element SHOULD be ignored." > Why not MUST? Can you describe reasons someone might reasonably = violate this SHOULD? I suppose there=E2=80=99s some forensic scenarios where someone might = choose not to ignore such data. At any rate, changed to MUST in = https://github.com/Matroska-Org/ebml-specification/commit/b2edf60e6c634e9a= d3c01c82c36c5ca4a22b6267. > Also, you meant "within", not "with=E2=80=9D. Also fixed in = https://github.com/Matroska-Org/ebml-specification/commit/b2edf60e6c634e9a= d3c01c82c36c5ca4a22b6267. > Also, to be pedantic, any data contained within a Master Element _is_ = part of an element: the Master Element that contains it. Otherwise you = are violating the "...MUST NOT contain any data that is not part of an = EBML Element" in the "EBML Document" section below. Changed "is not part of an Element=E2=80=9D to "is not part of a Child = Element=E2=80=9D in = https://github.com/Matroska-Org/ebml-specification/commit/7ea610207ec5ed8f= 470ab0d43b935be7a90d1892. Child Element=E2=80=99s definition in drafted = in the terminology branch at = https://github.com/Matroska-Org/ebml-specification/commit/347e54bb18c7f6c9= b7b65d0ec3cf8889856d9bfb. > In your Security Considerations section, you may wish to mention this = data as a potential side channel. Included within = https://github.com/Matroska-Org/ebml-specification/commit/a3850f30c51c1de6= 5e359ce4a3e4501e8e4e5217. >> "The contents of a Binary element should not be interpreted by the = EBML parser." > What is an EBML parser? This is the first mention of it in the = document. Changed to EBML Reader which already has an issue to be defined. = https://github.com/Matroska-Org/ebml-specification/commit/e2ab139149a77605= 518f23c1ff06013340972535 > =3D=3D EBML Document =3D=3D >=20 >> "An EBML Document MUST start with an EBML Header which declares = significant characteristics of the entire EBML Body." > Is the "which" clause meant to be restrictive or unrestrictive? =E2=80=9Cwhich=E2=80=9D -> =E2=80=9Cthat=E2=80=9D = https://github.com/Matroska-Org/ebml-specification/commit/02d6bab08a1d2844= 61520d9e178a7d91ed83173e >> "...MAY only..." > This is not how 2119 MAY is meant to work. also reworded in = https://github.com/Matroska-Org/ebml-specification/commit/02d6bab08a1d2844= 61520d9e178a7d91ed83173e > Since the concept of Master Element and level is used in multiple = places in the text prior to this section, it may be helpful to move this = short paragraph earlier in the document (possibly expanding it with = additional material required to understand the subsequent text without = forward references). >=20 > =3D EBML Header =3D >=20 >> "The EBML Header may be considered as analogous to an XML = Declaration." > MAY? Reworded to "The EBML Header of an EBML Document is analogous to the XML = Declaration of an XML Document.=E2=80=9D = https://github.com/Matroska-Org/ebml-specification/commit/02d6bab08a1d2844= 61520d9e178a7d91ed83173e >> "All EBML Documents MUST begin with a valid EBML Header." > You already said this in the preceding section. Marion already removed this.=20 >> "The EBML Header documents the EBML Schema (also known as the EBML = DocType) that may be used to semantically interpret the structure and = meaning of the EBML Document." > MAY? What else would you use? What would you do if you didn't use it? Changed =E2=80=98may=E2=80=99 to =E2=80=98is=E2=80=99, also in = https://github.com/Matroska-Org/ebml-specification/commit/02d6bab08a1d2844= 61520d9e178a7d91ed83173e >> "The EBML Header consists of a single Master-element with an Element = ID of 'EBML'." > Previously, you said, "The Element ID MUST be encoded as a Variable = Size Integer." How is 'EBML' encoded as a Variable Size Integer? I = assume what you actually meant is that one is supposed to take the = numeric Element ID frame the table of the element _named_ EBML in = Section 1.11.2, but since the reader hasn't seen that table yet, there's = no forward reference, a name is not an ID, and nowhere do you explain = how those IDs were derived, this is all pretty unclear. Thanks for pointing that out. Reworded to "The EBML Header consists of a = single Master Element with an Element Name of 'EBML' and Element ID of = `0x1A45DFA3`.=E2=80=9D = https://github.com/Matroska-Org/ebml-specification/commit/02d6bab08a1d2844= 61520d9e178a7d91ed83173e >> "The EBML Header MUST ONLY contain EBML Elements that are defined as = part of the EBML Specification." > ONLY is not a 2119 keyword (though at least in this case, "only" is = consistent with the use of 2119 MUST). Lowercased to =E2=80=98only=E2=80=99. = https://github.com/Matroska-Org/ebml-specification/commit/02d6bab08a1d2844= 61520d9e178a7d91ed83173e > What is "the EBML Specification"? I would just say, "...this = document.=E2=80=9D Reworded to "This document defines precisely what EBML Elements are to = be used within the EBML Header, but does not name or define what EBML = Elements are to be used within the EBML Body.=E2=80=9D = https://github.com/Matroska-Org/ebml-specification/commit/02d6bab08a1d2844= 61520d9e178a7d91ed83173e > Can an EBML Header contain an EBML Header element? No. In the EBML Header section we already have "The EBML Header MUST = only contain EBML Elements that are defined as part of this document.=E2=80= =9D and subsequently the =E2=80=98parent=E2=80=99 attribute definition = that Steve is working on clarifies what is a valid Parent Element of = each defined Element. > Can it contain an EBML Body element? No, for same reason as above. > Both are defined in this document. >=20 >> "All EBML Elements within the EBML Header MUST NOT utilize any = Element ID with a length greater than 4 octets. All EBML Elements = within the EBML Header MUST NOT utilize any Element Data Size with a = length greater than 4 octets." > What is a decoder supposed to do if these restrictions are violated? Added this question to the issue on the definition of EBML Reader. = https://github.com/Matroska-Org/ebml-specification/issues/88 > =3D EBML Body =3D >=20 >> "All data of an EBML Document following the EBML Header may be = considered the EBML Body." > MAY? How else would you consider it? =E2=80=98may be considered=E2=80=99 reworded to =E2=80=98is=E2=80=99. = https://github.com/Matroska-Org/ebml-specification/commit/02d6bab08a1d2844= 61520d9e178a7d91ed83173e >> "The end of the EBML Body, as well as the end of the EBML Document = that contains the EBML Body, is considered as whichever comes first: the = beginning of a new level 0 EBML Header or the end of the file." > Is it possible to have a non-level-0 EBML Header? Well a Matroska file could be stored as an attachment within another = Matroska file, though in that case the attached Matroska file considered = the content of a binary Element. I wrote it this way to imply that the = next occurrence of 0x1A45DFA3 is not necessarily the end of an EBML = Document. > Can an EBML Body contain an EBML Header? Can it contain another EBML = Body? Only in binary storage, such as the attachment scenario. Otherwise the = Elements that comprise the EBML Body and EBML Header are only defined to = occur at certain levels which don=E2=80=99t permit storage at other = levels. >> "...MAY only..." > This is not how 2119 MAY is meant to work. =E2=80=9CMAY only consist=E2=80=9D reworded to =E2=80=9CMUST consist = only=E2=80=9D = https://github.com/Matroska-Org/ebml-specification/commit/02d6bab08a1d2844= 61520d9e178a7d91ed83173e > =3D EBML Schema =3D >=20 >> "...XML Schema [1]..." > This URL should probably instead be at least an informative reference. Added informative references for XML Schema and XML Document in = https://github.com/Matroska-Org/ebml-specification/commit/470d4355143aab5e= 795be8785faa721fab371b5f. >> "An EBML Schema must be expressed as well-formed XML." > MUST? Although you give a URL for XML above, I think this statement = means it needs to be a normative reference. Upgraded in = https://github.com/Matroska-Org/ebml-specification/commit/f6882cc793d3f202= dc4d3ff2c25d0181d946d705. > Should there be an XML Schema for valid EBML Schemas? I was thinking the same. Issue filed in = https://github.com/Matroska-Org/ebml-specification/issues/91. >> "An EBML Document Type is identified by a unique string stored within = the EBML Header element called DocType; for example 'matroska' or = 'webm'." > What does it mean to be unique? How is this string stored within the = EBML Header? Is DocType an EBML Element? If so, you should call it that, = instead of just a string, and provide a forward reference. If someone reads the EBML specification and has knowledge of Matroska = and decides to invent a new EBML Document Type called =E2=80=98webm=E2=80=99= it would cause confusion since there is already a =E2=80=98webm=E2=80=99 = Document Type. Although there are not many EBML Document Types, without = a registry system it would require omniscience to know if anyone had = already used a given string to identify a document type. I adjusted the section to say "An EBML Document Type is identified by a = string stored within the EBML Header in the DocType Element; for example = `matroska` or `webm`. The DocType value for an EBML Document Type SHOULD = be unique and persistent.=E2=80=9D = https://github.com/Matroska-Org/ebml-specification/commit/220fc0297a6fa0e1= 79c8c3829b7996cff659d431 >> "...which are defined in the section on Section 1.11.1.1." > Delete "the section on=E2=80=9D. Waiting on the resolution of an issue with cross-referencing to fix = this. > Did you mean "that" instead of "which=E2=80=9D? Changed to =E2=80=98that=E2=80=99.=20 >> "...but MUST NOT conflict is the definitions..." > You meant "...with the definitions=E2=80=A6." Marion already fixed this. >> "Within the EBML Schema each EBML Element is defined to occur at a = specific level." > This and the following description of Global EBML Elements, Root = Element, and Top-Level Elements would have answered a number of the = questions I raised above if presented earlier in the document. Hoping that the new terminology section address this: = https://github.com/Matroska-Org/ebml-specification/pull/87. >> "...specificied..." > Typo. Marion already fixed this. >> "...specifiied..." > Typo. Marion already fixed this. >> "...the Child EBML Element refers to the EBML Elements..." > Plural/singular mis-match. Marion already fixed this. >> "...that may be immediately contained within that Master-element." > MAY? Changed =E2=80=98may be=E2=80=99 to =E2=80=98is=E2=80=99. >> "...the Parent EBML Element may be identified..." > MAY? How else would you identify it? Changed =E2=80=98may be=E2=80=99 to =E2=80=98is=E2=80=99, though this = section MAY soon change based on a proposal Steve is working on.=20 >> "The only exception for this rule are Global EBML Elements which may = occur within any Parent EBML Element within the restriction of the = Global EBML Element's range declaration." > MAY? Is the which clause meant to be restrictive or unrestrictive? Capitalized MAY. Will return here later as Steve=E2=80=99s refactor of = the EBML Schema will impact this significantly. >> "An EBML Schema MUST declare exactly one Element at Level 0 (referred = to as the Root Element) that MUST occur exactly once within an EBML = Document. The Root Element MUST be mandatory (with minOccurs set to 1) = and MUST be defined to occur exactly once (maxOccurs set to 1)." > Is the second sentence saying the same thing as the first sentence, or = are these different requirements? If so, how do they differ? You should = provide forward references to minOccurs and maxOccurs. Dropped the second second in = https://github.com/Matroska-Org/ebml-specification/commit/6505b07a8b342e54= d2564019553fd119b53ae91d. Will add forward references when working on = https://github.com/Matroska-Org/ebml-specification/issues/89. >> "Note that the EBML and Void Elements may also occur at Level 0 but = are not considered to be Root Elements." > Delete "Note that". You should provide a forward reference for Void = Elements. All =E2=80=9CNote that=E2=80=9Ds at the beginning of sentences are = removed in = https://github.com/Matroska-Org/ebml-specification/commit/85ac2cd42bba44b9= 354c4ec3bc2adaa61aa42a48. Will add forward references when working on = https://github.com/Matroska-Org/ebml-specification/issues/89. >> "...in the Section 1.11.2 section." > Delete the second "section=E2=80=9D. Removed in = https://github.com/Matroska-Org/ebml-specification/commit/9006d4d27ceb0868= 8ede2470e9548fabc08a9678. More cleanup on cross-references to be done = within https://github.com/Matroska-Org/ebml-specification/issues/89. > =3D EBML Schema Element Attributes =3D >=20 >> "...'' uses the following attributes to define the EBML = Schema:" > These attributes are not thMarion already fixed this.e whole = definition of an EBML Schema, surely. I'd just end the sentence after = "attributes=E2=80=9D. Marion already fixed this. >> "Elements may be introduced and deprecated by using the "minver" and = "maxver" attributes of ." > Broken cross-reference. Will fix when working on = https://github.com/Matroska-Org/ebml-specification/issues/89. >> "The value of the name MUST be in the form of an NCName as defined by = the XML Schema specification [3]." > [3] should be a normative reference, not just a URL. Corrected in = https://github.com/Matroska-Org/ebml-specification/commit/37baf1d18a7871c9= df087124581f19587854e25a. >> "...or may occur..." > MAY? Changed "The level notes at what hierarchical depth the EBML Element may = occur...=E2=80=9D to "The level notes at what hierarchical depth or = depths the EBML Element MUST occur if used=E2=80=A6.=E2=80=9D. = https://github.com/Matroska-Org/ebml-specification/commit/2dd46718bccfc07b= 7b6ad57a46dc87c7a15f7e71 >> "If the "global" attribute is not expressed..." > What does it mean for an attribute to be "expressed"? Did you just = mean "present=E2=80=9D? Updated phrasing of the meaning of non-mandatory attributes if not = =E2=80=98present=E2=80=99 in = https://github.com/Matroska-Org/ebml-specification/commit/43bd6f0c527d58c0= 3918e24903e12d190e8bb2a8. Mostly updating =E2=80=98expressed=E2=80=99 or =E2=80=99used' to = =E2=80=98present=E2=80=99 and dropped unneeded =E2=80=98it is assumed=E2=80= =99 language. >> "The Element ID expressed in hexadecimal notation prefixed by a = '0x'." > Is this meant to be the Variable Size Integer encoding of the element = ID, or the the integer that that encoding encodes? =46rom the examples, = I believe you mean the former, but you should say so explicitly rather = than requiring someone to work it out from the examples. If it is the = former, you should also specify the byte order, since the definition of = the Variable Size Integer merely specifies how it is encoded, bit-by-bit = but here you are giving a single large integer value. Given that = everything else is big-endian (except the CRC!), I can guess, but I = shouldn't have to guess. I clarified this in = https://github.com/Matroska-Org/ebml-specification/commit/b9306b9a967c0561= c4b3ceb770489c1f185e684e by changing "The Element ID expressed in = hexadecimal notation=E2=80=9D to "The Element ID encoded as a Variable = Size Integer expressed in hexadecimal notation=E2=80=9D. The sentence = ends with "that is read and stored in big-endian order=E2=80=9D so I = think the endianness is clear from this. >> "An integer to express the minimal number of occurrences that the = EBML Element MUST occur within its Parent Element if its Parent Element = is used." > That's pretty tortured. How about, "An integer expressing the minimum = number of occurrences of this EBML Element within its Parent Element. = Each instance of the Parent Element MUST contain at least this many = instances of this Element." What is one expected to do if it does not = contain this many? Changed as recommended in = https://github.com/Matroska-Org/ebml-specification/commit/40fe4cb33d7e5d06= e9f58d389f95d425437071ba. Moved the questions about expectations if = minoccurs is not met to = https://github.com/Matroska-Org/ebml-specification/issues/88#issuecomment-= 246198633. >> "If the Element has no Parent Level (as is the case with Elements at = Level 0), then minOccurs refers to constaints on the Element's = occurrence within the EBML Document." > Earlier you said there had to be exactly one element at level 0, and = that it could only occur once, and also that its minOccurs had to be set = to 1. So why is this sentence necessary? What is the point it's trying = to make? Earlier it said that "An EBML Schema MUST declare exactly one Element at = Level 0 (referred to as the Root Element) that MUST occur exactly once = within an EBML Document. The EBML and Void Elements may also occur at = Level 0 but are not considered to be Root Elements.=E2=80=9D So Void is = an exception here but still gets a maxOccurs value. For instance: is valid, and only the is considered a Root Element. = Potentially if Void is changed from 0+ to 1+ then a lot of this language = could be cleaned up. I started an issue at = https://github.com/Matroska-Org/ebml-specification/issues/93. >> "If the minOccurs attribute is not expressed for that Element then = that Element shall be considered to have a minOccurs value of 0. This = value of minOccurs MUST be a positive integer." > These two sentences seem to literally say that a minOccurs value of 0 = MUST be a positive integer. Did you mean "the" instead of "this"? Does = that mean that the only way to express a minOccurs of value of 0 is to = omit the attribute? Oops, changing positive to = non-negative.https://github.com/Matroska-Org/ebml-specification/commit/4e6= 92a8972cb74175aace57fe2de3a1a285e8b7c >> "...see the Section 1.11.1.6." > Delete "the=E2=80=9D. Marion fixed this. >> "A value to express the maximum number of occurrences that the EBML = Element MAY occur within its Parent Element if its Parent Element is = used." > Just as tortured as above, but can be fixed in a similar way. What is = one expected to do if there are more than this many occurrences? If = maxOccurs is a positive integer, and maxOccurs instances of the element = have already been encountered within a Master Element with an unknown = size, is the another instance of this element considered "...the = beginning of the next element that is not a valid sub-element of that = Master-element=E2=80=9D? Good question about the end of unknown-sized elements. I moved the = question to = https://github.com/Matroska-Org/ebml-specification/issues/94. Moved question about maxOccurs being exceeded to = https://github.com/Matroska-Org/ebml-specification/issues/88#issuecomment-= 246200237. And cleaned up the tortured language in = https://github.com/Matroska-Org/ebml-specification/commit/ef04a4acc6cd621f= 6fdb354f463ffa611b35b2bb. >> "...the Element is an Section 1.11.1.3." > Broken cross reference. Will fix with = https://github.com/Matroska-Org/ebml-specification/issues/89. >> "If specified that the value of the EBML Element MUST be within the = defined range inclusively." > Either you meant to remove "that" or this is an incomplete sentence = fragment. Marion fixed this. >> "See the Section 1.11.1.4..." > Delete "the=E2=80=9D. Marion fixed this. >> "A default value may be provided." > MAY? Is the existence of this attribute sufficient to provide it, or = even if the attribute exists, is providing a default value something it = merely MAY do? I removed the sentence and rewrote the following sentence to be "If an = Element is mandatory but not written within its Parent EBML Element or = stored as an Empty Element, then the EBML Reader of the EBML Document = MUST semantically insert this specified default value for the = Element.=E2=80=9D = https://github.com/Matroska-Org/ebml-specification/commit/31a54bbab0ba48d6= 6128543c92254092b95135ba=20 >> "If an Element is mandatory but not written within its Parent EBML = Element, then the parser of the EBML Document MUST insert the defined = default value of the Element." > What does it mean for an element to be mandatory? I think you mean = minOccurs > 0, Yes, I added that as a parenthetical to the above sentence so "is = mandatory (has a minOccurs value greater than zero) but not written=E2=80=9D= .=20 > though that then raises the question of what it means if minOccurs > = 1. Are parsers then meant to insert enough copies of the element set to = the default value to make up for any and all missing occurrences? Moved this question to = https://github.com/Matroska-Org/ebml-specification/issues/95. > Also, again, what is a parser? What does it mean for it to insert = something? Where does it insert it? Changed =E2=80=98insert=E2=80=99 to =E2=80=98semantically interpret=E2=80=99= . New version: "If an Element is mandatory (has a minOccurs value = greater than zero) but not written within its Parent EBML Element or = stored as an Empty Element, then the EBML Reader of the EBML Document = MUST semantically interpret the Element as present with this specified = default value for the Element.=E2=80=9D = https://github.com/Matroska-Org/ebml-specification/commit/ac34f1fd3eb36670= 7597ce6985c5109ca0eb6729 >> "As defined within the Section 1.8, the type MUST be set..." > Delete the first "the=E2=80=9D. Marion got this. > Section 1.8 does not define that the type MUST be set to anything. I = assume you mean that the section defines the content of the element for = each type, but that's not what this sentence says. Sentences re-ordered and clarified to: "The type MUST be set to one of = the following values: 'integer' (signed integer), 'uinteger' (unsigned = integer), 'float', 'string', 'date', 'utf-8', 'master', or 'binary'. The = content of each type is defined within [section on EBML Element = Types](#ebml-element-types).=E2=80=9D = https://github.com/Matroska-Org/ebml-specification/commit/275a8ee33bfbc678= b60bbf8c03027d162b77521b >> "The 'unknownsizeallowed' attribute only applies to Master-elements." > Does that mean it's legal for a non-Master Element to include an = "unknownsizeallowed" in its schema, but someone validating a document = with that schema is required to ignore it? Same question for the = "recursive" attribute. Reworded to: "Elements that are not Master Elements MUST NOT set = `unknownsizeallowed` to true.=E2=80=9D Same approach with recursive. = https://github.com/Matroska-Org/ebml-specification/commit/a28c86e5a6803a54= f6dc6161cd9de1499c0e495c >> "If the "maxver" attribute is not used it is assumed that the element = has a maximum version equal to the value stored in the "version" = attribute of ." > Broken cross-reference. Also, what does it mean for an attribute to be = "used"? Is that like "expressed"? Or does it really just meant "present" = again? Already fixed in = https://github.com/Matroska-Org/ebml-specification/commit/43bd6f0c527d58c0= 3918e24903e12d190e8bb2a8. Changed =E2=80=98used=E2=80=99 to = =E2=80=98present=E2=80=99. > What is one expected to do if "maxver" is smaller than "minver=E2=80=9D?= Added: `maxver` MUST be greater than or equal to `minver`. = https://github.com/Matroska-Org/ebml-specification/commit/7b6e2653046985d4= 5168dd63b035b3c68c72a597 >> "The '' nodes shall contain..." > SHALL? Removed =E2=80=98shall=E2=80=99. >> "The '' sub-element may use a 'lang' attribute which = may be set to the RFC 5646 value of the language of the element's = documentation." > MAY (two times)? I'm also allowed to set it to some other completely = unrelated language? RFC 5646 should be a normative reference. Reworded to drop one may and capitalize the other: "The = `` sub-element MAY use a `lang` attribute which is set to = the [@!RFC5646] value of the language of the element's documentation.=E2=80= =9D And RFC5646 is now normative. = https://github.com/Matroska-Org/ebml-specification/commit/effffba7f90a8368= ad85f47ad0b87e231b0a7d89 >> "The '' sub-element may use a 'type' attribute to = distinguish the meaning of the documentation." > MAY? Capitalized in above commit. >> "Recommended values for the '' sub-element's 'type' = attribute include..." > RECOMMENDED? What do these values mean? Why not MUST? Can you describe = reasons someone might reasonably choose to use other "type" attribute = values? Updated to MUST. = https://github.com/Matroska-Org/ebml-specification/commit/fbbe8b38e924399b= 9bffb27e74be48d1d9bc67fd >> "An '' node that defines an EBML Element which is a Child = Element of another Parent Element MUST be stored as an immediate = sub-element of the '' node that defines the Parent Element." > Is it allowed to have two nodes with the same Element ID as = children of different Parent Elements? Is one simply expected to = duplicate the whole contents of that node under both parents? = Are they allowed to have different attribute values for things like = "level", "minOccurs", "maxOccurs", etc.? If so, what is the complete = list of attributes that are allowed to differ? I think the answer to the first one is no. Suggest that this one is = resolved along with Steve=E2=80=99s refactor of the EBML Schema and use = of the parent attribute. > =3D Identically Recurring Elements =3D >=20 >> "An Identically Recurring Element is an Element that may occur..." > MAY? Capitalized. = https://github.com/Matroska-Org/ebml-specification/commit/a332fe9bd873ff08= 4c1c9be1eb0fd6171cec6ef6 >> "...and optimize the use of EBML in transmission." > What objective function is being optimized? How do Identically = Recurrent Elements optimize it? Trying to have a non-Matroska example, but added "For instance a = pertinent Top-Level Element could be periodically resent within a data = stream so that an EBML Reader which starts reading the stream from the = middle could better interpret the contents.=E2=80=9D = https://github.com/Matroska-Org/ebml-specification/commit/fcbb1bbde9465aa4= e2fbbbeab11823fc94f3cc7b >> "Identically Recurring Elements SHOULD include a CRC-32 Element as a = Child Element..." > Why not MUST? Can you describe reasons someone might reasonably = violate this? Ping to Moritz. ;) > This section is good for describing what someone is supposed to do if = the Identically Recurring Elements are not, in fact, identical. Though, = you may wish to point out in the Security Considerations that the = extraneous copies or copies with an invalid CRC-32 Element are a = potential side channel. Included within = https://github.com/Matroska-Org/ebml-specification/commit/a3850f30c51c1de6= 5e359ce4a3e4501e8e4e5217. > =3D Expression of range [sic] =3D >=20 >> "The 'range' attribute does not support date EBML Elements." > The definition of the range attribute said it could be used for Dates. Oops, dates added. = https://github.com/Matroska-Org/ebml-specification/commit/ba0b0d9986193315= 8b14ed2630da50d57cd0dfe4 >> "The 'range' expression may contain whitespace for readability but = whitespace within a 'range' expression MUST NOT convey meaning." > How would whitespace within a 'range' expression convey meaning? I = think this is a vague attempt to say _where_ whitespace is allowed, but = as vague attempts go, it's pretty vague. For instance there=E2=80=99s a range value which says "> 0x0p+0=E2=80=9D= . Slightly easier to read than ">0x0p+0=E2=80=9D, but intended to = mean the same thing. Also later the prefix `not ` is used which includes = a space as `not 0` is more readable than `not0`. >> "'x-y' where x and y are integers or floats and 'y' must be greater = than "x", meaning that the value must be greater than or equal to 'x' = and less than or equal to 'y'." > MUST (two times)? Capitalized in = https://github.com/Matroska-Org/ebml-specification/commit/fa54ea4d365f78b0= 07970b1e908f9e6e521a0ff7.=20 > What is one supposed to do if 'y' is less than or equal to 'x'? Is = there even any reason to forbid 'x' and 'y' from being the same? Added a mandate that "`x` MUST be less than `y`.=E2=80=9D in = https://github.com/Matroska-Org/ebml-specification/commit/444973f62008fdf9= 77c2cad2c9eb014d2a55f672. >> "'>x' where 'x' is an integer or float, meaning that the value MUST = be greater than 'x'." > Earlier in the definition of the range attribute, you said something = about "within the defined range inclusively", but this appears to define = a range that excludes its end-point. So which is it, inclusive or = exclusive? Is this meant to be treated as >=3D? Good catch. I prefer to let the range expression define whether it is = inclusive or exclusive, so I removed the work =E2=80=98inclusively=E2=80=99= from the sentence you quote. = https://github.com/Matroska-Org/ebml-specification/commit/9c24ab7dba5a2f54= ace4cf0e50839cd4286e3be6 I also added expressions for greater than or = equal, less than, and less than or equal. = https://github.com/Matroska-Org/ebml-specification/commit/0d9aef8904a5c4ed= 9878fc9f779d28ffe5e815f4 We have `x-y` as an inclusive range. Do we need expressions for an = exclusive range? >> "The 'range' may use the prefix 'not' to indicate that the expressed = range is negated." > MAY? What does this do to the inclusivity rules? I think this is resolved by moving inclusive/exclusive to range = expression rather than within the range definition. >> "Please also see the section on Section 1.11.1.5." > Delete "the section on=E2=80=9D. Marion fixed this. >> "...as defined in the C11 standard ISO/IEC 9899:2011 [6]..." > This should be a normative reference. Done. = https://github.com/Matroska-Org/ebml-specification/commit/96f9893d1f1ed5b8= 22514aa284828ea30a97c7d6 > The discussion on hyphens also leaves out how the hyphen that = separates the range is to be disambiguated from a signed (negative) = float. Perhaps it's worth defining an EBNF grammar for this field? Started issue here = https://github.com/Matroska-Org/ebml-specification/issues/96. > =3D Note on the Use of default attributes to define [sic] Mandatory = EBML Elements =3D >=20 >> "If a Mandatory EBML Element..." > This is the first time this term has appeared, at least in capitalized = form. It appears here with no definition. Added this sentence to the minOccurs definition: "An EBML Element = defined with a `minOccurs` value greater than zero is called a Mandatory = EBML Element.=E2=80=9D = https://github.com/Matroska-Org/ebml-specification/commit/77ef392a86698c1f= 3b88681eb4bdd127758ddd97 >> "In this case, the default value of the Mandatory EBML Element may be = assumed although the EBML Element is not present within its Parent EBML = Element." > MAY? I'm allowed to not do any of that, also? Correct to a MUST and rewritten to: "In this case, the default value of = the Mandatory EBML Element MUST be interpretted by the EBML Reader = although the EBML Element is not present within its Parent EBML = Element.=E2=80=9D = https://github.com/Matroska-Org/ebml-specification/commit/b1e7cf9ebd8de3e1= 0a5cff48a574cfbaf16da213 >> "Also in this case the parser of the EBML Document MUST insert the = defined default value of the Element." > Again, what is a parser? What does it mean to insert it? Where does it = get inserted? Also, why say this again when you already said it? (with = 2119 language both times!) That's just a recipe for having it = interpreted two different ways. Redundant final sentence removed, also in = https://github.com/Matroska-Org/ebml-specification/commit/b1e7cf9ebd8de3e1= 0a5cff48a574cfbaf16da213. >> "...than the EBML Element must be present as well." > You meant "then=E2=80=9D. Marion fixed this. > MUST? Capitalized in = https://github.com/Matroska-Org/ebml-specification/commit/8aece89a65fdc287= 0a6c580fefce71d227beae74. >> "...then the EBML Element MUST be used." > What does it mean to be used? Changed to present, rather than =E2=80=98used=E2=80=99. = https://github.com/Matroska-Org/ebml-specification/commit/f4d03f6bedf520c7= 63ea517e74b85da18aa03600 > If the Element is an Identically Recurring Element, how do I satisfy = this rule (which says I must use every such value) and also the rule = that I only use the first one with a valid CRC (or failing that, just = the first one)? Tim, could you restate this question. I=E2=80=99m confused about the = =E2=80=9Cevery such value=E2=80=9D part. >> "Is the Parent Element used?" > What does it mean to be used? Same. = https://github.com/Matroska-Org/ebml-specification/commit/f4d03f6bedf520c7= 63ea517e74b85da18aa03600 >> "Then is storing the EBML Element required?" > REQUIRED? Capitalized in = https://github.com/Matroska-Org/ebml-specification/commit/8aece89a65fdc287= 0a6c580fefce71d227beae74. > The last two lines of this table are also identical to the preceding = two. Good catch, removed in = https://github.com/Matroska-Org/ebml-specification/commit/52f908dd57554b32= a7bf8b863ee89f10a1e25839. > =3D Note on the Use of default attributes to define [sic] = non-Mandatory EBML Elements =3D >=20 > This section appears to simply be repeating something already said = elsewhere. This section seems redundant as you say. I removed it in = https://github.com/Matroska-Org/ebml-specification/commit/6b3172965684aaba= 49a72534bc443e3a1a863f4c. > =3D EBML Header Elements =3D >=20 > You've gone to a lot of trouble to define an XML schema for defining = EBML elements, and then this section doesn't use it. I=E2=80=99ve been uncertain as to how to handle this. For instance there = is no means in an XML Schema to define an XML Declaration (the xml = equivalent of an EBML Header). To define EBML Header in an EBML Schema = would be incomplete since the EBML Schema needs a non-EBMLHeader Root = Element. > Do you think it would be a good idea to at least have the table row = names match the schema attribute names? (capitalization, "EBML ID" vs. = "id", "Mandatory"/"Multiple" vs. "minOccurs"/"maxOccurs", etc.) Does the = lack of a Mandatory attribute in the XML schema and the lack of a = definition for "mandatory" in this document mean that only elements = defined in this section and the next can ever be Mandatory? Should the = specification of invalid values like "-" for Range and Default be taken = to mean that those attributes should be treated as "not expressed" or is = it also legal to include values like that in an EBML Schema? The language should be adjusted as noted. I filed a ticket. = https://github.com/Matroska-Org/ebml-specification/issues/97 >> "Range | 1" > Do I take this to mean that the _only_ valid values for EBMLVersion = and EBMLReadVersion are 1? What is one expected to do if one encounters = a different version number? E.g., what if EBMLReadVersion is 1 but = EBMLVersion is 2? EBMLVersion as 2 is undefined, there=E2=80=99s no such definition. If = CELLAR and/or IETF defines EBML version 2 then this range would be = changed from =E2=80=981=E2=80=99 to =E2=80=981-2=E2=80=99. I added a mandate that version>=3Dreadversion in both EBMLVersion and = DocTypeVersion in = https://github.com/Matroska-Org/ebml-specification/commit/f486236285f0af3b= 5ad959821eb3f1d989140f46. >> "The version of EBML parser used to create the EBML Document." > Why is a parser creating documents? Changed to EBML Writer and updated the issue: = https://github.com/Matroska-Org/ebml-specification/issues/88. > This is why I ask what you might think is the obvious question, "What = is a parser?=E2=80=9D Also in issue #88. > Should there be a maximum values allowed for EBMLMaxIDLength and = EBMLMaxSizeLength? How large of a value can an implementation put for = these with a reasonable expectation that it will be supported? Filed issue: = https://github.com/Matroska-Org/ebml-specification/issues/98. > =3D Global elements (used everywhere in the format) [sic] =3D >=20 >> "All Top-Level Elements of an EBML Document SHOULD include a CRC-32 = Element as a Child Element." > Why not MUST? Can you describe reasons someone might reasonably = violate this? Moritz, ping. >> "The CRC in use is the IEEE-CRC-32 algorithm as used in the ISO 3309 = standard and in section 8.1.1.6.2 of ITU-T recommendation V.42..." > This should be cited as a normative reference. Both ISO and ITU references now formatted as normative. = https://github.com/Matroska-Org/ebml-specification/commit/8b4eaf9eb6d262ac= 07e1fbafe30c35026bd9de2d > Void elements should probably also be called out in the Security = Considerations section as a possible side channel. Included within = https://github.com/Matroska-Org/ebml-specification/commit/a3850f30c51c1de6= 5e359ce4a3e4501e8e4e5217. Thanks much for this comprehensive review! Dave Rice= From nobody Tue Sep 13 06:35:40 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 51D8D12B6A0 for ; Tue, 13 Sep 2016 06:35:39 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.7 X-Spam-Level: X-Spam-Status: No, score=-2.7 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=nypl.org Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hXlI7alUUDOq for ; Tue, 13 Sep 2016 06:35:34 -0700 (PDT) Received: from mail-vk0-x22d.google.com (mail-vk0-x22d.google.com [IPv6:2607:f8b0:400c:c05::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1AF2712B407 for ; Tue, 13 Sep 2016 06:21:04 -0700 (PDT) Received: by mail-vk0-x22d.google.com with SMTP id m62so122878784vkd.3 for ; Tue, 13 Sep 2016 06:21:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nypl.org; s=google; h=mime-version:from:date:message-id:subject:to; bh=P9zWza6adKMNWVZenXbCARaF542ZKgwa8bxAJV/2cyA=; b=fAuUnj+/CheWuKtuwqXaAutzBSRj0q3iVJ53QT65pV5rW1LlGUowPBVU6ImurQLIob QWeC51/LdThsv2ywqbL5/emWO1qJJEkZPt4CZ3TAtRkKQGVGOdn41QPbx4DfN1I83Zu6 w9Y8oWnt9GKme4/O9V07vpXsTYmYj28kFKbdq+nouyuXffewLQGRYK0XW8TEwiOVPWpf DpWnQGeemRWoaQlxqtSzPkTUB4KYG9mvjlctdgz+2RDkhJG/1wDur3TvtT083t/kQaiQ ujz2I26LqleoflPY/XFp1EcaWLmyfkRTycdXfgUnZImVITIVET/B1PthWJMZv+Ypz3k+ jOxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=P9zWza6adKMNWVZenXbCARaF542ZKgwa8bxAJV/2cyA=; b=EuZvVdaMfaokyWwu8KJjoSK2X4xuHugznJbstEDeubwQ25CAcb0luXy4YC4MqZ5eCk RZY+l6+tP7F0FOKhTA7jDZPyFDW49m0uHXxZBmc8pfEtrG9bmTcvWKne45GYY4noG9yL /YK2bmOlgSfOL066gOiIHRXbBT4x6W+oyXk+sq6H7HL0rXqFAho0ZQHzXebAbm8D2D6d xubjD1RScsnsaRjIgTWp+aqHfUes2pbsM64K27VLK4fWWRdxWLu43lta5BuS6c/MnDF4 Zhtu+GVUPrMaZEyjsp1zZP1qRWBL+Oqs3ggQI4EDw3Q8717vHrNrT9DXXaWmK4o2AuRu wdtw== X-Gm-Message-State: AE9vXwNk+9fHEHn7Xo/Bhl7x3077VcRU/1WC6m3XePB5vriV2iTHOU1LoICfuaI0JBpFoPA3Y7/h2bfiW/Z7MYk5 X-Received: by 10.31.9.137 with SMTP id 131mr654406vkj.117.1473772862920; Tue, 13 Sep 2016 06:21:02 -0700 (PDT) MIME-Version: 1.0 Received: by 10.176.2.21 with HTTP; Tue, 13 Sep 2016 06:20:42 -0700 (PDT) From: Nick Krabbenhoeft Date: Tue, 13 Sep 2016 09:20:42 -0400 Message-ID: To: cellar@ietf.org Content-Type: multipart/alternative; boundary=001a11440eb8f97836053c637bcd Archived-At: Subject: [Cellar] Standardizing FLAC X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Sep 2016 13:35:39 -0000 --001a11440eb8f97836053c637bcd Content-Type: text/plain; charset=UTF-8 What's the status of standardizing FLAC? I work at NYPL where we are using WAVE32 for our audio preservation masters. However, WAVE32's have a 4GB size limit, which means that long media like 2+ hour open-reel recordings are split into part files. I'm looking at FLAC as a potential solution to my container problem, and I would be happy to help with standardization efforts. Best, Nick -- *Nick Krabbenhoeft | The New York Public Library* *Head of Digital Preservation* Special Collections Administration 476 Fifth Avenue, New York, NY 10018-2788 T: (917) 229-9596 | nickkrabbenhoeft@nypl.org --001a11440eb8f97836053c637bcd Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
What's the status of standardizing FLAC?

I work at NYPL where we are using WAVE32 for our audio preservation m= asters. However, WAVE32's have a 4GB size limit, which means that long = media like 2+ hour open-reel recordings are split into part files.

I'm looking at FLAC as a potential solution to my cont= ainer problem, and I would be happy to help with standardization efforts.

Best,
Nick=C2=A0
--

Nick Krabbenhoeft | The New York Public Library
Head of Digital = Preservation

Special Collections Administration
476 Fifth Ave= nue, New York, NY 10018-2788
T:=C2=A0(917) 229-9596=C2=A0|=C2=A0nickkrabbenhoeft@nypl.org

<= /div>
--001a11440eb8f97836053c637bcd-- From nobody Tue Sep 13 07:32:13 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1920C12B607 for ; Tue, 13 Sep 2016 07:32:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.699 X-Spam-Level: X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id j9iTnwlVNX18 for ; Tue, 13 Sep 2016 07:32:06 -0700 (PDT) Received: from mail-it0-x234.google.com (mail-it0-x234.google.com [IPv6:2607:f8b0:4001:c0b::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6319C12B624 for ; Tue, 13 Sep 2016 06:53:09 -0700 (PDT) Received: by mail-it0-x234.google.com with SMTP id r192so36563495ita.0 for ; Tue, 13 Sep 2016 06:53:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=ZLsUHGDvN8bhirFdqRE+ESXM2B5P95WbiBauF0EUk2g=; b=zrZ6H81Sfz3IChMDokxrpIWyR3OpwNBfE9fGUSGyVEx1vPc7K1B1y1Fs719bSTwsr2 7AOZBDAvIjTqU8p1pZblSXTYMq4NXhwGSNbBwhQ0gsWDMa2M223U2ekQW3RtYn2IrHIA RcySPpaO2AXiIn2qOcVxV3lzriPGZdrO/5wDCGTr5owzRJPjtMNJO8Kd8pW0Oeun0w48 IzxrfuZQNfJNdkqDbefigr3GnGKxNMQaYOjMI6wbNDM396f0p+Xlf6vUlIupmjMUNMCM L+lqLAazhalgqJdmB4QRcp/FwKFj2Pd2pllR51bjdB79Dl5OQsRjE5mj+lDZvJ/4wVtj V3+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=ZLsUHGDvN8bhirFdqRE+ESXM2B5P95WbiBauF0EUk2g=; b=Nx2LYUCdLPZM1BKX0wOtlBQyZdCYVQBN1JCOZEJ/c5likShtN/6k+nTqqz0KmwuYbW mo/EmBmDsOIqZXo3eY6+ZvhxI3WtSv+jqX74kHMjaFWe5u+SiSEnvX4wcxNB4+GhyzxS JHUmGN7Xhi9I5WifLKN9OQft3+w2jpckfd+PpeDU4YkvyVblpNISR30X10e0AYrcp9Ju kSghwRHFWFPJgQ06LHj3rDMGBzaloiVCzS+OjXQvesfeVvAuCBGE3Jxaw/pQZ7bEtsRk 4B0PRi5hqj4pSAkoRdLiFiw6EvAiZ1pU9R7X71aDIWkcZ+Rk2WGrbhWmdEPxoVSVZhDe g5Cw== X-Gm-Message-State: AE9vXwP7KIQWLPNHZ55N91hnYrUA+9jDAC6RYEyqeIOp0gMNDEoUr9EIws3+X9KTD5vrDjYj9Xly+S+NIMA5Kg== X-Received: by 10.107.12.140 with SMTP id 12mr1738629iom.6.1473774788599; Tue, 13 Sep 2016 06:53:08 -0700 (PDT) MIME-Version: 1.0 Received: by 10.107.40.4 with HTTP; Tue, 13 Sep 2016 06:52:28 -0700 (PDT) In-Reply-To: References: From: Tessa Fallon Date: Tue, 13 Sep 2016 09:52:28 -0400 Message-ID: To: Nick Krabbenhoeft Content-Type: multipart/alternative; boundary=001a113f8f1cc100d8053c63ee25 Archived-At: Cc: cellar@ietf.org Subject: Re: [Cellar] Standardizing FLAC X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Sep 2016 14:32:11 -0000 --001a113f8f1cc100d8053c63ee25 Content-Type: text/plain; charset=UTF-8 HI Nick, To date there has not been much discussion of FLAC as efforts have been focused on FFV1 and Matroska. If you're interested in moving this forward, you're welcome to initiate the discussion, We did get initial buy-in from the FLAC developer community when we chartered the working group. I'm not aware of any crossover between that group and CELLAR at the current time, and if that is really the case I'd very much like it to change. Happy to answer any questions. Best, Tessa On Tue, Sep 13, 2016 at 9:20 AM, Nick Krabbenhoeft < nickkrabbenhoeft@nypl.org> wrote: > What's the status of standardizing FLAC? > > I work at NYPL where we are using WAVE32 for our audio preservation > masters. However, WAVE32's have a 4GB size limit, which means that long > media like 2+ hour open-reel recordings are split into part files. > > I'm looking at FLAC as a potential solution to my container problem, and I > would be happy to help with standardization efforts. > > Best, > Nick > -- > > *Nick Krabbenhoeft | The New York Public Library* > *Head of Digital Preservation* > > Special Collections Administration > 476 Fifth Avenue, New York, NY 10018-2788 > T: (917) 229-9596 | nickkrabbenhoeft@nypl.org > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > > --001a113f8f1cc100d8053c63ee25 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
HI Nick,

To date there has not been muc= h discussion of FLAC as efforts have been focused on FFV1 and Matroska.=C2= =A0 If you're interested in moving this forward, you're welcome to = initiate the discussion, =C2=A0We did get initial buy-in from the FLAC deve= loper community when we chartered the working group. I'm not aware of a= ny crossover between that group and CELLAR at the current time, and if that= is really the case I'd very much like it to change.=C2=A0 Happy to ans= wer any questions.

Best,

Tessa
=

On Tue, Sep= 13, 2016 at 9:20 AM, Nick Krabbenhoeft <nickkrabbenhoeft@nypl.org= > wrote:
W= hat's the status of standardizing FLAC?

I work at NY= PL where we are using WAVE32 for our audio preservation masters. However, W= AVE32's have a 4GB size limit, which means that long media like 2+ hour= open-reel recordings are split into part files.

I= 'm looking at FLAC as a potential solution to my container problem, and= I would be happy to help with standardization efforts.

Best,
Nick=C2=A0
--

Nick Krabbenhoeft | The New York Public Library
Head of Digit= al Preservation

Special Collections Administration
476 Fifth = Avenue, New York, NY 10018-2788
T:=C2=A0(917) 229-9596=C2=A0|=C2=A0nickkrabbenhoeft@nypl.org


_______________________________________________
Cellar mailing list
Cellar@ietf.org
https://www.ietf.org/mailman/listinfo/cellar

--001a113f8f1cc100d8053c63ee25-- From nobody Tue Sep 13 07:37:00 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6409312B504 for ; Tue, 13 Sep 2016 07:36:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.12 X-Spam-Level: X-Spam-Status: No, score=-1.12 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qe-upWB0PuYW for ; Tue, 13 Sep 2016 07:36:53 -0700 (PDT) Received: from s172.web-hosting.com (s172.web-hosting.com [68.65.122.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6B9E312B65F for ; Tue, 13 Sep 2016 06:57:27 -0700 (PDT) Received: from [146.96.19.240] (port=29230 helo=[10.10.202.53]) by server172.web-hosting.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.86_1) (envelope-from ) id 1bjoD6-003mvB-7T; Tue, 13 Sep 2016 09:57:20 -0400 Content-Type: multipart/alternative; boundary="Apple-Mail=_E628544F-EABC-43FD-B1C0-1CB347D6C966" Mime-Version: 1.0 (Mac OS X Mail 9.2 \(3112\)) From: Dave Rice In-Reply-To: Date: Tue, 13 Sep 2016 09:57:14 -0400 Message-Id: <4BA8D689-137D-4B26-8C39-2C76F4A6CCDD@dericed.com> References: To: Nick Krabbenhoeft X-Mailer: Apple Mail (2.3112) X-OutGoing-Spam-Status: No, score=-2.9 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Cc: cellar@ietf.org Subject: Re: [Cellar] Standardizing FLAC X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Sep 2016 14:36:59 -0000 --Apple-Mail=_E628544F-EABC-43FD-B1C0-1CB347D6C966 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hi Nick, > On Sep 13, 2016, at 9:20 AM, Nick Krabbenhoeft = wrote: >=20 > What's the status of standardizing FLAC? AFAIK there's no progress (yet) towards the CELLAR goals on FLAC. > I work at NYPL where we are using WAVE32 for our audio preservation = masters. However, WAVE32's have a 4GB size limit, which means that long = media like 2+ hour open-reel recordings are split into part files. >=20 > I'm looking at FLAC as a potential solution to my container problem, = and I would be happy to help with standardization efforts. FLAC doesn't have a limit in that area and includes a lot of other = preservation features: checksum of decoded audio in header, audio frames = with crcs, ability to store any WAV metadata of the source file via = -keep-foreign-metadata. I wrote up some preservation considerations of = flac here . If you'd like to contribute I suggest this as first steps: Should check with FLAC community whether it's preferable to work in = https://github.com/xiph/flac or possibly start a new repo such as = https://github.com/xiph/flac-specification so that the specification = work can be separated into a more specific repository. Re-format existing document: The existing FLAC spec is in html at = https://github.com/xiph/flac/blob/master/doc/html/format.html which is = rendered at https://xiph.org/flac/format.html. With both FFV1, EBML, and = Matroska we first re-formatted the existing specifications from their = original state to markdown (FFV1 was lyx to markdown, EBML and Matroska = were HTML to markdown). We have a markdown to rfc process working in the = other repos of the working group and the approach could be copied to = FLAC. Produce a Makefile to render the flac spec markdown into rfc form as = well as whatever other formats that the community needs. See the version = for FFV1 and EBML = .= Once the FLAC spec is in markdown, it could likely use a review and = update of RFC2119 terminology and new needed sections for an RFC such as = an Abstract and Security Considerations. At the IETF96 CELLAR working group meeting I volunteered to start the = steps listed above, but have been lately focused on EBML, so help or = volunteering on FLAC would be greatly appreciated. [...] Best Regards, Dave Rice --Apple-Mail=_E628544F-EABC-43FD-B1C0-1CB347D6C966 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii
Hi Nick,

On Sep 13, 2016, at 9:20 AM, = Nick Krabbenhoeft <nickkrabbenhoeft@nypl.org> wrote:

What's the status of standardizing = FLAC?

AFAIK = there's no progress (yet) towards the CELLAR goals on FLAC.

I work at NYPL where we are using = WAVE32 for our audio preservation masters. However, WAVE32's have a 4GB = size limit, which means that long media like 2+ hour open-reel = recordings are split into part files.

I'm looking at FLAC as a potential = solution to my container problem, and I would be happy to help with = standardization efforts.

FLAC doesn't have a limit in that area and = includes a lot of other preservation features: checksum of decoded audio = in header, audio frames with crcs, ability to store any WAV metadata of = the source file via -keep-foreign-metadata. I wrote up some preservation = considerations of flac here.

If you'd like = to contribute I suggest this as first steps:

Should check with FLAC community whether it's = preferable to work in https://github.com/xiph/flac or possibly start a new repo = such as https://github.com/xiph/flac-specification so that the = specification work can be separated into a more specific = repository.

Re-format existing = document: The existing FLAC spec is in html at https://github.com/xiph/flac/blob/master/doc/html/format.html which is rendered at https://xiph.org/flac/format.html. With both FFV1, EBML, = and Matroska we first re-formatted the existing specifications from = their original state to markdown (FFV1 was lyx to markdown, EBML and = Matroska were HTML to markdown). We have a markdown to rfc process = working in the other repos of the working group and the approach could = be copied to FLAC.

Produce a = Makefile to render the flac spec markdown into rfc form as well as = whatever other formats that the community needs. See the version = for FFV1 and EBML.

Once the = FLAC spec is in markdown, it could likely use a review and update of = RFC2119 terminology and new needed sections for an RFC such as an = Abstract and Security Considerations.

At the IETF96 CELLAR working group meeting I = volunteered to start the steps listed above, but have been lately = focused on EBML, so help or volunteering on FLAC would be greatly = appreciated.

[...]

Best Regards,
Dave Rice

= --Apple-Mail=_E628544F-EABC-43FD-B1C0-1CB347D6C966-- From nobody Tue Sep 13 07:39:28 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A5E7B12B512 for ; Tue, 13 Sep 2016 07:39:26 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.699 X-Spam-Level: X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OcqOHgZW-raP for ; Tue, 13 Sep 2016 07:39:20 -0700 (PDT) Received: from mail-it0-x22a.google.com (mail-it0-x22a.google.com [IPv6:2607:f8b0:4001:c0b::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BF6A112B515 for ; Tue, 13 Sep 2016 06:59:20 -0700 (PDT) Received: by mail-it0-x22a.google.com with SMTP id o3so38220961ita.1 for ; Tue, 13 Sep 2016 06:59:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=aQxNqh1awvy5AetcBtmBuojEWQimL7Vw0oQPV6hB7hU=; b=oF9SjUnboTtaS0UQtShnAAZLyJIj0nJNfOvB7j46Camkp0RSsllmU2mVVkW+EZvQg0 P5wg1cu+IxMKl5Ir3BCKo13+/ZBePsTp8tZjd7/S1mF6v7L645uzCw+nTYFm5fyJzB+X 3f6xnZNL8qkURdeYotwMuWLRKJ13rutu7Nghz8urZCcAyMu5oxqWc3903CDQ85MFk0C7 iJO2Jg7kv+pxYcNzMlrWQKUGI2Kc+bszbenBC7wKpR9+hMNv6px8N17iHKo1FFPxvl/y /u5K0D6MGXtYqnt3oYulNgxaB2hWY7tuLXRvYaZV+lC2bjNKtHy2f6BZIJAmZUjJKhvS WyMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=aQxNqh1awvy5AetcBtmBuojEWQimL7Vw0oQPV6hB7hU=; b=jksTKbIp8JY+11RTyozb+3CltvIGb4CNHw2UgYOyi+qXmP1DYvr/jjG7JkgcmGYjRU kNgZIIRLEf0dzpXAL2SiZnOvXfBF5FJczNVN8VIZnqtALrGQKe+c47swO4V6yv3T9S6G mST7XQshavM0imwkKv75mOHjcXKAKG6EE0pSkZ8h9nMnJJZGNaWUuQ0II+6ZyhWJcQ6x UD8C46+iZHfadyGEh0gaPyVk6zkdSqfbhiYuxXrn6nceQcsIIFg8hD5R5uvIHu27fJ8I 6TqmoCROVl2xgKwWGUj3rBC94synBqqn4+wtQ9Wb1o/Y/z858g54W92jpdMSZ9KAGcv5 j3Qw== X-Gm-Message-State: AE9vXwOMVrgsufudK1JFz4mVUEUoAhjm5NRdsgZ2m335h0iAISOtcc6KijIBYR6me2p4D5clyCZyqdyFy0a6sg== X-Received: by 10.107.132.28 with SMTP id g28mr1886452iod.11.1473775159562; Tue, 13 Sep 2016 06:59:19 -0700 (PDT) MIME-Version: 1.0 Received: by 10.107.40.4 with HTTP; Tue, 13 Sep 2016 06:58:39 -0700 (PDT) From: Tessa Fallon Date: Tue, 13 Sep 2016 09:58:39 -0400 Message-ID: To: cellar@ietf.org Content-Type: multipart/alternative; boundary=001a113f377add67fb053c640427 Archived-At: Subject: [Cellar] IETF current milestones: request for feedback X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Sep 2016 14:39:27 -0000 --001a113f377add67fb053c640427 Content-Type: text/plain; charset=UTF-8 At the CELLAR meeting in July, it was decided that working group milestones would need to be revised. The current list is below. Would those working on FFV1, EBML, and Matroska weigh in on the state of the specifications and projected timelines for work to be completed? I note that the most recent changes to EBML were submitted on Sunday 9/11. Where does this leave the document? Much thanks, Tessa Date Dec 2016 Submit specification for FLAC audio codec to IESG (Standards Track) Sep 2016 Submit specification for FFV1 video codec version 4 to IESG (Standards Track) Jul 2016 Submit specification for Matroska container format version 4 to IESG (Standards Track) Jul 2016 Submit informational specification for FFV1 video codec versions 0, 1 and 3 to IESG for publication Jun 2016 Submit specification for EBML to IESG (Standards Track) Jun 2016 Submit informational specification for Matroska container format versions 1, 2 and 3 to IESG for publication --001a113f377add67fb053c640427 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
At the CELLAR meeting in July, it was decided that working= group milestones would need to be revised. =C2=A0 The current list is belo= w.

Would those working on FFV1, EBML, and Matroska weigh= in on the state of the specifications and projected timelines for work to = be completed?=C2=A0 I note that the most recent changes to EBML were submit= ted on Sunday 9/11.=C2=A0 Where does this leave the document?=C2=A0

Much thanks,
Tessa=C2=A0

Date
Dec 2016 Submit specification for FLAC audio codec to IESG = (Standards Track)
Sep 2016 Submit specification for FFV1 video codec v= ersion 4 to IESG (Standards Track)
Jul 2016 Submit specification for M= atroska container format version 4 to IESG (Standards Track)
Jul = 2016 = Submit informational specification for FFV1 video codec versions 0, 1 and 3= to IESG for publication
Jun 2016 Submit specification for EBML to IES= G (Standards Track)
Jun 2016 Submit informational specification for Ma= troska container format versions 1, 2 and 3 to IESG for publication
--001a113f377add67fb053c640427-- From nobody Tue Sep 13 07:53:41 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7BD6612B595 for ; Tue, 13 Sep 2016 07:53:40 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.699 X-Spam-Level: X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nlMNXFH4fNrL for ; Tue, 13 Sep 2016 07:53:37 -0700 (PDT) Received: from mail-yw0-x22d.google.com (mail-yw0-x22d.google.com [IPv6:2607:f8b0:4002:c05::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B798812B3F2 for ; Tue, 13 Sep 2016 07:21:12 -0700 (PDT) Received: by mail-yw0-x22d.google.com with SMTP id i129so33124037ywb.0 for ; Tue, 13 Sep 2016 07:21:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=ldBowI1DwvvyUcy3LzdmQ1TfRAyexyrbuzjSo6FPVyA=; b=g7iJIGP2+SB7MS4EIMBnQsQgnnPA68Tnys+0KvvXAVkbkt/0GUzm9PJg+K1kd5f4Di 7NuTNpWPoyAjU2asBsh58wzhRfi4X+YvN+bAdnZ3zLoUhUtJxmmWnr7Wd3pUbCEO29/w W2/JH4f3lA/lddMcFIx9LcwR9EMpm2rnol69RWyDaLWW3Qf9K1RwtU6pJihcmPLObn5s TgJNHq8rULFxqw95cFMYHTYS8G1liHtVwUklBOp3WXKRgy5mLICwOmMUAJnpkf+W4OjL bScgt8/IHR2Fw7NHWyNBB1zf8uwB45nlji3El/bz0STRxKKrlf10S0oAvgpvkNj+QO8N g4vQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=ldBowI1DwvvyUcy3LzdmQ1TfRAyexyrbuzjSo6FPVyA=; b=XPXZMhTRT6nMUPIfr6TD1CKZoo2B2wT+Swh9cw3CzbO6XlDyioAdnetc1UFOjoamTa mQ7IHKQX53P/LJ7O10B/8d0k0+vOBMM969FGufjqkjNgAWrWAgZSaDtXy1MjPNn+Nbon sZdefY0+II2+WV8dkaZH6hSY025PoEUPI9B0MW11Y62Owe1fi8JYFPwtXzCmqpakNDga aGo+5l+/23G68vQ92AUC9Taoac+rufggLd9vOvx/Y7svHFFfsHLvtgRGNx4nFoAoBZvj mXHSALer1UxBkDKSgr8bhlCAAZO7tWJHBrURZ46AoLQYUWDNXh6nizmAjm6i2vztPj2G 5GLw== X-Gm-Message-State: AE9vXwMVVwjJLA1iXQBzCJ1hpERK0JKup+8vbvfEeeflsUXbGjH3ZD6VVWkEGoGF88il8ssUMJcrQB9oykMBdg== X-Received: by 10.129.111.8 with SMTP id k8mr895370ywc.235.1473776471734; Tue, 13 Sep 2016 07:21:11 -0700 (PDT) MIME-Version: 1.0 Received: by 10.129.116.3 with HTTP; Tue, 13 Sep 2016 07:21:11 -0700 (PDT) In-Reply-To: References: From: Nithin Mathew Kurien Date: Tue, 13 Sep 2016 19:51:11 +0530 Message-ID: To: Nick Krabbenhoeft Content-Type: multipart/alternative; boundary=001a1145ec6a138044053c645366 Archived-At: Cc: Codec Encoding for LossLess Archiving and Realtime transmission Subject: Re: [Cellar] Standardizing FLAC X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Sep 2016 14:53:40 -0000 --001a1145ec6a138044053c645366 Content-Type: text/plain; charset=UTF-8 The RF64 file format [1] adopted by EBU is a backward-compatible extension of WAV supporting files of size more than 4 GB. It is currently supported by many programs including ffmpeg and libav. Recently, the ITU has standardised the BW64 file format along with Audio Definition Model [2,3,4,5]. In addition to large file support, it also specifies metadata for immersive and object-based audio. It is being used for encoding next generation immersive audio codecs like MPEG-H 3D Audio, Dolby AC-4 and DTS:X for UHDTV broadcasts. If FLAC is being standardised by CELLAR, IMHO it would be good if it would be able to preserve the metadata in BW64. It may also be necessary to increase the supported channel count of FLAC, which is currently limited to 8. [1] http://tech.ebu.ch/docs/tech/tech3306-2009.pdf [2] https://www.itu.int/rec/R-REC-BS.2088-0-201510-I/en [3] http://www.itu.int/rec/R-REC-BS.2076-0-201506-I [4] https://www.itu.int/rec/R-REC-BS.2094-0-201604-I/en [5] http://www.itu.int/rec/R-REC-BS.2051-0-201402-I On Tue, Sep 13, 2016 at 6:50 PM, Nick Krabbenhoeft < nickkrabbenhoeft@nypl.org> wrote: > What's the status of standardizing FLAC? > > I work at NYPL where we are using WAVE32 for our audio preservation > masters. However, WAVE32's have a 4GB size limit, which means that long > media like 2+ hour open-reel recordings are split into part files. > > I'm looking at FLAC as a potential solution to my container problem, and I > would be happy to help with standardization efforts. > > Best, > Nick > -- > > *Nick Krabbenhoeft | The New York Public Library* > *Head of Digital Preservation* > > Special Collections Administration > 476 Fifth Avenue, New York, NY 10018-2788 > T: (917) 229-9596 | nickkrabbenhoeft@nypl.org > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > > --001a1145ec6a138044053c645366 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
The RF64 file format [1] adopted by EBU is a backward-comp= atible extension of WAV supporting files of size more than 4 GB. It is curr= ently supported by many programs including ffmpeg and libav.

=
Recently, the ITU has standardised the BW64 file format along with Aud= io Definition Model [2,3,4,5]. In addition to large file support, it also s= pecifies metadata for immersive and object-based audio. It is being used fo= r encoding next generation immersive audio codecs like MPEG-H 3D Audio, Dol= by AC-4 and DTS:X for UHDTV broadcasts. If FLAC is being standardised by CE= LLAR, IMHO it would be good if it would be able to preserve the metadata in= BW64. It may also be necessary to increase the supported channel count of = FLAC, which is currently limited to 8.

[4]=C2=A0https://www.itu.int/rec/R-REC-BS.2094-0-201604-I/en


On Tue, Sep 13= , 2016 at 6:50 PM, Nick Krabbenhoeft <nickkrabbenhoeft@nypl.org> wrote:
What= 's the status of standardizing FLAC?

I work at NYPL = where we are using WAVE32 for our audio preservation masters. However, WAVE= 32's have a 4GB size limit, which means that long media like 2+ hour op= en-reel recordings are split into part files.

I= 9;m looking at FLAC as a potential solution to my container problem, and I = would be happy to help with standardization efforts.

Best,
_______________________________________________
Cellar mailing list
Cellar@ietf.org
https://www.ietf.org/mailman/listinfo/cellar

--001a1145ec6a138044053c645366-- From nobody Tue Sep 13 07:59:56 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 93C5B12B4F8 for ; Tue, 13 Sep 2016 07:59:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.699 X-Spam-Level: X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GbuXaQSLTwOa for ; Tue, 13 Sep 2016 07:59:53 -0700 (PDT) Received: from mail-oi0-x234.google.com (mail-oi0-x234.google.com [IPv6:2607:f8b0:4003:c06::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1574F12B645 for ; Tue, 13 Sep 2016 07:35:00 -0700 (PDT) Received: by mail-oi0-x234.google.com with SMTP id w11so13215453oia.2 for ; Tue, 13 Sep 2016 07:35:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=8cmYfRtr1CP44QgLvTt0dxq0SUpVuFktKe4Sp4+A9qU=; b=eNdDTvUuYEXV6gDUeoi+v5vHty0hliLvc91+7Ce5l2WKMHbdDnuwSP07ReUm67KaO5 dWEmu9Z9zenEWAzTfaWUm8mDKhZ6XatfvfN8YRlQ3j3Ctmb6PHJFDHB0DL6+dJPfJeJS ZydlQkLvX52MDMm1mn1haaQ1KS2q64LAhPy87huFpMDN3hLuyRpiQBG1r18VDgE2Qi55 CRLRMntUL7beyh45BpLgs3Fyj84w25pmo0AZ4jn9EHA/+SsFy6uxuj0lhP/1JyrbNzz6 zgLfwt7I4zaZlNS8HV7sTPaeb4rnnYrdJ6A7rVh8fczqRfndEjRM+V5PLeVI6DYOAHDz It5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=8cmYfRtr1CP44QgLvTt0dxq0SUpVuFktKe4Sp4+A9qU=; b=iM6b1DjBH7SylTOa37J5rVUlP+hywtvS2HCyI9KVx72MVh4N4TZXV/Ghivhs7UILIV 4vJDLhYWLPoJhs4J007bhw4qGuBymfQjGN57Jt/ZJFjK6kIWCzxw70BhJo7IZHCGmjQX A9YsL9Svg40sRFPN1yJZV0rjG3Q2ovh+nUhnsizszMDIiwYCCG2nvWHw1hCoKlEtlhGk PGJ6R1Lvg4/8Npvca3AsBlf2Zr4eBd3nPONS+i0OKVAqQWmmHt+1D0eRztK6agDP2DHl 1RBNH+xhmvhA/20nhAtwo0xfz8ELAHz7q9vP0fhRY+JhtV5ZOMvBoaiWhKasDA6jo2lF 0HNA== X-Gm-Message-State: AE9vXwM5EBYRAf22BTcPRo+mGinWJUg2x5lckJpfZmH7dHl4SkuN2rSNFZRTr5RrzRXUphbT3qqzUYue6nUjuQ== X-Received: by 10.107.138.200 with SMTP id c69mr2383954ioj.43.1473777299312; Tue, 13 Sep 2016 07:34:59 -0700 (PDT) MIME-Version: 1.0 Received: by 10.79.6.146 with HTTP; Tue, 13 Sep 2016 07:34:58 -0700 (PDT) In-Reply-To: References: From: Ashley Blewer Date: Tue, 13 Sep 2016 10:34:58 -0400 Message-ID: To: Tessa Fallon Content-Type: multipart/alternative; boundary=001a113e8268677470053c6484d2 Archived-At: Cc: Nick Krabbenhoeft , cellar@ietf.org Subject: Re: [Cellar] Standardizing FLAC X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Sep 2016 14:59:55 -0000 --001a113e8268677470053c6484d2 Content-Type: text/plain; charset=UTF-8 All, Is anyone within CELLAR currently also involved in the FLAC development community, or know of a particular person to reach out to? I am also interested and can facilitate bridging this gap by posting to their developer's list, et al, but maybe someone else already has an established relationship, or this may be something suitable for the CELLAR chairs. Ashley On Tue, Sep 13, 2016 at 9:52 AM, Tessa Fallon wrote: > HI Nick, > > To date there has not been much discussion of FLAC as efforts have been > focused on FFV1 and Matroska. If you're interested in moving this forward, > you're welcome to initiate the discussion, We did get initial buy-in from > the FLAC developer community when we chartered the working group. I'm not > aware of any crossover between that group and CELLAR at the current time, > and if that is really the case I'd very much like it to change. Happy to > answer any questions. > > Best, > > Tessa > > On Tue, Sep 13, 2016 at 9:20 AM, Nick Krabbenhoeft < > nickkrabbenhoeft@nypl.org> wrote: > >> What's the status of standardizing FLAC? >> >> I work at NYPL where we are using WAVE32 for our audio preservation >> masters. However, WAVE32's have a 4GB size limit, which means that long >> media like 2+ hour open-reel recordings are split into part files. >> >> I'm looking at FLAC as a potential solution to my container problem, and >> I would be happy to help with standardization efforts. >> >> Best, >> Nick >> -- >> >> *Nick Krabbenhoeft | The New York Public Library* >> *Head of Digital Preservation* >> >> Special Collections Administration >> 476 Fifth Avenue, New York, NY 10018-2788 >> T: (917) 229-9596 | nickkrabbenhoeft@nypl.org >> >> _______________________________________________ >> Cellar mailing list >> Cellar@ietf.org >> https://www.ietf.org/mailman/listinfo/cellar >> >> > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > > --001a113e8268677470053c6484d2 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
All,

Is anyone within CELLAR currently = also involved in the FLAC development community, or know of a particular pe= rson to reach out to? I am also interested and can facilitate bridging this= gap by posting to their developer's list, et al, but maybe someone els= e already has an established relationship, or this may be something suitabl= e for the CELLAR chairs.

Ashley

On Tue, Sep 13, 2016 at = 9:52 AM, Tessa Fallon <tessa.fallon@gmail.com> wrote:
HI Nick,

To date there has not been much discussion of FLAC as efforts have been = focused on FFV1 and Matroska.=C2=A0 If you're interested in moving this= forward, you're welcome to initiate the discussion, =C2=A0We did get i= nitial buy-in from the FLAC developer community when we chartered the worki= ng group. I'm not aware of any crossover between that group and CELLAR = at the current time, and if that is really the case I'd very much like = it to change.=C2=A0 Happy to answer any questions.

Best,

Tessa

On Tue, Sep 13, 2016 at 9:20 AM= , Nick Krabbenhoeft <nickkrabbenhoeft@nypl.org> wrot= e:
What's the status of standardizing FLAC?

I work at NYPL where we are using WAVE32 for our audio preservation mast= ers. However, WAVE32's have a 4GB size limit, which means that long med= ia like 2+ hour open-reel recordings are split into part files.
<= br>
I'm looking at FLAC as a potential solution to my contain= er problem, and I would be happy to help with standardization efforts.

Best,
Nick=C2=A0
--

Nick Krabbenhoeft | The New York Public Library
Head of Digital = Preservation

Special Collections Administration
476 Fifth Ave= nue, New York, NY 10018-2788
T:=C2=A0(917) 229-9596=C2=A0|=C2=A0nickkrabbenhoeft@nypl.org

<= /div>

_______________________________________________
Cellar mailing list
Cellar@ietf.org https://www.ietf.org/mailman/listinfo/cellar


_______________________________________________
Cellar mailing list
Cellar@ietf.org
https://www.ietf.org/mailman/listinfo/cellar

--001a113e8268677470053c6484d2-- From nobody Tue Sep 13 08:30:33 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BDEFB12B3BA for ; Tue, 13 Sep 2016 08:30:30 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.7 X-Spam-Level: X-Spam-Status: No, score=-2.7 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=nypl.org Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id owEiYiWeTe_S for ; Tue, 13 Sep 2016 08:30:25 -0700 (PDT) Received: from mail-vk0-x230.google.com (mail-vk0-x230.google.com [IPv6:2607:f8b0:400c:c05::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6575D12B42C for ; Tue, 13 Sep 2016 07:57:55 -0700 (PDT) Received: by mail-vk0-x230.google.com with SMTP id f76so182937827vke.0 for ; Tue, 13 Sep 2016 07:57:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nypl.org; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=j1sGlxMMdNmpTvQ2RpU0UOrRYWTuEKshuhovRlbxS2w=; b=VxzBPwaTDzl796eurilkaMMIrU3Clpgu+/s1tIHryNkdtvPtqyszlB/74Xfi+Y2dIz ahofaTYI8R/qEMNJ3Jb1CBIlcLvm1COIJlKtmLnFzP71R4vnxIRvpz5HeJ6Z6hCJGWkq KOZCjWTq/GNjsrl8BlhRJzy13UqbyU/7xGO71D+b+kV7vGc+tMJwtHgfge+kKtvZ0sDN Aj08z4PpxwByTMRJV6rGmtwiCoK2OFPzL3J0g+Bc2QXMsiqMo9n9dg9YO+NkDK+VpVou f4/J8hIu+Dhk2DaOTnDCzo4jEDkt398sWgXCv0V2xYpirqO0d0Xw1WlBESIpBzWGXFyh Ebkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=j1sGlxMMdNmpTvQ2RpU0UOrRYWTuEKshuhovRlbxS2w=; b=cfJ19m8L3M33+NTpW334jd6CmNkhF9dRTYJgNWfzPh2Y8pt+0318GF9NTy/vrhfdrP XCLgx8cZz4oFKlPkoXbAsOpClHOIKzRZe+YvFWmBemv2nmyLxpBaG62crWLLRWG76kyp 1lP46uY5xNy7yr3bEZo8Aff5r9CJMNdqso1s8+UZWXVN4zYVqx/9XALwKU7j2xyWg1I6 6jnYJMmxMzPFVv5JHM/Ft3VPEeyUw2p6DZj45fnTX6vskREDcj5BRHB4tKH068sp6jHO ESM1MkLPGPZrQgoiXNvSSkhEFyK9cFf53UR1AF20dFow2CELB86NCUfNsJ93Z1OKXHRS KYgg== X-Gm-Message-State: AE9vXwOmorxv82/MZkq4PjkCcvsiOROC18HoH2UyO9CmbCrtApwsTzUg8MDa9ric/UNvNBmgLWfE9vL6NDp0KiIq X-Received: by 10.31.76.2 with SMTP id z2mr1156169vka.105.1473778674368; Tue, 13 Sep 2016 07:57:54 -0700 (PDT) MIME-Version: 1.0 Received: by 10.176.2.21 with HTTP; Tue, 13 Sep 2016 07:57:34 -0700 (PDT) In-Reply-To: <4BA8D689-137D-4B26-8C39-2C76F4A6CCDD@dericed.com> References: <4BA8D689-137D-4B26-8C39-2C76F4A6CCDD@dericed.com> From: Nick Krabbenhoeft Date: Tue, 13 Sep 2016 10:57:34 -0400 Message-ID: To: Dave Rice Content-Type: multipart/alternative; boundary=001a114db7d25d1c07053c64d6e8 Archived-At: Cc: cellar@ietf.org Subject: Re: [Cellar] Standardizing FLAC X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Sep 2016 15:30:31 -0000 --001a114db7d25d1c07053c64d6e8 Content-Type: text/plain; charset=UTF-8 Dave, I'm happy to help with what you outlined. I think Ashley volunteered to contact flac-dev, but I can get started on html->md. Nick On Tue, Sep 13, 2016 at 9:57 AM, Dave Rice wrote: > Hi Nick, > > On Sep 13, 2016, at 9:20 AM, Nick Krabbenhoeft > wrote: > > What's the status of standardizing FLAC? > > > AFAIK there's no progress (yet) towards the CELLAR goals on FLAC. > > I work at NYPL where we are using WAVE32 for our audio preservation > masters. However, WAVE32's have a 4GB size limit, which means that long > media like 2+ hour open-reel recordings are split into part files. > > I'm looking at FLAC as a potential solution to my container problem, and I > would be happy to help with standardization efforts. > > > FLAC doesn't have a limit in that area and includes a lot of other > preservation features: checksum of decoded audio in header, audio frames > with crcs, ability to store any WAV metadata of the source file via > -keep-foreign-metadata. I wrote up some preservation considerations of flac > here . > > If you'd like to contribute I suggest this as first steps: > > Should check with FLAC community whether it's preferable to work in > https://github.com/xiph/flac or possibly start a new repo such as > https://github.com/xiph/flac-specification so that the specification work > can be separated into a more specific repository. > > Re-format existing document: The existing FLAC spec is in html at > https://github.com/xiph/flac/blob/master/doc/html/format.html which is > rendered at https://xiph.org/flac/format.html. With both FFV1, EBML, and > Matroska we first re-formatted the existing specifications from their > original state to markdown (FFV1 was lyx to markdown, EBML and Matroska > were HTML to markdown). We have a markdown to rfc process working in the > other repos of the working group and the approach could be copied to FLAC. > > Produce a Makefile to render the flac spec markdown into rfc form as well > as whatever other formats that the community needs. See the version for > FFV1 and EBML > . > > Once the FLAC spec is in markdown, it could likely use a review and update > of RFC2119 terminology and new needed sections for an RFC such as an > Abstract and Security Considerations. > > At the IETF96 CELLAR working group meeting I volunteered to start the > steps listed above, but have been lately focused on EBML, so help or > volunteering on FLAC would be greatly appreciated. > > [...] > > Best Regards, > Dave Rice > > -- *Nick Krabbenhoeft | The New York Public Library* *Head of Digital Preservation* Special Collections Administration 476 Fifth Avenue, New York, NY 10018-2788 T: (917) 229-9596 | nickkrabbenhoeft@nypl.org *Inspiring Lifelong Learning* | *Advancing Knowledge* | *Strengthening Our Communities* --001a114db7d25d1c07053c64d6e8 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Dave,

I'm happy to help with what y= ou outlined. I think Ashley volunteered to contact flac-dev, but I can get = started on html->md.

Nick

On Tue, Sep 13, 2016 at 9:5= 7 AM, Dave Rice <dave@dericed.com> wrote:
Hi Nick,

<= div>
On Sep 13, 2016, at 9:2= 0 AM, Nick Krabbenhoeft <nickkrabbenhoeft@nypl.org> wrote:

What's the status of standardizing FLAC?

AFAIK there's no progress (yet) towar= ds the CELLAR goals on FLAC.

I work at NYPL where we are using WAVE32 f= or our audio preservation masters. However, WAVE32's have a 4GB size li= mit, which means that long media like 2+ hour open-reel recordings are spli= t into part files.

I'm looking at FLAC as a po= tential solution to my container problem, and I would be happy to help with= standardization efforts.

FLAC doesn't have a limit in that area and includes a lot of ot= her preservation features: checksum of decoded audio in header, audio frame= s with crcs, ability to store any WAV metadata of the source file via -keep= -foreign-metadata. I wrote up some preservation considerations of flac=C2= =A0here.

If you'd like to contribute I su= ggest this as first steps:

Should check with = FLAC community whether it's preferable to work in https://github.com/xiph/flac or p= ossibly start a new repo such as https://github.com/xiph/flac-specificat= ion so that the specification work can be separated into a more specifi= c repository.

Re-format existing document: T= he existing FLAC spec is in html at=C2=A0https://github.co= m/xiph/flac/blob/master/doc/html/format.html which is rendere= d at=C2=A0h= ttps://xiph.org/flac/format.html. With both FFV1, EBML, and Matros= ka we first re-formatted the existing specifications from their original st= ate to markdown (FFV1 was lyx to markdown, EBML and Matroska were HTML to m= arkdown). We have a markdown to rfc process working in the other repos of t= he working group and the approach could be copied to FLAC.

Produce a Makefile to render the flac spec markdown into rfc form = as well as whatever other formats that the community needs. See the version= for=C2=A0FFV1=C2=A0and=C2=A0EBML= .

Once the FLAC spec is in markdown, it could like= ly use a review and update of RFC2119 terminology and new needed sections f= or an RFC such as an Abstract and Security Considerations.

At the IETF96 CELLAR working group meeting I volunteered to start = the steps listed above, but have been lately focused on EBML, so help or vo= lunteering on FLAC would be greatly appreciated.

[= ...]

Best Regards,
Dave Rice
<= br>



--
=

Nick Krabbenhoeft | The New York Public Library
Head of Digital Preservation

Special Collections = Administration
476 Fifth Avenue, New York, NY 10018-2788
T:=C2=A0(917= ) 229-9596=C2=A0|=C2=A0nickkrabbenhoeft@ny= pl.org

Inspiring Lifelong Learning | Advancing Knowledge | Strengthening Our Communities

=
--001a114db7d25d1c07053c64d6e8-- From nobody Tue Sep 13 09:01:29 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 377EF12B4E2 for ; Tue, 13 Sep 2016 09:01:27 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.439 X-Spam-Level: X-Spam-Status: No, score=-2.439 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, HTML_OBFUSCATE_05_10=0.26, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id clg9Irf_vwC1 for ; Tue, 13 Sep 2016 09:01:19 -0700 (PDT) Received: from mail-it0-x236.google.com (mail-it0-x236.google.com [IPv6:2607:f8b0:4001:c0b::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A30CE12B500 for ; Tue, 13 Sep 2016 08:34:56 -0700 (PDT) Received: by mail-it0-x236.google.com with SMTP id o3so1598893ita.1 for ; Tue, 13 Sep 2016 08:34:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=tfyz50nQZOQvSy9A2J6IzagHdcbALPhFJ7ivpj8lZ48=; b=eagAWf4HZRUdBwpuGPRL2gMMtYJRxg5LP/uW24KqgaWDfVWZp2LOzj7cII9ej3GQ43 n9EH57Ijn0hAsLeyFYwv18t6YNWJTTrGh8LSu7+XI/7UgZOz8SaaFPDJYtoR8S7bgsJQ OE4o2UICgP8MOl2gsNW9Ynw5jDBWtltb/ACV1h/IHanYE4HD5dIdqUgPFWAKUFvWUBUn MKzO24WBfeXJLoXy6DcfjIKpyTx7DDzecenUUjo8HfioeJYsryXXq8DJT1u0gu5V+BD3 BfWvBZhSjSeaK02H2vdzK8dzzuSUYTtMQD/up9SzOFC/4g6ly3KZAqCyjYdZ3HSDnkjK Hrlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=tfyz50nQZOQvSy9A2J6IzagHdcbALPhFJ7ivpj8lZ48=; b=W27uFFT4HY6i/1S+9ek0EIMsXnwbPBhlmSi2FfoxVRXWt3W1FZiDF46Dm/zcjqjtTM xlovJqvIOYse43S/1WiIJ7RTfB7GijEJtHVRsDeoPJxrlp90wcrnVmrX1szMUldUP5Kt uzFM/C/LgiqjukdnSGNDdg7/DxIIv+9u37WvilsfQa5sfAFEYbse3CDTJbGiEYz1MpNY UJRaoqzqSdlIYCiqUqSVePq8T+wuQQ6nM2wtOoncLJ56ELe6fEhvcnKH9vY+V3Wq/+su BTPyZO100TWj+UGd6SLbpumlcdVnfNr9QU+BCRZuu0jMR0yUk3bGIVRubk5L9vqJ+x3s HsXw== X-Gm-Message-State: AE9vXwM2O21Hfm1cLIwug7NvI12IkkMUZ92cZEXd7RPfFUkDas/UOiyF73eK6xp7OWOrrINbKGHs8tsUrKAziw== X-Received: by 10.107.186.138 with SMTP id k132mr2490996iof.84.1473780895766; Tue, 13 Sep 2016 08:34:55 -0700 (PDT) MIME-Version: 1.0 Received: by 10.107.40.4 with HTTP; Tue, 13 Sep 2016 08:34:15 -0700 (PDT) From: Tessa Fallon Date: Tue, 13 Sep 2016 11:34:15 -0400 Message-ID: To: cellar@ietf.org Content-Type: multipart/alternative; boundary=94eb2c07ffa4c4e6ca053c655a68 Archived-At: Subject: [Cellar] New thread: Work to be done on FLAC, call for volunteers X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Sep 2016 16:01:27 -0000 --94eb2c07ffa4c4e6ca053c655a68 Content-Type: text/plain; charset=UTF-8 In an effort to be more co-chairperson-like: Moving this from previous thread: https://mailarchive.ietf.org/arch/msg/cellar/NW8nx-cgwYvaQexUxD3d7_YxEDk If you're interested in working on FLAC, please respond to this thread. If you're interested in possibly/maybe/no commitment spearheading the work on FLAC (helping keep work on task, meeting deadlines, possible document shepherding), please say so as well. Is there an outline of the markdown to rfc process that is easily accessible for others? >From Dave Rice's email: If you'd like to contribute I suggest this as first steps: - Should check with FLAC community whether it's preferable to work in https://github.com/xiph/flac or possibly start a new repo such as https://github.com/xiph/flac-specification so that the specification work can be separated into a more specific repository. - Re-format existing document: The existing FLAC spec is in html at https://github.com/xiph/flac/blob/master/doc/html/format.html which is rendered at https://xiph.org/flac/format.html. With both FFV1, EBML, and Matroska we first re-formatted the existing specifications from their original state to markdown (FFV1 was lyx to markdown, EBML and Matroska were HTML to markdown). We have a markdown to rfc process working in the other repos of the working group and the approach could be copied to FLAC. - Produce a Makefile to render the flac spec markdown into rfc form as well as whatever other formats that the community needs. See the version for FFV1 and EBML . - Once the FLAC spec is in markdown, it could likely use a review and update of RFC2119 terminology and new needed sections for an RFC such as an Abstract and Security Considerations. --94eb2c07ffa4c4e6ca053c655a68 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
In an effort to be more co= -chairperson-like:


If you're interested in working on FLAC, please respond to t= his thread.=C2=A0 If you're interested in possibly/maybe/no commitment = spearheading the work on FLAC (helping keep work on task, meeting deadlines= , possible document shepherding), please say so as well.

Is there an ou= tline of the markdown to rfc process that is easily accessible for others?<= /div>

From Dave Rice's email:

If you'd like to contribute I sugge= st this as first steps:

  • Should check with FLAC community whether it's preferabl= e to work in=C2=A0https://github.com/xiph/flac=C2=A0or possibly start a new repo such as=C2=A0<= /span>https://github.com/xiph/flac-specificat= ion=C2=A0so that the specification wor= k can be separated into a more specific repository.
  • Re-format existing document: The existing FLAC spe= c is in html at=C2=A0htt= ps://github.com/xiph/flac/blob/master/doc/html/format.html=C2=A0which is rendered at=C2=A0https://xiph.org/flac/format.html. With both FFV1, EBML, and Matroska we first re-formatted the ex= isting specifications from their original state to markdown (FFV1 was lyx t= o markdown, EBML and Matroska were HTML to markdown). We have a markdown to= rfc process working in the other repos of the working group and the approa= ch could be copied to FLAC.
  • Produce a Makefile to render the flac spec markdown into rfc form as well = as whatever other formats that the community needs. See the version for=C2= =A0FFV1=C2=A0and=C2=A0EBML.
  • Once the FLAC spec is in markdown, it could like= ly use a review and update of RFC2119 terminology and new needed sections f= or an RFC such as an Abstract and Security Considerations.
    • <= /ul>

--94eb2c07ffa4c4e6ca053c655a68-- From nobody Tue Sep 13 09:41:18 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D598512B1C5 for ; Tue, 13 Sep 2016 09:41:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.699 X-Spam-Level: X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=johnwarburton.net Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bnScEvmBtp2z for ; Tue, 13 Sep 2016 09:41:12 -0700 (PDT) Received: from mail-wm0-x234.google.com (mail-wm0-x234.google.com [IPv6:2a00:1450:400c:c09::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A2BF312B3CB for ; Tue, 13 Sep 2016 09:41:07 -0700 (PDT) Received: by mail-wm0-x234.google.com with SMTP id b187so4705437wme.1 for ; Tue, 13 Sep 2016 09:41:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=johnwarburton.net; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=x8QCjYSurZETuQJJseKTIriBhA3DwEgcyZhx++QhzHc=; b=Zdkw42ZWjgtNnSTL942BNgSaf2OzuVg4fRhlgDr9VcQ4Fp0mdtDvXw5wlc9Hle1F6l 9mPEQM8AR/d+OHAzH+qWDwuTZykwIF6cN4rh/NQeuPKxMImMWG4dZJkP56D/eeZFwFYG s0wM8MzgFiIeSotMlFQIGgwr1gZEE6MvQ+ZvQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=x8QCjYSurZETuQJJseKTIriBhA3DwEgcyZhx++QhzHc=; b=WlNmT1fHyJwVZF5JSD6paF5HRi3NF+9ri5D4tRBPX3Lp/s3+NV8eHXfm3JNSCDCtJJ /lwYhm3UJtygQz/kc9lztoeblDl66LESq8BgC9peid+d0hP916HJ5uwKy6L/Uyy85wDA S2HAT0zqMa1leIlSPsUNqhy6pw0VE3aSHwRhLGdRYc+TuuzC3es5z7E/5qzhRGA9i4yK uRBRGDOrJS8BeJR8BnumgysCVIS8JTpT/QSKPSIGUmjTGYD02tX6De+0PTfy9x6ctuf+ K8dReC3GK3s0YNhrWAZDDopEGU0+ZI/lzezNPSy+SS3SSPAY4V1wvF9x5fAxB9SpVaQk 6cEA== X-Gm-Message-State: AE9vXwPGaHCP5ntPnR7HUFh7WEcGEr88J6n39gxG4FlHsljl0mAW4r1+j1gWzYKE1j/MpOFWPRhcGUa2F2HXzA== X-Received: by 10.28.137.212 with SMTP id l203mr6323814wmd.26.1473784865869; Tue, 13 Sep 2016 09:41:05 -0700 (PDT) MIME-Version: 1.0 Received: by 10.80.151.220 with HTTP; Tue, 13 Sep 2016 09:41:05 -0700 (PDT) In-Reply-To: References: From: John Warburton Date: Tue, 13 Sep 2016 17:41:05 +0100 Message-ID: To: cellar@ietf.org Content-Type: multipart/alternative; boundary=001a11444d9068437a053c664787 Archived-At: Subject: Re: [Cellar] New thread: Work to be done on FLAC, call for volunteers X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Sep 2016 16:41:17 -0000 --001a11444d9068437a053c664787 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable =E2=80=8B On Tue, Sep 13, 2016 at 4:34 PM, Tessa Fallon wrote: > If you're interested in working on FLAC, please respond to this thread. > If you're interested in possibly/maybe/no commitment spearheading the wor= k > on FLAC (helping keep work on task, meeting deadlines, possible document > shepherding), please say so as well. > > If you'd like to contribute I suggest this as first steps: > > > - =E2=80=8B.... > > > > - =E2=80=8B > Re-format existing document: The existing FLAC spec is in html at > https://github.com/xiph/flac/blob/master/doc/html/format.html > which > is rendered at https://xiph.org/flac/format.html. With both FFV1, > EBML, and Matroska we first re-formatted the existing specifications f= rom > their original state to markdown (FFV1 was lyx to markdown, EBML and > Matroska were HTML to markdown). We have a markdown to rfc process wor= king > in the other repos of the working group and the approach could be copi= ed to > FLAC. > - =E2=80=8B...=E2=80=8B > > > If I can help...? I maintain a binary distribution, compiled from development source, of the FLAC library and many other multimedia utilities (Windows 10, 64-bit only) mainly for our own use here; and am a Tonmeister graduate, about to begin teaching part-time in the same department at Surrey University. Document editing, I can certainly do. Is it a once-only conversion, or would you like me to keep an eye on changes? I update my Git tree from all sources approximately once a day. John Warburton --001a11444d9068437a053c664787 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
=E2=80=8B
On Tue, Sep 13, 2016= at 4:34 PM, Tessa Fallon=C2=A0<tessa.fallon@gmail.com>= =C2=A0wrote:
I= f you're interested in working on FLAC, please respond to this thread.= =C2=A0 If you're interested in possibly/maybe/no commitment spearheadin= g the work on FLAC (helping keep work on task, meeting deadlines, possible = document shepherding), please say so as well.

If you'd l= ike to contribute I suggest this as first steps:
=C2=A0
  • =E2= =80=8B....
    =C2=A0
  • =E2=80=8B
    Re-format existing document= : The existing FLAC spec is in html at=C2=A0https://github.com/xiph/flac/blob/master/doc/html/format.= html=C2=A0which is rendered at=C2= =A0https://xiph.org/flac/format.html. With both FFV1, EBML, and Matroska we first re-= formatted the existing specifications from their original state to markdown= (FFV1 was lyx to markdown, EBML and Matroska were HTML to markdown). We ha= ve a markdown to rfc process working in the other repos of the working grou= p and the approach could be copied to FLAC.
  • =E2=80=8B...=E2=80=8B

If I can help...? I maintain a binary distribution, compiled from dev= elopment source, of the FLAC library and many other multimedia utilities (W= indows 10, 64-bit only) mainly for our own use here; and am a Tonmeister gr= aduate, about to begin teaching part-time in the same department at Surrey = University. Document editing, I can certainly do. Is it a once-only convers= ion, or would you like me to keep an eye on changes? I update my Git tree f= rom all sources approximately once a day.

Jo= hn Warburton
--001a11444d9068437a053c664787-- From nobody Tue Sep 13 09:50:51 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7356212B1C5 for ; Tue, 13 Sep 2016 09:50:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.902 X-Spam-Level: X-Spam-Status: No, score=-1.902 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id i4MEBgqTY3G2 for ; Tue, 13 Sep 2016 09:50:48 -0700 (PDT) Received: from 1.mo173.mail-out.ovh.net (1.mo173.mail-out.ovh.net [178.33.111.180]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 51F4C12B397 for ; Tue, 13 Sep 2016 09:50:48 -0700 (PDT) Received: from player157.ha.ovh.net (b9.ovh.net [213.186.33.59]) by mo173.mail-out.ovh.net (Postfix) with ESMTP id 4E522FF9B77 for ; Tue, 13 Sep 2016 18:50:46 +0200 (CEST) Received: from [192.168.2.101] (p5DDB5F23.dip0.t-ipconnect.de [93.219.95.35]) (Authenticated sender: jerome@francoallemand.eu) by player157.ha.ovh.net (Postfix) with ESMTPSA id E0F31500076 for ; Tue, 13 Sep 2016 18:50:45 +0200 (CEST) To: cellar@ietf.org References: From: Jerome Martinez Message-ID: <2b5aab81-8260-b1a6-96ec-8f882ecef401@mediaarea.net> Date: Tue, 13 Sep 2016 18:50:45 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-Ovh-Tracer-Id: 3343359776626118801 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 0 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrfeeluddrjedtgddutdegucdltddurdefledtrddttddmucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfenuceurghilhhouhhtmecufedttdenuc Archived-At: Subject: Re: [Cellar] Standardizing FLAC X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Sep 2016 16:50:50 -0000 Le 13/09/2016 15:20, Nick Krabbenhoeft a crit : > However, WAVE32's have a 4GB size limit, which means that long media > like 2+ hour open-reel recordings are split into part files. > > I'm looking at FLAC as a potential solution to my container problem (...) Beside all other answers, especially the one from Dave citing some advantages of FLAC, I think that it is not good to say things done by FLAC not better than others. In the case of the 4GB size limit, there are already updates of the original WAV formats for supporting 4GB+ files (RF64 and Wave64) with only small changes in the workflow that saying that we should move to FLAC due to 4GB size limit is not believable. Not citing QuickTime or any other modern container able to support PCM. I would say that RF64 is the solution to your container problem with a very small cost but that you should move to FLAC for other reasons (detection of corrupted data, storage cost improvement...). note that this is the same reason I recommend FFV1 vs uncompressed video (v210...) Jrme From nobody Tue Sep 13 14:05:57 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6CA5012B00A for ; Tue, 13 Sep 2016 14:05:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=nypl.org Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jn4cUNRRxkeg for ; Tue, 13 Sep 2016 14:05:53 -0700 (PDT) Received: from mail-yb0-x236.google.com (mail-yb0-x236.google.com [IPv6:2607:f8b0:4002:c09::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BBC4B12B005 for ; Tue, 13 Sep 2016 14:05:53 -0700 (PDT) Received: by mail-yb0-x236.google.com with SMTP id u125so67804798ybg.3 for ; Tue, 13 Sep 2016 14:05:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nypl.org; s=google; h=mime-version:from:date:message-id:subject:to; bh=/+o7EVlO6zz9VuIcdiCdHJv/NUBSFmBoU9IgyQ6jvUE=; b=lumKBrhQH2tbLWlCATG3sc3GLi52/2eyGtuseGjbJJ0K+mJkvtLwZ031+vX2AVlVOG Obck8EdCWsZXx+M5oPgABniYlYbTO0UGtl3iSdW/CvZsQ03mRCiF5KHhMp3Sn1P3AULO OK3XxWwhP9Ii88pQZv8sKqYvdvcWwhiuCZvA8iZVMrD2ol043gTzl9wCV+nWG7gqbUmn cb9j9aJ+o8IdoCkNgUIgejEkYpknl+yvniVU8aW2WBq8Ygc6P0OPJH6kgZv5K40zGFd8 5KwXxy+KdByDXq62RH89QNCFEJHw+XFInhTNPhEMPb2ByZMqpZVvlpZeSgxkk/wpBzUk k6Dg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=/+o7EVlO6zz9VuIcdiCdHJv/NUBSFmBoU9IgyQ6jvUE=; b=G6IHedzccaeaKNdmITblzzfmWNLm5VIHdxBd+63ZtZghn/tGyqhD/9PGkhL1TIj0f3 xiB7dfa47++6kxgIl1LeEtD/oqmwrWSDsM6AEgxtZFPG0c7NwDGzw5hdxux8zteaBmUw RiVNS/sylyksduNoDgihgUsMwUDs/Thj3bGqYcW8LyZDf43xWqJMAsPVcFXJxr47mqDG 1QCbahudgR490bAZvQFPZNyd8VzOCVPxDHdINqCepK+0YI0bW6+7r/eKLWjmQ9I1eRWm 0fDOrt7BXroMipJWQL2DLhC/vXma/O7yRO+b4A33DhrZ6epd6QfNPHICmrMF1U8SAOa5 kqxQ== X-Gm-Message-State: AE9vXwO3QBV4NamlZ5LRBtjCfAbJytuIve9tUBnqNjZQ7hFd2it1XUs5LQE6Et+gRovatYkHwy17l4Mwp5Sxu+sH X-Received: by 10.37.197.130 with SMTP id v124mr3220813ybe.101.1473800752539; Tue, 13 Sep 2016 14:05:52 -0700 (PDT) MIME-Version: 1.0 Received: by 10.159.35.8 with HTTP; Tue, 13 Sep 2016 14:05:31 -0700 (PDT) From: Nick Krabbenhoeft Date: Tue, 13 Sep 2016 17:05:31 -0400 Message-ID: To: cellar@ietf.org Content-Type: multipart/alternative; boundary=94eb2c0458485362a2053c69fa35 Archived-At: Subject: Re: [Cellar] Standardizing FLAC X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Sep 2016 21:05:55 -0000 --94eb2c0458485362a2053c69fa35 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Tue, Sep 13, 2016 at 2:50 PM, J=C3=A9r=C3=B4me wrote: > > I would say that RF64 is the solution to your container problem with a > very small cost but that you should move to FLAC for other reasons > (detection of corrupted data, storage cost improvement...). note that thi= s > is the same reason I recommend FFV1 vs uncompressed video (v210...) > I agree that RF64 would be the lowest cost solution. We have looked at RF64, Wave64, and other containers, but we're finding there is a high political cost to convince internal and external partners to use anything besides WAVE32. My initial reasons to use a non-WAVE32 container were because of the problems managing part files during ingest, storage, description, and access. I'm looking at FLAC now because its other features address the objections we've heard to RF64. Unfortunately, FLAC does not have the weight of a standardization process behind it yet, which is why I'm looking to contribute. -Nick --94eb2c0458485362a2053c69fa35 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
On Tue, Sep 13, 2016 at 2:50 PM,=C2=A0J=C3=A9r=C3=B4me=C2= =A0wrote:
I would say that RF64 is the solution to your container pr= oblem with a very small cost but that you should move to FLAC for other rea= sons (detection of corrupted data, storage cost improvement...). note that = this is the same reason I recommend FFV1 vs uncompressed video (v210...)

I agree that RF64 would be the low= est cost solution. We have looked at RF64, Wave64, and other containers, bu= t we're finding there is a high political cost to convince internal and= external partners to use anything besides WAVE32.

My initial reasons to use a non-WAVE32 container were because of the probl= ems managing part files during ingest, storage, description, and access. I&= #39;m looking at FLAC now because its other features address the objections= we've heard to RF64.

Unfortunately, FLAC = does not have the weight of a standardization process behind it yet, which = is why I'm looking to contribute.

-Nick
--94eb2c0458485362a2053c69fa35-- From nobody Tue Sep 13 19:41:34 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E7B7412B193 for ; Tue, 13 Sep 2016 19:41:32 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.699 X-Spam-Level: X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NP7eHYtlbm_T for ; Tue, 13 Sep 2016 19:41:30 -0700 (PDT) Received: from mail-it0-x231.google.com (mail-it0-x231.google.com [IPv6:2607:f8b0:4001:c0b::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 92FAF12B523 for ; Tue, 13 Sep 2016 19:40:43 -0700 (PDT) Received: by mail-it0-x231.google.com with SMTP id r192so24687661ita.0 for ; Tue, 13 Sep 2016 19:40:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=Pem8wvSuJho0AbiEW4LG3Y7PhDkws3KHl2Mh/tUPc2Q=; b=zfEHpnvl6ZMkUolEKyV5yI0YxjqVB/ebx3WtB2fNxABrXIaMowgTs44YS78DW+tPlB h4SsSgh7eIYKy8+O1/0UUTly5rYkMf4eAESbO2R71UJq+e171ArQmqsBU5bVMAD2mKgm wqJKME6U/ZLCdcX4dj+rNJQkVFY59lGJJLx5UjrGBWCT5cKlDNzFXv2pga1Z9/G9a5C9 iIyiPycl0a6IJMf0zwJv6uWXjrvB1MxSnynjw9fXaLYEICFSrlO13H+/EOnH5CsuxqHA oeOsaxEYqsBXe4JxTCXvt3xFmLlGxeoHgzQUJE63POr+dqNawg11ON+RqOK2xFNBlDG2 Lf7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=Pem8wvSuJho0AbiEW4LG3Y7PhDkws3KHl2Mh/tUPc2Q=; b=FxAXQkZspy4/AVgvzmKeteMVcPQ0HPRyzUMNy0LwLz7CqJ8OsmrWInFAxQO//2Dfi/ VoOj8QNdJ7eifVDkx5QSg2P25xzgykHQjB2XzUzCW1mh0sMMme4Dm6ixBpX6h39VJanb mPZYvVbkwGZIUIbk9qfBQgSYBHTSypl2rRviyhg4iMK33BdCvMOhkiAT2YIgOiAFAuN/ JkCCrfJHdWPcu5P8pO62uulK4GxvBDYblXTCGeZ7ccnHmcIMHLSrxulSgqKx+4oEIxvg bYslkQgZmkofTy1s9ckqcQyN7v3MTpBeQZTNigICoLshHKK6qeOeLYVDo04ZdM4pAXjo BZcg== X-Gm-Message-State: AE9vXwMn59jYJ+JcWqdIDf0SlGdgt8Rplzm1b2DkYxOcM8OXoY7ocnaABzCxfTUJMJix1RGd62AjZYYaLapqPw== X-Received: by 10.107.182.9 with SMTP id g9mr1453871iof.66.1473820842669; Tue, 13 Sep 2016 19:40:42 -0700 (PDT) MIME-Version: 1.0 Received: by 10.107.40.4 with HTTP; Tue, 13 Sep 2016 19:40:02 -0700 (PDT) In-Reply-To: References: From: Tessa Fallon Date: Tue, 13 Sep 2016 22:40:02 -0400 Message-ID: To: cellar@ietf.org Content-Type: multipart/alternative; boundary=001a114aca0eca4391053c6ea72e Archived-At: Subject: Re: [Cellar] Standardizing FLAC X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Sep 2016 02:41:33 -0000 --001a114aca0eca4391053c6ea72e Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable RE: "Is anyone within CELLAR currently also involved in the FLAC development community, or know of a particular person to reach out to?" Via Tim Terriberry, the group reached out to Erik de Castro Lopo to discuss standardizing FLAC. Erik is the current maintainer of FLAC. Ashley, I'm happy to reach out to Erik and can cc you if you want For those interested in html to md (Nick, John Warburton), I imagine the best starting point is to talk with the person who's been guiding the process so far --Ashley, this is you, correct? Also if there is any documentation for this process, pls share I started another thread re: FLAC work, but perhaps it's better to loop back to this one. Cheers, Tessa On Tue, Sep 13, 2016 at 5:05 PM, Nick Krabbenhoeft < nickkrabbenhoeft@nypl.org> wrote: > On Tue, Sep 13, 2016 at 2:50 PM, J=C3=A9r=C3=B4me wrote: >> >> I would say that RF64 is the solution to your container problem with a >> very small cost but that you should move to FLAC for other reasons >> (detection of corrupted data, storage cost improvement...). note that th= is >> is the same reason I recommend FFV1 vs uncompressed video (v210...) >> > > I agree that RF64 would be the lowest cost solution. We have looked at > RF64, Wave64, and other containers, but we're finding there is a high > political cost to convince internal and external partners to use anything > besides WAVE32. > > My initial reasons to use a non-WAVE32 container were because of the > problems managing part files during ingest, storage, description, and > access. I'm looking at FLAC now because its other features address the > objections we've heard to RF64. > > Unfortunately, FLAC does not have the weight of a standardization process > behind it yet, which is why I'm looking to contribute. > > -Nick > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > > --001a114aca0eca4391053c6ea72e Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
RE: "Is anyone withi= n=C2=A0CELLAR=C2=A0currently also in= volved in the FLAC development community, or know of a particular person to= reach out to?"

Via Tim Terriberry, the group = reached out to Erik de Castro Lopo to discuss standardizing FLAC.=C2=A0 Eri= k is the current maintainer of FLAC.=C2=A0

As= hley, I'm happy to reach out to Erik and can cc you if you want<= /div>

For those interested in html to md (Nick, John Warbu= rton), I imagine the best starting point is to talk with the person who'= ;s been guiding the process so far
--Ashley, this is you, correct?=C2=A0 Also if there is any docume= ntation for this process, pls share

I started= another thread re: FLAC work, but perhaps it's better to loop back to = this one.

Cheers,

Tessa=C2=A0

On Tue, Sep 13, 2016 at 5:05 PM, Nick Krabbenhoeft <nickkrabbenhoeft@nypl.org> wrote:
On Tue, Sep 13, 2016 at 2:50 PM,=C2=A0J=C3= =A9r=C3=B4me=C2=A0wrote:
<= span style=3D"font-size:12.8px">I would say that RF64 is the solution to yo= ur container problem with a very small cost but that you should move to FLA= C for other reasons (detection of corrupted data, storage cost improvement.= ..). note that this is the same reason I recommend FFV1 vs uncompressed vid= eo (v210...)

I agree that= RF64 would be the lowest cost solution. We have looked at RF64, Wave64, an= d other containers, but we're finding there is a high political cost to= convince internal and external partners to use anything besides WAVE32.

My initial reasons to use a non-WAVE32 container wer= e because of the problems managing part files during ingest, storage, descr= iption, and access. I'm looking at FLAC now because its other features = address the objections we've heard to RF64.

Unfortunately, FLAC does not have the weight of a standardization process= behind it yet, which is why I'm looking to contribute.

-Nick

_______________________________________________
Cellar mailing list
Cellar@ietf.org https://www.ietf.org/mailman/listinfo/cellar

--001a114aca0eca4391053c6ea72e-- From nobody Wed Sep 14 00:00:12 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4DE5612B1F7 for ; Wed, 14 Sep 2016 00:00:10 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jXqoBnSfqp3G for ; Wed, 14 Sep 2016 00:00:03 -0700 (PDT) Received: from p1002.netstorage.at (p1002.netstorage.at [89.207.146.186]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D844112B1F1 for ; Wed, 14 Sep 2016 00:00:02 -0700 (PDT) Received: from wien.noa-archive.com (unknown [78.142.187.230]) by p1002.netstorage.at (Postfix) with ESMTPSA id EC6E4832EC for ; Wed, 14 Sep 2016 08:59:57 +0200 (CEST) Received: from EX2K14.noa-vienna.local (223.5.5.66) by EX2K14.noa-vienna.local (223.5.5.66) with Microsoft SMTP Server (TLS) id 15.0.995.29; Wed, 14 Sep 2016 08:59:56 +0200 Received: from EX2K14.noa-vienna.local ([223.5.5.66]) by EX2K14.noa-vienna.local ([223.5.5.66]) with mapi id 15.00.0995.028; Wed, 14 Sep 2016 08:59:55 +0200 From: Jean-Christophe Kummer To: "cellar@ietf.org" Thread-Topic: [Cellar] Standardizing FLAC Thread-Index: AQHSDgKPdYiXwmpBn0SHIg/4+BxWaKB4iaEw Date: Wed, 14 Sep 2016 06:59:54 +0000 Message-ID: <0aa5181096cd482ea9481c3e3dfdf6c9@EX2K14.noa-vienna.local> References: In-Reply-To: Accept-Language: de-AT, en-US Content-Language: de-DE X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [223.5.5.6] x-gfi-smtp-submission: 1 x-gfi-smtp-hellodomain: EX2K14.noa-vienna.local x-gfi-smtp-remoteip: 223.5.5.66 Content-Type: multipart/alternative; boundary="_000_0aa5181096cd482ea9481c3e3dfdf6c9EX2K14noaviennalocal_" MIME-Version: 1.0 X-PPP-Message-ID: <20160914065958.25525.6453@p1002.netstorage.at> X-PPP-Vhost: noa-archive.com Archived-At: Subject: Re: [Cellar] Standardizing FLAC X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Sep 2016 07:00:10 -0000 --_000_0aa5181096cd482ea9481c3e3dfdf6c9EX2K14noaviennalocal_ Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 SSBjYW4gdGVsbCB0aGF0IGZyb20gYXJvdW5kIDMgbWlvIGZpbGVzIHdoaWNoIGhhdmUgYmVlbiBk aWdpdGlzZWQgbWFpbmx5IGluIHRoZSBhcmNoaXZhbCBjb21tdW5pdHkgZnJvbSBsZXQgYWxvbmUg b3VyIGNsaWVudHMsIHRoZSBjaG9pY2UgbmV2ZXIgaGFzIGJlZW4gRkxBQyBidXQgbGluZWFyIHdh diDigJMgaW4gdGhlIGJlZ2lubmluZyB3aXRoIFdBViB3aXRoIHRoZSBmYW1vdXMgMmhpZ2ggMzEg bGltaXRhdGlvbiwgYW5kIGxhdGVyIG9uIHdpdGggUkY2NC4NCg0KU28gc3RpY2tpbmcgdG8gV0FW L1JGNjQgd291bGQgYmUgbXkgZmlyc3QgaWRlYSB0byBtYWtlIGl0IGFuIGFjY2VwdGFibGUgY2hv aWNlLg0KDQpUaGUgcmV2b2x1dGlvbiBpbiB0aGUgYXJjaGl2ZSB3b3JsZCBpcyBub3QgZmxhYyBi dXQgZmZ2MS4NCg0KTWF0cm9za2EgYXMgY29udGFpbmVyIHdpbGwgb3ZlcmNvbWUgc29tZSBhdmkg bGltaXRhdGlvbnMgd2hpY2ggd2VyZSBhbHJlYWR5IGluIHVzZSwNCkJ1dCBwbGVhc2UgbGV0wrRz IG5vdCBpbnRyb2R1Y2UgYSBuZXcgd2FyIOKAkyBhdWRpbyBjb2RlY3MgZm9yIGFyY2hpdmVzIGhh dmUgYmVlbiBzb2x2ZWQgMTUgeWVhcnMgYWdvLg0KDQpDaHJpc3RvcGhlDQoNCg0KDQpWb246IENl bGxhciBbbWFpbHRvOmNlbGxhci1ib3VuY2VzQGlldGYub3JnXSBJbSBBdWZ0cmFnIHZvbiBOaWNr IEtyYWJiZW5ob2VmdA0KR2VzZW5kZXQ6IERpZW5zdGFnLCAxMy4gU2VwdGVtYmVyIDIwMTYgMjM6 MDYNCkFuOiBjZWxsYXJAaWV0Zi5vcmcNCkJldHJlZmY6IFJlOiBbQ2VsbGFyXSBTdGFuZGFyZGl6 aW5nIEZMQUMNCg0KT24gVHVlLCBTZXAgMTMsIDIwMTYgYXQgMjo1MCBQTSwgSsOpcsO0bWUgd3Jv dGU6DQpJIHdvdWxkIHNheSB0aGF0IFJGNjQgaXMgdGhlIHNvbHV0aW9uIHRvIHlvdXIgY29udGFp bmVyIHByb2JsZW0gd2l0aCBhIHZlcnkgc21hbGwgY29zdCBidXQgdGhhdCB5b3Ugc2hvdWxkIG1v dmUgdG8gRkxBQyBmb3Igb3RoZXIgcmVhc29ucyAoZGV0ZWN0aW9uIG9mIGNvcnJ1cHRlZCBkYXRh LCBzdG9yYWdlIGNvc3QgaW1wcm92ZW1lbnQuLi4pLiBub3RlIHRoYXQgdGhpcyBpcyB0aGUgc2Ft ZSByZWFzb24gSSByZWNvbW1lbmQgRkZWMSB2cyB1bmNvbXByZXNzZWQgdmlkZW8gKHYyMTAuLi4p DQoNCkkgYWdyZWUgdGhhdCBSRjY0IHdvdWxkIGJlIHRoZSBsb3dlc3QgY29zdCBzb2x1dGlvbi4g V2UgaGF2ZSBsb29rZWQgYXQgUkY2NCwgV2F2ZTY0LCBhbmQgb3RoZXIgY29udGFpbmVycywgYnV0 IHdlJ3JlIGZpbmRpbmcgdGhlcmUgaXMgYSBoaWdoIHBvbGl0aWNhbCBjb3N0IHRvIGNvbnZpbmNl IGludGVybmFsIGFuZCBleHRlcm5hbCBwYXJ0bmVycyB0byB1c2UgYW55dGhpbmcgYmVzaWRlcyBX QVZFMzIuDQoNCk15IGluaXRpYWwgcmVhc29ucyB0byB1c2UgYSBub24tV0FWRTMyIGNvbnRhaW5l ciB3ZXJlIGJlY2F1c2Ugb2YgdGhlIHByb2JsZW1zIG1hbmFnaW5nIHBhcnQgZmlsZXMgZHVyaW5n IGluZ2VzdCwgc3RvcmFnZSwgZGVzY3JpcHRpb24sIGFuZCBhY2Nlc3MuIEknbSBsb29raW5nIGF0 IEZMQUMgbm93IGJlY2F1c2UgaXRzIG90aGVyIGZlYXR1cmVzIGFkZHJlc3MgdGhlIG9iamVjdGlv bnMgd2UndmUgaGVhcmQgdG8gUkY2NC4NCg0KVW5mb3J0dW5hdGVseSwgRkxBQyBkb2VzIG5vdCBo YXZlIHRoZSB3ZWlnaHQgb2YgYSBzdGFuZGFyZGl6YXRpb24gcHJvY2VzcyBiZWhpbmQgaXQgeWV0 LCB3aGljaCBpcyB3aHkgSSdtIGxvb2tpbmcgdG8gY29udHJpYnV0ZS4NCg0KLU5pY2sNCg== --_000_0aa5181096cd482ea9481c3e3dfdf6c9EX2K14noaviennalocal_ Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: base64 PGh0bWwgeG1sbnM6dj0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTp2bWwiIHhtbG5zOm89InVy bjpzY2hlbWFzLW1pY3Jvc29mdC1jb206b2ZmaWNlOm9mZmljZSIgeG1sbnM6dz0idXJuOnNjaGVt YXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6d29yZCIgeG1sbnM6bT0iaHR0cDovL3NjaGVtYXMubWlj cm9zb2Z0LmNvbS9vZmZpY2UvMjAwNC8xMi9vbW1sIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcv VFIvUkVDLWh0bWw0MCI+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIg Y29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjxtZXRhIG5hbWU9IkdlbmVyYXRv ciIgY29udGVudD0iTWljcm9zb2Z0IFdvcmQgMTUgKGZpbHRlcmVkIG1lZGl1bSkiPg0KPHN0eWxl PjwhLS0NCi8qIEZvbnQgRGVmaW5pdGlvbnMgKi8NCkBmb250LWZhY2UNCgl7Zm9udC1mYW1pbHk6 V2luZ2RpbmdzOw0KCXBhbm9zZS0xOjUgMCAwIDAgMCAwIDAgMCAwIDA7fQ0KQGZvbnQtZmFjZQ0K CXtmb250LWZhbWlseToiQ2FtYnJpYSBNYXRoIjsNCglwYW5vc2UtMToyIDQgNSAzIDUgNCA2IDMg MiA0O30NCkBmb250LWZhY2UNCgl7Zm9udC1mYW1pbHk6Q2FsaWJyaTsNCglwYW5vc2UtMToyIDE1 IDUgMiAyIDIgNCAzIDIgNDt9DQovKiBTdHlsZSBEZWZpbml0aW9ucyAqLw0KcC5Nc29Ob3JtYWws IGxpLk1zb05vcm1hbCwgZGl2Lk1zb05vcm1hbA0KCXttYXJnaW46MGNtOw0KCW1hcmdpbi1ib3R0 b206LjAwMDFwdDsNCglmb250LXNpemU6MTIuMHB0Ow0KCWZvbnQtZmFtaWx5OiJUaW1lcyBOZXcg Um9tYW4iLHNlcmlmO30NCmE6bGluaywgc3Bhbi5Nc29IeXBlcmxpbmsNCgl7bXNvLXN0eWxlLXBy aW9yaXR5Ojk5Ow0KCWNvbG9yOiMwNTYzQzE7DQoJdGV4dC1kZWNvcmF0aW9uOnVuZGVybGluZTt9 DQphOnZpc2l0ZWQsIHNwYW4uTXNvSHlwZXJsaW5rRm9sbG93ZWQNCgl7bXNvLXN0eWxlLXByaW9y aXR5Ojk5Ow0KCWNvbG9yOiM5NTRGNzI7DQoJdGV4dC1kZWNvcmF0aW9uOnVuZGVybGluZTt9DQpw Lk1zb0xpc3RQYXJhZ3JhcGgsIGxpLk1zb0xpc3RQYXJhZ3JhcGgsIGRpdi5Nc29MaXN0UGFyYWdy YXBoDQoJe21zby1zdHlsZS1wcmlvcml0eTozNDsNCgltYXJnaW4tdG9wOjBjbTsNCgltYXJnaW4t cmlnaHQ6MGNtOw0KCW1hcmdpbi1ib3R0b206MGNtOw0KCW1hcmdpbi1sZWZ0OjM2LjBwdDsNCglt YXJnaW4tYm90dG9tOi4wMDAxcHQ7DQoJZm9udC1zaXplOjEyLjBwdDsNCglmb250LWZhbWlseToi VGltZXMgTmV3IFJvbWFuIixzZXJpZjt9DQpzcGFuLkUtTWFpbEZvcm1hdHZvcmxhZ2UxNw0KCXtt c28tc3R5bGUtdHlwZTpwZXJzb25hbC1yZXBseTsNCglmb250LWZhbWlseToiQ2FsaWJyaSIsc2Fu cy1zZXJpZjsNCgljb2xvcjojMUY0OTdEO30NCi5Nc29DaHBEZWZhdWx0DQoJe21zby1zdHlsZS10 eXBlOmV4cG9ydC1vbmx5Ow0KCWZvbnQtZmFtaWx5OiJDYWxpYnJpIixzYW5zLXNlcmlmOw0KCW1z by1mYXJlYXN0LWxhbmd1YWdlOkVOLVVTO30NCkBwYWdlIFdvcmRTZWN0aW9uMQ0KCXtzaXplOjYx Mi4wcHQgNzkyLjBwdDsNCgltYXJnaW46NzAuODVwdCA3MC44NXB0IDIuMGNtIDcwLjg1cHQ7fQ0K ZGl2LldvcmRTZWN0aW9uMQ0KCXtwYWdlOldvcmRTZWN0aW9uMTt9DQovKiBMaXN0IERlZmluaXRp b25zICovDQpAbGlzdCBsMA0KCXttc28tbGlzdC1pZDo2MjI5MTk1MDsNCgltc28tbGlzdC10eXBl Omh5YnJpZDsNCgltc28tbGlzdC10ZW1wbGF0ZS1pZHM6NzM1NTYxODIgLTY3MDUzODU1OCAyMDE3 ODUzNDcgMjAxNzg1MzQ5IDIwMTc4NTM0NSAyMDE3ODUzNDcgMjAxNzg1MzQ5IDIwMTc4NTM0NSAy MDE3ODUzNDcgMjAxNzg1MzQ5O30NCkBsaXN0IGwwOmxldmVsMQ0KCXttc28tbGV2ZWwtc3RhcnQt YXQ6MDsNCgltc28tbGV2ZWwtbnVtYmVyLWZvcm1hdDpidWxsZXQ7DQoJbXNvLWxldmVsLXRleHQ6 LTsNCgltc28tbGV2ZWwtdGFiLXN0b3A6bm9uZTsNCgltc28tbGV2ZWwtbnVtYmVyLXBvc2l0aW9u OmxlZnQ7DQoJdGV4dC1pbmRlbnQ6LTE4LjBwdDsNCglmb250LWZhbWlseToiQ2FsaWJyaSIsc2Fu cy1zZXJpZjsNCgltc28tZmFyZWFzdC1mb250LWZhbWlseTpDYWxpYnJpOw0KCW1zby1iaWRpLWZv bnQtZmFtaWx5OiJUaW1lcyBOZXcgUm9tYW4iO30NCkBsaXN0IGwwOmxldmVsMg0KCXttc28tbGV2 ZWwtbnVtYmVyLWZvcm1hdDpidWxsZXQ7DQoJbXNvLWxldmVsLXRleHQ6bzsNCgltc28tbGV2ZWwt dGFiLXN0b3A6bm9uZTsNCgltc28tbGV2ZWwtbnVtYmVyLXBvc2l0aW9uOmxlZnQ7DQoJdGV4dC1p bmRlbnQ6LTE4LjBwdDsNCglmb250LWZhbWlseToiQ291cmllciBOZXciO30NCkBsaXN0IGwwOmxl dmVsMw0KCXttc28tbGV2ZWwtbnVtYmVyLWZvcm1hdDpidWxsZXQ7DQoJbXNvLWxldmVsLXRleHQ6 74KnOw0KCW1zby1sZXZlbC10YWItc3RvcDpub25lOw0KCW1zby1sZXZlbC1udW1iZXItcG9zaXRp b246bGVmdDsNCgl0ZXh0LWluZGVudDotMTguMHB0Ow0KCWZvbnQtZmFtaWx5OldpbmdkaW5nczt9 DQpAbGlzdCBsMDpsZXZlbDQNCgl7bXNvLWxldmVsLW51bWJlci1mb3JtYXQ6YnVsbGV0Ow0KCW1z by1sZXZlbC10ZXh0Ou+CtzsNCgltc28tbGV2ZWwtdGFiLXN0b3A6bm9uZTsNCgltc28tbGV2ZWwt bnVtYmVyLXBvc2l0aW9uOmxlZnQ7DQoJdGV4dC1pbmRlbnQ6LTE4LjBwdDsNCglmb250LWZhbWls eTpTeW1ib2w7fQ0KQGxpc3QgbDA6bGV2ZWw1DQoJe21zby1sZXZlbC1udW1iZXItZm9ybWF0OmJ1 bGxldDsNCgltc28tbGV2ZWwtdGV4dDpvOw0KCW1zby1sZXZlbC10YWItc3RvcDpub25lOw0KCW1z by1sZXZlbC1udW1iZXItcG9zaXRpb246bGVmdDsNCgl0ZXh0LWluZGVudDotMTguMHB0Ow0KCWZv bnQtZmFtaWx5OiJDb3VyaWVyIE5ldyI7fQ0KQGxpc3QgbDA6bGV2ZWw2DQoJe21zby1sZXZlbC1u dW1iZXItZm9ybWF0OmJ1bGxldDsNCgltc28tbGV2ZWwtdGV4dDrvgqc7DQoJbXNvLWxldmVsLXRh Yi1zdG9wOm5vbmU7DQoJbXNvLWxldmVsLW51bWJlci1wb3NpdGlvbjpsZWZ0Ow0KCXRleHQtaW5k ZW50Oi0xOC4wcHQ7DQoJZm9udC1mYW1pbHk6V2luZ2RpbmdzO30NCkBsaXN0IGwwOmxldmVsNw0K CXttc28tbGV2ZWwtbnVtYmVyLWZvcm1hdDpidWxsZXQ7DQoJbXNvLWxldmVsLXRleHQ674K3Ow0K CW1zby1sZXZlbC10YWItc3RvcDpub25lOw0KCW1zby1sZXZlbC1udW1iZXItcG9zaXRpb246bGVm dDsNCgl0ZXh0LWluZGVudDotMTguMHB0Ow0KCWZvbnQtZmFtaWx5OlN5bWJvbDt9DQpAbGlzdCBs MDpsZXZlbDgNCgl7bXNvLWxldmVsLW51bWJlci1mb3JtYXQ6YnVsbGV0Ow0KCW1zby1sZXZlbC10 ZXh0Om87DQoJbXNvLWxldmVsLXRhYi1zdG9wOm5vbmU7DQoJbXNvLWxldmVsLW51bWJlci1wb3Np dGlvbjpsZWZ0Ow0KCXRleHQtaW5kZW50Oi0xOC4wcHQ7DQoJZm9udC1mYW1pbHk6IkNvdXJpZXIg TmV3Ijt9DQpAbGlzdCBsMDpsZXZlbDkNCgl7bXNvLWxldmVsLW51bWJlci1mb3JtYXQ6YnVsbGV0 Ow0KCW1zby1sZXZlbC10ZXh0Ou+CpzsNCgltc28tbGV2ZWwtdGFiLXN0b3A6bm9uZTsNCgltc28t bGV2ZWwtbnVtYmVyLXBvc2l0aW9uOmxlZnQ7DQoJdGV4dC1pbmRlbnQ6LTE4LjBwdDsNCglmb250 LWZhbWlseTpXaW5nZGluZ3M7fQ0Kb2wNCgl7bWFyZ2luLWJvdHRvbTowY207fQ0KdWwNCgl7bWFy Z2luLWJvdHRvbTowY207fQ0KLS0+PC9zdHlsZT48IS0tW2lmIGd0ZSBtc28gOV0+PHhtbD4NCjxv OnNoYXBlZGVmYXVsdHMgdjpleHQ9ImVkaXQiIHNwaWRtYXg9IjEwMjYiIC8+DQo8L3htbD48IVtl bmRpZl0tLT48IS0tW2lmIGd0ZSBtc28gOV0+PHhtbD4NCjxvOnNoYXBlbGF5b3V0IHY6ZXh0PSJl ZGl0Ij4NCjxvOmlkbWFwIHY6ZXh0PSJlZGl0IiBkYXRhPSIxIiAvPg0KPC9vOnNoYXBlbGF5b3V0 PjwveG1sPjwhW2VuZGlmXS0tPg0KPC9oZWFkPg0KPGJvZHkgbGFuZz0iREUtQVQiIGxpbms9IiMw NTYzQzEiIHZsaW5rPSIjOTU0RjcyIj4NCjxkaXYgY2xhc3M9IldvcmRTZWN0aW9uMSI+DQo8cCBj bGFzcz0iTXNvTm9ybWFsIj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImZvbnQtc2l6ZToxMS4w cHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5zLXNlcmlmO2NvbG9yOiMxRjQ5 N0Q7bXNvLWZhcmVhc3QtbGFuZ3VhZ2U6RU4tVVMiPkkgY2FuIHRlbGwgdGhhdCBmcm9tIGFyb3Vu ZCAzIG1pbyBmaWxlcyB3aGljaCBoYXZlIGJlZW4gZGlnaXRpc2VkIG1haW5seSBpbiB0aGUgYXJj aGl2YWwgY29tbXVuaXR5IGZyb20gbGV0IGFsb25lIG91cg0KIGNsaWVudHMsIHRoZSBjaG9pY2Ug bmV2ZXIgaGFzIGJlZW4gRkxBQyBidXQgbGluZWFyIHdhdiDigJMgaW4gdGhlIGJlZ2lubmluZyB3 aXRoIFdBViB3aXRoIHRoZSBmYW1vdXMgMmhpZ2ggMzEgbGltaXRhdGlvbiwgYW5kIGxhdGVyIG9u IHdpdGggUkY2NC4NCjxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwi PjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTom cXVvdDtDYWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWY7Y29sb3I6IzFGNDk3RDttc28tZmFyZWFzdC1s YW5ndWFnZTpFTi1VUyI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1z b05vcm1hbCI+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQt ZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssc2Fucy1zZXJpZjtjb2xvcjojMUY0OTdEO21zby1m YXJlYXN0LWxhbmd1YWdlOkVOLVVTIj5TbyBzdGlja2luZyB0byBXQVYvUkY2NCB3b3VsZCBiZSBt eSBmaXJzdCBpZGVhIHRvIG1ha2UgaXQgYW4gYWNjZXB0YWJsZSBjaG9pY2UuDQo8bzpwPjwvbzpw Pjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5 bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5z LXNlcmlmO2NvbG9yOiMxRjQ5N0Q7bXNvLWZhcmVhc3QtbGFuZ3VhZ2U6RU4tVVMiPjxvOnA+Jm5i c3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIGxhbmc9IkVO LVVTIiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1 b3Q7LHNhbnMtc2VyaWY7Y29sb3I6IzFGNDk3RDttc28tZmFyZWFzdC1sYW5ndWFnZTpFTi1VUyI+ VGhlIHJldm9sdXRpb24gaW4gdGhlIGFyY2hpdmUgd29ybGQgaXMgbm90IGZsYWMgYnV0IGZmdjEu DQo8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBsYW5n PSJFTi1VUyIgc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJy aSZxdW90OyxzYW5zLXNlcmlmO2NvbG9yOiMxRjQ5N0Q7bXNvLWZhcmVhc3QtbGFuZ3VhZ2U6RU4t VVMiPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxz cGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVv dDtDYWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWY7Y29sb3I6IzFGNDk3RDttc28tZmFyZWFzdC1sYW5n dWFnZTpFTi1VUyI+TWF0cm9za2EgYXMgY29udGFpbmVyIHdpbGwgb3ZlcmNvbWUgc29tZSBhdmkg bGltaXRhdGlvbnMgd2hpY2ggd2VyZSBhbHJlYWR5IGluIHVzZSwNCjxvOnA+PC9vOnA+PC9zcGFu PjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iZm9u dC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWY7 Y29sb3I6IzFGNDk3RDttc28tZmFyZWFzdC1sYW5ndWFnZTpFTi1VUyI+QnV0IHBsZWFzZSBsZXTC tHMgbm90IGludHJvZHVjZSBhIG5ldyB3YXIg4oCTIGF1ZGlvIGNvZGVjcyBmb3IgYXJjaGl2ZXMg aGF2ZSBiZWVuIHNvbHZlZCAxNSB5ZWFycyBhZ28uPG86cD48L286cD48L3NwYW4+PC9wPg0KPHAg Y2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJmb250LXNpemU6MTEu MHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssc2Fucy1zZXJpZjtjb2xvcjojMUY0 OTdEO21zby1mYXJlYXN0LWxhbmd1YWdlOkVOLVVTIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48 L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImZvbnQt c2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5zLXNlcmlmO2Nv bG9yOiMxRjQ5N0Q7bXNvLWZhcmVhc3QtbGFuZ3VhZ2U6RU4tVVMiPkNocmlzdG9waGU8bzpwPjwv bzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBsYW5nPSJFTi1VUyIg c3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90Oyxz YW5zLXNlcmlmO2NvbG9yOiMxRjQ5N0Q7bXNvLWZhcmVhc3QtbGFuZ3VhZ2U6RU4tVVMiPjxvOnA+ Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIGxhbmc9 IkVOLVVTIiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJp JnF1b3Q7LHNhbnMtc2VyaWY7Y29sb3I6IzFGNDk3RDttc28tZmFyZWFzdC1sYW5ndWFnZTpFTi1V UyI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNw YW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90 O0NhbGlicmkmcXVvdDssc2Fucy1zZXJpZjtjb2xvcjojMUY0OTdEO21zby1mYXJlYXN0LWxhbmd1 YWdlOkVOLVVTIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9y bWFsIj48Yj48c3BhbiBsYW5nPSJERSIgc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1p bHk6JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5zLXNlcmlmIj5Wb246PC9zcGFuPjwvYj48c3BhbiBs YW5nPSJERSIgc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJy aSZxdW90OyxzYW5zLXNlcmlmIj4gQ2VsbGFyIFttYWlsdG86Y2VsbGFyLWJvdW5jZXNAaWV0Zi5v cmddDQo8Yj5JbSBBdWZ0cmFnIHZvbiA8L2I+TmljayBLcmFiYmVuaG9lZnQ8YnI+DQo8Yj5HZXNl bmRldDo8L2I+IERpZW5zdGFnLCAxMy4gU2VwdGVtYmVyIDIwMTYgMjM6MDY8YnI+DQo8Yj5Bbjo8 L2I+IGNlbGxhckBpZXRmLm9yZzxicj4NCjxiPkJldHJlZmY6PC9iPiBSZTogW0NlbGxhcl0gU3Rh bmRhcmRpemluZyBGTEFDPG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1h bCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+T24g VHVlLCBTZXAgMTMsIDIwMTYgYXQgMjo1MCBQTSwmbmJzcDtKw6lyw7RtZSZuYnNwO3dyb3RlOjxv OnA+PC9vOnA+PC9wPg0KPGJsb2NrcXVvdGUgc3R5bGU9ImJvcmRlcjpub25lO2JvcmRlci1sZWZ0 OnNvbGlkICNDQ0NDQ0MgMS4wcHQ7cGFkZGluZzowY20gMGNtIDBjbSA2LjBwdDttYXJnaW4tbGVm dDo0LjhwdDttYXJnaW4tcmlnaHQ6MGNtIj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0 eWxlPSJmb250LXNpemU6OS41cHQiPkkgd291bGQgc2F5IHRoYXQgUkY2NCBpcyB0aGUgc29sdXRp b24gdG8geW91ciBjb250YWluZXIgcHJvYmxlbSB3aXRoIGEgdmVyeSBzbWFsbCBjb3N0IGJ1dCB0 aGF0IHlvdSBzaG91bGQgbW92ZSB0byBGTEFDIGZvciBvdGhlciByZWFzb25zIChkZXRlY3Rpb24g b2YgY29ycnVwdGVkIGRhdGEsIHN0b3JhZ2UgY29zdCBpbXByb3ZlbWVudC4uLikuIG5vdGUgdGhh dA0KIHRoaXMgaXMgdGhlIHNhbWUgcmVhc29uIEkgcmVjb21tZW5kIEZGVjEgdnMgdW5jb21wcmVz c2VkIHZpZGVvICh2MjEwLi4uKTwvc3Bhbj48bzpwPjwvbzpwPjwvcD4NCjwvYmxvY2txdW90ZT4N CjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjwvZGl2 Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPkkgYWdyZWUgdGhhdCBSRjY0IHdvdWxkIGJl IHRoZSBsb3dlc3QgY29zdCBzb2x1dGlvbi4gV2UgaGF2ZSBsb29rZWQgYXQgUkY2NCwgV2F2ZTY0 LCBhbmQgb3RoZXIgY29udGFpbmVycywgYnV0IHdlJ3JlIGZpbmRpbmcgdGhlcmUgaXMgYSBoaWdo IHBvbGl0aWNhbCBjb3N0IHRvIGNvbnZpbmNlIGludGVybmFsIGFuZCBleHRlcm5hbCBwYXJ0bmVy cyB0byB1c2UgYW55dGhpbmcgYmVzaWRlcyBXQVZFMzIuPG86cD48L286cD48L3A+DQo8L2Rpdj4N CjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjwvZGl2 Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPk15IGluaXRpYWwgcmVhc29ucyB0byB1c2Ug YSBub24tV0FWRTMyIGNvbnRhaW5lciB3ZXJlIGJlY2F1c2Ugb2YgdGhlIHByb2JsZW1zIG1hbmFn aW5nIHBhcnQgZmlsZXMgZHVyaW5nIGluZ2VzdCwgc3RvcmFnZSwgZGVzY3JpcHRpb24sIGFuZCBh Y2Nlc3MuIEknbSBsb29raW5nIGF0IEZMQUMgbm93IGJlY2F1c2UgaXRzIG90aGVyIGZlYXR1cmVz IGFkZHJlc3MgdGhlIG9iamVjdGlvbnMgd2UndmUgaGVhcmQgdG8NCiBSRjY0LjxvOnA+PC9vOnA+ PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286 cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj5VbmZvcnR1bmF0ZWx5 LCBGTEFDIGRvZXMgbm90IGhhdmUgdGhlIHdlaWdodCBvZiBhIHN0YW5kYXJkaXphdGlvbiBwcm9j ZXNzIGJlaGluZCBpdCB5ZXQsIHdoaWNoIGlzIHdoeSBJJ20gbG9va2luZyB0byBjb250cmlidXRl LjxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PG86 cD4mbmJzcDs8L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj4t TmljazxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9ib2R5Pg0KPC9o dG1sPg0K --_000_0aa5181096cd482ea9481c3e3dfdf6c9EX2K14noaviennalocal_-- From nobody Wed Sep 14 07:01:13 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B86F312BCA2 for ; Wed, 14 Sep 2016 07:01:10 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.699 X-Spam-Level: X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wd4tloASWhM6 for ; Wed, 14 Sep 2016 07:01:05 -0700 (PDT) Received: from mail-it0-x233.google.com (mail-it0-x233.google.com [IPv6:2607:f8b0:4001:c0b::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 85DD612B777 for ; Wed, 14 Sep 2016 06:18:24 -0700 (PDT) Received: by mail-it0-x233.google.com with SMTP id r192so43337027ita.0 for ; Wed, 14 Sep 2016 06:18:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=wGAQtRkfNSzT0LkBPXy7HfevxT8qH0FWxH+mIwdiCI4=; b=vkPFfTN+zm1kgbsDJGHwYvsP/2AO+JQNj5IDODO53Ht+TXBQigVpe9XMe0hhYkmIWH lGlc9dxTaSz3eLxvYerpGofNKxOwHLEzw8gMg8pNLcPKjw/UYI4khDvlh3xJPesX8dhi 7fbBnij9M6Ru1QTh1YiHPSI+Wd2bnBEM/kI/xMMDyhJB/siLDtlMR926PchgBpE6RObo HZNzB66sK2tB8LLdQJAhLPYcI3iRyikayXniIxRoIXbEA/5NABeeM0e3iiArpSEXyjCz tds4WUrqSla/RA3GNQGue4hROD5MgPephlpRSdeSEQ5hSHer+8OhLvc7DBcmlXWDgNZG TONQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=wGAQtRkfNSzT0LkBPXy7HfevxT8qH0FWxH+mIwdiCI4=; b=BGAL7i+TjEHzATAopW/5fH3otLeoCLCDI+1oXei2WGxaL1+ckMrviJKwDyLsd0ul0o jb8tS7Vav8g3pe64bbeLDwvzl2Jvq1peXYmZE3lVg2Ow/NwDkIPiluvKhDg+NTiF7PLn TJPF7UlTLLZ3OIfDG4NanYeDKfrRvkrZdKi3ouhGs0qmu/dGAby4yyq8Ue/flVJD/iZY IDfumxP3tssrI66c9E1PnTj+QKFgILIhL4OBnIohv99Y2mB+qLx1/0THmMqnG4GpiZFY 3NxilMYGelavzqXA2Z7XCoUxX5m7jLMqgFnWYQdxTMBJAgOoXjAAGdlR8gBdtdIC9/Ln uoBQ== X-Gm-Message-State: AE9vXwOnPrbpLUQCszXij6C4m298BsQYdVq2K6N1wX/aA6vGxzMLyHlfOGrLcybpKi1CHBZyD6b6kf/OvFRMmA== X-Received: by 10.107.176.144 with SMTP id z138mr6744106ioe.133.1473859103788; Wed, 14 Sep 2016 06:18:23 -0700 (PDT) MIME-Version: 1.0 Received: by 10.79.6.146 with HTTP; Wed, 14 Sep 2016 06:18:23 -0700 (PDT) In-Reply-To: <0aa5181096cd482ea9481c3e3dfdf6c9@EX2K14.noa-vienna.local> References: <0aa5181096cd482ea9481c3e3dfdf6c9@EX2K14.noa-vienna.local> From: Ashley Blewer Date: Wed, 14 Sep 2016 09:18:23 -0400 Message-ID: To: Jean-Christophe Kummer Content-Type: multipart/alternative; boundary=001a11438ad454bba6053c77902c Archived-At: Cc: "cellar@ietf.org" Subject: Re: [Cellar] Standardizing FLAC X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Sep 2016 14:01:12 -0000 --001a11438ad454bba6053c77902c Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Revolutions and format politics aside, CELLAR is tasked with the standardization of MKV, FFV1, *and* FLAC and there's plenty of work to do. On Wed, Sep 14, 2016 at 2:59 AM, Jean-Christophe Kummer < c.kummer@noa-archive.com> wrote: > I can tell that from around 3 mio files which have been digitised mainly > in the archival community from let alone our clients, the choice never ha= s > been FLAC but linear wav =E2=80=93 in the beginning with WAV with the fam= ous 2high > 31 limitation, and later on with RF64. > > > > So sticking to WAV/RF64 would be my first idea to make it an acceptable > choice. > > > > The revolution in the archive world is not flac but ffv1. > > > > Matroska as container will overcome some avi limitations which were > already in use, > > But please let=C2=B4s not introduce a new war =E2=80=93 audio codecs for = archives have > been solved 15 years ago. > > > > Christophe > > > > > > > > *Von:* Cellar [mailto:cellar-bounces@ietf.org] *Im Auftrag von *Nick > Krabbenhoeft > *Gesendet:* Dienstag, 13. September 2016 23:06 > *An:* cellar@ietf.org > *Betreff:* Re: [Cellar] Standardizing FLAC > > > > On Tue, Sep 13, 2016 at 2:50 PM, J=C3=A9r=C3=B4me wrote: > > I would say that RF64 is the solution to your container problem with a > very small cost but that you should move to FLAC for other reasons > (detection of corrupted data, storage cost improvement...). note that thi= s > is the same reason I recommend FFV1 vs uncompressed video (v210...) > > > > I agree that RF64 would be the lowest cost solution. We have looked at > RF64, Wave64, and other containers, but we're finding there is a high > political cost to convince internal and external partners to use anything > besides WAVE32. > > > > My initial reasons to use a non-WAVE32 container were because of the > problems managing part files during ingest, storage, description, and > access. I'm looking at FLAC now because its other features address the > objections we've heard to RF64. > > > > Unfortunately, FLAC does not have the weight of a standardization process > behind it yet, which is why I'm looking to contribute. > > > > -Nick > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > > --001a11438ad454bba6053c77902c Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Revolutions and format politics aside, CELLAR is tasked wi= th the standardization of MKV, FFV1, and=C2=A0FLAC and there's p= lenty of work to do.=C2=A0

On Wed, Sep 14, 2016 at 2:59 AM, Jean-Christophe Kummer <c.kummer@noa-archive.com> wrote:

I can tell that from a= round 3 mio files which have been digitised mainly in the archival communit= y from let alone our clients, the choice never has been FLAC but linear wav =E2=80=93 in the be= ginning with WAV with the famous 2high 31 limitation, and later on with RF6= 4.

=C2=A0

So sticking to WAV/RF6= 4 would be my first idea to make it an acceptable choice.

=C2=A0

The revolution in the = archive world is not flac but ffv1.

=C2=A0

Matroska as container = will overcome some avi limitations which were already in use,

But please let=C2=B4s = not introduce a new war =E2=80=93 audio codecs for archives have been solve= d 15 years ago.

=C2=A0

Christophe

=C2=A0

=C2=A0

=C2=A0

Von: Cellar = [mailto:cellar= -bounces@ietf.org] Im Auftrag von Nick Krabbenhoeft
Gesendet: Dienstag, 13. September 2016 23:06
An: cellar@ietf= .org
Betreff: Re: [Cellar] Standardizing FLAC

=C2=A0

On Tue, Sep 13, 2016 at 2:50 PM,=C2=A0J=C3=A9r=C3=B4= me=C2=A0wrote:

I would say that RF6= 4 is the solution to your container problem with a very small cost but that= you should move to FLAC for other reasons (detection of corrupted data, st= orage cost improvement...). note that this is the same reason I recommend FFV1 vs uncompressed video (v210...)

=C2=A0

I agree that RF64 would be the lowest cost solution.= We have looked at RF64, Wave64, and other containers, but we're findin= g there is a high political cost to convince internal and external partners= to use anything besides WAVE32.

=C2=A0

My initial reasons to use a non-WAVE32 container wer= e because of the problems managing part files during ingest, storage, descr= iption, and access. I'm looking at FLAC now because its other features = address the objections we've heard to RF64.

=C2=A0

Unfortunately, FLAC does not have the weight of a st= andardization process behind it yet, which is why I'm looking to contri= bute.

=C2=A0

-Nick


_______________________________________________
Cellar mailing list
Cellar@ietf.org
https://www.ietf.org/mailman/listinfo/cellar

--001a11438ad454bba6053c77902c-- From nobody Wed Sep 14 07:04:04 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 130B712BD21 for ; Wed, 14 Sep 2016 07:04:03 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zK2u6Jhy_mLT for ; Wed, 14 Sep 2016 07:03:57 -0700 (PDT) Received: from mail-it0-x229.google.com (mail-it0-x229.google.com [IPv6:2607:f8b0:4001:c0b::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B7FA412B77E for ; Wed, 14 Sep 2016 06:21:23 -0700 (PDT) Received: by mail-it0-x229.google.com with SMTP id 186so23234555itf.0 for ; Wed, 14 Sep 2016 06:21:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=txiT+EhEJgIj3wsgIMccy4qYh4fS8ht99qE3PVS+sNY=; b=VA78pJ06ePOMufHz6ITc226yD4dXqlIRzMra8e/my8EvV5D3V+PSUoGfWxeNB6CA3f b8KMaxOzf6G9ea0Hl1Vi+4NRMXHNzyGSipwoEPDKIJbsnpn8RWOyIbUQj/Z6uuJIkwLD OlXlj72i/L9qwq6ewubH06sBij4zqLuBFn9jIluRTf9uiLlXVW1M2WA8XjaX82UICVUv H3VLHZsw75Hf6VobLC4VbpfFIcYMV+qNw/0hp2rMMs06cbgog1W4oXalTS2kzvo8ZxGK 1ZjbFBagk9/XoCvgDVVc8QmS48iFsERZvS9wnWejPqka7TiKxUYjXtcd8CKl6N5mqWiu sBug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=txiT+EhEJgIj3wsgIMccy4qYh4fS8ht99qE3PVS+sNY=; b=AY6fxIKo7rNXj14XazI5oOlAgSWo/Acu6r0WKC11ff2OO+6CeMt8kQNQp2BKPTu+2s Gzi+xxwjHV2FAgTu7DGgH5Ulk3Sahvx4kMmaFYBS/qjkcCgwFpg3kAS0jB4baArvQKpm oSHChkSIQ9ZkG9zHIRc746t3hDMREAS3PG0kq37NtoAsw17x0NoE3RYXNycoVVUUy/G4 GycwvoDh2ihDEvvW0beym59GR0MI2BbZmvmXPOzAhMS6gLhekuvuhULhjst5mkvttupH TI04ihJeOg7SB4XEtwZOI1sYdx2SVjmWJNVe7Tgn3dYNa93JsL6EKjJI1HYHQTGDk1Iu 1qCw== X-Gm-Message-State: AE9vXwOeX/XVULQ4Z8gNlSwCk+auwQARDo3WhG+30GxKtV3JDnALLNEHUKPa1grC1lbi+YS7L93dw/gi+eOvtg== X-Received: by 10.107.10.77 with SMTP id u74mr7257572ioi.22.1473859282950; Wed, 14 Sep 2016 06:21:22 -0700 (PDT) MIME-Version: 1.0 Received: by 10.79.6.146 with HTTP; Wed, 14 Sep 2016 06:21:22 -0700 (PDT) In-Reply-To: References: From: Ashley Blewer Date: Wed, 14 Sep 2016 09:21:22 -0400 Message-ID: To: Tessa Fallon , cellar@ietf.org Content-Type: multipart/alternative; boundary=001a113eca4e0265fb053c779bb4 Archived-At: Subject: Re: [Cellar] Standardizing FLAC X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Sep 2016 14:04:03 -0000 --001a113eca4e0265fb053c779bb4 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable In-line. On Tue, Sep 13, 2016 at 10:40 PM, Tessa Fallon wrote: > RE: "Is anyone within CELLAR currently also involved in the FLAC > development community, or know of a particular person to reach out to?" > > Via Tim Terriberry, the group reached out to Erik de Castro Lopo to > discuss standardizing FLAC. Erik is the current maintainer of FLAC. > > Ashley, I'm happy to reach out to Erik and can cc you if you want > > This would be a great start! Thanks! > For those interested in html to md (Nick, John Warburton), I imagine the > best starting point is to talk with the person who's been guiding the > process so far > --Ashley, this is you, correct? Also if there is any documentation for > this process, pls share > I did a lot of the conversion for FFV1 and MKV (as did Dave Rice) but haven't done any work towards FLAC. It has proven to be a useful way to review the spec in a granular way and facilitate contributions although not required. All roads eventually lead to a makefile that converts to an RFC so it makes sense to follow the same steps as we have with the other two, though. > > I started another thread re: FLAC work, but perhaps it's better to loop > back to this one. > > Cheers, > > Tessa > > On Tue, Sep 13, 2016 at 5:05 PM, Nick Krabbenhoeft < > nickkrabbenhoeft@nypl.org> wrote: > >> On Tue, Sep 13, 2016 at 2:50 PM, J=C3=A9r=C3=B4me wrote: >>> >>> I would say that RF64 is the solution to your container problem with a >>> very small cost but that you should move to FLAC for other reasons >>> (detection of corrupted data, storage cost improvement...). note that t= his >>> is the same reason I recommend FFV1 vs uncompressed video (v210...) >>> >> >> I agree that RF64 would be the lowest cost solution. We have looked at >> RF64, Wave64, and other containers, but we're finding there is a high >> political cost to convince internal and external partners to use anythin= g >> besides WAVE32. >> >> My initial reasons to use a non-WAVE32 container were because of the >> problems managing part files during ingest, storage, description, and >> access. I'm looking at FLAC now because its other features address the >> objections we've heard to RF64. >> >> Unfortunately, FLAC does not have the weight of a standardization proces= s >> behind it yet, which is why I'm looking to contribute. >> >> -Nick >> >> _______________________________________________ >> Cellar mailing list >> Cellar@ietf.org >> https://www.ietf.org/mailman/listinfo/cellar >> >> > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > > --001a113eca4e0265fb053c779bb4 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
In-line.

On Tue, Sep 13, 2016 at 10:40 PM, Tessa Fallon <tessa.fal= lon@gmail.com> wrote:
RE: "Is anyone within=C2= =A0CELLAR=C2=A0currently also involv= ed in the FLAC development community, or know of a particular person to rea= ch out to?"

Via Tim Terriberry, the group reac= hed out to Erik de Castro Lopo to discuss standardizing FLAC.=C2=A0 Erik is= the current maintainer of FLAC.=C2=A0

Ashley= , I'm happy to reach out to Erik and can cc you if you want

=

This would be a great start! Thanks!

=C2=A0
For those interested in html to md (Nick, John Warburton), I imagine = the best starting point is to talk with the person who's been guiding t= he process so far
--Ashle= y, this is you, correct?=C2=A0 Also if there is any documentation for this = process, pls share

I did= a lot of the conversion for FFV1 and MKV (as did Dave Rice) but haven'= t done any work towards FLAC. It has proven to be a useful way to review th= e spec in a granular way and facilitate contributions although not required= . All roads eventually lead to a makefile that converts to an RFC so it mak= es sense to follow the same steps as we have with the other two, though.
=C2=A0

I started another thread re: FLAC work, but perhaps it's bet= ter to loop back to this one.

Cheers,<= /div>

Tessa=C2=A0

On Tue, Sep 13, 20= 16 at 5:05 PM, Nick Krabbenhoeft <nickkrabbenhoeft@nypl.org>= ; wrote:
On Tue, Sep 13, 2016 at 2:50 PM,=C2=A0J=C3= =A9r=C3=B4me=C2=A0wrote:
<= span style=3D"font-size:12.8px">I would say that RF64 is the solution to yo= ur container problem with a very small cost but that you should move to FLA= C for other reasons (detection of corrupted data, storage cost improvement.= ..). note that this is the same reason I recommend FFV1 vs uncompressed vid= eo (v210...)

I agree that= RF64 would be the lowest cost solution. We have looked at RF64, Wave64, an= d other containers, but we're finding there is a high political cost to= convince internal and external partners to use anything besides WAVE32.

My initial reasons to use a non-WAVE32 container wer= e because of the problems managing part files during ingest, storage, descr= iption, and access. I'm looking at FLAC now because its other features = address the objections we've heard to RF64.

Unfortunately, FLAC does not have the weight of a standardization process= behind it yet, which is why I'm looking to contribute.

-Nick

_____________________________________= __________
Cellar mailing list
Cellar@ietf.org https://www.ietf.org/mailman/listinfo/cellar


_______________________________________________
Cellar mailing list
Cellar@ietf.org
https://www.ietf.org/mailman/listinfo/cellar

--001a113eca4e0265fb053c779bb4-- From nobody Wed Sep 14 07:57:31 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CD2AD12BB0E for ; Wed, 14 Sep 2016 07:57:30 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.702 X-Spam-Level: X-Spam-Status: No, score=-0.702 tagged_above=-999 required=5 tests=[BAYES_40=-0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, RCVD_IN_SORBS_SPAM=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eq5m7XjKYMeb for ; Wed, 14 Sep 2016 07:57:28 -0700 (PDT) Received: from smtp-sh2.infomaniak.ch (smtp-sh2.infomaniak.ch [128.65.195.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 18C2212B932 for ; Wed, 14 Sep 2016 07:25:25 -0700 (PDT) Received: from smtp6.infomaniak.ch (smtp6.infomaniak.ch [83.166.132.19]) by smtp-sh.infomaniak.ch (8.14.5/8.14.5) with ESMTP id u8EEPOqI005692 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL) for ; Wed, 14 Sep 2016 16:25:24 +0200 Received: from Castor.local (dynamic.wline.6rd.res.cust.swisscom.ch [IPv6:2a02:1205:c69b:8640:11a6:a56f:64bc:fc5] (may be forged)) (authenticated bits=0) by smtp6.infomaniak.ch (8.14.5/8.14.5) with ESMTP id u8EEPN7i059005 for ; Wed, 14 Sep 2016 16:25:23 +0200 Date: Wed, 14 Sep 2016 16:25:23 +0200 From: Reto Kromer To: cellar@ietf.org X-Priority: 3 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Mailer: Mailsmith 2.4 (470) X-Antivirus: Dr.Web (R) for Unix mail servers drweb plugin ver.6.0.2.8 X-Antivirus-Code: 0x100000 Archived-At: Subject: Re: [Cellar] Standardizing FLAC X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Sep 2016 14:57:31 -0000 Hello Cellar, I also wish to refrain also this one from 2016-07-19: Timothy B. Terriberry wrote: >Michael Niedermayer wrote: > >>I would be interrested in working on compression/algorithmic >>improvments on FLAC or in designing a new higher compression rate >>lossless audio codec, if CELLAR would be interrested in going that >>direction ... > >To do that we would need to recharter the working group, as our=20 >current charter is expressly to develop a formal specification=20 >for the existing FLAC format to allow it to be used in archival=20 >contexts where this is required. > >That is something we could try to do if people wanted to=20 >(working groups get rechartered all the time). I am just=20 >pointing out that it's out of scope of our current charter. In my opinion, it would be indeed a good move to amend the chart. Best regards, Reto From nobody Wed Sep 14 08:03:19 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7434A12BB47 for ; Wed, 14 Sep 2016 08:03:18 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.699 X-Spam-Level: X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=johnwarburton.net Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Csze1xRjlTe8 for ; Wed, 14 Sep 2016 08:03:16 -0700 (PDT) Received: from mail-wm0-x232.google.com (mail-wm0-x232.google.com [IPv6:2a00:1450:400c:c09::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 80C1112BDC1 for ; Wed, 14 Sep 2016 07:29:58 -0700 (PDT) Received: by mail-wm0-x232.google.com with SMTP id k186so10838007wmd.0 for ; Wed, 14 Sep 2016 07:29:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=johnwarburton.net; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=c6GazI2ayxVlUZfM6qdgH/2SSF5adB4GJD96MQX9I/A=; b=KlEbhJbl+LpXsXi3uGsTMXl4wJg8VSMZ0+EYcefTUi03wdr3kbPARpL4GB0zPiUo+A MT1XyvxZjtFepYTqdVj+fdOxCPLhKMXizxliRfh32yhlwK6EQpmN3tH/BJO0Sx8kj2Qo 4jgwMdebb44OD2LhlxNlmvKk8EEEhACFuSroI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=c6GazI2ayxVlUZfM6qdgH/2SSF5adB4GJD96MQX9I/A=; b=WZ4fEw/D4PE1/Rt7zHjnWRJW9UzHrVjIFr8vJbUN18ywA2aykofMJXcSsnPzQJ4W0D kb06zTGmQa74hxSZWs1YAuWtlVvT4npCjephrpLMsquMmZ8M7eAio4L3ygZnwO8iX8bP LMWkIXsS36k7witomzb0jTBxlgVa1k4cnrOZKfwso1rA9f+Du1SX94T7k+kLCB4BGBoE iHptO/xEHcAayFHdVtW2/q4uaT1u57AHWQ1ECQ+8rxPdBJnbzNywG5g3P93EFD0GbA9E NhiL/q3nyATYjVawD3fIMnrHfZL66En2+XeMbmvu//sF1gq6KmCXP3mofwl4PB06SNoe +9zQ== X-Gm-Message-State: AE9vXwPFt+Vm/5kCVuQNB91Md08LfD7sJuyoujpZ9AOHSaayLgLn5RjY420MpPXvvFhGQEZh//XF4n51uEB0XQ== X-Received: by 10.194.95.36 with SMTP id dh4mr2963154wjb.156.1473863397124; Wed, 14 Sep 2016 07:29:57 -0700 (PDT) MIME-Version: 1.0 Received: by 10.80.151.202 with HTTP; Wed, 14 Sep 2016 07:29:56 -0700 (PDT) In-Reply-To: References: From: John Warburton Date: Wed, 14 Sep 2016 15:29:56 +0100 Message-ID: To: cellar@ietf.org Content-Type: multipart/alternative; boundary=047d7beb91763c8841053c789089 Archived-At: Subject: Re: [Cellar] Standardizing FLAC X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Sep 2016 15:03:18 -0000 --047d7beb91763c8841053c789089 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Wed, Sep 14, 2016 at 2:21 PM, Ashley Blewer wrote: > > I did a lot of the conversion for FFV1 and MKV (as did Dave Rice) but > haven't done any work towards FLAC. It has proven to be a useful way to > review the spec in a granular way and facilitate contributions although n= ot > required. All roads eventually lead to a makefile that converts to an RFC > so it makes sense to follow the same steps as we have with the other two, > though. > =E2=80=8BI'll read the spec contained within the library doucments from beg= inning to end, therefore. Even just to get a feel for the documents, rather than the command line options, will be a greater education than that which I have at present. JW =E2=80=8B --047d7beb91763c8841053c789089 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
On Wed, Sep 14, 2016 at 2:21 PM, Ashley Blewer <ashley.blewer@gma= il.com> wrote:

I did a lot of the conversion for FFV1 and MKV (as did Dave Rice)= but haven't done any work towards FLAC. It has proven to be a useful w= ay to review the spec in a granular way and facilitate contributions althou= gh not required. All roads eventually lead to a makefile that converts to a= n RFC so it makes sense to follow the same steps as we have with the other = two, though.

=E2=80=8BI'll read the spec contained within the libra= ry doucments from beginning to end, therefore. Even just to get a feel for = the documents, rather than the command line options, will be a greater educ= ation than that which I have at present.

JW
=E2=80=8B
=C2=A0
--047d7beb91763c8841053c789089-- From nobody Sun Sep 18 00:44:34 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4F05312B03C for ; Sun, 18 Sep 2016 00:44:32 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.6 X-Spam-Level: X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3ZBMZFtFZ789 for ; Sun, 18 Sep 2016 00:44:30 -0700 (PDT) Received: from mail-yw0-x232.google.com (mail-yw0-x232.google.com [IPv6:2607:f8b0:4002:c05::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C262812B01F for ; Sun, 18 Sep 2016 00:44:30 -0700 (PDT) Received: by mail-yw0-x232.google.com with SMTP id t67so110670476ywg.3 for ; Sun, 18 Sep 2016 00:44:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=BUUfoc46s+ZfZoBe745u00TstwfG1TkrOh4HcWVMgWQ=; b=G603pLGOjRgaFMHVU+xVpnLfw8Bq5rgcv8v7WY9xPKj4UdFmALXofo1T01Txop//ll bO1/+HepuR8S6TXpQoa1tmoHDWcw58cJBWhY2fz7m+C+wlBd3t+ZNekMPkZdu3SrNP+a gEJzpKWPFAUqRdvrr79Mp8IdSYs7UMnCXQOL2KK39R2b9UiWeX7GM0AT7ojwjff6Tuom NgtqBkbymsXkviNAdO8pQ3qCgqGhfTpFWxcIlSiSBVsITNS4bbkpxltF+jOE12PXky32 YDAFLiAI6eeuzGgqEFNzcSzmK3NxY07/B5GC0hyu3uTLb0Qt9/VlgQsffx4DHii9wkY3 jFnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=BUUfoc46s+ZfZoBe745u00TstwfG1TkrOh4HcWVMgWQ=; b=RwugxwrDz/d5o+geMN3pKTTO7dyLHR2LZdfzX5muTU+gryUfM0DgA+xP13OZwl3GP5 d9b/rd7LnMkDJOHFtHCMAGri63gkAapomsPSrFS83G0URdXYNyHtWMaIckrENFJR8nKG L7EOiqQz0DGN1TfRpXiO0zl7uyDJC0caIk3ajYeSAqLpJuiXcFxREgSsaJwXaGfflaza IAnmxdIMfgtXqJatORJ8oRckM4nyk/pEljcsgg6Cq0eQV2Os9B15mNxwyBZzhqDpiXUz RZ5L5P6Nl97fJNDI/mKCR9nEHa4SEhgLZUU2AjKiStg7U12Zvm6/NJ6FHlmGFTOLjdUp fR7A== X-Gm-Message-State: AE9vXwOytp3W8AVSMbSgxTAxdvNNhPwW3ilFmTQYEL1QpwSgLqQQo/LxUFcvRw6xZ989NBoKVMhQd5LDeZSGvA== X-Received: by 10.129.158.195 with SMTP id v186mr13324274ywg.43.1474184669963; Sun, 18 Sep 2016 00:44:29 -0700 (PDT) MIME-Version: 1.0 Received: by 10.83.20.135 with HTTP; Sun, 18 Sep 2016 00:44:29 -0700 (PDT) In-Reply-To: References: From: Steve Lhomme Date: Sun, 18 Sep 2016 09:44:29 +0200 Message-ID: To: Tessa Fallon Content-Type: text/plain; charset=UTF-8 Archived-At: Cc: Codec Encoding for LossLess Archiving and Realtime transmission Subject: Re: [Cellar] IETF current milestones: request for feedback X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Sep 2016 07:44:32 -0000 2016-09-13 15:58 GMT+02:00 Tessa Fallon : > At the CELLAR meeting in July, it was decided that working group milestones > would need to be revised. The current list is below. > > Would those working on FFV1, EBML, and Matroska weigh in on the state of the > specifications and projected timelines for work to be completed? I note > that the most recent changes to EBML were submitted on Sunday 9/11. Where > does this leave the document? It's an on-going effort. There are currently 16 issues opened on the EBML specs alone: https://github.com/Matroska-Org/ebml-specification/issues There are 5 on Matroska but that's because most of the effort has been concentrated on EBML yet: https://github.com/Matroska-Org/matroska-specification/issues There are way more elements to defined in Matroska, various interactions between them and all kinds of loopholes that we need to iron out. > Much thanks, > Tessa > > Date > Dec 2016 Submit specification for FLAC audio codec to IESG (Standards Track) > Sep 2016 Submit specification for FFV1 video codec version 4 to IESG > (Standards Track) > Jul 2016 Submit specification for Matroska container format version 4 to > IESG (Standards Track) > Jul 2016 Submit informational specification for FFV1 video codec versions 0, > 1 and 3 to IESG for publication > Jun 2016 Submit specification for EBML to IESG (Standards Track) > Jun 2016 Submit informational specification for Matroska container format > versions 1, 2 and 3 to IESG for publication > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > -- Steve Lhomme Matroska association Chairman From nobody Wed Sep 21 11:15:36 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AAA7D12B829 for ; Wed, 21 Sep 2016 11:15:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.703 X-Spam-Level: X-Spam-Status: No, score=-0.703 tagged_above=-999 required=5 tests=[BAYES_40=-0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id esE15fYZ6Jr5 for ; Wed, 21 Sep 2016 11:15:31 -0700 (PDT) Received: from smtp-sh2.infomaniak.ch (smtp-sh2.infomaniak.ch [128.65.195.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C34C612B70D for ; Wed, 21 Sep 2016 11:15:30 -0700 (PDT) Received: from smtp6.infomaniak.ch (smtp6.infomaniak.ch [83.166.132.19]) by smtp-sh.infomaniak.ch (8.14.5/8.14.5) with ESMTP id u8LIFRIr021886 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL) for ; Wed, 21 Sep 2016 20:15:28 +0200 Received: from Castor.local (77-56-128-186.dclient.hispeed.ch [77.56.128.186]) (authenticated bits=0) by smtp6.infomaniak.ch (8.14.5/8.14.5) with ESMTP id u8LIFRGp046812 for ; Wed, 21 Sep 2016 20:15:27 +0200 Date: Wed, 21 Sep 2016 20:15:27 +0200 From: Reto Kromer To: cellar@ietf.org X-Priority: 3 In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Mailer: Mailsmith 2.4 (470) X-Antivirus: Dr.Web (R) for Unix mail servers drweb plugin ver.6.0.2.8 X-Antivirus-Code: 0x100000 Archived-At: Subject: Re: [Cellar] New thread: Work to be done on FLAC, call for volunteers X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Sep 2016 18:15:35 -0000 Tessa Fallon wrote: >If you're interested in working on FLAC, please respond to >this thread. I'm more than interested in the FLAC standardisation, and would like to contribute, yet I have to admit that I really hate markdown. Best regards from Reto From nobody Wed Sep 21 11:24:41 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8714412B632 for ; Wed, 21 Sep 2016 11:24:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.121 X-Spam-Level: X-Spam-Status: No, score=-1.121 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Lj68OYfuHuO9 for ; Wed, 21 Sep 2016 11:24:34 -0700 (PDT) Received: from s172.web-hosting.com (s172.web-hosting.com [68.65.122.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C117912B7CA for ; Wed, 21 Sep 2016 11:24:34 -0700 (PDT) Received: from [146.96.19.240] (port=24183 helo=[10.10.202.53]) by server172.web-hosting.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.86_1) (envelope-from ) id 1bmmC9-000ZFf-20; Wed, 21 Sep 2016 14:24:34 -0400 Mime-Version: 1.0 (Mac OS X Mail 9.2 \(3112\)) Content-Type: text/plain; charset=us-ascii From: Dave Rice X-Priority: 3 In-Reply-To: Date: Wed, 21 Sep 2016 14:24:31 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Reto Kromer X-Mailer: Apple Mail (2.3112) X-OutGoing-Spam-Status: No, score=-2.9 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Cc: cellar@ietf.org Subject: Re: [Cellar] New thread: Work to be done on FLAC, call for volunteers X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Sep 2016 18:24:35 -0000 > On Sep 21, 2016, at 2:15 PM, Reto Kromer wrote: >=20 > Tessa Fallon wrote: >=20 >> If you're interested in working on FLAC, please respond to >> this thread. >=20 > I'm more than interested in the FLAC standardisation, and > would like to contribute, yet I have to admit that I really > hate markdown. I think Markdown was initially suggested to encourage more volunteers in = the FFV1 specification, which prior to using markdown was in lyx. IMHO = markdown was an easier choice to work with than lyx. Currently the FLAC = format documentation is in HTML so I suppose it's feasible to work on it = in HTML but at some point we'll need to discover a way to convert the = HTML to RFC. Or I suppose FLAC work could be done in another = intermediate format. Suggestions? Dave= From nobody Wed Sep 21 12:02:14 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 925C212B8C2 for ; Wed, 21 Sep 2016 12:02:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.602 X-Spam-Level: X-Spam-Status: No, score=-2.602 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id E4k1vNbZFZdb for ; Wed, 21 Sep 2016 12:02:10 -0700 (PDT) Received: from smtp-sh2.infomaniak.ch (smtp-sh2.infomaniak.ch [128.65.195.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 60F4512BCA6 for ; Wed, 21 Sep 2016 12:01:55 -0700 (PDT) Received: from smtp5.infomaniak.ch (smtp5.infomaniak.ch [83.166.132.18]) by smtp-sh.infomaniak.ch (8.14.5/8.14.5) with ESMTP id u8LJ1rkK009074 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL) for ; Wed, 21 Sep 2016 21:01:53 +0200 Received: from Castor.local (77-56-128-186.dclient.hispeed.ch [77.56.128.186]) (authenticated bits=0) by smtp5.infomaniak.ch (8.14.5/8.14.5) with ESMTP id u8LJ1q8Z011323 for ; Wed, 21 Sep 2016 21:01:53 +0200 Date: Wed, 21 Sep 2016 21:01:53 +0200 From: Reto Kromer To: cellar@ietf.org X-Priority: 3 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Mailer: Mailsmith 2.4 (470) X-Antivirus: Dr.Web (R) for Unix mail servers drweb plugin ver.6.0.2.8 X-Antivirus-Code: 0x100000 Archived-At: Subject: Re: [Cellar] New thread: Work to be done on FLAC, call for volunteers X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Sep 2016 19:02:11 -0000 Dave Rice wrote: >I think Markdown was initially suggested to encourage more >volunteers in the FFV1 specification, which prior to using >markdown was in lyx. IMHO markdown was an easier choice to >work with than lyx. Indeed! If this kind is chosen, then I would suggest LaTeX instead of LyX, because it's more widely known, I guess. >Currently the FLAC format documentation >is in HTML so I suppose it's feasible to work on it in HTML >but at some point we'll need to discover a way to convert >the HTML to RFC. Or I suppose FLAC work could be done in >another intermediate format. Suggestions? I never used "xml2rfc", but this seems to exist. The current HTML5 can be processed by any XML tool (after serialising where needed). Hope this helps! Reto From nobody Wed Sep 21 12:14:41 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 771B812B8ED for ; Wed, 21 Sep 2016 12:14:39 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.621 X-Spam-Level: X-Spam-Status: No, score=-2.621 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RoKaDTcZk7Uu for ; Wed, 21 Sep 2016 12:14:37 -0700 (PDT) Received: from 14.mo7.mail-out.ovh.net (14.mo7.mail-out.ovh.net [178.33.251.19]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id EB6ED12B67D for ; Wed, 21 Sep 2016 12:14:36 -0700 (PDT) Received: from player796.ha.ovh.net (b7.ovh.net [213.186.33.57]) by mo7.mail-out.ovh.net (Postfix) with ESMTP id 9D185FF9B63 for ; Wed, 21 Sep 2016 21:14:32 +0200 (CEST) Received: from [192.168.2.101] (p5DDB5F8D.dip0.t-ipconnect.de [93.219.95.141]) (Authenticated sender: jerome@francoallemand.eu) by player796.ha.ovh.net (Postfix) with ESMTPSA id 312215C007B for ; Wed, 21 Sep 2016 21:14:32 +0200 (CEST) To: cellar@ietf.org References: From: Jerome Martinez Message-ID: <65afdf8a-18f7-d110-abfb-fea05733c4ab@mediaarea.net> Date: Wed, 21 Sep 2016 21:14:31 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-Ovh-Tracer-Id: 15859426088761888913 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 0 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrfeelvddrtddvgddufedtucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfenuceurghilhhouhhtmecufedttdenuc Archived-At: Subject: Re: [Cellar] New thread: Work to be done on FLAC, call for volunteers X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Sep 2016 19:14:39 -0000 Le 21/09/2016 21:01, Reto Kromer a crit : > Dave Rice wrote: > >> I think Markdown was initially suggested to encourage more >> volunteers in the FFV1 specification, which prior to using >> markdown was in lyx. IMHO markdown was an easier choice to >> work with than lyx. > Indeed! If this kind is chosen, then I would suggest LaTeX > instead of LyX, because it's more widely known, I guess. as a volunteer but maybe not the main one, I suggest to use the nowadays de facto standard for the Markup language. I understand that some people may "hate" it (can we really hate a markup "language"?) but I argue that this is the language used by Wikipedia, that we have direct display in e.g. GitHub (e.g. FFV1 specs https://github.com/FFmpeg/FFV1/blob/master/ffv1.md ), that there is a standardization effort of Markdown (including IETF RFC 7764), and that LaTeX is known by less people than Markdown. From micasey@indiana.edu Wed Sep 21 11:49:18 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F0FB812B739 for ; Wed, 21 Sep 2016 11:49:17 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.537 X-Spam-Level: X-Spam-Status: No, score=-6.537 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-2.316, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ECnXW_GAcOB4 for ; Wed, 21 Sep 2016 11:49:16 -0700 (PDT) Received: from hartman.uits.indiana.edu (hartman.uits.indiana.edu [129.79.1.194]) (using TLSv1.2 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3C2FD12B113 for ; Wed, 21 Sep 2016 11:49:16 -0700 (PDT) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A2A8BgC81eJX/0kBT4FbAxwBAQQBAQoBA?= =?us-ascii?q?YMoEwEBAQEBHleBA7hvggQBG4dpOhIBAgEBAQEBAQFeHAuEaDpRATUJQiYBBBu?= =?us-ascii?q?IQwGfQJlFAQaCZwEKAQEBAR0FhjiIaREBWwiCBQuDBwWINpE/AYE5hG2JNI9zk?= =?us-ascii?q?GMlBCuDGRyBUIUYgSABfgEBAQ?= X-IPAS-Result: =?us-ascii?q?A2A8BgC81eJX/0kBT4FbAxwBAQQBAQoBAYMoEwEBAQEBHle?= =?us-ascii?q?BA7hvggQBG4dpOhIBAgEBAQEBAQFeHAuEaDpRATUJQiYBBBuIQwGfQJlFAQaCZ?= =?us-ascii?q?wEKAQEBAR0FhjiIaREBWwiCBQuDBwWINpE/AYE5hG2JNI9zkGMlBCuDGRyBUIU?= =?us-ascii?q?YgSABfgEBAQ?= X-IronPort-AV: E=Sophos;i="5.30,374,1470715200"; d="scan'208";a="37515485" Received: from mssg-relay.indiana.edu ([129.79.1.73]) by irpt-internal-relay.indiana.edu with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 21 Sep 2016 14:49:15 -0400 Received: from bl-cci-exch0f.ads.iu.edu (bl-cci-exch0f.ads.iu.edu [10.79.80.141]) by mssg-relay.indiana.edu (8.14.7/8.14.4/IU Campus Communications Team) with ESMTP id u8LInFWW009561 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NOT) for ; Wed, 21 Sep 2016 14:49:15 -0400 Received: from in-cci-exch08.ads.iu.edu (2001:18e8:3:cc1::10b) by bl-cci-exch0f.ads.iu.edu (2001:18e8:2:cc1::24a) with Microsoft SMTP Server (TLS) id 15.0.1210.3; Wed, 21 Sep 2016 14:49:15 -0400 Received: from in-cci-exch08.ads.iu.edu ([fe80::383e:ce1d:3835:718c]) by in-cci-exch08.ads.iu.edu ([fe80::383e:ce1d:3835:718c%18]) with mapi id 15.00.1210.000; Wed, 21 Sep 2016 14:49:14 -0400 From: "Casey, Michael T" To: "cellar@ietf.org" Thread-Topic: Risk assessment for FFV1 and Matroska Thread-Index: AdIUOMYE9b5vwXxKTa6q4p7s4EadRw== Date: Wed, 21 Sep 2016 18:49:14 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [129.79.49.223] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: Subject: [Cellar] Risk assessment for FFV1 and Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Sep 2016 19:22:10 -0000 All, I have a couple of questions from a large digitization project in Europe ab= out the risk associated with choosing FFV1 and Matroska. This is from their= email: =20 You guys have chosen for FFv1 last year, but the standardisation process is= still going on (IETF, CELLAR) as you know.=20 =20 What did you think about the risk of going for a not-yet-standardized codec= ? Did you somehow take into account the chance that the standardized FFv1 w= ill look a little different than the one you're using now? =20 I am wondering if you have any thoughts on this that I could pass along? =20 Thanks, =20 Mike ------------ Mike Casey Director of Technical Operations Media Digitization and Preservation Initiative Indiana University 812-855-8090 https://mdpi.iu.edu/ From nobody Wed Sep 21 12:49:26 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 37C5312BA76 for ; Wed, 21 Sep 2016 12:49:25 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.621 X-Spam-Level: X-Spam-Status: No, score=-2.621 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PWGchDEajNrK for ; Wed, 21 Sep 2016 12:49:03 -0700 (PDT) Received: from 15.mo7.mail-out.ovh.net (15.mo7.mail-out.ovh.net [87.98.180.21]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 96FDB12B186 for ; Wed, 21 Sep 2016 12:49:00 -0700 (PDT) Received: from player796.ha.ovh.net (b7.ovh.net [213.186.33.57]) by mo7.mail-out.ovh.net (Postfix) with ESMTP id E1329FFCFED for ; Wed, 21 Sep 2016 21:48:58 +0200 (CEST) Received: from [192.168.2.101] (p5DDB5F8D.dip0.t-ipconnect.de [93.219.95.141]) (Authenticated sender: jerome@francoallemand.eu) by player796.ha.ovh.net (Postfix) with ESMTPSA id 5D83E5C0080 for ; Wed, 21 Sep 2016 21:48:58 +0200 (CEST) To: cellar@ietf.org References: From: Jerome Martinez Message-ID: Date: Wed, 21 Sep 2016 21:48:57 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-Ovh-Tracer-Id: 16440953393298149521 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 0 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrfeelvddrtddvgddufeekucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfenuceurghilhhouhhtmecufedttdenuc Archived-At: Subject: Re: [Cellar] Risk assessment for FFV1 and Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Sep 2016 19:49:25 -0000 Le 21/09/2016 20:49, Casey, Michael T a crit : > [...] > What did you think about the risk of going for a not-yet-standardized codec? Did you somehow take into account the chance that the standardized FFv1 will look a little different than the one you're using now? The goal of CELLAR is to describe what is currently the most used FFV1 (definition of FFV1 version 0/1/3), before creating a new (and breaking compatibility) version 4. So the standard will not look a little different: it will be the description of the most used FFV1. As a proof that we prioritize what you are using instead of the draft of specification, an example: https://www.ietf.org/mail-archive/web/cellar/current/msg00823.html You can see that we discovered a little (16-bit YUV is not widely used) difference between the main implementation and the draft of spec, we decided to update the draft of spec instead than creating an incompatibility between the standard and the files in the wild (and we'll simplify the bitstream format in FFV1 version 4, not yet in the wild). The most used FFV1 encoder/decoder is FFmpeg. There may be a risk in case you don't use FFmpeg for encoding. Do you? Another argument is that as it is lossless compression, it would be relatively easy to move from the "little different" bitstream to the "standard" without losing quality, in the case we decide (we won't) to have a little difference somewhere. That said, which FFV1 encoder do you use? I can say that I deeply reviewed FFmpeg 3.1 (the last one) for FFV1 8-, 10- and 16-bit, for Gray, YUV and RGB, with Golomb Rice (8-bit) and Range Coder (8/10/16-bit) coders, with version 3 of FFV1, I can say that I created another decoder which confirms that the current draft of spec is correct, so I can say that I have big doubts that the spec and FFmpeg 3.1 have or will have a little difference. if you use one of these configurations, go ahead, else provide the configuration you use and I'll prioritize it in my review. The same applies to Matroska but I am less involved in it, I can only say that the idea is the same and the risk is only in case you use tools we don't check (e.g. if you use a super rare muxer) and in case you use super rare Matroska features (e.g. encryption, deprecated signature...), else we review files from several muxers e.g. tools using libmatroska or FFmpeg. Our priority is to have no "little difference" with the most used muxers. Jrme From nobody Wed Sep 21 12:56:29 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DBB0F12BD2C for ; Wed, 21 Sep 2016 12:56:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GfdrAiUdtoBs for ; Wed, 21 Sep 2016 12:56:27 -0700 (PDT) Received: from relay6-d.mail.gandi.net (relay6-d.mail.gandi.net [IPv6:2001:4b98:c:538::198]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1E08912BD6E for ; Wed, 21 Sep 2016 12:56:26 -0700 (PDT) Received: from mfilter47-d.gandi.net (mfilter47-d.gandi.net [217.70.178.178]) by relay6-d.mail.gandi.net (Postfix) with ESMTP id 8AF2BFB89E for ; Wed, 21 Sep 2016 21:56:24 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at mfilter47-d.gandi.net Received: from relay6-d.mail.gandi.net ([IPv6:::ffff:217.70.183.198]) by mfilter47-d.gandi.net (mfilter47-d.gandi.net [::ffff:10.0.15.180]) (amavisd-new, port 10024) with ESMTP id hBrgLUCvYAUR for ; Wed, 21 Sep 2016 21:56:23 +0200 (CEST) X-Originating-IP: 213.47.41.20 Received: from localhost (213-47-41-20.cable.dynamic.surfer.at [213.47.41.20]) (Authenticated sender: michael@niedermayer.cc) by relay6-d.mail.gandi.net (Postfix) with ESMTPSA id EC303FB887 for ; Wed, 21 Sep 2016 21:56:21 +0200 (CEST) Date: Wed, 21 Sep 2016 21:56:06 +0200 From: Michael Niedermayer To: cellar@ietf.org Message-ID: <20160921195606.GI4983@nb4> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="vH3HHxf962mwD/qo" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Archived-At: Subject: Re: [Cellar] New thread: Work to be done on FLAC, call for volunteers X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Sep 2016 19:56:29 -0000 --vH3HHxf962mwD/qo Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Sep 21, 2016 at 09:01:53PM +0200, Reto Kromer wrote: > Dave Rice wrote: >=20 > >I think Markdown was initially suggested to encourage more > >volunteers in the FFV1 specification, which prior to using > >markdown was in lyx. IMHO markdown was an easier choice to > >work with than lyx. >=20 > Indeed! If this kind is chosen, then I would suggest LaTeX > instead of LyX, because it's more widely known, I guess. lyx syntax / language knowledge isnt needed to use lyx. lyx has a nice GUI editor which is what i used for IIRC 100% of the spec before markdown The real disadvanatge of lyx was that when multiple people try to work on a spec and review changes on git backed .lyx files things get somewhat messy if one tries to work with high level tools like the lyx GUI editor ... OTOH lyx works much better and reproduceable to get the formatting one wants, markdown is less predictable with different tools and versions If i would start writing a new spec, then id choose lyx if i expect that i wouldnt need to collaborate with others and markdown or some other "easy hand editable in a texteditor" format if collaboration between multiple people is expected [...] --=20 Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB If you think the mosad wants you dead since a long time then you are either wrong or dead since a long time. --vH3HHxf962mwD/qo Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAlfi5dYACgkQYR7HhwQLD6tY5gCfRE+f5pQlOHy+6NoMQuZ671JJ KX4AoIFXQ5zxRdLZ8Qf0ZJxClN5Ae8j+ =1trz -----END PGP SIGNATURE----- --vH3HHxf962mwD/qo-- From nobody Wed Sep 21 13:02:32 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6362812BE30 for ; Wed, 21 Sep 2016 13:02:31 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.12 X-Spam-Level: X-Spam-Status: No, score=-1.12 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bn_r_Sas-aTl for ; Wed, 21 Sep 2016 13:02:29 -0700 (PDT) Received: from s172.web-hosting.com (s172.web-hosting.com [68.65.122.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3FD4312BE2E for ; Wed, 21 Sep 2016 13:02:29 -0700 (PDT) Received: from [146.96.19.240] (port=28405 helo=[10.10.202.53]) by server172.web-hosting.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.86_1) (envelope-from ) id 1bmnir-001MII-8B; Wed, 21 Sep 2016 16:02:28 -0400 Content-Type: multipart/alternative; boundary="Apple-Mail=_3894DBF1-0563-46A7-994F-0813EB6F09AE" Mime-Version: 1.0 (Mac OS X Mail 9.2 \(3112\)) From: Dave Rice In-Reply-To: Date: Wed, 21 Sep 2016 16:02:23 -0400 Message-Id: <0EEF4F55-2102-496D-BFB7-776766385597@dericed.com> References: To: Jerome Martinez X-Mailer: Apple Mail (2.3112) X-OutGoing-Spam-Status: No, score=-2.9 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Cc: cellar@ietf.org Subject: Re: [Cellar] Risk assessment for FFV1 and Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Sep 2016 20:02:31 -0000 --Apple-Mail=_3894DBF1-0563-46A7-994F-0813EB6F09AE Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 Hi all, > On Sep 21, 2016, at 3:48 PM, Jerome Martinez = wrote: >=20 > Le 21/09/2016 =E0 20:49, Casey, Michael T a =E9crit : >> [...] >> What did you think about the risk of going for a not-yet-standardized = codec? Did you somehow take into account the chance that the = standardized FFv1 will look a little different than the one you're using = now? >=20 > The goal of CELLAR is to describe what is currently the most used FFV1 = (definition of FFV1 version 0/1/3), before creating a new (and breaking = compatibility) version 4. > So the standard will not look a little different: it will be the = description of the most used FFV1. This is clarified in the charter = where we are = standardizing the existing formats and working on the next version of = the formats. > As a proof that we prioritize what you are using instead of the draft = of specification, an example: > https://www.ietf.org/mail-archive/web/cellar/current/msg00823.html > You can see that we discovered a little (16-bit YUV is not widely = used) difference between the main implementation and the draft of spec, = we decided to update the draft of spec instead than creating an = incompatibility between the standard and the files in the wild (and = we'll simplify the bitstream format in FFV1 version 4, not yet in the = wild). >=20 > The most used FFV1 encoder/decoder is FFmpeg. > There may be a risk in case you don't use FFmpeg for encoding. Do you? Isn't this just as true with libav as well. Is there another FFV1 = encoder beyond FFmpeg and libav? > Another argument is that as it is lossless compression, it would be = relatively easy to move from the "little different" bitstream to the = "standard" without losing quality, in the case we decide (we won't) to = have a little difference somewhere. >=20 > That said, which FFV1 encoder do you use? I can say that I deeply = reviewed FFmpeg 3.1 (the last one) for FFV1 8-, 10- and 16-bit, for = Gray, YUV and RGB, with Golomb Rice (8-bit) and Range Coder = (8/10/16-bit) coders, with version 3 of FFV1, I can say that I created = another decoder which confirms that the current draft of spec is = correct, so I can say that I have big doubts that the spec and FFmpeg = 3.1 have or will have a little difference. if you use one of these = configurations, go ahead, else provide the configuration you use and = I'll prioritize it in my review. Mike, the beginning of the output of=20 ffmpeg -debug 1 -i MikeCaseysConfiguration.mkv -f null - would answer these questions. > The same applies to Matroska but I am less involved in it, I can only = say that the idea is the same and the risk is only in case you use tools = we don't check (e.g. if you use a super rare muxer) and in case you use = super rare Matroska features (e.g. encryption, deprecated signature...), = else we review files from several muxers e.g. tools using libmatroska or = FFmpeg. Our priority is to have no "little difference" with the most = used muxers. I think it's possible that some Matroska muxer could have exploited the = vagueness of earlier versions of the Matroska spec so that what could be = earlier argued as valid is now clarified as not valid by the CELLAR = work. This is sort of the entire point of standardizing the = documentation though. For instance, someone could have a file with = CRC-32 in the middle of a Master Element of have = are argue for that as not = restricted from pre-CELLAR documents whereas it is clearly invalid from = current CELLAR drafts. But for most sane muxers I think a reverse = compatibility issue is not a big concern. As part of the PREFORMA = project I've been reviewing the validity of ~100,000 Matroska files from = the Internet Archive and such files as the scenario I mentioned almost = never happen. Also on the topic, when we worked on the proposal for the working group, = there was some concern that with the movement of change control and = custody of the formats from Matroska and FFmpeg to the IETF that the = IETF could change the specs to create a spec that do not match the = existing use of the formats. I think this is unlikely as a proposal to = the working group to break the formats would likely not be well-received = unless it is not to fix something or close a design loophole. Many of = the working group members also maintain software that is using these = formats and having a specification for the format that is different from = the use of the format is not likely to get wide support from the group. Best Regards, Dave Rice --Apple-Mail=_3894DBF1-0563-46A7-994F-0813EB6F09AE Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=windows-1252 Hi all,

On Sep 21, 2016, at 3:48 PM, = Jerome Martinez <jerome@mediaarea.net> wrote:

Le = 21/09/2016 =E0 20:49, Casey, Michael T a =E9crit :
[...]
What = did you think about the risk of going for a not-yet-standardized codec? = Did you somehow take into account the chance that the standardized FFv1 = will look a little different than the one you're using now?

The goal of CELLAR is to describe = what is currently the most used FFV1 (definition of FFV1 version 0/1/3), = before creating a new (and breaking compatibility) version 4.
So the standard will not look a little different: it will be = the description of the most used FFV1.

This = is clarified in the charter where we are standardizing the existing = formats and working on the next version of the formats.

As a proof that we prioritize what you are using instead of = the draft of specification, an example:
https://www.ietf.org/mail-archive/web/cellar/current/msg00823.h= tml
You can see that we discovered a little (16-bit = YUV is not widely used) difference between the main implementation and = the draft of spec, we decided to update the draft of spec instead than = creating an incompatibility between the standard and the files in the = wild (and we'll simplify the bitstream format in FFV1 version 4, not yet = in the wild).

The most used FFV1 = encoder/decoder is FFmpeg.
There may be a risk in case you = don't use FFmpeg for encoding. Do you?

Isn't = this just as true with libav as well. Is there another FFV1 encoder = beyond FFmpeg and libav?

Another argument is that as = it is lossless compression, it would be relatively easy to move from the = "little different" bitstream to the "standard" without losing quality, = in the case we decide (we won't) to have a little difference = somewhere.

That said, which FFV1 encoder do = you use? I can say that I deeply reviewed FFmpeg 3.1 (the last one) for = FFV1 8-, 10- and 16-bit, for Gray, YUV and RGB, with Golomb Rice (8-bit) = and Range Coder (8/10/16-bit) coders, with version 3 of FFV1, I can say = that I created another decoder which confirms that the current draft of = spec is correct, so I can say that I have big doubts that the spec and = FFmpeg 3.1 have or will have a little difference. if you use one of = these configurations, go ahead, else provide the configuration you use = and I'll prioritize it in my review.

Mike, = the beginning of the output of 
ffmpeg -debug 1 -i = MikeCaseysConfiguration.mkv -f null -
would answer these = questions.

The same applies to Matroska but I am less = involved in it, I can only say that the idea is the same and the risk is = only in case you use tools we don't check (e.g. if you use a super rare = muxer) and in case you use super rare Matroska features (e.g. = encryption, deprecated signature...), else we review files from several = muxers e.g. tools using libmatroska or FFmpeg. Our priority is to have = no "little difference" with the most used muxers.

I = think it's possible that some Matroska muxer could have exploited the = vagueness of earlier versions of the Matroska spec so that what could be = earlier argued as valid is now clarified as not valid by the CELLAR = work. This is sort of the entire point of standardizing the = documentation though. For instance, someone could have a file with = CRC-32 in the middle of a Master Element of have = <EBML><Segment><Segment><CRC-32><Segment> = are argue for that as not restricted from pre-CELLAR documents whereas = it is clearly invalid from current CELLAR drafts. But for most sane = muxers I think a reverse compatibility issue is not a big concern. As = part of the PREFORMA project I've been reviewing the validity of = ~100,000 Matroska files from the Internet Archive and such files as the = scenario I mentioned almost never happen.

Also on the topic, when we worked on the proposal = for the working group, there was some concern that with the movement of = change control and custody of the formats from Matroska and FFmpeg to = the IETF that the IETF could change the specs to create a spec that do = not match the existing use of the formats. I think this is unlikely as a = proposal to the working group to break the formats would likely not be = well-received unless it is not to fix something or close a design = loophole. Many of the working group members also maintain software that = is using these formats and having a specification for the format that is = different from the use of the format is not likely to get wide support = from the group.

Best = Regards,
Dave Rice

= --Apple-Mail=_3894DBF1-0563-46A7-994F-0813EB6F09AE-- From nobody Thu Sep 22 00:18:01 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CF3D312BE11 for ; Thu, 22 Sep 2016 00:17:59 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.6 X-Spam-Level: X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QXu0oAbu9D57 for ; Thu, 22 Sep 2016 00:17:58 -0700 (PDT) Received: from zucker.schokokeks.org (zucker.schokokeks.org [178.63.68.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id AA07912B0A4 for ; Thu, 22 Sep 2016 00:17:56 -0700 (PDT) Received: from [10.0.0.2] (1360030434.d-dsl.at [::ffff:81.16.106.226]) (AUTH: PLAIN bubestinger@schokokeks.org, TLS: TLSv1/SSLv3, 128bits, ECDHE-RSA-AES128-GCM-SHA256) by zucker.schokokeks.org with ESMTPSA; Thu, 22 Sep 2016 09:17:55 +0200 id 000000000000017E.0000000057E385A3.00002A54 Message-ID: <57E385A1.3060807@das-werkstatt.com> Date: Thu, 22 Sep 2016 09:17:53 +0200 From: "Peter B." User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: "cellar@ietf.org" References: In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [Cellar] Risk assessment for FFV1 and Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Sep 2016 07:18:00 -0000 Dear Michael, Since we started using FFV1 at the Austrian Mediathek in 2010, without assuming it'd become any standard soon, we worked on risk-assessment considerations. If you're interested, I'll write in more detail later :) Kind regards, Peter On 09/21/2016 08:49 PM, Casey, Michael T wrote: > I have a couple of questions from a large digitization project in Europe about the risk associated with choosing FFV1 and Matroska. This is from their email: > > You guys have chosen for FFv1 last year, but the standardisation process is still going on (IETF, CELLAR) as you know. > > What did you think about the risk of going for a not-yet-standardized codec? Did you somehow take into account the chance that the standardized FFv1 will look a little different than the one you're using now? > > I am wondering if you have any thoughts on this that I could pass along? > From nobody Thu Sep 22 00:22:15 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5FE7E12D77F for ; Thu, 22 Sep 2016 00:22:14 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.6 X-Spam-Level: X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id H7WCI1nm-W5c for ; Thu, 22 Sep 2016 00:22:13 -0700 (PDT) Received: from zucker.schokokeks.org (zucker.schokokeks.org [178.63.68.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 915C912D779 for ; Thu, 22 Sep 2016 00:22:12 -0700 (PDT) Received: from [10.0.0.2] (1360030434.d-dsl.at [::ffff:81.16.106.226]) (AUTH: PLAIN bubestinger@schokokeks.org, TLS: TLSv1/SSLv3, 128bits, ECDHE-RSA-AES128-GCM-SHA256) by zucker.schokokeks.org with ESMTPSA; Thu, 22 Sep 2016 09:22:12 +0200 id 0000000000000057.0000000057E386A4.000031B9 Message-ID: <57E386A2.6050908@das-werkstatt.com> Date: Thu, 22 Sep 2016 09:22:10 +0200 From: "Peter B." User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: cellar@ietf.org References: In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [Cellar] Risk assessment for FFV1 and Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Sep 2016 07:22:14 -0000 On 09/21/2016 09:48 PM, Jerome Martinez wrote: > The most used FFV1 encoder/decoder is FFmpeg. > There may be a risk in case you don't use FFmpeg for encoding. Do you? To my knowledge, there are only 2 implementations existing for FFV1 encoder/decoder: 1) FFmpeg 2) LibAV Yet, both originate from the very same code, and FATE tests should make sure that they don't deviate beyond compatibility level. Therefore, I think the danger of having completely different FFV1 files by a badly implemented, proprietary 3rd party encoder is very low :) Does anyone of you know of another FFV1 encoder/decoder? (MediaConch would only count as decoder, I guess :)) Kind regards, Peter From nobody Thu Sep 22 00:37:06 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C135C12D990 for ; Thu, 22 Sep 2016 00:37:03 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.602 X-Spam-Level: X-Spam-Status: No, score=-2.602 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IV5AW2svQOIK for ; Thu, 22 Sep 2016 00:37:01 -0700 (PDT) Received: from smtp-sh.infomaniak.ch (smtp-sh.infomaniak.ch [128.65.195.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6617812D986 for ; Thu, 22 Sep 2016 00:37:01 -0700 (PDT) Received: from smtp6.infomaniak.ch (smtp6.infomaniak.ch [83.166.132.19]) by smtp-sh.infomaniak.ch (8.14.5/8.14.5) with ESMTP id u8M7awvx005461 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL) for ; Thu, 22 Sep 2016 09:36:58 +0200 Received: from Castor.local (dynamic.wline.6rd.res.cust.swisscom.ch [IPv6:2a02:1205:c69b:8640:711e:bc21:dfaa:b57d] (may be forged)) (authenticated bits=0) by smtp6.infomaniak.ch (8.14.5/8.14.5) with ESMTP id u8M7av3P017734 for ; Thu, 22 Sep 2016 09:36:58 +0200 Date: Thu, 22 Sep 2016 09:36:58 +0200 From: Reto Kromer To: cellar@ietf.org X-Priority: 3 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Mailer: Mailsmith 2.4 (470) X-Antivirus: Dr.Web (R) for Unix mail servers drweb plugin ver.6.0.2.8 X-Antivirus-Code: 0x100000 Archived-At: Subject: Re: [Cellar] New thread: Work to be done on FLAC, call for volunteers X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Sep 2016 07:37:04 -0000 Jerome Martinez wrote: >I understand that some people may "hate" it (can we really >hate a markup "language"?) but I argue that this is the >language used by Wikipedia, that we have direct display in >e.g. GitHub (e.g. FFV1 specs=20 >https://github.com/FFmpeg/FFV1/blob/master/ffv1.md ), that >there is a standardization effort of Markdown (including >IETF RFC 7764), and that LaTeX is known by less people than >Markdown. To not be misunderstood: I do *NOT* suggest to use LaTeX here, because I don't think that would simplify the job. Yes, J=C3=A9r=C3=B4me, I'm afraid, I do hate Markdown! In fact it forces me to waste my time. This is my personal position that I don't wish to impose to others. Therefore I do *NOT* volunteer to lead the FLAC standardisation process, but - of course! - I do volunteer to collaborate. And I'm also fine to do so in Markdown. Hope this helps! Reto P.S. - The fact that many likes Markdown is not an argument for me. If the majority decides, then we can stop working on =46FV1, because we are still a niche, a small minority in the AV archival community. From nobody Thu Sep 22 09:53:19 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6E97412D97E for ; Thu, 22 Sep 2016 09:53:18 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.537 X-Spam-Level: X-Spam-Status: No, score=-6.537 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-2.316, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qy1jdu9ULFAq for ; Thu, 22 Sep 2016 09:53:16 -0700 (PDT) Received: from hartman.uits.indiana.edu (belushi.uits.indiana.edu [129.79.1.188]) (using TLSv1.2 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2452012D999 for ; Thu, 22 Sep 2016 09:50:08 -0700 (PDT) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A2D6AgBYC+RX/0kBT4FeGQEBAQEBAQEBA?= =?us-ascii?q?QEBBwEBAQEBgygTAQEBAQEeV3wHp2GTFAEYC4UwSgKBaTwQAQIBAQEBAQEBXie?= =?us-ascii?q?EYQEBAQQBAQE3NBcEAgEIEQQBAQEVCQkHJwsUCQgBAQQBEgiIQwENuBEBBoJnA?= =?us-ascii?q?QEBAQEBAQECAQEBAQEBAQEBGQWGOIRThBYRAVuFHwWZdQGPXo9zjGiDfDUfgxk?= =?us-ascii?q?cgVByhHCBIAF+AQEB?= X-IPAS-Result: =?us-ascii?q?A2D6AgBYC+RX/0kBT4FeGQEBAQEBAQEBAQEBBwEBAQEBgyg?= =?us-ascii?q?TAQEBAQEeV3wHp2GTFAEYC4UwSgKBaTwQAQIBAQEBAQEBXieEYQEBAQQBAQE3N?= =?us-ascii?q?BcEAgEIEQQBAQEVCQkHJwsUCQgBAQQBEgiIQwENuBEBBoJnAQEBAQEBAQECAQE?= =?us-ascii?q?BAQEBAQEBGQWGOIRThBYRAVuFHwWZdQGPXo9zjGiDfDUfgxkcgVByhHCBIAF+A?= =?us-ascii?q?QEB?= X-IronPort-AV: E=Sophos;i="5.30,379,1470715200"; d="scan'208";a="37007996" Received: from mssg-relay.indiana.edu ([129.79.1.73]) by irpt-internal-relay.indiana.edu with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 22 Sep 2016 12:50:08 -0400 Received: from in-cci-exch0c.ads.iu.edu (in-cci-exch0c.ads.iu.edu [10.234.33.139]) by mssg-relay.indiana.edu (8.14.7/8.14.4/IU Campus Communications Team) with ESMTP id u8MGo7IO013987 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NOT); Thu, 22 Sep 2016 12:50:07 -0400 Received: from in-cci-exch08.ads.iu.edu (2001:18e8:3:cc1::10b) by in-cci-exch0c.ads.iu.edu (2001:18e8:3:cc1::10f) with Microsoft SMTP Server (TLS) id 15.0.1210.3; Thu, 22 Sep 2016 12:50:07 -0400 Received: from in-cci-exch08.ads.iu.edu ([fe80::383e:ce1d:3835:718c]) by in-cci-exch08.ads.iu.edu ([fe80::383e:ce1d:3835:718c%18]) with mapi id 15.00.1210.000; Thu, 22 Sep 2016 12:50:07 -0400 From: "Casey, Michael T" To: "'Peter B.'" , "cellar@ietf.org" Thread-Topic: [Cellar] Risk assessment for FFV1 and Matroska Thread-Index: AdIUOMYE9b5vwXxKTa6q4p7s4EadRwAii/yAAAuYXuA= Date: Thu, 22 Sep 2016 16:50:07 +0000 Message-ID: <91c9a832455c44f0ae5de446aeb45d55@in-cci-exch08.ads.iu.edu> References: <57E385A1.3060807@das-werkstatt.com> In-Reply-To: <57E385A1.3060807@das-werkstatt.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [129.79.49.223] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: Subject: Re: [Cellar] Risk assessment for FFV1 and Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Sep 2016 16:53:18 -0000 Peter, Yes, I'm interested and would like to know more details. I suspect that the= European project that I am dialoguing with would also be interested. Thanks, Mike -----Original Message----- From: Cellar [mailto:cellar-bounces@ietf.org] On Behalf Of Peter B. Sent: Thursday, September 22, 2016 3:18 AM To: cellar@ietf.org Subject: Re: [Cellar] Risk assessment for FFV1 and Matroska Dear Michael, Since we started using FFV1 at the Austrian Mediathek in 2010, without assu= ming it'd become any standard soon, we worked on risk-assessment considerat= ions. If you're interested, I'll write in more detail later :) Kind regards, Peter On 09/21/2016 08:49 PM, Casey, Michael T wrote: > I have a couple of questions from a large digitization project in Europe = about the risk associated with choosing FFV1 and Matroska. This is from the= ir email: > =20 > You guys have chosen for FFv1 last year, but the standardisation process = is still going on (IETF, CELLAR) as you know.=20 > =20 > What did you think about the risk of going for a not-yet-standardized cod= ec? Did you somehow take into account the chance that the standardized FFv1= will look a little different than the one you're using now? > =20 > I am wondering if you have any thoughts on this that I could pass along? > _______________________________________________ Cellar mailing list Cellar@ietf.org https://www.ietf.org/mailman/listinfo/cellar From nobody Fri Sep 23 20:08:31 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 458EE12B4F8 for ; Fri, 23 Sep 2016 20:08:27 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.121 X-Spam-Level: X-Spam-Status: No, score=-1.121 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UZpaRsEdARoo for ; Fri, 23 Sep 2016 20:08:23 -0700 (PDT) Received: from s172.web-hosting.com (s172.web-hosting.com [68.65.122.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7660612B6B3 for ; Fri, 23 Sep 2016 20:08:23 -0700 (PDT) Received: from cpe-184-152-63-84.nyc.res.rr.com ([184.152.63.84]:48031 helo=[10.0.1.77]) by server172.web-hosting.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.86_1) (envelope-from ) id 1bndK6-001cAn-BY; Fri, 23 Sep 2016 23:08:22 -0400 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) From: Dave Rice In-Reply-To: <57A9C469.4000509@xiph.org> Date: Fri, 23 Sep 2016 23:08:16 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <6E18AA8A-424C-4452-AB38-2975312CC3B7@dericed.com> References: <578E3BD0.3090106@xiph.org> <57A9C469.4000509@xiph.org> To: "Timothy B. Terriberry" X-Mailer: Apple Mail (2.3124) X-OutGoing-Spam-Status: No, score=-2.9 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Cc: Codec Encoding for LossLess Archiving and Realtime transmission Subject: Re: [Cellar] Confirming consensus to adopt draft-lhomme-cellar-ebml-00 as a WG item X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Sep 2016 03:08:28 -0000 Hi Tim, > On Aug 9, 2016, at 7:54 AM, Timothy B. Terriberry = wrote: >=20 > The chairs believe this consensus is confirmed. Authors, please submit = a new version of the draft as draft-ietf-cellar-ebml-00. For = change-tracking purposes, it may be best to submit a version very close = to what was in draft-lhomme-cellar-ebml-00, and then do an -01 update = with any changes that have happened since then. I submitted draft-ietf-cellar-ebml-00 from a checkout from the same day = as draft-lhomme-cellar-ebml-00 from commit 9ce99c8d. There have been 112 = commits since that point. I suggest posting a -01 update after a few = pull requests are reviewed and merged, in particular: https://github.com/Matroska-Org/ebml-specification/pulls/105 https://github.com/Matroska-Org/ebml-specification/pulls/103 https://github.com/Matroska-Org/ebml-specification/pulls/83 Best Regards, Dave Rice= From nobody Fri Sep 23 20:44:21 2016 Return-Path: X-Original-To: cellar@ietf.org Delivered-To: cellar@ietfa.amsl.com Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 2C70E12B567; Fri, 23 Sep 2016 20:44:19 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: internet-drafts@ietf.org To: X-Test-IDTracker: no X-IETF-IDTracker: 6.33.1 Auto-Submitted: auto-generated Precedence: bulk Message-ID: <147468865917.20393.14263086993215132384.idtracker@ietfa.amsl.com> Date: Fri, 23 Sep 2016 20:44:19 -0700 Archived-At: Cc: cellar@ietf.org Subject: [Cellar] I-D Action: draft-ietf-cellar-ebml-00.txt X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Sep 2016 03:44:19 -0000 A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Codec Encoding for LossLess Archiving and Realtime transmission of the IETF. Title : Extensible Binary Meta Language Authors : Steve Lhomme <> Dave Rice <> Moritz Bunkus <> Filename : draft-ietf-cellar-ebml-00.txt Pages : 27 Date : 2016-09-23 Abstract: This document defines the Extensible Binary Meta Language (EBML) format as a genearlized file format for any type of data in a hierarchical form. EBML is designed as a binary equivalent to XML and utilizes a storage-efficient approach to building nested Elements with identifiers, lengths, and values. Similar to how an XML Schema defines the structure and semantics of an XML Document, this document defines an EBML Schema to convey the semantics of an EBML Document. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-cellar-ebml/ There's also a htmlized version available at: https://tools.ietf.org/html/draft-ietf-cellar-ebml-00 Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From nobody Sat Sep 24 11:05:02 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D2BAB12B01F for ; Sat, 24 Sep 2016 11:04:59 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.101 X-Spam-Level: X-Spam-Status: No, score=-2.101 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cA51RiEW73Cp for ; Sat, 24 Sep 2016 11:04:58 -0700 (PDT) Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9EFE112B008 for ; Sat, 24 Sep 2016 11:04:57 -0700 (PDT) Received: from [192.168.2.129] ([84.61.100.56]) by mail.gmx.com (mrgmx101) with ESMTPSA (Nemesis) id 0Lj61K-1bF5b61mv8-00dHL0 for ; Sat, 24 Sep 2016 20:04:54 +0200 To: cellar@ietf.org References: <147468865917.20393.14263086993215132384.idtracker@ietfa.amsl.com> From: "Sebastian G. " Openpgp: id=BFE90DE515B6F548CDE298939902921C2B944DAE Message-ID: Date: Sat, 24 Sep 2016 20:04:53 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: <147468865917.20393.14263086993215132384.idtracker@ietfa.amsl.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-Provags-ID: V03:K0:rkm7Au7DW9xAFDs8jZZNjgzIgV/SQxEE7tfFZh9+d6aevfOQFX/ nDoHJK2sTN4NZaAydQyIYVKWogklhkXic4QLvSAZ7g48JnDDxWCgEJ/8jo5l1oCYTM47fs1 fnwbIELcgmlJVkt9tPJzNUNq7Z9+thP71xJF2AjA5yHa/xSuSiE+l/A/+26P4Mlap7V1gPR vzBnFJ/FScV09Rej5P4TA== X-UI-Out-Filterresults: notjunk:1;V01:K0:H19GNoYOto4=:rLuLZ3PhEsCC3EClN08MNF t0rKbIIJRJo7T7l1aVb6iVjRSox8r1kQ6poz+9bDCMBpHK7BVpArrUflQKQxjkiXyhfWWjYgw 5T9Hjtkb6UNupWs4/P9tookI1e82Y0vcA+zUHFuE+rOZNdDTgKInPIwyvWEvueeA1GnQwJNSA PIe2GDUaNDSZX7VreID5lrmKdTW2R6PiUFuYtbFidR5+uvApkfON9HFIt0lPIHfMrfz8bnEKr K8+soTwAnjUsHucIwca2JCYpJDg1OwkvZUia1wtGUoPRBpecaYO9YxfhKG1k3y+v50aOfUGjS sw3sfsjpiFRzU17XVUeKk8xpDRiTFwj4vbCShBvHEvdk75AsGfale8wpDOaXAriYMbpN7RR/F 5DI69S9j7lw0I5FGh76lKXpNfgEwAjpWF6kphAt3StDkIrgdkgApBgmP0+skjqXmQaQrpmF8D nksZDBIxVmL5Esxhr+95li29137kaQvZb0Bjig02dtSxFam3+s4F+Rb3WZjcneynHpOPetKOQ xMKwwSpID4sIM0u07b+fiTKN05ElPCBn8zbZeu52tEQ7zKku6sonsQUyCyZt5jfkwBWI+UQzp nIwkrGq3H9Apu/QCj+VYulrwrXVNDEk/CFKUblMpQckekG7ckIsgyz85+EPEwPL1Zik55zxdC jaJCQmXhbXyW1OzV1Cxx8IW44up5kdnA/FBz12j7U+Vh438y9+q0KXNyB9dUZkMWBaJVDS+3q dK6MIXfyUPkVhtsncnthe497CgQ4Gb3wD97IhVwH9zow+irKx7Inr95l09rwqLsIxan/JdJrn 1fTORPv Archived-At: Subject: Re: [Cellar] I-D Action: draft-ietf-cellar-ebml-00.txt X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Sep 2016 18:05:00 -0000 24.09.2016, 05:44 internet-drafts@ietf.org: > > A New Internet-Draft is available from the on-line Internet-Drafts directories. > This draft is a work item of the Codec Encoding for LossLess Archiving and Realtime transmission of the IETF. > > Title : Extensible Binary Meta Language > Authors : Steve Lhomme <> > Dave Rice <> > Moritz Bunkus <> > Filename : draft-ietf-cellar-ebml-00.txt > Pages : 27 > Date : 2016-09-23 I think this date means that this is a recent document. > Abstract: > This document defines the Extensible Binary Meta Language (EBML) > format as a genearlized file format for any type of data in a > hierarchical form. EBML is designed as a binary equivalent to XML > and utilizes a storage-efficient approach to building nested Elements > with identifiers, lengths, and values. Similar to how an XML Schema > defines the structure and semantics of an XML Document, this document > defines an EBML Schema to convey the semantics of an EBML Document. However this abstract has been changed. [0] > > The IETF datatracker status page for this draft is: > https://datatracker.ietf.org/doc/draft-ietf-cellar-ebml/ > And this document, does not include the merged changes either. For the 'lhomme' version I assumed it would have to be the case, because it had to be submitted as it was. Now I wonder if the current document shouldn't have been updated. If it was supposed to be updated I am not aware of other changes that have not been included, yet. Best Regards, Sebastian [0] https://github.com/Matroska-Org/ebml-specification/pull/75/commits/4d777b5ad1b40df2f235ed2db4f1468f6466d464 From nobody Sat Sep 24 11:32:58 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1F0A212B020 for ; Sat, 24 Sep 2016 11:32:57 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.119 X-Spam-Level: X-Spam-Status: No, score=-1.119 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, MIME_QP_LONG_LINE=0.001, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zQ0NLjlCJeP6 for ; Sat, 24 Sep 2016 11:32:55 -0700 (PDT) Received: from s172.web-hosting.com (s172.web-hosting.com [68.65.122.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id F1A4012B008 for ; Sat, 24 Sep 2016 11:32:54 -0700 (PDT) Received: from [172.56.35.213] (port=63513 helo=[IPv6:2607:fb90:2402:343c:6da7:2cfe:811b:53da]) by server172.web-hosting.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.86_1) (envelope-from ) id 1bnrkq-000s4P-He; Sat, 24 Sep 2016 14:32:54 -0400 Content-Type: multipart/alternative; boundary=Apple-Mail-07D3FC2B-2AC5-45AE-B678-6F36BBD9119A Mime-Version: 1.0 (1.0) From: Dave Rice X-Mailer: iPhone Mail (13F69) In-Reply-To: Date: Sat, 24 Sep 2016 14:32:51 -0400 Content-Transfer-Encoding: 7bit Message-Id: <4A7DC26B-5B22-4144-BDC5-61D363764F9A@dericed.com> References: <147468865917.20393.14263086993215132384.idtracker@ietfa.amsl.com> To: "Sebastian G. " X-OutGoing-Spam-Status: No, score=-2.9 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Cc: cellar@ietf.org Subject: Re: [Cellar] I-D Action: draft-ietf-cellar-ebml-00.txt X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Sep 2016 18:32:57 -0000 --Apple-Mail-07D3FC2B-2AC5-45AE-B678-6F36BBD9119A Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Hi Sebastian, > On Sep 24, 2016, at 14:04, Sebastian G. wrote: >=20 > 24.09.2016, 05:44 internet-drafts@ietf.org: >>=20 >> A New Internet-Draft is available from the on-line Internet-Drafts direct= ories. >> This draft is a work item of the Codec Encoding for LossLess Archiving an= d Realtime transmission of the IETF. >>=20 >> Title : Extensible Binary Meta Language >> Authors : Steve Lhomme <> >> Dave Rice <> >> Moritz Bunkus <> >> Filename : draft-ietf-cellar-ebml-00.txt >> Pages : 27 >> Date : 2016-09-23 >=20 > I think this date means that this is a recent document. The content is not recent. This is a (late) attempt to follow Tim's suggesti= ons here: https://mailarchive.ietf.org/arch/msg/cellar/uNTIzKdbFdBersnQ9BNwK= l5tTNg. >> Abstract: >> This document defines the Extensible Binary Meta Language (EBML) >> format as a genearlized file format for any type of data in a >> hierarchical form. EBML is designed as a binary equivalent to XML >> and utilizes a storage-efficient approach to building nested Elements >> with identifiers, lengths, and values. Similar to how an XML Schema >> defines the structure and semantics of an XML Document, this document >> defines an EBML Schema to convey the semantics of an EBML Document. >=20 > However this abstract has been changed. [0] >=20 >>=20 >> The IETF datatracker status page for this draft is: >> https://datatracker.ietf.org/doc/draft-ietf-cellar-ebml/ >=20 > And this document, does not include the merged changes either. >=20 > For the 'lhomme' version I assumed it would have to be the case, because > it had to be submitted as it was. Now I wonder if the current document > shouldn't have been updated. >=20 > If it was supposed to be updated I am not aware of other changes that > have not been included, yet. A new version may be submitted. I suggest waiting until a few of the open pu= ll requests are resolved. Dave > Best Regards, > Sebastian >=20 > [0] > https://github.com/Matroska-Org/ebml-specification/pull/75/commits/4d777b5= ad1b40df2f235ed2db4f1468f6466d464 >=20 > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar --Apple-Mail-07D3FC2B-2AC5-45AE-B678-6F36BBD9119A Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
Hi Sebastian,
On Sep 24, 2016, at 14:04, Sebastian G. <bastik> <bastik.public.mailinglist@gmx.de>= wrote:

24.09.2016, 05:44 <= a href=3D"mailto:internet-drafts@ietf.org">internet-drafts@ietf.org:

A New Internet-Draft is available from the on-line Intern= et-Drafts directories.
This draft is a work item of the Codec Encoding for LossLess Archiving and= Realtime transmission of the IETF.

 =       Title      &nbs= p;    : Extensible Binary Meta Language
      =  Authors         : Steve Lhomme= <>
 &nbs= p;            &n= bsp;          Dave Rice &l= t;>
  &= nbsp;            = ;          Moritz Bunkus &= lt;>
   = ;Filename        : draft-ietf-cellar-ebml= -00.txt
   = ;Pages           : 27
   Date  =           : 2016-09-23

I think this date means that this= is a recent document.

The c= ontent is not recent. This is a (late) attempt to follow Tim's suggestions h= ere: https://mailarchive.ietf.org/arch/msg/cellar/uNTIzKdbFdBe= rsnQ9BNwKl5tTNg.

Abstract:
=   This document defines the Extensible Binary Meta Language= (EBML)
  = format as a genearlized file format for any type of data in a
  hierarchical form. &nb= sp;EBML is designed as a binary equivalent to XML
  and utilizes a storage-efficient a= pproach to building nested Elements
  with identifiers, lengths, and values.  Simil= ar to how an XML Schema
  defines the structure and semantics of an XML Document, this= document
 &nbs= p;defines an EBML Schema to convey the semantics of an EBML Document.=

However this abstract has been chang= ed. [0]

The IETF datatracker status p= age for this draft is:
https= ://datatracker.ietf.org/doc/draft-ietf-cellar-ebml/


= And this document, does not include the merged changes either.<= br>
For the 'lhomme' version I assumed it would have t= o be the case, because
it had to be submitted as it was. Now= I wonder if the current document
shouldn't have been update= d.

If it was supposed to be updated I am no= t aware of other changes that
have not been included, yet.

A new version may be submitt= ed. I suggest waiting until a few of the open pull requests are resolved.
Dave

= --Apple-Mail-07D3FC2B-2AC5-45AE-B678-6F36BBD9119A-- From nobody Sat Sep 24 15:06:02 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3BE1E12B05E for ; Sat, 24 Sep 2016 15:06:01 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.121 X-Spam-Level: X-Spam-Status: No, score=-1.121 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id P_wVcxx-r7Kx for ; Sat, 24 Sep 2016 15:05:59 -0700 (PDT) Received: from s172.web-hosting.com (s172.web-hosting.com [68.65.122.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id AA1B012B05D for ; Sat, 24 Sep 2016 15:05:59 -0700 (PDT) Received: from cpe-184-152-63-84.nyc.res.rr.com ([184.152.63.84]:33142 helo=[10.0.1.77]) by server172.web-hosting.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.86_1) (envelope-from ) id 1bnv53-002UY7-CG for cellar@ietf.org; Sat, 24 Sep 2016 18:05:58 -0400 From: Dave Rice Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Message-Id: <38E61A25-422A-482B-AFC0-0A6735572FBB@dericed.com> Date: Sat, 24 Sep 2016 18:05:55 -0400 To: Codec Encoding for LossLess Archiving and Realtime transmission Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) X-Mailer: Apple Mail (2.3124) X-OutGoing-Spam-Status: No, score=-2.9 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Subject: [Cellar] review of codec_specs.md for defining use of encodings in Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Sep 2016 22:06:01 -0000 Hi all, Here=E2=80=99s a review of the codec_specs.md page in the = matroska-specification repository. I also compared the Codec IDs listed = in this page to a collection of sample files from the Internet Archive = to get an idea on how much this page and real use overlap. Currently there is a page that documents how certain encodings are = stored in Matroska. The original document is on the matroska site at = https://matroska.org/technical/specs/codecid/index.html and there is a = representation in markdown in the matroska-specification repository at = https://github.com/Matroska-Org/matroska-specification/blob/gh-pages/codec= _specs.md. There=E2=80=99s a related version on Haali=E2=80=99s site at = http://haali.su/mkv/codecs.pdf. Within this page three values are used to define Matroska=E2=80=99s = support for an encoding: 1. Codec ID 2. Name 3. Description The Codec ID appears to have a form to it but it is not defined though = should be. The description appears to consistently. The terms =E2=80=98private data=E2=80=99 and the `CodecPrivate Element` = seems to be referenced in the same way. Are these the same things? If = so, they should be made consistent. The phrase "The private data is void.=E2=80=9D is used several times in = the document. Does this mean that the `CodecPrivate Element` is stored = but as an `Empty Element`? Could the `CodecPrivate Element` just not be = present if not needed? The tables of codecs has some odd nested that breaks the table in = markdown. I propose that the data be rearranged into a simple table. Are there encodings that Matroska supports that are not in this list?=20 This Codec ID is listed as =E2=80=9Dsupported later=E2=80=9D but is it = working now? Codec ID: V_MJPEG Sample file: = http://archive.org/download/2VideoMP4MPEG44096x216063625KbpsHiRez/34_Video= _MKV_(mjpeg)________640x480______3166Kbps__________________.mkv Codec ID: V_VP8 There are many files around with this Codec ID. Supported? Sample file: = http://archive.org/download/20140825071_201408/2014-08-25-070-1.mkv Codec ID: V_MPEGH/ISO/HEVC Even more of these files are around but not in the spec. Supported? Sample file: http://archive.org/download/1A71zv37dr/1A71zv37dr.mkv Codec ID: V_VP9 Also popular, but unlisted. Supported? Sample file: http://archive.org/download/yt-adonfjv/Super Mario 64 Disk = Version 64DD - Powerfull Mario song-MGeGIaJr8gc.mkv Codec ID: V_MPEG4/ISO/AVC This is possibly the most widely used Codec ID in Matroska, but = unlisted. Supported? Sample: http://archive.org/download/00013862/0001-3862.mkv Codec ID: A_AAC This is very popular as a Matroska audio codec, but it deviates from the = forms listed in codec_specs.md Sample file: http://archive.org/download/150069/150069.mkv Codec ID: A_OPUS Opus is common in Matroska but unlisted here. Supported? Sample; = http://archive.org/download/JigwallysNesTheGreatWaldoSearchIn0052.41/great= waldosearch-tas-jigwally.mkv Codec ID: A_EAC3 More rare, but there are around. Unlisted in specs. Sample file: = http://archive.org/download/Barcelona1VS1EspanyolPenaltiesAndCrowning720pH= DBarcelonaHD.blogspot.com/Barcelona 1 VS 1 Espanyol penalties and = crowning 720pHD Barcelona-HD.blogspot.com.mkv Codec ID: A_TRUEHD Unlisted but found in real world. Supported? Sample file: = http://archive.org/download/mad.max.fury.road.2015.1080p.bluray.hevc.trueh= d.7.1.atmos.absinth/Mad.Max.Fury.Road.2015.1080p.BluRay.TrueHD.7.1.ATMOS-A= bsinth.mkv Note that for many of the Codec IDs noted that are not in the Matroska = specification, they are in the webm specification at = http://www.webmproject.org/docs/container/: such as V_VP8, V_VP9, = A_OPUS. Are these valid in Matroska as well? A discussion on adding = these was started at = https://lists.matroska.org/pipermail/matroska-devel/2015-October/004852.ht= ml but did not resolve. Other questions: Should the Matroska specification provide enough info for a user to = independently define support for an encoding in Matroska? Or should = codec support somehow be =E2=80=98registered=E2=80=99 via Matroska = and/or CELLAR? Can the structure of a Codec mapping be more clearly defined, so that a = user could define their own support and understand if it is correct or = not? For a Matroska validation that finds something like V_MPEG4/ISO/AVC in = Matroska. Should it be considered as unsupported (most MKV files!)? Best Regards, Dave Rice From nobody Sat Sep 24 22:45:13 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7B78D12B0CD for ; Sat, 24 Sep 2016 22:45:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.601 X-Spam-Level: X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KUqx11cUwKGv for ; Sat, 24 Sep 2016 22:45:09 -0700 (PDT) Received: from mout.gmx.net (mout.gmx.net [212.227.15.18]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 670F412B0CE for ; Sat, 24 Sep 2016 22:45:08 -0700 (PDT) Received: from [192.168.2.129] ([84.61.100.56]) by mail.gmx.com (mrgmx001) with ESMTPSA (Nemesis) id 0LqylH-1bIZQg2rv1-00efWB; Sun, 25 Sep 2016 07:44:17 +0200 To: Dave Rice References: <147468865917.20393.14263086993215132384.idtracker@ietfa.amsl.com> <4A7DC26B-5B22-4144-BDC5-61D363764F9A@dericed.com> From: "Sebastian G. " Openpgp: id=BFE90DE515B6F548CDE298939902921C2B944DAE Message-ID: Date: Sun, 25 Sep 2016 07:44:13 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: <4A7DC26B-5B22-4144-BDC5-61D363764F9A@dericed.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:a8fUV7TQGaQR7on01/ZRqJ++rcRrqArccaejxJaBlvlvApFXPYD BBboQ0eWQxD+LlWjBKT74ofA7X80R4z/DiJf9Hmn4k+mNTMoBtSKfekVYqhHNTf8JfLIUTr 6irw/06V6EMAj8Pwk9Xya8gkZnGtwg+0X0XdLbDdiHMSGE6lpAQaWDwDa3Jf/3nrRk29ohC +VJMrtzYWOLDemGXejYBA== X-UI-Out-Filterresults: notjunk:1;V01:K0:drXhFySLIkE=:yn/ZeJqw0Bs5z7uK/15gtx VjnSxVMmot/tH8O0MtjWPMUyKb7xTyijAyYHO7GN7GSmKqQNbUvb8QNgb0WNVazFyoEL+uFCI n/Ulfic0TOaAXvY//9EX3B/WutPeVGsiHdjevgr62OV8r/LKZkji6ku7fPW2OO0U008qFbnHD YiOhAmSieROUFiyEya6ezIlmSEur3Em7cH3B79nXegIQGOiSdmtUu1WHF0yVjikB4lvldLlEW p4QdComBNboUfi8N/hAj8+XQqC8dhJZtXNqMZHRV1uy7dh4NeOTEk3zrTndnjAg17BLRdE9h7 Gp3hL4b9RWzfL7c2cja72jBA1zKLBN9d6KFd9xH2ctnYKlnJurbxZLvnCukZkdHTe7ArQgDmv 9tWsBaoPOwxm7P20Q+EMSJohIzaym78zwrfrmPTp2Uw51T+0SaVRnPEUJVEDUbruLfA2YB79M m1DAJ6t4cXQTy8q13Rv5w6bZ+vPfh1mqNytwAH/JdJd+bro1a4OiRl84U/YuWCq9TKQA+n2/A slg6R3h4rAbcAIGeC+zb3+1kLFwLbGmkriV6rNwAhNVtNRUC+G3NjLVPNRoqk5Do5ASyxEG7W RV60jr7K2vXpoym8qMDQZS5qqRTxilI5olCIQU9y2m4YBIzXtbQD9lkBlzSyNMXTh7yHDw6iE g6xZXUzrtX4ZR+rJguaA+8Ekjbm7jsD9Ix22wuxusoh9iT2sXkcGEqxP5//XnSqdoaH4KEaI0 UyLxuAtERiSshZ722MrmWAgskf4NazxrUQy46Y3EwSigtj8vW0sXuog6x4BTIzO0VmvEw0v86 tfEcR/d Archived-At: Cc: cellar@ietf.org Subject: Re: [Cellar] I-D Action: draft-ietf-cellar-ebml-00.txt X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Sep 2016 05:45:11 -0000 24.09.2016, 20:32 Dave Rice: > Hi Sebastian, > >> On Sep 24, 2016, at 14:04, Sebastian G. wrote: >> >> 24.09.2016, 05:44 internet-drafts@ietf.org: >>> >>> A New Internet-Draft is available from the on-line Internet-Drafts directories. >>> This draft is a work item of the Codec Encoding for LossLess Archiving and Realtime transmission of the IETF. >>> >>> Title : Extensible Binary Meta Language >>> Authors : Steve Lhomme <> >>> Dave Rice <> >>> Moritz Bunkus <> >>> Filename : draft-ietf-cellar-ebml-00.txt >>> Pages : 27 >>> Date : 2016-09-23 >> >> I think this date means that this is a recent document. > > The content is not recent. This is a (late) attempt to follow Tim's suggestions here: https://mailarchive.ietf.org/arch/msg/cellar/uNTIzKdbFdBersnQ9BNwKl5tTNg. Now I understand that the contents of the document have not been updated. >>> Abstract: >>> This document defines the Extensible Binary Meta Language (EBML) >>> format as a genearlized file format for any type of data in a >>> hierarchical form. EBML is designed as a binary equivalent to XML >>> and utilizes a storage-efficient approach to building nested Elements >>> with identifiers, lengths, and values. Similar to how an XML Schema >>> defines the structure and semantics of an XML Document, this document >>> defines an EBML Schema to convey the semantics of an EBML Document. >> >> However this abstract has been changed. [0] >> >>> >>> The IETF datatracker status page for this draft is: >>> https://datatracker.ietf.org/doc/draft-ietf-cellar-ebml/ >> >> And this document, does not include the merged changes either. >> >> For the 'lhomme' version I assumed it would have to be the case, because >> it had to be submitted as it was. Now I wonder if the current document >> shouldn't have been updated. >> >> If it was supposed to be updated I am not aware of other changes that >> have not been included, yet. > > A new version may be submitted. I suggest waiting until a few of the open pull requests are resolved. > Dave I am OK with that. Best Regards, Sebastian p.s.: Just to have my understanding and my position on the list anyway, as you say that responses are helpful. >> Best Regards, >> Sebastian >> >> [0] >> https://github.com/Matroska-Org/ebml-specification/pull/75/commits/4d777b5ad1b40df2f235ed2db4f1468f6466d464 >> From nobody Sat Sep 24 23:47:12 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 54E8E12B0DB for ; Sat, 24 Sep 2016 23:47:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.6 X-Spam-Level: X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 29WYd9Dd6nJf for ; Sat, 24 Sep 2016 23:47:09 -0700 (PDT) Received: from mout.gmx.net (mout.gmx.net [212.227.15.15]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4CDC412B0D9 for ; Sat, 24 Sep 2016 23:47:08 -0700 (PDT) Received: from [192.168.2.129] ([84.61.100.56]) by mail.gmx.com (mrgmx003) with ESMTPSA (Nemesis) id 0LreCz-1apKMc1lgD-013KqR; Sun, 25 Sep 2016 08:46:23 +0200 To: cellar@ietf.org References: <38E61A25-422A-482B-AFC0-0A6735572FBB@dericed.com> From: "Sebastian G. " Openpgp: id=BFE90DE515B6F548CDE298939902921C2B944DAE Message-ID: <295a496f-03b0-b78e-b8fc-b36887649825@gmx.de> Date: Sun, 25 Sep 2016 08:46:19 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: <38E61A25-422A-482B-AFC0-0A6735572FBB@dericed.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Provags-ID: V03:K0:geC4I+yYqPkVWgHqphhPpLrBi64UX+GMEPZRPXFTJhUAhs4L0fJ rF8jT5tZePwd5m88pRV7UQCivAOuDpgdM2e1x1SaZhry+4Z3UMV28V4AhNiiX+U2VEb2DmB /Ow/HeGRMzd+capTFJlACPDUO1G5xfOefNmODP+WBeMKXsNzSuz6A/5n0htnni/23mwrf6R hhC6ZYDL9bMX5arwpUNtA== X-UI-Out-Filterresults: notjunk:1;V01:K0:8xrQtsRTg+g=:hN/BGsCAduQR7PIfuS0nsq u76MdQIjSy09VhUHCT1tUxhvcikGEJ49FFOGHJ5ftHnIAfNEe+T76OTNFNj+xcyivAvb7lpny 8xm6r0UO6Q40YtGVW+AP7/q5vmXaLlcu2LIYzWHHLgTgewPx//CCraNYfHK9FiWnl7qokTwyQ YdOsBPDJyie5f9go4CiJOTqvzHN+LaBSXHamgYwOrkphXF3AlLcPNHKuvWAs1tPsZu9sOoPKH qLdS1KUcHmJ9P60nosYszr3+FG01o8Jp97mhKJtSzQloGX/839dbbzdDMf+251jC8Uq4+OeRv Hd0mCew8QCuvaPr03F2yeBWuSU8gZK9N5SumBS2DFXmr3p1bniR2rPx2NoP9KkWCkfRGsmO59 Bh2jJV9TlQo1l8osEXJeqY0RGoQvahUw5bdv9KTUvCy+/cc/mmxFeY0Krw6J1KwzzgKCjjzrP OjBh46xdqAA08tfTgU9U4GOh6njGfdt6RRP2vf1FWLP/CFIxCnVbHOEV7qVW6As87qXHZFI2q zPuwGyzx/8urh1vm1lcuPPKCxelHjQ1P01dNsVvBABw+W9sp3ju21eSf5vprHXfaP89giTPEU 2wb01KtyeoGHdVoeofVZ1R39c9s/N9V/6PvQqvER9Baq7WFR3eWz34gEDrGItpdo4a9/Jqagr /Hhwa5eBBh9iJAwrxMXwm9yLNHJkBy+4DZM4PJTq02rN2Wl6lQnTyMYcY5b7mXKViEDqwfkvz t73YrFMYekOLJLpmnBXhoZN83aNJb8yqSspZ9IVqol7VaoRQ/wHbPBtyi0CW3jMo/lsOtbatT KHZhxiv Archived-At: Cc: Dave Rice Subject: Re: [Cellar] review of codec_specs.md for defining use of encodings in Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Sep 2016 06:47:11 -0000 25.09.2016, 00:05 Dave Rice: > Hi all, > > Here’s a review of the codec_specs.md page in the matroska-specification repository. I also compared the Codec IDs listed in this page to a collection of sample files from the Internet Archive to get an idea on how much this page and real use overlap. > > Currently there is a page that documents how certain encodings are stored in Matroska. The original document is on the matroska site at https://matroska.org/technical/specs/codecid/index.html and there is a representation in markdown in the matroska-specification repository at https://github.com/Matroska-Org/matroska-specification/blob/gh-pages/codec_specs.md. There’s a related version on Haali’s site at http://haali.su/mkv/codecs.pdf. > > Within this page three values are used to define Matroska’s support for an encoding: > > 1. Codec ID > 2. Name > 3. Description > > The Codec ID appears to have a form to it but it is not defined though should be. The obvious pattern to me is that a Codec ID starts with an V for a video codec, with an A for a audio codec and with an S for subtitles (and B for button). Then the Codec ID includes the name in some way. For subtitles it states if it is a text or image based subtitle, though not for all of them. > > The description appears to consistently. > > The terms ‘private data’ and the `CodecPrivate Element` seems to be referenced in the same way. Are these the same things? If so, they should be made consistent. > > The phrase "The private data is void.” is used several times in the document. Does this mean that the `CodecPrivate Element` is stored but as an `Empty Element`? Could the `CodecPrivate Element` just not be present if not needed? > > The tables of codecs has some odd nested that breaks the table in markdown. I propose that the data be rearranged into a simple table. I assume that the separation of video, audio, subtitles and other codec related stuff will be possible with a simple table, in which case I would be OK with replacing the nested table with a simple one. > Are there encodings that Matroska supports that are not in this list? Yes, I think so. VP9 and OPUS should be already supported, but are not listed. There might be even more. > This Codec ID is listed as ”supported later” but is it working now? > Codec ID: V_MJPEG > Sample file: http://archive.org/download/2VideoMP4MPEG44096x216063625KbpsHiRez/34_Video_MKV_(mjpeg)________640x480______3166Kbps__________________.mkv > > Codec ID: V_VP8 > There are many files around with this Codec ID. Supported? > Sample file: http://archive.org/download/20140825071_201408/2014-08-25-070-1.mkv I think it is the basis for WebM, so either it had been supported before or Google would have added support for it to WebM. > Codec ID: V_MPEGH/ISO/HEVC > Even more of these files are around but not in the spec. Supported? > Sample file: http://archive.org/download/1A71zv37dr/1A71zv37dr.mkv > > Codec ID: V_VP9 > Also popular, but unlisted. Supported? > Sample file: http://archive.org/download/yt-adonfjv/Super Mario 64 Disk Version 64DD - Powerfull Mario song-MGeGIaJr8gc.mkv The same that applies to WebM I think. > Codec ID: V_MPEG4/ISO/AVC > This is possibly the most widely used Codec ID in Matroska, but unlisted. Supported? > Sample: http://archive.org/download/00013862/0001-3862.mkv > > Codec ID: A_AAC > This is very popular as a Matroska audio codec, but it deviates from the forms listed in codec_specs.md > Sample file: http://archive.org/download/150069/150069.mkv > > Codec ID: A_OPUS > Opus is common in Matroska but unlisted here. Supported? > Sample; http://archive.org/download/JigwallysNesTheGreatWaldoSearchIn0052.41/greatwaldosearch-tas-jigwally.mkv There have been changes to Matroska in order to support pre-roll for OPUS. There is a related email, but I can't find the list thread discussing changes to Matroska.[0] > Codec ID: A_EAC3 > More rare, but there are around. Unlisted in specs. > Sample file: http://archive.org/download/Barcelona1VS1EspanyolPenaltiesAndCrowning720pHDBarcelonaHD.blogspot.com/Barcelona 1 VS 1 Espanyol penalties and crowning 720pHD Barcelona-HD.blogspot.com.mkv > > Codec ID: A_TRUEHD > Unlisted but found in real world. Supported? > Sample file: http://archive.org/download/mad.max.fury.road.2015.1080p.bluray.hevc.truehd.7.1.atmos.absinth/Mad.Max.Fury.Road.2015.1080p.BluRay.TrueHD.7.1.ATMOS-Absinth.mkv > > Note that for many of the Codec IDs noted that are not in the Matroska specification, they are in the webm specification at http://www.webmproject.org/docs/container/: such as V_VP8, V_VP9, A_OPUS. Are these valid in Matroska as well? A discussion on adding these was started at https://lists.matroska.org/pipermail/matroska-devel/2015-October/004852.html but did not resolve. > > Other questions: A question that I see is: What about Codec ID other people used and got popular? Should they just be included in the codec spec? And even if they go against some naming scheme? > Should the Matroska specification provide enough info for a user to independently define support for an encoding in Matroska? Or should codec support somehow be ‘registered’ via Matroska and/or CELLAR? I quote the codec specs: "[...] to list those codecs that are currently supported in matroska (or will be supported soon), and therfore need a well defined codec ID so that all developers supporting matroska will use the same ID." (There is a spelling mistake in 'therfore') I'm not sure if people come up with the same name independently for every possible new format, even if the spec contains detailed information on how to derive them. Well, arguably the quoted section did not turn out well, as you show examples where there are no defined IDs, but there are ones that are used. However, to me it still appears to be better to have a governing body to ensure that in the long run, if not from the beginning of the usage of an ID, that only one Codec ID will be used. > Can the structure of a Codec mapping be more clearly defined, so that a user could define their own support and understand if it is correct or not? Certainly for the sake of clarity, to increase understanding and avoid errors or mismatches some information on how it is supposed to work should be included. > For a Matroska validation that finds something like V_MPEG4/ISO/AVC in Matroska. Should it be considered as unsupported (most MKV files!)? For the sake of supporting the current state, there could be a primary Codec ID and a secondary Codec ID. While a muxer that complies with the specs would write the primary ID, a demuxer could support both. The file would remain valid in the sense that a player would not refuse it. A validator could point out that an .mk? contains a secondary Codec ID, that better should be replaced with the primary Codec ID. Tools that would aim to bring files in a (more) compliant state would replace the secondary ID with the primary ID, unless the user would choose not to, because his player would not understand the primary ID. > Best Regards, > Dave Rice > Best Regards, Sebastian [0] https://lists.matroska.org/pipermail/matroska-devel/2013-September/004573.html From nobody Sun Sep 25 00:53:07 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1DE7012B0E2 for ; Sun, 25 Sep 2016 00:53:06 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IVj0WbGUtFOr for ; Sun, 25 Sep 2016 00:53:03 -0700 (PDT) Received: from mail-yb0-x22e.google.com (mail-yb0-x22e.google.com [IPv6:2607:f8b0:4002:c09::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 68AF612B0BE for ; Sun, 25 Sep 2016 00:53:03 -0700 (PDT) Received: by mail-yb0-x22e.google.com with SMTP id x93so81794474ybh.1 for ; Sun, 25 Sep 2016 00:53:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-transfer-encoding; bh=DGYCchFgxHziFVZacx8em5/llZJ7ITAppWU2PVZXvzg=; b=dK3OLgDysmLTTgDMetzHlO9deZ0D0BpLuUO1UolIZnixxf7QfiX57nv7s61BdpcZp3 4piTTAADaZrK3KaDb5RdMK0cpVd93bHmYJUs7jAcNiC7Aclv4lTChEOZyoQDxiDb0S8a oID60sK1yLNaQ+rciXG3ysZdI4aSFh+Yx0lhhQsEH7Ifo3XvNkjxd58wYwrhCam04F45 4fuBZfUEoj0bHstT8yVOreBCwnpZQE16hju47WJR2h3wPWWjr/QHVfUa9edvNrInD/O7 mdk+wosakXb+DvI1qteKzi7WEzVHaYloDzkkqNO8p5BqGjsijhbcQtagaNL6da7/Cu4m 5c9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-transfer-encoding; bh=DGYCchFgxHziFVZacx8em5/llZJ7ITAppWU2PVZXvzg=; b=RrOczGqAREfLUOYTGDCawhNpVqqNxEuXaEqtwrt5R4tCcXtxxsOfVfBILePY4JDd/c WrscGHL9vdXxF4ABakP8BYp2vKwoh9FY/3fvwEslcratiA1P6WGs+FORhH+9DXOKH6e6 II1EVWiBMpAMZ2kAOufRLIemXXoIiHcenGj1JchVRw8mm/O15ZcbwXJzEJgYXsI3oiJ3 2WQkd3fNx9oyiM+IybI4uMuFlrmcLl7ubeDPXqut21WtY8TWSbpwMY8+W6HhWmqUYKU8 Dt5ttDhQp0yfqOvAdBaHYyZeCLB/Ya0L9KjrylsMtiz3/k7iMqAWZciIfzzXOP91ulWO zRoQ== X-Gm-Message-State: AA6/9Rn/4u3pDIRX7nCmh3ZYLwNMLkgGMc5UT9KoVSRw4Y3eH/P4V5PNBCg7LszilYViNix4lcmVATb3ZS17gQ== X-Received: by 10.37.53.68 with SMTP id c65mr9022858yba.64.1474789982301; Sun, 25 Sep 2016 00:53:02 -0700 (PDT) MIME-Version: 1.0 Received: by 10.83.20.135 with HTTP; Sun, 25 Sep 2016 00:53:01 -0700 (PDT) In-Reply-To: <7208DD13-54A2-4C49-AF09-26A6E52C74A9@dericed.com> References: <4DC31048-4D9E-4039-B576-ECDC434A6A1E@dericed.com> <7208DD13-54A2-4C49-AF09-26A6E52C74A9@dericed.com> From: Steve Lhomme Date: Sun, 25 Sep 2016 09:53:01 +0200 Message-ID: To: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: Subject: Re: [Cellar] EBML Element Parenting X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Sep 2016 07:53:06 -0000 We keep on updating the draft for this (big) change on Github. To defined CRC-32 and Void that can have all kinds of parents we need some kind of regular expression. For now I think `/+` and `/*` respectively are good candidates to describe them. For the parent description we need to make these expressions unambiguously interpreted so it should be based on some regular expression format. We don't want to reinvent the wheel here so something already standardized would be better. We need a format that allows: - basic paths (`/xx/yy/c`) - any path possible (`/*`) - any path possible except the root (`/+`) optional but welcome: - repeatable path that would describe programmatically how the recursion works (`/Tags/Tag[/SimpleTag]+`) - a combination of pathes (`/xx/yy/[a|b|c]`) While thinking about this it becomes clearer that what we need for each element is not to define the parent (or the level) but actually its valid path(es). even more optional - a way to make pathes mutually exclusive - a way to make pathes relate to each other (all timestamps that depend on the timestampscale) - a way to mark elements that must both be present (TimestampScaleNumerator & TimestampScaleDenominator) By adopting a regular expression format we may impose some restrictions on what is possible too. A very complex regular expression handling would mean valid parsers would be required to handle complex files for no good reasons. 2016-08-23 21:00 GMT+02:00 Dave Rice : > >> On Aug 23, 2016, at 2:41 PM, Steve Lhomme wrote: >> >> 2016-08-23 20:39 GMT+02:00 Dave Rice : >>> >>>> On Aug 23, 2016, at 2:35 PM, Steve Lhomme wrote= : >>>> >>>> I updated the pull request with something that merges with the current= master. >>>> >>>> Anyone disagree with the move before I merge it ? >>> >>> I've been thinking about it and have slowly starting thinking that this= is a better approach than I first thought. It does deviate from the XML Sc= hema analogy but maybe the complexity of XML Schemas aren't for the best he= re. You're right that it would be easier to add extensions (divx) with this= approach. It also resolves the issue with xml being normative, since the t= ext and xml versions would be more semantically identical without depending= on xml nesting. >>> >>>> The next steps after that: >>>> - unflatten the XML definition >>> >>> What do you mean? >> >> >> >> >> Container of data and >> attributes representing one or many files. >> >> > maxOccurs=3D"unbounded"> >> An attached file. >> >> > minOccurs=3D"1"> >> Filename of the attached >> file. >> >> > minOccurs=3D"1"> >> MIME type of the >> file. >> >> > type=3D"date" minOccurs=3D"1"> >> Modification timestamp o= f >> the file. >> >> > minOccurs=3D"1"> >> The data of the >> file. >> >> > > Ah, I would have considered that action as 'flattening', but I support th= e example that we'll use parent rather than nested elements. > >> No more hierarchy that could potentially interfere with the one >> defined in "parent". Also as discussed before elements like Void have >> no real hierarchy. > > +1 > Dave > >>>> - remove the level altogether, including all text references and in >>>> the Matroska specs. >>> >>> Yes. >>> Dave >>> >>>> 2016-08-05 17:34 GMT+02:00 Steve Lhomme : >>>>> 2016-08-05 17:26 GMT+02:00 Dave Rice : >>>>>> >>>>>>> On Aug 5, 2016, at 11:13 AM, Steve Lhomme wr= ote: >>>>>>> >>>>>>> 2016-08-05 17:09 GMT+02:00 Steve Lhomme : >>>>>>>> 2016-08-02 2:50 GMT+02:00 Dave Rice : >>>>>>>>> >>>>>>>>> On Jul 31, 2016, at 11:41 AM, Steve Lhomme = wrote: >>>>>>>>> >>>>>>>>> As we're moving towards splitting the Matroska specs in many smal= ler >>>>>>>>> documents, putting not essential things in a document it becomes = more >>>>>>>>> important to define things differently than the "level" element w= e >>>>>>>>> currently rely on. It makes it very difficult to define a single >>>>>>>>> element without having to copy all its parents to explain where i= t >>>>>>>>> should go in the EBML format. >>>>>>>>> >>>>>>>>> I think we should set the name of the parent(s) explicitely to av= oid >>>>>>>>> that confusion. We will also not rely anymore on the order in whi= ch >>>>>>>>> elements are defined in the specs. >>>>>>>>> >>>>>>>>> I did a pull request with this addition: >>>>>>>>> https://github.com/Matroska-Org/ebml-specification/pull/83 >>>>>>>>> >>>>>>>>> This change will probably mean many paragraphs need to be rewritt= en to >>>>>>>>> rely less on the level system. >>>>>>>>> >>>>>>>>> I will do the pull request for Matroska as well once it's settled= . >>>>>>>>> >>>>>>>>> >>>>>>>>> Sorry to join this conversation late. I think the current state o= f the EBML >>>>>>>>> implies two different ways of documenting nesting. The EBML Schem= a example >>>>>>>>> simply nests the within one another. For example: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> maxOccurs=3D=E2=80=9Cunbounded"> >>>>>>>>> >>>>>>>> minOccurs=3D=E2=80=9C1=E2=80=9D/> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> The nesting of the EBML Schema implies that FileName occurs withi= n File and >>>>>>>>> File occurs within Files. The use of the =E2=80=98level=E2=80=99 = attribute is redundant to >>>>>>>>> the structure. >>>>>>>> >>>>>>>> Yes it's redundant and should probably be removed. It only creates >>>>>>>> possible issues and gives no extra information. Except when it's 1= + >>>>>>>> which doesn't have any equivalent in XML and is currently not defi= ned. >>>>>>>> >>>>>>>>> However, later we list out the elements of the EBML Header in a s= eries of >>>>>>>>> tables. The tables don=E2=80=99t imply the nested structure as th= e xml does, so the >>>>>>>>> level attribute here is needed, though it is certainly a bit uncl= ear since >>>>>>>>> determining the parent requiring reading backwards to the last ma= ster >>>>>>>>> element of a higher value. >>>>>>>>> >>>>>>>>> I prefer to keep the XML analogy where elements are defined in an= XML schema >>>>>>>>> where the arrangement of the elements within one another defines = the nesting >>>>>>>>> structure. Then since we can compile our documentation the =E2=80= =9CParent=E2=80=9D values >>>>>>>>> can be decipher from the XML-based EBML Schema. However this gets= to a >>>>>>>>> conversation at the CELLAR working group meeting on whether XML s= hould be >>>>>>>>> considered normative or not. >>>>>>>> >>>>>>>> While I support having an up to date machine readable version of t= he >>>>>>>> specs. I don't think it should/will be the actual reference for th= e >>>>>>>> specifications. >>>>>>>> >>>>>>>> Also the XML parent analogy only works if there's a fixed parent a= nd >>>>>>>> the element can only be found at that level. How do you define an >>>>>>>> element that can be found anywhere under /File/Attribute/* (to use= the >>>>>>>> path system) (or elements similar to Void and CRC32 which can be >>>>>>>> defined for a certain DocType) ? Either we add an attibute in the = XML >>>>>>>> element or we define how level with a + should be interpreted. Sin= ce >>>>>>>> it's extra information it's probably better to put it out of the l= evel >>>>>>>> (which should go from the XML anyway). How would you define an ele= ment >>>>>>>> that be put anywhere within a DocType in the XML format ? I agree = that >>>>>>>> it would be odd to have a "parent" attribute in a non flat XML fil= e. >>>>>>>> It's also odd to have a "parent" on a flat XML file. >>>>>>> >>>>>>> Also there's still the problem how to define a (set of) element(s) >>>>>>> outside of the core Matroska speficications. >>>>>> >>>>>> You mean to define a custom extra section to Matroska? >>>>> >>>>> Yes, outside of the original spec. For example if a company does like >>>>> DivX and creates a set of new elements. We don't want to update the >>>>> original Matroska specs for that. >>>>> >>>>>>> I can't be the core >>>>>>> specifications including an external file. It should be that extern= al >>>>>>> (XML) that tells where it should be inserted. Is there a common XML >>>>>>> mechanism to inject one file into another ? >>>>>> >>>>>> Yes, in XML Schema there is a way to define an element to contain xs= d:any meaning that any elements may occur within it. Within that section th= e sub-element should ideally have a namespace that points to another XML Sc= hema that defines that particular section. >>>>>> >>>>>>> Does it involve defining >>>>>>> all the elements in the original file to define where we want the >>>>>>> insertion ? With possible mismatch or missing updates. >>>>>> >>>>>> It involves the parent format defining where the flexibility can occ= ur so that any valid XML is allowed in a defined area and then the contents= of that optionally being defined by another schema. >>>>> >>>>> That does not sound very practical/flexible in our case. Or we'd have >>>>> to define all Master elements with such a system. And that would have >>>>> to be done for all EBML-based formats. >>>>> >>>>>> dave >>>>>> >>>>>>>>> So I think I would prefer not to have a =E2=80=98parent=E2=80=99 = attribute but to allow that >>>>>>>>> to be implied by the nesting of elements within an EBML Schema. T= hen the >>>>>>>>> parent data could be presented for users when we compile document= ation from >>>>>>>>> the EBML Schema. For instance this markdown is build from the EBM= L Schema >>>>>>>>> and lists parent values derived from the structure of the EBML Sc= hema. >>>>>>>>> https://gist.github.com/dericed/dd1710e47dd3b6779463a85e4fc0f788 >>>>>>>>> >>>>>>>>> Dave Rice >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Steve Lhomme >>>>>>>> Matroska association Chairman >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Steve Lhomme >>>>>>> Matroska association Chairman >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Cellar mailing list >>>>>>> Cellar@ietf.org >>>>>>> https://www.ietf.org/mailman/listinfo/cellar >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Steve Lhomme >>>>> Matroska association Chairman >>>> >>>> >>>> >>>> -- >>>> Steve Lhomme >>>> Matroska association Chairman >>>> >>>> _______________________________________________ >>>> Cellar mailing list >>>> Cellar@ietf.org >>>> https://www.ietf.org/mailman/listinfo/cellar >>> >> >> >> >> -- >> Steve Lhomme >> Matroska association Chairman >> >> _______________________________________________ >> Cellar mailing list >> Cellar@ietf.org >> https://www.ietf.org/mailman/listinfo/cellar > --=20 Steve Lhomme Matroska association Chairman From nobody Sun Sep 25 01:11:51 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9C34212B0E2 for ; Sun, 25 Sep 2016 01:11:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.6 X-Spam-Level: X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id o--nfUmZP6yL for ; Sun, 25 Sep 2016 01:11:48 -0700 (PDT) Received: from mail-yw0-x231.google.com (mail-yw0-x231.google.com [IPv6:2607:f8b0:4002:c05::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id F207A12B0BE for ; Sun, 25 Sep 2016 01:11:47 -0700 (PDT) Received: by mail-yw0-x231.google.com with SMTP id t67so140878010ywg.3 for ; Sun, 25 Sep 2016 01:11:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-transfer-encoding; bh=ojSxtEyCVSWyl9R5sz/WDjE3la4F2F+ge56YWnKhOT0=; b=kEelabH3lUh5Kw9gl/C6v2Lud3473RuPAbgwBjaAkT2h9t9dR9JCwS4A0c0I9/SFoj vEAfQ6YnztSsy/GVOvGek0lmRCM65hpjbycZP97uNSMEJqdVN2Z6N9L4Wt/2nhKT+6qc Pcz418sE9Wz6KvzjoanHMUu0sFPRFWIgDZJpH2JVv7Rv2LfMixUNIJffIEo0LfFHiiO5 p8yYGL9Dbpvyph2iw40TF9zCAP+yeHTHKb/is3X5h0nu9Jb5k1HmT8OMziiaf0hVUsd+ q1A9QUnFvm8gfJKwS0LU3FZfPp72xdLBVb/T2kzDmDrb5WqieSbNME/8V64sNwuasbHf qYfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-transfer-encoding; bh=ojSxtEyCVSWyl9R5sz/WDjE3la4F2F+ge56YWnKhOT0=; b=TEp+pQG78xB2TQ9mxNFtaeH0Yi5mu5xkVhPzGclS07mEvU/bzNKiErnHHBBvV443ky gzmB1pmMxssOMheTu/A46rlZj3xCWoxsvvTSUqZnIbU0cBhnSFaids6gosrf8ZcMDXdG xkDH+WArT/hSrVOOQ6Cirvuo7s0jLLMdliwEgwHFPSZCge9sw5mExCCK1HZwKIdS6OuW 1FiYZlvFQNodaUKCdyVeNGV/1odStCjcvWlyPz7MEFZbtp7F7G35mf0nwieZolMG9x2p b97ZU/Kq5zSWxb4ddFGLwEkyh24yiTD+8oH1LuiVVbLPg6jWrukv+C9YEii/+83sgZh1 wUrQ== X-Gm-Message-State: AE9vXwOgpgy+huWm93uFG8E64U6CreltqU7DIvtS+RYaJmDj6mj9jCTZ/kuifcCSP2G0REsUrSrd7Qb2jDBkWA== X-Received: by 10.129.166.194 with SMTP id d185mr12223095ywh.247.1474791107026; Sun, 25 Sep 2016 01:11:47 -0700 (PDT) MIME-Version: 1.0 Received: by 10.83.20.135 with HTTP; Sun, 25 Sep 2016 01:11:46 -0700 (PDT) In-Reply-To: References: <4DC31048-4D9E-4039-B576-ECDC434A6A1E@dericed.com> <7208DD13-54A2-4C49-AF09-26A6E52C74A9@dericed.com> From: Steve Lhomme Date: Sun, 25 Sep 2016 10:11:46 +0200 Message-ID: To: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: Subject: Re: [Cellar] EBML Element Parenting X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Sep 2016 08:11:50 -0000 I found the POSIX regex https://en.wikipedia.org/wiki/Regular_expression#Standards The ABNF and EBNF https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_Form https://en.wikipedia.org/wiki/Augmented_Backus%E2%80%93Naur_Form The W3C derivative of the ABNF https://www.w3.org/TR/REC-xml/#sec-notation The problem with all these is that they define "character" or "binary" regular expressions. What we want is something of higher level so we can use the human friendly names rather than the binary representation for each element. Maybe we need to define our own ABNF derivative. But it's harder to parse than an POSIX-like regex. When looking at the mess it is to do a regex for a URL or email address I don't the ABNF is doing a great job. 2016-09-25 9:53 GMT+02:00 Steve Lhomme : > We keep on updating the draft for this (big) change on Github. > > To defined CRC-32 and Void that can have all kinds of parents we need > some kind of regular expression. For now I think `/+` and `/*` > respectively are good candidates to describe them. > > For the parent description we need to make these expressions > unambiguously interpreted so it should be based on some regular > expression format. We don't want to reinvent the wheel here so > something already standardized would be better. > > We need a format that allows: > - basic paths (`/xx/yy/c`) > - any path possible (`/*`) > - any path possible except the root (`/+`) > > optional but welcome: > - repeatable path that would describe programmatically how the > recursion works (`/Tags/Tag[/SimpleTag]+`) > - a combination of pathes (`/xx/yy/[a|b|c]`) > > While thinking about this it becomes clearer that what we need for > each element is not to define the parent (or the level) but actually > its valid path(es). > > even more optional > - a way to make pathes mutually exclusive > - a way to make pathes relate to each other (all timestamps that > depend on the timestampscale) > - a way to mark elements that must both be present > (TimestampScaleNumerator & TimestampScaleDenominator) > > By adopting a regular expression format we may impose some > restrictions on what is possible too. A very complex regular > expression handling would mean valid parsers would be required to > handle complex files for no good reasons. > > 2016-08-23 21:00 GMT+02:00 Dave Rice : >> >>> On Aug 23, 2016, at 2:41 PM, Steve Lhomme wrote: >>> >>> 2016-08-23 20:39 GMT+02:00 Dave Rice : >>>> >>>>> On Aug 23, 2016, at 2:35 PM, Steve Lhomme wrot= e: >>>>> >>>>> I updated the pull request with something that merges with the curren= t master. >>>>> >>>>> Anyone disagree with the move before I merge it ? >>>> >>>> I've been thinking about it and have slowly starting thinking that thi= s is a better approach than I first thought. It does deviate from the XML S= chema analogy but maybe the complexity of XML Schemas aren't for the best h= ere. You're right that it would be easier to add extensions (divx) with thi= s approach. It also resolves the issue with xml being normative, since the = text and xml versions would be more semantically identical without dependin= g on xml nesting. >>>> >>>>> The next steps after that: >>>>> - unflatten the XML definition >>>> >>>> What do you mean? >>> >>> >>> >>> >>> Container of data and >>> attributes representing one or many files. >>> >>> >> maxOccurs=3D"unbounded"> >>> An attached file. >>> >>> >> minOccurs=3D"1"> >>> Filename of the attache= d >>> file. >>> >>> >> minOccurs=3D"1"> >>> MIME type of the >>> file. >>> >>> >> type=3D"date" minOccurs=3D"1"> >>> Modification timestamp = of >>> the file. >>> >>> >> minOccurs=3D"1"> >>> The data of the >>> file. >>> >>> >> >> Ah, I would have considered that action as 'flattening', but I support t= he example that we'll use parent rather than nested elements. >> >>> No more hierarchy that could potentially interfere with the one >>> defined in "parent". Also as discussed before elements like Void have >>> no real hierarchy. >> >> +1 >> Dave >> >>>>> - remove the level altogether, including all text references and in >>>>> the Matroska specs. >>>> >>>> Yes. >>>> Dave >>>> >>>>> 2016-08-05 17:34 GMT+02:00 Steve Lhomme : >>>>>> 2016-08-05 17:26 GMT+02:00 Dave Rice : >>>>>>> >>>>>>>> On Aug 5, 2016, at 11:13 AM, Steve Lhomme w= rote: >>>>>>>> >>>>>>>> 2016-08-05 17:09 GMT+02:00 Steve Lhomme : >>>>>>>>> 2016-08-02 2:50 GMT+02:00 Dave Rice : >>>>>>>>>> >>>>>>>>>> On Jul 31, 2016, at 11:41 AM, Steve Lhomme wrote: >>>>>>>>>> >>>>>>>>>> As we're moving towards splitting the Matroska specs in many sma= ller >>>>>>>>>> documents, putting not essential things in a document it becomes= more >>>>>>>>>> important to define things differently than the "level" element = we >>>>>>>>>> currently rely on. It makes it very difficult to define a single >>>>>>>>>> element without having to copy all its parents to explain where = it >>>>>>>>>> should go in the EBML format. >>>>>>>>>> >>>>>>>>>> I think we should set the name of the parent(s) explicitely to a= void >>>>>>>>>> that confusion. We will also not rely anymore on the order in wh= ich >>>>>>>>>> elements are defined in the specs. >>>>>>>>>> >>>>>>>>>> I did a pull request with this addition: >>>>>>>>>> https://github.com/Matroska-Org/ebml-specification/pull/83 >>>>>>>>>> >>>>>>>>>> This change will probably mean many paragraphs need to be rewrit= ten to >>>>>>>>>> rely less on the level system. >>>>>>>>>> >>>>>>>>>> I will do the pull request for Matroska as well once it's settle= d. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Sorry to join this conversation late. I think the current state = of the EBML >>>>>>>>>> implies two different ways of documenting nesting. The EBML Sche= ma example >>>>>>>>>> simply nests the within one another. For example: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> maxOccurs=3D=E2=80=9Cunbounded"> >>>>>>>>>> >>>>>>>>> minOccurs=3D=E2=80=9C1=E2=80=9D/> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> The nesting of the EBML Schema implies that FileName occurs with= in File and >>>>>>>>>> File occurs within Files. The use of the =E2=80=98level=E2=80=99= attribute is redundant to >>>>>>>>>> the structure. >>>>>>>>> >>>>>>>>> Yes it's redundant and should probably be removed. It only create= s >>>>>>>>> possible issues and gives no extra information. Except when it's = 1+ >>>>>>>>> which doesn't have any equivalent in XML and is currently not def= ined. >>>>>>>>> >>>>>>>>>> However, later we list out the elements of the EBML Header in a = series of >>>>>>>>>> tables. The tables don=E2=80=99t imply the nested structure as t= he xml does, so the >>>>>>>>>> level attribute here is needed, though it is certainly a bit unc= lear since >>>>>>>>>> determining the parent requiring reading backwards to the last m= aster >>>>>>>>>> element of a higher value. >>>>>>>>>> >>>>>>>>>> I prefer to keep the XML analogy where elements are defined in a= n XML schema >>>>>>>>>> where the arrangement of the elements within one another defines= the nesting >>>>>>>>>> structure. Then since we can compile our documentation the =E2= =80=9CParent=E2=80=9D values >>>>>>>>>> can be decipher from the XML-based EBML Schema. However this get= s to a >>>>>>>>>> conversation at the CELLAR working group meeting on whether XML = should be >>>>>>>>>> considered normative or not. >>>>>>>>> >>>>>>>>> While I support having an up to date machine readable version of = the >>>>>>>>> specs. I don't think it should/will be the actual reference for t= he >>>>>>>>> specifications. >>>>>>>>> >>>>>>>>> Also the XML parent analogy only works if there's a fixed parent = and >>>>>>>>> the element can only be found at that level. How do you define an >>>>>>>>> element that can be found anywhere under /File/Attribute/* (to us= e the >>>>>>>>> path system) (or elements similar to Void and CRC32 which can be >>>>>>>>> defined for a certain DocType) ? Either we add an attibute in the= XML >>>>>>>>> element or we define how level with a + should be interpreted. Si= nce >>>>>>>>> it's extra information it's probably better to put it out of the = level >>>>>>>>> (which should go from the XML anyway). How would you define an el= ement >>>>>>>>> that be put anywhere within a DocType in the XML format ? I agree= that >>>>>>>>> it would be odd to have a "parent" attribute in a non flat XML fi= le. >>>>>>>>> It's also odd to have a "parent" on a flat XML file. >>>>>>>> >>>>>>>> Also there's still the problem how to define a (set of) element(s) >>>>>>>> outside of the core Matroska speficications. >>>>>>> >>>>>>> You mean to define a custom extra section to Matroska? >>>>>> >>>>>> Yes, outside of the original spec. For example if a company does lik= e >>>>>> DivX and creates a set of new elements. We don't want to update the >>>>>> original Matroska specs for that. >>>>>> >>>>>>>> I can't be the core >>>>>>>> specifications including an external file. It should be that exter= nal >>>>>>>> (XML) that tells where it should be inserted. Is there a common XM= L >>>>>>>> mechanism to inject one file into another ? >>>>>>> >>>>>>> Yes, in XML Schema there is a way to define an element to contain x= sd:any meaning that any elements may occur within it. Within that section t= he sub-element should ideally have a namespace that points to another XML S= chema that defines that particular section. >>>>>>> >>>>>>>> Does it involve defining >>>>>>>> all the elements in the original file to define where we want the >>>>>>>> insertion ? With possible mismatch or missing updates. >>>>>>> >>>>>>> It involves the parent format defining where the flexibility can oc= cur so that any valid XML is allowed in a defined area and then the content= s of that optionally being defined by another schema. >>>>>> >>>>>> That does not sound very practical/flexible in our case. Or we'd hav= e >>>>>> to define all Master elements with such a system. And that would hav= e >>>>>> to be done for all EBML-based formats. >>>>>> >>>>>>> dave >>>>>>> >>>>>>>>>> So I think I would prefer not to have a =E2=80=98parent=E2=80=99= attribute but to allow that >>>>>>>>>> to be implied by the nesting of elements within an EBML Schema. = Then the >>>>>>>>>> parent data could be presented for users when we compile documen= tation from >>>>>>>>>> the EBML Schema. For instance this markdown is build from the EB= ML Schema >>>>>>>>>> and lists parent values derived from the structure of the EBML S= chema. >>>>>>>>>> https://gist.github.com/dericed/dd1710e47dd3b6779463a85e4fc0f788 >>>>>>>>>> >>>>>>>>>> Dave Rice >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Steve Lhomme >>>>>>>>> Matroska association Chairman >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Steve Lhomme >>>>>>>> Matroska association Chairman >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Cellar mailing list >>>>>>>> Cellar@ietf.org >>>>>>>> https://www.ietf.org/mailman/listinfo/cellar >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Steve Lhomme >>>>>> Matroska association Chairman >>>>> >>>>> >>>>> >>>>> -- >>>>> Steve Lhomme >>>>> Matroska association Chairman >>>>> >>>>> _______________________________________________ >>>>> Cellar mailing list >>>>> Cellar@ietf.org >>>>> https://www.ietf.org/mailman/listinfo/cellar >>>> >>> >>> >>> >>> -- >>> Steve Lhomme >>> Matroska association Chairman >>> >>> _______________________________________________ >>> Cellar mailing list >>> Cellar@ietf.org >>> https://www.ietf.org/mailman/listinfo/cellar >> > > > > -- > Steve Lhomme > Matroska association Chairman --=20 Steve Lhomme Matroska association Chairman From nobody Sun Sep 25 01:58:47 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AA55112B138 for ; Sun, 25 Sep 2016 01:58:45 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.703 X-Spam-Level: X-Spam-Status: No, score=-0.703 tagged_above=-999 required=5 tests=[BAYES_20=-0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TeFrGAjwO89i for ; Sun, 25 Sep 2016 01:58:43 -0700 (PDT) Received: from smtp-sh2.infomaniak.ch (smtp-sh2.infomaniak.ch [128.65.195.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DE43D12B12B for ; Sun, 25 Sep 2016 01:58:42 -0700 (PDT) Received: from smtp5.infomaniak.ch (smtp5.infomaniak.ch [83.166.132.18]) by smtp-sh.infomaniak.ch (8.14.5/8.14.5) with ESMTP id u8P8wetl001864 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL) for ; Sun, 25 Sep 2016 10:58:40 +0200 Received: from Castor.local (77-56-128-186.dclient.hispeed.ch [77.56.128.186]) (authenticated bits=0) by smtp5.infomaniak.ch (8.14.5/8.14.5) with ESMTP id u8P8wd9T030849 for ; Sun, 25 Sep 2016 10:58:40 +0200 Date: Sun, 25 Sep 2016 10:58:40 +0200 From: Reto Kromer To: Codec Encoding for LossLess Archiving and Realtime transmission X-Priority: 3 In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Mailer: Mailsmith 2.4 (470) X-Antivirus: Dr.Web (R) for Unix mail servers drweb plugin ver.6.0.2.8 X-Antivirus-Code: 0x100000 Archived-At: Subject: Re: [Cellar] EBML Element Parenting X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Sep 2016 08:58:46 -0000 Steve Lhomme wrote: >I found the POSIX regex >https://en.wikipedia.org/wiki/Regular_expression#Standards The formal standard is: ISO/IEC/IEEE 9945:2009 Information technology -- Portable Operating System Interface (POSIX) Base Specifications, Issue 7 which contains much more than "just" the regex. Best regards, Reto AV Preservation by reto.ch chemin du Suchet 5 | 1024 Ecublens | Switzerland Web: https://reto.ch | Twitter: @retoch From nobody Sun Sep 25 06:34:35 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CD6BE12B219 for ; Sun, 25 Sep 2016 06:34:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.121 X-Spam-Level: X-Spam-Status: No, score=-1.121 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rS0DKoObanFn for ; Sun, 25 Sep 2016 06:34:32 -0700 (PDT) Received: from s172.web-hosting.com (s172.web-hosting.com [68.65.122.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 142BF12B218 for ; Sun, 25 Sep 2016 06:34:31 -0700 (PDT) Received: from cpe-184-152-63-84.nyc.res.rr.com ([184.152.63.84]:37060 helo=[10.0.1.77]) by server172.web-hosting.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.86_1) (envelope-from ) id 1bo9Zc-000yw3-Gk; Sun, 25 Sep 2016 09:34:31 -0400 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) From: Dave Rice In-Reply-To: <295a496f-03b0-b78e-b8fc-b36887649825@gmx.de> Date: Sun, 25 Sep 2016 09:34:24 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: References: <38E61A25-422A-482B-AFC0-0A6735572FBB@dericed.com> <295a496f-03b0-b78e-b8fc-b36887649825@gmx.de> To: "Sebastian G. " X-Mailer: Apple Mail (2.3124) X-OutGoing-Spam-Status: No, score=-2.9 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Cc: cellar@ietf.org Subject: Re: [Cellar] review of codec_specs.md for defining use of encodings in Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Sep 2016 13:34:35 -0000 > On Sep 25, 2016, at 2:46 AM, Sebastian G. = wrote: >=20 > 25.09.2016, 00:05 Dave Rice: >> Hi all, >>=20 >> Here=E2=80=99s a review of the codec_specs.md page in the = matroska-specification repository. I also compared the Codec IDs listed = in this page to a collection of sample files from the Internet Archive = to get an idea on how much this page and real use overlap. >>=20 >> Currently there is a page that documents how certain encodings are = stored in Matroska. The original document is on the matroska site at = https://matroska.org/technical/specs/codecid/index.html and there is a = representation in markdown in the matroska-specification repository at = https://github.com/Matroska-Org/matroska-specification/blob/gh-pages/codec= _specs.md. There=E2=80=99s a related version on Haali=E2=80=99s site at = http://haali.su/mkv/codecs.pdf. >>=20 >> Within this page three values are used to define Matroska=E2=80=99s = support for an encoding: >>=20 >> 1. Codec ID >> 2. Name >> 3. Description >>=20 >> The Codec ID appears to have a form to it but it is not defined = though should be. >=20 > The obvious pattern to me is that a Codec ID starts with an V for a > video codec, with an A for a audio codec and with an S for subtitles > (and B for button). Then the Codec ID includes the name in some way. = For > subtitles it states if it is a text or image based subtitle, though = not > for all of them. But the identifiers are not complete. For instance the TrackType = Element, https://matroska.org/technical/specs/index.html#TrackType, also = lists `complex`, `logo`, and `control` track types. Would encodings of = these types have different letter prefixes in Codec ID. Off topic, but are `complex` track types for encodings such as DV where = audio, video, timecode, captions, and metadata information is all = arranged within a single encoding? >> The description appears to consistently. >>=20 >> The terms =E2=80=98private data=E2=80=99 and the `CodecPrivate = Element` seems to be referenced in the same way. Are these the same = things? If so, they should be made consistent. >>=20 >> The phrase "The private data is void.=E2=80=9D is used several times = in the document. Does this mean that the `CodecPrivate Element` is = stored but as an `Empty Element`? Could the `CodecPrivate Element` just = not be present if not needed? >>=20 >> The tables of codecs has some odd nested that breaks the table in = markdown. I propose that the data be rearranged into a simple table. >=20 > I assume that the separation of video, audio, subtitles and other = codec > related stuff will be possible with a simple table, in which case I > would be OK with replacing the nested table with a simple one. >=20 >> Are there encodings that Matroska supports that are not in this list?=20= >=20 > Yes, I think so. VP9 and OPUS should be already supported, but are not > listed. There might be even more. >=20 >> This Codec ID is listed as =E2=80=9Dsupported later=E2=80=9D but is = it working now? >> Codec ID: V_MJPEG >> Sample file: = http://archive.org/download/2VideoMP4MPEG44096x216063625KbpsHiRez/34_Video= _MKV_(mjpeg)________640x480______3166Kbps__________________.mkv >>=20 >> Codec ID: V_VP8 >> There are many files around with this Codec ID. Supported? >> Sample file: = http://archive.org/download/20140825071_201408/2014-08-25-070-1.mkv >=20 > I think it is the basis for WebM, so either it had been supported = before > or Google would have added support for it to WebM. >=20 >> Codec ID: V_MPEGH/ISO/HEVC >> Even more of these files are around but not in the spec. Supported? >> Sample file: http://archive.org/download/1A71zv37dr/1A71zv37dr.mkv >>=20 >> Codec ID: V_VP9 >> Also popular, but unlisted. Supported? >> Sample file: http://archive.org/download/yt-adonfjv/Super Mario 64 = Disk Version 64DD - Powerfull Mario song-MGeGIaJr8gc.mkv >=20 > The same that applies to WebM I think. Yes it would be good if it were feasible to remux a valid webm to a = valid Matroska and still consider the encoding as supported. I suggest = that first the manner in which Matroska encoding support is written is = reviewed and clarified and then consider adopting the encoding support = from the webm specification. >> Codec ID: V_MPEG4/ISO/AVC >> This is possibly the most widely used Codec ID in Matroska, but = unlisted. Supported? >> Sample: http://archive.org/download/00013862/0001-3862.mkv >>=20 >> Codec ID: A_AAC >> This is very popular as a Matroska audio codec, but it deviates from = the forms listed in codec_specs.md >> Sample file: http://archive.org/download/150069/150069.mkv >>=20 >> Codec ID: A_OPUS >> Opus is common in Matroska but unlisted here. Supported? >> Sample; = http://archive.org/download/JigwallysNesTheGreatWaldoSearchIn0052.41/great= waldosearch-tas-jigwally.mkv >=20 > There have been changes to Matroska in order to support pre-roll for > OPUS. There is a related email, but I can't find the list thread > discussing changes to Matroska.[0] >=20 >> Codec ID: A_EAC3 >> More rare, but there are around. Unlisted in specs. >> Sample file: = http://archive.org/download/Barcelona1VS1EspanyolPenaltiesAndCrowning720pH= DBarcelonaHD.blogspot.com/Barcelona 1 VS 1 Espanyol penalties and = crowning 720pHD Barcelona-HD.blogspot.com.mkv >>=20 >> Codec ID: A_TRUEHD >> Unlisted but found in real world. Supported? >> Sample file: = http://archive.org/download/mad.max.fury.road.2015.1080p.bluray.hevc.trueh= d.7.1.atmos.absinth/Mad.Max.Fury.Road.2015.1080p.BluRay.TrueHD.7.1.ATMOS-A= bsinth.mkv >>=20 >> Note that for many of the Codec IDs noted that are not in the = Matroska specification, they are in the webm specification at = http://www.webmproject.org/docs/container/: such as V_VP8, V_VP9, = A_OPUS. Are these valid in Matroska as well? A discussion on adding = these was started at = https://lists.matroska.org/pipermail/matroska-devel/2015-October/004852.ht= ml but did not resolve. >>=20 >> Other questions: >=20 > A question that I see is: What about Codec ID other people used and = got > popular? Should they just be included in the codec spec? And even if > they go against some naming scheme? Presently there is no naming scheme to comply with or go against. I = suggest one be written according to the forms of existing supported = Codec ID (as you described a bit above). With a Codec ID expression and = more clarity in how a encoding support is expressed, it would be easier = to adopt externally defined Codec IDs. >> Should the Matroska specification provide enough info for a user to = independently define support for an encoding in Matroska? Or should = codec support somehow be =E2=80=98registered=E2=80=99 via Matroska = and/or CELLAR? >=20 > I quote the codec specs: "[...] to list those codecs that are = currently > supported in matroska (or will be supported soon), and therfore need a > well defined codec ID so that all developers supporting matroska will > use the same ID." >=20 > (There is a spelling mistake in 'therfore=E2=80=99) Also I think =E2=80=9Csupported soon=E2=80=9D should be removed from the = draft and possibly moved to branches or discussion on the list instead. > I'm not sure if people come up with the same name independently for > every possible new format, even if the spec contains detailed > information on how to derive them. >=20 > Well, arguably the quoted section did not turn out well, as you show > examples where there are no defined IDs, but there are ones that are = used. >=20 > However, to me it still appears to be better to have a governing body = to > ensure that in the long run, if not from the beginning of the usage of > an ID, that only one Codec ID will be used. V_MS/VFW/FOURCC provides a good workaround anyhow as it allows a lot of = encodings to be used with fourccs in AVI style. Still it sounds messy to = recommend that users create their own Codec IDs without some registry = (similar issue to official vs user TagNames). But supposing the Matroska RFC is finalized. How would a new Codec ID be = registered? Update the Matroska RFC with an addedum or extended Codec ID = part of the specification? Or create a new RFC for the use of encoding = in Matroska as a separate document? IIUC the latter option is how MXF = registers new encodings, by new documents. >> Can the structure of a Codec mapping be more clearly defined, so that = a user could define their own support and understand if it is correct or = not? >=20 > Certainly for the sake of clarity, to increase understanding and avoid > errors or mismatches some information on how it is supposed to work > should be included. >=20 >> For a Matroska validation that finds something like V_MPEG4/ISO/AVC = in Matroska. Should it be considered as unsupported (most MKV files!)? >=20 > For the sake of supporting the current state, there could be a primary > Codec ID and a secondary Codec ID. While a muxer that complies with = the > specs would write the primary ID, a demuxer could support both. The = file > would remain valid in the sense that a player would not refuse it. >=20 > A validator could point out that an .mk? contains a secondary Codec = ID, > that better should be replaced with the primary Codec ID. >=20 > Tools that would aim to bring files in a (more) compliant state would > replace the secondary ID with the primary ID, unless the user would > choose not to, because his player would not understand the primary ID. I think that makes sense, to have some Codec ID aliases that should not = be written but could be read to be interpreted with an official Codec = ID. >> Best Regards, >> Dave Rice >>=20 >=20 > Best Regards, > Sebastian >=20 > [0] > = https://lists.matroska.org/pipermail/matroska-devel/2013-September/004573.= html Dave= From nobody Sun Sep 25 08:51:42 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C7CB312B24C for ; Sun, 25 Sep 2016 08:51:40 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.6 X-Spam-Level: X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xCFpD-op9sWF for ; Sun, 25 Sep 2016 08:51:39 -0700 (PDT) Received: from mail-yw0-x230.google.com (mail-yw0-x230.google.com [IPv6:2607:f8b0:4002:c05::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DBB8412B24A for ; Sun, 25 Sep 2016 08:51:38 -0700 (PDT) Received: by mail-yw0-x230.google.com with SMTP id i129so144536479ywb.0 for ; Sun, 25 Sep 2016 08:51:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=Ep3EH6VcHi2aNECXoZBMEaY2KM0Sjz0OODCHy7VB804=; b=C0BK0mXRAxjchrgeAcuAgjj6UuUYYnEWzJd6+FInP43C+VwYfCskk1xNvaVxL9SMkt QifmH2TXSMc8q2P4R4Az8JY0RH5U/YmIYu3f0D1EEEGsLEfGQopghpWSkvKIu81inAqR 9oYA4dck2iGgcghauYUmHeMhJtCULJhLOHpnxrG104zg2wa0qHaMCfnoYQdIZSCrmzZJ eEqiX3/ozNLWmm/oV4DqWLCQz8nCweZwE2a5yQHrNEYupuFQHWhU7LXV00IiDxeFd6KK CJGQNG9yg8lGAZCrrtyjsQoN4R4qDkBRhxPlJe//y8xH/Xejib80Kwn4gibGiJtZc87/ dQvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=Ep3EH6VcHi2aNECXoZBMEaY2KM0Sjz0OODCHy7VB804=; b=d+Y6RbALzaIZyMrLbnrfnnkG2uYMzxw4eVnFyig99GRPmLmrkG0ueu/kSdPWk2plSt vTPm5mckZ9lkjH+P+yHO4l/I63zTz2pb3UwIQnoygd2bqJy4GvnZmNq0tFuFeFiBOuXx Myd+HH+SQl8hPgFfagqZcb7V2VRTNu4D/xVA4ASF3JNKFqxs0jRSmu9x8wIMUp2nWiLz 8LtZOVNdbEiW7Z58N7sOqiQicqSbKlDg7owtzc+jBmPHEDpgczSin0zW7HKhFw5ptDmU ZG/xdFvxS3trcanNMs+yciOTSyKbTuuWWuVWgWRjUxCYihoSGh/gJ+STpilh8mQPGvnV RoFw== X-Gm-Message-State: AE9vXwP50EMoVl+3kd/BZmZzw9mGmO45mLKbqv/fOFUF1PV4Fj3oJoPAVwN6g3CHoTtujx/49w1hLCbVz13RQg== X-Received: by 10.13.207.1 with SMTP id r1mr14203960ywd.177.1474818697649; Sun, 25 Sep 2016 08:51:37 -0700 (PDT) MIME-Version: 1.0 Received: by 10.83.20.135 with HTTP; Sun, 25 Sep 2016 08:51:37 -0700 (PDT) In-Reply-To: References: From: Steve Lhomme Date: Sun, 25 Sep 2016 17:51:37 +0200 Message-ID: To: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset=UTF-8 Archived-At: Subject: Re: [Cellar] EBML Element Parenting X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Sep 2016 15:51:41 -0000 I went ahead and did a new Pull Request with an EBNF based notation to define the path. I merged the minOccurs/maxOccurs as well as it's simple enough to add it to the "regular expression" in a way anyone can understand. Just reading that path line of an element gives you a quick view of where it is found and how often (including recursively). https://github.com/Matroska-Org/ebml-specification/pull/106 Now after doing that I realized the XML schema wasn't as easy to parse as before. Let me know what you think. We may have the EBNF notation for each element definition in the RFC and the XML definition used different parts of the EBNF definition in various attributes (likely the ones we have before). I struggled a bit to find a way to define the Void properly without making the EBNF syntax too complex. I left out more complex recursive possibilities like /a/b(/c/d)+ for recursion jumping a level. It is not supported by current EBML parsers and was never considered anyway. I used the classic EBNF base with concatenation using the "," character. 2016-09-25 10:58 GMT+02:00 Reto Kromer : > Steve Lhomme wrote: > >>I found the POSIX regex >>https://en.wikipedia.org/wiki/Regular_expression#Standards > > The formal standard is: > > ISO/IEC/IEEE 9945:2009 > Information technology -- Portable Operating System > Interface (POSIX) Base Specifications, Issue 7 > > which contains much more than "just" the regex. > > Best regards, Reto > > > AV Preservation by reto.ch > chemin du Suchet 5 | 1024 Ecublens | Switzerland > Web: https://reto.ch | Twitter: @retoch > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar -- Steve Lhomme Matroska association Chairman From nobody Sun Sep 25 09:01:16 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 610FE12B24E for ; Sun, 25 Sep 2016 09:01:15 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.101 X-Spam-Level: X-Spam-Status: No, score=-2.101 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ohx-qvxGYgRq for ; Sun, 25 Sep 2016 09:01:12 -0700 (PDT) Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1B0BC12B24A for ; Sun, 25 Sep 2016 09:01:11 -0700 (PDT) Received: from [192.168.2.129] ([84.61.100.56]) by mail.gmx.com (mrgmx103) with ESMTPSA (Nemesis) id 0M6ioC-1b0SuU1qCa-00wRAd; Sun, 25 Sep 2016 18:00:26 +0200 To: Dave Rice References: <38E61A25-422A-482B-AFC0-0A6735572FBB@dericed.com> <295a496f-03b0-b78e-b8fc-b36887649825@gmx.de> From: "Sebastian G. " Openpgp: id=BFE90DE515B6F548CDE298939902921C2B944DAE Message-ID: Date: Sun, 25 Sep 2016 18:00:24 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Provags-ID: V03:K0:YvWZZvYaxhfPRwjcD3tL7TzFduU5LFa+sIacLxmvQj/mMlGi+TD gqvuX4wUBGMAXDdQrMr25D2m05FYKLuNTXCmrVjuJwQm+CeGdGdjkqkqrkyggkaSLy7WL0p DNpFzhODGI53EikTm0+ZmtJk/NwjuEJNSQ+Z4faaSkfWLGoTwGteQpUIp4eRwxrwyiN1Ux0 3mwgppIsV6mFlIskKQN2Q== X-UI-Out-Filterresults: notjunk:1;V01:K0:jcupYGuhr8A=:kSgexCEW/+umxlQnifLODl rm3aCkDeY4kxW26BaFHwoHJg0QfxTNBZx3L/fgcWLju1rMPkN1ugfX5KhvfwAE5oV9LZAuCx2 Ed3DG74KQ9du/mIXi1zwUS15uW7Ps4xzKp3tW/RPw+KlfpLb4Tb2giiml85XoZkmtcUi5hDIY /SSwnrnPlw4Q1crbKoUnKz2tMwoautHZ+A9ddzXHKSsmUsa5UGgpYoWK0LL444IdjyRCyBlHD znwIxvGwEEd9wb6XBN/3MfFeYkXe3vAt35yaW87Z42OicRWz1kJHsQ9rwwzD2KE4x3s9poSkO lT+jEezSuiRJAlMQCWH4Ll4S9BKxct+4K9MnMIsTSDtU4iuGxu4Mj4MkQwFJ6C2Z1ulrN4gkK df6l6nr9oItfFRi5Ap9T5KfiSyZZWLPNCGMKsQSaGDokO/G7THV4e415W39CUUxNVmlpJTu9Y SJ0gXJKz9b1/HFKZcllDBj618qtAY/xNvhs80kFKDXW9okaAlipdrsMwHKOIp/qxjzrfwpfIT mwRrIz4F6eT6xLgKEsg7vFthak6+KDdwLVvvkPA0RQaNo3YKWXox/sHBEuoiWIMXziDssMtw8 gVwrzjCrrS0t0C9eu8JgLFxQin3eNuU50ND7TwzFy0pB/uAVQ3MUnh7HK2qXyLCL+nu0k9pzA bGBzjL9k0p8kfy45+6yu0rPqPuv7ddI31Wh0clAHyr0hHkkVSIrrFntISGM+50d5uoUnL1nSr vMBW4STSVa/xiIEKc+Qn+MLgy2GSzh6BvkL4t673LUs/L65EJiCWMD8HImmD2v0ij+RBPn3fL 4KX4TSd Archived-At: Cc: cellar@ietf.org Subject: Re: [Cellar] review of codec_specs.md for defining use of encodings in Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Sep 2016 16:01:15 -0000 25.09.2016, 15:34 Dave Rice: > >> On Sep 25, 2016, at 2:46 AM, Sebastian G. wrote: >> >> 25.09.2016, 00:05 Dave Rice: >>> Hi all, >>> >>> Here’s a review of the codec_specs.md page in the matroska-specification repository. I also compared the Codec IDs listed in this page to a collection of sample files from the Internet Archive to get an idea on how much this page and real use overlap. >>> >>> Currently there is a page that documents how certain encodings are stored in Matroska. The original document is on the matroska site at https://matroska.org/technical/specs/codecid/index.html and there is a representation in markdown in the matroska-specification repository at https://github.com/Matroska-Org/matroska-specification/blob/gh-pages/codec_specs.md. There’s a related version on Haali’s site at http://haali.su/mkv/codecs.pdf. >>> >>> Within this page three values are used to define Matroska’s support for an encoding: >>> >>> 1. Codec ID >>> 2. Name >>> 3. Description >>> >>> The Codec ID appears to have a form to it but it is not defined though should be. >> >> The obvious pattern to me is that a Codec ID starts with an V for a >> video codec, with an A for a audio codec and with an S for subtitles >> (and B for button). Then the Codec ID includes the name in some way. For >> subtitles it states if it is a text or image based subtitle, though not >> for all of them. > > But the identifiers are not complete. For instance the TrackType Element, https://matroska.org/technical/specs/index.html#TrackType, also lists `complex`, `logo`, and `control` track types. Would encodings of these types have different letter prefixes in Codec ID. > > Off topic, but are `complex` track types for encodings such as DV where audio, video, timecode, captions, and metadata information is all arranged within a single encoding? > >>> The description appears to consistently. >>> >>> The terms ‘private data’ and the `CodecPrivate Element` seems to be referenced in the same way. Are these the same things? If so, they should be made consistent. >>> >>> The phrase "The private data is void.” is used several times in the document. Does this mean that the `CodecPrivate Element` is stored but as an `Empty Element`? Could the `CodecPrivate Element` just not be present if not needed? >>> >>> The tables of codecs has some odd nested that breaks the table in markdown. I propose that the data be rearranged into a simple table. >> >> I assume that the separation of video, audio, subtitles and other codec >> related stuff will be possible with a simple table, in which case I >> would be OK with replacing the nested table with a simple one. >> >>> Are there encodings that Matroska supports that are not in this list? >> >> Yes, I think so. VP9 and OPUS should be already supported, but are not >> listed. There might be even more. >> >>> This Codec ID is listed as ”supported later” but is it working now? >>> Codec ID: V_MJPEG >>> Sample file: http://archive.org/download/2VideoMP4MPEG44096x216063625KbpsHiRez/34_Video_MKV_(mjpeg)________640x480______3166Kbps__________________.mkv >>> >>> Codec ID: V_VP8 >>> There are many files around with this Codec ID. Supported? >>> Sample file: http://archive.org/download/20140825071_201408/2014-08-25-070-1.mkv >> >> I think it is the basis for WebM, so either it had been supported before >> or Google would have added support for it to WebM. >> >>> Codec ID: V_MPEGH/ISO/HEVC >>> Even more of these files are around but not in the spec. Supported? >>> Sample file: http://archive.org/download/1A71zv37dr/1A71zv37dr.mkv >>> >>> Codec ID: V_VP9 >>> Also popular, but unlisted. Supported? >>> Sample file: http://archive.org/download/yt-adonfjv/Super Mario 64 Disk Version 64DD - Powerfull Mario song-MGeGIaJr8gc.mkv >> >> The same that applies to WebM I think. > > Yes it would be good if it were feasible to remux a valid webm to a valid Matroska and still consider the encoding as supported. I suggest that first the manner in which Matroska encoding support is written is reviewed and clarified and then consider adopting the encoding support from the webm specification. +1 to both of that. >>> Codec ID: V_MPEG4/ISO/AVC >>> This is possibly the most widely used Codec ID in Matroska, but unlisted. Supported? >>> Sample: http://archive.org/download/00013862/0001-3862.mkv >>> >>> Codec ID: A_AAC >>> This is very popular as a Matroska audio codec, but it deviates from the forms listed in codec_specs.md >>> Sample file: http://archive.org/download/150069/150069.mkv >>> >>> Codec ID: A_OPUS >>> Opus is common in Matroska but unlisted here. Supported? >>> Sample; http://archive.org/download/JigwallysNesTheGreatWaldoSearchIn0052.41/greatwaldosearch-tas-jigwally.mkv >> >> There have been changes to Matroska in order to support pre-roll for >> OPUS. There is a related email, but I can't find the list thread >> discussing changes to Matroska.[0] >> >>> Codec ID: A_EAC3 >>> More rare, but there are around. Unlisted in specs. >>> Sample file: http://archive.org/download/Barcelona1VS1EspanyolPenaltiesAndCrowning720pHDBarcelonaHD.blogspot.com/Barcelona 1 VS 1 Espanyol penalties and crowning 720pHD Barcelona-HD.blogspot.com.mkv >>> >>> Codec ID: A_TRUEHD >>> Unlisted but found in real world. Supported? >>> Sample file: http://archive.org/download/mad.max.fury.road.2015.1080p.bluray.hevc.truehd.7.1.atmos.absinth/Mad.Max.Fury.Road.2015.1080p.BluRay.TrueHD.7.1.ATMOS-Absinth.mkv >>> >>> Note that for many of the Codec IDs noted that are not in the Matroska specification, they are in the webm specification at http://www.webmproject.org/docs/container/: such as V_VP8, V_VP9, A_OPUS. Are these valid in Matroska as well? A discussion on adding these was started at https://lists.matroska.org/pipermail/matroska-devel/2015-October/004852.html but did not resolve. >>> >>> Other questions: >> >> A question that I see is: What about Codec ID other people used and got >> popular? Should they just be included in the codec spec? And even if >> they go against some naming scheme? > > Presently there is no naming scheme to comply with or go against. I suggest one be written according to the forms of existing supported Codec ID (as you described a bit above). With a Codec ID expression and more clarity in how a encoding support is expressed, it would be easier to adopt externally defined Codec IDs. +1 >>> Should the Matroska specification provide enough info for a user to independently define support for an encoding in Matroska? Or should codec support somehow be ‘registered’ via Matroska and/or CELLAR? >> >> I quote the codec specs: "[...] to list those codecs that are currently >> supported in matroska (or will be supported soon), and therfore need a >> well defined codec ID so that all developers supporting matroska will >> use the same ID." >> >> (There is a spelling mistake in 'therfore’) > > Also I think “supported soon” should be removed from the draft and possibly moved to branches or discussion on the list instead. +1 The document seems to have been written a long time ago. It has seen some updates, but some stuff is supported and not listed. It appears to be good if the document would list what is supported. >> I'm not sure if people come up with the same name independently for >> every possible new format, even if the spec contains detailed >> information on how to derive them. >> >> Well, arguably the quoted section did not turn out well, as you show >> examples where there are no defined IDs, but there are ones that are used. >> >> However, to me it still appears to be better to have a governing body to >> ensure that in the long run, if not from the beginning of the usage of >> an ID, that only one Codec ID will be used. > > V_MS/VFW/FOURCC provides a good workaround anyhow as it allows a lot of encodings to be used with fourccs in AVI style. Still it sounds messy to recommend that users create their own Codec IDs without some registry (similar issue to official vs user TagNames). > > But supposing the Matroska RFC is finalized. How would a new Codec ID be registered? Update the Matroska RFC with an addedum or extended Codec ID part of the specification? Or create a new RFC for the use of encoding in Matroska as a separate document? IIUC the latter option is how MXF registers new encodings, by new documents. There are multiple possibilities as you line out. 1) Include the codec IDs in the main RFC. If I understand RFCs correctly they are not updated/changed, but extended or replaced by other RFCs. 2) Have the Matroska RFC contain what makes a Matroska container a Matroska container, how to create, how to read and how to change them. And in the same specs it is listed how the Codec ID system works. Then the codec ID list stays independent and can be updated whenever needed. 2a) Have the Matroska RFC contain anything from 2) besides how the codec ID system works. Everything related to the codecs would go to the codecs specs. This list might not end here, there are other ways to achieve this. I am unsure to what the best option would be. If one wants to support OPUS and VP9 (as example) and has to read the main RFC, how many more documents would have to be read? if 1) the follow-up RFC defining support for OPUS and VP9, plus the RFC for format changes, if lucky that would be one document more, in the worst case one for OPUS, one for VP9. if 2) and 2a) the RFCs for possible format changes and the document listing the supported IDs. That would make one in the best case, two if there actually have been changes. For the codecs ID that does not get worser it is just one document. At first it appears to stay more centralized with version 2) or 2a), but what if the main specs of Matroska have to be updated for a new codec, because it needs new elements? Wouldn't it be better to have one RFC defining the changes to Matroska and the newly introduced codec ID? Therefore I am not sure what the is the more reasonable approach here. >>> Can the structure of a Codec mapping be more clearly defined, so that a user could define their own support and understand if it is correct or not? >> >> Certainly for the sake of clarity, to increase understanding and avoid >> errors or mismatches some information on how it is supposed to work >> should be included. >> >>> For a Matroska validation that finds something like V_MPEG4/ISO/AVC in Matroska. Should it be considered as unsupported (most MKV files!)? >> >> For the sake of supporting the current state, there could be a primary >> Codec ID and a secondary Codec ID. While a muxer that complies with the >> specs would write the primary ID, a demuxer could support both. The file >> would remain valid in the sense that a player would not refuse it. >> >> A validator could point out that an .mk? contains a secondary Codec ID, >> that better should be replaced with the primary Codec ID. >> >> Tools that would aim to bring files in a (more) compliant state would >> replace the secondary ID with the primary ID, unless the user would >> choose not to, because his player would not understand the primary ID. > > I think that makes sense, to have some Codec ID aliases that should not be written but could be read to be interpreted with an official Codec ID. Well, at least for those 'false' Codec IDs that have been established. It is bad if developers would depend on the RFC too closely and reject files that they would not have rejected otherwise. >>> Best Regards, >>> Dave Rice >>> Best Regards, Sebastian From nobody Sun Sep 25 09:38:54 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A53DA12B25C for ; Sun, 25 Sep 2016 09:38:52 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.121 X-Spam-Level: X-Spam-Status: No, score=-1.121 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kONNxkq6RpDX for ; Sun, 25 Sep 2016 09:38:50 -0700 (PDT) Received: from s172.web-hosting.com (s172.web-hosting.com [68.65.122.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9AC981200DF for ; Sun, 25 Sep 2016 09:38:50 -0700 (PDT) Received: from mbd2536d0.tmodns.net ([208.54.37.189]:29257 helo=[IPv6:2607:fb90:244d:ea8b:535:3725:cb42:e767]) by server172.web-hosting.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.86_1) (envelope-from ) id 1boCRz-002U0W-L3; Sun, 25 Sep 2016 12:38:49 -0400 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable From: Dave Rice Mime-Version: 1.0 (1.0) Date: Sun, 25 Sep 2016 12:34:33 -0400 Message-Id: <89CDA2AB-7A2E-4C5A-B85D-165FC33DAB6C@dericed.com> References: In-Reply-To: To: Steve Lhomme X-Mailer: iPhone Mail (13F69) X-OutGoing-Spam-Status: No, score=-2.1 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Cc: Codec Encoding for LossLess Archiving and Realtime transmission Subject: Re: [Cellar] EBML Element Parenting X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Sep 2016 16:38:52 -0000 Hi, > On Sep 25, 2016, at 11:51, Steve Lhomme wrote: >=20 > I went ahead and did a new Pull Request with an EBNF based notation to > define the path. I merged the minOccurs/maxOccurs as well as it's > simple enough to add it to the "regular expression" in a way anyone > can understand. Just reading that path line of an element gives you a > quick view of where it is found and how often (including recursively). >=20 > https://github.com/Matroska-Org/ebml-specification/pull/106 This looks good though I suggest that we have a mechanism to express a paren= t without the full path. In xpath '//' is used in this way. For instance "//Info/MuxingApp" could mean that MuxingApp could be a Child E= lement of any Info Element. Subsequently Info could be defined only at "/Seg= ment/Info". Through extrapolating one could see that MuxingApp can only be a= t "/Segment/Info/MuxingApp" without needed the full path. > Now after doing that I realized the XML schema wasn't as easy to parse > as before. Let me know what you think. We may have the EBNF notation > for each element definition in the RFC and the XML definition used > different parts of the EBNF definition in various attributes (likely > the ones we have before). >=20 > I struggled a bit to find a way to define the Void properly without > making the EBNF syntax too complex. >=20 > I left out more complex recursive possibilities like /a/b(/c/d)+ for > recursion jumping a level. It is not supported by current EBML parsers > and was never considered anyway. >=20 > I used the classic EBNF base with concatenation using the "," character. Such as "//Tags/SimpleTag,//SimpleTag/SimpleTag"? Dave > 2016-09-25 10:58 GMT+02:00 Reto Kromer : >> Steve Lhomme wrote: >>=20 >>> I found the POSIX regex >>> https://en.wikipedia.org/wiki/Regular_expression#Standards >>=20 >> The formal standard is: >>=20 >> ISO/IEC/IEEE 9945:2009 >> Information technology -- Portable Operating System >> Interface (POSIX) Base Specifications, Issue 7 >>=20 >> which contains much more than "just" the regex. >>=20 >> Best regards, Reto >>=20 >>=20 >> AV Preservation by reto.ch >> chemin du Suchet 5 | 1024 Ecublens | Switzerland >> Web: https://reto.ch | Twitter: @retoch >>=20 >> _______________________________________________ >> Cellar mailing list >> Cellar@ietf.org >> https://www.ietf.org/mailman/listinfo/cellar >=20 >=20 >=20 > --=20 > Steve Lhomme > Matroska association Chairman >=20 > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar From nobody Sun Sep 25 11:22:37 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A664712B007 for ; Sun, 25 Sep 2016 11:22:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8lpYjzVm0HqG for ; Sun, 25 Sep 2016 11:22:33 -0700 (PDT) Received: from mail-yw0-x22c.google.com (mail-yw0-x22c.google.com [IPv6:2607:f8b0:4002:c05::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 37F3212B011 for ; Sun, 25 Sep 2016 11:22:33 -0700 (PDT) Received: by mail-yw0-x22c.google.com with SMTP id i129so145741198ywb.0 for ; Sun, 25 Sep 2016 11:22:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=f8wR5CQzZ4iXuKEpRIBqoQj8PkRhwMe7/P6RfFkqVAk=; b=VVSr0j4Oh9RfPz11k8W22lYh0wIRo6i7Lc8GxdZnBd1bbgbgcr8LpkvAs8/Fb8doyl PO3r2diHwij69Ht4GIOzuVRn/svHUYJBfCCvV0isv3fFtL3iIEsXpdKPXfJybs/BhVB7 7Gi5N4j1bxdb6vUTtxu0U6Jh/cnFy9AquuEoqMCq+/PT97vcPeFsHBGytIYocuScyF2r MnvkILmPkOq4T9wk1/TrB0+C/gqeVksAPwWek94psn7aH5Q2Ww5uBu6ModXLROtiiz92 QGCv+KQ0aK3pXfHyA52YshvDECvxZL3N/Cpbn3gHdNur9kiN6gKroDRm3TKcuJSbdngz L5Ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=f8wR5CQzZ4iXuKEpRIBqoQj8PkRhwMe7/P6RfFkqVAk=; b=ZwM/qxxJwPex1iO0+lDKz07xuhe0U9YHhU0X4E+Kwe2LAL4DGqtR1AQsZP4oFq1Bgl +8MTgAqlvDANQZcSoZlVDt/e4eL656RFZrUG57TAfZ8olidzWbVzUDtkYx9wJv0zMqOl oS/X7TGIas6HWb4nV10QByhcslfsbUu1krLt+ECWP8c9LMn6OeH8weqGBr5RsPUcV20f SG8KNp97FI3fvoiCNqOq3k8d9ZLZIGgSpvv5maEHFrxP71v6p7Ji2YNO2EjheZ5ioXSq acnVRBqcxWfRKS/juvKsnwUfqUO1RElauWW8nbU3wowVEaSGDwJg2NEMRjAVlFFdyIjA 1HGA== X-Gm-Message-State: AE9vXwPMOd2yPwjG6m7bsZp14+ciSqJyXNGYVufoZt9AExva4SIN3RqY2G73DddQCW6EtYZ2gaUcEB6ZD9iE2w== X-Received: by 10.129.167.193 with SMTP id e184mr14708112ywh.60.1474827752462; Sun, 25 Sep 2016 11:22:32 -0700 (PDT) MIME-Version: 1.0 Received: by 10.83.20.135 with HTTP; Sun, 25 Sep 2016 11:22:31 -0700 (PDT) Received: by 10.83.20.135 with HTTP; Sun, 25 Sep 2016 11:22:31 -0700 (PDT) In-Reply-To: <89CDA2AB-7A2E-4C5A-B85D-165FC33DAB6C@dericed.com> References: <89CDA2AB-7A2E-4C5A-B85D-165FC33DAB6C@dericed.com> From: Steve Lhomme Date: Sun, 25 Sep 2016 20:22:31 +0200 Message-ID: To: Dave Rice Content-Type: multipart/alternative; boundary=001a114adddc4a85cf053d591807 Archived-At: Cc: Codec Encoding for LossLess Archiving and Realtime transmission Subject: Re: [Cellar] EBML Element Parenting X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Sep 2016 18:22:35 -0000 --001a114adddc4a85cf053d591807 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Le 25 sept. 2016 18:38, "Dave Rice" a =C3=A9crit : > > Hi, > > > On Sep 25, 2016, at 11:51, Steve Lhomme wrote: > > > > I went ahead and did a new Pull Request with an EBNF based notation to > > define the path. I merged the minOccurs/maxOccurs as well as it's > > simple enough to add it to the "regular expression" in a way anyone > > can understand. Just reading that path line of an element gives you a > > quick view of where it is found and how often (including recursively). > > > > https://github.com/Matroska-Org/ebml-specification/pull/106 > > This looks good though I suggest that we have a mechanism to express a parent without the full path. In xpath '//' is used in this way. > > For instance "//Info/MuxingApp" could mean that MuxingApp could be a Child Element of any Info Element. Subsequently Info could be defined only at "/Segment/Info". Through extrapolating one could see that MuxingApp can only be at "/Segment/Info/MuxingApp" without needed the full path. We could have something like that but the idea is that part of Matroska or different EBML formats could be defined elsewhere. It's a lot easier to understand unambiguously if you have the full path. Also if two formats use the same name for different things that becomes a problem. You may also have math formats using X, Y, Z all over the place. But that should be easy to add an EBMLElementParent in there. > > Now after doing that I realized the XML schema wasn't as easy to parse > > as before. Let me know what you think. We may have the EBNF notation > > for each element definition in the RFC and the XML definition used > > different parts of the EBNF definition in various attributes (likely > > the ones we have before). > > > > I struggled a bit to find a way to define the Void properly without > > making the EBNF syntax too complex. > > > > I left out more complex recursive possibilities like /a/b(/c/d)+ for > > recursion jumping a level. It is not supported by current EBML parsers > > and was never considered anyway. > > > > I used the classic EBNF base with concatenation using the "," character= . > > Such as "//Tags/SimpleTag,//SimpleTag/SimpleTag"? No. "a", "b" equals "ab". > Dave > > > 2016-09-25 10:58 GMT+02:00 Reto Kromer : > >> Steve Lhomme wrote: > >> > >>> I found the POSIX regex > >>> https://en.wikipedia.org/wiki/Regular_expression#Standards > >> > >> The formal standard is: > >> > >> ISO/IEC/IEEE 9945:2009 > >> Information technology -- Portable Operating System > >> Interface (POSIX) Base Specifications, Issue 7 > >> > >> which contains much more than "just" the regex. > >> > >> Best regards, Reto > >> > >> > >> AV Preservation by reto.ch > >> chemin du Suchet 5 | 1024 Ecublens | Switzerland > >> Web: https://reto.ch | Twitter: @retoch > >> > >> _______________________________________________ > >> Cellar mailing list > >> Cellar@ietf.org > >> https://www.ietf.org/mailman/listinfo/cellar > > > > > > > > -- > > Steve Lhomme > > Matroska association Chairman > > > > _______________________________________________ > > Cellar mailing list > > Cellar@ietf.org > > https://www.ietf.org/mailman/listinfo/cellar > --001a114adddc4a85cf053d591807 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

Le=C2=A025 sept. 2016 18:38, "Dave Rice" <dave@dericed.com> a =C3=A9crit=C2=A0:=
>
> Hi,
>
> > On Sep 25, 2016, at 11:51, Steve Lhomme <slhomme@matroska.org> wrote:
> >
> > I went ahead and did a new Pull Request with an EBNF based notati= on to
> > define the path. I merged the minOccurs/maxOccurs as well as it&#= 39;s
> > simple enough to add it to the "regular expression" in = a way anyone
> > can understand. Just reading that path line of an element gives y= ou a
> > quick view of where it is found and how often (including recursiv= ely).
> >
> > https://github.com/Matroska-Org/ebml-specification/pull/106
>
> This looks good though I suggest that we have a mechanism to express a= parent without the full path. In xpath '//' is used in this way. >
> For instance "//Info/MuxingApp" could mean that MuxingApp co= uld be a Child Element of any Info Element. Subsequently Info could be defi= ned only at "/Segment/Info". Through extrapolating one could see = that MuxingApp can only be at "/Segment/Info/MuxingApp" without n= eeded the full path.

We could have something like that but the idea is that part = of Matroska or different EBML formats could be defined elsewhere. It's = a lot easier to understand unambiguously if you have the full path. Also if= two formats use the same name for different things that becomes a problem.= You may also have math formats using X, Y, Z all over the place.

But that should be easy to add an EBMLElementParent in there= .

> > Now after doing that I realized the XML schema was= n't as easy to parse
> > as before. Let me know what you think. We may have the EBNF notat= ion
> > for each element definition in the RFC and the XML definition use= d
> > different parts of the EBNF definition in various attributes (lik= ely
> > the ones we have before).
> >
> > I struggled a bit to find a way to define the Void properly witho= ut
> > making the EBNF syntax too complex.
> >
> > I left out more complex recursive possibilities like /a/b(/c/d)+ = for
> > recursion jumping a level. It is not supported by current EBML pa= rsers
> > and was never considered anyway.
> >
> > I used the classic EBNF base with concatenation using the ",= " character.
>
> Such as "//Tags/SimpleTag,//SimpleTag/SimpleTag"?

No. "a", "b" equals "ab".

> Dave
>
> > 2016-09-25 10:58 GMT+02:00 Reto Kromer <lists@reto.ch>:
> >> Steve Lhomme wrote:
> >>
> >>> I found the POSIX regex
> >>> https://en.wikipedia.org/wiki/Regular_expression#Standards
> >>
> >> The formal standard is:
> >>
> >>=C2=A0 ISO/IEC/IEEE 9945:2009
> >>=C2=A0 Information technology -- Portable Operating System
> >>=C2=A0 Interface (POSIX) Base Specifications, Issue 7
> >>
> >> which contains much more than "just" the regex.
> >>
> >> Best regards, Reto
> >>
> >>
> >> AV Preservation by reto.ch
> >> chemin du Suchet 5 | 1024 Ecublens | Switzerland
> >> Web: https://reto.ch | Twitte= r: @retoch
> >>
> >> _______________________________________________
> >> Cellar mailing list
> >> Cellar@ietf.org
> >> http= s://www.ietf.org/mailman/listinfo/cellar
> >
> >
> >
> > --
> > Steve Lhomme
> > Matroska association Chairman
> >
> > _______________________________________________
> > Cellar mailing list
> > Cellar@ietf.org
> > https://= www.ietf.org/mailman/listinfo/cellar
>

--001a114adddc4a85cf053d591807-- From nobody Sun Sep 25 12:07:17 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 321ED12B011 for ; Sun, 25 Sep 2016 12:07:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.921 X-Spam-Level: X-Spam-Status: No, score=-1.921 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id w7cGKfqkL-o9 for ; Sun, 25 Sep 2016 12:07:14 -0700 (PDT) Received: from 6.mo173.mail-out.ovh.net (6.mo173.mail-out.ovh.net [46.105.43.93]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 07218127071 for ; Sun, 25 Sep 2016 12:07:13 -0700 (PDT) Received: from player157.ha.ovh.net (b9.ovh.net [213.186.33.59]) by mo173.mail-out.ovh.net (Postfix) with ESMTP id EA119FF9EBE for ; Sun, 25 Sep 2016 21:07:11 +0200 (CEST) Received: from [192.168.2.101] (p5DDB5F8D.dip0.t-ipconnect.de [93.219.95.141]) (Authenticated sender: jerome@francoallemand.eu) by player157.ha.ovh.net (Postfix) with ESMTPSA id A3A41500072 for ; Sun, 25 Sep 2016 21:07:11 +0200 (CEST) To: cellar@ietf.org References: <38E61A25-422A-482B-AFC0-0A6735572FBB@dericed.com> From: Jerome Martinez Message-ID: <2164b95e-4f9b-80bb-9bbd-4d246f08bf74@mediaarea.net> Date: Sun, 25 Sep 2016 21:07:11 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: <38E61A25-422A-482B-AFC0-0A6735572FBB@dericed.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Ovh-Tracer-Id: 2332583133001617553 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 0 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrfeelvddruddugdduheefucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfenuceurghilhhouhhtmecufedttdenuc Archived-At: Subject: Re: [Cellar] review of codec_specs.md for defining use of encodings in Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Sep 2016 19:07:16 -0000 Le 25/09/2016 à 00:05, Dave Rice a écrit : > [...] > > The terms ‘private data’ and the `CodecPrivate Element` seems to be referenced in the same way. Are these the same things? If so, they should be made consistent. They are same thing, as described: "CodecPrivate element: Private data only known to the codec." but right, maybe we need to add more information about it. > The phrase "The private data is void.” is used several times in the document. Does this mean that the `CodecPrivate Element` is stored but as an `Empty Element`? Could the `CodecPrivate Element` just not be present if not needed? Not present if not needed is the implemented way. Maybe rephrasing the sentence would make it more explicit (e.g. "no CodecPrivate Element" instead, for stream format not needing configuration bytes). > [...] > > Other questions: > Should the Matroska specification provide enough info for a user to independently define support for an encoding in Matroska? Or should codec support somehow be ‘registered’ via Matroska and/or CELLAR? First, the list of supported formats and their respective definition should be in a separate document (updated more often than the core spec). And yes, each new format should be registered, because there is an unique codec name namespace. Maybe defining "X_" prefix as private (but mime RFC now discourages use of "x-", so maybe not a good idea). Now, the question is where to define CodecPrivate. e.g. for FFV1 we already defined it in the FFV1 spec but we can not update AVC specs for that. > [...] > > For a Matroska validation that finds something like V_MPEG4/ISO/AVC in Matroska. Should it be considered as unsupported (most MKV files!)? Matroska-centric validation tools should not validate raw streams, that includes CodecPrivate part. This is the role of a more general validation tool (a tool described as "validation of Matroska files" would not validate the CodecPrivate part, a tool described as "validation of Matroska files + FFV1 and FLAC" would use the description of how FFV1 and FLAC are registered) From nobody Sun Sep 25 14:31:50 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C833712B02D for ; Sun, 25 Sep 2016 14:31:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.12 X-Spam-Level: X-Spam-Status: No, score=-1.12 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GmO9Gu_HUIxK for ; Sun, 25 Sep 2016 14:31:46 -0700 (PDT) Received: from s172.web-hosting.com (s172.web-hosting.com [68.65.122.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0671B12B028 for ; Sun, 25 Sep 2016 14:31:45 -0700 (PDT) Received: from cpe-184-152-63-84.nyc.res.rr.com ([184.152.63.84]:48184 helo=[10.0.1.77]) by server172.web-hosting.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.86_1) (envelope-from ) id 1boH1S-000S2j-2z; Sun, 25 Sep 2016 17:31:45 -0400 Content-Type: multipart/alternative; boundary="Apple-Mail=_97E38AC1-5F01-46D6-A8A4-ED320FD1AC62" Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) From: Dave Rice In-Reply-To: Date: Sun, 25 Sep 2016 17:31:37 -0400 Message-Id: References: <89CDA2AB-7A2E-4C5A-B85D-165FC33DAB6C@dericed.com> To: Steve Lhomme X-Mailer: Apple Mail (2.3124) X-OutGoing-Spam-Status: No, score=-2.9 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Cc: Codec Encoding for LossLess Archiving and Realtime transmission Subject: Re: [Cellar] EBML Element Parenting X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Sep 2016 21:31:49 -0000 --Apple-Mail=_97E38AC1-5F01-46D6-A8A4-ED320FD1AC62 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi, I finished a review of = https://github.com/Matroska-Org/ebml-specification/pull/106 = with many = comments there. Overall I like the new version of EBML Path expression = based on an EBNF-defined convention. > On Sep 25, 2016, at 2:22 PM, Steve Lhomme = wrote: > Le 25 sept. 2016 18:38, "Dave Rice" > a =C3=A9crit : > > > > Hi, > > > > > On Sep 25, 2016, at 11:51, Steve Lhomme > wrote: > > > > > > I went ahead and did a new Pull Request with an EBNF based = notation to > > > define the path. I merged the minOccurs/maxOccurs as well as it's > > > simple enough to add it to the "regular expression" in a way = anyone > > > can understand. Just reading that path line of an element gives = you a > > > quick view of where it is found and how often (including = recursively). > > > > > > https://github.com/Matroska-Org/ebml-specification/pull/106 = > > > > This looks good though I suggest that we have a mechanism to express = a parent without the full path. In xpath '//' is used in this way. > > > > For instance "//Info/MuxingApp" could mean that MuxingApp could be a = Child Element of any Info Element. Subsequently Info could be defined = only at "/Segment/Info". Through extrapolating one could see that = MuxingApp can only be at "/Segment/Info/MuxingApp" without needed the = full path. >=20 > We could have something like that but the idea is that part of = Matroska or different EBML formats could be defined elsewhere. It's a = lot easier to understand unambiguously if you have the full path. Also = if two formats use the same name for different things that becomes a = problem. You may also have math formats using X, Y, Z all over the = place. >=20 Yes but it could cut some of the semantic redundancy. For instance = `Language`, `ChapLanguage`, and `TagLanguage` could be all defined as = documenting the language of their Parent Element=E2=80=99s context. name=3D=E2=80=9CLanguage=E2=80=9D = path=3D"//TrackEntry/Language|//ChapterDisplay/Language|//SimpleTag/Langua= ge=E2=80=9D Also in the PR the concepts of minOccur and maxOccur are moved from = attributes into the path expression such as = =E2=80=9C/EBML/EBMLVersion(1,1)=E2=80=9D. This makes it a bit more = difficult as one would now have to parse the XML and then parse the = path. Not a blocking issue, but for tools that will depend on the EBML = Schema, parsing out queries such as =E2=80=98find all mandatory = elements=E2=80=99 or =E2=80=98all non-repeating element=E2=80=99 may be = more difficult. > But that should be easy to add an EBMLElementParent in there. >=20 > > > Now after doing that I realized the XML schema wasn't as easy to = parse > > > as before. Let me know what you think. We may have the EBNF = notation > > > for each element definition in the RFC and the XML definition used > > > different parts of the EBNF definition in various attributes = (likely > > > the ones we have before). > > > > > > I struggled a bit to find a way to define the Void properly = without > > > making the EBNF syntax too complex. > > > > > > I left out more complex recursive possibilities like /a/b(/c/d)+ = for > > > recursion jumping a level. It is not supported by current EBML = parsers > > > and was never considered anyway. > > > > > > I used the classic EBNF base with concatenation using the "," = character. > > > > Such as "//Tags/SimpleTag,//SimpleTag/SimpleTag"? >=20 > No. "a", "b" equals "ab". >=20 > > Dave > > > > > 2016-09-25 10:58 GMT+02:00 Reto Kromer >: > > >> Steve Lhomme wrote: > > >> > > >>> I found the POSIX regex > > >>> https://en.wikipedia.org/wiki/Regular_expression#Standards = > > >> > > >> The formal standard is: > > >> > > >> ISO/IEC/IEEE 9945:2009 > > >> Information technology -- Portable Operating System > > >> Interface (POSIX) Base Specifications, Issue 7 > > >> > > >> which contains much more than "just" the regex. > > >> > > >> Best regards, Reto > > >> > > >> > > >> AV Preservation by reto.ch > > >> chemin du Suchet 5 | 1024 Ecublens | Switzerland > > >> Web: https://reto.ch | Twitter: @retoch > > >> > > >> _______________________________________________ > > >> Cellar mailing list > > >> Cellar@ietf.org > > >> https://www.ietf.org/mailman/listinfo/cellar = > > > > > > > > > > > > -- > > > Steve Lhomme > > > Matroska association Chairman > > > > > > _______________________________________________ > > > Cellar mailing list > > > Cellar@ietf.org > > > https://www.ietf.org/mailman/listinfo/cellar = > > >=20 Dave Rice= --Apple-Mail=_97E38AC1-5F01-46D6-A8A4-ED320FD1AC62 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
Hi,

I finished a review of  https://github.com/Matroska-Org/ebml-specification/pull/106=  with many comments there. Overall I like the new version of EBML = Path expression based on an EBNF-defined convention.

On = Sep 25, 2016, at 2:22 PM, Steve Lhomme <slhomme@matroska.org> wrote:

Le 25 sept. 2016 18:38, "Dave Rice" <dave@dericed.com> a = =C3=A9crit :
>
> Hi,
>
> > On Sep 25, 2016, at 11:51, Steve Lhomme <slhomme@matroska.org> wrote:
> >
> > I went ahead and did a new Pull Request with an EBNF based = notation to
> > define the path. I merged the minOccurs/maxOccurs as well as = it's
> > simple enough to add it to the "regular expression" in a way = anyone
> > can understand. Just reading that path line of an element = gives you a
> > quick view of where it is found and how often (including = recursively).
> >
> > https://github.com/Matroska-Org/ebml-specification/pull/106=
>
> This looks good though I suggest that we have a mechanism to = express a parent without the full path. In xpath '//' is used in this = way.
>
> For instance "//Info/MuxingApp" could mean that MuxingApp could be = a Child Element of any Info Element. Subsequently Info could be defined = only at "/Segment/Info". Through extrapolating one could see that = MuxingApp can only be at "/Segment/Info/MuxingApp" without needed the = full path.

We could have something like = that but the idea is that part of Matroska or different EBML formats = could be defined elsewhere. It's a lot easier to understand = unambiguously if you have the full path. Also if two formats use the = same name for different things that becomes a problem. You may also have = math formats using X, Y, Z all over the place. =

Yes but it could cut some of the semantic = redundancy. For instance `Language`, `ChapLanguage`, and `TagLanguage` = could be all defined as documenting the language of their Parent = Element=E2=80=99s context.

name=3D=E2=80=9CLanguage=E2=80=9D
path=3D"= //TrackEntry/Language|//ChapterDisplay/Language|//SimpleTag/Language=E2=80= =9D

Also in the PR the concepts of = minOccur and maxOccur are moved from <element> attributes into the = path expression such as =E2=80=9C/EBML/EBMLVersion(1,1)=E2=80=9D. This = makes it a bit more difficult as one would now have to parse the XML and = then parse the path. Not a blocking issue, but for tools that will = depend on the EBML Schema, parsing out queries such as =E2=80=98find all = mandatory elements=E2=80=99 or =E2=80=98all non-repeating element=E2=80=99= may be more difficult.

But that should be easy to add an = EBMLElementParent in there.

> > Now = after doing that I realized the XML schema wasn't as easy to parse
> > as before. Let me know what you think. We may have the EBNF = notation
> > for each element definition in the RFC and the XML definition = used
> > different parts of the EBNF definition in various attributes = (likely
> > the ones we have before).
> >
> > I struggled a bit to find a way to define the Void properly = without
> > making the EBNF syntax too complex.
> >
> > I left out more complex recursive possibilities like = /a/b(/c/d)+ for
> > recursion jumping a level. It is not supported by current EBML = parsers
> > and was never considered anyway.
> >
> > I used the classic EBNF base with concatenation using the "," = character.
>
> Such as "//Tags/SimpleTag,//SimpleTag/SimpleTag"?

No. "a", "b" equals "ab".

> = Dave
>
> > 2016-09-25 10:58 GMT+02:00 Reto Kromer <lists@reto.ch>:
> >> Steve Lhomme wrote:
> >>
> >>> I found the POSIX regex
> >>> https://en.wikipedia.org/wiki/Regular_expression#Standards<= br class=3D""> > >>
> >> The formal standard is:
> >>
> >>  ISO/IEC/IEEE 9945:2009
> >>  Information technology -- Portable Operating = System
> >>  Interface (POSIX) Base Specifications, Issue 7
> >>
> >> which contains much more than "just" the regex.
> >>
> >> Best regards, Reto
> >>
> >>
> >> AV Preservation by reto.ch
> >> chemin du Suchet 5 | 1024 Ecublens | Switzerland
> >> Web: https://reto.ch | Twitter: @retoch
> >>
> >> _______________________________________________
> >> Cellar mailing list
> >> Cellar@ietf.org
> >> https://www.ietf.org/mailman/listinfo/cellar
= > >
> >
> >
> > --
> > Steve Lhomme
> > Matroska association Chairman
> >
> > _______________________________________________
> > Cellar mailing list
> > Cellar@ietf.org
> > https://www.ietf.org/mailman/listinfo/cellar
= >

Dave Rice= --Apple-Mail=_97E38AC1-5F01-46D6-A8A4-ED320FD1AC62-- From nobody Sun Sep 25 14:37:40 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B656512B043 for ; Sun, 25 Sep 2016 14:37:38 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.121 X-Spam-Level: X-Spam-Status: No, score=-1.121 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id haXq_6Ndf3MS for ; Sun, 25 Sep 2016 14:37:37 -0700 (PDT) Received: from s172.web-hosting.com (s172.web-hosting.com [68.65.122.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 361E212B042 for ; Sun, 25 Sep 2016 14:37:37 -0700 (PDT) Received: from cpe-184-152-63-84.nyc.res.rr.com ([184.152.63.84]:48438 helo=[10.0.1.77]) by server172.web-hosting.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.86_1) (envelope-from ) id 1boH79-000Tpj-1C; Sun, 25 Sep 2016 17:37:36 -0400 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) From: Dave Rice In-Reply-To: <2164b95e-4f9b-80bb-9bbd-4d246f08bf74@mediaarea.net> Date: Sun, 25 Sep 2016 17:37:30 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <3B99BB70-4228-4DAD-999E-8DBFEA024838@dericed.com> References: <38E61A25-422A-482B-AFC0-0A6735572FBB@dericed.com> <2164b95e-4f9b-80bb-9bbd-4d246f08bf74@mediaarea.net> To: Jerome Martinez X-Mailer: Apple Mail (2.3124) X-OutGoing-Spam-Status: No, score=-2.9 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Cc: cellar@ietf.org Subject: Re: [Cellar] review of codec_specs.md for defining use of encodings in Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Sep 2016 21:37:39 -0000 > On Sep 25, 2016, at 3:07 PM, Jerome Martinez = wrote: >=20 > Le 25/09/2016 =C3=A0 00:05, Dave Rice a =C3=A9crit : >> [...] >>=20 >> The terms =E2=80=98private data=E2=80=99 and the `CodecPrivate = Element` seems to be referenced in the same way. Are these the same = things? If so, they should be made consistent. >=20 > They are same thing, as described: "CodecPrivate element: Private data = only known to the codec." but right, maybe we need to add more = information about it. If CodecPrivate is referenced by nearly every definition of an encoding = in Matroska, perhaps we should split the =E2=80=98Description=E2=80=99 = into =E2=80=98CodecPrivate Usage=E2=80=99 and =E2=80=98Description=E2=80=99= . >> The phrase "The private data is void.=E2=80=9D is used several times = in the document. Does this mean that the `CodecPrivate Element` is = stored but as an `Empty Element`? Could the `CodecPrivate Element` just = not be present if not needed? >=20 > Not present if not needed is the implemented way. Maybe rephrasing the = sentence would make it more explicit (e.g. "no CodecPrivate Element" = instead, for stream format not needing configuration bytes). >=20 >> [...] >>=20 >> Other questions: >> Should the Matroska specification provide enough info for a user to = independently define support for an encoding in Matroska? Or should = codec support somehow be =E2=80=98registered=E2=80=99 via Matroska = and/or CELLAR? >=20 > First, the list of supported formats and their respective definition = should be in a separate document (updated more often than the core = spec). +1 For the chairs, if we are splitting an intention of one document = (Matroska) into several documents (Matroska, Matroska Encoding Registry, = Matroska Metadata Registry) do we need to update the charter? > And yes, each new format should be registered, because there is an = unique codec name namespace. Maybe defining "X_" prefix as private (but = mime RFC now discourages use of "x-", so maybe not a good idea). +1 > Now, the question is where to define CodecPrivate. e.g. for FFV1 we = already defined it in the FFV1 spec but we can not update AVC specs for = that. >=20 >> [...] >>=20 >> For a Matroska validation that finds something like V_MPEG4/ISO/AVC = in Matroska. Should it be considered as unsupported (most MKV files!)? >=20 > Matroska-centric validation tools should not validate raw streams, = that includes CodecPrivate part. This is the role of a more general = validation tool (a tool described as "validation of Matroska files" = would not validate the CodecPrivate part, a tool described as = "validation of Matroska files + FFV1 and FLAC" would use the description = of how FFV1 and FLAC are registered) There are still some tests permitted by only the Matroska specification. = For instance with a revision we could make it clear which Codec ID MUST = or MUST NOT have an associated `CodecPrivate Element`. Dave Rice= From nobody Sun Sep 25 18:00:38 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A7C8112B056 for ; Sun, 25 Sep 2016 18:00:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.699 X-Spam-Level: X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8DWTniMCeGEi for ; Sun, 25 Sep 2016 18:00:33 -0700 (PDT) Received: from mail-io0-x22c.google.com (mail-io0-x22c.google.com [IPv6:2607:f8b0:4001:c06::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D733712B053 for ; Sun, 25 Sep 2016 18:00:32 -0700 (PDT) Received: by mail-io0-x22c.google.com with SMTP id m186so167523221ioa.2 for ; Sun, 25 Sep 2016 18:00:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=KfHWEywlYcA0Ix2UYVH46KQq6vMqDWtS9h9z5MH9tHQ=; b=D5nZwDMhRNJ+HimrI3hwefRjJrzwmOn7ymwG+04TtXroxWD0ZpYkPPx5uQ2RyCx3hp pddQzjhrmUUkyBdXGU8NQBndeuKgtaimsfYC3LUB0lx/1DvpukHEThIG+GN34hzi3FVu K7OL746XN0VO7hWmsE6LNx0RsXnzPF6mlSLLsG07bBrC+DyfGfzIUrG/DD56vC96XV/8 pPntkF7b+Cg6U+ymrYQlEcGXNsYUy/Pli/in/VXeY/n1e3ug3BZYI8A6iQGfz9rBtXa7 jqhD8QIzsPu8S4Lf7LiocrMvPUBiX7oaFALhrpgkDLb7EjcAvuDJVdmShBftun58mP4A z5gQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=KfHWEywlYcA0Ix2UYVH46KQq6vMqDWtS9h9z5MH9tHQ=; b=We8I2/3TjbNN/5JPGrTPWUOGsa4I1VTGcDLVq1/oRrzBsgE9CYlJBvnZt8LQH/UNlX GAliZPn8NsSB5aoqyLGtBJ46hOZtfUFGxq+AmOm621FpBVkGU8mUem0qsDE9icNRgYUF 5N4JEkAeGjZi8yRkTNLMgdaSAXa/C+lkO+YcsJOMtvbeQc9tAYcTeBXsPLDBWukiDGng u5qWipBpNjBigAnYg55Z3mnv6fHN6JdfXAx4sl5xOUQVfF8ajlZuai244fqHJ1iS7X5I 2A1376Vj+a+dNPhxmtIX1ZRtiuQaJywMzSX6VqnoDQtPjPZyjJu3uARuGDfpO2n5MZf8 LkKQ== X-Gm-Message-State: AE9vXwOo4sJuHT1dPQtBGDzAQxNd+6EI959tv9N1xDcq1SL1lPSDOfNMO+QVa8DtcAEI5t3cXQVJ5R0IbdgNjQ== X-Received: by 10.107.130.140 with SMTP id m12mr19630344ioi.8.1474851632147; Sun, 25 Sep 2016 18:00:32 -0700 (PDT) MIME-Version: 1.0 Received: by 10.107.40.4 with HTTP; Sun, 25 Sep 2016 18:00:30 -0700 (PDT) Received: by 10.107.40.4 with HTTP; Sun, 25 Sep 2016 18:00:30 -0700 (PDT) In-Reply-To: <3B99BB70-4228-4DAD-999E-8DBFEA024838@dericed.com> References: <38E61A25-422A-482B-AFC0-0A6735572FBB@dericed.com> <2164b95e-4f9b-80bb-9bbd-4d246f08bf74@mediaarea.net> <3B99BB70-4228-4DAD-999E-8DBFEA024838@dericed.com> From: Tessa Fallon Date: Sun, 25 Sep 2016 21:00:30 -0400 Message-ID: To: Dave Rice Content-Type: multipart/alternative; boundary=001a113fbf1ea1733e053d5ea7b4 Archived-At: Cc: Jerome Martinez , cellar@ietf.org Subject: Re: [Cellar] review of codec_specs.md for defining use of encodings in Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Sep 2016 01:00:36 -0000 --001a113fbf1ea1733e053d5ea7b4 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Re: multiple documents, I am not 100% sure and defer to those with more experience at the IETF to indicate if there is precedent; if it is not part of the core spec, then it/they could be published as informational drafts. However, before that I would ask for group consensus to ensure that this fragmentation is absolutely necessary, and that the implications are fully outlined and understood. Also, I'd ask for it to be made clear why, if the items in question are not part of the core spec, they merit publication under the CELLAR charter, as we'd be adding two more documents to our already substantial list of documents to be published. Best Tessa On Sep 25, 2016 17:37, "Dave Rice" wrote: > > > On Sep 25, 2016, at 3:07 PM, Jerome Martinez > wrote: > > > > Le 25/09/2016 =C3=A0 00:05, Dave Rice a =C3=A9crit : > >> [...] > >> > >> The terms =E2=80=98private data=E2=80=99 and the `CodecPrivate Element= ` seems to be > referenced in the same way. Are these the same things? If so, they should > be made consistent. > > > > They are same thing, as described: "CodecPrivate element: Private data > only known to the codec." but right, maybe we need to add more informatio= n > about it. > > If CodecPrivate is referenced by nearly every definition of an encoding i= n > Matroska, perhaps we should split the =E2=80=98Description=E2=80=99 into = =E2=80=98CodecPrivate > Usage=E2=80=99 and =E2=80=98Description=E2=80=99. > > >> The phrase "The private data is void.=E2=80=9D is used several times i= n the > document. Does this mean that the `CodecPrivate Element` is stored but as > an `Empty Element`? Could the `CodecPrivate Element` just not be present = if > not needed? > > > > Not present if not needed is the implemented way. Maybe rephrasing the > sentence would make it more explicit (e.g. "no CodecPrivate Element" > instead, for stream format not needing configuration bytes). > > > >> [...] > >> > >> Other questions: > >> Should the Matroska specification provide enough info for a user to > independently define support for an encoding in Matroska? Or should codec > support somehow be =E2=80=98registered=E2=80=99 via Matroska and/or CELLA= R? > > > > First, the list of supported formats and their respective definition > should be in a separate document (updated more often than the core spec). > > +1 > > For the chairs, if we are splitting an intention of one document > (Matroska) into several documents (Matroska, Matroska Encoding Registry, > Matroska Metadata Registry) do we need to update the charter? > > > And yes, each new format should be registered, because there is an > unique codec name namespace. Maybe defining "X_" prefix as private (but > mime RFC now discourages use of "x-", so maybe not a good idea). > > +1 > > > Now, the question is where to define CodecPrivate. e.g. for FFV1 we > already defined it in the FFV1 spec but we can not update AVC specs for > that. > > > >> [...] > >> > >> For a Matroska validation that finds something like V_MPEG4/ISO/AVC in > Matroska. Should it be considered as unsupported (most MKV files!)? > > > > Matroska-centric validation tools should not validate raw streams, that > includes CodecPrivate part. This is the role of a more general validation > tool (a tool described as "validation of Matroska files" would not valida= te > the CodecPrivate part, a tool described as "validation of Matroska files = + > FFV1 and FLAC" would use the description of how FFV1 and FLAC are > registered) > > There are still some tests permitted by only the Matroska specification. > For instance with a revision we could make it clear which Codec ID MUST o= r > MUST NOT have an associated `CodecPrivate Element`. > > Dave Rice > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > --001a113fbf1ea1733e053d5ea7b4 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

Re: multiple documents, I am not 100% sure and defer to thos= e with more experience at the IETF to indicate if there is precedent; if it= is not part of the core spec, then it/they could be published as informati= onal drafts.=C2=A0 However, before that I would ask for group consensus to = ensure that this fragmentation is absolutely necessary, and that the implic= ations are fully outlined and understood.

Also, I'd ask for it to be made clear why, if the items = in question are not part of the core spec, they merit publication under the= CELLAR charter, as we'd be adding two more documents to our already su= bstantial list of documents to be published.

Best

Tessa


On Sep 25, 2016 1= 7:37, "Dave Rice" <dave@de= riced.com> wrote:

> On Sep 25, 2016, at 3:07 PM, Jerome Martinez <Jerome@MediaArea.net&= gt; wrote:
>
> Le 25/09/2016 =C3=A0 00:05, Dave Rice a =C3=A9crit :
>> [...]
>>
>> The terms =E2=80=98private data=E2=80=99 and the `CodecPrivate Ele= ment` seems to be referenced in the same way. Are these the same things? If= so, they should be made consistent.
>
> They are same thing, as described: "CodecPrivate element: Private= data only known to the codec." but right, maybe we need to add more i= nformation about it.

If CodecPrivate is referenced by nearly every definition of an encoding in = Matroska, perhaps we should split the =E2=80=98Description=E2=80=99 into = =E2=80=98CodecPrivate Usage=E2=80=99 and =E2=80=98Description=E2=80=99.

>> The phrase "The private data is void.=E2=80=9D is used severa= l times in the document. Does this mean that the `CodecPrivate Element` is = stored but as an `Empty Element`? Could the `CodecPrivate Element` just not= be present if not needed?
>
> Not present if not needed is the implemented way. Maybe rephrasing the= sentence would make it more explicit (e.g. "no CodecPrivate Element&q= uot; instead, for stream format not needing configuration bytes).
>
>> [...]
>>
>> Other questions:
>> Should the Matroska specification provide enough info for a user t= o independently define support for an encoding in Matroska? Or should codec= support somehow be =E2=80=98registered=E2=80=99 via Matroska and/or CELLAR= ?
>
> First, the list of supported formats and their respective definition s= hould be in a separate document (updated more often than the core spec).
+1

For the chairs, if we are splitting an intention of one document (Matroska)= into several documents (Matroska, Matroska Encoding Registry, Matroska Met= adata Registry) do we need to update the charter?

> And yes, each new format should be registered, because there is an uni= que codec name namespace. Maybe defining "X_" prefix as private (= but mime RFC now discourages use of "x-", so maybe not a good ide= a).

+1

> Now, the question is where to define CodecPrivate. e.g. for FFV1 we al= ready defined it in the FFV1 spec but we can not update AVC specs for that.=
>
>> [...]
>>
>> For a Matroska validation that finds something like V_MPEG4/ISO/AV= C in Matroska. Should it be considered as unsupported (most MKV files!)? >
> Matroska-centric validation tools should not validate raw streams, tha= t includes CodecPrivate part. This is the role of a more general validation= tool (a tool described as "validation of Matroska files" would n= ot validate the CodecPrivate part, a tool described as "validation of = Matroska files + FFV1 and FLAC" would use the description of how FFV1 = and FLAC are registered)

There are still some tests permitted by only the Matroska specification. Fo= r instance with a revision we could make it clear which Codec ID MUST or MU= ST NOT have an associated `CodecPrivate Element`.

Dave Rice
_______________________________________________
Cellar mailing list
Cellar@ietf.org
https://www.ietf.org/mailman/listinfo/cellar
--001a113fbf1ea1733e053d5ea7b4-- From nobody Wed Sep 28 04:20:37 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2DF0E12B110 for ; Wed, 28 Sep 2016 04:20:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.6 X-Spam-Level: X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OggqlHgwMMg6 for ; Wed, 28 Sep 2016 04:20:32 -0700 (PDT) Received: from zucker.schokokeks.org (zucker.schokokeks.org [178.63.68.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 52A9B12B43A for ; Wed, 28 Sep 2016 04:20:32 -0700 (PDT) Received: from [10.0.0.2] (1360030002.d-dsl.at [::ffff:81.16.105.50]) (AUTH: PLAIN bubestinger@schokokeks.org, TLS: TLSv1/SSLv3, 128bits, ECDHE-RSA-AES128-GCM-SHA256) by zucker.schokokeks.org with ESMTPSA; Wed, 28 Sep 2016 13:20:30 +0200 id 000000000000011A.0000000057EBA77E.00005B6B Message-ID: <57EBA77C.60908@das-werkstatt.com> Date: Wed, 28 Sep 2016 13:20:28 +0200 From: "Peter B." User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: cellar@ietf.org References: <57E385A1.3060807@das-werkstatt.com> <91c9a832455c44f0ae5de446aeb45d55@in-cci-exch08.ads.iu.edu> In-Reply-To: <91c9a832455c44f0ae5de446aeb45d55@in-cci-exch08.ads.iu.edu> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [Cellar] Risk assessment for FFV1 and Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Sep 2016 11:20:35 -0000 Dear Mike, Dear list :) At the Mediathek, we have never written a formal paper - but we've put together certain aspects to consider - and how we are planning to deal with them. I've now finally taken the time to write them down in a way that's more suitable for public viewing: http://download.das-werkstatt.com/pb/mthk/info/video/risk_assessment.html I wrote it in a way which is short enough, so people read it - and not too technical, while being detailed enough to reproduce our decisions and considerations. Please let me know if you find it useful or have comments :) Kind regards, Pb On 09/22/2016 06:50 PM, Casey, Michael T wrote: > Peter, > > Yes, I'm interested and would like to know more details. I suspect that the European project that I am dialoguing with would also be interested. > > Thanks, > > Mike > > -----Original Message----- > From: Cellar [mailto:cellar-bounces@ietf.org] On Behalf Of Peter B. > Sent: Thursday, September 22, 2016 3:18 AM > To: cellar@ietf.org > Subject: Re: [Cellar] Risk assessment for FFV1 and Matroska > > Dear Michael, > > Since we started using FFV1 at the Austrian Mediathek in 2010, without assuming it'd become any standard soon, we worked on risk-assessment considerations. > > If you're interested, I'll write in more detail later :) > > > Kind regards, > Peter > > > On 09/21/2016 08:49 PM, Casey, Michael T wrote: >> I have a couple of questions from a large digitization project in Europe about the risk associated with choosing FFV1 and Matroska. This is from their email: >> >> You guys have chosen for FFv1 last year, but the standardisation process is still going on (IETF, CELLAR) as you know. >> >> What did you think about the risk of going for a not-yet-standardized codec? Did you somehow take into account the chance that the standardized FFv1 will look a little different than the one you're using now? >> >> I am wondering if you have any thoughts on this that I could pass along? >> > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > From nobody Wed Sep 28 04:54:40 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 96B1C12B0E9 for ; Wed, 28 Sep 2016 04:54:38 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.202 X-Spam-Level: X-Spam-Status: No, score=-1.202 tagged_above=-999 required=5 tests=[BAYES_05=-0.5, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mbBH3kCi09GW for ; Wed, 28 Sep 2016 04:54:36 -0700 (PDT) Received: from smtp-sh.infomaniak.ch (smtp-sh.infomaniak.ch [128.65.195.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B1B7E12B0BE for ; Wed, 28 Sep 2016 04:54:35 -0700 (PDT) Received: from smtp5.infomaniak.ch (smtp5.infomaniak.ch [83.166.132.18]) by smtp-sh.infomaniak.ch (8.14.5/8.14.5) with ESMTP id u8SBsX27016623 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL) for ; Wed, 28 Sep 2016 13:54:33 +0200 Received: from Castor.local (dynamic.wline.6rd.res.cust.swisscom.ch [IPv6:2a02:1205:c69b:8640:6137:49aa:b5bd:23ff] (may be forged)) (authenticated bits=0) by smtp5.infomaniak.ch (8.14.5/8.14.5) with ESMTP id u8SBsWAY054861 for ; Wed, 28 Sep 2016 13:54:33 +0200 Date: Wed, 28 Sep 2016 13:54:33 +0200 From: Reto Kromer To: cellar@ietf.org X-Priority: 3 In-Reply-To: <57EBA77C.60908@das-werkstatt.com> Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Mailer: Mailsmith 2.4 (470) X-Antivirus: Dr.Web (R) for Unix mail servers drweb plugin ver.6.0.2.8 X-Antivirus-Code: 0x100000 Archived-At: Subject: Re: [Cellar] Risk assessment for FFV1 and Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Sep 2016 11:54:38 -0000 Peter B. wrote: >I've now finally taken the time to write them down in a way >that's more suitable for public viewing: > >http://download.das-werkstatt.com/pb/mthk/info/video/risk_assessment.html In my opinion, this is a very useful document for advocating purposes. Thank you very much indeed, Peter! Best regards, Reto AV Preservation by reto.ch chemin du Suchet 5 | 1024 Ecublens | Switzerland Web: https://reto.ch | Twitter: @retoch From nobody Wed Sep 28 05:36:02 2016 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E84E112B0F8 for ; Wed, 28 Sep 2016 05:36:00 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.6 X-Spam-Level: X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XF3muRGEKJZo for ; Wed, 28 Sep 2016 05:35:59 -0700 (PDT) Received: from zucker.schokokeks.org (zucker.schokokeks.org [178.63.68.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0E0B812B107 for ; Wed, 28 Sep 2016 05:35:58 -0700 (PDT) Received: from [10.0.0.2] (1360030002.d-dsl.at [::ffff:81.16.105.50]) (AUTH: PLAIN bubestinger@schokokeks.org, TLS: TLSv1/SSLv3, 128bits, ECDHE-RSA-AES128-GCM-SHA256) by zucker.schokokeks.org with ESMTPSA; Wed, 28 Sep 2016 14:35:57 +0200 id 0000000000000120.0000000057EBB92E.0000453B Message-ID: <57EBB92C.9050105@das-werkstatt.com> Date: Wed, 28 Sep 2016 14:35:56 +0200 From: "Peter B." User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: cellar@ietf.org References: In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [Cellar] Risk assessment for FFV1 and Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Sep 2016 12:36:01 -0000 On 09/28/2016 01:54 PM, Reto Kromer wrote: > Peter B. wrote: > >> http://download.das-werkstatt.com/pb/mthk/info/video/risk_assessment.html >> > > In my opinion, this is a very useful document for advocating > purposes. Thank you very much indeed, Peter! Happy to hear you find it useful :D Thank you for the compliments! I hope it doesn't sound like an Open Source evangelism paper. It's just so that from my technical point of view, it *is* the open nature which makes things better for us archivists. Feedback/criticism greatly welcome. Regards, Pb From nobody Thu Sep 29 15:11:11 2016 Return-Path: X-Original-To: cellar@ietf.org Delivered-To: cellar@ietfa.amsl.com Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 9132A12B23B; Thu, 29 Sep 2016 15:11:08 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: "\"IETF Meeting Session Request Tool\"" To: X-Test-IDTracker: no X-IETF-IDTracker: 6.34.1 Auto-Submitted: auto-generated Precedence: bulk Message-ID: <147518706856.18617.2354258240369083459.idtracker@ietfa.amsl.com> Date: Thu, 29 Sep 2016 15:11:08 -0700 Archived-At: Cc: ben@nostrum.com, tessa.fallon@gmail.com, cellar@ietf.org, cellar-chairs@ietf.org Subject: [Cellar] cellar - Not having a session at IETF 97 X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.17 List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Sep 2016 22:11:08 -0000 Tessa Fallon, a chair of the cellar working group, indicated that the cellar working group does not plan to hold a session at IETF 97. This message was generated and sent by the IETF Meeting Session Request Tool.