From nobody Mon Jul 2 00:39:06 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7D4A1130F11 for ; Mon, 2 Jul 2018 00:38:57 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.909 X-Spam-Level: X-Spam-Status: No, score=-1.909 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, T_DKIMWL_WL_MED=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8XZQ0PjXCGvS for ; Mon, 2 Jul 2018 00:38:55 -0700 (PDT) Received: from mail-pl0-x236.google.com (mail-pl0-x236.google.com [IPv6:2607:f8b0:400e:c01::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 48C14130E54 for ; Mon, 2 Jul 2018 00:38:55 -0700 (PDT) Received: by mail-pl0-x236.google.com with SMTP id z9-v6so7527783plo.1 for ; Mon, 02 Jul 2018 00:38:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=srnBCtW0M5+UsaYWzvdF3UDB+TVGvu/kPJJEXlyqIx8=; b=OzjHUACN4o80aNctbpn8yydz3ThoJcnv0EDqTR7DD3GCqryzONaRS8kweCKbgECD4U AEnH/yaTunQskTe6Gt8UqXVbBOpaVKP92aWfLUIWrFBBLmsMzxtX6XR8RD8doZgXfMwD c3y3iy1CzCH1e2VRw4VvPD9KL1rBn6TEa/Q+OR937/ln0Lwya5bUzcqEO8jSU/P5dK1P iO6/xlrkanRjis/gLw64pbRa6JsEIRof6pJBRWRq9q2aBxHI9KV0hFiN1xN88VNf+x4l WwnO1npg79NWm/FXPbF+8LUhf1NT5suiigMMYwq6/bIi4wJYd8SHOjzHW6cv5WkyQfjw Su6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=srnBCtW0M5+UsaYWzvdF3UDB+TVGvu/kPJJEXlyqIx8=; b=WCZOJVEJSWlcbUw9jE8e5IPkQiUv/WjKy1lm4EMzWOGFYFKnzEsoz/wG18wsjMbzza TybjCShRNM2MsA1lCTi3FjqtnZvTru9xUDlvE1rMqMoN7X4Fu1q6d4gViGIyaJSVwrQ5 bgNPBHXTzCB+emOpRyHJripkMeEMl061ZrOcls4UDOJZSP9gLzryBHFoM25lhUxYvK5W 24TQM3RLoDHAwH1yRTvbBwiDf2OM/7JSqp+49TuYLvH1Q0uCAGgiiqWU0yrvU278nE1i ZgZJYzOvP96KcasyZ/61pB63g+tfv0aJp3JKFAqJRkDcqz3pEo58tKGri2MPzBHJg234 3FxA== X-Gm-Message-State: APt69E3u3IZ9BCzfoaSAc0iBX7igGfFoxU8Gyh8mAJcd3Uc0jIOE8VDb S7Qb+DU7M8LP3qzPGtAlOqvcC8PUfPFQY/YIp/ZcVw== X-Google-Smtp-Source: ADUXVKIwMokwvknoLRzLsz6zOAByBFG3NrVaCAs26+OczfDgkFrl4S8Tp8slMf3GCeCZulc6bGt+rhSRFQBAIvxhfv4= X-Received: by 2002:a17:902:2f43:: with SMTP id s61-v6mr24533410plb.274.1530517134534; Mon, 02 Jul 2018 00:38:54 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a17:90a:17af:0:0:0:0 with HTTP; Mon, 2 Jul 2018 00:38:53 -0700 (PDT) In-Reply-To: References: From: Steve Lhomme Date: Mon, 2 Jul 2018 09:38:53 +0200 Message-ID: To: "Timothy B. Terriberry" Cc: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cellar] AV1 mapping Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2018 07:39:06 -0000 2018-06-30 15:54 GMT+02:00 Timothy B. Terriberry : > Steve Lhomme wrote: >> >> You're right. This is a mistake on my part. Instead of MUST it's a >> SHOULD. Because Matroska allows to override the codec internal values >> to display this differently, including changing the aspect ratio or >> forcing physical dimensions. I will change that. > > > Keep in mind that "SHOULD" tends to have a somewhat stronger force in an > IETF specification than in normal English. If you're going to use SHOULD and > not MUST, it is a good idea to explain when someone might reasonably violate > that SHOULD (if you can't because you can't think of a reason, then you want > MUST; if you can't because there are too many reasons, then you want MAY). I changed the DisplayWidth/DisplayHeight mapping to MAY because there is no strong requirement needed. PixelWidth/PixelHeight should be fixed for the whole Segment and are mandatory. But the display ones can be edited even by a user to change the aspect ratio. Using MAY just means that the default values should come from those bits mentioned. >> You mean in my document or the AV1 specs ? My understanding is that >> the AV1 bitstream allow Sequence Header OBUs to be found within the >> stream. And doesn't mention which elements are allowed to change or >> not. Maybe there will be profiles for such restrictions. It won't be > > > See Section 7.5 "Ordering of OBUs": > > "Sequence header OBUs may appear in any order within a coded video sequence. > Within a particular coded video sequence, the contents of > sequence_header_obu must be bit-identical each time the sequence header > appears except for the contents of operating_parameters_info. A new coded > video sequence is required if the sequence header parameters change." Ah I kinda missed that part. > So the question you want to answer is how what is stored in a Matroska file > maps to a "coded video sequence". My opinion (as an individual) is that a > single EBML Document should correspond to a single coded video sequence. That would be a Segment in Matroska. > That means that nothing in the sequence header can change (except for the > operating_parameters_info). Section 7.5 starts with this: "A bitstream conforming to this specification consists of one or more coded video sequences." So there can be bitstreams with various coded video sequences, which parameters may change as the text right after the one you pasted suggests: "A new coded video sequence is required if the sequence header parameters change. Any sequence header in a bitstream which changes the parameters must be contained in a temporal unit with temporal_id equal to zero." My understanding is that an AV1 bitstream can change a lot of parameters for a whole stream. A coded video sequence is strict but so much that people might tend to use more than one even in simple files (although it would probably lose some of the codec efficiency). So either we allow more than one coded video sequence (the direction I took) and so we need to define what parameters must not change for all the Sequence Header OBUs found. Or we only allow one coded video sequence per Segment. It simplifies things a lot, but I fear we'll end up with more than one Segment per "file" which is not what most people use nowadays, even if it's legal to do so. I'd like more input on this on what they expect the best practices for "coded video sequence" will be. It's not obvious to me from just reading the specs. Looking at the term "coded video sequence" (CVS) in other codecs (H.264 and H.265) it seems it's a common term. And for those codec one Segment correspond to one CVS, with the parameters of that CVS stored in the CodecPrivate (SPS + PPS for H.264 for example). So we should probably go in that simple way. > > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar -- Steve Lhomme Matroska association Chairman From nobody Mon Jul 2 00:48:05 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 32A4A130E41 for ; Mon, 2 Jul 2018 00:48:03 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.001 X-Spam-Level: X-Spam-Status: No, score=-2.001 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (4096-bit key) header.d=bunkus.org Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AoyMN0XN8ZLL for ; Mon, 2 Jul 2018 00:48:02 -0700 (PDT) Received: from adara.bunkus.org (adara.bunkus.org [144.76.6.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id AFAD2130DC4 for ; Mon, 2 Jul 2018 00:48:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=bunkus.org; s=mail2017070101; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Message-ID:Date:In-reply-to:Subject:Cc:To:From:References; bh=hNk2TRCHGDzp/GWU2mrV2n4qRqH0U8VAnq4NABRyyZE=; b=ofEhxdGRCYa/P3acNmDAbT0bMWA68+K3DT0PHHxPnUHZqZxZFOcEZ/j0ePnXQPKidbY1eiLWOLxBf4inXO9rAi7w8DtmCh4S9HBdEShhwE+UyaVoBUPuUpZ3N1fTr8ydaCi3tJjl+g/BQMWi0Wh24ZhFJ6BoKb+dS4RGXB+HzLFxB8gRu0zoj5ZmOmtQIgulwTBu8iCja9iW7sKBaIKRHyfX/xdkjS/O1GiREtaZrviOYW9uU8qLFRrP555q4wLjnTbFT6yWE7aC0+eWPPqH5spvbbBa2h/OgUE3J9CGVACsL7FWxNXgQSzP5e8NzEhoAQC1I56/W1XiJt8xZZt7aDsg5I6pNVOMdKq23r6KhSlrc8PnDueWxmqaB2ZYxxxv/xDoXp2aU00ilnhLZBp45/1Npr3oWJOMWGu+J4QQLuPOF7ZtmZorgltE4LAv9UobTHz2k4SSY9GhAHlyMDVFiSHLoIgy4Rbs+5OrnsFZMQ22ucIlWQkaTHt3XjaNLYXUwI8byHZcaN1C7Hbcfj2Nev7aFUet8o/GtRR5h3JovgQt87gQP7fqxN/ywua13Up+mr0X2RmLwN7HBE3HhQR5EzNv3TW1amYUiTYZoVw+s1hXqB+n67B8gcNLOxBC9Ws9FeMPN8vysPMbypHMjd5PhfOk9Ho8bC7xcSvQWhTC7vs=; Received: from liselle.bunkus.org ([2a01:4f8:190:8147::105:1]:36698) by adara.bunkus.org with esmtps (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.82_1-5b7a7c0-XX) (envelope-from ) id 1fZtYu-0002p5-1z for cellar@ietf.org; Mon, 02 Jul 2018 09:47:52 +0200 X-Virus-Scanned: amavisd-new at bunkus.org Received: from sweet-chili.local (unknown [192.168.191.4]) by liselle.bunkus.org (Postfix) with ESMTPS id 4EC3165411B1 for ; Mon, 2 Jul 2018 09:47:46 +0200 (CEST) Received: from sweet-chili (localhost [IPv6:::1]) by sweet-chili.local (Postfix) with ESMTP id 8BB383BFCE48 for ; Mon, 2 Jul 2018 09:47:45 +0200 (CEST) References: User-agent: mu4e 1.0; emacs 26.1 From: Moritz Bunkus To: Codec Encoding for LossLess Archiving and Realtime transmission Cc: In-reply-to: Date: Mon, 02 Jul 2018 09:47:45 +0200 Message-ID: <87efgmdqou.fsf@bunkus.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Archived-At: Subject: Re: [Cellar] AV1 mapping Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2018 07:48:04 -0000 Hey, > Looking at the term "coded video sequence" (CVS) in other codecs (H.264 > and H.265) it seems it's a common term. And for those codec one Segment > correspond to one CVS, with the parameters of that CVS stored in the > CodecPrivate (SPS + PPS for H.264 for example). So we should probably go > in that simple way. I have quite a lot of h.264 samples here, mainly M2TS from DVB, where SPS/PPS change mid-stream, often multiple times. For such files the first occurrences of SPS/PPS make up the AvcC in CodecPrivate, but all key frames are still prefixed with the currently active SPS/PPS =E2=80=94 because that= 's the only way to signal that stuff has actually changed. Requiring a new segment each time those parameters change would make for quite a bad experience, mainly due to two reasons: 1. There's no index listing where each segment starts making seeking inside multi-segment files all but impossible. 2. Software support for multi-segment files is pretty much non-existent. I'm very pessimistic 2 would ever change, even if we mandated it in the AV1 mapping. m. From nobody Mon Jul 2 01:40:57 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 02EF8126F72 for ; Mon, 2 Jul 2018 01:40:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.909 X-Spam-Level: X-Spam-Status: No, score=-1.909 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, T_DKIMWL_WL_MED=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 55xW5ftywude for ; Mon, 2 Jul 2018 01:40:50 -0700 (PDT) Received: from mail-pg0-x235.google.com (mail-pg0-x235.google.com [IPv6:2607:f8b0:400e:c05::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B04EB130E41 for ; Mon, 2 Jul 2018 01:40:50 -0700 (PDT) Received: by mail-pg0-x235.google.com with SMTP id r1-v6so1125024pgp.11 for ; Mon, 02 Jul 2018 01:40:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=6qM2J+iBa7D85Wb868oVp+ptvgf3Uqx2abupqkN5zHU=; b=ydiSMmrsSC2RXcih6TNdVFS0worxapXsPzHHJAs/SSrXi9juHSJGNmH7p0mAmqu3NE AOaSQOT/IJ/9Elo4YPddWZ1P9QkcEZJSn4M1KSWIO2lkK53zwy8HDxQiHdOAGfJxvbrJ aAvZKrowRBTat6bful96MGmXRWSvmhJ7ZVFEihrZu9BVchR0enbxxS8I2gx4zIJbUKaE 5ngtCzw7QYyKDuMFDyssgbhgNhm+P/iukI0ijzri/lKA0kMcL/S8bMz8jKrhjFAePMeR y0hldEBjt+cBbTlLMX/KKLjicSDktI/eQ8oCCCHyX7HtCYwHrAW2zimh/9HBp7SOKRQb O0wA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=6qM2J+iBa7D85Wb868oVp+ptvgf3Uqx2abupqkN5zHU=; b=VAmk/0e+kbq7+yC1pWsoV1jeTJL7VhkN0qyFxUQU17aWnTqUTT6dlsRMxDWkwtGglq TYF+kX44j0HV/4Dfcb9OyWe3LuIsh/N3FiyvGaTRs7/kyMs8P7Dwf995xkjVc6RtHerk jg5VAxabGIDZAIn0m6zGeQY1D+rFT6OMkh1x5CWFrH22b7b2ovJOgM6jGkWfJzfY6148 rsnAY4rGkIanALIkjPzlzI0ER6Sqrn5cfmdIeAsyB37qpn6ca+KcIH3GcnqAuFFqLOS+ fZI4p5hTWWf//C4Q9zb07n8act/WbID1uIhuN/WYBKi8lDXqqcHnc82fPnmtI0y2Dw3M Ut/g== X-Gm-Message-State: APt69E2j+2UTkd6t4zI2jkgQ+9H8iPLKLvffKF2zlQlNvqtHHTWAqzp2 TWolzfzLCZvsIaVxG5wn8yzKf/YVP4/eVHkoiWeTAg== X-Google-Smtp-Source: ADUXVKI+ALvoF3LH+rjHbUrXeiUULLCGg0yFtJ7MlE3woZAhwc0OcXG9JzZzOHuc0U6qduIhsROj1WUS3OuBDmpbGlU= X-Received: by 2002:a65:4b0f:: with SMTP id r15-v6mr21301922pgq.103.1530520849964; Mon, 02 Jul 2018 01:40:49 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a17:90a:17af:0:0:0:0 with HTTP; Mon, 2 Jul 2018 01:40:49 -0700 (PDT) In-Reply-To: References: From: Steve Lhomme Date: Mon, 2 Jul 2018 10:40:49 +0200 Message-ID: To: Andreas Rheinhardt Cc: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cellar] AV1 mapping Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2018 08:40:55 -0000 2018-06-30 23:02 GMT+02:00 Andreas Rheinhardt : > Hello, > > 1. > Steve Lhomme: >>> 3. Given that AV1 seems to only use one Sequence Header OBU at a time >>> couldn't one use the CodecState and CueCodecState elements to designate >>> where the currently active Sequence Header OBU can be found so that one >>> doesn't need to repeat the Sequence Header with every keyframe? >> >> Yes, that would be a good use for when seeking is used. But these >> elements were ruled out as deprecated because they were of no use >> until now. I know VLC doesn't read them for example. > Are you sure that it is deprecated? I couldn't find anything that says No, you're right it's not. Only CueRefCodecState is. > this and the current version of the Codec Specs contains this: "When the > Initialisation is updated within a track then that updated > Initialisation data MUST be written into the CodecState Element of the > first Cluster to require it." (Btw: This requirement is not fulfilled > for the way H.264 is commonly muxed into Matroska.) Your draft also > includes nothing that indicates that CodecState is deprecated. > > Anyway, given that this element is not part of WebM means that it can't > be used in Matroska either if one (sensibly) wants only one codec > mapping for both. Correct. >>> (Btw: There are currently conflicting recommendations regarding the >>> existence of Sequence Header OBU if there is actually only one Sequence >>> Header OBU. On the one hand, they should be omitted, on the other hand >>> every block marked as keyframe should start with a Sequence Header OBU.) >> >> You mean in my document or the AV1 specs ? > In your document. I have already made a PR for this. Timothy B. > Terriberry has misunderstood what I had in mind: At one place the draft > said that every keyframe should have a sequence header OBU and at > another place it said that there shouldn't be in-band sequence header > OBUs if all of them coincide. This is contradictory in case all of the > sequence header OBUs coincide. OK I will try to clarify this. > 2. How does one signal the difference between a real KEY_FRAME and an > INTRA_ONLY_FRAME (that isn't a valid random access point) when using > blocks (that don't have a keyframe flag)? According to your draft, the > relevant `BlockGroup` won't have any `ReferenceBlocks` so that both > frame types are indistinguishable at the container level when using > `BlockGroup`s (if the muxer knows how to handle AV1 it should of course > not reference an INTRA_ONLY_FRAME in the cues, but it would really be > advantageous to be able to infer this without resorting to cues which > are optional anyway and unavailable for e.g. live-streaming). Indeed, there's an issue here. I think we had a similar discussion in the past and we concluded that a ReferenceBlock with a value of 0 can be used. Meaning it's referencing itself. I should add that to the document. > 3. There is something wrong with the invisible flag: > a) According to 7.5. every temporal unit contains at least one frame for > which [show_frame] || [show_existing_frame] equals 1, i.e. for each > Matroska block there is an output frame (even if said output frame is > only from a frame buffer and not from data directly contained in the > Matroska block). This implies that actually no block should have the > invisible flag bit set. And if it is set, it actually means that the > frame that should normally be output due to said frame should not be > displayed (regardless of whether the frame that would normally be output > is one of the already existing ([show_existing_frame] == 1) frames or > not). Otherwise we'd be breaking the semantics of the invisible flag > (and therefore make it impossible to hide individual frames of an > already encoded AV1 track). Indeed, it's either " Each temporal unit must have exactly one shown frame." or "Every layer that has a coded frame in a temporal unit must have exactly one shown frame that is the last frame of that layer in the temporal unit." So each temporal unit has at least one visible frame. And since we map 1 Temporal Unit = 1 Block we don't have to extract the invisible part. > b) If one nevertheless wants to map the invisible flag to properties of > the track, it IMO shouldn't be done the way it is currently proposed: > i) It is possible that [show_frame] equals 1 (meaning the frame should > be immediately be output) and [showable_frame] is 1 (namely if > [frame_type] is not KEY_FRAME). In your draft the invisible bit should > be set to 1 for such a `Block` meaning that the frame should not be > displayed although it should be immediately output. This is obviously wrong. > ii) Apart from that there is the problem that a temporal unit can > contain more than one frame. > iii) A frame might also be displayed later if [showable_frame] equals 1. > So the closest to a invisible frame seems to be a frame with > [show_frame] and [showable_frame] equal to zero. So a temporal unit that > contains only frames with [show_frame] and [showable_frame] equal to 0 > seems to be a sensible choice to merit the invisible flag. (Such a > temporal unit must contain a frame with [show_existing_frame] equal to 1.) > > 4. There is currently no hard requirement that only real random access > points get flagged as keyframes in Matroska/WebM. The current wording is > only a "SHOULD". Is this really intended? I'll double check on that. > 5. Given that changing extradata are not that uncommon in H.264 I don't > deem it a good idea to restrict Matroska to one coded video sequence. > This might end up excluding a lot of content from being muxed into > Matroska (and it makes it much harder to append different tracks). I'm looking at how it's officially handled in MP4. It seems they have different modes, but the "avc1" only has the SPS and PPS in the extradata and not in the stream. Also they have this note: "NOTE 1 It is recommended that when several parameter sets are used and parameter set updating is desired, a separate parameter set elementary stream be used" Meaning a single file should not have changing parameters others that the one expressed in the header. Thanks a lot for your feedback ! > Andreas Rheinhardt > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar -- Steve Lhomme Matroska association Chairman From nobody Mon Jul 2 03:35:10 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9EF6A13107E for ; Mon, 2 Jul 2018 03:35:01 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.62 X-Spam-Level: X-Spam-Status: No, score=-2.62 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oQQsOXQD-VPa for ; Mon, 2 Jul 2018 03:34:59 -0700 (PDT) Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [217.70.183.194]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1382E13102E for ; Mon, 2 Jul 2018 03:34:58 -0700 (PDT) X-Originating-IP: 213.47.41.20 Received: from localhost (213-47-41-20.cable.dynamic.surfer.at [213.47.41.20]) (Authenticated sender: michael@niedermayer.cc) by relay2-d.mail.gandi.net (Postfix) with ESMTPSA id 5D2144000B for ; Mon, 2 Jul 2018 10:34:56 +0000 (UTC) Date: Mon, 2 Jul 2018 12:34:55 +0200 From: Michael Niedermayer To: cellar@ietf.org Message-ID: <20180702103455.GQ4839@michaelspb> References: <87efgmdqou.fsf@bunkus.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="qKOY9RTXA4DvnaVz" Content-Disposition: inline In-Reply-To: <87efgmdqou.fsf@bunkus.org> User-Agent: Mutt/1.5.24 (2015-08-30) Archived-At: Subject: Re: [Cellar] AV1 mapping Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2018 10:35:07 -0000 --qKOY9RTXA4DvnaVz Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi On Mon, Jul 02, 2018 at 09:47:45AM +0200, Moritz Bunkus wrote: > Hey, >=20 > > Looking at the term "coded video sequence" (CVS) in other codecs (H.264 > > and H.265) it seems it's a common term. And for those codec one Segment > > correspond to one CVS, with the parameters of that CVS stored in the > > CodecPrivate (SPS + PPS for H.264 for example). So we should probably go > > in that simple way. >=20 > I have quite a lot of h.264 samples here, mainly M2TS from DVB, where > SPS/PPS change mid-stream, often multiple times. For such files the first > occurrences of SPS/PPS make up the AvcC in CodecPrivate, but all key fram= es > are still prefixed with the currently active SPS/PPS =E2=80=94 because th= at's the > only way to signal that stuff has actually changed. IMHO All SPS/PPS should be in the "global header" (CodecPrivate) which the global header applies to, not just the first. This way an application can know all resolutions, aspect ratios and so on =66rom just the header without the need to scan through all random access points. [...] --=20 Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Those who are too smart to engage in politics are punished by being governed by those who are dumber. -- Plato=20 --qKOY9RTXA4DvnaVz Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAls5/88ACgkQYR7HhwQLD6uMcQCfevN/aifLvxKjW7XqnM9Sw1ua QncAnRpqe2RxJkpFF69qEc/LIEeiqaKT =dZVo -----END PGP SIGNATURE----- --qKOY9RTXA4DvnaVz-- From nobody Mon Jul 2 03:51:47 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C15E2130F13 for ; Mon, 2 Jul 2018 03:51:44 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.909 X-Spam-Level: X-Spam-Status: No, score=-1.909 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, T_DKIMWL_WL_MED=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eQhmBm14nGjU for ; Mon, 2 Jul 2018 03:51:41 -0700 (PDT) Received: from mail-pl0-x22d.google.com (mail-pl0-x22d.google.com [IPv6:2607:f8b0:400e:c01::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B6412130E06 for ; Mon, 2 Jul 2018 03:51:41 -0700 (PDT) Received: by mail-pl0-x22d.google.com with SMTP id m16-v6so7769776pls.11 for ; Mon, 02 Jul 2018 03:51:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=KTT7aFV64X90IztfWJwPHAgPa5o+u0cPsbTUEvlB0DU=; b=wnuy3JrQ+HKrCNd2Uk33qPpTUGEJxjRPHIluf5N+EdUqmx/KPSZLlcFXn96cAcr0iy E5dE8a2vq73NUlZFHwJs8neWzNjgAm4spL33LTPxSWJhrqizSOgA9d78p0WL9qS4IzLF ZoalRI9kH3ENXiKpucF9qZiCZkFDq9B3j2162V5e5VojXwngHr6pUCcpqbzbYj1X1+hI W7ZdqPCEtvW6XiOB1JVqgOdQkkMLfTuKB/1ZZFiIjNlal0WSjvPuSikZJPBQWHfl8vJ2 /OM9yvmTDu0Lh8inCi2G8C8G1z2k9PO8DEs6CkZnvQl5SK16ARqTyydP0LMjtFPnnJOD x6SQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=KTT7aFV64X90IztfWJwPHAgPa5o+u0cPsbTUEvlB0DU=; b=rpFaL90XLkpIAxpRQCyXnLvFjdwEG6NA8TVSzzwbh/AdTApwk3bGM+b+UyllTEyPSd av5m78ZTcEEtYUvvrtAVSQTHSqF8XVOQvNrkt7NfMANDfKQsKP8+WcS1AIQc7Mk45Mzz Ez9zBO/EGMYKBVfK4nF/tht7gaafZuUycpwJPhH8l8frWiU4f2fqfz5OgoU6/DGnrPQv yYDWY/S8iupTUmG3+9oLWyhv9d0Jl4FBg09O0AXl+yv8LcU6OfGkQxt7upF+D9TCmZ7V kc9IBGwpnTATBmJUE2mTVlZl3O/LThTRkiuQKNuvDblSTmuDn0Ei1hjBexdUcuDw5wLr ia3Q== X-Gm-Message-State: APt69E276ULa+i7e8zx7c2cv2iltG5AplAn5tnJV9bwHstm5g5qbyGUK mVkdRRaEsRyrxGNum3XF67fUlA7a3i4H/n7JIvTdCQ== X-Google-Smtp-Source: ADUXVKLSV1m9Ms9SnDrkn53HqV2uVJj5VQY+fz28nR3WY+PZRNzqrBK9YKJ+Pol0XOZm2i9pt/jtxlUVlk7Zt6AaD24= X-Received: by 2002:a17:902:8c95:: with SMTP id t21-v6mr25665186plo.306.1530528701040; Mon, 02 Jul 2018 03:51:41 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a17:90a:17af:0:0:0:0 with HTTP; Mon, 2 Jul 2018 03:51:40 -0700 (PDT) In-Reply-To: <20180702103455.GQ4839@michaelspb> References: <87efgmdqou.fsf@bunkus.org> <20180702103455.GQ4839@michaelspb> From: Steve Lhomme Date: Mon, 2 Jul 2018 12:51:40 +0200 Message-ID: To: Michael Niedermayer Cc: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Archived-At: Subject: Re: [Cellar] AV1 mapping Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2018 10:51:45 -0000 2018-07-02 12:34 GMT+02:00 Michael Niedermayer : > Hi > > On Mon, Jul 02, 2018 at 09:47:45AM +0200, Moritz Bunkus wrote: >> Hey, >> >> > Looking at the term "coded video sequence" (CVS) in other codecs (H.26= 4 >> > and H.265) it seems it's a common term. And for those codec one Segmen= t >> > correspond to one CVS, with the parameters of that CVS stored in the >> > CodecPrivate (SPS + PPS for H.264 for example). So we should probably = go >> > in that simple way. >> >> I have quite a lot of h.264 samples here, mainly M2TS from DVB, where >> SPS/PPS change mid-stream, often multiple times. For such files the firs= t >> occurrences of SPS/PPS make up the AvcC in CodecPrivate, but all key fra= mes >> are still prefixed with the currently active SPS/PPS =E2=80=94 because t= hat's the >> only way to signal that stuff has actually changed. > > IMHO All SPS/PPS should be in the "global header" (CodecPrivate) which th= e > global header applies to, not just the first. > This way an application can know all resolutions, aspect ratios and so on > from just the header without the need to scan through all random access > points. My understanding is that it's how it's done in MP4: "Each AVC sample entry, which contains the AVC video stream decoder specific information, includes a group of SPSs and PPSs. This group of parameter sets functions much like a codebook. Each parameter set has an identifier, and each slice references the parameter set it was coded against using the parameter set's identifier." I have to check but I don't think Sequence Headers OBUs have identifiers like that in AV1. By the way, this is one mapping for V_AV1. There could be other AV1 mapping in Matroska where there is no CodecPrivate and there could be multiple coded video sequences in it. But it would require the use of CodecState and CueCodecState which is currently not an option for WebM. So for now I will concentrate on a single coded video sequence. And if needed in the future we can do a different one. > [...] > > -- > Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB > > Those who are too smart to engage in politics are punished by being > governed by those who are dumber. -- Plato > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > --=20 Steve Lhomme Matroska association Chairman From nobody Mon Jul 2 04:14:12 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8B77A130EE3 for ; Mon, 2 Jul 2018 04:14:10 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xbzGQAdLQB8Q for ; Mon, 2 Jul 2018 04:14:07 -0700 (PDT) Received: from 9.mo178.mail-out.ovh.net (9.mo178.mail-out.ovh.net [46.105.75.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C9A0D1292AD for ; Mon, 2 Jul 2018 04:14:07 -0700 (PDT) Received: from player760.ha.ovh.net (unknown [10.109.108.51]) by mo178.mail-out.ovh.net (Postfix) with ESMTP id 299491A5AA for ; Mon, 2 Jul 2018 13:14:05 +0200 (CEST) Received: from [192.168.2.120] (p3EE2DF2E.dip0.t-ipconnect.de [62.226.223.46]) (Authenticated sender: jerome@mediaarea.net) by player760.ha.ovh.net (Postfix) with ESMTPSA id E76FE20088 for ; Mon, 2 Jul 2018 13:14:03 +0200 (CEST) To: cellar@ietf.org References: <87efgmdqou.fsf@bunkus.org> <20180702103455.GQ4839@michaelspb> From: Jerome Martinez Message-ID: <36a0dc7e-8579-d99c-4beb-e8f4ce019eff@mediaarea.net> Date: Mon, 2 Jul 2018 13:14:02 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Ovh-Tracer-Id: 647673925405184145 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 0 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedtiedrvdejgdefiecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemuceftddtnecu Archived-At: Subject: Re: [Cellar] AV1 mapping Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2018 11:14:11 -0000 On 02/07/2018 12:51, Steve Lhomme wrote: > 2018-07-02 12:34 GMT+02:00 Michael Niedermayer : >> Hi >> >> On Mon, Jul 02, 2018 at 09:47:45AM +0200, Moritz Bunkus wrote: >>> Hey, >>> >>>> Looking at the term "coded video sequence" (CVS) in other codecs (H.264 >>>> and H.265) it seems it's a common term. And for those codec one Segment >>>> correspond to one CVS, with the parameters of that CVS stored in the >>>> CodecPrivate (SPS + PPS for H.264 for example). So we should probably go >>>> in that simple way. >>> I have quite a lot of h.264 samples here, mainly M2TS from DVB, where >>> SPS/PPS change mid-stream, often multiple times. For such files the first >>> occurrences of SPS/PPS make up the AvcC in CodecPrivate, but all key frames >>> are still prefixed with the currently active SPS/PPS — because that's the >>> only way to signal that stuff has actually changed. >> IMHO All SPS/PPS should be in the "global header" (CodecPrivate) which the >> global header applies to, not just the first. Not possible with e.g. real time streaming (you don't know in advance the next SPS/PPS) >> This way an application can know all resolutions, aspect ratios and so on >> from just the header without the need to scan through all random access >> points. > My understanding is that it's how it's done in MP4: > > "Each AVC sample entry, which contains the AVC video stream decoder > specific information, includes a group of SPSs and PPSs. This group of > parameter sets functions much like a codebook. Each parameter set has > an identifier, and each slice references the parameter set it was > coded against using the parameter set's identifier." > > I have to check but I don't think Sequence Headers OBUs have > identifiers like that in AV1. Not in AV1 (I just updated my own code for AV1  1.0.0 spec, still not there). But it actually does not answer the issue because in the example there is usually still only the "set id" of 0 (the new SPS replaces the old one, not a new number). Actually IIRC in HEVC (same for AVC) there are explicitly 2 different codec identifiers: - "hvc1" is for *all* SPS/PPS in the global header (CodecPrivate), and there is no SPS/PPS in the stream itself - "hev1" is for *no* SPS/PPS in the global header (CodecPrivate), and key frames are prefixed with the currently active SPS/PPS (I personally don't see the need of 2 different identifiers, as it quick to see if SPS/PPS is in the file, but there is surely a reason) The issue is not only for AV1, but for all formats permitting to have the init in the stream itself (AVC, HEVC, AV1...). IMO we should not restrict to only one possibility in Matroska spec, as they are for different needs (non real time vs real time), whatever is the format (AVC, HEVC, AV1...). > By the way, this is one mapping for V_AV1. There could be other AV1 > mapping in Matroska where there is no CodecPrivate and there could be > multiple coded video sequences in it. But it would require the use of > CodecState and CueCodecState which is currently not an option for > WebM. So for now I will concentrate on a single coded video sequence. > And if needed in the future we can do a different one. If WebM decides that real time AV1 is for them, they may decide to discard CodecPrivate usage and put sequence header in the "Block" (as they currently do with up to date AOM build) So restriction will not be there in practice. Jérôme From nobody Mon Jul 2 04:27:52 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 742A4130F36 for ; Mon, 2 Jul 2018 04:27:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.909 X-Spam-Level: X-Spam-Status: No, score=-1.909 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, T_DKIMWL_WL_MED=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SHEf5okTODP8 for ; Mon, 2 Jul 2018 04:27:48 -0700 (PDT) Received: from mail-pg0-x234.google.com (mail-pg0-x234.google.com [IPv6:2607:f8b0:400e:c05::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DB983130F2F for ; Mon, 2 Jul 2018 04:27:47 -0700 (PDT) Received: by mail-pg0-x234.google.com with SMTP id y5-v6so743914pgv.1 for ; Mon, 02 Jul 2018 04:27:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=L4OTMSSYc4wjGZBMaxqb+l1z0V9e/2zxrrMxcMP89/E=; b=ZZ0K/stevxa7jJqCVZpx+wGdI36JkC9KRfTJWIPL2CLXHEAOjFxpBbYP6mL2F4tg3A B/6utqbJdGkbPjNTT8h6pb/wXTVCbdWW9V/w9ZM0LlYjOfkk9Xth+qqlqqQoa0msj88W b4hYLUfxkCxc4e2D2peRuysUTAdMC1cvL0T+XVf1xVljVm+A+X39yhlQdUlU6Iqrx2FA F/gsyHPNy09DxuWmNnIIAJK9ImsaB0nMvuPXC/YaksjA+Zr/jSOv9bZdDE5kMvkSAeyP 0XtG+9ngSalVqbvq243kkYbUrjvVtqmkqKPCW+t2XFU8q8fNuYU4G0/6Fv8PmWQrEI/a gqXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=L4OTMSSYc4wjGZBMaxqb+l1z0V9e/2zxrrMxcMP89/E=; b=JYh+Xaxxm2Md9y30fo3nNCkuN9bulyBwxoFl5CE2Fi08FaPMYzzl7hbcjUTP6YkMLR r+CsXVjTi6pS9oUPMcGMFrKaE6o+XNNK6nP8dIqHFCXJMaMOtNAPz67G5fmuAFxR6yYu oSWwye4v7zu5OAC/nyaL6RRTDH3aYuNE7u9047s8G5da8HcoUe8/06ZZbIyEvJsw+d7M Er/ULet5sg+EjYOzruSyLAFELQ+hfF7F17hrYygOUHJNSB8u31aOclMDCcwjCHYNjFQn 9Xm78eDuncjebuoBSQ48HTdP2Zfl2hZDev+6tLgnYvd504bkCqi64vympwRBMRcx21Km HS5g== X-Gm-Message-State: APt69E3KR9gsH/bt+FAlMjAmCjxUtYj4hWEGt4xirAdHYZ7DJ5/Xrnnp KeiVsyu5EtmpJXGViGcYb5/r5LO3GIermJBlXsy3mlt3 X-Google-Smtp-Source: AAOMgpe1thVCIEumTWq6lH31T+c6vCH6q7+kdXN8P8atXkzsa8G8lsQI/emsPqOO/LCZdLS0MaxlQGY4x1jmph0KpXE= X-Received: by 2002:a62:b20c:: with SMTP id x12-v6mr16038070pfe.64.1530530867294; Mon, 02 Jul 2018 04:27:47 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a17:90a:17af:0:0:0:0 with HTTP; Mon, 2 Jul 2018 04:27:46 -0700 (PDT) In-Reply-To: <36a0dc7e-8579-d99c-4beb-e8f4ce019eff@mediaarea.net> References: <87efgmdqou.fsf@bunkus.org> <20180702103455.GQ4839@michaelspb> <36a0dc7e-8579-d99c-4beb-e8f4ce019eff@mediaarea.net> From: Steve Lhomme Date: Mon, 2 Jul 2018 13:27:46 +0200 Message-ID: To: Jerome Martinez Cc: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Archived-At: Subject: Re: [Cellar] AV1 mapping Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2018 11:27:51 -0000 2018-07-02 13:14 GMT+02:00 Jerome Martinez : > On 02/07/2018 12:51, Steve Lhomme wrote: >> >> 2018-07-02 12:34 GMT+02:00 Michael Niedermayer : >>> >>> Hi >>> >>> On Mon, Jul 02, 2018 at 09:47:45AM +0200, Moritz Bunkus wrote: >>>> >>>> Hey, >>>> >>>>> Looking at the term "coded video sequence" (CVS) in other codecs (H.2= 64 >>>>> and H.265) it seems it's a common term. And for those codec one Segme= nt >>>>> correspond to one CVS, with the parameters of that CVS stored in the >>>>> CodecPrivate (SPS + PPS for H.264 for example). So we should probably >>>>> go >>>>> in that simple way. >>>> >>>> I have quite a lot of h.264 samples here, mainly M2TS from DVB, where >>>> SPS/PPS change mid-stream, often multiple times. For such files the >>>> first >>>> occurrences of SPS/PPS make up the AvcC in CodecPrivate, but all key >>>> frames >>>> are still prefixed with the currently active SPS/PPS =E2=80=94 because= that's >>>> the >>>> only way to signal that stuff has actually changed. >>> >>> IMHO All SPS/PPS should be in the "global header" (CodecPrivate) which >>> the >>> global header applies to, not just the first. > > > Not possible with e.g. real time streaming (you don't know in advance the > next SPS/PPS) > >>> This way an application can know all resolutions, aspect ratios and so = on >>> from just the header without the need to scan through all random access >>> points. >> >> My understanding is that it's how it's done in MP4: >> >> "Each AVC sample entry, which contains the AVC video stream decoder >> specific information, includes a group of SPSs and PPSs. This group of >> parameter sets functions much like a codebook. Each parameter set has >> an identifier, and each slice references the parameter set it was >> coded against using the parameter set's identifier." >> >> I have to check but I don't think Sequence Headers OBUs have >> identifiers like that in AV1. > > > Not in AV1 (I just updated my own code for AV1 1.0.0 spec, still not > there). > But it actually does not answer the issue because in the example there is > usually still only the "set id" of 0 (the new SPS replaces the old one, n= ot > a new number). > > Actually IIRC in HEVC (same for AVC) there are explicitly 2 different cod= ec > identifiers: > - "hvc1" is for *all* SPS/PPS in the global header (CodecPrivate), and th= ere > is no SPS/PPS in the stream itself > - "hev1" is for *no* SPS/PPS in the global header (CodecPrivate), and key > frames are prefixed with the currently active SPS/PPS > (I personally don't see the need of 2 different identifiers, as it quick = to > see if SPS/PPS is in the file, but there is surely a reason) > > The issue is not only for AV1, but for all formats permitting to have the > init in the stream itself (AVC, HEVC, AV1...). > > IMO we should not restrict to only one possibility in Matroska spec, as t= hey > are for different needs (non real time vs real time), whatever is the for= mat > (AVC, HEVC, AV1...). > > > >> By the way, this is one mapping for V_AV1. There could be other AV1 >> mapping in Matroska where there is no CodecPrivate and there could be >> multiple coded video sequences in it. But it would require the use of >> CodecState and CueCodecState which is currently not an option for >> WebM. So for now I will concentrate on a single coded video sequence. >> And if needed in the future we can do a different one. > > > If WebM decides that real time AV1 is for them, they may decide to discar= d > CodecPrivate usage and put sequence header in the "Block" (as they curren= tly > do with up to date AOM build) > So restriction will not be there in practice. Seeking in such a file would not be possible. I'm not sure it's desirable even when doing adaptive streaming with short sequences. We will need CodecState and CueCodecState for that. We could define such a case (with a different codec ID) for Matroska and WebM could pick that from there. Also while working on the case where the Sequence Header OBU doesn't change, I noticed an issue. The Sequence Header OBU can still change the [operating_parameters_info] during the lifetime of the stream (paragraph 7.5 pointed out by Tim). That means even applying the stricter restrictions of the Codec Video Sequence in AV1 means we'd still need CodecState and CueCodecState for seeking to work. This change is only possible in a specific decoder model (Annex E) when [decoder_model_info_present_flag] is 1. When it's 0 then the [operating_parameters_info] doesn't apply. So we should probably restrict the single Sequence Header OBU to streams where [decoder_model_info_present_flag] is 0. > J=C3=A9r=C3=B4me > > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar --=20 Steve Lhomme Matroska association Chairman From nobody Mon Jul 2 04:35:16 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 71564130E0A for ; Mon, 2 Jul 2018 04:35:15 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HBjztX7m8KNE for ; Mon, 2 Jul 2018 04:35:13 -0700 (PDT) Received: from mail-wm0-x234.google.com (mail-wm0-x234.google.com [IPv6:2a00:1450:400c:c09::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0FA20130EBB for ; Mon, 2 Jul 2018 04:35:11 -0700 (PDT) Received: by mail-wm0-x234.google.com with SMTP id w16-v6so8007855wmc.2 for ; Mon, 02 Jul 2018 04:35:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :content-transfer-encoding; bh=gnyChOSJCxJhT3rs5wTXwJloenfRrdunTxJAk+6CudI=; b=qys7DvImVckbxGOiWnrWh3V5+wxvLGdU6Vun//m1yyX2C+muuTv2/sH7i3pxOYgne2 dcERTPMThJSxy+Djpj9fQBi2+UFjheogktCGIBA8/8VECun1NVlrbfLLIvf4Fd2cfV/9 VJxBzaEUwEgSMl54r4+pmMsbiLFvLcR99DJuTpbCDXhD4+SD+nbaxXDwJn/FiEkEFoB8 MBwZcLHSuiMPBzkA/9hiXB9qTH9s7Zct4NhyMh65g3VNElz+Qrwk+lr0Ttb1DNs7uC8/ XN6bX9nixCDn6R804Tb4um4emI+o2BStQD3EdG/barUa3Fp06UD2CKE4Hrdq+a+fQaby Mafw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:content-transfer-encoding; bh=gnyChOSJCxJhT3rs5wTXwJloenfRrdunTxJAk+6CudI=; b=Qdgclau/JtLWVBuNZylaN8zM7kT9Frzpx85YgGCSCq/Wb4wdKSCbqy6cEsHK5cIi5N 6rLfO3Z9Dfv9QZjmdxWQsOQ1Cd36yvl3e8IktfxsKsJxBt3X5IGkc508uztwZQfuaWgl gxkPGQxjXdCMk02qhnDno8Ho8WVyJxI49RcLD8FtlwpSeBIhUAPAG2tn1tf0CfLt+V+W N30CXf3oHywU+ns6xqbierKUpA+GT1JhQT66hpWfBB9uVYDOeibkFH+sZfcKE2o/gSWu NkphKB/yFOZG573Pi5ZGgmQhry2OhSV6J6HHy3pBcCLVAJH1Rir62ZjsljzToMbvKjSj QTfQ== X-Gm-Message-State: APt69E2WoeEfm6E1UfT0MAL+op19+3icKeBkg6y3b42B1X+5YRleJBje 5IAtnKSOJZDuWbeEu0aGxaO8FG5lkgXjsO2pTaPzdvf3 X-Google-Smtp-Source: AAOMgpfZQ1k/kyKZwh0kl54aC7yV/DZx6v1RBDPQUM4NoMz8EsNhcSvcoGAqGKZEjPQZyDDjfn53BI6Xl+IB5mbkCWw= X-Received: by 2002:a1c:8d15:: with SMTP id p21-v6mr7357699wmd.145.1530531309319; Mon, 02 Jul 2018 04:35:09 -0700 (PDT) MIME-Version: 1.0 References: <87efgmdqou.fsf@bunkus.org> <20180702103455.GQ4839@michaelspb> <36a0dc7e-8579-d99c-4beb-e8f4ce019eff@mediaarea.net> In-Reply-To: From: Hendrik Leppkes Date: Mon, 2 Jul 2018 13:35:00 +0200 Message-ID: To: CELLAR list Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Archived-At: Subject: Re: [Cellar] AV1 mapping Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2018 11:35:15 -0000 On Mon, Jul 2, 2018 at 1:27 PM Steve Lhomme wrote: > > 2018-07-02 13:14 GMT+02:00 Jerome Martinez : > > On 02/07/2018 12:51, Steve Lhomme wrote: > >> > >> 2018-07-02 12:34 GMT+02:00 Michael Niedermayer : > >>> > >>> Hi > >>> > >>> On Mon, Jul 02, 2018 at 09:47:45AM +0200, Moritz Bunkus wrote: > >>>> > >>>> Hey, > >>>> > >>>>> Looking at the term "coded video sequence" (CVS) in other codecs (H= .264 > >>>>> and H.265) it seems it's a common term. And for those codec one Seg= ment > >>>>> correspond to one CVS, with the parameters of that CVS stored in th= e > >>>>> CodecPrivate (SPS + PPS for H.264 for example). So we should probab= ly > >>>>> go > >>>>> in that simple way. > >>>> > >>>> I have quite a lot of h.264 samples here, mainly M2TS from DVB, wher= e > >>>> SPS/PPS change mid-stream, often multiple times. For such files the > >>>> first > >>>> occurrences of SPS/PPS make up the AvcC in CodecPrivate, but all key > >>>> frames > >>>> are still prefixed with the currently active SPS/PPS =E2=80=94 becau= se that's > >>>> the > >>>> only way to signal that stuff has actually changed. > >>> > >>> IMHO All SPS/PPS should be in the "global header" (CodecPrivate) whic= h > >>> the > >>> global header applies to, not just the first. > > > > > > Not possible with e.g. real time streaming (you don't know in advance t= he > > next SPS/PPS) > > > >>> This way an application can know all resolutions, aspect ratios and s= o on > >>> from just the header without the need to scan through all random acce= ss > >>> points. > >> > >> My understanding is that it's how it's done in MP4: > >> > >> "Each AVC sample entry, which contains the AVC video stream decoder > >> specific information, includes a group of SPSs and PPSs. This group of > >> parameter sets functions much like a codebook. Each parameter set has > >> an identifier, and each slice references the parameter set it was > >> coded against using the parameter set's identifier." > >> > >> I have to check but I don't think Sequence Headers OBUs have > >> identifiers like that in AV1. > > > > > > Not in AV1 (I just updated my own code for AV1 1.0.0 spec, still not > > there). > > But it actually does not answer the issue because in the example there = is > > usually still only the "set id" of 0 (the new SPS replaces the old one,= not > > a new number). > > > > Actually IIRC in HEVC (same for AVC) there are explicitly 2 different c= odec > > identifiers: > > - "hvc1" is for *all* SPS/PPS in the global header (CodecPrivate), and = there > > is no SPS/PPS in the stream itself > > - "hev1" is for *no* SPS/PPS in the global header (CodecPrivate), and k= ey > > frames are prefixed with the currently active SPS/PPS > > (I personally don't see the need of 2 different identifiers, as it quic= k to > > see if SPS/PPS is in the file, but there is surely a reason) > > > > The issue is not only for AV1, but for all formats permitting to have t= he > > init in the stream itself (AVC, HEVC, AV1...). > > > > IMO we should not restrict to only one possibility in Matroska spec, as= they > > are for different needs (non real time vs real time), whatever is the f= ormat > > (AVC, HEVC, AV1...). > > > > > > > >> By the way, this is one mapping for V_AV1. There could be other AV1 > >> mapping in Matroska where there is no CodecPrivate and there could be > >> multiple coded video sequences in it. But it would require the use of > >> CodecState and CueCodecState which is currently not an option for > >> WebM. So for now I will concentrate on a single coded video sequence. > >> And if needed in the future we can do a different one. > > > > > > If WebM decides that real time AV1 is for them, they may decide to disc= ard > > CodecPrivate usage and put sequence header in the "Block" (as they curr= ently > > do with up to date AOM build) > > So restriction will not be there in practice. > > Seeking in such a file would not be possible. I'm not sure it's > desirable even when doing adaptive streaming with short sequences. We > will need CodecState and CueCodecState for that. We could define such > a case (with a different codec ID) for Matroska and WebM could pick > that from there. > > Also while working on the case where the Sequence Header OBU doesn't > change, I noticed an issue. The Sequence Header OBU can still change > the [operating_parameters_info] during the lifetime of the stream > (paragraph 7.5 pointed out by Tim). That means even applying the > stricter restrictions of the Codec Video Sequence in AV1 means we'd > still need CodecState and CueCodecState for seeking to work. This > change is only possible in a specific decoder model (Annex E) when > [decoder_model_info_present_flag] is 1. When it's 0 then the > [operating_parameters_info] doesn't apply. So we should probably > restrict the single Sequence Header OBU to streams where > [decoder_model_info_present_flag] is 0. > In this particular case, doesn't the stream already include in-band Sequence Header OBUs, and judging from the requirement of some parameters being allowed to change, I would wager that decoders would be setup to handle them as well. So all this comes down to is setting up Cues properly to point to clean seek points with a Sequence Header + Keyframe, much like a H264 stream with minor SPS/PPS changes would be muxed right now. Now if an encoder would produce a stream that uses multiple different Sequence Headers but doesn't regularly repeat the sequence header (ie. with keyframes), then such a stream would be unseekable in whatever medium, and that would be the encoders fault. (PS: re-posted to the list, can someone fix this list to set reply-to heade= rs?) - Hendrik From nobody Mon Jul 2 04:54:42 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EA60D130DF7 for ; Mon, 2 Jul 2018 04:54:40 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.909 X-Spam-Level: X-Spam-Status: No, score=-1.909 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, T_DKIMWL_WL_MED=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XQdolLP5RPSP for ; Mon, 2 Jul 2018 04:54:38 -0700 (PDT) Received: from mail-pf0-x22b.google.com (mail-pf0-x22b.google.com [IPv6:2607:f8b0:400e:c00::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 809D6129385 for ; Mon, 2 Jul 2018 04:54:38 -0700 (PDT) Received: by mail-pf0-x22b.google.com with SMTP id s21-v6so7414419pfm.6 for ; Mon, 02 Jul 2018 04:54:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=blgU5yyjE17MtBySND2djZm4KleoxKS2Kl/sOog2PKY=; b=04FeWnPzGzVlyRDUwCceGc9KajmnMk2xVsjRvYcN2UjMWy+3Qz7dcr7MHS2mu6dhcu 0iKhM/lxhseH6FWjrX9w2WmZX5KZyBYGt+S/sCbGfoYDgYH4n6kYAKCJb8DsxYX1mAhb ktic7BalQJMi2BqJTgP5lSgyHZ8LDkCZ5iDoHji0cDYGZ2TVzEvfqSCPN8C6s7fcovkB vi+C40yRuuXhy/vzphUXnFR8S5oZzmBpFtABm8zm4G78j0+qSMS4HL9Ox5tisLIqgXls m9K7vjqHRdbxbGqVuOonJuqfIH3MO1wd7RFoaNkm5iTslpZy6iI/zYJXJYRWPl5xGmAD E/HA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=blgU5yyjE17MtBySND2djZm4KleoxKS2Kl/sOog2PKY=; b=UqyY081VR4xSofSLoxXkN7c04Fih9H6oFgBiHTlJPe1VplIUy70yUyAwetvXILSe/+ Q2cit0tSMNXU1vrKeymwtLvFgebSQKXJYpQuVPL2m5Z1ry0MTxvK8OCDE7Lm8A2Z8mZY UePSS+KqOROy2uGh9PnozPIDQ5J9FYeWlVm2HjvU0Skhh9geFCWHvjtCGtktfjRPl3Ee 7M4GC98hMIZ9CIW672eLYELwzzBevGL1UEH8QMCd++vS7im+DWunAVvY0CYmFAr2UpSv xVENZgSbtWqLqpBNNsfyuP4dBM5uEwL/CX7g1mNHGDCj5nfzWB439XcNlMNeu2m8UCA3 gEYQ== X-Gm-Message-State: APt69E1IkV+17GidCgCvVKN2En27wOIw+h8ZFQwx62QJ6wToZdhsNB0J ETe2Xwv4CNFG6mFRs3UzekTeLntyBCJIVYAK7nzbfw== X-Google-Smtp-Source: AAOMgpf8DxkxYMdzap9u1LvaulbLXG4JiYLL8HaaYfEARwym4m+6O+fRiHSDeZjN9XsVNQ8eqX6HfgV49mChsv8f4oo= X-Received: by 2002:a62:b20c:: with SMTP id x12-v6mr16128177pfe.64.1530532477944; Mon, 02 Jul 2018 04:54:37 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a17:90a:17af:0:0:0:0 with HTTP; Mon, 2 Jul 2018 04:54:37 -0700 (PDT) In-Reply-To: References: <87efgmdqou.fsf@bunkus.org> <20180702103455.GQ4839@michaelspb> <36a0dc7e-8579-d99c-4beb-e8f4ce019eff@mediaarea.net> From: Steve Lhomme Date: Mon, 2 Jul 2018 13:54:37 +0200 Message-ID: To: Hendrik Leppkes Cc: CELLAR list Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Archived-At: Subject: Re: [Cellar] AV1 mapping Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2018 11:54:41 -0000 2018-07-02 13:35 GMT+02:00 Hendrik Leppkes : > On Mon, Jul 2, 2018 at 1:27 PM Steve Lhomme wrote: >> >> 2018-07-02 13:14 GMT+02:00 Jerome Martinez : >> > On 02/07/2018 12:51, Steve Lhomme wrote: >> >> >> >> 2018-07-02 12:34 GMT+02:00 Michael Niedermayer : >> >>> >> >>> Hi >> >>> >> >>> On Mon, Jul 02, 2018 at 09:47:45AM +0200, Moritz Bunkus wrote: >> >>>> >> >>>> Hey, >> >>>> >> >>>>> Looking at the term "coded video sequence" (CVS) in other codecs (= H.264 >> >>>>> and H.265) it seems it's a common term. And for those codec one Se= gment >> >>>>> correspond to one CVS, with the parameters of that CVS stored in t= he >> >>>>> CodecPrivate (SPS + PPS for H.264 for example). So we should proba= bly >> >>>>> go >> >>>>> in that simple way. >> >>>> >> >>>> I have quite a lot of h.264 samples here, mainly M2TS from DVB, whe= re >> >>>> SPS/PPS change mid-stream, often multiple times. For such files the >> >>>> first >> >>>> occurrences of SPS/PPS make up the AvcC in CodecPrivate, but all ke= y >> >>>> frames >> >>>> are still prefixed with the currently active SPS/PPS =E2=80=94 beca= use that's >> >>>> the >> >>>> only way to signal that stuff has actually changed. >> >>> >> >>> IMHO All SPS/PPS should be in the "global header" (CodecPrivate) whi= ch >> >>> the >> >>> global header applies to, not just the first. >> > >> > >> > Not possible with e.g. real time streaming (you don't know in advance = the >> > next SPS/PPS) >> > >> >>> This way an application can know all resolutions, aspect ratios and = so on >> >>> from just the header without the need to scan through all random acc= ess >> >>> points. >> >> >> >> My understanding is that it's how it's done in MP4: >> >> >> >> "Each AVC sample entry, which contains the AVC video stream decoder >> >> specific information, includes a group of SPSs and PPSs. This group o= f >> >> parameter sets functions much like a codebook. Each parameter set has >> >> an identifier, and each slice references the parameter set it was >> >> coded against using the parameter set's identifier." >> >> >> >> I have to check but I don't think Sequence Headers OBUs have >> >> identifiers like that in AV1. >> > >> > >> > Not in AV1 (I just updated my own code for AV1 1.0.0 spec, still not >> > there). >> > But it actually does not answer the issue because in the example there= is >> > usually still only the "set id" of 0 (the new SPS replaces the old one= , not >> > a new number). >> > >> > Actually IIRC in HEVC (same for AVC) there are explicitly 2 different = codec >> > identifiers: >> > - "hvc1" is for *all* SPS/PPS in the global header (CodecPrivate), and= there >> > is no SPS/PPS in the stream itself >> > - "hev1" is for *no* SPS/PPS in the global header (CodecPrivate), and = key >> > frames are prefixed with the currently active SPS/PPS >> > (I personally don't see the need of 2 different identifiers, as it qui= ck to >> > see if SPS/PPS is in the file, but there is surely a reason) >> > >> > The issue is not only for AV1, but for all formats permitting to have = the >> > init in the stream itself (AVC, HEVC, AV1...). >> > >> > IMO we should not restrict to only one possibility in Matroska spec, a= s they >> > are for different needs (non real time vs real time), whatever is the = format >> > (AVC, HEVC, AV1...). >> > >> > >> > >> >> By the way, this is one mapping for V_AV1. There could be other AV1 >> >> mapping in Matroska where there is no CodecPrivate and there could be >> >> multiple coded video sequences in it. But it would require the use of >> >> CodecState and CueCodecState which is currently not an option for >> >> WebM. So for now I will concentrate on a single coded video sequence. >> >> And if needed in the future we can do a different one. >> > >> > >> > If WebM decides that real time AV1 is for them, they may decide to dis= card >> > CodecPrivate usage and put sequence header in the "Block" (as they cur= rently >> > do with up to date AOM build) >> > So restriction will not be there in practice. >> >> Seeking in such a file would not be possible. I'm not sure it's >> desirable even when doing adaptive streaming with short sequences. We >> will need CodecState and CueCodecState for that. We could define such >> a case (with a different codec ID) for Matroska and WebM could pick >> that from there. >> >> Also while working on the case where the Sequence Header OBU doesn't >> change, I noticed an issue. The Sequence Header OBU can still change >> the [operating_parameters_info] during the lifetime of the stream >> (paragraph 7.5 pointed out by Tim). That means even applying the >> stricter restrictions of the Codec Video Sequence in AV1 means we'd >> still need CodecState and CueCodecState for seeking to work. This >> change is only possible in a specific decoder model (Annex E) when >> [decoder_model_info_present_flag] is 1. When it's 0 then the >> [operating_parameters_info] doesn't apply. So we should probably >> restrict the single Sequence Header OBU to streams where >> [decoder_model_info_present_flag] is 0. >> > > In this particular case, doesn't the stream already include in-band > Sequence Header OBUs, and judging from the requirement of some > parameters being allowed to change, I would wager that decoders would > be setup to handle them as well. > So all this comes down to is setting up Cues properly to point to > clean seek points with a Sequence Header + Keyframe, much like a H264 > stream with minor SPS/PPS changes would be muxed right now. Ah yes, good point. In Paragraph 7.5 there is this: "Note:There is not a requirement that every temporal unit with a key frame also contains a sequence header, just that the sequence header has been sent before the first key frame. However, note that temporal units without sequence header OBUs are not considered to be random access points." So we can assume that differing Sequence Headers OBUs will be found in-band. And must be bit-identical to the first one in the CVS except for [operating_parameters_info]. It may also mean that our keyframe flag may need to be refined. And by "temporal unit" I think it means "temporal unit with a key frame" wihout a sequence header OBU are not RAP. In practice I don't think there will not be keyframes where the sequence header OBU is not in that temporal unit. > Now if an encoder would produce a stream that uses multiple different > Sequence Headers but doesn't regularly repeat the sequence header (ie. > with keyframes), then such a stream would be unseekable in whatever > medium, and that would be the encoders fault. > > (PS: re-posted to the list, can someone fix this list to set reply-to hea= ders?) > > - Hendrik > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar --=20 Steve Lhomme Matroska association Chairman From nobody Mon Jul 2 06:46:18 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EF9F0131089 for ; Mon, 2 Jul 2018 06:46:09 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.909 X-Spam-Level: X-Spam-Status: No, score=-1.909 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, T_DKIMWL_WL_MED=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zr1sWoAxOcxS for ; Mon, 2 Jul 2018 06:46:08 -0700 (PDT) Received: from mail-pl0-x22f.google.com (mail-pl0-x22f.google.com [IPv6:2607:f8b0:400e:c01::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id F00EB130F8B for ; Mon, 2 Jul 2018 06:46:07 -0700 (PDT) Received: by mail-pl0-x22f.google.com with SMTP id d10-v6so7989234plo.5 for ; Mon, 02 Jul 2018 06:46:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=tj2iN6AcHBy5/8eg9rjtZ8M605Tcn5viVKohBnz1d7I=; b=IG/oFbuzcy+0hsN/ehW/zkZVuwCR8tpwn61m4qijEL29FMiY1FpCx198MCXbZ02tkX TiFYEa7cPG7cp04iXxndy/GvJp8DhAk0LsRVPRBIib33YtImvWxQezXktC/hVa6BC3It Cpox0Wn+wAl9D2pd+fyxUuqkkuJjFw3Ri6oeXRvUa1o0cHIZXFR507F6rvIF5baGg1Qo P1red54tV06yDvmzpm5PpqUT5nDQ5V9jNQZ3F8pnsA21am4spktF6kUFu8hrQiGlViw0 tO/Xl1TEazHIg7tHp/695fPkxGNOixeqoXjSyns3KP7icpYxDXiPUZP5cIgRiNyKUy+6 eoEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=tj2iN6AcHBy5/8eg9rjtZ8M605Tcn5viVKohBnz1d7I=; b=bUlOLCx2IThget/s48srhuK1gWUjpqr40i0ajsDLriD98UVr+l8yySLlGtsTRUE0ap wCT90QjI5Lj9Kbca3SqXmfpTpi814tLTWI77jwP6G0tIuip2/inSOl3XaokEtDBgpj9E lPPiLaMM6yHrNDK5zbjqsm230bPD8H8TKi+9rIFUwWyvTyNErROi/KGO5ycSqaYaB5q9 A1R4EU99dLb197PTiBhaBSMFzfn1Tag62rHOMLzDELUSZxRRP7D2UQ6n+Ujhv4VMHjyk VV/kJ9F0D9v8kPKcLs8+R5k2v/gBJXDmKtx4k3Q41lzO9dlBQx1VRaLGvHbNTgcY9EtA X10Q== X-Gm-Message-State: APt69E0S2/ojEJ2fzlGqywlpZs907btBuoINMLlCBtg4pqpH446OE7yV 1DIQGwIFGyqh9SVyurbbtURyN2s1ycQelzX4KZT6THQQ X-Google-Smtp-Source: AAOMgpchdLEGO90P5hTocYUXr+AjmHmOWWXbr8C7yqY4J+6Iar26TYNGDelHBwQwLYDK0dJYuOCh9bfFef/4K+8bVNE= X-Received: by 2002:a17:902:82c3:: with SMTP id u3-v6mr19103931plz.83.1530539167201; Mon, 02 Jul 2018 06:46:07 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a17:90a:17af:0:0:0:0 with HTTP; Mon, 2 Jul 2018 06:46:06 -0700 (PDT) In-Reply-To: References: <87efgmdqou.fsf@bunkus.org> <20180702103455.GQ4839@michaelspb> <36a0dc7e-8579-d99c-4beb-e8f4ce019eff@mediaarea.net> From: Steve Lhomme Date: Mon, 2 Jul 2018 15:46:06 +0200 Message-ID: To: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cellar] AV1 mapping Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2018 13:46:17 -0000 I updated the document with the following changes: - the Sequence Header OBU in CodecPrivate has the same restrictions as a coded video sequence in AV1 - if all Sequence Header OBU in Blocks are going to be the same (decoder_model_info_present_flag =0) they can be omitted from the Blocks. - if decoder_model_info_present_flag =1 in the CodecPrivate then all keyframes MUST have a Sequence Header OBU - the low overhead syntax is a MUST - set the keyframe flag/ReferenceBlock for valid random access points (not intra frame). - remove the Inisible bit mention (it's actually supposed to be used for prerolling when editing files). - DisplayWidth/Height are just hints of how to get the value - typos (from mkver) https://github.com/Matroska-Org/matroska-specification/blob/av1-mappin/codec/av1.md -- Steve Lhomme Matroska association Chairman From nobody Mon Jul 2 12:07:21 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DAC671312C9 for ; Mon, 2 Jul 2018 12:07:06 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.619 X-Spam-Level: X-Spam-Status: No, score=-2.619 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vCC20kAVwK_U for ; Mon, 2 Jul 2018 12:07:04 -0700 (PDT) Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [217.70.183.194]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5A1621312C8 for ; Mon, 2 Jul 2018 12:07:04 -0700 (PDT) X-Originating-IP: 213.47.41.20 Received: from localhost (213-47-41-20.cable.dynamic.surfer.at [213.47.41.20]) (Authenticated sender: michael@niedermayer.cc) by relay2-d.mail.gandi.net (Postfix) with ESMTPSA id 2318F40013 for ; Mon, 2 Jul 2018 19:07:01 +0000 (UTC) Date: Mon, 2 Jul 2018 21:07:01 +0200 From: Michael Niedermayer To: cellar@ietf.org Message-ID: <20180702190701.GR4839@michaelspb> References: <87efgmdqou.fsf@bunkus.org> <20180702103455.GQ4839@michaelspb> <36a0dc7e-8579-d99c-4beb-e8f4ce019eff@mediaarea.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="HR4ti3j4RKbrMpBw" Content-Disposition: inline In-Reply-To: <36a0dc7e-8579-d99c-4beb-e8f4ce019eff@mediaarea.net> User-Agent: Mutt/1.5.24 (2015-08-30) Archived-At: Subject: Re: [Cellar] AV1 mapping Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2018 19:07:14 -0000 --HR4ti3j4RKbrMpBw Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jul 02, 2018 at 01:14:02PM +0200, Jerome Martinez wrote: > On 02/07/2018 12:51, Steve Lhomme wrote: > >2018-07-02 12:34 GMT+02:00 Michael Niedermayer : > >>Hi > >> > >>On Mon, Jul 02, 2018 at 09:47:45AM +0200, Moritz Bunkus wrote: > >>>Hey, > >>> > >>>>Looking at the term "coded video sequence" (CVS) in other codecs (H.2= 64 > >>>>and H.265) it seems it's a common term. And for those codec one Segme= nt > >>>>correspond to one CVS, with the parameters of that CVS stored in the > >>>>CodecPrivate (SPS + PPS for H.264 for example). So we should probably= go > >>>>in that simple way. > >>>I have quite a lot of h.264 samples here, mainly M2TS from DVB, where > >>>SPS/PPS change mid-stream, often multiple times. For such files the fi= rst > >>>occurrences of SPS/PPS make up the AvcC in CodecPrivate, but all key f= rames > >>>are still prefixed with the currently active SPS/PPS =E2=80=94 because= that's the > >>>only way to signal that stuff has actually changed. > >>IMHO All SPS/PPS should be in the "global header" (CodecPrivate) which = the > >>global header applies to, not just the first. >=20 > Not possible with e.g. real time streaming (you don't know in advance the > next SPS/PPS) In cases where the SPS/PPS can change and it is not known when the global header needs to be produced (that is for example real time streaming). it follows that there should be no global header or it should not contain= =20 any SPS/PPS. But only things which are known for the whole stream. I would argue this is kind of the definition of a global header. It applies globally to the whole stream.=20 Storing only one out of several PPS/SPS in the header=20 (which was mentioned above) feels rather incorrect to me. And any software which makes decissions based on SPS data like resolution and aspect ratio could benefit from knowing if it needs to scan the whole stream for SPS changes or if it can trust the first or global header [...] --=20 Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB If you drop bombs on a foreign country and kill a hundred thousand innocent people, expect your government to call the consequence "unprovoked inhuman terrorist attacks" and use it to justify dropping more bombs and killing more people. The technology changed, the idea is old. --HR4ti3j4RKbrMpBw Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAls6d9QACgkQYR7HhwQLD6tbdwCfQR12aZWekb/bfr7QBcImxkAq 5PoAn0aN6uW3uLKobYfDbnUnRmDM7KIh =qhs5 -----END PGP SIGNATURE----- --HR4ti3j4RKbrMpBw-- From nobody Mon Jul 2 12:45:29 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9E35F1311BB for ; Mon, 2 Jul 2018 12:45:27 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.119 X-Spam-Level: X-Spam-Status: No, score=-1.119 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, SPF_NEUTRAL=0.779, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lU269Yz5I8Uq for ; Mon, 2 Jul 2018 12:45:24 -0700 (PDT) Received: from server172-3.web-hosting.com (server172-3.web-hosting.com [68.65.122.111]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E3C981311EA for ; Mon, 2 Jul 2018 12:45:24 -0700 (PDT) Received: from [146.96.19.240] (port=24785 helo=[10.10.201.39]) by server172.web-hosting.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.91) (envelope-from ) id 1fa4lC-001lLx-Be; Mon, 02 Jul 2018 15:45:23 -0400 From: Dave Rice Message-Id: Content-Type: multipart/alternative; boundary="Apple-Mail=_1C0CF7A5-C88E-47F5-A2EC-B68AB079461A" Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Date: Mon, 2 Jul 2018 15:45:17 -0400 In-Reply-To: Cc: Codec Encoding for LossLess Archiving and Realtime transmission To: Michael Richardson , Steve Lhomme References: <15361.1528336434@localhost> X-Mailer: Apple Mail (2.3273) X-OutGoing-Spam-Status: No, score=-2.9 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Subject: Re: [Cellar] IANA Considerations for EBML X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2018 19:45:28 -0000 --Apple-Mail=_1C0CF7A5-C88E-47F5-A2EC-B68AB079461A Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi Michael, Steve, I might be mistaken here, but from the draft the process of registering = an Element ID via IANA only seems interested in the Element ID itself. = For a curious onlooker of the IANA registrations it might be helpful to = see what the associated DocType is so that the context and semantics = could be understood, otherwise the registration only seems to indicate = that somewhere someone registered it via first-come first-serve. Additionally I think I=E2=80=99d prefer (except for the IDs used by EBML = itself) that the combination of DocType and ElementID be required as = unique rather than only the ElementID itself. This seems analogous to = how the same node name is allowed in various XML Schemas but don=E2=80=99t= necessarily have any relation to one another in semantics. Is it acceptable to extend the IANA Considerations section to clarify = the required information for registrations? =46rom the discussion in the = pull request these scenarios seem to be: option a - the VINT_DATA of the Element ID option b - the VINT_DATA of the Element ID - the associated DocType option c - the VINT_DATA of the Element ID - the associated DocType - the full Element Definition as described at = https://tools.ietf.org/html/draft-ietf-cellar-ebml-04#section-14.1.3 = Dave Would it be possible to an interest > On Jun 17, 2018, at 10:57 AM, Steve Lhomme = wrote: >=20 > I think in general it would be better to use hexadecimal values for > the ranges. It's friendlier to know why these values when reading then > in "binary". Also they are coded as VINT but in the end are not > intrepreted as the integer values they represent but really the whole > ID as one. So the integer values are not what people will use/see. >=20 > 2018-06-07 3:53 GMT+02:00 Michael Richardson : >>=20 >> I've made a pull request, = https://github.com/Matroska-Org/ebml-specification/pull/179 >> but I include the text here so that we can discuss it on the list. >> I havne't gotten the references done correctly for mmark, so don't = worry >> about that for now. >>=20 >> Note: "Specification Required" does not imply IETF specification. >> It could be any SDO, and many other entities that might publish = things. >> First Come/First Served requires no document at all... just ask IANA. >> RFC Required does not imply standards track. >>=20 >> see https://tools.ietf.org/html/rfc8126 >>=20 >>=20 >> # IANA Considerations >>=20 >> This document creates a new IANA Registry called >> "CELLAR EBML Element ID Registry". >>=20 >> Element IDs are described in section {{#element-id}}. Element IDs = are >> encoded using the VINT mechanism described in >> section {{#variable-sized-integer}} can be between one and five bytes >> long. Five byte long Element IDs are possible only if declared in the = header. >>=20 >> One byte Element IDs are numbers between 1 and 126. These items are = valuable >> because they short, and need to be used for commonly repeated = elements. >> Values from 1 to 126 are to be allocated according to RFC Required. >>=20 >> Two byte Element IDs are numbers between 127 and 16382. >> Numbers may be allocated within this range according to Specification >> Required. >>=20 >> Three byte Element IDs are numbers between 16383 and 2097150. >> Numbers may be allocated within this range according to First Come = First >> Served. >>=20 >> Four byte Element IDs are numbers between 2097151 and 268435456. >> Four byte Element IDs are somewhat special in that they are useful = for >> resynchronizing to major structures in the event of data corruption = or >> loss. As such four byte Element IDs are split into two categories. >> Four byte Element IDs whose lower three bytes (as encoded) would make >> printable 7-bit ASCII values may be allocated only Specification = Required. >> Sequential allocation of values is not required: specifications = SHOULD >> include a specific request, and are encouraged to do early = allocations. >>=20 >> To be clear about the above category: Four Byte Element IDs always = start >> with hex 0x10 to 0x1F, and that byte may be chosen so that the = entire number >> has some desirable property, such as a specific CRC. The other three = bytes, >> when ALL having values between 33 (ASCII !) and 126 (ASCII ~), fall = into >> this catgory. >>=20 >> Other Four Byte Element IDs may be allocated by First Come First = Served. >>=20 >> Five Byte Element IDs (values from 268435457 upwards) are reserved = for >> Experimental use. >>=20 >>=20 >> -- >> Michael Richardson , Sandelman Software Works >> -=3D IPv6 IoT consulting =3D- >>=20 >>=20 >>=20 >>=20 >> _______________________________________________ >> Cellar mailing list >> Cellar@ietf.org >> https://www.ietf.org/mailman/listinfo/cellar >>=20 >=20 >=20 >=20 > --=20 > Steve Lhomme > Matroska association Chairman >=20 > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar --Apple-Mail=_1C0CF7A5-C88E-47F5-A2EC-B68AB079461A Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
Hi Michael, Steve,

I might be mistaken = here, but from the draft the process of registering an Element ID via = IANA only seems interested in the Element ID itself. For a curious = onlooker of the IANA registrations it might be helpful to see what the = associated DocType is so that the context and semantics could be = understood, otherwise the registration only seems to indicate that = somewhere someone registered it via first-come first-serve.

Additionally I think = I=E2=80=99d prefer (except for the IDs used by EBML itself) that the = combination of DocType and ElementID be required as unique rather than = only the ElementID itself. This seems analogous to how the same node = name is allowed in various XML Schemas but don=E2=80=99t necessarily = have any relation to one another in semantics.

Is it acceptable to extend the IANA = Considerations section to clarify the required information for = registrations? =46rom the discussion in the pull request these scenarios = seem to be:

option a
- the VINT_DATA of the Element = ID

option = b
- the VINT_DATA of the Element ID
- the associated DocType

option c
- the VINT_DATA of the Element ID
- the = associated DocType
- the full Element = Definition as described at https://tools.ietf.org/html/draft-ietf-cellar-ebml-04#section-1= 4.1.3

Dave

Would it be possible to an interest

On = Jun 17, 2018, at 10:57 AM, Steve Lhomme <slhomme@matroska.org> wrote:

I = think in general it would be better to use hexadecimal values for
the ranges. It's friendlier to know why these values when = reading then
in "binary". Also they are coded as VINT but = in the end are not
intrepreted as the integer values they = represent but really the whole
ID as one. So the integer = values are not what people will use/see.

2018-06-07 3:53 GMT+02:00 Michael Richardson <mcr+ietf@sandelman.ca>:

I've made a pull request, https://github.com/Matroska-Org/ebml-specification/pull/179=
but I include the text here so that we can discuss it on = the list.
I havne't gotten the references done correctly = for mmark, so don't worry
about that for now.

Note: "Specification Required" does not imply = IETF specification.
It could be any SDO, and many other = entities that might publish things.
First Come/First = Served requires no document at all... just ask IANA.
RFC = Required does not imply standards track.

see = https://tools.ietf.org/html/rfc8126


# IANA Considerations

This document creates a new IANA Registry called
"CELLAR EBML Element ID Registry".

Element IDs are described in section {{#element-id}}. =  Element IDs are
encoded using the VINT mechanism = described in
section {{#variable-sized-integer}} can be = between one and five bytes
long. Five byte long Element = IDs are possible only if declared in the header.

One byte Element IDs are numbers between 1 and 126. These = items are valuable
because they short, and need to be used = for commonly repeated elements.
Values from 1 to 126 are = to be allocated according to RFC Required.

Two byte Element IDs are numbers between 127 and 16382.
Numbers may be allocated within this range according to = Specification
Required.

Three = byte Element IDs are numbers between 16383 and 2097150.
Numbers may be allocated within this range according to First = Come First
Served.

Four byte = Element IDs are numbers between 2097151 and 268435456.
Four = byte Element IDs are somewhat special in that they are useful for
resynchronizing to major structures in the event of data = corruption or
loss.  As such four byte Element IDs = are split into two categories.
Four byte Element IDs whose = lower three bytes (as encoded) would make
printable 7-bit = ASCII values may be allocated only Specification Required.
Sequential allocation of values is not required: = specifications SHOULD
include a specific request, and are = encouraged to do early allocations.

To be = clear about the above category: Four Byte Element IDs always start
with hex 0x10 to 0x1F,  and that byte may be chosen so = that the entire number
has some desirable property, such = as a specific CRC.  The other three bytes,
when ALL = having values between 33 (ASCII !) and 126 (ASCII ~), fall into
this catgory.

Other Four Byte = Element IDs may be allocated by First Come First Served.
Five Byte Element IDs (values from 268435457 upwards) are = reserved for
Experimental use.


--
Michael Richardson <mcr+IETF@sandelman.ca>, Sandelman Software Works
-=3D IPv6 IoT consulting =3D-



_______________________________________________
Cellar mailing list
Cellar@ietf.org
https://www.ietf.org/mailman/listinfo/cellar




-- =
Steve Lhomme
Matroska association = Chairman

_______________________________________________
Cellar mailing list
Cellar@ietf.org
https://www.ietf.org/mailman/listinfo/cellar

= --Apple-Mail=_1C0CF7A5-C88E-47F5-A2EC-B68AB079461A-- From nobody Thu Jul 5 14:00:42 2018 Return-Path: X-Original-To: cellar@ietf.org Delivered-To: cellar@ietfa.amsl.com Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id BA79C130F89; Thu, 5 Jul 2018 14:00:33 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: Matthew Miller To: Cc: draft-ietf-cellar-ffv1.all@ietf.org, cellar@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 6.81.3 Auto-Submitted: auto-generated Precedence: bulk Message-ID: <153082443369.26281.17163340643478720739@ietfa.amsl.com> Date: Thu, 05 Jul 2018 14:00:33 -0700 Archived-At: Subject: [Cellar] Genart early review of draft-ietf-cellar-ffv1-03 X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.26 List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jul 2018 21:00:35 -0000 Reviewer: Matthew Miller Review result: On the Right Track I am the assigned Gen-ART reviewer for this draft. The General Area Review Team (Gen-ART) reviews all IETF documents being processed by the IESG for the IETF Chair. Please treat these comments just like any other last call comments. For more information, please see the FAQ at . Document: draft-ietf-cellar-ffv1-03 Reviewer: Matthew A. Miller Review Date: 2018-07-05 IETF LC End Date: N/A IESG Telechat date: N/A Summary: This document is moving in the right direction, but needs work. Overall, the document feels unfinished. It's clear a lot of work has gone into this, and there's been tremendous effort put into the details. However, it's lacking some clarity, that was present in older revisions that were removed; speculatively is it due to more generic coverage that later revisions covered? The introduction is quite helpful in answering the inevitable question "What is a FFV1?". For the most part, the flow of the document seems to make sense. Major issues: I can understand relying on pseudo-code for something very math- and/or algorithm-heavy, but some prose would be quite helpful in understanding how and why the parts relate to one another. The Definitions section is essentially what I would look for, but only accounts for some of the terms used within the rest of the document. As written, this document depends entirely on the reader being intimately familiar with the subject matter. Minor issues: * Please consider moving the text of Section 2.2.6. "Pseudo-code" up a level to 2.2. "Conventions" or as the first sub-section under 2.2. "Conventions". This points seems to me to warrant more significance than it currently has. * In reading this, I wondered if it would help improve the understanding of this document if Sections 3. and 4. swapped places. I think it's worth considering, but accept this suggestion could be rejected. * Please consider moving Section 4.8. "Parameters" to immediately proceed from Section 4.1. "Configuration Record". I think it would help with understanding the document. * Please consider moving the ASCII diagram of a Frame from Section 9.1.1. "Multi-threading support and independence of slices" to Section 4.2. "Frame". Nits/editorial comments: * In Section 2.1. "Definitions", a short description of what a "Frame" and "Slice" are conceptually would be very helpful. * In Section 2.2.4. "Mathematical functions", the definition of "ceil(a)"" appears to be a copy from "floor(a)"; I would suggest: """ ceil(a) the smallest integer greater than or equal to a """ * In Section 2.2.6. "Pseudo-code", the word "as" ought to be "and" in "as uses its". * The first use of "JEP2000-RCT" (in Section 3.3.) is not immediately followed with its reference. * In Section 3.7.2. "RGB", there is a paragraph that is solely ""[ISO.15444-1.2016]"", almost as if it's a reference that was meant a section heading. * In Section 3.8.1. "Range coding mode", there appears to be some odd formatting with "_G. Nigel_ and "N. Martin_"; is this an attempt to italicize? * Most of the subsection contents with Section 4. seem to have extraneous newlines (e.g., 4.1.1. "reserved_for_future_use"). * In Section 4.4.9. "sar_den", the text is identical to the preceding section. I think this is meant to be the denominator to complement "sar_num" as the numerator. * In Section 4.5.1. "primary_color_count", the pseudo-code is inline with no quotes, which is inconsistent with the rest of the document. * In Section 4.7.1. "slice_size", the "Note:" appears to be two sentence fragments. I think the following conveys the same meaning: """ Note: this allows finding the start of slices before previous slices have been fully decoded, and allows parallel decoding as well as error resilience. """ * Section 11. "ToDo" still as one item remaining. From nobody Fri Jul 6 08:23:34 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2BAEF13104A; Fri, 6 Jul 2018 08:23:17 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.119 X-Spam-Level: X-Spam-Status: No, score=-1.119 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, SPF_NEUTRAL=0.779, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4K_Ie1hGRec5; Fri, 6 Jul 2018 08:23:14 -0700 (PDT) Received: from server172-3.web-hosting.com (server172-3.web-hosting.com [68.65.122.111]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3BFD3131067; Fri, 6 Jul 2018 08:23:11 -0700 (PDT) Received: from [146.96.19.240] (port=49774 helo=[10.10.201.39]) by server172.web-hosting.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.91) (envelope-from ) id 1fbSZf-0026RJ-Tt; Fri, 06 Jul 2018 11:23:10 -0400 From: Dave Rice Message-Id: <4016F335-7C19-4F25-AED8-B4A97E0F3A97@dericed.com> Content-Type: multipart/alternative; boundary="Apple-Mail=_3475854A-DC9D-4597-9FCE-E4A62EBEC603" Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Date: Fri, 6 Jul 2018 11:23:06 -0400 In-Reply-To: <153082443369.26281.17163340643478720739@ietfa.amsl.com> Cc: gen-art@ietf.org, draft-ietf-cellar-ffv1.all@ietf.org, cellar@ietf.org To: Matthew Miller References: <153082443369.26281.17163340643478720739@ietfa.amsl.com> X-Mailer: Apple Mail (2.3273) X-OutGoing-Spam-Status: No, score=-2.1 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Subject: Re: [Cellar] Genart early review of draft-ietf-cellar-ffv1-03 X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2018 15:23:27 -0000 --Apple-Mail=_3475854A-DC9D-4597-9FCE-E4A62EBEC603 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi, Thank you Matthew for this detailed review. Here are my comments related = to a PR at https://github.com/FFmpeg/FFV1/pull/119 = which responds to some of = these comments. Other comments below may require more discussion. > On Jul 5, 2018, at 5:00 PM, Matthew Miller = wrote: >=20 > Reviewer: Matthew Miller > Review result: On the Right Track >=20 > I am the assigned Gen-ART reviewer for this draft. The General Area > Review Team (Gen-ART) reviews all IETF documents being processed > by the IESG for the IETF Chair. Please treat these comments just > like any other last call comments. >=20 > For more information, please see the FAQ at >=20 > . >=20 > Document: draft-ietf-cellar-ffv1-03 > Reviewer: Matthew A. Miller > Review Date: 2018-07-05 > IETF LC End Date: N/A > IESG Telechat date: N/A >=20 > Summary: >=20 > This document is moving in the right direction, but needs > work. Overall, the document feels unfinished. It's clear > a lot of work has gone into this, and there's been tremendous > effort put into the details. However, it's lacking some > clarity, that was present in older revisions that were removed; > speculatively is it due to more generic coverage that > later revisions covered? I=E2=80=99m not sure I understand as it hasn=E2=80=99t seemed like many = parts have been removed during the drafting work. Could you provide an = example? > The introduction is quite helpful in answering the inevitable > question "What is a FFV1?". For the most part, the flow of > the document seems to make sense. >=20 > Major issues: >=20 > I can understand relying on pseudo-code for something very math- > and/or algorithm-heavy, but some prose would be quite helpful in > understanding how and why the parts relate to one another. The > Definitions section is essentially what I would look for, but only > accounts for some of the terms used within the rest of the > document. As written, this document depends entirely on the > reader being intimately familiar with the subject matter. This has been discussed before and I=E2=80=99ve appreciated how some = other IETF documents which include pseudo-code include an =E2=80=9Cas = read=E2=80=9D narrative version of the code as well. I think there may = have been some concern of any risk of discrepancy between what the = pseudo-code and the narrative of the pseudo-code communicate. If anyone = know some boilerplate for managing this, please share. > Minor issues: >=20 > * Please consider moving the text of Section 2.2.6. "Pseudo-code" > up a level to 2.2. "Conventions" or as the first sub-section under > 2.2. "Conventions". This points seems to me to warrant more > significance than it currently has. I started a pull request at https://github.com/FFmpeg/FFV1/pull/119 = and moved this section. > * In reading this, I wondered if it would help improve the > understanding of this document if Sections 3. and 4. swapped > places. I think it's worth considering, but accept this > suggestion could be rejected. I reviewed these sections and feel hesitant swapping the order outright. = Any other comments from cellar on this proposal? > * Please consider moving Section 4.8. "Parameters" to immediately > proceed from Section 4.1. "Configuration Record". I think it > would help with understanding the document. I agree. I moved it in the pull request. > * Please consider moving the ASCII diagram of a Frame from > Section 9.1.1. "Multi-threading support and independence of slices" > to Section 4.2. "Frame=E2=80=9D. I agree. I moved it in the pull request. > Nits/editorial comments: >=20 > * In Section 2.1. "Definitions", a short description of what a > "Frame" and "Slice" are conceptually would be very helpful. I agree. I moved it in the pull request. Comments welcome on these = definitions: `Frame`: An encoded representation of a complete static image. `Slice`: A spatial sub-section of a `Frame` that is encoded separately = from an other region of the same frame. > * In Section 2.2.4. "Mathematical functions", the definition of > "ceil(a)"" appears to be a copy from "floor(a)"; I would suggest: >=20 > """ > ceil(a) the smallest integer greater than or equal to a > =E2=80=9C"" Already fixed in = https://github.com/FFmpeg/FFV1/commit/a6191aae2beb29879ac7f82530fa92a950f8= a4bb = . > * In Section 2.2.6. "Pseudo-code", the word "as" ought to be > "and" in "as uses its=E2=80=9D. Fixed in PR. > * The first use of "JEP2000-RCT" (in Section 3.3.) is not > immediately followed with its reference. I added the reference to the first use in the PR. > * In Section 3.7.2. "RGB", there is a paragraph that is solely > ""[ISO.15444-1.2016]"", almost as if it's a reference that was > meant a section heading. Fixed in = https://github.com/FFmpeg/FFV1/commit/8b6304ebf25764b5d2820497be1ad90ebadc= 624f = . > * In Section 3.8.1. "Range coding mode", there appears to be some > odd formatting with "_G. Nigel_ and "N. Martin_"; is this an > attempt to italicize? Removed in PR. > * Most of the subsection contents with Section 4. seem to have > extraneous newlines (e.g., 4.1.1. "reserved_for_future_use=E2=80=9D). I suggest some discussion on this. I remember that it was discussed by = not resolved. See https://github.com/FFmpeg/FFV1/pull/85 = . > * In Section 4.4.9. "sar_den", the text is identical to the > preceding section. I think this is meant to be the denominator > to complement "sar_num" as the numerator. Thanks. Fixed in PR. > * In Section 4.5.1. "primary_color_count", the pseudo-code is > inline with no quotes, which is inconsistent with the rest of the > document. Thanks. Fixed in PR. > * In Section 4.7.1. "slice_size", the "Note:" appears to be two > sentence fragments. I think the following conveys the same > meaning: >=20 > """ > Note: this allows finding the start of slices before previous > slices have been fully decoded, and allows parallel decoding as > well as error resilience. > =E2=80=9C"" Thanks. Fixed in PR. > * Section 11. "ToDo" still as one item remaining. This is moved to a github issue. Thanks again, Dave Rice --Apple-Mail=_3475854A-DC9D-4597-9FCE-E4A62EBEC603 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
Hi,
Thank you Matthew = for this detailed review. Here are my comments related to a PR = at https://github.com/FFmpeg/FFV1/pull/119 which = responds to some of these comments. Other comments below may require = more discussion.

On Jul 5, 2018, at 5:00 PM, Matthew Miller = <linuxwolf+ietf@outer-planes.net> wrote:

Reviewer: Matthew Miller
Review result: On the = Right Track

I am the assigned Gen-ART = reviewer for this draft. The General Area
Review Team = (Gen-ART) reviews all IETF documents being processed
by = the IESG for the IETF Chair.  Please treat these comments just
like any other last call comments.

For more information, please see the FAQ at

<https://trac.ietf.org/trac/gen/wiki/GenArtfaq>.

Document: draft-ietf-cellar-ffv1-03
Reviewer: Matthew A. Miller
Review Date: = 2018-07-05
IETF LC End Date: N/A
IESG = Telechat date: N/A

Summary:
This document is moving in the right direction, but needs
work.  Overall, the document feels unfinished. =  It's clear
a lot of work has gone into this, and = there's been tremendous
effort put into the details. =  However, it's lacking some
clarity, that was present = in older revisions that were removed;
speculatively is it = due to more generic coverage that
later revisions = covered?

I=E2=80=99m not sure I understand as it hasn=E2=80=99= t seemed like many parts have been removed during the drafting work. = Could you provide an example?

The = introduction is quite helpful in answering the inevitable
question "What is a FFV1?".  For the most part, the flow = of
the document seems to make sense.

Major issues:

I can understand = relying on pseudo-code for something very math-
and/or = algorithm-heavy, but some prose would be quite helpful in
understanding how and why the parts relate to one another. =  The
Definitions section is essentially what I would = look for, but only
accounts for some of the terms used = within the rest of the
document.  As written, this = document depends entirely on the
reader being intimately = familiar with the subject matter.

This = has been discussed before and I=E2=80=99ve appreciated how some other = IETF documents which include pseudo-code include an =E2=80=9Cas read=E2=80= =9D narrative version of the code as well. I think there may have been = some concern of any risk of discrepancy between what the pseudo-code and = the narrative of the pseudo-code communicate. If anyone know some = boilerplate for managing this, please share.

Minor issues:

* Please consider = moving the text of Section 2.2.6. "Pseudo-code"
up a level = to 2.2. "Conventions" or as the first sub-section under
2.2.= "Conventions".  This points seems to me to warrant more
significance than it currently has.

I = started a pull request at https://github.com/FFmpeg/FFV1/pull/119 and moved = this section.

* In reading this, I wondered = if it would help improve the
understanding of this = document if Sections 3. and 4. swapped
places.  I = think it's worth considering, but accept this
suggestion = could be rejected.

I reviewed these sections and feel hesitant = swapping the order outright. Any other comments from cellar on this = proposal?

* Please consider moving Section 4.8. = "Parameters" to immediately
proceed from Section 4.1. = "Configuration Record".  I think it
would help with = understanding the document.

I = agree. I moved it in the pull request.

* Please = consider moving the ASCII diagram of a Frame from
Section = 9.1.1. "Multi-threading support and independence of slices"
to Section 4.2. "Frame=E2=80=9D.

I = agree. I moved it in the pull request.

Nits/editorial = comments:

* In Section 2.1. "Definitions", = a short description of what a
"Frame" and "Slice" are = conceptually would be very helpful.

I = agree. I moved it in the pull request. Comments welcome on these = definitions:

`Frame`: An encoded = representation of a complete static image.

`Slice`: A spatial sub-section of a `Frame` that is encoded = separately from an other region of the same frame.
* In Section 2.2.4. "Mathematical functions", the definition = of
"ceil(a)"" appears to be a copy from "floor(a)"; I = would suggest:

  """
  ceil(a) the smallest integer greater than or = equal to a
  =E2=80=9C""


* In Section 2.2.6. = "Pseudo-code", the word "as" ought to be
"and" in "as uses = its=E2=80=9D.

Fixed in PR.

* The first use = of "JEP2000-RCT" (in Section 3.3.) is not
immediately = followed with its reference.

I = added the reference to the first use in the PR.

* In Section 3.7.2. "RGB", there is a paragraph that is = solely
""[ISO.15444-1.2016]"", almost as if it's a = reference that was
meant a section heading.


* In Section 3.8.1. "Range = coding mode", there appears to be some
odd formatting with = "_G. Nigel_ and "N. Martin_"; is this an
attempt to = italicize?

Removed in PR.

* Most of the = subsection contents with Section 4. seem to have
extraneous = newlines (e.g., 4.1.1. "reserved_for_future_use=E2=80=9D).

I = suggest some discussion on this. I remember that it was discussed by not = resolved. See https://github.com/FFmpeg/FFV1/pull/85.

* In Section 4.4.9. "sar_den", the text is identical to = the
preceding section.  I think this is meant to be = the denominator
to complement "sar_num" as the = numerator.

Thanks. Fixed in PR.

* In Section = 4.5.1. "primary_color_count", the pseudo-code is
inline = with no quotes, which is inconsistent with the rest of the
document.

Thanks. Fixed in PR.

* In Section = 4.7.1. "slice_size", the "Note:" appears to be two
sentence = fragments.  I think the following conveys the same
meaning:

  """
  Note: this allows finding the start of slices = before previous
  slices have been fully = decoded, and allows parallel decoding as
  well = as error resilience.
  =E2=80=9C""

Thanks.= Fixed in PR.

* Section 11. "ToDo" still as = one item remaining.

This is moved to a github issue.

Thanks again,
Dave Rice

= --Apple-Mail=_3475854A-DC9D-4597-9FCE-E4A62EBEC603-- From nobody Fri Jul 6 10:50:49 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 06EAF129C6A for ; Fri, 6 Jul 2018 10:50:45 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, T_DKIMWL_WL_MED=-0.01, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=outer-planes-net.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gzQ-lj8usiMg for ; Fri, 6 Jul 2018 10:50:42 -0700 (PDT) Received: from mail-oi0-x244.google.com (mail-oi0-x244.google.com [IPv6:2607:f8b0:4003:c06::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1CCED130E6C for ; Fri, 6 Jul 2018 10:50:40 -0700 (PDT) Received: by mail-oi0-x244.google.com with SMTP id v8-v6so24984546oie.5 for ; Fri, 06 Jul 2018 10:50:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outer-planes-net.20150623.gappssmtp.com; s=20150623; h=sender:subject:to:cc:references:from:openpgp:autocrypt:message-id :date:user-agent:mime-version:in-reply-to; bh=74xlgS7T4SV3piyy/e1HiZUrCe8ab41U2JiJMO3w6Ts=; b=Q+VavmZzLH0pepmFA1himcD6K6uzN+cMw70R7B1y5C3dBbvMTslVLBT+U7bnfRasML LsuIkK/uNas5KpTrnSF8+8/erUF9qb6+Q6jCXkJ8To9t4jpFD/vqRaFlwv8FDeQf6uAJ +Crw3x5M4JNYXXZ7f/WBuMSjCdOalYLVVijmWVzH75N2xef00XM8tzWRQ2w9qXQbEBUv v8L6ejocTbIhwvJfAs4hhrcA1uk2MDJnUumOjPScSAIZH6Xp5S/Kttw7FNeq+hBJiLQm qHB0c+1Gym8Swe8xiqRSZnZbOx0olRIavLJOQnbvErbMxXQwO3kK70TJnzWyzKm/hMuJ vUCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:to:cc:references:from:openpgp :autocrypt:message-id:date:user-agent:mime-version:in-reply-to; bh=74xlgS7T4SV3piyy/e1HiZUrCe8ab41U2JiJMO3w6Ts=; b=AGlcXb/LRUZ05S/WM5FmnjThgtBpGDvYwMkpYqNzygX7rEht/zif8n1NdzGh8c3DqY N0MjgUNYEr4560dQogBxm+CsxQ0cvRr7Jot9iYBI2C148PcFuLsZT4Pe2mOH+msT+a2+ xR66w3vlA2OPJI55OHvzWw2C2J809Fgg7F0BrhIr0urelKDO1c/TdDczBKbA3pD39RE6 VQWETESDWJql7oMoTwEGkHueEjOdMHLFpGN327h1vD425gr7n6ADRdh/5wuGyIoPmsYH sqpQtYXgg/DP09BX5Rm0g60Zkrf2yKgJ7mEYikHv3dVex6oIhm4BJkqolo0Bw/9DJSwA i2JQ== X-Gm-Message-State: APt69E0woec+6inprP/9EkALyhcbxnWfyJUqJZh36txJDdKh6IAx3ioS zUsM2Xz8ifHDKUY0malV3Tg/w5FWtOs= X-Google-Smtp-Source: AAOMgpfRupIIYjbTt2dVxUAAEJ22PtL6EE0pndUEzER2F771kquxAcEjGmg1o/7YnMiahuNhvLR4IQ== X-Received: by 2002:aca:4455:: with SMTP id r82-v6mr7062027oia.260.1530899439128; Fri, 06 Jul 2018 10:50:39 -0700 (PDT) Received: from [10.6.21.160] ([128.177.113.102]) by smtp.gmail.com with ESMTPSA id v85-v6sm1782134oie.57.2018.07.06.10.50.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Jul 2018 10:50:38 -0700 (PDT) Sender: Matthew Miller To: Dave Rice Cc: gen-art@ietf.org, draft-ietf-cellar-ffv1.all@ietf.org, cellar@ietf.org References: <153082443369.26281.17163340643478720739@ietfa.amsl.com> <4016F335-7C19-4F25-AED8-B4A97E0F3A97@dericed.com> From: "Matthew A. Miller" Openpgp: preference=signencrypt Autocrypt: addr=linuxwolf+ietf@outer-planes.net; prefer-encrypt=mutual; keydata= xsBNBFJoAooBCADQmEtpbpY/4wTeKgZIuyG7HkxIFgiUeqOvtiBKj/pCA73d7Q5hCvQdGcKJ 6uZsYz3Il9oKoKFxVt90iEXspbE39g6ek19e6RsB4j0Q10l4QvH+EqeD760gs0H2yf/eYj9i uk9/VY6axdQlPsmid1zoQgCNjSM7X4/K26WGMs03sbXJpKdoonelzIlJSNfzi0q546iplo72 D2cCm9BriMkQvcGnsm4B9eBIBn3GKmVx1tsmPNeNTyun2DvaLnrYxbA0Ivo1DzZReds9NZ25 uROI/+b+lcg9/kmHzhK+q8NMQCFWmqpS/lZRKxVBSijKGpGr5h8VLVf5iURHtwG+B/QxABEB AAHNLk1hdHRoZXcgQS4gTWlsbGVyIDxsaW51eHdvbGZAb3V0ZXItcGxhbmVzLm5ldD7CwIAE EwEKACoCGwMFCwkIBwMFFQoJCAsFFgIDAQACHgECF4AFCQvHJDEFAlirCeQCGQEACgkQ7PRy ThCeBbt+sAgAzUQokr+f+ArieIrv2JkiQLqiBaZX29Aph9YwG3OPLWSdESEKkFOSJT0LWbsC cAKHLrVfgl2+6iPhf4OOacTdqK7wS6vruPZC1ChdO7NZTgbVa0hP/Q/QKEoaMGNdfc1/lgxY 5kwh+bvGIF1+HyadytgCBBHxdVEhYI7G3ejKqA8iVwri1VW0Wjp8iWdjpF74swIHhid5GcAu 6VJgVNJw3P+WkTkNrkd2tx5yUfNXQuGyFhxwlpiuaOpIk3p74P6e8h/riMpkJ5mIH/ryGTH7 qxpEIuep2bLQZmGwBen8kf3MO/VbiA/NMY6OHdc93EBKr0g7n2BA5uFLdy79FqAA3M7ATQRS aAKKAQgAwP67h8GJUO6XYyWOrcJGXDJnnZEDS+q+bTQXkJMFa74rVIx0yioqY8QdpBJFGaMT 4DCNYe/3pw61ZTDDKqukSCfOh/ssdd8zSGTQZSX5lR4B4+00/LKWugP6iHHHYiETbBVb5bxc aR/LE41Wx3z2HsW3TkeZB6WVk82MTclS1zCuY3p9AeCvr424BSQL7KC38y2eQc95G+nabsVD c6oQ8oZOf1D2giBb2VgbYkSppKj8BKvBtmjCauWeEq/AkZKaDAdua8Qj0vEfgcoh8aavlPJi rqj1YNSyc3AO4R5prPGgTepcUpW8ip8xIPAFoJXfuvsZSV7uVP36gwApU4+ZnwARAQABwsB8 BBgBCgAmAhsMFiEEMddYjeyQaQ1rzJjg7PRyThCeBbsFAlpvpIsFCQvLWoEACgkQ7PRyThCe BbuNHAf/cchJ7kHoIr5i+jgVRuR71AGlxlMbVolnS5tza3bi9Ie63LRdOtMUE3pDUQo25cWd cP7pzwwRBCDD2GxfIuyMCWaES0xtQdTIyNOAFFOtBtCFOrsNEk+iLAu6GBr4QzSQKW1QW4/b vcfpM2pLQn7Zd6naUioEYfTHCMmYHr7hQXaPNEQ7V/J4pLVAN8bHyVgQ9ciQN91DUs6jnueM BUW7DNvuHq0RDzA+ufYdpQAjwl4z1v+rnJ79P3HTxfFdiTTAk9MjyVQklHxS067cmLYkSOku dnCOHhDmSFwkKd9EwOBNuztpjCzmM5SgOT+U/iHH+IM/Hv6bjVCiAQ5WOihe6Q== Message-ID: <31b43c76-1c3f-a669-fc1d-9f08999cebf3@outer-planes.net> Date: Fri, 6 Jul 2018 11:50:37 -0600 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:60.0) Gecko/20100101 Thunderbird/60.0 MIME-Version: 1.0 In-Reply-To: <4016F335-7C19-4F25-AED8-B4A97E0F3A97@dericed.com> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="3u1ditHKoHblvT2tUoxlmJIBmJaPvscYq" Archived-At: Subject: Re: [Cellar] Genart early review of draft-ietf-cellar-ffv1-03 X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2018 17:50:46 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --3u1ditHKoHblvT2tUoxlmJIBmJaPvscYq Content-Type: multipart/mixed; boundary="BzHV6VTEnhfTgcQIelHQzUM5upOugXmU6"; protected-headers="v1" From: "Matthew A. Miller" To: Dave Rice Cc: gen-art@ietf.org, draft-ietf-cellar-ffv1.all@ietf.org, cellar@ietf.org Message-ID: <31b43c76-1c3f-a669-fc1d-9f08999cebf3@outer-planes.net> Subject: Re: Genart early review of draft-ietf-cellar-ffv1-03 References: <153082443369.26281.17163340643478720739@ietfa.amsl.com> <4016F335-7C19-4F25-AED8-B4A97E0F3A97@dericed.com> In-Reply-To: <4016F335-7C19-4F25-AED8-B4A97E0F3A97@dericed.com> --BzHV6VTEnhfTgcQIelHQzUM5upOugXmU6 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable Thanks Dave for the quick work! I've read through the PR and commits you provided, and think it will be in much better shape. I look forward to reviewing the next published revision. I've made contextual responses below to outstanding questions. On 18/07/06 09:23, Dave Rice wrote: > Hi, > Thank you Matthew for this detailed review. Here are my comments relate= d > to a PR at=C2=A0https://github.com/FFmpeg/FFV1/pull/119=C2=A0which resp= onds to > some of these comments. Other comments below may require more discussio= n. >=20 >> On Jul 5, 2018, at 5:00 PM, Matthew Miller >> > > wrote: >> >> Reviewer: Matthew Miller >> Review result: On the Right Track >> >> I am the assigned Gen-ART reviewer for this draft. The General Area >> Review Team (Gen-ART) reviews all IETF documents being processed >> by the IESG for the IETF Chair. =C2=A0Please treat these comments just= >> like any other last call comments. >> >> For more information, please see the FAQ at >> >> . >> >> Document: draft-ietf-cellar-ffv1-03 >> Reviewer: Matthew A. Miller >> Review Date: 2018-07-05 >> IETF LC End Date: N/A >> IESG Telechat date: N/A >> >> Summary: >> >> This document is moving in the right direction, but needs >> work. =C2=A0Overall, the document feels unfinished. =C2=A0It's clear >> a lot of work has gone into this, and there's been tremendous >> effort put into the details. =C2=A0However, it's lacking some >> clarity, that was present in older revisions that were removed; >> speculatively is it due to more generic coverage that >> later revisions covered? >=20 > I=E2=80=99m not sure I understand as it hasn=E2=80=99t seemed like many= parts have been > removed during the drafting work. Could you provide an example? >=20 When I first opened this document, I felt overwhelmed by some of the terminology and unclear on what the structure/architecture was, so I did some spelunking. It's largely why I took my review is five days late despite having 30 days to get through it (that, and I still procrastinated some in the middle). I went all the way back to versions prior to WG adoption and saw there were some additional prose throughout; although it seemed the earlier revisions were more restricted in what could be represented in FFV1 than the current document allows, it helped me better understand. I apologize for not taking more detailed note of which sections in those previous revisions helped me the most, but I can try to do that. However, I think the additions and reorganization in the PR goes a long way to helping already. >> The introduction is quite helpful in answering the inevitable >> question "What is a FFV1?". =C2=A0For the most part, the flow of >> the document seems to make sense. >> >> Major issues: >> >> I can understand relying on pseudo-code for something very math- >> and/or algorithm-heavy, but some prose would be quite helpful in >> understanding how and why the parts relate to one another. =C2=A0The >> Definitions section is essentially what I would look for, but only >> accounts for some of the terms used within the rest of the >> document. =C2=A0As written, this document depends entirely on the >> reader being intimately familiar with the subject matter. >=20 > This has been discussed before and I=E2=80=99ve appreciated how some ot= her IETF > documents which include pseudo-code include an =E2=80=9Cas read=E2=80=9D= narrative > version of the code as well. I think there may have been some concern o= f > any risk of discrepancy between what the pseudo-code and the narrative > of the pseudo-code communicate. If anyone know some boilerplate for > managing this, please share. >=20 My experience has been that (pseud-)code is great to cover the details, but the overall forest can get lost reading each tree. I think the additional terms added plus some of the reorganizing addresses this concern, but I'd need to re-read the document with all the changes to be sure. >> Minor issues: >> >> * Please consider moving the text of Section 2.2.6. "Pseudo-code" >> up a level to 2.2. "Conventions" or as the first sub-section under >> 2.2. "Conventions". =C2=A0This points seems to me to warrant more >> significance than it currently has. >=20 > I started a pull request at=C2=A0https://github.com/FFmpeg/FFV1/pull/11= 9=C2=A0and > moved this section. >=20 >> * In reading this, I wondered if it would help improve the >> understanding of this document if Sections 3. and 4. swapped >> places. =C2=A0I think it's worth considering, but accept this >> suggestion could be rejected. >=20 > I reviewed these sections and feel hesitant swapping the order outright= =2E > Any other comments from cellar on this proposal? >=20 I personally felt I had a better grasp of the document after reading Section 4 and re-reading Section 3. If such a big change is too much, think it might be possible to expand the introduction or background a bit to restate/duplicate some of the high-level structure; e.g., how Containers, Frames, and Slices fit together. >> * Please consider moving Section 4.8. "Parameters" to immediately >> proceed from Section 4.1. "Configuration Record". =C2=A0I think it >> would help with understanding the document. >=20 > I agree. I moved it in the pull request. >=20 >> * Please consider moving the ASCII diagram of a Frame from >> Section 9.1.1. "Multi-threading support and independence of slices" >> to Section 4.2. "Frame=E2=80=9D. >=20 > I agree. I moved it in the pull request. >=20 >> Nits/editorial comments: >> >> * In Section 2.1. "Definitions", a short description of what a >> "Frame" and "Slice" are conceptually would be very helpful. >=20 > I agree. I moved it in the pull request. Comments welcome on these > definitions: >=20 > `Frame`: An encoded representation of a complete static image. >=20 > `Slice`: A spatial sub-section of a `Frame` that is encoded separately > from an other region of the same frame. >=20 >> * In Section 2.2.4. "Mathematical functions", the definition of >> "ceil(a)"" appears to be a copy from "floor(a)"; I would suggest: >> >> =C2=A0=C2=A0""" >> =C2=A0=C2=A0ceil(a) the smallest integer greater than or equal to a >> =C2=A0=C2=A0=E2=80=9C"" >=20 > Already fixed > in=C2=A0https://github.com/FFmpeg/FFV1/commit/a6191aae2beb29879ac7f8253= 0fa92a950f8a4bb. >=20 >> * In Section 2.2.6. "Pseudo-code", the word "as" ought to be >> "and" in "as uses its=E2=80=9D. >=20 > Fixed in PR. >=20 >> * The first use of "JEP2000-RCT" (in Section 3.3.) is not >> immediately followed with its reference. >=20 > I added the reference to the first use in the PR. >=20 >> * In Section 3.7.2. "RGB", there is a paragraph that is solely >> ""[ISO.15444-1.2016]"", almost as if it's a reference that was >> meant a section heading. >=20 > Fixed > in=C2=A0https://github.com/FFmpeg/FFV1/commit/8b6304ebf25764b5d2820497b= e1ad90ebadc624f. >=20 >> * In Section 3.8.1. "Range coding mode", there appears to be some >> odd formatting with "_G. Nigel_ and "N. Martin_"; is this an >> attempt to italicize? >=20 > Removed in PR. >=20 >> * Most of the subsection contents with Section 4. seem to have >> extraneous newlines (e.g., 4.1.1. "reserved_for_future_use=E2=80=9D). >=20 > I suggest some discussion on this. I remember that it was discussed by > not resolved. See=C2=A0https://github.com/FFmpeg/FFV1/pull/85. >=20 I read the commentary from that PR, and appreciate the argument. I'm also anticipating the RFC Editor coming back with proposed changes that better match their conventions. If I may suggest after read the PR comments, taking the current line breaks and treating them as paragraphs seems like a reasonable editorial change to me that appears to align with the comments. >> * In Section 4.4.9. "sar_den", the text is identical to the >> preceding section. =C2=A0I think this is meant to be the denominator >> to complement "sar_num" as the numerator. >=20 > Thanks. Fixed in PR. >=20 >> * In Section 4.5.1. "primary_color_count", the pseudo-code is >> inline with no quotes, which is inconsistent with the rest of the >> document. >=20 > Thanks. Fixed in PR. >=20 >> * In Section 4.7.1. "slice_size", the "Note:" appears to be two >> sentence fragments. =C2=A0I think the following conveys the same >> meaning: >> >> =C2=A0=C2=A0""" >> =C2=A0=C2=A0Note: this allows finding the start of slices before previ= ous >> =C2=A0=C2=A0slices have been fully decoded, and allows parallel decodi= ng as >> =C2=A0=C2=A0well as error resilience. >> =C2=A0=C2=A0=E2=80=9C"" >=20 > Thanks. Fixed in PR. >=20 >> * Section 11. "ToDo" still as one item remaining. >=20 > This is moved to a github issue. >=20 > Thanks again, > Dave Rice >=20 And thank you, - m&m Matthew A. Miller --BzHV6VTEnhfTgcQIelHQzUM5upOugXmU6-- --3u1ditHKoHblvT2tUoxlmJIBmJaPvscYq Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCgAdFiEEMddYjeyQaQ1rzJjg7PRyThCeBbsFAls/q+0ACgkQ7PRyThCe BbvWVgf6Ak2tkJTPDBatJQXoYXfrpFmYxhD57AIxH/DnSTOOcjT473X9lgyENH/S ri7obc2chyJqE4hWIiaa70ndkRzB4Yh2ghxlltpkRIdiSoJOdLkN/Vv/CFr2iBHe oV3ySCcH2hyrcDmNIh1j3iiqViF4e6/1l+31KRU6Rjj3iBI9+MIKfARmJhCv4Tiw MnhvjLbotPf5W/M/Br7t4F+BubXn91yCxpE1xGENbuuWDAZvitz45NFYueckM25G libzsoDg6jiEt6p4WowovtM1aMtLN6eVuWhbnPGiLLmTcJoE3o5mx3P3JU34YiOy 2P94/EthHiKeVfNivXMUBOJKJZf1mA== =OqY4 -----END PGP SIGNATURE----- --3u1ditHKoHblvT2tUoxlmJIBmJaPvscYq-- From nobody Mon Jul 9 01:17:46 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 77BAC130F1B for ; Mon, 9 Jul 2018 01:17:44 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, T_DKIMWL_WL_MED=-0.01] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WubsHSAUgvbD for ; Mon, 9 Jul 2018 01:17:41 -0700 (PDT) Received: from mail-pf0-x235.google.com (mail-pf0-x235.google.com [IPv6:2607:f8b0:400e:c00::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C3DFA130E27 for ; Mon, 9 Jul 2018 01:17:41 -0700 (PDT) Received: by mail-pf0-x235.google.com with SMTP id x13-v6so3198144pfh.5 for ; Mon, 09 Jul 2018 01:17:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-transfer-encoding; bh=sWs5L73ybUg4rwbSWMECBI6oHK/Zol3zQ5cXtHL3fMk=; b=EwHq5S+VYyBr+xOUQuGZaB86M+kQi2Y2oJoT5czwm1LCMX3D9Lau0nPQwcMeQnZLCJ usBn7votqdEQJluzMuKYuZCocJngtSbPMxQ5uFtjsjel436kAPcsd4cTC8aKu4/5C9lq JiDN5lukVQVJX9wuoMXVBfMRIuF+aMzNUBoHr0p5BYJ0cmoSkmQOk6zCpiP1S1iyrGuX cQtsHfxmwTMhDJEo8LG/Y4p6Ly9eMUjGnd2HEOd5qM+N/A9OLL4+L0Jj13enCW4MB58b /PpFEDvlYxo8mepznv6y+oZFrXdv6Rm2IKj+yZpoEOBwDAoLLBs0Dqg2rk6QJXHTrM+4 o7VA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-transfer-encoding; bh=sWs5L73ybUg4rwbSWMECBI6oHK/Zol3zQ5cXtHL3fMk=; b=R1KQ25/9b9wHSmwDYk9ie00LYiG4+gAT+Fs9MV0ORdzuEqriEFsrMgziVIKo9IrGOK alxSucHOZGuMAOh8MB7XwPaK6KXlH/dwa7xQhOjIFQDOfPIvZssOwRzL+eOgKGIwa0VS oaaQNv/bwdQ2nQXa0affCvu6bwqTEYhDfdE0c0XsEYpsZ1ofvCt67vQR+5jPqDlpD4sO 4cdCH7mF6gFsSJ98ShaOxM49pOtRfCDmcoF4hciJhjTM1OWDZ9YgQHxKNvHIDOwkASG0 /9sISOAbWmsHpJQjJ/u6ZHe/4BRs/rz3ia8UnY0Y8XrangaO/71qAdnkDBDcx9julNc/ SqAQ== X-Gm-Message-State: APt69E2tAtyW0ojm8AX39uVspXGFcmvmLoFl84yAvNqN7KDLFTGLqjkX lPFQCzavSnAytJtHEdLBff6WQiEEWF0e6y/8VH5VeQ== X-Google-Smtp-Source: AAOMgpf37zi5pgnz5ZuRXA9nYnOC5yW0FwZ78LGGJPPaW6lKU8xQoAkDUlI+//FNK5E0kuT7RYyRFptbgoajZi+Q2+8= X-Received: by 2002:a62:384:: with SMTP id 126-v6mr2985557pfd.11.1531124260940; Mon, 09 Jul 2018 01:17:40 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a17:90a:17af:0:0:0:0 with HTTP; Mon, 9 Jul 2018 01:17:40 -0700 (PDT) In-Reply-To: <20180702190701.GR4839@michaelspb> References: <87efgmdqou.fsf@bunkus.org> <20180702103455.GQ4839@michaelspb> <36a0dc7e-8579-d99c-4beb-e8f4ce019eff@mediaarea.net> <20180702190701.GR4839@michaelspb> From: Steve Lhomme Date: Mon, 9 Jul 2018 10:17:40 +0200 Message-ID: To: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Archived-At: Subject: Re: [Cellar] AV1 mapping Matroska X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jul 2018 08:17:45 -0000 2018-07-02 21:07 GMT+02:00 Michael Niedermayer : > On Mon, Jul 02, 2018 at 01:14:02PM +0200, Jerome Martinez wrote: >> On 02/07/2018 12:51, Steve Lhomme wrote: >> >2018-07-02 12:34 GMT+02:00 Michael Niedermayer = : >> >>Hi >> >> >> >>On Mon, Jul 02, 2018 at 09:47:45AM +0200, Moritz Bunkus wrote: >> >>>Hey, >> >>> >> >>>>Looking at the term "coded video sequence" (CVS) in other codecs (H.= 264 >> >>>>and H.265) it seems it's a common term. And for those codec one Segm= ent >> >>>>correspond to one CVS, with the parameters of that CVS stored in the >> >>>>CodecPrivate (SPS + PPS for H.264 for example). So we should probabl= y go >> >>>>in that simple way. >> >>>I have quite a lot of h.264 samples here, mainly M2TS from DVB, where >> >>>SPS/PPS change mid-stream, often multiple times. For such files the f= irst >> >>>occurrences of SPS/PPS make up the AvcC in CodecPrivate, but all key = frames >> >>>are still prefixed with the currently active SPS/PPS =E2=80=94 becaus= e that's the >> >>>only way to signal that stuff has actually changed. >> >>IMHO All SPS/PPS should be in the "global header" (CodecPrivate) which= the >> >>global header applies to, not just the first. >> >> Not possible with e.g. real time streaming (you don't know in advance th= e >> next SPS/PPS) > > In cases where the SPS/PPS can change and it is not known when the global > header needs to be produced (that is for example real time streaming). > it follows that there should be no global header or it should not contain > any SPS/PPS. But only things which are known for the whole stream. In Matroska this can be achieved by starting a new Segment when the codec parameters change. That has an impact on all tracks at the same time so it may not be good/friendly in all cases. We could describe this scenario in the specs when a `PrevUID` should be used to specify the new Segment is a continuation of the previous segment. Actually even the NextUID could be generated for each Segment and written when it starts. Technically it would be a correct segment hard-linking but in the context of live streaming (unknown duration per segment). > I would argue this is kind of the definition of a global header. > It applies globally to the whole stream. > Storing only one out of several PPS/SPS in the header > (which was mentioned above) feels rather incorrect to me. And any > software which makes decissions based on SPS data like resolution and > aspect ratio could benefit from knowing if it needs to scan the whole > stream for SPS changes or if it can trust the first or global header > > > [...] > -- > Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB > > If you drop bombs on a foreign country and kill a hundred thousand > innocent people, expect your government to call the consequence > "unprovoked inhuman terrorist attacks" and use it to justify dropping > more bombs and killing more people. The technology changed, the idea is o= ld. > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > --=20 Steve Lhomme Matroska association Chairman From nobody Mon Jul 9 03:24:22 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C9B41130F69 for ; Mon, 9 Jul 2018 03:24:20 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, T_DKIMWL_WL_MED=-0.01] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xmGO9Rnci2fl for ; Mon, 9 Jul 2018 03:24:19 -0700 (PDT) Received: from mail-pf0-x235.google.com (mail-pf0-x235.google.com [IPv6:2607:f8b0:400e:c00::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 135F9130E6B for ; Mon, 9 Jul 2018 03:24:19 -0700 (PDT) Received: by mail-pf0-x235.google.com with SMTP id x10-v6so3505368pfm.4 for ; Mon, 09 Jul 2018 03:24:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=+OcJVrwGHlxTtE5BKREdIRwerWnthT9xJKvMe3IDuTc=; b=zsNXm93s0wZGi2Y/txzSUrRjDOBI4RuBIt4H0ymlsEeVEUSkzM0Tfyy2W1QhlCNhhj Rf+PJ524idPvd5gkJFAN3AJ5HEIK+GLH0zm8mytE9QaD0DgtLK0r/8tNPuE6jjn7oCn/ Jq6SzrYCdzenVd1BhDdZA4C/8982R9gIVlujuyza2XnZDGvaFRDWHclZpP3jNnCRVpkz XSWgXxGlPQUgTWMII09X7bl6vljwBMy+8w9gyV0GXj05n9bwUtpObTBWIXdARgkR162n KPzuLqpuJSRWHxS6BvT4rfqPbbQ3XYnjawAT+jiBb2/dbUJ8xJpVWXhKk1oVdb6BV8sh +oxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=+OcJVrwGHlxTtE5BKREdIRwerWnthT9xJKvMe3IDuTc=; b=qKu+2XUX25cxH/0LRAtaaJ77Bog+CVyA+E3/AofRjCrtpkfofdwzax+BBfzkkEljZm a15CPsIoKS264KJ6/9AcMPJ1kFKweYtN7nDSFMgn8AtkuVoxrfY/5jfRpbhQl88CXkrJ nvk+G0ELBG/Lwduyp2jemwYgsx5QnHgW7Qr+A/qlbbfeiO/DQuDztyQcZAr/D5mmXCJ6 9fIDYZo8R0+Qz8GnZ9BF+MpNOiPaxdiOREI3u8htUiRFibRNNSHKSoyMeGQ0p9C/a45+ x1+mujAZg/JjVfHWRp4lvR4Q9msTU2moJ/gwA8nGXF1P/p9eYKCn+PwRmHEbu1bvG08K nY0w== X-Gm-Message-State: APt69E2CDVMqDlQ/MlV+zUXHkuclfI0bb3S4JL49mcXa9gchqex8Lgja QnWwsoDYM3lm7c2FU4ukxf4bX+7+vq20s9nIap35milL X-Google-Smtp-Source: AAOMgpeUJVAsrdzmgSiY4/1iLqMmz5ADfzY69hB2Hmjv7yYPMZ8Q7lxUC2LzXyNjfOhl7wqBQ/q40nnib5VdsP4q/Vo= X-Received: by 2002:a62:9652:: with SMTP id c79-v6mr20885437pfe.114.1531131858418; Mon, 09 Jul 2018 03:24:18 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a17:90a:17af:0:0:0:0 with HTTP; Mon, 9 Jul 2018 03:24:17 -0700 (PDT) From: Steve Lhomme Date: Mon, 9 Jul 2018 12:24:17 +0200 Message-ID: To: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: [Cellar] AV1 mapping update X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jul 2018 10:24:21 -0000 I updated the AV1 mapping to clean a few sentences. Since we allow stripping the Sequence Header OBU from the stream when it's equal to the CodecPrivate one, we need to add it back to the bitstream for compliance. At least when seeking on keyframes. So I added a section to explain that. IMO that's an extra feature of the CodecPrivate that it's meant to be added to the bistream as-is. And in this case on startup and when seeking. I wonder if we should add an element next to the CodecPrivate to describe that. Because in this case it's not entirely opaque to the demuxer. Or maybe it's implied by the CodecID and is up to the decoder to use it how it's supposed to be (in this case detecting keyframes and possibly adding back the Sequence Header OBU). https://github.com/Matroska-Org/matroska-specification/blob/av1-mappin/codec/av1.md and the list of changes can be found here https://github.com/Matroska-Org/matroska-specification/commits/av1-mappin/codec/av1.md Let me know what you think so we can settle this spec for good. -- Steve Lhomme Matroska association Chairman From nobody Wed Jul 11 06:48:24 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 43F18130DC9 for ; Wed, 11 Jul 2018 06:48:17 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=googlemail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qSC5XIH8DNa9 for ; Wed, 11 Jul 2018 06:48:13 -0700 (PDT) Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1F7BC130E19 for ; Wed, 11 Jul 2018 06:48:13 -0700 (PDT) Received: by mail-wr1-x42f.google.com with SMTP id b15-v6so18280305wrv.10 for ; Wed, 11 Jul 2018 06:48:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20161025; h=subject:to:references:from:message-id:date:mime-version:in-reply-to :content-transfer-encoding; bh=M1ETHoJD99r24YvH9zYnG7QAEeUXb0ON7oKZMe1OYok=; b=dQ03co0KTsEmMlkd8Fd3OOgdn/R9GHsmiLtK7DdIKd060zom4gWzDcsdQZYDjVqxjQ bfYLmCNY4LnRjbCp5nXQyIACjux2d8Yux15cejpYh183P7D5Iv+0ifgPmvrk9k79/7oP 6+If4Dq7Odv7tvleS0z210hcJLFX1sHybWzo1xuNLqrFPTbFzmHRkt2RfqWeRQv4pn8E AHKrCRx4357JLbr3qVFTHYvzWbGCAT+Owec4Up2HCLGNSlTOnyZpgGvwgqVzNiNVJGqC aN7gRfKnjA31PzWKSEMJx1ZUAuenXpNBKmxTA68S4DL1wLPv9k7erknnV3yHOKMtEl7r W4TA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :mime-version:in-reply-to:content-transfer-encoding; bh=M1ETHoJD99r24YvH9zYnG7QAEeUXb0ON7oKZMe1OYok=; b=FNYXct0djluQYEuKgwRj/zFKm2o7T++SR8Hu9jwJAZZB59FaaSIcC+ne6LPbSeSXQd qywPHB1/PYrTOXFSdpvK5hzsPHVy71QzI1IY1GiWxYt9+yClXiAjiG49PBqY3xasc52i a5KnUHwNCX58m2eAB4YZAWMl199DS5N6ve7SPOB2AxPL0BtuB/Fu1V701ucFoQmjAQ3u 8yU5pUR69rmXlfEbGNHEdu2Jyga/aeRu0GAeh4aWDlGYqBoOgXwniznXxqdIXnBPqxeA 4J3NiVbp8xmMdUMfCRqd4EFKlKROAP02amJS6wQfpdnzIqjbVtmGrR4Rqn9Fx9IHlMnZ TJZw== X-Gm-Message-State: APt69E0prjOY0UVG4pFe4WJbbOrlZOGoQnuOc0o1NK0UEGgY19m59zYe x5ufRmf/Qm9+q7PICci+qEykAd4M X-Google-Smtp-Source: AAOMgpehknHN3MOaSfTBxFdE8Cyy8r2iLjgPuMLfMCERkiupHeNaoqyUr4qQfSk33rlrJ7zdr1Hv9A== X-Received: by 2002:adf:ac66:: with SMTP id v93-v6mr20251637wrc.7.1531316891238; Wed, 11 Jul 2018 06:48:11 -0700 (PDT) Received: from [127.0.0.1] ([31.131.2.19]) by smtp.googlemail.com with ESMTPSA id y203-v6sm3321849wme.42.2018.07.11.06.48.09 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Jul 2018 06:48:10 -0700 (PDT) To: cellar@ietf.org References: From: Andreas Rheinhardt Message-ID: Date: Wed, 11 Jul 2018 13:47:00 +0000 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [Cellar] AV1 mapping update X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jul 2018 13:48:18 -0000 Steve Lhomme: > I updated the AV1 mapping to clean a few sentences. > > > https://github.com/Matroska-Org/matroska-specification/blob/av1-mappin/codec/av1.md > and the list of changes can be found here > https://github.com/Matroska-Org/matroska-specification/commits/av1-mappin/codec/av1.md > 1. Whether `DisplayWidth` and `DisplayHeight` needs to be written actually depends on the value of `DisplayUnit`. 2. You forgot `OBU_PADDING` in the list of OBU types that mustn't be in the `CodecPrivate`. (Either that or your sentence that only `OBU_SEQUENCE_HEADER` and `OBU_METADATA` are currently allowed in the `CodecPrivate` should be changed.) 3. "They SHOULD have the [obu_has_size_field] set to 1 except for the last OBU in the sample, for which [obu_has_size_field] MAY be set to 0, in which case it is assumed to fill the remaining of the sample." "The OBUs in the Block MUST follow the [Low Overhead Bitstream Format syntax]." The first sentence leaves the possibility that [obu_has_size_field] is 0 for OBUs other than the last OBU of a block (only a SHOULD). And the requirement in the second sentence actually makes MUST out of the SHOULD in the first sentence (making this part of the first sentence redundant) and contradicts/voids the MAY part of the first sentence. In other words, the two sentences should be merged to something like: "The `OBUs` in the block must follow the `Low Overhead Bitstream Format` (in which [obu_has_size_field] MUST be equal to one for every OBU) for every `OBU` with the possible exception of the very last `OBU` in which [obu_has_size_field] MAY be set to 0, in which case the `OBU` is assumed to consist of the remainder of the block." 4. "ReferenceBlocks inside a BlockGroup MUST reference frames according to the [ref_frame_idx] values of frame that is neither a KEYFRAME nor an INTRA_ONLY_FRAME.": The problem with this sentence is that [ref_frame_idx] needn't be present. It depends upon [frame_refs_short_signaling] and [show_existing_frame]. If one uses a Block inside a Blockgroup and if [show_exsting_frame] equals one one should reference the block that contained the showable frame that is now output (and that this should be the only `ReferenceBlock` written). In case of [frame_refs_short_signaling] == 1 the obvious candidates for `ReferenceBlocks` are the blocks containing the `last_frame_idx` and `gold_frame_idx` that are explicitly signalled. If I am not mistaken, then there are also other reference frames that are not explicitly signalled, but computed. I don't know if we should really write a `ReferenceBlock` entry for every reference as the current proposal seems to imply. This would be quite a bit of overhead for no gain (and furthermore, it would complicate muxers that would have to compute the references that are not explicitly signalled in case that [frame_refs_short_signaling] is 1). One `ReferenceBlock` would be enough to distinguish keyframes from non-keyframes. By the way: If a temporal unit contains multiple frames with references, whose references should end up as `ReferenceBlocks`? Or may the muxer choose some? 5. AV1 may use spatial scalability and/or temporal scalability. What do we make of these? They are currently not forbidden if I am not mistaken, but if e.g. the spatial dimensions of different layers disagree, the `PixelWidth` and `PixelHeight` values can't be true for all layers. Matroska seems to be missing some features here. 6. Depending on [frame_size_override_flag] there is even the possibility that the size of the frames differs even without scalability (if I am not mistaken). Should this be allowed? 7. Then there is another thing with keyframes and cues (for this point it is always presumed that the relevant sequence header OBUs are available regardless of whether this is done in-band or via CodecPrivate): a) The proposal currently does not take into account that key frames reset the decoder when they are output, not when they are decoded. A key frame needn't be immediately output; if it is (i.e. [show_frame] equaling 1), it is called a "key frame random access point" in section 7.6 of the standard and is the equivalent of an IDR frame in H.264. Everything's fine here. But a key frame can also be declared a showable_frame (but only if [show_frame] equals 0) and output later via the show_existing_frame mechanism. This is similar to an open GOP in other codecs (but in contrast to them, the block that contains the coded keyframe doesn't have the same timestamp (pts) as the first frame that can be output after a seek). The coded key frame with [show_frame] equal to zero is called a delayed random access point and a key frame dependent recovery point is a frame where a key frame with [showable_frame] equal to 1 is output via the show_existing_frame mechanism. If one starts decoding at the delayed random access point, all the output frames up to but not including the key frame dependent recovery point can depend both on the delayed random access point frame and on other earlier frames so that these frames can't be correctly decoded in general. But all the frames from the key frame dependent recovery point onwards can be correctly decoded if one starts decoding at the delayed random access point (because the decoder is reset after displaying the key frame). If one starts decoding at the key frame dependent recovery point, one doesn't have the key frame that should be shown via the show_existing_frames mechanism at all, so that this frame is simply not a real key frame. b) But although a key frame dependent recovery point is not a "real" key frame, it has the same [frame_type] as the frame that is output, i.e. its [frame_type] is KEY_FRAME. According to our current proposal this would mean that it should be treated as a keyframe in Matroska which is obviously wrong. c) Marking a delayed random access point as keyframe deviates from the way that flag has been traditionally understood: If one starts decoding at this point, one doesn't get the frame that should be output for the temporal unit containing the delayed random access point. But I nevertheless think that these are the right keyframes, because they are the points at which random access has to begin when there aren't key frame random access points available; this also means that one can split the stream at this point and the second part will still play so that a muxer like mkvmerge needn't be rewritten too much. d) A consequence of this is that a `Blockgroup` containing a delayed random access point mustn't contain a `ReferenceBlock` (although the actual frame that is output for that temporal unit very likely uses other reference frames than the key frame that is contained in the same temporal unit). e) Yes, this proposal means that it is impossible to tell from Matroska alone (well, from the block structure that is; see f) for a way for which one could put this information into the Cues) whether it is a key frame random access point or a delayed random access point. One will have to decode it (or parse deeper) to know. f) This also leads to problems with seeking: If one simply added a CuePoint for the keyframe (i.e. for the delayed random access point) and the user wants to seek to a point between the delayed random access point (inclusive) and the dependent recovery point (exclusive) and the player used the cues to seek to the nearest keyframe in front of the desired point, then decoding at the point referenced in the cues would not yield the desired frame (it would be either corrupted or not output at all). Therefore I think it is best to add a CuePoint for every key frame random access point and every key frame dependent recovery point. The CuePoint for the key frame random access point would be an ordinary CuePoint as usual. But the CuePoint for the key frame dependent recovery point wouldn't be (my favourite is iv) (and if I were allowed to play God it would be i))): i) A comprehensive way of doing it is this: The CueTime would be the timestamp of the block containing the dependent recovery point; it would include a CueTrackPositions for the video track we are talking about that contains the right CueTrack, the CueClusterPosition containing the position of the dependent recovery point block and a CueReference with CueRefTime and CueRefCluster, both corresponding to the valus of the delayed random access point. This proposal has several downsides: It uses Cue elements that are deprecated in Matroska and not part of Webm. So this would require a quite nontrivial change in both projects. (Btw: If one does this, one should add a default value for `CueRefCluster`: It should be the same as `CueClusterPosition` as both blocks that we are talking about will probably end up in the same cluster anyway.) ii) One uses the CueTime of the dependent recovery point, but the position of the Cluster of the delayed access point (and `CueRelativePosition` (if used) should also point to this block). Pro: It only uses elements that are supported by both Matroska and Webm. Furthermore, the specs only say that `CueClusterPosition` should point to the cluster containing the "required block; they don't explicitly say that said block needs to have the same timestamp as `CueTime`. Contra: How does a demuxer know from which block onwards it should feed the data to the decoder? It might use the `CueRelativePosition`, but probably a lot of demuxers would simply read the cluster until they come to the block with timestamp `CueTime` (i.e. they interpret the specs so that the "required block" is the block with the timestamp `CueTime`) and then they would either deliver this to the decoder or conclude that the file is damaged (because the block they found is no keyframe). iii) The last is the same as i) with the difference that `CueRefCluster` is omitted. It is also incompatible with current Webm, but at least it has the advantage that it doesn't use any currently deprecated elements of Matroska. One could add a requirement that the delayed random access point and the dependent recovery point need to be in the same cluster and then omitting `CueRefCluster` is not a problem any more. iv) And then there is the possibility of creating a normal CuePoint for the dependent recovery point, writing the dependent recovery point as a Block in a Blockgroup with exactly one ReferenceBlock which points to the delayed access point block and let the demuxer seek backwards from the dependent recovery point to the delayed access point. Pro: Would only use things that are already supported by Matroska and Webm. It would also not be AV1 specific. The demuxer doesn't need to know anything about AV1, everything is signalled at the container level. Contra: Demuxers would have to be adapted not to expect any more that only keyframes are referenced in the cues. They would also have to be adapted to actually make use of the value of `ReferenceBlock` and seek backwards. This also implies more seeks, but this should be quite limited when one puts both the delayed random access point and the dependent recovery point in the same cluster -- hopefully the data is still cached. (Maybe one should add a SHOULD clause that says that both blocks should be in the same cluster.) g) Of course there are two easy alternative solutions: i) Restrict the type of AV1 that is allowed in Matroska even further so that all key frames are of key frame random access type. (This could exclude quite a lot of AV1 and therefore I recommend not doing so.) ii) Create cues as usual, i.e. reference every delayed random access point, and don't care about the fact that seeking will be partially broken in this case. h) It should be noted that exactly the same situation exists with periodic intra refresh in general. There was a short discussion on the Matroska developer mailing list in April 2011, but nothing came out of it. Every solution I outlined here for AV1 is also applicable for this case. Steve Lhomme: > Since we allow stripping the Sequence Header OBU from the stream when > it's equal to the CodecPrivate one, we need to add it back to the > bitstream for compliance. At least when seeking on keyframes. So I > added a section to explain that. > > IMO that's an extra feature of the CodecPrivate that it's meant to be > added to the bistream as-is. And in this case on startup and when > seeking. I wonder if we should add an element next to the CodecPrivate > to describe that. Because in this case it's not entirely opaque to the > demuxer. Or maybe it's implied by the CodecID and is up to the decoder > to use it how it's supposed to be (in this case detecting keyframes > and possibly adding back the Sequence Header OBU). 8. I think we can relax the requirements on the existence of in-band sequence header OBUs a bit: If a keyframe (i.e. a key random access point or a delayed random access point, not a dependent recovery point) uses the same sequence header OBU as in the CodecPrivate (including the same operating_parameters_info), then the sequence header OBU needn't be prepended to the block with the keyframe, because seeking already works without it provided one always adds the sequence header OBU from the CodecPrivate back in the bitstream on seeking. For example, consider the following scenario: One has an elementary stream that uses two different sequence header OBUs A and B that only differ in the operating_parameters_info. The first three keyframes use A, between the third and the B is contained in a temporal unit between the third and the fourth keyframe. Between the sixth and the seventh keyframe is a temporal unit containing sequence header A again. Then a muxer that wants to put this elementary stream into Matroska may put A in the CodecPrivate can strip the very first occurence of A away; it must leave B inside the temporal unit that it was in (so that a player that plays the file linearly is notified about the change) and has to make sure that keyframes #4 to #6 contain sequence header B (so that one has the correct sequence header when seeking to said keyframes). It mustn't strip A between the sixth and the seventh keyframe away (so that a player that plays the file linearly notices the change of sequence header), but it needn't preprend keyframes #7 and following with A. That way one can save a few bytes. This is consistent with an interpretation of the CodecPrivate as the default extradata/header (but it is not necessarily a truly global header). Before `CueCodecState` was deprecated it had a default value of 0 that mandated that one should look in the CodecPrivate for the CodecState upon seking. So the only thing specific to AV1 in this case is the "as-is" part; that one should reset the decoder to whatever initialization information is contained in the CodecPrivate upon seeking is nothing new. 9. Image that future AV1 encoders would find out that changing the sequence header (with a change of the CVS) enables better compression. What would we do in this case? Simply lift the restriction of one track = one CVS even when this means that some players that can't cope with changing coded video sequences won't know in advance whether they can play the files at all? I'm asking because we don't include a version field in the CodecPrivate (in contrast to how it is done in mp4 with avcC). Steve Lhomme: > Let me know what you think so we can settle this spec for good. > I think we are not even close to settling this for good. - Andreas Rheinhardt From nobody Thu Jul 12 02:54:37 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 58B7B1310AE for ; Thu, 12 Jul 2018 02:54:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.909 X-Spam-Level: X-Spam-Status: No, score=-1.909 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, T_DKIMWL_WL_MED=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3fiFiucmlSiq for ; Thu, 12 Jul 2018 02:54:30 -0700 (PDT) Received: from mail-pf0-x22f.google.com (mail-pf0-x22f.google.com [IPv6:2607:f8b0:400e:c00::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C5D51130DC0 for ; Thu, 12 Jul 2018 02:54:30 -0700 (PDT) Received: by mail-pf0-x22f.google.com with SMTP id v15-v6so280364pff.5 for ; Thu, 12 Jul 2018 02:54:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=4SmP/4UmnGseQCwn7T2Zzj2E8wyHxgPnnix6mMWb+Ug=; b=nZYGsBP6PBCQbX9TiDyldQzhpM2kaL6PdX8y/gnJWWj80d6wKtSLKI9U954QnLiZyD ITgYolStvs4/XUS4aY3ZAS0mPJY6inO2fCNQZfg6WY2vhPLVBRfRMY8f5ixavcvM/hFm o4v34Sk7NAEHS2/iqdSTBabja+ZQ6HNDD4N0/ASJqzR85zas+RQj0epHiOFCTW6li85I W39+E1kxGeL31xjuN0b86Sg676D+cg0ML9xKtrmT22Ntxu2w2OPT24wt6TGsIR8ZZHO5 /qZArQg9rLs2ZagV6ajvjS1ymBO8ODkh0WIKVSGNUZF4LBiMS/8g7h6cKwq2pjOleL4V NiKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=4SmP/4UmnGseQCwn7T2Zzj2E8wyHxgPnnix6mMWb+Ug=; b=ExvX6VlD/Yz/mgftAgAuJly1x/40o0ay5jtQp6770oJAOgiz1rgMV5iU7EaEVU/IYf 6sGSYl67fRyLJHY5dWr1f/eEt67vFpM0hOTFbb+nzetKzT5aSRD+7T/yCZdrFp8yp89a 3jVU6xmfKvX/b8RboaIP13QPKRzWBc7faFSMYa3wB5SZ15q08JgOS7p8e6NfFpxyc/lW LvqACTMOqg/S5AOO9unAIVG0jiRa0q/vSM6tUOhNeT4CqtTYkVbSrYqL/S+P8J+ps0IT 6FcW6pva151eMOWp98MNBS3KZHNa4OF7Ltyw2a+5zJgklLmMybyxd8/xZQBygdTo6bHm qn9Q== X-Gm-Message-State: AOUpUlEQkh/zFDr2L7tAFcM6YJdWQQTF6Gkg9rCxt01nxwi94uvoggsk EWBE4uNfjhomJhi70DCdgVlJ5EThpLvO3PY2EhKc8LCv X-Google-Smtp-Source: AAOMgpc6Nvtn4AqAeP8DU5SE65m6VE0ZE1Y+2LL3VTT8rjHFl+illNu9xTS+QPLRK5mFVo+jRNZ1GsuBTv64NcUG1nY= X-Received: by 2002:a62:8d84:: with SMTP id p4-v6mr1642725pfk.251.1531389270081; Thu, 12 Jul 2018 02:54:30 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a17:90a:17af:0:0:0:0 with HTTP; Thu, 12 Jul 2018 02:54:29 -0700 (PDT) In-Reply-To: References: From: Steve Lhomme Date: Thu, 12 Jul 2018 11:54:29 +0200 Message-ID: To: Andreas Rheinhardt Cc: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Archived-At: Subject: Re: [Cellar] AV1 mapping update X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jul 2018 09:54:36 -0000 Hi Andreas, Thanks for your detailed feedback. 2018-07-11 15:47 GMT+02:00 Andreas Rheinhardt : > Steve Lhomme: >> I updated the AV1 mapping to clean a few sentences. >> >> >> https://github.com/Matroska-Org/matroska-specification/blob/av1-mappin/c= odec/av1.md >> and the list of changes can be found here >> https://github.com/Matroska-Org/matroska-specification/commits/av1-mappi= n/codec/av1.md >> > 1. Whether `DisplayWidth` and `DisplayHeight` needs to be written > actually depends on the value of `DisplayUnit`. True, I will mention that. > 2. You forgot `OBU_PADDING` in the list of OBU types that mustn't be in > the `CodecPrivate`. (Either that or your sentence that only > `OBU_SEQUENCE_HEADER` and `OBU_METADATA` are currently allowed in the > `CodecPrivate` should be changed.) Indeed. The MP4 is a bit fuzzy on what is allowed. It's Sequence Header and Metadata only if they apply to all samples. But anything that is is valid to put before a sync point is OK. But in our case it's better to be more strict. Remuxing from MP4 might require some cleaning... > 3. "They SHOULD have the [obu_has_size_field] set to 1 except for the > last OBU in the sample, for which [obu_has_size_field] MAY be set to 0, > in which case it is assumed to fill the remaining of the sample." > "The OBUs in the Block MUST follow the [Low Overhead Bitstream Format > syntax]." > The first sentence leaves the possibility that [obu_has_size_field] is 0 > for OBUs other than the last OBU of a block (only a SHOULD). And the > requirement in the second sentence actually makes MUST out of the SHOULD > in the first sentence (making this part of the first sentence redundant) > and contradicts/voids the MAY part of the first sentence. In other > words, the two sentences should be merged to something like: "The `OBUs` > in the block must follow the `Low Overhead Bitstream Format` (in which > [obu_has_size_field] MUST be equal to one for every OBU) for every `OBU` > with the possible exception of the very last `OBU` in which > [obu_has_size_field] MAY be set to 0, in which case the `OBU` is assumed > to consist of the remainder of the block." Indeed there's a contradiction here. If we use MUST (can't be must lowercase) on [Low Overhead Bitstream Format] then the [obu_has_size_field] MUST be 1. On MP4 for the CodecPrivate the [obu_has_size_field] MUST be 1. But in the Blocks it can be 0: "Each OBU SHALL have the obu_has_size_field set to 1 except for the last OBU in the sample, for which obu_has_size_field MAY be set to 0, in which case it is assumed to fill the remaining of the sample" I think we should mimic that. I'll rephrase it. > 4. "ReferenceBlocks inside a BlockGroup MUST reference frames according > to the [ref_frame_idx] values of frame that is neither a KEYFRAME nor an > INTRA_ONLY_FRAME.": The problem with this sentence is that > [ref_frame_idx] needn't be present. It depends upon > [frame_refs_short_signaling] and [show_existing_frame]. If one uses a > Block inside a Blockgroup and if [show_exsting_frame] equals one one > should reference the block that contained the showable frame that is now > output (and that this should be the only `ReferenceBlock` written). In > case of [frame_refs_short_signaling] =3D=3D 1 the obvious candidates for > `ReferenceBlocks` are the blocks containing the `last_frame_idx` and > `gold_frame_idx` that are explicitly signalled. If I am not mistaken, > then there are also other reference frames that are not explicitly > signalled, but computed. I don't know if we should really write a > `ReferenceBlock` entry for every reference as the current proposal seems > to imply. This would be quite a bit of overhead for no gain (and > furthermore, it would complicate muxers that would have to compute the > references that are not explicitly signalled in case that This is how `ReferenceBlock` is supposed to be used. So a muxer that has no idea of any codec can cut a file and keep the relevant references. So they all have to be there. It's one of the reasons SimpleBlock was added, to simplify things a little (and reduce overhead). > [frame_refs_short_signaling] is 1). One `ReferenceBlock` would be enough > to distinguish keyframes from non-keyframes. > By the way: If a temporal unit contains multiple frames with references, > whose references should end up as `ReferenceBlocks`? Or may the muxer > choose some? I think a Temporal Unit can only have one (visible frame). I don't know if golden frames can have extra references. But the BlockGroup should contain all frames needed to decode this frame, that includes all the frames in the Block (even if not visible). > 5. AV1 may use spatial scalability and/or temporal scalability. What do > we make of these? They are currently not forbidden if I am not mistaken, > but if e.g. the spatial dimensions of different layers disagree, the > `PixelWidth` and `PixelHeight` values can't be true for all layers. > Matroska seems to be missing some features here. Our spec says that the Sequence Header OBU should be valid for all frames. That can't be used for spatial scalability. We don't support that mode for now. It may technically possible to add different sizes in BlockAddition. > 6. Depending on [frame_size_override_flag] there is even the possibility > that the size of the frames differs even without scalability (if I am > not mistaken). Should this be allowed? It's not restricted by our spec. But then it's up to the codec to handle, not the container. It's used with a SWITCH frame. The MP4 spec don't make any special case for that either. (please add spacing between your paragraph, it's hard to read this big block of text) > 7. Then there is another thing with keyframes and cues (for this point > it is always presumed that the relevant sequence header OBUs are > available regardless of whether this is done in-band or via CodecPrivate)= : > a) The proposal currently does not take into account that key frames > reset the decoder when they are output, not when they are decoded. A key > frame needn't be immediately output; if it is (i.e. [show_frame] > equaling 1), it is called a "key frame random access point" in section > 7.6 of the standard and is the equivalent of an IDR frame in H.264. > Everything's fine here. But a key frame can also be declared a > showable_frame (but only if [show_frame] equals 0) and output later via > the show_existing_frame mechanism. This is similar to an open GOP in > other codecs (but in contrast to them, the block that contains the coded > keyframe doesn't have the same timestamp (pts) as the first frame that > can be output after a seek). The coded key frame with [show_frame] equal > to zero is called a delayed random access point and a key frame > dependent recovery point is a frame where a key frame with > [showable_frame] equal to 1 is output via the show_existing_frame > mechanism. If one starts decoding at the delayed random access point, > all the output frames up to but not including the key frame dependent > recovery point can depend both on the delayed random access point frame > and on other earlier frames so that these frames can't be correctly > decoded in general. But all the frames from the key frame dependent > recovery point onwards can be correctly decoded if one starts decoding > at the delayed random access point (because the decoder is reset after > displaying the key frame). If one starts decoding at the key frame > dependent recovery point, one doesn't have the key frame that should be > shown via the show_existing_frames mechanism at all, so that this frame > is simply not a real key frame. > b) But although a key frame dependent recovery point is not a "real" key > frame, it has the same [frame_type] as the frame that is output, i.e. > its [frame_type] is KEY_FRAME. According to our current proposal this > would mean that it should be treated as a keyframe in Matroska which is > obviously wrong. That's not how I understand it. Here's section 7.6.3: "Informally, the requirement for decoder conformance is that decoding can start at any key frame random access point or delayed random access point." And 7.6.2: "delayed random access point is defined as being a frame: =E2=80=A2 with frame_type equal to KEY_FRAME =E2=80=A2 with show_frame equal to 0 =E2=80=A2 that is contained in a temporal unit that also contains a sequenc= e header OBU" So as long as we seek on frames of type KEY_FRAME we should be able to seek. Wether it's a visible frame or not. But because this is a bit loose in the 7.6.3 section they add this: "To support the different modes of operation, a conformant decoder is required to be able to decode bitstreams consisting of: =E2=80=A2 a temporal unit containing a delayed random access point =E2=80=A2 immediately followed by a temporal unit containing the associated key frame dependent recovery point" So the invisible KEY_FRAME should be immediately followed by the recovery point data. So effectively it will work. There's also a note that if it's not followed immediately then what is done with the intermediate frames is implementation dependent. I don't think we need to care too much. > c) Marking a delayed random access point as keyframe deviates from the > way that flag has been traditionally understood: If one starts decoding > at this point, one doesn't get the frame that should be output for the > temporal unit containing the delayed random access point. But I > nevertheless think that these are the right keyframes, because they are > the points at which random access has to begin when there aren't key > frame random access points available; this also means that one can split > the stream at this point and the second part will still play so that a > muxer like mkvmerge needn't be rewritten too much. Yes, IMO this is the correct way. > d) A consequence of this is that a `Blockgroup` containing a delayed > random access point mustn't contain a `ReferenceBlock` (although the > actual frame that is output for that temporal unit very likely uses > other reference frames than the key frame that is contained in the same > temporal unit). This won't happen if it's a proper random access point, ie it doesn't need past frames to start decoding. If it's not then it's not a RAP and then it can/should have ReferenceBlock. > e) Yes, this proposal means that it is impossible to tell from Matroska > alone (well, from the block structure that is; see f) for a way for > which one could put this information into the Cues) whether it is a key > frame random access point or a delayed random access point. One will > have to decode it (or parse deeper) to know. No, this is independent of the codec. Also Cues can target a frames that can't be seeked to directly but that's beside the point. S SimpleBlock marked keyframe or BlockGroup with no ReferenceBlock can be seeked to directly and that's why they equal Random Access Points as defined in AV1 (and other codecs). > f) This also leads to problems with seeking: If one simply added a > CuePoint for the keyframe (i.e. for the delayed random access point) and > the user wants to seek to a point between the delayed random access > point (inclusive) and the dependent recovery point (exclusive) and the > player used the cues to seek to the nearest keyframe in front of the > desired point, then decoding at the point referenced in the cues would > not yield the desired frame (it would be either corrupted or not output > at all). Therefore I think it is best to add a CuePoint for every key > frame random access point and every key frame dependent recovery point. > The CuePoint for the key frame random access point would be an ordinary > CuePoint as usual. But the CuePoint for the key frame dependent recovery > point wouldn't be (my favourite is iv) (and if I were allowed to play > God it would be i))): Cues are quite loose. It would be possible to do Cues only for frames that are not delayed RAP and that's valid. IMO it's fine to reference the delayed RAP. But they are RAP so it's legal to seek there. The AV1 specs have this to say about this tricky case: "Note:In practice, decoder implementations are expected to be able to start decoding bitstreams from a delayed random access point when the intermediate temporal units are still present. The decoder should correctly produce all output frames from the next key frame or key frame dependent recovery point onwards, while the preceding frames are implementation defined. For example: a streaming decoder may choose to decode and display all frames even when the reference frames are not available (tolerating some errors in the output), a low latency decoder may choose to decode and display all frames that are guaranteed to be correct (e.g. an inter frame that only uses inter prediction from the delayed random access point), a media player decoder may choose to decode and display only frames starting from a key frame or key frame dependent recovery point (guaranteeing smooth playback once display starts)." It's not up to the container to solve this. > i) A comprehensive way of doing it is this: The CueTime would be the > timestamp of the block containing the dependent recovery point; it would > include a CueTrackPositions for the video track we are talking about > that contains the right CueTrack, the CueClusterPosition containing the > position of the dependent recovery point block and a CueReference with > CueRefTime and CueRefCluster, both corresponding to the valus of the > delayed random access point. This proposal has several downsides: It > uses Cue elements that are deprecated in Matroska and not part of Webm. > So this would require a quite nontrivial change in both projects. (Btw: > If one does this, one should add a default value for `CueRefCluster`: It > should be the same as `CueClusterPosition` as both blocks that we are > talking about will probably end up in the same cluster anyway.) > ii) One uses the CueTime of the dependent recovery point, but the > position of the Cluster of the delayed access point (and > `CueRelativePosition` (if used) should also point to this block). > Pro: It only uses elements that are supported by both Matroska and Webm. > Furthermore, the specs only say that `CueClusterPosition` should point > to the cluster containing the "required block; they don't explicitly say > that said block needs to have the same timestamp as `CueTime`. > Contra: How does a demuxer know from which block onwards it should feed > the data to the decoder? It might use the `CueRelativePosition`, but > probably a lot of demuxers would simply read the cluster until they come > to the block with timestamp `CueTime` (i.e. they interpret the specs so > that the "required block" is the block with the timestamp `CueTime`) and > then they would either deliver this to the decoder or conclude that the > file is damaged (because the block they found is no keyframe). > iii) The last is the same as i) with the difference that `CueRefCluster` > is omitted. It is also incompatible with current Webm, but at least it > has the advantage that it doesn't use any currently deprecated elements > of Matroska. One could add a requirement that the delayed random access > point and the dependent recovery point need to be in the same cluster > and then omitting `CueRefCluster` is not a problem any more. > iv) And then there is the possibility of creating a normal CuePoint for > the dependent recovery point, writing the dependent recovery point as a > Block in a Blockgroup with exactly one ReferenceBlock which points to > the delayed access point block and let the demuxer seek backwards from > the dependent recovery point to the delayed access point. > Pro: Would only use things that are already supported by Matroska and > Webm. It would also not be AV1 specific. The demuxer doesn't need to > know anything about AV1, everything is signalled at the container level. > Contra: Demuxers would have to be adapted not to expect any more that > only keyframes are referenced in the cues. They would also have to be > adapted to actually make use of the value of `ReferenceBlock` and seek > backwards. This also implies more seeks, but this should be quite > limited when one puts both the delayed random access point and the > dependent recovery point in the same cluster -- hopefully the data is > still cached. (Maybe one should add a SHOULD clause that says that both > blocks should be in the same cluster.) > g) Of course there are two easy alternative solutions: > i) Restrict the type of AV1 that is allowed in Matroska even further so > that all key frames are of key frame random access type. (This could > exclude quite a lot of AV1 and therefore I recommend not doing so.) > ii) Create cues as usual, i.e. reference every delayed random access > point, and don't care about the fact that seeking will be partially > broken in this case. > h) It should be noted that exactly the same situation exists with > periodic intra refresh in general. There was a short discussion on the > Matroska developer mailing list in April 2011, but nothing came out of > it. Every solution I outlined here for AV1 is also applicable for this ca= se. > > Steve Lhomme: >> Since we allow stripping the Sequence Header OBU from the stream when >> it's equal to the CodecPrivate one, we need to add it back to the >> bitstream for compliance. At least when seeking on keyframes. So I >> added a section to explain that. >> >> IMO that's an extra feature of the CodecPrivate that it's meant to be >> added to the bistream as-is. And in this case on startup and when >> seeking. I wonder if we should add an element next to the CodecPrivate >> to describe that. Because in this case it's not entirely opaque to the >> demuxer. Or maybe it's implied by the CodecID and is up to the decoder >> to use it how it's supposed to be (in this case detecting keyframes >> and possibly adding back the Sequence Header OBU). > > 8. I think we can relax the requirements on the existence of in-band > sequence header OBUs a bit: If a keyframe (i.e. a key random access > point or a delayed random access point, not a dependent recovery point) > uses the same sequence header OBU as in the CodecPrivate (including the > same operating_parameters_info), then the sequence header OBU needn't be > prepended to the block with the keyframe, because seeking already works > without it provided one always adds the sequence header OBU from the > CodecPrivate back in the bitstream on seeking. For example, consider the > following scenario: > One has an elementary stream that uses two different sequence header > OBUs A and B that only differ in the operating_parameters_info. The > first three keyframes use A, between the third and the B is contained in > a temporal unit between the third and the fourth keyframe. Between the > sixth and the seventh keyframe is a temporal unit containing sequence > header A again. Then a muxer that wants to put this elementary stream > into Matroska may put A in the CodecPrivate can strip the very first > occurence of A away; it must leave B inside the temporal unit that it > was in (so that a player that plays the file linearly is notified about > the change) and has to make sure that keyframes #4 to #6 contain > sequence header B (so that one has the correct sequence header when > seeking to said keyframes). It mustn't strip A between the sixth and the > seventh keyframe away (so that a player that plays the file linearly > notices the change of sequence header), but it needn't preprend > keyframes #7 and following with A. That works but then we need to tell in the specs that following a Sequence Header MUST not be stripped if the previous Sequence Header was not bit identical to the one in CodecPrivate. IMO it's valid to output A and then B before the rest of the data in frame #4. That should be done if it's a keyframe. But if it's not a keyframe the demuxer/decoder doesn't know it has to prepend the CodecPrivate there. That could be a problem. Even though it doesn't make much sense to change Sequence Header data before a non keyframe (RAP). So maybe your proposal should be added. We'll gain a bit of weight but it's safer. > That way one can save a few bytes. > This is consistent with an interpretation of the CodecPrivate as the > default extradata/header (but it is not necessarily a truly global > header). Before `CueCodecState` was deprecated it had a default value of > 0 that mandated that one should look in the CodecPrivate for the > CodecState upon seking. So the only thing specific to AV1 in this case > is the "as-is" part; that one should reset the decoder to whatever > initialization information is contained in the CodecPrivate upon seeking > is nothing new. > > 9. Image that future AV1 encoders would find out that changing the > sequence header (with a change of the CVS) enables better compression. > What would we do in this case? Simply lift the restriction of one track > =3D one CVS even when this means that some players that can't cope with > changing coded video sequences won't know in advance whether they can > play the files at all? I'm asking because we don't include a version > field in the CodecPrivate (in contrast to how it is done in mp4 with avcC= ). Another CodecID should be used if the restrictions/format is defined. It's safer than reusing one with different data and hoping the system using the old one paid attention to the version bit that was always the same anyway. > Steve Lhomme: >> Let me know what you think so we can settle this spec for good. >> > I think we are not even close to settling this for good. \o/ > > - Andreas Rheinhardt > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar --=20 Steve Lhomme Matroska association Chairman From nobody Thu Jul 12 02:58:24 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0D28D130DEA for ; Thu, 12 Jul 2018 02:58:22 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.909 X-Spam-Level: X-Spam-Status: No, score=-1.909 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, T_DKIMWL_WL_MED=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WGIoSigGydch for ; Thu, 12 Jul 2018 02:58:18 -0700 (PDT) Received: from mail-pf0-x22a.google.com (mail-pf0-x22a.google.com [IPv6:2607:f8b0:400e:c00::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 940B6130DC0 for ; Thu, 12 Jul 2018 02:58:18 -0700 (PDT) Received: by mail-pf0-x22a.google.com with SMTP id c21-v6so15677523pfn.8 for ; Thu, 12 Jul 2018 02:58:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=IeGGEXGTqxzkZplZXFpd7QIqnWwVesT84NpzY1ofdfs=; b=x3SKvQJvsr9mD0SENifpfMVxfMXVwVN/BgiJFMZpHtVY0EoXuZG9hDlexw1Kw0rZsp Xijaf9ZYKSvSxlQmh6mkKleLqTo0CbLRiEKL5RxLnBIggYvX0dqOjLh2eqIn8ZXIjs9Q 0RlbQH5V3RWOYU/3Jty0OaIEFZPFrX3SyzvDkWnkTIdJR+W9CYtpPRZITkElzrZx83Rb zJCc/C25wruyIfd/j2TAQARqWPYlgD+ymvzH/N3DayUYzuWUVSvmlqFZoZtcvvDYJoPw IOzXaGNi9ABBF0vtJnr9jP2jIoZPlo7QyDjUZ2bCPuufxvg65HyvwgZF8gWNnJ6JG6MI +8hw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=IeGGEXGTqxzkZplZXFpd7QIqnWwVesT84NpzY1ofdfs=; b=ui4Whm7lTlOotB3TnDcbs936joltcS4p0e0+oZRfjY472LV7GOkGJF0Wthm9HDm0c1 XoQDMTY0w5vKjxZf1wJ3YSuyo/rBKN1l0bClhfQLHwZwCkhbcyOmUDkrovc/F7NLjIs3 Wua5UjTEts56UEXMYMrgnOy5b5ZghHnt0xd/xfEAkLOcZySV9EBICfrzV9nH/+3gfJIq UK2w1DgA9Qpj3cR/JRpBSnbcOdhLt7Vspr3bZEaghcEbE0TeKjy68+6bD9MwoutoyuOU QgWK0LiqDfYUrk1EKmWfrmHiTKvAAcNXyohxhFYMnvmo5a0nDY++oiBHAbUM7ppFzgFD aZQw== X-Gm-Message-State: AOUpUlE5UnXuNjcmkzklLpJYALeQybuTtZXVVr1ZTq6iZB6AyCbA4Td/ tUO/CyJTmo301Mhn4eE6EY/lAAW2IxZ0gj6yynuP4w== X-Google-Smtp-Source: AAOMgpcS2Ve1J8Gr+FIOVjLaQGWzwTSsfBCt+hT++4N5BWAF2Lkz2nKp9sltNH7IlOjS5fSMgNGyNX4+p665EC8Hc6E= X-Received: by 2002:a62:384:: with SMTP id 126-v6mr1641307pfd.11.1531389498033; Thu, 12 Jul 2018 02:58:18 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a17:90a:17af:0:0:0:0 with HTTP; Thu, 12 Jul 2018 02:58:17 -0700 (PDT) In-Reply-To: References: From: Steve Lhomme Date: Thu, 12 Jul 2018 11:58:17 +0200 Message-ID: To: Andreas Rheinhardt Cc: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Archived-At: Subject: Re: [Cellar] AV1 mapping update X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jul 2018 09:58:23 -0000 New update after the points from Andreas were addressed: https://github.com/Matroska-Org/matroska-specification/blob/av1-mappin/code= c/av1.md and the list of changes can be found here https://github.com/Matroska-Org/matroska-specification/commits/av1-mappin/c= odec/av1.md 2018-07-12 11:54 GMT+02:00 Steve Lhomme : > Hi Andreas, > > Thanks for your detailed feedback. > > 2018-07-11 15:47 GMT+02:00 Andreas Rheinhardt > : >> Steve Lhomme: >>> I updated the AV1 mapping to clean a few sentences. >>> >>> >>> https://github.com/Matroska-Org/matroska-specification/blob/av1-mappin/= codec/av1.md >>> and the list of changes can be found here >>> https://github.com/Matroska-Org/matroska-specification/commits/av1-mapp= in/codec/av1.md >>> >> 1. Whether `DisplayWidth` and `DisplayHeight` needs to be written >> actually depends on the value of `DisplayUnit`. > > True, I will mention that. > >> 2. You forgot `OBU_PADDING` in the list of OBU types that mustn't be in >> the `CodecPrivate`. (Either that or your sentence that only >> `OBU_SEQUENCE_HEADER` and `OBU_METADATA` are currently allowed in the >> `CodecPrivate` should be changed.) > > Indeed. The MP4 is a bit fuzzy on what is allowed. It's Sequence > Header and Metadata only if they apply to all samples. But anything > that is is valid to put before a sync point is OK. But in our case > it's better to be more strict. Remuxing from MP4 might require some > cleaning... > >> 3. "They SHOULD have the [obu_has_size_field] set to 1 except for the >> last OBU in the sample, for which [obu_has_size_field] MAY be set to 0, >> in which case it is assumed to fill the remaining of the sample." >> "The OBUs in the Block MUST follow the [Low Overhead Bitstream Format >> syntax]." >> The first sentence leaves the possibility that [obu_has_size_field] is 0 >> for OBUs other than the last OBU of a block (only a SHOULD). And the >> requirement in the second sentence actually makes MUST out of the SHOULD >> in the first sentence (making this part of the first sentence redundant) >> and contradicts/voids the MAY part of the first sentence. In other >> words, the two sentences should be merged to something like: "The `OBUs` >> in the block must follow the `Low Overhead Bitstream Format` (in which >> [obu_has_size_field] MUST be equal to one for every OBU) for every `OBU` >> with the possible exception of the very last `OBU` in which >> [obu_has_size_field] MAY be set to 0, in which case the `OBU` is assumed >> to consist of the remainder of the block." > > Indeed there's a contradiction here. If we use MUST (can't be must > lowercase) on [Low Overhead Bitstream Format] then the > [obu_has_size_field] MUST be 1. > > On MP4 for the CodecPrivate the [obu_has_size_field] MUST be 1. But in > the Blocks it can be 0: > > "Each OBU SHALL have the obu_has_size_field set to 1 except for the > last OBU in the sample, for which obu_has_size_field MAY be set to 0, > in which case it is assumed to fill the remaining of the sample" > > I think we should mimic that. I'll rephrase it. > >> 4. "ReferenceBlocks inside a BlockGroup MUST reference frames according >> to the [ref_frame_idx] values of frame that is neither a KEYFRAME nor an >> INTRA_ONLY_FRAME.": The problem with this sentence is that >> [ref_frame_idx] needn't be present. It depends upon >> [frame_refs_short_signaling] and [show_existing_frame]. If one uses a >> Block inside a Blockgroup and if [show_exsting_frame] equals one one >> should reference the block that contained the showable frame that is now >> output (and that this should be the only `ReferenceBlock` written). In >> case of [frame_refs_short_signaling] =3D=3D 1 the obvious candidates for >> `ReferenceBlocks` are the blocks containing the `last_frame_idx` and >> `gold_frame_idx` that are explicitly signalled. If I am not mistaken, >> then there are also other reference frames that are not explicitly >> signalled, but computed. I don't know if we should really write a >> `ReferenceBlock` entry for every reference as the current proposal seems >> to imply. This would be quite a bit of overhead for no gain (and >> furthermore, it would complicate muxers that would have to compute the >> references that are not explicitly signalled in case that > > This is how `ReferenceBlock` is supposed to be used. So a muxer that > has no idea of any codec can cut a file and keep the relevant > references. So they all have to be there. It's one of the reasons > SimpleBlock was added, to simplify things a little (and reduce > overhead). > >> [frame_refs_short_signaling] is 1). One `ReferenceBlock` would be enough >> to distinguish keyframes from non-keyframes. >> By the way: If a temporal unit contains multiple frames with references, >> whose references should end up as `ReferenceBlocks`? Or may the muxer >> choose some? > > I think a Temporal Unit can only have one (visible frame). I don't > know if golden frames can have extra references. But the BlockGroup > should contain all frames needed to decode this frame, that includes > all the frames in the Block (even if not visible). > >> 5. AV1 may use spatial scalability and/or temporal scalability. What do >> we make of these? They are currently not forbidden if I am not mistaken, >> but if e.g. the spatial dimensions of different layers disagree, the >> `PixelWidth` and `PixelHeight` values can't be true for all layers. >> Matroska seems to be missing some features here. > > Our spec says that the Sequence Header OBU should be valid for all > frames. That can't be used for spatial scalability. We don't support > that mode for now. > > It may technically possible to add different sizes in BlockAddition. > >> 6. Depending on [frame_size_override_flag] there is even the possibility >> that the size of the frames differs even without scalability (if I am >> not mistaken). Should this be allowed? > > It's not restricted by our spec. But then it's up to the codec to > handle, not the container. It's used with a SWITCH frame. The MP4 spec > don't make any special case for that either. > > (please add spacing between your paragraph, it's hard to read this big > block of text) >> 7. Then there is another thing with keyframes and cues (for this point >> it is always presumed that the relevant sequence header OBUs are >> available regardless of whether this is done in-band or via CodecPrivate= ): >> a) The proposal currently does not take into account that key frames >> reset the decoder when they are output, not when they are decoded. A key >> frame needn't be immediately output; if it is (i.e. [show_frame] >> equaling 1), it is called a "key frame random access point" in section >> 7.6 of the standard and is the equivalent of an IDR frame in H.264. >> Everything's fine here. But a key frame can also be declared a >> showable_frame (but only if [show_frame] equals 0) and output later via >> the show_existing_frame mechanism. This is similar to an open GOP in >> other codecs (but in contrast to them, the block that contains the coded >> keyframe doesn't have the same timestamp (pts) as the first frame that >> can be output after a seek). The coded key frame with [show_frame] equal >> to zero is called a delayed random access point and a key frame >> dependent recovery point is a frame where a key frame with >> [showable_frame] equal to 1 is output via the show_existing_frame >> mechanism. If one starts decoding at the delayed random access point, >> all the output frames up to but not including the key frame dependent >> recovery point can depend both on the delayed random access point frame >> and on other earlier frames so that these frames can't be correctly >> decoded in general. But all the frames from the key frame dependent >> recovery point onwards can be correctly decoded if one starts decoding >> at the delayed random access point (because the decoder is reset after >> displaying the key frame). If one starts decoding at the key frame >> dependent recovery point, one doesn't have the key frame that should be >> shown via the show_existing_frames mechanism at all, so that this frame >> is simply not a real key frame. >> b) But although a key frame dependent recovery point is not a "real" key >> frame, it has the same [frame_type] as the frame that is output, i.e. >> its [frame_type] is KEY_FRAME. According to our current proposal this >> would mean that it should be treated as a keyframe in Matroska which is >> obviously wrong. > > That's not how I understand it. Here's section 7.6.3: > > "Informally, the requirement for decoder conformance is that decoding > can start at any key frame random access point or delayed random > access point." > > And 7.6.2: > > "delayed random access point is defined as being a frame: > =E2=80=A2 with frame_type equal to KEY_FRAME > =E2=80=A2 with show_frame equal to 0 > =E2=80=A2 that is contained in a temporal unit that also contains a seque= nce header OBU" > > So as long as we seek on frames of type KEY_FRAME we should be able to > seek. Wether it's a visible frame or not. > > But because this is a bit loose in the 7.6.3 section they add this: > > "To support the different modes of operation, a conformant decoder is > required to be able to decode bitstreams consisting of: > =E2=80=A2 a temporal unit containing a delayed random access point > =E2=80=A2 immediately followed by a temporal unit containing the associat= ed > key frame dependent recovery point" > > So the invisible KEY_FRAME should be immediately followed by the > recovery point data. So effectively it will work. There's also a note > that if it's not followed immediately then what is done with the > intermediate frames is implementation dependent. I don't think we need > to care too much. > > >> c) Marking a delayed random access point as keyframe deviates from the >> way that flag has been traditionally understood: If one starts decoding >> at this point, one doesn't get the frame that should be output for the >> temporal unit containing the delayed random access point. But I >> nevertheless think that these are the right keyframes, because they are >> the points at which random access has to begin when there aren't key >> frame random access points available; this also means that one can split >> the stream at this point and the second part will still play so that a >> muxer like mkvmerge needn't be rewritten too much. > > Yes, IMO this is the correct way. > >> d) A consequence of this is that a `Blockgroup` containing a delayed >> random access point mustn't contain a `ReferenceBlock` (although the >> actual frame that is output for that temporal unit very likely uses >> other reference frames than the key frame that is contained in the same >> temporal unit). > > This won't happen if it's a proper random access point, ie it doesn't > need past frames to start decoding. If it's not then it's not a RAP > and then it can/should have ReferenceBlock. > >> e) Yes, this proposal means that it is impossible to tell from Matroska >> alone (well, from the block structure that is; see f) for a way for >> which one could put this information into the Cues) whether it is a key >> frame random access point or a delayed random access point. One will >> have to decode it (or parse deeper) to know. > > No, this is independent of the codec. Also Cues can target a frames > that can't be seeked to directly but that's beside the point. S > SimpleBlock marked keyframe or BlockGroup with no ReferenceBlock can > be seeked to directly and that's why they equal Random Access Points > as defined in AV1 (and other codecs). > >> f) This also leads to problems with seeking: If one simply added a >> CuePoint for the keyframe (i.e. for the delayed random access point) and >> the user wants to seek to a point between the delayed random access >> point (inclusive) and the dependent recovery point (exclusive) and the >> player used the cues to seek to the nearest keyframe in front of the >> desired point, then decoding at the point referenced in the cues would >> not yield the desired frame (it would be either corrupted or not output >> at all). Therefore I think it is best to add a CuePoint for every key >> frame random access point and every key frame dependent recovery point. >> The CuePoint for the key frame random access point would be an ordinary >> CuePoint as usual. But the CuePoint for the key frame dependent recovery >> point wouldn't be (my favourite is iv) (and if I were allowed to play >> God it would be i))): > > Cues are quite loose. It would be possible to do Cues only for frames > that are not delayed RAP and that's valid. IMO it's fine to reference > the delayed RAP. But they are RAP so it's legal to seek there. > > The AV1 specs have this to say about this tricky case: > > "Note:In practice, decoder implementations are expected to be able to > start decoding bitstreams from a delayed random access point when the > intermediate temporal units are still present. The decoder should > correctly produce all output frames from the next key frame or key > frame dependent recovery point onwards, while the preceding frames are > implementation defined. For example: a streaming decoder may choose to > decode and display all frames even when the reference frames are not > available (tolerating some errors in the output), a low latency > decoder may choose to decode and display all frames that are > guaranteed to be correct (e.g. an inter frame that only uses inter > prediction from the delayed random access point), a media player > decoder may choose to decode and display only frames starting from a > key frame or key frame dependent recovery point (guaranteeing smooth > playback once display starts)." > > It's not up to the container to solve this. > >> i) A comprehensive way of doing it is this: The CueTime would be the >> timestamp of the block containing the dependent recovery point; it would >> include a CueTrackPositions for the video track we are talking about >> that contains the right CueTrack, the CueClusterPosition containing the >> position of the dependent recovery point block and a CueReference with >> CueRefTime and CueRefCluster, both corresponding to the valus of the >> delayed random access point. This proposal has several downsides: It >> uses Cue elements that are deprecated in Matroska and not part of Webm. >> So this would require a quite nontrivial change in both projects. (Btw: >> If one does this, one should add a default value for `CueRefCluster`: It >> should be the same as `CueClusterPosition` as both blocks that we are >> talking about will probably end up in the same cluster anyway.) >> ii) One uses the CueTime of the dependent recovery point, but the >> position of the Cluster of the delayed access point (and >> `CueRelativePosition` (if used) should also point to this block). >> Pro: It only uses elements that are supported by both Matroska and Webm. >> Furthermore, the specs only say that `CueClusterPosition` should point >> to the cluster containing the "required block; they don't explicitly say >> that said block needs to have the same timestamp as `CueTime`. >> Contra: How does a demuxer know from which block onwards it should feed >> the data to the decoder? It might use the `CueRelativePosition`, but >> probably a lot of demuxers would simply read the cluster until they come >> to the block with timestamp `CueTime` (i.e. they interpret the specs so >> that the "required block" is the block with the timestamp `CueTime`) and >> then they would either deliver this to the decoder or conclude that the >> file is damaged (because the block they found is no keyframe). >> iii) The last is the same as i) with the difference that `CueRefCluster` >> is omitted. It is also incompatible with current Webm, but at least it >> has the advantage that it doesn't use any currently deprecated elements >> of Matroska. One could add a requirement that the delayed random access >> point and the dependent recovery point need to be in the same cluster >> and then omitting `CueRefCluster` is not a problem any more. >> iv) And then there is the possibility of creating a normal CuePoint for >> the dependent recovery point, writing the dependent recovery point as a >> Block in a Blockgroup with exactly one ReferenceBlock which points to >> the delayed access point block and let the demuxer seek backwards from >> the dependent recovery point to the delayed access point. >> Pro: Would only use things that are already supported by Matroska and >> Webm. It would also not be AV1 specific. The demuxer doesn't need to >> know anything about AV1, everything is signalled at the container level. >> Contra: Demuxers would have to be adapted not to expect any more that >> only keyframes are referenced in the cues. They would also have to be >> adapted to actually make use of the value of `ReferenceBlock` and seek >> backwards. This also implies more seeks, but this should be quite >> limited when one puts both the delayed random access point and the >> dependent recovery point in the same cluster -- hopefully the data is >> still cached. (Maybe one should add a SHOULD clause that says that both >> blocks should be in the same cluster.) >> g) Of course there are two easy alternative solutions: >> i) Restrict the type of AV1 that is allowed in Matroska even further so >> that all key frames are of key frame random access type. (This could >> exclude quite a lot of AV1 and therefore I recommend not doing so.) >> ii) Create cues as usual, i.e. reference every delayed random access >> point, and don't care about the fact that seeking will be partially >> broken in this case. >> h) It should be noted that exactly the same situation exists with >> periodic intra refresh in general. There was a short discussion on the >> Matroska developer mailing list in April 2011, but nothing came out of >> it. Every solution I outlined here for AV1 is also applicable for this c= ase. >> >> Steve Lhomme: >>> Since we allow stripping the Sequence Header OBU from the stream when >>> it's equal to the CodecPrivate one, we need to add it back to the >>> bitstream for compliance. At least when seeking on keyframes. So I >>> added a section to explain that. >>> >>> IMO that's an extra feature of the CodecPrivate that it's meant to be >>> added to the bistream as-is. And in this case on startup and when >>> seeking. I wonder if we should add an element next to the CodecPrivate >>> to describe that. Because in this case it's not entirely opaque to the >>> demuxer. Or maybe it's implied by the CodecID and is up to the decoder >>> to use it how it's supposed to be (in this case detecting keyframes >>> and possibly adding back the Sequence Header OBU). >> >> 8. I think we can relax the requirements on the existence of in-band >> sequence header OBUs a bit: If a keyframe (i.e. a key random access >> point or a delayed random access point, not a dependent recovery point) >> uses the same sequence header OBU as in the CodecPrivate (including the >> same operating_parameters_info), then the sequence header OBU needn't be >> prepended to the block with the keyframe, because seeking already works >> without it provided one always adds the sequence header OBU from the >> CodecPrivate back in the bitstream on seeking. For example, consider the >> following scenario: >> One has an elementary stream that uses two different sequence header >> OBUs A and B that only differ in the operating_parameters_info. The >> first three keyframes use A, between the third and the B is contained in >> a temporal unit between the third and the fourth keyframe. Between the >> sixth and the seventh keyframe is a temporal unit containing sequence >> header A again. Then a muxer that wants to put this elementary stream >> into Matroska may put A in the CodecPrivate can strip the very first >> occurence of A away; it must leave B inside the temporal unit that it >> was in (so that a player that plays the file linearly is notified about >> the change) and has to make sure that keyframes #4 to #6 contain >> sequence header B (so that one has the correct sequence header when >> seeking to said keyframes). It mustn't strip A between the sixth and the >> seventh keyframe away (so that a player that plays the file linearly >> notices the change of sequence header), but it needn't preprend >> keyframes #7 and following with A. > > That works but then we need to tell in the specs that following a > Sequence Header MUST not be stripped if the previous Sequence Header > was not bit identical to the one in CodecPrivate. > > IMO it's valid to output A and then B before the rest of the data in > frame #4. That should be done if it's a keyframe. But if it's not a > keyframe the demuxer/decoder doesn't know it has to prepend the > CodecPrivate there. That could be a problem. Even though it doesn't > make much sense to change Sequence Header data before a non keyframe > (RAP). > > So maybe your proposal should be added. We'll gain a bit of weight but > it's safer. > >> That way one can save a few bytes. >> This is consistent with an interpretation of the CodecPrivate as the >> default extradata/header (but it is not necessarily a truly global >> header). Before `CueCodecState` was deprecated it had a default value of >> 0 that mandated that one should look in the CodecPrivate for the >> CodecState upon seking. So the only thing specific to AV1 in this case >> is the "as-is" part; that one should reset the decoder to whatever >> initialization information is contained in the CodecPrivate upon seeking >> is nothing new. >> >> 9. Image that future AV1 encoders would find out that changing the >> sequence header (with a change of the CVS) enables better compression. >> What would we do in this case? Simply lift the restriction of one track >> =3D one CVS even when this means that some players that can't cope with >> changing coded video sequences won't know in advance whether they can >> play the files at all? I'm asking because we don't include a version >> field in the CodecPrivate (in contrast to how it is done in mp4 with avc= C). > > Another CodecID should be used if the restrictions/format is defined. > It's safer than reusing one with different data and hoping the system > using the old one paid attention to the version bit that was always > the same anyway. > >> Steve Lhomme: >>> Let me know what you think so we can settle this spec for good. >>> >> I think we are not even close to settling this for good. > > \o/ > >> >> - Andreas Rheinhardt >> >> _______________________________________________ >> Cellar mailing list >> Cellar@ietf.org >> https://www.ietf.org/mailman/listinfo/cellar > > > > -- > Steve Lhomme > Matroska association Chairman --=20 Steve Lhomme Matroska association Chairman From nobody Thu Jul 12 11:21:46 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 138D4131150 for ; Thu, 12 Jul 2018 11:21:45 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=googlemail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QJeXRk5Ela7a for ; Thu, 12 Jul 2018 11:21:41 -0700 (PDT) Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id F1E21130E63 for ; Thu, 12 Jul 2018 11:21:40 -0700 (PDT) Received: by mail-wr1-x435.google.com with SMTP id m1-v6so9922309wrg.5 for ; Thu, 12 Jul 2018 11:21:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20161025; h=subject:to:references:from:message-id:date:mime-version:in-reply-to :content-transfer-encoding; bh=01DASMNcW+Cglw9U2C5cXnQhKQmKUw4SHO4dhCsQZck=; b=X4ExU13oKVqS3id5ZaKRh2TJtW7jv1rRhuYfLcI3Uw0vRMjimiJdxAcEDcX3HH3l5Q NVJz8s7UhpSR5mLxQqIvTZzPObr68DX1/JTTx+fGGLPebOxguRairD1LMyHo3locD4t2 I3Ej63OpQxFr5Ig6heNLJlyBwwcKhGQaFyZwQvKuNhrR6LlHqleUkOcwyt9aqZAVLpD0 H6zw8w7d3/Wg73nGiISU/lkMgTjMHT5Fwyl6Gw6SlrM/jnYkbkK8GqsaFSdreXPtgDSt l0t+C9RnTuAMOE1TzoEI+2ujALOx818K2MJPQd0nPsnYkBouHcC8tCOBUBWVUo306/Td mkEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :mime-version:in-reply-to:content-transfer-encoding; bh=01DASMNcW+Cglw9U2C5cXnQhKQmKUw4SHO4dhCsQZck=; b=PyV1C09JNRCn35tzu7DOe5iPjN8fHrTR+Jl4Vspc0q88ErsbLRKpIGZopXNujwhDqC fUdhtLwmjnEIYxwkCiE1f+zWnXgDBEm0l2BfEF6x3d+CpYZRjPhsuUQx6dYBQFyLevcG fXmNfp0WasSfblmMEBAqOVn+mR/2xtTxBF9o5vCNKLAgIfJvkjOsBZFvXqpIJ5nZ7RNP X+3H3UZ/KZ1W6eB9jk0vRZHAkeAMDEE4KJkbVxe6u2QBSsBUg3w2654nbjRvh3IoDJ2y cRKs2uRLOJhX7UVXe7xNI2miHboz0okzNCboA24dGJIKtRjgyWOlZdRjHz2KVQT5UvAB lqSQ== X-Gm-Message-State: AOUpUlESl9B014DlN77XcjurWAGgjG8sGOuyRFgTGPM1Tu9gU1DxsnNy D1t/r5PoAHs/qBIVSumMg/TOW3ks X-Google-Smtp-Source: AAOMgpdmlPbeU2WPpkigeZiQctZzqIWyOMS9XKO+aFJ7yYO0K5bo/kPzhg0UCUiFAVW6jc1/BYHebA== X-Received: by 2002:adf:fc86:: with SMTP id g6-v6mr2531878wrr.216.1531419698973; Thu, 12 Jul 2018 11:21:38 -0700 (PDT) Received: from [127.0.0.1] (tor-exit-01.jelleschneiders.com. [145.239.90.27]) by smtp.googlemail.com with ESMTPSA id r125-v6sm4353655wmb.27.2018.07.12.11.21.37 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 12 Jul 2018 11:21:38 -0700 (PDT) To: Codec Encoding for LossLess Archiving and Realtime transmission References: From: Andreas Rheinhardt Message-ID: Date: Thu, 12 Jul 2018 18:20:00 +0000 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Archived-At: Subject: Re: [Cellar] AV1 mapping update X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jul 2018 18:21:45 -0000 Hello, Steve Lhomme: > Hi Andreas, > > Thanks for your detailed feedback. > >> 1. Whether `DisplayWidth` and `DisplayHeight` needs to be written >> actually depends on the value of `DisplayUnit`. > > True, I will mention that. > The "Notes" still only cover the case of `DisplayUnit` indicating pixels. > 2018-07-11 15:47 GMT+02:00 Andreas Rheinhardt > : >> 3. "They SHOULD have the [obu_has_size_field] set to 1 except for the >> last OBU in the sample, for which [obu_has_size_field] MAY be set to 0, >> in which case it is assumed to fill the remaining of the sample." >> "The OBUs in the Block MUST follow the [Low Overhead Bitstream Format >> syntax]." >> The first sentence leaves the possibility that [obu_has_size_field] is 0 >> for OBUs other than the last OBU of a block (only a SHOULD). And the >> requirement in the second sentence actually makes MUST out of the SHOULD >> in the first sentence (making this part of the first sentence redundant) >> and contradicts/voids the MAY part of the first sentence. In other >> words, the two sentences should be merged to something like: "The `OBUs` >> in the block must follow the `Low Overhead Bitstream Format` (in which >> [obu_has_size_field] MUST be equal to one for every OBU) for every `OBU` >> with the possible exception of the very last `OBU` in which >> [obu_has_size_field] MAY be set to 0, in which case the `OBU` is assumed >> to consist of the remainder of the block." > > Indeed there's a contradiction here. If we use MUST (can't be must > lowercase) on [Low Overhead Bitstream Format] then the > [obu_has_size_field] MUST be 1. > > On MP4 for the CodecPrivate the [obu_has_size_field] MUST be 1. But in > the Blocks it can be 0: > > "Each OBU SHALL have the obu_has_size_field set to 1 except for the > last OBU in the sample, for which obu_has_size_field MAY be set to 0, > in which case it is assumed to fill the remaining of the sample" > > I think we should mimic that. I'll rephrase it. > The current version is: "The OBUs in the Block follow the [Low Overhead Bitstream Format syntax]. They SHOULD have the [obu_has_size_field] set to 1 except for the last OBU in the sample, for which [obu_has_size_field] MAY be set to 0, in which case it is assumed to fill the remaining of the sample." If one interprets the first sentence as meaning "The OBUs in the Block MUST follow the [Low Overhead Bitstream Format syntax]", then given that this syntax mandates [obu_has_size_field] to be equal to 1 the first part of the second sentence is redundant (given that MUST is stronger than SHOULD) and the second part is again in contradiction to/voided by the first sentence because the first sentence doesn't allow "[obu_has_size_field]" set to zero at all. If one interprets the first sentence as not conveying a MUST, then it is allowed (albeit strongly discouraged) to use [obe_has_size_field] equal to 0 for an OBU that is not the last OBU in the sample. This is not what we want, isn't it? How about: "The OBUs in the `Block` MUST follow the [Low Overhead Bitstream Format syntax] with the possible exception of the last OBU of a `Block` for which [obu_has_size_field] MAY be set to 0, in which case it is assumed to fill the remainder of the `Block`." >> 4. "ReferenceBlocks inside a BlockGroup MUST reference frames according >> to the [ref_frame_idx] values of frame that is neither a KEYFRAME nor an >> INTRA_ONLY_FRAME.": The problem with this sentence is that >> [ref_frame_idx] needn't be present. It depends upon >> [frame_refs_short_signaling] and [show_existing_frame]. If one uses a >> Block inside a Blockgroup and if [show_exsting_frame] equals one one >> should reference the block that contained the showable frame that is now >> output (and that this should be the only `ReferenceBlock` written). In >> case of [frame_refs_short_signaling] == 1 the obvious candidates for >> `ReferenceBlocks` are the blocks containing the `last_frame_idx` and >> `gold_frame_idx` that are explicitly signalled. If I am not mistaken, >> then there are also other reference frames that are not explicitly >> signalled, but computed. I don't know if we should really write a >> `ReferenceBlock` entry for every reference as the current proposal seems >> to imply. This would be quite a bit of overhead for no gain (and >> furthermore, it would complicate muxers that would have to compute the >> references that are not explicitly signalled in case that > > This is how `ReferenceBlock` is supposed to be used. So a muxer that > has no idea of any codec can cut a file and keep the relevant > references. So they all have to be there. It's one of the reasons > SimpleBlock was added, to simplify things a little (and reduce > overhead). > Actually a muxer can cut a file if it just knows the keyframes, the decoding order and the display order. It doesn't need to have complete information about reference frames. After all, one can cut files that exclusively use `SimpleBlocks`. (If one wants to cut the beginning away, one can cut according to the keyframes; and at the end one can cut every block with timestamp >t_0 away if there is no block with timestamp >t_0 that precedes a block with timestamp <=t_0 in coding/storage order.) >> [frame_refs_short_signaling] is 1). One `ReferenceBlock` would be enough >> to distinguish keyframes from non-keyframes. >> By the way: If a temporal unit contains multiple frames with references, >> whose references should end up as `ReferenceBlocks`? Or may the muxer >> choose some? > > I think a Temporal Unit can only have one (visible frame). I don't > know if golden frames can have extra references. But the BlockGroup > should contain all frames needed to decode this frame, that includes > all the frames in the Block (even if not visible). > >> 5. AV1 may use spatial scalability and/or temporal scalability. What do >> we make of these? They are currently not forbidden if I am not mistaken, >> but if e.g. the spatial dimensions of different layers disagree, the >> `PixelWidth` and `PixelHeight` values can't be true for all layers. >> Matroska seems to be missing some features here. > > Our spec says that the Sequence Header OBU should be valid for all > frames. That can't be used for spatial scalability. We don't support > that mode for now. > Then this should be explicitly stated in the codec mapping. And I also fail to see why the fact that the Sequence Header OBU should be valid for all frames should be incompatible with spatial scalability (after all, in my reading of the spec the various share the same Sequence Header OBU). >> 7. Then there is another thing with keyframes and cues (for this point >> it is always presumed that the relevant sequence header OBUs are >> available regardless of whether this is done in-band or via CodecPrivate): >> a) The proposal currently does not take into account that key frames >> reset the decoder when they are output, not when they are decoded. A key >> frame needn't be immediately output; if it is (i.e. [show_frame] >> equaling 1), it is called a "key frame random access point" in section >> 7.6 of the standard and is the equivalent of an IDR frame in H.264. >> Everything's fine here. But a key frame can also be declared a >> showable_frame (but only if [show_frame] equals 0) and output later via >> the show_existing_frame mechanism. This is similar to an open GOP in >> other codecs (but in contrast to them, the block that contains the coded >> keyframe doesn't have the same timestamp (pts) as the first frame that >> can be output after a seek). The coded key frame with [show_frame] equal >> to zero is called a delayed random access point and a key frame >> dependent recovery point is a frame where a key frame with >> [showable_frame] equal to 1 is output via the show_existing_frame >> mechanism. If one starts decoding at the delayed random access point, >> all the output frames up to but not including the key frame dependent >> recovery point can depend both on the delayed random access point frame >> and on other earlier frames so that these frames can't be correctly >> decoded in general. But all the frames from the key frame dependent >> recovery point onwards can be correctly decoded if one starts decoding >> at the delayed random access point (because the decoder is reset after >> displaying the key frame). If one starts decoding at the key frame >> dependent recovery point, one doesn't have the key frame that should be >> shown via the show_existing_frames mechanism at all, so that this frame >> is simply not a real key frame. >> b) But although a key frame dependent recovery point is not a "real" key >> frame, it has the same [frame_type] as the frame that is output, i.e. >> its [frame_type] is KEY_FRAME. According to our current proposal this >> would mean that it should be treated as a keyframe in Matroska which is >> obviously wrong. > > That's not how I understand it. Here's section 7.6.3: > > "Informally, the requirement for decoder conformance is that decoding > can start at any key frame random access point or delayed random > access point." > > And 7.6.2: > > "delayed random access point is defined as being a frame: > • with frame_type equal to KEY_FRAME > • with show_frame equal to 0 > • that is contained in a temporal unit that also contains a sequence header OBU" > > So as long as we seek on frames of type KEY_FRAME we should be able to > seek. Wether it's a visible frame or not. a) According to the Uncompressed Header Syntax, the [frame_type] of a frame that is output via the [show_existing_frame] mechanism is the [frame_type] of the [showable] frame that is output. This implies that a key frame dependent recovery point (KFDRP from now on) has [frame_type] KEY_FRAME. And this means that a KFDRP is a keyframe according to the codec mapping (actually it is only heavily implied to be a keyframe, because the current codec mapping does not require to label any frame a keyframe). b) The first passage you cited says that decoding can start at key frame random access points (KFRAP from now on) or delayed random access points (DRAP from now on). It does not say that one can seek to frames of type KFDRP. And these frames are of type KEY_FRAME, too. > > But because this is a bit loose in the 7.6.3 section they add this: > > "To support the different modes of operation, a conformant decoder is > required to be able to decode bitstreams consisting of: > • a temporal unit containing a delayed random access point > • immediately followed by a temporal unit containing the associated > key frame dependent recovery point" > > So the invisible KEY_FRAME should be immediately followed by the > recovery point data. So effectively it will work. There's also a note > that if it's not followed immediately then what is done with the > intermediate frames is implementation dependent. I don't think we need > to care too much. > I don't think we should assume that the DRAP block is immediately followed by a KFDRP (and I don't see how the assumption that there are no temporal units between DRAP and KFDRP simplifies anything for us container guys). That's just the only place where being able to decode is absolutely required. But just read the note (that you actually quote below) and you will see that they expect more: "In practice, decoder implementations are expected to be able to start decoding bitstreams from a delayed random access point when the intermediate temporal units are still present. The decoder should correctly produce all output frames from the next key frame or key frame dependent recovery point onwards, while the preceding frames are implementation defined." So it's reasonable to assume that there will be temporal units between DRAP and KFDRP. And I think we should care about such stuff and in particular make sure that seeking works well (because when there are non-KFRAP keyframes, AV1 is very much like intra decoder refresh when it comes to seeking and intra decoder refresh seeking currently doesn't work well because the cues only contain the timestamps of the frames where one should start decoding in order to output frames, but not the timestamp of the first frame that is undamaged after one has started decoding; this is a problem when one wants to seek to one of the frames inbetween these two frames). > >> c) Marking a delayed random access point as keyframe deviates from the >> way that flag has been traditionally understood: If one starts decoding >> at this point, one doesn't get the frame that should be output for the >> temporal unit containing the delayed random access point. But I >> nevertheless think that these are the right keyframes, because they are >> the points at which random access has to begin when there aren't key >> frame random access points available; this also means that one can split >> the stream at this point and the second part will still play so that a >> muxer like mkvmerge needn't be rewritten too much. > > Yes, IMO this is the correct way. > >> d) A consequence of this is that a `Blockgroup` containing a delayed >> random access point mustn't contain a `ReferenceBlock` (although the >> actual frame that is output for that temporal unit very likely uses >> other reference frames than the key frame that is contained in the same >> temporal unit). > > This won't happen if it's a proper random access point, ie it doesn't > need past frames to start decoding. If it's not then it's not a RAP > and then it can/should have ReferenceBlock. > The DRAP frame itself (being coded intra) won't reference other frames, but the other frames (including the shown frame) in the temporal unit containing said DRAP will of course reference earlier frames (the H.264 equivalent of this scenario is an open gop and the other frames in this case are B-frames shared between two GOPs (i.e. the B-frames that precede the open GOP's keyframe in display order and follow it in coding order) -- and they of course reference both the keyframe and earlier frames, that is after all the whole point of having an open GOP). So it does happen although it is a random access point; it does happen, because it is just a delayed random access point. >> e) Yes, this proposal means that it is impossible to tell from Matroska >> alone (well, from the block structure that is; see f) for a way for >> which one could put this information into the Cues) whether it is a key >> frame random access point or a delayed random access point. One will >> have to decode it (or parse deeper) to know. > > No, this is independent of the codec. Also Cues can target a frames > that can't be seeked to directly but that's beside the point. S > SimpleBlock marked keyframe or BlockGroup with no ReferenceBlock can > be seeked to directly and that's why they equal Random Access Points > as defined in AV1 (and other codecs). > Given that your reply began with "No" I presume that you believe that you refuted me, but I don't see that you have. After all, how can one tell from the Matroska layer alone whether a `Block` is a KFRAP or a DRAP? (Of course I know that Cues can also target frames that can't be seeked to directly -- after all, my proposal below is about adding Cues for such frames.) >> f) This also leads to problems with seeking: If one simply added a >> CuePoint for the keyframe (i.e. for the delayed random access point) and >> the user wants to seek to a point between the delayed random access >> point (inclusive) and the dependent recovery point (exclusive) and the >> player used the cues to seek to the nearest keyframe in front of the >> desired point, then decoding at the point referenced in the cues would >> not yield the desired frame (it would be either corrupted or not output >> at all). Therefore I think it is best to add a CuePoint for every key >> frame random access point and every key frame dependent recovery point. >> The CuePoint for the key frame random access point would be an ordinary >> CuePoint as usual. But the CuePoint for the key frame dependent recovery >> point wouldn't be (my favourite is iv) (and if I were allowed to play >> God it would be i))): > > Cues are quite loose. It would be possible to do Cues only for frames > that are not delayed RAP and that's valid. IMO it's fine to reference > the delayed RAP. But they are RAP so it's legal to seek there. > Cues are loose, but we can (and I think we should) add a SHOULD clause that describes what cues muxers are expected to produce. And referencing the delayed RAP brings the problem that the cues only contain where to start decoding, but not from when on the output is undamaged/valid. > The AV1 specs have this to say about this tricky case: > > "Note:In practice, decoder implementations are expected to be able to > start decoding bitstreams from a delayed random access point when the > intermediate temporal units are still present. The decoder should > correctly produce all output frames from the next key frame or key > frame dependent recovery point onwards, while the preceding frames are > implementation defined. For example: a streaming decoder may choose to > decode and display all frames even when the reference frames are not > available (tolerating some errors in the output), a low latency > decoder may choose to decode and display all frames that are > guaranteed to be correct (e.g. an inter frame that only uses inter > prediction from the delayed random access point), a media player > decoder may choose to decode and display only frames starting from a > key frame or key frame dependent recovery point (guaranteeing smooth > playback once display starts)." > > It's not up to the container to solve this. > It is not up to the container to decide whether the possibly damaged frames between the DRAP and the KFDRP should be displayed, but it very much is up to the container to make it as easy as possible to find the data needed for random access. And so the cues should answer the question "I want to play from point t_0 on. Where do I have to seek to?" >> i) A comprehensive way of doing it is this: The CueTime would be the >> timestamp of the block containing the dependent recovery point; it would >> include a CueTrackPositions for the video track we are talking about >> that contains the right CueTrack, the CueClusterPosition containing the >> position of the dependent recovery point block and a CueReference with >> CueRefTime and CueRefCluster, both corresponding to the valus of the >> delayed random access point. This proposal has several downsides: It >> uses Cue elements that are deprecated in Matroska and not part of Webm. >> So this would require a quite nontrivial change in both projects. (Btw: >> If one does this, one should add a default value for `CueRefCluster`: It >> should be the same as `CueClusterPosition` as both blocks that we are >> talking about will probably end up in the same cluster anyway.) >> ii) One uses the CueTime of the dependent recovery point, but the >> position of the Cluster of the delayed access point (and >> `CueRelativePosition` (if used) should also point to this block). >> Pro: It only uses elements that are supported by both Matroska and Webm. >> Furthermore, the specs only say that `CueClusterPosition` should point >> to the cluster containing the "required block; they don't explicitly say >> that said block needs to have the same timestamp as `CueTime`. >> Contra: How does a demuxer know from which block onwards it should feed >> the data to the decoder? It might use the `CueRelativePosition`, but >> probably a lot of demuxers would simply read the cluster until they come >> to the block with timestamp `CueTime` (i.e. they interpret the specs so >> that the "required block" is the block with the timestamp `CueTime`) and >> then they would either deliver this to the decoder or conclude that the >> file is damaged (because the block they found is no keyframe). >> iii) The last is the same as i) with the difference that `CueRefCluster` >> is omitted. It is also incompatible with current Webm, but at least it >> has the advantage that it doesn't use any currently deprecated elements >> of Matroska. One could add a requirement that the delayed random access >> point and the dependent recovery point need to be in the same cluster >> and then omitting `CueRefCluster` is not a problem any more. >> iv) And then there is the possibility of creating a normal CuePoint for >> the dependent recovery point, writing the dependent recovery point as a >> Block in a Blockgroup with exactly one ReferenceBlock which points to >> the delayed access point block and let the demuxer seek backwards from >> the dependent recovery point to the delayed access point. >> Pro: Would only use things that are already supported by Matroska and >> Webm. It would also not be AV1 specific. The demuxer doesn't need to >> know anything about AV1, everything is signalled at the container level. >> Contra: Demuxers would have to be adapted not to expect any more that >> only keyframes are referenced in the cues. They would also have to be >> adapted to actually make use of the value of `ReferenceBlock` and seek >> backwards. This also implies more seeks, but this should be quite >> limited when one puts both the delayed random access point and the >> dependent recovery point in the same cluster -- hopefully the data is >> still cached. (Maybe one should add a SHOULD clause that says that both >> blocks should be in the same cluster.) >> g) Of course there are two easy alternative solutions: >> i) Restrict the type of AV1 that is allowed in Matroska even further so >> that all key frames are of key frame random access type. (This could >> exclude quite a lot of AV1 and therefore I recommend not doing so.) >> ii) Create cues as usual, i.e. reference every delayed random access >> point, and don't care about the fact that seeking will be partially >> broken in this case. >> h) It should be noted that exactly the same situation exists with >> periodic intra refresh in general. There was a short discussion on the >> Matroska developer mailing list in April 2011, but nothing came out of >> it. Every solution I outlined here for AV1 is also applicable for this case. >> >> Steve Lhomme: >>> Since we allow stripping the Sequence Header OBU from the stream when >>> it's equal to the CodecPrivate one, we need to add it back to the >>> bitstream for compliance. At least when seeking on keyframes. So I >>> added a section to explain that. >>> >>> IMO that's an extra feature of the CodecPrivate that it's meant to be >>> added to the bistream as-is. And in this case on startup and when >>> seeking. I wonder if we should add an element next to the CodecPrivate >>> to describe that. Because in this case it's not entirely opaque to the >>> demuxer. Or maybe it's implied by the CodecID and is up to the decoder >>> to use it how it's supposed to be (in this case detecting keyframes >>> and possibly adding back the Sequence Header OBU). >> >> 8. I think we can relax the requirements on the existence of in-band >> sequence header OBUs a bit: If a keyframe (i.e. a key random access >> point or a delayed random access point, not a dependent recovery point) >> uses the same sequence header OBU as in the CodecPrivate (including the >> same operating_parameters_info), then the sequence header OBU needn't be >> prepended to the block with the keyframe, because seeking already works >> without it provided one always adds the sequence header OBU from the >> CodecPrivate back in the bitstream on seeking. For example, consider the >> following scenario: >> One has an elementary stream that uses two different sequence header >> OBUs A and B that only differ in the operating_parameters_info. The >> first three keyframes use A, between the third and the B is contained in >> a temporal unit between the third and the fourth keyframe. Between the >> sixth and the seventh keyframe is a temporal unit containing sequence >> header A again. Then a muxer that wants to put this elementary stream >> into Matroska may put A in the CodecPrivate can strip the very first >> occurence of A away; it must leave B inside the temporal unit that it >> was in (so that a player that plays the file linearly is notified about >> the change) and has to make sure that keyframes #4 to #6 contain >> sequence header B (so that one has the correct sequence header when >> seeking to said keyframes). It mustn't strip A between the sixth and the >> seventh keyframe away (so that a player that plays the file linearly >> notices the change of sequence header), but it needn't preprend >> keyframes #7 and following with A. > > That works but then we need to tell in the specs that following a > Sequence Header MUST not be stripped if the previous Sequence Header > was not bit identical to the one in CodecPrivate. > > IMO it's valid to output A and then B before the rest of the data in > frame #4. That should be done if it's a keyframe. But if it's not a > keyframe the demuxer/decoder doesn't know it has to prepend the > CodecPrivate there. That could be a problem. Even though it doesn't > make much sense to change Sequence Header data before a non keyframe > (RAP). Why should the demuxer ever want to prepend the CodecPrivate in front of a non-keyframe? This doesn't make sense to me; after all, one should only seek to keyframes and only add the CodecPrivate in front of a keyframe if one has just seeked to said keyframe, not if one just encounters a keyframe (after the seek one just uses whatever Sequence Header one encounters in-band (if any)). > > So maybe your proposal should be added. We'll gain a bit of weight but > it's safer. > My wording would be: "Upon seeking to a keyframe the player/demuxer MUST prepend the `Block` with the `Sequence Header OBU` contained in the `CodecPrivate`. A muxer MUST make sure that the correct `Sequence Header OBU` is in force both during linear access and also after seeking to a keyframe `Block`. So in particular a keyframe `Block` where a `Sequence Header OBU` that is not bit-identical to the one in the `CodecPrivate` is in force for the decoding of the first frame contained in said `Block` MUST contain the `Sequence Header OBU` that is in force for the decoding of the first frame contained in said `Block` in front of the first frame contained in said `Block`." One could also relax this a bit and only make the correct linear access a MUST and the rest a SHOULD. This might be useful for applications where seeking isn't desired (although it really should be included even for those scenarios to support resuming playback after a transmission error). And maybe one should add a clause like "Seeking to non-keyframes is undefined and not recommended." And if one wants to one can add a clause recommending to strip out unnecessary `Sequence Header OBUs` >> Steve Lhomme: >>> Let me know what you think so we can settle this spec for good. >>> >> I think we are not even close to settling this for good. > > \o/ > I honestly don't know what this means. - Andreas Rheinhardt From nobody Thu Jul 12 12:28:47 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4C282130F2D for ; Thu, 12 Jul 2018 12:28:45 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (4096-bit key) header.d=bunkus.org Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id g4gJDTG5Igkp for ; Thu, 12 Jul 2018 12:28:42 -0700 (PDT) Received: from adara.bunkus.org (adara.bunkus.org [144.76.6.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 34C46130E2A for ; Thu, 12 Jul 2018 12:28:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=bunkus.org; s=mail2017070101; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Message-ID:Date:Subject:To:From; bh=fCsOii5yzvDm+6+jNITU/eEJxdTlrPvv7pJ6ZHzeuYw=; b=tt25MvRtmzU57e38Xf0XF+bjhH1HW7EoGTBqyB0W+YrHxLVuToaldW63s4xJBHYvZhUuP0sbtBZwNa8yfd6PY3F1aJdGVYJIqkGLh797d10Kd3AdNOJLj5E0OIUM0qh1E6KD/JK88lZnyHQpntPfr0POkOcG6bCd48mJB374tUxLG/wAySwQte9LVywJKlRa4wn1RFmb+DFggWNqwIX2Sqm8Gca8MAu2tE6GdYWnF+dFk3q3jjkIYPreIprDIIXrAVd3rr/QbOvYUvIT/U/vJcac9Z9CdEcuY5Gy39IWJy5DkkADv12QOGmbVmoSCfaF08swzpWAQg2efhGkKlEk60xle0xoNnt2tmfii4KJczbFdPm64VVBZvnmGOxn41BFfXjeqw70H9JB/8qg0XDFrueEG5IgTVhnMss5FvddmKC5zl2XAdSI5xvz8cjsGGgTjwy3NXLEsZs+01cSPRfWC8RIx2dHZeFZoT4BPFd2Rlo7wrlKmiu1p/AC7ljUMjqzspRO2iGOFuTY2N4hfcirJP7Nh9mfuY0befc63vTmnhb/ZziilhGzCg5ZJuTdLJ0VcdSHJTgJKNm6+t1y7ag7yzNKp1dkDQOinG7O2woqwmSXOoPnedXzmXdwtqGf2f5SUNokf0NQ4fyfnvAn8x0fx6cBQQUcpDpMShhbVTCNmfc=; Received: from liselle.bunkus.org ([2a01:4f8:190:8147::105:1]:40952) by adara.bunkus.org with esmtps (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.82_1-5b7a7c0-XX) (envelope-from ) id 1fdhGV-0006TV-1M; Thu, 12 Jul 2018 21:28:35 +0200 X-Virus-Scanned: amavisd-new at bunkus.org Received: from sweet-chili.local (unknown [192.168.191.4]) by liselle.bunkus.org (Postfix) with ESMTPS id 69AC1654158E; Thu, 12 Jul 2018 21:28:28 +0200 (CEST) Received: from sweet-chili (localhost [IPv6:::1]) by sweet-chili.local (Postfix) with ESMTP id C0B673CD9609; Thu, 12 Jul 2018 21:28:27 +0200 (CEST) User-agent: mu4e 1.0; emacs 26.1 From: Moritz Bunkus To: Cellar list , help Questions Date: Thu, 12 Jul 2018 21:28:27 +0200 Message-ID: <87a7qwb6ec.fsf@bunkus.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Archived-At: Subject: [Cellar] MKVToolNix v25.0.0 released X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jul 2018 19:28:46 -0000 Hello everyone, July's release of MKVToolNix is here: v25.0.0. After two smaller ones, this one packs quite a number of bug fixes and enhancements, including fixing a regression in the GUI's header editor preventing elements from being removed that was introduced in v24.0.0. Sorry about that. AV1 support hasn't changed. While the bitstream format's been finalized in the meantime, the mapping to Matroska & WebM hasn't. It's currently being discussed on the CELLAR mailing list. Here are the usual links: =E2=80=A6to the source code: https://mkvtoolnix.download/source.html =E2=80=A6to the binaries: https://mkvtoolnix.download/downloads.html The Windows and macOS binaries as well as the Linux AppImage are available already. The other Linux binaries are still being built and will be available of the course of the next couple of hours. Here are the NEWS since the previous release: ------------------------------------------------------------ # Version 25.0.0 "Prog Noir" 2018-07-12 ## New features and enhancements * mkvmerge: SRT/ASS/SSA text subtitles: for files for which no encoding has been specified, mkvmerge will try UTF-8 first before falling back to the system's default encoding. Part of the implementation of #2246. * mkvmerge: SRT/ASS/SSA/WebVTT text subtitles: a warning is now emitted if invalid 8-bit characters are encountered outside valid multi-byte UTF-8 sequences. Part of the implementation of #2246. * mkvmerge: Matroska & MPEG transport stream readers: the encoding of text subtitles read from Matroska files can now be changed with the `--sub-charset` parameter. * Linux: starting with release 25 an AppImage will be provided which should run on any Linux distribution released around the time of CentOS 7/Ubuntu 14.04 or later. * macOS: translations: updated the `build.sh` script to build `libiconv` an= d a complete `gettext`. Together with an additional fix to how translation fi= les are located, MKVToolNix can now use all interface languages on macOS, too. Fixes #2110, #2307, #2323. ## Bug fixes * mkvmerge: AVC/h.264: fixed file identification failing for certain elementary streams due to internal buffers not being cleared properly. Fi= xes #2325. * mkvmerge: HEVC/h.265: fixed file identification failing for certain elementary streams due to internal buffers not being cleared properly. Th= is is the HEVC analog to what was fixed for AVC in #2325. * mkvmerge: MLP code: fixed various issues preventing MLP from being parsed correctly. Fixes #2326. * mkvmerge: TrueHD/MLP packetizer; dialog volume normalization removal isn't attempted if the track is an MLP track as the operation is only supported for TrueHD, not MLP. * mkvmerge: MPEG TS reader: when reading MPLS mkvmerge will now compare the MPLS's start and end timestamps against the transport stream's PTS instead of its DTS. Otherwise the first key frame of a video track might be dropp= ed if it isn't the first in presentation order. Fixes #2321. * mkvmerge: JSON identification: mkvmerge will ensure that all strings pass= ed to the JSON output modules are valid UTF-8 encoded strings by replacing invalid bytes with placeholder characters. This avoids the JSON library throwing an exception and mkvmerge aborting on such data. Fixes #2327. * mkvmerge: audio packetizers: mkvmerge will now keep discard padding values if they're present for packets read from Matroska files. Fixes #2296. * mkvmerge: Ogg Opus reader: packet timestamps aren't calculated by summing= up the duration of all packets starting with timestamp 0 anymore. Instead the algorithm is based on the Ogg page's granule position and which packet number is currently timestamped (special handling for the first and last packets in the stream). * This fixes the first timestamp if the first Ogg packet's granule positi= on is larger than the number of samples in the first packet (=3D if the fi= rst sample's timestamp is bigger than 0). mkvmerge will keep those offsets = now and inserts "discard padding" only where it's actually needed. * It also improves handling of invalid files where the first Ogg packet's granule position is smaller than the number of samples in the first pac= ket (=3D the first sample's timestamp is smaller than 0). mkvmerge will now shift all timestamps up to 0 in such a case instead of inserting "disca= rd padding" elements all over the place. * mkvmerge will no longer insert "discard padding" elements if the difference between a) the calculated number of samples in the packet according to the granule position and b) the actual number of samples as calculated from the bitstream is one sample or less and if the packet isn't the last one in the stream. This circumvents certain rounding errors. * The timestamp of the first packet after a gap in the middle of the stre= am is now calculated based on the Ogg page the packet belongs to, and not based on the timestamps before the gap. Fixes #2280. * mkvmerge: complete rewrite of the progress handling. It's now based upon = the total size of all source files and the current position within them inste= ad of the number of frames/blocks to be processed. This simplifies calculati= on when appending files and fixes rare cases of when progress report was obvious wrong (e.g. stuck at 0% right until the end). Fixes #2150 and #23= 30. * MKVToolNix GUI: header editor: non-mandatory elements couldn't be removed anymore due to a regression while fixing #2320. They can now be removed again. Fixes #2322. ------------------------------------------------------------ Have fun :) mosu From nobody Fri Jul 13 00:24:26 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AEFBA130DCC for ; Fri, 13 Jul 2018 00:24:23 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, T_DKIMWL_WL_MED=-0.01] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id v3tF_5lYvqh9 for ; Fri, 13 Jul 2018 00:24:20 -0700 (PDT) Received: from mail-pg1-x52d.google.com (mail-pg1-x52d.google.com [IPv6:2607:f8b0:4864:20::52d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id EDF5B126F72 for ; Fri, 13 Jul 2018 00:24:19 -0700 (PDT) Received: by mail-pg1-x52d.google.com with SMTP id p23-v6so4563817pgv.13 for ; Fri, 13 Jul 2018 00:24:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=qNNBBqyG0Q8ptRUbwk95gHswefxe58f94mW9ALTvWKs=; b=uWTOjjzWYzia1GeOTTbVqfsKxI9AaQnhsTAzRexLQjlJYVUj+G1CwI1EurGVJsfbnr 3Oxgjcbgi9wAYK9fPEZvUFSvszpmyLy7ICpq6p40UVkhgNoYP5hGLqeV5ger3rebdDyS 8Ny0BSUspfCm2IHARi50pu60Wvl+YIOSO0qloKaVnw/913/x/N1ctqXahopNn2zuqhoi u8btcTdO/nOGGOGrr7eyFWSfqgx38oiYKmFOgY48KaZ8c+3fp9ng9Nz09tLwMHUBMAVW d+yXAfxSzfAc2Nn0LPqtOMESq+iJc+491DhR5AXJ9WIrLVu8XUhHHV28zPsOWOuvUlwm P1rA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=qNNBBqyG0Q8ptRUbwk95gHswefxe58f94mW9ALTvWKs=; b=siIB9GHCw0J+7GcsTrskpERxjdh085/JNlrnOBJE3abUGYYQi+9ZZDAwNFCoPpv2To xkQEd3bnf7nb1WmUbtCTqnTrt9kWp8KXhwSEEdHCEOADnCDEX772f1Zw6GjvkJ8ju8xX rEfuY3fh4sbLwFDsLft8F+Vp+q8/Fq1ae/qLsyKQQOkW7ayW6n+MZYLanjjj9vhdsnrv 0NUbA+Ew06G7K27xxdXSWrdmG8bU5iu/ZWdQjlI2BYt0TIhAFE/Svj5u7HRsypkeRBn6 Ntye/KdhqglhMGkrmthJw+juBfc1lk4NLUFoFgrUYc6zP3sglc7y0I0hkJIzwrQjsGtU i67Q== X-Gm-Message-State: AOUpUlEB46uQua83RUSwWjlmlkH91ZYWBrpCZ/1L+5Z+yZf+idWhqnZY kAUvOfoXr0xRHCMwRljcDA9rNb3E1d1IeNamUAscSJOd X-Google-Smtp-Source: AAOMgpdy+HaPpQnbQXimOVkn2PM0ii7X9iBMLRTkQZpysAZ4rszlUlEmbHe2icxmxrZ/OAF07NJYYriNQ3XEkS/VLgQ= X-Received: by 2002:a62:9652:: with SMTP id c79-v6mr5861820pfe.114.1531466659239; Fri, 13 Jul 2018 00:24:19 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a17:90a:9c13:0:0:0:0 with HTTP; Fri, 13 Jul 2018 00:24:18 -0700 (PDT) In-Reply-To: References: From: Steve Lhomme Date: Fri, 13 Jul 2018 09:24:18 +0200 Message-ID: To: Andreas Rheinhardt Cc: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Archived-At: Subject: Re: [Cellar] AV1 mapping update X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Jul 2018 07:24:24 -0000 Hi, 2018-07-12 20:20 GMT+02:00 Andreas Rheinhardt : > Hello, > > Steve Lhomme: >> Hi Andreas, >> >> Thanks for your detailed feedback. >> >>> 1. Whether `DisplayWidth` and `DisplayHeight` needs to be written >>> actually depends on the value of `DisplayUnit`. >> >> True, I will mention that. >> > The "Notes" still only cover the case of `DisplayUnit` indicating pixels. If they are not in pixels then the values should not be taken from the codec anyway. The general Matroska rules apply. IMO we are not going to repeat here the way DisplayWidth/Height has to be interpreted. >> 2018-07-11 15:47 GMT+02:00 Andreas Rheinhardt >> : >>> 3. "They SHOULD have the [obu_has_size_field] set to 1 except for the >>> last OBU in the sample, for which [obu_has_size_field] MAY be set to 0, >>> in which case it is assumed to fill the remaining of the sample." >>> "The OBUs in the Block MUST follow the [Low Overhead Bitstream Format >>> syntax]." >>> The first sentence leaves the possibility that [obu_has_size_field] is = 0 >>> for OBUs other than the last OBU of a block (only a SHOULD). And the >>> requirement in the second sentence actually makes MUST out of the SHOUL= D >>> in the first sentence (making this part of the first sentence redundant= ) >>> and contradicts/voids the MAY part of the first sentence. In other >>> words, the two sentences should be merged to something like: "The `OBUs= ` >>> in the block must follow the `Low Overhead Bitstream Format` (in which >>> [obu_has_size_field] MUST be equal to one for every OBU) for every `OBU= ` >>> with the possible exception of the very last `OBU` in which >>> [obu_has_size_field] MAY be set to 0, in which case the `OBU` is assume= d >>> to consist of the remainder of the block." >> >> Indeed there's a contradiction here. If we use MUST (can't be must >> lowercase) on [Low Overhead Bitstream Format] then the >> [obu_has_size_field] MUST be 1. >> >> On MP4 for the CodecPrivate the [obu_has_size_field] MUST be 1. But in >> the Blocks it can be 0: >> >> "Each OBU SHALL have the obu_has_size_field set to 1 except for the >> last OBU in the sample, for which obu_has_size_field MAY be set to 0, >> in which case it is assumed to fill the remaining of the sample" >> >> I think we should mimic that. I'll rephrase it. >> > > The current version is: > "The OBUs in the Block follow the [Low Overhead Bitstream Format > syntax]. They SHOULD have the [obu_has_size_field] set to 1 except for > the last OBU in the sample, for which [obu_has_size_field] MAY be set to > 0, in which case it is assumed to fill the remaining of the sample." > > If one interprets the first sentence as meaning "The OBUs in the Block > MUST follow the [Low Overhead Bitstream Format syntax]", then given that > this syntax mandates [obu_has_size_field] to be equal to 1 the first > part of the second sentence is redundant (given that MUST is stronger > than SHOULD) and the second part is again in contradiction to/voided by > the first sentence because the first sentence doesn't allow > "[obu_has_size_field]" set to zero at all. > If one interprets the first sentence as not conveying a MUST, then it is > allowed (albeit strongly discouraged) to use [obe_has_size_field] equal > to 0 for an OBU that is not the last OBU in the sample. This is not what > we want, isn't it? How about: > "The OBUs in the `Block` MUST follow the [Low Overhead Bitstream Format > syntax] with the possible exception of the last OBU of a `Block` for > which [obu_has_size_field] MAY be set to 0, in which case it is assumed > to fill the remainder of the `Block`." MUST is wrong IMO if you add an exception. Then it should be a SHOULD and explain the cases where it shouldn't. I left it out of the first sentence on purpose because the "normative" SHOULD is on the next sentence and the one that should apply. >>> 4. "ReferenceBlocks inside a BlockGroup MUST reference frames according >>> to the [ref_frame_idx] values of frame that is neither a KEYFRAME nor a= n >>> INTRA_ONLY_FRAME.": The problem with this sentence is that >>> [ref_frame_idx] needn't be present. It depends upon >>> [frame_refs_short_signaling] and [show_existing_frame]. If one uses a >>> Block inside a Blockgroup and if [show_exsting_frame] equals one one >>> should reference the block that contained the showable frame that is no= w >>> output (and that this should be the only `ReferenceBlock` written). In >>> case of [frame_refs_short_signaling] =3D=3D 1 the obvious candidates fo= r >>> `ReferenceBlocks` are the blocks containing the `last_frame_idx` and >>> `gold_frame_idx` that are explicitly signalled. If I am not mistaken, >>> then there are also other reference frames that are not explicitly >>> signalled, but computed. I don't know if we should really write a >>> `ReferenceBlock` entry for every reference as the current proposal seem= s >>> to imply. This would be quite a bit of overhead for no gain (and >>> furthermore, it would complicate muxers that would have to compute the >>> references that are not explicitly signalled in case that >> >> This is how `ReferenceBlock` is supposed to be used. So a muxer that >> has no idea of any codec can cut a file and keep the relevant >> references. So they all have to be there. It's one of the reasons >> SimpleBlock was added, to simplify things a little (and reduce >> overhead). >> > Actually a muxer can cut a file if it just knows the keyframes, the > decoding order and the display order. It doesn't need to have complete > information about reference frames. After all, one can cut files that > exclusively use `SimpleBlocks`. I'm not sure it's true for modern codec where the referenced frame may be older than the previous keyframes. Also the ReferenceBlock allows picking only the frames necessary to render that particular frame, regardless of the internals of the codec. > (If one wants to cut the beginning away, one can cut according to the > keyframes; and at the end one can cut every block with timestamp >t_0 > away if there is no block with timestamp >t_0 that precedes a block with > timestamp <=3Dt_0 in coding/storage order.) > > > >>> [frame_refs_short_signaling] is 1). One `ReferenceBlock` would be enoug= h >>> to distinguish keyframes from non-keyframes. >>> By the way: If a temporal unit contains multiple frames with references= , >>> whose references should end up as `ReferenceBlocks`? Or may the muxer >>> choose some? >> >> I think a Temporal Unit can only have one (visible frame). I don't >> know if golden frames can have extra references. But the BlockGroup >> should contain all frames needed to decode this frame, that includes >> all the frames in the Block (even if not visible). >> >>> 5. AV1 may use spatial scalability and/or temporal scalability. What do >>> we make of these? They are currently not forbidden if I am not mistaken= , >>> but if e.g. the spatial dimensions of different layers disagree, the >>> `PixelWidth` and `PixelHeight` values can't be true for all layers. >>> Matroska seems to be missing some features here. >> >> Our spec says that the Sequence Header OBU should be valid for all >> frames. That can't be used for spatial scalability. We don't support >> that mode for now. >> > Then this should be explicitly stated in the codec mapping. And I also > fail to see why the fact that the Sequence Header OBU should be valid > for all frames should be incompatible with spatial scalability (after > all, in my reading of the spec the various share the same Sequence > Header OBU). I didn't fully understand how the spatial scalability works. If the same Sequence Header OBU supports it then we can support it. After all the PixelWidth/Height use the "maximum" width/height. But internally it may be less. >>> 7. Then there is another thing with keyframes and cues (for this point >>> it is always presumed that the relevant sequence header OBUs are >>> available regardless of whether this is done in-band or via CodecPrivat= e): >>> a) The proposal currently does not take into account that key frames >>> reset the decoder when they are output, not when they are decoded. A ke= y >>> frame needn't be immediately output; if it is (i.e. [show_frame] >>> equaling 1), it is called a "key frame random access point" in section >>> 7.6 of the standard and is the equivalent of an IDR frame in H.264. >>> Everything's fine here. But a key frame can also be declared a >>> showable_frame (but only if [show_frame] equals 0) and output later via >>> the show_existing_frame mechanism. This is similar to an open GOP in >>> other codecs (but in contrast to them, the block that contains the code= d >>> keyframe doesn't have the same timestamp (pts) as the first frame that >>> can be output after a seek). The coded key frame with [show_frame] equa= l >>> to zero is called a delayed random access point and a key frame >>> dependent recovery point is a frame where a key frame with >>> [showable_frame] equal to 1 is output via the show_existing_frame >>> mechanism. If one starts decoding at the delayed random access point, >>> all the output frames up to but not including the key frame dependent >>> recovery point can depend both on the delayed random access point frame >>> and on other earlier frames so that these frames can't be correctly >>> decoded in general. But all the frames from the key frame dependent >>> recovery point onwards can be correctly decoded if one starts decoding >>> at the delayed random access point (because the decoder is reset after >>> displaying the key frame). If one starts decoding at the key frame >>> dependent recovery point, one doesn't have the key frame that should be >>> shown via the show_existing_frames mechanism at all, so that this frame >>> is simply not a real key frame. >>> b) But although a key frame dependent recovery point is not a "real" ke= y >>> frame, it has the same [frame_type] as the frame that is output, i.e. >>> its [frame_type] is KEY_FRAME. According to our current proposal this >>> would mean that it should be treated as a keyframe in Matroska which is >>> obviously wrong. >> >> That's not how I understand it. Here's section 7.6.3: >> >> "Informally, the requirement for decoder conformance is that decoding >> can start at any key frame random access point or delayed random >> access point." >> >> And 7.6.2: >> >> "delayed random access point is defined as being a frame: >> =E2=80=A2 with frame_type equal to KEY_FRAME >> =E2=80=A2 with show_frame equal to 0 >> =E2=80=A2 that is contained in a temporal unit that also contains a sequ= ence header OBU" >> >> So as long as we seek on frames of type KEY_FRAME we should be able to >> seek. Wether it's a visible frame or not. > > a) According to the Uncompressed Header Syntax, the [frame_type] of a > frame that is output via the [show_existing_frame] mechanism is the > [frame_type] of the [showable] frame that is output. This implies that a > key frame dependent recovery point (KFDRP from now on) has [frame_type] > KEY_FRAME. And this means that a KFDRP is a keyframe according to the > codec mapping (actually it is only heavily implied to be a keyframe, > because the current codec mapping does not require to label any frame a > keyframe). In "6.8.2 Uncompressed header semantics" it says: "If obu_type is equal to OBU_FRAME, it is a requirement of bitstream conformance that show_existing_frame is equal to 0." So the KFDRP which has show_existing_frame=3D1 is not an OBU_FRAME. I think it's in an OBU of type OBU_FRAME_HEADER. Also a `Block` is one Temporal Unit which has at least one shown OBU frame (exactly 1 if spatial scalability is not used). It's not entirely clear if an OBU of type OBU_FRAME_HEADER is a frame or not. We could add the requirement for our keyframe definition that the OBU with the keyframe flag must be of type OBU_FRAME. Currently we say: A `SimpleBlock` MUST only be marked as a Keyframe if the first `Frame Header OBU` in the `Block` has a __[frame_type]__ of `KEY_FRAME` This is actually the opposite of what we want... It should not be a `Frame Header OBU` but a `Frame OBU`. I think the Frame Header OBU is a frame placeholder for the show_existing_frame flag. It's missing the Tile Group that contains the actual data. I'll fix that in my next push. > b) The first passage you cited says that decoding can start at key frame > random access points (KFRAP from now on) or delayed random access points > (DRAP from now on). It does not say that one can seek to frames of type > KFDRP. And these frames are of type KEY_FRAME, too. I think this is exactly what Random Access Point means. You can seek there and start decoding safely. And that's exactly when a Block must be a keyframe in Matroska and not any other time. And for that I prefer to use Chapter 7.6. >> But because this is a bit loose in the 7.6.3 section they add this: >> >> "To support the different modes of operation, a conformant decoder is >> required to be able to decode bitstreams consisting of: >> =E2=80=A2 a temporal unit containing a delayed random access point >> =E2=80=A2 immediately followed by a temporal unit containing the associa= ted >> key frame dependent recovery point" >> >> So the invisible KEY_FRAME should be immediately followed by the >> recovery point data. So effectively it will work. There's also a note >> that if it's not followed immediately then what is done with the >> intermediate frames is implementation dependent. I don't think we need >> to care too much. > > I don't think we should assume that the DRAP block is immediately > followed by a KFDRP (and I don't see how the assumption that there are > no temporal units between DRAP and KFDRP simplifies anything for us > container guys). That's just the only place where being able to decode > is absolutely required. But just read the note (that you actually quote > below) and you will see that they expect more: > > "In practice, decoder implementations are expected to be able to start > decoding bitstreams from a delayed random access point when the > intermediate temporal units are still present. The decoder should > correctly produce all output frames from the next key frame or key frame > dependent recovery point onwards, while the preceding frames are > implementation defined." > > So it's reasonable to assume that there will be temporal units between > DRAP and KFDRP. > > And I think we should care about such stuff and in particular make sure > that seeking works well (because when there are non-KFRAP keyframes, AV1 > is very much like intra decoder refresh when it comes to seeking and > intra decoder refresh seeking currently doesn't work well because the > cues only contain the timestamps of the frames where one should start > decoding in order to output frames, but not the timestamp of the first > frame that is undamaged after one has started decoding; this is a > problem when one wants to seek to one of the frames inbetween these two > frames). >> >>> c) Marking a delayed random access point as keyframe deviates from the >>> way that flag has been traditionally understood: If one starts decoding >>> at this point, one doesn't get the frame that should be output for the >>> temporal unit containing the delayed random access point. But I >>> nevertheless think that these are the right keyframes, because they are >>> the points at which random access has to begin when there aren't key >>> frame random access points available; this also means that one can spli= t >>> the stream at this point and the second part will still play so that a >>> muxer like mkvmerge needn't be rewritten too much. >> >> Yes, IMO this is the correct way. >> >>> d) A consequence of this is that a `Blockgroup` containing a delayed >>> random access point mustn't contain a `ReferenceBlock` (although the >>> actual frame that is output for that temporal unit very likely uses >>> other reference frames than the key frame that is contained in the same >>> temporal unit). >> >> This won't happen if it's a proper random access point, ie it doesn't >> need past frames to start decoding. If it's not then it's not a RAP >> and then it can/should have ReferenceBlock. >> > The DRAP frame itself (being coded intra) won't reference other frames, > but the other frames (including the shown frame) in the temporal unit > containing said DRAP will of course reference earlier frames (the H.264 > equivalent of this scenario is an open gop and the other frames in this > case are B-frames shared between two GOPs (i.e. the B-frames that > precede the open GOP's keyframe in display order and follow it in coding > order) -- and they of course reference both the keyframe and earlier > frames, that is after all the whole point of having an open GOP). So it > does happen although it is a random access point; it does happen, > because it is just a delayed random access point. > > >>> e) Yes, this proposal means that it is impossible to tell from Matroska >>> alone (well, from the block structure that is; see f) for a way for >>> which one could put this information into the Cues) whether it is a key >>> frame random access point or a delayed random access point. One will >>> have to decode it (or parse deeper) to know. >> >> No, this is independent of the codec. Also Cues can target a frames >> that can't be seeked to directly but that's beside the point. S >> SimpleBlock marked keyframe or BlockGroup with no ReferenceBlock can >> be seeked to directly and that's why they equal Random Access Points >> as defined in AV1 (and other codecs). >> > Given that your reply began with "No" I presume that you believe that > you refuted me, but I don't see that you have. After all, how can one > tell from the Matroska layer alone whether a `Block` is a KFRAP or a DRAP= ? A RAP is exactly what we need, no matter the time. If there are glitches because it's a DRAP it's fine according to the specs. But that's were proper seek should be done. > (Of course I know that Cues can also target frames that can't be seeked > to directly -- after all, my proposal below is about adding Cues for > such frames.) > >>> f) This also leads to problems with seeking: If one simply added a >>> CuePoint for the keyframe (i.e. for the delayed random access point) an= d >>> the user wants to seek to a point between the delayed random access >>> point (inclusive) and the dependent recovery point (exclusive) and the >>> player used the cues to seek to the nearest keyframe in front of the >>> desired point, then decoding at the point referenced in the cues would >>> not yield the desired frame (it would be either corrupted or not output >>> at all). Therefore I think it is best to add a CuePoint for every key >>> frame random access point and every key frame dependent recovery point. >>> The CuePoint for the key frame random access point would be an ordinary >>> CuePoint as usual. But the CuePoint for the key frame dependent recover= y >>> point wouldn't be (my favourite is iv) (and if I were allowed to play >>> God it would be i))): >> >> Cues are quite loose. It would be possible to do Cues only for frames >> that are not delayed RAP and that's valid. IMO it's fine to reference >> the delayed RAP. But they are RAP so it's legal to seek there. >> > Cues are loose, but we can (and I think we should) add a SHOULD clause > that describes what cues muxers are expected to produce. And referencing > the delayed RAP brings the problem that the cues only contain where to > start decoding, but not from when on the output is undamaged/valid. >> The AV1 specs have this to say about this tricky case: >> >> "Note:In practice, decoder implementations are expected to be able to >> start decoding bitstreams from a delayed random access point when the >> intermediate temporal units are still present. The decoder should >> correctly produce all output frames from the next key frame or key >> frame dependent recovery point onwards, while the preceding frames are >> implementation defined. For example: a streaming decoder may choose to >> decode and display all frames even when the reference frames are not >> available (tolerating some errors in the output), a low latency >> decoder may choose to decode and display all frames that are >> guaranteed to be correct (e.g. an inter frame that only uses inter >> prediction from the delayed random access point), a media player >> decoder may choose to decode and display only frames starting from a >> key frame or key frame dependent recovery point (guaranteeing smooth >> playback once display starts)." >> >> It's not up to the container to solve this. >> > It is not up to the container to decide whether the possibly damaged > frames between the DRAP and the KFDRP should be displayed, but it very > much is up to the container to make it as easy as possible to find the > data needed for random access. And so the cues should answer the > question "I want to play from point t_0 on. Where do I have to seek to?" In Matroska to the CuePoint with no CueReference with the closest time before point_t_0. In AV1 to the RAP with the closest time before point_t_0. >>> i) A comprehensive way of doing it is this: The CueTime would be the >>> timestamp of the block containing the dependent recovery point; it woul= d >>> include a CueTrackPositions for the video track we are talking about >>> that contains the right CueTrack, the CueClusterPosition containing the >>> position of the dependent recovery point block and a CueReference with >>> CueRefTime and CueRefCluster, both corresponding to the valus of the >>> delayed random access point. This proposal has several downsides: It >>> uses Cue elements that are deprecated in Matroska and not part of Webm. >>> So this would require a quite nontrivial change in both projects. (Btw: >>> If one does this, one should add a default value for `CueRefCluster`: I= t >>> should be the same as `CueClusterPosition` as both blocks that we are >>> talking about will probably end up in the same cluster anyway.) >>> ii) One uses the CueTime of the dependent recovery point, but the >>> position of the Cluster of the delayed access point (and >>> `CueRelativePosition` (if used) should also point to this block). >>> Pro: It only uses elements that are supported by both Matroska and Webm= . >>> Furthermore, the specs only say that `CueClusterPosition` should point >>> to the cluster containing the "required block; they don't explicitly sa= y >>> that said block needs to have the same timestamp as `CueTime`. >>> Contra: How does a demuxer know from which block onwards it should feed >>> the data to the decoder? It might use the `CueRelativePosition`, but >>> probably a lot of demuxers would simply read the cluster until they com= e >>> to the block with timestamp `CueTime` (i.e. they interpret the specs so >>> that the "required block" is the block with the timestamp `CueTime`) an= d >>> then they would either deliver this to the decoder or conclude that the >>> file is damaged (because the block they found is no keyframe). >>> iii) The last is the same as i) with the difference that `CueRefCluster= ` >>> is omitted. It is also incompatible with current Webm, but at least it >>> has the advantage that it doesn't use any currently deprecated elements >>> of Matroska. One could add a requirement that the delayed random access >>> point and the dependent recovery point need to be in the same cluster >>> and then omitting `CueRefCluster` is not a problem any more. >>> iv) And then there is the possibility of creating a normal CuePoint for >>> the dependent recovery point, writing the dependent recovery point as a >>> Block in a Blockgroup with exactly one ReferenceBlock which points to >>> the delayed access point block and let the demuxer seek backwards from >>> the dependent recovery point to the delayed access point. >>> Pro: Would only use things that are already supported by Matroska and >>> Webm. It would also not be AV1 specific. The demuxer doesn't need to >>> know anything about AV1, everything is signalled at the container level= . >>> Contra: Demuxers would have to be adapted not to expect any more that >>> only keyframes are referenced in the cues. They would also have to be >>> adapted to actually make use of the value of `ReferenceBlock` and seek >>> backwards. This also implies more seeks, but this should be quite >>> limited when one puts both the delayed random access point and the >>> dependent recovery point in the same cluster -- hopefully the data is >>> still cached. (Maybe one should add a SHOULD clause that says that both >>> blocks should be in the same cluster.) >>> g) Of course there are two easy alternative solutions: >>> i) Restrict the type of AV1 that is allowed in Matroska even further so >>> that all key frames are of key frame random access type. (This could >>> exclude quite a lot of AV1 and therefore I recommend not doing so.) >>> ii) Create cues as usual, i.e. reference every delayed random access >>> point, and don't care about the fact that seeking will be partially >>> broken in this case. >>> h) It should be noted that exactly the same situation exists with >>> periodic intra refresh in general. There was a short discussion on the >>> Matroska developer mailing list in April 2011, but nothing came out of >>> it. Every solution I outlined here for AV1 is also applicable for this = case. >>> >>> Steve Lhomme: >>>> Since we allow stripping the Sequence Header OBU from the stream when >>>> it's equal to the CodecPrivate one, we need to add it back to the >>>> bitstream for compliance. At least when seeking on keyframes. So I >>>> added a section to explain that. >>>> >>>> IMO that's an extra feature of the CodecPrivate that it's meant to be >>>> added to the bistream as-is. And in this case on startup and when >>>> seeking. I wonder if we should add an element next to the CodecPrivate >>>> to describe that. Because in this case it's not entirely opaque to the >>>> demuxer. Or maybe it's implied by the CodecID and is up to the decoder >>>> to use it how it's supposed to be (in this case detecting keyframes >>>> and possibly adding back the Sequence Header OBU). >>> >>> 8. I think we can relax the requirements on the existence of in-band >>> sequence header OBUs a bit: If a keyframe (i.e. a key random access >>> point or a delayed random access point, not a dependent recovery point) >>> uses the same sequence header OBU as in the CodecPrivate (including the >>> same operating_parameters_info), then the sequence header OBU needn't b= e >>> prepended to the block with the keyframe, because seeking already works >>> without it provided one always adds the sequence header OBU from the >>> CodecPrivate back in the bitstream on seeking. For example, consider th= e >>> following scenario: >>> One has an elementary stream that uses two different sequence header >>> OBUs A and B that only differ in the operating_parameters_info. The >>> first three keyframes use A, between the third and the B is contained i= n >>> a temporal unit between the third and the fourth keyframe. Between the >>> sixth and the seventh keyframe is a temporal unit containing sequence >>> header A again. Then a muxer that wants to put this elementary stream >>> into Matroska may put A in the CodecPrivate can strip the very first >>> occurence of A away; it must leave B inside the temporal unit that it >>> was in (so that a player that plays the file linearly is notified about >>> the change) and has to make sure that keyframes #4 to #6 contain >>> sequence header B (so that one has the correct sequence header when >>> seeking to said keyframes). It mustn't strip A between the sixth and th= e >>> seventh keyframe away (so that a player that plays the file linearly >>> notices the change of sequence header), but it needn't preprend >>> keyframes #7 and following with A. >> >> That works but then we need to tell in the specs that following a >> Sequence Header MUST not be stripped if the previous Sequence Header >> was not bit identical to the one in CodecPrivate. >> >> IMO it's valid to output A and then B before the rest of the data in >> frame #4. That should be done if it's a keyframe. But if it's not a >> keyframe the demuxer/decoder doesn't know it has to prepend the >> CodecPrivate there. That could be a problem. Even though it doesn't >> make much sense to change Sequence Header data before a non keyframe >> (RAP). > > Why should the demuxer ever want to prepend the CodecPrivate in front of > a non-keyframe? This doesn't make sense to me; after all, one should > only seek to keyframes and only add the CodecPrivate in front of a > keyframe if one has just seeked to said keyframe, not if one just > encounters a keyframe (after the seek one just uses whatever Sequence > Header one encounters in-band (if any)). Because they can be omitted. But given we changed the way we omit it so that changes always go back to the "reference" one it's not needed anymore. The AV1 specs say a RAP must contains a Sequence Header OBU. In our case it may be omitted but it's up to the codec/demuxer to add it back in the stream when seeking. >> >> So maybe your proposal should be added. We'll gain a bit of weight but >> it's safer. >> > My wording would be: > "Upon seeking to a keyframe the player/demuxer MUST prepend the `Block` > with the `Sequence Header OBU` contained in the `CodecPrivate`. > A muxer MUST make sure that the correct `Sequence Header OBU` is in > force both during linear access and also after seeking to a keyframe > `Block`. So in particular a keyframe `Block` where a `Sequence Header > OBU` that is not bit-identical to the one in the `CodecPrivate` is in > force for the decoding of the first frame contained in said `Block` MUST > contain the `Sequence Header OBU` that is in force for the decoding of > the first frame contained in said `Block` in front of the first frame > contained in said `Block`." > > One could also relax this a bit and only make the correct linear access > a MUST and the rest a SHOULD. This might be useful for applications > where seeking isn't desired (although it really should be included even > for those scenarios to support resuming playback after a transmission > error). OK, I'll try to add something for seeking. > And maybe one should add a clause like "Seeking to non-keyframes is > undefined and not recommended." > > And if one wants to one can add a clause recommending to strip out > unnecessary `Sequence Header OBUs` > > >>> Steve Lhomme: >>>> Let me know what you think so we can settle this spec for good. >>>> >>> I think we are not even close to settling this for good. >> >> \o/ >> > I honestly don't know what this means. Raised arms. > - Andreas Rheinhardt > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar --=20 Steve Lhomme Matroska association Chairman From nobody Fri Jul 13 00:26:39 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 54875130DCC for ; Fri, 13 Jul 2018 00:26:37 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, T_DKIMWL_WL_MED=-0.01] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id I8Ger5EgZZak for ; Fri, 13 Jul 2018 00:26:36 -0700 (PDT) Received: from mail-pl0-x22b.google.com (mail-pl0-x22b.google.com [IPv6:2607:f8b0:400e:c01::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E5FD6126F72 for ; Fri, 13 Jul 2018 00:26:35 -0700 (PDT) Received: by mail-pl0-x22b.google.com with SMTP id 30-v6so11813852pld.13 for ; Fri, 13 Jul 2018 00:26:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=qEAI1QTh+0dzPQabUNFlmeTPYAU+9ZX/zYPS8SWgw9E=; b=JFRsb1wrzzbH6cdS9njpxCIzloSBL5g7k8aDie7AeLx9RSvetWB06XgzP/wNnUWiYt cpM/5NH/TeqTsamf5T0thIAcNnM23BBz+9q3V3aQs2tVwwvfY6Ik+xIWeTqpB/BjNkVk oHCjAIN0DDZ4ar5/EhC7Ct6t5/d4kX+oZAC5wTCCg900FAAPTD9hDOuRI2vy0gNzdZqt kq4ukpRUm6Nr2ibBeLMgAc8o5VnlKU/dQI92vFzT6Ks+AHUC0GgtFhPBtnBpJw60XS3u WuqCim2JR28nUH8EwWhEktwu0bLt0nhHzLegWCNwmesBY7pT4tqXMZbyPAukWTLcZiyP MwrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=qEAI1QTh+0dzPQabUNFlmeTPYAU+9ZX/zYPS8SWgw9E=; b=RU61WjjGlf2IzP6tp2C6wAhrbC/3w1vTA4HCUrwylEN+Q1z1wJrra6XBxBnagW59z/ Y4ntCbRTETOttRiPncRcAYHEhczQ9Jv7mpvz2z28IeyU39lUJHQl/e/mWG90/3w/Evy5 Fn9vVJ4YDsP6x2d9H24oBdsgYFFYgx5a2tN6yyoqCGTk2Ts5tR7JLVHKnRvnMdO8pgDA 0MxlHaBq04z41AXLuXhvayDXVHSOcuwV/oa5PXQxQ9t0fNcEaw7EuWmgr8m+onP6TCct jC1MpH+KuOG8DapaehBj/oLRGE/wsXAqc6M0zX1yea4kWT/LQsGdaVhoKxJesjffUyUf 8+xw== X-Gm-Message-State: AOUpUlEMMk5bnnV+r84jiEUATBRTIbD1wLT5oKyUerTHYaMvG5YYEV2z kP4XBVs5JvcOnpOZFCk/NMhW+NwZ/R4EU1WhtQY1Bg== X-Google-Smtp-Source: AAOMgpeGO1l/Xh3Acptj2nCd71I7fJp+hsRHFbdyr5kbyIS5nFYcm0pEOACXxCvQTjK61WVaMAD5ZZNq9Pt/wxVD9RY= X-Received: by 2002:a17:902:7793:: with SMTP id o19-v6mr5342166pll.306.1531466795495; Fri, 13 Jul 2018 00:26:35 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a17:90a:9c13:0:0:0:0 with HTTP; Fri, 13 Jul 2018 00:26:35 -0700 (PDT) In-Reply-To: References: From: Steve Lhomme Date: Fri, 13 Jul 2018 09:26:35 +0200 Message-ID: To: Andreas Rheinhardt Cc: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cellar] AV1 mapping update X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Jul 2018 07:26:38 -0000 >>> So maybe your proposal should be added. We'll gain a bit of weight but >>> it's safer. >>> >> My wording would be: >> "Upon seeking to a keyframe the player/demuxer MUST prepend the `Block` >> with the `Sequence Header OBU` contained in the `CodecPrivate`. >> A muxer MUST make sure that the correct `Sequence Header OBU` is in >> force both during linear access and also after seeking to a keyframe >> `Block`. So in particular a keyframe `Block` where a `Sequence Header >> OBU` that is not bit-identical to the one in the `CodecPrivate` is in >> force for the decoding of the first frame contained in said `Block` MUST >> contain the `Sequence Header OBU` that is in force for the decoding of >> the first frame contained in said `Block` in front of the first frame >> contained in said `Block`." >> >> One could also relax this a bit and only make the correct linear access >> a MUST and the rest a SHOULD. This might be useful for applications >> where seeking isn't desired (although it really should be included even >> for those scenarios to support resuming playback after a transmission >> error). > > OK, I'll try to add something for seeking. Actually we already have it in the Segment Restrictions: Given a `Sequence Header OBU` can be omitted from a `Block` if __[decoder_model_info_present_flag]__ is 0 and it is bit identical to the one found in `CodecPrivate`, when seeking to a keyframe, that omitted `Sequence Header OBU` MUST be added back to the bitstream for compliance with the Random Access Decoding section of the [AV1 Specifiations](#av1-specifications). -- Steve Lhomme Matroska association Chairman From nobody Fri Jul 13 23:47:49 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C70D5130EA1 for ; Fri, 13 Jul 2018 23:47:47 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, T_DKIMWL_WL_MED=-0.01] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id daAOcW2QXiUf for ; Fri, 13 Jul 2018 23:47:46 -0700 (PDT) Received: from mail-pl0-x229.google.com (mail-pl0-x229.google.com [IPv6:2607:f8b0:400e:c01::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1F4D8129C6A for ; Fri, 13 Jul 2018 23:47:46 -0700 (PDT) Received: by mail-pl0-x229.google.com with SMTP id z9-v6so13094753plo.1 for ; Fri, 13 Jul 2018 23:47:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=Uz+E+xwWFlBNNjvjqV/suFdfRrEpahzCQ7eutl7By2s=; b=fm1gJBaKKafn720xtQBtk/1FGuiJ/3vYj3+GwDTVzcg+lgEjk7+p7hXff8sFvMEOOw J+aStzsab0Sld/RmkAKHGSbJzM9cXLUIybg0vWDGibDH/0aDtZ/twbHVX7h1sQ49B39I +p6L833LRqYpZAQo37eBp3NfxNuLHXvG44Cym9pxbWXG1bxvcaFn/pXP4HJgucJI1+DX TOZ3IkGYjMT6wKKTXitdesbYX+enhvb7hJZuKmbbNQTc0jqMvZ3M/ffhmNxCTPq+j2cB 7vvOcLNPiCfrjSDF9Lj2ogAfsXNWEv6oeyDGWYEL/2rojcIwT/D5hof0EGAOP/JVhY9l 8UbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=Uz+E+xwWFlBNNjvjqV/suFdfRrEpahzCQ7eutl7By2s=; b=V0DbSAp9ctdsXolaGAeNpMNUOlOz4LkdCbX6b78CiBHVVBjJ8zymnSNlFUBWjiVd1q B+GEplfvesd+olDAbnJ0dyO1oESebGdGN8T2Not60Srq8BoVR+b42XiLzUMMzl1AaD+q tTzmw/kqI3QPvpeRzKQx6oQePIxPZS23u+rX5oXCku2iPRVUuHwJmbwwxjSpdCQNCtAY 8cXHoXZa2quEsQPh2+M5Zm+ePXev4NfWmUD76rSpTyXwsKr2+89XWG8i45TIL8FWtd7x /+RtmsEuF7ytuooOP/Yfdg5caPoavwBmVguCr3cajAWAjwpOvkdjnA0AJcGcLHrRhNYc heZw== X-Gm-Message-State: AOUpUlGlWK1ZI953UR/DdE7nBby25YXaxD7guk4Nl2g0AyO+NUl8kZBX wD9HgSB8NaOkHWnxjAkbvDxuyoxYhOrZ4JFMlujP79+I X-Google-Smtp-Source: AAOMgpfNitQ6Bw06du5UVw2qqH7QbFLkgZ304fA93V450CeNdzNPO4ay+0Knv2B8uz+7inaCaFwg+i/zmsAb1orikZw= X-Received: by 2002:a17:902:82c7:: with SMTP id u7-v6mr9094567plz.83.1531550865515; Fri, 13 Jul 2018 23:47:45 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a17:90a:9c13:0:0:0:0 with HTTP; Fri, 13 Jul 2018 23:47:45 -0700 (PDT) From: Steve Lhomme Date: Sat, 14 Jul 2018 08:47:45 +0200 Message-ID: To: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: [Cellar] AV1 seeking X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Jul 2018 06:47:48 -0000 Hi, Following Andreas' feedback on the AV1 thread it seems there's something that needs to be looked deeper in AV1. There is a kind of RAP (Random Access Point) that is actually a not visible frame but that is a keyframe and a seekpoint (definition of RAP). That frame is actually displayed later (maybe sometimes it isn't) using a dummy frame (Frame Header OBU but not a Frame OBU). Sounds good so far. The problem is that between the keyframe and its visible sibling, there can be other frames. They may depend on frame before the RAP. This is grey area in the specs where it's up to the decoder to chose how to properly (or not) display them. Here's an example stream a b [C d] e f g h i [c] j k l C is the non visible keyframe which (from what I understand) always have another frame in the Temporal Unit, d in this case. And [c] is displayed later in the stream. If a user wants to seek to frame 'f', the normal seeking would start decoding at 'C'. But f may depend on frame 'a' (which hopefully is a keyframe otherwise we need the frames referenced by 'a'). Also this is very much the concept of B frames since 'c' is displayed after 'f' but with a different name. Using BlockReference we can actually know where to seek for these particular frames to get all the frames they need. But most people use SimpleBlock (or we could forbid its use for such streams) and it would mean referencing all frames in the Cues which is not a good idea. In MP4 they have [initial_presentation_delay_minus_one] in the CodecPrivate. I did not understand it so far because it's not found in the AV1 spec. But it seems to guarantee that to read frame 'f' you need to decode X frame before that one. In our case that would be 5 to have at least a decoded. IMO this is a bad design because it mixes the container with the codec (something done in the past in ogg with bad results). Seeking should not ask the codec how it should be done. Also that means to mux such an MP4 you'd need to scan the whole source first to verify the value is correct or that information needs to be given to the muxer by the encoder/packetizer (this information is not found in the stream). We have the option of not caring as it's allowed to leave it in grey area by the AV1 spec. Or we can think of a proper "container" solution. ie storing this information in the TrackInfo but not in the CodecPrivate. We already have a CodecDelay but IMO it doesn't fit the bill. Each frame has the timestamp modified by it. Which is not the case here. We may need a SeekingDelay so that looking for a seek point takes this amount in account. -- Steve Lhomme Matroska association Chairman From nobody Sat Jul 14 13:31:17 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8ABEC130DC5 for ; Sat, 14 Jul 2018 13:31:15 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.901 X-Spam-Level: X-Spam-Status: No, score=-6.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Vr7rwwvV2wTx for ; Sat, 14 Jul 2018 13:31:14 -0700 (PDT) Received: from smtp.mozilla.org (mx2.scl3.mozilla.com [63.245.214.156]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1983612F1A5 for ; Sat, 14 Jul 2018 13:31:13 -0700 (PDT) Received: from localhost (localhost6.localdomain [127.0.0.1]) by mx2.mail.scl3.mozilla.com (Postfix) with ESMTP id AB578C0B13; Sat, 14 Jul 2018 20:31:13 +0000 (UTC) X-Virus-Scanned: amavisd-new at mozilla.org Received: from smtp.mozilla.org ([127.0.0.1]) by localhost (mx2.mail.scl3.mozilla.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6nfIHtijuYbe; Sat, 14 Jul 2018 20:31:13 +0000 (UTC) Received: from [31.133.140.210] (dhcp-8cd2.meeting.ietf.org [31.133.140.210]) (Authenticated sender: tterriberry@mozilla.com) by mx2.mail.scl3.mozilla.com (Postfix) with ESMTPSA id 15248C04F3; Sat, 14 Jul 2018 20:31:13 +0000 (UTC) To: Steve Lhomme , Codec Encoding for LossLess Archiving and Realtime transmission References: From: "Timothy B. Terriberry" Message-ID: <71376bcb-ec70-4239-f2e5-e4202d660545@xiph.org> Date: Sat, 14 Jul 2018 13:31:11 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 SeaMonkey/2.49.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [Cellar] AV1 seeking X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Jul 2018 20:31:16 -0000 Steve Lhomme wrote: > In MP4 they have [initial_presentation_delay_minus_one] in the > CodecPrivate. I did not understand it so far because it's not found in > the AV1 spec. But it seems to guarantee that to read frame 'f' you > need to decode X frame before that one. In our case that would be 5 to > have at least a decoded. You want to search for InitialPresentationDelay in the AV1 specification. It is derived from the decoder model info if it is available (see decode_process()). From nobody Sat Jul 14 13:34:04 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 52DF5130DC5 for ; Sat, 14 Jul 2018 13:34:02 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.901 X-Spam-Level: X-Spam-Status: No, score=-6.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6_oj7yWgQRD1 for ; Sat, 14 Jul 2018 13:34:00 -0700 (PDT) Received: from smtp.mozilla.org (mx2.scl3.mozilla.com [63.245.214.156]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id AFF5812F1A5 for ; Sat, 14 Jul 2018 13:34:00 -0700 (PDT) Received: from localhost (localhost6.localdomain [127.0.0.1]) by mx2.mail.scl3.mozilla.com (Postfix) with ESMTP id 9F4BEC0B13 for ; Sat, 14 Jul 2018 20:34:00 +0000 (UTC) X-Virus-Scanned: amavisd-new at mozilla.org Received: from smtp.mozilla.org ([127.0.0.1]) by localhost (mx2.mail.scl3.mozilla.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EssRK-I6GBn0 for ; Sat, 14 Jul 2018 20:33:59 +0000 (UTC) Received: from [31.133.140.210] (dhcp-8cd2.meeting.ietf.org [31.133.140.210]) (Authenticated sender: tterriberry@mozilla.com) by mx2.mail.scl3.mozilla.com (Postfix) with ESMTPSA id C2900C04F3 for ; Sat, 14 Jul 2018 20:33:59 +0000 (UTC) To: Codec Encoding for LossLess Archiving and Realtime transmission References: From: "Timothy B. Terriberry" Message-ID: <65e82a9f-26b2-11d8-44d5-7b1d6ed131a2@xiph.org> Date: Sat, 14 Jul 2018 13:33:59 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 SeaMonkey/2.49.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [Cellar] AV1 mapping update X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Jul 2018 20:34:03 -0000 Andreas Rheinhardt wrote: >> It's not up to the container to solve this. >> > It is not up to the container to decide whether the possibly damaged > frames between the DRAP and the KFDRP should be displayed, but it very I would argue that this does have implications for file trimming. The Matroska mapping is allowed to place more requirements on an implementation than the AV1 specification. So there is a choice to make. Just considering the default (no further restrictions) or requiring one of the three example behaviors in the spec has the following implications: If you choose to leave the handling of frames between the DRAP and the KFDRP implementation-defined, then someone cutting a stream has no idea how they will be handled. So the only safe thing to do is to strip them all. This adds some minor complexity burden to the remuxer, but guarantees that all such useless data will get stripped by a correct implementation. Choosing to require decoding and display even when the reference frames are not available would produce a similar result (since packet loss concealment is not normative). Choosing to require decoding and display of all frames that are guaranteed to be correct might allow a remuxer to cut at slightly more places without including an entire extra keyframe interval, but given that such frames are not required to be consecutive, this would get pretty complicated to implement (but at least it is optional: you can always fall back to stripping all such frames as above). Choosing to require that only frames starting from the KFDRP be decoded and displayed means that a remuxer does not need to strip such frames to guarantee consistent output from all players, allowing it to be slightly simpler, at the cost of allowing it to include some data that is useless. None of this helps you for seeking, because you don't have a choice about which frames to include, so one must rely on the player not to display garbage (in the case that references are missing) or stuttering output (in the case that only some frames between the DRAP and KFDRP are correctly decodable). From nobody Sat Jul 14 14:56:09 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0434D130EFB for ; Sat, 14 Jul 2018 14:56:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=googlemail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bvnDxlOjeqEY for ; Sat, 14 Jul 2018 14:56:04 -0700 (PDT) Received: from mail-qt0-x22d.google.com (mail-qt0-x22d.google.com [IPv6:2607:f8b0:400d:c0d::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 924E012F1AC for ; Sat, 14 Jul 2018 14:56:04 -0700 (PDT) Received: by mail-qt0-x22d.google.com with SMTP id d4-v6so14566862qtn.13 for ; Sat, 14 Jul 2018 14:56:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20161025; h=subject:to:references:from:message-id:date:mime-version:in-reply-to :content-transfer-encoding; bh=Ft21uOwpBg9Qci9I6XuaJMddJDskDaK30O6wYE9w6qY=; b=T9yFkNG8xqAKAsPstzPcugPgxhOZJnHtpTuP3y3a1tUbW/VXxXgHsP7f0HwFaqe3M3 c/3iEVbecG4y3YRrqWwZUIfsEFXmqmmvn1yZrtxGqAHu5Zq4WvpHcFmg4n2N/qWNMOFu 56Yawoa0wJ9fxwzVKa3vE+IpiDGfroQCZZcC46yn0xTSHsF+b4VhCEheh9rlU7rVEuyg VXO79mj2mZXf9MwCEmWhYQEdVPcULIB0qhTC0ZMIUrDzSuju0yo8J5ArbYoUoQVdGZAp JV9P0y1sx3joQr+tK5ndmUvsJvpFOTyMdCcoIoWK484jJYFhIwG6TiI90JPSTp4qctSl pyEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :mime-version:in-reply-to:content-transfer-encoding; bh=Ft21uOwpBg9Qci9I6XuaJMddJDskDaK30O6wYE9w6qY=; b=fyp+SSOwALHfeoxxCsT0GPm6mR7r6W45g7aIZXvN+rXaoJDHHQEnimWHjlkVTpnuKs h19zjqZjHmx6fJ9LdroB03Ki1pl2u9B54U/B22ZLRuBgoA4fK2sMnUGqut1iGpw4umck nxU7hnqJuT92vckAIUOfHA3KMEf0qhn1i6H0IOTi8qj9wFkx4c+SzBRj9MHOKy32T7j1 6M0NrYhkMvzsq0NUveSwqnPFsplvEDaC88zXG8JWz+Q0jLkMr3P6cCcBIMbhcDjyIh6p fKg/bKUQ2GxjNPJR5bbQZWS3X96c57NhYSncrtuXLdFN+WKqlWCvq8t7FqhdJ2x7wK9+ Z4rA== X-Gm-Message-State: AOUpUlH8ynuZzQXcOzgGMwHp9uLj0lSSEOBhJWF8tWE2HIegLtqSmVyL GS1YNfwo40wJtUr+3FPyBnptSAMj X-Google-Smtp-Source: AAOMgpeHviCcQBJ07keP6zMtAfdDybjr3ycUBzHm4+RxWtlvunIdtPuMY197uL/RHFfgRUUGAgQ0oA== X-Received: by 2002:a0c:80a8:: with SMTP id 37-v6mr12437030qvb.13.1531605363299; Sat, 14 Jul 2018 14:56:03 -0700 (PDT) Received: from [127.0.0.1] ([2604:9a00:2010:a08d:10::23]) by smtp.googlemail.com with ESMTPSA id x7-v6sm15297962qtc.66.2018.07.14.14.56.01 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 14 Jul 2018 14:56:02 -0700 (PDT) To: Codec Encoding for LossLess Archiving and Realtime transmission References: From: Andreas Rheinhardt Message-ID: Date: Sat, 14 Jul 2018 21:55:00 +0000 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [Cellar] AV1 mapping update X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Jul 2018 21:56:08 -0000 Hello, what I have to say about the "seeking with delayed random access points" topic will be said in a separate email as a reply to your other email. Here is the rest: 1. "The [timing_info_present_flag] of the Sequence Header OBU SHOULD be 0. Even when it is 1 the presentation time of the Frame Header OBUs in Blocks should be discarded. In other words, only the timestamps given by the Matroska container MUST be used." a) This clause should not be in the section about the `CodecPrivate` as it pertains to all `Sequence Header OBUs`, not just the one in the `CodecPrivate`. b) I don't see a reason why the bitstream should be cleaned of these values. This isn't done for H.264 or other codecs either. c) If you want to keep this recommendation in, then we should merge it with the recommendation to discard [temporal_point_info] (after all, setting [timing_info_present_flag] to 0 automatically does the same with [decoder_model_info_present_flag] and then [[temporal_point_info]] mustn't be present anyway) that uses [frame_presentation_time]. It's probably good to adapt the mp4 version: "The presentation times of AV1 samples are given by the Matroska container. The [timing_info_present_flag] in the `Sequence Header OBU` (in the `CodecPrivate` or in the bitstream) SHOULD be set to 0. If set to 1, the [timing_info] structure of the `Sequence Header OBU`, the [frame_presentation_time] and [buffer_removal_time] fields of the `Frame Header OBUs`, if present, SHALL be ignored for the purpose of timed processing of the Matroska file." 2. Steve Lhomme: >> The current version is: >> "The OBUs in the Block follow the [Low Overhead Bitstream Format >> syntax]. They SHOULD have the [obu_has_size_field] set to 1 except for >> the last OBU in the sample, for which [obu_has_size_field] MAY be set to >> 0, in which case it is assumed to fill the remaining of the sample." >> >> If one interprets the first sentence as meaning "The OBUs in the Block >> MUST follow the [Low Overhead Bitstream Format syntax]", then given that >> this syntax mandates [obu_has_size_field] to be equal to 1 the first >> part of the second sentence is redundant (given that MUST is stronger >> than SHOULD) and the second part is again in contradiction to/voided by >> the first sentence because the first sentence doesn't allow >> "[obu_has_size_field]" set to zero at all. >> If one interprets the first sentence as not conveying a MUST, then it is >> allowed (albeit strongly discouraged) to use [obe_has_size_field] equal >> to 0 for an OBU that is not the last OBU in the sample. This is not what >> we want, isn't it? How about: >> "The OBUs in the `Block` MUST follow the [Low Overhead Bitstream Format >> syntax] with the possible exception of the last OBU of a `Block` for >> which [obu_has_size_field] MAY be set to 0, in which case it is assumed >> to fill the remainder of the `Block`." > > MUST is wrong IMO if you add an exception. Then it should be a SHOULD > and explain the cases where it shouldn't. I left it out of the first > sentence on purpose because the "normative" SHOULD is on the next > sentence and the one that should apply. My interpretation of my proposal is: "If the exception doesn't apply, then it is a MUST (a real MUST), otherwise the exception (with its MAY) applies." I thought this is what is really wanted: After all, there used to be the sentence "The OBUs in the `Block` MUST follow the __[Low Overhead Bitstream Format syntax]__." And the commit message of the commit that changed this part to the way it currently is reads: "av1: the low overhead format must be used in Blocks except for the last OBU" Anyway, given that my earlier proposal can apparently be misunderstood in ways I didn't imagine let me rephrase it: "If an OBU is not the last OBU in a `Block`, it MUST follow the [Low Overhead Bitstream Format syntax] (i.e. it MUST have [obu_has_size_field] set to 1); the last OBU in a `Block` MAY have [obu_has_size_field] set to 0, in which case it is assumed to fill the remainder of the `Block`." I can't think of any ambiguity in the above wording. 3. >>>> 4. "ReferenceBlocks inside a BlockGroup MUST reference frames according >>>> to the [ref_frame_idx] values of frame that is neither a KEYFRAME nor an >>>> INTRA_ONLY_FRAME.": The problem with this sentence is that >>>> [ref_frame_idx] needn't be present. It depends upon >>>> [frame_refs_short_signaling] and [show_existing_frame]. If one uses a >>>> Block inside a Blockgroup and if [show_exsting_frame] equals one one >>>> should reference the block that contained the showable frame that is now >>>> output (and that this should be the only `ReferenceBlock` written). In >>>> case of [frame_refs_short_signaling] == 1 the obvious candidates for >>>> `ReferenceBlocks` are the blocks containing the `last_frame_idx` and >>>> `gold_frame_idx` that are explicitly signalled. If I am not mistaken, >>>> then there are also other reference frames that are not explicitly >>>> signalled, but computed. I don't know if we should really write a >>>> `ReferenceBlock` entry for every reference as the current proposal seems >>>> to imply. This would be quite a bit of overhead for no gain (and >>>> furthermore, it would complicate muxers that would have to compute the >>>> references that are not explicitly signalled in case that >>> >>> This is how `ReferenceBlock` is supposed to be used. So a muxer that >>> has no idea of any codec can cut a file and keep the relevant >>> references. So they all have to be there. It's one of the reasons >>> SimpleBlock was added, to simplify things a little (and reduce >>> overhead). >>> >> Actually a muxer can cut a file if it just knows the keyframes, the >> decoding order and the display order. It doesn't need to have complete >> information about reference frames. After all, one can cut files that >> exclusively use `SimpleBlocks`. > > I'm not sure it's true for modern codec where the referenced frame may > be older than the previous keyframes. Also the ReferenceBlock allows > picking only the frames necessary to render that particular frame, > regardless of the internals of the codec. > If by cut a file one means to specify an interval of the (output) movie that should be preserved, then I have to correct myself: The criteria I mentioned only work if the keyframes have the property that any block with a timestamp bigger than the timestamp of the keyframe can be correctly decoded if one starts decoding from the keyframe on. If not, one also needs the information from which frame on the output is ok. One doesn't need every reference for every frame for that, one could also convey this information with one of the ways I proposed. 4. >>>> 5. AV1 may use spatial scalability and/or temporal scalability. What do >>>> we make of these? They are currently not forbidden if I am not mistaken, >>>> but if e.g. the spatial dimensions of different layers disagree, the >>>> `PixelWidth` and `PixelHeight` values can't be true for all layers. >>>> Matroska seems to be missing some features here. >>> >>> Our spec says that the Sequence Header OBU should be valid for all >>> frames. That can't be used for spatial scalability. We don't support >>> that mode for now. >>> >> Then this should be explicitly stated in the codec mapping. And I also >> fail to see why the fact that the Sequence Header OBU should be valid >> for all frames should be incompatible with spatial scalability (after >> all, in my reading of the spec the various share the same Sequence >> Header OBU). > > I didn't fully understand how the spatial scalability works. If the > same Sequence Header OBU supports it then we can support it. After all > the PixelWidth/Height use the "maximum" width/height. But internally > it may be less. > It uses the same `Sequence Header OBU`. But it is not confined to something internal to the codec. Different spatial layers can have different output resolutions; in fact even without this the frame dimensions may change from frame to frame. And if I am not mistaken, then this is not something internal to the codec: The output pictures (that have to be scaled by the renderer) can have varying dimensions, too. Currently the segment restrictions contain the following clause: "Matroska doesn't allow dynamic changes within a codec for the whole Segment. The parameters that should not change for a video Track are the dimensions and the CodecPrivate." I take this as meaning that all output frames must have the same dimension and I think (this point is not clearly stated) it is meant to be that said output dimensions coincide with what must be put into `DisplayWidth` and `DisplayHeight` so that one actually has the added restriction for AV1 in Matroska that every output frame has the same width as [max_frame_width_minus_1]+1 (similar for height). But what I'd like to know is how much of AV1 will probably be excluded from Matroska by these requirements? Could we contact some of the codec designers and ask them about their opinions on this? (Honestly, "we" probably means "you" as you are the chairman of Matroska.) And maybe other things where we might have questions for them. (We should collect the questions first.) 5. >>> My wording would be: >>> "Upon seeking to a keyframe the player/demuxer MUST prepend the `Block` >>> with the `Sequence Header OBU` contained in the `CodecPrivate`. >>> A muxer MUST make sure that the correct `Sequence Header OBU` is in >>> force both during linear access and also after seeking to a keyframe >>> `Block`. So in particular a keyframe `Block` where a `Sequence Header >>> OBU` that is not bit-identical to the one in the `CodecPrivate` is in >>> force for the decoding of the first frame contained in said `Block` MUST >>> contain the `Sequence Header OBU` that is in force for the decoding of >>> the first frame contained in said `Block` in front of the first frame >>> contained in said `Block`." >>> >>> One could also relax this a bit and only make the correct linear access >>> a MUST and the rest a SHOULD. This might be useful for applications >>> where seeking isn't desired (although it really should be included even >>> for those scenarios to support resuming playback after a transmission >>> error). >> >> OK, I'll try to add something for seeking. > > Actually we already have it in the Segment Restrictions: > > Given a `Sequence Header OBU` can be omitted from a `Block` if > __[decoder_model_info_present_flag]__ is 0 and it is bit identical to > the one found in `CodecPrivate`, when seeking to a keyframe, that > omitted `Sequence Header OBU` MUST be added back to the bitstream for > compliance with the Random Access Decoding section of the [AV1 > Specifiations](#av1-specifications). No, we haven't. But before I come to this here are two more points that I would like to mention first: a) [decoder_model_info_present_flag] controls more than just whether [operating_parameters_info] is present: It also influences the presence of [temporal_point_info] which is AV1's way of storing the timestamps (possibly VFR) of the video. The real condition whether the `Sequence Header OBUs` of a CVS can change is whether [decoder_model_present_for_this_op[i]] is 1 for an operating point. But in my proposal it doesn't matter anymore anyway, as the way of specifying when a `Sequence Header OBU` has to be present is entirely independent of the one track = one CVS restriction (it would work just as well if this restriction were dropped). b) The current proposal mandates that one track can only consist of one (subset of a) CVS in a suboptimal manner (if it mandates it at all): It says: "Matroska doesn't allow dynamic changes within a codec for the whole Segment. The parameters that should not change for a video Track are the dimensions and the CodecPrivate." The `CodecPrivate` can't change in a valid Matroska track anyway, because it can only have one. Therefore this requirement is empty and the only requirement of one track = one CVS is in the next sentence: "The first Sequence Header OBU of a CVS is stored in the CodecPrivate of a Track. So this AV1 Track has the same requirements as the CVS." Here the "so" is currently a non-sequitur. Better wording: "Matroska doesn't allow dynamic changes within a `Track` for the whole `Segment`. Therefore the `Sequence Header OBUs` of the `Track` (both the one in the `CodecPrivate` as well as the in-band `Sequence Header OBUs`) MUST adhere to the restriction that the `Sequence Header OBUs` of a `CVS` must fulfill: Their content MUST be bit-identical each time a `Sequence Header OBU` appears except for the contents of [operating_parameters_info]. Furthermore the dimensions of all output frames MUST be equal." c) Here is what the proposal currently has to say about `Sequence Header OBUs`: "Sequence Header OBUs SHOULD be omitted when they are bit-identical to the one found in CodecPrivate and [decoder_model_info_present_flag] is 0 and the previous Sequence Header OBUs in the bistream was also bit-identical to the one found in CodecPrivate. They can be kept when encryption constraints require it." "The first Sequence Header OBU of a CVS is stored in the CodecPrivate of a Track. So this AV1 Track has the same requirements as the CVS." "If the [decoder_model_info_present_flag] of this Sequence Header OBU is set to 1 then each keyframe Block MUST contain a Sequence Header OBU before the Frame Header OBUs." "Given a Sequence Header OBU can be omitted from a Block if [decoder_model_info_present_flag] is 0 and it is bit identical to the one found in CodecPrivate, when seeking to a keyframe, that omitted Sequence Header OBU MUST be added back to the bitstream for compliance with the Random Access Decoding section of the AV1 Specifiations." "A SimpleBlock MUST be marked as a Keyframe only if the first Frame OBU in the Block has a [frame_type] of KEY_FRAME and the SimpleBlock contains a Sequence Header OBU or if the Sequence Header OBU is correctly omitted (see above)." "A Block inside a BlockGroup MUST use ReferenceBlock elements if the first Frame OBU in the Block has a [frame_type] other than KEY_FRAME or the Block doesn't contain a Sequence Header OBU when it should not be omitted." Clause 1 and clause 4 only apply when [decoder_model_info_present_flag] is 0. In this case (by our requirements of each track only containing a single CVS) all `Sequence Header OBUs` must be bit-identical; in particular, the part of clause 1 dealing with the previous `Sequence Header OBU` is unnecessary because it is automatically fulfilled. Clause 3 directly contradicts your claim that the current proposal already allows to strip away some of the `Sequence Header OBUs` when they are bit-identical to the one in the `CodecPrivate` and redundant for linear access. Clause 2 is actually an unnecessary restriction. It might be that the very first `Sequence Header OBU` is rare and that one could save more bytes when putting a different one in the `CodecPrivate`. In any case I fail to see why the first `Sequence Header OBU` should be treated specially in the standard; that it allows for simpler muxers will of course make most muxer put the first `Sequence Header OBU` in the `CodecPrivate`, no doubt about that. But if one requires this, then splitting AV1 tracks in Matroska will be more complicated: One has to change the `CodecPrivate` to what is the `Sequence Header OBU` used by the first frame. Even ignoring this I regard the whole requirements as confusing. The requirements should not be stated in terms of [decoder_model_info_present_flag] (or [decoder_model_present_for_this_op[i]]); instead one should state them as a corollary of the general principle that the necessary codec initialisation data/extradata must be available to the decoder during linear access as well as after random access (to a keyframe) and not hide this fact behind some conditions involving the aforementioned flags. It's much more intelligible this way. And it encourages muxers not to hard-code to treat the cases of [decoder_model_info_present_flag] being 0 or 1 (or [decoder_model_present_for_this_op[i]] being 0 for all applicable i or not) separately, but to use a system that works even if multiple CVS were allowed in the future. d) Here is a proposal for said section in case we make it a MUST that keyframes must have the correct `Sequence Header OBU` (whether because it is there or because it is prepended from the `CodecPrivate`): "Upon seeking to a random access point (RAP) or starting playback from the beginning the player/demuxer MUST prepend the `Block` where decoding starts with the `Sequence Header OBU` contained in the `CodecPrivate`. Afterwards playback proceeds normally including consuming any `Sequence Header OBUs` that are found in the bitstream. Seeking to a non-RAP is undefined and not recommended. A muxer MUST make sure that when using a conformant demuxer/player the correct `Sequence Header OBU` is active both during linear access and also after seeking to a random access `Block`. In particular, if a `Sequence Header OBU` that differs from the `Sequence Header OBU` in the `CodecPrivate` is active during consuming of the first `Frame Header OBU` of a random access point sample, then said sample MUST contain said `Sequence Header OBU` in front of the first `Frame Header OBU`. The muxer MAY omit (strip away and discard) `Sequence Header OBUs` provided the above criteria are still fulfilled afterwards. Note: A `Sequence Header OBUs` that fulfills any of these criteria can be omitted without changing compliance to the above criteria: If, after potentially stripping away all `Temporal Delimiter OBUs` there are more than one `Sequence Header OBUs` immediately after each other, then all but the last of these `Sequence Header OBUs` can be omitted. A `Sequence Header OBU` that differs from the `Sequence Header OBU` in the `CodecPrivate` can be omitted if there was a preceding `Sequence Header OBU` in the bitstream, if the preceding `Sequence Header OBU` was bit-identical to the current `Sequence Header OBU` and if the current `Sequence Header OBU` is not the first OBU after a `Temporal Delimiter OBU` that starts a temporal unit whose corresponding `Block` is a random access point. A `Sequence Header OBU` that is bit-identical to the `Sequence Header OBU` in the `CodecPrivate` can be omitted if there was no preceding `Sequence Header OBU` or if there was a preceding `Sequence Header OBU` and the previous `Sequence Header OBU` was bit-identical to the current `Sequence Header OBU`." If this is adopted, clauses 2-4 from c) above should be omitted and clauses 5 and 6 can be adapted, too. That's it from me today. - Andreas Rheinhardt From nobody Sat Jul 14 18:19:19 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B0CC8130E6C for ; Sat, 14 Jul 2018 18:19:18 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.901 X-Spam-Level: X-Spam-Status: No, score=-6.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Yybbu35ZZcQp for ; Sat, 14 Jul 2018 18:19:17 -0700 (PDT) Received: from smtp.mozilla.org (mx2.scl3.mozilla.com [63.245.214.156]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 10F6B1274D0 for ; Sat, 14 Jul 2018 18:19:17 -0700 (PDT) Received: from localhost (localhost6.localdomain [127.0.0.1]) by mx2.mail.scl3.mozilla.com (Postfix) with ESMTP id C9180C0227 for ; Sun, 15 Jul 2018 01:19:16 +0000 (UTC) X-Virus-Scanned: amavisd-new at mozilla.org Received: from smtp.mozilla.org ([127.0.0.1]) by localhost (mx2.mail.scl3.mozilla.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KfpmwxEuSfHs for ; Sun, 15 Jul 2018 01:19:16 +0000 (UTC) Received: from [192.168.0.104] (modemcable234.246-178-173.mc.videotron.ca [173.178.246.234]) (Authenticated sender: tterriberry@mozilla.com) by mx2.mail.scl3.mozilla.com (Postfix) with ESMTPSA id C3712C0226 for ; Sun, 15 Jul 2018 01:19:15 +0000 (UTC) To: Codec Encoding for LossLess Archiving and Realtime transmission References: From: "Timothy B. Terriberry" Message-ID: <10d56d2a-3053-6069-7805-54bb4fd1d4e0@xiph.org> Date: Sat, 14 Jul 2018 18:19:11 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 SeaMonkey/2.49.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [Cellar] AV1 mapping update X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jul 2018 01:19:19 -0000 Andreas Rheinhardt wrote: > "The presentation times of AV1 samples are given by the Matroska > container. The [timing_info_present_flag] in the `Sequence Header OBU` > (in the `CodecPrivate` or in the bitstream) SHOULD be set to 0. If set I'll ask the usual question: when is it reasonable to violate this SHOULD? > width as [max_frame_width_minus_1]+1 (similar for height). But what I'd > like to know is how much of AV1 will probably be excluded from Matroska > by these requirements? Could we contact some of the codec designers and > ask them about their opinions on this? (Honestly, "we" probably means Hi, I am one of the codec designers. I think the expectation is that DisplayWidth and DisplayHeight will remain constant for a coded video sequence (and these should match [max_frame_width_minus_1] + 1 and [max_frame_height_minus_1] + 1). If the output dimensions of a frame change, it should be scaled to the display resolution before being displayed. Spatial scalability depends on this (at least if you ever intend to drop some of the layers, and if you don't then there is no point to using scalability). Even without spatial scalability, it is sometimes desirable to reduce the coded resolution to maintain quality while fitting within the instantaneous channel bandwidth. That mostly applies to live/low-latency streams, but I think an important use case for Matroska is to be able to capture such streams for long-term storage (for example, via WebRTC and the MediaStream Recording API in web browsers). > [operating_parameters_info]. Furthermore the dimensions of all output > frames MUST be equal." As such, I disagree with this part (as an individual). > "A SimpleBlock MUST be marked as a Keyframe only if the first Frame OBU > in the Block has a [frame_type] of KEY_FRAME and the SimpleBlock > contains a Sequence Header OBU or if the Sequence Header OBU is > correctly omitted (see above)." This is mathematically correct (assuming you really mean "only if" and not "if and only if), but I think it could be easily misread. Perhaps "MUST NOT... unless..."? > Seeking to a non-RAP is undefined and not recommended. RECOMMENDED is an RFC 2119 keyword (but "NOT RECOMMENDED" is not explicitly listed as one). You might want to rephrase to avoid confusion. From nobody Sun Jul 15 10:28:32 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2E5AF130E74 for ; Sun, 15 Jul 2018 10:28:31 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=googlemail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TtYNkVA0oiv2 for ; Sun, 15 Jul 2018 10:28:28 -0700 (PDT) Received: from mail-wm0-x22a.google.com (mail-wm0-x22a.google.com [IPv6:2a00:1450:400c:c09::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 373711294D0 for ; Sun, 15 Jul 2018 10:28:28 -0700 (PDT) Received: by mail-wm0-x22a.google.com with SMTP id z13-v6so13557893wma.5 for ; Sun, 15 Jul 2018 10:28:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20161025; h=subject:to:references:from:message-id:date:mime-version:in-reply-to :content-transfer-encoding; bh=VnZy5uLpq/W8Svpkg7iZo1XMGU9iuF1CSaCwsf5AhJ8=; b=n5uhc2+/RLFWvMXNzarEX8oqxtKm7e0tnKVP80I/Bk0aKWXXdfbSHeSOauMt3rzwm5 Ltg7XesyQ6qUJE21hNDBxlZ33jeMhSIuF/iX8fUGSXkP/HZfX9R7sICJZiemq0T/vrth x3E14u/98kSDqRaBWZ8gNq2MgEWtjVYPdFQiujcsW2E6Vh9wP7IS2kEl7ZUyHhKdAjrx LX9qYOqJRJrdrPLJUgeiSb/rEjqvEQAOBfdaVEqhgWiC7dn55yw79m9hEhjpR9bmUzrx UKbXb9OmwBr1GTfJiXiYh+NCZwZonMojLAkKtbBOE//FOjZawT7Rp7ppuLx7rEP1Fm8O dHbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :mime-version:in-reply-to:content-transfer-encoding; bh=VnZy5uLpq/W8Svpkg7iZo1XMGU9iuF1CSaCwsf5AhJ8=; b=lCjXfgLhIRYJDUTmlr/vsjJ52eVOUClBM35uwrxKhOU0/hs08KwQWkJnjUpFTC3rYD /Qw8kVykbUDySgqi0rlAkyisCidem1DfHDyfpgNteAfbx922olfIZXMm1srObx29S5IA Loam718cF9mLobBpta5qOswBW9Ev4YTRBe+Yvotnx9xNyxweMdfVTOKp53AmRPpCcRVf edJyhyQVxe/ngUJtf2hBvOajej0Ntcq3ogI+P+mbhkW9id6ygHxYI/yQFGOiWAX2oLNC mOyq5CmV7rUn2xETb11/lcTTA91cLEQcQYZBX+Xnu/H7JRBUzEminf7NDDHI+9Y4RAlu pH/g== X-Gm-Message-State: AOUpUlGJ/1UfLtdJxlsbSzCRTXVADARX0OX3LGLixxU2C8gvcU4sOOR5 1IV1fS5QMGtVsAJQysr+Mbbq9jTQ X-Google-Smtp-Source: AAOMgpdtA3KnMGrdf2t2EwCO2fqRQcDYe8WKPNWQLosHiVS4d8WpNJNc/sEjyjL0U1cec0VyzhmkXA== X-Received: by 2002:a1c:5e08:: with SMTP id s8-v6mr8443268wmb.88.1531675706409; Sun, 15 Jul 2018 10:28:26 -0700 (PDT) Received: from [127.0.0.1] ([178.175.135.102]) by smtp.googlemail.com with ESMTPSA id g75-v6sm13588963wmd.38.2018.07.15.10.28.24 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 15 Jul 2018 10:28:25 -0700 (PDT) To: cellar@ietf.org References: <10d56d2a-3053-6069-7805-54bb4fd1d4e0@xiph.org> From: Andreas Rheinhardt Message-ID: Date: Sun, 15 Jul 2018 17:27:00 +0000 MIME-Version: 1.0 In-Reply-To: <10d56d2a-3053-6069-7805-54bb4fd1d4e0@xiph.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [Cellar] AV1 mapping update X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jul 2018 17:28:31 -0000 Hello, Timothy B. Terriberry: > Andreas Rheinhardt wrote: >> "The presentation times of AV1 samples are given by the Matroska >> container. The [timing_info_present_flag] in the `Sequence Header OBU` >> (in the `CodecPrivate` or in the bitstream) SHOULD be set to 0. If set > > I'll ask the usual question: when is it reasonable to violate this SHOULD? > every timestamp in Matroska is a multiple of 1 ns (this number is hard-coded) so that NTSC timings can't be exactly represented on the container level: E.g. the default duration field for 24/1.001fps content would indicate 41708333ns, although it is 41708333.33333333...ns. So by preserving the bitstream information it is possible to increase the certainity that 41708333ns actually means 24/1.001 fps. >> width as [max_frame_width_minus_1]+1 (similar for height). But what I'd >> like to know is how much of AV1 will probably be excluded from Matroska >> by these requirements? Could we contact some of the codec designers and >> ask them about their opinions on this? (Honestly, "we" probably means > > Hi, I am one of the codec designers. I think the expectation is that > DisplayWidth and DisplayHeight will remain constant for a coded video > sequence (and these should match [max_frame_width_minus_1] + 1 and > [max_frame_height_minus_1] + 1). If the output dimensions of a frame > change, it should be scaled to the display resolution before being > displayed. Spatial scalability depends on this (at least if you ever > intend to drop some of the layers, and if you don't then there is no > point to using scalability). Even without spatial scalability, it is > sometimes desirable to reduce the coded resolution to maintain quality > while fitting within the instantaneous channel bandwidth. That mostly > applies to live/low-latency streams, but I think an important use case > for Matroska is to be able to capture such streams for long-term storage > (for example, via WebRTC and the MediaStream Recording API in web > browsers).> >> [operating_parameters_info]. Furthermore the dimensions of all output >> frames MUST be equal." > > As such, I disagree with this part (as an individual). > a) I don't like restrictions on the content that can be put into Matroska either. The reason why I put it in is because this is (in my understanding) currently an absolute requirement of Matroska (`PixelWidth` and `PixelHeight` are mandatory and valid for every frame). Given that decoders will have to support these varying output dimensions anyway (it is a mandatory part of AV1, isn't it?), it would be possible to drop this requirement, but for this the specifications of both Matroska and Webm would have to be changed. I'm not sure if everyone is ok with this. b) Just to be sure: "Normally" the size of an output frame is given by [max_frame_height_minus_1]+1 and [max_frame_width_minus_1]+1, although part 7.18 that deals with the output process says that the width is given by `UpscaledWidth`. The reason for this is that for intra frames, key frames and some inter frames the initial `FrameWidth` (directly inferred from [max_frame_width_minus_1] or [frame_width_minus_1]) is saved to `UpscaledWidth` and then `FrameWidth` gets overwritten with a smaller value (that is the value that is used during much of the decoding process, but only internally, so that we can ignore it). Is that correct? (I have to admit I thought for a time that the output dimensions also depend on the superres parameters and that they can be even bigger than [max_frame_height/width_minus_1]+1.) >> "A SimpleBlock MUST be marked as a Keyframe only if the first Frame OBU >> in the Block has a [frame_type] of KEY_FRAME and the SimpleBlock >> contains a Sequence Header OBU or if the Sequence Header OBU is >> correctly omitted (see above)." > > This is mathematically correct (assuming you really mean "only if" and > not "if and only if), but I think it could be easily misread. Perhaps > "MUST NOT... unless..."? > I agree that this should be changed and your proposal is certainly an improvement. However this is not the only thing in this definition that needs to be looked at (see 4. below). >> Seeking to a non-RAP is undefined and not recommended. > > RECOMMENDED is an RFC 2119 keyword (but "NOT RECOMMENDED" is not > explicitly listed as one). You might want to rephrase to avoid confusion. > Ok: "Seeking to a non-RAP is undefined and discouraged." (We also need to explicitly add what a RAP is.) And because you are one of the AV1 designers, here are a few questions: 1. Section 7.5 includes the following: "If scalability is not being used (OperatingPointIdc equal to 0), then all frames are part of the operating point. The following constraints must hold: The first frame header must have frame_type equal to KEY_FRAME and show_frame equal to 1. Each temporal unit must have exactly one shown frame. If scalability is being used (OperatingPointIdc not equal to 0), then only a subset of frames are part of the operating point. For each operating point, the following constraints must hold: The first frame header that will be decoded must have frame_type equal to KEY_FRAME and show_frame equal to 1. Every layer that has a coded frame in a temporal unit must have exactly one shown frame that is the last frame of that layer in the temporal unit." I am wondering whether the the last two requirements only apply if scalability is being used (OperatingPointIdc not equal to 0) or in any case. The actual reason why I am asking myself this is that in case that scalability is not being used there is no restriction that a shown frame is the last frame in that temporal unit. I don't see a reason why there should be another frame afterwards in the same temporal unit, but is it actually allowed? 2. Would it be legal to code a frame with [show_frame] 0, [showable_frame] 1 followed (perhaps immediately) by a frame header in the same temporal unit that has [show_existing_frame] set to 1 and outputs the frame just coded? If yes, is there any reason to ever use it? I don't see any. According to the definition, this temporal unit contains both delayed random access point and a key frame dependent access point and according to 7.6.3 a decoder is not required to be able to handle such a situation for random access. But obviously it is as good as a key frame random access point. 3. Would it be legal to have a temporal unit that contains a frame that is not shown (but might be declared as showable) followed by a keyframe with [show_frame] equal to 1? I'm asking because both the current proposal as well as the mp4/ISOBMFF definition of sync sample require the keyframe to be the first frame in the temporal unit. 4. You have only criticized the "MUST only" wording of the definition of keyframe simpleblocks. You said nothing about the requirement that it is the first frame that needs to be of type KEY_FRAME. The standard doesn't require this and I fail to see why this should be needed. Moreover, in contrast to 2. and 3. there seems to be a usecase for this: Think about a situation where all reference frames available before/at the start of the temporal unit that contains a delayed RAP constitute a very good set of reference frames for a frame that is to be shown between the output frame of the temporal unit that contains the delayed RAP and the output frame of the temporal unit that contains the dependent keyframe recovery point, but where the combination of the dependent RAP frame + all but one of the reference frames available at the start of the temporal unit that contains the delayed RAP frame constitute a worse set of reference frames (regardless of which of the initial reference frames gets overwritten by the delayed RAP frame). Then it may make sense to first code this frame as a showable frame using all the reference frames and then code the delayed RAP frame followed by the frame that is actually output for the temporal unit containing the delayed RAP. Is my usecase sound and should the "first"-requirement be dropped? 5. What do you think of the restriction of a Matroska track to a (subset of) a CVS? - Andreas Rheinhardt From nobody Sun Jul 15 15:20:50 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DFDFF130E8C for ; Sun, 15 Jul 2018 15:20:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=googlemail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ey2DgI_sINUT for ; Sun, 15 Jul 2018 15:20:46 -0700 (PDT) Received: from mail-wr1-x442.google.com (mail-wr1-x442.google.com [IPv6:2a00:1450:4864:20::442]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4D894130E88 for ; Sun, 15 Jul 2018 15:20:46 -0700 (PDT) Received: by mail-wr1-x442.google.com with SMTP id a3-v6so20795416wrt.2 for ; Sun, 15 Jul 2018 15:20:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20161025; h=subject:to:references:from:message-id:date:mime-version:in-reply-to :content-transfer-encoding; bh=hRkfIk2PNO8D4kru70fJ6RRUcNWIellb6URYh1ukfKo=; b=AIGic6c3Nqx0GE2ai3uzv7F3tXUIPwjanjVXFOKbykvS10fNbNwSQmNhgzE0pcEDq7 lmKnGBZ2QyHaOzRgRke3wIzW1+bW9ZS71WT8cCeZaTkQO/umYOCL+6TEOfBfItsJ9K+t kDBvHxPxdCLVNVbOJJSrkrdYYdO8g7lUlVGB7LhSKrYjibU8Mj14BF8thQMA2umVgwlp ee6qJSmy0GfUtac+gSEHy+2YoR/TAJWm2NnN6BV1pDUaGxRZSjjczdpdevnzG+7ym48g cB6ogppDRZFhb0IDNGXgv2Gv2dZe2vBim52Na1vMKp/G7naI1ELF3XaodzJlKrj13hLN IKUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :mime-version:in-reply-to:content-transfer-encoding; bh=hRkfIk2PNO8D4kru70fJ6RRUcNWIellb6URYh1ukfKo=; b=B36X9AkeTjFueVASHqHv/TCzCmi1OvXUU1akDUa23hO1v9OK/qzrmEm/DWV6Y2otJR sthLjZ4aT/elXVGviszITOfhveuua0kWxu7Q4+uSr//jilHh9u2ZEgMdEY2XdPvWtlcS tzihIaY/fYjeYiX8x4lwAxJisEFW1ud44QpgQUoMamCBocfd2QgnIA+pGeCiWk9J5PU0 omq8+SIYBPDLTm9wtlyviKu6YzyxbDRBgLV+jAk/46FJSLVLAmgMRekbd3M83wNEuBvQ BGbNVV2SmQDgBTP+DfuR9aOA41SN22OIWf3mxSnnHgJhPfsXYtTFCE1V0/iLYVUN6f2B eHag== X-Gm-Message-State: AOUpUlGEe5JL3GnPavkwhVoTWf9Y/YNYP0oRuCJ/dbstS2ouP6faaryM PpFcOWL0qLhc4cCZy1H2Go4k25Me X-Google-Smtp-Source: AAOMgpenuIVuKCSU1gyrvr4jbrqhmDEQORGUqETI0OKcgedA53IHYxhYCoq7xJMRLT10luJyAv1nzw== X-Received: by 2002:adf:e491:: with SMTP id i17-v6mr10868606wrm.145.1531693244508; Sun, 15 Jul 2018 15:20:44 -0700 (PDT) Received: from [127.0.0.1] ([2a00:1298:8011:212::165]) by smtp.googlemail.com with ESMTPSA id l7-v6sm10759186wmh.1.2018.07.15.15.20.43 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 15 Jul 2018 15:20:43 -0700 (PDT) To: cellar@ietf.org References: From: Andreas Rheinhardt Message-ID: <62c29889-49f5-6634-049a-a2d73315bb3c@googlemail.com> Date: Sun, 15 Jul 2018 22:19:00 +0000 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [Cellar] AV1 seeking X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jul 2018 22:20:49 -0000 Hello, Steve Lhomme: > Using BlockReference we can actually know where to seek for these > particular frames to get all the frames they need. But most people use > SimpleBlock (or we could forbid its use for such streams) and it would > mean referencing all frames in the Cues which is not a good idea. > I actually have already made some proposals that allow the demuxer to know where to seek to. Here are two old ones and a new one (c)) which is my new favourite: a) One could reference the recovery points and the keyframe (i.e. non-delayed) RAPs in the cues. Both would be normal cue entries. The recovery points would have to be put into a `Block` in a `Blockgroup` and said `Blockgroup` must have a `ReferenceBlock` that points to the delayed RAP. Pro: Normally the delayed RAP and the recovery point end up in the same cluster and so hopefully the delayed RAP can be quickly accessed because it is still cached. Also this approach doesn't rely on anything deprecated in Matroska or unavailable in Webm. It can be generalized to the other gradual decoder refresh scenarios.* It has very low overhead. Con: Depending on the demuxer implementation this might result in two seeks; it is also not really very backward-friendly: If one has a delayed RAP at t_0, the recovery point at t_1 and the next RAP at t_2 with t_0 In MP4 they have [initial_presentation_delay_minus_one] in the > CodecPrivate. I did not understand it so far because it's not found in > the AV1 spec. But it seems to guarantee that to read frame 'f' you > need to decode X frame before that one. In our case that would be 5 to > have at least a decoded. > You completely misunderstood this field. It is the equivalent of the max_num_reorder_frames value from H.264. Let me explain it in MPEG terminology (with b-frames) as you probably have way more experience with this: Consider a decoder that can only decode one frame per unit of time and a stream like this (left to right is decoding order; the numbers are presentation order): I0 P2 B1 ... If one displayed the leading I frame immediately after decoding it, one would not have the right frame to display at time 1, because at that time only I0 and P2 has been decoded, not B1. Therefore one has to decode I0 and P2 before one outputs the first frame and and max_num_reorder_frames would be 1. The typical b-pyramid would require to decode the first three frames before the display of the first frame and max_num_reorder_frames would be 2. The [initial_presentation_delay_minus_one] is the AV1 analogue of this. This number is not an upper bound for the amount of temporal units between delayed RAP and recovery point/for the amount of frames shared between a GOP. Just look at this example: MPEG example I0 P5 B1 B2 B3 B4 I10 B6 B7 B8 B9 AV1 example (I kept the MPEG-naming with P and B to make it easier comparable to the above; furthermore the pointer *x denotes that a frame is showable and x without * is a frame header that outputs *x via show_existing_frame; square brackets are the delimiters of temporal units; I10 is the delayed RAP frame) [I0] [*P5 B1] [B2] [B3] [B4] [P5] [*I10 B6] [B7] [B8] [B9] [I10] ... This stream can have [initial_presentation_delay_minus_one] equal to 1, yet in order to seek to [I10] one has to decode the temporal unit [*I10 B6] (or at least the decodable keyframe in it) which is four temporal units in front of [I10]. > IMO this is a bad design because it mixes the container with the codec > (something done in the past in ogg with bad results). Seeking should > not ask the codec how it should be done. Also that means to mux such > an MP4 you'd need to scan the whole source first to verify the value > is correct or that information needs to be given to the muxer by the > encoder/packetizer (this information is not found in the stream). > > We have the option of not caring as it's allowed to leave it in grey > area by the AV1 spec. Or we can think of a proper "container" > solution. ie storing this information in the TrackInfo but not in the > CodecPrivate. We already have a CodecDelay but IMO it doesn't fit the > bill. Each frame has the timestamp modified by it. Which is not the > case here. We may need a SeekingDelay so that looking for a seek point > takes this amount in account. > yes, this is completely different than CodecDelay. There is a Matroska field that could be used for this: SeekPreRoll. But there are two problems with this: This value is valid for the whole track, whereas the amount of time between delayed RAP and recovery point can vary. And of course the needed value is generally unknown at the beginning of the muxing process. Not to mention the fact that actually SeekPreRoll is valid if one starts decoding at an arbitrary frame, not necessarily a (delayed) keyframe. Therefore a valid SeekPreRoll element for the track would have to be so large that it would be impractical. - Andreas Rheinhardt From nobody Sun Jul 15 23:33:59 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 428DB130E5C for ; Sun, 15 Jul 2018 23:33:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.909 X-Spam-Level: X-Spam-Status: No, score=-1.909 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, T_DKIMWL_WL_MED=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3At-yojEJ7vi for ; Sun, 15 Jul 2018 23:33:57 -0700 (PDT) Received: from mail-pg1-x52d.google.com (mail-pg1-x52d.google.com [IPv6:2607:f8b0:4864:20::52d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0176D130DE1 for ; Sun, 15 Jul 2018 23:33:56 -0700 (PDT) Received: by mail-pg1-x52d.google.com with SMTP id r5-v6so7079948pgv.0 for ; Sun, 15 Jul 2018 23:33:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=8v9hD9UZxqlRSz8hWiz8MaWUlZxF8rZGgvygqccCXfk=; b=bht66XSj/X+ZBuef8GnnSp94XeTovx+JI68GoWvU9/xd0TSQDPmfhc3lOZoG5igI8r an5ngzU+pZuszL3Rx0nMOdRmV/I2/Wqzu1t4vv9vKdCvT8HwoeMjgwWXQw6jMJkDK2tE U8rqYaPACYaSK5tlp+KTYw6Py8TFqCaBKIufw9tqlelPX9jjWwAr0IksrIilX2ZG7N/7 aYZFSzyVlhemp3O5YORKuXTia0hnAu2hDLkMQ6MJOPtLX2XmTMUYAqFTNsqAubgLCVyS Ckk4NTeTiG5fLFXhBsLH7RJ6JAMp+1afETKC5vz3XUIVaswmm9H1EXdb0mKueOPf+qH0 /g9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=8v9hD9UZxqlRSz8hWiz8MaWUlZxF8rZGgvygqccCXfk=; b=MGz6LsQVarcmDHbg/9kvap8uvgGPWL1+57SNpQMu6BA2l5bmoJoToO+yZXXVXqnf6O vxNx9VMi+toUE+DGooH65hU5e5jh9teRv1F1lA8URPNxBvD/CYsU3l8oRh7alx3Ufeay q6316lRaFRXCm62tt+MZO2w2E9YB/X69ctTRxwkz3ucZ7XRCpnREWxJD8nNTRw0oAtet 71qpbIdMyfYusHQC/0mewmmkRzlO9ObX8xVqse2U3oEXBsfaz42I3MAhDdjcjK6ijBof oDJw2/gvAOzzQhP6Qodl8INHVEdgPq8mH0LQYm4CFb5PB/FLh3G3YtsHosi4g6s+sqqi kT/A== X-Gm-Message-State: AOUpUlHiGMAvfJ1c4JG4e/GQq+D08Y/fBSXlVFyBad/HJXg2bYA0DOiY W8mWFIJsY5IG5MOMUKo6pWHwbU9iQ/XATcE11EFcv+ZW X-Google-Smtp-Source: AAOMgpd74ZkbpUSce3iPtycIH8xa1VVeFAmjtjArsi7hRQsVFDu6+kWOlUQbMHEaP6QnPoaCd8/BokRBvh4hVJtMlEU= X-Received: by 2002:a65:5245:: with SMTP id q5-v6mr14183720pgp.67.1531722836374; Sun, 15 Jul 2018 23:33:56 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a17:90a:9c13:0:0:0:0 with HTTP; Sun, 15 Jul 2018 23:33:55 -0700 (PDT) In-Reply-To: <71376bcb-ec70-4239-f2e5-e4202d660545@xiph.org> References: <71376bcb-ec70-4239-f2e5-e4202d660545@xiph.org> From: Steve Lhomme Date: Mon, 16 Jul 2018 08:33:55 +0200 Message-ID: To: "Timothy B. Terriberry" Cc: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cellar] AV1 seeking X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jul 2018 06:33:58 -0000 Ah yes, thanks. I overlooked that section (E.4.7) (for some reason the PDF searching doesn't catch everything so I'll you the github repo source). The field definition also says: "If not signaled then initial_display_delay_minus_1[ i ] = BufferPoolMaxSize - 1." and "it is indicated by a VBI slot that points to this frame buffer. BufferPoolMaxSize is equal to 10." So if I understand correctly by default the value of that field is assumed to be 9 if [initial_display_delay_present_flag] is set but not the individual values for each Operating Point. And it can go up to 15 (4 bits) meaning 16 frames are needed in the decoding buffer. 2018-07-14 22:31 GMT+02:00 Timothy B. Terriberry : > Steve Lhomme wrote: >> >> In MP4 they have [initial_presentation_delay_minus_one] in the >> CodecPrivate. I did not understand it so far because it's not found in >> the AV1 spec. But it seems to guarantee that to read frame 'f' you >> need to decode X frame before that one. In our case that would be 5 to >> have at least a decoded. > > > You want to search for InitialPresentationDelay in the AV1 specification. It > is derived from the decoder model info if it is available (see > decode_process()). -- Steve Lhomme Matroska association Chairman From nobody Mon Jul 16 01:01:52 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 00573130DD4 for ; Mon, 16 Jul 2018 01:01:51 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.909 X-Spam-Level: X-Spam-Status: No, score=-1.909 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, T_DKIMWL_WL_MED=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hzWWc40LIAHL for ; Mon, 16 Jul 2018 01:01:48 -0700 (PDT) Received: from mail-pl0-x230.google.com (mail-pl0-x230.google.com [IPv6:2607:f8b0:400e:c01::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C768C130DE0 for ; Mon, 16 Jul 2018 01:01:48 -0700 (PDT) Received: by mail-pl0-x230.google.com with SMTP id f4-v6so10725490plb.9 for ; Mon, 16 Jul 2018 01:01:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=BKeGucWJ50dAzJzYcUMduS3bIMOqf8nYUZ51gKR94Bs=; b=ZJ4Uj+FxBZfpxirSCwJ3lxMc6pK5Uk35ldSyleZ3MUOQzUM6rucC3FsokH4bxwQ5kA Z2cqXIBGdTE6HTD4r4LvcjCNtjPLyybsi4NCjUjSFBczdXm+Hy3eb6HGtL251pn75xoK HFo5uN1fJSeoq07N8Yr2j1O4m78Y4W8p4Itn4zbOF5c0Hbgyc4/etLZqP6QuyYzxXq7K RY+DuVTOaczHacqX0f/zMKWP6wuIKlfydHNkL5bKP7FutUy57yBv+m6fGl8ejD5IxYUf tdZ81E56D2HWDqlBejCjzQTE/MS7QJz8EnKQ6qR6K3ds07f9hMokwxM2unNiAkwqCKOY 52ow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=BKeGucWJ50dAzJzYcUMduS3bIMOqf8nYUZ51gKR94Bs=; b=fIo/CFTo8ZGSYwaEtJYZOEfMCIlE+WLLLqc2+LW4/NMKiUcgjprGNAJBm46EJHVJaL hJuHMVoYFOVQpN8ZUdJTM7oGqheTaX20l/J4wth8qsW5vIpVFtTwDHatL5ueyWYR6p6c f9GI/J2nzcZCcHXKbMXD5zvpZM7g08jIWXkwuUen85UuQ3CrWziPulT4HAMm3gm7XYxZ luEJXbyIOQBoC63JFi8q3bk3VGxRley370gPVOWXND9xkPTMkPMR4leOCy3t3dSoeM7V 4RYaG4BKVq94KKdpKsDGYXoZkxalRJNsFHn0H9IGIlotQURmZtmLpvEQCr6qjxwcuOjX sITg== X-Gm-Message-State: AOUpUlF1V+xqBrvhv4n3cq+uG5ZX34toeUqWIFQ/SSI9FzHH7fTbfkMy 98MuyvbdZNCTtizp0ZJHQtLvNE5jRkGIXxbZn6rit2tp X-Google-Smtp-Source: AAOMgpdyfmNXdVtwdvF3YkHP7tpelEdPwKDK4DgUSHiUGI/A+wlVZq9jVOiWm7Xxvm+CWa+z4reN3BE+vTKBtQ0ajlU= X-Received: by 2002:a17:902:2f43:: with SMTP id s61-v6mr15460711plb.274.1531728108250; Mon, 16 Jul 2018 01:01:48 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a17:90a:9c13:0:0:0:0 with HTTP; Mon, 16 Jul 2018 01:01:47 -0700 (PDT) In-Reply-To: <62c29889-49f5-6634-049a-a2d73315bb3c@googlemail.com> References: <62c29889-49f5-6634-049a-a2d73315bb3c@googlemail.com> From: Steve Lhomme Date: Mon, 16 Jul 2018 10:01:47 +0200 Message-ID: To: Andreas Rheinhardt Cc: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cellar] AV1 seeking X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jul 2018 08:01:51 -0000 Hi, 2018-07-16 0:19 GMT+02:00 Andreas Rheinhardt : >> In MP4 they have [initial_presentation_delay_minus_one] in the >> CodecPrivate. I did not understand it so far because it's not found in >> the AV1 spec. But it seems to guarantee that to read frame 'f' you >> need to decode X frame before that one. In our case that would be 5 to >> have at least a decoded. >> > You completely misunderstood this field. It is the equivalent of the > max_num_reorder_frames value from H.264. Let me explain it in MPEG > terminology (with b-frames) as you probably have way more experience > with this: Consider a decoder that can only decode one frame per unit of > time and a stream like this (left to right is decoding order; the > numbers are presentation order): > I0 P2 B1 ... > If one displayed the leading I frame immediately after decoding it, one > would not have the right frame to display at time 1, because at that > time only I0 and P2 has been decoded, not B1. Therefore one has to > decode I0 and P2 before one outputs the first frame and and > max_num_reorder_frames would be 1. The typical b-pyramid would require > to decode the first three frames before the display of the first frame > and max_num_reorder_frames would be 2. > The [initial_presentation_delay_minus_one] is the AV1 analogue of this. > This number is not an upper bound for the amount of temporal units > between delayed RAP and recovery point/for the amount of frames shared > between a GOP. Just look at this example: > MPEG example > I0 P5 B1 B2 B3 B4 I10 B6 B7 B8 B9 > AV1 example (I kept the MPEG-naming with P and B to make it easier > comparable to the above; furthermore the pointer *x denotes that a frame > is showable and x without * is a frame header that outputs *x via > show_existing_frame; square brackets are the delimiters of temporal > units; I10 is the delayed RAP frame) > [I0] [*P5 B1] [B2] [B3] [B4] [P5] [*I10 B6] [B7] [B8] [B9] [I10] ... > This stream can have [initial_presentation_delay_minus_one] equal to 1, A value of 1 means 2 frame delay. That would mean B6 needs *I10 and P5 ? > yet in order to seek to [I10] one has to decode the temporal unit [*I10 > B6] (or at least the decodable keyframe in it) which is four temporal > units in front of [I10]. That seems more like it. In the Frame presentation timing paragraph (E.4.7) it says: InitialPresentationDelay = Removal [ initial_display_delay_minus_1 ] + TimeToDecode [ initial_display_delay_minus_1 ] and PresentationTime[ 0 ] = InitialPresentationDelay PresentationTime[ j ] = InitialPresentationDelay + ( frame_presentation_time[ j ] - frame_presentation_time[ 0 ] ) * DispCT or in constant bitrate mode PresentationTime[ 0 ] = InitialPresentationDelay PresentationTime[ j ] = PresentationTime[ j - 1 ] + ( num_ticks_per_picture_minus_1 + 1 ) * DispCT And our `Block` timestamp derives directly from this [PresentationTime]. The delay is carried over every frame. So it does seem like we need to globally shift these timestamps for the Track. Possibly with `CodecDelay`. Basically each frame has its timestamp on which this delay has to be added (can't be negative). While `CodecDelay` is a positive value that needs to be substracted from the timestamps. It is available in WebM, because Opus needs it. Let's assume CodecDelay = InitialPresentationDelay The `Block` timestamps would be stored like this: Block[ 0 ] = 0 Block[ 1 ] = frame_presentation_time[ 1 ] * DispCT Block[ 2 ] = frame_presentation_time[ 2 ] * DispCT ... And on output of the demuxer we would get: Block[ 0 ] = -InitialPresentationDelay Block[ 1 ] = -InitialPresentationDelay + frame_presentation_time[ 1 ] * DispCT Block[ 2 ] = -InitialPresentationDelay + frame_presentation_time[ 2 ] * DispCT ... `CodecDelay` cannot be used here. But it's very close to what we need. It's not the SeekPreRoll either because the actual frame timestamps of each frame is affected, not just when seeking (although it may be needed for delayed RAP). We also need to figure out whether the Blocks need to be stored with or without the delay. If that's with the delay then we don't even need to care about it. The frames will just not start at 0 but that's already the case in the original stream. But that may just be the tricky part here, the reference point. The PresentationTime[ 0 ] doesn't start at 0 because some frames were needed to decode before getting actual usable data out of the decoder. But this is really the first frame to display so it would actually be 0 on the output of the demuxer. So it does seem exactly like what CodecDelay does: "CodecDelay is The codec-built-in delay in nanoseconds. This value must be subtracted from each block timestamp in order to get the actual timestamp." The `Block` timestamp would be PresentationTime[ 0 ] which isn't 0. But on the output Block[ 0 ] should really give 0. So we would have something like this in the file: Block[ 0 ] = InitialPresentationDelay Block[ 1 ] = InitialPresentationDelay + frame_presentation_time[ 1 ] * DispCT Block[ 2 ] = InitialPresentationDelay + frame_presentation_time[ 2 ] * DispCT ... And on the output of the demuxer we would have the following, when `CodecDelay` is [InitialPresentationDelay]: Block[ 0 ] = 0 Block[ 1 ] = frame_presentation_time[ 1 ] * DispCT Block[ 2 ] = frame_presentation_time[ 2 ] * DispCT ... It does look like what the internal AV1 display delay is trying to achieve. The `CodecDelay` also has this note on proper muxing: The value SHOULD be small so the muxing of tracks with the same actual timestamp are in the same Cluster. Because muxing is done based on the stored `Block` timestamp. It usually doesn't take in account CodecDelay (but it could/should). So I'll update my AV1 codec mapping saying `CodecDelay` is [InitialPresentationDelay] and how to use it. From nobody Mon Jul 16 07:05:55 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4C380130DFF for ; Mon, 16 Jul 2018 07:05:53 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.901 X-Spam-Level: X-Spam-Status: No, score=-6.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Bz0RZ88toslx for ; Mon, 16 Jul 2018 07:05:51 -0700 (PDT) Received: from smtp.mozilla.org (mx1.scl3.mozilla.com [63.245.214.155]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 14E59130E9E for ; Mon, 16 Jul 2018 07:05:51 -0700 (PDT) Received: from localhost (localhost6.localdomain [127.0.0.1]) by mx1.mail.scl3.mozilla.com (Postfix) with ESMTP id C718DC0434 for ; Mon, 16 Jul 2018 14:05:50 +0000 (UTC) X-Virus-Scanned: amavisd-new at mozilla.org Received: from smtp.mozilla.org ([127.0.0.1]) by localhost (mx1.mail.scl3.mozilla.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YPo5ehWvF-rO for ; Mon, 16 Jul 2018 14:05:50 +0000 (UTC) Received: from [31.133.142.81] (dhcp-8e51.meeting.ietf.org [31.133.142.81]) (Authenticated sender: tterriberry@mozilla.com) by mx1.mail.scl3.mozilla.com (Postfix) with ESMTPSA id 0402DBFFFB for ; Mon, 16 Jul 2018 14:05:49 +0000 (UTC) From: "Timothy B. Terriberry" To: Codec Encoding for LossLess Archiving and Realtime transmission References: <1ae2f2ce-3218-a19f-330e-0bd5407452a3@xiph.org> Message-ID: <6ce57954-4d49-730c-db4c-e85c80729c53@xiph.org> Date: Mon, 16 Jul 2018 07:05:49 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 SeaMonkey/2.49.7.0 MIME-Version: 1.0 In-Reply-To: <1ae2f2ce-3218-a19f-330e-0bd5407452a3@xiph.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [Cellar] Call for adoption of three new WG items X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jul 2018 14:05:53 -0000 Timothy B. Terriberry wrote: > We request that people indicate support or objection to the adoption of > the following three drafts as working group drafts: > draft-lhomme-cellar-matroska-04 > draft-lhomme-cellar-codec-00 > draft-lhomme-cellar-tags-00 Now that the draft submission window has re-opened, and having heard several comments in favor and none against, I think that we can consider these drafts adopted. Would the authors/editors please submit updated versions with the draft-ietf-cellar-* naming convention? From nobody Mon Jul 16 08:11:48 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8498C130EB7 for ; Mon, 16 Jul 2018 08:11:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.901 X-Spam-Level: X-Spam-Status: No, score=-6.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PjJri1eMWIFH for ; Mon, 16 Jul 2018 08:11:40 -0700 (PDT) Received: from smtp.mozilla.org (mx2.scl3.mozilla.com [63.245.214.156]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 281571310C5 for ; Mon, 16 Jul 2018 08:11:40 -0700 (PDT) Received: from localhost (localhost6.localdomain [127.0.0.1]) by mx2.mail.scl3.mozilla.com (Postfix) with ESMTP id E56FDC0A44 for ; Mon, 16 Jul 2018 15:11:39 +0000 (UTC) X-Virus-Scanned: amavisd-new at mozilla.org Received: from smtp.mozilla.org ([127.0.0.1]) by localhost (mx2.mail.scl3.mozilla.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CxNrOYNKkvIE for ; Mon, 16 Jul 2018 15:11:38 +0000 (UTC) Received: from [31.133.142.81] (dhcp-8e51.meeting.ietf.org [31.133.142.81]) (Authenticated sender: tterriberry@mozilla.com) by mx2.mail.scl3.mozilla.com (Postfix) with ESMTPSA id 42B80C0654 for ; Mon, 16 Jul 2018 15:11:38 +0000 (UTC) To: cellar@ietf.org References: <10d56d2a-3053-6069-7805-54bb4fd1d4e0@xiph.org> From: "Timothy B. Terriberry" Message-ID: Date: Mon, 16 Jul 2018 08:11:37 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 SeaMonkey/2.49.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [Cellar] AV1 mapping update X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jul 2018 15:11:45 -0000 Andreas Rheinhardt wrote: > every timestamp in Matroska is a multiple of 1 ns (this number is > hard-coded) so that NTSC timings can't be exactly represented on the > container level: E.g. the default duration field for 24/1.001fps content > would indicate 41708333ns, although it is 41708333.33333333...ns. So by > preserving the bitstream information it is possible to increase the > certainity that 41708333ns actually means 24/1.001 fps. This seems like a good enough reason that it should probably be explicitly described in the draft (especially if you want any of that to work in practice). If that's the only reason then okay, but if there are a lot of other reasons, then SHOULD may be too strong. As an aside the timestamp situation still makes me sad. > a) I don't like restrictions on the content that can be put into > Matroska either. The reason why I put it in is because this is (in my > understanding) currently an absolute requirement of Matroska > (`PixelWidth` and `PixelHeight` are mandatory and valid for every > frame). Given that decoders will have to support these varying output > dimensions anyway (it is a mandatory part of AV1, isn't it?), it would > be possible to drop this requirement, but for this the specifications of > both Matroska and Webm would have to be changed. I'm not sure if > everyone is ok with this. I don't think this is too burdensome. A system that requires a constant pixel size for the decoded frame can put an upscaler between the underlying AV1 decoder implementation and the input to that system, and pretend that rescaling is part of the AV1 decoder black box (implementing an upscaler is vastly simpler than implementing a whole video codec). The fact that there are different dimensions in the AV1 frame header shouldn't be something it has to worry about. > decoding process, but only internally, so that we can ignore it). Is > that correct? (I have to admit I thought for a time that the output I believe that is correct. The UpscaledWidth allows the codec to partially do what I was describing for real-time/low-latency streams internally, but still run some of the loop filters at the upscaled resolution, but due to hardware decoder complexity concerns it can only rescale in the horizontal direction, so there's a limited operating space where that is sufficient. > is the last frame in that temporal unit. I don't see a reason why there > should be another frame afterwards in the same temporal unit, but is it > actually allowed? I don't know. I agree it's ambiguous as written. I'll ask someone and get back to you. > 2. Would it be legal to code a frame with [show_frame] 0, > [showable_frame] 1 followed (perhaps immediately) by a frame header in > the same temporal unit that has [show_existing_frame] set to 1 and > outputs the frame just coded? If yes, is there any reason to ever use > it? I don't see any. According to the definition, this temporal unit I think it is legal. I don't know why you would ever do it, but it wouldn't cause any great harm. Does it complicate anything to allow it any more than DRAPs complicate things already? > 3. Would it be legal to have a temporal unit that contains a frame that > is not shown (but might be declared as showable) followed by a keyframe > with [show_frame] equal to 1? I'm asking because both the current > proposal as well as the mp4/ISOBMFF definition of sync sample require > the keyframe to be the first frame in the temporal unit. I think this is legal. I also think this would be entirely wasteful. The key frame would reset all of the reference buffers, so the non-shown frame would never contribute anything to a shown frame. > 4. You have only criticized the "MUST only" wording of the definition of > keyframe simpleblocks. You said nothing about the requirement that it is > the first frame that needs to be of type KEY_FRAME. The standard doesn't Please assume that if I don't comment on something, it may just be because I have not thought it all of the way through (you might not be wrong to assume that about things I do comment on, also). > overwritten by the delayed RAP frame). Then it may make sense to first > code this frame as a showable frame using all the reference frames and > then code the delayed RAP frame followed by the frame that is actually > output for the temporal unit containing the delayed RAP. Is my usecase > sound and should the "first"-requirement be dropped? I agree that, because a DRAP does not immediately reset all of the reference buffers, it may not be wasteful to precede it by a non-shown frame, and because you will (presumably) consume at least one reference buffer slot to store the DRAP, you cannot in general re-order them. It gets more complicated when you consider spatial scalability. It is possible for you to have a RAP for a frame other than the base layer, so you may actually have one or more shown frames in the same temporal unit (for different spatial layers) before a (delayed or even non-delayed) RAP, and again be unable to re-order them, and now whether you can start display at that frame or a following frame can depend on which operating point you are trying to use. In particular, the problem is that the AV1 specification defines a RAP in terms of a *frame*, and not in terms of a temporal unit. Because Matroska defines its blocks as containing a temporal unit, and RAPs in terms of a block, you will have to define the details of when a temporal unit should be considered a RAP (or DRAP, KFDRP, etc.) based on the frames contained inside of it. I think it may have been a mistake to do things this way in the AV1 specification, since the idea was always for temporal units to be the unit of muxing in any container, but I expect it was done this way out of expediency to avoid having to think through all of these cases. We have to think through them now. As I said in another mail, I think it may be okay to impose additional restrictions on top of the AV1 specification, especially if it makes everyone else's lives easier. This may be one such case. While the situations described above may confer some marginal advantage, and I can imagine an encoder that could take advantage of them, I don't think there would be a big loss by forbidding them. I do think it would be a very good idea to have the *same* restrictions (if any) for Matroska and MP4. If something is technically allowed by the AV1 specification but you can't actually put that into any container, then in practical terms it might as well not be allowed. But if we allow some things in Matroska but don't allow them in MP4, or vice versa, then you just have an interoperability mess. > 5. What do you think of the restriction of a Matroska track to a (subset > of) a CVS? If you look back to my mail on June 30th, I thought I was the one who suggested it in the first place. From nobody Mon Jul 16 11:37:37 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2B63F130E62 for ; Mon, 16 Jul 2018 11:37:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.901 X-Spam-Level: X-Spam-Status: No, score=-6.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LiQMPWrf0wHL for ; Mon, 16 Jul 2018 11:37:34 -0700 (PDT) Received: from smtp.mozilla.org (mx2.scl3.mozilla.com [63.245.214.156]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 80265130E55 for ; Mon, 16 Jul 2018 11:37:34 -0700 (PDT) Received: from localhost (localhost6.localdomain [127.0.0.1]) by mx2.mail.scl3.mozilla.com (Postfix) with ESMTP id 51DE0C0B1F for ; Mon, 16 Jul 2018 18:37:34 +0000 (UTC) X-Virus-Scanned: amavisd-new at mozilla.org Received: from smtp.mozilla.org ([127.0.0.1]) by localhost (mx2.mail.scl3.mozilla.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vkA8eX9bF_b9 for ; Mon, 16 Jul 2018 18:37:34 +0000 (UTC) Received: from [31.133.134.62] (dhcp-863e.meeting.ietf.org [31.133.134.62]) (Authenticated sender: tterriberry@mozilla.com) by mx2.mail.scl3.mozilla.com (Postfix) with ESMTPSA id BCFDCC0A9D for ; Mon, 16 Jul 2018 18:37:33 +0000 (UTC) From: "Timothy B. Terriberry" To: cellar@ietf.org References: <10d56d2a-3053-6069-7805-54bb4fd1d4e0@xiph.org> Message-ID: <5eeb3376-7297-205c-109b-e73558c33471@xiph.org> Date: Mon, 16 Jul 2018 11:37:32 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 SeaMonkey/2.49.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [Cellar] AV1 mapping update X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jul 2018 18:37:36 -0000 Timothy B. Terriberry wrote: >> is the last frame in that temporal unit. I don't see a reason why there >> should be another frame afterwards in the same temporal unit, but is it >> actually allowed? > > I don't know. I agree it's ambiguous as written. I'll ask someone and > get back to you. I asked the person whom I think wrote that text, Peter de Rivaz, who said: "I believe this is an oversight. I believe the intention is that the shown frame should be the last frame in all cases." From nobody Mon Jul 16 22:14:15 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A462D130F1F for ; Mon, 16 Jul 2018 22:14:13 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.201 X-Spam-Level: X-Spam-Status: No, score=-4.201 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pwbpps3NUIKY for ; Mon, 16 Jul 2018 22:14:11 -0700 (PDT) Received: from tuna.sandelman.ca (tuna.sandelman.ca [IPv6:2607:f0b0:f:3:216:3eff:fe7c:d1f3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1C65B130F13 for ; Mon, 16 Jul 2018 22:14:11 -0700 (PDT) Received: from sandelman.ca (obiwan.sandelman.ca [209.87.249.21]) by tuna.sandelman.ca (Postfix) with ESMTP id 48F742008C; Tue, 17 Jul 2018 01:29:53 -0400 (EDT) Received: by sandelman.ca (Postfix, from userid 179) id 2D25C2686; Tue, 17 Jul 2018 01:09:37 -0400 (EDT) Received: from sandelman.ca (localhost [127.0.0.1]) by sandelman.ca (Postfix) with ESMTP id 2AF1A267E; Tue, 17 Jul 2018 01:09:37 -0400 (EDT) From: Michael Richardson To: Dave Rice cc: Steve Lhomme , Codec Encoding for LossLess Archiving and Realtime transmission In-Reply-To: References: <15361.1528336434@localhost> X-Mailer: MH-E 8.6; nmh 1.7+dev; GNU Emacs 24.5.1 X-Face: $\n1pF)h^`}$H>Hk{L"x@)JS7<%Az}5RyS@k9X%29-lHB$Ti.V>2bi.~ehC0; <'$9xN5Ub# z!G,p`nR&p7Fz@^UXIn156S8.~^@MJ*mMsD7=QFeq%AL4m Archived-At: Subject: Re: [Cellar] IANA Considerations for EBML X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jul 2018 05:14:14 -0000 --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Dave Rice wrote: > I might be mistaken here, but from the draft the process of > registering an Element ID via IANA only seems interested in the > Element ID itself. For a curious onlooker of the IANA registrations it > might be helpful to see what the associated DocType is so that the > context and semantics could be understood, otherwise the registration > only seems to indicate that somewhere someone registered it via > first-come first-serve. I've been reading ebml more deeply in the last two weeks, and I actually ha= ve significant concerns. I don't really understand what is going on. We have a virtual interim on the 31st, and I'd like to discuss this in person, perhaps. Earlier 1:1 if you think you could help me. In particular, I have no idea what the binary format of EBML even is! I thought I'd just missed the description before... > Additionally I think I=E2=80=99d prefer (except for the IDs used by E= BML > itself) that the combination of DocType and ElementID be required as > unique rather than only the ElementID itself. This seems analogous to > how the same node name is allowed in various XML Schemas but don=E2= =80=99t > necessarily have any relation to one another in semantics. That's a choice. Given 2^35 ElementIDs, I don't see why you'd want to complicate your life that way.=20 > Is it acceptable to extend the IANA Considerations section to clarify > the required information for registrations? From the discussion in the > pull request these scenarios seem to be: If you want to make the ElementID a function of DocType, then put all the ElementID allocations into the document that defines that DocType. =2D-=20 ] Never tell me the odds! | ipv6 mesh network= s [=20 ] Michael Richardson, Sandelman Software Works | network architect= [=20 ] mcr@sandelman.ca http://www.sandelman.ca/ | ruby on rails = [=20 =09 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBW016DoCLcPvd0N1lAQIQxQgAiv55Ydn+1z4HVZXsN8xdcxQ9EEzrZeVD 759otsI+dzQ3R2cp8is3Ailt53dE0nVtB7Bb2fFIvnot1sWlOgYNAOchMyCz2oaI lujLy53/s13qjHgdOdU44FGVAWEGS9r3vHZwU4B2nelSu5E99leWHq6UPaGI7DqK OOh4wNwEu5HYowyt3wc2ZBEllOB+mB1NiynjPpw/VnV2zCfNr6Avjo/ZThHj34SC EFWAr4yaMJq8mrd5XA+b8gL5fPauinOm9FR553EJjmweGr1Ax/QcorC/qeXhEkn1 1JJBrW1ln2RdIx1Trs8HrFyMBF7qa8Is+3qNwVfWBKIbYWZMfAkn5g== =19mF -----END PGP SIGNATURE----- --=-=-=-- From nobody Tue Jul 17 06:24:54 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E3A47130F69 for ; Tue, 17 Jul 2018 06:24:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=googlemail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DR1ivn2nrv2N for ; Tue, 17 Jul 2018 06:24:44 -0700 (PDT) Received: from mail-wm0-x231.google.com (mail-wm0-x231.google.com [IPv6:2a00:1450:400c:c09::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 048F8130FB2 for ; Tue, 17 Jul 2018 06:24:44 -0700 (PDT) Received: by mail-wm0-x231.google.com with SMTP id s13-v6so12563527wmc.1 for ; Tue, 17 Jul 2018 06:24:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20161025; h=subject:to:references:from:message-id:date:mime-version:in-reply-to :content-transfer-encoding; bh=SHe/RpwTsAuC01amH41H8ztPx5xnF2cr4TsJtwIAyi4=; b=rtOghvbKknewfPYNxwMogOdyFi8tlfjATUt4IPeFm9apwMOyZGxrjX0Q/jkKDkdxd3 NqwvLSZW9zoZbV9fW+pW7UyPx1hBSE1ukEAPlKIyfLvHX7UyBVGu9M5fppIhDffclBZx g5uZQXo2XPT64HnS+CAtB94dTtxkXi3WB/YorMRSNs1LxMBqijxErC9E2Hy1WWq0DNHN o/9rxNhWnrVQ93OaSRpx/6V+ntYHCPa7kN7XI3+7LzNbAA+aaobOUxQEYWKBaC166Lbx Jk4UTUhTP+nsdBi6jVHwJTuYaFpfjoAwQKJGhj+6Lwn7zI7TmB8aPOXLSRi8IEggJK4g 1TJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :mime-version:in-reply-to:content-transfer-encoding; bh=SHe/RpwTsAuC01amH41H8ztPx5xnF2cr4TsJtwIAyi4=; b=HNGh1P16YyrmWIOQG0LuU4Ne+md4OGRhSCVFO1HtPa7ngx3UqGwe9qtRQWBi9mFlAd ViLQigT2GdTAVZQi694blzgXTU8fnTR+jQI6kZPksTZ3oGPUJmKM7KjTkFCelPkwvwZQ ZRt/W8NQntMSL2buEccd8vVzKSrKaNaw4kB2mjzlCQKK/J+qVeT1nBfcnK2KsDVQxywd Qf2drKk+Hf/aqHHllVDPFnMe+YPlOZkM5cYHzlvBx2nUfaE0NAQxcYzbntMbOZXujnQk PFrsXaxumUCjS/Qi5m61Ive1rjCvwfQ2QrNMKpegbmfjT6OZn2YeOZW3uQ3O9vXxO3uN pSKg== X-Gm-Message-State: AOUpUlEKeTHjxILdUHRSEkdLvayYjM7FwytoNefuxw0AExtYRjGBM2La OELMwAy2F8FpWVLfyn11cXBrqndL X-Google-Smtp-Source: AAOMgpcCCiNNxyeY+PEspxqKlqgE3jnIv6V4PBMA8FoqKT99lfeBLJ3Fx8U/vNRgZbBOB7T1Q5arKg== X-Received: by 2002:a1c:3c4:: with SMTP id 187-v6mr1283756wmd.96.1531833882001; Tue, 17 Jul 2018 06:24:42 -0700 (PDT) Received: from [127.0.0.1] ([141.255.162.34]) by smtp.googlemail.com with ESMTPSA id y189-v6sm1587923wmd.19.2018.07.17.06.24.40 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 17 Jul 2018 06:24:41 -0700 (PDT) To: Codec Encoding for LossLess Archiving and Realtime transmission References: <62c29889-49f5-6634-049a-a2d73315bb3c@googlemail.com> From: Andreas Rheinhardt Message-ID: <4ff1f1f8-3b66-e229-7884-c851dae8e2e0@googlemail.com> Date: Tue, 17 Jul 2018 13:23:00 +0000 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [Cellar] AV1 seeking X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jul 2018 13:24:53 -0000 Hello, Steve Lhomme: > Hi, > > 2018-07-16 0:19 GMT+02:00 Andreas Rheinhardt > : >>> In MP4 they have [initial_presentation_delay_minus_one] in the >>> CodecPrivate. I did not understand it so far because it's not found in >>> the AV1 spec. But it seems to guarantee that to read frame 'f' you >>> need to decode X frame before that one. In our case that would be 5 to >>> have at least a decoded. >>> >> You completely misunderstood this field. It is the equivalent of the >> max_num_reorder_frames value from H.264. Let me explain it in MPEG >> terminology (with b-frames) as you probably have way more experience >> with this: Consider a decoder that can only decode one frame per unit of >> time and a stream like this (left to right is decoding order; the >> numbers are presentation order): >> I0 P2 B1 ... >> If one displayed the leading I frame immediately after decoding it, one >> would not have the right frame to display at time 1, because at that >> time only I0 and P2 has been decoded, not B1. Therefore one has to >> decode I0 and P2 before one outputs the first frame and and >> max_num_reorder_frames would be 1. The typical b-pyramid would require >> to decode the first three frames before the display of the first frame >> and max_num_reorder_frames would be 2. >> The [initial_presentation_delay_minus_one] is the AV1 analogue of this. >> This number is not an upper bound for the amount of temporal units >> between delayed RAP and recovery point/for the amount of frames shared >> between a GOP. Just look at this example: >> MPEG example >> I0 P5 B1 B2 B3 B4 I10 B6 B7 B8 B9 >> AV1 example (I kept the MPEG-naming with P and B to make it easier >> comparable to the above; furthermore the pointer *x denotes that a frame >> is showable and x without * is a frame header that outputs *x via >> show_existing_frame; square brackets are the delimiters of temporal >> units; I10 is the delayed RAP frame) >> [I0] [*P5 B1] [B2] [B3] [B4] [P5] [*I10 B6] [B7] [B8] [B9] [I10] ... >> This stream can have [initial_presentation_delay_minus_one] equal to 1, > > A value of 1 means 2 frame delay. That would mean B6 needs *I10 and P5 ? > No, absolutely not. This field tells you nothing at all about seeking and in particular it does not say that "2 frame delay" implies that it is sufficient to seek to a frame two frames before said frame and start decoing there to get the (undamaged) desired frame. But judging from your next comment, you have already realized this on your own. The frame delay field is there to solve a problem that is completely orthogonal to the seeking problem, hence it is actually wrong to discuss it under the label "[Cellar] AV1 seeking". I will nevertheless reply to your latest change of the current proposal in this email. >> yet in order to seek to [I10] one has to decode the temporal unit [*I10 >> B6] (or at least the decodable keyframe in it) which is four temporal >> units in front of [I10]. > > That seems more like it. > > In the Frame presentation timing paragraph (E.4.7) it says: > > InitialPresentationDelay = Removal [ initial_display_delay_minus_1 ] > + TimeToDecode [ initial_display_delay_minus_1 ] > > and > > PresentationTime[ 0 ] = InitialPresentationDelay > PresentationTime[ j ] = InitialPresentationDelay + ( > frame_presentation_time[ j ] - frame_presentation_time[ 0 ] ) * DispCT > > or in constant bitrate mode > > PresentationTime[ 0 ] = InitialPresentationDelay > PresentationTime[ j ] = PresentationTime[ j - 1 ] + ( > num_ticks_per_picture_minus_1 + 1 ) * DispCT > > > And our `Block` timestamp derives directly from this > [PresentationTime]. The delay is carried over every frame. So it does > seem like we need to globally shift these timestamps for the Track. > Possibly with `CodecDelay`. > > Basically each frame has its timestamp on which this delay has to be > added (can't be negative). While `CodecDelay` is a positive value that > needs to be substracted from the timestamps. It is available in WebM, > because Opus needs it. > > Let's assume CodecDelay = InitialPresentationDelay > > The `Block` timestamps would be stored like this: > Block[ 0 ] = 0 > Block[ 1 ] = frame_presentation_time[ 1 ] * DispCT > Block[ 2 ] = frame_presentation_time[ 2 ] * DispCT > ... > > And on output of the demuxer we would get: > Block[ 0 ] = -InitialPresentationDelay > Block[ 1 ] = -InitialPresentationDelay + frame_presentation_time[ 1 ] * DispCT > Block[ 2 ] = -InitialPresentationDelay + frame_presentation_time[ 2 ] * DispCT > ... > > `CodecDelay` cannot be used here. But it's very close to what we need. > > It's not the SeekPreRoll either because the actual frame timestamps of > each frame is affected, not just when seeking (although it may be > needed for delayed RAP). > > We also need to figure out whether the Blocks need to be stored with > or without the delay. If that's with the delay then we don't even need > to care about it. The frames will just not start at 0 but that's > already the case in the original stream. > > But that may just be the tricky part here, the reference point. The > PresentationTime[ 0 ] doesn't start at 0 because some frames were > needed to decode before getting actual usable data out of the decoder. > But this is really the first frame to display so it would actually be > 0 on the output of the demuxer. So it does seem exactly like what > CodecDelay does: > "CodecDelay is The codec-built-in delay in nanoseconds. This value > must be subtracted from each block timestamp in order to get the > actual timestamp." > > The `Block` timestamp would be PresentationTime[ 0 ] which isn't 0. > But on the output Block[ 0 ] should really give 0. So we would have > something like this in the file: > Block[ 0 ] = InitialPresentationDelay > Block[ 1 ] = InitialPresentationDelay + frame_presentation_time[ 1 ] * DispCT > Block[ 2 ] = InitialPresentationDelay + frame_presentation_time[ 2 ] * DispCT > ... > > And on the output of the demuxer we would have the following, when > `CodecDelay` is [InitialPresentationDelay]: > Block[ 0 ] = 0 > Block[ 1 ] = frame_presentation_time[ 1 ] * DispCT > Block[ 2 ] = frame_presentation_time[ 2 ] * DispCT > ... > > It does look like what the internal AV1 display delay is trying to achieve. > > The `CodecDelay` also has this note on proper muxing: > The value SHOULD be small so the muxing of tracks with the same actual > timestamp are in the same Cluster. > > Because muxing is done based on the stored `Block` timestamp. It > usually doesn't take in account CodecDelay (but it could/should). > > So I'll update my AV1 codec mapping saying `CodecDelay` is > [InitialPresentationDelay] and how to use it. > 1. Good that you thought about the initial presentation delay. I have personally experienced how unfortunate it is that the H.264 packetization doesn't provide a way to indicate the necessary number of reorder frames (see e.g. ) and I have talked to Moritz Bunkus about using the `MinCache` value to store the number of reorder frames. (Btw: The "reference pseudo-cache system" this element talks about seems to be undefined.) 2. Nevertheless I think that your proposal is totally wrong: a) The semantics of `CodecDelay` are unclear: Although the current wording only speaks about a delay that should be subtracted from the timestamps, the muxers are actually using this value to map the Opus preskip to a Matroska field, i.e. they are treating it as if it would not only be a delay that should be subtracted from the timestamps, but as if it also signaled `DiscardPadding` at the beginning. There was a discussion in May 2016 in which everyone agreed that `CodecDelay` included discarding samples at the beginning and the following proposal for an alternative wording was made (see ): "CodecDelay is the duration of the codec-built-in delay in nanoseconds. The decoded frames from the beginning of a stream should be discarded until CodecDelay duration has passed, and CodecDelay must be subtracted from each block timestamp in order to get the presentation timestamp. The value should be small so the muxing of tracks with the same presentation timestamp are in the same Cluster." This has not been agreed on not because the discard-part was controversial per se, but because it was unclear how exactly to convert this ns value to precise samples given that they use different timebases. Notice that one thing has not been brought up during the linked discussion: What exactly will be discarded? Is it always the first `CodecDelay` output or is it only the output which has a negative timestamp after the timestamp shift specified by `CodecDelay` has been applied? This is important if the lowest timestamp is >0 (before shifting by `CodecDelay`). In the second case one would have to additionaly use a `DiscardPadding` element at the beginning to signal that the appropriate number of samples should be discarded. No muxer I know of currently does this, i.e. they work as if not only the samples that end up with a negative timestamp after the shift were discarded. And if we keep this logic then the first frames will be decoded, but discarded under your proposal; if we don't keep it, then some files will have been wrongly muxed (i.e. then current muxers haven't signalled the Opus `PreSkip` correctly at all and people that relied on this interpretation are screwed). b) The elementary streams of other codecs also has such an offset (See e.g. equation (C-12) in the H.264 standard.), but we don't store it for them either. And it works. c) The time in Annex E is obviously a dts given that it starts at zero when decoding begins. (Here I consider the arrival of the data in a buffer as part of the decoding process.) But Matroska uses pts. The only thing that your proposal does is signalling the pts to dts difference. But only for the very first picture. This is enough so that one can play the track smoothly from the beginning; but it is not enough in general. Let me explain: Nothing guarantees you that the pts to dts offset for the first frame is the pts to dts offset for any further keyframes (even when we restrict this to the keyframe RAP and not the delayed RAPs; after all, if one discards from a CVS all OBUs in front of a keyframe RAP, one still has a valid CVS (provided that the `Sequence Header OBU` is still available)). That's because it may depend on [buffer_removal_time] which needs to be signalled for every frame. Now the obvious solution for this would be to use the highest pts to dts offset for any RAP (or only for every keyframe RAP; I actually don't know if one would have to use the offset between pts and dts for the delayed RAPs, too). This has the downside that you would have to do the muxing in two passes; after all, if you notice that you need to use an increased value of `CodecDelay`, you would have to rewrite not only the track header, but the timestamps written so far as well if you tried to mux in one pass. Here is a situation where the above scenario might be realistic: You have a vfr track that actually consists of several sections that are cfr (but when combined are nevertheless vfr). The beginning is (say) 50p, the rest is 25p (this really happens: if the movie is shot in 25p, but the title and end credits are overlayed as 50p, then the result is effectively 50p at the beginning and end and 25p in the middle; I have several TV broadcasts like that). Then it is quite likely that the individual 50p frames are smaller (coded size) so that they can be transmitted faster and so one can use a lower ScheduledRemovalTiming for the DFGs corresponding to the 50p frames. This means that the Removal times (in decoding schedule mode) will be smaller as well. Hence the InitialPresentationDelay for decoding from the very first frame is smaller than the InitialPresentationDelay for decoding from one of the later RAPs. For this reason decoders would probably rely on the [initial_display_delay_minus_1] values (that are luckily not stripped away). But then there is no point in using `CodecDelay` at all as this value can also be used at the beginning. d) There are additional complications: i) It would complicate muxers that would have to implement the procedures described in E.4. ii) It breaks compability with mp4/ISOBMFF because they only store the initial_presentation_delay_minus_one value (storing it is recommended, but not required) and (that's a SHOULD) they should set the [timing_info_present] flag to zero which means that the bitstream itself wouldn't contain an explicitly signalled presentation delay any more. One would have to recalculate the `CodecDelay` value, but there is a problem with this approach: Appendix E.3 knows two modes: Resource availability mode and the decoding schedule mode. The latter needs the buffer_removal_time information from the frame header, but they SHOULD be stripped away during muxing into mp4, so they are likely not available and the calculation can't be performed at all. The former has the prerequisite that it needs to be crf ([equal_picture_interval] equal to 1). Given that the timing information has been discarded, [equal_picture_interval] is generally unavailable in mp4, but we can still check whether it is crf. And what if it isn't? What `CodecDelay` value does one use? iii) It is not possible every time even when muxing from an AV1 elementary stream: A bitstream can contain [initial_display_delay_minus_1] even when it doesn't signal explicit timing info or when it doesn't signal the decoder_model_info or the operating_parameters_info or if it doesn't signal the buffer removal time. In my experience with H.264, it was enough to explicitly signal the [max_num_reorder_frames] for correct playback; it was not necessary to also add the SEI messages necessary for conformance testing of the track. (This is also how x264 works by default: No unnessary SEIs, but the [max_num_reorder_frames] is set.) I actually wouldn't be surprised if an x264-like AV1 encoder would use the same defaults: Signal [initial_display_delay_minus_1], don't signal the buffer_removal_time (too many wasted bits for no gain) and don't signal the operating parameters. iv) Of course the `CodecDelay` would also depend on the operating point (if scalability is used). I have to admit this could be easily rectified: Simply calculate the `CodecDelay` for every operating point and use the maximum. v) Purely Matroska-wise, there are further complications: Splitting such a file would mean that the muxer has to use the correct `CodecDelay` value for the second file. This is in general a different value than the value for the first file, but the value can't be calculated because the timing information has been discarded from the bitstream. Alternatively appending files where two tracks that are to be appended to one another have different `CodecDelay` values also requires changes; if the second file has the higher `CodecDelay` value, one would have to increase the `CodecDelay` for the whole output track to that value and this entails to shift the timestamps of the first file, too. There probably are more complications further down the road. 3. Here is a counterproposal: We more or less copy how mp4 does it. I.e. we ignore InitialPresentationDelay when determining the timestamps and we add a field that is equivalent to their [initial_presentation_delay_minus_one]. Honstely I don't even know what they exactly mean by "display model verification algorithm". Is it the procedure described in annex E? If so, they can run this algorithm only if the video is cfr (resource availability mode) or the buffer_removal_time is given in the frame headers (decoding schedule mode), but the latter SHOULD have been striped away, so that leaves a crf requirement. (Or they could have kept the relevant paramters from the elementary stream (maybe not in the mp4 file, but somewhere else).) Anyway, for our purposes it will be enough to use the maximum of all [initial_display_delay_minus_1] values if said value is present for every operating point that occurs in the file. It is one of the fields which can't change mid-way in a CVS so it works well for one-pass muxers. The easiest way would be probably to add a byte to the beginning of `CodecPrivate` (whose definition would have to be changed). How about the MSB bit being the "initial_presentation_delay_present" bit, then three reserved bits (set to zero, may hypothetically be used to indicate a new version of the AV1 packetization in Matroska if we wished to keep the AV1 CodecID) and then the `initial_presentation_delay_minus_one`, coded on 4 bits. If someone wonders why I propose to add such a field even when the information is in the 'Sequence Header OBU` (after all, this is not stripped away): Because it actually needn't be in the `Sequence Header OBU`! E.g. think of an AV1 bitstream inside mp4 where the `Sequence Header OBU` delay field doesn't exist, but where the initial delay is signalled on the container level. Muxing from mp4 to Matroska would loose this information when it is not kept somewhere. Furthermore, just as a muxer can actually count the number of reorder frames necessary for H.264, it can probably do the equivalent with the initial presentation delay for AV1 (this is not to say that this should be required or even a SHOULD) and then it can put the value into this field (rewriting a field in the track header is possible even for a one-pass muxer). 4. How helpful is this buffer removal timing information actually? Is it needed for streaming? I don't think so, because then the mp4 guys (one of the main authors of the mp4 packetization works for Netflix) would not have recommended to strip it away. But maybe conformance testing should be mentioned as another reason not to strip the [timing_info] away. (I would only keep the timing_info parameters for a crf track in order to have the real framerate stored, but I would discard the decoder_model_info stuff.) - Andreas Rheinhardt From nobody Tue Jul 17 06:46:58 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D1C34130E58 for ; Tue, 17 Jul 2018 06:46:52 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.12 X-Spam-Level: X-Spam-Status: No, score=-1.12 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cnXfrfCP1Xc3 for ; Tue, 17 Jul 2018 06:46:51 -0700 (PDT) Received: from server172-3.web-hosting.com (server172-3.web-hosting.com [68.65.122.111]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4F16612D7F8 for ; Tue, 17 Jul 2018 06:46:51 -0700 (PDT) Received: from [146.96.19.240] (port=1630 helo=[10.10.201.39]) by server172.web-hosting.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.91) (envelope-from ) id 1ffQJU-003Sdv-Vj; Tue, 17 Jul 2018 09:46:50 -0400 From: Dave Rice Message-Id: <9EFC559C-7C60-4B6E-80F3-2623232AF071@dericed.com> Content-Type: multipart/alternative; boundary="Apple-Mail=_54671CE6-7D8D-4C36-94D4-7AE39C09E848" Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Date: Tue, 17 Jul 2018 09:46:47 -0400 In-Reply-To: <6ce57954-4d49-730c-db4c-e85c80729c53@xiph.org> Cc: Codec Encoding for LossLess Archiving and Realtime transmission To: "Timothy B. Terriberry" References: <1ae2f2ce-3218-a19f-330e-0bd5407452a3@xiph.org> <6ce57954-4d49-730c-db4c-e85c80729c53@xiph.org> X-Mailer: Apple Mail (2.3273) X-OutGoing-Spam-Status: No, score=-2.9 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Subject: Re: [Cellar] Call for adoption of three new WG items X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jul 2018 13:46:53 -0000 --Apple-Mail=_54671CE6-7D8D-4C36-94D4-7AE39C09E848 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On Jul 16, 2018, at 10:05 AM, Timothy B. Terriberry = wrote: >=20 > Timothy B. Terriberry wrote: >> We request that people indicate support or objection to the adoption = of the following three drafts as working group drafts: >> draft-lhomme-cellar-matroska-04 >> draft-lhomme-cellar-codec-00 >> draft-lhomme-cellar-tags-00 >=20 > Now that the draft submission window has re-opened, and having heard = several comments in favor and none against, I think that we can consider = these drafts adopted. Would the authors/editors please submit updated = versions with the draft-ietf-cellar-* naming convention? I adjusted the name of all three from *-lhomme-* to *-ietf-* and = submitted all three (matroska, tags, and codec). I reset the version = numbering of all three back to 00. The changes for the renaming and versioning are in = https://github.com/Matroska-Org/matroska-specification/pull/241/files = . Dave Rice= --Apple-Mail=_54671CE6-7D8D-4C36-94D4-7AE39C09E848 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii
On Jul 16, 2018, at 10:05 AM, Timothy B. Terriberry <tterribe@xiph.org> = wrote:

Timothy B. Terriberry wrote:
We request that people indicate support or = objection to the adoption of the following three drafts as working group = drafts:
draft-lhomme-cellar-matroska-04
draft-lhomme-cellar-codec-00
draft-lhomme-cellar-tags-00

Now that the draft submission window has re-opened, and = having heard several comments in favor and none against, I think that we = can consider these drafts adopted. Would the authors/editors please = submit updated versions with the draft-ietf-cellar-* naming = convention?

I adjusted the name of all three from = *-lhomme-* to *-ietf-* and submitted all three (matroska, tags, and = codec). I reset the version numbering of all three back to 00.
The changes for the renaming and versioning are in https://github.com/Matroska-Org/matroska-specification/pull/241= /files.

Dave= Rice
= --Apple-Mail=_54671CE6-7D8D-4C36-94D4-7AE39C09E848-- From nobody Tue Jul 17 06:57:48 2018 Return-Path: X-Original-To: cellar@ietf.org Delivered-To: cellar@ietfa.amsl.com Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id F3C7B130FAE; Tue, 17 Jul 2018 06:57:42 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: internet-drafts@ietf.org To: Cc: cellar@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 6.82.0 Auto-Submitted: auto-generated Precedence: bulk Reply-To: cellar@ietf.org Message-ID: <153183586284.12630.9635880100971594845@ietfa.amsl.com> Date: Tue, 17 Jul 2018 06:57:42 -0700 Archived-At: Subject: [Cellar] I-D Action: draft-ietf-cellar-matroska-00.txt X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jul 2018 13:57:46 -0000 A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Codec Encoding for LossLess Archiving and Realtime transmission WG of the IETF. Title : Matroska Specifications Authors : Steve Lhomme Moritz Bunkus Dave Rice Filename : draft-ietf-cellar-matroska-00.txt Pages : 159 Date : 2018-07-17 Abstract: This document defines the Matroska audiovisual container, including definitions of its structural elements, as well as its terminology, vocabulary, and application. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-cellar-matroska/ There are also htmlized versions available at: https://tools.ietf.org/html/draft-ietf-cellar-matroska-00 https://datatracker.ietf.org/doc/html/draft-ietf-cellar-matroska-00 Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From nobody Tue Jul 17 06:59:25 2018 Return-Path: X-Original-To: cellar@ietf.org Delivered-To: cellar@ietfa.amsl.com Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 130FC130F74; Tue, 17 Jul 2018 06:59:06 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: internet-drafts@ietf.org To: Cc: cellar@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 6.82.0 Auto-Submitted: auto-generated Precedence: bulk Reply-To: cellar@ietf.org Message-ID: <153183594601.12793.8464055023886307742@ietfa.amsl.com> Date: Tue, 17 Jul 2018 06:59:06 -0700 Archived-At: Subject: [Cellar] I-D Action: draft-ietf-cellar-codec-00.txt X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jul 2018 13:59:15 -0000 A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Codec Encoding for LossLess Archiving and Realtime transmission WG of the IETF. Title : Matroska Codec Authors : Steve Lhomme Moritz Bunkus Dave Rice Filename : draft-ietf-cellar-codec-00.txt Pages : 44 Date : 2018-07-17 Abstract: This document defines the Matroska codec mappings, including the codec ID, layout of data in a "Block Element" and in an optional "CodecPrivate Element". The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-cellar-codec/ There are also htmlized versions available at: https://tools.ietf.org/html/draft-ietf-cellar-codec-00 https://datatracker.ietf.org/doc/html/draft-ietf-cellar-codec-00 Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From nobody Tue Jul 17 06:59:38 2018 Return-Path: X-Original-To: cellar@ietf.org Delivered-To: cellar@ietfa.amsl.com Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 6FF62130FB7; Tue, 17 Jul 2018 06:59:27 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: internet-drafts@ietf.org To: Cc: cellar@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 6.82.0 Auto-Submitted: auto-generated Precedence: bulk Reply-To: cellar@ietf.org Message-ID: <153183596739.12760.2184492698691114394@ietfa.amsl.com> Date: Tue, 17 Jul 2018 06:59:27 -0700 Archived-At: Subject: [Cellar] I-D Action: draft-ietf-cellar-tags-00.txt X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jul 2018 13:59:34 -0000 A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Codec Encoding for LossLess Archiving and Realtime transmission WG of the IETF. Title : Matroska Tags Authors : Steve Lhomme Moritz Bunkus Dave Rice Filename : draft-ietf-cellar-tags-00.txt Pages : 20 Date : 2018-07-17 Abstract: This document defines the Matroska tags, namely the tag names and their respective semantic meaning. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-cellar-tags/ There are also htmlized versions available at: https://tools.ietf.org/html/draft-ietf-cellar-tags-00 https://datatracker.ietf.org/doc/html/draft-ietf-cellar-tags-00 Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From nobody Tue Jul 17 07:38:51 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8BB8C130E14 for ; Tue, 17 Jul 2018 07:38:39 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.901 X-Spam-Level: X-Spam-Status: No, score=-6.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zO6EUijsOM_q for ; Tue, 17 Jul 2018 07:38:36 -0700 (PDT) Received: from smtp.mozilla.org (mx2.scl3.mozilla.com [63.245.214.156]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BBE41130F63 for ; Tue, 17 Jul 2018 07:38:36 -0700 (PDT) Received: from localhost (localhost6.localdomain [127.0.0.1]) by mx2.mail.scl3.mozilla.com (Postfix) with ESMTP id A4321C003A for ; Tue, 17 Jul 2018 14:38:35 +0000 (UTC) X-Virus-Scanned: amavisd-new at mozilla.org Received: from smtp.mozilla.org ([127.0.0.1]) by localhost (mx2.mail.scl3.mozilla.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id amj8Q-0fhMIy for ; Tue, 17 Jul 2018 14:38:35 +0000 (UTC) Received: from [192.168.0.111] (modemcable234.246-178-173.mc.videotron.ca [173.178.246.234]) (Authenticated sender: tterriberry@mozilla.com) by mx2.mail.scl3.mozilla.com (Postfix) with ESMTPSA id 06BF8C0022 for ; Tue, 17 Jul 2018 14:38:34 +0000 (UTC) To: Codec Encoding for LossLess Archiving and Realtime transmission References: <1ae2f2ce-3218-a19f-330e-0bd5407452a3@xiph.org> <6ce57954-4d49-730c-db4c-e85c80729c53@xiph.org> <9EFC559C-7C60-4B6E-80F3-2623232AF071@dericed.com> From: "Timothy B. Terriberry" Message-ID: <542f74de-8413-d4d2-84b1-94812b8c5078@xiph.org> Date: Tue, 17 Jul 2018 07:38:33 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 SeaMonkey/2.49.7.0 MIME-Version: 1.0 In-Reply-To: <9EFC559C-7C60-4B6E-80F3-2623232AF071@dericed.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [Cellar] Call for adoption of three new WG items X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jul 2018 14:38:50 -0000 Dave Rice wrote: > I adjusted the name of all three from *-lhomme-* to *-ietf-* and > submitted all three (matroska, tags, and codec). I reset the version > numbering of all three back to 00. > The changes for the renaming and versioning are in > https://github.com/Matroska-Org/matroska-specification/pull/241/files. Thanks, Dave. All three drafts should be posted now. From nobody Wed Jul 18 02:41:39 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D7203130DD8 for ; Wed, 18 Jul 2018 02:41:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.909 X-Spam-Level: X-Spam-Status: No, score=-1.909 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, T_DKIMWL_WL_MED=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mYIVOgFrd2hY for ; Wed, 18 Jul 2018 02:41:34 -0700 (PDT) Received: from mail-pg1-x52e.google.com (mail-pg1-x52e.google.com [IPv6:2607:f8b0:4864:20::52e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 33AAB12777C for ; Wed, 18 Jul 2018 02:41:34 -0700 (PDT) Received: by mail-pg1-x52e.google.com with SMTP id n7-v6so1763175pgq.4 for ; Wed, 18 Jul 2018 02:41:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=I8ZGYjcVA2LC+EiZMUtzgMvIXmedvbWJ+Dfd07cUAtg=; b=lMAyuC2LG4FJY+GgWvuqSOj7efux5OHHvDr3AllKzBgoqQe99dISM5/+WIsbPmOeDa W2dhbrgDumIFcviVzGg64bZLwcTCIT3jw/LD9pGTHpCIOIIfj0kZzcbXt0Aco17MySqd p7+r4z8XfqcHsmPTnSxY9XVA6wkQTEfNbiMH0bahexDlPwb8MFt89Soblq4VOFA2qs+S mMBVsdLinAZ8Fjdn4wnpL2/0Rpnc5cwV1dp5gfFaKwxvuJanOLdTvsgisxGcYVmUAY2C Wf1tjsk9XIKZtLxr57+V7bLozHy+r+RIpqmpqE9OSnDm0bo3sswb6VxpOewPsQGRRju/ N8ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=I8ZGYjcVA2LC+EiZMUtzgMvIXmedvbWJ+Dfd07cUAtg=; b=c+cCRjGsVWHnEpqV3Ay9SH3ep7gmzJxO+Gev1DJwNPsGXOW2237eR1ufpKyEP78JRN i+CaRn16nrJu502KcYn1b5jrf7Q/LWumdsRHvEGQZfbxjedwmReN+jy+ewvof8y5/jSi XwjxZDgjZ59dP6umryAZXVdaqqbRY4cEE1VR+rdwL5PDy20f/MU9SxTg8oC6aesvZ+kx KH2rhwwEATteV2pdcT3L9vbqNgMlnDOUjlYTunZqINEUx6iuW5Co2T0A/cQfrfRXkRc7 mparRnkf7jF5oNfOjIRpho5qq0se6l7Y/C4pFW+iaSQHLJul3B95IGy99gk7GXJ+sVx/ tweA== X-Gm-Message-State: AOUpUlHvgTsPfI8CgNb6e1mj77df000Um9yXFL9QhdkEiC1by39yblDN H730YFHybbx/NYo2Ksmo6+bOTxcuaA5lJsiIHC5JcC2X X-Google-Smtp-Source: AAOMgpe/Ml3NMMjuTmy2y0WSQ1KsvZTJo/gsY+7k1lMrFmijFAX7Fg7oHFId8LIdEF0DfFIlyorb7M4xTodYVWhbwc8= X-Received: by 2002:a65:5245:: with SMTP id q5-v6mr5016308pgp.67.1531906893622; Wed, 18 Jul 2018 02:41:33 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a17:90a:9c13:0:0:0:0 with HTTP; Wed, 18 Jul 2018 02:41:32 -0700 (PDT) In-Reply-To: References: <10d56d2a-3053-6069-7805-54bb4fd1d4e0@xiph.org> From: Steve Lhomme Date: Wed, 18 Jul 2018 11:41:32 +0200 Message-ID: To: "Timothy B. Terriberry" Cc: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cellar] AV1 mapping update X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jul 2018 09:41:37 -0000 Hi Tim, 2018-07-16 17:11 GMT+02:00 Timothy B. Terriberry : > Andreas Rheinhardt wrote: >> >> every timestamp in Matroska is a multiple of 1 ns (this number is >> hard-coded) so that NTSC timings can't be exactly represented on the >> container level: E.g. the default duration field for 24/1.001fps content >> would indicate 41708333ns, although it is 41708333.33333333...ns. So by >> preserving the bitstream information it is possible to increase the >> certainity that 41708333ns actually means 24/1.001 fps. > > > This seems like a good enough reason that it should probably be explicitly > described in the draft (especially if you want any of that to work in > practice). If that's the only reason then okay, but if there are a lot of > other reasons, then SHOULD may be too strong. In any cases the container time will apply. It's fine to "resample" a 24fps to display it in 1fps (slow motion mode) without changing the coded data. So it doesn't really matter how the source was encoded. (I'm not sure decoders do temporal adjustments, at least when interlacing is not involved). > As an aside the timestamp situation still makes me sad. All of us :( We still haven't come up with a good backward compatible solution. For future Matroska versions it's certainly doable. >> 2. Would it be legal to code a frame with [show_frame] 0, >> [showable_frame] 1 followed (perhaps immediately) by a frame header in >> the same temporal unit that has [show_existing_frame] set to 1 and >> outputs the frame just coded? If yes, is there any reason to ever use >> it? I don't see any. According to the definition, this temporal unit > > > I think it is legal. I don't know why you would ever do it, but it wouldn't > cause any great harm. Does it complicate anything to allow it any more than > DRAPs complicate things already? No, it's just not optimal. It may happen in cases where some frames that were in between were dropped (or some capture system that would only keep keyframes from a source stream). > It gets more complicated when you consider spatial scalability. It is > possible for you to have a RAP for a frame other than the base layer, so you > may actually have one or more shown frames in the same temporal unit (for > different spatial layers) before a (delayed or even non-delayed) RAP, and > again be unable to re-order them, and now whether you can start display at > that frame or a following frame can depend on which operating point you are > trying to use. > > In particular, the problem is that the AV1 specification defines a RAP in > terms of a *frame*, and not in terms of a temporal unit. Because Matroska > defines its blocks as containing a temporal unit, and RAPs in terms of a > block, you will have to define the details of when a temporal unit should be > considered a RAP (or DRAP, KFDRP, etc.) based on the frames contained inside > of it. > > I think it may have been a mistake to do things this way in the AV1 > specification, since the idea was always for temporal units to be the unit > of muxing in any container, but I expect it was done this way out of > expediency to avoid having to think through all of these cases. We have to > think through them now. Sadly it's too late to fix this. And on our side we cannot just assume people will just do "the right thing" if the spec allows different/odd ways. > As I said in another mail, I think it may be okay to impose additional > restrictions on top of the AV1 specification, especially if it makes > everyone else's lives easier. This may be one such case. While the > situations described above may confer some marginal advantage, and I can > imagine an encoder that could take advantage of them, I don't think there > would be a big loss by forbidding them. I agree. We could also have a different CodecID for a more loose/less seek friendly version. > I do think it would be a very good idea to have the *same* restrictions (if > any) for Matroska and MP4. If something is technically allowed by the AV1 > specification but you can't actually put that into any container, then in > practical terms it might as well not be allowed. But if we allow some things > in Matroska but don't allow them in MP4, or vice versa, then you just have > an interoperability mess. I agree. That's why I had a few picks there to see how things are done. There doesn't seem to be much restrictions on the bitstream itself (other that the ones I already copied like the low overhead bitstream format). But that also means if we want to add new restrictions we need to have them on the MP4 side as well. >> 5. What do you think of the restriction of a Matroska track to a (subset >> of) a CVS? > > > If you look back to my mail on June 30th, I thought I was the one who > suggested it in the first place. > > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar -- Steve Lhomme Matroska association Chairman From nobody Wed Jul 18 05:22:37 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1F083130E57 for ; Wed, 18 Jul 2018 05:22:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.909 X-Spam-Level: X-Spam-Status: No, score=-1.909 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, T_DKIMWL_WL_MED=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4rFQTmiE4tCC for ; Wed, 18 Jul 2018 05:22:32 -0700 (PDT) Received: from mail-pl0-x233.google.com (mail-pl0-x233.google.com [IPv6:2607:f8b0:400e:c01::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0726D1277C8 for ; Wed, 18 Jul 2018 05:22:31 -0700 (PDT) Received: by mail-pl0-x233.google.com with SMTP id m16-v6so1962265pls.11 for ; Wed, 18 Jul 2018 05:22:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=Ds1cnajsvzRRRd7bxc0E4CXW/2dQVTsVqAArrG3hKWo=; b=qIg9C9lc2QvqThb1qxrsmftKqI9mg4wdEiYb9qzcGzCR3erfyjef19dr1qtaG+iyCT btHojggEqMGRS48Xy19NV7vQG7dOafCLErGmsYoBJ52WyPKBMTK5p7uzQoXTeLx91ihp 3eY30qVbk5Gzjtxb8os0/x/pDYEzQ33ID9A5t3vcFo4ip2HpABtQgZBvHqZcWCKwUoYE RXn3HKXWHUCnxu+QH0ek7y7C8OqYdlsaAHpicaQrOCEI7S1VAzKow8iSoF0dyvvcFc0H ZgUC3goug0tgrWm8ue7JK+9IVxEnZ1puiUHwRb52X4HmIMqCH74XWmsOpKc3BO0tQzR3 oYRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=Ds1cnajsvzRRRd7bxc0E4CXW/2dQVTsVqAArrG3hKWo=; b=kaQ+EViC5/15RCaF4SHBz13iVHo7AAqsFq57CJZuT/1xb588dqxoiHkoGs7GHi6gxV 4W6ajTHJ709qISg7cpcHnaWZC7U2tqSL3CBHk6bLnf4Ek1NDGxnyLMKJn4afYG5vlu7a mY5T3Xpc6VYp2y5CJUGogpGeHPFNTEPdrHI5jx5z1Uj7lm7JRpjTFuRm0qktjSxiqAou EPRaM5bxmWk0BkimkH9LG2rtbVRH4EdAe/Rdvhyd3fKJVel9d9X0XdkW7FVzrbMs2EWK ZU904FgSztu4jtyzdZx5tJKWXJKwHdg2mQJwFA4JQKT/NxsStFnrkP38N8Vz+/OhsDE9 Yc2A== X-Gm-Message-State: AOUpUlF+1REQfPMPBBexVIvWDJhUonMP43gIgvezgyrdB6ckGebgSD2n zfS2KIk0zJGlJnG/7SysgmCJZkBY4hJDZLcgShgnlC2n X-Google-Smtp-Source: AAOMgpd4Jrvrg4IvEt4yCdjdsgWWmj3Obo6wZ+4Fsypn7hO5pSIvd1hSbEqzGPKRb3Rx9eBAP1s0bDSaDAcAfv8KGNs= X-Received: by 2002:a17:902:704c:: with SMTP id h12-v6mr5654739plt.237.1531916551129; Wed, 18 Jul 2018 05:22:31 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a17:90a:9c13:0:0:0:0 with HTTP; Wed, 18 Jul 2018 05:22:30 -0700 (PDT) In-Reply-To: <4ff1f1f8-3b66-e229-7884-c851dae8e2e0@googlemail.com> References: <62c29889-49f5-6634-049a-a2d73315bb3c@googlemail.com> <4ff1f1f8-3b66-e229-7884-c851dae8e2e0@googlemail.com> From: Steve Lhomme Date: Wed, 18 Jul 2018 14:22:30 +0200 Message-ID: To: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cellar] AV1 seeking X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jul 2018 12:22:36 -0000 2018-07-17 15:23 GMT+02:00 Andreas Rheinhardt : >> So I'll update my AV1 codec mapping saying `CodecDelay` is >> [InitialPresentationDelay] and how to use it. >> > 1. Good that you thought about the initial presentation delay. I have > personally experienced how unfortunate it is that the H.264 > packetization doesn't provide a way to indicate the necessary number of > reorder frames (see e.g. ) and > I have talked to Moritz Bunkus about using the `MinCache` value to store > the number of reorder frames. (Btw: The "reference pseudo-cache system" > this element talks about seems to be undefined.) > > 2. Nevertheless I think that your proposal is totally wrong: It's may be wrong but it's not *totally* wrong. > a) The semantics of `CodecDelay` are unclear: Although the current > wording only speaks about a delay that should be subtracted from the > timestamps, the muxers are actually using this value to map the Opus > preskip to a Matroska field, i.e. they are treating it as if it would > not only be a delay that should be subtracted from the timestamps, but > as if it also signaled `DiscardPadding` at the beginning. There was a > discussion in May 2016 in which everyone agreed that `CodecDelay` > included discarding samples at the beginning and the following proposal > for an alternative wording was made (see > ): > "CodecDelay is the duration of the codec-built-in delay in nanoseconds. > The decoded frames from the beginning of a stream should be discarded > until CodecDelay duration has passed, and CodecDelay must be subtracted > from each block timestamp in order to get the presentation timestamp. > The value should be small so the muxing of tracks with the same > presentation timestamp are in the same Cluster." OK, something needs to be done here. It seems clear that the data dropping is needed for Opus. It's not entirely clear for AV1. Since the DRAP is delayed it may have data before the actual keyframe is shown. We could decide that it should not be shown and so it would be like Opus (after clarifying the dropping of CodecDelay). I added an issue so we don't forget to clarify this: https://github.com/Matroska-Org/matroska-specification/issues/242 > This has not been agreed on not because the discard-part was > controversial per se, but because it was unclear how exactly to convert > this ns value to precise samples given that they use different timebases. > > Notice that one thing has not been brought up during the linked > discussion: What exactly will be discarded? Is it always the first > `CodecDelay` output or is it only the output which has a negative As you quoted the CodecDelay definition and the change proposal you know it applies to all timestamps in the track. > timestamp after the timestamp shift specified by `CodecDelay` has been > applied? This is important if the lowest timestamp is >0 (before > shifting by `CodecDelay`). In the second case one would have to > additionaly use a `DiscardPadding` element at the beginning to signal > that the appropriate number of samples should be discarded. No muxer I > know of currently does this, i.e. they work as if not only the samples > that end up with a negative timestamp after the shift were discarded. > And if we keep this logic then the first frames will be decoded, but > discarded under your proposal; if we don't keep it, then some files will > have been wrongly muxed (i.e. then current muxers haven't signalled the > Opus `PreSkip` correctly at all and people that relied on this > interpretation are screwed). > > b) The elementary streams of other codecs also has such an offset (See > e.g. equation (C-12) in the H.264 standard.), but we don't store it for > them either. And it works. I'm not well versed in other codecs. I can tell you about a lot issues with hardware decoder that fail after seeking though. So maybe it works most of the time but it's not clear cut. > c) The time in Annex E is obviously a dts given that it starts at zero > when decoding begins. (Here I consider the arrival of the data in a > buffer as part of the decoding process.) But Matroska uses pts. The only > thing that your proposal does is signalling the pts to dts difference. > But only for the very first picture. This is enough so that one can play > the track smoothly from the beginning; but it is not enough in general. > > Let me explain: Nothing guarantees you that the pts to dts offset for > the first frame is the pts to dts offset for any further keyframes (even > when we restrict this to the keyframe RAP and not the delayed RAPs; > after all, if one discards from a CVS all OBUs in front of a keyframe > RAP, one still has a valid CVS (provided that the `Sequence Header OBU` > is still available)). That's because it may depend on > [buffer_removal_time] which needs to be signalled for every frame. Now > the obvious solution for this would be to use the highest pts to dts > offset for any RAP (or only for every keyframe RAP; I actually don't > know if one would have to use the offset between pts and dts for the > delayed RAPs, too). This has the downside that you would have to do the > muxing in two passes; after all, if you notice that you need to use an > increased value of `CodecDelay`, you would have to rewrite not only the > track header, but the timestamps written so far as well if you tried to > mux in one pass. > Here is a situation where the above scenario might be realistic: > You have a vfr track that actually consists of several sections that are > cfr (but when combined are nevertheless vfr). The beginning is (say) > 50p, the rest is 25p (this really happens: if the movie is shot in 25p, > but the title and end credits are overlayed as 50p, then the result is > effectively 50p at the beginning and end and 25p in the middle; I have > several TV broadcasts like that). Then it is quite likely that the > individual 50p frames are smaller (coded size) so that they can be > transmitted faster and so one can use a lower ScheduledRemovalTiming for > the DFGs corresponding to the 50p frames. This means that the Removal > times (in decoding schedule mode) will be smaller as well. Hence the > InitialPresentationDelay for decoding from the very first frame is > smaller than the InitialPresentationDelay for decoding from one of the > later RAPs. > For this reason decoders would probably rely on the > [initial_display_delay_minus_1] values (that are luckily not stripped > away). But then there is no point in using `CodecDelay` at all as this > value can also be used at the beginning. It depends if CodecDelay always means dropping the amount of CodecDelay before rendering data. In this case it doesn't correspond to the [InitialPresentationDelay]. But if it doesn't (and it could be managed by a separate flag) then it can make sense. I believe the values are added in MP4 so that seeking happens smoothly, ie you know how much you need to buffer when seeking. It may also be a way for adaptive streaming to tell how many frames need to be grabbed in advance when switching to another bitrate/resolution. It would be an odd way to do it though. I don't agree that it's the dts though. Because the dts would be strictly incrementing. But the DRAP is in fact decoded many frames before being displayed (in the normal usage case). > d) There are additional complications: > > i) It would complicate muxers that would have to implement the > procedures described in E.4. Yes. This is also the case for MP4 to find the relevant "initial_presentation_delay_minus_one" value. > ii) It breaks compability with mp4/ISOBMFF because they only store the > initial_presentation_delay_minus_one value (storing it is recommended, > but not required) and (that's a SHOULD) they should set the I don't see where it says it's a SHOULD. Only the pseudo-algorithm "SHALL" be applied to the data and still be a valid bitstream. It's not a MUST either so it may be optional, but it's not clear. > [timing_info_present] flag to zero which means that the bitstream itself > wouldn't contain an explicitly signalled presentation delay any more. > One would have to recalculate the `CodecDelay` value, but there is a > problem with this approach: Appendix E.3 knows two modes: Resource > availability mode and the decoding schedule mode. The latter needs the > buffer_removal_time information from the frame header, but they SHOULD > be stripped away during muxing into mp4, so they are likely not > available and the calculation can't be performed at all. The former has > the prerequisite that it needs to be crf ([equal_picture_interval] equal > to 1). Given that the timing information has been discarded, > [equal_picture_interval] is generally unavailable in mp4, but we can > still check whether it is crf. And what if it isn't? What `CodecDelay` > value does one use? > > iii) It is not possible every time even when muxing from an AV1 > elementary stream: A bitstream can contain > [initial_display_delay_minus_1] even when it doesn't signal explicit > timing info or when it doesn't signal the decoder_model_info or the > operating_parameters_info or if it doesn't signal the buffer removal time. > In my experience with H.264, it was enough to explicitly signal the > [max_num_reorder_frames] for correct playback; it was not necessary to > also add the SEI messages necessary for conformance testing of the > track. (This is also how x264 works by default: No unnessary SEIs, but > the [max_num_reorder_frames] is set.) I actually wouldn't be surprised > if an x264-like AV1 encoder would use the same defaults: Signal > [initial_display_delay_minus_1], don't signal the buffer_removal_time > (too many wasted bits for no gain) and don't signal the operating > parameters. > > iv) Of course the `CodecDelay` would also depend on the operating point > (if scalability is used). I have to admit this could be easily > rectified: Simply calculate the `CodecDelay` for every operating point > and use the maximum. If it's [InitialPresentationDelay] then it's implied. If it's based on something else like [initial_display_delay_minus_1] then it needs to be the maximum, yes. > v) Purely Matroska-wise, there are further complications: Splitting such > a file would mean that the muxer has to use the correct `CodecDelay` > value for the second file. This is in general a different value than the > value for the first file, but the value can't be calculated because the > timing information has been discarded from the bitstream. No. At least in the case of Opus, splitting a file keeps the same `CodecDelay` value. Splitting is just like seeking in this case. > Alternatively appending files where two tracks that are to be appended > to one another have different `CodecDelay` values also requires changes; You can't append tracks with a different CodecPrivate. And if it's the same I would assume the CodecDelay is the same (either in Opus or AV1). > if the second file has the higher `CodecDelay` value, one would have to > increase the `CodecDelay` for the whole output track to that value and > this entails to shift the timestamps of the first file, too. > There probably are more complications further down the road. > > 3. Here is a counterproposal: We more or less copy how mp4 does it. I.e. > we ignore InitialPresentationDelay when determining the timestamps and > we add a field that is equivalent to their > [initial_presentation_delay_minus_one]. > Honstely I don't even know what they exactly mean by "display model > verification algorithm". Is it the procedure described in annex E? If > so, they can run this algorithm only if the video is cfr (resource > availability mode) or the buffer_removal_time is given in the frame > headers (decoding schedule mode), but the latter SHOULD have been > striped away, so that leaves a crf requirement. (Or they could have kept > the relevant paramters from the elementary stream (maybe not in the mp4 > file, but somewhere else).) > Anyway, for our purposes it will be enough to use the maximum of all > [initial_display_delay_minus_1] values if said value is present for > every operating point that occurs in the file. It is one of the fields > which can't change mid-way in a CVS so it works well for one-pass muxers. I still don't understand this part in MP4. This is information that is already in the Sequence Header OBU that is (but not always) following this data. And in our case the Sequence Header OBU is mandatory so no need to copy this information again. > The easiest way would be probably to add a byte to the beginning of > `CodecPrivate` (whose definition would have to be changed). How about > the MSB bit being the "initial_presentation_delay_present" bit, then > three reserved bits (set to zero, may hypothetically be used to indicate > a new version of the AV1 packetization in Matroska if we wished to keep > the AV1 CodecID) and then the `initial_presentation_delay_minus_one`, > coded on 4 bits. No, we're not going to change the interpretation of a field that's already widely used (WebM). We can add an extra element with a default value that matches what is done with Opus. For example whether the CodecDelay means discarding that amount of data after a discontinuity (start or seek). For the record VLC doesn't discard this amount of data on output of the decoder. I'm not sure avformat/avcodec is capable of doing that either. > If someone wonders why I propose to add such a field even when the > information is in the 'Sequence Header OBU` (after all, this is not > stripped away): Because it actually needn't be in the `Sequence Header > OBU`! E.g. think of an AV1 bitstream inside mp4 where the `Sequence > Header OBU` delay field doesn't exist, but where the initial delay is > signalled on the container level. Muxing from mp4 to Matroska would > loose this information when it is not kept somewhere. Furthermore, just It wouldn't be lost. To mux in Matroska you'd need to recover the Sequence Header OBU from the MP4 to put it in the CodecPrivate (it's an OBU that preceeds the first keyframe). > as a muxer can actually count the number of reorder frames necessary for > H.264, it can probably do the equivalent with the initial presentation > delay for AV1 (this is not to say that this should be required or even a > SHOULD) and then it can put the value into this field (rewriting a field > in the track header is possible even for a one-pass muxer). I think we went full circle. Do we need to do anything at all ? The initial problem is with DRAP and seeking, right ? The Block with a "keyframe" data is marked as a keyframe whether it's displayed or not. So the real problem is what to do with the frames before the keyframe is displayed. The spec is blurry on what to do with frames in between (big Note in 7.6.3: "the preceding frames are implementation defined"). And it's likely not a choice to be made at the container level. So maybe we just don't care. > 4. How helpful is this buffer removal timing information actually? Is it > needed for streaming? I don't think so, because then the mp4 guys (one > of the main authors of the mp4 packetization works for Netflix) would > not have recommended to strip it away. But maybe conformance testing > should be mentioned as another reason not to strip the [timing_info] > away. (I would only keep the timing_info parameters for a crf track in > order to have the real framerate stored, but I would discard the > decoder_model_info stuff.) > > - Andreas Rheinhardt Thanks for taking the time for all these detailed talks. It really helps to have a foolproof solution in the end. > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar -- Steve Lhomme Matroska association Chairman From nobody Wed Jul 18 06:42:58 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B7AE4127333 for ; Wed, 18 Jul 2018 06:42:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, T_DKIMWL_WL_MED=-0.01] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UbLBiDZtwuJV for ; Wed, 18 Jul 2018 06:42:54 -0700 (PDT) Received: from mail-pl0-x22e.google.com (mail-pl0-x22e.google.com [IPv6:2607:f8b0:400e:c01::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1209C126F72 for ; Wed, 18 Jul 2018 06:42:53 -0700 (PDT) Received: by mail-pl0-x22e.google.com with SMTP id s17-v6so2064658plp.7 for ; Wed, 18 Jul 2018 06:42:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=L18+yXQ6AwYg/4DuPi6xjRTMavQbH3vZqBmey1oUt+A=; b=idbz1Ju5hv9gZhIGSpmGKwjvKQN/bkz/GOk6+2GfTI/Wo+pVhGrP+fGC8dDEuCkf1u n7gpcru/4KCPGqe4uLU1ftla/PhAHLOTrySPtKrsYrwY5AaYjWd0mB15mywnA/liY2tb Fyh/JQ3XctFjA9U/Wkpy5R2bEe8xDuapZ5eVuj8MfSvtu2WNNqi9x7wMuY6K94dhn4y7 NweLjzPE/1Wd0oU38bIdEUaH4QoLq2BtmAUIp25PBBrjGyoiRFicyTgBsUKBrVqSedIq h8Ohy16kZG2uyT/PZ3pKiIe+c2uqz8cUl+uyuBDyuEP1qeTZg4l7pBrMcwlfKyzYkq3V ZutQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=L18+yXQ6AwYg/4DuPi6xjRTMavQbH3vZqBmey1oUt+A=; b=f3wU3J/Ng8IKvpAyQR3+b+RDTRJcJv8PypH61vBEUzprpYNdiIY8FMXGpy0UlUoXa1 0xReomV4Bd2v41GenbgiZ1w9QAYjR8t8F4dEjfLMwpw2b09+tFAaB6TF/R2KDE8iQXT9 h0S9hi+5Hve5lTmgEPlZLc7kZTXP8i4MdZoorwoMbN618ExXRjRalRTEDxF8KNwKc+sr +gaofjKipRxF9LrYhNaDNvtHdEX7KmQ9sQOopdc7QAaH4RxB5wA+vSTb2KFMTxOCIUF4 hPLp+BUegoIJijnr4WE/bJ+llO1spIJya1oYN/5Iqk5Jb7lfDLb3WBz9i914p3OHsWeI 54YQ== X-Gm-Message-State: AOUpUlFnNiLeMbT3NO+L5qhRWbsZ5siemo2o3rh8VEM5Jo0GcxzhlEOp 8BhS4FHcuVKBKa6ay39XORl9JO7BJRhnxoso5GkeXrsy X-Google-Smtp-Source: AAOMgpfkzW+HEQyT2NxU6toWirhaO6UaOwb+rndPOD8JseAPtZvJb/4kOSr+60Ns8kVpJtq0Fl4boO/5HoSk97tqLTM= X-Received: by 2002:a17:902:1025:: with SMTP id b34-v6mr5887761pla.112.1531921373391; Wed, 18 Jul 2018 06:42:53 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a17:90a:9c13:0:0:0:0 with HTTP; Wed, 18 Jul 2018 06:42:52 -0700 (PDT) In-Reply-To: References: <62c29889-49f5-6634-049a-a2d73315bb3c@googlemail.com> <4ff1f1f8-3b66-e229-7884-c851dae8e2e0@googlemail.com> From: Steve Lhomme Date: Wed, 18 Jul 2018 15:42:52 +0200 Message-ID: To: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cellar] AV1 seeking X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jul 2018 13:42:56 -0000 I asked Cyril Concolato to join the conversation so we can discuss the differences between our current mapping and the ISOBMFF/MP4 one he is working on. IMO there are two main differences for now: - initial_presentation_delay_present / initial_presentation_delay_minus_one which we may or may not store (and where does it come from) - the configOBU that may be completely empty, whereas in our case we have exactly one Sequence Header OBU and the ones in the Segment must be very close to that one. -- Steve Lhomme Matroska association Chairman From cconcolato@netflix.com Wed Jul 18 07:59:51 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B5627130FD1 for ; Wed, 18 Jul 2018 07:59:51 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.009 X-Spam-Level: X-Spam-Status: No, score=-2.009 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, T_DKIMWL_WL_HIGH=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=netflix.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nmyxjUafhEvQ for ; Wed, 18 Jul 2018 07:59:48 -0700 (PDT) Received: from mail-it0-x235.google.com (mail-it0-x235.google.com [IPv6:2607:f8b0:4001:c0b::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B740F130E19 for ; Wed, 18 Jul 2018 07:59:47 -0700 (PDT) Received: by mail-it0-x235.google.com with SMTP id j185-v6so4740125ite.1 for ; Wed, 18 Jul 2018 07:59:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netflix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=DGMYIz8ZgM21UcpDgpnPvk9X9N32tI9PyWADVw2kCQk=; b=hpOI+xdQZ8U0AE5mKcandt6A5jQ7Ga5NOj/uwGHBWeOHmMGb7LBQCGcKHszm5QPTIZ wjfTbLAlpLMWmAFqVNzuLnNsKr7qgjP3UGvLajw1bpaUijq0S8955v701UgHS0bU0Bcx Y9B4aXlDBs7vR7VOBhajIZgExgtxPEPyZkv+w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=DGMYIz8ZgM21UcpDgpnPvk9X9N32tI9PyWADVw2kCQk=; b=VUjdUiwDU+8ouTFI9kYoHt9KbA3Wr4bBi+0S4xVt9crpZeuNPEIMupKIazqRsmDwfG 3Vu9TbWJxve+xeaSBeYcelLUklLwgEuSi//VMYUtWlyPS5pw0LdpkLKzJ1Z3R3k+zZno C6eT/kDergvz9V53pLHepgao6Nsy9119JBwu0FPFraTFYtNMzmalPj4BdbBTXupXxl1d YZjPaTGR8KBkCQ5n+HNgBsok6/knJvODUKIvI4nEAIkYdSGSuAlDBP2l4a0M69LZGdCY QfeMgI23ExUlSWKZq1fkBFw5/7uask/jATZFWgGxbWLQTlCb0Bea7XOU+JIl7AEFqxkv uyCA== X-Gm-Message-State: AOUpUlHxmk7pxr1eJM0u5QcN1r9xyZJK/r5w/Yuuq+VCQ7tC7zqN9ZU6 QuAUemWz74dB5ZJen7+JU1INbPoQcVfRGt7Ng9uNjM6IaXo= X-Google-Smtp-Source: AAOMgpeEe/YeHc/oQYBYPIZm8ECFkWIiYWb7LClWLc55PS6MKf/71p3WnbVtAufwl2WMqrh9XM5Vx7jQsDz48Zg0g/k= X-Received: by 2002:a02:687:: with SMTP id 129-v6mr5594195jav.59.1531925986922; Wed, 18 Jul 2018 07:59:46 -0700 (PDT) MIME-Version: 1.0 References: <62c29889-49f5-6634-049a-a2d73315bb3c@googlemail.com> <4ff1f1f8-3b66-e229-7884-c851dae8e2e0@googlemail.com> In-Reply-To: From: Cyril Concolato Date: Wed, 18 Jul 2018 16:59:35 +0200 Message-ID: To: slhomme@matroska.org Cc: cellar@ietf.org Content-Type: multipart/alternative; boundary="00000000000045fd7a0571475060" Archived-At: Subject: Re: [Cellar] AV1 seeking X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jul 2018 15:00:46 -0000 --00000000000045fd7a0571475060 Content-Type: text/plain; charset="UTF-8" On Wed, Jul 18, 2018 at 3:43 PM Steve Lhomme wrote: > I asked Cyril Concolato to join the conversation so we can discuss the > differences between our current mapping and the ISOBMFF/MP4 one he is > working on. > Thanks Steve. Some comments regarding the previous messages I read in the archives of this list: You wrote: "Here's an example stream a b [C d] e f g h i [c] j k l C is the non visible keyframe which (from what I understand) always have another frame in the Temporal Unit, d in this case. And [c] is displayed later in the stream. If a user wants to seek to frame 'f', the normal seeking would start decoding at 'C'. But f may depend on frame 'a' (which hopefully is a keyframe otherwise we need the frames referenced by 'a'). Also this is very much the concept of B frames since 'c' is displayed after 'f' but with a different name." [CC] In this configuration, to seek to frame 'f' you need to start decoding at frame 'a' (visible Key Frame). To seek to frame 'k' you need to decode only 'C' and then 'c', and 'j'. "In MP4 they have [initial_presentation_delay_minus_one] in the CodecPrivate. I did not understand it so far because it's not found in the AV1 spec. But it seems to guarantee that to read frame 'f' you need to decode X frame before that one. In our case that would be 5 to have at least a decoded." [CC] "initial_presentation_delay_minus_one" in the AV1-ISOBMFF is related to the "initial_display_delay" in the AV1 codec spec. They express the same delay but in different units: the latter expresses it in number of decoded frames, while the former expresses it in number of presented frames. The reason for this difference is that an application may not be able to know when a frame is not decoded it only sees when it is output. That said, this delay is not at all related to the seeking feature. It tells you how many frames have to be decoded (or output) to guarantee smooth playback if the decoder decodes at the decode rate of the level indicated in the bitstream. It may be useful for low latency applications. "IMO this is a bad design because it mixes the container with the codec (something done in the past in ogg with bad results). Seeking should not ask the codec how it should be done. Also that means to mux such an MP4 you'd need to scan the whole source first to verify the value is correct or that information needs to be given to the muxer by the encoder/packetizer (this information is not found in the stream)." [CC] I don't understand what this means. > > IMO there are two main differences for now: > > - initial_presentation_delay_present / > initial_presentation_delay_minus_one which we may or may not store > (and where does it come from) > [CC] Hope it's clearer. > > - the configOBU that may be completely empty, whereas in our case we > have exactly one Sequence Header OBU and the ones in the Segment must > be very close to that one. > [CC] The intent of the AV1-ISOBMFF spec is that the Sequence Header OBU has to be present in-band and may be present in the configOBU and if present shall be identical. In your spec, I don't understand what "very close means". AV1 does not allow variation of Sequence Header within a CVS. HTH, Cyril > > -- > Steve Lhomme > Matroska association Chairman > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > --00000000000045fd7a0571475060 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

On Wed, Jul 18, 2018 at 3:43 PM Steve Lhomme <slhomme@matroska.org> wr= ote:
I asked Cyril Concolato to joi= n the conversation so we can discuss the
differences between our current mapping and the ISOBMFF/MP4 one he is
working on.
Thanks Steve.

S= ome comments regarding the previous messages I read in the archives of this= list:

You wrote:
"Here's an example stream<= /div>
a b [C d] e f g h i = [c] j k l
C is the n= on visible keyframe which (from what I understand) always
have another frame in the Temporal = Unit, d in this case. And [c] is
displayed later in the stream.

If a user wants to seek to frame 'f', the normal seeking w= ould start
decoding = at 'C'. But f may depend on frame 'a' (which hopefully is a=
keyframe otherwise = we need the frames referenced by 'a'). Also this is
very much the concept of B frames s= ince 'c' is displayed after 'f' but
with a different name."

[CC] In this configuration, to seek to frame '= ;f' you need to start decoding at frame 'a' (visible Key Frame)= . To seek to frame 'k' you need to decode only 'C' and then= 'c', and 'j'.

= "In MP4 they have [initial_presentation_delay_minus_one] in the
<= div style=3D"font-size:small;background-color:rgb(255,255,255);text-decorat= ion-style:initial;text-decoration-color:initial">CodecPrivate. I did not un= derstand it so far because it's not found in
the AV1 spec. But it seems to guarantee that t= o read frame 'f' you
need to decode X frame before that one. In our case that would be = 5 to
have at least a= decoded."
[CC]= "initial_presentation_delay_minus_one" in the AV1-ISOBMFF is r= elated to the "initial_display_delay" in the AV1 codec spec. They= express the same delay but in different units: the latter expresses it in = number of decoded frames, while the former expresses it in number of presen= ted frames. The reason for this difference is that an application may not b= e able to know when a frame is not decoded it only sees when it is output.= =C2=A0
That= said, this delay is not at all related to the seeking feature. It tells yo= u how many frames have to be decoded (or output) to guarantee smooth playba= ck if the decoder decodes at the decode rate of the level indicated in the = bitstream. It may be useful for low latency applications.=C2=A0

"IMO this is a bad design because = it mixes the container with the codec
(something done in the past in ogg with bad results). See= king should
not ask = the codec how it should be done. Also that means to mux such
an MP4 you'd need to scan the = whole source first to verify the value
is correct or that information needs to be given to the = muxer by the
encoder= /packetizer (this information is not found in the stream)."
[CC] I don't understand wh= at this means.

<= /div>
=C2=A0

IMO there are two main differences for now:

- initial_presentation_delay_present /
initial_presentation_delay_minus_one which we may or may not store
(and where does it come from)
[CC] Hope it's clear= er.
=C2=A0

- the configOBU that may be completely empty, whereas in our case we
have exactly one Sequence Header OBU and the ones in the Segment must
be very close to that one.
[CC] The intent of the AV1-= ISOBMFF spec is that the Sequence Header OBU has to be present in-band and = may be present in the configOBU and if present shall be identical. In your = spec, I don't understand what "very close means". AV1 does no= t allow variation of Sequence Header within a CVS.

HTH,
Cyril
=C2=A0

--
Steve Lhomme
Matroska association Chairman

_______________________________________________
Cellar mailing list
Cellar@ietf.org https://www.ietf.org/mailman/listinfo/cellar
--00000000000045fd7a0571475060-- From nobody Wed Jul 18 08:36:16 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5469F13119E for ; Wed, 18 Jul 2018 08:36:13 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.909 X-Spam-Level: X-Spam-Status: No, score=-1.909 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, T_DKIMWL_WL_MED=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lrozlmrcMxlp for ; Wed, 18 Jul 2018 08:36:10 -0700 (PDT) Received: from mail-pg1-x530.google.com (mail-pg1-x530.google.com [IPv6:2607:f8b0:4864:20::530]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A3381130F63 for ; Wed, 18 Jul 2018 08:36:10 -0700 (PDT) Received: by mail-pg1-x530.google.com with SMTP id y5-v6so2185475pgv.1 for ; Wed, 18 Jul 2018 08:36:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=DHLlXf60ql39Uont6ku8v/Ey3e8uR5TmuLJgkVH2OIs=; b=PmJTcYATAHNODZYUigiwE7qa+XSoObvviqDtgLFXX46OeGkwinsbrq28nT53iTM/my iqKHThbdVE1QiypCSsNodaQRSzD4QxZGAVastHnZiywVFxt1VX631TqSno2EwwP/qbve cZVVqPrxpn6G0zMen8nYj9G/Nnnbbhy9ZkT1igmBsrZIFbeBILh3nIFDNH+k4Zp/wRgm G318cXxenron5Cc9SS2yseiFrS1OFQvDEp1er9BOHcsHbLEwqu2jSJjY9kkon4CHJJKN Ky9+Uwez0R2onmnNx1UOwb8ptx3b9BTE05WrXevVbfhUXRsjhZ9brncILoDxjEtrKfLj 83Jw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=DHLlXf60ql39Uont6ku8v/Ey3e8uR5TmuLJgkVH2OIs=; b=tYKviCGBjSjPrjdbdrNtQ/z5eEViLgIlwPoGpghxP4hYqQUFuTQ9Pq+oeDkJs18B2e 3E+sduzDFtTduMfkiTHHvLWVa/FKnhEoOPbMepIMuEH+dMbzz/mwHY3xywVY8qwTUx1i H8KjtgSfcLgoE7z14VxXpZb4M8pPHEuZLEzQYWoGZcG65jr/J52M0qbIZzuGbVHmvSlu XIaia4kHkZNPtvGk7Q75fVIPpYDUT2gzZcTq/srQmiNFYUi3yJg35FZSk//298czW7Op y9BMe/9EbbqtdIj925cRB6OfsH/ah5bIlm9a/XNM44I3giEPWceQM+C/bwHSVRTfa8e9 +m0A== X-Gm-Message-State: AOUpUlFr1D5r6fypnAbn9f6eHy/jq652dcbMhYm1IL+p1dQtANsvRaBv JJ/cpJxIga77/+KWDTWJK+vPIvPQvx4Mxjhsi+KXZQ== X-Google-Smtp-Source: AAOMgpfB9jwUOr3lpzz7a9rHdi0ZGI1H0ykOQWZmGSoUGS9g8VfoGoXQ+w8oCO/BJQlIx+HT98gHk7j7/5nGXkvA2B0= X-Received: by 2002:a65:5245:: with SMTP id q5-v6mr6165330pgp.67.1531928170129; Wed, 18 Jul 2018 08:36:10 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a17:90a:9c13:0:0:0:0 with HTTP; Wed, 18 Jul 2018 08:36:09 -0700 (PDT) In-Reply-To: References: <62c29889-49f5-6634-049a-a2d73315bb3c@googlemail.com> <4ff1f1f8-3b66-e229-7884-c851dae8e2e0@googlemail.com> From: Steve Lhomme Date: Wed, 18 Jul 2018 17:36:09 +0200 Message-ID: To: Cyril Concolato Cc: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cellar] AV1 seeking X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jul 2018 15:36:14 -0000 Hi Cyril, Thanks for clarifying some things. 2018-07-18 16:59 GMT+02:00 Cyril Concolato : > > On Wed, Jul 18, 2018 at 3:43 PM Steve Lhomme wrote: >> >> I asked Cyril Concolato to join the conversation so we can discuss the >> differences between our current mapping and the ISOBMFF/MP4 one he is >> working on. > > Thanks Steve. > > Some comments regarding the previous messages I read in the archives of this > list: > > "In MP4 they have [initial_presentation_delay_minus_one] in the > CodecPrivate. I did not understand it so far because it's not found in > the AV1 spec. But it seems to guarantee that to read frame 'f' you > need to decode X frame before that one. In our case that would be 5 to > have at least a decoded." > [CC] "initial_presentation_delay_minus_one" in the AV1-ISOBMFF is related to > the "initial_display_delay" in the AV1 codec spec. They express the same > delay but in different units: the latter expresses it in number of decoded > frames, while the former expresses it in number of presented frames. The > reason for this difference is that an application may not be able to know > when a frame is not decoded it only sees when it is output. > That said, this delay is not at all related to the seeking feature. It tells > you how many frames have to be decoded (or output) to guarantee smooth > playback if the decoder decodes at the decode rate of the level indicated in > the bitstream. It may be useful for low latency applications. OK but it is a duplicate value from the Sequence Header OBU may (or may not) be found in the configOBU. Maybe it would be worth having a specific atom just for that ? And you have either this atom or the one with a Sequence Header OBU but not both. There are many [initial_display_delay_minus_1] values in a single Sequence Header OBU. So I assume the value stored in MP4 is the maximum value ? IMO it should be said in the document. Also in what cases a Sequence Header OBU would not be found in the configOBU ? IMO it should be specified. We may have the same requirement but it seems better to me to always have one. It helps knowing the profile of the codec to initiate the decoder instance. Otherwise we need to wait for the actual data when playback has already started (along with audio and subtitles). > "IMO this is a bad design because it mixes the container with the codec > (something done in the past in ogg with bad results). Seeking should > not ask the codec how it should be done. Also that means to mux such > an MP4 you'd need to scan the whole source first to verify the value > is correct or that information needs to be given to the muxer by the > encoder/packetizer (this information is not found in the stream)." > [CC] I don't understand what this means. Ogg is notorious for tying the container and the codec data. You can't know the timestamp from just reading ogg, you need to know the codec and analyse the data inside. So if seeking in AV1-in-ISOBMFF required the demuxer to parse the "initial_presentation_delay_minus_one" data would be a bit similar. But as you explained, it's not the case. >> >> >> IMO there are two main differences for now: >> >> - initial_presentation_delay_present / >> initial_presentation_delay_minus_one which we may or may not store >> (and where does it come from) > > [CC] Hope it's clearer. Yes >> - the configOBU that may be completely empty, whereas in our case we >> have exactly one Sequence Header OBU and the ones in the Segment must >> be very close to that one. > > [CC] The intent of the AV1-ISOBMFF spec is that the Sequence Header OBU has > to be present in-band and may be present in the configOBU and if present > shall be identical. In your spec, I don't understand what "very close > means". AV1 does not allow variation of Sequence Header within a CVS. "Very close" was in my email but that's not the wording in the MKV/WebM mapping. In that document there is this CVS documentation: "A Coded Video Sequence is a sequence of video frames where the contents of [sequence_header_obu] must be bit-identical for all the Sequence Header OBUs found in the bitstream before Matroska encapsulation except for the contents of [operating_parameters_info]." [operating_parameters_info] is allowed to change in a CVS in our case. This is the same as found in the AV1 specification 7.5 Ordering of OBUs: "Sequence header OBUs may appear in any order within a coded video sequence. Within a particular coded video sequence, the contents of sequence_header_obu must be bit-identical each time the sequence header appears except for the contents of operating_parameters_info." I suppose by "shall be identical" that's the change that is allowed. But is there are reason not to put a Sequence Header OBU in the AV1CodecConfigurationRecord ? In Matroska we advise to strip the similar ones from the bitstream to save space so that's a direct benefit of putting it there. But there's also a benefit for playback (a similar feature is missing from VP9 and causes hardware detection issues in VLC). > HTH, > Cyril Thanks again. >> >> -- >> Steve Lhomme >> Matroska association Chairman >> >> _______________________________________________ >> Cellar mailing list >> Cellar@ietf.org >> https://www.ietf.org/mailman/listinfo/cellar -- Steve Lhomme Matroska association Chairman From nobody Wed Jul 18 12:33:35 2018 Return-Path: X-Original-To: cellar@ietf.org Delivered-To: cellar@ietfa.amsl.com Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id C078D130F7F; Wed, 18 Jul 2018 12:33:29 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: internet-drafts@ietf.org To: Cc: cellar@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 6.82.0 Auto-Submitted: auto-generated Precedence: bulk Reply-To: cellar@ietf.org Message-ID: <153194240973.2968.9117202950899720357@ietfa.amsl.com> Date: Wed, 18 Jul 2018 12:33:29 -0700 Archived-At: Subject: [Cellar] I-D Action: draft-ietf-cellar-ebml-05.txt X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jul 2018 19:33:30 -0000 A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Codec Encoding for LossLess Archiving and Realtime transmission WG of the IETF. Title : Extensible Binary Meta Language Authors : Steve Lhomme Dave Rice Moritz Bunkus Filename : draft-ietf-cellar-ebml-05.txt Pages : 38 Date : 2018-07-18 Abstract: This document defines the Extensible Binary Meta Language (EBML) format as a generalized file format for any type of data in a hierarchical form. EBML is designed as a binary equivalent to XML and uses a storage-efficient approach to build nested Elements with identifiers, lengths, and values. Similar to how an XML Schema defines the structure and semantics of an XML Document, this document defines how EBML Schemas are created to convey the semantics of an EBML Document. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-cellar-ebml/ There are also htmlized versions available at: https://tools.ietf.org/html/draft-ietf-cellar-ebml-05 https://datatracker.ietf.org/doc/html/draft-ietf-cellar-ebml-05 A diff from the previous version is available at: https://www.ietf.org/rfcdiff?url2=draft-ietf-cellar-ebml-05 Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From nobody Wed Jul 18 21:29:36 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 475AF130E37 for ; Wed, 18 Jul 2018 21:29:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, T_DKIMWL_WL_MED=-0.01] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0vx0WGnA5Zzb for ; Wed, 18 Jul 2018 21:29:32 -0700 (PDT) Received: from mail-pg1-x52c.google.com (mail-pg1-x52c.google.com [IPv6:2607:f8b0:4864:20::52c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 860BC12872C for ; Wed, 18 Jul 2018 21:29:32 -0700 (PDT) Received: by mail-pg1-x52c.google.com with SMTP id r1-v6so2990086pgp.11 for ; Wed, 18 Jul 2018 21:29:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to:cc; bh=dA9aJ/BEBZ0BAxxJSsuBxUPTmyf8yOilmAJBKCQ3ayQ=; b=HLEGKMiNJ6eNaebDkojRjduM115JNDKsM6VccnwR3IImS79JYViT8S8j5Gta2xoHWi Vu4lgA7UA7AMgqJgqnBgqO9YzmV+kVfraBtPEhdjgOgl8YKkXDv+VGU2G5OGOokloKkp HVOtUb0EmhbpcuVUrITQsLePyr96DfXavHpr4/l+5JO5lTNFldkx/khG02WV3Esl9xfe GT3hJw8WhLV+ExgTmYc3CqGh4vnHg9bIlGCAgFMgnkcMMjbmJ63CV7PagWi72jrtpkPq H+Z4PkwplSWzFX2fGQl3nfSOw0iqU0djdJPfJTiQJQfOooft+ZNTPbsqMaBymVFYaZt1 HECw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=dA9aJ/BEBZ0BAxxJSsuBxUPTmyf8yOilmAJBKCQ3ayQ=; b=pSQHzlLLF1Il0LqaVDBmMeVZc16c43Y5GSBy8u/RUurZHXoA4Wr4Hy/hedfEc+BE1c 7JoEE951mUwwE8g+b2fWlZP5U7GIralHutsQ2Un6YeIJC+kRKzx/R2lxqgWnztIa6vU1 CmWOTzP43xjIpzu84+Z5TWv2RIzU0ur6jqyl3aGLwz3yHHx2t6fc5lcUHuEj+bxN1+J8 jcyi5YySYLChE5sq/GPZlhhUyw4L46ebhxlufYgXZ2bP68p/znKIARi8YG0oZgwY/il8 J2O1LczoL/JNZQ+MxsTY2uL5cc2c73llGxsmr9mllQCw+BpzY9hKKcUvZ7cxjs3l6wIE Evgg== X-Gm-Message-State: AOUpUlH5oZmx0SGWacOpytk4CJPL4eDT6N0a2CQklJ9Odf4tFZwUC0rF do4aE2Og0/sf+wPGA3sasrLmas+OPnJ4G/Mw6Tpqel4c X-Google-Smtp-Source: AAOMgpd3NUm7x4kbOC0yljmZX8ptRzW4Il0KMA/sIgDK5bUhT1WiMi7jWN4PWBurRucaghaqkhEcda1wDY64r5TZD9w= X-Received: by 2002:a65:5307:: with SMTP id m7-v6mr8580158pgq.431.1531974571415; Wed, 18 Jul 2018 21:29:31 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a17:90a:9c13:0:0:0:0 with HTTP; Wed, 18 Jul 2018 21:29:31 -0700 (PDT) From: Steve Lhomme Date: Thu, 19 Jul 2018 06:29:31 +0200 Message-ID: To: Codec Encoding for LossLess Archiving and Realtime transmission Cc: Cyril Concolato Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: [Cellar] AV1 init X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jul 2018 04:29:35 -0000 Hi, As the CodecDelay seems to be unnecessary, we're getting close to finalizing the AV1 mapping for Matroska and WebM. It would be beneficial to everyone that Matroska and MP4 (ISOBMFF) share as much as possible so it's easy to transmux from one to the other. The initial_presentation_delay_present we may not deal with at all or we should have a specific Matroska element that can be used by all codecs. So it doesn't really go into the AV1 codec mapping (more an MKV/ISOBMFF compatibility document). The other outstanding issue is the configOBU in MP4 that can be empty. In our case we mandate that the Sequence Header OBU that represent the Coded Video Sequence be put in there. Because it's not mandatory in MP4 it's fair to say people will not bother too much writing it. In fact the only currently available encoder doesn't export it, one has to parse the bitstream to find it. See this ffmpeg thread [1] which led to this "fix" request in libaom [2]. When that is done, libaom should be able to output WebM that follow our current mapping [3]. We have the choice of make the Sequence Header mandatory in CodecPrivate (MUST) make it highly recommended (SHOULD), just suggested (MAY) or not put it at all. IMO it should always be there. Because as stated above, if not people will likely not write it. And players will have to handle the case where it's not there. And the reason why it should be there is for playback reasons, so that the right decoder can be selected before having to parse the data. There's a reason H264/H265 have profiles, it's so that you can tell if you can decode something or not and how. In many cases you get hardware decoding and in other cases not. This will likely be the same for AV1, for example there might never be hardware decoders doing 12 bits, that doesn't mean hardware decoding should never be used. In the case of avcodec, deciding if HW is going to be used needs to be done before starting the decoder, before parsing any codec data. Because you need to chose whether 1 thread should be used (a limitation of HW decoders on Windows and Linux at least) or multiple. And you can't switch once the codec is initialized. That situation is still not too bad. If one has to use completely different code for HW decoding (like Intel's MediaSDK API) and for SW decoding (libaom) it gets worse. You'd need to parse the codec data before deciding which one to use. That also requires an extra buffering step at the beginning to probe the codec data and not lose these data (still need to be fed to the codec once it's initialized) (while the interleaved audio is going through the decoder already, especially in WebM where audio should come before video). We also have the counter example of not having the codec characteristics known in advance with VP9. Some GPU are capable of decoding 8 bits VP9 but not 10 bits. In VLC (and other apps using libavcodec) we can't decide whether to use 1 thread or many before intializing the decoder. So we use many which is the case that works best. But when the HW decoder is used it's from multiple threads which is not how it's designed to be used, has performance issues and sometimes crashes. If we make the Sequence Header OBU mandatory in the CodecPrivate we avoid all these issues on the playback side. It puts the trouble of extracting that data on the muxer/encoder side. So I think it's the right way for Matroska/WebM and also the right way for ISOBMFF, so I cc Cyril in this thread. [1] http://ffmpeg.org/pipermail/ffmpeg-devel/2018-July/232084.html [2] https://bugs.chromium.org/p/aomedia/issues/detail?id=2012 [3] https://bugs.chromium.org/p/aomedia/issues/detail?id=2027 -- Steve Lhomme Matroska association Chairman From nobody Thu Jul 19 01:47:47 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B2107130DDF for ; Thu, 19 Jul 2018 01:47:44 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=googlemail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6NpEMjyKpY7b for ; Thu, 19 Jul 2018 01:47:42 -0700 (PDT) Received: from mail-wm0-x244.google.com (mail-wm0-x244.google.com [IPv6:2a00:1450:400c:c09::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DD4A4127148 for ; Thu, 19 Jul 2018 01:47:41 -0700 (PDT) Received: by mail-wm0-x244.google.com with SMTP id s9-v6so5346091wmh.3 for ; Thu, 19 Jul 2018 01:47:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20161025; h=subject:to:references:from:cc:message-id:date:mime-version :in-reply-to:content-transfer-encoding; bh=3MCNipV6Fx3Nr9y0tHlWlq5Vq3j6HYDNQq+wGdyETmc=; b=bhzjgINGjrzFLIH/RYqSSHi0X19NhcSWxgLH7MaLpvcRcvpfodBruIXVm+1RS0ZG8M OAh/7S65isVuKo3lTqWTlPLUZ+aQLF5Ol7Jc7vzpzAXR0SUyAy9W/GN2CqzNpKgIfmbf YkjxB9Wx+lvbVaYjeBqZ7R4h3s6+BMqmTvDVqMQn40LziKNjP8uuptmyaAsKYiCVrRGD G2inB2r+sbN0BOTJsZcxC6p+szFKbCQOfXJA5Z9ljv+R1Ziw6G5/Gy8hqJRMkVY4iR/c cGgr9un1YjCX0lY+jXo6z8ZzkUwdqi2Dn7SMmZHWdNlGPJvHpMcuSoM3KE3pPkcPPgIT +HoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:cc:message-id:date :mime-version:in-reply-to:content-transfer-encoding; bh=3MCNipV6Fx3Nr9y0tHlWlq5Vq3j6HYDNQq+wGdyETmc=; b=turIjMaE+w1SMpnmLTOi+ubhVF1UJXk2fHq8WLR9tregz1gMD2/kx40jA5c/eyBynC yDh5dTTESegjQovr+Z4RU1jQNvCL6XZ8z1jgcCEh2xhIhGHZhCq5xCnVygSaj0oAsWh/ 5ZO/NS8xhL+vc4onAkMgZVy8zH2Zp/+mqBWLUx/BNpArh7O8tQu7uSjT+zLg0j/FCaar f5LQICett9oLfO5zsfT1tM1YMuCDkfAeIm1179KK83AY8DH6AHFgXiDcPCsvhaXaWWEl L8a11yR9R12Bu0joTTyZrM7SpRVJTp2TyaGNED6A03e/CnMKUNiVwbh6Qv8QIbF47xz+ of3A== X-Gm-Message-State: AOUpUlEq8/GqkPoVMSW+eRBNhXLfRSYNZ9AkdMJ5SRuxRHfBW1NYfk4e 0dT6f6ux28kntaAr+I1zolI= X-Google-Smtp-Source: AAOMgpf80/gpqH/62L4TEujITPiZ07AZ7FVfhCS2nKVJQwz8RyOiQfhBRQ1tuDzhz+haaWRNuwASLA== X-Received: by 2002:a1c:ce0b:: with SMTP id e11-v6mr3486017wmg.47.1531990059099; Thu, 19 Jul 2018 01:47:39 -0700 (PDT) Received: from [127.0.0.1] ([185.225.68.13]) by smtp.googlemail.com with ESMTPSA id c10-v6sm4307490wrs.6.2018.07.19.01.47.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 19 Jul 2018 01:47:38 -0700 (PDT) To: cellar@ietf.org References: <62c29889-49f5-6634-049a-a2d73315bb3c@googlemail.com> <4ff1f1f8-3b66-e229-7884-c851dae8e2e0@googlemail.com> From: Andreas Rheinhardt Cc: Cyril Concolato Message-ID: <7d95865e-0314-ee21-640d-69ea29cf4716@googlemail.com> Date: Thu, 19 Jul 2018 08:46:00 +0000 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [Cellar] AV1 seeking X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jul 2018 08:47:45 -0000 Hello Cyril, thanks for joining us here. Cyril Concolato: > "In MP4 they have [initial_presentation_delay_minus_one] in the > CodecPrivate. I did not understand it so far because it's not found in > the AV1 spec. But it seems to guarantee that to read frame 'f' you > need to decode X frame before that one. In our case that would be 5 to > have at least a decoded." > [CC] "initial_presentation_delay_minus_one" in the AV1-ISOBMFF is related > to the "initial_display_delay" in the AV1 codec spec. They express the same > delay but in different units: the latter expresses it in number of decoded > frames, while the former expresses it in number of presented frames. The > reason for this difference is that an application may not be able to know > when a frame is not decoded it only sees when it is output. > That said, this delay is not at all related to the seeking feature. It > tells you how many frames have to be decoded (or output) to guarantee > smooth playback if the decoder decodes at the decode rate of the level > indicated in the bitstream. It may be useful for low latency applications. > That confirms what I have been thinking all along. But nevertheless I have some questions (My interest stems from the fact that I am in favour of including a field indicating how many frames need to be decoded in advance): 1. What exactly is the "display model verification algorithm" that the AV1-ISOBMFF speaks about? Does this just mean that the bitstream so constructed should be tested whether it fulfills the requirements of annex E of the AV1 specification? 2. If yes, then how can one test this precisely? a) There are two operating modes in annex E.3: Resource availability mode and decoding schedule mode. The latter mode needs buffer_removal_time values for the check; but these values SHOULD have been removed from the bitstream when muxing into ISOBMFF so this can in general not be checked with the ISOBMFF file alone. Is it intended that the muxer checks this during muxing when the original file with the original values are still available or not? Or do you just test this in resource availability mode. For the resource availability mode equal_picture_interval equal should have been set to 1 (i.e. crf video), but E.3.1 also says "If the parameters listed above are not specified by the bitstream, the parameters necessary to input into this model can be signaled by the application or some other means." I have not found that this crf requirement is in any way necessary. Are you simply using the timestamps derived from the ISOBMFF file (even if vfr) and use them to check in resource availability mode? b) If one has verified that one can use a value of k for initial_presentation_delay_minus_one, is this value also automatically applicable for starting the presentation at any RAP (or at least any keyframe/non-delayed RAP because -- after all, discarding the samples in front of a keyframe RAP should give a conformant bitstream)? The answer is of course tautologically true if "display model verification algorithm" includes checking presentation from every RAP onwards, but the current spec proposal talks only about constructing a (singular) hypothetical bitstream consisting of all the OBUs in the sample entry followed by all OBUs in all the samples. The reason I am asking this are bitstreams like this one (everything inside square brackets forms a temporal unit; &x means that a showable frame x is output via the show_existing_frames mechanism (if I am not mistaken, then it is allowed to output frames more than once -- and it might even make sense to do this at the beginning if there are several static frames at the beginning which sometimes happens with logos/black frames)): [A] [B] [&B] [C] [D E] [&D] A is a keyframe RAP, B an ordinary frame that is allowed to be output more than once. And it is output more than once. C is another non-delayed keyframe. One can start presentation as soon as one has A decoded (assuming the decoder can decode one frame during the presentation time of one frame): When one reaches the temporal unit [B], one has already decoded B. One can decode C while B is shown. When one reaches [&B], one has of course B still available. When one shows B a second time, one can decode D. This effectively means that when one starts at the very beginning, one enters the second CVS with two frames already decoded (as if one started decoding at C with initial_display_delay_minus_1 set to 1) and therefore one can use temporal units like [D E] with more than one frame to decode. In the above bitstream, one could use initial_presentation_delay_minus_one equal to zero for presentation from the beginning, but for decoding from C onwards one would need a value of 1. 3. a) The ISOBMFF specs contain the clause: "set the first initial_display_delay_minus_1 field of each Sequence Header OBU to the number of frames contained in the first initial_presentation_delay_minus_one + 1 samples," Is this really intended? Shouldn't it read: "set the first initial_display_delay_minus_1 field of each Sequence Header OBU to the number of frames - 1 contained in the first initial_presentation_delay_minus_one + 1 samples," After all initial_display_delay_minus_1 includes a minus 1, too; without the minus 1 one would have to decode the first frame of the sample number initial_presentation_delay_minus_one + 1 (zero-based), too, before the presentation can start. b) If one has a compliant AV1 bitstream where the Sequence Headers all include the initial_display_delay_minus_1 for every operating point, then one can use the maximum of all these initial_display_delay_minus_1 as initial_presentation_delay_minus_one if my above corrections turns out to be right and is accepted? (If it's not accepted, then one could even use the first the maximum - 1, couldn't one?) - Andreas Rheinhardt From nobody Thu Jul 19 03:21:32 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A03B412F1AB for ; Thu, 19 Jul 2018 03:21:29 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, T_DKIMWL_WL_MED=-0.01] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id p1oOwW5gxTM5 for ; Thu, 19 Jul 2018 03:21:27 -0700 (PDT) Received: from mail-pf0-x22d.google.com (mail-pf0-x22d.google.com [IPv6:2607:f8b0:400e:c00::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id AE230126BED for ; Thu, 19 Jul 2018 03:21:27 -0700 (PDT) Received: by mail-pf0-x22d.google.com with SMTP id d14-v6so3704243pfo.3 for ; Thu, 19 Jul 2018 03:21:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=WuBDSZzRoZRH5bjsCspBj6MilSA4dWFdTJwezGzX5TE=; b=PJYE1OzL4JF2dHt7opSeRGB5k+QIF15qllLc/8bfKwZEsmB8VN1zAj3oZ/YXRXzPLW WLTOsi+ubtCPOr3ZLrmE6h2QMdsIeFHkVi6n1wymHoTyCoP91jNQ2qAIp1I/vv9QBij1 5QQRtYMeQ1LZ2eE13NRZlfUqxpkynZBf4iXKXI27QKGqDnkXxvINU8827muVqyxGzLHN g5vZY6Y0mRHzZo0NG5ME50gZ9OefVPLIAilTf9wmZwMqUzZYNQgjfPEhVxJXSoD+Y++3 ysBvsTMpl4kHzOJeHWP0xrmONPrFq1MUvgXPaI7ak9wvoDn0OF/ere0slHPGs2IAHUiO PJLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=WuBDSZzRoZRH5bjsCspBj6MilSA4dWFdTJwezGzX5TE=; b=sR7tGHx8jbiKGARkEzjqW5RPImPixYydwo56mpcse8GJ4qehcOqmsPc7JP8hBUAuwD C76ekACWW5goaj6acuiuQre1jdmof3D05VgbjaOCjHWEfZyIe6J2Y2qzRj6GDXHkb2cA wyGEO+OrDk4+FBV/fWJPLE+r7hNKxdwrufinWLBlNDGQysDRcC/Rxy7/xQMHICR71tTz Mihldrw2AHzy3Uq6toRo6vZFRks8O2NMAOCKOJWIG18jTBHHKYjJpnLU3rmXGNC12B/4 +y8tM4NweCqREUemA60vOfaX1jFNVRPuGiWNf9PKZUq7bSmS33zdg8cGaX8SCD8A7vWM m4aA== X-Gm-Message-State: AOUpUlHhzLUnkvX0oTFJpax3EMO6qoYxyGkH9mxM4DP9QqPKiXmNiNr0 EiPENIxB0jKb4EcJjYhWNCWJvZ0ZedIZN+O6lSWCaFxk6a8= X-Google-Smtp-Source: AAOMgpf2jadXLkuohooYJH+R9IcYJH2m7aI1c9XCD4qsFWKzKO349s2MdhN4xj5m/ugcKsIuIqDWsm/9Upc3DrTsMnY= X-Received: by 2002:a65:5245:: with SMTP id q5-v6mr2004690pgp.67.1531995687053; Thu, 19 Jul 2018 03:21:27 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a17:90a:9c13:0:0:0:0 with HTTP; Thu, 19 Jul 2018 03:21:26 -0700 (PDT) In-Reply-To: References: From: Steve Lhomme Date: Thu, 19 Jul 2018 12:21:26 +0200 Message-ID: To: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cellar] AV1 init X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jul 2018 10:21:30 -0000 2018-07-19 6:29 GMT+02:00 Steve Lhomme : > IMO it should always be there. Because as stated above, if not people > will likely not write it. And players will have to handle the case > where it's not there. > > And the reason why it should be there is for playback reasons, so that > the right decoder can be selected before having to parse the data. > There's a reason H264/H265 have profiles, it's so that you can tell if > you can decode something or not and how. In many cases you get > hardware decoding and in other cases not. This will likely be the same > for AV1, for example there might never be hardware decoders doing 12 > bits, that doesn't mean hardware decoding should never be used. Reading the libaom code it seems that it does not support profile 2 https://aomedia.googlesource.com/aom/+/master/av1/decoder/obu.c#205 That's also some information that would be good to have before starting using that decoder. -- Steve Lhomme Matroska association Chairman From nobody Thu Jul 19 06:04:15 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8C513130DD8 for ; Thu, 19 Jul 2018 06:04:13 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=googlemail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xKJMI5srrsui for ; Thu, 19 Jul 2018 06:04:11 -0700 (PDT) Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1545F1277C8 for ; Thu, 19 Jul 2018 06:04:11 -0700 (PDT) Received: by mail-wr1-x429.google.com with SMTP id h10-v6so7993724wre.6 for ; Thu, 19 Jul 2018 06:04:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20161025; h=subject:to:references:from:message-id:date:mime-version:in-reply-to :content-transfer-encoding; bh=AS5O/Y+MA0B3ufR2D7VAC554aeEPVcEflso4c2wRWso=; b=OCubJ331g0DE+KDPyJldocR1eCZFp+iJ0LkxONwlX3F2STMqDVvKkBLMrJiEW+8/Pe +2xh3Pzl2ujI3vSOqsRcSYy+C9TqqRtGJxRiWhhCvOW8O6nDgja6sUA+mD7wBVsZwQjP +ND5E96BeZ+7I5/1ZGWLcw1bq9j09Tq7b5ZfNSF/zIosZPCG/g/UmfpQe/JJNcd4C+4g v8qhNiaC0ngx97ID3AH8q1zHqHz6qX4srrS/UGzhso8jeu0yWMDzRjm2IJ/rK5qkSThC MS+U4ryr8XxTzlycE15bgMIVuAdpbbe6ODk75b0FTDKQmEpto1K26guHAG3IlMjl07uW kwUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :mime-version:in-reply-to:content-transfer-encoding; bh=AS5O/Y+MA0B3ufR2D7VAC554aeEPVcEflso4c2wRWso=; b=n5K+aGFX6rQdx7tK+AHN9gFlWFqfpkqEjuTCzaV5LsC5FubO4SIy61wgjC6kPMYftF qMJwUM1FoRWQVmgGIsgg4mCDmDXLgN076pYfG9irMOVb5QalVw//RC9ZWX5cXHr9bz9p tet6O9Cut43mtiKsoaHJ1Vr4ujReHvdE2HdwLD7jct/dtZ2WQ4kst/MNVTfwwI3Iz850 mnxAL8XOlpefuh0z53qT3p3kqTrr41Or1hE7NAksQ14p7pP+ubIVX9WLIBkrPthFYiYC DFWxtVU7Icqf53XaFlU7KHkw+g9xSHyF2ufCCBQHBBT7pwInmB4mATMIxiHFDaw8OqNB L+2g== X-Gm-Message-State: AOUpUlGOPFvxgUTKgiS+j/8Iidsl0lDJusVaIDsK8Pcmc43psryhJzSj d13qB25MfLKqF/HyAQa4zk9h78OQbbg= X-Google-Smtp-Source: AAOMgpfRrouamCUEobdyJ80nkpnMQIey6z65/JZ8vM6szsf2lquXz5FjuLNi91st17RvYlRK3v8loA== X-Received: by 2002:a5d:428a:: with SMTP id k10-v6mr7414206wrq.225.1532005449315; Thu, 19 Jul 2018 06:04:09 -0700 (PDT) Received: from [127.0.0.1] ([2a0b:f4c1::4]) by smtp.googlemail.com with ESMTPSA id w9-v6sm8485352wrr.77.2018.07.19.06.04.08 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 19 Jul 2018 06:04:08 -0700 (PDT) To: cellar@ietf.org References: From: Andreas Rheinhardt Message-ID: <336a1f43-e223-dc57-78f7-ad1f88e7d488@googlemail.com> Date: Thu, 19 Jul 2018 13:03:00 +0000 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [Cellar] AV1 init X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jul 2018 13:04:14 -0000 Hello, I agree with everything you said regarding the availability of the necessary data for codec initialization in the `CodecPrivate`. But there are some other points: 1. The new proposal says that `DisplayHeight` and `DisplayWidth` are not mandatory. This is news to me as the website (outdated, I know) says that they are. That's why they can have default values. That the default values only apply if `DisplayUnit` is 0 (its default value) doesn't change this IMO. (Or is it possible to not signal an aspect ratio with using `DisplayUnit` 4?) 2. The current version still contains this: "The Block timestamp is translated from the [PresentationTime], including the [InitialPresentationDelay], which is then substracted by the CodecDelay. Audio/Video/Subtitle interleaving SHOULD take the CodecDelay in account so that Blocks with similar timestamps (adjusted with the CodecDelay) are close to each other in a Cluster." I think you want to discard this. 3. The current proposal does allow to have a `CodecPrivate` that consists of multiple OBUs none of which has [obu_has_size_field] set to 1. This is the same situation as for the blocks. Then the commit message read as if you only wanted to allow this (not having [obu_has_size_field] set to 1) for the very last OBU of a `Block`. If this is really what is wanted then the current wording doesn't convey the current intention and should be changed. I already made a proposal (in my email from July 14th) for this for the Block: "If an OBU is not the last OBU in a `Block`, it MUST follow the [Low Overhead Bitstream Format syntax] (i.e. it MUST have [obu_has_size_field] set to 1); the last OBU in a `Block` MAY have [obu_has_size_field] set to 0, in which case it is assumed to fill the remainder of the `Block`." Of course, if it is intended to allow [obu_has_size_field] set to zero everywhere, then the above proposal can be ignored. 4. Steve Lhomme: > As the CodecDelay seems to be unnecessary, we're getting close to > finalizing the AV1 mapping for Matroska and WebM. It would be > beneficial to everyone that Matroska and MP4 (ISOBMFF) share as much > as possible so it's easy to transmux from one to the other. > > The initial_presentation_delay_present we may not deal with at all or > we should have a specific Matroska element that can be used by all > codecs. So it doesn't really go into the AV1 codec mapping (more an > MKV/ISOBMFF compatibility document). > I think we really should deal with the initial delay; actually, knowing how many frames should be decoded before the first one can be output should be considered part of the initialization data. But if you want to create a new element for this (and not put it into the CodecPrivate), then I have to tell you that the way this delay is counted is codec-dependent: H.264 has max_num_reorder_frames and the important thing is that this does not translate 1-1 to `Blocks`, but that complementary field pairs are only counted as 1, whole frames are counted as 1 and unpaired fields are counted as 1 whereas complementary field pairs are stored as two `Blocks` in Matroska. So I don't think a good way to do is in a codec-agnostic manner exists. But given that adding this to existing codecs via a modification of the `CodecPrivate` is an absolute no-no because it breaks everything, not putting it here in the `CodecPrivate` seems to be the right choice. But the semantics of this new field would have to be codec-dependent, i.e. they would have to be part of every codec mapping for every codec that can benefit from such a field. Ideally, this new field would already be used in the first version of the AV1 codec mapping. 5. As you are saying that we are getting close: What about my proposal regarding when the `Sequence Header OBU` needs to be present in the bitstream (see my email from July 14th for this)? Should I prepare a PR for this? And what do you think about my proposal c) from my email from July the 16th regarding the introduction of a new child of `CueTrackPositions` to solve the seeking problem? - Andreas Rheinhardt From nobody Thu Jul 19 06:24:10 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E46FA130DD8 for ; Thu, 19 Jul 2018 06:24:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, T_DKIMWL_WL_MED=-0.01] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oEdYfkD9vLtM for ; Thu, 19 Jul 2018 06:24:06 -0700 (PDT) Received: from mail-pg1-x52f.google.com (mail-pg1-x52f.google.com [IPv6:2607:f8b0:4864:20::52f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 15989129619 for ; Thu, 19 Jul 2018 06:24:06 -0700 (PDT) Received: by mail-pg1-x52f.google.com with SMTP id n7-v6so3762358pgq.4 for ; Thu, 19 Jul 2018 06:24:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=Pd/oUxfoKXys1GLdRp3AaAXEOpyriTcx6l37wR6UDzk=; b=lAfUnaPuCr2jYC2MHCphgSh7VFp4qgq7GdUTpcSwkUo0x3hRLnKR15wBGShTLtJgxL 6rY2ouMtR0ZQkTgCCdvyar97wgDMOgBRWtJPD/rEO4VxFms1g+Pg8vY9MZ4BfrgIE22x AZ89gCo55cvRJ4oGQwCEeEq8e3EWRbhrwR16TKHXgUkaohK6TFEXUxrEIBzuSsi0bn5T 5GZllhFeWBV7G49ucR8fO87GqLAFDP9M4Z5l09RtykYlOf7sBZ4IxgDHrKbWa1gboADK j3sF9pIG0u37TE1OnbLyXlWaJnEYNXGLy3+RRZWM3hVjqqxIDDvS2PTSgopfpKdnzTtA FavQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=Pd/oUxfoKXys1GLdRp3AaAXEOpyriTcx6l37wR6UDzk=; b=Tm5FgbCTcHqxC4QM00woCKA57+O+lktWS4Jm5GWMMMZI29pY1alYRK6AfIHT+O6h1y 7MtECR4jp+CS7n0+0hFWLwvfCvuF1CXCuMnrOBNhvuPIfOzW/keHonL2PC+Qhxb6FtoL akyivKigppxXb4t9zY2U+O10F9td2UdfzyV5OMwOache9VxsRQbZM8qR4z7+iv0qaQfo 0jD5RddZYQdZ6eLBBd1nccduwNQg1pG5N6tj0951GuyXGfLP20Rgy2dnAzWNdCzHaTnw PyecUS8gutgo0DWkh29y5BN5TloiBA4MVK6Ftxr5kxs9EqGDSQMYvRvvKC+pM4Rmg5z+ X2ZA== X-Gm-Message-State: AOUpUlH48ZCtRjdVm5SJWkRmaWr1Y6ve/b9efFOQbCb8k4uyjC/uaqKM kwZTQ9I8XNNrzMT/g5RSDQQZKCqSjSjHz/gKLzchCWGX7Dk= X-Google-Smtp-Source: AAOMgpfj10iZHIIPSzvRo/2skw8sOWTtdyhVqsshPhBC1UG6q6owJMK2cRzAc8fUGRHbAgiieKRiDnW/qetEFs73Xos= X-Received: by 2002:a65:45cc:: with SMTP id m12-v6mr10028121pgr.160.1532006645375; Thu, 19 Jul 2018 06:24:05 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a17:90a:9c13:0:0:0:0 with HTTP; Thu, 19 Jul 2018 06:24:04 -0700 (PDT) In-Reply-To: <336a1f43-e223-dc57-78f7-ad1f88e7d488@googlemail.com> References: <336a1f43-e223-dc57-78f7-ad1f88e7d488@googlemail.com> From: Steve Lhomme Date: Thu, 19 Jul 2018 15:24:04 +0200 Message-ID: To: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cellar] AV1 init X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jul 2018 13:24:09 -0000 2018-07-19 15:03 GMT+02:00 Andreas Rheinhardt : > Hello, > > I agree with everything you said regarding the availability of the > necessary data for codec initialization in the `CodecPrivate`. But there > are some other points: > > 1. The new proposal says that `DisplayHeight` and `DisplayWidth` are not > mandatory. This is news to me as the website (outdated, I know) says > that they are. That's why they can have default values. That the default > values only apply if `DisplayUnit` is 0 (its default value) doesn't > change this IMO. (Or is it possible to not signal an aspect ratio with > using `DisplayUnit` 4?) If you are referring to this page, it doesn't say it's mandatory: https://matroska.org/technical/specs/index.html > 2. The current version still contains this: "The Block timestamp is > translated from the [PresentationTime], including the > [InitialPresentationDelay], which is then substracted by the CodecDelay. > Audio/Video/Subtitle interleaving SHOULD take the CodecDelay in account > so that Blocks with similar timestamps (adjusted with the CodecDelay) > are close to each other in a Cluster." > I think you want to discard this. Indeed, well spotted. > 3. The current proposal does allow to have a `CodecPrivate` that > consists of multiple OBUs none of which has [obu_has_size_field] set to > 1. This is the same situation as for the blocks. Then the commit message > read as if you only wanted to allow this (not having > [obu_has_size_field] set to 1) for the very last OBU of a `Block`. If > this is really what is wanted then the current wording doesn't convey > the current intention and should be changed. I already made a proposal > (in my email from July 14th) for this for the Block: > "If an OBU is not the last OBU in a `Block`, it MUST follow the [Low > Overhead Bitstream Format syntax] (i.e. it MUST have > [obu_has_size_field] set to 1); the last OBU in a `Block` MAY have > [obu_has_size_field] set to 0, in which case it is assumed to fill the > remainder of the `Block`." > Of course, if it is intended to allow [obu_has_size_field] set to zero > everywhere, then the above proposal can be ignored. This is an issue in ISOBMFF as well https://github.com/AOMediaCodec/av1-isobmff/issues/26 There it is in the sample (Block for us) but it's tricky to enforce this requirement on the size mode when the original bitstream was in a different format. That means the muxers has to be ready to rewrite OBUs and they won't even be restored to their original format on playback (since they should be prepended on keyframes). IMO any solution has its drawback. But I would lean towards keeping the original format (so the current wording is OK). The main goal of the Sequence Header OBU in the `CodecPrivate` is to find the profile and we don't need to know the size for that, it's at the beggining the OBU data (the size parsing still need to be handled but that's the case anyway if the SH_OBU is the only one). > 4. > > Steve Lhomme: >> As the CodecDelay seems to be unnecessary, we're getting close to >> finalizing the AV1 mapping for Matroska and WebM. It would be >> beneficial to everyone that Matroska and MP4 (ISOBMFF) share as much >> as possible so it's easy to transmux from one to the other. >> >> The initial_presentation_delay_present we may not deal with at all or >> we should have a specific Matroska element that can be used by all >> codecs. So it doesn't really go into the AV1 codec mapping (more an >> MKV/ISOBMFF compatibility document). >> I think we really should deal with the initial delay; actually, knowing > how many frames should be decoded before the first one can be output > should be considered part of the initialization data. But if you want to > create a new element for this (and not put it into the CodecPrivate), > then I have to tell you that the way this delay is counted is > codec-dependent: H.264 has max_num_reorder_frames and the important > thing is that this does not translate 1-1 to `Blocks`, but that > complementary field pairs are only counted as 1, whole frames are > counted as 1 and unpaired fields are counted as 1 whereas complementary > field pairs are stored as two `Blocks` in Matroska. So I don't think a > good way to do is in a codec-agnostic manner exists. But given that > adding this to existing codecs via a modification of the `CodecPrivate` > is an absolute no-no because it breaks everything, not putting it here > in the `CodecPrivate` seems to be the right choice. But the semantics of > this new field would have to be codec-dependent, i.e. they would have to > be part of every codec mapping for every codec that can benefit from > such a field. Ideally, this new field would already be used in the first > version of the AV1 codec mapping. I added a version number in case we need to explain things more or add more things. It would be OK to add a new element for cases were this would be needed. And the way it is done would be discussed if/when we introduce this element. Although we never needed it so far so I'm not sure there's some real benefit. As for the value it would be ok to count frames/fields rather than durations. > 5. As you are saying that we are getting close: What about my proposal > regarding when the `Sequence Header OBU` needs to be present in the > bitstream (see my email from July 14th for this)? Should I prepare a PR Yes, a PR is always good. Although I update the document many times a day so it's still a moving target. > for this? And what do you think about my proposal c) from my email from > July the 16th regarding the introduction of a new child of > `CueTrackPositions` to solve the seeking problem? IMO a new element for seeking is not an option for now, at least because we can't guarantee it will be implemented in WebM. > - Andreas Rheinhardt > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar -- Steve Lhomme Matroska association Chairman From nobody Thu Jul 19 06:25:36 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 77A57130E0F for ; Thu, 19 Jul 2018 06:25:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, T_DKIMWL_WL_MED=-0.01] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id j4FWah-sXPxJ for ; Thu, 19 Jul 2018 06:25:33 -0700 (PDT) Received: from mail-pg1-x530.google.com (mail-pg1-x530.google.com [IPv6:2607:f8b0:4864:20::530]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CC14A130E0D for ; Thu, 19 Jul 2018 06:25:32 -0700 (PDT) Received: by mail-pg1-x530.google.com with SMTP id e6-v6so3773771pgv.2 for ; Thu, 19 Jul 2018 06:25:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=lIdo0ANUExrLJxSEvFVxvPeEEBDA/TClm3ZHugohWNQ=; b=Z19kuZap1WXcAVl+9QR2hocPuUHKCVzzUF2nxwlmduuPEX3DtohJiBpoZUjQhcfgDM jHfJFAJy5lpTYMIfAb74B/ZR9eEZOuwZlA92XNEng2e2ZXnScRoNn5t3TiASzESgJcrG Pzd1QUARnNB8AnaSgYuxc5e1RAENV76aYO6CmclDOS5AnA9VUp/wI1KtTNZg8gVJYJji ECzbT9NemOIliz90TYAQYFhINOfpoqFaFGKdKIFmjHosxm8aIMNAaMNw/yepFqI2uizB r6+8Yq9B8Jd8EQRKyF0LIEknBI2u7J4XQsz21+zyg9X5pU314JY4PUM1zFqsv1Gke5tn vsCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=lIdo0ANUExrLJxSEvFVxvPeEEBDA/TClm3ZHugohWNQ=; b=PUhVkHL8sZek4FYtb6CK1wTw6dfQ1J7iw9lmPmtQgSkQqk2Xk6rCBkGAuurmrmAkqz oLIA2RjnsQ5SmXXRX0HeF1fz6PSuQpqz4VmBmgZPpXkR1GuFVopLslc0OtLzT5yQRcpi qSBWQUGjGOTXmU4rVTCjGec0NHgoOQTE5/ZsYeh73i7OmB3xLvIPz1chSArH8NPuO4mx 4QOW64BqCwKAzj5r5hHcrqvtklU5I5O15KbdgkX9Wlm1AzzIFtT621HMZ2pFdx+4kdRP JC52vtb6fyDjnI9aPqND041/b7MxE0QoP6f1XRIihWV2D5IkQD9OBK/BycmmSyI5r8cS 2wkg== X-Gm-Message-State: AOUpUlFIT+AC24rY6Ti8rhNlAJSc/KDFGyZrGQnmNjoj9SIWSHdtDZHW XDtDe8Gjby1TYLkofSmKAuapClHo5ay4Vlc7iXtlDbNOrp0= X-Google-Smtp-Source: AAOMgpe1BNzYFFVg01JjvHsbogY3BUFcQ+gLbx2CdL1Hl+9d+MVqIru40aKvljA9XwAZRRsnTp2yzGCzAI5rX6F2A+k= X-Received: by 2002:a65:5307:: with SMTP id m7-v6mr10224376pgq.431.1532006732310; Thu, 19 Jul 2018 06:25:32 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a17:90a:9c13:0:0:0:0 with HTTP; Thu, 19 Jul 2018 06:25:31 -0700 (PDT) In-Reply-To: References: <336a1f43-e223-dc57-78f7-ad1f88e7d488@googlemail.com> From: Steve Lhomme Date: Thu, 19 Jul 2018 15:25:31 +0200 Message-ID: To: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cellar] AV1 init X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jul 2018 13:25:35 -0000 BTW I added an issue on the ISOBMFF spec to ask for the Sequence Header OBU in the codec extradata be made mandatory: https://github.com/AOMediaCodec/av1-isobmff/issues/46 From nobody Thu Jul 19 07:10:45 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6758D131030 for ; Thu, 19 Jul 2018 07:10:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=googlemail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VDVfnjh7X4Ia for ; Thu, 19 Jul 2018 07:10:30 -0700 (PDT) Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [IPv6:2a00:1450:4864:20::52e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 700551310BE for ; Thu, 19 Jul 2018 07:10:30 -0700 (PDT) Received: by mail-ed1-x52e.google.com with SMTP id b10-v6so7195684edi.2 for ; Thu, 19 Jul 2018 07:10:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20161025; h=subject:to:references:from:message-id:date:mime-version:in-reply-to :content-transfer-encoding; bh=y7yY44rdsrSzFlmcU4LsGHI4LqC4JR6JY/G/SzGc01A=; b=dbx/YaYiHJMpzz3YroTd3NmBE97c+gN3TwpkMt7QQGJZg5H2SmsqxYybanZEQO+oUg G+D+OFmOuEBh/lpZdzYwTIMoPbxzcMjX4L0ApCmAaUY0CZV8ID8ygGrosDYa+yGchBsj i8J0/XTdXHHo86IPlAvWqDK7iXvUS14NUdQEZoge+t8TxQZs7fQfYnmVzmysbMnZid4D 7kHbOZJwJTLdqPZr32AgPo2SNBtpBCIs8RsTuMgm0BQNuMMu4RqFAMiNCVOHCqOV1A6e EeXIrtIq5BKxegvIx0qEbw3hOYBjSg2GXsa4YVqxBmQyEvHS0ny9ORqJNXnfuPd0J+QV Syiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :mime-version:in-reply-to:content-transfer-encoding; bh=y7yY44rdsrSzFlmcU4LsGHI4LqC4JR6JY/G/SzGc01A=; b=RdggjLle+PISi6WiSThKH1PDf3H87mkAQ4rPudB1Q4rS06cAKFKdzhzT7yWWyQo8SM Yut+YSOH4jBNLN9U8fZdqj6Cv/Rz1dD/58ebvMGjANpgnG0hKulDQUUi1uz0FcI/u4r+ /JEw0ByJqX5dy6yF6/1DF0aGhpCWO8GARV88r+6frNVzRwxqU5JteQDIAUhAFLww333N nXQJgp+dR5X3xCd6m+70NUqeqW6CwNpvT5lnroHKCFE9P2jkRSYg3j633+IAZklPKBZQ Jztztu6MmAGP5tUbJwi6/Ogp3ECtKHJD2DR5qWAdeb7NRZty704FEIjJqaLLM9qOyWM5 rC3w== X-Gm-Message-State: AOUpUlESVtMkFhHAmGth7GQPbcRty1OEu7zbp6WBWRGAaxIZxdHjvod7 PKfBFv/fkKDMCChhLneQkKyvreAu X-Google-Smtp-Source: AAOMgpdqPicdByAW4cWtOtsd5XYB2i9tjXRltC63nJRN/OSumaPNIYNb1Qs6BjYwwUSClrf7hACrYA== X-Received: by 2002:a50:adaa:: with SMTP id a39-v6mr11710003edd.194.1532009428629; Thu, 19 Jul 2018 07:10:28 -0700 (PDT) Received: from [127.0.0.1] ([2a00:1768:1001:21::32a3:201a]) by smtp.googlemail.com with ESMTPSA id o7-v6sm1863523edk.94.2018.07.19.07.10.27 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 19 Jul 2018 07:10:28 -0700 (PDT) To: cellar@ietf.org References: <336a1f43-e223-dc57-78f7-ad1f88e7d488@googlemail.com> From: Andreas Rheinhardt Message-ID: <301d46e9-262d-babc-09b6-da1c5bc49bf3@googlemail.com> Date: Thu, 19 Jul 2018 14:09:00 +0000 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [Cellar] AV1 init X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jul 2018 14:10:43 -0000 Hello, Steve Lhomme: > 2018-07-19 15:03 GMT+02:00 Andreas Rheinhardt > : >> Hello, >> >> I agree with everything you said regarding the availability of the >> necessary data for codec initialization in the `CodecPrivate`. But there >> are some other points: >> >> 1. The new proposal says that `DisplayHeight` and `DisplayWidth` are not >> mandatory. This is news to me as the website (outdated, I know) says >> that they are. That's why they can have default values. That the default >> values only apply if `DisplayUnit` is 0 (its default value) doesn't >> change this IMO. (Or is it possible to not signal an aspect ratio with >> using `DisplayUnit` 4?) > > If you are referring to this page, it doesn't say it's mandatory: > https://matroska.org/technical/specs/index.html > true. I could have sworn it was, but apparently I was wrong.> >> 3. The current proposal does allow to have a `CodecPrivate` that >> consists of multiple OBUs none of which has [obu_has_size_field] set to >> 1. This is the same situation as for the blocks. Then the commit message >> read as if you only wanted to allow this (not having >> [obu_has_size_field] set to 1) for the very last OBU of a `Block`. If >> this is really what is wanted then the current wording doesn't convey >> the current intention and should be changed. I already made a proposal >> (in my email from July 14th) for this for the Block: >> "If an OBU is not the last OBU in a `Block`, it MUST follow the [Low >> Overhead Bitstream Format syntax] (i.e. it MUST have >> [obu_has_size_field] set to 1); the last OBU in a `Block` MAY have >> [obu_has_size_field] set to 0, in which case it is assumed to fill the >> remainder of the `Block`." >> Of course, if it is intended to allow [obu_has_size_field] set to zero >> everywhere, then the above proposal can be ignored. > > This is an issue in ISOBMFF as well > https://github.com/AOMediaCodec/av1-isobmff/issues/26 > > There it is in the sample (Block for us) but it's tricky to enforce > this requirement on the size mode when the original bitstream was in a > different format. That means the muxers has to be ready to rewrite > OBUs and they won't even be restored to their original format on > playback (since they should be prepended on keyframes). > > IMO any solution has its drawback. But I would lean towards keeping > the original format (so the current wording is OK). The main goal of > the Sequence Header OBU in the `CodecPrivate` is to find the profile > and we don't need to know the size for that, it's at the beggining the > OBU data (the size parsing still need to be handled but that's the > case anyway if the SH_OBU is the only one). > > Ok. Then that is settled as well. >> 5. As you are saying that we are getting close: What about my proposal >> regarding when the `Sequence Header OBU` needs to be present in the >> bitstream (see my email from July 14th for this)? Should I prepare a PR > > Yes, a PR is always good. Although I update the document many times a > day so it's still a moving target. > Ok, I will try to make it this evening. >> for this? And what do you think about my proposal c) from my email from >> July the 16th regarding the introduction of a new child of >> `CueTrackPositions` to solve the seeking problem? > > IMO a new element for seeking is not an option for now, at least > because we can't guarantee it will be implemented in WebM. > I don't think that a satisfactory solution exists without new elements. And given that Webm is a member of the Alliance for Open Media I don't think that they will resist if we add a new element which is very benificial for AV1. Who is actually in charge of the Webm bitstream format? Maybe we should simply informally ask him. Maybe he is even a member of this mailing list. - Andreas Rheinhardt From nobody Fri Jul 20 00:48:31 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E93A413107E for ; Fri, 20 Jul 2018 00:48:29 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.909 X-Spam-Level: X-Spam-Status: No, score=-1.909 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, T_DKIMWL_WL_MED=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kWXrcRPv6bZj for ; Fri, 20 Jul 2018 00:48:28 -0700 (PDT) Received: from mail-pl0-x22b.google.com (mail-pl0-x22b.google.com [IPv6:2607:f8b0:400e:c01::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 51B6113107D for ; Fri, 20 Jul 2018 00:48:28 -0700 (PDT) Received: by mail-pl0-x22b.google.com with SMTP id o7-v6so4828936plk.10 for ; Fri, 20 Jul 2018 00:48:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=h3QidATvfgbtm3jS6oyYRORu2ydez/ItES+CaPplQcw=; b=N6atFcY4Qc68zpBL6weEtSFhwbdJZzeArsh8aSbX/7or94c/ELTzPJzGm8+/FCL5Mb QX8kxzT+pO4f0c2gXBVrGrQLQZpXC2GILea5/7K2/X+YGnymZYers3pdNCv0xNWYvDZm V2t6BWE/rBa9DOKlrDtHCyYIi+9W5caHRdoXspM362jskkJRbDEeXVAOAL1cFDS6VEoy NmfXICQTais1GLDwZ238IgRsHxWS3jAbSVJNkFC7IZcvpZOAzIS133wTOSMS5zrirKdb fstxCx8prxIkrAV46x/Dr0zv+jMr2xU9EsrRjjMxqLi/gLrWJ71R7GHEZdK7qQg8Rwig n3+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=h3QidATvfgbtm3jS6oyYRORu2ydez/ItES+CaPplQcw=; b=rFfKTsik/IPRTCBCWqczKl2I4F8lV55jQiH5It8alXV6O3rg/fYC6qB8z+wlZLJe8j FBqE6ehLRyDiwyafMgjqdj34JCQIRHvKUeuZiySjiB9WODclmZmlE1H0FwMVgvzfMmuS i1j/eEJCHtQ01Ueze598Kq3LtCQOmPEThdc7Zx2T4K266Czth0pDcC6iWtPy1fsJhIyi SznUauVPFKIiU658aq5Jb40ejYCNrk93aoxjjaFZ/bT1hWNuiMbPb44Xy7UFAAdcNo09 PTPe4xW0dVavNPYOv1TfzHpmYlXaCJNEHL/cyEyJeM2f5r7FXqC0yn5okRFA04xxOJUd sTcg== X-Gm-Message-State: AOUpUlH4y9o/48p6VsRzAYyNKMaQVdY3C9cWc4WSZAcdUs0ybH5qllyg LQzWl8XqMJRFUPqbZE6bGEyvZ9hyhjNiyrWoMiKTnq0g X-Google-Smtp-Source: AAOMgpdZtZkgIKW8fclacAFNIqi89imdfH5piKA69g6wX+HSfYXq0eITfFDgveRJot4avBTxUHe3E7fX6vBok0+HMK4= X-Received: by 2002:a17:902:7793:: with SMTP id o19-v6mr1083760pll.306.1532072907729; Fri, 20 Jul 2018 00:48:27 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a17:90a:9c13:0:0:0:0 with HTTP; Fri, 20 Jul 2018 00:48:27 -0700 (PDT) In-Reply-To: <301d46e9-262d-babc-09b6-da1c5bc49bf3@googlemail.com> References: <336a1f43-e223-dc57-78f7-ad1f88e7d488@googlemail.com> <301d46e9-262d-babc-09b6-da1c5bc49bf3@googlemail.com> From: Steve Lhomme Date: Fri, 20 Jul 2018 09:48:27 +0200 Message-ID: To: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cellar] AV1 init X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jul 2018 07:48:30 -0000 2018-07-19 16:09 GMT+02:00 Andreas Rheinhardt : >>> for this? And what do you think about my proposal c) from my email from >>> July the 16th regarding the introduction of a new child of >>> `CueTrackPositions` to solve the seeking problem? >> >> IMO a new element for seeking is not an option for now, at least >> because we can't guarantee it will be implemented in WebM. >> > I don't think that a satisfactory solution exists without new elements. > And given that Webm is a member of the Alliance for Open Media I don't > think that they will resist if we add a new element which is very > benificial for AV1. Who is actually in charge of the Webm bitstream > format? Maybe we should simply informally ask him. Maybe he is even a > member of this mailing list. I think Frank Galligan is the one to contact. -- Steve Lhomme Matroska association Chairman From nobody Fri Jul 20 16:09:03 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1906D130E21 for ; Fri, 20 Jul 2018 16:09:02 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=googlemail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nB_Gpm-_AvS2 for ; Fri, 20 Jul 2018 16:09:00 -0700 (PDT) Received: from mail-wm0-x230.google.com (mail-wm0-x230.google.com [IPv6:2a00:1450:400c:c09::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B4514127148 for ; Fri, 20 Jul 2018 16:08:59 -0700 (PDT) Received: by mail-wm0-x230.google.com with SMTP id o11-v6so10773042wmh.2 for ; Fri, 20 Jul 2018 16:08:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20161025; h=subject:to:references:from:message-id:date:mime-version:in-reply-to :content-transfer-encoding; bh=zszAylZA0S/r3EHjNuUJraIQpvoMkQWjanHZth6E7yQ=; b=RtSIEDlOcJl7Gdd4K45ty9KQCdfJps8nqW8YD6wdwEZ0H5hVWaZxLBH0MRtWAQDFIb oc5G98JyeLt0Oht0QzbnxIj2KP7zC27sEuJcSauCt3Rb/OVEOCKi+JfJ7RCg9RGwZwZH dmVBnlBcbR+fYobGOiQq+jzqA0GS3cn+MjuNo9RNrh0Lnx3zdblzHDt2LrYKrEDgikTd JhOyYPm8OpkRlOgIMsoqn85l9yvAY7wpselV/JCZtuxYUDJQNEUw93kTl0OfPD7riUvJ dUHkJQ4A8dWIPtC4ie+WMT+o/oNIl/VOzImGYwOiskS/Q8Sm7BCBhtfzj4r33zWGCfpw yjcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :mime-version:in-reply-to:content-transfer-encoding; bh=zszAylZA0S/r3EHjNuUJraIQpvoMkQWjanHZth6E7yQ=; b=btkt5bbNoJ0MfRnWFoR3dQWa6nE8DvW6l1TD3lnBelDxjzYYXP93mdOuoqPZZtnqYo eUDCUmzRVX3OfZ1D2dN+R+kM8DQ3KW8poZT7d5Xjc9++gobFo8LlTFrHhtvkYDqNjvow LSmbEIb3e5LaSshKRAEc+6bgsjoobm5/wilQ0ygjeAqwcXdCNERW6lfOzgLF65GUjeiL OUFThMR69wqpS57pnyougLw6BqPC1HkvQPPQX/I8rwwNDyqxZ6vSTB9DvTGQE60R89/X Jz4rjxVM8FRulvtPOpnMgQ8Yn89Oscg8PL9bf9EkOXqwJ/TfVqF1//ILdic9t8mdt92m Tejw== X-Gm-Message-State: AOUpUlHHPBDAtrV91ywmTaHNJDFknc8F3m2EDTnEledEvBI8zEv+uAnV /sOxYsEjsF65UHyqAxxStRqQ/Xw2n08= X-Google-Smtp-Source: AAOMgpcnX/X5KT9RBWxo9PnJgYwuImCUi2ZCEOsLAO1NBfhcLx4IOo014MpWuEVEtxVJwbdOdB608w== X-Received: by 2002:a1c:2352:: with SMTP id j79-v6mr2567057wmj.124.1532128138066; Fri, 20 Jul 2018 16:08:58 -0700 (PDT) Received: from [127.0.0.1] ([141.255.162.34]) by smtp.googlemail.com with ESMTPSA id u25-v6sm4637605wrd.61.2018.07.20.16.08.56 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 20 Jul 2018 16:08:57 -0700 (PDT) To: cellar@ietf.org References: <336a1f43-e223-dc57-78f7-ad1f88e7d488@googlemail.com> From: Andreas Rheinhardt Message-ID: Date: Fri, 20 Jul 2018 23:07:00 +0000 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [Cellar] AV1 init X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jul 2018 23:09:02 -0000 Hello, Steve Lhomme: > Yes, a PR is always good. Although I update the document many times a > day so it's still a moving target. Done: https://github.com/Matroska-Org/matroska-specification/pull/247 I also converted the clause "The parameters that should not change for a video Track are the dimensions..." to "Furthermore the dimensions of all output frames MUST be equal", although I am not sure if we should make this hard requirement or not. But I did not want to ambush you with such a change buried in a big PR. - Andreas Rheinhardt From nobody Mon Jul 23 03:25:18 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8B7C2130E17 for ; Mon, 23 Jul 2018 03:25:17 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iW2pGC7LjKZu for ; Mon, 23 Jul 2018 03:25:15 -0700 (PDT) Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 350A0130E29 for ; Mon, 23 Jul 2018 03:25:15 -0700 (PDT) Received: by mail-wr1-x433.google.com with SMTP id a3-v6so155421wrt.2 for ; Mon, 23 Jul 2018 03:25:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=XdxVfQbBOcZ+I8JWv6pZKwzorg1nON2ijbuSxOfOYyk=; b=d/aRI21BN3xRJP31Oxs0hHMZycggRLZDf3H78WvWxszsjYwf+oolfkcZ+tR37z7JdW NhbCYb2YXDLCRNOvcaw1Kzp2oWUFZvBXpSNTY3Zzy/UtyAfh/bL8hpyWuNr/yYekIrLC M+gnkJmFbBOBIznfRg/+O65jJtwjTeoQO7tGVxyz1iwNo5QWjJ5g105ZaAArhy5AQyn/ 5ZD5iIFGBAGxgcfPQrxk/9tZHbMwgQx3OIt0XdTlKvhPqP33CaeO0DHiduRmv5pkpM8O J0k1zoDb7+TMMPb/s4G4EAPWFHiOe/ylW4Iy3cItNQ0as+6MnJedeOnoXejkXAIt8tBE /PpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=XdxVfQbBOcZ+I8JWv6pZKwzorg1nON2ijbuSxOfOYyk=; b=VTcmI2LosJDD6T3A6KlRuf94VUxFsAtFsmq/CLc76EhPo4JZMKjrz9QSsLiT0bCk68 qOqDN1YDzPCvhujJ7SUMGPPbvWw6Gq3yp3Jv+1CTxH/FgEfiKt8MbaDZ20hAaelZD6Ga AyqsvDLOYaW91pj3GHpJ38TWCFD3h5H91LUIBvonfloYmLIirUlohvmec5Tq4BEmhEJ7 RVvUlYD4VP0+fZQm3KgC6ZpnskGWirVrtG3k1Lm2Nea8iCozZ0xMeZBPxyZwK4PhHyhi VjGC4uP0b30E6wgTJqGkDaL1NXiYoCwrD7SIixiiutpPFzUJeIemdnBI/2KyyqibmUGB 6fOw== X-Gm-Message-State: AOUpUlHTaWHazhsprvzLpF+uxIb6uiDkehJLM9VN5kH1YmtWx94j9Fcc aUwTdBG12/PxjjT0vbZ2ASG570X6JKKRVh+SbM6CwzA= X-Google-Smtp-Source: AAOMgpfjaCBO5pG3QpHYiXh+bqH8lMe0K+dQDgfifI5ym8/bMmVxzo/SPRsO4QtTl9LyoJtYWKilZNz5sEWJQkLV+Ts= X-Received: by 2002:adf:ec41:: with SMTP id w1-v6mr7913816wrn.128.1532341513311; Mon, 23 Jul 2018 03:25:13 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a7b:c098:0:0:0:0:0 with HTTP; Mon, 23 Jul 2018 03:25:12 -0700 (PDT) From: Kieran O Leary Date: Mon, 23 Jul 2018 11:25:12 +0100 Message-ID: To: cellar@ietf.org Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: [Cellar] Matroska schema Table of Contents numbering X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jul 2018 10:25:18 -0000 Hi all, I've been dipping in and out of the Matroska draft spec in the last week, and I had one suggestion regarding the table of contents. Currently, section 8.2 (Matroska Schema) has 243 child entries. I think that it would make the document easier to parse through if the numbering system for the ToC reflected the parent child relationships of the elements. For example: 8.2.112. BitsPerChannel Element has a path of "0*1(\Segment\Tracks\TrackEntry\Video\Colour\BitsPerChannel) So, perhaps this element could be numbered: 8.2.3. Segment 8.2.3.3. Segment\Tracks 8.2.3.3.1. Segment\Tracks\TrackEntry 8.2.3.3.1.9. Segment\Tracks\TrackEntry\Video 8.2.3.3.1.9.10 . Segment\Tracks\TrackEntry\Video\Color 8.2.3.3.1.9.10.2. Segment\Tracks\TrackEntry\Video\Color\BitsPerChannel While that ends up being a long index number, it would allow the table of contents to represent the parent/child relationship between elements. This might mess with automation or whatever, so I'm just curious to know if anyone else would find it useful? If this had to be done manually, I'd volunteer to do so. It would allow these elements to look a bit more like the Menu example: 14. Menu Specifications . . . . . . . . . . . . . . . . . . . 142 14.1. Introduction . . . . . . . . . . . . . . . . . . . . . 142 14.2. Requirements . . . . . . . . . . . . . . . . . . . 143 14.2.1. Highlights/Hotspots . . . . . . . . . . . .143 14.2.2. Playback features . . . . . . . . . . . . . 144 14.2.3. Player requirements . . . . . . . . . . . 144 14.3. Working Graph . . . . . . . . . . . . . . . . . 144 14.4. Ideas . . . . . . . . . . . . . . . . . . . . . . . . . 145 14.5. Data Structure . . . . . . . . . . . . . . . . . . 145 I found this approach useful when scanning the PREMIS data dictionary ( http://www.loc.gov/standards/premis/v3/premis-hierarchical-3-0.html) , which looks a bit like this: 1.5 objectCharacteristics (M, R) [File, Bitstream] 1.5.1 compositionLevel (O, NR) [File, Bitstream] 1.5.2 fixity (O, R) [File, Bitstream] 1.5.2.1 messageDigestAlgorithm (M, NR) [File, Bitstream] 1.5.2.2 messageDigest (M, NR) [File, Bitstream] 1.5.2.3 messageDigestOriginator (O, NR) [File, Bitstream] 1.5.3 size (O, NR) [File, Bitstream] 1.5.4 format (M, R) [File, Bitstream] 1.5.4.1 formatDesignation (O, NR) [File, Bitstream] 1.5.4.1.1 formatName (M, NR) [File, Bitstream] 1.5.4.1.2 formatVersion (O, NR) [File, Bitstream] 1.5.4.2 formatRegistry (O, NR) [File, Bitstream] 1.5.4.2.1 formatRegistryName (M, NR) [File, Bitstream] 1.5.4.2.2 formatRegistryKey (M, NR) [File, Bitstream] 1.5.4.2.3 formatRegistryRole (O, NR) [File, Bitstream] 1.5.4.3 formatNote (O, R) [File, Bitstream] Best, Kieran O'Leary, IFI Irish Film Archive. From nobody Mon Jul 23 05:23:34 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E4206130E6F for ; Mon, 23 Jul 2018 05:23:31 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1p0OtLps6Q2f for ; Mon, 23 Jul 2018 05:23:30 -0700 (PDT) Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5AA98130E64 for ; Mon, 23 Jul 2018 05:23:30 -0700 (PDT) Received: by mail-wr1-x431.google.com with SMTP id t6-v6so497152wrn.7 for ; Mon, 23 Jul 2018 05:23:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=UvbcsX6gL8RJFDJWyhzYe5qyUgNPJuT1NEiY3rKTcuM=; b=RNBs4Ey3yZ+YX93YpOXzlxHAOWuw4bdZxGrzHeRcG0/U4X1nKFt02D32bMCkTxgk7Z b26RLPffpdGksL7NhWHigMncHgzY+14X1Wf4aQX8vnNh/VU6FYbCndqZNI+zmgHOnVQe OvQ9WMsprvSHlWneJz8NgAo3Dx9zH53O6JNTNCZYs3nI8iGnMRqWkBK0mE8J4dnFsmaq JLTA0/eyxMDY7VWiG7oK1TmPHLJ+yj0Kk6wEqE6JxG2e3YSMVpaOYspvYJgfnIXbOJ4s rogprQLOs/w2Fe2RwyzRfXj9ntKdjbSmYVO6cMDoi/qvQDmNhJU1qCINA9WhkbxXHTHL vhxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=UvbcsX6gL8RJFDJWyhzYe5qyUgNPJuT1NEiY3rKTcuM=; b=Oxe4CxpHJVWs35L3NNlLL4d3IYBZGPvPS93CGvQnQ6zDAK4c4sVSxg/m6Jk6JYlA2h C6gYy/cA0CV7OLPapYcme5KVfa4uKhxpW+kg9GOiSd+VBDu3HoodNl6Ht6BVVhm8hFNO 10BDuim+xblP0IX1ZuQEbwXWkrHLhkCXmG0oOL8iTx2HTZCY4GAyL5BGreV3YEkUgMvM gYE3YeefvnTLS8NBw5A6xbGbNXS1BjzzMM5M3nu1qouNwTwpG/0alyC+Up4EVQ3UkK4B Z7gnZcuQHsXGOswOVfjEa1brjqvmT8JTVDZ7gwmNYXajbnJuKzl26dyiM5/AHhKcRc+u 2UWQ== X-Gm-Message-State: AOUpUlFH0gS1RA7R1Cenr5wLdYb07yoeifJSWgva8vP37HRZCv1V1aMa bScTRwsumAUzwM0aOpkRMGVmRNaHhnUtXpNSq7xc X-Google-Smtp-Source: AAOMgpejF3sqX+YddlBwDo7vCbmd2qzpIsRBucdDefeqqQ1rl8EDXrSHGgDEeZm7otZv56mkP7Ii7aXWEEMrhVC0TWg= X-Received: by 2002:adf:c5d3:: with SMTP id v19-v6mr8512188wrg.169.1532348608731; Mon, 23 Jul 2018 05:23:28 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Kieran O Leary Date: Mon, 23 Jul 2018 13:23:16 +0100 Message-ID: To: cellar@ietf.org Content-Type: multipart/alternative; boundary="0000000000007edec70571a9b652" Archived-At: Subject: Re: [Cellar] Matroska schema Table of Contents numbering X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jul 2018 12:23:32 -0000 --0000000000007edec70571a9b652 Content-Type: text/plain; charset="UTF-8" Ok,so it looks like the XML transformation would prevent this from occurring. --0000000000007edec70571a9b652 Content-Type: text/html; charset="UTF-8"
Ok,so it looks like the XML transformation would prevent this from occurring.
--0000000000007edec70571a9b652-- From nobody Mon Jul 23 06:10:16 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 96618130ECB for ; Mon, 23 Jul 2018 06:10:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.121 X-Spam-Level: X-Spam-Status: No, score=-1.121 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id h_rq--JmYSGk for ; Mon, 23 Jul 2018 06:10:02 -0700 (PDT) Received: from server172-3.web-hosting.com (server172-3.web-hosting.com [68.65.122.111]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id ADB34130F09 for ; Mon, 23 Jul 2018 06:10:02 -0700 (PDT) Received: from [146.96.19.240] (port=51419 helo=[10.10.201.39]) by server172.web-hosting.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.91) (envelope-from ) id 1fhab5-003pfv-QH; Mon, 23 Jul 2018 09:10:02 -0400 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) From: Dave Rice In-Reply-To: Date: Mon, 23 Jul 2018 09:09:54 -0400 Cc: cellar@ietf.org Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Kieran O Leary X-Mailer: Apple Mail (2.3273) X-OutGoing-Spam-Status: No, score=-2.9 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Subject: Re: [Cellar] Matroska schema Table of Contents numbering X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jul 2018 13:10:14 -0000 Hi, When I was working on the xml transformation I did at one point have it = nesting of the sections adjust according to the depth of the associated = element. Basically I just had a counter add an extra =E2=80=98#=E2=80=99 = character in the section header for every level of depth for that = element. However the output rfc document then had a table of contents = that only went down to a certain depth maybe 4 or 5. In Matroska some = elements goes much deeper so have a nested element table of contents = with our current document build system, resulting in some elements being = hidden from the toc. I=E2=80=99m unclear if there is a depth limit for = RFC TOC=E2=80=99s or if it=E2=80=99s a limitation of mmark or rfc2xml, = but if it=E2=80=99s the latter and there=E2=80=99s interest in having = the TOC as Kieran suggests then we could retry it. Dave Rice > On Jul 23, 2018, at 8:23 AM, Kieran O Leary = wrote: >=20 > Ok,so it looks like the XML transformation would prevent this from = occurring. > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar From nobody Mon Jul 23 07:23:13 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 05DDE130EA1 for ; Mon, 23 Jul 2018 07:23:12 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.201 X-Spam-Level: X-Spam-Status: No, score=-4.201 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sHswt4OWAB2z for ; Mon, 23 Jul 2018 07:23:10 -0700 (PDT) Received: from tuna.sandelman.ca (tuna.sandelman.ca [IPv6:2607:f0b0:f:3:216:3eff:fe7c:d1f3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CBCD8130EA0 for ; Mon, 23 Jul 2018 07:23:09 -0700 (PDT) Received: from sandelman.ca (obiwan.sandelman.ca [209.87.249.21]) by tuna.sandelman.ca (Postfix) with ESMTP id 694512008F; Mon, 23 Jul 2018 10:39:13 -0400 (EDT) Received: by sandelman.ca (Postfix, from userid 179) id 49E4B15B9; Mon, 23 Jul 2018 10:23:07 -0400 (EDT) Received: from sandelman.ca (localhost [127.0.0.1]) by sandelman.ca (Postfix) with ESMTP id 450DE15B0; Mon, 23 Jul 2018 10:23:07 -0400 (EDT) From: Michael Richardson To: Dave Rice cc: Kieran O Leary , cellar@ietf.org In-Reply-To: References: X-Mailer: MH-E 8.6; nmh 1.7+dev; GNU Emacs 24.5.1 X-Face: $\n1pF)h^`}$H>Hk{L"x@)JS7<%Az}5RyS@k9X%29-lHB$Ti.V>2bi.~ehC0; <'$9xN5Ub# z!G,p`nR&p7Fz@^UXIn156S8.~^@MJ*mMsD7=QFeq%AL4m Archived-At: Subject: Re: [Cellar] Matroska schema Table of Contents numbering X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jul 2018 14:23:12 -0000 --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Dave Rice wrote: > When I was working on the xml transformation I did at one point have = it > nesting of the sections adjust according to the depth of the associat= ed > element. Basically I just had a counter add an extra =E2=80=98#=E2=80= =99 character in > the section header for every level of depth for that element. However > the output rfc document then had a table of contents that only went > down to a certain depth maybe 4 or 5. In Matroska some elements goes > much deeper so have a nested element table of contents with our curre= nt > document build system, resulting in some elements being hidden from t= he > toc. I=E2=80=99m unclear if there is a depth limit for RFC TOC=E2=80= =99s or if it=E2=80=99s a > limitation of mmark or rfc2xml, but if it=E2=80=99s the latter and th= ere=E2=80=99s > interest in having the TOC as Kieran suggests then we could retry it. I think that the xml2rfc tool has a setting for depth of ToC produced. The RFC-Editor might mildly object, but it may be worth tweaking things in this case, if the WG would prefer a deeper ToC. If we want to go this way, then I'll query the RFC-editor for their blessing. =2D- Michael Richardson , Sandelman Software Works -=3D IPv6 IoT consulting =3D- --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBW1XkyICLcPvd0N1lAQLWqAgAoz7skbZgtOkkAh6oYdeRGjMa+unGa1OM zQqy3KL9Cypeqw2byxfe6/2+eS6oXJmmT7BA6nU7ydosshhX5jRfiK+NFmz+slmV bsX2J/8VZHk5/Vk/4ywpar4vhLF7mGpZRZlnFw+yM9ujPCtX7Fb3nGcPzfmMHmme 9OFNa6OypwZL0VN+PXRKymkuqSKGjF3hMVbjUgkJXvsGoveqUjSCNAT3AKzJ8c1b 5+CYpZicwQzyjms21PfqJY0Moe3rUZXBEn5NTL6ql2sdSvVLm2BrACC8SB68vP9u 9K3Iv/ngqseE7El9eRUOu0waECbgchoppEdLcuX7hKnTBMjPacIemg== =V6U3 -----END PGP SIGNATURE----- --=-=-=-- From nobody Mon Jul 23 07:35:52 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 22E83130EA1 for ; Mon, 23 Jul 2018 07:35:51 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.121 X-Spam-Level: X-Spam-Status: No, score=-1.121 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Z_vesu7NleGs for ; Mon, 23 Jul 2018 07:35:49 -0700 (PDT) Received: from server172-3.web-hosting.com (server172-3.web-hosting.com [68.65.122.111]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BE3D3130E95 for ; Mon, 23 Jul 2018 07:35:49 -0700 (PDT) Received: from [146.96.19.240] (port=40540 helo=[10.10.201.39]) by server172.web-hosting.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.91) (envelope-from ) id 1fhbwC-000DxP-9P; Mon, 23 Jul 2018 10:35:49 -0400 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) From: Dave Rice In-Reply-To: <24367.1532355787@localhost> Date: Mon, 23 Jul 2018 10:35:47 -0400 Cc: Kieran O Leary , cellar@ietf.org Content-Transfer-Encoding: quoted-printable Message-Id: <6915E7A9-3F8E-4B11-8626-2E1425F3E06C@dericed.com> References: <24367.1532355787@localhost> To: Michael Richardson X-Mailer: Apple Mail (2.3273) X-OutGoing-Spam-Status: No, score=-2.9 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Subject: Re: [Cellar] Matroska schema Table of Contents numbering X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jul 2018 14:35:51 -0000 > On Jul 23, 2018, at 10:23 AM, Michael Richardson = wrote: >=20 >=20 > Dave Rice wrote: >> When I was working on the xml transformation I did at one point have = it >> nesting of the sections adjust according to the depth of the = associated >> element. Basically I just had a counter add an extra =E2=80=98#=E2=80=99= character in >> the section header for every level of depth for that element. However >> the output rfc document then had a table of contents that only went >> down to a certain depth maybe 4 or 5. In Matroska some elements goes >> much deeper so have a nested element table of contents with our = current >> document build system, resulting in some elements being hidden from = the >> toc. I=E2=80=99m unclear if there is a depth limit for RFC TOC=E2=80=99= s or if it=E2=80=99s a >> limitation of mmark or rfc2xml, but if it=E2=80=99s the latter and = there=E2=80=99s >> interest in having the TOC as Kieran suggests then we could retry it. >=20 > I think that the xml2rfc tool has a setting for depth of ToC produced. >=20 > The RFC-Editor might mildly object, but it may be worth tweaking = things in > this case, if the WG would prefer a deeper ToC. If we want to go = this way, > then I'll query the RFC-editor for their blessing. Such to consider the sanity of it, we=E2=80=99d have a TOC that goes to = a depth of 9 subsections if each element was to be referenced in the = toc. Dave= From nobody Mon Jul 23 08:41:06 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E202B130EE9 for ; Mon, 23 Jul 2018 08:41:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, T_DKIMWL_WL_MED=-0.01] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HQfhmQ0WiRym for ; Mon, 23 Jul 2018 08:41:01 -0700 (PDT) Received: from mail-pg1-x531.google.com (mail-pg1-x531.google.com [IPv6:2607:f8b0:4864:20::531]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2FCC2130EDF for ; Mon, 23 Jul 2018 08:41:01 -0700 (PDT) Received: by mail-pg1-x531.google.com with SMTP id x5-v6so656270pgp.7 for ; Mon, 23 Jul 2018 08:41:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=0q1vucvFam6jRe6kYtt63FY0scgSQ4zOyC9Kr/4cEhg=; b=hyQ956FYOZQrbakjm8WUkEvFgIpXM9UU9o4oFDktZsiooXx8tCHBwY2yx0v47TXelA 2t9SHBS5MXRKqB6CZYj8CLh8vSo10j5jFibkYrV2fU4SWkUHl3p9ohq8mKHsB9MR0tM9 P8KVXcygLtTTb6WcofXDuzI3SDSFiYncXDTukLZi2HkVqiSLrgCOdA56xPiHUXHbqm1n hcMtI7H/PbCecXVuP+pLqkIY772xHuIC2KxCR8gqic0FqxEcrSsbRqAvT6faCkCPlUK0 MslLQ2Nk4He+EUW/YHuVAKtIooOADeduMmKUksy91mgMjzx7Up2XTlm/KASWq7wD04FN chNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=0q1vucvFam6jRe6kYtt63FY0scgSQ4zOyC9Kr/4cEhg=; b=a8TatuFOKBgPkXd1fO/EapEAYufD2i3b7rtPqFgapiX5ZCH7pMYeMndCzU/uRzE6eG 5SjTvDol9ApyQ1L78TQFtbrDHjIhY+rvnBUzh8+L02KIyXh/7l9p9Ff7jHclQrLk5HQS 8L4pXqWSCOoNTpNaciizI8nH9VXM25HmoeOjBieu6MdEPhEH8wGxGGAb2bo6lMFH8eNU GLgKyfsAE32I/aVLwohIX2RC5GwblyiOrto552coNg23axdB3CWMiI6AUHiOPpXJyxxn 7OKQA00KxB3eMfaSc/CsyXZwhABbt6WgeiTpw8y9xwY+6RK5hSc/hgVKLHDz/W0pQyuK x0ig== X-Gm-Message-State: AOUpUlH52CBeavl/4Y9bQnEKaRC5Rkk0/WmzHTkqxOVGQvAVax8EcnWc +uukXKGtEaNHNs8k3WwW7kp8VpcwxnSQjNIaVKXwfA== X-Google-Smtp-Source: AAOMgpdmt2bZckBlqFUe6oC6SzaXdtUdGhBfqmVPIJvqYzO6T5Zwm5yz6bjljwgNSL6IVfP0OArRRJdfbmXo7HqjdiE= X-Received: by 2002:a62:8d84:: with SMTP id p4-v6mr13952184pfk.251.1532360460759; Mon, 23 Jul 2018 08:41:00 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a17:90a:9c13:0:0:0:0 with HTTP; Mon, 23 Jul 2018 08:41:00 -0700 (PDT) In-Reply-To: <24367.1532355787@localhost> References: <24367.1532355787@localhost> From: Steve Lhomme Date: Mon, 23 Jul 2018 17:41:00 +0200 Message-ID: To: Michael Richardson Cc: Dave Rice , Codec Encoding for LossLess Archiving and Realtime transmission , Kieran O Leary Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Archived-At: Subject: Re: [Cellar] Matroska schema Table of Contents numbering X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jul 2018 15:41:05 -0000 It seems adding something like in the XML source would be enough (number 10 picked randomly). Source: https://xml2rfc.tools.ietf.org/authoring/README.html We have to be careful with global elements that don't actually have a parent. There's none in Matroska but some in EBML. 2018-07-23 16:23 GMT+02:00 Michael Richardson : > > Dave Rice wrote: > > When I was working on the xml transformation I did at one point hav= e it > > nesting of the sections adjust according to the depth of the associ= ated > > element. Basically I just had a counter add an extra =E2=80=98#=E2= =80=99 character in > > the section header for every level of depth for that element. Howev= er > > the output rfc document then had a table of contents that only went > > down to a certain depth maybe 4 or 5. In Matroska some elements goe= s > > much deeper so have a nested element table of contents with our cur= rent > > document build system, resulting in some elements being hidden from= the > > toc. I=E2=80=99m unclear if there is a depth limit for RFC TOC=E2= =80=99s or if it=E2=80=99s a > > limitation of mmark or rfc2xml, but if it=E2=80=99s the latter and = there=E2=80=99s > > interest in having the TOC as Kieran suggests then we could retry i= t. > > I think that the xml2rfc tool has a setting for depth of ToC produced. > > The RFC-Editor might mildly object, but it may be worth tweaking things i= n > this case, if the WG would prefer a deeper ToC. If we want to go this w= ay, > then I'll query the RFC-editor for their blessing. > > -- > Michael Richardson , Sandelman Software Works > -=3D IPv6 IoT consulting =3D- > > > > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > --=20 Steve Lhomme Matroska association Chairman From nobody Mon Jul 23 09:31:48 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A33A1130EE4 for ; Mon, 23 Jul 2018 09:31:46 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.201 X-Spam-Level: X-Spam-Status: No, score=-4.201 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id W5TXUG0hv_dP for ; Mon, 23 Jul 2018 09:31:44 -0700 (PDT) Received: from tuna.sandelman.ca (tuna.sandelman.ca [209.87.249.19]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id ABA0D130EF8 for ; Mon, 23 Jul 2018 09:31:44 -0700 (PDT) Received: from sandelman.ca (obiwan.sandelman.ca [IPv6:2607:f0b0:f:2::247]) by tuna.sandelman.ca (Postfix) with ESMTP id A2D7420008; Mon, 23 Jul 2018 12:47:49 -0400 (EDT) Received: by sandelman.ca (Postfix, from userid 179) id 41E1C1297; Mon, 23 Jul 2018 12:31:43 -0400 (EDT) Received: from sandelman.ca (localhost [127.0.0.1]) by sandelman.ca (Postfix) with ESMTP id 3EEBEFF4; Mon, 23 Jul 2018 12:31:43 -0400 (EDT) From: Michael Richardson To: Dave Rice cc: Kieran O Leary , cellar@ietf.org In-Reply-To: <6915E7A9-3F8E-4B11-8626-2E1425F3E06C@dericed.com> References: <24367.1532355787@localhost> <6915E7A9-3F8E-4B11-8626-2E1425F3E06C@dericed.com> X-Mailer: MH-E 8.6; nmh 1.7+dev; GNU Emacs 24.5.1 X-Face: $\n1pF)h^`}$H>Hk{L"x@)JS7<%Az}5RyS@k9X%29-lHB$Ti.V>2bi.~ehC0; <'$9xN5Ub# z!G,p`nR&p7Fz@^UXIn156S8.~^@MJ*mMsD7=QFeq%AL4m Archived-At: Subject: Re: [Cellar] Matroska schema Table of Contents numbering X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jul 2018 16:31:47 -0000 Dave Rice wrote: > Such to consider the sanity of it, we=E2=80=99d have a TOC that goes = to a depth > of 9 subsections if each element was to be referenced in the toc. Understood. If it organizes things better then great. If it makes sense to have a shallower ToC, omitting some detail in the ToC, then do that. I suggest trying it out. From nobody Tue Jul 24 02:30:38 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A862F130E40 for ; Tue, 24 Jul 2018 02:30:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1wwo3Tl28gMj for ; Tue, 24 Jul 2018 02:30:34 -0700 (PDT) Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1A9F812872C for ; Tue, 24 Jul 2018 02:30:34 -0700 (PDT) Received: by mail-wr1-x435.google.com with SMTP id j5-v6so3407464wrr.8 for ; Tue, 24 Jul 2018 02:30:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=PFT/sxlMCwx/7j0Cpnn5CQb6C88eCh2RYjZsgq0ISSQ=; b=OrcRIgnM/Wvqsq04oGXoq8NV3hOKtC4XrkDu3zmLCQcI3Oa/mT/F1wYC9Js7TLenDR j4ezd3tFcDtb8npcPzTNuI29M7Se1YNn79mgMjJ1JlIHHcuqFosnr5jPxZurEpd+3fbR OKoMFRDpuavnbgdBXgjDBKkXBar4T9mDDKLjMRPaKEiTmBVHLWJ31CwbFEbZ4PtST/QY Hvjkb6d21g+1zLbyBrbCEO7x8htfGg7v25/gtccyrjew9byFSWBLDKiHc9SwaJrOZv+e pS5yyWh5reVDLAFrtBEtGbrXUHv+2GAC7/iSeAvXHFHqdIxRkVUzyXMSyLmTP5sWMRZR POAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=PFT/sxlMCwx/7j0Cpnn5CQb6C88eCh2RYjZsgq0ISSQ=; b=CL6ILzOpeWU2LvFgk8VAksqRDdr5e9gg2klZ0MQC+UR5K7Pkrm+UkS4Xr8mf6R1XX2 bBCOsCk9vaGXVx6oKI2STHNdu+TnsOCm1r3KZcZ3S1m5bmmuOD08Zij/yqB+8gcWV6mR CO3j4iAu9Dj+eKcHV6Bif5mxmHdXamglq6p1H2cETQaTPFRnZVFv4DDXM5hqPxfgEfhh evtUygECJxlG9iH0Zma5fycSpLhknrZ98clDoNMqLHouZPrd+udCy1KFzEMX0Mgw53hG kD1Sp+3F6t9/bFGQlUX3FSNSbwm5itiqkgqnNOUnNYaoYMD1foTM5kFm0Xg9pOL04TY2 urUg== X-Gm-Message-State: AOUpUlGB1fn6YmITf9nhnC8buIt4988wr3jiRG5PjFvrORnHtWYvXOUA bj6MKpotDw4k9IzYVBn2YKSsEGbEjlDlSR6jqtYC67k= X-Google-Smtp-Source: AAOMgpdDG5Pryns4NvBcrALeJW8fWQ0bWZ9uFIj3bTzEhEf8xqxdEPYiDVizHVLssTjWoVFA2RcetYXBuAilchlMNWQ= X-Received: by 2002:adf:f60a:: with SMTP id t10-v6mr9993522wrp.170.1532424632316; Tue, 24 Jul 2018 02:30:32 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a7b:c098:0:0:0:0:0 with HTTP; Tue, 24 Jul 2018 02:30:31 -0700 (PDT) From: Kieran O Leary Date: Tue, 24 Jul 2018 10:30:31 +0100 Message-ID: To: cellar@ietf.org Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: [Cellar] Matroska element descriptions and defaults X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jul 2018 09:30:37 -0000 Is it within the scope of the IETF specification that the meaning of the integer values should be explained? I find that if I want to understand Matroska, I also have to refer to https://matroska.org/technical/specs/index.html. In this instance, I wanted to know what was the default value of FlagInterlaced, as this is the presumed value when the element is not present in a Matroska file. I was able to see from : https://tools.ietf.org/html/draft-ietf-cellar-matroska-00#section-8.2.92 tha the default is 0, and the range is 0-2, but I have to go to matroska.org in order to know what these mean. If this is something that needs to be manually added, I could have a go, but perhaps it's outside of the scope of the IETF spec work. Best, Kieran O'Leary IFI Irish Film Archive. From nobody Tue Jul 24 02:35:33 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 35830130E40 for ; Tue, 24 Jul 2018 02:35:31 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6JPJlpPx-3S7 for ; Tue, 24 Jul 2018 02:35:29 -0700 (PDT) Received: from mail-wm0-x22e.google.com (mail-wm0-x22e.google.com [IPv6:2a00:1450:400c:c09::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 072A213105D for ; Tue, 24 Jul 2018 02:35:29 -0700 (PDT) Received: by mail-wm0-x22e.google.com with SMTP id o11-v6so1822348wmh.2 for ; Tue, 24 Jul 2018 02:35:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=4cQ7RCC3sz+dKoNkhHviUH4xcj3iZK8qcqIpogOPBYY=; b=ge1hqsoWHTliyxvSsSNA3/TJ4p4ohAaofqgRiaW1mbhHRV5G4SSu/ChnC8xFa9KmG3 ob2RGQjXH9lPWatpKgE1UNc9DwqmDcuhYwNZOvGyKX0rwmJerUTr3Hr3v5NhYQeLOzML yTQGKqbWBtsYNNpCZNZzAb147lmoUpwpt/duLI8xzdrDFJeaohkuuyVTwqKlQSRnxzSc fnXgQoVN87OwFHBWMSYUiQLByloiot7rHS5vVOnjPtfzYm/Lau/jqgammbZ6eZMIpsKy sjkqILhDXQtcbvz2Iph191F97JAa9vq/u228Gj6GWc9C7QclXbnMfYdFqX2AFacjph3u jF6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=4cQ7RCC3sz+dKoNkhHviUH4xcj3iZK8qcqIpogOPBYY=; b=ZGnLug6D8DvBSxoBubPbzl8dK+zUhsFL6zwtvtfxQsuuWz4Ra3v234N3ZcD3JY9eH3 /hwlvLZsKyv6O8K5atPXxwKktqlPHKo8nvdIDWCIh0ji8N8S0d06T5+fwX9fERDsn8jk IzwmFsxWMwi53PlR9W+LiajL5T1uWSaXD1JVEd/PbjJUWFm8CksvHUPoKgA7ieru5Ump IgFWqES37b3tvAAgCyBMsuGnHQLlbiSRJEz+QTA5G21n6klIfzKf+71RkX+Gb2VO135N jvOf3CLRHy25I+Yp8JsuvftSmV7E4m93GtQoFTqmn+oQyHgCnrvYNcWCdzbVaUO/gZur 14AA== X-Gm-Message-State: AOUpUlGWWzxerdKvQNj0ia0B3y6tocS0uqRk8CZbueLm15oWQwlxpMIk NcoBsWQwDuV6YBcS1Wo+sDmK/rVF/8u+xrcTGs62F/4= X-Google-Smtp-Source: AAOMgpc68UJcy2q4VNnv270f2wrDsOx6SBJce/CYeQ0Ou3n9LNglHNcLk/3QDizDWKYD5sVfHWLVbcUCO48fMhGUeHI= X-Received: by 2002:a1c:d946:: with SMTP id q67-v6mr1461216wmg.156.1532424927274; Tue, 24 Jul 2018 02:35:27 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a7b:c098:0:0:0:0:0 with HTTP; Tue, 24 Jul 2018 02:35:26 -0700 (PDT) In-Reply-To: References: From: Kieran O Leary Date: Tue, 24 Jul 2018 10:35:26 +0100 Message-ID: To: cellar@ietf.org Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cellar] Matroska element descriptions and defaults X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jul 2018 09:35:31 -0000 P.S I see that this information is actually present in the XML file in the form of , but it isn't making its way into the IETF drafts? https://github.com/Matroska-Org/matroska-specification/blob/master/ebml_matroska.xml#L311 -K From nobody Tue Jul 24 08:47:55 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 61A20131151 for ; Tue, 24 Jul 2018 08:47:53 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.501 X-Spam-Level: X-Spam-Status: No, score=-5.501 tagged_above=-999 required=5 tests=[BAYES_05=-0.5, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ybw38vqNlRHq for ; Tue, 24 Jul 2018 08:47:50 -0700 (PDT) Received: from smtp.mozilla.org (mx2.scl3.mozilla.com [63.245.214.156]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id AEBF7130ED8 for ; Tue, 24 Jul 2018 08:47:50 -0700 (PDT) Received: from localhost (localhost6.localdomain [127.0.0.1]) by mx2.mail.scl3.mozilla.com (Postfix) with ESMTP id 6080BC057D for ; Tue, 24 Jul 2018 15:47:50 +0000 (UTC) X-Virus-Scanned: amavisd-new at mozilla.org Received: from smtp.mozilla.org ([127.0.0.1]) by localhost (mx2.mail.scl3.mozilla.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ozJaINXOBd-z for ; Tue, 24 Jul 2018 15:47:50 +0000 (UTC) Received: from [10.252.28.233] (unknown [63.245.221.198]) (Authenticated sender: tterriberry@mozilla.com) by mx2.mail.scl3.mozilla.com (Postfix) with ESMTPSA id 4BCA8BFFE7 for ; Tue, 24 Jul 2018 15:47:50 +0000 (UTC) To: cellar@ietf.org References: From: "Timothy B. Terriberry" Message-ID: Date: Tue, 24 Jul 2018 08:47:50 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 SeaMonkey/2.49.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [Cellar] Matroska element descriptions and defaults X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jul 2018 15:47:54 -0000 Kieran O Leary wrote: > If this is something that needs to be manually added, I could have a > go, but perhaps it's outside of the scope of the IETF spec work. This is completely in scope. It's okay to point to other specifications to define what something means, when there is a good specification available, but even then the IETF draft should at least have the pointer. From nobody Wed Jul 25 13:46:52 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CE2A0130E1B for ; Wed, 25 Jul 2018 13:46:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.12 X-Spam-Level: X-Spam-Status: No, score=-1.12 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vBBoROluEa_w for ; Wed, 25 Jul 2018 13:46:49 -0700 (PDT) Received: from server172-3.web-hosting.com (server172-3.web-hosting.com [68.65.122.111]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 59628130DE4 for ; Wed, 25 Jul 2018 13:46:49 -0700 (PDT) Received: from [146.96.19.240] (port=4189 helo=[10.10.201.39]) by server172.web-hosting.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.91) (envelope-from ) id 1fiQgH-001Iid-5e; Wed, 25 Jul 2018 16:46:47 -0400 From: Dave Rice Message-Id: <930D0F1F-5A08-44D8-9C72-DB1DED0370B9@dericed.com> Content-Type: multipart/alternative; boundary="Apple-Mail=_ECE6B966-1496-4C7F-85A3-DD0E05CD770F" Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Date: Wed, 25 Jul 2018 16:46:43 -0400 In-Reply-To: Cc: cellar@ietf.org To: "Timothy B. Terriberry" References: X-Mailer: Apple Mail (2.3273) X-OutGoing-Spam-Status: No, score=-2.9 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Subject: Re: [Cellar] Matroska element descriptions and defaults X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jul 2018 20:46:51 -0000 --Apple-Mail=_ECE6B966-1496-4C7F-85A3-DD0E05CD770F Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On Jul 24, 2018, at 11:47 AM, Timothy B. Terriberry = wrote: >=20 > Kieran O Leary wrote: >> If this is something that needs to be manually added, I could have a >> go, but perhaps it's outside of the scope of the IETF spec work. >=20 > This is completely in scope. It's okay to point to other = specifications to define what something means, when there is a good = specification available, but even then the IETF draft should at least = have the pointer. I added changes that put the enumeration vocabularies into the RFC in = https://github.com/Matroska-Org/matroska-specification/pull/251 = . Dave Rice= --Apple-Mail=_ECE6B966-1496-4C7F-85A3-DD0E05CD770F Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii
On Jul 24, 2018, at 11:47 AM, Timothy B. Terriberry <tterribe@xiph.org> = wrote:

Kieran O Leary wrote:
If this is something that needs to be manually added, I could = have a
go, but perhaps it's outside of the scope of the = IETF spec work.

This is = completely in scope. It's okay to point to other specifications to = define what something means, when there is a good specification = available, but even then the IETF draft should at least have the = pointer.

I added changes that put the enumeration vocabularies into = the RFC in https://github.com/Matroska-Org/matroska-specification/pull/251= .
Dave Rice
= --Apple-Mail=_ECE6B966-1496-4C7F-85A3-DD0E05CD770F-- From nobody Thu Jul 26 07:30:34 2018 Return-Path: X-Original-To: cellar@ietf.org Delivered-To: cellar@ietfa.amsl.com Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 75D3D13104A; Thu, 26 Jul 2018 07:30:32 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: internet-drafts@ietf.org To: Cc: cellar@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 6.83.0 Auto-Submitted: auto-generated Precedence: bulk Reply-To: cellar@ietf.org Message-ID: <153261543245.25998.3507214246260206603@ietfa.amsl.com> Date: Thu, 26 Jul 2018 07:30:32 -0700 Archived-At: Subject: [Cellar] I-D Action: draft-ietf-cellar-matroska-01.txt X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jul 2018 14:30:32 -0000 A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Codec Encoding for LossLess Archiving and Realtime transmission WG of the IETF. Title : Matroska Specifications Authors : Steve Lhomme Moritz Bunkus Dave Rice Filename : draft-ietf-cellar-matroska-01.txt Pages : 165 Date : 2018-07-26 Abstract: This document defines the Matroska audiovisual container, including definitions of its structural elements, as well as its terminology, vocabulary, and application. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-cellar-matroska/ There are also htmlized versions available at: https://tools.ietf.org/html/draft-ietf-cellar-matroska-01 https://datatracker.ietf.org/doc/html/draft-ietf-cellar-matroska-01 A diff from the previous version is available at: https://www.ietf.org/rfcdiff?url2=draft-ietf-cellar-matroska-01 Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From nobody Thu Jul 26 07:34:05 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 10D0113104A for ; Thu, 26 Jul 2018 07:34:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.119 X-Spam-Level: X-Spam-Status: No, score=-1.119 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, SPF_NEUTRAL=0.779, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KYVuUoJTrP0r for ; Thu, 26 Jul 2018 07:34:02 -0700 (PDT) Received: from server172-3.web-hosting.com (server172-3.web-hosting.com [68.65.122.111]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 93196130DF5 for ; Thu, 26 Jul 2018 07:34:02 -0700 (PDT) Received: from [146.96.19.240] (port=9993 helo=[10.10.201.39]) by server172.web-hosting.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.91) (envelope-from ) id 1fihL5-001rk7-P7; Thu, 26 Jul 2018 10:34:02 -0400 From: Dave Rice Message-Id: <3FBFC3C9-62A6-4D73-B307-DF7C665A2582@dericed.com> Content-Type: multipart/alternative; boundary="Apple-Mail=_15E5142B-AD94-47D4-9100-94639C927B59" Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Date: Thu, 26 Jul 2018 10:33:58 -0400 In-Reply-To: <930D0F1F-5A08-44D8-9C72-DB1DED0370B9@dericed.com> Cc: cellar@ietf.org To: "Timothy B. Terriberry" References: <930D0F1F-5A08-44D8-9C72-DB1DED0370B9@dericed.com> X-Mailer: Apple Mail (2.3273) X-OutGoing-Spam-Status: No, score=-2.9 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Subject: Re: [Cellar] Matroska element descriptions and defaults X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jul 2018 14:34:04 -0000 --Apple-Mail=_15E5142B-AD94-47D4-9100-94639C927B59 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On Jul 25, 2018, at 4:46 PM, Dave Rice wrote: >=20 >=20 >> On Jul 24, 2018, at 11:47 AM, Timothy B. Terriberry = > wrote: >>=20 >> Kieran O Leary wrote: >>> If this is something that needs to be manually added, I could have a >>> go, but perhaps it's outside of the scope of the IETF spec work. >>=20 >> This is completely in scope. It's okay to point to other = specifications to define what something means, when there is a good = specification available, but even then the IETF draft should at least = have the pointer. >=20 > I added changes that put the enumeration vocabularies into the RFC in = https://github.com/Matroska-Org/matroska-specification/pull/251 = . > Dave Rice These changes are changed and a new 01 version of matroska is at = https://datatracker.ietf.org/doc/draft-ietf-cellar-matroska/ = . The main = changes since 00 are: - mkv elements are now organized into subsections based on elemental = depth (up to a depth of 6) - enumeration vocabularies are now in tables in the rfc draft - updated colour primary and transfer characteristic vocabularies - other typo fixes. Best Regards, Dave Rice= --Apple-Mail=_15E5142B-AD94-47D4-9100-94639C927B59 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii
On Jul 25, 2018, at 4:46 PM, Dave Rice <dave@dericed.com> = wrote:


On Jul 24, 2018, at 11:47 AM, Timothy B. Terriberry <tterribe@xiph.org> = wrote:

Kieran O Leary wrote:
If this is something that needs to be manually added, I could = have a
go, but perhaps it's outside of the scope of the = IETF spec work.

This is = completely in scope. It's okay to point to other specifications to = define what something means, when there is a good specification = available, but even then the IETF draft should at least have the = pointer.

I added changes that put the enumeration vocabularies into = the RFC in https://github.com/Matroska-Org/matroska-specification/pull/251= .
Dave = Rice

These changes are changed and a new 01 version of matroska is = at https://datatracker.ietf.org/doc/draft-ietf-cellar-matroska/. The main changes since 00 are:
- mkv elements = are now organized into subsections based on elemental depth (up to a = depth of 6)
- enumeration vocabularies are now in = tables in the rfc draft
- updated colour primary = and transfer characteristic vocabularies
- other = typo fixes.

Best= Regards,
Dave Rice
= --Apple-Mail=_15E5142B-AD94-47D4-9100-94639C927B59-- From nobody Fri Jul 27 03:45:27 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 418A0130F18 for ; Fri, 27 Jul 2018 03:45:26 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.979 X-Spam-Level: X-Spam-Status: No, score=-0.979 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, MISSING_HEADERS=1.021, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lpg4NGkIjjtp for ; Fri, 27 Jul 2018 03:45:25 -0700 (PDT) Received: from mail-wm0-x235.google.com (mail-wm0-x235.google.com [IPv6:2a00:1450:400c:c09::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BE3F4130F09 for ; Fri, 27 Jul 2018 03:45:24 -0700 (PDT) Received: by mail-wm0-x235.google.com with SMTP id s9-v6so4820196wmh.3 for ; Fri, 27 Jul 2018 03:45:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:cc; bh=zwBnRlvbPYNKWGHI5+jcqO6SXOg985CeRib/B3jQ+fs=; b=ObPCtMOrb32v3Kf5wYZ/NDYQiG6SHzVoNsZ3HmIz5P8XxhIQ//TPWGGoWq8rjSPcaa 9Sfq3d2koF68FyDy350UWY6HN/xP1QehEPv1QdpqDQU2NYNtDQWGzAYavHkLO0Cnq5SG 1j7tUxxtAYyGq/WkOLr/5OksRIwGUFc/zNXKVE70PHNlXUIydVdz5xN1lC4qKvLAei39 5y3Q6OGpJ9EvNt9Q0u6PsnoZDZqA/JQqWjYcRRQcYsL3oVtPmGx/sof9ZRN5FvrRFdnY 9JD5pEnj08c9ggycIPBdP+lPsB2D+lZAXdZ/Hr4hDu2pMJHBj3sk3jVspFxeaxbEACuQ VUWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:cc; bh=zwBnRlvbPYNKWGHI5+jcqO6SXOg985CeRib/B3jQ+fs=; b=ExrziQVSyUdBZTWQOQC3YXXloct1PRgqo5IvJEzGR5tisWK7k8mHrxMcZgV1OAT3o8 fmCB8/rU4W6aMIWnyW9cXEeMgGMbCffVaUPfahfZh0zWyb23AdRvQEnDyFXOHfipK18y 0Ul4m8Yi3xG5oosSCiPnsnNP3Z11Z7bRvyJvAFvgJ6c0uNQBqkN4E5WuA5eZIPIxxM/5 ayysa9VMKkdOqGM9eQ152BOb+ViaKVvy1tSdVOqRyJ4eqVD08sfblMBSDVWtsiUOZg2f quXGf5/hNyZsFNlSFJqeiHa+ezuR2bVwb0GJzs1rzI7YpkNKaXfks3Uu6xyHf08oRBsE BB/A== X-Gm-Message-State: AOUpUlHtDrIZ7ex0Fy0OqtNSN9fcANelE8Q8JT6c95KhL00DyIXKO5cz o+hoEeerksiTt21EW1wG0Ir2r7eSgP8IoI3ChGmX X-Google-Smtp-Source: AAOMgpcXX4Aij9AQVXb4dkkLSBDZ2VISQWhRkIAbS7x0oxTkaxYo+QjkPFJOZ1iPpESCGwE/D2Sia/vviqOpCm3b9ms= X-Received: by 2002:a1c:d946:: with SMTP id q67-v6mr3916472wmg.156.1532688322946; Fri, 27 Jul 2018 03:45:22 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a7b:c098:0:0:0:0:0 with HTTP; Fri, 27 Jul 2018 03:45:22 -0700 (PDT) In-Reply-To: References: <538B01A1-5411-43B4-818C-82A704370CD4@gmail.com> From: Kieran O Leary Date: Fri, 27 Jul 2018 11:45:22 +0100 Message-ID: Cc: cellar@ietf.org Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cellar] Contributing recommendations (for new users) (according to me) X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jul 2018 10:45:26 -0000 Sorry for the bump, but I wonder if there is a value in adding some contributor guidelines to the github pages? I'm about to start a review of the Matroska spec and I was looking for some guidance on how best to approach it. Ashley's guidance is pretty much exactly what I needed. -Kieran O'Leary IFI Irish Film Archive. On Mon, Aug 14, 2017 at 3:47 AM, Katherine Frances wrote: > Hey Ashley, > > Thanks for this timely post. Dave Rice contacted me some months ago to help > with the specs, but new job & general life responsibilities got in the way. > So it's a useful mail for me, hopefully I can dive into some spec work in > the next while! > > Cheers, K > > > On Tue, Aug 8, 2017 at 12:09 PM, Ashley Blewer > wrote: >> >> Hi CELLAR, >> >> I've talked to a couple of lurkers on this list that want to get involved >> but don't know where to start. Here are a few guidelines just based on my >> experience: >> >> EBML, Matroska, FFV1, FLAC: >> >> Review for inconsistencies: typos, contradictions, ambiguity in the >> documentation. So just picking one of the specs and giving it a good review >> is incredibly helpful. The Matroska spec needs a lot of work here, as it is >> written in a colloquial manner inconsistent with "standards-style" language. >> There's plenty of work to do before that can really start to matter, but >> it's a good starting point to get familiar with the docs. That's what I've >> started doing. This can be brought up on the CELLAR list (preferred), >> brought up as a Github issue for the specification, or a pull request can be >> sent to change the codebase and a conversation can happen there. >> >> Specs can be reviewed purely with an eye towards compliance with RFC2119 >> (https://www.ietf.org/rfc/rfc2119.txt): Are these words being used >> appropriately? Is there something listed as a SHOULD but should be a MUST? >> Is the word being used but not capitalized, and so recommendation is being >> unintentionally implied, and the word should either become full-caps or >> changed? This kind of tedious-work passthrough is a good way to familiarize >> yourself with one of the documents with plans to analyze more specifically >> later. >> >> EBML: >> >> EBML is in a more advanced stage than others, but can still use more eyes >> for general review. This initial review of EBML is helpful as a guideline >> for what to look for and how to look for it: >> https://mailarchive.ietf.org/arch/msg/cellar/Mfj44tK1gyfqjU3Uu1tI5fMovtk >> >> Matroska: >> >> I've been just doing that general work of cleaning up language and looking >> for inconsistencies. Matroska docs are VERY long and helpful for users but >> less "specification standard". Lots of eyes here would be helpful since >> there is a lot of ground to cover. There's a handful of issues on the Github >> tracker (https://github.com/Matroska-Org/matroska-specification/issues) but >> not sure how many are suitable for novices. A review of the Matroska spec >> that seeks for any redundant or conflicting information is a worthwhile >> endeavor. We are very fortunate to have Moritz and Steve as active members >> and contributors to this group. :) >> >> FFV1: >> >> FFV1 is a little harder to break into, but there was a review by Derek >> Buitenhuis posted to the CELLAR list on July 28 pointing out questions ( >> https://mailarchive.ietf.org/arch/msg/cellar/qItSPq06vc9MaxsZFPJ3GmREsr4 ). >> Work towards reviewing and deciding what can be done with this can be used >> as a basis for where to start looking. Again, just searching for typos and >> contradictions and noting them is important. >> >> FLAC: >> >> FLAC has some issues on the issue tracker that could be worked on: >> https://github.com/privatezero/flac_markdown/issues >> >> Issue #25 is of note -- when I have more time, I'll dig into that one a >> bit more and make new tickets. It seems like the code and the specification >> have diverged slightly in terms of what things are named, and the spec >> should be updated to reflect what is in the source code. That's some fun >> ("fun") digging-around work that takes time but isn't intrinsically hard or >> requires expertise in the languages. The original ticket is about getting >> anchors to work, there's some discussion in the ticket about what to do >> about that. >> >> In general, FLAC needs a more permanent Github home that isn't Andrew's >> personal Github (as reported by the IETF meeting notes -- unsure what kind >> of nudging should be done here), and more nudging on the flac-dev list about >> any questions in the spec that need explaining. It'd be good to get a closer >> feedback loop with the FLAC crew, which can be done by bringing up issues on >> both listservs. Not that much work has been started on FLAC yet so it's open >> terrain. >> >> Overall, just reading and questioning the specifications is valuable work. >> >> If you are not very familiar with Github or are generally unsure how to >> contribute there, please feel free to contact me off-list and I'd be happy >> to explain/clarify questions. >> >> My best, >> >> Ashley >> >> _______________________________________________ >> Cellar mailing list >> Cellar@ietf.org >> https://www.ietf.org/mailman/listinfo/cellar >> > > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > From nobody Fri Jul 27 08:44:14 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D2E33130F8D for ; Fri, 27 Jul 2018 08:44:12 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.201 X-Spam-Level: X-Spam-Status: No, score=-4.201 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SNbPYSJ5OVkM for ; Fri, 27 Jul 2018 08:44:09 -0700 (PDT) Received: from tuna.sandelman.ca (tuna.sandelman.ca [IPv6:2607:f0b0:f:3:216:3eff:fe7c:d1f3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9B55F130EFF for ; Fri, 27 Jul 2018 08:44:09 -0700 (PDT) Received: from sandelman.ca (obiwan.sandelman.ca [IPv6:2607:f0b0:f:2::247]) by tuna.sandelman.ca (Postfix) with ESMTP id 4927320008; Fri, 27 Jul 2018 12:00:24 -0400 (EDT) Received: by sandelman.ca (Postfix, from userid 179) id 56D0E1572; Fri, 27 Jul 2018 11:44:03 -0400 (EDT) Received: from sandelman.ca (localhost [127.0.0.1]) by sandelman.ca (Postfix) with ESMTP id 530201517; Fri, 27 Jul 2018 11:44:03 -0400 (EDT) From: Michael Richardson To: Kieran O Leary cc: cellar@ietf.org In-Reply-To: References: <538B01A1-5411-43B4-818C-82A704370CD4@gmail.com> X-Mailer: MH-E 8.6; nmh 1.7+dev; GNU Emacs 24.5.1 X-Face: $\n1pF)h^`}$H>Hk{L"x@)JS7<%Az}5RyS@k9X%29-lHB$Ti.V>2bi.~ehC0; <'$9xN5Ub# z!G,p`nR&p7Fz@^UXIn156S8.~^@MJ*mMsD7=QFeq%AL4m Archived-At: Subject: Re: [Cellar] Contributing recommendations (for new users) (according to me) X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jul 2018 15:44:13 -0000 --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Kieran O Leary wrote: > Sorry for the bump, but I wonder if there is a value in adding some > contributor guidelines to the github pages? What exactly are you thinking about? Are you thinking about text formatting issues? The IETF NOTEWELL applies, as far as all legal issues. See: https://www.ietf.org/about/note-well/ We should probably pull it into the README.md, like: https://github.com/anima-wg/voucher does. > I'm about to start a review of the Matroska spec and I was looking for > some guidance on how best to approach it. Ashley's guidance is pretty > much exactly what I needed. Thank you for the review. Please post your review as email to this email list. > -Kieran O'Leary > IFI Irish Film Archive. > On Mon, Aug 14, 2017 at 3:47 AM, Katherine Frances wrote: >> Hey Ashley, >>=20 >> Thanks for this timely post. Dave Rice contacted me some months ago = to help >> with the specs, but new job & general life responsibilities got in t= he way. >> So it's a useful mail for me, hopefully I can dive into some spec wo= rk in >> the next while! >>=20 >> Cheers, K >>=20 >>=20 >> On Tue, Aug 8, 2017 at 12:09 PM, Ashley Blewer >> wrote: >>>=20 >>> Hi CELLAR, >>>=20 >>> I've talked to a couple of lurkers on this list that want to get in= volved >>> but don't know where to start. Here are a few guidelines just based= on my >>> experience: >>>=20 >>> EBML, Matroska, FFV1, FLAC: >>>=20 >>> Review for inconsistencies: typos, contradictions, ambiguity in the >>> documentation. So just picking one of the specs and giving it a goo= d review >>> is incredibly helpful. The Matroska spec needs a lot of work here, = as it is >>> written in a colloquial manner inconsistent with "standards-style" = language. >>> There's plenty of work to do before that can really start to matter= , but >>> it's a good starting point to get familiar with the docs. That's wh= at I've >>> started doing. This can be brought up on the CELLAR list (preferred= ), >>> brought up as a Github issue for the specification, or a pull reque= st can be >>> sent to change the codebase and a conversation can happen there. >>>=20 >>> Specs can be reviewed purely with an eye towards compliance with RF= C2119 >>> (https://www.ietf.org/rfc/rfc2119.txt): Are these words being used >>> appropriately? Is there something listed as a SHOULD but should be = a MUST? >>> Is the word being used but not capitalized, and so recommendation i= s being >>> unintentionally implied, and the word should either become full-cap= s or >>> changed? This kind of tedious-work passthrough is a good way to fam= iliarize >>> yourself with one of the documents with plans to analyze more speci= fically >>> later. >>>=20 >>> EBML: >>>=20 >>> EBML is in a more advanced stage than others, but can still use mor= e eyes >>> for general review. This initial review of EBML is helpful as a gui= deline >>> for what to look for and how to look for it: >>> https://mailarchive.ietf.org/arch/msg/cellar/Mfj44tK1gyfqjU3Uu1tI5f= Movtk >>>=20 >>> Matroska: >>>=20 >>> I've been just doing that general work of cleaning up language and = looking >>> for inconsistencies. Matroska docs are VERY long and helpful for us= ers but >>> less "specification standard". Lots of eyes here would be helpful s= ince >>> there is a lot of ground to cover. There's a handful of issues on t= he Github >>> tracker (https://github.com/Matroska-Org/matroska-specification/iss= ues) but >>> not sure how many are suitable for novices. A review of the Matrosk= a spec >>> that seeks for any redundant or conflicting information is a worthw= hile >>> endeavor. We are very fortunate to have Moritz and Steve as active = members >>> and contributors to this group. :) >>>=20 >>> FFV1: >>>=20 >>> FFV1 is a little harder to break into, but there was a review by De= rek >>> Buitenhuis posted to the CELLAR list on July 28 pointing out questi= ons ( >>> https://mailarchive.ietf.org/arch/msg/cellar/qItSPq06vc9MaxsZFPJ3Gm= REsr4 ). >>> Work towards reviewing and deciding what can be done with this can = be used >>> as a basis for where to start looking. Again, just searching for ty= pos and >>> contradictions and noting them is important. >>>=20 >>> FLAC: >>>=20 >>> FLAC has some issues on the issue tracker that could be worked on: >>> https://github.com/privatezero/flac_markdown/issues >>>=20 >>> Issue #25 is of note -- when I have more time, I'll dig into that o= ne a >>> bit more and make new tickets. It seems like the code and the speci= fication >>> have diverged slightly in terms of what things are named, and the s= pec >>> should be updated to reflect what is in the source code. That's som= e fun >>> ("fun") digging-around work that takes time but isn't intrinsically= hard or >>> requires expertise in the languages. The original ticket is about g= etting >>> anchors to work, there's some discussion in the ticket about what t= o do >>> about that. >>>=20 >>> In general, FLAC needs a more permanent Github home that isn't Andr= ew's >>> personal Github (as reported by the IETF meeting notes -- unsure wh= at kind >>> of nudging should be done here), and more nudging on the flac-dev l= ist about >>> any questions in the spec that need explaining. It'd be good to get= a closer >>> feedback loop with the FLAC crew, which can be done by bringing up = issues on >>> both listservs. Not that much work has been started on FLAC yet so = it's open >>> terrain. >>>=20 >>> Overall, just reading and questioning the specifications is valuabl= e work. >>>=20 >>> If you are not very familiar with Github or are generally unsure ho= w to >>> contribute there, please feel free to contact me off-list and I'd b= e happy >>> to explain/clarify questions. >>>=20 >>> My best, >>>=20 >>> Ashley >>>=20 >>> _______________________________________________ >>> Cellar mailing list >>> Cellar@ietf.org >>> https://www.ietf.org/mailman/listinfo/cellar >>>=20 >>=20 >>=20 >> _______________________________________________ >> Cellar mailing list >> Cellar@ietf.org >> https://www.ietf.org/mailman/listinfo/cellar >>=20 > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar =2D-=20 Michael Richardson , Sandelman Software Works -=3D IPv6 IoT consulting =3D- --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBW1s9w4CLcPvd0N1lAQLurwf/T1QrL1rP6zcCHyqQTpn+c3krNmEIQAkA YUwVVaMi08OvmI0vMETDP/2lgJKyIJQexVno0J5m9m0ekuMG7bn/lWZP5y8FKZhE +PUCVVTC4VRS41jw/Pp9NSHC9yT6fm2N8GI996AI2gblSkFMF5vSm1X4/S2aPTLV AS1fg9AQX9yMnhNwgT5GXdfYjz9HVBs8kwiHxXlr/OjIG/CNg0sIC/oPudgmuOBp e9DZnoGxbAJZN14dfU2c8++bxQeV2VHCt9aMb3CgTws0evbOXC0iicHmwOLI2UsB JKLp4n8txk/GPl5e4xHibSqTa30/jVPM3ik2XJCW7dhGCAr1WHxNtw== =qppI -----END PGP SIGNATURE----- --=-=-=-- From nobody Fri Jul 27 13:34:01 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 65604131083 for ; Fri, 27 Jul 2018 13:33:52 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.201 X-Spam-Level: X-Spam-Status: No, score=-4.201 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bNqnTA2EzAJY for ; Fri, 27 Jul 2018 13:33:49 -0700 (PDT) Received: from tuna.sandelman.ca (tuna.sandelman.ca [IPv6:2607:f0b0:f:3:216:3eff:fe7c:d1f3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0E25F130ED7 for ; Fri, 27 Jul 2018 13:33:49 -0700 (PDT) Received: from sandelman.ca (obiwan.sandelman.ca [IPv6:2607:f0b0:f:2::247]) by tuna.sandelman.ca (Postfix) with ESMTP id AF38E20491 for ; Fri, 27 Jul 2018 16:50:06 -0400 (EDT) Received: by sandelman.ca (Postfix, from userid 179) id 59A651572; Fri, 27 Jul 2018 16:33:46 -0400 (EDT) Received: from sandelman.ca (localhost [127.0.0.1]) by sandelman.ca (Postfix) with ESMTP id 55B6E1517 for ; Fri, 27 Jul 2018 16:33:46 -0400 (EDT) From: Michael Richardson To: cellar@ietf.org X-Attribution: mcr X-Mailer: MH-E 8.6; nmh 1.7+dev; GNU Emacs 24.5.1 X-Face: $\n1pF)h^`}$H>Hk{L"x@)JS7<%Az}5RyS@k9X%29-lHB$Ti.V>2bi.~ehC0; <'$9xN5Ub# z!G,p`nR&p7Fz@^UXIn156S8.~^@MJ*mMsD7=QFeq%AL4m Archived-At: Subject: [Cellar] REMINDER, Virtual Interim, 2018-07-31 (Tuesday), 20:00 UTC X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jul 2018 20:34:00 -0000 --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable CELLAR -- Agenda for Virtual Interim Meeting July 31, 2018 20:00 UTC INFO: https://datatracker.ietf.org/meeting/interim-2018-cellar-03/session= /cellar WEBEX: Meeting number: 313 505 170 Meeting password: dq64cWsm Meeting link: https://ietf.webex.com/ietf/j.php?MTID=3Dme21026faa14be85507cdef66b= 860014e Host key: 763872 Audio connection: 1-650-479-3208 Call-in toll number (US/Canada) Agenda: 1. Note Well. 2. Draft minutes from Last meeting 3. Logistics for Meeting. 2a) Etherpad for notes https://etherpad.tools.ietf.org/p/notes-cellar-virtual03?useMonospac= eFont=3Dtrue 2b) Roll call 4. Any updates from IETF102. - GENART review. 5. EBML document status. How to get this into WG Last Call. 6. Milestones for WG. Proposed changes: Jul 2018 Adopt matroska specifications as WG documents Aug 2018 Submit specification for EBML to IESG (Standards Track) Aug 2018 Adopt flac specifications as WG documents Oct 2018 Submit informational specification for FFV1 video codec versions 0, 1 and 3 to IESG for publication draft-ietf-cellar-ffv1 Dec 2019 Submit specification for FFV1 video codec version 4 to IESG (= Standards Track) xxx 2019 Submit informational specification for Matroska container format versions 1, 2 and 3 to IESG for publication xxx 2019 Submit specification for FLAC audio codec to IESG (Standards = Track) draft-xiph-cellar-flac xxx 2019 Submit specification for Matroska container format version 4 = to IESG (Standards Track) draft-lhomme-cellar-matroska What would be appropriate values for xxx above?=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20 7. responses/plan to address GENART review of ffv1. =2D-=20 Michael Richardson , Sandelman Software Works -=3D IPv6 IoT consulting =3D- --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBW1uBqoCLcPvd0N1lAQKWGQf/dbC5Rj+U61frmg78pRHcsXcgNBKBhqQS KjpyBWUerJOf9ysIjXRrfMojVsZwXizEUtcRWFg/YoU4RkdMMQVwyT+Kdg65i9v8 npD30GVyG3rl7iOnKKW9VE819aFhDQo4/n4byuI0Ypg9Qkxb8ZPBGJdhlgSUGAcA 1vC8n9iIJEL4dvXywaNPwpX1i08FkrJbnWUDCIlYlczw/6xcohE6VSoo1q6u4+Kz 1mGo2AaRh/Cib5GLRSTG3kpO9tNcjQ3FAbeQ8v4dhhkIVSN7JunxdfzvwesxE9sS H2s+pStpQsdJSWXqoJ5qTECPIj4ypIR76dMfjGxMvdior4OLXLNpEA== =ufL4 -----END PGP SIGNATURE----- --=-=-=-- From nobody Fri Jul 27 14:07:57 2018 Return-Path: X-Original-To: cellar@ietf.org Delivered-To: cellar@ietfa.amsl.com Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 24F4C130E11; Fri, 27 Jul 2018 14:07:50 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: internet-drafts@ietf.org To: Cc: cellar@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 6.83.0 Auto-Submitted: auto-generated Precedence: bulk Reply-To: cellar@ietf.org Message-ID: <153272567012.32667.15191169339749098218@ietfa.amsl.com> Date: Fri, 27 Jul 2018 14:07:50 -0700 Archived-At: Subject: [Cellar] I-D Action: draft-ietf-cellar-ffv1-04.txt X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jul 2018 21:07:50 -0000 A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Codec Encoding for LossLess Archiving and Realtime transmission WG of the IETF. Title : FFV1 Video Coding Format Version 0, 1, and 3 Authors : Michael Niedermayer Dave Rice Jerome Martinez Filename : draft-ietf-cellar-ffv1-04.txt Pages : 41 Date : 2018-07-27 Abstract: This document defines FFV1, a lossless intra-frame video encoding format. FFV1 is designed to efficiently compress video data in a variety of pixel formats. Compared to uncompressed video, FFV1 offers storage compression, frame fixity, and self-description, which makes FFV1 useful as a preservation or intermediate video format. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-cellar-ffv1/ There are also htmlized versions available at: https://tools.ietf.org/html/draft-ietf-cellar-ffv1-04 https://datatracker.ietf.org/doc/html/draft-ietf-cellar-ffv1-04 A diff from the previous version is available at: https://www.ietf.org/rfcdiff?url2=draft-ietf-cellar-ffv1-04 Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From nobody Fri Jul 27 14:12:39 2018 Return-Path: X-Original-To: cellar@ietf.org Delivered-To: cellar@ietfa.amsl.com Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id A18D4130E10; Fri, 27 Jul 2018 14:12:32 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: internet-drafts@ietf.org To: Cc: cellar@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 6.83.0 Auto-Submitted: auto-generated Precedence: bulk Reply-To: cellar@ietf.org Message-ID: <153272595263.32671.13541394953597629903@ietfa.amsl.com> Date: Fri, 27 Jul 2018 14:12:32 -0700 Archived-At: Subject: [Cellar] I-D Action: draft-ietf-cellar-ffv1-v4-01.txt X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jul 2018 21:12:33 -0000 A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Codec Encoding for LossLess Archiving and Realtime transmission WG of the IETF. Title : FFV1 Video Coding Format Version 4 Authors : Michael Niedermayer Dave Rice Jerome Martinez Filename : draft-ietf-cellar-ffv1-v4-01.txt Pages : 42 Date : 2018-07-27 Abstract: This document defines FFV1, a lossless intra-frame video encoding format. FFV1 is designed to efficiently compress video data in a variety of pixel formats. Compared to uncompressed video, FFV1 offers storage compression, frame fixity, and self-description, which makes FFV1 useful as a preservation or intermediate video format. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-cellar-ffv1-v4/ There are also htmlized versions available at: https://tools.ietf.org/html/draft-ietf-cellar-ffv1-v4-01 https://datatracker.ietf.org/doc/html/draft-ietf-cellar-ffv1-v4-01 A diff from the previous version is available at: https://www.ietf.org/rfcdiff?url2=draft-ietf-cellar-ffv1-v4-01 Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From nobody Tue Jul 31 03:16:44 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 515A8124BE5 for ; Tue, 31 Jul 2018 03:16:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id P62SMfVxUE0a for ; Tue, 31 Jul 2018 03:16:37 -0700 (PDT) Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D3207130DEC for ; Tue, 31 Jul 2018 03:16:36 -0700 (PDT) Received: by mail-wr1-x430.google.com with SMTP id j5-v6so16044685wrr.8 for ; Tue, 31 Jul 2018 03:16:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=Fu961aJP5BbdPtjVTks+hh152G6Sf+l3l47ArTQzD/0=; b=WynYKmDM+TnpsbXVEhcTjU/vbF7qrxsTMiVlsreXe3LDvVNJQf2VBTOpC+rXwuDhOH eJpD41PtW/P4zSaZfn4EGo+2mY7yd5GWYn1RggqMwDYRizL2IUjIVloJfILOg5PkrcnL FL9GxvqE6mOpNi7bNgzqMdVhkYWjCD8gpy3vlgVJaWitk5f0fgrhtuCi97r9PrzMZZ9j Zplwi4HKVtG4jNafvD1QqOsoAJ2/U+kVVJfgjJ+Q04ZVX2smADgk8DGDD+vw3i6S71/h rWmfdW2UWl5iUOnKndKCZ+4IGz5fDjg8ugUtwhZpTo5wKlCks16X09Vp6b1I9J4ql+O5 PRwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Fu961aJP5BbdPtjVTks+hh152G6Sf+l3l47ArTQzD/0=; b=bDTErOHOYXlap3RJPB/cvLaVef+f5/YJz8FnuOaFj/PAYGiAyezG9GkJNEJFYAvkKk hZ0BewSoQl1yQNaiECYhibbGgSQLxejgRvHl/j9fWmwT0ffPd/gWIwEP/dR5DK1cuxtz 5yNp8Z5CgNgV7A5qqZJzLSaSl+Qrs/RjC2xneOA4FBQi/Yb+IZCG1T+DbhdFOoV8aL8N fpK5DDxaMPWKWvztbkwttYisp8RMAmp5oYl7ccVM8RW9em7z2j+3hR1y+OageguM6Jvp H8H7GEKsD9yYWBPjDrAc0WWN32r2aaP00+z47isv27sh6QZRf2UWI17cklnxJryJ9b4p VEpg== X-Gm-Message-State: AOUpUlE69kEIZjE6nriRz/G93nptV2++sl48Vq9lheIJUZ3viqtVVUvW VxC5d7PkOxB65syYpWbHNwGcErxDYQbck5sG9SbN X-Google-Smtp-Source: AAOMgpcjm/a5J4pUPs1UXgZF2ZfjGiwj80QXiLV04c2vUbrF1rIxynNOR09FBvT7eQFXPVpUFqLeekDY43KsZVtvMjk= X-Received: by 2002:a5d:4a07:: with SMTP id m7-v6mr21717759wrq.8.1533032195355; Tue, 31 Jul 2018 03:16:35 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a7b:c098:0:0:0:0:0 with HTTP; Tue, 31 Jul 2018 03:16:34 -0700 (PDT) In-Reply-To: <2898.1532706243@localhost> References: <538B01A1-5411-43B4-818C-82A704370CD4@gmail.com> <2898.1532706243@localhost> From: Kieran O Leary Date: Tue, 31 Jul 2018 11:16:34 +0100 Message-ID: To: Michael Richardson Cc: cellar@ietf.org Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cellar] Contributing recommendations (for new users) (according to me) X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 10:16:42 -0000 Hi Michael, On Fri, Jul 27, 2018 at 4:44 PM, Michael Richardson wrote: > > Kieran O Leary wrote: > > Sorry for the bump, but I wonder if there is a value in adding some > > contributor guidelines to the github pages? > > What exactly are you thinking about? I was thinking more along the lines of what Ashley mentioned earlier in the thread. How to contribute to the project. Specifically, I wanted some guidance on what kind of reviews might be needed, depending on your knowledge level, and how to deliver the feedback (github/cellar) > Are you thinking about text formatting issues? > The IETF NOTEWELL applies, as far as all legal issues. > See: https://www.ietf.org/about/note-well/ > > We should probably pull it into the README.md, like: > https://github.com/anima-wg/voucher > > does. > > > I'm about to start a review of the Matroska spec and I was looking for > > some guidance on how best to approach it. Ashley's guidance is pretty > > much exactly what I needed. > > Thank you for the review. > Please post your review as email to this email list. Will do! Best, Kieran. From nobody Tue Jul 31 16:52:15 2018 Return-Path: X-Original-To: cellar@ietf.org Delivered-To: cellar@ietfa.amsl.com Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 9AB24130EA3 for ; Tue, 31 Jul 2018 16:52:13 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit To: X-Test-IDTracker: no X-IETF-IDTracker: 6.83.0 Auto-Submitted: auto-generated Precedence: bulk Message-ID: <153308113362.3174.2732663077888411877.idtracker@ietfa.amsl.com> Date: Tue, 31 Jul 2018 16:52:13 -0700 From: IETF Secretariat Archived-At: Subject: [Cellar] Milestones changed for cellar WG X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.27 List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 23:52:14 -0000 Changed milestone "Submit specification for EBML to IESG (Standards Track)", set due date to August 2018 from April 2017, added draft-ietf-cellar-ebml to milestone. Changed milestone "Submit informational specification for FFV1 video codec versions 0, 1 and 3 to IESG for publication", set due date to October 2018 from April 2017, added draft-ietf-cellar-ffv1 to milestone. Changed milestone "Submit specification for FFV1 video codec version 4 to IESG (Standards Track)", set due date to December 2018 from July 2017, added draft-ietf-cellar-ffv1-v4 to milestone. Changed milestone "Submit informational specification for Matroska container format versions 1, 2 and 3 to IESG for publication", set due date to April 2019 from July 2017, added draft-ietf-cellar-matroska to milestone. Changed milestone "Submit specification for Matroska container format version 4 to IESG (Standards Track)", set due date to April 2019 from September 2017. Changed milestone "Submit specification for FLAC audio codec to IESG (Standards Track)", set due date to April 2019 from December 2017. URL: https://datatracker.ietf.org/wg/cellar/about/