From nobody Wed Aug 1 01:58:39 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2880B130F82 for ; Wed, 1 Aug 2018 01:58:38 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.299 X-Spam-Level: X-Spam-Status: No, score=-2.299 tagged_above=-999 required=5 tests=[RCVD_IN_DNSWL_MED=-2.3, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2NNmTfIqV32M for ; Wed, 1 Aug 2018 01:58:35 -0700 (PDT) Received: from mailhost.informatik.uni-bremen.de (mailhost.informatik.uni-bremen.de [IPv6:2001:638:708:30c9::12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 621D9129385 for ; Wed, 1 Aug 2018 01:58:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at informatik.uni-bremen.de Received: from submithost.informatik.uni-bremen.de (submithost.informatik.uni-bremen.de [134.102.201.11]) by mailhost.informatik.uni-bremen.de (8.14.5/8.14.5) with ESMTP id w718wUGi008313; Wed, 1 Aug 2018 10:58:30 +0200 (CEST) Received: from client-0081.vpn.uni-bremen.de (client-0081.vpn.uni-bremen.de [134.102.107.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by submithost.informatik.uni-bremen.de (Postfix) with ESMTPSA id 41gRzp46J2zDXMg; Wed, 1 Aug 2018 10:58:30 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) From: Carsten Bormann In-Reply-To: <20180731225854.GA18176@davidb.org> Date: Wed, 1 Aug 2018 10:58:29 +0200 Cc: cbor@ietf.org X-Mao-Original-Outgoing-Id: 554806707.625007-caae92523d804331f7818c5409045530 Content-Transfer-Encoding: quoted-printable Message-Id: <8D8BDD10-8139-44D9-B028-194524A6C295@tzi.org> References: <20180731225854.GA18176@davidb.org> To: David Brown X-Mailer: Apple Mail (2.3445.9.1) Archived-At: Subject: Re: [Cbor] Parsing CDDL X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 08:58:38 -0000 > On Aug 1, 2018, at 00:58, David Brown wrote: >=20 > I have written up a small (220 line) parser based on the > draft-ietf-cbor-cddl-03 document. The code can be found at > https://github.com/d3zd3z/suit-play/blob/master/cddl.rkt. I used the > Racket lexer and parser tools to make this easier. >=20 > I believe this parser parses (nearly) the same language described in > the ABNF grammar in the draft (see below about spaces). However, a > couple of observations: >=20 > - in the rule for 'rule', since a 'type' by itself is a valid > 'grpent', this rule is ambiguous. I made the parser work by > simply commenting out the first alternate. I would gather that a > tool using the output of this parser would have to store all > entities, and check that a name used elsewhere as a type must be > restricted to the =E2=80=98type' form. Yes, that is the way the existing cddl tool is implemented. Specifically, the ABNF given is =E2=80=9Cambiguous=E2=80=9D in the sense = of A =3D B being parsable as both a type and a group. This =E2=80=9Cambiguity=E2=80=9D= does not cause any problems with the grammar, but needs unification = before feeding the grammar to a classic parser generator). (In a PEG parser, the first rule always matches, so there is no = =E2=80=9Cambiguity=E2=80=9D.) (I=E2=80=99m using =E2=80=9Cambiguous=E2=80=9D in quotes, because the = spec is not ambiguous here =E2=80=94 just the assignment of the parse to = one or the other rule; both generate the same language.) > - Spaces are a little complicated, and I'm not sure quite as > intended. One production (for #6(...)) has a comment "note no > space", however there are a few other places that are missing > spaces (such as the space after the open paren on the >=20 > "(" type ")" >=20 > alternate. =20 Right. The grammar is trying to keep the opportunities for inserting = spaces explicit, but that did not quite work out here (=E2=80=9Ctype=E2=80= =9D allows for trailing, but not leading white space): type =3D type1 S *(=E2=80=9C/=E2=80=9C S type1 S) This probably should be cleaned up to type =3D type1 *(S =E2=80=9C/=E2=80=9C S type1) and then changing type2 as in: type2 =3D value / typename [genericarg] / =E2=80=9C(=E2=80=9C S type S =E2=80=9C)=E2=80=9D > I found this much easier to handle by consuming spaces > in the lexer and not making them part of the grammar at all. =20 Lexers made sense for certain environments and languages. ABNF does not have them, so CDDL's ABNF grammar is not trying to = introduce them. > In > order to do this, three things: >=20 > "#" "6" ["." uint] "(" S type S ")" > "#" DIGIT ["." uint] >=20 > [uint] "*" [uint] >=20 > must be captured by the lexer, since these production would make > the grammar ambiguous unless the space is not allowed. =20 Right, if you insist on doing the lexer/parser split, you will have to = split up the grammar to your liking. There is an opportunity for catching more issues at the syntax level = here by replacing DIGIT with NONTAGMAJOR as in NONTAGMAJOR =3D %x30-35 / =E2=80=9C7=E2=80=9D (Choose your preferred way to remove the cognitive dissonance the above = line invariably creates.) > I assume > the intent is to interpret these with "maximal munch" (mostly > thanks to the comma being optional). Not sure what =E2=80=9Cthese=E2=80=9D is here =E2=80=94 there are no = commas in the above rules. > I also capture the repetition operation with numbers at lex time > to avoid additional ambiguity. Again, if you must have a lexer, that approach does help here. (The main remaining argument I sometimes hear for having a lexer is that = it keeps noise out of the AST. I prefer to have a parser with an AST generator that allows me to get a = clean AST closer to the semantics I actually need.) Gr=C3=BC=C3=9Fe, Carsten From nobody Wed Aug 1 16:02:37 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C133E130E9E for ; Wed, 1 Aug 2018 16:02:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.898 X-Spam-Level: X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zToMlyo5UykR for ; Wed, 1 Aug 2018 16:02:33 -0700 (PDT) Received: from p3plsmtpa06-09.prod.phx3.secureserver.net (p3plsmtpa06-09.prod.phx3.secureserver.net [173.201.192.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6F0101277BB for ; Wed, 1 Aug 2018 16:02:33 -0700 (PDT) Received: from [192.168.1.23] ([73.35.165.162]) by :SMTPAUTH: with ESMTPSA id l08WfhW5XkuRKl08WfmvnQ; Wed, 01 Aug 2018 16:02:32 -0700 From: Laurence Lundblade Content-Type: multipart/alternative; boundary="Apple-Mail=_2296A2ED-0AAD-4C75-9F1F-F853DF272871" Mime-Version: 1.0 (Mac OS X Mail 11.3 \(3445.6.18\)) Message-Id: <80DFAEF6-BE25-4F43-9CCF-A8FCBEB1F4BE@island-resort.com> Date: Wed, 1 Aug 2018 16:02:32 -0700 To: cbor@ietf.org X-Mailer: Apple Mail (2.3445.6.18) X-CMAE-Envelope: MS4wfCngN7YvLs5gMwPQoBSwBPd3c98l+HZ5pYikchleG3qaEkGecQsxvgBsekhzb4bKTvLhWJiRUDauwNbUudRfC0V0hgmhxh0vQyzg490NWr6gLCn4LsEV ur65eb2emahgXt7NpwP5ToS0jDbEJxig7SLZqpxcr9480KkaYLgmc/4G Archived-At: Subject: [Cbor] Preferred encoding of floats X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 23:02:36 -0000 --Apple-Mail=_2296A2ED-0AAD-4C75-9F1F-F853DF272871 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Thinking through the preferred encoding for floats=E2=80=A6 For 0, NaN and +/- Infinity, converting down to half-precision to save = space is seems OK. Conversion between half, single and double-precision = for these is fully deterministic. If these values occur frequently in an = a protocol there will be some space saving. For other than these values, you could have an algorithm for converting = double and single to half where you only do it when the significand bits = that are to be truncated are 0. This would preserve the precision and = save space. Assuming even distribution of values, the odds of this = happening are very low (1 in 8192 when converting single, 23 bits, to = half, 10 bits) and thus the space saving is negligible (0.01%). I think = this is what is meant by the text about floating point in section 3.9 on = canonicalization. Am I right about that?=20 This seems weird, complicating and confusing for very little benefit, = especially for those unfamiliar with float point. The size reduction is = the only benefit and it is negligible for many and probably most use = cases.=20 I thus do not think there should be any conversion of floating point = types for preferred encoding / decoding. That=E2=80=99s why I didn=E2=80=99= t put any in the PR . I=E2=80=99m still thinking about it, but it does seem that we should = require NaN to be always in one particular form and to disallow denormal = numbers o the wire. That will require some active code on the encoding = side. You can=E2=80=99t just take any IEEE float (e.g. the C language = float or double), convert it to network byte order and send it. I = didn=E2=80=99t put any text in the PR for this yet. Last, my view on half-precision is that preferred encoders should not = send it, but preferred decoders should accept it. This just based on = Postel and the fact that half-precision is not widely supported.=20 LL --Apple-Mail=_2296A2ED-0AAD-4C75-9F1F-F853DF272871 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
Thinking through the preferred encoding for = floats=E2=80=A6

For 0, NaN and +/- Infinity, converting down = to half-precision to save space is seems OK. Conversion between half, = single and double-precision for these is fully deterministic. If these = values occur frequently in an a protocol there will be some space = saving.

For other than these values, you could have an algorithm for = converting double and single to half where you only do it when the = significand bits that are to be truncated are 0. This would preserve the = precision and save space.  Assuming even distribution of values, = the odds of this happening are very low (1 in 8192 when converting = single, 23 bits, to half, 10 bits) and thus the space saving is = negligible (0.01%).  I think this is what is meant by the text = about floating point in section 3.9 on canonicalization. Am I right = about that? 

This seems weird, complicating and confusing for very little = benefit, especially for those unfamiliar with float point. The size = reduction is the only benefit and it is negligible for many and probably = most use cases. 

I thus do not think there should be any conversion of = floating point types for preferred encoding / decoding.  That=E2=80=99= s why I didn=E2=80=99t put any in the PR.


I=E2=80=99m still = thinking about it, but it does seem that we should require NaN to be = always in one particular form and to disallow denormal numbers o the = wire. That will require some active code on the encoding side. You = can=E2=80=99t just take any IEEE float (e.g. the C language float or = double), convert it to network byte order and send it.  I didn=E2=80=99= t put any text in the PR for this yet.


Last, my view on half-precision is that preferred encoders = should not send it, but preferred decoders should accept it. This just = based on Postel and the fact that half-precision is not widely = supported. 

LL






= --Apple-Mail=_2296A2ED-0AAD-4C75-9F1F-F853DF272871-- From nobody Wed Aug 1 16:23:18 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DF4CA130E94 for ; Wed, 1 Aug 2018 16:23:15 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.749 X-Spam-Level: X-Spam-Status: No, score=-1.749 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sC5tUmdwqITW for ; Wed, 1 Aug 2018 16:23:14 -0700 (PDT) Received: from mail-wm0-x235.google.com (mail-wm0-x235.google.com [IPv6:2a00:1450:400c:c09::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A5AD1130DC3 for ; Wed, 1 Aug 2018 16:23:14 -0700 (PDT) Received: by mail-wm0-x235.google.com with SMTP id q8-v6so397604wmq.4 for ; Wed, 01 Aug 2018 16:23:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=qPKteVnjTSxaelBAl7827KZENKnrXM45iD35D48GGGI=; b=pO6uKst0yGJtcmeKKVYmhUMz4scFeAqvFsS98OSFGHWbjZPNNYgFQjuSzg/8zUF0sc 4LEztdY3tCcBdCn8lYAL755pC8iYMr6bSDnVAXrGzGz5VCpFVuzFK1NDnoXn7mlpx1Oz D4Nce2EHyee+RX7+kn3quauo0QnR2iI8p7iazzkRxrTvgEgV+PTdebHUwSy8RnVWupnL CkscHet9Qru8BHrkPnf1R1QaqkcGeGA9lonoQtr9XVxNpqn5f/p9yBzw3j6d5XkyqgOE P3aUyPwJiLKPDwqjiNr1LbpspQmutcMjXR8wmD/LkvZE2GPE+asrasG/Z3i8XktX89Tj xzbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=qPKteVnjTSxaelBAl7827KZENKnrXM45iD35D48GGGI=; b=GMf642iZuauGJXIr3mN5Cif5ihwtt8FgY5fyjpiQ+xMY88hGXXJhkIY875oMaSIDj8 r6Dsf2SuThLRPERFRmZ0TsV2rPxc42XOmqG2lWPJgqHvKv4JNY5rUb9dHYVtIAoGuJBX yTusOcPQpJwHdFLLvlXvmffsKTQmQ3vgPevgsShVJZ0sf/f2PL15DRhiYmc9zfyyS2TG gZiBgrnuie3nK8AADYUfen4byAAP0+eBIKugPnMolhb2OFsq403jX7Y99i0cGkLedYC7 vd4AN/qWlYfJMuzVEZevn78tQjpITZyUxQEIvYtt1OYpd9bbxsNOAaocHDdqTHPN2S3m 6sWw== X-Gm-Message-State: AOUpUlFR/xRQr7Mmf1YBsvBtoOimM6Jvx7qGsnQ9G2sIYC3BFBYIR+D7 usm2VJl1XVg4rGHNUiwk1fzdwHW6o7nZBOqFEeZZeP8l X-Google-Smtp-Source: AAOMgpf7j+C2eKRjLxme/TARRAJMfcGu+Q3L1UnrOvlWEBY3p/EK0rDx7ZHR2MCy/Eu7ujpyc6rz4s+14hyfN09AO9k= X-Received: by 2002:a1c:9e4d:: with SMTP id h74-v6mr335992wme.70.1533165792819; Wed, 01 Aug 2018 16:23:12 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a5d:4d92:0:0:0:0:0 with HTTP; Wed, 1 Aug 2018 16:23:12 -0700 (PDT) In-Reply-To: <80DFAEF6-BE25-4F43-9CCF-A8FCBEB1F4BE@island-resort.com> References: <80DFAEF6-BE25-4F43-9CCF-A8FCBEB1F4BE@island-resort.com> From: Sean Solberg Date: Wed, 1 Aug 2018 18:23:12 -0500 Message-ID: To: cbor@ietf.org Content-Type: multipart/alternative; boundary="000000000000767ed4057267fa45" Archived-At: Subject: [Cbor] Fwd: Preferred encoding of floats X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 23:23:16 -0000 --000000000000767ed4057267fa45 Content-Type: text/plain; charset="UTF-8" In my opinion, the CBOR protocol level should never be converting data from one type to another for the sake of being able to represent the same value on the wire more compactly. The receiver may want to distinguish between a particular value that can be represented as either a double, single or half float. A double "0" is not the same "value" as a single "0" because they in fact do have different precision. One of the big benefits (imho) that CBOR has over other mechanisms such as JSON, etc. is that it does transmit explicitly defined data types and the choice for those data types are at the application level, not the protocol/serialization level. Please do not propose any CBOR protocol level conversion of data types for any reason whatsoever, including "more compact on the wire". ..Sean Solberg --000000000000767ed4057267fa45 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
In my opinion, the CBOR protocol level should never be con= verting data from one type to another for the sake of being able to represe= nt the same value on the wire more compactly.=C2=A0 The receiver may want t= o distinguish between a particular value that can be represented as either = a double, single or half float.=C2=A0=20 A double "0" is not th= e same "value" as a single "0" because they in fact do = have different precision.=C2=A0
One of the big benefits=20 (imho)=C2=A0= that CBOR has over other mechanisms such as JSON, etc. is that it does tran= smit explicitly defined data types and the choice for those data types are = at the application level, not the protocol/serialization level.=C2=A0 Pleas= e do not propose any CBOR protocol level conversion of data types for any r= eason whatsoever, including "more compact on the wire".=C2=A0=C2= =A0

..Sean Solberg



<= /div>

--000000000000767ed4057267fa45-- From nobody Wed Aug 1 20:57:06 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A8BA5130E79 for ; Wed, 1 Aug 2018 20:57:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.201 X-Spam-Level: X-Spam-Status: No, score=-4.201 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id R5V4Cw8HcYHB for ; Wed, 1 Aug 2018 20:57:02 -0700 (PDT) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A5F26127598 for ; Wed, 1 Aug 2018 20:57:02 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Aug 2018 20:57:02 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,434,1526367600"; d="scan'208";a="61387771" Received: from orsmsx101.amr.corp.intel.com ([10.22.225.128]) by orsmga007.jf.intel.com with ESMTP; 01 Aug 2018 20:57:02 -0700 Received: from tjmaciei-mobl1.localnet (10.254.78.100) by ORSMSX101.amr.corp.intel.com (10.22.225.128) with Microsoft SMTP Server (TLS) id 14.3.319.2; Wed, 1 Aug 2018 20:57:01 -0700 From: Thiago Macieira To: CC: Sean Solberg Date: Wed, 1 Aug 2018 20:56:20 -0700 Message-ID: <3101844.7X5zPJ143S@tjmaciei-mobl1> Organization: Intel Corporation In-Reply-To: References: <80DFAEF6-BE25-4F43-9CCF-A8FCBEB1F4BE@island-resort.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Originating-IP: [10.254.78.100] Archived-At: Subject: Re: [Cbor] Fwd: Preferred encoding of floats X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 03:57:05 -0000 On Wednesday, 1 August 2018 16:23:12 PDT Sean Solberg wrote: > In my opinion, the CBOR protocol level should never be converting data from > one type to another for the sake of being able to represent the same value > on the wire more compactly. The receiver may want to distinguish between a > particular value that can be represented as either a double, single or half > float. A double "0" is not the same "value" as a single "0" because they > in fact do have different precision. One of the big benefits (imho) that > CBOR has over other mechanisms such as JSON, etc. is that it does transmit > explicitly defined data types and the choice for those data types are at > the application level, not the protocol/serialization level. Please do not > propose any CBOR protocol level conversion of data types for any reason > whatsoever, including "more compact on the wire". I think that boat has sailed. A lot of higher-level API dealing with CBOR does type-erasure on floating points, if not all numbers. Requiring a specific FP type to remain after a round of decoding and re-encoding is not feasible. I would also advise against protocols requiring a specific one of the FP types. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center From nobody Thu Aug 2 14:14:16 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AB4A1130E20 for ; Thu, 2 Aug 2018 14:14:14 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pBa1RBF4PX8P for ; Thu, 2 Aug 2018 14:14:12 -0700 (PDT) Received: from p3plsmtpa06-03.prod.phx3.secureserver.net (p3plsmtpa06-03.prod.phx3.secureserver.net [173.201.192.104]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 717E0130E1E for ; Thu, 2 Aug 2018 14:14:12 -0700 (PDT) Received: from [192.168.1.23] ([73.35.165.162]) by :SMTPAUTH: with ESMTPSA id lKvCfxzXKyz3QlKvDfzboE; Thu, 02 Aug 2018 14:14:11 -0700 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.3 \(3445.6.18\)) From: Laurence Lundblade In-Reply-To: <3101844.7X5zPJ143S@tjmaciei-mobl1> Date: Thu, 2 Aug 2018 14:14:10 -0700 Cc: cbor@ietf.org, Sean Solberg Content-Transfer-Encoding: quoted-printable Message-Id: <3B65AA30-5A89-47DD-8563-57EDA6653BBD@island-resort.com> References: <80DFAEF6-BE25-4F43-9CCF-A8FCBEB1F4BE@island-resort.com> <3101844.7X5zPJ143S@tjmaciei-mobl1> To: Thiago Macieira X-Mailer: Apple Mail (2.3445.6.18) X-CMAE-Envelope: MS4wfK9k4Ymc/lraePH84CQWZpIU2FOiyvl/pF1M/47kzf6bZWKuBI3IoONJFAQpYLV9ZUhCrUbCYsC8Bg/AVwQVoh6qJ0ufY9n57CSWx/iWzatHnoRoQwz2 6ZsJDWgPU5dhKHUgmUMlZcAzLSD9LGxgpCdOoJ6fGJ71g61o5JZ2sbLPjXmCWFgOFp7JPulkpb3jYcnAE5oDI8sPYZv7W0aqHXKhcwaxzV+gaUDBCg9d7pWC Archived-At: Subject: Re: [Cbor] Fwd: Preferred encoding of floats X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 21:14:15 -0000 I think the job of a CBOR library is to convert to/from data in a = particular programing environment from/to the CBOR on-the-wire format. = Different environments will do this differently because they support = different types.=20 I don=E2=80=99t think we want to reduce all CBOR use to the least common = denominator of all programming environments. While some protocols will = prefer this reduction to maximize interop and ease implementation, I = don=E2=80=99t think that is a given or a requirement. If we were to make = it a given or requirement, it would be a sad waste of something already = built into CBOR and that it does well.=20 For example: JSON does a poor job with floating point and CBOR is = commonly converted to it. A custom tweaked compiler and a recent model = CPU (or GPU) can handle half-precision very efficiently in HW. Someone = should be able to built a super efficient protocol with CBOR for the = half-precision environment if they want. They probably can=E2=80=99t = expect it to work that with well JSON, but that is their choice and that = is OK. Sean, I agree that nothing in the CBOR standard should preclude a = protocol design that does no conversion. A comprehensive CBOR library = for the C / C++ / assembly programing environment would allow precise = control of the floating point types encoded and decoded, including = half-precision. In other environments libraries will necessarily be more = crude about this and that is OK too.=20 I=E2=80=99ve seen some use of CBOR for representing data in marshalled = IPC/RPC mechanisms. In these use cases the data never goes on a wire and = never gets close to JSON, Java, Python and such. It just goes between = processes and maybe between locally connected CPUs (a typical SoC in a = smart phone has lots and lots of CPUs of different sizes and types). LL > On Aug 1, 2018, at 8:56 PM, Thiago Macieira = wrote: >=20 > On Wednesday, 1 August 2018 16:23:12 PDT Sean Solberg wrote: >> In my opinion, the CBOR protocol level should never be converting = data from >> one type to another for the sake of being able to represent the same = value >> on the wire more compactly. The receiver may want to distinguish = between a >> particular value that can be represented as either a double, single = or half >> float. A double "0" is not the same "value" as a single "0" because = they >> in fact do have different precision. One of the big benefits (imho) = that >> CBOR has over other mechanisms such as JSON, etc. is that it does = transmit >> explicitly defined data types and the choice for those data types are = at >> the application level, not the protocol/serialization level. Please = do not >> propose any CBOR protocol level conversion of data types for any = reason >> whatsoever, including "more compact on the wire". >=20 > I think that boat has sailed. A lot of higher-level API dealing with = CBOR does=20 > type-erasure on floating points, if not all numbers. Requiring a = specific FP=20 > type to remain after a round of decoding and re-encoding is not = feasible. >=20 > I would also advise against protocols requiring a specific one of the = FP=20 > types. >=20 > --=20 > Thiago Macieira - thiago.macieira (AT) intel.com > Software Architect - Intel Open Source Technology Center >=20 >=20 >=20 > _______________________________________________ > CBOR mailing list > CBOR@ietf.org > https://www.ietf.org/mailman/listinfo/cbor >=20 From nobody Thu Aug 2 15:22:16 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B2AED130E3B for ; Thu, 2 Aug 2018 15:22:13 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.2 X-Spam-Level: X-Spam-Status: No, score=-4.2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Nid9oSi7NPag for ; Thu, 2 Aug 2018 15:22:11 -0700 (PDT) Received: from mailhost.informatik.uni-bremen.de (mailhost.informatik.uni-bremen.de [IPv6:2001:638:708:30c9::12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 32E101286E3 for ; Thu, 2 Aug 2018 15:22:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at informatik.uni-bremen.de Received: from submithost.informatik.uni-bremen.de (submithost.informatik.uni-bremen.de [134.102.201.11]) by mailhost.informatik.uni-bremen.de (8.14.5/8.14.5) with ESMTP id w72MM4wO027786; Fri, 3 Aug 2018 00:22:04 +0200 (CEST) Received: from [192.168.217.114] (p54A6C84F.dip0.t-ipconnect.de [84.166.200.79]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by submithost.informatik.uni-bremen.de (Postfix) with ESMTPSA id 41hPmX3CRgzDXgd; Fri, 3 Aug 2018 00:22:04 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) From: Carsten Bormann In-Reply-To: <3B65AA30-5A89-47DD-8563-57EDA6653BBD@island-resort.com> Date: Fri, 3 Aug 2018 00:22:03 +0200 Cc: Thiago Macieira , cbor@ietf.org, Sean Solberg X-Mao-Original-Outgoing-Id: 554941321.3879091-e4f205a5d9cce5dc29032d6f750a229d Content-Transfer-Encoding: quoted-printable Message-Id: References: <80DFAEF6-BE25-4F43-9CCF-A8FCBEB1F4BE@island-resort.com> <3101844.7X5zPJ143S@tjmaciei-mobl1> <3B65AA30-5A89-47DD-8563-57EDA6653BBD@island-resort.com> To: Laurence Lundblade X-Mailer: Apple Mail (2.3445.9.1) Archived-At: Subject: Re: [Cbor] Fwd: Preferred encoding of floats X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 22:22:14 -0000 Interesting discussion. Hardware support for binary16 (half-precision floating point) would add = very little to the speed of a CBOR encoder/decoder. The argument =E2=80=9C= no support for half-precision=E2=80=9D is a red herring. Because we = expected less people to be familiar with it in 2013 than with single and = double precision, we added code examples in Appendix D of RFC 7049. = (And, BTW, half-precision converts great to JSON decimal, at least if = you think decimal is great in the first place.) Half-precision is = definitely the preferred encoding for numbers that can use it (as is = often the case with sensor readings). On an ARM that does floating = point at all, you have hardware support for half-precision (and C = compiler support in the form of __fp16/_Float16), but none of that is = really needed if you just want to encode/decode CBOR. JSON uses decimal floating point, which requires a relatively expensive = conversion from and to the more common binary floating point numbers = (which also is often done wrong, even to the point that there are = several CVEs describing security vulnerabilities caused by bugs in that = conversion). CBOR offers decimal, too (Tag 4), but its base support is for binary = floating point, in the three most common forms defined in IEEE 754 = (binary16, binary32, binary64). (The larger IEEE 754 types can be = re-arranged into Tag 5 values, if needed, or we could add special tags = that don=E2=80=99t need any rearranging once that becomes a serious = burden for an application.) CBOR=E2=80=99s preference for binary floating point also means that = converting the more common CBOR floating point numbers to JSON is = relatively expensive. This is very different from simply rearranging = the bits as required for converting between the = binary64/binary32/binary16 numbers. My theory is that many people that = use JSON really don=E2=80=99t care about floating point at all, and = those who do, don=E2=80=99t care about that level of efficiency (they = would be using HDF5 otherwise=E2=80=A6). The advantage of an encoder that always uses the shortest form of = floating point is not that a very low fraction of numbers would get = shortened with a truly random distribution. The advantage is that an = application that operates on binary32 or binary16 floating point can = hand these to the encoder without having to make sure they don=E2=80=99t = accidentally are handed over as binary64 somewhere on the way. If your = environment never does that, there is probably little point in checking = whether a shorter form is possible. BTW, there is nothing wrong with denormalized numbers in IEEE 754 = floating point; there is no reason not to send or process them. Gr=C3=BC=C3=9Fe, Carsten From nobody Fri Aug 3 04:27:17 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8F19B130FF1 for ; Fri, 3 Aug 2018 04:27:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jUd6dVffGRFd for ; Fri, 3 Aug 2018 04:27:14 -0700 (PDT) Received: from ppsa-online.com (ppsa-online.com [217.199.162.192]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E1D4A130ED7 for ; Fri, 3 Aug 2018 04:27:13 -0700 (PDT) Received: (qmail 2785 invoked from network); 3 Aug 2018 12:15:52 +0100 Received: from host109-158-44-24.range109-158.btcentralplus.com (HELO ?192.168.1.72?) (109.158.44.24) by lvps217-199-162-217.vps.webfusion.co.uk with ESMTPSA (DHE-RSA-AES128-SHA encrypted, authenticated); 3 Aug 2018 12:15:52 +0100 To: Carsten Bormann Cc: cbor@ietf.org References: <46C63155-C2AB-4DBF-9348-3D42782A452E@tzi.org> From: Pete Cordell Message-ID: <23e77222-917c-46c4-f7cd-98c59a22f514@codalogic.com> Date: Fri, 3 Aug 2018 12:27:09 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <46C63155-C2AB-4DBF-9348-3D42782A452E@tzi.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-GB Content-Transfer-Encoding: 8bit Archived-At: Subject: Re: [Cbor] ABNF in place of regex (Was: Re: CDDL: Solving the regexp issue) X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 11:27:16 -0000 Hi Carsten, Thanks for responding to this - way back! I had started replying earlier, but couldn't quite crystallize any useful thoughts. One thought is that in JCR we have multi-line directives. These are 'global level' constructs. It could be that CDDL2.0 could add something similar. That way you can define a block of ABNF in a CDDL definition and then refer to it. For example, something like: ; A 'multi-line directive' containing a block of ABNF #{abnf three-digits = 1*DIGIT 2*(“.” 1*DIGIT) four-digits = 1*DIGIT 3*(“.” 1*DIGIT) DIGIT = %x30-39 } ; CDDL rules that refer to ABNF rules triple-digit = text .abnf three-digits quad-digit = text .abnf four-digits That way the same ABNF rule (e.g. DIGIT) can be used by multiple CDDL rules without being 'weird'! I think the concept of multi-line global directives (and global directives in general) are a useful concept in their own right, as they provide a convenient "get out of jail free" card. As such they are worth considering for CDDL2.0 independent of the above. Grüße, Pete Cordell Codalogic Ltd C++ tools for C++ programmers, http://codalogic.com Read & write XML in C++, http://www.xml2cpp.com On 17/07/2018 21:54, Carsten Bormann wrote: > Hi Pete, > > Mixing in ABNF into CDDL is still something I’d like to do (for 2.0). > > A simple control like the .abnf you used in March is not enough, since most ABNF right hand sides require one or more rules to become useful. In this case, DIGIT (even if that happens to be a core rule — you still have to provide it). A good place to keep those additional rules hasn’t occurred to me. > > Also, it would be nice for this to have an equivalent of Python’s “”” for this — you used | (which would be quite confusing for people that think in classic BNF). > > Ugly straw man: maybe something like > > triple-digit = text .abnf “”” 1*DIGIT 2*(“.” 1*DIGIT) > DIGIT = %x30-39 > “”” > > (i.e., the first line of the string is used as the RHS for an anonymous ABNF rule, the others provide additional rules.) > > Grüße, Carsten > > >> On Mar 30, 2018, at 08:09, Pete Cordell wrote: >> >> Thinking laterally, if specifying a regex format is problematic, is it worth considering ABNF as an alternative? >> >> The pipe character could act as a suitable delimiter. For example: >> >> triple-digit = tstr .abnf | 1*DIGIT "." 1*DIGIT "." 1*DIGIT | >> >> Over multiple lines it could look like: >> >> triple-digit = tstr .abnf | >> 1*DIGIT "." 1*DIGIT "." 1*DIGIT >> | >> >> The ABNF could be loosely defined; as in: >> >> opague-abnf = *( text / NOT-PIPE ) >> NOT-PIPE = CRLF / %x20-5B / %x7D-10FFFD >> >> It's more verbose. And it's unconventional. But... >> >> Pete. >> -- >> --------------------------------------------------------------------- >> Pete Cordell >> http://json-content-rules.org Rule >> --------------------------------------------------------------------- >> >> _______________________________________________ >> CBOR mailing list >> CBOR@ietf.org >> https://www.ietf.org/mailman/listinfo/cbor >> > > From nobody Fri Aug 3 05:41:19 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 80A3B129619 for ; Fri, 3 Aug 2018 05:41:17 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.748 X-Spam-Level: X-Spam-Status: No, score=-1.748 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jFTYTfbwJss4 for ; Fri, 3 Aug 2018 05:41:15 -0700 (PDT) Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C9E67126F72 for ; Fri, 3 Aug 2018 05:41:14 -0700 (PDT) Received: by mail-wr1-x431.google.com with SMTP id r16-v6so5267558wrt.11 for ; Fri, 03 Aug 2018 05:41:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=iPB9K6SXfjSwVYG5hSCUaZQNTwD5jsj/Pddx9sIKv8E=; b=Xhm/Mc87Z51KJK5cI2pW0Y6vkJcmsR3C8qgEJqqBbJL8X5oaCMlxpyePQeAalRysCH oGK2eABoLxODC+FwcEfUPpuR0RNr8Ra1RZQYri/NIlSDHgN2RSXg0YSLLrwKXgmCPMbR y+ETkHnTUWKKdQ3bAxPaJ1amPFwBmojOShsnSrNjTaWHfCs8kmVmDnWU01noVQX3zh7D 9r6x7E4lboXd9tUVIjrrI1gQrxEF8lR57UHUYBeH22xd/aaTZCpWIRYLGcHKDb+Tf4UD RH1OP5LehDfIxQwLGTCMu/LA59kXPdO7BAFLNM/4flYn7VIyWDQHaRFjsnfB7NEbfH6Y iZwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=iPB9K6SXfjSwVYG5hSCUaZQNTwD5jsj/Pddx9sIKv8E=; b=Q1wKNO/B2BSQdWAXO8di4yF4c5l5hKWVaDODOXzQD/A88Y4fAEz+CyYos71UXcSiX7 T6aWIFt6tNg5aJSWfyLX8gi5xJB1gJGlaoMUO3Ov/IUY9YckD1+M2mn21sgH4oTNk/uG QbfZ2WkfJnKdX5B+8XSNEAG8R8A0+rx/2HdXq6QokLEf2U6p14Fh7J8Iqx1rcN2L1xaV TkvPRNngARQA9+KRxK7v4QFNmaUssOg2uAVWCzZEgqMWUEcP0hS7nmcs6AjpRT8LhFw/ gopmYoyFumdy1YeMwIMmkLhvW91/5H5hfqbkZpR6CXwJs5N/CTS9fh9s52VmRSQgXL76 rinA== X-Gm-Message-State: AOUpUlGuIPr1gvIJNfwFXaMNgg0R8CgvJXEsW5YubdHLftNjneuMue/U PHE+PLCRYctVFjAU6tDSMQjrZnujdmz6jXtBf7HndQ== X-Google-Smtp-Source: AAOMgpfonsfr9VSTacTsC0NudVRbLyje6wbXNXByGC4Kri2lY+95mgEGICTl5D8YAk3MVoOv8/7CKQyhkOsa5REy+gU= X-Received: by 2002:a5d:4acc:: with SMTP id y12-v6mr2334784wrs.132.1533300072908; Fri, 03 Aug 2018 05:41:12 -0700 (PDT) MIME-Version: 1.0 References: <80DFAEF6-BE25-4F43-9CCF-A8FCBEB1F4BE@island-resort.com> <3101844.7X5zPJ143S@tjmaciei-mobl1> <3B65AA30-5A89-47DD-8563-57EDA6653BBD@island-resort.com> In-Reply-To: <3B65AA30-5A89-47DD-8563-57EDA6653BBD@island-resort.com> From: Sean Solberg Date: Fri, 3 Aug 2018 07:41:00 -0500 Message-ID: To: cbor@ietf.org Content-Type: multipart/alternative; boundary="0000000000002e0a820572873e62" Archived-At: Subject: [Cbor] Fwd: Fwd: Preferred encoding of floats X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 12:41:18 -0000 --0000000000002e0a820572873e62 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I just hope there aren't any words in the spec that says it "must" or "should" reduce to the smallest data type that can represent that value. If the spec says that a cbor implementation "may" do that, then great. ..sean ---------- Forwarded message --------- From: Laurence Lundblade Date: Thu, Aug 2, 2018, 4:14 PM Subject: Re: [Cbor] Fwd: Preferred encoding of floats To: Thiago Macieira Cc: , Sean Solberg I think the job of a CBOR library is to convert to/from data in a particular programing environment from/to the CBOR on-the-wire format. Different environments will do this differently because they support different types. I don=E2=80=99t think we want to reduce all CBOR use to the least common denominator of all programming environments. While some protocols will prefer this reduction to maximize interop and ease implementation, I don=E2= =80=99t think that is a given or a requirement. If we were to make it a given or requirement, it would be a sad waste of something already built into CBOR and that it does well. For example: JSON does a poor job with floating point and CBOR is commonly converted to it. A custom tweaked compiler and a recent model CPU (or GPU) can handle half-precision very efficiently in HW. Someone should be able to built a super efficient protocol with CBOR for the half-precision environment if they want. They probably can=E2=80=99t expect it to work tha= t with well JSON, but that is their choice and that is OK. Sean, I agree that nothing in the CBOR standard should preclude a protocol design that does no conversion. A comprehensive CBOR library for the C / C++ / assembly programing environment would allow precise control of the floating point types encoded and decoded, including half-precision. In other environments libraries will necessarily be more crude about this and that is OK too. I=E2=80=99ve seen some use of CBOR for representing data in marshalled IPC/= RPC mechanisms. In these use cases the data never goes on a wire and never gets close to JSON, Java, Python and such. It just goes between processes and maybe between locally connected CPUs (a typical SoC in a smart phone has lots and lots of CPUs of different sizes and types). LL > On Aug 1, 2018, at 8:56 PM, Thiago Macieira wrote: > > On Wednesday, 1 August 2018 16:23:12 PDT Sean Solberg wrote: >> In my opinion, the CBOR protocol level should never be converting data from >> one type to another for the sake of being able to represent the same value >> on the wire more compactly. The receiver may want to distinguish between a >> particular value that can be represented as either a double, single or half >> float. A double "0" is not the same "value" as a single "0" because the= y >> in fact do have different precision. One of the big benefits (imho) that >> CBOR has over other mechanisms such as JSON, etc. is that it does transmit >> explicitly defined data types and the choice for those data types are at >> the application level, not the protocol/serialization level. Please do not >> propose any CBOR protocol level conversion of data types for any reason >> whatsoever, including "more compact on the wire". > > I think that boat has sailed. A lot of higher-level API dealing with CBOR does > type-erasure on floating points, if not all numbers. Requiring a specific FP > type to remain after a round of decoding and re-encoding is not feasible. > > I would also advise against protocols requiring a specific one of the FP > types. > > -- > Thiago Macieira - thiago.macieira (AT) intel.com > Software Architect - Intel Open Source Technology Center > > > > _______________________________________________ > CBOR mailing list > CBOR@ietf.org > https://www.ietf.org/mailman/listinfo/cbor > --0000000000002e0a820572873e62 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I just hope there aren't any words in the spec that s= ays it "must" or "should" reduce to the smallest data t= ype that can represent that value.=C2=A0 If the spec says that a cbor imple= mentation "may" do that, then great.

<= div dir=3D"auto">..sean

---------- Forwarded message ---------
From: Laurence Lundblade <lgl@island-resort.com>=
Date: Thu, Aug 2, 2018, 4:14 PM
Subject: Re: [Cbor] Fwd: Pre= ferred encoding of floats
To: Thiago Macieira <thiago.macieira@intel.com>
Cc: <cbor@ietf.org>, Sean Solberg <ssolbs99@gmail.com>


= I think the job of a CBOR library is to convert to/from data in a particula= r programing environment from/to the CBOR on-the-wire format. Different env= ironments will do this differently because they support different types.
I don=E2=80=99t think we want to reduce all CBOR use to the least common de= nominator of all programming environments. While some protocols will prefer= this reduction to maximize interop and ease implementation, I don=E2=80=99= t think that is a given or a requirement. If we were to make it a given or = requirement, it would be a sad waste of something already built into CBOR a= nd that it does well.

For example: JSON does a poor job with floating point and CBOR is commonly = converted to it. A custom tweaked compiler and a recent model CPU (or GPU) = can handle half-precision very efficiently in HW. Someone should be able to= built a super efficient protocol with CBOR for the half-precision environm= ent if they want. They probably can=E2=80=99t expect it to work that with w= ell JSON, but that is their choice and that is OK.

Sean, I agree that nothing in the CBOR standard should preclude a protocol = design that does no conversion. A comprehensive CBOR library for the C / C+= + / assembly programing environment would allow precise control of the floa= ting point types encoded and decoded, including half-precision. In other en= vironments libraries will necessarily be more crude about this and that is = OK too.

I=E2=80=99ve seen some use of CBOR for representing data in marshalled IPC/= RPC mechanisms. In these use cases the data never goes on a wire and never = gets close to JSON, Java, Python and such. It just goes between processes a= nd maybe between locally connected CPUs (a typical SoC in a smart phone has= lots and lots of CPUs of different sizes and types).

LL







> On Aug 1, 2018, at 8:56 PM, Thiago Macieira <thiago.macieira= @intel.com> wrote:
>
> On Wednesday, 1 August 2018 16:23:12 PDT Sean Solberg wrote:
>> In my opinion, the CBOR protocol level should never be converting = data from
>> one type to another for the sake of being able to represent the sa= me value
>> on the wire more compactly.=C2=A0 The receiver may want to disting= uish between a
>> particular value that can be represented as either a double, singl= e or half
>> float.=C2=A0 A double "0" is not the same "value&qu= ot; as a single "0" because they
>> in fact do have different precision. One of the big benefits (imho= ) that
>> CBOR has over other mechanisms such as JSON, etc. is that it does = transmit
>> explicitly defined data types and the choice for those data types = are at
>> the application level, not the protocol/serialization level.=C2=A0= Please do not
>> propose any CBOR protocol level conversion of data types for any r= eason
>> whatsoever, including "more compact on the wire".
>
> I think that boat has sailed. A lot of higher-level API dealing with C= BOR does
> type-erasure on floating points, if not all numbers. Requiring a speci= fic FP
> type to remain after a round of decoding and re-encoding is not feasib= le.
>
> I would also advise against protocols requiring a specific one of the = FP
> types.
>
> --
> Thiago Macieira - thiago.macieira (AT) intel.com
>=C2=A0 Software Architect - Intel Open Source Technology Center
>
>
>
> _______________________________________________
> CBOR mailing list
> = CBOR@ietf.org
> https://www.ietf.org/mailman/listinfo/cbor=
>

--0000000000002e0a820572873e62-- From nobody Fri Aug 3 06:01:16 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2E05C130E16 for ; Fri, 3 Aug 2018 06:01:14 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.2 X-Spam-Level: X-Spam-Status: No, score=-4.2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eTs5qjraFqE7 for ; Fri, 3 Aug 2018 06:01:12 -0700 (PDT) Received: from mailhost.informatik.uni-bremen.de (mailhost.informatik.uni-bremen.de [IPv6:2001:638:708:30c9::12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BABFE128CE4 for ; Fri, 3 Aug 2018 06:01:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at informatik.uni-bremen.de Received: from submithost.informatik.uni-bremen.de (submithost.informatik.uni-bremen.de [134.102.201.11]) by mailhost.informatik.uni-bremen.de (8.14.5/8.14.5) with ESMTP id w73D18c4007770; Fri, 3 Aug 2018 15:01:08 +0200 (CEST) Received: from [192.168.217.114] (p54A6C84F.dip0.t-ipconnect.de [84.166.200.79]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by submithost.informatik.uni-bremen.de (Postfix) with ESMTPSA id 41hnGr4RwDzDXpW; Fri, 3 Aug 2018 15:01:08 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) From: Carsten Bormann In-Reply-To: Date: Fri, 3 Aug 2018 15:01:08 +0200 Cc: cbor@ietf.org X-Mao-Original-Outgoing-Id: 554994065.974257-659efc75549dd2feac169081bccde581 Content-Transfer-Encoding: quoted-printable Message-Id: <4FE5660E-0E53-4445-9863-8016454AEBF6@tzi.org> References: <80DFAEF6-BE25-4F43-9CCF-A8FCBEB1F4BE@island-resort.com> <3101844.7X5zPJ143S@tjmaciei-mobl1> <3B65AA30-5A89-47DD-8563-57EDA6653BBD@island-resort.com> To: Sean Solberg X-Mailer: Apple Mail (2.3445.9.1) Archived-At: Subject: Re: [Cbor] Fwd: Fwd: Preferred encoding of floats X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 13:01:14 -0000 On Aug 3, 2018, at 14:41, Sean Solberg wrote: >=20 > I just hope there aren't any words in the spec that says it "must" or = "should" reduce to the smallest data type that can represent that value. = If the spec says that a cbor implementation "may" do that, then great. It=E2=80=99s certainly a MAY (RFC 2119 sense of MAY: the peer has to be = prepared to encounter it). It is definitely not a SHOULD or a MUST. The term =E2=80=9Cpreferred encoding=E2=80=9D in any case is more of a = =E2=80=9Cquality of implementation expectation=E2=80=9D, which is = somewhat orthogonal to the =E2=80=9Cinteroperability considerations=E2=80=9D= that require RFC 2119 wording. Gr=C3=BC=C3=9Fe, Carsten From nobody Sun Aug 5 13:21:46 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EC15A130DC5 for ; Sun, 5 Aug 2018 13:21:44 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.898 X-Spam-Level: X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TZRoOhlxMYUq for ; Sun, 5 Aug 2018 13:21:42 -0700 (PDT) Received: from p3plsmtpa09-10.prod.phx3.secureserver.net (p3plsmtpa09-10.prod.phx3.secureserver.net [173.201.193.239]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 87ADE129385 for ; Sun, 5 Aug 2018 13:21:42 -0700 (PDT) Received: from [10.12.102.53] ([66.228.17.5]) by :SMTPAUTH: with ESMTPSA id mPX2frRo9itREmPX2fCNk7; Sun, 05 Aug 2018 13:21:41 -0700 From: Laurence Lundblade Message-Id: <46A693AD-5AF4-4A84-8A43-3575D10F705E@island-resort.com> Content-Type: multipart/alternative; boundary="Apple-Mail=_746D6437-9A4D-4B61-B4FF-75A8AB663846" Mime-Version: 1.0 (Mac OS X Mail 11.3 \(3445.6.18\)) Date: Sun, 5 Aug 2018 13:21:40 -0700 In-Reply-To: <4FE5660E-0E53-4445-9863-8016454AEBF6@tzi.org> Cc: Sean Solberg , cbor@ietf.org To: Carsten Bormann References: <80DFAEF6-BE25-4F43-9CCF-A8FCBEB1F4BE@island-resort.com> <3101844.7X5zPJ143S@tjmaciei-mobl1> <3B65AA30-5A89-47DD-8563-57EDA6653BBD@island-resort.com> <4FE5660E-0E53-4445-9863-8016454AEBF6@tzi.org> X-Mailer: Apple Mail (2.3445.6.18) X-CMAE-Envelope: MS4wfKY7Kpo6rO1Qogd5dVif/TzPn+RydnczPNZc7qnM0mKibJ3XThpjC/kKwx4ze3zj2N9ncWaGLxDwBm+mZc0iwySruOJ7zWTnvtDBKGNxbnlnJHuOQi1D ORuiwFuykpNKICWAdyLkILLAvx21O+VRKd5KFBRTsXhu878AUD2mg8opZjUU/FERPRo6QnJHnmq1QmooAdziDSEqgg1MufJ+Kio= Archived-At: Subject: Re: [Cbor] Fwd: Fwd: Preferred encoding of floats X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Aug 2018 20:21:45 -0000 --Apple-Mail=_746D6437-9A4D-4B61-B4FF-75A8AB663846 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 My comments below > On Aug 3, 2018, at 6:01 AM, Carsten Bormann wrote: >=20 > On Aug 3, 2018, at 14:41, Sean Solberg wrote: >>=20 >> I just hope there aren't any words in the spec that says it "must" or = "should" reduce to the smallest data type that can represent that value. = If the spec says that a cbor implementation "may" do that, then great. >=20 > It=E2=80=99s certainly a MAY (RFC 2119 sense of MAY: the peer has to = be prepared to encounter it). >=20 > It is definitely not a SHOULD or a MUST. >=20 > The term =E2=80=9Cpreferred encoding=E2=80=9D in any case is more of a = =E2=80=9Cquality of implementation expectation=E2=80=9D, which is = somewhat orthogonal to the =E2=80=9Cinteroperability considerations=E2=80=9D= that require RFC 2119 wording. I don=E2=80=99t think IETF does =E2=80=9Cquality of implementation=E2=80=9D= . I don=E2=80=99t think it should. Formally, that is the realm of = certifications like Common Criteria and FIPS. Informally, we all want to = do a good job of our code, have code reviews, fuzz test and such, but = none of that goes in an RFC. I also don=E2=80=99t think any particular feature set or mode of CBOR = should be considered necessary for higher quality implementation, to = defend against buffer overruns or general SW security. In this day and = age all features and options in CBOR should be implementable in equally = and very secure ways. So I think preferred encoding / decoding is only about interoperability. = Interoperability is the focus of this pull request = on preferred encoding / = decoding. (I think we=E2=80=99ve determined canonicalization for the = sake or repeatable deterministic encoding is a lesser focus).=20 For interop for half, float and double, the main thing I think is = important is that preferred decoders support all three. They are liberal = in what they accept. Some implementations, those on small CPUs without = any floating point support, will not be able to implement this and thus = can=E2=80=99t be preferred decoders. You have to support float to be a = preferred decoder.=20 I=E2=80=99m still trying to get my head fully around what to do with = denormal and varying NaNs for interop.=20 It seems like preferred encoding should require the canonical form for = NaN, but I=E2=80=99m not sure if it should support both sNaN and qNaN, = or just one NaN. Denormal seems to have historical controversy. HW / compilers have = options to turn it on and off. Because of this, I=E2=80=99d say suggest = encoding should require denormal-as-zero. It=E2=80=99s tempting to say = that it=E2=80=99s not a big deal if a decoder folds denormal it receives = to zero, but maybe the sender didn=E2=80=99t want that. It would = certainly not be OK to fold small integer values 0x01, 0x02, 0x03 to = 0x00. LL --Apple-Mail=_746D6437-9A4D-4B61-B4FF-75A8AB663846 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 My = comments below

On Aug 3, 2018, at 6:01 AM, Carsten Bormann = <cabo@tzi.org> = wrote:

On Aug 3, 2018, at 14:41, Sean Solberg <ssolbs99@gmail.com> = wrote:

I= just hope there aren't any words in the spec that says it "must" or = "should" reduce to the smallest data type that can represent that value. =  If the spec says that a cbor implementation "may" do that, then = great.

It=E2=80=99s certainly = a MAY (RFC 2119 sense of MAY: the peer has to be prepared to encounter = it).

It is definitely not a SHOULD or a = MUST.

The term =E2=80=9Cpreferred = encoding=E2=80=9D in any case is more of a =E2=80=9Cquality of = implementation expectation=E2=80=9D, which is somewhat orthogonal to the = =E2=80=9Cinteroperability considerations=E2=80=9D that require RFC 2119 = wording.

I don=E2=80=99t think IETF does =E2=80=9Cquality = of implementation=E2=80=9D. I don=E2=80=99t think it should. Formally, = that is the realm of certifications like Common Criteria and FIPS. = Informally, we all want to do a good job of our code, have code reviews, = fuzz test and such, but none of that goes in an RFC.

I also don=E2=80=99t think any particular feature = set or mode of CBOR should be considered necessary for higher quality = implementation, to defend against buffer overruns or general SW = security. In this day and age all features and options in CBOR should be = implementable in equally and very secure ways.

So I think preferred encoding / decoding is only = about interoperability. Interoperability is the focus of this pull = request on preferred encoding / decoding.  (I think = we=E2=80=99ve determined canonicalization for the sake or repeatable = deterministic encoding is a lesser focus). 

For interop for half, float and double, the main = thing I think is important is that preferred decoders support all three. = They are liberal in what they accept. Some implementations, those on = small CPUs without any floating point support, will not be able to = implement this and thus can=E2=80=99t be preferred decoders. You have to = support float to be a preferred decoder. 

I=E2=80=99m still trying to get my head fully = around what to do with denormal and varying NaNs for = interop. 

It seems like = preferred encoding should require the canonical form for NaN, but I=E2=80=99= m not sure if it should support both sNaN and qNaN, or just one = NaN.

Denormal seems to have = historical controversy. HW / compilers have options to turn it on and = off. Because of this, I=E2=80=99d say suggest encoding should require = denormal-as-zero.  It=E2=80=99s tempting to say that it=E2=80=99s = not a big deal if a decoder folds denormal it receives to zero, but = maybe the sender didn=E2=80=99t want that. It would certainly not be OK = to fold small integer values 0x01, 0x02, 0x03 to 0x00.

LL











= --Apple-Mail=_746D6437-9A4D-4B61-B4FF-75A8AB663846-- From nobody Mon Aug 6 01:21:26 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1F87B130DFF for ; Mon, 6 Aug 2018 01:21:25 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.2 X-Spam-Level: X-Spam-Status: No, score=-4.2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RFT66Bujfku3 for ; Mon, 6 Aug 2018 01:21:20 -0700 (PDT) Received: from mailhost.informatik.uni-bremen.de (mailhost.informatik.uni-bremen.de [IPv6:2001:638:708:30c9::12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3AFC9127598 for ; Mon, 6 Aug 2018 01:21:20 -0700 (PDT) X-Virus-Scanned: amavisd-new at informatik.uni-bremen.de Received: from submithost.informatik.uni-bremen.de (submithost.informatik.uni-bremen.de [IPv6:2001:638:708:30c9::b]) by mailhost.informatik.uni-bremen.de (8.14.5/8.14.5) with ESMTP id w768LFeU028298; Mon, 6 Aug 2018 10:21:15 +0200 (CEST) Received: from [192.168.217.102] (p54A6C84F.dip0.t-ipconnect.de [84.166.200.79]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by submithost.informatik.uni-bremen.de (Postfix) with ESMTPSA id 41kVwW34TfzDWjR; Mon, 6 Aug 2018 10:21:15 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) From: Carsten Bormann In-Reply-To: <46A693AD-5AF4-4A84-8A43-3575D10F705E@island-resort.com> Date: Mon, 6 Aug 2018 10:21:14 +0200 Cc: Sean Solberg , cbor@ietf.org X-Mao-Original-Outgoing-Id: 555236472.447896-ce20f482c4cbd1b525265199f5dd8abe Content-Transfer-Encoding: quoted-printable Message-Id: References: <80DFAEF6-BE25-4F43-9CCF-A8FCBEB1F4BE@island-resort.com> <3101844.7X5zPJ143S@tjmaciei-mobl1> <3B65AA30-5A89-47DD-8563-57EDA6653BBD@island-resort.com> <4FE5660E-0E53-4445-9863-8016454AEBF6@tzi.org> <46A693AD-5AF4-4A84-8A43-3575D10F705E@island-resort.com> To: Laurence Lundblade X-Mailer: Apple Mail (2.3445.9.1) Archived-At: Subject: Re: [Cbor] Fwd: Preferred encoding of floats X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Aug 2018 08:21:25 -0000 >> The term =E2=80=9Cpreferred encoding=E2=80=9D in any case is more of = a =E2=80=9Cquality of implementation expectation=E2=80=9D, which is = somewhat orthogonal to the =E2=80=9Cinteroperability considerations=E2=80=9D= that require RFC 2119 wording. >=20 > I don=E2=80=99t think IETF does =E2=80=9Cquality of implementation=E2=80= =9D. You may be surprised how often it does: = https://www.google.de/search?q=3D%22quality+of+implementation%22+site%3Arf= c-editor.org (These are just the RFCs that use that exact phrase, but there are tons = other that provide guidance of a similar kind using different words.) > I don=E2=80=99t think it should. I=E2=80=99d say, we are doing it much less than we actually should. =20 We are defining protocols to create benefit. This includes giving = implementers guidance what to implement. Beyond what is absolutely = needed for interoperability per se, there may also guidance on what = implementation decisions may further increase benefit. > Formally, that is the realm of certifications like Common Criteria and = FIPS. Informally, we all want to do a good job of our code, have code = reviews, fuzz test and such, but none of that goes in an RFC. Neither certification nor code review do. But the above mentioned = =E2=80=9Cquality of implementation=E2=80=9D guidance does. Implementers are often looking for a cop-out to do something the easy = way (and, I=E2=80=99d say, legitimately so!). Allowing some undesirable behavior in an RFC (just because it is not = fatal for interoperability) may serve as such a cop-out. That=E2=80=99s = why additional =E2=80=9Cquality of implementation=E2=80=9D guidance is = so useful. (Or implementers may simply not be aware of the benefits of = achieving some quality of implementation objective.) > I also don=E2=80=99t think any particular feature set or mode of CBOR = should be considered necessary for higher quality implementation, to = defend against buffer overruns or general SW security. In this day and = age all features and options in CBOR should be implementable in equally = and very secure ways. I=E2=80=99m not sure where security comes in here, but I agree CBOR is = implementable. =20 But it sure helps new implementers to know what is a good way to do = things. > So I think preferred encoding / decoding is only about = interoperability. As I said above, interoperability is an important benefit, but this is = about overall benefit, not just interoperability. > Interoperability is the focus of this pull request on preferred = encoding / decoding. (I think we=E2=80=99ve determined canonicalization = for the sake or repeatable deterministic encoding is a lesser focus).=20 Different focus. (Whether it=E2=80=99s lesser is dependent on what the = application needs.) > For interop for half, float and double, the main thing I think is = important is that preferred decoders support all three. They are liberal = in what they accept. Some implementations, those on small CPUs without = any floating point support, will not be able to implement this and thus = can=E2=80=99t be preferred decoders. You have to support float to be a = preferred decoder.=20 I don=E2=80=99t think we want to have =E2=80=9Cpreferred decoders=E2=80=9D= and =E2=80=9Cnon-preferred decoders=E2=80=9D. A decoder implements all = of CBOR or an application-specific subset. =E2=80=9CAll of CBOR=E2=80=9D = is certainly preferred for what tries to be a generic decoder, but that = is so obvious that it probably does not need to be stated. =E2=80=9CPreferred=E2=80=9D mostly comes in where there are *encoding* = choices that are all =E2=80=9Cvalid=E2=80=9D, but some of which are = better than others. Talking about quality of implementation in a = specification may help an implementer to make the right decisions here. = Generally, it is good for an ecosystem if the non-essential decisions = mostly go into the same direction, e.g., because decoder implementers = can then optimize for that. > I=E2=80=99m still trying to get my head fully around what to do with = denormal and varying NaNs for interop.=20 As I said, denormal is a normal thing in IEEE 754 (called subnormal = there). Varying NaNs are not disallowed by CBOR; I don=E2=80=99t think there is = a good expectation for interoperability with =E2=80=9CNaN boxing=E2=80=9D = or similar approaches, though. > It seems like preferred encoding should require the canonical form for = NaN, but I=E2=80=99m not sure if it should support both sNaN and qNaN, = or just one NaN. The current text for recommended canonical, maps all NaNs to a single = (quiet) NaN. That may indeed not be the best way for all applications. > Denormal seems to have historical controversy. Yes, at the time probably the biggest controversy around IEEE 754. = Settled some 30 years ago=E2=80=A6 > HW / compilers have options to turn it on and off. And that is fine. But now you are into application processing of data = *before* they go into interchange. (BTW, CDDL does not have a way to say that a number may not be = subnormal. =20 Could be added easily as a control.) > Because of this, I=E2=80=99d say suggest encoding should require = denormal-as-zero. =20 Absolutely not. Subnormal is needed for what is called =E2=80=9Cgradual = underflow=E2=80=9D, so normally you *want* to transfer them unharmed. = There also is no point in loading every single encoder with = considerations about what limitations the floating point environment of = the decoder might have. (Just because a receiver *might* have some = undesirable handling of certain numbers should not mean you should = *force* that undesirable behavior on every sender. Unless, of course, = the specific application wants it, but then it=E2=80=99s an application = thing.) > It=E2=80=99s tempting to say that it=E2=80=99s not a big deal if a = decoder folds denormal it receives to zero, but maybe the sender = didn=E2=80=99t want that. A receiving software environment may have all kinds of weird properties = the sender did not want. If a sender wants receivers with both subnormal-challenged and full IEEE = 754 implementations to operate on the data in the same way [we are in a = bit of canonical land here, again], it will need to do flush-to-zero in = the application. But there is no need to burden the CBOR environment = with this. (There is nothing to stop a CBOR library implementer from = providing flush-to-zero as an encoder option for the convenience of the = application. But I wouldn=E2=80=99t expect an encoder to provide that = =E2=80=94 it=E2=80=99s a nice feature for some applications, but it is = not =E2=80=9Cpreferred=E2=80=9D.) > It would certainly not be OK to fold small integer values 0x01, 0x02, = 0x03 to 0x00. Sure, but integers are exact numbers, so the comparison is not very = useful for inexact numbers. Gr=C3=BC=C3=9Fe, Carsten From nobody Tue Aug 7 10:00:54 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 916B012777C; Tue, 7 Aug 2018 10:00:51 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.799 X-Spam-Level: X-Spam-Status: No, score=0.799 tagged_above=-999 required=5 tests=[BAYES_50=0.8, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id E5N0TlwlnBmy; Tue, 7 Aug 2018 10:00:50 -0700 (PDT) Received: from mail2.augustcellars.com (augustcellars.com [50.45.239.150]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C611212F1A5; Tue, 7 Aug 2018 10:00:49 -0700 (PDT) Received: from Jude (73.180.8.170) by mail2.augustcellars.com (192.168.0.56) with Microsoft SMTP Server (TLS) id 15.0.1347.2; Tue, 7 Aug 2018 09:57:03 -0700 From: Jim Schaad To: CC: Date: Tue, 7 Aug 2018 10:00:41 -0700 Message-ID: <006c01d42e70$2dbf65c0$893e3140$@augustcellars.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 16.0 Thread-Index: AdQucCNY6kXpy38DRUCCxyRh6hjQrw== Content-Language: en-us X-Originating-IP: [73.180.8.170] Archived-At: Subject: [Cbor] Agenda for tomorrow X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Aug 2018 17:00:52 -0000 Is there an agenda for the call tomorrow? Jim From nobody Tue Aug 7 10:45:30 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 10B1F130F25 for ; Tue, 7 Aug 2018 10:45:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.896 X-Spam-Level: X-Spam-Status: No, score=-1.896 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.001, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MFoDG1AYtV6x for ; Tue, 7 Aug 2018 10:45:25 -0700 (PDT) Received: from mail-io0-x229.google.com (mail-io0-x229.google.com [IPv6:2607:f8b0:4001:c06::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 52A70130E0F for ; Tue, 7 Aug 2018 10:45:25 -0700 (PDT) Received: by mail-io0-x229.google.com with SMTP id z20-v6so14742090iol.0 for ; Tue, 07 Aug 2018 10:45:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:from:date:message-id:subject:to; bh=8LryF+tHhKxgjnQpd1hLmsv28p/kChrv4FdO96GtONA=; b=koUzfz+e07wA9WX6ur0ddHYVOtTDBGS2RDMGSy3huDnaf8wwmHc2OoeZ6xlTUI3Krx yoAyo0HfmlmHFRK/kTRSnmOR1CMc4m1cZ++YbD7NITHl1LgBEcV/doRuS7/+bcc1NsQ5 y+FKq+vJE1A90Yjyo36348x1JzwqymhzV4m+D+0Np6SApi9biEUFbHlRhY41SIJMdJ0+ uPqiaLx82Vn1u8lP8a6wGHXtb9sNXUebqs9Qp95N87kSyR96pUlQ41FLqfLvlY7kOqnH bz3/bfcYFnpYdcbNH4khvpzynDWCESWp80367Ic9iRAAjGzJAdQFb66HQl21Vdf69mbR PK7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:from:date:message-id:subject :to; bh=8LryF+tHhKxgjnQpd1hLmsv28p/kChrv4FdO96GtONA=; b=RDZRq2a13MzGPApCAC8PT75pBYVPYHh3x39NQHMazEEO/Zz3ROsnZSR2o/K1aR2FGN KcxMmHlsKUyq5R+B6hw48SNr7bWN2/dHhilj774OgVZ2g9LK2pnf0mtSEa4UFwKmFDur s1mLKJGNYEg0Fg+S7qCMWUONCX12hASWqcWdOEDRpwlO8Udvzjtl9jl1i58wSlgNzKVH dfZrYMH8wCQvUNjyGrrLRvQmC4N15waQmD/GJkv9rpojJtAkaFaubq4qSTzP+e5ZP7Yo C0rx887DrF7umGVm0qLAa74eyqhnYci9XhKUTflHuanjuVLg9lEV77nU04uT3bO3Nrh/ U6ZQ== X-Gm-Message-State: AOUpUlHVhN80gSmnUhBf1tpC8UfaoeWNkPr3WXaSHFD6I0ld+wXEd25h /Xm9fCfCmqc0hQbHjJJs9v7KWKsBo+fzAPUDVgfV0g== X-Google-Smtp-Source: AA+uWPykf4tKZsFHHgTa1WWXROhy+Sh8ay+5dCG4NrPX0B8v695V2Jh8uaA6/+SYGUwt3PtVgnrbBdVvasfTTrtKWfs= X-Received: by 2002:a6b:9bd1:: with SMTP id d200-v6mr21132187ioe.147.1533663924257; Tue, 07 Aug 2018 10:45:24 -0700 (PDT) MIME-Version: 1.0 Sender: barryleiba@gmail.com Received: by 2002:ac0:8e88:0:0:0:0:0 with HTTP; Tue, 7 Aug 2018 10:45:23 -0700 (PDT) From: Barry Leiba Date: Tue, 7 Aug 2018 13:45:23 -0400 X-Google-Sender-Auth: Q1MusZ3e7Y2EJTCL5og_s3cr4Ks Message-ID: To: cbor@ietf.org Content-Type: multipart/mixed; boundary="00000000000069b0b00572dbf5af" Archived-At: Subject: [Cbor] Webex meeting invitation: CBOR working group conference call X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Aug 2018 17:45:28 -0000 --00000000000069b0b00572dbf5af Content-Type: multipart/alternative; boundary="00000000000069b0ad0572dbf5ad" --00000000000069b0ad0572dbf5ad Content-Type: text/plain; charset="UTF-8" Here is the WebEx information for the CBOR conference calls. There's an ICS attachment here, or you can click the "Add to Calendar" button to get the standard WebEx one. The calls will repeat every two weeks, and the purpose of the calls is to check the progress of the work on the CDDL and CBOR-bis documents, and to keep it moving forward. REMINDER: The first call is on Wednesday, 8 August, at 15:00 UTC. That's "tomorrow" as I write this. (Note that there is a meeting password, "constrained"; you're likely not to need it, but there it is in case you do. I wanted to just make it "ietf", but the WebEx system wouldn't let me.) *CBOR working group conference call* Every 2 weeks on Wednesday, from Wednesday, August 8, 2018, to Wednesday, October 31, 2018 11:00 am | Eastern Daylight Time (New York, GMT-04:00) | 1 hr Meeting number (access code): 311 714 693 Meeting password: constrained Add to Calendar When it's time, join the meeting . *Join by phone* *1-650-479-3208* Call-in toll number (US/Canada) Can't join the meeting? IMPORTANT NOTICE: Please note that this WebEx service allows audio and other information sent during the session to be recorded, which may be discoverable in a legal matter. By joining this session, you automatically consent to such recordings. If you do not consent to being recorded, discuss your concerns with the host or do not join the session. --00000000000069b0ad0572dbf5ad Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Here is the WebEx information for the CBOR conference= calls.=C2=A0 There's an ICS attachment here, or you can click the &quo= t;Add to Calendar" button to get the standard WebEx one.
The calls will repeat every two weeks, and the purpose of the c= alls is to check the progress of the work on the CDDL and CBOR-bis document= s, and to keep it moving forward.

REMINDER: The fi= rst call is on Wednesday, 8 August, at 15:00 UTC.
That's &quo= t;tomorrow" as I write this.=C2=A0 (Note that there is a meeting passw= ord, "constrained"; you're likely not to need it, but there i= t is in case you do.=C2=A0 I wanted to just make it "ietf", but t= he WebEx system wouldn't let me.)

CBOR wor= king group conference call
Every 2 weeks on Wednesday, from Wednesday, August 8, 2018, to = Wednesday, October 31, 2018
11:00 am=C2=A0=C2=A0|=C2=A0=C2=A0Eastern Daylight Time (New Yor= k, GMT-04:00)=C2=A0=C2=A0|=C2=A0=C2=A01 hr
Meeting number (access code): 311 714 693
=09
Meeting password: constrained

=09
= =C2=A0
Add to Calendar
When it's time, join the meeting.
= =C2=A0
=09
Join by phone
1-650-479-3208=C2=A0Call-in toll num= ber (US/Canada)
=C2= =A0
Can't join the meeting?
=C2= =A0
IMPORTANT NOTICE: Please note that this WebEx service allows audio= and other information sent during the session to be recorded, which may be= discoverable in a legal matter. By joining this session, you automatically= consent to such recordings. If you do not consent to being recorded, discu= ss your concerns with the host or do not join the session.



--00000000000069b0ad0572dbf5ad-- --00000000000069b0b00572dbf5af Content-Type: text/calendar; charset="US-ASCII"; name="CBOR working group conference call.ics" Content-Disposition: attachment; filename="CBOR working group conference call.ics" Content-Transfer-Encoding: base64 X-Attachment-Id: f_jkjzvcr80 QkVHSU46VkNBTEVOREFSClBST0RJRDotLy9NaWNyb3NvZnQgQ29ycG9yYXRpb24vL091dGxvb2sg MTAuMCBNSU1FRElSLy9FTgpWRVJTSU9OOjIuMApNRVRIT0Q6UFVCTElTSApCRUdJTjpWVElNRVpP TkUKVFpJRDpFYXN0ZXJuIFRpbWUKQkVHSU46U1RBTkRBUkQKRFRTVEFSVDoyMDE2MTEwMVQwMjAw MDAKUlJVTEU6RlJFUT1ZRUFSTFk7SU5URVJWQUw9MTtCWURBWT0xU1U7QllNT05USD0xMQpUWk9G RlNFVEZST006LTA0MDAKVFpPRkZTRVRUTzotMDUwMApUWk5BTUU6U3RhbmRhcmQgVGltZQpFTkQ6 U1RBTkRBUkQKQkVHSU46REFZTElHSFQKRFRTVEFSVDoyMDE2MDMwMVQwMjAwMDAKUlJVTEU6RlJF UT1ZRUFSTFk7SU5URVJWQUw9MTtCWURBWT0yU1U7QllNT05USD0zClRaT0ZGU0VURlJPTTotMDUw MApUWk9GRlNFVFRPOi0wNDAwClRaTkFNRTpEYXlsaWdodCBTYXZpbmdzIFRpbWUKRU5EOkRBWUxJ R0hUCkVORDpWVElNRVpPTkUKQkVHSU46VkVWRU5UCkNMQVNTOlBVQkxJQwpERVNDUklQVElPTjpc blxuSk9JTiBXRUJFWCBNRUVUSU5HXG5odHRwczovL2lldGYud2ViZXguY29tL2lldGYvai5waHA/ TVRJRD1tMmE4ODIxZjg0YmQ4YzU2MTg5YmU4MWYwMTIyN2U1OWZcbk1lZXRpbmcgbnVtYmVyIChh Y2Nlc3MgY29kZSk6IDMxMSA3MTQgNjkzXG5Ib3N0IGtleTogMjA1Mjc2XG5NZWV0aW5nIHBhc3N3 b3JkOiBjb25zdHJhaW5lZFxuXG5cblxuSk9JTiBCWSBQSE9ORVxuMS02NTAtNDc5LTMyMDggQ2Fs bC1pbiB0b2xsIG51bWJlciAoVVMvQ2FuYWRhKVxuXG5cblxuQ2FuJ3Qgam9pbiB0aGUgbWVldGlu Zz8gQ29udGFjdCBzdXBwb3J0IGhlcmU6XG5odHRwczovL2lldGYud2ViZXguY29tL2lldGYvbWNc blxuXG5JTVBPUlRBTlQgTk9USUNFOiBQbGVhc2Ugbm90ZSB0aGF0IHRoaXMgV2ViRXggc2Vydmlj ZSBhbGxvd3MgYXVkaW8gYW5kIG90aGVyIGluZm9ybWF0aW9uIHNlbnQgZHVyaW5nIHRoZSBzZXNz aW9uIHRvIGJlIHJlY29yZGVkLCB3aGljaCBtYXkgYmUgZGlzY292ZXJhYmxlIGluIGEgbGVnYWwg bWF0dGVyLiBZb3Ugc2hvdWxkIGluZm9ybSBhbGwgbWVldGluZyBhdHRlbmRlZXMgcHJpb3IgdG8g cmVjb3JkaW5nIGlmIHlvdSBpbnRlbmQgdG8gcmVjb3JkIHRoZSBtZWV0aW5nLlxuCkRUU1RBTVA6 MjAxODA4MDhUMTUwMDAwWgpVSUQ6OTY4YmEwNTItMDIzMy00Njk2LWI1OTUtNDY5YmY0YmU2OWNh ClBSSU9SSVRZOjUKRFRTVEFSVDtUWklEPSJFYXN0ZXJuIFRpbWUiOjIwMTgwODA4VDExMDAwMApE VEVORDtUWklEPSJFYXN0ZXJuIFRpbWUiOjIwMTgwODA4VDEyMDAwMApSUlVMRTpGUkVRPVdFRUtM WTtJTlRFUlZBTD0yO0JZREFZPVdFO0NPVU5UPTcKU0VRVUVOQ0U6MTUzMzY2MzM5NgpTVU1NQVJZ OkNCT1Igd29ya2luZyBncm91cCBjb25mZXJlbmNlIGNhbGwKVFJBTlNQOk9QQVFVRQpMT0NBVElP TjpodHRwczovL2lldGYud2ViZXguY29tL2lldGYvai5waHA/TVRJRD1tMmE4ODIxZjg0YmQ4YzU2 MTg5YmU4MWYwMTIyN2U1OWYKRU5EOlZFVkVOVApFTkQ6VkNBTEVOREFSCg== --00000000000069b0b00572dbf5af-- From nobody Tue Aug 7 10:52:32 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8C3D4130EED for ; Tue, 7 Aug 2018 10:52:30 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.898 X-Spam-Level: X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.001, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fZilDPKC4UGS for ; Tue, 7 Aug 2018 10:52:29 -0700 (PDT) Received: from mail-qt0-x234.google.com (mail-qt0-x234.google.com [IPv6:2607:f8b0:400d:c0d::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DBEC4130E0F for ; Tue, 7 Aug 2018 10:52:28 -0700 (PDT) Received: by mail-qt0-x234.google.com with SMTP id h4-v6so18861948qtj.7 for ; Tue, 07 Aug 2018 10:52:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=NBP/kEIDFbqdRGMQViQQWerEthBWf2H6pTamMKlbex4=; b=ljT6zgyoDxhPN5Rmuyi7amJs5K1RZiNKrjjit4lvfWuu+Gwpqz7fHwfsbAlhE4iAd0 3HTiH+z3kZEFqmfbxiZo1Sr2yK9Q71/Nlze+Vz+Jrn24JsHGOGpaWoOkA1tFtiRtW7E6 kMG5MOOYawLKUqwDQIxviR+4Px1kYsFY80X4Je4kYnLP4Zxnm4Vg/ZoH59hdpVWKQyKy 7vJJeJf3eC4nv/9RCgFK28RBhtIcDVctjM48U4+/bCcuy58At7we/cPXdWLy4K6DWYtp AJ2CeeASgjDpQaPRyDunkrQ31GIPSrUKtGU6br4LAI8cjK3L+QiGPzuIKVoOnQ/5EH82 tFKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=NBP/kEIDFbqdRGMQViQQWerEthBWf2H6pTamMKlbex4=; b=QdDwcr31IlkDrgpjpp9ThNTMIxBhdrLuS1MJ6lbJZxM7hM0RlnNFy6hyloAKehEKca fOq0x0D8rTw6vwV4t0bFlo+a3wDA48E+FDBuLimuEsM2h0v2RaCvbZzYIxq39i/zsB8l t0/tHNXbBi3115fa6qN9MZm1nAbztwhc5iDfoycs7LuH7u/aNyLewv8oAx6sKJA/xo5L 9mqrQLpuLrDAPBvECErnO+FPnucxnofzX4O6sR7UX0mEqw/S9O99SSDx6bVNsXjF0jVK rmL1KuRyaZzzA3e+uVQVYW66j+dyQ6tkIMYbvT90mJTwCWUCSWaxL7qOVB7T6dQjRa0K XjoQ== X-Gm-Message-State: AOUpUlGx+U3nc04LN/LQ82YJ07Bk1LXT91udL2lYuo/2CXxLTLNIM+rH lprmirtGtYJU85sLPZNMcMR32gjcbp8fV67ss/xgaA== X-Google-Smtp-Source: AAOMgpeUEtb8FI7r666P6FwA4baIb8nDioS71GU8RpeJO0h9iBuVf25cpDYf8grXX0VemTSTNnN93EXsezy5elHShDg= X-Received: by 2002:a0c:8745:: with SMTP id 5-v6mr17630914qvi.223.1533664347982; Tue, 07 Aug 2018 10:52:27 -0700 (PDT) MIME-Version: 1.0 Sender: barryleiba.mailing.lists@gmail.com Received: by 2002:ac8:2c26:0:0:0:0:0 with HTTP; Tue, 7 Aug 2018 10:52:27 -0700 (PDT) In-Reply-To: <006c01d42e70$2dbf65c0$893e3140$@augustcellars.com> References: <006c01d42e70$2dbf65c0$893e3140$@augustcellars.com> From: Barry Leiba Date: Tue, 7 Aug 2018 13:52:27 -0400 X-Google-Sender-Auth: 6AY1K0Rj_uQ1JxvIqdYvH7okZKg Message-ID: To: Jim Schaad Cc: cbor@ietf.org Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cbor] Agenda for tomorrow X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Aug 2018 17:52:31 -0000 > Is there an agenda for the call tomorrow? The agenda for all the call is to check the work status, work out whatever issues/impediments there are, and keep the work progressing. For this first call, the main thing on the table is that Carsten promised a CDDL document update by the end of July, which we don't seem to have yet, so we'll be discussing that -- is there something we need to do in order to keep CDDL moving? Perhaps get an alternative active editor who can keep progress going if Carsten is too busy. Barry From nobody Tue Aug 7 12:37:45 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DD30F130F02 for ; Tue, 7 Aug 2018 12:37:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.898 X-Spam-Level: X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oEfy7zgMTEXD for ; Tue, 7 Aug 2018 12:37:39 -0700 (PDT) Received: from p3plsmtpa11-04.prod.phx3.secureserver.net (p3plsmtpa11-04.prod.phx3.secureserver.net [68.178.252.105]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0ED08130EA2 for ; Tue, 7 Aug 2018 12:37:38 -0700 (PDT) Received: from [10.12.102.53] ([66.228.17.5]) by :SMTPAUTH: with ESMTPSA id n7nVfQgvlUsN2n7nVf88hj; Tue, 07 Aug 2018 12:37:38 -0700 From: Laurence Lundblade Message-Id: <87170077-3229-4FA2-A179-2A7F979951B0@island-resort.com> Content-Type: multipart/alternative; boundary="Apple-Mail=_98CE3F00-47B4-4EA0-8607-11E84A5C4AF8" Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Date: Tue, 7 Aug 2018 12:37:37 -0700 In-Reply-To: Cc: cbor@ietf.org, Sean Solberg To: Carsten Bormann References: <80DFAEF6-BE25-4F43-9CCF-A8FCBEB1F4BE@island-resort.com> <3101844.7X5zPJ143S@tjmaciei-mobl1> <3B65AA30-5A89-47DD-8563-57EDA6653BBD@island-resort.com> <4FE5660E-0E53-4445-9863-8016454AEBF6@tzi.org> <46A693AD-5AF4-4A84-8A43-3575D10F705E@island-resort.com> X-Mailer: Apple Mail (2.3445.9.1) X-CMAE-Envelope: MS4wfABT20Y02SU+l2ilT4HeQUHvdDdUmqyr0uwTicB9iu1cgRvbvTI0NpTbKy/utiub+HIuwrdWlYtdOD/bzQB7BLbyPBVIVI9lwyRNbMMD6tELMEKnlXLW R7E7ZKE2cwMaOSTIxZfRibFFcKvSXSHKHWDfo8BpRfBJyVDhekRavMW2Oa74uFCxnDk5pmryUBzZw6XyF+mLkyeo8KBtSYrm0p4= Archived-At: Subject: Re: [Cbor] Preferred encoding of floats X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Aug 2018 19:37:43 -0000 --Apple-Mail=_98CE3F00-47B4-4EA0-8607-11E84A5C4AF8 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Comments below. > On Aug 6, 2018, at 1:21 AM, Carsten Bormann wrote: >=20 >=20 >=20 >>> The term =E2=80=9Cpreferred encoding=E2=80=9D in any case is more of = a =E2=80=9Cquality of implementation expectation=E2=80=9D, which is = somewhat orthogonal to the =E2=80=9Cinteroperability considerations=E2=80=9D= that require RFC 2119 wording. >>=20 >> I don=E2=80=99t think IETF does =E2=80=9Cquality of = implementation=E2=80=9D. >=20 > You may be surprised how often it does: >=20 > = https://www.google.de/search?q=3D%22quality+of+implementation%22+site%3Arf= c-editor.org >=20 > (These are just the RFCs that use that exact phrase, but there are = tons other that provide guidance of a similar kind using different = words.) >=20 >> I don=E2=80=99t think it should. >=20 > I=E2=80=99d say, we are doing it much less than we actually should. =20= >=20 > We are defining protocols to create benefit. This includes giving = implementers guidance what to implement. Beyond what is absolutely = needed for interoperability per se, there may also guidance on what = implementation decisions may further increase benefit. >=20 >> Formally, that is the realm of certifications like Common Criteria = and FIPS. Informally, we all want to do a good job of our code, have = code reviews, fuzz test and such, but none of that goes in an RFC. >=20 > Neither certification nor code review do. But the above mentioned = =E2=80=9Cquality of implementation=E2=80=9D guidance does. > Implementers are often looking for a cop-out to do something the easy = way (and, I=E2=80=99d say, legitimately so!). > Allowing some undesirable behavior in an RFC (just because it is not = fatal for interoperability) may serve as such a cop-out. That=E2=80=99s = why additional =E2=80=9Cquality of implementation=E2=80=9D guidance is = so useful. (Or implementers may simply not be aware of the benefits of = achieving some quality of implementation objective.) >=20 >> I also don=E2=80=99t think any particular feature set or mode of CBOR = should be considered necessary for higher quality implementation, to = defend against buffer overruns or general SW security. In this day and = age all features and options in CBOR should be implementable in equally = and very secure ways. >=20 > I=E2=80=99m not sure where security comes in here, but I agree CBOR is = implementable. =20 > But it sure helps new implementers to know what is a good way to do = things. So I had/have some idea that: A) if some text affects how bits on the wire are sent or received, it = should be mainline, complete, non-subjective, non-qualified, normative = text B) quality of implementation text is not about bits on the wire, but = rather about UX, SW security, certification, APIs and such =46rom looking at RFC in your google search, quality of implementation = sections sometimes address A) and sometimes address B). I still prefer = the A), B) separation as helpful towards sharp clear text specifying = interop and bits on the wire, but now understand that is not always the = way it is done. Thanks for the Google search and other info. >=20 >> So I think preferred encoding / decoding is only about = interoperability. >=20 > As I said above, interoperability is an important benefit, but this is = about overall benefit, not just interoperability. >=20 >> Interoperability is the focus of this pull request on preferred = encoding / decoding. (I think we=E2=80=99ve determined canonicalization = for the sake or repeatable deterministic encoding is a lesser focus).=20 >=20 > Different focus. (Whether it=E2=80=99s lesser is dependent on what = the application needs.) >=20 >> For interop for half, float and double, the main thing I think is = important is that preferred decoders support all three. They are liberal = in what they accept. Some implementations, those on small CPUs without = any floating point support, will not be able to implement this and thus = can=E2=80=99t be preferred decoders. You have to support float to be a = preferred decoder.=20 >=20 > I don=E2=80=99t think we want to have =E2=80=9Cpreferred decoders=E2=80=9D= and =E2=80=9Cnon-preferred decoders=E2=80=9D. A decoder implements all = of CBOR or an application-specific subset. =E2=80=9CAll of CBOR=E2=80=9D = is certainly preferred for what tries to be a generic decoder, but that = is so obvious that it probably does not need to be stated. All of CBOR is too much to expect any majority of decoders and encoders = to implement. Somethings like float, double and half are a little hard = to figure out (as I am discovering). Other things like bignums seem = useful for a very very narrow set of use cases (they aren=E2=80=99t even = used for COSE keys). Since we can=E2=80=99t expect =E2=80=9Call=E2=80=9D from the majority of = decoders, all we have is application-specific subsets. That=E2=80=99s a = little hard on those of us working on generic libraries.=20 >=20 > =E2=80=9CPreferred=E2=80=9D mostly comes in where there are *encoding* = choices that are all =E2=80=9Cvalid=E2=80=9D, but some of which are = better than others. Talking about quality of implementation in a = specification may help an implementer to make the right decisions here. = Generally, it is good for an ecosystem if the non-essential decisions = mostly go into the same direction, e.g., because decoder implementers = can then optimize for that. My PR is basically proposing that there be a =E2=80=9Cpreferred=E2=80=9D = CBOR subset that will cover the vast majority of use cases and that all = uses of this preferred subset are guaranteed to interoperate. To do this = the preferred subset must apply to decoding as well as encoding.=20 This notion of =E2=80=9Cpreferred" makes it easy for protocol designers. = They just stick to the preferred set which is should be enough for most = use cases. They don=E2=80=99t have to think too hard about interop. It = makes it easy for generic library implementors and it make it easy for = an application to select a library. Protocol implementors just design = for preferred and select a library that is preferred. It should also = make translation to JSON more uniform.=20 I think we also have to thoroughly acknowledge that not all = implementations will be able to implement all of preferred for very good = reason that they are on a constrained device or have a special use case. = There is / was sentiment here that such uses of CBOR are bad and = non-interoperable. That is not the case.=20 >=20 >> I=E2=80=99m still trying to get my head fully around what to do with = denormal and varying NaNs for interop.=20 >=20 > As I said, denormal is a normal thing in IEEE 754 (called subnormal = there). OK. Understood about denormal. Thanks for taking the time to explain. > Varying NaNs are not disallowed by CBOR; I don=E2=80=99t think there = is a good expectation for interoperability with =E2=80=9CNaN boxing=E2=80=9D= or similar approaches, though. >=20 >> It seems like preferred encoding should require the canonical form = for NaN, but I=E2=80=99m not sure if it should support both sNaN and = qNaN, or just one NaN. >=20 > The current text for recommended canonical, maps all NaNs to a single = (quiet) NaN. > That may indeed not be the best way for all applications. I assume NaN boxing and NaN payloads are the same thing.=20 NaN boxing/payloads is not compatible with a canonical / preferred = representation NaN. You have to pick one or the other.=20 So in CBOR we can choose one of these paths: A) Always use the one single representation of NaN and entirely disallow = boxing/payloads B) In preferred/canonical CBOR use the one single representation, which = makes boxing/payloads work for regular, but not canonical/preferred CBOR C) Have no recommendation at all. NaNs are just carried as they come = (like denormals) My preference is for A), but could go with C) - I like sparing users of CBOR the work of having to figure out more = weird stuff like NaN boxing. NaN boxing/payloads seems like a very = marginal feature.=20 - I think B) is weird and less interoperable because sometimes NaN = boxing/payloads work in CBR and sometimes they do not. - I think C) is not as crisp as A) LL --Apple-Mail=_98CE3F00-47B4-4EA0-8607-11E84A5C4AF8 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Comments below.

On Aug 6, 2018, at 1:21 AM, = Carsten Bormann <cabo@tzi.org> wrote:



The term =E2=80=9Cpreferred encoding=E2=80=9D = in any case is more of a =E2=80=9Cquality of implementation = expectation=E2=80=9D, which is somewhat orthogonal to the = =E2=80=9Cinteroperability considerations=E2=80=9D that require RFC 2119 = wording.

I don=E2=80=99t think = IETF does =E2=80=9Cquality of implementation=E2=80=9D.

You may be surprised how often it = does:

https://www.google.de/search?q=3D%22quality+of+implementation%2= 2+site%3Arfc-editor.org

(These are just = the RFCs that use that exact phrase, but there are tons other that = provide guidance of a similar kind using different words.)

I don=E2=80= =99t think it should.

I=E2=80=99= d say, we are doing it much less than we actually should.  

We are defining protocols to create benefit. =  This includes giving implementers guidance what to implement. =  Beyond what is absolutely needed for interoperability per se, = there may also guidance on what implementation decisions may further = increase benefit.

Formally, that is the realm of certifications like Common = Criteria and FIPS. Informally, we all want to do a good job of our code, = have code reviews, fuzz test and such, but none of that goes in an = RFC.

Neither certification nor = code review do.  But the above mentioned =E2=80=9Cquality of = implementation=E2=80=9D guidance does.
Implementers are = often looking for a cop-out to do something the easy way (and, I=E2=80=99d= say, legitimately so!).
Allowing some undesirable = behavior in an RFC (just because it is not fatal for interoperability) = may serve as such a cop-out.  That=E2=80=99s why additional = =E2=80=9Cquality of implementation=E2=80=9D guidance is so useful. =  (Or implementers may simply not be aware of the benefits of = achieving some quality of implementation objective.)

I also don=E2=80=99t = think any particular feature set or mode of CBOR should be considered = necessary for higher quality implementation, to defend against buffer = overruns or general SW security. In this day and age all features and = options in CBOR should be implementable in equally and very secure = ways.

I=E2=80=99m not sure = where security comes in here, but I agree CBOR is implementable. =  
But it sure helps new implementers to know what is = a good way to do things.

So I had/have some idea = that:
A) if some text affects how bits on = the wire are sent or received, it should be mainline, complete, = non-subjective, non-qualified, normative text
B) = quality of implementation text is not about bits on the wire, but rather = about UX, SW security, certification, APIs and = such

=46rom = looking at RFC in your google search, quality of implementation sections = sometimes address A) and sometimes address B). I still prefer the A), B) = separation as helpful towards sharp clear text specifying interop and = bits on the wire, but now understand that is not always the way it is = done.  Thanks for the Google search and other info.



So I think preferred encoding / decoding is only about = interoperability.

As I said = above, interoperability is an important benefit, but this is about = overall benefit, not just interoperability.

Interoperability is the = focus of this pull request on preferred encoding / decoding.  (I = think we=E2=80=99ve determined canonicalization for the sake or = repeatable deterministic encoding is a lesser focus).

Different focus.  (Whether = it=E2=80=99s lesser is dependent on what the application needs.)

For = interop for half, float and double, the main thing I think is important = is that preferred decoders support all three. They are liberal in what = they accept. Some implementations, those on small CPUs without any = floating point support, will not be able to implement this and thus = can=E2=80=99t be preferred decoders. You have to support float to be a = preferred decoder.

I don=E2=80=99= t think we want to have =E2=80=9Cpreferred decoders=E2=80=9D and = =E2=80=9Cnon-preferred decoders=E2=80=9D.  A decoder implements all = of CBOR or an application-specific subset.  =E2=80=9CAll of CBOR=E2=80= =9D is certainly preferred for what tries to be a generic decoder, but = that is so obvious that it probably does not need to be stated.

All = of CBOR is too much to expect any majority of decoders and encoders to = implement. Somethings like float, double and half are a little hard to = figure out (as I am discovering). Other things like bignums seem useful = for a very very narrow set of use cases (they aren=E2=80=99t even used = for COSE keys).

Since we can=E2=80=99t= expect =E2=80=9Call=E2=80=9D from the majority of decoders, all we have = is application-specific subsets. That=E2=80=99s a little hard on those = of us working on generic libraries. 



=E2=80=9CPreferred=E2=80=9D = mostly comes in where there are *encoding* choices that are all = =E2=80=9Cvalid=E2=80=9D, but some of which are better than others. =  Talking about quality of implementation in a specification may = help an implementer to make the right decisions here.  Generally, = it is good for an ecosystem if the non-essential decisions mostly go = into the same direction, e.g., because decoder implementers can then = optimize for that.


My PR is basically = proposing that there be a =E2=80=9Cpreferred=E2=80=9D CBOR subset that = will cover the vast majority of use cases and that all uses of this = preferred subset are guaranteed to interoperate. To do this the = preferred subset must apply to decoding as well as = encoding. 

This notion of = =E2=80=9Cpreferred" makes it easy for protocol designers. They just = stick to the preferred set which is should be enough for most use cases. = They don=E2=80=99t have to think too hard about interop. It makes it = easy for generic library implementors and it make it easy for an = application to select a library. Protocol implementors just design for = preferred and select a library that is preferred. It should also make = translation to JSON more uniform. 

I think we also have to thoroughly acknowledge = that not all implementations will be able to implement all of preferred = for very good reason that they are on a constrained device or have a = special use case. There is / was sentiment here that such uses of CBOR = are bad and non-interoperable. That is not the = case. 



I=E2=80=99m= still trying to get my head fully around what to do with denormal and = varying NaNs for interop.

As = I said, denormal is a normal thing in IEEE 754 (called subnormal = there).

OK. Understood about denormal. Thanks for taking = the time to explain.


Varying NaNs are not disallowed by CBOR; I don=E2=80=99t = think there is a good expectation for interoperability with =E2=80=9CNaN = boxing=E2=80=9D or similar approaches, = though.

It seems like preferred encoding should require the canonical = form for NaN, but I=E2=80=99m not sure if it should support both sNaN = and qNaN, or just one NaN.

The = current text for recommended canonical, maps all NaNs to a single = (quiet) NaN.
That may indeed not be the best way for all = applications.

I assume NaN boxing and NaN payloads are the = same thing. 

NaN = boxing/payloads is not compatible with a canonical / preferred = representation NaN. You have to pick one or the = other. 

So in CBOR we can = choose one of these paths:
A) = Always use the one single representation of NaN and entirely disallow = boxing/payloads
B) In = preferred/canonical CBOR use the one single representation, which makes = boxing/payloads work for regular, but not canonical/preferred = CBOR
C) Have no recommendation at all. = NaNs are just carried as they come (like = denormals)

My preference is for A), but could go with = C)
- I like sparing users of = CBOR the work of having to figure out more weird stuff like NaN boxing. = NaN boxing/payloads seems like a very marginal = feature. 
- I think B) is weird and = less interoperable because sometimes NaN boxing/payloads work in CBR and = sometimes they do not.
- I think C) is = not as crisp as A)

LL



= --Apple-Mail=_98CE3F00-47B4-4EA0-8607-11E84A5C4AF8-- From nobody Tue Aug 7 14:55:07 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C0E14130DC5 for ; Tue, 7 Aug 2018 14:55:05 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.199 X-Spam-Level: X-Spam-Status: No, score=-4.199 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6C2ZanzQo1Sz for ; Tue, 7 Aug 2018 14:55:03 -0700 (PDT) Received: from mailhost.informatik.uni-bremen.de (mailhost.informatik.uni-bremen.de [IPv6:2001:638:708:30c9::12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 475CE130DC3 for ; Tue, 7 Aug 2018 14:55:02 -0700 (PDT) X-Virus-Scanned: amavisd-new at informatik.uni-bremen.de Received: from submithost.informatik.uni-bremen.de (submithost.informatik.uni-bremen.de [IPv6:2001:638:708:30c9::b]) by mailhost.informatik.uni-bremen.de (8.14.5/8.14.5) with ESMTP id w77LsQiP026920; Tue, 7 Aug 2018 23:54:26 +0200 (CEST) Received: from client-0211.vpn.uni-bremen.de (client-0211.vpn.uni-bremen.de [134.102.107.211]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by submithost.informatik.uni-bremen.de (Postfix) with ESMTPSA id 41lSwL2ZcTzDX4b; Tue, 7 Aug 2018 23:54:26 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) From: Carsten Bormann In-Reply-To: Date: Tue, 7 Aug 2018 23:54:25 +0200 Cc: Jim Schaad , cbor@ietf.org X-Mao-Original-Outgoing-Id: 555371663.534018-1f04eaac57de2386c2696e7ee996ba68 Content-Transfer-Encoding: quoted-printable Message-Id: References: <006c01d42e70$2dbf65c0$893e3140$@augustcellars.com> To: Barry Leiba X-Mailer: Apple Mail (2.3445.9.1) Archived-At: Subject: Re: [Cbor] Agenda for tomorrow X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Aug 2018 21:55:06 -0000 If we have time, we could also look at the current draft: https://cbor-wg.github.io/cddl/ https://cbor-wg.github.io/cddl/draft-ietf-cbor-cddl.html = https://tools.ietf.org/rfcdiff?url1=3Dhttps://tools.ietf.org/id/draft-ietf= -cbor-cddl.txt&url2=3Dhttps://cbor-wg.github.io/cddl/draft-ietf-cbor-cddl.= txt (We could have submitted that state as a -04, but I=E2=80=99d assume = further fixes are likely to be made right after the Interim. In = particular:) The solution for the .ne/.eq question is somewhat novel, let=E2=80=99s = see if we can live with that. (We tried to keep as many of the current specifications valid as = possible, but that means that .ne/.eq is a bit of a .) I also went ahead and made h=E2=80=99f000=E2=80=99 and b64=E2=80=99BAA=E2=80= =99 case-insensitive, so they can be written as H=E2=80=99f000=E2=80=99 = and B64=E2=80=99BAA=E2=80=99 =E2=80=94 a result of some side-track we = had on this list. This makes all of the surface syntax of CDDL = case-insensitive (the rule names, of course, stay case-sensitive). Gr=C3=BC=C3=9Fe, Carsten > On Aug 7, 2018, at 19:52, Barry Leiba wrote: >=20 >> Is there an agenda for the call tomorrow? >=20 > The agenda for all the call is to check the work status, work out > whatever issues/impediments there are, and keep the work progressing. >=20 > For this first call, the main thing on the table is that Carsten > promised a CDDL document update by the end of July, which we don't > seem to have yet, so we'll be discussing that -- is there something we > need to do in order to keep CDDL moving? Perhaps get an alternative > active editor who can keep progress going if Carsten is too busy. >=20 > Barry >=20 > _______________________________________________ > CBOR mailing list > CBOR@ietf.org > https://www.ietf.org/mailman/listinfo/cbor >=20 From nobody Tue Aug 7 15:37:43 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E9586131102 for ; Tue, 7 Aug 2018 15:37:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.2 X-Spam-Level: X-Spam-Status: No, score=-4.2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yFrcJgV_qIsV for ; Tue, 7 Aug 2018 15:37:40 -0700 (PDT) Received: from mailhost.informatik.uni-bremen.de (mailhost.informatik.uni-bremen.de [IPv6:2001:638:708:30c9::12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4E21B127148 for ; Tue, 7 Aug 2018 15:37:40 -0700 (PDT) X-Virus-Scanned: amavisd-new at informatik.uni-bremen.de Received: from submithost.informatik.uni-bremen.de (submithost.informatik.uni-bremen.de [IPv6:2001:638:708:30c9::b]) by mailhost.informatik.uni-bremen.de (8.14.5/8.14.5) with ESMTP id w77Mbaot002634; Wed, 8 Aug 2018 00:37:36 +0200 (CEST) Received: from client-0211.vpn.uni-bremen.de (client-0211.vpn.uni-bremen.de [134.102.107.211]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by submithost.informatik.uni-bremen.de (Postfix) with ESMTPSA id 41lTt76M3BzDX4s; Wed, 8 Aug 2018 00:37:35 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) From: Carsten Bormann In-Reply-To: <87170077-3229-4FA2-A179-2A7F979951B0@island-resort.com> Date: Wed, 8 Aug 2018 00:37:35 +0200 Cc: cbor@ietf.org, Sean Solberg X-Mao-Original-Outgoing-Id: 555374253.370826-48186897ee017daca79f264cbd89f6a3 Content-Transfer-Encoding: quoted-printable Message-Id: <3A7B27E3-9DCD-48D3-BDEB-BDA41E7AD1B1@tzi.org> References: <80DFAEF6-BE25-4F43-9CCF-A8FCBEB1F4BE@island-resort.com> <3101844.7X5zPJ143S@tjmaciei-mobl1> <3B65AA30-5A89-47DD-8563-57EDA6653BBD@island-resort.com> <4FE5660E-0E53-4445-9863-8016454AEBF6@tzi.org> <46A693AD-5AF4-4A84-8A43-3575D10F705E@island-resort.com> <87170077-3229-4FA2-A179-2A7F979951B0@island-resort.com> To: Laurence Lundblade X-Mailer: Apple Mail (2.3445.9.1) Archived-At: Subject: Re: [Cbor] Preferred encoding of floats X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Aug 2018 22:37:43 -0000 Hi Laurence, thanks for keeping up the discussion of floats; they are often neglected = in circles like the IETF (say, YANG doesn=E2=80=99t not even have them!) = and we have to get this right. >> I don=E2=80=99t think we want to have =E2=80=9Cpreferred decoders=E2=80= =9D and =E2=80=9Cnon-preferred decoders=E2=80=9D. A decoder implements = all of CBOR or an application-specific subset. =E2=80=9CAll of CBOR=E2=80= =9D is certainly preferred for what tries to be a generic decoder, but = that is so obvious that it probably does not need to be stated. >=20 > All of CBOR is too much to expect any majority of decoders and = encoders to implement. Actually, since anyone can *extend* CBOR at any time by defining another = Tag, that is an impossible expectation. But we can describe what a full = generic CBOR decoder would do: 1 =E2=80=94 decode any valid CBOR, 2 =E2=80=94 present all information in there that goes beyond = representation variants to the application. Now, number 2 is the fun part. Is the difference between 1 and = 2(h=E2=80=9901=E2=80=99) just a representation variant? For many implementations, it will be. However, implementations that are = stuck with a BigInt that isn=E2=80=99t even numeric may think this is a = valid distinction (e.g., a JavaScript implementation might want to = encode 1n as 2(h=E2=80=9901=E2=80=99) to distinguish it from 1). = We=E2=80=99ll need to make some taste decisions there. > Somethings like float, double and half are a little hard to figure out = (as I am discovering). I just extracted the float decoder implementation from my Elixir code. = (This code is a bit more complicated than needed because Erlang does not = handle nonfinites, so there is about twice as much code as needed.) =E2=80=A6 7 -> case bin do << 0xf9, sign::size(1), exp::size(5), mant::size(10), _::binary >> -> {decode_half(sign, exp, mant, = treatment), rest} << 0xfa, value::[float, size(32)], _::binary >> -> = {value, rest} << 0xfa, sign::size(1), 255::size(8), mant::size(23), _::binary >> -> {treatment.nonfinites.(sign, mant), = rest} << 0xfb, value::[float, size(64)], _::binary >> -> = {value, rest} << 0xfb, sign::size(1), 2047::size(11), mant::size(52), _::binary >> -> {treatment.nonfinites.(sign, mant), = rest} =E2=80=A6 defp decode_half(sign, 31, mant, treatment), do: = treatment.nonfinites.(sign, mant) defp decode_half(sign, exp, mant, _) do << value::[float, size(32)] >> =3D << sign::size(1), exp::size(8), mant::size(10), 0::size(13) >> value * 5192296858534827628530496329220096.0 # 2**112 =E2=80=94 = difference in bias Just showing that this is not particularly hard even in a language that = doesn=E2=80=99t give you a lot of tools to handle floating point. > Other things like bignums seem useful for a very very narrow set of = use cases (they aren=E2=80=99t even used for COSE keys). The JavaScript people are just now finding out that they were lacking = them all the time=E2=80=A6 It is just needed for many applications that can=E2=80=99t survive = artificial limitations. (Each of us has a certain set of experience behind their tastes, and it = is too easy to fall into the trap that just because something does not = occur in that experience that it is unimportant=E2=80=A6) > Since we can=E2=80=99t expect =E2=80=9Call=E2=80=9D from the majority = of decoders, all we have is application-specific subsets. That=E2=80=99s = a little hard on those of us working on generic libraries.=20 I would expect generic libraries to differ a lot between differently = capable programming environments. A Lua implementation will always be more limited than a Haskell one. I don=E2=80=99t think we can normalize that. But we can make sure that = all full generic implementations at least enable applications to handle = some types (awkwardly) that the programing environment doesn=E2=80=99t. = Like the ugly nonfinites above=E2=80=A6 > My PR is basically proposing that there be a =E2=80=9Cpreferred=E2=80=9D= CBOR subset that will cover the vast majority of use cases and that all = uses of this preferred subset are guaranteed to interoperate. To do this = the preferred subset must apply to decoding as well as encoding.=20 That would be the I-JSON of CBOR. Meaning the other half of CBOR can no = longer be used. We don=E2=80=99t have to make that mistake, I think. > This notion of =E2=80=9Cpreferred" makes it easy for protocol = designers. They just stick to the preferred set which is should be = enough for most use cases. They don=E2=80=99t have to think too hard = about interop. It makes it easy for generic library implementors and it = make it easy for an application to select a library. Protocol = implementors just design for preferred and select a library that is = preferred. It should also make translation to JSON more uniform.=20 More shades of I-JSON=E2=80=A6 > I assume NaN boxing and NaN payloads are the same thing.=20 Yes. > NaN boxing/payloads is not compatible with a canonical / preferred = representation NaN. You have to pick one or the other.=20 An application could define their =E2=80=9Ccanonical=E2=80=9D with the = right way of NaN handling. CBOR definitely can transport NaN payloads. > So in CBOR we can choose one of these paths: > A) Always use the one single representation of NaN and entirely = disallow boxing/payloads > B) In preferred/canonical CBOR use the one single representation, = which makes boxing/payloads work for regular, but not = canonical/preferred CBOR > C) Have no recommendation at all. NaNs are just carried as they come = (like denormals) >=20 > My preference is for A), but could go with C) > - I like sparing users of CBOR the work of having to figure out more = weird stuff like NaN boxing. NaN boxing/payloads seems like a very = marginal feature.=20 > - I think B) is weird and less interoperable because sometimes NaN = boxing/payloads work in CBR and sometimes they do not. > - I think C) is not as crisp as A) I=E2=80=99m leaning the same way, but I still think it is valid for an = application to go ahead and use the full capabilities of the CBOR = serialization (e.g., to carry around NaN payloads). It is probably a = mistake in my implementation that I always canonicalize NaNs in the = encoder =E2=80=94 I should only do this when canonical encoding was = requested. I still think that it is good to turn both = 0xFB7FF8000000000000 and 0xFA7FC00000 into 0xF97E00 in the generic = encoder. Gr=C3=BC=C3=9Fe, Carsten From nobody Wed Aug 8 08:51:20 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2EE1F130E7B for ; Wed, 8 Aug 2018 08:51:18 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.31 X-Spam-Level: X-Spam-Status: No, score=-4.31 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, T_DKIMWL_WL_HIGH=-0.01] autolearn=unavailable autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=ericsson.com header.b=b8BMX2io; dkim=pass (1024-bit key) header.d=ericsson.com header.b=KbXybVrt Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id D1j1mhb0J6eX for ; Wed, 8 Aug 2018 08:51:16 -0700 (PDT) Received: from sessmg23.ericsson.net (sessmg23.ericsson.net [193.180.251.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 89AC4130E7A for ; Wed, 8 Aug 2018 08:51:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; d=ericsson.com; s=mailgw201801; c=relaxed/simple; q=dns/txt; i=@ericsson.com; t=1533743472; h=From:Sender:Reply-To:Subject:Date:Message-ID:To:CC:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=q63OdszTDNQkDhvw4SKvv5CWvzNaUzfIXhDMTcwpo+A=; b=b8BMX2io7Jl57UWx/yHO2D4UuKybJpThiYT/SyE9h0uUD8KxfD5DkQiKTrbw3FH7 2wxjar0sQJ36fNJz3K7v6p5ahExRD5HTEVrL5JSmQvui/U0U79MYqOl476KhebQJ hV6mapiuWfdV0x87jeJcp7JQbPk6JXeh4A5sc+448AY=; X-AuditID: c1b4fb2d-223ff700000055ff-f6-5b6b1170556d Received: from ESESSMB503.ericsson.se (Unknown_Domain [153.88.183.121]) by sessmg23.ericsson.net (Symantec Mail Security) with SMTP id 2F.D0.22015.0711B6B5; Wed, 8 Aug 2018 17:51:12 +0200 (CEST) Received: from ESESSMB503.ericsson.se (153.88.183.164) by ESESSMB503.ericsson.se (153.88.183.164) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1466.3; Wed, 8 Aug 2018 17:51:12 +0200 Received: from EUR03-AM5-obe.outbound.protection.outlook.com (153.88.183.157) by ESESSMB503.ericsson.se (153.88.183.164) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1466.3 via Frontend Transport; Wed, 8 Aug 2018 17:51:12 +0200 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8jftX9oz4/FhGrQHAlAlN2PhxbXk97Of4yA+hFhIbh0=; b=KbXybVrtYCw3rcSgh6RMeA/XL6FKAqy6zYLk1xMGn7ULSuK+EoDPRmtolW0hXicoPa4VPC7TUabHYWmleidgIYpfZH7wEHq0hMx57hCW4C+efpN1GJNdV5U21Jc7zgiJhJEdzCizY++7ARn6LwYIpfj9Oilb8DfLSSYPUBl6uyo= Received: from AM5PR0701MB2737.eurprd07.prod.outlook.com (10.173.93.139) by AM5PR0701MB3010.eurprd07.prod.outlook.com (10.168.156.148) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1038.13; Wed, 8 Aug 2018 15:51:11 +0000 Received: from AM5PR0701MB2737.eurprd07.prod.outlook.com ([fe80::9d14:49dc:68aa:8bef]) by AM5PR0701MB2737.eurprd07.prod.outlook.com ([fe80::9d14:49dc:68aa:8bef%2]) with mapi id 15.20.1038.013; Wed, 8 Aug 2018 15:51:11 +0000 From: Francesca Palombini To: "cbor@ietf.org" CC: "cbor-chairs@ietf.org" Thread-Topic: Minutes from 1st webex call Thread-Index: AdQvL5/CcTrFrFbfQ+GN9fW4eBOkUg== Date: Wed, 8 Aug 2018 15:51:11 +0000 Message-ID: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=francesca.palombini@ericsson.com; x-originating-ip: [5.94.252.163] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; AM5PR0701MB3010; 6:wfHPTeap9sNUBy0/0XpIn0xtXBl5m9EqoZkpwynhCMf4DRO6EecNZDIhwCZfevGCs1ewkKK2MOiP51JGp9pP9q7IcOTu13sFYZqUh38BhcqdvcFWm8d5vw/q6svJFxuvdCWPBIT+DyGpL132F027CH8TEcI/si8tQX1tGvYdUBPTlu2Oz6UQQvjBuO/OTrEUyMGO2a16oKDgqM1K4sX2mXvnaBjowAunOIuC23oM3zwfHMAArRZA50kCBkL+ITZOzZaOJm8D8t++a8TgDHc4hikyIhYIh1TaJ1X5lX2dNlDMkw8mM1nF6esRlTJZrR3F5xRIoYsuyDo3vkxKMFheJrXP2G9934OPVWbNYzor5Z4eRO1pNjQ5oSdA1Z0ZBM8FgEP8G25mJYeKx6hd/tm5zJDHTCzhJsbgfSEXDPcrDSKtpI9neaFan3+wSn17B73vjRO/eH7WuNd1CYh03QoEMQ==; 5:LNfy112G7bf4ZhynTiOCBwippnrSweeh3Yt38oSBmBhn98QOINBmyO2HgXTYxs/pwPCVOUdP/bgRwqMYEj1hBdQL3Aupm0Hmh4jls+ScI3MkTiS1G1Qj3gQvjyuQh4TYj5fX2b8Ft10E2+VwYzhSRCkQ5oPgUj/uUc2hka/JJ+k=; 7:5/Q3b1u6P/C6tlgWW0r+cwtX9/50FwYEYEnrXQ335ODIa5ozthOjQBOk0DkqMuVlsMTWc/OZqpmUJsl0T18GFjttJ3FkC2uBnHw+5dJnHKUsLs/Ly7O0bixXVQpEkwmUhWzWJYSnoOzSLXmZGXi6Q8bjp1ueRwugrBHzfJZYxH8T6Bm1t1XyMVH6H475w9vtAXpx9gNcDJqbZ/mcmhzholrYx5LEHFJWicgPs27ZJvyKoiULVkkFxODtXE1jkrwD x-ms-exchange-antispam-srfa-diagnostics: SOS; x-ms-office365-filtering-correlation-id: a6e14ed5-b9e2-4f91-cad3-08d5fd46c3aa x-microsoft-antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989117)(5600074)(711020)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(2017052603328)(7153060)(7193020); SRVR:AM5PR0701MB3010; x-ms-traffictypediagnostic: AM5PR0701MB3010: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(28532068793085)(21748063052155); x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(93006095)(93001095)(3231311)(944501410)(52105095)(10201501046)(3002001)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(20161123560045)(20161123562045)(20161123558120)(6072148)(201708071742011)(7699016); SRVR:AM5PR0701MB3010; BCL:0; PCL:0; RULEID:; SRVR:AM5PR0701MB3010; x-forefront-prvs: 07584EDBCD x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(136003)(39860400002)(346002)(376002)(366004)(396003)(53754006)(189003)(199004)(25786009)(256004)(6116002)(5630700001)(790700001)(14444005)(4326008)(486006)(19609705001)(44832011)(476003)(450100002)(186003)(3846002)(53936002)(68736007)(2906002)(71446004)(86362001)(54896002)(9686003)(5250100002)(6306002)(2501003)(99286004)(33656002)(478600001)(316002)(102836004)(55016002)(2900100001)(561944003)(6916009)(26005)(6506007)(2351001)(74316002)(97736004)(81166006)(5640700003)(6436002)(66066001)(8676002)(81156014)(1730700003)(7736002)(105586002)(8936002)(7696005)(106356001)(14454004)(5660300001); DIR:OUT; SFP:1101; SCL:1; SRVR:AM5PR0701MB3010; H:AM5PR0701MB2737.eurprd07.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: ericsson.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: G8Q2BuWep4HK1j5cwuThYXf2m7K2l8w7a6RMPbTbM/iWoQAy/cC5+Wt9sDGqGmIXRVuNEyN6qy+wnXrADM+s7Ks8wP2X6j7tlkmEctIi5/A3wGZGJ6G8Khvg7gptGeoPh+5W7/rLWZluXAT1hpow4mfSy1CVQmMoo/hYPao3PGIBCtD/91lSY8LqxgXb12OUf5vdJmLDMI1qzzKipbaiWqJS2AvreokJ3ieCEA9MUEGR492VXknNiwWiWxDG3v7GsZFuin+Cm283xurbEgI/kuptxgI0/dIuRnDE6txetu0hMd3II7/mcBnHnE+zytRcFa+Fpi2meQWI82gTjPU9bRc/2Y2JzCLH7hwZq0O3F7k= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: multipart/alternative; boundary="_000_AM5PR0701MB273766AD3A8AC4ABB6EFC57698260AM5PR0701MB2737_" MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: a6e14ed5-b9e2-4f91-cad3-08d5fd46c3aa X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Aug 2018 15:51:11.3987 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0701MB3010 X-OriginatorOrg: ericsson.com X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrEKsWRmVeSWpSXmKPExsUyM2J7pW6BYHa0waJnZhaz/25htJjZMZ3F gcljyZKfTAGMUVw2Kak5mWWpRfp2CVwZj95tZio4klWx9VJxA+OnmC5GTg4JAROJ1y+OMXcx cnEICRxllHjz/RYrhPOVUaLx5SE2CGcxk8TmXzfYQFpYBCYwS6y+bgWRmM4k8br9JDtIQkjg KaPEw+nBIDabgI3EhYfvWUFsEQFlialTDgM1c3AwCxhK/NxTCBIWFlCRaJ93hx2iRFNidtsE qHI9iQf3jzGClLMA1ZyfKg8S5hVIkLgydSVYCaOArMSXxtXMIDazgLjErSfzmSC+EZBYsuc8 M4QtKvHy8T+o+mSJK7f72CHiChLfzp1hgbBlJS7N72YEeUVC4AC7xNVVD6ASuhIfpk6FGuQr 8erLfDaIopOMEv0fl0FN0pE48/w8G4SdL/Hv0HOoK7wk/p/ph2qWk1jV+5AFovkQs8SyH/+h NshIzLz7hnkCo94sJF9A2PkSb97eY50F9rWgxMmZT1gg4joSC3Z/YoOwtSWWLXzNDGOfOfCY CVl8ASP7KkbR4tTi4tx0I2O91KLM5OLi/Dy9vNSSTYzAlHNwy2/dHYyrXzseYhTgYFTi4T3P kh0txJpYVlyZe4hRgoNZSYQ3uS4zWog3JbGyKrUoP76oNCe1+BCjNAeLkjiv3qo9UUIC6Ykl qdmpqQWpRTBZJg5OqQbGpG7HLp3Y1889D8bvMbXJKW3xYP+3Kqrtp4fPlufFbJNMwx8djWN8 8mR588Kf14/+Syi8u1xrLavGnN9Tz0k94ixvqtjzZN61ULavtlXRWm6GVbm7vjx8UyKvW9O4 LK/KPzzP5rB+0CWnqYlxJ6XfhHcqblgnvzUkhW/eEp4VNxpWV2w7JvhYiaU4I9FQi7moOBEA iWUUljUDAAA= Archived-At: Subject: [Cbor] Minutes from 1st webex call X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Aug 2018 15:51:18 -0000 --_000_AM5PR0701MB273766AD3A8AC4ABB6EFC57698260AM5PR0701MB2737_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi all, I took some notes from the call today, where Carsten gave us an update of t= he status of CDDL. Participants, feel free to add anything I might have missed. Thanks, Francesca -- CBOR WG Conference Call Wednesday, Aug 8, 2018, 15:00 - 16:00 UTC Chairs: Barry Leiba, Francesca Palombini Minute takers: Francesca Participants: * Francesca Palombini * Barry Leiba * David Brown * Laurence Lundblade * Michael Richardson * Carsten Bormann # CDDL Barry: Goal of this meeting is to make sure the doc progresses. Carsten has= made updates in the github, not updated in the datatracker yet. Carsten: presenting the diff between last draft (-03) and github versions. = All formal comments should be here (github editor version), might need some= polishing - Standard Prelude moved to appendix D (Jim's comment) - number of changes are editorials that are still in the document (Jim'= s comment) - one objective is still there that we decided not to do, number of cha= nges related to that - in terminology: explicit def of term byte - section 2.1 explains how group work following issue 14 - fixes to figure - 2.2.2 explicit about delivery - 2.2.2.1 explicit ranges are either between integers or floating + con= trol operators - 2.2.3 representation types, comments welcome if the text is clear - 3.5.3 confusion about cut means, trying to make this clearer - 3.8 control operators have fw considerations to iana registry - .size is more explicitly explained what it is for - 3.8.6 .ne and .eq : leaving specification as people have been using i= t but explicit - 3.10 Jim's comments about generic rules (+ editorial) - 3.11 encourage people to use parenthesis when precedence is not clear - Jim's noticed tilde was missing in the operator table, fixed, and amp= as well - CDDL operator registry defined - using the . is IETF review, otherwise specification required. - added reference on parsing expression grammar - App B: Fixed a comment from Jim + "h" and "b64" expression case insen= sitive. One discussion about spacing in the mailing list (see mail from Car= sten from a couple of days ago), but a part from that spacing should be cor= rect now. - Editorial comments Barry: what of these ones need to be solved before we go to publication Carsten: only one is the syntax (spacing before the type), the others are c= overed (or for future version). Barry: proposal makes sense. Carsten: Probably need 2 more hours of testing to make sure it does not bre= ak anything. # Summary Barry: update -04 in the next couple of days (by the end of the week), then= we'll start WGLC. CBORbis: on the agenda for next meeting (in two weeks), as well as any WGLC= comments for CDDL. --_000_AM5PR0701MB273766AD3A8AC4ABB6EFC57698260AM5PR0701MB2737_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi all,

 

I took some notes from the call today, where Carsten= gave us an update of the status of CDDL.

Participants, feel free to add anything I might have= missed.

 

Thanks,
Francesca

 

--

 

= CBOR WG Conference Call

= Wednesday, Aug 8, 2018, 15:00 - 16:00 UTC

= Chairs: Barry Leiba, Francesca Palombini

=  

= Minute takers: Francesca

=  

= Participants:

= * Francesca Palombini

= * Barry Leiba

= * David Brown

= * Laurence Lundblade

= * Michael Richardson

= * Carsten Bormann

=  

= # CDDL

=  

= Barry: Goal of this meeting is to make sure the doc progresses. Carsten has= made updates in the github, not updated in the datatracker yet.=

=  

= Carsten: presenting the diff between last draft (-03) and github versions. = All formal comments should be here (github editor version), might need some= polishing

=     - Standard Prelude moved to appendix D (Jim's comment)

=     - number of changes are editorials that are still in the= document (Jim's comment)

=     - one objective is still there that we decided not to do= , number of changes related to that

=     - in terminology: explicit def of term byte

=     - section 2.1 explains how group work following issue 14=

=     - fixes to figure  

=     - 2.2.2 explicit about delivery

=     - 2.2.2.1 explicit ranges are either between integers or= floating + control operators

=     - 2.2.3 representation types, comments welcome if the te= xt is clear

=     - 3.5.3 confusion about cut means, trying to make this c= learer

=     - 3.8 control operators have fw considerations to iana r= egistry

=     - .size is more explicitly explained what it is for=

=     - 3.8.6 .ne and .eq : leaving specification as people ha= ve been using it but explicit

=     - 3.10 Jim's comments about generic rules (+ editori= al)

=     - 3.11 encourage people to use parenthesis when preceden= ce is not clear

=     - Jim's noticed tilde was missing in the operator table,= fixed, and amp as well

=     - CDDL operator registry defined

=     - using the . is IETF review, otherwise specification re= quired.

=     - added reference on parsing expression grammar

=     - App B: Fixed a comment from Jim + "h" an= d "b64" expression case insensitive. One discussion about spacing= in the mailing list (see mail from Carsten from a couple of days ago), but= a part from that spacing should be correct now.

=     - Editorial comments

= Barry: what of these ones need to be solved before we go to publication

= Carsten: only one is the syntax (spacing before the type), the others are c= overed (or for future version).

= Barry: proposal makes sense.

= Carsten: Probably need 2 more hours of testing to make sure it does not bre= ak anything.

=  

= # Summary

=  

= Barry: update -04 in the next couple of days (by the end of the week), then= we'll start WGLC.

=  

= CBORbis: on the agenda for next meeting (in two weeks), as well as any WGLC= comments for CDDL.

--_000_AM5PR0701MB273766AD3A8AC4ABB6EFC57698260AM5PR0701MB2737_-- From nobody Thu Aug 9 03:23:14 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D5B28130E04 for ; Thu, 9 Aug 2018 03:23:12 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.199 X-Spam-Level: X-Spam-Status: No, score=-4.199 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1JaC2c_t68ya for ; Thu, 9 Aug 2018 03:23:10 -0700 (PDT) Received: from haggis.mythic-beasts.com (haggis.mythic-beasts.com [IPv6:2a00:1098:0:86:1000:0:2:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C363412F1AB for ; Thu, 9 Aug 2018 03:23:10 -0700 (PDT) Received: from [2001:b98:204:102:fff1::11] (port=61518 helo=Jims-iMac.local) by haggis.mythic-beasts.com with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1fni5x-0000AJ-9v for cbor@ietf.org; Thu, 09 Aug 2018 11:23:09 +0100 To: cbor@ietf.org From: Jim Hague Openpgp: preference=signencrypt Autocrypt: addr=jim@sinodun.com; prefer-encrypt=mutual; keydata= xsFNBFjma08BEADHz5x1FKpV7GGMzaXlnC88O+f9iJjkajqdk8YJFS9GBB4X0RnJvew5Ek/Q yTbalhhg8lkMcDkJlW6/l+XPuwTeChLdF83c8yUW49gJ1xN2YP0pew38JzXUpeHC/JLY1FYb canohDqWojY/chE4tscGKehmmHRkneTdtZgreCNMi4cpqu4PsWiJkbBLwRlTTtq0eIUBLCcF 3/Q8Um3SXuKcXhO9XVSNIP/wHUKkZcIBJ2ZWIQYHefuos56BAZOuaV5lPkgaAcb/o72hyU+l yKF87J/srK0PIXK9WcyOYW77N8XDjr9z9W9YIPVrYfX/9DuwibJd9KfdN36T7MbYZW4sensn FsJ9SRhv7cCAcpYCLQtzVYaJ8d8OY4UJSpV3uXH/TZ0lmIH2h0M9m+5LxjyFkoeR1Dbb+OO5 P7X+lx+7kNz34fbwFoXgSWqCPOtJLHR4qejUG3oXGgfi4rBC2336Fv26og27j7TVwRlLeYF1 79/drAcsNS2SiH4v2/Frf44nKh0f6GlufcZVJSnXviMzubDedgTWsqqf8t2pjHDwWTfcGa1s plQ5rClRv43aa/QLC4+lid1MA3bHgtufPDZWz1kFmEMm+lAuFNzoQakcWm8OweN5s/MgkUFj lV/VtvmsrIt9D3BL4F5fxDG9hWwOOIwUXiCRAg57XYdTqTwa6wARAQABzRxKaW0gSGFndWUg PGppbUBsdW5jaC5vcmcudWs+wsF6BBMBCgAkAhsDBQsJCAcDBRUKCQgLBRYCAwEAAh4BAheA BQJY5mwQAhkBAAoJEO4RoliWIemRZnEQAJMfSE1ylcKl22uaFKyE2Yg5sH0FyEdTiA8cFr2w 3Xc05Aqpv2MDko39iUfqR9AROPZNyHJtVrWrG4xicEVI/ZVHq5ujtFgllHWeoTD9X3LzvvJr 6Xogh6ZHtuJkRW05sA6dz0J/eD1bGAi7vcE0/hUo7+zpaUxNMXhn9eSjMLcSNPTfCRbeLSZb 1ooe88jNzJ5oDHz8+rCckTe38Qea+jGI0+EeG5fZkRGZiNcOV7xEcjJPRRsANX71oufj+7ZH rYAF0zSst+W29siAxL/rpMcZzqakt8bt25uMvbLlqgusLYeyKKkNRPaGYmqVzekWL2OceaLf yKr05X2LsZqcvgDfnpK0XJqOqztHudQdgutyhuzsla1JEV8TBg0Siqv7i7/osGo36EO5+FTr BQmX8tqQUiGTAOuEowr9UxznBlflGTrwd1Nfln6bNeGaAFs9k9yPQSlbnnY+VAU0fuffGYt/ Yyp3r3R+EyBWsejbXiba0/wBNczDqaJPJAhnswel1qSV1Aw6UyvG673ewLwqPdI+qz67x7yO K0MRTDW/5Vec3MyTBHjcycbkvP9gr1hyNZhHLUiK3/vfzhocMUGzRYpFuF82r/ouPuFgRsC7 KHVh2LKJk7tKvxB3ozpnHv8GAgQ3GvFVrGIzzYxF3ZL3mVJO1JNH+cJU84F9RRgErdcVzsFN BFjma08BEADxnGIWD0IRk/SMBL/nIebJN2i86SNLmjGWBbzM/DAIFylRrxzGL3eewsPd15Va IToMb7OsxK22aAXv/CqoTQFpLvNIV3gKuSvUiUun2Aas9V3yKFWnjyvmJpSzWIfhr+IsBdC5 StlQtgVlk9jcFoIpqaVTgPK8DUZzrWKEHObqeUQde9rbqlbL3wUiYaAsC9R2JQrdoSEG8fhS jdmDGicTFJPJdyoaHQz/YhyqpH5aEs5eLTgtWMeNRkgBH7wgJmH0Gn8oSHEf4JVmSdz+TgWK DaHvoP9KgoLOZEK3Q1pCDT6/EQgo9B07nej1e22ld7JGEbVCWy9IeQrOZ95YHypWiqXyQsi4 vYTp4bYhLqW4aHZJeF2Ic+3sTng53OrV5oqL4ExuIYbNbG/6To7xxvcnVlQme5x7bNbPuiZy rPyj8Cid0xI0FuWiGOh9v0nEC0zVTaAwjX82h5f9f0USYRfdYTIkoQiVE57kCQdNF7aJnJdB lxWIxKlrsfN2WgMVv4NBSgQCbq6uZF3bHPfAhF82j47JeC1sFkRnJf5y7EsKNkvT++1J6bRJ 0qD6WC+1v53iMh3Nx63/F4a7L3iBY6Q/4ITUPzA0OkVqw4oqHHgIsqx18pIzeDgCYMK7d/Vx awWiFsNNExtIpTjsSYJyxB9rOJO3yccVjxO1C1pExwMXTQARAQABwsFfBBgBCgAJBQJY5mtP AhsMAAoJEO4RoliWIemRBRUP/3LMVCCtcvHUAd61nkr2LTSPQW7Y9+he/BU672mUp3OPrtL/ wF3Cg+JQR30bJC6ztY5wAWrKoB8A380GmhhBa4havapzfp+vXvHxU4LW+ie/lE0sGneRDN1t wGkvhLRH6NGhjOEd5zWDm4D/zOzubBB5FZ4E70yadzZmzsQC7XIk28WfGDHysHJTEMmOklWy UEMF6oWvygaEMHC/lgxXYORYDA5LeElOtPbvHeRaazX74yUCTaA7w4810Dxd7aXMS+7yK3/e 2qucM6VVWb5O7bdKckJBrBxAe7BpzH2in+nA2TDoMQRCukGsnOQreQX8ulQVDC3ZEmvHmErd NHVBrcbxb5hBrnykNHy7jkNqxsS5xNJqfdo0Xa0NdD5QIZGydF4Q2J102f1eB3FpzGCugx+z sPdMFv0YKEBfS00GwCZepv/2TS0fMRwgVsjKuxkTFWklIgOWIiDrq8taSS5VXiOzxWmG0/CK XUXRviyrZ7ATwI+CqJvnbKVat1Tzu2AjtsY/jtCOpDsfwNrESQllrh4LIclNi5EojBC+h5H9 6bByZCh2/UszlhlEewHglNXSAXElQJRmC/R8ON0uHU30ZcWTzS1ohTTz1Rzo2Y27A54OxXRM IjMZr6g+IfF9Ya5kMWMIK/+c7y0Q2YWmpCO0cAWQjYN0TPWSQ0H08pfNFG9P Organization: Sinodun Internet Technologies Ltd. Message-ID: <8df323d4-16b5-1415-50ee-15964e996b0f@sinodun.com> Date: Thu, 9 Aug 2018 11:23:03 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-GB Content-Transfer-Encoding: 8bit X-BlackCat-Spam-Score: 0 Archived-At: Subject: [Cbor] CDDL: expressing allowed values in bitfield X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Aug 2018 10:23:13 -0000 I'm trying to specify a bitfield in CDDL which contains flags and a value, which is one of a specified set. An example C structure: enum Vals{A=0, B, C, D}; struct Field { unsigned flag1 : 1; enum Vals val : 4; /* 2 bits spare for future new Vals */ unsigned flag2 : 1; }; So I'm using the .bits control: Vals = &( A: 0, B: 1, C: 2, D: 3 ) FieldBits = &( flag1 : 0, flag2 : 5) / (1..4) Field = uint .bits FieldBits Is there a cunning way to express the restricted value set expected in bits 1..4 in CDDL? Thanks. -- Jim Hague - jim@sinodun.com Never trust a computer you can't lift. From nobody Thu Aug 9 04:10:00 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 97866130E1B for ; Thu, 9 Aug 2018 04:09:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.199 X-Spam-Level: X-Spam-Status: No, score=-4.199 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eodGiTULuiLb for ; Thu, 9 Aug 2018 04:09:56 -0700 (PDT) Received: from mailhost.informatik.uni-bremen.de (mailhost.informatik.uni-bremen.de [IPv6:2001:638:708:30c9::12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3A974130E17 for ; Thu, 9 Aug 2018 04:09:56 -0700 (PDT) X-Virus-Scanned: amavisd-new at informatik.uni-bremen.de Received: from submithost.informatik.uni-bremen.de (submithost.informatik.uni-bremen.de [134.102.201.11]) by mailhost.informatik.uni-bremen.de (8.14.5/8.14.5) with ESMTP id w79B9rix019174; Thu, 9 Aug 2018 13:09:53 +0200 (CEST) Received: from [192.168.217.114] (p54A6C84F.dip0.t-ipconnect.de [84.166.200.79]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by submithost.informatik.uni-bremen.de (Postfix) with ESMTPSA id 41mQWh69z1zDXPQ; Thu, 9 Aug 2018 13:09:52 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) From: Carsten Bormann In-Reply-To: <8df323d4-16b5-1415-50ee-15964e996b0f@sinodun.com> Date: Thu, 9 Aug 2018 13:09:52 +0200 Cc: cbor@ietf.org X-Mao-Original-Outgoing-Id: 555505790.4378181-1a109f230775f5044c4a0d9779298dbb Content-Transfer-Encoding: quoted-printable Message-Id: <6B50A906-DD20-4322-BE1E-EFABD06DB3FB@tzi.org> References: <8df323d4-16b5-1415-50ee-15964e996b0f@sinodun.com> To: Jim Hague X-Mailer: Apple Mail (2.3445.9.1) Archived-At: Subject: Re: [Cbor] CDDL: expressing allowed values in bitfield X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Aug 2018 11:09:59 -0000 Hi Jim, great question. CDDL has limited capabilities to =E2=80=9Creach into=E2=80=9D the = (non-container) component data. We have ranges and .lt etc. for numeric control. We have regexes for text strings, but there is a lot of interest to be = able to embed/relate to ABNF for reaching into text strings. We have .bits for byte strings (and uints that essentially serve as = such), and we could try to come up with something ABNF-like for that as = well. Say: Field =3D uint .bitfield Fieldbits Fieldbits =3D [ flag1: [1, bool], val: [4, Vals], flag2: [1, bool], ] Vals =3D &(A: 0, B: 1, C: 2, D: 3) This is just a strawman, but can serve as a proof of concept how this = could be added as a control. If we want, we can start developing this = right away and then register it into CDDL 1.0. I have put this into the = freezer document for now=E2=80=A6 Gr=C3=BC=C3=9Fe, Carsten > On Aug 9, 2018, at 12:23, Jim Hague wrote: >=20 > I'm trying to specify a bitfield in CDDL which contains flags and a > value, which is one of a specified set. An example C structure: >=20 > enum Vals{A=3D0, B, C, D}; > struct Field { > unsigned flag1 : 1; > enum Vals val : 4; /* 2 bits spare for future new Vals */ > unsigned flag2 : 1; > }; >=20 > So I'm using the .bits control: >=20 > Vals =3D &( > A: 0, > B: 1, > C: 2, > D: 3 > ) > FieldBits =3D &( > flag1 : 0, > flag2 : 5) / (1..4) > Field =3D uint .bits FieldBits >=20 > Is there a cunning way to express the restricted value set expected in > bits 1..4 in CDDL? >=20 > Thanks. > --=20 > Jim Hague - jim@sinodun.com Never trust a computer you can't = lift. >=20 > _______________________________________________ > CBOR mailing list > CBOR@ietf.org > https://www.ietf.org/mailman/listinfo/cbor >=20 From nobody Thu Aug 9 09:02:42 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 40DA9130E4C for ; Thu, 9 Aug 2018 09:02:40 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.199 X-Spam-Level: X-Spam-Status: No, score=-4.199 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AIVKiQg66K3v for ; Thu, 9 Aug 2018 09:02:37 -0700 (PDT) Received: from balrog.mythic-beasts.com (balrog.mythic-beasts.com [IPv6:2a00:1098:0:82:1000:0:2:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A76AF12777C for ; Thu, 9 Aug 2018 09:02:37 -0700 (PDT) Received: from [2001:b98:204:102:fff1::11] (port=63182 helo=Jims-iMac.local) by balrog.mythic-beasts.com with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1fnnOS-0004U9-4X; Thu, 09 Aug 2018 17:02:36 +0100 To: Carsten Bormann Cc: cbor@ietf.org References: <8df323d4-16b5-1415-50ee-15964e996b0f@sinodun.com> <6B50A906-DD20-4322-BE1E-EFABD06DB3FB@tzi.org> From: Jim Hague Openpgp: preference=signencrypt Autocrypt: addr=jim@sinodun.com; prefer-encrypt=mutual; keydata= xsFNBFjma08BEADHz5x1FKpV7GGMzaXlnC88O+f9iJjkajqdk8YJFS9GBB4X0RnJvew5Ek/Q yTbalhhg8lkMcDkJlW6/l+XPuwTeChLdF83c8yUW49gJ1xN2YP0pew38JzXUpeHC/JLY1FYb canohDqWojY/chE4tscGKehmmHRkneTdtZgreCNMi4cpqu4PsWiJkbBLwRlTTtq0eIUBLCcF 3/Q8Um3SXuKcXhO9XVSNIP/wHUKkZcIBJ2ZWIQYHefuos56BAZOuaV5lPkgaAcb/o72hyU+l yKF87J/srK0PIXK9WcyOYW77N8XDjr9z9W9YIPVrYfX/9DuwibJd9KfdN36T7MbYZW4sensn FsJ9SRhv7cCAcpYCLQtzVYaJ8d8OY4UJSpV3uXH/TZ0lmIH2h0M9m+5LxjyFkoeR1Dbb+OO5 P7X+lx+7kNz34fbwFoXgSWqCPOtJLHR4qejUG3oXGgfi4rBC2336Fv26og27j7TVwRlLeYF1 79/drAcsNS2SiH4v2/Frf44nKh0f6GlufcZVJSnXviMzubDedgTWsqqf8t2pjHDwWTfcGa1s plQ5rClRv43aa/QLC4+lid1MA3bHgtufPDZWz1kFmEMm+lAuFNzoQakcWm8OweN5s/MgkUFj lV/VtvmsrIt9D3BL4F5fxDG9hWwOOIwUXiCRAg57XYdTqTwa6wARAQABzRxKaW0gSGFndWUg PGppbUBsdW5jaC5vcmcudWs+wsF6BBMBCgAkAhsDBQsJCAcDBRUKCQgLBRYCAwEAAh4BAheA BQJY5mwQAhkBAAoJEO4RoliWIemRZnEQAJMfSE1ylcKl22uaFKyE2Yg5sH0FyEdTiA8cFr2w 3Xc05Aqpv2MDko39iUfqR9AROPZNyHJtVrWrG4xicEVI/ZVHq5ujtFgllHWeoTD9X3LzvvJr 6Xogh6ZHtuJkRW05sA6dz0J/eD1bGAi7vcE0/hUo7+zpaUxNMXhn9eSjMLcSNPTfCRbeLSZb 1ooe88jNzJ5oDHz8+rCckTe38Qea+jGI0+EeG5fZkRGZiNcOV7xEcjJPRRsANX71oufj+7ZH rYAF0zSst+W29siAxL/rpMcZzqakt8bt25uMvbLlqgusLYeyKKkNRPaGYmqVzekWL2OceaLf yKr05X2LsZqcvgDfnpK0XJqOqztHudQdgutyhuzsla1JEV8TBg0Siqv7i7/osGo36EO5+FTr BQmX8tqQUiGTAOuEowr9UxznBlflGTrwd1Nfln6bNeGaAFs9k9yPQSlbnnY+VAU0fuffGYt/ Yyp3r3R+EyBWsejbXiba0/wBNczDqaJPJAhnswel1qSV1Aw6UyvG673ewLwqPdI+qz67x7yO K0MRTDW/5Vec3MyTBHjcycbkvP9gr1hyNZhHLUiK3/vfzhocMUGzRYpFuF82r/ouPuFgRsC7 KHVh2LKJk7tKvxB3ozpnHv8GAgQ3GvFVrGIzzYxF3ZL3mVJO1JNH+cJU84F9RRgErdcVzsFN BFjma08BEADxnGIWD0IRk/SMBL/nIebJN2i86SNLmjGWBbzM/DAIFylRrxzGL3eewsPd15Va IToMb7OsxK22aAXv/CqoTQFpLvNIV3gKuSvUiUun2Aas9V3yKFWnjyvmJpSzWIfhr+IsBdC5 StlQtgVlk9jcFoIpqaVTgPK8DUZzrWKEHObqeUQde9rbqlbL3wUiYaAsC9R2JQrdoSEG8fhS jdmDGicTFJPJdyoaHQz/YhyqpH5aEs5eLTgtWMeNRkgBH7wgJmH0Gn8oSHEf4JVmSdz+TgWK DaHvoP9KgoLOZEK3Q1pCDT6/EQgo9B07nej1e22ld7JGEbVCWy9IeQrOZ95YHypWiqXyQsi4 vYTp4bYhLqW4aHZJeF2Ic+3sTng53OrV5oqL4ExuIYbNbG/6To7xxvcnVlQme5x7bNbPuiZy rPyj8Cid0xI0FuWiGOh9v0nEC0zVTaAwjX82h5f9f0USYRfdYTIkoQiVE57kCQdNF7aJnJdB lxWIxKlrsfN2WgMVv4NBSgQCbq6uZF3bHPfAhF82j47JeC1sFkRnJf5y7EsKNkvT++1J6bRJ 0qD6WC+1v53iMh3Nx63/F4a7L3iBY6Q/4ITUPzA0OkVqw4oqHHgIsqx18pIzeDgCYMK7d/Vx awWiFsNNExtIpTjsSYJyxB9rOJO3yccVjxO1C1pExwMXTQARAQABwsFfBBgBCgAJBQJY5mtP AhsMAAoJEO4RoliWIemRBRUP/3LMVCCtcvHUAd61nkr2LTSPQW7Y9+he/BU672mUp3OPrtL/ wF3Cg+JQR30bJC6ztY5wAWrKoB8A380GmhhBa4havapzfp+vXvHxU4LW+ie/lE0sGneRDN1t wGkvhLRH6NGhjOEd5zWDm4D/zOzubBB5FZ4E70yadzZmzsQC7XIk28WfGDHysHJTEMmOklWy UEMF6oWvygaEMHC/lgxXYORYDA5LeElOtPbvHeRaazX74yUCTaA7w4810Dxd7aXMS+7yK3/e 2qucM6VVWb5O7bdKckJBrBxAe7BpzH2in+nA2TDoMQRCukGsnOQreQX8ulQVDC3ZEmvHmErd NHVBrcbxb5hBrnykNHy7jkNqxsS5xNJqfdo0Xa0NdD5QIZGydF4Q2J102f1eB3FpzGCugx+z sPdMFv0YKEBfS00GwCZepv/2TS0fMRwgVsjKuxkTFWklIgOWIiDrq8taSS5VXiOzxWmG0/CK XUXRviyrZ7ATwI+CqJvnbKVat1Tzu2AjtsY/jtCOpDsfwNrESQllrh4LIclNi5EojBC+h5H9 6bByZCh2/UszlhlEewHglNXSAXElQJRmC/R8ON0uHU30ZcWTzS1ohTTz1Rzo2Y27A54OxXRM IjMZr6g+IfF9Ya5kMWMIK/+c7y0Q2YWmpCO0cAWQjYN0TPWSQ0H08pfNFG9P Organization: Sinodun Internet Technologies Ltd. Message-ID: <504b2483-9863-280c-cd4f-98f6e8509632@sinodun.com> Date: Thu, 9 Aug 2018 17:02:25 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <6B50A906-DD20-4322-BE1E-EFABD06DB3FB@tzi.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-GB Content-Transfer-Encoding: 8bit X-BlackCat-Spam-Score: 0 Archived-At: Subject: Re: [Cbor] CDDL: expressing allowed values in bitfield X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Aug 2018 16:02:41 -0000 On 09/08/2018 12:09, Carsten Bormann wrote: > CDDL has limited capabilities to “reach into” the (non-container) component data. > We have ranges and .lt etc. for numeric control. > We have regexes for text strings, but there is a lot of interest to be able to embed/relate to ABNF for reaching into text strings. > We have .bits for byte strings (and uints that essentially serve as such), and we could try to come up with something ABNF-like for that as well. > > Say: > > Field = uint .bitfield Fieldbits > > Fieldbits = [ > flag1: [1, bool], > val: [4, Vals], > flag2: [1, bool], > ] > > Vals = &(A: 0, B: 1, C: 2, D: 3) > > This is just a strawman, but can serve as a proof of concept how this could be added as a control. If we want, we can start developing this right away and then register it into CDDL 1.0. I have put this into the freezer document for now… I'm happy to hear I didn't miss something. The above looks an excellent approach. I'd certainly like to be able to specify types for bitfields; even with a field that's only a single bit, it can be valuable to distinguish a numeric value that can only be 0 or 1 from a boolean. -- Jim Hague - jim@sinodun.com Never trust a computer you can't lift. From nobody Thu Aug 9 19:46:04 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A08A9130DDC for ; Thu, 9 Aug 2018 19:46:01 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.899 X-Spam-Level: X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VETxbQ9DiHo7 for ; Thu, 9 Aug 2018 19:45:58 -0700 (PDT) Received: from p3plsmtpa09-08.prod.phx3.secureserver.net (p3plsmtpa09-08.prod.phx3.secureserver.net [173.201.193.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 732C01294D0 for ; Thu, 9 Aug 2018 19:45:58 -0700 (PDT) Received: from [10.12.102.53] ([66.228.17.5]) by :SMTPAUTH: with ESMTPSA id nxR6fpULZvKHXnxR6fACuE; Thu, 09 Aug 2018 19:45:57 -0700 From: Laurence Lundblade Message-Id: Content-Type: multipart/alternative; boundary="Apple-Mail=_2AE9FCA3-5D64-462F-8DC3-A0E8757E8DB1" Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Date: Thu, 9 Aug 2018 19:45:56 -0700 In-Reply-To: <3A7B27E3-9DCD-48D3-BDEB-BDA41E7AD1B1@tzi.org> Cc: cbor@ietf.org, Sean Solberg To: Carsten Bormann References: <80DFAEF6-BE25-4F43-9CCF-A8FCBEB1F4BE@island-resort.com> <3101844.7X5zPJ143S@tjmaciei-mobl1> <3B65AA30-5A89-47DD-8563-57EDA6653BBD@island-resort.com> <4FE5660E-0E53-4445-9863-8016454AEBF6@tzi.org> <46A693AD-5AF4-4A84-8A43-3575D10F705E@island-resort.com> <87170077-3229-4FA2-A179-2A7F979951B0@island-resort.com> <3A7B27E3-9DCD-48D3-BDEB-BDA41E7AD1B1@tzi.org> X-Mailer: Apple Mail (2.3445.9.1) X-CMAE-Envelope: MS4wfBsZjN2U9fhmA2V71NxH4QSeL75KkDaNp4pvttZdQQcl++AWQkxBZTQ6Qo45h5KObhXm9kbqpDfCOQpOZGXwqbW/PAOwULm/6KMv7+d2qpO+iW+fmzTm G34ukTqrfxPCGOSTeEubLrROYi8OKUFOgsZiXVVUR65ZXeANiY1UyVuxhbPfyPh75qNxEh8z9iWYapJ1tingyR/0mAkBcxbP/Pg= Archived-At: Subject: Re: [Cbor] Preferred encoding of floats X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Aug 2018 02:46:02 -0000 --Apple-Mail=_2AE9FCA3-5D64-462F-8DC3-A0E8757E8DB1 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 This thread is getting a bit frayed, but will try to go a bit further = with it. > On Aug 7, 2018, at 3:37 PM, Carsten Bormann wrote: >=20 > Hi Laurence, >=20 > thanks for keeping up the discussion of floats; they are often = neglected in circles like the IETF (say, YANG doesn=E2=80=99t not even = have them!) and we have to get this right. >=20 >>> I don=E2=80=99t think we want to have =E2=80=9Cpreferred decoders=E2=80= =9D and =E2=80=9Cnon-preferred decoders=E2=80=9D. A decoder implements = all of CBOR or an application-specific subset. =E2=80=9CAll of CBOR=E2=80= =9D is certainly preferred for what tries to be a generic decoder, but = that is so obvious that it probably does not need to be stated. >>=20 >> All of CBOR is too much to expect any majority of decoders and = encoders to implement. I kind of took this the wrong direction by discussing types. I should = have stuck to serialization since that=E2=80=99s the focus of the PR.=20 The PR does say say that a = preferred decoder should implement all serializations defined by CBOR. = It lists them out one by one and says a preferred decoder must implement = them.=20 So we agree on what a preferred decoder should do if we are to have one.=20= I am in favor of describing the preferred decoder in the RFC. I think = some decoders don=E2=80=99t implement all serializations. We want to = encourage them to for much better interop. CBOR libraries can also say = the support preferred and that helps completeness and interop. >=20 > Actually, since anyone can *extend* CBOR at any time by defining = another Tag, that is an impossible expectation. But we can describe = what a full generic CBOR decoder would do: >=20 > 1 =E2=80=94 decode any valid CBOR, > 2 =E2=80=94 present all information in there that goes beyond = representation variants to the application. >=20 > Now, number 2 is the fun part. Is the difference between 1 and = 2(h=E2=80=9901=E2=80=99) just a representation variant? > For many implementations, it will be. However, implementations that = are stuck with a BigInt that isn=E2=80=99t even numeric may think this = is a valid distinction (e.g., a JavaScript implementation might want to = encode 1n as 2(h=E2=80=9901=E2=80=99) to distinguish it from 1). = We=E2=80=99ll need to make some taste decisions there. >=20 >> Somethings like float, double and half are a little hard to figure = out (as I am discovering). >=20 > I just extracted the float decoder implementation from my Elixir code. = (This code is a bit more complicated than needed because Erlang does = not handle nonfinites, so there is about twice as much code as needed.) >=20 > =E2=80=A6 > 7 -> case bin do > << 0xf9, sign::size(1), exp::size(5), mant::size(10), > _::binary >> -> {decode_half(sign, exp, mant, = treatment), rest} > << 0xfa, value::[float, size(32)], _::binary >> -> = {value, rest} > << 0xfa, sign::size(1), 255::size(8), mant::size(23), > _::binary >> -> {treatment.nonfinites.(sign, mant), = rest} > << 0xfb, value::[float, size(64)], _::binary >> -> = {value, rest} > << 0xfb, sign::size(1), 2047::size(11), = mant::size(52), > _::binary >> -> {treatment.nonfinites.(sign, mant), = rest} > =E2=80=A6 > defp decode_half(sign, 31, mant, treatment), do: = treatment.nonfinites.(sign, mant) > defp decode_half(sign, exp, mant, _) do > << value::[float, size(32)] >> =3D > << sign::size(1), exp::size(8), mant::size(10), 0::size(13) >> > value * 5192296858534827628530496329220096.0 # 2**112 =E2=80=94 = difference in bias >=20 > Just showing that this is not particularly hard even in a language = that doesn=E2=80=99t give you a lot of tools to handle floating point. Well it was hard for me to do the thorough job I wanted to do. :-). I = agree the code isn=E2=80=99t very long. I had to learn a lot to support = half-precision. Then I had to learn about denormal and NaN payloads to = know if the job was really done. >=20 >> Other things like bignums seem useful for a very very narrow set of = use cases (they aren=E2=80=99t even used for COSE keys). >=20 > The JavaScript people are just now finding out that they were lacking = them all the time=E2=80=A6 > It is just needed for many applications that can=E2=80=99t survive = artificial limitations. >=20 > (Each of us has a certain set of experience behind their tastes, and = it is too easy to fall into the trap that just because something does = not occur in that experience that it is unimportant=E2=80=A6) >=20 >> Since we can=E2=80=99t expect =E2=80=9Call=E2=80=9D from the majority = of decoders, all we have is application-specific subsets. That=E2=80=99s = a little hard on those of us working on generic libraries.=20 Like I said above, I kind of take this back. We should expect decoders = to implement all serializations and might want to call those preferred = decoders. >=20 > I would expect generic libraries to differ a lot between differently = capable programming environments. > A Lua implementation will always be more limited than a Haskell one. > I don=E2=80=99t think we can normalize that. But we can make sure = that all full generic implementations at least enable applications to = handle some types (awkwardly) that the programing environment doesn=E2=80=99= t. Like the ugly nonfinites above=E2=80=A6 >=20 >> My PR is basically proposing that there be a =E2=80=9Cpreferred=E2=80=9D= CBOR subset that will cover the vast majority of use cases and that all = uses of this preferred subset are guaranteed to interoperate. To do this = the preferred subset must apply to decoding as well as encoding.=20 >=20 > That would be the I-JSON of CBOR. Meaning the other half of CBOR can = no longer be used. We don=E2=80=99t have to make that mistake, I think. >=20 >> This notion of =E2=80=9Cpreferred" makes it easy for protocol = designers. They just stick to the preferred set which is should be = enough for most use cases. They don=E2=80=99t have to think too hard = about interop. It makes it easy for generic library implementors and it = make it easy for an application to select a library. Protocol = implementors just design for preferred and select a library that is = preferred. It should also make translation to JSON more uniform.=20 >=20 > More shades of I-JSON=E2=80=A6 >=20 >> I assume NaN boxing and NaN payloads are the same thing.=20 >=20 > Yes. >=20 >> NaN boxing/payloads is not compatible with a canonical / preferred = representation NaN. You have to pick one or the other.=20 >=20 > An application could define their =E2=80=9Ccanonical=E2=80=9D with the = right way of NaN handling. CBOR definitely can transport NaN payloads. >=20 >> So in CBOR we can choose one of these paths: >> A) Always use the one single representation of NaN and entirely = disallow boxing/payloads >> B) In preferred/canonical CBOR use the one single representation, = which makes boxing/payloads work for regular, but not = canonical/preferred CBOR >> C) Have no recommendation at all. NaNs are just carried as they come = (like denormals) >>=20 >> My preference is for A), but could go with C) >> - I like sparing users of CBOR the work of having to figure out more = weird stuff like NaN boxing. NaN boxing/payloads seems like a very = marginal feature.=20 >> - I think B) is weird and less interoperable because sometimes NaN = boxing/payloads work in CBR and sometimes they do not. >> - I think C) is not as crisp as A) >=20 > I=E2=80=99m leaning the same way, but I still think it is valid for an = application to go ahead and use the full capabilities of the CBOR = serialization (e.g., to carry around NaN payloads). It is probably a = mistake in my implementation that I always canonicalize NaNs in the = encoder =E2=80=94 I should only do this when canonical encoding was = requested. I still think that it is good to turn both = 0xFB7FF8000000000000 and 0xFA7FC00000 into 0xF97E00 in the generic = encoder. Supporting NaN payloads in C/native on a CPU that supports IEEE 754 is = seems pretty easy. It is actually less code than A) too. But what about = other languages and environments? That would push me towards A) instead = of C), though I suspect in the end use of NaN payloads is extremely rare = and this ends up not mattering very much.=20 LL >=20 > Gr=C3=BC=C3=9Fe, Carsten >=20 > _______________________________________________ > CBOR mailing list > CBOR@ietf.org > https://www.ietf.org/mailman/listinfo/cbor --Apple-Mail=_2AE9FCA3-5D64-462F-8DC3-A0E8757E8DB1 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 This = thread is getting a bit frayed, but will try to go a bit further with = it.

On Aug 7, 2018, at 3:37 PM, Carsten Bormann = <cabo@tzi.org> = wrote:

Hi Laurence,

thanks for keeping = up the discussion of floats; they are often neglected in circles like = the IETF (say, YANG doesn=E2=80=99t not even have them!) and we have to = get this right.

I don=E2=80=99t think we = want to have =E2=80=9Cpreferred decoders=E2=80=9D and =E2=80=9Cnon-preferr= ed decoders=E2=80=9D.  A decoder implements all of CBOR or an = application-specific subset.  =E2=80=9CAll of CBOR=E2=80=9D is = certainly preferred for what tries to be a generic decoder, but that is = so obvious that it probably does not need to be stated.

All of CBOR is too much to expect = any majority of decoders and encoders to = implement.

I kind of took this the wrong direction by = discussing types. I should have stuck to serialization since that=E2=80=99= s the focus of the PR. 

The PR does say say that a preferred decoder = should implement all serializations defined by CBOR. = It lists them out one by one and says a preferred decoder must = implement them. 

So we agree on = what a preferred decoder should do if we are to have = one. 

I am in favor of = describing the preferred decoder in the RFC. I think some decoders = don=E2=80=99t implement all serializations. We want to encourage them to = for much better interop. CBOR libraries can also say the support = preferred and that helps completeness and interop.



Actually, since anyone can = *extend* CBOR at any time by defining another Tag, that is an impossible = expectation.  But we can describe what a full generic CBOR decoder = would do:

1 =E2=80=94 decode any valid = CBOR,
2 =E2=80=94 present all information in there that = goes beyond representation variants to the application.

Now, number 2 is the fun part.  Is the difference = between 1 and 2(h=E2=80=9901=E2=80=99) just a representation variant?
For many implementations, it will be.  However, = implementations that are stuck with a BigInt that isn=E2=80=99t even = numeric may think this is a valid distinction (e.g., a JavaScript = implementation might want to encode 1n as 2(h=E2=80=9901=E2=80=99) to = distinguish it from 1).  We=E2=80=99ll need to make some taste = decisions there.

Somethings like float, double and half are a little hard to = figure out (as I am discovering).

I just extracted the float decoder implementation from my = Elixir code.  (This code is a bit more complicated than needed = because Erlang does not handle nonfinites, so there is about twice as = much code as needed.)

 =E2=80=A6
         7 = -> case bin do
=             &n= bsp;   << 0xf9, sign::size(1), exp::size(5), = mant::size(10),
=             &n= bsp;      _::binary >> -> = {decode_half(sign, exp, mant, treatment), rest}
=             &n= bsp;   << 0xfa, value::[float, size(32)], _::binary = >> -> {value, rest}
=             &n= bsp;   << 0xfa, sign::size(1), 255::size(8), = mant::size(23),
=             &n= bsp;      _::binary >> -> = {treatment.nonfinites.(sign, mant), rest}
=             &n= bsp;   << 0xfb, value::[float, size(64)], _::binary = >> -> {value, rest}
=             &n= bsp;   << 0xfb, sign::size(1), 2047::size(11), = mant::size(52),
=             &n= bsp;      _::binary >> -> = {treatment.nonfinites.(sign, mant), rest}
 =E2=80=A6  defp decode_half(sign, 31, mant, treatment), do: = treatment.nonfinites.(sign, mant)
 defp = decode_half(sign, exp, mant, _) do
=    << value::[float, size(32)] >> =3D
     << sign::size(1), = exp::size(8), mant::size(10), 0::size(13) >>
=    value * 5192296858534827628530496329220096.0 # 2**112 = =E2=80=94 difference in bias

Just showing = that this is not particularly hard even in a language that doesn=E2=80=99t= give you a lot of tools to handle floating point.

Well = it was hard for me to do the thorough job I wanted to do. :-).  I = agree the code isn=E2=80=99t very long.  I had to learn a lot to = support half-precision. Then I had to learn about denormal and NaN = payloads to know if the job was really done.



Other things like bignums seem useful for a very very narrow = set of use cases (they aren=E2=80=99t even used for COSE keys).

The JavaScript people are just = now finding out that they were lacking them all the time=E2=80=A6
It is just needed for many applications that can=E2=80=99t = survive artificial limitations.

(Each of us = has a certain set of experience behind their tastes, and it is too easy = to fall into the trap that just because something does not occur in that = experience that it is unimportant=E2=80=A6)

Since we can=E2=80=99t = expect =E2=80=9Call=E2=80=9D from the majority of decoders, all we have = is application-specific subsets. That=E2=80=99s a little hard on those = of us working on generic libraries.

Like I said above, I kind of take this back. We should = expect decoders to implement all serializations and = might want to call those preferred decoders.


I would expect generic libraries to differ a = lot between differently capable programming environments.
A = Lua implementation will always be more limited than a Haskell one.
I don=E2=80=99t think we can normalize that.  But we can = make sure that all full generic implementations at least enable = applications to handle some types (awkwardly) that the programing = environment doesn=E2=80=99t.  Like the ugly nonfinites above=E2=80=A6=

My PR = is basically proposing that there be a =E2=80=9Cpreferred=E2=80=9D CBOR = subset that will cover the vast majority of use cases and that all uses = of this preferred subset are guaranteed to interoperate. To do this the = preferred subset must apply to decoding as well as encoding.

That would be the I-JSON of CBOR. =  Meaning the other half of CBOR can no longer be used.  We = don=E2=80=99t have to make that mistake, I think.

This notion of = =E2=80=9Cpreferred" makes it easy for protocol designers. They just = stick to the preferred set which is should be enough for most use cases. = They don=E2=80=99t have to think too hard about interop. It makes it = easy for generic library implementors and it make it easy for an = application to select a library. Protocol implementors just design for = preferred and select a library that is preferred. It should also make = translation to JSON more uniform.

More shades of I-JSON=E2=80=A6

I assume NaN boxing and = NaN payloads are the same thing.

Yes.

NaN boxing/payloads is not compatible with a canonical / = preferred representation NaN. You have to pick one or the other.

An application could define their = =E2=80=9Ccanonical=E2=80=9D with the right way of NaN handling. =  CBOR definitely can transport NaN payloads.

So in CBOR we can choose = one of these paths:
A) Always use the one single = representation of NaN and entirely disallow boxing/payloads
B) In preferred/canonical CBOR use the one single = representation, which makes boxing/payloads work for regular, but not = canonical/preferred CBOR
C) Have no recommendation at all. = NaNs are just carried as they come (like denormals)

My preference is for A), but could go with C)
- = I like sparing users of CBOR the work of having to figure out more weird = stuff like NaN boxing. NaN boxing/payloads seems like a very marginal = feature.
- I think B) is weird and less interoperable = because sometimes NaN boxing/payloads work in CBR and sometimes they do = not.
- I think C) is not as crisp as A)

I=E2=80=99m leaning the same way, = but I still think it is valid for an application to go ahead and use the = full capabilities of the CBOR serialization (e.g., to carry around NaN = payloads).  It is probably a mistake in my implementation that I = always canonicalize NaNs in the encoder =E2=80=94 I should only do this = when canonical encoding was requested.  I still think that it is = good to turn both 0xFB7FF8000000000000 and 0xFA7FC00000 into 0xF97E00 in = the generic encoder.

Supporting NaN payloads in C/native on a CPU that = supports IEEE 754 is seems pretty easy. It is actually less code than A) = too.  But what about other languages and environments? That would = push me towards A) instead of C), though I suspect in the end use of NaN = payloads is extremely rare and this ends up not mattering very = much. 

LL






Gr=C3=BC=C3=9Fe, Carsten

_______________________________________________
CBOR mailing list
CBOR@ietf.org
https://www.ietf.org/mailman/listinfo/cbor

= --Apple-Mail=_2AE9FCA3-5D64-462F-8DC3-A0E8757E8DB1-- From nobody Thu Aug 9 23:16:54 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D25D9130EFF for ; Thu, 9 Aug 2018 23:16:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.391 X-Spam-Level: X-Spam-Status: No, score=-1.391 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RAZOR2_CF_RANGE_51_100=1.886, RAZOR2_CHECK=0.922, RCVD_IN_DNSWL_MED=-2.3, URIBL_DBL_ABUSE_REDIR=0.001] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kF3PDPO0o6lV for ; Thu, 9 Aug 2018 23:16:49 -0700 (PDT) Received: from mailhost.informatik.uni-bremen.de (mailhost.informatik.uni-bremen.de [IPv6:2001:638:708:30c9::12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DF562130DFC for ; Thu, 9 Aug 2018 23:16:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at informatik.uni-bremen.de Received: from submithost.informatik.uni-bremen.de (submithost.informatik.uni-bremen.de [IPv6:2001:638:708:30c9::b]) by mailhost.informatik.uni-bremen.de (8.14.5/8.14.5) with ESMTP id w7A6GjMJ004401 for ; Fri, 10 Aug 2018 08:16:45 +0200 (CEST) Received: from client-0059.vpn.uni-bremen.de (client-0059.vpn.uni-bremen.de [134.102.107.59]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by submithost.informatik.uni-bremen.de (Postfix) with ESMTPSA id 41mvz133nGzDXXp; Fri, 10 Aug 2018 08:16:45 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) From: Carsten Bormann Date: Fri, 10 Aug 2018 08:16:44 +0200 X-Mao-Original-Outgoing-Id: 555574602.952009-b3d209e5c1f5546d7584fb50171b3419 Content-Transfer-Encoding: quoted-printable Message-Id: <77BEDB78-0C2D-4BD4-95B4-DB3556B22353@tzi.org> References: <153388117963.28907.13862811574346222830.idtracker@ietfa.amsl.com> To: cbor@ietf.org X-Mailer: Apple Mail (2.3445.9.1) Archived-At: Subject: [Cbor] CDDL -04 ready; new Version for draft-bormann-cbor-cddl-freezer-01.txt X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Aug 2018 06:16:51 -0000 I believe CDDL is now ready for submission as -04; I intend to do this = at 1200Z (I.e., about 5=C2=BE hours from now). If you have a bit of time to look at this, please consult: http://gg.gg/cddl-editor-diff https://github.com/cbor-wg/cddl/ In particular, it may be worth to look at the cleaned up ABNF. I have already submitted an update for the freezer document (see below); = comments and pointers to omissions welcome. Gr=C3=BC=C3=9Fe, Carsten > Name: draft-bormann-cbor-cddl-freezer > Revision: 01 > Title: A feature freezer for the Concise Data = Definition Language (CDDL) > Document date: 2018-08-10 > Pages: 7 > Status: = https://datatracker.ietf.org/doc/draft-bormann-cbor-cddl-freezer/ > Htmlized: = https://tools.ietf.org/html/draft-bormann-cbor-cddl-freezer-01 > Diff: = https://www.ietf.org/rfcdiff?url2=3Ddraft-bormann-cbor-cddl-freezer-01 >=20 > Abstract: > In defining the Concise Data Definition Language (CDDL), some > features have turned up that would be nice to have. In the interest > of completing this specification in a timely manner, the present > document collects nice-to-have features that did not make it into = the > first RFC for CDDL. From nobody Thu Aug 9 23:58:34 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C3ED0130DF3 for ; Thu, 9 Aug 2018 23:58:31 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -9.508 X-Spam-Level: X-Spam-Status: No, score=-9.508 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, T_DKIMWL_WL_HIGH=-0.01, URIBL_DBL_ABUSE_REDIR=0.001, USER_IN_DEF_SPF_WL=-7.5] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=chromium.org Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8-aN_DG9ENz2 for ; Thu, 9 Aug 2018 23:58:29 -0700 (PDT) Received: from mail-yw1-xc41.google.com (mail-yw1-xc41.google.com [IPv6:2607:f8b0:4864:20::c41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1BF4B127AC2 for ; Thu, 9 Aug 2018 23:58:29 -0700 (PDT) Received: by mail-yw1-xc41.google.com with SMTP id e23-v6so7830158ywe.13 for ; Thu, 09 Aug 2018 23:58:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=5ZV5klXUIBnOTRU5qYEAqvTB6RlLtPJHxYi7kKyRXYA=; b=PaW8jIrxxl/IKA4kkrxGNN8T0xHYulTxADpZeMOi7QUEqIXBgEYRTe7aND7QgCVr/a w6j9QJtFH3OU/ICeUwLmlsFHcPHu+ggow4kdvYF5z1Axrit8cDixWsT1yniGA8IUHO9H NpT9L3UVHFfww8Uo2EapBNV3qAx4SafzwZ55k= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=5ZV5klXUIBnOTRU5qYEAqvTB6RlLtPJHxYi7kKyRXYA=; b=tUzIvrCjKq2TiOdhf+hEnDM8n4JE7hwEV0JX+13Qn7DNODi9ktlz7Eag+UGzDV3+da GUfYis4byNZDHwuKhswVto/1xGJxJ0R2UF+zlXBNKw5JPnp1qUR5PvH/CqiGmzguLRvC LBU/RRdL2mPaEDPmdysI6xFLHZhRRu30PAbHBBhUmXlq3T/2MVP4Jb6M/xuIe7j9JqgN lHIBindN12vXlpvB95L94ix32ub5/lLIkFIpCXgnF8s7oxx9fr9CoEDEokhq/4ZuKdx8 XXkR848xNswMfTpqHJsvVA2NhBGmokXuPescA/efkX3fNX5XaAa2xHfFswHzhs5O+tYA ZRsg== X-Gm-Message-State: AOUpUlGI/ZLT+LmPAjX4rtdEHMBmlU8ojozcEbhELOT2QGLTR/8VxZaj /v8t07RN2VAI/hk8uhkqhXEyxIpjLk954Zqc1SFmHtlQPFA= X-Google-Smtp-Source: AA+uWPznZqbABSTllONIxUMoBub+UfYJCjw3iD3o/4dFuS7PQs4CaimPCRQSB6jElVALg3p+WTc/jsEefN/kdpgcHZs= X-Received: by 2002:a81:cb06:: with SMTP id q6-v6mr2792371ywi.199.1533884307720; Thu, 09 Aug 2018 23:58:27 -0700 (PDT) MIME-Version: 1.0 References: <153388117963.28907.13862811574346222830.idtracker@ietfa.amsl.com> <77BEDB78-0C2D-4BD4-95B4-DB3556B22353@tzi.org> In-Reply-To: <77BEDB78-0C2D-4BD4-95B4-DB3556B22353@tzi.org> From: Jeffrey Yasskin Date: Thu, 9 Aug 2018 23:58:16 -0700 Message-ID: To: Carsten Bormann Cc: cbor@ietf.org Content-Type: multipart/alternative; boundary="0000000000004a6fe905730f45d3" Archived-At: Subject: Re: [Cbor] CDDL -04 ready; new Version for draft-bormann-cbor-cddl-freezer-01.txt X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Aug 2018 06:58:32 -0000 --0000000000004a6fe905730f45d3 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable As I mentioned in https://github.com/cbor-wg/cddl/issues/14, replacing the undefined term "match" with the undefined term "can produce" doesn't improve the situation. I suspect it's also inconsistent with the idea of using a Parse Expression Grammar, since "can produce" seems to imply backtracking in quantifiers, which PEGs don't do. This doesn't need to block submission as -04, but it should block submission as an RFC. Thanks, Jeffrey On Thu, Aug 9, 2018 at 11:16 PM Carsten Bormann wrote: > I believe CDDL is now ready for submission as -04; I intend to do this at > 1200Z (I.e., about 5=C2=BE hours from now). > If you have a bit of time to look at this, please consult: > > http://gg.gg/cddl-editor-diff > https://github.com/cbor-wg/cddl/ > > In particular, it may be worth to look at the cleaned up ABNF. > > I have already submitted an update for the freezer document (see below); > comments and pointers to omissions welcome. > > Gr=C3=BC=C3=9Fe, Carsten > > > > Name: draft-bormann-cbor-cddl-freezer > > Revision: 01 > > Title: A feature freezer for the Concise Data Definition > Language (CDDL) > > Document date: 2018-08-10 > > Pages: 7 > > Status: > https://datatracker.ietf.org/doc/draft-bormann-cbor-cddl-freezer/ > > Htmlized: > https://tools.ietf.org/html/draft-bormann-cbor-cddl-freezer-01 > > Diff: > https://www.ietf.org/rfcdiff?url2=3Ddraft-bormann-cbor-cddl-freezer-01 > > > > Abstract: > > In defining the Concise Data Definition Language (CDDL), some > > features have turned up that would be nice to have. In the interest > > of completing this specification in a timely manner, the present > > document collects nice-to-have features that did not make it into the > > first RFC for CDDL. > > > > _______________________________________________ > CBOR mailing list > CBOR@ietf.org > https://www.ietf.org/mailman/listinfo/cbor > --0000000000004a6fe905730f45d3 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
As I mentioned in=C2=A0https://github.com/cbor-wg/cddl/issues/14, replacing= the undefined term "match" with the undefined term "can pro= duce" doesn't improve the situation. I suspect it's also incon= sistent with the idea of using a Parse Expression Grammar, since "can = produce" seems to imply backtracking in quantifiers, which PEGs don= 9;t do.

This doesn't need to block submission as -04= , but it should block submission as an RFC.

Thanks= ,
Jeffrey

On Thu, Aug 9, 2018 at 11:16 PM Carsten Bormann <cabo@tzi.org> wrote:
I believe CDDL is now ready for submission as -04; I intend to do thi= s at 1200Z (I.e., about 5=C2=BE hours from now).
If you have=C2=A0 a bit of time to look at this, please consult:

http://gg.gg/cddl-editor-diff
https://github.com/cbor-wg/cddl/

In particular, it may be worth to look at the cleaned up ABNF.

I have already submitted an update for the freezer document (see below); co= mments and pointers to omissions welcome.

Gr=C3=BC=C3=9Fe, Carsten


> Name:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0draft-bormann-cbor-cddl-freezer=
> Revision:=C2=A0 =C2=A0 =C2=A001
> Title:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 A featur= e freezer for the Concise Data Definition Language (CDDL)
> Document date:=C2=A0 =C2=A0 =C2=A0 =C2=A0 2018-08-10
> Pages:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 7
> Status:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0https://datatracker.ietf.org/doc/draft-bormann-cbor-cddl-freeze= r/
> Htmlized:=C2=A0 =C2=A0 =C2=A0 =C2=A0https://tools.ietf.org/html/draft-bormann-cbor-cddl-freezer-01
> Diff:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0https://www.ietf.org/rfcdiff?url2=3Ddraft-bormann-cbor= -cddl-freezer-01
>
> Abstract:
>=C2=A0 =C2=A0In defining the Concise Data Definition Language (CDDL), s= ome
>=C2=A0 =C2=A0features have turned up that would be nice to have.=C2=A0 = In the interest
>=C2=A0 =C2=A0of completing this specification in a timely manner, the p= resent
>=C2=A0 =C2=A0document collects nice-to-have features that did not make = it into the
>=C2=A0 =C2=A0first RFC for CDDL.



_______________________________________________
CBOR mailing list
CBOR@ietf.org
https://www.ietf.org/mailman/listinfo/cbor
--0000000000004a6fe905730f45d3-- From nobody Fri Aug 10 04:53:53 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 08B43130DE4 for ; Fri, 10 Aug 2018 04:53:52 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.199 X-Spam-Level: X-Spam-Status: No, score=-4.199 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, URIBL_DBL_ABUSE_REDIR=0.001] autolearn=unavailable autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eN7eytxY6oXC for ; Fri, 10 Aug 2018 04:53:50 -0700 (PDT) Received: from mailhost.informatik.uni-bremen.de (mailhost.informatik.uni-bremen.de [IPv6:2001:638:708:30c9::12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 33FA6130DE5 for ; Fri, 10 Aug 2018 04:53:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at informatik.uni-bremen.de Received: from submithost.informatik.uni-bremen.de (submithost.informatik.uni-bremen.de [IPv6:2001:638:708:30c9::b]) by mailhost.informatik.uni-bremen.de (8.14.5/8.14.5) with ESMTP id w7ABrfnR004260; Fri, 10 Aug 2018 13:53:41 +0200 (CEST) Received: from [192.168.217.114] (p54A6C84F.dip0.t-ipconnect.de [84.166.200.79]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by submithost.informatik.uni-bremen.de (Postfix) with ESMTPSA id 41n3Rn3lfqzDXcG; Fri, 10 Aug 2018 13:53:41 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) From: Carsten Bormann In-Reply-To: Date: Fri, 10 Aug 2018 13:53:40 +0200 Cc: cbor@ietf.org X-Mao-Original-Outgoing-Id: 555594819.1706899-a5c50feb37369c0124026f7f5f38164f Content-Transfer-Encoding: quoted-printable Message-Id: <36637D9F-B783-4446-BA39-DFE9C7270907@tzi.org> References: <153388117963.28907.13862811574346222830.idtracker@ietfa.amsl.com> <77BEDB78-0C2D-4BD4-95B4-DB3556B22353@tzi.org> To: Jeffrey Yasskin X-Mailer: Apple Mail (2.3445.9.1) Archived-At: Subject: Re: [Cbor] CDDL -04 ready; new Version for draft-bormann-cbor-cddl-freezer-01.txt X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Aug 2018 11:53:52 -0000 Hi Jeffrey, Did you see the changes in=20 = https://github.com/cbor-wg/cddl/commit/111b90b11b96d1e8fdc8614693a4164086b= 371b4 This commit has some additional text to define =E2=80=9Ccan produce=E2=80=9D= , based on your comment that it wasn=E2=80=99t sufficiently defined. If this new text is not sufficient, at this point, the best way forward = would probably some proposed text. I also would like to understand if you think there is an actual = interoperability problem lurking here (and, if yes, I=E2=80=99d like a = =E2=80=9Cfailing test=E2=80=9D to develop this further), or whether the = text just isn=E2=80=99t perfect (which it isn=E2=80=99t). Also, =E2=80=9C= can produce=E2=80=9D certainly is not the ultimate wording, maybe we can = define a better term. (I=E2=80=99ll go ahead with submitting now; we can discuss this further = during WGLC.) Gr=C3=BC=C3=9Fe, Carsten > On Aug 10, 2018, at 08:58, Jeffrey Yasskin = wrote: >=20 > As I mentioned in https://github.com/cbor-wg/cddl/issues/14, replacing = the undefined term "match" with the undefined term "can produce" doesn't = improve the situation. I suspect it's also inconsistent with the idea of = using a Parse Expression Grammar, since "can produce" seems to imply = backtracking in quantifiers, which PEGs don't do. >=20 > This doesn't need to block submission as -04, but it should block = submission as an RFC. >=20 > Thanks, > Jeffrey >=20 > On Thu, Aug 9, 2018 at 11:16 PM Carsten Bormann wrote: > I believe CDDL is now ready for submission as -04; I intend to do this = at 1200Z (I.e., about 5=C2=BE hours from now). > If you have a bit of time to look at this, please consult: >=20 > http://gg.gg/cddl-editor-diff > https://github.com/cbor-wg/cddl/ >=20 > In particular, it may be worth to look at the cleaned up ABNF. >=20 > I have already submitted an update for the freezer document (see = below); comments and pointers to omissions welcome. >=20 > Gr=C3=BC=C3=9Fe, Carsten >=20 >=20 > > Name: draft-bormann-cbor-cddl-freezer > > Revision: 01 > > Title: A feature freezer for the Concise Data = Definition Language (CDDL) > > Document date: 2018-08-10 > > Pages: 7 > > Status: = https://datatracker.ietf.org/doc/draft-bormann-cbor-cddl-freezer/ > > Htmlized: = https://tools.ietf.org/html/draft-bormann-cbor-cddl-freezer-01 > > Diff: = https://www.ietf.org/rfcdiff?url2=3Ddraft-bormann-cbor-cddl-freezer-01 > >=20 > > Abstract: > > In defining the Concise Data Definition Language (CDDL), some > > features have turned up that would be nice to have. In the = interest > > of completing this specification in a timely manner, the present > > document collects nice-to-have features that did not make it into = the > > first RFC for CDDL. >=20 >=20 >=20 > _______________________________________________ > CBOR mailing list > CBOR@ietf.org > https://www.ietf.org/mailman/listinfo/cbor > _______________________________________________ > CBOR mailing list > CBOR@ietf.org > https://www.ietf.org/mailman/listinfo/cbor From nobody Fri Aug 10 05:16:31 2018 Return-Path: X-Original-To: cbor@ietf.org Delivered-To: cbor@ietfa.amsl.com Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id CA4E31286E3; Fri, 10 Aug 2018 05:16:23 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: internet-drafts@ietf.org To: Cc: cbor@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 6.83.0 Auto-Submitted: auto-generated Precedence: bulk Reply-To: cbor@ietf.org Message-ID: <153390338374.1696.2312375920423852282@ietfa.amsl.com> Date: Fri, 10 Aug 2018 05:16:23 -0700 Archived-At: Subject: [Cbor] I-D Action: draft-ietf-cbor-cddl-04.txt X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Aug 2018 12:16:24 -0000 A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Concise Binary Object Representation Maintenance and Extensions WG of the IETF. Title : Concise data definition language (CDDL): a notational convention to express CBOR and JSON data structures Authors : Henk Birkholz Christoph Vigano Carsten Bormann Filename : draft-ietf-cbor-cddl-04.txt Pages : 59 Date : 2018-08-10 Abstract: This document proposes a notational convention to express CBOR data structures (RFC 7049). Its main goal is to provide an easy and unambiguous way to express structures for protocol messages and data formats that use CBOR or JSON. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-cbor-cddl/ There are also htmlized versions available at: https://tools.ietf.org/html/draft-ietf-cbor-cddl-04 https://datatracker.ietf.org/doc/html/draft-ietf-cbor-cddl-04 A diff from the previous version is available at: https://www.ietf.org/rfcdiff?url2=draft-ietf-cbor-cddl-04 Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From nobody Fri Aug 10 11:23:13 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0442B130E47 for ; Fri, 10 Aug 2018 11:23:12 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -9.508 X-Spam-Level: X-Spam-Status: No, score=-9.508 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, T_DKIMWL_WL_HIGH=-0.01, URIBL_DBL_ABUSE_REDIR=0.001, USER_IN_DEF_SPF_WL=-7.5] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=chromium.org Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YrTD30o9ite0 for ; Fri, 10 Aug 2018 11:23:09 -0700 (PDT) Received: from mail-yw1-xc42.google.com (mail-yw1-xc42.google.com [IPv6:2607:f8b0:4864:20::c42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BFBF7130DF9 for ; Fri, 10 Aug 2018 11:23:08 -0700 (PDT) Received: by mail-yw1-xc42.google.com with SMTP id s68-v6so9263884ywg.2 for ; Fri, 10 Aug 2018 11:23:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=+x6KMSHXTUd+de27arZrBPAm0odIYVqzFGunXauMgLI=; b=mwo9Hn+M3aEB+K0pwypqwWDdixTM9z8hkMsR8aY70R23QwAqsK/L4l5WzYJILKs2c4 YKhJpzZY5Q2/uWFUDpVy1uYvRFpmmZQ0zIXlcWsRxjuVXYp+TGe88ygTQkwn2nktDWI9 6lN6OeVjqVnzIAGrSN6YulokRw8U8HDEaGOj4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=+x6KMSHXTUd+de27arZrBPAm0odIYVqzFGunXauMgLI=; b=ApcmbrNeP6mckrn4e3C3flijUnnI+ib7A9J/WAyxwpUmRFA5qeWvqO44qHsYXoxf4i sHqso2KdlH/AHK0Mc5MlFViK3YdjFBa7id0krGhfgLF/J8HnyPgQiFLPUa5+isfjhplm NVpPjuZow1RvDdbKAAhzAfUJ6XOGESZpQsoNI8r49uLUG0UM8fyTTMibY1I5FRil8HiS f6aRbtA6ZjtAfsfvL/1HAzcIwDFY3mHVsOrLbIc2V3G1c/htA6aw4O7rqwBbeqX7zuF0 xFWpxYoGrIUBLqN9LLDwpVKp+jDwvWh6o7I7H+i6pZDHUcuNKUUhDMx6Phzh/2UR6lzD mKSA== X-Gm-Message-State: AOUpUlF3EMkkQkfwPWujX6gCDepl7fRYzk2hBeyzpnAgcG5m3G1D8XC1 yuj+zeB2wzaQOnpUdPgNpR80qWboAFVGWyLuPqOBjixg7jaFwg== X-Google-Smtp-Source: AA+uWPxKInByFqmsWjN7SSVntLRYu2ohxk0M8E1zC3LKPBv0OFOYsZubZ95OR/e9Dlx5fCXLkIkST928SbLcItzmWEg= X-Received: by 2002:a25:ab8f:: with SMTP id v15-v6mr4094579ybi.408.1533925387360; Fri, 10 Aug 2018 11:23:07 -0700 (PDT) MIME-Version: 1.0 References: <153388117963.28907.13862811574346222830.idtracker@ietfa.amsl.com> <77BEDB78-0C2D-4BD4-95B4-DB3556B22353@tzi.org> <36637D9F-B783-4446-BA39-DFE9C7270907@tzi.org> In-Reply-To: <36637D9F-B783-4446-BA39-DFE9C7270907@tzi.org> From: Jeffrey Yasskin Date: Fri, 10 Aug 2018 11:22:55 -0700 Message-ID: To: Carsten Bormann Cc: Jeffrey Yasskin , cbor@ietf.org Content-Type: multipart/alternative; boundary="000000000000d43f8a057318d513" Archived-At: Subject: [Cbor] CDDL and group matching [was: CDDL -04 ready] X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Aug 2018 18:23:12 -0000 --000000000000d43f8a057318d513 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, Aug 10, 2018 at 4:53 AM Carsten Bormann wrote: > Hi Jeffrey, > > Did you see the changes in > > https://github.com/cbor-wg/cddl/commit/111b90b11b96d1e8fdc8614693a4164086= b371b4 I hadn't seen that. That's definitely an improvement in precision, but it's not the definition of matching that https://rubygems.org/gems/cddl/versions/0.8.5 uses. Is there a newer version of that tool, or a test suite I can add to? https://github.com/cabo/cddl doesn't work for me. Specifically, tstrs =3D [*tstr, +tstr] "can produce" ["a", "a"] because `tstr` can produce "a" per https://tools.ietf.org/html/draft-ietf-cbor-cddl-04#section-2.1.2, `*tstr` can produce () (the empty sequence) per https://tools.ietf.org/html/draft-ietf-cbor-cddl-04#section-3.2, and `+tstr` can produce ("a", "a") also per section 3.2. However, $ cddl <(echo 'tstrs =3D [*tstr, +tstr]') validate <(echo '["a", "a"]'|diag2cbor.rb) returns CDDL validation failure (nil for ["a", "a"]): > > ["a", > > [:prim, 3], > "[\"a\", [:prim, 3], \"occur not reached in array [\\\"a\\\", \\\"a\\\"] > for [:array, [:member, 0, Infinity, nil, [:prim, 3]], [:member, 1, > Infinity, nil, [:prim, 3]]]\"] -- cannot complete (false, 2) array [\"a\"= , > \"a\"] for [:array, [:member, 0, Infinity, nil, [:prim, 3]], [:member, 1, > Infinity, nil, [:prim, 3]]]"] I also suspect the cut text does the wrong thing, but I'm having trouble understanding A cut in this position means that once the map key matches the name part of > an entry that can be produced and carries a cut, other potential matches > for the key that occur in later entries in the group of the map are no > longer allowed. if matching is driven by generating sequences from a rule. I can't offer to write the better text because I don't know what you intend= . Jeffrey > This commit has some additional text to define =E2=80=9Ccan produce=E2=80= =9D, based on > your comment that it wasn=E2=80=99t sufficiently defined. > > If this new text is not sufficient, at this point, the best way forward > would probably some proposed text. > I also would like to understand if you think there is an actual > interoperability problem lurking here (and, if yes, I=E2=80=99d like a = =E2=80=9Cfailing > test=E2=80=9D to develop this further), or whether the text just isn=E2= =80=99t perfect > (which it isn=E2=80=99t). Also, =E2=80=9Ccan produce=E2=80=9D certainly = is not the ultimate > wording, maybe we can define a better term. > > (I=E2=80=99ll go ahead with submitting now; we can discuss this further d= uring > WGLC.) > > Gr=C3=BC=C3=9Fe, Carsten > > > > On Aug 10, 2018, at 08:58, Jeffrey Yasskin 40chromium.org@dmarc.ietf.org> wrote: > > > > As I mentioned in https://github.com/cbor-wg/cddl/issues/14, replacing > the undefined term "match" with the undefined term "can produce" doesn't > improve the situation. I suspect it's also inconsistent with the idea of > using a Parse Expression Grammar, since "can produce" seems to imply > backtracking in quantifiers, which PEGs don't do. > > > > This doesn't need to block submission as -04, but it should block > submission as an RFC. > > > > Thanks, > > Jeffrey > > > > On Thu, Aug 9, 2018 at 11:16 PM Carsten Bormann wrote: > > I believe CDDL is now ready for submission as -04; I intend to do this > at 1200Z (I.e., about 5=C2=BE hours from now). > > If you have a bit of time to look at this, please consult: > > > > http://gg.gg/cddl-editor-diff > > https://github.com/cbor-wg/cddl/ > > > > In particular, it may be worth to look at the cleaned up ABNF. > > > > I have already submitted an update for the freezer document (see below)= ; > comments and pointers to omissions welcome. > > > > Gr=C3=BC=C3=9Fe, Carsten > > > > > > > Name: draft-bormann-cbor-cddl-freezer > > > Revision: 01 > > > Title: A feature freezer for the Concise Data > Definition Language (CDDL) > > > Document date: 2018-08-10 > > > Pages: 7 > > > Status: > https://datatracker.ietf.org/doc/draft-bormann-cbor-cddl-freezer/ > > > Htmlized: > https://tools.ietf.org/html/draft-bormann-cbor-cddl-freezer-01 > > > Diff: > https://www.ietf.org/rfcdiff?url2=3Ddraft-bormann-cbor-cddl-freezer-01 > > > > > > Abstract: > > > In defining the Concise Data Definition Language (CDDL), some > > > features have turned up that would be nice to have. In the interes= t > > > of completing this specification in a timely manner, the present > > > document collects nice-to-have features that did not make it into t= he > > > first RFC for CDDL. > > > > > > > > _______________________________________________ > > CBOR mailing list > > CBOR@ietf.org > > https://www.ietf.org/mailman/listinfo/cbor > > _______________________________________________ > > CBOR mailing list > > CBOR@ietf.org > > https://www.ietf.org/mailman/listinfo/cbor > > --000000000000d43f8a057318d513 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Fri, Aug 10= , 2018 at 4:53 AM Carsten Bormann <cabo@= tzi.org> wrote:
Hi Jeffrey,

Did you see the changes in
https://github.com/cb= or-wg/cddl/commit/111b90b11b96d1e8fdc8614693a4164086b371b4
=

I hadn't seen that. That's definitely an improv= ement in precision, but it's not the definition of matching that=C2=A0<= a href=3D"https://rubygems.org/gems/cddl/versions/0.8.5">https://rubygems.o= rg/gems/cddl/versions/0.8.5 uses.=C2=A0Is there a newer version of that= tool, or a test suite I can add to?=C2=A0https://github.com/cabo/cddl doesn't work for me.

Specifically,

<= div>tstrs =3D [*tstr, +tstr]

"can produce"

=
["a", "a"]

because `tstr` can produce= "a" per=C2=A0https://tools.ietf.org/html/draft-ietf-cbor-cddl= -04#section-2.1.2, `*tstr` can produce () (the empty sequence) per=C2= =A0https://tools.ietf.org/html/draft-ietf-cbor-cddl-04#section-3.2, an= d `+tstr` can produce ("a", "a") also per section 3.2.= =C2=A0

However,=C2=A0

$ cddl= <(echo 'tstrs =3D [*tstr, +tstr]') validate <(echo '[&qu= ot;a", "a"]'|diag2cbor.rb)

returns

CDDL validation failure (nil for ["a", "a"]):=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=C2=A0
["= ;a",=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=C2=A0
=C2= =A0[:prim, 3],
=C2=A0"[\"a\", [:prim, 3], \"occur no= t reached in array [\\\"a\\\", \\\"a\\\"] for [:array, = [:member, 0, Infinity, nil, [:prim, 3]], [:member, 1, Infinity, nil, [:prim= , 3]]]\"] -- cannot complete (false, 2) array [\"a\", \"= ;a\"] for [:array, [:member, 0, Infinity, nil, [:prim, 3]], [:member, = 1, Infinity, nil, [:prim, 3]]]"]

I also suspect the cut text does t= he wrong thing, but I'm having trouble understanding

A cut in this position means that once the map key matches the name pa= rt of an entry that can be produced and carries a cut, other potential matc= hes for the key that occur in later entries in the group of the map are no = longer allowed.
<= div>
if matching is driven by generating sequences from a rul= e.

I can't offer to write the better text beca= use I don't know what you intend.

Jeffrey
=C2=A0
This commit has some additional text to define =E2=80=9Ccan produce=E2=80= =9D, based on your comment that it wasn=E2=80=99t sufficiently defined.

If this new text is not sufficient, at this point, the best way forward wou= ld probably some proposed text.
I also would like to understand if you think there is an actual interoperab= ility problem lurking here (and, if yes, I=E2=80=99d like a =E2=80=9Cfailin= g test=E2=80=9D to develop this further), or whether the text just isn=E2= =80=99t perfect (which it isn=E2=80=99t).=C2=A0 Also, =E2=80=9Ccan produce= =E2=80=9D certainly is not the ultimate wording, maybe we can define a bett= er term.

(I=E2=80=99ll go ahead with submitting now; we can discuss this further dur= ing WGLC.)

Gr=C3=BC=C3=9Fe, Carsten


> On Aug 10, 2018, at 08:58, Jeffrey Yasskin <jyasskin=3D40chromium.org@dmarc= .ietf.org> wrote:
>
> As I mentioned in https://github.com/cbor-wg/cddl/issu= es/14, replacing the undefined term "match" with the undefine= d term "can produce" doesn't improve the situation. I suspect= it's also inconsistent with the idea of using a Parse Expression Gramm= ar, since "can produce" seems to imply backtracking in quantifier= s, which PEGs don't do.
>
> This doesn't need to block submission as -04, but it should block = submission as an RFC.
>
> Thanks,
> Jeffrey
>
> On Thu, Aug 9, 2018 at 11:16 PM Carsten Bormann <cabo@tzi.org> wrote:
> I believe CDDL is now ready for submission as -04; I intend to do this= at 1200Z (I.e., about 5=C2=BE hours from now).
> If you have=C2=A0 a bit of time to look at this, please consult:
>
> http://gg.gg/cddl-editor-diff
> https://github.com/cbor-wg/cddl/
>
> In particular, it may be worth to look at the cleaned up ABNF.
>
> I have already submitted an update for the freezer document (see below= ); comments and pointers to omissions welcome.
>
> Gr=C3=BC=C3=9Fe, Carsten
>
>
> > Name:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0draft-bormann-cbor-cddl-fr= eezer
> > Revision:=C2=A0 =C2=A0 =C2=A001
> > Title:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 A f= eature freezer for the Concise Data Definition Language (CDDL)
> > Document date:=C2=A0 =C2=A0 =C2=A0 =C2=A0 2018-08-10
> > Pages:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 7 > > Status:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0https://datatracker.ietf.org/doc/draft-bormann-cbor-cddl-fr= eezer/
> > Htmlized:=C2=A0 =C2=A0 =C2=A0 =C2=A0https://tools.ietf.org/html/draft-bormann-cbor-cddl-freezer-01<= br> > > Diff:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0https://www.ietf.org/rfcdiff?url2=3Ddraft-bormann= -cbor-cddl-freezer-01
> >
> > Abstract:
> >=C2=A0 =C2=A0In defining the Concise Data Definition Language (CDD= L), some
> >=C2=A0 =C2=A0features have turned up that would be nice to have.= =C2=A0 In the interest
> >=C2=A0 =C2=A0of completing this specification in a timely manner, = the present
> >=C2=A0 =C2=A0document collects nice-to-have features that did not = make it into the
> >=C2=A0 =C2=A0first RFC for CDDL.
>
>
>
> _______________________________________________
> CBOR mailing list
> CBOR@ietf.org > https://www.ietf.org/mailman/listinfo/cbor
> _______________________________________________
> CBOR mailing list
> CBOR@ietf.org > https://www.ietf.org/mailman/listinfo/cbor

--000000000000d43f8a057318d513-- From nobody Fri Aug 10 12:33:31 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8EDB5130EAE for ; Fri, 10 Aug 2018 12:33:30 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.2 X-Spam-Level: X-Spam-Status: No, score=-4.2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4bHxN1u0hLCI for ; Fri, 10 Aug 2018 12:33:27 -0700 (PDT) Received: from mailhost.informatik.uni-bremen.de (mailhost.informatik.uni-bremen.de [IPv6:2001:638:708:30c9::12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4E3F1130E81 for ; Fri, 10 Aug 2018 12:33:27 -0700 (PDT) X-Virus-Scanned: amavisd-new at informatik.uni-bremen.de Received: from submithost.informatik.uni-bremen.de (submithost.informatik.uni-bremen.de [134.102.201.11]) by mailhost.informatik.uni-bremen.de (8.14.5/8.14.5) with ESMTP id w7AJXNcP019218; Fri, 10 Aug 2018 21:33:23 +0200 (CEST) Received: from client-0088.vpn.uni-bremen.de (client-0088.vpn.uni-bremen.de [134.102.107.88]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by submithost.informatik.uni-bremen.de (Postfix) with ESMTPSA id 41nFfC0XhMzDXg7; Fri, 10 Aug 2018 21:33:23 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) From: Carsten Bormann In-Reply-To: Date: Fri, 10 Aug 2018 21:33:22 +0200 Cc: cbor@ietf.org X-Mao-Original-Outgoing-Id: 555622400.570672-301b912d0ce960950d424bcb495886c6 Content-Transfer-Encoding: quoted-printable Message-Id: <27A69EAD-DEDC-4FD6-8812-F912BE9564DF@tzi.org> References: <153388117963.28907.13862811574346222830.idtracker@ietfa.amsl.com> <77BEDB78-0C2D-4BD4-95B4-DB3556B22353@tzi.org> <36637D9F-B783-4446-BA39-DFE9C7270907@tzi.org> To: Jeffrey Yasskin X-Mailer: Apple Mail (2.3445.9.1) Archived-At: Subject: Re: [Cbor] CDDL and group matching [was: CDDL -04 ready] X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Aug 2018 19:33:31 -0000 > On Aug 10, 2018, at 20:22, Jeffrey Yasskin = wrote: >=20 > On Fri, Aug 10, 2018 at 4:53 AM Carsten Bormann wrote: > Hi Jeffrey, >=20 > Did you see the changes in=20 > = https://github.com/cbor-wg/cddl/commit/111b90b11b96d1e8fdc8614693a4164086b= 371b4 >=20 > I hadn't seen that. That's definitely an improvement in precision, but = it's not the definition of matching that = https://rubygems.org/gems/cddl/versions/0.8.5 uses. Is there a newer = version of that tool, or a test suite I can add to? = https://github.com/cabo/cddl doesn=E2=80=99t work for me. I am working on a newer version of the tool. But the behavior is not going to change for: > tstrs =3D [*tstr, +tstr] =E2=80=94 the PEG semantics that is exhibited by the current version is = intended. Sorry, I was on a goose chase here, because I thought you were = discussing cut semantics and not the general PEG semantics. So I gather = your point is improving the description of those. (The text in Appendix C about occurrence acting as a potentially = infinite group choice together with the text pointing out that choices = =E2=80=94 both type and group =E2=80=94 use PEG semantics provides a = full definition. I might help to point to that from the introductory = text.) > "can produce" >=20 > [=E2=80=9Ca", "a"] Right. > because `tstr` can produce "a" per = https://tools.ietf.org/html/draft-ietf-cbor-cddl-04#section-2.1.2, = `*tstr` can produce () (the empty sequence) per = https://tools.ietf.org/html/draft-ietf-cbor-cddl-04#section-3.2, and = `+tstr` can produce ("a", "a") also per section 3.2.=20 >=20 > However,=20 >=20 > $ cddl <(echo 'tstrs =3D [*tstr, +tstr]') validate <(echo '["a", = "a"]'|diag2cbor.rb) >=20 > returns >=20 > CDDL validation failure (nil for [=E2=80=9Ca", "a"]): =20 As it should according to the PEG semantics. > I also suspect the cut text does the wrong thing, but I'm having = trouble understanding >=20 > A cut in this position means that once the map key matches the name = part of an entry that can be produced and carries a cut, other potential = matches for the key that occur in later entries in the group of the map = are no longer allowed. >=20 > if matching is driven by generating sequences from a rule. >=20 > I can=E2=80=99t offer to write the better text because I don't know = what you intend. The intention is to use the PEG semantics for cuts as well. Gr=C3=BC=C3=9Fe, Carsten From nobody Fri Aug 10 13:19:50 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8A002130EB8 for ; Fri, 10 Aug 2018 13:19:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -9.509 X-Spam-Level: X-Spam-Status: No, score=-9.509 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, T_DKIMWL_WL_HIGH=-0.01, USER_IN_DEF_SPF_WL=-7.5] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=chromium.org Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id S2Q-2zD09exk for ; Fri, 10 Aug 2018 13:19:47 -0700 (PDT) Received: from mail-yw1-xc2a.google.com (mail-yw1-xc2a.google.com [IPv6:2607:f8b0:4864:20::c2a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2D097130E2F for ; Fri, 10 Aug 2018 13:19:47 -0700 (PDT) Received: by mail-yw1-xc2a.google.com with SMTP id r184-v6so9515302ywg.6 for ; Fri, 10 Aug 2018 13:19:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=u6gMBWLkqYkaQZyOrfGa0RimQ/gYdcJEIEvBjT7AnyI=; b=XSV/4n8446AKU9pooiZ86DT54QzYF08z67P/5OQYZR6ZB0frT6neuPPMGCA11uxjkX qe/Hqk0Swt1C/P+wS0l2uI1nG+GwDtEnLUJVb36PsRpc6c4aFt5YoOpQGQV0U3ArPeTN fd6oY8GZMOjPfW56+qZPjzpUJmo2q2Hw2RXxs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=u6gMBWLkqYkaQZyOrfGa0RimQ/gYdcJEIEvBjT7AnyI=; b=il4ni9SCxXuMYkMwN3o5Y8F7g4miU2oJrn4B8DnpONlrNj5ko4i/1qmq/hCtQwrIi4 X17dZfkSm2EOyAgD0fjqPMOywkTkwaLL4T0Ka7jjH2WjraRHtga3CgmgWuTn95IbRg85 7B5I/C0gU6kyTpeBwnzokBgVm/hy087ABHmGVZ59Hybe5pohihzbd74BFRuPhgeb9HuZ J0lc5zKKk0gWKad8/di9+iE6qUoqdGT4wSE6gZrmULBP5HELxjesmUkNWNOpuXnixKfR lL93pcMXnrQA5mRfb1xpaIr0K8/OoR69Z6UZlowiuB636wh2A2MGoOOn9BZ80CelCVTc kezg== X-Gm-Message-State: AOUpUlFSGuVe3fX9OEuHDSslP7S/WHOQCL1idk9XO/8ZM/HVM1NajzTL zDjm2LCqXjGlaCCsd31lCXzVzy6lAgpO9Gf/+cw+pQ== X-Google-Smtp-Source: AA+uWPxxfWU+NqsqPdbEkz4AIdOAHmXO/s2vLOxDavB7nFj8SDWfC4ixtMe36fSZqyJ0GB6uOu7BNlIr4nH5gaUc+HQ= X-Received: by 2002:a25:8806:: with SMTP id c6-v6mr1872995ybl.12.1533932385651; Fri, 10 Aug 2018 13:19:45 -0700 (PDT) MIME-Version: 1.0 References: <153388117963.28907.13862811574346222830.idtracker@ietfa.amsl.com> <77BEDB78-0C2D-4BD4-95B4-DB3556B22353@tzi.org> <36637D9F-B783-4446-BA39-DFE9C7270907@tzi.org> <27A69EAD-DEDC-4FD6-8812-F912BE9564DF@tzi.org> In-Reply-To: <27A69EAD-DEDC-4FD6-8812-F912BE9564DF@tzi.org> From: Jeffrey Yasskin Date: Fri, 10 Aug 2018 13:19:33 -0700 Message-ID: To: Carsten Bormann Cc: Jeffrey Yasskin , cbor@ietf.org Content-Type: multipart/alternative; boundary="000000000000f5b1c705731a764e" Archived-At: Subject: Re: [Cbor] CDDL and group matching [was: CDDL -04 ready] X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Aug 2018 20:19:50 -0000 --000000000000f5b1c705731a764e Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, Aug 10, 2018 at 12:33 PM Carsten Bormann wrote: > > > > On Aug 10, 2018, at 20:22, Jeffrey Yasskin > wrote: > > > > On Fri, Aug 10, 2018 at 4:53 AM Carsten Bormann wrote: > > Hi Jeffrey, > > > > Did you see the changes in > > > https://github.com/cbor-wg/cddl/commit/111b90b11b96d1e8fdc8614693a4164086= b371b4 > > > > I hadn't seen that. That's definitely an improvement in precision, but > it's not the definition of matching that > https://rubygems.org/gems/cddl/versions/0.8.5 uses. Is there a newer > version of that tool, or a test suite I can add to? > https://github.com/cabo/cddl doesn=E2=80=99t work for me. > > I am working on a newer version of the tool. > > But the behavior is not going to change for: > > > tstrs =3D [*tstr, +tstr] > > =E2=80=94 the PEG semantics that is exhibited by the current version is i= ntended. > > Sorry, I was on a goose chase here, because I thought you were discussing > cut semantics and not the general PEG semantics. So I gather your point = is > improving the description of those. > I'm discussing both: there has to be 1 definition of when a CDDL rule matches a CBOR item, and that definition needs to cover both the behavior of quantifiers and alternation, and the behavior of cuts. The intention is to use the PEG semantics for cuts as well. > It's not clear what that means. https://dl.acm.org/citation.cfm?id=3D964001.964011 defines how a PEG parses input *strings*, but a CBOR map is not ordered, and so it's not clear how to run a PEG over it. Without the meaning of a PEG over maps, the spec can't define how cuts change that meaning. Jeffrey --000000000000f5b1c705731a764e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Fri, Aug 10= , 2018 at 12:33 PM Carsten Bormann <cabo= @tzi.org> wrote:


> On Aug 10, 2018, at 20:22, Jeffrey Yasskin <jyasskin@chromium.org> wrote: >
> On Fri, Aug 10, 2018 at 4:53 AM Carsten Bormann <cabo@tzi.org> wrote:
> Hi Jeffrey,
>
> Did you see the changes in
> https://github.c= om/cbor-wg/cddl/commit/111b90b11b96d1e8fdc8614693a4164086b371b4
>
> I hadn't seen that. That's definitely an improvement in precis= ion, but it's not the definition of matching that h= ttps://rubygems.org/gems/cddl/versions/0.8.5 uses. Is there a newer ver= sion of that tool, or a test suite I can add to? https://github.com/cabo= /cddl doesn=E2=80=99t work for me.

I am working on a newer version of the tool.

But the behavior is not going to change for:

> tstrs =3D [*tstr, +tstr]

=E2=80=94 the PEG semantics that is exhibited by the current version is int= ended.

Sorry, I was on a goose chase here, because I thought you were discussing c= ut semantics and not the general PEG semantics.=C2=A0 So I gather your poin= t is improving the description of those.

I'm discussing both: there has to be 1 definition of when a CDDL rule= matches a CBOR item, and that definition needs to cover both the behavior = of quantifiers and alternation, and the behavior of cuts.

The intention is to use the PEG semantics for cuts as well.

It's not clear what that means.=C2=A0https://dl.acm.org/citati= on.cfm?id=3D964001.964011 defines how a PEG parses input *strings*, but= a CBOR map is not ordered, and so it's not clear how to run a PEG over= it. Without the meaning of a PEG over maps, the spec can't define how = cuts change that meaning.

Jeffrey
--000000000000f5b1c705731a764e-- From nobody Fri Aug 10 14:52:50 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EBB13127333 for ; Fri, 10 Aug 2018 14:52:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.2 X-Spam-Level: X-Spam-Status: No, score=-4.2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3] autolearn=unavailable autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0SNLm6qlqmjs for ; Fri, 10 Aug 2018 14:52:47 -0700 (PDT) Received: from mailhost.informatik.uni-bremen.de (mailhost.informatik.uni-bremen.de [IPv6:2001:638:708:30c9::12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 86851130E66 for ; Fri, 10 Aug 2018 14:52:47 -0700 (PDT) X-Virus-Scanned: amavisd-new at informatik.uni-bremen.de Received: from submithost.informatik.uni-bremen.de (submithost.informatik.uni-bremen.de [134.102.201.11]) by mailhost.informatik.uni-bremen.de (8.14.5/8.14.5) with ESMTP id w7ALqfXe009135; Fri, 10 Aug 2018 23:52:41 +0200 (CEST) Received: from client-0088.vpn.uni-bremen.de (client-0088.vpn.uni-bremen.de [134.102.107.88]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by submithost.informatik.uni-bremen.de (Postfix) with ESMTPSA id 41nJkx4x8czDXgr; Fri, 10 Aug 2018 23:52:41 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) From: Carsten Bormann In-Reply-To: Date: Fri, 10 Aug 2018 23:52:40 +0200 Cc: Jeffrey Yasskin , cbor@ietf.org X-Mao-Original-Outgoing-Id: 555630759.211176-6eb20307cf3ac1dc4ca3a91062c64e95 Content-Transfer-Encoding: quoted-printable Message-Id: <750143B4-0703-4237-B283-695F040E1C00@tzi.org> References: <153388117963.28907.13862811574346222830.idtracker@ietfa.amsl.com> <77BEDB78-0C2D-4BD4-95B4-DB3556B22353@tzi.org> <36637D9F-B783-4446-BA39-DFE9C7270907@tzi.org> <27A69EAD-DEDC-4FD6-8812-F912BE9564DF@tzi.org> To: Jeffrey Yasskin X-Mailer: Apple Mail (2.3445.9.1) Archived-At: Subject: Re: [Cbor] CDDL and group matching [was: CDDL -04 ready] X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Aug 2018 21:52:49 -0000 Hi Jeffrey, thanks for helping us to get this fixed. > I=E2=80=99m discussing both: there has to be 1 definition of when a = CDDL rule matches a CBOR item, and that definition needs to cover both = the behavior of quantifiers and alternation, and the behavior of cuts. Yes. I was reading your messages thinking that you had a problem just = with cuts and not with the rest of the matching rules, and so I didn=E2=80= =99t think about fixing the latter. I now see what you are trying to = achieve. > > The intention is to use the PEG semantics for cuts as well. >=20 > It's not clear what that means. = https://dl.acm.org/citation.cfm?id=3D964001.964011 defines how a PEG = parses input *strings*, but a CBOR map is not ordered, and so it's not = clear how to run a PEG over it. Without the meaning of a PEG over maps, = the spec can=E2=80=99t define how cuts change that meaning. Right. So, while applying the matching rules of Section 3.3 of that = paper to maps instead of strings, we are not chopping off the front of a = string, we are picking off elements from that map. Since the picking = can occur anywhere in that map (there is no order), this adds = non-determinism. In real-world specifications, that problem is usually = dodged intuitively by going from more-specific to more-general entries. = But it is possible to construct pathological cases: A =3D { text =3D> int, text =3D> 2 } A matches {=E2=80=9Ca=E2=80=9D: 2, =E2=80=9Cb=E2=80=9D: 4} if the first = entry of the group happens to pick off =E2=80=9Cb=E2=80=9D: 4 and not if = it happens to pick =E2=80=9Ca=E2=80=9D: 2. (Playing around with version 0.8.5 of the tool is little help here, as = its map matching algorithm is not quite up to the current spec. This is = one of the two main reasons I=E2=80=99m rewriting the tool right now.) Gr=C3=BC=C3=9Fe, Carsten From nobody Mon Aug 13 09:44:29 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E82B9130F9D for ; Mon, 13 Aug 2018 09:44:27 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -9.508 X-Spam-Level: X-Spam-Status: No, score=-9.508 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, T_DKIMWL_WL_HIGH=-0.01, URIBL_BLOCKED=0.001, USER_IN_DEF_SPF_WL=-7.5] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=chromium.org Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id arPcvE14m8kU for ; Mon, 13 Aug 2018 09:44:25 -0700 (PDT) Received: from mail-yw1-xc36.google.com (mail-yw1-xc36.google.com [IPv6:2607:f8b0:4864:20::c36]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8922612F1A2 for ; Mon, 13 Aug 2018 09:44:25 -0700 (PDT) Received: by mail-yw1-xc36.google.com with SMTP id q129-v6so14032952ywg.8 for ; Mon, 13 Aug 2018 09:44:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Zu/548v7Mb10rqwMlPluqUTlDMes9pJ2AR+Qz2DFTS0=; b=IEIlJNJzstrgYt/Hm9hfd6cewUsG4yMZca0CFT1Q2Pxpj91+BnSwXXfQ1/UWkFxgu3 22ycmQ/jWJb3czo09RFgfKgv/dMCHbvk93FinJ9MyHpKEMAupbnsP+qqfrOqsvbLYTWA 181AkIRxkS0YVwQEFdwEtqqxdGZYsBRP7HXQU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Zu/548v7Mb10rqwMlPluqUTlDMes9pJ2AR+Qz2DFTS0=; b=mNJft4+6IfChjoWXIzcp8baSUHG2wL0bJzCQ5NSKw+Nkcf4Pc9ZQp9PPLthONYaQ9x u53Wq41i7T0/nP/rzeD7aVsGL7QHLQPu12zBuozU8HKl6Oy9LbbxmAfY25Tf7+KZLHj+ Jx697vsagQLYAHSwT1dkQOwW8lA3y2bIkFBn2EC8+L2lNEo7GCT9l7g5MSW8fIUB/Ot6 d71zKIgn9Tq3d5qbA/ofmb8xX0PB07ENv/QXU9BKrkhlsu4BEYRAagWXJFpbFNB2TmY4 S+Lb2piteBsyuLRSqDLN3IcdZCfmVWbZdSlnh+CH8SOvR4px2rSMQcs1v0pPlMLrHVpn Lljw== X-Gm-Message-State: AOUpUlFc2jbylr1n5UueNpYkvu8TGsZwe49AMuLHdKT73Y1eXCf2DD+B XtLaexpWdU0ZgYo+juaz9kP5GobwgITDYfgPVkhyw11PHF4= X-Google-Smtp-Source: AA+uWPxmV8Ho9TIDhZjusWBKheqEc5bujkR18liVKTGDsHaf+WWIzct6n6bufWUPRrFZfvFlm3TQRgWLv4M9RpFZB2A= X-Received: by 2002:a0d:d105:: with SMTP id t5-v6mr10296413ywd.449.1534178664110; Mon, 13 Aug 2018 09:44:24 -0700 (PDT) MIME-Version: 1.0 References: <153388117963.28907.13862811574346222830.idtracker@ietfa.amsl.com> <77BEDB78-0C2D-4BD4-95B4-DB3556B22353@tzi.org> <36637D9F-B783-4446-BA39-DFE9C7270907@tzi.org> <27A69EAD-DEDC-4FD6-8812-F912BE9564DF@tzi.org> <750143B4-0703-4237-B283-695F040E1C00@tzi.org> In-Reply-To: <750143B4-0703-4237-B283-695F040E1C00@tzi.org> From: Jeffrey Yasskin Date: Mon, 13 Aug 2018 09:44:12 -0700 Message-ID: To: Carsten Bormann Cc: cbor@ietf.org Content-Type: multipart/alternative; boundary="0000000000004cc4d6057353ce5f" Archived-At: Subject: Re: [Cbor] CDDL and group matching [was: CDDL -04 ready] X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Aug 2018 16:44:28 -0000 --0000000000004cc4d6057353ce5f Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, Aug 10, 2018 at 2:52 PM Carsten Bormann wrote: > > > The intention is to use the PEG semantics for cuts as well. > > > > It's not clear what that means. > https://dl.acm.org/citation.cfm?id=3D964001.964011 defines how a PEG pars= es > input *strings*, but a CBOR map is not ordered, and so it's not clear how > to run a PEG over it. Without the meaning of a PEG over maps, the spec > can=E2=80=99t define how cuts change that meaning. > > Right. So, while applying the matching rules of Section 3.3 of that pape= r > to maps instead of strings, we are not chopping off the front of a string= , > we are picking off elements from that map. Since the picking can occur > anywhere in that map (there is no order), this adds non-determinism. In > real-world specifications, that problem is usually dodged intuitively by > going from more-specific to more-general entries. It's going to be an interoperability problem if there's non-determinism in whether an item matches a CDDL spec. The spec needs to either give a deterministic way to pick the next map item, or give a rule for declaring CDDL specs invalid so that all valid ones don't care what order the implementation picks map items. > But it is possible to construct pathological cases: > > A =3D { text =3D> int, text =3D> 2 } > > A matches {=E2=80=9Ca=E2=80=9D: 2, =E2=80=9Cb=E2=80=9D: 4} if the first e= ntry of the group happens to pick > off =E2=80=9Cb=E2=80=9D: 4 and not if it happens to pick =E2=80=9Ca=E2=80= =9D: 2. > > (Playing around with version 0.8.5 of the tool is little help here, as it= s > map matching algorithm is not quite up to the current spec. This is one = of > the two main reasons I=E2=80=99m rewriting the tool right now.) > Unless I'm missing something, the current spec doesn't have a map matching algorithm. Even if it said "implementations MUST translate the group to PEG in ... way and then run it", which it doesn't, as described above that wouldn't be sufficient. 0.8.5 may not be up to what you intend, but your intent hasn't made it into the spec, and it needs to before there'll be a point to a WGLC. Jeffrey --0000000000004cc4d6057353ce5f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Fri, Aug 10= , 2018 at 2:52 PM Carsten Bormann <cabo@= tzi.org> wrote:
> > The intention is to use the PEG semantics for cuts as well.
>
> It's not clear what that means. https://d= l.acm.org/citation.cfm?id=3D964001.964011 defines how a PEG parses inpu= t *strings*, but a CBOR map is not ordered, and so it's not clear how t= o run a PEG over it. Without the meaning of a PEG over maps, the spec can= =E2=80=99t define how cuts change that meaning.

Right.=C2=A0 So, while applying the matching rules of Section 3.3 of that p= aper to maps instead of strings, we are not chopping off the front of a str= ing, we are picking off elements from that map.=C2=A0 Since the picking can= occur anywhere in that map (there is no order), this adds non-determinism.= =C2=A0 In real-world specifications, that problem is usually dodged intuiti= vely by going from more-specific to more-general entries.
=
It's going to be an interoperability problem if there= 9;s non-determinism in whether an item matches a CDDL spec. The spec needs = to either give a deterministic way to pick the next map item, or give a rul= e for declaring CDDL specs invalid so that all valid ones don't care wh= at order the implementation picks map items.
=C2=A0
=C2=A0 But it is possible to construct pathological = cases:

A =3D { text =3D> int, text =3D> 2 }

A matches {=E2=80=9Ca=E2=80=9D: 2, =E2=80=9Cb=E2=80=9D: 4} if the first ent= ry of the group happens to pick off =E2=80=9Cb=E2=80=9D: 4 and not if it ha= ppens to pick =E2=80=9Ca=E2=80=9D: 2.

(Playing around with version 0.8.5 of the tool is little help here, as its = map matching algorithm is not quite up to the current spec.=C2=A0 This is o= ne of the two main reasons I=E2=80=99m rewriting the tool right now.)

Unless I'm missing something, the curren= t spec doesn't have a map matching algorithm. Even if it said "imp= lementations MUST translate the group to PEG in ... way and then run it&quo= t;, which it doesn't, as described above that wouldn't be sufficien= t. 0.8.5 may not be up to what you intend, but your intent hasn't made = it into the spec, and it needs to before there'll be a point to a WGLC.=

Jeffrey
--0000000000004cc4d6057353ce5f-- From nobody Mon Aug 13 12:32:57 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8DD52130E1D for ; Mon, 13 Aug 2018 12:32:56 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.2 X-Spam-Level: X-Spam-Status: No, score=-4.2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3] autolearn=unavailable autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VC3rE0coqMOs for ; Mon, 13 Aug 2018 12:32:55 -0700 (PDT) Received: from mailhost.informatik.uni-bremen.de (mailhost.informatik.uni-bremen.de [IPv6:2001:638:708:30c9::12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4C0EF130E1C for ; Mon, 13 Aug 2018 12:32:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at informatik.uni-bremen.de Received: from submithost.informatik.uni-bremen.de (submithost.informatik.uni-bremen.de [IPv6:2001:638:708:30c9::b]) by mailhost.informatik.uni-bremen.de (8.14.5/8.14.5) with ESMTP id w7DJWnqm010499; Mon, 13 Aug 2018 21:32:49 +0200 (CEST) Received: from [192.168.217.114] (p54A6C84F.dip0.t-ipconnect.de [84.166.200.79]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by submithost.informatik.uni-bremen.de (Postfix) with ESMTPSA id 41q5V9300nzDWfl; Mon, 13 Aug 2018 21:32:49 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) From: Carsten Bormann In-Reply-To: Date: Mon, 13 Aug 2018 21:32:48 +0200 Cc: cbor@ietf.org X-Mao-Original-Outgoing-Id: 555881567.003253-72bd00c757f401d2d83299294904ee7c Content-Transfer-Encoding: quoted-printable Message-Id: References: <153388117963.28907.13862811574346222830.idtracker@ietfa.amsl.com> <77BEDB78-0C2D-4BD4-95B4-DB3556B22353@tzi.org> <36637D9F-B783-4446-BA39-DFE9C7270907@tzi.org> <27A69EAD-DEDC-4FD6-8812-F912BE9564DF@tzi.org> <750143B4-0703-4237-B283-695F040E1C00@tzi.org> To: Jeffrey Yasskin X-Mailer: Apple Mail (2.3445.9.1) Archived-At: Subject: Re: [Cbor] CDDL and group matching [was: CDDL -04 ready] X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Aug 2018 19:32:57 -0000 > It's going to be an interoperability problem if there's = non-determinism in whether an item matches a CDDL spec. The spec needs = to either give a deterministic way to pick the next map item, or give a = rule for declaring CDDL specs invalid so that all valid ones don=E2=80=99t= care what order the implementation picks map items. Right. Reminds me of the =E2=80=9Cambiguous content model=E2=80=9D = (again, this was not about ambiguity but non-determinism) discussion = when SGML was finished. Here, this was really complicated, as SGML had = the =E2=80=9C&=E2=80=9D operator, which allows the DTD specification to = leave open in which sequence required elements were to be included. = =E2=80=9C&=E2=80=9D was thrown out for XML because it made things = complicated that didn=E2=80=99t need to be. (Anne Br=C3=BCggemann-Klein = wrote a whole thesis about this subject in 1993.) Unfortunately, for JSON (and due to its JSON heritage, for CBOR as = well), we can=E2=80=99t just throw arbitrary ordering out. So maybe we should do what SGML and then XML did: outlaw =E2=80=9Cambiguou= s content models=E2=80=9D, as in =E2=80=9CFor compatibility, it is an = error if an element in the document can match more than one occurrence = of an element type in the content model.=E2=80=9D = (http://www.w3.org/TR/REC-xml#sec-element-content) The fun part about making something an =E2=80=9Cerror=E2=80=9D in XML is = that a processor does not have to check it. The problem we are discussing has not occurred in practice. The reason = is that it is natural, when writing the group for a map, to be quite = specific. Non-determinism only occurs in contrived examples. I=E2=80=99d= be unhappy to damage CDDL because of contrived examples. I=E2=80=99d = also be unhappy having to wait for Anne to write another thesis. So just saying =E2=80=9Cpathological specs introduce non-determinism, = don=E2=80=99t do that=E2=80=9D is probably the best way to handle this = for CDDL 1.0. > Unless I=E2=80=99m missing something, the current spec doesn't have a = map matching algorithm. No, the semantics of matching a map against a group is not given as an = algorithm: =E2=80=9Ca map expression [=E2=80=A6] matches a valid CBOR map the = key/value pairs of which can be ordered in such a way that the resulting sequence matches the group expression [=E2=80=A6]=E2=80=9D (Not having an algorithm, but a logical expression, is a feature, not a = bug.) The definition of the group matching mentions PEGs, but only for the = choice, this probably needs to be added once more. > Even if it said "implementations MUST translate the group to PEG in = ... way and then run it", which it doesn't, as described above that = wouldn't be sufficient. 0.8.5 may not be up to what you intend, but your = intent hasn=E2=80=99t made it into the spec, and it needs to before = there'll be a point to a WGLC. Anything else to fix in addition to=20 =E2=80=94 calling out PEG semantics for occurrence indicators =E2=80=94 making specifications where the non-determinism can leads to = multiple interpretations invalid ? Gr=C3=BC=C3=9Fe, Carsten From nobody Mon Aug 13 14:21:55 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 86BB8130DFC for ; Mon, 13 Aug 2018 14:21:53 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -9.508 X-Spam-Level: X-Spam-Status: No, score=-9.508 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, T_DKIMWL_WL_HIGH=-0.01, URIBL_BLOCKED=0.001, USER_IN_DEF_SPF_WL=-7.5] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=chromium.org Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Vz0pVWkyR8We for ; Mon, 13 Aug 2018 14:21:50 -0700 (PDT) Received: from mail-yw1-xc35.google.com (mail-yw1-xc35.google.com [IPv6:2607:f8b0:4864:20::c35]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DD7101310FD for ; Mon, 13 Aug 2018 14:21:49 -0700 (PDT) Received: by mail-yw1-xc35.google.com with SMTP id s68-v6so14692141ywg.2 for ; Mon, 13 Aug 2018 14:21:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=1kuttbRYxzulEiqJJMHAJgTMgSzjD13bNBCK1g4tUbY=; b=XcdJvDOHnGPEBkD7BSlHyfOJ5vsMamOwYMuKQIYYJKW7DnOmeCh9caxLSF5dm2EQPV jiugZikR4gJSqMMHutUyzyMfUganJYUE1DyNuoDawZDhVQo31722AqZxJ44kkf/Mz4pN sOYfJaKlGvKavkeo6VWSXLXOKXhnmgWaYr/Fg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=1kuttbRYxzulEiqJJMHAJgTMgSzjD13bNBCK1g4tUbY=; b=RfQqsLWuHBsjZ00qiue3cJqKNDOtAhnx1tQB3ZpbqkckJP9Y/8DPgq1iuW/5AvqIxq 2PuAKeI1xKi9X2GNnWAws8G+FkYaSMcpQC2ODKQh8le8D7ZWWVzmQ91AJsGK41Ye9dJw ZTMAnvP7gsakALnqa0hSATh8ApJcss/+5tm4z7wXOhH2+kZ810ZZNqH/vP2svlkEL87u 6vf24ap/KgismuNOkG1Xd8p0oaBelQgqLV89FOaYBm4vh/SuBFWEXbi05TA3GuOETrG4 FEVBJXostGeql+m19MG5DTqHEymy/LjTg0WfM79l5Arkp0ErwJTfZwZHfpicW1WPcv/p jsnA== X-Gm-Message-State: AOUpUlHjwYyoESnXZJcdgPn4NnB06I72atbv1R0IXUjSJ/WteLKXYhb1 9tHZKG1UaHZu+1JtDRqVC5sgOl3nywxZ4M9D2AbV+w== X-Google-Smtp-Source: AA+uWPwSvTVw/FytSVf7GISxXlHN0Bpn2TDdPrBrl8aTskzJCRIYkgl//Fp61Dj/U6BTMgwDKwafwIc44KAKjqGIOR8= X-Received: by 2002:a81:250a:: with SMTP id l10-v6mr10044862ywl.222.1534195308682; Mon, 13 Aug 2018 14:21:48 -0700 (PDT) MIME-Version: 1.0 References: <153388117963.28907.13862811574346222830.idtracker@ietfa.amsl.com> <77BEDB78-0C2D-4BD4-95B4-DB3556B22353@tzi.org> <36637D9F-B783-4446-BA39-DFE9C7270907@tzi.org> <27A69EAD-DEDC-4FD6-8812-F912BE9564DF@tzi.org> <750143B4-0703-4237-B283-695F040E1C00@tzi.org> In-Reply-To: From: Jeffrey Yasskin Date: Mon, 13 Aug 2018 14:21:36 -0700 Message-ID: To: Carsten Bormann Cc: Jeffrey Yasskin , cbor@ietf.org Content-Type: multipart/alternative; boundary="0000000000006481fb057357ae70" Archived-At: Subject: Re: [Cbor] CDDL and group matching [was: CDDL -04 ready] X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Aug 2018 21:21:54 -0000 --0000000000006481fb057357ae70 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Mon, Aug 13, 2018 at 12:32 PM Carsten Bormann wrote: > > > It's going to be an interoperability problem if there's non-determinism > in whether an item matches a CDDL spec. The spec needs to either give a > deterministic way to pick the next map item, or give a rule for declaring > CDDL specs invalid so that all valid ones don=E2=80=99t care what order t= he > implementation picks map items. > > Right. Reminds me of the =E2=80=9Cambiguous content model=E2=80=9D (agai= n, this was not > about ambiguity but non-determinism) discussion when SGML was finished. > Here, this was really complicated, as SGML had the =E2=80=9C&=E2=80=9D op= erator, which > allows the DTD specification to leave open in which sequence required > elements were to be included. =E2=80=9C&=E2=80=9D was thrown out for XML= because it made > things complicated that didn=E2=80=99t need to be. (Anne Br=C3=BCggemann= -Klein wrote a > whole thesis about this subject in 1993.) > > Unfortunately, for JSON (and due to its JSON heritage, for CBOR as well), > we can=E2=80=99t just throw arbitrary ordering out. > > So maybe we should do what SGML and then XML did: outlaw =E2=80=9Cambiguo= us > content models=E2=80=9D, as in =E2=80=9CFor compatibility, it is an error= if an element in > the document can match more than one occurrence of an element type in the > content model.=E2=80=9D (http://www.w3.org/TR/REC-xml#sec-element-content= ) > The fun part about making something an =E2=80=9Cerror=E2=80=9D in XML is = that a processor > does not have to check it. > > The problem we are discussing has not occurred in practice. The reason i= s > that it is natural, when writing the group for a map, to be quite > specific. Non-determinism only occurs in contrived examples. I=E2=80=99= d be > unhappy to damage CDDL because of contrived examples. I=E2=80=99d also b= e unhappy > having to wait for Anne to write another thesis. > So just saying =E2=80=9Cpathological specs introduce non-determinism, don= =E2=80=99t do > that=E2=80=9D is probably the best way to handle this for CDDL 1.0. > > > Unless I=E2=80=99m missing something, the current spec doesn't have a m= ap > matching algorithm. > > No, the semantics of matching a map against a group is not given as an > algorithm: > > =E2=80=9Ca map expression [=E2=80=A6] matches a valid CBOR map the key/va= lue pairs > of which can be ordered in such a way that the resulting sequence > matches the group expression [=E2=80=A6]=E2=80=9D > > (Not having an algorithm, but a logical expression, is a feature, not a > bug.) > I'd count that as an algorithm, thanks. It requires that implementations be equivalent to doing the O(n!) search over all permutations of the map contents. Of course, no implementation will actually implement an O(n!) algorithm, so they'll pick something O(n) that we hope gives the same answer in all cases. If it doesn't give the same answer, that's an interoperability bug. The definition of the group matching mentions PEGs, but only for the > choice, this probably needs to be added once more. > > > Even if it said "implementations MUST translate the group to PEG in ...= . > way and then run it", which it doesn't, as described above that wouldn't = be > sufficient. 0.8.5 may not be up to what you intend, but your intent hasn= =E2=80=99t > made it into the spec, and it needs to before there'll be a point to a WG= LC. > > Anything else to fix in addition to > =E2=80=94 calling out PEG semantics for occurrence indicators > The PEG paper needs to be a normative reference. =E2=80=94 making specifications where the non-determinism can leads to mult= iple > interpretations invalid > If "pathological spec" is clearly defined, such that creative spec-writers can be sure they've written non-pathological CDDL without talking to this group and without being experts on parsing, I'm happy with that. - I'm not confident that the current "cut" language is clear: "A cut in this position means that once the map key matches the name part of an entry that can be produced and carries a cut, other potential matches for the key that occur in later entries in the group of the map are no longer allowed." Do you mean that any item whose key matches the 'cut' key, ignoring the 'cut' key's occurrence indicators, does not match a later group-entry in the same group? If you mean something else, it didn't come through. I think that'll be sufficient. Jeffrey --0000000000006481fb057357ae70 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Mon, Aug 13= , 2018 at 12:32 PM Carsten Bormann <cabo= @tzi.org> wrote:

> It's going to be an interoperability problem if there's non-de= terminism in whether an item matches a CDDL spec. The spec needs to either = give a deterministic way to pick the next map item, or give a rule for decl= aring CDDL specs invalid so that all valid ones don=E2=80=99t care what ord= er the implementation picks map items.

Right.=C2=A0 Reminds me of the =E2=80=9Cambiguous content model=E2=80=9D (a= gain, this was not about ambiguity but non-determinism) discussion when SGM= L was finished.=C2=A0 Here, this was really complicated, as SGML had the = =E2=80=9C&=E2=80=9D operator, which allows the DTD specification to lea= ve open in which sequence required elements were to be included.=C2=A0 =E2= =80=9C&=E2=80=9D was thrown out for XML because it made things complica= ted that didn=E2=80=99t need to be.=C2=A0 (Anne Br=C3=BCggemann-Klein wrote= a whole thesis about this subject in 1993.)

Unfortunately, for JSON (and due to its JSON heritage, for CBOR as well), w= e can=E2=80=99t just throw arbitrary ordering out.

So maybe we should do what SGML and then XML did: outlaw =E2=80=9Cambiguous= content models=E2=80=9D, as in =E2=80=9CFor compatibility, it is an error = if an element in the document can match more than one occurrence of an elem= ent type in the content model.=E2=80=9D (http://www.= w3.org/TR/REC-xml#sec-element-content)
The fun part about making something an =E2=80=9Cerror=E2=80=9D in XML is th= at a processor does not have to check it.

The problem we are discussing has not occurred in practice.=C2=A0 The reaso= n is that it is natural, when writing the group for a map, to be quite spec= ific.=C2=A0 Non-determinism only occurs in contrived examples.=C2=A0 I=E2= =80=99d be unhappy to damage CDDL because of contrived examples.=C2=A0 I=E2= =80=99d also be unhappy having to wait for Anne to write another thesis. So just saying =E2=80=9Cpathological specs introduce non-determinism, don= =E2=80=99t do that=E2=80=9D is probably the best way to handle this for CDD= L 1.0.

> Unless I=E2=80=99m missing something, the current spec doesn't hav= e a map matching algorithm.

No, the semantics of matching a map against a group is not given as an algo= rithm:

=E2=80=9Ca map expression [=E2=80=A6] matches a valid CBOR map the key/valu= e pairs
of which can be ordered in such a way that the resulting sequence
matches the group expression [=E2=80=A6]=E2=80=9D

(Not having an algorithm, but a logical expression, is a feature, not a bug= .)

I'd count that as an algorithm, = thanks. It requires that=C2=A0implementations be equivalent to doing the O(= n!) search over all permutations of the map contents. Of course, no impleme= ntation will actually implement an O(n!) algorithm, so they'll pick som= ething O(n) that we hope gives the same answer in all cases. If it doesn= 9;t give the same answer, that's an interoperability bug.
The definition of the group matching mentions PEGs, but only for the choice= , this probably needs to be added once more.

> Even if it said "implementations MUST translate the group to PEG = in .... way and then run it", which it doesn't, as described above= that wouldn't be sufficient. 0.8.5 may not be up to what you intend, b= ut your intent hasn=E2=80=99t made it into the spec, and it needs to before= there'll be a point to a WGLC.

Anything else to fix in addition to
=E2=80=94 calling out PEG semantics for occurrence indicators

The PEG paper needs to be a normative reference.

=E2=80=94 making specifications where the non-determinism can leads to mult= iple interpretations invalid

If "p= athological spec" is clearly defined, such that creative spec-writers = can be sure they've written non-pathological CDDL without talking to th= is group and without being experts on parsing, I'm happy with that.


- I'm not confident that the curre= nt "cut" language is clear: "A cut in this position means th= at once the map key matches the name part of an entry that can be produced = and carries a cut, other potential matches for the key that occur in later = entries in the group of the map are no longer allowed." Do you mean th= at any item whose key matches the 'cut' key, ignoring the 'cut&= #39; key's occurrence indicators, does not match a later group-entry in= the same group? If you mean something else, it didn't come through.

I think that'll be sufficient.

Jeffrey
--0000000000006481fb057357ae70-- From nobody Mon Aug 13 14:36:33 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DCC151310BF for ; Mon, 13 Aug 2018 14:36:32 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.897 X-Spam-Level: X-Spam-Status: No, score=-1.897 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.001, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Fz14rsVqDLYD for ; Mon, 13 Aug 2018 14:36:30 -0700 (PDT) Received: from mail-qk0-x243.google.com (mail-qk0-x243.google.com [IPv6:2607:f8b0:400d:c09::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B22BD1310A7 for ; Mon, 13 Aug 2018 14:36:30 -0700 (PDT) Received: by mail-qk0-x243.google.com with SMTP id n85-v6so12073519qke.8 for ; Mon, 13 Aug 2018 14:36:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to; bh=jLnE8C+rpp0VdMwGUaf9xKuRwVpieqazHATi3Avux1I=; b=BnthMJl2wPv8Si3MyPc8MFcL9hQEhNqo06OvULnpbFzPEz/plj8r3kdZOpHXnP/3b5 F47PLj6j+Ou+T1Nwd6IgRawSPYtBQOUy1B5NROs2W1MBe6wOGGu9WiF6jmOBITmDcCtk zmkuN65m3f1ysaNTLdv76ouVa9UYEl50PLubwI0nsgug9/Yvz0h7yy5MUmz+8V1N1FOE QK9bc0U8FWzux/JY1/p3Qh6hqJxzm1u476RL5et/xmSd/LIqrc3RU3cDQrCgHYJ+2vz9 1QKmj5enhnAIeb/7mxKTLS778BZIow6SB5zbxHzjqpvl1yCichXqxVSijQa/jXkRYnrY ziSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to; bh=jLnE8C+rpp0VdMwGUaf9xKuRwVpieqazHATi3Avux1I=; b=UwlsRTtSHIvPr4KwlEXZkbDNyZzThr9XTJjnplq4fwzR8pGn6WiksWk/r/qNm16T9r +JCeb9CYdlzaNAZCQYJY4zmz1ZKnUPdQDqIDd2I41iB82qbSi50M6bQZSOJGE4NFzdNM jqkBq3KFm8ig0GM1hK8HMwoImpzcSgMqSYhhVJQ+Rnc2Poen7bVv/Yeo1tvO5oKSpZHV NQQek1qhCNlhvZ+bC3zTDdQ2+LwtxtOHnnT3FUswR8Rd9N/N4Ce6VXOC1Vu0OWrM+m0j 7pYd7g1JwKLh5YuVrBPLytQeefBJVv771Pl7yf7gngnuvK53AOpWDQdKnWoHvORvuXqu 4k8w== X-Gm-Message-State: AOUpUlFg5Dho/tvv1zBvVlLcfOLQqPDmUyQoYMsi5q/+Uc8sd+Egz3Z6 e9Wrm6HFc89HwiFLyIqFt9hEVq6D3LJ0qeqBw4dkgg== X-Google-Smtp-Source: AA+uWPzy1v5Vh+kZcUQNbSqP2F91JlPf1yAyJxyHS0pEJEdPUTvN0kzNCPu4TOsYRZB9O1q1fVRJPYCD4wMoOXIUrPU= X-Received: by 2002:a37:1127:: with SMTP id b39-v6mr4023322qkh.149.1534196189574; Mon, 13 Aug 2018 14:36:29 -0700 (PDT) MIME-Version: 1.0 Sender: barryleiba.mailing.lists@gmail.com Received: by 2002:aed:2e65:0:0:0:0:0 with HTTP; Mon, 13 Aug 2018 14:36:29 -0700 (PDT) In-Reply-To: <153390338374.1696.2312375920423852282@ietfa.amsl.com> References: <153390338374.1696.2312375920423852282@ietfa.amsl.com> From: Barry Leiba Date: Mon, 13 Aug 2018 17:36:29 -0400 X-Google-Sender-Auth: 7ChZiEF8xh7jaMfR4vu8w3dZWTY Message-ID: To: cbor@ietf.org Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cbor] I-D Action: draft-ietf-cbor-cddl-04.txt X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Aug 2018 21:36:33 -0000 The chairs are starting a working group last call on this document, ending on Wednesday, 22 August. Everyone, please check this version out and let us know if there's something that's still missed. Also let us know if you think it's ready to go. Jeffrey and Carsten are already discussing Jeffrey's issue with the "can produce" terminology, so that's good.... Barry On Fri, Aug 10, 2018 at 8:16 AM, wrote: > > A New Internet-Draft is available from the on-line Internet-Drafts directories. > This draft is a work item of the Concise Binary Object Representation Maintenance and Extensions WG of the IETF. > > Title : Concise data definition language (CDDL): a notational convention to express CBOR and JSON data structures > Authors : Henk Birkholz > Christoph Vigano > Carsten Bormann > Filename : draft-ietf-cbor-cddl-04.txt > Pages : 59 > Date : 2018-08-10 > > Abstract: > This document proposes a notational convention to express CBOR data > structures (RFC 7049). Its main goal is to provide an easy and > unambiguous way to express structures for protocol messages and data > formats that use CBOR or JSON. > > > The IETF datatracker status page for this draft is: > https://datatracker.ietf.org/doc/draft-ietf-cbor-cddl/ > > There are also htmlized versions available at: > https://tools.ietf.org/html/draft-ietf-cbor-cddl-04 > https://datatracker.ietf.org/doc/html/draft-ietf-cbor-cddl-04 > > A diff from the previous version is available at: > https://www.ietf.org/rfcdiff?url2=draft-ietf-cbor-cddl-04 > > > Please note that it may take a couple of minutes from the time of submission > until the htmlized version and diff are available at tools.ietf.org. > > Internet-Drafts are also available by anonymous FTP at: > ftp://ftp.ietf.org/internet-drafts/ > > _______________________________________________ > CBOR mailing list > CBOR@ietf.org > https://www.ietf.org/mailman/listinfo/cbor From nobody Mon Aug 13 14:38:13 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8C4AD1310DE for ; Mon, 13 Aug 2018 14:38:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.897 X-Spam-Level: X-Spam-Status: No, score=-1.897 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.001, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id O0Cm-ijIuiPT for ; Mon, 13 Aug 2018 14:38:08 -0700 (PDT) Received: from mail-qt0-x244.google.com (mail-qt0-x244.google.com [IPv6:2607:f8b0:400d:c0d::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4D42B1310E0 for ; Mon, 13 Aug 2018 14:38:08 -0700 (PDT) Received: by mail-qt0-x244.google.com with SMTP id y5-v6so19069795qti.12 for ; Mon, 13 Aug 2018 14:38:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:from:date:message-id:subject:to; bh=rdKvarP9ydiSYnSUK06CL9lLH9igdtp0XP5iiRQ4BO0=; b=HzcDStZnXgKCWnfCpX6ty9Y9x1mReIlDNuc1aBINanp4nufZ7UrhxTBeVjvX8KwkKX Slo2uRhAfMo0LsyHoB96ch7fykgDLCTaGcT1n7+bEhGd0Xr2mMA6DwaqVhbxKeL6yOxw 2oayFuYPud3kekA17wXdApoIIvsEU3zuVpYb14JMb42avdOBmeSbu8WOoOmBcMdYe5Dl EMXJx8DwW2Jf4roHxYpDfJVhr2XQ5SgfGfm26pimpAUoMm1Vd3/qFqoI9/szrqKNJQI/ fzrsx7pwwGiak/ZKv1sLTdEsSha21sRa7cFud+Ugg/yQieuiXblmHAM8edMSQtF57TP2 UOKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:from:date:message-id:subject :to; bh=rdKvarP9ydiSYnSUK06CL9lLH9igdtp0XP5iiRQ4BO0=; b=Y/vaiBhiM4LN+zl1NGt9u4KyAF9gewjhFSawSXXb9tB0UkdrAp+wXd6iRia5tg5J7q 5G7GutEHLiRla05OfDDBysItYaB+RIAvRbMXWVh1rkY3jwtqIovjlZpl01OflLDyJiGt aJpFxoIYynhb+qAKryVFFk6Hvuaj/Wa7nQcny7l0vtb/VlKC5snDddsQWBOMLobVCW7k shTHuSN0ojMCpl0r1fsZ+bIISP2Pu2WIKWmMgajCepJ+VflgSlugxycHcTJ7d+izk2jl DIPUMHqZRDoa6KD2TQgHNv0npcsHfMUTksPBWWsaByO6vEubFRR2IOE+TU+AqORmA+sM ScdA== X-Gm-Message-State: AOUpUlHuHQPugwvJQKxtzkDEseJwSFzFZIsbg17q9j2K2eIyI9+RrdGr u7iJ/RRDK+C5jIXmH0KPD4eYhWFhkAir9o4B+EIywA== X-Google-Smtp-Source: AA+uWPwo6MYq45t+VQxAuTBkWfAAtY9gsoAeiQGQHCSHga9kLWEsHzJtwXcdEHYrjgZ+DMTW5x8n7Xrf/kBpSVGRdzU= X-Received: by 2002:ac8:3961:: with SMTP id t30-v6mr18777315qtb.56.1534196287097; Mon, 13 Aug 2018 14:38:07 -0700 (PDT) MIME-Version: 1.0 Sender: barryleiba.mailing.lists@gmail.com Received: by 2002:aed:2e65:0:0:0:0:0 with HTTP; Mon, 13 Aug 2018 14:38:06 -0700 (PDT) From: Barry Leiba Date: Mon, 13 Aug 2018 17:38:06 -0400 X-Google-Sender-Auth: 17do9-2oLpStlitECEE9umMZxeE Message-ID: To: cbor@ietf.org Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: [Cbor] Working Group Last Call on draft-ietf-cbor-cddl-04 X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Aug 2018 21:38:12 -0000 (Re-sending this, because I meant to change the subject line and forgot to, and it's too easy to miss a WGLC that doesn't have an explicit subject.) The chairs are starting a working group last call on this document, ending on Wednesday, 22 August. Everyone, please check this version out and let us know if there's something that's still missed. Also let us know if you think it's ready to go. Jeffrey and Carsten are already discussing Jeffrey's issue with the "can produce" terminology, so that's good.... Barry On Fri, Aug 10, 2018 at 8:16 AM, wrote: > > A New Internet-Draft is available from the on-line Internet-Drafts directories. > This draft is a work item of the Concise Binary Object Representation Maintenance and Extensions WG of the IETF. > > Title : Concise data definition language (CDDL): a notational convention to express CBOR and JSON data structures > Authors : Henk Birkholz > Christoph Vigano > Carsten Bormann > Filename : draft-ietf-cbor-cddl-04.txt > Pages : 59 > Date : 2018-08-10 > > Abstract: > This document proposes a notational convention to express CBOR data > structures (RFC 7049). Its main goal is to provide an easy and > unambiguous way to express structures for protocol messages and data > formats that use CBOR or JSON. > > > The IETF datatracker status page for this draft is: > https://datatracker.ietf.org/doc/draft-ietf-cbor-cddl/ > > There are also htmlized versions available at: > https://tools.ietf.org/html/draft-ietf-cbor-cddl-04 > https://datatracker.ietf.org/doc/html/draft-ietf-cbor-cddl-04 > > A diff from the previous version is available at: > https://www.ietf.org/rfcdiff?url2=draft-ietf-cbor-cddl-04 > > > Please note that it may take a couple of minutes from the time of submission > until the htmlized version and diff are available at tools.ietf.org. > > Internet-Drafts are also available by anonymous FTP at: > ftp://ftp.ietf.org/internet-drafts/ > > _______________________________________________ > CBOR mailing list > CBOR@ietf.org > https://www.ietf.org/mailman/listinfo/cbor From nobody Thu Aug 16 08:57:12 2018 Return-Path: X-Original-To: cbor@ietf.org Delivered-To: cbor@ietfa.amsl.com Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id C8A1D1252B7; Thu, 16 Aug 2018 08:57:10 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: IETF Meeting Session Request Tool To: Cc: cbor@ietf.org, cbor-chairs@ietf.org, barryleiba@gmail.com, aamelnikov@fastmail.fm X-Test-IDTracker: no X-IETF-IDTracker: 6.83.1 Auto-Submitted: auto-generated Precedence: bulk Message-ID: <153443503074.12066.4245088514406138070.idtracker@ietfa.amsl.com> Date: Thu, 16 Aug 2018 08:57:10 -0700 Archived-At: Subject: [Cbor] cbor - New Meeting Session Request for IETF 103 X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Aug 2018 15:57:11 -0000 A new meeting session request has just been submitted by Barry Leiba, a Chair of the cbor working group. --------------------------------------------------------- Working Group Name: Concise Binary Object Representation Maintenance and Extensions Area Name: Applications and Real-Time Area Session Requester: Barry Leiba Number of Sessions: 1 Length of Session(s): 2 Hours Number of Attendees: 50 Conflicts to Avoid: First Priority: artarea dispatch secdispatch saag extra jmap uta dmarc dcrup iasa2 core ace t2trg Second Priority: suit lwig anima 6tisch Third Priority: acme roll detnet sacm lpwan dots People who must be present: Carsten Bormann Barry Leiba Alexey Melnikov Francesca Palombini Resources Requested: Special Requests: Please avoid conflict with ART and IoT-related BoFs Please avoid Friday: one chair has to leave on Thursday night --------------------------------------------------------- From nobody Mon Aug 20 22:41:25 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 52A65130EC7 for ; Mon, 20 Aug 2018 22:41:23 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.901 X-Spam-Level: X-Spam-Status: No, score=-1.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gMivHjA0cNZW for ; Mon, 20 Aug 2018 22:41:21 -0700 (PDT) Received: from mail2.augustcellars.com (augustcellars.com [50.45.239.150]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6FA3D130EB9 for ; Mon, 20 Aug 2018 22:41:21 -0700 (PDT) Received: from Jude (73.180.8.170) by mail2.augustcellars.com (192.168.0.56) with Microsoft SMTP Server (TLS) id 15.0.1347.2; Mon, 20 Aug 2018 22:37:28 -0700 From: Jim Schaad To: References: In-Reply-To: Date: Mon, 20 Aug 2018 22:41:11 -0700 Message-ID: <02a301d43911$930a9ac0$b91fd040$@augustcellars.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 16.0 Content-Language: en-us Thread-Index: AQElaBaDJH3FKqmIGQcQ+7fjwawqBKYmhG/w X-Originating-IP: [73.180.8.170] Archived-At: Subject: Re: [Cbor] Working Group Last Call on draft-ietf-cbor-cddl-04 X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Aug 2018 05:41:23 -0000 Section 2.1 - the following could probably be made simpler: each of which can be a name/value pair or a more complex group expression composed of name/ value pairs. Perhaps - each of which can be a name/value pair or a group. Section 2.1 - for clarity suggest: s/group is important, as it is the information/ group is important, as the sequence is the information/ Section 2.1 - It is unclear for Figure 1 if the term 'right' is being used in the sense of directly or in the sense of correctly. Section 2.1 - I still suggest adding the parens rather than have an implicit group definition in figure 1 for better clarity. Either that or reverse the order of Figure 2 and Figure 3 so that you define a group first and then do in an implicit manner. I really don't like the order of presentation of these figures. They should be in the order of education that the order of common usage - this is a teaching document. (Thus 2, 3, 4, 1 is my preferred order.) Section 2.1.2 - According to the text in the syntax section the following is legal Group = ( high:low The start is optional, the end is optional. Nothing is said about pairing. Section 2.1.2 - third bullet - the third case that a colon is ok would be if the key is never going to be used (i.e. a group for an array) Section 2.2.1 - para #2 - sentence 1 - you seem to be missing some type of transition or jump before the reference to appendix B Appendix B - per the grammar the following is legal s /= "string" s //= (foo:bar) This is because all of the assign operators are in a single rule rather than being broken up according to if a type or a group assignment is being made. Section 2.2.2.1 In the last paragraph, did you intend to say the right hand side no the left hand side? I do not believe that the following would ever be legal. min .. max = 5 Section 2.2.3 - The description in the first paragraph does not appear to match the ABNF syntax. Nor from the description would I know how to deal with encoding some of the options. Per the text - the following appear to be legal f1 = #(tstr) ; optionally followed by major type option not positive if the additional info is optional f2 = #.999(tstr) ; optional major type followed by additional info f3 = #1.22(tstr) ; no restriction on the value of the major type There is nothing in the description that talks about when the parens would be used and when thy are omitted. Section 3.1 - What should the behavior be for a hex value which for which | val | % 2 == 1? Don't' see anything that says that the length must be even. Not a problem for B64 if one assumes that trailing equals are optional. Section 3.5.1 - Example PersonalData - it says that the first and family names are a map of text strings, but I think they are just text strings unless my eyes are deceiving me. Section 3.8 - I am not sure what is meant by combining multiple controls on a single target. I assume that the syntax "(target .control-operator1 controller1) .control-operator2 controller2" is allowed. In what way would this not normally be equivalent and should this be discussed in the document? Section 3.8.6 - the order of the controls and the description of the constraint do not match in order. Current text says that .gt is "equal to" Section 3.9 - the rule that definitions for socket names MUST be augmentations conflicts with the previous rule which says that the first definition could use '='. Section 5 - I think that it would be good to add some text about extension points. If nothing else they make the security analysis for an application that uses them more difficult. This is esp. true given that we have no agreement on how things do or do not match when using extension points and anys. Section 6 - Are there any requirements that the Expert Reviewer should apply to the syntax to make sure that the resulting items will all be able to be parsed? Specifically something like .operator X Y might cause problems. Appendix A - Better text might be "This appendix is intentionally left empty." As text of the appendix. This is not necessary if there is an expectation that the appendixes are going to be renumbered during publication. You should also include a comment to the RFC Editor about the fact that you want to keep this appendix section and why. I would expect the IESG to complain about this. Appendix C - You say that the order of the rules is significant in to cases, but I have no idea what those two cases are. s/his point/this point/ jim From nobody Tue Aug 21 02:37:46 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D8A3C130EAC for ; Tue, 21 Aug 2018 02:37:44 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.198 X-Spam-Level: X-Spam-Status: No, score=-4.198 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, URIBL_BLOCKED=0.001, URIBL_DBL_ABUSE_REDIR=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GvFanX97Pthz for ; Tue, 21 Aug 2018 02:37:42 -0700 (PDT) Received: from mailhost.informatik.uni-bremen.de (mailhost.informatik.uni-bremen.de [IPv6:2001:638:708:30c9::12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 43E69130DC3 for ; Tue, 21 Aug 2018 02:37:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at informatik.uni-bremen.de Received: from submithost.informatik.uni-bremen.de (submithost.informatik.uni-bremen.de [IPv6:2001:638:708:30c9::b]) by mailhost.informatik.uni-bremen.de (8.14.5/8.14.5) with ESMTP id w7L9b014017233; Tue, 21 Aug 2018 11:37:00 +0200 (CEST) Received: from [192.168.217.114] (p54A6C24D.dip0.t-ipconnect.de [84.166.194.77]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by submithost.informatik.uni-bremen.de (Postfix) with ESMTPSA id 41vlv00yzqzDWZ4; Tue, 21 Aug 2018 11:37:00 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) From: Carsten Bormann In-Reply-To: <02a301d43911$930a9ac0$b91fd040$@augustcellars.com> Date: Tue, 21 Aug 2018 11:36:59 +0200 Cc: cbor@ietf.org X-Mao-Original-Outgoing-Id: 556537018.1663539-5b389130af6292f5b801b1ad92c4969c Content-Transfer-Encoding: quoted-printable Message-Id: References: <02a301d43911$930a9ac0$b91fd040$@augustcellars.com> To: Jim Schaad X-Mailer: Apple Mail (2.3445.9.1) Archived-At: Subject: Re: [Cbor] Working Group Last Call on draft-ietf-cbor-cddl-04 X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Aug 2018 09:37:45 -0000 Hi Jim, thank you! I have prepared a new version at https://cbor-wg.github.io/cddl/draft-ietf-cbor-cddl.html http://gg.gg/cddl-editor-diff This should cover all your comments except for the first one below; some = more replies to specific ones follow: > Section 2.1 - I still suggest adding the parens rather than have an = implicit > group definition in figure 1 for better clarity. Either that or = reverse the > order of Figure 2 and Figure 3 so that you define a group first and = then do > in an implicit manner. I really don't like the order of presentation = of > these figures. They should be in the order of education that the = order of > common usage - this is a teaching document. (Thus 2, 3, 4, 1 is my > preferred order.) Hmm, the current sequence is the result of acting on an earlier comment = that the examples (which were then closer to the sequence you suggest) where = starting with an uncommon case. If more people have an opinion on this, I=E2=80=99d like to hear that. > Appendix B - per the grammar the following is legal > s /=3D "string" > s //=3D (foo:bar) >=20 > This is because all of the assign operators are in a single rule = rather than > being broken up according to if a type or a group assignment is being = made. Right. (This example actually triggers a bug in the cddl tool, thank = you.) The grammar allows a bit more than is semantically meaningful. Do you think this is a big problem? I have split assign into assignt and assigng, but I=E2=80=99m not sure I = like that very much. > Section 2.2.2.1 In the last paragraph, did you intend to say the = right hand > side no the left hand side? I do not believe that the following would = ever > be legal. > min .. max =3D 5 This is about the LHS of the range operator, not of the whole rule. = Clarified. > Section 3.1 - What should the behavior be for a hex value which for = which | > val | % 2 =3D=3D 1? Don't' see anything that says that the length = must be even. > Not a problem for B64 if one assumes that trailing equals are = optional. (That is actually a problem for b64 as well, because the length cannot = be 1 modulo 4.) Clarified a bit. > Section 3.8 - I am not sure what is meant by combining multiple = controls on > a single target. I assume that the syntax "(target .control-operator1 > controller1) .control-operator2 controller2" is allowed. In what way = would > this not normally be equivalent and should this be discussed in the > document? This was a comment about a limitation of the CDDL tool; we don't need that comment. (The example you give is indeed allowed by CDDL.) > Section 6 - Are there any requirements that the Expert Reviewer should = apply > to the syntax to make sure that the resulting items will all be able = to be > parsed? Specifically something like .operator X Y might cause = problems. The registration cannot add syntax, it only can add control operators that use the existing syntax. > Appendix A - Better text might be "This appendix is intentionally = left > empty." As text of the appendix. This is not necessary if there is an > expectation that the appendixes are going to be renumbered during > publication. You should also include a comment to the RFC Editor = about the > fact that you want to keep this appendix section and why. I would = expect > the IESG to complain about this. Good point. Filled the gap with the examples (as Appendix H). > Appendix C - You say that the order of the rules is significant in to = cases, > but I have no idea what those two cases are. Hmm, text for those two cases is in the appendix: # The first rule defines the semantics of the entire specification; = [=E2=80=A6] # The creation of the type choices and group choices from the # right hand sides of rules is the other case where rule order can be = significant. Do we need to clarify that more? Gr=C3=BC=C3=9Fe, Carsten From nobody Tue Aug 21 10:07:11 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 53D59130DBE for ; Tue, 21 Aug 2018 10:06:56 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.896 X-Spam-Level: X-Spam-Status: No, score=-1.896 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.001, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id arTR069iWn3r for ; Tue, 21 Aug 2018 10:06:53 -0700 (PDT) Received: from mail-qt0-x232.google.com (mail-qt0-x232.google.com [IPv6:2607:f8b0:400d:c0d::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4D549130F9A for ; Tue, 21 Aug 2018 10:06:50 -0700 (PDT) Received: by mail-qt0-x232.google.com with SMTP id z8-v6so21011436qto.9 for ; Tue, 21 Aug 2018 10:06:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:from:date:message-id:subject:to; bh=Wxgix+mTviXKkslt7bSnc4e619v3YY0hMWxG02UUqyg=; b=kxhPrtye/isS4kXXveYcjON1RahrnX7fIC1VjmHRZIr9uAYj6dBc3f17imHxKOcvco vSeY5vOlm3E63ChXOj8gh/aWsBzs57SNmZIHm8xWwp2V4xAqzzmaqtGGlI75PPBQDFXm 9cztEicLN1H5k50xPEX9X4Ubb1iXQojFOZUbAAi2Pf0n4voVxR96+uoA59U4BIPTmupG vf1Po76jPyNlr7tgY9PVUptc+dFOQaTHNBkdZQ1Mfs47j0YNki0yT1VOfEvdwJ/bDurh 5tL9JWGlOyu01OdoGugUNXs0dQY015g7hYgXpWl6//cctxodJApRVNv3HDU5uTrW1+qb ylgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:from:date:message-id:subject :to; bh=Wxgix+mTviXKkslt7bSnc4e619v3YY0hMWxG02UUqyg=; b=Z5nYwkuOKC0uzzFRmmJqORCwTNE67OAk5hh0gjkkhaajxX5vG4DfkS+gLXKXXSjoSf dhCbLoqHxxjYEtGxKvElhxxGn8OY+GzgWc+Qe+gMq9iduN+alqeJZ+3OG3yEBigYPZxb EmkRBLUO/cXex/rXiqH8SajxRwuRb0Wkyj2f5NZHc1+a17MmPY2mSwcxG95E+tYhoWfN sUzRLMfw1rDymALF+6oKnBsDXD0vAkluddNgzNt2X5qfZVPMzICGBlZDx1+2r25ubfvn kJ3MufUykNqPA4ZOIFp59epm9jY/RFoOhUUawdRAJlbytfVIo11WWLfJWIQf+p6mZPSi K+tg== X-Gm-Message-State: AOUpUlHA4SMBBNmOhQJ//Oe5IFjuKUKiqbiecvduqBNHvUIV5ZImzvHd D7kCczMMVQUgPXlmDu8bHr7gAxIYe6nPPQsaLRmoR9o8 X-Google-Smtp-Source: AA+uWPy3gnvgFIj/di1QOsP1l7KVKUjQBj6fe+EsT5vXuENeH6wta3u0LJPjVYDHDvt0bOH4DW816ZKswdEVrlCfxms= X-Received: by 2002:aed:2e02:: with SMTP id j2-v6mr9324716qtd.180.1534871209197; Tue, 21 Aug 2018 10:06:49 -0700 (PDT) MIME-Version: 1.0 Sender: barryleiba.mailing.lists@gmail.com Received: by 2002:ac8:7484:0:0:0:0:0 with HTTP; Tue, 21 Aug 2018 10:06:48 -0700 (PDT) From: Barry Leiba Date: Tue, 21 Aug 2018 13:06:48 -0400 X-Google-Sender-Auth: iFhmFXgxOgOMtu7gIzVbguTicF8 Message-ID: To: cbor@ietf.org Content-Type: multipart/mixed; boundary="0000000000003363850573f50d5e" Archived-At: Subject: [Cbor] Webex meeting invitation: CBOR working group conference call X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Aug 2018 17:07:10 -0000 --0000000000003363850573f50d5e Content-Type: multipart/alternative; boundary="0000000000003363820573f50d5c" --0000000000003363820573f50d5c Content-Type: text/plain; charset="UTF-8" REMINDER: The second call is on Wednesday, 22 August, at 15:00 UTC. That's "tomorrow" as I write this. (Note that there is a meeting password, "constrained"; you're likely not to need it, but there it is in case you do.) ---------------------------------- Here is the WebEx information for the CBOR conference calls. There's an ICS attachment here, or you can click the "Add to Calendar" button to get the standard WebEx one. The calls will repeat every two weeks, and the purpose of the calls is to check the progress of the work on the CDDL and CBOR-bis documents, and to keep it moving forward. *CBOR working group conference call* Every 2 weeks on Wednesday, from Wednesday, August 8, 2018, to Wednesday, October 31, 2018 11:00 am | Eastern Daylight Time (New York, GMT-04:00) | 1 hr Meeting number (access code): 311 714 693 Meeting password: constrained Add to Calendar When it's time, join the meeting . *Join by phone* *1-650-479-3208* Call-in toll number (US/Canada) Can't join the meeting? IMPORTANT NOTICE: Please note that this WebEx service allows audio and other information sent during the session to be recorded, which may be discoverable in a legal matter. By joining this session, you automatically consent to such recordings. If you do not consent to being recorded, discuss your concerns with the host or do not join the session. _______________________________________________ CBOR mailing list CBOR@ietf.org https://www.ietf.org/mailman/listinfo/cbor --0000000000003363820573f50d5c Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
REMINDER: The second call is on We= dnesday, 22 August, at 15:00 UTC.
That's "tomorrow" as I write this.=C2=A0 (Note = that there is a meeting password, "constrained"; you're likel= y not to need it, but there it is in case you do.)

----= ------------------------------
Here is the WebEx information for = the CBOR conference calls.=C2=A0 There's an ICS attachment here, or you= can click the "Add to Calendar" button to get the standard WebEx= one.

The calls will repeat every two weeks, and t= he purpose of the calls is to check the progress of the work on the CDDL an= d CBOR-bis documents, and to keep it moving forward.

CBOR working group conference call
Every 2 weeks on Wednesday, from Wednesday, August 8, 2018, to = Wednesday, October 31, 2018
11:00 am=C2=A0=C2=A0|=C2=A0=C2=A0Eastern Daylight Time (New Yor= k, GMT-04:00)=C2=A0=C2=A0|=C2=A0=C2=A01 hr
Meeting number (access code): 311 714 693
=09
Meeting password: constrained

=09
= =C2=A0
Add to Calendar
When it's time, join the meeting.
= =C2=A0
=09
Join by phone
1-650-479-3208=C2=A0Call-in toll num= ber (US/Canada)
=C2= =A0
Can't join the meeting?
=C2= =A0
IMPORTANT NOTICE: Please note that this WebEx service allows audio= and other information sent during the session to be recorded, which may be= discoverable in a legal matter. By joining this session, you automatically= consent to such recordings. If you do not consent to being recorded, discu= ss your concerns with the host or do not join the session.




_______________________________________________
CBOR mailing list
CBOR@ietf.org
https://www.ietf.org/mailman/listinfo/cbor


--0000000000003363820573f50d5c-- --0000000000003363850573f50d5e Content-Type: text/calendar; charset="US-ASCII"; name="CBOR working group conference call.ics" Content-Disposition: attachment; filename="CBOR working group conference call.ics" Content-Transfer-Encoding: base64 X-Attachment-Id: f_jkjzvcr80 QkVHSU46VkNBTEVOREFSClBST0RJRDotLy9NaWNyb3NvZnQgQ29ycG9yYXRpb24vL091dGxvb2sg MTAuMCBNSU1FRElSLy9FTgpWRVJTSU9OOjIuMApNRVRIT0Q6UFVCTElTSApCRUdJTjpWVElNRVpP TkUKVFpJRDpFYXN0ZXJuIFRpbWUKQkVHSU46U1RBTkRBUkQKRFRTVEFSVDoyMDE2MTEwMVQwMjAw MDAKUlJVTEU6RlJFUT1ZRUFSTFk7SU5URVJWQUw9MTtCWURBWT0xU1U7QllNT05USD0xMQpUWk9G RlNFVEZST006LTA0MDAKVFpPRkZTRVRUTzotMDUwMApUWk5BTUU6U3RhbmRhcmQgVGltZQpFTkQ6 U1RBTkRBUkQKQkVHSU46REFZTElHSFQKRFRTVEFSVDoyMDE2MDMwMVQwMjAwMDAKUlJVTEU6RlJF UT1ZRUFSTFk7SU5URVJWQUw9MTtCWURBWT0yU1U7QllNT05USD0zClRaT0ZGU0VURlJPTTotMDUw MApUWk9GRlNFVFRPOi0wNDAwClRaTkFNRTpEYXlsaWdodCBTYXZpbmdzIFRpbWUKRU5EOkRBWUxJ R0hUCkVORDpWVElNRVpPTkUKQkVHSU46VkVWRU5UCkNMQVNTOlBVQkxJQwpERVNDUklQVElPTjpc blxuSk9JTiBXRUJFWCBNRUVUSU5HXG5odHRwczovL2lldGYud2ViZXguY29tL2lldGYvai5waHA/ TVRJRD1tMmE4ODIxZjg0YmQ4YzU2MTg5YmU4MWYwMTIyN2U1OWZcbk1lZXRpbmcgbnVtYmVyIChh Y2Nlc3MgY29kZSk6IDMxMSA3MTQgNjkzXG5Ib3N0IGtleTogMjA1Mjc2XG5NZWV0aW5nIHBhc3N3 b3JkOiBjb25zdHJhaW5lZFxuXG5cblxuSk9JTiBCWSBQSE9ORVxuMS02NTAtNDc5LTMyMDggQ2Fs bC1pbiB0b2xsIG51bWJlciAoVVMvQ2FuYWRhKVxuXG5cblxuQ2FuJ3Qgam9pbiB0aGUgbWVldGlu Zz8gQ29udGFjdCBzdXBwb3J0IGhlcmU6XG5odHRwczovL2lldGYud2ViZXguY29tL2lldGYvbWNc blxuXG5JTVBPUlRBTlQgTk9USUNFOiBQbGVhc2Ugbm90ZSB0aGF0IHRoaXMgV2ViRXggc2Vydmlj ZSBhbGxvd3MgYXVkaW8gYW5kIG90aGVyIGluZm9ybWF0aW9uIHNlbnQgZHVyaW5nIHRoZSBzZXNz aW9uIHRvIGJlIHJlY29yZGVkLCB3aGljaCBtYXkgYmUgZGlzY292ZXJhYmxlIGluIGEgbGVnYWwg bWF0dGVyLiBZb3Ugc2hvdWxkIGluZm9ybSBhbGwgbWVldGluZyBhdHRlbmRlZXMgcHJpb3IgdG8g cmVjb3JkaW5nIGlmIHlvdSBpbnRlbmQgdG8gcmVjb3JkIHRoZSBtZWV0aW5nLlxuCkRUU1RBTVA6 MjAxODA4MDhUMTUwMDAwWgpVSUQ6OTY4YmEwNTItMDIzMy00Njk2LWI1OTUtNDY5YmY0YmU2OWNh ClBSSU9SSVRZOjUKRFRTVEFSVDtUWklEPSJFYXN0ZXJuIFRpbWUiOjIwMTgwODA4VDExMDAwMApE VEVORDtUWklEPSJFYXN0ZXJuIFRpbWUiOjIwMTgwODA4VDEyMDAwMApSUlVMRTpGUkVRPVdFRUtM WTtJTlRFUlZBTD0yO0JZREFZPVdFO0NPVU5UPTcKU0VRVUVOQ0U6MTUzMzY2MzM5NgpTVU1NQVJZ OkNCT1Igd29ya2luZyBncm91cCBjb25mZXJlbmNlIGNhbGwKVFJBTlNQOk9QQVFVRQpMT0NBVElP TjpodHRwczovL2lldGYud2ViZXguY29tL2lldGYvai5waHA/TVRJRD1tMmE4ODIxZjg0YmQ4YzU2 MTg5YmU4MWYwMTIyN2U1OWYKRU5EOlZFVkVOVApFTkQ6VkNBTEVOREFSCg== --0000000000003363850573f50d5e-- From nobody Tue Aug 21 10:38:02 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0CB63130F80 for ; Tue, 21 Aug 2018 10:37:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.2 X-Spam-Level: X-Spam-Status: No, score=-4.2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DG8URYazx0A6 for ; Tue, 21 Aug 2018 10:37:47 -0700 (PDT) Received: from mailhost.informatik.uni-bremen.de (mailhost.informatik.uni-bremen.de [IPv6:2001:638:708:30c9::12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 57E21130F84 for ; Tue, 21 Aug 2018 10:37:47 -0700 (PDT) X-Virus-Scanned: amavisd-new at informatik.uni-bremen.de Received: from submithost.informatik.uni-bremen.de (submithost.informatik.uni-bremen.de [134.102.201.11]) by mailhost.informatik.uni-bremen.de (8.14.5/8.14.5) with ESMTP id w7LHbhtU002415; Tue, 21 Aug 2018 19:37:43 +0200 (CEST) Received: from [192.168.217.114] (p54A6C24D.dip0.t-ipconnect.de [84.166.194.77]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by submithost.informatik.uni-bremen.de (Postfix) with ESMTPSA id 41vyYg1s93zDWfS; Tue, 21 Aug 2018 19:37:43 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) From: Carsten Bormann In-Reply-To: Date: Tue, 21 Aug 2018 19:37:42 +0200 Cc: cbor@ietf.org X-Mao-Original-Outgoing-Id: 556565860.69458-f2bbf4c94233b36bd047e5ad382e30ea Content-Transfer-Encoding: quoted-printable Message-Id: <49E5A7B2-049D-41AB-AE7A-E698AFBECDFD@tzi.org> References: <153388117963.28907.13862811574346222830.idtracker@ietfa.amsl.com> <77BEDB78-0C2D-4BD4-95B4-DB3556B22353@tzi.org> <36637D9F-B783-4446-BA39-DFE9C7270907@tzi.org> <27A69EAD-DEDC-4FD6-8812-F912BE9564DF@tzi.org> <750143B4-0703-4237-B283-695F040E1C00@tzi.org> To: Jeffrey Yasskin X-Mailer: Apple Mail (2.3445.9.1) Archived-At: Subject: Re: [Cbor] CDDL and group matching [was: CDDL -04 ready] X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Aug 2018 17:38:00 -0000 Hi Jeffrey, > If =E2=80=9Cpathological spec" is clearly defined, such that creative = spec-writers can be sure they've written non-pathological CDDL without = talking to this group and without being experts on parsing, I'm happy = with that. Please have a look at the most recent editor=E2=80=99s copy. > - I'm not confident that the current "cut" language is clear: "A cut = in this position means that once the map key matches the name part of an = entry that can be produced and carries a cut, other potential matches = for the key that occur in later entries in the group of the map are no = longer allowed." Do you mean that any item whose key matches the =E2=80=98= cut' key, ignoring the 'cut' key's occurrence indicators, There are no occurrence indicators on the key =E2=80=94 occurrence = indicators are always on a whole group entry. They are not ignored (why should they be?), but there are only a few = cases that make a lot of sense: ? Key: value That is the classical =E2=80=9Cif there is a map entry named =E2=80=9CKey=E2= =80=9D, it needs to have value type =E2=80=9Cvalue=E2=80=9D. Key: value The cut is a bit inconsequential here, as there needs to be a member in = the map with the right key/value, since =E2=80=9CKey=E2=80=9D is a = single-value type, this makes it impossible to match a =C2=BB{ = =E2=80=9CKey=E2=80=9D: =E2=80=9Cfoo=E2=80=9D }=C2=AB (wrong value type) = anyway. type =3D> value with an occurrence indicator of 1*1 tends to give you non-deterministic = results if the members could be matched by another entry, and this is = less likely to occur. * type =3D> value essentially cuts after all members whose keys have =E2=80=9Ctype=E2=80=9D,= so this does have a meaning if it is followed by something like * any =3D> any > does not match a later group-entry in the same group? If you mean = something else, it didn=E2=80=99t come through. Please do have a look at the additional text. Let=E2=80=99s see if we can lay this to rest tomorrow! Gr=C3=BC=C3=9Fe, Carsten From nobody Tue Aug 21 19:32:51 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1D240130E60 for ; Tue, 21 Aug 2018 19:32:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_DBL_ABUSE_REDIR=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id A84n2wSVFo5c for ; Tue, 21 Aug 2018 19:32:46 -0700 (PDT) Received: from mail2.augustcellars.com (augustcellars.com [50.45.239.150]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4981D130E7E for ; Tue, 21 Aug 2018 19:32:45 -0700 (PDT) Received: from Jude (73.180.8.170) by mail2.augustcellars.com (192.168.0.56) with Microsoft SMTP Server (TLS) id 15.0.1347.2; Tue, 21 Aug 2018 19:28:46 -0700 From: Jim Schaad To: 'Carsten Bormann' CC: References: <02a301d43911$930a9ac0$b91fd040$@augustcellars.com> In-Reply-To: Date: Tue, 21 Aug 2018 19:32:30 -0700 Message-ID: <000201d439c0$617c7f00$24757d00$@augustcellars.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 16.0 Thread-Index: AQElaBaDJH3FKqmIGQcQ+7fjwawqBALdLU+2AZEc87OmBS3j4A== Content-Language: en-us X-Originating-IP: [73.180.8.170] Archived-At: Subject: Re: [Cbor] Working Group Last Call on draft-ietf-cbor-cddl-04 X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Aug 2018 02:32:49 -0000 > -----Original Message----- > From: Carsten Bormann > Sent: Tuesday, August 21, 2018 2:37 AM > To: Jim Schaad > Cc: cbor@ietf.org > Subject: Re: [Cbor] Working Group Last Call on draft-ietf-cbor-cddl-04 >=20 > Hi Jim, >=20 > thank you! >=20 > I have prepared a new version at >=20 > https://cbor-wg.github.io/cddl/draft-ietf-cbor-cddl.html > http://gg.gg/cddl-editor-diff >=20 > This should cover all your comments except for the first one below; = some more > replies to specific ones follow: >=20 > > Section 2.1 - I still suggest adding the parens rather than have an > > implicit group definition in figure 1 for better clarity. Either = that > > or reverse the order of Figure 2 and Figure 3 so that you define a > > group first and then do in an implicit manner. I really don't like > > the order of presentation of these figures. They should be in the > > order of education that the order of common usage - this is a = teaching > > document. (Thus 2, 3, 4, 1 is my preferred order.) >=20 > Hmm, the current sequence is the result of acting on an earlier = comment that > the examples (which were then closer to the sequence you suggest) = where > starting with an uncommon case. > If more people have an opinion on this, I=E2=80=99d like to hear that. >=20 > > Appendix B - per the grammar the following is legal > > s /=3D "string" > > s //=3D (foo:bar) > > > > This is because all of the assign operators are in a single rule > > rather than being broken up according to if a type or a group = assignment is > being made. >=20 > Right. (This example actually triggers a bug in the cddl tool, thank = you.) The > grammar allows a bit more than is semantically meaningful. > Do you think this is a big problem? > I have split assign into assignt and assigng, but I=E2=80=99m not sure = I like that very > much. I found this while looking at trying to make a version of the grammar = that would work as LALR(n). (I doubt that I will get this mail out = before the call tomorrow.) I guess the question is one of should the = grammar be correct or does it just need to be good enough. >=20 > > Section 2.2.2.1 In the last paragraph, did you intend to say the > > right hand side no the left hand side? I do not believe that the > > following would ever be legal. > > min .. max =3D 5 >=20 > This is about the LHS of the range operator, not of the whole rule. = Clarified. Looks fine >=20 > > Section 3.1 - What should the behavior be for a hex value which for > > which | val | % 2 =3D=3D 1? Don't' see anything that says that the = length must be > even. > > Not a problem for B64 if one assumes that trailing equals are = optional. >=20 > (That is actually a problem for b64 as well, because the length cannot = be 1 > modulo 4.) Clarified a bit. That helps - and avoids the question of the hex string being right or = left padded. >=20 > > Section 3.8 - I am not sure what is meant by combining multiple > > controls on a single target. I assume that the syntax "(target > > .control-operator1 > > controller1) .control-operator2 controller2" is allowed. In what = way > > would this not normally be equivalent and should this be discussed = in > > the document? >=20 > This was a comment about a limitation of the CDDL tool; we don't need = that > comment. (The example you give is indeed allowed by CDDL.) >=20 > > Section 6 - Are there any requirements that the Expert Reviewer = should > > apply to the syntax to make sure that the resulting items will all = be > > able to be parsed? Specifically something like .operator X Y might = cause > problems. >=20 > The registration cannot add syntax, it only can add control operators = that use > the existing syntax. >=20 > > Appendix A - Better text might be "This appendix is intentionally > > left empty." As text of the appendix. This is not necessary if = there > > is an expectation that the appendixes are going to be renumbered > > during publication. You should also include a comment to the RFC > > Editor about the fact that you want to keep this appendix section = and > > why. I would expect the IESG to complain about this. >=20 > Good point. Filled the gap with the examples (as Appendix H). >=20 > > Appendix C - You say that the order of the rules is significant in = to > > cases, but I have no idea what those two cases are. >=20 > Hmm, text for those two cases is in the appendix: >=20 > # The first rule defines the semantics of the entire specification; = [=E2=80=A6] >=20 > # The creation of the type choices and group choices from the # right = hand > sides of rules is the other case where rule order can be significant. I don't even know what that sentence says. >=20 > Do we need to clarify that more? Yes I think that you do. I would like to get a mention of what the = second rule is right after saying that there are two cases where it is = significant. Having to fish a long ways down in the text to find the = second example makes things difficult. Jim >=20 > Gr=C3=BC=C3=9Fe, Carsten From nobody Wed Aug 22 05:30:29 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EF2BA130E0E for ; Wed, 22 Aug 2018 05:30:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.2 X-Spam-Level: X-Spam-Status: No, score=-4.2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iB4GcsvsVUXq for ; Wed, 22 Aug 2018 05:30:26 -0700 (PDT) Received: from mailhost.informatik.uni-bremen.de (mailhost.informatik.uni-bremen.de [IPv6:2001:638:708:30c9::12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6E84B127333 for ; Wed, 22 Aug 2018 05:30:26 -0700 (PDT) X-Virus-Scanned: amavisd-new at informatik.uni-bremen.de Received: from submithost.informatik.uni-bremen.de (submithost.informatik.uni-bremen.de [134.102.201.11]) by mailhost.informatik.uni-bremen.de (8.14.5/8.14.5) with ESMTP id w7MCTmOS006425; Wed, 22 Aug 2018 14:29:48 +0200 (CEST) Received: from [192.168.217.114] (p54A6C24D.dip0.t-ipconnect.de [84.166.194.77]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by submithost.informatik.uni-bremen.de (Postfix) with ESMTPSA id 41wRgw0SKzzDWnp; Wed, 22 Aug 2018 14:29:48 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) From: Carsten Bormann In-Reply-To: <000201d439c0$617c7f00$24757d00$@augustcellars.com> Date: Wed, 22 Aug 2018 14:29:47 +0200 Cc: cbor@ietf.org X-Mao-Original-Outgoing-Id: 556633785.446052-08bcca85c99b1834147a80b7daaf8741 Content-Transfer-Encoding: quoted-printable Message-Id: References: <02a301d43911$930a9ac0$b91fd040$@augustcellars.com> <000201d439c0$617c7f00$24757d00$@augustcellars.com> To: Jim Schaad X-Mailer: Apple Mail (2.3445.9.1) Archived-At: Subject: Re: [Cbor] Working Group Last Call on draft-ietf-cbor-cddl-04 X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Aug 2018 12:30:29 -0000 On Aug 22, 2018, at 04:32, Jim Schaad wrote: >=20 >> Hmm, text for those two cases is in the appendix: >>=20 >> # The first rule defines the semantics of the entire specification; = [=E2=80=A6] >>=20 >> # The creation of the type choices and group choices from the # right = hand >> sides of rules is the other case where rule order can be significant. >=20 > I don=E2=80=99t even know what that sentence says. The example is: A /=3D a A /=3D b A /=3D c The order of the rules matters, because the order in the resulting = choice matters: A =3D a / b / c >> Do we need to clarify that more? >=20 > Yes I think that you do. I would like to get a mention of what the = second rule is right after saying that there are two cases where it is = significant. Having to fish a long ways down in the text to find the = second example makes things difficult. OK, I cleaned this up a bit in the commit = https://github.com/cbor-wg/cddl/commit/ca3a044 =E2=80=94 I hope that = helps. Gr=C3=BC=C3=9Fe, Carsten From nobody Wed Aug 22 07:16:32 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1A0E4129385 for ; Wed, 22 Aug 2018 07:16:30 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.901 X-Spam-Level: X-Spam-Status: No, score=-1.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cGAWOBZaME9V for ; Wed, 22 Aug 2018 07:16:28 -0700 (PDT) Received: from mail2.augustcellars.com (augustcellars.com [50.45.239.150]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 36F84124BE5 for ; Wed, 22 Aug 2018 07:16:28 -0700 (PDT) Received: from Jude (73.180.8.170) by mail2.augustcellars.com (192.168.0.56) with Microsoft SMTP Server (TLS) id 15.0.1347.2; Wed, 22 Aug 2018 07:12:35 -0700 From: Jim Schaad To: Date: Wed, 22 Aug 2018 07:16:20 -0700 Message-ID: <002901d43a22$b4230500$1c690f00$@augustcellars.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 16.0 Thread-Index: AdQ6Ig1UdmJanL5yQ2yq8COZmj5IAQ== Content-Language: en-us X-Originating-IP: [73.180.8.170] Archived-At: Subject: [Cbor] Ambiguous CDDL statement X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Aug 2018 14:16:30 -0000 Using the grammar as defined today, the following statement is not well defined a = ( uint ) It is not clear if 'a' is a type or a group. rule - typename assignt type type - type1 type1 - type2 type2 - "(" type ")" or rule - groupname assign grpent grpent - "(" group ")" group - grpchoice grpchoice - grpent grpent - type jim From nobody Wed Aug 22 07:49:00 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A89D6130E11 for ; Wed, 22 Aug 2018 07:48:59 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.901 X-Spam-Level: X-Spam-Status: No, score=-1.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LnF2wkwYZiXo for ; Wed, 22 Aug 2018 07:48:58 -0700 (PDT) Received: from mail2.augustcellars.com (augustcellars.com [50.45.239.150]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E2BE1130E0F for ; Wed, 22 Aug 2018 07:48:57 -0700 (PDT) Received: from Jude (73.180.8.170) by mail2.augustcellars.com (192.168.0.56) with Microsoft SMTP Server (TLS) id 15.0.1347.2; Wed, 22 Aug 2018 07:45:05 -0700 From: Jim Schaad To: 'Carsten Bormann' CC: References: <02a301d43911$930a9ac0$b91fd040$@augustcellars.com> <000201d439c0$617c7f00$24757d00$@augustcellars.com> In-Reply-To: Date: Wed, 22 Aug 2018 07:48:49 -0700 Message-ID: <002c01d43a27$3e0ea630$ba2bf290$@augustcellars.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 16.0 Thread-Index: AQElaBaDJH3FKqmIGQcQ+7fjwawqBALdLU+2AZEc87MCxAe/uwK89AVKpdoEEXA= Content-Language: en-us X-Originating-IP: [73.180.8.170] Archived-At: Subject: Re: [Cbor] Working Group Last Call on draft-ietf-cbor-cddl-04 X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Aug 2018 14:49:00 -0000 That fixes my issue. > -----Original Message----- > From: Carsten Bormann > Sent: Wednesday, August 22, 2018 5:30 AM > To: Jim Schaad > Cc: cbor@ietf.org > Subject: Re: [Cbor] Working Group Last Call on draft-ietf-cbor-cddl-04 >=20 > On Aug 22, 2018, at 04:32, Jim Schaad wrote: > > > >> Hmm, text for those two cases is in the appendix: > >> > >> # The first rule defines the semantics of the entire specification; > >> [=E2=80=A6] > >> > >> # The creation of the type choices and group choices from the # = right > >> hand sides of rules is the other case where rule order can be = significant. > > > > I don=E2=80=99t even know what that sentence says. >=20 > The example is: >=20 > A /=3D a > A /=3D b > A /=3D c >=20 > The order of the rules matters, because the order in the resulting = choice > matters: >=20 > A =3D a / b / c >=20 > >> Do we need to clarify that more? > > > > Yes I think that you do. I would like to get a mention of what the = second rule > is right after saying that there are two cases where it is = significant. Having to > fish a long ways down in the text to find the second example makes = things > difficult. >=20 > OK, I cleaned this up a bit in the commit https://github.com/cbor- > wg/cddl/commit/ca3a044 =E2=80=94 I hope that helps. >=20 > Gr=C3=BC=C3=9Fe, Carsten From nobody Wed Aug 22 09:14:31 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8AAF712D7F8 for ; Wed, 22 Aug 2018 09:14:29 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.2 X-Spam-Level: X-Spam-Status: No, score=-4.2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xHePbKgo932d for ; Wed, 22 Aug 2018 09:14:27 -0700 (PDT) Received: from mailhost.informatik.uni-bremen.de (mailhost.informatik.uni-bremen.de [IPv6:2001:638:708:30c9::12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 054071274D0 for ; Wed, 22 Aug 2018 09:14:26 -0700 (PDT) X-Virus-Scanned: amavisd-new at informatik.uni-bremen.de Received: from submithost.informatik.uni-bremen.de (submithost.informatik.uni-bremen.de [134.102.201.11]) by mailhost.informatik.uni-bremen.de (8.14.5/8.14.5) with ESMTP id w7MGDmVU002730; Wed, 22 Aug 2018 18:13:48 +0200 (CEST) Received: from [192.168.217.114] (p54A6C24D.dip0.t-ipconnect.de [84.166.194.77]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by submithost.informatik.uni-bremen.de (Postfix) with ESMTPSA id 41wXfM5xsnzDWqk; Wed, 22 Aug 2018 18:13:47 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) From: Carsten Bormann In-Reply-To: <002901d43a22$b4230500$1c690f00$@augustcellars.com> Date: Wed, 22 Aug 2018 18:13:47 +0200 Cc: cbor@ietf.org X-Mao-Original-Outgoing-Id: 556647225.374867-4728afea3267fcade74a77f04abc33b0 Content-Transfer-Encoding: quoted-printable Message-Id: <09B8AFB0-17D4-4776-B7FE-06C03DF3389E@tzi.org> References: <002901d43a22$b4230500$1c690f00$@augustcellars.com> To: Jim Schaad X-Mailer: Apple Mail (2.3445.9.1) Archived-At: Subject: Re: [Cbor] Ambiguous CDDL statement X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Aug 2018 16:14:30 -0000 Hi Jim, thanks for this comment and for the good discussion we had in the = interim meeting. https://github.com/cbor-wg/cddl/commit/d9512e has the text I=E2=80=99m = proposing to add to -05 to address this. Gr=C3=BC=C3=9Fe, Carsten > On Aug 22, 2018, at 16:16, Jim Schaad wrote: >=20 > Using the grammar as defined today, the following statement is not = well > defined >=20 > a =3D ( uint ) >=20 > It is not clear if 'a' is a type or a group. >=20 > rule - typename assignt type > type - type1 > type1 - type2 > type2 - "(" type ")" >=20 > or >=20 > rule - groupname assign grpent > grpent - "(" group ")" > group - grpchoice > grpchoice - grpent > grpent - type >=20 >=20 >=20 > jim >=20 >=20 > _______________________________________________ > CBOR mailing list > CBOR@ietf.org > https://www.ietf.org/mailman/listinfo/cbor >=20 From nobody Wed Aug 22 12:21:59 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 08B61130DD3 for ; Wed, 22 Aug 2018 12:21:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.2 X-Spam-Level: X-Spam-Status: No, score=-4.2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pVWUNyfZa6PF for ; Wed, 22 Aug 2018 12:21:55 -0700 (PDT) Received: from mailhost.informatik.uni-bremen.de (mailhost.informatik.uni-bremen.de [IPv6:2001:638:708:30c9::12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4BE111292AD for ; Wed, 22 Aug 2018 12:21:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at informatik.uni-bremen.de Received: from submithost.informatik.uni-bremen.de (submithost.informatik.uni-bremen.de [IPv6:2001:638:708:30c9::b]) by mailhost.informatik.uni-bremen.de (8.14.5/8.14.5) with ESMTP id w7MJLqOJ028773 for ; Wed, 22 Aug 2018 21:21:52 +0200 (CEST) Received: from [192.168.217.114] (p54A6C24D.dip0.t-ipconnect.de [84.166.194.77]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by submithost.informatik.uni-bremen.de (Postfix) with ESMTPSA id 41wcqM6YfqzDWrX; Wed, 22 Aug 2018 21:21:51 +0200 (CEST) From: Carsten Bormann Content-Type: text/plain; charset=utf-8 X-Mao-Original-Outgoing-Id: 556658509.958876-9d0f428ace12b361dfda7dd21e122c53 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Date: Wed, 22 Aug 2018 21:21:51 +0200 Message-Id: <8AF4702A-42D6-416A-8517-669AA552FD86@tzi.org> To: cbor@ietf.org X-Mailer: Apple Mail (2.3445.9.1) Archived-At: Subject: [Cbor] CBOR interop tests? X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Aug 2018 19:21:58 -0000 This may sound strange five years after the specification was published, = with > 50 documented implementations available, but: During the interim today, when we discussed the CBORbis work, the = question came up whether some structured interop testing might be useful = at this time to really get to the ground of some of the finer points we = have been discussing recently (NaN payloads, anyone?). It might also = help in coming up with a more comprehensive set of test cases for = implementers to check their implementations, but the focus will be on = interop between different implementations. To do such an interop, we=E2=80=99d need to write some test descriptions = (TD, (*)) so people know what is going to be tested. We thought it might be possible to have a first set of test = descriptions, plus implementations of stubs or other kinds of support on = the respective implementation side, at some time in October. We could = do some online testing (say, in a Jitsi-style web conference event) = then, and pick up the work with an updated set of test descriptions in = Bangkok during the hackathon. I=E2=80=99m going on a two-week vacation now, but maybe we can start = some of the collection of interesting corners to look at, before I = return. First question: Who would be interested in making these interops happen? = I=E2=80=99m pretty sure that much of the work on the TDs could be done = by the editors of CBORbis, but we need the experience of implementers = (and the stubs that talk to the implementations) to make this a = meaningful exercise. Second question: What were the corners of CBOR encoding/decoding itself, = of CBOR c18n/deterministic encoding, of JSON conversion/interop, etc., = where you felt like you had to make a decision that wasn=E2=80=99t = really supported either way by the existing spec? Gr=C3=BC=C3=9Fe, Carsten (*) for CoAP, we used https://github.com/cabo/td-coap4; maybe we can use = some of that structure. But then encoding and decoding (and possibly = round-tripping) are much simpler than what we needed for CoAP. From nobody Wed Aug 22 12:45:48 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 988E712F1A2 for ; Wed, 22 Aug 2018 12:45:46 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.2 X-Spam-Level: X-Spam-Status: No, score=-4.2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KeXSRpXuoIpt for ; Wed, 22 Aug 2018 12:45:45 -0700 (PDT) Received: from mailhost.informatik.uni-bremen.de (mailhost.informatik.uni-bremen.de [IPv6:2001:638:708:30c9::12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B37931292AD for ; Wed, 22 Aug 2018 12:45:44 -0700 (PDT) X-Virus-Scanned: amavisd-new at informatik.uni-bremen.de Received: from submithost.informatik.uni-bremen.de (submithost.informatik.uni-bremen.de [134.102.201.11]) by mailhost.informatik.uni-bremen.de (8.14.5/8.14.5) with ESMTP id w7MJjedi016106 for ; Wed, 22 Aug 2018 21:45:40 +0200 (CEST) Received: from [192.168.217.114] (p54A6C24D.dip0.t-ipconnect.de [84.166.194.77]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by submithost.informatik.uni-bremen.de (Postfix) with ESMTPSA id 41wdLr2v52zDWrf; Wed, 22 Aug 2018 21:45:40 +0200 (CEST) From: Carsten Bormann Content-Type: text/plain; charset=utf-8 X-Mao-Original-Outgoing-Id: 556659937.798617-60513bbeb40bddff72ffaa9df18d4f0f Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Date: Wed, 22 Aug 2018 21:45:39 +0200 Message-Id: <1E00D1D4-5054-4EE6-864C-BD4D17FA25EF@tzi.org> To: cbor@ietf.org X-Mailer: Apple Mail (2.3445.9.1) Archived-At: Subject: [Cbor] Five years soon... X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Aug 2018 19:45:47 -0000 (I=E2=80=99m already almost in vacation mode; SCNR.) For those who want to prepare a proper fifth birthday party, the = announcement that CBOR was approved by IESG went out on September 13, = 2013 at 2210Z (so it was September 14 already where I live). It was = then published as RFC 7049 on October 24, but that is more of an = inauguration event than a birthday. Gr=C3=BC=C3=9Fe, Carsten PS.: There were two *Friday* *13*s in 20*13*. We managed to hit the = first one :-) From nobody Wed Aug 22 12:50:23 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D8A7F12D949 for ; Wed, 22 Aug 2018 12:50:21 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.901 X-Spam-Level: X-Spam-Status: No, score=-1.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MGPdOBzNguDD for ; Wed, 22 Aug 2018 12:50:19 -0700 (PDT) Received: from mail2.augustcellars.com (augustcellars.com [50.45.239.150]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 176551292AD for ; Wed, 22 Aug 2018 12:50:19 -0700 (PDT) Received: from Jude (73.180.8.170) by mail2.augustcellars.com (192.168.0.56) with Microsoft SMTP Server (TLS) id 15.0.1347.2; Wed, 22 Aug 2018 12:46:26 -0700 From: Jim Schaad To: 'Carsten Bormann' , References: <8AF4702A-42D6-416A-8517-669AA552FD86@tzi.org> In-Reply-To: <8AF4702A-42D6-416A-8517-669AA552FD86@tzi.org> Date: Wed, 22 Aug 2018 12:50:11 -0700 Message-ID: <006d01d43a51$5798a310$06c9e930$@augustcellars.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 16.0 Thread-Index: AQKuiSrJnPcPSw4veAug5bTrryfSraMXj6cQ Content-Language: en-us X-Originating-IP: [73.180.8.170] Archived-At: Subject: Re: [Cbor] CBOR interop tests? X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Aug 2018 19:50:22 -0000 >From my point of view, this should also cover one more issue. Part of = the process of moving the standard some form of interop report is going = to be needed. Thus I would like to see all of the items which we think = need to be covered in that report covered as well. I don't know what = the chairs and ADs are currently looking for in such an interop report. Jim > -----Original Message----- > From: CBOR On Behalf Of Carsten Bormann > Sent: Wednesday, August 22, 2018 12:22 PM > To: cbor@ietf.org > Subject: [Cbor] CBOR interop tests? >=20 > This may sound strange five years after the specification was = published, with > > 50 documented implementations available, but: >=20 > During the interim today, when we discussed the CBORbis work, the = question > came up whether some structured interop testing might be useful at = this time > to really get to the ground of some of the finer points we have been = discussing > recently (NaN payloads, anyone?). It might also help in coming up = with a more > comprehensive set of test cases for implementers to check their > implementations, but the focus will be on interop between different > implementations. >=20 > To do such an interop, we=E2=80=99d need to write some test = descriptions (TD, (*)) so > people know what is going to be tested. >=20 > We thought it might be possible to have a first set of test = descriptions, plus > implementations of stubs or other kinds of support on the respective > implementation side, at some time in October. We could do some online > testing (say, in a Jitsi-style web conference event) then, and pick up = the work > with an updated set of test descriptions in Bangkok during the = hackathon. >=20 > I=E2=80=99m going on a two-week vacation now, but maybe we can start = some of the > collection of interesting corners to look at, before I return. >=20 > First question: Who would be interested in making these interops = happen? I=E2=80=99m > pretty sure that much of the work on the TDs could be done by the = editors of > CBORbis, but we need the experience of implementers (and the stubs = that talk > to the implementations) to make this a meaningful exercise. >=20 > Second question: What were the corners of CBOR encoding/decoding = itself, of > CBOR c18n/deterministic encoding, of JSON conversion/interop, etc., = where > you felt like you had to make a decision that wasn=E2=80=99t really = supported either way > by the existing spec? >=20 > Gr=C3=BC=C3=9Fe, Carsten >=20 >=20 > (*) for CoAP, we used https://github.com/cabo/td-coap4; maybe we can = use > some of that structure. But then encoding and decoding (and possibly = round- > tripping) are much simpler than what we needed for CoAP. >=20 > _______________________________________________ > CBOR mailing list > CBOR@ietf.org > https://www.ietf.org/mailman/listinfo/cbor From nobody Thu Aug 23 00:46:16 2018 Return-Path: X-Original-To: cbor@ietf.org Delivered-To: cbor@ietfa.amsl.com Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 64F271292F1; Thu, 23 Aug 2018 00:46:14 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: internet-drafts@ietf.org To: Cc: cbor@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 6.83.1 Auto-Submitted: auto-generated Precedence: bulk Reply-To: cbor@ietf.org Message-ID: <153501037436.12184.3015150361178685489@ietfa.amsl.com> Date: Thu, 23 Aug 2018 00:46:14 -0700 Archived-At: Subject: [Cbor] I-D Action: draft-ietf-cbor-cddl-05.txt X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Aug 2018 07:46:14 -0000 A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Concise Binary Object Representation Maintenance and Extensions WG of the IETF. Title : Concise data definition language (CDDL): a notational convention to express CBOR and JSON data structures Authors : Henk Birkholz Christoph Vigano Carsten Bormann Filename : draft-ietf-cbor-cddl-05.txt Pages : 59 Date : 2018-08-23 Abstract: This document proposes a notational convention to express CBOR data structures (RFC 7049). Its main goal is to provide an easy and unambiguous way to express structures for protocol messages and data formats that use CBOR or JSON. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-cbor-cddl/ There are also htmlized versions available at: https://tools.ietf.org/html/draft-ietf-cbor-cddl-05 https://datatracker.ietf.org/doc/html/draft-ietf-cbor-cddl-05 A diff from the previous version is available at: https://www.ietf.org/rfcdiff?url2=draft-ietf-cbor-cddl-05 Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From nobody Thu Aug 23 06:10:52 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8A8AF130EBE for ; Thu, 23 Aug 2018 06:10:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.898 X-Spam-Level: X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.001, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tCBHIVpSnSno for ; Thu, 23 Aug 2018 06:10:48 -0700 (PDT) Received: from mail-io0-x231.google.com (mail-io0-x231.google.com [IPv6:2607:f8b0:4001:c06::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A8174130EB8 for ; Thu, 23 Aug 2018 06:10:48 -0700 (PDT) Received: by mail-io0-x231.google.com with SMTP id q4-v6so4216715iob.8 for ; Thu, 23 Aug 2018 06:10:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:from:date:message-id:subject:to :content-transfer-encoding; bh=3yZrCFful2wvPrs4p+6KIW+B70dI5g7fGIerA+Rqd54=; b=Uczi0RpZTnSP7MGQq5cTs0VfVOscRZ5DVJnUgwBnEK9L1cX6x54FYWqjSiCfaZqi4C EYWpAx9mOvjRa3x0GEwJeL0rOlhpxMN99LgEkKiqLUnYCLGRAaBvDWcXh20nloAZw9NT S8NBoqeyNvHgVGc8mxL66fxPkIB7PncRspyj2R2yN9rscEJmtOnjBBlm5bArO0i+AgXO mI3josiow/fSarLoMpQsjLn3xckbuL4S7S77KaFyr192fh6SXpU4v8L2rjgHl5BjamaW IiKAH9Z3d/JUGXOS1MeqYWvQmGduP5ADxRw8N75HOP5GIMTFz3jGcDXmLVbBMrKeX4sj spEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:from:date:message-id:subject :to:content-transfer-encoding; bh=3yZrCFful2wvPrs4p+6KIW+B70dI5g7fGIerA+Rqd54=; b=tF2z0U2O04G3LYtOUigDzile5kaG+tPsoK2It5f8hVNT+gD713oeFoAdv81uyllHQi t5h8A0MiTofSKVXgj15PCUOl+nCX2VNFLPouVAchf0RpVhX/xq897eRVtsBYmRBZscfT dBoXTKZWKx2GkZK1uqt6qdQ+pTPtoGXxp8jZpN+D0t7Lzs1uWX1n4dEH30SdA/4tbqfr EWy5oLaTO6bL4o/WhPVwvRTbKZjYXPKnRbzaj5GcdDjyKS3G55t3HyXXbnuL0R89wRiC /agp8btaQ54qHUaDmW8IuqcQKhque1tjY7qiPajeb0JQOrEgrRb+5tmKDLpNRULK6Hls vSNg== X-Gm-Message-State: APzg51CCz5sM2UB0ZDF18f+MvLpbo9d4e+nX20n9yYt5IaCn2Zo73Mzm 61D79OHGxg6yleUWOEnrKLjCtMsVmHqdKzRl7vGnoXcY X-Google-Smtp-Source: ANB0Vdb7VMqnKvF87V908FeEcQs0phgSYaDnLeUhH5wlE6NAZR9OC4kaKR7eg9PfwxDecCbhbUP91l1GX3CteJzBobY= X-Received: by 2002:a6b:b445:: with SMTP id d66-v6mr18215006iof.163.1535029847548; Thu, 23 Aug 2018 06:10:47 -0700 (PDT) MIME-Version: 1.0 Sender: barryleiba@gmail.com Received: by 2002:ac0:8e88:0:0:0:0:0 with HTTP; Thu, 23 Aug 2018 06:10:47 -0700 (PDT) From: Barry Leiba Date: Thu, 23 Aug 2018 09:10:47 -0400 X-Google-Sender-Auth: LGTOY16MUJDQkBbuC3OtCxnvM3c Message-ID: To: cbor@ietf.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Archived-At: Subject: [Cbor] Notes from CBOR teleconference, 22 Aug X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Aug 2018 13:10:50 -0000 Here are my brief notes from yesterday's CBOR teleconference. Attendees: Barry Leiba Carsten Bormann Jim Schaad Lawrence Lundblade Michael Breuer Carsten will leave for vacation tomorrow morning (23 Aug), back on 7 Sept. CDDL status: CDDL issues mostly resolved in Carsten=E2=80=99s working version Thinks he=E2=80=99s addressed Jeffrey=E2=80=99s concerns, but would like to= hear from Jeffrey, who appears to be on vacation. Addressed most of what Jim raised, but Jim sent another message 30 minutes before the call; will discuss later. CBOR-bis: need to go through the issues in detail after Carsten=E2=80=99s vacation base64 padding will need much discussion - some want to give up on it, but that causes implementors problems - better for us to nail that down: revisit an old in-room decision some other issues that are less technical will need discussion brief look at issues and pull requests on the call; most not a big deal - canonicalization, extract into separate document - does it make sense to do an interop test for CBOR? (Jim: would be helpful to find corner cases) - Carsten will start a thread on the mailing list collecting interop cases for an online test in October; will also do a Hackathon project in Bangkok. Back to CDDL, discussing Jim=E2=80=99s recent message to the list. - Issue stops Jim from building a syntax tree. - Discussion of options, looking at editorial changes. - Carsten will work with Jim today on text. - If the issue is still open after today, we can note it in the shepherd writeup and finish dealig with it in IETF last call. Call on 5 Sept will be cancelled unless we have something to talk about without Carsten. Next call after that is on 19 Sept; Carsten will be on it. --=20 Barry From nobody Thu Aug 23 06:28:32 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 05DE9130EF2 for ; Thu, 23 Aug 2018 06:28:20 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.898 X-Spam-Level: X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.001, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RB_zmPO-MMJE for ; Thu, 23 Aug 2018 06:28:18 -0700 (PDT) Received: from mail-qt0-x22c.google.com (mail-qt0-x22c.google.com [IPv6:2607:f8b0:400d:c0d::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3BCD5130EDA for ; Thu, 23 Aug 2018 06:28:18 -0700 (PDT) Received: by mail-qt0-x22c.google.com with SMTP id o15-v6so6193389qtk.6 for ; Thu, 23 Aug 2018 06:28:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=5p0kyxIQwoYZ2ID/iDjZq/sKsyKkIahgzgqqSIEBg30=; b=Nn3UnO2NL+haN7NtH3Y3fTXssUHugc1CRHkLxNzyoTDBegKtq7WE0uiKb7DcamDoW5 ifNZ1kElD6fZMH1njIlDriuRKwnLJKP8k9ZwZX+mrgVJwPkKR9yh9mDoBE/K+ZOZOnv1 KfiNrVh80jlgDZwF2qTq9rdYkXHulGEWY6lh6o4/vzhnuXpE5bhCkC+PsF++2DMc/vkB fQfW27lT5iYRkVXl9WqjCAV1S4zIBDxcZTcUyW+PgfWihXd5GcY7T2SftvdIR5atwDPF TwlWRptZlNmg61WtW10vZSslppcEd8rLtsNFqcTahN/AhztPYnKFEYlWvn45s941xiP1 Ytlw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=5p0kyxIQwoYZ2ID/iDjZq/sKsyKkIahgzgqqSIEBg30=; b=I0ChEDn5mYqFjt/bl0Wo+WC9YeCcAPuYjTyqIUuEUkufXD4pFJEx0tY0+6QjfOdCED v0eoklUXJjMHM3S49l5wnmhbaH+w2sX79ljyxLWR+0R8jb/Vj1MnOxuUI1meW89tmX8Q BJfN0A3T3d66Z+PytSHcB5FMn0nWmFelkrlYIohrnh/eaIvpdvJOtdLWQua5s2oeNam+ YDGdp5dPx/TD/t6nYS/laLkxccO/JjNn/LsuZ29S2+Qvv2ddf2wY+gS1t2g7UqKKqx4Q zmrBdUn4JwE+A2+i/EqO9A6Yu9s3A3DPD0dW/VZJFSnTcH0r5ByIepR2UHVKWdYTAAts UEtg== X-Gm-Message-State: APzg51DPTNuo7XvP+XCy5vq3z+t5fByZ+iGGxD0FC8NgcNVwBsVj542A 5Q86tw3Ftoi158/adhScVLqij4aq+PLNk97BzlFvTR/M X-Google-Smtp-Source: ANB0VdYHYBp1BmbBbnc6eDyb4rYhGaW4M8gP9yk8XRHIhfLYS1Togo8e/lIj+oopT+NGG2mk9tMJpEvrY0iLneZE/04= X-Received: by 2002:ac8:2fa4:: with SMTP id l33-v6mr20288701qta.83.1535030896823; Thu, 23 Aug 2018 06:28:16 -0700 (PDT) MIME-Version: 1.0 Sender: barryleiba.mailing.lists@gmail.com Received: by 2002:ac8:7484:0:0:0:0:0 with HTTP; Thu, 23 Aug 2018 06:28:16 -0700 (PDT) In-Reply-To: <006d01d43a51$5798a310$06c9e930$@augustcellars.com> References: <8AF4702A-42D6-416A-8517-669AA552FD86@tzi.org> <006d01d43a51$5798a310$06c9e930$@augustcellars.com> From: Barry Leiba Date: Thu, 23 Aug 2018 09:28:16 -0400 X-Google-Sender-Auth: ptYhB9nScyaSZ7NMAuxVpvtz6RM Message-ID: To: Jim Schaad Cc: Carsten Bormann , cbor@ietf.org Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cbor] CBOR interop tests? X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Aug 2018 13:28:32 -0000 > From my point of view, this should also cover one more issue. Part of the process of > moving the standard some form of interop report is going to be needed. Not really. We do have to convince the IESG that what we have is interoperable, but RFC 6410 eliminated the need for an extensive, formal report (see section 3.2, which says, "The IETF Standards Process no longer requires a formal interoperability report, recognizing that deployment and use is sufficient to show interoperability."). > I don't know what the chairs and ADs are currently looking for in such an interop report. What we're looking for now is a reasonable demonstration that there are a number of interoperable implementations, for which a brief summary of an interop test would serve nicely, and perhaps a statement that we are not aware of interoperability issues among compliant implementations, which will likely fall out of the former anyway. Barry From nobody Thu Aug 23 06:33:41 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0FAAF130EC6 for ; Thu, 23 Aug 2018 06:33:38 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.897 X-Spam-Level: X-Spam-Status: No, score=-1.897 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.001, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id S7bpaZ3r4P14 for ; Thu, 23 Aug 2018 06:33:36 -0700 (PDT) Received: from mail-qk0-x243.google.com (mail-qk0-x243.google.com [IPv6:2607:f8b0:400d:c09::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1787D130EBE for ; Thu, 23 Aug 2018 06:33:36 -0700 (PDT) Received: by mail-qk0-x243.google.com with SMTP id d131-v6so3577877qke.11 for ; Thu, 23 Aug 2018 06:33:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to; bh=Aa7B+qVE7LJ279o2/NfNOLTSdL6rUWzhIzqWSf6TDYk=; b=T9TgLND6sDjusKVCUUla0mhtUafczAlwjeAXdyZKk++goCyjeYM6e5gTGJWJr99qU0 6QbxK3FU0563U9dypGjZsNzzT8mn0VXLayf10ICo1tYKer3F2iepNqNfFCNIJf1K+9vm EYvIqZXxRZhvRqGoEIDy1xOb0NMF/7zRgU5iZSuOMCB3d0EwRcN7Yx5WDg+jVVNJQUgJ 5bqgOFuUI2uI4VALk7t70NuGKqhEpuDx/qd1k4jtqqZrwzeqa+/2z+alWUPYQkDyo4f1 6M3Q74LqjV5rnjblhcoSo6ve3paoO5eRealfG09X37srW9d/cnBkHHQrkjGSlmv9ilx4 Fwjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to; bh=Aa7B+qVE7LJ279o2/NfNOLTSdL6rUWzhIzqWSf6TDYk=; b=dzaF+UuLI/+us3fIZevaryGLfr8SZQO2F74i/dYlSSFNRCcKv3H/gaBp+MpZqcYohe oI4PkhVk3uDpzbqjtXgGdz77qKRvu4As735IWEXdkncu2W1GmNWBgEgF7UkU9C8NoObh tyVqt7XG3va61m1cve4Sv1+ZcRwqp2MmrDGJfndp4FJgWTpfqLnfoS/4T/pZB+vMj85n 8isxWU+rxQPGno+SsvcW2mN0D/Wx+WcmeKIK6dW0HEmQVPg/EuztqhPVRdJ+FIfgC/kN 43Y6RR51G6UD9mZkAA5W61KR5ZiSMlicyP0i/4mYFr/hwMhx+pP7uSI9ucps5BLsDYe6 5cJg== X-Gm-Message-State: AOUpUlHhlDRTkU/Qh+H3f8nKTkrpYQLKCGS3e8P19uLUXzOXbR9W9FO9 6u65Nl+zzuxYOAMdrrt24DX7flXG3zGzVlSaNoBJsA== X-Google-Smtp-Source: AA+uWPwS/9Blr3cLL9MlEOgK9KanwpmmBn7/juPRRwv8MNdVIK73/e72iXHQsxj8feUxeMn0PLk+pjjuGrRKhW4dE2A= X-Received: by 2002:a37:4843:: with SMTP id v64-v6mr56894516qka.209.1535031214894; Thu, 23 Aug 2018 06:33:34 -0700 (PDT) MIME-Version: 1.0 Sender: barryleiba.mailing.lists@gmail.com Received: by 2002:ac8:7484:0:0:0:0:0 with HTTP; Thu, 23 Aug 2018 06:33:34 -0700 (PDT) In-Reply-To: <153501037436.12184.3015150361178685489@ietfa.amsl.com> References: <153501037436.12184.3015150361178685489@ietfa.amsl.com> From: Barry Leiba Date: Thu, 23 Aug 2018 09:33:34 -0400 X-Google-Sender-Auth: QhMYinZd_09xonBWBtytA2FvUX8 Message-ID: To: cbor@ietf.org Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cbor] I-D Action: draft-ietf-cbor-cddl-05.txt X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Aug 2018 13:33:38 -0000 We believe this version addresses the issues raised in the working group and that it's ready for us to request publication. A publication request will get us a review from the responsible AD (Alexey), with perhaps another round of updates to address that, followed by a two-week IETF-wide last call. Any remaining issues can be dealt with during that period (including any further concerns from Jeffrey when he returns from vacation). I am, therefore, going to do the document shepherd writeup and then request publication, which I should do by the end of today, Thursday. If anyone sees an immediate reason to halt, please shout out now. Barry On Thu, Aug 23, 2018 at 3:46 AM, wrote: > > A New Internet-Draft is available from the on-line Internet-Drafts directories. > This draft is a work item of the Concise Binary Object Representation Maintenance and Extensions WG of the IETF. > > Title : Concise data definition language (CDDL): a notational convention to express CBOR and JSON data structures > Authors : Henk Birkholz > Christoph Vigano > Carsten Bormann > Filename : draft-ietf-cbor-cddl-05.txt > Pages : 59 > Date : 2018-08-23 > > Abstract: > This document proposes a notational convention to express CBOR data > structures (RFC 7049). Its main goal is to provide an easy and > unambiguous way to express structures for protocol messages and data > formats that use CBOR or JSON. > > > The IETF datatracker status page for this draft is: > https://datatracker.ietf.org/doc/draft-ietf-cbor-cddl/ > > There are also htmlized versions available at: > https://tools.ietf.org/html/draft-ietf-cbor-cddl-05 > https://datatracker.ietf.org/doc/html/draft-ietf-cbor-cddl-05 > > A diff from the previous version is available at: > https://www.ietf.org/rfcdiff?url2=draft-ietf-cbor-cddl-05 > > > Please note that it may take a couple of minutes from the time of submission > until the htmlized version and diff are available at tools.ietf.org. > > Internet-Drafts are also available by anonymous FTP at: > ftp://ftp.ietf.org/internet-drafts/ > > _______________________________________________ > CBOR mailing list > CBOR@ietf.org > https://www.ietf.org/mailman/listinfo/cbor From nobody Thu Aug 23 09:52:40 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8AEE3130E35 for ; Thu, 23 Aug 2018 09:52:37 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 45MrtQh5uGBL for ; Thu, 23 Aug 2018 09:52:36 -0700 (PDT) Received: from mail2.augustcellars.com (augustcellars.com [50.45.239.150]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CBBEA130DE4 for ; Thu, 23 Aug 2018 09:52:35 -0700 (PDT) Received: from Jude (73.180.8.170) by mail2.augustcellars.com (192.168.0.56) with Microsoft SMTP Server (TLS) id 15.0.1347.2; Thu, 23 Aug 2018 09:48:42 -0700 From: Jim Schaad To: 'Barry Leiba' CC: 'Carsten Bormann' , References: <8AF4702A-42D6-416A-8517-669AA552FD86@tzi.org> <006d01d43a51$5798a310$06c9e930$@augustcellars.com> In-Reply-To: Date: Thu, 23 Aug 2018 09:52:27 -0700 Message-ID: <00cc01d43b01$ae3cc010$0ab64030$@augustcellars.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 16.0 Thread-Index: AQKuiSrJnPcPSw4veAug5bTrryfSrQH8HFMeAa7Abbei+5l/cA== Content-Language: en-us X-Originating-IP: [73.180.8.170] Archived-At: Subject: Re: [Cbor] CBOR interop tests? X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Aug 2018 16:52:38 -0000 > -----Original Message----- > From: barryleiba.mailing.lists@gmail.com > On Behalf Of Barry Leiba > Sent: Thursday, August 23, 2018 6:28 AM > To: Jim Schaad > Cc: Carsten Bormann ; cbor@ietf.org > Subject: Re: [Cbor] CBOR interop tests? >=20 > > From my point of view, this should also cover one more issue. Part = of > > the process of moving the standard some form of interop report is = going to > be needed. >=20 > Not really. We do have to convince the IESG that what we have is > interoperable, but RFC 6410 eliminated the need for an extensive, = formal > report (see section 3.2, which says, "The IETF Standards Process no = longer > requires a formal interoperability report, recognizing that deployment = and use > is sufficient to show interoperability."). >=20 > > I don't know what the chairs and ADs are currently looking for in = such an > interop report. >=20 > What we're looking for now is a reasonable demonstration that there = are a > number of interoperable implementations, for which a brief summary of = an > interop test would serve nicely, and perhaps a statement that we are = not aware > of interoperability issues among compliant implementations, which will = likely > fall out of the former anyway. Does that mean that you would think that the interop which is currently = on the Wiki is sufficient or would we need to look at covering more of = the document that just the basic I can encode/decode this Major.Minor = type for some basic types? As an example, should the issues that have = come up with base64 encoding have been on that matrix initially? Jim >=20 > Barry From nobody Fri Aug 24 05:46:23 2018 Return-Path: X-Original-To: cbor@ietf.org Delivered-To: cbor@ietfa.amsl.com Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 3D082130E98; Fri, 24 Aug 2018 05:46:22 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: Barry Leiba To: X-Test-IDTracker: no X-IETF-IDTracker: 6.83.1 Auto-Submitted: auto-generated Precedence: bulk Cc: cbor@ietf.org, Barry Leiba , barryleiba@computer.org, iesg-secretary@ietf.org, cbor-chairs@ietf.org Message-ID: <153511478224.23035.2990020783447466729.idtracker@ietfa.amsl.com> Date: Fri, 24 Aug 2018 05:46:22 -0700 Archived-At: Subject: [Cbor] Publication has been requested for draft-ietf-cbor-cddl-05 X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Aug 2018 12:46:22 -0000 Barry Leiba has requested publication of draft-ietf-cbor-cddl-05 as Proposed Standard on behalf of the CBOR working group. Please verify the document's state at https://datatracker.ietf.org/doc/draft-ietf-cbor-cddl/ From nobody Sat Aug 25 13:38:35 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 392B8130E2C for ; Sat, 25 Aug 2018 13:38:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.898 X-Spam-Level: X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fv3pyJxgQRPl for ; Sat, 25 Aug 2018 13:38:32 -0700 (PDT) Received: from p3plsmtpa12-06.prod.phx3.secureserver.net (p3plsmtpa12-06.prod.phx3.secureserver.net [68.178.252.235]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8C8C9130DD5 for ; Sat, 25 Aug 2018 13:38:32 -0700 (PDT) Received: from [192.168.1.82] ([76.192.164.238]) by :SMTPAUTH: with ESMTPSA id tfKJfXIQJ7eECtfKJfWF21; Sat, 25 Aug 2018 13:38:31 -0700 From: Laurence Lundblade Content-Type: multipart/alternative; boundary="Apple-Mail=_AEAC0E7A-70C5-4A73-9D47-D45178D5190E" Mime-Version: 1.0 (Mac OS X Mail 11.4 \(3445.8.2\)) Message-Id: <2B5BDE55-EE9E-4CAD-A2BF-8435C46103AE@island-resort.com> Date: Sat, 25 Aug 2018 13:38:31 -0700 To: cbor@ietf.org X-Mailer: Apple Mail (2.3445.8.2) X-CMAE-Envelope: MS4wfDqklW4XBlyR8IyKJXqDB8I/onCZCPRvHVnqz+SU5kzCgsytnwXcKVC6Qoj6vWrdlJHUqW/CMPT253LsJVAu+pF0/5CtdtkdJe2EVQQ2pD1RBlROx36H iHQtMaO6jSsQZS583IEBAMlxHurL9XF65GXXkkcOt8XL7hX7MoZ+05Qb Archived-At: Subject: [Cbor] New pull request proposal for shortest float encoding X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Aug 2018 20:38:34 -0000 --Apple-Mail=_AEAC0E7A-70C5-4A73-9D47-D45178D5190E Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii https://github.com/cbor-wg/CBORbis/pull/32 This is a proposal to reduce CBOR message size without losing floating = point precision by encoding the larger floating point values as smaller = ones when it can be done without dropping any significant bits. I personally don't think this should be part of canonicalization, or = part of preferred encoding. It's just a very useful way of encoding = floating points that seems worth including in the RFC. LL --Apple-Mail=_AEAC0E7A-70C5-4A73-9D47-D45178D5190E Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii https://github.com/cbor-wg/CBORbis/pull/32

This is a proposal to reduce CBOR message size without losing = floating point precision by encoding the larger floating point values as = smaller ones when it can be done without dropping any significant = bits.

I = personally don't think this should be part of canonicalization, or part = of preferred encoding. It's just a very useful way of encoding floating = points that seems worth including in the RFC.

LL

= --Apple-Mail=_AEAC0E7A-70C5-4A73-9D47-D45178D5190E-- From nobody Sun Aug 26 20:25:38 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 11B5E1277BB for ; Sun, 26 Aug 2018 20:25:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.749 X-Spam-Level: X-Spam-Status: No, score=-1.749 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cocCyTiPxQhA for ; Sun, 26 Aug 2018 20:25:35 -0700 (PDT) Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 97359126F72 for ; Sun, 26 Aug 2018 20:25:34 -0700 (PDT) Received: by mail-wr1-x42b.google.com with SMTP id w11-v6so12184537wrc.5 for ; Sun, 26 Aug 2018 20:25:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=qK439wPh2KQxWbQTflN6U0LjVCRYY6G1b88B3JhPZmk=; b=YLW0eUWHsuF4PwnkVZBhFB45SP0wSHuGjXSrXLhe5VHhh8P/R1RRKTVP+lV5Et4Il/ 6pL7bwsRGfBLB714RwFos0Q29vsw4iXyY6HOknEf1QrbhTIx2q5YYlvYiIn4igdYklzU IYCxHwf3ZhHh65nHwR8i6u0NI1xazYiyxfg2cJh4wxvP9WZzMiG4UpFzP3nUINjIQJax Ex6qYqNAhtTEHbj4THWIyKBpVVAUQ1qjPRA4uqLXdGFsLRcqqN8DzUrCmgkqGIaCSonn SHB2FKYUrBmKB9yV0uYxGDkOGqVQUmrznk191mSajIqa9XYPX6Pt2dRyh1eYHc+CNntb bDQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=qK439wPh2KQxWbQTflN6U0LjVCRYY6G1b88B3JhPZmk=; b=bDMTQuUP54DypkYtSvPo6WJfMt+DiNDg+WAtA+fMbnPNPQsMwqwY37NxF2wKGgowS9 LxuBYsWqsxlYzKm2d0bDvRtIU7RjXkFUVXV2j47VAJ9BAyjN+Ho/oAsTaRoNGi0ObucV brBGRbMux2t0QITY7eLjsKEBld77pp28Nqd7/KytBnltEuJmR0J5Z3FtxgNISsOB+Urm XlebXf7K9ItV/Rwt4GyseqYMBAPSNwJSejPfFr3FIJJ9N58rqyPgAgGigTdf8ZOrKKMK N2+ySbYLIl11kQVXEWjteaTns5Nk+Z4BsVBFBhlLEN3GLl2FoglPA6F9ebSCzfgAyrHr U5IA== X-Gm-Message-State: APzg51At/a2Cm0ZDmeardJdtL47UcCrmczwaxL3ijXS/jSpXtVelkjAp BzQrnpjxBji5df+BnEvFboQZZqK6M8rLYU1A6FVQfA== X-Google-Smtp-Source: ANB0VdbwtS8qc9wHij97FahtuVra385CU7XkSwk9sJiVAXtvF9WegsE88pCbd/Gpv8PTMgU8TKDedGUjaSIXdicVZx4= X-Received: by 2002:adf:ed88:: with SMTP id c8-v6mr7001144wro.264.1535340333057; Sun, 26 Aug 2018 20:25:33 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:adf:9bcb:0:0:0:0:0 with HTTP; Sun, 26 Aug 2018 20:25:32 -0700 (PDT) In-Reply-To: <2B5BDE55-EE9E-4CAD-A2BF-8435C46103AE@island-resort.com> References: <2B5BDE55-EE9E-4CAD-A2BF-8435C46103AE@island-resort.com> From: Sean Solberg Date: Sun, 26 Aug 2018 22:25:32 -0500 Message-ID: To: Laurence Lundblade Cc: cbor@ietf.org Content-Type: multipart/alternative; boundary="000000000000294c1d0574624706" Archived-At: Subject: Re: [Cbor] New pull request proposal for shortest float encoding X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2018 03:25:36 -0000 --000000000000294c1d0574624706 Content-Type: text/plain; charset="UTF-8" Question on this: Even though the actual value transmitted has not lost any precision for that particular value, how does the receiver then know that the intended data type actually does have higher precision than what was transmitted? Are there some flags somewhere that notify the receiver that it's actually a double instead of a single or its actually a single instead of a half? ..Sean On Sat, Aug 25, 2018 at 3:38 PM, Laurence Lundblade wrote: > https://github.com/cbor-wg/CBORbis/pull/32 > > This is a proposal to reduce CBOR message size without losing floating > point precision by encoding the larger floating point values as smaller > ones when it can be done without dropping any significant bits. > I personally don't think this should be part of canonicalization, or part > of preferred encoding. It's just a very useful way of encoding floating > points that seems worth including in the RFC. > > LL > > > _______________________________________________ > CBOR mailing list > CBOR@ietf.org > https://www.ietf.org/mailman/listinfo/cbor > > --000000000000294c1d0574624706 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Question on this:=C2=A0 Even though the actual value trans= mitted has not lost any precision for that particular value, how does the r= eceiver then know that the intended data type actually does have higher pre= cision than what was transmitted?=C2=A0 Are there some flags somewhere that= notify the receiver that it's actually a double instead of a single or= its actually a single instead of a half?

..Sean

On Sat, Aug 2= 5, 2018 at 3:38 PM, Laurence Lundblade <lgl@island-resort.com><= /span> wrote:
https://github.com/cbor-wg/CBORbis= /pull/32

This is a proposal to reduce CBOR m= essage size without losing floating point precision by encoding the larger = floating point values as smaller ones when it can be done without dropping = any significant bits.

I personally don't think this should = be part of canonicalization, or part of preferred encoding. It's just a= very useful way of encoding floating points that seems worth including in = the RFC.

LL


____= ___________________________________________
CBOR mailing list
CBOR@ietf.org
https://www.ietf.org/mailman/listinfo/cbor


--000000000000294c1d0574624706-- From nobody Sun Aug 26 20:35:02 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D4EE31274D0 for ; Sun, 26 Aug 2018 20:35:00 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.749 X-Spam-Level: X-Spam-Status: No, score=-1.749 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id l7IrCazhx5kp for ; Sun, 26 Aug 2018 20:34:58 -0700 (PDT) Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 34B21126F72 for ; Sun, 26 Aug 2018 20:34:58 -0700 (PDT) Received: by mail-wr1-x42e.google.com with SMTP id a108-v6so12171660wrc.13 for ; Sun, 26 Aug 2018 20:34:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=1KJn7umIboERQef88gimJQY6mL3o9sRksGGbwFQPBzc=; b=Kt8Xw9+xlzs/za+UKLTPQHuX3eBBEM1Tm6Yuo221ZCxh4Z9e0QLuOGW2iQIz3bRrK7 CIuuKveEaj9rNoR/KPM0KfqGc4tEdH5wPeJ+NlbaXT+VqBBqJn/GNzqiBatv9rXY+gZd q6VlW5dmdVMf4fsfp8rfjDs57Nl41I61WuVA6g3VHMKFUBl2n8mMtE1kLQLKZVBXLBKe 24B9h7Dl2DaeadMVv9hkVu2U/kRe8VP8aPO1yY7C9OIVDzEe8nHtO4N9Mxf5niFiKQJk gSs4y924WbPxzxrnCZmvAk5v7xEC2C6phqN9Y8BAPoeuYFZ+g2oeil5aLzEFnbG/ioPO EfvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=1KJn7umIboERQef88gimJQY6mL3o9sRksGGbwFQPBzc=; b=gre0Tur+dUoLdRdpx5ww6Jx+UjkDKcIQX/Btdn9vzwnqRH7kvhH48DDHV0GmZUwr4L N1tUORqJBvFrIVqh2ZujP9mmTefo8aVFhW/N7/PzUpO4Fa0NASroWJML7i2QH6Er4vz4 cZm8uYhLzpwbi7rUpRnL/Ffjh3e7Jrd/UwWf/4IZn++8wunw0f1+V5BPrJkpatjM+bX9 bIFiMl6otxBqOY5nHNvSBbRIxg7xKH5vyyi7iPuIJHY3mVkpIoI4ks+zoi5nkU9UZ1hK UZ5o7o4lfE+l7fTQrfzqLldKXXjRhSBEUPtayR9X9s5EVMydLR4idbydZ8woKcJ/+9dR 2K3w== X-Gm-Message-State: APzg51C9SyN8qSL4Pt1CzkKz8uHC9vT1RZg2xWyI9TDVcwiTqpPS7HPF WSPYk6ogVivzkZj1LUbOEZ4JiY7DQfR7Eg0oEAU= X-Google-Smtp-Source: ANB0VdZhtAvFlSso0zk8L1irYhJq5JDWkiBLbGEM2/zxvgJ/lCw9BvBMNEBieDSqvktWcIZjOSGtrQ+QXEIDsaVFLlQ= X-Received: by 2002:adf:ed88:: with SMTP id c8-v6mr7014446wro.264.1535340896714; Sun, 26 Aug 2018 20:34:56 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:adf:9bcb:0:0:0:0:0 with HTTP; Sun, 26 Aug 2018 20:34:56 -0700 (PDT) In-Reply-To: References: <2B5BDE55-EE9E-4CAD-A2BF-8435C46103AE@island-resort.com> From: Sean Solberg Date: Sun, 26 Aug 2018 22:34:56 -0500 Message-ID: To: Laurence Lundblade Cc: cbor@ietf.org Content-Type: multipart/alternative; boundary="000000000000c205bc0574626871" Archived-At: Subject: Re: [Cbor] New pull request proposal for shortest float encoding X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2018 03:35:01 -0000 --000000000000c205bc0574626871 Content-Type: text/plain; charset="UTF-8" Follow up question: the proposal states: Generally the decoder should promote half- and single-precision values to double-precision when they are received as it is implied that all half- and single-precision values transmitted are actually double precision. Why is it implied that a transmitted single precision value is actually a double precision value? That makes no sense. A single precision value is implied to be a single precision value in my mind. If there was a "please treat as a double" flag sent along with a single precision value, then the transmitted data explicitly defines that the transmitted single precision value should be treated as a double precision value by the receiver. without the flag, the single precision value transmitted should be treated by the receiver as a single precision value. Explicit behavior is great. Implicitly expecting a receiver to treat all floats (half, single & double) as doubles when received is implicitly a wrong choice. In general, any time things are done "implicitly", we end up with compatibility problems because people make different implicit choices/assumptions. I believe keeping things explicit is a much better way to go. ..Sean On Sun, Aug 26, 2018 at 10:25 PM, Sean Solberg wrote: > Question on this: Even though the actual value transmitted has not lost > any precision for that particular value, how does the receiver then know > that the intended data type actually does have higher precision than what > was transmitted? Are there some flags somewhere that notify the receiver > that it's actually a double instead of a single or its actually a single > instead of a half? > > ..Sean > > On Sat, Aug 25, 2018 at 3:38 PM, Laurence Lundblade > wrote: > >> https://github.com/cbor-wg/CBORbis/pull/32 >> >> This is a proposal to reduce CBOR message size without losing floating >> point precision by encoding the larger floating point values as smaller >> ones when it can be done without dropping any significant bits. >> I personally don't think this should be part of canonicalization, or part >> of preferred encoding. It's just a very useful way of encoding floating >> points that seems worth including in the RFC. >> >> LL >> >> >> _______________________________________________ >> CBOR mailing list >> CBOR@ietf.org >> https://www.ietf.org/mailman/listinfo/cbor >> >> > --000000000000c205bc0574626871 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Follow up question:=C2=A0 the proposal states:=C2=A0=C2=A0 Generally = the decoder should promote half- and single-precision values to double-= precision when they are received as it is implied that all half- and single-precision values transmitted are actually double pre= cision. Why is it implied that a = transmitted single precision value is actually a double precision value?=C2= =A0 =C2=A0That makes no sense.=C2=A0 A single precision value is implied to= be a single precision value in my mind.=C2=A0=C2=A0=C2=A0If there w= as a "please treat as a double" flag sent along with a single pre= cision value, then the transmitted data explicitly defines that the transmi= tted single precision value should be treated as a double precision value b= y the receiver.=C2=A0 without the flag, the single precision value transmit= ted should be treated by the receiver as a single precision value.=C2=A0 Ex= plicit behavior is great.=C2=A0 Implicitly expecting a receiver to treat al= l floats (half, single & double) as doubles when received is implicitly= a wrong choice.=C2=A0 =C2=A0=C2=A0

In gener= al, any time things are done "implicitly", we end up with compati= bility problems because people make different implicit choices/assumptions.= =C2=A0 I believe keeping things explicit is a much better way to go.=C2=A0= =C2=A0
..Sean

On Sun, Aug 26, 2018 at 10:25 PM, Sean Solberg <= ssolbs99@gmail.com> wrote:
=
Question on this:=C2=A0 Even though the actual value trans= mitted has not lost any precision for that particular value, how does the r= eceiver then know that the intended data type actually does have higher pre= cision than what was transmitted?=C2=A0 Are there some flags somewhere that= notify the receiver that it's actually a double instead of a single or= its actually a single instead of a half?

..Sean

On Sat, Aug 25, 2018 at 3:38 PM, Laurence Lundblade <lgl@isl= and-resort.com> wrote:
https://github.com/cbor-wg/CBORbis/pull/32
=

This is a proposal to reduce CBOR message size witho= ut losing floating point precision by encoding the larger floating point va= lues as smaller ones when it can be done without dropping any significant b= its.

I personally don't think this should be part of canoni= calization, or part of preferred encoding. It's just a very useful way = of encoding floating points that seems worth including in the RFC.

LL

_______________________________________________
CBOR mailing list
CBOR@ietf.org
https://www.ietf.org/mailman/listinfo/cbor



--000000000000c205bc0574626871-- From nobody Sun Aug 26 22:32:25 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 66074130E6D for ; Sun, 26 Aug 2018 22:32:23 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.201 X-Spam-Level: X-Spam-Status: No, score=-4.201 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lo92p3hnPsm6 for ; Sun, 26 Aug 2018 22:32:22 -0700 (PDT) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 43958130E6A for ; Sun, 26 Aug 2018 22:32:22 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Aug 2018 22:32:22 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,294,1531810800"; d="scan'208";a="68077926" Received: from orsmsx101.amr.corp.intel.com ([10.22.225.128]) by orsmga007.jf.intel.com with ESMTP; 26 Aug 2018 22:32:21 -0700 Received: from tjmaciei-mobl1.localnet (10.252.206.187) by ORSMSX101.amr.corp.intel.com (10.22.225.128) with Microsoft SMTP Server (TLS) id 14.3.319.2; Sun, 26 Aug 2018 22:32:21 -0700 From: Thiago Macieira To: Date: Sun, 26 Aug 2018 22:32:20 -0700 Message-ID: <1645839.CNUdieBceP@tjmaciei-mobl1> Organization: Intel Corporation In-Reply-To: References: <2B5BDE55-EE9E-4CAD-A2BF-8435C46103AE@island-resort.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Originating-IP: [10.252.206.187] Archived-At: Subject: Re: [Cbor] New pull request proposal for shortest float encoding X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2018 05:32:23 -0000 On Sunday, 26 August 2018 20:25:32 PDT Sean Solberg wrote: > Question on this: Even though the actual value transmitted has not lost any > precision for that particular value, how does the receiver then know that > the intended data type actually does have higher precision than what was > transmitted? That's not part of CBOR's purpose. That's why integrals are also sent in the smallest form depending on the value, regardless of the original data type. > Are there some flags somewhere that notify the receiver that > it's actually a double instead of a single or its actually a single instead > of a half? No. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center From nobody Sun Aug 26 22:46:52 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 66B6A130E70 for ; Sun, 26 Aug 2018 22:46:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.201 X-Spam-Level: X-Spam-Status: No, score=-4.201 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ryzmvc3IKxqs for ; Sun, 26 Aug 2018 22:46:49 -0700 (PDT) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1E1AB130E6F for ; Sun, 26 Aug 2018 22:46:49 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Aug 2018 22:46:48 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,294,1531810800"; d="scan'208";a="69353273" Received: from orsmsx101.amr.corp.intel.com ([10.22.225.128]) by orsmga006.jf.intel.com with ESMTP; 26 Aug 2018 22:44:45 -0700 Received: from tjmaciei-mobl1.localnet (10.252.206.187) by ORSMSX101.amr.corp.intel.com (10.22.225.128) with Microsoft SMTP Server (TLS) id 14.3.319.2; Sun, 26 Aug 2018 22:44:44 -0700 From: Thiago Macieira To: Date: Sun, 26 Aug 2018 22:44:44 -0700 Message-ID: <4729660.88A3kqaN6S@tjmaciei-mobl1> Organization: Intel Corporation In-Reply-To: <1645839.CNUdieBceP@tjmaciei-mobl1> References: <2B5BDE55-EE9E-4CAD-A2BF-8435C46103AE@island-resort.com> <1645839.CNUdieBceP@tjmaciei-mobl1> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Originating-IP: [10.252.206.187] Archived-At: Subject: Re: [Cbor] New pull request proposal for shortest float encoding X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2018 05:46:51 -0000 On Sunday, 26 August 2018 22:32:20 PDT Thiago Macieira wrote: > > Are there some flags somewhere that notify the receiver that > > it's actually a double instead of a single or its actually a single > > instead > > of a half? > > No. That said, we could have a set of tags that could be applied to integrals as well as floating point to denote the original type. For example: Original8Bit Original16Bit Original32Bit Original64Bit OriginalUnsigned The 16, 32 and 64-bit ones can be used with both floating point and integrals; the integral ones can be combined with the Unsigned one to indicate the type was originally unsigned. It may be even possible to combine them with the Decimal tags to indicate decimal32 an decimal64. Possibly add Original80Bit, Original128Bit too, allowing for long double, __float128, decimal128. Or expand out the combinations, thus gaining us Float64x different from Float64. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center From nobody Mon Aug 27 13:58:57 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A6235130DD3 for ; Mon, 27 Aug 2018 13:58:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.899 X-Spam-Level: X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dZzIP9cKGhzk for ; Mon, 27 Aug 2018 13:58:53 -0700 (PDT) Received: from p3plsmtpa12-07.prod.phx3.secureserver.net (p3plsmtpa12-07.prod.phx3.secureserver.net [68.178.252.236]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1D9A6124BE5 for ; Mon, 27 Aug 2018 13:58:52 -0700 (PDT) Received: from [192.168.1.82] ([76.192.164.238]) by :SMTPAUTH: with ESMTPSA id uOb5fgjcwj39luOb5faKfO; Mon, 27 Aug 2018 13:58:51 -0700 From: Laurence Lundblade Message-Id: <07152864-D276-451E-B001-DE429705B76B@island-resort.com> Content-Type: multipart/alternative; boundary="Apple-Mail=_B8755BBE-2E40-4482-A2E8-8AF4712AF70A" Mime-Version: 1.0 (Mac OS X Mail 11.4 \(3445.8.2\)) Date: Mon, 27 Aug 2018 13:58:51 -0700 In-Reply-To: Cc: cbor@ietf.org To: Sean Solberg References: <2B5BDE55-EE9E-4CAD-A2BF-8435C46103AE@island-resort.com> X-Mailer: Apple Mail (2.3445.8.2) X-CMAE-Envelope: MS4wfLrLbH3PHxzvlPZZC9rFC4nAWTCUvmqU/38XeoH7JRbNFz9VB0ZOkKP+hQtt9cvwZMBWrIebgn/rTwMuQDsbl/mkLdXHIPU2kwZ4ptZImRP6Fvsqqrik JCkeQWCzKVX3cv6xdyOfh9uG0/2dfxCLZ1KSYmUIrQQOJMKEn6HLlYa19uE25jIgc3JPh+sgDkcItwgS8WHR3f+D0fwk+D7lFJQ= Archived-At: Subject: Re: [Cbor] New pull request proposal for shortest float encoding X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2018 20:58:56 -0000 --Apple-Mail=_B8755BBE-2E40-4482-A2E8-8AF4712AF70A Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Sean, really glad you brought all this up and wrote it down. I was = thinking some of it myself and it needs to be addressed. Seems like we have a few options for indicating precision for shortest = float encoding - Each CBOR protocol specification explicitly states that shortest float = encoding is used for particular values. There is nothing explicit in the = protocol messages - We use a tag like Thiago suggested - Both of the above are allowed Since the tag takes up space, maybe the tag could be applied to maps and = arrays to indicate all float data items in them are of a particular = type. This would complicate the decoder as it has to keep track of some = context at each nesting level. I don=E2=80=99t think the preference for shortest form that we apply to = integers should apply to floats. Floats are fundamentally different from = integers. Floats are only approximations of the actual value. Precision = tells you how good the approximation is which matters in some use cases. = If you encode 3 as 0x00000003 instead of 0x03 you don=E2=80=99t really = get any more information, but there is a difference between 3.000 and = 3.000000. That is why I propose that float and double are not subject to = any shortest-form rule in the preferred serialization pull request = . I thought about leaving shortest float encoding entirely out of the RFC = for most simplicity. If we leave it out, any CBOR protocol could still = define it and use it.=20 However, shortest float encoding does seems truly useful in reducing = message sizes a lot for a lot of use cases. It also isn=E2=80=99t easy = to figure out (at least it took me a while=E2=80=A6), so I=E2=80=99m = inclined to put it in. Note that I believe the only purpose of shortest = float encoding is to reduce message size. I see no value in it for = canonicalization or for interop. In fact it is extra complexity so it = will hurt interop. LL > On Aug 26, 2018, at 8:34 PM, Sean Solberg wrote: >=20 > Follow up question: the proposal states: Generally the decoder = should promote half- and single-precision values to double-precision = when they are received as it is implied that all half- and = single-precision values transmitted are actually double precision.=20 >=20 > Why is it implied that a transmitted single precision value is = actually a double precision value? That makes no sense. A single = precision value is implied to be a single precision value in my mind. = If there was a "please treat as a double" flag sent along with a single = precision value, then the transmitted data explicitly defines that the = transmitted single precision value should be treated as a double = precision value by the receiver. without the flag, the single precision = value transmitted should be treated by the receiver as a single = precision value. Explicit behavior is great. Implicitly expecting a = receiver to treat all floats (half, single & double) as doubles when = received is implicitly a wrong choice. =20 >=20 > In general, any time things are done "implicitly", we end up with = compatibility problems because people make different implicit = choices/assumptions. I believe keeping things explicit is a much better = way to go. =20 > ..Sean >=20 > On Sun, Aug 26, 2018 at 10:25 PM, Sean Solberg > wrote: > Question on this: Even though the actual value transmitted has not = lost any precision for that particular value, how does the receiver then = know that the intended data type actually does have higher precision = than what was transmitted? Are there some flags somewhere that notify = the receiver that it's actually a double instead of a single or its = actually a single instead of a half? >=20 > ..Sean >=20 > On Sat, Aug 25, 2018 at 3:38 PM, Laurence Lundblade = > wrote: > https://github.com/cbor-wg/CBORbis/pull/32 = >=20 > This is a proposal to reduce CBOR message size without losing floating = point precision by encoding the larger floating point values as smaller = ones when it can be done without dropping any significant bits. >=20 > I personally don't think this should be part of canonicalization, or = part of preferred encoding. It's just a very useful way of encoding = floating points that seems worth including in the RFC. >=20 > LL >=20 >=20 > _______________________________________________ > CBOR mailing list > CBOR@ietf.org > https://www.ietf.org/mailman/listinfo/cbor = >=20 >=20 >=20 > _______________________________________________ > CBOR mailing list > CBOR@ietf.org > https://www.ietf.org/mailman/listinfo/cbor --Apple-Mail=_B8755BBE-2E40-4482-A2E8-8AF4712AF70A Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Sean,= really glad you brought all this up and wrote it down. I was thinking = some of it myself and it needs to be addressed.

Seems like we have a few options for = indicating precision for shortest float encoding
- = Each CBOR protocol specification explicitly states that shortest float = encoding is used for particular values. There is nothing explicit in the = protocol messages
- We use a tag like Thiago = suggested
- Both of the above are allowed

Since the tag takes up = space, maybe the tag could be applied to maps and arrays to indicate all = float data items in them are of a particular type. This would complicate = the decoder as it has to keep track of some context at each nesting = level.


I don=E2=80=99t think the preference = for shortest form that we apply to integers should apply to floats. = Floats are fundamentally different from integers.  Floats are only = approximations of the actual value. Precision tells you how good the = approximation is which matters in some use cases.  If you encode 3 = as 0x00000003 instead of 0x03 you don=E2=80=99t really get any more = information, but there is a difference between 3.000 and 3.000000. That = is why I propose that float and double are not subject to any = shortest-form rule in the preferred = serialization pull request.


I = thought about leaving shortest float encoding entirely out of the RFC = for most simplicity. If we leave it out, any CBOR protocol could still = define it and use it. 

However, shortest float encoding does seems truly useful in = reducing message sizes a lot for a lot of use cases. It also isn=E2=80=99t= easy to figure out (at least it took me a while=E2=80=A6), so I=E2=80=99m= inclined to put it in.  Note that I believe the only purpose of = shortest float encoding is to reduce message size. I see no value in it = for canonicalization or for interop.  In fact it is extra = complexity so it will hurt interop.

LL




On Aug 26, 2018, at 8:34 PM, Sean Solberg <ssolbs99@gmail.com> = wrote:

Follow up question:  the proposal = states:   Generally the decoder should promote half- and single-precision values to = double-precision when they are = received as it is implied that all half- and single-precision values transmitted are = actually double precision. Why is it = implied that a transmitted single precision value is actually a double = precision value?   That makes no sense.  A single = precision value is implied to be a single precision value in my = mind.   If there was a "please treat as a double" = flag sent along with a single precision value, then the transmitted data = explicitly defines that the transmitted single precision value should be = treated as a double precision value by the receiver.  without the = flag, the single precision value transmitted should be treated by the = receiver as a single precision value.  Explicit behavior is = great.  Implicitly expecting a receiver to treat all floats (half, = single & double) as doubles when received is implicitly a wrong = choice.    

In general, any time things are done = "implicitly", we end up with compatibility problems because people make = different implicit choices/assumptions.  I believe keeping things = explicit is a much better way to go.  
..Sean

On Sun, = Aug 26, 2018 at 10:25 PM, Sean Solberg <ssolbs99@gmail.com> wrote:
Question on this:  Even though the actual value = transmitted has not lost any precision for that particular value, how = does the receiver then know that the intended data type actually does = have higher precision than what was transmitted?  Are there some = flags somewhere that notify the receiver that it's actually a double = instead of a single or its actually a single instead of a half?

..Sean

On Sat, Aug 25, 2018 at 3:38 PM, Laurence = Lundblade <lgl@island-resort.com> wrote:
https://github.com/cbor-wg/CBORbis/pull/32

This is a proposal to reduce CBOR message size without losing = floating point precision by encoding the larger floating point values as = smaller ones when it can be done without dropping any significant = bits.

I personally don't think this should be part of = canonicalization, or part of preferred encoding. It's just a very useful = way of encoding floating points that seems worth including in the = RFC.

LL


_______________________________________________
CBOR mailing list
CBOR@ietf.org
https://www.ietf.org/mailman/listinfo/cbor



_______________________________________________
CBOR = mailing list
CBOR@ietf.org
https://www.ietf.org/mailman/listinfo/cbor

= --Apple-Mail=_B8755BBE-2E40-4482-A2E8-8AF4712AF70A-- From kbraun@obj-sys.com Wed Aug 29 11:02:53 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1F874130DFE for ; Wed, 29 Aug 2018 11:02:53 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.899 X-Spam-Level: X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LOgm5FguM5vU for ; Wed, 29 Aug 2018 11:02:51 -0700 (PDT) Received: from server.obj-sys.com (server.obj-sys.com [74.50.121.244]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 171A6130DDE for ; Wed, 29 Aug 2018 11:02:51 -0700 (PDT) Received: from 75-147-126-222-philadelphia.hfc.comcastbusiness.net ([75.147.126.222]:61489 helo=[10.0.0.100]) by server.obj-sys.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.91) (envelope-from ) id 1fv4np-0002bc-Mw for cbor@ietf.org; Wed, 29 Aug 2018 14:02:49 -0400 To: cbor@ietf.org From: Kevin Braun Message-ID: Date: Wed, 29 Aug 2018 14:02:52 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="------------FDCE6FD4DEBF6BFEABD570FD" Content-Language: en-US X-Antivirus: Avast (VPS 180829-2, 08/29/2018), Outbound message X-Antivirus-Status: Clean X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server.obj-sys.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - obj-sys.com X-Get-Message-Sender-Via: server.obj-sys.com: authenticated_id: kbraun@obj-sys.com X-Authenticated-Sender: server.obj-sys.com: kbraun@obj-sys.com X-Source: X-Source-Args: X-Source-Dir: Archived-At: Subject: [Cbor] draft-ietf-cbor-cddl-05 and maps X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Aug 2018 18:12:17 -0000 This is a multi-part message in MIME format. --------------FDCE6FD4DEBF6BFEABD570FD Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Hi folks, I've looked at draft-ietf-cbor-cddl-05 and read some of the discussion on this list regarding maps, which appear to have been made relative to draft-ietf-cbor-cddl-04. It is not entirely clear to me how map validation is supposed to work.  As this has been an area of some recent discussion already, perhaps this mail is superfluous, but in case it is not, I'll try to explain what I don't find clear... The discussion of "pathological non-determinism" is a little confusing.  Is the problem with the example one of specification or implementation - i.e. is the validity of {3: 5, 4: 6} against do-not-do-this undecided/undefined, or it is merely difficult for tools to decide this?  It seems to me {3: 5, 4: 6} is certainly valid against do-not-do-this, so I think the issue is really about implementation difficulty. do-not-do-this = { int => int, int => 6, } The text says, "This pathological non-determinism is caused by specifying more general before more specific....", so I take it the following would NOT be considered pathological: do-this = { int => 6, int => int, } Okay, so why is one pathological and not the other?  I guess because the PEG algorithm requires us to match the first entry first, though this isn't stated anywhere that I see.  But, even with such a rule, if it is a rule, I think do-this and do-not-do-this would validate the exact same set of maps, which seems to make sense given what Appendix C says about ordering:  a map expression "matches a valid CBOR map the key/value pairs of which can be ordered in such a way that the resulting sequence matches the group expression".  So, it appears that, at with respect to do-this and do-not-do-this, PEG is not actually relevant to specifying map validity but only to ensuring that an algorithm can more easily determine map validity. Also, I had to ask myself, "Wait, what exactly is non-deterministic here?".  I considered that you might be thinking of ambiguity in the attribution of input key-value pairs to the group entries, but I believe you had the algorithm in mind.  Attribution would be ambiguous for both do-not-do-this and do-this for the input { 4: 6, 5: 6 }, and there is no ambiguity of attribution for either case for the input { 4: 6, 5: 7 }. I think PEG does become relevant to the definition of map validity when there are occurrence indicators on the entries.  However, I don't see anything saying that occurrence indicators have greedy behavior as in PEG.  From the discussion of labeled-values in 3.5.3, I am guessing that if we write: bad-labeled-values = { * label => value fritz: number, } then we've created a map expression that can't validate any maps, as the first entry will consume any entry that could satisfy the second entry.  That, however, assumes some rule that I'm not quite sure how to state precisely.  It seems to depend on a model of execution that involves pulling key-value pairs out of the map to satisfy the group entries, a rule that occurrence indicators have greedy behavior (they require pulling out of the map everything that can be matched), and a rule that the pulling of entries from the map must be done for the group entries according to the order in which they are given in the map expression.  That suggests we need an algorithmic definition of map validity, which is in contrast to the description in Appendix C.  It seems that if bad-labeled-values ought to validate no maps, the key to that is that occurrence indicators be understood to be greedy.  However, saying occurrence indicators are greedy in the context of map validation lacks meaning (since the input is not ordered), unless you say that in the context of an algorithmic description that imposes some order on how things must be done, in which case "greedy" can mean pulling out all possible matches for the "current" group entry, up to the maximum allowed by the occurrences indicator. So, having said all of that, I think where I've ended up is that, assuming I've understood the spec's intentions, you really need an algorithmic definition of map validity in order to have a model that PEG principles can be applied to (e.g. greedy occurrences); the description in appendix C just doesn't seem to accomplish that. I hope you find these comments helpful. Regards, Kevin --------------FDCE6FD4DEBF6BFEABD570FD Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit Hi folks,

I've looked at draft-ietf-cbor-cddl-05 and read some of the discussion on this list regarding maps, which appear to have been made relative to draft-ietf-cbor-cddl-04.

It is not entirely clear to me how map validation is supposed to work.  As this has been an area of some recent discussion already, perhaps this mail is superfluous, but in case it is not, I'll try to explain what I don't find clear...

The discussion of "pathological non-determinism" is a little confusing.  Is the problem with the example one of specification or implementation - i.e. is the validity of {3: 5, 4: 6} against do-not-do-this undecided/undefined, or it is merely difficult for tools to decide this?  It seems to me {3: 5, 4: 6} is certainly valid against do-not-do-this, so I think the issue is really about implementation difficulty.
   do-not-do-this = {
     int => int,
     int => 6,
   }
The text says, "This pathological non-determinism is caused by specifying more general before more specific....", so I take it the following would NOT be considered pathological:
   do-this = {
     int => 6,
     int => int,
   }
Okay, so why is one pathological and not the other?  I guess because the PEG algorithm requires us to match the first entry first, though this isn't stated anywhere that I see.  But, even with such a rule, if it is a rule, I think do-this and do-not-do-this would validate the exact same set of maps, which seems to make sense given what Appendix C says about ordering:  a map expression "matches a valid CBOR map the key/value pairs of which can be ordered in such a way that the resulting sequence matches the group expression".  So, it appears that, at with respect to do-this and do-not-do-this, PEG is not actually relevant to specifying map validity but only to ensuring that an algorithm can more easily determine map validity.

Also, I had to ask myself, "Wait, what exactly is non-deterministic here?".  I considered that you might be thinking of ambiguity in the attribution of input key-value pairs to the group entries, but I believe you had the algorithm in mind.  Attribution would be ambiguous for both do-not-do-this and do-this for the input { 4: 6, 5: 6 }, and there is no ambiguity of attribution for either case for the input { 4: 6, 5: 7 }.

I think PEG does become relevant to the definition of map validity when there are occurrence indicators on the entries.  However, I don't see anything saying that occurrence indicators have greedy behavior as in PEG.  From the discussion of labeled-values in 3.5.3, I am guessing that if we write:
bad-labeled-values = {
     * label => value
     fritz: number,
   }
then we've created a map expression that can't validate any maps, as the first entry will consume any entry that could satisfy the second entry.  That, however, assumes some rule that I'm not quite sure how to state precisely.  It seems to depend on a model of execution that involves pulling key-value pairs out of the map to satisfy the group entries, a rule that occurrence indicators have greedy behavior (they require pulling out of the map everything that can be matched), and a rule that the pulling of entries from the map must be done for the group entries according to the order in which they are given in the map expression.  That suggests we need an algorithmic definition of map validity, which is in contrast to the description in Appendix C.  It seems that if bad-labeled-values ought to validate no maps, the key to that is that occurrence indicators be understood to be greedy.  However, saying occurrence indicators are greedy in the context of map validation lacks meaning (since the input is not ordered), unless you say that in the context of an algorithmic description that imposes some order on how things must be done, in which case "greedy" can mean pulling out all possible matches for the "current" group entry, up to the maximum allowed by the occurrences indicator.

So, having said all of that, I think where I've ended up is that, assuming I've understood the spec's intentions, you really need an algorithmic definition of map validity in order to have a model that PEG principles can be applied to (e.g. greedy occurrences); the description in appendix C just doesn't seem to accomplish that.

I hope you find these comments helpful.

Regards,
Kevin
--------------FDCE6FD4DEBF6BFEABD570FD-- From nobody Wed Aug 29 11:28:36 2018 Return-Path: X-Original-To: cbor@ietfa.amsl.com Delivered-To: cbor@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3A8F2130E06 for ; Wed, 29 Aug 2018 11:28:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.499 X-Spam-Level: X-Spam-Status: No, score=-0.499 tagged_above=-999 required=5 tests=[BAYES_05=-0.5, HTML_MESSAGE=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4LSwUff4cwBg for ; Wed, 29 Aug 2018 11:28:32 -0700 (PDT) Received: from server.obj-sys.com (server.obj-sys.com [74.50.121.244]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CAA35126DBF for ; Wed, 29 Aug 2018 11:28:32 -0700 (PDT) Received: from 75-147-126-222-philadelphia.hfc.comcastbusiness.net ([75.147.126.222]:61624 helo=[10.0.0.100]) by server.obj-sys.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.91) (envelope-from ) id 1fv5Ch-0003NX-Sh for cbor@ietf.org; Wed, 29 Aug 2018 14:28:31 -0400 To: cbor@ietf.org From: Kevin Braun Message-ID: <25aa170c-2267-e5ac-1525-c42ef52a09fa@obj-sys.com> Date: Wed, 29 Aug 2018 14:28:34 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="------------78735EC012AECFB13B61F7AE" Content-Language: en-US X-Antivirus: Avast (VPS 180829-2, 08/29/2018), Outbound message X-Antivirus-Status: Clean X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server.obj-sys.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - obj-sys.com X-Get-Message-Sender-Via: server.obj-sys.com: authenticated_id: kbraun@obj-sys.com X-Authenticated-Sender: server.obj-sys.com: kbraun@obj-sys.com X-Source: X-Source-Args: X-Source-Dir: Archived-At: Subject: [Cbor] draft-ietf-cbor-cddl-05 and representation types X-BeenThere: cbor@ietf.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Concise Binary Object Representation \(CBOR\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Aug 2018 18:28:34 -0000 This is a multi-part message in MIME format. --------------78735EC012AECFB13B61F7AE Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Hi folks, "2.2.3. Representation Types" says: 'Note that although this notation is based on the CBOR serialization, it is about a set of values at the data model level, e.g. "#7.25" specifies the set of values that can be represented as half-precision floats; it does not mandate that these values also do have to be serialized as half-precision floats: CDDL does not provide any language means to restrict the choice of serialization variants.' So, to be clear, uint (#0) could match a non-negative integer value encoded as a half-precision float (#7.25) or as a BigNum (#2)?  I just want to be clear this isn't something special for floats. Perhaps it would be better to say, 'e.g. "#7.25" specifies the set of values that can be represented as half-precision floats; it does not mandate that these values also do have to be serialized as half-precision floats, /or even as floats at all/: CDDL does not....'. Regards, Kevin --------------78735EC012AECFB13B61F7AE Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit Hi folks,

"2.2.3. Representation Types" says:

'Note that although this notation is based on the CBOR serialization, it is about a set of values at the data model level, e.g. "#7.25" specifies the set of values that can be represented as half-precision floats; it does not mandate that these values also do have to be serialized as half-precision floats: CDDL does not provide any language means to restrict the choice of serialization variants.'

So, to be clear, uint (#0) could match a non-negative integer value encoded as a half-precision float (#7.25) or as a BigNum (#2)?  I just want to be clear this isn't something special for floats.  Perhaps it would be better to say, 'e.g. "#7.25" specifies the set of values that can be represented as half-precision floats; it does not mandate that these values also do have to be serialized as half-precision floats, or even as floats at all: CDDL does not....'.

Regards,
Kevin
--------------78735EC012AECFB13B61F7AE--