Subject: [core] yang cbor encoding without SID

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 20 Jul 2017 10:42:34 +0200

Is it possible to define the cbor encoding in such a way that it is not
required to use SID? SID may be great for environments that need it, but
there are environments where it is not necessary and the complication of
dealing with a distributed set of registries may be considered costly.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 20 Jul 2017 10:50:51 +0200

> dealing with a distributed set of registries may be considered costly.

Yang-cbor also allows the use of string identifiers (search for “member name”).

Grüße, Carsten

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 20 Jul 2017 10:57:43 +0200

> Yang-cbor also allows the use of string identifiers (search for “member name”).

Thanks, much appreciated. Do I understand correctly that COMI requires
that the CBOR encoding uses SID?

/js

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 20 Jul 2017 11:24:22 +0200

> that the CBOR encoding uses SID?

A constrained device that wants to support both SID- and string-based identifiers would need to store all those strings to do identifier comparisons on incoming requests.  It seems to me it is a good tradeoff for COMI, which is meant to be able to run on constrained devices, to focus on the use of SIDs for identifiers.

Grüße, Carsten

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 20 Jul 2017 11:40:47 +0200

> A constrained device that wants to support both SID- and string-based identifiers would need to store all those strings to do identifier comparisons on incoming requests.  It seems to me it is a go…

OK. This limits COMI applicability to constrained devices in the sense
of a few 100k of memory or devices living on rather limited links.

I am interested in IoT devices that are able to run an embedded Linux
inside. So I am then probably more interested in using RESTCONF with
CBOR encoding since my devices are not that memory limited and I fear
the pain of a global single number managed in a decentralized way will
raise development pain. (So I would love to use CBOR without SID.)
Well, perhaps even RESTCONF with JSON is just good enough (clearly
more attractive for application writers).

/js

---

From: Andy Bierman <[email-masked]>
Date: Thu, 20 Jul 2017 22:58:23 -0700

Hi,

I agree with Juergen that CoMI without SID is a much better solution for
network management.
The SID approach is brand new, untested, and many network management
experts are concerned
that perfect registries and perfect implementations are unlikely.  The
question "why can't I just
use CBOR?" keeps coming up in private and WG emails.

If SID is so crucial, then why is plain CBOR good enough for CoAP now?
Seems to me that if it was so critical to collapse all identifier names down
to a single integer, that this requirement would apply to all CoAP servers,
not just CoAP using CoMI.

IMO CoMI is unusable (experimental at best) because of SID.
Perhaps a standards track RFC without SID and an Experimental RFC with SID
would be better.

Andy

On Thu, Jul 20, 2017 at 2:40 AM, Juergen Schoenwaelder <
[email-masked]> wrote:

> > On Jul 20, 2017, at 10:57, Juergen Schoenwaelder <
> [email-masked]> wrote:
> > >
> > A constrained device that wants to support both SID- and string-based
> identifiers would need to store all those strings to do identifier
> comparisons on incoming requests.  It seems to me it is a good tradeoff for
> COMI, which is meant to be able to run on constrained devices, to focus on
> the use of SIDs for identifiers.
> >
>
> _______________________________________________
> core mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/core
>

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 21 Jul 2017 08:14:38 +0200

On Jul 21, 2017, at 07:58, Andy Bierman <[email-masked]> wrote:
> 
> I agree with Juergen that CoMI without SID is a much better solution for network management.

For class-10+ (*), text string identifiers are fine.  Yang-cbor is neutral on this question.

COMI defines a number of media types, and these are right now using Yang-cbor with SIDs.  Maybe we should just define another set of media types with text string identifiers?  After all, COMI is used in the CoRE environment where we can use media types.

Grüße, Carsten

(*) https://tools.ietf.org/html/draft-bormann-lwig-7228bis-01#section-3

---

From: Michel Veillette <[email-masked]>
Date: Fri, 21 Jul 2017 06:56:03 +0000

Hi Andy

I won't object to the introduction of names within CoMI if someone takes the lead to introduce it.

However, I want to say that names can be error-prone also.
Similar as SID, names rely on a registry of files (i.e. yang files) to enable different implementations to share the same set of identifiers.
Considering that SID registries can automatically detect any misuse or duplicate use of SIDs, I don't see the incremental risk.

I also want to mention that the use of binary identifier is not a new or experimental concept. It have been used in different protocols with success for decades.

Regards,
Michel

-----Original Message-----
From: core [mailto:[email-masked]] On Behalf Of Carsten Bormann
Sent: Friday, July 21, 2017 8:15 AM
To: Andy Bierman <[email-masked]>
Cc: Core <[email-masked]>
Subject: Re: [core] yang cbor encoding without SID

Grüße, Carsten

---

From: Andy Bierman <[email-masked]>
Date: Fri, 21 Jul 2017 00:45:42 -0700

> For class-10+ (*), text string identifiers are fine.  Yang-cbor is neutral

But ALL CoAP devices using CBOR now use string identifiers right?
There is no alternative now, even for Class 1 devices, right?
So why is CBOR appropriate at all now, since identifiers are strings?

>
> COMI defines a number of media types, and these are right now using
> Yang-cbor with SIDs.  Maybe we should just define another set of media
> types with text string identifiers?  After all, COMI is used in the CoRE
> environment where we can use media types.
>
> Grüße, Carsten
>
Andy

---

From: Andy Bierman <[email-masked]>
Date: Fri, 21 Jul 2017 00:56:12 -0700

On Thu, Jul 20, 2017 at 11:56 PM, Michel Veillette <
[email-masked]> wrote:

> Similar as SID, names rely on a registry of files (i.e. yang files) to
> enable different implementations to share the same set of identifiers.
> Considering that SID registries can automatically detect any misuse or
> duplicate use of SIDs, I don't see the incremental risk.
>
>

I do not agree.
Currently, it is IMPOSSIBLE to create a YANG module for which an
instance document cannot be 100% correctly encoded and decoded using XML or
JSON.
This is not the case with YANG to CBOR at all.  Developers need special
tools to
check the YANG module to make sure it does not contain certain union types.
If it does, the YANG module must be changed or not used at all.

Also, YANG updates allow statements to be moved and altered in many ways.
None of these changes impact the XML or JSON encoding at all, but they do
affect CBOR.  It is critical to maintain previous SID assignments.
YANG identifiers do not have to be long. They can be 1 byte if you want.
YANG modules for constrained devices can use short identifiers.
Since XML and JSON are hierarchical, the identifier length is actually
the node name length (e.g., 'name') not the full path string from root
(e.g. /interfaces/interface/name).

Andy

> I also want to mention that the use of binary identifier is not a new or
> experimental concept. It have been used in different protocols with success
> for decades.
>
> Regards,
> Michel
>

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 21 Jul 2017 10:12:56 +0200

> of a few 100k of memory or devices living on rather limited links.

Not being affluent in memory may be one reason to use a constrained management solution; another reason may be not being affluent in bandwidth (or in total energy).  The third reason may be wanting to stay compatible with nodes that are in one of these two situations.  In other words: I don’t think the above mentioned limit in applicability really exists; you don’t have to abandon COMI as soon as you add some memory.

Grüße, Carsten

---

From: Michel Veillette <[email-masked]>
Date: Fri, 21 Jul 2017 08:17:34 +0000

Hi Andy

About the union, the CBOR encoding avoid any ambiguities between datatypes, something not archived by the xml mapping and partially archived by JSON. For example, in xml, it is impossible to differentiate the string "1" from the integer 1. The only ambiguities not resolved by the CBOR encoding is an union composed of two enumerations defining the same value associated to different meanings. I will argue that such definition is incorrect.

About your second point, if the JSON encoding is not impacted, I don't see why SID will be impacted. As you said yesterday, SIDs is a simple mapping between JSON paths and numeric IDs.

Regards,
Michel

From: Andy Bierman [mailto:[email-masked]]
Sent: Friday, July 21, 2017 9:56 AM
To: Michel Veillette <[email-masked]>
Cc: Carsten Bormann <[email-masked]>; Core <[email-masked]>
Subject: Re: [core] yang cbor encoding without SID

On Thu, Jul 20, 2017 at 11:56 PM, Michel Veillette <[email-masked]<mailto:[email-masked]>> wrote:
Hi Andy

I do not agree.
Currently, it is IMPOSSIBLE to create a YANG module for which an
instance document cannot be 100% correctly encoded and decoded using XML or JSON.
This is not the case with YANG to CBOR at all.  Developers need special tools to
check the YANG module to make sure it does not contain certain union types.
If it does, the YANG module must be changed or not used at all.

Andy

Regards,
Michel

-----Original Message-----
From: core [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Carsten Bormann
Sent: Friday, July 21, 2017 8:15 AM
To: Andy Bierman <[email-masked]<mailto:[email-masked]>>
Cc: Core <[email-masked]<mailto:[email-masked]>>
Subject: Re: [core] yang cbor encoding without SID

Grüße, Carsten

_______________________________________________
core mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/core

---

From: Andy Bierman <[email-masked]>
Date: Fri, 21 Jul 2017 08:26:50 +0000

I think a good compromise would be to:
1)  continue CoMI with SID as planned
2) create 2 different YANG to CBOR media types
3) RESTCONF can use the unoptimized media type
4) CoMI can use the optimized media type

Andy

On Fri, Jul 21, 2017 at 10:13 Carsten Bormann <[email-masked]> wrote:

> On Jul 20, 2017, at 11:40, Juergen Schoenwaelder <
> Not being affluent in memory may be one reason to use a constrained
> management solution; another reason may be not being affluent in bandwidth
> (or in total energy).  The third reason may be wanting to stay compatible
> with nodes that are in one of these two situations.  In other words: I
> don’t think the above mentioned limit in applicability really exists; you
> don’t have to abandon COMI as soon as you add some memory.
>
> Grüße, Carsten
>

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 21 Jul 2017 10:40:24 +0200

> 1)  continue CoMI with SID as planned

+1

> 2) create 2 different YANG to CBOR media types

This would be two sets of media types, one expecting SID support and one expecting the string identifiers to be known.
(Is there a need for mixing?  Not sure.)

> 4) CoMI can use the optimized media type

This would be what we would expect to be supported (MTI).  
Since this is REST, either could also support the other set of media types.

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 21 Jul 2017 11:01:56 +0200

> On Jul 21, 2017, at 10:17, Michel Veillette <[email-masked]> wrote:
> 
> Hi Andy
>  
> About the union, the CBOR encoding avoid any ambiguities between datatypes, something not archived by the xml mapping and partially archived by JSON. For example, in xml, it is impossible to differentiate the string "1" from the integer 1.

I think Andy’s point is that the serialization idiosyncrasies that leak into YANG are at this point well-known if it is the idiosyncrasies from the XML serialization (which have been maintained quite well in the JSON serialization, too) but not so for the CBOR serialization.  With some effort, it is possible to write YANG that exposes these idiosyncrasies:

> The only ambiguities not resolved by the CBOR encoding is an union composed of two enumerations defining the same value associated to different meanings. I will argue that such definition is incorrect.

Not sure that is the only case, but it still is a case.  Right now, the YANG tools don’t know about these cases yet.  So having, say, a pyang extension that tests for this might be a useful thing to have.

Grüße, Carsten

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 21 Jul 2017 11:51:27 +0200

> COMI defines a number of media types, and these are right now using Yang-cbor with SIDs.  Maybe we should just define another set of media types with text string identifiers?  After all, COMI is us…

Having separate media types would clearly help.

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 21 Jul 2017 11:55:24 +0200

> Considering that SID registries can automatically detect any misuse or duplicate use of SIDs, I don't see the incremental risk.

Michel,

Andy and I have many years of MIB doctor and YANG doctor experience
and getting people to write reasonably correct modules is already
hard. Getting multiple organizations to maintain a global unique
number space for all YANG identifiers where everybody fully
understands the rules (a number once assigned is never allowed to
change) is from my own experience a non-starter.

But, if people believe this is doable, go for it - as long as there
are ways to fall back to an encoding that does not require this global
number space to work I am fine to let people run the experiement and
collect their own experience.

/js

---

From: Laurent Toutain <[email-masked]>
Date: Fri, 21 Jul 2017 12:07:26 +0200

> 1)  continue CoMI with SID as planned

+1 there is not only constrained device, there is also constrained network
where names can be too long to be sent.

> 2) create 2 different YANG to CBOR media types

may be a problem for interoperability

-- 
Laurent Toutain
+--- VoIP (recommended) ---+----------- Télécom Bretagne -----------+
| Tel: +33 2 22 06 8156    | Tel: + 33 2 99 12 7026                 | Visit
:
| Mob: +33 6 800 75 900    |                                        |
| Fax: +33 2 22 06 8445    | Fax: +33 2 99 12 7030                  |
http://class.touta.in
| [email-masked]         | [email-masked]    |
+--------------------------+----------------------------------------+

---

From: Alexander Pelov <[email-masked]>
Date: Fri, 21 Jul 2017 12:15:52 +0200

Dear Juergen, all,

We’ve had these discussions for some time now and I am very glad that fresh eyeballs are looking at the problem. 

At some point we’ve had both the « name » and SID in the YANG-CBOR draft. The choice depends on the problem space you are trying to address. We chose to have a pragmatic approach - solve at least the problems of constrained devices and constrained networks.

If you are having a NETCONF/RESTCONF server anyway, and just want an efficient CBOR representation, then you could do it more simply than following the YANG-CBOR draft. Get your JSON, convert to CBOR, then on the receiver side convert back to JSON and process it. 

The issue is when you want to address constrained devices, where you DON’T want to keep string identifiers lying around in case someone sends you a name. This is where things get more challenging - and this is the question we’re solving here. 

Yes, it is trivial to add item names as identifiers. I think that’s exactly why we should not add them today. (we are not blocking any future development - and all will remain perfectly compatible).

Alexander

> Le 21 juil. 2017 à 11:55, Juergen Schoenwaelder <[email-masked]> a écrit :
>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 21 Jul 2017 12:29:19 +0200

> If you are having a NETCONF/RESTCONF server anyway, and just want an efficient CBOR representation, then you could do it more simply than following the YANG-CBOR draft. Get your JSON, convert to CB…

So does the YANG-CBOR draft exist only so that we have SID? Or are
there any other differences between [1] YANG->JSON->CBOR (RFC 7951 +
RFC 7049 section 4) and [2] YANG->CBOR (draft-ietf-core-yang-cbor-04)?

I understand that I can implement [1] without having to generate JSON
first. Perhaps a discussion of this would be a nice addition to
draft-ietf-core-yang-cbor-04.

/js

---

From: Alexander Pelov <[email-masked]>
Date: Fri, 21 Jul 2017 12:35:38 +0200

See inline.

> Le 21 juil. 2017 à 12:29, Juergen Schoenwaelder <[email-masked]> a écrit :
> 

Differences exist. YANG-CBOR is more efficient than YANG->JSON->CBOR, but definitely not on a scale that would be important for a 16 MB device and a 200kbps+ network.

What I’m saying is: if you don’t care that much for the amount of data and the impact on the code on the end-device, then YANG->JSON->CBOR is good enough.
If you do care a lot for these things, then you will be willing to go an extra mile.

If you are doing YANG->JSON->CBOR you are not doing YANG->CBOR. You are doing JSON->CBOR and it’s already in the CBOR RFC. Maybe you’re right that we should add a paragraph for people that don’t want to use SIDs.

Alexander

---

From: Michel Veillette <[email-masked]>
Date: Fri, 21 Jul 2017 10:38:55 +0000

Hi Juergen, Hi Alexander

I just want to mention that the following text relative to this topic
is already present In the YANG to CBOR draft.

https://tools.ietf.org/html/draft-ietf-core-yang-cbor-04#section-3

   This specification supports two type of CBOR keys; YANG
   Schema Item iDentifier (SID) as defined in Section 2.1 and member
   names as defined in [RFC7951].  Each of these key types is encoded
   using a specific CBOR type which allows their interpretation during
   the deserialization process.  The end user of this mapping
   specification (e.g.  RESTCONF [RFC8040], CoMI [I-D.ietf-core-comi])
   can mandate the use of a specific key type.

Regards,
Michel

-----Original Message-----
From: Juergen Schoenwaelder [mailto:[email-masked]] 
Sent: Friday, July 21, 2017 12:29 PM
To: Alexander Pelov <[email-masked]>
Cc: Michel Veillette <[email-masked]>; Core <[email-masked]>
Subject: Re: [core] yang cbor encoding without SID

> If you are having a NETCONF/RESTCONF server anyway, and just want an efficient CBOR representation, then you could do it more simply than following the YANG-CBOR draft. Get your JSON, convert to CB…

So does the YANG-CBOR draft exist only so that we have SID? Or are there any other differences between [1] YANG->JSON->CBOR (RFC 7951 + RFC 7049 section 4) and [2] YANG->CBOR (draft-ietf-core-yang-cbor-04)?

I understand that I can implement [1] without having to generate JSON first. Perhaps a discussion of this would be a nice addition to draft-ietf-core-yang-cbor-04.

/js

---

From: Alexander Pelov <[email-masked]>
Date: Fri, 21 Jul 2017 12:48:28 +0200

Oh, thanks for the reminder, Michel !

I remember having long discussions on removing the member names from the spec.. and then we had the decision to keep them and leave to the implementers. 

So, it’s solved then?

Alexander

> Le 21 juil. 2017 à 12:38, Michel Veillette <[email-masked]> a écrit :
> 
> Hi Juergen, Hi Alexander
>

---

From: Andy Bierman <[email-masked]>
Date: Fri, 21 Jul 2017 15:57:10 -0700

On Fri, Jul 21, 2017 at 2:01 AM, Carsten Bormann <[email-masked]> wrote:

>
> > On Jul 21, 2017, at 10:17, Michel Veillette <
> > About the union, the CBOR encoding avoid any ambiguities between
> datatypes, something not archived by the xml mapping and partially archived
> by JSON. For example, in xml, it is impossible to differentiate the string
> "1" from the integer 1.
>
> I think Andy’s point is that the serialization idiosyncrasies that leak
> into YANG are at this point well-known if it is the idiosyncrasies from the
> XML serialization (which have been maintained quite well in the JSON
> serialization, too) but not so for the CBOR serialization.  With some
> effort, it is possible to write YANG that exposes these idiosyncrasies:
>
>

What I mean is that XML and JSON encodings align with YANG by design.
The YANG identifiers are strings. The value spaces are specified such that
any
collision is invalid YANG.

The  XML/JSON encoding formats are "protected" by the rules of YANG.
It is not possible for identifier or value space errors to occur if the
YANG module is valid.
This protection is built into the data modeling language by design for
numbered
identifiers as well (e.g., SMIv2, Zigbee, etc.)

But SID is completely unprotected from these errors because the SID
identifier and
data type encoding can change even though no YANG rules have been violated.
Statement position carries (almost) no semantic or identifier value at all
in YANG.
Rely on it at your own risk.

> The only ambiguities not resolved by the CBOR encoding is an union
> composed of two enumerations defining the same value associated to
> different meanings. I will argue that such definition is incorrect.
>
> Not sure that is the only case, but it still is a case.  Right now, the
> YANG tools don’t know about these cases yet.  So having, say, a pyang
> extension that tests for this might be a useful thing to have.
>
> Grüße, Carsten
>
>
Andy

---

From: Michel Veillette <[email-masked]>
Date: Sat, 22 Jul 2017 02:45:54 +0000

Hi Andy

I will agree with you on one point, some implementers might have defined carelessly 'value' and 'position' because these statements was not useful for their NETCONF or RESTCONF implementations. This is not an issue with YANG but with those developers. I don't agree that the designers of the YANG modeling language put these statements in the schema definition just for fun. They certainly foresaw the time YANG will be implemented in binary.

The impacts on the yang to cbor mapping is however limited. The YANG specification and YANG validators guarantee the uniqueness of enumeration values and bits positions. When used in a union, the conclusion of the design team was to add a check in validation tools to generate a warning when duplicate values or positions are detected. Such check can be part of a validation performed when registering SIDs. Since those values or position are not likely to have been used, fixing them won't have any impacts on prior implementations. Nobody however have reported so far any instances of this issue.

I don't see any reasons why YANG won't be suitable for binary implementations and designers of this modeling language should be proud of this accomplishment.

Regards,
Michel

From: Andy Bierman [mailto:[email-masked]]
Sent: Saturday, July 22, 2017 12:57 AM
To: Carsten Bormann <[email-masked]>
Cc: Michel Veillette <[email-masked]>; Core <[email-masked]>
Subject: Re: [core] yang cbor encoding without SID

On Fri, Jul 21, 2017 at 2:01 AM, Carsten Bormann <[email-masked]<mailto:[email-masked]>> wrote:

> On Jul 21, 2017, at 10:17, Michel Veillette <[email-masked]<mailto:[email-masked]>> wrote:
>
> Hi Andy
>

What I mean is that XML and JSON encodings align with YANG by design.
The YANG identifiers are strings. The value spaces are specified such that any
collision is invalid YANG.

The  XML/JSON encoding formats are "protected" by the rules of YANG.
It is not possible for identifier or value space errors to occur if the YANG module is valid.
This protection is built into the data modeling language by design for numbered
identifiers as well (e.g., SMIv2, Zigbee, etc.)

But SID is completely unprotected from these errors because the SID identifier and
data type encoding can change even though no YANG rules have been violated.
Statement position carries (almost) no semantic or identifier value at all in YANG.
Rely on it at your own risk.

Grüße, Carsten

Andy

---

From: Andy Bierman <[email-masked]>
Date: Fri, 21 Jul 2017 21:53:15 -0700

Hi,

I am not going to explain it anymore.
The order most statements appear in a YANG file is irrelevant to the
semantics of the YANG module
except (1) auto-numbering of position or value and (2) evaluation order of
union member types.
Keeping the SID files perfectly aligned with the YANG fie is a challenge.
There is no way for
a module to get out of synch with itself wrt/ schema node naming or
instance value evaluation.

So of course the tools will be perfect and the developers will be perfect
and never let the registries and implementations get out of synch.
Otherwise SID will produce incorrect results.
If you cannot recognize the additional risks introduced by SID,
you probably need to study RFC 7950 a bit more.

Andy

On Fri, Jul 21, 2017 at 7:45 PM, Michel Veillette <
[email-masked]> wrote:

> Hi Andy
>
>
>
> I will agree with you on one point, some implementers might have defined
> carelessly 'value' and 'position' because these statements was not useful
> for their NETCONF or RESTCONF implementations. This is not an issue with
> YANG but with those developers. I don't agree that the designers of the
> YANG modeling language put these statements in the schema definition just
> for fun. They certainly foresaw the time YANG will be implemented in binary.
>
>
>
> The impacts on the yang to cbor mapping is however limited. The YANG
> specification and YANG validators guarantee the uniqueness of enumeration
> values and bits positions. When used in a union, the conclusion of the
> design team was to add a check in validation tools to generate a warning
> when duplicate values or positions are detected. Such check can be part of
> a validation performed when registering SIDs. Since those values or
> position are not likely to have been used, fixing them won't have any
> impacts on prior implementations. Nobody however have reported so far any
> instances of this issue.
>
>
>
> I don't see any reasons why YANG won't be suitable for binary
> implementations and designers of this modeling language should be proud of
> this accomplishment.
>
>
>
> Regards,
>
> Michel
>
>
>
>
>
> *From:* Andy Bierman [mailto:[email-masked]]
> *Sent:* Saturday, July 22, 2017 12:57 AM
> *To:* Carsten Bormann <[email-masked]>
> *Cc:* Michel Veillette <[email-masked]>; Core <
> [email-masked]>
> *Subject:* Re: [core] yang cbor encoding without SID
>
>
>
>
>
>
>

---

From: Michel Veillette <[email-masked]>
Date: Sat, 22 Jul 2017 06:47:27 +0000

Hi Andy

We do recognize the additional risks if SIDs are unmanaged.
This is why we work actively on the development of the registration WEB site.
As soon a beta version is online, I let you try it to see if it address your concerns.

Regards,
Michel

Sent: Saturday, July 22, 2017 6:53 AM

Hi,

I am not going to explain it anymore.
The order most statements appear in a YANG file is irrelevant to the semantics of the YANG module
except (1) auto-numbering of position or value and (2) evaluation order of union member types.
Keeping the SID files perfectly aligned with the YANG fie is a challenge. There is no way for
a module to get out of synch with itself wrt/ schema node naming or instance value evaluation.

Andy

On Fri, Jul 21, 2017 at 7:45 PM, Michel Veillette <[email-masked]<mailto:[email-masked]>> wrote:
Hi Andy

Regards,
Michel

From: Andy Bierman [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Saturday, July 22, 2017 12:57 AM
To: Carsten Bormann <[email-masked]<mailto:[email-masked]>>
Cc: Michel Veillette <[email-masked]<mailto:[email-masked]>>; Core <[email-masked]<mailto:[email-masked]>>
Subject: Re: [core] yang cbor encoding without SID

Grüße, Carsten

Andy

---

From: Andy Bierman <[email-masked]>
Date: Sat, 22 Jul 2017 07:43:20 +0000

> Hi Andy
I think 2 sets of media types addresses all my concerns. It is very simple
to define a YANG to CBOR encoding that aligns with the standard data naming
and instance value encoding. It is very easy to use the Accept header to
insure interoperability. This "plain" encoding can be quite useful for
RESTCONF. We have already implemented RESTCONF over CoAP and plan to use it
for that as well.

Andy

> *Sent:* Saturday, July 22, 2017 6:53 AM
> *To:* Michel Veillette <[email-masked]>
> *Cc:* Carsten Bormann <[email-masked]>; Core <[email-masked]>
>
>

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Sat, 22 Jul 2017 10:35:04 +0000

Hello Michel

Maybe some thoughts should be given on how this ledger is implemented before we start. Looks to me that the problem fits the domain of block chains quite well but I may be wrong.

Regards,

Pascal

Le 22 juil. 2017 à 08:47, Michel Veillette <[email-masked]<mailto:[email-masked]>> a écrit :

Hi Andy

Regards,
Michel

From: Andy Bierman [mailto:[email-masked]]
Sent: Saturday, July 22, 2017 6:53 AM
To: Michel Veillette <[email-masked]<mailto:[email-masked]>>
Cc: Carsten Bormann <[email-masked]<mailto:[email-masked]>>; Core <[email-masked]<mailto:[email-masked]>>
Subject: Re: [core] yang cbor encoding without SID

Hi,

Andy

Regards,
Michel

Grüße, Carsten

Andy

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 22 Jul 2017 13:39:31 +0200

Yes, Alex had actually proposed this a couple of years ago in the first DIN side meeting. The beauty of the range application approach is that we can do the block chain thing later for a range once we have figured out a good way to run this ledger while starting out today with the approach shown by Alex. 

Sent from mobile

> On 22. Jul 2017, at 12:35, Pascal Thubert (pthubert) <[email-masked]> wrote:
> 
> Hello Michel 
> 
> Le 22 juil. 2017 à 08:47, Michel Veillette <[email-masked]> a écrit :
> 
>> Hi Andy
>>  
>> On Fri, Jul 21, 2017 at 7:45 PM, Michel Veillette <[email-masked]> wrote:
>> Hi Andy
>>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Sat, 22 Jul 2017 14:02:59 +0200

Michel,

do you really believe that well managed registries will for example
survive company mergers or acquisitions? What about modules under
development where people build prototypes and then later identifiers
get changed? Do you expect _all_ developers will go and update the
number schemes embedded in their code correctly?

Anyway, I do not want to prevent people from getting experience with
this as long as there is a way to use YANG-CBOR without SID. Make SID
an additional optimization. If you want to require SID for CoMI, fine.
Then in the worst case CoMI goes with the fate of SIDs.

/js

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 22 Jul 2017 15:37:33 +0200

> On Jul 22, 2017, at 14:02, Juergen Schoenwaelder <[email-masked]> wrote:
> 
> survive company mergers or acquisitions?

They mostly seem to manage that for their financial information :-)

> What about modules under

I think that the situation in 2017 is really different from that even a decade ago.
Developers have been getting used to version control, which does require a bit of a protocol to use it right.
We also have web applications that help with that (bitbucket, github, gitlab, …).

So I’m more optimistic that the same can be done for SID control.
The SID tool in pyang can be the basis of one way to do it, and we probably should evolve an informational document explaining a best practice way to handle SIDs.

> Then in the worst case CoMI goes with the fate of SIDs.

Sounds good to me.

So we would have the following levels of optimization:

1 RESTCONF with JSON-YANG
2 RESTCONF with CBOR-YANG
3 COMI with string identifiers
4 COMI with SIDs

Are 2 and 3 the same?

Do we have to write up something for RESTCONF over CoAP?

(There might also be NETCONF with CBOR-YANG, which doesn’t quite fit into the above hierarchy.)

Grüße, Carsten

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Sat, 22 Jul 2017 16:02:30 +0200

> 2 RESTCONF with CBOR-YANG

Option #2 makes sense to investigate, i.e., to find out how much
efficiency gain there is. Someone would need to implement both and
provide hard numbers. I would expect a certain gain on the
encoding/decoding side but then one needs to lock at the whole
picture. But I think this is a reasonable experiment to
make. Volunteers?

> 3 COMI with string identifiers

Up to COMI people to decide, perhaps #4 is sufficient if COMI's scope
is restricted to very constrained devices or boxes using highly
limited links and we do not need #3. All depends on what is the real
target for COMI.

> into the above hierarchy.)

NETCONF has no way to negotiate different content encodings, NETCONF
is pretty much hard wired to use XML since the whole protocol is
encoded in XML.

/js

---

From: Andy Bierman <[email-masked]>
Date: Sun, 23 Jul 2017 11:39:53 -0700

> picture. But I think this is a reasonable experiment to

Isn't CBOR-YANG covered by changes to the current document?

IMO, the following protocol-independent media types should be defined:

(2)
  application/yang-data+cbor : exact same identifier and instance value
mappings as XML and JSON versions of yang-data.

(4)
  application/yang-data+cbor.sid : identifier and instance value mappings
according to SID registry

Efficiency in system design is not limited to the protocol.
Designers of YANG modules for constrained devices can use numeric types
over string types
and increase the efficiency of the CBOR encoding.

The main use-case for CBOR for RESTCONF right now is pushing lots of
operational state telemetry data
off of routers. I can imagine routers using application/yang-data+cbor.sid
a little at a time, for spot solutions.
Some might want to see the technology mature before putting it in
production networks.

Andy

---

From: peter van der Stok <[email-masked]>
Date: Mon, 24 Jul 2017 08:42:47 +0200

For case 3, CoMI with string identifiers, I recommend to send hashes and 
fall back to strings when a clash occurs.
Strings are only used for clashing names, all others are sent as hashes.
For an installation with 1000 names and 30 bit hash, the clash 
probability is less than 5*10^-4.
With little extra code, string identifiers are only necessary when 3 or 
more names clash to the same hash value.
The probability of such a clash with 30 bit hash and 1000 names is less 
than 5*10^-8.
For comparison, the probability that a given flight will crash is 10^-7.

Peter

Andy Bierman schreef op 2017-07-23 20:39:
> On Sat, Jul 22, 2017 at 7:02 AM, Juergen Schoenwaelder
> <[email-masked]> wrote:
> 
>   application/yang-data+cbor : exact same identifier and instance
> value mappings as XML and JSON versions of yang-data.
> 
> (4)
>   application/yang-data+cbor.sid : identifier and instance value
> mappings according to SID registry
> 
> Designers of YANG modules for constrained devices can use numeric
> types over string types
> off of routers. I can imagine routers using
> application/yang-data+cbor.sid a little at a time, for spot solutions.
>>> (There might also be NETCONF with CBOR-YANG, which doesn’t quite
>> fit
>>> into the above hierarchy.)
>> 
>> Fax:   +49 421 200 3103         <http://www.jacobs-university.de/
>> [1]>
>> 
>> https://www.ietf.org/mailman/listinfo/core [2]
> 
> 
> 
> Links:
> ------
> [1] http://www.jacobs-university.de/
> [2] https://www.ietf.org/mailman/listinfo/core

---

From: Alexander Pelov <[email-masked]>
Date: Mon, 24 Jul 2017 10:55:36 +0200

Dear all,

Section "4.2.2. Member names as keys" of draft-ietf-core-yang-cbor-04 describes how you achieve what you want in your demand.

I think there is no point in continuing the discussion of should we include something that is already there. You can start using it as it is today.

Also, we will be doing an interop event by the end of August. Please, make sure to participate to this event, as without running code, a specification serves no useful purpose.

Alexander

> Le 24 juil. 2017 à 08:42, peter van der Stok <[email-masked]> a écrit :
> 
> For case 3, CoMI with string identifiers, I recommend to send hashes and fall back to strings when a clash occurs.
> For an installation with 1000 names and 30 bit hash, the clash probability is less than 5*10^-4.
> With little extra code, string identifiers are only necessary when 3 or more names clash to the same hash value.
> The probability of such a clash with 30 bit hash and 1000 names is less than 5*10^-8.
>>> [email-masked] <mailto:[email-masked]>
>>> https://www.ietf.org/mailman/listinfo/core <https://www.ietf.org/mailman/listinfo/core> [2]
>> Links:
>> ------
>> [1] http://www.jacobs-university.de/ <http://www.jacobs-university.de/>
>> [2] https://www.ietf.org/mailman/listinfo/core <https://www.ietf.org/mailman/listinfo/core>
>> [email-masked] <mailto:[email-masked]>
>> https://www.ietf.org/mailman/listinfo/core <https://www.ietf.org/mailman/listinfo/core>
> 
> [email-masked] <mailto:[email-masked]>
> https://www.ietf.org/mailman/listinfo/core <https://www.ietf.org/mailman/listinfo/core>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 24 Jul 2017 11:18:29 +0200

> I think there is no point in continuing the discussion of should we include something that is already there. You can start using it as it is today.

There need to be different media types and it needs to be clear what
is required to implement in which situation.

/js

---

From: Alexander Pelov <[email-masked]>
Date: Mon, 24 Jul 2017 11:31:03 +0200

Dear Juergen,

The Content Format is out of scope for YANG-CBOR. It is only pertinent for CoMI. 

Alexander

> Le 24 juil. 2017 à 11:18, Juergen Schoenwaelder <[email-masked]> a écrit :
>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 24 Jul 2017 12:00:37 +0200

Dear Alexander,

I think people what to implement YANG-CBOR without having to implement
the SID part. If the document makes it clear that this is possible
without breaking conformance, I think everybody is fine.

/js

---

From: Alexander Pelov <[email-masked]>
Date: Mon, 24 Jul 2017 12:02:28 +0200

Dear Juergen,

Sounds perfectly reasonable! Thanks for pointing it out - and you are right about it.

It is already in the text, but I’m taking note that we should make it more explicit and maybe add a sentence to underline it in the introduction / abstract.

Does that sound OK ?

Alexander

> Le 24 juil. 2017 à 12:00, Juergen Schoenwaelder <[email-masked]> a écrit :
> 
> Dear Alexander,
> 
> Fax:   +49 421 200 3103         <http://www.jacobs-university.de/ <http://www.jacobs-university.de/>>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 24 Jul 2017 12:13:00 +0200

> It is already in the text, but I’m taking note that we should make it more explicit and maybe add a sentence to underline it in the introduction / abstract.

Yes

/js

---

From: Alexander Pelov <[email-masked]>
Date: Mon, 24 Jul 2017 14:30:36 +0200

Ok, great - we’ll include it in the next version.

Thanks a lot, Juergen ! 
Alexander

> Le 24 juil. 2017 à 12:13, Juergen Schoenwaelder <[email-masked]> a écrit :
>