Subject: Re: [netmod] Comments on draft-schoenw-netmod-smi-yang-02

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 4 Apr 2011 17:02:27 +0200

On Wed, Mar 30, 2011 at 01:11:05PM -0600, Brian Hedstrom wrote:
> NETMOD WG:
> 
> I have some questions on the use cases around the intended use of draft-schoenw-netmod-smi-yang-02.
> Presuming we end up having standards-based tools that convert SMIv2 MIBs into YANG modules and we can easily convert all the RFC MIBs into YANG Modules, what if we don't want all the SMIv2 baggage in our YANG modules and corresponding data model implementations on the client and server?

The SMIv2 to YANG translation gives you read-only access to data
originally defined SNMP MIB modules. In other words, it provides
access to lots of defined operational state through NETCONF.
 
> Will there now be two YANG modules published for every RFC MIB?  One that is auto-converted using the translation tool built around this draft standard and another one that is built from the ground-up (i.e., YANG-centric, no SMI baggage), perhaps by this working group (for example, the SNMP Cfg module)?

I do not think there is a need to republish any MIB modules in YANG
format once we have settled on the translation algorithm. If you need
the YANG version of a MIB module, you simply run a translator and
there you go. Perhaps someone provides an online service for this
(exists already today) so you do not even have to have a tool locally
installed.

Since the translation is read-only, we will have to write proper YANG
configuration data models for those things we consider worth having a
YANG configuration data model. Most IETF MIB modules do not cover
configuration well (if at all), the SNMPv3 MIB modules being the clear
exception to this rule. In the case of the SNMPv3 MIB modules, it
turned out that it is possible to come up with a simpler YANG
configuration data model compared to the way the MIB tables are
structured. We looked at several command line interfaces and we
observed that they often have a different structure than the
underlying SNMPv3 MIB tables they are based on.

/js (writing as contributor)

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Brian Hedstrom <[email-masked]>
Date: Mon, 4 Apr 2011 14:48:31 -0600

Hi Juergen,
I ran the DOCS-IF-MIB RFC 4546 through the LIBSMI SMI to YANG translation tool (smidump 0.4.8) and see that it is keeping the read-write SMI objects as config true in the resulting YANG.
Based on this IETF draft-02, would the translation change this resulting YANG leaf to config false, or would the resulting YANG have such object not exist (would the SMI to YANG translation prune the read-write SMI objects out of the YANG)?
Also, I see the RowStatus objects are brought in.  Will they be pruned with draft-02?

Any idea when the libsmi tool might be updated to match draft-02? Or are there other such translation tools supporting draft-02?

Thanks
Brian

-----Original Message-----
From: Juergen Schoenwaelder [mailto:[email-masked]] 
Sent: Monday, April 04, 2011 9:02 AM
To: Brian Hedstrom
Cc: [email-masked]
Subject: Re: [netmod] Comments on draft-schoenw-netmod-smi-yang-02

> Presuming we end up having standards-based tools that convert SMIv2 MIBs into YANG modules and we can easily convert all the RFC MIBs into YANG Modules, what if we don't want all the SMIv2 baggage…

The SMIv2 to YANG translation gives you read-only access to data originally defined SNMP MIB modules. In other words, it provides access to lots of defined operational state through NETCONF.
 
> Will there now be two YANG modules published for every RFC MIB?  One that is auto-converted using the translation tool built around this draft standard and another one that is built from the ground…

I do not think there is a need to republish any MIB modules in YANG format once we have settled on the translation algorithm. If you need the YANG version of a MIB module, you simply run a translator and there you go. Perhaps someone provides an online service for this (exists already today) so you do not even have to have a tool locally installed.

Since the translation is read-only, we will have to write proper YANG configuration data models for those things we consider worth having a YANG configuration data model. Most IETF MIB modules do not cover configuration well (if at all), the SNMPv3 MIB modules being the clear exception to this rule. In the case of the SNMPv3 MIB modules, it turned out that it is possible to come up with a simpler YANG configuration data model compared to the way the MIB tables are structured. We looked at several command line interfaces and we observed that they often have a different structure than the underlying SNMPv3 MIB tables they are based on.

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 4 Apr 2011 23:07:24 +0200

On Mon, Apr 04, 2011 at 02:48:31PM -0600, Brian Hedstrom wrote:

> I ran the DOCS-IF-MIB RFC 4546 through the LIBSMI SMI to YANG
> translation tool (smidump 0.4.8) and see that it is keeping the
> read-write SMI objects as config true in the resulting YANG.

You are obvously not using the trunk version of the SVN repository...

> Based on this IETF draft-02, would the translation change this
> resulting YANG leaf to config false, or would the resulting YANG
> have such object not exist (would the SMI to YANG translation prune
> the read-write SMI objects out of the YANG)?

Quoting draft-schoenw-netmod-smi-yang-02.txt:

   o  Top-level container must be marked as config false.

There is no text in draft-schoenw-netmod-smi-yang-02.txt that talks
about pruning certain object types.

> Also, I see the RowStatus objects are brought in.  Will they be

No, did you find something in the ID that makes you believe so?

> Any idea when the libsmi tool might be updated to match draft-02?

I am working on the next revision of the ID and hopefully I can make
my code follow the ID. But until the ID gets stable, you will have to
live with some inconsistencies (or get involved and contribute
patches). An ID is by definition work in progress.

> Or are there other such translation tools supporting draft-02?

I don't know.

/js

---

From: Brian Hedstrom <[email-masked]>
Date: Tue, 12 Apr 2011 10:55:10 -0600

Hi Juergen,
Following up on the use case question, let me give one such case along with my questions.

If we develop YANG configuration modules for DOCSIS interfaces (e.g, RFC 4546, RFC 4639, RFC 4131) by building these from scratch, and then we use the smi-yang-02 tool to translate these same MIB modules into read-only YANG modules, we have 2 sets of YANG modules:
1. Config YANG modules in namespace A
2. Status/State/Counters/Statistics Yang Modules in namespace B generated by smi-to-yang-02 tool

The 1. Modules use a YANG-centric indexing, not an SNMP-centric indexing.  So for example, they may use Name or Id instead of IfIndex for indexing.
The 2. Modules I presume still use ifIndexes for indexing?

Since the State data that goes along with the Config data is not present in the container in the Config module, it now resides in a different module, at a different namespace, using a different indexing mechanism.  How do I correlate the data back together?  Back in the old days when everything was in the SMI model, it was all done under an ifIndex scheme.  If we had built the Config and State/Counter/Statistics module together from the ground up, it could have had the same Name/Id indexing scheme in the single container.

It would seem that vendors/operators/etc who will use this draft will all be faced with the same challenges, I think we need a best practices or guidelines for how to solve this problem.

Thanks
Brian

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 13 Apr 2011 11:51:27 +0200

> 2. Status/State/Counters/Statistics Yang Modules in namespace B generated by smi-to-yang-02 tool

In XML land, it is quite normal to deal with multiple namespaces. In
fact, the NETCONF related YANG data modules already use several
namespaces. (Perhaps this was not the smartest idea but this is what
sometimes happens during development.)
 
> The 2. Modules I presume still use ifIndexes for indexing?

This is why the interfaces configuration data model provides the
ifIndex value so that things can be mapped together. And this is
nothing really new if you look at existing CLIs and config files,
where it quite unpopular to refer to interfaces via ifIndex values.

> Since the State data that goes along with the Config data is not present in the container in the Config module, it now resides in a different module, at a different namespace, using a different ind…

In my reality, the config interfaces usually do not use SNMP indexes.
Perhaps things were different in the specific case of cable modems.
And even in the SNMP world, the mapping from say a bridge port number
to an interface ifIndex and to a physical entity representing the port
is not always trivial.

> It would seem that vendors/operators/etc who will use this draft will all be faced with the same challenges, I think we need a best practices or guidelines for how to solve this problem.

So what would these guidelines be?

- Applications should be prepared to deal with namespaces. The
  management information for a function or resource might live in
  multiple namespaces.

- Applications should be prepared that the keys used to configure a
  function or resource may be different than the keys used to obtain
  operational state about a function or resource. In case the keys are
  different, data model designers should provide appropriate mappings.

Is this what you are looking for?

/js

---

From: Carl Moberg <[email-masked]>
Date: Thu, 14 Apr 2011 13:48:56 +0200

Coming in slightly from the sidelines here, but since Brian
  and I are hard at work on bringing YANG models into life in
  various other forums, we are, I guess destined to end up with
  some overlapping concerns.

> where it quite unpopular to refer to interfaces via ifIndex values.

  But the issue IMHO is bigger than this. As you probably know, many
  MIBs are flattened and indexed versions of more abstract models
  described in plain text and/or graphical representation in document
  format. As I've worked on a YANG model based on IEEE 802.1ag and some
  additions, this became very evident and I think a useful data point
  for this discussion.

  The background document [1] actually puts the information model
  in a nice tree-structured way that are somewhat well aligned with
  how people intuitively think about Connectivity Fault Management
  (domains, maintenance groups, management entities, etc) and how you
  would e.g. draw it on a white board to explain it to peers.

  The MIB, as you can imagine, is decidedly not (and can't be) as
  easy to grasp for readers. Even though it is based on the same
  information model it took me literally weeks to understand the ins
  and outs of this "projection" of the information model. This probably
  says something about my level of experience with MIBs but IMHO also
  should give a hint about what the cost of applying SMIv2 incurs on
  the reader.

  Now, the idea of taking this flattened version, grind it through
  the SMIv2 to YANG process is just not very appealing to me, since:

   1. It really doesn't do the actual specification much justice as
      it provides a painfully mangled version of the information model.

   2. I've obviously reused the tree-structure for the configuration
      part of the model, it came out much in alignment with the
      sentiment of .1ag. Now, trying to cross-reference between
      the flattened, auto-generated (non-config) module where e.g.
      counters that belongs to particular list members will be a
      very painful process and will (probably) require multiple
      NETCONF requests to come up with a set of data that model-wise
      belong together.

   3. Finally, since I actually could simply remove/not implement
      large parts of what was in the MIB as it was largely support
      tables to allow simple(r) SNMP queries back and forth over the
      data. Keeping these SNMP-support tables and expose it for
      NETCONF queries is plainly confusing for users.

My deliberately vague conclusion from this is that this, as much
else, requires some thought based on how the underlying information
model is described. My opinion, though, is that the SMIv2 to YANG
translation described in the document discussed will be most useful
in the case of *very* simple MIBs.

[1] http://standards.ieee.org/getieee802/download/802.1ag-2007.pdf

[...]

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 14 Apr 2011 15:59:59 +0200

On Thu, Apr 14, 2011 at 01:48:56PM +0200, Carl Moberg wrote:

[...]
 
> [1] http://standards.ieee.org/getieee802/download/802.1ag-2007.pdf

Whether it makes sense to manually redo a MIB data model in YANG must
be decided on a case-by-case basis since the decision depends on the
circumstances (e.g. how long a MIB module has already been out, how
often it has been implemented and what its deployment is) and the
structure or "complexity" of the MIB module.

The SMI to YANG mapping is useful in situations where there is already
SNMP instrumentation and application writers prefer to access this
existing instrumentation via a single protocol (NETCONF) instead of
dealing with multiple protocols (and likely different security
mechanisms and key management headaches). An automated mapping
provides quick (because automated) access to operational state data
without redesigning data models and rewriting parts of the
instrumentation.

It is an illusion to believe that all MIB modules will be manually
rewritten in YANG based on the underlying information model. Only some
will deserve such an effort.

/js

---

From: Brian Hedstrom <[email-masked]>
Date: Thu, 14 Apr 2011 08:43:04 -0600

I see value in the IETF draft including a section which discusses these tradeoffs so that readers can make intelligent decisions as to which approach is best, use the SMI-to-YANG translation, or construct the YANG module from scratch.
We are faced with this decision now in projects I am dealing with, and it's not an easy decision.  Borrowing knowledge from what Carl has gone through with 802.1ag is very valuable information I believe.  Including such information and examples in a section I think would be a value add.

Thanks,
Brian

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Juergen Schoenwaelder
Sent: Thursday, April 14, 2011 8:00 AM
To: Carl Moberg
Cc: [email-masked]
Subject: Re: [netmod] Comments on draft-schoenw-netmod-smi-yang-02

[...]
 
>  Now, the idea of taking this flattened version, grind it through  the 
> SMIv2 to YANG process is just not very appealing to me, since:
> 
> My deliberately vague conclusion from this is that this, as much else, 
> requires some thought based on how the underlying information model is 
> described. My opinion, though, is that the SMIv2 to YANG translation 
> described in the document discussed will be most useful in the case of 
> *very* simple MIBs.
> 

Whether it makes sense to manually redo a MIB data model in YANG must be decided on a case-by-case basis since the decision depends on the circumstances (e.g. how long a MIB module has already been out, how often it has been implemented and what its deployment is) and the structure or "complexity" of the MIB module.

The SMI to YANG mapping is useful in situations where there is already SNMP instrumentation and application writers prefer to access this existing instrumentation via a single protocol (NETCONF) instead of dealing with multiple protocols (and likely different security mechanisms and key management headaches). An automated mapping provides quick (because automated) access to operational state data without redesigning data models and rewriting parts of the instrumentation.

It is an illusion to believe that all MIB modules will be manually rewritten in YANG based on the underlying information model. Only some will deserve such an effort.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>
_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: "Randy Presuhn" <[email-masked]>
Date: Thu, 14 Apr 2011 09:58:08 -0700

Hi -

> From: "Juergen Schoenwaelder" <[email-masked]>
> To: "Carl Moberg" <[email-masked]>
> Cc: <[email-masked]>
> Sent: Thursday, April 14, 2011 6:59 AM
...
> will deserve such an effort.

But this does lead to a potentially interesting issue.
   When re-written modules do appear (and they will, simply because the
   SMIv2 ones are sometimes spectacularly ugly and counter-intuitive, even
   if logically perfectly correct) systems, in the interest of interoperability, will
   end up supporting both.  Now if there are two sets of knobs available to configure
   the same set of things, how do we ensure that the access control policy
   is coherent, never mind dealing with conflicts between the two sets of
   potentially overlapping configuration data?  (I say potentially, because
   one might use one model to configure one subset of interfaces, and
   another for the rest because one has different management applications
   for fine-tuning different interface types.)

Randy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 15 Apr 2011 06:43:54 +0200

>    for fine-tuning different interface types.)

The SMI to YANG translation is read-only. 

/js