Subject: Re: [Netconf] [netmod] Alternative YANG library structure for 7895bis

From: Kent Watsen <[email-masked]>
Date: Fri, 8 Dec 2017 17:21:47 +0000

CC-ing NETCONF, where the draft is being worked on.

Kent

On Fri, 2017-12-08 at 16:34 +0100, Juergen Schoenwaelder wrote:
> On Fri, Dec 08, 2017 at 04:19:28PM +0100, Vladimir Vassilev wrote:
> > 
> > Yes. The default value for yang-library-datastore leaf is ds:operational
> > (the only possible one for the ds:operational datastore). This is backward
> > compatible. If one needs different model for 'running', etc. then a new
> > datastore identity has to be defined  and set in place of the default value.
> > Then this identity can be used to read the yang-library data with
> > <get-data>.
> > 
> 
> Sorry, but I have to ask this: How do I obtain the schema for the
> datastore (lets call it <running-library>) that reports the schema for
> <running>? Is there another <running-library-library> datastore? Will
> the recursion end? Perhaps it does since <running-library-library>
> might have itself listed as the schema defining datastore. I guess
> Lada will like these kind of meta and meta-meta datastores.

Not really. Metadata needn't be in datastores.

Lada

> /js
-- 
Ladislav Lhotka
Head, CZ.NIC Labs
PGP Key ID: 0xB8F92B08A9F76C67

_______________________________________________
netmod mailing list
[email-masked]
https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_netmod&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=5qj6BQUSwqYmkAVeKz5axFV8k3gxYEPSJ5Cp0RSnxrE&s=I7fR1GY5lN2hVMkDuvryrhDeRypike3wPeFRrvQI5l8&e=

---

From: Andy Bierman <[email-masked]>
Date: Fri, 8 Dec 2017 10:01:20 -0800

Hi,

A library per datastore sounds too complicated.
I prefer the proposal that was made at the IETF meeting that had
a 'not-implemented-in' leaf-list and a single module list.

Why is it interesting to have a separate module list for regular modules
and imported modules?
I prefer to keep the conformance leaf and not change the module list.

NMDA needs to be possible to implement with a single schema tree such that
a module
is implemented in all datastores, or a subset of all datastores.  Otherwise
it probably won't
get supported in clients.

Andy

On Fri, Dec 8, 2017 at 9:21 AM, Kent Watsen <[email-masked]> wrote:

> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.
> ietf.org_mailman_listinfo_netmod&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-
> ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=
> 5qj6BQUSwqYmkAVeKz5axFV8k3gxYEPSJ5Cp0RSnxrE&s=
> I7fR1GY5lN2hVMkDuvryrhDeRypike3wPeFRrvQI5l8&e=
>
>
> https://www.ietf.org/mailman/listinfo/netmod
>

---

From: Vladimir Vassilev <[email-masked]>
Date: Sat, 9 Dec 2017 12:49:18 +0100

> A library per datastore sounds too complicated.
I am not proposing that.

The fundamental point proposed is that the datastore relevant bits are 
kept in the ietf-datastores module instead of merging everything in a 
new ietf-yang-library entangled monster module. If needed 
ietf-datastores can augment ietf-yang-library but ietf-yang-library 
should be usable on its own without ietf-datastores. The solution is 
coherent and modular and addresses the problem statement.
> a 'not-implemented-in' leaf-list and a single module list.
This constraint is already specified in the text of the revised 
datastores draft. Clients conforming to the draft can expect servers to 
comply with the MUST requirement even if there is a separate 
yang-library data tree for each datastore the constraint of 
configuration stores mapping to 'operational' should be enforced 
according to the draft. There is no contradiction here.

That said I would be also be OK with ietf-datastores augmenting 
ietf-yang-library with such a leaf-list ('not-implemented-in' leaf-list) 
as a more constrained flavor of the same approach instead of going for 
independent copies of yang-library data. For any of that to happen 
change in ietf-datastores.yang is needed and change in the original 
rfc7895 ietf-yang-library is not needed at all.

Vladimir

>
> Why is it interesting to have a separate module list for regular 
> modules and imported modules?
> NMDA needs to be possible to implement with a single schema tree such 
> that a module
> is implemented in all datastores, or a subset of all datastores.  
> Otherwise it probably won't
> On Fri, Dec 8, 2017 at 9:21 AM, Kent Watsen <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     [email-masked] <mailto:[email-masked]>
>     <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_netmod&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=5qj6BQUSwqYmkAVeKz5axFV8k3gxYEPSJ5Cp0RSnxrE&s=I7fR1GY5lN2hVMkDuvryrhDeRypike3wPeFRrvQI5l8&e=>
>
>
>     [email-masked] <mailto:[email-masked]>
>     <https://www.ietf.org/mailman/listinfo/netmod>
>
>
>
>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 11 Dec 2017 07:29:36 +0100

On Fri, Dec 08, 2017 at 10:01:20AM -0800, Andy Bierman wrote:
> 
> NMDA needs to be possible to implement with a single schema tree
> such that a module is implemented in all datastores, or a subset of
> all datastores.  Otherwise it probably won't get supported in
> clients.

We believe it is. At the end, a+b = a+b+c-c. You can either use set
addition to create the schema of a datastore or you create a superset
and remove things from it. The result is the same. The key is that
configuration datastores may implement a+b and operational a+b+c (and
during transition even something different).

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Robert Wilton <[email-masked]>
Date: Mon, 11 Dec 2017 10:57:50 +0000

Hi,

> a 'not-implemented-in' leaf-list and a single module list.
The use case that this particular design doesn't work particularly well 
for is if you have a dynamic datastore that just contains a few modules 
that are not supported via the conventional datastores.

I think that there are future uses cases where the set of modules used 
for a dynamic datastore could be really quite different and separate 
from conventional configuration.  E.g. if dynamic subscribers were 
managed through a dynamic configuration datastore rather than RADIUS.

> modules and imported modules?
Several reasons:
1) It means that the list of implemented modules have a single key and 
hence any references to an implemented module are cleaner/simpler.
2) The model structure naturally more strictly enforces that only a 
single revision/version of a module is implemented.  (E.g. it prevents a 
server stating that two revisions of a module are both implemented).
3) I genuinely think that the list of implemented modules is more 
interesting to the client than the imported, but not implemented modules.

For a server, I would design it to "implement" one revision of every 
module that it uses (including those that don't contain any data nodes, 
RPCs, actions, notifications, or deviations), and then the "import-only" 
list becomes the list of modules that the server implements to satisfy 
"import-by-revision" and these are stated in the implemented schema anyway.

> get supported in clients.
All solutions accommodate this requirement.

For me, some of the interesting design questions have revolved around:
- is it better to reduce duplication in the list of modules reported at 
the cost of increased model complexity?
- does the solution extend to schema mount?
- how well does the solution cope with with configuration datastores 
that support very different sets of modules?

To a lesser extent we have also been considering how well the solution 
extends to packaging and semantic versioning, but I think that it is 
quite tricky to know who these are going to pan out. E.g. I think that 
the restriction that a given schema will only implement a single 
revision of a module will end up still holding, but I'm not sure that 
everyone has that same view point.

Thanks,
Rob

> Netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf

---

From: Robert Wilton <[email-masked]>
Date: Mon, 11 Dec 2017 11:16:23 +0000

Hi Vladimir,

>> A library per datastore sounds too complicated.
I'm slightly lost, because I thought that was exactly what you were 
proposing! ;-)

> coherent and modular and addresses the problem statement.

The issue with this is that datastore augmentations to YANG library 
would end up changing the meaning of the existing YANG library nodes.  
E.g. an old client that ignores the datastore augmentations is going to 
get a nasty surprise when the server does not behave how it expects.  
E.g. because the configuration node that it thinks should be there isn't 
there because it only supported in <operational>.

This was one of the reasons for changing YANG library.

In terms of the idea of just re-using YANG library but export a separate 
copy for each datastore I think that this has its own problems:
- I don't like the idea of returning meta-data along with configuration 
for a <get-data> on any of the configuration datastores.
- How does a client know whether the YANG library for <running> applies 
to the whole server (as it does today) vs just applies to the <running> 
datastore (as it would for an NMDA server)?
- It requires more handshaking between the client and server to get the 
schema, since a separate request would be required for each datastore 
that is supported.

So, for me, I think that the only way that this solution works, would be 
to define a new <get-server-metadata> RPC, but even then I think that it 
would make sense to combine the data together into a new YANG library 
structure.

At the end of the day, I don't think that a new YANG library is going to 
be were the real cost for supporting NMDA comes from.  I think that the 
real work is supporting <operational> independently from <running> both 
in the client and servers. But I also think that once servers start 
implementing this properly that it will simplify automation, because 
rather than a client having to guess what state a server is in, it can 
actually querey, or be notified of it, without having to write a lot of 
model specific code.

Thanks,
Rob

> datastores draft. Clients conforming to the draft can expect servers 
> to comply with the MUST requirement even if there is a separate 
> ietf-yang-library with such a leaf-list ('not-implemented-in' 
> leaf-list) as a more constrained flavor of the same approach instead 
> of going for independent copies of yang-library data. For any of that 
> to happen change in ietf-datastores.yang is needed and change in the 
> original rfc7895 ietf-yang-library is not needed at all.
>
> Vladimir
>
>>
>>     > On Fri, Dec 08, 2017 at 04:19:28PM +0100, Vladimir Vassilev wrote:
>>     > >
>>     > > Yes. The default value for yang-library-datastore leaf is
>>     ds:operational
>>     > > (the only possible one for the ds:operational datastore). This
>>     is backward
>>     > > compatible. If one needs different model for 'running', etc.
>>     then a new
>>     > > datastore identity has to be defined  and set in place of the
>>     default value.
>>     > > Then this identity can be used to read the yang-library data 
>> with
>>     > > <get-data>.
>>     > >
>>     >
>>     > Sorry, but I have to ask this: How do I obtain the schema for the
>>     > datastore (lets call it <running-library>) that reports the
>>     schema for
>>     > <running>? Is there another <running-library-library> datastore?
>>     Will
>>     > the recursion end? Perhaps it does since <running-library-library>
>>     > might have itself listed as the schema defining datastore. I guess
>>     > Lada will like these kind of meta and meta-meta datastores.
>>
>>     >
>>     > /js
>>     >

---

From: Vladimir Vassilev <[email-masked]>
Date: Mon, 11 Dec 2017 16:53:30 +0100

> I'm slightly lost, because I thought that was exactly what you were 
I propose a solution that keeps ietf-yang-library a data model of a 
single YANG context specification doing the datastore specific modeling 
in ietf-datastores model instead. The solution can be both flexible 
(independent yang-library data instances per datastore as my example was 
focused on) or constrained ('not-implemented-in' modules leaf-list). 
Here is everything that is needed:

module: ietf-datastores
     +--ro datastores-state
        +--ro datastore* [name]
        +--ro (model)?
           +--:(same-as-operational)
           +--:(constrained-to-operational)
           |  +--ro not-implemented-in*       -> 
/yanglib:module-state/module/name
           +--:(unconstrained)
              +--ro yang-library-datastore?   identityref

YANG:
...
container datastores-state {
     config false;
     list datastore {
       key name;
     }
     choice model {
         case same-as-operational {
         }
         case constrained-to-operational {
           leaf-list not-implemented-in {
             type leafref {
               path "/yanglib:module-state/yanglib:module/yanglib:name";
             }
           }
         }
         case unconstrained {
           leaf yang-library-datastore {
             type identityref {
               base ds:datastore;
           }
         }
       }
     }
   }
...

IMO ietf-yang-library as defined in rfc7895 is modular and reusable. I 
do not see why this has to be compromised.
>>
>> The fundamental point proposed is that the datastore relevant bits 
>> are kept in the ietf-datastores module instead of merging everything 
>> in a new ietf-yang-library entangled monster module. If needed 
> E.g. an old client that ignores the datastore augmentations is going 
> to get a nasty surprise when the server does not behave how it 
> expects.  E.g. because the configuration node that it thinks should be 
> there isn't there because it only supported in <operational>.
>
A well written client that finds out unsupported newer versions of 
ietf-yang-library (which is reported in the capabilities) or any of the 
NMDA modules is deployed should not do any damage. How exactly did you 
solve the problem for the bad clients by changing the structure of 
yang-library in a incompatible way is something I do not understand.
>
> In terms of the idea of just re-using YANG library but export a 
> separate copy for each datastore I think that this has its own problems:
> - I don't like the idea of returning meta-data along with 
> configuration for a <get-data> on any of the configuration datastores.
There is no meta data. The datastores contain only the config false; 
/modules-state tree. I think I explained that very clearly.
> - How does a client know whether the YANG library for <running> 
> applies to the whole server (as it does today) vs just applies to the 
> <running> datastore (as it would for an NMDA server)?
All datastore models are "case same-as-operational" for such systems.
> - It requires more handshaking between the client and server to get 
> the schema, since a separate request would be required for each 
> datastore that is supported.
Correct. Flexibility and modularity come at a price. IMO modularity is 
more important in this case. And there are solutions for the problem 
e.g. send all <get-data> requests before waiting for replies. NETCONF 
supports this.
>
> So, for me, I think that the only way that this solution works, would 
> be to define a new <get-server-metadata> RPC, but even then I think 
> that it would make sense to combine the data together into a new YANG 
> library structure.
As I said my focus is on keeping ietf-yang-library modular (single YANG 
model context). <get-server-metadata> or just adding datastore 
identities allowing the client to use <get-data> to achieve the 
retrieval of the data is of little importance to me.

>
> At the end of the day, I don't think that a new YANG library is going 
> to be were the real cost for supporting NMDA comes from.  I think that 
> the real work is supporting <operational> independently from <running> 
> both in the client and servers. But I also think that once servers 
> start implementing this properly that it will simplify automation, 
> because rather than a client having to guess what state a server is 
> in, it can actually querey, or be notified of it, without having to 
> write a lot of model specific code.
+1

Vladimir
>>> such that a module

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 11 Dec 2017 17:15:56 +0100

I do not fully understand why you think it is worth to preserve the
old YANG module library structure. Clearly, it won't be backwards
compatible since an NMDA implementation may list modules that are not
implemented in all datastores and an old client talking to a new
server will thus misunderstand what is exposed via the old YANG module
library structure. Your proposal "looks" backwards compatible but it
is not if one takes a closer look.

One can start making changes to say the conformance-type but then this
does not solve the problem since an old client does not know what a
new conformance type value means. The design team did look into the
option of keeping the old structure unchanged some weeks ago and we
finally arrived at the conclusion that it gets (i) ugly and (ii) does
not really provide backwards compatibility for a client written agains
the old YANG module library structure.

Note that with the new proposals on the table, it should be possible
to provide a backwards compatible view on YANG library for systems
that implement the exact same module set (with the exact same set of
features and deviations) on all datastores they support. But for
systems where this is not true, it seems better to use new
definitions instead of tweaking semantics.

Perhaps it helps if you can clearly phrase the objective of keeping
the old structure. What is the goal you want to achieve with that?

/js

On Mon, Dec 11, 2017 at 04:53:30PM +0100, Vladimir Vassilev wrote:
> I propose a solution that keeps ietf-yang-library a data model of a single
> YANG context specification doing the datastore specific modeling in
> ietf-datastores model instead. The solution can be both flexible
> focused on) or constrained ('not-implemented-in' modules leaf-list). Here is
> everything that is needed:
> 
> module: ietf-datastores
>     +--ro datastores-state
>        +--ro datastore* [name]
>        +--ro (model)?
> IMO ietf-yang-library as defined in rfc7895 is modular and reusable. I do
> not see why this has to be compromised.
> > > 
> ietf-yang-library (which is reported in the capabilities) or any of the NMDA
> modules is deployed should not do any damage. How exactly did you solve the
> problem for the bad clients by changing the structure of yang-library in a
> incompatible way is something I do not understand.
> > 
> Correct. Flexibility and modularity come at a price. IMO modularity is more
> important in this case. And there are solutions for the problem e.g. send
> all <get-data> requests before waiting for replies. NETCONF supports this.
> > 
> model context). <get-server-metadata> or just adding datastore identities
> allowing the client to use <get-data> to achieve the retrieval of the data
> is of little importance to me.
> 
> > 
> > > of going for independent copies of yang-library data. For any of
> > > that to happen change in ietf-datastores.yang is needed and change
> > > in the original rfc7895 ietf-yang-library is not needed at all.
> > > 
> > > Vladimir
> > > 
> > > > 
> > > > is implemented in all datastores, or a subset of all
> > > > datastores.  Otherwise it probably won't
> > > >     > > Then this identity can be used to read the yang-library
> > > > data with
> > > >     > > <get-data>.
> > > >     > >
> > > >     >

---

From: Robert Wilton <[email-masked]>
Date: Mon, 11 Dec 2017 16:37:46 +0000

On 11/12/2017 15:53, Vladimir Vassilev wrote:
> single YANG context specification doing the datastore specific 
> modeling in ietf-datastores model instead. The solution can be both 
> flexible (independent yang-library data instances per datastore as my 
> example was focused on) or constrained ('not-implemented-in' modules 
> leaf-list). Here is everything that is needed:
>
> module: ietf-datastores
>     +--ro datastores-state
>        +--ro datastore* [name]
>        +--ro (model)?

But a major version change to YANG library is happening here.

In rfc7895, if a module is implemented then it is implemented in all 
configuration datastores, and <operational> doesn't exist.

Alas, that is not what is required for NMDA, it needs different 
semantics with more granularity.  You cannot achieve this without 
changing the meaning of the existing YANG library data nodes, which will 
break clients.

>> would end up changing the meaning of the existing YANG library 
>> nodes.  E.g. an old client that ignores the datastore augmentations 
>> is going to get a nasty surprise when the server does not behave how 
>> it expects.  E.g. because the configuration node that it thinks 
>> should be there isn't there because it only supported in <operational>.
>>
> ietf-yang-library (which is reported in the capabilities) or any of 
> the NMDA modules is deployed should not do any damage. How exactly did 
> you solve the problem for the bad clients by changing the structure of 

Servers have a choice:
If a server just wants to supports pre NMDA models + RPCs then it would 
use rfc7895.
If a server only wants to support post NMDA models + RPCs + clients, 
then it can use the new YANG library (and not implement the 
modules-state container).

If a server wants to support old and new clients then there are few choices:
(1) Use configuration to control which mode the server is running in.
(2) Run separate copies of the mgmt protocols on separate port numbers.
(3) Implement the NMDA models, and then publish the schema of 
<operational>, or perhaps <running>, in the modules-state container.  
Pre NMDA clients, using <get> would probably not see all system 
information.  This approach starts to have issues if there are 
differences between what is configurable and what is in operational.

> /modules-state tree. I think I explained that very clearly.

YANG library doesn't represent regular server operational data, but 
instead it is meta-data about the server instance, which is why it is 
allowed to differ between NETCONF and RESTCONF (which I don't really like).

Yes, you explained returning /modules-state from candidate, running, 
intended, i2rs, etc.  Unfortunately I really dislike this approach 
because it feels like a backwards step where we have trying to get a 
clean split between configuration and state.   Hence defining a 
<get-server-metadata> RPC would be a better choice.

> All datastore models are "case same-as-operational" for such systems.

This isn't about how the server implements it, but instead my concern is 
about how the client interacts with it.

I appreciate that it avoids a client having to parse a new YANG library, 
but more concerning for me is that the existing YANG clients will not 
see the behavior that they expect, and rather than failing, they will 
work in an unknown degraded fashion.

> e.g. send all <get-data> requests before waiting for replies. NETCONF 

I perceive that your argument is less about modularity, and instead it 
seems to be more centered around not making any changes to the structure 
of the existing YANG library.

> As I said my focus is on keeping ietf-yang-library modular (single 
> YANG model context). <get-server-metadata> or just adding datastore 
This seems to be a complex approach, introducing new protocol semantics, 
or overloading operations, to avoid revising an existing module structure.

I don't think the new proposed YANG library structures are less modular, 
in fact, I would argue that some of the approaches are designed to make 
the structure more modular and reusable (e.g. by having named schema).

Thanks,
Rob

>> the real work is supporting <operational> independently from 
>> <running> both in the client and servers. But I also think that once 
>> servers start implementing this properly that it will simplify 
>> automation, because rather than a client having to guess what state a 
>> server is in, it can actually querey, or be notified of it, without 
>> having to write a lot of model specific code.
> +1
>
> Vladimir
>>
>> Thanks,
>> Rob

---

From: Andy Bierman <[email-masked]>
Date: Mon, 11 Dec 2017 12:55:28 -0800

On Mon, Dec 11, 2017 at 2:57 AM, Robert Wilton <[email-masked]> wrote:

> I think that there are future uses cases where the set of modules used for
> a dynamic datastore could be really quite different and separate from
> conventional configuration.  E.g. if dynamic subscribers were managed
> through a dynamic configuration datastore rather than RADIUS.
>
>

IMO you are replacing universally meaningful keys  (module-name,
revision-date) with an arbitrary name,
It is not cleaner and not simpler for a client.

2) The model structure naturally more strictly enforces that only a single
> revision/version of a module is implemented.  (E.g. it prevents a server
> stating that two revisions of a module are both implemented).
>

How is that the case if the schema list includes its own module list?
You mean there is a "unique" statement in the outer list that insures that
a module/revision
shows up at most once in all instances of the inner module list?

> interesting to the client than the imported, but not implemented modules.

The conformance leaf was good enough.
Duplicating the module list and removing the conformance leaf is
aggressively non-backward compatible.

> All solutions accommodate this requirement.

Seems to me all new solutions allow a server to violate the MUST in the
NMDA draft that
there is a superset of all modules.  A client has to look for every module
in a server-specific
set of named schema sets, and then reconcile all these sets.
I still prefer the single module list with a conformance leaf and a
leaf-list indicating
the supported (or unsupported) datastores.

> - how well does the solution cope with with configuration datastores that
> support very different sets of modules?
>
> extends to packaging and semantic versioning, but I think that it is quite
> tricky to know who these are going to pan out.  E.g. I think that the
> restriction that a given schema will only implement a single revision of a
> module will end up still holding, but I'm not sure that everyone has that
> same view point.
>
> Thanks,
> Rob
>
>
>

Andy

>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.iet
>> f.org_mailman_listinfo_netmod&d=DwICAg&c=HAkYuh63rsuhr6Scbfh
>> 0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTv
>> jISlaJdcZo&m=5qj6BQUSwqYmkAVeKz5axFV8k3gxYEPSJ5Cp0RSnxrE&s=I
>> 7fR1GY5lN2hVMkDuvryrhDeRypike3wPeFRrvQI5l8&e=
>>
>>
> Netconf mailing [email-masked]://www.ietf.org/mailman/listinfo/netconf
>
>
>

---

From: Vladimir Vassilev <[email-masked]>
Date: Tue, 12 Dec 2017 11:37:58 +0100

> the old structure. What is the goal you want to achieve with that?
I prefer to call it the current structure for now. I will try to 
explain. My goal is to efficiently build transactional systems based on 
YANG models and open standards.

For this I need some reusable bricks. Such a brick is the rfc7895 
ietf-yang-libraray module. Even in a system with multiple YANG model 
contexts eventually processing breaks down to a single model context 
e.g. for example a YANG data validation task performed in a command line 
application comes down to:
  $ yang-data-validator yang-library-data.xml data.xml

The command can be called multiple times for different data and 
different contexts but it is the same very well tested simple tool. So 
my point is why can we not use one of the 2 newly introduced methods for 
instantiating this very well designed model for the needs of NMDA.

1. Use datastore for instantiation.
2. Use schema mount to mount rfc 7895 ietf-yang-library in the 
ietf-datastores list. (an alternative I assume is especially applicable 
to modules like ietf-yang-library that are self-contained)

In general having a solution that is flexible can easily be trimmed 
down. Adding special cases that are less flexible e.g. the 
(not-implemented-in leaf-list or none at all for the case of fully 
transactional systems with a single model context (no extra work for 
these)). The advantage is that even with the not-implemented-in 
leaf-list solution or other diff based the data processing tasks will 
eventually convert the differential information to a data structure 
implementing the rfc7895 ietf-yang-library data tree and the 
yang-data-validator application will be backward compatible and 
reusable. With changes that introduce a whole new level of abstraction 
in place of the simple list with module name and revision as keys one 
can't even reuse the groupings defined in that module and that is a 
problem I try to avoid.

Vladimir

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 12 Dec 2017 11:46:17 +0100

On Tue, Dec 12, 2017 at 11:37:58AM +0100, Vladimir Vassilev wrote:
> I prefer to call it the current structure for now. I will try to explain. My
> goal is to efficiently build transactional systems based on YANG models and
> open standards.
> 
> contexts eventually processing breaks down to a single model context e.g.
> for example a YANG data validation task performed in a command line

I think you can implement

$ yang-data-validator yang-library-data-bis.xml data.xml

and it will work for any datastore data in data.xml with and without
schema mount.

> The command can be called multiple times for different data and different
> contexts but it is the same very well tested simple tool.

Looks like you do not want to make changes but then you have to make
changes. I am not sure how to address such a requirement.

> So my point is why
> can we not use one of the 2 newly introduced methods for instantiating this
> very well designed model for the needs of NMDA.
> 
> ietf-datastores list. (an alternative I assume is especially applicable to
> modules like ietf-yang-library that are self-contained)
> 
> In general having a solution that is flexible can easily be trimmed down.
> Adding special cases that are less flexible e.g. the (not-implemented-in
> leaf-list or none at all for the case of fully transactional systems with a
> single model context (no extra work for these)). The advantage is that even
> with the not-implemented-in leaf-list solution or other diff based the data
> processing tasks will eventually convert the differential information to a
> data structure implementing the rfc7895 ietf-yang-library data tree and the
> yang-data-validator application will be backward compatible and reusable.
> With changes that introduce a whole new level of abstraction in place of the
> simple list with module name and revision as keys one can't even reuse the
> groupings defined in that module and that is a problem I try to avoid.

I think you can easily express the ietf-yang-library model in
ietf-yang-library-bis so moving to ietf-yang-library-bis will
give you a clean solution.

---

From: Vladimir Vassilev <[email-masked]>
Date: Wed, 13 Dec 2017 08:19:02 +0100

On 12/12/2017 08:20 PM, Martin Bjorklund wrote:

> Hi,
>
> Vladimir Vassilev <[email-masked]> wrote:
>>
>> On 12/08/2017 04:06 PM, Juergen Schoenwaelder wrote:
>>> On Fri, Dec 08, 2017 at 04:03:06PM +0100, Martin Bjorklund wrote:
>>>> Vladimir Vassilev <[email-masked]> wrote:
>>>>> On 11/15/2017 06:29 PM, Robert Wilton wrote:
>>>>>
>>>>>> I don't think that this is really a good idea.  You would end up
>>>>>> returning server metadata in addition to the configuration.
>>>>> Obviously RFC 7895 defines only config false; data and I was not
>>>>> proposing a change to that. But I agree something has to be added to
>>>>> complete the solution. Special purpose datastore identities can be
>>>>> defined that return instance of yang-library data when read with
>>>>> <get-data>. (Datastores with yang-library config false; only data not
>>>>> represented in 'operational')
>>>>>
>>>>> Adding this special yang-library-datastore to the proposed
>>>>> ietf-datastores container e.g.
>>>>>
>>>>> module: ietf-datastores
>>>>> +--ro datastores
>>>>> |  +--ro datastore* [name]
>>>>> |     +--ro name          identityref
>>>>> |     +--ro yang-library-datastore          identityref
>>>>>
>>>> I don't understand this proposal.  How would a client learn the
>>>> library for <running>?  For <operational>?
>>> My interpretation is that the client reads the datastores list from
>>> <operational> and the list entries give you the identity of a separate
>>> datastore that gives you the content of the yang library for that
>>> datastore. (For each datastore, you have a separate datastore to
>>> report its yang library.)
>> Yes. The default value for yang-library-datastore leaf is
>> ds:operational (the only possible one for the ds:operational
>> datastore). This is backward compatible. If one needs different model
>> for 'running', etc. then a new datastore identity has to be defined
>> and set in place of the default value. Then this identity can be used
>> to read the yang-library data with <get-data>.
> Ah, ok.  This is a clever solution, but quite complicated.  It
> requires several round trips for a client to learn all library
> instances.  Also, w/o any changes, it is not clear which module-set-id
> is sent in the capability, and a client must query all module-set-ids
> in all (meta)datastores in order to just check if it has the latest
> version or not.  It is also not clear how the existing notification
> "yang-library-change" would work when there are multiple instances
> involved.
How about this?
module: ietf-datastores
...
   augment /yanglib:yang-library-change:
     +---- datastore?   identityref

Vladimir
>    So I don't think that this solution will actually work w/o
> an update to 7895 - but not updating 7895 was the whole reason for
> doing this.
>
>
> /martin
>

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 13 Dec 2017 08:40:30 +0100 (CET)

>     +---- datastore?   identityref

This has the same issue as augementing a datastore leaf-list into the
current /modules-state/modules list would have - old clients won't
understand this new node.

/martin

---

From: Vladimir Vassilev <[email-masked]>
Date: Wed, 13 Dec 2017 08:50:33 +0100

On 12/13/2017 08:19 AM, Vladimir Vassilev wrote:
>> is sent in the capability,
The module-set-id sent in capabilities has to be the one for operational 
since the client needs to get started by reading /datastores-state.
>> and a client must query all module-set-ids
>> version or not.
Adding a /datastores-state/datastore/module-set-id leaf in operational 
can resolve this problem.

At the current point I think all issues raised are addressed with the 
following model (notice the added anydata option which can replace the 
use of yang-library-datastore and if implemented as an alternative can 
make retrieval as a single tree):

module: ietf-datastores
     +--ro datastores-state
        +--ro datastore* [name]
        +--ro module-set-id             string
        +--ro (model)?
           +--:(same-as-operational)
           +--:(constrained-to-operational)
           |  +--ro not-implemented* [name revision]
           |     +--ro name        -> /yanglib:module-state/module/name
           |     +--ro revision    -> /yanglib:module-state/module/revision
           +--:(unconstrained-datastore-instance)
           |  +--ro yang-library-datastore    identityref
           +--:(unconstrained-anydata)
              +--ro yang-library?             <anydata>

Vladimir

>>   It is also not clear how the existing notification

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 13 Dec 2017 09:29:39 +0100

On Wed, Dec 13, 2017 at 08:50:33AM +0100, Vladimir Vassilev wrote:
> following model (notice the added anydata option which can replace the use
> of yang-library-datastore and if implemented as an alternative can make
> retrieval as a single tree):
> 
> module: ietf-datastores
>     +--ro datastores-state
>        +--ro datastore* [name]

It is (a) architecturally backwards to have ietf-datastores depend on
ietf-yang-library and (b) the proposal does not solve the issue that
you are silently changing the semantics of the definitions in the old
ietf-yang-library. It remains unclear to me which problem this is
approach solving, i.e., what the benfit is over the other proposals
(since the semantics of ietf-yang-library change, it is _not_
backwards compatible).

/js

---

From: Vladimir Vassilev <[email-masked]>
Date: Wed, 13 Dec 2017 09:56:30 +0100

> understand this new node.
IMO it is different. Augmenting the data tree under /modules-state can 
potentially compromise the bootstrap resolution of the model context 
(depending on whether ignoring the data with unknown namespaces does 
change the original model semantics). However I do not see any issue 
with the case of the proposed augmentation to yang-library-change 
notification. Correctly implemented client needs to read /modules-state 
and make sure there are no unsupported modules,features etc. (e.g. 
ietf-datastores is supported) before going further (e.g. subscribing to 
notifications editing configuration etc.).

---

From: Robert Wilton <[email-masked]>
Date: Wed, 13 Dec 2017 14:55:30 +0000

On 11/12/2017 20:55, Andy Bierman wrote:
>
>
> On Mon, Dec 11, 2017 at 2:57 AM, Robert Wilton <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     Hi,
>
>
>     The use case that this particular design doesn't work particularly
>     well for is if you have a dynamic datastore that just contains a
>     few modules that are not supported via the conventional datastores.
>
>     I think that there are future uses cases where the set of modules
>     used for a dynamic datastore could be really quite different and
>     separate from conventional configuration.  E.g. if dynamic
>     subscribers were managed through a dynamic configuration datastore
>     rather than RADIUS.
>
>>
>     1) It means that the list of implemented modules have a single key
>     and hence any references to an implemented module are cleaner/simpler.
>
>
>
No, for alternatives A and B, the list key is the module name itself, 
nor an arbitrary name.
Alternative C uses an arbitrary name.

>
>
>     2) The model structure naturally more strictly enforces that only
>     a single revision/version of a module is implemented. (E.g. it
>     prevents a server stating that two revisions of a module are both
>     implemented).
>
>
>
> You mean there is a "unique" statement in the outer list that insures 
> that a module/revision
For alt A, each schema represents a single list (so no duplicated 
implemented modules are possible).

For alt B, yes, the rule is that there must be no duplicates in the 
module-sets that are combined into a single schema.

>     For a server, I would design it to "implement" one revision of
>     every module that it uses (including those that don't contain any
>     data nodes, RPCs, actions, notifications, or deviations), and then
>     the "import-only" list becomes the list of modules that the server
>     implements to satisfy "import-by-revision" and these are stated in
>     the implemented schema anyway.
>
>
> Seems to me all new solutions allow a server to violate the MUST in 
> the NMDA draft that
> there is a superset of all modules.  A client has to look for every 
> module in a server-specific
So, this is a trade off between a more expressive model vs a more 
constrained model.

It is worth noting that the existing YANG library (RFC 7895) allows 
servers to produce illegal module lists because they could implement 
multiple revisions of the same module.

Thanks,
Rob

>     - is it better to reduce duplication in the list of modules
>     reported at the cost of increased model complexity?
>     - how well does the solution cope with with configuration
>     datastores that support very different sets of modules?
>
>     To a lesser extent we have also been considering how well the
>     solution extends to packaging and semantic versioning, but I think
>     that it is quite tricky to know who these are going to pan out. 
>     E.g. I think that the restriction that a given schema will only
>     implement a single revision of a module will end up still holding,
>     but I'm not sure that everyone has that same view point.
>
>     Thanks,
>     Rob
>
>
>
>
> Andy
>
>>
>>
>>     Andy
>>
>>
>>
>>     [email-masked] <mailto:[email-masked]>
>>     <https://www.ietf.org/mailman/listinfo/netconf>
>
>

---

From: Andy Bierman <[email-masked]>
Date: Wed, 20 Dec 2017 12:16:44 -0800

On Wed, Dec 13, 2017 at 6:55 AM, Robert Wilton <[email-masked]> wrote:

>> for a dynamic datastore could be really quite different and separate from
> No, for alternatives A and B, the list key is the module name itself, nor
> an arbitrary name.

The alt-B approach that seems to be favored is not constrained, but even if
the server does conform
(and not have any overlaps between schema lists) this approach is rather
complicated in order
to answer the simple question "What datastores are supported for module
foo, revision A?"

If the previous design was used, a client could simply retrieve the module
entry and check
the 'not-implemented-in' leaf-list.  Now a client has to retrieve the
entire library, then analyze the
datastore lists (built from the schema entries for the datastore). The
client has to match
instances and look for module "foo" in multiple places.  The client then
assumes that
any missing entry means the module is not supported in that datastore
(assumes read access control
is never applied to the YANG library).

The corner case used as an excuse to redesign the library (ephemeral
datastore with a few modules in it)
is not very interesting, and also not expensive in terms of processing
complexity and bandwidth.
IMO the library should be optimized for conventional + operational, and
also be optimized for
the long-term, not a short-term transition period, biased for server
developers who want
to release partial implementations.

Andy