Subject: [Netconf] Decision on the Intended Status of the Revised DS Draft WAS:RE: :candidate, :writable-running and RESTCONF edits

From: "Mehmet Ersue" <[email-masked]>
Date: Fri, 6 Jan 2017 19:04:14 +0100

Hi All,

[NETMOD CCed as this is relevant to the revised DS draft]

> > > > I wrote it already several times: I support(ed) Mehmet's proposal to make the revised-datastores I-D informative. The document was adopted as a Standard Track WG item although I didn't see anything close to rough consensus during the adoption poll.
> > > >
> > > It appears to me that there is consensus to making this a standards track solution.
> > >
> > Where is any evidence of this?
>
> I think there was approval in the room in Seoul, and it is being confirmed on the mailing list.

It is correct that we agreed in Seoul to start a consensus call on both maillists for the adoption of the DT draft and executed such an adoption call.

However we did not have any decision on the intended draft status in Seoul and did not have any agreement during the adoption call. I explained my concerns on the currently indicated status being standard track and did not hear any other argument.

It is correct that we need a standard track document for the new DS framework - to provide a basis for other RFCs to develope.  However the current DT solution draft has not been prepared as a standard track document nor it has standard relevant content. Such concept description is usually prepared as an architecture document (see example in  <https://tools.ietf.org/html/rfc6244> RFC 6244).

As I stated earlier I believe “a new protocol- and language-independent standard document” should be prepared defining the generic datastore framework (based on and following the concept in the DT solution draft).

IMO the intended status of the DS draft is still open and subject to decide.

Regards,

Mehmet

From: Netconf [mailto:[email-masked]] On Behalf Of Andy Bierman
Sent: Wednesday, December 28, 2016 6:37 PM
To: Ladislav Lhotka <[email-masked]>
Cc: Netconf <[email-masked]>
Subject: Re: [Netconf] :candidate, :writable-running and RESTCONF edits

On Wed, Dec 28, 2016 at 1:06 AM, Ladislav Lhotka <[email-masked] <mailto:[email-masked]> > wrote:

> On 27 Dec 2016, at 19:34, Andy Bierman <[email-masked] <mailto:[email-masked]> > wrote:
>
>
>
> On Tue, Dec 27, 2016 at 10:01 AM, Ladislav Lhotka <[email-masked] <mailto:[email-masked]> > wrote:
>
> > On 27 Dec 2016, at 16:37, Juergen Schoenwaelder <[email-masked] <mailto:[email-masked]> > wrote:
> >
> > On Tue, Dec 27, 2016 at 03:47:10PM +0100, Ladislav Lhotka wrote:
> >>
> >>>
> >>> A remote API for management applications was our original target and
> >>> this is where YANG is doing well (and we apparently still have work to
> >>> do to improve things).
> >>
> >> By "management application" I mean a specific implementation with a fixed set of datastores and precise semantics. I do argue that neither RFC 6241, nor "revised-datastores" nor any other *specific* setup of datastores is going to work for all use cases, yet the protocols and YANG can be pretty universal.
> >>
> >> We have already seen YANG being used in areas that are, strictly speaking, not supported by 6020/7950. Rather than tolerating such uses (which is a slippery slope), I would prefer to make them - within reasonable limits - officially supported.
> >>
> >
> > This is too abstract for me. What do you suggest to change in the
> > revised datastore architecture? I do believe that having a number of
> > well defined datastores with specific semantics is necessary for
> > having interoperability.
>
> I wrote it already several times: I support(ed) Mehmet's proposal to make the revised-datastores I-D informative. The document was adopted as a Standard Track WG item although I didn't see anything close to rough consensus during the adoption poll.
>
>
>
> It appears to me that there is consensus to making this a standards track solution.

Where is any evidence of this?

I think there was approval in the room in Seoul, and it is being confirmed on the mailing list.

>
> Datastores allow universal concepts about managed data to be standardized
> across multiple protocols.  In a protocol, the datastore name serves as
> an input parameter value.  It needs to be normative in order for a
> standards-track protocol to use it.

For the protocol, the RPC parameter needs to be normative, not a particular datastore name or semantics.

>
> Changing standards track status never addresses real technical problems.
> I do not understand your objections or alternate proposals.

I think this proposal will lead to rather significant changes to the protocols and YANG, and their extent IMO isn't clear yet. For one, the draft aims at moving validation from "running" to "intended". But if "intended" is optional to implement (sec. 6.1), I don't get how validation in YANG can be (re)defined to apply also to implementations that don't have "intended".

It should be possible to add new operations to NETCONF and

new query parameters to RESTCONF that support the operator requirements,

and still preserve existing operations.

I am not in favor of rewriting operations that move the validation to 'intended'.

That would not be backward-compatible so it is a non-starter.

What I would like to do is to make protocols and YANG work equally well for all datastore architectures so that the protocols and YANG needn't be changed each time the current architecture is found to require adjustments.

This does not make sense to me.

First, I do not care about non-standard, private architectures, just the

standard architecture.  Applications cannot be forced to hard-wire all the

details about configuration management.  Let's not go back to data silo architecture.

The standards need to expose the details in a common framework to enable

data-driven automation tools.  That means the protocols and the data models

have to support a standard datastore framework.

>
> My Applicability Statement concerns are related to developers deciding
> if a server implementation needs to expose 'intended' and 'applied'.
> I would like a definitive statement like "If an intended value takes more than 5 seconds
> to become applied (due to implementation, not missing hardware), then
> the 'intended' and 'applied' datastores SHOULD be supported."
>
> I think the definitions of the datastores apply to all servers, and the
> set of new datastores is the correct and the semantics are clear.

For some servers running-intended-applied is way too complicated, and good old "running" may be all that's needed.

Agreed.  These servers do not need to implement the new datastores.

The same is true for the 'candidate' and 'startup' datastores already.

Lada

Andy

>
>
> Lada
>
> Andy
>
>
> >
> > /js
> >
> > --
> > Juergen Schoenwaelder           Jacobs University Bremen gGmbH
> > Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
> > Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>
>
> --
> Ladislav Lhotka, CZ.NIC Labs
> PGP Key ID: 0xB8F92B08A9F76C67

--
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: 0xB8F92B08A9F76C67

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 6 Jan 2017 19:45:13 +0100

> As I stated earlier I believe “a new protocol- and language-independent standard document” should be prepared defining the generic datastore framework (based on and following the concept in the DT…

To me, this sounds like duplicate work for no real technical value. If
the existance of two WG results into actions like this, we should
seriously consider the option to merge NETMOD and NETCONF into one WG.

/js

---

From: "Mehmet Ersue" <[email-masked]>
Date: Fri, 6 Jan 2017 20:05:17 +0100

Hi Juergen,

I don't think it is duplicate work. One is as I understand the architecture and concept document you were asking for 
and the other draft is the standard DS framework RFC to be used as the basis for different documents.

Cheers,
Mehmet

-----Original Message-----
From: Juergen Schoenwaelder [mailto:[email-masked]] 
Sent: Friday, January 6, 2017 7:45 PM
To: Mehmet Ersue <[email-masked]>
Cc: 'Netconf' <[email-masked]>; 'NetMod WG' <[email-masked]>
Subject: Re: [Netconf] Decision on the Intended Status of the Revised DS Draft WAS:RE: :candidate, :writable-running and RESTCONF edits

> As I stated earlier I believe “a new protocol- and language-independent standard document” should be prepared defining the generic datastore framework (based on and following the concept in the DT…

To me, this sounds like duplicate work for no real technical value. If the existance of two WG results into actions like this, we should seriously consider the option to merge NETMOD and NETCONF into one WG.

/js

---

From: Andy Bierman <[email-masked]>
Date: Fri, 6 Jan 2017 11:57:44 -0800

On Fri, Jan 6, 2017 at 11:05 AM, Mehmet Ersue <[email-masked]> wrote:

> Hi Juergen,
>
> I don't think it is duplicate work. One is as I understand the
> architecture and concept document you were asking for
I do not understand the difference between these 2 documents.

There should be 1 document that is protocol-independent that includes:
   - definition of datastores
   - discussion of interactions between datastores
   - use-cases in scope for new datastores
   - applicability guidance for server developers

Each protocol that wants to use these new datastores needs to define
mechanisms to do that in a separate document.

Cheers,

Andy

> Subject: Re: [Netconf] Decision on the Intended Status of the Revised DS
> Draft WAS:RE: :candidate, :writable-running and RESTCONF edits
>
> > It is correct that we need a standard track document for the new DS
> framework - to provide a basis for other RFCs to develope.  However the
> current DT solution draft has not been prepared as a standard track
> document nor it has standard relevant content. Such concept description is
> usually prepared as an architecture document (see example in  <
> https://tools.ietf.org/html/rfc6244> RFC 6244).
> >
> > As I stated earlier I believe “a new protocol- and language-independent
> standard document” should be prepared defining the generic datastore
> framework (based on and following the concept in the DT solution draft).
> >
>
> To me, this sounds like duplicate work for no real technical value. If the
> existance of two WG results into actions like this, we should seriously
> consider the option to merge NETMOD and NETCONF into one WG.
>
> /js
>
> --
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod
>

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 9 Jan 2017 13:24:29 +0100

> On 6 Jan 2017, at 20:57, Andy Bierman <[email-masked]> wrote:
> 
> 
> 

The current document involves quite a lot of hand-waving, and that's why I was also reluctant to accept it as a WG standard-track deliverable. I don't care that much about the number of documents that depend on it because any mistakes may be pretty expensive here. I pointed out some gaps in my previous review, e.g. it talks about templates without defining what they are. If running contains templates, does it mean it needn't be valid according to the data model?

> mechanisms to do that in a separate document.

That's one part, the other are changes inflicted to YANG. I think the best way would be to make YANG independent of a particular setup of datastores and their semantics. Then I2RS or anybody else can do whatever they need without affecting YANG spec any more.

I believe this wouldn't be a terribly difficult thing to do, but Juergen wants me to write a meta-model document first.

Lada

---

From: Lou Berger <[email-masked]>
Date: Mon, 09 Jan 2017 07:38:11 -0500

On January 9, 2017 7:25:24 AM Ladislav Lhotka <[email-masked]> wrote:
>
> The current document involves quite a lot of hand-waving, and that's why I 
> was also reluctant to accept it as a WG standard-track deliverable.

IMO I think we should do and document the work and then, once the is 
general agreement,  worry about number and publication status of documents.

Lou

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 9 Jan 2017 13:50:45 +0100

> On 9 Jan 2017, at 13:38, Lou Berger <[email-masked]> wrote:
> 
> 
> 
>> The current document involves quite a lot of hand-waving, and that's why I was also reluctant to accept it as a WG standard-track deliverable.
> 
> IMO I think we should do and document the work and then, once the is general agreement,  worry about number and publication status of documents.

I agree, but we have to accept it will take some time. The problem I see is that quite a few people already work with the solution.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Mon, 9 Jan 2017 10:37:16 -0800

On Mon, Jan 9, 2017 at 4:50 AM, Ladislav Lhotka <[email-masked]> wrote:

> >> The current document involves quite a lot of hand-waving, and that's
> why I was also reluctant to accept it as a WG standard-track deliverable.
> >
>

I do not agree with this hand-waving assessment.
What are the show-shoppers?

I agree the draft is light on use-cases.
There are no standard mechanisms that cause <running> to be
different from <intended>, so I would agree the intended datastore
needs a lot more standards support before it is useful.

> I agree, but we have to accept it will take some time. The problem I see
> is that quite a few people already work with the solution.
>
>
The solutions I've seen were not very good, so they need a lot of work.
I expect that the operational datastore will be more widely implemented
than any of the others.  Designing a <get> replacement is not that
difficult.

Lada

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 9 Jan 2017 21:18:46 +0100

> On 9 Jan 2017, at 19:37, Andy Bierman <[email-masked]> wrote:
> 
> 
> 

I am more concerned about use cases that are not known so far, and so I am against standardizing this (or any other) workflow as the only one supported by NETCONF/RESTCONF and YANG. I believe both the protocols and YANG can work with any set of datastores, but their choice depends on the use case at hand. Why should IoT developers be exposed to the running-intended-applied complexity, even if they are allowed to lump all three into one?   

> needs a lot more standards support before it is useful.

The only difference seems to be the presence of templates but I don't know what they are.

Lada

> than any of the others.  Designing a <get> replacement is not that difficult.
> 
> 
> 
> Lada
> 
> Andy
>  
> 
> >
> > Lou
> >
> >
> 
> --

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 9 Jan 2017 21:51:11 +0100

> I am more concerned about use cases that are not known so far, and so I am against standardizing this (or any other) workflow as the only one supported by NETCONF/RESTCONF and YANG. I believe both…

Please point me to the statement in the I-D that makes you believe an
implementation is required to support all datastores.

> The only difference seems to be the presence of templates but I don't know what they are.

The I-D says:

                            |        // e.g., removal of 'inactive'
                            |        // nodes, expansion of templates

So it is not just templates. And yes, these are things several
real-world implementations can do but where the IETF did not yet
managed to standardize anything. The basic question is whether we want
a model that is (a) capable to match real-world implementation and
that allows for future standards of existing proprietary technology or
(b) we go with what we have today (either chartered or published) and
we keep revising the model as we move ahead.

/js

---

From: Andy Bierman <[email-masked]>
Date: Mon, 9 Jan 2017 13:17:09 -0800

On Mon, Jan 9, 2017 at 12:51 PM, Juergen Schoenwaelder <
[email-masked]> wrote:

> > I am more concerned about use cases that are not known so far, and so I
> am against standardizing this (or any other) workflow as the only one
> supported by NETCONF/RESTCONF and YANG. I believe both the protocols and
> YANG can work with any set of datastores, but their choice depends on the
> use case at hand. Why should IoT developers be exposed to the
> running-intended-applied complexity, even if they are allowed to lump all
> three into one?
> >
>

I think itt is not realistic to say that datastores are optional.

e.g. <enabled> leaf:  If there is a standard way to enable/disable config
then individual "enabled" leafs are redundant. However XPath (must/when)
has no way to describe if the subtree is enabled (which is a show-stopper)

<foo-config> vs <foo-oper>.  If the applied or operational datastore is
assumed,
then there is no need to model the redundant config-as-operstate.
If this is left out of the model, then the datastore becomes mandatory.
If it is left in the model, the datasore becomes redundant.

The basic premise that these datastores are optional is flawed.
One cannot design a YANG module assuming the datastores are present
if they are in fact optional.

/js
Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 10 Jan 2017 08:21:03 +0100

> has no way to describe if the subtree is enabled (which is a show-stopper)

I may not understand what you are saying. From what I know, there are
implementations that allow to 'comment out' nodes and subtrees and
that work with clients in a backwards compatible way.
 
> if they are in fact optional.

The claim that all datastores are mandatory is equally flawed.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 10 Jan 2017 09:20:36 +0100

> On 9 Jan 2017, at 21:51, Juergen Schoenwaelder <[email-masked]> wrote:
> 

Sec. 6.3: "YANG currently describes validation in terms of the <running> configuration datastore while it really happens on the <intended> configuration datastore."

This seems to indicate that <running> and <intended> are required, at least conceptually.

The original datastore model (sec. 4) was hardwired into NETCONF and YANG specs but it was found insufficient for some use cases. So now we come up with another datastore model, and I am against doing the same mistake again. What if some future use cases again need something else?

We could instead define the protocols and YANG in terms of reasonable abstractions (e.g. resource and data tree), and use a run-time specification of the datastore model (there could also be several standardized alternatives). To some extent, we do it already now by using capabilites such as "candidate", "startup" and "writable-running".

> we keep revising the model as we move ahead.

I think we need protocol and YANG specs that are not tied to any particular model and that are thus capable of matching unforeseen real-world implementations. This is no sci-fi, HTTP and XML schema languages work this way.

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 10 Jan 2017 09:39:43 +0100

> I think we need protocol and YANG specs that are not tied to any particular model and that are thus capable of matching unforeseen real-world implementations. This is no sci-fi, HTTP and XML schema…

I disagree that HTTP and XML schema languages do the same thing. Our
goal is interoperable configuration of network devices; the notion of
configuration datastore validation and specification of semantics goes
well beyond what you usually find in web interactions, where typically
all semantics are left to the implementor of a service. We validate
configurations (residing in datastores), not the syntactic aspects of
protocol messages. But yes, I am looking forward to a concrete I-D.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 10 Jan 2017 10:20:35 +0100

> On 10 Jan 2017, at 09:39, Juergen Schoenwaelder <[email-masked]> wrote:
> 

Even now, a client that's programmed to write straight to running isn't interoperable with a server that has candidate and read-only running. A RESTCONF server that supports only JSON isn't interoperable with a client that supports only XML.

We are not in a situation that every pair of a randomly chosen server and client need to be interoperable. It's IMO perfectly fine if IoT and ISP networks use different clients. Yet, both can still use the same RESTCONF, same YANG, and even same YANG modules. 

> protocol messages. But yes, I am looking forward to a concrete I-D.

Where did I write anything about syntactic aspects of protocol messages? Of course we need to validate datastores. In the case of YANG, my point is that its spec could just define what a valid datastore (data tree) is. What it needn't state is that running, intended or whatever MUST be valid - it can be said elsewhere.

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 10 Jan 2017 10:34:21 +0100

> We are not in a situation that every pair of a randomly chosen server and client need to be interoperable. It's IMO perfectly fine if IoT and ISP networks use different clients. Yet, both can still…

Yes, there are optional protocol features and the protocols support
negotiations of protocol features. Not a big deal. HTTP is full of
this as well.

The point is that writing to candidate has a well defined meaning
because we defined how candidate behaves and interacts with the other
datastores. If a client commits candidate, it knows what kind of
changes it can expect to see in other datastores. If you want to do
away with well-known datastores, you (a) either provide all the
semantics we currently have hard-wired into well-known datastores as
meta-data at runting (and you require that every client is able to
interpret this meta-information correctly) or (b) you significantly
loose functionality and we end up with generic editing protocols that
unfortunately do not help much with configuration management (since we
lost how information in differnet datastores interacts).

In case I completely mis-understand you, consider to write an I-D.

/js

---

From: Robert Wilton <[email-masked]>
Date: Tue, 10 Jan 2017 14:31:56 +0000

Hi,

On 09/01/2017 21:17, Andy Bierman wrote:
>
>
> On Mon, Jan 9, 2017 at 12:51 PM, Juergen Schoenwaelder 
> <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     > I am more concerned about use cases that are not known so far,
>     and so I am against standardizing this (or any other) workflow as
>     the only one supported by NETCONF/RESTCONF and YANG. I believe
>     both the protocols and YANG can work with any set of datastores,
>     but their choice depends on the use case at hand. Why should IoT
>     developers be exposed to the running-intended-applied complexity,
>     even if they are allowed to lump all three into one?
>     >
>
> <foo-config> vs <foo-oper>.  If the applied or operational datastore 
> is assumed,

I agree that these datastores are not all really optional.

Current YANG models are structured assuming that there is no operational 
state datastore.  They could be used on devices that support an 
operational state datastore, but there would likely be duplication of 
some of the data because it is represented in two places.

But to design clean models with combined config and state, it is 
necessary to assume support for both "running" and also "operational 
state" datatstores.  Such models could theoretically be used by devices 
that don't support the operational state datastore, but key information 
would be unavailable, so this doesn't feel like a pragmatic solution.  
However, it would to straight forward to write tooling (i.e. a plugin to 
pyang) that takes a YANG module designed assuming an operational state 
datastore and generate the missing config false leaves so that the model 
could also be used on devices that don't support an operational state 
datastore.

This raises the question, or how do you know whether a model has been 
designed assuming support for an operational state datastore? For me the 
simplest solution appears to be to mark the models as YANG version 2.0.  
I would also propose revising NETCONF and RESTCONF to version 2, 
allowing the protocols to change to support an operational state 
datastore in a clean way.  E.g. if a device supports an operational 
state datastore then it doesn't really make sense to continue to support 
the existing NETCONF GET operation.

Thanks,
Rob

>     Fax:   +49 421 200 3103         <http://www.jacobs-university.de/
>     <http://www.jacobs-university.de/>>
>
>
>
>

---

From: Robert Wilton <[email-masked]>
Date: Tue, 10 Jan 2017 14:43:15 +0000

> general agreement,  worry about number and publication status of 
+1

I would like to get on with the interesting work of actually working out 
the solution.  Personally, I think that having it in one document makes 
it easier to review/discuss/etc.  I also think that it would help this 
work progress more quickly which I still regard as being particularly 
important.

I do also agree with Mehmet's suggestion of restructuring the existing 
YANG docs to make them more consistent, but I think that it would be a 
mistake to slow down this important work to what seems like a 
documentation cleanup task.

Rob

> .

---

From: Andy Bierman <[email-masked]>
Date: Tue, 10 Jan 2017 07:30:51 -0800

> The claim that all datastores are mandatory is equally flawed.
correct -- nobody is saying that.

The reason this is different is that the YANG objects are impacted.
Candidate vs. running has no impact whatsoever on the set of
YANG modules.  The protocol is not self-selecting some objects
and making other objects invisible.

But if I want to model <foo-state>, I will soon have to decide
to use <foo-state> and allow all protocols to read it or
model get-state(foo) and require a different module for each
protocol.

/js

Andy

---

From: Robert Wilton <[email-masked]>
Date: Tue, 10 Jan 2017 16:11:21 +0000

On 10/01/2017 15:30, Andy Bierman wrote:
>
>
> On Mon, Jan 9, 2017 at 11:21 PM, Juergen Schoenwaelder 

The same module can be used for the set of protocols that are 
conceptually built around the same set of mandatory datastores. I.e. it 
isn't necessary to have a separate module for each protocol, and that 
hasn't been proposed.

Rob

> Netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 10 Jan 2017 17:16:43 +0100

> correct -- nobody is saying that.

Well, I originally commented on the statement that intened would be
required and adding complexity - it does not.
 
> and making other objects invisible.

Yes. And the same is true for intended as long as an implementation
does not support templates or inactive configuration objects.

> model get-state(foo) and require a different module for each

If you do /foo and /foo-state, things will just work with or without
an operational state datastore. If you have only /foo, then an
operational state datastore may come in handy if you have to support
config and state with different lifetimes.

/js

---

From: Robert Wilton <[email-masked]>
Date: Tue, 10 Jan 2017 17:07:54 +0000

On 10/01/2017 16:16, Juergen Schoenwaelder wrote:
> an operational state datastore.
True, but there would also be an undesirable duplication of data in the 
data tree.

>   If you have only /foo, then an
I think that this may be more than "come in handy".  I think that there 
would be key information that clients would expect to be available in a 
model but wouldn't be easily retrievable without supporting the 
operational state datastore.  Specifically you lose the ability to 
easily query an operational property of a system that can be configured, 
but hasn't been configured.

Example: Consider an Ethernet interface speed leaf that in the running 
ds represents the configured speed, and in the operational state ds 
represents the actual operational speed in use.  Normally, the 
operational speed would default to the maximum speed supported by the 
hardware (or the negotiated value if auto-neg is in effect). If a device 
doesn't support the operational state datastore then you wouldn't be 
able to query the operational speed of an interface if it hasn't also 
been configured.

If the device support the with-defaults extension and appropriate 
options then they could presumably retrieve the "complex default" value 
from the device using one of the with-default query parameters.

Rob

---

From: Andy Bierman <[email-masked]>
Date: Tue, 10 Jan 2017 09:25:08 -0800

On Tue, Jan 10, 2017 at 9:07 AM, Robert Wilton <[email-masked]> wrote:

>>>>> has no way to describe if the subtree is enabled (which is a
>>>>>
>>>> show-stopper)
>>>>
> model but wouldn't be easily retrievable without supporting the operational
> state datastore.  Specifically you lose the ability to easily query an
> operational property of a system that can be configured, but hasn't been
> configured.
>
> Example: Consider an Ethernet interface speed leaf that in the running ds
> represents the configured speed, and in the operational state ds represents
> the actual operational speed in use.  Normally, the operational speed would
> default to the maximum speed supported by the hardware (or the negotiated
> value if auto-neg is in effect). If a device doesn't support the
> operational state datastore then you wouldn't be able to query the
> operational speed of an interface if it hasn't also been configured.
>

This is my concern -- that data modelers will put in the <oper-speed> leaf
to make sure
all protocols (including existing NETCONF) can retrieve the oper-value.

For many decades, this has been the design approach.
There have not been many leafs where interactions with control-plane
protocols
is a factor.  The SNMP-style solution is ad-hoc, but the problem is
somewhat rare,
so it didn't really matter.

The premise now seems to be that the problem is no longer rare
and lots of <oper-speed> type of data is needed.  I am not even sure this
will
be true if I2RS is constrained to RIB data (as the charter dictates).

Presumably, the same instrumentation gets invoked for get(oper-speed) as
get-state(admin-speed)

> If the device support the with-defaults extension and appropriate options
> then they could presumably retrieve the "complex default" value from the
> device using one of the with-default query parameters.
>

with-defaults is a bit different because the YANG module can provide the
default
even if the server won't.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Tue, 10 Jan 2017 10:17:46 -0800

On Tue, Jan 10, 2017 at 8:16 AM, Juergen Schoenwaelder <
[email-masked]> wrote:

> is

Will they really work without /foo-state?

How will I write XPath must/when/leafref statements for foo-state if it
doesn't exist?

(BTW, why do I need foo-state in both applied and operational datastores?
This part is not clear)

If the solution is to introduce an XPath function like
operational-value(/top/speed)
then this is both complicated and mandatory (YANG designers need the same
XPath function set on every server)

Andy

---

From: Robert Wilton <[email-masked]>
Date: Tue, 10 Jan 2017 18:18:15 +0000

On 10/01/2017 17:25, Andy Bierman wrote:
>
>
> On Tue, Jan 10, 2017 at 9:07 AM, Robert Wilton <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>
>
>             [email-masked]
>     True, but there would also be an undesirable duplication of data
>     in the data tree.
>
>
>     I think that this may be more than "come in handy".  I think that
>     there would be key information that clients would expect to be
>     available in a model but wouldn't be easily retrievable without
>     supporting the operational state datastore.  Specifically you lose
>     the ability to easily query an operational property of a system
>     that can be configured, but hasn't been configured.
>
>     Example: Consider an Ethernet interface speed leaf that in the
>     running ds represents the configured speed, and in the operational
>     state ds represents the actual operational speed in use. 
>     Normally, the operational speed would default to the maximum speed
>     supported by the hardware (or the negotiated value if auto-neg is
>     in effect). If a device doesn't support the operational state
>     datastore then you wouldn't be able to query the operational speed
>     of an interface if it hasn't also been configured.
>
>
> This is my concern -- that data modelers will put in the <oper-speed> 
> leaf to make sure
I think that there may be a better way here:  The data modelers design 
the model on the assumption that an operational state datastore will be 
present.  We can then use a pyang plugin to generate an extra YANG model 
that contains the missing state leaves that would be required for the 
split config/state trees.  E.g. if it finds a config leaf in foo/speed 
it creates a module that contains foo-state/speed.  I've been playing 
around with pyang and I don't think that this would be too hard to do.

> so it didn't really matter.
Yes, OK.  But I think that SNMP failed for programmatic configuration, 
it only seemed to get traction returning operational state.

> and lots of <oper-speed> type of data is needed. I am not even sure 
> this will
I think that I2RS is orthogonal to what the operational state datastore 
is really solving, it just happens to help for I2RS as well.

>
> Presumably, the same instrumentation gets invoked for get(oper-speed) 
> as get-state(admin-speed)
Yes, in the general case, I think that using the same instrumentation 
would be a valid implementation.

But you may also be able to optimize this to fetching the information 
from the running configuration datastore (if you know for sure that the 
value will be the same).

If the additional metadata is being supported and returned then it may 
also need to compare the operational value with the configured value and 
schema default to choose the correct metadata annotation.

>     options then they could presumably retrieve the "complex default"
>     value from the device using one of the with-default query parameters.
>
>
> with-defaults is a bit different because the YANG module can provide 
> the default
I was thinking of the "report-all" option.  Wouldn't that mean that the 
server would have to return the actual value for a config leaf that had 
a complex default value (i.e. based on the hardware present)?

Rob

---

From: Andy Bierman <[email-masked]>
Date: Tue, 10 Jan 2017 10:31:49 -0800

On Tue, Jan 10, 2017 at 10:18 AM, Robert Wilton <[email-masked]> wrote:

> I think that there may be a better way here:  The data modelers design the
> model on the assumption that an operational state datastore will be
> that contains the missing state leaves that would be required for the split
> config/state trees.  E.g. if it finds a config leaf in foo/speed it creates
> a module that contains foo-state/speed.  I've been playing around with
> pyang and I don't think that this would be too hard to do.
>
>
>

This is a real hack.

I liked the if-feature approach much better
e.g.

   leaf oper-speed {
       if-feature "not operational-datastore";
       ...
   }

> Yes, OK.  But I think that SNMP failed for programmatic configuration, it
> only seemed to get traction returning operational state.
>
>

It failed because there are no transactions,
not because the oper-speed leaf exists or not.

> I think that I2RS is orthogonal to what the operational state datastore is
> really solving, it just happens to help for I2RS as well.
>
>
OK -- Joel informs me the I2RS charter is not constrained to RIB data at
all.

> But you may also be able to optimize this to fetching the information from
> the running configuration datastore (if you know for sure that the value
> will be the same).
>
> server would have to return the actual value for a config leaf that had a
> complex default value (i.e. based on the hardware present)?
>

basic=report-all just means the server never suppresses defaults.
It always returns them in <rpc-reply> messages.

Andy

---

From: Kent Watsen <[email-masked]>
Date: Tue, 10 Jan 2017 19:08:47 +0000

I think that there may be a better way here:  The data modelers design the model on the assumption that an operational state datastore will be present.  We can then use a pyang plugin to generate an extra YANG model that contains the missing state leaves that would be required for the split config/state trees.  E.g. if it finds a config leaf in foo/speed it creates a module that contains foo-state/speed.  I've been playing around with pyang and I don't think that this would be too hard to do.

I generally support this approach as stop-gap solution while the industry goes thru the transition.  However, I recommend a variant whereby the pyang plugin only migrates the config false values (not also the config true values).  The reason for this is as follows:

Migrating only the config false values is sufficient for matching existing functionality (read: a must-have requirement); that is, currently top-level /foo-state is used to support conveying the opstate for system-generated objects, that have lifetimes independent of config.

Migrating the config true nodes also is possible, but only needed if wanting to report the opstate value for config true nodes (e.g., hostname), but this would be above and beyond what is possible today (read: a nice-to-have requirement), and hence I’d rather steer folks towards the new approach rather than double-down on the approach we’re trying to get away from.

>       if-feature "not operational-datastore";

Is your proposal for the YANG modules to simultaneously define both opstate-aware and opstate-unaware trees?  Wouldn’t that make the modules less readable, largely redundant, and ripe for inconsistencies?

Kent  // as a contributor

---

From: Andy Bierman <[email-masked]>
Date: Tue, 10 Jan 2017 12:00:40 -0800

On Tue, Jan 10, 2017 at 11:08 AM, Kent Watsen <[email-masked]> wrote:

> I generally support this approach as stop-gap solution while the industry
> goes thru the transition.  However, I recommend a variant whereby the pyang
> plugin only migrates the config false values (not also the config true
> values).  The reason for this is as follows:
>
>
>
> Migrating only the config false values is sufficient for matching existing
> functionality (read: a must-have requirement); that is, currently top-level
> /foo-state is used to support conveying the opstate for system-generated
> objects, that have lifetimes independent of config.
>
>
>
> Migrating the config true nodes also is possible, but only needed if
> wanting to report the opstate value for config true nodes (e.g., hostname),
> but this would be above and beyond what is possible today (read: a
> nice-to-have requirement), and hence I’d rather steer folks towards the new
> approach rather than double-down on the approach we’re trying to get away
> from.
>
>
>
>
>
>
>
> > This is a real hack.
>
> >
>
> Is your proposal for the YANG modules to simultaneously define both
> opstate-aware and opstate-unaware trees?  Wouldn’t that make the modules
> less readable, largely redundant, and ripe for inconsistencies?
>
>
>
>
>

I think it is better to have a human decide what is in the module
instead of relying on a pyang plugin to generate some additional module
that follows some simplistic pattern.

Of course this solution only works if the value-set of operational data
is exactly the same as the configurable value-set (which is sometimes
not the case at all).

What is an  "opstate-aware tree".

The point of this work is that the opstate tree goes away and is replaced by
a protocol operation instead.

In fact, there are no new datastores needed whatsoever to
add the RPC <resource-ready>, or even <get-operational>.

Andy

---

From: Kent Watsen <[email-masked]>
Date: Tue, 10 Jan 2017 21:20:04 +0000

> that follows some simplistic pattern.

It may be simple, but I’m thinking that’s only because it’s not tricky  ;)

> is exactly the same as the configurable value-set (which is sometimes

The value-set of the operational data would be the combination of both the config true and config false nodes.   [Did you see this in my last response? I received an offline message indicating that my previous response was somewhat malformed, so you might’ve missed it...].   The config false nodes are obviously part of the operational data.  For the config true nodes, I’ve been swayed to believe that every config true node can also have an operational value.  I didn’t think so at first, using ‘hostname’ as an example to make my point, but it was pointed out that the admin might circumvent the YANG-driven config engine completely and set the hostname via the UNIX command line instead.  In this case, the intended hostname value might differ from the operational hostname value.  Even for nodes where we’re convinced there can never be an operational value that differs from the intended value, there still might be a propagation delay that causes a difference to be perceived in time.  All this points to a rather conservative and thankfully simple solution that the value-set of opstate is the combination of *all* the config true and config false nodes.

> What is an "opstate-aware tree".

I should’ve written “opstate-aware data model”.   

To be clear, by “opstate-aware tree”, I meant the YANG module would only have a top-level /foo tree that has both config true and config false nodes, with the expectation that the solution provided by draft-ietf-netmod-revised-datastores enables 1) opstate for both system-generated objects as well as for config true nodes and 2) opstate for all config true nodes also (note: this doesn’t remove the need for explicit config false nodes for negotiated values like MTU).

Similarly, by “opstate-unaware tree”, I meant the YANG module would have both top-level /foo and /foo-state trees.  Essentially, what we have today, which we’re trying to get away from.

> a protocol operation instead.

Correct, the hope is that the top-level /foo-state tree no longer needs to be defined in YANG modules.  This is the goal as we believe it will make YANG modules easier to read and write.

> add the RPC <resource-ready>, or even <get-operational>.

I’m not sure what you mean by RPC <resource-ready>.  True, a <get-operational> RPC could be defined now, but we’d still want the YANG modules to be a certain way and we’d still want to define a conceptual operational-state datastore so that we could describe it unambiguously.

Kent // as a contributor

---

From: Andy Bierman <[email-masked]>
Date: Tue, 10 Jan 2017 14:23:23 -0800

> It may be simple, but I’m thinking that’s only because it’s not tricky  ;)
The client and server developers still need to know about this
auto-generated module
and implement it.  Operators might have to know about it to use it.

> The value-set of the operational data would be the combination of both the
> config true and config false nodes.   [Did you see this in my last
> response? I received an offline message indicating that my previous
> response was somewhat malformed, so you might’ve missed it...].   The
> config false nodes are obviously part of the operational data.  For the
> config true nodes, I’ve been swayed to believe that every config true node
> can also have an operational value.  I didn’t think so at first, using
> ‘hostname’ as an example to make my point, but it was pointed out that the
> admin might circumvent the YANG-driven config engine completely and set the
> hostname via the UNIX command line instead.  In this case, the intended
> hostname value might differ from the operational hostname value.  Even for
> nodes where we’re convinced there can never be an operational value that
> differs from the intended value, there still might be a propagation delay
> that causes a difference to be perceived in time.  All this points to a
> rather conservative and thankfully simple solution that the value-set of
> opstate is the combination of *all* the config true and config false nodes.
>
>

But the foo-state node is being omitted from the module.
How does the pyang plugin know how to produce the extra values so the
auto-generated foo-state node has all the combined values?

> To be clear, by “opstate-aware tree”, I meant the YANG module would only
> have a top-level /foo tree that has both config true and config false
> nodes, with the expectation that the solution provided by
> draft-ietf-netmod-revised-datastores enables 1) opstate for both
> system-generated objects as well as for config true nodes and 2) opstate
> for all config true nodes also (note: this doesn’t remove the need for
> explicit config false nodes for negotiated values like MTU).
>
> Similarly, by “opstate-unaware tree”, I meant the YANG module would have
> both top-level /foo and /foo-state trees.  Essentially, what we have today,
> which we’re trying to get away from.
>
>
> Correct, the hope is that the top-level /foo-state tree no longer needs to
> be defined in YANG modules.  This is the goal as we believe it will make
> YANG modules easier to read and write.
>
>
> I’m not sure what you mean by RPC <resource-ready>.  True, a
> <get-operational> RPC could be defined now, but we’d still want the YANG
> modules to be a certain way and we’d still want to define a conceptual
> operational-state datastore so that we could describe it unambiguously.
>
>

I could have an RPC that tested a config subtree.
It would return 'true' if intended=applied for that subtree.

I agree it is good to have clear definitions that are widely understood.
It would be nice to have any clear definition of config=false YANG nodes,
whether datastores are used or not.  (e.g., does operational state
include counters?)

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 11 Jan 2017 00:54:13 +0100

I believe that shortening tranistion pain is in the longer term better
than prociding tools that at the end just extend the transition pain.

/js

---

From: Andy Bierman <[email-masked]>
Date: Tue, 10 Jan 2017 16:34:43 -0800

> than prociding tools that at the end just extend the transition pain.

That is a good goal, but ending up with a stable and useful solution
is more important.

The WG decided that an RPC-based solution was preferrred under the
assumption
that the data model design would not be impacted and existing YANG modules
could
report intended vs. actual values without any changes.

I certainly agree with Kent that real systems can override config values in
ways
that were not anticipated at module design-time.   All the more reason for
an RPC-based solution.

I am willing to accept the design team output as the architectural model.
I liked Phil's approach -- identify all the state transitions and decide
which
ones need to be exposed in the standard. I don't see how tweaking this model
will have any impact on the transition pain of the solution path.

Until the basic show-stoppers are solved, the redundant opstate objects are
not important.
Removing the foo-state objects means they are now invisible wrt/ YANG
constraints
(must, when, leafref, min/max, etc).  IMO this is a show-shopper.  YANG can
only cross-reference
YANG statements.  Invisible opstate hiding behind a datastore label seems
elegant
wrt/ <get>, but it looks like a disaster wrt/ YANG.

/js

Andy

> > I generally support this approach as stop-gap solution while the
> industry goes thru the transition.  However, I recommend a variant whereby
> the pyang plugin only migrates the config false values (not also the config
> true values).  The reason for this is as follows:
> >
> > Migrating only the config false values is sufficient for matching
> existing functionality (read: a must-have requirement); that is, currently
> top-level /foo-state is used to support conveying the opstate for
> system-generated objects, that have lifetimes independent of config.
> >

---

From: Kent Watsen <[email-masked]>
Date: Wed, 11 Jan 2017 00:53:32 +0000

Hi Andy,

> Until the basic show-stoppers are solved, the redundant opstate objects are not important.
> Removing the foo-state objects means they are now invisible wrt/ YANG constraints
> (must, when, leafref, min/max, etc).  IMO this is a show-shopper.  YANG can only cross-reference
> YANG statements.  Invisible opstate hiding behind a datastore label seems elegant

Nothing has been removed.  All the config false nodes are still available, but now they’re no longer separated into a top-level /foo-state tree for the sole purpose of being able to report opstate for system-generated objects.  Likewise, all YANG constraints continue to work, but rather than reference nodes in /foo-state, they’ll now reference nodes in /foo.   Does this make sense?  Do you still have an issue?

Kent // as a contributor

---

From: Andy Bierman <[email-masked]>
Date: Tue, 10 Jan 2017 16:57:38 -0800

On Tue, Jan 10, 2017 at 4:53 PM, Kent Watsen <[email-masked]> wrote:

> Nothing has been removed.  All the config false nodes are still available,
> but now they’re no longer separated into a top-level /foo-state tree for
> the sole purpose of being able to report opstate for system-generated
> objects.  Likewise, all YANG constraints continue to work, but rather than
> reference nodes in /foo-state, they’ll now reference nodes in /foo.   Does
> this make sense?  Do you still have an issue?
>
>
>
>
>

This does not work. There are no config=false nodes if they are overlaid
onto the config=true nodes.
There is no way to say in the YANG XPath that you mean the configured value
of /foo
vs. the operational value of /foo.  There is just 1 leaf that YANG says has
0 or 1 instance
(and therefore 0 or 1 value).

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 11 Jan 2017 10:22:09 +0100 (CET)

> and implement it.  Operators might have to know about it to use it.

Exactly.  I agree that this is a real hack.  Implementations can use
whatever transformation tricks they want in order to comply with
different standards, but the standard modules should be very clear.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 11 Jan 2017 10:27:29 +0100 (CET)

> (and therefore 0 or 1 value).

This is correct.  But note that YANG doesn't allow config true nodes
to refer to config false nodes anyway, so this is less of an issue.
Also note that draft-ietf-netmod-revised-datastores-00 proposes that
semantic constraints don't apply to the operational-state datastore
(see section 5.3).

BTW, it has been suggested before to add a function similar to the
XSLT 1.0 function "document", that could be used to refer to nodes in
other documents (or rather other datastores in our case).

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 11 Jan 2017 10:36:13 +0100 (CET)

Ladislav Lhotka <[email-masked]> wrote:
> 
> > On 10 Jan 2017, at 09:39, Juergen Schoenwaelder
> > <[email-masked]> wrote:
> > 
> >> I think we need protocol and YANG specs that are not tied to any
> >> particular model and that are thus capable of matching unforeseen
> >> real-world implementations. This is no sci-fi, HTTP and XML schema
> >> languages work this way.
> >> 
> > 
> Even now, a client that's programmed to write straight to running
> isn't interoperable with a server that has candidate and read-only
> running. A RESTCONF server that supports only JSON isn't interoperable
> with a client that supports only XML.
> 
> We are not in a situation that every pair of a randomly chosen server
> and client need to be interoperable. It's IMO perfectly fine if IoT
> and ISP networks use different clients. Yet, both can still use the
> same RESTCONF, same YANG, and even same YANG modules.

The fact is that that data models are written with a certain set of
protocol features and datastores in mind (the "meta-model").  Some
examples:

If we had an "operational-state" datastore like the one proposed, we
would not see the /foo vs /foo-state split.

If SNMP would have had a CREATE operation, MIBs would not have used
RowStatus.  If NETCONF didn't have a way to create instances, we would
have seen something similar in YANG models.

If NETCONF had a way to add comments to any node in a datastore, we
wouldn't have "leaf description" sprinkled throughout the models.

If NETCONF didn't have a generic way to filter retreived data, we'd
see lots of specific get-* rpcs in YANG models.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 11 Jan 2017 12:05:00 +0100

> On 11 Jan 2017, at 10:36, Martin Bjorklund <[email-masked]> wrote:
> 

Yes, but I assume this will go away anyway. However, we can still have YANG modules (and complete schemas) designed for the operational datastore. The important property of the "meta-model" so far has been that config and state data are separate, and this is not going to change.

> see lots of specific get-* rpcs in YANG models.

Maybe, but are the last three points relevant to this discussion?

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 11 Jan 2017 12:16:15 +0100 (CET)

Ladislav Lhotka <[email-masked]> wrote:
> Yes, but I assume this will go away anyway. However, we can still have
> YANG modules (and complete schemas) designed for the operational
> datastore. The important property of the "meta-model" so far has been
> that config and state data are separate, and this is not going to
> change.
> 
> > 

The point is that data models are designed with some meta-model in
mind.  The meta-model includes (some) datastores.  You wrote:

  I believe both the protocols and YANG can work with any set of
  datastores [...]

And I don't think that this is true (practically).  For example, a
YANG module that is designed with the new operational state datastore
in mind will be of limited use in a legacy NETCONF server.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 11 Jan 2017 12:43:40 +0100

> On 11 Jan 2017, at 12:16, Martin Bjorklund <[email-masked]> wrote:
> 

But where and how is this reflected in existing YANG modules (except for the foo and foo-state split, which is IMO a minor issue)?

> in mind will be of limited use in a legacy NETCONF server.

Please explain. My idea what could be done e.g. with ietf-interfaces is this:

1. Split it into two modules, say ietf-interfaces-config and ietf-interfaces-state. The former would contain exactly what's now inside "interfaces", and the latter will augment it with extra state data that are now under "interfaces-state".

2. The data model for configuration datastores will be defined to contain only ietf-interfaces-config whereas for operational-state datastore it will be ietf-interfaces-config *and* ietf-interfaces-state.

Am I completely misguided here? If not, then I don't see where the new modules refer to any particular datastore model. Yes, they do reflect that there is configuration and state data, but we don't want to get rid of this distinction, right?

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 11 Jan 2017 13:27:59 +0100 (CET)

Ladislav Lhotka <[email-masked]> wrote:
> But where and how is this reflected in existing YANG modules (except
> for the foo and foo-state split, which is IMO a minor issue)?

I don't this split is a minor issue.  For the openconfig group, this
is one of the major problems with YANG, leading to their design with
duplicate leafs.  The reason for adding the operational state
datastore in the form we propose in the draft it to be able to get rid
of this split.

> Please explain.

If a YANG module is designed with this new architecture in mind, it
will have a single top-level tree, which can support pre-configuration
and different instances in the config and operational state.

If such a module is implemented in a legacy NETCONF server, the only
way to get the operational state is to used <get/>.  But <get/> will
return the union between running and operational state.  The client
can't tell if an instance is really present in the operational state,
or just in the config.

My idea what could be done e.g. with ietf-interfaces
> is this:
> 
> 1. Split it into two modules, say ietf-interfaces-config and
> ietf-interfaces-state. The former would contain exactly what's now
> inside "interfaces", and the latter will augment it with extra state
> data that are now under "interfaces-state".
> 
> 2. The data model for configuration datastores will be defined to
> contain only ietf-interfaces-config whereas for operational-state
> datastore it will be ietf-interfaces-config *and*
> ietf-interfaces-state.

If we do this for all modules then we haven't gained anything; we
still have duplicate definitions.

/martin

> Am I completely misguided here? If not, then I don't see where the new
> modules refer to any particular datastore model. Yes, they do reflect
> that there is configuration and state data, but we don't want to get
> rid of this distinction, right?
> 
> Lada
> 
> > 
> > 
> > 
> > /martin
> 
> --

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 11 Jan 2017 13:45:29 +0100

> On 11 Jan 2017, at 13:27, Martin Bjorklund <[email-masked]> wrote:
> 

Show me a single YANG data node definition that's duplicate in my concept above. But then maybe I didn't explain it properly.

Note also that you slightly misinterpreted my statement that you cited:

I didn't say that there cannot be *modules* that are somehow designed for a particular datastore model - I meant YANG the language. 

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 11 Jan 2017 13:53:59 +0100 (CET)

Ladislav Lhotka <[email-masked]> wrote:
> Show me a single YANG data node definition that's duplicate in my
> concept above. But then maybe I didn't explain it properly.

The interface's "type" leaf.  With the new operational-state
datastore, /interfaces/interface/type in operational-state and
/interfaces-state/interface/type are duplicate.

> I didn't say that there cannot be *modules* that are somehow designed
> for a particular datastore model - I meant YANG the language.

Ok.  Yes, you're right, but then we'd probably need some new statement
in each module that tells which meta-model the YANG module is written
for.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 11 Jan 2017 14:05:13 +0100

> On 11 Jan 2017, at 13:53, Martin Bjorklund <[email-masked]> wrote:
> 

As I said, ietf-interfaces-state state would consist of augments containing extra state nodes (i.e. those that are not in configuration). So "type" won't be there.

> in each module that tells which meta-model the YANG module is written

I would prefer to have it as state data, basically separate YANG libraries for configuration datastores and operational-state.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 11 Jan 2017 14:19:59 +0100

> On 11 Jan 2017, at 14:05, Ladislav Lhotka <[email-masked]> wrote:
> 
>> 

BTW, this approach could also solve the issue of different "levels" of data that Rob asked about previously:

https://www.ietf.org/mail-archive/web/netmod/current/msg17125.html

A server could then provide additional operational-state datastores, for example "operational-state-diagnostics" and "operational-state-registers". This is IMO quite a clean solution.

Lada

---

From: Robert Wilton <[email-masked]>
Date: Wed, 11 Jan 2017 14:18:19 +0000

Hi,

>> Ladislav Lhotka <[email-masked]> wrote:
<snip>
> As I said, ietf-interfaces-state state would consist of augments containing extra state nodes (i.e. those that are not in configuration). So "type" won't be there.
I think that this effectively just achieves the same thing that the 
"config: true|false" statement indicates in a combined config/state tree.

Personally, I think that a file of augmentations is probably going to be 
harder to read than having the config and state schema in one tree in 
one file.

The models may also be slightly more inconvenient to use because the 
state tree leaves would presumably be in a different namespace from the 
configuration?

If you wanted this file level split then using submodules would allow 
for separate config/state files but still be managed as a single 
combined module.

Rob

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 11 Jan 2017 15:37:03 +0100 (CET)

Ladislav Lhotka <[email-masked]> wrote:
> As I said, ietf-interfaces-state state would consist of augments
> containing extra state nodes (i.e. those that are not in
> configuration). So "type" won't be there.

So how would a client learn the type of a system-controlled interface?

> I would prefer to have it as state data, basically separate YANG
> libraries for configuration datastores and operational-state.

But the use case is that a particular module is designed for a certain
datastore model (which you wrote above).  This is a design-time
property, not a rum-time property, so state data (run-time) is not the
right solution.   If a module is designed for a certain datastore
model, that module cannot be implemented in a server with some other
datastore model.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 11 Jan 2017 15:43:17 +0100 (CET)

> So how would a client learn the type of a system-controlled interface?

Never mind; I misread your proposal, sorry about that.

But I agree with Rob; how is the proposal with two modules different than
having them in the same module (apart from the additional namespace
required)?

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 11 Jan 2017 15:48:06 +0100

> On 11 Jan 2017, at 15:18, Robert Wilton <[email-masked]> wrote:
> 
> Hi,
> 
> I think that this effectively just achieves the same thing that the "config: true|false" statement indicates in a combined config/state tree.
> 
> Personally, I think that a file of augmentations is probably going to be harder to read than having the config and state schema in one tree in one file.

Possibly we could also use schema mount. On the other hand, it doesn't enforce the use of operational-state datastore. A simple device like a WiFi-controlled electric socket could easily use just ietf-interfaces-config (and ietf-ip-config), i.e. no state data.

Another example I am dealing with now is OpenWRT: with some effort, it would be possible to translate our nice configuration data into UCI files without touching the OpenWRT system itself. On the other hand, serving state data according to our YANG modules is a non-starter.

> 
> The models may also be slightly more inconvenient to use because the state tree leaves would presumably be in a different namespace from the configuration?
> 

Yes, but I don't think it is a big problem - even for human readers.

> If you wanted this file level split then using submodules would allow for separate config/state files but still be managed as a single combined module.

But it means you have to implement both.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 11 Jan 2017 16:10:17 +0100

> On 11 Jan 2017, at 15:37, Martin Bjorklund <[email-masked]> wrote:
> 

Not necessarily, and certainly not all modules. In my example with ietf-interfaces, one could still emulate the current situation with "interfaces" and "interfaces-state" by defining these containers as two mount points and then schema-mounting the necessary schemas under each. There are some minor annoyances regarding namespaces but it could IMO also work fine for a client that uses the "old" datastore model.

Lada

---

From: Robert Wilton <[email-masked]>
Date: Wed, 11 Jan 2017 15:12:42 +0000

>> and implement it.  Operators might have to know about it to use it.
My idea is not to auto generate models on the fly.

My aim is to allow folks to start writing models in the desired long 
term format (i.e. combined config and state tree) with the model 
designer being able to assume the existence of the operational state 
datastore.

The tooling would be there to statically generate the extra foo-state 
config false node modules for servers that don't support the operational 
state datastore.  This could be done once, and the extra foo-state 
modules committed to the github YANG respository in the same way that 
models are extracted from IETF RFCs today.

The aim here is that the single model being produced by IETF would be 
usable both by new client/servers that support an operational state 
datastore, and also by existing NETCONF client/servers that don't 
implement an operational state datastore.

I'm not proposing that as a long term solution, but as a path to make it 
easier for folk to migrate, and to not slow down the model writing 
effort.  Otherwise, it may be hard to get a protocol model writer to 
design the YANG model in a way that is not fully usable on any current 
devices.

As an illustration, an RFC published combined ietf-interfaces model may 
look like this:

module: ietf-interfaces-combined
     +--rw interfaces
        +--rw interface* [name]
           +--rw name                        string
           +--rw description?                string
           +--rw type                        identityref
           +--rw enabled?                    boolean
           +--rw link-up-down-trap-enable?   enumeration {if-mib}?
           +--ro oper-status                 enumeration
           +--ro last-change? yang:date-and-time
           +--ro if-index                    int32 {if-mib}?
           +--ro phys-address? yang:phys-address
           +--ro higher-layer-if*            interface-ref
           +--ro lower-layer-if*             interface-ref
           +--ro speed?                      yang:gauge64
           +--ro statistics
              +--ro discontinuity-time    yang:date-and-time
              +--ro in-octets?            yang:counter64
              +--ro in-unicast-pkts?      yang:counter64
              +--ro in-broadcast-pkts?    yang:counter64
              +--ro in-multicast-pkts?    yang:counter64
              +--ro in-discards?          yang:counter32
              +--ro in-errors?            yang:counter32
              +--ro in-unknown-protos?    yang:counter32
              +--ro out-octets?           yang:counter64
              +--ro out-unicast-pkts?     yang:counter64
              +--ro out-broadcast-pkts?   yang:counter64
              +--ro out-multicast-pkts?   yang:counter64
              +--ro out-discards?         yang:counter32
              +--ro out-errors?           yang:counter32

The extra generated model would look like this:

module: ietf-interfaces-combined-state
     +--ro interfaces-state
        +--ro interface* [name]
           +--ro name                        string
           +--ro description?                string
           +--ro type                        identityref
           +--ro enabled?                    boolean
           +--ro link-up-down-trap-enable?   enumeration {if:if-mib}?
           +--ro oper-status                 enumeration
           +--ro last-change? yang:date-and-time
           +--ro if-index                    int32 {if:if-mib}?
           +--ro phys-address? yang:phys-address
           +--ro higher-layer-if* if:interface-ref
           +--ro lower-layer-if* if:interface-ref
           +--ro speed?                      yang:gauge64
           +--ro statistics
              +--ro discontinuity-time    yang:date-and-time
              +--ro in-octets?            yang:counter64
              +--ro in-unicast-pkts?      yang:counter64
              +--ro in-broadcast-pkts?    yang:counter64
              +--ro in-multicast-pkts?    yang:counter64
              +--ro in-discards?          yang:counter32
              +--ro in-errors?            yang:counter32
              +--ro in-unknown-protos?    yang:counter32
              +--ro out-octets?           yang:counter64
              +--ro out-unicast-pkts?     yang:counter64
              +--ro out-broadcast-pkts?   yang:counter64
              +--ro out-multicast-pkts?   yang:counter64
              +--ro out-discards?         yang:counter32
              +--ro out-errors?           yang:counter32

Servers that support operational-state would just implement 
ietf-interfaces-combined

Servers that don't support operational-state could implement 
ietf-interfaces-combined and ietf-interfaces-combined-state, probably 
not implementing the duplicate config false leaves under the interfaces 
config tree.  Deviations could also be auto-generated to remove the 
config false leaves from the config tree so that they are only in the 
state tree.

Of course, Clients may need to support both schemes depending on what 
types of devices they are interacting with.

Finally, I've illustrated this using ietf-interfaces, but I'm not 
actually proposing immediately changing that model.  I was more thinking 
about IETF protocols that in the process of working on their YANG models.

Rob

---

From: Robert Wilton <[email-masked]>
Date: Wed, 11 Jan 2017 15:18:09 +0000

> Another example I am dealing with now is OpenWRT: with some effort, it would be possible to translate our nice configuration data into UCI files without touching the OpenWRT system itself. On the o…
Isn't the proper answer here to generate deviations to remove the state 
leaves that won't be implemented.

> Yes, but I don't think it is a big problem - even for human readers.
I'm not sure.  I think that you might end up with a variation of the 
OpenConfig models, and based on experience I would say that those models 
are hard to read if they haven't been compiled first to expand the 
groupings and reveal the actual structure.

Rob

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 11 Jan 2017 16:37:19 +0100

> On 11 Jan 2017, at 16:18, Robert Wilton <[email-masked]> wrote:
> 
> 
> 
> Isn't the proper answer here to generate deviations to remove the state leaves that won't be implemented.

This is of course possible but deviations are generally frowned upon, so why not use the modularity features for making it a little more flexible? I know some people will now say that without state data it is no proper network management but, speaking pragmatically, automated configuration would be a big boon by itself.

> I'm not sure.  I think that you might end up with a variation of the OpenConfig models, and based on experience I would say that those models are hard to read if they haven't been compiled first to expand the groupings and reveal the actual structure.

One thing that makes modules really hard to read are augments, and we already have them all over the place. It's a trade-off between readability and modularity.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Wed, 11 Jan 2017 08:56:14 -0800

Hi,

On Wed, Jan 11, 2017 at 7:12 AM, Robert Wilton <[email-masked]> wrote:

> My aim is to allow folks to start writing models in the desired long term
> format (i.e. combined config and state tree) with the model designer being
> able to assume the existence of the operational state datastore.
>
>

I am not convinced this "new format" has solved anything.
Don't you need separate description-stmts in every node for each
datastore?  What does the value mean if pre-configured? configured?
operational?  Will the auto-generated objects be exactly correct
and never need any alterations or additional text?
They still need to be used by developers and YANG tools.

Is is that realistic to force the config structure and operational structure
to be the same? Seems it is quite common to monitor data structures
with additional keys or different keys.  This is completely unsupported
so separate /foo and /foo-state trees will still exist.

IMO this combination of trees needs to be proven.
Take ietf-interfaces and show how much better it will work
if the /interfaces and /interfaces-state trees were combined.

Andy

> state datastore.  This could be done once, and the extra foo-state modules
> committed to the github YANG respository in the same way that models are
> extracted from IETF RFCs today.
>
> datastore, and also by existing NETCONF client/servers that don't implement
> easier for folk to migrate, and to not slow down the model writing effort.
> Otherwise, it may be hard to get a protocol model writer to design the YANG
> model in a way that is not fully usable on any current devices.
>
> ietf-interfaces-combined and ietf-interfaces-combined-state, probably not
> implementing the duplicate config false leaves under the interfaces config
> tree.  Deviations could also be auto-generated to remove the config false
> leaves from the config tree so that they are only in the state tree.
>
> Finally, I've illustrated this using ietf-interfaces, but I'm not actually
> proposing immediately changing that model.  I was more thinking about IETF
> protocols that in the process of working on their YANG models.
>
> Rob
>
>

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 11 Jan 2017 18:21:19 +0100

> On 11 Jan 2017, at 17:56, Andy Bierman <[email-masked]> wrote:
> 
> Hi,
> 
> 
> My aim is to allow folks to start writing models in the desired long term format (i.e. combined config and state tree) with the model designer being able to assume the existence of the operational state datastore.
> 
> 
> 

Right, this is one problem of this "deduplication": even if two nodes - one config and the other state - have the same name or even type (which is not always the case, as we know), their semantics is often different. An IP address in configuration means a manually configured address whereas in state it may come from any source. So writing sensible descriptions will become tricky.

> so separate /foo and /foo-state trees will still exist.

I agree.

Lada

> The tooling would be there to statically generate the extra foo-state config false node modules for servers that don't support the operational state datastore.  This could be done once, and the extra foo-state modules committed to the github YANG respository in the same way that models are extracted from IETF RFCs today.
> 
> The aim here is that the single model being produced by IETF would be usable both by new client/servers that support an operational state datastore, and also by existing NETCONF client/servers that don't implement an operational state datastore.
> 
> I'm not proposing that as a long term solution, but as a path to make it easier for folk to migrate, and to not slow down the model writing effort.  Otherwise, it may be hard to get a protocol model writer to design the YANG model in a way that is not fully usable on any current devices.
> 
> As an illustration, an RFC published combined ietf-interfaces model may look like this:
> 
> Servers that support operational-state would just implement ietf-interfaces-combined
> 
> Servers that don't support operational-state could implement ietf-interfaces-combined and ietf-interfaces-combined-state, probably not implementing the duplicate config false leaves under the interfaces config tree.  Deviations could also be auto-generated to remove the config false leaves from the config tree so that they are only in the state tree.
> 
> Of course, Clients may need to support both schemes depending on what types of devices they are interacting with.
> 
> Finally, I've illustrated this using ietf-interfaces, but I'm not actually proposing immediately changing that model.  I was more thinking about IETF protocols that in the process of working on their YANG models.
> 
> Rob
> 
>

---

From: Andy Bierman <[email-masked]>
Date: Wed, 11 Jan 2017 09:34:25 -0800

On Wed, Jan 11, 2017 at 1:27 AM, Martin Bjorklund <[email-masked]> wrote:

> > > > Until the basic show-stoppers are solved, the redundant opstate
> objects
> > > are not important.
> > >
> > > > (must, when, leafref, min/max, etc).  IMO this is a show-shopper.
> YANG
> > > can only cross-reference
> > >

So valid YANG constraints applying to config=false nodes would now be
ignored?  Just put in the YANG module to fool people?
How can you propose to ignore YANG constraints on operational data?

> other documents (or rather other datastores in our case).
So a new version of YANG is needed to support combining
config and oper into 1 tree?

Andy

---

From: Andy Bierman <[email-masked]>
Date: Wed, 11 Jan 2017 12:32:25 -0800

On Wed, Jan 11, 2017 at 9:21 AM, Ladislav Lhotka <[email-masked]> wrote:

> term format (i.e. combined config and state tree) with the model designer
> being able to assume the existence of the operational state datastore.
> >
> >
> >
> Right, this is one problem of this "deduplication": even if two nodes -
> one config and the other state - have the same name or even type (which is
> not always the case, as we know), their semantics is often different. An IP
> address in configuration means a manually configured address whereas in
> state it may come from any source. So writing sensible descriptions will
> become tricky.
>
> >

OK -- let me see if I understand the value of combining ietf-interfaces.

Here is the starting tree:

     +--rw interfaces
      |  +--rw interface* [name]
      |     +--rw name                        string
      |     +--rw description?                string
      |     +--rw type                        identityref
      |     +--rw enabled?                    boolean
      |     +--rw link-up-down-trap-enable?   enumeration
      +--ro interfaces-state
         +--ro interface* [name]
            +--ro name               string
            +--ro type               identityref
            +--ro admin-status       enumeration
            +--ro oper-status        enumeration
            +--ro last-change?       yang:date-and-time
            +--ro if-index           int32
            +--ro phys-address?      yang:phys-address
            +--ro higher-layer-if*   interface-state-ref
            +--ro lower-layer-if*    interface-state-ref
            +--ro speed?             yang:gauge64
            +--ro statistics
               +--ro discontinuity-time    yang:date-and-time
               +--ro in-octets?            yang:counter64
               +--ro in-unicast-pkts?      yang:counter64
               +--ro in-broadcast-pkts?    yang:counter64
               +--ro in-multicast-pkts?    yang:counter64
               +--ro in-discards?          yang:counter32
               +--ro in-errors?            yang:counter32
               +--ro in-unknown-protos?    yang:counter32

So these are the objects that would no longer be duplicated:

    - name
    - type

Neither one is supposed to have a different value in operational state vs
configuration.

   - enabled
   - link-up-down-trap-enable

These 2 could be different in operational state I suppose.
An RPC can provide the operational value without changing the YANG module

    rpc get-oper-value {
      input {
         leaf node {
            type instance-identifier;
             description "the config=true node to check";
          }
      }
      output {
          anydata value {
             description
               "contains 1 child node matching the input 'node' parameter.
                The value of the node is the current operational value."
          }
     }
   }

   <rpc>
      <get-oper-value>
          <node>/if:interfaces/if:interface[if:name='eth0']/enabled</node>
      </get-oper-value>
   </rpc>

   <rpc-reply>
       <value>
          <if:enabled>false</if:enabled>
        </value>
     </rpc-reply>

I don't need to change the YANG module at all to support operational state.

Andy

---

From: Robert Wilton <[email-masked]>
Date: Thu, 12 Jan 2017 09:51:37 +0000

On 11/01/2017 20:32, Andy Bierman wrote:
>
>
> On Wed, Jan 11, 2017 at 9:21 AM, Ladislav Lhotka <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>
>     > On 11 Jan 2017, at 17:56, Andy Bierman <[email-masked]
>     <mailto:[email-masked]>> wrote:
>     >
>     > Hi,
>     >
>     >
>     > On Wed, Jan 11, 2017 at 7:12 AM, Robert Wilton
>     <[email-masked] <mailto:[email-masked]>> wrote:
>     >
>     >
>     > Andy Bierman <[email-masked] <mailto:[email-masked]>> wrote:
>     > On Tue, Jan 10, 2017 at 1:20 PM, Kent Watsen
>     <[email-masked] <mailto:[email-masked]>> wrote:
>     >
>     > My aim is to allow folks to start writing models in the desired
>     long term format (i.e. combined config and state tree) with the
>     model designer being able to assume the existence of the
>     operational state datastore.
>     >
>     >
>     >
>     Right, this is one problem of this "deduplication": even if two
>     nodes - one config and the other state - have the same name or
>     even type (which is not always the case, as we know), their
>     semantics is often different. An IP address in configuration means
>     a manually configured address whereas in state it may come from
>     any source. So writing sensible descriptions will become tricky.
>
I don't disagree that in some cases there is some nuanced differences 
between configured values and values that may be learned via other 
dynamic mechanisms.

The pragmatic point solution for these sorts of cases is to add an extra 
leaf to indicate the source of the value.  The existing ietf-ip YANG 
model already has a "source" leaf to indicate whether the address is 
static, dhcp, link-layer, random.  This type of adhoc mechanism will 
continue to work with a combined config/state data tree as well.

However, draft-ietf-netmod-revised-datastores also provides a 
generalized solution to this problem, via using origin metadata 
annotations (sections 5.2 and 8).  This allows the server to annotate 
whether a node in the operational state datastore has been populated via 
running configuration, or has been acquired via some other mechanism.

Yes, the descriptions may have to be written in a slightly different 
way, but I don't think that it is going to be too tricky to understand 
that was is in running represents the configuration given to the system, 
vs what is in the operational state datastore represents what 
configuration the device is actually running, along with all of the 
associated other operational state.

Rob

>     > The tooling would be there to statically generate the extra
>     foo-state config false node modules for servers that don't support
>     the operational state datastore.  This could be done once, and the
>     extra foo-state modules committed to the github YANG respository
>     in the same way that models are extracted from IETF RFCs today.
>     >
>     > The aim here is that the single model being produced by IETF
>     would be usable both by new client/servers that support an
>     operational state datastore, and also by existing NETCONF
>     client/servers that don't implement an operational state datastore.
>     >
>     > I'm not proposing that as a long term solution, but as a path to
>     make it easier for folk to migrate, and to not slow down the model
>     writing effort.  Otherwise, it may be hard to get a protocol model
>     writer to design the YANG model in a way that is not fully usable
>     on any current devices.
>     >
> Neither one is supposed to have a different value in operational state 
> vs configuration.
>
>    - enabled
>     ietf-interfaces-combined and ietf-interfaces-combined-state,
>     probably not implementing the duplicate config false leaves under
>     the interfaces config tree.  Deviations could also be
>     auto-generated to remove the config false leaves from the config
>     tree so that they are only in the state tree.
>     >
>     > Of course, Clients may need to support both schemes depending on
>     what types of devices they are interacting with.
>     >
>     > Finally, I've illustrated this using ietf-interfaces, but I'm
>     not actually proposing immediately changing that model.  I was
>     more thinking about IETF protocols that in the process of working
>     on their YANG models.
>     >
>     > Rob
>     >
>     >
>     > [email-masked] <mailto:[email-masked]>
>     <https://www.ietf.org/mailman/listinfo/netmod>
>     >
>     >
>     > [email-masked] <mailto:[email-masked]>
>     <https://www.ietf.org/mailman/listinfo/netmod>
>
>     --

---

From: Robert Wilton <[email-masked]>
Date: Thu, 12 Jan 2017 10:32:44 +0000

On 11/01/2017 16:56, Andy Bierman wrote:
> Hi,
>
>
> On Wed, Jan 11, 2017 at 7:12 AM, Robert Wilton <[email-masked] 
> datastore?
No, generally I don't this so.  Assuming clients understand the general 
semantics and datastore architecture it should be fairly easy for them 
to understand the difference between the configured value and 
operational value.

There might need to be some nuance as to how the description statements 
are written, but this still seems better than requiring two separate 
leaves for each property so that first one can have a description that 
says "this is the configured value" and the second says "this is the 
operational value".  Anywhere where it is necessary to have manual 
duplication leads to mistakes and inconsistencies creeping in.

>   What does the value mean if pre-configured? configured?
> operational?

The datastore architecture defines the semantic meaning of what a node 
means in a particular datastore.

This is true even today between startup, candidate, and running datastores.

>   Will the auto-generated objects be exactly correct

Yes, they will be exactly correct, and never need any alterations, since 
they have exactly the same semantic meaning as for the equivalent config 
true leaf in the operational state datastore.

The auto-generation script could trivially append to each description 
statement for nodes in the generated model to indicate that it 
represents the operational value.

> so separate /foo and /foo-state trees will still exist.

It is not forcing the config and operational state structure to be the 
same, only that the operational state must extend from the config structure.

These "monitor data structures" can continue to exist as config false 
structures just as they do today somewhere in the combined tree.  They 
could even be put in top level foo-state objects if there was nowhere 
lower down the tree that is a better place for them, but I think that in 
the general case (for components that can be configured on or off, they 
can live under a single top level, config true, component container).

Finally, if you allow the core structure of the config tree and state 
tree to deviate that it makes life much harder for management clients 
because they have to be hardcoded to know where the corresponding state 
lives for a particular set of configuration.

> if the /interfaces and /interfaces-state trees were combined.

Yes, this might be a good idea.  But doing this for the base IETF 
interface model on its own isn't really going to be informative.  I 
think that a larger example is necessary to properly see the benefits.

Rob

>     The aim here is that the single model being produced by IETF would
>     be usable both by new client/servers that support an operational
>     state datastore, and also by existing NETCONF client/servers that
>     don't implement an operational state datastore.
>
>     actually proposing immediately changing that model.  I was more
>     thinking about IETF protocols that in the process of working on
>     their YANG models.
>
>     Rob
>
>

---

From: Robert Wilton <[email-masked]>
Date: Thu, 12 Jan 2017 10:45:07 +0000

> Neither one is supposed to have a different value in operational state 

I (personally) would make the config "enabled" and the state 
"admin-status" leaf the same as well.

But this example is really too simple on its own, a larger model needs 
to be considered to properly see the benefits.

> I don't need to change the YANG module at all to support operational 

This solution would naively seem to be very limited.

Rob

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 12 Jan 2017 13:47:37 +0100 (CET)

>                 The value of the node is the current operational value."

This is essentially what we propose, except that we have generalized
it so that more than one value can be retreived: <get-state> or
<get-data> which takes a filter just like <get>.

> I don't need to change the YANG module at all to support operational state.

Correct.  Old modules will continue to work.  Clients that <get-state>
both /interfaces and /interfaces-state will receive some duplicate
data.

However, the new model allows for combined trees to be defined.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Thu, 12 Jan 2017 09:19:54 -0800

On Thu, Jan 12, 2017 at 4:47 AM, Martin Bjorklund <[email-masked]> wrote:

> > > > It may be simple, but I’m thinking that’s only because it’s not
> tricky
> > > ;)
> > > >
> > > >
> designer
> modules
> make it
> effort.
OK

How do I retrieve the same subtree from multiple contexts at once
so I can reduce time-skew caused by retrieving config-tree then oper-tree?

> However, the new model allows for combined trees to be defined.

Here are the things that do not work anymore if a designer uses the new
tree approach:

YANG statements:
   - It is not possible to define these statements so they are different
for config and oper
      - must
      - when
      - unique
      - key
      - min-elements
      - max-elements
      - leafref (path)
      - if-feature
      - deviation
      - type (or any sub-statements of type-stmt)
      - status
      - description
      - reference

YANG allows must-when to reference state data nodes in every XPath context
except 1:
   - state data
   - RPC input
   - RPC output
   - action input
   - action output
   - notification payload

Seems like you are removing a lot of YANG functionality in order to make
the problem-space
fit your solution-space.

Andy

> not
> thinking

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 12 Jan 2017 18:34:02 +0100

>       - type (or any sub-statements of type-stmt)

Considering statements that constraint 'values', it is not entirely
clear to me what they mean for state nodes. If a server has
operational state that violates a must or range or ... constraint in
the YANG model, what is the server expected to do?

/js

---

From: Andy Bierman <[email-masked]>
Date: Thu, 12 Jan 2017 09:38:46 -0800

> the YANG model, what is the server expected to do?

The client uses the YANG validation to check on what the server is sending.
The server is buggy if it is sending data that violates YANG constraints.
If any of these statements need to be different for config and oper
then the old style YANG has to be used instead.

Andy

---

From: Robert Wilton <[email-masked]>
Date: Thu, 12 Jan 2017 17:54:03 +0000

On 12/01/2017 17:38, Andy Bierman wrote:
>
>
> On Thu, Jan 12, 2017 at 9:34 AM, Juergen Schoenwaelder 
You just have a separate state leaf.  These are still allowed in a 
combined tree.

The config true leaf in the operational state datastore represents the 
applied configuration.
The additional state leaf represents some more complex state derived 
from the applied configuration, just like the rest of the config false 
leaves.

Rob

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 12 Jan 2017 19:44:52 +0100

> then the old style YANG has to be used instead.

OK. So the client does the validation. What does the client do if the
operational state it got is not valid according to the YANG constraints?

/js

---

From: Alex Campbell <[email-masked]>
Date: Thu, 12 Jan 2017 20:54:38 +0000

IMO it should be treated like any other protocol error.
Which means that an ideal client will report an error - but in practice they'll end up ignoring the constraint violation because it's easier to not do validation.

This problem isn't specific to YANG - what happens if I make a request to an HTTP server ("GET / HTTP/1.1") and the server sends back nonsense ("FOOBAR/-1.3i xyz Didn't feel like it")?
A good client will report that there was an error parsing the response; a bad client might call atoi on the second field (recording the status code as 0) and ignore the other fields.
Is there anything we can do about that? I don't think there is.

Alex

________________________________________
From: netmod <[email-masked]> on behalf of Juergen Schoenwaelder <[email-masked]>
Sent: Friday, 13 January 2017 7:44 a.m.
To: Andy Bierman
Cc: Netconf; [email-masked]
Subject: Re: [netmod] [Netconf] Decision on the Intended Status of the Revised DS Draft WAS:RE: :candidate, :writable-running and RESTCONF edits

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 12 Jan 2017 22:20:44 +0100

> On 12 Jan 2017, at 19:44, Juergen Schoenwaelder <[email-masked]> wrote:
> 

Don't forget that data models also provide guidelines to server implementors. It is not without reason to write a test suite that validates server responses, including state data.

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 12 Jan 2017 22:27:55 +0100

> Don't forget that data models also provide guidelines to server implementors. It is not without reason to write a test suite that validates server responses, including state data.

OK. But what do you expect a regular client to do?

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 12 Jan 2017 22:59:37 +0100 (CET)

Ladislav Lhotka <[email-masked]> wrote:
> 
> > On 12 Jan 2017, at 19:44, Juergen Schoenwaelder
> Don't forget that data models also provide guidelines to server
> implementors.

Yes, and that it is all that can be done currently.  I don't think any
implemention that receives a <get> request today freezes the system in
order to get a guaranteed consistent snapshot.  Instead, the different
subsystems will be queried, sequentially or in parallell, and the end
result is shipped to the client.  The result may or may not be
consistent.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Thu, 12 Jan 2017 14:05:07 -0800

> result is shipped to the client.  The result may or may not be
So this thread is questioning why YANG allows constraints on config=false
data nodes.
>From the generic toolbuilder POV, it allows the YANG engine to report
issues to
the operator without custom programming for each little issue.  Who knows
why the
foo-table requires 3 entries to be valid min-elements 3).
The tool can report to the operator "foo-table does not have enough entries"
and let the operator decide what to do about it.

/martin
Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 13 Jan 2017 09:09:17 +0100

> On 12 Jan 2017, at 22:27, Juergen Schoenwaelder <[email-masked]> wrote:
> 

Complain to the vendor that their server is broken.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 13 Jan 2017 09:22:12 +0100

> On 12 Jan 2017, at 22:59, Martin Bjorklund <[email-masked]> wrote:
> 

Which doesn't mean that inconsistent (format of) state data is acceptable. My colleague develops a BGP looking glass, and it is really a terrible work because he has to do a lot of screen-scraping. Each time a vendor changes the data format, he has to update his software. I keep telling him that our nice protocols would save him from this drudgery.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 13 Jan 2017 09:42:41 +0100

> On 12 Jan 2017, at 23:05, Andy Bierman <[email-masked]> wrote:
> 
> 
> 
> So this thread is questioning why YANG allows constraints on config=false data nodes.

Partly it is a fault of YANG spec because RFC 7950 only says in sec. 8.1 that

   The running configuration datastore MUST always be valid.

This is exactly what I would like to remove. *Any* data tree for which we have a corresponding YANG data model can be validated, and it is IMO up to a particular management application (with a certain datastore model) to decide what needs to be validated and what not.

A particular workflow, datastore model and validation procedure can also be standardized, but it should be done outside the YANG spec.

Lada

> From the generic toolbuilder POV, it allows the YANG engine to report issues to
> the operator without custom programming for each little issue.  Who knows why the

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 13 Jan 2017 09:51:38 +0100

> So this thread is questioning why YANG allows constraints on config=false

The point I am trying to make is that for configuration data, we have
a clear model what validation means and when it is applied. The basic
idea is that the running configuration of a system is always valid and
configuration changes will never leave the running configuration in an
invalid state, i.e., configuration change requests are rejected if
they would leave to invalid configuration state.

For operational state data, it is not entirely clear where and when
validation will occur or if it will occur at all. In fact, if a system
is in a weird state, it might actually be useful for certain purposes
to expose the weird state. And, as Martin pointed out, inconsistencies
may also result from the technical difficulties to take a consistent
snapshort of a system with several concurrent subsystems. For these
reasons, the assumption that a server always returns valid operational
state data is likely not true.

> From the generic toolbuilder POV, it allows the YANG engine to
> report issues to the operator without custom programming for each
> little issue. Who knows why the foo-table requires 3 entries to be
> valid min-elements 3). The tool can report to the operator
> "foo-table does not have enough entries" and let the operator decide
> what to do about it.

Yes, it will be up to the client's functionality and its implemention
specifics to deal with the question whether it is useful to validate
operational state data or what to do if the received operational state
data is invalid.

While servers are expected to be strict about the validity of
configuration data, I think clients need to be lenient about the
validity of operational state data.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 13 Jan 2017 10:26:02 +0100

> On 13 Jan 2017, at 09:51, Juergen Schoenwaelder <[email-masked]> wrote:
> 

These are valid reasons and could IMO be lived with, but I think we have to require "best effort" from the server. If we drop validity of state data altogether, their invalidity will be commonplace, and (I suspect) caused mainly by laziness of server implementors.

That said, I would be in favour of decoupling data models of configuration and state data. If a device produces native state data in a format that's difficult to translate to a standard form, vendors could be allowed to use a proprietary model for state data. It is IMO still far better than declaring compliance to RFC XXXX on paper and then sending something else.

> validity of operational state data.

It is all about the Postel Principle: the server should be strict, and the client may be lenient. It also depends on the character of the state data and specific aspect of validity (broken leafref is not the same as missing mandatory leaf).

Lada

---

From: Andy Bierman <[email-masked]>
Date: Fri, 13 Jan 2017 08:50:17 -0800

> validity of operational state data.

Actually, a client that is not restricted by NACM can use the YANG
validation just fine.
There are opstate tables that churn but that is the exception.

Also note that the rpc/input and action/input constraints are validated
by the server, and MUST be implemented.  The XPath in these contexts
are allowed to reference state data (starting in YANG 1.1)

Andy

---

From: Phil Shafer <[email-masked]>
Date: Wed, 18 Jan 2017 16:20:37 -0500

>state data is likely not true.

Amen.  It's more important to give data values that are accurate than
to give values that can be validated.  The client wants to see what's
going on right now; raw, real, and not sugar coated.

Consider the case where a list is limited to 1000 entries, but as
the device is reporting the list, entries are being created and
deleted.  The result may be that more than 1000 entries are emitted.
But we would not want the device to be forced to stop at 1000 just
to avoid breaking the constraints of the data model.

The client should be prepared to handle operational data that does
not conform to the constraints of the data model.  Or more exactly,
we should be describing the constraints that may be violated and
example scenarios where it might occur.   Some constraints (identifiers)
should never be violated.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Wed, 18 Jan 2017 16:23:22 -0500

Ladislav Lhotka writes:
>Which doesn't mean that inconsistent (format of) state data is
>acceptable. My colleague develops a BGP looking glass, and it is
>really a terrible work because he has to do a lot of screen-scraping.
>Each time a vendor changes the data format, he has to update his
>software. I keep telling him that our nice protocols would save him
>from this drudgery.

This exact scenario was the motivation for our XML API all those
years ago.  Your colleague should be happy using a NETCONF/YANG-based
solution.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Wed, 18 Jan 2017 16:26:12 -0500

>You just have a separate state leaf.  These are still allowed in a 

We're trying to avoid a "separate state leaf" solution.  That's imho
the motivation for this whole exercise.

Thanks,
 Phil