Subject: Re: [Netconf] New Version Notification for draft-clemm-netconf-nmda-diff-00.txt

From: Alexander Clemm <[email-masked]>
Date: Thu, 5 Oct 2017 22:18:06 +0000 (UTC)

Hi,

We have just posted a new draft entitled “Discrepancy detection between NMDA datastores”. 

This draft defines a capability that allows to report discrepancies between management datastores that comply with the NMDA architecture. For this purpose, the draft introduces a new RPC that is defined as part of a YANG data model. We anticipate that this capability will be useful in a variety of scenarios, specifically to detect situations in which data does not propagate between datastores (e.g. <intended> and <operational>) the way it should. 

Please let us know your feedback. We hope that we can interest the working group to adopt it as a work item. 

We submitted the draft to netconf, since at the core is a new RPC, i.e. an operation that you might request via Netconf or Restconf. It does define a YANG data model and the context is NMDA; in that sense netmod might be an alternative landing target. 

Thanks

--- Alex, Yingzhen, Jeff

From: [email-masked]

Sent: Thursday, October 5, 3:02 PM

Subject: New Version Notification for draft-clemm-netconf-nmda-diff-00.txt

To: Yingzhen Qu, Alexander Clemm, Jeff Tantsura

A new version of I-D, draft-clemm-netconf-nmda-diff-00.txt has been successfully submitted by Alexander Clemm and posted to the IETF repository. Name:		draft-clemm-netconf-nmda-diff Revision:	00 Title:		Discrepancy detection between NMDA datastores Document date:	2017-10-05 Group:		Individual Submission Pages:		10 URL: https://www.ietf.org/internet-drafts/draft-clemm-netconf-nmda-diff-00.txt Status: https://datatracker.ietf.org/doc/draft-clemm-netconf-nmda-diff/ Htmlized: https://tools.ietf.org/html/draft-clemm-netconf-nmda-diff-00 Htmlized: https://datatracker.ietf.org/doc/html/draft-clemm-netconf-nmda-diff-00 Abstract: This document defines a capability that allows to report discrepancies between management datastores in Netconf or Restconf servers that comply with the NMDA architecture. The capability is based on a set of RPCs that are defined as part of a YANG data model and that are intended to be used in conjunction with Netconf and Restconf. Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. The IETF Secretariat

---

From: Robert Wilton <[email-masked]>
Date: Fri, 6 Oct 2017 17:05:43 +0100

Hi,

I've (quickly) read through this draft, some brief thought/comments below.

I think that the basic idea is reasonable, and likely to be useful, 
although it may need some tweaks.

Some of the introduction descriptive text isn't quite right: E.g. if 
<running> is changed then <intended> must reflect that change 
immediately.  I.e. a client shouldn't be able to see an change to 
<running> that hasn't yet been reflected in <intended>.   Of course you 
would expect to see any differences due to inactive config and template 
expansions between the two.  Also, the applied configuration will only 
be represented in <operational>, never <intended>.

I don't agree with the idea of detecting "fleeting" changes or the 
associated dampening option.  I think that this would potentially be a 
lot more complicated and expensive for a device to have to check. 
Instead, I would make the operation semantically the same as if the 
client performed the comparison directly.

At the moment, the compare operation probably only makes sense between 
<intended> and <operational>, particularly if there is any templating or 
inactive configuration that has come into play.

There may also be others reasons why the value in <operational> doesn't 
match <intended> even though the configuration has been successfully 
applied.  E.g. system created objects, IP addresses learned through 
DHCP, timer values learned via a protocol interaction from a designated 
master node that overrides any configured value, bugs, etc.  The origin 
metadata information is designed to help in these scenarios because the 
origin would have to be something other than "intended" in all of these 
scenarios.  So, having the option to include the origin annotations may 
be useful.

In summary, I think that this means that the system can answer the 
following two questions:
1) Which nodes in the intended configuration have not been reflected 
with the same value in <operational>?
2) Which nodes in <operational> with a origin <intended> don't exist in 
<intended>?

These questions are similar to asking what configuration has not been 
applied, or not removed, but the questions have been weakened to reflect 
what is actually externally deducible by comparing the intended and 
operational datastores.

Thanks,
Rob

On 05/10/2017 23:18, Alexander Clemm wrote:
> This draft defines a capability that allows to report discrepancies 
> between management datastores that comply with the NMDA architecture. 
> For this purpose, the draft introduces a new RPC that is defined as 
> part of a YANG data model. We anticipate that this capability will be 
> useful in a variety of scenarios, specifically to detect situations in 
> which data does not propagate between datastores (e.g. <intended> and 
> <operational>) the way it should.
>
> Please let us know your feedback. We hope that we can interest the 
> working group to adopt it as a work item.
>
> We submitted the draft to netconf, since at the core is a new RPC, 
> i.e. an operation that you might request via Netconf or Restconf. It 
> does define a YANG data model and the context is NMDA; in that sense 
> netmod might be an alternative landing target.
>
> Thanks
>
> --- Alex, Yingzhen, Jeff
>
>
> A new version of I-D, draft-clemm-netconf-nmda-diff-00.txt has been 
> successfully submitted by Alexander Clemm and posted to the IETF 
> repository. Name: draft-clemm-netconf-nmda-diff Revision: 00 Title: 
> Discrepancy detection between NMDA datastores Document date: 
> 2017-10-05 Group: Individual Submission Pages: 10 URL: 
> https://www.ietf.org/internet-drafts/draft-clemm-netconf-nmda-diff-00.txt 
> Status: 
> https://datatracker.ietf.org/doc/draft-clemm-netconf-nmda-diff/ 
> Htmlized: https://tools.ietf.org/html/draft-clemm-netconf-nmda-diff-00 
> Htmlized: 
> https://datatracker.ietf.org/doc/html/draft-clemm-netconf-nmda-diff-00 
> Abstract: This document defines a capability that allows to report 
> discrepancies between management datastores in Netconf or Restconf 
> servers that comply with the NMDA architecture. The capability is 
> based on a set of RPCs that are defined as part of a YANG data model 
> and that are intended to be used in conjunction with Netconf and 
> Restconf. Please note that it may take a couple of minutes from the 
> time of submission until the htmlized version and diff are available 
> at tools.ietf.org. The IETF Secretariat
>
>
>
> _______________________________________________
> Netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf

---

From: "Alexander Clemm" <[email-masked]>
Date: Fri, 6 Oct 2017 21:06:54 -0700

Hi Robert, 

Thank you for your feedback!  

We clearly do have <intended> and <operational> in mind, but think this can be framed general enough to apply to other datastores.  We don’t mention <running>, so I am a bit confused by that comment as to which part of the descriptive isn’t right, but we will gladly take suggestions.  

If we don’t include the dampening option, it certainly makes implementation simpler.  However, my concern would be that if we report every discrepancy, even very shortlived ones, that the usefulness of this suffers since some propagation delays will naturally occur.  A user could of course always indicate a dampening period of 0.  

Including the origin annotation as an option when reporting the differences is a good suggestion.   Can we accomplish this with yang-patch, or will we need an extension there?  

--- Alex

From: Robert Wilton [mailto:[email-masked]] 
Sent: Friday, October 6, 2017 9:06 AM
To: Alexander Clemm <[email-masked]>; Yingzhen Qu <[email-masked]>; Jeff Tantsura <[email-masked]>; [email-masked]
Subject: Re: [Netconf] New Version Notification for draft-clemm-netconf-nmda-diff-00.txt

Hi,

I think that the basic idea is reasonable, and likely to be useful, although it may need some tweaks.

Some of the introduction descriptive text isn't quite right: E.g. if <running> is changed then <intended> must reflect that change immediately.  I.e. a client shouldn't be able to see an change to <running> that hasn't yet been reflected in <intended>.   Of course you would expect to see any differences due to inactive config and template expansions between the two.  Also, the applied configuration will only be represented in <operational>, never <intended>.

I don't agree with the idea of detecting "fleeting" changes or the associated dampening option.  I think that this would potentially be a lot more complicated and expensive for a device to have to check.  Instead, I would make the operation semantically the same as if the client performed the comparison directly.

At the moment, the compare operation probably only makes sense between <intended> and <operational>, particularly if there is any templating or inactive configuration that has come into play.

There may also be others reasons why the value in <operational> doesn't match <intended> even though the configuration has been successfully applied.  E.g. system created objects, IP addresses learned through DHCP, timer values learned via a protocol interaction from a designated master node that overrides any configured value, bugs, etc.  The origin metadata information is designed to help in these scenarios because the origin would have to be something other than "intended" in all of these scenarios.  So, having the option to include the origin annotations may be useful.

In summary, I think that this means that the system can answer the following two questions:
1) Which nodes in the intended configuration have not been reflected with the same value in <operational>?
2) Which nodes in <operational> with a origin <intended> don't exist in <intended>?

These questions are similar to asking what configuration has not been applied, or not removed, but the questions have been weakened to reflect what is actually externally deducible by comparing the intended and operational datastores.

Thanks,
Rob

Hi,

Thanks

--- Alex, Yingzhen, Jeff

From: [email-masked] <mailto:[email-masked]> 

_______________________________________________
Netconf mailing list
[email-masked] <mailto:[email-masked]> 
https://www.ietf.org/mailman/listinfo/netconf

---

From: Robert Wilton <[email-masked]>
Date: Mon, 9 Oct 2017 11:13:01 +0100

Hi Alex,

On 07/10/2017 05:06, Alexander Clemm wrote:
> We clearly do have <intended> and <operational> in mind, but think 
> this can be framed general enough to apply to other datastores.  We 
> don’t mention <running>, so I am a bit confused by that comment as to 
> which part of the descriptive isn’t right, but we will gladly take 
> suggestions.
>

It was this paragraph that seemed to imply that applied configuration 
appears in <intended>:

Some of this is
    entirely expected, as there may be a time lag between when a
    configuration is applied to the device and reflected e.g. in
    <intended>, until when it actually takes effect and is reflected in
    <operational>.

Perhaps just changing it as follows may be more clear:

Some of this is
    entirely expected, as there may be a time lag between when
    configuration is given to the device and reflected in
    <intended>, until when it actually applied by the device and
    is reflected in <operational>.

> If we don’t include the dampening option, it certainly makes 
> implementation simpler.  However, my concern would be that if we 
> report every discrepancy, even very shortlived ones, that the 
> usefulness of this suffers since some propagation delays will 
> naturally occur.  A user could of course always indicate a dampening 
> period of 0.
>
So the purpose of your dampening is to filter out changes that are 
configuration changes that may still be in the act of being applied?

I think that your current definition of dampening may require some 
additional per node time tracking to implement correctly.

A similar, but cheaper way to implement this could be to filter the list 
of differences to exclude any nodes that have been re-configured within 
the last X seconds.  This isn't quite the same, but potentially good enough.

But, overall, I'm still not convinced whether the dampening is a good 
idea or not.  Or at least whether it shouldn't be something that is 
optional to implement in the RFC.

> Including the origin annotation as an option when reporting the 
> differences is a good suggestion.   Can we accomplish this with 
> yang-patch, or will we need an extension there?
>

I'm not sure, but it looks like YANG patch is allowed to include meta 
data encoded via RFC 7952.

Thanks,
Rob

> --- Alex
>
> *From:*Robert Wilton [mailto:[email-masked]]
> *Sent:* Friday, October 6, 2017 9:06 AM
> *To:* Alexander Clemm <[email-masked]>; Yingzhen Qu 
> <[email-masked]>; Jeff Tantsura <[email-masked]>; 
> [email-masked]
> *Subject:* Re: [Netconf] New Version Notification for 
> draft-clemm-netconf-nmda-diff-00.txt
>
> Hi,
>
> <running> that hasn't yet been reflected in <intended>.   Of course 
> you would expect to see any differences due to inactive config and 
> template expansions between the two.  Also, the applied configuration 
> will only be represented in <operational>, never <intended>.
>
> <intended> and <operational>, particularly if there is any templating 
> or inactive configuration that has come into play.
>
> There may also be others reasons why the value in <operational> 
> doesn't match <intended> even though the configuration has been 
> successfully applied.  E.g. system created objects, IP addresses 
> learned through DHCP, timer values learned via a protocol interaction 
> from a designated master node that overrides any configured value, 
> bugs, etc. The origin metadata information is designed to help in 
> these scenarios because the origin would have to be something other 
> than "intended" in all of these scenarios.  So, having the option to 
> include the origin annotations may be useful.
>
> applied, or not removed, but the questions have been weakened to 
> reflect what is actually externally deducible by comparing the 
> intended and operational datastores.
>
> Thanks,
> Rob
>
>     This draft defines a capability that allows to report
>     discrepancies between management datastores that comply with the
>     NMDA architecture. For this purpose, the draft introduces a new
>     RPC that is defined as part of a YANG data model. We anticipate
>     that this capability will be useful in a variety of scenarios,
>     specifically to detect situations in which data does not propagate
>     between datastores (e.g. <intended> and <operational>) the way it
>     should.
>
>     i.e. an operation that you might request via Netconf or Restconf.
>     It does define a YANG data model and the context is NMDA; in that
>     sense netmod might be an alternative landing target.
>
>     Thanks
>
>     --- Alex, Yingzhen, Jeff
>
>     A new version of I-D, draft-clemm-netconf-nmda-diff-00.txt has
>     been successfully submitted by Alexander Clemm and posted to the
>     IETF repository. Name: draft-clemm-netconf-nmda-diff Revision: 00
>     Title: Discrepancy detection between NMDA datastores Document
>     date: 2017-10-05 Group: Individual Submission Pages: 10 URL:
>     https://tools.ietf.org/html/draft-clemm-netconf-nmda-diff-00
>     Htmlized:
>     based on a set of RPCs that are defined as part of a YANG data
>     model and that are intended to be used in conjunction with Netconf
>     and Restconf. Please note that it may take a couple of minutes
>     from the time of submission until the htmlized version and diff
>     are available at tools.ietf.org. The IETF Secretariat
>
>
>
>

---

From: "Alexander Clemm" <[email-masked]>
Date: Thu, 12 Oct 2017 11:01:43 -0700

Hi Rob,

Thank you!  

A couple of responses inline, <ALEX>

--- Alex

From: Robert Wilton [mailto:[email-masked]] 
Sent: Monday, October 9, 2017 3:13 AM
To: Alexander Clemm <[email-masked]>; 'Yingzhen Qu' <[email-masked]>; 'Jeff Tantsura' <[email-masked]>; [email-masked]
Subject: Re: [Netconf] New Version Notification for draft-clemm-netconf-nmda-diff-00.txt

…

It was this paragraph that seemed to imply that applied configuration appears in <intended>:

Some of this is

Some of this is

<ALEX> Sure, will update to make this clearer.  I guess I was using “applied” as an overloaded term. </ALEX>

So the purpose of your dampening is to filter out changes that are configuration changes that may still be in the act of being applied?

I think that your current definition of dampening may require some additional per node time tracking to implement correctly.

A similar, but cheaper way to implement this could be to filter the list of differences to exclude any nodes that have been re-configured within the last X seconds.  This isn't quite the same, but potentially good enough.

But, overall, I'm still not convinced whether the dampening is a good idea or not.  Or at least whether it shouldn't be something that is optional to implement in the RFC.

<ALEX> Making this optional is certainly an option.  Certainly dampening makes implementation harder.  However, I do think that it is a worthwhile feature, since it can be used to weed out “false positives” and direct attention to inconsistencies that persist - this way, facilitating e.g. service-assurance related operations.  

There are different ways to frame it and one of the questions will be to decide which should be the preferred option.  Focusing on items that have been reconfigured within X seconds seems to point to another possible feature on how one might articulate a filter, but not what we had in mind here.  I think what may provide a good compromise between usefulness and ease of implementation would be to have a dampening parameter refer not to a “precise” time, but to be more indicative of a range.  For example, it could be the interval between a first snapshot of discrepancies, and a subsequent second snapshot.  Straightforward semantics, but different from saying “it has to persist for exactly x seconds”.  

</ALEX>

I'm not sure, but it looks like YANG patch is allowed to include meta data encoded via RFC 7952.

<ALEX> Need to check; if we can do that this will be straightforward and we should include this.  Thanks for the suggestion! </ALEX>

…