Subject: [Netconf] review of draft-ietf-netconf-yang-push-11

From: Martin Bjorklund <[email-masked]>
Date: Tue, 28 Nov 2017 10:37:35 +0100 (CET)

Hi,

I have now reviewed draft-ietf-netconf-yang-push-11.  I have one
somewhat more important comment, and several others.

Important issue:

o  3.10

  I don't think the proposed YANG extension is the correct solution to
  the stated problem, for several reasons:

    1.  In most cases, this is not a property of the data model, but
        of the implementation, and possibly even the deployment.  So
        having a YANG extension statement is not a good solution.

    2.  With NDMA, the same schema node is present in different
        datastores.  It might be the case that an implementation
        supports on-change for the node in a configuration datastore,
        but not in operational.  Again, marking a node in the schema
        is not a good solution.

    3.  Since the on-change property is implementation dependent, it
        means the information will be available to clients only in
        deviation modules.  This is quite an expensive and complicated
        way to pass the information to the clients.

  An alternative solution could be to have an ordered list of
  instance-identifiers that list this property, per datastore, for
  example:

   <entry>
     <path>/sys:system/sys:system-time<path>
     <notifiable-on-change>false</notifiable-on-change>
   </entry>
   <entry>
     <path>/sys:system<path>
     <notifiable-on-change>true</notifiable-on-change>
   </entry>

  Yet another alternative would be to leave this to future work.

Other comments:

o  2

  The document uses the term "YANG datastore" (it is even in the title
  of the document).  This term is not used elsewhere, and it is not
  defined in this document.  I suggest you change this to simply
  "datastore".

  Also, I think you should "import" the term "datastore" from
  draft-ietf-netmod-revised-datastores, instead of having a slightly
  different term in this document.

  It is also unfortunate that you use the term "data node" in a
  different meaning than RFC 7950.  Maybe use "datastore node"
  instead?

o  3.1

  I'm not sure I understand the dampening period concept.  Let's
  assume that the dampening period is 10s.  Then changes happen at
  times:

    2  3  4  11  13  14  15

  From the description, it seems I would receive 4 notifications, from
  times:

    2  (containing only change from 2)
    12 (containing change from 3,4,11)
    13 (containing only change from 13)
    23 (containing changes from 14,15)

  Is this correct?

  In any case, I suggest the description in the YANG module is
  clarified - currently the RFC text contains more details than the
  YANG module.

o  3.2

  The text says:

     Therefore, in order to minimize the number of subscription
     iterations between subscriber and publisher, dynamic
     subscriptions SHOULD support a simple negotiation between
     subscribers and publishers for subscription parameters.

  To whom is this "SHOULD" directed?  I mean, dynamic subscriptions as
  specified in this draft does indeed support simple "negotiation".
  Maybe s/SHOULD support/supports/ ?

o  3.4

  The text says:

    Please see [promise] for
    more on the transactional basis underlying the publisher and
    subscriber interactions within this document.

  So I did that.  But I didn't find any text in the referenced
  document that talked about "the transactional basis".

  I suggest you remove this sentence from the draft.

o  3.5

  The text says:

     A publisher MUST support XML encoding and MAY support other
     encodings such as JSON encoding.

  I don't think this is correct.  A RESTCONF sever is not required to
  support XML.  I think you should remove this sentence.

o  3.5.1

     In a periodic subscription, the data included as part of an update
     corresponds to data that could have been simply retrieved using a get
     operation and is encoded in the same way.

  What is "a get operation"?  Do you mean RESTCONF GET or NETCONF
  <get/> or something else?  Whatabout other protocols?

  (In 3.6, you use simply "a GET" for probably the same thing.)

o  3.6

     Subscription policy specifies both the selection filters and the
     datastores against which these selection filters will be applied.
     The result is the push of information necessary to remotely maintain
     an extract of the publisher's datastore.

  It seems this paragraph defines the term "Subscription policy".  But
  this term is not used in the document.  What does it means that the
  result of a policy is "the push of information"?

  I think I don't understand what this paragraph tries to tell me.

o  3.6

     o  xpath: An xpath selection filter is an XPath expression which may
        be meaningfully applied to a datastore.

   What does "meaningfully applied" mean?

o  3.6

     Selection filters are not intended to be used to filter objects
     based on a non-key property.  Supporting non-key property
     filtering so would have a number of implications that would
     result in significant complexity.

     [...]

     the goal is to
     provide equivalent capabilities to what is available with a GET.

  In GET you can filter on "non-key properties".

  I think you should remove the text about "non-key properties".  If
  anything, I think you can allow an implementation to reject a filter
  that would be too complex to implement / evaluate (in fact I think
  the text already allows a server to reject such filters.)

  With the current text, is this filter ok:

     /interfaces/interface[contains(name, "eth")]

  It filters on a key, so it should be ok, right?

o  3.7

  The XML examples are not using the correct XML namespace for the
  nodes from the "ietf-interface" module.

  The YANG Patch example also shows an interesting effect in the
  "patch-id" and "edit-id" leafs.  I think the draft should mention
  how implementations are suppose to fill in these leafs.

  The "target" leaf is not specified correctly.  It should be:

         <target>/ietf-interfaces:interfaces-state</target>

o  3.8

  The example has:

   <establish-subscription
       xmlns="urn:ietf:params:xml:ns:yang:ietf-subscribed-notifications"
       xmlns:yp="urn:ietf:params:xml:ns:yang:ietf-yang-push">
      <yp:datastore>
        <yp:source xmlns="urn:ietf:params:xml:ns:yang:ietf-datastores">

  This doesn't match the YANG model; there is no container called
  "datastore" in the model.

  Further is has:

        <yp:subtree-filter netconf:type="xpath"
            xmlns:ex="http://example.com/sample-data/1.0"
            select="/ex:foo"/>

  a subtree-filter of type xpath?  I think ot should be:

        <yp:xpath-filter xmlns:ex="http://example.com/sample-data/1.0">
           /ex:foo
        </yp:xpath-filter>

  Also, it has:

        <yp:source xmlns="urn:ietf:params:xml:ns:yang:ietf-datastores">
          operational
        </yp:source>

  which should be:

        <yp:source xmlns:ds="urn:ietf:params:xml:ns:yang:ietf-datastores">
          ds:operational
        </yp:source>

o  3.9

  This section lists three cases for which:

     the error identity "data-unavailable" SHOULD be returned.

  One of the cases is:

    o  the authorization privileges of a receiver change over the course
       of the subscription.

  But how can a server know this when "establish-subscription" is sent?

o  3.9

     The contextual authorization model for data in YANG datastores is
     the NETCONF Access Control Model [RFC6536bis], Section 3.2.4.

  Did you mean s/contextual/conceptual/ ?

o  3.11.1

  OLD:

       If
       this is not possible and the synch-on-start option is configured,

  NEW:

       If
       this is not possible and the "no-synch-on-start" option is not
       present for the subscription.

  (fixes incorrect name of leaf, and also covers dynamic
  subscriptions)

o  4.3.2

     A subscription-id MUST be transported along with the subscribed
     contents.

  Then the leaf "subscription-id" should be mandatory.

     A "time-of-update" which represents the time an update record
     snapshot was generated.  A receiver MAY assume that a publisher's
     objects have these pushed values at this point in time.

  Should "time-of-update" be mandatory?

  How is "time-of-update" different from "eventTime" in the
  notification?

o  4.3.2

     If the application detects an informational discontinuity

  What is an "informational discontinuity"?

o  4.4.1 / 4.4.2

  The XML examples are broken; compare with my comments for 3.8.

o  5

  OLD:

    "This module contains conceptual YANG specifications
     for YANG push.";

  NEW:

    "This module contains YANG specifications for YANG push.";

o  5 - identities

    identity qos-unsupported {
      base sn:error;
      description
        "Subscription QoS parameters not supported on this platform.";
    }

  This identity is not mentioned anywhere in the text.  Instead of
  having this identity, wouldn't it be better to define a feature for
  "qos", and mark the nodes you have in mind with an if-feature?

    identity on-change-unsupported {
      base sn:error;
      description
        "On-change not supported.";
    }

  When will this identity be used?  There is already a feature
  "on-change" and corresponding if-feature statements.

    identity on-change-synch-unsupported {
      base sn:error;
      description
        "On-change synch-on-start and resynchonization not supported.";
    }

  The leaf is called "no-sync-on-start", which implies that sync on
  start is the default.  So when will this identity be used?

    identity reference-mismatch {
     base sn:error;
      description
       "Mismatch in filter key and referenced yang subtree.";
    }

  I don't understand the description of this identity.  Please
  clarify.

     identity datatree-size {

  Should it be "result-too-big" or something?

    identity no-such-datastore {

  I think this one should be removed.  The normal "invalid-value"
  error-tag covers this error.

      identity custom-datastore {
        base ds:datastore;
        description
          "A datastore with boundaries not defined within
           draft-ietf-netmod-revised-datastores";
      }

  This identity needs to be removed.  If someone defines a custom
  datastore, it would get a specific identity, and that identity can
  be used as "source".

o  5 - "change-type"

  The descriptions of the enums need to be improved.  This is about
  reporting a change in a datastore.  The value "create" is described
  as:

        description
          "Create a new data resource if it does not already exist.  If
          it already exists, replace.";

  Also, the description of the typedef has:

      "RFC 8072 section 2.5, with a delta that it is ok to receive
      ability create on an existing node, or receive a delete on a
      missing node.";

  But what does this mean?  The type is used to *exclude* some changes
  from a yang patch record.

o  5 - selection filter

  The XPath expression is not properly defined.

  OLD:

          "This parameter contains an XPath expression identifying the
          portions of the target datastore to retrieve.";

  NEW:

          "This parameter contains an XPath expression identifying the
           portions of the target datastore to retrieve.

           If the expression returns a node-set, all nodes in the
           node-set are selected by the filter.  Otherwise, if the
           expression does not return a node-set, the filter
           doesn't select any nodes.

           FIXME: (*)

           The expression is evaluated in the following XPath context:

             o  The set of namespace declarations are those in scope on
                the 'xpath-filter' leaf element.

             o  The set of variable bindings is empty.

             o  The function library is the core function library, and
                the XPath functions defined in section 10 in RFC 7950.

             o  The context node is the root node of the target
                datastore.

  (*) - We need to describe when the filter is evaluated.  This is
  also true for the subtree filter.  Is it evaluated when the
  subscription is started and explicitly modified, or everytime a
  change is detected?

  [Side note: this description is also missing from the "xpath-filter"
  leaf in "get-data" in draft-ietf-netconf-nmda-netconf.  I have
  updated the description in that draft.]

o  5 - terminology

  The term "agent" is used a couple of times.  Use "publisher" or
  "server" instead.

o  5 - dampening

          leaf dampening-period {
            type yang:timeticks;
            mandatory true;

   Should this instead be:

          leaf dampening-period {
            type yang:timeticks;
            default "0";

   So that the request is the same if the "on-change" feature is
   support or not.

o  5 - no-synch-on-start

             When
             present, pushing a full selection per the terms of the
             selection filter MAY NOT be done for this subscription.

   Did you mean s/MAY NOT/MUST NOT/ ?

   MAY NOT is not a 2119 term.

o  5 - QoS

  Why are the qos parameters defined in yang push, and not in
  subscribed notifications?  It seems that they are generic.

o  5 - establish-subscription params

  I think a "when" expression should be added to the first augment:

    augment "/sn:establish-subscription/sn:input" {
      when "sn:target/yp:datastore/yp:source";

o  5 - push-update

  What does the presence of "updates-not-sent" mean in "push-update"?
  "push-update" contains a snapshot, not a diff, so why is
  "updates-not-sent" present?

o  5 - push-change-update

        "This contains an incremental set of datastore changes needed
         to update a remote datastore starting at the time of the
         previous update, per the terms of the subscription.

   What is an "incremental set"?  Probably s/incremental set/set/

o  General

  Use double quotes for node names.  "push-update", "subscription-id"
  etc.  Quotes are currently used inconsistently, and it makes the
  text harder to read.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Tue, 28 Nov 2017 09:10:51 -0800

>   Yet another alternative would be to leave this to future work.

I would prefer to leave this to future work.
I agree this is an implementation property, and not a data model property.

Andy

> _______________________________________________
> Netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf
>

---

From: "Eric Voit (evoit)" <[email-masked]>
Date: Wed, 29 Nov 2017 17:37:25 +0000

Hi Martin,

Thanks for the comments.   Many thoughts in-line...

> From: Martin Bjorklund, November 28, 2017 4:38 AM
> 
> Hi,
> 
> 
> I have now reviewed draft-ietf-netconf-yang-push-11.  I have one somewhat
> more important comment, and several others.
> 
> Important issue:
> 
> o  3.10
> 

In general we agree there are several alternatives to solve this issue.  And any mechanism supported by the WG would be fine.   

My read of your proposal below is that it does reuse elements of 3.10.  For example, I *think* you are proposing hierarchical inheritance of on-change property markings of the parent rather than making every instance.  Because marking every instance within a routing table would be prohibitively expensive.  So based on whatever commonality we can establish across the proposed alternatives, perhaps we can come to a common way to set this information.    If not, we can back out on-change marking as something implementation-specific for this current specification.

Stepping back a minute, it is good to think about who would use this on-change support data.  Application developers are going to make subscription requests.  And they are far more likely to want to design based on the schema level information exposed rather than instance data.   Driven by that constraint, the current proposal is framed as is -- mostly since there was no interest in Balazs'
https://tools.ietf.org/html/draft-lengyel-netmod-schema-annotation-00   
which was designed to support application developers needing this information.   Without this draft, at the time this was proposed the next-best thing was an extension which works based on the established pattern of deviations.   

To cover this issue, I have re-opened YP#10 to track and resolve...
https://github.com/netconf-wg/yang-push/issues/10 

Below are some specific pros/cons I see...

>         having a YANG extension statement is not a good solution.

Fully agree this can be implementation level issue.  But then again, so is any deviation.  

So to cover "property of the implementation" case, the proposed extension would typically be included in deviations file per implementation.  Such an approach does mirror existing mechanisms for deviations, and has the benefit of not requiring another a new totally separate file to document the desired behavior.  

As for times this might be deployment specific, we saw potential capacity issues determining whether something might not be on-change subscribable.   Here it would be up to the implementation to determine whether to use the error "on-change-unsupported" from YANG Push, or the error "insufficient-resources" from subscribed notifications.  A choice between the two can be driven based on whether a deployment would *ever* have capacity for the requested subscription.

>         but not in operational.  Again, marking a node in the schema

As the current solution was developed before NMDA, we didn't consider differences in on-change support for different datastores.  So this is a real consideration.

If for a minute we assume we are going to follow the current approach in the draft, the extension proposed in 3.10 could have a default of the operational datastore.  And we could add the option of including a list of datastores in the extension whenever on-change for something else is needed/supported..

>         way to pass the information to the clients.

Visibility to application developers was one reason we went down this path.   Any solution needs to frame this visibility as a key element.   Considering this, I actually see a need both for the schema based solution as framed, and the possibility of an instance based mechanism if someone wanted the lower level of granularity.  Per above, I don't think instance data within all instance objects like routing entries is possible anyplace datastore size becomes a factor.

>      <notifiable-on-change>true</notifiable-on-change>

This could work.   Several questions which would need to be addressed with this approach:

(1) how would application developers easily know what objects are on-change subscribable?

(2) If this is per-instance, is the property inherited from a parent?   If no, how do we deal with the size of the resulting information?

(3) If this is per-instance, how is this information populated?  Don't you need schema level guidance recorded somewhere anyway?

(4) Are there any issues with things like Schema mount?

>   Yet another alternative would be to leave this to future work.

If necessary, we can push this out.   I am hoping we can come up with something here, especially as the on-chance implementations I have seen have all used phased introduction of on-change for different schema objects.   

>   defined in this document.  I suggest you change this to simply

In general, I am fine with making the change to "Datastore" throughout the document.  For the title, I think it better to keep YANG to allow casual readers easy access to the context.

>   different term in this document.

Can do.

>   different meaning than RFC 7950.  Maybe use "datastore node"

Can do.
 
>   assume that the dampening period is 10s.  Then changes happen at

Should be 2, 12, & 22.   I have updated the definition (below) in a way which hopefully clarifies the proper behavior

         +  Dampening period: In an on-change subscription, detected object changes should be sent as quickly as possible.  However without adequate protections, a rapid series of object changes might exhaust of resources in the publisher or receiver.  In order to protect against that, a dampening period MAY be used to specify the interval which must pass before successive update records for the same subscription are generated for a receiver.  The dampening period collectively applies to the set of all data nodes selected by a single subscription and sent to a single receiver.  This means that when there is a change to a subscribed object, an update record containing that object is created either immediately when no dampening period is in effect, or at the end of a dampening period.  A dampening period is reset every time a new notification message is passed to transport.
 
>   clarified - currently the RFC text contains more details than the

I have updated the YANG module text as:

"Specifies the interval which must pass before successive update records for the same subscription are generated for a receiver.  The dampening period collectively applies to the set of all data nodes selected by a single subscription and sent to a single receiver.  This means that when there is a change to a subscribed object, an update record containing that object is created either immediately when no dampening period is in effect, or at the end of a dampening period.  A dampening period is reset every time a new notification message is passed to transport.  A dampening period is reset every time a new notification message is passed to transport.  A zero value indicates no dampening period, and all subscribed object changes are sent immediately."

>   Maybe s/SHOULD support/supports/ ?

Yes.  Will remove the SHOULD, showing just "supports".

>   I suggest you remove this sentence from the draft.

As this is informative, I can remove.  

>   support XML.  I think you should remove this sentence.

ok

>   <get/> or something else?  Whatabout other protocols?

As YANG Push is intended to be transport independent, it is better not to provide a specific transport.   Instead, how about the following text:

In a periodic subscription, the update record MUST include the datastore nodes which would have been retrieved using an equivalent datastore selection operation over that subscription's transport. 
 
>   (In 3.6, you use simply "a GET" for probably the same thing.)

If you are good with the text above, I will change 3.6 to:

" to what is available with a datastore selection operations".

>   this term is not used in the document. 

We don't intend to define a new term in this paragraph.  And you are correct, policy only appears elsewhere in the document as part of the YANG model group names.    (more below)

>  What does it means that the  result of a policy is "the push of information"?
> 

What if I changes the first paragraph of Section 3.6 to:

>    What does "meaningfully applied" mean?

There are plenty of valid xpath expressions which don't select data nodes.  How about: 

" xpath: An xpath selection filter is an XPath expression which references data nodes. When applied to a datastore, it is the results of this expression which will be pushed."

>   the text already allows a server to reject such filters.)

The issue with non-key properties is how to deal with the creation and deletion of objects when a non-key property goes in/out of the selection.   Others were uncomfortable with things like passing along a patch showing a node was deleted, when in reality the property simply changed from meeting the selection filter to failing the selection filter.  So people wanted to default to the simpler case of key properties for selection.

Based on that I can delete the words "but the goal is to provide equivalent capabilities to what is available with a GET".  

>   It filters on a key, so it should be ok, right?

Yes

>   how implementations are suppose to fill in these leafs.

Will add how to populate.  In summary, sequential numbering of "edit-id" was from the RFC-8072.   And a null "patch-id" is because patch-id is mandatory in RFC-8072, and was originally supposed to be used for debugging of failed datastore write operations.  That really isn't the case here, plus we have subscription-id and other object available.

>          <target>/ietf-interfaces:interfaces-state</target>

Good catch.  Making change.

>         <yp:xpath-filter xmlns:ex="http://example.com/sample-data/1.0">

Yes

>         <yp:source xmlns:ds="urn:ietf:params:xml:ns:yang:ietf-datastores">

At the last IETF, Kent pointed us to YANG Lint which should allow us to do example checking in ways not previously known to us.    I will install and attempt to use these to clear the example items you have listed.  

>   But how can a server know this when "establish-subscription" is sent?

It cannot know this at "establish-subscription".   So a publisher will have to track whether the permissions on subscribed objects change.   How is left to implementations.

>   Did you mean s/contextual/conceptual/ ?

Will make that change.

>   (fixes incorrect name of leaf, and also covers dynamic

Yes, good catch.  Will fix.

>   Then the leaf "subscription-id" should be mandatory.

The requirement is that a subscription-id must be in the notification message, this doesn't mean that the subscription-id must be within the two new notifications.   The reason for this is that when we add support for the notification-messages draft, but having the subscription-id optional in the push-update and push-change-update, we can enable implementations which do not duplicate this header item.  (And a duplication would be forced with the future header if we made this mandatory.)

>   Should "time-of-update" be mandatory?

Two reasons it is not:
(a) Other vendors have worried they can only support message-time, rather than notification-time.  
(b) Notification-time is the generalized name for "time-of-update" .  Having "time-of-update" as optional allows for eventual migration to common headers of  draft-ietf-netconf-notification-messages without information duplication

>   How is "time-of-update" different from "eventTime" in the

I think that answer is above.  But there is some good guidance here from draft-ietf-netconf-notification-messages.  The new draft ultimately will define and support the following times:

message-time: 
      "Header information consisting of time the message headers were placed generated prior to being sent to transport";

notification-time
      "Header information consisting of the time an originating process created the notification."

Notification-time should be equivalent to eventTime from RFC-5277.  But some other vendors have said they really only have message-time, and this is what they are populating in eventTime even though it doesn't explicitly fit the definition.  With the new definitions, at least they should be able to populate what they explicitly have.

>   What is an "informational discontinuity"?

Will change to:
  If the application detects any incompleteness in set of objects placed into a "push-update" or "push-change-update"

>   The XML examples are broken; compare with my comments for 3.8.

Will do.  I am hoping a new attempt using yanglint helps pick out everything I missed by hand previously.

>     "This module contains YANG specifications for YANG push.";

ok

>   "qos", and mark the nodes you have in mind with an if-feature?

It is possible to expose QoS explicitly as a feature.   I will make that addition if you are ok with my other QoS comment described below about subscribed-notifications.

But even in that case if QoS is not supported, and someone includes QoS objects in an "establish-subscription" we need this identity as an error.  This error allow an explicit identification of what was wrong in the RPC (such as a DSCP provided is not supported by the Publisher).  I will include this in the Identity description.

>   "on-change" and corresponding if-feature statements.

Per our discussion above, this can be used if an RPC asks for on-change for an object which is not available at a platform deployment (either marked in the schema, or a specific deployment doesn't support an object which is included).   I will enhance the description based on the discussion on this earlier in this thread.
 
>   start is the default.  So when will this identity be used?

Can be used in two places:

(1) Will be used if an RPC asks to synch on start, but it can't be supported for any reason (e.g., no nodes identifiable within the selection filter will ever be support on-change).   Will enhance the definition.

(2) The resynch RPC is invoked on a periodic subscription-id, or on an on-change subscription which can't support synchronization.

>   I don't understand the description of this identity.  Please

Will clarify description to explain that the key provided with the filter does not match the data type of the referenced yang subtree

>   Should it be "result-too-big" or something?

Yes.   I will change the name.

>   datastore, it would get a specific identity, and that identity can

Yes, any new custom datastore will get a new identity, but if that new identity uses this custom-datastore one as a base, then we have an umbrella identity which is a handle just for the custom ones.  That was the intent of this.   If you don't think that an identity which acts as a bundling mechanism is useful, we can remove.

>   from a yang patch record.

This is to identify churn within a datastore during a dampening period.  Two examples of when this is needed:

(1) For security applications, it is an absolute requirement that you know if a node was added and then removed during a dampening period.   For an example, a hacker adds a "permit any any" which is then quickly removed before the dampening period ends.  Remote applications need to know that churn occurred.

(2) For network management applications when a physical interface is going down and then quickly back up.   If the up/down is transient, how do you know that this occurred if there is no notification that churn occurred?

To support this, the YANG patch definition is loosened to allow the new creation on an existing node (based on what resulted from the last patch), or a delete on a missing node.  This acts as an indication that some churn in the datastore happened even though the data is in the same state as a per the previous update.  And if an application get to sees this indication when comparing to the previous state, they have the option of looking more closely at some logs on the publisher to determine what actually happened.

I can see that this behavior is under-described in the text.   I will put a small section between sections 3.10 & 3.11 describing this.  The title will be "Identification of a transient change within dampening period"

>   updated the description in that draft.]

I will update the descriptions to match your draft.

As for when the evaluation occurs, I am not sure what you mean.  Every time a change occurs, you need to see if the changed object would have fallen within the selection.  There is no attempt to support an event-condition-action context.

>   The term "agent" is used a couple of times.  Use "publisher" or

Will update

>    So that the request is the same if the "on-change" feature is

I like it, will update.

>    MAY NOT is not a 2119 term.

Will change to MUST NOT.

>   subscribed notifications?  It seems that they are generic.

This question has come up before.  Basically current 5277 event streams haven't shown need for QoS to date.   We know that QoS has been requested to handle congestion issues for YANG push.  So for simplicities sake, we bundled all QoS elements together in YANG push where we knew there was demonstrable need.    If someone really is asking for these QoS constructs for events, we can revisit.

>       when "sn:target/yp:datastore/yp:source";

So this is saying that the named datastore must exist before allowing the augment.   Works for me.

>   "updates-not-sent" present?

This is to indicate a system error where not all objects can be extracted from the datastore.   (e.g., the routing table got *lots* bigger, and the periodic extract cannot be performed as expected.)  At least some implementations have seen this.   I will add some clarifying text.

>    What is an "incremental set"?  Probably s/incremental set/set/

I can change to "set".

>   etc.  Quotes are currently used inconsistently, and it makes the

Will make the fix.

Thanks again for all the excellent thoughts!
Eric

---

From: Andy Bierman <[email-masked]>
Date: Wed, 29 Nov 2017 10:18:13 -0800

On Tue, Nov 28, 2017 at 1:37 AM, Martin Bjorklund <[email-masked]> wrote:

> ....

Seems to me (from a client POV) that I want the dampening to apply
to the entire subscription, not to each node within the subscription.
I want "at most, 1 notification per second".  I don't see why I would want
to be told about an individual data node once per second.  I could still
get 10,000 events/sec but for different data nodes.  The receiver does not
really care whet nodes are being reported in each notification. The goal
is to simply limit the network and processor load.

>From a server POV, I do not want a timer on every data node instance
and complex code to construct the next on-change notification.

RMON handles dampening very differently.
The rising and falling thresholds are used to arm and re-arm an event
trigger.
The time between changes is not used at all to determine how many events to
send.
(Not suggesting all yang-push use-cases are threshold-based.)

IMO, the operator should put events that require low-latency into a
separate subscription,
and the dampening period should apply to the entire subscription.

 ...

/martin

Andy

---

From: Kent Watsen <[email-masked]>
Date: Wed, 29 Nov 2017 19:03:46 +0000

Hi Andy,

>> <SNIP/>
> The rising and falling thresholds are used to arm and re-arm an event trigger.
> The time between changes is not used at all to determine how many events to send.
> IMO, the operator should put events that require low-latency into a separate subscription,

I think that this is already the case.  At least Section 3.1 says:

  The dampening period collectively applies to the set of all data
  nodes of a single subscription.

and Section 3.3 says:

   In cases where a subscriber wants to have separate dampening periods
   for different objects, multiple subscriptions with different objects
   in a selection filter can be created.

That said, the description statement for "dampening-period" could be
improved:

OLD

     "The shortest time duration which is allowed between the
       creation of independent yang object update messages.
       Effectively this is the amount of time that needs to have
       passed since the last update.  Zero indicates no delay.";

NEW

     "The shortest time duration allowed between two on-change
       update messages.   Effectively this is the amount of time that
       needs to have passed since the last update.  Zero, the default,
       indicates no delay.";

K.  // contributor

---

From: Alexander Clemm <[email-masked]>
Date: Wed, 29 Nov 2017 20:03:55 +0000

Yes, thank you for the comments!  

As the mail is long, rather than commenting inline, let me make just a few additional replies in addition to Eric's:

- On the topic of how to mark on-change notifiable YANG objects:

Yes, clearly what is on-change notifiable and what is not could differ by implementation.  We thought that asking implementors to define implementation-specific modules that would indicate the on-change notifiability through deviations would make sense, since this is really one of the function of deviations - indicate what is implementation-specific.  In that sense, the current solution does make sense to me and seems to be in the spirit of how implementation-specific YANG modules should be used.  

That said, the other proposal to have a "meta-module" that contains a list of objects for which on-change notification is supported could be certainly be an alternative.  However, the notifiability should be indicated per node at the schema level, not for each individual instance.  We would not want to have to enumerate individual list entries, for example. Also, are there additional considerations to accommodate schema-mount in that case?  

Another thought, perhaps on-change notifiability is something that could instead also be included in the YANG library.  Each module in the library would then list the nodes for which on-change is notifiable on a given server.  

Deferring this until later might also an option.  I am sure there will be other annotations/markings for objects that we will come up with our time.  However, one issue is that by default we did not want to make assumptions about on-change notifiability being supported.  Rather, when it is supported, it would be specifically indicated; the default would be that it's not supported.  If we defer the solution until later, then this won't work.  If we wanted to defer, it would seem what we should do is still propose a solution informally in an appendix, and indicate that this may be later replaced by another solution.  We would put the YANG extension into an own separate module.  

- on the Subscription Policy term (3.6).  Sure, we don't need to introduce a new term, but really what the parameters describe is in fact a policy.  We could of course call them "datastore subscription specific configuration parameters", or "datastore subscription specific behavior configuration" , but this sounds a bit awkward to me.  

- on the "no-such-datastore" eror tag:  true, invalid-value would cover this, but why be general when we can be specific?

--- Alex

> -----Original Message-----
> From: Netconf [mailto:[email-masked]] On Behalf Of Eric Voit
> (evoit)
> Sent: Wednesday, November 29, 2017 9:37 AM
> To: Martin Bjorklund <[email-masked]>; [email-masked]; Balazs Lengyel
> <[email-masked]> ([email-masked])
> <[email-masked]>; Andy Bierman ([email-masked])
> <[email-masked]>
> Subject: Re: [Netconf] review of draft-ietf-netconf-yang-push-11
> 
> Hi Martin,
> 
> In general we agree there are several alternatives to solve this issue.  And
> any mechanism supported by the WG would be fine.
> 
> My read of your proposal below is that it does reuse elements of 3.10.  For
> example, I *think* you are proposing hierarchical inheritance of on-change
> property markings of the parent rather than making every instance.  Because
> marking every instance within a routing table would be prohibitively
> expensive.  So based on whatever commonality we can establish across the
> proposed alternatives, perhaps we can come to a common way to set this
> information.    If not, we can back out on-change marking as something
> implementation-specific for this current specification.
> 
> Stepping back a minute, it is good to think about who would use this on-
> change support data.  Application developers are going to make subscription
> requests.  And they are far more likely to want to design based on the
> schema level information exposed rather than instance data.   Driven by that
> constraint, the current proposal is framed as is -- mostly since there was no
> interest in Balazs'
> which was designed to support application developers needing this
> information.   Without this draft, at the time this was proposed the next-best
> thing was an extension which works based on the established pattern of
> deviations.
> 
> So to cover "property of the implementation" case, the proposed extension
> would typically be included in deviations file per implementation.  Such an
> approach does mirror existing mechanisms for deviations, and has the
> benefit of not requiring another a new totally separate file to document the
> desired behavior.
> 
> As for times this might be deployment specific, we saw potential capacity
> issues determining whether something might not be on-change
> subscribable.   Here it would be up to the implementation to determine
> whether to use the error "on-change-unsupported" from YANG Push, or the
> error "insufficient-resources" from subscribed notifications.  A choice
> between the two can be driven based on whether a deployment would
> *ever* have capacity for the requested subscription.
> 
> As the current solution was developed before NMDA, we didn't consider
> differences in on-change support for different datastores.  So this is a real
> consideration.
> 
> If for a minute we assume we are going to follow the current approach in the
> draft, the extension proposed in 3.10 could have a default of the operational
> datastore.  And we could add the option of including a list of datastores in the
> extension whenever on-change for something else is needed/supported..
> 
> Visibility to application developers was one reason we went down this path.
> Any solution needs to frame this visibility as a key element.   Considering this,
> I actually see a need both for the schema based solution as framed, and the
> possibility of an instance based mechanism if someone wanted the lower
> level of granularity.  Per above, I don't think instance data within all instance
> objects like routing entries is possible anyplace datastore size becomes a
> factor.
> 
> (2) If this is per-instance, is the property inherited from a parent?   If no, how
> do we deal with the size of the resulting information?
> 
> (3) If this is per-instance, how is this information populated?  Don't you need
> schema level guidance recorded somewhere anyway?
> 
> If necessary, we can push this out.   I am hoping we can come up with
> something here, especially as the on-chance implementations I have seen
> have all used phased introduction of on-change for different schema objects.
> 
> > Other comments:
> >
> >
> > o  2
> >
> In general, I am fine with making the change to "Datastore" throughout the
> document.  For the title, I think it better to keep YANG to allow casual
> readers easy access to the context.
> 
>          +  Dampening period: In an on-change subscription, detected object
> changes should be sent as quickly as possible.  However without adequate
> protections, a rapid series of object changes might exhaust of resources in
> the publisher or receiver.  In order to protect against that, a dampening
> period MAY be used to specify the interval which must pass before
> successive update records for the same subscription are generated for a
> receiver.  The dampening period collectively applies to the set of all data
> nodes selected by a single subscription and sent to a single receiver.  This
> means that when there is a change to a subscribed object, an update record
> containing that object is created either immediately when no dampening
> period is in effect, or at the end of a dampening period.  A dampening period
> is reset every time a new notification message is passed to transport.
> 
> >   Is this correct?
> >
> "Specifies the interval which must pass before successive update records for
> the same subscription are generated for a receiver.  The dampening period
> collectively applies to the set of all data nodes selected by a single
> subscription and sent to a single receiver.  This means that when there is a
> change to a subscribed object, an update record containing that object is
> created either immediately when no dampening period is in effect, or at the
> end of a dampening period.  A dampening period is reset every time a new
> notification message is passed to transport.  A dampening period is reset
> every time a new notification message is passed to transport.  A zero value
> indicates no dampening period, and all subscribed object changes are sent
> immediately."
> 
> > o  3.2
> >
> >   The text says:
> >
> As YANG Push is intended to be transport independent, it is better not to
> provide a specific transport.   Instead, how about the following text:
> 
> In a periodic subscription, the update record MUST include the datastore
> nodes which would have been retrieved using an equivalent datastore
> selection operation over that subscription's transport.
> 
> We don't intend to define a new term in this paragraph.  And you are correct,
> policy only appears elsewhere in the document as part of the YANG model
> group names.    (more below)
> 
> An objective of YANG push is to allow the replication of a subset of a
> publisher's datastore within a receiver.  The datastore selection filter defines
> this subset.  Only a single selection filter can be applied to a subscription at a
> time.  The selection filter types defined in this include:
> 
> > o  3.6
> >
> " xpath: An xpath selection filter is an XPath expression which references
> data nodes. When applied to a datastore, it is the results of this expression
> which will be pushed."
> 
> > o  3.6
> >
> The issue with non-key properties is how to deal with the creation and
> deletion of objects when a non-key property goes in/out of the selection.
> Others were uncomfortable with things like passing along a patch showing a
> node was deleted, when in reality the property simply changed from
> meeting the selection filter to failing the selection filter.  So people wanted
> to default to the simpler case of key properties for selection.
> 
> Based on that I can delete the words "but the goal is to provide equivalent
> capabilities to what is available with a GET".
> 
> Will add how to populate.  In summary, sequential numbering of "edit-id"
> was from the RFC-8072.   And a null "patch-id" is because patch-id is
> mandatory in RFC-8072, and was originally supposed to be used for
> debugging of failed datastore write operations.  That really isn't the case
> here, plus we have subscription-id and other object available.
> 
> >         <yp:source
> > xmlns="urn:ietf:params:xml:ns:yang:ietf-datastores">
> >
> At the last IETF, Kent pointed us to YANG Lint which should allow us to do
> example checking in ways not previously known to us.    I will install and
> attempt to use these to clear the example items you have listed.
> 
> > o  3.9
> >
> It cannot know this at "establish-subscription".   So a publisher will have to
> track whether the permissions on subscribed objects change.   How is left to
> implementations.
> 
> > o  3.9
> >
> The requirement is that a subscription-id must be in the notification
> message, this doesn't mean that the subscription-id must be within the two
> new notifications.   The reason for this is that when we add support for the
> notification-messages draft, but having the subscription-id optional in the
> push-update and push-change-update, we can enable implementations
> which do not duplicate this header item.  (And a duplication would be forced
> with the future header if we made this mandatory.)
> 
> (b) Notification-time is the generalized name for "time-of-update" .  Having
> "time-of-update" as optional allows for eventual migration to common
> headers of  draft-ietf-netconf-notification-messages without information
> duplication
> 
> I think that answer is above.  But there is some good guidance here from
> draft-ietf-netconf-notification-messages.  The new draft ultimately will
> define and support the following times:
> 
> message-time:
>       "Header information consisting of time the message headers were placed
> generated prior to being sent to transport";
> 
> notification-time
> Notification-time should be equivalent to eventTime from RFC-5277.  But
> some other vendors have said they really only have message-time, and this is
> what they are populating in eventTime even though it doesn't explicitly fit
> the definition.  With the new definitions, at least they should be able to
> populate what they explicitly have.
> 
> > o  4.3.2
> >
> It is possible to expose QoS explicitly as a feature.   I will make that addition if
> you are ok with my other QoS comment described below about subscribed-
> notifications.
> 
> But even in that case if QoS is not supported, and someone includes QoS
> objects in an "establish-subscription" we need this identity as an error.  This
> error allow an explicit identification of what was wrong in the RPC (such as a
> DSCP provided is not supported by the Publisher).  I will include this in the
> Identity description.
> 
> Per our discussion above, this can be used if an RPC asks for on-change for an
> object which is not available at a platform deployment (either marked in the
> schema, or a specific deployment doesn't support an object which is
> included).   I will enhance the description based on the discussion on this
> earlier in this thread.
> 
> (1) Will be used if an RPC asks to synch on start, but it can't be supported for
> any reason (e.g., no nodes identifiable within the selection filter will ever be
> support on-change).   Will enhance the definition.
> 
> (2) The resynch RPC is invoked on a periodic subscription-id, or on an on-
> change subscription which can't support synchronization.
> 
> Will clarify description to explain that the key provided with the filter does
> not match the data type of the referenced yang subtree
> 
> >      identity datatree-size {
> >
> Yes, any new custom datastore will get a new identity, but if that new
> identity uses this custom-datastore one as a base, then we have an umbrella
> identity which is a handle just for the custom ones.  That was the intent of
> this.   If you don't think that an identity which acts as a bundling mechanism is
> useful, we can remove.
> 
> > o  5 - "change-type"
> >
> (1) For security applications, it is an absolute requirement that you know if a
> node was added and then removed during a dampening period.   For an
> example, a hacker adds a "permit any any" which is then quickly removed
> before the dampening period ends.  Remote applications need to know that
> churn occurred.
> 
> (2) For network management applications when a physical interface is going
> down and then quickly back up.   If the up/down is transient, how do you
> know that this occurred if there is no notification that churn occurred?
> 
> To support this, the YANG patch definition is loosened to allow the new
> creation on an existing node (based on what resulted from the last patch), or
> a delete on a missing node.  This acts as an indication that some churn in the
> datastore happened even though the data is in the same state as a per the
> previous update.  And if an application get to sees this indication when
> comparing to the previous state, they have the option of looking more
> closely at some logs on the publisher to determine what actually happened.
> 
> I can see that this behavior is under-described in the text.   I will put a small
> section between sections 3.10 & 3.11 describing this.  The title will be
> "Identification of a transient change within dampening period"
> 
> > o  5 - selection filter
> >
> As for when the evaluation occurs, I am not sure what you mean.  Every time
> a change occurs, you need to see if the changed object would have fallen
> within the selection.  There is no attempt to support an event-condition-
> action context.
> 
> > o  5 - terminology
> >
> This question has come up before.  Basically current 5277 event streams
> haven't shown need for QoS to date.   We know that QoS has been
> requested to handle congestion issues for YANG push.  So for simplicities
> sake, we bundled all QoS elements together in YANG push where we knew
> there was demonstrable need.    If someone really is asking for these QoS
> constructs for events, we can revisit.
> 
> This is to indicate a system error where not all objects can be extracted from
> the datastore.   (e.g., the routing table got *lots* bigger, and the periodic
> extract cannot be performed as expected.)  At least some implementations
> have seen this.   I will add some clarifying text.
> 
> > o  5 - push-change-update
> >

---

From: Alexander Clemm <[email-masked]>
Date: Wed, 29 Nov 2017 20:10:20 +0000

If we leave this to future work, ok, but what is the default behavior in that case?

Currently: on-change notifiable is supported only when specifically designated as such.

If we have no way to designate it, we can change it to assume it is supported, but reject on-change subscriptions that contain non-notifiable objects.  But how does a server know which ones they are?  We do not want to silently not send on-change updates, when the client assumes that we do.

One possibility would be to move the current solution to the appendix, defining this as one solution approach (and giving the YANG module for that).

--- Alex

From: Netconf [mailto:[email-masked]] On Behalf Of Andy Bierman
Sent: Tuesday, November 28, 2017 9:11 AM
To: Martin Bjorklund <[email-masked]>
Cc: Netconf <[email-masked]>
Subject: Re: [Netconf] review of draft-ietf-netconf-yang-push-11

On Tue, Nov 28, 2017 at 1:37 AM, Martin Bjorklund <[email-masked]<mailto:[email-masked]>> wrote:
Hi,

Important issue:

o  3.10

   <entry>

Andy

Other comments:

o  2

o  3.1

    2  3  4  11  13  14  15

  Is this correct?

o  3.2

  The text says:

o  3.4

  The text says:

    Please see [promise] for

o  3.5

  The text says:

o  3.5.1

o  3.6

o  3.6

o  3.6

     [...]

     the goal is to

o  3.7

o  3.8

  The example has:

   <establish-subscription

  Further is has:

  Also, it has:

  which should be:

o  3.9

  One of the cases is:

o  3.9

o  3.11.1

  OLD:

       If

  NEW:

       If

o  4.3.2

o  4.3.2

o  4.4.1 / 4.4.2

o  5

  OLD:

  NEW:

o  5 - identities

     identity datatree-size {

o  5 - "change-type"

        description

o  5 - selection filter

  OLD:

  NEW:

           FIXME: (*)

o  5 - terminology

o  5 - dampening

          leaf dampening-period {
            type yang:timeticks;
            mandatory true;

   Should this instead be:

          leaf dampening-period {
            type yang:timeticks;
            default "0";

o  5 - no-synch-on-start

             When

o  5 - QoS

o  5 - push-update

o  5 - push-change-update

o  General

/martin

_______________________________________________
Netconf mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/netconf

---

From: Alexander Clemm <[email-masked]>
Date: Wed, 29 Nov 2017 20:16:36 +0000

I thought we decided that we have combined dampening for all objects in the subscription.  In this case, we would send updates at 2, 12 (containing 3,4,11), and 22 (containing 13, 14, 15).

If the scope of the subscription becomes sufficiently large, this almost reverts back to a periodic subscription (since there is always going to be a change somewhere).  This is why I originally argued to have it indeed on a per-object basis, but I lost that argument.  For such fine-grained updates, where a client is indeed interest in getting delays of individual objects without delay,  a client could simply need to establish multiple “micro” subscriptions.

TCAs in RMON are something different altogether.  This is something we are trying to address with smart filters.

--- Alex

Sent: Wednesday, November 29, 2017 10:18 AM

....

o  3.1

    2  3  4  11  13  14  15

  Is this correct?

 ...

/martin

Andy

---

From: "Eric Voit (evoit)" <[email-masked]>
Date: Thu, 30 Nov 2017 00:51:24 +0000

The Section 3.1 text I proposed in my response to Martin on the dampening question:

Dampening period: In an on-change subscription, detected object changes should be sent as quickly as possible.  However without adequate protections, a rapid series of object changes might exhaust of resources in the publisher or receiver.  In order to protect against that, a dampening period MAY be used to specify the interval which must pass before successive update records for the same subscription are generated for a receiver.  The dampening period collectively applies to the set of all data nodes selected by a single subscription and sent to a single receiver.  This means that when there is a change to a subscribed object, an update record containing that object is created either immediately when no dampening period is in effect, or at the end of a dampening period.  A dampening period is reset every time a new notification message is passed to transport.

With the YANG description of:

Does this work for everyone?

Eric

From: Netconf [mailto:[email-masked]] On Behalf Of Alexander Clemm
Sent: Wednesday, November 29, 2017 3:17 PM
To: Andy Bierman <[email-masked]>; Martin Bjorklund <[email-masked]>
Cc: Netconf <[email-masked]>
Subject: Re: [Netconf] review of draft-ietf-netconf-yang-push-11

--- Alex

From: Netconf [mailto:[email-masked]] On Behalf Of Andy Bierman
Sent: Wednesday, November 29, 2017 10:18 AM
To: Martin Bjorklund <[email-masked]<mailto:[email-masked]>>
Cc: Netconf <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Netconf] review of draft-ietf-netconf-yang-push-11

....

o  3.1

    2  3  4  11  13  14  15

  Is this correct?

 ...

/martin

Andy

---

From: Alexander Clemm <[email-masked]>
Date: Thu, 30 Nov 2017 00:59:24 +0000

This works for me.  Perhaps one additional item we might add (for crystal-clear clarification) is that when the notification message is sent, it contains the most recent update for that object (i.e. the value that is in effect when the update is sent).  If there are some quickly oscillating values we don’t send the whole sequence of updates/values.

--- Alex

From: Eric Voit (evoit) [mailto:[email-masked]]
Sent: Wednesday, November 29, 2017 4:51 PM
To: Alexander Clemm <[email-masked]>; Andy Bierman <[email-masked]>; Martin Bjorklund <[email-masked]>; [email-masked]
Cc: Netconf <[email-masked]>
Subject: RE: [Netconf] review of draft-ietf-netconf-yang-push-11

Eric

To: Andy Bierman <[email-masked]<mailto:[email-masked]>>; Martin Bjorklund <[email-masked]<mailto:[email-masked]>>

--- Alex

....

o  3.1

    2  3  4  11  13  14  15

  Is this correct?

 ...

/martin

Andy

---

From: Rohit R Ranade <[email-masked]>
Date: Thu, 30 Nov 2017 06:30:27 +0000

Hi Eric,

We need also consider that commit maybe very big or in the dampening time so many records are updated that the <notification> message may need to be split across fragments.  I suggest reframing of the sentence to

“A dampening period is reset every time the last fragment of a new notification message is passed to transport”

With Regards,
Rohit R

From: Netconf [mailto:[email-masked]] On Behalf Of Alexander Clemm
Sent: 30 November 2017 06:29
To: Eric Voit (evoit) <[email-masked]>; Andy Bierman <[email-masked]>; Martin Bjorklund <[email-masked]>; [email-masked]
Cc: Netconf <[email-masked]>
Subject: Re: [Netconf] review of draft-ietf-netconf-yang-push-11

--- Alex

To: Alexander Clemm <[email-masked]<mailto:[email-masked]>>; Andy Bierman <[email-masked]<mailto:[email-masked]>>; Martin Bjorklund <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>

Eric

--- Alex

....

o  3.1

    2  3  4  11  13  14  15

  Is this correct?

 ...

/martin

Andy

---

From: "Eric Voit (evoit)" <[email-masked]>
Date: Thu, 30 Nov 2017 15:01:43 +0000

Hi Randy,
Hi Rohit,
Hi Alex,

Thanks for the thoughts.

Randy, on your point: it is true previous wording of the YANG description can be tightened up.  And your paraphrasing for the YANG description provided good guidance.  Below I give an attempt reframing yang description, also taking into account Alex & Rohit’s comments...

YANG Description

Specifies the minimum interval between the assembly of successive update records for a single receiver of a subscription. Whenever subscribed objects change, and a dampening period interval (which may be zero) has elapsed since the previous update record creation for a receiver, then any subscribed objects and properties which have changed since the previous update record will have their current values marshalled and placed into a new update record.

Rohit, on your point: to cover any such a possibility, it might be better to reset the dampening period after a specific update record is assembled.   This way any bundling of updates into notification messages, or fragmentation delays won’t result in an irregular or unpredictable dampening period.   Look below at my attempt to frame this within Section 3.10 text...

Alex, on your point: yes we need to be explicit about just the last value being sent.  I tweak it into the description in the 3.10 below.  I think I also covered in it the YANG description above...

Section 3.10

Dampening period: In an on-change subscription, detected object changes should be sent as quickly as possible.  However it may be undesirable to send a rapid series of object changes.  Such behavior has the potential to exhaust of resources in the publisher or receiver.  In order to protect against that, a dampening period MAY be used to specify the interval which must pass before successive update records for the same subscription are generated for a receiver.  The dampening period collectively applies to the set of all data nodes selected by a single subscription and sent to a single receiver.  This means that when there is a change to one or more subscribed objects, an update record containing those objects is created either immediately when no dampening period is in effect, or at the end of a dampening period.  If multiple changes to a single object occur during a dampening period, only the value that is in effect is included as part of the update record.  A dampening period is reset every time an update record has completed its assembly.

Eric

From: Rohit R Ranade, November 30, 2017 1:30 AM

Hi Eric,

With Regards,
Rohit R

To: Eric Voit (evoit) <[email-masked]<mailto:[email-masked]>>; Andy Bierman <[email-masked]<mailto:[email-masked]>>; Martin Bjorklund <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>

--- Alex

Eric

--- Alex

....

o  3.1

    2  3  4  11  13  14  15

  Is this correct?

 ...

/martin

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 30 Nov 2017 16:24:42 +0100 (CET)

Hi,

I'll reply to the on-change discussion here.

Alexander Clemm <[email-masked]> wrote:
> As the mail is long, rather than commenting inline, let me make just a
> few additional replies in addition to Eric's:
> 
> Yes, clearly what is on-change notifiable and what is not could differ
> by implementation.  We thought that asking implementors to define
> implementation-specific modules that would indicate the on-change
> notifiability through deviations would make sense, since this is
> really one of the function of deviations - indicate what is
> implementation-specific.

Actually, I think it mis-uses the deviation statement a bit.  From the
spec:

   Deviations define the way a server or class of servers deviate from a
   standard.

   [...]

   Server deviations are strongly discouraged and MUST only be used as a
   last resort.  Telling the application how a server fails to follow a
   standard is no substitute for implementing the standard correctly.  A
   server that deviates from a module is not fully compliant with the
   module.

We have discussed an "annotate" statement before, which would be used
to add information to schema nodes.  (we actually have such a
statement as a vendor-specific extension, and it is used a lot).  But
even if we had that, I'm not sure that would be the best solution in
this case.

> In that sense, the current solution does
> make sense to me and seems to be in the spirit of how
> implementation-specific YANG modules should be used.
> 
> That said, the other proposal to have a "meta-module" that contains a
> list of objects for which on-change notification is supported could be
> certainly be an alternative.  However, the notifiability should be
> indicated per node at the schema level, not for each individual
> instance.  We would not want to have to enumerate individual list
> entries, for example. Also, are there additional considerations to
> accommodate schema-mount in that case?
> 
> Another thought, perhaps on-change notifiability is something that
> could instead also be included in the YANG library.  Each module in
> the library would then list the nodes for which on-change is
> notifiable on a given server.

That's also an option.

I think we want somehting that is available on-line in runtime, but
that also can be stored in an off-line file (ala AGENT-CAPABILITIES).
Augmentation of yang library probably solved both these cases.

> Deferring this until later might also an option.

I think that's my preferred option.  At least if we want to finish
this document sooner rather than later...

> I am sure there will
> be other annotations/markings for objects that we will come up with
> our time.  However, one issue is that by default we did not want to
> make assumptions about on-change notifiability being supported.
> Rather, when it is supported, it would be specifically indicated; the
> default would be that it's not supported.  If we defer the solution
> until later, then this won't work.

Well, there's alread a feature to indicate *some* support.  If we
defer, we'd just write that how an implementation communicates which
nodes it supports for on-change is out of scope.

> If we wanted to defer, it would
> seem what we should do is still propose a solution informally in an
> appendix, and indicate that this may be later replaced by another
> solution.  We would put the YANG extension into an own separate
> module.

I don't think that's a good solution.  I don't think we should
encourage the extension-statement solution.

As one data point, in our implementation, all config nodes in the
conventional datastores would be available for on-change.  For
operational state, it is up to the instrumentation.

> - on the Subscription Policy term (3.6).  Sure, we don't need to
> - introduce a new term, but really what the parameters describe is in
> - fact a policy.  We could of course call them "datastore subscription
> - specific configuration parameters", or "datastore subscription
> - specific behavior configuration" , but this sounds a bit awkward to
> - me.

I would actually prefer more concrete words like the ones you
propose.

> - on the "no-such-datastore" eror tag: true, invalid-value would cover
> - this, but why be general when we can be specific?

I'll reply to this in my reply to Eric.

/martin

> > Fully agree this can be implementation level issue.  But then again,
> > so is any
> > deviation.
> > 
> > This could work.  Several questions which would need to be addressed
> > with
> > this approach:
> > 
> > Should be 2, 12, & 22.  I have updated the definition (below) in a way
> > which
> > hopefully clarifies the proper behavior
> > 
> > There are plenty of valid xpath expressions which don't select data
> > nodes.
> > How about:
> > 
> > (a) Other vendors have worried they can only support message-time,
> > rather
> > than notification-time.
> >       "Header information consisting of the time an originating process
> >       created
> > the notification."
> > 
> >   If the application detects any incompleteness in set of objects placed
> >   into a
> > "push-update" or "push-change-update"
> > 
> > > o  4.4.1 / 4.4.2
> > >
> > Will do.  I am hoping a new attempt using yanglint helps pick out
> > everything I
> > missed by hand previously.
> > 
> > > o  5
> > >
> > >   OLD:
> > >
> > This is to identify churn within a datastore during a dampening
> > period.  Two
> > examples of when this is needed:
> > 
> > So this is saying that the named datastore must exist before allowing
> > the
> > augment.   Works for me.
> > 
> > > o  5 - push-update
> > >

---

From: Alexander Clemm <[email-masked]>
Date: Thu, 30 Nov 2017 18:54:14 +0000

Hi Martin,

Instead of "deviation", we could also use "augmentation".  The point is that the implementation-specifics would be provided in an implementation-specific YANG module.  Clearly the possibility to have implementation-specific YANG modules is foreseen by the very fact that the deviation statement exists, and the genie is out of the bottle here.  

That said, yes, there was an original proposal to use annotations, which Balazs proposed.  When it became clear that the general annotations proposal wasn't moving forward, we settled for the next-best solution, which is the YANG extension (i.e. a very specific annotation to indicate on-change notifiability) that we have today.  

As indicated by you and others on this thread, alternatively we can certainly also simply defer this issue for now.  In this case, as you state, we would simply indicate in the text that clients need to be aware that not every object may support on-change notifiability; how do discover which ones do and which ones do not would be left outside the scope of the specification.  

However, in that case it would still seem to make sense to propose a way in which an implementation could address this in an informational appendix.  This informational appendix could then make the suggestions that implementations might support a module with such an extension, which we would then include in the appendix as an example.  Having a module enumerating the nodes (per your earlier suggestion), or suggesting a library augmentation (without introducing a dependency on that - can be informational only!) could be described there as well.  Perhaps it is a question to our WG chairs, Mahesh and Kent, whether putting this as an informational example for a way in which to addresss this would be feasible, in the same way as some drafts explain how something might be implemented - really this is a process question - or if it should simply be omitted entirely.  

We would also make clear that if a subscription contains objects that are not on-change notifiable, no updates will be received (rather than rejecting the subscription).  

The more I think about this, the more I think that addressing this as part of the library will be the technically preferable option.  Having this as an augmentation would work.  Is this something that we can include in the YANG library -bis draft?  Really this is where this should go.  This could also serve as an example there how the library might be used to keep track also of other "annotations" in the future.  Perhaps something to discuss on the other thread.

Thanks
--- Alex

> From: Martin Bjorklund [mailto:[email-masked]]
> Sent: Thursday, November 30, 2017 7:25 AM
> To: Alexander Clemm <[email-masked]>
> Cc: [email-masked]; [email-masked]; [email-masked];
> [email-masked]
> We have discussed an "annotate" statement before, which would be used to
> add information to schema nodes.  (we actually have such a statement as a
> vendor-specific extension, and it is used a lot).  But even if we had that, I'm
> not sure that would be the best solution in this case.
> 
> > In that sense, the current solution does make sense to me and seems to
> > be in the spirit of how implementation-specific YANG modules should be
> > used.
> >
> I think we want somehting that is available on-line in runtime, but that also
> can be stored in an off-line file (ala AGENT-CAPABILITIES).
> I think that's my preferred option.  At least if we want to finish this document
> sooner rather than later...
> 
> 
> > I am sure there will
> Well, there's alread a feature to indicate *some* support.  If we defer, we'd
> just write that how an implementation communicates which nodes it
> supports for on-change is out of scope.
> 
> I don't think that's a good solution.  I don't think we should encourage the
> extension-statement solution.
> 
> conventional datastores would be available for on-change.  For operational
> state, it is up to the instrumentation.
> 
> 
> I would actually prefer more concrete words like the ones you propose.
> 
> 
> > > To: Martin Bjorklund <[email-masked]>; [email-masked]; Balazs
> > > Lengyel <[email-masked]> ([email-masked])
> > > In general we agree there are several alternatives to solve this
> > > issue.  And any mechanism supported by the WG would be fine.
> > >
> > > example, I *think* you are proposing hierarchical inheritance of
> > > on-change property markings of the parent rather than making every
> > > instance.
> > > Because
> > > expensive.  So based on whatever commonality we can establish across
> > > the proposed alternatives, perhaps we can come to a common way to
> > > set this
> > > change support data.  Application developers are going to make
> > > subscription requests.  And they are far more likely to want to
> > > design based on the schema level information exposed rather than
> > > instance data.  Driven by that constraint, the current proposal is
> > > framed as is -- mostly since there was no interest in Balazs'
> > > https://tools.ietf.org/html/draft-lengyel-netmod-schema-annotation-0
> > > 0 which was designed to support application developers needing this
> > > information.  Without this draft, at the time this was proposed the
> > > next-best thing was an extension which works based on the
> > > established pattern of deviations.
> > >
> > > so is any deviation.
> > > So to cover "property of the implementation" case, the proposed
> > > extension would typically be included in deviations file per
> > > implementation.
> > > Such an
> > > benefit of not requiring another a new totally separate file to
> > > document the desired behavior.
> > >
> > > As for times this might be deployment specific, we saw potential
> > > capacity issues determining whether something might not be on-change
> > > whether to use the error "on-change-unsupported" from YANG Push, or
> > > the error "insufficient-resources" from subscribed notifications.  A
> > > choice between the two can be driven based on whether a deployment
> > > would
> > > As the current solution was developed before NMDA, we didn't
> > > consider differences in on-change support for different datastores.
> > > So this is a real consideration.
> > >
> > > If for a minute we assume we are going to follow the current
> > > approach in the draft, the extension proposed in 3.10 could have a
> > > default of the operational datastore.  And we could add the option
> > > of including a list of datastores in the extension whenever
> > > on-change for something else is needed/supported..
> > >
> > > I actually see a need both for the schema based solution as framed,
> > > and the possibility of an instance based mechanism if someone wanted
> > > the lower level of granularity.  Per above, I don't think instance
> > > data within all instance objects like routing entries is possible
> > > anyplace datastore size becomes a factor.
> > >
> > > with this approach:
> > > (1) how would application developers easily know what objects are
> > > on- change subscribable?
> > >
> > > (3) If this is per-instance, how is this information populated?
> > > Don't you need schema level guidance recorded somewhere anyway?
> > >
> > > something here, especially as the on-chance implementations I have
> > > seen have all used phased introduction of on-change for different
> > > schema objects.
> > >
> > > > Other comments:
> > > >
> > > >
> > > > o  2
> > > >
> > > In general, I am fine with making the change to "Datastore"
> > > throughout the document.  For the title, I think it better to keep
> > > YANG to allow casual readers easy access to the context.
> > >
> > > Should be 2, 12, & 22.  I have updated the definition (below) in a
> > > way which hopefully clarifies the proper behavior
> > >
> > >          +  Dampening period: In an on-change subscription, detected
> > > object changes should be sent as quickly as possible.  However
> > > without adequate protections, a rapid series of object changes might
> > > exhaust of resources in the publisher or receiver.  In order to
> > > protect against that, a dampening period MAY be used to specify the
> > > interval which must pass before successive update records for the
> > > same subscription are generated for a receiver.  The dampening
> > > period collectively applies to the set of all data nodes selected by
> > > a single subscription and sent to a single receiver.
> > > This
> > > means that when there is a change to a subscribed object, an update
> > > record containing that object is created either immediately when no
> > > dampening period is in effect, or at the end of a dampening period.
> > > A dampening period is reset every time a new notification message is
> > > passed to transport.
> > >
> > > >   Is this correct?
> > > >
> > > "Specifies the interval which must pass before successive update
> > > records for the same subscription are generated for a receiver.  The
> > > dampening period collectively applies to the set of all data nodes
> > > selected by a single subscription and sent to a single receiver.
> > > This means that when there is a change to a subscribed object, an
> > > update record containing that object is created either immediately
> > > when no dampening period is in effect, or at the end of a dampening
> > > period.  A dampening period is reset every time a new notification
> > > message is passed to transport.  A dampening period is reset every
> > > time a new notification message is passed to transport.  A zero
> > > value indicates no dampening period, and all subscribed object
> > > changes are sent immediately."
> > >
> > > > o  3.2
> > > >
> > > >   The text says:
> > > >
> > > In a periodic subscription, the update record MUST include the
> > > datastore nodes which would have been retrieved using an equivalent
> > > datastore selection operation over that subscription's transport.
> > >
> > > We don't intend to define a new term in this paragraph.  And you are
> > > correct, policy only appears elsewhere in the document as part of
> > > the YANG model
> > > An objective of YANG push is to allow the replication of a subset of
> > > a publisher's datastore within a receiver.  The datastore selection
> > > filter defines this subset.  Only a single selection filter can be
> > > applied to a subscription at a time.  The selection filter types
> > > defined in this include:
> > >
> > > > o  3.6
> > > >
> > > " xpath: An xpath selection filter is an XPath expression which
> > > references data nodes. When applied to a datastore, it is the
> > > results of this expression which will be pushed."
> > >
> > > > o  3.6
> > > >
> > > The issue with non-key properties is how to deal with the creation
> > > and deletion of objects when a non-key property goes in/out of the
> > > selection.
> > > Others were uncomfortable with things like passing along a patch
> > > showing a node was deleted, when in reality the property simply
> > > changed from meeting the selection filter to failing the selection
> > > filter.  So people wanted to default to the simpler case of key
> > > properties for selection.
> > >
> > > Based on that I can delete the words "but the goal is to provide
> > > equivalent capabilities to what is available with a GET".
> > >
> > > debugging of failed datastore write operations.  That really isn't
> > > the case here, plus we have subscription-id and other object
> > > available.
> > >
> > > At the last IETF, Kent pointed us to YANG Lint which should allow us
> > > to do example checking in ways not previously known to us.  I will
> > > install and attempt to use these to clear the example items you have
> > > listed.
> > >
> > > > o  3.9
> > > >
> > > It cannot know this at "establish-subscription".  So a publisher
> > > will have to track whether the permissions on subscribed objects
> > > change.  How is left to implementations.
> > >
> > > > o  3.9
> > > >
> > > The requirement is that a subscription-id must be in the
> > > notification message, this doesn't mean that the subscription-id
> > > must be within the two new notifications.  The reason for this is
> > > that when we add support for the notification-messages draft, but
> > > having the subscription-id optional in the push-update and
> > > push-change-update, we can enable implementations which do not
> > > duplicate this header item.  (And a duplication would be forced with
> > > the future header if we made this mandatory.)
> > >
> > > rather than notification-time.
> > > headers of draft-ietf-netconf-notification-messages without
> > > information duplication
> > >
> > > I think that answer is above.  But there is some good guidance here
> > > from draft-ietf-netconf-notification-messages.  The new draft
> > > ultimately will define and support the following times:
> > >
> > > message-time:
> > >       "Header information consisting of time the message headers
> > > were placed generated prior to being sent to transport";
> > >
> > > notification-time
> > > some other vendors have said they really only have message-time, and
> > > this is what they are populating in eventTime even though it doesn't
> > > explicitly fit the definition.  With the new definitions, at least
> > > they should be able to populate what they explicitly have.
> > >
> > > > o  4.3.2
> > > >
> > > everything I missed by hand previously.
> > >
> > > > o  5
> > > >
> > > >   OLD:
> > > >
> > > It is possible to expose QoS explicitly as a feature.  I will make
> > > that addition if you are ok with my other QoS comment described
> > > below about subscribed- notifications.
> > >
> > > But even in that case if QoS is not supported, and someone includes
> > > QoS objects in an "establish-subscription" we need this identity as
> > > an error.  This error allow an explicit identification of what was
> > > wrong in the RPC (such as a DSCP provided is not supported by the
> > > Publisher).  I will include this in the Identity description.
> > >
> > > Per our discussion above, this can be used if an RPC asks for
> > > on-change for an object which is not available at a platform
> > > deployment (either marked in the schema, or a specific deployment
> > > doesn't support an object which is included).  I will enhance the
> > > description based on the discussion on this earlier in this thread.
> > >
> > > (1) Will be used if an RPC asks to synch on start, but it can't be
> > > supported for any reason (e.g., no nodes identifiable within the
> > > selection filter will ever be
> > > (2) The resynch RPC is invoked on a periodic subscription-id, or on
> > > an
> > > on-
> > > Will clarify description to explain that the key provided with the
> > > filter does not match the data type of the referenced yang subtree
> > >
> > > >      identity datatree-size {
> > > >
> > > Yes, any new custom datastore will get a new identity, but if that
> > > new identity uses this custom-datastore one as a base, then we have
> > > an umbrella identity which is a handle just for the custom ones.
> > > That was the intent of this.  If you don't think that an identity
> > > which acts as a bundling mechanism is useful, we can remove.
> > >
> > > > o  5 - "change-type"
> > > >
> > > period.  Two examples of when this is needed:
> > >
> > > example, a hacker adds a "permit any any" which is then quickly
> > > removed before the dampening period ends.  Remote applications need
> > > to know that churn occurred.
> > >
> > > (2) For network management applications when a physical interface is
> > > going down and then quickly back up.  If the up/down is transient,
> > > how do you know that this occurred if there is no notification that
> > > churn occurred?
> > >
> > > To support this, the YANG patch definition is loosened to allow the
> > > new creation on an existing node (based on what resulted from the
> > > last patch), or a delete on a missing node.  This acts as an
> > > indication that some churn in the datastore happened even though the
> > > data is in the same state as a per the previous update.  And if an
> > > application get to sees this indication when comparing to the
> > > previous state, they have the option of looking more closely at some
> > > logs on the publisher to determine what actually happened.
> > >
> > > I can see that this behavior is under-described in the text.  I will
> > > put a small section between sections 3.10 & 3.11 describing this.
> > > The title will be "Identification of a transient change within
> > > dampening period"
> > >
> > > > o  5 - selection filter
> > > >
> > > As for when the evaluation occurs, I am not sure what you mean.
> > > Every time a change occurs, you need to see if the changed object
> > > would have fallen within the selection.  There is no attempt to
> > > support an
> > > event-condition-
> > > action context.
> > >
> > > > o  5 - terminology
> > > >
> > > requested to handle congestion issues for YANG push.  So for
> > > simplicities sake, we bundled all QoS elements together in YANG push
> > > where we knew there was demonstrable need.  If someone really is
> > > asking for these QoS constructs for events, we can revisit.
> > >
> > > So this is saying that the named datastore must exist before
> > > allowing the
> > > augment.   Works for me.
> > >
> > > > o  5 - push-update
> > > >
> > > This is to indicate a system error where not all objects can be
> > > extracted from the datastore.  (e.g., the routing table got *lots*
> > > bigger, and the periodic extract cannot be performed as expected.)
> > > At least some implementations

---

From: Henk Birkholz <[email-masked]>
Date: Thu, 30 Nov 2017 22:47:48 +0100

Hi all,

Admittedly, I am relatively new to this domain and before I provide my 
comment I want to acknowledge that:

- it is important not to off-load to much complexity to the agents that 
are part of the data store
- complex composite devices might create a lot of "noise" in wrt changes 
of data node values and that has to be dealt with in a sane and 
reasonable way

That said, YANG Push caught a lot of attention outside of NETCONF. In a 
nutshell, it is a solution that provides meaningful telemetry, which 
simplifies post-processing of subscribed notifications significantly.

But - there is always a but, I guess - "visibility" is a key capability 
in this context, I think. Yes, there might be oscillating values, yes, 
they might be of no value to most post-processing processes, but to 
exclude them entirely might reduce the usefulness of the on-change 
capability significantly.

There is the complementary topic of smart filters that might deal with 
providing metadata about this "omitted notifications" (coming back to 
the oscillating values example, Alex illustrated), but visibility of 
ongoing changes should be supported somehow, I think. Maybe not in the 
top level draft that is netconf-yang-push, but at some level, I hope.

Again, this is coming from a fresh pair of eyes still relatively 
unfamiliar with the emerging ecosystem around YANG Push.

In a nutshell, I just want to highlight that visibility of past changes 
and dealing with high frequency of changes of course requires a feasible 
compromise. But from my POV, "the most recent update" might not cut it. 
I only want to highlight that fine granular visibility is a vital 
characteristic of on-change emission of notifications. Maybe there can 
be a knob, as Sue might phrase it.

If this is already addressed by another draft, I apologize for my wall 
of text. If not, please take this POV into account :)

Viele Grüße,

Henk

On 11/30/2017 01:59 AM, Alexander Clemm wrote:
> This works for me.  Perhaps one additional item we might add (for 
> crystal-clear clarification) is that when the notification message is 
> sent, it contains the most recent update for that object (i.e. the value 
> that is in effect when the update is sent).  If there are some quickly 
> oscillating values we don’t send the whole sequence of updates/values.
> 
> --- Alex
> 
> *From:*Eric Voit (evoit) [mailto:[email-masked]]
> *Sent:* Wednesday, November 29, 2017 4:51 PM
> *To:* Alexander Clemm <[email-masked]>; Andy Bierman 
> <[email-masked]>; Martin Bjorklund <[email-masked]>; [email-masked]
> *Cc:* Netconf <[email-masked]>
> *Subject:* RE: [Netconf] review of draft-ietf-netconf-yang-push-11
> 
> Dampening period: In an on-change subscription, detected object changes 
> should be sent as quickly as possible.  However without adequate 
> protections, a rapid series of object changes might exhaust of resources 
> in the publisher or receiver. In order to protect against that, a 
> dampening period MAY be used to specify the interval which must pass 
> before successive update records for the same subscription are generated 
> for a receiver.  The dampening period collectively applies to the set of 
> all data nodes selected by a single subscription and sent to a single 
> receiver.  This means that when there is a change to a subscribed 
> object, an update record containing that object is created either 
> immediately when no dampening period is in effect, or at the end of a 
> dampening period.  A dampening period is reset every time a new 
> notification message is passed to transport.
> 
> "Specifies the interval which must pass before successive update records 
> for the same subscription are generated for a receiver.  The dampening 
> period collectively applies to the set of all data nodes selected by a 
> single subscription and sent to a single receiver.  This means that when 
> there is a change to a subscribed object, an update record containing 
> that object is created either immediately when no dampening period is in 
> effect, or at the end of a dampening period.  A dampening period is 
> reset every time a new notification message is passed to transport.  A 
> dampening period is reset every time a new notification message is 
> passed to transport.  A zero value indicates no dampening period, and 
> all subscribed object changes are sent immediately."
> 
> *From:*Netconf [mailto:[email-masked]] *On Behalf Of 
> *Alexander Clemm
> *Sent:* Wednesday, November 29, 2017 3:17 PM
> *To:* Andy Bierman <[email-masked] <mailto:[email-masked]>>; 
> Martin Bjorklund <[email-masked] <mailto:[email-masked]>>
> *Cc:* Netconf <[email-masked] <mailto:[email-masked]>>
> *Subject:* Re: [Netconf] review of draft-ietf-netconf-yang-push-11
> 
> I thought we decided that we have combined dampening for all objects in 
> the subscription.  In this case, we would send updates at 2, 12 
> (containing 3,4,11), and 22 (containing 13, 14, 15).
> 
> If the scope of the subscription becomes sufficiently large, this almost 
> reverts back to a periodic subscription (since there is always going to 
> be a change somewhere). This is why I originally argued to have it 
> indeed on a per-object basis, but I lost that argument.  For such 
> fine-grained updates, where a client is indeed interest in getting 
> delays of individual objects without delay,  a client could simply need 
> to establish multiple “micro” subscriptions.
> 
> TCAs in RMON are something different altogether.  This is something we 
> are trying to address with smart filters.
> 
> --- Alex
> 
> *From:*Netconf [mailto:[email-masked]] *On Behalf Of *Andy Bierman
> *Sent:* Wednesday, November 29, 2017 10:18 AM
> *To:* Martin Bjorklund <[email-masked] <mailto:[email-masked]>>
> *Cc:* Netconf <[email-masked] <mailto:[email-masked]>>
> *Subject:* Re: [Netconf] review of draft-ietf-netconf-yang-push-11
> 
> [email-masked] <mailto:[email-masked]>

---

From: "Eric Voit (evoit)" <[email-masked]>
Date: Thu, 30 Nov 2017 22:16:43 +0000

Hi Henk,

YANG Push does identify that churn has occurred, but not how much churn has occurred during a dampening period.   This allows things like security applications to know things like temporary ACL changes.  Or things like network management systems knowing that there are transient outages on an interface.

At this point, the thinking is that an application always has the option of going to Publisher to get the details of the churn.   Perhaps new drafts could also push the quantity of changes which occurred, or a histogram or changes, or some other metric.  At this point that topic is unexplored.  But there is no reason that such information couldn't be augmented into some header associated with the push-change-update record.

Eric

> From: Henk Birkholz, November 30, 2017 4:48 PM
> 
> Hi all,
> 
> - it is important not to off-load to much complexity to the agents that are
> part of the data store
> - complex composite devices might create a lot of "noise" in wrt changes of
> data node values and that has to be dealt with in a sane and reasonable
> way
> 
> But - there is always a but, I guess - "visibility" is a key capability in this
> context, I think. Yes, there might be oscillating values, yes, they might be of
> no value to most post-processing processes, but to exclude them entirely
> might reduce the usefulness of the on-change capability significantly.
> 
> providing metadata about this "omitted notifications" (coming back to the
> oscillating values example, Alex illustrated), but visibility of ongoing changes
> should be supported somehow, I think. Maybe not in the top level draft
> that is netconf-yang-push, but at some level, I hope.
> 
> Again, this is coming from a fresh pair of eyes still relatively unfamiliar with
> the emerging ecosystem around YANG Push.
> 
> In a nutshell, I just want to highlight that visibility of past changes and
> dealing with high frequency of changes of course requires a feasible
> I only want to highlight that fine granular visibility is a vital characteristic of
> on-change emission of notifications. Maybe there can be a knob, as Sue
> might phrase it.
> 
> If this is already addressed by another draft, I apologize for my wall of text.
> If not, please take this POV into account :)
> 
> Viele Grüße,
> 
> Henk
> 
> > sent, it contains the most recent update for that object (i.e. the
> > value that is in effect when the update is sent).  If there are some
> > quickly oscillating values we don’t send the whole sequence of
> updates/values.
> >
> > --- Alex
> >
> > Dampening period: In an on-change subscription, detected object
> > changes should be sent as quickly as possible.  However without
> > adequate protections, a rapid series of object changes might exhaust
> > of resources in the publisher or receiver. In order to protect against
> > that, a dampening period MAY be used to specify the interval which
> > must pass before successive update records for the same subscription
> > are generated for a receiver.  The dampening period collectively
> > applies to the set of all data nodes selected by a single subscription
> > and sent to a single receiver.  This means that when there is a change
> > to a subscribed object, an update record containing that object is
> > created either immediately when no dampening period is in effect, or
> > at the end of a dampening period.  A dampening period is reset every
> > time a new notification message is passed to transport.
> >
> > selected by a single subscription and sent to a single receiver.  This
> > dampening period is in effect, or at the end of a dampening period.  A
> > passed to transport.  A dampening period is reset every time a new
> > notification message is passed to transport.  A zero value indicates
> > no dampening period, and all subscribed object changes are sent
> immediately."
> >
> > I thought we decided that we have combined dampening for all objects
> > in the subscription.  In this case, we would send updates at 2, 12
> > If the scope of the subscription becomes sufficiently large, this
> > almost reverts back to a periodic subscription (since there is always
> > going to be a change somewhere). This is why I originally argued to
> > have it indeed on a per-object basis, but I lost that argument.  For
> > such fine-grained updates, where a client is indeed interest in
> > getting delays of individual objects without delay,  a client could
> > simply need to establish multiple “micro” subscriptions.
> >

---

From: Rohit R Ranade <[email-masked]>
Date: Fri, 1 Dec 2017 04:07:20 +0000

Hi Eric,

In your case , then there is a corner case where in if the dampening period is short, the device may have still not finished sending all its notification fragments and the next update record is already ready to get assembled.  Whether we can have a lower limit on the dampening period interval to avoid such a scenario ?

With Regards,
Rohit R

From: Eric Voit (evoit) [mailto:[email-masked]]
Sent: 30 November 2017 20:32
To: Rohit R Ranade <[email-masked]>; Alexander Clemm <[email-masked]>; Andy Bierman <[email-masked]>; Martin Bjorklund <[email-masked]>; [email-masked]; Randy Presuhn <[email-masked]>
Cc: Netconf <[email-masked]>
Subject: RE: [Netconf] review of draft-ietf-netconf-yang-push-11

Hi Randy,
Hi Rohit,
Hi Alex,

Thanks for the thoughts.

YANG Description

Section 3.10

Eric

With Regards,
Rohit R

--- Alex

Eric

--- Alex

....

o  3.1

    2  3  4  11  13  14  15

  Is this correct?

 ...

/martin

Andy