Subject: [Netconf] YP#12 Value filtering on non-key objects (was RE: review of draft-ietf-netconf-yang-push-11)

From: "Eric Voit (evoit)" <[email-masked]>
Date: Fri, 22 Dec 2017 19:52:30 +0000

I am looking to see if anyone has an issue with my opening and closing of:
https://github.com/netconf-wg/yang-push/issues/12 
as described below...

During Martin's recent review of yang-push, he argued that we should allow selection filtering using non-key properties.   I believe he is correct.  (In fact, this is what was supported in yang-push drafts -v00 through -v08.)

The reason we changed in -v09 is that if a non-key property changes, it might mean that the object as a whole is no longer selected.    The result is that an entire subtree may no longer should be visible to the receiver.  So the YANG patch sent from the subscriber to the receiver would need to add/remove entire subtree based on a simple value change for a non-key object in that subtree.   The thought for -v09 was that if we restricted on-change selection filtering just to keys, this might be simpler/easier to implement.  But per Martin's comments below, he pointed out other issues which come with placing such restrictions in filters.    

Does anyone have an objections to once again allowing selection filters to be placed on non-key properties?    (Alex & Andy, this was something we went back-and-forth on as part of the Dezign Team discussions.)

If there is no objection, this makes the conceptual selection process for creating an push-change-update:
		
 (1) Just before a change, or at the start of a dampening period, evaluate any filtering and any access control rules.  The result is a set "A" of datastore nodes and subtrees.

 (2) Just after a change, or at the end of a dampening period, evaluate any filtering and any (possibly new) access control rules.  The result is a set "B" of datastore nodes and subtrees.

 (3) Construct a YANG patch record for going from A to B.   If the record is non-empty, send it to the receiver.

Eric

> From: Martin Bjorklund,  December 5, 2017 5:03 AM
> 
> > > > >      the goal is to
> > > > >      provide equivalent capabilities to what is available with a GET.
> > > > >
> > > > >   In GET you can filter on "non-key properties".
> > > > >
> > > > >   I think you should remove the text about "non-key properties".  If
> > > > >   anything, I think you can allow an implementation to reject a filter
> > > > >   that would be too complex to implement / evaluate (in fact I think
> > > > >   the text already allows a server to reject such filters.)
> > > >
> > > > The issue with non-key properties is how to deal with the creation
> > > > and deletion of objects when a non-key property goes in/out of the
> > > > selection.  Others were uncomfortable with things like passing
> > > > along a patch showing a node was deleted, when in reality the
> > > > property simply changed from meeting the selection filter to
> > > > failing the selection filter.  So people wanted to default to the
> > > > simpler case of key properties for selection.
> > >
> > > But you are constraining the entire solution to support just this
> > > use case, when a simpler unconstrained solution just as easily
> > > supports this use case, *plus other use cases* - if a client does
> > > not want these "fake *deletes", they can simply construct filters
> > > based on keys only, which they would in the current solution anyway.
> >
> > This is true.
> >
> > > Note also that b/c of the last paragraph in section 3.9, a client
> > > must deal with such deletes anyway, even if it specificied only
> > > keys.
> >
> > This is true.
> >
> > > But I do understand your point.  This is also related to my comment
> > > below about when a filter is evaluated.  Based on you reply to that
> > > comment, I think the conceptual algorithm is like this:
> > >
> > >   Just before a change, evaluate the filter and any access control
> > >   rules.  The result is a set "A" of nodes (including subnodes).
> > >
> > >   Just after a change, evaluate the filter and any (possibly new)
> > >   access control rules.  The result is a set "B" of nodes (including
> > >   subnodes).
> > >
> > >   Construct a YANG patch record for going from A to B.   If the record
> > >   is non-empty, send it to the subscriber.
> > >
> > > (this is conceptual, an implementation can do lots of optimizations
> > > to avoid evaluating filters on un-related changes)
> >
> > Yes, this is it.  The only tweak I would make is to add the dampening
> > period.  As a result, I have added the following text at the end of
> > the "On-Change Considerations" section
> >
> > "Putting it all together, following is the conceptual process for
> > creating an push-change-update notification:
> >
> >   1.Just before a change, or at the start of a dampening period,
> >   evaluate any filtering and any access control rules.  The result is a
> >   set "A" of datastore nodes and subtrees.
> >
> >   2.Just after a change, or at the end of a dampening period, evaluate
> >   any filtering and any (possibly new) access control rules.  The result
> >   is a set "B" of datastore nodes and subtrees.
> >
> >   3. Construct a YANG patch record for going from A to B.  If the record
> >   is non-empty, send it to the receiver."
> >
> > > > Based on that I can delete the words "but the goal is to provide
> > > > equivalent capabilities to what is available with a GET".
> > > >
> > > > >   With the current text, is this filter ok:
> > > > >
> > > > >      /interfaces/interface[contains(name, "eth")]
> > > > >
> > > > >   It filters on a key, so it should be ok, right?
> > > >
> > > > Yes
> > >
> > > What about this one:
> > >
> > >    /interfaces/interface[contains(name, //name)]
> > >
> > > it also filters on a key is it ok?
> > >
> > > [if the answer is "yes" we have a problem; and if it is "no", we
> > > need to tighten the text]
> > >
> > >
> > > All this makes me wonder if it is correct to have "subtree" and
> > > "xpath"
> > > filters at all.  If they only can match on keys, they become
> > > severely constrained.
> > >
> > > An alternative could be to specify filters as
> > > "nacm:node-instance-identifier"
> > > instead; these are instance-identitifiers that allow missing keys.
> > > For
> > > example:
> > >
> > >   /if:interfaces/if:interface/if:oper-status
> > >
> > > This would be easier to understand and probably easier to optimize
> > > for in the server code.
> >
> > I want to generalize this "only-keys?" issue for the full WG.  It is
> > too important to handle this deep in the thread.  Look for me to open
> > an issue shortly.
> 
> Ok!
> 
> /martin