Subject: [netmod] NMDA indication of value "in use" vs. defaults

From: David Lamparter <[email-masked]>
Date: Mon, 17 Jul 2017 19:28:01 +0200

Hi all,

so, I need to partially walk back my comments made on mic earlier today.
The context where this came from is section 7.6.1 ("The leaf's default
value") of both RFCs 7950 and 6020.  It says:

   When the default value is in use, the server MUST operationally
   behave as if the leaf was present in the data tree with the default
   value as its value.

The point where my earlier arguments fall flat is where it says "the
server".  My memory is that both some servers as well as libraries
effectively "hard-implement" this paragraph; i.e. when you retrieve a
value from a loaded configuration, there is no way to distinguish
between "X passed in" and "the library filled in X from the model's
default".  The library doesn't give you the opportunity to break above
paragraph.

So, since the discussion was in regard to values read back out from the
server, the YANG language specs themselves theoretically allow this.

... if you accept that this only applies to configuration written on the
server.  You could as well read that paragraph to also apply to reading
out configuration.

And it gets worse.  RFC 6243 (with-defaults) starts off with this:
   1.  Data retrieval: A server is normally allowed to exclude data
       nodes that it considers to contain the default value.  The actual
       nodes omitted depend on the default-handling behavior used by the
       server.

If this is implemented in an older NETCONF server that does this
unconditionally, this effectively breaks operational state retrieval
even if the system software does provide correct data with or without
defaults.  The NETCONF server would just (validly) remove all values
that are the same as the model's default -- which then the client would
interpret as "these values are not in use".

In the same way, "report-all" and "trim" modes for with-defaults would
fundamentally break this model where absence of the value indicates
not-in-use.  "explicit" should work;  "report-all-tagged" may work
depending on the implementation.

...

This, for me, lends the conclusion that this would be massively fragile,
even if a specific combination could make it work.  My position hence
changes from "this doesn't work" to "please don't do this, even if it
may work."

The solution I would suggest would be to have "is the value in use?" as
metadata just like the origin.  It can either be a new value on the
"datastore" identity (=> if a value is not in use, there is no way to
report what datastore it came from), or alternatively it can be a
separate metadata annotation (=> you can report not-in-use and still
have the value and its origin - if that makes sense, which...)

(I'm leaning towards adding "not-in-use" on the origin metadata
annotation, as opposed to a separate boolean metadata annotation.)

Whether the value would have any meaning if "in-use=false" or
"datastore=not-in-use" are signalled is an open question with this
approach.  I would argue if the value isn't in use down in the system,
the value sent to the client (if any) is meaningless (and the system
usually wouldn't be able to provide it to begin with.)  I haven't
explored the details of this though.

Hope this is useful input,

-David

P.S.: The original case where I stumbled over this is OpenConfig's BGP
model having some leaves with default values, and then saying "if the
value is not present, use the parent's value".  This is a hard violation
of the YANG spec;  if a default value is in the model, the server isn't
permitted to deviate and hunt for the parent's value in lieu of using
the model default.  (This was a year ago, I think/hope the model
changed.)

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 17 Jul 2017 23:51:49 +0200

David,

I am still not sure what the problem is. We proposed that the
<operational> datastore always reports the values that are in use,
whatever their origin is. This seems simple and clear if you want to
troubleshoot something.

> interpret as "these values are not in use".

RFC 6243 was written before <operational> did exist and we propose
that <operational> always reports the values in use (regardless
whether they match defaults or not).

> changes from "this doesn't work" to "please don't do this, even if it

Always reporting the value in use should be fine. If libraries mess up
the data it seems a library issue. Libraries should not mess around
with <operational> data. The data in <operational> data is the ground
truth of what the device is really doing, messing around with it
before passing it to an application is not recommended.

> annotation, as opposed to a separate boolean metadata annotation.)

But if something is not in use, it may not even have a value. So I end
up reporting fake values for things that are not is use decorated with
metadata that these fake values are not in use? Does not sound very
efficient to me.

> the model default.  (This was a year ago, I think/hope the model

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: David Lamparter <[email-masked]>
Date: Tue, 18 Jul 2017 10:57:11 +0200

Jürgen,

> whatever their origin is. This seems simple and clear if you want to

This is not about the value - this is about the separate single bit of
information that answers the question "is this value in use?"

My understanding is that the NMDA design wants to communicate this
single bit of information to the client that retrieves data from the
operational datastore.

I'm arguing that this single bit of information should not be
transported as "is the value present or absent?", because defaults
specified in the model can interfere with the absence of values.

> whether they match defaults or not).

Yes, you can make this work if you specify that the NETCONF server MUST
NOT make any additions or removals regarding defaults for operational
requests.  I'm just saying this won't be robust.

> before passing it to an application is not recommended.

Sure -- I'm not concerned about the value.  I'm concerned I won't be
able to tell whether the value is "in use".  This is 2 error cases:

- the value in use is equal to the model's default, and due to mangling
  it's misrepresented as "there is no value" / "the value is not in use"
- the value is not in use, and due to mangling it's misrepresented as
  "the value in use happens to be the default value"

> metadata that these fake values are not in use? Does not sound very

I would actually suggest that the default for the _origin_metadata_ is
"not-in-use", so you can in fact leave it out of the reply, and the
client knows reliably that it's not in use -- even if something in the
data path processes/interferes with defaults.

Cheers,

-David

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 18 Jul 2017 14:50:12 +0200

> single bit of information to the client that retrieves data from the

Yes, 'in use' <=> 'there is a value' in <operational> (the conceptual
datastore).
 
> specified in the model can interfere with the absence of values.

They interfere (i) if you make them interfere or (ii) you have
intermediaries that mess around with the data without telling you that
they did so. We are trying to resolve (i) by definition of semantics
and (ii) seems outside of what specifications can deal with.

> requests.  I'm just saying this won't be robust.

This is what we are trying to specify.

>   "the value in use happens to be the default value"

I do not think server randomly mangle data. I am concerned about
reporting 'fake' values for not in use leafs with metadata that such
leafs are not in use. This seems pretty costly (there usually is lots
of stuff not in use).

> data path processes/interferes with defaults.

In XML, metadata is encoded as XML attributes so I need to ship an XML
element to ship the XML attribute and hence I need a value (and an
empty value might violate the YANG data type contract, so I need a
fake value). This does not seem a good proposal to me.

/js

---

From: David Lamparter <[email-masked]>
Date: Tue, 18 Jul 2017 16:50:19 +0200

> and (ii) seems outside of what specifications can deal with.

Well, I don't want to give people the rope to hang themselves with; I
fully understand your position and it's correct by the letter, yet I
uphold my objection rooted on philosopohical differences in design
practice ;)

> leafs are not in use. This seems pretty costly (there usually is lots

I'm not suggesting you make up fake values.  [See below.]

> fake value). This does not seem a good proposal to me.

Let me rephrase/develop my suggestion.

- don't send anything for a value that's not in use.
- specify that if a client can't find an origin attribute, e.g. because
  the entire tag is missing, it MUST assume "not-in-use"
This implies:
- data that actually is in use MUST always have an origin attribute,
  otherwise the client will go "not-in-use"

This kinda forces people to do the right thing not only for not-in-use
signaling, but also for the origin metadata.  No easy way out in just
not sending it, if you don't know you have to own up your amnesia and
admit to "unknown".

... and with the current wording in the draft, it already suggests a
value in all cases, since the last paragraph in in 4.7.3 is:
   In the case that a device cannot provide an accurate origin for a
   particular data node then it should use the origin "unknown".

I'm just saying that to determine whether a value is in use, the
data's consumer must look at the origin attribute (including its
absence), not at the fact whether there is some value.

-David

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 18 Jul 2017 17:29:14 +0200

> absence), not at the fact whether there is some value.

This is the relevant current text:

   Requests to retrieve nodes from <operational> always return the value
   in use if the node exists, regardless of any default value specified
   in the YANG module.  If no value is returned for a given node, then
   this implies that the node is not used by the device.

I am still unsure why the presence of an origin attribute adds value
here. The last sentence clearly says 'if no value is returned for a
given node, then this implies that the node is not used by the
device.'

FYI: There is another ongoing discussion whether origin metadata for
config false nodes makes sense at all and one option considered right
now is to have origin metadata only on config true nodes. If so, you
can't overload the presence of origin metadata with another meaning.

/js