Subject: Re: [Netconf] New Notification and Subscription Features WASFW: 3 Options for Subscription & Event Notification draft structure

From: Martin Bjorklund <[email-masked]>
Date: Wed, 01 Feb 2017 11:54:07 +0100 (CET)

Andy Bierman <[email-masked]> wrote:
> Hi,
> 
> I am not in favor of refactoring RFCs just to move content around.
> If/when NETCONF 1.2 gets done, then the NETCONF bits in RFC 7950 should
> get moved to that document. The NETCONF specific text is unfortunate,
> but not causing harm to interoperability.

I agree.  Among all the work we have on our plate at the moment, I
wouldn't give this highest priority.

/martin

> 
> 
> 
> 
> 
> On Tue, Jan 31, 2017 at 2:20 PM, Kent Watsen <[email-masked]> wrote:
> 
> >
> > > Since when has RFC 7950 JSON notification encoding sections? This
> > > email is confusing on multiple ends.
> >
> > True, RFC 7950 doesn't have any JSON encoding sections, so there is
> > nothing to be done on that front.   Also, while I agree that the XML
> > encoding sections should be move out of RFC 7950, I view this purely as a
> > NETMOD WG activity (not a NETCONF WG activity) and hence doesn't need to be
> > discussed here.
> >
> > What I think should be discussed here is moving all the NETCONF-isms in
> > RFC 7950 to the NETCONF WG including, but not limited to, all the NETCONF
> > XML Encoding Rules sections.
> >
> > RFC 7950 also contains a large number of NETCONF usage examples.  I think
> > the NETMOD WG needs to discuss if these should to be moved to the NETCONF
> > WG also, or if it's the better to convert half the examples to RESTCONF so
> > RFC 7590 shows impartiality.  Though this is a NETMOD issue to discuss, it
> > clearly might impact NETCONF, hence why I'm mentioning it here now.
> >
> > Separately, in addition to moving material from NETMOD drafts to NETCONF
> > drafts, I think there should be a discussion about moving material the
> > other direction as well.  For instance, I think these terms defined in RFC
> > 6241 might be better placed in the datastores draft:
> >
> >   o configuration data
> >   o configuration datastore
> >   o datastore
> >   o state data
> >
> >
> > Thoughts?
> >
> > Kent // a NETCONF contributor
> >
> >
> >
> >
> > _______________________________________________
> > Netconf mailing list
> > [email-masked]
> > https://www.ietf.org/mailman/listinfo/netconf
> >

---

From: "Mehmet Ersue" <[email-masked]>
Date: Wed, 1 Feb 2017 12:20:00 +0100

Hi Kent,

I believe the terms defined in 6241 are needed in the context of the network configuration framework.
I think it is perfectly fine if 7950bis uses terms defined in 6241.
We don't need to redefine or duplicate these terms in another draft.

Mehmet

-----Original Message-----
From: Kent Watsen [mailto:[email-masked]] 
Sent: Tuesday, January 31, 2017 11:20 PM
To: Juergen Schoenwaelder <[email-masked]>; Mehmet Ersue <[email-masked]>
Cc: [email-masked]
Subject: Re: [Netconf] New Notification and Subscription Features WASFW: 3 Options for Subscription & Event Notification draft structure

> email is confusing on multiple ends.

True, RFC 7950 doesn't have any JSON encoding sections, so there is nothing to be done on that front.   Also, while I agree that the XML encoding sections should be move out of RFC 7950, I view this purely as a NETMOD WG activity (not a NETCONF WG activity) and hence doesn't need to be discussed here.

What I think should be discussed here is moving all the NETCONF-isms in RFC 7950 to the NETCONF WG including, but not limited to, all the NETCONF XML Encoding Rules sections.  

RFC 7950 also contains a large number of NETCONF usage examples.  I think the NETMOD WG needs to discuss if these should to be moved to the NETCONF WG also, or if it's the better to convert half the examples to RESTCONF so RFC 7590 shows impartiality.  Though this is a NETMOD issue to discuss, it clearly might impact NETCONF, hence why I'm mentioning it here now.

Separately, in addition to moving material from NETMOD drafts to NETCONF drafts, I think there should be a discussion about moving material the other direction as well.  For instance, I think these terms defined in RFC 6241 might be better placed in the datastores draft:

  o configuration data

Thoughts?

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 1 Feb 2017 12:30:31 +0100

> We don't need to redefine or duplicate these terms in another draft.

We need to move towards a real architecture document that defines
overarching concepts and this architecture document should also
include the definintion of the core terminology we use.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Kent Watsen <[email-masked]>
Date: Wed, 1 Feb 2017 14:30:52 +0000

>> Andy writes:

I wouldn't refactor RFCs *just* to move content around.  I only
consider this in context of already creating (potentially) 6241bis
and 8040bis, per Mehmet's email.

In addition to potentially putting documents (b) and (c) from Mehmet's
email into said -bis docs, we should also discuss if we'd also want to
put the "how to support revised datastores" solution into said -bis
docs as well, or if we prefer these to be separate stand-alone docs.

> Martin writes:

Agreed, refactoring is not a high priority on its own.  I just see it
as timely to consider now.  I do think that the document collection is
beginning to become unwieldy, and we already know that RFC 7950 being
so NETCONF-oriented is rather misleading.

Kent

---

From: Kent Watsen <[email-masked]>
Date: Wed, 1 Feb 2017 14:44:26 +0000

> include the definition of the core terminology we use.

Agreed, and I'm thinking that the revised-datastores document might be
close enough to morph into this purpose.  If we did this, then we'd 
want to point both 6241bis and 8040bis to it.  

It's questionable if we'd want to point a 7950bis to such a doc.  On
one hand it is ideal for YANG to be independent of the datastore
concept (e.g., artifacts/data-structures) but, on the other hand,
there are YANG statements like 'config' that might require a
reference in order to make sense.

Kent

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 1 Feb 2017 16:17:22 +0100

> reference in order to make sense.

YANG can't be independent of datastores. YANG's notion of validation
is per configuration datastore. You mentioned the config statement.
Or do you suggest we have 'must' and 'when' in YANG without any
definition what they actually mean and then there is another document
that says what these statements mean for different datastores? Not
sure this makes YANG simpler to understand.

/js

---

From: Andy Bierman <[email-masked]>
Date: Wed, 1 Feb 2017 10:03:14 -0800

Hi,

I would really like to retire this misconception that datastores in YANG
are bad,
or a hack, or a mistake.

Datastores allow YANG to describe validation and other behavior quite
precisely,
independent of a protocol interaction model. E.g., NETCONF and RESTCONF
have very different interaction models, and they can both use the same YANG
modules.

YANG should map to datastores in a minimalist fashion.
Each protocol definition needs to map its use of datastores as detailed as
possible.

The biggest challenge is keeping the data model content as
datastore-independent
and protocol-independent as possible. We need to really understand the
revised
datastores architecture before changing YANG again.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 02 Feb 2017 11:49:19 +0100

> is per configuration datastore. You mentioned the config statement.

Not true: in RFC 7950 we already have e.g. this:

     o If the XPath expression is defined in a substatement to a data
       node that represents state data, the accessible tree is all state
       data in the server, and the running configuration datastore.

So in this case the instance data tree that is subject to validation is
constructed by combining the running configuration with state
data.

In another case the accessible tree is a merge of notification instance,
state data and running, etc.

I am pretty sure validity in YANG terms can be defined for *a* data
tree, no matter how the tree is constructed.

A particular protocol or application can then specify validity
requirements for any number of data trees (and check-points in the
protocol workflow) given that

- it is specified how each data tree is constructed

- the data model for each data tree is known.

If the data tree coincides with a datastore, then fine, but it is by no
means a necessity.

> sure this makes YANG simpler to understand.

IMO "must" and "when" work exactly the same way for any well-defined
accessible tree, regardless of how the tree is constructed, because it
is essentially XPath.

Lada

-- 
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: 0xB8F92B08A9F76C67

---

From: "Eric Voit (evoit)" <[email-masked]>
Date: Thu, 2 Feb 2017 22:25:10 +0000

> From: Netconf, January 31, 2017 5:18 PM
> 
> Thanks. It should read:
> 
> @All: Please comment on the documents a)-d) below. Especially on the
> questions:
> - whether the XML notification encoding section should be removed from 7950
> and put into NETCONF WG documents as well as

I don't see a need to touch 7950 at this time.  I do see a reason for a new notification draft which can support headers.  We should define that new notification in a separate draft, and collapse it into YANG 1.2 whenever there is sufficient business need.

> - whether the documents b) and c) are good to separate or the protocol-specific
> content should be put into 6241bis and RESTCONF-bis documents.

I would rather see (b) and (c) as separate documents.

For (b) benefits of separation include:
- security considerations and configuration for subscriptions
- considerations for parallel support / implementation compatibility with RFC-5277
- room for examples

In (c), the benefits of separation include:
- specifics of call-home and heartbeat for subscriptions
- security considerations and configuration for subscriptions
- benefits of HTTP2 for notifications (e.g., avoiding SSE), and HTTP2 Posts' interplay with GRPC.
- room for examples
it would be confusing to put all the above into Restconf.  And the ink on Restconf isn't dry yet.

> From: Juergen Schoenwaelder [mailto:[email-masked]]
> Sent: Tuesday, January 31, 2017 10:16 PM
> To: Mehmet Ersue <[email-masked]>
> Cc: 'Martin Bjorklund' <[email-masked]>; [email-masked]
> Subject: Re: [Netconf] New Notification and Subscription Features WASFW: 3
> Options for Subscription & Event Notification draft structure
> 
> On Tue, Jan 31, 2017 at 10:01:02PM +0100, Mehmet Ersue wrote:
> > The original document set proposed was:
> >
> > a) A document which defines the protocol-neutral notification
> > framework, i.e., explains the concepts of subscriptions, filters,
> > control plane notifications, replay, etc.  And also defines the
> > associated YANG data model, RPCs, etc. This is currently covered in
> > 5277bis document which would need to be renamed.

How about renaming to: "subscribed-notifications"

> > b) A document which defines how notifications are sent over NETCONF
> > (generalizing section 3.7 of RFC 5277) and how YANG notifications are
> > encoded in XML and JSON
> > (draft-ietf-netconf-netconf-event-notifications),
> >
> > c) A document which defines how notifications are sent over RESTCONF
> > (generalizing section 6 of RESTCONF RFC) and HTTP2.  Also defines how
> > YANG notifications are encoded in XML and JSON
> > (draft-ietf-netconf-restconf-notif)
> >
> > d) A document which defines the subscription and push mechanism for
> > YANG datastores allowing subscriber applications to request updates
> > from a YANG datastore (draft-ietf-netconf-yang-push).
> >
> >
> > This proposal was aiming to address the requirements Mahesh and I have
> > read on NETCONF and NETMOD maillists and discussed with Eric:
> >
> > - Provide the protocol-neutral notification framework in a separate
> > document.

Yes, I support a new protocol neutral notification draft which can support headers.

> > - Improve YANG specification RFC 7950 to be (as good as possible and
> > required) protocol-independent and remove XML and JSON notification
> > encoding sections for NETCONF. This was also the discussion result
> > between NETCONF and NETMOD co-chairs.

This is a good idea, but not urgent to me.

> > - Provide protocol-specific details e.g. how notifications are sent
> > and the notification encodings in separate documents. This part could
> > be for sure put into 6241bis and RESTCONF-bis. Though Eric and the
> > co-chairs were favoring to have separate documents.

Yes, these would be the (b) and (c) documents.

Eric

> >
> > That said I see Martin's comments below as valid.
> >
> > > > I think this is fine, but *if* this happens to coincide with a
> > rfc6241bis document, I would be nice to move the contents of the
> > second of these documents (i.e.,
> >
> > > > how notifications are sent over NETCONF) into rfc6241bis.  As has
> > > > been
> > stated before, bumping the protocol version to 1.2 would allow us to
> > change the
> >
> > > > "notification" element contents.
> >
> >
> >
> > @All: Please comment on the documents a)-d) above. Especially on the
> > questions:
> >
> > - whether the XML and JSON notification encoding sections should be
> > removed from 7950 and put into NETCONF WG documents as well as
> >
> > - whether the documents b) and c) are good to separate or they should
> > be put into 6241bis and RESTCONF-bis documents.
> >
> 
> Since when has RFC 7950 JSON notification encoding sections? This email is
> confusing on multiple ends.
> 
> /js
> 
> --

---

From: Kent Watsen <[email-masked]>
Date: Fri, 3 Feb 2017 00:34:34 +0000

> I don't see a need to touch 7950 at this time.  I do see a 
> reason for a new notification draft which can support headers.
> We should define that new notification in a separate draft,
> and collapse it into YANG 1.2 whenever there is sufficient
> business need.

Hi Eric, 

I don't understand the need to update 7950 YANG at all.  At 
least, the way I see it, headers are very much a function of
the protocol, and we're trying to move all protocol stuff out
of YANG...

When Andy and others mentioning wanting to model headers in
YANG, I think they were saying, for instance, augments against
NETCONF's YANG model.  At least that's what I thought.  

Maybe someone could post a snippet of YANG that illustrates
what is trying to be achieved?

K.

---

From: Andy Bierman <[email-masked]>
Date: Thu, 2 Feb 2017 16:56:37 -0800

Hi,

Using the yang-data extension, and a new extension:

module ex {
  prefix ex;

rc:yang-data notification {
  container notification {
     container hdr {
        leaf subscription-id { ... }
     }
     // no way to model the 2nd element which has a different name for each
replacement
     // this is the event element or a top-level element for YANG 1.1
nested notifications
  }
}

extension augment-yang-data {
  "top-level usage only
   argument is a string identifying the yang-data to augment";
}

ex:augment-yang-data ex:notification {
  augment "ex:notification/ex:hdr" {
     leaf sequence-id { ... }
  }
}

}

Andy

---

From: "Eric Voit (evoit)" <[email-masked]>
Date: Fri, 3 Feb 2017 01:47:53 +0000

Yes, this.

Hi,

module ex {
  prefix ex;

rc:yang-data notification {
  container notification {
     container hdr {
        leaf subscription-id { ... }
     }
     // no way to model the 2nd element which has a different name for each replacement
     // this is the event element or a top-level element for YANG 1.1 nested notifications
  }
}

}

Andy

Hi Eric,

K.

_______________________________________________
Netconf mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/netconf

---

From: Alexander Clemm <[email-masked]>
Date: Fri, 3 Feb 2017 02:42:26 +0000

+1 on (b) and (c) as separate documents (and hence a more modular document structure), for the reasons stated by Eric.  
--- Alex

-----Original Message-----
From: Netconf [mailto:[email-masked]] On Behalf Of Eric Voit (evoit)
Sent: Thursday, February 02, 2017 2:25 PM
To: Mehmet Ersue <[email-masked]>; [email-masked]
Subject: Re: [Netconf] New Notification and Subscription Features WASFW: 3 Options for Subscription & Event Notification draft structure

> - whether the XML notification encoding section should be removed from 
> 7950 and put into NETCONF WG documents as well as

> - whether the documents b) and c) are good to separate or the 
> protocol-specific content should be put into 6241bis and RESTCONF-bis documents.

> From: Juergen Schoenwaelder 
> [mailto:[email-masked]]
> Subject: Re: [Netconf] New Notification and Subscription Features 
> WASFW: 3 Options for Subscription & Event Notification draft structure
> 

> > (generalizing section 3.7 of RFC 5277) and how YANG notifications 
> > are encoded in XML and JSON 
> > (generalizing section 6 of RESTCONF RFC) and HTTP2.  Also defines 
> > how YANG notifications are encoded in XML and JSON
> > This proposal was aiming to address the requirements Mahesh and I 
> > have read on NETCONF and NETMOD maillists and discussed with Eric:
> >

> > and the notification encodings in separate documents. This part 
> > could be for sure put into 6241bis and RESTCONF-bis. Though Eric and 
> > the co-chairs were favoring to have separate documents.

Eric

> > > > how notifications are sent over NETCONF) into rfc6241bis.  As 
> > > > has been
> > - whether the documents b) and c) are good to separate or they 
> > should be put into 6241bis and RESTCONF-bis documents.
> >
>

---

From: t.petch <[email-masked]>
Date: Fri, 3 Feb 2017 12:24:42 +0000

----- Original Message -----
From: "Juergen Schoenwaelder" <[email-masked]>
Sent: Wednesday, February 01, 2017 11:30 AM

> include the definintion of the core terminology we use.

Yes, I think that this has been a need for quite a while now.  I would
see datastores as key to it and that means, for me, that it would to
some extent cut the ground from underneath YANG, RESTCONF, NETCONF,  ...
so while it is not 7950bis or 6241bis or ...it would be standards track
and would update many of the current documents.  Informational would,
for me, be pointless.

What I struggle with, always have, is having a single set of concepts
that spans, datastore, RPC, Notifications etc.  I have seen the much
discussed s.6.4 of RFC7960 and its predecessors as a symptom of this,
and likewise the often referred-to question of where is state data
vis-a-vis datastores.

It may be that there cannot be a single set of concepts, that the
architecture of an RPC has nothing meaningful in common with the
architecture of 'running', even if they share a DDL, but we should try.

Whether this belongs on the Netconf list or the Netmod list or another
list, I am less clear.

Tom Petch

---

From: "Mehmet Ersue" <[email-masked]>
Date: Tue, 7 Feb 2017 18:22:10 +0100

Hi Kent, All,

> Sent: Wednesday, February 1, 2017 3:44 PM
> close enough to morph into this purpose.  If we did this, then we'd want to
> point both 6241bis and 8040bis to it.

I think an architecture document would require to address a bigger picture concerning network configuration framework and its mechanisms.

That said I'm not sure what kind of architecture document we need and should strive for.
On the one hand I agree that a 'real architecture document' makes very much sense as a substantial document defining the "network configuration framework", its protocols and the interaction with configuration datastores. As a matter of fact preparing a normative architecture document beforehand and to build the protocol development on it has not been seen as necessary so far. We rather developed our documents iteratively based on a configuration framework written down mostly in 4741 and 4741bis. 8040 sees the NETCONF datastore concepts in 4741bis as substantial but uses only a subset of it.
RFC 6244 has been written after the finalization of NETCONF-4741bis and YANG 1.0. As an Informational RFC it was an introduction document bringing NETCONF and YANG into relation rather than providing a substantial architecture document.

OTOH I see it kind of late to produce a substantial architecture document which would become normative for our development. I assume people would be against waiting on the finalization of a normative architecture document. As I understand many are in favor of publishing the RFCs we have currently in our hands rapidly.

Though, comparable with 6244, it would be again very useful for people implementing configuration protocols to understand the bigger picture and environment if we provide an informational document describing the network configuration framework with all its pieces and the current IETF configuration protocols (including I2RS) as well as the usage of YANG for different cases. The revised datastore concept would be part of it as it addresses this architectural need.
 
> It's questionable if we'd want to point a 7950bis to such a doc.  On one hand it
> is ideal for YANG to be independent of the datastore concept (e.g.,
> artifacts/data-structures) but, on the other hand, there are YANG
> statements like 'config' that might require a reference in order to make
> sense.

Originally YANG has been specified as a modeling language for NETCONF. Now YANG is being prepared to use for diverse protocols or even more generally "for Network Configuration".

I think YANG as a modeling language should define the access to a data tree and the validation of a datastore on a generic level.
It is highly desirable to keep it, as much as possible, independent of a particular type of datastore and the details of the DS concept.