Subject: [netmod] Modeling NETCONF Event Notifications

From: Jernej Tuljak <[email-masked]>
Date: Tue, 03 Jul 2012 12:03:31 +0200

Hi,

I have questions regarding NETCONF Notifications and YANG. Which YANG 
module is the standard module that models RFC5277 notifications? I've 
got nc-notifications@2008-07-14 <mailto:nc-notifications@2008-07-14> and 
notifications@2008-07-14 <mailto:notifications@2008-07-14> over here, 
but it's quite evident that they are part of yuma's netconfd. Are these 
what I'm looking for?

More specifically should we generate XML schema for notifications in our 
generic client based on the root 
{urn:ietf:params:xml:ns:netconf:notification:1.0}notification container 
in the latter mentioned module and a combination of all available YANG 
notification statements which would represent some sort of an undefined 
merge of YANG statements? Are there rules on how this merge is supposed 
to happen? What are the XML encoding rules for the result of such a 
merge? I understand that the root notification container is there so 
modules can augment nodes to all notifications at once (instead of doing 
one augment per each available notification) and that netconfd handles 
this requirement in such a way, but what is the standard way of modeling 
this?

For event notifications in the following module (I hope the pretty 
printing remains untouched):

module augmented-notification {
     namespace "http://www.example.com/ns/augmented-notification";
     prefix an;
     import notifications {
         prefix ncEvent;
     }
     revision 2012-07-03;
     notification event {
         leaf severity {
             type enumeration {
                 enum major;
                 enum minor;
                 enum critical;
                 enum notice;
             }
         }
         leaf cause {
             type string;
         }
         leaf message {
             type string;
         }
     }
     augment "/ncEvent:notification" {
         leaf sequence-id {
             type int32;
         }
     }
}

how does a generic client determine for a notification like this 
(represents an instance of the merge):

<?xml version="1.0" encoding="utf-8"?>
<notification xmlns="urn:ietf:params:xml:ns:netconf:notification:1.0"
               xmlns:an="http://www.example.com/ns/augmented-notification">
<eventTime>2012-03-07T11:00:00Z</eventTime>
<an:event>
<an:severity>critical</an:severity>
<an:cause>this and that</an:cause>
<an:message>Stuff happened.</an:message>
</an:event>
<an:sequence-id>1</an:sequence-id>
</notification>

which child element is the actual notification content and which are the 
augmented nodes? How does it even know that an additional node will be 
present? Suppose that the above module would add an "event" node instead 
of "sequence-id". Should such a module still be considered a valid 
module or not (note that this creates a name conflict).

I can see that it can be very useful to have access to the "base" 
notification, but one should be able to model this within YANG (forget 
extensions unless they are standardized), not resort to some sort of 
external hacks, right?

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 3 Jul 2012 13:58:32 +0200

On Tue, Jul 03, 2012 at 12:03:31PM +0200, Jernej Tuljak wrote:
 
> I have questions regarding NETCONF Notifications and YANG. Which
> YANG module is the standard module that models RFC5277
> notifications? I've got nc-notifications@2008-07-14
> <mailto:nc-notifications@2008-07-14> and notifications@2008-07-14
> <mailto:notifications@2008-07-14> over here, but it's quite evident
> that they are part of yuma's netconfd. Are these what I'm looking
> for?

RFC 5277 predates the 'yangification' of the NETCONF specification and
there likely is a need to update/redo RFC 5277 at some point in time.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: "Ersue, Mehmet (NSN - DE/Munich)" <[email-masked]>
Date: Tue, 3 Jul 2012 16:01:33 +0200

True. Is there any volunteer willing to yangify RFC 5277?

Mehmet 

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
Behalf Of ext
> Juergen Schoenwaelder
> Sent: Tuesday, July 03, 2012 1:59 PM
> To: Jernej Tuljak
> Cc: NETMOD Working Group
> Subject: Re: [netmod] Modeling NETCONF Event Notifications
> 
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

---

From: Andy Bierman <[email-masked]>
Date: Tue, 3 Jul 2012 08:53:59 -0700

On Tue, Jul 3, 2012 at 3:03 AM, Jernej Tuljak <[email-masked]> wrote:
> Hi,
>
> I have questions regarding NETCONF Notifications and YANG. Which YANG module
> is the standard module that models RFC5277 notifications? I've got
> nc-notifications@2008-07-14 and notifications@2008-07-14 over here, but it's
> quite evident that they are part of yuma's netconfd. Are these what I'm
> looking for?

These are internal to the yuma server.
All the protocol data structures are modeling in YANG and data-driven
in the server.
I think it represents the standard (with a sequence ID leaf added).
There are no YANG modules for the XSDs in RFC 5277.

> {urn:ietf:params:xml:ns:netconf:notification:1.0}notification container in
> the latter mentioned module and a combination of all available YANG
> merge of YANG statements? Are there rules on how this merge is supposed to
> happen? What are the XML encoding rules for the result of such a merge? I
> understand that the root notification container is there so modules can
> augment nodes to all notifications at once (instead of doing one augment per
> each available notification) and that netconfd handles this requirement in
> such a way, but what is the standard way of modeling this?
>

This is an internal implementation decision.
The notification root is there in yuma for internal parsing.

I suggest using YANG for modeling notifications,
but if your internal tools use XSD then implement with XSD
and advertise the equivalent YANG module.

> For event notifications in the following module (I hope the pretty printing
> remains untouched):
>

Using YANG to model the notification message is an implementation decision.
Not sure why you would not just use YANG notification-stmt instead.

Andy

> how does a generic client determine for a notification like this (represents
> an instance of the merge):
>
> present? Suppose that the above module would add an "event" node instead of
> "sequence-id". Should such a module still be considered a valid module or
> not (note that this creates a name conflict).
>

---

From: Jernej Tuljak <[email-masked]>
Date: Wed, 04 Jul 2012 09:43:48 +0200

Dne 3.7.2012 17:53, piše Andy Bierman:
> On Tue, Jul 3, 2012 at 3:03 AM, Jernej Tuljak<[email-masked]>  wrote:
>> Hi,
>>

I'd be glad to do that if the YANG notifications could express the model 
of notifications sent by netconfd and potentially others. I consider 
netconfd as the next best thing to a reference implementation of a 
NETCONF server. I was quite surprised to receive notifications from it 
that could not possibly be expressed through YANG without some 
additional knowledge (at least not through the module set advertised by 
it). So I assumed there had to be something behind this.

As I said once before, a generic client must know the exact structure of 
every possible payload that could be formed and sent by the server. And 
the model of these payloads is expressed through YANG (also if I 
understood correctly from previous discussions the plan for the future 
is to integrate YANG into NETCONF?). RFC6020 suggests so in 5.6.1. [

    The model defines a contract between the NETCONF client and server,
    which allows both parties to have faith the other knows the syntax
    and semantics behind the modeled data.  The strength of YANG lies in
    the strength of this contract.

]. A client should therefore require only YANG and nothing else to be 
able to understand what the server is saying.

Our client could easily consider sequence-id as a breach of this 
contract and simply refuse to accept such notifications. But I don't 
think this is the answer to the problem at hand. I agree that RFC5277 
should be revisited. It should be possible to add server specific 
information to each notification in a way which makes the client aware 
of it.

Jernej

---

From: Andy Bierman <[email-masked]>
Date: Wed, 4 Jul 2012 07:05:37 -0700

On Wed, Jul 4, 2012 at 12:43 AM, Jernej Tuljak <[email-masked]> wrote:
>>> module
>>> in
>>> to
> I'd be glad to do that if the YANG notifications could express the model of
> notifications sent by netconfd and potentially others. I consider netconfd
> as the next best thing to a reference implementation of a NETCONF server. I
> was quite surprised to receive notifications from it that could not possibly
> be expressed through YANG without some additional knowledge (at least not
> through the module set advertised by it). So I assumed there had to be
> something behind this.
>
> every possible payload that could be formed and sent by the server. And the
> model of these payloads is expressed through YANG (also if I understood
> correctly from previous discussions the plan for the future is to integrate
> YANG into NETCONF?). RFC6020 suggests so in 5.6.1. [
>
> ]. A client should therefore require only YANG and nothing else to be able
> to understand what the server is saying.
>

No -- this was not the charter of the NETMOD WG to create YANG.
YANG very explicitly covers the NETCONF content and operation layers.
The original proposal did not even cover RPCs and notifications --
just the database.

Yuma uses YANG internally for more layers but that is just an
implementation choice.

> Our client could easily consider sequence-id as a breach of this contract
> and simply refuse to accept such notifications.

I have to check the XSD again, but I thought it was extensible.
YANG only covers the event element payload.  I guess I should
add a CLI parameter to netconfd to disable sequence-id if it
conflicts with the XSD in RFC 5277.

 But I don't think this is
> the answer to the problem at hand. I agree that RFC5277 should be revisited.
> It should be possible to add server specific information to each
> notification in a way which makes the client aware of it.
>

I think XSD or RelaxNG has to be used for this purpose, not YANG.
YANG allows only the event payload to be augmented, not the message.
Standards often leave out features like sequence-id that would help
operators and developers
determine if access control and notification filters are doing what is
expected or not.
The sequence-id is an easy way to debug this sort of issue.   At the
time, the NETCONF
WG did not think notifications needed any sort of diagnostic support, so it
was included in the standard.

Andy

>>> of

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 04 Jul 2012 20:38:56 +0200

> YANG very explicitly covers the NETCONF content and operation layers.

Hmm, I would say that YANG very explicitly covers the NETCONF content layer and it was also (mis)used for the operations layer in RFC 6241. Note that in order to model XML attributes, the "ietf-netconf" module had to introduce an extension.

> Yuma uses YANG internally for more layers but that is just an

No wonder people are misled into thinking that YANG is just another XML schema language.

...

> I think XSD or RelaxNG has to be used for this purpose, not YANG.

A set of extensible RELAX NG schemas covering base NETCONF can be found in my long-expired draft http://tools.ietf.org/id/draft-lhotka-netconf-relaxng-00.txt.
Extensible schemas for notifications (as well as other optional capabilities) could be easily added in a similar vein.

Lada

-- 
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: E74E8C0C

---

From: Jernej Tuljak <[email-masked]>
Date: Thu, 05 Jul 2012 11:55:05 +0200

Dne 4.7.2012 20:38, piše Ladislav Lhotka:
> Andy Bierman<[email-masked]>  writes:

Even if this was not the WG intention, the current RFCs suggest the 
following. Notifications are part of operations layer and have _no_ 
representation in the RPC layer. Take a look at RFC5277 (1. 
Introduction) and compare the diagram of NETCONF protocol layers to the 
one in RFC6241. The diagram in the first document clearly allows RPC 
layer to be "bypassed" for the sole purpose of notifications. RFC6020 
states that it models notifications. Therefore YANG simply must handle 
the entire model of a notification payload, not just the event 
(especially if we are talking about server-specific information, 
eventTime leaf is not an issue as examples of XML encoding for 
notification-stmt indicate). This would be quite simple if an additional 
restriction would be put in there which would require that the event 
always appears after eventTime leaf and that garbage might follow.

Yuma's netconfd server violates the YANG contract by adding stuff where 
it currently cannot be added. It doesn't matter if RFC5277 has an 
extensible schema since you have no way of telling the client what's 
going on. Which is the event and which is the appended information? Not 
without hacking around (or by augmenting to every notification-stmt). 
What you did here is an attempt of augmentation of the RPC layer which 
does not exist for notifications.

It would be different if notifications where in the RPC layer, so a 
custom wrapper around the event could be easily identified. But i think 
YANG handling this in a future version would be a better way to go. Why 
make RPC and transport layers which are well known for everyone, as not 
such? Let YANG handle the rest.

I don't really care if the server sends debug information along with a 
notification. But I must be able to unambiguously determine what the 
event was and what the added information is. There's no well known way 
to determine/inform of this in a hello message or a notification itself. 
Therefore it should not be done.

Jernej

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 5 Jul 2012 12:07:43 +0200

On Thu, Jul 05, 2012 at 11:55:05AM +0200, Jernej Tuljak wrote:
> Introduction) and compare the diagram of NETCONF protocol layers to
> the one in RFC6241. The diagram in the first document clearly allows
> RPC layer to be "bypassed" for the sole purpose of notifications.
> RFC6020 states that it models notifications. Therefore YANG simply
> must handle the entire model of a notification payload, not just the
> event (especially if we are talking about server-specific
> information, eventTime leaf is not an issue as examples of XML
> encoding for notification-stmt indicate). This would be quite simple
> if an additional restriction would be put in there which would
> require that the event always appears after eventTime leaf and that
> garbage might follow.
> 
> Yuma's netconfd server violates the YANG contract by adding stuff
> where it currently cannot be added. It doesn't matter if RFC5277 has
> an extensible schema since you have no way of telling the client
> what's going on. Which is the event and which is the appended
> information? Not without hacking around (or by augmenting to every
> notification-stmt). What you did here is an attempt of augmentation
> of the RPC layer which does not exist for notifications.
> 
> custom wrapper around the event could be easily identified. But i
> think YANG handling this in a future version would be a better way
> to go. Why make RPC and transport layers which are well known for
> everyone, as not such? Let YANG handle the rest.
> 
> I don't really care if the server sends debug information along with
> a notification. But I must be able to unambiguously determine what
> the event was and what the added information is. There's no well
> known way to determine/inform of this in a hello message or a
> notification itself. Therefore it should not be done.

I think it is important to keep in mind the history of these
documents. The event notifications addition to NETCONF was published
in July 2008, the YANG working group got established in April 2008.
The idea to redo the NETCONF base specification in YANG was born even
much later. Hence, it is no wonder that things might not align as well
as they perhaps could.

I think your real issue is that RFC 5277 kind of hard codes what the
basic information is that goes into the notification. But then there
was no YANG when this was done (and the authors/editors of that
document believed that XSD is anyway the industry solution to data
modeling and perhaps the XSD is extensible... ;-).

My take is that RFC 5277 may need a revision. But that would be the
job of the NETCONF working group, not the job of the NETMOD working
group (this list).

/js

---

From: Andy Bierman <[email-masked]>
Date: Thu, 05 Jul 2012 07:38:34 -0700

On 07/05/2012 02:55 AM, Jernej Tuljak wrote:
>> Hmm, I would say that YANG very explicitly covers the NETCONF content layer and it was also (mis)used for the operations layer in RFC 6241. Note that in order to model XML attributes, the 
>> "ietf-netconf" module had to introduce an extension.
>
> Even if this was not the WG intention, the current RFCs suggest the following. Notifications are part of operations layer and have _no_ representation in the RPC layer. Take a look at RFC5277 (1. 
> Introduction) and compare the diagram of NETCONF protocol layers to the one in RFC6241. The diagram in the first document clearly allows RPC layer to be "bypassed" for the sole purpose of 
> notifications. RFC6020 states that it models notifications. Therefore YANG simply must handle the entire model of a notification payload, not just the event (especially if we are talking about 
> server-specific information, eventTime leaf is not an issue as examples of XML encoding for notification-stmt indicate). This would be quite simple if an additional restriction would be put in 
> there which would require that the event always appears after eventTime leaf and that garbage might follow.
>
> Yuma's netconfd server violates the YANG contract by adding stuff where it currently cannot be added. It doesn't matter if RFC5277 has an extensible schema since you have no way of telling the 
> client what's going on. Which is the event and which is the appended information? Not without hacking around (or by augmenting to every notification-stmt). What you did here is an attempt of 
> augmentation of the RPC layer which does not exist for notifications.
>

I don't agree -- it violates the XSD in RFC 5277, not YANG.
I will change netconfd to disable this extension by default.

There are problems with NETCONF notifications that can't be fixed
by tweaking RFC 5277, such as the call-home problem and the open session
to every server.

 From a modeling POV, it is incorrect to augment each event payload with
meta-data related to the notification itself.  I don't know if YANG needs to support
augmenting the notification element.  IMO, a really low priority.

> It would be different if notifications where in the RPC layer, so a custom wrapper around the event could be easily identified. But i think YANG handling this in a future version would be a better 
> way to go. Why make RPC and transport layers which are well known for everyone, as not such? Let YANG handle the rest.
>
> I don't really care if the server sends debug information along with a notification. But I must be able to unambiguously determine what the event was and what the added information is. There's no 
> well known way to determine/inform of this in a hello message or a notification itself. Therefore it should not be done.
>
> Jernej

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 5 Jul 2012 18:09:09 +0200

On Jul 5, 2012, at 11:55 AM, Jernej Tuljak wrote:

> Even if this was not the WG intention, the current RFCs suggest the following. Notifications are part of operations layer and have _no_ representation in the RPC layer. Take a look at RFC5277 (1. Introduction) and compare the diagram of NETCONF protocol layers to the one in RFC6241. The diagram in the first document clearly allows RPC layer to be "bypassed" for the sole purpose of notifications. RFC6020 states that it models notifications. Therefore YANG simply must handle the entire model of a notification payload, not just the event (especially if we are talking about server-specific information, eventTime leaf is not an issue as examples of XML encoding for notification-stmt indicate). This would be quite simple if an additional restriction would be put in there which would require that the event always appears after eventTime leaf and that garbage might follow.
> 
> Yuma's netconfd server violates the YANG contract by adding stuff where it currently cannot be added. It doesn't matter if RFC5277 has an extensible schema since you have no way of telling the client what's going on.

Why not? You could use additional capabilities.

Lada

> Which is the event and which is the appended information? Not without hacking around (or by augmenting to every notification-stmt). What you did here is an attempt of augmentation of the RPC layer which does not exist for notifications.
> 
> It would be different if notifications where in the RPC layer, so a custom wrapper around the event could be easily identified. But i think YANG handling this in a future version would be a better way to go. Why make RPC and transport layers which are well known for everyone, as not such? Let YANG handle the rest.
> 
> I don't really care if the server sends debug information along with a notification. But I must be able to unambiguously determine what the event was and what the added information is. There's no well known way to determine/inform of this in a hello message or a notification itself. Therefore it should not be done.
> 
> Jernej

---

From: Andy Bierman <[email-masked]>
Date: Thu, 05 Jul 2012 09:39:19 -0700

> Why not? You could use additional capabilities.

You could rewrite a lot of things, but right now, the XSD in RFC 5277
does not allow extensions (no <xsd:any namespace="##other"> element in the sequence).

Andy

---

From: "Alexander Clemm (alex)" <[email-masked]>
Date: Thu, 5 Jul 2012 20:29:29 +0000

Regarding some basic information in Netconf event notifications,  I am not sure why it would need to be defined via YANG.  Aspects that cover information such as time stamp, severity, event type, reporting entity - basically, the type of information for which the format is also defined in syslog, and which is common to any kind of event - could IMHO very well be "hard coded" as notification parameters, just like operation parameters for other Netconf operations. It is not clear to me what would be gained by defining such parameters as YANG modules.  
--- Alex  

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Juergen Schoenwaelder
Sent: Thursday, July 05, 2012 3:08 AM
To: Jernej Tuljak
Cc: [email-masked]
Subject: Re: [netmod] Modeling NETCONF Event Notifications

> >Andy Bierman<[email-masked]>  writes:
.....

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 5 Jul 2012 23:51:15 +0200

> Regarding some basic information in Netconf event notifications,  I am not sure why it would need to be defined via YANG.  Aspects that cover information such as time stamp, severity, event type, r…

I think the thread started because someone found the things currently
hard coded not sufficient and there is no easy way to augment things.
If you are fine with what is there, feel happy. ;-)

/js

---

From: Andy Bierman <[email-masked]>
Date: Thu, 05 Jul 2012 18:53:08 -0700

> True. Is there any volunteer willing to yangify RFC 5277?

Yes -- since I already started with notifications.yang and nc-notifications.yang.

Andy

> Netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf

---

From: Andy Bierman <[email-masked]>
Date: Thu, 05 Jul 2012 19:08:23 -0700

> Regarding some basic information in Netconf event notifications,  I am not sure why it would need to be defined via YANG.  Aspects that cover information such as time stamp, severity, event type, r…

At 1 level, it is the same.  You could either advertise a special capability,
like notification:1.0, interleave:1.0, etc., or you could advertise a YANG module
capability instead (ietf-netconf-notifications.yang).

The difference is in the tool automation.
Every little foo:1.x has hard-wired semantics.
The <filter> hack in ietf-netconf.yang is the YANG version the same thing.

But using plain standard YANG with augmentations is not hard-wired.
That's why Jernej is right, and there should either be no vendor/WG extensibility
at all to the <notification> header, or it should be done in a standard way, preferably YANG.
We may also need to support multiple encodings, so I prefer not to rely on undocumented
XML attributes, ala <rpc> header attributes.

Andy

---

From: Chris Elliott <[email-masked]>
Date: Thu, 5 Jul 2012 22:37:26 -0400

On Thursday, July 5, 2012 at 10:08 PM, Andy Bierman wrote:
> at all to the <notification> header,  
>  
>  

In which case we're just duplicating syslog functionality and we might as well not do notifications at all, in my opinion.
  
> or it should be done in a standard way, preferably YANG.
>  
>  

I strongly prefer using YANG and making notifications extensible.

Chris.