Subject: [Netconf] Fwd: New Version Notification for draft-varga-netconf-exi-capability-00.txt

From: Robert Varga <[email-masked]>
Date: Thu, 11 Jul 2013 08:22:23 +0200

Hi everyone,

I have just uploaded a new draft specifying an extension to NETCONF 
protocol which allows the message encoding to be switched to Efficient 
XML Interchange (EXI).

Any comments/feedback are welcome.

Thanks,
Robert

-------- Original Message --------
Subject: 	New Version Notification for 
draft-varga-netconf-exi-capability-00.txt
Date: 	Wed, 10 Jul 2013 23:11:48 -0700
From: 	[email-masked]
To: 	Robert Varga <[email-masked]>

A new version of I-D, draft-varga-netconf-exi-capability-00.txt
has been successfully submitted by Robert Varga and posted to the
IETF repository.

Filename:	 draft-varga-netconf-exi-capability
Revision:	 00
Title:		 Efficient XML Interchange Capability for NETCONF
Creation date:	 2013-07-11
Group:		 Individual Submission
Number of pages: 7
URL:             http://www.ietf.org/internet-drafts/draft-varga-netconf-exi-capability-00.txt
Status:          http://datatracker.ietf.org/doc/draft-varga-netconf-exi-capability
Htmlized:        http://tools.ietf.org/html/draft-varga-netconf-exi-capability-00

Abstract:
    The Network Configuration Protocol (NETCONF) provides mechanisms to
    install, manipulate, and delete the configuration of network devices
    via exchange of XML messages in textual representation.  Efficient
    XML Interchange (EXI) is a W3C-recommended binary representation of
    XML Information Set, which is more efficient from both CPU and
    bandwidth utilization perspective.  This document defines a
    capability-based extension to the NETCONF protocol that allows peers
    to agree to exchange protocol messages using EXI encoding.

The IETF Secretariat

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 11 Jul 2013 09:23:14 +0200

On Thu, Jul 11, 2013 at 08:22:23AM +0200, Robert Varga wrote:
> protocol which allows the message encoding to be switched to
> Efficient XML Interchange (EXI).

Did you implement this? If so, what is the gain in

a) consumed bandwidth,
b) reduced code size,
c) runtime overhead?

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Michal 'vorner' Vaner <[email-masked]>
Date: Thu, 11 Jul 2013 09:44:46 +0200

Good morning

> Did you implement this? If so, what is the gain in

May I suggest to compare it with enabling compression in the underlying ssh
connection, with `ssh -C` or equivalent?

With regards

-- 
Michal 'vorner' Vaner

---

From: "ietfdbh" <[email-masked]>
Date: Thu, 11 Jul 2013 06:50:31 -0400

Hi,

A major reason for moving from SNMP's ASN.1 binary encoding to XML's
text-based encoding was operator request.

Operators preferred a solution that was readable on-the-wire, so they could
look at packet-dumps and see the information being passed.
Operators found that SNMP packets had to be decoded by (non-standard)
applications to make the data human-readable, and often an application UI
never showed the raw varbinds; and the information presented to the operator
might have been cached, or might have been "interpreted" before being shown
to the operator. Even when the decoded raw varbinds were shown, such as with
a MIB browser, the OIDs were not human-friendly. By having the data encoded
in XML, the on-the-wire packet content was readable in a packet dump, and
the labels were meaningful. (Of course, with encryption, the content was
unreadable.)

I would be interested in seeing a discussion of these points in a comparison
of XML, EXI, and "ssh -C" encodings. Such discussion should probably include
whether popular packet sniffing tools, both commercial and open source,
supported the formats, and at what level in the transfer of information
between systems (i.e. in the network stack and/or the Netconf architecture)
an operator can access something human-readable.

It would also be interesting to discuss the adoption level of each approach.
EXI was approved in 2011; When was "ssh -C" approved as a standard?
Do we have any information about how many commercial/open-source toolkits
support the standards?

I don't know EXI. Wikipedia indicates EXI uses "schema-informed"
compression, and the decoder needs a copy of the same schema that the
encoder used . How would this potentially affect Netconf? Assuming multiple
Netconf clients (NMSes) talk to multiple Netconf servers, does that mean
that each client must have the managed-device-specific schema for each
device, and each netconf server must have a copy of the schema used by each
client? While such an approach might be viable for a web browser/web server
relationship, where each is hosted on a device that is not very resource
constrained, but what impact would this have on devices with significant
resource constraints (as is common with Netconf)?

David Harrington
[email-masked]
+1-603-828-1401

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Michal 'vorner' Vaner
> Sent: Thursday, July 11, 2013 3:45 AM
> To: [email-masked]
> Subject: Re: [Netconf] Fwd: New Version Notification for draft-varga-
> netconf-exi-capability-00.txt
> 
> Good morning
> 
> _______________________________________________
> Netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf

---

From: "David Harrington" <[email-masked]>
Date: Thu, 11 Jul 2013 07:24:55 -0400

Hi,

Since EXI is schema-informed, would a EXI-standard-compliant implementation
work with a YANG schema, or would a compliant implementation require a
particular type of schema, such as W3C-standard XML Schema (XSD)? 

At what point in the process does EXI becomes schema-informed? Is this a
special schema that is used only during the encode/decode steps, totally
independent of where in the process a YANG schema or RELAX-NG schema would
get used with Netconf?

David Harrington
[email-masked]
+1-603-828-1401

> From: ietfdbh [mailto:[email-masked]]
> Sent: Thursday, July 11, 2013 6:51 AM
> To: [email-masked]
> Subject: RE: [Netconf] Fwd: New Version Notification for draft-varga-
> never showed the raw varbinds; and the information presented to the
> operator might have been cached, or might have been "interpreted" before
> being shown to the operator. Even when the decoded raw varbinds were
> shown, such as with a MIB browser, the OIDs were not human-friendly. By
> having the data encoded in XML, the on-the-wire packet content was
> readable in a packet dump, and the labels were meaningful. (Of course,
with
> encryption, the content was
> unreadable.)
> 
> Do we have any information about how many commercial/open-source
> toolkits support the standards?
> 
> device, and each netconf server must have a copy of the schema used by
> each client? While such an approach might be viable for a web browser/web
> server relationship, where each is hosted on a device that is not very
> resource constrained, but what impact would this have on devices with
> significant resource constraints (as is common with Netconf)?
> 
> David Harrington
> [email-masked]
> +1-603-828-1401
> 
> > > > I have just uploaded a new draft specifying an extension to
> > > > NETCONF protocol which allows the message encoding to be switched
> > > > to Efficient XML Interchange (EXI).
> > >
> > May I suggest to compare it with enabling compression in the
> > underlying
> ssh

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 11 Jul 2013 13:31:17 +0200

> particular type of schema, such as W3C-standard XML Schema (XSD)?

It seems the mapping to RELAX NG (RFC 6020) could be used for this purpose.

Lada

--
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: E74E8C0C

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 11 Jul 2013 15:49:03 +0200

> It seems the mapping to RELAX NG (RFC 6020) could be used for this purpose.

Oh, the RFC number was supposed to be 6110 (what a shame:-).

Lada

---

From: Andy Bierman <[email-masked]>
Date: Fri, 12 Jul 2013 13:13:21 -0700

Hi,

some comments inline...

> the labels were meaningful. (Of course, with encryption, the content was

I don't think operators asked for monitoring data to be returned
in readable format. That was a config-related request.

I have not studied this draft or EXI in detail yet.
One question: what are the use-cases for enabling
and disabling an "EXI mode"? I prefer the same encoding
for the whole session.

FYI:

Efficient XML Interchange
http://www.w3.org/TR/2011/REC-exi-20110310/

IMO, plain and binary JSON looks easier to implement:

Binary Encodings for JSON
http://www.ietf.org/id/draft-hallambaker-jsonbcd-00.txt

> an operator can access something human-readable.

An operator might have to enable plain encoding,
in addition to providing keys to the sniffer so it can
decode the SSH packets. (I can't see XML in wireshark
for NETCONF sessions).

> resource constraints (as is common with Netconf)?

IMO, the encoding is a second order problem.
Real efficiency is derived from the appropriate set
of protocol operations. The standards are behind
industry in this area.

(If you are scrubbing the floor with a toothbrush, you might
be inclined to invest in better bristle technology ;-)

> [email-masked]

Andy

---

From: Robert Varga <[email-masked]>
Date: Mon, 15 Jul 2013 11:35:49 +0200

> Did you implement this? If so, what is the gain in

We do have a limited prototype.

> a) consumed bandwidth,

Message sizes are reduced by 28-92% in our testing, depending on payload 
and whether schema-informed grammar was used.

> b) reduced code size,

We cannot really measure this, as both XML and EXI codec are external 
libraries.

> c) runtime overhead?

We have not measured this one yet.

Thanks,
Robert

---

From: Robert Varga <[email-masked]>
Date: Mon, 15 Jul 2013 12:10:58 +0200

Hi,

> and disabling an "EXI mode"? I prefer the same encoding

There is not definite use case driving it. I was looking for the 
simplest way to extend NETCONF. Unfortunately, the number of useful EXI 
encoding parameters and deployment concerns made pure hello-based 
negotiation quite verbose and ugly. After defining the <start-exi> 
operation, it seemed a good idea to keep the symmetry and provide a way 
of going back to plain XML.

> http://www.ietf.org/id/draft-hallambaker-jsonbcd-00.txt

It may. On the other hand, you need to translate XML infoset into JSON 
first, which is not necessary with EXI.

> [snip]

Agreed. It may still prove useful to grow the toothbrush 30% ;-)

Thanks,
Robert

---

From: Robert Varga <[email-masked]>
Date: Mon, 15 Jul 2013 14:40:59 +0200

> connection, with `ssh -C` or equivalent?

Noted, I'll try to get the numbers, though it's gonna take some time.

Thanks,
Robert

---

From: Robert Varga <[email-masked]>
Date: Mon, 15 Jul 2013 15:28:18 +0200

Hi,

> Do we have any information about how many commercial/open-source toolkits

SSH compression is described in RFC4253 (2006), zlib compression in 
RFC1950(1996). I am not sure about commercial toolkits, but openssh 
supported it as far back as I remember (~2002).

As for EXI, existing implementations are listed at 
http://www.w3.org/XML/EXI/. There are three open-source (GPLv2, BSD-3 
for Java, Apache 2.0 for C/C++) and one commercial product.

> resource constraints (as is common with Netconf)?

Fortunately, EXI can work in absence of schemas, Current document 
defines two modes of schema-informed operation, one of which relies only 
on EXI-specified XML schema.

The questions you raise are completely valid and need a full state 
synchronization mechanism to solve. XEP-0322 
(http://xmpp.org/extensions/xep-0322.html) tries to do that for XMPP, 
but honestly I could not figure out a clean&easy way of doing the 
equivalent within NETCONF. There is a informational I-D dealing with 
schema synchronization, 
https://tools.ietf.org/html/draft-doi-exi-messaging-requirements-01, but 
I am afraid it is not applicable to NETCONF at this stage.

My preference would be to keep things simple for now and deal with 
schema synchronization in a separate document,

Thanks,
Robert

---

From: Robert Varga <[email-masked]>
Date: Mon, 15 Jul 2013 21:27:32 +0200

> particular type of schema, such as W3C-standard XML Schema (XSD)?

EXI as an encoding format itself is not tied to a particular schema 
language, but the EXI format specification (1.0) only defines how to 
derive EXI grammars from an XML Schema.

> independent of where in the process a YANG schema or RELAX-NG schema would

I am not sure I understand the question. EXI does not require the entire 
data set to be covered by a schema -- it will happily generate dynamic 
grammar to cover for parts which lie outside of the schema being used. 
What this means is that we can use these options, listed in order of 
increasing size efficiency:
- schema-less EXI
- schema-less EXI with built-in XML schema types
- schema-informed EXI with static protocol schema (e.g. just use 
netconf.xsd from RFC6241)
- schema-informed EXI with dynamic data-dependent schemas

The current document covers the first three options.

Bye,
Robert

---

From: Benoit Claise <[email-masked]>
Date: Thu, 25 Jul 2013 13:02:38 +0200

Hi,
> look at packet-dumps and see the information being passed.
I just re-read RFC 3535, section 3, hoping to see that requirement in 
the list, but could not find it.

Regards, Benoit

---

From: "Bert Wijnen \(IETF\)" <[email-masked]>
Date: Thu, 25 Jul 2013 13:46:48 +0200

Inline
----- Original Message ----- 
From: "Benoit Claise" <[email-masked]>
To: "ietfdbh" <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, July 25, 2013 1:02 PM
Subject: Re: [Netconf] Fwd: New Version Notification 
fordraft-varga-netconf-exi-capability-00.txt

> I just re-read RFC 3535, section 3, hoping to see that requirement in the 
> list, but could not find it.
>

Not sure if it is explicitly documented. But I also recall that Operators wanted 
asci
on the line. Now this extension would still allow non-binary transmission. So 
may
be that is acceptable?

Can any operators chime in?

Bert

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 25 Jul 2013 16:31:39 +0200

On Thu, Jul 25, 2013 at 01:02:38PM +0200, Benoit Claise wrote:
> I just re-read RFC 3535, section 3, hoping to see that requirement
> in the list, but could not find it.

I think operators actually preferred protocols that are _not_ readable
on the wire. And this is why we use SSH as the default transport -
without keys, the traffic is not readable on the wire by design. ;-)

I think the most important requirement was captured in #10: The
configuration itself should be in a format that allows to use
off-the-shelf tools to process them. (Not exactly the wording in 
RFC 3535.)

/js

---

From: Andy Bierman <[email-masked]>
Date: Thu, 25 Jul 2013 07:48:14 -0700

Hi,

What matters are the tools available to display the
protocol messages. For REST, it is very nice
that any browser will display JSON or XML.

If a common tool like a browser or an xterm is all you
need to debug protocol messages, that is better than
if special tools need to be installed first.

SSH is not wireshark-friendly. It does a much better job
with HTTP.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 25 Jul 2013 17:04:58 +0200

> SSH is not wireshark-friendly. It does a much better job

I do not care how wireshark-friendly is defined. I do agree that
availability of a large variety of tools and APIs for different
programming and scripting languages matters. And yes, this is where
REST and JSON wins today (but I am old enough to know that popular
encodings come and go).

/js

---

From: David Harrington <[email-masked]>
Date: Thu, 25 Jul 2013 18:38:04 -0400

Not every operator request got documented in rfc3535.
There was a mailing list for discussing nm directions, and other
discussions.

Part of the reason, as others are pointing out is ascii-tool compatibility
(like diff).

Another reason is that the ASN.1 was not easily read/translated/groked, so
specialty applications were needed to translate from varbinds into
something human readable.
But operators didn't fully trust the applications to accurately reflect
what was being sent across the wire.
Some applications translated certain types incorrectly, but some agents
sent certain types incorrectly, and the operator could not easily tell who
was really at fault.
Some applications used cached values, rather than sending new queries,
which impacted interpretation of the results.
So for various reasons, operators wanted an encoding they could understand
without having to rely on (potentially-faulty) specialty tools.

--
David Harrington
[email-masked]
+1-603-828-1401

On 7/25/13 7:02 AM, "Benoit Claise" <[email-masked]> wrote:

>>operator
>>each

---

From: t.petch <[email-masked]>
Date: Fri, 26 Jul 2013 09:32:41 +0100

<inline>
Tom Petch

----- Original Message -----
From: "Bert Wijnen (IETF)" <[email-masked]>
To: "Benoit Claise" <[email-masked]>; "ietfdbh" <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, July 25, 2013 12:46 PM
> > I just re-read RFC 3535, section 3, hoping to see that requirement
in the

Bert

No way am I an operator, but in RFC3535, I do see
"   o  SNMP requires applications to be useful.  SNMP was, from its
early
      days, designed as a programmatic interface between management
      applications and devices.  As such, using SNMP without management
      applications or smart tools appears to be more complicated."
and
"   12. Textual configuration files should be able to contain
       international characters.  Human-readable strings should utilize
       the least-bad internationalized character set and encoding, which
       this year almost certainly means UTF-8.  Protocol elements should
       be in case insensitive ASCII."

which I think is as close as we get.

Tom Petch

---

From: Balazs Lengyel <[email-masked]>
Date: Tue, 30 Jul 2013 14:34:11 +0200

Hello,
One thing that scared me in this approach was the schema dependent 
encoding, which would mean that anyone handling the messages needs to 
have a big set of schemas (all in their correct version). That could 
become a nightmare, so it is good we will avoid that :-)
Balazs

-- 
Balazs Lengyel                       Ericsson Hungary Ltd.
System Manager
ECN: 831 7320                        Tel: +36-1-437-7320
Mobile: +36-70-330-7909              email: [email-masked]