Subject: Re: [ipfix] [issue] INFO-29 Namespace definitions may need IANA considerations

From: "Jeff Meyer" <[email-masked]>
Date: Fri, 30 Jan 2004 13:06:56 -0800

Juergen,

  Thanks for continuing the dialog, hopefully it will help flesh out the 
issues (or misperceptions).

  More comments inline...

-- Jeff

>From: Juergen Quittek <[email-masked]>
>To: Jeff Meyer <[email-masked]>, [email-masked]
>Subject: Re: [ipfix] [issue] INFO-29 Namespace definitions may need IANA 
>considerations
>Date: Fri, 30 Jan 2004 13:59:10 +0100
>
>Jeff,
>
>--On 29.01.2004 10:20 Uhr -0800 Jeff Meyer wrote:
>
>>Juergen,
>>
>>   XML-Schema can be thought of as a pure Informaiton Model.
>
>Of course we can say:
>  "Here is a combined info and data model, please ignore the data model 
>part,
>   because this standard is just about the info model."
>
>But this appears to be - at least - curious.

Well, the authors of XML-Schema had XML documents in mind, so XML-Schema 
does address both in a single document.  That is a bit unfortunate as it 
does cause this confusion.

Carrying the ASN.1 analogy, the ASN.1 authors separated the modeling and 
encoding.  Initially they only had one encoding document, BER, but over time 
have created others.

IPDR and IPFIX-protocol are defining additional encodings from the same 
XML-Schema model.

>
>>XML-Schema and XML also define productions which can be used to create
>>instance documents which are a data model and encoded in ASCII.
>
>This is what people call a data model.

Yup, (or encoding model).  So we're in agreement here.

>
>>IPDR takes full advantage of this by providing both
>>ASCII and Binary encodings from the same information model.
>>I.e. XML-Schema CAN be used AS IS as a pure information model.
>
>Yes and we can have exactly the same with what I suggested.
>The current version of the IPFIX INFO model does not contain
>a binary encoding, but it does contain an ASCII encoding.

No.  IPFIX-info uses XML-Schema as an information model.  XML-Schema authors 
defined a specific encoding which is ASCII based, and as stated above, they 
didn't separate it from the encoding (data model).  IPFIX-protocol defines a 
binary model.  IPDR defines another binary data model which is equivalent to 
the ASCII model from XML-Schema (assuming the subset of XML-Schema defined 
in appendix A.4 of IPDR is followed).

>
>>   Think of ASN.1 which has a Syntax and various encoding rules such
>>as BER, DER and others (even an XER).  At some point you need to
>>formally define something, but that DOES NOT mean that the ASN.1
>>syntax is just a BINARY encoding model nor is it just
>>one encoding model.  Hence it IS separate from encoding.
>>
>>   I view XML-Schema as the ASN.1 of the 21st century, only with a
>>much richer set of off the shelf and often free tools and other
>>interoperable components.
>
>I am sure you do not claim to inherit all properties of ASN.1 ;-)

Nope, just the useful stuff ;-)

>
>>   Renaming things which are well defined (e.g. int) to duplicate
>>names (e.g. ipfix-int) is just a lot of unnecessary rework.
>>The temptation to simply redefine things is often large for
>>engineers.  However in the long run reusing existing work
>>increases the likelihood that people will converge on certain
>>key patterns.
>
>It is not just renaming.  It is replacing the concrete XML data type
>with an abstract one that can be mapped to XML as well as to many other
>domains.

The data types in XML-Schema can be thought of as abstract.  Let's look at a 
concrete example from XML-Schema Part-2 Datatypes 
(http://www.w3.org/TR/xmlschema-2/#long)

3.3.16 long
[Definition:]   long is ·derived· from integer by setting the value of 
·maxInclusive· to be 9223372036854775807 and ·minInclusive· to be 
-9223372036854775808. The ·base type· of long is integer.

3.3.16.1 Lexical representation
long has a lexical representation consisting of an optional sign followed by 
a finite-length sequence of decimal digits (#x30-#x39). If the sign is 
omitted, "+" is assumed. For example: -1, 0, 12678967543233, +100000.

3.3.16.2 Canonical representation
The canonical representation for long is defined by prohibiting certain 
options from the Lexical representation (§3.3.16.1). Specifically, the the 
optional "+" sign is prohibited and leading zeroes are prohibited.

The "Definition" I would consider abstract.  The subsections "Lexical rep" 
and "Cannonical rep" mix in the concerns of ASCII encoding, which we can 
ignore for IPFIX purposes.

>And if you look at other INFO models, for example defined
>in CIM by the DMTF, always abstract data types are used in info models.

Yeah, and these various specs all calls the same basic concept something 
different.  Big help that is!  I would refer to this as the "Tower of Babel" 
approach.

>Concrete ones are just used in data models.
>
>But even if you just consider it as renaming, it solves some problems.
>  [And having worked as software engineer, I know that renaming (e.g. by
>   endless sets of #DEFINEs in C and C++ is not really unusual.]

Are you saying this is a good thing?   I know one reason this is used is 
because of the ambiguity in the C language around what an "int" is.  Is it a 
32-bit integer or whatever is basic to the architecture (e.g. 16-bit or 
64-bit).  This ambiguity started the ball rolling as developers wanted to 
ensure that there was a single name for data types of a specific range of 
values.

Our model doesn't introduce that ambiguity, hence (at least part of) the 
reason for using such #defines (i.e. making up new names) is not necessary.

>The problems we can solve are
>
>  1. We can replace 'hexBinary' by 'octetArray' or something else.
>     Why should a binary data type use 'hex' in its name, if not ONLY
>     with respect to its ASCII data encoding?  And the encoding that really
>     is of importance for IPFIX standardization is the binary encoding
>     by the protocol.

Well the name reflects the initial focus of XML-Schema authors on ASCII 
encoding.  I would prefer "octetArray", but I also have an aversion to 
inventing new names for semantically equivalent things.

This is probably the worst example of mixing in concerns.

The other item I see this coming up in is dateTime.  The definition is 
ambiguous in terms of precision because they allow any valid ISO style time 
string, i.e. arbitrary precision.  They can get away with this because it is 
ASCII encoded.

For the basic types int, long, unsigned short...  I see no issue.

For these two I think clarifying their application in the IPFIX information 
model is sufficient.

>
>  2. We do not have to refer to an IPDR document as a normative reference.
>     This has three implications:
>
>     First, I do not know if this is (so far) possible at all for IETF
>     standard track documents.

First time I heard this.

>
>     Second, I am not sure if the IPDR definition of IPv4 addresses and 
>IPv6
>     addresses is what the IETF would like to use.  It does not make sense
>     to import IPv4 address XML encoding from the IPDR definitions in one
>     IETF standard and to use another source in another IETF standard.

Well, this was not the indication I got from Andrew Norton who has been 
involved in the application of XML w/in IETF (see bottom of this thread).  
I.e. he seemed in favor of reducing the "Tower of Babel".

>
>     Third, if we use our own ABSTRACT data types, users can more easily
>     choose other XML encodings than the one promoted by IPDR.
>     [Nothing against IPDR. I like it very much and my company is a member]
>     Management systems and database builders/integrators may/will already
>     have their own (XML) data types for flow recording. If we use an
>     abstract data type we do not explicitly request them to use the
>     IPDR definitions.

I don't quite follow this.  There are 20 or so types defined, 15 or so names 
come directly from XML-Schema base types.  The remainder address types which 
are important from an IPFIX perspective.

For serialization, users are not required to use IPDR XML or Binary, 
especially if they enjoy spending money on needless reinvention.

For instance since the Schema defines basic fields, the user is free to 
create an alternate enclosing complexType and put any other informaiton they 
like into it.  Now if they want to rename the fields themselves, that is 
more work (e.g. calling "flowCreationTime" "startTime" instead).  However, 
any form of information model wouldn't really help in that low level of 
renaming.

>
>
>    Juergen
>
>>-- Jeff
>>
>>
>>>From: Juergen Quittek <[email-masked]>
>>>To: Jeff Meyer <[email-masked]>, [email-masked]
>>>Subject: Re: [ipfix] [issue] INFO-29  Namespace definitions may need IANA
>>>considerations
>>>Date: Thu, 29 Jan 2004 13:31:19 +0100
>>>
>>>Jeff,
>>>
>>>I suggest to follow a completely different way of solving this
>>>problems such that everybody get what is needed.
>>>
>>>I see the cause in our dispute that the Appendix of the INFO model
>>>document is NOT just an information model, but a data model.
>>>
>>>As you argued well some time ago, it is a bad idea specifying
>>>binary encodings in the info model document, because this would
>>>already be a data model, which may vary among different protocols
>>>that potentially use the IPFIX INFO model.
>>>
>>>So, it is completely correct, that the binary encoding has become
>>>a part of the IPFIX PROTOCOL document.  But still the IPFIX INFO
>>>model document contains a data model: the XML ASCII encoding for
>>>each information element. This is a data model. IMHO this is wrong!
>>>
>>>We should follow a clean approach by just specifying an
>>>information model.  This would solve several of the open issues
>>>we are discussing, including the name space problem (iprd:),
>>>because we get completely independent of these data encoding issues
>>>and still can support IPDR data types in XML-based applications.
>>>
>>>Here is my suggestion:
>>>
>>>Currently, the Appendix is an XML schema defining which fixes ASCII
>>>encodings for each information element.  I suggest to replace this
>>>XML schema by an XML document.  The document would consist of a set
>>>of XML element, each of them specifying a single information elements
>>>with a set of attributes and/or contained XML elements exactly
>>>covering all properties of information elements as defined in section 3.
>>>(We can use a small, simple XML schema for checking consistency of this
>>>XML document.)
>>>
>>>Information elements defined this way would not have a standard XML,
>>>IPDR, or other data type (data model should be separated from info 
>>>model),
>>>but would be defined just by English text, for example
>>>  ipfix-int: integer value in the range of -2147483648 to 2147483647
>>>  ipfix-dateTime: number of seconds since Jan 1st, 1970, 00:00 h
>>>  ipfix-ipv4address: integral representation of a 32 bit IPv4 address
>>>  ...
>>>We would create our own IPFIX type space for the information elements.
>>>
>>>Based on this pure information model, we can use XSLT (as we do it now)
>>>to generate all the text in Section 6 describing the information 
>>>elements.
>>>
>>>Now, if you want to realize an XML-based application, you can take the
>>>new XML document and use XSLT for automatic generation of a schema, such
>>>as the one we have in the Appendix of the current version of the INFO 
>>>model
>>>document.  You see, you still have all the advantages of the machine-
>>>readability and all opportunities to generate IPFIX XML parsers
>>>automatically.
>>>But with the suggested change you would also have a clean information 
>>>model
>>>in the IPFIX INFO document avoiding all discussions on whether or not to
>>>use the IPDR data types.
>>>
>>>I am currently working together with Thomas, editor of the PSAMP INFO
>>>model, to implement the idea sketched above. We will send an example to
>>>this list in a few days.
>>>
>>>Regards,
>>>
>>>    Juergen
>>>
>>>
>>>--On 27.01.2004 23:55 h -0800 Jeff Meyer wrote:
>>>
>>>>The current info-model-02 specification contains references to XML type
>>>>names and semantics borrowed directly from the XML-Schema specification 
>>>>as
>>>>well as extensions.
>>>>
>>>>Currently these extensions borrow from existing work of IPDR as well as
>>>>newly identified types currently specific to IPFIX.
>>>>
>>>>There are a few ways to address this.
>>>>
>>>>The following e-mail thread discusses the options and opinion of this
>>>>editor as well as opinions from Andrew Norton who is involved in IETF's
>>>>and IANA's work in utilizing XML in RFC's.
>>>>
>>>>
>>>>Jeff,
>>>>
>>>>It would seem that this is all a matter of "taste" as any of these 
>>>>options
>>>>technically work.  Personally I would go with option 3; though it 
>>>>requires
>>>>more coordination between organizations, to an outsider it would seem 
>>>>more
>>>>cohesive.
>>>>
>>>>There is a practical difference between option 1 and option 3 in terms 
>>>>of
>>>>setting the standard.  With option 3, if IPDR upgrades their standard, 
>>>>the
>>>>IPFIX group needs to do nothing to incorporate the changes.  However, 
>>>>with
>>>>option 1, IPFIX will have to
>>>>standardize a new schema and issue new RFC's.
>>>>
>>>>-andy
>>>>
>>>>Jeff Meyer wrote:
>>>>     Hi Andy,
>>>>
>>>>I wanted to get your opinion on extending the XML-Schema type system for
>>>>some specific information modeling requirements encountered by the IPFIX
>>>>working group.
>>>>
>>>>IPFIX currently reuses about 15 types from XML-Schema directly, however
>>>>there are several types which are worthwile to distinguish from an IPFIX
>>>>persepective which are not defined by XML-Schema.
>>>>
>>>>These include types to identify timestamps of specific resolution (e.g.
>>>>mSec, uSec, nSec), addressing types (IPv4 and IPv6 addresses) and one 
>>>>for
>>>>UUIDs.
>>>>
>>>>Today most of these extension types are defined in the public NDM-U 
>>>>3.1.1
>>>>specification from the industry consortium called IPDR.org.  There are a
>>>>couple which are not are related to the high resolution timestamps (uSec
>>>>and nSec).
>>>>
>>>>The NDM-U 3.1.1 specificaiton is avaiable at
>>>>http://www.ipdr.org/documents/NDM-U_3.1.1.pdf and specifically section
>>>>A.4.4.
>>>>
>>>>The IPFIX-Info reference is available at:
>>>>
>>>>http://www.ipdr.org/documents/ipfix/infomodel/draft-ietf-ipfix-info-02.html#anchor4
>>>>
>>>>
>>>>This is a link to the specific secion in the HTML format of the info 
>>>>model
>>>>based on RFC2629.
>>>>
>>>>
>>>>The quesion at hand is whether to follow one of the following 3 options:
>>>>
>>>>    1. have a single XML-Schema which defines the extension types 
>>>>relevant
>>>>to IPFIX and contained in the IPFIX namespace
>>>>    2. reference the existing types targeted by IPDR and add only the 
>>>>new
>>>>types specific to the IPFIX Schema
>>>>    3. petition IPDR to define the remaining two IPFIX types for
>>>>timestamps in their upcoming 3.5 specification (likely timeframe Feb. or
>>>>March)
>>>>
>>>>I get the impression from IPFIX discussions that referencing IPDR is
>>>>something that some vocal participants find objectionable, i.e. they 
>>>>would
>>>>prefer to see option 1.
>>>>
>>>>My personal preference would be to go with option 3 or 2.  I believe 3 
>>>>is
>>>>doable and would reduce the number of places where people have to look 
>>>>for
>>>>things.  Option 1 I see as creating some confusion in the space, but
>>>>ultimately addressable through XSL
>>>>or other mapping functions, if that's what it takes to get consensus.
>>>>
>>>>
>>>>Any guidance on your part would be appreciated.
>>>>
>>>>Regards,
>>>>
>>>>Jeff Meyer
>>>>
>>>>_________________________________________________________________
>>>>Get a FREE online virus check for your PC here, from McAfee.
>>>>http://clinic.mcafee.com/clinic/ibuy/campaign.asp?cid=3963
>>>>
>>>>
>>>>--
>>>>Help        mailto:[email-masked] and say "help" in message
>>>>body
>>>>Unsubscribe mailto:[email-masked] and say
>>>>"unsubscribe ipfix" in message body
>>>>Archive     http://ipfix.doit.wisc.edu/archive/
>>>
>>>
>>>
>>>
>>
>>_________________________________________________________________
>>Find high-speed ‘net deals ? comparison-shop your local providers here. 
>>https://broadband.msn.com
>>
>
>
>
>

_________________________________________________________________
What are the 5 hot job markets for 2004? Click here to find out. 
http://msn.careerbuilder.com/Custom/MSN/CareerAdvice/WPI_WhereWillWeFindJobsIn2004.htm?siteid=CBMSN3006&sc_extcmp=JS_wi08_dec03_hotmail1

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: "Jeff Meyer" <[email-masked]>
Date: Fri, 30 Jan 2004 18:12:08 -0800

Juergen,

  I'll look forward to the XSL tools, basically the "information modeling 
specific" XML document to/from XML-Schema.

  The ability to completely disconnect the type space from XML encoding 
introduces an ambiguity whose value I question.  But it is effectively 
irrelevant/equivalent if one uses a default XSL mapping from your 
specification format to the obvious XML type analogues used in XML-Schema.

  So, overall I see some benefits to this format, not the least of which 
being that some folks might be more willing to move forward, as it 
disconnects the work from that of other rogue non-IETF groups, but still 
leaves a straightforward mapping thanks to the power of XML transformations.

  It also provides a separation of concern which I can appreciate.

  Good work!

-- Jeff

>Date: Sat, 31 Jan 2004 01:14:16 +0100
>
>Jeff,
>
>Please find some comments inline.
>
>--On 30.01.2004 13:36 Uhr -0800 Jeff Meyer wrote:
>
>>Juergen,
>>
>>   Some interesting work.  I like some of the elements.
>>
>>   Comments inline.
>>
>>-- Jeff
>>
>>
>>>Date: Fri, 30 Jan 2004 17:32:52 +0100
>>>
>>>Jeff and all,
>>>
>>>Here is an alternative suggestion for the XML representation of the
>>>info model that I developed together with Thomas, one of the editors of
>>>the PSAMP info model.  This XML representation of the info model uses
>>>an XML document (ipfix.xml) instead of an XML schema for specifying
>>>the IPFIX information elements (or fields as the IPFIX protocol document
>>>calls them).
>>
>>Do you have XSL templates to go to/from this format to the more
>>generally useful XML-Schema syntax?
>
>Thomas and I are working on it.
>
>>>It just shows the first 11 elements defined in the current draft,
>>>but this is enough to show how this XML format looks like.
>>>
>>>File ipfix.xml can be used for automatically generating Section 6 of
>>>the INFO model document as well as the XML schema used for the previous
>>>version.
>>
>>Do you have that XSL template available publicly?
>
>We will make it public, when it is complete.
>
>>>The difference is that it uses abstract data types (as a data
>>>model should) instead of concrete ones.
>>
>>How are these any more abstract?  Your ipfix.xsd in turn derives these
>>shiny new names from existing XML Schema Types.  In other words new names,
>>same thing.
>
>The difference is that the encoding - and even the ASCII encoding - is 
>open.
>
>>>
>>>In addition, file ipfix.xsd contains an XML schema, that can be used
>>>for validating the ipfix.xml file. It defines the properties of fields/
>>>information elements that are used in ipfix.xml and it also includes
>>>a definition of the abstract data types used in ipfix.xml.
>>
>>So, one thing this does is provide validation that required annotations
>>are present.  That could be useful.
>>
>>What you have lost is the ability to validate or have an XML format of
>>the information simply fall out of the work.  This could be mitigated
>>if you have the XML-Schema -> ipfix.xml XSL, do you have this?
>
>Sorry, what are you asking for?
>
>>>A nice consequence is that now not just Section 6 (field definitions)
>>>but also Section 3 (Properties of an IPFIX Flow Attribute) and Section 4
>>>(Type Space) can be generated out of XML input.
>>
>>This is interesting and potentially useful, with the exception of
>>creating new names for lots of existing types.
>>
>>One benefit I see is that it would constrain and allow validation that
>>only allowed types are present in this style information model.  Reusing
>>well defined names from XML-Schema Types would obviously be my preference.
>>
>>>
>>>While section 6 is generated from ipfix.xml, Sections 3 and 4 are
>>>generated from ipfix.xsd.
>>
>>Do you have the XSL for this?
>
>Not yet, but without them, this work would not make sense.
>
>>>Also the XML schema, we used for the previous versions of the INFO model
>>>can be generated out of the info model given in ipfix.xml.  You just have
>>>to define a mapping of the used abstract data types to concrete ones,
>>>such as the IPDR definitions of IPv4 and IPv6 addresses.
>>
>>OK, nice.
>>
>>>
>>>Conclusion:
>>>- This version is a pure data model using abstract data types and without
>>>  any data encodings.
>>
>>Umm, you mean "pure information model"?
>
>An information model that comes without a data model.
>
>>I would say "This version is enables the validation of IPFIX information
>>models using off the shelf tools" and enforces restrictions which are not
>>assigned in the technique of using XML-Schema directly as an information
>>model."
>>
>>>- It does not have dependencies on external data type definitions by 
>>>IPDR.
>>
>>Thank god for that!
>>
>>>- We do not loose anything, because the XML schema, we used so far
>>>  (including XML data types) can be generated automatically.
>>
>>If you have any of the XSL's I'd appreciate your making them public.
>
>As I said, without them the entire effort does not make sense.
>We will have them completed soon.
>
>>  Ultimately having a direct representation in XML-Schema is useful
>>for validating instance documents which contain sets of data serialized
>>in this format, for use of IPDR's XML and
>>XDR serialization tools and other things.
>>
>>So, in general, I agree there are some benefits which you've identified.
>>As long as it doesn't take away other existing benefits, i.e. immediate
>>mechanisms for serialization in XML and binary formats with open source
>>tools.  Having bi directional XSL
>>info-model.xml <-> info-model.xsd would be good.
>
>Agreed.
>
>Thanks,
>
>    Juergen
>

_________________________________________________________________
There are now three new levels of MSN Hotmail Extra Storage!  Learn more. 
http://join.msn.com/?pgmarket=en-us&page=hotmail/es2&ST=1

---

From: "Jeff Meyer" <[email-masked]>
Date: Fri, 30 Jan 2004 18:16:21 -0800

Juergen,

  The ability to completely disconnect the type space from the existing XML 
encoding introduces an ambiguity whose value I question.  But it is 
effectively irrelevant/equivalent if one uses a default XSL mapping from 
your specification format to the obvious XML type analogues used in 
XML-Schema.

  Good work!

-- Jeff

_________________________________________________________________
Learn how to choose, serve, and enjoy wine at Wine @ MSN. 
http://wine.msn.com/