Subject: Re: [ipfix] [issue] INFO-29 Namespace definitions may need IANA considerations

From: Sebastian Zander <[email-masked]>
Date: Mon, 02 Feb 2004 16:06:17 +0100

Hi Juergen,

good work! Some comments:

- Do we really need the semantics attribute? Often the semantics are already
   included in the description or even in the field name. I'd prefer to have that in
   the description.
- No signed types yet (do we need them?)
- Do we need both enumeratedRange and Range? I'd prefer to have only one element
   and let this be an enumeration or range. According to the current definition you
   can specify both which might not be a good idea.
- Allow multiple reference elements instead of comma separation in the value?
- Make some attributes of field elements instead to make the first line shorter
   and more readable. I'd suggest to have only name and *Type as attributes.

Cheers,

Sebastian

Juergen Quittek wrote:
> Jeff and all,
> 
> Here is an alternative suggestion for the XML representation of the
> info model that I developed together with Thomas, one of the editors of
> the PSAMP info model.  This XML representation of the info model uses
> an XML document (ipfix.xml) instead of an XML schema for specifying
> the IPFIX information elements (or fields as the IPFIX protocol document
> calls them). It just shows the first 11 elements defined in the current 
> draft,
> but this is enough to show how this XML format looks like.
> 
> File ipfix.xml can be used for automatically generating Section 6 of
> the INFO model document as well as the XML schema used for the previous
> version.  The difference is that it uses abstract data types (as a data
> model should) instead of concrete ones.
> 
> In addition, file ipfix.xsd contains an XML schema, that can be used
> for validating the ipfix.xml file. It defines the properties of fields/
> information elements that are used in ipfix.xml and it also includes
> a definition of the abstract data types used in ipfix.xml.
> 
> A nice consequence is that now not just Section 6 (field definitions)
> but also Section 3 (Properties of an IPFIX Flow Attribute) and Section 4
> (Type Space) can be generated out of XML input.
> 
> While section 6 is generated from ipfix.xml, Sections 3 and 4 are
> generated from ipfix.xsd.
> 
> Also the XML schema, we used for the previous versions of the INFO model
> can be generated out of the info model given in ipfix.xml.  You just have
> to define a mapping of the used abstract data types to concrete ones,
> such as the IPDR definitions of IPv4 and IPv6 addresses.
> 
> Conclusion:
> - This version is a pure data model using abstract data types and without
>  any data encodings.
> - It does not have dependencies on external data type definitions by IPDR.
> - We do not loose anything, because the XML schema, we used so far
>  (including XML data types) can be generated automatically.
> 
> Cheers,
> 
>    Juergen
> 
> --On 30.01.2004 13:59 Uhr +0100 Juergen Quittek wrote:
> 
>> Jeff,
>>
>> --On 29.01.2004 10:20 Uhr -0800 Jeff Meyer wrote:
>>
>>> Juergen,
>>>
>>>   XML-Schema can be thought of as a pure Informaiton Model.
>>
>>
>> Of course we can say:
>>   "Here is a combined info and data model, please ignore the data 
>> model part,
>>    because this standard is just about the info model."
>>
>> But this appears to be - at least - curious.
>>
>>> XML-Schema and XML also define productions which can be used to create
>>> instance documents which are a data model and encoded in ASCII.
>>
>>
>> This is what people call a data model.
>>
>>> IPDR takes full advantage of this by providing both
>>> ASCII and Binary encodings from the same information model.
>>> I.e. XML-Schema CAN be used AS IS as a pure information model.
>>
>>
>> Yes and we can have exactly the same with what I suggested.
>> The current version of the IPFIX INFO model does not contain
>> a binary encoding, but it does contain an ASCII encoding.
>>
>>>   Think of ASN.1 which has a Syntax and various encoding rules such
>>> as BER, DER and others (even an XER).  At some point you need to
>>> formally define something, but that DOES NOT mean that the ASN.1
>>> syntax is just a BINARY encoding model nor is it just
>>> one encoding model.  Hence it IS separate from encoding.
>>>
>>>   I view XML-Schema as the ASN.1 of the 21st century, only with a
>>> much richer set of off the shelf and often free tools and other
>>> interoperable components.
>>
>>
>> I am sure you do not claim to inherit all properties of ASN.1 ;-)
>>
>>>   Renaming things which are well defined (e.g. int) to duplicate
>>> names (e.g. ipfix-int) is just a lot of unnecessary rework.
>>> The temptation to simply redefine things is often large for
>>> engineers.  However in the long run reusing existing work
>>> increases the likelihood that people will converge on certain
>>> key patterns.
>>
>>
>> It is not just renaming.  It is replacing the concrete XML data type
>> with an abstract one that can be mapped to XML as well as to many other
>> domains.  And if you look at other INFO models, for example defined
>> in CIM by the DMTF, always abstract data types are used in info models.
>> Concrete ones are just used in data models.
>>
>> But even if you just consider it as renaming, it solves some problems.
>>   [And having worked as software engineer, I know that renaming (e.g. by
>>    endless sets of #DEFINEs in C and C++ is not really unusual.]
>> The problems we can solve are
>>
>>   1. We can replace 'hexBinary' by 'octetArray' or something else.
>>      Why should a binary data type use 'hex' in its name, if not ONLY
>>      with respect to its ASCII data encoding?  And the encoding that 
>> really
>>      is of importance for IPFIX standardization is the binary encoding
>>      by the protocol.
>>
>>   2. We do not have to refer to an IPDR document as a normative 
>> reference.
>>      This has three implications:
>>
>>      First, I do not know if this is (so far) possible at all for IETF
>>      standard track documents.
>>
>>      Second, I am not sure if the IPDR definition of IPv4 addresses 
>> and IPv6
>>      addresses is what the IETF would like to use.  It does not make 
>> sense
>>      to import IPv4 address XML encoding from the IPDR definitions in one
>>      IETF standard and to use another source in another IETF standard.
>>
>>      Third, if we use our own ABSTRACT data types, users can more easily
>>      choose other XML encodings than the one promoted by IPDR.
>>      [Nothing against IPDR. I like it very much and my company is a 
>> member]
>>      Management systems and database builders/integrators may/will 
>> already
>>      have their own (XML) data types for flow recording. If we use an
>>      abstract data type we do not explicitly request them to use the
>>      IPDR definitions.
>>
>>
>>     Juergen
>>
>>> -- Jeff
>>>
>>>
>>>> From: Juergen Quittek <[email-masked]>
>>>> To: Jeff Meyer <[email-masked]>, [email-masked]
>>>> Subject: Re: [ipfix] [issue] INFO-29  Namespace definitions may need 
>>>> IANA
>>>> considerations
>>>> Date: Thu, 29 Jan 2004 13:31:19 +0100
>>>>
>>>> Jeff,
>>>>
>>>> I suggest to follow a completely different way of solving this
>>>> problems such that everybody get what is needed.
>>>>
>>>> I see the cause in our dispute that the Appendix of the INFO model
>>>> document is NOT just an information model, but a data model.
>>>>
>>>> As you argued well some time ago, it is a bad idea specifying
>>>> binary encodings in the info model document, because this would
>>>> already be a data model, which may vary among different protocols
>>>> that potentially use the IPFIX INFO model.
>>>>
>>>> So, it is completely correct, that the binary encoding has become
>>>> a part of the IPFIX PROTOCOL document.  But still the IPFIX INFO
>>>> model document contains a data model: the XML ASCII encoding for
>>>> each information element. This is a data model. IMHO this is wrong!
>>>>
>>>> We should follow a clean approach by just specifying an
>>>> information model.  This would solve several of the open issues
>>>> we are discussing, including the name space problem (iprd:),
>>>> because we get completely independent of these data encoding issues
>>>> and still can support IPDR data types in XML-based applications.
>>>>
>>>> Here is my suggestion:
>>>>
>>>> Currently, the Appendix is an XML schema defining which fixes ASCII
>>>> encodings for each information element.  I suggest to replace this
>>>> XML schema by an XML document.  The document would consist of a set
>>>> of XML element, each of them specifying a single information elements
>>>> with a set of attributes and/or contained XML elements exactly
>>>> covering all properties of information elements as defined in 
>>>> section 3.
>>>> (We can use a small, simple XML schema for checking consistency of this
>>>> XML document.)
>>>>
>>>> Information elements defined this way would not have a standard XML,
>>>> IPDR, or other data type (data model should be separated from info 
>>>> model),
>>>> but would be defined just by English text, for example
>>>>  ipfix-int: integer value in the range of -2147483648 to 2147483647
>>>>  ipfix-dateTime: number of seconds since Jan 1st, 1970, 00:00 h
>>>>  ipfix-ipv4address: integral representation of a 32 bit IPv4 address
>>>>  ...
>>>> We would create our own IPFIX type space for the information elements.
>>>>
>>>> Based on this pure information model, we can use XSLT (as we do it now)
>>>> to generate all the text in Section 6 describing the information 
>>>> elements.
>>>>
>>>> Now, if you want to realize an XML-based application, you can take the
>>>> new XML document and use XSLT for automatic generation of a schema, 
>>>> such
>>>> as the one we have in the Appendix of the current version of the 
>>>> INFO model
>>>> document.  You see, you still have all the advantages of the machine-
>>>> readability and all opportunities to generate IPFIX XML parsers
>>>> automatically.
>>>> But with the suggested change you would also have a clean 
>>>> information model
>>>> in the IPFIX INFO document avoiding all discussions on whether or 
>>>> not to
>>>> use the IPDR data types.
>>>>
>>>> I am currently working together with Thomas, editor of the PSAMP INFO
>>>> model, to implement the idea sketched above. We will send an example to
>>>> this list in a few days.
>>>>
>>>> Regards,
>>>>
>>>>    Juergen
>>>>
>>>>
>>>> --On 27.01.2004 23:55 h -0800 Jeff Meyer wrote:
>>>>
>>>>> The current info-model-02 specification contains references to XML 
>>>>> type
>>>>> names and semantics borrowed directly from the XML-Schema 
>>>>> specification as
>>>>> well as extensions.
>>>>>
>>>>> Currently these extensions borrow from existing work of IPDR as 
>>>>> well as
>>>>> newly identified types currently specific to IPFIX.
>>>>>
>>>>> There are a few ways to address this.
>>>>>
>>>>> The following e-mail thread discusses the options and opinion of this
>>>>> editor as well as opinions from Andrew Norton who is involved in 
>>>>> IETF's
>>>>> and IANA's work in utilizing XML in RFC's.
>>>>>
>>>>>
>>>>> Jeff,
>>>>>
>>>>> It would seem that this is all a matter of "taste" as any of these 
>>>>> options
>>>>> technically work.  Personally I would go with option 3; though it 
>>>>> requires
>>>>> more coordination between organizations, to an outsider it would 
>>>>> seem more
>>>>> cohesive.
>>>>>
>>>>> There is a practical difference between option 1 and option 3 in 
>>>>> terms of
>>>>> setting the standard.  With option 3, if IPDR upgrades their 
>>>>> standard, the
>>>>> IPFIX group needs to do nothing to incorporate the changes.  
>>>>> However, with
>>>>> option 1, IPFIX will have to
>>>>> standardize a new schema and issue new RFC's.
>>>>>
>>>>> -andy
>>>>>
>>>>> Jeff Meyer wrote:
>>>>>     Hi Andy,
>>>>>
>>>>> I wanted to get your opinion on extending the XML-Schema type 
>>>>> system for
>>>>> some specific information modeling requirements encountered by the 
>>>>> IPFIX
>>>>> working group.
>>>>>
>>>>> IPFIX currently reuses about 15 types from XML-Schema directly, 
>>>>> however
>>>>> there are several types which are worthwile to distinguish from an 
>>>>> IPFIX
>>>>> persepective which are not defined by XML-Schema.
>>>>>
>>>>> These include types to identify timestamps of specific resolution 
>>>>> (e.g.
>>>>> mSec, uSec, nSec), addressing types (IPv4 and IPv6 addresses) and 
>>>>> one for
>>>>> UUIDs.
>>>>>
>>>>> Today most of these extension types are defined in the public NDM-U 
>>>>> 3.1.1
>>>>> specification from the industry consortium called IPDR.org.  There 
>>>>> are a
>>>>> couple which are not are related to the high resolution timestamps 
>>>>> (uSec
>>>>> and nSec).
>>>>>
>>>>> The NDM-U 3.1.1 specificaiton is avaiable at
>>>>> http://www.ipdr.org/documents/NDM-U_3.1.1.pdf and specifically section
>>>>> A.4.4.
>>>>>
>>>>> The IPFIX-Info reference is available at:
>>>>>
>>>>> http://www.ipdr.org/documents/ipfix/infomodel/draft-ietf-ipfix-info-02.html#anchor4 
>>>>>
>>>>>
>>>>>
>>>>> This is a link to the specific secion in the HTML format of the 
>>>>> info model
>>>>> based on RFC2629.
>>>>>
>>>>>
>>>>> The quesion at hand is whether to follow one of the following 3 
>>>>> options:
>>>>>
>>>>>    1. have a single XML-Schema which defines the extension types 
>>>>> relevant
>>>>> to IPFIX and contained in the IPFIX namespace
>>>>>    2. reference the existing types targeted by IPDR and add only 
>>>>> the new
>>>>> types specific to the IPFIX Schema
>>>>>    3. petition IPDR to define the remaining two IPFIX types for
>>>>> timestamps in their upcoming 3.5 specification (likely timeframe 
>>>>> Feb. or
>>>>> March)
>>>>>
>>>>> I get the impression from IPFIX discussions that referencing IPDR is
>>>>> something that some vocal participants find objectionable, i.e. 
>>>>> they would
>>>>> prefer to see option 1.
>>>>>
>>>>> My personal preference would be to go with option 3 or 2.  I 
>>>>> believe 3 is
>>>>> doable and would reduce the number of places where people have to 
>>>>> look for
>>>>> things.  Option 1 I see as creating some confusion in the space, but
>>>>> ultimately addressable through XSL
>>>>> or other mapping functions, if that's what it takes to get consensus.
>>>>>
>>>>>
>>>>> Any guidance on your part would be appreciated.
>>>>>
>>>>> Regards,
>>>>>
>>>>> Jeff Meyer
>>>>>
>>>>> _________________________________________________________________
>>>>> Get a FREE online virus check for your PC here, from McAfee.
>>>>> http://clinic.mcafee.com/clinic/ibuy/campaign.asp?cid=3963
>>>>>
>>>>>
>>>>> -- 
>>>>> Help        mailto:[email-masked] and say "help" in 
>>>>> message
>>>>> body
>>>>> Unsubscribe mailto:[email-masked] and say
>>>>> "unsubscribe ipfix" in message body
>>>>> Archive     http://ipfix.doit.wisc.edu/archive/
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>> _________________________________________________________________
>>> Find high-speed ‘net deals ? comparison-shop your local providers 
>>> here. https://broadband.msn.com
>>>
>>
>>
>>
>>
>>
>> -- 
>> Help        mailto:[email-masked] and say "help" in 
>> message body
>> Unsubscribe mailto:[email-masked] and say
>> "unsubscribe ipfix" in message body
>> Archive     http://ipfix.doit.wisc.edu/archive/
> 
> 
> 
> 

-- 
Sebastian Zander                         E-mail: [email-masked]
Fraunhofer FOKUS / METEOR                Tel: +49-30-3463-7287
Kaiserin-Augusta-Allee 31                Fax: +49-30-3463-8287
D-10589 Berlin, Germany                  www.fokus.fraunhofer.de/usr/sebastian.zander

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: Juergen Quittek <[email-masked]>
Date: Mon, 02 Feb 2004 22:10:04 +0100

Sebastian,

> good work! Some comments:

Thanks.

>    included in the description or even in the field name. I'd prefer to have that in

I copied them from draft -02.  I am not really a fan of the semantics attribute,
but I did not find time to look for a good alternative.  There is some benefit in
having them formally specified, but maybe just an explanation in the description
is fine. However, SMI distinguishes counters from gauges and plain integers.
I suggest to make this separate INFO model issue.

> - No signed types yet (do we need them?)

Maybe another issue.  It tried to find an example that requires negative integers,
but I could not find any.  Can soemone else?

>    can specify both which might not be a good idea.

Well, an enumerated range may assigns semantics to each individual numbers,
while a plain range just defines maximum and minimum value.  Again it would be
good to discuss this using an example.

> - Allow multiple reference elements instead of comma separation in the value?

I'm in favor of plain text in a single reference element.
You can list as many references as you like.

>    and more readable. I'd suggest to have only name and *Type as attributes.

It makes the first line shorter, but the number of lines larger.

Thanks,

    Juergen

---

From: Sebastian Zander <[email-masked]>
Date: Tue, 03 Feb 2004 11:16:38 +0100

> I suggest to make this separate INFO model issue.

Ok.

Another issue: Current bitset not only has a different semantics but also
needs a different presentation. Furthermore an octetArray might be
used as bitset too.

> but I could not find any.  Can soemone else?

The delta time encoding (as currently defined in the protocol draft) needs a
signed type.

> good to discuss this using an example.

I think my comment wasn't too clear. Can we do this:

<field name=bla...
<range>
   1: semantic1,
   2: semantic2
</range>
<field name=foo...
<range>1..50</range>

 From the value is pretty clear what is an enumeration and what a range. Do
we really need two tags?

Actually enumeratedRange is not really used right now. It is used for e.g.
samplingMethod _but_ in the Description!

> It makes the first line shorter, but the number of lines larger.

Exactly. IMO the first line should contain only the essential data allowing
the reader to skip the rest of the field definition. So e.g. I don't
think that sementics is so important to must appear in the first line.

Another proposal: It would be good to have a list (maybe two lists)
generated from the XML document which contain only field ids and field names
and is sorted by 1) field id ( 2) name) for convenience of the reader
and possible input for IANA.

Cheers,

Sebastian

>> D-10589 Berlin, Germany                  
>> www.fokus.fraunhofer.de/usr/sebastian.zander
>>
>>
>>
>>
> 
> 
> 
>