Subject: Re: [IPFIX] [Technical Errata Reported] RFC7012 (3852)

From: Stewart Bryant <[email-masked]>
Date: Thu, 02 Jan 2014 11:54:00 +0000

Brian

Firstly going back to my original problem, I think that would be
better addressed by

OLD

Definitions of timestamp data types have been clarified.

NEW

Definitions of timestamp data types have been clarified and
the definition of the epochs has been moved to sections
6.1.7 through 6.1.11 of RFC 7011

End

Your new text is also helpful.

Then there is the problem of the IANA registry pointing to
an obsolete RFC, which then takes you to RFC7012 which
does not define the IEs.

Consider IE 150, the reader is directed to RFC7012 by the
registry header, and finds the datatype isdateTimeSeconds
(which they dutifully look up in RFC7012 as indicated
in the registry) they find the definition of dateTimeSeconds
without the epoch and then what... For the definition
of the IE to be complete, the reader needs the epoch
which arguably now belongs in the registry, but is missing.
I think that the root cause is that when we made the
registry the definitive definition of the IEs, we
did not add sufficient information in the case of the
time definitions.

Now we have one other issue which I have been puzzling
over since you sent the email and can't quite get my
head around. You use the term "value" in the ADT
context. With for example i.e. 52 (min TTL) it's
easy to imagine what "value" means however you
represent it. However I am struggling to understand
whether it is meaningful to define the "value" of IE
150:"The absolute timestamp of the first packet of
this Flow" is without some specific epoch. This
issue is addressed in I.E. 1 by specifying the count
relative to a previous instance of count, so I think
whilst that is a value, the timestamps are not.

We may need a call to work though this and another issue
I am about to raise on the IPFIX list.

- Stewart

On 31/12/2013 12:44, Brian Trammell wrote:
> hi Stewart,
>
> Stewart Bryant (stbryant) wrote:
>> Certainly something is needed. I was thinking of using IPFIX as an information gathering method in a radio context, and went to look at the definition of the types and could not find the epoch, or even a reference to the epoch. Now part of the problem (which was my fault) was that I did not notice at the time that the elements were defined by the obsolete RFC5102 and went straight to the new RFC. However having the definitions in an obsolete RFC also seems problematic, since it puts the IEs in a strange state. Since RFC 7012 replaces RFC5102 and includes the definitions you would expect the definition in RFC7012 to replace the definition in RFC5102, but those definitions are, as I explained incomplete.
> This is intentional. See below.
>
>> I need to look at this some more when I get back to work, but I am now also concerned that if the epoch can change, the definition of the existing IEs is now unreliable.
> The definition of the existing IEs is based upon the definition of the
> ADT itself in 7012 as well as the definition of the ADT representation
> within the IPFIX protocol in 7011. It is not the intention that reading
> 7012 tells you how to encode IEs for use with IPFIX; that's what section
> 6.1 of 7011 is for.
>
>> You will notice in the errata that I did suggest that an alternative resolution would be to include a reference to the epoch text.
> The point is that the ADT _explicitly_ does not have a binding to an
> epoch, as epochs are only necessary when using integral representations
> of timestamps. IPFIX's representations for these ADTs are integral, but
> bindings to other representations need not be (see, for example,
> draft-trammell-ipfix-text-adt, which recommends ISO8601-style timestamps
> for textual representations of IE values).
>
> I'd suggest instead somehow expanding the present text in 7012 to
> reiterate that if you're using IPFIX, the representations of the ADTs
> are in 7011:
>
> OLD para 2 sec 3.1 7012:
>
>     The current encodings of these data types for use with the IPFIX
>     protocol are defined in [RFC7011]; encodings allowing the use of the
>     IPFIX Information Elements [IANA-IPFIX] with other protocols may be
>     defined in the future by referencing this document.
>
> NEW para 2 sec 3.1 7012:
>
>     The abstract data type definitions in this section are intended
>     only to define the values which can be taken by Information
>     Elements of each type. The encodings of these data types for
>     use with the IPFIX protocol are defined in Section 6.1 of
>     [RFC7011]; encodings  allowing the use of the IPFIX Information
>     Elements [IANA-IPFIX] with other protocols may be defined in the
>     future by referencing this document.
>
> Best regards,
>
> Brian
>
>
>> Stewart
>>
>>
>>
>> Sent from my iPad
>>
>>> On 31 Dec 2013, at 08:50, "Brian Trammell" <[email-masked]> wrote:
>>>
>>> hi Paul, all,
>>>
>>> +n.
>>>
>>> The separation between ADTs and ADT encoding between 7012 and 7011 was explicit and purposeful. Specifically, we do _not_ want to exclude other representations of the IPFIX Information Model from being based upon other encodings of these ADTs, whether ISO 8601 (which either has no epoch or epoch 0000-00-00 00:00 UTC, depending on how you count), NTP (1904), Julian day based counting, etc, etc, etc…
>>>
>>> If there is confusion on this point, I’d suggest adding more explanatory text on this point to Paragraph 2 of the front matter to section 3.1. But as is, I emphatically recommend rejection of this reported erratum.
>>>
>>> Best regards,
>>>
>>> Brian
>>>
>>>> On 31 Dec 2013, at 03:58, Paul Aitken <[email-masked]> wrote:
>>>>
>>>> +1
>>>>
>>>> If anyone was going to raise an errata on this, it would have been me ;-)
>>>>
>>>> I've pointed this issue out before, probably more than once - and have been encouraged to read RFC 3444.
>>>>
>>>> P.
>>>>
>>>>
>>>>> On 30/12/2013 09:23, Andrew Feren wrote:
>>>>> The encoding for the time data types is specified RFC 7011 Sections
>>>>> 6.1.7 through 6.1.10.
>>>>>
>>>>> -Andrew
>>>>>
>>>>>> On 12/30/2013 10:13 AM, RFC Errata System wrote:
>>>>>> The following errata report has been submitted for RFC7012,
>>>>>> "Information Model for IP Flow Information Export (IPFIX)".
>>>>>>
>>>>>> --------------------------------------
>>>>>> You may review the report below and at:
>>>>>> http://www.rfc-editor.org/errata_search.php?rfc=7012&eid=3852
>>>>>>
>>>>>> --------------------------------------
>>>>>> Type: Technical
>>>>>> Reported by: Stewart Bryant <[email-masked]>
>>>>>>
>>>>>> Section: 3.1.15-17
>>>>>>
>>>>>> Original Text
>>>>>> -------------
>>>>>> 3.1.15. dateTimeSeconds
>>>>>>
>>>>>>    The type "dateTimeSeconds" represents a time value expressed with
>>>>>>    second-level precision.
>>>>>>
>>>>>> 3.1.16. dateTimeMilliseconds
>>>>>>
>>>>>>    The type "dateTimeMilliseconds" represents a time value expressed
>>>>>>    with millisecond-level precision.
>>>>>>
>>>>>> 3.1.17. dateTimeMicroseconds
>>>>>>
>>>>>>    The type "dateTimeMicroseconds" represents a time value expressed
>>>>>>    with microsecond-level precision.
>>>>>>
>>>>>> 3.1.18. dateTimeNanoseconds
>>>>>>
>>>>>>    The type "dateTimeNanoseconds" represents a time value expressed with
>>>>>>    nanosecond-level precision.
>>>>>>
>>>>>>
>>>>>> Corrected Text
>>>>>> --------------
>>>>>> 3.1.15. dateTimeSeconds
>>>>>>
>>>>>>    The type "dateTimeSeconds" represents a time value in units of
>>>>>>    seconds based on coordinated universal time (UTC).  The choice of an
>>>>>>    epoch, for example, 00:00 UTC, January 1, 1970, is left to
>>>>>>    corresponding encoding specifications for this type, for example, the
>>>>>>    IPFIX protocol specification.  Leap seconds are excluded.  Note that
>>>>>>    transformation of values might be required between different
>>>>>>    encodings if different epoch values are used.
>>>>>>
>>>>>> 3.1.16. dateTimeMilliseconds
>>>>>>
>>>>>>    The type "dateTimeMilliseconds" represents a time value in units of
>>>>>>    milliseconds based on coordinated universal time (UTC).  The choice
>>>>>>    of an epoch, for example, 00:00 UTC, January 1, 1970, is left to
>>>>>>    corresponding encoding specifications for this type, for example, the
>>>>>>    IPFIX protocol specification.  Leap seconds are excluded.  Note that
>>>>>>    transformation of values might be required between different
>>>>>>    encodings if different epoch values are used.
>>>>>>
>>>>>> 3.1.17. dateTimeMicroseconds
>>>>>>
>>>>>>    The type "dateTimeMicroseconds" represents a time value in units of
>>>>>>    microseconds based on coordinated universal time (UTC).  The choice
>>>>>>    of an epoch, for example, 00:00 UTC, January 1, 1970, is left to
>>>>>>    corresponding encoding specifications for this type, for example, the
>>>>>>    IPFIX protocol specification.  Leap seconds are excluded.  Note that
>>>>>>    transformation of values might be required between different
>>>>>>    encodings if different epoch values are used.
>>>>>>
>>>>>> 3.1.18. dateTimeNanoseconds
>>>>>>
>>>>>>    The type "dateTimeNanoseconds" represents a time value in units of
>>>>>>    nanoseconds based on coordinated universal time (UTC).  The choice of
>>>>>>    an epoch, for example, 00:00 UTC, January 1, 1970, is left to
>>>>>>    corresponding encoding specifications for this type, for example, the
>>>>>>    IPFIX protocol specification.  Leap seconds are excluded.  Note that
>>>>>>    transformation of values might be required between different
>>>>>>    encodings if different epoch values are used.
>>>>>>
>>>>>>
>>>>>> Notes
>>>>>> -----
>>>>>> Although section 1.1 says : - "Definitions of timestamp data types have been clarified." The edited text has removed the epoch definition, and this does not seem to have been incorporated elsewhere in the RFC.
>>>>>>
>>>>>> Without a specified epoch, there is no unique definition of the timestamps.
>>>>>>
>>>>>> My proposal above is to revert to the RFC5102 definitions. RFC7102 is intended to be backwards compatible with RFC5102 and thus the definitions need to be technically identical. Alternatively, if the text is now included elsewhere in RFC7012 or in another RFC, it would be helpful to the reader to provide a reference to the epoch definition in an editorial update to dateTimeX definitions in RFC7102.
>>>>>>
>>>>>> Instructions:
>>>>>> -------------
>>>>>> This errata is currently posted as "Reported". If necessary, please
>>>>>> use "Reply All" to discuss whether it should be verified or
>>>>>> rejected. When a decision is reached, the verifying party (IESG)
>>>>>> can log in to change the status and edit the report, if necessary.
>>>>>>
>>>>>> --------------------------------------
>>>>>> RFC7012 (draft-ietf-ipfix-information-model-rfc5102bis-10)
>>>>>> --------------------------------------
>>>>>> Title               : Information Model for IP Flow Information Export (IPFIX)
>>>>>> Publication Date    : September 2013
>>>>>> Author(s)           : B. Claise, Ed., B. Trammell, Ed.
>>>>>> Category            : PROPOSED STANDARD
>>>>>> Source              : IP Flow Information Export
>>>>>> Area                : Operations and Management
>>>>>> Stream              : IETF
>>>>>> Verifying Party     : IESG
>>>>>> _______________________________________________
>>>>>> IPFIX mailing list
>>>>>> [email-masked]
>>>>>> https://www.ietf.org/mailman/listinfo/ipfix
>>>>> _______________________________________________
>>>>> IPFIX mailing list
>>>>> [email-masked]
>>>>> https://www.ietf.org/mailman/listinfo/ipfix
>>> _______________________________________________
>>> IPFIX mailing list
>>> [email-masked]
>>> https://www.ietf.org/mailman/listinfo/ipfix

-- 
For corporate legal information go to:

http://www.cisco.com/web/about/doing_business/legal/cri/index.html

---

From: Brian Trammell <[email-masked]>
Date: Fri, 3 Jan 2014 11:00:21 +0100

hi Stewart,

inline as per tradition…

> 6.1.7 through 6.1.11 of RFC 7011
Okay, good point. this works for me.
> registry header, and finds the datatype is dateTimeSeconds
Okay, I understand your issue better here. We should consider a new erratum changing the introduction as well as para 2 section 3.1.
> 150: "The absolute timestamp of the first packet of 
But each timestamp value has an underlying real value, a moment in time. If you have a sequence of octets, you need both the ADT (7012) as well as the encoding (7011) to interpret it. 

Take, for example, “09:45:00 UTC on Friday 3 January 2014 CE”. That string, interpreted by a human who makes a couple of basic assumptions (implicit selection of Gregorian over Julian calendar given the date, selection of a post-Julian solar calendar given the name of the month, presence in a negligibly accelerated reference frame with respect to Earth, a commonly agreed method of handling/ignoring leap seconds) unambiguously represents a single second in time. Which is what dateTimeSeconds is meant to do according to the 7012 definition. That second in time is the value.

The positive integer 1388742300 encoded as a big-endian unsigned32 is the IPFIX encoding of this value. This, as you say, requires an epoch, which is given in 7011. Without the epoch, you cannot interpret it at all.

The same value encoded as in draft-trammell-ipfix-text-adt would be “2014-01-03 09:45:00”: less efficient that the IPFIX encoding, but more readable. There’s an implicit epoch here as well (“2014” in this context taken to be years since 1 BCE), but it’s a different epoch from that given in 7011; this is defined by reference, eventually, to ISO 8601.

So each instance of an IE of an ADT can take a value, and that value exists separate from its encoding. 7011 and 7012 together combine to make an unambiguous mapping between the two possible in IPFIX.

Cheers,

Brian

---

From: Stewart Bryant <[email-masked]>
Date: Fri, 03 Jan 2014 12:11:37 +0000

Hi Brian

We are getting there.

On 03/01/2014 10:00, Brian Trammell wrote:
> hi Stewart,
>
> inline as per tradition…
>
> On 02 Jan 2014, at 12:54, Stewart Bryant <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>> Brian
>>
> Okay, I understand your issue better here. We should consider a new 
> erratum changing the introduction as well as para 2 section 3.1.
> But each timestamp value has an underlying real value, a moment in 
> time. If you have a sequence of octets, you need both the ADT (7012) 
> as well as the encoding (7011) to interpret it.
>
> Take, for example, “09:45:00 UTC on Friday 3 January 2014 CE”. That 
> string, interpreted by a human who makes a couple of basic assumptions 
> (implicit selection of Gregorian over Julian calendar given the date, 
> selection of a post-Julian solar calendar given the name of the month, 
> presence in a negligibly accelerated reference frame with respect to 
> Earth, a commonly agreed method of handling/ignoring leap seconds) 
> unambiguously represents a single second in time. Which is what 
> dateTimeSeconds is meant to do according to the 7012 definition. That 
> second in time is the value.
>
> The positive integer 1388742300 encoded as a big-endian unsigned32 is 
> the IPFIX encoding of this value. This, as you say, requires an epoch, 
> which is given in 7011. Without the epoch, you cannot interpret it at all.
>
> The same value encoded as in draft-trammell-ipfix-text-adt would be 
> “2014-01-03 09:45:00”: less efficient that the IPFIX encoding, but 
> more readable. There’s an implicit epoch here as well (“2014” in this 
> context taken to be years since 1 BCE), but it’s a different epoch 
> from that given in 7011; this is defined by reference, eventually, to 
> ISO 8601.
>
> So each instance of an IE of an ADT can take a value, and that value 
> exists separate from its encoding. 7011 and 7012 together combine to 
> make an unambiguous mapping between the two possible in IPFIX.

I think the key is the last sentence, and so long as we make that clear 
we are good.

Maybe some text of the form "relative to the defined epoch" would sort 
it out. At least
that would remind the reader that they needs to know rather than assume 
the epoch.

- Stewart

>>>>> On 31 Dec 2013, at 08:50, "Brian Trammell"<[email-masked]>  wrote:
>>>>>
>>>>> hi Paul, all,
>>>>>
>>>>> +n.
>>>>>
>>>>>> On 31 Dec 2013, at 03:58, Paul Aitken<[email-masked]>  wrote:
>>>>>>
>>>>>> +1
>>>>>>
>>>>>>>> Reported by: Stewart Bryant<[email-masked]>
>>>>>>>>
>>>>>>>> Section: 3.1.15-17
>>>>>>>>
>>>>>>>> Original Text
>>>>>>>> -------------
>>>>>>>> 3.1.15. dateTimeSeconds
>>>>>>>>

---

From: Brian Trammell <[email-masked]>
Date: Fri, 3 Jan 2014 16:39:49 +0100

Hi Stewart,

On 03 Jan 2014, at 13:11, Stewart Bryant <[email-masked]> wrote:

> I think the key is the last sentence, and so long as we make that clear we are good.
> 
> Maybe some text of the form "relative to the defined epoch" would sort it out. At least
> that would remind the reader that they needs to know rather than assume the epoch.

This may seem like a tiny little nit to pick, but we thought a _whole lot_ about this when moving the epochs out of 5102bis, and I still think the reasons therefor are valid. I’d ask you to reconsider insisting on timestamps as relative to a fixed epoch. I’d rather state instead something like "it is the responsibility of the encoding to ensure that each representation has an unambiguous mapping to a moment in time (e.g. relative to a defined epoch)."

Otherwise we go down the calendar rathole very, very quickly, and we’ve tried very hard not to do that. Indeed, one can define any time representation as relative to some epoch, more or less fuzzily. However, back to our ISO 8601 example, one can indeed interpret “2014-01-03 10:00:00 UTC” as a fixed number of years, months, days, hours, and so on from the implicit epoch 0001-01-01 00:00:00 UTC. One would be wrong, given variability in Julian-Gregorian calendar transitions, policies for accounting/ignoring leap seconds, and all those other assumptions one has to make when turning a human-readable timestamp (our “ideal” value) to some notion of the moment in time it references.

I submit that these differences are largely irrelevant on the timescales and for the types of comparisons in event timing — whether network management related or not — that the IPFIX information model is likely to be applied to. But if we bake fixed-epoch references into the information model we’re forced to care about them regardless.

Thanks, cheers,

Brian

---

From: Stewart Bryant <[email-masked]>
Date: Fri, 03 Jan 2014 16:21:10 +0000

On 03/01/2014 15:39, Brian Trammell wrote:
> Hi Stewart,
>
> On 03 Jan 2014, at 13:11, Stewart Bryant <[email-masked] 
> This may seem like a tiny little nit to pick, but we thought a _whole 
> lot_ about this when moving the epochs out of 5102bis, and I still 
> think the reasons therefor are valid. I’d ask you to reconsider 
> insisting on timestamps as relative to a fixed epoch. I’d rather state 
> instead something like "it is the responsibility of the encoding to 
> ensure that each representation has an unambiguous mapping to a moment 
> in time (e.g. relative to a defined epoch)."

If you are proposing that text, that would work for me.
>
> Otherwise we go down the calendar rathole very, very quickly, and 
> we’ve tried very hard not to do that. Indeed, one can define any time 
> representation as relative to some epoch, more or less fuzzily. 
> However, back to our ISO 8601 example, one can indeed interpret 
> “2014-01-03 10:00:00 UTC” as a fixed number of years, months, days, 
> hours, and so on from the implicit epoch 0001-01-01 00:00:00 UTC. One 
> would be wrong, given variability in Julian-Gregorian calendar 
> transitions, policies for accounting/ignoring leap seconds, and all 
> those other assumptions one has to make when turning a human-readable 
> timestamp (our “ideal” value) to some notion of the moment in time it 
> references.
>
> I submit that these differences are largely irrelevant on the 
> timescales and for the types of comparisons in event timing — whether 
> network management related or not — that the IPFIX information model 
> is likely to be applied to. But if we bake fixed-epoch references into 
> the information model we’re forced to care about them regardless.
No, I am fine with the IM not baking in the epoch, now I understand the 
approach, and the with the other changes that tell the reader that they 
moved. However, I think you do need to help the reader understand the 
importance of understanding the of the epoch (and now you remind me the 
discontinuities). The above sentence will do this.

Stewart

---

From: Benoit Claise <[email-masked]>
Date: Mon, 13 Jan 2014 16:08:06 +0100

Brian, Stewart,

Can you please let me know what's the latest conclusion on this errata, 
and update the errata accordingly.

Regards, Benoit
>>>> So each instance of an IE of an ADT can take a value, and that 
>>>> value exists separate from its encoding. 7011 and 7012 together 
>>>> combine to make an unambiguous mapping between the two possible in 
>>>> IPFIX.
>>>
>> think the reasons therefor are valid. I'd ask you to reconsider 
>> insisting on timestamps as relative to a fixed epoch. I'd rather 
>> state instead something like "it is the responsibility of the 
>> encoding to ensure that each representation has an unambiguous 
>> mapping to a moment in time (e.g. relative to a defined epoch)."
>
>> we've tried very hard not to do that. Indeed, one can define any time 
>> "2014-01-03 10:00:00 UTC" as a fixed number of years, months, days, 
>> timestamp (our "ideal" value) to some notion of the moment in time it 
>> references.
>>
>> timescales and for the types of comparisons in event timing --- 
>> whether network management related or not --- that the IPFIX 
>> information model is likely to be applied to. But if we bake 
>> fixed-epoch references into the information model we're forced to 
>> care about them regardless.
> No, I am fine with the IM not baking in the epoch, now I understand 
> the approach, and the with the other changes that tell the reader that 
> they moved. However, I think you do need to help the reader understand 
> the importance of understanding the of the epoch (and now you remind 
> me the discontinuities). The above sentence will do this.
>
> Stewart
>
>>
>> Thanks, cheers,
>>
>> Brian
>>
>>>>>>>> The separation between ADTs and ADT encoding between 7012 and 7011 was explicit and purposeful. Specifically, we do _not_ want to exclude other representations of the IPFIX Information Model from being based upon other encodings of these ADTs, whether ISO 8601 (which either has no epoch or epoch 0000-00-00 00:00 UTC, depending on how you count), NTP (1904), Julian day based counting, etc, etc, etc...
>>>>>>>>
>>>>>>>> If there is confusion on this point, I'd suggest adding more explanatory text on this point to Paragraph 2 of the front matter to section 3.1. But as is, I emphatically recommend rejection of this reported erratum.
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>>
>>>>>>>> Brian
>>>>>>>>