Subject: [ipfix-req] proposed changes for requirements document

From: Juergen Quittek <[email-masked]>
Date: Fri, 03 Jan 2003 17:11:27 +0100

Dear all,

Here is a list of suggestions for changes to the requirements document.
They are based on our discussion in Atlanta and on a comment by Benoit.

I would like to achieve rough consensus on this list of changes
in order to enter working group last call for the requirements document.

    Juergen

1. Replace in Section "2.1. IP Traffic Flow" in item 1., line 3:

   "RTP header fields" by "RTP header fields [RFC1889]"

2. Append to Section "2.1. IP Traffic Flow":

   "Also, please note that although packet properties may depend on
    application headers, there is no requirement defined in this document
    related to applicaton headers."

3. Move in section 6.1 "Information Model" item "7. if protocol
   type is ICMP: ICMP type and code" from the list of MUST
   attributes to the list of SHOULD attributes. Now it is item 17.

4. Replace the first paragraph in section "6.3.2. Reliability":

   "Absence of reliability of the data transfer, for example caused by
    loss or reordering of packets containing flow information, MUST be
    indicated."

   by

   "Different levels of reliability of the data transfer MAY be
    supported. The collecting process MUST have sufficient information
    for making a conservative assumption about the applied reliability
    level. This means that the applied level of reliability MUST be as
    high as or higher than indicated to the collecting process."

5. Add new paragraph after second paragraph in section "6.3.2":

   "The chosen method of data transfer MUST be extensible concerning
    reliability. It MUST be possible to increase reliability by
    extensions."

6. Add to References:

   "[RFC1889]   H. Schulzrinne, S. Casner, R. Frederick, V. Jacobson, "RTP:
                A Transport Protocol for Real-Time Applications", RFC 1889,
                January 1996."

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: Juergen Quittek <[email-masked]>
Date: Wed, 08 Jan 2003 18:54:06 +0100

Dear all,

If there are no comments on this suggestion until Wednesday
next week, I will prepare and submit a new version of the
requirements document at then end of next week. Hopefully,
this will be the version that enters WG last call.

    Juergen

---

From: "Tal Givoly" <[email-masked]>
Date: Tue, 14 Jan 2003 10:32:00 -0800

Juergen,

Some recommendation of modification to the proposed changes:

<snip>

   by

</snip>

<tal>

Regarding item 4, there seem to be a few concerns regarding the proposed
wording:

 - It implies that the collecting process must indicate the level
   of reliability it is interested in, hence, there must be a
   mechanism to indicate this.
 - The words "conservative assumptions" are ambiguous.
 - It implies the level of reliability may be higher than required by
   collecting process - it is unclear to me why this is implied by
   the consensus to allow the protocol to "MAY" support higher
   levels of reliability.

Therefore, may I propose the following alternative as a replacement for the
first paragraph in "6.3.2 Reliability":

   "The data exchange between the exporter and the collector MAY
    support different levels of reliability.

    For the most basic level of reliability, the following should
    be maintained:
     - The collecting process MUST be able to detect the level of
       reliability.
     - The collection process MUST be able to detect any loss of data.

    For higher levels of reliability, the following should be
    maintained:
     - All reliability properties requirements of lower levels of
       reliability MUST be maintained.
     - The exporter process SHOULD be able to communicate with more
       than one collection process.
     - In case of failure the collection process, or in the
       connectivity between the exporter process and one collection
       process, transmission of data SHOULD be redirected, in a
       fail-over fashion, to a secondary collection process.
     - In case of failover, data that has not been acknowledged
       SHOULD be retransmitted to the secondary collection process.
     - The collection process MUST be able to eliminate duplicate
       reporting of the same records emanating from the fail over
       mechanism."

</tal>

<snip>

</snip>

<tal>
Regarding item 5, I agree that a paragraph should be added after the second
paragraph in 6.3.2. The main point on which I differ is that your proposed
wording implies that there must be an "ability to ... by extensions" whereas
I don't believe that is mandatory to meet the essence of the requirements.
If a protocol supports adequate levels of reliability, as several of the
currently evaluated protocols do, there may not be a need for an extension
model. If, on the other hand, the protocol selected does not have a reliable
delivery mechanism, then a well-defined means to extend it must be present.

Therefore, may I suggest a slightly different wording than the version you
proposed:

   "If higher levels of reliability are not supported, there must be
    a well-defined mechanism to extend the protocol to allow for
    increasing the reliability to support higher degrees of
    reliability."

I believe that even though the wording above may seem to hint at an
implementation, in fact, it merely may state some lowest common denominator
features recurring in practically all existing reliable delivery protocols.

Best regards,

Tal
</tal>

---

From: Juergen Quittek <[email-masked]>
Date: Wed, 15 Jan 2003 00:26:21 +0100

Hi Tal,

Thank you for the comments. See my replies inline.

    Juergen

>    mechanism to indicate this.

No. It does not. At least, I cannot see this. Why do you think so?

>  - The words "conservative assumptions" are ambiguous.

I agree. They should be changed.

>    the consensus to allow the protocol to "MAY" support higher

Of course the level may be higher or lower than required by the collecting
process. It just says that the collecting process needs sufficient
information on the reliability in order to find out whether or not the
level is sufficient.

>    "The data exchange between the exporter and the collector MAY

I'd suggest "data transfer from the exporting process to the collecting process".

>     For higher levels of reliability, the following should be

Are these several higher levels or is it a single higher level
including everything below?

>        reliability MUST be maintained.

Above you say "the following should b maintained", within the
following you say "MUST". Is should x MUST = should?

>        than one collection process.

This is already covered as MAY requirement by section 8.3.

>        fail-over fashion, to a secondary collection process.

This is already covered by the last paragraph of section 6.3.2

>        SHOULD be retransmitted to the secondary collection process.

Isn't this a solution rather than a requirement?

>        reporting of the same records emanating from the fail over

This is already covered as MAY requirement by section 8.3.

> currently evaluated protocols do, there may not be a need for an extension

Weren't among the people teaching us that there cannot be sufficient
reliability? What is sufficient depends on the requirements of the
collecting process or the application processing the transfered data.

> delivery mechanism, then a well-defined means to extend it must be present.

Some people said that without application level ACKs there is no real reliability.
Consequently, all you ask for above is not reliable and would need to be
extensible.

The main lesson I learned is that there are several levels of reliability
and that you cannot say something is reliable or not without specifying
what you mean with "reliable". The only consequence is that you have to
ask for every level, that it still is open for further increasing reliability.

>    "If higher levels of reliability are not supported, there must be

The term "higher levels" is only implicitly defined by your text above
and still ambiguous. How many levels need to be supported such that you
need not to be extensible anymore?

>     increasing the reliability to support higher degrees of

Is "degree" equivqalent to "level"?

---

From: Juergen Quittek <[email-masked]>
Date: Thu, 23 Jan 2003 00:06:34 +0100

Dear all,

After receiving input from Tal and Jeff, Benoit and I modified the
proposal for changing the requirements draft. Again I am asking:
Can we agree on this?

    Juergen

4. Replace the entire section "6.3.2. Reliability" by
   "Loss of flow records during the data transfer from the exporting
    process to the collecting process MUST be indicated at the
    collecting process. This indication MUST allow the collecting
    process to gauge the number of flow records lost. Possible reasons
    for flow records loss include but is not limited to:

    1. Metering process limitations: lack of memory, processing power, etc.
       These limitations are already covered in section 5.1.
    2. Exporting process limitations: lack of memory, processing power, etc.
    3. Data transfer problems: packets that carry flow records sent from
       the exporting process to the collecting process, are dropped by the
       network. Examples are connection failures, congestions in combination
       with an unreliable transport protocol, etc.
    4. Collecting process limitations: it may be experiencing congestion
       and not able to buffer new flows records."

    Please note that if an unreliable transport protocol is used,
    reliability can be provided by higher layers. In such a case only
    lack of overall reliability MUST be indicated. For example reordering
    could be dealt with by adding a sequence number to each packet.

    The data transfer between exporting process and collecting process MUST
    be open to reliability extensions including at least
       - retransmission of lost flow records,
       - detection of disconnection and fail-over, and
    This extensibility MAY be used to provide additional reliability."

5. Add to References:

---

From: "Tal Givoly" <[email-masked]>
Date: Wed, 22 Jan 2003 15:41:42 -0800

Juergen,

Agreed with all with exception for a minor change I propose to item 4.

The problem with the current phrasing is that reliability extension may not
cover the most basic means for cost-effective deployments. Therefore, I
would like the following bullet added to the bullets beginning with "-":

"   - an indication of acknowledgement from the collection process to the
exporting process, indicating to the exporting process that it may discard
flow information records that have been received at the collection process."

I believe that by mandating the protocol be at least extensible enough to
support this, we can achieve the desired level of reliability and permit for
cost-effective deployments (less than 1-1 redundancy). This is one of the
least implementation-dependent ways to phrase this item. It is implemented
in slightly different ways in a large variety of protocols today, DIAMETER,
GTP', RADIUS, CRANE, and Streaming IPDR to name but a few. It crystalizes
what extension would be required to afford the right level of applicability
to the protocol.

Best regards,

Tal

-----Original Message-----
From: majordomo listserver [mailto:[email-masked]]On Behalf
Of Juergen Quittek
Sent: Wednesday, January 22, 2003 3:07 PM
To: [email-masked]
Subject: Re: [ipfix-req] proposed changes for requirements document

Dear all,

    Juergen

5. Add to References:

--
Help        mailto:[email-masked] and say "help" in message
body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: Nevil Brownlee <[email-masked]>
Date: Thu, 23 Jan 2003 12:46:36 +1300

> proposal for changing the requirements draft. Again I am asking:

From the WG chair perspective, those changes look fine to me.
Can we please try to agree on them quickly, and get the next version
of the requirements published?  Could Juergen submit the draft
next Monday, for instance?

We need to get it into WG last call, so that we can submit it to IESG well
before the draft deadline for the San Francisco IETF - that's starting to 
seem very close now!

Once the requirements document is done (or better, right now) we also
need to get on with the IPFIX Evaluation process - the WG goal is
to get agreement at/soon after the San Francisco meeting on the
IPFIX protocol so that we can restart progress on the Architecture
and Data Model documents.

Cheers, Nevil

-----------------------------------------------------------------------
   Nevil Brownlee                   Director, Technology Development
   Phone: +64 9 373 7599 x88941     ITSS, The University of Auckland
   FAX: +64 9 373 7021      Private Bag 92019, Auckland, New Zealand

-------------------------------------------------
This mail sent through University of Auckland
http://www.auckland.ac.nz/

---

From: Sebastian Zander <[email-masked]>
Date: Thu, 23 Jan 2003 06:06:24 +0100

Hi Juergen,

comments on Reliability inline

> proposal for changing the requirements draft. Again I am asking:
[...]
>       and not able to buffer new flows records."

The first sentence explicitly states an req. for loss during data transfer
because this is a section in the "Data Transfer" chapter. In the list there
are a couple of other reasons for loss (some which are even covered in a
different section). I find this quite confusing. I suggest to remove this
list from the draft (reason 1 and 3 are already covered within the draft)
or find a more appropriate place for it.

comment on reason 1: loss may not result in flow record loss. The loss may
even occur before there is a flow record.

>       - detection of disconnection and fail-over, and

point missing?

>    This extensibility MAY be used to provide additional reliability."

Cheers,

Sebastian

-- 
Sebastian Zander                         E-mail: [email-masked]
FhI FOKUS / Global Networking (GloNe)    Tel: +49-30-3463-7287
Kaiserin-Augusta-Allee 31                Fax: +49-30-3463-8287
D-10589 Berlin, Germany                  www.fokus.fhg.de/usr/sebastian.zander

---

From: Juergen Quittek <[email-masked]>
Date: Thu, 23 Jan 2003 09:56:05 +0100

Dear all,

I completely understand Tal's arguments here. My only concern
about this is that I think we declared application level
acknowledgements to be out of scope. But my memory might be wrong.

If noone else shares this memory or has other concerns, then I
suggest to accept Tal's proposal but use a less solution-oriented
phrasing instead of the suggested one:

   "- acknowledgement of flow records by the collecting process."

        Juergen

---

From: "Tal Givoly" <[email-masked]>
Date: Thu, 23 Jan 2003 09:41:40 -0800

Juergen,

This is less explicit but is a sufficient placeholder so that in
architecture discussions it may be refined.

I support this compromise.

Tal

-----Original Message-----
From: Juergen Quittek [mailto:[email-masked]]
Sent: Thursday, January 23, 2003 12:56 AM
To: Tal Givoly
Cc: [email-masked]
Subject: RE: [ipfix-req] proposed changes for requirements document

Dear all,

        Juergen

---

From: Nevil Brownlee <[email-masked]>
Date: Fri, 24 Jan 2003 08:45:56 +1300

>    "- acknowledgement of flow records by the collecting process."

1) Your memory is correct, application-level acks are out of scope.

2) But Tal said ..

>     be open to reliability extensions including at least

   so he's only proposing tht the precesses MUST 'be open to extensions,'
   which was the consensus from our Atlanta meeting.

Tal's text lists two obvious reliability extensions, but I don't think
that amounts to a 'solution-oriented phrasing.'  I think it's OK as it is.

Cheers, Nevil

---

From: Nevil Brownlee <[email-masked]>
Date: Fri, 24 Jan 2003 08:47:22 +1300

> I support this compromise.

Well in that case, so do I.  Maybe I should read all my mail before
I start answering it :-)

Cheers again, Nevil