Subject: [ipfix-protocol] PROTO-5, 6, 7, 8, 9, 10, 11: SCTP and SCTP-PR

From: Benoit Claise <[email-masked]>
Date: Fri, 09 Jul 2004 15:47:34 +0200

Dear all,

 From the last meeting minute:

    - Transport Protocol: finally agreed on SCTP MUST, TCP MAY, UDP MAY
    Q: [Juergen] Is an implementation compliant if it just supports PR-SCTP?
    A: [Benoit] MUST for PR-SCTP makes most sense, but I have to ask Nevil.
    A: [Nevil Brownlee (via Jabber)] I will talk to SCTP people on SCTP vs
    PR-SCTP.

I think we agree a MUST for PR-SCTP makes more sense that simply SCTP.
Note also that PR-SCTP is a standard now: RFC3758 has been published.

Section 5.1 "Transport Compliance and Transport Usage" specifies:

   SCTP [RFC2960] MUST be implemented by all compliant implementations. 
   UDP [UDP] and TCP [TCP] MAY also be implemented by compliant 
   implementations. 
    
   SCTP SHOULD be used in deployments where exporters and collectors 
   are communicating over links which are susceptible to congestion. 
    
   TCP MAY be used in deployments where exporters and collectors 
   communicate over links which are susceptible to congestion, but SCTP 
   is preferred, due to its ability to limit back pressure on exporters 
   (especially when using PR-SCTP [SCTP-PR]) and its message vs. stream 
   orientation. 
    
   Other non-congestion aware protocols (like UDP) MAY be used in 
   deployments where exporters and collectors always communicate over 
   dedicated links which are not susceptible to congestion. 

We would like to propose the following text

   SCTP [RFC2960] and SCTP-PR [RFC3758] MUST be implemented by all 
   compliant implementations. UDP [UDP] MAY also be implemented by 
   compliant implementations. TCP [TCP] MAY also be implemented by 
   compliant implementations. 
    
   SCTP-PR SHOULD be used in deployments where exporters and collectors 
   are communicating over links which are susceptible to congestion. 
   SCTP-PR is capable of providing any required degree of reliability.
    
   TCP MAY be used in deployments where exporters and collectors 
   communicate over links which are susceptible to congestion, but SCTP-PR 
   is preferred, due to its ability to limit back pressure on exporters 
   and its message vs. stream orientation. 
    
   Other non-congestion aware protocols (like UDP) MAY be used in 
   deployments where exporters and collectors always communicate over 
   dedicated links which are not susceptible to congestion. 

The attached file is a proposal to address the SCTP/SCTP-PR part of issues PROTO-5 till PROTO-11.

Many thanks to Randall Stewart for the technical correction.
Many thanks to Stewart Bryant for the text improvement ... manies anglish mistaicks ;)

If accepted, the same structure could be applied for UDP and TCP.

Feedback?

Regards, Randall, Stewart, and Benoit.

.3 SCTP 
 
This section describes how IPFIX can be transported over SCTP 
[RFC2960] using the PR-SCTP [RFC3758] extension.  

5.3.1    Congestion Avoidance 
 
The SCTP transport protocol provides the required level of 
congestion avoidance by design. 

5.3.2    Reliability 
 
The SCTP transport protocol is by default reliable, but has the 
capability to operate in unreliable and partially reliable modes 
[RFC3758]. 
 
Using reliable SCTP streams (referred to hereafter as "streams") for 
the IPFIX export is not in itself a guarantee that all Data Records are 
delivered. If there is congestion on the link from the Exporting Process to 
the Collecting Process, or if a significant number of retransmissions are 
required, the send queues on the Exporting Process may fill up: the 
Exporting Process MAY either suspent export or discard IPFIX Messages. 
If Data Records are discarded the sequence numbers used for export 
MUST reflect the loss of data. 

5.3.3 MTU

SCTP provides the required IPFIX Message fragmentation service based on
path MTU discovery.

5.3.4    Exporting Process 

5.3.4.1  Association 
 
The Exporting Process MUST create at least one assocation 
(connection "bundle" in SCTP terminology) to the Collecting Process.
However, the Exporting Process MAY create more than one association.
 
The Collecting Process MUST NOT initiate the connection. 

5.3.4.2 Source ID 
 
The IPFIX Message MUST contain a Message Header, which includes a 
source id (SID).  The Exporting Process uses the SID to
uniquely identify to the Collecting Process the Observation
Domain that metered the Flows.
 
If a Metering Process consists of a single Observation Domain, a 
single SID value MUST be used for all IPFIX Messages. 
 
If a Metering Process consists of multiple Observation Domains, 
a unique SID value for each Observation Domain MUST be used. 

5.3.4.3  Stream 
 
An Exporting Process MUST request at least two outbound streams 
per association. The first stream (referred to as stream zero in the 
rest of the document), is used to send the Template FlowSet and
the Options Template FlowSet. Stream zero MUST be fully reliable. 
Data FlowSets MUST NOT be sent on stream zero.

Depending on the application requirement, the Exporting Process 
selects the mode (unreliable, partially reliable, or fully 
reliable mode) of the stream, used to send the Data FlowSets. 
Unreliable mode MAY be used where the application does not 
require reliable transmission and the use of a retransmission 
queue is impractical.

An Exporter MAY use multiple streams to export Data FlowSet, 
in some cases different applications will have different 
requirements in terms of reliability. In such a case, the Observation 
Domain MUST use the same SID value on all of the multiple streams it uses. 
Data FlowSets from multiple Observation Domains MUST NOT be transmitted 
over the same stream; the Collecting Process should however verify that 
the SID values are the expected values. 

When Data FlowSets are exported over a partial reliable stream, 
they SHOULD be marked for retransmission as long as there is room 
in the SCTP send queues. However during times of congestion or other 
retransmission events, if the queue overflows, the oldest data 
record that has been transmitted and marked as partially reliable
should be freed and marked to be skipped per the PR-SCTP [RFC3758]
specification. The freed buffer space should then be re-used for the 
new Data FlowSets being exported. 
 
5.3.4.4  Template 

The Templates FlowSet and Option Template FlowSet MUST be sent on 
stream zero with full reliability. 

New Template Records SHOULD be transmitted as soon as they are created 
on the Metering Process, and preferably before any associated Data Record 
is transmitted. The Collecting Process SHOULD accept Data Records without 
the associated Template Record.

5.3.4 Collecting Process  
 
The Collecting Process SHOULD listen for a new association request 
from the Exporting Process. The Exporting Process will request a 
number of streams to use for export. A Collecting Process
MUST support a least two inbound streams per association. 
A Exporting Process and Collecting Process MAY ask for and support 
more than two streams.
 
The Collecting Process SHOULD verify that the received IPFIX 
Messages inside one stream does not have differing SID values and 
silently discard any data that does NOT meet the collected values. The 
Exporting Process SHOULD NOT transmit messages inside one stream with 
multiple SID values. The correlated Flow Records are then treated 
like a normal export Flow.

If the Collecting Process receives an IPFIX Message that it cannot 
decode, it SHOULD reset the SCTP association.

5.3.5 Failover

If the Collecting Process does not acknowledge the attempt by the
Exporting Process to establish an association it will retry using
the SCTP exponential backoff feature. The Exporter MAY log an alarm 
if the time to establish the association exceeds a specified
threshold.

If Collecting Process failover is supported by the Exporting Process
a second SCTP association MAY be opened in advance.

---

From: Stewart Bryant <[email-masked]>
Date: Wed, 14 Jul 2004 12:38:41 +0100

>decode, it SHOULD reset the SCTP association.
The last para should be changed to:

If the Collecting Process receives an IPFIX Message that it cannot
decode, it MUST reset the SCTP association, discard the message
and log the error.

and then we should add the following paragraph:

When an SCTP association is closed, the Collecting Process MUST discard 
all templates
received over that association and stop decoding IPFIX Messages that use 
those templates.

- Stewart

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: Robert Lowe <[email-masked]>
Date: Wed, 14 Jul 2004 13:26:06 -0500

Stewart Bryant wrote:

>> MUST support a least two inbound streams per association. A Exporting 
>> Process and Collecting Process MAY ask for and support more than two 
>> streams.
>>
>> The Collecting Process SHOULD verify that the received IPFIX Messages 
>> inside one stream does not have differing SID values and silently 
>> discard any data that does NOT meet the collected values. The 
>> multiple SID values. The correlated Flow Records are then treated like 
>> a normal export Flow.
>>

In light of that...

Section 13, paragraph 3:

    The Template IDs must remain constant for the life of the Metering
    Process and the Exporting Process. If the Exporting Process or the
    Metering Process restarts for any reason, all information about
    Templates will be lost and new Template IDs will be created.
    Template IDs are thus not guaranteed to be consistent across an
    Exporting Process or Metering Process restart.

Should "lost" be changed to "discarded"?  Other changes here?

Also noted:

Section 13, paragraph 4:

    A newly created Template record is assigned an unused Template ID
    from the Exporter. If the template configuration is changed, the
    current Template ID is abandoned and SHOULD NOT be reused until the
    Metering Process.

(The last shown sentence is incomplete.)

Section 14, paragraph 4/5:

    The life of a template at the Collecting Process is limited to a
    fixed refresh timeout. Templates not refreshed from the Exporting
    Process within the timeout are expired at the Collecting Process.

Why bother to refresh at all given SCTP?

    The Collecting Process MUST NOT attempt to decode the Flow or
    Options Data Records with an expired Template. At any given time the
    Collecting Process SHOULD maintain the following for all the current
    Template Records and Options Template Records: <Exporting Process,
    Observation Domain, Template ID, Template Definition, Last Received>
    Note that the Observation Domain is identified by the Source ID
    field from the IPFIX Message.

    Template IDs are unique per Exporting Process and per Observation
    Domain.

Dumb question... what uniquely identifies a template ID at the collector?
If the SID is assigned by the exporter, it can't be guaranteed to be
unique at the collector.  How does the exporting process identify itself
to the collector?  Socket?

-Robert

---

From: Benoit Claise <[email-masked]>
Date: Thu, 15 Jul 2004 15:21:31 +0200

> When an SCTP association is closed, the Collecting Process MUST 
> discard all templates
> received over that association and stop decoding IPFIX Messages that 
> use those templates.

Good feedback. Inserted.

Regards, Benoit.

---

From: Benoit Claise <[email-masked]>
Date: Thu, 15 Jul 2004 15:36:39 +0200

Robert,

You've got some good points below regarding the section 13 "Template 
Management"
This section was initially written for UDP as a transport (long time 
ago...). As a consequence, it's not up to date.
The issue PROTO-25 addresses this.

       PROTO-25: The section 11 "Template Management" will have to updated
       according to the transport protocol.
       - For example, the point 2 of the section "Template Management".
         Remark: the template management will vary with TCP, SCTP,
         etc...
         Must have both sections updated: transport updated and
         template  management sections (BTW, this is the same for
         the failover section).

We should first agree on the 3 transport sections, and then change this 
section 13.

Regards, Benoit.

> Stewart Bryant wrote:
>
>>
>>> 5.3.4 Collecting Process The Collecting Process SHOULD listen for a 
>>> new association request from the Exporting Process. The Exporting 
>>> Process will request a number of streams to use for export. A 
>>> Collecting Process
>>> MUST support a least two inbound streams per association. A 
>>> Exporting Process and Collecting Process MAY ask for and support 
>>> more than two streams.
>>>
>>> silently discard any data that does NOT meet the collected values. 
>>> The Exporting Process SHOULD NOT transmit messages inside one stream 
>>> with multiple SID values. The correlated Flow Records are then 
>>> treated like a normal export Flow.
>>>

---

From: Robert Lowe <[email-masked]>
Date: Thu, 15 Jul 2004 10:55:17 -0500

Benoit Claise wrote:

Benoit/Stewart,

> We should first agree on the 3 transport sections, and then change this 

Understood.  Any comments on section 14?  I'm mostly confused about how
the collector process is to uniquely identify template IDs as belonging
to a unique observation domain (Stewart's revised text wanted the collector
process to discard them under a couple of error conditions).  Unless I
am mistaken, observation domain/SID and template ID are assigned by the
exporter.  How are collisions avoided?  IPV4/V6 socket from the exporter?
If so, that should be included in the list of 'SHOULD maintain', right?

-Robert