Subject: [ipfix] Lost Flow Records - Synch Message

From: "Reinaldo Penno" <[email-masked]>
Date: Fri, 9 May 2003 11:07:12 -0700

The editors of the protocol draft had a conf call and we came up with the
main list of issues we need to sort out at this point.

The first issue we need to get closure is the method to account for lost
flow records. The proposal is in line with the "do not  make assumption that
the transport will be TCP". This does not preclude and/or is intended to fix
the higher reliability extensions requirement. Mark will send an email with
a proposal on this subject

The proposal on the table is to send a IPFix Sync message periodically
(periodicity is configurable), with the following information (aside the
standard IPFix header)  

* Number of flow records sent (for each template?)
* Packets and bytes sent (for each template?)

Please advise if this is okay/not okay, etc, etc

Regards,

Reinaldo

---

From: Alex Audu <[email-masked]>
Date: Fri, 09 May 2003 13:57:10 -0500

Is the sequence number in the header packet  not enough for tracking
lost flow records?

Alex.

Reinaldo Penno wrote:

>
>
> The editors of the protocol draft had a conf call and we came up with
> the main list of issues we need to sort out at this point.
>
> The first issue we need to get closure is the method to account for
> lost flow records. The proposal is in line with the "do not  make
> assumption that the transport will be TCP". This does not preclude
> and/or is intended to fix the higher reliability extensions
> requirement. Mark will send an email with a proposal on this subject
>
> (periodicity is configurable), with the following information (aside
> the standard IPFix header)
>

---

From: Sebastian Zander <[email-masked]>
Date: Mon, 12 May 2003 02:21:04 +0200

> Is the sequence number in the header packet  not enough for tracking 

Having only the sequence number the collector only would know the number
of packets lost. It would not know the number of flow records lost nor
which flow records had been lost.

-- 
Sebastian Zander                         E-mail: [email-masked]
Fraunhofer FOKUS / METEOR                Tel: +49-30-3463-7287
Kaiserin-Augusta-Allee 31                Fax: +49-30-3463-8287
D-10589 Berlin, Germany                  www.fokus.fraunhofer.de/usr/sebastian.zander

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: Sebastian Zander <[email-masked]>
Date: Mon, 12 May 2003 03:09:04 +0200

Hi Reinaldo,

having flow record counters per template is a very nice thing in my
opinion. But if we have this information why would we need the packets
and bytes sent per template information in addition?

What about information about lost template definitions? (A loss of a
template def could potentially cause flow records to be "lost" at the
collector...)

Cheers,

Sebastian

Reinaldo Penno wrote:
> flow records. The proposal is in line with the "do not  make assumption 
> that the transport will be TCP". This does not preclude and/or is 
> intended to fix the higher reliability extensions requirement. Mark will 
> send an email with a proposal on this subject
>

---

From: Mark Fullmer <[email-masked]>
Date: Sun, 11 May 2003 22:13:48 -0400

How lost templates are handled depends on the transport.

  UDP - templates are periodically retransmitted so a collector should
        buffer data until the template arrives or resource exhaustion.

  TCP - Templates shouldn't be lost.  There is a small window for
        a template to be lost in a primary/secondary failover
        situation.  The failover details have not been finalized
        yet.

  SCPT - Hopefully the templates can be done over a reliable
         channel.  Still have failover issues.

mark

---

From: Maurizio Molina <[email-masked]>
Date: Mon, 12 May 2003 12:46:54 +0200

> requirement. Mark will send an email with a proposal on this subject

I just wonder whether it makes sense discussing the details listed below
before the above mentioned proposal is circulated. Otherwise, we'd risk
to open a discussion with all the participants havig their (own) idea
about how the listed counters may be used by a protocol to achieve
reliability.
Regards,
Maurizio

---

From: Maurizio Molina <[email-masked]>
Date: Mon, 12 May 2003 14:39:04 +0200

> which flow records had been lost.

I think Alex's comment shows that there are different issues and discussing before the
_usage_ of the information could be beneficial, as I've asked in a previous mail. Here my
2 cents:
To build a reliable information transfer over an unreliable transport (which however in
the current requirement draft is NOT mandated)  the sequence number in the packet header
is probably enough.
The number of flow records sent would be on th econtrary useful for the collector to
judge on the reliability of the received information (draft-ietf-ipfix-reqs-09.txt, sec.
6.3.2, list points 3-5 (loss of flow records due to data transfer, mainteneance)).
In this respect, I'm noting that the mentioned list points 1 and 2 (loss of flow records
due to metering or exporting process limitation) would be difficult to quantitatively
report. A meter/exporter unable to create a flow record (e.g. because of memory
limitations) could be at most able to report the number of packets and bytes that it had
to drop, but not the number of _not even created_ flow records.
Regards,
Maurizio

---

From: Alex Audu <[email-masked]>
Date: Mon, 12 May 2003 09:11:51 -0500

My concern was sending periodic sync messages may take away from bandwidth resources
that could be put to better use exporting data. However, the sync message scheme would
have minimum impact if the collector initiated it ONLY after detecting a gap in the
sequence
numbers of packets received. I think this may be a good compromise.

Regards,
Alex.

---

From: "Tal Givoly" <[email-masked]>
Date: Mon, 12 May 2003 07:54:54 -0700

Hi,

Wasn't the suggestion of the evaluation committee and the WG chairs to begin
with a TCP version of the protocol and later, perhaps, add SCTP (or
SCTP-PR)? If that is the case, why are we discussing the behavior of a UDP
version of it as that should not be part of the IPFIX standard at all.

Regarding the TCP below, why wouldn't the templates be transmitted upon
re-establishing a connection or fail-over to avoid the unnecessary
complexities in the collector of receiving data for which it has not
information regarding its template?
=

Tal

-----Original Message-----
From: majordomo listserver [mailto:[email-masked]]On Behalf
Of Mark Fullmer
Sent: Sunday, May 11, 2003 7:14 PM
To: Sebastian Zander
Cc: [email-masked]'
Subject: Re: [ipfix] Lost Flow Records - Synch Message

mark

On Mon, May 12, 2003 at 03:09:04AM +0200, Sebastian Zander wrote:
> D-10589 Berlin, Germany
www.fokus.fraunhofer.de/usr/sebastian.zander

--
Help        mailto:[email-masked] and say "help" in message
body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: Mark Fullmer <[email-masked]>
Date: Mon, 12 May 2003 11:20:40 -0400

I'm only using UDP as an example of a lossy transport.
We're supposed to be transport agnostic.

The templates could be retransmitted on failover, and
that's probably the easiest thing to do.  Like I said
the details have not been worked out yet.  Things like
when to fail over, how to fail back, how to ensure stability
of the failover mechanism (prevent oscillations), do
the collectors know about each other, does the exporter
open connections to all the potential collectors and keep
them primed with template info, do we support load
balancing among collectors, etc.

mark

---

From: Mark Thibodeau <[email-masked]>
Date: Mon, 12 May 2003 11:19:17 -0400

I agree with Tal.  With TCP, we do not have to worry about losing templates.

For the lost flow statistics, a sequence number and exported flow count should be
adequate.  The collector can use the sequence number to detect that a packet has
been lost.

The total number of flows exported can be used to determine how many flows have
been lost.

It could be that this is redundant information though.  With just the total flow
count, the collector should be able to figure everything out.  But, the sequence
numbers make things a bit easier.

I vote that we keep both mechanisms.

Cheers,
Mark

--

****************************************************
* Mark Thibodeau, P Eng.
* 670 RSP Development - Firmware Designer
* Alcatel CID - (613) 784-5375
* Fax - (613) 599-3696
* [email-masked]
****************************************************

---

From: Mark Fullmer <[email-masked]>
Date: Mon, 12 May 2003 11:26:59 -0400

The synch message could be sent with a user configurable
period.  I was thinking the default would be O(minutes).

I generate graphs from 5 minute updates right now (v5
deployments) which seems to work well.

mark

---

From: "Tal Givoly" <[email-masked]>
Date: Mon, 12 May 2003 09:01:15 -0700

Mark,

But even if we do use SCTP-PR, the "control" information (such as templates,
indication of synch messages, etc.) is most likely to be transported over
the reliable SCTP link rather than the partially reliable one. This was
suggested by Benoit in the IETF meeting. In that case, should we introduce
undue complexity by supportinging arbitrary theoretical protocols?

Tal

-----Original Message-----
From: Mark Fullmer [mailto:[email-masked]]
Sent: Monday, May 12, 2003 8:21 AM
To: Tal Givoly
Cc: Sebastian Zander; [email-masked]
Subject: Re: [ipfix] Lost Flow Records - Synch Message

mark

> > > The first issue we need to get closure is the method to account for
lost
> > > (periodicity is configurable), with the following information (aside
the

---

From: "Tal Givoly" <[email-masked]>
Date: Mon, 12 May 2003 09:01:24 -0700

Mark,

All those items you mention below regarding the fail-over are very valid
items to be discussed. Perhaps we should create a set of items for each of
them and discuss them independently rather than all in one lump?

It seems to me that the mandatory requirement that the IPFIX protcol be
extensible enough to support reliability extensions basically implies that
at least one such extension needs to be demonstrated. Therefore, it makes
sense to make an attempt to articulate just such an extension (if not
standardize it).

- when to fail over
<tal>
a) when the window of outbound messages has been reached and an
acknowledgement has not been received from the collector.
b) when a timeout on an acknowledged message exceeds a pre-determined length
of time.
c) when the collector sends a message indicating it is incapable of handling
more traffic.
d) transport protocol disconnection from the collector (TCP link fails).
e) keep alives aren't received - if they are added, this would allow a
low-throughput link to detect the effective disconnection from the collector
before the transport protocol disconnection.
In either of these cases, establish connectivity to a back-up collector
(optionally as the connection could be already established - see below),
retransmit all unacknowledged data records to a stand-by collector;
</tal>

- how to fail back
<tal>
a) periodic retry mechanism to re-establish connection with "primary"
collector (introducing notion of priority of collectors is useful as the
"secondary" or "back-up" collector could be more regionally co-located and
shared by multiple exporters and may need to return to the role of back-up
asap.
b) failure of the secondary/back-up collector in the same way described
above.
c) collector establishes connection to the exporter to "take over" the
burdon of collecting the data.

- how to ensure stability of the failover mechanism (prevent oscillations)
<tal>
On one hand, neither of these fail-back modes causes short-term oscillations
(immediate). On the other hand, if the machines are working under load
conditions, and there is no way to communicate the load to the exporter,
there is no way to avoid oscillations by re-establishing export to a
collector that is already under load.
Therefore, it is probably left as an implementation issue to make sure that
a collector assume the collection process if it cannot handle the load.
</tal>

- do the collectors know about each other
<tal>
No. This introduces undue complexity. This could be left for developers of
collectors to figure out whether this is of utility. Standardizing the
methods of interactions between collectors is especially complex.
</tal>

- does the exporter open connections to all the potential collectors and
keep them primed with template info.
<tal>
This would be ideal as it implies minimum needed footprint on device
generating the information (information could be sent once conditions
require a fail-over. However, this could be optional in implementation as
from the collector-exporter interaction, there could be no implications and
exporters could have options to do either (maintain multiple connections to
collectors or maintain one active and create another upon failover). The
collectors can be unaffected as well as the protocol itself.
</tal>

- do we support load balancing among collectors
<tal>
Nice extension option, however, it doesn't seem that the requirements
document indicate this is a mandatory extension. If one can find a nice way
of doing this technically as a bi-product of reliability extensions, I would
support it. However, it is usually not merely a reliability extension.
</tal>

I recall someone (Paul :-) volunteering to track issues - I believe all
these issues could be considered sub-items under the category of
"reliability extensions" and may need to be discussed on separate threads.

Tal

-----Original Message-----
From: Mark Fullmer [mailto:[email-masked]]
Sent: Monday, May 12, 2003 8:21 AM
To: Tal Givoly
Cc: Sebastian Zander; [email-masked]
Subject: Re: [ipfix] Lost Flow Records - Synch Message

mark

lost
the

---

From: Sebastian Zander <[email-masked]>
Date: Tue, 13 May 2003 03:28:04 +0200

> the current requirement draft is NOT mandated)  the sequence number in the packet header

Probably not. It only enables the collector to find out which packets have been lost.
Actually unless you don't know the first sequence number the collector can't even find out
about packets lost before the first arrives...

 > to drop, but not the number of _not even created_ flow records.

I think that in case of point 2 you may be able to gauge the loss of flow records because
after the metering process you have flow records. I totally agree that you may not know
anything about lost flow records in case of point 1. That was and is my one of my issues
with section 6.3.2, see old post http://ipfix.doit.wisc.edu/archive/1532.html.

Cheers,

Sebastian

---

From: Sebastian Zander <[email-masked]>
Date: Tue, 13 May 2003 03:41:32 +0200

> I vote that we keep both mechanisms.

The sequence number would also be usefull for duplicate detection. There is a
timestamp but it has only ms accuracy...

>>>Sebastian Zander                         E-mail:
>>[email-masked]
>>

---

From: "Tal Givoly" <[email-masked]>
Date: Mon, 12 May 2003 21:33:41 -0700

Hi,

For duplicate detection, each flow must have sequence numbers unique enough
to disambiguate it from any other flow. Therefore, it usually makes sense to
assign them per template and perhaps per startup time of device. For most
devices it doesn't make sense to make them "globally unique" as context of
transmitting device is known. But it is necessary to disambiguate flows from
the same device after resets of device.

It would greatly assist duplicate elimination if any flow that has been
retransmitted is also marked as duplicate with some "duplicate bit". This is
a hint to the collector so that it only needs to check those flow records
marked as duplicate against other flows it has received. This has a benefit
of optimizing duplicate elimination at the collector while introducing no
overhead and minimum complexity to the exporter - as it already should know
whether it is sending a flow record for the first time or for any subsequent
retransmission.

Tal

-----Original Message-----
From: Sebastian Zander [mailto:[email-masked]]
Sent: Monday, May 12, 2003 6:42 PM
To: Mark Thibodeau
Cc: Tal Givoly; Mark Fullmer; [email-masked]
Subject: Re: [ipfix] Lost Flow Records - Synch Message

> I vote that we keep both mechanisms.

The sequence number would also be usefull for duplicate detection. There is
a
timestamp but it has only ms accuracy...

body
body

---

From: Mark Fullmer <[email-masked]>
Date: Thu, 15 May 2003 21:16:59 -0400

I don't think there's much more than periodic retransmission 
of templates and some option data to support a lossy control channel.

v9 has lifetimes for templates which is one way to solve some issues
with a connectionless lossy transport, that is something else that
can go away.

I agree though, if we don't need to support this I'd rather not, but there's
still DCCP.

mark

---

From: "Tal Givoly" <[email-masked]>
Date: Thu, 15 May 2003 23:22:45 -0700

Mark,

It seems to me pretty obvious that periodic retransmission of templates
introduces undue complexity in the collector to buffer data for an arbitrary
period of time until such time that it knows how to decode packets for which
it has not received the templates. By transmitting only the templates (not
necessarily the data) over a transport that is reliable (such as TCP or
SCTP), you avoid this undue complexity. This doesn't imply that the data
itself doesn't flow over a partially reliable transport (SCTP-PR or UDP).

Furthermore, templates don't need lifetimes. Templates could remain in
effect from the beginning of a session until changed during that session or
by tearing down that session and establishing a new one for the new
templates. Lifetimes introduce yet another unnecessary complexity.

In general, the way templates are defined in NetFlow v9 (with transient
identifier) implies that different record structures (templates) must be
identified IMPLICITLY by virtue of the specific set of attributes within the
template. So supposing a collector needs to decode a set of templates in a
communication - an attribute-by-attribute comparison needs to be used to
identify what is the meaning of the data in the template. There is no
reference to some externally defined record structure. I believe this is
going to be one of the major drawbacks of the current template mechanism
(this item has been discussed on the mailing list -
http://ipfx.doit.wisc.edu/list/ipfix/archive/1336.html).

Tal

-----Original Message-----
From: majordomo listserver [mailto:[email-masked]]On Behalf
Of Mark Fullmer
Sent: Thursday, May 15, 2003 6:17 PM
To: Tal Givoly
Cc: [email-masked]
Subject: Re: [ipfix] Lost Flow Records - Synch Message

mark

---

From: Ganesh Sadasivan <[email-masked]>
Date: Fri, 16 May 2003 15:25:45 -0700

- Periodic refreshing would be good - especially in a lossy network and
when  template config change is taking place even if the transport is
reliable. Periodicity can be configured, based on the exporter and the
network through which the export data traverses.

- The same holds good for life-time for templates. The end-of-life is
an alternative to using option templates or some other means to that
a template is unconfigured.

Thanks
Ganesh

---

From: "Tal Givoly" <[email-masked]>
Date: Fri, 16 May 2003 16:17:09 -0700

Ganesh,

Perhaps it's only me, but I still fail to understand why periodic refreshing
is good or even necessary if the control information is sent over TCP or
SCTP? Once a connection is established between an exporter and a collector,
the templates could be communicated and remain valid for the duration of the
session (or until changed).

Also, life-time does not seem to be a good solution for changing templates.
What if the life-time implies the template is still valid, but the template
must be changed at that point? That case already calls for the need to have
some method to "reconfigure" the template. If, on the other hand, templates
are short-lived, that implies excessive overhead is spent on communicating
these templates.

Both of these items seem to be UDP-related legacy that add little or no
value and undue complexity once you can use either TCP or SCTP (at least for
the control information).

Tal

-----Original Message-----
From: Ganesh Sadasivan [mailto:[email-masked]]
Sent: Friday, May 16, 2003 3:26 PM
To: Tal Givoly
Cc: Mark Fullmer; [email-masked]
Subject: Re: [ipfix] Lost Flow Records - Synch Message

Thanks
Ganesh

body

---

From: Ganesh Sadasivan <[email-masked]>
Date: Fri, 16 May 2003 18:10:02 -0700

Tal,

> session (or until changed).

In the event of congestion, though TCP does best effort, packet could be dropped

right at the exporter, anywhere in the network or at the collector even though
the
connection still remains valid. It is quite undesirable if the lost packet is
one that
contains templates. If there is no periodic update, then collector will never be
able
to decode flow records corresponding to the lost templates.

> are short-lived, that implies excessive overhead is spent on communicating

The reconfigured templates have a different ID. The old ones silently age out.
The life-time value is deployment specific.

---

From: "Tal Givoly" <[email-masked]>
Date: Sun, 18 May 2003 09:10:05 -0700

Ganesh,

Responses interspersed below:

-----Original Message-----
From: Ganesh Sadasivan [mailto:[email-masked]]
Sent: Friday, May 16, 2003 6:10 PM
To: Tal Givoly
Cc: Mark Fullmer; [email-masked]
Subject: Re: [ipfix] Lost Flow Records - Synch Message

Tal,

> session (or until changed).

In the event of congestion, though TCP does best effort, packet could be
dropped

right at the exporter, anywhere in the network or at the collector even
though
the
connection still remains valid. It is quite undesirable if the lost packet
is
one that
contains templates. If there is no periodic update, then collector will
never be
able
to decode flow records corresponding to the lost templates.

<Tal> Yes. TCP packets are merely IP packets that could get dropped.
However, this will trigger a retransmission by the TCP/IP stack and
eventually a disconnection (if the packets don't get through). If it will
not disconnect, the template will get to the collector - guaranteed
(irrespective of how many packets get lost trying). If it does disconnect,
the reconnected stream would send it. It is probably expected that there is
no way to send usage data either if these, retransmitted packets, do not get
through.
</Tal>

> are short-lived, that implies excessive overhead is spent on communicating

The reconfigured templates have a different ID. The old ones silently age
out.
The life-time value is deployment specific.

<Tal>
Once again, the template mechanism in NetFlow v9 is arbitrary and causes
undue complexity and ambiguity in the collector to "discover" the meaning of
the template (semantics) from the content. The life-time introduces
unnecessary complexity in the collector (for instance, what happens if raw
data is stored and reprocessed at a different pace than real-time?).
</Tal>

---

From: Ganesh Sadasivan <[email-masked]>
Date: Sun, 18 May 2003 19:20:01 -0700

Tal,

> no way to send usage data either if these, retransmitted packets, do not get

Yes, I agree that as long as the packet is in TCP send window of the exporter
it will get to the collector through retransmission. But it is possible that the

exporter is not able to queue the packet into send window due to delays
in retransmission which gets dropped in exporter floor. Whether the application
(in this case V9) keeps track of this loss or not, it needs to retransmit the
template
and this needs to happen before too much of flow records pertaining to this
template reach the collector.
Now since TCP is not the only targeted transport and one could use other
congestion aware transport which are partially reliable, and so one can expect
data
loss in the contorl connection. So a periodic scheme though not super efficient
works well regardless of the transport  like TCP/SCTP-PR/DCCP used
for control connnection.

> data is stored and reprocessed at a different pace than real-time?).

Agreed that there are better means to achive the same. But I don't agree that
there is any
difficulty in discovering the semantics of the template as the .template ID ->
data mapping
for a session is unique. .So there should not be any complication whenever the
records are processed. If you think this is not the case, please explain.
The roll-over case is when a large number of templates are generated in a short
period
of time. Eg. of one such case which can lead this to happen is ACLs in which
each rule
maps to a flow . But are we discussing any such cases here?

Thanks
Ganesh

---

From: Mark Fullmer <[email-masked]>
Date: Mon, 19 May 2003 01:11:39 -0400

> itself doesn't flow over a partially reliable transport (SCTP-PR or UDP).

I'm not sure the race condition that results in a collector receiving
data encoded with a template before the template can be avoided in
all situations.

  With a router the device may be in a situation where it starts
  forwarding packets and creating flow state before the template
  gets sent.  In this situation it will either have to drop the
  flows or buffer them to preserve the template first rule, the
  latter may not be possible.

  With SCTP I don't think it's possible to give the reliable
  stream with the templates priority over the stream with flow data.

A robust collector will probably want to handle with data before templates
by buffering the data.  A less robust one will drop the data.

I'm just not ready to give up on supporting a totally unreliable
transport if it's not difficult to do.

> templates. Lifetimes introduce yet another unnecessary complexity.

It's my understanding that the template lifetimes are there to prevent
race conditions that would exist if a router rebooted and reused the
same template with a different format.

Regardless, I agree the template lifetime should go.  There are
attributes that only need to be sent once, for example an exporter ID
or sysUpTime : Real Time mapping.  With a connection oriented protocol
there are no issues.  With a non connection oriented protocal something
needs to be done to emulate a session, for example a random ID (cookie)
sent along with every packet...If we need to support a non connection
oriented transport.  This could be done in the existing framework with
an option.

> http://ipfx.doit.wisc.edu/list/ipfix/archive/1336.html).

Even without transient ID's there are problems, for example what if 
two access lists mapped packets into two flow cache's that used the
same template.  The collector would not be able to map the flows back
to rules on the router.  This is a documentation problem though, not
a protocol problem.  There are many straightforward ways to accomodate
this behavior, such as an id in the flow or supplementary template info.

mark

---

From: Mark Fullmer <[email-masked]>
Date: Mon, 19 May 2003 01:16:16 -0400

If this happens the application layer write() will fail and the template
can immediatly be marked for retransmission.

I don't think the refresh is needed for a reliable control connection
such as TCP or SCTP/PR-SCTP.

mark

---

From: Ganesh Sadasivan <[email-masked]>
Date: Mon, 19 May 2003 11:22:10 -0700

> can immediatly be marked for retransmission.

Yes if it is a blocking send. This need not be the case for a non-blocking send.

Ganesh

---

From: Peter Ludemann <[email-masked]>
Date: Mon, 19 May 2003 21:32:54 -0700 (PDT)

A small clarification of what Mark Fullmer wrote: a *later*
application layer write or close will fail if the data cannot
be transmitted at all through repeated (automatic) retries at
the TCP layer [which is out of the control of application
layer programmers].

This is why even though TCP has transport-layer ACKs, you
also need application layer ACK -- TCP will tell you if it
cannot deliver data but none of the implementations that I've
seen will tell you what data wasn't delivered [and this is
also why there is a possibility of duplication with fail-over
processing]. Also, TCP can take a very long time to time out,
so you may want additional timers running to force fail-over
sooner. The common way of handling this is with non-blocking
I/O (in Unix-like systems, this is done with select() [the
old way] or poll() [the new way]; Java 1.4's "nio" package
has "channels" with similar functionality).

(Note that the only difference between blocking and
non-blocking I/O for output is that blocking will wait if the
transmit buffer is full whereas non-blocking will always
return ... in terms of detection of inability to determine
which bytes have been successfully transmitted, both methods
are the same.)

I don't know enough details about SCTP implementations to
know whether application layer ACKs are necessary or not; but
I suspect that they are because it's not the job of the
transport layer to wait until the application decides that
the data has been sufficiently processed.

For excellent references on this, I suggest:
   W. Richard Stevens: TPC/IP Illustrated, which has the *BSD
      source code plus excellent over-all commentary on
TCP/IP
      and details. This code forms the basis of the various 
      BSD operating systems, Solaris, Linux, and (I think) 
      Windows-XP [there have been rewrites, but this code is
      still the base as far as I know]. VxWorks now also 
      BSD-derived code; I don't know about other RTOSs, nor 
      Cisco's IOS.  (This book also has the code for UDP, IP,

      ICMP, etc.)
   W. Richard Stevens: Unix Network Programming
   Jon C. Snader, Effective TCP/IP Programming
   Douglas Comer, Internetworking with TCP/IP.

All these books are available in multi-volume sets from
Amazon and other booksellers.

- peter

---

From: Sebastian Zander <[email-masked]>
Date: Tue, 20 May 2003 08:22:07 +0200

Periodic refreshment of templates is in my opinion pointless when a
reliable connection-oriented transport is used. Either the transport
is working (the refreshment just wastes bandwidth) or it fails in
which case the sender (exporter) knows that it fails and can act
accordingly (such as failover and send the templates to the new
collector). In this case periodic refreshment just adds unecessary
complexity to the exporter.

For UDP it is necessary if there are no ACKs from collector to exporter.
Periodic refreshments are basically forward error correction. If there
would be application layer ACKs anyway (to achieve application layer
reliability as explained by other people on the list) why have periodic
template refreshments at all?

Just my 0.02 cents on the periodic refreshment discussion,

Sebastian

---

From: Mark Fullmer <[email-masked]>
Date: Tue, 20 May 2003 10:37:38 -0400

> Yes if it is a blocking send. This need not be the case for a non-blocking send.

If it's a blocking send it will block, a non blocking send will return
an error or partial write.

Regardless, the point I was trying to make is that a periodic template
refresh mechanism is not required for a reliable transport.  If buffers
are full the application layer will be informed and be able to schedule
the failed template send again.

mark