Subject: RE: fragmented packets (was: Re: [ipfix-req] IPFIX Requirements Q uestions)

From: "Logg, Connie A." <[email-masked]>
Date: Wed, 28 Aug 2002 08:34:15 -0700

I would certainly like to see fragmented packets some how identified with
the first packet. I have an algorithm for processing netflow data that works
"most of the time" for extracting these for my netflow analysis, but most of the time
is not really good enough.  Our AFS traffic is an example. We really would like to be
able to accurately quantify the AFS traffic, as we have a lot of it. For high 
utilizations, the algorithm breaks down, as it also does when there are more than one
AFS transaction going on at a time.

Connie Logg - Network Analyst - 650-926-2879
Stanford Linear Accelerator Center
MS 97; 2575 SandHill Road; Menlo Park CA 94025
"Happiness is found along the way, not at the end of the road"

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: Wednesday, August 28, 2002 6:07 AM
To: Juergen Quittek
Cc: req
Subject: Re: fragmented packets (was: Re: [ipfix-req] IPFIX Requirements
Questions)

Juergen Quittek wrote:
> 
> Hi Paul,
> 
> --On 27 August 2002 20:26 -0400 [email-masked] wrote:
> 
> > Juergen Quittek wrote:
> >>
> >> --On 27 August 2002 14:49 -0400 [email-masked] wrote:
> >>
> >> >
> >> > As I work on the advocates document some questions
> >> > have come to mind on the requirements document.
> >> >
> >> > 4.3 Distinguishing flows using Transport Header Fields
> >> >
> >> >       In the case of fragmented packets there are
> >> >       no port numbers. Are we saying flow state information
> >> >       MUST be maintained? In general, it is not clear from
> >>
> >> No, the requirements document does not say so.
> >>
> >> >       the document what the requirements are for fragmented
> >> >       packets.
> >>
> >> I think the requirements should not include a requirement for
> >> keeping state of fragmented packets. Would you like to see
> >> an explicit statement on that? This would be a NO NEED FOR
> >> statement :-)
> >
> >       I think we need to be explicit. Most people will think
> >       of fragements as being counted in the same flow as the
> >       first packet not as 2 differernt flows.
> 
> I see. Waht about adding a new subsection to Section "5. Metering
> Process":
> 
> 5.8.  Packet Fragmentation
> 
>    In case of IP packet fragmentation, only one fragment of a single
>    packet might contain sufficient information for classifying the
>    packet correctly. The metering process MAY keep state of
>    IP packet fragmentation in order to map fragments that do not
>    contain sufficient header information correctly to flows.

  How about a slight re-wording...

    In case of IP datagram fragmentation, only the first fragment might
    contain sufficient information for classifying the packet correctly. 
    The metering process MAY keep state of IP fragmentation in order to 
    map fragments without sufficient header information to the same
    flow as the first fragmented packet.
	
> 
>  [...]
> 
> >> > 6.1 information Model
> >> >
> >> >       9  Packet Counter
> >> >       10 Byte counter
> >> >
> >> >               As mentioned earlier, for fragments this
> >> >               requires state information? If there is no
> >> >               state info, then what flow are the counted
> >> >               towards?
> >>
> >> Do you suggest to include a requirement for keeping fragment state info?
> >
> >
> >       A MAY is as far as I would go. There may certainly be probes
> >       that have this capability. We should have a requirement that
> >       says fragment counting MUST be clearly defined.
> 
> OK. Currently, we have
> 
>       9. packet counter
>          If a packet is fragmented, each fragment is counted as an
>          individual packet.
>      10. byte counter
>          Which bytes of a packet are counted MUST be defined exactly.
> 
> What about appending
> 
>          The behavior of the byte counter in case of IP packet
>          fragmentation MUST be clearly defined.

	With the new section on fragmentation I think we can 
	eliminate any mention of it on the counters.

> ?
> 
> Please note that we also have in the MAY attributes section
> 
>      26. fragmented packet counter
>          counter of all packets for which the fragmented bit is set in
>          the IP header

	I'm not sure why this was added. If you want that info,
	make the IP fragment bit part of the flow definition.

>     Juergen

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: Juergen Quittek <[email-masked]>
Date: Wed, 28 Aug 2002 18:29:08 +0200

Connie,

I have a again the same comment for you:

The first fragment of a packet that is observed, is not necessarily the
fragment containing the TCP/UDP header. IP allows packet re-ordering.
The fragment containing the first bytes of the IP payload may travel
slower through the network than the other fragments.

    Juergen

---

From: Simon Leinen <[email-masked]>
Date: 29 Aug 2002 11:28:36 +0200

On Wed, 28 Aug 2002 18:29:08 +0200, Juergen Quittek <[email-masked]> said:
> The first fragment of a packet that is observed, is not necessarily
> the fragment containing the TCP/UDP header. IP allows packet
> re-ordering.  The fragment containing the first bytes of the IP
> payload may travel slower through the network than the other
> fragments.

Not only that, the source may not even send the first bytes of an
oversize datagram as the first fragment.  Notably Linux sends
fragments from the end of the datagram towards the start.  The
standards apparently don't mandate a specific order of the fragments -
that would be of limited use anyway since IP doesn't guarantee
in-order delivery.  Sending the last part of the packet first has the
advantage that, if the fragments do happen to arrive in order, the
receiver immediately learns the total size of the datagram, and can
allocate buffer space accordingly.

Anyway, I think caching of fragmented packets just cannot be done
efficiently, and that we should certainly not enforce it in IPFIX.

This does have the implication that malicious users can "hide" their
higher-layer information (e.g. TCP/UDP port numbers) by intentionally
fragmenting packets just after the IP header.  That's a limitation I
can personally live with (note that this is also a problem for
stateless packet filters that try to filter based on higher-layer
information).
-- 
Simon Leinen				       [email-masked]
SWITCH				   http://www.switch.ch/misc/leinen/

	       Computers hate being anthropomorphized.

---

From: Benoit Claise <[email-masked]>
Date: Thu, 29 Aug 2002 12:44:47 +0200

Simon,

>in-order delivery. 
RFC 791:
    The first portion of the data is placed in the first new internet 
datagram,
    and the total length field is set to the length of the first datagram.

> Sending the last part of the packet first has the
Agreed.

Regards, Benoit.