Subject: Re: [[email-masked]: RE: [ipfix-req] Ipfix features]

From: [email-masked]
Date: Tue, 18 Jun 2002 13:07:28 -0400

Isn't the other problem that if fragemented flows are lumped 
into src, dst, proto, zero-src-port, zero-dst-port then
there are possibly multiple fragements counted in
one flow? 

Paul

"Logg, Connie A." wrote:
> 
> Actually, given the additional experience I have had in the last 3 months on processing metflow and extracting parallel streams,
> it is likely (although I do not have time to look at it now), that by sorting the traffic by src, dest, and time, one can probably
> match up the data fragments.  However there is a catch here.  It would hard to do this on the fly as the packets may be split over 2 or more netflow files (I cut files every 10 minutes) and process each file on the fly for time series rrd's.  It would have to be done in the overnight processing. That is where I currently extract and match parallel streams.  I can look into it in August.
> 
> Connie Logg - Network Analyst - 650-926-2879
> Stanford Linear Accelerator Center
> MS 97; 2575 SandHill Road; Menlo Park CA 94025
> "Happiness is found along the way, not at the end of the road"
> 
> -----Original Message-----
> From: David Moore [mailto:[email-masked]]
> Sent: Tuesday, June 18, 2002 9:26 AM
> To: [email-masked]
> Cc: David Moore; Cottrell, Les; 'K.C. Norseth'; req; Logg, Connie A.;
> colleen
> Subject: Re: [[email-masked]: RE: [ipfix-req] Ipfix
> features]
> 
> On Tue, Jun 18, 2002 at 09:56:53AM -0400, [email-masked] wrote:
> >
> > Does reporting the fragment ID help? There is still an
> > issue with reusing the fragment ID but given the same
> > src/dst and a reasonably close time stamp might be
> > good enough to figure it out after the fact (i.e.
> > combining a fragment flow and a full flow at the collector).
> 
> Well, if you report the ipid on packets w/ MF set or offset != 0,
> that would be sufficient, however it will also create an awful lot
> of exported flows.  A typical tunneling situation with each original
> packet turning into 2 fragments at 1500 bytes and 20 bytes, using
> the ipid will result in one export record for each original packet.
> Which could readily result in hundreds to thousands of new flows
> per second for tunneled traffic of only a couple Mbit/sec.
> 
> >
> > Paul
> >
> > David Moore wrote:
> > >
> > > This may be difficult to get "correct".  I assume correct means
> > > something roughly like:
> > > 1. maintain state to determine what ports apply for a non-first fragmented
> > > packet.
> > > 2. once a fragmented packet's ports are determined, add it into the
> > > flow it would have gone into if it had ports.
> > >
> > > A few comments:
> > >
> > > A separate fragment/ipid timer would be needed to decide when to expire
> > > the state.  RFC 791 recommends 15 seconds, which also seems to match
> > > reasonably well with caida network measurements.
> > >
> > > Fragments may come in any order (in particular linux sends them
> > > from the back of the original packet backwards towards the front).
> > > This means that you may see a fragment w/o port information before
> > > you see the corresponding port information.  So you can't determine
> > > which flow the fragment belongs to until a later point.
> > >
> > > VPN and tunnel's cause a lot of fragmented traffic, which may occur
> > > at high-speed depending on the endpoints, which may result in a
> > > lot of state being needed.  A good garbage collection scheme may
> > > be needed for this (as well as DoS attacks with fragmented packets).
> > >
> > > Personally, I'd also like a flow field saying whether the packet
> > > was fragmented or not.  So fragmented packets to/from host/port &
> > > proto would be counted as a separate flow from non-fragmented
> > > packets to/from host/port & proto.  This would be in addition to
> > > any attempts to correctly assign ports to fragments.
> > >
> > > -- david
> > >
> > > ----- Forwarded message from "Cottrell, Les" <[email-masked]> -----
> > >
> > >   Date: Tue, 11 Jun 2002 15:00:47 -0700
> > >   From: "Cottrell, Les" <[email-masked]>
> > >   Subject: RE: [ipfix-req] Ipfix features
> > >   To: "'K.C. Norseth'" <[email-masked]>, req <[email-masked]>
> > >   Cc: "Logg, Connie A." <[email-masked]>
> > >   X-Mailer: Internet Mail Service (5.5.2653.19)
> > >
> > >   As an example of IP fragmentation see:
> > >
> > >   http://www.slac.stanford.edu/comp/net/netflow/afsexample.html
> > >
> > >   > -----Original Message-----
> > >   > From: K.C. Norseth [mailto:[email-masked]]
> > >   > Sent: Tuesday, June 11, 2002 10:33 AM
> > >   > To: Cottrell, Les; req
> > >   > Cc: Logg, Connie A.
> > >   > Subject: Re: [ipfix-req] Ipfix features
> > >   >
> > >   >
> > >   > Discusssion is healthy. Please elaborate on what you mean.
> > >   >
> > >   > K.C.
> > >   > ----- Original Message -----
> > >   > From: "Cottrell, Les" <[email-masked]>
> > >   > To: "req" <[email-masked]>
> > >   > Cc: "Logg, Connie A." <[email-masked]>
> > >   > Sent: Tuesday, June 11, 2002 11:16 AM
> > >   > Subject: [ipfix-req] Ipfix features
> > >   >
> > >   >
> > >   > | I hope this is not too off-subject, but in future realeases
> > >   > of ipfix
> > >   > | ttols
> > >   > such as Netflow, I would like to see:
> > >   > |
> > >   > | Max window size reported
> > >   > | Correct handling of fragmented packets
> > >   > |
> > >   > | --
> > >   > | Help        mailto:[email-masked] and say
> > >   > "help" in message
> > >   > body
> > >   > | Unsubscribe mailto:[email-masked] and say "unsubscribe
> > >   > | ipfix" in message body
> > >   > | Archive     http://ipfix.doit.wisc.edu/archive/
> > >   > |
> > >   >
> > >
> > >   --
> > >   Help        mailto:[email-masked] and say "help" in message body
> > >   Unsubscribe mailto:[email-masked] and say
> > >   "unsubscribe ipfix" in message body
> > >   Archive     http://ipfix.doit.wisc.edu/archive/
> > >
> > > ----- End forwarded message -----
> > >
> > > --
> > > Help        mailto:[email-masked] and say "help" in message body
> > > Unsubscribe mailto:[email-masked] and say
> > > "unsubscribe ipfix" in message body
> > > Archive     http://ipfix.doit.wisc.edu/archive/

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/