Subject: Re: [ipfix-protocol] PROTO-14: padding.

From: David Moore <[email-masked]>
Date: Thu, 05 Aug 2004 00:57:32 -0700

Hi Sebastion, Benoit,

I like the newer language (included in proto-4), since it fixes a bug
where the collector is unable to differentiate between padding and
legitimate flow records when the flow record size is small.

I'm wondering if it makes sense to change 'shorter than one Flow Data
Record' to something more like 'shorter than one Flow Data Record of the
corresponding Template ID' or 'shorter than any legitimate Flow Data
Record permissible in this FlowSet' or 'shorter than any allowable
Record in this FlowSet'.

-- david

Sebastian Zander <[email-masked]> writes:

> Hi Benoit,
>
> here is my proposal for the header format (change formats for other
> templates as well) and the Template Flowset Padding paragraph.
>
>         0                   1                   2                   3
>         0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |       FlowSet ID = 0          |          Length               |
>        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |       Template ID 1           |         Field Count           |
>        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |        Field Type 1           |         Field Length 1        |
>        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |        Field Type 2           |         Field Length 2        |
>        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |             ...               |              ...              |
>        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |        Field Type N           |         Field Length N        |
>        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |       Template ID 2           |         Field Count           |
>        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |        Field Type 1           |         Field Length 1        |
>        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |        Field Type 2           |         Field Length 2        |
>        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |             ...               |              ...              |
>        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |        Field Type M           |         Field Length M        |
>        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |             ...               |              ...              |
>        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |        Template ID K          |         Field Count           |
>        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |             ...               |              ...              |
>        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |                            Padding (opt)                      |
>        |                              ...			      |
>        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>
> Padding
> <your text>
> Because Template Flowsets are always 4-byte aligned by definition
> padding is only needed in case of other alignments e.g. on 8-byte
> boundaries.
>
> Cheers,
>
> Sebastian
>
> Benoit Claise wrote:
>
>> Hi Sebastian,
>>
>>> Hi Benoit,
>>>
>>> answer below
>>>
>>> Benoit Claise wrote:
>>>
>>>> Hi Sebastian,
>>>>
>>>> Actually this sentence is not part of the proposed text. It was
>>>> just a comment.
>>>> I wanted to express the fact:
>>>>    - Template FlowSet always have "4-byte aligned" boundaries."
>>>>    - Template FlowSet are only sent once with SCTP, TCP
>>>> So I think we don't care too much about the alignment for the
>>>> Template FlowSeet
>>>>
>>>> However, I could understand the need for some text
>>>> clarification. I see three solutions
>>>>
>>>> Solution 1
>>>>
>>>> We don't write down anything for padding & Template
>>>> FlowSet. Basically what I proposed initially. However, I can
>>>> understand the discrepancy between the Template FlowSet and the
>>>> other FlowSets.
>>>>
>>>> Solution 2
>>>>
>>>>  8.3   Template FlowSet Format
>>>>
>>>>        0                   1                   2
>>>> 3         0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7
>>>> 8 9 0 1
>>>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>>> |       FlowSet ID = 0          |          Length               |
>>>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>>> |       Template ID 1           |         Field Count           |
>>>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>>> |        Field Type 1           |         Field Length 1        |
>>>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>>> |        Field Type 2           |         Field Length 2        |
>>>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>>> |             ...               |              ...              |
>>>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>>>       |        Field Type N           |         Field Length N
>>>> |
>>>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>>>       |       Template ID 2           |         Field Count
>>>> |
>>>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>>>       |        Field Type 1           |         Field Length 1
>>>> |
>>>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>>>       |        Field Type 2           |         Field Length 2
>>>> |
>>>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>>> |             ...               |              ...              |
>>>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>>> |        Field Type M           |         Field Length M        |
>>>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>>> |             ...               |              ...              |
>>>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>>> |        Template ID K          |         Field Count           |
>>>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>>> |             ...               |              ...              |
>>>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>>>       |                           Padding                             |
>>>>
>>>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>>> Padding
>>>>           The Exporting Process MAY insert some padding bytes,
>>>>            so that the subsequent FlowSet starts at an aligned boundary.
>>>>            Padding MUST be composed of zero (0) bits.
>>>>            The padding length MUST be shorter
>>>>            than a Template definition with a Field Count value of one.
>>>>            It is important to note that the Length field includes
>>>> the padding bits.
>>>>
>>>> Solution 3
>>>>
>>>> Adding a 4 bytes padding field in the packet format could be
>>>> considered as confusing (see solution 2), as currently, the
>>>> alignement is always 4 bytes.
>>>> So we could add some generic text about padding in the "template
>>>> flowset section", without adding the Padding field in the packet
>>>> format.
>>>>
>>>> What's your feedback?
>>>
>>>
>>>
>>> I find solution 1 confusing because if there is a strong intention
>>> to allow arbitrary padding for templates (other than 4-byte
>>> aligments) than is should be possible for all templates. So IMO
>>> solution 2+3: having
>>> the field and a statement clarifying that it is already 4-byte
>>> aligned. Maybe the field name could be changed from Padding to
>>> Optional Padding to make clear that it is optional (for all
>>> templates)? The size is not 4 bytes but variable which should be
>>> indicated to avoid confusion (same for the other templates).
>> Can you please propose the text and packet format you would like to
>> see.
>  >
>> Regards, Benoit.
>>
>>>
>>> Cheers,
>>>
>>> Sebastian
>>>
>>>> Regards, Benoit.
>>>>
>>>>> Hi Benoit,
>>>>>
>>>>> shouldn't it always read "... starts at a _4-byte_ aligned boundary"?
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Sebastian
>>>>>
>>>>> Benoit Claise wrote:
>>>>>
>>>>>> Dear all,
>>>>>>
>>>>>>  >From the IPFIX protocol specification:
>>>>>>
>>>>>>     14. PROTO-14: Why do we need padding? Should we shift it to MAY?
>>>>>>            limit the size of the padding? Yes
>>>>>>            solution:
>>>>>>            - padding shorter than actual Record Length
>>>>>>            - fill with 0
>>>>>>            - only at end of flowset
>>>>>>            - applies to all flowsets
>>>>>>            - padding is OPTIONAL (MAY) , not RECOMMENDED (SHOULD)
>>>>>>
>>>>>>
>>>>>>  >From the Seoul meeting minutes:
>>>>>>
>>>>>>     - Padding: currently a should for the exporting process, and
>>>>>> a must for
>>>>>>      the collector.  Proposal to have a MAY for the exporting process
>>>>>>
>>>>>> In the current IPFIX protocol draft version 3, the same
>>>>>> definition is applied to the following section:
>>>>>>     8.4   Data FlowSet Format
>>>>>>     8.5.1   Options Template FlowSet Format     8.5.2   Options
>>>>>> Data Record Format
>>>>>>
>>>>>>    Padding
>>>>>>            The Exporting Process SHOULD insert some padding bytes so
>>>>>>            that the subsequent FlowSet starts at a 4-byte aligned
>>>>>>            boundary. It is important to note that the Length field
>>>>>>            includes the padding bits.
>>>>>>
>>>>>> The proposal is to change it to:
>>>>>>
>>>>>>     8.4   Data FlowSet Format
>>>>>>    Padding
>>>>>>            The Exporting Process MAY insert some padding bytes,
>>>>>>            so that the subsequent FlowSet starts at an aligned
>>>>>> boundary.
>>>>>>            Padding MUST be composed of zero (0) bits.
>>>>>>            The padding length MUST be shorter than one Flow Data
>>>>>> Record.
>>>>>>           It is important to note that the Length field includes
>>>>>> the padding bits.
>>>>>>
>>>>>>     8.5.1   Options Template FlowSet Format    Padding
>>>>>>           The Exporting Process MAY insert some padding bytes,
>>>>>>            so that the subsequent FlowSet starts at an aligned
>>>>>> boundary.
>>>>>>            Padding MUST be composed of zero (0) bits.
>>>>>>            The padding length MUST be shorter
>>>>>>            than one Options Template Record. It is important to
>>>>>> note that the
>>>>>>           Length field includes the padding bits.
>>>>>>
>>>>>>     8.5.2   Options Data Record Format
>>>>>>    Padding
>>>>>>           The Exporting Process MAY insert some padding bytes,
>>>>>>            so that the subsequent FlowSet starts at an aligned
>>>>>> boundary.
>>>>>>            Padding MUST be composed of zero (0) bits.
>>>>>>            The padding length MUST be shorter
>>>>>>            than one Options Data Record. It is important to note
>>>>>> that the
>>>>>>           Length field includes the padding bits.
>>>>>>
>>>>>>  I think we are OK with Template FlowSet, as it will always have
>>>>>> "4-byte aligned" boundaries.
>>>>>>
>>>>>> Feedback?
>>>>>>
>>>>>> Regards, Benoit.
>>>>>>
>>>>>
>>>>
>>>
>>
>
> -- 
> Sebastian Zander
> http://caia.swin.edu.au
>
>
> --
> Help        mailto:[email-masked] and say "help" in message body
> Unsubscribe mailto:[email-masked] and say
> "unsubscribe ipfix" in message body
> Archive     http://ipfix.doit.wisc.edu/archive/

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: Benoit Claise <[email-masked]>
Date: Fri, 06 Aug 2004 01:27:03 +0200

David,

That makes sense to me.

Regards, Benoit.

---

From: Sebastian Zander <[email-masked]>
Date: Fri, 06 Aug 2004 11:09:42 +1000

Makes it more precise. I like the first proposal.

Should we also change the wording for templates to "padding length must 
be shorter than the shortest possible template (with one field)"? Or 
rely on the fact that a 0 length template is obviously padding?

Cheers,

Sebastian

Benoit Claise wrote:

>>>>>> Padding                             |

-- 
Sebastian Zander
http://caia.swin.edu.au

---

From: David Moore <[email-masked]>
Date: Thu, 05 Aug 2004 22:19:42 -0700

Nevil Brownlee <[email-masked]> writes:

> Hang on a minute --
>
> Are the padding bytes included in the set length?  I think they
> should be, so that a collector can walk down the IPFIX message 
> and be sure where each Flow Record starts.

The length is for the entire FlowSet including any padding bytes.

> Given that they are, there's no doubt about where the last record in
> the set ends - it's the point where another Flow Record would
> extend past the end of the Flow record.

If my flow records are say 1 byte long (ie. I've asked for TTL only, and
have set my timeout to 0), then there is no way to differentiate zero
padding at the end from a sequence of 0 TTL packets, unless there is a
requirement that padding must be shorter than the size of a valid record
in this flowset.

Even for longer flow records, the padding must be bounded to prevent
misinterpretation.  The other choice is to provide a length/count of how
many valid entries are present.

> And, given that, there's no need to say anything at all about the
> padding bytes.

If you mean that nothing needs to be said about them being filled as
0-bits, I'd agree, although I think that's reasonable to say.  But we do
have to say something about either the amount of padding that can be
added to the end or add another field to the flowset take care of the
problem.

> Also, by the way, I assume that the lengths specified in a Template
> Record are fixed for every Flow record generated by that Template?
>
> Summary:  I suggest we change the 'length' field definitions for
> each of the Sets, to make it clear the length includes any padding
> bytes.

Hmm, I thought it already made that clear.  My only concern is that
whatever is done with padding must be prevented from being
misinterpreted by a collector as potential data.

-- david

---

From: Stewart Bryant <[email-masked]>
Date: Mon, 09 Aug 2004 11:40:31 +0100

>0-bits, I'd agree, although I think that's reasonable to say.
>

If you don't specify the value you get a security issue -

- Implementations that just export junk, are usually exporting
  the contents of memory which is a security risk.

- Implementations that can chose the value have a covert
  channel that may be exploited.

It is therefore good practise to explicitly specify the padding
value.

- Stewart