Subject: RE: [Forces-protocol] RE: GET/SET in one msg ?

From: "Khosravi, Hormuzd M" <[email-masked]>
Date: Sun, 17 Oct 2004 21:15:29 -0700

-----Original Message-----
From: Joel M. Halpern [mailto:[email-masked]] 

PS: On the structuring of messages and operations, I can live with
creating 
two kinds of operations, one type for query operations, used in query 
messages, and one type for manipulation  operations, used in config 
messages.  Clearly, we want the structures to be the same, even if we do

use two message types.

[HK] Agreed, seems like we have a concensus on this issue now
(CONFIG-REQUEST, CONFIG-RESPONSE, QUERY-REQUEST, QUERY-RESPONSE msgs).

Jamal, can we close on this one now for the protocol? Think you said
yes, but just want to confirm.

Thanks
Hormuzd

_______________________________________________
Forces-protocol mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/forces-protocol

---

From: "Weiming Wang" <[email-masked]>
Date: Mon, 18 Oct 2004 20:07:53 +0800

I'm glad to see this.  Let's just move one issue forward.

Weiming

----- Original Message -----
From: "Khosravi, Hormuzd M" <[email-masked]>

Thanks
Hormuzd

---

From: "Weiming Wang" <[email-masked]>
Date: Mon, 18 Oct 2004 20:07:53 +0800

Weiming

----- Original Message -----

Thanks
Hormuzd

---

From: Jamal Hadi Salim <[email-masked]>
Date: 18 Oct 2004 08:32:14 -0400

Welcome back Weiming. I have updated the text with the query/response.
The only outstanding issue is 6.7. Please weigh in.
I think we are ready top start making updates.

cheers,
jamal

---

From: "Weiming Wang" <[email-masked]>
Date: Mon, 18 Oct 2004 22:20:28 +0800

Hi Jamal,

main hdr (eg type = config)
     |
     |
     +--- T = LFBselect
     |        |
     |        +-- LFBCLASSID = target LFB class
     |        |
     |        |
     |        +-- LFBInstance = target LFB instance

[Weiming] The more I'm thinking, the more I see the value to address multipul
LFB instances here (I can now live with single LFB class). To speak of this, I
have an aspire  to show my yesterday experience with my GRMP test platform
(sorry I have to mention it). As you know, GRMP  does not support multipul LFB
instance addressing.  Yesterday, we had to prepare a show of the platform to
guests from our sponsors. Before the show, we spent near one hour to operate on
the menu to construct a scenario, in which there were 5 output port, 5
associated schedulers (LFBs), and several other LFBs that have many instances.
unfortunately, we faced a problem with one of the machine. Then we had to do it
again.  At that time, I had a VERY VERY strong desire that batch configuration
based on multipul LFB isntance addressing can be used.

I can see very simple scheme to include multipul instances here (by ranging, or
by enum, or by both). Definitely, I believe this will greatly improve our
protocol.

I sincerely hope this be considered seriously by gentlemen.

Best regards,
Weiming

     |        |
     |        |
     |        +-- T = operation { ADD, DEL, GET, etc}
     |        |   |
     |        |   +--  // one or more path targets
     |        |        // under discussion
     |        |
     |        +-- T = operation { ADD, DEL, GET, etc}
     |        |   |
     |        |   +--  // one or more path targets
     |        |        // under discussion
     |        |
     |        +-- T = operation { ADD, DEL, GET, etc}
     |        |   |
     |        |   +--  // one or more path targets
     |        |        // under discussion
     |        |

In other words: Very similar to the way we have it already except
the naming has changed and we can target multiple
operations and multiple paths in an LFB instance
----- Original Message -----
From: "Jamal Hadi Salim" <[email-masked]>

---

From: Zsolt Haraszti <[email-masked]>
Date: Mon, 18 Oct 2004 11:56:50 -0400

Attached is our proposal for data encoding over the wire
for ForCES.

This initial description has a blank chapter on ARRAY (table) encoding. 
We do have a proposal for that but we still need to
write it down (hope to do so today), but wanted to air the
rest of the proposal ASAP.

As always, eager to get bombarded with your comments.

Cheers,
Zsolt
-- 
Zsolt Haraszti                Phone:  +1 919-765-0027/2017
Modular Networks              Mobile:      +1 919-522-2337
                              Email:  [email-masked]
_______________________________________________
Forces-protocol mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/forces-protocol

---

From: Steven Blake <[email-masked]>
Date: Mon, 18 Oct 2004 11:24:49 -0400

On Mon, 2004-10-18 at 08:32, Jamal Hadi Salim wrote:

> 6.2 Query Messages
> -------------------
> 
> 6.2.1 Query message 
> -------------------
>
> main hdr (eg type = Query)
>      |
>      |
>      +--- T = LFBselect
>      |        |
>      |        +-- T = GET
> The LFB is targeted. So if you need to find a route
> from an LPM you target that LFB. If you need to find 
> neighbors of an FE you target the FE object LFB.
>
> The response is exactly the same as above with type = Query-response.
>
> 6.3 Configuration Message
> --------------------------
> 
> 6.3.1 Config request
> ----------------------
> 

If people agree that the protocol should preclude GET operations in
CONFIG_REQUEST messages, and only allow them in QUERY_REQUEST messages,
then the latter BNF needs to be edited to remove GET from the operation
list.  I was assuming that the GET would be encoded as an operation TLV
in the QUERY_REQUEST message.

Regards,

// Steve

---

From: Alan DeKok <[email-masked]>
Date: Mon, 18 Oct 2004 13:21:12 -0400

Zsolt Haraszti wrote:

> - It is assumed that the type of the data can be inferred by the
>   context in which data is used.  Hence, data will not include its
>   type information. 

   It may be useful to also be able to explicitly describe a type, if 
only for initialization purposes.  This ensures that the FE can 
communicate implementation-specific extensions to the CE, and that the 
CE can understand them.

   e.g. "ACL is array[n] of struct {ip, ip, proto, port, port}"

   These types SHOULD NOT be used outside of initialization, as they 
will make the protocol complex, and the implementation problematic.

> - Random sub-element access will not be guaranteed.  Specifically, random
>   sub-element access will not be provided when the sub-elements inlcude
>   one or more sub-elements with variable-size encoding (there are ARRAYs
>   and STRING[N]'s.

   I would phrase that as "random sub-element access is provided only 
when all sub-elements are of the same, fixed, size".  It means the same, 
but phrasing it in a positive way means that you're defining what's OK. 
  By definition, then, everything else is not OK, and you don't have to 
enumerate the infinite weird ways that can't be used for access.

> - The length of the encoded data block will always be a multiple of 32-bit
>   words.

   <nods violently>

...
> 	PADDING := 1, 2 or 3 bytes of 0x00 values to round up the resulting
> 		   data block to the next 32-bit boundary.

   I would say "0, 1, 2, or 3 bytes..."

   You've defined "DATA := PACK(type, value) + PADDING", so there should 
be some provision for PADDING to be zero.

> PACK(type, integer) :=	1/2/4/8 bytes of binary data in network
> 		 	byte order (big endian).

   I would say "network byte order", rather than "big endian".  It 
avoids terminology from "religious wars".

> PACK(type, float) :=	4/8 bytes of binary data in network byte
> 			order, using IEEE floating point numbers
> 			encoding.

   I think the reference is "ANSI/IEEE Standard 754-1985"

> 4.1.3 Encoding of STRING[N] Types
> 
> If type is STRING[N], its encoding is as follows:
> 
> PACK(STRING[N], string) := LEN + STRING + PADDING

   Where "+" is "concatenation".

> where:
>    
>    	LEN := 2 byte string length indicator encoded as uint16
> 	       (big endian).  Length includes only the consecutive
> 	       non-zero characters from the start of the string.

   Can strings contain embedded NULs?

> Examples:
> 
>    	string = "abcde", N = 16:   00 05 61 62
> 				    63 64 65 00
> 
> 	string = "abcde", N = 128:  same as above, encoding does not care
> 				    about max. size of string.

   I'm not sure I see why it is necessary to allow NUL terminated 
strings which have many, many, NULs at the end.  Or, why is "N=128" here?

(byte arrays)
> Note that the four sub-32-bit integer types (INT8, UINT8, INT16 and UINT16)
> preserve they sizes in PACK().  All other atomic types are packed
            ^^^^
   "their" (and other miscellaneous typos)

> 4.2.1  Struct Encoding
> 
> Struct encoding is rather straight-forward.  For struct with fixed-size
> content the encoding follows the ANSI C struct representation
> standards.

   What does that mean?

> If type is STRUCT (more precisely a derived type using the STRUCT construct),
> its content is encoded using the following rules:
...
> 2  Each sub-element (field) is pre-packed with its respective PACK
>    operation.

   What does that mean?

   I think you mean to say something like "the packing of the structure 
is the simple concatenation of the packing of it's elements".

   e.g.

   PACK(struct(foo)) = PACK(foo.element1) + PACK(foo.element2) ...

   If that's true, then it will be possible to write a function which 
takes a ForCES STRUCT definition and a pointer to a C structure 
containing the relevant data, and return a PACKed DATA field.  While 
this process often won't be necessary, it may be useful to give a sample 
function in pseudo-code, or C, to do this packing.  That way people 
implementing the protocol have a "known good" place to start from.

> 4  If a field is one of the 8-bit or 16-bit integer types, it is placed in the
>    resulting data block aligned on its natural length.  For example,
>    if the STRUCT includes a UINT16 field, the 2 bytes of a PACK(UINT16, int)
>    will start on an even byte offset inside PACK(STRUCT, struct element).

   Systems which have difficulties addressing byte-aligned data may not 
like this style of packing.  Aligning everything on 32-bit boundaries 
may not be efficient, though.

> 5  PACK-ed fields of 32-bit and larger sub-elements will be 32-bit
>    aligned.
> 
> 6  If alignment requires padding, 1, 2 or 3 all-zero bytes will be appended
>    to the already encoded block until the required alignment is achieved.

   I would say "zero, one, two, or three bytes of zero will be appended 
to the encoded block, to align it to a 32-bit boundary".

> Example:
>    STRUCT {
>    	UINT8		A;
> 	INT16		B;

   It would be great to have some sample code which read in simple 
STRUCT definitions, and produced the packed DATA.  Having a function to 
produce ASCII art would be even better...

> 4.2.2  Encoding of Array Types
> 
> This is the only non-trivial type in terms of encoding.  Before presenting
> the encoding format and rules, let's state some requirements:
> 
> - Number of elements must be specified in a header
> - If the type of the entries are not of fixed size, the encoded entries
>   will have variable sizes, so the size of each entry must be provided.

   Variable-sized entries are severely problematic.

   Is there a requirement to have arrays of variable sized elements?  To 
me, that sounds more like a big STRUCT.

   Alan DeKok.

---

From: Zsolt Haraszti <[email-masked]>
Date: Mon, 18 Oct 2004 15:00:12 -0400

Alan,

Thanks for your quick but rather exhaustive comments.
See some feedback below.

> will make the protocol complex, and the implementation problematic.

I agree that this could be useful.  We shall find a way to
accommodate this in the protocol somewhere.

> enumerate the infinite weird ways that can't be used for access.

OK

> be some provision for PADDING to be zero.

Right.

> avoids terminology from "religious wars".

OK, we can delete the "(big endian)"

>    I think the reference is "ANSI/IEEE Standard 754-1985"

I will check (or let me know when you know for sure :).

>    Where "+" is "concatenation".

That's right.

>    Can strings contain embedded NULs?

My suggestion would be to follow C string conventions.
So the first NUL and anything after it is ignored.

And naturally, only the initial non-NUL content is carried
over the wire in ForCES messages.

> strings which have many, many, NULs at the end.  Or, why is "N=128" here?

There may some misunderstanding here, so let me rewind a bit.

N in STRING[N] defines that max size of the string _in the model_.
This is analogous to the

	char 	if_name[16]

C construct (as opposed to the char *if_name).

What the above two examples illustrate is that max size N does
not really matter when sending the content over the wire (the
content still must be smaller then N), because the string
encoding will only transfer the useful part of the string,
i.e., everything till the terminating '\0'.

If people have problem with this we can define alternative string
model(s), but I thought this is sufficient.

> > content the encoding follows the ANSI C struct representation

I hoped this should be clear from the rest of the text, but apparently
not.

Let me try to clarify it a bit:  Suppose you define a STRUCT A data type
per ForCES data model.  Suppose furthermore that you implement that
struct in the FE/CE software by means of a C struct: struct A.  The C
struct will have a certain memory layout in the host memory.  That 
layout is what I referred to as "ANSI C struct representation" (I may
have used the wrong reference here; I shall look it up to be sure).

The statement above is that for structs which do not have variable
size fields, the ForCES encoded STRUCT will have the same layout as
the C struct in memory (minus endianness).  The obvious advantage of
this is that you can memcpy() between the message buffer and you
C struct.

>    PACK(struct(foo)) = PACK(foo.element1) + PACK(foo.element2) ...

Almost, but not quite.  Since we want to make this layout compatible
with C implementations, we need to include padding between some of the
elements.  The stated alignment rules serve this purpose.

The formula would be more precise if stated like this:

PACK(STRUCT, foo) = PACK(..., foo.field1) + PADDING1 +
                    PACK(..., foo.field2) + PADDING2 +
                    ...

where PADDINGn is k = 0, 1, 2, or 3 bytes of bytes, where k is
chosen to make the next sub-element be positioned at its required
alignment.

I believe the provided example illustrates the rules well.

> implementing the protocol have a "known good" place to start from.

Are you suggesting putting a pseudo-code into the description
here or into the final standard (or both)?

> may not be efficient, though.

One extreme would be to densely pack, not caring with alignments
at all.  The other would be to align everything on 32-bit boundaries,
and align the 64-bit float and int types on 64-bit.

A sweet spot is to align 32-bit or larger items on 32-bit, and
smaller items on their own size.  This follows the alignment rules of
C structs, which should be a huge merit.
 
> to the encoded block, to align it to a 32-bit boundary".

Again, not always 32-bit.  A uint16 or int16 will be aligned to the
next 16-bit boundary.

> produce ASCII art would be even better...

You mean "STRUCT definition and actual value, and produced the packed
DATA"?                      ^^^^^^^^^^^^^^^^

We can have a contest on who could post that program first.
Input: a) an the XML data specification per current ForCES model
          document
       b) value assignment for the fields
Output: an ASCII art showing the packet data.

>    Variable-sized entries are severely problematic.
I agree that they are more trouble-some.  But I find it too restrictive
to prohibit them.  Arrays of arrays will have this.  Or, if your array
is based on a struct that has string field(s), will have this problem.

This is one of those places where I would think good designs will avoid
such ARRAYs as much as possible, but in some cases it cannot be avoided
so we must support it in the protocol.

/zsolt

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 18 Oct 2004 15:19:43 -0400

Note that an array of structs, where the struct itself contains an array 
(or an array of arrays, which is our equivalent of a two dimensional array) 
is an array with variable sized components.

Note that with the encoding defined, if an array starts with its array 
count one can parse the array sequentially and find all the boundaries even 
if the elements are variable length.  The inner length (which I may have 
suggested) just makes decomposition much cleaner.

I did find one aspect of the description confusing, but technically 
correct.  The packing of 8 and 16 bit integers is defined without 
packing.  You do this so you can use that encoding in the building of 
structs.  However, the encoding of single 8 or 16 bit integers will require 
padding.  (And likely, an array of 8 bit integers should probably be padded 
so that iterative array processing is clean.  (After all, we have STRING 
and BYTES for the common packed cases.

Yours,
Joel M. Halpern

---

From: Alan DeKok <[email-masked]>
Date: Mon, 18 Oct 2004 15:46:00 -0400

> I will check (or let me know when you know for sure :).

http://grouper.ieee.org/groups/754/

http://stevehollasch.com/cgindex/coding/ieeefloat.html

   Yup.

> C construct (as opposed to the char *if_name).

   Ah, OK.  Having the protocol avoid the zeros is nice, but it means 
that many structs are now packed to variable-sized data.

> have used the wrong reference here; I shall look it up to be sure).

   The problem is that ANSI C struct representation is 
platform-specific.  Some platforms can pack multiple 'uint8_t' into a 
32-bit word.  Others can't.

   As an extreme example (not applicable here), big/little-endian 
systems pack bit arrays differently.  So I wouldn't depend on anything 
other than order, and maybe 32-bit alignment, for platform-independent C 
structs.  This makes mapping ForCES protocol packed data into C structs 
somewhat problematic.

> this is that you can memcpy() between the message buffer and you

   I would suggest adding a note that implementors are STRONGLY 
CAUTIONED to validate that the sizes of the structs are the same, and 
that the padding is the same between ForCES and the C compiler.  This 
kind of "memcpy" of complex structures has historically been open to 
implementation flaws and security attacks.

> elements.  The stated alignment rules serve this purpose.

   OK.  It's just that if you have variable rules for padding, the PACK 
function can't really be recursive, as it varies depending on whether or 
not it's in a struct, or not.

> I believe the provided example illustrates the rules well.

   Yes.

> here or into the final standard (or both)?

   Yes.  Code will clarify a LOT of issues.

> You mean "STRUCT definition and actual value, and produced the packed

   Sure.

> Output: an ASCII art showing the packet data.

   Hmm... I don't know if I'll have time.  But it would be extremely 
useful, even for other RFC's containing ASCII art.

> is based on a struct that has string field(s), will have this problem.

   How about having a packed length for structures?  That is, reserve a 
32-bit word at the start of the struct for it's length, PACK the struct, 
and then update the length field with the resulting value.

   For structs of variable size, this should make it easy to quickly 
find the N'th element in an array of those structs.  The clearest 
benefit is that you don't have to parse or even know the entries of the 
struct, in order to find the N'th element.

> so we must support it in the protocol.

   I agree.  I would, however, like to see the difference between 
fixed-size structs && variable-size structs highlighted.

   The problem is that if a data structure has a sub-structure 
*anywhere* in it which contains a STRING[N], then the entire data 
structure is "tainted", and it's size is variable instead of fixed.

   If we expect that most STRUCTs will contain STRING[N], then it may be 
reasonable to assume that ALL STRUCT packing includes a 32-bit prefix of 
the size of the struct, as this will decrease the complexity of the parser.

   If STRING[N] is the only variable-sized data type in the protocol, 
then it may be reasonable instead to separate the string packing from 
the struct packing.  i.e. Rather than packing the string in place, pack 
all of the strings together, and pack a "pointer" in the struct to the 
string.

   I'm not sure I like that idea, though, as packing will require 
multiple passes to get all of the cross-references correct.

   Alan DeKok.

---

From: Zsolt Haraszti <[email-masked]>
Date: Mon, 18 Oct 2004 15:51:15 -0400

> is an array with variable sized components.

Precisely.

> if the elements are variable length.
 
Yes, ineed.

>   The inner length (which I may have 

Well, we have pondered on this too.  Three pieces of info will be needed
to decode an array (and things that may come after the array):  the
total size of the array, the size of each entry, and the number of
entries.  Since there is a redundancy in the triplet, you don't need
to explicitly encode all these.

Entry size is needed if the entries have variable size, so we must
include this in the encoding, though (as will be described in the
text that we are working on) this info can be omitted in cases where
the size can be inferred from the type (means fixed type).

Total size and number of entries are redundant info.  Total size
helps quickly skipping the array (or passing the array block in its
entirety to a decoder), number of entries are useful if your
implementation needs to reserve memory.  We picked the latter,
but the former would work too.  We can also do both, but that
will make the ARRAY header bigger.

> structs.

Yes, indeed.

> However, the encoding of single 8 or 16 bit integers will require 
> padding.

Yes, but this is handled at the higher level in the current description:
If the int8 is all your data, then the top-level definition will pad it
to 32-bit.

If int8 is part of the struct, then padding entirely depends on what
is before and what comes after this field in the struct.

>   (And likely, an array of 8 bit integers should probably be padded 
> so that iterative array processing is clean.

Well, the proposal will allow tight packing of 8- and 16-bit integers,
i.e., with no padding.  This is for two reasons:
1. Byte efficiency over the wire
2. Encoding efficiency.  When you have an array of the form:

	typedef uint8 dscp_map_table[64];

   then a tight-packed encoding allows you to memcpy() the table when
   sending it over the wire.

> (After all, we have STRING 

Keep in mind that although BYTES can provide a simple mechanism to
define int8/uint8 tables, it does not provide the same semantics as
ARRAY.  Specifically it does not allow access to individual entries.

BYTES[N] are really good for things like MAC address (BYTE[6]
mac_address) and IPv6 address (BYTE[16] ipv6_address), or for
very small tables where you don't want to mess with indexing.

In addition, it does not cater for uint16/int16 arrays.

/zsolt

---

From: Zsolt Haraszti <[email-masked]>
Date: Mon, 18 Oct 2004 16:45:28 -0400

> http://stevehollasch.com/cgindex/coding/ieeefloat.html

Thanks.

> that many structs are now packed to variable-sized data.

Yes, its a trade-off.  You can use BYTES[N] instead of a STRING[N]
and then the full buffer will always be sent.  You sacrifice
bandwidth, but you have fixed-size.

I can see value in both.

> 32-bit word.  Others can't.

Can you give me an example of a platform where it is not done?

> systems pack bit arrays differently.

Actually, I checked the same simple C test code both on my Intel
and PowerPC hosts, and found that the layout was the same on
both, albeit the bytes ordering _inside_ 32-bit and 16-bit integer
fields were naturally the opposite.

Take, for example the following struct definition and field values:

struct t_t {
	char 	a;
	int	b;
	char	c;
	short	d;
} t = { 0x11, 0x22334455, 0x66, 0x7788 };

Here is what the memory layout looks like on an Intel P and PowerPC.
In both cases I reset the memory with 0's so that the padding is
obvious.

	Intel:				PowerPC:
	------------			--------------
	11000000			11000000
	55443322			22334455
	66008877			66007788
	Total length: 12 bytes		Total length: 12 bytes

I would be happy to see the result of this test program on other
architectures (I attached the C source file).

> So I wouldn't depend on anything 

OK, lets say I was wrong, and the memory layout is (very)
platform-specific.  E.g., 64-bit platforms do it differently
than others.  That means we cannot come up with a layout that
pleases everyone (every platform).  What shall we do then?

One approach is to screw everyone, but make it a clean design.
E.g., no alignment at all, i.e., only 8-bit alignment.

Another approach is to try to be nice to pre-dominant platforms,
and feel sorry for the rest.

Sure.

> not it's in a struct, or not.

That is why padding is defined at level[n+1] and not at level[n].
At level[n] you pack your data as its own type dictates.  If it is
uint8, it will be a byte, if it is uint32, it will be 4 bytes.
Then at level[n+1] it will be determined if and how much additional
padding is required.
 
> struct, in order to find the N'th element.

This is another trade-off issue.  You are absolutely right, inserting
the size info in front of each struct can be very useful, especially if
it is a variable size struct.  We will have this in ARRAYs, not
part of the struct itself, but part of the ARRAY encoding.
That does not help when the STRUCT is embedded in another struct though.

> structure is "tainted", and it's size is variable instead of fixed.

Yes, that's right.

> the size of the struct, as this will decrease the complexity of the parser.

My hunch is that we will see STRING[N] more in "informational"
attributes that are read infrequently (e.g., name of interfaces), and
see STRING[N] very infrequently in data that will be touched upon at
high frequencies (e.g., classifier table entries, FIB, etc.).
  
> all of the strings together, and pack a "pointer" in the struct to the 

It's not the only.  ARRAY in ARRAY is a more important one.

> multiple passes to get all of the cross-references correct.

Yeah, we thought about that venue too, but rejected it for the same
reason.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 18 Oct 2004 16:58:18 -0400

Why not just declare that structs start with their length.  Then we can 
declare that an array consists of its element count (or length and element 
count) followed by the representation of its contents.  [Ignoring for the 
moment the issue of dense vs sparse packing, subscript representation, etc.]

Yours,
Joel

---

From: Jamal Hadi Salim <[email-masked]>
Date: 18 Oct 2004 17:09:41 -0400

I guess i am gonna have to mention what i refered to as the ugly scheme
we use after all ;->
Beside it being ugly it _works_  regardless of wheterh you have 
tables of tables of tables and you wanted to reference that 10th index
3rd element;->

Every thing is a TLV.
The T is the ID as provided by the XML doc.

The disadvantage is clearly the abuse of bits. You could play tricks to
improve this of course. 
The advantage is that this is a well know interface.

You could go one step and say TLVs would only apply for variable sized 
items only; which means any struct will have to have _atomic_ attributes
first then variable sized at the end of it.

cheers,
jamal

---

From: Jamal Hadi Salim <[email-masked]>
Date: 18 Oct 2004 17:11:41 -0400

On Mon, 2004-10-18 at 11:24, Steven Blake wrote:
[..]
> in the QUERY_REQUEST message.

Fixed. 

cheers,
jamal

---

From: Jamal Hadi Salim <[email-masked]>
Date: 18 Oct 2004 17:14:20 -0400

So far you are the second person who has shown desire for this. I was
the other person; both of us are driven by implementation experience.
How about we just keep it as a note in the draft for now (for progress
reasons)?
Hopefully implementation experience will show the error of whats being
proposed right now, then we can come back and fix it?

cheers,
jamal

---

From: Zsolt Haraszti <[email-masked]>
Date: Mon, 18 Oct 2004 17:30:51 -0400

> moment the issue of dense vs sparse packing, subscript representation, etc.]

That's because when the struct is of fixed size, it is really not
needed and completely destroys even the possibility that you can
memcpy() a large array of these structs.

We may have put too much stress on the efficiency design goal.

But as we are talking to customers and partners, they consistent
concern is efficiency.

Therefore we decided that we try to design the data transfer such
that it scales 1:1 with the data size for large data elements, i.e.,
tables.  If your table is stored on X MBs in memory, it should ideally
require X MBs to transfer it over the wire.

/zsolt

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 18 Oct 2004 17:48:58 -0400

I understand the desire for efficiency.  But using a different encoding for 
the array depending upon the ability to know the size seems like asking for 
trouble.  After all, we are going to need to find a way to include 
additional information anyway.
As for including the size in the struct, if we want it there, and an 
implementor wants the memcpy efficiency, they can always include the size 
in the front of the struct.  (And if they want to generate the structs 
automatically from the  XML, they just need to teach the tool to add the 
size field.)

I want to be careful about letting implementation wrap us around to 
contorted a protocol encoding.

Yours,
Joel

---

From: "Wang,Weiming" <[email-masked]>
Date: Tue, 19 Oct 2004 12:00:10 +0800

Jamal,Hormuzd, and Joel,

I think we have already have the issue as an editorial note as below:

       Editorial Note:
                        1.  Under discussion is, when an 'FE Protocol ...

                        2.  Under discussion is, do we need to support
                            multiple objects addressing at the LFB Type
                            and LFB Instance layer? One simple way to
                            support multiple LFB types or instances is
                            to use TLVs to identify the group of Type
                            IDs and Instance IDs, rather than only one
                            Type and Instance ID.  A range of Instance
                            IDs may also be supported in this way.

Hormuzd and Joel, do you really think it is not the case? I remember Joel
supposed there might be thousands of instances with same LFB calss.  In this
case, if we do not support a range of intance addressing, it actually makes our
protocol unpractical.

regards,
Weiming

---

From: Zsolt Haraszti <[email-masked]>
Date: Tue, 19 Oct 2004 01:30:57 -0400

I suggested this already way back, and now I bring this up again.

First, I firmly believe that we must support multicast (and I think
that is what Weiming is bringing up in his recent emails; if not,
then I still do not understand the issue, so please explain; but the
rest of this email is still valid.).

Second, dividing the LFB address into two components, class ID and
instance ID, and splitting the two into two TLVs have nothing
to do with solving multicast.

I define multicast in the ForCES context as sending the same
configuration requests to a set of LFB instances, with the additional
constraints:
- the (LFB) instances that are targeted are of the same type (class)
- the instances may reside over multiple FEs
- although one or more instances of a certain class maybe target of a
  a multicast request on a certain FE, there may be other instances of
  the same class on the FE that are NOT part of the multicast request.  
  In other words, we must be able to selectively target only a sub-set
  of the instances of the same class on any FE (that's why it's 
  multicast and not broadcast).
- when LFB instances on multiple FEs are part of the same tree, the
  LFB instances on one FE may have a different set of instance
  IDs than LFBs on another FE.

Further observations:
- For all the practical cases where I see this useful, the multicast
  groups may change over time, but rather infrequently.  That is,
  potentially a large number of config requests may be sent to the
  LFBs before the group membership changes.
- In some cases the same LFB instance may be a member of multiple
  multicast groups simultaneously.

Example:

An FE may have 4 instances of IP prefix lookup LFBs,
each serving one channelized physical interface with potentially a
large number of channels, each channel configured as an IP interface.
There may be more than one FEs in the system.  Consider two cases: one
without VRFs and one with VRFs.

When there are no VRFs, the same FIB needs to be loaded into
all the 4 LFBs (to all FEs if you have more than one).  And
all subsequent updates to the FIB must be mirrored to all these
LFBs.

When there are VRFs and the interfaces can use overlapping address
spaces (an all too common case for modern IPv4 edge routers),
then we have a more refined situation.  The CE maintains as many
separate FIBs as many VRFs are in use.  What needs to be loaded
into a given LFB instance depends on the VRF membership of the
interfaces served by the given instance.  That may change when
the interface configurations change, which can be considered
infrequent compared to route changes in the FIB.  A brute-force
solution would be to mirror all VRFs to all LFB instances; this
does not scale very well.  A more desirable solution is to download
selectively the right content to the rigth LFBs.  So in this
case we must be able to selectively form a multicast group for
each VRF and make sure only those LFB instances are members which
serve one or more interfaces that belong to the VRF.  Since
the same LFB instance may serve interfaces of many VRFs, the
LFB must receive the config request of multiple groups.

The proposed model:

Considering all the above, the notion of configurable multicast
groups seems very appealing.  The idea is very similar to multicast
trees in IP networks.  This is what was suggested earlier:

We introduce a stateful multicast group model in which the CE can
configure the FE to define and manage multicast groups.  This
can be done, e.g., via a special attribute of the FE class instance.
The heart of the model is a multicast table attribute in the FE
object, outlined as follows:

MTABLE := ARRAY of MGROUP
MGROUP := {CID, MIID, IID-LIST}
IID-LIST := ARRAY of IID

where:
- CID is an LFB class ID
- IID is an LFB instance ID, valid on the FE
- MIID is a "virtual" instance ID from a pre-defined "multicast" range,
  i.e., it is not the actual IID of any instance on the FE.

The CE can add/remove entries in the MTABLE, creating/removing multicast
groups.

The CE can also add/remove IIDs in the IID-LIST of any existing group,
that is, grow or shrink the membership in a given group.

Once a group is set, all subsequent requests using a CID:MIID
address will be delivered to all LFB instances in the referred group.

Benefits:
- The above schema supports well the case of overlapping memberships.
- Using the same group address across mutiple FEs will enable efficient
  multi-FE multicasts, i.e., the same message can be sent to multiple
  FEs.  This is because the CE can assign an MIID for each multicast
  tree, so a config message with given CID:MIID address in the LFB TLV
  will address the right set of LFB instances on all FEs.

Cheers,
Zsolt

---

From: "Wang,Weiming" <[email-masked]>
Date: Tue, 19 Oct 2004 18:27:12 +0800

Zsolt,

Thank you for your intensive reply.

Yes, what I suggest is trying a multicast config (that is, the data is the same
for all instances) at pure instance layer by use of a very simple and explicit
multipul instance scheme. I understand your scheme for multicast at both FE
layer and instance layer. It's ture defining such a scheme can supply much more
powerful function than purely by use of explicit instances for object
addressing. Actually I think we have considered such scheme during protocol
design (by use of multicast FE id address and LFB isntance id address).I also
agree that the scheme you presented below to use an FE attribute to define
instance multicast group is practical to realize such multicast. What I just see
is, apart from using FEs and Instance multicast address, to use multipul
explicit instances is still valuable for customers. For example, consider there
are two LFBs that have one same attribute data to config, and the attribute data
is quite lenthy in protocol text. Because there is only one message to use,
users may be tired of defining a specific instance multicast group for such only
one time config. But they will definitely  be very tired of repeating the lenthy
protocol text. In this case, by use of multipul instances, I'm sure they will be
very happy.

Thank you.

Weiming

----- Original Message -----
From: "Zsolt Haraszti" <[email-masked]>

> to do with solving multicast.
What I think is to use a TLV to structuraly represent instances, for example, to
use a range structure for a range of instances.

> > > > LFB instances here (I can now live with single LFB class). To speak of
this,
> > I
> > > > (sorry I have to mention it). As you know, GRMP  does not support
multipul
> > LFB
> > > > guests from our sponsors. Before the show, we spent near one hour to
operate
> > on
> > > > unfortunately, we faced a problem with one of the machine. Then we had
to do
> > it
> > > > I can see very simple scheme to include multipul instances here (by
ranging,
> > or

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 19 Oct 2004 09:38:34 -0400

I do think that there may be thousands of instances.
I do not think that this requires multiple addressing.

I think that there are some interesting cases prompted by the possibility 
of very large numbers of LFBs, but they do not drive multiple addressing.

My reasoning is based on the following premise.
Firstly, if there are many LFBinstances s of a class, then it is likely 
that many fields of the LFB instances will be the same, but it is extremely 
unlikely that all fields of the LFB instances will be the same.
The simplest case to understand when one would need to setup all those LFBs 
at once is in a power recovery situation  (the FE lost power, then 
recovered to an empty state.)  The CE needs to send the configuration 
information to the FE.
Secondly, I believe that transaction count is much more important than data 
volume.  The FE is going to have to set every field in every LFB.  Encoding 
is not going to change that.
Given that there are distinct values in each LFB instance, there must be an 
operation against each LFB instance.

The marginal gain from having a single transaction to update the identical 
fields, and then individual transactions for the distinct fields is very 
small.  It does not reduce the number of IOs or the core transaction rate.

If it is desired to optimize for this case, we may want to introduce (in a 
future version of the protocol) some kind of block checkpoint / restart 
mechanism.  The CE would record its full state about the FE, and then ask 
the FE for a dump suitable for restoral of the FE state.  Upon restart, the 
CE would rebuild its state from its stored representation, and would ship 
the dump back to the FE to enable efficient rebuilding there.  I can see 
significant value in such a mechanism.  I do not however see a need to 
include this in the first deliverable of the protocol.

Yours,
Joel M. Halpern

---

From: Alan DeKok <[email-masked]>
Date: Tue, 19 Oct 2004 10:46:59 -0400

> The advantage is that this is a well know interface.

   There are intermediate possibilities.  Rather than putting every 
element of every data structure into a unique TLV, you can have a TLV 
like "LFB-Instance-Data", which contains all of the PACKed information 
that Zsolt is talking about.

   The benefits with this approach are:

- you have one LFB-Instance-Data TLV for all LFB's, rather than multiple 
data-specific TLV's, or multiple LFB-specific LFB's.

- The protocol is more compact, efficient, and easy to parse by things 
that don't understand the internals of LFB configuration.

- you can update the PACKed data and STRUCTs inside of LFB-Instance-Data 
without adding or deleting TLV's.

   I would propose the above scheme, and also a TLV such as 
LFB-Data-Structure, which could hold an encoded representation of the 
STRUCTs and data that the LFB understands, so that the CE can 
"bootstrap" it's knowledge of how to configure the LFB.

   This process could be very powerful.

   Alan DeKok.

---

From: Alan DeKok <[email-masked]>
Date: Tue, 19 Oct 2004 10:51:08 -0400

> Can you give me an example of a platform where it is not done?

   DSP's, for one.  sizeof(char) == sizeof(short) == sizeof(int) == 8.

> fields were naturally the opposite.

   Did you use the same compiler?  Not that that matters much, the 
struct packign should be the same across all compilers on the same 
architecture.

   Maybe my worries are not relevant.  I just remember having problems 
with this sort of thing in the past.

> E.g., no alignment at all, i.e., only 8-bit alignment.

   Yuck.

> and feel sorry for the rest.

   That sounds like a plan.

   Alan DeKok.

---

From: "Wang,Weiming" <[email-masked]>
Date: Wed, 20 Oct 2004 13:56:50 +0800

Joel,

----- Original Message -----
From: "Joel M. Halpern" <[email-masked]>
Subject: Re: [Forces-protocol] RE: GET/SET in one msg ?

> unlikely that all fields of the LFB instances will be the same.
[Weiming]That's just why explicit multipul addressing is needed. For the case of
fields of all LFBinsatnces are the same, we can simply use a broadcast to do so.
Let me try to show a scenario why multipul addressing is needed for different
fields case.

Firstly, because we have assumed the 16bit insatnce Id is not enough, we can
reasonably assmue there is a case where there are 64k or more instances (say
64k). Further, we suppose Id1 to Id32k need to config parameter 1, Id(32k+1) to
Id 64k to set parameter2.

Then, by using single instance addressing, the protocol becomes horrible, either
using one message or using separate messages. I just show the one message case.
The message format would like like:

Msghdr
LFBselect
LFBClassID
Instance ID1
Parameter1
Instance ID2
Parameter1
...
...
Instance ID 32k
Parameter1
Insatnce ID (32k+1)
Parameter2
....
Instance ID 64k
Parameter 2

Remember we then have 64k pair of instance and parameter in the protocol text.
In some cases, I'm just worried this make protocol unpractical.

By using multipul addressing, the only text is as:
Msghdr
LFBselect
LFBClassID
Instance ID1 to ID 32k
Parameter1
Instance ID (32k+1) to ID 64k
Parameter2

By using multicast scheme supposed by Zsolt, I suppose we need following steps:
1. send a FE attribute to FE object to setup a virtualID1 for Instance 1 to
Instance 32k
2. send a FE attribute to FE object to setup a virtualID2 for Instance (32k+1)
to Instance 64k
3. config:
Msghdr
LFBselect
LFBClassID
Virtual ID1
Parameter1
Virtual ID2
Parameter2
4. send a message to release virtual ID1
5. send a message to release virtual ID2

I can see the advatage of above explicit multipul addressing scheme very
clearly.

> recovered to an empty state.)  The CE needs to send the configuration
[Weiming]Yes, that's the one case but not the only.

> is not going to change that.
[Weiming]I'm not sure if you mean for every operation, we should maintain a
transaction count. If is, then I have to say our current one message with
multiple Operation definitions has already be against the requirement. My
opinion is transaction maintenance is moderate, while multicast and multiple
operation are more important.

> operation against each LFB instance.
I don't think multicast will loose operation individuality.
> small.  It does not reduce the number of IOs or the core transaction rate.
[Weiming]At least it saves bits greatly and makes protocol practical. Remember
the capability between CE and FE are quite limited, especially in muli-hop
ForCES application.
> include this in the first deliverable of the protocol.
[Weiming]I suppose this is only for restart case, I can see many cases where
multiple addressing can apply, such as delete, change of LFB parameter, load and
unload of LFBs, etc. And also I think the scheme above is much more complex than
multiple addressing. So, my conclusion is, with a little bit expansion, we can
gain much, then why not we adpot it?

Best regards,
Weiming

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 20 Oct 2004 07:08:57 -0400

Allow me to be a bit pedantic and repetitive please, because apparently my 
main point got lost.

Even when there are large numbers of LFB instances of a given class in an 
FE that need to be set, I have trouble envisioning that they will need to 
be set to the identical values.  Some of the values will be the same.  But 
some will be different.
If at least one value differs, then you need an operation directed at each 
individual LFB Instance.  If this is infeasible, then we have a basic 
problem that multicast at this level will not help.  If this is feasible, 
then multicast at this level seems a refinement that can wait until we know 
if we need it.

Yours,
Joel

PS: We are talking here about multicasting to LFB instances within an 
FE.  Although I am still doubtful about the multi-FE case, I can understand 
that rather better.

---

From: Jamal Hadi Salim <[email-masked]>
Date: 20 Oct 2004 09:08:07 -0400

> be set to the identical values.  Some of the values will be the same.  But 

In anglais:
"route add 10.0.0.0/24 gw 10.0.0.50 on port 10 of FE2";

Consider my case where I have six routing table instances (not VR)
(typically one per MAC) which i would typically want to in most cases 
to populate with the same value (and on rare occasions with different
values).
Note the above makes sense for multi-FE multicast as well but you dont
seem to have issues with that.

cheers,
jamal

---

From: "Weiming Wang" <[email-masked]>
Date: Wed, 20 Oct 2004 21:34:12 +0800

Joel,

> then multicast at this level seems a refinement that can wait until we know
I agree your mean of 'Even when' here, therefore the 'unpractical'  I mean may
not exist. I actually don't think the extremity is a good prove for the benifits
of multiple addressing.  I just can see the great benifits even when there are
100 instances. I think it may be much helpful if you could present some drawback
for such approach. What I can see it is only a very little bit of complexity
added to the instance select field. May be I'm too pedantic :)   Anyway, I don't
think we need to wait more if we find it valuble.

Thank you.
Weiming

---

From: "Wang,Weiming" <[email-masked]>
Date: Thu, 21 Oct 2004 12:48:07 +0800

Hi Jamal, Hormuzd, etc,

To summarize the discussions on multiple instances, I try to propose following
scheme for instance selection, which follows Robert's idea and Jamal's format,
as:

PL level PDU : = MAINHDR<LFBselect>+
LFBselect := LFBCLASSID InsSelect <OPER>+
InsSelect := InstanceID <RangeMark | Instance ID >+
RangeMark := '0xFFFFFFFF'; the value is the same as Broadcast Instance address,
no worry of ambiguity here.

The InsSelect is a TLV, whose structure is shown as:

main hdr (eg type = config)
     |
     |
     +-- T = LFBselect
     |        |
     |        +- LFBCLASSID = target LFB class
     |        |
     |        |
     |        +- T = InsSelect
     |        |   |
     |        |   V = InstanceID <RangeMark | Instance ID >+
     |        |
     |        +- T = operation { ADD, DEL, GET, etc}
     ...

Best Regards,
Weiming

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 21 Oct 2004 08:35:22 -0400

While I am concerned about feature creep in adding this multi-targetting, I 
will stop objecting to it at this point.
If we want to add such, there is no need to use a TLV to represent 
it.  Simply define the instance ID encoding so we can tell ranges from 
individual instances.  Using a TLV for information whose presence and 
position is mandatory does not add much, and can in fact cause problems.

Yours,
Joel

---

From: "Weiming Wang" <[email-masked]>
Date: Thu, 21 Oct 2004 21:46:03 +0800

Hi Joel,

----- Original Message -----
From: "Joel M. Halpern" <[email-masked]>
Subject: Re: Instance Select

> will stop objecting to it at this point.
Could you show why and what's the benifits to defer it. What I see the issue is
it is more a basic one than a value-added one, for it is quite  like a printer
which can only print one page at a time if without multiple selecting.

> individual instances.
Does it mean the coding as:

LFBCLASSID InstanceID <RangeMark | Instance ID >+  <OPER>+ ?

If is, I'm afraid there is a size problem for it, for the InstanceID number
changes and we can not tell an InstanceID with the OPER TLV type field.
If it means using some tag in InstanceID field to indicate the InstanceID and
the Range mark, then I think it will still limit the OPER TLV type definition.
Could you show more on this?

>Using a TLV for information whose presence
The presense is definite, for we at least need one Instance ID.

>and
Joel, I know you are quite experienced and I'm really very much interested in
what you think of the problems here. From my limited experience, I think TLV
followed by TLV approach is a very common PDU scheme for protocol design,  and
will lead to no problems that I can think of. Actually, in our current scheme, I
think we have already used LFBselectTLV followed by many other LFBselectTLV
format. And I also see we will use Path-Data TLV quene inevitably. Maybe I'v
missed something.

Thank you.
Weiming

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 21 Oct 2004 10:02:26 -0400

I think I was wrong in this case.  There are really two different kinds of 
instance targets, and you are right that a TLV is the best way to encode that.
Sorry.
Joel

---

From: Zsolt Haraszti <[email-masked]>
Date: Thu, 21 Oct 2004 14:26:42 -0400

Weiming,

I have a very hard time to resonate with your examples, and therefore
with your reasoning.

For one, if you end up needing 64k LFBs from the same class, I think
you or we did something wrong with the model.  Sure you mean it an
extreme case, but I think it is a misleading one.  I envision
practical LFB models having up to a few hundred LFB instances, where
multiple instances of the same class will be in very different roles
(e.g., a Classifier LFB supporting Diffserv versus a Classifier
LFB supporting IPsec, etc.), hence more often not sharing any config
data than sharing some.

For two, displacing the two parts of the LFB address (class ID and
instance ID) in the protocol is a much bigger concern to me than to
allow ad-hoc/state-less multicast addressing.  It would be in some
sense similar to if in the IP header the (sub-)network address and
host-address part of the IP address were placed in distant offsets.

For three, on state-less versus state-ful multicast:  Your example
below is based on the very extreme case when you have a SINGLE
config message addressed to a VERY LARGE number of LFB instances
of the same class.  I have not yet seen a practical case for this
scenario.  I reiterate that multicast groups are not ad-hoc, and
there will be typically a large number of config updates sent to a
group after the group is formed.

Regards,
Zsolt

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 21 Oct 2004 15:30:38 -0400

While I do not think it is common, I do think that some implementations 
without misusing the model will end up with that many or more LFB instances 
of a single class.
The example I gave before is an FE serving an OC48 which can be subdivided 
for fixed rate ports down to DS0.  You can argue that we should invent a 
multi-port LFB, but that does not seem an inevitable situation.

The odd aspect of this is the if we implement sending the same update to 
multiple LFB instances, it will actually be easier to make some changes if 
the ports are separate LFBs than it will be if they are table entries in a 
single LFB.
Or else we end up needing block selection syntax for SETs, with implicit 
element replication.  Which suggests we are doing something wrong, since 
that would mean 3 levels of multicast replication inside the system.

Yours,
Joel

At 02:26 PM 10/21/2004 -0400, Zsolt Haraszti wrote:
>you or we did something wrong with the model.

---

From: "Wang,Weiming" <[email-masked]>
Date: Fri, 22 Oct 2004 10:19:50 +0800

Zsolt,

----- Original Message -----
From: "Zsolt Haraszti" <[email-masked]>
Subject: Re: [Forces-protocol] RE: GET/SET in one msg ?

> LFB supporting IPsec, etc.), hence more often not sharing any config
I think Joel has expained some. Actually I have no doult agreeing with you that
different instances may have different roles there fore may have different
attribute values, but we can not exclude that there are many cases the
attributes of instances are the same. From my current experiences, I'v already
seen the same attribute values needed for schedulers, queues, ports, policers,
and even classifiers. Note that what I mean the 'same' is not the same for all
data of insatnces, there can be only one attribute data entry that are set the
same, then the value of multiple addressing appears. Even if we have say 10
instances, to repeat the same data for 10 times is already boring thing to do.

> host-address part of the IP address were placed in distant offsets.
I see your thought here, and I actually like this, not at all exclude this. I
just think both the multicast address based mulitple addressing and the explicit
multiple addressing should all be needed. For multicast address based, it's more
fit for those insatnces that have more same data to set and the multicast group
are more permanent comparatively. For explicit based, it's more fit for some
instant use of the same data setup for instances. I try to show an example to
clear this as, say 10 insatnces, 1-5 need to setup same parameter1, 2-6 to set
same parameter2, 3-10 to set p3, 4-6 to set pa4, etc. In this case, to setup
many multicast group is really not applicable, but by using a very handy
explicit muliple addressing, it just makes things very easy.

> group after the group is formed.
I'm not sure if you mean explicit multiple instance addressing is state-less and
defining specific multicat group is stateful. I agree the latter is not ad-hoc.

Thank you.
Weiming

---

From: "Ligang Dong" <[email-masked]>
Date: Fri, 22 Oct 2004 14:51:04 +0800

hi,

The following is my opinion about the debate whether multicast (i.e., multiple addressing) of LFB Instance is required or not. 

(1) In past several months, I am engaged in the implementation of ForCES & GRMP. Till now, a basic prototype is already constructed. Also it has been shown to several guys in this research field. From the view of implementation, the multicast of LFB instance is easy. 
(2) Furthermore, multicast of LFB instance is more efficient than the "unicast" approach and the "virtualID" approach. Therefore, why not adopt multicast of LFB instance in the protocol. 
best regards
Ligang

---

From: Jamal Hadi Salim <[email-masked]>
Date: 22 Oct 2004 07:01:08 -0400

Ligang,

Lets continue this discussion assuming we dont need to update anything
in the draft for now.

Can you explain under which circumstances it was desirable to do
multicast to several instances?
I can give you an example where this would be valuable to me:
Two of the (commodity) ASICs we use have a table per set of ports
ranging from 1-8 ports per table. So I could have anywhere
6-24 LPM tables for example on the same FE - depending on the chip port
enumeration or board layout. To me these are 6-48 instances of the same
(LPM) LFB. when i look at all of them as part of a single NE, I have
done some rough calculation and anywhere between 90-80% of the time
i would send the _same_ table values. The rest of the time they will
vary. 
So to me the mutlicast/broadcast/port range is valuable. I dont think
this is only valuable to me or you (Sorry i missed the reasoning on your
and Weiming part in the email thread), rather these are commodity ASICs
which i expcte to be used a lot to help ForCES become ubiquitous. Folks,
We cant just ignore the fact they exist and hope they will go away.

My opinion, since we are discussing this post draft release:
To bring back old discussions (Apologies in advance), to meet the above
requirements, one would need to split the grammar so we have a Instance
selector level where we could have specific instances within a class; as
well ability to select multiple in one instance selection; i.e something
along the lines of (hopefully diagrams look right):

     +--- T = LFBselect
     |        |    
     |        |
     |        +-- LFBCLASSID = 0x45 
     |        |
     |        +---T = LFBTARGET_UNICAST
     |        |     |
     |        |     +-- LFBInstance = 0x4321
     |        |     |
     |        |     +-- T = ADD
     |        |     |   |
     |        |     |   +--  // path-data 
     |        |     |        
     |        |    
     |        +---T = LFBTARGET_MCAST
     |        |     |
     |        |     +-- LFBInstance = 0x1234
     |        |     |
     |        |     |
     |        |     +-- LFBInstanceMask = 0xf
     |        |     |
     |        |     +-- T = ADD
     |        |     |   |
     |        |     |   +--  // path-data
     |        |     |      
     |        |     |
     |        |    
     |        +---T = LFBTARGET_UNICAST
     |        |     |
     |        |     +-- LFBInstance = 0x5678
     |        |     |
     |        |     +-- T = DEL
     |        |     |   |
     |        |     |   +--  // path-data
     |        |     |      
     |        |     |

Apologies if this is what is being discussed already in thread
involving Steve/Robert/Weiming.
Again, I may have miscontrued your need, but please chip in.

cheers,
jamal

---

From: "Ligang Dong" <[email-masked]>
Date: Mon, 25 Oct 2004 17:08:06 +0800

hi, Jamal, 
Sorry for the delay because of my holiday.
I like your example although I have not used ASIC in my implementation prototype.
My current experiences tell me that multicast addressing can make the message simpler and have better transmission efficiency.

In your design about the multicast, I notice that you use "LFBInstanceMask". It means that you use multicast address. It is obviously an approach. Another approach is to use a list of included LFB instance addresses.

best regards
Ligang

---

From: Jamal Hadi Salim <[email-masked]>
Date: 25 Oct 2004 12:49:56 -0400

Hi Ligang,

> In your design about the multicast, I notice that you use "LFBInstanceMask". It means that you use multicast address. It is obviously an approach. Another approach is to use a list of included LFB…

I realized after i responded to you that the approach you posted is
slightly different, but end goals the same. So far i think there are
three approaches being talked about.
1) Yours
2) Steve/Robert with MIID (whcih unfortunately made it into the draft
before consensus was reached)
3) What i posted 

Maybe someone needs to present at the meeting.

Clearly this issue needs revisiting and we may have to make mods
to the LFBSelect TLV.

cheers,
jamal

---

From: Robert Haas <[email-masked]>
Date: Mon, 25 Oct 2004 22:28:04 +0200

Jamal,

>before consensus was reached)
Sorry, that was not my intent. My understanding was that there was no 
other proposal addressing this issue in particular.

>3) What i posted 
I must have missed that.

>Maybe someone needs to present at the meeting.
Definitely. A short presentation summarizing all the options. I could 
take care of that (probably need 30 min with the discussion).

-- 
Robert Haas
IBM Zurich Research Laboratory
Säumerstrasse 4
CH-8803 Rüschlikon/Switzerland
phone +41-1-724-8698  fax +41-1-724-8578  http://www.zurich.ibm.com/~rha

---

From: "Ligang Dong" <[email-masked]>
Date: Tue, 26 Oct 2004 09:26:52 +0800

hi, 
Jamal, thank you for the summarization about this issue.
Robert, thank you for the presentation about this issue. 
We have the same objective of making the protocol simple and efficient.
best regards
Ligang

---

From: Jamal Hadi Salim <[email-masked]>
Date: 26 Oct 2004 07:17:33 -0400

I concur with Ligang. Robert if you can ask for a slot, I think this
issue needs to be resolved.

cheers,
jamal