Subject: RE: [Forces-protocol] Issue 0: Application addressing

From: "Deleganes, Ellen M" <[email-masked]>
Date: Wed, 10 Mar 2004 16:20:11 -0800

So if you have a proxy that is "fronting" for multiple FEs, what does
the ID contain? If it is the ID of the proxy, how does the proxy know
which FE to configure? I think even in this case the ID should identify
the FE so the proxy knows which FE the CE wishes to configure. If this
is the case FE ID seems to be appropriate nomenclature. 

How the message gets to the proxy should be transparent to the ForCES
protocol. This can be handled by correctly addressing the message (or
putting into the right socket or whatever) at the underlying transport
the proxy ID should not have to be part of the ForCES protocol. This
seems like the connection to a proxy is part of the pre-association
phase and outside of ForCES.

Regards,
Ellen

-----Original Message-----
From: Jamal Hadi Salim [mailto:[email-masked]] 
Sent: Wednesday, March 10, 2004 4:04 PM
To: [email-masked]
Cc: Deleganes, Ellen M; [email-masked]
Subject: Re: [Forces-protocol] Issue 0: Application addressing

On Wed, 2004-03-10 at 18:37, Alex Audu wrote:
> What I am saying is there  should be no ambiguity as to which FE an
> FEID maps to.

There is none. 

> An FEID maps to one and only one FE instance.  There should be no
> other interpretation
> of this FEID.  At least I don't see a use case for what you are
> proposing. May be you can explain some more,..

Actually let me throw that at you instead; you said:

On Wed, 2004-03-10 at 18:45, Alex Audu wrote: 
> A machine that is proxying for multiple FEs should be transparent to
> the protocol. This is 
> important for interoperability. In short, how the message gets routed
> to the FE of interest
> should be transparent to the protocol.
> 

So is this machine an FE, an FE proxy or what?
Do you have multiple FEIDs destined to it or do you have a single one?
The protocol is still using only the ID field (i dislike the FEID naming
for this reason - its an ID). In other words no extra header info is
needed. 

cheers,
jamal

_______________________________________________
Forces-protocol mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/forces-protocol

---

From: Jamal Hadi Salim <[email-masked]>
Date: 10 Mar 2004 19:46:43 -0500

On Wed, 2004-03-10 at 19:20, Deleganes, Ellen M wrote:
> the ID contain? 

I would think there would be a unique ID for each FE being addressed.
Protocol doesnt care who is handling it; 

> If it is the ID of the proxy, how does the proxy know
> which FE to configure? 
> I think even in this case the ID should identify
> the FE so the proxy knows which FE the CE wishes to configure.

Exactly. So message goes say to IP X which is a proxy; but destination
ID refers uniquely to the FE.

>  If this

So far we seem to be saying the address looks like:

ID:=FEID: LFBID: LFBInstanceID

So you are refering to that one portion of the ID i hope?
in which case we may have no issues. 

> protocol. 

It is.

> This can be handled by correctly addressing the message (or
> the proxy ID should not have to be part of the ForCES protocol. 

It is not. Nothing in the Forces header helps identify it as a proxy.

cheers,
jamal

---

From: Alex Audu <[email-masked]>
Date: Thu, 11 Mar 2004 09:42:35 -0600

Hello Jamal,

I think I see the crux of your argument.  So, you route a message to the 
proxy with
an ID (say FEID), and you are wondering how you address the dumb FEs 
that are
being proxied. And you are refering to these dumb FEs as instances of 
the proxy FE.
Im I correct?

But you forget that the proxy and the dumb FEs share the same IP 
address. They
probably have different transport addresses (different ports).  So, lets 
say there are
two dumb FEs (FE1 and FE2) attached to this proxy at address A.  If the 
CE sends
message to FE1, it will be routed to the proxy at address A,  who will 
then forward
it to FE1 based  on the FEID in the message.

The CE will create objects for FE1 and FE2 such that FE1 has  FEID=1 and
IpAddress = A, and FE2 has FEID=2, and IpAddress=A .

I don't see any conflict; do you?

Regards,
Alex.

---

From: Jamal Hadi Salim <[email-masked]>
Date: 11 Mar 2004 15:01:24 -0500

Alex,

On Thu, 2004-03-11 at 10:42, Alex Audu wrote:
> Hello Jamal, 
> 
> I think I see the crux of your argument.  So, you route a message to
> the proxy with 

The point i was trying to make is you can have one end point that
owns several FEIDs. Call it a proxy etc. 

> address. 

It doesnt matter if the "proxy" has one or more IPs and whether
they are multihomed or aliased on the same interface;

> They 
> probably have different transport addresses (different ports).  So,
> lets say there are 
> two dumb FEs (FE1 and FE2) attached to this proxy at address A.  If
> the CE sends 

Now you are suggesting an implementation. Another implementation is they
all go to the same IP transport protocol port and the process muxes them
using pigeons to the correct FE based on FEID etc. 

> I don't see any conflict; do you?

There are no conflicts, but your proposal is implementation specific.
I think we have reached a general understanding on this topic. 

In your other email you say:

On Thu, 2004-03-11 at 12:38, Alex Audu wrote: 
> I agree, ..I don't see any need for GroupFEID, and other such group
> IDs.
> 

I actually wasnt agreeing with what you said above. I think the grouping
of IDs is needed; the mechanics is what i was commenting on.

BTW, momentum is dead. Who is the taker for topic #3? I could take it.
Starting a timer ...

cheers,
jamal

---

From: Alex Audu <[email-masked]>
Date: Thu, 11 Mar 2004 15:43:41 -0600

Issue 3:

We can start by agreeing that Unicast is a MUST implement, and MCAST, 
BCAST a
SHOULD.

Any comments?

As an asside, on the GroupFEID  issue, certainly you can define any 
grouping to
help the application accomplish its goal, but we don't want that bubled 
up to the
protocol addressing structure.

Regards,
Alex.

---

From: Patrick Droz <[email-masked]>
Date: Fri, 12 Mar 2004 11:30:54 +0100

Hi Alex,

why not take the same approach that OSPF is taking.
Unicast a MUST
Mcast/Bcast a must if available.

Regards,
Patrick

-- 
   Dr. Patrick Droz 			| [email-masked]
   IBM Zurich Research Laboratory 	| http://www.zurich.ibm.com/~dro
   Saumerstrasse 4 			| Tel. +41-1-724-85-25 CH-8803
   Rueschlikon/Switzerland 		| Fax. +41-1-724-85-78

---

From: Jamal Hadi Salim <[email-masked]>
Date: 12 Mar 2004 06:13:00 -0500

Alex,
Sorry I slowed down a little there.   

> We can start by agreeing that Unicast is a MUST implement, and MCAST,

Well I like the AD suggestions, which to quote David:

---
"dual mandatory" approach.  I.e.:

If your underlying interconnect supports multicast,
then you MUST implement the following (multicast)
method of communication.

If your underlying interconnect supports unicast, then
you MUST implement the following (unicast) method
of communication.

This will guarantee interop, although at a potential
extra cost (cost being having to potentially support
both when both are present).  While this isn't optimal,
it does at least give some flexibility.
-------

In my opinion multicast should be used only within a 
"close proximity" only.

cheers,
jamal

---

From: Jamal Hadi Salim <[email-masked]>
Date: 12 Mar 2004 06:30:28 -0500

On Thu, 2004-03-11 at 16:43, Alex Audu wrote:
> help the application accomplish its goal, but we don't want that
> bubled up to the

I dont disagree with your general sentiment. I think it may end up being
tricky to do it right without adding a little tweak. The ways i see this
done is:

a) such a ID is known apriori; we saw the broadcast group for the three
address pieces listed as this (0xffff)

b) such ID is configured on all devices via FEM/CEM interface.
The view ive been hearing is to avoid that interface whenever possible.
My belief is that this is a configuration issue which belongs to the
FEM/CEM interface. Otherwise you are only left with c).

c) Part of the async capability exchange. Which can be done to not
complicate the protocol.

You can repsond to this topic or defer it (on my part this is the last
response for now) - i think on topic 0 we are there and this is a
detail; lets use our energies on the other issues.

cheers,
jamal

---

From: jeff pickering <[email-masked]>
Date: Fri, 12 Mar 2004 09:35:30 -0500

I agree on the dual must and think it will save a lot of interoperability
problems down the road. I also like the close proximity idea (although
it could be seen to contradict the dual must) because it could save us from
doing something silly about multicast reliability just to address the 
"not close"
case.

Jeff

---

From: [email-masked]
Date: Fri, 12 Mar 2004 23:52:17 +0800

Hi Jamal,

> In my opinion multicast should be used only within a 

[Weiming Re] That's very good point, I think. To separately considier the Issue 
3 for the close proximity and for the remote multihops deployment may be useful 
to resolve this Issue.

Yours,
Weiming

-------------------------------------------------
This mail sent through HUCNIC Webmail system.

---

From: Jamal Hadi Salim <[email-masked]>
Date: 12 Mar 2004 11:23:09 -0500

On Fri, 2004-03-12 at 09:35, jeff pickering wrote: 
> "not close" case.

Agree. Reliability and Congestion control MUST be addressed.
The nice thing about being able to limit the scope is to reduce
complexity. I know we did.
OTOH, if it ends being that we have to use something
like RMT (I would hate to have to do that given the amount of fat 
that comes with it) then so be it. 

> 3 for the close proximity and for the remote multihops deployment may be useful 

I think we should be able to close the issue if we agree to the
following:
a) that we will proceed with the thought that dual mandatory approach
is where we are going
b) that we will have a close proximity vs non-close proximity approach
to partitioning the domains.

Theres a lot we can learn from something like OSPF for mechanisms
(Areas, broadcast/multicast/NBMA, etc)  but i dont think we have to
reach agreement on the details in order to close the issue.

Thoughts?

cheers,
jamal

---

From: Jamal Hadi Salim <[email-masked]>
Date: 12 Mar 2004 18:29:54 -0500

Alex,
Seems like you are trying to take my spot as most frequent poster;->

On Fri, 2004-03-12 at 15:49, Alex Audu wrote:

> 1) In a critical environment like the one we are talking about,  we need 
> ACKs to give a warm-fuzzy feeling that FEs are getting what the CEs are 
> blasting at them. 

Acks are MUST for reliability. In some cases, you dont need to receive
ACKs (eg unidirectional heartbeats). 

>  What is the implication of this as number of FEs rise into the thousands?  
> You run the risk of  tons of ACKs comming back and overwhleming the CE. 
> That is  essentially a flooding attack on the CE. 
> I believe it is called ACK implosion in  literature.  A lot
>       of ways have been proposed to avoid this,..like selective 
> ACK/NACK,..etc. These
>      suggested solutions are pretty complex.

You want all those ACKs from All FE if you want to be reliable in the
case of ForCES; lets do the math though, because for what you have
described i think you are mistaken in the case that the CE is
overwhelmed. You may have meant some of the FEs will be overwhelmed.

Lets say you had 10000 FEs.
The CE sends one multicast message and receives 10000 ACks.
Thats 10001 messages
Consider the case of unicast. You send 10000 messages and receive
10000 ACKs. 
The math says you are sending 50% less traffic with multicast.

> 2) As the number of FEs grows, the  chance of packet loss increases in 
> the network when doing multicast.  You have to fall back on retransmits  to 
> accommodate this.

Again refer to the math above. It seems to me its a lot worse
for unicast.
Retransmits are a necessary part of reliability.

> That essentially whipes out the percieved benefits of mulitcast.

;->

> 3) How do you synchronize your packets and make sure they are recieved 
> in order at the the FEs?
> You have to build something extra into  your application 
> or the FoRCES protocol to take care of such issues.  

Absolutely you have to build something into the protocol.
The ordering in Forces is at the application level.
Thats why we have two phase commits. Until the FE receives all the
messages it cant commit a transactions.
This has nothing to do with multicast.

> And if you don't  specify it, 
> there is danger of different implementations resulting in interoperability 
> issues.

Refer to above.

> 4) And there are issues of security to address.

I think we will have to address this within scope of locality.
But this is a separate issue.

> In short, I am not sure we know enough about multicast to mandate its 
> use in a
> critical envirnonment. If we are not concerned about timing, sure, we 
> can retransmit
> a couple of times untill all the endpoints get the data. But we resorted 
> to multicast  to
> save time in the first place.

Refer to my comments and repost.

cheers,
jamal

---

From: jeff pickering <[email-masked]>
Date: Fri, 12 Mar 2004 21:32:16 -0500

Alex,

Using unicast to avoid multicast ack implosion devolves into the worst 
case of  not only
a unicast ack for every packet, but also the unicast packet causing the 
ack. Im afraid we
need to accept some level of complexity to enable the scaling we need.

Cheers,
Jeff

Alex Audu wrote:

> Hello Patrick,
>
> Sorry it took me this late to respond.  I really don't know what a 
> "MUST if  available"
> means.  The main issue with multicast is that there are still a lot of 
> unresolved issues
> about it viz:
> 1) In a critical environment like the one we are talking about,  we 
> need ACKs to give
>     a warm-fuzzy feeling that FEs are getting what the CEs are 
> blasting at them.  What is
>     the implication of this as number of FEs rise into the thousands?  
> You run the risk
>     of  tons of ACKs comming back and overwhleming the CE. That is 
> essentially a
>     flooding attack on the CE. I believe it is called ACK implosion in 
> literature.  A lot
> the network
>     when doing multicast.  You have to fall back on retransmits  to 
> accommodate this.
> in order at the
>     the FEs?  You have to build something extra into  your application 
> or the FoRCES
>     protocol to take care of such issues.  And if you don't specify 
> it, there is danger of
>     different implementations resulting in interoperability issues.
>
> a couple of times untill all the endpoints get the data. But we 
> resorted to multicast  to
> Patrick Droz wrote:
>>> We can start by agreeing that Unicast is a MUST implement, and 
>>> MCAST, BCAST a
>>> SHOULD.
>>>
>>> Any comments?
>>>
>>>

---

From: Alex Audu <[email-masked]>
Date: Fri, 12 Mar 2004 14:49:59 -0600

Hello Patrick,

Sorry it took me this late to respond.  I really don't know what a "MUST 
if  available"
means.  The main issue with multicast is that there are still a lot of 
unresolved issues
about it viz:
1) In a critical environment like the one we are talking about,  we need 
ACKs to give
     a warm-fuzzy feeling that FEs are getting what the CEs are blasting 
at them.  What is
     the implication of this as number of FEs rise into the thousands?  
You run the risk
     of  tons of ACKs comming back and overwhleming the CE. That is 
essentially a
     flooding attack on the CE. I believe it is called ACK implosion in 
literature.  A lot
      of ways have been proposed to avoid this,..like selective 
ACK/NACK,..etc. These
     suggested solutions are pretty complex.

3) How do you synchronize your packets and make sure they are recieved 
in order at the
     the FEs?  You have to build something extra into  your application 
or the FoRCES
     protocol to take care of such issues.  And if you don't specify it, 
there is danger of
     different implementations resulting in interoperability issues.

Regards,
Alex.