Subject: RE: [Forces-protocol] Comments from chat with ADs

From: "Khosravi, Hormuzd M" <[email-masked]>
Date: Thu, 4 Mar 2004 18:11:35 -0800

Hi Alex,

In general, I prefer using language such as MUST so that there is no
confusion or interop issues in the protocol implementations. However,
you raise a reasonable point and let me tell you my thoughts on this. We
will definitely need Unicast between FEs and CEs at the minimum in the
protocol. In addition, if the interconnect technology supports Multicast
and there is a case where it is useful to the application (such as
downloading IPv4 routes) we must allow it as well without breaking
interop. In order to do that, we will need specific procedures in the
protocol that will allow the CE and FE to negotiate on using multicast
dynamically at run-time (say during the binding or joining phase between
the CE and FE) and also associate different multicast groups with LFBs.
We will also need to define how Reliability/CC etc. will be provided for
multicast...may be using RMT, TIPC, etc. (?) If we do this correctly,
then applications will be able to use both unicast and multicast without
any interop issues.

Let me know what you guys think about this.

Regards
Hormuzd

-----Original Message-----
From: Alex Audu [mailto:[email-masked]] 
Sent: Thursday, March 04, 2004 1:22 PM
To: Khosravi, Hormuzd M
Cc: [email-masked]
Subject: Re: [Forces-protocol] Comments from chat with ADs

Hormuzd,

Why should multicast be "mandated for certain cases where it will be 
useful" if there
is a simpler, more robust way to do the same thing?  I'll say we don't 
mandate this.
We can make it a "SHOULD" strength, not a "MUST".  We mandate UNICAST
and make MULTICAST optional.  Any issues with this?

Regards,
Alex.

Khosravi, Hormuzd M wrote:

>Hi David
>
>Your summary below of the discussion with Ads on the different issues
>seems like very reasonable approaches to me.
>
>I agree that it will be a good idea to present transport alternatives
at
>IP and non-IP level for the ForCES protocol. We can mandate one or more
>transport for IP and one or more transport for non-IP, this way the
>interoperability requirements are met and at the same time, the in-box,
>out of box scenarios are met.
>
>Also, agree with having multiple channels/connections to differentiate
C
>& D traffic between CEs and FEs. On the unicast/multicast topic, dual
>mandatory approach again seems a good idea to support interoperability.
>The one thing that I would like to point out is that for any ForCES
>protocol implementation to work, unicast will definitely be required at
>the minimum. One could also mandate using multicast for certain cases
>where it will be useful (e.g. IPv4/config table download), if it is
>supported by the interconnect as you stated. But just multicast without
>unicast will not work, atleast not efficiently.
>
>Regards
>Hormuzd
>
>-----Original Message-----
>From: [email-masked]
>[mailto:[email-masked]] On Behalf Of Putzolu, David
>Sent: Tuesday, March 02, 2004 5:06 AM
>To: [email-masked]
>Subject: [Forces-protocol] Comments from chat with ADs
>
>Patrick and I spent about 90 minutes last night
>chatting with Bill and Alex about ForCES, with
>maybe half of the time on the protocol.  In
>the conversations with them, along with some
>follow-up chats I had with Allison Mankin, the
>following topics where covered:  multiple 
>channels, multicast/unicast, and congestion 
>control.  Below is what I recall from memory 
>(Patrick please correct me/elaborate as needed :)
>
>
>
>Multiple channels:  
>
>I discussed the interference issue that 
>was identified by the GRMP team.  Bill &
>Alex agreed that many OS implementations can have
>an interference property (i.e. where two different
>connections will have queuing interference lower in
>the networking stack in the OS).  So it is necessary
>to make sure when implementing a FE (or CE probably
>as well) that this is taken into account.  Methods
>of doing that include:
>* Manually queuing packets (i.e. feed OS few enough
>  packets that it cannot get into trouble - although
>  you pay a performance price for that, in that fine
>  grained scheduling takes cycles)
>* Indicating to the OS to treat flows differently - 
>  some real-time OS will let you do this, that is they
>  are implemented to internally retain QoS between
>  flows.
>However, it was also mentioned that intra-FE 
>interference is only one kind of interference.  Good
>FE implementation (e.g. by manually queuing above the
>OS) helps to fix intra-FE interference, but it does
>not solve inter-FE interference.  That is, if all
>(C & D) packets are always on the same connection,
>that makes it impossible on the wire to differentiate
>them from one another.  Imagine you have a bunch of
>FEs with some of them being subject to DOS attack for
>D packets, and you want to query one of the FEs for
>statistics.  It would be good to have some way of
>ensuring the C packets (or any high priority packets -
>OSPF HELLO maybe) get through.  Separate channels
>(as Jamal indicated, don't even have to be C & D)
>makes this possible.   Putting everything on one
>channel makes it impossible to deal with inter-FE
>interference in a differentiated fashion. 
>
>
>Multicast & Unicast:  
>
>The discussion on this started out with a discussion
>of the IETF protocol design BKM that options can ruin
>compatibility.  That is, if there are four ways to
>implement something, all optional, then you can have
>client 1 implement A & B, and client 2 implement C & D,
>and even though both clients are 100% compliant, they
>cannot talk to one another. Better to say all clients
>must implement A and make B, C, and D optional, so that
>way you know that all implementations will be able to 
>at least talk.
>
>Patrick & I pushed on this a bit, explaining that we
>actually had potentially (at least) two different
>environments - very close environments where multicast
>support could be present in the interconnect,  and 
>not-as-close environments (and some close
>environments as well) where multicast was not available.
>
>Bill & Alex indicated that if there really are two
>quite different cases, then it is possible to have a
>"dual mandatory" approach.  I.e.:
>
>If your underlying interconnect supports multicast,
>then you MUST implement the following (multicast)
>method of communication.
>
>If your underlying interconnect supports unicast, then
>you MUST implement the following (unicast) method
>of communication.
>
>This will guarantee interop, although at a potential
>extra cost (cost being having to potentially support
>both when both are present).  While this isn't optimal,
>it does at least give some flexibility.
>
>
>
>Congestion control:
>
>Several times over the whole conversation the topic
>of congestion control came up.  Also, as part of some
>TIPC discussions, I ended up spending some time talking
>to Allison Mankin (transport AD) about congestion control
>as well.  All the ADs I talked to where consistent on 
>the following:
>
>If you are running over IP, you MUST act in a RFC2914
>compliant fashion (i.e. respond to congestion like TCP
>or SCTP or RMT does).  Two reasons where put forth for
>this:
>1) Any time you define an IP-based protocol you cannot
>guarantee that it will not go out over the Internet. As
>such, the IESG will NOT standardize IP protocols with
>"bad" (non-2914) behaviors.
>2) Lots of implementation experience showing that other
>congestion control mechanisms don't work (congestion
>collapse etc).
>
>They where very firm on the above and my impression was
>that convincing them otherwise would be very difficult.
>
>That being said, a second alternative did arise:  If
>you are not implementing on top of IP (e.g. running on
>Infiniband, ATM, Ethernet) then you could rely on mechanisms
>for congestion control built into what you are relying on,
>or even consider building your own congestion control
>mechanisms (ala TIPC).  This makes it possible to define
>(for example) a multicast transport for forces without
>having to go all the way and implementing RMT - as long
>as it is clear that it isn't over IP.  The ADs went as
>far as saying that the IETF could even accept standards-
>track drafts that say nothing about IP (e.g. define an
>Ethernet-only transport for ForCES), if it is necessary 
>for ForCES. GSMP and (some other group I forgot) was
>given as an example.
>
>Cheers,
>David
>
>_______________________________________________
>Forces-protocol mailing list
>[email-masked]
>https://www1.ietf.org/mailman/listinfo/forces-protocol
>
>
>_______________________________________________
>Forces-protocol mailing list
>[email-masked]
>https://www1.ietf.org/mailman/listinfo/forces-protocol
>  
>

_______________________________________________
Forces-protocol mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/forces-protocol

---

From: jeff pickering <[email-masked]>
Date: Fri, 05 Mar 2004 08:35:59 -0500

All,

Given that I dont believe we can scale well without multicast, I would 
like to see it
as a MUST. But thats just due to a focus on larger systems. For 
lightweight implementations,
one could envision unicast being sufficient. I agree with Hormuzd that 
some negotiation
is necessary if we make this a SHOULD. Then the question would become 
what happens
when 1 FE says it can only unicast. Does all traffic revert to unicast, 
or can there be a mix?

Jeff

---

From: Alex Audu <[email-masked]>
Date: Fri, 05 Mar 2004 09:52:01 -0600

Hi Hormuzd,

My point is: multicast's main purpose is getting the same data to 
multitargets
quickly.  But I am not sure I'd want to use it all the time. If I am 
sending to
less than a certain number of targets,..say 4, would I want to use 
multicasting,
given all the other problems it has?  I can achieve my goal with 
unicast. I think
what we need to do is mandate the basic (simplest) mechanism here (UNICAST)
and anything else (MULTICAST) as gravy.

Regars,
Alex.

---

From: Alex Audu <[email-masked]>
Date: Fri, 05 Mar 2004 10:38:39 -0600

Hi Jeff,

We can have the protocol fall back to UNICAST in this case. Supporting a mix
in a given transaction may add unnecessary complexity to the protocol.
What do you all think?

Alex.

jeff pickering wrote:

> when 1 FE says it can only unicast. Does all traffic revert to 
> unicast, or can there be a mix?
>
> Jeff
>
>
>> From: Alex Audu [mailto:[email-masked]] Sent: Thursday, March 
>> 04, 2004 1:22 PM
>> To: Khosravi, Hormuzd M
>> is a simpler, more robust way to do the same thing?  I'll say we 
>> don't mandate this.
>>> following topics where covered:  multiple channels, 
>>> multicast/unicast, and congestion control.  Below is what I recall 
>>> from memory (Patrick please correct me/elaborate as needed :)
>>>
>>>
>>>
>>> Multiple channels: 
>>> I discussed the interference issue that was identified by the GRMP 
>>> team.  Bill &
>>> * Indicating to the OS to treat flows differently - some real-time 
>>> OS will let you do this, that is they
>>> However, it was also mentioned that intra-FE interference is only 
>>> one kind of interference.  Good
>>> way you know that all implementations will be able to at least talk.
>>>
>>> support could be present in the interconnect,  and not-as-close 
>>> environments (and some close
>>> as well.  All the ADs I talked to where consistent on the following:
>>>
>>> Ethernet-only transport for ForCES), if it is necessary for ForCES. 
>>> GSMP and (some other group I forgot) was
>>> given as an example.
>>>
>>> Cheers,
>>> David
>>>