Subject: Shared bottleneck concern with CMT [was Re: FW: ssthressh]

From: Janardhan Iyengar <[email-masked]>
Date: Tue, 19 Jul 2005 13:24:30 -0400 (EDT)

Brian,

I do not think I agree with you. Stock SCTP or TCP tries to send a window 
of data over the time window of what should be one RTT. Given that, you 
shouldn't really be sending more that one window of data to the receiver 
in one RTT.

If you repeatedly did change_primary in SCTP, and kept sending to 
different destination addresses, you are being aggressive (if the paths to 
the destination addresses share a bottleneck). Which I think is close to 
what you are doing.

> If you want to get closer to my CMT algorithm, when you go to send,
> check the send buffer availability on both TCP connections, chose the
> one with the largest buffer space, and send to it until the buffer is
> full.  Then do the comparison again, and send again until the next
> connection's buffer is full, and so on.

If you assume a infinitely large file transfer (there is always data to 
send), you will always be sending as much as both cwnds allow.

Effectively, your sending algorithm is too aggressive if the bottleneck 
is shared.

> You cannot model either stock SCTP or a persistent CMT algorithm as
> multiple completely independent flows.  They are not independent at all.

I do not understand what you mean. For congestion control purposes, they 
are independent. There is SACK information that is shared across the 
paths, but I assume that is not what you are talking about.

> The papers you cite cover independent concurrent TCP flows, with, I
> assume, some determinate traffic splitting (round robin).  Try altering
> the algorithm for selecting the TCP stream as detailed above.

What I do is, in fact, quite similar to what you describe. When the sender 
is able to send, I pick a destination arbitrarily (this I think is where 
we differ), and send to it until the cwnd is full. I then move on to the 
next destination, and so on. If my application is a large file transfer, 
then I am sending as much as the cwnds allow, to all destinations.

- jana

On Fri, 15 Jul 2005, Brian F. G. Bidulock wrote:

> Janardhan,
>
> What does stock SCTP do?  Is it using one cwnd or multiple cwnds?  Well,
> I think it is using multiple cwnds (because I calculate a different one
> for each destination) but is fair through a bottleneck.  Why is that?
> Because there is a constraint on sending.  Only one cwnd is in effect
> over a time window, the time window that the primary destination is
> "it".  This make multiple cwnd calculations compete effectively as a
> single TCP flow (both though a bottle neck and not).
>
> For my CMT algorithm, I send persistently to the best destination.  This
> means that only one cwnd is in effect over a time window, the time
> window that the best destination is "it".  This makes multiple cwnd
> calculations compete effectively as a single TCP flow, through a
> bottleneck (and not when not through a bottleneck).
>
> Take your large file experiment and run it like this: open two TCP
> connections, send to the second connection only when the first TCP
> connection has exhausted its send window.  You could try this by filling
> the first socket's send buffer completely before sending to the second
> TCP socket.  Each time that you go to send a packet, check the first TCP
> socket to see if there is room, if there is, send to it.  Otherwise you
> can send to the second one.  See if that competes unfairly with another
> TCP stream through a barbell, and then how it competes against another
> TCP stream through a football.
>
>
>
> --brian
>
>
> On Fri, 15 Jul 2005, Janardhan Iyengar wrote:
>
>> Brian,
>>
>> I think you are missing Randall's and Mark's point. There are two ways in
>> which you could maintain cwnds when doing CMT:
>>
>> (i) use a single cwnd across all destination, OR
>> (ii) use multiple cwnds, one for each destination.
>>
>> * If the paths between a sender and a receiver have a shared bottleneck
>> link, the sender SHOULD maintain a single cwnd across all destinations, so
>> that the aggregate CMT flow gets only a fair share of the bottleneck
>> bandwidth. If the sender maintains several (say n) cwnds in the face of a
>> shared bottleneck, the sender is stuck being unfair - the flow get n times
>> the fair share. This is almost the same as using multiple TCP connections
>> to download a single file (almost because with CMT, the RTTs may be
>> different).
>>
>> Try a simple experiment downloading a large file in a single TCP
>> connection, then again with multiple TCP connections downloading different
>> parts of the same file in parallel. Multiple connections do better, partly
>> because they get an unfair share of the bottleneck bandwidth. Mark's
>> mention of "much literature" was with regard to this general point, not
>> about a comparison of CMT using SCTP against TCP. From the network's
>> perspective, CMT with multiple cwnds in the presence of a shared
>> bottleneck behaves as parallel TCP connections do.
>>
>>
>> * If the paths between a sender and a receiver have separate bottleneck
>> links, the sender would like to maintain multiple cwnds - one for each
>> destination to maximize throughput that the CMT flow would get. Each cwnd
>> would try to track available bandwidth to the corresponding destination.
>> The sender could choose to use a single cwnd in this situation, but then
>> could face severe throughput degradation due to "false sharing" - when one
>> cwnd is used to track two bottlenecks, the resultant send rate will be
>> limited by the lesser of the two bottleneck bandwidths. For more detail on
>> "false sharing", see
>> http://www-2.cs.cmu.edu/~aditya/papers/ccr-sharing.pdf
>> OR
>> http://www-2.cs.cmu.edu/~aditya/papers/icnp03.pdf
>>
>>
>> A major question in CMT is how does a sender decide whether to use a
>> single or multiple cwnds.
>>
>> If your algo is able to bypass this problem, you may just have nailed the
>> problem of shared bottleneck detection with a window-based sender using
>> very simple methods. But I do not think that is the case. Let me know if
>> you think otherwise.
>>
>> regards,
>> jana
>>
>>
>> On Tue, 12 Jul 2005, Brian F. G. Bidulock wrote:
>>
>>> Mark,
>>>
>>> I was looking for the many papers written on the subject of how
>>> CMT SCTP is unfair to TCP through bottlenecks.
>>>
>>> I take it they do not exist.
>>>
>>> --brian
>>>
>>> On Tue, 12 Jul 2005, Mark Allman wrote:
>>>
>>>>
>>>>> Well, that's hardly "much literature".
>>>>
>>>> I gave you an example.  You can, for instance, look at many of my papers
>>>> for fairness results.  Many, many papers (by plenty of authors) on
>>>> congestion control do fairness comparisons and end up illustrating
>>>> generic congestion control fairness as a matter of course, but not as
>>>> the real result of interest in the paper (e.g., the paper I cited had
>>>> nothing to do with the case we're discussing -- it just happened to show
>>>> stock TCP fairness as part of the scientific process).  However, I have
>>>> absorbed a fair amount of congestion control literature and know that
>>>> this fairness result has been arrived at by a broad range of authors and
>>>> in a range of studies.
>>>>
>>>>> RFC2960 congestion control is actually more conservative than TCP
>>>>> (even TCP+SACK), so I don't think that there is a comparison there.
>>>>
>>>> RFC2960 is meant to be quite similar to TCP.  I have seen results that
>>>> show it more and less aggressive than TCP.  But, it's meant to be
>>>> roughly the same and I have seen nothing that shows this intent to have
>>>> turned out drastically wrong.
>>>>
>>>> allman
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Brian F. G. Bidulock    ¦ The reasonable man adapts himself to the ¦
>>> [email-masked]    ¦ world; the unreasonable one persists in  ¦
>>> http://www.openss7.org/ ¦ trying  to adapt the  world  to himself. ¦
>>>                        ¦ Therefore  all  progress  depends on the ¦
>>>                        ¦ unreasonable man. -- George Bernard Shaw ¦
>>>
>>
>> ---------------------------------------------------------------
>> Janardhan R. Iyengar           http://www.cis.udel.edu/~iyengar
>> Protocol Engineering Lab  --   CIS   --  University Of Delaware
>> ---------------------------------------------------------------
>
>
> -- 
> Brian F. G. Bidulock    ¦ The reasonable man adapts himself to the ¦
> [email-masked]    ¦ world; the unreasonable one persists in  ¦
> http://www.openss7.org/ ¦ trying  to adapt the  world  to himself. ¦
>                        ¦ Therefore  all  progress  depends on the ¦
>                        ¦ unreasonable man. -- George Bernard Shaw ¦
>

---------------------------------------------------------------
Janardhan R. Iyengar           http://www.cis.udel.edu/~iyengar
Protocol Engineering Lab  --   CIS   --  University Of Delaware
---------------------------------------------------------------
------------=_1121793987-8891-0--