Subject: Outstanding data count in multi-homed association

From: Stephan Baucke <[email-masked]>
Date: Wed, 15 Feb 2006 15:40:53 +0100

Hi,

I have a question regarding the handling of retransmissions in a 
multi-homed association. RFC 2960 section 6.4 says the following:

  >Retransmissions do not affect the total outstanding data count.
  >However, if the DATA chunk is retransmitted onto a different
  >destination address, both the outstanding data counts on the new
  >destination address and the old destination address to which the data
  >chunk was last sent shall be adjusted accordingly.

What exactly does "adjusting the count on the old destination 
accordingly" mean? Consider the following example in a multi-homed 
association with 2 paths:

1) we have TSNs 1...5 in flight on the primary path
2) due to a failure, all chunks on the primary are lost
3) T3-rtx eventually expires with earliest outstanding TSN being 1
4) cwnd on the primary will now be set to 1 MTU
5) some of the lost TSNs 1...5 will now be retransmitted on the 
alternate path (as far as the alternate path cwnd allows)

My question now: Do the 5 TSNs still count as outstanding data *on the 
primary path*? If that is the case, I cannot send a new chunk (TSN 6) on 
the primary at this point, since the cwnd has been reduced to one and 
the earliest outstanding TSN is still 1 (i.e. I have to wait until TSN 5 
has been acked from the alternate). Or can I reset the oustanding data 
counter to zero on the primary at this point, allowing TSN 6 to be sent?

BR,
Stephan

---

From: Randall Stewart <[email-masked]>
Date: Wed, 15 Feb 2006 12:29:20 -0500

> counter to zero on the primary at this point, allowing TSN 6 to be sent?

What you should be doing is

Mark-all-on neta for retransmission
neta->flight_size = 0;
netb->flight_size += size of what you can fit on netb

Yes, its possible that not all the data will fit on
the new destination.. even though the wording imply's
that all of it will fit... at least that was
how it was written..

So making the appropriate adjust means that:

a) Where you sent it to looses the ones marked
    for retransmission
and
b) the size you actually send on the new net (assuming
    there is one) is incremented onto the new destination.
and
c) Any that don't fit are not considered outstanding..

R

-- 
Randall Stewart
ITD - Cisco Systems Inc.
803-345-0369 <or> 815-342-5222 (cell)

---

From: Stephan Baucke <[email-masked]>
Date: Wed, 15 Feb 2006 18:51:44 +0100

Randall Stewart wrote:

>> primary path*? If that is the case, I cannot send a new chunk (TSN 6) 
>> on the primary at this point, since the cwnd has been reduced to one 
>> and the earliest outstanding TSN is still 1 (i.e. I have to wait until 
>> TSN 5 has been acked from the alternate). Or can I reset the 
>> oustanding data counter to zero on the primary at this point, allowing 
>> TSN 6 to be sent?
> 

Ok, thanks, that's what I was hoping. BTW, it looks like this is not 
implemented correctly in the NS2 module. I'll see if I can come up with 
a fix.

BR,
Stephan.

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Wed, 15 Feb 2006 14:28:24 -0700

Randall,

Hey, what happens if they are acknowledged by SACK from the primary
path before they are actually transmitted on the secondary path (e.g.
because they were waiting on cwnd for the new path)?  How is oustanding
and acked bytes handled then?

--brian

-- 
Brian F. G. Bidulock    ¦ The reasonable man adapts himself to the ¦
[email-masked]    ¦ world; the unreasonable one persists in  ¦
http://www.openss7.org/ ¦ trying  to adapt the  world  to himself. ¦
                        ¦ Therefore  all  progress  depends on the ¦
                        ¦ unreasonable man. -- George Bernard Shaw ¦

---

From: Randall Stewart <[email-masked]>
Date: Wed, 15 Feb 2006 17:14:17 -0500

Brian:

Not quite sure I follow the question...

So I have

primary= dest-a

TSN1 - todesta (at sending cwnd is full)
TSN2 - todesta
TSN3 - todestb
TSN4 - todesta

(Now how we got TSN3 scheduled to destb is a different
  question.. but for simplicty lets assume user preference).

Now you are saying that a SACK comes
back

<----SACK(0 gap=1, 2-2)

Well.. if this is the case you have ack'd data on TSN2.
Its removed from the flight sizes (both the overall flight
size and dest-b's). Now you can't advance dest-a or dest-b's
cwnd.. since the cum-tsn did not move forward... And of
course you must keep TSN2 around... marked as ACK'd in
case of reneging... but the flight sizes being reduced
would let you transmit more to dest-b (if there happened
to be any)..

Did I answer your question... or did I mis-understand your
request?

R

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Wed, 15 Feb 2006 15:27:58 -0700

Randall,

No, I was meaning in the retransmission scenario described:

 TSN1 -> to dest A
 TSN2 -> to dest A
 TSN3 -> to dest A
 --- RTO dest A ---
 TSN1 -> to dest B
 TSN2 waiting cwnd on dest B
 TSN3 waiting cwnd on dest B
 dest A now has no bytes outstanding
 <- SACK acking TSN3.

Since TSN2 and TSN3 were only even sent to dest A, the SACK is
obviously from dest A.  The question is how do I then handle 
bytes outstanding and acked bytes for dest A based on the SACK?

--brian

---

From: Stephan Baucke <[email-masked]>
Date: Thu, 16 Feb 2006 10:17:57 +0100

> bytes outstanding and acked bytes for dest A based on the SACK?

IMO they should be left unchanged, i.e. the ACK should be treated as a 
spurious one. The problem is one of incomplete information. Lacking any 
acknowledgement from dest A, at some point you have to make a decision 
that a previously sent chunk is lost and no longer considered 
outstanding, even if the decision may be wrong in some corner cases. To 
me, the most logical point to do this is when SCTP decides to retransmit 
the chunks, i.e. when it marks them for retransmission. This is inline 
with the behaviour described by Randall.

BTW, if you left it in the outstanding count for dest A, you would still 
have an ambiguity if an ACK finally arrived (i.e. does the ACK refer to 
the original on dest A or the retransmission on B?).

Stephan.

---

From: Randall Stewart <[email-masked]>
Date: Thu, 16 Feb 2006 04:24:35 -0500

Brian:

Ok, now I see what you are asking.. let me illustrate
my answer with your picture below.. I will add some
variables...

> No, I was meaning in the retransmission scenario described:
Intially...
desta cwnd=3000 fs=0
destb cwnd=3000 fs=0
total_fs=0

>  TSN1(sz=1452) -> to dest A (Result cwnd=1548, fs=1452) tf=1452
>  TSN2(sz=1452) -> to dest A (Result cwnd=96,   fs=2904) tf=2904
>  TSN3(sz=1452) -> to dest A (Result cwnd=0,    fs=4356) tf=4356
<at this point destb remains the same as above>
>  --- RTO dest A ---
All TSN's 1-3 are marked for retransmission.
dest-a cwnd=1500 fs=0
dest-b cwnd=3000 fs=0
total_fs=0
>  TSN1 -> to dest B (Result cwnd=1548, fs=1452) tf=1452
Assuming that our RTO timer has not expired
on B we end up with:
dest-a cwnd=1500 fs=0
dest-b cwnd=1548, fs=1452
tf=1452
No advancement is made to any cwnd (since
the cum-ack is not moving forward).
But we now use this opp to do
TSN2(sz=1452) -> to dest B (Result cwnd=96,   fs=2904) tf=2904
<and if there was a TSN4 in the wings>
TSN4(sz=1542) -> to dest A (Result cwnd=48,   fs=1452) tf=4356
<and if there was a TSN5 in the wings>
TSN5(sz=1452) -> to dest A (Result cwnd=0,    fs=2904) tf=5808

Does that answer your question?

R