Subject: RE: PR-SCTP question.

From: "Sellar, Ronnie" <[email-masked]>
Date: Tue, 6 May 2003 10:35:28 +0100

Brian,

This does sound a better solution to me.  It means that
SHUTDOWN/SHUTDOWN-ACK processing is not affected and it also fixes Michael's
problem of stuck data in the middle of an association.

Regards

Ronnie
---
Ronnie Sellar			Artesyn Communications Products
Tel: +44 131 475 7015		Suite 4, 2 Anderson Place
Fax: +44 131 475 7001		Edinburgh EH6 5NP
mailto:[email-masked]	http://www.artesyncp.com

> -----Original Message-----
> From: Brian F. G. Bidulock [mailto:[email-masked]]
> Sent: 06 May 2003 10:04
> To: Michael Tüxen
> Cc: Randall Stewart (cisco); [email-masked]; Sellar, Ronnie; Sctp-Impl
> (E-mail); Phillip T. Conrad; Qiaobing Xie; Michael A. Ramalho
> Subject: Re: PR-SCTP question.
> 
> 
> Michael,
> 
> Yes, I was thinking there might be a problem during transmission too.
> 
> However, as t3-rtx is never running in this circumstance, yet 
> the sender
> of the FWD-TSN can indeed expect a SACK within an RTO for the FWD-TSN.
> 
> So, it would be possible to borrow the t3-rtx timer (rather 
> than adding
> a new one) to solve this siutation as follows:
> 
> When the sender sends a FWD-TSN and there is no unacknowledged data
> remaining, resulting from the dropped TSNs, that t3-rtx SHOULD NOT be
> stopped.  This would be an alteration to rule A2).
> 
> In addition, when the t3-trx timer expires, normal base protocol
> retransmission would occur and, in addition, if the 
> cummulative TSN ack
> point is less than the Advanced.Peer.Ack.Point that in addition to
> normal retransmission a FWD-TSN be bundled with any retransmitted DATA
> chunks indicating the Advanced.Peer.Ack.Point (again) to the receiver.
> This would be an additional rule, say A4.
> 
> That is, one could make retransmission of the FWD-TSN subject to the
> same rules as would have applied to the DATA had it not been dropped.
> 
> Coupled this with making the FWD-TSN at the receiver subject 
> to the same
> rules visa vi SACK as the received DATA would have been 
> subject.  (This
> would also be an addition as Randall notes.)
> 
> Then all cases of retransmission of FWD-TSN would be handled 
> in the same
> fashion as the base protocol and none of these problems can any longer
> ensue.  Modifications to SHUTDOWN would then no longer be required.
> Also, no additional timers would be required.
> 
> One can then think of the FWD-TSN as a compressed placeholder for the
> DATA that was dropped.  If the place holder is subject to the same
> retramission (t3-rtx) and acknowledgement (SACK) rules as the DATA it
> represents, then the only variation from the base protocol is that the
> data itself does not need to be retransmitted once expired, and the
> place holders for a number of DATA chunks can be represented 
> in a single
> FWD-TSN.
> 
> I think the change would be minor in implementation (not stop T3-rtx
> when only dropped data is outstanding and consider retransmission of
> FWD-TSN when considering DATA for retransmission).
> 
> Note that the SHUTDOWN case is also handled (because now the 
> FWD-TSN for
> the dropped TSN 97 would be retransmitted on T3-rtx, and 
> T3-rtx would be
> restarted, retransmission pegs counted, ...  just as though 
> the data for
> TSN 97 were being retransmitted.)
> 
> How does that sound?
> 
> Is that a better and more complete solution proposal?
> 
> --brian
> 
> On Tue, 06 May 2003, Michael Tüxen wrote:
> 
> > Hi all,
> > 
> > Brians solution handles the case at the end of live of an 
> association.
> > 
> > Consider this case:
> > 
> > A sends TSN 0,1,2 on stream 0, all ordered. TSN 0 gets 
> dropped in the 
> > network.
> > TSN 1 and TSN 2 arrive and get SACKed. Now the sender drops (in the 
> > sense of
> > PR-SCTP)  TSN 0 and sends a FWD-TSN chunk which is dropped in the 
> > network.
> > The sender now has no timers running, the receiver waits 
> for TSN 0 to 
> > deliver
> > TSN 1 and TSN 2 to the upper layer.
> > 
> > This situation is not changed until the Upper Layer on the 
> sender side 
> > decides
> > to send more data or initiates the shutdown procedure.
> > 
> > Did I miss something?
> > 
> > Best regards
> > Michael
> > 
> > On Monday, May 5, 2003, at 20:34 Europe/Berlin, Brian F. G. 
> Bidulock 
> > wrote:
> > 
> > > Randall,
> > >
> > > If the FWD-TSN is treated in the same fashion as DATA is in the
> > > base protocol with regard to the sending of SACK, in this case a
> > > SACK would already be required by the base protocol would it
> > > not?  So I don't think that there is a need for an alteration in
> > > SACK processing.  IHO the problem appears to be (as you allude)
> > > that for a normal DATA transmission there is a retransmit timer
> > > running whereas for FWD-TSN there is not.  During normal data
> > > transmission during the association, this is not problematic
> > > because of the rules for sending subsequent FWD-TSN in receipt
> > > of SACK.  It is only in this case where there is no further data
> > > to send.
> > >
> > > As this only appears to be a problem in this one situation (not
> > > subsequent DATA to send and no SACKs forthcoming), it might be
> > > possible to handle this with a SHUTDOWN rule such as follows:
> > >
> > > Whenever the PR-SCTP sender enters the SHUTDOWN-PENDING state,
> > > and there are only TSNs marked dropped for retransmission (for
> > > which FWD-TSNs have been sent) a SHUTDOWN chunk is sent and the
> > > sender enters the SHUTDOWN-SENT state.  (The same for
> > > SHUTDOWN-RECEIVED and SHUTDOWN-ACK-SENT states.)
> > >
> > > Whenever the PR-SCTP receiver receives a SHUTDOWN or
> > > SHUTDOWN-ACK and there are gaps in its received TSN sequence
> > > (messages awaiting reordering before delivery to the user), it
> > > will treat the SHUTDOWN or SHUTDOWN-ACK though it were a forward
> > > TSN for all missing data chunks.
> > >
> > > The trick here is to not delay the SHUTDOWN (or SHUTDOWN-ACK)
> > > chunk being sent waiting for acknowledgement of dropped TSNs
> > > but to send it anyways with the knowledge that the receiver can
> > > imply any missing TSNs as having been dropped.
> > >
> > > I think this would better meet the intent of PR-SCTP (and in
> > > particular timed-reliability) as it does not rely on a timer to
> > > arrive at the implication of a missed FWD-TSN.  The receipt of a
> > > SHUTDOWN or SHUTDOWN-ACK can be treated as an implied FWD-TSN
> > > for all outstanding data.
> > >
> > > Under the base protocol, what happens if a sender sends a
> > > SHUTDOWN chunk *before* all its sent data is acknowledged?  I
> > > think an implementation could either complete the shutdown and
> > > discard data awaiting reordering, or ABORT the association.  For
> > > PR-SCTP, receiving SHUTDOWN early could be construed as cause to
> > > mark as dropped all outstanding data, and move the cummulative
> > > ack point to the "highest" received TSN and complete the
> > > SHUTDOWN procedure.
> > >
> > > Does that make sense?
> > >
> > > So effectively the rules would be that a SHUTDOWN or
> > > SHUTDOWN-ACK should not be delayed in the SHUTDOWN-PENDING or
> > > SHUTDOWN-RECEIVED states solely on the basis of unacknowledged
> > > but "dropped" TSNs, and that a received SHUTDOWN or SHUTDOWN-ACK
> > > chunk that is otherwise valid should be construed by the
> > > receiver to have the same effect as a FWD-TSN dropping all
> > > outstanding TSNs.
> > >
> > > Is that simpler than a timer?
> > >
> > > --brian
> > >
> > >
> > > Whenever the receiver receives
> > >
> > >
> > > On Mon, 05 May 2003, Randall Stewart (cisco) wrote:
> > >
> > >> Ronnie:
> > >>
> > >> Nope, I don't think you are missing anything. I have 
> been testing the
> > >> internal Cisco-IOS implementation of PR-SCTP against the 
> KAME stack
> > >> and have reached the same conclusion. I think we need to 
> require that 
> > >> the
> > >> receiver of a FWD-TSN send a SACK in reponse to this. Otherwise
> > >> you have this deadlock scenario.... I have made the KAME stack
> > >> send the SACK.. but this then begs another question...
> > >>
> > >> If the SACK in the return path was lost (I am testing at 
> times with
> > >> error rates set to 20%... so that happens a lot).. what do we do?
> > >>
> > >> It almost sounds like we need not only to require a FWD-TSN get
> > >> a SACK in response.. but that the sender of the FWD-TSN, if no
> > >> other data is being sent, start a timer to retransmit the FWD-TSN
> > >> if no SACK is received.. not sure if I worded that clear 
> but I think
> > >> it will be needed...
> > >>
> > >> Co-authors.. what do you-all think?
> > >>
> > >> R
> > >>
> > >>
> > >> Sellar, Ronnie wrote:
> > >>
> > >>> Hi,
> > >>>
> > >>> We have just started looking at PR-SCTP with a view to having a 
> > >>> prototype
> > >>> implementation ready for the next Interop in Delaware 
> and have come 
> > >>> up with
> > >>> a question regarding the FORWARD-TSN chunk and how it 
> could affect
> > >>> association termination.
> > >>>
> > >>> Consider the following example.  Endpoint A has an 
> association with 
> > >>> endpoint
> > >>> B in which all the data it sends is ordered and marked as 
> > >>> 'unreliable' for
> > >>> whatever reason.  Endpoint A has now sent all the data 
> it wants to 
> > >>> send to B
> > >>> but can't yet close the association because B has not yet 
> > >>> acknowledged one
> > >>> of the TSNs.  Lets say that it has transmitted TSNs 
> 1-100 but has 
> > >>> not yet
> > >>> received an acknowledgement of TSN 97 since B has not 
> yet received 
> > >>> that TSN.
> > >>>
> > >>> In this situation A cannot yet send the SHUTDOWN since 
> not all data 
> > >>> is acked
> > >>> and B has three chunks it cannot deliver to it's user 
> (98, 99 and 
> > >>> 100) since
> > >>> it has not received 97.  When A receives the last SACK 
> it decides 
> > >>> that it no
> > >>> longer wants to retransmit chunk 97 and so issues a FORWARD-TSN 
> > >>> chunk to
> > >>> move the cumulative ack TSN on to 100.  How does A know that B 
> > >>> received the
> > >>> FORWARD-TSN chunk ?  There is no more data to send so B 
> will not be 
> > >>> sending
> > >>> any more SACKs.  If B sends no more SACKs A's 
> cumulative ack TSN 
> > >>> will not
> > >>> advance and so he will not be able to send the SHUTDOWN chunk.
> > >>>
> > >>> In B's case when he receives the FORWARD-TSN he will be happy, 
> > >>> deliver
> > >>> 98-100 to the user and move his cumulative TSN forward. 
>  But he is 
> > >>> not
> > >>> required to send a SACK to tell A he has done so.  Are 
> we now in a 
> > >>> stalemate
> > >>> situation ?
> > >>>
> > >>> Also consider the case where B never receives the 
> FORWARD-TSN.  He 
> > >>> is now
> > >>> stuck with 98-100 in his buffers which he cannot 
> deliver since he 
> > >>> doesn't
> > >>> know that 97 is not coming.  Have we reached a similar 
> stalemate ?
> > >>>
> > >>> Perhaps I'm missing something, but I do think there 
> seems to be a 
> > >>> problem if
> > >>> a FORWARD-TSN chunk is the last chunk to be transmitted on an 
> > >>> association.
> > >>> Please feel free to correct me if I'm wrong.
> > >>>
> > >>> Cheers
> > >>>
> > >>> Ronnie
> > >>>
> > >>> ---
> > >>> Ronnie Sellar			Artesyn Communications Products
> > >>> Tel: +44 131 475 7015		Suite 4, 2 Anderson Place
> > >>> Fax: +44 131 475 7001		Edinburgh EH6 5NP
> > >>> mailto:[email-masked]	http://www.artesyncp.com
> > >>>
> > >>>
> > >>>
> > >>
> > >>
> > >> -- 
> > >> Randall R. Stewart
> > >> ITD
> > >> Cisco Systems Inc.
> > >> [email-masked] 815-342-5222(cell) or 815-477-2127(office)
> > >>
> > >
> > > -- 
> > > Brian F. G. Bidulock    ¦ The reasonable man adapts 
> himself to the ¦
> > > [email-masked]    ¦ world; the unreasonable one 
> persists in  ¦
> > > http://www.openss7.org/ ¦ trying  to adapt the  world  to 
> himself. ¦
> > >                         ¦ Therefore  all  progress  
> depends on the ¦
> > >                         ¦ unreasonable man. -- George 
> Bernard Shaw ¦
> > >
> 
> -- 
> Brian F. G. Bidulock    ¦ The reasonable man adapts himself to the ¦
> [email-masked]    ¦ world; the unreasonable one persists in  ¦
> http://www.openss7.org/ ¦ trying  to adapt the  world  to himself. ¦
>                         ¦ Therefore  all  progress  depends on the ¦
>                         ¦ unreasonable man. -- George Bernard Shaw ¦
>

---

From: "Randall Stewart (cisco)" <[email-masked]>
Date: Tue, 06 May 2003 06:57:14 -0500

Ronnie/Brian:

I think this is the correct solution...

It then requires:

1) A note on the receiver side .. aka handle like data .. you must start the
    sack timer etc

and

2) A note on the sender side.. aka you don't stop the t3-timer if there
     is no outstanding data.. and when the t3-timer goes off you should
     send a fwd-tsn as well as any retransmmitted data..

I will see if I can't cut and past from this email and do a bit
of massaging to get the proper notes in the document..

I too view the FWD-TSN as nothing but a compressed "heres the data
I am skipping and not sending"...

R

---

From: Qiaobing Xie <[email-masked]>
Date: Tue, 06 May 2003 12:16:45 -0500

> I am skipping and not sending"...

This is indeed a good way to look at it. And I think the solution
summarized by Brian and Randy is the right approach.

-Qiaobing

---

From: Peter Lei <[email-masked]>
Date: Tue, 06 May 2003 14:43:02 -0500

> summarized by Brian and Randy is the right approach.

Looks like the right (and straight-forware) solution to me
as well... 

--peter

---

From: "Randall Stewart (cisco)" <[email-masked]>
Date: Tue, 06 May 2003 15:11:48 -0500

Dear all:

I have made what I think are the appropriate changes and I would
like you all to take a look. I have put up on

http://www.sctp.org

under the RFC page tab

A web link that will take you to a very nice diff of the 02/03 version.

Please take a few moments to look at the diff's and suggest any changes

I would like to get this sumbitted by Friday.

R