Subject: RE: iSCSI: MaxRecvPDULength simplification

From: [email-masked]
Date: Mon, 10 Jun 2002 11:29:31 -0600

Michael,

That is true where the limit on PDU length is due to TUF and network packet size limits. However, when the connection is migrating the limitation may be in the new receiver and that may not be flexible.

Pat

-----Original Message-----
From: Michael Schoberg [mailto:[email-masked]]
Sent: Thursday, March 28, 2002 8:48 AM
To: 'Mallikarjun C.'; IPS Reflector (E-mail)
Subject: RE: iSCSI: MaxRecvPDULength simplification

Thank you for the draft reference.  It's good that the iSCSI group is
looking at these other drafts.  Unfortunately, I haven't been following the
TUFs draft so my opinion may be way off.  From my brief read, it seems iSCSI
may be able to afford a little elasticity with MaxRecvPDULength than what's
being proposed.  The PMTU reduction scenario is referenced within TUFs as
rare and recoverable.  In your opinion, could an iSCSI client afford to
ignore resegmentation on previously sent PDUs (SNACK) or for recovered PDUs
(task reassignment) and simply apply the new MaxRecvPDULength negotiated
value on future PDUs?  This would turn the resegmentation feature into a MAY
from a MUST.
 
Thanks.

:-----Original Message-----
:From: Mallikarjun C. [mailto:[email-masked]]
:Sent: Wednesday, March 27, 2002 9:27 PM
:To: IPS Reflector (E-mail)
:Subject: Re: iSCSI: MaxRecvPDULength simplification
:
:
:One of the design goals of task reassign is to be able to reassign the
:connection allegiance of a task to a different connection 
:that's perhaps
:using an entire different network for a true failover 
:capability - for ex.,
:the original allegiant connection could be traversing a 
:corprorate intranet,
:and the reassigned connection could be using the public 
:internet.  That's
:the reason for not placing the equivalence requirement on the two
:connections.
:
:MaxRecvPDULength was made negotiable during the FFP because
:several of us felt that it's very useful for iSCSI to allow 
:implementations
:with the `ULPDU containment' property
:(http://search.ietf.org/internet-drafts/draft-ietf-tsvwg-tcp-ul
:p-frame-01.txt).
:This automatically means that MaxRecvPDULength should be changeable
:with changes in PMTU changes, particularly with dynamic PMTU 
:degradation.
:
:Let's look at the possibility you suggest - there being two 
:valid MaxRecvPDULength
:values at the same time.  If a change in FFP is being 
:attempted for this key, it must
:be due to PMTU changes, and the implicit expectation is that 
:initiator will sense
:it and initiate the negotiation process (thus enabling the 
:target to declare its changed
:MaxRecvPDULength, if any, as well in the text response).  
:There is no ambiguity for
:the initiator since the text negotiation can not be assumed 
:successful (meaning new
:MaxRecvPDULength can't be used) until the text response (with 
:F-bit) is received.
:As far as the target is concerned, it can not assume the text 
:negotiation to be effective
:(meaing new MaxRecvPDULength can't be used) until the StatSN 
:for the text response
:is ack'ed - perhaps we should make this clear in the text (and 
:allow explicit soliciting
:for StatSN acknowledgement using a NOP-ping).
:--
:Mallikarjun
:
:Mallikarjun Chadalapaka
:Networked Storage Architecture
:Network Storage Solutions Organization
:Hewlett-Packard MS 5668
:Roseville CA 95747
:[email-masked]
:
:----- Original Message -----
:From: "Michael Schoberg" <[email-masked]>
:To: "IPS Reflector (E-mail)" <[email-masked]>
:Sent: Wednesday, March 27, 2002 8:36 AM
:Subject: RE: iSCSI: MaxRecvPDULength simplification
:
:
:> SNACK with RL=0 is only one requirement.  Another is task allegiance
:> reassignment (6.1.2) which does not use SNACK (correct?)  
:Here the initiator
:> sends a Task-Management request with a Task-Reassign message 
:to the target.
:> The target must reorganize it's T->I messages to match the 
:MaxRecvPDULength
:> of the new connection.  Plus, the Task Reassign message includes the
:> ExpDataSN field which, if I'm reading right, on reassignment 
:the target must
:> remember the sequencing of the old connection in order to track the
:> ExpDataSN field then re-sequence for the new connection 
:(9.5.4).  So now
:> target implementations will have to keep track of sequence 
:indexes for
:> Data-In PDUs for conversion over to the new allegiance.
:>
:> On-the-fly modification of MaxRecvPDULength also means that 
:some compliance
:> standard must be set for in-flight messages.  An outstanding 
:read request
:> may have T->I PDUs that comply with multiple 
:MaxRecvPDULength values.  In a
:> sense, there will be times where multiple MaxRecvPDULength 
:values are valid.
:> At what point MUST the target comply with the new value?
:>
:> So I guess I'm not convinced that the benefit of simplification is
:> illusionary.  Is there a discussion thread that expressly states
:> MaxRecvPDULength must have the flexibility allowed for in 
:the draft?  The
:> discussions I've seen mainly centered around whether it 
:should be duplex
:> valued (I->T, T->I).
:>
:> Thanks.
:>
:> -----Original Message-----
:> From: Julian Satran [mailto:[email-masked]]
:> Sent: Wednesday, March 27, 2002 1:40 AM
:> To: Michael Schoberg
:> Cc: IPS Reflector (E-mail); [email-masked]
:> Subject: Re: iSCSI: MaxRecvPDULength simplification
:>
:>
:>
:> The simplification you are talking about is illusory. Currently this
:> requires SNACK to require all data(RL = 0).
:> The restriction you propose instead is far more severe (and 
:unwarranted).
:>
:> Julo
:>
:>
:>
:> Michael Schoberg <[email-masked]>
:> Sent by: [email-masked]
:>
:>
:> 27-03-02 01:19
:> Please respond to Michael Schoberg
:>
:>
:>
:>         To:        "IPS Reflector (E-mail)" <[email-masked]>
:>         cc:
:>         Subject:        iSCSI: MaxRecvPDULength simplification
:>
:>
:>
:>
:> I've looked over some of the reflector messages regarding 
:MaxRecvPDULength
:> negotiation (back to Oct 2001).  I can't find the discussion 
:of why this
:> value must be available for negotiation outside of login.  It really
:> complicates SNACK and Task Reassignment if the initiator can 
:change this
:> value on-the-fly.  Would it be too restrictive to propose 
:the following
:> rules?
:>
:> 1) MaxRecvPDULength is negotiated only at login before FFP.
:>
:> 2) For message-level recovery, a connection must be prepared 
:to accept the
:> MaxRecvPDULength of the connection it's providing fail over 
:capability for.
:> It doesn't seem unreasonable to expect fault tolerant 
:configurations to
:> comply with this.  It would simply be stating that 
:RecoveryLevel 2 devices
:> should be somewhat matched in this capability.
:>
:>
:>
:>  -----Original Message-----
:> From: Julian Satran [mailto:[email-masked]]
:> Sent: Tuesday, March 26, 2002 1:50 AM
:> To: Michael Schoberg
:> Cc: IPS Reflector (E-mail); [email-masked]
:> Subject: Re: iSCSI: SNACK RunLength
:>
:>
:> Michael,
:>
:> The paragraph says that if you issue a SNACK after the 
:MaxPDURecvLength has
:> decreased you should use RunLength 0 (meaning all) instead 
:of a specific
:> runlength which might not be accurate anymore.
:>
:> Julo
:>
:>
:> Michael Schoberg <[email-masked]>
:> Sent by: [email-masked]
:>
:>
:> 25-03-02 19:43
:> Please respond to Michael Schoberg
:>
:>
:>        To:        "IPS Reflector (E-mail)" <[email-masked]>
:>        cc:
:>        Subject:        iSCSI: SNACK RunLength
:>
:>
:>
:>
:>
:> Am I just not reading this or can this paragraph use some 
:help?  Can someone
:> give an interpretation?
:>
:>  9.16.3  RunLength
:>                                 ...
:>
:>          The first data SNACK, issued after initiator's 
:MaxRecvPDULength
:>          decreased, for a command issued on the same 
:connection before the
:>
:>          change in MaxRecvPDULength, MUST use RunLength "0" 
:to request
:>          retransmission of any number of PDUs (including 
:one).  The number
:> of
:>          retransmitted PDUs in this case, may or may not be 
:the same as
:> the
:>          original transmission, depending on whether loss 
:was before or
:> after
:>          the MaxRecvPDULength was changed at the target.
:>
:>
:> Does this refer to something like:
:>
:> For connections with unacknowledged PDUs that undergo 
:MaxRecvPDULength
:> negotiation ...
:>
:>
:>
:>
:>
:>
:>
:

---

From: Michael Schoberg <[email-masked]>
Date: Mon, 10 Jun 2002 13:09:07 -0500

By "connection is migrating", are you talking about iSCSI connection
recovery or is this something else?  If it's an iSCSI connection in
recovery, I think your scenario is unlikely (or I hope it's unlikely).  I
guess I'd argue that in those circumstances, the new receiver could choose a
different method for recovery.

TUFs appears to have more to do with the TCP layer and segment loss than
iSCSI.  Adjusting the MaxRecvPDULength dynamically seems like just another
costly draft feature.  My guess on the original intent of this feature was
to ensure the RDMA blocks were posted in-sync with the (ever changing) TCP
MSS.  If so, this should be handled between the IP and TCP layer where iSCSI
would be oblivious to it.

If MaxRecvPDULength is for dynamically changing the receive characteristics
of the iSCSI initiator/target, it seems like existing mechanisms could be
used without requiring this feature.   So far, I haven't seen a posted
scenario where this feature really makes sense and in my (humble) opinion,
these unlikely events could be handled simply by doing a logout & relogin.
However, it does appear there's a strong consensus for this as a spec
necessity, so I haven't kept the thread active.

:-----Original Message-----
:From: [email-masked] [mailto:[email-masked]]
:Sent: Monday, June 10, 2002 12:30 PM
:To: Michael Schoberg; [email-masked]; [email-masked]
:Subject: RE: iSCSI: MaxRecvPDULength simplification
:
:
:Michael,
:
:That is true where the limit on PDU length is due to TUF and 
:network packet size limits. However, when the connection is 
:migrating the limitation may be in the new receiver and that 
:may not be flexible.
:
:Pat
:
:-----Original Message-----
:From: Michael Schoberg [mailto:[email-masked]]
:Sent: Thursday, March 28, 2002 8:48 AM
:To: 'Mallikarjun C.'; IPS Reflector (E-mail)
:Subject: RE: iSCSI: MaxRecvPDULength simplification
:
:
:Thank you for the draft reference.  It's good that the iSCSI group is
:looking at these other drafts.  Unfortunately, I haven't been 
:following the
:TUFs draft so my opinion may be way off.  From my brief read, 
:it seems iSCSI
:may be able to afford a little elasticity with 
:MaxRecvPDULength than what's
:being proposed.  The PMTU reduction scenario is referenced 
:within TUFs as
:rare and recoverable.  In your opinion, could an iSCSI client afford to
:ignore resegmentation on previously sent PDUs (SNACK) or for 
:recovered PDUs
:(task reassignment) and simply apply the new MaxRecvPDULength 
:negotiated
:value on future PDUs?  This would turn the resegmentation 
:feature into a MAY
:from a MUST.
: 
:Thanks.
:
::-----Original Message-----
::From: Mallikarjun C. [mailto:[email-masked]]
::Sent: Wednesday, March 27, 2002 9:27 PM
::To: IPS Reflector (E-mail)
::Subject: Re: iSCSI: MaxRecvPDULength simplification
::
::
::One of the design goals of task reassign is to be able to reassign the
::connection allegiance of a task to a different connection 
::that's perhaps
::using an entire different network for a true failover 
::capability - for ex.,
::the original allegiant connection could be traversing a 
::corprorate intranet,
::and the reassigned connection could be using the public 
::internet.  That's
::the reason for not placing the equivalence requirement on the two
::connections.
::
::MaxRecvPDULength was made negotiable during the FFP because
::several of us felt that it's very useful for iSCSI to allow 
::implementations
::with the `ULPDU containment' property
::(http://search.ietf.org/internet-drafts/draft-ietf-tsvwg-tcp-ul
::p-frame-01.txt).
::This automatically means that MaxRecvPDULength should be changeable
::with changes in PMTU changes, particularly with dynamic PMTU 
::degradation.
::
::Let's look at the possibility you suggest - there being two 
::valid MaxRecvPDULength
::values at the same time.  If a change in FFP is being 
::attempted for this key, it must
::be due to PMTU changes, and the implicit expectation is that 
::initiator will sense
::it and initiate the negotiation process (thus enabling the 
::target to declare its changed
::MaxRecvPDULength, if any, as well in the text response).  
::There is no ambiguity for
::the initiator since the text negotiation can not be assumed 
::successful (meaning new
::MaxRecvPDULength can't be used) until the text response (with 
::F-bit) is received.
::As far as the target is concerned, it can not assume the text 
::negotiation to be effective
::(meaing new MaxRecvPDULength can't be used) until the StatSN 
::for the text response
::is ack'ed - perhaps we should make this clear in the text (and 
::allow explicit soliciting
::for StatSN acknowledgement using a NOP-ping).
::--
::Mallikarjun
::
::Mallikarjun Chadalapaka
::Networked Storage Architecture
::Network Storage Solutions Organization
::Hewlett-Packard MS 5668
::Roseville CA 95747
::[email-masked]
::
::----- Original Message -----
::From: "Michael Schoberg" <[email-masked]>
::To: "IPS Reflector (E-mail)" <[email-masked]>
::Sent: Wednesday, March 27, 2002 8:36 AM
::Subject: RE: iSCSI: MaxRecvPDULength simplification
::
::
::> SNACK with RL=0 is only one requirement.  Another is task allegiance
::> reassignment (6.1.2) which does not use SNACK (correct?)  
::Here the initiator
::> sends a Task-Management request with a Task-Reassign message 
::to the target.
::> The target must reorganize it's T->I messages to match the 
::MaxRecvPDULength
::> of the new connection.  Plus, the Task Reassign message includes the
::> ExpDataSN field which, if I'm reading right, on reassignment 
::the target must
::> remember the sequencing of the old connection in order to track the
::> ExpDataSN field then re-sequence for the new connection 
::(9.5.4).  So now
::> target implementations will have to keep track of sequence 
::indexes for
::> Data-In PDUs for conversion over to the new allegiance.
::>
::> On-the-fly modification of MaxRecvPDULength also means that 
::some compliance
::> standard must be set for in-flight messages.  An outstanding 
::read request
::> may have T->I PDUs that comply with multiple 
::MaxRecvPDULength values.  In a
::> sense, there will be times where multiple MaxRecvPDULength 
::values are valid.
::> At what point MUST the target comply with the new value?
::>
::> So I guess I'm not convinced that the benefit of simplification is
::> illusionary.  Is there a discussion thread that expressly states
::> MaxRecvPDULength must have the flexibility allowed for in 
::the draft?  The
::> discussions I've seen mainly centered around whether it 
::should be duplex
::> valued (I->T, T->I).
::>
::> Thanks.
::>
::> -----Original Message-----
::> From: Julian Satran [mailto:[email-masked]]
::> Sent: Wednesday, March 27, 2002 1:40 AM
::> To: Michael Schoberg
::> Cc: IPS Reflector (E-mail); [email-masked]
::> Subject: Re: iSCSI: MaxRecvPDULength simplification
::>
::>
::>
::> The simplification you are talking about is illusory. Currently this
::> requires SNACK to require all data(RL = 0).
::> The restriction you propose instead is far more severe (and 
::unwarranted).
::>
::> Julo
::>
::>
::>
::> Michael Schoberg <[email-masked]>
::> Sent by: [email-masked]
::>
::>
::> 27-03-02 01:19
::> Please respond to Michael Schoberg
::>
::>
::>
::>         To:        "IPS Reflector (E-mail)" <[email-masked]>
::>         cc:
::>         Subject:        iSCSI: MaxRecvPDULength simplification
::>
::>
::>
::>
::> I've looked over some of the reflector messages regarding 
::MaxRecvPDULength
::> negotiation (back to Oct 2001).  I can't find the discussion 
::of why this
::> value must be available for negotiation outside of login.  It really
::> complicates SNACK and Task Reassignment if the initiator can 
::change this
::> value on-the-fly.  Would it be too restrictive to propose 
::the following
::> rules?
::>
::> 1) MaxRecvPDULength is negotiated only at login before FFP.
::>
::> 2) For message-level recovery, a connection must be prepared 
::to accept the
::> MaxRecvPDULength of the connection it's providing fail over 
::capability for.
::> It doesn't seem unreasonable to expect fault tolerant 
::configurations to
::> comply with this.  It would simply be stating that 
::RecoveryLevel 2 devices
::> should be somewhat matched in this capability.
::>
::>
::>
::>  -----Original Message-----
::> From: Julian Satran [mailto:[email-masked]]
::> Sent: Tuesday, March 26, 2002 1:50 AM
::> To: Michael Schoberg
::> Cc: IPS Reflector (E-mail); [email-masked]
::> Subject: Re: iSCSI: SNACK RunLength
::>
::>
::> Michael,
::>
::> The paragraph says that if you issue a SNACK after the 
::MaxPDURecvLength has
::> decreased you should use RunLength 0 (meaning all) instead 
::of a specific
::> runlength which might not be accurate anymore.
::>
::> Julo
::>
::>
::> Michael Schoberg <[email-masked]>
::> Sent by: [email-masked]
::>
::>
::> 25-03-02 19:43
::> Please respond to Michael Schoberg
::>
::>
::>        To:        "IPS Reflector (E-mail)" <[email-masked]>
::>        cc:
::>        Subject:        iSCSI: SNACK RunLength
::>
::>
::>
::>
::>
::> Am I just not reading this or can this paragraph use some 
::help?  Can someone
::> give an interpretation?
::>
::>  9.16.3  RunLength
::>                                 ...
::>
::>          The first data SNACK, issued after initiator's 
::MaxRecvPDULength
::>          decreased, for a command issued on the same 
::connection before the
::>
::>          change in MaxRecvPDULength, MUST use RunLength "0" 
::to request
::>          retransmission of any number of PDUs (including 
::one).  The number
::> of
::>          retransmitted PDUs in this case, may or may not be 
::the same as
::> the
::>          original transmission, depending on whether loss 
::was before or
::> after
::>          the MaxRecvPDULength was changed at the target.
::>
::>
::> Does this refer to something like:
::>
::> For connections with unacknowledged PDUs that undergo 
::MaxRecvPDULength
::> negotiation ...
::>
::>
::>
::>
::>
::>
::>
::
: