Subject: RE: iSCSI: plugfest4 issues

From: [email-masked]
Date: Thu, 1 Aug 2002 10:56:30 -0400

Julian,
 
I think we need to do something here, as there are clearly
situations in which the residual count is important for commands that
complete with other than good status, making the "other point of
view" reported by Robert Russell incorrect.  Waiting for SPC-3 to
do something to clarify this isn't going to do much for iSCSI
interoperability in the short term.  Since Bob Snively was the
Technical Editor of FCP-2, he tends to be correct about what FCP-2
requires or intends - I suggest we follow FCP-2, and say that the
O/o/U/u bits are valid in all cases (of course, if none of them
are set, the Residual Count field is not valid).
 
Thanks,
--David
---------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 42 South St., Hopkinton, MA  01748
+1 (508) 249-6449            FAX: +1 (508) 497-8018
[email-masked]       Mobile: +1 (978) 394-7754
---------------------------------------------------

-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Thursday, August 01, 2002 3:26 AM
To: Santosh Rao
Cc: IPS Reflector; [email-masked]; Robert Snively;
[email-masked]; T10 Reflector
Subject: Re: iSCSI: plugfest4 issues

Santosh, 

I think that this behaviour should be specified by SPC3. I looked (again)
into the FCP docs and like iSCSI they do not say anything beyond 
iSCSI says. Like iSCSI they specify that the field is valid when the Oo/Uu
bits are set but nothing about how those bits relate to status. 
SPC says nothing about that either  (beyond that the bits set are not
necessarily an indication of error). 

Julo 

	Santosh Rao <[email-masked]> 
Sent by: [email-masked] 

08/01/2002 03:44 AM 

        To:        IPS Reflector <[email-masked]>, Julian
Satran/Haifa/IBM@IBMIL, [email-masked] 
        cc:        Robert Snively <[email-masked]>, T10 Reflector
<[email-masked]> 
        Subject:        Re: iSCSI: plugfest4 issues 

Julian & Robert [Russell],

I raised the same query regarding RESID for FCP/FCP-2 this time last
year. The response I got for FCP/FCP-2 was that RESID information shall
be valid, regardless of the scsi status returned. The RESID field, can
be checked by the scsi transport drivers independent of the SCSI STATUS.

I have enclosed the T10 response from Rob Snivelly below on that issue.
As per FC-PLDA, the RESID information is valid, regardless of the scsi
status returned by the device. 

An example of this is the case of "NO SENSE" or "RECOVERED ERROR" check
condition, when the data transfer may have taken place and a CHECK
CONDITION is returned. Also, for other CHECK CONDITION status', partial
data transfer may have taken place and hence, resid information should
be present.

It would be good to maintain consistent behaviour across the scsi
transports in this regard, since protocol bridging from iscsi to FCP
domain would expect RESID information in the FCP domain, regardless of
scsi status.

This also allows scsi transports to remain free of SCSI command set
details. (ex : the scsi transport drivers do not need to parse for CHECK
CONDITION or GOO status information.)

Thanks,
Santosh

-------------------------------------------------------------------------

Subject: Re: iSCSI: plugfest4 issues
Date:    Thu, 1 Aug 2002 02:52:19 +0300
From:   "Julian Satran" <[email-masked]>
To:     "Robert D. Russell" <[email-masked]>
CC:     [email-masked]

Bob, 

Thanks - some comments in text. Julo 

 "Robert D. Russell" <[email-masked]> 
                                           
Julian:

Four issues came up today at the iSCSI plugfest:

1. A question about whether or not the Residual Count field and the
 appropriate O and U bits need to be computed on all SCSI Response
 PDUs, regardless of the values in the Status and/or Response fields.

 One point of view says that the Residual Count field and the O and U
 bits appear to be strictly iSCSI values that are derived by the
 iSCSI target layer from the ExpectedDataTransferLength field of the
 SCSI Command PDU and the DataSegmentLength fields of the DataIn or
 DataOut PDUs sent as part of this command.  Therefore ,the iSCSI
 target always computes a Residual Count value without regard to the
 Status and/or Response fields, since these are SCSI values.

 The other point of view says that the Residual Count field, and the
 O and U bits, need only be set when the Status and Response fields
 indicate that the command was completed at the target with a GOOD
 Status, and the target does not have to compute or set the Residual
 Count field and the O or U bits for other values of the Status and/or
 Response fields.

 Which is it?  In any case, could this be clarified somewhere in the
 standard, most likely in section 9.4.4.

+++ Residual count fields are in fact carrioed over from the SCSI layer.
I know that none of the SCSI docs specifies
exactly their behavior and it strikes me as a bad idea to have protocols
specify them. 
The values should be valid any time the target decides to put them in. 
+++

-------------------------------------------------------------------------
Subject: RE: FCP_RSP Residual Checking.
Date:    Thu, 5 Jul 2001 13:18:42 -0700
From:    Robert Snively <[email-masked]>
To:      "'Santosh Rao'" <[email-masked]>, 
        T10 Reflector <[email-masked]>, 
        Fibre Channel T11 reflector <[email-masked]>

Robert Snively wrote:
> 
> >  Is the target required to initialize the fields FCP_RESID_UNDER,
> >  FCP_RESID_OVER & FCP_RESID when any I/O is completed
> >  without the data phase having transferred exactly
> >  FCP_DL bytes, regardless of the SCSI Status being returned ?
> 
> >  When the target generates a CHECK CONDITION on an I/O
> >  and may have returned less than FCP_DL bytes in the data
> >  phase for that I/O, is it
> >  required to set the FCP_RESID_UNDER to 1 and indicate the number of
> >  bytes not transferred in the FCP_RESID field?
> 
> The intent is that the answer to your second question is:
> FCP_RESID should appropriately regardless of the SCSI Status
> being returned.  The classic errors of that class are those
> involving successful completion with reporting, like
> the "NO SENSE" and "RECOVERED ERROR" series of errors.
> 
> >
> >  What is the behaviour initiators can expect under the above
> >  condition ?
> 
> The intent is that there be no conflict.  I believe that FCP-2
> was a bit less bald than FC-PLDA in stating the requirement.
> 
> >  Is there a conflict in the behaviours described by FCP/FCP-2
> >  & FC-PLDA ?
> >
> 
> Bob Snively                        e-mail:    [email-masked]
> Brocade Communications Systems     phone:  408 487 8135
> 1745 Technology Drive
> San Jose, CA 95110
> 
> >  -----Original Message-----
> >  From: Santosh Rao [mailto:[email-masked]]
> >  Sent: Thursday, July 05, 2001 12:15 PM
> >  To: T10 Reflector; Fibre Channel T11 reflector
> >  Subject: FCP_RSP Residual Checking.
> >
> >
> >  All,
> >
> >  I've got a question on target behaviour while sending a
> >  CHECK CONDITION
> >  SCSI status in its FCP_RSP IU.
> >
> >  Is the target required to initialize the fields FCP_RESID_UNDER,
> >  FCP_RESID_OVER & FCP_RESID when any I/O is completed without the data
> >  phase having transferred exactly FCP_DL bytes, regardless of the SCSI
> >  Status being returned ?
> >
> >  When the target generates a CHECK CONDITION on an I/O and may have
> >  returned less than FCP_DL bytes in the data phase for that I/O, is it
> >  required to set the FCP_RESID_UNDER to 1 and indicate the number of
> >  bytes not transferred in the FCP_RESID field?
> >
> >  FC-PLDA Section 8.2.4.1 states that :
> >  "SCSI targets that transfer less than FCP_DL bytes during
> >  the FCP_DATA
> >  IUs shall set the FCP_RESID_UNDER to 1".
> >
> >  No exceptions are specified in the case of a CHECK CONDITION in the
> >  above definition, implying that FCP_RSP residual checking can be
> >  performed irrespective of the SCSI Status that was returned in the
> >  FCP_RSP.
> >
> >  However, the wording descriptions of FCP_RESID_UNDER,
> >  FCP_RESID_OVER &
> >  FCP_RESID in SCSI-FCP & FCP-2 are not as stringent as
> >  FC-PLDA and do not
> >  mandate that FCP_RESID_UNDER shall be set when the data
> >  transferred is <
> >  FCP_DL.
> >
> >  What is the behaviour initiators can expect under the above
> >  condition ?
> >  Is there a conflict in the behaviours described by FCP/FCP-2
> >  & FC-PLDA ?
> >
> >  Thanks,
> >  Santosh Rao
> >

-- 
Education is when you read the fine print. 
Experience is what you get if you don't.

---

From: "Julian Satran" <[email-masked]>
Date: Fri, 2 Aug 2002 08:43:01 +0300

Bob,

Yes disconect is justified if the first reguest is not Login. 

I will add to 2.2.3

A target receiving any PDU except a Login request before the Login phase 
is started MUST immediately terminate the connection on which the PDU was 
received.

Julo

"Robert D. Russell" <[email-masked]>
Sent by: [email-masked]
08/01/2002 04:12 PM

        To:     Julian Satran/Haifa/IBM@IBMIL, <[email-masked]>
        cc: 
        Subject:        Re: iSCSI: plugfest4 issues

Julian:

Would you please clarify your answer to issue 2 below:

Do you mean the target should simply disconnect in all cases of a PDU
received during Login Phase that is not a Login Request PDU, or only
when the first PDU received during Login Phase is not a Login Request?

If the answer is "all cases", then there would appear to be no need
for the Login Response Status code 0x020b "Invalid Request type during
Login" and it should be removed to avoid further confusion.

I believe disconnect is fully justified when the first PDU in Login
Phase is not a Login Request, for both security and implementation
reasons, as stated in a posting by Eddy Quicksall.  However, after the
first Login Request PDU has been successfully received, the Login Phase
has clearly begun and it seems to me easier then to deal with all errors
on the target side, including reception of a non-Login Request PDU, in the
same manner, which is to send back a Login-Reject with the appropriate
status code and then close the connection.  I believe this is why the
0x020b Status code was introduced, and could anticipate the usefulness
of having this if, for example, the initiator concludes Login Phase
and begins sending FFP PDUs, but for whatever reason the target has not
concluded Login Phase and continues to expect to receive only Login
Request PDUs.

Thank you,

Bob Russell

> 2. The last paragraph of section 2.2.3 says:
>
>    "Before the Full Feature Phase is established, only Login Request and
>    Login Response PDUs are allowed. Any other PDU, when received at ini-
>    tiator or target, is a protocol error and MUST result in the connec-
>    tion being terminated. ..."
>
>    The question is the following:  is this rule literally true for the
>    target (i.e., can the target disconnect as soon as it receives a
>    non-Login PDU from the initiator) or does the target have to first
>    send a Login Response with Login reject PDU before disconnecting, as
>    it does for all other errors detected by the target during Login
>    Phase (according to section 4.3.1)?
>
>    A related question is: does the target take the same action when
>    the very first PDU it receives on a new TCP connection is not a
>    Login Request PDU?
>
>    If the target has to send the Login reject PDU before disconnecting,
>    then the last paragraph of section 2.2.3 should be reworded along
>    the following lines (modeled after the last paragraph of section 
4.3):
>
>    "Before the Full Feature Phase is established, only Login Request
>    and Login Response PDUs are allowed.  If the target receives any PDU
>    other than Login Request, it must send a Login reject (code 0x020b)
>    and then disconnect.  If the initiator receives any PDU other than
>    Login Response, it MUST drop the connection. ..."
>
>    This wording would also appear to cover the case of when the very 
first
>    PDU a target receives on a new TCP connection is not a Login Request.
> +++ I would suggest sticking with disconnecting. +++
>

---

From: "Julian Satran" <[email-masked]>
Date: Fri, 2 Aug 2002 08:43:05 +0300

David,

We DO EXACTLY what FCP did  - say nothing.
I went through the document - thetre is no relation mentioned and that is 
what we do too.

In any case we cannot enforce a SCSI behavior.
The expectation is obvious that if SCSI hands obver counts those will be 
carried by iSCSI.

I also suspect that the trouble may be deeper than we think and I find it 
much more prudent 
to say nothing (again as FCP did).

Julo

[email-masked]
08/01/2002 05:56 PM

        To:     Julian Satran/Haifa/IBM@IBMIL
        cc:     [email-masked]
        Subject:        RE: iSCSI: plugfest4 issues

Julian,

Thanks,
--David
-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Thursday, August 01, 2002 3:26 AM
To: Santosh Rao
Cc: IPS Reflector; [email-masked]; Robert Snively; 

Santosh, 

Julo 

Thanks,
Santosh

Bob, 

 
Julian:

---

From: "Julian Satran" <[email-masked]>
Date: Fri, 2 Aug 2002 08:55:03 +0300

Santosh,

I may change the words - thanks - but that will not solve the issue raised 
- that is when are the bits set in relation to the status.

Julo

08/01/2002 08:11 PM

        To:     Julian Satran/Haifa/IBM@IBMIL
        cc:     IPS Reflector <[email-masked]>, Robert Snively <[email-masked]>
        Subject:        Re: iSCSI: plugfest4 issues

Julian,

I agree with you that the wording should be sufficient as called out in
Section 9.4 description of the bits and the resid field.

However, in observance of the fact that non-interopability has been
noticed in this area, it may help to tighten the wording for the
description of the o/O and u/U bits in the "SCSI Response" description
in Section 9.4. Perhaps, the use of "MUST" directive when calling out
the condition for the u/U/o/O bits to be set would help.

Also, In Section 9.4.4, instead of :
"The Residual Count field is only valid in the case where either the U
bit or the O bit is set."

I suggest :
"The Residual Count field MUST be valid when either the U or the O bit
is set".

Similar wording for the Bi-di read residual count.

Thanks,
Santosh

Julian Satran wrote:
> 
> Santosh,
> 
> I think that this behaviour should be specified by SPC3. I looked
> (again) into the FCP docs and like iSCSI they do not say anything
> beyond
> iSCSI says. Like iSCSI they specify that the field is valid when the
> Oo/Uu bits are set but nothing about how those bits relate to status.
>  <[email-masked]>                     To:        IPS Reflector
>  Sent by:                          <[email-masked]>, Julian
>  [email-masked]        Satran/Haifa/IBM@IBMIL,
>                                    [email-masked]
>  08/01/2002 03:44 AM                       cc:        Robert Snively
>                                    <[email-masked]>, T10
>                                    Reflector <[email-masked]>
>                                            Subject:        Re: iSCSI:
>                                    plugfest4 issues
> 
> 
> 

-- 
Finish each day and be done with it.  You have done what you could;
Some blunders and absurdities have crept in; Forget them as soon as you
can.
Tomorrow is a new day; You shall begin it serenely and with too high a
spirit 
to be encumbered with your old nonsense.

---

From: [email-masked]
Date: Fri, 2 Aug 2002 18:50:55 -0400

Julian,
 
Matching FCP's wording resulted in a "go ask the expert" algorithm
for figuring out what is required - we ought to do better, and this
is an iSCSI protocol issue.  What I'm looking for in 9.4 is:
 
- Bits 3-6 (o, u, O, U) MAY be set by the Target and SHOULD be checked
    by the Initiator when the Response is "Command Completed at Target"
    no matter what the value of the Status field is.
- SCSI permits Residual Counts to be returned for Status values other
    than GOOD; see [SPC2].
 
I believe that both of these are implied by the current text, but since
this caused confusion at the plugfest, we ought to spell it out.
 
Thanks,
--David

-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Friday, August 02, 2002 1:43 AM
To: [email-masked]
Cc: [email-masked]
Subject: RE: iSCSI: plugfest4 issues

David, 

Julo 

	[email-masked] 

08/01/2002 05:56 PM 

Julian, 

Thanks, 
--David 

-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Thursday, August 01, 2002 3:26 AM
To: Santosh Rao
Cc: IPS Reflector; [email-masked]; Robert Snively;

Santosh, 

Julo 

08/01/2002 03:44 AM 

Thanks,
Santosh

Bob, 

                                          
Julian:

---

From: "Julian Satran" <[email-masked]>
Date: Sat, 3 Aug 2002 08:13:37 +0300

David,

We are in violent agreement. SPC3 is explicit about counts only when 
talking about extended copy.
The only thing I find objectionable is to tell the initiator when to CHECK 
(that is a device class/device/driver issue).
Some devices for some commands will generate CHECK CONDITION BECAUSE they 
have a residual count.

That is why I would suggest the following text:

The Residual Count field MUST be valid in the case where either the U bit 
or the O bit is set. If neither bit is set, the Residual Count field is 
reserved. Targets may set residual count and initiators may use it when 
the response code is completed at target. If the O bit is set, the 
Residual Count indicates the number of bytes that were not transferred 
because the initiator's Expected Data Transfer Length was not sufficient. 
If the U bit is set, the Residual Count indicates the number of bytes that 
were not transferred out of the number of bytes expected to be 
transferred.

Julo

[email-masked]
08/03/2002 01:50 AM

Julian,

 
 
Thanks,
--David
-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Friday, August 02, 2002 1:43 AM
To: [email-masked]
Cc: [email-masked]
Subject: RE: iSCSI: plugfest4 issues

David, 

Julo 

[email-masked] 
08/01/2002 05:56 PM 

Julian, 

Thanks, 
--David 
-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Thursday, August 01, 2002 3:26 AM
To: Santosh Rao
Cc: IPS Reflector; [email-masked]; Robert Snively; 

Santosh, 

Julo 

Thanks,
Santosh

Bob, 

 
Julian:

---

From: [email-masked]
Date: Mon, 5 Aug 2002 11:53:55 -0400

Julian,
 
Text is ok, but please add a sentence to avoid the confusion over
Status that occurred at the plugfest - SCSI permits
a residual count to be returned for status values other than GOOD.
 
Thanks,
--David

-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Saturday, August 03, 2002 1:14 AM
To: [email-masked]
Cc: [email-masked]
Subject: RE: iSCSI: plugfest4 issues

David, 

We are in violent agreement. SPC3 is explicit about counts only when talking
about extended copy. 
The only thing I find objectionable is to tell the initiator when to CHECK
(that is a device class/device/driver issue). 
Some devices for some commands will generate CHECK CONDITION BECAUSE they
have a residual count. 

The Residual Count field MUST be valid in the case where either the U bit or
the O bit is set. If neither bit is set, the Residual Count field is
reserved. Targets may set residual count and initiators may use it when the
response code is completed at target. If the O bit is set, the Residual
Count indicates the number of bytes that were not transferred because the
initiator's Expected Data Transfer Length was not sufficient. If the U bit
is set, the Residual Count indicates the number of bytes that were not
transferred out of the number of bytes expected to be transferred. 

Julo 

	[email-masked] 

08/03/2002 01:50 AM 

Julian, 

  
  
Thanks, 
--David 

-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Friday, August 02, 2002 1:43 AM
To: [email-masked]
Cc: [email-masked]
Subject: RE: iSCSI: plugfest4 issues

David, 

Julo 

	[email-masked] 

08/01/2002 05:56 PM 

Julian, 

Thanks, 
--David 

-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Thursday, August 01, 2002 3:26 AM
To: Santosh Rao
Cc: IPS Reflector; [email-masked]; Robert Snively;

Santosh, 

Julo 

08/01/2002 03:44 AM 

Thanks,
Santosh

Bob, 

                                         
Julian: