Subject: Re: iSCSI: draft 7: iSCSI response and SCSI sense data

From: "Jim Hafner" <[email-masked]>
Date: Wed, 1 Aug 2001 16:42:33 -0700

Rob,

I agree with you.  These layering violations (and some of the others you've
also noted on the reflector) should be removed from the draft.

Jim Hafner

"Elliott, Robert" <[email-masked]>@ece.cmu.edu on 08-01-2001
04:26:57 PM

Sent by:  [email-masked]

To:   "[email-masked]'" <[email-masked]>
cc:
Subject:  iSCSI: draft 7: iSCSI response and SCSI sense data

iSCSI draft 7 section 2.4.3 includes a table relating the iSCSI
response codes to various CHECK CONDITION status additional
sense codes:

  "0x01 Target Failure
    ASC=0x44 ASCQ=0x00 INTERNAL TARGET FAILURE
  0x02 Delivery Subsystem Failure
    ASC=0x47 ASCQ=0x03 INFORMATION UNIT CRC ERROR DETECTED
  0x03 Unsolicited data rejected
    ASC=0x49 ASCQ=0x00 INVALID MESSAGE ERROR
  0x04 Not enough unsolicited [data]
    ASC=0x49 ASCQ=0x00 INVALID MESSAGE ERROR
  0x05 SNACK rejected [Command in progress]
    ASC=0x47 ASCQ=0x03 INFORMATION UNIT CRC ERROR DETECTED

  As listed above, each defined response code MUST be used (under the
  conditions described in the 'Reason' field), only when the
  corresponding SCSI CHECK CONDITION is signaled, to convey additional
  protocol service information.  A SCSI CHECK CONDITION with the ASC
  and ASCQ values as tabulated MUST be signaled by iSCSI targets for
  all the instances in this document referring to usage of one of the
  above defined response codes."

No other SCSI protocol does this. iSCSI seems to be granting the
SCSI-level status a priority over the iSCSI-level response data.
I think the other protocols treat the response data as more
important; if the response data indicates a problem, the SCSI layer
is considered broken and the SCSI status is undefined.

Forcing the iSCSI layer to fill in those fields seems to break the
layering model.

I suggest removing the CHECK CONDITION status and additional sense
code linkage and leave the SCSI status undefined if the iSCSI
response is nonzero.

This is compatible with SAM-2, which only requires status be returned
when the service response is TASK COMPLETE (SAM-2 revision 18
section 5.3.1):
  "Status shall be sent from the logical unit to the application
   client whenever a command ends with a service response of
   TASK COMPLETE or LINKED COMMAND COMPLETE."

Furthermore, SAM-2 requires that status be ignored when the service
response indicates an error (SAM-2 revision 18 section 5.1):
  "Status: A one-byte field containing command completion status
  (see 5.3). If the command ends with a service response of
  SERVICE DELIVERY OR TARGET FAILURE, the application client
  shall consider this parameter to be undefined."

I think iSCSI's current wording may violate this rule.
---
Rob Elliott, Compaq Server Storage
[email-masked]

---

From: "Julian Satran" <[email-masked]>
Date: Thu, 2 Aug 2001 08:49:47 +0300

Steph,

You change opinions faster than draft changes :-)

David,

Could you please quote us which part of SAM-2 this is violating. Please
keep in mind
that SAM has 2 return parameters in its Remote Procedure Call model that
can be sent independently.
FTP has following the older standards and set them exclusively but that is
not IMHO required by SAM.
ACA or not - we do require an interlocking mechanism. ACA can't be broken
that easy and certainly not that
total as you suggest.  And if we do not use the SCSI interlocking we will
have to invent a brand new one of our own.

And a procedural call - please do not quote layering as a reason for
tearing something down. Please quote a way of doing interlocking that is as
cheap as this and does not violate layering and I'll buy it.

A second line of argument says that layering is not a goal.

And a third says that all SCSI protocols access both SCSI level and
protocol specific functions that are common to all at the "SCSI levels".
That type of engineering is very common in computing (OSes are built this
way).
Interlocking is a problem for many protocols (not only iSCSI) and its
solution (for all) is at SCSI level. And ACA is a way to do interlocking
(there are not many others and yhey are all described in SAM).
If having it described in the same book means violating layering so be it
although I think that you are mistaken.

Regards,
Julo

[email-masked] on 02-08-2001 04:19:27

Please respond to [email-masked]

To:   Julian Satran/Haifa/IBM@IBMIL, [email-masked]
cc:
Subject:  RE: iSCSI: draft 7: iSCSI response and SCSI sense data

In my lousy Indiana Jones impression - "ACA, why does it
always have to be ACA?" ....

This whole notion that ACA is a fundamental coordination
mechanism for shared SCSI access to a target is not headed
in a good direction - see the recent message from Ralph Weber
expressing T10's concern that expansion of the usage of ACA is
an invitation to undesirable denial of service possibilities.
I will note that the reservation example that Ralph used in his
message is the sort of thing that can break clustering software,
although I don't know for certain whether there's any existing
clustering software that it does in fact break.

I think Rob and Jim are correct when they describe the current
2.4.3 text as being at odds with SAM-2.  So, I think:

- The text in 2.4.3 that is at odds with SAM-2 should come out.
     The fact that it was caused by a desire to use ACA makes
     it doubly suspect.
- The current mandate for ACA in Section 8.2 exceeds the level
     stated in the iSCSI requirements draft and I don't believe
     it represents rough consensus of the WG.  Support for
     ACA should be a "SHOULD", not a "MUST".
- The entire effort to use ACA or something like it to serialize
     error conditions for multiple commands in flight has not been
     explained well to T10, and what has been explained has not
     received a good reception.

The last bullet has a couple of consequences.  First, we should
take out anything else that's crept into iSCSI in support of this
use of ACA.  Second, those who want to use ACA or something ACA-like
for this error coordination across multiple commands in flight
should write up a submission to T10 explaining what the problem
is and how ACA or something like it solves the problem.

I think Ralph is conveying a message from T10 that whatever the
problem iSCSI has, T10 is not inclined to believe that more use
of ACA is NOT the solution.  It may take a while to work with T10
to get the right solution designed, but I'd prefer doing nothing
over force-fitting ACA to a use that T10 doesn't think is appropriate.
When we figure out what the right thing is, we can put in the right
text to describe it.

Comments?
--David

---------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 42 South St., Hopkinton, MA  01748
+1 (508) 435-1000 x75140     FAX: +1 (508) 497-8500
[email-masked]       Mobile: +1 (978) 394-7754
---------------------------------------------------

> -----Original Message-----
> From: Julian Satran [mailto:[email-masked]]
> Sent: Wednesday, August 01, 2001 8:01 PM
> To: [email-masked]
> Subject: Re: iSCSI: draft 7: iSCSI response and SCSI sense data
>
>
>
> Elliot,
>
> This issue has some history.  After many took the position you are
> advocating
> and I had to drop the check condition all list participants that had
> something to say adopted the current position in order the
> benefit from the
> "serializing" effect of ACA whenever a check condition is
> originated at the
> target and the delivery subsystem is still operational.
>
> Julo
>
>
> "Elliott, Robert" <[email-masked]> on 02-08-2001 02:26:57
>
> Please respond to "Elliott, Robert" <[email-masked]>
>

---

From: [email-masked]
Date: Thu, 2 Aug 2001 10:50:08 -0400

I've got to stop writing email late at night ...

> of ACA is NOT the solution.

That should have been a single negative, not double. i.e., "T10
is inclined to believe that more use of ACA is NOT the solution."

> It may take a while to work with T10

At least that's still correct :-).

Sorry/Thanks,
--David

> From: [email-masked] [mailto:[email-masked]]
> Sent: Wednesday, August 01, 2001 9:19 PM
> To: [email-masked]; [email-masked]

---

From: "Julian Satran" <[email-masked]>
Date: Thu, 2 Aug 2001 18:37:54 +0300

That is the current situation - statuse are generated (and responses) are
generated at the target.

Sorry - Steph - I am wrong and the right guy to blame for anything :-)

Julo

Stephen Bailey <[email-masked]> on 02-08-2001 16:48:34

Please respond to Stephen Bailey <[email-masked]>

To:   Julian Satran/Haifa/IBM@IBMIL
cc:
Subject:  Re: iSCSI: draft 7: iSCSI response and SCSI sense data

> You change opinions faster than draft changes :-)

Actually, I think I do a bad job of explaining myself.

You already chastised me for a `reversal' on this topic.

My position, stated the last time you claimed I reversed (just a few
weeks ago, I think), is:
  1) target should report ALL forms of task completion using SCSI status
  2) the initiator should NOT synthesize SCSI status.

My originally stated opinion had to do with 2).  I never argued with
the target `synthesizing' SCSI status, because the target is NOT
synthesizing SCSI status, it's generating it.

I agree that I have been confused about the role of target-generated
response codes WRT to request protocol errors (that's FCP data length
& invalid FCP_CMND response codes).  I have decided that the FCP
approach is wrong and the ||SCSI approach is right.  The FCP targets
that I like use the ||SCSI approach anyway.

How many more times can I say uncle?  I said it on the list (check the
archive).  I'm saying it here.  You want me to say it on the list
again?

Are you ever wrong about anything?

Steph

---

From: "Julian Satran" <[email-masked]>
Date: Thu, 2 Aug 2001 19:58:21 +0300

David,

That's what will probably end-up doing.

Julo

[email-masked] on 02-08-2001 17:21:25

> Could you please quote us which part of SAM-2 this is
> violating.

> I think iSCSI's current wording may violate this rule.

So, just generate the CHECK CONDITION and don't try to use the
iSCSI Service Response to further qualify it, as Mallikarjun
suggests:

> Robert Elliott pointed out a violation by iSCSI in trying to
> additionally qualify ASC, ASCQ with its response codes.  I suggest
> that dropping this "additional description" attempt in all these
> cases is all iSCSI needs to do to meet SAM's expectations.

Thanks,
--David

---

From: Robert Snively <[email-masked]>
Date: Thu, 2 Aug 2001 14:38:38 -0700

I am not sure that complete removal of the response codes 
is the best approach.  There are some things
that SCSI devices can know nothing about, and only TCP links
can know about.  There are other things that iSCSI state machines
know about that logical units do not know about (usually software
generated inconsistencies in the protocol procedures).  These are
natural candidates for response codes.  Note that vendor 
specific is a synonym for non-interoperable and also for
undesirable.  It is far better to reserve anything like that,
since the behavior in the presence of a vendor specific value
is not defined.  While it is clear that may response codes should
move over to CHECK CONDITIONS, I think that there are
still some protocol related response codes that will be required.

Bob

-----Original Message-----
From: Mallikarjun C. [mailto:[email-masked]]
Sent: Wednesday, August 01, 2001 9:06 PM
To: Elliott, Robert
Cc: [email-masked]'
Subject: Re: iSCSI: draft 7: iSCSI response and SCSI sense data

Robert and all,

As the one who suggested that wording into the draft, let me 
add some comments. (Please see
http://www.pdl.cmu.edu/mailinglists/ips/mail/msg05089.html
for my initial proposal.)

> code linkage and leave the SCSI status undefined if the iSCSI

I agree, as Steph already alluded to in a separate email, the ideal 
solution appears to: 
          a) drop the response code *definitions* (ie. leave
             room for vendor-unique codes) altogether, and
          b) continue to specify the right SCSI CHECK CONDITION 
             (ASC, ASCQ) values for the iSCSI error events (such
             as digest failures).

As I said in the earlier referenced message "mandate a standard 
SCSI CHECK CONDITION on these errors with the current response codes 
acting as detailed descriptions.".  But you bring up a good point 
that this "detailed descriptions" may be violating SAM-2's expectations. 

> Forcing the iSCSI layer to fill in those fields seems to break the

I disagree.  If the affected SCSI task can be reliably ascertained
by the target, it appears highly desirable to report the status 
using a SCSI CHECK CONDITION.  Target iSCSI layer has to internally
terminate the pre-instantiated SCSI task anyway informing its SCSI 
layer of the fact.  Besides, there are precedents in FCP-2 for similar
cases where the task is identifiable.

Regards.
-- 
Mallikarjun 

Mallikarjun Chadalapaka
Networked Storage Architecture
Network Storage Solutions Organization
MS 5668 Hewlett-Packard, Roseville.
[email-masked]

---

From: "Julian Satran" <[email-masked]>
Date: Mon, 6 Aug 2001 23:27:47 +0300

That will change - Julo

"Eddy Quicksall" <[email-masked]>@ece.cmu.edu on 06-08-2001
22:22:42

Please respond to "Eddy Quicksall" <[email-masked]>

To:   "Elliott, Robert" <[email-masked]>, <[email-masked]>
cc:
Subject:  Re: iSCSI: draft 7: iSCSI response and SCSI sense data

Then what was the purpose of giving them separate fields and eliminating
the
S bit?

Eddy

----- Original Message -----
From: "Elliott, Robert" <[email-masked]>
To: <[email-masked]>
Sent: Thursday, August 02, 2001 6:04 PM
Subject: RE: iSCSI: draft 7: iSCSI response and SCSI sense data

> My view:
> If a non-zero response code is present, the Status field is undefined
> and the response data contains all information about the error (fed
> from the iSCSI layer).
>
> If the target is capable of reporting a Status, then the response
> field is zero and the sense data contains all information about the
> error (fed from the SCSI layer, which might communicate with the
> iSCSI layer internally to figure out some of the information).
>
> ---
> > From: Robert Snively [mailto:[email-masked]]
> > Sent: Thursday, August 02, 2001 4:39 PM
> > To: [email-masked]'; Elliott, Robert
> > Cc: [email-masked]'