Subject: iSCSI : Initiators expected to fake CHECK CONDITIONS.

From: Santosh Rao <[email-masked]>
Date: Tue, 09 Jan 2001 19:05:43 -0800

Julian,

My comments embedded below.

Thanks,
Santosh

> Section 4.4. Format Errors
> ====================
> Quoting from the draft :

> "When a session is active whenever an initiator receives an iSCSI PDU
>  with a format error, for which it has an outstanding task, it MUST
>  abort the target task and report the error as a SCSI check condition
>  status with a sense key of 4h (hardware error)."

> What is the reasoning behind this ? It seems to complicate Format
Error
> Handling with no clear benefit. Is it the expectation that iSCSI
> initiators
> will fake a format error as a CHECK CONDITION back to the SCSI
>Layer ? If so, why ? And why the choice of "Hardware Error" ?

[js} the error was generated by a faulty controller and I did not find
any
other SCSI sense fit for it[/js]

A clean layering should be maintained between iSCSI layer errors and
SCSI layer errors. A format error on a iSCSI PDU header does not
constitute a SCSI error. The specific error returned back to the
SCSI upper layer driver in this case is really O.S. specific since each
O.S. has its own error codes to deal with interconnect transport errors.

Expecting initiators to fabricate sense data on behalf of a target and
faking a CHECK CONDITION back to the upper layers is :
a) violating layering b/n iSCSI and SCSI layers.
b) adds un-necessary complexity to the initiator which now needs to
build sense data internally on behalf of a target.
c) will cause different error recovery paths to be taken in the upper
layer SCSI driver which is expecting to see some std O.S. defined error
codes to be returned on interconnect transport errors.

> Last, but not the least, the above statement is referring to all iSCSI

> PDUs and advocates that iSCSI initiators must fake a CHECK CONDITION
> back to the SCSI layer on any format error of any iSCSI PDU.

[js] only if a task is running [/js]

I'm somewhat confused here. Let us say a format error occurs on a
Logout. What is the error recovery to be done in this case if there are
other outstanding tasks ? Is this section advocating erroring ALL the
outstanding tasks [that did not encounter format error] ?

> This is not applicable for non-scsi iSCSI PDUs which have no SCSI
> context. I believe the REJECT PDU should only be used for non-scsi
> PDUs since Rejects to SCSI PDUs will need a different type of reason
> code/explanation information best fitted into the existing SCSI
Response

---

From: [email-masked]
Date: Sun, 14 Jan 2001 15:50:07 +0200

Santosh,

I understand your concerns.

The previous versions had the response field in the response PDU as a
summary of several format errors.  The format was selected to be close to
FCP.  FCP cumulates there the task management responses (for which we have
a separate  PDU) and the format errors.

Surfacing the Service Response can be accomplished either through a command
response  or through a  command reject  and we decided that with the format
we have we can give more details than with the response field.

However we could not find any mapping of the a service response neither in
SAM nor in FCP nor in CAM.  We could leave this mapping to be
implementation dependent (and that is a perfectly valid choice as it has to
be supported by the OS) or we could map it into a Status and Sense and we
though that this is closer to what we would like an initiator to do in
order to express its unhappiness with a specific target implementation
(regardless of the sacrosanct layering).

Nevertheless we are open to suggestions.

Regards,
Julo

Santosh Rao <[email-masked]> on 13/01/2001 06:14:20

Please respond to Santosh Rao <[email-masked]>

To:   Julian Satran/Haifa/IBM@IBMIL
cc:   [email-masked]
Subject:  Re: iSCSI : Initiators expected to fake CHECK CONDITIONS.

Julian,

I had 2 concerns on the format error handling issue :

1) This error recovery is applicable for the SCSI Command PDU. What is the
error recovery for a format error on a Login, Logout, Text, NOP-OUT,
NOP-IN,
SCSI Task Management Command PDU ? These commands are either :
not from the scsi layer and their responses do not get sent up the stack to
the scsi layer.
or
in the case of scsi task mgmt cmds, the response does not contain sense
data.

How are the above to be handled ?

2) On the concept of initiators generating sense data on behalf of targets
and
faking a HARDWARE ERROR, this is not the approach parallel scsi and fibre
channel take. In the case of fibre channel, a header format error results
in
Response Info being generated in FCP_RSP. (for scsi pdu's). However, I
noticed
that the latest version of the iSCSI draft has removed all references to
response length and response data, and hence, it looks like this approach
is
not being considered (?)

Generating sense data in an initiator is adding complexity and makes iSCSI
HBA
drivers SPC-2 aware in terms of the content and generation of sense data.
Also, we will need to check some implementations of the upper scsi layer
drivers to see if this can cause any different form of error recovery to be
taken such as issuing a BDR to recover from HARDWARE ERRORs ?

Regards,
Santosh

> Subject:  Re: iSCSI : Initiators expected to fake CHECK CONDITIONS.
> Date:    Fri, 12 Jan 2001 09:41:36 +0200
> From:  [email-masked]
> To:      [email-masked]

> Santosh,

> A machine producing a format error is DEFECTIVE and this a reported as
> hardware error by SCSI (as SCSI does not distinguish between hardware and
> firmware and rightfully so).

 - santoshr.vcf

---

From: Stephen Byan <[email-masked]>
Date: Mon, 15 Jan 2001 07:56:02 -0800

I completely concur with Stephen Bailey. Protocol errors belong to a
different name-space than SCSI status. iSCSI cannot hijack SCSI status codes
for protocol errors. iSCSI could union it's protocol-dependent error field
with the SCSI status field, as long as protocol-dependent errors are defined
as returning a service response of SERVICE DELIVERY OR TARGET FAILURE. (SAM
clearly states that the SCSI status value is invalid if the service response
is SERVICE DELIVERY OR TARGET FAILURE.)

If the protocol error field is unioned with the SCSI status field, it should
be given a different member name - similar to an anonymous union in C - in
order to prevent confusion regarding its semantics. In particular, there is
no allegiance created for protocol errors.

Regards,
-Steve

Steve Byan
<[email-masked]>
Design Engineer
MS 1-3/E23
333 South Street
Shrewsbury, MA 01545
(508)770-3414
fax: (508)770-2604

---

From: [email-masked]
Date: Tue, 16 Jan 2001 12:52:55 +0200

That is a good approximation although I am not sure about all of them.

Julo

Santosh Rao <[email-masked]> on 16/01/2001 09:49:28

Julian,

Thanks for accepting this change into the draft.

I hope I understood your resolution correctly as follows :

The service response exchanged between the LLP and ULP is specified
by each O.S' SCSI Stack  and the draft is not required to elaborate
on this.

The service response exchanged b/n the target and initiator in PDU
responses must be specified by the draft. (Response field
in SCSI Task Mgmt Response, Response Data in SCSI Response PDU, Reason
field in Reject PDU). Targets and Initiators will indicate a service
response of "Service Delivery or Target Failure" in case of header format
errors.

Thanks & Regards,
Santosh

>
>
>
> Santosh,
>
> OK - you have convinced me (and I've found the CAM3 list of statuses).  I
> will specify in 04 the relevant values for the service response in line
> with CAM3.
>
> Julo
>
> Santosh Rao <[email-masked]> on 15/01/2001 00:34:36
>
> > It would be simple if things where that clear-cut.
> >
> > What is a format error coming from a target?  IMHO it is a target error
> and
> > not a protocol failure.
>
> Julian,
>
> A SCSI Protocol(i.e. iSCSI) header format error constitutes a
> "Service Delivery or Target Failure" service response, since the failure
> occurs in the service delivery sub-system (SCSI Protocol). SAM-2 explains
> this service response as "The command has ended due to a service delivery
> failure or target device mal-function", which should be the
> service response to be returned on a header format error.
>
> Further, SAM-2 Section 5 also states that the application client
> (i.e. the SCSI upper layer driver) shall treat the SCSI Status to be
> un-defined if the command ends with a service response of
> "Service Delivery or Target Failure".
> This implies that the SCSI Upper Layer driver shall ignore the
> CHECK CONDITION scsi status that iSCSI initiators are going to fake to
> their upper layers on a header format error, due to a service response of
> "Service Delivery or Target Mal-function".
>
> So, is iSCSI proposing to return a service response of "Task Complete" to
> its upper layers on a header format error ? If not and it is the intent
> that a service response of "Service Delivery or Target Mal-function"
> should be returned, then, the upper layer scsi driver is going to ignore
> the CHECK CONDITION that iSCSI intends to fake.
>
> It would be interesting to hear what T10 has to say on this approach that
> iSCSI is currently proposing to adopt on a header format error.
>
> I see the following problems with the current approach being advocated :
> ------------------------------------------------------------------------
>
> 1) It does NOT solve the problem for commands other than the SCSI Command
> PDU. I am yet to see a response to my questions on how header format
> errors for Login, Logout, Text, NOP-OUT, NOP-IN and SCSI Task Management
> Command PDU are handled by the current proposal described in section 5.4
> of the 03 iSCSI draft.
>
> 2) It is not in line with [violates ?] Section 5 of SAM-2.
>
> 3) It can cause upper layers to initiate Auto Contingent Allegiance
> recovery such as CLEAR ACA due to the CHECK CONDITION returned.
>
> 4) It can cause upper layers to over-react on seeing a HARDWARE ERROR by
> resorting to error recovery such as BDR to recover from a HARDWARE ERROR.
>
> 5) It adds extra complexity to the iSCSI initiator drivers by making them
> SPC-2 aware and having to generate sense data on behalf of a target,
> not something that has been required in other SCSI Protocols such as
> parallel scsi and fibre channel.
>
> 6) It causes a mis-leading error log of a
> HARDWARE ERROR, where, a more specific error log of the
> header format error that occurred based on the response data would
> have been more useful in quick fault isolation.
>
> 7) Last, but not the least, it is a violation of layering between the
> ULP and LLP, wherein, an LLP Service Delivery Failure is being treated as
> a ULP SCSI error returned by the LUN.
>
> I would like to propose the following solution :
> ------------------------------------------------
> 1) On discovery of header format error, a target MUST convey the
> specific type of format error that was discovered through use of
> mechanisms like the Response Data in a SCSI Response PDU,
> Response Field in a SCSI Task Management Response PDU
> and the Reason Code field in the REJECT PDU.
>
> 2) On a header format error discovered at the initiator,
> a service response of "Service Delivery or Target Failure" MUST be
> returned to their upper layers [and the initiator may log the
> specific header format error that was discovered.]. The draft need not
> attempt to elaborate on these service response definitions since these
are
> defined by the SCSI Stack of each O.S. as a set of return values
exchanged
> b/n the LLP and the ULP.
>
> Benefits of the proposed solution :
> -----------------------------------
> 1) It addresses the problem for header format errors on ALL types of
PDUs.
>
> 2) It is in line with [complies with ?] SAM-2 semantics of "Execute
> Command".
>
> 3) It provides more value-added error logging based on the specific
header
> format error that occurred rather than a more general HARDWARE ERROR,
> allowing for quicker isolation and root cause of the problem.
>
> 4) It avoids ULPs resorting to inappropriate error recovery such as CLEAR
> ACA or a BDR on seeing CHECK CONDITION with HARDWARE ERROR.
>
> 5) It retains layering semantics and differentiates between LLP service
> delivery errors and ULP SCSI errors.
>
> I'd be happy to learn what implications of this issue I've missed.
>
> Thanks & Regards,
> Santosh Rao
>
>
>
>
>
>
> > Should target errors be reported in the service-response?   Except for
> task
> > management that is common to all protocols I did not see any other
thing
> > popping up in any SCSI driver.
> >
> > Preaching layering won't make the issue disappear.
> >
> > Julo
> >
> > Stephen Bailey <[email-masked]> on 14/01/2001 17:10:56
> >
> > Please respond to Stephen Bailey <[email-masked]>
> >
> > To:   [email-masked]
> > cc:
> > This seems like the zillionth time aired this same disagreement.
> >
> > I think we should try to reach a WG consensus on whether iSCSI should
> > use SCSI status as a means for reporting protocol-related errors, and
> > kill it once and for all.
> >
> > I'm strongly against it.
> >
> > > And an error in the iSCSI layer gets reported by the next layer -
that
> is
> > > the regular layering technique (and BTW I am getting a bit uneasy
about
> > all
> > > this preaching on layering when it is not obvious that you understand
> all
> > > the implications of the point).
> >
> > Santosh seems to understand 100%.  I agree with Santosh.
> >
> > SAM defines three pieces of status returned by Execute Command()
> > (which is, in turn implemented by each SCSI protocol, e.g. iSCSI):
> >   1) Service Response: task complete, linked command complete, service
> >      delivery or target failure.
> >   2) SCSI status byte
> >   3) SCSI sense data
> >
> > SAM clearly suggest that a protocol's means for signalling
> > protocol-detected errors is the service response status when it says:
> >
> >   The actual protocol events corresponding to a response of TASK
> >   COMPLETE, LINKED COMMAND COMPLETE or SERVICE DELIVERY OR TARGET
> >   FAILURE shall be specified in each protocol standard.
> >
> > Note that defining protocol error events in terms of TC, LCC, SDF and
> > TF, remains an abstraction.  An actual implementation can chose to
> > signal these events by whatever means.  All SCSI implementations I've
> > seen do have a status return component which exactly corresponds to
> > SAM's service response status, but has more than just these four
> > alternatives.  Typically, that includes things like success, command
> > timeout, addressing failure (selection timeout in ||SCSI, bad AL_PA in
> > FCAL, etc.), command aborted, bus parity error, etc..  What iSCSI does
> > need to do is clearly define its error events AS protocol events,
> > which is what describing them in terms of the SAM specified set does.
> >
> > ||SCSI, FCP and the other SCSI protocol standards do this.
> >
> > Operationally, this means that in iSCSI:
> >
> >   o protocol-specific errors for a task detected by the target without
> >     a CLEARLY corresponding SCSI error return should be signalled
> >     using the iSCSI response mechanism.
> >
> >     The initiator will handle these errors by recording them in some
> >     appropriate way, and selecting an appropriate service response
> >     status value.
> >
> >   o errors for a task detected directly by the initiator are handled
> >     by recording them in some appropriate way, and selecting an
> >     appropriate service response status value.
> >
> > I don't know that there's a single sentence or section in SAM which
> > says this, but it clearly implies that the components of SCSI status
> > (status byte and sense) are data which are CARRIED (not created) by
> > SCSI protocols, for the use of the protocol-independent components.
> > For example, a disk peripheral driver reacts to SCSI status and sense
> > generated by disks for SCSI operations that it starts.
> >
> > SCSI status should be equivalent to the SCSI status returned by the
> > logical unit.
> >
> > SCSI protocols are not citizens of the SCSI status space, which means
> > that the real citizens (the command standards, and SAM), may define
> > semantics of SCSI error codes which conflict with iSCSI's selections.
> > The `hardware error' sense key might be defined to mean something very
> > specific within a particular SCSI peripheral command set, and your
> > choice of synthesizing SCSI status within the protocol could cause
> > this behavior to misfire.
> >
> > > any other SCSI sense fit for it[/js]
> >
> > That's because there IS no SCSI sense fit for it.  It is a
> > protocol-detected, protocol-unique error.  Therefore it should be
> > signalled using the service response status.
> >
> > Steph
> >
> >
> >
> >
>
>
> --
> #################################
> Santosh Rao
> Software Design Engineer,
> HP, Cupertino.
> email : [email-masked]
> Phone : 408-447-3751
> #################################
>
>
>
>

--
#################################
Santosh Rao
Software Design Engineer,
HP, Cupertino.
email : [email-masked]
Phone : 408-447-3751
#################################