Return-Path: X-Sieve: cmu-sieve 2.0 Return-Path: Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id h0EGG5205308 for ips-outgoing; Tue, 14 Jan 2003 11:16:05 -0500 (EST) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from mail.ivivity.com ([64.238.111.99]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id h0EGFwW05293 for ; Tue, 14 Jan 2003 11:16:02 -0500 (EST) Received: by ATLOPS with Internet Mail Service (5.5.2653.19) id ; Tue, 14 Jan 2003 11:15:56 -0500 Message-ID: From: Eddy Quicksall To: Julian Satran Cc: Black_David@emc.com, dcuddihy@attotech.com, "Elliott, Robert (Server Storage)" , ips@ece.cmu.edu Subject: RE: iSCSI: Implicit Termination of Tasks Date: Tue, 14 Jan 2003 11:15:48 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2BBE8.3330FB10" Sender: owner-ips@ece.cmu.edu Precedence: bulk This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2BBE8.3330FB10 Content-Type: text/plain; charset="iso-8859-1" Yes and this is where I've been coming from since I started this thread. 6.5 should be changed as Bob pointed out below. I can't tell from your response "This is perfectly reasonable" if you are agreeing that Bob's suggestion is reasonable or if the current wording is reasonable. Please clear that up. Eddy -----Original Message----- From: Julian Satran [mailto:Julian_Satran@il.ibm.com] Sent: Tuesday, January 14, 2003 10:34 AM To: Eddy_Quicksall@iVivity.com Cc: Black_David@emc.com; dcuddihy@attotech.com; Elliott, Robert (Server Storage); ips@ece.cmu.edu Subject: RE: iSCSI: Implicit Termination of Tasks Eddy, Setting the UA is a purely SCSI matter and so is the code used. There is no iSCSI state surviving and we assume that the isCSI initiator produces some termination code for the SCSI initiator. It is not clear to me than anything besides "ACA active" code is needed. If the target messes with command from other initiators (single queue-per-target) those should be ready to accept the consequences of a common queue. UA is used (IMHO) mostly in cases of reset - that kill commands in other queues even when the operation is queue-per-initiator. Julo "Eddy Quicksall" 14/01/03 16:39 Please respond to To Julian Satran/Haifa/IBM@IBMIL cc , , , "Elliott, Robert \(Server Storage\)" Subject RE: iSCSI: Implicit Termination of Tasks >>nothing specific has to survive for ACA For ACA, doesn't the next command get the UA then the additional commands get ACA Active until ACA is cleared (all assuming NACA, the ACA attribute and QErr are being used appropriately)? So, doesn't the UA have to survive? Eddy -----Original Message----- From: Julian Satran [mailto:Julian_Satran@il.ibm.com] Sent: Tuesday, January 14, 2003 3:50 AM To: Elliott, Robert (Server Storage) Cc: Black_David@emc.com; dcuddihy@attotech.com; Eddy Quicksall; ips@ece.cmu.edu Subject: RE: iSCSI: Implicit Termination of Tasks Rob, This is perfectly reasonable. The clearing effects will wipe-out anyhow whatever was specific to the code for session termination and nothing specific has to survive for ACA. Regards, Julo "Elliott, Robert (Server Storage)" 14/01/03 09:59 To "Eddy Quicksall" , , , Julian Satran/Haifa/IBM@IBMIL, cc Subject RE: iSCSI: Implicit Termination of Tasks A few comments on this thread: * Autosense protocols simply do not have CA. At the T10 CAP meeting this week, Ralph Weber is proposing removing all references to CA from SAM-3 (see 03-002r2 on http://www.t10.org). As soon as a command completes, it is assumed that the status and sense data will be returned together. If they can't be delivered, it doesn't matter. * the clearing effects table in FCP-2 was removed because it had several incorrect entries and was missing quite a few additional SCSI items that are also affected by logouts. T10 proposals 02-143r3 and 02-232r2 have the details and include large tables listing all the considered SCSI effects. These proposals have been incorporated into the latest SAM-3, SPC-3, SBC-2, and SSC-2 standards. * Most protocols don't detail how their hypothetical SCSI layer confirms the abort of tasks to the protocol layer by specifying a CHECK CONDITION, much less one with a specific sense key and additional sense code. This level of detail is internal only and seems irrelevent to the iSCSI wire protocol. Specifying a "unit attention" sense key could cause particular confusion, since unit attentions are supposed to be cleared once reported. On the iSCSI wire, the goal is to generate a unit attention with an additional sense code whose ASC field is 29h (e.g. I_T NEXUS LOSS, POWER ON OCCURRED, etc.) on the next command. To minimize changes to iSCSI at this point, I recommend changing the wording like this (changed text noted between _ characters) so only status is mentioned and it is only mentioned as an example: If the tasks terminated in the above cases a), b, c) and d)are SCSI tasks, they must be internally terminated _with CHECK CONDITION status with a sense key of unit attention and ASC/ASCQ values of 0x6E/0x00 (COMMAND TO LOGICAL UNIT FAILED)_. This status is only meaningful for appropriately handling the internal SCSI state and SCSI side effects with respect to ordering (e.g., queued commands and ACA) because this status is never communicated back as a terminating status to the initiator. To: If the tasks terminated in the above cases a), b, c) and d) are SCSI tasks, they must be internally terminated _as if with CHECK CONDITION status_. This status is only meaningful for appropriately handling the internal SCSI state and SCSI side effects with respect to ordering (e.g., queued commands and ACA) because this status is never communicated back as a terminating status to the initiator. --- Rob Elliott, HP Industry Standard Server Storage elliott@hp.com -----Original Message----- From: Eddy Quicksall [mailto:eddy_quicksall@ivivity.com] Sent: Thursday, January 09, 2003 2:39 PM To: dcuddihy@attotech.com; ips@ece.cmu.edu Subject: RE: iSCSI: Implicit Termination of Tasks The way I figured it would work is that the UA would get set and would get reported when the next command arrives. Consider this: connection 0 and 1 are all quite connection 0 has a command x on the wire connection 0 is lost the UA is set command y comes in connection 1 command y gets the UA or ACA Active That way the upper layer at the initiator can figure out how to recover. Is this not the idea? Eddy -----Original Message----- From: dcuddihy@attotech.com [mailto:dcuddihy@attotech.com] Sent: Thursday, January 09, 2003 3:05 PM To: ips@ece.cmu.edu Subject: RE: iSCSI: Implicit Termination of Tasks David, Regarding Autosense clearing the CA: The problem that I have with this is that the CA should get cleared when the sense data is reported, not when the sense data is generated. In this case, the autosense cannot be reported (the port is logged out), so the CA may not be cleared. It doesn't seem to me that generating a sense data response, even for an interface in which autosense is mandatory, implicitly clears a CA. By the way, there was a question of Request Sense commands enqueued when a termination occurs .. my understanding is that the sense data need not be saved for a subsequent request sense command if it has been reported via autosense, so the request sense returns NO_SENSE. Regarding FCP handling of port logout and ACA: For us, ACA is a non-issue-- we don't support it, but it should be cleared only for the initiator which generated the log event. (ACA support seems to be a point of contention among SCSI implementers... I know it's in all the specs, but I have yet to see it implemented.) Rev 7A of FCP-2 has a nice chart of how task managment is affected via the various log events. I'm not why it was removed in rev 8, but "FCP-2 Rev 7A (1/1/2001)" Table 4 might help. regards, david David J Cuddihy Principal Engineer ATTO Technology, Inc. http://www.attotech.com/fcbridge.html dcuddihy@attotech.com ------------------------------------------------------------------------ ---- --------------------- ------------------------------------------------------------------------ ---- --------------------- David, > Forgive me for being confused, but what is the reason for the Check > Condition being issued? Good question ... > Say a target implements a blocking CA, (TST 0 Qerr 0) which would disallow > other initiator's tasks from running until the check condition could > be reported, or another command from the CA'd initiator completes with > good status. An internal, non- reported check-condition would cause > all other initiators' tasks to be blocked until someone times out and > issues a lun reset to the device, or a command from the first > initiator completed (not likely, since that initiator logged out and all its tasks have been > terminated ). All this caused by bad behavior on behalf of one > initiator. That shouldn't happen. iSCSI REQUIRES use of Autosense, which will immediately clear the CA by generating a response to report the sense. That response gets bit-bucketed as undeliverable for the implicit termination scenarios in Section 6.5, but its generation clears the CA. Not the most obvious thing to do/explain, though ... > Implicit termination is handled in FCP without the check condition > being issued... all the tasks from the 'logged out' initiator are > terminated, reservations cleared, etc. and a UA is generated for that > initiator. Every other initiator goes along with no problem, including > those sharing the lun with the initiator who abruptly logged out. > Seems to work. That looks like a cleaner way to accomplish this and avoids some of the confusion over what ASC/ASCQ values to use. Does FCP also avoid ACA in this circumstance (assuming that NACA is set to use ACA), and if so, can you suggest some specific text for iSCSI? Thanks, --David ---------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 176 South St., Hopkinton, MA 01748 +1 (508) 293-7953 **NEW** FAX: +1 (508) 293-7786 black_david@emc.com Mobile: +1 (978) 394-7754 ---------------------------------------------------- ------_=_NextPart_001_01C2BBE8.3330FB10 Content-Type: text/html; charset="iso-8859-1"

Yes and this is where I've been coming from since I started this thread. 6.5 should be changed as Bob pointed out below. I can't tell from your response "This is perfectly reasonable" if you are agreeing that Bob's suggestion is reasonable or if the current wording is reasonable.

Please clear that up.

Eddy

-----Original Message-----
From: Julian Satran [mailto:Julian_Satran@il.ibm.com]
Sent: Tuesday, January 14, 2003 10:34 AM
To: Eddy_Quicksall@iVivity.com
Cc: Black_David@emc.com; dcuddihy@attotech.com; Elliott, Robert (Server Storage); ips@ece.cmu.edu
Subject: RE: iSCSI: Implicit Termination of Tasks

Eddy,

Setting the UA is a purely SCSI matter and so is the code used. There is no iSCSI state surviving and we assume that the isCSI initiator
produces some termination code for the SCSI initiator. It is not clear to me than anything besides "ACA active" code is needed.
If the target messes with command from other initiators (single queue-per-target) those should be ready to accept the consequences of a common queue.
UA is used (IMHO) mostly in cases of reset - that kill commands in other queues even when the operation is queue-per-initiator.

Julo

"Eddy Quicksall" <Eddy_Quicksall@iVivity.com>
14/01/03 16:39

Please respond to
<Eddy_Quicksall@iVivity.com>

To
Julian Satran/Haifa/IBM@IBMIL

cc
<Black_David@emc.com>, <dcuddihy@attotech.com>, <ips@ece.cmu.edu>, "Elliott, Robert \(Server Storage\)" <Elliott@hp.com>

Subject
RE: iSCSI: Implicit Termination of Tasks

>>nothing specific has to survive for ACA

For ACA, doesn't the next command get the UA then the additional commands get ACA Active until ACA is cleared (all assuming NACA, the ACA attribute and QErr are being used appropriately)?

So, doesn't the UA have to survive?

Eddy
-----Original Message-----
From: Julian Satran [mailto:Julian_Satran@il.ibm.com]
Sent: Tuesday, January 14, 2003 3:50 AM
To: Elliott, Robert (Server Storage)
Cc: Black_David@emc.com; dcuddihy@attotech.com; Eddy Quicksall; ips@ece.cmu.edu
Subject: RE: iSCSI: Implicit Termination of Tasks

Rob,

This is perfectly reasonable.
The clearing effects will wipe-out anyhow whatever was specific to the code for session termination
and nothing specific has to survive for ACA.

Regards,
Julo

"Elliott, Robert (Server Storage)" <Elliott@hp.com>
14/01/03 09:59

To
"Eddy Quicksall" <eddy_quicksall@ivivity.com>, <dcuddihy@attotech.com>, <ips@ece.cmu.edu>, Julian Satran/Haifa/IBM@IBMIL, <Black_David@emc.com>

cc

Subject
RE: iSCSI: Implicit Termination of Tasks

A few comments on this thread: * Autosense protocols simply do not have CA. At the T10 CAP meeting this week, Ralph Weber is proposing removing all references to CA from SAM-3 (see 03-002r2 on http://www.t10.org). As soon as a command completes, it is assumed that the status and sense data will be returned together. If they can't be delivered, it doesn't matter. * the clearing effects table in FCP-2 was removed because it had several incorrect entries and was missing quite a few additional SCSI items that are also affected by logouts. T10 proposals 02-143r3 and 02-232r2 have the details and include large tables listing all the considered SCSI effects. These proposals have been incorporated into the latest SAM-3, SPC-3, SBC-2, and SSC-2 standards. * Most protocols don't detail how their hypothetical SCSI layer confirms the abort of tasks to the protocol layer by specifying a CHECK CONDITION, much less one with a specific sense key and additional sense code. This level of detail is internal only and seems irrelevent to the iSCSI wire protocol. Specifying a "unit attention" sense key could cause particular confusion, since unit attentions are supposed to be cleared once reported. On the iSCSI wire, the goal is to generate a unit attention with an additional sense code whose ASC field is 29h (e.g. I_T NEXUS LOSS, POWER ON OCCURRED, etc.) on the next command. To minimize changes to iSCSI at this point, I recommend changing the wording like this (changed text noted between _ characters) so only status is mentioned and it is only mentioned as an example: If the tasks terminated in the above cases a), b, c) and d)are SCSI tasks, they must be internally terminated _with CHECK CONDITION status with a sense key of unit attention and ASC/ASCQ values of 0x6E/0x00 (COMMAND TO LOGICAL UNIT FAILED)_. This status is only meaningful for appropriately handling the internal SCSI state and SCSI side effects with respect to ordering (e.g., queued commands and ACA) because this status is never communicated back as a terminating status to the initiator. To: If the tasks terminated in the above cases a), b, c) and d) are SCSI tasks, they must be internally terminated _as if with CHECK CONDITION status_. This status is only meaningful for appropriately handling the internal SCSI state and SCSI side effects with respect to ordering (e.g., queued commands and ACA) because this status is never communicated back as a terminating status to the initiator. --- Rob Elliott, HP Industry Standard Server Storage elliott@hp.com -----Original Message----- From: Eddy Quicksall [mailto:eddy_quicksall@ivivity.com] Sent: Thursday, January 09, 2003 2:39 PM To: dcuddihy@attotech.com; ips@ece.cmu.edu Subject: RE: iSCSI: Implicit Termination of Tasks The way I figured it would work is that the UA would get set and would get reported when the next command arrives. Consider this: connection 0 and 1 are all quite connection 0 has a command x on the wire connection 0 is lost the UA is set command y comes in connection 1 command y gets the UA or ACA Active That way the upper layer at the initiator can figure out how to recover. Is this not the idea? Eddy -----Original Message----- From: dcuddihy@attotech.com [mailto:dcuddihy@attotech.com] Sent: Thursday, January 09, 2003 3:05 PM To: ips@ece.cmu.edu Subject: RE: iSCSI: Implicit Termination of Tasks David, Regarding Autosense clearing the CA: The problem that I have with this is that the CA should get cleared when the sense data is reported, not when the sense data is generated. In this case, the autosense cannot be reported (the port is logged out), so the CA may not be cleared. It doesn't seem to me that generating a sense data response, even for an interface in which autosense is mandatory, implicitly clears a CA. By the way, there was a question of Request Sense commands enqueued when a termination occurs .. my understanding is that the sense data need not be saved for a subsequent request sense command if it has been reported via autosense, so the request sense returns NO_SENSE. Regarding FCP handling of port logout and ACA: For us, ACA is a non-issue-- we don't support it, but it should be cleared only for the initiator which generated the log event. (ACA support seems to be a point of contention among SCSI implementers... I know it's in all the specs, but I have yet to see it implemented.) Rev 7A of FCP-2 has a nice chart of how task managment is affected via the various log events. I'm not why it was removed in rev 8, but "FCP-2 Rev 7A (1/1/2001)" Table 4 might help. regards, david David J Cuddihy Principal Engineer ATTO Technology, Inc. http://www.attotech.com/fcbridge.html dcuddihy@attotech.com ------------------------------------------------------------------------ ---- --------------------- ------------------------------------------------------------------------ ---- --------------------- David, > Forgive me for being confused, but what is the reason for the Check > Condition being issued? Good question ... > Say a target implements a blocking CA, (TST 0 Qerr 0) which would disallow > other initiator's tasks from running until the check condition could > be reported, or another command from the CA'd initiator completes with > good status. An internal, non- reported check-condition would cause > all other initiators' tasks to be blocked until someone times out and > issues a lun reset to the device, or a command from the first > initiator completed (not likely, since that initiator logged out and all its tasks have been > terminated ). All this caused by bad behavior on behalf of one > initiator. That shouldn't happen. iSCSI REQUIRES use of Autosense, which will immediately clear the CA by generating a response to report the sense. That response gets bit-bucketed as undeliverable for the implicit termination scenarios in Section 6.5, but its generation clears the CA. Not the most obvious thing to do/explain, though ... > Implicit termination is handled in FCP without the check condition > being issued... all the tasks from the 'logged out' initiator are > terminated, reservations cleared, etc. and a UA is generated for that > initiator. Every other initiator goes along with no problem, including > those sharing the lun with the initiator who abruptly logged out. > Seems to work. That looks like a cleaner way to accomplish this and avoids some of the confusion over what ASC/ASCQ values to use. Does FCP also avoid ACA in this circumstance (assuming that NACA is set to use ACA), and if so, can you suggest some specific text for iSCSI? Thanks, --David ---------------------------------------------------- David L. Black, Senior Technologist EMC Corporation, 176 South St., Hopkinton, MA 01748 +1 (508) 293-7953 **NEW** FAX: +1 (508) 293-7786 black_david@emc.com Mobile: +1 (978) 394-7754 ----------------------------------------------------

------_=_NextPart_001_01C2BBE8.3330FB10--