Subject: aborting an out of sequence cmdSN

From: "Eddy Quicksall" <[email-masked]>
Date: Fri, 13 Apr 2001 18:39:54 -0400

If a command is received out of sequence, it is not passed to the SCSI
Target Layer. So, it must be queued waiting for the prior command. Now, if
the prior command never comes the operating system may attempt to abort the
lost command by sending an abort to the driver.

The abort can't be turned into an ABORT TASK TMF because the target iSCSI
has not acknowledged that it has received the original command yet. And,
even if the ABORT TASK was sent, it can't get to the SCSI Target Layer
because its cmdSN will be higher than the most recent command.

The iSCSI initiator drive could just jerk it out of its queue (knowing that
it hasn't been acknowledged yet) but then the iSCSI target may still execute
it when the missing PDU's arrive.

Does iSCSI have a way to abort the commands that are in the target but not
yet in the SCSI Target Layer?

Eddy

---

From: Charles Monia <[email-masked]>
Date: Tue, 17 Apr 2001 11:03:17 -0700

Hi:

> No, it doesn't.  Part of the problem is that just considering
> iSCSI info, the Initiator can't tell whether such commands
> are in the Target vs. still in-flight.  TCP ACK info may help,
> but if TCP hasn't ACKed, the Initiator still doesn't know.
> 

I don't understand what problem is being addressed.

I assume that iSCSI is just an extension of the pipeline for SCSI commands
and task management functions.  As long as nothing impedes pipeline flow,
the appropriate operations should eventually be presented to the SCSI layer
for execution in the proper order.

I thought it was decided a long time ago that the SCSI layer would interact
with the iSCSI transport such that flow through the pipeline never blocked
indefinitely.

Is that the case?

Charles
> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]
> Sent: Tuesday, April 17, 2001 12:03 AM
> To: [email-masked]; [email-masked]
> Subject: RE: aborting an out of sequence cmdSN
> 
> 
> Eddy,
> 
> No, it doesn't.  Part of the problem is that just considering
> iSCSI info, the Initiator can't tell whether such commands
> are in the Target vs. still in-flight.  TCP ACK info may help,
> but if TCP hasn't ACKed, the Initiator still doesn't know.
> 
> This is the issue that Doug's been posting about, advocating
> that iSCSI always reject all commands prior to the task abort
> on the assumption that the initiator can reissue them - given
> the lack of support this has received, I'm going to take this
> opportunity to state that I believe WG rough consensus exists
> for NOT pursuing this "reject all prior commands" approach.
> This is the last chance for anyone else to express support for
> this reject-based approach.
> 
> If we were to address the original issue, I think some sort
> of new mechanism is needed at the iSCSI level, as the
> attempts to use a side effect of SCSI task aborts don't
> seem to be getting much of anywhere (e.g., a previous
> attempt at this involved having iSCSI automatically send
> task management commands multiple times).  A SWAG at what
> such a mechanism might look like is some sort of iSCSI
> "Cancel" that names the CmdSN(s) to be cancelled and sent
> in the same PDU as the SCSI Task Abort or Task Set Abort
> that is supposed to abort them.  Canceling would apply
> both to commands waiting for CmdSN in iSCSI as well as
> those not yet received by the target (i.e., the target
> has to have some notion of "pending cancel" for commands
> it hasn't received yet.  The cancelled commands
> probably have to return some sort of iSCSI service error
> rather than SCSI sense data.  Something like this looks
> like it can be made to work as desired (issue TASK 
> ABORT with the corresponding Cancel for immediate delivery,
> and if SCSI declares the TASK ABORT to have succeeded,
> then that command will not subsequently execute), but it'll
> add complexity.  Is that worth the benefit of being able
> to reach out and definitively strangle an errant command?
> 
> --David
> 
> > -----Original Message-----
> > From:	Eddy Quicksall [SMTP:[email-masked]]
> > Sent:	Friday, April 13, 2001 6:40 PM
> > To:	Ips@Ece. Cmu. Edu (E-mail)
> > Subject:	aborting an out of sequence cmdSN
> > 
> > 
> > the prior command never comes the operating system may 
> attempt to abort
> > the
> > The iSCSI initiator drive could just jerk it out of its 
> queue (knowing
> > that

---

From: [email-masked]
Date: Wed, 18 Apr 2001 10:12:56 -0400

Sandeep,

> The "iSCSI cancel" proposal you describe below has been presented 
> once before.  If you recall, I was asking for a refCmdSN in the 
> TaskMgmt PDU.
> 
> The only difference between that and what you now describe below 
> is the addition of this concept of an "iSCSI service error" response.

Close enough - sorry for the lack of acknowledgement.  Of the four
options, (1) and (3) make the most sense to me.  Assuming that
we document the iSCSI-based effects of task management operations
(probably should do this not only for task abort, but also task set
abort and task set clear):

> (1) use connection allegiance for TASK MGMT PDU.

Ok, with the possible concern about something going wrong on
that connection.

> (2) reject all commands prior to cmdSN of TASK MGMT PDU.

I don't see anyone aside from Doug interested in this, so I consider
WG rough consensus to have rejected this approach.

> (3) cmdSN of original task is sent with TASK MGMT PDU and
>     target at the iSCSI layer keeps state.

The state's not a big deal - in essence the iSCSI target pretends it
received a NOP instead of the actual command if the abort arrives
ahead of the command.

> (4) iSCSI initiator retains state for deleted tasks to ensure
>     that R2T/Scsi Responses are appropriately handled.

Ugly, although errant R2Ts may show up if the abort/clear is
across connections.  With the iSCSI enhancements, the
initiator should be able to throw these away.

--David

---------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 42 South St., Hopkinton, MA  01748
+1 (508) 435-1000 x75140     FAX: +1 (508) 497-8500
[email-masked]       Mobile: +1 (978) 394-7754
---------------------------------------------------

---

From: [email-masked]
Date: Thu, 19 Apr 2001 08:33:01 +0200

Santosh,

The version 06 (to appear today) addresses your concerns with regard to
Task management - although through the mechanism I was alluding all the
time.

At David Black's suggestion - I've added a section to chapter 7 explaining
how to properly handle task management using CmdSN.
It is only a note to implementers - nothing needed change.

The error recovery chapter is still work-in-progress.

I would also appreciate if you would "tone down" your notes - recall that
we are always limited by our own imagination.

Julo

Santosh Rao <[email-masked]> on 18/04/2001 19:00:43

Please respond to Santosh Rao <[email-masked]>

To:   Julian Satran/Haifa/IBM@IBMIL
cc:   [email-masked]
Subject:  Re: aborting an out of sequence cmdSN

Julian,

I understand that nothing in the current draft prevents usage of
connection allegiance for an abort task to flush the stale PDUs of a
command.

However, as a target implementor, if such a guaranteed mechanism is not
mandated to flush stale PDUs of a command, targets will need to build
safeguards against possible stale PDUs from a command. Any such scheme
would cause targets to retire resources until a safe period (based on a
best guess !) has elapsed. The targets CANNOT depend on initiators
deploying this scheme in the absence of the spec mandating this and do
not have a guarantee that stale PDUs will not arrive after the Abort
Task has cleaned up the command at the target end.

Keeping in mind the above, I am asking that iSCSI mandate that the Abort
Task be sent on the same connection as the SCSI command being aborted.
In addition, [and this has been asked earlier], the iSCSI draft lacks a
section on I/O timeout handling under the error recovery section. Some
description must be provided on the actions an initiator should take on
an I/O timeout. (or timeout of other iSCSI PDUs).

For instance, should non-SCSI PDUs be timed ? (login, text, etc). If so,
what is their timeout values. What actions should an initiator take if
they time out. How is the task tag for that non-SCSI PDU cleaned up,
ensuring no stale PDUs are going to arrive for that task tag.

iSCSI Error Recovery descriptions MUST address task timeout handling.

- Santosh

[email-masked] wrote:
>
> First the current draft does not stop you from doing what you want - it
> just does not mandate it.
> Second with CmdSN now you have a better chance - before handing the
command
> over to SCSI task management
> mark all the relevant commands in the iSCSI queue as non-deliverable to a
> LUN or all LUNs or keep a stack of barriers for active aborts (I assume
> that this is a reasonably low number) to the same effect.
>
> Julo
>
> Santosh Rao <[email-masked]> on 18/04/2001 01:10:17
>
> Please respond to Santosh Rao <[email-masked]>
>
> To:   Sandeep Joshi <[email-masked]>
> cc:   [email-masked]
> Subject:  Re: aborting an out of sequence cmdSN
>
> I'd think option 1 is the simplest (with the caveat that the task mgmt
> PDU referred to is the Abort Task.) and only impacts the affected
> command/task.
>
> Pierre Labat and I have asked for this 4 months ago. (See :
> http://ips.pdl.cs.cmu.edu/mail/msg02958.html). The concept of connection
> allegiance should be extended to include the Abort Task. Also,
> connection allegiance should apply to the task (which spans multiple
> commands in the case of linked commands.), allowing for a deterministic
> clean up of stale PDUs of the task through the use of Abort Task.
>
> - Santosh
>
> Sandeep Joshi wrote:
>
> >
> > So our options for abort_task boil down to..
>  - santoshr.vcf
 - santoshr.vcf