Subject: Several iSCSI protocol questions

From: "Tony Battersby" <[email-masked]>
Date: Fri, 30 Aug 2002 09:01:49 -0400

Hi,

I have a few general iSCSI protocol questions.  All questions are based on
Draft 15.

(Sorry if this message is a duplicate.  I sent it once before but I don't
think it got through.)

--- Initiator Task Tag uniqueness ---

Section 2.1, "A SCSI task is a SCSI command or possibly a linked set of SCSI
commands."

Section 2.2.2.1, "Some iSCSI tasks are SCSI tasks, and many SCSI activities
are related to a SCSI task ([SAM2]). In all cases, the task is identified by
the Initiator Task Tag for the life of the task."

Does the above imply that the initiator must (or can?) use the same
Initiator Task Tag for all SCSI commands in a given linked set of SCSI
commands?  If so, then a given Initiator Task Tag may not uniquely identify
a particular SCSI command.  In this case, if the target receives an
unsolicited data PDU, it would have to rely on the requirement that
unsolicited data PDUs must be sent in the same order as the commands were
sent in order to identify the SCSI command associated with the unsolicited
data PDU.  My question: what is to prevent the target from associating an
unsolicited data PDU with the wrong command if the target discarded one of
the linked command PDUs in the task due to a digest error?

--- Task Management Functions ---

Section 9.5.1, "For all these functions, the Task Management function
response MUST be returned as detailed in Section 9.6 Task Management
Function Response."  Does this mean that a Task Management Function with one
of the listed functions cannot be rejected with a Reject PDU if some other
field of the PDU has a value that is a protocol error?

If I read the spec correctly, the ABORT TASK, ABORT TASK SET, CLEAR TASK
SET, and LOGICAL UNIT RESET functions operate on iSCSI tasks that:

1) Have a CmdSN that is lower than the CmdSN of the Task Management
Function, AND
2) Have a valid LUN that is equal to the LUN of the Task Management Function

Since the Task Management Function itself has a CmdSN and valid LUN, can one
Task Management Function affect (abort) another?  If not, then what is the
appropriate Task Management Function Response for an ABORT TASK that
attempts to operate on another Task Management Function?

--- AHS ---

Section 9.2.1.5 TotalAHSLength:

"Total length of all AHS header segments in four byte words including
padding, if any.

The TotalAHSLength is used only in PDUs that have an AHS and MUST be 0 in
all other PDUs."

For the first sentence, I recommend changing the wording to emphasize the
fact that the length is not in bytes (something like "... segments in units
of four byte words ..." with an example given).  I missed this fact the
first time I read it.

The second sentence is confusing since it is not spelled out which PDUs have
an AHS.  Is this sentence specifying a requirement that some pre-defined
list of PDUs that do not have an AHS MUST have TotalAHSLength equal to 0, or
is the sentence simply stating that a nonzero TotalAHSLength implies that an
AHS is present?  In the former case, the list of PDUs not allowed to have an
AHS can be determined by looking elsewhere in the spec (Task Management
Function Request, Task Management Function Response, R2T, Logout Request,
Logout Response).

Next question: if an AHS is present in a PDU allowing it, but the target
does not know what to do with it, should the target reject the PDU or ignore
the AHS?  I noticed that some older drafts had a bit that indicated which to
do, but the bit was dropped in more recent drafts.  If the target should
reject the PDU, what Reject reason should be used?

--- Reject ---

If a target decides to reject a non-immediate command when it can determine
the CmdSN of the command being rejected, should the target return the reject
response immediately, or should the target defer the reject response until
the command being rejected would have been delivered for execution based on
its CmdSN?  If the CmdSN of the command being rejected is outside the valid
command window, should the target ignore the PDU and not return the Reject
response, or should the target return the Reject response anyway?  My guess
is that any of these options are valid, but I want to make sure.  In any
case, it seems that the initiator needs to be prepared to receive multiple
Reject PDUs from the target for a single command if the initiator
re-transmitted the command due to a timeout before receiving (or noticing)
the reject.

--- Logout ---

Section 9.14, "A Logout for a CID may be performed on a different transport
connection when the TCP connection for the CID has already been terminated."
Was this intended to imply that a Logout for a CID MUST be performed on the
same transport connection when the TCP connection for the CID has _not_
already been terminated?

Thanks,
Anthony J. Battersby
Cybernetics

---

From: "Julian Satran" <[email-masked]>
Date: Fri, 30 Aug 2002 19:45:21 +0300

Fields that appear in every PDU are not described repeatedly - Julo

"Tony Battersby" <[email-masked]>
Sent by: [email-masked]
30/08/02 17:34
Please respond to tonyb

        To:     <[email-masked]>, <[email-masked]>
        cc: 
        Subject:        RE: Several iSCSI protocol questions

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]
> Sent: Friday, August 30, 2002 10:05 AM
> To: [email-masked]; [email-masked]
> Subject: RE: Several iSCSI protocol questions

> > commands?  If so, then a given Initiator Task Tag may not
> > uniquely identify a particular SCSI command.
>
> Yes, but the inference is incorrect.  SAM-2 defines "linked
> command" as:
>
>   3.1.58 linked command: One in a series of SCSI commands
> processed by a
>   single task that collectively make up a discrete I/O
> operation. In such
>   a series, each command is represented by the same I_T_L_x
> nexus, and all,
>   except the last, have the LINK bit in the CDB CONTROL byte
> set to one.
>
> Linked commands are always sequential, not concurrent, as
> they use the same
> I_T_L_x nexus - see the example in Section 5.7.2 of SAM-2.
> Only one command
> from a set of linked commands is active at any point in time,
> and hence the
> ITT for the linked commands uniquely identifies it.  Linked
> commands are
> rarely used in practice, FWIW.
>

Thanks.  I realized this after I sent the message.

> > Function Response."  Does this mean that a Task Management
> Function with
> > one of the listed functions cannot be rejected with a Reject PDU
> > if some other field of the PDU has a value that is a protocol error?
>
> It shouldn't - if there's a protocol error, the Task
> Management function
> cannot be processed.

Ok.

> > Task Management Function affect (abort) another?
>
> No, SCSI task management functions operate on *SCSI* tasks,
> not *iSCSI*
> tasks, and a task management function is not a *SCSI* task -
> see SAM-2.

This contradicts the text in Section 9.5.1: "The Task Management functions
provide an initiator with a way to explicitly control the execution of one
or more Tasks (SCSI and iSCSI tasks)."  Your interpretation makes more 
sense
to me, however.

> > If not, then what is the
> That can't happen.  The only task management function that
> could possibly
> attempt "to operate on another Task Management Function" is
> ABORT TASK, but
> it identifies the task to be aborted by an Initiator Task
> Tag, something
> that Task Management Functions *do not* have - see Section
> 9.5 of the iSCSI
> draft.

I am looking at Section 9.5.  I see "Initiator Task Tag" in the table
showing the PDU format, although it is not mentioned in the text.  Another
typo?

> > --- AHS ---
> > Next question: if an AHS is present in a PDU allowing it, but
> > the target does not know what to do with it, should the
> target reject
> > the PDU or ignore the AHS?
>
> Such a target is broken (if an AHS is allowed, the target has to
> expect that it might be present and be prepared to do whatever
> is necessary), and should be fixed.

Let me give a better example: Section 9.2.2.1, AHSType, lists "60- 63
Non-iSCSI extensions".  What should the target do if it comes across an 
AHS
with an AHSType of 60?  Is there some other document specifying what these
codes mean?

> > its CmdSN?
> Returning the Reject earlier will probably be appreciated by the
> initiator.  There's no requirement to wait.

Ok.

> > If the CmdSN of the command being rejected is
> > outside the valid
> > response, or should the target return the Reject response
> > anyway?
>
> Ignore it.  Section 2.2.2.1 says:
>
>   For non-immediate commands, the CmdSN field can take any value from
>   ExpCmdSN to MaxCmdSN inclusive. The target MUST silently ignore any
>   non-immediate command outside of this range or non-immediate dupli-
>   cates within the range.
>
> "silently" is the key word in this text.

Ok.  The exception of course is if the opcode field is unrecognized, the
target can't be sure if the CmdSN is present in the normal place or not.

> Not exactly.  The problem is that one can get into weird situations
> in which the two ends of a TCP connection have different views on
> whether it's been terminated.  If the TCP connection is alive,
> Logout for that connection should be sent on that connection.
>

Makes sense.

> Thanks,
> --David
> ---------------------------------------------------
> David L. Black, Senior Technologist
> EMC Corporation, 42 South St., Hopkinton, MA  01748
> +1 (508) 249-6449            FAX: +1 (508) 497-8018
> [email-masked]       Mobile: +1 (978) 394-7754
> ---------------------------------------------------
>

---

From: "Julian Satran" <[email-masked]>
Date: Fri, 30 Aug 2002 19:22:31 +0300

Tony answers in text - Julo

30/08/02 16:01
Please respond to tonyb

        To:     <[email-masked]>
        cc: 
        Subject:        Several iSCSI protocol questions

Hi,

--- Initiator Task Tag uniqueness ---

Section 2.1, "A SCSI task is a SCSI command or possibly a linked set of 
SCSI
commands."

Section 2.2.2.1, "Some iSCSI tasks are SCSI tasks, and many SCSI 
activities
are related to a SCSI task ([SAM2]). In all cases, the task is identified 
by
the Initiator Task Tag for the life of the task."

commands?  If so, then a given Initiator Task Tag may not uniquely 
identify
+++++

Linked commands are executed sequentially (that is the only purpose of 
linking) as long as
the "cahin" is not broken by an exception"

++++
--- Task Management Functions ---

Function Response."  Does this mean that a Task Management Function with 
one

1) Have a CmdSN that is lower than the CmdSN of the Task Management
Function, AND
2) Have a valid LUN that is equal to the LUN of the Task Management 
Function

Since the Task Management Function itself has a CmdSN and valid LUN, can 
one
Task Management Function affect (abort) another?  If not, then what is the
appropriate Task Management Function Response for an ABORT TASK that
attempts to operate on another Task Management Function?

+++ Task management are SCSI functions not iSCSI functions.
In theory a Task Management request may refer to another task management 
request in iSCSI (a side effect of ITT being used for everything).
SAM and  SPC do not talk about this as Task Management are not queued to 
the device but are executed synchronously by the Task Manager. Some 
protocols (SPI, FCP-2) limit the effect of some (e.g., abort task) by 
implementing them so that only one can be issued (use a link level 
function).

The best way to handle this is to avoid queueing while a Task Management 
is active (act synchronously on it).
Sometimes this feature can become useful as the "clear functions" will 
clear the TM requests too.

++++

--- AHS ---

For the first sentence, I recommend changing the wording to emphasize the
fact that the length is not in bytes (something like "... segments in 
units
of four byte words ..." with an example given).  I missed this fact the
first time I read it.

+++ I will add units +++

The second sentence is confusing since it is not spelled out which PDUs 
have
an AHS.  Is this sentence specifying a requirement that some pre-defined
list of PDUs that do not have an AHS MUST have TotalAHSLength equal to 0, 
or
is the sentence simply stating that a nonzero TotalAHSLength implies that 
an
AHS is present?  In the former case, the list of PDUs not allowed to have 
an
AHS can be determined by looking elsewhere in the spec (Task Management
Function Request, Task Management Function Response, R2T, Logout Request,
Logout Response).

+++ the sentence is just stating that this field is always reffering to 
AHS and will not be used for something else That enables simpler handling 
of the length fields. Unlike reserved it will be USED by receivers!+++

Next question: if an AHS is present in a PDU allowing it, but the target
does not know what to do with it, should the target reject the PDU or 
ignore
the AHS?  I noticed that some older drafts had a bit that indicated which 
to
do, but the bit was dropped in more recent drafts.  If the target should
reject the PDU, what Reject reason should be used?

+++ AHSs as defined today MUST be understood. You will have to reject with 
a Function Response of Function Rejected (AHS is used for SCSI commands). 
You may also use a Reject with protocol error if you get AHS where you 
should not have one

--- Reject ---

If a target decides to reject a non-immediate command when it can 
determine
the CmdSN of the command being rejected, should the target return the 
reject
response immediately, or should the target defer the reject response until
the command being rejected would have been delivered for execution based 
on
its CmdSN? 
+++ either way is fine although I suspect that doing the first may cause 
trouble with some initiators +++

 If the CmdSN of the command being rejected is outside the valid
command window, should the target ignore the PDU and not return the Reject
response, or should the target return the Reject response anyway?  My 
guess
is that any of these options are valid, but I want to make sure.  In any
case, it seems that the initiator needs to be prepared to receive multiple
Reject PDUs from the target for a single command if the initiator
re-transmitted the command due to a timeout before receiving (or noticing)
the reject.
+++ CmdSN outside window - results in ignore ++++
--- Logout ---

Section 9.14, "A Logout for a CID may be performed on a different 
transport
connection when the TCP connection for the CID has already been 
terminated."
Was this intended to imply that a Logout for a CID MUST be performed on 
the
same transport connection when the TCP connection for the CID has _not_
already been terminated?

+++ yes - you should attempt to do it on existing connection if you can.
However some caution - this is not a condition a target can chec - it can 
see
a connection that is dead on the other end +++

Thanks,
Anthony J. Battersby
Cybernetics

---

From: "Julian Satran" <[email-masked]>
Date: Fri, 30 Aug 2002 19:33:53 +0300

I already answered to this.

This is a result of using ITT for everything.

We considered it several times (adding a reject code) but the we thought 
that
the "good side-effects" outweight the bad (e.g., you can do a TASK SET 
CLEANUP including an TASK ABORT) and it is not worth adding reject code 
(mainly not worth testing for it).  I fail to see a scenario where it can 
do unintentional harm.

Julo

[email-masked]
30/08/02 17:58

        To:     [email-masked], [email-masked]
        cc: 
        Subject:        RE: Several iSCSI protocol questions

> > No, SCSI task management functions operate on *SCSI* tasks, not 
*iSCSI*
> > tasks, and a task management function is not a *SCSI* task - see 
SAM-2.
sense
> to me, however.

And there's more text further down in 9.5.1 that seems to imply that
one ought to be able to abort a Task Management Function.

  All these functions apply to the referenced tasks regard-
  less of whether they are proper SCSI tasks or tagged iSCSI opera-
  tions.

I think you've caught something we need to fix ...

> > That can't happen.  The only task management function that could
possibly
> > attempt "to operate on another Task Management Function" is ABORT 
TASK,
but
> > it identifies the task to be aborted by an Initiator Task Tag, 
something
> > 9.5 of the iSCSI draft.
> showing the PDU format, although it is not mentioned in the 
> text.  Another typo?

My mistake - I think we have some text cleanup to do, as SAM-2 definitely
considers task management functions not to be tasks, thus avoiding issues
of what happens when a task management function is applied to a task
management function - I believe iSCSI needs to follow this route.  Julian?

Thanks,
--David