Subject: iSCSI linked commands

From: [email-masked]
Date: Tue, 17 Apr 2001 12:30:51 +0200

Doug,

I think you would want to go back to SAM.  Linked command are broken by any
"irregularity" in execution.
The basic assumption is that the initiator is in charge of shipping linked
commands - one-by-one.
I assume that for high latency links they won't be very popular.

At a very early stage (about 2 years ago) we contemplated the idea of
"prefetching" linked commands and have the target
effect the serialization. We would have had to come up with a way of
conveying the initiator which command broke the chain (if it broke) or
caused a unit attention (if it caused) and it was not at all clear that
this was "in the spirit of SAM" .
There where also more esoteric issues with later command getting modified
by execution of prior commands etc. -:).

The 360 channels had the same class of issues.

I assume that T10 folks went over these issues many times.

Julo

"Douglas Otis" <[email-masked]> on 16/04/2001 23:41:49

Please respond to "Douglas Otis" <[email-masked]>

To:   "Santosh Rao" <[email-masked]>
cc:   "Ips" <[email-masked]>
Subject:  RE: iSCSI:flow control, acknowledgement, and a deterministic
      recovery

Santosh,

The iSCSI proposal ver 5-91 explicitly defines tasks and also includes the
option to allow linked commands to be sent across different connections.
Obviously, for sequential devices, particular attention must be paid to
command serialization as these commands tend to use relative addressing or
are dependent upon the successful completion of prior commands.  This
requirement is not helped with Auto-Sense and impels the need for a target
model change in SCSI.  An error injected into the SCSI layer as a result of
a network communication error will significantly reduce the utility of most
backup applications.  Such reliance on the SCSI layer to recover from such
uncertainty imposed as a result of the inability of the network transport
to
do minimal handshakes and retries is the wrong approach.  Regardless, you
are burdening the driver with the duty of tracking a transient connection
allegiance status.  The latest version has improved language with the
exception of Pg. 16 in two places.

Ver 5-91
Pg. 15
"Connection allegiance is strictly per-command and not per-task."

Pg. 16
"tasks that have allegiance to the connection"
"all outstanding tasks that have allegiance to the connection to conclude
and send their status."

Doug

> Doug,
>
> You seem to be referring to linked commands as a case wherein the
> approach of Abort Task will not flush stale PDUs.
>
> Linked Commands cannot work the way SCSI implementations are defined
> today, since linked commands require the initiator task tag (I_T_L_x
> nexus identifier in SAM-2 Execute Command terminology) to be generated
> by the SCSI ULP. However, in practice, the Initiator Task Tag (or the FC
> OX_ID) is typically generated in the SCSI LLP (or in some cases in the
> adapter firmware). IOW, there is no common reference handle like the
> task tag sent down from the ULP that allows for association of multiple
> commands to a task in several/most implementations today.
>
> When this is fixed up to get linked commands to work [& there exist
> examples of its usage], there is no reason connection allegiance could
> not be applied to all the commands within the task.
>
> I fail to see why you think Abort Task will not work with sequential
> devices (?).
>
> - Santosh
>
> Douglas Otis wrote:
> >
> > Santosh,
> >
> > I see a few problems with this approach.  Tasks as defined in
> > iSCSI do not maintain connection allegiance.  The driver binds all
> > SCSI commands to their connection for the most resent association.
> > Although there are several places within the iSCSI proposal that
> > make reference to a task having a connection allegiance, this is
> > in error.  Commands and not tasks carry such allegiance.  Your
> > recovery scheme will not allow a satisfactory recovery with a
> > sequential device.  In this case, repeating the command is not a
> > solution.  As a result, one connection falter and it will become a
> > difficult situation.  In addition, you have no clue from iSCSI your
> > delivery status.  You do not know if you are waiting for the target
> > or if you are waiting for the connection.  Some sequential devices
> > have rather long time-outs with these complications of deducing
> > status created by the multiple connections.
> >
> > The application will not know about these connection allegiance
> > problems. The iSCSI layer does not define interaction to provide
> > additional application status to allow these applications to respond
> > in a manner that may aid this situation nor should such additional
> > information be required.  With your scheme the SCSI driver must
> > examine the content of these commands to make a guess as to the
> > connection allegiance assignments.  Now the driver is expected to
> > understand what the intended action is of this SCSI management
> > command.  What signal is used to indicate a need for the iSCSI
> > immediate treatment?  The only obvious seems to be the task attribute
> > argument.  With the way iSCSI has defined iSCSI immediate, I
> > would expect those commands to be treated in a LIFO rather than the
> > normal FIFO fashion.
> >
> > Doug

---

From: "Douglas Otis" <[email-masked]>
Date: Tue, 17 Apr 2001 08:40:10 -0700

Julian,

I am aware that iSCSI can be used for parallel SCSI configurations as well
Fibre-channel that has a level of accommodation for tape linked commands but
I am unaware of such implementations.  Without getting involved with this FC
discussion, it is clear that parallel SCSI is still a valid means of
implementing linking.  The point that I was making as it relates to the
proposal was that on page 16 you indicate that the task carries the
allegiance.  It is the command as you clearly indicate on the prior page.
As there can only be one command in play at any point in time, this task can
become spread over any number of connections as you state.  As such, rather
than indicating an allegiance associated with the task, you may wish to say
with outstanding commands.  This should be taken only as an editorial
concern.  I was trying to make the point that connection allegiance is not
constant with respect to the task.  Santosh wishes to exclude the sequential
model from his thinking.  It was the examination of the command and the
overhead of tracking these allegiances I saw as undesirable.  The change to
implementing serialization prevents the problem if you introduce allegiance
for commands carrying the same serialization so this thread should become
stale as to the original concern.  The reason to introduce allegiance for
these commands carrying the same serialization is to prevent the command
window being closed.

With your present scheme, "immediate" commands should not advance the window
and should be placed in a prior position on the same connection.   If I have
missed this requirement, ignore this concern.

Doug

---

From: Ralph Weber <[email-masked]>
Date: Tue, 17 Apr 2001 19:51:45 -0700

Julian,

> "irregularity" in execution.

The specific statement you are looking for appears on SAM-2 r16
top of PDF page 66, "The receipt of any status, except INTERMEDIATE
or INTERMEDIATE-CONDITION MET, shall indicate that the associated
task has ended."

> The basic assumption is that the initiator is in charge of shipping linked

I am not an expert on linked commands (there are only three such
people that I know of).  However, my understanding is that linked
commands are REQUIRED to be shipped one-by-one.  The initiator is
allowed to pick the next linked command based on the results
returned by the previous linked command.

> I assume that for high latency links they won't be very popular.

They are not very popular, period.

> by execution of prior commands etc. -:).

Like I said above, I think pre-fetching linked commands violates
"the rules".

Thanks.

Ralph...