Subject: RE: iSCSI: Out of order commands

From: "John Hufferd" <[email-masked]>
Date: Thu, 8 Nov 2001 16:19:54 -0800

Most folks I know have a buffer that applies to all the connections, not to
just one buffer per connection.  I fail to see the problem for most
implimentations.  The window should apply to the session.

.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
Home Office (408) 997-6136, Cell: (408) 499-9702
Internet address: [email-masked]

"Somesh Gupta" <[email-masked]>@ece.cmu.edu on
11/08/2001 02:22:30 PM

Please respond to <[email-masked]>

Sent by:  [email-masked]

To:   "Robert D. Russell" <[email-masked]>
cc:   Julian Satran/Haifa/IBM@IBMIL, <[email-masked]>
Subject:  RE: iSCSI: Out of order commands

Comments below

> -----Original Message-----
> From: Robert D. Russell [mailto:[email-masked]]
> Sent: Thursday, November 08, 2001 11:46 AM
> To: Somesh Gupta
> Cc: Julian Satran; [email-masked]
> Subject: RE: iSCSI: Out of order commands
>
>
>
> Somesh:
>
> > > It seems to me that if a target can only handle x new commands,
> > > then its queueing capacity is x, and in the SCSI Response PDU it
> > > should set MaxCmdSN = (ExpCmdSn + x - 1), in accordance with the
> > > formula in section 2.2.2.1.  This in turn controls the number of
> > > commands the initiator can send, and thus prevents the incoming
> > > commands from overfilling the target's queue.
> >
> >   This is actually a weakness of the multiple connections over
> >   multiple adapters model (it is not related to ordering).
> >   The target advertises a command window on a session wide basis.
> >   At the same time, it has to provide the resource to the adapter
> >   to be able to DMA those commands to. Since there is no restriction
> >   on which connection the commands may be received, either the
> >   target has to allocate the max resources needed to each adapter
> >   (thus committing n times the resources actually required), or
> >   has to "lie" (it would not be a complete lie) which could
> >   result in running out of resources. One way to fix that would be
> >   to have the credit on a per connection basis.
>
> If I understand you correctly, you are saying that deadlock can
> occur even if we enforce ordered delivery by initiators and even
> if the advertisements are correct and even if both parties obey
> the advertisements.

  I would let Julian enlighten us on that, or whether it just
  leads to wholesale command drops and retries, or TASK SET FULL
  is returned (is TASK SET FULL a valid response in this case
  for a LINKED task when it is not the first command?)

>
> In other words, doesn't this mean that the standard can lead to a
> trap in implementing targets and that, in fact, the advertisements
> really should be on a per connection basis?
> However, what would be the consequences for initiators if
> the advertisements were on a per-connection basis?

  There is really no issue with adveritisement on a per
  connection basis. It leads to a protocol change. However, it
  also allows a much more performant flow of commands.

>
>
> Thanks,
>
> Bob Russell
> InterOperability Lab
> University of New Hampshire
> [email-masked]
> 603-862-3774
>
>
>

---

From: "Somesh Gupta" <[email-masked]>
Date: Thu, 8 Nov 2001 16:31:29 -0800

John,

You are correct if all the connections are on 
a single adapter. In this case the buffer pool
can be shared among all the connections on the
adapter.

However this is not true if the connections are
on different adapters (the point being discussed).
Think of there being a big buffer pool in the board
into which multiple adapters/chips are "plugged" in.
However, at any given point in time, these buffers
are allocated (handed over through command queues or
whatever) to each of the individual adapters. And this
is where the problem arises.

The assumption that adapters can share a typical host
command buffer pool on their own without host intervention
is incorrect.

Somesh

> From: John Hufferd [mailto:[email-masked]]
> Sent: Thursday, November 08, 2001 4:20 PM
> To: [email-masked]
> Cc: Robert D. Russell; Julian Satran; [email-masked]