Subject: RE: iSCSI: Out of order commands

From: "Julian Satran" <[email-masked]>
Date: Fri, 9 Nov 2001 12:11:11 +0200

Paul,

I should have seen this comming. 
iSCSI assumes that TCP is there to take care of the transport window and 
the transport window is as
large as it should be. However we are talking about the application 
window.  The application has advertize it's own window in addition to the 
transport windows. With transport having things in order we can avoid 
having to have a complete or even partial duplicate of the windows.  If we 
allow transport to deliver not in order we have to have them duplicated 
(or at least the control part - and whatever DMA schemes we put in place 
won't help).

Julo

Paul Koning <[email-masked]>
09-11-01 04:25
Please respond to Paul Koning

        To:     Julian Satran/Haifa/IBM@IBMIL
        cc:     [email-masked]
        Subject:        RE: iSCSI: Out of order commands

Excerpt of message (sent 8 November 2001) by Julian Satran:
> Ron,
>
> Targets will most likely advertise a total (command and data) window
> larger than they can accommodate on any long haul link.  With the 
current
> ordering rules nothing bad will happen.

This is a very bad idea.

The whole notion of a window is that you advertise what you can
handle.  You DO NOT, EVER, advertise more than you can handle in the
hope that you won't be caught.

Long haul links are no excuse.  The way you deal with those is by
having more buffer capacity, and issuing window increases early enough
to keep data flowing.

In a window flow control scheme, a target that advertises resources it
does not have is defective.

This is all VERY old hat.  TCP has understood how to do this for
ages.  In particular, it has always been well understood that you HAVE
to have more buffers in order to run at high speed over long delay
links.

In essence, what you're saying is that we have an application protocol
here that is so unusual that it should throw out the lessons of the
past 40 years, which have taught us how you construct correct
protocols.  This makes no sense whatsoever to me.

I think the notion of out of order commands on a single connection is
somewhat strange, but your reason for opposing it is several order of
magnitude stranger.

                    paul

---

From: Paul Koning <[email-masked]>
Date: Fri, 9 Nov 2001 11:36:24 -0500

> (or at least the control part - and whatever DMA schemes we put in place

I'm having trouble parsing some of this, but let me make some
comments.

The iSCSI spec says:

    The queuing capacity of the receiving iSCSI layer is MaxCmdSN -
    ExpCmdSN + 1.

That's quite unambiguous.  It means that you set the application
window to indicate what resources you have, NOT to something smaller
in the hope that the wire delays will save you from being caught at
cheating.

Each layer that has a window must use that window to control the
consumption of resources.  It must advertise the ability to accept
incoming stuff, i.e., resources are available at that layer to handle
incoming traffic.  So TCP advertises a window based on its resources,
and iSCSI (separately) advertises its own window based on iSCSI
resources needed to do iSCSI processing.  Yes, it can rely on in-order
delivery from TCP, because that's the promise TCP makes to the
application layer.  But overcommitting because you hope there will be
packets in flight is an entirely different matter.  The TCP/IP stack
makes no such guarantee, and assuming there is one is broken design.

      paul