Subject: RE: iSCSI: Iterating long text responses

From: Robert Snively <[email-masked]>
Date: Mon, 2 Jul 2001 07:55:41 -0700

I have looked over the options for iterating long text
responses.  In the context of SCSI driver stacks, #5 is 
a familiar behavior that could be applied very easily
to the text commands.  If anyone was interested, it could
even be extended to a future SCSI command within the SCSI 
command set.  I favor #5.

Bob Snively                        e-mail:    [email-masked]
Brocade Communications Systems     phone:  408 487 8135
1745 Technology Drive
San Jose, CA 95110

>  
>  5. Do #4, but allow the initiator to specify the amount of data
>     it is willing to accept:
>  
>     I->T  Text Command (F bit set, AcceptResponseLength=4096)
>     T->I  Text Response (first PDU, F bit set, 
>  TotalResponseLength=12288)
>  
>     In the above example, we have created a new text command field:
>  
>        AcceptResponseLength
>  
>     And in the text response PDU:
>  
>        TotalResponseLength
>  
>     The initiator indicated it was willing to accept a 4k response.
>     The target returned the first 4k in the text response, but set
>     the F bit since it was at its limit.  It also returned a
>     TotalResponseLength field.  Since this was greater than the
>     AcceptResponseLength, the initiator knows the amount of buffer
>     space it will need to get the full response.  It can then choose
>     whether it will re-send the command, and if so, can give it a
>     large enough response length:
>  
>     I->T  Text Command (F bit set, AcceptResponseLength=12288)
>     T->I  Text Response (first PDU, F bit clear)
>     T->I  Text Response (next PDU, F bit clear)
>     T->I  Text Response (last PDU, F bit set, 
>  TotalResponseLength=12288)
>  
>     Note that most initiators will probably send an 
>  AcceptResponseLength
>     of the largest response they would normally accept, and that most
>     target responses will fit in one or a few PDUs anyway.

---

From: "Julian Satran" <[email-masked]>
Date: Mon, 16 Jul 2001 15:24:53 +0300

I think that both 4 and 5 involve in fact some state to be kept at the
target between PDUs sent in something related to
to a "task control block" if we assume that all the text commands carry the
same ITT.
4 enables the initiator to reuse its parse buffer while 5 requires the
initiator to allocate a buffer for all the text responses (or keep the pipe
closed).
4 is simpler than 5.  If you add to 4 a handle that the initiator has to
give back the target next time (the bookmark)
then the target does not have to keep state.
A 0 bookmark says start from the top.  It is very much like an offset (that
was mentioned) but it is generic and opaque.

Regards,
Julo

Mark Bakke <[email-masked]> on 29-06-2001 23:18:20

Please respond to Mark Bakke <[email-masked]>

To:   IPS <[email-masked]>
cc:
Subject:  iSCSI: Iterating long text responses

Initiator developers-

   Please respond to the questions at the end.

Thanks,

Mark

The current iSCSI draft allows text command and response
PDUs of up to 4096 bytes.  While we don't see any real
problems for the command PDU size, commands such as SendTargets
can easily exceed the response size.

There are several ways we can fix this.  The first two solutions
require no differences in the current iSCSI text command and
response; the latter three involve the use of the F bit.

1. Assume that such commands are done on a "special" connection
   or are handled completely in software, and allow its response
   PDU to be as large as it needs to be.

   This one appears too restrictive to be a solid solution.  It
   will also weaken any data digests done on the longer PDU.

2. Create an iterative SendTargets key (and do the same for any
   other text commands that need this), that would allow the
   initiator to request the "next target" or "next address".

   This would work, but would require any new command that needed
   a large response to implement an iterator.  It also means that
   each part of the response from the iterator would have to fit
   in the 4k PDU.

The remainder of these require that only one text command sequence
be outstanding on a connection at a given time.  They use the F
bit to indicate the last PDU of the sequence.  Note that connection
allegiance is assumed for the entire sequence, and text commands
are never retried on another connection; a new command is issued
instead.

3. Do a text-response R2T, where the initiator controls when the
   next partial response is sent.  This would be more generic:

   I->T  Text Command (F bit set)
   T->I  Text Response (first PDU, F bit cleared)
   I->T  Text Command (with some indicator that we want more)
   T->I  Text Response (next PDU, F bit cleared)

   ...

   I->T  Text Command (with indicator that we want more)
   T->I  Text Response (last PDU, F bit set)

   The main problem with this is that the target must keep track
   of the state of its responses; if the initiator just stops asking,
   it may have to keep a buffered response around until it times out,
   the connection is dropped, or another text command comes along.

4. Allow multiple response PDUs to a single text command:

   I->T  Text Command (F bit set)
   T->I  Text Response (first PDU, F bit cleared)
   T->I  Text Response (next PDU, F bit cleared)

   ...

   T->I  Text Response (last PDU, F bit set)

   Basically, we are doing (3) without the R2T.  The initiator,
   upon sending the text command, must be prepared either to
   accept as many PDUs as come back, or throw them away if it
   can't handle them.  This solution is a lot like #1, but with
   the response spread across 4k PDUs.

   Also note that this (and the following scheme) avoid the problem
   of the target keeping state; it sends ALL of the response PDUs
   without the initiator asking for them.

   I->T  Text Command (F bit set, AcceptResponseLength=4096)
   T->I  Text Response (first PDU, F bit set, TotalResponseLength=12288)

      AcceptResponseLength

      TotalResponseLength

   I->T  Text Command (F bit set, AcceptResponseLength=12288)
   T->I  Text Response (first PDU, F bit clear)
   T->I  Text Response (next PDU, F bit clear)
   T->I  Text Response (last PDU, F bit set, TotalResponseLength=12288)

   Note that most initiators will probably send an AcceptResponseLength
   of the largest response they would normally accept, and that most
   target responses will fit in one or a few PDUs anyway.

   #5 is really a compromise between #3 and #4; the target has the
   benefit of being less statefull, and the initiator has the benefit
   of controlling the amount of data it receives.

I would like to recommend either #4 or #5.  I think that #5 is
probably the safest solution, but #4 may not be a problem for anyone.

Assuming that none of the implementors of initiators have a problem
with #4, I would pick that.

If they do have a problem with it, we should go with #5.

Targets probably don't care much whether we do #4 or #5.

Initiator developers-

  Please indicate which solution (#4 or #5) appeals to you.

--
Mark A. Bakke
Cisco Systems
[email-masked]
763.398.1054

---

From: "Jim Hafner" <[email-masked]>
Date: Mon, 16 Jul 2001 09:00:28 -0700

Julo,

On the other hand, 5 is closer to a READ operation (and aren't we talking
about something like an iSCSI read).  For both 4 and 5, the only state
needed between PDUs is within the context of the same command (as is the
case for a read), not between commands.   5 is a bit more efficient in that
the target only sends (and prepares to send) the amont of data requested.
And in 4, there's extra wire traffic for no reason (when the initiator
isn't prepared for all the data).  Admittedly, it's not a lot, but ...

How does use of a bookmark not imply state?  Doesn't the target have to
keep the reference of the bookmark to the data between commands?  Is the
bookmark valid only for the one initiator or for all initiators?  What
happens if the data set changes between commands?  What if the bookmark is
no longer valid?

I think 5 is the right approach.

Jim Hafner

Julian Satran/Haifa/IBM@[email-masked] on 07/16/2001 05:24:53 am

Sent by:  [email-masked]

To:   Mark Bakke <[email-masked]>
cc:   "IPS <ips" <[email-masked]>
Subject:  Re: iSCSI: Iterating long text responses

Regards,
Julo

Initiator developers-

Thanks,

Mark

   ...

   ...

      AcceptResponseLength

      TotalResponseLength

Initiator developers-

--
Mark A. Bakke
Cisco Systems
[email-masked]
763.398.1054

---

From: Mark Bakke <[email-masked]>
Date: Wed, 18 Jul 2001 14:32:44 -0500

The "bookmark" solution that Julian had mentioned is
currently in the -98 version of the iSCSI draft at:

http://www.haifa.il.ibm.com/satran/ips/draft-ietf-ips-iSCSI-06-98.txt

This solution is more akin to #3, since the target does need
to keep track of bookmarks, and the initiator requests each
piece.

This doesn't seem quite as simple as #5, but it will work.  Simple
targets that will not return enough addresses to fill a 4k text
response PDU would not have to implement the bookmarks at all.

Anyway, this is a change from what we had been discussing on the
list, so it's worth looking at, and would perhaps be a good thing
to do a consensus call on so we can put this thing to rest either
way.

--
Mark

-- 
Mark A. Bakke
Cisco Systems
[email-masked]
763.398.1054

---

From: "Jim Hafner" <[email-masked]>
Date: Tue, 24 Jul 2001 09:34:16 -0700

Mark et al,

I may be wrong here, but I read the general feeling of the group that
option #5 (simple request length from the initiator and total available
response length from the target) was the best option.

I would like someone (David?) to call for concensus on this issue.

I do not like the bookmark proposal.  It needlessly requires:
1) the target to create and maintain this bookmark between Text commands
2) requires a whole new error state when the bookmark is invalid
3) doesn't spell out how long the target has to maintain the bookmark
4) requires the initiator to track the bookmark between Text commands
etc.
In other words, it's a lot more work for no real gain.

Can we get closure on this issue, please?

Jim Hafner

Mark Bakke <[email-masked]>@ece.cmu.edu on 07-18-2001 12:32:44 PM

To:   Julian Satran/Haifa/IBM@IBMIL
cc:   "IPS <ips" <[email-masked]>
Subject:  Re: iSCSI: Iterating long text responses

--
Mark

--
Mark A. Bakke
Cisco Systems
[email-masked]
763.398.1054