Subject: Re: Target record not to span PDUs?

From: "Julian Satran" <[email-masked]>
Date: Wed, 22 Aug 2001 13:22:57 +0300

Good point. I think that we can do this as we have a limit on the size of
an individual record far lower than 4096.

I'll ad the following text to 2.9.5:

   A Key=value pair must be confined to a given text response even in the
   presence of bookmark - i.e., it must start and end within one Text
   Response.

Julo

Tow Wang <[email-masked]> on 22-08-2001 05:06:44

Please respond to Tow Wang <[email-masked]>

To:   Julian Satran/Haifa/IBM@IBMIL
cc:   "[email-masked]'" <[email-masked]>
Subject:  Target record not to span PDUs?

Julian (and all):

Hello. This is regarding draft 07; could we require that target records NOT
span across
PDU's if a text response for SendTargets is very long? Upon reading
appendix E, it seems
that a response (of 4096 bytes in length) could end with:

[Begin data segment]
...
TargetName=I.got.chopped.4096
TargetAddress=10.1.1.45
[End data segment]

In the above case, one could not determine whether the address is IP V4 or
V6. Even if it
had had enough space to put in the whole address, port and group tag, one
cannot parse and
process the record until inspecting the data segment of the next text
response PDU, and
this would involve cumulative buffering, back-parsing, etc. I think the
above complexity
could be avoided, as I can't imagine any single record exceeding 4096 bytes
in length.

Tow Wang

---

From: Robert Snively <[email-masked]>
Date: Wed, 22 Aug 2001 09:00:18 -0700

It sounds like there may be some confusion between "physical 
data unit" (the TCP/IP frame) and "iSCSI Protocol Data Unit" 
(the messaging unit for iSCSI).  I cannot see where Tom's 
problem arises, since Protocol Data Units are very well 
behaved and should not have the problems he is discussing. 
You cannot complete Protocol Data Unit processing until you
know you have all of them and that useful information does not
span them.  The assembly of protocol data units into useful
complete messages should be done at a layer below that where
you interpret the contiguous bytes of data in the context of
the complete messages.  I think as a general rule that any
iSCSI action should be able to span PDUs, including the 
Target Record.

Bob 

> -----Original Message-----
> From: Tow Wang [mailto:[email-masked]]
> Sent: Tuesday, August 21, 2001 7:07 PM
> To: 'Julian Satran'
> Cc: [email-masked]'
> Hello. This is regarding draft 07; could we require that 
> target records NOT span across
> PDU's if a text response for SendTargets is very long? Upon 
> reading appendix E, it seems
> In the above case, one could not determine whether the 
> address is IP V4 or V6. Even if it
> had had enough space to put in the whole address, port and 
> group tag, one cannot parse and
> process the record until inspecting the data segment of the 
> next text response PDU, and
> this would involve cumulative buffering, back-parsing, etc. I 
> think the above complexity
> could be avoided, as I can't imagine any single record 
> exceeding 4096 bytes in length.
> 
> Tow Wang
> 
>

---

From: "Julian Satran" <[email-masked]>
Date: Fri, 24 Aug 2001 00:00:35 +0300

Bob,

The record in question is a key=value pair. What Tow was asking is a about
a way to avoid having to "concatenate
strings" from two Text responses that carry the SendTargets answers one
after another and we have stated that
explicitly now.

Julo

Robert Snively <[email-masked]>@ece.cmu.edu on 22-08-2001 19:00:18

Please respond to Robert Snively <[email-masked]>

Sent by:  [email-masked]

To:   Tow Wang <[email-masked]>, Julian Satran/Haifa/IBM@IBMIL
cc:   "[email-masked]'" <[email-masked]>
Subject:  RE: Target record not to span PDUs?

Bob

---

From: [email-masked]
Date: Thu, 23 Aug 2001 16:25:58 -0700 (PDT)

> The approach you propose adds processing burden to every action on a target
> side to put a key=action into a PDU.  I envision the target side having
> contiguous memory to formulate the entire text response, and it then sends
> this in sequential PDUs. 

I may have to disagree with this statement. The response is dynamic and
is based on the request (the keys that the initiator is interested in)
and it is already doing some work here, in decoding the key and deciding
an appropriate action/value for the key. Therefore, I don't believe it is
either difficult or too much work to adjust the PDU to contain a complete
key/value pair in one PDU. If they span multiple PDUs, the initiator
has to do additional to work to coalesce them into meaningful pairs,
and I don't see any major gains in trying to do that.

-JP

---

From: JP Raghavendra Rao <[email-masked]>
Date: Thu, 23 Aug 2001 15:50:06 -0700 (PDT)

-JP

---

From: Robert Snively <[email-masked]>
Date: Thu, 23 Aug 2001 17:09:34 -0700

Folks,

Welcome to storage.

I am with Stephen on this one.  The choice of how to 
transmit a set of information is done at a relatively low layer,
often in hardware.  
That choice is made independent of what useful boundaries
may or may not exist in the data itself.  His vision of an
assemblage of data that is logically contiguous transmitted
by a layer that will cut it on arbitrary protocol data unit
boundaries is correct.

This same argument (and the same resolution) will naturally
be made for disk data blocks, tape data blocks, and everything
else which has an interesting internal boundary.  The
internal boundary is not part of the formation of the IP
packet, the TCP segment, or the iSCSI protocol data unit.

On those special cases where this is very important, each
data unit should be requested and transmitted in a separate
small interchange.  As an example, if you don't like the 
key=data to span a PDU, just ask for a single key=data unit
in each request.  Then the data will be small enough that,
no matter how many PDUs it uses in the transmission process,
you don't chew up very much buffer.

Once you have requested a set of data, you have committed to
accepting ALL the data that fits in the specified allocation
size.  You have also committed to processing that data only
after all retries and out-of-order transmissions have been
completed and the data buffer has been returned to you.  Looking
ahead based on knowledge of the PDU (whose boundaries are 
not exposed at the standard delivery interface) can be useful
only if you are willing to consider such look-aheads as 
provisional.  You cannot fully use the data until you have
accounted for and completed the analysis of any ambiguities
that were temporarily created by spanning of PDUs.

Note that part of this discussion seems to center on whether
you expect PDU transmission to be hardware or software.  I
expect it to be hardware, so creating the proposed restriction
is very burdensome.

Bob Snively                        e-mail:    [email-masked]
Brocade Communications Systems     phone:  408 487 8135
1745 Technology Drive
San Jose, CA 95110

> Sent: Thursday, August 23, 2001 3:49 PM
> To: Wheat, Stephen R; [email-masked]
> Subject: Re: Target record not to span PDUs?
> 
> 
> Stephen and all:
> 
> Hello. Let me explain myself further, as I do care about this 
> issue. As I
> interpret the appendix E, the amount of data (bytes) 
> generated in response to
> "SendTargets" can be arbitrarily long, and thus may require 
> an unpredictable
> number of text command/text response exchanges until all 
> target records have
> been delivered to the initiator.
> 
> I don't think it is a good idea to have to buffer all the 
> data segments of
> these text responses (until seeing a final bit) before 
> parsing the target
> records and freeing the buffers. I was not confusing between 
> the concepts of
> "having to buffer all TCP frames until all bytes of a single 
> PDU have been
> received"
> and
> "having to buffer and hold all PDU's sent in response to SendTargets"
> The former is a given due to the nature of networking, but 
> the latter is bad,
> and likely unfeasible, if one has limited buffering memory. It is more
> memory-efficient (and slightly improves performance) to FULLY 
> process each
> INDIVIDUAL PDU that has been delivered to the recipient, so 
> one can release the
> buffering resources and prepare to process the following PDUs 
> coming in.
> 
> Also, in this situation the target is much better informed 
> about the sizes,
> starting points and end points of the target records. While 
> the target writes
> each record into the "contiguous memory" buffer alluded to, 
> it must keep track
> of the offset for the amount of space already used in the 
> buffer (as it
> certainly is not infinite in size!). That will give an 
> immediate indication of
> whether a record will fit within the a PDU, or should be 
> shipped in the
> following PDU.
> 
> Tow
> 
> 
> "Wheat, Stephen R" wrote:
> 
> > Julian,
> >
> > Not to be pedantic, but, again, the arbitrary decision to avoid the
> > concatenation adds burden to the target side and restricts 
> the future
> > capability of key=values that exceed the max text PDU length.
> >
> > Again, the host side will need to buffer the input until it 
> has all arrived
> > anyway.  Is there a stronger technical reason for this 
> other than the
> > avoidance of concatenation?
> >
> > this in sequential PDUs.  Having to search for where a 
> key=value pair spans
> > the PDU boundary is work.
> >
> > Furthermore, being able to span PDUs facilitates a simpler 
> specification.
> >
> > Maybe Tow can explain the burden of concatenation.  If Tow 
> really doesn't
> > care, can we go back to allowing the records to span PDUs?
> >
> > Does anyone else care about this?
> >
> > Stephen
> >
> > From: Julian Satran [mailto:[email-masked]]
> > Sent: Thursday, August 23, 2001 2:01 PM
> > To: [email-masked]

---

From: JP Raghavendra Rao <[email-masked]>
Date: Thu, 23 Aug 2001 15:18:30 -0700 (PDT)

-JP

-JP