Subject: RE: iSCSI: padding on intermediate sequences

From: Eddy Quicksall <[email-masked]>
Date: Tue, 26 Mar 2002 19:28:46 -0500

Maybe I didn't say something right. Here is what I meant:
 
I-T Read CDB of 1024 bytes
T-I Data-In with 513 bytes and F bit set. So, pad is 3 bytes. I don't like
pad here.
T-I Data-In with 511 bytes and F bit set. So, pad is 1 byte. This is the
only place I would like to see pad.
 
Since the F bit ends a sequence and since a transfer can have several
sequences, the spec allows each sequence to have pad. It is a problem for
the HW guys to deal with pad except at the end of the whole transfer.
 
Eddy

-----Original Message-----
From: Anshul Chadda [mailto:[email-masked]]
Sent: Tuesday, March 26, 2002 5:25 PM
To: Sukha Ghosh; 'Robert D. Russell'; Eddy Quicksall
Cc: ips@ece. cmu. edu (E-mail)
Subject: Re: iSCSI: padding on intermediate sequences

suppose there is a requirement that a Data-In PDU with F-bit set (denoting
just end of sequence and not last data-in pdu for the command) have real
data (no pad bytes) in its last 4 byte word.

Now the target has to keep track of data over sequences. if a target doesn't
have enough real data to fill the last data-in pdu (in a sequence) to
integer number of 4 byte words, then it has to wait (don't know for how
long) for data that might belong to next sequence. this will increase the
work for target, won't it? and it will break the concept of sequences too as
we want the sequences to be independent of each other and not wait for each
other.

hope i got the question correct!!

Anshul

----- Original Message -----
From: "Sukha Ghosh" <[email-masked]>
To: "'Robert D. Russell'" <[email-masked]>; "Eddy Quicksall"
<[email-masked]>
Cc: "ips@ece. cmu. edu (E-mail)" <[email-masked]>
Sent: Tuesday, March 26, 2002 3:40 PM
Subject: RE: iSCSI: padding on intermediate sequences

> Just to clarify,
>
> What Eddy (please correct me if I am wrong) is asking is that if a Data-in
> response is multiple sequences of Data-In PDUs then the ending Data in
PDUs
> (F = 1)of intermediate sequences must also follow the 4-byte alignment
rule
> with pure data payload no padding. This means that only the last data-in
PDU
> can have padding, if necessary.
>
> -----Original Message-----
> From: Robert D. Russell [mailto:[email-masked]]
> Sent: Tuesday, March 26, 2002 2:48 PM
> To: Eddy Quicksall
> Cc: ips@ece. cmu. edu (E-mail)
> Subject: Re: iSCSI: padding on intermediate sequences
>
>
> Eddy:
>
> I may not be understanding you correctly, but the
> words in section 9.7.7 I believe mean that these data
> segments should contain an integer number of 4 byte words
> of data, not arbitrary "pad" bytes.  If the data segment
> length is a multiple of 4 then there will be NO pad bytes.
>
> There seems to be confusion over your use of "pad"
> and the standard's use of "pad".  Would you please clarify?
>
> Thanks,
> Bob
>
>
> On Tue, 26 Mar 2002, Eddy Quicksall wrote:
>
> > Section 9.7.7 says:
> >
> > The Data Segments of Data-in and Data-out PDUs SHOULD be filled to the
> > integer number of 4 byte words (real payload) unless the F bit is set
> > to 1.
> >
> > This allows one to pad intermediate sequences of a transfer. But, there
> are
> > reasons why padding on intermediate sequences within a transfer can make
> > life difficult.
> >
> > Can we forbid padding on all but the last PDU for the transfer? If that
is
> > objectionable, could we forbid padding on all but the last PDU of a
> transfer
> > if DataInOrder=yes.
> >
> > Eddy
> >

---

From: Eddy Quicksall <[email-masked]>
Date: Tue, 26 Mar 2002 19:41:44 -0500

It is the sequences I am speaking of, not the segments. The current wording
of the spec would allow you to send sequences with padding in each sequence
(the segments within the sequence are ok because they can't have padding).
But if a transfer is made up of several sequences then it is more work for
the hardware if intermediate sequences were allowed to have arbitrary pad
bytes.

And there is no good reason (that I am aware of) that intermediate sequences
could not just be pure data.

Now, if there is a reason, then we should come up with a scheme that allows
one party to "just say no". One possibility is that if DataInOrder=yes, then
padding would not be allowed except in the sequence that represents the last
portion of the transfer. 

If that is objectionable, then we should have a new key
(PaddingOnlyInLastSequenceOfTotalTransfer=yes). :-)

Eddy

-----Original Message-----
From: Robert D. Russell [mailto:[email-masked]]
Sent: Tuesday, March 26, 2002 2:48 PM
To: Eddy Quicksall
Cc: ips@ece. cmu. edu (E-mail)
Subject: Re: iSCSI: padding on intermediate sequences

Eddy:

Thanks,
Bob

are
> Can we forbid padding on all but the last PDU for the transfer? If that is
transfer

---

From: "Julian Satran" <[email-masked]>
Date: Wed, 27 Mar 2002 09:27:15 +0200

The last one is allowed to be padded - Julo

"Anshul Chadda" <[email-masked]>
Sent by: [email-masked]
27-03-02 00:25
Please respond to "Anshul Chadda"

        To:     "Sukha Ghosh" <[email-masked]>, "'Robert D. Russell'" 
<[email-masked]>, "Eddy Quicksall" <[email-masked]>
        cc:     "ips@ece. cmu. edu \(E-mail\)" <[email-masked]>
        Subject:        Re: iSCSI: padding on intermediate sequences

Now the target has to keep track of data over sequences. if a target 
doesn't
have enough real data to fill the last data-in pdu (in a sequence) to
integer number of 4 byte words, then it has to wait (don't know for how
long) for data that might belong to next sequence. this will increase the
work for target, won't it? and it will break the concept of sequences too 
as
we want the sequences to be independent of each other and not wait for 
each
other.

Anshul

----- Original Message -----
From: "Sukha Ghosh" <[email-masked]>
To: "'Robert D. Russell'" <[email-masked]>; "Eddy Quicksall"

PDUs
rule
PDU
is

---

From: "Julian Satran" <[email-masked]>
Date: Wed, 27 Mar 2002 10:03:22 +0200

Eddy,

Now that we understand better you issue here are some reasons why we may 
not want to do what you propose:

the target may ask for a number of bytes that is not a multiple of 4 
(should we outlaw this? I don't think we want)
there might be devices that require/generate in "bursts" that are not 
multiples of 4

I think that we avoided abuse by what we have and we should leave further 
optimization to the implementer for specific devices.

Julo

Eddy Quicksall <[email-masked]>
Sent by: [email-masked]
27-03-02 02:41
Please respond to Eddy Quicksall

        To:     "ips@ece. cmu. edu (E-mail)" <[email-masked]>
        cc: 
        Subject:        RE: iSCSI: padding on intermediate sequences

It is the sequences I am speaking of, not the segments. The current 
wording
of the spec would allow you to send sequences with padding in each 
sequence
(the segments within the sequence are ok because they can't have padding).
But if a transfer is made up of several sequences then it is more work for
the hardware if intermediate sequences were allowed to have arbitrary pad
bytes.

And there is no good reason (that I am aware of) that intermediate 
sequences
could not just be pure data.

Now, if there is a reason, then we should come up with a scheme that 
allows
one party to "just say no". One possibility is that if DataInOrder=yes, 
then
padding would not be allowed except in the sequence that represents the 
last
portion of the transfer. 

Eddy

-----Original Message-----
From: Robert D. Russell [mailto:[email-masked]]
Sent: Tuesday, March 26, 2002 2:48 PM
To: Eddy Quicksall
Cc: ips@ece. cmu. edu (E-mail)
Subject: Re: iSCSI: padding on intermediate sequences

Eddy:

Thanks,
Bob

are
is
transfer

---

From: Robert Snively <[email-masked]>
Date: Wed, 27 Mar 2002 08:43:40 -0800

Since there is no requirement by the SCSI ULP to say anything about data
until
the SCSI Status has been presented, there is no need to concern ourselves
with
device dependent burst boundaries.  We can make the boundaries convenient
for the technology.  In FCP, all bursts were required to begin on a 
4-byte boundary, and all except the last to end on a 4-byte boundary.  This
simplified DMA and segmentation/reassembly mechanisms
in all hardware paths.  Looking at the TCP/IP formats, I see a similar
bias toward 4-byte structures, which implies that the same rule would be
a constructive addition to the document.
 
This simply prohibited the target from asking for a number of intermediate
bytes that was not a multiple of 4.
 
The folks using odd byte block counts had to manage the buffers in
a manner appropriate to such behaviors.  Practically, I do not believe that
there were many such devices, nor was this a burden to any devices that did
support such behavior.
 
Bob

-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Wednesday, March 27, 2002 12:03 AM
To: Eddy Quicksall
Cc: ips@ece. cmu. edu (E-mail); [email-masked]
Subject: RE: iSCSI: padding on intermediate sequences

Eddy, 

Now that we understand better you issue here are some reasons why we may not
want to do what you propose: 

*	the target may ask for a number of bytes that is not a multiple of 4
(should we outlaw this? I don't think we want) 

*	there might be devices that require/generate in "bursts" that are
not multiples of 4 

Julo 

	27-03-02 02:41 

Eddy

-----Original Message-----
From: Robert D. Russell [mailto:[email-masked]]
Sent: Tuesday, March 26, 2002 2:48 PM
To: Eddy Quicksall
Cc: ips@ece. cmu. edu (E-mail)
Subject: Re: iSCSI: padding on intermediate sequences

Eddy:

Thanks,
Bob

are
transfer

---

From: Eddy Quicksall <[email-masked]>
Date: Wed, 27 Mar 2002 13:19:36 -0500

Julian,
 
Notice in the example I gave below, that the pad was only applied when the F
bit was set. Section 9.7.7 says: 

So my example is not forbidden. Can you please change section 9.7.7 to be
more clear and forbid this behavior? I suggest something like:
 
The Data Segments of Data-in and Data-out PDUs SHOULD be filled to the 
integer number of 4 byte words (real payload) unless the F bit is set 
to 1 on the last Data PDU that concludes the entire requested
data transfer for the task from the target's perspective.
 
Eddy

-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Wednesday, March 27, 2002 1:42 AM
To: Eddy Quicksall
Cc: Sanjay Goyal
Subject: RE: iSCSI: padding on intermediate sequences

The spec clearly forbids this behavior - data pdus (except the last) have to
filled (not padded) to a 4 byte boundary. That is what the text says. 

Regards, 
Julo 

26-03-02 18:52 

        To:        Julian Satran/Haifa/IBM@IBMIL 
        cc:        Sanjay Goyal <[email-masked]>, Sukha Ghosh
<[email-masked]> 
        Subject:        RE: iSCSI: padding on intermediate sequences 

  
  
Eddy 
-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Tuesday, March 26, 2002 11:09 AM
To: Eddy Quicksall
Subject: Re: iSCSI: padding on intermediate sequences

Eddy - it says real payload. They must be filled not padded. That is
mandataed to avoid padding abuse. 

But I assume that I missread your question. 

Julo 

26-03-02 16:52 

       To:        "ips@ece. cmu. edu (E-mail)" <[email-masked]> 
       cc:         
       Subject:        iSCSI: padding on intermediate sequences 

Section 9.7.7 says: 

This allows one to pad intermediate sequences of a transfer. But, there are
reasons why padding on intermediate sequences within a transfer can make
life difficult. 
 
Can we forbid padding on all but the last PDU for the transfer? If that is
objectionable, could we forbid padding on all but the last PDU of a transfer
if DataInOrder=yes. 
 
Eddy

---

From: Eddy Quicksall <[email-masked]>
Date: Thu, 28 Mar 2002 09:31:01 -0500

Apparently FC had to make this restriction (see EMAIL from Robert Snively
[[email-masked]] dated 3/27). We should consider that as a basis for
discussion.

until
with
 
Eddy

 -----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Wednesday, March 27, 2002 3:03 AM
To: Eddy Quicksall
Cc: ips@ece. cmu. edu (E-mail); [email-masked]
Subject: RE: iSCSI: padding on intermediate sequences

Eddy, 

Julo 

	27-03-02 02:41 

Eddy

-----Original Message-----
From: Robert D. Russell [mailto:[email-masked]]
Sent: Tuesday, March 26, 2002 2:48 PM
To: Eddy Quicksall
Cc: ips@ece. cmu. edu (E-mail)
Subject: Re: iSCSI: padding on intermediate sequences

Eddy:

Thanks,
Bob

are
transfer