Subject: RE: sector alignment for DataOut PDUs?

From: Eddy Quicksall <[email-masked]>
Date: Wed, 27 Feb 2002 11:41:31 -0500

Given that the TCP issues of reassembly are so much more complicated than
the concatenation issues necessary for target data alignment, is there
really a problem here?

If there is a problem, can someone explain that in more detail?

Eddy

-----Original Message-----
From: Buck Landry [mailto:[email-masked]]
Sent: Wednesday, February 27, 2002 12:15 AM
To: Rod Harrison; Paul Koning
Cc: [email-masked]
Subject: RE: sector alignment for DataOut PDUs?

>>> Also, I think we should consider making the "alignment" any
multiple of 512 the target wants
<<<

Well.. if we put no restrictions on it, and I decided my target should only
handle one data-out pdu per scsi write cmd, I might abuse this and force an
alignment of (512 * ReallyHugeNumber).  This might demand more resources or
complexity from the initiator than is wished for, although I can't think of
any reasons off the top of my head (perhaps I should think harder).  OTOH I
am fully aware that the target is the entity that is supposed to be cheap n'
easy to implement...

Perhaps you could limit the alignment size by the target's MaxRecvPDULength?
Or...  

I believe this "multiple-of" idea also needs to be reconciled with
MaxBurstSize.  If MaxBurstSize is negotiated by the initiator to 512 bytes,
then by my understanding the target still must send 512-byte R2Ts for writes
(and hence, expect 512-byte data-out pdus), even if the target's sector size
is 4k.

--buck 

-----Original Message-----
From: Rod Harrison [mailto:[email-masked]]
Sent: Tuesday, February 26, 2002 5:51 PM
To: Paul Koning
Cc: [email-masked]
Subject: RE: sector alignment for DataOut PDUs?

	I was talking about DATA-OUT PDUs.

	I would suggest we also restrict any target that wants to
negotiate for the 512 byte DATA-OUT option to not doing anything
weird with FirstBurstSize or MaxRecvPDULength. I think it would
unreasonable to allow a target to require 512 byte aligned
DATA-OUTs and have those other two parameters not multiples of
512 bytes.

	How do we handle the case where the iSCSI target node has
multiple LUNs and not all of them are disks? Perhaps the rule for
transfers lengths that are not multiples of 512 bytes works OK
here?

	Also, I think we should consider making the "alignment" any
multiple of 512 the target wants, not just 512. Otherwise it
seems to me you are back where you started if you have a disk
with 4096 byte sectors and the initiator chooses to send you
DATA-OUT PDUs that are not a multiple of 4k.

	- Rod

-----Original Message-----
From: Paul Koning [mailto:[email-masked]]
Sent: Tuesday, February 26, 2002 11:05 PM
To: [email-masked]
Cc: [email-masked]
Subject: RE: sector alignment for DataOut PDUs?

>>>>> "Rod" == Rod Harrison <[email-masked]> writes:

 Rod> I have no objection in principle to this but I have a
concern
 Rod> over non-media access commands. Things like mode select
 Rod> payloads, i.e. defect lists etc are not necessarily 512
byte
 Rod> multiples. The wording would need to allow "odd" sizes when
the
 Rod> SCSI transfer length is not a 512 byte multiple.

Are those DataOut or DataIn?  The target can arrange inbound PDU
sizes
to be whatever multiple is helpful if it wants to, that isn't the
case
I was talking about.

In any case, if there are cases where the total transfer length
for
data out isn't a multiple of 512, then the rule would have to
allow
the last transfer to have an odd length.  The earlier transfers
could
still be constrained if the constraint is enabled.

      paul

---

From: Paul Koning <[email-masked]>
Date: Wed, 27 Feb 2002 12:20:13 -0500

>>>>> "Eddy" == Eddy Quicksall <[email-masked]> writes:

 Eddy> Given that the TCP issues of reassembly are so much more
 Eddy> complicated than the concatenation issues necessary for target
 Eddy> data alignment, is there really a problem here?

 Eddy> If there is a problem, can someone explain that in more detail?

I'd describe it as an opportunity for optimization.

Yes, at the TCP level your segments can be strange sizes.  My goal is
to make things better once you get up to the iSCSI level, and you're
looking at an iSCSI PDU.

Right now, the initiator is allowed to send data out in strange sized
chunks.  There's a requirement for 4 byte multiples but nothing
stronger than that.

If you're implementing a disk target, what you want to do is to take
in the arriving stream of data from the initiator (DataOut or, if
applicable, immediate data) and send that off to disk as it arrives.
If data out boundaries are 512 byte multiples, this is
straightforward: each time an iSCSI PDU arrives, you can send a disk
write down your local disk I/O stack.  But as matters stand now, it's
more complex because the data out may end in the middle of a disk
block, so the bookkeeping for what disk I/Os go with what iSCSI PDUs
gets a lot more painful.

To add to this, it seems entirely likely that initiators will use 512
byte multiples, especially if you negotiate burst sizes and max pdu
sizes that are 512 byte multiples.  But the current wording in the
spec doesn't require that.  (It does hint that targets may object if
you don't do it that way -- which makes no sense at all unless there's
a MUST for initiators to do what targets are allowed to expect.)

So that's why I made the proposal.  An alternative would be to tighten
the spec so it requires all but the last PDU to be exactly the size of
the negotiated max size that applies.  That already appears to be the
intent, so it may be the easier change.

	paul

---

From: Paul Koning <[email-masked]>
Date: Wed, 27 Feb 2002 14:46:44 -0500

>>>>> "Rod" == Rod Harrison <[email-masked]> writes:

 Rod> Paul, I strongly object to your alternative of tightening the
 Rod> spec wording to force all PDUs to be max sized except the
 Rod> last. As I mentioned in an earlier email, since draft v09 we
 Rod> don't have a negotiated PDU size we have arbitrary maximums set
 Rod> by the receivers. Forcing an initiator to send "full" PDUs is
 Rod> unreasonable since the target may set a huge MaxRecvPDULength.

 Rod> The intent of the current wording was, if I remember correctly,
 Rod> to have the initiator always send as much unsolicited data as
 Rod> possible. This was to avoid the target having to wait for the F
 Rod> bit on the unsolicited data before issuing R2T(s) for the
 Rod> remaining payload. Note that unsolicited data, and its maximum
 Rod> size, are negotiated parameters and that that current SHOULD is
 Rod> quite different from forcing initiators to always send "full"
 Rod> DATA-OUT PDUs.

That's a good point.

So for unsolicited data we have a negotiated max length, but you're
allowed to send less than that.  Similarly, for R2T the target
requests a length and the initiator is allowed to send less than that.

That can be tightened up, but doing so doesn't constrain the
individual PDUs that are being sent, only the whole burst.  I was
concerned with the individual PDUs, so a different rule is needed
there.  And as you point out, the max PDU length is no longer suitable
for that.  (Undoing the change of what Max PDU length means back to
what it was in -08 doesn't sound like the right way to address this.)

 Rod> Please note that I don't object to your original proposal, just
 Rod> to the alternative which I believe would be a huge functional
 Rod> change.

Thanks... your comments make sense to me.

	  paul

---

From: "Rod Harrison" <[email-masked]>
Date: Wed, 27 Feb 2002 19:25:37 -0000

Paul,

	I strongly object to your alternative of tightening the spec
wording to force all PDUs to be max sized except the last. As I
mentioned in an earlier email, since draft v09 we don't have a
negotiated PDU size we have arbitrary maximums set by the
receivers. Forcing an initiator to send "full" PDUs is
unreasonable since the target may set a huge MaxRecvPDULength.

	The intent of the current wording was, if I remember correctly,
to have the initiator always send as much unsolicited data as
possible. This was to avoid the target having to wait for the F
bit on the unsolicited data before issuing R2T(s) for the
remaining payload. Note that unsolicited data, and its maximum
size, are negotiated parameters and that that current SHOULD is
quite different from forcing initiators to always send "full"
DATA-OUT PDUs.

	Please note that I don't object to your original proposal, just
to the alternative which I believe would be a huge functional
change.

	- Rod

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On
Behalf Of
Paul Koning
Sent: Wednesday, February 27, 2002 5:20 PM
To: [email-masked]
Cc: [email-masked]
Subject: RE: sector alignment for DataOut PDUs?

 Eddy> Given that the TCP issues of reassembly are so much more
 Eddy> complicated than the concatenation issues necessary for
target
 Eddy> data alignment, is there really a problem here?

 Eddy> If there is a problem, can someone explain that in more
detail?

Yes, at the TCP level your segments can be strange sizes.  My
goal is
to make things better once you get up to the iSCSI level, and
you're
looking at an iSCSI PDU.

Right now, the initiator is allowed to send data out in strange
sized
chunks.  There's a requirement for 4 byte multiples but nothing
stronger than that.

If you're implementing a disk target, what you want to do is to
take
in the arriving stream of data from the initiator (DataOut or, if
applicable, immediate data) and send that off to disk as it
arrives.
If data out boundaries are 512 byte multiples, this is
straightforward: each time an iSCSI PDU arrives, you can send a
disk
write down your local disk I/O stack.  But as matters stand now,
it's
more complex because the data out may end in the middle of a disk
block, so the bookkeeping for what disk I/Os go with what iSCSI
PDUs
gets a lot more painful.

To add to this, it seems entirely likely that initiators will use
512
byte multiples, especially if you negotiate burst sizes and max
pdu
sizes that are 512 byte multiples.  But the current wording in
the
spec doesn't require that.  (It does hint that targets may object
if
you don't do it that way -- which makes no sense at all unless
there's
a MUST for initiators to do what targets are allowed to expect.)

So that's why I made the proposal.  An alternative would be to
tighten
the spec so it requires all but the last PDU to be exactly the
size of
the negotiated max size that applies.  That already appears to be
the
intent, so it may be the easier change.

	paul