Subject: RE: sector alignment for DataOut PDUs?

From: "Julian Satran" <[email-masked]>
Date: Thu, 28 Feb 2002 08:58:34 +0200

Where is the hint you are alluding to? Julo

Paul Koning <[email-masked]>
Sent by: [email-masked]
27-02-02 19:20

        To:     [email-masked]
        cc:     [email-masked]
        Subject:        RE: sector alignment for DataOut PDUs?

>>>>> "Eddy" == Eddy Quicksall <[email-masked]> writes:

 Eddy> Given that the TCP issues of reassembly are so much more
 Eddy> complicated than the concatenation issues necessary for target
 Eddy> data alignment, is there really a problem here?

 Eddy> If there is a problem, can someone explain that in more detail?

I'd describe it as an opportunity for optimization.

Yes, at the TCP level your segments can be strange sizes.  My goal is
to make things better once you get up to the iSCSI level, and you're
looking at an iSCSI PDU.

Right now, the initiator is allowed to send data out in strange sized
chunks.  There's a requirement for 4 byte multiples but nothing
stronger than that.

If you're implementing a disk target, what you want to do is to take
in the arriving stream of data from the initiator (DataOut or, if
applicable, immediate data) and send that off to disk as it arrives.
If data out boundaries are 512 byte multiples, this is
straightforward: each time an iSCSI PDU arrives, you can send a disk
write down your local disk I/O stack.  But as matters stand now, it's
more complex because the data out may end in the middle of a disk
block, so the bookkeeping for what disk I/Os go with what iSCSI PDUs
gets a lot more painful.

To add to this, it seems entirely likely that initiators will use 512
byte multiples, especially if you negotiate burst sizes and max pdu
sizes that are 512 byte multiples.  But the current wording in the
spec doesn't require that.  (It does hint that targets may object if
you don't do it that way -- which makes no sense at all unless there's
a MUST for initiators to do what targets are allowed to expect.)

So that's why I made the proposal.  An alternative would be to tighten
the spec so it requires all but the last PDU to be exactly the size of
the negotiated max size that applies.  That already appears to be the
intent, so it may be the easier change.

                 paul

---

From: Paul Koning <[email-masked]>
Date: Thu, 28 Feb 2002 09:40:55 -0500

>>>>> "Julian" == Julian Satran <[email-masked]> writes:

 Julian> Where is the hint you are alluding to? 

 >> ....  (It does
 >> hint that targets may object if you don't do it that way --
 >> which makes no sense at all unless there's a MUST for
 >> initiators to do what targets are allowed to expect.)

In this section:

     9.5  Unsolicited Data and Performance

        Unsolicited data on write are meant to reduce the effect 
        of latency on throughput (no R2T is needed to start send-
        ing data).  In addition, immediate data are meant to 
        reduce the protocol overhead (both bandwidth and execu-
        tion time).

        However, negotiating an amount of unsolicited data for 
        writes and sending less than the negotiated amount when 
        the total data amount to be sent by a command is larger 
        than the negotiated amount may negatively impact perfor-
        mance and may not be supported by all the targets.

Specifically, the last sentence.  That statement is not a good thing,
because it explicitly permits failures to interoperate, which a
protocol standard must never do.  Instead, a receiver must always be
required to accept anything that the standard permits the sender to
send in a state that the sender can legitimately get to. 

There are two ways to fix this: one is to forbid the sender sending
less than the negotiated unsolicited data length.  The other is to
require the receiver to accept unsolicited data of less than the
negotiated length.  Either change works because either change
establishes interoperability.

	    paul

---

From: Paul Koning <[email-masked]>
Date: Thu, 28 Feb 2002 12:07:18 -0500

>>>>> "Sajay" == Sajay Selvaraj <[email-masked]> writes:

 Sajay> The sender should definitely avoid sending less data than the
 Sajay> negotiated value of unsolicited data length when more data is
 Sajay> available to be sent.

 Sajay> However, the total data length available to be sent itself
 Sajay> could be lesser than the negotiated value. This suggests that
 Sajay> while we should mandate the receiver to accept anything less
 Sajay> than the negotiated value, we should not force the initiator
 Sajay> to send exactly the same value as the negotiated value. In
 Sajay> other words, we cannot "forbid the sender sending less than
 Sajay> the negotiated value".  I'm for the first option suggested by
 Sajay> Paul.

Sajay,

My comments were meant in the context of the case that the spec
describes: " when the total data amount to be sent by a command is
larger than the negotiated amount".  So I abbreviated a bit.  The more
precise statement would be "one is to forbid the sender sending less
than the negotiated unsolicited data length or the total I/O length,
whichever is smaller".

	  paul

---

From: Sajay Selvaraj <[email-masked]>
Date: Thu, 28 Feb 2002 22:18:56 +0530

The sender should definitely avoid sending less data than the negotiated
value of unsolicited data length when more data is available to be sent.

However, the total data length available to be sent itself could be
lesser
than the negotiated value. This suggests that while we should mandate
the
receiver to accept anything less than the negotiated value, we should
not
force the initiator to send exactly the same value as the negotiated 
value. In other words, we  cannot "forbid the sender sending less than
the negotiated value".  I'm for the first option suggested by Paul.

BTW, can we change "data amount" to "size of data" to be sent ?

Regards,
Sajay

-- 
http://san.hcltech.com

---

From: Sajay Selvaraj <[email-masked]>
Date: Thu, 28 Feb 2002 22:44:42 +0530

Hi,

  Now both the options look valid. How about mandating both of them ?

- target should  accept immediate data which is lesser than the
  negotiated value of unsolicited data length
- initiator should not send "less
  than the negotiated unsolicited data length or the total I/O length,
  whichever is smaller".

  I suppose most implementations would work this way even now.

/Sajay

-- 
http://san.hcltech.com

---

From: Paul Koning <[email-masked]>
Date: Thu, 28 Feb 2002 12:33:34 -0500

>>>>> "Sajay" == Sajay Selvaraj <[email-masked]> writes:

 Sajay> Hi, Now both the options look valid. How about mandating both
 Sajay> of them ?

 Sajay> - target should accept immediate data which is lesser than the
 Sajay> negotiated value of unsolicited data length - initiator should
 Sajay> not send "less than the negotiated unsolicited data length or
 Sajay> the total I/O length, whichever is smaller".

You could do that. 

Unfortunately, as the spec points out, flexibility comes at a cost.
If you want to make the receiver as simple and efficient as possible,
the best answer is to take away options from the sender.  Is it
important for the sender to be able to send short data?  It's not
clear to me why it would be.  

So if it isn't a problem to require the sender to send the full amount
when there is enough total data to do so, then this allows the
receiver to rely on that and be simpler and possibly more efficient.

The current spec points in that direction; the issue is that it
doesn't state it strongly enough to ensure interoperability.  My
proposal #1 keeps the intent as it was but makes the requirement
stronger so you do get interoperability.

	 paul