Subject: RE: iSCSI: FirstBurstSize and unsolicited data

From: Sanjay Goyal <[email-masked]>
Date: Fri, 14 Jun 2002 09:37:58 -0400

The specifications says
 If you send any non-immediate unsolicited data-out PDUs, the total
unsolicited amount of data should be FirstBurstSize amount of data.

Regards
Sanjay Goyal

-----Original Message-----
From: Nandakumar Ramamurthy [mailto:[email-masked]]
Sent: Friday, June 14, 2002 7:16 AM
To: [email-masked]
Subject: iSCSI: FirstBurstSize and unsolicited data

Hi All,

Consider the case where InitialR2T=yes and ImmediateData=yes.
All other values are the defaults.

The expected data transfer length for a SCSI write
operation is a value exceeding FirstBurstSize (64KB).

In the above case, the initiator will send immediate
unsolicited data (MaxRecvPDULength=8192 bytes).
After that it has to wait for an R2T from the target.

In this scenario, FirstBurstSize of data will not be sent
to the target as unsolicited data-out's cannot be sent here.

My understanding is that the remaining data can be
sent only through solicited Data-out PDUs,
and this is the expected way according to the draft.

What does the following statement(in Section 9.4.6.2 Sense Data) mean in
this context ?

<quote>

The target reports the "Not enough unsolicited data" condition only
if it does not support output (write) operations in which the total

data length is greater than FirstBurstSize, but the initiator sent

less than FirstBurstSize amount of unsolicited data, and out-of-order

R2Ts cannot be used.

</quote>

If this is specific to the SCSI layer at the target,
what is the dependency on FirstBurstSize?

Please clarify.

Thanks,
Nandakumar
Member Technical Staff
HCL Technologies, Chennai, INDIA.
http://san.hcltech.com

---

From: "Julian Satran" <[email-masked]>
Date: Fri, 14 Jun 2002 17:24:28 +0300

... or all the data - if it is less than the FirstBurstSize. Julo

Sanjay Goyal <[email-masked]>
Sent by: [email-masked]
06/14/2002 04:37 PM
Please respond to Sanjay Goyal

        To:     "'Nandakumar Ramamurthy'" <[email-masked]>, [email-masked]
        cc: 
        Subject:        RE: iSCSI: FirstBurstSize and unsolicited data

The specifications says

Regards
Sanjay Goyal

-----Original Message-----
From: Nandakumar Ramamurthy [mailto:[email-masked]]
Sent: Friday, June 14, 2002 7:16 AM
To: [email-masked]
Subject: iSCSI: FirstBurstSize and unsolicited data

Hi All,

<quote>

R2Ts cannot be used.

</quote>

Please clarify.

Thanks,
Nandakumar
Member Technical Staff

---

From: [email-masked]
Date: Fri, 14 Jun 2002 15:15:19 -0600

> less than FirstBurstSize amount of unsolicited data, and out-of-order

This text does seem strange. Why does it say "only if it does not support output (write) operations in which" the initiator sent less data than the standard requires?
Why would the target support that given that the initiator is required to send FirstBurstSize of unsolicited data if it sends non-immediate data PDUs (and total data length is greater than FirstBurstSize). Even out-of-order R2Ts are enabled, the target shouldn't be required to send an R2T for data that should have been sent unsolicited. This will complicate implementations.

The text also doesn't deal with cases where the Initiator sent only Immediate data.

Also it doesn't deal with cases where the transfer is less than First Burst size and the initiator sent non-immediate unsolicited data with a length less than the required amount.

I think the text should be

"The target reports the "Not enough unsolicited data" condition if the Initiator sent  non-Immediate unsolicited data with a totla unsolicited data length less than the smaller of FirstBurstSize and Expected Data Transfer Length."

It also isn't clear why this error has an iSCSI condition code while other similar errors do not. In particular, it is just as possible that an Initiator sends less data in response to an explicit R2T. When it sends less unsolicited data than it should, it is violating an implicit R2T. Why is violation of an implicit R2T different from violation of an explicit R2T? And why is sending less data than expected flagged with a Sense Data error but sending more data than expected is not?

Regards,
Pat

---

From: [email-masked]
Date: Fri, 14 Jun 2002 16:09:22 -0600

While there is no need to reduce FirstBurstSize to MaxRecvDataSegmentLength (current name for MaxRecvPDULength below), 11.15 says:

"FirstBurstSize MUST NOT exceed MaxBurstSize."

and it doesn't moderate that with a statement such as "unless InitialR2T=yes" so it would probably be wise to reduce FirstBurstSize if needed to satisfy that condition in case your partner checks the relative values of the parameters even when FirstBurstSize isn't important.

Regards,
Pat

-----Original Message-----
From: Eddy Quicksall [mailto:[email-masked]]
Sent: Friday, June 14, 2002 1:43 PM
To: Bill Studenmund; Nandakumar Ramamurthy
Cc: [email-masked]
Subject: RE: iSCSI: FirstBurstSize and unsolicited data

The initiator should probably reduce FirstBurstSize to 8k.

Although the spec does not seem to mandate this.

Eddy

-----Original Message-----
From: Bill Studenmund [mailto:[email-masked]]
Sent: Friday, June 14, 2002 1:22 PM
To: Nandakumar Ramamurthy
Cc: [email-masked]
Subject: Re: iSCSI: FirstBurstSize and unsolicited data

> After that it has to wait for an R2T from the target.

That is correct.

> less than FirstBurstSize amount of unsolicited data, and out-of-order

I think what that's getting at is if you do a write command and send
unsolicited data-out PDUs, you've promised to send FirstBurstSize worth of
data. If you don't, to continue the operation, the target needs to send an
R2T for the missing data. But that would be an out-of-order R2T since by
sending unsolicited data, you are acting as if you received an R2T for
FirstBurstSize data; this error R2T would be going backwards.

Take care,

Bill

---

From: "Julian Satran" <[email-masked]>
Date: Thu, 20 Jun 2002 19:56:27 +0300

Pat,

That piece is a leftover (together with the piece found by Ron) from the 
time neither FirstBurstSize nor R2T where mandatory to fullfill.
FirstBurstSize was particularly nasty because if not done properly a 
target starting to send R2T in advance to compensate for latency
might end-up with less data than expected .

Now the only condition we have to take care is "Incorrect amount of data" 
and I renamed the condition to that.

The text reads also:

The target reports the "Incorrect amount of data" condition if dur-ing 
data output the total data length to output is greater than 
First-BurstSize, but the initiator sent an amount different than 
FirstBurstSize of unsolicited non-immediate data or the amount of data 
sent as a reply to an R2T does not match the amount requested.

Julo

[email-masked]
Sent by: [email-masked]
06/15/2002 12:15 AM
Please respond to pat_thaler

        To:     [email-masked]
        cc: 

> less than FirstBurstSize amount of unsolicited data, and out-of-order

This text does seem strange. Why does it say "only if it does not support 
output (write) operations in which" the initiator sent less data than the 
standard requires?
Why would the target support that given that the initiator is required to 
send FirstBurstSize of unsolicited data if it sends non-immediate data 
PDUs (and total data length is greater than FirstBurstSize). Even 
out-of-order R2Ts are enabled, the target shouldn't be required to send an 
R2T for data that should have been sent unsolicited. This will complicate 
implementations.

The text also doesn't deal with cases where the Initiator sent only 
Immediate data.

Also it doesn't deal with cases where the transfer is less than First 
Burst size and the initiator sent non-immediate unsolicited data with a 
length less than the required amount.

"The target reports the "Not enough unsolicited data" condition if the 
Initiator sent  non-Immediate unsolicited data with a totla unsolicited 
data length less than the smaller of FirstBurstSize and Expected Data 
Transfer Length."

It also isn't clear why this error has an iSCSI condition code while other 
similar errors do not. In particular, it is just as possible that an 
Initiator sends less data in response to an explicit R2T. When it sends 
less unsolicited data than it should, it is violating an implicit R2T. Why 
is violation of an implicit R2T different from violation of an explicit 
R2T? And why is sending less data than expected flagged with a Sense Data 
error but sending more data than expected is not?

Regards,
Pat

---

From: "Julian Satran" <[email-masked]>
Date: Fri, 21 Jun 2002 14:25:43 +0300

In your example (even before the correction) nothing bad would have 
happened (the behavior is correct).
If you would have decided to send (and had InitialR2T=No) some amount of 
unsolicited data but not all
the target could have answered with error.
The error explanation is (was) a residue from when FirstBurtSize was only 
a limit (not also a mandated to send value).

Julo

"Nandakumar Ramamurthy" <[email-masked]>
06/21/2002 12:50 PM
Please respond to "Nandakumar Ramamurthy"

        To:     Julian Satran/Haifa/IBM@IBMIL
        cc: 
        Subject:        Fw: iSCSI: FirstBurstSize and unsolicited data

Julian,

This was my mail to the IPS alias which started discussions on this 
thread.

Thanks,
Nandakumar.

---

From: "THALER,PAT (A-Roseville,ex1)" <[email-masked]>
Date: Fri, 21 Jun 2002 10:46:02 -0600

Nandakumar,
 
There are no restrictions on the amount of immediate data sent other than that it must be less than MaxRecvDataSegmentLength and less than FirstBurstSize. So in the conditions you have chosen, the initiator could send any amount of ImmediateData from 4 Bytes to 8 KB. Doing so should not cause an error at the target.
 
The purpose of allowing an implicit InitialR2T is to gain efficiency by letting data start flowing without having to wait a round-trip delay for an R2T. Gaining this efficiency requires that the target know how much unsolicited data will be sent when it receives the SCSI Command PDU so that it can immediately send the first explicit R2T. The rule on sending the full FirstBurstSize (or all the data) when unsolicited data is sent in Data-out PDUs achieves this. When the target sees the SCSI Command PDU with the F bit set to 0, it knows that the first data it needs to request with an R2T starts after FirstBurstSize bytes.
 
When the SCSI Command PDU has an F bit set to 1, then the target knows that DataSegmentLength is the amound of unsolicited data being sent and it can construct its first R2T. Therefore there is no reason to restrict the amount of unsolicited data sent when only immediate data is sent (other than that it not exceed the maximums). 
 
That is what is reflected in the rules.
 
Pat
 
-----Original Message-----
From: Nandakumar Ramamurthy [mailto:[email-masked]]
Sent: Friday, June 21, 2002 1:36 AM
To: Julian Satran
Cc: [email-masked]
Subject: Re: iSCSI: FirstBurstSize and unsolicited data

Hi,
 
I have been following the discussion on this thread.
 
I still have certain doubts regarding the
conditions I specified in my original mail.
 
The conditions are :
 
FirstBurstSize = 64KB
EDTL = 100KB ( EDTL > FirstBurstSize)
MaxRecvDataSegmentLength = 8KB
ImmediateData = Yes
InitialR2T = Yes
 
In the above case the initiator cannot send unsolicited Data-out PDUs.
Here unsolicited data(ImmediateData) < FirstBurstSize.
 
Will the target report any error in this case?
 
The modified text for Section 9.4.6.2 only refers to the cases
where unsolicited Data-outs can be sent.
 
Please clarify if I am missing something very obvious.

Thanks,
Nandakumar
Member Technical Staff
HCL Technologies, Chennai, INDIA.
http://san.hcltech.com <http://san.hcltech.com> 

----- Original Message ----- 
From: Julian Satran <mailto:[email-masked]>  
To: Eddy Quicksall <mailto:[email-masked]>  
Cc: [email-masked] <mailto:[email-masked]>  
Sent: Friday, June 21, 2002 11:41 AM
Subject: RE: iSCSI: FirstBurstSize and unsolicited data

Eddy, 

I assume you meant EDTL not DSL and then the answer is yes and I again forgot to subtract  the immediate. A better formulation would be: 

The target reports the "Incorrect amount of data" condition if dur-ing data output the total data length to output is greater than First-BurstSize and the initiator sent unsolicited non-immediate data but the total amount of unsolicited data is different than FirstBurst-Size. The target reports the same error when the amount of data sent as a reply to an R2T does not match the amount requested. 

Julo 

	"Eddy Quicksall" <[email-masked]> 

06/21/2002 12:22 AM 
Please respond to "Eddy Quicksall" 

Isn't this saying that if DSL > FirstBurstSize, it would be incorrect to send non-immediate unsolicited which is not equal to FirstBurstSize? 
  
The target reports the "Incorrect amount of data" condition if dur-ing data output the total data length to output is greater than First-BurstSize, but the initiator sent an amount different than FirstBurstSize of unsolicited non-immediate data or the amount of data sent as a reply to an R2T does not match the amount requested. 
  
Eddy