Subject: RE: Unsolicited Data.

From: "BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)" <[email-masked]>
Date: Wed, 6 Jun 2001 15:35:38 +0100

The current spec states that the F bit is "set to 1 when the immediate data
that accompany the command are all the data associated with this command. It
is an error if the Length and Expected Length do not match and this bit is
set to 1". 

I interpret this as there is no more data to follow and not that the
initiator has opted not to use the unsolicited data feature.  Therefore the
spec needs to be modified to indicate that if unsolicited data has been
negotiated (i.e. InitialR2T=no), then the initiator MUST send unsolicited
data of length = min( FirstBurstSize, ExpectedTransferLength ) minus any
immediate data sent).

Matthew Burbridge
NIS-Bristol
Hewlett Packard
Telnet: 312 7010
E-mail: [email-masked]

-----Original Message-----
From: Ayman Ghanem [mailto:[email-masked]]
Sent: Tuesday, June 05, 2001 6:31 PM
To: [email-masked]
Subject: RE: Unsolicited Data.

If the initiator decides not to send immediate or unsolicited data when it
has negotiated to do so, then the initiator must set the F-bit in the
command PDU. This prompts the target to send R2T.

I still agree that the spec should indicate that the initiator MUST use the
resources it has negotiated. If it has negotiated the option to send
immediate data or unsolicited data then it should do that to the limits
allowed. If it has negotiated a PDU length, it must not send data PDUs less
than the negotiated limit except for last one. While most implementation may
do that for performance reasons, I would prefer defining this in the spec.

-Ayman

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]On Behalf Of
> BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)
> Sent: Tuesday, June 05, 2001 11:58 AM
> To: [email-masked]'
> Subject: Unsolicited Data.
>
>
> I'm not sure if this has been discussed before but it is causing some
> confusion.  The statement below implies that if immediate data has been
> negotiated then the initiator MAY use it.
>
> "If ImmediateData is set to no and InitialR2T is set to no then the
> initiator MUST NOT send unsolicited immediate data but MAY send one
> unsolicited burst of Data-OUT PDUs."
>
> Therefore the target must wait for the initial burst of unsolicited data
> before issuing the first R2T (if there is subsequent data).  If the
> initiator decides not to send it then the target must timeout and
> issue the
> R2T for the initial data.  Can the spec be changed to state that if
> unsolicited data has been negotiated, then the initiator MUST use it.
>
> Thanks
>
> Matthew Burbridge
> NIS-Bristol
> Hewlett Packard
> Telnet: 312 7010
> E-mail: [email-masked]
>
>

---

From: "Ayman Ghanem" <[email-masked]>
Date: Wed, 6 Jun 2001 11:48:21 -0500

That section was modified. Julian posted the revised text here:

http://ips.pdl.cs.cmu.edu/mail/msg04647.html

-Ayman

> Sent: Wednesday, June 06, 2001 9:36 AM
> To: [email-masked]'

---

From: "Sanjeev Bhagat \(TRIPACE/Zoetermeer\)" <[email-masked]>
Date: Thu, 7 Jun 2001 01:05:39 +0200

Hello,

All that is said below is ok but I cannot understand one thing here
----- why would initiator choose not to send unsolicited immediate data when
it has negotiated for IntialR2T= Yes??

Sanjeev

---

From: Stephen Bailey <[email-masked]>
Date: Wed, 06 Jun 2001 21:19:48 -0400

Sanjeev,

> why would initiator choose not to send unsolicited immediate data when

I assume you mean InitialR2T=no.

I would do this because I think that the real benefit of unsolicited
data is for writes where ALL the write data can be sent unsolicited,
and I'm happy to have the target schedule the transfer of (ALL) data
for larger writes.  Of course, others would say I'm a moron.

Assuming the overwhelming majority of storage use is file access,
small writes are exclusively file system metadata writes.  Unsolicited
data is a huge win for metadata writes because the latency of these
metadata writes is frequently not covered by transferring other data
(within a single access thread).  On the other hand, the file system
(usually) generates enough concurrent demand for real data writes that
the flow control (R2T) latency is well hidden.

You might argue that if you're running a file system optimized fors
today's SANs over a link with a HUGE bandwidth * delay product
(greater than the expected demand created by the file system), sending
large amounts of unsolicited write data will substantially improve the
link utilization.  That is true, but I don't think many targets are
going to offer unsolicited data bursts remotely that large.

In a hardware accelerate implementation, unsolicited data will be
handled differently than solicited data.  The target can chose where
the solicited data is delivered, but unsolicited data arrives through
a general-delivery path.  Therefore, if the offered unsolicited data
burst size approaches the regular burst size, an implementation will
need TWICE as much buffer memory (half in the general-delivery area
and half in the flow-controlled buffer area) to support the same
operation load.  Maybe this tradeoff is OK, but given the cost
sensitivity of storage targets, I doubt it.

As always, I welcome comments from those who see things differently.

Steph

---

From: "Mallikarjun C." <[email-masked]>
Date: Thu, 07 Jun 2001 11:04:59 PDT

Steph,

I agree with you that mandating unsolicited data to be sent always
is limiting, even if InitialR2T=no is negotiated.

I suggest the following for consideration -
- If the command F-bit is set:
       - if immediate data is present, it's all the data
         for the command. (as in rev06)
       - if immediate data is not present, there are no
         unsolicited data PDUs following. 
- If the command F-bit is not set:
       - if immediate is present, there is more solicited
         data to follow.
       - if immediate data is not present, there may be
         solicited data depending on "Expected Data Transfer 
         Length".

The only case this disallows is that of having _both_ unsolicited
separate PDUs and immediate data for a command - which I personally 
am okay with.  This is disallowed in rev06 anyway, but for some 
reason seemed like was changed later.
--
Mallikarjun 

Mallikarjun Chadalapaka
Networked Storage Architecture
Network Storage Solutions Organization
MS 5668	Hewlett-Packard, Roseville.
[email-masked]

---

From: "Sanjeev Bhagat \(TRIPACE/Zoetermeer\)" <[email-masked]>
Date: Fri, 8 Jun 2001 00:50:30 +0200

Mallkarjun,

According to me it is possible for initiator **not** to send  unsolicited
data if we follow the latest changes on the section 2.3.1 posted at

According to me

When IntitialR2T= No, Immediate Data = Yes and target doesnt want to
send any immediate data but it does want to send unsolicited data it can be
done like this
Step 1
Send iSCSI CmdPDU and  Dont send F bit and Dont send any immediate data by
not including any extra length in datasegmentlength
Step 2
Send Unsolicited Data and set the F bit

When IntitialR2T= No, Immediate Data = Yes and target doesnt want to
send any immediate data nor does it want to send unsolicited data it can be
done as follows

Set F bit in Task and Flag Attributes and dont send any immediate data
by not including any extra length in datasegmentlength .

My only doubt here is that it is not any illegal condition to set this F bit
when there is no immediate data to be sent and no unsolicited data to be
sent. Current spec doesnt declare it illegal but it doesnt really suggest it
this way either.

PLEASE DO COMMENT ON THIS

Regards,
Sanjeev