Subject: iSCSI : Concerns regarding DataRN in READ Data PDU.

From: Santosh Rao <[email-masked]>
Date: Mon, 08 Jan 2001 20:03:03 -0800

Julian/All,

Can anybody comment on the true benefit that the DataRN feature provides
that justifies the complexity and performance issues it introduces to
the protocol ?

I see the following issues with DataRN :

1) It adds a performance penalty in that on every READ Data PDU
targets will have to initialize one additional 32-bit word in the
header.

2) Extra traffic on the outbound context from initiators sending
NOP-OUTs to acknowledge the DataRNs.

3) Performance path checks for the "P" bit on every READ PDU.
(Alternatively, checks for the DataNumber field from the Login
Text Keys).

4)  Implementing DataRNs per task is too short-lived to provide any
real benefit. Implementing them per connection or session can result
in quick use up of the 32-bit DataRN space (since each frame consumes
one DataRN), adding additional complexity to the target code to use the
"P"
bit appropriately.

5) Further, "DataNumber", which is the size of the un-acknowledged
DataRN window is negotiated at login time, whereas this is really
desired to be a dynamic variable based on the target's current I/O load
and the number of initiators speaking to it. Thus, DataRN can end up
being
under-utilized resulting in too much NOP-OUT traffic, or it can be
over-utilized resulting in too much usage of the "P" bit in the READ
PDUs, which, again, can cause too much NOP-OUT traffic.

6) Considering that the DataRN generation and acknowledgement are
functions
that would typically be in SCSI hardware assist engines and there is a
need to
keep these as simple as possible, adding such features in the spec means

that these SCSI assist engines MUST implement DataRN generation
and acknowledgement [since initiators have no way to turn it off if a
target decides to use it]

What is the true benefit of this feature ? Is it intended to be used as
:

a) Will allow targets to do partial freeing of their memory buffers as
and when DataRNs are received ?

b) Will allow targets to send back only the un-acknowledged data (as
seen from the ExpDataRN received on the last NOP-OUT from the initiator)
when the initiators retry commands with the "Retry" bit set ?

If the benefit intended is (a), then, in order to derive this benefit,
targets will have to advertise their DataNumber login key to be < the
avg. number of frames transmitted per READ I/O. Having a DataNumber >
the avg. number of frames per READ I/O implies the life of the task ends
[and buffers for the task get released] before the initiator sends
NOP-OUT, thereby, defeating benefit (a).

If the benefit intended is (b) [and i do not see this explicitly stated
in the draft], then, this is a dangerous assumption which can lead to
data corruption issues.

When commands are retried with the "Retry" bit set, are targets allowed
to send back partial data based on the last ExpDataRN they received ? If
so, this can have multiple issues like :

o    complexity in handling I/O underrun cases for the retried command
      which only saw partial data transfer from the target.

o    If the target only sent back partial data based
      on its last ExpDataRN received, then, it can open up possible data

      corruption windows.

All in all, I believe that the DataRN is adding too much complexity and
potential performance impact to iSCSI for the benefits it may provide.

Moreover and most importantly, the draft does not give initiators any
control over the usage of this feature and if a target decides to use
this feature, initiators will have to support it. This exposes the
initiators to all of the above issues without being able to turn off
this feature.

I would like to therefore ask that :

either,
a) DataRN support be removed from the iSCSI draft.

or
b) The support for DataRN be negotiated at login time giving initators
control over this feature and allowing them to disable this feature.
This will allow SCSI hardware assist engines to skip implementation of
this functionality and their associated software can merely turn off
DataNumber in the login negotiation.

Thanks,
Santosh

---

From: "VOIGT,DOUG (HP-Boise,ex1)" <[email-masked]>
Date: Wed, 10 Jan 2001 17:11:41 -0800

It seems to me that this feature is redundant with TCP's own delivery
guarantees, in an effort to minimize work loss in the event of TCP
connection failure.  I too believe this level of complexity and potential
normal performance impact compared with accelerated recovery from improbable
error events is not likely to yield a net gain.

Is recoverable TCP connection loss more common than disk array controller
failure?  Since different controllers in the same array are (quite
reasonably) viewed as different targets (pending SAM change), iSCSI level
failover won't compensate for controller failure.  This example leads me to
suggest that the benefit we are shooting for has a high likelihood of being
overshadowed by other failure modes whose recovery is less graceful.

Doug Voigt

> -----Original Message-----
> From: Santosh Rao [mailto:[email-masked]]
> Sent: Monday, January 08, 2001 9:03 PM
> To: IPS Reflector
> Subject: iSCSI : Concerns regarding DataRN in READ Data PDU.
> 
> 
> Julian/All,
>

---

From: [email-masked]
Date: Thu, 11 Jan 2001 19:10:47 +0200

Santosh,

DataRN was meant for what you call case B to be used by targets that
perform long inbound transfers
(like mirorring or tape copy) . Initiators have very few things to do to
suport it and targets will use as they see fit.
Setting a counter in the header is not exactly an large overhead - it is a
local counter in the command block (unprotected).

As for your statements about the data corruption window  - could you
explain please?

Julo

Santosh Rao <[email-masked]> on 09/01/2001 06:03:03

Please respond to Santosh Rao <[email-masked]>

To:   IPS Reflector <[email-masked]>
cc:
Subject:  iSCSI : Concerns regarding DataRN in READ Data PDU.

Julian/All,

      corruption windows.

either,

or

Thanks,
Santosh

 - santoshr.vcf

---

From: [email-masked]
Date: Thu, 11 Jan 2001 19:13:02 +0200

And BTW the same coments where made with regard to ftp!

Was it a good thing that it it was unrecoverable?

Should we repeat this story for the SCSI Extended Copy?

Julo

Julian/All,

      corruption windows.

either,

or

Thanks,
Santosh

 - santoshr.vcf

---

From: "Peglar, Robert" <[email-masked]>
Date: Fri, 12 Jan 2001 07:36:33 -0600

This is a very important feature indeed.

Think of block-sequential operations on
tens- (or even hundreds-) of TB of data.
Think of sessions that perform in-band continuous
management functions; they have to live forever,
by definition.
Think of sessions between virtual initiators
and virtual targets.  Forever, potentially.

Thank you,
Rob

Rob Peglar
Director, Storage Architecture
XIOtech Corporation
(314) 308-6983

> From: [email-masked] [mailto:[email-masked]]
> Sent: Friday, January 12, 2001 3:33 AM
> To: [email-masked]
> Subject: RE: iSCSI : Concerns regarding DataRN in READ Data PDU.
> 
> 
> 
> 
> This feature was meant for long lasting operations (tons of data) that
> could then recover
> with ease. As for complexity... it is local counter.  If that is not
> available all long lasting commands will have to provide their own
> checkpointing mechanisms or there will be no long-lasting commands.
> 
> Julo
> 
> "VOIGT,DOUG (HP-Boise,ex1)" <[email-masked]> on 11/01/2001 03:11:41
> 
> Please respond to "VOIGT,DOUG (HP-Boise,ex1)" <[email-masked]>
> 
> Subject:  RE: iSCSI : Concerns regarding DataRN in READ Data PDU.
> 
> 
> 
>

---

From: [email-masked]
Date: Sun, 14 Jan 2001 14:23:30 +0200

Santosh,

The general assumption was that a target is resource constrained.
If a target wants acks an initiator should be able to give those to the
target.
If a target gives a large number in the DataNumber it practically says that
it can do without and will request ACK explicitly or not at all. Letting an
initiator control this might negatively impact performance for unrelated
initiators as the target will be bound to keep resources for the lazy
initiator.

I hope this clarifies the issue.

Julo

Santosh Rao <[email-masked]> on 14/01/2001 00:16:33

To:   Julian Satran/Haifa/IBM@IBMIL
cc:   [email-masked]
Subject:  Re: iSCSI : Concerns regarding DataRN in READ Data PDU.

Julian,

My original request on this issue was suggesting 2 alternatives. If a WG
consensus cannot be achieved on eliminating the DataRN and all of its
complexity, I would like to re-iterate the alternative proposal that was
made, that initiators should be allowed to disable this
feature during login negotiation.

Regards,
Santosh

--
#################################
Santosh Rao
Software Design Engineer,
HP, Cupertino.
email : [email-masked]
Phone : 408-447-3751
#################################