Subject: iSCSI : Digest Error recovery causes data corruption

From: Santosh Rao <[email-masked]>
Date: Mon, 29 Jan 2001 20:28:35 -0800

Julian & All,

Section 5.5 on digest errors states that an initiator MUST "discard
and re-start" a task when it encounters a header or data digest
error, provided it can recognize the initiator task tag.

I assume the above reference to re-start is to the use of the "retry"
bit. (?)

If so, there is a possibility of this error recovery mechanism leading
to
data corruption. The probability is reduced with the removal of partial
data recovery (based on DataRN/SN). However, data corruption can
still occur as follows :

- Last Data PDU on a READ I/O is returned from target to initiator.

- Initiator detects a header or data digest error on this last Data PDU
   and discards the PDU.

- Initiator re-starts the task (using the "retry" bit).

- Target sends in the Status PDU on the previous instance of the
   command. (It is not clear from the spec what the initiator does with
   stale frames that continue to arrive on the previous instance of the
   I/O. For now, I assume the initiator will, by some mechanism
   discard such frames.)

- When the target receives the "retry" of this command, it thinks it
    has sent back all the data and so, it only sends back Status for
this
    "retry".

- Initiator has no count based checks and so, depends (trusts !)
   the target with its status, based on which it reports a successful
   I/O completion to the initiator's SCSI ULP, [indicating no residual
   count, since target thought it sent all the data].

- SCSI ULP assumes a completed I/O and notifies application,
   [since it depends on the initiator notifying it with an appropriate
   service response on an underflow, which the initiator in this case
   did not detect].

- Application encounters data corruption, due to the missing Data
    PDU which was discarded by the initiator on a digest error,
    and which was never re-sent by the target, since it does partial
    recovery by only sending the status.

The StatSN based partial status recovery can lead to such dangerous
corner cases causing possible data corruption scenarios.

Regards,
Santosh

---

From: Matt Wakeley <[email-masked]>
Date: Tue, 30 Jan 2001 00:24:08 -0800

Santosh Rao wrote:

> this "retry".

The retry means, retry the whole thing, not just the status.  The target must
resend all the data, and the status.

> corner cases causing possible data corruption scenarios.

Where do you get this notion of "StatSN based partial status recovery"?

> Santosh

-Matt

---

From: Santosh Rao <[email-masked]>
Date: Tue, 30 Jan 2001 00:54:58 -0800 (PST)

> Where do you get this notion of "StatSN based partial status recovery"?

From Section 5.1 :

"-upon receiving the new/retry commands the target will resume 
  command execution; ... <<snip snip>>
  data and for "terminated" commands retransmitting the
	  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  Status & Sense while retaining the original StatRN."
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The above seems to indicate that StatSN did allow for 
re-transmission of just the status [and sense, if present].

Regards,
Santosh

ps :
(Just the fact that 2 independent implementors could interpret
"retry" handling at the target in such different ways goes
to prove that the "retry" handling description in the draft 
needs a lot more clarity and a lot less ambiguity.)

-- 
#################################
Santosh Rao
Software Design Engineer,
HP, Cupertino.
email : [email-masked]
Phone : 408-447-3751
#################################