Subject: Re: iSCSI: Reject, CmdSN, and DataSN

From: "Julian Satran" <[email-masked]>
Date: Sun, 29 Jul 2001 08:53:42 +0300

Mallikurjun,

I thought that my earlier comment on this was clear - but apparently it was
not.
The problem is real.
There as trivial case in which a format error in a command, will force the
window to close as the reject may be the only trafic and it does not carry
ExpCmdSN etc.

There are several things to be done (and as I mentioned earlier I fixed
both):

   the reject command should carry the same updates as all others (ExpCmdSN
   etc.) - good practice anyhow
   the local fields should be updated even if the command is rejected -
   whenever possible (there is no digest error)
   The command slot should be considered filled

Here is the proposed new text for 2.19

1.1  Reject

   Byte /    0       |       1       |       2       |       3       |
      /              |               |               |               |
     |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
     +---------------+---------------+---------------+---------------+
    0|1|1| 0x3f      |1| Reserved (0)                                |
     +---------------+---------------+---------------+---------------+
    4| Reserved (0)  | DataSegmentLength                             |
     +---------------+---------------+---------------+---------------+
    8/ Reserved (0)                                                  /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   24| StatSN                                                        |
     +---------------+---------------+---------------+---------------+
   28| ExpCmdSN                                                      |
     +---------------+---------------+---------------+---------------+
   32| MaxCmdSN                                                      |
     +---------------+---------------+---------------+---------------+
   26| Reserved (0)                                                  |
     +---------------+---------------+---------------+---------------+
   40| Reason        | Reserved (0)  | First Bad Byte or Rsvd(0)     |
     +---------------+---------------+---------------+---------------+
   44| Reserved (0)                                                  |
     +---------------+---------------+---------------+---------------+
   48| Digests if any...                                             |
     +---------------+---------------+---------------+---------------+
   xx/ Complete Header of Bad PDU                                    /
    +/                                                               /
     +---------------+---------------+---------------+---------------+
   yy

   It may happen that a target receives a PDU with a format error (e.g.,
   inconsistent fields etc.) or a digest error (e.g., invalid payload or
   header). The target returns the header (not including digest) of the PDU
   in error as the data of the response.

1.1.1     Reason

   The reject Reason is coded as follows:

      1 - Format Error
      2 - Data (payload) Digest Error
      3 - Data-SNACK Reject
      4 - Command-Retry Reject
      5 - Protocol Error (e.g., SNACK request for a status that was already
      acknowledged)
      6 - Command-in-progress
      7 - Command Replay Not Supported
      8 - Immediate Command Reject - too many immediate commands
      9 - Immediate Command Retry Reject - task not found
      10 - Bookmark rejected (old or different ITT)
      11 - command not supported in this session type
      15 - Full Feature Phase Command before login

   Some of the reject reasons terminate or prevent the creation of a task
   at the target and no retry is possible in those cases. Format error for
   a command, Command Retry Reject and Full Feature Phase Command before
   login are in this category.

   In all the cases in which creation of a SCSI task is prevented or a SCSI
   task is terminated because of the reject, the target must issue a proper
   SCSI command response including a Check Condition Status (0x02). The
   sense key to be used is iSCSI REJECT (the numeric value and format for
   additional-sense-data to be coordinated with T10).  If the error is
   detected while data from the initiator is still expected (the command
   PDU did not contain all the data and the target has not received a Data
   PDU with the final bit Set) the target MUST wait until it receives the
   Data PDU with the F bit set before sending the Response PDU.

   In all the cases in which the rejected PDU is a non-retry numbered
   request PDU with a valid CmdSN field that number MUST be considered as
   received and acknowledged appropriately.

Julo

"Mallikarjun C." <[email-masked]> on 28-07-2001 22:32:41

Please respond to [email-masked]

To:   [email-masked]
cc:   [email-masked]
Subject:  Re: iSCSI: Reject, CmdSN, and DataSN

Mark,

When you say "uses up a SN", do you mean the ExpCmdSN/ExpDataSN
is updated?  If so, I see a problem with at least CmdSN.

Let's consider commands first.  The only Reject reason code that
I see where a target could update its ExpCmdSN is when there
is a payload-digest error on immediate data.  In all the other
cases, either the target doesn't care (immediate/retry/non-FFP),
or can't know for sure (format error).  I suggest that we _not_
advance the ExpCmdSN on immediate data digest error to give the
initiator an opportunity to re-issue the command to preserve
command ordering.

Let's now consider data.  Again the only Reject reason code I see
for advancing ExpDataSN is payload-digest error ("2").  Either the
target
  (a) would generate a recovery R2T (if DataSequenceOrder was
      negotiated as "no" and target supports Within-command
      recovery), or
  (b) eventually signal a service delivery subsystem failure for
      the command (iSCSI response code 0x02).

For (b), it doesn't matter if the target updates ExpDataSN or not.
For (a) OTOH, the target should update the ExpDataSN to deal with
the current data burst (that reminds me that I need to fix this
in Within-command algorithms in Appendix F!).

So to summarize, I think a Reject'ed CmdSN should not be considered
as successfully received on the target.

Your comments are welcome.
--
Mallikarjun

Mallikarjun Chadalapaka
Networked Storage Architecture
Network Storage Solutions Organization
MS 5668   Hewlett-Packard, Roseville.
[email-masked]

>When a PDU is rejected, I assume that the CmdSN is still
>updated, as well as the DataSN where applicable.  That is,
>a rejected command still uses up a SN.  It probably wouldn't
>hurt to state this in the reject section.
>
>Since the Reject response does not contain ExpCmdSN, if the
>last command before the window is closed is rejected, the
>initiator has to rely on prior commands completing to e-open
>the window.  This will usually work, but what if the window
>size is reduced to one outstanding command for some reason?
>Any command that is rejected will close the window for good.
>A sequence of rejected commands equal to the window size will
>do the same.
>
>Any thoughts?
>
>--
>Mark A. Bakke
>Cisco Systems
>[email-masked]
>763.398.1054
>

---

From: "Mallikarjun C." <[email-masked]>
Date: Sun, 29 Jul 2001 17:50:53 PDT

Julian,

Three comments.

- I am *not* arguing against your latest changes adding
  more fields to Reject PDU.  I think they are required,
  sorry if my note came across differently.

- You seem to think that targets can pick out CmdSN on format 
  errors, my argument is that targets may not be able to - from
  *badly-formatted* PDUs (certainly not, if in the worst case, 
  the "format errors" are a result of broken framing sync).

- OTOH, targets can definitely decode the CmdSN on one class of
  "data digest errors" - those on immediate data.  You seem to
  discount this possibility.

Please review my note closely and offer comments on my proposal
not to advance CmdSN for any Rejects as the general case.  I 
suppose we can talk about this as well during our scheduled phone
call tomorrow.  

Regards.
--
Mallikarjun 

Mallikarjun Chadalapaka

>To: [email-masked]
>From: "Julian Satran" <[email-masked]>
>Date: Sun, 29 Jul 2001 08:53:42 +0300
>Sender: [email-masked]
>Precedence: bulk
>Status: RO
>
>Mallikurjun,
>

---

From: "Julian Satran" <[email-masked]>
Date: Mon, 30 Jul 2001 10:18:49 +0300

Mallikarjun,

We are talking past one another. My comment did not say anything about
considering the rejected command as received rather than they may get the
window closed as the rejects did not convey feedback.

We will certainly take it up on the phone.

Regards,
Julo

"Mallikarjun C." <[email-masked]> on 30-07-2001 03:50:53

To:   [email-masked]
cc:

Julian,

Three comments.

Regards.
--
Mallikarjun

Mallikarjun Chadalapaka