Subject: iSCSI - change - negotiation reset

From: "Julian Satran" <[email-masked]>
Date: Sat, 13 Oct 2001 02:43:59 +0300

Dear colleagues,

I suggested earlier a slight modification in the way a text negotiation is
reset.
The current text  (08) assumes that this is done through a Task Management
request of ABORT TASK.

We amy want to use a simpler mechanism that is already defined for long
text responses - and extend it to long text exchanges in general.

The (slightly) modified text is attached.

Comments?

Julo

(See attached file: Change-negotiation-reset.txt)

Change Text Negotiation Reset
_____________________________
3.10 Text Request

The Text Request is provided to allow the exchange of information and for future extensions. It permits the initiator to inform a target of its capabilities or to request some special operations.

An initiator MUST have only one outstanding Text Request on a connection at any given time.

Byte /    0       |       1       |       2       |       3       |
   /              |               |               |               |
  |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
  +---------------+---------------+---------------+---------------+
 0|X|I| 0x04      |F| Reserved                                    |
  +---------------+---------------+---------------+---------------+
 4| Reserved      | DataSegmentLength                             |
  +---------------+---------------+---------------+---------------+
 8| Reserved                                                      |
  +                                                               +
12|                                                               |
  +---------------+---------------+---------------+---------------+
16| Initiator Task Tag                                            |
  +---------------+---------------+---------------+---------------+
20| Target Transfer Tag or 0xffffffff                             |
  +---------------+---------------+---------------+---------------+
24| CmdSN                                                         |
  +---------------+---------------+---------------+---------------+
28| ExpStatSN                                                     |
  +---------------+---------------+---------------+---------------+
32/ Reserved                                                      /
 +/                                                               /
  +---------------+---------------+---------------+---------------+
48| Digests if any...                                             |
  +---------------+---------------+---------------+---------------+
  / DataSegment (Text)                                            /
 +/                                                               /
  +---------------+---------------+---------------+---------------+

3.10.1 F (Final) Bit

When set to 1 it indicates that this is the last or only text request in a sequence of commands; otherwise it indicates that more commands will follow.

3.10.2 Initiator Task Tag

The initiator assigned identifier for this Text Request.
If the command is sent as part of a sequence of text requests and responses, the Initiator Task Tag MUST be the same for all the requests within the sequence (similar to linked SCSI commands).

3.10.3 Target Transfer Tag

When the Target Transfer Tag is set to the reserved value 0xffffffff, it tells the target that this is a new request and the target should reset any internal state associated with the Initiator Task Tag.

The target sets in a text response the Target Transfer Tag to a value other than the reserved value 0xffffffff whenever it indicates that it has more data to send or more operations to perform associated with the specified Initiator Task Tag (it MUST do so whenever it sets the F bit to 0 in the response). By copying the Target Transfer Tag from the response to the next Text Request, the initiator tells the target to continue the operation for the specific Initiator Task Tag.

This mechanism allows the initiator and target to transfer a large amount of textual data over a sequence of text-command/text-response exchange or to perform extended negotiation sequences  

A target MAY reset its internal state if an exchange is stalled by the initiator for a long time or if it is running out of resources.

Long text responses are handled as in the following example:

I->T Text SendTargets=all (F=1,TTT=0xffffffff)
T->I Text <part 1> (F=0,TTT=0x12345678) 
I->T Text <empty> (F=1, TTT=0x12345678)
T->I Text <part 2> (F=0, TTT=0x12345678) 
I->T Text <empty> (F=1, TTT=0x12345678)
...
T->I Text <part n> (F=1, TTT=0xffffffff) 

3.10.4 Text

The initiator sends the target a set of key=value or key=list pairs encoded in UTF-8 Unicode. All the text keys and text values specified in this document are to be presented and interpreted in the case they appear in this document (they are case sensitive). The key and value are separated by a '=' (0x3d) delimiter. Every key=value pair (including the last or only pair) MUST be followed by at least one null (0x00) delimiter.  A list is a set of values separated by comma (0x2c). 

Character strings are represented as plain text. Binary items can be encoded using their decimal representation (with or without leading zeros) or hexadecimal representation (e.g., 8190 is 0x1ffe).  Upper and lower case letters may be used interchangeably in hexadecimal notation (i.e., 0x1aBc, 0x1AbC, 0X1aBc and 0x1ABC are equivalent). Binary items can also be encoded using the more compact Base64 encoding as specified by [RFC2045] preceded by the 0b.  Key names MUST NOT exceed 63 bytes.

If not specified otherwise the maximum length of an individual value (not its encoded representation) is 255 bytes not including the delimiter (comma or null).

The data lengths of a text request or response MUST NOT exceed MaxRecvPDULength (a per connection negotiated parameter).  

A Key=value pair can span Text request or response boundaries (i.e. a key=value pair can start in one PDU and continue on the next).

The target responds by sending its response back to the initiator. The response text format is similar to the request text format. 

Some basic key=value pairs are described in Appendix D. All keys in Appendix D, except for the X- extension format, MUST be supported by iSCSI initiators and targets. 

Manufacturers may introduce new keys by prefixing them with X- followed by their (reversed) domain name, for example the company owning the domain acme.com can issue: 

X-com.acme.bar.foo.do_something=3

Any other key not understood by the target may be ignored by the target without affecting basic function. However the Text Response for a key that was not understood MUST be key=NotUnderstood.

Text operations are usually meant for parameter setting/negotiations but can be used also to perform some long lasting operations.

Text operations that will take a long time should be placed in their own Text request.  

3.11 Text Response

The Text Response PDU contains the target's responses to the initiator's Text request. The format of the Text field matches that of the Text request.

Byte /    0       |       1       |       2       |       3       |
   /              |               |               |               |
  |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
  +---------------+---------------+---------------+---------------+
 0|1|1| 0x24      |F| Reserved                                    |
  +---------------+---------------+---------------+---------------+
 4| Reserved      | DataSegmentLength                             |
  +---------------+---------------+---------------+---------------+
 8| Reserved                                                      |
  +                                                               +
12|                                                               |
  +---------------+---------------+---------------+---------------+
16| Initiator Task Tag                                            |
  +---------------+---------------+---------------+---------------+
20| Target Transfer Tag or 0xffffffff                             |
  +---------------+---------------+---------------+---------------+
24| StatSN                                                        |
  +---------------+---------------+---------------+---------------+
28| ExpCmdSN                                                      |
  +---------------+---------------+---------------+---------------+
32| MaxCmdSN                                                      |
  +---------------+---------------+---------------+---------------+
36/ Reserved                                                      /
 +/                                                               /
  +---------------+---------------+---------------+---------------+
48| Digests if any...                                             |
  +---------------+---------------+---------------+---------------+
  / DataSegment (Text)                                            /
 +/                                                               /
  +---------------+---------------+---------------+---------------+

3.11.1 F (Final) Bit

When set to 1 in response to a text request with the Final bit set to 1 the F bit indicates that the target has finished the whole operation.  Otherwise, if set to 0 in response to a text request with the Final Bit set to 1 it indicates that the target has more work to do (invites a follow-on text request).  A text response with the F bit set to 1 in response to a text request with the F bit set to 0 is a protocol error. 

A text response with the F bit set to 1 MUST NOT contain key=value pairs that may require additional answers from the initiator.

A text response with the F bit set to 0 MUST have a Target Transfer Tag field set to a value different than the reserved 0xffffffff.

3.11.2 Initiator Task Tag

The Initiator Task Tag matches the tag used in the initial Text request.

3.11.3 Target Transfer Tag

When a target has more work to do (e.g., can't transfer all the remaining text data in a single Text response or has to continue the negotiation) and has enough resources to proceed it MUST set the Target Transfer Tag to a value different from the reserved value of 0xffffffff.  

The initiator MUST copy this Target Transfer Tag in its next request to indicate that it wants the rest of the data.

If the target receives a Text Request with the Target Task Tag set to the reserved value of 0xffffffff it resets its internal state associated with the given Initiator Task Tag.  

When a target can't finish the operation in single text response but has not enough resources to continue it rejects the Text request with an appropriate Reject code. A target may reset its internal its internal state associated with an Initiator Task Tag and expressed through the Target Transfer Tag, if the initiator fails to continue the exchange for some time, and reject subsequent Text requests with the Target Transfer Tag set to the "stale" value.

3.11.4 Text Response Data

The Text Response Data Segment contains responses in the same key=value format as the Text request and with the same length and coding constraints. Appendix A and Appendix D lists some basic Text requests and their Responses.  

Although the initiator is the requesting party and controls the request-response initiation and termination the target can offer key=value pairs of its own as part of a sequence and not only in response to the initiator.

---------------------------------

3.17 Reject

Byte /    0       |       1       |       2       |       3       |
   /              |               |               |               |
  |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
  +---------------+---------------+---------------+---------------+
 0|1|1| 0x3f      |1| Reserved    | Reason        | Reserved      |
  +---------------+---------------+---------------+---------------+
 4| Reserved      | DataSegmentLength                             |
  +---------------+---------------+---------------+---------------+
 8/ Reserved                                                      /
 +/                                                               /
  +---------------+---------------+---------------+---------------+
24| StatSN                                                        |
  +---------------+---------------+---------------+---------------+
28| ExpCmdSN                                                      |
  +---------------+---------------+---------------+---------------+
32| MaxCmdSN                                                      |
  +---------------+---------------+---------------+---------------+
36| DataSN or Reserved                                            |
  +---------------+---------------+---------------+---------------+
40| Reserved                                                      |
  +---------------+---------------+---------------+---------------+
44| Reserved                                                      |
  +---------------+---------------+---------------+---------------+
48| Digest (if any)                                               |
  +---------------+---------------+---------------+---------------+
xx/ Complete Header of Bad PDU                                    /
 +/                                                               /
  +---------------+---------------+---------------+---------------+
yy/Vendor specific data (if any)                                  /
  /                                                               /
  +---------------+---------------+---------------+---------------+
zz

Reject is used to indicate an iSCSI error condition (protocol, unsupported option etc.).

3.17.1 Reason

The reject Reason is coded as follows:

+------+-----------------------------------------+------------------+
| Code | Explanation                             | Can the original |
| (hex)|                                         | PDU be re-sent?  |
+------+-----------------------------------------+------------------+
| 0x01 | Full Feature Phase Command before login | no               |
|      |                                         |                  |
| 0x02 | Data (payload) Digest Error             | yes  (Note 1)    |
|      |                                         |                  |
| 0x03 | Data-SNACK Reject                       | yes              |
|      |                                         |                  |
| 0x04 | Protocol Error (e.g., SNACK request for | no               |
|      | a status that was already acknowledged) |                  |
|      |                                         |                  |
| 0x05 | Command not supported in this session   | no               |
|      | type                                    |                  |
|      |                                         |                  |
| 0x06 | Immediate Command Reject - too many     | yes              |
|      | immediate commands                      |                  |
|      |                                         |                  |
| 0x07 | Task in progress                        | no               |
|      |                                         |                  |
| 0x08 | Invalid SNACK                           | no               |
|      |                                         |                  |
| 0x09 | Target Transfer Tag Reject for this     | no               |
|      | Initiator Task Tag                      |                  |
|      |                                         |                  |
| 0x0a | Long Operation Reject - Can't generate  | yes              |
|      | Target Transfer Tag - out of resources  |                  |
|      |                                         |                  |
| 0x0b | Negotiation Reset                       | no              |
+------+-----------------------------------------+------------------+

Note 1: For iSCSI data PDUS, this is done only if target requests retransmission with a recovery R2T.    However, if this is the data digest error on immediate data, no signal from the target is necessary for PDU retransmission if desired so by the initiator.

All other values for reason are reserved.

In all the cases in which a pre-instantiated SCSI task is terminated because of the reject, the target must issue a proper SCSI command response with CHECK CONDITION as described in section 3.4.3.  If the error is detected while data from the initiator is still expected (the command PDU did not contain all the data and the target has not received a Data-out PDU with the final bit Set) the target MUST wait until it receives the Data-out PDU with the F bit set before sending the Response PDU.  

For additional usage semantics of Reject PDU, please see section 8.2.

3.17.2 DataSN

This field is valid only if the Reason code is "Invalid SNACK" and the SNACK was a data SNACK.  The DataSN is the last sequence number that the target sent for the task.

3.17.3 Complete Header of Bad PDU

The target returns the header (not including digest) of the PDU in error as the data of the response.

----------------------------------
6. Operational Parameter Negotiation Outside the Login Phase

Some operational parameters MAY be negotiated outside (after) the login phase.

Parameter negotiation in full feature phase is done through Text requests and responses.

Operational parameter negotiation MAY involve several text request-response exchanges always started and terminated by the initiator. The initiator MUST indicate its intent to terminate the negotiation by setting the F bit to 1; the target sets the F bit to 1 on the last response.

If the target responds to a text request with the F bit set to 1, with a text response with the F bit set to 0, the initiator must keep sending the text request (even empty) with the F bit set to 1 until it gets the text response with the F bit set to 1. Responding to a text request with the F bit set to 1 with an empty (no key=value pairs) is not an error but is discouraged.

In a negotiation sequence, the F bit settings in one pair of text request-responses have no bearing on the F bit settings of the next pair.  An initiator having the F bit set to 1 in a request and being answered with an F bit setting of 0 may have the next request issued with the F bit set to 0.  

Whenever parameter action or acceptance is dependent on other parameters, the dependent parameters MUST be sent after the parameters they depend on; if they are sent within the same command a response for a parameter might imply responses for others.

Whenever the target responds with the F bit set to 0 it MUST set the Target Transfer Tag to a value other than the default 0xffffffff.

An initiator MAY reset an operational parameter negotiation by issuing a Text request with the Target Transfer Tag set to the value 0xffffffff after receiving a response with the Target Transfer Tag set to a value other than 0xffffffff.  A target may reset an operational parameter negotiation by answering a Text request with a Reject.

---

From: Santosh Rao <[email-masked]>
Date: Mon, 15 Oct 2001 09:53:46 -0700

Julian,

Please see comments below.

- Santosh

> When the Target Transfer Tag is set to the reserved value 0xffffffff, it tells the target that this is a new request and the target should reset any internal state associated with the Initiator Tas…

I believe the above is in violation with general protocol rules
regarding uniqueness of task tags. initiator Task Tags are required to
be unique for a given initiator and it should be considered a protocol
error when an initiator re-uses an initiator task tag while a prior
instance of the task tag is still active at the target.

Such re-uses of an active task tag by an initiator are REJECTed by the
target with a reason of "protocol error" or "task in progress". (It is
also not clear which of the 2 reason codes is to be used in this case
?).

In general, task tag resources MUST NOT be reused by initiators without
first ensuring that their prior usage had been completed, either through
successful completion of the prior task, or a completed error recovery
of the prior task through an Abort Task or some higher recovery
mechanism such as session teardown.

I suggest that operational parameters be cleared by an initiator by
either issuing an Abort Task for the text command, or tearing down the
connection, or re-negotiating new parameters and the above special-cased
rule be removed.

> T->I Text <part n> (F=1, TTT=0xffffffff)

Regarding the above description of long text exchange handling, is it a
requirement that the initiator should send an EMPTY text command in
response to a long text response ? Can it originate any keys or respond
to any keys in continuing a long text exchange ? (This would occur on
long text exchanges involving X-* keys, and not SendTargets).

> Text response PDU

> If the target receives a Text Request with the Target Task Tag set to the reserved value of 0xffffffff it resets its internal state associated with the given Initiator Task 
Tag.

Please see above comments on this rule. It violates "unique task tag"
protocol rules & requires targets to special case the handling of task
tags.

It is still not clear on when the target would need to use a
"Negotiation Reset" reject reason code. Any reject from the target of a
text command results in a reset of operational parameters and an
explicit reason code such as "Negotiation Reset" is not required.

I suggest the above rule not be allowed, since it special cases the
protocol handling of "non-unique task tags".

-- 
##################################
Santosh Rao
Software Design Engineer,
HP-UX iSCSI Driver Team,
Hewlett Packard, Cupertino.
email : [email-masked]
Phone : 408-447-3751
##################################