Subject: [Ips] iSCSI: Question about iscsi-impl-guide-00.txt, section 4 (TMF)

From: "Pittman, Joseph" <[email-masked]>
Date: Wed, 3 Aug 2005 12:31:09 -0400

This is a question about section 4 of the iSCSI implementer's
guide, draft-ietf-ips-iscsi-impl-guide-00.txt.
 
In particular, steps e) and f) in the target's sequence of actions
states the following:
 
  The Target:
 
     e) Takes note of last-sent StatSN on each of the connections
       in the iSCSI session(s) (one or more) sharing the affected
       tasks, and waits for acknowledgement of each StatSN (may
       solicit for acknowledgement by way of a NOP-In). If some
       tasks originate from non-iSCSI I_T_L nexuses then the means
       by which the target insures that all affected tasks have
       returned their status to the initiator are defined by the
       specific non-iSCSI transport protocol(s).
 
     f) Sends the task set management response to the issuing
        initiator.

That is:
- The target must wait until it has positive acknowledgement that all
  responses sent for all POTENTIALLY AFFECTED commands were received by
  their respective initiators
- THEN the target may send the TMF response.

For the purposes of this mail message, I would like to separate the
requirement imposed by steps e) and f), into two separate
sub-requirements:
 
Sub-requirement 1, pertaining to:
  Status/Responses for iSCSI requests (including SCSI tasks) being
  conducted on the SAME iSCSI session as the TMF itself.
 
Sub-requirement 2, pertaining to:
  Status/Responses for SCSI tasks being conducted on OTHER SCSI
  I_T_L nexuses

I understand Sub-requirement 1 above.  Section 4.1.3 of the
implementer's
guide, and especially bullet item 2 ('single ordered response flow')
clearly explains why the target steps e) and f) are necessary.
The iSCSI multi-connection session feature introduces the possibility
of out of order arrival of responses at the initiator; it is the
responsibility of the iSCSI initiator to reconstruct the in-order
response stream, to satisfy the expectations of the SCSI layer.

But I cannot understand why the iSCSI specification imposes
Sub-requirement 2.
 
- The SCSI-2 and SCSI-3 specifications do not impose ordering
  constraints between two separate SCSI nexuses
 
- iSCSI is merely a transport protocol for SCSI, so it seems
  out of place for the iSCSI spec to be imposing additional
  requirements on SCSI or other transports
 
- It may not be possible to satisfy this sub-requirement with
  certain other transports
 
- There are many examples of configurations with multi-pathed access
  from a single host system to SCSI targets using non-iSCSI transports,
  without the "benefit" of sub-requirement 2.

And the current version of the implementer's guide does not appear
to shed light on the matter.
 
Can anyone think of a justification for the inclusion of
Sub-requirement 2 in the iSCSI specification?  If so, should this
topic be discussed in the implementer's guide?
 
Thanks!
--
Joe Pittman
[email-masked] <mailto:[email-masked]> 

_______________________________________________
Ips mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ips

---

From: Julian Satran <[email-masked]>
Date: Wed, 3 Aug 2005 19:25:51 +0200

SCSI requires that no packets pertained to the aborted tasks be received 
after the response to the TMF (even if they are multi-NEXI).
However for the case of different NEXI you may have a point in that 
different NEXI may not even all be iSCSI (SAM does not required this type 
of homogeneity).
For other NEXI the requirement can be weakened with a statement of "if 
this function is available in the transport of the other NEXI".

Julo

"Pittman, Joseph" <[email-masked]> 
Sent by: [email-masked]
03-08-05 18:31

To
<[email-masked]>
cc

Subject
[Ips]   iSCSI: Question about iscsi-impl-guide-00.txt, section 4 (TMF)

 
  The Target:

That is:

 

 
I understand Sub-requirement 1 above.  Section 4.1.3 of the implementer's

 

 
 
Thanks!
--
Joe Pittman
[email-masked]
Ips mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ips

---

From: "Mallikarjun C." <[email-masked]>
Date: Wed, 03 Aug 2005 17:35:55 -0700

Joe,

 >   their respective initiators

Only for iSCSI-based I_T_L nexuses.

 > requirement imposed by steps e) and f), into two separate

There is also the intermediate "requirement":

Sub-requirement 1.5:
    Status/Responses for SCSI tasks being conducted on OTHER iSCSI
    I_T_L nexuses

But the caveat is that the "requirement" levels vary for iSCSI & 
non-iSCSI transports.

In implementation terms, the RFC 3720 text (also in Implementer's guide):
        "If some
        tasks originate from non-iSCSI I_T_L nexuses then the means
        by which the target insures that all affected tasks have
        returned their status to the initiator are defined by the
        specific non-iSCSI transport protocol(s)."

is, IMO, fairly benign, meant to address the obvious question of "what 
do we do with non-iSCSI nexuses?"

It is only saying that any task statuses have to be returned to 
respective initiators before the issuing initiator is sent the TMF 
Response - it isn't suggesting waiting for status acknowledgements 
across the board for all transports.  The "MUST wait for StatSN ack" 
rule obviously applies only to I_T nexuses of iSCSI transport type.  In 
addition, "the means" of "returning the status" are left to other 
transport protocol implementations as well - it may be a simple 
inter-layer enqueue operation in some implementations.

 >   constraints between two separate SCSI nexuses

Note that iSCSI only specifies this ordering constraint across iSCSI 
sessions for "iSCSI reasons".  It is not a SCSI nexus constraint in that 
SCSI layer is privy to this iSCSI behavior.

 > - It may not be possible to satisfy this sub-requirement with

As commented above, RFC 3720 text is neither imposing requirements on 
other transports nor the SCSI layer.

 > to shed light on the matter.

You're right, not completely.  The "iSCSI reasons" for this behavior are 
tied to the "single ordered response flow" principle.  There are two 
aspects of single ordered response flow -

1) The TMF response must not pass the affected task responses  (on the 
TMF-issuing session).  I think the current draft text makes this case.

2) The UA notification (set due to Clear Task Set) must not pass the 
task responses (on all the iSCSI sessions accessing the LU).  More on 
this below.

SAM requires Clear Task Set to trigger a UA on all I_T_L nexuses.  Once 
an initiator is notified of such an UA, the application client on the 
receiving initiator is required to clear its task state (clause 5.5) for 
each task in the task set.  It would thus be inappropriate to deliver a 
SCSI Response for a task after the task state is cleared on the 
initiator, i.e. after the UA is notified.

What #1 & #2 above imply is that the current text is not quite complete 
in that it says nothing about how iSCSI should deal with SCSI task 
reponses being handed down to it *after* the TMF response (the first 
SCSI task response carries the UA notification).  We should have text 
that mandates that SCSI reponses must be held back at the iSCSI layer 
until the affected task response acks are received.  IOW, the TMF 
reponse and the "unaffected" task responses (the first of which 
presumably carrying the UA) are all "released" once the affected StatSN 
acks are received.

Once we agree on this, I can propose specific text under a separate 
cover to address the issue.

Thanks!

Mallikarjun

Mallikarjun Chadalapaka
Networked Storage Architecture
StorageWorks Division
Hewlett-Packard MS 5668
Roseville CA 95747
cbm [at] rose.hp.com

Pittman, Joseph wrote:
> ------------------------------------------------------------------------
>