Subject: [Ips] TMF text with updates

From: "Mallikarjun C." <[email-masked]>
Date: Wed, 27 Dec 2006 20:41:39 -0800 (PST)

Attached is the latest text that incorporates David's proposed enhancement.  Please review and comment.  Note especially two things: new section 4.1.4 that summarizes generic implementation considerations for both "clarified" and "updated" semantics, the changed text in 4.1.2 that says "MAY wait for ....target transfer tags.....from third-party initiators" from the previous blanket MUST.

Mallikarjun

4.1.2 Clarified multi-task abort semantics
All iSCSI implementations MUST support the protocol behavior defined in this section as the default behavior.  The execution of ABORT TASK SET, CLEAR TASK SET, LOGICAL UNIT RESET, TARGET WARM RESET, and TARGET COLD RESET TMF Requests consists of the following sequence of actions in the specified order on the specified party. 
The initiator iSCSI layer:
a. MUST continue to respond to each TTT received for the affected tasks. 
b. Should receive any responses that the target may provide for some tasks among the affected tasks (may process them as usual because they are guaranteed to have chronologically originated prior to the TMF response). 
c. Should receive the TMF Response concluding all the tasks in the set of affected tasks. 

The target iSCSI layer:
a. MUST wait for currently valid target transfer tags of the affected tasks from the issuing initiator to be responded to.  MAY wait for responses on currently valid target transfer tags of the affected tasks from third-party initiators.
b. MUST wait (concurrent with the wait in Step.a) for all commands of the affected tasks to be received based on the CmdSN ordering.   SHOULD NOT wait for new commands on third-party affected sessions - only the instantiated tasks have to be considered for the purpose of determining the affected tasks.  In the case of target-scoped requests (i.e. TARGET WARM RESET and TARGET COLD RESET), all the commands that are not yet received on the issuing session in the command stream however can be considered to have been received with no command waiting period - i.e. the entire CmdSN space up to the CmdSN of the task management function can be "plugged".
c. MUST propagate the TMF request to and receive the response from the target SCSI layer. 
d. MUST address the Response Fence flag on the TMF Response on issuing session as defined in 3.3.2.
e. MUST address the Response Fence flag on the first post-TMF Response on third-party sessions as defined in 3.3.2.  If some tasks originate from non-iSCSI I_T_L nexuses then the means by which the target ensures that all affected tasks have returned their status to the initiator are defined by the specific non-iSCSI transport protocol(s).
Implementation note: Technically, the TMF servicing is complete in Step.d.  Data transfers corresponding to terminated tasks may however still be in progress on third-party iSCSI sessiosn even at the end of Step.e.  TMF Response MUST NOT be sent by the target iSCSI layer before the end of Step.d, and MAY be sent at the end of Step.d despite these outstanding Data transfers until after Step.e.
 
4.1.3 Updated multi-task abort semantics
Protocol behavior defined in this section MUST be implemented by all iSCSI implementations complying with this document.  Protocol behavior defined in this section MUST be exhibited by iSCSI implementations on an iSCSI session when they negotiate the TaskReporting (section 9.1) key to “FastAbort” on that session.  The execution of ABORT TASK SET, CLEAR TASK SET, LOGICAL UNIT RESET, TARGET WARM RESET, and TARGET COLD RESET TMF Requests consists of the following sequence of actions in the specified order on the specified party. 
The initiator iSCSI layer:
a. MUST NOT send any more Data-Out PDUs for affected tasks on the issuing connection of the issuing iSCSI session once the TMF is sent to the target.
b. Should receive any responses that the target may provide for some tasks among the affected tasks (may process them as usual because they are guaranteed to have chronologically originated prior to the TMF response).
c. MUST respond to Async Message PDU with AsyncEvent=5 as defined in section 8.1.
d. Should receive the TMF Response concluding all the tasks in the set of affected tasks.

The target iSCSI layer:
a. MUST wait for all commands of the affected tasks to be received based on the CmdSN ordering on the issuing session.  SHOULD NOT wait for new commands on third-party affected sessions - only the instantiated tasks have to be considered for the purpose of determining the affected tasks.  In the case of target-scoped requests (i.e. TARGET WARM RESET and TARGET COLD RESET), all the commands that are not yet received on the issuing session in the command stream however can be considered to have been received with no command waiting period - i.e. the entire CmdSN space up to the CmdSN of the task management function can be "plugged".
b. MUST propagate the TMF request to and receive the response from the target SCSI layer. 
c. MUST leave all active "affected TTTs" (i.e. active TTTs associated with affected tasks) valid along with any buffer allocations for the TTTs intact.
d. MUST generate an Asynchronous Message PDU with AsyncEvent=5 (section 8.1) on:
i) each connection of each third-party session that at least one affected task is allegiant to, and
ii) each connection except the issuing connection of the issuing session that has at least one allegiant affected task.
If there are multiple affected LUs (say due to a target reset), then one Async Message PDU MUST be sent for each such LU on each connection that has at least one allegiant affected task.
e. MUST address the Response Fence flag on the TMF Response on issuing session as defined in 3.3.2.
f. MUST address the Response Fence flag on the first post-TMF Response on third-party sessions as defined in 3.3.2. If some tasks originate from non-iSCSI I_T_L nexuses then the means by which the target ensures that all affected tasks have returned their status to the initiator are defined by the specific non-iSCSI transport protocol(s).
g. MUST free up the affected TTTs (and STags, if applicable) and the corresponding buffers once it receives the associated Nop-Out acknowledgement that the initiator generated in response to the Async Message.  
Implementation note: Technically, the TMF servicing is complete in Step.e.  Data transfers corresponding to terminated tasks may however still be in progress even at the end of Step.f.  TMF Response MUST NOT be sent by the target iSCSI layer before the end of Step.e, and MAY be sent at the end of Step.e despite these outstanding Data transfers until Step.g.  Step.g specifies an event to free up any such resources that may have been reserved to support outstanding data transfers.  
4.1.3.1 Clearing effects update
Appendix F.1 of [RFC3720] specifies the clearing effects of target and LU resets on “Incomplete TTTs” as “Y”.  This meant that a target warm reset or a target cold reset or an LU reset would clear the active TTTs upon completion.  The TaskReporting=FastAbort (section 9.1) semantics defined by this section however do not guarantee that the active TTTs are cleared by the end of the reset operations.  In fact, the new semantics are designed to allow clearing the TTTs in a “lazy” fashion after the TMF Response is delivered.  Thus, when TaskReporting=FastAbort is operational on a session, the clearing effects of reset operations on “Incomplete TTTs” is “N”.  
4.1.4 Implementation considerations
Both in clarified semantics (section 4.1.2) and updated semantics (section 4.1.3), there may be outstanding data transfers even after the TMF completion is reported on the issuing session.  In the case of iSCSI/iSER [iSER], these would be tagged data transfers for STags not owned by any active tasks.  Whether or not real buffers support these data transfers is implementation-dependent.  However, the data transfers logically MUST be silently discarded by the target iSCSI layer in all cases.  A target MAY, on an implementation-defined internal timeout, also choose to drop the connections on which it did not receive the expected Data-out sequences (section 4.1.2) or Nop-Out acknowledgements (section 4.1.3) so as to reclaim the associated buffer, STag and TTT resources as appropriate.

----- Original Message ----
From: "[email-masked]" <[email-masked]>
To: [email-masked]; [email-masked]; [email-masked]
Sent: Wednesday, December 20, 2006 2:33:43 PM
Subject: RE: [Ips] Implementer's Guide - Task Management Issue

> Let's try this line of reasoning - the target issues the Nop-Out, now
when
> can it free the resources?  Answer:
>     - The Nop-In response comes back, OR
>     - The connection times out and is torn down.
> Now, what if the Nop-Out is not issued - what does the target wait for
to
> free the resources?  Answer:
>     - The transfers complete, OR
>     - The connection times out and is torn down. 
> Those look similar enough at the target (the worst case is the same -
the
> resources are tied up until an uncooperative initiator times out) that
> I don't see the harm in allowing the early TMF return without the new
> key.  The clear distinction is that the first two bullets are
different;
> if the new key is not negotiated, the target has to wait for the
transfers
> to complete; the new key and the Nop-Out are necessary to walk away
earlier
> when the initiator involved is able to continue the transfers.

That got slightly twisted - what it should have talked about was the
target
issuing the new Async Message and the Nop-Out response coming back.

Thanks,
--David
----------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
[email-masked]        Mobile: +1 (978) 394-7754
----------------------------------------------------

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

_______________________________________________
Ips mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ips

---

From: Julian Satran <[email-masked]>
Date: Thu, 28 Dec 2006 12:31:49 +0200

Mallikarjun,

I would consider the text:

c. MUST leave all active "affected TTTs" (i.e. active TTTs associated with 
affected tasks) valid along with any buffer allocations for the TTTs 
intact.

somewhat excessive as it relates too much to implementation. I would 
rather say:

c. MUST leave all active "affected TTTs" (i.e. active TTTs associated with 
affected tasks) valid.

and leave the buffer issue to the implementer (as I have stated already on 
this list).

I also keep thinking (and mildly  objecting to) that the whole fast abort 
is excesively complex.

You could achive the same effect by issuing the TM command on every 
affected connection.
The third party story is even more puzzling as in order to negotiate any 
of the nem TM modes the taget will have to ascertain that all other 
initiators support it. I fail to understand how would you handle 
downgrading the mode for those that don't. And if you don't downgrade why 
not state that fast-abort and target scoped queues don't go together and 
simplify the mechanics to a multiple issue TM.

Thanks,
Julo

"Mallikarjun C." <[email-masked]> 
28/12/06 06:41

To
[email-masked]
cc

Subject
[Ips] TMF text with updates

Attached is the latest text that incorporates David's proposed 
enhancement.  Please review and comment.  Note especially two things: new 
section 4.1.4 that summarizes generic implementation considerations for 
both "clarified" and "updated" semantics, the changed text in 4.1.2 that 
says "MAY wait for ....target transfer tags.....from third-party 
initiators" from the previous blanket MUST.

Mallikarjun

4.1.2 Clarified multi-task abort semantics
All iSCSI implementations MUST support the protocol behavior defined in 
this section as the default behavior.  The execution of ABORT TASK SET, 
CLEAR TASK SET, LOGICAL UNIT RESET, TARGET WARM RESET, and TARGET COLD 
RESET TMF Requests consists of the following sequence of actions in the 
specified order on the specified party. 
The initiator iSCSI layer:
a. MUST continue to respond to each TTT received for the affected tasks. 
b. Should receive any responses that the target may provide for some tasks 
among the affected tasks (may process them as usual because they are 
guaranteed to have chronologically originated prior to the TMF response). 
c. Should receive the TMF Response concluding all the tasks in the set of 
affected tasks. 

The target iSCSI layer:
a. MUST wait for currently valid target transfer tags of the affected 
tasks from the issuing initiator to be responded to.  MAY wait for 
responses on currently valid target transfer tags of the affected tasks 
from third-party initiators.
b. MUST wait (concurrent with the wait in Step.a) for all commands of the 
affected tasks to be received based on the CmdSN ordering.   SHOULD NOT 
wait for new commands on third-party affected sessions - only the 
instantiated tasks have to be considered for the purpose of determining 
the affected tasks.  In the case of target-scoped requests (i.e. TARGET 
WARM RESET and TARGET COLD RESET), all the commands that are not yet 
received on the issuing session in the command stream however can be 
considered to have been received with no command waiting period - i.e. the 
entire CmdSN space up to the CmdSN of the task management function can be 
"plugged".
c. MUST propagate the TMF request to and receive the response from the 
target SCSI layer. 
d. MUST address the Response Fence flag on the TMF Response on issuing 
session as defined in 3.3.2.
e. MUST address the Response Fence flag on the first post-TMF Response on 
third-party sessions as defined in 3.3.2.  If some tasks originate from 
non-iSCSI I_T_L nexuses then the means by which the target ensures that 
all affected tasks have returned their status to the initiator are defined 
by the specific non-iSCSI transport protocol(s).
Implementation note: Technically, the TMF servicing is complete in Step.d. 
 Data transfers corresponding to terminated tasks may however still be in 
progress on third-party iSCSI sessiosn even at the end of Step.e.  TMF 
Response MUST NOT be sent by the target iSCSI layer before the end of 
Step.d, and MAY be sent at the end of Step.d despite these outstanding 
Data transfers until after Step.e.
 
4.1.3 Updated multi-task abort semantics
Protocol behavior defined in this section MUST be implemented by all iSCSI 
implementations complying with this document.  Protocol behavior defined 
in this section MUST be exhibited by iSCSI implementations on an iSCSI 
session when they negotiate the TaskReporting (section 9.1) key to 
?FastAbort? on that session.  The execution of ABORT TASK SET, CLEAR TASK 
SET, LOGICAL UNIT RESET, TARGET WARM RESET, and TARGET COLD RESET TMF 
Requests consists of the following sequence of actions in the specified 
order on the specified party. 
The initiator iSCSI layer:
a. MUST NOT send any more Data-Out PDUs for affected tasks on the issuing 
connection of the issuing iSCSI session once the TMF is sent to the 
target.
b. Should receive any responses that the target may provide for some tasks 
among the affected tasks (may process them as usual because they are 
guaranteed to have chronologically originated prior to the TMF response).
c. MUST respond to Async Message PDU with AsyncEvent=5 as defined in 
section 8.1.
d. Should receive the TMF Response concluding all the tasks in the set of 
affected tasks.

The target iSCSI layer:
a. MUST wait for all commands of the affected tasks to be received based 
on the CmdSN ordering on the issuing session.  SHOULD NOT wait for new 
commands on third-party affected sessions - only the instantiated tasks 
have to be considered for the purpose of determining the affected tasks. 
In the case of target-scoped requests (i.e. TARGET WARM RESET and TARGET 
COLD RESET), all the commands that are not yet received on the issuing 
session in the command stream however can be considered to have been 
received with no command waiting period - i.e. the entire CmdSN space up 
to the CmdSN of the task management function can be "plugged".
b. MUST propagate the TMF request to and receive the response from the 
target SCSI layer. 
c. MUST leave all active "affected TTTs" (i.e. active TTTs associated with 
affected tasks) valid along with any buffer allocations for the TTTs 
intact.
d. MUST generate an Asynchronous Message PDU with AsyncEvent=5 (section 
8.1) on:
i) each connection of each third-party session that at least one affected 
task is allegiant to, and
ii) each connection except the issuing connection of the issuing session 
that has at least one allegiant affected task.
If there are multiple affected LUs (say due to a target reset), then one 
Async Message PDU MUST be sent for each such LU on each connection that 
has at least one allegiant affected task.
e. MUST address the Response Fence flag on the TMF Response on issuing 
session as defined in 3.3.2.
f. MUST address the Response Fence flag on the first post-TMF Response on 
third-party sessions as defined in 3.3.2. If some tasks originate from 
non-iSCSI I_T_L nexuses then the means by which the target ensures that 
all affected tasks have returned their status to the initiator are defined 
by the specific non-iSCSI transport protocol(s).
g. MUST free up the affected TTTs (and STags, if applicable) and the 
corresponding buffers once it receives the associated Nop-Out 
acknowledgement that the initiator generated in response to the Async 
Message. 
Implementation note: Technically, the TMF servicing is complete in Step.e. 
 Data transfers corresponding to terminated tasks may however still be in 
progress even at the end of Step.f.  TMF Response MUST NOT be sent by the 
target iSCSI layer before the end of Step.e, and MAY be sent at the end of 
Step.e despite these outstanding Data transfers until Step.g.  Step.g 
specifies an event to free up any such resources that may have been 
reserved to support outstanding data transfers. 
4.1.3.1 Clearing effects update
Appendix F.1 of [RFC3720] specifies the clearing effects of target and LU 
resets on ?Incomplete TTTs? as ?Y?.  This meant that a target warm reset 
or a target cold reset or an LU reset would clear the active TTTs upon 
completion.  The TaskReporting=FastAbort (section 9.1) semantics defined 
by this section however do not guarantee that the active TTTs are cleared 
by the end of the reset operations.  In fact, the new semantics are 
designed to allow clearing the TTTs in a ?lazy? fashion after the TMF 
Response is delivered.  Thus, when TaskReporting=FastAbort is operational 
on a session, the clearing effects of reset operations on ?Incomplete 
TTTs? is ?N?. 
4.1.4 Implementation considerations
Both in clarified semantics (section 4.1.2) and updated semantics (section 
4.1.3), there may be outstanding data transfers even after the TMF 
completion is reported on the issuing session.  In the case of iSCSI/iSER 
[iSER], these would be tagged data transfers for STags not owned by any 
active tasks.  Whether or not real buffers support these data transfers is 
implementation-dependent.  However, the data transfers logically MUST be 
silently discarded by the target iSCSI layer in all cases.  A target MAY, 
on an implementation-defined internal timeout, also choose to drop the 
connections on which it did not receive the expected Data-out sequences 
(section 4.1.2) or Nop-Out acknowledgements (section 4.1.3) so as to 
reclaim the associated buffer, STag and TTT resources as appropriate.

----- Original Message ----
From: "[email-masked]" <[email-masked]>
To: [email-masked]; [email-masked]; [email-masked]
Sent: Wednesday, December 20, 2006 2:33:43 PM
Subject: RE: [Ips] Implementer's Guide - Task Management Issue

when
to
the
different;
transfers
earlier

Thanks,
--David

---

From: [email-masked]
Date: Thu, 28 Dec 2006 14:54:27 -0500

I agree with Julian that we should avoid discussing "buffer allocations"
and
the like, even though we know that something like that has to happen in
at
least iSER implementations.  A general discussion of "resources" works.
 
affected connection.
 
Not for TMF's that affect commands from other initiators.  Also, asking
the
target to coordinate receipt of the TM command on every connection in a
multi-connection session is also a bit much.
 
> The third party story is even more puzzling as in order to negotiate
any of
> the new TM modes the taget will have to ascertain that all other
initiators
> support it. I fail to understand how would you handle downgrading the
mode
> for those that don't.
 
I think the target has to track this initiator by initiator, and not
issue the
new async message to old initiators.  This increases the importance of
being
able to complete a TMF in the face of an uncooperative third party
"Legacy"
initiator.
 
> And if you don't downgrade why not state that fast-abort and target
scoped
> queues don't go together and simplify the mechanics to a multiple
issue TM. 

This didn't parse for me - could you explain in more detail?
 
Thanks,
--David

________________________________

	From: Julian Satran [mailto:[email-masked]] 
	Sent: Thursday, December 28, 2006 5:32 AM
	To: Mallikarjun C.
	Cc: [email-masked]
	Subject: Re: [Ips] TMF text with updates

	Mallikarjun, 

	c. MUST leave all active "affected TTTs" (i.e. active TTTs
associated with affected tasks) valid along with any buffer allocations
for the TTTs intact. 
	
	somewhat excessive as it relates too much to implementation. I
would rather say: 
	
	c. MUST leave all active "affected TTTs" (i.e. active TTTs
associated with affected tasks) valid. 
	
	and leave the buffer issue to the implementer (as I have stated
already on this list). 
	
	I also keep thinking (and mildly  objecting to) that the whole
fast abort is excesively complex. 
	
	You could achive the same effect by issuing the TM command on
every affected connection. 
	The third party story is even more puzzling as in order to
negotiate any of the nem TM modes the taget will have to ascertain that
all other initiators support it. I fail to understand how would you
handle downgrading the mode for those that don't. And if you don't
downgrade why not state that fast-abort and target scoped queues don't
go together and simplify the mechanics to a multiple issue TM. 
	
	Thanks, 
	Julo 

28/12/06 06:41 

To
[email-masked] 
cc
Subject

	Attached is the latest text that incorporates David's proposed
enhancement.  Please review and comment.  Note especially two things:
new section 4.1.4 that summarizes generic implementation considerations
for both "clarified" and "updated" semantics, the changed text in 4.1.2
that says "MAY wait for ....target transfer tags.....from third-party
initiators" from the previous blanket MUST.
	
	Mallikarjun

	4.1.2 Clarified multi-task abort semantics
	All iSCSI implementations MUST support the protocol behavior
defined in this section as the default behavior.  The execution of ABORT
TASK SET, CLEAR TASK SET, LOGICAL UNIT RESET, TARGET WARM RESET, and
TARGET COLD RESET TMF Requests consists of the following sequence of
actions in the specified order on the specified party. 
	The initiator iSCSI layer:
	a. MUST continue to respond to each TTT received for the
affected tasks. 
	b. Should receive any responses that the target may provide for
some tasks among the affected tasks (may process them as usual because
they are guaranteed to have chronologically originated prior to the TMF
response). 
	c. Should receive the TMF Response concluding all the tasks in
the set of affected tasks. 
	
	The target iSCSI layer:
	a. MUST wait for currently valid target transfer tags of the
affected tasks from the issuing initiator to be responded to.  MAY wait
for responses on currently valid target transfer tags of the affected
tasks from third-party initiators.
	b. MUST wait (concurrent with the wait in Step.a) for all
commands of the affected tasks to be received based on the CmdSN
ordering.   SHOULD NOT wait for new commands on third-party affected
sessions - only the instantiated tasks have to be considered for the
purpose of determining the affected tasks.  In the case of target-scoped
requests (i.e. TARGET WARM RESET and TARGET COLD RESET), all the
commands that are not yet received on the issuing session in the command
stream however can be considered to have been received with no command
waiting period - i.e. the entire CmdSN space up to the CmdSN of the task
management function can be "plugged".
	c. MUST propagate the TMF request to and receive the response
from the target SCSI layer. 
	d. MUST address the Response Fence flag on the TMF Response on
issuing session as defined in 3.3.2.
	e. MUST address the Response Fence flag on the first post-TMF
Response on third-party sessions as defined in 3.3.2.  If some tasks
originate from non-iSCSI I_T_L nexuses then the means by which the
target ensures that all affected tasks have returned their status to the
initiator are defined by the specific non-iSCSI transport protocol(s).
	Implementation note: Technically, the TMF servicing is complete
in Step.d.  Data transfers corresponding to terminated tasks may however
still be in progress on third-party iSCSI sessiosn even at the end of
Step.e.  TMF Response MUST NOT be sent by the target iSCSI layer before
the end of Step.d, and MAY be sent at the end of Step.d despite these
outstanding Data transfers until after Step.e.
	
	4.1.3 Updated multi-task abort semantics
	Protocol behavior defined in this section MUST be implemented by
all iSCSI implementations complying with this document.  Protocol
behavior defined in this section MUST be exhibited by iSCSI
implementations on an iSCSI session when they negotiate the
TaskReporting (section 9.1) key to "FastAbort" on that session.  The
execution of ABORT TASK SET, CLEAR TASK SET, LOGICAL UNIT RESET, TARGET
WARM RESET, and TARGET COLD RESET TMF Requests consists of the following
sequence of actions in the specified order on the specified party. 
	The initiator iSCSI layer:
	a. MUST NOT send any more Data-Out PDUs for affected tasks on
the issuing connection of the issuing iSCSI session once the TMF is sent
to the target.
	b. Should receive any responses that the target may provide for
some tasks among the affected tasks (may process them as usual because
they are guaranteed to have chronologically originated prior to the TMF
response).
	c. MUST respond to Async Message PDU with AsyncEvent=5 as
defined in section 8.1.
	d. Should receive the TMF Response concluding all the tasks in
the set of affected tasks.
	
	The target iSCSI layer:
	a. MUST wait for all commands of the affected tasks to be
received based on the CmdSN ordering on the issuing session.  SHOULD NOT
wait for new commands on third-party affected sessions - only the
instantiated tasks have to be considered for the purpose of determining
the affected tasks.  In the case of target-scoped requests (i.e. TARGET
WARM RESET and TARGET COLD RESET), all the commands that are not yet
received on the issuing session in the command stream however can be
considered to have been received with no command waiting period - i.e.
the entire CmdSN space up to the CmdSN of the task management function
can be "plugged".
	b. MUST propagate the TMF request to and receive the response
from the target SCSI layer. 
	c. MUST leave all active "affected TTTs" (i.e. active TTTs
associated with affected tasks) valid along with any buffer allocations
for the TTTs intact.
	d. MUST generate an Asynchronous Message PDU with AsyncEvent=5
(section 8.1) on:
	i) each connection of each third-party session that at least one
affected task is allegiant to, and
	ii) each connection except the issuing connection of the issuing
session that has at least one allegiant affected task.
	If there are multiple affected LUs (say due to a target reset),
then one Async Message PDU MUST be sent for each such LU on each
connection that has at least one allegiant affected task.
	e. MUST address the Response Fence flag on the TMF Response on
issuing session as defined in 3.3.2.
	f. MUST address the Response Fence flag on the first post-TMF
Response on third-party sessions as defined in 3.3.2. If some tasks
originate from non-iSCSI I_T_L nexuses then the means by which the
target ensures that all affected tasks have returned their status to the
initiator are defined by the specific non-iSCSI transport protocol(s).
	g. MUST free up the affected TTTs (and STags, if applicable) and
the corresponding buffers once it receives the associated Nop-Out
acknowledgement that the initiator generated in response to the Async
Message.  
	Implementation note: Technically, the TMF servicing is complete
in Step.e.  Data transfers corresponding to terminated tasks may however
still be in progress even at the end of Step.f.  TMF Response MUST NOT
be sent by the target iSCSI layer before the end of Step.e, and MAY be
sent at the end of Step.e despite these outstanding Data transfers until
Step.g.  Step.g specifies an event to free up any such resources that
may have been reserved to support outstanding data transfers.  
	4.1.3.1 Clearing effects update
	Appendix F.1 of [RFC3720] specifies the clearing effects of
target and LU resets on "Incomplete TTTs" as "Y".  This meant that a
target warm reset or a target cold reset or an LU reset would clear the
active TTTs upon completion.  The TaskReporting=FastAbort (section 9.1)
semantics defined by this section however do not guarantee that the
active TTTs are cleared by the end of the reset operations.  In fact,
the new semantics are designed to allow clearing the TTTs in a "lazy"
fashion after the TMF Response is delivered.  Thus, when
TaskReporting=FastAbort is operational on a session, the clearing
effects of reset operations on "Incomplete TTTs" is "N".  
	4.1.4 Implementation considerations
	Both in clarified semantics (section 4.1.2) and updated
semantics (section 4.1.3), there may be outstanding data transfers even
after the TMF completion is reported on the issuing session.  In the
case of iSCSI/iSER [iSER], these would be tagged data transfers for
STags not owned by any active tasks.  Whether or not real buffers
support these data transfers is implementation-dependent.  However, the
data transfers logically MUST be silently discarded by the target iSCSI
layer in all cases.  A target MAY, on an implementation-defined internal
timeout, also choose to drop the connections on which it did not receive
the expected Data-out sequences (section 4.1.2) or Nop-Out
acknowledgements (section 4.1.3) so as to reclaim the associated buffer,
STag and TTT resources as appropriate.

	----- Original Message ----
	From: "[email-masked]" <[email-masked]>
	To: [email-masked]; [email-masked]; [email-masked]
	Sent: Wednesday, December 20, 2006 2:33:43 PM
	Subject: RE: [Ips] Implementer's Guide - Task Management Issue

	when
	to
	the
	different;
	transfers
	earlier
	> when the initiator involved is able to continue the transfers.
	
	That got slightly twisted - what it should have talked about was
the
	target
	issuing the new Async Message and the Nop-Out response coming
back.
	
	Thanks,
	--David

---

From: [email-masked]
Date: Thu, 28 Dec 2006 15:15:58 -0500

This looks ok for a revised draft - I'll take a detailed look at
all the text in the next version of the draft.  Please try to remove
uses of the word "buffer" where a more general discussion of
"resources" is appropriate.

Thanks,
--David 

> -----Original Message-----
> From: Mallikarjun C. [mailto:[email-masked]] 
> Sent: Wednesday, December 27, 2006 11:42 PM
> To: [email-masked]
> Subject: [Ips] TMF text with updates
> 
> Attached is the latest text that incorporates David's 
> proposed enhancement.  Please review and comment.  Note 
> especially two things: new section 4.1.4 that summarizes 
> generic implementation considerations for both "clarified" 
> and "updated" semantics, the changed text in 4.1.2 that says 
> "MAY wait for ....target transfer tags.....from third-party 
> defined in this section as the default behavior.  The 
> execution of ABORT TASK SET, CLEAR TASK SET, LOGICAL UNIT 
> RESET, TARGET WARM RESET, and TARGET COLD RESET TMF Requests 
> consists of the following sequence of actions in the 
> b. Should receive any responses that the target may provide 
> for some tasks among the affected tasks (may process them as 
> usual because they are guaranteed to have chronologically 
> originated prior to the TMF response). 
> affected tasks from the issuing initiator to be responded to. 
>  MAY wait for responses on currently valid target transfer 
> tags of the affected tasks from third-party initiators.
> commands of the affected tasks to be received based on the 
> CmdSN ordering.   SHOULD NOT wait for new commands on 
> third-party affected sessions - only the instantiated tasks 
> have to be considered for the purpose of determining the 
> affected tasks.  In the case of target-scoped requests (i.e. 
> TARGET WARM RESET and TARGET COLD RESET), all the commands 
> that are not yet received on the issuing session in the 
> command stream however can be considered to have been 
> received with no command waiting period - i.e. the entire 
> CmdSN space up to the CmdSN of the task management function 
> can be "plugged".
> Response on third-party sessions as defined in 3.3.2.  If 
> some tasks originate from non-iSCSI I_T_L nexuses then the 
> means by which the target ensures that all affected tasks 
> have returned their status to the initiator are defined by 
> the specific non-iSCSI transport protocol(s).
> Implementation note: Technically, the TMF servicing is 
> complete in Step.d.  Data transfers corresponding to 
> terminated tasks may however still be in progress on 
> third-party iSCSI sessiosn even at the end of Step.e.  TMF 
> Response MUST NOT be sent by the target iSCSI layer before 
> the end of Step.d, and MAY be sent at the end of Step.d 
> despite these outstanding Data transfers until after Step.e.
>  
> Protocol behavior defined in this section MUST be implemented 
> by all iSCSI implementations complying with this document.  
> Protocol behavior defined in this section MUST be exhibited 
> by iSCSI implementations on an iSCSI session when they 
> negotiate the TaskReporting (section 9.1) key to "FastAbort" 
> on that session.  The execution of ABORT TASK SET, CLEAR TASK 
> SET, LOGICAL UNIT RESET, TARGET WARM RESET, and TARGET COLD 
> RESET TMF Requests consists of the following sequence of 
> the issuing connection of the issuing iSCSI session once the 
> TMF is sent to the target.
> b. Should receive any responses that the target may provide 
> for some tasks among the affected tasks (may process them as 
> usual because they are guaranteed to have chronologically 
> originated prior to the TMF response).
> received based on the CmdSN ordering on the issuing session.  
> SHOULD NOT wait for new commands on third-party affected 
> sessions - only the instantiated tasks have to be considered 
> for the purpose of determining the affected tasks.  In the 
> case of target-scoped requests (i.e. TARGET WARM RESET and 
> TARGET COLD RESET), all the commands that are not yet 
> received on the issuing session in the command stream however 
> can be considered to have been received with no command 
> waiting period - i.e. the entire CmdSN space up to the CmdSN 
> of the task management function can be "plugged".
> associated with affected tasks) valid along with any buffer 
> allocations for the TTTs intact.
> ii) each connection except the issuing connection of the 
> issuing session that has at least one allegiant affected task.
> If there are multiple affected LUs (say due to a target 
> reset), then one Async Message PDU MUST be sent for each such 
> LU on each connection that has at least one allegiant affected task.
> Response on third-party sessions as defined in 3.3.2. If some 
> tasks originate from non-iSCSI I_T_L nexuses then the means 
> by which the target ensures that all affected tasks have 
> returned their status to the initiator are defined by the 
> specific non-iSCSI transport protocol(s).
> g. MUST free up the affected TTTs (and STags, if applicable) 
> and the corresponding buffers once it receives the associated 
> Nop-Out acknowledgement that the initiator generated in 
> response to the Async Message.  
> Implementation note: Technically, the TMF servicing is 
> complete in Step.e.  Data transfers corresponding to 
> terminated tasks may however still be in progress even at the 
> end of Step.f.  TMF Response MUST NOT be sent by the target 
> iSCSI layer before the end of Step.e, and MAY be sent at the 
> end of Step.e despite these outstanding Data transfers until 
> Step.g.  Step.g specifies an event to free up any such 
> resources that may have been reserved to support outstanding 
> data transfers.  
> target and LU resets on "Incomplete TTTs" as "Y".  This meant 
> that a target warm reset or a target cold reset or an LU 
> reset would clear the active TTTs upon completion.  The 
> TaskReporting=FastAbort (section 9.1) semantics defined by 
> this section however do not guarantee that the active TTTs 
> are cleared by the end of the reset operations.  In fact, the 
> new semantics are designed to allow clearing the TTTs in a 
> "lazy" fashion after the TMF Response is delivered.  Thus, 
> when TaskReporting=FastAbort is operational on a session, the 
> clearing effects of reset operations on "Incomplete TTTs" is "N".  
> semantics (section 4.1.3), there may be outstanding data 
> transfers even after the TMF completion is reported on the 
> issuing session.  In the case of iSCSI/iSER [iSER], these 
> would be tagged data transfers for STags not owned by any 
> active tasks.  Whether or not real buffers support these data 
> transfers is implementation-dependent.  However, the data 
> transfers logically MUST be silently discarded by the target 
> iSCSI layer in all cases.  A target MAY, on an 
> implementation-defined internal timeout, also choose to drop 
> the connections on which it did not receive the expected 
> Data-out sequences (section 4.1.2) or Nop-Out 
> acknowledgements (section 4.1.3) so as to reclaim the 
> associated buffer, STag and TTT resources as appropriate.
> 
>

---

From: Julian Satran <[email-masked]>
Date: Fri, 29 Dec 2006 13:15:44 +0200

David,

My point was that we can solve the TM issue only for a single initiator. 
So besides puting the burden on SCSI to cleanly finish other and advise 
that fast solutions really work if there are no target scoped queues there 
is little we can do. And if we limit ourselves to a single initiator the 
multiple TM solution is a simpler (and basically does not deviate from 
3270).  If you consider how software stacks are layered I assume that some 
clever implementers have figured that already (and did it).

Regards,
Julo

[email-masked] 
28/12/06 21:54

To
Julian Satran/Haifa/IBM@IBMIL, <[email-masked]>
cc
<[email-masked]>
Subject
RE: [Ips] TMF text with updates

I agree with Julian that we should avoid discussing "buffer allocations" 
and
the like, even though we know that something like that has to happen in at
least iSER implementations.  A general discussion of "resources" works.
 
affected connection.

> The third party story is even more puzzling as in order to negotiate any 
of
initiators
mode
> for those that don't.
 
I think the target has to track this initiator by initiator, and not issue 
the
new async message to old initiators.  This increases the importance of 
being
able to complete a TMF in the face of an uncooperative third party 
"Legacy"
initiator.
 
scoped
> queues don't go together and simplify the mechanics to a multiple issue 
TM. 
 
Thanks,
--David

Mallikarjun, 

Thanks, 
Julo 

To
[email-masked] 
cc

Subject

Mallikarjun

The target iSCSI layer:

The target iSCSI layer:

----- Original Message ----
From: "[email-masked]" <[email-masked]>
To: [email-masked]; [email-masked]; [email-masked]
Sent: Wednesday, December 20, 2006 2:33:43 PM
Subject: RE: [Ips] Implementer's Guide - Task Management Issue

when
to
the
different;
transfers
earlier

Thanks,
--David

---

From: [email-masked]
Date: Fri, 29 Dec 2006 12:45:25 -0500

Julian,
 
I'm not sure I completely understand what you wrote, but if you're
suggesting that for a TMF that affects tasks from multiple initiators:
- Fast abort (early termination of data transfers) is used for the
sending
    initiator.
- The existing 3270 abort mechanism is used with other initiators, with
    the update that the TMF response does not have to wait for data
transfer
    completion.
I think that suggestion is reasonable, and it actually helps with the
use
case that started this (3rd party initiator is dead).  The draft text
would
need to change to allow this (right now it mandates fast abort in all
cases
if the key is negotiated to support it)
 
I'm not sure how target-scoped queues enter into this.
 
Thanks,
--David

	From: Julian Satran [mailto:[email-masked]] 
	Sent: Friday, December 29, 2006 6:16 AM
	To: Black, David
	Cc: [email-masked]; [email-masked]
	Subject: RE: [Ips] TMF text with updates

	David, 
	
	My point was that we can solve the TM issue only for a single
initiator. So besides puting the burden on SCSI to cleanly finish other
and advise that fast solutions really work if there are no target scoped
queues there is little we can do. And if we limit ourselves to a single
initiator the multiple TM solution is a simpler (and basically does not
deviate from 3270).  If you consider how software stacks are layered I
assume that some clever implementers have figured that already (and did
it). 
	
	Regards, 
	Julo 

[email-masked] 

28/12/06 21:54 

To

	I agree with Julian that we should avoid discussing "buffer
allocations" and 
	the like, even though we know that something like that has to
happen in at 
	least iSER implementations.  A general discussion of "resources"
works. 

	Not for TMF's that affect commands from other initiators.  Also,
asking the 
	target to coordinate receipt of the TM command on every
connection in a 
	multi-connection session is also a bit much. 
	  
	> The third party story is even more puzzling as in order to
negotiate any of 
	> the new TM modes the taget will have to ascertain that all
other initiators 
	> support it. I fail to understand how would you handle
downgrading the mode 
	> for those that don't. 
	  
	I think the target has to track this initiator by initiator, and
not issue the 
	new async message to old initiators.  This increases the
importance of being 
	able to complete a TMF in the face of an uncooperative third
party "Legacy" 
	initiator. 
	  
	> And if you don't downgrade why not state that fast-abort and
target scoped 
	> queues don't go together and simplify the mechanics to a
multiple issue TM. 
	  
	Thanks, 
	--David 

	

	Mallikarjun, 

	

	

	Thanks, 
	Julo 

28/12/06 06:41 

To
[email-masked] 
cc
Subject

	

	Mallikarjun

	
	The target iSCSI layer:

	The target iSCSI layer:

	----- Original Message ----
	From: "[email-masked]" <[email-masked]>
	To: [email-masked]; [email-masked]; [email-masked]
	Sent: Wednesday, December 20, 2006 2:33:43 PM
	Subject: RE: [Ips] Implementer's Guide - Task Management Issue

	when
	to
	the
	different;
	transfers
	earlier

	Thanks,
	--David