Subject: Re: Fw: [Ips] Response Fence Flag

From: "Mallikarjun C." <[email-masked]>
Date: Sat, 6 Jan 2007 14:54:49 -0800 (PST)

Julian,

I am surprised that you have not referred to the SCSI task attributes in the command ordering discussion.  In any case, in the interest of staying on the response fence topic, I will not respond to the command ordering points you made (debate topics for some other time, :-)).

The whole notion of response fence though is all about ordering responses, where such ordering is needed (only a very small % of responses, which are listed in section 3.3.3 of IG draft as we know them today).  The updated draft, which I will forward for publication today, lists one additional case - that of Persistent Reserve/Abort Tasks.

I am not sure of your intended context of how SCSI-level rigorous exception handling relates to response fence.

And yes, response fence is not for cross-session collaboration, it is in fact limited to a single I_T_L nexus (please refer to section 3.3 of implementer's guide).

Regarding why iSCSI should define it, because it is required for succinctly describing iSCSI protocol behavior in multi-channel I_T nexus situations (iSCSI and SAS, as we know them today).  As mentioned to Eddy yesterday, it (or a concept like it with any other name - this concept is in section 10.6.2 of RFC 3720 as well even though we didn't call it such) is required for iSCSI's proper behavior.

Thanks.

Mallikarjun 

----- Original Message ----
From: Julian Satran <[email-masked]>
To: Mallikarjun C. <[email-masked]>
Cc: IPS <[email-masked]>; [email-masked]
Sent: Saturday, January 6, 2007 6:22:52 AM
Subject: Re: Fw: [Ips] Response Fence Flag

Mallikarjun & All,

As I pointed out in a private discussion with you I never considered in 
dept the response-fence (beyond what is needed for TM).
It is a serious issue for any "transaction type" operations to storage - 
but the last interface that had all the pieces correctly aligned was the 
360 channel.
SCSI had taken (traditionally) the positions that sequencing and 
barriers are rarely needed and when needed applications should provide them.
And that is exactly what all of the relevant applications do (all 
transaction monitors, databases etc.).

The way all the application handle cases when the start of some SCSI 
command has to follow the end of a previous set of operations is to just 
wait for the end of the operations before issuing the "dependent commands".

Is this solution optimal? As you probably observed already this solution 
implies that all the queues in the "stack" (at initiator, transport and 
target) get empty before the dependent commands can be issued and that 
is bad for performance.

Can fencing (bad name BTW because it is in widespread use for defect 
isolation) improve this? Yes (by keeping the pipes full!) provided it is 
associated with rigorous exception handling and the later is a SCSI 
issue (exceptions will require emptying the queues). With most 
applications being distributed I doubt also that fencing for a single 
I_T nexus is interesting and coordinating fencing and exception handling 
over several I_Ts is a complex exercise.

And as transport alone brings no value to the whole issue - why  should 
we go through the exercise of defining a solution before SCSI has 
defined one and we are confident that it is good enough and has value? 
Again I agree that we a mechanism for TM but a good enough mechanism for 
TM is already in and it gets now even better even without resorting to a 
generalized fencing/barrier support.

Regards,
Julo

Mallikarjun C. wrote:
> Trying again after bouncing (>50KB), tail is now clipped....
>
> ----- Forwarded Message ----
> From: Mallikarjun C. <[email-masked]>
> To: [email-masked]
> Sent: Friday, January 5, 2007 2:40:56 PM
> Subject: Re: [Ips] Response Fence Flag
>
> Hi John,
>  
> As you correctly point out, there are two distinct but related 
> topics here.
>  
> (1) proper response ordering across participating connections in a 
> multi-connection session, for a handful of task/TMF responses
> (2) proper way to terminate and signal tasks when actions on one 
> session can impact multiple initiators
>  
> (1) is all about response fence, it is covered separately in section 
> 3.3 of IG draft.  That's what this email thread started off with.
>  
> (2) covers cases that IG had called as as "multi-task abort" cases 
> that typically have a shared task set across sessions, or are the 
> result of a Logical Unit Reset TMF.  Section 4.1 of IG adresses (2).
>  
> So how are (1) and (2) related?
>  
> - Some cases covered under (2) overlap with (1), but some cases in (2) 
> are for single connection sessions.  E.g. LU Reset on a single 
> connection session impacting several 3rd party sessions
>  
> - Some cases under (1) overlap with (2), but some cases in (1) have 
> nothing to do with other sessions.  E.g. establishment of ACA on a 
> non-shared task set
>  
> - Section 4.1 (i.e. text for (2)) uses response fence (i.e. text for 
> (1)) whenever multi-connection sessions are involved in multi-task 
> abort situations.
>  
> I hope that clarifies it.  Feel free to ask if that is not clear.  The 
> net is that the response fence is the underpinning notion to describe 
> the correct iSCSI behavior in a few cases and some of those cases are 
> about task terminations across third-party sessions.
>  
> Mallikarjun
>  
>
>
>  
> ----- Original Message ----
> From: John Hufferd <[email-masked]>
> To: Mallikarjun C. <[email-masked]>; [email-masked]
> Sent: Friday, January 5, 2007 1:01:06 PM
> Subject: RE: [Ips] Response Fence Flag
>
> Mallikarjun,
>
> I must admit, that I have been a bit confused with the direction of 
> the conversation here.
>
>  
>
> Therefore, I went back and reviewed the charts from Dallas .  And as I 
> remembered, the initial focus was to address the issue of Multiple 
> Connections per Session (MC/S) (as stated on chart 4 – “Non-issue for 
> single-connection iSCSI sessions”).  So I think I have missed the step 
> where we have morphed the discussion into one dealing with multiple 
> sessions.  (I am not sure how that happened, or if I miss-read the 
> charts from Dallas , or have not followed the discussion adequately.)
>
>  
>
> If we are attempting to define two different issues, one with MC/S and 
> one with Multiple Session from different Initiators, I think it would 
> be useful to break down the conversation into Topic A – MC/S and Topic 
> B Multiple Sessions.  It is possible that one solution will addresses 
> both, but I for one think I am hearing arguments that might be 
> appropriate for Topic B, while I am thinking about its applicability 
> to Topic A.
>
>  
>
> Perhaps, you could address the issue as either being all about MC/S or 
> explicitly state that it is intended to affect Multiple Sessions also, 
> and then address the issues and solution for each separately.  For 
> example, I believe Robert was addressing the issue from a view of 
> Multiple Sessions and if we only intended to address MC/S then I 
> expect the response might be somewhat different.
>
>  
>
> Anyway, if you could clear-up some of this, I think it would be useful 
> (at least to me).
>
>  
>
> .
>
> .
>
> .
>
> John L Hufferd
>
> Sr. Executive Director of Technology
>
> Brocade Communications Systems, Inc
>
> [email-masked] <mailto:[email-masked]>
>
> Office Phone: (408) 333-5244; eFAX: (408) 904-4688
>
> Alt Office Phone: (408) 997-6136; Cell: (408) 627-9606
>
>  
>
> ------------------------------------------------------------------------
>
> *From:* Mallikarjun C. [mailto:[email-masked]]
> *Sent:* Friday, January 05, 2007 10:08 AM
> *To:* [email-masked]
> *Subject:* Re: [Ips] Response Fence Flag
>
>  
>
> Not really.  Current draft text is intentionally written to not have 
> any dependencies on T10 dynamics.  The point is that iSCSI needs such 
> a notion for succinctly describing the proper iSCSI protocol actions 
> in a few places - ACA, TMF, Persistent reserve/Abort to name a few.  
> We certainly hope it will be approved by T10 and be a part of SAM-4 
> soon, but that isn't required per se for describing what iSCSI needs 
> for its correct behavior.
>
>  
>
> IPS WG has adopted what it needs in the past - staying ahead of T10 
> review/approval cycle if necessary.  I_T nexus loss notification, 
> iSCSI target/port naming, clearing effects are a few I recall.
>
>  
>
> Mallikarjun
>
>
>
>  
>
> ----- Original Message ----
> From: Eddy Quicksall <[email-masked]>
> To: Robert Snively <[email-masked]>; "Elliott, Robert (Server 
> Storage)" <[email-masked]>; [email-masked]
> Sent: Friday, January 5, 2007 8:58:47 AM
> Subject: Re: [Ips] Response Fence Flag
>
> From an earlier email I think that Response Fence is only a proposal 
> in T10 (http://www.t10.org:80/doc06.htm 
> <http://www.t10.org/doc06.htm>). If so shouldn't iSCSI wait a bit 
> until this has been ratified?
>
>  
>
> Eddy
>
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam? Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
> ------------------------------------------------------------------------
>
> _______________________________________________
> Ips mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/ips
>   

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

_______________________________________________
Ips mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ips

---

From: Julian Satran <[email-masked]>
Date: Sun, 7 Jan 2007 09:46:26 -0500

Mallikarjun,

I did reffer to the SCSI task atributes and why the ordering attribute is 
ill conceived as an ordering mechanism. I am repeating here the arguments:

based on receiver time ordering (not controlled by application)  makes 
them unsuitable for applications 
they are done either on every command or none while applications 
(transactions) just need a "barrier" between groups

Reponse ordering servs just this as the reponse with the barrier has to be 
(hold) the last of a group and allows the next group to start.
The ordering of responses is a way for an application to achieve a partial 
ordering in execution.

This is the way I read the draft.

In order to be valuable exceptions have to result in dropping and 
reissuing all queued commands including the barrier and after.

As for multi I_T_L nexuses (or even multi I_T) for the barrier to be 
used/relevant it has to accomodate multiple units (nexi) as interesting 
applications hold multiple nexi.
This accomodation is especially difficult in the face on of failures. If 
this is what scsi had intended it for it looks to me that it is not 
finished yet (SCSI will need a synchronized barrier accomodating several 
nexi) and makes whatever we do of questionable value.

I am still OK with how it is written (as I I said before) - my change in 
mind is only in "if ought to be doing it" (as I doubt its value without 
additions) or we have to wait for SCSI to finish its work.

For TM we already did what we had to do (without the barrier) and doing 
the barrier is IMHO premature (although it answers nicely the single nexi 
issue).

Regards,
Julo

"Mallikarjun C." <[email-masked]> 
06/01/07 17:54

To
IPS <[email-masked]>
cc

Subject
Re: Fw: [Ips] Response Fence Flag

Julian,

I am surprised that you have not referred to the SCSI task attributes in 
the command ordering discussion.  In any case, in the interest of staying 
on the response fence topic, I will not respond to the command ordering 
points you made (debate topics for some other time, :-)).

The whole notion of response fence though is all about ordering responses, 
where such ordering is needed (only a very small % of responses, which are 
listed in section 3.3.3 of IG draft as we know them today).  The updated 
draft, which I will forward for publication today, lists one additional 
case - that of Persistent Reserve/Abort Tasks.

I am not sure of your intended context of how SCSI-level rigorous 
exception handling relates to response fence.

And yes, response fence is not for cross-session collaboration, it is in 
fact limited to a single I_T_L nexus (please refer to section 3.3 of 
implementer's guide).

Regarding why iSCSI should define it, because it is required for 
succinctly describing iSCSI protocol behavior in multi-channel I_T nexus 
situations (iSCSI and SAS, as we know them today).  As mentioned to Eddy 
yesterday, it (or a concept like it with any other name - this concept is 
in section 10.6.2 of RFC 3720 as well even though we didn't call it such) 
is required for iSCSI's proper behavior.

Thanks.

Mallikarjun 

----- Original Message ----
From: Julian Satran <[email-masked]>
To: Mallikarjun C. <[email-masked]>
Cc: IPS <[email-masked]>; [email-masked]
Sent: Saturday, January 6, 2007 6:22:52 AM
Subject: Re: Fw: [Ips] Response Fence Flag

Mallikarjun & All,

barriers are rarely needed and when needed applications should provide 
them.

The way all the application handle cases when the start of some SCSI 
command has to follow the end of a previous set of operations is to just 
wait for the end of the operations before issuing the "dependent 
commands".

Regards,
Julo

Mallikarjun C. wrote:
> Connections per Session (MC/S) (as stated on chart 4 ? ?Non-issue for 
> single-connection iSCSI sessions?).  So I think I have missed the step 
> be useful to break down the conversation into Topic A ? MC/S and Topic

---

From: "Eddy Quicksall" <[email-masked]>
Date: Sun, 7 Jan 2007 10:39:48 -0500

Maybe this was already mentioned but wouldn't the Response Fence be necessary for even commands with ordered tags?

If an ordered command were sent on one connection then an ordered command were sent on another connection, the response for the 2nd ordered command may get back to the initiator before the response for the first ordered command. But that is not what would be expected by the initiator. A case for this could also be made regarding one or more simples followed by an ordered.

Eddy
  ----- Original Message ----- 
  From: Julian Satran 
  To: Mallikarjun C. 
  Cc: IPS 
  Sent: Sunday, January 07, 2007 9:46 AM
  Subject: Re: Fw: [Ips] Response Fence Flag

  Mallikarjun, 

  I did reffer to the SCSI task atributes and why the ordering attribute is ill conceived as an ordering mechanism. I am repeating here the arguments: 

    a.. based on receiver time ordering (not controlled by application)  makes them unsuitable for applications 
    b.. they are done either on every command or none while applications (transactions) just need a "barrier" between groups

  Reponse ordering servs just this as the reponse with the barrier has to be (hold) the last of a group and allows the next group to start. 
  The ordering of responses is a way for an application to achieve a partial ordering in execution. 

  In order to be valuable exceptions have to result in dropping and reissuing all queued commands including the barrier and after. 

  As for multi I_T_L nexuses (or even multi I_T) for the barrier to be used/relevant it has to accomodate multiple units (nexi) as interesting applications hold multiple nexi. 
  This accomodation is especially difficult in the face on of failures. If this is what scsi had intended it for it looks to me that it is not finished yet (SCSI will need a synchronized barrier accomodating several nexi) and makes whatever we do of questionable value. 

  I am still OK with how it is written (as I I said before) - my change in mind is only in "if ought to be doing it" (as I doubt its value without additions) or we have to wait for SCSI to finish its work. 

  For TM we already did what we had to do (without the barrier) and doing the barrier is IMHO premature (although it answers nicely the single nexi issue). 

  Regards, 
  Julo 

        "Mallikarjun C." <[email-masked]> 
        06/01/07 17:54 
       To IPS <[email-masked]>  
              cc  
              Subject Re: Fw: [Ips] Response Fence Flag 

  Julian,

  Thanks.

  Mallikarjun 

  ----- Original Message ----
  From: Julian Satran <[email-masked]>
  To: Mallikarjun C. <[email-masked]>
  Cc: IPS <[email-masked]>; [email-masked]
  Sent: Saturday, January 6, 2007 6:22:52 AM
  Subject: Re: Fw: [Ips] Response Fence Flag

  Mallikarjun & All,

  Regards,
  Julo

  Mallikarjun C. wrote:
  > Connections per Session (MC/S) (as stated on chart 4 - "Non-issue for 
  > single-connection iSCSI sessions").  So I think I have missed the step 
  > be useful to break down the conversation into Topic A - MC/S and Topic 

------------------------------------------------------------------------------

---

From: Julian Satran <[email-masked]>
Date: Sun, 7 Jan 2007 11:04:08 -0500

Odered commands guarantee a complete ordered excution at the target (in 
the order the commands are received by the target). Unordered responses 
can be  expected (as the standards are completely mute about them). The 
trouble is that the appliaction has no control about the order the 
commands are received at the target.

Julo

"Eddy Quicksall" <[email-masked]> 
07/01/07 10:39

To
"Mallikarjun C." <[email-masked]>, Julian Satran/Haifa/IBM@IBMIL
cc
"IPS" <[email-masked]>
Subject
Re: Fw: [Ips] Response Fence Flag

Maybe this was already mentioned but wouldn't the Response Fence be 
necessary for even commands with ordered tags?
 
If an ordered command were sent on one connection then an ordered command 
were sent on another connection, the response for the 2nd ordered command 
may get back to the initiator before the response for the first ordered 
command. But that is not what would be expected by the initiator. A case 
for this could also be made regarding one or more simples followed by an 
ordered.
 
Eddy
----- Original Message ----- 
From: Julian Satran 
To: Mallikarjun C. 
Cc: IPS 
Sent: Sunday, January 07, 2007 9:46 AM
Subject: Re: Fw: [Ips] Response Fence Flag

Mallikarjun, 

Regards, 
Julo 

To
IPS <[email-masked]> 
cc

Subject

Julian,

Thanks.

Mallikarjun 

----- Original Message ----
From: Julian Satran <[email-masked]>
To: Mallikarjun C. <[email-masked]>
Cc: IPS <[email-masked]>; [email-masked]
Sent: Saturday, January 6, 2007 6:22:52 AM
Subject: Re: Fw: [Ips] Response Fence Flag

Mallikarjun & All,

Regards,
Julo

---

From: "Eddy Quicksall" <[email-masked]>
Date: Sun, 7 Jan 2007 12:07:42 -0500

I would think ordered commands would correctly for a single instance of a single application.

Is your example that there may be multiple instances of an application (or multiple applications), each issuing ordered commands to the same I_T Nexus and that a single instance/application would not be able to guarantee order with respect to other instances/applications?

Eddy
  ----- Original Message ----- 
  From: Julian Satran 
  To: Eddy Quicksall 
  Cc: Mallikarjun C. ; IPS 
  Sent: Sunday, January 07, 2007 11:04 AM
  Subject: Re: Fw: [Ips] Response Fence Flag

  Odered commands guarantee a complete ordered excution at the target (in the order the commands are received by the target). Unordered responses can be  expected (as the standards are completely mute about them). The trouble is that the appliaction has no control about the order the commands are received at the target. 

  Julo 

        "Eddy Quicksall" <[email-masked]> 
        07/01/07 10:39 
       To "Mallikarjun C." <[email-masked]>, Julian Satran/Haifa/IBM@IBMIL  
              cc "IPS" <[email-masked]>  
              Subject Re: Fw: [Ips] Response Fence Flag 

    
    
  Eddy 
  ----- Original Message ----- 
  From: Julian Satran 
  To: Mallikarjun C. 
  Cc: IPS 
  Sent: Sunday, January 07, 2007 9:46 AM 
  Subject: Re: Fw: [Ips] Response Fence Flag 

  Mallikarjun, 

  Regards, 
  Julo 

              To IPS <[email-masked]>  
              cc  

  Julian,

  Thanks.

  Mallikarjun 

  ----- Original Message ----
  From: Julian Satran <[email-masked]>
  To: Mallikarjun C. <[email-masked]>
  Cc: IPS <[email-masked]>; [email-masked]
  Sent: Saturday, January 6, 2007 6:22:52 AM
  Subject: Re: Fw: [Ips] Response Fence Flag

  Mallikarjun & All,

  Regards,
  Julo