Subject: iSCSI: update on OOO CmdSNs/connection

From: "Mallikarjun C." <[email-masked]>
Date: Fri, 09 Nov 2001 13:49:19 -0800

To those interested in this discussion:

Julian and I had a phone conversation on the topic
of OOO CmdSNs on a connection.  An update follows.

Julian agrees that there are valid error recovery
scenarios where CmdSNs will legitimately end up OOO
on a given connection.

OTOH, I agree with two of Julian's scenarios that he 
pointed out right away - the "cleaning command" (command
required to be sent after a retry copy to ensure flushing 
within 2^31 -1), and an immediate Logout posted with 
unacknowledged commands.  Neither of this can be shipped 
OOO - since the former undoes the flushing intent, and 
the latter breaks the rule that nothing more follows a 
Logout on the connection (and troublesome in other ways,
see below).  

In general, I share the concern with Julian that we 
have not closely scrutinized all possibilities.

With that said, something along the following lines
seemed reasonable -

A)Initiator MUST send commands in increasing order of 
  CmdSN on a connection if both the following are true -
	- operational ErrorRecoveryLevel is 0,
	- MaxConnections is negotiated to 1.
B)In all the other cases, initiator SHOULD send commands
  in increasing order of CmdSN on a connection.  It is 
  strongly encouraged that commands with out-of-order
  CmdSNs be sent on a connection only if they are 
  retransmitted commands due to digest error recovery 
  and connection recovery.

I also suggest the following upon further reflection-

C)Add wording in section 2.2.2.1 to mandate that
  the cleaning command MUST be sent in-order after 
  the retried command.
D)Warn clearly that sending an immediate Logout command 
  in the presence of other unacknowledged commands MAY 
  create inadvertent discarding of certain commands (even
  if it is a recovery Logout), and MAY cause protocol 
  errors leading to ungraceful shutdown of the connection.

Hopefully A will bring the determinism that Somesh was 
looking for certain design points.  B describes the more 
general n-connection session case.  C & D are fixes for 
two identfied areas (so far) which will break. 
 
Comments?
-- 
Mallikarjun 

Mallikarjun Chadalapaka
Networked Storage Architecture
Network Storage Solutions Organization
MS 5668	Hewlett-Packard, Roseville.
[email-masked]

---

From: "Somesh Gupta" <[email-masked]>
Date: Fri, 9 Nov 2001 15:26:18 -0800

Mallikarjun,

I never liked the SHOULD. It is not a design point.
If we really want to allow it, perhaps a negotiation
parameter is a better choice (which is only
marginally better). Error detection and recovery
have completely different design requirements than
the data path.

So ideally we say MUST or nothing
or we negotiate it

Also on your point A, it "MUST" be a MUST on
a single connection case except for when required
for error recovery (i.e. the very very rare -
case of digest error).

Somesh

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]On Behalf Of
> Mallikarjun C.
> Sent: Friday, November 09, 2001 1:49 PM
> To: ips
> Subject: iSCSI: update on OOO CmdSNs/connection
> 
>

---

From: "Mallikarjun C." <[email-masked]>
Date: Fri, 09 Nov 2001 17:50:07 -0800

Somesh,

I am in general saying the same thing as you describe,
we are perhaps debating on how to phrase the sentiment.

>Error detection and recovery

Completely agreed, as I have been saying all along.  But 
we're faced with the task of crafting wording that works 
for both - my wording was B.  Please suggest alternate 
wording that you propose.

> So ideally we say MUST or nothing

In what way saying nothing would help the performance/
interoperability issues on hand?

> for error recovery (i.e. the very very rare -

Please see the two conditions that distinguish A from B.

Regards.
-- 
Mallikarjun 

Mallikarjun Chadalapaka

---

From: "Somesh Gupta" <[email-masked]>
Date: Tue, 13 Nov 2001 12:53:11 -0800

Julian,

I am surprised to see the text in the latest rev
of the doc change as suggested by Mallikarjun.
I did not think there was a consensus on this
subject.

Just because I did not respond to Mallikarjun's
last comment publicly should not be construed
as agreement. Having the last word is hopefully
not assumed to be consensus, otherwise a thread
may never end.

Somesh

> Somesh Gupta
> Sent: Friday, November 09, 2001 3:26 PM
> To: ips
> Subject: RE: iSCSI: update on OOO CmdSNs/connection
> 
> 
> Mallikarjun,
>