Subject: iSCSI connection recovery

From: [email-masked] (Sandeep Joshi)
Date: Fri, 27 Jul 2001 23:37:33 -0400 (EDT)

I had a "big-picture" question about the connection
recovery model.

The current draft suggests that, once the broken connection 
is logged out, the commands allegiant to the old connection 
can now be retried over *any* connection. (See Section 7.11.3 
bullet 1 and also Appendix F Session Recovery algo for 
initiator.)

I may be mistaken, but in previous drafts, this was not
the case and commands always stayed allegiant to a CID.

So the question is.. how does one use a Status cache
belonging to the old connection in this new model (now that 
the commands are going to be retried over any connection)
Doesnt this become more complex ?

Secondly, this also means that one must walk the command
list at the target and quiesce connection allegiance during 
logout - which may not be required if the commands were to be 
retried over the same connection always.

Comments  ?

-Sandeep

---

From: [email-masked] (Sandeep Joshi)
Date: Sat, 28 Jul 2001 18:01:54 -0400 (EDT)

Er..I mixed up the terminology.  By "same connection", I meant
"same CID" - so one could retry cmds ONLY on the new TCP connection 
with the same CID, after the old TCP connection had failed.  
This avoids having to change connection allegiance on
the stuck commands, as part of connection logout.

The main questions, however, were these :

1) What is the effect of a CID logout on the status (statSN)
   cache ?  Can it be reused when the commands are retried ?
   (Think not..)

2) After one does a login for recovery using an old CID,
   can new SCSI commands be issued on that new TCP connection.
   (though it bears an old CID identifier)

-Sandeep

> Sandeep,
> 
> Retry over any connection was always the case.
> Commands can change allegiance only after a logout.
> The commands get quiesced anyway and you have to mark them ready for a
> retry anyhow (you don't want retry at arbitrary times to hit unpunished the
> target). After that it doesn't matter where you retry (same connection,
> another old one, a new one).
> 
> The only new thing is command replay (after status was sent but before it
> is acked).
> 
> Regards,
> Julo
> 
> [email-masked] (Sandeep Joshi) on 28-07-2001 06:37:33
> 
> Please respond to [email-masked] (Sandeep Joshi)
> 
> To:   [email-masked]
> cc:
> Subject:  iSCSI connection recovery
> 
> 
> 
>

---

From: "Julian Satran" <[email-masked]>
Date: Sat, 28 Jul 2001 09:05:19 +0300

Sandeep,

Regards,
Julo

To:   [email-masked]
cc:

Comments  ?

-Sandeep

---

From: Sandeep Joshi <[email-masked]>
Date: Mon, 30 Jul 2001 10:33:49 -0400

Julian,

The explanation helps.  Now that the model is clear, let me 
ask if something like this would work.. 

It seems possible for the target to send back SCSI responses 
during recovery logout, even before commands are retried.
The recovery logout could act as a final (& appetizing) SNACK.

Since the target still has a statSN index on the responses,
it could use the expStatSN field in the Logout Command
and send all responses from [expStatSN-endStatSN].

Initiator            Target
---------            ---------
Logout (for recovery) ---->>
   <<--- SCSI Data+Responses from [expStatSN-endStatSN]
   <<--- Logout Response	

Once the logout occurs, the statSN ranges for the CID are 
lost and command retry cannot be avoided.

This logout optimization has larger benefits for Writes.
Retrying Write Commands (e.g. tape backup) may involve 
sending all the data (or minimally FirstBurst) all over 
again!   Of course, cost-benefit depends on queue lengths, 
bandwidth, frequency of connection recovery, transfer 
sizes, ULP timeouts, etc.

Comments ?

-Sandeep

Julian Satran wrote:
> 
> Sandeep,
> 
> Your status cache is made of some task  related structures.  You can reuse
> those - just link them to a new connection.
> As for StatSN - you can't reuse those as each connection establishes its
> own (new) StatSN at login (even if reusing the old CID).
> 
> Regardless of what CID the connection bears - it is a new connection and
> you can issue new commands. Even for the old ones by reissuing you in fact
> indicate the new allegiance (the logout suspended them and the retry
> establishes the new allegiance).
> 
> [email-masked] (Sandeep Joshi) on 29-07-2001 01:01:54
> 
> Subject:  Re: iSCSI connection recovery
>

---

From: "Julian Satran" <[email-masked]>
Date: Mon, 30 Jul 2001 18:15:54 +0300

Sandeep,

Both Login (with the X bit it is a logout/login) and Logout have an
ExpStatSN just for this reason.
If this does not come through clear in the text please let me know.

Regards,
Julo

Sandeep Joshi <[email-masked]> on 30-07-2001 17:33:49

Please respond to Sandeep Joshi <[email-masked]>

To:   Julian Satran/Haifa/IBM@IBMIL, [email-masked]
cc:
Subject:  Re: iSCSI connection recovery

Julian,

Initiator            Target
---------            ---------

Comments ?

-Sandeep

---

From: "Julian Satran" <[email-masked]>
Date: Tue, 31 Jul 2001 09:33:56 +0300

Sandeep,

Don't blame me for size.  I would prefer the spec to be 10-20 pages and
have a formal spec.
The past of this list has clearly shown that more text helps at least to
gain consensus :-)

Regards,
Julo

Sandeep Joshi <[email-masked]> on 30-07-2001 21:03:04

To:   [email-masked]
cc:

> I assume you are not suggesting a FinalLogout ;-), are you?

No I was not :-)  I missed reading the 4th paragraph of
Section 2.14 which alludes to this logout processing.
The spec is getting monstrous at 185 pages.

regards,
-Sandeep

Julian Satran wrote:
>
> Sandeep,
>
> There is nothing to stop them sending the state but there is no way for
the
> initiator to acknowledge those (Logout takes it out of the full feature
> phase - nothing valid after) and a new connection is a new day ...
>
> I assume you are not suggesting a FinalLogout ;-), are you?
>
> Regards,
> Julo
>
> Sandeep Joshi <[email-masked]> on 30-07-2001 18:38:48
>
> To:   Julian Satran/Haifa/IBM@IBMIL
> cc:
> Julian
> Wait..!  I was proposing that the expStatSN could be used to send back
> the responses (even *before* commands get retried)   See below
>
> -Sandeep
>
> Julian Satran wrote:
> >
> > Sandeep,
> >
> > > > The commands get quiesced anyway and you have to mark them ready
for
> a
> > > > retry anyhow (you don't want retry at arbitrary times to hit
> unpunished
> > > the
> > > > The only new thing is command replay (after status was sent but
> before
> > it
> > > > is acked).
> > > >
> > > > Regards,
> > > > Julo
> > > >