Subject: RE: iSCSI: reusing ISID for recovery

From: "KRUEGER,MARJORIE (HP-Roseville,ex1)" <[email-masked]>
Date: Fri, 24 Aug 2001 20:51:43 -0400

I don't see how Option A is prone to "wild closing of sessions".  The target
is only looking for sessions with this particular initiator and closing them
if the ISID matches.

If an initiator doesn't have a valid TSID (login w/TSID=0), it means it has
lost state entirely (reboot) or knows it wants to immediately reset a
session (NIC failure).  How could there possibly be a case where the
initiator has an active valid session with the same ISID, but just doesn't
know about it??  Rejecting the login seems pointless, since obviously the
initiator either has a bug or intends to quickly reset the session. 

The behavior chosen (Option C) will cause the initiator's recovery to be
delayed while the target NOPs all the connections and waits for them to time
out - this will only delay the initiators recovery unnecessarily.  I can't
help but think this will cause long term problems for the protocol.

Marjorie Krueger
Networked Storage Architecture
Networked Storage Solutions Org.
Hewlett-Packard
tel: +1 916 785 2656
fax: +1 916 785 0391
email: [email-masked] 

> -----Original Message-----
> From: Julian Satran [mailto:[email-masked]]
> Sent: Thursday, August 23, 2001 9:39 PM
> To: [email-masked]
> Subject: Re: iSCSI: reusing ISID for recovery (Was: RE: iSCSI 
> - Change -
> Login/Text...)
> 
> 
> 
> Mallikarjun,
> 
> On  your point 1 that is what is stated today in the draft.
> 
> On your point 2 option C is the one we took in the draft, after some
> debate.
> 
> Option A is prone to "wild closing of sessions" and option B is also
> relaying to much on the good behaviour of the
> client. It also introduces a "feature" that complicates login/logout.
> 
> Our postion on this is  that the initiator should logout the session
> explicitly if it can (and in this case it can as the target 
> has ascertained
> that the session is alive).
> 
> I agree that you may want to update the stayte diagram to 
> reflect this.
> 
> Julo
> 
> 
> "Mallikarjun C." <[email-masked]>@ece.cmu.edu on 24-08-2001 01:46:40
> 
> Please respond to [email-masked]
> 
> Sent by:  [email-masked]
> 
> 
> To:   [email-masked]
> cc:
> Subject:  iSCSI: reusing ISID for recovery (Was: RE: iSCSI - Change -
>       Login/Text...)
> 
> 
> 
> Julian and all:
> 
> This thread mirrors another discussion some of us are
> having in a different forum.  Following (two bullets 1
> & 2 below) is what I proposed there, attempting to address
> two issues -
>      a) how to recover sessions when target and the initiator
>            have conflicting views of the same TCP connection(s)?
>            (Initiator NIC fails, but there's no I/O activity,
>            and the target doesn't see any connection failure.)
>      b) More specifically, how to address the above problem
>            when the initiator *does not want* to re-instate failed
>            connections since it only implements the mandatory
>            session recovery?
> 
> This could add clarity or muddle things up here, though hopefully
> the former...
> 
> 1 If login is sent with the same ISID, same TSID, same CID and X-bit,
>   then it means a failed connection is being re-instated (whether
>   or not there are multiple connections in the session).  This login
>   attempt must be done before the connection timeout (transition M1),
>   or if this is the only connection in the session, also before the
>   session timeout (transition N6) - to be counted as a connection
>   reinstatement effort.
>            o CmdSN counters (CmdSN, ExpCmdSN) are continued.  
> Initiator
>              must do command plugging when there's a mismatch
>              between its CmdSN and ExpCmdSN in the login response.
>            o Since this is an implicit connection logout, all 
> the active
>              tasks on the connection are either internally terminated,
>              or made non-allegiant (based on ErrorRecoveryLevel=x/y,
>              TBD) for recovery.
> 
> 2 If login is sent with the same ISID and TSID=0, the session (if it
>   exists on the target) is being cleared and any active connections
>   that the target sees must be immediately (at the end of the login
>   process including any initiator authentication) transport reset.
>   Initiator may attempt this only after it ascertains a 
> session failure
>   on its end (ie. all connections entered RECOVERY_START).
>            o CmdSN counters get reset.  Initiator has to perform the
>              currently defined session recovery actions.
>            o All active tasks of the session are internally 
> terminated.
> 
> 
> Essentially, I was proposing extending the same notion of "implicit
> logout" of a connection to the session level.  The options that I
> see are -
> 
>        A) Should iSCSI let it happen by default as stated above (ie.
>           same ISID, TSID=0 always wipes out the pre-existing session
>           on target, since we are mandating it to be used only when
>           initiator sees a session failure)?
>        B) Should iSCSI mandate making this intended cleanup explicit
>           by setting a bit (Say C-bit, for Clear) in the Login Command
>           PDU to prevent an accidental session cleanup with a buggy
>           initiator code?
>        C) Should iSCSI merely state that targets must ascertain
>           the connection state(s) whenever a new session creation
>           attempt is made with a known ISID and TSID=0?  (sort of
>           defeats the intention of the initiator wanting quicker
>           session recovery since the Login command PDU would have
>           to idle till target ascertains the connection state(s)).
> 
> I prefer A, or B.
> 
> Going with A or B means that the description of transition N3
> in the session state diagram would have to change to:
>      Last LOGGED_IN connection had ceased to be LOGGED_IN,
>         or a Login Command requesting clearing the session (also
>         with C-bit set, if option B) was received by the target.
> 
> The transition N7's description would have to be augmented as
> well to:
>         Session recovery attempt with an implicit logout,
>         or connection reinstatement/new CID addition.
> 
> Comments?
> --
> Mallikarjun
> 
> 
> Mallikarjun Chadalapaka
> Networked Storage Architecture
> Network Storage Solutions Organization
> MS 5668   Hewlett-Packard, Roseville.
> [email-masked]
> 
> 
> >Stephen,
> >
> >That can happen as the target may set-up completely new TCP 
> connections
> >(the old sockets are still there and look OK).
> >Untill the login is  progessing he assumes that this is just another
> >open-session attempt. Then he checks the old session and the 
> session is
> >dead (initiator has closed the connections).
> >
> >The target has to distinguish only between a session that is 
> alive (and
> >reject the new one) and one that its dead in which case it 
> can clean-up.
> >
> >Julo
> >
> >"Wheat, Stephen R" <[email-masked]> on 23-08-2001 22:50:56
> >
> >Please respond to "Wheat, Stephen R" <[email-masked]>
> >
> >To:   Julian Satran/Haifa/IBM@IBMIL, [email-masked]
> >cc:
> >Subject:  RE: iSCSI - Change - Login/Text commands with the 
> binary stage
> co
> >          de
> >
> >
> >
> >Julian,
> >
> >I don't understand your answer.  For the scenario given, I 
> would presume
> >then that the target would reject the new session attempt, 
> as it sees the
> >previous session still "alive".  What is there to tell the 
> target that
> this
> >is any different from when the Initiator is erroneously using the
> >repetitive
> >session id?
> >
> >Thanks,
> >Stephen
> >
> >-----Original Message-----
> >From: Julian Satran [mailto:[email-masked]]
> >Sent: Thursday, August 23, 2001 11:15 AM
> >To: [email-masked]
> >Subject: Re: iSCSI - Change - Login/Text commands with the 
> binary stage
> >co de
> >
> >
> >Stephen,
> >
> >1.If the initiator goes away for a while and reboots and there was no
> >activity on the connections
> >the target may see a session alive (I am not sure that it 
> has to appear on
> >the state diagram but maybe).
> >
> >2.Again - I am not sure that the curent state diagram 
> includes death of
> the
> >initiator
> >
> >Julo
> >
> >"Wheat, Stephen R" <[email-masked]>@ece.cmu.edu 
> on 23-08-2001
> >19:58:34
> >
> >Please respond to "Wheat, Stephen R" <[email-masked]>
> >
> >Sent by:  [email-masked]
> >
> >
> >To:   [email-masked]
> >cc:
> >Subject:  Re: iSCSI - Change - Login/Text commands with the 
> binary stage
> co
> >      de
> >
> >
> >
> >Julian,
> >
> >1.3.6 ISID states that the target should check to see if the 
> old session
> is
> >still active when a duplicate session is detected.
> >
> >I have two questions, the second only if you answer in the 
> affirmative on
> >the first ;^)
> >
> >1. Is there a properly executed sequence of events (i.e., no 
> coding error
> >on
> >the target side) where the session is not active, but the 
> target hadn't
> >taken notice of it?  It appears this as a protocol-specified 
> means to work
> >around a flaw in a target's implementation.  I interpret the 
> state diagram
> >transitions as being atomic wrt other commands.  I.e., the 
> last logout
> >would
> >result in the various transitions of the connection/session 
> prior to the
> >initiator starting the session up again.  And the target would have
> >completed the transitions prior to handling a new session request.
> >
> >2. If you answered (1) in the affirmative, then the word 
> "Active" is not
> >consistent with the 6.3 Session State Diagram.  Does this 
> mean the target
> >got lost, due to transport failures of any sort, in its 
> transition from
> >Q3-to-Q2-to-Q1?  It sounds like the intent is to close the 
> old session if
> >the session was in Q2 or Q4, presuming if it were in Q1,  it 
> would not
> have
> >been found as a duplicate.
> >
> >Stephen
> >
> >
> >
> >
> >
> >
> >
> >
> 
> 
> 
> 
>

---

From: "Julian Satran" <[email-masked]>
Date: Sat, 25 Aug 2001 10:22:33 +0300

It may be another OS running on the same machine or CPU complex or it could
be an attack.
In any case if the initiator is up and fine he can as well do logout.
It is a rare enough event for us not to try to optimize.

The reboot is the only case in which the initiator can't logout and about
which we care.

And I would like to remind you all that we where on this exact thread more
tan 3 months ago (other players).
I just restated the rationale for an (apparent) newcomer.

Julo

"KRUEGER,MARJORIE (HP-Roseville,ex1)" <[email-masked]>@ece.cmu.edu
on 25-08-2001 03:51:43

Please respond to "KRUEGER,MARJORIE (HP-Roseville,ex1)"
      <[email-masked]>

To:   [email-masked]
cc:
Subject:  RE: iSCSI: reusing ISID for recovery

I don't see how Option A is prone to "wild closing of sessions".  The
target
is only looking for sessions with this particular initiator and closing
them
if the ISID matches.

The behavior chosen (Option C) will cause the initiator's recovery to be
delayed while the target NOPs all the connections and waits for them to
time
out - this will only delay the initiators recovery unnecessarily.  I can't
help but think this will cause long term problems for the protocol.

Marjorie Krueger

---

From: "Jim Hafner" <[email-masked]>
Date: Mon, 27 Aug 2001 07:16:33 -0700

Julo,
You wrote:
  And I would like to remind you all that we where on this exact thread
more
  tan 3 months ago (other players).
  I just restated the rationale for an (apparent) newcomer.

On the other hand, I don't recall this issue ever being called for
concensus.  Opinions were posted and nothing followed up.

I happen to agree with Marjorie.

Jim Hafner

Julian Satran/Haifa/IBM@[email-masked] on 08/25/2001 12:22:33 AM

To:   [email-masked]
cc:

Julo

To:   [email-masked]
cc:

Marjorie Krueger

---

From: "Julian Satran" <[email-masked]>
Date: Mon, 27 Aug 2001 17:50:09 +0300

Jim,

It would be interesting to hear your arguments. The functions is clearly
there - only that an initiator has to perform two steps
Why is it important (except for boot) that a living initiator do it in one
step.

Regards,
Julo

Jim Hafner/Almaden/IBM@[email-masked] on 27-08-2001 17:16:33

Please respond to Jim Hafner/Almaden/IBM@IBMUS

To:   [email-masked]
cc:

Julo,
You wrote:

Jim Hafner

To:   [email-masked]
cc:

Julo

To:   [email-masked]
cc:

Marjorie Krueger

---

From: "Julian Satran" <[email-masked]>
Date: Mon, 27 Aug 2001 19:50:00 +0300

Jim,

Option A is clearly unacceptable as an initiator may do harm.
Recovery is not done by the target neither in B nor in C.

In B an initiator has to add a bit (he may be tempted to put it in as "my
code is safe" while in C
it requires the initiator to logout first before he can reinitiate the
session.

B is an optimization over C - and quite dangerous.

Julo

Jim Hafner/Almaden/IBM@[email-masked] on 27-08-2001 19:22:12

To:   [email-masked]
cc:

Julo,

Here's the way I look at it.  I think this is an "error detection/recovery"
issue.

With a rule that the ISID doesn't get reused by an initiator to a given
target portal group,  it's the obligation of the initiator not to do it.
If  the initiator does it anyway, then it is acting in protocol error and
it is the job of the target to enforce that rule.   If the initiator still
does it anyway, it's probably because it has "lost its way", then it is in
error state (but how does it know this?).  I prefer option B. The target
enforces the protocol rule only.  An unexpected reject is the "error
detection" mechanism for the intiator.  The "forced" login is part of the
error recovery, as driven by the initiator which is doing the error
recovery.

I see option A as a simpler mechanim that makes the "explicit" error
recovery request by the initiator an "implicit" request with no direct
detection mechanism on the initiator side.   I'd go with this choice if it
is decided that the only time we'd expect an initiator to make this request
is after a reboot or other massive reset, where it wouldn't have clue about
it's previous state.  If there are cases where we think it won't "do the
right thing" when it could, then I would say that option B as a safer
mechanism.

In short, it gets the target out of doing error recovery on behalf of the
initiator without explicit (option B) or implicit (Option A) direction.
It's not in the business of "guessing" if the initiator needs recovery.

Jim Hafner

Julian Satran/Haifa/IBM@[email-masked] on 08/27/2001 07:50:09 am

To:   [email-masked]
cc:

Jim,

Regards,
Julo

To:   [email-masked]
cc:

Julo,
You wrote:

Jim Hafner

To:   [email-masked]
cc:

Julo

To:   [email-masked]
cc:

Marjorie Krueger

---

From: "Jim Hafner" <[email-masked]>
Date: Mon, 27 Aug 2001 09:22:12 -0700

Julo,

Jim Hafner

To:   [email-masked]
cc:

Jim,

Regards,
Julo

To:   [email-masked]
cc:

Julo,
You wrote:

Jim Hafner

To:   [email-masked]
cc:

Julo

To:   [email-masked]
cc:

Marjorie Krueger

---

From: "Mallikarjun C." <[email-masked]>
Date: Mon, 27 Aug 2001 10:07:08 PDT

Julian,

>It may be another OS running on the same machine or CPU complex

I know that the rules for iSCSI-Names aren't final yet, but I
would think that the second O/S copy in the case you mention will
have a different iSCSI-Name.

>or it could

That's the reason in my description I had the following phrase
"at the end of the login process including any initiator authentication".
I assume that's adequate guarantee.

>In any case if the initiator is up and fine he can as well do logout.

I am not sure I understand this - are you suggesting a connection
reinstatement (within-session recovery), or second connection addition?
If I hadn't made it clear, I was searching for a decent solution for
single-connection, session-recovery-only case.  This implicit session
logout is the only one I came up with.  Please comment if I missed 
something (keep in mind that old TSID could very well be lost if
it's a NIC failure and the session manager is hosted by the NIC itself).

For these reasons, Option A is my preferred.  I also suggested that 
"Initiator may attempt this only after it ascertains a session failure
on its end." - so this is only to take care of the asymmetric session 
views on either end.

As I stated, I can live with option B which I suspect is the case of
protocol making allowances for incorrect implementations.

Option C, IMHO, would be needlessly slowing down recovery (and could
have an impact on boot times, depending on initiator boot probe
algorithm).

Regards.
--
Mallikarjun 

Mallikarjun Chadalapaka

---

From: "Jim Hafner" <[email-masked]>
Date: Mon, 27 Aug 2001 10:23:47 -0700

Julo,

 Comments in line of your last response.

Jim Hafner

Julian Satran/Haifa/IBM@[email-masked] on 08/27/2001 09:50:00 am

To:   [email-masked]
cc:

Jim,

Option A is clearly unacceptable as an initiator may do harm.
<JLH>
How can it do harm? It should only do this when it's "lost it's way" and so
doesn't have any state to harm.  If it can't tell that it has a session
with that ISID live, then having that session killed at the target end
isn't going to cause any problems.   If it can tell and still blows the
protocol, then all bets are off anyway.  [As an aside, I believe this is
the behavior of FC, though admittedly, it's at a lower and less
sophisticated layer then we are talking about.]  There's a further summary
of these issues below.
</JLH>

Recovery is not done by the target neither in B nor in C.
<JLH>
Certainly, recovery is done by the target! Perhaps your definition of
"recovery" is different than mine.  I include "cleanup" in "recovery".
It's not trying to get back the previous state, so much as get out from
under the bad state it's in.  So, in both cases, it's supposed to cleanup
dead session/connections.  In C, it's not even clear to the initiator that
this is going to happen!  In B, it knows it's explicitly asking for this
cleanup (with the bit set).
</JLH>

B is an optimization over C - and quite dangerous.
<JLH>
If B uses the bit indiscriminantly, then it's its own fault and tough
cookies.  So, I don't see how this is "quite dangerous".

I personally think that C is more dangerous, because the behavior is less
predictable to the initiator.  How can the initiator judge what the target
thinks of the state of the connections/sessions?  Maybe the target sees the
connections as dead, even though the initiator doesn't (yet?).
</JLH>

<JLH>
Ask the question why the initiator is taking this action and what state is
he in.  I see three cases:

1) If the initiator knows he has (or thinks he has) a session that he wants
to restart, then he can logout and login and this ISID rule enforcement is
*never* triggered. This is true whether he sees the session as live or
dead.

2) If the initiator knows he has a session and still proposes to break the
rule, then tough.  Either he has to work harder (option B) or he gets what
he gets (option A) in predictable fashion.  He knows what state he's in and
he knows the consequences of his actions.

3) The only time he should break his side of the protocol (reuse the ISID)
is if he has lost knowledge of the previous session.  In that case,
everyone might as well kill that session since context is lost on the
initiator side anyway.  That means that option A is just fine. Option B
carries the extra benefit of assisting him in error detection (that he's
lost his way).   He may not know enough a priori to know he's lost his way
(which is why I have a weak preference for B, though all of this discussion
is leaning me more towards A).
</JLH>

Julo

To:   [email-masked]
cc:

Julo,

Jim Hafner

To:   [email-masked]
cc:

Jim,

Regards,
Julo

To:   [email-masked]
cc:

Julo,
You wrote:

Jim Hafner

To:   [email-masked]
cc:

Julo

To:   [email-masked]
cc:

Marjorie Krueger

---

From: "Jim Hafner" <[email-masked]>
Date: Mon, 27 Aug 2001 14:18:31 -0700

Marj,

I agree (that is, I don't like option C).  I'm waffling between A and B.  A
is simpler and B is "more polite".

But I have a twist on option B that may make it a bit more palettable.

The problem with option B is that requires extra protocol to have the
additional bit to force the logout of the "dead" session, mostly because
the initiator doesn't have the context left for the old session.  Suppose
we have the target reject but in the response it puts what it is using for
TSID in the usual place.  In effect, this is saying to the initiator "I
already have a session in place with this ISID/TSID value".

I think this gives the initiator the handle it needs to expressly do a
logout of the old session (with a valid ISID/TSID pair) or at least enough
information for the initiator to figure out what's gone wrong.  That is, it
can do session recovery or whatever else is needed.  In this way, we don't
need anything like a "force" bit in the login.  He has more steps to do,
but they would be the same steps he would do if he just wanted to cleanup a
old session he did know about.

Jim Hafner

"KRUEGER,MARJORIE (HP-Roseville,ex1)" <[email-masked]>@ece.cmu.edu
on 08/27/2001 01:39:52 pm

To:   [email-masked]
cc:

If it is another OS, it should be a different iSCSI initiator, and hence
will have a different initiator name, so this case shouldn't be a factor.
It's not the ISID along which identifies the initiator, it's the initiator
name  + ISID.

I believe the case in which this behavior (option C) will become an issue
is
in the case of initiator reboot, which will be the norm, not the exception.
IOTW, most of the cases where the target will be faced with an initiator
login w/an "in use" ISID, TSID=0 will be in the case of an initiator
reboot.
That is the case we should optimize for.  I could live with Option B, but
Option A seems sufficient.

It seems the primary value of option C is to guard against malfunctioning
initiators.  As has been said many times, we cannot design a protocol to
guard against coding errors.

Marj

> Sent: Saturday, August 25, 2001 12:23 AM
> To: [email-masked]

---

From: "Jim Hafner" <[email-masked]>
Date: Mon, 27 Aug 2001 16:12:28 -0700

Marj,

All of your questions point out the lack of completeness in my thinking
(which is why I said "I think this gives...").

I guess I was thinking that the initiator could act like it wants to join a
new connection to the existing session, so that it can logout that session.
You're right that this is a problem for 1-connection-per-session targets,
but don't we have that problem anyway (if there is one connection and the
host looses it, what does it do now for recovery?)

As I said, I'm starting to lean a bit more toward A.  It's the cleanest and
simplest.

Jim Hafner

"KRUEGER,MARJORIE (HP-Roseville,ex1)" <[email-masked]>@ece.cmu.edu
on 08/27/2001 03:46:23 pm

To:   [email-masked]
cc:

How would the initiator log out the "dead" session?  Even if it knows a
TSID, it doesn't have any state, or any connections (assuming reboot).  Are
you suggesting the initiator perform some new sort of recovery where it
rebuilds a partial session state and "acts" like it wants to join the dead
session, for the purpose of logging it out?   Presumeably, this wouldn't
work with target implementations that only allow one connection? (see
Mallikarjun's last email) In my mind, any sort of recovery only makes sense
when there is surviving state information on both parties (I & T).  Is your
thinking different?

IMO, if option B were provided, initiator implementations would always just
turn on the "logout" bit, "just to be safe" and therefore, it would
effectively be no different than option A (just more protocol bits).

Marj

> From: Jim Hafner [mailto:[email-masked]]
> Sent: Monday, August 27, 2001 2:19 PM
> To: [email-masked]

---

From: "Mallikarjun C." <[email-masked]>
Date: Mon, 27 Aug 2001 18:45:08 PDT

Jim,

I am glad you're leaning towards A.  I believe that's the only
option that makes most sense.

>if there is one connection and the

Here's an excerpt from my first email with this subject heading -

---->
<----

There are two sub-cases under one-connection-per-session targets -
   case-a: Target supports connection reinstatement for the same CID
           (ie. within-session recovery supported).
   case-b: Target doesn't support connection replacement, mandatory
           session recovery is the only supported form of recovery.

The above excerpt from my first email only addresses case-a (it is 
defined in the rev07 draft).

As I said in my last posting (can't quote a URL since the mail
archive appears broken), I was attempting to find a decent solution
for case-b (not to mention the rebooting-host-after-a-crash scenario), 
and I believe option A is the best for it (though option A can be 
used whenever one/all connections reach the RECOVERY_START state on 
the initiator, or initiator doesn't have an active session at all).

Regards.
--
Mallikarjun 

Mallikarjun Chadalapaka

---

From: "Jim Hafner" <[email-masked]>
Date: Tue, 28 Aug 2001 09:04:34 -0700

Julo,

I'm unclear why you think that option A isn't viable and why you think a
more complicated protocol is required here.

If the initiator 'lost his way' such that he thinks he has an existing
session, then he does the normal error recovery for that scenario (a la the
mechanisms Mallikarjun noted).  If he 'lost his way' so badly that he
doesn't even know he has an existing session, then option A is probably the
right thing to do.  In this case, he won't even know he's lost his way so
can't expect to set the X bit rationally.  He'd either not set it, and then
we need a rule for what the target does then OR he sets it knee-jerk in
which case there's no point in having it (as Marj, Deva, et al, have
argued).

In short, if he has knowledge of a lost session, he does session recovery.
If not, he just logs in and the target cleans up its end.

I'd like to close this issue as well, should there be a call for consensus?

Jim Hafner

Julian Satran/Haifa/IBM@[email-masked] on 08/27/2001 09:39:03 pm

To:   [email-masked]
cc:

Marjorie, Jim, Mallikarjun, Deva & all

Let's attemp to close that optimization that is so dear to your hearts.
If we go for the following scheme (very close to B):

The initiator knows "that he lost his way" (in the initiator jungle tough)
and issues a Login + ISID + TSID=0 + X bit on (meaning
" I think I have an old session - please clean it up for me" and open a new
one) then the target may do one of the following:

- before going to the effort of authenticating the new initiator check if a
session exist (if he has the initiator name) or go to authentication if it
does not
- If he has a session then clean it and create a new one
- If he does not have a session - we have 3 options:
a)- tell him hid did not have an old session and drop him
b)- tell him he did not have an old session but nevertheless create a new
one
c)- create a new session and do not tell him

With option a we are close to the old option c but the initiator does not
have to find the old session he can do now safely a new login
With b & c  we are back at the old b.

Which one would you give your kingdom for ( and my peace!).

Julo

"Dev" <[email-masked]>@ece.cmu.edu on 28-08-2001 01:44:15

Please respond to <[email-masked]>

To:   "KRUEGER,MARJORIE \(HP-Roseville,ex1\)" <[email-masked]>,
      <[email-masked]>
cc:
Subject:  RE: iSCSI: reusing ISID for recovery

All,

When a session is already running authenticated, a login request with the
same
ISID + initiator fails authentication. Will the session continue to run?
Or
if the target fails during parameter negotiation?

I guess, yes. i.e. target will not act on the existing session, until
the login completes successfully. Or are there exception to these?

Thanks

Deva
Platys Communications

---

From: "Julian Satran" <[email-masked]>
Date: Tue, 28 Aug 2001 20:28:36 +0300

Marjorie,

When I say without authetication I meant only on option A.
And it is option B with some details and using the X bit.

Julo

on 28-08-2001 20:15:29

To:   [email-masked]
cc:

I am opposed to doing anything before authentication as this opens
opportunities for DOS attacks or some new form of hacking.  Also, this
would
be a "special case" behavior for login, and I'm sure I speak for all
implementors when I beg you not to create unnecessary special case
behavior.
We like a simple state machine for the login process.

I agree, w/Jim, this looks much like a more complicated option B, and the
value escapes me?

Marj

> Sent: Monday, August 27, 2001 9:39 PM
> To: [email-masked]

---

From: "KRUEGER,MARJORIE (HP-Roseville,ex1)" <[email-masked]>
Date: Tue, 28 Aug 2001 13:15:29 -0400

I am opposed to doing anything before authentication as this opens
opportunities for DOS attacks or some new form of hacking.  Also, this would
be a "special case" behavior for login, and I'm sure I speak for all
implementors when I beg you not to create unnecessary special case behavior.
We like a simple state machine for the login process.

Marj

---

From: "Mallikarjun C." <[email-masked]>
Date: Tue, 28 Aug 2001 15:43:48 PDT

Julian,

I disagree that it is an optimization that most of us are
seeking, but more to the point...

I continue to prefer option A even after this proposal.  

Reasons?

o Attaching more semantics to X-bit is only more complexity.
  Only "plug command" and "reinstate connection" until now; 
  with this additionally, "lost my way" (if TSID = 0). 

o This is option B (overloading X-bit instead of a new C-bit), 
  which we all (I thought including you) agreed shouldn't be 
  adopted.

o I am unclear on the model you have in mind on authentication
  - going with any of these models.  I am further perplexed with
  this response you gave to Marjorie -

>When I say without authetication I meant only on option A.

  To quote my original email yet again, here is my phrasing:

  "at the end of the login process including any initiator authentication"

  I believe that's the only time a target must decide to invoke
  option A.

Regards.
--
Mallikarjun 

Mallikarjun Chadalapaka

---

From: "Julian Satran" <[email-masked]>
Date: Wed, 29 Aug 2001 20:46:18 +0300

Thanks - I started feeling bad. I could not get a message through. Julo

"Dev" <[email-masked]> on 29-08-2001 19:03:40

To:   Julian Satran/Haifa/IBM@IBMIL, <[email-masked]>
cc:
Subject:  RE: iSCSI: reusing ISID for recovery

Julian,

>But - as this is bound to bring
>many an initiator writer to set always
>the bit to "just cover for the case" I propose that the command fails if
>the X bit was on but there was no need for it to be on.

The only need to set the X-bit (when ISID, TSID=0) could be to forcibly
close a pre-existing session, if any in the target right?

I agree with this proposal, as this forces the initiator to issue the
command
with an X-bit only when there is an error (a session already exists).
I also like that returning an error when there is no need to set an X-bit,
as it discourages the initiators to set the X-bit by default, for opening
a session.

Thanks

Deva
Platys Communications.

I think that making a new session bring down an old one makes just too easy
for common mistakes.

The scheme I've proposed - like the original option B - avoids just that by
letting the target know that this
was intentional - by adding the X bit.   But - as this is bound to bring
many an initiator writer to set always
the bit to "just cover for the case" I propose that the command fails if
the X bit was on but there was no need for it to be on.

I hope that this will get us to have a scheme in which both the common
mistakes can't do harm and the initiator writers
will not get away by always having the X bit on.

The price is a login in 2 steps in case of recovery - and I think this is
acceptable.

Julo

Jim Hafner/Almaden/IBM@[email-masked] on 28-08-2001 19:04:34

To:   [email-masked]
cc:

Julo,

Jim Hafner

To:   [email-masked]
cc:

Julo

All,

Thanks

Deva
Platys Communications

---

From: "Julian Satran" <[email-masked]>
Date: Wed, 29 Aug 2001 22:58:13 +0300

Please read my answer to Marjorie. Julo

Santosh Rao <[email-masked]>@ece.cmu.edu on 29-08-2001 22:32:21

Please respond to Santosh Rao <[email-masked]>

To:   [email-masked]
cc:
Subject:  Re: iSCSI: reusing ISID for recovery

Julian,

I dis-agree with the below proposal. I am in favor of proposal (A) as
suggested by Mallikarjun, Marj & Jim Hafner. The same was earlier proposed
by
myself in a previous thread on this issue a few months ago. The iscsi draft
should not be optimizing to deal with coding errors, but to optimize the
behaviour for the typical functional scenario.

For initiators that use session recovery on any iscsi errors (ex : format
error, digest error, protocol error, sequence error), it is an optimization
to
perform the logout + re-login in 1 step using the session login with the
same
ISID and 0 TSID to indicate an implicit logout + re-login. (what has been
described as option A).

The above semantics are also required in the case wherein the initiator
lost
its prior state information and attempts to establish a session with the
same
ISID and 0 TSID.

I disagree with the proposal made below since it adds additional complexity
to
initiators cleanup & re-login in the case where they are booting up and
establishing the session for the first time. The initiator has no idea if a
previously established session exists, that requires clean-up. It would set
the X-bit [or C-bit as proposed in option B] to clear any pre-existing
session. This would cause the target to reject the session login since
there
was no session that required cleanup, thereby, requiring the initiator to
login again without the X bit set.

All these extra handshakes in the above case during system startup are
going
to cause a bloat in login time which should be avoided. There is no need
for
all this extra complexity just to protect against coding errors by an
initiator that accidentally cleared its own session.

The semantics of option (A) are also used in FC and there are no complaints
with that model there. In fact, that is an often used technique to logout &
re-login in FC.

Regards,
Santosh

> > > >"Wheat, Stephen R" <[email-masked]> on

 - santoshr.vcf

---

From: "Jim Hafner" <[email-masked]>
Date: Wed, 29 Aug 2001 13:56:59 -0700

Julian,

In the first place, a multi OS machine should have a different iSCSI name
for each OS image (that's what the iSCSI Name is supposed to name).  So,
this sort of collision shouldn't occur.  If these OS's are sharing the same
SCSI stack or iSCSI stack (and same iSCSI name), then the iSCSI layer
should be handling this problem directly.  This is one of the reasons why
"configuring and partitioning of the ISID"s among the components (session
coordinators) that drive/coordinate/handle/process/instantiate/etc. the
sessions is an important and desireable feature of these components.  You
should be able to stuff an iSCSI Name and a set of ISIDs down through the
layers to the session coordinators (be they hardware or software) so that
they can independently handle their responsibility without name collisions
with other components.  The problem you think is going to happen, I think,
should never happen in a well-behaved system.

So, as I read your model now, you've still got the target "checking to see
if the session is alive".   I think we're trying to avoid this.  It's not
so much for optimization (as you alluded to).  For me, its to make the
target's behavior completely predictable.  Can the initiator judge a priori
what the target thinks is going on with the other session if it doesn't
have a context to check on itself?

If the old session is alive enough to answer to NOP, isn't it alive enough
to clean it up?  When wouldn't it be?

Jim Hafner

Julian Satran/Haifa/IBM@[email-masked] on 08/29/2001 12:56:24 pm

To:   [email-masked]
cc:

Marjorie,

I probably have some difficulties in explaining and you in reading ... and
that contributes to increasing distance.
Doing an automatic logout when a new session is seen can be performed by
mistake by an OS in a multi OS machine. It is just to easy to let it
through.  After a reboot the right procedure would be to login cleanly (no
previous knowledge needed). The target will ascertain that the old session
is dead and let you in.

You will need the X only when the old session is alive (answering to NOP)
but you don't know how to clean it.

Reboot behavior is close to what you have in the current draft ( a single
login needed - no X).

The difference is when you have a living connection - then you have for the
session the same thing as for a connection you have to restart it.

Julo

on 29-08-2001 22:27:35

To:   [email-masked]
cc:

It would help to get your message thru if you could answer our requests for
an explanation of your thinking.  We have tried several times to explain
our
logic (w/examples) but I haven't seen an example from you supporting a
scenario in which you see a problem.

If an initiator reboots, and has no context information, how can it know
whether or not a target has a pre-existing session?  Since there is no nice
way to know that, I would probably code my initiator to request a login
with
the X bit set (but as Mallikarjun said, I don't like this overloading of
the
X bit, it's a special case and makes the coding extra convoluted).  In your
preferred scenario, this would cause the target to reject the login "cause
there is no pre-existing session", and the initiator would re-issues the
login without the X bit set.  What have you saved anyone from here?  You've
just added latency to the login process.  And either way the initiator
codes
it's login after reboot, there's a presumably equal probability of
encountering this extra exchange.

I still haven't seen a plausable example where it does harm to have a login
w/ ISID=n, TSID=0 close an existing session with this initiator.  I can see
no case where this would be the wrong decision.  If "this isn't what the
initiator intended", this is a defective initiator implementation and
closing the other session at least does no harm.

Marjorie Krueger

> Sent: Wednesday, August 29, 2001 10:46 AM
> To: [email-masked]

---

From: "KRUEGER,MARJORIE (HP-Roseville,ex1)" <[email-masked]>
Date: Wed, 29 Aug 2001 12:27:35 -0700

It would help to get your message thru if you could answer our requests for
an explanation of your thinking.  We have tried several times to explain our
logic (w/examples) but I haven't seen an example from you supporting a
scenario in which you see a problem.

If an initiator reboots, and has no context information, how can it know
whether or not a target has a pre-existing session?  Since there is no nice
way to know that, I would probably code my initiator to request a login with
the X bit set (but as Mallikarjun said, I don't like this overloading of the
X bit, it's a special case and makes the coding extra convoluted).  In your
preferred scenario, this would cause the target to reject the login "cause
there is no pre-existing session", and the initiator would re-issues the
login without the X bit set.  What have you saved anyone from here?  You've
just added latency to the login process.  And either way the initiator codes
it's login after reboot, there's a presumably equal probability of
encountering this extra exchange.

Marjorie Krueger

---

From: "Julian Satran" <[email-masked]>
Date: Thu, 30 Aug 2001 08:33:35 +0300

With  regard to time bloat I have two things to say:

1- It is a rare event (connection don't stay alive long if a machine goes
down)
2 - we could remove the checking - going only on reporting and have in
those cases the command reissued with X

As for the mistake I am afraid that anything that is so easy to do badly
will be done badly and causes can range from simple bugs, to management
servers that direct you to same target under two different names and the
target does not care to two user programs doing uncoordinated login (but
only one is bad).

I assume that are many scenarios that we don't even think about.

My long experience tells me not make it easy for a mistake to ruin good
work.

And I think that the proposal that is out (removing the checking and having
Marjory irritated) is a good compromise between simplicity a sound
engineering.

Julo

Santosh Rao <[email-masked]>@ece.cmu.edu on 30-08-2001 02:54:09

To:   [email-masked]
cc:

Julian Satran wrote:

> through.

A multi-OS system would use a different iSCSI initiator name for each O.S.

> After a reboot the right procedure would be to login cleanly (no

Are you suggesting a combination of the existing login target behaviour
wherein the target checks the olde session if it exists when the X bit is
not
set, coupled with an unconditional cleanup when the X bit is set ?

Having the target test the old session, prior to responding to a login can
bloat login time to unacceptable levels and increase system bootup time as
well as I/O scan time.

> but you don't know how to clean it.

The initiator may not know if an old session had been established by it
previously. Hence, it will either always need to set the X bit, or risk the
chance of a login reject.

What are we trying to achieve by having the target validate the initiator's
attempt to clean up an old session ? The target must trust the initiator's
decision to clean up [after first completing login authentication to avoid
malicious logout type attacks], rather than have to validate it first.
Targets
should not be expected to do all this extra work of validation to protect
against initiator coding bugs.

The draft should not be trying to optimize for a coding bug scenario,
wherein
the initiator accidentally re-logs in with the same (iscsi initiator name,
ISID, NULL TSID).

FC exhibits the same semantics that are being asked for here. Implicit
logout
+ re-login behaviour is oft used by FC initiators. Similar login semantics
will make the life of iscsi-fc convertor products easier as well.

> Reboot behavior is close to what you have in the current draft ( a single

On a reboot, the initiator does not know if it has previously logged out of
the session prior to the reboot. Thus, it would need to clean up any stale
sessions, if they exist. Rather than attempt 2 logins [one with an X and
then,
one without], the login should be allowed to imply a logout of any stale
session, if one such exists.

This issue is applicable to the reboot of initiators.

- Santosh

 - santoshr.vcf

---

From: "Jim Hafner" <[email-masked]>
Date: Thu, 30 Aug 2001 07:26:48 -0700

Venkat,
You wrote that optionA doesn't protect:
  where a rogue software on the initiator running in user
mode which simply issues ISID=existing, TSID=0 to the target.

But what's to prevent that same rogue software from setting the C bit?
It's the same exposure either way. In option A, the old session gets killed
regardless.  In option B the old session gets killed with C=1 (and that's
hardly any sort of extra hurdle you'd want to place on the rogue software).

You also wrote:
  It is a bad initiator in
that it doesn't respect the presence of other sessions at the initiator
system, but the protocol can't bloat handling these cases of bad initiators
.

That's the point exactly.  It's a "bad initiator" that carelessly reuses an
existing ISID (regardless of whether it has to set an extra bit or not).
We can't protect against that at the target.

Jim Hafner

Venkat Rangan <[email-masked]>@ece.cmu.edu on 08/29/2001
10:25:27 PM

To:   "'Stephen Bailey'" <[email-masked]>, Santosh Rao
      <[email-masked]>
cc:   [email-masked]
Subject:  RE: iSCSI: reusing ISID for recovery

Steph,

This is a long thread, almost as long as the thread "iSCSI: response to
second login (with same ISID)" we had a few months ago. One aspect from the
other thread that may not have been mentioned is the fact that you could
have a situation where a rogue software on the initiator running in user
mode which simply issues ISID=existing, TSID=0 to the target. (This may be
what was meant by "wild closing of connections" in this thread.) Choosing
option A) would destroy an otherwise well protected ISID managed in the
kernel space. This could also occur when you accidentally start another
application that does not coordinate its ISID selection.

The earlier thread was leaning towards "Reject the second login attempt
with
In-Use error code" but the issue of targets cleaning up inactive and failed
sessions still lingers on.

Some observations:

1. There is no way to protect against rogue software at the intiator; even
the best IPSec policies and key management can not prevent such software
from disturbing other well-behaved components. So trying to design
mechanisms to prevent this at the target would be fruitless.

2. There is some value in protecting an accidental use of an existing ISID
by a second client (ISID=<existing>, TSID=0). This can be done with a
Reject
of the login attempt, even without verification of login credentials -
(less
of a DOS attack). You also do not need to analyze connection states in this
case.

3. From the perspective of the Target, case 2) above is no different from
an
initiator rebooting and accidentally choosing an existing ISID with TSID=0.
To distinguish 2) from 3), we can make use of the Clear bit (or the C-bit)
and avoid many unnecessary trips at reboot time, trying to probe for an
unused ISID. The usage is that if C-bit is present, any previous sessions
are also closed by the target. I think this was Option B) in the original
email.

4. If initiators always set the C-bit, then they can't benefit from the
duplicate session identification capability of 2). It is a bad initiator in
that it doesn't respect the presence of other sessions at the initiator
system, but the protocol can't bloat handling these cases of bad
initiators.

5. If an initiator only implements mandatory session recovery, it can help
a
target recover faster by explicitly logging out the previous session and
use
same or different ISID with TSID=0 and an C-bit to really ensure that it
can
recover on a new session. This is a fast operation because we are not
waiting for lengthy round trips or timeouts, and the presence of the C-bit
makes the intent clear to the target that it must wipe off the existing
session. The perceived delay noted in Option C) does not arise.

6. If a target determines that all connections in a session are dead (TCP
keepalives can help here), and there is no activity on a session, it can
simply close the session and release the ISID and the TSID and recover
target side resources. This is not driven be explicit protocol action from
initiator, but simply target timer based, and is designed for initiators
that disappear without cleanup of sessions and don't reappear (no reboot
for
a long time).

With this limited understanding, my preference would be Option B). One
could
argue that B) offers protection only against an accidental use of the same
ISID, but if (as was pointed out earlier and in the other thread by Nick),
there are multiple ways in which iSCSI applications are packaged, so that
there is no shared visibility into ISID space, the protection offered by B)
can be useful.

Venkat Rangan
Rhapsody Networks Inc.
http://www.rhapsodynetworks.com

-----Original Message-----
From: Stephen Bailey [mailto:[email-masked]]
Sent: Wednesday, August 29, 2001 1:53 PM
To: Santosh Rao
Cc: [email-masked]
Subject: Re: iSCSI: reusing ISID for recovery

> The same was earlier proposed by myself in a previous thread on this
> issue a few months ago.

You're not the only one.  I'm also in favor of (A).  I have also
carefully laid out the chain that Marjorie's following on at least one
(and probably several) past occasions.  It seems sound to me.  An
initiator that has lost its mind (this might be an OS reboot, or
`simply' a fat adapter crash & restart) is going to want to stomp out
any existing sessions in its name.  It's not going to want to pick up
old context unless it's aware that there IS old context, in which case
it hasn't completely lost its mind.

I've been keeping my mouth shut in hopes that it would actually pass,
but I guess I don't have to articulate a position so much as just hold
it to ensure it gets killed.  Given my track record, I'm now for sale
to the highest bidder.

Steph

---

From: "Jim Hafner" <[email-masked]>
Date: Thu, 30 Aug 2001 07:29:26 -0700

Julian,

You wrote:
    to management servers that direct you to same target under two
different names

I didn't understand who has the different name, the target or the
initiator.  In either case, however, it doesn't matter because the ISID
scoping is within the *named* pair of (initiator,target), so that there
won't be a collision!

Jim Hafner

Julian Satran/Haifa/IBM@[email-masked] on 08/29/2001 10:33:35 PM

To:   [email-masked]
cc:

Julo

To:   [email-masked]
cc:

- Santosh

 - santoshr.vcf

---

From: "John Hufferd" <[email-masked]>
Date: Thu, 30 Aug 2001 12:47:31 -0700

Folks,
Following your notes, I think  that if an initiator does not know what it
is doing, rather then having a new semantic on Login, why not just Logout,
then re-Login.
Since this should only happen like "Never", if you have coded things well,
and the infrequency should mean that the extra hand shake is not a problem.

However,  if you do not know what your session's TSID is, you probably are
two screwed up to make even the above Logout work, so I think this is what
you are all trying to address.  That is,  we  need to have a method to blow
everything away and start again when the initiator has lost the TSID of its
Session.  This is why we have option A, and Option B.

So I think the issue here is do we add additional responses at boot up
time, or not.  (Quit frankly I do not think this is an important issue,
either way.) However, it is a consideration.  If mode B is used, the clean
up bit is available, and initiators that use it all the time, will usually
cause the Extra interaction to occur at boot, because the normal case would
be that things come up clean.  (Since I think we said that if things are
clean, an error response would be issued. Therefore, most of the time, the
extra response is going to happen).  If, as some have suggested, the right
way to code this is to use  the Clean up bit only when there is a rejection
of the Login, because of existing sessions, and require that the initiator
should come back with the Login again but this time with the Clean up bit
set. The argument is made that there will not normally  be any extra
interaction.  Just extra interactions when clean-up is needed. (This
approach has been called Option B.)  The other approach is known as Option
A.  It will implicitly start the Login and clear out any existing session
when the Login is issued.   There will never be extra responses with Option
A, and there will only be extra responses in option B when the Target
rejects the Login because of a existing session. The Option B clean-up
action will not happen often, and the code is not hard, but there is more
code, perhaps 100 lines.

Now the folks that support option A say that they will not have to do
anything special since they want the Login to always succeed and blow away
any other thing that might be going on.  It is clear that this has less
code and interaction at boot/Startup. (However 100 lines of code is not a
big deal anyway.)
It also seem to me that there is probably less code on the target since
they do not have to do inspection of whether or not sessions are active or
not, etc. They just blow them away if they exist.

So all this flapping has to do with whether we need to add (on either side)
some (perhaps small) amount of code so that we use to protect from a buggy
Initiator.

I think this is a Nit in the Draft and is needlessly divisive, over a small
amount of code.

There has also been stated the opinion that option B is needed when their
is more then one OS sharing the Link.  I do not think this applies since
there should be another iSCSI Initiator Name for each OS, and therefore it
is a different session and has nothing to do with the discussion at hand.

Therefore, since the simplification (less code) is on the side of option A,
and since this is really not important anyway, I suggest that all the B
folks roll over since you will not be able to convince the A folks anyway,
and since they do have a valid (small) argument that Option B requires more
complexity and lines of code.

So I recommend Option A, and lets move on.  This does not deserver the list
space, or the emotion  it has been given.

.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
Home Office (408) 997-6136
Internet address: [email-masked]

---

From: "KRUEGER,MARJORIE (HP-Roseville,ex1)" <[email-masked]>
Date: Thu, 30 Aug 2001 17:00:10 -0400

Thanks John.  I'm not concerned about the amount of code.  The sticking
point for option B is the delay in initiator boot time for what I feel is
unnecessary protection against software defects (IOTW, adds delay to boot to
inform initiator of existing session, which initiator wants cleaned up
anyway).  I recall very ugly boot times in FC testing because of delay in
target response to initiator login that would cause some OS (MS!) to give up
and time out the target device at great expense to total SAN recovery times.

Marj 

> From: John Hufferd [mailto:[email-masked]]
> Sent: Thursday, August 30, 2001 12:48 PM
> To: [email-masked]