Subject: Re: iSCSI: multiple sessions b/n a pair of WWUIs.

From: Santosh Rao <[email-masked]>
Date: Thu, 03 May 2001 17:19:27 -0700

Matt Wakeley wrote:
> 
> One clarification: these are unique between any I-T *pair*.
> (you make it sound like an initiator must have a unique ISID for all sessions
> with all targets)
> 

Matt,

While it is true that uniqueness is strictly required only b/n a given
I-T pair, using the ISID as a unique key for all the sessions an
initiator establishes allows the O.S. to use this session id as a port
identifier within the node space. This is a useful utilization of this
construct rather than inventing yet another unique token to represent
each target within the node name space for targets.

Regards,
Santosh

> -Matt
> 
> Santosh Rao wrote:
> >
> > Jim,
> >
> > Thanks for the clarification. Both the iSCSI and the name-disc drafts
> > need to explicitly state that ISID is uniquely assigned for all sessions
> > within a given initiator. Similarly, TSID is uniquely assigned for all
> > sessions within a given target.
> >
> > Regards,
> > Santosh

---

From: "KRUEGER,MARJORIE (HP-Roseville,ex1)" <[email-masked]>
Date: Fri, 4 May 2001 17:39:35 -0700

> (you make it sound like an initiator must have a unique ISID 

The ISID must be unique within the iSCSI layer on an iSCSI device - that
amounts to "initiator must have unique ISID for all sessions with all
targets".  

To quote Jim "The requirement in name+disc that a given initiator name
cannot reuse an ISID for two different sessions comes as a consequence a
number of things (which are described in the draft).  The gist is that this
is needed to provide the correct context for restoration of reservations
state (and other nexus state) to a particular nexus after logout/login. In
other words, if the session goes down for some reason, the target needs
clear context to restore nexus state to a rebuilt session. The only tool it
has is uniqueness of Name+ISID combination within its name space."

-Marj

---

From: [email-masked]
Date: Mon, 7 May 2001 22:52:08 -0400

While Jim's correct ... in Marj's defense, what
she wrote is a fairly obvious way to implement
it, and that might be worth noting in the draft.

--David

---------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 42 South St., Hopkinton, MA  01748
+1 (508) 435-1000 x75140     FAX: +1 (508) 497-8500
[email-masked]       Mobile: +1 (978) 394-7754
---------------------------------------------------

> -----Original Message-----
> From:	Jim Hafner [SMTP:[email-masked]]
> Sent:	Monday, May 07, 2001 10:44 PM
> To:	KRUEGER,MARJORIE (HP-Roseville,ex1)
> Cc:	[email-masked]
> Subject:	RE: iSCSI: multiple sessions b/n a pair of WWUIs.
> 
> 
> Marj,
> 
> I think you've stretched my words too far.  The requirement is for a
> single
> logical unit to recover enough identity information provided through the
> I_T_L nexus to reestablish the reservation.  That means that the name+ISID
> must be unique *from the viewpoint of a single iSCSI target device*.  So
> it's more like the
> "initiator must have unique ISID for all sessions with <delete>all
> targets</delete> any given target device".
> 
> The best way to think about this is the following:
> 1) only a single target's (actually logical unit's) view of the initiator
> counts in this regard
> 2) there are no rules in SCSI concerning multi-target device interactions
> In other words, for these rules, look only from inside a target going out.
> 
> Jim Hafner
> 
> 
> "KRUEGER,MARJORIE (HP-Roseville,ex1)"
> <[email-masked]>@ece.cmu.edu
> on 05/04/2001 05:39:35 PM
> 
> Sent by:  [email-masked]
> 
> 
> To:   "'Matt Wakeley'" <[email-masked]>, [email-masked]
> cc:
> Subject:  RE: iSCSI: multiple sessions b/n a pair of WWUIs.
> 
> 
>

---

From: "John Hufferd" <[email-masked]>
Date: Tue, 8 May 2001 20:08:59 -0700

Santosh,
In the attached you are getting you wishes in the way of what is Required.
The Hard requirement is that the iSCSI Initiator Port have a ISID that is
unique to a specific target Node (actually Target iSCSI Port).  It is nice
that you want to used it (the ISID in the way you suggest, however, there
may be several iSCSI Initiator Ports on any iSCSI Initiator Node and that
means that you must find a way to allocate out the ISID value range to each
iSCSI Initiator  Port.   Quite frankly I believe you should probably should
use another indexing method, and let a single ISID be used by each iSCSI
Initiator Port.

However, I am getting the feeling that we are talking past each other, and
some how not communicating.  Not sure if it is my fault or not.

.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
(408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
Internet address: [email-masked]

Santosh Rao <[email-masked]>@ece.cmu.edu on 05/08/2001 11:03:27 AM

To:   [email-masked]
cc:
Subject:  Re: iSCSI: multiple sessions b/n a pair of WWUIs.

Douglas Otis wrote:

>  The usefulness of ISID is also
> diminished if the client is not expected to maintain this value as unique
> across all Targets.

I agree with the above and have stated to similar effect in my response
to Matt. A practical usage of ISID within an initiator node name space
is to assign it as unique across all targets, allowing the same ISID to
be used as a lookup key of the session for any given target port/node.

Not applying uniqueness in ISID assignment across all targets would then
require initiators to implement a second key mechanism for target
lookup.

- Santosh

---

From: Santosh Rao <[email-masked]>
Date: Wed, 09 May 2001 11:29:49 -0700

Jim,

Agreed on the ISID issue. One additional comment below.

Regards,
Santosh

Jim Hafner wrote:

> The spec might make a note that says that multiple sessions
> handlers within a given iSCSI Node sharing a common iSCSI Name should
> coordinate their use of ISIDs (carve the name space between them) to avoid
> rejected login by one session handler because it reused an ISID already in
> use by another session handler. 

As I've brought up before, login rejection when a target receives a
second session login from the same (iSCSI Initiator Name, ISID) when the
first session login is still active may not be a good option.

This scenario can typically occur when the host rebooted without having
done a logout and then attempts to re-login. The host must be allowed to
maintain persistence of its ISID without being rejected in its login
attempt.

I'd suggest something along the following lines :
" When the target receives a second session login while one is already
active for a given
(iSCSI Initiator Name, ISID), it MUST treat this as an implicit logout
of the prior session and then accept the second login". [provided the
login authentication was successful].

I believe this is also what Spteh Bailey had in mind when writing up
algorithms for ERT (?). Do you see any issues with such an approach. (?)

---

From: "Jim Hafner" <[email-masked]>
Date: Wed, 9 May 2001 11:45:51 -0700

Santosh,

Comments in line <JLH> ... </JLH>

Jim Hafner

Santosh Rao <[email-masked]>@cup.hp.com on 05-09-2001 08:54:54 AM

Sent by:  [email-masked]

To:   Jim Hafner/Almaden/IBM@IBMUS
cc:   [email-masked]
Subject:  Re: iSCSI: multiple sessions b/n a pair of WWUIs.

Jim,

I agree that the only requirement is uniqueness of ISID among all the
sessions to a given target node. Maintaining uniqueness across all
targets only extends compliance to this requirement a step further and
does not violate it.

<JLH> Correct, it does not violate it and that's all that matters.  Do this
if you want, it's your implementation :-{) </JLH>

All we are debating is David's suggestion that a note could be added to
indicate that implementations may choose to apply this uniqueness across
all target nodes so as to use the ISID as a target lookup mechanism.

<JLH> I'm beginning to believe strongly that such a note is NOT a good
thing as it implies a preferred implementation and (a) that's not the job
of the spec and (b) we haven't hashed out all the possible issues to be
sure we would be recommending a "best" implementation (and such hashing is
NOT the job of this reflector). </JLH>

If there is strong resistance against adding such a note, then, let us
omit it. [as long as the spec does'nt preclude usage of ISID as unique
across all targets].

<JLH> As I said, I'm moving rapidly to "strong" against.   I think the spec
(and any related document or note) currently does not say (and agreeably
shouldn't say) that the ISIDs must be reused in any particular way.  That
option would be simply falling on the other side of the implementation
fence.  The spec might make a note that says that multiple sessions
handlers within a given iSCSI Node sharing a common iSCSI Name should
coordinate their use of ISIDs (carve the name space between them) to avoid
rejected login by one session handler because it reused an ISID already in
use by another session handler.  But that's about all it should say.</JLH>

Regards,
Santosh

Jim Hafner wrote:
>
> Santosh,
>
> As I've said, I agree with the sentiment, but that is not the only
> implementation option.  For example, why can't the database index on the
> pointer to the data structure that has the names, etc, or hash on the
names
> or a million other things.  We shouldn't spec implementations, only
> interoperability requirements.
>
> Jim Hafner
>
> Santosh Rao <[email-masked]>@ece.cmu.edu on 05-08-2001 11:03:27 AM
>

---

From: "John Hufferd" <[email-masked]>
Date: Wed, 9 May 2001 18:03:38 -0700

Santosh,
Notes in text below between [Huff] and [/Huff].
.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
(408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
Internet address: [email-masked]

Santosh Rao <[email-masked]>@ece.cmu.edu on 05/09/2001 10:41:34 AM

To:   John Hufferd/San Jose/IBM@IBMUS
cc:   [email-masked], [email-masked]
Subject:  Re: iSCSI: multiple sessions b/n a pair of WWUIs.

John Hufferd wrote:
>
> Santosh,
> I think I am beginning to see the problem.  A given iSCSI Initiator Port
> can NOT have a second session with the same iSCSI Initiator Port and be

John,
                                             ^^^^^^^^^^^^^^^^^^^^^
Is the above "same iSCSI target port" (?).

[Huff]  correct, it was a Type-O, or Brain-check.[/Huff]

> consistent with SCSIness.  The second session could be started with
another
> ISID, (to the same iSCSI Target Port) but if the session was established
it
> would not have any of its commands and data handled via any techniques
> defined by SCSI or iSCSI.  In fact its use would require a wedge driver.

Perhaps, some of my confusion arises from a lack of understanding of
your definition of iSCSI Initiator Port and iSCSI Target Port. Some
clarification on your definition of these would help make things more
clear.

[Huff] In our Naming and Discovery Document the thing I am calling iSCSI
Initiator Port was shown as the SCSI Initiator Port.  It is the thing that
creates the sessions that talk to the Target Devices,  the Target Device
must logically receive the connection and assign it to what I was calling
an iSCSI Target Port, which in the  N&D document was shown as the SCSI
Port.

It is the (i)SCSI Initiator Port that creates Sessions and in our iSCSI
model it does this through the LOGIN command sending the iSCSI Initiator
Node Name and an approprate ISID.  The Target side's SCSI port, in our
iSCSI model exchanges its iSCSI Target Node Name and an approprate TSID.
(And Jim has been talking about how the (i)SCSI Target Port  should keep
persistent reservation information, including the ISID & TSIDs,  and use
that information to re-establish the Session and assign to the Session the
approprate Reservations that might be out standing for the approprate
(i)SCSI Initiator Port -- This port is identified by the iSCSI Initiator
Node Name and the ISID, under which the Reservation was taken.)[/Huff]

> This is the exact confusion I was trying to avoid.  Technically if you
were
> actually able to start another Session to the same Target Port, it would
> be, by definition another iSCSI Initiator Port.   The two different Ports
> would need to be coordinated by what we have been calling a Wedge Driver.

I'm not sure why you believe a wedge driver would be required to handle
such a configuration.(?) Per my understanding [which may be missing some
aspect], the host can operate in such a configuration in the absence of
a wedge driver.

Also, the host could create such multiple sessions to the same target
port in order to separate sub-sets of LUNs into different sessions and
apply different session properties to each such LUN subset.

[Huff] As for Wedge Drivers, this is a NON SCSI process that is either
Dynamic and therefore in the path to the Port, or is Static and performs
setup before the I/O is done (such as isolation of one set of LUNs from
others even though they are on the same Target).   But some one has to do
something to prevent the LUN I/O from being spread across different
sessions, such that order can not be assured.  The best performing Wedge
Drivers are those that  can send the I/O down any Session, but will insure
that it will not send any LUN I/O down another Session as long as the LUN
has I/O out standing.  However, if no I./O is outstanding to the LUN it can
balance the I/O across the different session.  So in my terms, your wedge
driver either separate the LUN I/O ahead of time statically, or it does it
dynamically (or both).  The point to remember is:  if Multiple Connections
per Sessions (SC/S) can be used, there is no Wedge Driver requirement and
the code is already in place to balance I/O across connections.   Since
SC/S is documented by the protocol, we should probably give preference in
the document to those design approaches which use the protocol to its best
advantage.   [/Huff]

>
> Today, the way we use multiple Fibre Channel Initiator Ports connected to
> the same Fibre Channel Target Ports is by use of Wedge Drivers.

With FC, connecting multiple initiator ports from the *same* host to a
single target port makes little or no sense. The host would see the same
LUNs twice through different initiator ports.

[Huff] In fact this is a frequent type of connection used by large servers,
and this is where the Wedge drivers come into action [/Huff]

A more typical configuration is to connect multiple FC initiator ports
from *different* hosts to the same target port as a high availability
configuration. Yes, in such a case, some type of cluster application
would be managing the switch-over from one initiator port to another.
(Is this the case being referred to ?).

[Huff] This is not what I was talking about, this is the typical type of
connections with Shared File Systems. and that does not need a Wedge
Driver, since the Shared File System (or Database) protects its self in
such a cluster environment.  Also no application can assume any
inter-arrival  of I/O with respect to the other system unless it does its
own coordination.  This is NOT a Wedge Driver. [/Huff]

> I think
> what you are suggesting causes the need for a Wedge Drivers to integrate
> the iSCSI Initiator Ports.

(I guess we wandered off into a different line of discussion which is
"whether multiple sessions between an initiator port and target port are
legal and required" ? :)

My answer to that would be, yes, they are legal and required. They also
do not require the use of a wedge driver. The host may configure
multiple sessions from one iSCSI initiator NIC to a given iSCSI
TargetAddress so as to access LUN subsets from different sessions, and
apply different session properties to different LUN sub-sets.

[Huff] The point is their sematicts are NOT defined by SCSI, you need what
I was calling a Wedge Driver either dynamic or Staticly, and I think you
are saying the same thing.  Yes, it can be done, but the deffinition of
what happens is NOT Standarized. [/Huff]

Such multiple sessions that are originated on the same initiator NIC
need to use different ISIDs [for all of the sessions for that given
target node] and thereby, are considered multiple iSCSI Initiator Ports,
based on the current name-disc draft's defn.

[Huff] Correct, we are back on the same page now. [/Huff]

 Not sure that we want to cause this type of
> thinking, by accident.  One of the reasons for Multiple Connections per
> Session was to remove the "Hard" need for Wedge Drivers.

IMHO, "multiple connections per session" complements wedge drivers in
their functionality but may not comletely replace them. A lot of wedge
drivers have code specific to the SCSI properties of the target and
moving all of this functionality into the iSCSI layer would imply adding
device specific code into this layer. Not the cleanest of layering
schemes.

[Huff] Correct again, there are some dynamic wedge drivers that not only
balance workload but also spring into action to help with fail over
situations.  Also the wedge drivers sometimes are needed to ensure that for
the best use of the target Raid Array Caches, by assuring that the various
LUN traffic is isolated/fixed to one connection or another,until fail over
at the target.  These types of storage controllers will probably continue
to need Wedge Drivers, but that makes it very hard to come up with generic
Desktop and Laptop Initiators that can efficiently use those types of RAID
Arrays. IMHO it is unlikely that Microsoft will ship various vendor
specific Wedge Drivers, so it will be up to the RAID manufacture, and that
is always problematical in large installations.  [/Huff]

Also, some wedge drivers claim to utilize device proprietary features,
which a device independent iSCSI initiator layer could not provide
through "multiple connections per session".

[Huff] And again Correct, as I agreed above.  However, it would be very
good if the vendors that are making the various generic iSCSI initiators,
think through how they will handle the Single Connection case with one of
the above mentioned RAID Array Controllers, so that upon failure of one
side of a duplexed Raid Storage Controller, the Session can be restarted
from the initiator to the surviving side of the Raid Controller (which may
have a different TCP/IP address, or have taken over the failing IP
address).  If a clean new session is started, the target  still needs to
handle Persistent Reserves, and if the surviving Storage controller board
actually takes over the IP address, then I am not sure what we should be
doing since the surviving side may or may not have approprate state to
continue the session as if nothing had happen.  (I would be interested in
opinions here.)

In any event, as I suspected, we are in basic agreement, but were talking
past each other.  So the question is should David's suggestion about
including, in the Protocol Document, the concept of multiple ISIDs per
(i)SCSI Port.   My opinion is not, since it would probably cause a whole
lot of  discussion about Wedge Drivers, etc., and I do not think that would
be approprate in the iSCSI spec. [/Huff]

Regards,
Santosh