Subject: iSCSI: ISID and the New Persistent Reservations Proposals

From: "John Hufferd" <[email-masked]>
Date: Thu, 4 Oct 2001 17:51:59 -0700

David,
I have spent some time stepping through the issues of Target Generation of
SSID or ISID along with Persistent Reserves.  I will start with Persistent
Reserves and then move to the SSID/ISID issue.

I have tried to lay it out persistent reserves so that everyone can follow
it (may not work out but that is the attempt), I have also reviewed this
with Jim Hafner, so unless I have some typos I think it is correct.

Persistent Reserves have three new proposed versions: (in each of the
following assume that the Initiator Node has 2 Ports A, and B, and the
Target Node has 2 Ports X, and Y. Also the subject LUN is called LU5).  I
will start with point #0 that represents Persistent Reservations as of
today, and then step through the 3 new options.

0. Today, Initiator Port A, can create a Persistent Reservation for LU5
with Target Port X, and then only I/O to LU5 can occur via the path from
Initiator port A to Target Port X, all other I/Os to LU5 will be blocked.
That is LU5 I/O via A to Y will be blocked as will B to X and B to Y.

Three New additional proposals for handling Persistent reservations are
being pushed by folks in T10:
1. Persistent reservations to LU5 can be made via Port A, and that will
apply on the A to X path and the A to Y path.   I/O to LU5 from Port B to
either X or Y will be blocked.
2. Persistent reservation to LU5 made via Port A  or Port B to Target Port
X, will permit  I/O to LU5 via Port A or B, if sent to Target Port X.
However, I/O to LU5 via Initiator Ports A or Port B will be blocked if it
is sent to Target Port Y.
3. Persistent reservation to LU5 made via Initiator Port A or Port B to
Target Ports X or Y may occur, and I/O to LU5 may then occur via any path A
to X, A to Y, B to X, or B to Y.

In both todays model (#0) or in proposal #1 the Target needs to place the
Persistent  reservation against the Full iSCSI Initiator Port Name.  This
is the iSCSI Initiator Node Name Plus the ISID.  In proposal #1 the Full
iSCSI Initiator Port Name is then made known to all the target ports (in
the same SCSI Device), and that name is then used to determine the
Persistent Reservation State for each of the Target Ports relative to the
subject LU.

In proposal 2 and 3, above,  the name to be used in the Reservations is
just the iSCSI Initiator Node Name (No ISID).  In proposal #2 the Name is
not shared with the other Target Ports (in that SCSI Device) so Persistence
only apples to I/O arriving at that Target Port from any Initiator port on
the Initiator Node which made the Reservation.  And in proposal 3 the iSCSI
Initiator Node Name is shared among the Target Ports so that Persistence
can be applied to any Target Port (in that SCSI Device), which is addressed
by any Port on the Initiator Node which made the Reservation.

Now, with regards to the issue that has been kicking around about the
target assigning the SSID,  it does not matter one way or the other with
today's position #0 or proposal numbers 2 or 3. where the SSID comes from.
However, proposal #1 needs a unique ISID, and can not use a dynamically
created SSID generated by each target Port.  There are other reasons why
folks do not even want the ISID, or the SSID generated by the target, but
those reasons have nothing to do with these Persistent Reservation
proposals.

I have been suggesting to the N &D team and others that if we limit the
Dynamic Target Allocation to only the ISID (and of course the TSID) but not
a complete SSID, that there may be a workable solution.  I say ISID, since
I believe it must be assured uniqueness per Initiator Port.  One reason can
be seen in #0, & #1 above.

Now suppose that one of the Target's Job was to Generate a unique ISID for
each Initiator Port, within and Initiator Node, which wanted to go into
session.  If you combine with that, the rule that if the Initiator Port
sets and sends its own ISID  on the Login the Target will accept it, and
then use the Full Initiator Port Name to check for Persistent Reserves, I
think we approach a working solution.

Now you also have to ensure that upon a Target Boot, (which has to remember
the Persistent Reservation Status), it needs to also remember the high
water mark of ISIDs given out to each Initiator Node.  With that high water
mark the Target can continue assigning the ISIDs for that Initiator Node.
Then I think your words were do not aggressively reused ISIDs.  I think
that if the Target has not gone down for a day or so, ISIDs not in use
might be re-used, etc.
We have 65K valid values per Initiator Node so reuse should not be a
problem.

OK now lets review what we have said.  There is a way for a very simple
routine on the Target to assign the ISIDs.  If Sent an ISID in the Login it
should accept it (if it can), and check on persistent reserves of type #0,
or #1.  The Initiator can save the ISID, in case it goes down and want to
come back up, and reclaim the reserves, or it can blow the reserves away
and start again with a Target assigned ISID.

OK now in one of your last notes, I think you said a similar thing.  This
is also what I attempted to explain to the N &D team.

So anyway, as good engineers we have been able to come up with something
that works.  The problem that the N & D team had is, so what.  What we have
now works, and has a smaller impact on the Target.

So if it works we need to decide if we should do it or NOT, and focus the
conversation on that.

I had sent a note in previously that explained that if we have a way to set
the iSCSI Initiator Node Name then we must have a way to set the ISID by
the Initiator Node.  Therefore, the question goes, why get the Target
involved.

Here is what I said in another note:

If you accept that HBAs should get the iSCSI Initiator Node Name passed to
it in some manner from the OS, then we did not see why it can not get the
ISID at the same time also.  If one were to argue that the HBA can not get
the iSCSI Initiator Node Name, I think this is a bit strange, since almost
every thing I install in systems, has to go through some type of Wizard, or
vendor install routine  that actually installs the support driver, etc.
(This is all set when the OS is fully up and the Registry etc. is clearly
available.)  I would expect that with current OSs, this would be a normal
approach.  In the future, I suppose it might be possible that  the OSs
might have a more dynamic routine to hand out the iSCSI Initiator Node Name
and ISID so that Hot Swap HBAs could come up without effort.  However,
until then, I am not sure it is unreasonable for a vendor to require the
customer to run an update routine,  even after performing an HBA Hot Swap.

There is of course, the first up boot situation.  I think it should be
possible to have a default, first time HBA boot Name that uses the iqn form
of the iSCSI Name, and an ISID of FFFF (not 0 as said previously).  This
should be enough to bring up a "first time up boot", since after the OS is
started, the approprate install routine should be kicked off, and the real
iSCSI Initiator Node Name and real ISID set in the Flash for that, and any
other iSCSI HBAs.

By the way I have a SCSI HBA in my personal system that runs its own BIOS
routine and when it comes up, but before it boots, it asks questions if
needed or just announces it is there.  Configuration value can be entered
at that time.  So even that approach is possible.

So based on the above, I do not think requiring the HBAs to have an
install/update routine (which sets its Flash) is even a problem.

This means that a Single iSCSI Initiator Node Name should be easy to obtain
and will apply for all the iSCSI Initiators, whether HW or SW. And likewise
it should be straight forward to secure the ISID at the same time.  All
this following well understood current install processes.

Therefore, additional routines on the Target for handing out ISIDs should
not be needed.

OK, I tried to bring all the ISID issues together here so we could all
examine them.  I hope this was useful.

.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
Home Office (408) 997-6136
Internet address: [email-masked]

---

From: [email-masked]
Date: Fri, 5 Oct 2001 10:43:04 -0400

John,

Thanks, that's a useful summary.  There are some important points to add.

> created SSID generated by each target Port.

Or it needs an iSCSI text key to serve the port naming function of the ISID
- the reason to do this rather than the ISID is that it cleanly splits
persistent reservation management away from iSCSI session management.
This has a couple of advantages:
- It allows us to exactly accommodate whatever T10 does by defining the
	semantics of the text key appropriately.  We won't have the same
	degree of flexibility in redefining ISID behavior.
- It allows us to resolve the "blow it away" issue caused by ISID reuse
	without having to trade off aspects of that resolution against
	consequences for proposal #1, whose status in T10 is unknown.
I would prefer deferring this whole issue of "New Persistent Reservation
Proposals" until T10's direction is clear, and in the interim advise
T10 that proposal #1 is a poor fit to iSCSI.

As I indicated earlier, ISIDs in their current form aren't sufficient
to deal with proposal #1 - while the specification will map onto iSCSI,
the current specification of ISIDs will not ensure that the added
functionality will work as intended.  We would have to significantly
expand the ISID rule to do so (that expansion will be problematic),
and hence if proposal #1 is adopted, we may wind up with not only
the ISID, but also a text key to deal with the resulting mismatch.
This is one more reason to stop trying to design in anticipation of
what T10 may or may not do.

problem.

This is actually a "SHOULD", not a "MUST", because a properly coded
Initiator
can figure out what's going on even if TSIDs (ISIDs in the above paragraph)
are reused with some care.  I'll spare everyone the details, as they're a
bit
subtle and deal with a situation in which the Target has already made a
mistake
(lost a persistent reserve), and hence it's reasonable to wonder what else
the Target will get wrong.  We agree that it's better to avoid depending on
completely correct behavior in this sort of error situation.

> now works, and has a smaller impact on the Target.

I'm not sure that what we have now "works" courtesy of the Option A/Option B
issue on when a Target is supposed to blow away a previous session.  The
current requirement that the Initiator specify the ISID, along with the
expectation that ISID values will be aggressively reused is the direct cause
of that problem.  The best solution I've seen to it is to remove ISIDs,
yielding a clear distinction between an Initiator who specifies a TSID
(wants its old state back) vs. sends 0 as the TSID (wants a brand new
session).

I also take issue with the "smaller impact on the Target" statement
above -- it may be based on the incorrect assumption that Targets would
have to manage ISIDs in addition to TSIDs.  ISIDs can be removed in their
entirety, and TSID management would continue to work in essentially the
current fashion.  There's been at least one statement on the list that
having Targets completely control the allocation of identifiers used to
manage target resources (e.g., persistent reserves) simplifies Targets.

> the Initiator Node.  Therefore, the question goes, why get the Target

Wrong question, IMHO.  Try "If TSIDs already solve the problem, why get the
Initiator involved?" :-).  Those who want to keep the ISID need to defend
the additional complexity that it adds to the protocol, rather than
uniformly
criticize any proposed change as disruptive, IMHO.

As to the statement that the ISID can be configured to an HBA in the same
manner as the iSCSI Initiator Node Name, I observe that fewer things to
configure lead to fewer opportunities to make mistakes and better scaling
of management.

> Therefore, additional routines on the Target for handing out ISIDs should

I agree, just get rid of ISIDs entirely :-).

--David

> -----Original Message-----
> From: John Hufferd [mailto:[email-masked]]
> Sent: Thursday, October 04, 2001 8:52 PM
> To: [email-masked]
> Cc: [email-masked]
> Subject: iSCSI: ISID and the New Persistent Reservations Proposals
> 
> 
> David,
problem.

---

From: "Jim Hafner" <[email-masked]>
Date: Fri, 5 Oct 2001 08:34:36 -0700

David,

I think I'm finally coming around to understand the full nature of what
you're proposing.

However, I'm not comfortable with it, though I have no more strong
technical arguments against it.

My strongest discomfort is this:  in essence you are removing the SCSI
Initiator Port Name from the iSCSI implementation of the model.  You're
hoping to fill that gap later with a text key whose semantics have yet to
be defined.  My expectation is that once you attempt to put the name back
in you're recreated the OptionA/OptionB debate and will never be able to
solve that satisfactorially.   I'm concerned that removing the notion of
SCSI Initiator Port Name from the model today, you will have strongly
hindered iSCSI's ability to take advantage of a number of things that T10
will be able to do,  now that Names are part of the lexicon.   I also
believe as I've stated that the OS's are going to prefer to see a more
stable SCSI port view of the initiator's ports than we're currently
anticipating in iSCSI.  Finally, I think
   - the reason to do this rather than the ISID is that it cleanly splits
   persistent reservation management away from iSCSI session management.
is a major violation of layering.  You're going to burden the reservation
management with having to coordinate iSCSI actions.  [I've argued that
conservative reuse of ISIDs to different target portal groups provides a
clean consistent view of SCSI Initiator Ports to the upper layers and
that's all that's needed.]

None of those are strong technical arguments however.

I think we agree that the problem comes up because the belief that
coordination of ISIDs on the initiator side is going to be difficult.  I'm
not convinced that's the case, but let me try a different approach, which
may make the implementation easier.

Let's postulate two things (a) that initiator portal groups can get their
iSCSI Name from the OS (without that, the issue is moot because theyn each
portal group would have its own name and its management of ISIDs) and (b)
that the initiator portal groups can be enumerated by the OS (as they are
enabled/discovered/installed/etc.) (/dev/iscsi0,...).  In other words, each
initiator portal group would have access to the iSCSI Name and it's portal
group tag.  [You outlined a similar thing on the target side.]

We have two choices:
(a) embed the initiator portal group tag in the ISID (that provides
effective partitioning without management intervention).
(b) put the initiator portal group tag in a text key (then I don't need to
partition ISIDs, as each will be qualified by the tag.

These are semantically equivalent (it's just as if the ISID field got
bigger in (b)).  The only reason to choose option (b) over (a) then is if
the ISID field (2bytes) isn't big enough.

You've suggested that (a) is a "preferred implementation" for the target
side.  Is there any reason that can't be done in the same way on the
initiator side? All devices get enumerated in some form or another in every
OS.  We can then take advantage of that inherent enumeration.

Some final points.

You wrote:
  As I indicated earlier, ISIDs in their current form aren't sufficient
  to deal with proposal #1 - while the specification will map onto iSCSI,
  the current specification of ISIDs will not ensure that the added
  functionality will work as intended.  We would have to significantly
  expand the ISID rule to do so (that expansion will be problematic),
  and hence if proposal #1 is adopted, we may wind up with not only
  the ISID, but also a text key to deal with the resulting mismatch.
I don't understand these claims still.   Why aren't ISIDs sufficient with
conservative reuse (as noted above).  What other expansion to ISID rule do
you foresee?  As I've said, there must be a rule that prevents parallel I_T
nexus and that will have to be based on the SCSI Initiator Port name
(however that is defined).  If you don't use the ISID as part of the name
but a key (or both), then you'll get a new rule with exactly the same
language, just replacing ISID with "key" or "key + ISID".
If you don't provide a SCSI Initiator Port name, you will never be able to
enable proposal #1 (and you'll look an awful lot like parallel SCSI in
functionality).

I'd like to hear from some of the other T10 folks on this list, if any have
opinions on this issue.  After that, I'd suggest we call concensus as I've
run my fingers ragged (and my brain as well).

Jim Hafner

[email-masked]@ece.cmu.edu on 10/05/2001 07:43:04 am

Sent by:  [email-masked]

To:   John Hufferd/San Jose/IBM@IBMUS, [email-masked]
cc:
Subject:  RE: iSCSI: ISID and the New Persistent Reservations Proposals

John,

problem.

> now works, and has a smaller impact on the Target.

I'm not sure that what we have now "works" courtesy of the Option A/Option
B
issue on when a Target is supposed to blow away a previous session.  The
current requirement that the Initiator specify the ISID, along with the
expectation that ISID values will be aggressively reused is the direct
cause
of that problem.  The best solution I've seen to it is to remove ISIDs,
yielding a clear distinction between an Initiator who specifies a TSID
(wants its old state back) vs. sends 0 as the TSID (wants a brand new
session).

--David

problem.

---

From: [email-masked]
Date: Fri, 5 Oct 2001 21:52:59 -0400

Jim,

> be defined.

Having the text key semantics be "yet to be defined" seems reasonable
because the behavior required by T10 is in the same state.  Horse first,
then cart :-).

> My expectation is that once you attempt to put the name back
> in you're recreated the OptionA/OptionB debate and will never 
> be able to solve that satisfactorially.

I disagree.  ISIDs aren't negotiable, text keys are.  In the text key
situation analogous to the ISID situation that causes the OptionA/OptionB
mess, the Target can set the text key to some sort of "Port Identity
Conflict Detected" value and ship everything back to the Initiator
with no harm done.  Trying to do that sort of thing with ISIDs would
be a mess, as indicated by the amount of email on OptionA/OptionB.

> I'm concerned that removing the notion of SCSI Initiator Port Name
> from the model today, you will have strongly hindered iSCSI's ability
> to take advantage of a number of things that T10
> will be able to do, now that Names are part of the lexicon.

And I have a strong discomfort with designing for functionality that isn't
specified.  We already have a mismatch where the current T10 proposal #1
does not match current ISID specification (see below on conservative reuse).
As T10 does things with the Port Name, we can put in the text key(s)
required
to make them work.

> I also believe as I've stated that the OS's are going to prefer to see
> a more stable SCSI port view of the initiator's ports than we're currently
> anticipating in iSCSI.

As long as the TSID is not used to provide this view, there is no problem
here (e.g., the enumeration of interface cards on boot can be used).

> Finally, I think
> management with having to coordinate iSCSI actions.

I disagree. The interaction required is something that will have to happen
anyway if T10 adopts any proposal that has reservations span sessions at the
Target.  Suppose that port X opens a session to port A, places a persistent
reservation that spans target ports, and then opens a session to port B -
as part of setting up that second session, port B has to interact with the
persistent reservation logic to discover that the reservation exists and
applies to the second session.  This is all that's required, and it's true
for any transport - in each cases the transport-specific port identification
has to be passed from the transport to the SCSI reservation logic.  The
only wrinkle is that iSCSI gets this name from a text key instead of a WWN
or a parallel SCSI bus ID - at the SCSI level, this shouldn't matter (the
IDs are opaque and are being checked for equality).  In other words, if
there's a layering problem here, it's in SAM2 ...

> [I've argued that conservative reuse of ISIDs to different target portal
groups
> provides a clean consistent view of SCSI Initiator Ports to the upper
layers and
> that's all that's needed.]

But that requirement is NOT in the current draft, and inserting it would
impact implementation approaches such as that originally described by Nick
Martin in which an iSCSI implementation has to cons up an ISID in the
potential absence of coordination with other implementations that share
the same iSCSI Initiator Node Name.  The nasty problems result when there
are multiple Initiators connected to multiple Targets, but not all
Initiators
are connected to all Targets - an Initiator could set up several sessions
and only then discover that the ISID it used is taken and have to tear down
those sessions and start over.  This is why ISIDs as currently specified
aren't sufficient to deal with proposal #1 and changing them to do so would
be a visible technical change.

> may make the implementation easier.

The summary is to have the OS enumeration provide the portal group tag,
and make that part of the ISID either explicitly or via a text key that
implicitly extends the ISID.  It's a good try, but I think it runs into
problems because device enumeration order in an operating system can
change between reboots for all sorts of reasons ranging from the
unavoidable (interface card fails in a fashion that causes it not to
be discovered on boot - this is discovery by plug-and-play logic or the
like, not iSCSI discovery) to the inane (stupid plug and play tricks,
new driver revisions).  This may wind up adding the Initiator Portal
Group number to the list of things that an Initiator has to remember
in order to get its persistent reservation state restored.

--David

---------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 42 South St., Hopkinton, MA  01748
+1 (508) 435-1000 x75140     FAX: +1 (508) 497-8500
[email-masked]       Mobile: +1 (978) 394-7754
---------------------------------------------------

---

From: "Julian Satran" <[email-masked]>
Date: Sun, 7 Oct 2001 12:30:07 +0300

David,

What puzzles me is that the you bring the negotiation argument as an
argument is favor of text keys. I think that having a response of "conflict
detected" is completely equivalent to option c (if I recall correctly the
number).

And handling keys that do not conflict trough the OS may prove trickier
than handle 16 bit numbers.

An Item that Jim suggested (more permanent initiator port numbers) may
favor another structure based on the portal-group-tag
and a session number within the tag with slow reuse. We may want then to
have a different split between ISID and TSID (and/or include a similar
split for TSID).

Julo

                    Black_David@em                                                            
                    c.com                To:     [email-masked], [email-masked]      
                    Sent by:             cc:                                                  
                    owner-ips@ece.       Subject:     RE: iSCSI: ISID and the New Persistent  
                    cmu.edu               Reservations Proposals                              

                    06-10-01 03:52                                                            
                    Please respond                                                            
                    to Black_David                                                            

Jim,

And I have a strong discomfort with designing for functionality that isn't
specified.  We already have a mismatch where the current T10 proposal #1
does not match current ISID specification (see below on conservative
reuse).
As T10 does things with the Port Name, we can put in the text key(s)
required
to make them work.

anyway if T10 adopts any proposal that has reservations span sessions at
the
for any transport - in each cases the transport-specific port
identification

groups
layers and

--David

---

From: "Jim Hafner" <[email-masked]>
Date: Mon, 8 Oct 2001 08:46:22 -0700

David,

A few (hopefiully final) comments in line.

Jim Hafner

[email-masked] on 10/05/2001 06:52:59 pm

To:   Jim Hafner/Almaden/IBM@IBMUS, [email-masked]
cc:
Subject:  RE: iSCSI: ISID and the New Persistent Reservations Proposals

Jim,

Having the text key semantics be "yet to be defined" seems reasonable
because the behavior required by T10 is in the same state.  Horse first,
then cart :-).
<JLH>
The behavior required by T10 is not "yet to be defined".  What is missing
is crossing a few t's and dotting a few i's to make the proposed new
behavior more backward compatible with existing implementations.  The
general model is defined, though not yet approved (but I think you can
expect something in the next 3 months).
</JLH>

I disagree.  ISIDs aren't negotiable, text keys are.  In the text key
situation analogous to the ISID situation that causes the OptionA/OptionB
mess, the Target can set the text key to some sort of "Port Identity
Conflict Detected" value and ship everything back to the Initiator
with no harm done.  Trying to do that sort of thing with ISIDs would
be a mess, as indicated by the amount of email on OptionA/OptionB.
<JLH>
I don't get this?  You're going to have the target "negotiate the value of
this key"?  We don't negotiate the value of the iSCSIName.  This would fall
under the same category (I would hope!).   Saying "conflict" for key=value
isn't any different from saying "conflict" for ISID.
</JLH>

<JLH>
I don't see a mismatch, further comment below.
</JLH>

As long as the TSID is not used to provide this view, there is no problem
here (e.g., the enumeration of interface cards on boot can be used).
<JLH>
Then I can use the enumeration for ISIDs as well, as I outlined.
</JLH>

I disagree. The interaction required is something that will have to happen
anyway if T10 adopts any proposal that has reservations span sessions at
the
Target.  Suppose that port X opens a session to port A, places a persistent
reservation that spans target ports, and then opens a session to port B -
as part of setting up that second session, port B has to interact with the
persistent reservation logic to discover that the reservation exists and
applies to the second session.  This is all that's required, and it's true
for any transport - in each cases the transport-specific port
identification
has to be passed from the transport to the SCSI reservation logic.  The
only wrinkle is that iSCSI gets this name from a text key instead of a WWN
or a parallel SCSI bus ID - at the SCSI level, this shouldn't matter (the
IDs are opaque and are being checked for equality).  In other words, if
there's a layering problem here, it's in SAM2 ...
<JLH>
The interaction I was talking about (which is what I thought you were
addressing) was in the OS side (not the target).  It was about having the
application that is coordinating the reservation passing key stuff through
the layers (within the initiator) to the iSCSI initiator layer to get this
key over to the target.

You had talked about the reservation application passing key=values down
through the layers for iSCSI to use for SCSI Initiator port identification
(or some such).  That's the layering I was referring to.

The target has always had to do some (minimal) cross port coordination.
The newer proposals in T10 will require more (but no more in iSCSI than in
any other protocol, except SPI where this proposal has no relevance).
</JLH>

groups
layers and

But that requirement is NOT in the current draft, and inserting it would
impact implementation approaches such as that originally described by Nick
Martin in which an iSCSI implementation has to cons up an ISID in the
potential absence of coordination with other implementations that share
the same iSCSI Initiator Node Name.  The nasty problems result when there
are multiple Initiators connected to multiple Targets, but not all
Initiators
are connected to all Targets - an Initiator could set up several sessions
and only then discover that the ISID it used is taken and have to tear down
those sessions and start over.  This is why ISIDs as currently specified
aren't sufficient to deal with proposal #1 and changing them to do so would
be a visible technical change.
<JLH>
I disagree here.
(a) The current spec says what the ISIDs are used for and how they ought to
be generated (loosely).
(b) It does not make conservative reuse a REQUIREMENT, but it does suggest
reasons for that.
(c) Conservative reuse is NOT a requirement anyway (and I've never said it
is). It makes for some things being simpler.

I'm confused by your use of the terms "multiple initiators" and "multiple
targets".  Are we talking with the same iSCSI Name or different names?
This issue has nothing to do with three or more entities.  It is strictly
scoped between one iSCSI initiator (by name) and one iSCSI target (by
name).
</JLH>

The summary is to have the OS enumeration provide the portal group tag,
and make that part of the ISID either explicitly or via a text key that
implicitly extends the ISID.  It's a good try, but I think it runs into
problems because device enumeration order in an operating system can
change between reboots for all sorts of reasons ranging from the
unavoidable (interface card fails in a fashion that causes it not to
be discovered on boot - this is discovery by plug-and-play logic or the
like, not iSCSI discovery) to the inane (stupid plug and play tricks,
new driver revisions).  This may wind up adding the Initiator Portal
Group number to the list of things that an Initiator has to remember
in order to get its persistent reservation state restored.
<JLH>
But this isn't a problem at all.  So things move around.  The only affect
this would have is on a persistent reservation of type #1 and in that case,
perhaps enough resets and reboots have occured to have clean up and cluster
recovery so that the problem isn't there.
</JLH>

--David