Subject: Re: iscsi : changes involving tgt portal group tag.

From: "Jim Hafner" <[email-masked]>
Date: Mon, 4 Mar 2002 08:01:00 -0800

Santosh,

Some comments in line (the net is that I think you're right, but with a
slightly different reason in the first case).

Jim Hafner

Santosh Rao <[email-masked]>@t10.org on 03/01/2002 06:02:54 PM

Sent by:    [email-masked]

To:    IPS Reflector <[email-masked]>
cc:    T10 Reflector <[email-masked]>
Subject:    iscsi : changes involving tgt portal group tag.

* From the T10 Reflector ([email-masked]), posted by:
* Santosh Rao <[email-masked]>
*
All,

I believe iscsi needs to make 2 changes :

1) to mandate persistence of the portal group tag (and thus, persistence
of its scsi target port identifier).

2) to send the portal group tag (scsi target port identifier) as a part
of the iscsi login request.

The rationale for (1) is :
--------------------------
SAM-2 requires scsi port names to be persistent and world-wide-unique.
(SAM-2 Rev 22 Section 4.7.7). Quoting from SAM-2 :

"A scsi port name shall never change and may be used to persistently
identify a scsi initiator port or target port...".

iscsi defines scsi target port name as :

iscsi target name + NULL delimiter + 't' + NULL + (0 - 3) bytes of NULL
padding + 2 byte portal group tag (in big endian format).

In order for the above scsi target port name to meet SAM-2's requirement
of being persistent, the portal group tag needs to be persistent.

<JLH>
There are different ways that one can interpret this "persistent" rule. The
intent was that names shouldn't change over time when *identifying the same
object*.   What that means is that if the object changes (in any
discernable way), then the name should change.  So, the object can move to
a different piece of hardware, but it can still be named the same way.  If
the object changes, e.g., in this case, reconfigures as a different set of
network portals with different addressing elements, then I would think that
the name should change as well.   That's all the persistence one really
needs.

I'm not sure what that implies about your recommendation.  I don't see any
problem with it, either.
</JLH>

The rationale for (2) is :
--------------------------
Consider an initiator node establishing multiple sessions to a scsi tgt
port, with each session established to a subset of the network portals
within the tgt portal group.

Consider an iscsi transport event involving the re-configuration of
target portal groups on the iscsi target node. This may be preceeded by
the iscsi sessions seeing an async message "target requests logout",
followed by session logout, portal group re-configuration, and then, the
initiator re-establishes sessions.

Across a transport event that results in such session termination and
re-establishment, the initiator needs to authenticate that it is still
speaking to the same [i]scsi target port, in order to ensure that any
open/active I-T-L nexus traffic on that session is not incorrectly
routed to a wrong LUN after such a transport event.
<JLH>
Under these events, shouldn't all "open/active I_T_L traffic" have been
aborted, closed or otherwise ended?  So this shouldn't be an issue at all.

Note that the session end-points in such sessions  are individual
network portals within a portal group (tgt port) and a target
re-configuration involving the moving of network portals from 1 portal
group to another can occur.

Following such a re-configuration, if an initiator were to re-establish
a session to the same session end-points, since there is no addressing
information carried in the iscsi login request which carries the target
port identifier, the session may be established to the same network
portal, but under a different scsi target port and the I-T-L nexus
traffic incorrectly routed to a different scsi tgt port, resulting in
potential data corruption.

Other session oriented SCSI transports like FCP and SRP address this
problem by defining authentication schemes as well and/or explicitly
carrying the target port identifier in their login requests, which allow
them to identify such an authentication mis-match.

To prevent such authentication issues, iscsi can send the iscsi target
port identifier (portal group tag) explicitly in the login request, and
the login can be rejected by the target on a portal group tag mis-match.
(if the network portal does not belong to the addressed portal group
tag).
<JLH>
Did you mean for the initiator to send this TPGT?
</JLH>

Also, on any portal group re-configuration, iscsi targets MUST terminate
the session to avoid such scenarios as described above.
<JLH>
I would have thought that was implicit in the session logout.
</JLH>

Comments ?

<JLH>
Final comment:
I think you may have hit an odd corner case.  I think we need a rule that
says that (visibile) target port reconfiguration should involve new "names"
(either node or port). That such a reconfiguration should only occur after
all sessions have been dropped (with all the attendent clearing effects).
And that including the TPGT in the login with target reject if it isn't
correct should also be added.
</JLH>

Regards,
Santosh

--
##################################
Santosh Rao
Software Design Engineer,
HP-UX iSCSI Driver Team,
Hewlett Packard, Cupertino.
email : [email-masked]
Phone : 408-447-3751
##################################
*
* For T10 Reflector information, send a message with
* 'info t10' (no quotes) in the message body to [email-masked]

---

From: Santosh Rao <[email-masked]>
Date: Mon, 04 Mar 2002 10:40:45 -0800

Jim,

I agree that a "complete re-configuration" of a target port can result
in a new port name & port identifier. However, the tricky part is the
definition of a "complete re-configuration of an iscsi target port", due
to the concepts of portal groups involving multiple network portals.

For example, if a portal group (aka, an iscsi target port) were to be
re-configured to include a new network portal [moved from another portal
group], then, the target port itself has not changed, since it is still
accessible through its previously used network portals. What has changed
is the individual network portal that has moved from 1 target port to
another. 

Hence, it is sufficient, in this case, to maintain persistence of the
target port name/identifier, without requiring any change in port
name/identifier. By requiring initiators to send the intended TPGT (scsi
target port name/identifier) along with the login request, this allows
the target port to detect that the network portal is being accessed to
connect to a different target port and it can reject the login.

It may be helpful to call out the specific case when a port
name/identifier MUST change. How about something like :

"If a portal group is re-configured such that all its previously
advertised network portals are no longer a part of the portal group,
then, the portal group tag (and thereby, the port name/identifier)
*MUST* be changed to indicate a new target port."

This would allow access to the target port through its un-altered
network portals to continue un-disrupted. More comments inline, in
response to some of your queries.

Thanks,
Santosh

NOTE : In this discussion target port and target portal group are used
to imply the same entity, within a target node.

> I'm not sure what that implies about your recommendation.  I don't see any

I think we may be in agreement. (See reasoning above). 

> aborted, closed or otherwise ended?  So this shouldn't be an issue at all.

On a session logout & re-login, it is not efficient/necessary to close
and re-open each LUN behind that target port, due to the fact that a
target port may have hundred's of LUNs behind it. 

All outstanding I/Os need to be aborted. Once the session is
re-established [and the target port is authenticated to be the same],
further I/O traffic can be resumed without requiring the SCSI ULP to
close and re-open each LUN. Some transport specific clearing effects may
have occurred, which may require additional LUN level activity, in some
cases. 

(There are analogies to the above model in FCP & SRP, which authenticate
port name/identifier on login, allowing I/O resumption after session
termination and re-establishment.)

> Did you mean for the initiator to send this TPGT?

Yes. The initiator login request must include the target portal group
tag, thus identifying the target port to which a session is being
established. 

Login currently carries no addressing information, since the addressing
is implicit, based on the TCP connection in use. The problem with this
approach is that the login implicitly addresses a network portal, and
not the target port. As seen in the earlier example, the association of
network portal <-> target port can change, and such changes can be
detected, if the initiator were to explicitly identify the target port
being logged into.

---

From: "Mallikarjun C." <[email-masked]>
Date: Mon, 4 Mar 2002 15:12:15 -0800

Santosh and Jim,

It appears a good idea to add in the portal group tag in the Login 
Request header for a sanity check by the receiving target portal.

I generally agree with Jim's comments that the duration of persistence
of a portal group tag is intricately linked to the extent of portal reconfiguration.
Not every target reconfiguration may result in a portal group tag reassignment.
OTOH, some reconfigurations may be so extensive as to cause even a node 
name change.

Some comments on Santosh's message -

> *MUST* be changed to indicate a new target port."

I am not sure this solves the problem you're trying to get at.  If none of
the earlier IP addresses can get an initiator to the SCSI target port that it 
knew of (your scenario), it appears to me that it doesn't matter if the 
portal group tags are changed or not....A new discovery process should
update the initiator of the changed portal addresses.

I suggest the following text -

   After a portal group reconfiguration which changed the view of LUs
   for an initiator with a given set of privileges, the target MUST change
   the portal group tag that represents the reconfigured target portal group.

> target port may have hundred's of LUNs behind it. 

I agree with Jim on this one - there should be *no* open/active I_T_L nexus 
traffic after a reconfiguration, targets can enforce this via simple iSCSI means
(reject initiator advances to continue the session for DefaultTime2Wait+
DefaultTime2Retain seconds).  In fact, Async logout request requires a
clean closure that implicitly aborts I/Os.

What you're describing is typical O/S "LUN open" and "LUN close"
interactions.  I agree that they wouldn't have to be repeated, but care 
must be taken to ensure that new I/Os (on the new session after reconfiguration)
are not delivered to the incorrect LUs.  It seems that the addition of 
TPGT in the login header and the proposed new text (above) would take
care of this.

Regards.
--
Mallikarjun

Mallikarjun Chadalapaka
Networked Storage Architecture
Network Storage Solutions Organization
Hewlett-Packard MS 5668 
Roseville CA 95747