Subject: Re: iscsi : changes involving tgt portal group tag.

From: "John Hufferd" <[email-masked]>
Date: Tue, 12 Mar 2002 11:07:35 -0800

Mallikarjun,
I do not see the absolute requirement for this TPGT in the Login.  I think
it would be approprate to say that if the time for the Reconnect has been
more then the "...Time2Wait" that the Initiator, if it cares about the
TPGT,  SHOULD perform a rediscovery.

That would address the issue with out any protocol changes.

.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
Home Office (408) 997-6136, Cell: (408) 499-9702
Internet address: [email-masked]

"Mallikarjun C." <[email-masked]>@ece.cmu.edu on 03/12/2002 10:30:29 AM

Sent by:    [email-masked]

To:    "Shailesh Manjrekar" <[email-masked]>
cc:    <[email-masked]>, <[email-masked]>
Subject:    Re: iscsi : changes involving tgt portal group tag.

Shailesh,

The failure resulting out of a TPGT mismatch (assuming we have the TPGT in
the Login Request PDU), would have to trigger a discovery
(SendTargets/SLP/...)
updating the initiator of the latest configuration.  This discovery process
is
similar
to the FCP ADISC process you refer to.

Alternatively, if there has been a change in the LU view of  a given target
portal
group tag (meaning that the TPGT would match correctly), the LUs in the
view
should have established UA with an ASC of REPORTED LUNS DATA HAS
CHANGED since the SCSI port association has changed for the LUs.  This
again
should trigger a discovery process from the initiator.

It seems to be that we are now sufficiently covered either way.
--
Mallikarjun

Mallikarjun Chadalapaka
Networked Storage Architecture
Network Storage Solutions Organization
Hewlett-Packard MS 5668
Roseville CA 95747
[email-masked]

----- Original Message -----
From: "Shailesh Manjrekar" <[email-masked]>
To: "'Mallikarjun C.'" <[email-masked]>; "'Julian Satran'"
<[email-masked]>
Cc: <[email-masked]>; <[email-masked]>
Sent: Monday, March 11, 2002 7:51 PM
Subject: RE: iscsi : changes involving tgt portal group tag.

> Hi,
>
> On a TPG reconfiguration or TPGT reassignment, shouldn't there be a
> discovery followed by SCSI Inquiry and Report_LUNS which would make the
> Initiator aware of this change? Can the  Async message - iscsi Event
> Data notify the Initiator of the change, which would force an discovery.
> This would be similar to an ADISC for FCP. Because including the TPGT in
> the login would prevent inadvertent logins but would still not notify
> the initiator of the changed configuration?
>
> Thanks,
> Shailesh.
> Aarohi Communications.
> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Mallikarjun C.
> Sent: Wednesday, March 06, 2002 10:17 AM
> To: Julian Satran
> Cc: [email-masked]; [email-masked]
> Subject: Re: iscsi : changes involving tgt portal group tag.
>
> * From the T10 Reflector ([email-masked]), posted by:
> * "Mallikarjun C." <[email-masked]>
> *
> > The issue is that I am not sure that a target is checking today
> anything -
> > (adapter drivers may check oif in their realm they can give out a
> TSID).
> > Nothing else is needed.
>
> Target is required to derive the TPGT and use it in both cases of Login
> -
>      a) when TSID = 0, to ascertain if a session reinstatement needs to
>          be effected for that TPGT.
>      b) when TSID != 0, to ascertain the validity of TSID and add in
>           the new connection to an existing session if it is valid for
> that TPGT.
> --
> Mallikarjun
>
> Mallikarjun Chadalapaka
> Networked Storage Architecture
> Network Storage Solutions Organization
> Hewlett-Packard MS 5668
> Roseville CA 95747
>
> ----- Original Message -----
> From: "Julian Satran" <[email-masked]>
> To: "Mallikarjun C." <[email-masked]>
> Cc: <[email-masked]>; <[email-masked]>
> Sent: Tuesday, March 05, 2002 11:56 PM
> Subject: Re: iscsi : changes involving tgt portal group tag.
>
>
> > The issue is that I am not sure that a target is checking today
> anything -
> > (adapter drivers may check oif in their realm they can give out a
> TSID).
> > Nothing else is needed. Does SCSI have to be aware of it? Perhaps but
> > iSCSI certainly not. Does a "front-end" have to be - again probably
> not
> > the name identifies the node.
> >
> > Julo
> >
> >
> >
> >
> > "Mallikarjun C." <[email-masked]>
> > 05-03-02 22:34
> > Please respond to "Mallikarjun C."
> >
> >
> >         To:     Julian Satran/Haifa/IBM@IBMIL
> >         cc:     <[email-masked]>, <[email-masked]>
> >         Subject:        Re: iscsi : changes involving tgt portal group
> tag.
> >
> >
> >
> > Julian,
> >
> > Whatever methods a target is expected to use today to derive the
> implicit
> > TPGT to preclude a duplicate I-T nexus would work after the change as
> > well,
> > except the derived value needs to be compared against the (proposed)
> > TPGT in the Login Request payload.
> >
> > Please comment if we're missing something.
> >
> > Regards.
> > --
> > Mallikarjun
> >
> > Mallikarjun Chadalapaka
> > Networked Storage Architecture
> > Network Storage Solutions Organization
> > Hewlett-Packard MS 5668
> > Roseville CA 95747
> >
> > ----- Original Message -----
> > From: "Julian Satran" <[email-masked]>
> > To: "Mallikarjun C." <[email-masked]>
> > Cc: <[email-masked]>; <[email-masked]>; "Santosh Rao"
> > <[email-masked]>;
> > <[email-masked]>
> > Sent: Monday, March 04, 2002 10:24 PM
> > Subject: Re: iscsi : changes involving tgt portal group tag.
> >
> >
> > > Has a decent target implementation and easy way of checking that the
> TPG
> > > is correct?
> > >
> > > Julo
> > >
> > >
> > >
> > >
> > > "Mallikarjun C." <[email-masked]>
> > > Sent by: [email-masked]
> > > 05-03-02 01:12
> > > Please respond to "Mallikarjun C."
> > >
> > >
> > >         To:     "Santosh Rao" <[email-masked]>,
> <[email-masked]>
> > >         cc:     <[email-masked]>
> > >         Subject:        Re: iscsi : changes involving tgt portal
> group
> > tag.
> > >
> > >
> > >
> > > Santosh and Jim,
> > >
> > > It appears a good idea to add in the portal group tag in the Login
> > > Request header for a sanity check by the receiving target portal.
> > >
> > > I generally agree with Jim's comments that the duration of
> persistence
> > > of a portal group tag is intricately linked to the extent of portal
> > > reconfiguration.
> > > Not every target reconfiguration may result in a portal group tag
> > > reassignment.
> > > OTOH, some reconfigurations may be so extensive as to cause even a
> node
> > > name change.
> > >
> > > Some comments on Santosh's message -
> > >
> > > > "If a portal group is re-configured such that all its previously
> > > > advertised network portals are no longer a part of the portal
> group,
> > > > then, the portal group tag (and thereby, the port name/identifier)
> > > > *MUST* be changed to indicate a new target port."
> > >
> > > I am not sure this solves the problem you're trying to get at.  If
> none
> > of
> > > the earlier IP addresses can get an initiator to the SCSI target
> port
> > that
> > > it
> > > knew of (your scenario), it appears to me that it doesn't matter if
> the
> > > portal group tags are changed or not....A new discovery process
> should
> > > update the initiator of the changed portal addresses.
> > >
> > > I suggest the following text -
> > >
> > >    After a portal group reconfiguration which changed the view of
> LUs
> > >    for an initiator with a given set of privileges, the target MUST
> > change
> > >    the portal group tag that represents the reconfigured target
> portal
> > > group.
> > >
> > > > > Under these events, shouldn't all "open/active I_T_L traffic"
> have
> > > been
> > > > > aborted, closed or otherwise ended?  So this shouldn't be an
> issue
> > at
> > > all.
> > > >
> > > > On a session logout & re-login, it is not efficient/necessary to
> close
> > > > and re-open each LUN behind that target port, due to the fact that
> a
> > > > target port may have hundred's of LUNs behind it.
> > >
> > > I agree with Jim on this one - there should be *no* open/active
> I_T_L
> > > nexus
> > > traffic after a reconfiguration, targets can enforce this via simple
>
> > iSCSI
> > > means
> > > (reject initiator advances to continue the session for
> DefaultTime2Wait+
> > > DefaultTime2Retain seconds).  In fact, Async logout request requires
> a
> > > clean closure that implicitly aborts I/Os.
> > >
> > > What you're describing is typical O/S "LUN open" and "LUN close"
> > > interactions.  I agree that they wouldn't have to be repeated, but
> care
> > > must be taken to ensure that new I/Os (on the new session after
> > > reconfiguration)
> > > are not delivered to the incorrect LUs.  It seems that the addition
> of
> > > TPGT in the login header and the proposed new text (above) would
> take
> > > care of this.
> > >
> > > Regards.
> > > --
> > > Mallikarjun
> > >
> > > Mallikarjun Chadalapaka
> > > Networked Storage Architecture
> > > Network Storage Solutions Organization
> > > Hewlett-Packard MS 5668
> > > Roseville CA 95747
> > >
> > > ----- Original Message -----
> > > From: "Santosh Rao" <[email-masked]>
> > > To: <[email-masked]>
> > > Cc: <[email-masked]>
> > > Sent: Monday, March 04, 2002 10:40 AM
> > > Subject: Re: iscsi : changes involving tgt portal group tag.
> > >
> > >
> > > > * From the T10 Reflector ([email-masked]), posted by:
> > > > * Santosh Rao <[email-masked]>
> > > > *
> > > > Jim,
> > > >
> > > > I agree that a "complete re-configuration" of a target port can
> result
> > > > in a new port name & port identifier. However, the tricky part is
> the
> > > > definition of a "complete re-configuration of an iscsi target
> port",
> > due
> > > > to the concepts of portal groups involving multiple network
> portals.
> > > >
> > > > For example, if a portal group (aka, an iscsi target port) were to
> be
> > > > re-configured to include a new network portal [moved from another
> > portal
> > > > group], then, the target port itself has not changed, since it is
> > still
> > > > accessible through its previously used network portals. What has
> > changed
> > > > is the individual network portal that has moved from 1 target port
> to
> > > > another.
> > > >
> > > > Hence, it is sufficient, in this case, to maintain persistence of
> the
> > > > target port name/identifier, without requiring any change in port
> > > > name/identifier. By requiring initiators to send the intended TPGT
>
> > (scsi
> > > > target port name/identifier) along with the login request, this
> allows
> > > > the target port to detect that the network portal is being
> accessed to
> > > > connect to a different target port and it can reject the login.
> > > >
> > > > It may be helpful to call out the specific case when a port
> > > > name/identifier MUST change. How about something like :
> > > >
> > > > "If a portal group is re-configured such that all its previously
> > > > advertised network portals are no longer a part of the portal
> group,
> > > > then, the portal group tag (and thereby, the port name/identifier)
> > > > *MUST* be changed to indicate a new target port."
> > > >
> > > > This would allow access to the target port through its un-altered
> > > > network portals to continue un-disrupted. More comments inline, in
> > > > response to some of your queries.
> > > >
> > > > Thanks,
> > > > Santosh
> > > >
> > > > NOTE : In this discussion target port and target portal group are
> used
> > > > to imply the same entity, within a target node.
> > > >
> > > >
> > > > Jim Hafner wrote:
> > > >
> > > > > SAM-2 requires scsi port names to be persistent and
> > world-wide-unique.
> > > > > (SAM-2 Rev 22 Section 4.7.7). Quoting from SAM-2 :
> > > > >
> > > > > "A scsi port name shall never change and may be used to
> persistently
> > > > > identify a scsi initiator port or target port...".
> > > > >
> > > > > <JLH>
> > > > > There are different ways that one can interpret this
> "persistent"
> > > rule. The
> > > > > intent was that names shouldn't change over time when
> *identifying
> > the
> > > same
> > > > > object*.   What that means is that if the object changes (in any
> > > > > discernable way), then the name should change.  So, the object
> can
> > > move to
> > > > > a different piece of hardware, but it can still be named the
> same
> > way.
> > >  If
> > > > > the object changes, e.g., in this case, reconfigures as a
> different
> > > set of
> > > > > network portals with different addressing elements, then I would
>
> > think
> > > that
> > > > > the name should change as well.   That's all the persistence one
> > > really
> > > > > needs.
> > > > >
> > > > > I'm not sure what that implies about your recommendation.  I
> don't
> > see
> > > any
> > > > > problem with it, either.
> > > > > </JLH>
> > > >
> > > > I think we may be in agreement. (See reasoning above).
> > > >
> > > >
> > > > >
> > > > > The rationale for (2) is :
> > > > > --------------------------
> > > > > Consider an initiator node establishing multiple sessions to a
> scsi
> > > tgt
> > > > > port, with each session established to a subset of the network
> > portals
> > > > > within the tgt portal group.
> > > > >
> > > > > Consider an iscsi transport event involving the re-configuration
> of
> > > > > target portal groups on the iscsi target node. This may be
> preceeded
> > > by
> > > > > the iscsi sessions seeing an async message "target requests
> logout",
> > > > > followed by session logout, portal group re-configuration, and
> then,
> > > the
> > > > > initiator re-establishes sessions.
> > > > >
> > > > > Across a transport event that results in such session
> termination
> > and
> > > > > re-establishment, the initiator needs to authenticate that it is
>
> > still
> > > > > speaking to the same [i]scsi target port, in order to ensure
> that
> > any
> > > > > open/active I-T-L nexus traffic on that session is not
> incorrectly
> > > > > routed to a wrong LUN after such a transport event.
> > > >
> > > > > <JLH>
> > > > > Under these events, shouldn't all "open/active I_T_L traffic"
> have
> > > been
> > > > > aborted, closed or otherwise ended?  So this shouldn't be an
> issue
> > at
> > > all.
> > > >
> > > > On a session logout & re-login, it is not efficient/necessary to
> close
> > > > and re-open each LUN behind that target port, due to the fact that
> a
> > > > target port may have hundred's of LUNs behind it.
> > > >
> > > > All outstanding I/Os need to be aborted. Once the session is
> > > > re-established [and the target port is authenticated to be the
> same],
> > > > further I/O traffic can be resumed without requiring the SCSI ULP
> to
> > > > close and re-open each LUN. Some transport specific clearing
> effects
> > may
> > > > have occurred, which may require additional LUN level activity, in
>
> > some
> > > > cases.
> > > >
> > > > (There are analogies to the above model in FCP & SRP, which
> > authenticate
> > > > port name/identifier on login, allowing I/O resumption after
> session
> > > > termination and re-establishment.)
> > > >
> > > >
> > > > > To prevent such authentication issues, iscsi can send the iscsi
> > target
> > > > > port identifier (portal group tag) explicitly in the login
> request,
> > > and
> > > > > the login can be rejected by the target on a portal group tag
> > > mis-match.
> > > > > (if the network portal does not belong to the addressed portal
> group
> > > > > tag).
> > > > > <JLH>
> > > > > Did you mean for the initiator to send this TPGT?
> > > > > </JLH>
> > > >
> > > > Yes. The initiator login request must include the target portal
> group
> > > > tag, thus identifying the target port to which a session is being
> > > > established.
> > > >
> > > > Login currently carries no addressing information, since the
> > addressing
> > > > is implicit, based on the TCP connection in use. The problem with
> this
> > > > approach is that the login implicitly addresses a network portal,
> and
> > > > not the target port. As seen in the earlier example, the
> association
> > of
> > > > network portal <-> target port can change, and such changes can be
> > > > detected, if the initiator were to explicitly identify the target
> port
> > > > being logged into.
> > > >
> > > > --
> > > > ##################################
> > > > Santosh Rao
> > > > Software Design Engineer,
> > > > HP-UX iSCSI Driver Team,
> > > > Hewlett Packard, Cupertino.
> > > > email : [email-masked]
> > > > Phone : 408-447-3751
> > > > ##################################
> > > > *
> > > > * For T10 Reflector information, send a message with
> > > > * 'info t10' (no quotes) in the message body to [email-masked]
> > > >
> > >
> > >
> > >
> >
> >
> >
> >
> *
> * For T10 Reflector information, send a message with
> * 'info t10' (no quotes) in the message body to [email-masked]
>
>

---

From: "Mallikarjun C." <[email-masked]>
Date: Tue, 12 Mar 2002 16:09:33 -0800

John,

> I do not see the absolute requirement for this TPGT in the Login.  

That was my initial inclination as well, but now I believe there's value
in adding the TPGT to the Login Request PDU.

> more then the "...Time2Wait" that the Initiator, if it cares about the

If I understand you correctly, you are suggesting that the DefaultTimeWait
value negotiated *for a session* should be used for decisions after the 
end of that session.  I am afraid that it's a slippery slope...

OTOH, regardless of the proposed change, it would be reasonable to add text 
that generally expresses this idea of potential volatility of portal<->portal group 
association in the iSCSI (or perhaps NDT) draft.

On to reasons that convinced me....

- An iSCSI session is an I_T nexus, and it is between an initiator *port*
   and a target *port*.  Login as a mechanism to instantiate/add to an
   iSCSI session should identify the target port in question, not just the 
   target node.  The initiator port is currently identified in the Login dialogue 
  (the InitiatorName text key and the ISID field), but not the target port.

- When there has been a portal group change for an IP address (portal) -
   meaning its TPGT had changed - it would be much more quicker to 
   identify the fact and prevent an I_T nexus establishment by way of 
   the TPGT in Login Request PDU.  The other option of having each 
   LU assert its UA (REPORTED LUNS DATA HAS CHANGED) on 
    the first command is prone to errors (see next bullet), and simply ineffcient.

- A target cold reset or a powercycle (possibly done after the portal group 
   reconfiguration) would clear the aforementioned UA (but would assert
   a different UA for the power-on reset, but this generic UA gives no 
   clue to initiators on the need to re-discover the target ports).

- At the moment, it is unclear to me if SPC-3 is mandating an "interlocked"
   style UA for the REPORTED LUNS DATA HAS CHANGED condition
   (though it seems like it... I will raise it only on t10 reflector under a different 
    cover).  If that isn't the case, the UA interlock capability (T10/00-359) would 
    need to deployed as well to realibly deal with this portal group reconfiguration.
  
> That would address the issue with out any protocol changes.

It is certainly a PDU change, but one can argue (successfully, I think) that 
it is not a "protocol" change per se -  since the implicit usage of TPGT so far
(see my message to Julian earlier on this thread) is merely being turned into 
an explicit usage.

To summarize, I realize the sensitivity of changes this late but I believe there 
are reasonable grounds here.

Regards.
--
Mallikarjun

Mallikarjun Chadalapaka