Subject: RE: iSCSI: response to second login (with same ISID)

From: "Martin, Nick" <[email-masked]>
Date: Thu, 24 May 2001 01:38:17 -0500

[The message I am replying to is below this one.  A similar one was posted
while I was composing this reply.]

Santosh,

If the iSCSI HBA is exposing SCSI targets to the OS, then the OS's SCSI
target driver (or class driver if I have my Microsoft terminology correct)
can support "exclusive access" semantics which can protect the current user
of a tape device from other users on the same system attempting to use the
same device through the same driver, or can allow a disk device (and the
session for it) to be shared by multiple users when that is appropriate.  If
the target device supports some form of SCSI reserve/release, then this
would protect the device from access via a second session from anywhere.  In
the target, iSCSI could also refuse a second concurrent session login to the
same tape target protecting initiators which do not reserve/release.  (The
trick is: does the target think this is a new session using the same ISID
(reject), or an attempt to clean-up and re-establish an existing session
(reset, accept).)

Think about an iSCSI enabled version of tar or cpio (simple applications
used for tape backups).  These could run from user space over a TCP
connection on a standard NIC using an OS that has no awareness of iSCSI.  If
a second copy of the program is started, how is it to know that something is
already using the same tape device?  How is it to know the ISID used for
that other copy?  How is it to know whether ANY particular ISID is really
currently in use, or what it is being used for?  How can we prevent the
second copy from unknowingly stealing the device from the first by simply
using the same ISID?  The second user thinks everything is fine until the
first user restarts his command that just failed.  In this model there is no
HBA driver or target driver.  There is nothing that knows about both
sessions being attempted except the target.

You may think this is a bit far fetched, but I almost have such a program.
I expect other folks will be clever enough to think of other ways to use
this "user mode access" capability of TCP/IP to use remote iSCSI devices
without kernel co-operation.  Can we enable them to peacefully co-exist with
each other?  Can they peacefully co-exist with kernel mode drivers?

There may be other scenarios such as administrators dynamically adding and
removing iSCSI devices to an OS while a system is running rather than at
boot time (think 24x7 operations, what choice to they have?).  If the ISID's
are going to have to be manually configured, then there is a high likelihood
of human error causing disruptions.  If we can enable non-disruptive
automatic ISID probing to find an available ISID during device configuration
or initialization, this seems safer.  Once we pick an ISID, we should still
remember it across re-boot, to avoid repeated probing, and to assist manual
clean-up after ungraceful shutdown.  We also need the "logout this old
session" capability both for recovery after an initiator crash or error
which failed to logout, and for when the initiator realizes it has no
remaining working TCP connections to an existing session and wants to be
sure to free target resources and start fresh.

So, I think there are valid arguments for both behaviors 1 and 2, and we
need to enable the initiator to select the behavior appropriate for each
application.

Stated another way, we need to allow for the case when the initiator is sure
that this ISID is unique and it takes full responsibility for all
consequences, and for the case where the initiator can not be sure this ISID
is unique and wants there to be no side effects for others if it turns out
not to be unique.

Thanks,
Nick
-----Original Message-----
From: Santosh Rao [mailto:[email-masked]]
Sent: Wednesday, May 23, 2001 8:36 PM
To: Martin, Nick
Cc: [email-masked]
Subject: Re: iSCSI: response to second login (with same ISID)

"Martin, Nick" wrote:

> 
> I know from personal experience how easy it is for an authorized user to
> unintentionally start a second copy of a backup program, while the tape
> drive is still in use.  This should be a harmless mistake.  What kind of
> traffic cop would prevent this request from making it to the iSCSI target
> and terminating the session already in progress?  (Rhetorical question,
> please do not describe for me such a mechanism ;)

Nick,

I know you don't want an answer to your rhetorical question ;-}, but, I
could'nt resist one quick query here.

In the scenario you describe, is it not the case that HBA drivers would
only send out a Login on the first open of that target, and on
subsequent opens would only bump an open_count or a ref_count of some
kind rather than send a login on the wire on every open ?

If that is correct, then, the scenario you describe would still cause
disruption even with the modified semantics, only, when the WRITE is
issued by the 2nd execution of the backup application (?).

Regards,
Santosh

---

From: Santosh Rao <[email-masked]>
Date: Thu, 24 May 2001 10:26:08 -0700

Nick,

I agree with you that there is a case for both usage models and a bit in
the Login PDU that allows for both behaviours would address both sets of
issues.

I'm in support of adding a bit ("login unconditionally") to the Login
PDU that allows the initiator to specify whether any previous session
for that ISID, if it exists, is to be logged out and a new session login
established.

Thanks,
Santosh

---

From: Stephen Bailey <[email-masked]>
Date: Thu, 24 May 2001 17:24:34 -0400

Nick & Santosh,

> each other?  Can they peacefully co-exist with kernel mode drivers?

There's a nice, elegant can of worms, I mean solution, to this
problem.  In fact, I'm sure this is probably just a rehash of somebody
else's (Jim Hafner's) proposal, but I'm going to beat out some
implications.  Some people might say this is already the way things
work.  I don't see anything in the spec that precludes this
interpretation, anyway.

If a SCSI I (as in I_T) is defined by a unique Initiator Name, then
you simply say that each distinct iSCSI entity within a system (and
across systems too, of course) has it's own Initiator Name.  That
means that each entity has it's own, distinct SID space.

In other words, the kernel mode iSCSI entity will have a different
Initiator Name from the user-mode programs.  Each iSCSI entity (user
program or kernel) is responsible for producing its own Initiator
Name, AND authenticating that name with the target.  A user mode
program could find the kernel's initiator name, but it couldn't
perform the proper authentication, so it would not be able to `hijack'
the TSID.  Similarly noncooperative user mode programs could not
authenticate themselves as each other either.

> You may think this is a bit far fetched, but I almost have such a

Not at all.  I think it's a very exciting possibility.  One of the
features which an iSCSI running on RDMA can offer is fully accelerated
(as in, hardware handled data), user-mode (OS-bypassed) iSCSI
connections.  This is a major reason why I keep pushing iSCSI/RDMA as
The Right Thing.

> Santosh said:

Given the idea above, it's not necessary.  Even if you discount the
idea above (which you better not, or I'm going to be really MAD!), I
don't see how such a bit would be used.  What is the algorithm for
setting the bit?

If you allow multiple entities (e.g. a user-client and a kernel
client) to use the same Initiator Name, one of those entities can deny
service from the other, accidentally, or deliberately by having the
bit set.  Specifically, to allow access at all in this scenario, it
must either be the case that:

  o two entities can authenticate with the same Initiator Name
or
  o no authentication is required

In this case, you can not force a malicious or clumsy entity not to
set the bit, which means that one entity can deny service to the other
by hijacking (forcing implicit logout) a session.

Even if you disallow multiple entities using the same Initiator Name,
the policing behavior of the `~login unconditionally' setting doesn't
buy you much.  The first time in your life you login with a TSID, you
must assume that you may have been logged in to the same target in a
previous life (reboot), within RA_TOV.  You have no way (no memory of
the previous life) to know that you did not do that.  Therefore,
clearly, the first time in your life you use a TSID, you will login
with `login unconditionally'.  On subsequent logins with a TSID within
the same life you could set '~login unconditionally' and you might
learn that you blew your TSID allocation policy (or something else),
but frankly, it seems like a pretty small gain.  Even if you always
login with `login unconditionally', you will still certainly know if
you blown your TSID allocation policy.

Steph

P.S.

> (or class driver if I have my Microsoft terminology correct)

Actually, that's VMS[++ == WNT] terminology, and yep you got it :^)