Subject: iSCSI: Canonical Targets

From: Mark Bakke <[email-masked]>
Date: Fri, 11 May 2001 16:04:24 -0500

It was pointed out during the interim meeting that the
canonical target is defined somewhat ambiguously, and may
be a bit too flexible for interoperability purposes.  This
message is a stake in the ground for defining this a little
more tightly.

So here are the new canonical target rules.  Please let me
know if there are major problems with any of them.  I tried
to define them a-la-carte, to make it easier to pick out any
that might cause trouble.

1. Each iSCSI implementation MUST include a canonical target.

2. A canonical target MUST be accessible at the default, IANA-
   assigned TCP port on each IP address on which the iSCSI
   implementation is listening for iSCSI connections.

3. A canonical target MUST NOT be used for SCSI commands.

   A canonical target is an iSCSI construct only, and does not
   have a corresponding SCSI device.  This means it may not
   be used to access Logical Units.

   A session created to a canonical target is a discovery session
   only, and once in full feature phase, is used only for text
   commands and asynchronous messages.  (Do any other commands make
   sense)?

4. A device containing a single target MUST provide both the
   canonical target and the real target.  (This is implied by the
   above requirements).

   An initiator connecting to such a device using only its IP address
   would first connect to the canonical target, and use SendTargets
   to obtain the iSCSI name of the real target.  It would then create
   a separate session to the real target.  Essentially, this means
   there's nothing special about a single-target device. 

5. An iSCSI device MUST provide a unique iSCSI name for each of its
   targets.  Using the canonical target as a nameless iSCSI target
   is not supported.

6. We can further specify the order in which the SendTargets response
   fields are returned, to simplify things further, e.g. each target
   in the SendTargets response MUST return these fields in this order:

   - A TargetName= field
   - A TargetAlias= field (value left blank if there's no alias)
   - One or more TargetAddress= fields
   - Any vendor-specific fields (ignored by standard initiators)

The above rules will make the behavior of various target implementations
identical, regardless of the number of interfaces or targets they
support.  This will reduce the number of end-cases that initiator
implementations will have to handle.

If we agree on these, we can edit them into the iSCSI and NDT documents.

Additional questions to send input on:

1. Should a non-canonical target respond to a SendTargets command?

2. If so, should it respond only with addresses for its own target,
   or should it respond with other targets, as a canonical target
   might do?

3. If not, the initiator must connect to a canonical target to find
   the other addresses of a target to which it is already connected;
   is the information it has sufficient to do so?  (I think the answer
   is yes, given canonical requirement #2 above).

-- 
Mark A. Bakke
Cisco Systems
[email-masked]
763.398.1054

---

From: Stephen Bailey <[email-masked]>
Date: Sun, 13 May 2001 22:54:28 -0400

Mark,

This is mostly fine with me.  However, I KNOW implementors are going
to gripe about having to know the target name before performing an
operational login.  In general, the spec seems a bit capricious about
when implementors just have to suck it up and do the hard thing,
versus when we try to make it easy for them.  Realistically, it's not
a big deal to learn the name (just an extra login), but most end
points are going to have a single target, and from a configuration
standpoint endpoint (and, hence the target) is going to be specified
by transport coordinates (address, port).  This extra login for
discovery is simply an irritant.

>    - Any vendor-specific fields (ignored by standard initiators)

I can only guess that you are implying that target descriptions
following this rule may be tiled arbitrarily into Text PDUs returned
from the target?  If so, we should specify that.

> 1. Should a non-canonical target respond to a SendTargets command?

I'd say no.  Given the nature of the interaction, why bother?  Now, I
know the iSCSI philosophy seems to be `if it's not complete gibberish,
why NOT bother?' but that results in complicated implementation.
Specifically, each end (target and initiator) should be able to audit
the other for correctness when possible.  Hard rules like `thou shalt
not issue a SendTargets in an operational session' make the auditing a
zillion times easier.

To this end, I would go a step further and allow only a single
outstanding SendTargets exchange in a target discovery session.

Steph

---

From: "Ayman Ghanem" <[email-masked]>
Date: Mon, 14 May 2001 08:38:25 -0500

I think a non-canonical target should respond to SendTargets. Some
implementations may chose to drop the session to the canonical target after
connecting to the target devices. If new devices become available, it is
easier for a non-canonical target to respond with target names, rather than
having the initiator relogin to the canonical target.

-Ayman

---

From: Mark Bakke <[email-masked]>
Date: Mon, 14 May 2001 15:30:25 -0500

> discovery is simply an irritant.

True; it is an extra login.  In actual use, however, an initiator might
be configured with the IP address of a target to use, and it would have
to do the extra session after the SendTargets.  Once the initiator has
discovered the actual iSCSI name of the target, it will probably have to
keep it in a registry or configuration file anyway, since it will need to
look for it next time it boots.  Some operating systems are particularly
fussy about making the same LUs map to the same /dev entries, and so on,
and the only way to make them appear in the same place every time is to
keep track.  Given this, it is not always effective to just discover by
address after the first use of a device.

You had mentioned in the interim meeting that we could provide a separate
"default" target (different from the canonical target), that would represent
the only target of a single-target device, and could be logged into without
having to know its name.  This would make these devices simpler, but an
initiator would still need to have the functionality to handle multiple-
target devices anyway.

I also don't think that there will be a lot of single-target devices out
there, at least not right away.  Most devices will likely be accessed through
iSCSI-FC or iSCSI-SCSI gateways in the near future, before iSCSI interfaces are
added to the devices themselves.  Our gateway, for example, provides more
than one target; my guess is that others will do the same.  In this case,
an initiator has to deal with multiple targets anyway.

> from the target?  If so, we should specify that.

That's pretty much what it does now; I was trying to specify it a little
tighter.  Do you have any suggestions?

> not issue a SendTargets in an operational session' make the auditing a

I'm leaning that way as well.

> outstanding SendTargets exchange in a target discovery session.

That's a good idea; I can't imagine a reason that the initiator would
want to do this more than once in a session, and it would be nice for
a target to know it can reject a SendTargets command if it already has
one outstanding on the same session.

-- 
Mark A. Bakke
Cisco Systems
[email-masked]
763.398.1054

---

From: Stephen Bailey <[email-masked]>
Date: Mon, 14 May 2001 18:11:24 -0400

Mark,

> there, at least not right away.

Perhaps not right away, but definitely eventually.  Using the LUN
model is the way to ensure that a target (true target, not bridge)
implementation can be relatively transport independent.  That's proven
to be a major survival characteristic for targets.

> Most devices will likely be accessed through iSCSI-FC or iSCSI-SCSI
> gateways in the near future, before iSCSI interfaces are added to
> the devices themselves.  Our gateway, for example, provides more
> than one target; my guess is that others will do the same.

Agreed.  The only SST target in the world is a gateway, and it
provides multiple target support as well.  Our solution to this
problem was to constrain the target addressing range so that the
initiator can do the traditional big crawl.  SST was aimed at a
different design point, so I'm not saying we should adopt the same
solution with iSCSI, I'm just agreeing that allowing multiple targets
behind an endpoint in the protocol design is a good idea.

> In this case, an initiator has to deal with multiple targets
> anyway.

Absolutely.  However, in the limit, the bridges usually wither away.
Sometimes they wither away a lot faster than you might expect.  There
were some early attempts at FC/||SCSI bridging that worked from the
same set of assumptions.  You probably don't need to know how
successful those were.  That would never happen to today's modern
bridge vendor, since they've learned how to apply all their
accumulated cleverness to building native targets in short order :^)

The bridged scenario shouldn't be at the expense of the unbridged one.

My proposal of allowing:
  1) target discovery login
  2) operational login to a default (Julian's right---the term
     canonical is inappropriate) target
  3) operational login to a specified target

seems to cover all bases.  If your bridge does not want to support a
default target (I bet it will.... how else will you be able to slap it
under the skins with a Symmetrix and call it an iSCSI storage array?),
you can reject the login, with a `no default' reason code (TBS).

I don't mind adding Jim Hafner's multi-layer discovery proposal if you
guys really think it will help.  Personally, I'm skeptical.  Whatever
the case, it seems like it can be done in such a way as to still
permit a single-layer login when given nothing but socket
coordinates.

Let me beat on this one final time.  I think EVERYBODY on this list is
aware that the biggest selling point of iSCSI across the entire range
of applications is that it leverages existing IP management skills.
As such, using socket coordinates to specify targets is what IP
managers understand.  That doesn't mean we can't introduce additional
concepts, but the users should be able to get going without them.

Steph

---

From: Mark Bakke <[email-masked]>
Date: Tue, 15 May 2001 08:20:24 -0500

Steph-

Good discussion.  My comments are below.

>   3) operational login to a specified target

I'm OK with adding the default target if we feel we need it; again,
I just don't want to over-simplify the initiators to the point where
they don't support more than one.

BTW, I'm not all that attached to "canonical"; if anyone has a better
name for either the discovery target or the default operational
target, please send it to the list.

> you can reject the login, with a `no default' reason code (TBS).

At that point, the initiator can either give up, or log into the
discovery/canonical target, do SendTargets, and then log in.  As long
as we make the latter a MUST, I'm OK with the default target.  If the
latter is a SHOULD, then I'm not.

> concepts, but the users should be able to get going without them.

Absolutely.  However, there are two points of view from which one can
count a concept as "additional":

1. The FC SAN point of view:  People who have already deployed Fibre
   Channel, and wish to extend their storage network to a greater
   number of hosts.  To these people, iSCSI/IP is just another network
   interface.  These people have the storage management skills, and
   many have the IP networking skills as well.  Most of the storage
   vendors themselves probably belong to this category.

2. The intranetworking point of view:  People who don't know storage,
   but are accustomed to managing web servers, file servers, and
   the like.  To these people, an iSCSI device is just another server,
   and should be managed as such.  These people have the IP networking
   skills, and are not likely to be familiar with managing storage.
   Most potential iSCSI customers probably belong to this category.

Taking the FC SAN point of view might be expedient for many vendors,
but in the end, group #2 is probably much bigger than group #1, and
my guess is that the storage folks know more about networking than
the networking folks know about storage anyway.

So I think that to leverage existing IP management skills, taking
a server point of view will be easier to handle than a storage device
point of view.  The distinctions are perhaps subtle, but putting things
like multiple targets into the storage array probably counts as an
additional concept to the FC SAN folks, but not to the networking
folks.

-- 
Mark A. Bakke
Cisco Systems
[email-masked]
763.398.1054

---

From: Stephen Bailey <[email-masked]>
Date: Tue, 15 May 2001 12:49:40 -0400

> Any other ideas?

My suggestion was to not supply any target name for a discovery login
(which is how a discovery login would be distinguished) and call
`iSCSI' the `default' (operational) target.

Steph

---

From: Mark Bakke <[email-masked]>
Date: Tue, 15 May 2001 11:45:37 -0500

How about using the term "well-known target" instead of
canonical target?  It would fit in well with the networking
world.  Any other ideas?

--
Mark

-- 
Mark A. Bakke
Cisco Systems
[email-masked]
763.398.1054

---

From: Mark Bakke <[email-masked]>
Date: Wed, 16 May 2001 14:14:35 -0500

That kind of makes sense.  If we limit SendTargets to the discovery
session, and disallow SCSI commands to the discovery session, then
we really don't have a target at all from SCSI's point of view; but
we do have a "special" target from iSCSI's point of view since we
are doing the full login to it.

So if we used

  TargetName=

we would get the discovery target; if we used

  TargetName=iSCSI    # Note: this is case-insensitive

we would get whatever default target the iSCSI device wanted
our initiator to see.  Even if we do end up supporting both, I
guess I'd rather see a target name, rather than leaving it
blank.  How about

   TargetName=discovery

BTW, nobody else has spoken up for having this default target
that would avoid having to use SendTargets if there were not
multiple targets available to an initiator.

Who plans to make use of this?  I don't mind putting it in
(we had sort of implied the functionality before), but if
it is not in anyone's plans, I'd rather go for a simpler spec.

-- 
Mark A. Bakke
Cisco Systems
[email-masked]
763.398.1054