Subject: RE: iSCSI Target Reset

From: "John Hufferd" <[email-masked]>
Date: Sun, 22 Apr 2001 14:52:25 -0700

This is at least better.  But I do not have the issue of it being vendor
unique.  This is a shut down and restart of the complete Target, and will
probably be part of the vendors' operator console or their own remote
support functions, it is not clear that it needs to be a general management
function that works the same on all iSCSI Storage Controllers.

Many of the major Storage Controller do not support this feature today.

I do not believe that most SNMP implementations are very secure.  Most
folks do not want to have a changeable MIB until they have secure SNMP, and
even though there is a version of SNMP that has security features, this has
not been well supported.

I do NOT think that Target Reset should be in the base iSCSI protocol, but
I think it is reasonable to hold this discussion apart from the base
protocol document, and the question should be asked if this is a general
management function or a vendor specific console or remote support
function.

.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
(408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
Internet address: [email-masked]

"Dillard, David" <[email-masked]>@ece.cmu.edu on 04/22/2001
08:13:49 AM

Sent by:  [email-masked]

To:   [email-masked]
cc:
Subject:  RE: Target Reset

John,

I understand the danger of issuing a target reset and I agree that it
should
not be a part of the an initiator's normal error recovery procedure.
However, looking at this from a management perspective I'd like to see a
standardized way of resetting a target.  I don't want to see a variety of
vendor unique methods of resetting targets sprout up.

If resetting a target using the protocol is not desirable from your
perspective would incorporating this feature into the MIB be acceptable?
(MIBs are for management after all)

----------------------------------------------------------------
David Dillard                          [email-masked]
Management Software Group
Adaptec, Inc.                          www.adaptec.com

-----Original Message-----
From: John Hufferd [mailto:[email-masked]]
Sent: Sunday, April 22, 2001 4:59 AM
To: [email-masked]
Subject: Target Reset

I thought we had a number of discussion previously about Target Reset (Warm
or Cold).  I thought there was general feeling that this command is so
dangerous that it should not be supported by iSCSI.  The long distance
capability of iSCSI makes the risks involved unmanageable.  There should
only be an Admin way to do this.

Some folks have said that we could permit it and have special authorization
etc.  This would probably cause a separate section in the spec. to define
the authorization approach,  and what ever other security is needed to
prevent this from inappropriately being used.  All for what purpose?  This
can not be part of error recovery from a normal initiator.  The wide spread
effect is too great for that.

I would like to hear from the list about their feeling on this item.

.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
(408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
Internet address: [email-masked]

---

From: "Elliott, Robert" <[email-masked]>
Date: Mon, 23 Apr 2001 12:51:50 -0500

[iSCSI added to subject]

> From: Eddy Quicksall [mailto:[email-masked]]
> Sent: Monday, April 23, 2001 11:44 AM
> Cc: [email-masked]
> Subject: Re: Target Reset
> 
> I am wondering how clustering will work on NT without some 
> sort of reset.
> 
> On parallel SCSI, NT will issue a SCSI BUS RESET to break reservations
> during a challenge for the quorum drive.
> 
> On iSCSI, there is no full equivalent to the SCSI BUS RESET so I would
> assume the NT driver would have to issue a TARGET RESET to 
> each target that it is supporting.
> 
> How would you propose this would be done without a TARGET RESET?
> 
> Eddy

LOGICAL UNIT RESET breaks the reservations with the correct scope.

According to a WinHEC presentation, the new STORPORT port driver will 
attempt resets in this order:
1. LOGICAL UNIT RESET
2. TARGET RESET
3. Bus reset 

LOGICAL UNIT RESET isn't supported by all existing logical units, so 
a fallback scheme (with its unfortunate side effects) is necessary.  
This is mandatory as of SAM-2 revision 16, so new devices should 
start supporting it (on all protocols).

TARGET RESET is the first level of fallback (medium hammer).  This 
is optional for protocols to support as of SAM-2 revision 16.  
Protocols that don't support it require their logical units to play 
nicely and support LOGICAL UNIT RESET.  Protocols that do support
it leave themselves open to all the problems Fibre Channel went
through.

Bus reset is the final level of fallback (big hammer), which doesn't
exist on serial busses.

A miniport driver for SCSIPORT (the existing port driver) can convert 
a "target reset" request into LOGICAL UNIT RESETs for each logical 
unit known to that system to help limit the damage.

From what I understand, Linux also relies on target resets and 
bus resets and will need some mid-layer work to play well in a fabric.

---
Rob Elliott, Compaq Server Storage
[email-masked]

---

From: "Dillard, David" <[email-masked]>
Date: Mon, 23 Apr 2001 16:25:02 -0400

> function that works the same on all iSCSI Storage Controllers.

From a management vendor's perspective they want to be able to fully manage
storage devices so the less vendor unique stuff there is the better.  From a
customer's perspective many customers do not like having to spawn off
separate consoles within the main management application (e.g. products from
Veritas, Tivoli, et.al.) to manage specific features of a device.  So,
standardizing features makes it easier for the manage app vendors to support
those features, making some customers happier.

> folks do not want to have a changeable MIB until they have 
> secure SNMP, and even though there is a version of SNMP that has security 
> features, this has not been well supported.

This is an excellent point (slapping forehead! - should have thought before
I sent my message).  Target reset is obviously something that must be
secure, so if it were put into the MIB it could only be read/write if the
method of setting the value were secure.

---

From: Charles Monia <[email-masked]>
Date: Mon, 23 Apr 2001 16:11:53 -0700

Hi:

These seem to be implementation decisions. I don't see how that justifies
removing support from the protocol.

Charles

> Sent: Monday, April 23, 2001 2:34 PM
> To: Santosh Rao
> Cc: [email-masked]
> Subject: Re: iSCSI Target Reset
> 
> 
> 
> Absolutely not,  Why would we think that impacting 32 different other
> initiators is an OK thing to do.  By the way there are lots 
> more Initiators
> possible with FC on Shark, and would hope that there would be 
> even more
> with iSCSI.
> 
> I have been told that these large Storage Controllers do not 
> support Target
> Reset today.  So I see no loss in not supporting such an item in iSCSI
> especially since many Initiators will be beyond even the distances and
> mischief that is possible with FC.
> 
> .
> .
> .
> John L. Hufferd
> Santosh Rao <[email-masked]>@ece.cmu.edu on 04/23/2001 
> 01:24:02 PM
> 
> Subject:  Re: iSCSI Target Reset
> 
> 
> 
> "Dillard, David" wrote:
> >
> > When will STORPORT be generally available?  The latest 
> STORPORT document
> > that I found on the MS web site is version 0.6a, dated 
> March 18, 2001.
> > Given this it seems like STORPORT might not be available 
> soon.  In that
> case
> > do you know what happens with the current drivers?  Are we 
> going to be
> > telling customers that if they want to use iSCSI and NT 
> clustering they
> have
> > to update to Whistler?
> 
> 
> [One would hope that this list does not turn into a Microsoft
> release/product discussion mailing list (?) ]
> 
> Without going into specifics of A certain O.S., does it suffice to
> require that iSCSI not break existing legacy SCSI applications ?
> 
> If the above is a valid requirement, then, knowing that legacy
> applications continue to use SCSI-2 Reserve/Release and the 
> target reset
> as a mechanism of breaking SCSI-2 reservations, should'nt 
> iSCSI continue
> to support the target reset ?
> 
> - Santosh
> 
> 
> 
>

---

From: [email-masked]
Date: Mon, 23 Apr 2001 19:31:24 -0400

I agree with Charles that this is an implementation
issue.  If a Shark wants to reset all 32 adapters
when it receives a Target Reset on one of them, that's
a Shark implementation decision.  It's completely valid
to reset only the adapter that the Target Reset is
received on (common Fibre Channel behavior) or
only the iSCSI target to which the Target Reset is
addressed if there's more than one Target behind  
the adapter.

As for leaving things out of iSCSI - the default modus
operandi should be to put in everything that's described
in SAM2 unless we can convince T10 to take the feature
out of SAM2.  Let's not go deciding to cast things out
of SCSI on T10's behalf.

Thanks,
--David

> From:	Charles Monia [SMTP:[email-masked]]
> Sent:	Monday, April 23, 2001 7:12 PM
> To:	[email-masked]
> Subject:	RE: iSCSI Target Reset
> 
> Hi:
>

---

From: "Elliott, Robert" <[email-masked]>
Date: Mon, 23 Apr 2001 19:19:42 -0500

It's not a very good situation when each device chooses the
interpretation of TARGET RESET that it thinks is appropriate.  
IBM's Shark might choose a different hack than Compaq's
RA8000.  How is software supposed to make sense of this?

The rule in SAM-2 is that TARGET RESET resets every logical
unit (subject to access controls, if implemented).  The fact
is that in Fibre Channel, not many multi-LUN multi-port
targets followed that rule.  The result is that software 
cannot tell what's going to happen and may have to handle
targets from each vendor differently.  This is not very 
interoperable.

Charles suggested in the T10 meeting that we allow it to 
be implemented as a no-op rather than let protocols drop 
support for it.  That doesn't help software like Windows that 
does expect certain effects - a no-op implementation 
would break clustering.  By removing it from the protocol, 
software is forced to find a suitable replacement (e.g. use 
LOGICAL UNIT RESET or switch to persistent reservations).
In Windows, this can be done at the port driver level
(STORPORT improving on SCSIPORT) or at the miniport level
(convert each target reset request into multiple LOGICAL UNIT
RESETs).

Note that other protocols like NFS and HTTP over IP don't
seem to have "server resets."

The recent SAM-2 change was expressly designed to encourage 
iSCSI and SRP to drop support for TARGET RESET.  Please don't 
keep it because you think T10 would be offended :-)

---

> From: [email-masked] [mailto:[email-masked]]
> Sent: Monday, April 23, 2001 6:31 PM
> To: [email-masked]; [email-masked]

---

From: Charles Monia <[email-masked]>
Date: Mon, 23 Apr 2001 20:36:20 -0700

Hi:

Sorry to reopen old issues, but unfortunately that seems necessary since
something has been lost in translation.

My concerns about TARGET RESET center on three areas:

1.  Adverse side effects at the transport layer that could affect other
users.
2.  Similar adverse side effects on the affected devices,
3.  The impact on legacy software.

The gist of my opinion on the first issue, (as expressed on the T10
reflector) is as follows:

"> > > > The ... issue is whether mechanisms, such as terget reset, 
> > > > are appropriate for a given transport.  In my view, the only 
> > > > immutable requirement is to preserve the transport-independant 
> > > > part of the semantics.  The definition of transport-specific
> > > > side effects is best handled in the appropriate transport
> > > > specification."

The point of the above is that, in my view, a protocol specification has
leeway to define the protocol-specific side effects in a rational and benign
manner provided that the observable effects on the attached devices are
preserved.

Regarding adverse device-level side effects, I also stated that:

"....restricting the operation [target reset] means providing hooks so that
only a
trusted class of initiators can perform the function.  It's a bit like
controlling access to a file so that lots of users can read it but only a
trusted few can perform a write or delete operation." 

Finally, with regard to legacy implementations, my main intent was to avoid
the situation where an operation that was previously legal becomes illegal.
In that regard, I also recall suggesting that the function be treated as a
LUN RESET broadcast to all the logical units to which the initator had
access privileges.  That would also support the notion of preventing adverse
effects on other users as well. 

At any rate, since there is a proposal to make LUN RESET mandatory (see
below), I believed this was a reasonable and consistent alternative. I
therefore felt (and continue to believe) there is no justification for
making the function optional.

Anyhow, thess views did not prevail at the meeting in question. For that
reason, the so-called NOP proposal was made as a last ditch effort to
accomodate legacy implementations by preserving a measure of backwards
compatibility (albeit token compatibility). In that regard, it seemed the
lesser evil.

> > out of SAM2.  Let's not go deciding to cast things out

So, I guess that anyone wishing to support a change in the spec is, of
course, free to pursue it in that forum. 

Charles

PS: The proposal referenced above can be found at:
ftp://ftp.t10.org/t10/document.01/01-015r2.pdf

> From: Elliott, Robert [mailto:[email-masked]]
> Sent: Monday, April 23, 2001 5:20 PM
> To: [email-masked]; [email-masked]'; [email-masked]'

---

From: "John Hufferd" <[email-masked]>
Date: Mon, 23 Apr 2001 22:52:41 -0700

David,
you described the action in FC terms.  In iSCSI terms, I think the Target
Device would be a complete Symmetrix.  Do you think that is correct?  What
does Symmetrix do today to support Target Reset?  And how do you think it
should act in the future?  Surly you do not want the complete controller
reset, do you?

What type of flexibility do you think T10 gives you in this situation?

.
.
.
John L. Hufferd

[email-masked]@ece.cmu.edu on 04/23/2001 04:31:24 PM

To:   [email-masked], [email-masked]
cc:
Subject:  RE: iSCSI Target Reset

Thanks,
--David

---

From: [email-masked]
Date: Tue, 24 Apr 2001 11:43:34 -0400

Starting with John's comments:

> described the action in FC terms.  In iSCSI terms, I think the Target
> Device would be a complete Symmetrix.  Do you think that is correct?

If existing cluster software continues to issue Target Resets, then
no, because such Target Resets must not reset the complete
Symmetrix for correct operation - they don't currently.

> What does Symmetrix do today to support Target Reset?

Reset the port on which it arrives (SCSI or FC), and generally
configure things so that anything else that could be negatively
impacted by side effects of that reset is not connected to that port.
IIRC, AIX clusters use Target Resets.

> And how do you think it should act in the future?

Whatever it takes to keep existing cluster software happy.

> Surly you do not want the complete controller reset, do you?

If "controller" means the entire array, then of course not -
don't be ridiculous.

> keep it because you think T10 would be offended :-)

If dropping support for it breaks existing clustering software, that doesn't
seem like a good idea.  Such breakage will be viewed as an iSCSI
problem (it works fine on Fibre Channel ...), and will not contribute
to the adoption or acceptance of iSCSI :-(.

I appreciate T10's intention to obsolete Target Reset and would offer
two possible courses of action:

- If someone writes up a convincing case (Internet-Draft) for being
	able to map things that would have caused Target Resets to
	LUN Resets in iSCSI device drivers and the like in a fashion that
	will keep existing cluster software working (and unaware that
	this change has happened beneath it), then dropping
	Target Reset may be feasible.  This is akin to the discussion
	we went through about mandating Autosense and dropping
	Contingent Allegiance (it's easy to make a CA device do
	Autosense transparently), except that it'll have to consider what
	the various clustering software implementations actually do,
	as opposed to all the possible ways in which Target Reset
	could be used/abused.  Anyone want to volunteer?

- Put in some SHOULDs and SHOULD NOTs encouraging the use
	of LU Reset in place of Target Reset, and making Target Reset
	either OPTIONAL or NOT RECOMMENDED to implement.

The second bullet seems to match what T10 has done in practice
- this feature is not a good idea, but can't be removed from the
specification, at least not yet.

One more question for T10 - do the tape folks still need Target
Reset to whack an uncooperative robot/drive into submission
or is LU Reset good enough?

Comments?

Thanks,
--David

---------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 42 South St., Hopkinton, MA  01748
+1 (508) 435-1000 x75140     FAX: +1 (508) 497-8500
[email-masked]       Mobile: +1 (978) 394-7754
---------------------------------------------------

---

From: "Jim Hafner" <[email-masked]>
Date: Thu, 26 Apr 2001 07:46:12 -0700

John,

I'm really getting lost in this thread, but let me throw in my 2 lire.

Has anybody read the latest draft of SAM-2 rev 16? It says:
(a)Target Reset is not more than a set of Logical Unit Resets (see 6.6)
[Earlier drafts use the term "hard reset"-- this is no longer there.]
(b) Target Reset shall only affect the logical units to which the initiator
has access controls rights (see 6.0) [this is exactly what Shark does
though their "access controls" are pre-standard]

In short, I think we've spent a lot of time debating an issue that is no
longer a problem: the undesirable side effects of Target Reset just aren't
in the spec any more!

There *is* language that Logical Unit Reset "shall perform any additional
functions required by the applicable standards" (namely, the protocol
standards).

So, we
(a) can allow Target Reset in iSCSI without major concerns (but it would be
nice to "discourage it's use" as mentioned in SAM-2, rev 16, 6.6)
(b) should state explicitly that Logical Unit Reset has no additional
effects in iSCSI beyond those specified in SAM-2, rev 16, 5.7.7.

Jim Hafner

John Hufferd/San Jose/IBM@[email-masked] on 04-26-2001 01:24:17 AM

To:   Julian Satran/Haifa/IBM@IBMIL
cc:   [email-masked]
Subject:  RE: iSCSI Target Reset

I received the following statement from the IBM Shark development team.

" For Fibre Channel, Shark supports Target Reset and it is used by almost
all the hosts as well.

Shark supports Target Reset by resetting all the LUNS that are configured
to the host that issues
the Target Reset.   The Target Reset, therefore, will not affect the LUNs
seen by other initiators,
unless they are in sharing the same LUNs. "

Therefore, it probably make some since to state, in the draft, that this
kind of approach should be considered by iSCSI implementers.

I do not think this is a large controller only problem, since with iSCSI,
lots of different desktop and Laptops will be getting at RAID arrays, which
might have previously been attached to only 2 Host via SCSI BUSes or a few
more hosts with Fibre Channel.

IOW, Target Reset has a larger impact then before, even on Storage
controllers that are in the "Mid Range" (and lower), which previously may
not have even worried about the issue.

So the mention of an approach such as the above might be a useful note in
the Spec.

.
.
.
John L. Hufferd

Julian Satran/Haifa/IBM@[email-masked] on 04/26/2001 12:41:04 AM

To:   [email-masked]
cc:

I agree that this is an implementation decision.  However I wonder it it
won't be fair to tell the guy trying to do reset that although everything
is fine the reset was not performed - and do this in a way that does not
harm legacy initiators.

Julo

[email-masked] on 24/04/2001 02:31:24

Please respond to [email-masked]

To:    [email-masked]
cc:

Thanks,
--David