Subject: Review of the 07 draft

From: "Robert Griswold" <[email-masked]>
Date: Fri, 3 Aug 2001 14:33:56 -0500

Julian:

First, great work on this initiative and please forgive my huge email.  

We have undertaken a review of the 07 version, and even though
Crossroads will be at the meetings in London, we thought it would be
prudent to post these observations on the current specification.  If we
have completely misread the document, or have missed resolutions that
have come across the reflector, we apologize up front.  I have also sent
you some editorial only comments for your review.

++++++++++

Section 1.2.2.1 (and others):
The third paragraph up from the bottom of page 19 refers to the action
that a target should take when receiving commands outside of a range,
that are not correctly marked as immediate, with the right flags set.
The question is in regards to the MUST instruction on "silently ignore".
Assuming the I_T nexus is established, the devices are communicating and
the target receives what could only correctly be termed a protocol error
from the initiator it has established communication with, why would the
target not respond with an error back?  If in fact the error is
transport related, and the initiator really did send the correct PDU,
then the target being mute on the reception of the this command leaves
the initiator guessing.  There are other points in the specification
where the target is instructed to "silently ignore."  

Section 1.2.2.2:
The fifth paragraph indicates that a "large difference" between StatSN
and ExpStatSN may indicate a problem, but the number required by the
specification as the maximum before an initiator must take action is 2.1
Billion!  This seems like a very big number, and most implementers would
want to take action long before this limit was reached.  What is the
thinking behind this number?  Is there something we aren't
understanding?

Section 1.2.5:
On page 25, the second full paragraph indicates that the definition of
the use of target tags is not specifically specified by the protocol.
While we see possible uses for Target Transfer Tags, we would like to
understand if the author's feel that limiting the interpretation of its
use to that described in 2.17.4 would be helpful.  Clearly this 32-bit
field plays little role in the overall usefulness iSCSI command
protocol, so someone must have felt that it would have been a great idea
to have this in the specification.  Can that be described?

Section 1.2.6
This section covers the topic of connection termination, which is also
covered in the discussion of the Logout command in section 2.14.  There
seems to be some mingling of the concept of TCP logout with the concepts
around termination of a communication channel for storage protocols.
The concept of sending a logout, then waiting for the receiving end to
do some clean-up, flushing, and or additional communication seems odd to
the command.  The Logout directive should require no further
communication, and should put neither initiator nor target into the
state of expecting any such communication.  We would like to see the
Logout command (not the request for a logout) be free of any
expectations once sent, or received.  It seems logical that since the
initiator is the responsible party for sending this command (we would
also like to begin some discussion on a target only Logout command, like
SRP is working on now) that it would have done all of the required
cleanup and preparation before issuing the command.  Implementation of
target code to make decisions on what outstanding commands it should or
shouldn't give some credence to before breaking the connection seems out
of place. 

Section 1.2.7
On page 28, in the fourth to last paragraph in this section, there is a
description on how a target can assume an iSCSI Initiator Name, as
learned through authentication.  This is a confusing paragraph, and we
cannot determine if it is implying that the target should use the
initiator name it has been given through some service, or that it was
authorized to use by an initiator that discovered its iSCSI functions.
Can this paragraph be further expanded to clarify how the name was
determined?

Section 1.2.9.1
The last paragraph on this section has as 42 words in it's first
sentence, with the word "message" being used six times.  Suggested
rewording:
"Relying solely on the "message length" information from the iSCSI
message header may make it impossible to find iSCSI message boundaries
in subsequent TCP segments; due to the loss of a TCP segment containing
the iSCSI message length."

Section 1.2.9.2
The second full paragraph's last two sentences have odd wording.
Suggested rewording:
"In practice, though, iSCSI is not expected to handle infinitely long
streams; stream addressing will wrap around at 2**32-1."

The first sentence of the next paragraph is difficult to interpret.
Suggested rewording:
"This model assumes that the iSCSI layer will deliver complete PDUs to
underlying layers in single (atomic) operations."
You may also want to consider adding words that explain the reasoning
behind this assumption, as some who don't care about those layers may
not understand why it would be a reasonable assumption.

The rest of this section uses the word MUST a few times in the
description of actions or implementation assumptions.  While the use of
the word MUST may make complete sense to the topic, and the points they
are being applied to, it seems out of place for an optional layer of the
model.  If the Synch and Steering layer is truly optional, then how can
the specification make claims to what MUST happen?

Section 1.5.3
The second paragraph on page 39 uses the term nexuses, but does not
clearly build the relationship that the ISID rule is attempting to
explain.  Can the level (I_T, I_T_L, I_T_L_x) of nexus be identified
that is being referred to here, to help clarify the ISID rule mentioned
in the previous paragraph?

Section 2.2.2
The definition of the AHS (see below, 2.2.3.1) seems to be covered as a
global PDU property, but only op-code 0x01 (SCSI Command) seems to make
use of it.  Since the AHS seems to be only needed to encapsulate an
extended CDB, then why define its possible existence for all PDUs?  Can
the AHS be used in other commands?

Section 2.2.2.4
This section defines the behavior of non-specific fields in op-code use,
but not completely.  One portion of the field is explained (P/F bit),
but the use of the rest of the field is left to the reader to find in
the other op-codes.  Would it make more sense to pull the definition of
the P/F bit out of this section, then simply tell the reader that the
other bits are defined in the op-code specific sections?  Maybe tabulate
the P/F usage in this section, to the op-code specific use?

Section 2.2.2.7
The second sentence of this paragraph defines that the LUN field can be
used for "some other purpose."  While there may be a good reason for
overloading this field, we would like to see some more information on
the nature of this use.  What about inserting "opcode-specific" in from
of the word "purpose".

Section 2.2.3.1
The inclusion of a bit that identifies if the CDB is extended should not
live in an optional header.  Expecting that all implementations would
correctly code the use of an optional header to return critical data
that should be identified with the required portion of the CDB seems out
of place.  We would suggest that the use of a reserved bit or a bit in a
reserved byte be used, from the BHS, be used to identify an extended
CDB.  Putting the spillover of the CDB into the AHS (section 2.3.6) is
not an issue, but flagging if there is spillover should not be in the
AHS.

Section 2.4.3
This section has seen a lot of discussion, and we agree with the
position that Rob Elliot and David Black took on the disassociation of
the iSCSI responses from the ASC/ASCQ SCSI status codes.  While the
discussion of ACA and it's relationship to iSCSI may not be resolved
quickly, there needs to be a way to avoid polluting the expectation of
the SAM status to the devices with the iSCSI response codes.

Section 2.5.1
This section seems to build on the work done in FCP for out of CDB
delivery of task management.  But the behavior of a target on receipt of
a Target Cold Reset should be similar to that of a Logout (assuming
behavior as explained above in comments on section 1.2.6), where the
target returns as little information as possible, and does not attempt
to clean-up or respond upon reception of the command.  Also, this is the
only section in the document where the term MIB is used, and since
management of iSCSI is outside of the scope of this document, we would
prefer to see the last sentence in the paragraph on Target Cold Reset
removed.

Section 2.8
While the section on Bookmarks is brand new, and will probably go
through more definition, the double definition of this field should be
removed.  If it stays as Bookmark, and it is optional, then it is not
reserved.

Section 2.8.5
This section allows (in the first paragraph) large binary items to be
encoded as hex or decimal.  This discussion is ongoing now, and we
believe the encoding should be hex.

The third to last sentence in this section alludes to the ability to use
key=value pairs to perform active operations.  Since iSCSI active
operations are interpreted by us as related to SCSI operations, what is
being said here?  Does this mean that an initiator could perform
mode-parameter or task management operations inside of Text Messages?
If so, then more definition of this behavior is needed.

Section 2.9.5
The second paragraph attempts to instruct the reader to order response
key=value pairs in the same order they were received, but provides
little direction on why, or what possible consequences would be if it
were not done in that order.  Why is this sentence needed, except for
instruction on best practices?

Section 2.10.1
The instructions to targets on supporting multiple connections run up
against the minimum requirement of targets to support only one
connection.  If this section is a better reason to require at least two
connections at a minimum, then the specification should call that the
minimum.  If initiators are going to expect to be able to use this
command, but targets that only support one connection would refuse it,
than what good is this instruction going to be?  Would a target designer
who develops an iSCSI target create the ability to open two connections
to support only this situation, and not expand the interface to run two
connections in Full Feature Mode?

Section 2.12.4
If the limit of reflected data is 4096 bytes, then the instruction to
the initiator should not be "avoid sending", it should be "MUST NOT"
send more than 4096 bytes.  Is there a reason why the initiator would
want to send more than those number of bytes?

Section 2.14
This section was discussed above in relationship to section 1.2.6.  The
Logout command should be simple and immediate, with no further work
needed by the target then the clearing of internal buffers, and the
reporting on the next Login (CHECK CONDITION) that the Logout was
performed. 

Section 2.16
The last sentence in this section seems at odds with good error
detection for targets, and at odds with section 2.19.1.  If a SNACK
received has and error, whether or not it was a miss-directed PDU, or a
transit error, why would a target just silently ignore it?  Should it
use the SNACK Rejected code of the Reject response, or at least tell the
initiator something?  Besides, the use of the word "must" seems
inconsistent with what this sentence is trying to say.  If there is a
good reason for a target to ignore this, then it should be mentioned,
and the directive should be "MUST" if the expectations of the initiator
are set that way.

Section 2.16.1
The last paragraph of this section is confusing.  We cannot determine
how it should be rewritten, since the intent is not clear.  Why would
the target issue a message requesting Logout simply because it cannot
process as SNACK that is optional anyway?  How does the initiator know
if the target supports in-connection or out-of-connection recovery with
regards to what the target does after discarding the SNACK?

Section 2.17.3
What is the purpose of the last sentence on page 87?  It seems to
indicate that there is a good reason (besides the obvious) why the
Desired Data Length should not be zero, but falls short of explaining,
or directing the reader with a "MUST," or "SHOULD."

Section 2.17.4
This topic was discussed above in section 1.2.5.  

Section 3
We would like to see the explanation in this section clarified with
regards to the use of mode parameter changes for SCSI settings, versus
iSCSI settings.  Suggested rewording for the first sentence of the
second paragraph:
"The mode parameters for iSCSI behavior cannot be set by SCSI mode-set
but can be retrieved by SCSI mode-sense commands."
Suggested additional wording:
"SCSI mode parameters (non-iSCSI specific) cannot be set or retrieved
from within iSCSI mode page commands."  -  Or something like that.

Section 4.2
Under the key SecurityContextComplete=yes, there is a description of
what the party sending this key should do, finishing with the phrase
"until the handshake is complete."  What is the scenario for avoiding a
lock condition in this handshake?  Should the party use some standard
timeout? Where is this defined?

Section 7
The second paragraph in this section defines an assumption of the
specification.  Potential manufacturers of iSCSI targets will want to
know under what circumstances will the saving of sense and status
information differ than those already done for FC or SCSI operation;
specifically with regards to iSCSI Logout and iSCSI task management
commands.  Can this paragraph have more details added?

Section 7.1c
This description of the use of the Retry bit seems to put the onus of
interpretation of the Retry bit on the target for command replay.  Is it
at all possible for a Replay request to get issued, within the scope of
an initiator delaying its acknowledgement, such that the command
actually did get completed, but the Retry bit was set too soon?  Would
one assume that an initiator just avoids doing something like this?  

Section 7.6
This section uses the term "ULP timeout", but there is no definition or
reference to where it can be found.

Section 7.9
By recommending (in the first paragraph) that a connection be tested in
HA environments with iSCSI NOP-ping command, aren't the authors
recommending that designers use bandwidth consuming methods for
determining this?  If you assume a multiple path connection (fabric) for
such iSCSI implementations, then couldn't a bunch of initiators
potentially flood the fabric with such commands to "recognize a failing
connection?"

Section 7.11.1 and 7.11.2
Specifically, what time-outs are being referred to here?  The time a
target or initiator should or might wait for acknowledgement from the
other node that error recovery is being attempted?

Section 7.11.3
This section seems to not agree with the methods for completing a Logout
command as discussed earlier.  Item number (2) in this section indicates
that the initiator should treat the reception of an Asynchronous Message
requesting a Logout command to that of a TCP failure, yet the target is
instructed (section 2.14) to treat the reception of a Logout command as
an opportunity to complete any commands "it deems necessary."  Again,
the need for a Target Logout and Initiator Logout seems apparent.

Thanks,
Bob

Robert Griswold
Technologist
Crossroads Systems, Inc.
512-928-7272

---

From: "Julian Satran" <[email-masked]>
Date: Sun, 5 Aug 2001 17:52:01 +0300

Robert,

Thanks for you kind opening and for taking the time to read it.
Comments in text.

Julo

"Robert Griswold" <[email-masked]> on 03-08-2001 22:33:56

Please respond to "Robert Griswold" <[email-masked]>

To:   Julian Satran/Haifa/IBM@IBMIL
cc:   "Bill Moody" <[email-masked]>, <[email-masked]>
Subject:  Review of the 07 draft

Julian:

++++++++++

+++ This point has a long history remains of which you can find in the mail
archives.  I and several other authors where inclined to "silently discard"
whatever can't be repaired (e.g., bugs in the initiator, delayed packets or
packets arriving late) and explicitly reject only what can be repaired.
Debugging ease can be achieved by other means (event logs etc.). However
many of our colleagues on the list wanted error rejects to be explicit
whenever it is feasible and that is what you have now except for the cases
in which discards are due to a "legitimate packet" arriving late (e.g.,
from a previous transmission attempt that is already recovered).
Discards are also better in preventing DOS attacks by stream replay++++

Section 1.2.2.2:

+++ That is very much dependent on implementation and infrastructure (round
trip delay). We assumed that at several hundreds you will start to worry :
-)) +++

Section 1.2.5:

+++ We envisioned TTTs to be used by the target to identify parts of the
data stream that arrive in response of an R2T. Additionally they identify
NOP-Outs that are sent in response to a polling Nop-IN and in general when
valid indicate that the target expects something and identify the response
(link it to the "cause") for the target. We mentioned that the protocol
does not care about the value mainly to stress that, unlike ITT, the
protocol has no uniqueness requirement for them and whenever they appear
the LUN is also present and can serve as an additional differentiator.
++++

Section 1.2.6

+++ Logout has also history. We started without a logout but realized soon
that we will need a synchronized logout to enable command allegiance change
for recovery (that is possible only after logout). Besides it is a simple
and painless way to do a clean shutdown of a connection. As for a target
only logout that is what the Asynch Message does (after some timeouts)
Please keep in mind that there is also a fine interplay between logout and
recovery and that both initiator and target may decide to start some
recovery action+++

Section 1.2.7

+++ This is getting thrown out (it is already) considering that T10 has
gone a long way to expand the addressing of third party command source and
destinatiantion +++

Section 1.2.9.1

++ OK +++

Section 1.2.9.2

+++ OK +++

+++ I've made it:

This model assumes that the iSCSI layer will deliver complete PDUs to
underlying layers in single (atomic) operations.  The underlying layer doe
not need to examine the stream content to discover the PDU boundaries.

But please note that this whole sectionmight go away :-)

++++

+++ it says that IF you doit that MUST be the way to do it ++++

Section 1.5.3
+++ I_T - will fix the text +++

Section 2.2.2
+++ There is another one - BiDirectional command fields. +++

Section 2.2.2.4

+++ Good point I'll remove the P/F text - it appears wherever it is used.
+++

Section 2.2.2.7

+++ I've changed it to:

1.1.1.1   LUN

   Some opcodes operate on a specific Logical Unit. The Logical Unit Number
   (LUN) field identifies which Logical Unit.  If the opcode does not
   relate to a Logical Unit, this field either is ignored or may be used in
   an opcode specific way.  The LUN field is 64-bits and it is to be
   formatted in accordance with [SAM2].

 ++++

Section 2.2.3.1

+++ That must be a "misreading" - We are talking about a coded field one of
the values is Extended CDB. The whole code is the AHSType.  And if you have
an extended CDB this header is not optional - it is the way to code the
extended CDB (no other way is decribed)
++++

Section 2.4.3

+++ It is going to change. We have to find also a good code (ASC/ASQ) for
Insufficient immediate data
++++

Section 2.5.1

+++ I can refer to some "opaque" management structure instead of MIB
As for not  returning anything this is acceptable for Target Cold Reset -
where the TCP connections are teared down but not for those that leave the
connections
standing +++

Section 2.8

+++ The Bookmark field is a bookmark if the B bit is 1 else it is reserved
+++

Section 2.8.5

+++ We think that hex will be the preferred encoding used but sometimes
people will copy decimal numbers. Once you have to have a decimal
conversion routine it makes little sense to force hexadecimal only by
dictate
+++

+++ I will change it to "long lasting operations" and those can be vendor
specific +++

Section 2.9.5

+++ It is not a MUST but it was thought by many (with some vehemence) as
helping matching.  IMHO it has no meaning and would suggest removing it.
+++

Section 2.10.1

+++ The text only says that if you are going to support connection recovery
you should be able to support opening a second connection in order to clean
the first connection and close it. It also explicitly says that you don't
have to support 2 connections in full feature phase. And even on 2 TCP
connections it is just a recommendation - using IP-over-pidgeons can be an
good alternative for forcing a TCP connection to close. On a more serious
note the second TCP connection can be on the same physical connection.
+++

Section 2.12.4

+++ The implications of saying MUST NOT would be that the target has to
check and to reject with a format error. We don't want this do we? +++

Section 2.14

+++ the answer is the same as before +++

Section 2.16

+++ A SNACK that is not supported is answered either by an Asynch Message
asking logout - for failover - or total silence - in which case we assume
SCSI will timeout and do its thing (abort, reissue the command or whatever
neded.
There no need to answer. A badly formed SNACK is rejected as any othe badly
formed PDU (although even this is perceived by some as excessive when it
does not help the initiator take repair action).
++++

Section 2.16.1
+++ The target should request logout if it supports connection recovery but
not
data snack +++

Section 2.17.3

+++ the reason is in 2.17.1 the number of R2Ts in a command should be less
than x'ffffffff'
+++

Section 2.17.4

+++ I hope the answer was adequate +++

Section 3
+++ OK +++

+++ I think we can leave this to be implementation dependent. If I would
have to chose a "way out" would say limited number of text exchanges rather
than a timeout.  ++++

Section 7

+++ The status and sense are saved for command recovery while the status is
not acknowledged. Te recovery may involve status only, or some lost data
(detected) when status arrived) or the whole command replay. Details are
... I think we have now more than 50 pages of detail +++

Section 7.1c

+++ I will come back to this later. +++

Section 7.6

+++ It is not defined here - it is assumed that when we let SCSI commands
dry out (drop data packets, status etc.) the SCSI drivers will recover
after a timeout and do some cleanup +++

Section 7.9

+++ We don't think that a a NOP once a second or once 100ms is a bandwidth
consuming event.  +++

+++ Yes and the values are fabric and device dependent +++

Section 7.11.3

+++ Item 2 says only that the recovery procedure - if the parties are
capable
and willing to do within session recovery - should be initiated (like on a
complete TCP failure).  +++

Thanks,
Bob

Robert Griswold
Technologist
Crossroads Systems, Inc.
512-928-7272

---

From: "Robert Griswold" <[email-masked]>
Date: Thu, 9 Aug 2001 10:27:56 -0500

Julian:

Thanks, responses in line...

Bob

Robert Griswold
Technologist
Crossroads Systems, Inc.
512-928-7272

From: 	Julian Satran [mailto:[email-masked]] 
Sent:	Sunday, August 05, 2001 9:52 AM
To:	Robert Griswold
Subject:	Re: Review of the 07 draft

Robert,

Julo

Julian:

...

++++++++++

...

+++ This point has a long history remains of which you can find in the
mail
archives.  I and several other authors where inclined to "silently
discard"
whatever can't be repaired (e.g., bugs in the initiator, delayed packets
or
packets arriving late) and explicitly reject only what can be repaired.
Debugging ease can be achieved by other means (event logs etc.). However
many of our colleagues on the list wanted error rejects to be explicit
whenever it is feasible and that is what you have now except for the
cases
in which discards are due to a "legitimate packet" arriving late (e.g.,
from a previous transmission attempt that is already recovered).
Discards are also better in preventing DOS attacks by stream replay++++

[Bob Griswold]  Thanks, that is good clarification.  Again, while I
realize there may have been a lot of history on some of these comments,
having the evolution of the thinking helps.

Section 1.2.5:
...

+++ We envisioned TTTs to be used by the target to identify parts of the
data stream that arrive in response of an R2T. Additionally they
identify
NOP-Outs that are sent in response to a polling Nop-IN and in general
when
valid indicate that the target expects something and identify the
response
(link it to the "cause") for the target. We mentioned that the protocol
does not care about the value mainly to stress that, unlike ITT, the
protocol has no uniqueness requirement for them and whenever they appear
the LUN is also present and can serve as an additional differentiator.
++++

[Bob Griswold]  This will help in understanding what TTTs will be used
for.

Section 1.2.6
...

+++ Logout has also history. We started without a logout but realized
soon
that we will need a synchronized logout to enable command allegiance
change
for recovery (that is possible only after logout). Besides it is a
simple
and painless way to do a clean shutdown of a connection. As for a target
only logout that is what the Asynch Message does (after some timeouts)
Please keep in mind that there is also a fine interplay between logout
and
recovery and that both initiator and target may decide to start some
recovery action+++

[Bob Griswold]  I see the need for the logout to be as quick and as
clean as possible.  If one assumes the Initiator and Target have agreed
on some form of possible recovery scenario, then having the target spend
cycles trying to deliver some recovery information in response to a
Logout seems reasonable; but I am not confident divergent iSCSI
platforms could come to such an agreement, short of it being defined in
one of the iSCSI specifications.

Section 1.2.7
...

+++ This is getting thrown out (it is already) considering that T10 has
gone a long way to expand the addressing of third party command source
and
destinatiantion +++

[Bob Griswold]  This is good news...

This model assumes that the iSCSI layer will deliver complete PDUs to
underlying layers in single (atomic) operations.  The underlying layer
doe
not need to examine the stream content to discover the PDU boundaries.

[Bob Griswold]  OK.

++++

[Bob Griswold]  My main concern is over the ability to have this
specification define implementations of internal "sync and steering" for
different host implementations.  Certainly the IETF should define the
right method for on-the-wire communication between two systems that are
implementing such layers, wherever that communication may exist, but
internal implementations for iSCSI drivers, and how they handle these
"sync and steering" delivered PDUs seems out of place.  Assuming each OS
needs its separate driver base, separate assumptions and separate
legacies, there will be very little hope for having these PDUs
delivered, internally in divergent operating systems, in the exact same
manner.  Have I made these comments clearer, or just confused it more?

Section 2.2.2

[Bob Griswold]  Sorry, emphases on the wrong point here, my fault.  The
use of the AHS is only for iSCSI PDU op-code 0x01, but is defined under
section 2.2 (granted, optional) as a part of the general iSCSI PDU
format.  Since the AHS is only used for a SCSI Command, should it be
defined only under this instance, or is there some future use of the AHS
envisioned?  This comment borderlines on just editorial, and for
education for me.

Section 2.2.2.7
...

+++ I've changed it to:

1.1.1.1   LUN

   Some opcodes operate on a specific Logical Unit. The Logical Unit
Number
   (LUN) field identifies which Logical Unit.  If the opcode does not
   relate to a Logical Unit, this field either is ignored or may be used
in
   an opcode specific way.  The LUN field is 64-bits and it is to be
   formatted in accordance with [SAM2].

 ++++

[Bob Griswold]  Great, that helps a lot.

Section 2.2.3.1

+++ That must be a "misreading" - We are talking about a coded field one
of
the values is Extended CDB. The whole code is the AHSType.  And if you
have
an extended CDB this header is not optional - it is the way to code the
extended CDB (no other way is decribed)
++++

[Bob Griswold]  I am clear on the fact that the Extended CDB bit is in
the AHSType field; I am assuming that you are referring to section 2.3.6
when you say the header is not optional.  Can a developer assume that if
a SCSI Command PDU is received, and the TotalAHSLength has a correctly
formatted value, that there either must be an Extended CDB AHS, or
Bi-directional AHS present?  This may be sufficient to alert from inside
of the required header that the additional header exists.

Section 2.5.1
...

[Bob Griswold]  Sounds good.

As for not  returning anything this is acceptable for Target Cold Reset
-
where the TCP connections are teared down but not for those that leave
the
connections standing +++

[Bob Griswold]  I have reread this comment, but cannot understand what
you are saying.

Section 2.8.5
...

[Bob Griswold]  Well, if you assume that someone somewhere will have to
do the conversion of decimal to hex, then the burden has to live
somewhere, right?  If the wire is going to be exposed to two different
encoding schemes, then how is a lowly target going to know if its
getting decimal or hex values?  Will there be some marker, key=value
system, or something that will allow the receiver of an ambiguous data
type to know what it is getting?

Section 2.9.5
..

[Bob Griswold]  Agreed, there is no reason the same folks couldn't drive
a great "best-practices" RFC, or some SNIA document that could be used
by the industry developers to make life easier.  The out-of-order
key=value issue will never go away, but keeping code efficient is always
good.

Section 2.10.1
...

+++ The text only says that if you are going to support connection
recovery
you should be able to support opening a second connection in order to
clean
the first connection and close it. It also explicitly says that you
don't
have to support 2 connections in full feature phase. And even on 2 TCP
connections it is just a recommendation - using IP-over-pidgeons can be
an
good alternative for forcing a TCP connection to close. On a more
serious
note the second TCP connection can be on the same physical connection.
+++

[Bob Griswold]  Thanks for the explanation, and the great visual.

Section 2.12.4

[Bob Griswold]  Since the implications of ping data escape me (sorry,
just a storage guy so pinging is something I don't understand) I
probably need to defer this to the networking thinkers.  I just hate to
have language in a specification that allows the receiver of what should
be a field overrun to do whatever it sees fit.  Can you give a two
sentence explanation on what the target would do with ping data that is
truncated, and what the initiator would expect to see in such a case?

Section 2.16

+++ A SNACK that is not supported is answered either by an Asynch
Message
asking logout - for failover - or total silence - in which case we
assume
SCSI will timeout and do its thing (abort, reissue the command or
whatever
neded.
There no need to answer. A badly formed SNACK is rejected as any othe
badly
formed PDU (although even this is perceived by some as excessive when it
does not help the initiator take repair action).
++++

[Bob Griswold]  Well, I will continue to say that a target that provides
no information on a received packet of information that is part of a
SCSI transfer is not a great target.  If some well-intentioned initiator
keeps sending what it believes to be valid PDUs, but appears to the
target to be invalid, then where and how does the initiator learn about
what is going on at the target level?

Section 2.16.1
...
+++ The target should request logout if it supports connection recovery
but
not
data snack +++

[Bob Griswold]  So is that information (supporting recovery but no data
snack) inclusive in the discovery information?

Section 4.2
...

+++ I think we can leave this to be implementation dependent. If I would
have to chose a "way out" would say limited number of text exchanges
rather
than a timeout.  ++++

[Bob Griswold]  OK, then can that be defined, or should that also be
left for implementation?

Section 7.1c

[Bob Griswold]  Great, I am looking forward to it.

Section 7.6
...
+++ It is not defined here - it is assumed that when we let SCSI
commands
dry out (drop data packets, status etc.) the SCSI drivers will recover
after a timeout and do some cleanup +++

[Bob Griswold]  OK, but can ULP still be defined as an acronym for this
document?

Thanks,
Bob

Robert Griswold
Technologist
Crossroads Systems, Inc.
512-928-7272

---

From: "Julian Satran" <[email-masked]>
Date: Fri, 10 Aug 2001 09:24:25 +0300

Robert,

Snippets and comments in text - Julo

Section 1.2.6
...

>>>> Logout brings commands associted with the coonection in a state in
which they can change allegiance. This is specified in 7 and recovery
appendix <<<<<<

>>>> Cold reset tears down TCP connections <<<<

Section 2.8.5
...

>>>>> Syntat - Hexadecimal start with 0x <<<<<<<<

Section 2.16.1
...

>>> That will be negotiated recovery level <<<<

Section 7.1c

>>> We will at tiny encoded field to say what the retry means in a day or
two <<<

Thanks,
Bob

Robert Griswold
Technologist
Crossroads Systems, Inc.
512-928-7272

---

From: "Julian Satran" <[email-masked]>
Date: Fri, 10 Aug 2001 10:20:20 +0300

This issue was raised several times. Having parameters changed in two
layers is not a good idea and
leaving only SCSI change them will not enable us to restrict some changes
to login or initial login.
The compromise we have now does not break any existing software and allows
us to do what we want.

Julo

"Robert Griswold" <[email-masked]>@ece.cmu.edu on 09-08-2001
23:36:14

Please respond to "Robert Griswold" <[email-masked]>

Sent by:  [email-masked]

To:   "Eddy Quicksall" <[email-masked]>, Jim
      Hafner/Almaden/IBM@IBMUS
cc:   <[email-masked]>
Subject:  RE: Review of the 07 draft

Eddy:

I actually have no problem with all mode pages (SCSI and iSCSI) being
accessible and changeable from standard SCSI mode commands.  My initial
response to this section was to propose a method that would be
acceptable to the authors of the iSCSI draft.  If there is desire in the
group to allow mode pages for the entire target to be manipulated from
the SCSI level, I think that is a better idea that the one I proposed.
I would assume that an iSCSI aware SCSI utility would understand the
iSCSI specific settings, and allow the user to make those changes.  What
I am really against, is the ability to modify standard SCSI mode page
settings from text messages, as that could lead to target behavior
changes outside of the understanding of the SCSI nexus.

To recap your thinking:  Allow iSCSI text messages to modify and read
iSCSI only mode settings (potentially allowing the reading if SCSI mode
settings), but allow SCSI mode commands to modify and read all target
mode settings, including iSCSI settings.  Is that what you are saying.
If so, I agree.

Bob

Robert Griswold
Technologist
Crossroads Systems, Inc.
512-928-7272

 -----Original Message-----
From:     Eddy Quicksall [mailto:[email-masked]]
Sent:     Thursday, August 09, 2001 2:08 PM
To:  Robert Griswold; Jim Hafner
Cc:  [email-masked]
Subject:  Re: Review of the 07 draft

I don't like the idea of not letting a user of a SCSI utility be able to
change some of the parameters for iSCSI. Because they may be relevant to
him
and there may not be a user interface to the iSCSI driver. pSCSI sets
these
low level parameters via a standard mode set, so why not iSCSI?

It would be best if we could work out something where only the SCSI
layer
can set the mode pages. That would solve everything.

Eddy

----- Original Message -----
From: "Jim Hafner" <[email-masked]>
To: "Robert Griswold" <[email-masked]>
Cc: <[email-masked]>
Sent: Friday, August 03, 2001 5:26 PM
Subject: Re: Review of the 07 draft

> Well, in fact, the draft is supposed to say that MODE_SELECT for the
> transport-specific mode page will NOT be done via SCSI, only via Text
> commands.  I read that as saying that from the SCSI layer, all fields
in
> these pages are "unchangeable" (even though they can change in the
iSCSI
> layer).  Of course, the draft doesn't say whether MODE PARAMETERS HAVE
> CHANGED unit attentions get thrown up at the SCSI layer when this
happens
> at the iSCSI layer.... You later have a clarifying question (Section
3) on
> this as well.
> Regards,
> Jim Hafner

---

From: "John Hufferd" <[email-masked]>
Date: Fri, 10 Aug 2001 08:48:32 -0700

I am not sure that I feel real good about what you are stating here.  I
think the iSCSI Mode Pages are the persistent location for the iSCSI Modes,
but I think iSCSI will have the values in variables that they have in
memory/cache that iSCSI uses ongoing.  If a SCSI command was to change
those values I am not sure that iSCSI will see the changes.

I thought we had previously agreed (for the reasons above) that iSCSI sets
its pages and SCSI sets its pages.

.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
Home Office (408) 997-6136
Internet address: [email-masked]

"Robert Griswold" <[email-masked]>@ece.cmu.edu on 08/09/2001
01:36:14 PM

Eddy:

Bob

Robert Griswold
Technologist
Crossroads Systems, Inc.
512-928-7272

 -----Original Message-----
From:     Eddy Quicksall [mailto:[email-masked]]
Sent:     Thursday, August 09, 2001 2:08 PM
To:  Robert Griswold; Jim Hafner
Cc:  [email-masked]
Subject:  Re: Review of the 07 draft

Eddy

----- Original Message -----
From: "Jim Hafner" <[email-masked]>
To: "Robert Griswold" <[email-masked]>
Cc: <[email-masked]>
Sent: Friday, August 03, 2001 5:26 PM
Subject: Re: Review of the 07 draft

in
iSCSI
happens
3) on

---

From: "Robert Griswold" <[email-masked]>
Date: Mon, 13 Aug 2001 08:08:22 -0500

John:

You are saying what I originally described in my comments on this topic,
if that is what this group decides is important.  The fact that I would
like to see all mode pages available (as others do) to the SCSI mode
commands does not mean that limiting iSCSI to iSCSI and SCSI to SCSI is
not workable.  I personally think that limiting any mode page (the
assumption here is that that the target entity owns the responsibility
for keeping that mode page data) access to a command outside of the
domain of the entity itself (SCSI interface) is not consistent with the
historical use of SCSI.  But again, if the overriding agreement is that
separation is the best way to go, then so be it, it can be made to work.

Bob

Robert Griswold
Technologist
Crossroads Systems, Inc.
512-928-7272

 -----Original Message-----
From: 	John Hufferd [mailto:[email-masked]] 
Sent:	Friday, August 10, 2001 10:49 AM
To:	Robert Griswold
Cc:	Eddy Quicksall; Jim Hafner; [email-masked]
Subject:	RE: Review of the 07 draft

think the iSCSI Mode Pages are the persistent location for the iSCSI
Modes,

I thought we had previously agreed (for the reasons above) that iSCSI
sets
its pages and SCSI sets its pages.

.
.
.
John L. Hufferd

Eddy:

Bob

Robert Griswold
Technologist
Crossroads Systems, Inc.
512-928-7272

 -----Original Message-----
From:     Eddy Quicksall [mailto:[email-masked]]
Sent:     Thursday, August 09, 2001 2:08 PM
To:  Robert Griswold; Jim Hafner
Cc:  [email-masked]
Subject:  Re: Review of the 07 draft

Eddy

----- Original Message -----
From: "Jim Hafner" <[email-masked]>
To: "Robert Griswold" <[email-masked]>
Cc: <[email-masked]>
Sent: Friday, August 03, 2001 5:26 PM
Subject: Re: Review of the 07 draft

in
iSCSI
happens
3) on

---

From: "Binford, Charles" <[email-masked]>
Date: Mon, 13 Aug 2001 10:49:10 -0400

I agree with Julian about removing EMDP and FirstBurstSize from iSCSI
control.  I disagree on MaxBurstSize.

MaxBustSize defines DataPDU length only because we said it does.  We could
just as easily remove the relationship between DataPDU length and
MaxBurstSize.  I would prefer we let iSCSI control DataPDU length as it does
today and change the meaning of MaxBurstSize under iSCSI to be more in line
with what MaxBurstSize means under other SCSI protocols.  As pointed out
earlier on this thread a generic SCSI mode page utility doesn't/shouldn't
know/care what the transport it.

The meaning of MaxBurstSize under FCP is largest FC Sequence a target can
send.  Since an FC class 3 target can send multiple Read data sequences to
the host with no handshake, the parameter is essentially a don't care for
reads.  For Writes it effectively governs the largest amount of data a
target can ask for on an XFER_RDY.

For iSCSI, I'd like to see MaxBurstSize be defined as one of the following:
- largest amount of data a target can ask for under a single R2T; or
- ignored.

My problem with the current definition is typical iSCSI DataPDU lengths are
relatively small (default 8K in the spec.)  The typical MaxBurstSize for
other protocols is much larger - 64K to 512K.  

Charles Binford
Pirus Networks
316.315.0382 x222

-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Saturday, August 11, 2001 5:50 PM
To: Robert Snively
Cc: [email-masked]'; Robert Griswold; Eddy Quicksall; Jim Hafner;
[email-masked]

I think that for regularity it would be wise to remove completely EMDP and
FirstBurstsize from iSCSI control.
MaximumBurstSize defines the DataPDU length and is transport related
(affects iSCSI and less SCSI) and should stay under iSCSI control.

Comments?

Julo

Robert Snively <[email-masked]>@ece.cmu.edu on 10-08-2001 10:32:20

Please respond to Robert Snively <[email-masked]>

To:   "[email-masked]'" <[email-masked]>, Robert Griswold
      <[email-masked]>, Eddy Quicksall <[email-masked]>,
      Jim Hafner/Almaden/IBM@IBMUS
cc:   [email-masked]
Subject:  RE: Review of the 07 draft

Deva,

Both of you are right if iSCSI sticks to negotiating only
parameters related with the TCP connections, the iSCSI
session operational parameters other than EMDP and
burst length, and iSCSI security parameters.  Then, all
parameters relating to the normal SCSI activities, including
EMDP and burst length (which are required for SCSI, not iSCSI,
buffer management) would be negotiated in the normal
SCSI manner through the MODE SENSE/SELECT pages and everybody
would live happily ever after.

Otherwise, and especially with respect to EMDP and burst length,
I have to agree with Robert.

Bob Snively                        e-mail:    [email-masked]
Brocade Communications Systems     phone:  408 487 8135
1745 Technology Drive
San Jose, CA 95110

-----Original Message-----
From: Dev [mailto:[email-masked]]
Sent: Thursday, August 09, 2001 5:33 PM
To: Robert Griswold; Eddy Quicksall; Jim Hafner
Cc: [email-masked]
Subject: RE: Review of the 07 draft

All,

I tend to disagree. The parameters for the iSCSI are negotiated between the
initiator and target.
If we allow the parameters to be changed through SCSI Mode select, how will
this work?

If parameters can be changed through SCSI Mode select then it will apply
only to Lead only connections.

So, we'll have

a) two methods to change the Lead only parameters common for the entire
session (one through mode select
and the other through text parameters)
b) Just iSCSI text command to change the connection specific parameters.

Does it not add to complexity having multiple ways to change the same
stuff?

Thanks

Deva
Platys communications

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf Of
Robert Griswold
Sent: Thursday, August 09, 2001 1:36 PM
To: Eddy Quicksall; Jim Hafner
Cc: [email-masked]
Subject: RE: Review of the 07 draft

Eddy:

Bob

Robert Griswold
Technologist
Crossroads Systems, Inc.
512-928-7272

 -----Original Message-----
From:     Eddy Quicksall [mailto:[email-masked]]
Sent:     Thursday, August 09, 2001 2:08 PM
To:  Robert Griswold; Jim Hafner
Cc:  [email-masked]
Subject:  Re: Review of the 07 draft

Eddy

----- Original Message -----
From: "Jim Hafner" <[email-masked]>
To: "Robert Griswold" <[email-masked]>
Cc: <[email-masked]>
Sent: Friday, August 03, 2001 5:26 PM
Subject: Re: Review of the 07 draft

in
iSCSI
happens
3) on