Subject: Re: iSCSI : Reject PDU Concerns.

From: [email-masked]
Date: Fri, 19 Jan 2001 15:46:50 +0200

Santosh,

my answers in text

Julo

Santosh Rao <[email-masked]> on 18/01/2001 23:53:38

Please respond to Santosh Rao <[email-masked]>

To:   IPS Reflector <[email-masked]>
cc:
Subject:  iSCSI : Reject PDU Concerns.

Ref : Section 2.19 (Reject PDU), Section 5.5 (Digest Errors)

Issue :
=====
1) Reject PDU must return an Initiator Task Tag in the Reject PDU
header, which provides the initiator sufficient context to lookup the
outbound exchange that was transmitted.

<js> reject PDU can have a multitude of causes and those are/will be
detailed to certain level
(not too much.  Whenever the Initiator Task Tag is available and relevant
it can be read from the returned information. However I am still looking to
what it would take to get it back into the reject PDU itself </js>

2) An error_byte field should be considered for inclusion in the Reject
PDU which contains the byte offset of the errant byte[or word] in the
received header/payload that caused the target to send a Reject response
to the command.

This is a simple solution that provides for quick fault isolation and
root cause of the reason for the reject. This also rids the Reject PDU
of any detailed elaboration of reason codes meant to allow root cause of
the reason for the reject.

<js> that is no big help since there can be more than one error and it
helps mostly in debugging. For implementation errors in the target and/or
initiator I am sure the protocol analyzer will be happy to help but running
implementation should not be burdened with. For those that are running
errors on good implementations we will provide some details when necessary
</js>

3) The Reject PDU should only be used for " Format Error" since the
Reject mechanism  is a per-task error recovery to deal with header
format problems.
"Digest Error" handling is too complex dealing with too many situations.
Digest error recovery should be simplified to a connection level error
recovery which involves Logout or connection close/re-start.

The current digest recovery is a ping-pong as follows :
- Target sends REJECT
- Initiator on seeing REJECT with "Digest Error" sends Logout
- Target sends Logout response.
(the spec is not clear on who drives the connection close. does the
initiator close the connection on receiving logout response, or does the
target close connection and send logout reponse on another connection
??).

The above procedure is quite convoluted and the followinf simplification
should be considered :
- On a digest error, target sends Logout, indicating appropriate reason
code.
- Target follows up the Logout with a TCP connection close.

<js> No. Command Response flow follows a regular client-server pattern and
I am
not willing to consider all the consequences of transforming it into a
"peer to peer" </js>

4) There is no value add in expecting a logout response on a logout
which is intended to close a connection and is sent on the same
connection. The logout can be followed up with a connection close.

<js> That can be done at extreme anyhow. Logout is a late addition to this
protocol  mainly to clean-up a failing connection before recovery. Logout
response is there for symmetry and convenience. Even if the target follows
logout with a connection close - the response can get to the initiator and
recovery may proceed faster that in a complete failure in which it would
have to wait for a timeout </js>

Regards,
Santosh

---

From: Matt Wakeley <[email-masked]>
Date: Tue, 23 Jan 2001 10:54:54 -0800

> errors on good implementations we will provide some details when necessary

Julian,

With only in-band framing [the markers that are currently defined in the
spec], there is no way a protocol analyzer can be used in the traditional
way.  In order for the protocol analyzer to work, it would have to be
continuously running since the tcp connection(s) was  established.  Besides,
early on there will not be any protocol analyzers.

-Matt

---

From: Santosh Rao <[email-masked]>
Date: Tue, 23 Jan 2001 11:35:06 -0800

Matt Wakeley wrote:

> > helps mostly in debugging.

Julian,

1) The iSCSI layer, on parsing a PDU, will stop on detecting the first error.
So, there should really be only one error to report and that is the first one
discovered.

2) Helping in debugging is a big issue since one of the major problems
iSCSI implementations will need to deal with is inter-operability. This is not a
one-time problem as new implmentations of devices and initiators keep coming
along. The iSCSI draft must attempt to provide some aids for quick fault
isolation.

3) The cost of providing this error_byte based fault indication feature is
really minimal, compared to the benefits it provides.

Regards,
Santosh

> For implementation errors in the target and/or

---

From: "Douglas Otis" <[email-masked]>
Date: Tue, 23 Jan 2001 13:49:22 -0800

Matt,

Unless encoding headers to hide typical PDU signatures, an analyzer will
have little difficulty in discovering PDU boundaries.  Should a scheme force
PDU alignment every 8192 bytes as example, discovery would be a certainty
within 8k bytes.  To prevent blockage, the size of the PDU is limited and so
the search is not impossible as you imply nor even difficult.  Either
markers or alignment seems a good solution for minimizing impact of segment
loss to that of the interval rather than 2RTT x rate.  For either a periodic
marker or alignment scheme to work for sending, no changes to TCP is
required.

As for the diagnostics, I would suggest there is no reason to encode data
into decimal text for the same reason and so a hexadecimal encoding should
be the default for all numeric encoded settings.  This information is not
for human consumption and such decimal conversion requires additional
processing for dubious benefit.  If there is an analyzer dissecting content,
such human aid can be added much as text is appended to a code dump.

Doug

---

From: Matt Wakeley <[email-masked]>
Date: Tue, 23 Jan 2001 14:29:45 -0800

I disagree with you Doug,

Douglas Otis wrote:
> have little difficulty in discovering PDU boundaries.

I think that at 10Gig, it will be very difficult for an analyzer to find an
iSCSI header in a byte stream and keep up with the incoming traffic while it's
doing it. Furthermore, I could come up with a data pattern that would make it
really difficult for your searching analyzer to synchronize on the real PDUs
vs the fake ones in the data.

> Should a scheme force
> within 8k bytes.

How is that?  Remember, TCP's initial sequence number is random, so how will
you know where the 8K boundary is?

> To prevent blockage, the size of the PDU is limited and so

Tweeking TCP a little by adding even an option bit in the header signalling a
start of a new PDU would be a lot better.  TCP and any other protocol should
evolve to track the evolution of the rest of computer technology.

-Matt

---

From: "Douglas Otis" <[email-masked]>
Date: Tue, 23 Jan 2001 16:57:38 -0800

Matt,

As both schemes are periodic, once an interval has been deciphered, a
boundary is discovered and can be applied to later data already in progress.
Once a sample has been scanned for typical PDU signatures, a trial intercept
should confirm success with little effort even at OC-192 rates.  By making
the intervals a modulo that is a power of 2, you would not need to be
concerned about potential wraps between efforts.

The evolution is called SCTP for transforming TCP into a frame aligned
object based protocol.  Such changes to TCP as you suggest will hardly end
with a flag here or there.

Doug