Subject: Re: iSCSI: Header Digest Failure (was R2TDataSN)

From: [email-masked]
Date: Thu, 8 Mar 2001 13:24:08 +0200

I understood that. But markers are meant to solve this too aren't they?

Julo

Matt Wakeley <[email-masked]> on 08/03/2001 01:54:10

Please respond to Matt Wakeley <[email-masked]>

To:   [email-masked]
cc:
Subject:  iSCSI: Header Digest Failure (was R2TDataSN)

Julian,

What Somesh is talking about has nothing to do with R2T specifically.  His
point is, the length field in the PDU header determines how long *this* PDU
is, and where to find the *next* PDU.  If *this* PDU fails its header
digest
check, you cannot trust the length field, therefore, you cannot know where
the
*next* PDU is in the byte stream.

Given that, is seems like whenever there is a header digest error, you have
no
choice but to terminate the TCP connection and start a new one.

-Matt Wakeley
Agilent Technologies

Somesh Gupta wrote:
>
> Julian,
>
> How do I know it is an R2T if the header digest failed?
> I am sorry if I am missing something very obvious here.
>
> Let us say that a header digest failed. What information
> in the header am I able to trust at this point? I don't
> know if it is an R2T or not. I don't know if the length
> is ok or not.
>
> Thanks,
> Somesh
>
> > -----Original Message-----
> > From: [email-masked] [mailto:[email-masked]]On Behalf Of
> > [email-masked]
> > Sent: Tuesday, March 06, 2001 4:01 PM
> > To: [email-masked]
> > Subject: RE: R2TDataSN
> >
> >
> >
> >
> > Somesh,
> >
> >  I am still lost. R2T is fixed length and you know that you have a bad
> > digest after reading it.
> > No length involved.   ???
> >
> > Julo
> >
> > "Somesh Gupta" <[email-masked]> on 07/03/2001 00:26:10
> >
> > Please respond to [email-masked]
> >
> > To:   Julian Satran/Haifa/IBM@IBMIL, [email-masked]
> > cc:
> > Subject:  RE: R2TDataSN
> >
> >
> >
> >
> > Julian,
> >
> > > -----Original Message-----
> > > From: [email-masked] [mailto:[email-masked]]On Behalf
Of
> > > [email-masked]
> > > Sent: Tuesday, March 06, 2001 9:31 AM
> > > To: [email-masked]
> > > Subject: RE: R2TDataSN
> > >
> > >
> > >
> > >
> > > Somesh,
> > >
> > > You've lost me. I do not propose that you look at the bad R2tT but to
> > find
> > > that you have missed one
> > > by looking at the next.
> >
> > Since iSCSI PDUs define how long they are, you have to look at
> > one PDU to determine where the next PDU is. (unless ofcourse
> > the sync and steering layer is doing the work - see my exchange
> > with Venkat on that).
> >
> > On the long transfer etc., I was not sure what scenarios
> > an R2TDataSN was providing recovery from. Since you clarified
> > that it is to recover from a header digest error, we can
> > focus on that scenario.
> >
> >
> > This interesting for long transfers that have
> > > several outstanding R2Ts.
> > > What is speculative here? There was never a consensus that there
> > > will be no
> > > more than one outstanding R2T.
> > >
> > > Regards,
> > > Julo
> > >
> > > "Somesh Gupta" <[email-masked]> on 06/03/2001 17:23:04
> > >
> > > Please respond to [email-masked]
> > >
> > > To:   Julian Satran/Haifa/IBM@IBMIL, [email-masked]
> > > cc:
> > > Subject:  RE: R2TDataSN
> > >
> > >
> > >
> > >
> > > I beg to disagree. If an R2T PDU (header) has bad digest, or any
other
> > > header has a bad digest - since you always need the PDU length from
> > > the header, there is some uncertainty associated with further
> > processing.
> > >
> > > Are you proposing that the processing machine go into a "speculative"
> > > mode where the processing of the next PDU determines whether we were
> > > successfuly able to skip a bad PDU header? When there is a data
digest
> > > error, further stream parsing is deterministic. But not when the PDU
> > > header digest error.
> > >
> > > Also the consensus (in my interpretation) was on applications
> > > not transfering very large amounts of data using a single command or
> > > read/write PDU.
> > >
> > > > -----Original Message-----
> > > > From: [email-masked] [mailto:[email-masked]]On Behalf
Of
> > > > [email-masked]
> > > > Sent: Monday, March 05, 2001 5:41 PM
> > > > To: [email-masked]
> > > > Subject: Re: R2TDataSN
> > > >
> > > >
> > > >
> > > >
> > > > Somesh,
> > > >
> > > > 1.The only consensus I heard is not to transfer a large amount of
> > > > data with
> > > > one PDU.
> > > >
> > > > 2.With DatasN and Sack we dont need any data in a bad header.
> > > >
> > > > 3. If an R2T is lost (received at initiator with bad digest) - the
> > > > initiator will know that from
> > > > the next R2T if the target has several outstanding - very likely at
> > long
> > > > distances - and will not have to way for a timeout.   Other uses
are
> > > > marginal.  Basically it is "part of a command execution" and we can
> > > > painless recover
> > > > from failures for this case too.
> > > >
> > > > Regards,
> > > > Julo
> > > >
> > > > "Somesh Gupta" <[email-masked]> on 05/03/2001 20:40:06
> > > >
> > > > Please respond to [email-masked]
> > > >
> > > > To:   Julian Satran/Haifa/IBM@IBMIL, [email-masked]
> > > > cc:
> > > > Subject:  R2TDataSN
> > > >
> > > >
> > > >
> > > >
> > > > > R2TDataSN
> > > > > ----------
> > > > > Sec 6.7.1 has some new content on how to handle lost R2Ts using
> > > > > SACKs.  But I noticed that the SACK request (Sec 2.16) has not
> > > > > changed to indicate whether the DataSN is a R2T DataSN or just
> > > > > a Read PDU DataSN (D bit)
> > > > > So do we demux on the read/write operation type?
> > > > > And how does this affect PDU loss in bidirectional commands ?
> > > > > +++ SACK is ascking for data (DataSN) the target knows
> > > > >
> > > >
> > > > Julian,
> > > >
> > > > Regarding the R2TDataSN, I have a comments and a
> > > > question.
> > > >
> > > > I think that when a PDU header fails a CRC/checksum check etc,
> > > > it is a problem to depend on any information in the header
(including
> > > > length fields), thereby making any further processing on
> > > > the connection unreliable.
> > > >
> > > > What scenarios do you envision where the R2TDataSN is useful.
> > > > IN Orlando I think there was clear consensus that application
> > > > do not try to transfer very large amounts of data using a
> > > > single command.
> > > >
> > > > Thanks,
> > > > Somesh
> > > >
> >
> > Thanks,
> > Somesh Gupta
> >
> >
> > _________________________________________________________
> > Do You Yahoo!?
> > Get your free @yahoo.com address at http://mail.yahoo.com
> >
> >
> >
>
> _________________________________________________________
> Do You Yahoo!?
> Get your free @yahoo.com address at http://mail.yahoo.com

---

From: Douglas Otis <[email-masked]>
Date: Thu, 08 Mar 2001 13:03:02 -0800

Julian,

It should follow that frame markers are not an option if header digests are
used otherwise dropping the connection becomes required in the event of an
error.  If header digests are used, there should also be an advisory
recommending checking both values of the frame marker PDU pointer to detect
a potential error at this level, as well as, a recommended null stuffing of
the connection to allow resynchronization between framing markers.  Here in
the draft, you have overlooked a concern regarding a digest error as it
pertains to framing.

Page 20:
   "In situations where IP packets are delivered in order from the
   network, iSCSI message framing is not an issue; messages are
   processed one after the other. In the presence of IP packet
   reordering (e.g., frames being dropped), legacy TCP implementations
   store the "out of order" TCP segments in temporary buffers until the
   missing TCP segments arrive, upon which the data must be copied to
   the application buffers.  In iSCSI it is desirable to steer the SCSI
   data within these out of order TCP segments into the pre-allocated
   SCSI buffers rather than store them in temporary buffers. This
   decreases the need for dedicated reassembly buffers as well as the
   latency and bandwidth related to extra copies."

Page 12:

   "The iSCSI target layer MUST deliver the commands to the SCSI target
   layer in the order specified by CmdSN."

What happens if there is an unrecovered gap in the CmdSN due to a failed
NIC?  Why are these 31 bits? (2**31-1) is the limit set for several values.
This is 2,147,483,647.  In other words, this is a 31 bit limitation.  If the
MS bit is not to be used, perhaps this bit could be used to signal Immediate
without disrupting the command sequence.  I doubt this is intended however
as there are examples that indicate a count at 0xffffffff.  If the MS bit
was not used, then the maximum count would have been 0x7fffffff.  I hope I
am not repeating a prior notice.

Page 18:
      - IPv6 address, in dotted decimal notation.  Assumed if the
      name contains more than four, but at most 16 numbers, separated
      by dots (.), where each number is in the range 0..255.

Staying with decimal byte notation for IPv6, define how a value with fewer
than 16 digits is interpreted by means of padding.  Do you zero pad MS
positions?  This will require scanning all values with reversed order
placement.  I doubt there is a more expensive scheme.

On page 25, perhaps a better choice of words would be 'section' rather than
'segment' to describe portions of the various structures that comprise the
PDU layers.

On page 83, you go to some length explaining R2T recovery based on a timeout
but fail to provide any indication of a suitable timeout value.  How are
these timeouts negotiated?
Can equipment be expected to interoperate without timeouts specified?

Doug

---

From: "Venkat Rangan" <[email-masked]>
Date: Thu, 8 Mar 2001 16:13:46 -0800

Julian,

Immediately following the initial marker-less interval, what appears on the
stream? Is it the first marker or is it the first PDU?

Also, in heading 19 (page 110), "Default is 4096" would imply the default
initial markerless-interval is 16K, but in earlier section (page 105), it
states the default is 4K.

The marker offsets are in bytes, and the left most two bits are reserved.
So, one would be unable to store a marker for a LongDataWord PDU. We don't
think it's a good idea to send such a long PDU, but providing smaller number
of bits in marker offsets appears to be a sneaky way to impose a limit.

Venkat Rangan
Rhapsody Networks Inc.
http://www.rhapsodynetworks.com

---

From: Matt Wakeley <[email-masked]>
Date: Thu, 08 Mar 2001 17:31:08 -0800

What you say is that some method of framing is *required* to perform recovery
after a header digest failure.  So, is it then to be inferred that framing in
iSCSI is mandatory?

-Matt Wakeley

---

From: Matt Wakeley <[email-masked]>
Date: Fri, 09 Mar 2001 00:37:03 -0800

I believe that the "initial marker-less interval" is unnecessary.  At
connection time, there are no "markers".  When both sides agree on markers,
they should also agree on when to start them.  The time might well be before
or after the default "initial marker-less interval" defined in the standard.

-Matt Wakeley
Agilent Technologies