Subject: Re: [Tsvwg] [SCTP checksum problems]

From: [email-masked]
Date: Wed, 18 Apr 2001 08:27:04 +0200

Randall,

We are going to publish an I-D justifying our choice in about 2 weeks.
It is more extensive than the attached (and has some additional authors)
but here is a preliminary:

(See attached file: draft-sheinwald-iSCSI-CRC-00.txt)

The summary is:

-Checksums (Adler, Fletcher are weak and sensitive to data bias)
-CRCs are far better
-Not all CRCs are equal - out of the CRCs for which there is experimental
and analytical experience CRC32C is the best.

Regards,
Julo

Randall Stewart <[email-masked]> on 18/04/2001 00:15:17

Please respond to Randall Stewart <[email-masked]>

To:   "WENDT,JIM (HP-Roseville,ex1)" <[email-masked]>
cc:   Julian Satran/Haifa/IBM@IBMIL, [email-masked], [email-masked],
      "'Craig Partridge'" <[email-masked]>, Jonathan Wood
      <[email-masked]>, [email-masked], Jonathan Stone
      <[email-masked]>
Subject:  Re: [Tsvwg] [SCTP checksum problems]

Julian:

Your input would be invaluable.. please send us any comments
or input when you are ready.. I think we need to be
looking seriously at this in the early May time frame... so
any input you could give would be most welcome. We are limited
to a 32 bit checksum (the size of the common header CRC area).

But within that restriction any input you may have as to the
best for data integrety would be wonderful!

Regards

R

"WENDT,JIM (HP-Roseville,ex1)" wrote:
>
> Julian,
> The SCTP folks are right now discussing changing the SCTP checksum to be
a
> CRC-32 (or other). This is a very good thing and really what needs to
happen
> with SCTP for it to support iSCSI and other data-critical applications
> effectively (and also relieve iSCSI from having to implement data
integrity
> checking and transport-like functionality over SCTP).
>
> They are looking for inputs as to which CRC-32 or checksum to use. The
iSCSI
> WG's CRC investigation work and conclusion would be a valuable input into
> their decision. The sooner that you can provide the iSCSI recommended CRC
> and reasoning behind it to them, the better, even before the forthcoming
I-D
> is distributed.
>
> Jim Wendt
> Networked Storage Architecture
> Hewlett-Packard Company
> [email-masked] 916-785-5198
>
>
----------------------------------------------------------------------------

> -
>
> > -----Original Message-----
> > From: [email-masked] [mailto:[email-masked]]
> > Sent: Sunday, April 15, 2001 7:58 AM
> > To: [email-masked]
> > Subject: CRCs
> >
> >
> >
> >
> > Dear colleagues,
> >
> > We will probably not be able to finish the CRC/checksum
> > document in time
> > for Nashua but we hope it will be out very soon after that.
> > However I
> > would like to inform you that while in Orlando and
> > Minneapolis we where
> > still talking about different CRCs we (Dafna Sheinwald, Pat
> > Thaler, Matt
> > Wakeley, Vince Cavanna and myself) have agreed on a CRC and
> > the forthcoming
> > ID will give all the reasons and why we recomend it.
> >
> > Regards,
> > Julo
> >
>
>
----------------------------------------------------------------------------

> -
>
> -----Original Message-----
> From: Randall Stewart [mailto:[email-masked]]
> Sent: Tuesday, April 17, 2001 4:31 AM
> To: Jonathan Wood
> Cc: [email-masked]; [email-masked]; Jim Wendt; Jonathan Stone; Craig
> Partridge
> Subject: Re: [Tsvwg] [SCTP checksum problems]
>
> Jonathan:
>
> I will make sure everyone at the bakeoff is aware of the upcoming
> "checksum" change... Now one of the big questions yet is
> what checksum should we use?
>
> I kinda lean towards crc-32 myself (but of course I have no technical
> basis for this and need to keep silent on which one to use anyway :->),
> but do we have other candidates besides fletcher-32 and possibly
> modified
> Adler-32 (i.e. 16 bit adds instead of 8)??
>
> I will take the above 3 and do a bit of performance work this
> week and post some numbers... thats about all I can do i.e.  tell
> how much time the options I know of take...
>
> If you have some other candidates let me know and I can possibly get
> some performance numbers on these as well...
>
> As far as which is the best... I encourage all of you check-sum
> experts out there to please join the thread :)
>
> Oh, I know Jonathan Stone's paper will NOT be ready until sometime
> in May.. so we may want to proceed slowly so that Craig Partridge and
> he can have some cycles to add to this dicussion :)
>
> R
>
> Jonathan Wood wrote:
> >
> > As an SCTP implementor and someone who will want to get the hardware
folks
> to
> > help with checksumming, I wholeheartedly agree with Randy. Remember
that
> SCTP is
> > just a proposed standard, and is as such not all that far along the
> > standardization process. We should still be able to make changes like
this
> if
> > necessary.
> >
> > Jon
> >
> > >
> > >Q:
> > >
> > >The only problem with an additional "CRC chunk" is that
> > >it makes hardware assistance to error correction much
> > >more difficult. It is better (I think) to just realize
> > >we made a mistake. Get the opinions of the experts as to
> > >what checksum to use... i.e.:
> > >
> > >- CRC-32
> > >- Modified Adler-32 (16 bit word sums)
> > >- Fletcher-32
> > >- ???
> > >
> > >And then go with this as a replacement... Admit we were wrong
> > >and fix the problem..
> > >
> > >This way you have ONE and only ONE checksum algorithm making
> > >hardware designers life much easier...
> > >
> > >R
> > >
> > >Qiaobing Xie wrote:
> > >>
> > >> Another solution could be (I think I mentioned this to Randy and a
few
> > >> others at last IETF):
> > >>
> > >> - Define a CRC-32 (or other strong checksum) control chunk and when
the
> > >> sender wishes to use a stronger checksum protection, in addition to
the
> > >> Adler-32 in the common SCTP header it includes this CRC-32 chuck in
the
> > >> outbound packet. When the packet arrives, the receiver will do the
> > >> Adler-32 first, and then if the receiver supports the CRC-32 and
sees
> > >> the presence of the CRC-32 chunk in the packet it will further
verify
> > >> the CRC-32.
> > >>
> > >> We could also use a bit pattern in the chunk type of the CRC-32
chunk
> so
> > >> that if the receiver doesn't understand the CRC-32 chunk it would
drop
> > >> it with a report back to the sender.
> > >>
> > >> -Qiaobing
> > >>
> > >> _______________________________________________
> > >> tsvwg mailing list
> > >> [email-masked]
> > >> http://www1.ietf.org/mailman/listinfo/tsvwg
> > >
> > >--
> > >Randall R. Stewart
> > >Systems & Solutions Engineering
> > >Cisco Systems Inc.
> > >[email-masked] 815-342-5222 or 815-477-2127
> > >
> > >_______________________________________________
> > >tsvwg mailing list
> > >[email-masked]
> > >http://www1.ietf.org/mailman/listinfo/tsvwg
>
> --
> Randall R. Stewart
> Systems & Solutions Engineering
> Cisco Systems Inc.
> [email-masked] 815-342-5222 or 815-477-2127
>
> >

--
Randall R. Stewart
Systems & Solutions Engineering
Cisco Systems Inc.
[email-masked] 815-342-5222 or 815-477-2127

---

From: Craig Partridge <[email-masked]>
Date: Wed, 18 Apr 2001 10:08:05 -0400

Hi Julian:

I read your ID quickly and may have more comments later.

I think the bit error study is flawed.  The key issue here is that
the TCP, and Fletcher checksum (and I think Adler) will catch all
1 bit errors.  Indeed, the TCP checksum will catch up to 16 distinct
bit errors, provided that the bit offsets are all distinct, modulo 16.

Expressing this simply, assuming my system did its Poisson calculations
right:

    * we don't have to worry about single bit errors in a chunk of
      data, the checksum will always catch it.

    * the probability of 2 bit errors occuring in a single 1KB packet
      is about 1 in 3* 10^15.  The probability that the two bits will
      have the same offset mod 16 is around .008 (if I did the distribution
      right) and the chance is 50% that the two bits will offset each other
      (i.e. they'll flip opposite ways).  Ergo, your chance of an
      undetected error is about 1 in 8*10^17.  I believe (without having
      done all the match) that is as good or better than you'll get for
      CRC32 in almost all cases.  Since the checksums are cheaper to compute,
      they're a win.

    * more than 2 bit errors per chunk is so rare we probably don't care

So the big issue is burst errors (and, incidentally, burst erasures).

Thanks!

Craig