Subject: RE: iSCSI CRC: A CRC-checking example

From: Sanjay Goyal <[email-masked]>
Date: Fri, 17 Aug 2001 17:35:03 -0400

Hi
 For all zeros the value after complementing the CRC result is 0x556c8951.
If it is bit-swapped it becomes  0x8a9136aa.
 
 The way you will get the constant remainder 0x1c2d19ed is that you append
 0x556c8951 and not 0xaa36918a to the all-zero data bytes. The crc-check
engine would assume as if it is processing 4 more bytes. If we bit-swap the
CRC while transmitting, don't we have to again swap it at the receive side
and then pass it through the crc-check engine.

Sanjay Goyal                                  
www.ivivity.com

---

From: "Julian Satran" <[email-masked]>
Date: Sat, 18 Aug 2001 01:38:45 +0300

Mark,

I will get this out when we have it right.
As for the endian - it looks to me that the CRC is not different to the
rest - it is sent in network order (and calculated this way).

Regards,
Julo

Please respond to Mark Bakke <[email-masked]>

Sent by:  [email-masked]

To:   Julian Satran/Haifa/IBM@IBMIL, IPS <[email-masked]>
cc:
Subject:  iSCSI CRC: A CRC-checking example

Julian and CRC enthusiasts-

After several people ran the iSCSI CRC through their
hardware and software implementations, we've found that
there are a few important things to add to the CRC spec;
that the bit order must be flipped, and the CRC bytes
must be sent in network order.

We should also specify that when running the CRC check
on an iSCSI PDU with its CRC, that the remainder (before
complement and bit-reversal) is a constant 0x1c2d19ed.

How about replacing:

The generator polynomial selected is evaluated in [Castagnioli93].
When using the CRC the CRC register must be initialized to all 1s
(0xFFFFFFFF) and the CRC bits must be complemented before
transmission. Padding bytes, when present, in a segment covered by a
CRC, should be set to 0 and are included in the CRC.

With:

The generator polynomial selected is evaluated in [Castagnioli93].
When using the CRC the CRC register must be initialized to all 1s
(0xFFFFFFFF).  Input bytes are processed with bit 7 being the most
significant bit.  Before transmission, the CRC bits must be
reflected (bit 31 swapped with bit 0, bit 30 swapped with bit 1,
etc.), and complemented.  The CRC bytes must be transmitted in
network order.  Padding bytes, when present, in a segment covered
by a CRC, should be set to 0 and are included in the CRC.

Running the CRC-32C generator on an input stream that includes a
valid CRC-32C will result in the constant remainder 0x1c2d19ed,
(without being reversed and complemented).

--
Mark A. Bakke
Cisco Systems
[email-masked]
763.398.1054
Return-Path: [email-masked]
Received: from sj-msg-core-3.cisco.com (sj-msg-core-3.cisco.com
[171.70.157.152]) by dogwood.cisco.com (8.8.6
(PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id OAA13149 for
<[email-masked]>; Fri, 17 Aug 2001 14:31:30 -0400 (EDT)
Received: from sj-msg-av-1.cisco.com (sj-msg-av-1.cisco.com
[171.69.11.130])    by sj-msg-core-3.cisco.com (8.11.3/8.9.1) with ESMTP id
f7HITbD04150   for <[email-masked]>; Fri, 17 Aug 2001 11:29:38 -0700
(PDT)
Received: from proxy3.cisco.com (localhost [127.0.0.1])      by
sj-msg-av-1.cisco.com (8.10.1/8.10.1) with ESMTP id f7HIVdW25226  for
<[email-masked]>; Fri, 17 Aug 2001 11:31:39 -0700 (PDT)
Received: from baucis.sc.intel.com (ns3.intel.com [143.183.152.22])    by
proxy3.cisco.com (8.11.2/8.11.2) with ESMTP id f7HIW3Q20439  for
<[email-masked]>; Fri, 17 Aug 2001 11:32:03 -0700 (PDT)
Received: from SMTP (fmsmsxvs03-1.fm.intel.com [132.233.42.203])  by
baucis.sc.intel.com (8.9.1a+p1/8.9.1/d: relay.m4,v 1.41 2001/07/09 21:06:22
root Exp $) with SMTP id SAA03408  for <[email-masked]>; Fri, 17 Aug 2001
18:31:16 GMT
Received: from fmsmsx26.fm.intel.com ([132.233.48.26]) by 132.233.48.203
(Norton AntiVirus for Internet Email Gateways 1.0) ;  Fri, 17 Aug 2001
18:31:15 0000 (GMT)
Received: by fmsmsx26.fm.intel.com with Internet Mail Service (5.5.2653.19)
id <RBZ4BF1K>; Fri, 17 Aug 2001 11:31:14 -0700
Message-ID:
<[email-masked]>
From: "Wheat, Stephen R" <[email-masked]>
To: "'Mark Bakke'" <[email-masked]>
Subject: RE: iSCSI: [Fwd: Crc-32c example in iSCSI spec]
Date: Fri, 17 Aug 2001 11:31:10 -0700
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;     charset="iso-8859-1"
X-Mozilla-Status2: 00000000

Mark,

OK, we're cooking with fire now.  Yup, the reflected-complemented result
matches yours.

Thanks,
Stephen

-----Original Message-----
From: Mark Bakke [mailto:[email-masked]]
Sent: Friday, August 17, 2001 10:53 AM
To: Wheat, Stephen R
Subject: Re: iSCSI: [Fwd: Crc-32c example in iSCSI spec]

Stephen-

The remainder was from running an entire test case through
the CRC generator, including the CRC itself.  I then had to
complement and bit-flip the remainder to make it work out the
same as the documented Ethernet remainder.  So here's how
I get the remainder for the 32-bytes of ones example:

Run

       0:        ff ff ff ff
       4:        ff ff ff ff
       8:        ff ff ff ff
      12:        ff ff ff ff
      16:        ff ff ff ff
      20:        ff ff ff ff
      24:        ff ff ff ff
      28:        ff ff ff ff

      32:        43 ab a8 62

Through the generator to get the CRC of the whole thing,
including the CRC that was sent.

In all cases, if the CRC is correct for the data, the CRC
will be 0x48674bc7.  Since the CRC that came out of my
generator was not really the remainder, it was the bit-flipped,
complemented remainder, I have to reverse both of these to
get 0x1c2d19ed.

--
Mark

"Wheat, Stephen R" wrote:
>
> OK, until now, I was matching everything you posted (after the
correction).
> Now, I cannot get the same result as you on the remainder, either with a
> simple engine, or a table driven.
>
> So, what exactly are you passing through the engine for this latest
> confirmation calculation?
>
> I have been assuming you've been passing a 4-byte stream to the
crc-engine,
> where the 4-byte stream consists of the generation polynomial, byte order
> 0x41 0x6f 0xdc 0x1e
>
> I'm not looking for you to help debug my engine; just to confirm the
input
> to it.
>
> Thanks,
> Stephen
>
> -----Original Message-----
> From: Mark Bakke [mailto:[email-masked]]
> Sent: Friday, August 17, 2001 10:06 AM
> To: Sanjay Goyal
> Cc: Ips (E-mail)
> Subject: Re: iSCSI: [Fwd: Crc-32c example in iSCSI spec]
>
> Sanjay Goyal wrote:
> >
> > Hi
> >  I get the remainder for iSCSI as 0x1c2d19ed if I complement my CRC and
> then
> > pass it through the CRC engine.
> >
> >  As per CRC generation for data: the thing which is not clear to me is
> >    why do we need to bit-swap the CRC reminader as is done in all your
> > examples.
>
> As far as I can tell, bit-swapping does nothing to help or hurt
> the effectiveness of the CRC.  When I ran my examples, I thought
> that it would be the simplest for our CRC to different from the
> Ethernet CRC only in the polynomial.  Ethernet, FDDI, FC, and SCSI
> all do this, so I figured it would cause the least confusion if
> we did the same.
>
> > We can just complement it and append it after the DATA bytes.
> >  The other side also can just pass it through the CRC engine to check
it.
> >
> > Regards
> > Sanjay Goyal
> >
> > -----Original Message-----
> > From: Mark Bakke [mailto:[email-masked]]
> > Sent: Thursday, August 16, 2001 1:43 PM
> > To: Steve Blightman; [email-masked]
> > Subject: Re: [Fwd: Crc-32c example in iSCSI spec]
> >
> > One more thing that might be helpful.  When the Ethernet
> > polynomial is used in SCSI to generate its CRC, the T10
> > doc specifies the remainder polynomial that one should see
> > after running the data with a valid CRC through.  For
> > the Ethernet CRC, this was specified as 0xc704dd7b.  This
> > remainder polynomial is taken before the CRC is complemented
> > and bit-reflected.  For iSCSI, I came up with a remainder of
> > 0x1c2d19ed.  Can anyone verify this result?
> >
> > --
> > Mark
> >
> > Mark Bakke wrote:
> > >
> > > Steve-
> > >
> > > I just ran some Ethernet packets with known CRCs through
> > > my iSCSI/Ethernet CRC generator, and found the same thing
> > > as you did.
> > >
> > > All of my examples need to be byte-swapped, along with the
> > > fix I already posted for the all-ones example.  Here is a
> > > new set of examples, which will be in -08.  I also ran
> > > 64 bytes of zeroes and ones, which now agree with your
> > > numbers as well.
> > >
> > > Thanks again for bringing this up.
> > >
> > > --
> > > Mark
> > >
> > >      07 CRC Examples
> > >
> > >         N.B. all Values are Hexadecimal
> > >
> > >           Byte:        0  1  2  3
> > >
> > >              0:       01 a0 00 00
> > >              4:       00 00 00 00
> > >              8:       00 00 00 00
> > >             12:       00 00 00 00
> > >             16:       04 05 00 00
> > >             20:       00 01 00 00
> > >             24:       00 00 00 05
> > >             28:       00 00 00 04
> > >             32:       2a 00 00 00
> > >             36:       00 00 00 00
> > >             40:       80 00 00 00
> > >             44:       00 00 00 00
> > >
> > >            CRC:       93 70 51 db
> > >
> > >         32 bytes of zeroes:
> > >
> > >           Byte:        0  1  2  3
> > >
> > >              0:       00 00 00 00
> > >            ...
> > >             28:       00 00 00 00
> > >
> > >            CRC:       aa 36 91 8a
> > >
> > >         32 bytes of ones:
> > >
> > >           Byte:        0  1  2  3
> > >
> > >              0:       ff ff ff ff
> > >            ...
> > >             28:       ff ff ff ff
> > >
> > >            CRC:       43 ab a8 62
> > >
> > >         32 bytes of incrementing 00..1f:
> > >
> > >           Byte:        0  1  2  3
> > >
> > >              0:       00 01 02 03
> > >            ...
> > >             28:       1c 1d 1e 1f
> > >
> > >            CRC:       4e 79 dd 46
> > >
> > >
> > > Steve Blightman wrote:
> > > >
> > > > I believe the examples for the ISCSI CRC  have the wrong
endianness.
> > > >
> > > > As yiou suugested over the phone I ran some Ethernet frames through
a
> > > > simulation. I have some difficulty running the exact simulations
you
> > > > wanted becuase the minimum size Ethernet frame is 64 bytes.
> > > >
> > > > However using the Ethernet CRC polynomial,
> > > > Running 64 bytes of 0 onto the wire, we append "36 63 8d 75" onto
the
> > > > wire for the CRC - "36" goes out first.
> > > > Running 64 bytes of all 1, we append "ba 87 61 0f" onto the wire -
> "ba"
> > > > goes out first.
> > > >
> > > > Using the same logic for the ISCSI polynomial
> > > > Running 64 bytes of 0 I think we should append "67 eb c8 03" - "67"
> > > > going out first
> > > > and running 64 bytes of all 1 we should append "66 4e cd 2f" - "66"
> > > > going out first
> > > >
> > > > And now for 32 bytes with the ISCSI polynomial
> > > >
> > > > Running 32 bytes of 0 we should append "aa 36 91 8a" - "aa" going
out
> > > > first
> > > > Running 32 bytes of all 1 we should append "43 ab a8 62" - "43"
going
> > > > out first
> > > >
> > > > I don't want to get into an endless endian debate, but I believe it
is
> > > > important to get the order of these bytes in the right order, so
that
> we
> > > > can use the same hardware to check as well as to generate CRCs.
> > > >
> > > > Thanks for your help on this,
> > > > Steve Blightman
> > >
> > > --
> > > Mark A. Bakke
> > > Cisco Systems
> > > [email-masked]
> > > 763.398.1054
> >
> > --
> > Mark A. Bakke
> > Cisco Systems
> > [email-masked]
> > 763.398.1054
> >
> >
>
----------------------------------------------------------------------------

> ------------------------
> >
> >    Part 1.2    Type: application/ms-tnef
> >            Encoding: base64
>
> --
> Mark A. Bakke
> Cisco Systems
> [email-masked]
> 763.398.1054

--
Mark A. Bakke
Cisco Systems
[email-masked]
763.398.1054

---

From: "Julian Satran" <[email-masked]>
Date: Wed, 22 Aug 2001 23:28:37 +0300

Vince & all,

I've added some statement about order (byte and bit) to the draft (you have
probably missed that piece of the mail).

I could be more graphic and less explicit as in SPI but I think that won't
help too much.

People did it for ethernet for years without too much ado.

And don't complain to me - IBM did consistent bit ordering for years
(Big-endian).  IBM even sent ASCII big endian (nobody else did and we had
to recode for VT100 access).

As for what to choose - It seems that we are in an are where almost
everybody does it now in this crazy way.
I strongly suspect that this was also the reason for the way SPI does it.
They have gone all the way to acommodate
x86 byte ordering in registers - so that the revert the bytes before
calculating -and do this on a parallel bus!
360 was certainly the last bastion of regularity.

Regards,
Julo

"CAVANNA,VICENTE V (A-Roseville,ex1)" <[email-masked]> on
22-08-2001 20:42:50

Please respond to "CAVANNA,VICENTE V (A-Roseville,ex1)"
      <[email-masked]>

To:   Julian Satran/Haifa/IBM@IBMIL, "'Mark Bakke'" <[email-masked]>
cc:   "CAVANNA,VICENTE V (A-Roseville,ex1)" <[email-masked]>,
      [email-masked], [email-masked]
Subject:  RE: iSCSI CRC: A CRC-checking example

Julian and Mark and others,

After I took into account the bit "reflection"  and the byte order (Big
Endian on Bytes and Little Endian on bits) I did finally obtain, using
polynomial math, the (corrected) results you show in the CRC examples. The
results have also been confirmed independently by an implementor here at
Agilent.

I am the one who originally suggested to Julian that we specify the CRC
algorithm the same way as in ethernet even though for iSCSI it is not
really
important to initialize the CRC register to all 1s and to complement the
CRC
before transmission since there are other means to detect extra or missing
PDU bytes. However I had not realzied until recently that conformance with
the ethernet algorithm implies bit reflection. I had not been aware that in
ethernet the bits are sent out on the serial link with the least
significant
bit first AND that the corresponding message polynomial is formed from the
bits in the sequence that they appear on the serial link; thus the need for
bit "reflection".

Now that I understand the need for bit reflection (taken into account in
the
rocksoft parameterized CRC generator by setting the in and out reflection
flag parameters to TRUE) I am not sure I agree that we want it in iSCSI.
The
penalty for full conformity with ethernet seems too great. If people feel
strongly that we must keep the bit reflection I think that to make the
existing documentation clear and unambiguous we would need to explicitly
show the mapping of bits in the iSCSI PDU to coefficients of the message
polynomial that represents the iSCSI PDU. We would also need to show the
mapping of the CRC bits to the coefficients of its representative
polynomial.

If you don't agree I will elaborate further later this week to try to
convince you. My objective is to be able to easily and unambiguously
describe the polynomial math behind the algorithm; right now, with the bit
reflection and without the explicit mapping it is awkward.

Vince

---

From: [email-masked]
Date: Thu, 23 Aug 2001 18:31:23 -0400

From a practical standpoint, using the same implementation framework
(for lack of a better word) as Ethernet reduces the opportunity
to get this wrong - a designer who's done an Ethernet CRC does
the same thing with the new CRC polynomial and the result works.

If this aspect of CRC usage "ain't broke", I would think
that "don't fix it" is the preferred course of action ;-).

Thanks,
--David

> From: Douglas Otis [mailto:[email-masked]]
> Sent: Thursday, August 23, 2001 5:43 PM
> To: Sanjay Goyal; 'CAVANNA,VICENTE V (A-Roseville,ex1)'
> Cc: [email-masked]
> Sanjay,
> In short, the reflected table is likely to require fewer instructions to
> implement a lookup than a non-reflected table.  Secondly, if the processor
> is Little Endian, then the result is already in network order without byte
> swapping.  Obviously this last point only helps if you are using a Little
> Endian processor but this happens only once per packet whereas the lookup
> may happen every byte.  Third, the inverted store improves the sensitivity
> to the packet length.  All these Ethernet techniques seems to be good
> things.  One difference however is the polynomial.
> 
> Doug
> 
> > The last line says
> >  "but I suppose it is advantageous to do things the ethernet way."
> > May I know in what ways it would be advantageous.
> >
> > Regards
> > Sanjay Goyal
> >

---

From: Robert Snively <[email-masked]>
Date: Thu, 23 Aug 2001 19:04:25 -0700

But Doug, that was what Dave was proposing we would not have to
do by copying Ethernet  :-)

> It may mean tightening the
> straps on the thinking cap when assigning terms.
> 
> Doug

---

From: "THALER,PAT (A-Roseville,ex1)" <[email-masked]>
Date: Fri, 24 Aug 2001 11:36:02 -0600

Joe and Bob,

I doubt that any implementation today will be doing CRC computation in a bit
serial circuit. The computation is generally done either byte parallel or,
increasingly, multi-byte parallel. The order in which the bits of a byte are
assigned to polynomial coefficients makes no difference to byte parallel CRC
implementation complexity. The only time complexity would be effected by bit
order is when one was doing a serial shift register implementation of the
CRC generator as some of the earliest Ethernet implementations did. The
technical reasons which applied to Ethernet in chosing the CRC bit checking
order, burst protection and serial implementation complexity, don't apply to
iSCSI.

The only reason for choosing one bit order over another for iSCSI CRC
checking is the limits of human comprehension. Which bit order is most
likely to be gotten right by the humans implementing the standard?

 Ethernet 
          + many people are already accustomed to its bit order
          - most significant byte first, least significant bit first can
cause confusion
 Most significant everything first 
          + more internally consistent than Ethernet
          - different from Ethernet

The most important thing is that we specify whichever one we chose as
clearly as possible.

Regards,
Pat 

-----Original Message-----
From: Joe Gervais [mailto:[email-masked]]
Sent: Thursday, August 23, 2001 7:21 PM
To: Robert Snively; [email-masked]; [email-masked];
[email-masked]

Bob,

If the data was going on token ring or some other media, sure, who cares
about an Ethernet approach, but the data is generally being sent and
received over Ethernet, and there is advantage at least in some
implementations to be in Ethernet bit/byte order.

Additionally, as David said earlier today, if it isn't broke, let's not fix
it. Can we get consensus and move on?

Joe
-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf Of
Robert Snively

... snip ...
So I would have expected that we would avoid any choice that
makes that less convenient.  The Ethernet approach to those
bytes would strike me as being the second-least convenient
approach.  Clearly, throwing hardware at this could correct
for that inconvenience, but it seems to me that any
optimizations we make should focus on what we want to do, not
what Ethernet did.

Bob

---

From: Robert Snively <[email-masked]>
Date: Fri, 24 Aug 2001 10:43:36 -0700

Pat,

I agree with you.

Bob

> From: THALER,PAT (A-Roseville,ex1) [mailto:[email-masked]]
> Sent: Friday, August 24, 2001 10:36 AM
> To: [email-masked]; Robert Snively; [email-masked];
> [email-masked]; [email-masked]