Subject: RE: iSCSI: [Fwd: Crc-32c example in iSCSI spec]

From: "Wheat, Stephen R" <[email-masked]>
Date: Fri, 17 Aug 2001 13:25:45 -0700

Quoting from Richard Black,
www.cl.cam.ac.uk/Research/SRG/bluebook/21/crc/node2.html
the inversion of the CRC before sending it precludes erroneous addition of
trailing zeros.

Without complementing the remainder, the result of a checked message would
be 0x00000000.  If the message, for some reason, had any number of zero bits
appended to it (after the crc word), the result would still be 0x00000000.

Thus, it seems that complementing the remainder is a relic from the bit
serial days where the end of the message was not determined beforehand, as
is the case for iSCSI PDUs.

As Mark originally opined, keeping with the reflected/complemented algorithm
has a certain consistency to it.

So, it appears that complementing the result for iSCSI CRC is an arbitrary
decision, that consistency seems to have won out.

Does anyone have an opinion as to whether checking for a zero result, versus
a non-zero result, has a performance implication?

Stephen

-----Original Message-----
From: Douglas Otis [mailto:[email-masked]]
Sent: Friday, August 17, 2001 11:54 AM
To: Mark Bakke; Sanjay Goyal
Cc: Ips (E-mail)
Subject: RE: iSCSI: [Fwd: Crc-32c example in iSCSI spec]

Mark,

The reason for the bit swap within the table is to allow a serial hardware
scheme as CRC is processed ms to ls over the entire stream as if a single
number.  Ethernet sends the byte out least significant bit first.  The
entire table is also swapped ls to ms and actually reduces the operations
needed within the table calculations.  This reflects the method used for
Ethernet as it is being stored inverted and the initial value is started as
all ones.  The reason for the initial value being set to all ones is clear
for leading zero interaction, but I do not understand the value in storing
the CRC inverted.  I thought to include my ignorance to the mix.

Doug

> Sanjay Goyal wrote:
> >
> > Hi
> >  I get the remainder for iSCSI as 0x1c2d19ed if I complement my
> CRC and then
> > pass it through the CRC engine.
> >
> >  As per CRC generation for data: the thing which is not clear to me is
> >    why do we need to bit-swap the CRC reminader as is done in all your
> > examples.
>
> As far as I can tell, bit-swapping does nothing to help or hurt
> the effectiveness of the CRC.  When I ran my examples, I thought
> that it would be the simplest for our CRC to different from the
> Ethernet CRC only in the polynomial.  Ethernet, FDDI, FC, and SCSI
> all do this, so I figured it would cause the least confusion if
> we did the same.
>
> > We can just complement it and append it after the DATA bytes.
> >  The other side also can just pass it through the CRC engine to
> check it.
> >
> > Regards
> > Sanjay Goyal
> >
> > -----Original Message-----
> > From: Mark Bakke [mailto:[email-masked]]
> > Sent: Thursday, August 16, 2001 1:43 PM
> > To: Steve Blightman; [email-masked]
> > Subject: Re: [Fwd: Crc-32c example in iSCSI spec]
> >
> > One more thing that might be helpful.  When the Ethernet
> > polynomial is used in SCSI to generate its CRC, the T10
> > doc specifies the remainder polynomial that one should see
> > after running the data with a valid CRC through.  For
> > the Ethernet CRC, this was specified as 0xc704dd7b.  This
> > remainder polynomial is taken before the CRC is complemented
> > and bit-reflected.  For iSCSI, I came up with a remainder of
> > 0x1c2d19ed.  Can anyone verify this result?
> >
> > --
> > Mark
> >
> > Mark Bakke wrote:
> > >
> > > Steve-
> > >
> > > I just ran some Ethernet packets with known CRCs through
> > > my iSCSI/Ethernet CRC generator, and found the same thing
> > > as you did.
> > >
> > > All of my examples need to be byte-swapped, along with the
> > > fix I already posted for the all-ones example.  Here is a
> > > new set of examples, which will be in -08.  I also ran
> > > 64 bytes of zeroes and ones, which now agree with your
> > > numbers as well.
> > >
> > > Thanks again for bringing this up.
> > >
> > > --
> > > Mark
> > >
> > >      07 CRC Examples
> > >
> > >         N.B. all Values are Hexadecimal
> > >
> > >           Byte:        0  1  2  3
> > >
> > >              0:       01 a0 00 00
> > >              4:       00 00 00 00
> > >              8:       00 00 00 00
> > >             12:       00 00 00 00
> > >             16:       04 05 00 00
> > >             20:       00 01 00 00
> > >             24:       00 00 00 05
> > >             28:       00 00 00 04
> > >             32:       2a 00 00 00
> > >             36:       00 00 00 00
> > >             40:       80 00 00 00
> > >             44:       00 00 00 00
> > >
> > >            CRC:       93 70 51 db
> > >
> > >         32 bytes of zeroes:
> > >
> > >           Byte:        0  1  2  3
> > >
> > >              0:       00 00 00 00
> > >            ...
> > >             28:       00 00 00 00
> > >
> > >            CRC:       aa 36 91 8a
> > >
> > >         32 bytes of ones:
> > >
> > >           Byte:        0  1  2  3
> > >
> > >              0:       ff ff ff ff
> > >            ...
> > >             28:       ff ff ff ff
> > >
> > >            CRC:       43 ab a8 62
> > >
> > >         32 bytes of incrementing 00..1f:
> > >
> > >           Byte:        0  1  2  3
> > >
> > >              0:       00 01 02 03
> > >            ...
> > >             28:       1c 1d 1e 1f
> > >
> > >            CRC:       4e 79 dd 46
> > >
> > >
> > > Steve Blightman wrote:
> > > >
> > > > I believe the examples for the ISCSI CRC  have the wrong endianness.
> > > >
> > > > As yiou suugested over the phone I ran some Ethernet frames
> through a
> > > > simulation. I have some difficulty running the exact simulations you
> > > > wanted becuase the minimum size Ethernet frame is 64 bytes.
> > > >
> > > > However using the Ethernet CRC polynomial,
> > > > Running 64 bytes of 0 onto the wire, we append "36 63 8d
> 75" onto the
> > > > wire for the CRC - "36" goes out first.
> > > > Running 64 bytes of all 1, we append "ba 87 61 0f" onto the
> wire - "ba"
> > > > goes out first.
> > > >
> > > > Using the same logic for the ISCSI polynomial
> > > > Running 64 bytes of 0 I think we should append "67 eb c8 03" - "67"
> > > > going out first
> > > > and running 64 bytes of all 1 we should append "66 4e cd 2f" - "66"
> > > > going out first
> > > >
> > > > And now for 32 bytes with the ISCSI polynomial
> > > >
> > > > Running 32 bytes of 0 we should append "aa 36 91 8a" - "aa"
> going out
> > > > first
> > > > Running 32 bytes of all 1 we should append "43 ab a8 62" -
> "43" going
> > > > out first
> > > >
> > > > I don't want to get into an endless endian debate, but I
> believe it is
> > > > important to get the order of these bytes in the right
> order, so that we
> > > > can use the same hardware to check as well as to generate CRCs.
> > > >
> > > > Thanks for your help on this,
> > > > Steve Blightman
> > >
> > > --
> > > Mark A. Bakke
> > > Cisco Systems
> > > [email-masked]
> > > 763.398.1054
> >
> > --
> > Mark A. Bakke
> > Cisco Systems
> > [email-masked]
> > 763.398.1054
> >
> >
> ------------------------------------------------------------------
> ----------------------------------
> >
> >    Part 1.2    Type: application/ms-tnef
> >            Encoding: base64
>
> --
> Mark A. Bakke
> Cisco Systems
> [email-masked]
> 763.398.1054
>

---

From: Sanjay Goyal <[email-masked]>
Date: Fri, 17 Aug 2001 17:40:37 -0400

Stephen
 I would agree that checking for all zeros is always faster.

Regards
Sanjay G

-----Original Message-----
From: Wheat, Stephen R [mailto:[email-masked]]
Sent: Friday, August 17, 2001 4:26 PM
To: 'Douglas Otis'; Mark Bakke; Sanjay Goyal
Cc: Ips (E-mail)
Subject: RE: iSCSI: [Fwd: Crc-32c example in iSCSI spec]

Stephen

-----Original Message-----
From: Douglas Otis [mailto:[email-masked]]
Sent: Friday, August 17, 2001 11:54 AM
To: Mark Bakke; Sanjay Goyal
Cc: Ips (E-mail)
Subject: RE: iSCSI: [Fwd: Crc-32c example in iSCSI spec]

Mark,

Doug

---

From: "THALER,PAT (A-Roseville,ex1)" <[email-masked]>
Date: Fri, 17 Aug 2001 18:54:25 -0400

Mark,

I agree that for iSCSI there is no technical advantage to the bit order
chosen for the CRC. Bit order through the CRC checker helps in serial data
transmission such as over an Ethernet. If the bits are checked in the same
order as they are transmitted, then they will get the full length of burst
error detection. If they aren't, then burst error protection can shrink
about in half. When iSCSI packets are on an Ethernet, the Ethernet CRC
provides this protection. The iSCSI CRC protects the packet when it isn't on
the LAN and, in that case, it is often moving byte parallel and when it is
moving serial the bytes might be sent in either order. 

The important thing is that the order be specified clearly so everyone does
the same thing.

Regards,
Pat

-----Original Message-----
From: Mark Bakke [mailto:[email-masked]]
Sent: Friday, August 17, 2001 10:06 AM
To: Sanjay Goyal
Cc: Ips (E-mail)
Subject: Re: iSCSI: [Fwd: Crc-32c example in iSCSI spec]

Sanjay Goyal wrote:
> 
> Hi
>  I get the remainder for iSCSI as 0x1c2d19ed if I complement my CRC and
then

>  The other side also can just pass it through the CRC engine to check it.
> 
> Regards
> Sanjay Goyal
> 
> > > As yiou suugested over the phone I ran some Ethernet frames through a
> > > Running 64 bytes of 0 onto the wire, we append "36 63 8d 75" onto the
> > > Running 64 bytes of all 1, we append "ba 87 61 0f" onto the wire -
"ba"
> > > Running 32 bytes of 0 we should append "aa 36 91 8a" - "aa" going out
> > > first
> > > Running 32 bytes of all 1 we should append "43 ab a8 62" - "43" going
> > > out first
> > >
> > > I don't want to get into an endless endian debate, but I believe it is
> > > important to get the order of these bytes in the right order, so that
we
> > > Thanks for your help on this,
----------------------------------------------------------------------------
------------------------

-- 
Mark A. Bakke
Cisco Systems
[email-masked]
763.398.1054

---

From: "Julian Satran" <[email-masked]>
Date: Sun, 19 Aug 2001 21:22:19 +0300

Here are some clarifications about how the CRC is to be calculated (Network
byte order etc.),
the assumptions on data flow, the value of the CRC register for a good
segment and a corrected set of examples:

The Appendix A part will read:

   The following table lists cyclic integrity checksums that can be
   negotiated for the digests and MUST be implemented by every iSCSI
   initiator and target. Note that these digest options have only error
   detection significance.

   +---------------------------------------------+
   | Name          | Description     | Generator |
   +---------------------------------------------+
   | crc-32C       | 32 bit CRC      | 11EDC6F41 |
   +---------------------------------------------+
   | none          | no digest                   |
   +---------------------------------------------+

   The generator polynomial for this digest is given in hex-notation, for
   example 3b stands for 0011 1011 - the polynomial x**5+X**4+x**3+x+1.

   The generator polynomial selected is evaluated in [Castagnioli93].
   When using the CRC the CRC register must be initialized to all 1s
   (0xFFFFFFFF) and the CRC bits must be complemented before transmission.
   Padding bytes, when present, in a segment covered by a CRC, should be
   set to 0 and are included in the CRC. The CRC should be calculated as
   follows:

      - data are assumed to be  in the numbering order that appears in the
      draft - start with byte 0 bit 0 continue byte 1 bit 0 etc. (Big
      Endian on bytes / Little Endian on bits)
      - the CRC register is initialized with all 1s (equivalent to
      complementing the first 32 bits of the message)
      - the n PDU bits are considered coefficients of a polynomial M(x) of
      order n-1, with bit 0 of byte 0 being x^(n-1)
      - the polynomial is multiplied by x^32 and divided by G(x)- the
      generator polynomial - producing a remainder R(x) of degree <= 31
      - the coefficients of R(x) are considered a 32 bit sequence
      - the bit sequence is complemented and the result is the CRC
      - after the last bit of the original segment the CRC bits are
      transmitted with x^31 first followed by x^30 etc. ( whenever examples
      are given the value to be specified in examples follows the same
      rules of representation as the rest of this document)
      - a receiver of a "good" segment (data or header) built using the
      generator 0x11EDC6F41 will end-up having in the CRC register the
      value 0x1c2d19ed (this a register value and not a word as outlined in
      this draft)

CRC Examples will read:

01   CRC Examples

   32 bytes of zeroes:

     Byte:        0  1  2  3

        0:       00 00 00 00
      ...
       28:       00 00 00 00

      CRC:       aa 36 91 8a

   32 bytes of ones:

     Byte:        0  1  2  3

        0:       ff ff ff ff
      ...
       28:       ff ff ff ff

      CRC:       43 ab a8 62

     Byte:        0  1  2  3

        0:       00 01 02 03
      ...
       28:       1c 1d 1e 1f

      CRC:       4e 79 dd 46

   32 bytes of decrementing ff..e0:

     Byte:        0  1  2  3

        0:       ff fe fd fc
      ...
       28:       e3 e2 e1 e0

      CRC:       5c db 3f 11