Subject: RE: iSCSI CRC: A CRC-checking example

From: "CAVANNA,VICENTE V (A-Roseville,ex1)" <[email-masked]>
Date: Wed, 22 Aug 2001 11:42:50 -0600

Julian and Mark and others,

After I took into account the bit "reflection"  and the byte order (Big
Endian on Bytes and Little Endian on bits) I did finally obtain, using
polynomial math, the (corrected) results you show in the CRC examples. The
results have also been confirmed independently by an implementor here at
Agilent. 

I am the one who originally suggested to Julian that we specify the CRC
algorithm the same way as in ethernet even though for iSCSI it is not really
important to initialize the CRC register to all 1s and to complement the CRC
before transmission since there are other means to detect extra or missing
PDU bytes. However I had not realzied until recently that conformance with
the ethernet algorithm implies bit reflection. I had not been aware that in
ethernet the bits are sent out on the serial link with the least significant
bit first AND that the corresponding message polynomial is formed from the
bits in the sequence that they appear on the serial link; thus the need for
bit "reflection". 

Now that I understand the need for bit reflection (taken into account in the
rocksoft parameterized CRC generator by setting the in and out reflection
flag parameters to TRUE) I am not sure I agree that we want it in iSCSI. The
penalty for full conformity with ethernet seems too great. If people feel
strongly that we must keep the bit reflection I think that to make the
existing documentation clear and unambiguous we would need to explicitly
show the mapping of bits in the iSCSI PDU to coefficients of the message
polynomial that represents the iSCSI PDU. We would also need to show the
mapping of the CRC bits to the coefficients of its representative
polynomial. 

If you don't agree I will elaborate further later this week to try to
convince you. My objective is to be able to easily and unambiguously
describe the polynomial math behind the algorithm; right now, with the bit
reflection and without the explicit mapping it is awkward.

Vince

---

From: "Douglas Otis" <[email-masked]>
Date: Wed, 22 Aug 2001 12:41:05 -0700

Vince,

The table reflection is a good thing as it actually reduces the instructions
required within the algorithm.

For the non-reflected table lookup you have:
CRC32(c,d) (c=(c<<8)^crc32tab[(c>>24)&0xFF])

For the reflected table lookup you have:
CRC32(c,d) (c=(c>>8)^crc32tab[(c^(d))&0xFF])

In addition, for storage with a little endian processor, there is no need to
exchange byte order to store in big endian order as the reflected table has
already provided that byte order.

The only thing that I think should be considered is whether the CRC is
stored inverted.

Julian has already responded to my query on this subject with this
statement:

"The 2 "additions" you will find in the CRC calculation:

   initializing the "remainder" field with all ones
   (instead of the implied shifting in of 0's) and
   XORing the result with a mask (usually "full house")

are done to improve the error correction capabilities of CRCs.
CRCs are weak in detecting the spurious addition of leading 0s (that can
result from clocking errors) and the deletion of initial 1 with spurious
insertion of a 1 at the end."

If the CRC is initialized with all one's at the beginning, I fail to see the
need to invert before storing the CRC as this ensures there will be a
pattern created even with an all zeros block.  How does inverting this
pattern aid in detecting errors?

Doug

---

From: "Douglas Otis" <[email-masked]>
Date: Wed, 22 Aug 2001 12:51:33 -0700

Follow on:

Slight correction:

 For the non-reflected table lookup you have:
 CRC32(c,d) (c=(c<<8)^crc32tab[((c>>24)^(d))&0xFF])

Doug

---

From: Steve Blightman <[email-masked]>
Date: Wed, 22 Aug 2001 14:39:52 -0700

Just a small point -
If c is an unsigned 32 bit value and d is an unsigned 8 bit value, on
most machines for the non-reflected lookup I don't think the "&" is
necessary, is it?  Shifting right 24 places ensures the first 24 bits
are 0.

i.e For the non-reflected table lookup you have:
 CRC32(c,d) (c=(c<<8)^crc32tab[(c>>24)^(d)])

and the same number of instructions as for the reflected case.

As for all these issues about the reflection and the iversion, I suspect
we have all spent too much time worrying about this already.  These must
be very minor issues - the important thing is that the specification
clearly state which way it is to be done.  For those of us preparing to
implement the CRC in hardware, the arguments are very unsettling.  Must
I now add even more gates to make the reflection and the inversion
programmable, in case the committee changes its mind?

Steve

---

From: "Douglas Otis" <[email-masked]>
Date: Wed, 22 Aug 2001 15:13:32 -0700

Steve,

You are right, but a minor point in assembly, this masking issue can be
handled by a register designation rather than an actual instruction.  This
still leaves you with a 24 bit shift as an additional instruction to deal
with. I was suggesting that we not change the current approach and would not
support such a change.  I questioned the need for the inversion of the
stored CRC and if this is removed, it would mean fewer logic gates to
implement or perhaps an exclusive-or rather than an inverter if you wished
it to be programmable.  I have no great concern one way of the other, but in
reviewing all that is included in the CRC calculation, this inverted store
is perhaps one area that is perhaps not needed if we were wishing to reduce
complexity.

Doug