Subject: RE: iSCSI: Nailing down CRC-32C

From: "mehyar nima" <[email-masked]>
Date: Tue, 08 May 2001 14:56:58 -0700

Assumptions:

1) The iSCSI DATA PDU Header and Data Integrity checks use CRC-32
2) Support of Integrity check is mandatory for both header and Data 
   digest
3) An iSCSI PDU may span multiple TCP segments. 
4) Any of these segments may arrive out of order at the receiver.
5) To regenerate the CRC for comparison with the value present in the 
   PDU received, the receiver needs to keep in a temporary buffers 
   all segments, waiting for the missing one, as CRC must be computed 
   using the whole re-assembled PDU

Question: Does CRC-32 support partial computation and compensation 
          for wrong initial values later (I guess NO!), to enable 
          partial CRC calc on avail parts of PDU?

Possible answer: No

Possible Conclusion: The RCV side is further complicated b/c of this 
                     CRC

Possible solution:

1) Use CRC that allows partial calc
2) Limit iSCSI PDU to TCP MSS if data CRC is used (added o/h)...
3) Leave as it, though it has cost , performance implication and hurts
   scaling to 10G..

Nima

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: Saturday, May 05, 2001 9:21 AM
To: [email-masked]
Subject: Re: iSCSI: Nailing down CRC-32C

The CRC part of the appendix (for 07) reads:

   The following table lists cyclic integrity checksums that can be
   negotiated for the digests and MUST be implemented by every iSCSI
   initiator and target. Note that these digest options have only error
   detection significance.

   +---------------------------------------------+
   | Name          | Description                 |
   +---------------------------------------------+
   | crc-32C       | 32 bit CRC      | 11EDC6F41 |
   +---------------------------------------------+
   | none          | no digest                   |
   +---------------------------------------------+

   The generator polynomials for those digests are given in hex-notation,
   for example 3a stands for 0011 1010 - the polynomial x**5+X**4+x**3+x+1.

   The generator polynomial selected is evaluated in [Castagnioli93].
   When using the CRC the CRC register must be initialized to all 1s
   (0xFFFFFFFF) and the CRC bits must be complemented before transmission.
   Padding bytes, when presents in a segment covered by a CRC, have to be
   set to 0 and are included in the CRC.

   Regards,
   Julo

Mark Bakke <[email-masked]> on 05-05-2001 00:13:09

Please respond to Mark Bakke <[email-masked]>

To:   IPS <[email-masked]>
cc:
Subject:  iSCSI: Nailing down CRC-32C

At the interim meeting, it was stated that iSCSI has selected
the CRC-32C polynomial as its required iSCSI-level header and
data CRC.  Now that we have it, it's time to move on and make
sure we can implement it.

In the interest of interoperability, we need to not only specify
the polynomial, but also the initial values, bit and byte
ordering, etc.

"A Painless Guide to CRC Error Detection Algorithms"
(http://www.ross.net/crc/crcpaper.html) specifies a
method to unambiguously characterize these parameters
(sections 15 and 16).  Has anyone taken a shot at defining
these yet?  Otherwise, here is what it might look like:

Name   : "CRC-32C"
Width  : 32
Poly   : 1EDC6F41   (note that the leading "1" is implied)
Init   : FFFFFFFF
RefIn  : True
RefOut : True
XorOut : FFFFFFFF
Check  : ?

I haven't attempted to create check data based on these yet.  As
soon as the other parameters are nailed down, we need to do this.

Anyway, I am not a CRC expert, and can't make any statement about
the above values being the "best" way to do this, but instead just
copied them (except the polynomial itself) from the Ethernet CRC,
since that is likely the easiest for everyone implementing hardware
to deal with.

If someone else is already doing this, let me know; I just wanted
to start this thread so we can get closure.

Regards,

Mark

--
Mark A. Bakke
Cisco Systems
[email-masked]
763.398.1054

Get 250 color business cards for FREE!
http://businesscards.lycos.com/vp/fastpath/

---

From: Venkat Rangan <[email-masked]>
Date: Tue, 8 May 2001 20:30:43 -0700

Julian,

Is this example correct? How do we get the final "+1" is the last bit is
'0'?

>   for example 3a stands for 0011 1010 - the polynomial x**5+X**4+x**3+x+1.

Venkat Rangan
Rhapsody Networks Inc.
http://www.rhapsodynetworks.com

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: Saturday, May 05, 2001 9:21 AM
To: [email-masked]
Subject: Re: iSCSI: Nailing down CRC-32C

   Regards,
   Julo

Name   : "CRC-32C"
Width  : 32

Regards,

Mark

--
Mark A. Bakke
Cisco Systems
[email-masked]
763.398.1054

---

From: [email-masked]
Date: Tue, 8 May 2001 17:38:08 +0300

Mark,

My basic assumption was that everything is (as in all IP related standards)
in network order.

Regards,
Julo

Mark Bakke <[email-masked]> on 07-05-2001 23:00:28

To:   Julian Satran/Haifa/IBM@IBMIL
cc:   [email-masked]
Subject:  Re: iSCSI: Nailing down CRC-32C

Julian-

That's great; it covers nearly everything.  The only thing left
is the byte order; are they taken in the same order Ethernet uses?

If so, I can generate a few test patterns that our implementations
can all check against.

Thanks,

--
Mark

--
Mark A. Bakke
Cisco Systems
[email-masked]
763.398.1054

---

From: [email-masked]
Date: Wed, 9 May 2001 08:26:09 +0300

It is a typo - the text reads now 3b 00111011 - Julo

Venkat Rangan <[email-masked]> on 09-05-2001 06:30:43

Please respond to Venkat Rangan <[email-masked]>

To:   Julian Satran/Haifa/IBM@IBMIL, [email-masked]
cc:
Subject:  RE: iSCSI: Nailing down CRC-32C

Julian,

Venkat Rangan
Rhapsody Networks Inc.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: Saturday, May 05, 2001 9:21 AM
To: [email-masked]
Subject: Re: iSCSI: Nailing down CRC-32C

   Regards,
   Julo

Name   : "CRC-32C"
Width  : 32

Regards,

Mark

--
Mark A. Bakke
Cisco Systems
[email-masked]
763.398.1054

---

From: Venkat Rangan <[email-masked]>
Date: Tue, 15 May 2001 14:51:52 -0700

Mark,

Is there any specific advantage to iSCSI Reflection Parameters
to be different from that of Ethernet? Does CRC32-C benefit by
a different set of parameters? If so, what are they, and how do we
get closure on a particular set of parameters?

Purely from the description of the parameters, RefIn=TRUE means that each
byte from the stream is 'reflected' so that bit 0 is fed in first and
then bit 1 etc. RefOut=TRUE means that the 32-bit value of the CRC is also
reflected before the final XOR is done.

Currently, Ethernet has 'RefIn=TRUE' and 'RefOut= TRUE'.

Thanks,

Venkat Rangan
Rhapsody Networks Inc.

-----Original Message-----
From: Mark Bakke [mailto:[email-masked]]
Sent: Tuesday, May 15, 2001 5:16 AM
To: Venkat Rangan
Cc: [email-masked]'; [email-masked]
Subject: Re: iSCSI: Nailing down CRC-32C

Venkat-

I assumed the same parameters (iSCSI has not yet specified
the reflection parameters), and obtained the same check you
did with their code.

--
Mark

Venkat Rangan wrote:
> 
> All,
> 
> Based on the finalization of the parameters of CRC-32C per following
> parameters, I obtained a check of 0xE3069283 for the nine-byte string of
> "123456789" (9 bytes) that the Rocksoft CRC document refers to.
> 
> Name   : "CRC-32C"
> Width  : 32
> Check  : E3069283
> It would be nice if other independant checks can validate this.
> 
> Venkat Rangan
> Rhapsody Networks Inc.
> The table and code follows:
> 
> /*****************************************************************/
> /*                                                               */
> /* CRC LOOKUP TABLE                                              */
> /* ================                                              */
> /* The following CRC lookup table was generated automagically    */
> /* by the Rocksoft^tm Model CRC Algorithm Table Generation       */
> /* Program V1.0 using the following model parameters:            */
> /*                                                               */
> /*    Width   : 4 bytes.                                         */
> /*    Poly    : 0x1EDC6F41L                                      */
> /*    Reverse : TRUE.                                            */
> /*                                                               */
> /* For more information on the Rocksoft^tm Model CRC Algorithm,  */
> /* see the document titled "A Painless Guide to CRC Error        */
> /* Detection Algorithms" by Ross Williams                        */
> /* ([email-masked].). This document is likely to be  */
> /* in the FTP archive "ftp.adelaide.edu.au/pub/rocksoft".        */
> /*                                                               */
> /*****************************************************************/
> 
> unsigned long  crctable[256] =
> {
>  0x00000000L, 0xF26B8303L, 0xE13B70F7L, 0x1350F3F4L,
>  0xC79A971FL, 0x35F1141CL, 0x26A1E7E8L, 0xD4CA64EBL,
>  0x8AD958CFL, 0x78B2DBCCL, 0x6BE22838L, 0x9989AB3BL,
>  0x4D43CFD0L, 0xBF284CD3L, 0xAC78BF27L, 0x5E133C24L,
>  0x105EC76FL, 0xE235446CL, 0xF165B798L, 0x030E349BL,
>  0xD7C45070L, 0x25AFD373L, 0x36FF2087L, 0xC494A384L,
>  0x9A879FA0L, 0x68EC1CA3L, 0x7BBCEF57L, 0x89D76C54L,
>  0x5D1D08BFL, 0xAF768BBCL, 0xBC267848L, 0x4E4DFB4BL,
>  0x20BD8EDEL, 0xD2D60DDDL, 0xC186FE29L, 0x33ED7D2AL,
>  0xE72719C1L, 0x154C9AC2L, 0x061C6936L, 0xF477EA35L,
>  0xAA64D611L, 0x580F5512L, 0x4B5FA6E6L, 0xB93425E5L,
>  0x6DFE410EL, 0x9F95C20DL, 0x8CC531F9L, 0x7EAEB2FAL,
>  0x30E349B1L, 0xC288CAB2L, 0xD1D83946L, 0x23B3BA45L,
>  0xF779DEAEL, 0x05125DADL, 0x1642AE59L, 0xE4292D5AL,
>  0xBA3A117EL, 0x4851927DL, 0x5B016189L, 0xA96AE28AL,
>  0x7DA08661L, 0x8FCB0562L, 0x9C9BF696L, 0x6EF07595L,
>  0x417B1DBCL, 0xB3109EBFL, 0xA0406D4BL, 0x522BEE48L,
>  0x86E18AA3L, 0x748A09A0L, 0x67DAFA54L, 0x95B17957L,
>  0xCBA24573L, 0x39C9C670L, 0x2A993584L, 0xD8F2B687L,
>  0x0C38D26CL, 0xFE53516FL, 0xED03A29BL, 0x1F682198L,
>  0x5125DAD3L, 0xA34E59D0L, 0xB01EAA24L, 0x42752927L,
>  0x96BF4DCCL, 0x64D4CECFL, 0x77843D3BL, 0x85EFBE38L,
>  0xDBFC821CL, 0x2997011FL, 0x3AC7F2EBL, 0xC8AC71E8L,
>  0x1C661503L, 0xEE0D9600L, 0xFD5D65F4L, 0x0F36E6F7L,
>  0x61C69362L, 0x93AD1061L, 0x80FDE395L, 0x72966096L,
>  0xA65C047DL, 0x5437877EL, 0x4767748AL, 0xB50CF789L,
>  0xEB1FCBADL, 0x197448AEL, 0x0A24BB5AL, 0xF84F3859L,
>  0x2C855CB2L, 0xDEEEDFB1L, 0xCDBE2C45L, 0x3FD5AF46L,
>  0x7198540DL, 0x83F3D70EL, 0x90A324FAL, 0x62C8A7F9L,
>  0xB602C312L, 0x44694011L, 0x5739B3E5L, 0xA55230E6L,
>  0xFB410CC2L, 0x092A8FC1L, 0x1A7A7C35L, 0xE811FF36L,
>  0x3CDB9BDDL, 0xCEB018DEL, 0xDDE0EB2AL, 0x2F8B6829L,
>  0x82F63B78L, 0x709DB87BL, 0x63CD4B8FL, 0x91A6C88CL,
>  0x456CAC67L, 0xB7072F64L, 0xA457DC90L, 0x563C5F93L,
>  0x082F63B7L, 0xFA44E0B4L, 0xE9141340L, 0x1B7F9043L,
>  0xCFB5F4A8L, 0x3DDE77ABL, 0x2E8E845FL, 0xDCE5075CL,
>  0x92A8FC17L, 0x60C37F14L, 0x73938CE0L, 0x81F80FE3L,
>  0x55326B08L, 0xA759E80BL, 0xB4091BFFL, 0x466298FCL,
>  0x1871A4D8L, 0xEA1A27DBL, 0xF94AD42FL, 0x0B21572CL,
>  0xDFEB33C7L, 0x2D80B0C4L, 0x3ED04330L, 0xCCBBC033L,
>  0xA24BB5A6L, 0x502036A5L, 0x4370C551L, 0xB11B4652L,
>  0x65D122B9L, 0x97BAA1BAL, 0x84EA524EL, 0x7681D14DL,
>  0x2892ED69L, 0xDAF96E6AL, 0xC9A99D9EL, 0x3BC21E9DL,
>  0xEF087A76L, 0x1D63F975L, 0x0E330A81L, 0xFC588982L,
>  0xB21572C9L, 0x407EF1CAL, 0x532E023EL, 0xA145813DL,
>  0x758FE5D6L, 0x87E466D5L, 0x94B49521L, 0x66DF1622L,
>  0x38CC2A06L, 0xCAA7A905L, 0xD9F75AF1L, 0x2B9CD9F2L,
>  0xFF56BD19L, 0x0D3D3E1AL, 0x1E6DCDEEL, 0xEC064EEDL,
>  0xC38D26C4L, 0x31E6A5C7L, 0x22B65633L, 0xD0DDD530L,
>  0x0417B1DBL, 0xF67C32D8L, 0xE52CC12CL, 0x1747422FL,
>  0x49547E0BL, 0xBB3FFD08L, 0xA86F0EFCL, 0x5A048DFFL,
>  0x8ECEE914L, 0x7CA56A17L, 0x6FF599E3L, 0x9D9E1AE0L,
>  0xD3D3E1ABL, 0x21B862A8L, 0x32E8915CL, 0xC083125FL,
>  0x144976B4L, 0xE622F5B7L, 0xF5720643L, 0x07198540L,
>  0x590AB964L, 0xAB613A67L, 0xB831C993L, 0x4A5A4A90L,
>  0x9E902E7BL, 0x6CFBAD78L, 0x7FAB5E8CL, 0x8DC0DD8FL,
>  0xE330A81AL, 0x115B2B19L, 0x020BD8EDL, 0xF0605BEEL,
>  0x24AA3F05L, 0xD6C1BC06L, 0xC5914FF2L, 0x37FACCF1L,
>  0x69E9F0D5L, 0x9B8273D6L, 0x88D28022L, 0x7AB90321L,
>  0xAE7367CAL, 0x5C18E4C9L, 0x4F48173DL, 0xBD23943EL,
>  0xF36E6F75L, 0x0105EC76L, 0x12551F82L, 0xE03E9C81L,
>  0x34F4F86AL, 0xC69F7B69L, 0xD5CF889DL, 0x27A40B9EL,
>  0x79B737BAL, 0x8BDCB4B9L, 0x988C474DL, 0x6AE7C44EL,
>  0xBE2DA0A5L, 0x4C4623A6L, 0x5F16D052L, 0xAD7D5351L
> };
> 
> /*****************************************************************/
> /*                   End of CRC Lookup Table                     */
> /*****************************************************************/
> 
> /*****************************************************************/
> /*                   ComputeCRC                                  */
> /*****************************************************************/
> #include <stdio.h>
> #include <stdlib.h>
> #include "crctable.h"
> 
> #define TB_INIT 0xFFFFFFFF
> #define TB_INIT_REFLECTED 0xFFFFFFFF
> #define TB_XOROT 0xFFFFFFFF
> 
>    unsigned long crc_reflected ();
>    unsigned long crc_reflected (blk_adr,blk_len)
>    unsigned char *blk_adr;
>    unsigned long  blk_len;
>    {
>     unsigned long crc = TB_INIT_REFLECTED;
>     while (blk_len--)
>        crc = crctable[(crc ^ *blk_adr++) & 0xFFL] ^ (crc >> 8);
>     return crc ^ TB_XOROT;
>    }
> 
> Sent: Tuesday, May 08, 2001 7:38 AM
> To: [email-masked]

-- 
Mark A. Bakke
Cisco Systems
[email-masked]
763.398.1054