Subject: Markers

From: [email-masked]
Date: Tue, 6 Aug 2002 17:57:36 -0600

I was reading the text of Appendix A and noticed an ambiguity.

The text says:
The Marker indicates the offset to the next iSCSI PDU header. The 
   Marker is eight bytes in length and contains two 32-bit offset fields 
   that indicate how many bytes to skip in the TCP stream in order to 
   find the next iSCSI PDU header. The marker uses two copies of the 
   pointer so that a marker that spans a TCP packet boundary should 
   leave at least one valid copy in one of the packets.

From this text, it is not clear whether the two "copies" of the marker are identical copies or whether each marker specifies how many bytes to skip from its position in the stream. It also isn't entirely clear where the count of bytes to skip starts (which is related to the first). 

For example, if a PDU starts right after a marker:

      Next-iSCSI-PDU-start pointer - copy #1
      Next-iSCSI-PDU-start pointer - copy #2
      PDU header start

Do Copy #1 and Copy #2 both contain 0x00000000, or does copy #1 contain 0x00000004 because the packet starts 4 bytes after copy 1 while copy #2 contains 0x00000000
because the packet starts 0 bytes after copy 2? 

Since "copy" implies they are the same, I think they should both contain 0x00000000.

This could be clarified by adding, "The count of bytes begins with the first byte after the Marker." One could also include the example above.

Regards,
Pat

---

From: "Michael J. S. Smith (PacBell)" <[email-masked]>
Date: Tue, 06 Aug 2002 19:16:42 -0700

MarkersI helped write our hardware specification on this issue and going over the specification again just now, I realized that I referenced the Internet draft for the definitive description of marker insertion. So, I certainly have an interest in getting this right.

>> From this text, it is not clear whether the two "copies" of the marker are identical copies 

Pat, you are right, you need to look beyond the text that you quoted to get clarity. Some ambiguity is resolved when you realize that markers "don't count". 

 [quote]

   The Marker scheme uses payload byte stream counting that includes 
   every byte placed by iSCSI in the TCP stream except for the markers 
   themselves. It also excludes any bytes that TCP counts but are not 
   originated by iSCSI.
    
[end quote]

Thus,

1. Since "counting ... includes every byte ... except for the markers themselves," I believe the markers are both identical (in particular, the count in the two markers is the same, since we don't count the marker bytes).

[quote]
   Marker is eight bytes in length and contains two 32-bit offset fields 
   that indicate how many bytes to skip in the TCP stream in order to 
   find the next iSCSI PDU header. 
[end quote]

[quote]

   The marker interval and the initial marker-less interval are counted 
   in terms of the bytes placed in the TCP stream data by iSCSI. 
[end quote]

>> From this text, it is not clear ... whether each marker specifies how many bytes to skip from its position in the stream. It also isn't entirely clear where the count of bytes to skip starts (which is related to the first). 

Again the two markers don't matter. Thus,

2. The count is how many TCP stream bytes to skip, forget about the markers. The definition of "TCP Stream" needs a stretch, but between the first and third sections quoted above, I think it is "TCP Stream := every byte placed by iSCSI in the TCP stream except for the markers themselves and any bytes that TCP counts but are not originated by iSCSI".

I had to think a few times about "bytes that TCP counts but are not originated by iSCSI", and I can't think of any strange cases that leave ambiguity in this definition, but can anyone else? 

On the other hand if we were to add ""The count of bytes begins with the first byte after the Marker", I think there is a possibility for confusion unless you again declare "count of bytes" means TCP Stream (as defined).

I'm probably too close to this now to render an objective opinion on whether ambiguity exists, but I think an example will help, and Pat's is as good as any to resolve this particular (possible) ambiguity (in her example, I believe both marker copies contain zero).

Onto a very related issue: Our technical team just got back from the UNH plugfest and were unable to perform more testing on our marker implementation because nobody else at UNH had implemented markers yet. We would be very interested in hearing from anyone who would like to test against us. Please contact me as a starting point. Thanks!

Aloha

Mike Smith
CTO, iReady
[email-masked]

----- Original Message ----- 
From: [email-masked] 
To: [email-masked] 
Sent: Tuesday, August 06, 2002 4:57 PM
Subject: Markers

The text says: 

Regards, 
Pat

---

From: "Michael J. S. Smith (PacBell)" <[email-masked]>
Date: Tue, 06 Aug 2002 19:16:42 -0700

[quote]

[end quote]

Thus,

[quote]

[quote]

Aloha

Mike Smith
CTO, iReady
[email-masked]

----- Original Message ----- 
From: [email-masked] 
To: [email-masked] 
Sent: Tuesday, August 06, 2002 4:57 PM
Subject: Markers

The text says: 

Regards, 
Pat

---

From: Bill Studenmund <[email-masked]>
Date: Wed, 7 Aug 2002 12:46:51 -0700 (PDT)

On Tue, 6 Aug 2002, Michael J. S. Smith (PacBell) wrote:

> MarkersI helped write our hardware specification on this issue and
> going over the specification again just now, I realized that I
> referenced the Internet draft for the definitive description of marker
> insertion. So, I certainly have an interest in getting this right.
>
> Pat, you are right, you need to look beyond the text that you quoted
> to get clarity. Some ambiguity is resolved when you realize that
> markers "don't count".
>
>  [quote]
>
> 1. Since "counting ... includes every byte ... except for the markers
> themselves," I believe the markers are both identical (in particular,
> the count in the two markers is the same, since we don't count the
> marker bytes).
>
> [quote]
> >> From this text, it is not clear ... whether each marker specifies
> >> how many bytes to skip from its position in the stream. It also isn't
> >> entirely clear where the count of bytes to skip starts (which is
> >> related to the first).
>
> 2. The count is how many TCP stream bytes to skip, forget about the
> markers. The definition of "TCP Stream" needs a stretch, but between
> the first and third sections quoted above, I think it is "TCP Stream
> := every byte placed by iSCSI in the TCP stream except for the markers
> themselves and any bytes that TCP counts but are not originated by
> iSCSI".
>
> I had to think a few times about "bytes that TCP counts but are not
> originated by iSCSI", and I can't think of any strange cases that
> leave ambiguity in this definition, but can anyone else?

No, I think that's a good definition.

Though do you have examples of "bytes that TCP counts but are not
originated by iSCSI" that aren't markers?

> On the other hand if we were to add ""The count of bytes begins with
> the first byte after the Marker", I think there is a possibility for
> confusion unless you again declare "count of bytes" means TCP Stream
> (as defined).
>
> I'm probably too close to this now to render an objective opinion on
> whether ambiguity exists, but I think an example will help, and Pat's
> is as good as any to resolve this particular (possible) ambiguity (in
> her example, I believe both marker copies contain zero).

I'd like to suggest another, more complicated example that I think would
remove all ambiguity.

Scenario: We have data digests enabled for CRC32C, and markers every 2k of
data (every 512 int32's of data). We are 4 bytes away from our next
marker, and we send 8k worth of zeros.

We would write (I believe, please help check me on this):

four bytes of zero: 00 00 00 00

Marker (we're 8k away from next pdu): 00 00 20 00 00 00 20 00

2k of data: all zeros  (00 00 00 00 .....)

Marker (we're now 6k away):           00 00 18 00 00 00 18 00

2k of data: all zeros  (00 00 00 00 .....)

Marker (now 4k away):                 00 00 10 00 00 00 10 00

2k of data: all zeros  (00 00 00 00 .....)

Marker (now 2k away):                 00 00 08 00 00 00 08 00

2044 bytes of data, all that's left: (00 00 00 00 .....)

The data digest: (in transmission order) 23 46 44 90

Marker (now 0 away):                  00 00 00 00 00 00 00 00

[Next PDU]

Does that look right?

Also, this example can serve as another CRC example.

Take care,

Bill