Subject: iSCSI Marker questions

From: Dean Scoville <[email-masked]>
Date: Mon, 10 Dec 2001 17:09:11 -0800

The iSCSI Draft 9 Appendix C makes the following statements about 
Markers and the Initial Marker-less Interval:

     "The offset to the next iSCSI PDU header is counted in terms
      of the TCP stream data. Anything counted in the TCP 
      sequence-number is counted for the offset. Specifically this 
      includes any bytes "inserted" in the TCP stream by an UFL and
      it excludes any other markers inserted between the one we are
      examining and the next PDU header."... 

     "To enable the connection setup including the login phase 
      negotiation, marking (if any) is started only at the first 
      marker interval after the end of the login phase."

I understand that markers are not inserted until after login phase. 
Am I correct to assume that the placement of the first marker 
determined by the TCP sequence numbers on the final Login Request/
Response PDUs, or is initial marker position determined by the 
TCP sequence numbers at connection establishment?

Assume the following interaction:

I->  SYN     (TCP sequenceNum=1000)  -- irrelevant to this discussion?

T->  SYN-ACK (TCP sequenceNum=2000)  -- irrelevant to this discussion?

I->  Login Request PDU, T=0,CSG=1,NSG=0:
     InitiatorName=xxx
     TargetName=yyy
     SessionType=normal
     ...
     FMarker=send-receive
     RFMarkInt=512,1024

T->  Login Response PDU, T=0,CSG=1,NSG=0:
     ...
     FMarker=send-receive
     SFMarkInt=1024
     RFMarkInt=1024

I->  Login Request PDU, T=1,CSG=1,NSG=3:
     SFMarkInt=1024
     (64-byte PDU... TCP sequenceNum=1301-1364)

T->  Login Response PDU, T=1,CSG=1,NSG=3:
     (48-byte PDU... TCP sequenceNum=2201-2248) 

The above interaction designates a 1024 x 4 = 4096-byte marker 
interval in both directions. The first PDU byte sent by the 
intitiator in full-feature mode will have sequenceNum=1365, and 
the first byte sent by the target will have sequenceNum=2249.

Assuming the markerless interval starts at the end of login 
phase, the first two markers in each direction will have the 
following TCP sequence numbers:

               TCP SeqNum of    TCP SeqNum of
               First Marker     Second Marker
               ------------     -------------
Initiator:     5461-5468        9565-9572
Target:        6345-6352        10449-10456

Is this the correct interpretation of marker usage in iSCSI 
Draft 9, or does marker placement depend on the connection's
initial sequence numbers?

Also, is "RFMarkInt=..." always considered an offer, and "SFMarkInt="
considered a reply to that offer? If an offer is sent with "FMarker=..."
and "RFMarkInt=...", MUST the reply contain either "FMarker=no" or
BOTH "FMarker=yes" and "SFMarkInt=..."?

Thanks,
Dean Scoville
QLogic Corp.

---

From: Dean Scoville <[email-masked]>
Date: Fri, 14 Dec 2001 10:10:47 -0800

Julian,
Thanks for the response. Some examples would be very helpful.
Your response shows a parameter negotiation with SFMarkInt=1,512
but Draft 9 doesn't allow a range for SFMarkInt... only for RFMarkInt. 
Is a range also being allowed for SFMarkInt?
 
>
> I assume a normal dialogue may go like: 
>
> I->FMarker=send-receive,RFMarkInt=1,4,SFMarkInt=1,512 
> T->FMarker=send-receive,RFMarkInt=8, SFMarkInt=2 
>

Section 2.2.4 of the draft currently requires all integer parameter 
negotiations to have a response . (Responses are optional only 
with boolean negotiations where the offered value allows only a
single outcome -- such as an "OR" boolean function with a "Yes"
offer). Are responses always required for RFMarkInt and SFMarkInt,
or is the following acceptable?
 
I-> FMarker=send-receive,RFMarkInt=1,4,SFMarkInt=1,512
T-> FMarker=send,SFMarkInt=4
 
Would it also be allowed to respond with the following and, if so,
is one preferred over the other? (I'm not sure about the current
state of using "0" as a "don't care" value, but is that also
permitted in cases where the response would be irrelevant?)
 
I-> FMarker=send-receive,RFMarkInt=1,4,SFMarkInt=1,512
T-> FMarker=send,SFMarkInt=4,RFMarkInt=irrelevant
 
With the following exchange, is the target required to respond
with any RFMarkInt or SFMarkInt values, since there is no
real choice of values? Agreeing to send markers implies that 
the interval is 512, and not agreeing to receive markers implies
that there is no receive interval. Is the following allowed, or
should "RFMarkInt=4" and "SFMarkInt=irrelevant" be returned 
by the target?
 
I-> FMarker=send-receive,RFMarkInt=4,SFMarkInt=512
T-> FMarker=send
 
Is the following acceptable, even though it doesn't reply to 
RFMarkInt and SFMarkInt?
 
I-> FMarker=send-receive,RFMarkInt=4,SFMarkInt=512
T-> FMarker=no
 
It would also be helpful to explain what happens if the login
phase goes beyond the negotiated first marker interval.
Assume, for example, the following:
 
  SYN - at TCP sequence num 1000.
  SYN-ACK - at TCP sequence num 2000.
  negotiate to FMarker=send-receive,RFMarkInt=100, SFMarkInt=100.
 
If the 1st byte of 1st initiator PDU in full-feature mode has TCP
seqNum=1404,
full-feature phase starts in the middle of what would have been a marker.
Is a partial marker inserted? What would be the TCP seqNum of the first and
second marker?
 
If the 1st byte of 1st target PDU in full-feature mode has TCP seqNum=2810,
what would be the TCP seqNum of the first and second marker?
 
thanks,
Dean

-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Tuesday, December 11, 2001 4:54 PM
To: [email-masked]
Subject: Fw: Re: iSCSI Marker questions

----- Forwarded by Julian Satran/Haifa/IBM on 12-12-01 02:39 ----- 

	Julian Satran 

11-12-01 14:44 

        To:        IPS List 
        cc:         
        Subject:        Re: iSCSI Marker questions Link
<Notes:///C225670D0041573F/3C1739896E8426C58525643000741AA5/4A77BD027CFE6266
C2256B1F00068C0D>  	

Dean, 

>                First Marker     Second Marker
No - the correct numbers are dependent only on the marker interval (not the
length of the login phase) and are: 

Initiator        5096-5103        9200-9201 
Target           6096-6103        10200-10201 

> BOTH "FMarker=yes" and "SFMarkInt=..."?

Fmarker is not boolean - legal values are no, send, receive, send-receive 
The sender and receiver must set the interval it wants/is ready to use 
otherwise the responder can't answer. 
I assume a normal dialogue may go like: 

I->FMarker=send-receive,RFMarkInt=1,4,SFMarkInt=1,512 
T->FMarker=send-receive,RFMarkInt=8, SFMarkInt=2 

Please observe that target answers with RFMarkInt to the initiators
SFMarkInt and viceversa. 

I will attempt an example in draft 10 (last?).

---

From: "Julian Satran" <[email-masked]>
Date: Sat, 15 Dec 2001 05:57:32 +0200

You are right there is no range today for SFMarkint - we may want to 
change this to enable a one-shot negotiation.

Julo

Dean Scoville <[email-masked]>
Sent by: [email-masked]
14-12-01 20:10

        To:     "ips@ece. cmu. edu (E-mail)" <[email-masked]>
        cc: 
        Subject:        RE: Re: iSCSI Marker questions

Julian,

 

 

 

If the 1st byte of 1st initiator PDU in full-feature mode has TCP 
seqNum=1404,
full-feature phase starts in the middle of what would have been a marker.
Is a partial marker inserted? What would be the TCP seqNum of the first 
and
second marker?
 
If the 1st byte of 1st target PDU in full-feature mode has TCP 
seqNum=2810,
what would be the TCP seqNum of the first and second marker?
 
thanks,
Dean
-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Tuesday, December 11, 2001 4:54 PM
To: [email-masked]
Subject: Fw: Re: iSCSI Marker questions

----- Forwarded by Julian Satran/Haifa/IBM on 12-12-01 02:39 ----- 

Julian Satran 
11-12-01 14:44 

        To:        IPS List 
        cc:         
        Subject:        Re: iSCSI Marker questionsLink 

Dean, 

>                First Marker     Second Marker
No - the correct numbers are dependent only on the marker interval (not 
the length of the login phase) and are:

---

From: Julian_Satran/Haifa/[email-masked]
Date: Sun, 16 Dec 2001 21:53:59 +0200

Somesh,

Marker interval is from the end-of-marker to the beginning-of-next marker. 

The pointers to the next header do not include markers either.

Calculating the marker position is trivial anyway.
Pointers are not dependent on the marker interval and can be computed 
without knowing the TCP sequence numbers (makes layering cleaner).

Regards,
Julo

"Somesh Gupta" <[email-masked]>
16-12-01 19:22
Please respond to somesh_gupta

        To:     Julian Satran/Haifa/IBM@IBMIL, <[email-masked]>
        cc: 
        Subject:        RE: Re: iSCSI Marker questions

Julian,
 
Should the marker interval be counted from the begining of 1 marker to the
begining of the next, or from the end of one marker to the begining of the 
next.
Having the marker interval counted from the begining of 1 marker to the 
begining
of the next makes it easier for the receiver to determine where the next 
marker
will be.
 
Somesh
-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf Of Julian Satran
Sent: Tuesday, December 11, 2001 4:54 PM
To: [email-masked]
Subject: Fw: Re: iSCSI Marker questions

----- Forwarded by Julian Satran/Haifa/IBM on 12-12-01 02:39 ----- 

Julian Satran 
11-12-01 14:44 

        To:        IPS List 
        cc:         

Dean,

---

From: Dean Scoville <[email-masked]>
Date: Mon, 17 Dec 2001 10:13:47 -0800

Julian,
An undesirable side affect of starting the marker interval at connection
establishment time and including the SYN char in the interval is that the
alignment of markers will ALWAYS be different than the alignment of iSCSI
PDUs. This is messy for anyone who is looking at things in 4-byte pieces.
The alignment of the markers will ALWAYS differ by one byte from the PDU
alignment. Using the example cited in the earlier posting, the first marker
byte will ALWAYS have an EVEN-numbered TCP sequence number, and the first
PDU byte will ALWAYS have an ODD-numbered TCP sequence number.
 
> I->  SYN     (TCP sequenceNum=1000)
> 
> T->  SYN-ACK (TCP sequenceNum=2000)
> 
>                TCP SeqNum of    
>                First Marker     
>                ------------     
> Initiator:     5096-5103        
> Target:        6096-6103 
 
Since the layer that inserts the markers needs to be aware of login anyway
(it can't insert markers during login phase), I would rather that the marker
interval didn't start until the end of login phase. It would avoid some
messy corner cases (see next paragraph). If this is unacceptable, then let's
at least not count the SYN char in the marker interval, so that the markers
can be on the same 4-byte alignment as the PDUs.
 
If the marker interval remains as it is currently defined, how are partial
markers treated at the end of login phase? Modifying the previous example a
little, if the initiator's final PDU during login phase had TCP sequence
numbers 5035-5098 should we insert the last 5-bytes of the marker into the
data stream? If the final PDU were at 5039-5102, would we insert 1-byte of
the marker into the data stream? Or would we skip the first marker
altogether because part of it would have begun during login phase? Either
way, it is messy.
 
-Dean

-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Friday, December 14, 2001 7:58 PM
To: Dean Scoville
Cc: ips@ece. cmu. edu (E-mail); [email-masked]
Subject: RE: Re: iSCSI Marker questions

You are right there is no range today for SFMarkint - we may want to change
this to enable a one-shot negotiation. 

Julo 

14-12-01 20:10 

Julian, 

  

  

  

  
  
thanks, 
Dean 
-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Tuesday, December 11, 2001 4:54 PM
To: [email-masked]
Subject: Fw: Re: iSCSI Marker questions

----- Forwarded by Julian Satran/Haifa/IBM on 12-12-01 02:39 ----- 
	Julian Satran 

11-12-01 14:44 

       To:        IPS List 
       cc:         
       Subject:        Re: iSCSI Marker questions
<Notes:///C225670D0041573F/3C1739896E8426C58525643000741AA5/4A77BD027CFE6266
C2256B1F00068C0D> Link 	

Dean,

---

From: "Julian Satran" <[email-masked]>
Date: Mon, 17 Dec 2001 21:15:45 +0200

Dean,

Very good point.

I think we will want to explicitly say that the TCP sequence number to 
start counting will be the first sequence number used by iSCSI (the first 
Login Request and the first Login Response). 

Thanks for bringing this to our attention.

Julo

[email-masked] wrote on 17-12-2001 20:13:47:

> Julian,
> 
> An undesirable side affect of starting the marker interval at 
> connection establishment time and including the SYN char in the 
> interval is that the alignment of markers will ALWAYS be different 
> than the alignment of iSCSI PDUs. This is messy for anyone who is 
> looking at things in 4-byte pieces. The alignment of the markers 
> will ALWAYS differ by one byte from the PDU alignment. Using the 
> example cited in the earlier posting, the first marker byte will 
> ALWAYS have an EVEN-numbered TCP sequence number, and the first PDU 
> byte will ALWAYS have an ODD-numbered TCP sequence number.
> 
> 
> 
> Since the layer that inserts the markers needs to be aware of login 
> anyway (it can't insert markers during login phase), I would rather 
> that the marker interval didn't start until the end of login phase. 
> It would avoid some messy corner cases (see next paragraph). If this
> is unacceptable, then let's at least not count the SYN char in the 
> marker interval, so that the markers can be on the same 4-byte 
> alignment as the PDUs.
> 
> 
> 
> If the marker interval remains as it is currently defined, how are 
> partial markers treated at the end of login phase? Modifying the 
> previous example a little, if the initiator's final PDU during login
> phase had TCP sequence numbers 5035-5098 should we insert the last 
> 5-bytes of the marker into the data stream? If the final PDU were at
> 5039-5102, would we insert 1-byte of the marker into the data 
> stream? Or would we skip the first marker altogether because part of
> it would have begun during login phase? Either way, it is messy.
> 
> 
> 
> -Dean
> 
seqNum=1404,
and 
seqNum=2810, 
> No - the correct numbers are dependent only on the marker interval 
> (not the length of the login phase) and are: 
>