Subject: RE: [Ips] MPA alignment

From: "John Hufferd" <[email-masked]>
Date: Tue, 18 Apr 2006 11:56:24 -0700

Eddy,

I agree with your statement about software RDMA being a bit silly, but
only when the thought is about implementations on servers.

There is at least one type of environment where using RDMA with a
software iWARP stack may not be too silly, and that is when the object
of the interconnect is to optimize the over all costs and save Server
Cycles.  In that case is may make sense to operate with a software RDMA
stack on the desktop systems or workstations.  Those environments often
have a plentiful amount of available CPU cycles on the
desktop/workstation.  So, the approach is to burn CPU cycles at the
desktop/workstation and save CPU cycles on the servers, which have
RNICs.  This type of a configuration actually seems to be a useful
configuration.

The same argument can be used with iSCSI and iSER, but only in the case
where there are RNICs in the target.  It is a little bit strange, as you
point out, to have the storage controller implement a software version
of iSER, except perhaps for some type of compatibility or migration
reasons.

.

.

.

John L Hufferd

Sr. Executive Director of Technology

Brocade Communications Systems, Inc

[email-masked]

Office Phone: (408) 333-5244; eFAX: (408) 904-4688

Alt Office Phone: (408) 997-6136; Cell: (408) 627-9606

  _____  

From: Eddy Quicksall [mailto:[email-masked]] 
Sent: Tuesday, April 18, 2006 5:48 AM
To: Mike Ko
Cc: [email-masked]
Subject: Re: [Ips] MPA alignment

Thanks,

Here is my feeling on this. If one were to use a standard TCP/IP stack
(that does copying) then it is a bit silly to use iSER. If one were to
use a software implementation and write/modify the stack to get the
desired effects of DDP then he will probably heed to the alignment notes
in the spec. So if one were to support misaligned segments using a
slower method then it would be OK.

Eddy

	----- Original Message ----- 

	From: Mike Ko <mailto:[email-masked]>  

	To: Eddy Quicksall
<mailto:[email-masked]>  

	Cc: [email-masked] 

	Sent: Tuesday, April 18, 2006 2:16 AM

	Subject: Re: [Ips] MPA alignment

	Eddy, 
	
	If your question is about MPA behavior in general, then it
should be directed to the rddp forum. 
	
	Assuming your question is related to iSER, then if iWARP is
implemented in software and uses the existing TCP/IP stack, then it
cannot guarantee that FPDU's will be aligned with TCP segments.  For the
general case where iWARP is implemented in an RNIC with an integrated
TCP/IP stack, then it is to the RNIC vendor's advantage to optimize his
product's performance and therefore FPDU's should be aligned with TCP
segments.  However, the receiver can still receive FPDU's which are not
aligned with TCP segments if there are resegmenting middleboxes
somewhere in the connection path. 
	
	Mike   

	To:        <[email-masked]> 
	cc:         
	Subject:        [Ips] MPA alignment 

	It is preferable to have TCP sender to align MPA header where a
TCP segment begins and to keep the MPA FPDU within a TCP segment, so
that the receiver hardware doesn't need to temporarily buffer the FPDU
for the CRC checking.   
	  
	Does anyone know of any software or hardware that is not an FPDU
alignment sender? 
	Does anyone have a feel for the likelihood of software or
hardware to not align FPDU's on a TCP segment? 
	  
	Eddy_______________________________________________
	Ips mailing list
	[email-masked]
	https://www1.ietf.org/mailman/listinfo/ips

  _____  

	_______________________________________________
	Ips mailing list
	[email-masked]
	https://www1.ietf.org/mailman/listinfo/ips

_______________________________________________
Ips mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ips

---

From: "Dan Bar Dov" <[email-masked]>
Date: Sun, 23 Apr 2006 11:47:02 +0300

I just wanted to note in addition to what John said, that ISER is not a symmetric protocol.
Only the target side performs the RDMA operations. The initiator does only the setup.
 
Dan

________________________________

	From: John Hufferd [mailto:[email-masked]] 
	Sent: Tuesday, April 18, 2006 9:56 PM
	To: Eddy Quicksall; Mike Ko
	Cc: [email-masked]
	Subject: RE: [Ips] MPA alignment

	Eddy,

	I agree with your statement about software RDMA being a bit silly, but only when the thought is about implementations on servers.

	There is at least one type of environment where using RDMA with a software iWARP stack may not be too silly, and that is when the object of the interconnect is to optimize the over all costs and save Server Cycles.  In that case is may make sense to operate with a software RDMA stack on the desktop systems or workstations.  Those environments often have a plentiful amount of available CPU cycles on the desktop/workstation.  So, the approach is to burn CPU cycles at the desktop/workstation and save CPU cycles on the servers, which have RNICs.  This type of a configuration actually seems to be a useful configuration.

	The same argument can be used with iSCSI and iSER, but only in the case where there are RNICs in the target.  It is a little bit strange, as you point out, to have the storage controller implement a software version of iSER, except perhaps for some type of compatibility or migration reasons.

	.

	.

	.

	John L Hufferd

	[email-masked]

	
________________________________

	Thanks,

	Here is my feeling on this. If one were to use a standard TCP/IP stack (that does copying) then it is a bit silly to use iSER. If one were to use a software implementation and write/modify the stack to get the desired effects of DDP then he will probably heed to the alignment notes in the spec. So if one were to support misaligned segments using a slower method then it would be OK.

	Eddy

		----- Original Message ----- 

		To: Eddy Quicksall <mailto:[email-masked]>  

		Cc: [email-masked] 

		
		Eddy, 
		
		If your question is about MPA behavior in general, then it should be directed to the rddp forum. 
		
		Assuming your question is related to iSER, then if iWARP is implemented in software and uses the existing TCP/IP stack, then it cannot guarantee that FPDU's will be aligned with TCP segments.  For the general case where iWARP is implemented in an RNIC with an integrated TCP/IP stack, then it is to the RNIC vendor's advantage to optimize his product's performance and therefore FPDU's should be aligned with TCP segments.  However, the receiver can still receive FPDU's which are not aligned with TCP segments if there are resegmenting middleboxes somewhere in the connection path. 
		
		Mike   

		To:        <[email-masked]> 
		cc:         

		It is preferable to have TCP sender to align MPA header where a TCP segment begins and to keep the MPA FPDU within a TCP segment, so that the receiver hardware doesn't need to temporarily buffer the FPDU for the CRC checking.   
		  
		Does anyone know of any software or hardware that is not an FPDU alignment sender? 
		Does anyone have a feel for the likelihood of software or hardware to not align FPDU's on a TCP segment? 

________________________________

---

From: "John Hufferd" <[email-masked]>
Date: Mon, 24 Apr 2006 00:31:19 -0700

Well, the initiator side must still field the RDMA messages sent by the
target (RDMA Read & RDMA write). And that means that the RDMA Read
response must be handled by initiator side, and of course the RDMA write
must be handled and data moved into the appropriate buffers when the
RDMA write message is received from the target.  

The software RDMA stack on the Initiator is really quite busy.

.

.

.

John L Hufferd

[email-masked]

  _____  

From: Dan Bar Dov [mailto:[email-masked]] 
Sent: Sunday, April 23, 2006 1:47 AM
To: John Hufferd; Eddy Quicksall; Mike Ko
Cc: [email-masked]
Subject: RE: [Ips] MPA alignment

I just wanted to note in addition to what John said, that ISER is not a
symmetric protocol.

Only the target side performs the RDMA operations. The initiator does
only the setup.

Dan

  _____  

	Eddy,

	I agree with your statement about software RDMA being a bit
silly, but only when the thought is about implementations on servers.

	There is at least one type of environment where using RDMA with
a software iWARP stack may not be too silly, and that is when the object
of the interconnect is to optimize the over all costs and save Server
Cycles.  In that case is may make sense to operate with a software RDMA
stack on the desktop systems or workstations.  Those environments often
have a plentiful amount of available CPU cycles on the
desktop/workstation.  So, the approach is to burn CPU cycles at the
desktop/workstation and save CPU cycles on the servers, which have
RNICs.  This type of a configuration actually seems to be a useful
configuration.

	The same argument can be used with iSCSI and iSER, but only in
the case where there are RNICs in the target.  It is a little bit
strange, as you point out, to have the storage controller implement a
software version of iSER, except perhaps for some type of compatibility
or migration reasons.

	.

	.

	.

	John L Hufferd

	[email-masked]

	
  _____  

	From: Eddy Quicksall
[mailto:[email-masked]] 
	Sent: Tuesday, April 18, 2006 5:48 AM
	To: Mike Ko
	Cc: [email-masked]
	Subject: Re: [Ips] MPA alignment

	Thanks,

	Here is my feeling on this. If one were to use a standard TCP/IP
stack (that does copying) then it is a bit silly to use iSER. If one
were to use a software implementation and write/modify the stack to get
the desired effects of DDP then he will probably heed to the alignment
notes in the spec. So if one were to support misaligned segments using a
slower method then it would be OK.

	Eddy

		----- Original Message ----- 

		To: Eddy Quicksall

		Cc: [email-masked] 

		
		Eddy, 
		
		If your question is about MPA behavior in general, then
it should be directed to the rddp forum. 
		
		Assuming your question is related to iSER, then if iWARP
is implemented in software and uses the existing TCP/IP stack, then it
cannot guarantee that FPDU's will be aligned with TCP segments.  For the
general case where iWARP is implemented in an RNIC with an integrated
TCP/IP stack, then it is to the RNIC vendor's advantage to optimize his
product's performance and therefore FPDU's should be aligned with TCP
segments.  However, the receiver can still receive FPDU's which are not
aligned with TCP segments if there are resegmenting middleboxes
somewhere in the connection path. 
		
		Mike   

		To:        <[email-masked]> 
		cc:         

		It is preferable to have TCP sender to align MPA header
where a TCP segment begins and to keep the MPA FPDU within a TCP
segment, so that the receiver hardware doesn't need to temporarily
buffer the FPDU for the CRC checking.   
		  
		Does anyone know of any software or hardware that is not
an FPDU alignment sender? 
		Does anyone have a feel for the likelihood of software
or hardware to not align FPDU's on a TCP segment? 

  _____