Return-Path: X-Sieve: cmu-sieve 2.0 Return-Path: Received: (from majordom@localhost) by ece.cmu.edu (8.11.0/8.10.2) id h3I7ar511572 for ips-outgoing; Fri, 18 Apr 2003 03:36:53 -0400 (EDT) X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [194.196.100.235]) by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id h3I7ao311566; Fri, 18 Apr 2003 03:36:51 -0400 (EDT) Received: from d12relay02.de.ibm.com (d12relay02.de.ibm.com [9.165.215.23]) by d12lmsgate-2.de.ibm.com (8.12.9/8.12.8) with ESMTP id h3I7aiUm061338; Fri, 18 Apr 2003 09:36:44 +0200 Received: from d10ml001.telaviv.ibm.com (d10ml001.telaviv.ibm.com [9.148.216.55]) by d12relay02.de.ibm.com (8.12.8/NCO/VER6.5) with ESMTP id h3I7aeHZ178152; Fri, 18 Apr 2003 09:36:43 +0200 In-Reply-To: <200304172248.40859.mingz@ele.uri.edu> To: mingz@ele.uri.edu Cc: David Woolf , ips@ece.cmu.edu, owner-ips@ece.cmu.edu Subject: Re: iSCSI read/write cost difference MIME-Version: 1.0 X-Mailer: Lotus Notes Release 6.0.1 February 07, 2003 From: "Julian Satran" Message-ID: Date: Fri, 18 Apr 2003 10:36:38 +0300 X-MIMETrack: Serialize by Router on D10ML001/10/M/IBM(Release 5.0.9a |January 7, 2002) at 18/04/2003 10:36:44, Serialize complete at 18/04/2003 10:36:44 Content-Type: multipart/alternative; boundary="=_alternative 00281FB4C2256D0C_=" Sender: owner-ips@ece.cmu.edu Precedence: bulk This is a multipart message in MIME format. --=_alternative 00281FB4C2256D0C_= Content-Type: text/plain; charset="US-ASCII" For UNH code you have to address UNH. I suspect they do not incorporate the status in the last data and that slows write. For large data sizes on write you may also be slowed down by the target not issuing R2Ts in time. The jump should be visible when you start needing R2T. A high performance target will issue the required R2T as soon as it has the command while a test target migh be more "relaxed". Some tcp traces may help you - but please don't send them to me - that was a suggestion for you to analyze. TCP traces contain timestamps. Julo mingz Sent by: owner-ips@ece.cmu.edu 18/04/03 05:48 Please respond to mingz@ele.uri.edu To David Woolf cc ips@ece.cmu.edu Subject Re: iSCSI read/write cost difference yes, i test these three situations and get the results as follows 1) for all InitialR2T =Y/N, ImmediateData = Y/N, the read results are all same. 2) initialr2t has no influence on write 3) immediatedata has influence on write. IOPS: (scsi ram disk) 2k 4k 8k 16k 32k read 1975 1781 1557 1081 767 write 2877 2225 1588 812 533 (immediatedata =yes) write 1971 1527 1199 880 566 (immediatedata =no) when immediaredata=yes, for small write, data go with the command, so should be faster, but why with large data size, like 16k and 32k, the write is slower? and why write is always slower than read with large data size? btw, can u answer this question, 'Also, does the target piggyback SCSI Response in the Final Datain pdu?' since this code is from your unh iol, hehe. thx. On Thursday 17 April 2003 14:19, you wrote: > Hello, > Have you done these experiments with InitialR2T=Yes and Immediatedata=No, > InitialR2T=Yes and Immediatedata=Yes, and InitialR2T=No and > ImmediateData=Yes? I'd be interested to see how the results change. > > thanks, > > David Woolf > ************************************************ > University of New Hampshire Interoperability Lab > iSCSI and Fibre Channel Consortiums > Durham, NH 03824 > (603) 862 0701 > ************************************************ > > On Thu, 17 Apr 2003, mingz wrote: > > I recently did some experiments on iSCSI benchmarking. I used the UNH > > iSCSi ref18_03 implementation and use a ram based scsi device for iscsi > > target use. so there is no any disk overhead. > > > > I used a linux kernel module to send fixed size read/write requests to > > iscsi initiator, which eventually will be filled by iscsi target. i use > > interl pro1000 gigabit nic and intel 470 gigabit switch. and both > > initiator and target use same type piii 866 pc with 1g ram. > > > > now the IOPS result shows that > > > > 2k 4k 8k 16k 32k > > read 1975 1781 1557 1081 767 > > write 2877 2225 1588 812 533 > > > > for small reuqests, read is slower than write, while for large requests, > > read is faster than write. i redo the experiments on another enviroment, > > also get similar results. > > > > can anybody explain why this happens? > > > > thanks a lot. > > > > > > ming -- -------------------------------------------------- | Ming Zhang, PhD. Student | Dept. of Electrical & Computer Engineering | College of Engineering | University of Rhode Island | Kingston RI. 02881 | e-mail: mingz@ele.uri.edu | Tel. (401) 874-2293 Fax (401) 782-6422 | http://www.ele.uri.edu/~mingz -------------------------------------------------- --=_alternative 00281FB4C2256D0C_= Content-Type: text/html; charset="US-ASCII"
For UNH code you have to address UNH. I suspect they do not incorporate the status in the last data and that slows write.
For large data sizes on write you may also be slowed down by the target not issuing R2Ts in time. The jump should be visible when you start needing R2T.

A high performance target will issue the required R2T as soon as it has the command while a test target migh be more "relaxed".

Some tcp traces may help you - but please don't send them to me - that was a suggestion for you to analyze.
TCP traces contain timestamps.

Julo


mingz <mingz@ele.uri.edu>
Sent by: owner-ips@ece.cmu.edu

18/04/03 05:48
Please respond to
mingz@ele.uri.edu
To
David Woolf <djwoolf@io.iol.unh.edu>
cc
ips@ece.cmu.edu
Subject
Re: iSCSI read/write cost difference





yes, i test these three situations and get the results as follows

1) for all InitialR2T =Y/N, ImmediateData = Y/N, the read results are all
same.
2) initialr2t has no influence on write
3) immediatedata has influence on write.

IOPS: (scsi ram disk)
                                  2k                                  4k                                  8k                                  16k                                  32k
read                                  1975                 1781                 1557                 1081                 767
write                  2877                 2225                 1588                 812                                  533 (immediatedata =yes)
write                 1971                 1527                 1199                 880                                  566 (immediatedata =no)

when immediaredata=yes,  for small write, data go with the command, so should
be faster, but why with large data size, like 16k and 32k, the write is
slower? and why write is always slower than read with large data size?

btw, can u answer this question,  'Also, does the target piggyback SCSI
Response in the Final Datain pdu?' since this code is from your unh iol,
hehe. thx.


On Thursday 17 April 2003 14:19, you wrote:
> Hello,
> Have you done these experiments with InitialR2T=Yes and Immediatedata=No,
> InitialR2T=Yes and Immediatedata=Yes, and InitialR2T=No and
> ImmediateData=Yes? I'd be interested to see how the results change.
>
> thanks,
>
> David Woolf
> ************************************************
> University of New Hampshire Interoperability Lab
> iSCSI and Fibre Channel Consortiums
> Durham, NH 03824
> (603) 862 0701
> ************************************************
>
> On Thu, 17 Apr 2003, mingz wrote:
> > I recently did some experiments on iSCSI benchmarking. I used the UNH
> > iSCSi ref18_03 implementation and use a ram based scsi device for iscsi
> > target use. so there is no any disk overhead.
> >
> > I used a linux kernel module to send fixed size read/write requests to
> > iscsi initiator, which eventually will be filled by iscsi target. i use
> > interl pro1000 gigabit nic and intel 470 gigabit switch. and both
> > initiator and target use same type piii 866 pc with 1g ram.
> >
> > now the IOPS result shows that
> >
> >                                   2k                                  4k                                  8k                                  16k                                  32k
> > read                                  1975                 1781                 1557                 1081                 767
> > write                  2877                 2225                 1588                 812                                  533
> >
> > for small reuqests, read is slower than write, while for large requests,
> > read is faster than write. i redo the experiments on another enviroment,
> > also get similar results.
> >
> > can anybody explain why this happens?
> >
> > thanks a lot.
> >
> >
> > ming

--
--------------------------------------------------
| Ming Zhang, PhD. Student                                    
| Dept. of Electrical & Computer Engineering      
| College of Engineering                          
| University of Rhode Island                      
| Kingston RI. 02881                                  
| e-mail: mingz@ele.uri.edu                        
| Tel. (401) 874-2293 Fax  (401) 782-6422          
| http://www.ele.uri.edu/~mingz                    
--------------------------------------------------
--=_alternative 00281FB4C2256D0C_=--