Subject: RE: [Ips] : Considerations for Asymmetric iSCSI Implementati ons Draft Interest?

From: John Hufferd <[email-masked]>
Date: Mon, 17 Nov 2003 13:26:10 -0800

I am not sure why we would ever want to do this.  How does this improve 
the performance over normal Multiple Connections per Session?  And what is 
the projected improvement (%)?
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/System Group, San Jose CA
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688
Alt Office: (408) 997-6136, Cell: (408) 499-9702
Internet Address: [email-masked]

"Nicholas A. Bellinger" <[email-masked]>
Sent by: [email-masked]
11/17/2003 12:01 PM

        To:     [email-masked]
        cc:     [email-masked], Julian Satran <[email-masked]>
        Subject:        RE: [Ips] [IPS]: Considerations for Asymmetric iSCSI Implementati ons 
Draft Interest?

Pat,

http://www.haifa.il.ibm.com/Workshops/Storage2002/papers/iscsiDesign.pdf

The following paper from Kalman Meth decribes them on page 12:

Asymmetric:
Single Control Channel / iSCSI Connection.

Multiple Data Channels / iSCSI Connection.

Control Channel used to transfer commands, status, and task management.

Symmetric:
All Channels / iSCSI Connections identical.

Send Data and Status over same Channel / iSCSI Connection as 
corresponding command.

I am not exactly sure why the single control channel restriction was
imposed in the asymmetric case, but imagine its no longer an issue to
have multiple control channels.  Perhaps this was before the advent of
session wide CmdSN ordering?

Some of the more interesting questions:

Connection recovery case, which connection an inflight command is
alligent to when the control and/or data connection fails.

Requirements for falling back when an session runs out of control and/or
data channels.  Or is it just easier to do session reinstatement into
an symmetric session.

How strict are the requirements for data on the control channel and
control primatives in the data channel?  Immediate DataOut comes to mind
in the former case, and Phase Collapse for DataIn in the latter.

Thanks!

On Mon, 2003-11-17 at 11:17, [email-masked] wrote:
> I'm not clear on what you mean by "asymetric".
> 
> -----Original Message-----
> From: Nicholas A. Bellinger [mailto:[email-masked]]
> Sent: Monday, November 17, 2003 10:20 AM
> To: [email-masked]; [email-masked]
> Subject: [Ips] [IPS]: Considerations for Asymmetric iSCSI
> Implementations Draft Interest?
> 
> 
> Greetings All,
> 
>                I am inquiring about interest in seeing a draft and 
possibly RFC
> regarding Considerations for Asymmetric iSCSI Implementations.  I recall
> seeing some discussion on this topic back in 2000, and aside from a few
> other mentions of the model in various design documents I have not seen
> any followups.  I am curious if the model had all but been abandoned for
> the symmetric model in traditional and iSER iSCSI, or if there is still
> room in the protocol for asymmetric operations?
> 
> I have been thinking about this topic for a while and specifically its
> affects on ErrorRecoveryLevel=2 implemenatations and wanted to get some
> feedback from list members if it would be worthwhile for someone (mabye
> even me!) to put the thoughts down for other interested parties to
> consider.
> 
> Aside from the expected responses of complexity issues for simple
> implementations,  I do not see a terrible amount of extra functionality
> required for ErrorRecoveryLevel=1 and above implementations to operate
> in an Asymmetric model, and literally no changes to RFC 3347.  As to the
> benefits of running in a Asymmetric model I can only speculate, 
> although I suppose most are related performance in multiple connection
> ErrorRecoveryLevel=1 and above software implementations.
> 
> Any feedback is welcome! 
> 
> -- 
> Nicholas A. Bellinger <[email-masked]>
> 
> 
> _______________________________________________
> Ips mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/ips
-- 
Nicholas A. Bellinger <[email-masked]>

_______________________________________________
Ips mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ips

---

From: "Nicholas A. Bellinger" <[email-masked]>
Date: 17 Nov 2003 14:42:46 -0800

John,

I can think of a few reasons while trying not to go too deep into
implementation specific details.  These have to do with my idea of
achieving maximum parallelism,  which probably is not the same as other
peoples. Again these are purely speculation. (speculation == more
threads doing simpler operations will be faster as connection count
increases)

Non Immedate Unsolicited DataOUT Case, assume FirstBurstLength >= SCSI
Request size.

Asymmetric:

2 Control Channels and 2 Data Channels:
CID 0 and 1: Control Channels 0 and 1
CID 2 and 3: Data Channels 0 and 1

Only operation is checking CmdSN window to send commands: 

0) TX threads for CID 0 and 1 checking CmdSN window to send non
immediate commands.
1) CmdSN window opens for CID 0, iSCSI command becomes alligent to CID
0.
2) iSCSI Scsi Cmnd Opcode is built on CID 0 and unsolicited DataOUT
request gets added to CID 2 from CID 0.
3) A and B can happen in any order:
3A) iSCSI Scsi Cmnd Opcode gets sent on CID 0.
3B) iSCSI Unsolicited DataOUT request gets queued and Unsolicited
DataOUT is built on CID 2.
4) iSCSI Unsolicited DataOUT is sent on CID 2 after iSCSI Scsi Cmnd
Opcode is sent on CID 0.
5) TX thread for CID 1 detects open CmdSN window and the process starts
   over again with CID 1 and CID 3.

Problem: 
Latency issue of iSCSI Unsolicited DataOUT arriving before iSCSI Scsi
Cmnd Opcode.

Symmetric:  

2 iSCSI Connections:
CID 0 and 1

0) TX threads for CID 0 and 1 checking CmdSN window to send non
immediate commands.
1) CmdSN window opens for CID 0, iSCSI command becomes alligent to CID
0.
2) iSCSI Scsi Cmnd Opcode is built on CID 0.
3) iSCSI Scsi Cmnd Opcode is sent on CID 0, and adds Unsolicited DataOUT
request to CID 0 queue.
4) Either A or B may happen next (matter of implementation)
4A) CID 0 builds Unsolicited DataOUT.
4B) CID 0 detects CmdSN window is open and sends more non-immediate
commands, only until after CmdSN window closes does it send Unsolicited
DataOUT.
5) TX thread for CID 1 detects CmdSN window and the process starts over
again with CID 1.

Problem:
CID 0 having to build and send DataOUT while it could be sending another
iSCSI Scsi Cmnd Opcode while the CmdSN window is open.

This is just one scenario off the top of my head and could be written
off as a matter of implementation,  but the asymmetric goal is to
achieve maximum parallelism by A) threads doing smaller specialized
jobs, and B) Control threads queuing up more iSCSI Scsi Cmnds faster to
Data Threads checking a single queue.  As far as performance percentage
improvement I cannot begin to dream up numbers until an asymmetric
prototype is complete, and very well could only show improvements in
specific configurations,  but it is anyones guess.

Thanks!

On Mon, 2003-11-17 at 13:26, John Hufferd wrote:
> I am not sure why we would ever want to do this.  How does this
> improve the performance over normal Multiple Connections per Session?
> And what is the projected improvement (%)?
> .
> .
> John L. Hufferd
>         cc:      
> [email-masked], Julian
> Satran
> <[email-masked]>
>         Subject:      
> RE: [Ips] [IPS]:
> Considerations for
> Asymmetric iSCSI
> Implementati        ons
> Draft Interest?
> 
> 
> Pat,
>