Subject: RE: [Ips] [IPS]: Considerations for Asymmetric iSCSI Implementat i ons Draft Interest?

From: [email-masked]
Date: Mon, 17 Nov 2003 15:59:27 -0700

The asymmetric model looks simple at first glance, but when one probes, it appears to require significant extra complexity compared to the current model where command, data and status for a command are carried on a single connection. 

For example, when unsolicited data is sent on a data connection, it may arrive before the command so it may have to be held until the command arrives. More importantly, the SCSI protocol assumes an underlying ordered delivery to the SCSI layer between data and status. If data and status travel in different channels, then there would need to be some interlock between the cards to ensure that the status was not delivered to SCSI before the associated data.

I expect the need for the interlock and the need to coordinate between the ends of the link which link the data was to be sent on would require changes to the iSCSI headers.

The communication required between adapters (or adapter to driver to adapter) is also a disadvantage.

Pat

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf Of Mark S.
Edwards
Sent: Monday, November 17, 2003 2:24 PM
To: [email-masked]
Subject: RE: [Ips] [IPS]: Considerations for Asymmetric iSCSI
Implementati ons Draft Interest?

I can't access the doc you reference, but I believe that it is very old and 
part of the idea bouncing during the early stages of iSCSI.

I've been trying to remember the various discussions at the time as to the 
pro's and con's of the different models, but I do remember that I was quite 
opposed to this one largely due to the pain of what happens if one or other 
of the control or data plane fabrics breaks.  Remember, any IETF based 
solution MUST be scalable to the Internet.  Stuff that works when two hosts 
are side by side in a data centre has a habit of biting back when they are 
on opposite sides of the World.

The earliest reference I have in my mail store goes back April 2001 and the 
Last Call discussions on the requirements document.  There were a couple of 
emails asking Marjorie and team to remove the connection model discussions.

This was particularly apt from Bob Snively and also tells you where to 
trawl for further enlightenment on the original discussions, presumably 
there may be some IPS meeting notes on this in there, too.

>6)  ALTERNATE CONNECTION BINDING
>
>         The section in 2.4 discussing an alternate mechanism for
>         connection binding merely serves to weaken the stand
>         in favor of the selected binding relationship.  The following
>         text should be deleted:
>
>    "An alternate approach that was extensively discussed involved
>    sending all commands on a single connection and the associated data
>    and status on a different connection (asymetric approach). In this
>    scheme, the transport ensures the commands arrive in order. The
>    protocol on the data and status connections is simpler, perhaps
>    lending itself to a simpler realization in hardware.  One
>    disadvantage of this approach is that the recovery procedure is
>    different if a command connection fails vs. a data connection. Some
>    argued that this approach would require greater inter-processor
>    communication when connections are spread across processors.
>    The reader may reference the mail archives of the IPS mailing list
>    between June and September of 2000 for extensive discussions on
>    symmetric vs asymmetric connection models."

regards,

Mark.

At 08:01 PM 11/17/2003, Nicholas A. Bellinger wrote:
>Pat,
>
>http://www.haifa.il.ibm.com/Workshops/Storage2002/papers/iscsiDesign.pdf
>
>The following paper from Kalman Meth decribes them on page 12:
>
>Asymmetric:
>Single Control Channel / iSCSI Connection.
>
>Multiple Data Channels / iSCSI Connection.
>
>Control Channel used to transfer commands, status, and task management.
>
>Symmetric:
>All Channels / iSCSI Connections identical.
>
>Send Data and Status over same Channel / iSCSI Connection as
>corresponding command.
>
>
>
>I am not exactly sure why the single control channel restriction was
>imposed in the asymmetric case, but imagine its no longer an issue to
>have multiple control channels.  Perhaps this was before the advent of
>session wide CmdSN ordering?
>
>Some of the more interesting questions:
>
>Connection recovery case, which connection an inflight command is
>alligent to when the control and/or data connection fails.
>
>Requirements for falling back when an session runs out of control and/or
>data channels.  Or is it just easier to do session reinstatement into
>an symmetric session.
>
>How strict are the requirements for data on the control channel and
>control primatives in the data channel?  Immediate DataOut comes to mind
>in the former case, and Phase Collapse for DataIn in the latter.
>
>Thanks!
>
>On Mon, 2003-11-17 at 11:17, [email-masked] wrote:
> > I'm not clear on what you mean by "asymetric".
> >
> > -----Original Message-----
> > From: Nicholas A. Bellinger [mailto:[email-masked]]
> > Sent: Monday, November 17, 2003 10:20 AM
> > To: [email-masked]; [email-masked]
> > Subject: [Ips] [IPS]: Considerations for Asymmetric iSCSI
> > Implementations Draft Interest?
> >
> >
> > Greetings All,
> >
> >       I am inquiring about interest in seeing a draft and possibly RFC
> > regarding Considerations for Asymmetric iSCSI Implementations.  I recall
> > seeing some discussion on this topic back in 2000, and aside from a few
> > other mentions of the model in various design documents I have not seen
> > any followups.  I am curious if the model had all but been abandoned for
> > the symmetric model in traditional and iSER iSCSI, or if there is still
> > room in the protocol for asymmetric operations?
> >
> > I have been thinking about this topic for a while and specifically its
> > affects on ErrorRecoveryLevel=2 implemenatations and wanted to get some
> > feedback from list members if it would be worthwhile for someone (mabye
> > even me!) to put the thoughts down for other interested parties to
> > consider.
> >
> > Aside from the expected responses of complexity issues for simple
> > implementations,  I do not see a terrible amount of extra functionality
> > required for ErrorRecoveryLevel=1 and above implementations to operate
> > in an Asymmetric model, and literally no changes to RFC 3347.  As to the
> > benefits of running in a Asymmetric model I can only speculate,
> > although I suppose most are related performance in multiple connection
> > ErrorRecoveryLevel=1 and above software implementations.
> >
> > Any feedback is welcome!
> >
> > --
> > Nicholas A. Bellinger <[email-masked]>
> >
> >
> > _______________________________________________
> > Ips mailing list
> > [email-masked]
> > https://www1.ietf.org/mailman/listinfo/ips
>--
>Nicholas A. Bellinger <[email-masked]>
>
>
>_______________________________________________
>Ips mailing list
>[email-masked]
>https://www1.ietf.org/mailman/listinfo/ips

_______________________________________________
Ips mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ips

_______________________________________________
Ips mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ips

---

From: "Nicholas A. Bellinger" <[email-masked]>
Date: 17 Nov 2003 15:23:01 -0800

Pat,

> The communication required between adapters (or adapter to driver to adapter) is also a disadvantage.

I take the stance that handling and not discarding any Unsolicited
DataOUT for an unknown Initiator Task Tag is a bad idea.  In the
Unsolicited DataOut case, when the DataOUT request is only sent (but can
be built) on the data channel only after the control channel has
successfuly sent the iSCSI Scsi Cmnd.

Of course then the problem arises of handling out of order CmdSNs for
multiple control channels, and Unsolicited DataOut still arriving before
(although this time a Initiator Task Tag would be present) the CmdSN of
the iSCSI SCSI Cmnd is executed by the Storage Transport Layer.

Thanks for the observations!