Subject: Re: FW: [Ips] Recent comments about FCoE and iSCSI

From: Julian Satran <[email-masked]>
Date: Wed, 25 Apr 2007 08:09:43 -0400

"John Hufferd" <[email-masked]> wrote on 25/04/2007 02:45:51:

> Julian, 
> To be sure you understand our position; Brocade is pushing iSCSI as an
> outreach protocol from the Data Center. We also believe iSCSI is very
> useful for installations that do not have a Fibre Channel
> infrastructure, and in that case we will be able to sell them our new
> iSCSI and TOE offload HBAs. 
> 
> When I say iSCSI is an outreach protocol, this is a statement that iSCSI
> is very important to connect "stranded" servers to the Fibre Channel
> Fabric.  That is, we sell iSCSI-to-FC Gateway devices which will permit
> iSCSI Servers (software or HBA iSCSI initiators) to connect to the
> Enterprise "Bet Your Business" FC Storage.  This of course also applies
> to Desktops and Laptop systems, and systems at distance.
>

You make it sound like:
most of the servers in the world have their storage on the network - and 
that is not the case
FCP is basically better performing than iSCSI - and that is not true 
either
Gatewaying is expensive - and it is perhaps so but only if you are 
completely relying on FCP storage (and there are plenty of good iSCSI 
vendors of storage)and pushing the price on the servers is not cheap 
either - at least not for the server buyer

> Now with that positioning, it is important to understand the limitation
> to this strategy.  The primary problem is that iSCSI to FC Bridging
> (Gatewaying) is relatively expensive (compared to simple FC
> connections).  Though we have some of the best priced Gateways on the
> market, it is not cost feasible to replace all the server connectivity
> to FC storage with iSCSI for the hundred to thousands of servers in the
> Data Center.  And since, if there is to be a consolidated Network
> connection to the servers in the Data Center, there must be an
> evolutionary replacement of Server Connections to Storage.  That means
> there must be a bridge/Gateway approach.  And as I mentioned before,
> there is just too much cost in the iSCSI to FC Gateway.
> 
> The issue is the server requirement to have a single connection type to
> handle cluster messaging, general messaging, and storage.  iSCSI is
> clearly an option for the storage, however, the gateway costs are too
> high for iSCSI to be used as the "normal" server connect into a FC based
> Fabric. That is true for the current 1GE; and for 10GE the cost is just
> out of sight.  The reason for this is the requirement for TCP/IP
> termination and re-initiation with FC at the Gateway.
> 
> Now with respect to FC over Ethernet the important thing to understand
> is that it is not Ethernet as we have known it up to know.  The Ethernet
> we are talking about is a type of Ethernet that can only be deployed in
> a constrained environment such as a Data Center.  This form of Ethernet
> is called DCE (Data Center Ethernet) or CEE (Convergence Enhanced
> Ethernet).  This form of Ethernet is a Loss-less type Ethernet, with
> multi-priority and Flow Control.  This is NOT an Internet or Intranet
> type of Ethernet.
> 
> FCoE is all about using the DCE (CEE) to carry FC frames.  The rest of
> the Host and storage stack remain the same, the functions and features
> of the switches also remain the same and add the capability to provide
> Cluster Message Switching which has latency close to InfiniBand speeds.
> 
> 
> Because the FC frames are transported to the switches intact via a DCE
> frame, the Bridging, if you want to call it that, is virtually non
> existent.  Hence you can deliver the FC frames to FC devices, or send FC
> frames to DCE FCoE devices, just like one would do if it was all FC. And
> all this is done while performing Cluster message switching and general
> message trucking to the IP outfacing network. 
> 

The rosy future of the yet to appear DCE/CEE and a layer 2 only world.
First you have some terms confused:

Bridging is the term commonly used for Layer-2 switching and routing is 
therm used for layer-3 (switching).

Bridging has some advantages (less management) that have created a 
movement towards an enterprise wide
LAN. But this has a long way to go and will require significant equipment 
and protocol changes.
Even its proponents do not call for transportless networks, lossless 
networks etc.
The second trouble with your argument is that there are no known large 
scale networking technologies that
really work at full speed (high speed) and are lossless (flow-controlled) 
and errorless like FCoE assumes.
The TCP/IP has solved this issue for every generation using the proven 
end-to-end principle (and is doing so now).
And it is not by chance so and that is why all networking applications are 
built above layer-3 and not dropping layer-3 (like FCoE) does.

Although I can understand the DCE arguments as a management statement I 
would prefer like any rational engineer, to base my building blocks on 
structures that are proven and long lasting. And those are still the 
end-to-end TCP/IP that can accommodate even your FCP addicts. The IPS TWG 
has developed the iFCP that does exactly what FCoE claims to do an a 
better base.

> This means an evolutionary process is possible to the solution of
> getting a single Fabric connection for all networks connected to a
> server, further, the process has very low interconnection cost on the
> Data Center Fabric. And it maintains all the FC Fabric Services, and all
> the same Storage Management processes. 
> 
> By the way, this is primarily a Server driven value statement, there
> seems to be little value in having FCoE on the storage controller.
> Therefore FC storage controllers (and FICON) will be the very last
> things that connect using FCoE and that evolution will take at least a
> decade or more.
> 

It is server cost statement. It costs nothing to connect a modern server 
to ethernet it will cost a bundle to connect to FCoE and it will force 
users in short lived bad solutions.

> We see value in offering switches and Directors that can support DCE
> switching, FC switching as well as iSCSI interconnect, and the
> "Trunking" of general messaging to the Outfacing IP network.  That said;
> we do not see FCoE going beyond the constraints of the Data Center.

Data Centers now grow to tens of thousands of nodes. There is no layer-2 
technology for errorless/lossless operation at this scale and there is no 
good reason to pursue one. The only possible reason (good reason) is the 
bridging infrastructure but that infrastructure has a completely different 
rationale than the flowcontrol.

> This issue and message is quite different from the issues and messages
> we struggled with when we started iSCSI.  There is a consortium of folks
> both working on the DCE (CEE) and the FCoE.  Without the DCE the FCoE
> will not happen. 
> 
> None of the above cancels out the value of iSCSI in numerous
> environments.
> 
> 

iSCSI is good for all environments. Business consideration (and some 
politics) keep it form "exploding" and large storage vendors are 
completely indifferent to the network connection they are using. 
You and I have also slightly different views of DCE. I expect DCE (that 
still has a way to go) to improve the QoS in the data-center (and for 
storage too). You expect it to bring the loss rates down to the levels 
that FCP assumes (FCP has no transport layer) and that is probably a pipe 
dream. Todays transport solution for loss mitigation are far more cost 
effective - and that's why iFCP is a better proposition as a transition 
technology than FCoE and iSCSI with gateways is propably better in the 
long run.

> 
> .
> .
> .
> John L Hufferd
> Sr. Executive Director of Technology
> [email-masked]
> Office Phone: (408) 333-5244; eFAX: (408) 904-4688
> Alt Office Phone: (408) 997-6136; Cell: (408) 627-9606
> 
> -----Original Message-----
> From: John Hufferd 
> Sent: Tuesday, April 24, 2007 12:57 PM
> To: [email-masked]'
> Subject: Re: [Ips] Recent comments about FCoE and iSCSI
> 
> Julian,
> I think you are wrong on this one.  The arguments are quite different
> then the ones we had in pre iSCSI days.  (By the way I missed you on
> today's Renato meeting/conf call where Brocade took the IBM technology
> group through FCoE as it is being placed in our plans). 
> I will send you more info when I get to my computer.  But you probably
> were sent the Brocade charts.  Please review them and I will follow up
> with more information.
> This does NOT replace iSCSI it applies only to a DataCenter envuornment
> with lossless DCE ethernet.
> --------------------------
> John L. Hufferd
> Sr. Ex. Director of Technology
> Brocade Communications Systems, Inc.
> Phone: (408) 333-5244
> Mobile: (408) 627-9606
> eMail: [email-masked]
> (Sent from my BlackBerry Wireless)
> 
> 
> ----- Original Message -----
> From: Julian Satran <[email-masked]>
> To: [email-masked] <[email-masked]>
> Sent: Tue Apr 24 12:10:29 2007
> Subject: [Ips] Recent comments about FCoE and iSCSI
> 
> 
> Dear All, 
> 
> The trade press is lately full with comments about the latest and
> greatest reincarnation of Fiber Channel over ethernet. 
> It made me try and summarize all the long and hot debates that preceded
> the advent of iSCSI. 
> Although FCoE proponents make it look like no debate preceded iSCSI that
> was not so - FCoE was considered even then and was dropped as a dumb
> idea. 
> 
> Here is a summary (as afar as I can remember) of the main arguments.
> They are not bad arguments even in retrospect and technically FCoE
> doesn't look better than it did then. 
> 
> Feel free to use this material in a nay form. I expect this group to
> seriously  expand my arguments and make them public - in personal or
> collective form. 
> 
> And do not forget - it is a technical dispute - although we all must
> have some doubts about the way it is pursued. 
> 
> Regards, 
> Julo 
> 
> --------------------------------------------------------------------- 
> 
> What a piece of nostalgia :-) 
> 
> Around 1997 when a team at IBM Research (Haifa and Almaden) started
> looking at connecting storage to servers using the "regular network"
> (the ubiquitous LAN) we considered many alternatives (another team even
> had a look at ATM - still a computer network candidate at the time). I
> won't get you over all of our rationale (and we went over some of them
> again at the end of 1999 with a team from CISCO before we convened the
> first IETF BOF in 2000 at Adelaide that resulted in iSCSI and all the
> rest) but some of the reasons we choose to drop Fiber Channel over raw
> Ethernet where multiple: 
> 
> 
> *   Fiber Channel Protocol (SCSI over Fiber Channel Link) is
> "mildly" effective because: 
> 
>    *   it implements endpoints in a dedicated engine (Offload) 
>    *   it has no transport layer (recovery is done at the
> application layer under the assumption that the error rate will be very
> low) 
>    *   the network is limited in physical span and logical span
> (number of switches) 
>    *   flow-control/congestion control is achieved with a
> mechanism adequate for a limited span network (credits). The packet loss
> rate is almost nil and that allows FCP to avoid using a transport
> (end-to-end) layer
> 
>    *   FCP she switches are simple (addresses are local and the
> memory requirements cam be limited through the credit mechanism) 
>    *   However FCP endpoints are inherently costlier than
> simple NICs - the cost argument (initiators are more expensive) 
>    *   The credit mechanisms is highly unstable for large
> networks (check switch vendors planning docs for the network diameter
> limits) - the scaling argument 
>    *   The assumption of low losses due to errors might
> radically change when moving from 1 to 10 Gb/s - the scaling argument 
>    *   Ethernet has no credit mechanism and any mechanism with
> a similar effect increases the end point cost. Building a transport
> layer in the protocol stack has always been the preferred choice of the
> networking community - the community argument 
>    *   The "performance penalty" of a complete protocol stack
> has always been overstated (and overrated). Advances in protocol stack
> implementation and finer tuning of the congestion control mechanisms
> make conventional TCP/IP performing well even at 10 Gb/s and over.
> Moreover the multicore processors that become dominant on the computing
> scene have enough compute cycles available to make any "offloading"
> possible as a mere code restructuring exercise (see the stack reports
> from Intel, IBM etc.) 
>    *   Building on a complete stack makes available a wealth of
> operational and management mechanisms built over the years by the
> networking community (routing, provisioning, security, service location
> etc.) - the community argument 
>    *   Higher level storage access over an IP network is widely
> available and having both block and file served over the same connection
> with the same support and management structure is compelling - the
> community argument 
>    *   Highly efficient networks are easy to build over IP with
> optimal (shortest path) routing while Layer 2 networks use bridging and
> are limited by the logical tree structure that bridges must follow. The
> effort to combine routers and bridges (rbridges) is promising to change
> that but it will take some time to finalize (and we don't know exactly
> how it will operate). Untill then the scale of Layer 2 network is going
> to seriously limited - the scaling argument
> 
> 
> 
>       As a side argument - a performance comparison made in
> 1998 showed SCSI over TCP (a predecessor of the later iSCSI) to perform
> better than FCP at 1Gbs for block sizes typical for OLTP (4-8KB). That
> was what convinced us to take the path that lead to iSCSI - and we used
> plain vanilla x86 servers with plain-vanilla NICs and Linux (with
> similar measurements conducted on Windows). 
>    The networking and storage community acknowledged those
> arguments and developed iSCSI and the companion protocols for service
> discovery, boot etc. 
> 
>    The community also acknowledged the need to support existing
> infrastructure and extend it in a reasonable fashion and developed 2
> protocols iFCP (to support hosts with FCP drivers and IP connections to
> connect to storage by a simple conversion from FCP to TCP packets) FCPIP
> to extend the reach of FCP through IP (connects FCP islands through TCP
> links). Both have been 
>    implemented and their foundation is solid. 
> 
>    The current attempt of developing a "new-age" FCP over an
> Ethernet link is going against most of the arguments that have given us
> iSCSI etc. 
> 
>    It ignores the networking layering practice, build an
> application protocol directly above a link and thus limits scaling,
> mandates elements at the link layer and application layer that make
> applications more expensive and leaves aside the whole "ecosystem" that
> accompanies TCP/IP (and not Ethernet). 
> 
>    In some related effort (and at a point also when developing
> iSCSI) we considered also moving away from SCSI (like some "no
> standardized" but popular in some circles software did - e.g., NBP) but
> decided against. SCSI is a mature and well understood access
> architecture for block storage and is implemented by many device
> vendors. Moving away from it would not have been justified at the time. 
> 
_______________________________________________
Ips mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ips

---

From: "Eddy Quicksall" <[email-masked]>
Date: Wed, 25 Apr 2007 10:07:21 -0400

Basically, it is sending FC frames over Ethernet. This localizes the traffic unless you route based on MAC addresses. So you send 2146 bytes of FC frame plus 18 bytes of Ethernet overhead as FCoE "standard" packet. 18 bytes of Ethernet gets stripped and you have straight FC frame that can go through any FC network. Now you can have 10G Ethernet pipes into existing FC SANs. Limited market potential as far as I can see. The key argument is it much easier to implement than iSCSI and also has less overhead and uses all the benefits of FC. End to End credits are simulated using PAUSE command on Ethernet and MAC addresses are mapped into WWNs.

Biggest knock is that it will not route on the "global" scale like TCP/IP would.

Eddy

 ----- Original Message ----- 

  From: Julian Satran 
  To: John Hufferd 
  Cc: [email-masked] 
  Sent: Wednesday, April 25, 2007 8:09 AM
  Subject: Re: FW: [Ips] Recent comments about FCoE and iSCSI

  > to Desktops and Laptop systems, and systems at distance.

  You make it sound like: 
    1.. most of the servers in the world have their storage on the network - and that is not the case 
    2.. FCP is basically better performing than iSCSI - and that is not true either 
    3.. Gatewaying is expensive - and it is perhaps so but only if you are completely relying on FCP storage (and there are plenty of good iSCSI vendors of storage)and pushing the price on the servers is not cheap either - at least not for the server buyer

  Bridging is the term commonly used for Layer-2 switching and routing is therm used for layer-3 (switching). 

  Bridging has some advantages (less management) that have created a movement towards an enterprise wide 
  LAN. But this has a long way to go and will require significant equipment and protocol changes. 
  Even its proponents do not call for transportless networks, lossless networks etc. 
  The second trouble with your argument is that there are no known large scale networking technologies that 
  really work at full speed (high speed) and are lossless (flow-controlled) and errorless like FCoE assumes. 
  The TCP/IP has solved this issue for every generation using the proven end-to-end principle (and is doing so now). 
  And it is not by chance so and that is why all networking applications are built above layer-3 and not dropping layer-3 (like FCoE) does. 

  Although I can understand the DCE arguments as a management statement I would prefer like any rational engineer, to base my building blocks on structures that are proven and long lasting. And those are still the end-to-end TCP/IP that can accommodate even your FCP addicts. The IPS TWG has developed the iFCP that does exactly what FCoE claims to do an a better base. 

  > things that connect using FCoE and that evolution will take at least a

  It is server cost statement. It costs nothing to connect a modern server to ethernet it will cost a bundle to connect to FCoE and it will force users in short lived bad solutions. 

  > we do not see FCoE going beyond the constraints of the Data Center. 

  Data Centers now grow to tens of thousands of nodes. There is no layer-2 technology for errorless/lossless operation at this scale and there is no good reason to pursue one. The only possible reason (good reason) is the bridging infrastructure but that infrastructure has a completely different rationale than the flowcontrol. 

  > None of the above cancels out the value of iSCSI in numerous

  iSCSI is good for all environments. Business consideration (and some politics) keep it form "exploding" and large storage vendors are completely indifferent to the network connection they are using. 
  You and I have also slightly different views of DCE. I expect DCE (that still has a way to go) to improve the QoS in the data-center (and for storage too). You expect it to bring the loss rates down to the levels that FCP assumes (FCP has no transport layer) and that is probably a pipe dream. Todays transport solution for loss mitigation are far more cost effective - and that's why iFCP is a better proposition as a transition technology than FCoE and iSCSI with gateways is propably better in the long run. 

  > with lossless DCE ethernet.

------------------------------------------------------------------------------

---

From: Julian Satran <[email-masked]>
Date: Wed, 25 Apr 2007 11:04:43 -0400

Eddy,

That is oversimplified and ignore the drop rate assumption and error rate 
assumptions made in FCP(FCP has no transport layer). To get to it on a 
sizable network requires more than PAUSE.

Julo

"Eddy Quicksall" <[email-masked]> 
25/04/07 10:07

To
"John Hufferd" <[email-masked]>, Julian Satran/Haifa/IBM@IBMIL
cc
<[email-masked]>
Subject
Re: FW: [Ips] Recent comments about FCoE and iSCSI

Basically, it is sending FC frames over Ethernet. This localizes the 
traffic unless you route based on MAC addresses. So you send 2146 bytes of 
FC frame plus 18 bytes of Ethernet overhead as FCoE ?standard? packet. 18 
bytes of Ethernet gets stripped and you have straight FC frame that can go 
through any FC network. Now you can have 10G Ethernet pipes into existing 
FC SANs. Limited market potential as far as I can see. The key argument is 
it much easier to implement than iSCSI and also has less overhead and uses 
all the benefits of FC. End to End credits are simulated using PAUSE 
command on Ethernet and MAC addresses are mapped into WWNs.
Biggest knock is that it will not route on the ?global? scale like TCP/IP 
would.
Eddy
 ----- Original Message ----- 
From: Julian Satran 
To: John Hufferd 
Cc: [email-masked] 
Sent: Wednesday, April 25, 2007 8:09 AM
Subject: Re: FW: [Ips] Recent comments about FCoE and iSCSI

> to Desktops and Laptop systems, and systems at distance.

You make it sound like: 
1.      most of the servers in the world have their storage on the network 
- and that is not the case 
2.      FCP is basically better performing than iSCSI - and that is not 
true either 
3.      Gatewaying is expensive - and it is perhaps so but only if you are 
completely relying on FCP storage (and there are plenty of good iSCSI 
vendors of storage)and pushing the price on the servers is not cheap 
either - at least not for the server buyer

---

From: "Eddy Quicksall" <[email-masked]>
Date: Wed, 25 Apr 2007 11:08:56 -0400

I basically said that in the summery line by saying "it will not route on the "global" scale like TCP/IP would".
  ----- Original Message ----- 
  From: Julian Satran 
  To: Eddy Quicksall 
  Cc: [email-masked] ; John Hufferd 
  Sent: Wednesday, April 25, 2007 11:04 AM
  Subject: Re: FW: [Ips] Recent comments about FCoE and iSCSI

  Eddy, 

  That is oversimplified and ignore the drop rate assumption and error rate assumptions made in FCP(FCP has no transport layer). To get to it on a sizable network requires more than PAUSE. 

  Julo 

        "Eddy Quicksall" <[email-masked]> 
        25/04/07 10:07 
       To "John Hufferd" <[email-masked]>, Julian Satran/Haifa/IBM@IBMIL  
              cc <[email-masked]>  
              Subject Re: FW: [Ips] Recent comments about FCoE and iSCSI 

  Eddy 

   ----- Original Message ----- 

  From: Julian Satran 
  To: John Hufferd 
  Cc: [email-masked] 

  > to Desktops and Laptop systems, and systems at distance.

  You make it sound like: 
  1.        most of the servers in the world have their storage on the network - and that is not the case 
  2.        FCP is basically better performing than iSCSI - and that is not true either 
  3.        Gatewaying is expensive - and it is perhaps so but only if you are completely relying on FCP storage (and there are plenty of good iSCSI vendors of storage)and pushing the price on the servers is not cheap either - at least not for the server buyer

---

From: Julian Satran <[email-masked]>
Date: Wed, 25 Apr 2007 11:52:04 -0400

Sorry Eddy by sizable I meant even at the size of a modern data center. 
Julo

25/04/07 11:08

To
Julian Satran/Haifa/IBM@IBMIL
cc
[email-masked], John Hufferd <[email-masked]>
Subject
Re: FW: [Ips] Recent comments about FCoE and iSCSI

I basically said that in the summery line by saying "it will not route on 
the ?global? scale like TCP/IP would".
----- Original Message ----- 
From: Julian Satran 
To: Eddy Quicksall 
Cc: [email-masked] ; John Hufferd 
Sent: Wednesday, April 25, 2007 11:04 AM
Subject: Re: FW: [Ips] Recent comments about FCoE and iSCSI

Eddy, 

Julo 

To

 ----- Original Message ----- 
From: Julian Satran 
To: John Hufferd 
Cc: [email-masked] 
Sent: Wednesday, April 25, 2007 8:09 AM 
Subject: Re: FW: [Ips] Recent comments about FCoE and iSCSI 

> to Desktops and Laptop systems, and systems at distance.

You make it sound like: 
1.        most of the servers in the world have their storage on the 
network - and that is not the case 
2.        FCP is basically better performing than iSCSI - and that is not 
true either 
3.        Gatewaying is expensive - and it is perhaps so but only if you 
are completely relying on FCP storage (and there are plenty of good iSCSI 
vendors of storage)and pushing the price on the servers is not cheap 
either - at least not for the server buyer

---

From: "Eddy Quicksall" <[email-masked]>
Date: Wed, 25 Apr 2007 14:55:34 -0400

If it was not obvious in my wording, I meant to agree with you on your observations.

Eddy
  ----- Original Message ----- 
  From: Julian Satran 
  To: Eddy Quicksall 
  Cc: [email-masked] ; John Hufferd 
  Sent: Wednesday, April 25, 2007 11:52 AM
  Subject: Re: FW: [Ips] Recent comments about FCoE and iSCSI

  Sorry Eddy by sizable I meant even at the size of a modern data center. Julo 

        "Eddy Quicksall" <[email-masked]> 
        25/04/07 11:08 
       To Julian Satran/Haifa/IBM@IBMIL  
              cc [email-masked], John Hufferd <[email-masked]>  
              Subject Re: FW: [Ips] Recent comments about FCoE and iSCSI 

  ----- Original Message ----- 
  From: Julian Satran 
  To: Eddy Quicksall 
  Cc: [email-masked] ; John Hufferd 
  Sent: Wednesday, April 25, 2007 11:04 AM 
  Subject: Re: FW: [Ips] Recent comments about FCoE and iSCSI 

  Eddy, 

  Julo 

              

  Eddy 

   ----- Original Message ----- 

  From: Julian Satran 
  To: John Hufferd 
  Cc: [email-masked] 

  You make it sound like: 

  _______________________________________________
  Ips mailing list
  [email-masked]
  https://www1.ietf.org/mailman/listinfo/ips_______________________________________________
  Ips mailing list
  [email-masked]
  https://www1.ietf.org/mailman/listinfo/ips

---

From: "Eddy Quicksall" <[email-masked]>
Date: Wed, 25 Apr 2007 17:41:05 -0400

I notice that Cisco has a patent on Fibre Channel over Ethernet. Is FCoE something different?

       

  ----- Original Message ----- 
  From: Julian Satran 
  To: Eddy Quicksall 
  Cc: [email-masked] ; John Hufferd 
  Sent: Wednesday, April 25, 2007 11:04 AM 
  Subject: Re: FW: [Ips] Recent comments about FCoE and iSCSI 

  Eddy, 

  Julo 

              

  Eddy 

   ----- Original Message ----- 

  From: Julian Satran 
  To: John Hufferd 
  Cc: [email-masked] 

  You make it sound like:

---

From: "John Hufferd" <[email-masked]>
Date: Wed, 25 Apr 2007 17:42:20 -0700

Eddy,

The pause that is being analyzed as part of a Data Center Ethernet is
NOT the total stuff that is being handled.  The proposal that is being
worked on is called PPP (Per Priority Pause - which probably should be
called Per Class Pause).  The purpose of this is to use it as a flow
control alternative to a credit based approach which seems to be "not
going to happen" in the 802.1 committee.  This is a pause by class or
priority instead of the current Pause that effects every thing on the
link.  However, this is only a part of the total solution.  Another
committee is working on the Congestion issues, however, a number of
vendors, after looking at all the issues believe that we need more than
just the various congestion management approaches that are being
examined.  Therefore, many vendor think that adding a flow control such
as PPP can be part of the total solution to providing management of
congestion, at least for some of the higher priorities (classes).  

That being said let me say, that FCoE is not to be considered a
replacement for iSCSI.  It is just another tool for providing storage to
the application.  One approach does not fit all needs.  Fibre Channel is
important to the Enterprise Market.  They are not going to rip out and
replace their current FC infrastructure and replace it with an all iSCSI
or all FCoE network.  So FC is going to be with us.  I do not understand
why folks are going so hyper over the FC frames being transported on a
special Data Center Ethernet link instead of a FC physical Link.  It is
still FC, and we have that today. 

Many servers are asking for an evolutionary way to combine their
Networking connections from the Server.  The customers I have dealt with
do NOT want to rip out FC, they want to provide a single Link for
transport of all networking needs, including storage, exiting their
servers.  One needs to understand: this is still FC. But now the needs
of a single connection type can be phased in one Host system at a time,
and at some other time even the storage can be connected via FCoE.
However, there does not seem to be a compelling argument for the use of
FCoE to storage, at least not as compelling as the Server side.  But
some vendors will probably offer an FCoE connected storage controller.
This however, has little need for link consolidation, and FC will
continue to operate very well and I expect that even is FCoE is accepted
at the server side, most storage controller will remain FC.  

iSCSI still has an important place in the enterprise, but one approach
(FC, iSCSI, or FCoE) does not fit all.

.

.

.

John L Hufferd

[email-masked] <mailto:[email-masked]> 

________________________________

From: Eddy Quicksall [mailto:[email-masked]] 
Sent: Wednesday, April 25, 2007 12:15 PM
To: Eddy Quicksall; Julian Satran
Cc: [email-masked]; John Hufferd
Subject: Re: FW: [Ips] Recent comments about FCoE and iSCSI

Further, I think PAUSE is only for point-to-point and I think that is
too restrictive. 

Eddy

	----- Original Message ----- 

	From: Eddy Quicksall <mailto:[email-masked]>  

	To: Julian Satran <mailto:[email-masked]>  

	Cc: [email-masked] ; John Hufferd <mailto:[email-masked]>  

	Sent: Wednesday, April 25, 2007 2:55 PM

	If it was not obvious in my wording, I meant to agree with you
on your observations.

	Eddy

		----- Original Message ----- 

		From: Julian Satran <mailto:[email-masked]>  

		To: Eddy Quicksall
<mailto:[email-masked]>  

		Cc: [email-masked] ; John Hufferd
<mailto:[email-masked]>  

		Subject: Re: FW: [Ips] Recent comments about FCoE and
iSCSI

		Sorry Eddy by sizable I meant even at the size of a
modern data center. Julo 

25/04/07 11:08 

To

cc

Subject

 

		I basically said that in the summery line by saying "it
will not route on the "global" scale like TCP/IP would". 
		----- Original Message ----- 
		From: Julian Satran <mailto:[email-masked]>  
		To: Eddy Quicksall
<mailto:[email-masked]>  
		Cc: [email-masked] ; John Hufferd
<mailto:[email-masked]>  
		Sent: Wednesday, April 25, 2007 11:04 AM 
		Subject: Re: FW: [Ips] Recent comments about FCoE and
iSCSI 

		Eddy, 
		
		That is oversimplified and ignore the drop rate
assumption and error rate assumptions made in FCP(FCP has no transport
layer). To get to it on a sizable network requires more than PAUSE. 
		
		Julo 

"Eddy Quicksall" <[email-masked]
<mailto:[email-masked]> > 

25/04/07 10:07 

To

"John Hufferd" <[email-masked] <mailto:[email-masked]> >,
Julian Satran/Haifa/IBM@IBMIL <mailto:Satran/Haifa/IBM@IBMIL>  

cc

<[email-masked] <mailto:[email-masked]> > 

Subject

 

		
		Basically, it is sending FC frames over Ethernet. This
localizes the traffic unless you route based on MAC addresses. So you
send 2146 bytes of FC frame plus 18 bytes of Ethernet overhead as FCoE
"standard" packet. 18 bytes of Ethernet gets stripped and you have
straight FC frame that can go through any FC network. Now you can have
10G Ethernet pipes into existing FC SANs. Limited market potential as
far as I can see. The key argument is it much easier to implement than
iSCSI and also has less overhead and uses all the benefits of FC. End to
End credits are simulated using PAUSE command on Ethernet and MAC
addresses are mapped into WWNs. 

		Biggest knock is that it will not route on the "global"
scale like TCP/IP would. 

		Eddy

---

From: "Sandars, Ken" <[email-masked]>
Date: Wed, 25 Apr 2007 18:41:13 -0700

Hey John,
 
[Hufferd] Many servers are asking for an evolutionary way to combine
their Networking connections from the Server.  The customers I have
dealt with do NOT want to rip out FC, they want to provide a single Link
for transport of all networking needs, including storage, exiting their
servers.

I'm not sure what you are saying here. Are you saying servers should
have a single type of physical network connection, presumably ethernet?
How does that align with not wanting to rip out FC?
 
Thanks
Ken

---

From: "John Hufferd" <[email-masked]>
Date: Wed, 25 Apr 2007 21:51:50 -0700

Ken,

The term FCoE has as its primary component FC.  Consider the possibility
that the DCE Link from the Host connects to a switch/device that is able
to deal with the FC part of the FCoE.

.

.

.

John L Hufferd

From: Sandars, Ken [mailto:[email-masked]] 
Sent: Wednesday, April 25, 2007 6:41 PM
To: John Hufferd; Eddy Quicksall; Julian Satran
Cc: [email-masked]
Subject: RE: FW: [Ips] Recent comments about FCoE and iSCSI

Hey John,

Thanks

Ken

---

From: "Eddy Quicksall" <[email-masked]>
Date: Wed, 25 Apr 2007 15:15:22 -0400

Further, I think PAUSE is only for point-to-point and I think that is too restrictive.

Eddy
  ----- Original Message ----- 
  From: Eddy Quicksall 
  To: Julian Satran 
  Cc: [email-masked] ; John Hufferd 
  Sent: Wednesday, April 25, 2007 2:55 PM
  Subject: Re: FW: [Ips] Recent comments about FCoE and iSCSI

  Eddy

         

    ----- Original Message ----- 
    From: Julian Satran 
    To: Eddy Quicksall From [email-masked] Thu Apr 26 09:12:29 2007
From: "Eddy Quicksall" <[email-masked]>
To: "Eddy Quicksall" <[email-masked]>,
	"Julian Satran" <[email-masked]>
Subject: Re: FW: [Ips] Recent comments about FCoE and iSCSI
Date: Wed, 25 Apr 2007 15:15:22 -0400
Cc: [email-masked], John Hufferd <[email-masked]>

This is a multi-part message in MIME format.
Further, I think PAUSE is only for point-to-point and I think that is too restrictive.

Eddy
  ----- Original Message ----- 
  From: Eddy Quicksall 
  To: Julian Satran 
  Cc: [email-masked] ; John Hufferd 
  Sent: Wednesday, April 25, 2007 2:55 PM
  Subject: Re: FW: [Ips] Recent comments about FCoE and iSCSI

  Eddy

         

    ----- Original Message ----- 
    From: Julian Satran 
    To: Eddy Quicksall 
    Cc: [email-masked] ; John Hufferd 
    Sent: Wednesday, April 25, 2007 11:04 AM 
    Subject: Re: FW: [Ips] Recent comments about FCoE and iSCSI 

    Eddy, 

    Julo 

                

    Eddy 

     ----- Original Message ----- 

    From: Julian Satran 
    To: John Hufferd 
    Cc: [email-masked] 

    You make it sound like: 

    > handle cluster messaging, gener

    Eddy, 

    Julo 

                

    Eddy 

     ----- Original Message ----- 

    From: Julian Satran 
    To: John Hufferd 
    Cc: [email-masked] 

    You make it sound like: 

    > things that connect using FCoE and that evolution will take at least a

    It is server cost statement. It costs nothing to connect a modern server to ethernet it will cost a bundle to connect to FCoE and it will force users in short al messaging, and storage.  iSCSI is

    > Feel free to use this lived bad solutions. 

    > effort to combine routers and bridges (rbridmaterial in a nay form. I expect this group to

----------------------------------------------------------------------------

----------------------------------------------------------------------------

---

From: "Robert Snively" <[email-masked]>
Date: Wed, 25 Apr 2007 16:04:13 -0700

The standardization process is just beginning on FCoE.  Brocade and 
at least two other organizations have done significant work on the 
subject.  I expect content similar to the Cisco patent to be
a significant contribution to the FCoE standard.  As you know,
standards development often involves negotiation of details of
a technology.
 
Bob

From: Eddy Quicksall [mailto:[email-masked]] 
Sent: Wednesday, April 25, 2007 2:41 PM
To: Julian Satran
Cc: [email-masked]; John Hufferd
Subject: Re: FW: [Ips] Recent comments about FCoE and iSCSI

I notice that Cisco has a patent on Fibre Channel over Ethernet. Is FCoE
something different?

	----- Original Message ----- 
	From: Julian Satran <mailto:[email-masked]>  
	To: Eddy Quicksall <mailto:[email-masked]>  
	Cc: [email-masked] ; John Hufferd <mailto:[email-masked]>  
	Sent: Wednesday, April 25, 2007 11:52 AM
	Subject: Re: FW: [Ips] Recent comments about FCoE and iSCSI

	Sorry Eddy by sizable I meant even at the size of a modern data
center. Julo 

25/04/07 11:08 

To

	I basically said that in the summery line by saying "it will not
route on the "global" scale like TCP/IP would". 
	----- Original Message ----- 
	From: Julian Satran <mailto:[email-masked]>  
	To: Eddy Quicksall <mailto:[email-masked]>  
	Cc: [email-masked] <mailto:[email-masked]>  ; John Hufferd

	Eddy, 
	
	That is oversimplified and ignore the drop rate assumption and
error rate assumptions made in FCP(FCP has no transport layer). To get
to it on a sizable network requires more than PAUSE. 
	
	Julo 

25/04/07 10:07 

To

	

	Basically, it is sending FC frames over Ethernet. This localizes
the traffic unless you route based on MAC addresses. So you send 2146
bytes of FC frame plus 18 bytes of Ethernet overhead as FCoE "standard"
packet. 18 bytes of Ethernet gets stripped and you have straight FC
frame that can go through any FC network. Now you can have 10G Ethernet
pipes into existing FC SANs. Limited market potential as far as I can
see. The key argument is it much easier to implement than iSCSI and also
has less overhead and uses all the benefits of FC. End to End credits
are simulated using PAUSE command on Ethernet and MAC addresses are
mapped into WWNs. 

	Biggest knock is that it will not route on the "global" scale
like TCP/IP would. 

	Eddy 

	 ----- Original Message ----- 

	From: Julian Satran <mailto:[email-masked]>  
	To: John Hufferd <mailto:[email-masked]>  
	Cc: [email-masked] <mailto:[email-masked]>  
	Sent: Wednesday, April 25, 2007 8:09 AM 
	Subject: Re: FW: [Ips] Recent comments about FCoE and iSCSI 

	"John Hufferd" <[email-masked]
	
	> to Desktops and Laptop systems, and systems at distance.
	
	You make it sound like: 
	1.        most of the servers in the world have their storage on
the network - and that is not the case 
	2.        FCP is basically better performing than iSCSI - and
that is not true either 
	3.        Gatewaying is expensive - and it is perhaps so but
only if you are completely relying on FCP storage (and there are plenty
of good iSCSI vendors of storage)and pushing the price on the servers is
not cheap either - at least not for the server buyer 
	
	> message trucking to the IP outfacing network. 
	
	The rosy future of the yet to appear DCE/CEE and a layer 2 only
world. 
	First you have some terms confused: 
	
	Bridging is the term commonly used for Layer-2 switching and
routing is therm used for layer-3 (switching). 
	
	Bridging has some advantages (less management) that have created
a movement towards an enterprise wide 
	LAN. But this has a long way to go and will require significant
equipment and protocol changes. 
	Even its proponents do not call for transportless networks,
lossless networks etc. 
	The second trouble with your argument is that there are no known
large scale networking technologies that 
	really work at full speed (high speed) and are lossless
(flow-controlled) and errorless like FCoE assumes. 
	The TCP/IP has solved this issue for every generation using the
proven end-to-end principle (and is doing so now). 
	And it is not by chance so and that is why all networking
applications are built above layer-3 and not dropping layer-3 (like
FCoE) does. 
	
	Although I can understand the DCE arguments as a management
statement I would prefer like any rational engineer, to base my building
blocks on structures that are proven and long lasting. And those are
still the end-to-end TCP/IP that can accommodate even your FCP addicts.
The IPS TWG has developed the iFCP that does exactly what FCoE claims to
do an a better base. 
	
	> things that connect using FCoE and that evolution will take at
least a
	
	It is server cost statement. It costs nothing to connect a
modern server to ethernet it will cost a bundle to connect to FCoE and
it will force users in short lived bad solutions. 

	> we do not see FCoE going beyond the constraints of the Data
Center. 
	
	Data Centers now grow to tens of thousands of nodes. There is no
layer-2 technology for errorless/lossless operation at this scale and
there is no good reason to pursue one. The only possible reason (good
reason) is the bridging infrastructure but that infrastructure has a
completely different rationale than the flowcontrol. 
	
	> None of the above cancels out the value of iSCSI in numerous
	
	iSCSI is good for all environments. Business consideration (and
some politics) keep it form "exploding" and large storage vendors are
completely indifferent to the network connection they are using. 
	You and I have also slightly different views of DCE. I expect
DCE (that still has a way to go) to improve the QoS in the data-center
(and for storage too). You expect it to bring the loss rates down to the
levels that FCP assumes (FCP has no transport layer) and that is
probably a pipe dream. Todays transport solution for loss mitigation are
far more cost effective - and that's why iFCP is a better proposition as
a transition technology than FCoE and iSCSI with gateways is propably
better in the long run. 

	
https://www1.ietf.org/mailman/listinfo/ips______________________________
_________________
	Ips mailing list
	[email-masked]
	https://www1.ietf.org/mailman/listinfo/ips

---

From: [email-masked]
Date: Thu, 26 Apr 2007 14:48:33 -0400

How would an FCoE-based initiator communicate with iSCSI-based storage? 

In other words, can the same host adapter be used for FCoE connections
and iSCSI sessions, and if so, would the performance be similar?

thanks,
dj

	From: John Hufferd [mailto:[email-masked]] 
	Sent: Thursday, April 26, 2007 12:52 AM
	To: Sandars, Ken; Eddy Quicksall; Julian Satran
	Cc: [email-masked]
	Subject: RE: FW: [Ips] Recent comments about FCoE and iSCSI

	Ken,

	The term FCoE has as its primary component FC.  Consider the
possibility that the DCE Link from the Host connects to a switch/device
that is able to deal with the FC part of the FCoE.

	.

	.

	.

	John L Hufferd

	Hey John,

	[Hufferd] Many servers are asking for an evolutionary way to
combine their Networking connections from the Server.  The customers I
have dealt with do NOT want to rip out FC, they want to provide a single
Link for transport of all networking needs, including storage, exiting
their servers.

	I'm not sure what you are saying here. Are you saying servers
should have a single type of physical network connection, presumably
ethernet? How does that align with not wanting to rip out FC?

	Thanks

	Ken

---

From: "John Hufferd" <[email-masked]>
Date: Thu, 26 Apr 2007 14:25:18 -0700

David,
The first part of you question has perhaps a different answer than you
"In other words" part.

The first answer is it communicates the same way a FC adapter
communicates with iSCSI based storage --- via FC to iSCSI gateways.

The answer you maybe asking (in your Other Words part) is that it is
possible for a smart NIC that supports maybe iSCSI, TOE, NIC, and even
perhaps iWARP, to have another interface that supports FCoE.  The FCoE
part would send FC frames through the same NIC that was used by the
iSCSI part, but the FCoE frames would be headed (without a gateway) to a
different place then the iSCSI frames were headed. 

.
.
.
John L Hufferd

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Thursday, April 26, 2007 11:49 AM
To: [email-masked]
Subject: RE: FW: [Ips] Recent comments about FCoE and iSCSI

thanks,
dj

	Ken,

	.

	.

	.

	John L Hufferd

	Hey John,

	Thanks

	Ken

---

From: Michael Krause <[email-masked]>
Date: Thu, 26 Apr 2007 15:06:19 -0700

Out of curiosity, unless someone is proposing a change in the iSCSI spec to 
accommodate something unique to FCoE, isn't this entire debate really not 
an IETF topic?   Until the proponents of FCoE make their draft proposals 
available for public consumption and debate, this discussion is somewhat 
occurring in a vacuum.   Same goes for comprehending all that is in 
DCE.   It needs to be completely public to understand whether what is 
proposed makes sense to the broader implementation community w.r.t. to 
storage.

Mike

---

From: "Silvano Gai" <[email-masked]>
Date: Thu, 26 Apr 2007 15:29:32 -0700

I agree,
FCoE is being discussed in T11, in the FC-BB working group, as an
additional transport for FC.

-- Silvano

> From: Michael Krause [mailto:[email-masked]]
> Sent: Thursday, April 26, 2007 3:06 PM
> To: [email-masked]
> Out of curiosity, unless someone is proposing a change in the iSCSI
spec
> to