Subject: RE:iFCP: iFCP -06 comments

From: Charles Monia <[email-masked]>
Date: Sun, 18 Nov 2001 20:56:24 -0800

Hi David:

Thanks for reviewing the specification.  The following are responses to
selected comments. We (the co-authors) are in agreement with all others and
will change the spec accordingly.

> 9.2.1 - The last paragraph on what to do on loss of 
> synchronization seems
> inadequate.  It's current state appears to allow stale frame 
> propagation
> by a compliant implementation.  Was this the intended outcome?
> 

The issue is what to do if a gateway that was enforcing stale frame
detection enters the unsynchronized state. The original intent was to make
the gateway response discretionary.  i.e..  Either continue operation but
suspend enforcement or terminate operation.

For consistency, we propose that an implementation enforcing the detection
of stale frames should always do the latter (terminate operation).
Otherwise, of course, stale frame detection would not have been enabled to
begin with.

> 10.4 b) What happens when the broadcast frame exceeds the MTU 
> size?  This
> seems to result in a rather unreliable broadcast service, as 
> all of the UDP
> datagrams could well be dropped, consistently and repeatedly for large
> enough broadcast frames.

We will rewrite the section to:

a) Specify the use of source fragmentation if the datagram exceeds the PMTU.
b) Require that the receiving broadcast server be capable of accepting the
maximum FC frame size.

As a point of information, the expected frame sizes are small. In FC,
factors limiting broadcast frame size are the fabric and N_PORT MTUs. Fibre
Channel requires an N_PORT MTU to be at least 256 bytes. Therefore, for a
broadcast frame to be receivable by all N_PORTs, the frame size should not
exceed that limit.

> 7.3.7 - Why is PLOGI in a subsection of 7.3 when it's not augmented?
> 

iFCP defines an augmented ELS as any ELS requiring gateway intervention,
whether or not there's supplemental information to be processed.

> 7.3.8 - Three translation types are shown for the REC 
> Exchange originator.
> Again, I think this should be type 1.  Variants of this 
> problem are also
> present in all of 7.3.9 through 7.3.15.
> 

The issue relates to the manner in which the following ELSs are augmented:

7.3.10, 7.3.11  -- Read Exchange Status Block (RES) and RES accept
7.3.12 -- Read Link Error Status (RLS)
7.3.13 -- Read Sequence Status Block (RSS)
7.13.14 -- Reinstate Recovery Qualifier (RRQ)
7.13.15 -- Request Sequence Initiative (RSI)

(REC has since been deleted from FC-FS and will be removed from the iFCP
spec.)

As we interpret FC-FS, RES, RLS, and RSS contain N_PORT identifiers in the
payload which may not necessarily correspond to those of the frame source or
destination.  In the case of RES, for example, FC-FS states:

"The RES ELS Request Sequence requests an N_Port to return the Exchange
Status Block as defined in 17.8.1 for the RX_ID or OX_ID originated by the
S_ID specified in the Payload of this Request Sequence..."

To us, it appears that nothing in the spec requires the S_ID to correspond
to that of the requesting or responding N_PORTs. We will request
clarification on this point from T11.

For RSI (Request Sequence Initiative) and RRQ (Reinstate Recovery
Qualifier), the parameter in question is the S_ID of the exchange
originator, which may correspond to either the ELS originator or recipient,
hence we believe a translation type of 1 (frame originator) or 2 (frame
recipient) is correct.

> 12.2 - Please delete d) as MPLS is *NOT* a QoS technology.  

After a discussion of over-provisioning, IntServ and DiffServ, we feel MPLS
is appropriate since it touches on traffic engineering. For instance, one
may load a MPLS forwarding equivalence class (FEC) with QoS class
significance, in addition to other considerations such as protection and
diversity for the given path. The complementarity and compatibility of MPLS
with, say, DiffServ is explored in draft-ietf-mpls-diff-ext-09.txt (wherein
the PHB bits are copied to the EXP bits of the MPLS shim header). 

> In addition,
> the entire paragraph after d) appears to have very little content, and
> I'm not at all sure about the value of discussion SLAs here.

While we are willing to remove this, we believe there is value in taking the
reader through a QoS use case for iFCP.

If possible, we'd like to salvage this material and would appreciate
suggestions on enhancing or presenting it more appropriately.

Charles
Charles Monia
Senior Technology Consultant
Nishan Systems
email: [email-masked]
voice: (408) 519-3986
fax:   (408) 435-8385

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]
> Sent: Tuesday, November 13, 2001 5:22 PM
> To: [email-masked]
> Subject: iFCP -06 comments
> 
> 
> At the request of the iFCP authors, I've reviewed the -06 version of
> the iFCP draft.  Here are my comments divided into Major, Minor and/or
> Editorial, and Formatting categories.  
> 
> ----- Major -----
> 
> 5.3.2/5.3.2.1/5.3.2.1.1 - These are written as descriptions 
> of a possible
> implementation.  That's fine, but it's necessary to be crisp 
> about what
> the requirements are on an implementation that may not go 
> about this in
> the fashion envisioned here.  Please go over these sections, 
> figure out
> the requirements on an arbitrary implementation and put in 
> the requisite
> MUSTs and SHOULDs in addition to the current description of how things
> could be implemented.  I believe the functional equivalent of the
> translation
> table and its entries MUST exist, so that's somewhere to start from.
> Other areas of the document have lesser versions of this 
> problem, so please
> go over the entire document and check it for this.  I also 
> saw a number
> of places with lower case requirements words (e.g., "shall") 
> that probably
> need to be upper case.
> 
> 6.2.1
>          An N_PORT is identified by its network address consisting of:
> 
>          a) The N_PORT I/D assigned by the gateway to which 
> the N_PORT is
>             locally attached and
> 
>          b) The IP address of the gateway's iFCP Portal.
> 
> Use of an IP address to identify a remote gateway needs to be 
> reconsidered.
> Please add something to allow multiple iFCP implementations 
> to exist at
> different TCP ports on the same IP address (or explain why this has to
> be prohibited).  This is strongly related to the NAPT issues that the
> FCIP folks are in the midst of working through.
> 
> 6.2.2.2 - This section carries a strong risk of over-specification.
> As work on TCP evolves, there's a strong risk of entries in this
> table conflicting with the then-applicable RFCs.  The ground 
> rule should
> be to only specify something here when it has serious consequences
> (i.e., there are potentially very bad effects from ignoring 
> the SHOULD).
> I think I can accept this argument for the first 4 lines in the table,
> but I think the last three should be removed as I don't think
> they're crucial to iFCP per se (the discussion of them seems to
> amount to "good things to do in general", and there's a strong risk
> of further requirements changes/tweaks in this area).  Keep in mind
> that I'm one of the co-authors of RFC 3168 (ECN).  Also, I think
> keeping the "should"s and "should not"s lower case (so that they're
> advice to implementers rather than protocol requirements) is a good
> thing to do in this section.
> 
> 6.2.2.3 - I think this Section needs to go away.  6.2.2.3.1 
> is essentially
> repeating information about TCP implementations in general 
> that is already
> to be found elsewhere and should be removed.  6.2.2.3.2 
> should be moved
> to 6.2.3.2.
> 
> I think the order of Sections 7 and 8 should be reversed, as the TCP
> connection control material is needed to understand how iFCP functions
> before discussing the interesting things it has to do to some ELSs.
> 
> 9.2.1 - The last paragraph on what to do on loss of 
> synchronization seems
> inadequate.  It's current state appears to allow stale frame 
> propagation
> by a compliant implementation.  Was this the intended outcome?
> 
> 10.4 b) - How can one be sure that the local gateway knows 
> about all the
> remote gateways?  I suspect this involves iSNS, and needs to 
> be explained.
> 
> 10.4 b) What happens when the broadcast frame exceeds the MTU 
> size?  This
> seems to result in a rather unreliable broadcast service, as 
> all of the UDP
> datagrams could well be dropped, consistently and repeatedly for large
> enough broadcast frames.
> 
> 
> ---- Minor and/or Editorial -----
> 
> 4.4 a) It's not exactly correct to describe the Node WWN
> as being an identifier for the device.  While that
> was the original intent, it isn't always implemented
> that way.  In practice, I don't think Node WWNs are
> used for much, and that might be worth noting.
> 
> 4.7.1 - It might be worth describing how Area ID and Port ID
> are assigned when an FC-AL loop is attached to a switch
> to give the reader a slightly better feel for this
> (Area ID is assigned to switch port and Port ID is the
> loop port identifier).
> 
> 4.8 - Track state of things in T11 in determining whether
> its appropriate to add Class 4 and 6 to this description.
> From FCIP discussions, it sounds like at least Class 4
> should be added.
> 
> 4.9 - Might want to add a sentence to make it clear that these
> login processes occur at FC-2, and ULP (FC-4) protocol
> setup takes place over the established FC-2 N_Port to N_Port
> session in whatever manner the ULP desires
> 
> Figure 7 - I think "IP network" would be a better term for what's
> below the line than "IP fabric".  I understand what an
> "iFCP fabric" is, but I'm not at all sure about an "IP fabric".
> 
> 5.2.1 - Might want to add a sentence indicating how an FC device
> discovers that Class 1 isn't supported (gets told by the
> iFCP gateway on Fabric Login).
> 
> p. 20
>          All iFCP implementations MUST support operation in address
>          translation mode. Support for address transparent 
> mode is optional
> 
> "optional" should be all upper-case.
> 
>          The implementation MUST NOT allow the mode to be
>          changed after iFCP sessions have been established.
> 
> So, the mode can be changed while a session is being 
> established?  I don't
> think so and suggest that the above wording be changed.  One 
> possibility
> would be to prohibit a mode change while any device is logged into it
> via FLOGI.
> 
> 5.3.1 - Somewhere near the end of this section please say that FC
> frame CRCs have to be regenerated as a consequence of performing
> address translation.
> 
> Both 5.3 and 5.3.1 contain some rationale text about the 
> differences between
> address-transparent and address-translation mode and why one 
> might select
> one or the other that might be better separated out from the 
> description
> of how iFCP works in these modes.  In particular, iSNS pops 
> up without any
> introduction in 5.3.1 a) - this text really needs to come 
> after a discussion
> of iSNS and its responsibilities for/interaction with iFCP.
> 
> 
> 5.3.1.1 - The whole discussion of domain ID assignment is 
> rather imprecise.
> Please put in a "MUST" requirement for unique domain IDs, and 
> indicate that
> iSNS can do this (with a specific description of how) in 
> addition to manual
> assignment.
> 
> 5.3.1.2 - Please upper-case "shall" in the first sentence here.  Also
> applies
> to 5.3.2.3.
> 
> 6.2.2.1 - For a), please indicate how the gateway knows which remote
> gateway to use and what it's address is.  This involves translation
> table entries that were initialized by iSNS replies.
> 
> 6.2.3 - Please be clear about what exactly is being 
> terminated or aborted.
> I think the iFCP session (TCP connection) between gateways is
> what's involved, but there's enough FC mechanism discussion here
> to be unclear.
> 
> 6.3 - I guess this reference to RFC1323 is ok, although it strikes me
> as superfluous.  Please indicate that it's Appendix B of RFC 1323.
> 
> 6.4.4 - State that the FC CRC MUST be recalculated due to the
> address translation.
> 
> 7.3.1 - Two translation types are shown for the ABTX Exchange 
> originator.
> I think this should be Type 1.
> 
> 7.3.5 - Translation table is incomplete for FARP-REQ.
> 
> 7.3.7 - Why is PLOGI in a subsection of 7.3 when it's not augmented?
> 
> 7.3.8 - Three translation types are shown for the REC 
> Exchange originator.
> Again, I think this should be type 1.  Variants of this 
> problem are also
> present in all of 7.3.9 through 7.3.15.
> 
> 7.4 - Table 4 consists entirely of "n" and "M" entries, so delete the
> explanation of the unused "y" entry.
> 
> 8.1 - For clarity indicate that the connection handle is 
> needed to unbind
> connections (yes, I know this is discussed in the next section).
> 
> 11.3.1 - Please remove the "MUST implement" for DES (it's ok 
> to make DES
> OPTIONAL), and think about rewording the "As stated in" statements, as
> we are overriding some of the requirements in some of the 
> reference RFCs.
> 
> 11.3.1.1 - This is ok in a working version of the draft, but 
> will vanish
> in the final version (ditto the subject to availability of an RFC
> language about AES earlier) because either we will make a normative
> reference to the AES RFCs or RFC-to-be, or we will delete requirements
> for AES.
> 
> 11.3.2
>          If, however, the TCP session is
>          maintained, then a Phase 2 delete message shall 
> trigger a new Quick
>          Mode exchange.  
> 
> Probably not a good idea.  The issue here is that some hardware crypto
> accelerators only have room for a fixed number of phase 2 SAs 
> and hence
> delete old ones to make room for new ones (ideally deleting the least
> recently used, but ...).  Triggering a new Quick Mode immediately in
> response to any delete of an SA for an open TCP connection could
> thrash the SA state resources in such an accelerator.  Triggering
> the new Quick Mode based on traffic sent over the SA should 
> work better.
> 
> 12.2 - Please delete d) as MPLS is *NOT* a QoS technology.  
> In addition,
> the entire paragraph after d) appears to have very little content, and
> I'm not at all sure about the value of discussion SLAs here.  The
> discussion of [00-603] is also questionable.
> 
> Appendix B - Is there an external version of this material that could
> be referenced rather than including it here?
> 
> ----- Formatting -----
> 
> There are a bunch of places where MS Word
> has inserted non-standard MS punctuation characters
> (mostly a u with a circumflex instead of a hyphen)
> Turn off Smart Quotes and Auto Correct and correct these.
> 
> 6.3 has a formatting problem - probably an MS Word style
> that should not have been used.
> 
> Thanks,
> --David
> 
> ---------------------------------------------------
> David L. Black, Senior Technologist
> EMC Corporation, 42 South St., Hopkinton, MA  01748
> +1 (508) 435-1000 x75140     FAX: +1 (508) 497-8500
> [email-masked]       Mobile: +1 (978) 394-7754
> ---------------------------------------------------
>

---

From: [email-masked]
Date: Tue, 20 Nov 2001 11:19:20 -0500

Charles,

> > 9.2.1 - The last paragraph on what to do on loss of synchronization
seems
> > inadequate.  Its current state appears to allow stale frame propagation

Ok, also please make sure that clear guidance and requirements are provided
for when to use stale frame detection - I think this is at least a SHOULD,
and we'll need to make this consistent among FCIP and iFCP to the extent
possible.

> > 10.4 b) What happens when the broadcast frame exceeds the MTU size?
This
> > seems to result in a rather unreliable broadcast service, as all of the
UDP
> a) Specify the use of source fragmentation if the datagram exceeds the
PMTU.

Details will need to be supplied, as the large FC frame is being fragmented
into UDP datagrams, and hence the UDP datagrams each need to contain
information about how to reassemble them into an FC frame (unlike TCP
where one can simply follow the bytestream delivered by TCP).  Also, please
add a discussion about the different levels of reliability assumed/provided
by Fibre Channel vs. UDP - FC expects that all frames will be delivered in
the absence of unusual/extraordinary circumstances, whereas UDP is best
effort, and the network is free to drop UDP datagrams if delivering them
would be seriously inconvenient (e.g., router or switch ran out of buffer
space).  The latter discussion is needed to check whether UDP is an
acceptable implementation of FC broadcast.

> b) Require that the receiving broadcast server be capable of accepting the

Fine.

> whether or not there's supplemental information to be processed.

This strikes me as excessive consistency and a poor definition of
augmentation.  PLOGI is already discussed in the material on iFCP
session setup in Section 5.  A note at the top of Section 7.3 pointing
back to that discussion of special handling of PLOGI is sufficient
and may allow the PLOGI frame diagram to be deleted (if it's not
deleted, moving it to Section 5 would be a good idea).

> > 7.3.8 - Three translation types are shown for the REC Exchange
originator.
> > Again, I think this should be type 1.  Variants of this problem are also

It might also be useful to ask Bob Snively directly, as the most
important ULP for these ELSs is FCP-2.

> hence we believe a translation type of 1 (frame originator) or 2 (frame

In any case, it is necessary to spell out the rules/procedures that the
gateway originating the augmented ELS MUST follow in order to determine
which type of translation to use.  The current text leaves it to the
gateway implementer's discretion which is almost certain to result in
incorrect choices being made.

> the PHB bits are copied to the EXP bits of the MPLS shim header). 

The text starting with "For instance" above is fine.  Insert that
discussion,
remove the existing MPLS RFC reference and replace it with a reference to
that draft (which has been approved by the IESG to become an RFC).

> suggestions on enhancing or presenting it more appropriately.

I offer the following guidance ...
- Discussing the QoS requirements of iFCP and how existing IETF mechanisms
	can be used to meet them is fine and encouraged.  Explaining how and
	why Fibre Channel traffic may need low jitter, high bandwidth, etc.
	from an IP network is relevant.
- Discussion of SLAs and related contractual arrangements is not a good
idea.
	The IETF considers an SLA to be a contractual vehicle that will
	not be specified in any RFC.  The terms closest to what you're
trying
	to convey are Service Level Specification and Traffic Conditioning
	Specification -- see Section 2 of
draft-ietf-diffserv-new-terms-06.txt .

reconsidered.
> > Please add something to allow multiple iFCP implementations to exist at

Please inform the list of how the authors propose to resolve this.

Thanks,
--David

---

From: Charles Monia <[email-masked]>
Date: Tue, 20 Nov 2001 14:48:37 -0800

David:

> Sent: Tuesday, November 20, 2001 8:19 AM
> To: [email-masked]
> Cc: [email-masked]
> Subject: RE: iFCP: iFCP -06 comments
> 
<stuff deleted>

> > a) Specify the use of source fragmentation if the datagram 
> exceeds the
> PMTU.
> 
> information about how to reassemble them into an FC frame

We had intended to use IP datagram fragmentation (this should have been
mentioned explicitly, I guess).  What more do we need to specify in the way
of a fragmentation and reassembly mechanism?

Charles

---

From: [email-masked]
Date: Tue, 20 Nov 2001 18:55:44 -0500

> > > > 10.4 b) What happens when the broadcast frame exceeds the MTU size?

[... snip ...]

> We had intended to use IP datagram fragmentation

Oh ... I'm not sure that's a real good idea.  Using IP fragmentation is
one of the few ways to make UDP datagram delivery even less reliable
than it already is.  This puts FC broadcast at the mercy of code that
doesn't
fragment correctly at both iFCP and network nodes (infrequently used code
paths are subject to bit rot) and systems in the network that can't or
don't want to deal with IP fragments (e.g., firewalls and NAT/NAPT boxes)
and hence drop rather than fragment.  For source fragmentation, discovering
the MTU to fragment to may also be a problem area.

While I'm not going to put my foot down yet, I would advise thinking about
using TCP to deliver FC broadcast frames over iFCP.  If FC broadcasts are
infrequent and not performance critical, perhaps iSNS could provide the
FC broadcast forwarding service?  This would also clean up any issues around
whether an iFCP gateway knows about all the other gateways in the fabric,
as the iSNS server always knows this.

Thanks,
--David