From owner-forces@PEACH.EASE.LSOFT.COM Wed Oct 9 20:10:52 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA08142 for ; Wed, 9 Oct 2002 20:10:47 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <21.00763330@cherry.ease.lsoft.com>; Wed, 9 Oct 2002 20:12:53 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 28626767 for FORCES@PEACH.EASE.LSOFT.COM; Wed, 9 Oct 2002 20:12:51 -0400 Received: from 192.55.52.25 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Wed, 9 Oct 2002 20:10:51 -0400 Received: from talaria.fm.intel.com (talaria.fm.intel.com [10.1.192.39]) by caduceus.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.51 2002/09/23 20:43:23 dmccart Exp $) with ESMTP id g9A074W10053 for ; Thu, 10 Oct 2002 00:07:04 GMT Received: from fmsmsxv040-1.fm.intel.com (fmsmsxvs040.fm.intel.com [132.233.42.124]) by talaria.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.25 2002/09/23 20:43:13 dmccart Exp $) with SMTP id g9A0Dlj08326 for ; Thu, 10 Oct 2002 00:13:47 GMT Received: from FMSMSX016.fm.intel.com ([132.233.42.195]) by fmsmsxv040-1.fm.intel.com (NAVGW 2.5.2.11) with SMTP id M2002100917112428688 for ; Wed, 09 Oct 2002 17:11:24 -0700 Received: by fmsmsx016.fm.intel.com with Internet Mail Service (5.5.2653.19) id ; Wed, 9 Oct 2002 17:10:48 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/mixed; boundary="----_=_NextPart_000_01C26FF1.753256F0" Approved-By: "Yang, Lily L" Message-ID: Date: Wed, 9 Oct 2002 17:10:36 -0700 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: "Yang, Lily L" Subject: Please consider last call on draft-ietf-forces-framework-02.txt To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_000_01C26FF1.753256F0 Content-Type: text/plain; charset="ISO-8859-1" Hi, We just submitted a new version of the ForCES framework draft. Several people on the list have given us comments on the previous (ver01) draft and we've incorporated all into this version. Most of the change are minor, including: 1) fixing Fig 6 (a) to make it really full mesh 2) Moving the Abstract and Table of Contents to the front. 3) Other editorial changes to improve clarity throughout the draft A more significant modification is in section 5.7. "Application Layer -- Network Management Protocol". Per comments from T. Sridhar, when we talk about how SNMP is supported in ForCES, we suggest that "AgentX framework defined in RFC2741 may be applied here such that CEs act in the role of master agent to process SNMP protocol messages while FEs act in the role of subagent to provide access to the MIB objects residing on FEs. AgentX protocol messages between the master agent (CE) and the subagent (FE) are encapsulated and transported via ForCES, just like data packets from any other application layer protocols." Accordingly, a new reference to RFC2741 is also added in the "normative reference" section. As always, comments are welcome. The co-authors feel that this framework draft has been through several iterations and seems pretty stable now. So we would like the working group chairs to consider last call on this version. Thanks, Lily <> ------_=_NextPart_000_01C26FF1.753256F0 Content-Type: text/plain; name="draft-ietf-forces-framework-02.txt" Content-Disposition: attachment; filename="draft-ietf-forces-framework-02.txt" Content-Transfer-Encoding: quoted-printable Internet Draft L. Yang=20 Expiration: April 2003 Intel Corp.=20 File: draft-ietf-forces-framework-02.txt R. Dantu=20 Working Group: ForCES Univ. of Texas = Dallas=20 T. Anderson=20 Intel Corp. =20 Oct 2002 = =20 =20 =20 ForCES Architectural Framework=20 =20 =20 =20 draft-ietf-forces-framework-02.txt=20 =20 =20 =20 =20 Status of this Memo=20 =20 This document is an Internet-Draft and is in full conformance = with=20 all provisions of Section 10 of RFC2026. Internet-Drafts are=20 working documents of the Internet Engineering Task Force = (IETF), its=20 areas, and its working groups. Note that other groups may also = distribute working documents as Internet-Drafts.=20 =20 Internet-Drafts are draft documents valid for a maximum of six=20 months and may be updated, replaced, or obsoleted by other = documents=20 at any time. It is inappropriate to use Internet-Drafts as=20 reference material or to cite them other than as ``work in=20 progress.''=20 =20 The list of current Internet-Drafts can be accessed at=20 http://www.ietf.org/ietf/1id-abstracts.txt.=20 =20 The list of Internet-Draft Shadow Directories can be accessed = at =20 http://www.ietf.org/shadow.html.=20 =20 Abstract=20 =20 This document defines the architectural framework for ForCES=20 (Forwarding and Control Element Separation) network elements = (NE),=20 and identifies the associated entities and the interaction = among=20 them. This framework is intended to satisfy the requirements=20 specified in the ForCES requirements draft [FORCES-REQ].=20 =20 Table of Contents=20 =20 1. = Definitions.....................................................2=20 2. Introduction to Forwarding and Control Element Separation=20 = (ForCES)...........................................................4=20 3. = Architecture....................................................7=20 3.1. Control Elements and Fr Reference = Point....................8=20 =20 =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 3.2. Forwarding Elements and Fi reference = point.................8=20 3.3. CE = Managers...............................................11=20 3.4. FE = Managers...............................................11=20 4. Operational = Phases.............................................11=20 4.1. Pre-association = Phase.....................................12=20 4.1.1. Fl Reference = Point...................................12=20 4.1.2. Ff Reference = Point...................................13=20 4.1.3. Fc Reference = Point...................................13=20 4.2. Post-association Phase and Fp reference = point.............14=20 4.2.1. Proximity and Interconnect between CEs and = FEs.......14=20 4.2.2. Association = Establishment............................14=20 4.2.3. Steady-state = Communication...........................15=20 4.2.4. Data Packets across Fp reference = point...............16=20 4.2.5. Proxy = FE.............................................17=20 4.3. Association = Re-establishment..............................17=20 5. Applicability to = RFC1812.......................................18=20 5.1. General Router = Requirements...............................18=20 5.2. Link = Layer................................................19=20 5.3. Internet Layer = Protocols..................................20=20 5.4. Internet Layer = Forwarding.................................20=20 5.5. Transport = Layer...........................................21=20 5.6. Application Layer -- Routing = Protocols....................21=20 5.7. Application Layer -- Network Management = Protocol..........21=20 6. = Summary........................................................22=20 7. Security = Considerations........................................22=20 8. Intellectual Property = Right....................................22=20 9. = References.....................................................22=20 9.1. Normative = References......................................22=20 9.2. Informative = References....................................23=20 10. = Acknowledgments...............................................23=20 11. Authors' = Addresses............................................23=20 =20 Conventions used in this document =20 =20 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL = NOT", =20 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" = in=20 this document are to be interpreted as described in [RFC-2119]. = =20 1. Definitions=20 =20 A set of terminology associated with the ForCES requirements is = defined in [FORCES-REQ] and we only include the ones that are = most=20 relevant to this document here. =20 =20 Forwarding Element (FE) - A logical entity that implements the=20 ForCES protocol. FEs use the underlying hardware to provide = per- packet processing and handling as directed by a CE via the = ForCES=20 protocol. =20 =20 Control Element (CE) - A logical entity that implements the = ForCES=20 protocol and uses it to instruct one or more FEs how to process = packets. CEs handle functionality such as the execution of = control =20 and signaling protocols. =20 =20 Yang, et. al. Expires April 2003 [Page 2] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 =20 ForCES Network Element (NE) - An entity composed of one or more = CEs=20 and one or more FEs. To entities outside a NE, the NE = represents a=20 single point of management. Similarly, a NE usually hides its=20 internal organization from external entities. =20 =20 Pre-association Phase - The period of time during which a FE = Manager=20 (see below) and a CE Manager (see below) are determining which = FE=20 and CE should be part of the same network element. Any = partitioning=20 of PFEs and PCEs occurs during this phase. =20 =20 Post-association Phase - The period of time during which a FE = does=20 know which CE is to control it and vice versa, including the = time=20 during which the CE and FE are establishing communication with = one=20 another. =20 =20 ForCES Protocol - While there may be multiple protocols used = within=20 the overall ForCES architecture, the term "ForCES protocol" = refers=20 only to the ForCES post-association phase protocol (see below). = =20 =20 ForCES Post-Association Phase Protocol - The protocol used for = post- association phase communication between CEs and FEs. This = protocol=20 does not apply to CE-to-CE communication, FE-to-FE = communication, or=20 to communication between FE and CE managers. The ForCES = protocol is=20 a master-slave protocol in which FEs are slaves and CEs are = masters. =20 This protocol includes both the management of the communication = channel (e.g., connection establishment, heartbeats) and the = control=20 messages themselves. This protocol could be a single protocol = or=20 could consist of multiple protocols working together. =20 FE Manager - A logical entity that operates in the = pre-association=20 phase and is responsible for determining to which CE(s) a FE = should=20 communicate. This process is called CE discovery and may = involve=20 the FE manager learning the capabilities of available CEs. A = FE=20 manager may use anything from a static configuration to a pre- association phase protocol (see below) to determine which CE(s) = to=20 use. Being a logical entity, a FE manager might be physically=20 combined with any of the other logical entities mentioned in = this=20 section. =20 =20 CE Manager - A logical entity that operates in the = pre-association =20 phase and is responsible for determining to which FE(s) a CE = should =20 communicate. This process is called FE discovery and may = involve=20 the CE manager learning the capabilities of available FEs. A = CE=20 manager may use anything from a static configuration to a pre- association phase protocol (see below) to determine which FE to = use. =20 Being a logical entity, a CE manager might be physically = combined=20 with any of the other logical entities mentioned in this = section. =20 =20 Pre-association Phase Protocol - A protocol between FE managers = and=20 CE managers that is used to determine which CEs or FEs to use. = A=20 pre-association phase protocol may include a CE and/or FE = capability=20 discovery mechanism. Note that this capability discovery = process is=20 =20 Yang, et. al. Expires April 2003 [Page 3] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 wholly separate from (and does not replace) that used within = the=20 ForCES protocol. However, the two capability discovery = mechanisms=20 may utilize the same FE model. =20 =20 FE Model - A model that describes the logical processing = functions=20 of a FE. =20 =20 ForCES Protocol Element - A FE or CE. =20 =20 2. Introduction to Forwarding and Control Element Separation = (ForCES) =20 =20 An IP network element (NE) appears to external entities as a=20 monolithic piece of network equipment, e.g., a router, NAT,=20 firewall, or load balancer. Internally, however, an IP network=20 element (NE) (such as a router) is composed of numerous = logically=20 separated entities that cooperate to provide a given = functionality=20 (such as routing). Two types of network element components = exist:=20 control element (CE) in control plane and forwarding element = (FE) in=20 forwarding plane (or data plane). Forwarding elements = typically are=20 ASIC, network-processor, or general-purpose processor-based = devices=20 that handle data path operations for each packet. Control = elements=20 are typically based on general-purpose processors that provide=20 control functionality like routing and signaling protocols. =20 =20 ForCES aims to define a framework and associated protocol(s) to = standardize information exchange between the control and = forwarding=20 plane. Having standard mechanisms allows CEs and FEs to become=20 physically separated standard components. This physical = separation=20 accrues several benefits to the ForCES architecture. Separate=20 components would allow component vendors to specialize in one=20 component without having to become experts in all components.=20 Standard protocol also allows CEs and FEs from different = component=20 vendors to interoperate with each other and hence it becomes=20 possible for system vendors to integrate together CEs and FEs = from=20 different component vendors. This interoperability translates = into a=20 lot more design choices and flexibility to the system vendors.=20 Overall, ForCES will enable rapid innovation in both the = control and=20 forwarding planes while maintaining interoperability. = Scalability is=20 also easily provided by this architecture in that additional=20 forwarding or control capacity can be added to existing network = elements without the need for forklift upgrades.=20 =20 One example of such physical separation is at the blade level.=20 Figure 1 shows such an example configuration of a router, with = two=20 control blades and multiple router (forwarding) blades, all=20 interconnected into a switch fabric backplane. In such chassis=20 configuration, the control blades are the CEs while the router=20 blades are FEs, and the switch fabric backplane provides the=20 physical interconnect for all the blades. Control blade A may = be the=20 primary CE while control blade B is the backup CE providing=20 redundancy. It is also possible to have a redundant switch = fabric=20 for high availability support. Routers today with this kind of=20 configuration use proprietary interface for messaging between = CEs=20 =20 Yang, et. al. Expires April 2003 [Page 4] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 and FEs. The goal of ForCES is to replace such proprietary = interface=20 with a standard protocol. With a standard protocol like ForCES=20 implemented on all blades, it becomes possible for control = blades=20 from vendor X and routing blades from vendor Y to work = seamlessly=20 together in one chassis. =20 =20 ------------------------- -------------------------=20 | Control Blade A | | Control Blade B |=20 | (CE) | | (CE) |=20 ------------------------- -------------------------=20 ^ | ^ |=20 | | | |=20 | V | V=20 ---------------------------------------------------------=20 | Switch Fabric Backplane |=20 ---------------------------------------------------------=20 ^ | ^ | ^ | = | | | | =85 | | = =20 | V | V | V = ------------ ------------ ------------ = |Router | |Router | |Router |=20 |Blade #1 | |Blade #2 | |Blade #N |=20 | (FE) | | (FE) | | (FE) |=20 ------------ ------------ ------------ = ^ | ^ | ^ | = | | | | =85 | | = =20 | V | V | V = =20 Figure 1. A router configuration example with separate = blades.=20 =20 =20 ------- -------=20 | CE1 | | CE2 |=20 ------- -------=20 ^ ^=20 | |=20 V V=20 = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Ethernet=20 ^ ^ =85 ^=20 | | |=20 V V V=20 ------- ------- --------=20 | FE#1| | FE#2| | FE#n |=20 ------- ------- --------=20 ^ | ^ | ^ | =20 | | | | | | =20 | V | V | V =20 =20 Figure 2. A router configuration example with separate = boxes.=20 =20 Another level of physical separation between the CEs and FEs = can be=20 at the box level. In such configuration, all the CEs and FEs = are=20 physically separated boxes, interconnected with some kind of = high=20 =20 Yang, et. al. Expires April 2003 [Page 5] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 speed LAN connection (like Gigabit Ethernet). These separated = CEs=20 and FEs are only one hop away from each other within a local = area=20 network. The CEs and FEs communicate to each other by running=20 ForCES, and the collection of these CEs and FEs together become = one=20 routing unit to the external world. Figure 2 shows such an = example.=20 =20 In both examples shown here, the same physical interconnect is = used=20 for both CE-to-FE and FE-to-FE communication. However, that = does not=20 have to be the case. One reason to use different interconnect = is=20 that CE-to-FE interconnect does not have to be as fast as the = FE-to- FE interconnect, so the more expensive fast connections can be = saved=20 for FE-to-FE only. The separate interconnects may also provide = reliability and redundancy benefits for the NE.=20 =20 -------------------------------------------------=20 | | | | | | |=20 |OSPF |RIP |BGP |CAC |LDP |=85 |=20 | | | | | | |=20 -------------------------------------------------=20 | ForCES Interface |=20 -------------------------------------------------=20 ^ ^ =20 ForCES | |data =20 control | |packets =20 messages| |(e.g., routing packets) =20 v v=20 -------------------------------------------------=20 | ForCES Interface |=20 -------------------------------------------------=20 | | | | | | |=20 |LPM Fwd|Meter |Shaper |NAT |Classi-|=85 |=20 | | | | |fier | |=20 -------------------------------------------------=20 | FE resources |=20 -------------------------------------------------=20 =20 Figure 3. Examples of CE and FE functions=20 =20 Some examples of control functions that can be implemented in = the CE=20 include routing protocols like RIP, OSPF and BGP, control and=20 signaling protocols like CAC (Call Admission Control), LDP = (Label=20 Distribution Protocol) for MPLS, etc. Examples of forwarding=20 functions in FE include LPM (longest prefix match) forwarder,=20 classifiers, traffic shaper, meter, NAT, etc. Figure 3 shows a=20 diagram with examples in both CE and FE. Any given NE may = contain=20 one or many of these CE and FE functions in it. The diagram = also=20 shows that ForCES protocol is used to transport both the = control=20 messages for ForCES itself and the data packets that are=20 originated/destined from/to the control functions in CE (e.g.,=20 routing packets). Section 5.2.4 provides more detail on this. =20 =20 A set of requirements for control and forwarding separation is=20 identified in [FORCES-REQ]. This document describes a ForCES=20 =20 Yang, et. al. Expires April 2003 [Page 6] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 architecture that satisfies the architectural requirements of = that=20 document and defines a framework for ForCES network elements = and=20 associated entities to facilitate protocol definition.=20 =20 3. Architecture=20 =20 This section defines the ForCES architectural framework and the = associated logical components. This ForCES framework defines=20 components of ForCES NEs including several ancillary = components. =20 These components may be connected in different kinds of = topologies=20 for flexible packet processing. =20 =20 --------------------------------------- = | ForCES Network Element | = -------------- Fc | -------------- -------------- | = | CE Manager |---------+-| CE 1 |------| CE 2 | | = -------------- | | | Fr | | | = | | -------------- -------------- | = | Fl | | | Fp / | = | | Fp| |----------| / | = | | | |/ | = | | | | | = | | | Fp /|----| | = | | | /--------/ | | = -------------- Ff | -------------- -------------- | = | FE Manager |---------+-| FE 1 | Fi | FE 2 | | = -------------- | | |------| | | = =20 | -------------- -------------- | = | | | | | | | | | | = ----+--+--+--+----------+--+--+--+----- = | | | | | | | |=20 | | | | | | | |=20 Fi/f Fi/f=20 =20 Figure 4. ForCES Architectural Diagram=20 =20 The diagram in Figure 4 shows the logical components of the = ForCES=20 architecture and their relationships. There are two kinds of=20 components inside a ForCES network element: control element = (CE) and=20 forwarding element (FE). The framework allows multiple = instances of=20 CE and FE inside one NE. Each FE contains one or more physical = media=20 interfaces for receiving and transmitting packets from/to the=20 external world. The aggregation of these FE interfaces becomes = the=20 NE=92s external interfaces. In addition to the external = interfaces,=20 there must also exist some kind of interconnect within the NE = so=20 that the CE and FE can communicate with each other, and one FE = can=20 forward packets to another FE. The diagram also shows two = entities=20 outside of the ForCES NE: CE Manager and FE Manager. These two=20 entities provide configuration to the corresponding CE or FE in = the=20 pre-association phase (see Section 5.1). There is no defined = role=20 for FE Manager and CE Manager in post-association phase, thus = these=20 logical components are not considered part of the ForCES NE. =20 =20 =20 Yang, et. al. Expires April 2003 [Page 7] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 For convenience, the logical interactions between these = components=20 are labeled by reference points Fp, Fc, Ff, Fr, Fl, and Fi, as = shown=20 in Figure 4. The FE external interfaces are labeled as Fi/f. = More=20 detail is provided in Section 4 and 5 for each of these = reference=20 points. All these reference points are important in = understanding=20 the ForCES architecture, however, the ForCES protocol is only=20 defined over one reference point -- Fp. =20 =20 The interface between two ForCES NEs is identical to the = interface=20 between two conventional routers and these two NEs exchange the = protocol packets through the external interfaces at Fi/f. = ForCES NEs=20 connect to existing routers transparently.=20 =20 3.1. Control Elements and Fr Reference Point=20 =20 It is not necessary to define any protocols across the Fr = reference=20 point to enable control and forwarding separation for simple=20 configurations like single CE and multiple FEs. However, this=20 architecture permits multiple CEs to be present in a network=20 element. In cases where an implementation uses multiple CEs, it = is=20 expected the invariant that the CEs and FEs together appear as = a=20 single NE MUST be maintained.=20 =20 Multiple CEs may be used for redundancy, load sharing, = distributed=20 control, or other purposes. Redundancy is the case where one = or=20 more CEs are prepared to take over should an active CE fail. = Load=20 sharing is the case where two or more CEs are concurrently = active=20 and where any request that can be serviced by one of the CEs = can=20 also be serviced by any of the other CEs. In both redundancy = and=20 load sharing, the CEs involved are equivalently capable. The = only=20 difference between these two cases is in terms of how many = active=20 CEs there are. Distributed control is the case where two or = more=20 CEs are concurrently active but where certain requests can only = be=20 serviced by certain CEs.=20 =20 When multiple CEs are employed in a ForCES NE, their internal=20 organization is considered an implementation issue that is = beyond=20 the scope of ForCES. CEs are wholly responsible for = coordinating=20 amongst themselves via the Fr reference point to provide = consistency=20 and synchronization. However, ForCES does not define the=20 implementation or protocols used between CEs, nor does it = define how=20 to distribute functionality among CEs. Nevertheless, ForCES = will=20 support mechanisms for CE redundancy or fail over, and it is=20 expected that vendors will provide redundancy or fail over = solutions=20 within this framework.=20 =20 =20 3.2. Forwarding Elements and Fi reference point=20 =20 FEs perform per-packet processing and handling as directed by = CEs.=20 FEs have no initiative of their own. Instead, FEs are slaves = and=20 only do as they are told. FEs may communicate with one or more = CEs=20 concurrently across reference point Fp. FEs have no notion of = CE=20 =20 Yang, et. al. Expires April 2003 [Page 8] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 redundancy, load sharing, or distributed control. Instead, FEs = accept commands from any CE authorized to control them, and it = is up=20 to the CEs to coordinate among themselves to achieve = redundancy,=20 load sharing or distributed control. The idea is to keep FEs as = simple and dumb as possible so that FEs can focus its resource = on=20 the packet processing functions. =20 =20 =20 ------- Fr -------=20 | CE1 | ------| CE2 |=20 ------- -------=20 | \ / |=20 | \ / |=20 | \ / |=20 | \/Fp |=20 | /\ |=20 | / \ |=20 | / \ |=20 ------- Fi -------=20 | FE1 |<----->| FE2 |=20 ------- -------=20 =20 Figure 5. CE redundancy example.=20 =20 For example, in Figure 5, FE1 and FE2 can be configured to = accept=20 commands from both the primary CE (CE1) and the backup CE = (CE2). At=20 the beginning, CE1 issues commands to FEs while CE2 silently = remains=20 in sync with CE1 via CE to CE protocol over Fr reference point. = When=20 CE1 fails, CE2 detects it and starts to take over. Before CE2 = starts=20 issuing commands to the FEs, it might need to recheck the FEs' = state=20 and instruct FEs whether or not it is ok to preserve their = current=20 state. =20 =20 Distributed control can be achieved in the similar fashion, = without=20 much intelligence on the part of FEs. For example, FEs can be=20 configured to detect RSVP and BGP protocol packets, and forward = RSVP=20 packets to one CE and BGP packets to another CE. Hence, FEs may = need=20 to do packet filtering for forwarding packets to specific CEs.=20 =20 This architecture permits multiple FEs to be present in a NE.=20 [FORCES-REQ] dictates that the ForCES protocol MUST be able to = scale=20 to at least hundreds of FEs (see [FORCES-REQ] Section 5, = requirement=20 #11). Each of these FEs may potentially have a different set of = packet processing functions, with different media interfaces. = FEs=20 are responsible for basic maintenance of layer-2 connectivity = with=20 other FEs and with external entities. Many layer-2 media = include=20 sophisticated control protocols. The FORCES protocol (over the = Fp=20 reference point) will be able to carry messages for such = protools so=20 that, in keeping with the "dumb FE=92=92 model, the CE can = provide=20 appropriate intelligence and control over these media.=20 =20 When multiple FEs are present, ForCES requires that packets = MUST be=20 able to arrive at the NE by one FE and leave the NE via a = different=20 =20 Yang, et. al. Expires April 2003 [Page 9] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 FE (See [FORCES-REQ], Section 5, Requirement #3). Packets that = enter the NE via one FE and leave the NE via a different FE are = transferred between FEs across the Fi reference point. The Fi=20 reference point is a separate protocol from the Fp reference = point=20 and is not currently defined by the ForCES architecture.=20 =20 FEs could be connected in different kinds of topologies and = packet=20 processing may spread across several FEs in the topology. = Hence,=20 logical packet flow may be different from physical FE topology. = Figure 6 provides some topology examples. When it is necessary = to=20 forward packets between FEs, CE needs to understand the FE = topology.=20 The FE topology can be queried from FEs by CEs. =20 =20 ----------------- =20 | CE | =20 ----------------- =20 ^ ^ ^=20 / | \=20 / v \=20 / ------- \=20 / +->| FE3 |<-+ \=20 / | | | | \=20 v | ------- | v=20 ------- | | -------=20 | FE1 |<-+ +->| FE2 |=20 | |<--------------->| |=20 ------- -------=20 ^ | ^ |=20 | | | |=20 | v | v=20 =20 (a) Full mesh among FE1, FE2 and FE3.=20 =20 =20 ----------- =20 | CE | =20 ----------- =20 ^ ^ ^ ^=20 / | | \=20 /------ | | ------\=20 v v v v=20 ------- ------- ------- -------=20 | FE1 |<->| FE2 |<->| FE3 |<->| FE4 |=20 ------- ------- ------- -------=20 ^ | ^ | ^ | ^ |=20 | | | | | | | |=20 | v | v | v | v=20 =20 (b) Multiple FEs in a daisy chain=20 =20 =20 =20 =20 Yang, et. al. Expires April 2003 [Page 10] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 ^ |=20 | v=20 ----------- =20 | FE1 |<-----------------------| =20 ----------- |=20 ^ ^ |=20 / \ | =20 | ^ / \ ^ | V=20 v | v v | v ---------- = --------- --------- | | = | FE2 | | FE3 |<------------>| CE | = --------- --------- | | = ^ ^ ^ ---------- = | \ / ^ ^=20 | \ / | |=20 | v v | |=20 | ----------- | |=20 | | FE4 |<----------------------| | =20 | ----------- |=20 | | ^ |=20 | v | |=20 | |=20 |----------------------------------------|=20 =20 (c) Multiple FEs connected by a ring=20 =20 Figure 6. Some examples of FE topology.=20 =20 3.3. CE Managers=20 =20 CE managers are responsible for determining which FEs a CE = should=20 control. It is legitimate for CE managers to be hard-coded = with the=20 knowledge of with which FEs its CEs should communicate. A CE = manager=20 may also be physically embedded into a CE and be implemented as = a=20 simple keypad or other direct configuration mechanism on the = CE. =20 Finally, CE managers may be physically and logically separate=20 entities that configure the CE with FE information via such=20 mechanisms as COPS-PR [RFC3084] or SNMP [RFC1157].=20 =20 3.4. FE Managers=20 =20 FE managers are responsible for determining to which CE any=20 particular FE should initially communicate. Like CE managers, = no=20 restrictions are placed on how a FE manager decides to which = CEs its=20 FEs should communicate, nor are restrictions placed on how FE=20 managers are implemented.=20 =20 4. Operational Phases=20 =20 Both FEs and CEs require some configuration in place before = they can=20 start information exchange and function as a coherent network=20 element. Two operational phases are identified in this = framework --=20 pre-association and post-association. =20 =20 Yang, et. al. Expires April 2003 [Page 11] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 =20 4.1.Pre-association Phase=20 =20 Pre-association phase is the period of time during which a FE=20 Manager and a CE Manager are determining which FE and CE should = be=20 part of the same network element. The protocols used during = this=20 phase may include all or some of the message exchange over Fl, = Ff=20 and Fc reference points. However, all these may be optional and = none=20 of this is within the scope of ForCES protocol.=20 =20 4.1.1. Fl Reference Point=20 =20 =20 FE Manager FE CE Manager CE=20 | | | |=20 | | | |=20 |(security exchange) | |=20 1|<------------------------------>| |=20 | | | |=20 |(a list of CEs and their attributes) |=20 2|<-------------------------------| |=20 | | | |=20 |(a list of FEs and their attributes) |=20 3|------------------------------->| |=20 | | | |=20 | | | |=20 |<----------------Fl------------>| |=20 =20 Figure 7. An example of message exchange over Fl reference = point=20 =20 CE managers and FE managers may communicate across the Fl = reference=20 point in the pre-association phase in order to determine which = CEs=20 and FEs should communicate with each other. Communication = across=20 the Fl reference point is optional in this architecture. No=20 requirements are placed on this reference point.=20 =20 CE managers and FE managers may be operated by different = entities. =20 The operator of the CE manager may not want to divulge, except = to=20 specified FE managers, any characteristics of the CEs it = manages. =20 Similarly, the operator of the FE manager may not want to = divulge FE=20 characteristics, except to authorized entities. As such, CE=20 managers and FE managers may need to authenticate one another. = Subsequent communication between CE managers and FE managers = may=20 require other security functions such as privacy, = non-repudiation,=20 freshness, and integrity.=20 =20 Once the necessary security functions have been performed, the = CE=20 and FE managers communicate to determine which CEs and FEs = should=20 communicate with each other. At the very minimum, the CE and = FE=20 managers need to learn of the existence of available FEs and = CEs=20 respectively. This discovery process may or may not entail one = or=20 both managers learning the capabilities of the discovered = ForCES=20 =20 Yang, et. al. Expires April 2003 [Page 12] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 protocol elements. Figure 7 shows an example of possible = message=20 exchange between CE manager and FE manager over Fl reference = point.=20 =20 4.1.2. Ff Reference Point=20 =20 FE Manager FE CE Manager CE=20 | | | |=20 | | | |=20 |(security exchange) |(security exchange)=20 1|<------------>|authentication = 1|<----------->|authentication | | | |=20 |(FE ID, attributes) |(CE ID, attributes)=20 2|<-------------|request 2|<------------|request=20 | | | |=20 3|------------->|response 3|------------>|response=20 |(corresponding CE ID) |(corresponding FE ID)=20 | | | |=20 | | | |=20 |<-----Ff----->| |<-----Fc---->|=20 =20 Figure 8. Examples of message exchange=20 over Ff and Fc reference points.=20 =20 The Ff reference point is used to inform forwarding elements of = the=20 association decisions made by FE managers in pre-association = phase. =20 Only authorized entities may instruct a FE with respect to = which CE=20 should control it. Therefore, privacy, integrity, freshness, = and=20 authentication are necessary between FE manager and FEs when = the FE=20 manager is remote to the FE. Once the appropriate security has = been=20 established, FE manager instructs FEs across this reference = point to=20 join a new NE or to disconnect from an existing NE. Figure 8 = shows=20 example of message exchange over Ff reference point.=20 =20 Note that the FE manager function may be co-located with the FE = (such as by manual keypad entry of the CE IP address), in which = case=20 this reference point is reduced to a built-in function.=20 =20 4.1.3. Fc Reference Point=20 =20 The Fc reference point is used to inform control elements of = the=20 association decisions made by CE managers in pre-association = phase. =20 When the CE manager is remote, only authorized entities may = instruct=20 a CE to control certain FEs. Privacy, integrity, freshness and = authentication are also required across this reference point in = such=20 a configuration. Once appropriate security has been = established,=20 the CE manager instructs CEs as to which FEs they should = control and=20 how they should control them. Figure 7 shows example of message = exchange over Fc reference point.=20 =20 As with the FE manager and FEs, configurations are possible = where=20 the CE manager and CE are co-located and no protocol is used = for=20 this function.=20 =20 =20 Yang, et. al. Expires April 2003 [Page 13] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 4.2. Post-association Phase and Fp reference point=20 =20 Post-association phase is the period of time during which a FE = and=20 CE have been configured with information necessary to contact = each=20 other and includes both association establishment and = steady-state=20 communication. The communication between CE and FE is performed = across the Fp ("p" meaning protocol) reference point. ForCES=20 protocol is exclusively used for all communication across the = Fp=20 reference point. =20 =20 4.2.1. Proximity and Interconnect between CEs and FEs=20 =20 The ForCES Working Group has made a conscious decision that the = first version of ForCES will not be designed to support=20 configurations where the CE and FE are located arbitrarily in = the=20 network. In particular, ForCES is intended for "very close" = CE/FE=20 localities in IP networks, as defined by ForCES Applicability=20 Statement ([FORCES-APP]). Very Close localities consist of = control=20 and forwarding elements that either are components in the same=20 physical box, or are separated at most by one local network = hop. =20 =20 CEs and FEs can be connected by a variety of interconnect=20 technologies, including Ethernet connections, backplanes, ATM = (cell)=20 fabrics, etc. ForCES should be able to support each of these=20 interconnects (see [FORCES-REQ] Section 5, requirement #1). = ForCES=20 will make use of an existing RFC2914 compliant L4 protocol with = adequate reliability, security and congestion control (e.g. = TCP,=20 SCTP) for transport purposes.=20 =20 4.2.2. Association Establishment=20 =20 As an example, figure 9 shows some of the message exchange that = need=20 to happen before the association between CE and FE is fully=20 established. Typically, FE would need to inform the CE of its = own=20 capability and its topology in relation to other FEs. The = capability=20 of FE is represented by FE model, described in another separate = document. The model would allow FE to describe what kind of = packet=20 processing functions it contains, in what order these = processing=20 happen, what kind of configurable parameters it allows, what=20 statistics it collects and what events it might throw, etc. = Once=20 such information is available to CE, CE sends all necessary=20 configuration to FE so that FE can start receiving and = processing=20 packets. For example, CE might need to send a snapshot of the=20 current routing table to FE so that FE can start routing = packets=20 correctly. Once FE starts accepting packets for processing, we = say=20 the association of this FE with its CE is now established. From = then=20 on, CE and FE enter steady-state communication as described = next.=20 =20 =20 FE CE=20 | |=20 |(Hello, are you there?)|=20 1|<----------------------|=20 =20 Yang, et. al. Expires April 2003 [Page 14] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 | |=20 |(Yes. let me join the NE please.) =20 2|---------------------->|=20 | |=20 |(Security exchange.) |=20 3|<--------------------->|=20 | |=20 |(What kind of FE are you? -- capability query)=20 4|<----------------------|=20 | |=20 |(Here is my FE functions/state: use model to = describe)=20 5|---------------------->|=20 | |=20 |(How are you connected with others? -- topology = query)=20 6|<----------------------|=20 | |=20 |(Here is the topology info)=20 7|---------------------->|=20 | |=20 |(Config for FE initialization, e.g. routing = table)=20 8|<----------------------|=20 | |=20 |(I am ready to go. Shall I?) =20 9|---------------------->|=20 | |=20 |(Go ahead!) |=20 10|<----------------------|=20 | |=20 =20 Figure 9. Example of message exchange between CE and FE=20 over Fp to establish NE association=20 =20 =20 4.2.3. Steady-state Communication=20 =20 Once an association is established between the CE and the FE, = the=20 ForCES protocol is used by the CE and the FE over Fp reference = point=20 to exchange information to facilitate packet processing. =20 =20 FE CE=20 | |=20 |(Add these new routes.)|=20 1|<----------------------|=20 | |=20 |(Successful.) |=20 2|---------------------->|=20 | |=20 | |=20 |(Query some stats.) |=20 1|<----------------------|=20 | |=20 |(Reply with stats collected.) =20 2|---------------------->|=20 =20 Yang, et. al. Expires April 2003 [Page 15] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 | |=20 | |=20 |(My port is down, with port #.) = =20 1|---------------------->|=20 | |=20 |(Route to this port instead...) =20 2|<----------------------|=20 | |=20 | |=20 =20 Figure 10. Examples of message exchange between CE and FE = over Fp during steady-state communication=20 =20 Based on the information acquired through CEs' control = processing,=20 CEs will frequently need to manipulate the packet-forwarding=20 behaviors of their FE(s) by sending instructions to FEs. For=20 example, Figure 10 shows message exchange examples in which CE = sends=20 new routes to FE so that FE can add them to its routing table. = CE=20 can also query FE for statistics collected by FE and FE can = also=20 notify CE of some important events, like interface up and down, = etc.=20 =20 4.2.4. Data Packets across Fp reference point=20 =20 =20 =20 --------------------- ----------------------=20 | | | |=20 | +--------+ | | +--------+ |=20 | |CE(OSPF)| | | |CE(OSPF)| |=20 | +--------+ | | +--------+ |=20 | | | | ^ |=20 | |Fp | | |Fp |=20 | v | | | |=20 | +--------+ | | +--------+ |=20 | | FE | | | | FE | |=20 | +--------+ | | +--------+ |=20 | | | | ^ |=20 | Router | | | Router | |=20 | A | | | B | |=20 ---------+----------- -----------+----------=20 v ^=20 | |=20 | |=20 ------------------->--------------- =20 =20 Figure 11. Example to show data packet flow between two = NEs.=20 =20 Control plane protocol packets (such as RIP, OSPF messages)=20 addressed to any of NE's interfaces are typically redirected by = the=20 receiving FE to its CE, and CE may originate packets and have = its FE=20 deliver them to other NEs. Therefore, ForCES protocol over Fp = not=20 only transports the ForCES protocol messages between CEs and = FEs,=20 but also encapsulates the data packets from control plane = protocols.=20 =20 Yang, et. al. Expires April 2003 [Page 16] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 Moreover, one FE may be controlled by multiple CEs for = distributed=20 control. In this configuration, the control protocols supported = by=20 the FORCES NEs may spread across multiple CEs. For example, one = CE=20 may support routing protocols like OSPF and BGP, while = signaling and=20 admission control protocol like RSVP is supported in another = CE. FEs=20 are configured to recognize and filter these protocol packets = and=20 forward them to the corresponding CE. =20 =20 Figure 11 shows one example of how the OSPF packets originated = by=20 router A are passed to router B. In this example, ForCES = protocol is=20 used to transport the packets from CE to FE inside router A, = and=20 then from FE to CE inside router B. In light of the fact that = ForCES=20 protocol is responsible to transport both the control messages = and=20 the data packets between CE and FE over Fp reference point, it = is=20 possible to use either a single protocol or multiple protocols = to=20 achieve that.=20 =20 4.2.5. Proxy FE=20 =20 In the case where a physical FE cannot implement (e.g., due to = the=20 lack of a general purpose CPU) the ForCES protocol directly, a = proxy=20 FE can be used in the middle of Fp reference point. This allows = the=20 CE communicate to the physical FE via the proxy by using = ForCES,=20 while the proxy manipulates the physical FE using some = intermediary=20 form of communication (e.g., a non-ForCES protocol or DMA). In = such=20 an implementation, the combination of the proxy and the = physical FE=20 becomes one logical FE entity. =20 =20 4.3. Association Re-establishment=20 =20 FEs and CEs may join and leave NEs dynamically (see = [FORCES-REQ]=20 Section 5, requirements #12). When a FE or CE leaves the NE, = the=20 association with the NE is broken. If the leaving party rejoins = a NE=20 later, to re-establish the association, it may or may not need = to=20 re-enter the pre-association phase. Loss of association can = also=20 happen unexpectedly due to loss of connection between the CE = and the=20 FE. Therefore, the framework allows the bi-directional = transition=20 between these two phases, but the ForCES protocol is only = applicable=20 for the post-association phase. However, the protocol should = provide=20 mechanisms to support association re-establishment (see = [FORCES-REQ]=20 Section 5, requirement #7).=20 =20 The example in Figure 5 is used to illustrate what happens when = the=20 association is broken and later re-established again. Section = 4.2=20 already explains what happens if CE1 fails and how CE2 can take = over. If no CE redundancy is provided, at the association=20 establishment phase FEs need to be told what to do in the case = of CE=20 failure. FEs may be told to stop packet processing all together = if=20 its CE fails. Or, FEs may be told to continue forwarding = packets for=20 a limited time even in the face of CE failure. No matter what = the=20 instruction is, it needs to be part of the configuration when = the=20 association is established. =20 =20 =20 Yang, et. al. Expires April 2003 [Page 17] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 In the same example in Figure 5, assuming CE1 is the working CE = for=20 the moment, what would happen if it is one of the FEs, say FE1, = leaves the NE temporarily? FE1 may voluntarily decide to leave = the=20 association. Or, FE1 may stop functioning simply due to = unexpected=20 failure. In former case, CE1 receives a "leave-association = request"=20 from FE1. In the latter, CE1 detects the failure of FE1 by some = other mean. In both cases, CE1 keeps a note of such event from = FE1=20 while continue commanding FE2. When FE1 decides to rejoin = again, or=20 when it is back up again from the failure, FE1 needs to = re-discover=20 its master (CE). This can be achieved by several means. It may = re- enter the pre-association phase and get that information from = its FE=20 manager. It may retrieve the previous CE information from its = cache,=20 if it can validate the information freshness. Once it discovers = its=20 CE, it starts message exchange with CE to re-establish the=20 association just as outlined in Figure 9, with the possible=20 exception that it might be able to bypass the transport of the=20 complete initialization information. Suppose that FE1 still has = its=20 routing table and other state information from the last = association,=20 instead of sending all the information again from scratch, it = can=20 choose to use more efficient mechanism to re-sync up the state = with=20 its CE. For example, a checksum of the state might give a quick = indication of whether or not the state is in-sync with its CE. = By=20 comparing its state with CE first, it sends information update = only=20 if it is needed.=20 =20 =20 5. Applicability to RFC1812 =20 =20 [FORCES-REQ] Section 5, requirement #9 dictates that "All = proposed=20 ForCES architecture MUST explain how that architecture supports = all=20 of the router functions as defined in [RFC1812]." RFC1812 = discusses=20 many important requirements for IPv4 routers from the link = layer to=20 the application layer. This section addresses the relevant=20 requirements in RFC1812 for implementing IPv4 routers based on=20 ForCES architecture and explain how ForCES satisfies these=20 requirements.=20 =20 5.1. General Router Requirements=20 =20 Routers have at least two or more logical interfaces. When CEs = and=20 FEs are separated by ForCES within a single NE, some additional = interfaces are needed for intra-NE communications. Figure 12 = shows=20 an example to illustrate that. This NE contains one CE and two = FEs.=20 Each FE has four interfaces; two of them are used for receiving = and=20 transmitting packets to the external world, while the other two = are=20 for intra-NE connections. CE has two logical interfaces #9 and = #10,=20 connected to interfaces #3 and #6 from FE1 and FE2, = respectively.=20 Interface #4 and #5 are connected for FE1-FE2 communication.=20 Therefore, this router NE provides four external interfaces = (#1, 2,=20 7 and 8). =20 =20 IPv4 routers must implement IP to support its packet forwarding = function, which is driven by its FIB (Forwarding Information = Base).=20 =20 Yang, et. al. Expires April 2003 [Page 18] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 This Internet layer forwarding (see [RFC1812] Section 5)=20 functionality naturally belongs to FEs in the ForCES = architecture. =20 =20 ---------------------------------=20 | router NE |=20 | ----------- ----------- |=20 | | FE1 | | FE2 | |=20 | ----------- ----------- |=20 | 1| 2| 3| 4| 5| 6| 7| 8| |=20 | | | | | | | | | |=20 | | | | +----+ | | | |=20 | | | | | | | |=20 | | | 9| 10| | | |=20 | | | -------------- | | |=20 | | | | CE | | | |=20 | | | -------------- | | |=20 | | | | | |=20 -----+--+----------------+--+----=20 | | | | =20 | | | | =20 =20 Figure 12. A router NE example with four = interfaces.=20 =20 =20 A router may implement transport layer protocols (like TCP and = UDP)=20 that are required to support application layer protocols (see=20 [RFC1812] Section 6). One important class of application = protocols=20 is routing protocols (see [RFC1812] Section 7). In ForCES=20 architecture, routing protocols are naturally implemented by = CEs.=20 Routing protocols require routers communicate with each other. = This=20 communication between CEs in different routers is supported in=20 ForCES by FEs' ability to redirect data packets addressed to = routers=20 (i.e., NEs) and CEs' ability to originate packets and have them = delivered by their FEs. This communication occurs across Fp=20 reference point inside each router and between neighboring = routers'=20 external interfaces, as illustrated in Figure 11. =20 =20 5.2.Link Layer=20 =20 Since FEs own all the external interfaces for the router, FEs = need=20 to conform to the link layer requirements in RFC1812. Arguably, = ARP=20 support may be implemented in either CEs or FEs. As we will see = later, a number of behaviors that RFC1812 mandates fall into = this=20 category -- they may be performed by the FE and may be = performed by=20 the CE. A general guideline is needed to ensure = interoperability=20 between separated control and forwarding planes. The guideline = we=20 offer here is that in general CEs are required to be capable of = these kind of operations while FEs may or may not choose to=20 implement them. FE model should indicate its capabilities in = this=20 regard so that CEs can decide where these functions are = implemented.=20 =20 Interface parameters, including MTU, IP address, etc., must be=20 configurable by CEs via ForCES. CEs must be able to determine=20 =20 Yang, et. al. Expires April 2003 [Page 19] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 whether a physical interface in an FE is available to send = packets=20 or not. FEs must also inform CEs the status change of the = interfaces=20 (like link up/down) via ForCES.=20 =20 5.3.Internet Layer Protocols=20 =20 Both FEs and CEs must implement IP protocol and all mandatory=20 extensions as RFC1812 specified. CEs should implement IP = options=20 like source route and record route while FEs may choose to = implement=20 those as well. Timestamp option should be implemented by FEs to = insert the timestamp most accurately. FE must interpret the IP=20 options that it understands and preserve the rest unchanged for = use=20 by CEs. Both FEs and CEs might choose to silently discard = packets=20 without sending ICMP errors, but such events should be logged = and=20 counted. FEs may report statistics for such events to CEs via=20 ForCES. =20 =20 When multiple FEs are involved to process packets, the = appearance of=20 single NE must be strictly maintained. For example, = Time-To-Live=20 (TTL) must be decremented only once within a single NE. For = example,=20 it can be always decremented by the last FE with egress = function. =20 =20 FEs must receive and process normally any packets with a = broadcast=20 destination address or a multicast destination address that the = router has asked to receive. When IP multicast is supported in=20 routers, IGMP is implemented in CEs. CEs are also required of = ICMP=20 support, while it is optional for FEs to support ICMP. Such an=20 option can be communicated to CEs as part of the FE model.=20 Therefore, FEs can always rely upon CEs to send out ICMP error=20 messages, but FEs also have the option to generate ICMP error=20 messages themselves. =20 =20 5.4.Internet Layer Forwarding=20 =20 IP forwarding is implemented by FEs. When the routing table is=20 updated at CEs, ForCES is used to send the new route entries = from=20 CEs to FEs. Each FE has its own forwarding table and uses this = table=20 to direct packets to the next hop interface. =20 =20 Upon receiving IP packets, FE verifies the IP header and = processes=20 most of the IP options. Some options can't be processed until = the=20 routing decision has been made. Routing decision is made after=20 examining the destination IP address. If the destination = address=20 belongs to the router itself, the packets are filtered and = either=20 processed locally or forwarded to CE, depending upon the=20 instructions set-up by CE. Otherwise, FE determines the next = hop IP=20 address by looking up in its forwarding table. FE also = determines=20 the network interface it uses to send the packets. Sometimes FE = may=20 need to forward the packets to another FE before packets can be = forwarded out to the next hop. Right before packets are = forwarded=20 out to the next hop, FE decrements TTL by 1 and processes any = IP=20 options that cannot be processed before. FE performs any IP=20 fragmentation if necessary, determines link layer address = (e.g., by=20 =20 Yang, et. al. Expires April 2003 [Page 20] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 ARP), and encapsulates the IP datagram (or each of the = fragments=20 thereof) in an appropriate link layer frame and queues it for = output=20 on the interface selected.=20 =20 Other options mentioned in RFC1812 for IP forwarding may also = be=20 implemented at FEs, for example, packet filtering.=20 =20 FEs typically forward packets destined locally to CEs. FEs may = also=20 forward exceptional packets (packets that FEs don't know how to = handle) to CEs. CEs are required to handle packets forwarded by = FEs=20 for whatever different reasons. It might be necessary for = ForCES to=20 attach some meta-data with the packets to indicate the reasons = of=20 forwarding from FEs to CEs. Upon receiving packets with = meta-data=20 from FEs, CEs can decide to either process the packets = themselves,=20 or pass the packets to the upper layer protocols including = routing=20 and management protocols. If CEs are to process the packets by=20 themselves, CEs may choose to discard the packets, or modify = and re- send the packets. CEs may also originate new packets and = deliver=20 them to FEs for further forwarding. =20 =20 Any state change during router operation must also be handled=20 correctly according to RFC1812. For example, when an FE ceases=20 forwarding, the entire NE may continue forwarding packets, but = it=20 needs to stop advertising routes that are affected by the = failed FE. =20 =20 5.5.Transport Layer =20 =20 Transport layer is typically implemented at CEs to support = higher=20 layer application protocols like routing protocols. In = practice,=20 this means that most CEs implement both the Transmission = Control=20 Protocol (TCP) and the User Datagram Protocol (UDP).=20 =20 Both CEs and FEs need to implement ForCES protocol. If some = layer-4=20 transport is used to support ForCES, then both CEs and FEs need = to=20 implement the L4 transport and ForCES protocols. It is possible = that=20 all FEs inside an NE implements only one such protocol entity. = =20 5.6. Application Layer -- Routing Protocols=20 =20 Both interior routing protocols and exterior routing protocols = are=20 implemented on CEs. The routing packets originated by CEs are=20 forwarded to FEs for delivery. The results of such protocols = (like=20 routing table update) are communicated to FEs via ForCES.=20 =20 5.7. Application Layer -- Network Management Protocol=20 =20 RFC1812 also dictates "Routers MUST be manageable by SNMP." = (see=20 [RFC1157] Section 8) In general, for post-association phase, = most=20 external management tasks (including SNMP) should be done = through=20 interaction with the CE in order to support the appearance of a = single functional device. Therefore, it is recommended that = SNMP=20 management agent be implemented by CEs and the SNMP messages=20 received by FEs be redirected to their CEs. AgentX framework = defined=20 =20 Yang, et. al. Expires April 2003 [Page 21] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 in RFC2741 may be applied here such that CEs act in the role of = master agent to process SNMP protocol messages while FEs act in = the=20 role of subagent to provide access to the MIB objects residing = on=20 FEs. AgentX protocol messages between the master agent (CE) and = the=20 subagent (FE) are encapsulated and transported via ForCES, just = like=20 data packets from any other application layer protocols. =20 =20 6. Summary=20 =20 This document defines an architectural framework for ForCES. It = identifies the relevant components for a ForCES network = element,=20 including (one or more) FEs, (one or more) CEs, one optional FE = manager, and one optional CE manager. It also identifies the=20 interaction among these components and discusses all the major=20 reference points. It is important to point out that, among all = the=20 reference points, only the interface between CEs and FEs is = within=20 the scope of ForCES. ForCES alone may not be enough to support = all=20 desirablet NE configurations. However, we believe that ForCES = is the=20 most important element in realizing physical separation and=20 interoperability of CEs and FEs, and hence the first interface = that=20 ought to be standardized. Simple and useful configurations can = still=20 be implemented with only CE-FE interface being standardized, = e.g.,=20 single CE with full-meshed FEs and static configuration without = the=20 need for CE/FE managers.=20 =20 7. Security Considerations=20 =20 The security necessary across each reference point except Fp is = discussed throughout the document. In general, the physical=20 separation of two entities usually requires much stricter = security=20 measurement in place. For example, we pointed out in Section = 5.1=20 that authentication becomes necessary between CE manager and FE = manager, between CE and CE manager, between FE and FE manager = in=20 some configuration. The physical separation of CE and FE also=20 imposes serious security requirement for ForCES protocol. The=20 security requirements for reference point Fp (i.e., ForCES = protocol)=20 are discussed in detail in [FORCES-REQ] Section 8.=20 =20 8. Intellectual Property Right=20 =20 The authors are not aware of any intellectual property right = issues=20 pertaining to this document.=20 =20 9. References=20 =20 9.1. Normative References=20 =20 [RFC1157] J. Case, et. al., "A Simple Network Management = Protocol=20 (SNMP)", RFC1157, May 1990.=20 =20 [RFC1812] F. Baker, "Requirements for IP Version 4 Routers",=20 RFC1812, June 1995.=20 =20 =20 Yang, et. al. Expires April 2003 [Page 22] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 [RFC2741] M. Daniele, et. al., "Agent Extensibility (AgentX)=20 Protocol Version 1", RFC2741, Jan 2000.=20 =20 [RFC2914] S. Floyd, "Congestion Control Principles", RFC2914,=20 September 2000.=20 =20 [RFC3084] K. Chan, et. al., "COPS Usage for Policy Provisioning = (COPS-PR)", RFC3084, March 2001.=20 =20 9.2. Informative References=20 =20 [FORCES-REQ] T. Anderson, et. al., "Requirements for Separation = of=20 IP Control and Forwarding", work in progress, July 2002, = .=20 =20 [FORCES-APP] A. Crouch, et. al., "ForCES Applicability = Statement",=20 work in progress, June 2002, .=20 =20 10. Acknowledgments=20 =20 Joel M. Halpern gave us many insightful comments and = suggestions and=20 pointed out several major issues. Many of our colleagues and = people=20 in the ForCES mailing list also provided valuable feedback.=20 =20 11. Authors' Addresses=20 =20 Lily L. Yang=20 Intel Corp., MS JF3-206,=20 2111 NE 25th Avenue=20 Hillsboro, OR 97124 USA=20 Phone: +1 503 264 8813=20 Email: lily.l.yang@intel.com=20 =20 Ram Dantu=20 University of Texas Dallas=20 2601 North Flyod Road=20 Richardson Texas 75082=20 Phone: +1 972 883 4653=20 Email: ram.dantu@utdallas.edu=20 =20 Todd A. Anderson=20 Intel Corp.=20 2111 NE 25th Avenue=20 Hillsboro, OR 97124 USA=20 Phone: +1 503 712 1760=20 Email: todd.a.anderson@intel.com=20 =20 =20 =20 Yang, et. al. Expires April 2003 [Page 23] = =0C ------_=_NextPart_000_01C26FF1.753256F0-- From owner-forces@PEACH.EASE.LSOFT.COM Wed Oct 9 20:14:04 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA08231 for ; Wed, 9 Oct 2002 20:14:04 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <5.007631E0@cherry.ease.lsoft.com>; Wed, 9 Oct 2002 20:16:10 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 28626897 for FORCES@PEACH.EASE.LSOFT.COM; Wed, 9 Oct 2002 20:16:09 -0400 Received: from 192.55.52.25 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Wed, 9 Oct 2002 20:16:09 -0400 Received: from petasus.fm.intel.com (petasus.fm.intel.com [10.1.192.37]) by caduceus.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.51 2002/09/23 20:43:23 dmccart Exp $) with ESMTP id g9A0CNW12452 for ; Thu, 10 Oct 2002 00:12:23 GMT Received: from fmsmsxvs041.fm.intel.com (fmsmsxvs041.fm.intel.com [132.233.42.126]) by petasus.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.25 2002/09/23 20:43:13 dmccart Exp $) with SMTP id g9A0CRc24856 for ; Thu, 10 Oct 2002 00:12:27 GMT Received: from FMSMSX017.fm.intel.com ([132.233.42.196]) by fmsmsxvs041.fm.intel.com (NAVGW 2.5.2.11) with SMTP id M2002100917145400768 for ; Wed, 09 Oct 2002 17:14:54 -0700 Received: by fmsmsx017.fm.intel.com with Internet Mail Service (5.5.2653.19) id ; Wed, 9 Oct 2002 17:16:07 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain Approved-By: "Putzolu, David" Message-ID: Date: Wed, 9 Oct 2002 17:15:55 -0700 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: "Putzolu, David" Subject: ForCES Requirements Status Update, 10/09/2002 To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list All, Our area directors have provided feedback on the requirements draft - see below for a summary. The requirements design team is going to respond to the feedback and post a new revision for the WG to review. This new rev will have a brief last call comment period (one week), and then will be returned to the IESG for review for publication. Cheers, David & Patrick, co-chairs, ForCES WG --- AD Feedback --- Section 6.5 Requirement 5) Vendor-Specific Functions The ADs believe that the IESG will be very hesitant to approve this part of the draft as worded. This is due to a concern that vendors would simply use the extensibility mechanisms in the protocol to do everything in a proprietary fashion, including "common" functions. The ADs suggested rewording the section to talk about extensibility to support new, currently unknown functions. Make sure that it is clear that one is *not* compliant with ForCES if one controls all the standard/common functions in a proprietary manner. Section 7 Requirement 6) Reliability The ADs where confused by the mention of non-IP networks (hey, they are IETF ADs, IP is all they think of - just kidding!!!). Recommend breaking up the one long reliability paragraph into three paragraphs: Para 1: Talks about IP networks. Essentially the same content. Para 2: Elaborates what non-IP networks are (FastEth, GigE, cell fabric between control and forwarding elements). Para 3: Take the last sentence and expand on it to make clear that the ability to acknowledge the *content* and execution of messages, rather than just their delivery. I.e., TCP may guarantee delivery, but it tells you nothing about whether the forwarding entry add operation succeeded or not. It is the latter function that is required for all types of networks (IP and non-IP). Section 7 Requirement 9) Packet Redirection The ADs where concerned that as written this seems more like a requirement on the FEs/CEs rather than on the protocol. Suggest rewriting it to say that ForCES must define a way to deliver packets from FE's that are destined to the CE (i.e. RIP packets from other routers) and vice versa. This makes it a protocol requirement instead of a CE or FE requirement. --- Other Feedback --- Other feedback based on http://www.ietf.org/internet-drafts/draft-rfc-editor-rfc2223bis-02.txt * Looks like 58 or more lines per page. Reduce this to 55 lines per page as per section 3.1 item 3. * Table of contents should go directly after abstract per section 4. * References should appear before security considerations per section 4 * Make normative and informative references subsections rather than top level sections. --- end --- From owner-forces@PEACH.EASE.LSOFT.COM Thu Oct 10 09:54:53 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA04535 for ; Thu, 10 Oct 2002 09:54:52 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <1.00764BA8@cherry.ease.lsoft.com>; Thu, 10 Oct 2002 9:56:58 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 28682764 for FORCES@PEACH.EASE.LSOFT.COM; Thu, 10 Oct 2002 09:56:58 -0400 Received: from 65.194.140.130 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Thu, 10 Oct 2002 09:56:58 -0400 Received: by UNIWEST1 with Internet Mail Service (5.5.2653.19) id ; Thu, 10 Oct 2002 09:56:57 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Approved-By: "Syed, Jeelani" Message-ID: <49FF5C6DDBD8D311BBBD009027DE980C03FC300E@UNIWEST1> Date: Thu, 10 Oct 2002 09:55:46 -0400 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: "Syed, Jeelani" Subject: Re: Please consider last call on draft-ietf-forces-framework-02.t xt To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list Hi, Sometimes you may have to run protocol counterparts(agents) on FEs for protocols running on CEs to avoid packet latencies and increase scalability. This draft doesn't talk abt that, Is it because its too soon to talk abt it? regards, -jeelani.sm -----Original Message----- From: Yang, Lily L [mailto:lily.l.yang@intel.com] Sent: Wednesday, October 09, 2002 8:11 PM To: FORCES@PEACH.EASE.LSOFT.COM Subject: Please consider last call on draft-ietf-forces-framework-02.txt Hi, We just submitted a new version of the ForCES framework draft. Several people on the list have given us comments on the previous (ver01) draft and we've incorporated all into this version. Most of the change are minor, including: 1) fixing Fig 6 (a) to make it really full mesh 2) Moving the Abstract and Table of Contents to the front. 3) Other editorial changes to improve clarity throughout the draft A more significant modification is in section 5.7. "Application Layer -- Network Management Protocol". Per comments from T. Sridhar, when we talk about how SNMP is supported in ForCES, we suggest that "AgentX framework defined in RFC2741 may be applied here such that CEs act in the role of master agent to process SNMP protocol messages while FEs act in the role of subagent to provide access to the MIB objects residing on FEs. AgentX protocol messages between the master agent (CE) and the subagent (FE) are encapsulated and transported via ForCES, just like data packets from any other application layer protocols." Accordingly, a new reference to RFC2741 is also added in the "normative reference" section. As always, comments are welcome. The co-authors feel that this framework draft has been through several iterations and seems pretty stable now. So we would like the working group chairs to consider last call on this version. Thanks, Lily <> From owner-forces@PEACH.EASE.LSOFT.COM Thu Oct 10 10:08:13 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA05259 for ; Thu, 10 Oct 2002 10:08:12 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <11.00764D0C@cherry.ease.lsoft.com>; Thu, 10 Oct 2002 10:09:52 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 28683839 for FORCES@PEACH.EASE.LSOFT.COM; Thu, 10 Oct 2002 10:08:16 -0400 Received: from 194.196.100.238 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Thu, 10 Oct 2002 10:08:16 -0400 Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-5.de.ibm.com (8.12.3/8.12.3) with ESMTP id g9AE8BH3010524 for ; Thu, 10 Oct 2002 16:08:11 +0200 Received: from sihl.zurich.ibm.com (sihl.zurich.ibm.com [9.4.16.232]) by d12relay01.de.ibm.com (8.12.3/NCO/VER6.4) with SMTP id g9AE887o047084 for ; Thu, 10 Oct 2002 16:08:11 +0200 Received: from dhcp70-126.zurich.ibm.com by sihl.zurich.ibm.com (AIX 4.3/UCB 5.64/4.03) id AA25990 from ; Thu, 10 Oct 2002 16:08:07 +0200 User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.1) Gecko/20020823 Netscape/7.0 X-Accept-Language: en-us, en Mime-Version: 1.0 References: <49FF5C6DDBD8D311BBBD009027DE980C03FC300E@UNIWEST1> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Approved-By: Patrick Droz Message-ID: <3DA589C7.2010301@zurich.ibm.com> Date: Thu, 10 Oct 2002 16:08:07 +0200 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: Patrick Droz Organization: IBM Research Subject: Re: Please consider last call on draft-ietf-forces-framework-02.t xt To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list Content-Transfer-Encoding: 7bit Jeelani, there are some statments about this in the requirements draft. http://www.ietf.org/internet-drafts/draft-ietf-forces-requirements-06.txt Section 6.5. Point 8) Off-load Functions Regards, Patrick Syed, Jeelani wrote: > Hi, > > Sometimes you may have to run protocol counterparts(agents) on > FEs for protocols running on CEs to avoid packet latencies and > increase scalability. This draft doesn't talk abt that, > Is it because its too soon to talk abt it? > > regards, > > -jeelani.sm > > -----Original Message----- > From: Yang, Lily L [mailto:lily.l.yang@intel.com] > Sent: Wednesday, October 09, 2002 8:11 PM > To: FORCES@PEACH.EASE.LSOFT.COM > Subject: Please consider last call on draft-ietf-forces-framework-02.txt > > > Hi, > We just submitted a new version of the ForCES framework draft. > Several people on the list have given us comments on the previous (ver01) > draft and we've incorporated all into this version. > > Most of the change are minor, including: > 1) fixing Fig 6 (a) to make it really full mesh > 2) Moving the Abstract and Table of Contents to the front. > 3) Other editorial changes to improve clarity throughout the draft > > A more significant modification is in section 5.7. "Application Layer -- > Network Management Protocol". Per comments from T. Sridhar, when we talk > about how SNMP is supported in ForCES, we suggest that "AgentX framework > defined in RFC2741 may be applied here such that CEs act in the role of > master agent to process SNMP protocol messages while FEs act in the role of > subagent to provide access to the MIB objects residing on FEs. AgentX > protocol messages between the master agent (CE) and the subagent (FE) are > encapsulated and transported via ForCES, just like data packets from any > other application layer protocols." > > Accordingly, a new reference to RFC2741 is also added in the "normative > reference" section. > > As always, comments are welcome. > > The co-authors feel that this framework draft has been through several > iterations and seems pretty stable now. So we would like the working group > chairs to consider last call on this version. > > > Thanks, > Lily > > > <> > > -- Dr. Patrick Droz | dro@zurich.ibm.com IBM Zurich Research Laboratory | http://www.zurich.ibm.com/~dro Saumerstrasse 4 | Tel. +41-1-724-85-25 CH-8803 Rueschlikon/Switzerland | Fax. +41-1-724-85-78 From owner-forces@PEACH.EASE.LSOFT.COM Thu Oct 10 10:25:48 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA05782 for ; Thu, 10 Oct 2002 10:25:47 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <21.00764F5A@cherry.ease.lsoft.com>; Thu, 10 Oct 2002 10:27:45 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 28686165 for FORCES@PEACH.EASE.LSOFT.COM; Thu, 10 Oct 2002 10:27:34 -0400 Received: from 65.194.140.130 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Thu, 10 Oct 2002 10:27:28 -0400 Received: by UNIWEST1 with Internet Mail Service (5.5.2653.19) id ; Thu, 10 Oct 2002 10:27:28 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Approved-By: "Syed, Jeelani" Message-ID: <49FF5C6DDBD8D311BBBD009027DE980C03FC300F@UNIWEST1> Date: Thu, 10 Oct 2002 10:26:40 -0400 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: "Syed, Jeelani" Subject: Re: Please consider last call on draft-ietf-forces-framew ork-02.t xt To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list Yep that's exactly what i was gettin at, Thanx Patrick. Don't you think that the section "5.6. Application Layer -- Routing Protocols" in framework draft is missing or ignoring that requirement. Just a concern. Regards -jeelani.sm -----Original Message----- From: Patrick Droz [mailto:dro@zurich.ibm.com] Sent: Thursday, October 10, 2002 10:08 AM To: FORCES@PEACH.EASE.LSOFT.COM Subject: Re: Please consider last call on draft-ietf-forces-framework-02.t xt Jeelani, there are some statments about this in the requirements draft. http://www.ietf.org/internet-drafts/draft-ietf-forces-requirements-06.txt Section 6.5. Point 8) Off-load Functions Regards, Patrick Syed, Jeelani wrote: > Hi, > > Sometimes you may have to run protocol counterparts(agents) on > FEs for protocols running on CEs to avoid packet latencies and > increase scalability. This draft doesn't talk abt that, > Is it because its too soon to talk abt it? > > regards, > > -jeelani.sm > > -----Original Message----- > From: Yang, Lily L [mailto:lily.l.yang@intel.com] > Sent: Wednesday, October 09, 2002 8:11 PM > To: FORCES@PEACH.EASE.LSOFT.COM > Subject: Please consider last call on draft-ietf-forces-framework-02.txt > > > Hi, > We just submitted a new version of the ForCES framework draft. > Several people on the list have given us comments on the previous (ver01) > draft and we've incorporated all into this version. > > Most of the change are minor, including: > 1) fixing Fig 6 (a) to make it really full mesh > 2) Moving the Abstract and Table of Contents to the front. > 3) Other editorial changes to improve clarity throughout the draft > > A more significant modification is in section 5.7. "Application Layer -- > Network Management Protocol". Per comments from T. Sridhar, when we talk > about how SNMP is supported in ForCES, we suggest that "AgentX framework > defined in RFC2741 may be applied here such that CEs act in the role of > master agent to process SNMP protocol messages while FEs act in the role of > subagent to provide access to the MIB objects residing on FEs. AgentX > protocol messages between the master agent (CE) and the subagent (FE) are > encapsulated and transported via ForCES, just like data packets from any > other application layer protocols." > > Accordingly, a new reference to RFC2741 is also added in the "normative > reference" section. > > As always, comments are welcome. > > The co-authors feel that this framework draft has been through several > iterations and seems pretty stable now. So we would like the working group > chairs to consider last call on this version. > > > Thanks, > Lily > > > <> > > -- Dr. Patrick Droz | dro@zurich.ibm.com IBM Zurich Research Laboratory | http://www.zurich.ibm.com/~dro Saumerstrasse 4 | Tel. +41-1-724-85-25 CH-8803 Rueschlikon/Switzerland | Fax. +41-1-724-85-78 From owner-forces@PEACH.EASE.LSOFT.COM Thu Oct 10 12:27:02 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA13915 for ; Thu, 10 Oct 2002 12:27:01 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <21.0076545B@cherry.ease.lsoft.com>; Thu, 10 Oct 2002 12:29:06 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 28693762 for FORCES@PEACH.EASE.LSOFT.COM; Thu, 10 Oct 2002 12:29:05 -0400 Received: from 143.182.124.5 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Thu, 10 Oct 2002 12:29:05 -0400 Received: from fmsmsxvs043.fm.intel.com (fmsmsxvs043.fm.intel.com [132.233.42.129]) by petasus.ch.intel.com (8.11.6/8.11.6/d: solo.mc,v 1.46 2002/09/23 20:41:22 dmccart Exp $) with SMTP id g9AGUdi12897 for ; Thu, 10 Oct 2002 16:30:39 GMT Received: from fmsmsx26.fm.intel.com ([132.233.42.26]) by fmsmsxvs043.fm.intel.com (NAVGW 2.5.2.11) with SMTP id M2002101009272410201 for ; Thu, 10 Oct 2002 09:27:24 -0700 Received: by fmsmsx26.fm.intel.com with Internet Mail Service (5.5.2653.19) id ; Thu, 10 Oct 2002 09:29:04 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Approved-By: "Yang, Lily L" Message-ID: Date: Thu, 10 Oct 2002 09:29:03 -0700 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: "Yang, Lily L" Subject: Re: Please consider last call on draft-ietf-forces-framew ork-02.t xt To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list Jeelani -- That is a very good point. I have been thinking about it lately. The concept and practice of "off-loading CE functions onto FE" or "split control plane" challenge ForCES's way of defining FE and CE -- our terminology has been that FE and CE are both "logical" entities -- but what really is the boundary here? Is it logical boundary or physical boundary? Our goal is to strive for physical sepration. So I think it is actually the physical boundary that defines CE and FE. Maybe we should revise the definition to reflect this. What do people think? I was keenly aware of the fact that this was not addressed in the framework. I was not sure where this should be addressed. I think you just point out a good place to me. I will get that fixed soon. Thank you for your comment, Lily > -----Original Message----- > From: Syed, Jeelani [mailto:jsyed@juniper.net] > Sent: Thursday, October 10, 2002 7:27 AM > To: FORCES@PEACH.EASE.LSOFT.COM > Subject: Re: Please consider last call on draft-ietf-forces-framew > ork-02.t xt > > > Yep that's exactly what i was gettin at, Thanx Patrick. > Don't you think that the section "5.6. Application Layer -- Routing > Protocols" > in framework draft is missing or ignoring that requirement. > Just a concern. > > Regards > -jeelani.sm > > -----Original Message----- > From: Patrick Droz [mailto:dro@zurich.ibm.com] > Sent: Thursday, October 10, 2002 10:08 AM > To: FORCES@PEACH.EASE.LSOFT.COM > Subject: Re: Please consider last call on > draft-ietf-forces-framework-02.t xt > > > Jeelani, > > there are some statments about this in the requirements > draft. > http://www.ietf.org/internet-drafts/draft-ietf-forces-requirem > ents-06.txt > Section 6.5. Point 8) Off-load Functions > > Regards, > Patrick > > > Syed, Jeelani wrote: > > Hi, > > > > Sometimes you may have to run protocol counterparts(agents) on > > FEs for protocols running on CEs to avoid packet latencies and > > increase scalability. This draft doesn't talk abt that, > > Is it because its too soon to talk abt it? > > > > regards, > > > > -jeelani.sm > > > > -----Original Message----- > > From: Yang, Lily L [mailto:lily.l.yang@intel.com] > > Sent: Wednesday, October 09, 2002 8:11 PM > > To: FORCES@PEACH.EASE.LSOFT.COM > > Subject: Please consider last call on > draft-ietf-forces-framework-02.txt > > > > > > Hi, > > We just submitted a new version of the ForCES framework draft. > > Several people on the list have given us comments on the > previous (ver01) > > draft and we've incorporated all into this version. > > > > Most of the change are minor, including: > > 1) fixing Fig 6 (a) to make it really full mesh > > 2) Moving the Abstract and Table of Contents to the front. > > 3) Other editorial changes to improve clarity > throughout the draft > > > > A more significant modification is in section 5.7. > "Application Layer -- > > Network Management Protocol". Per comments from T. Sridhar, > when we talk > > about how SNMP is supported in ForCES, we suggest that > "AgentX framework > > defined in RFC2741 may be applied here such that CEs act in > the role of > > master agent to process SNMP protocol messages while FEs > act in the role > of > > subagent to provide access to the MIB objects residing on > FEs. AgentX > > protocol messages between the master agent (CE) and the > subagent (FE) are > > encapsulated and transported via ForCES, just like data > packets from any > > other application layer protocols." > > > > Accordingly, a new reference to RFC2741 is also added in > the "normative > > reference" section. > > > > As always, comments are welcome. > > > > The co-authors feel that this framework draft has been > through several > > iterations and seems pretty stable now. So we would like > the working group > > chairs to consider last call on this version. > > > > > > Thanks, > > Lily > > > > > > <> > > > > > > > -- > Dr. Patrick Droz | dro@zurich.ibm.com > IBM Zurich Research Laboratory | > http://www.zurich.ibm.com/~dro > Saumerstrasse 4 | Tel. +41-1-724-85-25 CH-8803 > Rueschlikon/Switzerland | Fax. +41-1-724-85-78 > From owner-forces@PEACH.EASE.LSOFT.COM Thu Oct 10 15:24:28 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA20625 for ; Thu, 10 Oct 2002 15:24:26 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <17.007658A3@cherry.ease.lsoft.com>; Thu, 10 Oct 2002 15:26:33 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 28707851 for FORCES@PEACH.EASE.LSOFT.COM; Thu, 10 Oct 2002 15:26:32 -0400 Received: from 63.78.179.217 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Thu, 10 Oct 2002 15:26:32 -0400 Received: from davir02nok.americas.nokia.com (davir02nok.americas.nokia.com [172.18.242.85]) by mgw-dax2.ext.nokia.com (Switch-2.2.1/Switch-2.2.0) with ESMTP id g9AJRAM17444 for ; Thu, 10 Oct 2002 14:27:11 -0500 (CDT) Received: from daebh001.NOE.Nokia.com (unverified) by davir02nok.americas.nokia.com (Content Technologies SMTPRS 4.2.5) with ESMTP id for ; Thu, 10 Oct 2002 14:26:29 -0500 Received: from bsebe001.NOE.Nokia.com ([172.19.160.13]) by daebh001.NOE.Nokia.com with Microsoft SMTPSVC(5.0.2195.5329); Thu, 10 Oct 2002 12:25:42 -0700 X-MimeOLE: Produced By Microsoft Exchange V6.0.6249.0 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Thread-Topic: Please consider last call on draft-ietf-forces-framew ork-02.t xt Thread-Index: AcJweiphYm4vQl+lRe+pZ1+gvwMgLQAFqPkA X-OriginalArrivalTime: 10 Oct 2002 19:25:42.0769 (UTC) FILETIME=[D3059610:01C27092] Approved-By: ram.gopal@NOKIA.COM Message-ID: Date: Thu, 10 Oct 2002 15:25:41 -0400 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: "(Ram Gopal )" Subject: Re: Please consider last call on draft-ietf-forces-framew ork-02.t xt To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by ietf.org id PAA20625 Greetings, In my view off-loading part of CE's functions to line card is considered as part of CE and not part of FE. In this case a line card is capable of performing both forwarding functions and control functions and provides both CE and FE logical endpoint. In other words, the line card has capability to act as CE as well as FE. FE represents forwarding plane entities only. The logical separation of FE and CE should not mixed with physical processing capability of hardware. IMHO it is should treated as another CE endpoint contained in the line card rather than a function inside a FE. Regards Ramg > -----Original Message----- > From: ext Yang, Lily L [mailto:lily.l.yang@intel.com] > Sent: Thursday, October 10, 2002 12:29 PM > To: FORCES@PEACH.EASE.LSOFT.COM > Subject: Re: Please consider last call on draft-ietf-forces-framew > ork-02.t xt > > > Jeelani -- > That is a very good point. I have been thinking about it > lately. The concept > and practice of "off-loading CE functions onto FE" or "split > control plane" > challenge ForCES's way of defining FE and CE -- our > terminology has been > that FE and CE are both "logical" entities -- but what really is the > boundary here? Is it logical boundary or physical boundary? > Our goal is to > strive for physical sepration. So I think it is actually the physical > boundary that defines CE and FE. Maybe we should revise the > definition to > reflect this. What do people think? > > I was keenly aware of the fact that this was not addressed in > the framework. > I was not sure where this should be addressed. I think you > just point out a > good place to me. > I will get that fixed soon. > > Thank you for your comment, > Lily > > > -----Original Message----- > > From: Syed, Jeelani [mailto:jsyed@juniper.net] > > Sent: Thursday, October 10, 2002 7:27 AM > > To: FORCES@PEACH.EASE.LSOFT.COM > > Subject: Re: Please consider last call on draft-ietf-forces-framew > > ork-02.t xt > > > > > > Yep that's exactly what i was gettin at, Thanx Patrick. > > Don't you think that the section "5.6. Application Layer -- Routing > > Protocols" > > in framework draft is missing or ignoring that requirement. > > Just a concern. > > > > Regards > > -jeelani.sm > > > > -----Original Message----- > > From: Patrick Droz [mailto:dro@zurich.ibm.com] > > Sent: Thursday, October 10, 2002 10:08 AM > > To: FORCES@PEACH.EASE.LSOFT.COM > > Subject: Re: Please consider last call on > > draft-ietf-forces-framework-02.t xt > > > > > > Jeelani, > > > > there are some statments about this in the requirements > > draft. > > http://www.ietf.org/internet-drafts/draft-ietf-forces-requirem > > ents-06.txt > > Section 6.5. Point 8) Off-load Functions > > > > Regards, > > Patrick > > > > > > Syed, Jeelani wrote: > > > Hi, > > > > > > Sometimes you may have to run protocol counterparts(agents) on > > > FEs for protocols running on CEs to avoid packet latencies and > > > increase scalability. This draft doesn't talk abt that, > > > Is it because its too soon to talk abt it? > > > > > > regards, > > > > > > -jeelani.sm > > > > > > -----Original Message----- > > > From: Yang, Lily L [mailto:lily.l.yang@intel.com] > > > Sent: Wednesday, October 09, 2002 8:11 PM > > > To: FORCES@PEACH.EASE.LSOFT.COM > > > Subject: Please consider last call on > > draft-ietf-forces-framework-02.txt > > > > > > > > > Hi, > > > We just submitted a new version of the ForCES framework draft. > > > Several people on the list have given us comments on the > > previous (ver01) > > > draft and we've incorporated all into this version. > > > > > > Most of the change are minor, including: > > > 1) fixing Fig 6 (a) to make it really full mesh > > > 2) Moving the Abstract and Table of Contents to the front. > > > 3) Other editorial changes to improve clarity > > throughout the draft > > > > > > A more significant modification is in section 5.7. > > "Application Layer -- > > > Network Management Protocol". Per comments from T. Sridhar, > > when we talk > > > about how SNMP is supported in ForCES, we suggest that > > "AgentX framework > > > defined in RFC2741 may be applied here such that CEs act in > > the role of > > > master agent to process SNMP protocol messages while FEs > > act in the role > > of > > > subagent to provide access to the MIB objects residing on > > FEs. AgentX > > > protocol messages between the master agent (CE) and the > > subagent (FE) are > > > encapsulated and transported via ForCES, just like data > > packets from any > > > other application layer protocols." > > > > > > Accordingly, a new reference to RFC2741 is also added in > > the "normative > > > reference" section. > > > > > > As always, comments are welcome. > > > > > > The co-authors feel that this framework draft has been > > through several > > > iterations and seems pretty stable now. So we would like > > the working group > > > chairs to consider last call on this version. > > > > > > > > > Thanks, > > > Lily > > > > > > > > > <> > > > > > > > > > > > > -- > > Dr. Patrick Droz | dro@zurich.ibm.com > > IBM Zurich Research Laboratory | > > http://www.zurich.ibm.com/~dro > > Saumerstrasse 4 | Tel. > +41-1-724-85-25 CH-8803 > > Rueschlikon/Switzerland | Fax. +41-1-724-85-78 > > > From owner-forces@PEACH.EASE.LSOFT.COM Thu Oct 10 15:38:17 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA21153 for ; Thu, 10 Oct 2002 15:38:16 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <11.007658DD@cherry.ease.lsoft.com>; Thu, 10 Oct 2002 15:40:23 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 28709243 for FORCES@PEACH.EASE.LSOFT.COM; Thu, 10 Oct 2002 15:40:20 -0400 Received: from 65.194.140.130 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Thu, 10 Oct 2002 15:40:20 -0400 Received: by UNIWEST1 with Internet Mail Service (5.5.2653.19) id ; Thu, 10 Oct 2002 15:40:20 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Approved-By: "Syed, Jeelani" Message-ID: <49FF5C6DDBD8D311BBBD009027DE980C03FC3013@UNIWEST1> Date: Thu, 10 Oct 2002 15:39:32 -0400 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: "Syed, Jeelani" Subject: Re: Please consider last call on draft-ietf-forces-framew ork-02.t xt To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list Yes I agree, but there is a whole set of interactions between CE and FE's "CE logical end point" that needs to be standardized in order to acheive the phyiscal seperation of CE & FE. Regards -jeelani.sm -----Original Message----- From: (Ram Gopal ) [mailto:ram.gopal@nokia.com] Sent: Thursday, October 10, 2002 3:26 PM To: FORCES@PEACH.EASE.LSOFT.COM Subject: Re: Please consider last call on draft-ietf-forces-framew ork-02.t xt Greetings, In my view off-loading part of CE's functions to line card is considered as part of CE and not part of FE. In this case a line card is capable of performing both forwarding functions and control functions and provides both CE and FE logical endpoint. In other words, the line card has capability to act as CE as well as FE. FE represents forwarding plane entities only. The logical separation of FE and CE should not mixed with physical processing capability of hardware. IMHO it is should treated as another CE endpoint contained in the line card rather than a function inside a FE. Regards Ramg > -----Original Message----- > From: ext Yang, Lily L [mailto:lily.l.yang@intel.com] > Sent: Thursday, October 10, 2002 12:29 PM > To: FORCES@PEACH.EASE.LSOFT.COM > Subject: Re: Please consider last call on draft-ietf-forces-framew > ork-02.t xt > > > Jeelani -- > That is a very good point. I have been thinking about it > lately. The concept > and practice of "off-loading CE functions onto FE" or "split > control plane" > challenge ForCES's way of defining FE and CE -- our > terminology has been > that FE and CE are both "logical" entities -- but what really is the > boundary here? Is it logical boundary or physical boundary? > Our goal is to > strive for physical sepration. So I think it is actually the physical > boundary that defines CE and FE. Maybe we should revise the > definition to > reflect this. What do people think? > > I was keenly aware of the fact that this was not addressed in > the framework. > I was not sure where this should be addressed. I think you > just point out a > good place to me. > I will get that fixed soon. > > Thank you for your comment, > Lily > > > -----Original Message----- > > From: Syed, Jeelani [mailto:jsyed@juniper.net] > > Sent: Thursday, October 10, 2002 7:27 AM > > To: FORCES@PEACH.EASE.LSOFT.COM > > Subject: Re: Please consider last call on draft-ietf-forces-framew > > ork-02.t xt > > > > > > Yep that's exactly what i was gettin at, Thanx Patrick. > > Don't you think that the section "5.6. Application Layer -- Routing > > Protocols" > > in framework draft is missing or ignoring that requirement. > > Just a concern. > > > > Regards > > -jeelani.sm > > > > -----Original Message----- > > From: Patrick Droz [mailto:dro@zurich.ibm.com] > > Sent: Thursday, October 10, 2002 10:08 AM > > To: FORCES@PEACH.EASE.LSOFT.COM > > Subject: Re: Please consider last call on > > draft-ietf-forces-framework-02.t xt > > > > > > Jeelani, > > > > there are some statments about this in the requirements > > draft. > > http://www.ietf.org/internet-drafts/draft-ietf-forces-requirem > > ents-06.txt > > Section 6.5. Point 8) Off-load Functions > > > > Regards, > > Patrick > > > > > > Syed, Jeelani wrote: > > > Hi, > > > > > > Sometimes you may have to run protocol counterparts(agents) on > > > FEs for protocols running on CEs to avoid packet latencies and > > > increase scalability. This draft doesn't talk abt that, > > > Is it because its too soon to talk abt it? > > > > > > regards, > > > > > > -jeelani.sm > > > > > > -----Original Message----- > > > From: Yang, Lily L [mailto:lily.l.yang@intel.com] > > > Sent: Wednesday, October 09, 2002 8:11 PM > > > To: FORCES@PEACH.EASE.LSOFT.COM > > > Subject: Please consider last call on > > draft-ietf-forces-framework-02.txt > > > > > > > > > Hi, > > > We just submitted a new version of the ForCES framework draft. > > > Several people on the list have given us comments on the > > previous (ver01) > > > draft and we've incorporated all into this version. > > > > > > Most of the change are minor, including: > > > 1) fixing Fig 6 (a) to make it really full mesh > > > 2) Moving the Abstract and Table of Contents to the front. > > > 3) Other editorial changes to improve clarity > > throughout the draft > > > > > > A more significant modification is in section 5.7. > > "Application Layer -- > > > Network Management Protocol". Per comments from T. Sridhar, > > when we talk > > > about how SNMP is supported in ForCES, we suggest that > > "AgentX framework > > > defined in RFC2741 may be applied here such that CEs act in > > the role of > > > master agent to process SNMP protocol messages while FEs > > act in the role > > of > > > subagent to provide access to the MIB objects residing on > > FEs. AgentX > > > protocol messages between the master agent (CE) and the > > subagent (FE) are > > > encapsulated and transported via ForCES, just like data > > packets from any > > > other application layer protocols." > > > > > > Accordingly, a new reference to RFC2741 is also added in > > the "normative > > > reference" section. > > > > > > As always, comments are welcome. > > > > > > The co-authors feel that this framework draft has been > > through several > > > iterations and seems pretty stable now. So we would like > > the working group > > > chairs to consider last call on this version. > > > > > > > > > Thanks, > > > Lily > > > > > > > > > <> > > > > > > > > > > > > -- > > Dr. Patrick Droz | dro@zurich.ibm.com > > IBM Zurich Research Laboratory | > > http://www.zurich.ibm.com/~dro > > Saumerstrasse 4 | Tel. > +41-1-724-85-25 CH-8803 > > Rueschlikon/Switzerland | Fax. +41-1-724-85-78 > > > From owner-forces@PEACH.EASE.LSOFT.COM Thu Oct 10 15:44:11 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA21286 for ; Thu, 10 Oct 2002 15:44:10 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <20.00765890@cherry.ease.lsoft.com>; Thu, 10 Oct 2002 15:46:16 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 28709622 for FORCES@PEACH.EASE.LSOFT.COM; Thu, 10 Oct 2002 15:46:16 -0400 Received: from 66.21.69.202 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Thu, 10 Oct 2002 15:46:16 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Approved-By: Alex Vasquez Message-ID: Date: Thu, 10 Oct 2002 15:31:12 -0400 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: Alex Vasquez Subject: Re: Please consider last call on draft-ietf-forces-framew ork-02.t xt To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list All, I do not think we should be mixing logical entities with physical ones. There should not be off-loading CE functions onto FE, this contradicts current CE and FE definitions. CE and FE are logical entities with concrete responsibilities. I can see that one can off-load a CE function into a physical entity that can perform that CE functions. I can also see that the same physical entity performing FE functions. Both, CE and FE, running in the same physical entity should interact similarly as if they were running in different physical entities. Thoughts? Cheers, Alex > The information contained in this e-mail is LVL7 > Confidential. Any use except that authorized by LVL7 is > prohibited. If you get this in error, please notify sender > and delete this e-mail. > -----Original Message----- From: Yang, Lily L [mailto:lily.l.yang@intel.com] Sent: Thursday, October 10, 2002 12:29 PM To: FORCES@PEACH.EASE.LSOFT.COM Subject: Re: Please consider last call on draft-ietf-forces-framew ork-02.t xt Jeelani -- That is a very good point. I have been thinking about it lately. The concept and practice of "off-loading CE functions onto FE" or "split control plane" challenge ForCES's way of defining FE and CE -- our terminology has been that FE and CE are both "logical" entities -- but what really is the boundary here? Is it logical boundary or physical boundary? Our goal is to strive for physical sepration. So I think it is actually the physical boundary that defines CE and FE. Maybe we should revise the definition to reflect this. What do people think? I was keenly aware of the fact that this was not addressed in the framework. I was not sure where this should be addressed. I think you just point out a good place to me. I will get that fixed soon. Thank you for your comment, Lily > -----Original Message----- > From: Syed, Jeelani [mailto:jsyed@juniper.net] > Sent: Thursday, October 10, 2002 7:27 AM > To: FORCES@PEACH.EASE.LSOFT.COM > Subject: Re: Please consider last call on draft-ietf-forces-framew > ork-02.t xt > > > Yep that's exactly what i was gettin at, Thanx Patrick. > Don't you think that the section "5.6. Application Layer -- Routing > Protocols" > in framework draft is missing or ignoring that requirement. > Just a concern. > > Regards > -jeelani.sm > > -----Original Message----- > From: Patrick Droz [mailto:dro@zurich.ibm.com] > Sent: Thursday, October 10, 2002 10:08 AM > To: FORCES@PEACH.EASE.LSOFT.COM > Subject: Re: Please consider last call on > draft-ietf-forces-framework-02.t xt > > > Jeelani, > > there are some statments about this in the requirements > draft. > http://www.ietf.org/internet-drafts/draft-ietf-forces-requirem > ents-06.txt > Section 6.5. Point 8) Off-load Functions > > Regards, > Patrick > > > Syed, Jeelani wrote: > > Hi, > > > > Sometimes you may have to run protocol counterparts(agents) on > > FEs for protocols running on CEs to avoid packet latencies and > > increase scalability. This draft doesn't talk abt that, > > Is it because its too soon to talk abt it? > > > > regards, > > > > -jeelani.sm > > > > -----Original Message----- > > From: Yang, Lily L [mailto:lily.l.yang@intel.com] > > Sent: Wednesday, October 09, 2002 8:11 PM > > To: FORCES@PEACH.EASE.LSOFT.COM > > Subject: Please consider last call on > draft-ietf-forces-framework-02.txt > > > > > > Hi, > > We just submitted a new version of the ForCES framework draft. > > Several people on the list have given us comments on the > previous (ver01) > > draft and we've incorporated all into this version. > > > > Most of the change are minor, including: > > 1) fixing Fig 6 (a) to make it really full mesh > > 2) Moving the Abstract and Table of Contents to the front. > > 3) Other editorial changes to improve clarity > throughout the draft > > > > A more significant modification is in section 5.7. > "Application Layer -- > > Network Management Protocol". Per comments from T. Sridhar, > when we talk > > about how SNMP is supported in ForCES, we suggest that > "AgentX framework > > defined in RFC2741 may be applied here such that CEs act in > the role of > > master agent to process SNMP protocol messages while FEs > act in the role > of > > subagent to provide access to the MIB objects residing on > FEs. AgentX > > protocol messages between the master agent (CE) and the > subagent (FE) are > > encapsulated and transported via ForCES, just like data > packets from any > > other application layer protocols." > > > > Accordingly, a new reference to RFC2741 is also added in > the "normative > > reference" section. > > > > As always, comments are welcome. > > > > The co-authors feel that this framework draft has been > through several > > iterations and seems pretty stable now. So we would like > the working group > > chairs to consider last call on this version. > > > > > > Thanks, > > Lily > > > > > > <> > > > > > > > -- > Dr. Patrick Droz | dro@zurich.ibm.com > IBM Zurich Research Laboratory | > http://www.zurich.ibm.com/~dro > Saumerstrasse 4 | Tel. +41-1-724-85-25 CH-8803 > Rueschlikon/Switzerland | Fax. +41-1-724-85-78 > From owner-forces@PEACH.EASE.LSOFT.COM Thu Oct 10 15:50:31 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA21536 for ; Thu, 10 Oct 2002 15:50:30 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <19.007657F2@cherry.ease.lsoft.com>; Thu, 10 Oct 2002 15:52:37 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 28709954 for FORCES@PEACH.EASE.LSOFT.COM; Thu, 10 Oct 2002 15:52:36 -0400 Received: from 66.21.69.202 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Thu, 10 Oct 2002 15:52:36 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Approved-By: Alex Vasquez Message-ID: Date: Thu, 10 Oct 2002 15:37:32 -0400 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: Alex Vasquez Subject: Re: Please consider last call on draft-ietf-forces-framew ork-02.t xt To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list Jeelani, It seems to me that one should be able to use the ForCes messaging regardless where the CE and FE are physically located. Am I missing something? Cheers, Alex -----Original Message----- From: Syed, Jeelani [mailto:jsyed@juniper.net] Sent: Thursday, October 10, 2002 3:40 PM To: FORCES@PEACH.EASE.LSOFT.COM Subject: Re: Please consider last call on draft-ietf-forces-framew ork-02.t xt Yes I agree, but there is a whole set of interactions between CE and FE's "CE logical end point" that needs to be standardized in order to acheive the phyiscal seperation of CE & FE. Regards -jeelani.sm -----Original Message----- From: (Ram Gopal ) [mailto:ram.gopal@nokia.com] Sent: Thursday, October 10, 2002 3:26 PM To: FORCES@PEACH.EASE.LSOFT.COM Subject: Re: Please consider last call on draft-ietf-forces-framew ork-02.t xt Greetings, In my view off-loading part of CE's functions to line card is considered as part of CE and not part of FE. In this case a line card is capable of performing both forwarding functions and control functions and provides both CE and FE logical endpoint. In other words, the line card has capability to act as CE as well as FE. FE represents forwarding plane entities only. The logical separation of FE and CE should not mixed with physical processing capability of hardware. IMHO it is should treated as another CE endpoint contained in the line card rather than a function inside a FE. Regards Ramg > -----Original Message----- > From: ext Yang, Lily L [mailto:lily.l.yang@intel.com] > Sent: Thursday, October 10, 2002 12:29 PM > To: FORCES@PEACH.EASE.LSOFT.COM > Subject: Re: Please consider last call on draft-ietf-forces-framew > ork-02.t xt > > > Jeelani -- > That is a very good point. I have been thinking about it > lately. The concept > and practice of "off-loading CE functions onto FE" or "split > control plane" > challenge ForCES's way of defining FE and CE -- our > terminology has been > that FE and CE are both "logical" entities -- but what really is the > boundary here? Is it logical boundary or physical boundary? > Our goal is to > strive for physical sepration. So I think it is actually the physical > boundary that defines CE and FE. Maybe we should revise the > definition to > reflect this. What do people think? > > I was keenly aware of the fact that this was not addressed in > the framework. > I was not sure where this should be addressed. I think you > just point out a > good place to me. > I will get that fixed soon. > > Thank you for your comment, > Lily > > > -----Original Message----- > > From: Syed, Jeelani [mailto:jsyed@juniper.net] > > Sent: Thursday, October 10, 2002 7:27 AM > > To: FORCES@PEACH.EASE.LSOFT.COM > > Subject: Re: Please consider last call on draft-ietf-forces-framew > > ork-02.t xt > > > > > > Yep that's exactly what i was gettin at, Thanx Patrick. > > Don't you think that the section "5.6. Application Layer -- Routing > > Protocols" > > in framework draft is missing or ignoring that requirement. > > Just a concern. > > > > Regards > > -jeelani.sm > > > > -----Original Message----- > > From: Patrick Droz [mailto:dro@zurich.ibm.com] > > Sent: Thursday, October 10, 2002 10:08 AM > > To: FORCES@PEACH.EASE.LSOFT.COM > > Subject: Re: Please consider last call on > > draft-ietf-forces-framework-02.t xt > > > > > > Jeelani, > > > > there are some statments about this in the requirements > > draft. > > http://www.ietf.org/internet-drafts/draft-ietf-forces-requirem > > ents-06.txt > > Section 6.5. Point 8) Off-load Functions > > > > Regards, > > Patrick > > > > > > Syed, Jeelani wrote: > > > Hi, > > > > > > Sometimes you may have to run protocol counterparts(agents) on > > > FEs for protocols running on CEs to avoid packet latencies and > > > increase scalability. This draft doesn't talk abt that, > > > Is it because its too soon to talk abt it? > > > > > > regards, > > > > > > -jeelani.sm > > > > > > -----Original Message----- > > > From: Yang, Lily L [mailto:lily.l.yang@intel.com] > > > Sent: Wednesday, October 09, 2002 8:11 PM > > > To: FORCES@PEACH.EASE.LSOFT.COM > > > Subject: Please consider last call on > > draft-ietf-forces-framework-02.txt > > > > > > > > > Hi, > > > We just submitted a new version of the ForCES framework draft. > > > Several people on the list have given us comments on the > > previous (ver01) > > > draft and we've incorporated all into this version. > > > > > > Most of the change are minor, including: > > > 1) fixing Fig 6 (a) to make it really full mesh > > > 2) Moving the Abstract and Table of Contents to the front. > > > 3) Other editorial changes to improve clarity > > throughout the draft > > > > > > A more significant modification is in section 5.7. > > "Application Layer -- > > > Network Management Protocol". Per comments from T. Sridhar, > > when we talk > > > about how SNMP is supported in ForCES, we suggest that > > "AgentX framework > > > defined in RFC2741 may be applied here such that CEs act in > > the role of > > > master agent to process SNMP protocol messages while FEs > > act in the role > > of > > > subagent to provide access to the MIB objects residing on > > FEs. AgentX > > > protocol messages between the master agent (CE) and the > > subagent (FE) are > > > encapsulated and transported via ForCES, just like data > > packets from any > > > other application layer protocols." > > > > > > Accordingly, a new reference to RFC2741 is also added in > > the "normative > > > reference" section. > > > > > > As always, comments are welcome. > > > > > > The co-authors feel that this framework draft has been > > through several > > > iterations and seems pretty stable now. So we would like > > the working group > > > chairs to consider last call on this version. > > > > > > > > > Thanks, > > > Lily > > > > > > > > > <> > > > > > > > > > > > > -- > > Dr. Patrick Droz | dro@zurich.ibm.com > > IBM Zurich Research Laboratory | > > http://www.zurich.ibm.com/~dro > > Saumerstrasse 4 | Tel. > +41-1-724-85-25 CH-8803 > > Rueschlikon/Switzerland | Fax. +41-1-724-85-78 > > > From owner-forces@PEACH.EASE.LSOFT.COM Thu Oct 10 15:54:36 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA21632 for ; Thu, 10 Oct 2002 15:54:34 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <4.00765A6D@cherry.ease.lsoft.com>; Thu, 10 Oct 2002 15:56:41 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 28710125 for FORCES@PEACH.EASE.LSOFT.COM; Thu, 10 Oct 2002 15:56:41 -0400 Received: from 63.78.179.217 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Thu, 10 Oct 2002 15:56:40 -0400 Received: from davir04nok.americas.nokia.com (davir04nok.americas.nokia.com [172.18.242.87]) by mgw-dax2.ext.nokia.com (Switch-2.2.1/Switch-2.2.0) with ESMTP id g9AJvKM21821 for ; Thu, 10 Oct 2002 14:57:20 -0500 (CDT) Received: from daebh002.NOE.Nokia.com (unverified) by davir04nok.americas.nokia.com (Content Technologies SMTPRS 4.2.5) with ESMTP id for ; Thu, 10 Oct 2002 14:56:40 -0500 Received: from bsebe001.NOE.Nokia.com ([172.19.160.13]) by daebh002.NOE.Nokia.com with Microsoft SMTPSVC(5.0.2195.5329); Thu, 10 Oct 2002 14:56:28 -0500 X-MimeOLE: Produced By Microsoft Exchange V6.0.6249.0 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Thread-Topic: Please consider last call on draft-ietf-forces-framew ork-02.t xt Thread-Index: AcJwlahw+Ki+o6u7RDuNxPVaUUFAIQAALGXQ X-OriginalArrivalTime: 10 Oct 2002 19:56:28.0505 (UTC) FILETIME=[1F2A8C90:01C27097] Approved-By: ram.gopal@NOKIA.COM Message-ID: Date: Thu, 10 Oct 2002 15:56:27 -0400 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: "(Ram Gopal )" Subject: Re: Please consider last call on draft-ietf-forces-framew ork-02.t xt To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by ietf.org id PAA21632 Hello Jeelani.sm, Yes, the requirement and architecture document describes FE and CE with respect to logical endpoints and describe the functions. The interaction between CE and FE is going to be same whether they are in same card or on different card (physical separation). Regards Ramg > -----Original Message----- > From: ext Syed, Jeelani [mailto:jsyed@juniper.net] > Sent: Thursday, October 10, 2002 3:40 PM > To: FORCES@PEACH.EASE.LSOFT.COM > Subject: Re: Please consider last call on draft-ietf-forces-framew > ork-02.t xt > > > Yes I agree, but there is a whole set of interactions between > CE and FE's > "CE logical end point" that needs to > be standardized in order to acheive the phyiscal seperation > of CE & FE. > > Regards > > -jeelani.sm > > -----Original Message----- > From: (Ram Gopal ) [mailto:ram.gopal@nokia.com] > Sent: Thursday, October 10, 2002 3:26 PM > To: FORCES@PEACH.EASE.LSOFT.COM > Subject: Re: Please consider last call on draft-ietf-forces-framew > ork-02.t xt > > > Greetings, > > In my view off-loading part of CE's functions to line card > is considered > as part of CE > and not part of FE. In this case a line card is capable of > performing both > forwarding > functions and control functions and provides both CE and FE logical > endpoint. > In other words, the line card has capability to act as CE > as well as FE. > > FE represents forwarding plane entities only. The logical > separation of FE > and CE should not mixed > with physical processing capability of hardware. IMHO it is > should treated > as another CE endpoint > contained in the line card rather than a function inside a FE. > > Regards > Ramg > > > > -----Original Message----- > > From: ext Yang, Lily L [mailto:lily.l.yang@intel.com] > > Sent: Thursday, October 10, 2002 12:29 PM > > To: FORCES@PEACH.EASE.LSOFT.COM > > Subject: Re: Please consider last call on draft-ietf-forces-framew > > ork-02.t xt > > > > > > Jeelani -- > > That is a very good point. I have been thinking about it > > lately. The concept > > and practice of "off-loading CE functions onto FE" or "split > > control plane" > > challenge ForCES's way of defining FE and CE -- our > > terminology has been > > that FE and CE are both "logical" entities -- but what really is the > > boundary here? Is it logical boundary or physical boundary? > > Our goal is to > > strive for physical sepration. So I think it is actually > the physical > > boundary that defines CE and FE. Maybe we should revise the > > definition to > > reflect this. What do people think? > > > > I was keenly aware of the fact that this was not addressed in > > the framework. > > I was not sure where this should be addressed. I think you > > just point out a > > good place to me. > > I will get that fixed soon. > > > > Thank you for your comment, > > Lily > > > > > -----Original Message----- > > > From: Syed, Jeelani [mailto:jsyed@juniper.net] > > > Sent: Thursday, October 10, 2002 7:27 AM > > > To: FORCES@PEACH.EASE.LSOFT.COM > > > Subject: Re: Please consider last call on draft-ietf-forces-framew > > > ork-02.t xt > > > > > > > > > Yep that's exactly what i was gettin at, Thanx Patrick. > > > Don't you think that the section "5.6. Application Layer > -- Routing > > > Protocols" > > > in framework draft is missing or ignoring that requirement. > > > Just a concern. > > > > > > Regards > > > -jeelani.sm > > > > > > -----Original Message----- > > > From: Patrick Droz [mailto:dro@zurich.ibm.com] > > > Sent: Thursday, October 10, 2002 10:08 AM > > > To: FORCES@PEACH.EASE.LSOFT.COM > > > Subject: Re: Please consider last call on > > > draft-ietf-forces-framework-02.t xt > > > > > > > > > Jeelani, > > > > > > there are some statments about this in the requirements > > > draft. > > > http://www.ietf.org/internet-drafts/draft-ietf-forces-requirem > > > ents-06.txt > > > Section 6.5. Point 8) Off-load Functions > > > > > > Regards, > > > Patrick > > > > > > > > > Syed, Jeelani wrote: > > > > Hi, > > > > > > > > Sometimes you may have to run protocol counterparts(agents) on > > > > FEs for protocols running on CEs to avoid packet latencies and > > > > increase scalability. This draft doesn't talk abt that, > > > > Is it because its too soon to talk abt it? > > > > > > > > regards, > > > > > > > > -jeelani.sm > > > > > > > > -----Original Message----- > > > > From: Yang, Lily L [mailto:lily.l.yang@intel.com] > > > > Sent: Wednesday, October 09, 2002 8:11 PM > > > > To: FORCES@PEACH.EASE.LSOFT.COM > > > > Subject: Please consider last call on > > > draft-ietf-forces-framework-02.txt > > > > > > > > > > > > Hi, > > > > We just submitted a new version of the ForCES framework draft. > > > > Several people on the list have given us comments on the > > > previous (ver01) > > > > draft and we've incorporated all into this version. > > > > > > > > Most of the change are minor, including: > > > > 1) fixing Fig 6 (a) to make it really full mesh > > > > 2) Moving the Abstract and Table of Contents to > the front. > > > > 3) Other editorial changes to improve clarity > > > throughout the draft > > > > > > > > A more significant modification is in section 5.7. > > > "Application Layer -- > > > > Network Management Protocol". Per comments from T. Sridhar, > > > when we talk > > > > about how SNMP is supported in ForCES, we suggest that > > > "AgentX framework > > > > defined in RFC2741 may be applied here such that CEs act in > > > the role of > > > > master agent to process SNMP protocol messages while FEs > > > act in the role > > > of > > > > subagent to provide access to the MIB objects residing on > > > FEs. AgentX > > > > protocol messages between the master agent (CE) and the > > > subagent (FE) are > > > > encapsulated and transported via ForCES, just like data > > > packets from any > > > > other application layer protocols." > > > > > > > > Accordingly, a new reference to RFC2741 is also added in > > > the "normative > > > > reference" section. > > > > > > > > As always, comments are welcome. > > > > > > > > The co-authors feel that this framework draft has been > > > through several > > > > iterations and seems pretty stable now. So we would like > > > the working group > > > > chairs to consider last call on this version. > > > > > > > > > > > > Thanks, > > > > Lily > > > > > > > > > > > > <> > > > > > > > > > > > > > > > > > -- > > > Dr. Patrick Droz | dro@zurich.ibm.com > > > IBM Zurich Research Laboratory | > > > http://www.zurich.ibm.com/~dro > > > Saumerstrasse 4 | Tel. > > +41-1-724-85-25 CH-8803 > > > Rueschlikon/Switzerland | Fax. +41-1-724-85-78 > > > > > > From owner-forces@PEACH.EASE.LSOFT.COM Thu Oct 10 15:57:10 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA21714 for ; Thu, 10 Oct 2002 15:57:09 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <7.007659AE@cherry.ease.lsoft.com>; Thu, 10 Oct 2002 15:59:15 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 28710240 for FORCES@PEACH.EASE.LSOFT.COM; Thu, 10 Oct 2002 15:59:15 -0400 Received: from 63.78.179.217 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Thu, 10 Oct 2002 15:58:55 -0400 Received: from davir04nok.americas.nokia.com (davir04nok.americas.nokia.com [172.18.242.87]) by mgw-dax2.ext.nokia.com (Switch-2.2.1/Switch-2.2.0) with ESMTP id g9AJxYM22081 for ; Thu, 10 Oct 2002 14:59:34 -0500 (CDT) Received: from daebh002.NOE.Nokia.com (unverified) by davir04nok.americas.nokia.com (Content Technologies SMTPRS 4.2.5) with ESMTP id for ; Thu, 10 Oct 2002 14:58:53 -0500 Received: from bsebe001.NOE.Nokia.com ([172.19.160.13]) by daebh002.NOE.Nokia.com with Microsoft SMTPSVC(5.0.2195.5329); Thu, 10 Oct 2002 14:57:42 -0500 X-MimeOLE: Produced By Microsoft Exchange V6.0.6249.0 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Thread-Topic: Please consider last call on draft-ietf-forces-framew ork-02.t xt Thread-Index: AcJwlk4tNf5fqZvyTzanHjMkz6Y5bwAAN/RA X-OriginalArrivalTime: 10 Oct 2002 19:57:42.0256 (UTC) FILETIME=[4B200F00:01C27097] Approved-By: ram.gopal@NOKIA.COM Message-ID: Date: Thu, 10 Oct 2002 15:57:41 -0400 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: "(Ram Gopal )" Subject: Re: Please consider last call on draft-ietf-forces-framew ork-02.t xt To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by ietf.org id PAA21714 I agree with this comment and this is what i meant in my last email. Regards Ramg > -----Original Message----- > From: ext Alex Vasquez [mailto:AVasquez@LVL7.com] > Sent: Thursday, October 10, 2002 3:31 PM > To: FORCES@PEACH.EASE.LSOFT.COM > Subject: Re: Please consider last call on draft-ietf-forces-framew > ork-02.t xt > > > All, > I do not think we should be mixing logical entities with > physical ones. > There should not be off-loading CE functions onto FE, this contradicts > current CE and FE definitions. CE and FE are logical entities with > concrete responsibilities. I can see that one can off-load a > CE function > into a physical entity that can perform that CE functions. I > can also see > that the same physical entity performing FE functions. Both, > CE and FE, > running in the same physical entity should interact similarly > as if they > were running in different physical entities. Thoughts? > > Cheers, > > Alex > > > The information contained in this e-mail is LVL7 > > Confidential. Any use except that authorized by LVL7 is > > prohibited. If you get this in error, please notify sender > > and delete this e-mail. > > > > > -----Original Message----- > From: Yang, Lily L [mailto:lily.l.yang@intel.com] > Sent: Thursday, October 10, 2002 12:29 PM > To: FORCES@PEACH.EASE.LSOFT.COM > Subject: Re: Please consider last call on draft-ietf-forces-framew > ork-02.t xt > > > Jeelani -- > That is a very good point. I have been thinking about it > lately. The concept > and practice of "off-loading CE functions onto FE" or "split > control plane" > challenge ForCES's way of defining FE and CE -- our > terminology has been > that FE and CE are both "logical" entities -- but what really is the > boundary here? Is it logical boundary or physical boundary? > Our goal is to > strive for physical sepration. So I think it is actually the physical > boundary that defines CE and FE. Maybe we should revise the > definition to > reflect this. What do people think? > > I was keenly aware of the fact that this was not addressed in > the framework. > I was not sure where this should be addressed. I think you > just point out a > good place to me. > I will get that fixed soon. > > Thank you for your comment, > Lily > > > -----Original Message----- > > From: Syed, Jeelani [mailto:jsyed@juniper.net] > > Sent: Thursday, October 10, 2002 7:27 AM > > To: FORCES@PEACH.EASE.LSOFT.COM > > Subject: Re: Please consider last call on draft-ietf-forces-framew > > ork-02.t xt > > > > > > Yep that's exactly what i was gettin at, Thanx Patrick. > > Don't you think that the section "5.6. Application Layer -- Routing > > Protocols" > > in framework draft is missing or ignoring that requirement. > > Just a concern. > > > > Regards > > -jeelani.sm > > > > -----Original Message----- > > From: Patrick Droz [mailto:dro@zurich.ibm.com] > > Sent: Thursday, October 10, 2002 10:08 AM > > To: FORCES@PEACH.EASE.LSOFT.COM > > Subject: Re: Please consider last call on > > draft-ietf-forces-framework-02.t xt > > > > > > Jeelani, > > > > there are some statments about this in the requirements > > draft. > > http://www.ietf.org/internet-drafts/draft-ietf-forces-requirem > > ents-06.txt > > Section 6.5. Point 8) Off-load Functions > > > > Regards, > > Patrick > > > > > > Syed, Jeelani wrote: > > > Hi, > > > > > > Sometimes you may have to run protocol counterparts(agents) on > > > FEs for protocols running on CEs to avoid packet latencies and > > > increase scalability. This draft doesn't talk abt that, > > > Is it because its too soon to talk abt it? > > > > > > regards, > > > > > > -jeelani.sm > > > > > > -----Original Message----- > > > From: Yang, Lily L [mailto:lily.l.yang@intel.com] > > > Sent: Wednesday, October 09, 2002 8:11 PM > > > To: FORCES@PEACH.EASE.LSOFT.COM > > > Subject: Please consider last call on > > draft-ietf-forces-framework-02.txt > > > > > > > > > Hi, > > > We just submitted a new version of the ForCES framework draft. > > > Several people on the list have given us comments on the > > previous (ver01) > > > draft and we've incorporated all into this version. > > > > > > Most of the change are minor, including: > > > 1) fixing Fig 6 (a) to make it really full mesh > > > 2) Moving the Abstract and Table of Contents to the front. > > > 3) Other editorial changes to improve clarity > > throughout the draft > > > > > > A more significant modification is in section 5.7. > > "Application Layer -- > > > Network Management Protocol". Per comments from T. Sridhar, > > when we talk > > > about how SNMP is supported in ForCES, we suggest that > > "AgentX framework > > > defined in RFC2741 may be applied here such that CEs act in > > the role of > > > master agent to process SNMP protocol messages while FEs > > act in the role > > of > > > subagent to provide access to the MIB objects residing on > > FEs. AgentX > > > protocol messages between the master agent (CE) and the > > subagent (FE) are > > > encapsulated and transported via ForCES, just like data > > packets from any > > > other application layer protocols." > > > > > > Accordingly, a new reference to RFC2741 is also added in > > the "normative > > > reference" section. > > > > > > As always, comments are welcome. > > > > > > The co-authors feel that this framework draft has been > > through several > > > iterations and seems pretty stable now. So we would like > > the working group > > > chairs to consider last call on this version. > > > > > > > > > Thanks, > > > Lily > > > > > > > > > <> > > > > > > > > > > > > -- > > Dr. Patrick Droz | dro@zurich.ibm.com > > IBM Zurich Research Laboratory | > > http://www.zurich.ibm.com/~dro > > Saumerstrasse 4 | Tel. > +41-1-724-85-25 CH-8803 > > Rueschlikon/Switzerland | Fax. +41-1-724-85-78 > > > From owner-forces@PEACH.EASE.LSOFT.COM Thu Oct 10 15:57:31 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA21757 for ; Thu, 10 Oct 2002 15:57:30 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <4.00765A77@cherry.ease.lsoft.com>; Thu, 10 Oct 2002 15:59:37 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 28710289 for FORCES@PEACH.EASE.LSOFT.COM; Thu, 10 Oct 2002 15:59:36 -0400 Received: from 65.194.140.130 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Thu, 10 Oct 2002 15:59:36 -0400 Received: by UNIWEST1 with Internet Mail Service (5.5.2653.19) id ; Thu, 10 Oct 2002 15:59:35 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Approved-By: "Syed, Jeelani" Message-ID: <49FF5C6DDBD8D311BBBD009027DE980C03FC3014@UNIWEST1> Date: Thu, 10 Oct 2002 15:58:40 -0400 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: "Syed, Jeelani" Subject: Re: Please consider last call on draft-ietf-forces-framew ork-02.t xt To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list Good point, Then off-loading CE functions" should be considered as a CE-CE interaction, which is nothing but load-sharing. CEs can load-share responsibilities among themselves within protocol itself, for ex: Within OSPF one of them will take care of running SPF and other responding to Hellos. Regards -jeelani.sm -----Original Message----- From: Alex Vasquez [mailto:AVasquez@LVL7.com] Sent: Thursday, October 10, 2002 3:31 PM To: FORCES@PEACH.EASE.LSOFT.COM Subject: Re: Please consider last call on draft-ietf-forces-framew ork-02.t xt All, I do not think we should be mixing logical entities with physical ones. There should not be off-loading CE functions onto FE, this contradicts current CE and FE definitions. CE and FE are logical entities with concrete responsibilities. I can see that one can off-load a CE function into a physical entity that can perform that CE functions. I can also see that the same physical entity performing FE functions. Both, CE and FE, running in the same physical entity should interact similarly as if they were running in different physical entities. Thoughts? Cheers, Alex > The information contained in this e-mail is LVL7 > Confidential. Any use except that authorized by LVL7 is > prohibited. If you get this in error, please notify sender > and delete this e-mail. > -----Original Message----- From: Yang, Lily L [mailto:lily.l.yang@intel.com] Sent: Thursday, October 10, 2002 12:29 PM To: FORCES@PEACH.EASE.LSOFT.COM Subject: Re: Please consider last call on draft-ietf-forces-framew ork-02.t xt Jeelani -- That is a very good point. I have been thinking about it lately. The concept and practice of "off-loading CE functions onto FE" or "split control plane" challenge ForCES's way of defining FE and CE -- our terminology has been that FE and CE are both "logical" entities -- but what really is the boundary here? Is it logical boundary or physical boundary? Our goal is to strive for physical sepration. So I think it is actually the physical boundary that defines CE and FE. Maybe we should revise the definition to reflect this. What do people think? I was keenly aware of the fact that this was not addressed in the framework. I was not sure where this should be addressed. I think you just point out a good place to me. I will get that fixed soon. Thank you for your comment, Lily > -----Original Message----- > From: Syed, Jeelani [mailto:jsyed@juniper.net] > Sent: Thursday, October 10, 2002 7:27 AM > To: FORCES@PEACH.EASE.LSOFT.COM > Subject: Re: Please consider last call on draft-ietf-forces-framew > ork-02.t xt > > > Yep that's exactly what i was gettin at, Thanx Patrick. > Don't you think that the section "5.6. Application Layer -- Routing > Protocols" > in framework draft is missing or ignoring that requirement. > Just a concern. > > Regards > -jeelani.sm > > -----Original Message----- > From: Patrick Droz [mailto:dro@zurich.ibm.com] > Sent: Thursday, October 10, 2002 10:08 AM > To: FORCES@PEACH.EASE.LSOFT.COM > Subject: Re: Please consider last call on > draft-ietf-forces-framework-02.t xt > > > Jeelani, > > there are some statments about this in the requirements > draft. > http://www.ietf.org/internet-drafts/draft-ietf-forces-requirem > ents-06.txt > Section 6.5. Point 8) Off-load Functions > > Regards, > Patrick > > > Syed, Jeelani wrote: > > Hi, > > > > Sometimes you may have to run protocol counterparts(agents) on > > FEs for protocols running on CEs to avoid packet latencies and > > increase scalability. This draft doesn't talk abt that, > > Is it because its too soon to talk abt it? > > > > regards, > > > > -jeelani.sm > > > > -----Original Message----- > > From: Yang, Lily L [mailto:lily.l.yang@intel.com] > > Sent: Wednesday, October 09, 2002 8:11 PM > > To: FORCES@PEACH.EASE.LSOFT.COM > > Subject: Please consider last call on > draft-ietf-forces-framework-02.txt > > > > > > Hi, > > We just submitted a new version of the ForCES framework draft. > > Several people on the list have given us comments on the > previous (ver01) > > draft and we've incorporated all into this version. > > > > Most of the change are minor, including: > > 1) fixing Fig 6 (a) to make it really full mesh > > 2) Moving the Abstract and Table of Contents to the front. > > 3) Other editorial changes to improve clarity > throughout the draft > > > > A more significant modification is in section 5.7. > "Application Layer -- > > Network Management Protocol". Per comments from T. Sridhar, > when we talk > > about how SNMP is supported in ForCES, we suggest that > "AgentX framework > > defined in RFC2741 may be applied here such that CEs act in > the role of > > master agent to process SNMP protocol messages while FEs > act in the role > of > > subagent to provide access to the MIB objects residing on > FEs. AgentX > > protocol messages between the master agent (CE) and the > subagent (FE) are > > encapsulated and transported via ForCES, just like data > packets from any > > other application layer protocols." > > > > Accordingly, a new reference to RFC2741 is also added in > the "normative > > reference" section. > > > > As always, comments are welcome. > > > > The co-authors feel that this framework draft has been > through several > > iterations and seems pretty stable now. So we would like > the working group > > chairs to consider last call on this version. > > > > > > Thanks, > > Lily > > > > > > <> > > > > > > > -- > Dr. Patrick Droz | dro@zurich.ibm.com > IBM Zurich Research Laboratory | > http://www.zurich.ibm.com/~dro > Saumerstrasse 4 | Tel. +41-1-724-85-25 CH-8803 > Rueschlikon/Switzerland | Fax. +41-1-724-85-78 > From owner-forces@PEACH.EASE.LSOFT.COM Thu Oct 10 16:53:59 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA23866 for ; Thu, 10 Oct 2002 16:53:58 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <5.00765BC1@cherry.ease.lsoft.com>; Thu, 10 Oct 2002 16:56:00 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 28730216 for FORCES@PEACH.EASE.LSOFT.COM; Thu, 10 Oct 2002 16:55:56 -0400 Received: from 192.55.52.18 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Thu, 10 Oct 2002 16:55:48 -0400 Received: from petasus.fm.intel.com (petasus.fm.intel.com [10.1.192.37]) by hermes.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.51 2002/09/23 20:43:23 dmccart Exp $) with ESMTP id g9AKsUE26878 for ; Thu, 10 Oct 2002 20:54:30 GMT Received: from fmsmsxvs042.fm.intel.com (fmsmsxvs042.fm.intel.com [132.233.42.128]) by petasus.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.26 2002/10/10 20:36:29 dmccart Exp $) with SMTP id g9AKq6529231 for ; Thu, 10 Oct 2002 20:52:06 GMT Received: from FMSMSX018.fm.intel.com ([132.233.42.197]) by fmsmsxvs042.fm.intel.com (NAVGW 2.5.2.11) with SMTP id M2002101013572800405 for ; Thu, 10 Oct 2002 13:57:28 -0700 Received: by fmsmsx018.fm.intel.com with Internet Mail Service (5.5.2653.19) id ; Thu, 10 Oct 2002 13:55:47 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Approved-By: "Yang, Lily L" Message-ID: Date: Thu, 10 Oct 2002 13:55:45 -0700 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: "Yang, Lily L" Subject: Re: Please consider last call on draft-ietf-forces-framew ork-02.t xt To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list (a)Physical boundary view: +--------------+ | CE | +--------------+ | +------------------+ | +-----------+ | | |CE offload | | | +-----------+ | | FE | +------------------+ (b)Logical boundary view: +------------------+ | CE | | +-----------+ | | |CE offload | | | +-----------+ | +------------------+ | +--------------+ | FE | +--------------+ Conceptually, I like to stick with the "logical" boundary view and consider the CE offload as a case of CE-CE coordination for load sharing. But it is still a special kind of CE-CE coordination because it directly gets into our way of achieving the grand vision of "physical separation". And because this CE offload function really resides on the FE, we have to use CE-FE interface to carry this "CE-CE" communication messages. It is in this sense that I think the "physical" view actually reflects more accurately the impact on the architecture. ForCES needs to encapsulate a lot of things any way, so I don't see why it can't encapsulate this CE-CE messages. But this kind of go against the statement that "ForCES is for CE-FE only" and it also means CE-CE interaction now gets into the picture of interoperability. Say a line card (FE) from vendor X can tell a control card (CE) from vendor Y that "I can do OSPF hello offload". Now a bit more extra need to be in place before these two cards (physical entieis) can interoperate. That is some kind of CE-CE interface for OSPF hello offloading. All I am trying to get to is an understanding of how much ForCES can or cannot enable by itself in such scenario if we insist on the logical view. Framework draft calls out things like "if you want to do CE redundancy, you need CE-CE in addition to ForCES". Now similarly, I would say "if you want to do CE offloading, you also need more than ForCES". Is that a right thing to say? Or, alternatively, we can take the physical boundary view and say "CE offload" is part of FE. Then ForCES is still exclusively used for CE-FE. And as requirement draft already points out, FE model need to be able to express such capability. Does it mean "OSPF Hello offload" is a functional block just like a "classifier" or "IPv4 Forwarder"? Lily > -----Original Message----- > From: Syed, Jeelani [mailto:jsyed@juniper.net] > Sent: Thursday, October 10, 2002 12:59 PM > To: FORCES@PEACH.EASE.LSOFT.COM > Subject: Re: Please consider last call on draft-ietf-forces-framew > ork-02.t xt > > > Good point, Then off-loading CE functions" should be > considered as a CE-CE interaction, which is nothing > but load-sharing. CEs can load-share responsibilities > among themselves within protocol itself, for ex: Within > OSPF one of them will take care of running SPF and other > responding to Hellos. > > Regards > -jeelani.sm > > -----Original Message----- > From: Alex Vasquez [mailto:AVasquez@LVL7.com] > Sent: Thursday, October 10, 2002 3:31 PM > To: FORCES@PEACH.EASE.LSOFT.COM > Subject: Re: Please consider last call on draft-ietf-forces-framew > ork-02.t xt > > > All, > I do not think we should be mixing logical entities with > physical ones. > There should not be off-loading CE functions onto FE, this contradicts > current CE and FE definitions. CE and FE are logical entities with > concrete responsibilities. I can see that one can off-load a > CE function > into a physical entity that can perform that CE functions. I > can also see > that the same physical entity performing FE functions. Both, > CE and FE, > running in the same physical entity should interact similarly > as if they > were running in different physical entities. Thoughts? > > Cheers, > > Alex > > > The information contained in this e-mail is LVL7 > > Confidential. Any use except that authorized by LVL7 is > > prohibited. If you get this in error, please notify sender > > and delete this e-mail. > > > > > -----Original Message----- > From: Yang, Lily L [mailto:lily.l.yang@intel.com] > Sent: Thursday, October 10, 2002 12:29 PM > To: FORCES@PEACH.EASE.LSOFT.COM > Subject: Re: Please consider last call on draft-ietf-forces-framew > ork-02.t xt > > > Jeelani -- > That is a very good point. I have been thinking about it > lately. The concept > and practice of "off-loading CE functions onto FE" or "split > control plane" > challenge ForCES's way of defining FE and CE -- our > terminology has been > that FE and CE are both "logical" entities -- but what really is the > boundary here? Is it logical boundary or physical boundary? > Our goal is to > strive for physical sepration. So I think it is actually the physical > boundary that defines CE and FE. Maybe we should revise the > definition to > reflect this. What do people think? > > I was keenly aware of the fact that this was not addressed in > the framework. > I was not sure where this should be addressed. I think you > just point out a > good place to me. > I will get that fixed soon. > > Thank you for your comment, > Lily > > > -----Original Message----- > > From: Syed, Jeelani [mailto:jsyed@juniper.net] > > Sent: Thursday, October 10, 2002 7:27 AM > > To: FORCES@PEACH.EASE.LSOFT.COM > > Subject: Re: Please consider last call on draft-ietf-forces-framew > > ork-02.t xt > > > > > > Yep that's exactly what i was gettin at, Thanx Patrick. > > Don't you think that the section "5.6. Application Layer -- Routing > > Protocols" > > in framework draft is missing or ignoring that requirement. > > Just a concern. > > > > Regards > > -jeelani.sm > > > > -----Original Message----- > > From: Patrick Droz [mailto:dro@zurich.ibm.com] > > Sent: Thursday, October 10, 2002 10:08 AM > > To: FORCES@PEACH.EASE.LSOFT.COM > > Subject: Re: Please consider last call on > > draft-ietf-forces-framework-02.t xt > > > > > > Jeelani, > > > > there are some statments about this in the requirements > > draft. > > http://www.ietf.org/internet-drafts/draft-ietf-forces-requirem > > ents-06.txt > > Section 6.5. Point 8) Off-load Functions > > > > Regards, > > Patrick > > > > > > Syed, Jeelani wrote: > > > Hi, > > > > > > Sometimes you may have to run protocol counterparts(agents) on > > > FEs for protocols running on CEs to avoid packet latencies and > > > increase scalability. This draft doesn't talk abt that, > > > Is it because its too soon to talk abt it? > > > > > > regards, > > > > > > -jeelani.sm > > > > > > -----Original Message----- > > > From: Yang, Lily L [mailto:lily.l.yang@intel.com] > > > Sent: Wednesday, October 09, 2002 8:11 PM > > > To: FORCES@PEACH.EASE.LSOFT.COM > > > Subject: Please consider last call on > > draft-ietf-forces-framework-02.txt > > > > > > > > > Hi, > > > We just submitted a new version of the ForCES framework draft. > > > Several people on the list have given us comments on the > > previous (ver01) > > > draft and we've incorporated all into this version. > > > > > > Most of the change are minor, including: > > > 1) fixing Fig 6 (a) to make it really full mesh > > > 2) Moving the Abstract and Table of Contents to the front. > > > 3) Other editorial changes to improve clarity > > throughout the draft > > > > > > A more significant modification is in section 5.7. > > "Application Layer -- > > > Network Management Protocol". Per comments from T. Sridhar, > > when we talk > > > about how SNMP is supported in ForCES, we suggest that > > "AgentX framework > > > defined in RFC2741 may be applied here such that CEs act in > > the role of > > > master agent to process SNMP protocol messages while FEs > > act in the role > > of > > > subagent to provide access to the MIB objects residing on > > FEs. AgentX > > > protocol messages between the master agent (CE) and the > > subagent (FE) are > > > encapsulated and transported via ForCES, just like data > > packets from any > > > other application layer protocols." > > > > > > Accordingly, a new reference to RFC2741 is also added in > > the "normative > > > reference" section. > > > > > > As always, comments are welcome. > > > > > > The co-authors feel that this framework draft has been > > through several > > > iterations and seems pretty stable now. So we would like > > the working group > > > chairs to consider last call on this version. > > > > > > > > > Thanks, > > > Lily > > > > > > > > > <> > > > > > > > > > > > > -- > > Dr. Patrick Droz | dro@zurich.ibm.com > > IBM Zurich Research Laboratory | > > http://www.zurich.ibm.com/~dro > > Saumerstrasse 4 | Tel. > +41-1-724-85-25 CH-8803 > > Rueschlikon/Switzerland | Fax. +41-1-724-85-78 > > > From owner-forces@PEACH.EASE.LSOFT.COM Thu Oct 10 18:11:22 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA25667 for ; Thu, 10 Oct 2002 18:11:21 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <3.00765FFA@cherry.ease.lsoft.com>; Thu, 10 Oct 2002 18:13:26 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 28740702 for FORCES@PEACH.EASE.LSOFT.COM; Thu, 10 Oct 2002 18:13:24 -0400 Received: from 65.194.140.130 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Thu, 10 Oct 2002 18:13:24 -0400 Received: by UNIWEST1 with Internet Mail Service (5.5.2653.19) id ; Thu, 10 Oct 2002 18:13:23 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Approved-By: "Syed, Jeelani" Message-ID: <49FF5C6DDBD8D311BBBD009027DE980C03FC3015@UNIWEST1> Date: Thu, 10 Oct 2002 18:12:26 -0400 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: "Syed, Jeelani" Subject: Re: Please consider last call on draft-ietf-forces-framew ork-02.t xt To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list I see a drawback with first approach(CE offload is beyond ForCES), a vendor making a forwarding plane have to take care of all control plane vendors that are compatible with it for offloading. But thats not the case with redundancy 'coz an assumption can be made that usually redundant and active control planes belong to the same vendor. Any thoughts? -jeelani.sm -----Original Message----- From: Yang, Lily L [mailto:lily.l.yang@intel.com] Sent: Thursday, October 10, 2002 4:56 PM To: FORCES@PEACH.EASE.LSOFT.COM Subject: Re: Please consider last call on draft-ietf-forces-framew ork-02.t xt (a)Physical boundary view: +--------------+ | CE | +--------------+ | +------------------+ | +-----------+ | | |CE offload | | | +-----------+ | | FE | +------------------+ (b)Logical boundary view: +------------------+ | CE | | +-----------+ | | |CE offload | | | +-----------+ | +------------------+ | +--------------+ | FE | +--------------+ Conceptually, I like to stick with the "logical" boundary view and consider the CE offload as a case of CE-CE coordination for load sharing. But it is still a special kind of CE-CE coordination because it directly gets into our way of achieving the grand vision of "physical separation". And because this CE offload function really resides on the FE, we have to use CE-FE interface to carry this "CE-CE" communication messages. It is in this sense that I think the "physical" view actually reflects more accurately the impact on the architecture. ForCES needs to encapsulate a lot of things any way, so I don't see why it can't encapsulate this CE-CE messages. But this kind of go against the statement that "ForCES is for CE-FE only" and it also means CE-CE interaction now gets into the picture of interoperability. Say a line card (FE) from vendor X can tell a control card (CE) from vendor Y that "I can do OSPF hello offload". Now a bit more extra need to be in place before these two cards (physical entieis) can interoperate. That is some kind of CE-CE interface for OSPF hello offloading. All I am trying to get to is an understanding of how much ForCES can or cannot enable by itself in such scenario if we insist on the logical view. Framework draft calls out things like "if you want to do CE redundancy, you need CE-CE in addition to ForCES". Now similarly, I would say "if you want to do CE offloading, you also need more than ForCES". Is that a right thing to say? Or, alternatively, we can take the physical boundary view and say "CE offload" is part of FE. Then ForCES is still exclusively used for CE-FE. And as requirement draft already points out, FE model need to be able to express such capability. Does it mean "OSPF Hello offload" is a functional block just like a "classifier" or "IPv4 Forwarder"? Lily > -----Original Message----- > From: Syed, Jeelani [mailto:jsyed@juniper.net] > Sent: Thursday, October 10, 2002 12:59 PM > To: FORCES@PEACH.EASE.LSOFT.COM > Subject: Re: Please consider last call on draft-ietf-forces-framew > ork-02.t xt > > > Good point, Then off-loading CE functions" should be > considered as a CE-CE interaction, which is nothing > but load-sharing. CEs can load-share responsibilities > among themselves within protocol itself, for ex: Within > OSPF one of them will take care of running SPF and other > responding to Hellos. > > Regards > -jeelani.sm > > -----Original Message----- > From: Alex Vasquez [mailto:AVasquez@LVL7.com] > Sent: Thursday, October 10, 2002 3:31 PM > To: FORCES@PEACH.EASE.LSOFT.COM > Subject: Re: Please consider last call on draft-ietf-forces-framew > ork-02.t xt > > > All, > I do not think we should be mixing logical entities with > physical ones. > There should not be off-loading CE functions onto FE, this contradicts > current CE and FE definitions. CE and FE are logical entities with > concrete responsibilities. I can see that one can off-load a > CE function > into a physical entity that can perform that CE functions. I > can also see > that the same physical entity performing FE functions. Both, > CE and FE, > running in the same physical entity should interact similarly > as if they > were running in different physical entities. Thoughts? > > Cheers, > > Alex > > > The information contained in this e-mail is LVL7 > > Confidential. Any use except that authorized by LVL7 is > > prohibited. If you get this in error, please notify sender > > and delete this e-mail. > > > > > -----Original Message----- > From: Yang, Lily L [mailto:lily.l.yang@intel.com] > Sent: Thursday, October 10, 2002 12:29 PM > To: FORCES@PEACH.EASE.LSOFT.COM > Subject: Re: Please consider last call on draft-ietf-forces-framew > ork-02.t xt > > > Jeelani -- > That is a very good point. I have been thinking about it > lately. The concept > and practice of "off-loading CE functions onto FE" or "split > control plane" > challenge ForCES's way of defining FE and CE -- our > terminology has been > that FE and CE are both "logical" entities -- but what really is the > boundary here? Is it logical boundary or physical boundary? > Our goal is to > strive for physical sepration. So I think it is actually the physical > boundary that defines CE and FE. Maybe we should revise the > definition to > reflect this. What do people think? > > I was keenly aware of the fact that this was not addressed in > the framework. > I was not sure where this should be addressed. I think you > just point out a > good place to me. > I will get that fixed soon. > > Thank you for your comment, > Lily > > > -----Original Message----- > > From: Syed, Jeelani [mailto:jsyed@juniper.net] > > Sent: Thursday, October 10, 2002 7:27 AM > > To: FORCES@PEACH.EASE.LSOFT.COM > > Subject: Re: Please consider last call on draft-ietf-forces-framew > > ork-02.t xt > > > > > > Yep that's exactly what i was gettin at, Thanx Patrick. > > Don't you think that the section "5.6. Application Layer -- Routing > > Protocols" > > in framework draft is missing or ignoring that requirement. > > Just a concern. > > > > Regards > > -jeelani.sm > > > > -----Original Message----- > > From: Patrick Droz [mailto:dro@zurich.ibm.com] > > Sent: Thursday, October 10, 2002 10:08 AM > > To: FORCES@PEACH.EASE.LSOFT.COM > > Subject: Re: Please consider last call on > > draft-ietf-forces-framework-02.t xt > > > > > > Jeelani, > > > > there are some statments about this in the requirements > > draft. > > http://www.ietf.org/internet-drafts/draft-ietf-forces-requirem > > ents-06.txt > > Section 6.5. Point 8) Off-load Functions > > > > Regards, > > Patrick > > > > > > Syed, Jeelani wrote: > > > Hi, > > > > > > Sometimes you may have to run protocol counterparts(agents) on > > > FEs for protocols running on CEs to avoid packet latencies and > > > increase scalability. This draft doesn't talk abt that, > > > Is it because its too soon to talk abt it? > > > > > > regards, > > > > > > -jeelani.sm > > > > > > -----Original Message----- > > > From: Yang, Lily L [mailto:lily.l.yang@intel.com] > > > Sent: Wednesday, October 09, 2002 8:11 PM > > > To: FORCES@PEACH.EASE.LSOFT.COM > > > Subject: Please consider last call on > > draft-ietf-forces-framework-02.txt > > > > > > > > > Hi, > > > We just submitted a new version of the ForCES framework draft. > > > Several people on the list have given us comments on the > > previous (ver01) > > > draft and we've incorporated all into this version. > > > > > > Most of the change are minor, including: > > > 1) fixing Fig 6 (a) to make it really full mesh > > > 2) Moving the Abstract and Table of Contents to the front. > > > 3) Other editorial changes to improve clarity > > throughout the draft > > > > > > A more significant modification is in section 5.7. > > "Application Layer -- > > > Network Management Protocol". Per comments from T. Sridhar, > > when we talk > > > about how SNMP is supported in ForCES, we suggest that > > "AgentX framework > > > defined in RFC2741 may be applied here such that CEs act in > > the role of > > > master agent to process SNMP protocol messages while FEs > > act in the role > > of > > > subagent to provide access to the MIB objects residing on > > FEs. AgentX > > > protocol messages between the master agent (CE) and the > > subagent (FE) are > > > encapsulated and transported via ForCES, just like data > > packets from any > > > other application layer protocols." > > > > > > Accordingly, a new reference to RFC2741 is also added in > > the "normative > > > reference" section. > > > > > > As always, comments are welcome. > > > > > > The co-authors feel that this framework draft has been > > through several > > > iterations and seems pretty stable now. So we would like > > the working group > > > chairs to consider last call on this version. > > > > > > > > > Thanks, > > > Lily > > > > > > > > > <> > > > > > > > > > > > > -- > > Dr. Patrick Droz | dro@zurich.ibm.com > > IBM Zurich Research Laboratory | > > http://www.zurich.ibm.com/~dro > > Saumerstrasse 4 | Tel. > +41-1-724-85-25 CH-8803 > > Rueschlikon/Switzerland | Fax. +41-1-724-85-78 > > > From owner-forces@PEACH.EASE.LSOFT.COM Fri Oct 11 07:36:00 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA17087 for ; Fri, 11 Oct 2002 07:36:00 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <8.0076760B@cherry.ease.lsoft.com>; Fri, 11 Oct 2002 7:38:05 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 28803239 for FORCES@PEACH.EASE.LSOFT.COM; Fri, 11 Oct 2002 07:38:05 -0400 Received: from 194.196.100.235 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Fri, 11 Oct 2002 07:38:05 -0400 Received: from d12relay02.de.ibm.com (d12relay02.de.ibm.com [9.165.215.23]) by d12lmsgate-2.de.ibm.com (8.12.3/8.12.3) with ESMTP id g9BBc3lR018564 for ; Fri, 11 Oct 2002 13:38:03 +0200 Received: from etzel.zurich.ibm.com (etzel.zurich.ibm.com [9.4.64.140]) by d12relay02.de.ibm.com (8.12.3/NCO/VER6.4) with SMTP id g9BBc1hW109380 for ; Fri, 11 Oct 2002 13:38:02 +0200 Received: from dhcp70-86.zurich.ibm.com by etzel.zurich.ibm.com (AIX 4.3/UCB 5.64/4.03) id AA28940 from ; Fri, 11 Oct 2002 13:38:00 +0200 X-Mailer: Mozilla 4.76 [en] (Windows NT 5.0; U) X-Accept-Language: en Mime-Version: 1.0 References: Content-Type: text/plain; charset=iso-8859-1 X-MIME-Autoconverted: from 8bit to quoted-printable by d12lmsgate-2.de.ibm.com id g9BBc3lR018564 Approved-By: Robert Haas Message-ID: <3DA6B818.984EF1A@zurich.ibm.com> Date: Fri, 11 Oct 2002 13:38:00 +0200 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: Robert Haas Subject: Re: Please consider last call on draft-ietf-forces-framework-02.t xt To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by ietf.org id HAA17087 Lily, Jeelani, "Syed, Jeelani" wrote [excerpt]: > I see a drawback with first approach(CE offload is beyond ForCES), a > vendor making a forwarding plane have to take care of all > control plane vendors that are compatible with it for offloading. "Yang, Lily L" wrote [excerpt]: > Or, alternatively, we can take the physical boundary view and say "CE > offload" is part of FE. Then ForCES is still exclusively used for CE-FE. And > as requirement draft already points out, FE model need to be able to express > such capability. Does it mean "OSPF Hello offload" is a functional block > just like a "classifier" or "IPv4 Forwarder"? Short answer: you are right. Long answer: The debate boils down to: what are today functions that we envision as "FE functions" ? Is generating an ICMP error message something that an FE could do ? Obviously yes. Then the FE has to model this for the CE to be able to drive this function. In this case, the CE would simply tell the FE: "please take care of ICMP processing locally ( and don't bother me with this noise )". The same argument holds for TCP termination or OSPF-Hellos state machine: to ensure interoperability between FE vendors that incorporate such functions and CE vendors, we must make sure ForCES defines the appropriate semantics for the CE to drive these functions in the FEs. So it's up to us to decide what we think are useful functions that FEs could perform, and then ensure that ForCES has the semantics to allow the CE to drive/control such functions. That was the reason why I suggested the addition of the "Off-loaded Functions" section into the requirements draft, and this should be reflected into the framework draft somehow. Regards, -- Robert Haas IBM Zurich Research Laboratory Säumerstrasse 4 CH-8803 Rüschlikon/Switzerland phone +41-1-724-8698 fax +41-1-724-8578 http://www.zurich.ibm.com/~rha From owner-forces@PEACH.EASE.LSOFT.COM Fri Oct 11 07:36:55 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA17106 for ; Fri, 11 Oct 2002 07:36:54 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <8.0076760D@cherry.ease.lsoft.com>; Fri, 11 Oct 2002 7:39:00 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 28803105 for FORCES@PEACH.EASE.LSOFT.COM; Fri, 11 Oct 2002 07:39:00 -0400 Received: from 132.151.1.176 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Fri, 11 Oct 2002 07:29:00 -0400 Received: from CNRI.Reston.VA.US (localhost [127.0.0.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA16548; Fri, 11 Oct 2002 07:26:53 -0400 (EDT) Mime-Version: 1.0 Content-Type: Multipart/Mixed; Boundary="NextPart" Approved-By: nsyracus@CNRI.RESTON.VA.US Message-ID: <200210111126.HAA16548@ietf.org> Date: Fri, 11 Oct 2002 07:26:53 -0400 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation Comments: RFC822 error: Incorrect or incomplete address field found and ignored. From: Internet-Drafts@ietf.org Subject: I-D ACTION:draft-ietf-forces-framework-02.txt To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list --NextPart A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Forwarding and Control Element Separation Working Group of the IETF. Title : ForCES Architectural Framework Author(s) : L. Yang et al. Filename : draft-ietf-forces-framework-02.txt Pages : 23 Date : 2002-10-10 This document defines the architectural framework for ForCES (Forwarding and Control Element Separation) network elements (NE), and identifies the associated entities and the interaction among them. This framework is intended to satisfy the requirements specified in the ForCES requirements draft [FORCES-REQ]. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-forces-framework-02.txt To remove yourself from the IETF Announcement list, send a message to ietf-announce-request with the word unsubscribe in the body of the message. Internet-Drafts are also available by anonymous FTP. Login with the username "anonymous" and a password of your e-mail address. After logging in, type "cd internet-drafts" and then "get draft-ietf-forces-framework-02.txt". A list of Internet-Drafts directories can be found in http://www.ietf.org/shadow.html or ftp://ftp.ietf.org/ietf/1shadow-sites.txt Internet-Drafts can also be obtained by e-mail. Send a message to: mailserv@ietf.org. In the body type: "FILE /internet-drafts/draft-ietf-forces-framework-02.txt". NOTE: The mail server at ietf.org can return the document in MIME-encoded form by using the "mpack" utility. To use this feature, insert the command "ENCODING mime" before the "FILE" command. To decode the response(s), you will need "munpack" or a MIME-compliant mail reader. Different MIME-compliant mail readers exhibit different behavior, especially when dealing with "multipart" MIME messages (i.e. documents which have been split up into multiple messages), so check your local documentation on how to manipulate these messages. Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. --NextPart Content-Type: Multipart/Alternative; Boundary="OtherAccess" --OtherAccess Content-Type: Message/External-body; access-type="mail-server"; server="mailserv@ietf.org" Content-Type: text/plain Content-ID: <2002-10-10140607.I-D@ietf.org> ENCODING mime FILE /internet-drafts/draft-ietf-forces-framework-02.txt --OtherAccess Content-Type: Message/External-body; name="draft-ietf-forces-framework-02.txt"; site="ftp.ietf.org"; access-type="anon-ftp"; directory="internet-drafts" Content-Type: text/plain Content-ID: <2002-10-10140607.I-D@ietf.org> --OtherAccess-- --NextPart-- From owner-forces@PEACH.EASE.LSOFT.COM Fri Oct 11 09:25:46 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA20780 for ; Fri, 11 Oct 2002 09:25:45 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <2.00767ABB@cherry.ease.lsoft.com>; Fri, 11 Oct 2002 9:27:52 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 28808048 for FORCES@PEACH.EASE.LSOFT.COM; Fri, 11 Oct 2002 09:27:52 -0400 Received: from 209.238.9.57 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Fri, 11 Oct 2002 09:27:52 -0400 Received: from www.zagrosnetworks.com (208.55.110.97) by mail07a.vwh1.net (RS ver 1.0.63s) with SMTP id 077909 for ; Fri, 11 Oct 2002 09:27:42 -0400 (EDT) Received: from 192.168.0.116 ([192.168.0.116]) by mail (WinRoute Pro 4.1.27) with SMTP; Fri, 11 Oct 2002 09:21:46 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.2627 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal X-Loop-Detect: 1 Approved-By: Vip Sharma Message-ID: <005b01c27129$24099200$7400a8c0@zagrosnetworks.com> Date: Fri, 11 Oct 2002 09:21:43 -0400 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: Vip Sharma Subject: Re: Please consider last call on draft-ietf-forces-framew ork-02.t xt To: FORCES@PEACH.EASE.LSOFT.COM In-Reply-To: Precedence: list Content-Transfer-Encoding: 7bit All, I agree with Alex's comments... CE and FE are logical entities .. Where they are physically located, though interesting from OEM point of view, is not very germane. Wherever they are located they interact in the "same" way... vip -----Original Message----- From: Forwarding and Control Element Separation [mailto:FORCES@PEACH.EASE.LSOFT.COM] On Behalf Of Alex Vasquez Sent: Thursday, October 10, 2002 3:31 PM To: FORCES@PEACH.EASE.LSOFT.COM Subject: Re: Please consider last call on draft-ietf-forces-framew ork-02.t xt All, I do not think we should be mixing logical entities with physical ones. There should not be off-loading CE functions onto FE, this contradicts current CE and FE definitions. CE and FE are logical entities with concrete responsibilities. I can see that one can off-load a CE function into a physical entity that can perform that CE functions. I can also see that the same physical entity performing FE functions. Both, CE and FE, running in the same physical entity should interact similarly as if they were running in different physical entities. Thoughts? Cheers, Alex > The information contained in this e-mail is LVL7 > Confidential. Any use except that authorized by LVL7 is > prohibited. If you get this in error, please notify sender > and delete this e-mail. > -----Original Message----- From: Yang, Lily L [mailto:lily.l.yang@intel.com] Sent: Thursday, October 10, 2002 12:29 PM To: FORCES@PEACH.EASE.LSOFT.COM Subject: Re: Please consider last call on draft-ietf-forces-framew ork-02.t xt Jeelani -- That is a very good point. I have been thinking about it lately. The concept and practice of "off-loading CE functions onto FE" or "split control plane" challenge ForCES's way of defining FE and CE -- our terminology has been that FE and CE are both "logical" entities -- but what really is the boundary here? Is it logical boundary or physical boundary? Our goal is to strive for physical sepration. So I think it is actually the physical boundary that defines CE and FE. Maybe we should revise the definition to reflect this. What do people think? I was keenly aware of the fact that this was not addressed in the framework. I was not sure where this should be addressed. I think you just point out a good place to me. I will get that fixed soon. Thank you for your comment, Lily > -----Original Message----- > From: Syed, Jeelani [mailto:jsyed@juniper.net] > Sent: Thursday, October 10, 2002 7:27 AM > To: FORCES@PEACH.EASE.LSOFT.COM > Subject: Re: Please consider last call on draft-ietf-forces-framew > ork-02.t xt > > > Yep that's exactly what i was gettin at, Thanx Patrick. > Don't you think that the section "5.6. Application Layer -- Routing > Protocols" > in framework draft is missing or ignoring that requirement. > Just a concern. > > Regards > -jeelani.sm > > -----Original Message----- > From: Patrick Droz [mailto:dro@zurich.ibm.com] > Sent: Thursday, October 10, 2002 10:08 AM > To: FORCES@PEACH.EASE.LSOFT.COM > Subject: Re: Please consider last call on > draft-ietf-forces-framework-02.t xt > > > Jeelani, > > there are some statments about this in the requirements > draft. > http://www.ietf.org/internet-drafts/draft-ietf-forces-requirem > ents-06.txt > Section 6.5. Point 8) Off-load Functions > > Regards, > Patrick > > > Syed, Jeelani wrote: > > Hi, > > > > Sometimes you may have to run protocol counterparts(agents) on > > FEs for protocols running on CEs to avoid packet latencies and > > increase scalability. This draft doesn't talk abt that, > > Is it because its too soon to talk abt it? > > > > regards, > > > > -jeelani.sm > > > > -----Original Message----- > > From: Yang, Lily L [mailto:lily.l.yang@intel.com] > > Sent: Wednesday, October 09, 2002 8:11 PM > > To: FORCES@PEACH.EASE.LSOFT.COM > > Subject: Please consider last call on > draft-ietf-forces-framework-02.txt > > > > > > Hi, > > We just submitted a new version of the ForCES framework draft. > > Several people on the list have given us comments on the > previous (ver01) > > draft and we've incorporated all into this version. > > > > Most of the change are minor, including: > > 1) fixing Fig 6 (a) to make it really full mesh > > 2) Moving the Abstract and Table of Contents to the front. > > 3) Other editorial changes to improve clarity > throughout the draft > > > > A more significant modification is in section 5.7. > "Application Layer -- > > Network Management Protocol". Per comments from T. Sridhar, > when we talk > > about how SNMP is supported in ForCES, we suggest that > "AgentX framework > > defined in RFC2741 may be applied here such that CEs act in > the role of > > master agent to process SNMP protocol messages while FEs > act in the role > of > > subagent to provide access to the MIB objects residing on > FEs. AgentX > > protocol messages between the master agent (CE) and the > subagent (FE) are > > encapsulated and transported via ForCES, just like data > packets from any > > other application layer protocols." > > > > Accordingly, a new reference to RFC2741 is also added in > the "normative > > reference" section. > > > > As always, comments are welcome. > > > > The co-authors feel that this framework draft has been > through several > > iterations and seems pretty stable now. So we would like > the working group > > chairs to consider last call on this version. > > > > > > Thanks, > > Lily > > > > > > <> > > > > > > > -- > Dr. Patrick Droz | dro@zurich.ibm.com > IBM Zurich Research Laboratory | > http://www.zurich.ibm.com/~dro > Saumerstrasse 4 | Tel. +41-1-724-85-25 CH-8803 > Rueschlikon/Switzerland | Fax. +41-1-724-85-78 > From owner-forces@PEACH.EASE.LSOFT.COM Fri Oct 11 09:46:31 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA21467 for ; Fri, 11 Oct 2002 09:46:30 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <2.00767B32@cherry.ease.lsoft.com>; Fri, 11 Oct 2002 9:48:37 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 28809270 for FORCES@PEACH.EASE.LSOFT.COM; Fri, 11 Oct 2002 09:48:33 -0400 Received: from 208.2.156.7 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Fri, 11 Oct 2002 09:48:33 -0400 Received: from jzny.localdomain ([208.2.156.2]) by lotus.znyx.com (Lotus Domino Release 5.0.11) with SMTP id 2002101106531302:34469 ; Fri, 11 Oct 2002 06:53:13 -0700 X-Mailer: KMail [version 1.2] MIME-Version: 1.0 X-MIMETrack: Itemize by SMTP Server on Lotus/Znyx(Release 5.0.11 |July 24, 2002) at 10/11/2002 06:53:13 AM, Serialize by Router on Lotus/Znyx(Release 5.0.11 |July 24, 2002) at 10/11/2002 06:53:16 AM Content-Type: text/plain; charset="iso-8859-1" Approved-By: Jamal Hadi Salim Message-ID: <02101108331102.01177@jzny.localdomain> Date: Fri, 11 Oct 2002 09:48:42 -0400 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: Jamal Hadi Salim Organization: Znyx Networks Subject: architecture draft comments To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by ietf.org id JAA21467 I realize that examples in the draft are just that, examples; however, I am a little concerned about some of the "suggestions" made and and how they may be interpreted as the way the protocol should behave. For instance: -paragraph under fig. 5 describes an example of how CE1 remains in sync with CE2 via a CE-CE protocol. I dont see it happening this way - section 4.1.2 and how the FE tells the FE manager of it FE ID. An alternate is for the FE to be told what its FE (perhaps by the CE via CE Manager to FE Manager). There are other ways. -Fig 9: I could see msg 2 "Let me join" or "I wish to register" as being the first msg. I could go on but hopefully with those few examples i have illustrated my concern. There should be clear text that indicates that the architecture examples are just that - mere examples; that try to describe from a sky-high view the concepts that might be employed. I just scanned the draft, so if that text is already there then ignore me. cheers, jamal From owner-forces@PEACH.EASE.LSOFT.COM Fri Oct 11 09:46:32 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA21469 for ; Fri, 11 Oct 2002 09:46:31 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <22.00767A50@cherry.ease.lsoft.com>; Fri, 11 Oct 2002 9:48:37 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 28809267 for FORCES@PEACH.EASE.LSOFT.COM; Fri, 11 Oct 2002 09:48:33 -0400 Received: from 208.2.156.7 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Fri, 11 Oct 2002 09:48:33 -0400 Received: from jzny.localdomain ([208.2.156.2]) by lotus.znyx.com (Lotus Domino Release 5.0.11) with SMTP id 2002101106531068:34467 ; Fri, 11 Oct 2002 06:53:10 -0700 X-Mailer: KMail [version 1.2] References: MIME-Version: 1.0 X-MIMETrack: Itemize by SMTP Server on Lotus/Znyx(Release 5.0.11 |July 24, 2002) at 10/11/2002 06:53:11 AM, Serialize by Router on Lotus/Znyx(Release 5.0.11 |July 24, 2002) at 10/11/2002 06:53:15 AM Content-Type: text/plain; charset="iso-8859-1" Approved-By: Jamal Hadi Salim Message-ID: <02101022551500.01177@jzny.localdomain> Date: Fri, 11 Oct 2002 08:20:39 -0400 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: Jamal Hadi Salim Organization: Znyx Networks Subject: Re: Please consider last call on draft-ietf-forces-framew ork-02.t xt Comments: To: "Yang, Lily L" To: FORCES@PEACH.EASE.LSOFT.COM In-Reply-To: Precedence: list Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by ietf.org id JAA21469 I am confused by the excitement. There is functionality which resides on the FE - some block. Its controller resides in the CE and tells the block how to behave via forces.... Some event: either a packet arrival or timer makes this block behave in a certain manner (as directed by its controller); Why should it matter whether it cooks, dices, meters packets, terminates or emits ospf hellos? The model already takes care of this (havent read the model draft in a while, so if it doesnt then it should). cheers, jamal From owner-forces@PEACH.EASE.LSOFT.COM Fri Oct 11 11:24:20 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA25367 for ; Fri, 11 Oct 2002 11:24:19 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <17.00767CCC@cherry.ease.lsoft.com>; Fri, 11 Oct 2002 11:26:26 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 28818714 for FORCES@PEACH.EASE.LSOFT.COM; Fri, 11 Oct 2002 11:26:25 -0400 Received: from 65.194.140.130 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Fri, 11 Oct 2002 11:26:25 -0400 Received: by UNIWEST1 with Internet Mail Service (5.5.2653.19) id ; Fri, 11 Oct 2002 11:26:24 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Approved-By: "Syed, Jeelani" Message-ID: <49FF5C6DDBD8D311BBBD009027DE980C03FC301A@UNIWEST1> Date: Fri, 11 Oct 2002 11:25:12 -0400 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: "Syed, Jeelani" Subject: Re: Please consider last call on draft-ietf-forces-framework-02.t xt To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by ietf.org id LAA25367 I agree, ForCES can define a standard for FE to announce its capabilities to CE. Regards, -jeelani.sm -----Original Message----- From: Robert Haas [mailto:rha@zurich.ibm.com] Sent: Friday, October 11, 2002 7:38 AM To: FORCES@PEACH.EASE.LSOFT.COM Subject: Re: Please consider last call on draft-ietf-forces-framework-02.t xt Lily, Jeelani, "Syed, Jeelani" wrote [excerpt]: > I see a drawback with first approach(CE offload is beyond ForCES), a > vendor making a forwarding plane have to take care of all > control plane vendors that are compatible with it for offloading. "Yang, Lily L" wrote [excerpt]: > Or, alternatively, we can take the physical boundary view and say "CE > offload" is part of FE. Then ForCES is still exclusively used for CE-FE. And > as requirement draft already points out, FE model need to be able to express > such capability. Does it mean "OSPF Hello offload" is a functional block > just like a "classifier" or "IPv4 Forwarder"? Short answer: you are right. Long answer: The debate boils down to: what are today functions that we envision as "FE functions" ? Is generating an ICMP error message something that an FE could do ? Obviously yes. Then the FE has to model this for the CE to be able to drive this function. In this case, the CE would simply tell the FE: "please take care of ICMP processing locally ( and don't bother me with this noise )". The same argument holds for TCP termination or OSPF-Hellos state machine: to ensure interoperability between FE vendors that incorporate such functions and CE vendors, we must make sure ForCES defines the appropriate semantics for the CE to drive these functions in the FEs. So it's up to us to decide what we think are useful functions that FEs could perform, and then ensure that ForCES has the semantics to allow the CE to drive/control such functions. That was the reason why I suggested the addition of the "Off-loaded Functions" section into the requirements draft, and this should be reflected into the framework draft somehow. Regards, -- Robert Haas IBM Zurich Research Laboratory Säumerstrasse 4 CH-8803 Rüschlikon/Switzerland phone +41-1-724-8698 fax +41-1-724-8578 http://www.zurich.ibm.com/~rha From owner-forces@PEACH.EASE.LSOFT.COM Fri Oct 11 11:31:16 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA25550 for ; Fri, 11 Oct 2002 11:31:16 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <13.00767BF8@cherry.ease.lsoft.com>; Fri, 11 Oct 2002 11:33:23 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 28819121 for FORCES@PEACH.EASE.LSOFT.COM; Fri, 11 Oct 2002 11:33:22 -0400 Received: from 143.209.238.80 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Fri, 11 Oct 2002 11:33:22 -0400 Received: from alcatel.com (localhost [127.0.0.1]) by auds951.usa.alcatel.com (8.10.2/8.10.2) with ESMTP id g9BFXMM28035 for ; Fri, 11 Oct 2002 10:33:22 -0500 (CDT) X-Mailer: Mozilla 4.72 [en] (WinNT; U) X-Accept-Language: en MIME-Version: 1.0 References: <02101108331102.01177@jzny.localdomain> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Approved-By: Alex Audu Message-ID: <3DA6EFE5.1540CF1E@alcatel.com> Date: Fri, 11 Oct 2002 10:36:06 -0500 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: Alex Audu Subject: Re: architecture draft comments To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list Content-Transfer-Encoding: 7bit Jamal, How else do you see CE and CE communicating with each other? Regards, Alex. Jamal Hadi Salim wrote: > I realize that examples in the draft are just that, examples; however, I am a > little concerned about some of the "suggestions" made and and how they may be > interpreted as the way the protocol should behave. For instance: > > -paragraph under fig. 5 describes an example of how CE1 remains in sync with > CE2 via a CE-CE protocol. I dont see it happening this way > - section 4.1.2 and how the FE tells the FE manager of it FE ID. An alternate > is for the FE to be told what its FE (perhaps by the CE via CE Manager to FE > Manager). There are other ways. > -Fig 9: I could see msg 2 "Let me join" or "I wish to register" as being the > first msg. > > I could go on but hopefully with those few examples i have illustrated my > concern. > There should be clear text that indicates that the architecture examples are > just that - mere examples; that try to describe from a sky-high view the > concepts that might be employed. > I just scanned the draft, so if that text is already there then ignore me. > > cheers, > jamal > From owner-forces@PEACH.EASE.LSOFT.COM Fri Oct 11 17:31:29 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA05539 for ; Fri, 11 Oct 2002 17:31:28 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <23.00768959@cherry.ease.lsoft.com>; Fri, 11 Oct 2002 17:33:36 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 28855998 for FORCES@PEACH.EASE.LSOFT.COM; Fri, 11 Oct 2002 17:33:35 -0400 Received: from 192.55.52.18 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Fri, 11 Oct 2002 17:33:35 -0400 Received: from talaria.fm.intel.com (talaria.fm.intel.com [10.1.192.39]) by hermes.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.51 2002/09/23 20:43:23 dmccart Exp $) with ESMTP id g9BLWGF16414 for ; Fri, 11 Oct 2002 21:32:16 GMT Received: from fmsmsxv040-1.fm.intel.com (fmsmsxvs040.fm.intel.com [132.233.42.124]) by talaria.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.26 2002/10/10 20:36:29 dmccart Exp $) with SMTP id g9BLaWE00114 for ; Fri, 11 Oct 2002 21:36:32 GMT Received: from fmsmsx29.FM.INTEL.COM ([132.233.42.29]) by fmsmsxv040-1.fm.intel.com (NAVGW 2.5.2.11) with SMTP id M2002101114340919776 ; Fri, 11 Oct 2002 14:34:09 -0700 Received: by fmsmsx29.fm.intel.com with Internet Mail Service (5.5.2653.19) id ; Fri, 11 Oct 2002 14:33:33 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Approved-By: "Yang, Lily L" Message-ID: Date: Fri, 11 Oct 2002 14:33:31 -0700 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: "Yang, Lily L" Subject: FW: comment on architecture Comments: To: "ram.gopal@nokia.com" To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list Ram -- Thanks for the comment. I will incorporate that into the next rev. Lily -----Original Message----- From: ram.gopal@nokia.com [mailto:ram.gopal@nokia.com] Sent: Thursday, October 10, 2002 1:43 PM To: lily.l.yang@intel.com Subject: comment on architecture Hello Lily, Architecture document describes the security message exchange between FE Manager and CE manager that its ok, as CEmanager and FEmanager are considered outside scope of Forces. But the security message exchange is being described between FE and CE in section 4.2.2. This is not within the scope of Forces protocol. We had these discussion in the requirements meeting couple of month back and RamD and Todd is aware of this and we have made explicit statement in requirement that key management etc are not part of the Forces. All key establishment ,rekeying etc are leveraged to TLS or IPsec or other underlying existing security framework . FEManager and CEManger configures all the security attributes using those information FE and CE communicates and there is no need to have any explicit security negotiation between them. IMHO we can remove the explicit "Security message " in section 4.2.2. Regards Ramg From owner-forces@PEACH.EASE.LSOFT.COM Fri Oct 11 17:49:34 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA05801 for ; Fri, 11 Oct 2002 17:49:34 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <14.0076887E@cherry.ease.lsoft.com>; Fri, 11 Oct 2002 17:51:41 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 28857371 for FORCES@PEACH.EASE.LSOFT.COM; Fri, 11 Oct 2002 17:51:41 -0400 Received: from 192.55.52.25 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Fri, 11 Oct 2002 17:51:41 -0400 Received: from petasus.fm.intel.com (petasus.fm.intel.com [10.1.192.37]) by caduceus.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.51 2002/09/23 20:43:23 dmccart Exp $) with ESMTP id g9BLlsR23916 for ; Fri, 11 Oct 2002 21:47:54 GMT Received: from fmsmsxvs042.fm.intel.com (fmsmsxvs042.fm.intel.com [132.233.42.128]) by petasus.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.26 2002/10/10 20:36:29 dmccart Exp $) with SMTP id g9BLlwq23233 for ; Fri, 11 Oct 2002 21:47:58 GMT Received: from fmsmsx28.fm.intel.com ([132.233.42.28]) by fmsmsxvs042.fm.intel.com (NAVGW 2.5.2.11) with SMTP id M2002101114532115066 for ; Fri, 11 Oct 2002 14:53:21 -0700 Received: by fmsmsx28.fm.intel.com with Internet Mail Service (5.5.2653.19) id ; Fri, 11 Oct 2002 14:51:39 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Approved-By: "Yang, Lily L" Message-ID: Date: Fri, 11 Oct 2002 14:51:37 -0700 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: "Yang, Lily L" Subject: Re: architecture draft comments To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list Jamal -- Thanks for the comments! Itemized responses or further questions below ... > -paragraph under fig. 5 describes an example of how CE1 > remains in sync with > CE2 via a CE-CE protocol. I dont see it happening this way Could you elaborate on how you think it works? > - section 4.1.2 and how the FE tells the FE manager of it FE > ID. An alternate > is for the FE to be told what its FE (perhaps by the CE via > CE Manager to FE > Manager). There are other ways. I guess that you meant FE can be assigned FE ID by its FE manager? I agree there are many ways go about doing that. It is outside ForCES anyway. > -Fig 9: I could see msg 2 "Let me join" or "I wish to > register" as being the > first msg. Absolutely. > There should be clear text that indicates that the > architecture examples are > just that - mere examples; that try to describe from a > sky-high view the > concepts that might be employed. Agree. Will make sure such text is there. Lily > -----Original Message----- > From: Jamal Hadi Salim [mailto:hadi@znyx.com] > Sent: Friday, October 11, 2002 6:49 AM > To: FORCES@PEACH.EASE.LSOFT.COM > Subject: architecture draft comments > > > > I realize that examples in the draft are just that, examples; > however, I am a > little concerned about some of the "suggestions" made and and > how they may be > interpreted as the way the protocol should behave. For instance: > > -paragraph under fig. 5 describes an example of how CE1 > remains in sync with > CE2 via a CE-CE protocol. I dont see it happening this way > - section 4.1.2 and how the FE tells the FE manager of it FE > ID. An alternate > is for the FE to be told what its FE (perhaps by the CE via > CE Manager to FE > Manager). There are other ways. > -Fig 9: I could see msg 2 "Let me join" or "I wish to > register" as being the > first msg. > > I could go on but hopefully with those few examples i have > illustrated my > concern. > There should be clear text that indicates that the > architecture examples are > just that - mere examples; that try to describe from a > sky-high view the > concepts that might be employed. > I just scanned the draft, so if that text is already there > then ignore me. > > cheers, > jamal > > > From owner-forces@PEACH.EASE.LSOFT.COM Fri Oct 11 20:40:21 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA08105 for ; Fri, 11 Oct 2002 20:40:15 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <2.00768E24@cherry.ease.lsoft.com>; Fri, 11 Oct 2002 20:42:21 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 28867962 for FORCES@PEACH.EASE.LSOFT.COM; Fri, 11 Oct 2002 20:42:18 -0400 Received: from 143.182.124.5 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Fri, 11 Oct 2002 20:42:17 -0400 Received: from fmsmsxvs043.fm.intel.com (fmsmsxvs043.fm.intel.com [132.233.42.129]) by petasus.ch.intel.com (8.11.6/8.11.6/d: solo.mc,v 1.47 2002/10/10 20:35:15 dmccart Exp $) with SMTP id g9C0hor05497 for ; Sat, 12 Oct 2002 00:43:50 GMT Received: from fmsmsx019.fm.intel.com ([132.233.42.130]) by fmsmsxvs043.fm.intel.com (NAVGW 2.5.2.11) with SMTP id M2002101117403621424 for ; Fri, 11 Oct 2002 17:40:36 -0700 Received: by fmsmsx019.fm.intel.com with Internet Mail Service (5.5.2653.19) id <4W0VCWA4>; Fri, 11 Oct 2002 17:42:15 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/mixed; boundary="----_=_NextPart_000_01C27188.309B72E0" Approved-By: "Yang, Lily L" Message-ID: Date: Fri, 11 Oct 2002 17:42:06 -0700 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: "Yang, Lily L" Subject: new rev of framework draft, please comment To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_000_01C27188.309B72E0 Content-Type: text/plain; charset="ISO-8859-1" Hi, all After hearing back from the list, I've incorporated most of the comments into this new version. Please take a look and provide feedback. I will wait a bit before submitting this version. Summary of changes since ver02: 1) Per suggestion from Jamal, the following is added at the end of Section 3 (Introduction): "Whenever necessary, this document uses many examples to illustrate the issues and/or possible solutions in ForCES. These examples are intended to be just examples, and should not be taken as the only or definite ways of doing certain things. It is expected that separate document will be produced by the ForCES working group to specify the ForCES protocol(s)." 2) Per suggestion from Jeelani and others about the "off-loaded functions", the following is added at the end of Section 3 (Introduction): "For performance or scalability reasons, portions of the control plane functions that need faster response may be moved from the CEs and off-loaded onto the FEs. For example in OSPF, the Hello protocol packets are generated and processed periodically. When done at CEs, the inbound Hello packets have to traverse from the external interfaces at the FEs to the CEs via the internal CE-FE channel. Similarly, the outbound Hello packets have to go from the CEs to the FEs and to the external interfaces. Frequent Hello updates place heavy processing overhead on the CEs and can overwhelm the CE-FE channel as well. Since typically there are far more FEs than CEs in a router, when off-loaded, the Hello packets are processed in a much more distributed and scalable fashion. Once off-loaded onto the forwarding plane, such CE functions, even though logically belong to the control plane, now become part of the FE logical functions. Just like the other logical functions performed by FEs (e.g., IPv4 forwarding, classification, etc.), such off-loaded functions must be expressed as part of the FE model so that the CEs can take advantage of these off-loaded functions when present on the FEs." 3) Per suggestion from Ram Gopal, the explicit security exchange is deleted from the message exchange example (Fig 9) in Section 4.2.2. 4) More editorial changes here and there. All minor. Thanks, Lily <> ------_=_NextPart_000_01C27188.309B72E0 Content-Type: text/plain; name="draft-ietf-forces-framework-03.txt" Content-Disposition: attachment; filename="draft-ietf-forces-framework-03.txt" Content-Transfer-Encoding: quoted-printable Internet Draft L. Yang=20 Expiration: April 2003 Intel Corp.=20 File: draft-ietf-forces-framework-03.txt R. Dantu=20 Working Group: ForCES Univ. of Texas = Dallas=20 T. Anderson=20 Intel Corp. =20 Oct 2002 = =20 =20 =20 ForCES Architectural Framework=20 =20 =20 =20 draft-ietf-forces-framework-03.txt=20 =20 =20 =20 =20 Status of this Memo=20 =20 This document is an Internet-Draft and is in full conformance = with=20 all provisions of Section 10 of RFC2026. Internet-Drafts are=20 working documents of the Internet Engineering Task Force = (IETF), its=20 areas, and its working groups. Note that other groups may also = distribute working documents as Internet-Drafts.=20 =20 Internet-Drafts are draft documents valid for a maximum of six=20 months and may be updated, replaced, or obsoleted by other = documents=20 at any time. It is inappropriate to use Internet-Drafts as=20 reference material or to cite them other than as ``work in=20 progress.''=20 =20 The list of current Internet-Drafts can be accessed at=20 http://www.ietf.org/ietf/1id-abstracts.txt.=20 =20 The list of Internet-Draft Shadow Directories can be accessed = at =20 http://www.ietf.org/shadow.html.=20 =20 Abstract=20 =20 This document defines the architectural framework for the = ForCES=20 (Forwarding and Control Element Separation) network elements = (NE),=20 and identifies the associated entities and the interaction = among=20 them. This framework is intended to satisfy the requirements=20 specified in the ForCES requirements draft [FORCES-REQ].=20 =20 Table of Contents=20 =20 1. = Definitions.....................................................2=20 2. Introduction to Forwarding and Control Element Separation=20 = (ForCES)...........................................................4=20 3. = Architecture....................................................7=20 3.1. Control Elements and Fr Reference = Point....................8=20 =20 =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 3.2. Forwarding Elements and Fi reference = point.................9=20 3.3. CE = Managers...............................................11=20 3.4. FE = Managers...............................................11=20 4. Operational = Phases.............................................12=20 4.1. Pre-association = Phase.....................................12=20 4.1.1. Fl Reference = Point...................................12=20 4.1.2. Ff Reference = Point...................................13=20 4.1.3. Fc Reference = Point...................................13=20 4.2. Post-association Phase and Fp reference = point.............14=20 4.2.1. Proximity and Interconnect between CEs and = FEs.......14=20 4.2.2. Association = Establishment............................14=20 4.2.3. Steady-state = Communication...........................15=20 4.2.4. Data Packets across Fp reference = point...............16=20 4.2.5. Proxy = FE.............................................17=20 4.3. Association = Re-establishment..............................17=20 5. Applicability to = RFC1812.......................................18=20 5.1. General Router = Requirements...............................18=20 5.2. Link = Layer................................................19=20 5.3. Internet Layer = Protocols..................................20=20 5.4. Internet Layer = Forwarding.................................20=20 5.5. Transport = Layer...........................................21=20 5.6. Application Layer -- Routing = Protocols....................21=20 5.7. Application Layer -- Network Management = Protocol..........22=20 6. = Summary........................................................22=20 7. Security = Considerations........................................22=20 8. Intellectual Property = Right....................................23=20 9. = References.....................................................23=20 9.1. Normative = References......................................23=20 9.2. Informative = References....................................23=20 10. = Acknowledgments...............................................23=20 11. Authors' = Addresses............................................23=20 =20 Conventions used in this document =20 =20 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL = NOT", =20 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" = in=20 this document are to be interpreted as described in [RFC-2119]. = =20 1. Definitions=20 =20 A set of terminology associated with the ForCES requirements is = defined in [FORCES-REQ] and we only include the ones that are = most=20 relevant to this document here. =20 =20 Forwarding Element (FE) - A logical entity that implements the=20 ForCES protocol. FEs use the underlying hardware to provide = per- packet processing and handling as directed by a CE via the = ForCES=20 protocol. =20 =20 Control Element (CE) - A logical entity that implements the = ForCES=20 protocol and uses it to instruct one or more FEs how to process = packets. CEs handle functionality such as the execution of = control =20 and signaling protocols. =20 =20 Yang, et. al. Expires April 2003 [Page 2] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 =20 ForCES Network Element (NE) - An entity composed of one or more = CEs=20 and one or more FEs. To entities outside a NE, the NE = represents a=20 single point of management. Similarly, a NE usually hides its=20 internal organization from external entities. =20 =20 Pre-association Phase - The period of time during which a FE = Manager=20 (see below) and a CE Manager (see below) are determining which = FE=20 and CE should be part of the same network element.=20 =20 Post-association Phase - The period of time during which a FE = does=20 know which CE is to control it and vice versa, including the = time=20 during which the CE and FE are establishing communication with = one=20 another. =20 =20 ForCES Protocol - While there may be multiple protocols used = within=20 the overall ForCES architecture, the term "ForCES protocol" = refers=20 only to the ForCES post-association phase protocol (see below). = =20 =20 ForCES Post-Association Phase Protocol - The protocol used for = post- association phase communication between CEs and FEs. This = protocol=20 does not apply to CE-to-CE communication, FE-to-FE = communication, or=20 to communication between FE and CE managers. The ForCES = protocol is=20 a master-slave protocol in which FEs are slaves and CEs are = masters. =20 This protocol includes both the management of the communication = channel (e.g., connection establishment, heartbeats) and the = control=20 messages themselves. This protocol could be a single protocol = or=20 could consist of multiple protocols working together. =20 =20 FE Manager - A logical entity that operates in the = pre-association=20 phase and is responsible for determining to which CE(s) a FE = should=20 communicate. This process is called CE discovery and may = involve=20 the FE manager learning the capabilities of available CEs. A = FE=20 manager may use anything from a static configuration to a pre- association phase protocol (see below) to determine which CE(s) = to=20 use. Being a logical entity, a FE manager might be physically=20 combined with any of the other logical entities mentioned in = this=20 section. =20 =20 CE Manager - A logical entity that operates in the = pre-association =20 phase and is responsible for determining to which FE(s) a CE = should =20 communicate. This process is called FE discovery and may = involve=20 the CE manager learning the capabilities of available FEs. A = CE=20 manager may use anything from a static configuration to a pre- association phase protocol (see below) to determine which FE to = use. =20 Being a logical entity, a CE manager might be physically = combined=20 with any of the other logical entities mentioned in this = section. =20 =20 Pre-association Phase Protocol - A protocol between FE managers = and=20 CE managers that is used to determine which CEs or FEs to use. = A=20 pre-association phase protocol may include a CE and/or FE = capability=20 discovery mechanism. Note that this capability discovery = process is=20 wholly separate from (and does not replace) that used within = the=20 =20 Yang, et. al. Expires April 2003 [Page 3] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 ForCES protocol. However, the two capability discovery = mechanisms=20 may utilize the same FE model. =20 =20 FE Model - A model that describes the logical processing = functions=20 of a FE. =20 =20 ForCES Protocol Element - A FE or CE. =20 =20 2. Introduction to Forwarding and Control Element Separation = (ForCES) =20 =20 An IP network element (NE) appears to external entities as a=20 monolithic piece of network equipment, e.g., a router, NAT,=20 firewall, or load balancer. Internally, however, an IP network=20 element (NE) (such as a router) is composed of numerous = logically=20 separated entities that cooperate to provide a given = functionality=20 (such as routing). Two types of network element components = exist:=20 control element (CE) in control plane and forwarding element = (FE) in=20 forwarding plane (or data plane). Forwarding elements = typically are=20 ASIC, network-processor, or general-purpose processor-based = devices=20 that handle data path operations for each packet. Control = elements=20 are typically based on general-purpose processors that provide=20 control functionality like routing and signaling protocols. =20 =20 ForCES aims to define a framework and associated protocol(s) to = standardize information exchange between the control and = forwarding=20 plane. Having standard mechanisms allows CEs and FEs to become=20 physically separated standard components. This physical = separation=20 accrues several benefits to the ForCES architecture. Separate=20 components would allow component vendors to specialize in one=20 component without having to become experts in all components.=20 Standard protocol also allows the CEs and FEs from different=20 component vendors to interoperate with each other and hence it=20 becomes possible for system vendors to integrate together the = CEs=20 and FEs from different component suppliers. This = interoperability=20 translates into a lot more design choices and flexibility to = the=20 system vendors. Overall, ForCES will enable rapid innovation in = both=20 the control and forwarding planes while maintaining=20 interoperability. Scalability is also easily provided by this=20 architecture in that additional forwarding or control capacity = can=20 be added to existing network elements without the need for = forklift=20 upgrades.=20 =20 One example of such physical separation is at the blade level.=20 Figure 1 shows such an example configuration of a router, with = two=20 control blades and multiple router (forwarding) blades, all=20 interconnected into a switch fabric backplane. In such chassis=20 configuration, the control blades are the CEs while the router=20 blades are FEs, and the switch fabric backplane provides the=20 physical interconnect for all the blades. Control blade A may = be the=20 primary CE while control blade B is the backup CE providing=20 redundancy. It is also possible to have a redundant switch = fabric=20 for high availability support. Routers today with this kind of=20 configuration use proprietary interface for messaging between = CEs=20 =20 Yang, et. al. Expires April 2003 [Page 4] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 and FEs. The goal of ForCES is to replace such proprietary = interface=20 with a standard protocol. With a standard protocol like ForCES=20 implemented on all blades, it becomes possible for control = blades=20 from vendor X and routing blades from vendor Y to work = seamlessly=20 together in one chassis. =20 =20 ------------------------- -------------------------=20 | Control Blade A | | Control Blade B |=20 | (CE) | | (CE) |=20 ------------------------- -------------------------=20 ^ | ^ |=20 | | | |=20 | V | V=20 ---------------------------------------------------------=20 | Switch Fabric Backplane |=20 ---------------------------------------------------------=20 ^ | ^ | ^ | = | | | | =85 | | = =20 | V | V | V = ------------ ------------ ------------ = |Router | |Router | |Router |=20 |Blade #1 | |Blade #2 | |Blade #N |=20 | (FE) | | (FE) | | (FE) |=20 ------------ ------------ ------------ = ^ | ^ | ^ | = | | | | =85 | | = =20 | V | V | V = =20 Figure 1. A router configuration example with separate = blades.=20 =20 =20 ------- -------=20 | CE1 | | CE2 |=20 ------- -------=20 ^ ^=20 | |=20 V V=20 = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Ethernet=20 ^ ^ =85 ^=20 | | |=20 V V V=20 ------- ------- --------=20 | FE#1| | FE#2| | FE#n |=20 ------- ------- --------=20 ^ | ^ | ^ | =20 | | | | | | =20 | V | V | V =20 =20 Figure 2. A router configuration example with separate = boxes.=20 =20 Another level of physical separation between the CEs and FEs = can be=20 at the box level. In such configuration, all the CEs and FEs = are=20 physically separated boxes, interconnected with some kind of = high=20 =20 Yang, et. al. Expires April 2003 [Page 5] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 speed LAN connection (like Gigabit Ethernet). These separated = CEs=20 and FEs are only one hop away from each other within a local = area=20 network. The CEs and FEs communicate to each other by running=20 ForCES, and the collection of these CEs and FEs together become = one=20 routing unit to the external world. Figure 2 shows such an = example.=20 =20 In both examples shown here, the same physical interconnect is = used=20 for both CE-to-FE and FE-to-FE communication. However, that = does not=20 have to be the case. One reason to use different interconnect = is=20 that CE-to-FE interconnect does not have to be as fast as the = FE-to- FE interconnect, so the more expensive fast connections can be = saved=20 for FE-to-FE only. The separate interconnects may also provide = reliability and redundancy benefits for the NE.=20 =20 -------------------------------------------------=20 | | | | | | |=20 |OSPF |RIP |BGP |CAC |LDP |=85 |=20 | | | | | | |=20 -------------------------------------------------=20 | ForCES Interface |=20 -------------------------------------------------=20 ^ ^ =20 ForCES | |data =20 control | |packets =20 messages| |(e.g., routing packets) =20 v v=20 -------------------------------------------------=20 | ForCES Interface |=20 -------------------------------------------------=20 | | | | | | |=20 |LPM Fwd|Meter |Shaper |NAT |Classi-|=85 |=20 | | | | |fier | |=20 -------------------------------------------------=20 | FE resources |=20 -------------------------------------------------=20 =20 Figure 3. Examples of CE and FE functions=20 =20 Some examples of control functions that can be implemented in = the CE=20 include routing protocols like RIP, OSPF and BGP, control and=20 signaling protocols like CAC (Call Admission Control), LDP = (Label=20 Distribution Protocol) for MPLS, etc. Examples of forwarding=20 functions in FE include LPM (longest prefix match) forwarder,=20 classifiers, traffic shaper, meter, NAT, etc. Figure 3 shows a=20 diagram with examples in both CE and FE. Any given NE may = contain=20 one or many of these CE and FE functions in it. The diagram = also=20 shows that ForCES protocol is used to transport both the = control=20 messages for ForCES itself and the data packets that are=20 originated/destined from/to the control functions in CE (e.g.,=20 routing packets). Section 4.2.4 provides more detail on this. =20 =20 A set of requirements for control and forwarding separation is=20 identified in [FORCES-REQ]. This document describes a ForCES=20 =20 Yang, et. al. Expires April 2003 [Page 6] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 architecture that satisfies the architectural requirements of = that=20 document and defines a framework for ForCES network elements = and the=20 associated entities to facilitate protocol definition. Whenever = necessary, this document uses many examples to illustrate the = issues=20 and/or possible solutions in ForCES. These examples are = intended to=20 be just examples, and should not be taken as the only or = definite=20 ways of doing certain things. It is expected that separate = document=20 will be produced by the ForCES working group to specify the = ForCES=20 protocol(s).=20 =20 3. Architecture=20 =20 This section defines the ForCES architectural framework and the = associated logical components. This ForCES framework defines=20 components of ForCES NEs including several ancillary = components. =20 These components may be connected in different kinds of = topologies=20 for flexible packet processing. =20 =20 --------------------------------------- = | ForCES Network Element | = -------------- Fc | -------------- -------------- | = | CE Manager |---------+-| CE 1 |------| CE 2 | | = -------------- | | | Fr | | | = | | -------------- -------------- | = | Fl | | | Fp / | = | | Fp| |----------| / | = | | | |/ | = | | | | | = | | | Fp /|----| | = | | | /--------/ | | = -------------- Ff | -------------- -------------- | = | FE Manager |---------+-| FE 1 | Fi | FE 2 | | = -------------- | | |------| | | = =20 | -------------- -------------- | = | | | | | | | | | | = ----+--+--+--+----------+--+--+--+----- = | | | | | | | |=20 | | | | | | | |=20 Fi/f Fi/f=20 =20 Figure 4. ForCES Architectural Diagram=20 =20 The diagram in Figure 4 shows the logical components of the = ForCES=20 architecture and their relationships. There are two kinds of=20 components inside a ForCES network element: control element = (CE) and=20 forwarding element (FE). The framework allows multiple = instances of=20 CE and FE inside one NE. Each FE contains one or more physical = media=20 interfaces for receiving and transmitting packets from/to the=20 external world. The aggregation of these FE interfaces becomes = the=20 NE=92s external interfaces. In addition to the external = interfaces,=20 there must also exist some kind of interconnect within the NE = so=20 that the CE and FE can communicate with each other, and one FE = can=20 forward packets to another FE. The diagram also shows two = entities=20 =20 Yang, et. al. Expires April 2003 [Page 7] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 outside of the ForCES NE: CE Manager and FE Manager. These two=20 entities provide configuration to the corresponding CE or FE in = the=20 pre-association phase (see Section 5.1). There is no defined = role=20 for FE Manager and CE Manager in post-association phase, thus = these=20 logical components are not considered part of the ForCES NE. =20 =20 For convenience, the logical interactions between these = components=20 are labeled by reference points Fp, Fc, Ff, Fr, Fl, and Fi, as = shown=20 in Figure 4. The FE external interfaces are labeled as Fi/f. = More=20 detail is provided in Section 4 and 5 for each of these = reference=20 points. All these reference points are important in = understanding=20 the ForCES architecture, however, the ForCES protocol is only=20 defined over one reference point -- Fp. =20 =20 The interface between two ForCES NEs is identical to the = interface=20 between two conventional routers and these two NEs exchange the = protocol packets through the external interfaces at Fi/f. = ForCES NEs=20 connect to existing routers transparently.=20 =20 3.1. Control Elements and Fr Reference Point=20 =20 It is not necessary to define any protocols across the Fr = reference=20 point to enable control and forwarding separation for simple=20 configurations like single CE and multiple FEs. However, this=20 architecture permits multiple CEs to be present in a network=20 element. In cases where an implementation uses multiple CEs, = the=20 invariant that the CEs and FEs together appear as a single NE = must=20 be maintained.=20 =20 Multiple CEs may be used for redundancy, load sharing, = distributed=20 control, or other purposes. Redundancy is the case where one = or=20 more CEs are prepared to take over should an active CE fail. = Load=20 sharing is the case where two or more CEs are concurrently = active=20 and where any request that can be serviced by one of the CEs = can=20 also be serviced by any of the other CEs. For both redundancy = and=20 load sharing, the CEs involved are equivalently capable. The = only=20 difference between these two cases is in terms of how many = active=20 CEs there are. Distributed control is the case where two or = more=20 CEs are concurrently active but where certain requests can only = be=20 serviced by certain CEs.=20 =20 When multiple CEs are employed in a ForCES NE, their internal=20 organization is considered an implementation issue that is = beyond=20 the scope of ForCES. CEs are wholly responsible for = coordinating=20 amongst themselves via the Fr reference point to provide = consistency=20 and synchronization. However, ForCES does not define the=20 implementation or protocols used between CEs, nor does it = define how=20 to distribute functionality among CEs. Nevertheless, ForCES = will=20 support mechanisms for CE redundancy or fail over, and it is=20 expected that vendors will provide redundancy or fail over = solutions=20 within this framework.=20 =20 =20 =20 Yang, et. al. Expires April 2003 [Page 8] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 3.2. Forwarding Elements and Fi reference point=20 =20 FEs perform per-packet processing and handling as directed by = CEs.=20 FEs have no initiative of their own. Instead, FEs are slaves = and=20 only do as they are told. FEs may communicate with one or more = CEs=20 concurrently across reference point Fp. FEs have no notion of = CE=20 redundancy, load sharing, or distributed control. Instead, FEs = accept commands from any CE authorized to control them, and it = is up=20 to the CEs to coordinate among themselves to achieve = redundancy,=20 load sharing or distributed control. The idea is to keep FEs as = simple and dumb as possible so that FEs can focus its resource = on=20 the packet processing functions. =20 =20 =20 ------- Fr -------=20 | CE1 | ------| CE2 |=20 ------- -------=20 | \ / |=20 | \ / |=20 | \ / |=20 | \/Fp |=20 | /\ |=20 | / \ |=20 | / \ |=20 ------- Fi -------=20 | FE1 |<----->| FE2 |=20 ------- -------=20 =20 Figure 5. CE redundancy example.=20 =20 For example, in Figure 5, FE1 and FE2 can be configured to = accept=20 commands from both the primary CE (CE1) and the backup CE = (CE2). At=20 the beginning, CE1 issues commands to FEs while CE2 silently = remains=20 in sync with CE1 via CE to CE protocol over Fr reference point. = When=20 CE1 fails, CE2 detects it and starts to take over. Before CE2 = starts=20 issuing commands to the FEs, it might need to recheck the FEs' = state=20 and instruct FEs whether or not it is ok to preserve their = current=20 state. =20 =20 Distributed control can be achieved in the similar fashion, = without=20 much intelligence on the part of FEs. For example, FEs can be=20 configured to detect RSVP and BGP protocol packets, and forward = RSVP=20 packets to one CE and BGP packets to another CE. Hence, FEs may = need=20 to do packet filtering for forwarding packets to specific CEs.=20 =20 This architecture permits multiple FEs to be present in a NE.=20 [FORCES-REQ] dictates that the ForCES protocol must be able to = scale=20 to at least hundreds of FEs (see [FORCES-REQ] Section 5, = requirement=20 #11). Each of these FEs may potentially have a different set of = packet processing functions, with different media interfaces. = FEs=20 are responsible for basic maintenance of layer-2 connectivity = with=20 other FEs and with external entities. Many layer-2 media = include=20 sophisticated control protocols. The FORCES protocol (over the = Fp=20 =20 Yang, et. al. Expires April 2003 [Page 9] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 reference point) will be able to carry messages for such = protools so=20 that, in keeping with the "dumb FE=92=92 model, the CE can = provide=20 appropriate intelligence and control over these media.=20 =20 When multiple FEs are present, ForCES requires that packets = must be=20 able to arrive at the NE by one FE and leave the NE via a = different=20 FE (See [FORCES-REQ], Section 5, Requirement #3). Packets that = enter the NE via one FE and leave the NE via a different FE are = transferred between FEs across the Fi reference point. The Fi=20 reference point is a separate protocol from the Fp reference = point=20 and is not currently defined by the ForCES architecture.=20 =20 =20 =20 ----------------- =20 | CE | =20 ----------------- =20 ^ ^ ^=20 / | \=20 / v \=20 / ------- \=20 / +->| FE3 |<-+ \=20 / | | | | \=20 v | ------- | v=20 ------- | | -------=20 | FE1 |<-+ +->| FE2 |=20 | |<--------------->| |=20 ------- -------=20 ^ | ^ |=20 | | | |=20 | v | v=20 =20 (a) Full mesh among FE1, FE2 and FE3.=20 =20 =20 ----------- =20 | CE | =20 ----------- =20 ^ ^ ^ ^=20 / | | \=20 /------ | | ------\=20 v v v v=20 ------- ------- ------- -------=20 | FE1 |<->| FE2 |<->| FE3 |<->| FE4 |=20 ------- ------- ------- -------=20 ^ | ^ | ^ | ^ |=20 | | | | | | | |=20 | v | v | v | v=20 =20 (b) Multiple FEs in a daisy chain=20 =20 =20 =20 =20 Yang, et. al. Expires April 2003 [Page 10] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 ^ |=20 | v=20 ----------- =20 | FE1 |<-----------------------| =20 ----------- |=20 ^ ^ |=20 / \ | =20 | ^ / \ ^ | V=20 v | v v | v ---------- = --------- --------- | | = | FE2 | | FE3 |<------------>| CE | = --------- --------- | | = ^ ^ ^ ---------- = | \ / ^ ^=20 | \ / | |=20 | v v | |=20 | ----------- | |=20 | | FE4 |<----------------------| | =20 | ----------- |=20 | | ^ |=20 | v | |=20 | |=20 |----------------------------------------|=20 =20 (c) Multiple FEs connected by a ring=20 =20 Figure 6. Some examples of FE topology.=20 =20 FEs could be connected in different kinds of topologies and = packet=20 processing may spread across several FEs in the topology. = Hence,=20 logical packet flow may be different from physical FE topology. = Figure 6 provides some topology examples. When it is necessary = to=20 forward packets between FEs, the CE needs to understand the FE=20 topology. The FE topology can be queried from the FEs by CEs. =20 =20 3.3. CE Managers=20 =20 CE managers are responsible for determining which FEs a CE = should=20 control. It is legitimate for CE managers to be hard-coded = with the=20 knowledge of with which FEs its CEs should communicate. A CE = manager=20 may also be physically embedded into a CE and be implemented as = a=20 simple keypad or other direct configuration mechanism on the = CE. =20 Finally, CE managers may be physically and logically separate=20 entities that configure the CE with FE information via such=20 mechanisms as COPS-PR [RFC3084] or SNMP [RFC1157].=20 =20 3.4. FE Managers=20 =20 FE managers are responsible for determining to which CE any=20 particular FE should initially communicate. Like CE managers, = no=20 restrictions are placed on how a FE manager decides to which CE = its=20 FEs should communicate, nor are restrictions placed on how FE=20 managers are implemented.=20 =20 Yang, et. al. Expires April 2003 [Page 11] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 =20 =20 4. Operational Phases=20 =20 Both FEs and CEs require some configuration in place before = they can=20 start information exchange and function as a coherent network=20 element. Two operational phases are identified in this = framework --=20 pre-association and post-association. =20 =20 4.1.Pre-association Phase=20 =20 Pre-association phase is the period of time during which a FE=20 Manager and a CE Manager are determining which FE and CE should = be=20 part of the same network element. The protocols used during = this=20 phase may include all or some of the message exchange over Fl, = Ff=20 and Fc reference points. However, all these may be optional and = none=20 of this is within the scope of ForCES protocol.=20 =20 4.1.1. Fl Reference Point=20 =20 =20 FE Manager FE CE Manager CE=20 | | | |=20 | | | |=20 |(security exchange) | |=20 1|<------------------------------>| |=20 | | | |=20 |(a list of CEs and their attributes) |=20 2|<-------------------------------| |=20 | | | |=20 |(a list of FEs and their attributes) |=20 3|------------------------------->| |=20 | | | |=20 | | | |=20 |<----------------Fl------------>| |=20 =20 Figure 7. An example of message exchange over Fl reference = point=20 =20 CE managers and FE managers may communicate across the Fl = reference=20 point in the pre-association phase in order to determine which = CEs=20 and FEs should communicate with each other. Communication = across=20 the Fl reference point is optional in this architecture. No=20 requirements are placed on this reference point.=20 =20 CE managers and FE managers may be operated by different = entities. =20 The operator of the CE manager may not want to divulge, except = to=20 specified FE managers, any characteristics of the CEs it = manages. =20 Similarly, the operator of the FE manager may not want to = divulge FE=20 characteristics, except to authorized entities. As such, CE=20 managers and FE managers may need to authenticate one another. = Subsequent communication between CE managers and FE managers = may=20 require other security functions such as privacy, = non-repudiation,=20 freshness, and integrity.=20 =20 Yang, et. al. Expires April 2003 [Page 12] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 =20 Once the necessary security functions have been performed, the = CE=20 and FE managers communicate to determine which CEs and FEs = should=20 communicate with each other. At the very minimum, the CE and = FE=20 managers need to learn of the existence of available FEs and = CEs=20 respectively. This discovery process may or may not entail one = or=20 both managers learning the capabilities of the discovered = ForCES=20 protocol elements. Figure 7 shows an example of possible = message=20 exchange between CE manager and FE manager over Fl reference = point.=20 =20 4.1.2. Ff Reference Point=20 =20 FE Manager FE CE Manager CE=20 | | | |=20 | | | |=20 |(security exchange) |(security exchange)=20 1|<------------>|authentication = 1|<----------->|authentication | | | |=20 |(FE ID, attributes) |(CE ID, attributes)=20 2|<-------------|request 2|<------------|request=20 | | | |=20 3|------------->|response 3|------------>|response=20 |(corresponding CE ID) |(corresponding FE ID)=20 | | | |=20 | | | |=20 |<-----Ff----->| |<-----Fc---->|=20 =20 Figure 8. Examples of message exchange=20 over Ff and Fc reference points.=20 =20 The Ff reference point is used to inform forwarding elements of = the=20 association decisions made by FE managers in pre-association = phase. =20 Only authorized entities may instruct a FE with respect to = which CE=20 should control it. Therefore, privacy, integrity, freshness, = and=20 authentication are necessary between FE manager and FEs when = the FE=20 manager is remote to the FE. Once the appropriate security has = been=20 established, FE manager instructs FEs across this reference = point to=20 join a new NE or to disconnect from an existing NE. Figure 8 = shows=20 example of message exchange over Ff reference point.=20 =20 Note that the FE manager function may be co-located with the FE = (such as by manual keypad entry of the CE IP address), in which = case=20 this reference point is reduced to a built-in function.=20 =20 4.1.3. Fc Reference Point=20 =20 The Fc reference point is used to inform control elements of = the=20 association decisions made by CE managers in pre-association = phase. =20 When the CE manager is remote, only authorized entities may = instruct=20 a CE to control certain FEs. Privacy, integrity, freshness and = authentication are also required across this reference point in = such=20 a configuration. Once appropriate security has been = established,=20 the CE manager instructs CEs as to which FEs they should = control and=20 =20 Yang, et. al. Expires April 2003 [Page 13] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 how they should control them. Figure 7 shows example of message = exchange over Fc reference point.=20 =20 As with the FE manager and FEs, configurations are possible = where=20 the CE manager and CE are co-located and no protocol is used = for=20 this function.=20 =20 4.2. Post-association Phase and Fp reference point=20 =20 Post-association phase is the period of time during which a FE = and=20 CE have been configured with information necessary to contact = each=20 other and includes both association establishment and = steady-state=20 communication. The communication between CE and FE is performed = across the Fp ("p" meaning protocol) reference point. ForCES=20 protocol is exclusively used for all communication across the = Fp=20 reference point. =20 =20 4.2.1. Proximity and Interconnect between CEs and FEs=20 =20 The ForCES Working Group has made a conscious decision that the = first version of ForCES will not be designed to support=20 configurations where the CE and FE are located arbitrarily in = the=20 network. In particular, ForCES is intended for "very close" = CE/FE=20 localities in IP networks, as defined by ForCES Applicability=20 Statement ([FORCES-APP]). Very Close localities consist of = control=20 and forwarding elements that either are components in the same=20 physical box, or are separated at most by one local network = hop. =20 =20 CEs and FEs can be connected by a variety of interconnect=20 technologies, including Ethernet connections, backplanes, ATM = (cell)=20 fabrics, etc. ForCES should be able to support each of these=20 interconnects (see [FORCES-REQ] Section 5, requirement #1). = ForCES=20 will make use of an existing RFC2914 compliant L4 protocol with = adequate reliability, security and congestion control (e.g. = TCP,=20 SCTP) for transport purposes.=20 =20 4.2.2. Association Establishment=20 =20 As an example, figure 9 shows some of the message exchange that = may=20 happen before the association between the CE and FE is fully=20 established. Either the CE or FE can initiate the connection. = The FE=20 needs to inform the CE of its own capability and its topology = in=20 relation to other FEs. The capability of the FE is represented = by=20 the FE model, described in another separate document. The model = would allow a FE to describe what kind of packet processing=20 functions it contains, in what order the processing happens, = what=20 kinds of configurable parameters it allows, what statistics it=20 collects and what events it might throw, etc. Once such = information=20 is available to the CE, the CE sends all necessary = configuration to=20 the FE so that the FE can start receiving and processing = packets=20 correctly. For example, the CE might need to send a snapshot of = the=20 current forwarding table to the FE so that the FE can start = routing=20 packets correctly. Once FE starts accepting packets for = processing,=20 =20 Yang, et. al. Expires April 2003 [Page 14] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 we say the association of this FE with its CE is now = established.=20 From then on, the CE and FE enter steady-state communication.=20 =20 =20 FE CE=20 | |=20 |(Hello, are you there?)|=20 1|<----------------------|=20 | |=20 |(Yes. let me join the NE please.) =20 2|---------------------->|=20 | |=20 |(What kind of FE are you? -- capability query)=20 3|<----------------------|=20 | |=20 |(Here is my FE functions/state: use model to = describe)=20 4|---------------------->|=20 | |=20 |(How are you connected with others? -- topology = query)=20 5|<----------------------|=20 | |=20 |(Here is the topology info)=20 6|---------------------->|=20 | |=20 |(Config for FE, e.g. forwarding table)=20 7|<----------------------|=20 | |=20 |(I am ready to go. Shall I?) =20 8|---------------------->|=20 | |=20 |(Go ahead!) |=20 9|<----------------------|=20 | |=20 =20 Figure 9. Example of message exchange between CE and FE=20 over Fp to establish NE association=20 =20 =20 4.2.3. Steady-state Communication=20 =20 Once an association is established between the CE and FE, the = ForCES=20 protocol is used by the CE and FE over Fp reference point to=20 exchange information to facilitate packet processing. =20 =20 FE CE=20 | |=20 |(Add these new routes.)|=20 1|<----------------------|=20 | |=20 |(Successful.) |=20 2|---------------------->|=20 | |=20 | |=20 =20 Yang, et. al. Expires April 2003 [Page 15] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 |(Query some stats.) |=20 1|<----------------------|=20 | |=20 |(Reply with stats collected.) =20 2|---------------------->|=20 | |=20 | |=20 |(My port is down, with port #.) = =20 1|---------------------->|=20 | |=20 |(Route to this port instead...) =20 2|<----------------------|=20 | |=20 | |=20 =20 Figure 10. Examples of message exchange between CE and FE = over Fp during steady-state communication=20 =20 Based on the information acquired through CEs' control = processing,=20 CEs will frequently need to manipulate the packet-forwarding=20 behaviors of their FE(s) by sending instructions to FEs. For=20 example, Figure 10 shows message exchange examples in which the = CE=20 sends new routes to the FE so that the FE can add them to its=20 forwarding table. The CE may query the FE for statistics = collected=20 by the FE and the FE may notify the CE of important events such = as=20 port failure.=20 =20 4.2.4. Data Packets across Fp reference point=20 =20 Control plane protocol packets (such as RIP, OSPF messages)=20 addressed to any of NE's interfaces are typically redirected by = the=20 receiving FE to its CE, and CE may originate packets and have = its FE=20 deliver them to other NEs. Therefore, ForCES protocol over Fp = not=20 only transports the ForCES protocol messages between CEs and = FEs,=20 but also encapsulates the data packets from control plane = protocols.=20 Moreover, one FE may be controlled by multiple CEs for = distributed=20 control. In this configuration, the control protocols supported = by=20 the FORCES NEs may spread across multiple CEs. For example, one = CE=20 may support routing protocols like OSPF and BGP, while = signaling and=20 admission control protocol like RSVP is supported in another = CE. FEs=20 are configured to recognize and filter these protocol packets = and=20 forward them to the corresponding CE. =20 =20 Figure 11 shows one example of how the BGP packets originated = by=20 router A are passed to router B. In this example, the ForCES=20 protocol is used to transport the packets from the CE to the FE = inside router A, and then from the FE to the CE inside router = B. In=20 light of the fact that the ForCES protocol is responsible to=20 transport both the control messages and the data packets = between the=20 CE and FE over Fp reference point, it is possible to use either = a=20 single protocol or multiple protocols to achieve that.=20 =20 --------------------- ----------------------=20 =20 Yang, et. al. Expires April 2003 [Page 16] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 | | | |=20 | +--------+ | | +--------+ |=20 | |CE(BGP) | | | |CE(BGP) | |=20 | +--------+ | | +--------+ |=20 | | | | ^ |=20 | |Fp | | |Fp |=20 | v | | | |=20 | +--------+ | | +--------+ |=20 | | FE | | | | FE | |=20 | +--------+ | | +--------+ |=20 | | | | ^ |=20 | Router | | | Router | |=20 | A | | | B | |=20 ---------+----------- -----------+----------=20 v ^=20 | |=20 | |=20 ------------------->--------------- =20 =20 Figure 11. Example to show data packet flow between two = NEs.=20 =20 =20 4.2.5. Proxy FE=20 =20 In the case where a physical FE cannot implement (e.g., due to = the=20 lack of a general purpose CPU) the ForCES protocol directly, a = proxy=20 FE can be used in the middle of Fp reference point. This allows = the=20 CE communicate to the physical FE via the proxy by using = ForCES,=20 while the proxy manipulates the physical FE using some = intermediary=20 form of communication (e.g., a non-ForCES protocol or DMA). In = such=20 an implementation, the combination of the proxy and the = physical FE=20 becomes one logical FE entity. =20 =20 4.3. Association Re-establishment=20 =20 FEs and CEs may join and leave NEs dynamically (see = [FORCES-REQ]=20 Section 5, requirements #12). When a FE or CE leaves the NE, = the=20 association with the NE is broken. If the leaving party rejoins = a NE=20 later, to re-establish the association, it may or may not need = to=20 re-enter the pre-association phase. Loss of association can = also=20 happen unexpectedly due to loss of connection between the CE = and the=20 FE. Therefore, the framework allows the bi-directional = transition=20 between these two phases, but the ForCES protocol is only = applicable=20 for the post-association phase. However, the protocol should = provide=20 mechanisms to support association re-establishment (see = [FORCES-REQ]=20 Section 5, requirement #7).=20 =20 The example in Figure 5 is used to illustrate what happens when = the=20 association is broken and later re-established again. Section = 4.2=20 already explains what happens if CE1 fails and how CE2 can take = over. If no CE redundancy is provided, at the association=20 establishment phase FEs need to be told what to do in the case = of CE=20 failure. FEs may be told to stop packet processing all together = if=20 =20 Yang, et. al. Expires April 2003 [Page 17] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 its CE fails. Or, FEs may be told to continue forwarding = packets for=20 a limited time even in the face of CE failure. No matter what = the=20 instruction is, it needs to be part of the configuration when = the=20 association is established. =20 =20 In the same example in Figure 5, assuming CE1 is the working CE = for=20 the moment, what would happen if one of the FEs, say FE1, = leaves the=20 NE temporarily? FE1 may voluntarily decide to leave the = association.=20 Or, FE1 may stop functioning simply due to unexpected failure. = In=20 former case, CE1 receives a "leave-association request" from = FE1. In=20 the latter, CE1 detects the failure of FE1 by some other mean. = In=20 both cases, CE1 keeps a note of such event from FE1 while = continue=20 commanding FE2. When FE1 decides to rejoin again, or when it is = back=20 up again from the failure, FE1 needs to re-discover its master = (CE).=20 This can be achieved by several means. It may re-enter the pre- association phase and get that information from its FE manager. = It=20 may retrieve the previous CE information from its cache, if it = can=20 validate the information freshness. Once it discovers its CE, = it=20 starts message exchange with CE to re-establish the association = just=20 as outlined in Figure 9, with the possible exception that it = might=20 be able to bypass the transport of the complete initialization=20 information. Suppose that FE1 still has its routing table and = other=20 state information from the last association, instead of sending = all=20 the information again from scratch, it can choose to use more=20 efficient mechanism to re-sync up the state with its CE. For=20 example, a checksum of the state might give a quick indication = of=20 whether or not the state is in-sync with its CE. By comparing = its=20 state with CE first, it sends information update only if it is=20 needed.=20 =20 =20 5. Applicability to RFC1812 =20 =20 [FORCES-REQ] Section 5, requirement #9 dictates that "All = proposed=20 ForCES architecture must explain how that architecture supports = all=20 of the router functions as defined in [RFC1812]." RFC1812 = discusses=20 many important requirements for IPv4 routers from the link = layer to=20 the application layer. This section addresses the relevant=20 requirements in RFC1812 for implementing IPv4 routers based on=20 ForCES architecture and explains how ForCES satisfies these=20 requirements.=20 =20 5.1. General Router Requirements=20 =20 Routers have at least two or more logical interfaces. When CEs = and=20 FEs are separated by ForCES within a single NE, some additional = interfaces are needed for intra-NE communications. Figure 12 = shows=20 an example to illustrate that. This NE contains one CE and two = FEs.=20 Each FE has four interfaces; two of them are used for receiving = and=20 transmitting packets to the external world, while the other two = are=20 for intra-NE connections. CE has two logical interfaces #9 and = #10,=20 connected to interfaces #3 and #6 from FE1 and FE2, = respectively.=20 Interface #4 and #5 are connected for FE1-FE2 communication.=20 =20 Yang, et. al. Expires April 2003 [Page 18] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 Therefore, this router NE provides four external interfaces = (#1, 2,=20 7 and 8). =20 =20 IPv4 routers must implement IP to support its packet forwarding = function, which is driven by its FIB (Forwarding Information = Base).=20 This Internet layer forwarding (see [RFC1812] Section 5)=20 functionality naturally belongs to FEs in the ForCES = architecture. =20 =20 ---------------------------------=20 | router NE |=20 | ----------- ----------- |=20 | | FE1 | | FE2 | |=20 | ----------- ----------- |=20 | 1| 2| 3| 4| 5| 6| 7| 8| |=20 | | | | | | | | | |=20 | | | | +----+ | | | |=20 | | | | | | | |=20 | | | 9| 10| | | |=20 | | | -------------- | | |=20 | | | | CE | | | |=20 | | | -------------- | | |=20 | | | | | |=20 -----+--+----------------+--+----=20 | | | | =20 | | | | =20 =20 Figure 12. A router NE example with four = interfaces.=20 =20 =20 A router may implement transport layer protocols (like TCP and = UDP)=20 that are required to support application layer protocols (see=20 [RFC1812] Section 6). One important class of application = protocols=20 is routing protocols (see [RFC1812] Section 7). In ForCES=20 architecture, routing protocols are naturally implemented by = CEs.=20 Routing protocols require routers communicate with each other. = This=20 communication between CEs in different routers is supported in=20 ForCES by FEs' ability to redirect data packets addressed to = routers=20 (i.e., NEs) and CEs' ability to originate packets and have them = delivered by their FEs. This communication occurs across Fp=20 reference point inside each router and between neighboring = routers'=20 external interfaces, as illustrated in Figure 11. =20 =20 5.2.Link Layer=20 =20 Since FEs own all the external interfaces for the router, FEs = need=20 to conform to the link layer requirements in RFC1812. Arguably, = ARP=20 support may be implemented in either CEs or FEs. As we will see = later, a number of behaviors that RFC1812 mandates fall into = this=20 category -- they may be performed by the FE and may be = performed by=20 the CE. A general guideline is needed to ensure = interoperability=20 between separated control and forwarding planes. The guideline = we=20 offer here is that in general CEs are required to be capable of = these kind of operations while FEs may or may not choose to=20 =20 Yang, et. al. Expires April 2003 [Page 19] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 implement them. FE model should indicate its capabilities in = this=20 regard so that CEs can decide where these functions are = implemented.=20 =20 Interface parameters, including MTU, IP address, etc., must be=20 configurable by CEs via ForCES. CEs must be able to determine=20 whether a physical interface in an FE is available to send = packets=20 or not. FEs must also inform CEs the status change of the = interfaces=20 (like link up/down) via ForCES.=20 =20 5.3.Internet Layer Protocols=20 =20 Both FEs and CEs must implement IP protocol and all mandatory=20 extensions as RFC1812 specified. CEs should implement IP = options=20 like source route and record route while FEs may choose to = implement=20 those as well. Timestamp option should be implemented by FEs to = insert the timestamp most accurately. FE must interpret the IP=20 options that it understands and preserve the rest unchanged for = use=20 by CEs. Both FEs and CEs might choose to silently discard = packets=20 without sending ICMP errors, but such events should be logged = and=20 counted. FEs may report statistics for such events to CEs via=20 ForCES. =20 =20 When multiple FEs are involved to process packets, the = appearance of=20 single NE must be strictly maintained. For example, = Time-To-Live=20 (TTL) must be decremented only once within a single NE. For = example,=20 it can be always decremented by the last FE with egress = function. =20 =20 FEs must receive and process normally any packets with a = broadcast=20 destination address or a multicast destination address that the = router has asked to receive. When IP multicast is supported in=20 routers, IGMP is implemented in CEs. CEs are also required of = ICMP=20 support, while it is optional for FEs to support ICMP. Such an=20 option can be communicated to CEs as part of the FE model.=20 Therefore, FEs can always rely upon CEs to send out ICMP error=20 messages, but FEs also have the option to generate ICMP error=20 messages themselves. =20 =20 5.4.Internet Layer Forwarding=20 =20 IP forwarding is implemented by FEs. When the routing table is=20 updated at CEs, ForCES is used to send the new route entries = from=20 CEs to FEs. Each FE has its own forwarding table and uses this = table=20 to direct packets to the next hop interface. =20 =20 Upon receiving IP packets, FE verifies the IP header and = processes=20 most of the IP options. Some options can't be processed until = the=20 routing decision has been made. Routing decision is made after=20 examining the destination IP address. If the destination = address=20 belongs to the router itself, the packets are filtered and = either=20 processed locally or forwarded to CE, depending upon the=20 instructions set-up by CE. Otherwise, FE determines the next = hop IP=20 address by looking up in its forwarding table. FE also = determines=20 the network interface it uses to send the packets. Sometimes FE = may=20 =20 Yang, et. al. Expires April 2003 [Page 20] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 need to forward the packets to another FE before packets can be = forwarded out to the next hop. Right before packets are = forwarded=20 out to the next hop, FE decrements TTL by 1 and processes any = IP=20 options that cannot be processed before. FE performs any IP=20 fragmentation if necessary, determines link layer address = (e.g., by=20 ARP), and encapsulates the IP datagram (or each of the = fragments=20 thereof) in an appropriate link layer frame and queues it for = output=20 on the interface selected.=20 =20 Other options mentioned in RFC1812 for IP forwarding may also = be=20 implemented at FEs, for example, packet filtering.=20 =20 FEs typically forward packets destined locally to CEs. FEs may = also=20 forward exceptional packets (packets that FEs don't know how to = handle) to CEs. CEs are required to handle packets forwarded by = FEs=20 for whatever different reasons. It might be necessary for = ForCES to=20 attach some meta-data with the packets to indicate the reasons = of=20 forwarding from FEs to CEs. Upon receiving packets with = meta-data=20 from FEs, CEs can decide to either process the packets = themselves,=20 or pass the packets to the upper layer protocols including = routing=20 and management protocols. If CEs are to process the packets by=20 themselves, CEs may choose to discard the packets, or modify = and re- send the packets. CEs may also originate new packets and = deliver=20 them to FEs for further forwarding. =20 =20 Any state change during router operation must also be handled=20 correctly according to RFC1812. For example, when an FE ceases=20 forwarding, the entire NE may continue forwarding packets, but = it=20 needs to stop advertising routes that are affected by the = failed FE. =20 =20 5.5.Transport Layer =20 =20 Transport layer is typically implemented at CEs to support = higher=20 layer application protocols like routing protocols. In = practice,=20 this means that most CEs implement both the Transmission = Control=20 Protocol (TCP) and the User Datagram Protocol (UDP).=20 =20 Both CEs and FEs need to implement ForCES protocol. If some = layer-4=20 transport is used to support ForCES, then both CEs and FEs need = to=20 implement the L4 transport and ForCES protocols. It is possible = that=20 all FEs inside an NE implements only one such protocol entity. = =20 5.6. Application Layer -- Routing Protocols=20 =20 Both interior routing protocols and exterior routing protocols = are=20 implemented on CEs. The routing packets originated by CEs are=20 forwarded to FEs for delivery. The results of such protocols = (like=20 forwarding table update) are communicated to FEs via ForCES.=20 =20 For performance or scalability reasons, portions of the control = plane functions that need faster response may be moved from the = CEs=20 and off-loaded onto the FEs. For example in OSPF, the Hello = protocol=20 packets are generated and processed periodically. When done at = CEs,=20 =20 Yang, et. al. Expires April 2003 [Page 21] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 the inbound Hello packets have to traverse from the external=20 interfaces at the FEs to the CEs via the internal CE-FE = channel.=20 Similarly, the outbound Hello packets have to go from the CEs = to the=20 FEs and to the external interfaces. Frequent Hello updates = place=20 heavy processing overhead on the CEs and can overwhelm the = CE-FE=20 channel as well. Since typically there are far more FEs than = CEs in=20 a router, when off-loaded, the Hello packets are processed in a = much=20 more distributed and scalable fashion. =20 =20 Once off-loaded onto the forwarding plane, such CE functions, = even=20 though logically belong to the control plane, now become part = of the=20 FE logical functions. Just like the other logical functions=20 performed by FEs (e.g., IPv4 forwarding, classification, etc.), = such=20 off-loaded functions must be expressed as part of the FE model = so=20 that the CEs can take advantage of these off-loaded functions = when=20 present on the FEs.=20 =20 5.7. Application Layer -- Network Management Protocol=20 =20 RFC1812 also dictates "Routers MUST be manageable by SNMP." = (see=20 [RFC1157] Section 8) In general, for post-association phase, = most=20 external management tasks (including SNMP) should be done = through=20 interaction with the CE in order to support the appearance of a = single functional device. Therefore, it is recommended that = SNMP=20 management agent be implemented by CEs and the SNMP messages=20 received by FEs be redirected to their CEs. AgentX framework = defined=20 in RFC2741 may be applied here such that CEs act in the role of = master agent to process SNMP protocol messages while FEs act in = the=20 role of subagent to provide access to the MIB objects residing = on=20 FEs. AgentX protocol messages between the master agent (CE) and = the=20 subagent (FE) are encapsulated and transported via ForCES, just = like=20 data packets from any other application layer protocols. =20 =20 6. Summary=20 =20 This document defines an architectural framework for ForCES. It = identifies the relevant components for a ForCES network = element,=20 including (one or more) FEs, (one or more) CEs, one optional FE = manager, and one optional CE manager. It also identifies the=20 interaction among these components and discusses all the major=20 reference points. It is important to point out that, among all = the=20 reference points, only the interface between CEs and FEs is = within=20 the scope of ForCES. ForCES alone may not be enough to support = all=20 desirablet NE configurations. However, we believe that ForCES = is the=20 most important element in realizing physical separation and=20 interoperability of CEs and FEs, and hence the first interface = that=20 ought to be standardized. Simple and useful configurations can = still=20 be implemented with only CE-FE interface being standardized, = e.g.,=20 single CE with full-meshed FEs and static configuration without = the=20 need for CE/FE managers.=20 =20 7. Security Considerations=20 =20 =20 Yang, et. al. Expires April 2003 [Page 22] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 The security necessary across each reference point except Fp is = discussed throughout the document. In general, the physical=20 separation of two entities usually requires much stricter = security=20 measurement in place. For example, we pointed out in Section = 5.1=20 that authentication becomes necessary between CE manager and FE = manager, between CE and CE manager, between FE and FE manager = in=20 some configuration. The physical separation of CE and FE also=20 imposes serious security requirement for ForCES protocol. The=20 security requirements for reference point Fp (i.e., ForCES = protocol)=20 are discussed in detail in [FORCES-REQ] Section 8.=20 =20 8. Intellectual Property Right=20 =20 The authors are not aware of any intellectual property right = issues=20 pertaining to this document.=20 =20 9. References=20 =20 9.1. Normative References=20 =20 [RFC1157] J. Case, et. al., "A Simple Network Management = Protocol=20 (SNMP)", RFC1157, May 1990.=20 =20 [RFC1812] F. Baker, "Requirements for IP Version 4 Routers",=20 RFC1812, June 1995.=20 =20 [RFC2741] M. Daniele, et. al., "Agent Extensibility (AgentX)=20 Protocol Version 1", RFC2741, Jan 2000.=20 =20 [RFC2914] S. Floyd, "Congestion Control Principles", RFC2914,=20 September 2000.=20 =20 [RFC3084] K. Chan, et. al., "COPS Usage for Policy Provisioning = (COPS-PR)", RFC3084, March 2001.=20 =20 9.2. Informative References=20 =20 [FORCES-REQ] T. Anderson, et. al., "Requirements for Separation = of=20 IP Control and Forwarding", work in progress, July 2002, = .=20 =20 [FORCES-APP] A. Crouch, et. al., "ForCES Applicability = Statement",=20 work in progress, June 2002, .=20 =20 10. Acknowledgments=20 =20 Joel M. Halpern gave us many insightful comments and = suggestions and=20 pointed out several major issues. Many of our colleagues and = people=20 in the ForCES mailing list also provided valuable feedback.=20 =20 11. Authors' Addresses=20 =20 Yang, et. al. Expires April 2003 [Page 23] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 =20 Lily L. Yang=20 Intel Corp., MS JF3-206,=20 2111 NE 25th Avenue=20 Hillsboro, OR 97124 USA=20 Phone: +1 503 264 8813=20 Email: lily.l.yang@intel.com=20 =20 Ram Dantu=20 University of Texas Dallas=20 2601 North Flyod Road=20 Richardson Texas 75082=20 Phone: +1 972 883 4653=20 Email: ram.dantu@utdallas.edu=20 =20 Todd A. Anderson=20 Intel Corp.=20 2111 NE 25th Avenue=20 Hillsboro, OR 97124 USA=20 Phone: +1 503 712 1760=20 Email: todd.a.anderson@intel.com=20 =20 =20 =20 Yang, et. al. Expires April 2003 [Page 24] = =0C ------_=_NextPart_000_01C27188.309B72E0-- From owner-forces@PEACH.EASE.LSOFT.COM Sat Oct 12 03:30:15 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA22146 for ; Sat, 12 Oct 2002 03:30:15 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <10.00769854@cherry.ease.lsoft.com>; Sat, 12 Oct 2002 3:32:21 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 28891518 for FORCES@PEACH.EASE.LSOFT.COM; Sat, 12 Oct 2002 03:32:16 -0400 Received: from 194.196.100.235 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Sat, 12 Oct 2002 03:32:16 -0400 Received: from d12relay02.de.ibm.com (d12relay02.de.ibm.com [9.165.215.23]) by d12lmsgate-2.de.ibm.com (8.12.3/8.12.3) with ESMTP id g9C7WFSZ028844 for ; Sat, 12 Oct 2002 09:32:15 +0200 Received: from etzel.zurich.ibm.com (etzel.zurich.ibm.com [9.4.64.140]) by d12relay02.de.ibm.com (8.12.3/NCO/VER6.4) with SMTP id g9C7WE1F087774 for ; Sat, 12 Oct 2002 09:32:14 +0200 Received: from [9.164.17.5] by etzel.zurich.ibm.com (AIX 4.3/UCB 5.64/4.03) id AA29738 from ; Sat, 12 Oct 2002 09:32:12 +0200 X-Mailer: Mozilla 4.76 [en] (Windows NT 5.0; U) X-Accept-Language: en Mime-Version: 1.0 References: Content-Type: text/plain; charset=iso-8859-1 X-MIME-Autoconverted: from 8bit to quoted-printable by d12lmsgate-2.de.ibm.com id g9C7WFSZ028844 Approved-By: Robert Haas Message-ID: <3DA7CFFB.4AFD6DAB@zurich.ibm.com> Date: Sat, 12 Oct 2002 09:32:11 +0200 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: Robert Haas Subject: Re: new rev of framework draft, please comment To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by ietf.org id DAA22146 Lily, I see you added some text regarding "Off-loaded functions" into section 5.6. "Application Layer -- Routing Protocols". This restricts the purpose of off-loaded functions to routing, i.e. OSPF Hello processing. In fact, as suggested in my previous note, parts of ICMP or TCP processing could as well be off-loaded into the FE. Could you please update the draft to make sure that no such restriction is placed on the type of off-loaded functions ? Thanks, -Robert "Yang, Lily L" wrote: > Hi, all > > After hearing back from the list, I've incorporated most of the comments > into this new version. Please take a look and provide feedback. I will wait > a bit before submitting this version. > > Summary of changes since ver02: > 1) Per suggestion from Jamal, the following is added at the end of Section 3 > (Introduction): > "Whenever necessary, this document uses many examples to illustrate > the issues and/or possible solutions in ForCES. These examples are intended > to be just examples, and should not be taken as the only or definite ways of > doing certain things. It is expected that separate document will be produced > by the ForCES working group to specify the ForCES protocol(s)." > > 2) Per suggestion from Jeelani and others about the "off-loaded functions", > the following is added at the end of Section 3 (Introduction): > "For performance or scalability reasons, portions of the control > plane functions that need faster response may be moved from the CEs and > off-loaded onto the FEs. For example in OSPF, the Hello protocol packets are > generated and processed periodically. When done at CEs, the inbound Hello > packets have to traverse from the external interfaces at the FEs to the CEs > via the internal CE-FE channel. Similarly, the outbound Hello packets have > to go from the CEs to the FEs and to the external interfaces. Frequent Hello > updates place heavy processing overhead on the CEs and can overwhelm the > CE-FE channel as well. Since typically there are far more FEs than CEs in a > router, when off-loaded, the Hello packets are processed in a much more > distributed and scalable fashion. > > Once off-loaded onto the forwarding plane, such CE functions, even > though logically belong to the control plane, now become part of the FE > logical functions. Just like the other logical functions performed by FEs > (e.g., IPv4 forwarding, classification, etc.), such off-loaded functions > must be expressed as part of the FE model so that the CEs can take advantage > of these off-loaded functions when present on the FEs." > > 3) Per suggestion from Ram Gopal, the explicit security exchange is deleted > from the message exchange example (Fig 9) in Section 4.2.2. > > 4) More editorial changes here and there. All minor. > > Thanks, > Lily > > <> > > ------------------------------------------------------------------------ > Name: draft-ietf-forces-framework-03.txt > draft-ietf-forces-framework-03.txt Type: Plain Text (text/plain) > Encoding: quoted-printable -- Robert Haas IBM Zurich Research Laboratory Säumerstrasse 4 CH-8803 Rüschlikon/Switzerland phone +41-1-724-8698 fax +41-1-724-8578 http://www.zurich.ibm.com/~rha From owner-forces@PEACH.EASE.LSOFT.COM Sun Oct 13 09:28:26 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA19395 for ; Sun, 13 Oct 2002 09:28:25 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <9.0076C871@cherry.ease.lsoft.com>; 13 Oct 2002 9:30:33 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 28950695 for FORCES@PEACH.EASE.LSOFT.COM; Sun, 13 Oct 2002 09:30:14 -0400 Received: from 208.2.156.2 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Sun, 13 Oct 2002 09:30:14 -0400 Received: from localhost (jamalhadi@localhost) by znyx.com (8.9.3/8.9.3) with ESMTP id GAA24384 for ; Sun, 13 Oct 2002 06:36:17 -0700 X-Sender: jamalhadi@electra.znyx.com MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Approved-By: Jamal Hadi Salim Message-ID: Date: Sun, 13 Oct 2002 06:36:17 -0700 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: Jamal Hadi Salim Subject: Re: architecture draft comments To: FORCES@PEACH.EASE.LSOFT.COM In-Reply-To: Precedence: list On Fri, 11 Oct 2002, Yang, Lily L wrote: > Jamal -- > Thanks for the comments! Itemized responses or further questions below ... > > > -paragraph under fig. 5 describes an example of how CE1 > > remains in sync with > > CE2 via a CE-CE protocol. I dont see it happening this way > > Could you elaborate on how you think it works? > Here's the wording i was refering to: "... FE1 and FE2 can be configured to accept commands from both the primary CE (CE1) and the backup CE (CE2). At the beginning, CE1 issues commands to FEs while CE2 silently remains in sync with CE1 via CE to CE protocol over Fr reference point. Before CE2 starts issuing commands to the FEs, it might need to recheck the FEs' state and instruct FEs whether or not it is ok to preserve their current state." 1st sentence uses the word "configured" to say somehow both CEs will send commands to the FEs. It doesnt say what configured means; therefore it doesnt describe the how - i am happy with this. The second and third sentence go into unnecessary details of the "how". How i think it should work is unimportant really since this is beyond forces scope; as far as i am concerned it could be proprietary. The danger is in making "suggestions" in a blackbox document. Since both yourself and Alex asked this same question i am going to assume you have come to the conclusion the way it is worded right now is "how it should work" ;-) Heres one of ways it could work as well: Assume communication between CEs and FEs is visible to both CEs. Maybe a mcast channel. They can both see the transactions/results, although only one is active. In this case synchronization is continual as opposed using a "CE to CE protocol over Fr reference point". I could describe two other ways (and i am sure there are 101 ways) to do it, but thats beside the point. I think you have enough material on the Fr point of reference that you dont need those two extra sentences above; so please consider deleting them and replacing with: "Upon detection of CE1 failure, perhaps across the Fr reference point, CE2 is configured to take over activities of CE1. This is beyond the scope of Forces and is not discussed further". cheers, jamal From owner-forces@PEACH.EASE.LSOFT.COM Tue Oct 15 23:09:57 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA22326 for ; Tue, 15 Oct 2002 23:09:56 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <1.0077331B@cherry.ease.lsoft.com>; Tue, 15 Oct 2002 23:12:07 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 29181086 for FORCES@PEACH.EASE.LSOFT.COM; Tue, 15 Oct 2002 23:12:07 -0400 Received: from 192.55.52.25 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Tue, 15 Oct 2002 23:12:06 -0400 Received: from talaria.fm.intel.com (talaria.fm.intel.com [10.1.192.39]) by caduceus.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.51 2002/09/23 20:43:23 dmccart Exp $) with ESMTP id g9G38Fg20632 for ; Wed, 16 Oct 2002 03:08:15 GMT Received: from fmsmsxvs042.fm.intel.com (fmsmsxvs042.fm.intel.com [132.233.42.128]) by talaria.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.26 2002/10/10 20:36:29 dmccart Exp $) with SMTP id g9G3F1Z10139 for ; Wed, 16 Oct 2002 03:15:01 GMT Received: from fmsmsx28.fm.intel.com ([132.233.42.28]) by fmsmsxvs042.fm.intel.com (NAVGW 2.5.2.11) with SMTP id M2002101520135200153 for ; Tue, 15 Oct 2002 20:13:52 -0700 Received: by fmsmsx28.fm.intel.com with Internet Mail Service (5.5.2653.19) id <403P93FS>; Tue, 15 Oct 2002 20:12:05 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Approved-By: "Yang, Lily L" Message-ID: Date: Tue, 15 Oct 2002 20:12:05 -0700 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: "Yang, Lily L" Subject: Re: architecture draft comments To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list Jamal -- Thank you for your detailed explanation. Now I understand your point. I will incorporate that into the draft. Lily -----Original Message----- From: Jamal Hadi Salim [mailto:hadi@znyx.com] Sent: Sunday, October 13, 2002 6:36 AM To: FORCES@PEACH.EASE.LSOFT.COM Subject: Re: architecture draft comments On Fri, 11 Oct 2002, Yang, Lily L wrote: > Jamal -- > Thanks for the comments! Itemized responses or further questions below ... > > > -paragraph under fig. 5 describes an example of how CE1 > > remains in sync with > > CE2 via a CE-CE protocol. I dont see it happening this way > > Could you elaborate on how you think it works? > Here's the wording i was refering to: "... FE1 and FE2 can be configured to accept commands from both the primary CE (CE1) and the backup CE (CE2). At the beginning, CE1 issues commands to FEs while CE2 silently remains in sync with CE1 via CE to CE protocol over Fr reference point. Before CE2 starts issuing commands to the FEs, it might need to recheck the FEs' state and instruct FEs whether or not it is ok to preserve their current state." 1st sentence uses the word "configured" to say somehow both CEs will send commands to the FEs. It doesnt say what configured means; therefore it doesnt describe the how - i am happy with this. The second and third sentence go into unnecessary details of the "how". How i think it should work is unimportant really since this is beyond forces scope; as far as i am concerned it could be proprietary. The danger is in making "suggestions" in a blackbox document. Since both yourself and Alex asked this same question i am going to assume you have come to the conclusion the way it is worded right now is "how it should work" ;-) Heres one of ways it could work as well: Assume communication between CEs and FEs is visible to both CEs. Maybe a mcast channel. They can both see the transactions/results, although only one is active. In this case synchronization is continual as opposed using a "CE to CE protocol over Fr reference point". I could describe two other ways (and i am sure there are 101 ways) to do it, but thats beside the point. I think you have enough material on the Fr point of reference that you dont need those two extra sentences above; so please consider deleting them and replacing with: "Upon detection of CE1 failure, perhaps across the Fr reference point, CE2 is configured to take over activities of CE1. This is beyond the scope of Forces and is not discussed further". cheers, jamal From owner-forces@PEACH.EASE.LSOFT.COM Tue Oct 15 23:18:59 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA22447 for ; Tue, 15 Oct 2002 23:18:57 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <18.007731FA@cherry.ease.lsoft.com>; Tue, 15 Oct 2002 23:21:07 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 29181324 for FORCES@PEACH.EASE.LSOFT.COM; Tue, 15 Oct 2002 23:21:06 -0400 Received: from 192.55.52.25 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Tue, 15 Oct 2002 23:21:06 -0400 Received: from petasus.fm.intel.com (petasus.fm.intel.com [10.1.192.37]) by caduceus.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.51 2002/09/23 20:43:23 dmccart Exp $) with ESMTP id g9G3HEW24375 for ; Wed, 16 Oct 2002 03:17:14 GMT Received: from fmsmsxvs041.fm.intel.com (fmsmsxvs041.fm.intel.com [132.233.42.126]) by petasus.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.26 2002/10/10 20:36:29 dmccart Exp $) with SMTP id g9G3HKb17711 for ; Wed, 16 Oct 2002 03:17:20 GMT Received: from FMSMSX017.fm.intel.com ([132.233.42.196]) by fmsmsxvs041.fm.intel.com (NAVGW 2.5.2.11) with SMTP id M2002101520194805367 for ; Tue, 15 Oct 2002 20:19:48 -0700 Received: by fmsmsx017.fm.intel.com with Internet Mail Service (5.5.2653.19) id ; Tue, 15 Oct 2002 20:21:05 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" X-MIME-Autoconverted: from 8bit to quoted-printable by caduceus.fm.intel.com id g9G3HEW24375 Approved-By: "Yang, Lily L" Message-ID: Date: Tue, 15 Oct 2002 20:20:41 -0700 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: "Yang, Lily L" Subject: Re: new rev of framework draft, please comment To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by ietf.org id XAA22447 Robert -- I didn't mean to restrict the off-loaded functions to routing. It is just that this particular paragraph is under section 5.6."Application Layer -- Routing Protocols" and so it is only concerned with the routing protocols in particular. Since this comes up as a general issue of deciding which functions go to CE and which go to FE, I might need to state something more general up front. The architecture should not in any way restrict what can or should be off-loaded in the future as long as mechanism (=FE model) exists to communicate that to ensure interoperability. I will fix the draft along that line. Does it make sense? thanks, Lily -----Original Message----- From: Robert Haas [mailto:rha@zurich.ibm.com] Sent: Saturday, October 12, 2002 12:32 AM To: FORCES@PEACH.EASE.LSOFT.COM Subject: Re: new rev of framework draft, please comment Lily, I see you added some text regarding "Off-loaded functions" into section 5.6. "Application Layer -- Routing Protocols". This restricts the purpose of off-loaded functions to routing, i.e. OSPF Hello processing. In fact, as suggested in my previous note, parts of ICMP or TCP processing could as well be off-loaded into the FE. Could you please update the draft to make sure that no such restriction is placed on the type of off-loaded functions ? Thanks, -Robert "Yang, Lily L" wrote: > Hi, all > > After hearing back from the list, I've incorporated most of the comments > into this new version. Please take a look and provide feedback. I will wait > a bit before submitting this version. > > Summary of changes since ver02: > 1) Per suggestion from Jamal, the following is added at the end of Section 3 > (Introduction): > "Whenever necessary, this document uses many examples to illustrate > the issues and/or possible solutions in ForCES. These examples are intended > to be just examples, and should not be taken as the only or definite ways of > doing certain things. It is expected that separate document will be produced > by the ForCES working group to specify the ForCES protocol(s)." > > 2) Per suggestion from Jeelani and others about the "off-loaded functions", > the following is added at the end of Section 3 (Introduction): > "For performance or scalability reasons, portions of the control > plane functions that need faster response may be moved from the CEs and > off-loaded onto the FEs. For example in OSPF, the Hello protocol packets are > generated and processed periodically. When done at CEs, the inbound Hello > packets have to traverse from the external interfaces at the FEs to the CEs > via the internal CE-FE channel. Similarly, the outbound Hello packets have > to go from the CEs to the FEs and to the external interfaces. Frequent Hello > updates place heavy processing overhead on the CEs and can overwhelm the > CE-FE channel as well. Since typically there are far more FEs than CEs in a > router, when off-loaded, the Hello packets are processed in a much more > distributed and scalable fashion. > > Once off-loaded onto the forwarding plane, such CE functions, even > though logically belong to the control plane, now become part of the FE > logical functions. Just like the other logical functions performed by FEs > (e.g., IPv4 forwarding, classification, etc.), such off-loaded functions > must be expressed as part of the FE model so that the CEs can take advantage > of these off-loaded functions when present on the FEs." > > 3) Per suggestion from Ram Gopal, the explicit security exchange is deleted > from the message exchange example (Fig 9) in Section 4.2.2. > > 4) More editorial changes here and there. All minor. > > Thanks, > Lily > > <> > > ------------------------------------------------------------------------ > Name: draft-ietf-forces-framework-03.txt > draft-ietf-forces-framework-03.txt Type: Plain Text (text/plain) > Encoding: quoted-printable -- Robert Haas IBM Zurich Research Laboratory Säumerstrasse 4 CH-8803 Rüschlikon/Switzerland phone +41-1-724-8698 fax +41-1-724-8578 http://www.zurich.ibm.com/~rha From owner-forces@PEACH.EASE.LSOFT.COM Wed Oct 16 03:57:41 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA21499 for ; Wed, 16 Oct 2002 03:57:40 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <23.00773C45@cherry.ease.lsoft.com>; Wed, 16 Oct 2002 3:59:51 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 29200170 for FORCES@PEACH.EASE.LSOFT.COM; Wed, 16 Oct 2002 03:59:29 -0400 Received: from 194.196.100.238 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Wed, 16 Oct 2002 03:59:29 -0400 Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22]) by d12lmsgate-5.de.ibm.com (8.12.3/8.12.3) with ESMTP id g9G7xSjk005672 for ; Wed, 16 Oct 2002 09:59:28 +0200 Received: from etzel.zurich.ibm.com (etzel.zurich.ibm.com [9.4.64.140]) by d12relay01.de.ibm.com (8.12.3/NCO/VER6.4) with SMTP id g9G7xRJ4050520 for ; Wed, 16 Oct 2002 09:59:27 +0200 Received: from dhcp70-86.zurich.ibm.com by etzel.zurich.ibm.com (AIX 4.3/UCB 5.64/4.03) id AA72556 from ; Wed, 16 Oct 2002 09:59:27 +0200 X-Mailer: Mozilla 4.76 [en] (Windows NT 5.0; U) X-Accept-Language: en Mime-Version: 1.0 References: Content-Type: text/plain; charset=iso-8859-1 X-MIME-Autoconverted: from 8bit to quoted-printable by d12lmsgate-5.de.ibm.com id g9G7xSjk005672 Approved-By: Robert Haas Message-ID: <3DAD1C5F.ACAEDBDC@zurich.ibm.com> Date: Wed, 16 Oct 2002 09:59:27 +0200 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: Robert Haas Subject: Re: new rev of framework draft, please comment To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by ietf.org id DAA21499 Lily, Yes, it makes perfect sense to have a more general statement upfront (under "5. Applicability to RFC1812" ), and possibly give examples of such off-loading in the following subsections. Thanks, -Robert "Yang, Lily L" wrote: > Robert -- > I didn't mean to restrict the off-loaded functions to routing. It is just > that this particular paragraph is under section 5.6."Application Layer -- > Routing Protocols" and so it is only concerned with the routing protocols in > particular. Since this comes up as a general issue of deciding which > functions go to CE and which go to FE, I might need to state something more > general up front. The architecture should not in any way restrict what can > or should be off-loaded in the future as long as mechanism (=FE model) > exists to communicate that to ensure interoperability. I will fix the draft > along that line. Does it make sense? > > thanks, > Lily > > -----Original Message----- > From: Robert Haas [mailto:rha@zurich.ibm.com] > Sent: Saturday, October 12, 2002 12:32 AM > To: FORCES@PEACH.EASE.LSOFT.COM > Subject: Re: new rev of framework draft, please comment > > Lily, > I see you added some text regarding "Off-loaded functions" into section 5.6. > "Application Layer -- Routing Protocols". This restricts the purpose of > off-loaded > functions to routing, i.e. OSPF Hello processing. In fact, as suggested in > my > previous note, parts of ICMP or TCP processing could as well be off-loaded > into the > FE. Could you please update the draft to make sure that no such restriction > is placed > on the type of off-loaded functions ? > > Thanks, > -Robert > > "Yang, Lily L" wrote: > > > Hi, all > > > > After hearing back from the list, I've incorporated most of the comments > > into this new version. Please take a look and provide feedback. I will > wait > > a bit before submitting this version. > > > > Summary of changes since ver02: > > 1) Per suggestion from Jamal, the following is added at the end of Section > 3 > > (Introduction): > > "Whenever necessary, this document uses many examples to > illustrate > > the issues and/or possible solutions in ForCES. These examples are > intended > > to be just examples, and should not be taken as the only or definite ways > of > > doing certain things. It is expected that separate document will be > produced > > by the ForCES working group to specify the ForCES protocol(s)." > > > > 2) Per suggestion from Jeelani and others about the "off-loaded > functions", > > the following is added at the end of Section 3 (Introduction): > > "For performance or scalability reasons, portions of the control > > plane functions that need faster response may be moved from the CEs and > > off-loaded onto the FEs. For example in OSPF, the Hello protocol packets > are > > generated and processed periodically. When done at CEs, the inbound Hello > > packets have to traverse from the external interfaces at the FEs to the > CEs > > via the internal CE-FE channel. Similarly, the outbound Hello packets have > > to go from the CEs to the FEs and to the external interfaces. Frequent > Hello > > updates place heavy processing overhead on the CEs and can overwhelm the > > CE-FE channel as well. Since typically there are far more FEs than CEs in > a > > router, when off-loaded, the Hello packets are processed in a much more > > distributed and scalable fashion. > > > > Once off-loaded onto the forwarding plane, such CE functions, even > > though logically belong to the control plane, now become part of the FE > > logical functions. Just like the other logical functions performed by FEs > > (e.g., IPv4 forwarding, classification, etc.), such off-loaded functions > > must be expressed as part of the FE model so that the CEs can take > advantage > > of these off-loaded functions when present on the FEs." > > > > 3) Per suggestion from Ram Gopal, the explicit security exchange is > deleted > > from the message exchange example (Fig 9) in Section 4.2.2. > > > > 4) More editorial changes here and there. All minor. > > > > Thanks, > > Lily > > > > <> > > > > ------------------------------------------------------------------------ > > Name: > draft-ietf-forces-framework-03.txt > > draft-ietf-forces-framework-03.txt Type: Plain Text (text/plain) > > Encoding: quoted-printable > > -- > Robert Haas > IBM Zurich Research Laboratory > Säumerstrasse 4 > CH-8803 Rüschlikon/Switzerland > phone +41-1-724-8698 fax +41-1-724-8578 http://www.zurich.ibm.com/~rha -- Robert Haas IBM Zurich Research Laboratory Säumerstrasse 4 CH-8803 Rüschlikon/Switzerland phone +41-1-724-8698 fax +41-1-724-8578 http://www.zurich.ibm.com/~rha From owner-forces@PEACH.EASE.LSOFT.COM Fri Oct 18 16:12:47 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA08640 for ; Fri, 18 Oct 2002 16:12:37 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <7.0077F67A@cherry.ease.lsoft.com>; Fri, 18 Oct 2002 16:14:49 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 29493957 for FORCES@PEACH.EASE.LSOFT.COM; Fri, 18 Oct 2002 16:14:22 -0400 Received: from 192.55.52.18 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Fri, 18 Oct 2002 16:14:20 -0400 Received: from talaria.fm.intel.com (talaria.fm.intel.com [10.1.192.39]) by hermes.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.51 2002/09/23 20:43:23 dmccart Exp $) with ESMTP id g9IKCsD07866 for ; Fri, 18 Oct 2002 20:12:54 GMT Received: from fmsmsx29.FM.INTEL.COM (fmsmsx29.fm.intel.com [132.233.42.29]) by talaria.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.27 2002/10/16 23:46:59 dmccart Exp $) with ESMTP id g9IKHB020151 for ; Fri, 18 Oct 2002 20:17:11 GMT Received: by fmsmsx29.fm.intel.com with Internet Mail Service (5.5.2653.19) id ; Fri, 18 Oct 2002 13:13:12 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/mixed; boundary="----_=_NextPart_000_01C276E2.C76C86A0" Approved-By: "Yang, Lily L" Message-ID: Date: Fri, 18 Oct 2002 13:13:10 -0700 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: "Yang, Lily L" Subject: newer framework-03 draft To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_000_01C276E2.C76C86A0 Content-Type: text/plain; charset="iso-8859-1" Hi, all Here is another rev of the framework draft, incorporating comments from Jamal and Robert. Change summary: 1) Per comments from Jamal, in Section 3.2, regarding redundancy support in Fig 5., the paragraph is revised as: For example, in Figure 5, FE1 and FE2 can be configured to accept commands from both the primary CE (CE1) and the backup CE (CE2). Upon detection of CE1 failure, perhaps across the Fr reference point, CE2 is configured to take over activities of CE1. This is beyond the scope of ForCES and is not discussed further. 2) Per comments from Robert, in Section 5 "Applicability to RFC1812", a new paragraph is added to address the general issue of off-loaded functions. In general, the forwarding plane carries out the bulk of the per-packet processing that is required at line speed, while the control plane carries most of the computationally complex operations that are typical of the control and signaling protocols. However, it is not practical to draw a rigid line to divide the processing into CEs and FEs cleanly. Nor should the ForCES architecture limits the innovative approaches in control and forwarding plane separation. As more and more processing power is available in the FEs, some of the control functions that traditionally are performed by CEs may now be moved to FEs for better performance and scalability. Such offloaded functions may include part of ICMP or TCP processing, or part of routing protocols. Once off-loaded onto the forwarding plane, such CE functions, even though logically belong to the control plane, now become part of the FE logical functions. Just like the other logical functions performed by FEs, such off-loaded functions must be expressed as part of the FE model so that the CEs can decide how to best take advantage of these off-loaded functions when present on the FEs. Then in sub-section 5.6 "Application Protocols -- Routing Protocols", a more specific paragraph is devoted to off-loaded functions in routing, like OSPF Hello example. For performance or scalability reasons, portions of the control plane functions that need faster response may be moved from the CEs and off-loaded onto the FEs. For example in OSPF, the Hello protocol packets are generated and processed periodically. When done at CEs, the inbound Hello packets have to traverse from the external interfaces at the FEs to the CEs via the internal CE-FE channel. Similarly, the outbound Hello packets have to go from the CEs to the FEs and to the external interfaces. Frequent Hello updates place heavy processing overhead on the CEs and can overwhelm the CE-FE channel as well. Since typically there are far more FEs than CEs in a router, when off-loaded, the Hello packets are processed in a much more distributed and scalable fashion. Interoperability is ensured by expressing such off-loaded functions in the FE model. Please check out the draft and provide comments. We would like to submit the draft officially sometime next week. Thanks, Lily <> <> ------_=_NextPart_000_01C276E2.C76C86A0 Content-Type: text/plain; name="draft-ietf-forces-framework-03.txt" Content-Disposition: attachment; filename="draft-ietf-forces-framework-03.txt" Content-Transfer-Encoding: quoted-printable Internet Draft L. Yang=20 Expiration: April 2003 Intel Corp.=20 File: draft-ietf-forces-framework-03.txt R. Dantu=20 Working Group: ForCES Univ. of Texas = Dallas=20 T. Anderson=20 Intel Corp. =20 Oct 2002 = =20 =20 =20 ForCES Architectural Framework=20 =20 =20 =20 draft-ietf-forces-framework-03.txt=20 =20 =20 =20 =20 Status of this Memo=20 =20 This document is an Internet-Draft and is in full conformance = with=20 all provisions of Section 10 of RFC2026. Internet-Drafts are=20 working documents of the Internet Engineering Task Force = (IETF), its=20 areas, and its working groups. Note that other groups may also = distribute working documents as Internet-Drafts.=20 =20 Internet-Drafts are draft documents valid for a maximum of six=20 months and may be updated, replaced, or obsoleted by other = documents=20 at any time. It is inappropriate to use Internet-Drafts as=20 reference material or to cite them other than as ``work in=20 progress.''=20 =20 The list of current Internet-Drafts can be accessed at=20 http://www.ietf.org/ietf/1id-abstracts.txt.=20 =20 The list of Internet-Draft Shadow Directories can be accessed = at =20 http://www.ietf.org/shadow.html.=20 =20 Abstract=20 =20 This document defines the architectural framework for the = ForCES=20 (Forwarding and Control Element Separation) network elements = (NE),=20 and identifies the associated entities and the interaction = among=20 them. This framework is intended to satisfy the requirements=20 specified in the ForCES requirements draft [FORCES-REQ].=20 =20 Table of Contents=20 =20 1. = Definitions.....................................................2=20 2. Introduction to Forwarding and Control Element Separation=20 = (ForCES)...........................................................4=20 3. = Architecture....................................................7=20 3.1. Control Elements and Fr Reference = Point....................8=20 =20 =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 3.2. Forwarding Elements and Fi reference = point.................9=20 3.3. CE = Managers...............................................11=20 3.4. FE = Managers...............................................11=20 4. Operational = Phases.............................................12=20 4.1. Pre-association = Phase.....................................12=20 4.1.1. Fl Reference = Point...................................12=20 4.1.2. Ff Reference = Point...................................13=20 4.1.3. Fc Reference = Point...................................13=20 4.2. Post-association Phase and Fp reference = point.............14=20 4.2.1. Proximity and Interconnect between CEs and = FEs.......14=20 4.2.2. Association = Establishment............................14=20 4.2.3. Steady-state = Communication...........................15=20 4.2.4. Data Packets across Fp reference = point...............16=20 4.2.5. Proxy = FE.............................................17=20 4.3. Association = Re-establishment..............................17=20 5. Applicability to = RFC1812.......................................18=20 5.1. General Router = Requirements...............................19=20 5.2. Link = Layer................................................20=20 5.3. Internet Layer = Protocols..................................20=20 5.4. Internet Layer = Forwarding.................................21=20 5.5. Transport = Layer...........................................21=20 5.6. Application Layer -- Routing = Protocols....................22=20 5.7. Application Layer -- Network Management = Protocol..........22=20 6. = Summary........................................................22=20 7. Security = Considerations........................................23=20 8. Intellectual Property = Right....................................23=20 9. = References.....................................................23=20 9.1. Normative = References......................................23=20 9.2. Informative = References....................................23=20 10. = Acknowledgments...............................................24=20 11. Authors' = Addresses............................................24=20 =20 Conventions used in this document =20 =20 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL = NOT", =20 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" = in=20 this document are to be interpreted as described in [RFC-2119]. = =20 1. Definitions=20 =20 A set of terminology associated with the ForCES requirements is = defined in [FORCES-REQ] and we only include the ones that are = most=20 relevant to this document here. =20 =20 Forwarding Element (FE) - A logical entity that implements the=20 ForCES protocol. FEs use the underlying hardware to provide = per- packet processing and handling as directed by a CE via the = ForCES=20 protocol. =20 =20 Control Element (CE) - A logical entity that implements the = ForCES=20 protocol and uses it to instruct one or more FEs how to process = packets. CEs handle functionality such as the execution of = control =20 and signaling protocols. =20 =20 Yang, et. al. Expires April 2003 [Page 2] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 =20 ForCES Network Element (NE) - An entity composed of one or more = CEs=20 and one or more FEs. To entities outside a NE, the NE = represents a=20 single point of management. Similarly, a NE usually hides its=20 internal organization from external entities. =20 =20 Pre-association Phase - The period of time during which a FE = Manager=20 (see below) and a CE Manager (see below) are determining which = FE=20 and CE should be part of the same network element.=20 =20 Post-association Phase - The period of time during which a FE = does=20 know which CE is to control it and vice versa, including the = time=20 during which the CE and FE are establishing communication with = one=20 another. =20 =20 ForCES Protocol - While there may be multiple protocols used = within=20 the overall ForCES architecture, the term "ForCES protocol" = refers=20 only to the ForCES post-association phase protocol (see below). = =20 =20 ForCES Post-Association Phase Protocol - The protocol used for = post- association phase communication between CEs and FEs. This = protocol=20 does not apply to CE-to-CE communication, FE-to-FE = communication, or=20 to communication between FE and CE managers. The ForCES = protocol is=20 a master-slave protocol in which FEs are slaves and CEs are = masters. =20 This protocol includes both the management of the communication = channel (e.g., connection establishment, heartbeats) and the = control=20 messages themselves. This protocol could be a single protocol = or=20 could consist of multiple protocols working together. =20 =20 FE Manager - A logical entity that operates in the = pre-association=20 phase and is responsible for determining to which CE(s) a FE = should=20 communicate. This process is called CE discovery and may = involve=20 the FE manager learning the capabilities of available CEs. A = FE=20 manager may use anything from a static configuration to a pre- association phase protocol (see below) to determine which CE(s) = to=20 use. Being a logical entity, a FE manager might be physically=20 combined with any of the other logical entities mentioned in = this=20 section. =20 =20 CE Manager - A logical entity that operates in the = pre-association =20 phase and is responsible for determining to which FE(s) a CE = should =20 communicate. This process is called FE discovery and may = involve=20 the CE manager learning the capabilities of available FEs. A = CE=20 manager may use anything from a static configuration to a pre- association phase protocol (see below) to determine which FE to = use. =20 Being a logical entity, a CE manager might be physically = combined=20 with any of the other logical entities mentioned in this = section. =20 =20 Pre-association Phase Protocol - A protocol between FE managers = and=20 CE managers that is used to determine which CEs or FEs to use. = A=20 pre-association phase protocol may include a CE and/or FE = capability=20 discovery mechanism. Note that this capability discovery = process is=20 wholly separate from (and does not replace) that used within = the=20 =20 Yang, et. al. Expires April 2003 [Page 3] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 ForCES protocol. However, the two capability discovery = mechanisms=20 may utilize the same FE model. =20 =20 FE Model - A model that describes the logical processing = functions=20 of a FE. =20 =20 ForCES Protocol Element - A FE or CE. =20 =20 2. Introduction to Forwarding and Control Element Separation = (ForCES) =20 =20 An IP network element (NE) appears to external entities as a=20 monolithic piece of network equipment, e.g., a router, NAT,=20 firewall, or load balancer. Internally, however, an IP network=20 element (NE) (such as a router) is composed of numerous = logically=20 separated entities that cooperate to provide a given = functionality=20 (such as routing). Two types of network element components = exist:=20 control element (CE) in control plane and forwarding element = (FE) in=20 forwarding plane (or data plane). Forwarding elements = typically are=20 ASIC, network-processor, or general-purpose processor-based = devices=20 that handle data path operations for each packet. Control = elements=20 are typically based on general-purpose processors that provide=20 control functionality like routing and signaling protocols. =20 =20 ForCES aims to define a framework and associated protocol(s) to = standardize information exchange between the control and = forwarding=20 plane. Having standard mechanisms allows CEs and FEs to become=20 physically separated standard components. This physical = separation=20 accrues several benefits to the ForCES architecture. Separate=20 components would allow component vendors to specialize in one=20 component without having to become experts in all components.=20 Standard protocol also allows the CEs and FEs from different=20 component vendors to interoperate with each other and hence it=20 becomes possible for system vendors to integrate together the = CEs=20 and FEs from different component suppliers. This = interoperability=20 translates into a lot more design choices and flexibility to = the=20 system vendors. Overall, ForCES will enable rapid innovation in = both=20 the control and forwarding planes while maintaining=20 interoperability. Scalability is also easily provided by this=20 architecture in that additional forwarding or control capacity = can=20 be added to existing network elements without the need for = forklift=20 upgrades.=20 =20 One example of such physical separation is at the blade level.=20 Figure 1 shows such an example configuration of a router, with = two=20 control blades and multiple router (forwarding) blades, all=20 interconnected into a switch fabric backplane. In such chassis=20 configuration, the control blades are the CEs while the router=20 blades are FEs, and the switch fabric backplane provides the=20 physical interconnect for all the blades. Control blade A may = be the=20 primary CE while control blade B is the backup CE providing=20 redundancy. It is also possible to have a redundant switch = fabric=20 for high availability support. Routers today with this kind of=20 configuration use proprietary interface for messaging between = CEs=20 =20 Yang, et. al. Expires April 2003 [Page 4] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 and FEs. The goal of ForCES is to replace such proprietary = interface=20 with a standard protocol. With a standard protocol like ForCES=20 implemented on all blades, it becomes possible for control = blades=20 from vendor X and routing blades from vendor Y to work = seamlessly=20 together in one chassis. =20 =20 ------------------------- -------------------------=20 | Control Blade A | | Control Blade B |=20 | (CE) | | (CE) |=20 ------------------------- -------------------------=20 ^ | ^ |=20 | | | |=20 | V | V=20 ---------------------------------------------------------=20 | Switch Fabric Backplane |=20 ---------------------------------------------------------=20 ^ | ^ | ^ | = | | | | =85 | | = =20 | V | V | V = ------------ ------------ ------------ = |Router | |Router | |Router |=20 |Blade #1 | |Blade #2 | |Blade #N |=20 | (FE) | | (FE) | | (FE) |=20 ------------ ------------ ------------ = ^ | ^ | ^ | = | | | | =85 | | = =20 | V | V | V = =20 Figure 1. A router configuration example with separate = blades.=20 =20 =20 ------- -------=20 | CE1 | | CE2 |=20 ------- -------=20 ^ ^=20 | |=20 V V=20 = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Ethernet=20 ^ ^ =85 ^=20 | | |=20 V V V=20 ------- ------- --------=20 | FE#1| | FE#2| | FE#n |=20 ------- ------- --------=20 ^ | ^ | ^ | =20 | | | | | | =20 | V | V | V =20 =20 Figure 2. A router configuration example with separate = boxes.=20 =20 Another level of physical separation between the CEs and FEs = can be=20 at the box level. In such configuration, all the CEs and FEs = are=20 physically separated boxes, interconnected with some kind of = high=20 =20 Yang, et. al. Expires April 2003 [Page 5] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 speed LAN connection (like Gigabit Ethernet). These separated = CEs=20 and FEs are only one hop away from each other within a local = area=20 network. The CEs and FEs communicate to each other by running=20 ForCES, and the collection of these CEs and FEs together become = one=20 routing unit to the external world. Figure 2 shows such an = example.=20 =20 In both examples shown here, the same physical interconnect is = used=20 for both CE-to-FE and FE-to-FE communication. However, that = does not=20 have to be the case. One reason to use different interconnect = is=20 that CE-to-FE interconnect does not have to be as fast as the = FE-to- FE interconnect, so the more expensive fast connections can be = saved=20 for FE-to-FE only. The separate interconnects may also provide = reliability and redundancy benefits for the NE.=20 =20 -------------------------------------------------=20 | | | | | | |=20 |OSPF |RIP |BGP |CAC |LDP |=85 |=20 | | | | | | |=20 -------------------------------------------------=20 | ForCES Interface |=20 -------------------------------------------------=20 ^ ^ =20 ForCES | |data =20 control | |packets =20 messages| |(e.g., routing packets) =20 v v=20 -------------------------------------------------=20 | ForCES Interface |=20 -------------------------------------------------=20 | | | | | | |=20 |LPM Fwd|Meter |Shaper |NAT |Classi-|=85 |=20 | | | | |fier | |=20 -------------------------------------------------=20 | FE resources |=20 -------------------------------------------------=20 =20 Figure 3. Examples of CE and FE functions=20 =20 Some examples of control functions that can be implemented in = the CE=20 include routing protocols like RIP, OSPF and BGP, control and=20 signaling protocols like CAC (Call Admission Control), LDP = (Label=20 Distribution Protocol) for MPLS, etc. Examples of forwarding=20 functions in FE include LPM (longest prefix match) forwarder,=20 classifiers, traffic shaper, meter, NAT, etc. Figure 3 shows a=20 diagram with examples in both CE and FE. Any given NE may = contain=20 one or many of these CE and FE functions in it. The diagram = also=20 shows that ForCES protocol is used to transport both the = control=20 messages for ForCES itself and the data packets that are=20 originated/destined from/to the control functions in CE (e.g.,=20 routing packets). Section 4.2.4 provides more detail on this. =20 =20 A set of requirements for control and forwarding separation is=20 identified in [FORCES-REQ]. This document describes a ForCES=20 =20 Yang, et. al. Expires April 2003 [Page 6] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 architecture that satisfies the architectural requirements of = that=20 document and defines a framework for ForCES network elements = and the=20 associated entities to facilitate protocol definition. Whenever = necessary, this document uses many examples to illustrate the = issues=20 and/or possible solutions in ForCES. These examples are = intended to=20 be just examples, and should not be taken as the only or = definite=20 ways of doing certain things. It is expected that separate = document=20 will be produced by the ForCES working group to specify the = ForCES=20 protocol(s).=20 =20 3. Architecture=20 =20 This section defines the ForCES architectural framework and the = associated logical components. This ForCES framework defines=20 components of ForCES NEs including several ancillary = components. =20 These components may be connected in different kinds of = topologies=20 for flexible packet processing. =20 =20 --------------------------------------- = | ForCES Network Element | = -------------- Fc | -------------- -------------- | = | CE Manager |---------+-| CE 1 |------| CE 2 | | = -------------- | | | Fr | | | = | | -------------- -------------- | = | Fl | | | Fp / | = | | Fp| |----------| / | = | | | |/ | = | | | | | = | | | Fp /|----| | = | | | /--------/ | | = -------------- Ff | -------------- -------------- | = | FE Manager |---------+-| FE 1 | Fi | FE 2 | | = -------------- | | |------| | | = =20 | -------------- -------------- | = | | | | | | | | | | = ----+--+--+--+----------+--+--+--+----- = | | | | | | | |=20 | | | | | | | |=20 Fi/f Fi/f=20 =20 Figure 4. ForCES Architectural Diagram=20 =20 The diagram in Figure 4 shows the logical components of the = ForCES=20 architecture and their relationships. There are two kinds of=20 components inside a ForCES network element: control element = (CE) and=20 forwarding element (FE). The framework allows multiple = instances of=20 CE and FE inside one NE. Each FE contains one or more physical = media=20 interfaces for receiving and transmitting packets from/to the=20 external world. The aggregation of these FE interfaces becomes = the=20 NE=92s external interfaces. In addition to the external = interfaces,=20 there must also exist some kind of interconnect within the NE = so=20 that the CE and FE can communicate with each other, and one FE = can=20 forward packets to another FE. The diagram also shows two = entities=20 =20 Yang, et. al. Expires April 2003 [Page 7] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 outside of the ForCES NE: CE Manager and FE Manager. These two=20 entities provide configuration to the corresponding CE or FE in = the=20 pre-association phase (see Section 5.1). There is no defined = role=20 for FE Manager and CE Manager in post-association phase, thus = these=20 logical components are not considered part of the ForCES NE. =20 =20 For convenience, the logical interactions between these = components=20 are labeled by reference points Fp, Fc, Ff, Fr, Fl, and Fi, as = shown=20 in Figure 4. The FE external interfaces are labeled as Fi/f. = More=20 detail is provided in Section 4 and 5 for each of these = reference=20 points. All these reference points are important in = understanding=20 the ForCES architecture, however, the ForCES protocol is only=20 defined over one reference point -- Fp. =20 =20 The interface between two ForCES NEs is identical to the = interface=20 between two conventional routers and these two NEs exchange the = protocol packets through the external interfaces at Fi/f. = ForCES NEs=20 connect to existing routers transparently.=20 =20 3.1. Control Elements and Fr Reference Point=20 =20 It is not necessary to define any protocols across the Fr = reference=20 point to enable control and forwarding separation for simple=20 configurations like single CE and multiple FEs. However, this=20 architecture permits multiple CEs to be present in a network=20 element. In cases where an implementation uses multiple CEs, = the=20 invariant that the CEs and FEs together appear as a single NE = must=20 be maintained.=20 =20 Multiple CEs may be used for redundancy, load sharing, = distributed=20 control, or other purposes. Redundancy is the case where one = or=20 more CEs are prepared to take over should an active CE fail. = Load=20 sharing is the case where two or more CEs are concurrently = active=20 and where any request that can be serviced by one of the CEs = can=20 also be serviced by any of the other CEs. For both redundancy = and=20 load sharing, the CEs involved are equivalently capable. The = only=20 difference between these two cases is in terms of how many = active=20 CEs there are. Distributed control is the case where two or = more=20 CEs are concurrently active but where certain requests can only = be=20 serviced by certain CEs.=20 =20 When multiple CEs are employed in a ForCES NE, their internal=20 organization is considered an implementation issue that is = beyond=20 the scope of ForCES. CEs are wholly responsible for = coordinating=20 amongst themselves via the Fr reference point to provide = consistency=20 and synchronization. However, ForCES does not define the=20 implementation or protocols used between CEs, nor does it = define how=20 to distribute functionality among CEs. Nevertheless, ForCES = will=20 support mechanisms for CE redundancy or fail over, and it is=20 expected that vendors will provide redundancy or fail over = solutions=20 within this framework.=20 =20 =20 =20 Yang, et. al. Expires April 2003 [Page 8] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 3.2. Forwarding Elements and Fi reference point=20 =20 FEs perform per-packet processing and handling as directed by = CEs.=20 FEs have no initiative of their own. Instead, FEs are slaves = and=20 only do as they are told. FEs may communicate with one or more = CEs=20 concurrently across reference point Fp. FEs have no notion of = CE=20 redundancy, load sharing, or distributed control. Instead, FEs = accept commands from any CE authorized to control them, and it = is up=20 to the CEs to coordinate among themselves to achieve = redundancy,=20 load sharing or distributed control. The idea is to keep FEs as = simple and dumb as possible so that FEs can focus its resource = on=20 the packet processing functions. =20 =20 =20 ------- Fr -------=20 | CE1 | ------| CE2 |=20 ------- -------=20 | \ / |=20 | \ / |=20 | \ / |=20 | \/Fp |=20 | /\ |=20 | / \ |=20 | / \ |=20 ------- Fi -------=20 | FE1 |<----->| FE2 |=20 ------- -------=20 =20 Figure 5. CE redundancy example.=20 =20 For example, in Figure 5, FE1 and FE2 can be configured to = accept=20 commands from both the primary CE (CE1) and the backup CE = (CE2).=20 Upon detection of CE1 failure, perhaps across the Fr reference=20 point, CE2 is configured to take over activities of CE1. This = is=20 beyond the scope of ForCES and is not discussed further.=20 =20 Distributed control can be achieved in the similar fashion, = without=20 much intelligence on the part of FEs. For example, FEs can be=20 configured to detect RSVP and BGP protocol packets, and forward = RSVP=20 packets to one CE and BGP packets to another CE. Hence, FEs may = need=20 to do packet filtering for forwarding packets to specific CEs.=20 =20 This architecture permits multiple FEs to be present in a NE.=20 [FORCES-REQ] dictates that the ForCES protocol must be able to = scale=20 to at least hundreds of FEs (see [FORCES-REQ] Section 5, = requirement=20 #11). Each of these FEs may potentially have a different set of = packet processing functions, with different media interfaces. = FEs=20 are responsible for basic maintenance of layer-2 connectivity = with=20 other FEs and with external entities. Many layer-2 media = include=20 sophisticated control protocols. The FORCES protocol (over the = Fp=20 reference point) will be able to carry messages for such = protools so=20 =20 Yang, et. al. Expires April 2003 [Page 9] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 that, in keeping with the "dumb FE=92=92 model, the CE can = provide=20 appropriate intelligence and control over these media.=20 =20 When multiple FEs are present, ForCES requires that packets = must be=20 able to arrive at the NE by one FE and leave the NE via a = different=20 FE (See [FORCES-REQ], Section 5, Requirement #3). Packets that = enter the NE via one FE and leave the NE via a different FE are = transferred between FEs across the Fi reference point. The Fi=20 reference point is a separate protocol from the Fp reference = point=20 and is not currently defined by the ForCES architecture.=20 =20 =20 =20 ----------------- =20 | CE | =20 ----------------- =20 ^ ^ ^=20 / | \=20 / v \=20 / ------- \=20 / +->| FE3 |<-+ \=20 / | | | | \=20 v | ------- | v=20 ------- | | -------=20 | FE1 |<-+ +->| FE2 |=20 | |<--------------->| |=20 ------- -------=20 ^ | ^ |=20 | | | |=20 | v | v=20 =20 (a) Full mesh among FE1, FE2 and FE3.=20 =20 =20 ----------- =20 | CE | =20 ----------- =20 ^ ^ ^ ^=20 / | | \=20 /------ | | ------\=20 v v v v=20 ------- ------- ------- -------=20 | FE1 |<->| FE2 |<->| FE3 |<->| FE4 |=20 ------- ------- ------- -------=20 ^ | ^ | ^ | ^ |=20 | | | | | | | |=20 | v | v | v | v=20 =20 (b) Multiple FEs in a daisy chain=20 =20 =20 =20 ^ |=20 =20 Yang, et. al. Expires April 2003 [Page 10] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 | v=20 ----------- =20 | FE1 |<-----------------------| =20 ----------- |=20 ^ ^ |=20 / \ | =20 | ^ / \ ^ | V=20 v | v v | v ---------- = --------- --------- | | = | FE2 | | FE3 |<------------>| CE | = --------- --------- | | = ^ ^ ^ ---------- = | \ / ^ ^=20 | \ / | |=20 | v v | |=20 | ----------- | |=20 | | FE4 |<----------------------| | =20 | ----------- |=20 | | ^ |=20 | v | |=20 | |=20 |----------------------------------------|=20 =20 (c) Multiple FEs connected by a ring=20 =20 Figure 6. Some examples of FE topology.=20 =20 FEs could be connected in different kinds of topologies and = packet=20 processing may spread across several FEs in the topology. = Hence,=20 logical packet flow may be different from physical FE topology. = Figure 6 provides some topology examples. When it is necessary = to=20 forward packets between FEs, the CE needs to understand the FE=20 topology. The FE topology can be queried from the FEs by CEs. =20 =20 3.3. CE Managers=20 =20 CE managers are responsible for determining which FEs a CE = should=20 control. It is legitimate for CE managers to be hard-coded = with the=20 knowledge of with which FEs its CEs should communicate. A CE = manager=20 may also be physically embedded into a CE and be implemented as = a=20 simple keypad or other direct configuration mechanism on the = CE. =20 Finally, CE managers may be physically and logically separate=20 entities that configure the CE with FE information via such=20 mechanisms as COPS-PR [RFC3084] or SNMP [RFC1157].=20 =20 3.4. FE Managers=20 =20 FE managers are responsible for determining to which CE any=20 particular FE should initially communicate. Like CE managers, = no=20 restrictions are placed on how a FE manager decides to which CE = its=20 FEs should communicate, nor are restrictions placed on how FE=20 managers are implemented.=20 =20 =20 Yang, et. al. Expires April 2003 [Page 11] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 =20 4. Operational Phases=20 =20 Both FEs and CEs require some configuration in place before = they can=20 start information exchange and function as a coherent network=20 element. Two operational phases are identified in this = framework --=20 pre-association and post-association. =20 =20 4.1.Pre-association Phase=20 =20 Pre-association phase is the period of time during which a FE=20 Manager and a CE Manager are determining which FE and CE should = be=20 part of the same network element. The protocols used during = this=20 phase may include all or some of the message exchange over Fl, = Ff=20 and Fc reference points. However, all these may be optional and = none=20 of this is within the scope of ForCES protocol.=20 =20 4.1.1. Fl Reference Point=20 =20 =20 FE Manager FE CE Manager CE=20 | | | |=20 | | | |=20 |(security exchange) | |=20 1|<------------------------------>| |=20 | | | |=20 |(a list of CEs and their attributes) |=20 2|<-------------------------------| |=20 | | | |=20 |(a list of FEs and their attributes) |=20 3|------------------------------->| |=20 | | | |=20 | | | |=20 |<----------------Fl------------>| |=20 =20 Figure 7. An example of message exchange over Fl reference = point=20 =20 CE managers and FE managers may communicate across the Fl = reference=20 point in the pre-association phase in order to determine which = CEs=20 and FEs should communicate with each other. Communication = across=20 the Fl reference point is optional in this architecture. No=20 requirements are placed on this reference point.=20 =20 CE managers and FE managers may be operated by different = entities. =20 The operator of the CE manager may not want to divulge, except = to=20 specified FE managers, any characteristics of the CEs it = manages. =20 Similarly, the operator of the FE manager may not want to = divulge FE=20 characteristics, except to authorized entities. As such, CE=20 managers and FE managers may need to authenticate one another. = Subsequent communication between CE managers and FE managers = may=20 require other security functions such as privacy, = non-repudiation,=20 freshness, and integrity.=20 =20 =20 Yang, et. al. Expires April 2003 [Page 12] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 Once the necessary security functions have been performed, the = CE=20 and FE managers communicate to determine which CEs and FEs = should=20 communicate with each other. At the very minimum, the CE and = FE=20 managers need to learn of the existence of available FEs and = CEs=20 respectively. This discovery process may or may not entail one = or=20 both managers learning the capabilities of the discovered = ForCES=20 protocol elements. Figure 7 shows an example of possible = message=20 exchange between CE manager and FE manager over Fl reference = point.=20 =20 4.1.2. Ff Reference Point=20 =20 FE Manager FE CE Manager CE=20 | | | |=20 | | | |=20 |(security exchange) |(security exchange)=20 1|<------------>|authentication = 1|<----------->|authentication | | | |=20 |(FE ID, attributes) |(CE ID, attributes)=20 2|<-------------|request 2|<------------|request=20 | | | |=20 3|------------->|response 3|------------>|response=20 |(corresponding CE ID) |(corresponding FE ID)=20 | | | |=20 | | | |=20 |<-----Ff----->| |<-----Fc---->|=20 =20 Figure 8. Examples of message exchange=20 over Ff and Fc reference points.=20 =20 The Ff reference point is used to inform forwarding elements of = the=20 association decisions made by FE managers in pre-association = phase. =20 Only authorized entities may instruct a FE with respect to = which CE=20 should control it. Therefore, privacy, integrity, freshness, = and=20 authentication are necessary between FE manager and FEs when = the FE=20 manager is remote to the FE. Once the appropriate security has = been=20 established, FE manager instructs FEs across this reference = point to=20 join a new NE or to disconnect from an existing NE. Figure 8 = shows=20 example of message exchange over Ff reference point.=20 =20 Note that the FE manager function may be co-located with the FE = (such as by manual keypad entry of the CE IP address), in which = case=20 this reference point is reduced to a built-in function.=20 =20 4.1.3. Fc Reference Point=20 =20 The Fc reference point is used to inform control elements of = the=20 association decisions made by CE managers in pre-association = phase. =20 When the CE manager is remote, only authorized entities may = instruct=20 a CE to control certain FEs. Privacy, integrity, freshness and = authentication are also required across this reference point in = such=20 a configuration. Once appropriate security has been = established,=20 the CE manager instructs CEs as to which FEs they should = control and=20 =20 Yang, et. al. Expires April 2003 [Page 13] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 how they should control them. Figure 7 shows example of message = exchange over Fc reference point.=20 =20 As with the FE manager and FEs, configurations are possible = where=20 the CE manager and CE are co-located and no protocol is used = for=20 this function.=20 =20 4.2. Post-association Phase and Fp reference point=20 =20 Post-association phase is the period of time during which a FE = and=20 CE have been configured with information necessary to contact = each=20 other and includes both association establishment and = steady-state=20 communication. The communication between CE and FE is performed = across the Fp ("p" meaning protocol) reference point. ForCES=20 protocol is exclusively used for all communication across the = Fp=20 reference point. =20 =20 4.2.1. Proximity and Interconnect between CEs and FEs=20 =20 The ForCES Working Group has made a conscious decision that the = first version of ForCES will not be designed to support=20 configurations where the CE and FE are located arbitrarily in = the=20 network. In particular, ForCES is intended for "very close" = CE/FE=20 localities in IP networks, as defined by ForCES Applicability=20 Statement ([FORCES-APP]). Very Close localities consist of = control=20 and forwarding elements that either are components in the same=20 physical box, or are separated at most by one local network = hop. =20 =20 CEs and FEs can be connected by a variety of interconnect=20 technologies, including Ethernet connections, backplanes, ATM = (cell)=20 fabrics, etc. ForCES should be able to support each of these=20 interconnects (see [FORCES-REQ] Section 5, requirement #1). = ForCES=20 will make use of an existing RFC2914 compliant L4 protocol with = adequate reliability, security and congestion control (e.g. = TCP,=20 SCTP) for transport purposes.=20 =20 4.2.2. Association Establishment=20 =20 As an example, figure 9 shows some of the message exchange that = may=20 happen before the association between the CE and FE is fully=20 established. Either the CE or FE can initiate the connection. = The FE=20 needs to inform the CE of its own capability and its topology = in=20 relation to other FEs. The capability of the FE is represented = by=20 the FE model, described in another separate document. The model = would allow a FE to describe what kind of packet processing=20 functions it contains, in what order the processing happens, = what=20 kinds of configurable parameters it allows, what statistics it=20 collects and what events it might throw, etc. Once such = information=20 is available to the CE, the CE sends all necessary = configuration to=20 the FE so that the FE can start receiving and processing = packets=20 correctly. For example, the CE might need to send a snapshot of = the=20 current forwarding table to the FE so that the FE can start = routing=20 packets correctly. Once FE starts accepting packets for = processing,=20 =20 Yang, et. al. Expires April 2003 [Page 14] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 we say the association of this FE with its CE is now = established.=20 From then on, the CE and FE enter steady-state communication.=20 =20 =20 FE CE=20 | |=20 |(Hello, are you there?)|=20 1|<----------------------|=20 | |=20 |(Yes. let me join the NE please.) =20 2|---------------------->|=20 | |=20 |(What kind of FE are you? -- capability query)=20 3|<----------------------|=20 | |=20 |(Here is my FE functions/state: use model to = describe)=20 4|---------------------->|=20 | |=20 |(How are you connected with others? -- topology = query)=20 5|<----------------------|=20 | |=20 |(Here is the topology info)=20 6|---------------------->|=20 | |=20 |(Config for FE, e.g. forwarding table)=20 7|<----------------------|=20 | |=20 |(I am ready to go. Shall I?) =20 8|---------------------->|=20 | |=20 |(Go ahead!) |=20 9|<----------------------|=20 | |=20 =20 Figure 9. Example of message exchange between CE and FE=20 over Fp to establish NE association=20 =20 =20 4.2.3. Steady-state Communication=20 =20 Once an association is established between the CE and FE, the = ForCES=20 protocol is used by the CE and FE over Fp reference point to=20 exchange information to facilitate packet processing. =20 =20 FE CE=20 | |=20 |(Add these new routes.)|=20 1|<----------------------|=20 | |=20 |(Successful.) |=20 2|---------------------->|=20 | |=20 | |=20 =20 Yang, et. al. Expires April 2003 [Page 15] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 |(Query some stats.) |=20 1|<----------------------|=20 | |=20 |(Reply with stats collected.) =20 2|---------------------->|=20 | |=20 | |=20 |(My port is down, with port #.) = =20 1|---------------------->|=20 | |=20 |(Route to this port instead...) =20 2|<----------------------|=20 | |=20 | |=20 =20 Figure 10. Examples of message exchange between CE and FE = over Fp during steady-state communication=20 =20 Based on the information acquired through CEs' control = processing,=20 CEs will frequently need to manipulate the packet-forwarding=20 behaviors of their FE(s) by sending instructions to FEs. For=20 example, Figure 10 shows message exchange examples in which the = CE=20 sends new routes to the FE so that the FE can add them to its=20 forwarding table. The CE may query the FE for statistics = collected=20 by the FE and the FE may notify the CE of important events such = as=20 port failure.=20 =20 4.2.4. Data Packets across Fp reference point=20 =20 Control plane protocol packets (such as RIP, OSPF messages)=20 addressed to any of NE's interfaces are typically redirected by = the=20 receiving FE to its CE, and CE may originate packets and have = its FE=20 deliver them to other NEs. Therefore, ForCES protocol over Fp = not=20 only transports the ForCES protocol messages between CEs and = FEs,=20 but also encapsulates the data packets from control plane = protocols.=20 Moreover, one FE may be controlled by multiple CEs for = distributed=20 control. In this configuration, the control protocols supported = by=20 the FORCES NEs may spread across multiple CEs. For example, one = CE=20 may support routing protocols like OSPF and BGP, while = signaling and=20 admission control protocol like RSVP is supported in another = CE. FEs=20 are configured to recognize and filter these protocol packets = and=20 forward them to the corresponding CE. =20 =20 Figure 11 shows one example of how the BGP packets originated = by=20 router A are passed to router B. In this example, the ForCES=20 protocol is used to transport the packets from the CE to the FE = inside router A, and then from the FE to the CE inside router = B. In=20 light of the fact that the ForCES protocol is responsible to=20 transport both the control messages and the data packets = between the=20 CE and FE over Fp reference point, it is possible to use either = a=20 single protocol or multiple protocols to achieve that.=20 =20 --------------------- ----------------------=20 =20 Yang, et. al. Expires April 2003 [Page 16] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 | | | |=20 | +--------+ | | +--------+ |=20 | |CE(BGP) | | | |CE(BGP) | |=20 | +--------+ | | +--------+ |=20 | | | | ^ |=20 | |Fp | | |Fp |=20 | v | | | |=20 | +--------+ | | +--------+ |=20 | | FE | | | | FE | |=20 | +--------+ | | +--------+ |=20 | | | | ^ |=20 | Router | | | Router | |=20 | A | | | B | |=20 ---------+----------- -----------+----------=20 v ^=20 | |=20 | |=20 ------------------->--------------- =20 =20 Figure 11. Example to show data packet flow between two = NEs.=20 =20 =20 4.2.5. Proxy FE=20 =20 In the case where a physical FE cannot implement (e.g., due to = the=20 lack of a general purpose CPU) the ForCES protocol directly, a = proxy=20 FE can be used in the middle of Fp reference point. This allows = the=20 CE communicate to the physical FE via the proxy by using = ForCES,=20 while the proxy manipulates the physical FE using some = intermediary=20 form of communication (e.g., a non-ForCES protocol or DMA). In = such=20 an implementation, the combination of the proxy and the = physical FE=20 becomes one logical FE entity. =20 =20 4.3. Association Re-establishment=20 =20 FEs and CEs may join and leave NEs dynamically (see = [FORCES-REQ]=20 Section 5, requirements #12). When a FE or CE leaves the NE, = the=20 association with the NE is broken. If the leaving party rejoins = a NE=20 later, to re-establish the association, it may or may not need = to=20 re-enter the pre-association phase. Loss of association can = also=20 happen unexpectedly due to loss of connection between the CE = and the=20 FE. Therefore, the framework allows the bi-directional = transition=20 between these two phases, but the ForCES protocol is only = applicable=20 for the post-association phase. However, the protocol should = provide=20 mechanisms to support association re-establishment (see = [FORCES-REQ]=20 Section 5, requirement #7).=20 =20 The example in Figure 5 is used to illustrate what happens when = the=20 association is broken and later re-established again. Section = 4.2=20 already explains what happens if CE1 fails and how CE2 can take = over. If no CE redundancy is provided, at the association=20 establishment phase FEs need to be told what to do in the case = of CE=20 failure. FEs may be told to stop packet processing all together = if=20 =20 Yang, et. al. Expires April 2003 [Page 17] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 its CE fails. Or, FEs may be told to continue forwarding = packets for=20 a limited time even in the face of CE failure. No matter what = the=20 instruction is, it needs to be part of the configuration when = the=20 association is established. =20 =20 In the same example in Figure 5, assuming CE1 is the working CE = for=20 the moment, what would happen if one of the FEs, say FE1, = leaves the=20 NE temporarily? FE1 may voluntarily decide to leave the = association.=20 Or, FE1 may stop functioning simply due to unexpected failure. = In=20 former case, CE1 receives a "leave-association request" from = FE1. In=20 the latter, CE1 detects the failure of FE1 by some other mean. = In=20 both cases, CE1 keeps a note of such event from FE1 while = continue=20 commanding FE2. When FE1 decides to rejoin again, or when it is = back=20 up again from the failure, FE1 needs to re-discover its master = (CE).=20 This can be achieved by several means. It may re-enter the pre- association phase and get that information from its FE manager. = It=20 may retrieve the previous CE information from its cache, if it = can=20 validate the information freshness. Once it discovers its CE, = it=20 starts message exchange with CE to re-establish the association = just=20 as outlined in Figure 9, with the possible exception that it = might=20 be able to bypass the transport of the complete initialization=20 information. Suppose that FE1 still has its routing table and = other=20 state information from the last association, instead of sending = all=20 the information again from scratch, it can choose to use more=20 efficient mechanism to re-sync up the state with its CE. For=20 example, a checksum of the state might give a quick indication = of=20 whether or not the state is in-sync with its CE. By comparing = its=20 state with CE first, it sends information update only if it is=20 needed.=20 =20 =20 5. Applicability to RFC1812 =20 =20 [FORCES-REQ] Section 5, requirement #9 dictates that "All = proposed=20 ForCES architecture must explain how that architecture supports = all=20 of the router functions as defined in [RFC1812]." RFC1812 = discusses=20 many important requirements for IPv4 routers from the link = layer to=20 the application layer. This section addresses the relevant=20 requirements in RFC1812 for implementing IPv4 routers based on=20 ForCES architecture and explains how ForCES satisfies these=20 requirements by providing guidelines on how to separate the=20 functionalities required into forwarding plane and control = plane. =20 =20 In general, the forwarding plane carries out the bulk of the = per- packet processing that is required at line speed, while the = control=20 plane carries most of the computationally complex operations = that=20 are typical of the control and signaling protocols. However, it = is=20 impossible to draw a rigid line to divide the processing into = CEs=20 and FEs cleanly. Nor should the ForCES architecture limits the=20 innovative approaches in control and forwarding plane = separation. As=20 more and more processing power is available in the FEs, some of = the=20 control functions that traditionally are performed by CEs may = now be=20 moved to FEs for better performance and scalability. Such = offloaded=20 =20 Yang, et. al. Expires April 2003 [Page 18] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 functions may include part of ICMP or TCP processing, or part = of=20 routing protocols. Once off-loaded onto the forwarding plane, = such=20 CE functions, even though logically belong to the control = plane, now=20 become part of the FE logical functions. Just like the other = logical=20 functions performed by FEs, such off-loaded functions must be=20 expressed as part of the FE model so that the CEs can decide = how to=20 best take advantage of these off-loaded functions when present = on=20 the FEs.=20 =20 =20 5.1. General Router Requirements=20 =20 Routers have at least two or more logical interfaces. When CEs = and=20 FEs are separated by ForCES within a single NE, some additional = interfaces are needed for intra-NE communications. Figure 12 = shows=20 an example to illustrate that. This NE contains one CE and two = FEs.=20 Each FE has four interfaces; two of them are used for receiving = and=20 transmitting packets to the external world, while the other two = are=20 for intra-NE connections. CE has two logical interfaces #9 and = #10,=20 connected to interfaces #3 and #6 from FE1 and FE2, = respectively.=20 Interface #4 and #5 are connected for FE1-FE2 communication.=20 Therefore, this router NE provides four external interfaces = (#1, 2,=20 7 and 8). =20 =20 IPv4 routers must implement IP to support its packet forwarding = function, which is driven by its FIB (Forwarding Information = Base).=20 This Internet layer forwarding (see [RFC1812] Section 5)=20 functionality naturally belongs to FEs in the ForCES = architecture. =20 =20 ---------------------------------=20 | router NE |=20 | ----------- ----------- |=20 | | FE1 | | FE2 | |=20 | ----------- ----------- |=20 | 1| 2| 3| 4| 5| 6| 7| 8| |=20 | | | | | | | | | |=20 | | | | +----+ | | | |=20 | | | | | | | |=20 | | | 9| 10| | | |=20 | | | -------------- | | |=20 | | | | CE | | | |=20 | | | -------------- | | |=20 | | | | | |=20 -----+--+----------------+--+----=20 | | | | =20 | | | | =20 =20 Figure 12. A router NE example with four = interfaces.=20 =20 =20 A router may implement transport layer protocols (like TCP and = UDP)=20 that are required to support application layer protocols (see=20 [RFC1812] Section 6). One important class of application = protocols=20 =20 Yang, et. al. Expires April 2003 [Page 19] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 is routing protocols (see [RFC1812] Section 7). In ForCES=20 architecture, routing protocols are naturally implemented by = CEs.=20 Routing protocols require routers communicate with each other. = This=20 communication between CEs in different routers is supported in=20 ForCES by FEs' ability to redirect data packets addressed to = routers=20 (i.e., NEs) and CEs' ability to originate packets and have them = delivered by their FEs. This communication occurs across Fp=20 reference point inside each router and between neighboring = routers'=20 external interfaces, as illustrated in Figure 11. =20 =20 5.2.Link Layer=20 =20 Since FEs own all the external interfaces for the router, FEs = need=20 to conform to the link layer requirements in RFC1812. Arguably, = ARP=20 support may be implemented in either CEs or FEs. As we will see = later, a number of behaviors that RFC1812 mandates fall into = this=20 category -- they may be performed by the FE and may be = performed by=20 the CE. A general guideline is needed to ensure = interoperability=20 between separated control and forwarding planes. The guideline = we=20 offer here is that in general CEs are required to be capable of = these kind of operations while FEs may or may not choose to=20 implement them. FE model should indicate its capabilities in = this=20 regard so that CEs can decide where these functions are = implemented.=20 =20 Interface parameters, including MTU, IP address, etc., must be=20 configurable by CEs via ForCES. CEs must be able to determine=20 whether a physical interface in an FE is available to send = packets=20 or not. FEs must also inform CEs the status change of the = interfaces=20 (like link up/down) via ForCES.=20 =20 5.3.Internet Layer Protocols=20 =20 Both FEs and CEs must implement IP protocol and all mandatory=20 extensions as RFC1812 specified. CEs should implement IP = options=20 like source route and record route while FEs may choose to = implement=20 those as well. Timestamp option should be implemented by FEs to = insert the timestamp most accurately. FE must interpret the IP=20 options that it understands and preserve the rest unchanged for = use=20 by CEs. Both FEs and CEs might choose to silently discard = packets=20 without sending ICMP errors, but such events should be logged = and=20 counted. FEs may report statistics for such events to CEs via=20 ForCES. =20 =20 When multiple FEs are involved to process packets, the = appearance of=20 single NE must be strictly maintained. For example, = Time-To-Live=20 (TTL) must be decremented only once within a single NE. For = example,=20 it can be always decremented by the last FE with egress = function. =20 =20 FEs must receive and process normally any packets with a = broadcast=20 destination address or a multicast destination address that the = router has asked to receive. When IP multicast is supported in=20 routers, IGMP is implemented in CEs. CEs are also required of = ICMP=20 support, while it is optional for FEs to support ICMP. Such an=20 =20 Yang, et. al. Expires April 2003 [Page 20] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 option can be communicated to CEs as part of the FE model.=20 Therefore, FEs can always rely upon CEs to send out ICMP error=20 messages, but FEs also have the option to generate ICMP error=20 messages themselves. =20 =20 5.4.Internet Layer Forwarding=20 =20 IP forwarding is implemented by FEs. When the routing table is=20 updated at CEs, ForCES is used to send the new route entries = from=20 CEs to FEs. Each FE has its own forwarding table and uses this = table=20 to direct packets to the next hop interface. =20 =20 Upon receiving IP packets, FE verifies the IP header and = processes=20 most of the IP options. Some options can't be processed until = the=20 routing decision has been made. Routing decision is made after=20 examining the destination IP address. If the destination = address=20 belongs to the router itself, the packets are filtered and = either=20 processed locally or forwarded to CE, depending upon the=20 instructions set-up by CE. Otherwise, FE determines the next = hop IP=20 address by looking up in its forwarding table. FE also = determines=20 the network interface it uses to send the packets. Sometimes FE = may=20 need to forward the packets to another FE before packets can be = forwarded out to the next hop. Right before packets are = forwarded=20 out to the next hop, FE decrements TTL by 1 and processes any = IP=20 options that cannot be processed before. FE performs any IP=20 fragmentation if necessary, determines link layer address = (e.g., by=20 ARP), and encapsulates the IP datagram (or each of the = fragments=20 thereof) in an appropriate link layer frame and queues it for = output=20 on the interface selected.=20 =20 Other options mentioned in RFC1812 for IP forwarding may also = be=20 implemented at FEs, for example, packet filtering.=20 =20 FEs typically forward packets destined locally to CEs. FEs may = also=20 forward exceptional packets (packets that FEs don't know how to = handle) to CEs. CEs are required to handle packets forwarded by = FEs=20 for whatever different reasons. It might be necessary for = ForCES to=20 attach some meta-data with the packets to indicate the reasons = of=20 forwarding from FEs to CEs. Upon receiving packets with = meta-data=20 from FEs, CEs can decide to either process the packets = themselves,=20 or pass the packets to the upper layer protocols including = routing=20 and management protocols. If CEs are to process the packets by=20 themselves, CEs may choose to discard the packets, or modify = and re- send the packets. CEs may also originate new packets and = deliver=20 them to FEs for further forwarding. =20 =20 Any state change during router operation must also be handled=20 correctly according to RFC1812. For example, when an FE ceases=20 forwarding, the entire NE may continue forwarding packets, but = it=20 needs to stop advertising routes that are affected by the = failed FE. =20 =20 5.5.Transport Layer =20 =20 =20 Yang, et. al. Expires April 2003 [Page 21] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 Transport layer is typically implemented at CEs to support = higher=20 layer application protocols like routing protocols. In = practice,=20 this means that most CEs implement both the Transmission = Control=20 Protocol (TCP) and the User Datagram Protocol (UDP).=20 =20 Both CEs and FEs need to implement ForCES protocol. If some = layer-4=20 transport is used to support ForCES, then both CEs and FEs need = to=20 implement the L4 transport and ForCES protocols. It is possible = that=20 all FEs inside an NE implements only one such protocol entity. = =20 5.6. Application Layer -- Routing Protocols=20 =20 Both interior routing protocols and exterior routing protocols = are=20 implemented on CEs. The routing packets originated by CEs are=20 forwarded to FEs for delivery. The results of such protocols = (like=20 forwarding table update) are communicated to FEs via ForCES.=20 =20 For performance or scalability reasons, portions of the control = plane functions that need faster response may be moved from the = CEs=20 and off-loaded onto the FEs. For example in OSPF, the Hello = protocol=20 packets are generated and processed periodically. When done at = CEs,=20 the inbound Hello packets have to traverse from the external=20 interfaces at the FEs to the CEs via the internal CE-FE = channel.=20 Similarly, the outbound Hello packets have to go from the CEs = to the=20 FEs and to the external interfaces. Frequent Hello updates = place=20 heavy processing overhead on the CEs and can overwhelm the = CE-FE=20 channel as well. Since typically there are far more FEs than = CEs in=20 a router, when off-loaded, the Hello packets are processed in a = much=20 more distributed and scalable fashion. Interoperability is = ensured=20 by expressing such off-loaded functions in the FE model.=20 =20 5.7. Application Layer -- Network Management Protocol=20 =20 RFC1812 also dictates "Routers MUST be manageable by SNMP." = (see=20 [RFC1157] Section 8) In general, for post-association phase, = most=20 external management tasks (including SNMP) should be done = through=20 interaction with the CE in order to support the appearance of a = single functional device. Therefore, it is recommended that = SNMP=20 management agent be implemented by CEs and the SNMP messages=20 received by FEs be redirected to their CEs. AgentX framework = defined=20 in RFC2741 may be applied here such that CEs act in the role of = master agent to process SNMP protocol messages while FEs act in = the=20 role of subagent to provide access to the MIB objects residing = on=20 FEs. AgentX protocol messages between the master agent (CE) and = the=20 subagent (FE) are encapsulated and transported via ForCES, just = like=20 data packets from any other application layer protocols. =20 =20 6. Summary=20 =20 This document defines an architectural framework for ForCES. It = identifies the relevant components for a ForCES network = element,=20 including (one or more) FEs, (one or more) CEs, one optional FE = manager, and one optional CE manager. It also identifies the=20 =20 Yang, et. al. Expires April 2003 [Page 22] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 interaction among these components and discusses all the major=20 reference points. It is important to point out that, among all = the=20 reference points, only the interface between CEs and FEs is = within=20 the scope of ForCES. ForCES alone may not be enough to support = all=20 desirablet NE configurations. However, we believe that ForCES = is the=20 most important element in realizing physical separation and=20 interoperability of CEs and FEs, and hence the first interface = that=20 ought to be standardized. Simple and useful configurations can = still=20 be implemented with only CE-FE interface being standardized, = e.g.,=20 single CE with full-meshed FEs and static configuration without = the=20 need for CE/FE managers.=20 =20 7. Security Considerations=20 =20 The security necessary across each reference point except Fp is = discussed throughout the document. In general, the physical=20 separation of two entities usually requires much stricter = security=20 measurement in place. For example, we pointed out in Section = 5.1=20 that authentication becomes necessary between CE manager and FE = manager, between CE and CE manager, between FE and FE manager = in=20 some configuration. The physical separation of CE and FE also=20 imposes serious security requirement for ForCES protocol. The=20 security requirements for reference point Fp (i.e., ForCES = protocol)=20 are discussed in detail in [FORCES-REQ] Section 8.=20 =20 8. Intellectual Property Right=20 =20 The authors are not aware of any intellectual property right = issues=20 pertaining to this document.=20 =20 9. References=20 =20 9.1. Normative References=20 =20 [RFC1157] J. Case, et. al., "A Simple Network Management = Protocol=20 (SNMP)", RFC1157, May 1990.=20 =20 [RFC1812] F. Baker, "Requirements for IP Version 4 Routers",=20 RFC1812, June 1995.=20 =20 [RFC2741] M. Daniele, et. al., "Agent Extensibility (AgentX)=20 Protocol Version 1", RFC2741, Jan 2000.=20 =20 [RFC2914] S. Floyd, "Congestion Control Principles", RFC2914,=20 September 2000.=20 =20 [RFC3084] K. Chan, et. al., "COPS Usage for Policy Provisioning = (COPS-PR)", RFC3084, March 2001.=20 =20 9.2. Informative References=20 =20 =20 Yang, et. al. Expires April 2003 [Page 23] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 [FORCES-REQ] T. Anderson, et. al., "Requirements for Separation = of=20 IP Control and Forwarding", work in progress, July 2002, = .=20 =20 [FORCES-APP] A. Crouch, et. al., "ForCES Applicability = Statement",=20 work in progress, June 2002, .=20 =20 10. Acknowledgments=20 =20 Joel M. Halpern gave us many insightful comments and = suggestions and=20 pointed out several major issues. Many of our colleagues and = people=20 in the ForCES mailing list also provided valuable feedback.=20 =20 11. Authors' Addresses=20 =20 Lily L. Yang=20 Intel Corp., MS JF3-206,=20 2111 NE 25th Avenue=20 Hillsboro, OR 97124 USA=20 Phone: +1 503 264 8813=20 Email: lily.l.yang@intel.com=20 Ram Dantu=20 University of Texas Dallas=20 2601 North Flyod Road=20 Richardson Texas 75082=20 Phone: +1 972 883 4653=20 Email: ram.dantu@utdallas.edu=20 =20 Todd A. Anderson=20 Intel Corp.=20 2111 NE 25th Avenue=20 Hillsboro, OR 97124 USA=20 Phone: +1 503 712 1760=20 Email: todd.a.anderson@intel.com=20 =20 =20 =20 Yang, et. al. Expires April 2003 [Page 24] = =0C ------_=_NextPart_000_01C276E2.C76C86A0 Content-Type: text/plain; name="draft-ietf-forces-framework-03.txt" Content-Disposition: attachment; filename="draft-ietf-forces-framework-03.txt" Content-Transfer-Encoding: quoted-printable Internet Draft L. Yang=20 Expiration: April 2003 Intel Corp.=20 File: draft-ietf-forces-framework-03.txt R. Dantu=20 Working Group: ForCES Univ. of Texas = Dallas=20 T. Anderson=20 Intel Corp. =20 Oct 2002 = =20 =20 =20 ForCES Architectural Framework=20 =20 =20 =20 draft-ietf-forces-framework-03.txt=20 =20 =20 =20 =20 Status of this Memo=20 =20 This document is an Internet-Draft and is in full conformance = with=20 all provisions of Section 10 of RFC2026. Internet-Drafts are=20 working documents of the Internet Engineering Task Force = (IETF), its=20 areas, and its working groups. Note that other groups may also = distribute working documents as Internet-Drafts.=20 =20 Internet-Drafts are draft documents valid for a maximum of six=20 months and may be updated, replaced, or obsoleted by other = documents=20 at any time. It is inappropriate to use Internet-Drafts as=20 reference material or to cite them other than as ``work in=20 progress.''=20 =20 The list of current Internet-Drafts can be accessed at=20 http://www.ietf.org/ietf/1id-abstracts.txt.=20 =20 The list of Internet-Draft Shadow Directories can be accessed = at =20 http://www.ietf.org/shadow.html.=20 =20 Abstract=20 =20 This document defines the architectural framework for the = ForCES=20 (Forwarding and Control Element Separation) network elements = (NE),=20 and identifies the associated entities and the interaction = among=20 them. This framework is intended to satisfy the requirements=20 specified in the ForCES requirements draft [FORCES-REQ].=20 =20 Table of Contents=20 =20 1. = Definitions.....................................................2=20 2. Introduction to Forwarding and Control Element Separation=20 = (ForCES)...........................................................4=20 3. = Architecture....................................................7=20 3.1. Control Elements and Fr Reference = Point....................8=20 =20 =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 3.2. Forwarding Elements and Fi reference = point.................9=20 3.3. CE = Managers...............................................11=20 3.4. FE = Managers...............................................11=20 4. Operational = Phases.............................................12=20 4.1. Pre-association = Phase.....................................12=20 4.1.1. Fl Reference = Point...................................12=20 4.1.2. Ff Reference = Point...................................13=20 4.1.3. Fc Reference = Point...................................13=20 4.2. Post-association Phase and Fp reference = point.............14=20 4.2.1. Proximity and Interconnect between CEs and = FEs.......14=20 4.2.2. Association = Establishment............................14=20 4.2.3. Steady-state = Communication...........................15=20 4.2.4. Data Packets across Fp reference = point...............16=20 4.2.5. Proxy = FE.............................................17=20 4.3. Association = Re-establishment..............................17=20 5. Applicability to = RFC1812.......................................18=20 5.1. General Router = Requirements...............................19=20 5.2. Link = Layer................................................20=20 5.3. Internet Layer = Protocols..................................20=20 5.4. Internet Layer = Forwarding.................................21=20 5.5. Transport = Layer...........................................21=20 5.6. Application Layer -- Routing = Protocols....................22=20 5.7. Application Layer -- Network Management = Protocol..........22=20 6. = Summary........................................................22=20 7. Security = Considerations........................................23=20 8. Intellectual Property = Right....................................23=20 9. = References.....................................................23=20 9.1. Normative = References......................................23=20 9.2. Informative = References....................................23=20 10. = Acknowledgments...............................................24=20 11. Authors' = Addresses............................................24=20 =20 Conventions used in this document =20 =20 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL = NOT", =20 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" = in=20 this document are to be interpreted as described in [RFC-2119]. = =20 1. Definitions=20 =20 A set of terminology associated with the ForCES requirements is = defined in [FORCES-REQ] and we only include the ones that are = most=20 relevant to this document here. =20 =20 Forwarding Element (FE) - A logical entity that implements the=20 ForCES protocol. FEs use the underlying hardware to provide = per- packet processing and handling as directed by a CE via the = ForCES=20 protocol. =20 =20 Control Element (CE) - A logical entity that implements the = ForCES=20 protocol and uses it to instruct one or more FEs how to process = packets. CEs handle functionality such as the execution of = control =20 and signaling protocols. =20 =20 Yang, et. al. Expires April 2003 [Page 2] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 =20 ForCES Network Element (NE) - An entity composed of one or more = CEs=20 and one or more FEs. To entities outside a NE, the NE = represents a=20 single point of management. Similarly, a NE usually hides its=20 internal organization from external entities. =20 =20 Pre-association Phase - The period of time during which a FE = Manager=20 (see below) and a CE Manager (see below) are determining which = FE=20 and CE should be part of the same network element.=20 =20 Post-association Phase - The period of time during which a FE = does=20 know which CE is to control it and vice versa, including the = time=20 during which the CE and FE are establishing communication with = one=20 another. =20 =20 ForCES Protocol - While there may be multiple protocols used = within=20 the overall ForCES architecture, the term "ForCES protocol" = refers=20 only to the ForCES post-association phase protocol (see below). = =20 =20 ForCES Post-Association Phase Protocol - The protocol used for = post- association phase communication between CEs and FEs. This = protocol=20 does not apply to CE-to-CE communication, FE-to-FE = communication, or=20 to communication between FE and CE managers. The ForCES = protocol is=20 a master-slave protocol in which FEs are slaves and CEs are = masters. =20 This protocol includes both the management of the communication = channel (e.g., connection establishment, heartbeats) and the = control=20 messages themselves. This protocol could be a single protocol = or=20 could consist of multiple protocols working together. =20 =20 FE Manager - A logical entity that operates in the = pre-association=20 phase and is responsible for determining to which CE(s) a FE = should=20 communicate. This process is called CE discovery and may = involve=20 the FE manager learning the capabilities of available CEs. A = FE=20 manager may use anything from a static configuration to a pre- association phase protocol (see below) to determine which CE(s) = to=20 use. Being a logical entity, a FE manager might be physically=20 combined with any of the other logical entities mentioned in = this=20 section. =20 =20 CE Manager - A logical entity that operates in the = pre-association =20 phase and is responsible for determining to which FE(s) a CE = should =20 communicate. This process is called FE discovery and may = involve=20 the CE manager learning the capabilities of available FEs. A = CE=20 manager may use anything from a static configuration to a pre- association phase protocol (see below) to determine which FE to = use. =20 Being a logical entity, a CE manager might be physically = combined=20 with any of the other logical entities mentioned in this = section. =20 =20 Pre-association Phase Protocol - A protocol between FE managers = and=20 CE managers that is used to determine which CEs or FEs to use. = A=20 pre-association phase protocol may include a CE and/or FE = capability=20 discovery mechanism. Note that this capability discovery = process is=20 wholly separate from (and does not replace) that used within = the=20 =20 Yang, et. al. Expires April 2003 [Page 3] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 ForCES protocol. However, the two capability discovery = mechanisms=20 may utilize the same FE model. =20 =20 FE Model - A model that describes the logical processing = functions=20 of a FE. =20 =20 ForCES Protocol Element - A FE or CE. =20 =20 2. Introduction to Forwarding and Control Element Separation = (ForCES) =20 =20 An IP network element (NE) appears to external entities as a=20 monolithic piece of network equipment, e.g., a router, NAT,=20 firewall, or load balancer. Internally, however, an IP network=20 element (NE) (such as a router) is composed of numerous = logically=20 separated entities that cooperate to provide a given = functionality=20 (such as routing). Two types of network element components = exist:=20 control element (CE) in control plane and forwarding element = (FE) in=20 forwarding plane (or data plane). Forwarding elements = typically are=20 ASIC, network-processor, or general-purpose processor-based = devices=20 that handle data path operations for each packet. Control = elements=20 are typically based on general-purpose processors that provide=20 control functionality like routing and signaling protocols. =20 =20 ForCES aims to define a framework and associated protocol(s) to = standardize information exchange between the control and = forwarding=20 plane. Having standard mechanisms allows CEs and FEs to become=20 physically separated standard components. This physical = separation=20 accrues several benefits to the ForCES architecture. Separate=20 components would allow component vendors to specialize in one=20 component without having to become experts in all components.=20 Standard protocol also allows the CEs and FEs from different=20 component vendors to interoperate with each other and hence it=20 becomes possible for system vendors to integrate together the = CEs=20 and FEs from different component suppliers. This = interoperability=20 translates into a lot more design choices and flexibility to = the=20 system vendors. Overall, ForCES will enable rapid innovation in = both=20 the control and forwarding planes while maintaining=20 interoperability. Scalability is also easily provided by this=20 architecture in that additional forwarding or control capacity = can=20 be added to existing network elements without the need for = forklift=20 upgrades.=20 =20 One example of such physical separation is at the blade level.=20 Figure 1 shows such an example configuration of a router, with = two=20 control blades and multiple router (forwarding) blades, all=20 interconnected into a switch fabric backplane. In such chassis=20 configuration, the control blades are the CEs while the router=20 blades are FEs, and the switch fabric backplane provides the=20 physical interconnect for all the blades. Control blade A may = be the=20 primary CE while control blade B is the backup CE providing=20 redundancy. It is also possible to have a redundant switch = fabric=20 for high availability support. Routers today with this kind of=20 configuration use proprietary interface for messaging between = CEs=20 =20 Yang, et. al. Expires April 2003 [Page 4] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 and FEs. The goal of ForCES is to replace such proprietary = interface=20 with a standard protocol. With a standard protocol like ForCES=20 implemented on all blades, it becomes possible for control = blades=20 from vendor X and routing blades from vendor Y to work = seamlessly=20 together in one chassis. =20 =20 ------------------------- -------------------------=20 | Control Blade A | | Control Blade B |=20 | (CE) | | (CE) |=20 ------------------------- -------------------------=20 ^ | ^ |=20 | | | |=20 | V | V=20 ---------------------------------------------------------=20 | Switch Fabric Backplane |=20 ---------------------------------------------------------=20 ^ | ^ | ^ | = | | | | =85 | | = =20 | V | V | V = ------------ ------------ ------------ = |Router | |Router | |Router |=20 |Blade #1 | |Blade #2 | |Blade #N |=20 | (FE) | | (FE) | | (FE) |=20 ------------ ------------ ------------ = ^ | ^ | ^ | = | | | | =85 | | = =20 | V | V | V = =20 Figure 1. A router configuration example with separate = blades.=20 =20 =20 ------- -------=20 | CE1 | | CE2 |=20 ------- -------=20 ^ ^=20 | |=20 V V=20 = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Ethernet=20 ^ ^ =85 ^=20 | | |=20 V V V=20 ------- ------- --------=20 | FE#1| | FE#2| | FE#n |=20 ------- ------- --------=20 ^ | ^ | ^ | =20 | | | | | | =20 | V | V | V =20 =20 Figure 2. A router configuration example with separate = boxes.=20 =20 Another level of physical separation between the CEs and FEs = can be=20 at the box level. In such configuration, all the CEs and FEs = are=20 physically separated boxes, interconnected with some kind of = high=20 =20 Yang, et. al. Expires April 2003 [Page 5] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 speed LAN connection (like Gigabit Ethernet). These separated = CEs=20 and FEs are only one hop away from each other within a local = area=20 network. The CEs and FEs communicate to each other by running=20 ForCES, and the collection of these CEs and FEs together become = one=20 routing unit to the external world. Figure 2 shows such an = example.=20 =20 In both examples shown here, the same physical interconnect is = used=20 for both CE-to-FE and FE-to-FE communication. However, that = does not=20 have to be the case. One reason to use different interconnect = is=20 that CE-to-FE interconnect does not have to be as fast as the = FE-to- FE interconnect, so the more expensive fast connections can be = saved=20 for FE-to-FE only. The separate interconnects may also provide = reliability and redundancy benefits for the NE.=20 =20 -------------------------------------------------=20 | | | | | | |=20 |OSPF |RIP |BGP |CAC |LDP |=85 |=20 | | | | | | |=20 -------------------------------------------------=20 | ForCES Interface |=20 -------------------------------------------------=20 ^ ^ =20 ForCES | |data =20 control | |packets =20 messages| |(e.g., routing packets) =20 v v=20 -------------------------------------------------=20 | ForCES Interface |=20 -------------------------------------------------=20 | | | | | | |=20 |LPM Fwd|Meter |Shaper |NAT |Classi-|=85 |=20 | | | | |fier | |=20 -------------------------------------------------=20 | FE resources |=20 -------------------------------------------------=20 =20 Figure 3. Examples of CE and FE functions=20 =20 Some examples of control functions that can be implemented in = the CE=20 include routing protocols like RIP, OSPF and BGP, control and=20 signaling protocols like CAC (Call Admission Control), LDP = (Label=20 Distribution Protocol) for MPLS, etc. Examples of forwarding=20 functions in FE include LPM (longest prefix match) forwarder,=20 classifiers, traffic shaper, meter, NAT, etc. Figure 3 shows a=20 diagram with examples in both CE and FE. Any given NE may = contain=20 one or many of these CE and FE functions in it. The diagram = also=20 shows that ForCES protocol is used to transport both the = control=20 messages for ForCES itself and the data packets that are=20 originated/destined from/to the control functions in CE (e.g.,=20 routing packets). Section 4.2.4 provides more detail on this. =20 =20 A set of requirements for control and forwarding separation is=20 identified in [FORCES-REQ]. This document describes a ForCES=20 =20 Yang, et. al. Expires April 2003 [Page 6] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 architecture that satisfies the architectural requirements of = that=20 document and defines a framework for ForCES network elements = and the=20 associated entities to facilitate protocol definition. Whenever = necessary, this document uses many examples to illustrate the = issues=20 and/or possible solutions in ForCES. These examples are = intended to=20 be just examples, and should not be taken as the only or = definite=20 ways of doing certain things. It is expected that separate = document=20 will be produced by the ForCES working group to specify the = ForCES=20 protocol(s).=20 =20 3. Architecture=20 =20 This section defines the ForCES architectural framework and the = associated logical components. This ForCES framework defines=20 components of ForCES NEs including several ancillary = components. =20 These components may be connected in different kinds of = topologies=20 for flexible packet processing. =20 =20 --------------------------------------- = | ForCES Network Element | = -------------- Fc | -------------- -------------- | = | CE Manager |---------+-| CE 1 |------| CE 2 | | = -------------- | | | Fr | | | = | | -------------- -------------- | = | Fl | | | Fp / | = | | Fp| |----------| / | = | | | |/ | = | | | | | = | | | Fp /|----| | = | | | /--------/ | | = -------------- Ff | -------------- -------------- | = | FE Manager |---------+-| FE 1 | Fi | FE 2 | | = -------------- | | |------| | | = =20 | -------------- -------------- | = | | | | | | | | | | = ----+--+--+--+----------+--+--+--+----- = | | | | | | | |=20 | | | | | | | |=20 Fi/f Fi/f=20 =20 Figure 4. ForCES Architectural Diagram=20 =20 The diagram in Figure 4 shows the logical components of the = ForCES=20 architecture and their relationships. There are two kinds of=20 components inside a ForCES network element: control element = (CE) and=20 forwarding element (FE). The framework allows multiple = instances of=20 CE and FE inside one NE. Each FE contains one or more physical = media=20 interfaces for receiving and transmitting packets from/to the=20 external world. The aggregation of these FE interfaces becomes = the=20 NE=92s external interfaces. In addition to the external = interfaces,=20 there must also exist some kind of interconnect within the NE = so=20 that the CE and FE can communicate with each other, and one FE = can=20 forward packets to another FE. The diagram also shows two = entities=20 =20 Yang, et. al. Expires April 2003 [Page 7] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 outside of the ForCES NE: CE Manager and FE Manager. These two=20 entities provide configuration to the corresponding CE or FE in = the=20 pre-association phase (see Section 5.1). There is no defined = role=20 for FE Manager and CE Manager in post-association phase, thus = these=20 logical components are not considered part of the ForCES NE. =20 =20 For convenience, the logical interactions between these = components=20 are labeled by reference points Fp, Fc, Ff, Fr, Fl, and Fi, as = shown=20 in Figure 4. The FE external interfaces are labeled as Fi/f. = More=20 detail is provided in Section 4 and 5 for each of these = reference=20 points. All these reference points are important in = understanding=20 the ForCES architecture, however, the ForCES protocol is only=20 defined over one reference point -- Fp. =20 =20 The interface between two ForCES NEs is identical to the = interface=20 between two conventional routers and these two NEs exchange the = protocol packets through the external interfaces at Fi/f. = ForCES NEs=20 connect to existing routers transparently.=20 =20 3.1. Control Elements and Fr Reference Point=20 =20 It is not necessary to define any protocols across the Fr = reference=20 point to enable control and forwarding separation for simple=20 configurations like single CE and multiple FEs. However, this=20 architecture permits multiple CEs to be present in a network=20 element. In cases where an implementation uses multiple CEs, = the=20 invariant that the CEs and FEs together appear as a single NE = must=20 be maintained.=20 =20 Multiple CEs may be used for redundancy, load sharing, = distributed=20 control, or other purposes. Redundancy is the case where one = or=20 more CEs are prepared to take over should an active CE fail. = Load=20 sharing is the case where two or more CEs are concurrently = active=20 and where any request that can be serviced by one of the CEs = can=20 also be serviced by any of the other CEs. For both redundancy = and=20 load sharing, the CEs involved are equivalently capable. The = only=20 difference between these two cases is in terms of how many = active=20 CEs there are. Distributed control is the case where two or = more=20 CEs are concurrently active but where certain requests can only = be=20 serviced by certain CEs.=20 =20 When multiple CEs are employed in a ForCES NE, their internal=20 organization is considered an implementation issue that is = beyond=20 the scope of ForCES. CEs are wholly responsible for = coordinating=20 amongst themselves via the Fr reference point to provide = consistency=20 and synchronization. However, ForCES does not define the=20 implementation or protocols used between CEs, nor does it = define how=20 to distribute functionality among CEs. Nevertheless, ForCES = will=20 support mechanisms for CE redundancy or fail over, and it is=20 expected that vendors will provide redundancy or fail over = solutions=20 within this framework.=20 =20 =20 =20 Yang, et. al. Expires April 2003 [Page 8] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 3.2. Forwarding Elements and Fi reference point=20 =20 FEs perform per-packet processing and handling as directed by = CEs.=20 FEs have no initiative of their own. Instead, FEs are slaves = and=20 only do as they are told. FEs may communicate with one or more = CEs=20 concurrently across reference point Fp. FEs have no notion of = CE=20 redundancy, load sharing, or distributed control. Instead, FEs = accept commands from any CE authorized to control them, and it = is up=20 to the CEs to coordinate among themselves to achieve = redundancy,=20 load sharing or distributed control. The idea is to keep FEs as = simple and dumb as possible so that FEs can focus its resource = on=20 the packet processing functions. =20 =20 =20 ------- Fr -------=20 | CE1 | ------| CE2 |=20 ------- -------=20 | \ / |=20 | \ / |=20 | \ / |=20 | \/Fp |=20 | /\ |=20 | / \ |=20 | / \ |=20 ------- Fi -------=20 | FE1 |<----->| FE2 |=20 ------- -------=20 =20 Figure 5. CE redundancy example.=20 =20 For example, in Figure 5, FE1 and FE2 can be configured to = accept=20 commands from both the primary CE (CE1) and the backup CE = (CE2).=20 Upon detection of CE1 failure, perhaps across the Fr reference=20 point, CE2 is configured to take over activities of CE1. This = is=20 beyond the scope of ForCES and is not discussed further.=20 =20 Distributed control can be achieved in the similar fashion, = without=20 much intelligence on the part of FEs. For example, FEs can be=20 configured to detect RSVP and BGP protocol packets, and forward = RSVP=20 packets to one CE and BGP packets to another CE. Hence, FEs may = need=20 to do packet filtering for forwarding packets to specific CEs.=20 =20 This architecture permits multiple FEs to be present in a NE.=20 [FORCES-REQ] dictates that the ForCES protocol must be able to = scale=20 to at least hundreds of FEs (see [FORCES-REQ] Section 5, = requirement=20 #11). Each of these FEs may potentially have a different set of = packet processing functions, with different media interfaces. = FEs=20 are responsible for basic maintenance of layer-2 connectivity = with=20 other FEs and with external entities. Many layer-2 media = include=20 sophisticated control protocols. The FORCES protocol (over the = Fp=20 reference point) will be able to carry messages for such = protools so=20 =20 Yang, et. al. Expires April 2003 [Page 9] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 that, in keeping with the "dumb FE=92=92 model, the CE can = provide=20 appropriate intelligence and control over these media.=20 =20 When multiple FEs are present, ForCES requires that packets = must be=20 able to arrive at the NE by one FE and leave the NE via a = different=20 FE (See [FORCES-REQ], Section 5, Requirement #3). Packets that = enter the NE via one FE and leave the NE via a different FE are = transferred between FEs across the Fi reference point. The Fi=20 reference point is a separate protocol from the Fp reference = point=20 and is not currently defined by the ForCES architecture.=20 =20 =20 =20 ----------------- =20 | CE | =20 ----------------- =20 ^ ^ ^=20 / | \=20 / v \=20 / ------- \=20 / +->| FE3 |<-+ \=20 / | | | | \=20 v | ------- | v=20 ------- | | -------=20 | FE1 |<-+ +->| FE2 |=20 | |<--------------->| |=20 ------- -------=20 ^ | ^ |=20 | | | |=20 | v | v=20 =20 (a) Full mesh among FE1, FE2 and FE3.=20 =20 =20 ----------- =20 | CE | =20 ----------- =20 ^ ^ ^ ^=20 / | | \=20 /------ | | ------\=20 v v v v=20 ------- ------- ------- -------=20 | FE1 |<->| FE2 |<->| FE3 |<->| FE4 |=20 ------- ------- ------- -------=20 ^ | ^ | ^ | ^ |=20 | | | | | | | |=20 | v | v | v | v=20 =20 (b) Multiple FEs in a daisy chain=20 =20 =20 =20 ^ |=20 =20 Yang, et. al. Expires April 2003 [Page 10] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 | v=20 ----------- =20 | FE1 |<-----------------------| =20 ----------- |=20 ^ ^ |=20 / \ | =20 | ^ / \ ^ | V=20 v | v v | v ---------- = --------- --------- | | = | FE2 | | FE3 |<------------>| CE | = --------- --------- | | = ^ ^ ^ ---------- = | \ / ^ ^=20 | \ / | |=20 | v v | |=20 | ----------- | |=20 | | FE4 |<----------------------| | =20 | ----------- |=20 | | ^ |=20 | v | |=20 | |=20 |----------------------------------------|=20 =20 (c) Multiple FEs connected by a ring=20 =20 Figure 6. Some examples of FE topology.=20 =20 FEs could be connected in different kinds of topologies and = packet=20 processing may spread across several FEs in the topology. = Hence,=20 logical packet flow may be different from physical FE topology. = Figure 6 provides some topology examples. When it is necessary = to=20 forward packets between FEs, the CE needs to understand the FE=20 topology. The FE topology can be queried from the FEs by CEs. =20 =20 3.3. CE Managers=20 =20 CE managers are responsible for determining which FEs a CE = should=20 control. It is legitimate for CE managers to be hard-coded = with the=20 knowledge of with which FEs its CEs should communicate. A CE = manager=20 may also be physically embedded into a CE and be implemented as = a=20 simple keypad or other direct configuration mechanism on the = CE. =20 Finally, CE managers may be physically and logically separate=20 entities that configure the CE with FE information via such=20 mechanisms as COPS-PR [RFC3084] or SNMP [RFC1157].=20 =20 3.4. FE Managers=20 =20 FE managers are responsible for determining to which CE any=20 particular FE should initially communicate. Like CE managers, = no=20 restrictions are placed on how a FE manager decides to which CE = its=20 FEs should communicate, nor are restrictions placed on how FE=20 managers are implemented.=20 =20 =20 Yang, et. al. Expires April 2003 [Page 11] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 =20 4. Operational Phases=20 =20 Both FEs and CEs require some configuration in place before = they can=20 start information exchange and function as a coherent network=20 element. Two operational phases are identified in this = framework --=20 pre-association and post-association. =20 =20 4.1.Pre-association Phase=20 =20 Pre-association phase is the period of time during which a FE=20 Manager and a CE Manager are determining which FE and CE should = be=20 part of the same network element. The protocols used during = this=20 phase may include all or some of the message exchange over Fl, = Ff=20 and Fc reference points. However, all these may be optional and = none=20 of this is within the scope of ForCES protocol.=20 =20 4.1.1. Fl Reference Point=20 =20 =20 FE Manager FE CE Manager CE=20 | | | |=20 | | | |=20 |(security exchange) | |=20 1|<------------------------------>| |=20 | | | |=20 |(a list of CEs and their attributes) |=20 2|<-------------------------------| |=20 | | | |=20 |(a list of FEs and their attributes) |=20 3|------------------------------->| |=20 | | | |=20 | | | |=20 |<----------------Fl------------>| |=20 =20 Figure 7. An example of message exchange over Fl reference = point=20 =20 CE managers and FE managers may communicate across the Fl = reference=20 point in the pre-association phase in order to determine which = CEs=20 and FEs should communicate with each other. Communication = across=20 the Fl reference point is optional in this architecture. No=20 requirements are placed on this reference point.=20 =20 CE managers and FE managers may be operated by different = entities. =20 The operator of the CE manager may not want to divulge, except = to=20 specified FE managers, any characteristics of the CEs it = manages. =20 Similarly, the operator of the FE manager may not want to = divulge FE=20 characteristics, except to authorized entities. As such, CE=20 managers and FE managers may need to authenticate one another. = Subsequent communication between CE managers and FE managers = may=20 require other security functions such as privacy, = non-repudiation,=20 freshness, and integrity.=20 =20 =20 Yang, et. al. Expires April 2003 [Page 12] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 Once the necessary security functions have been performed, the = CE=20 and FE managers communicate to determine which CEs and FEs = should=20 communicate with each other. At the very minimum, the CE and = FE=20 managers need to learn of the existence of available FEs and = CEs=20 respectively. This discovery process may or may not entail one = or=20 both managers learning the capabilities of the discovered = ForCES=20 protocol elements. Figure 7 shows an example of possible = message=20 exchange between CE manager and FE manager over Fl reference = point.=20 =20 4.1.2. Ff Reference Point=20 =20 FE Manager FE CE Manager CE=20 | | | |=20 | | | |=20 |(security exchange) |(security exchange)=20 1|<------------>|authentication = 1|<----------->|authentication | | | |=20 |(FE ID, attributes) |(CE ID, attributes)=20 2|<-------------|request 2|<------------|request=20 | | | |=20 3|------------->|response 3|------------>|response=20 |(corresponding CE ID) |(corresponding FE ID)=20 | | | |=20 | | | |=20 |<-----Ff----->| |<-----Fc---->|=20 =20 Figure 8. Examples of message exchange=20 over Ff and Fc reference points.=20 =20 The Ff reference point is used to inform forwarding elements of = the=20 association decisions made by FE managers in pre-association = phase. =20 Only authorized entities may instruct a FE with respect to = which CE=20 should control it. Therefore, privacy, integrity, freshness, = and=20 authentication are necessary between FE manager and FEs when = the FE=20 manager is remote to the FE. Once the appropriate security has = been=20 established, FE manager instructs FEs across this reference = point to=20 join a new NE or to disconnect from an existing NE. Figure 8 = shows=20 example of message exchange over Ff reference point.=20 =20 Note that the FE manager function may be co-located with the FE = (such as by manual keypad entry of the CE IP address), in which = case=20 this reference point is reduced to a built-in function.=20 =20 4.1.3. Fc Reference Point=20 =20 The Fc reference point is used to inform control elements of = the=20 association decisions made by CE managers in pre-association = phase. =20 When the CE manager is remote, only authorized entities may = instruct=20 a CE to control certain FEs. Privacy, integrity, freshness and = authentication are also required across this reference point in = such=20 a configuration. Once appropriate security has been = established,=20 the CE manager instructs CEs as to which FEs they should = control and=20 =20 Yang, et. al. Expires April 2003 [Page 13] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 how they should control them. Figure 7 shows example of message = exchange over Fc reference point.=20 =20 As with the FE manager and FEs, configurations are possible = where=20 the CE manager and CE are co-located and no protocol is used = for=20 this function.=20 =20 4.2. Post-association Phase and Fp reference point=20 =20 Post-association phase is the period of time during which a FE = and=20 CE have been configured with information necessary to contact = each=20 other and includes both association establishment and = steady-state=20 communication. The communication between CE and FE is performed = across the Fp ("p" meaning protocol) reference point. ForCES=20 protocol is exclusively used for all communication across the = Fp=20 reference point. =20 =20 4.2.1. Proximity and Interconnect between CEs and FEs=20 =20 The ForCES Working Group has made a conscious decision that the = first version of ForCES will not be designed to support=20 configurations where the CE and FE are located arbitrarily in = the=20 network. In particular, ForCES is intended for "very close" = CE/FE=20 localities in IP networks, as defined by ForCES Applicability=20 Statement ([FORCES-APP]). Very Close localities consist of = control=20 and forwarding elements that either are components in the same=20 physical box, or are separated at most by one local network = hop. =20 =20 CEs and FEs can be connected by a variety of interconnect=20 technologies, including Ethernet connections, backplanes, ATM = (cell)=20 fabrics, etc. ForCES should be able to support each of these=20 interconnects (see [FORCES-REQ] Section 5, requirement #1). = ForCES=20 will make use of an existing RFC2914 compliant L4 protocol with = adequate reliability, security and congestion control (e.g. = TCP,=20 SCTP) for transport purposes.=20 =20 4.2.2. Association Establishment=20 =20 As an example, figure 9 shows some of the message exchange that = may=20 happen before the association between the CE and FE is fully=20 established. Either the CE or FE can initiate the connection. = The FE=20 needs to inform the CE of its own capability and its topology = in=20 relation to other FEs. The capability of the FE is represented = by=20 the FE model, described in another separate document. The model = would allow a FE to describe what kind of packet processing=20 functions it contains, in what order the processing happens, = what=20 kinds of configurable parameters it allows, what statistics it=20 collects and what events it might throw, etc. Once such = information=20 is available to the CE, the CE sends all necessary = configuration to=20 the FE so that the FE can start receiving and processing = packets=20 correctly. For example, the CE might need to send a snapshot of = the=20 current forwarding table to the FE so that the FE can start = routing=20 packets correctly. Once FE starts accepting packets for = processing,=20 =20 Yang, et. al. Expires April 2003 [Page 14] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 we say the association of this FE with its CE is now = established.=20 From then on, the CE and FE enter steady-state communication.=20 =20 =20 FE CE=20 | |=20 |(Hello, are you there?)|=20 1|<----------------------|=20 | |=20 |(Yes. let me join the NE please.) =20 2|---------------------->|=20 | |=20 |(What kind of FE are you? -- capability query)=20 3|<----------------------|=20 | |=20 |(Here is my FE functions/state: use model to = describe)=20 4|---------------------->|=20 | |=20 |(How are you connected with others? -- topology = query)=20 5|<----------------------|=20 | |=20 |(Here is the topology info)=20 6|---------------------->|=20 | |=20 |(Config for FE, e.g. forwarding table)=20 7|<----------------------|=20 | |=20 |(I am ready to go. Shall I?) =20 8|---------------------->|=20 | |=20 |(Go ahead!) |=20 9|<----------------------|=20 | |=20 =20 Figure 9. Example of message exchange between CE and FE=20 over Fp to establish NE association=20 =20 =20 4.2.3. Steady-state Communication=20 =20 Once an association is established between the CE and FE, the = ForCES=20 protocol is used by the CE and FE over Fp reference point to=20 exchange information to facilitate packet processing. =20 =20 FE CE=20 | |=20 |(Add these new routes.)|=20 1|<----------------------|=20 | |=20 |(Successful.) |=20 2|---------------------->|=20 | |=20 | |=20 =20 Yang, et. al. Expires April 2003 [Page 15] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 |(Query some stats.) |=20 1|<----------------------|=20 | |=20 |(Reply with stats collected.) =20 2|---------------------->|=20 | |=20 | |=20 |(My port is down, with port #.) = =20 1|---------------------->|=20 | |=20 |(Route to this port instead...) =20 2|<----------------------|=20 | |=20 | |=20 =20 Figure 10. Examples of message exchange between CE and FE = over Fp during steady-state communication=20 =20 Based on the information acquired through CEs' control = processing,=20 CEs will frequently need to manipulate the packet-forwarding=20 behaviors of their FE(s) by sending instructions to FEs. For=20 example, Figure 10 shows message exchange examples in which the = CE=20 sends new routes to the FE so that the FE can add them to its=20 forwarding table. The CE may query the FE for statistics = collected=20 by the FE and the FE may notify the CE of important events such = as=20 port failure.=20 =20 4.2.4. Data Packets across Fp reference point=20 =20 Control plane protocol packets (such as RIP, OSPF messages)=20 addressed to any of NE's interfaces are typically redirected by = the=20 receiving FE to its CE, and CE may originate packets and have = its FE=20 deliver them to other NEs. Therefore, ForCES protocol over Fp = not=20 only transports the ForCES protocol messages between CEs and = FEs,=20 but also encapsulates the data packets from control plane = protocols.=20 Moreover, one FE may be controlled by multiple CEs for = distributed=20 control. In this configuration, the control protocols supported = by=20 the FORCES NEs may spread across multiple CEs. For example, one = CE=20 may support routing protocols like OSPF and BGP, while = signaling and=20 admission control protocol like RSVP is supported in another = CE. FEs=20 are configured to recognize and filter these protocol packets = and=20 forward them to the corresponding CE. =20 =20 Figure 11 shows one example of how the BGP packets originated = by=20 router A are passed to router B. In this example, the ForCES=20 protocol is used to transport the packets from the CE to the FE = inside router A, and then from the FE to the CE inside router = B. In=20 light of the fact that the ForCES protocol is responsible to=20 transport both the control messages and the data packets = between the=20 CE and FE over Fp reference point, it is possible to use either = a=20 single protocol or multiple protocols to achieve that.=20 =20 --------------------- ----------------------=20 =20 Yang, et. al. Expires April 2003 [Page 16] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 | | | |=20 | +--------+ | | +--------+ |=20 | |CE(BGP) | | | |CE(BGP) | |=20 | +--------+ | | +--------+ |=20 | | | | ^ |=20 | |Fp | | |Fp |=20 | v | | | |=20 | +--------+ | | +--------+ |=20 | | FE | | | | FE | |=20 | +--------+ | | +--------+ |=20 | | | | ^ |=20 | Router | | | Router | |=20 | A | | | B | |=20 ---------+----------- -----------+----------=20 v ^=20 | |=20 | |=20 ------------------->--------------- =20 =20 Figure 11. Example to show data packet flow between two = NEs.=20 =20 =20 4.2.5. Proxy FE=20 =20 In the case where a physical FE cannot implement (e.g., due to = the=20 lack of a general purpose CPU) the ForCES protocol directly, a = proxy=20 FE can be used in the middle of Fp reference point. This allows = the=20 CE communicate to the physical FE via the proxy by using = ForCES,=20 while the proxy manipulates the physical FE using some = intermediary=20 form of communication (e.g., a non-ForCES protocol or DMA). In = such=20 an implementation, the combination of the proxy and the = physical FE=20 becomes one logical FE entity. =20 =20 4.3. Association Re-establishment=20 =20 FEs and CEs may join and leave NEs dynamically (see = [FORCES-REQ]=20 Section 5, requirements #12). When a FE or CE leaves the NE, = the=20 association with the NE is broken. If the leaving party rejoins = a NE=20 later, to re-establish the association, it may or may not need = to=20 re-enter the pre-association phase. Loss of association can = also=20 happen unexpectedly due to loss of connection between the CE = and the=20 FE. Therefore, the framework allows the bi-directional = transition=20 between these two phases, but the ForCES protocol is only = applicable=20 for the post-association phase. However, the protocol should = provide=20 mechanisms to support association re-establishment (see = [FORCES-REQ]=20 Section 5, requirement #7).=20 =20 The example in Figure 5 is used to illustrate what happens when = the=20 association is broken and later re-established again. Section = 4.2=20 already explains what happens if CE1 fails and how CE2 can take = over. If no CE redundancy is provided, at the association=20 establishment phase FEs need to be told what to do in the case = of CE=20 failure. FEs may be told to stop packet processing all together = if=20 =20 Yang, et. al. Expires April 2003 [Page 17] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 its CE fails. Or, FEs may be told to continue forwarding = packets for=20 a limited time even in the face of CE failure. No matter what = the=20 instruction is, it needs to be part of the configuration when = the=20 association is established. =20 =20 In the same example in Figure 5, assuming CE1 is the working CE = for=20 the moment, what would happen if one of the FEs, say FE1, = leaves the=20 NE temporarily? FE1 may voluntarily decide to leave the = association.=20 Or, FE1 may stop functioning simply due to unexpected failure. = In=20 former case, CE1 receives a "leave-association request" from = FE1. In=20 the latter, CE1 detects the failure of FE1 by some other mean. = In=20 both cases, CE1 keeps a note of such event from FE1 while = continue=20 commanding FE2. When FE1 decides to rejoin again, or when it is = back=20 up again from the failure, FE1 needs to re-discover its master = (CE).=20 This can be achieved by several means. It may re-enter the pre- association phase and get that information from its FE manager. = It=20 may retrieve the previous CE information from its cache, if it = can=20 validate the information freshness. Once it discovers its CE, = it=20 starts message exchange with CE to re-establish the association = just=20 as outlined in Figure 9, with the possible exception that it = might=20 be able to bypass the transport of the complete initialization=20 information. Suppose that FE1 still has its routing table and = other=20 state information from the last association, instead of sending = all=20 the information again from scratch, it can choose to use more=20 efficient mechanism to re-sync up the state with its CE. For=20 example, a checksum of the state might give a quick indication = of=20 whether or not the state is in-sync with its CE. By comparing = its=20 state with CE first, it sends information update only if it is=20 needed.=20 =20 =20 5. Applicability to RFC1812 =20 =20 [FORCES-REQ] Section 5, requirement #9 dictates that "All = proposed=20 ForCES architecture must explain how that architecture supports = all=20 of the router functions as defined in [RFC1812]." RFC1812 = discusses=20 many important requirements for IPv4 routers from the link = layer to=20 the application layer. This section addresses the relevant=20 requirements in RFC1812 for implementing IPv4 routers based on=20 ForCES architecture and explains how ForCES satisfies these=20 requirements by providing guidelines on how to separate the=20 functionalities required into forwarding plane and control = plane. =20 =20 In general, the forwarding plane carries out the bulk of the = per- packet processing that is required at line speed, while the = control=20 plane carries most of the computationally complex operations = that=20 are typical of the control and signaling protocols. However, it = is=20 not practical to draw a rigid line to divide the processing = into CEs=20 and FEs cleanly. Nor should the ForCES architecture limits the=20 innovative approaches in control and forwarding plane = separation. As=20 more and more processing power is available in the FEs, some of = the=20 control functions that traditionally are performed by CEs may = now be=20 moved to FEs for better performance and scalability. Such = offloaded=20 =20 Yang, et. al. Expires April 2003 [Page 18] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 functions may include part of ICMP or TCP processing, or part = of=20 routing protocols. Once off-loaded onto the forwarding plane, = such=20 CE functions, even though logically belong to the control = plane, now=20 become part of the FE logical functions. Just like the other = logical=20 functions performed by FEs, such off-loaded functions must be=20 expressed as part of the FE model so that the CEs can decide = how to=20 best take advantage of these off-loaded functions when present = on=20 the FEs.=20 =20 =20 5.1. General Router Requirements=20 =20 Routers have at least two or more logical interfaces. When CEs = and=20 FEs are separated by ForCES within a single NE, some additional = interfaces are needed for intra-NE communications. Figure 12 = shows=20 an example to illustrate that. This NE contains one CE and two = FEs.=20 Each FE has four interfaces; two of them are used for receiving = and=20 transmitting packets to the external world, while the other two = are=20 for intra-NE connections. CE has two logical interfaces #9 and = #10,=20 connected to interfaces #3 and #6 from FE1 and FE2, = respectively.=20 Interface #4 and #5 are connected for FE1-FE2 communication.=20 Therefore, this router NE provides four external interfaces = (#1, 2,=20 7 and 8). =20 =20 IPv4 routers must implement IP to support its packet forwarding = function, which is driven by its FIB (Forwarding Information = Base).=20 This Internet layer forwarding (see [RFC1812] Section 5)=20 functionality naturally belongs to FEs in the ForCES = architecture. =20 =20 ---------------------------------=20 | router NE |=20 | ----------- ----------- |=20 | | FE1 | | FE2 | |=20 | ----------- ----------- |=20 | 1| 2| 3| 4| 5| 6| 7| 8| |=20 | | | | | | | | | |=20 | | | | +----+ | | | |=20 | | | | | | | |=20 | | | 9| 10| | | |=20 | | | -------------- | | |=20 | | | | CE | | | |=20 | | | -------------- | | |=20 | | | | | |=20 -----+--+----------------+--+----=20 | | | | =20 | | | | =20 =20 Figure 12. A router NE example with four = interfaces.=20 =20 =20 A router may implement transport layer protocols (like TCP and = UDP)=20 that are required to support application layer protocols (see=20 [RFC1812] Section 6). One important class of application = protocols=20 =20 Yang, et. al. Expires April 2003 [Page 19] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 is routing protocols (see [RFC1812] Section 7). In ForCES=20 architecture, routing protocols are naturally implemented by = CEs.=20 Routing protocols require routers communicate with each other. = This=20 communication between CEs in different routers is supported in=20 ForCES by FEs' ability to redirect data packets addressed to = routers=20 (i.e., NEs) and CEs' ability to originate packets and have them = delivered by their FEs. This communication occurs across Fp=20 reference point inside each router and between neighboring = routers'=20 external interfaces, as illustrated in Figure 11. =20 =20 5.2.Link Layer=20 =20 Since FEs own all the external interfaces for the router, FEs = need=20 to conform to the link layer requirements in RFC1812. Arguably, = ARP=20 support may be implemented in either CEs or FEs. As we will see = later, a number of behaviors that RFC1812 mandates fall into = this=20 category -- they may be performed by the FE and may be = performed by=20 the CE. A general guideline is needed to ensure = interoperability=20 between separated control and forwarding planes. The guideline = we=20 offer here is that in general CEs are required to be capable of = these kind of operations while FEs may or may not choose to=20 implement them. FE model should indicate its capabilities in = this=20 regard so that CEs can decide where these functions are = implemented.=20 =20 Interface parameters, including MTU, IP address, etc., must be=20 configurable by CEs via ForCES. CEs must be able to determine=20 whether a physical interface in an FE is available to send = packets=20 or not. FEs must also inform CEs the status change of the = interfaces=20 (like link up/down) via ForCES.=20 =20 5.3.Internet Layer Protocols=20 =20 Both FEs and CEs must implement IP protocol and all mandatory=20 extensions as RFC1812 specified. CEs should implement IP = options=20 like source route and record route while FEs may choose to = implement=20 those as well. Timestamp option should be implemented by FEs to = insert the timestamp most accurately. FE must interpret the IP=20 options that it understands and preserve the rest unchanged for = use=20 by CEs. Both FEs and CEs might choose to silently discard = packets=20 without sending ICMP errors, but such events should be logged = and=20 counted. FEs may report statistics for such events to CEs via=20 ForCES. =20 =20 When multiple FEs are involved to process packets, the = appearance of=20 single NE must be strictly maintained. For example, = Time-To-Live=20 (TTL) must be decremented only once within a single NE. For = example,=20 it can be always decremented by the last FE with egress = function. =20 =20 FEs must receive and process normally any packets with a = broadcast=20 destination address or a multicast destination address that the = router has asked to receive. When IP multicast is supported in=20 routers, IGMP is implemented in CEs. CEs are also required of = ICMP=20 support, while it is optional for FEs to support ICMP. Such an=20 =20 Yang, et. al. Expires April 2003 [Page 20] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 option can be communicated to CEs as part of the FE model.=20 Therefore, FEs can always rely upon CEs to send out ICMP error=20 messages, but FEs also have the option to generate ICMP error=20 messages themselves. =20 =20 5.4.Internet Layer Forwarding=20 =20 IP forwarding is implemented by FEs. When the routing table is=20 updated at CEs, ForCES is used to send the new route entries = from=20 CEs to FEs. Each FE has its own forwarding table and uses this = table=20 to direct packets to the next hop interface. =20 =20 Upon receiving IP packets, FE verifies the IP header and = processes=20 most of the IP options. Some options can't be processed until = the=20 routing decision has been made. Routing decision is made after=20 examining the destination IP address. If the destination = address=20 belongs to the router itself, the packets are filtered and = either=20 processed locally or forwarded to CE, depending upon the=20 instructions set-up by CE. Otherwise, FE determines the next = hop IP=20 address by looking up in its forwarding table. FE also = determines=20 the network interface it uses to send the packets. Sometimes FE = may=20 need to forward the packets to another FE before packets can be = forwarded out to the next hop. Right before packets are = forwarded=20 out to the next hop, FE decrements TTL by 1 and processes any = IP=20 options that cannot be processed before. FE performs any IP=20 fragmentation if necessary, determines link layer address = (e.g., by=20 ARP), and encapsulates the IP datagram (or each of the = fragments=20 thereof) in an appropriate link layer frame and queues it for = output=20 on the interface selected.=20 =20 Other options mentioned in RFC1812 for IP forwarding may also = be=20 implemented at FEs, for example, packet filtering.=20 =20 FEs typically forward packets destined locally to CEs. FEs may = also=20 forward exceptional packets (packets that FEs don't know how to = handle) to CEs. CEs are required to handle packets forwarded by = FEs=20 for whatever different reasons. It might be necessary for = ForCES to=20 attach some meta-data with the packets to indicate the reasons = of=20 forwarding from FEs to CEs. Upon receiving packets with = meta-data=20 from FEs, CEs can decide to either process the packets = themselves,=20 or pass the packets to the upper layer protocols including = routing=20 and management protocols. If CEs are to process the packets by=20 themselves, CEs may choose to discard the packets, or modify = and re- send the packets. CEs may also originate new packets and = deliver=20 them to FEs for further forwarding. =20 =20 Any state change during router operation must also be handled=20 correctly according to RFC1812. For example, when an FE ceases=20 forwarding, the entire NE may continue forwarding packets, but = it=20 needs to stop advertising routes that are affected by the = failed FE. =20 =20 5.5.Transport Layer =20 =20 =20 Yang, et. al. Expires April 2003 [Page 21] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 Transport layer is typically implemented at CEs to support = higher=20 layer application protocols like routing protocols. In = practice,=20 this means that most CEs implement both the Transmission = Control=20 Protocol (TCP) and the User Datagram Protocol (UDP).=20 =20 Both CEs and FEs need to implement ForCES protocol. If some = layer-4=20 transport is used to support ForCES, then both CEs and FEs need = to=20 implement the L4 transport and ForCES protocols. It is possible = that=20 all FEs inside an NE implements only one such protocol entity. = =20 5.6. Application Layer -- Routing Protocols=20 =20 Both interior routing protocols and exterior routing protocols = are=20 implemented on CEs. The routing packets originated by CEs are=20 forwarded to FEs for delivery. The results of such protocols = (like=20 forwarding table update) are communicated to FEs via ForCES.=20 =20 For performance or scalability reasons, portions of the control = plane functions that need faster response may be moved from the = CEs=20 and off-loaded onto the FEs. For example in OSPF, the Hello = protocol=20 packets are generated and processed periodically. When done at = CEs,=20 the inbound Hello packets have to traverse from the external=20 interfaces at the FEs to the CEs via the internal CE-FE = channel.=20 Similarly, the outbound Hello packets have to go from the CEs = to the=20 FEs and to the external interfaces. Frequent Hello updates = place=20 heavy processing overhead on the CEs and can overwhelm the = CE-FE=20 channel as well. Since typically there are far more FEs than = CEs in=20 a router, when off-loaded, the Hello packets are processed in a = much=20 more distributed and scalable fashion. Interoperability is = ensured=20 by expressing such off-loaded functions in the FE model.=20 =20 5.7. Application Layer -- Network Management Protocol=20 =20 RFC1812 also dictates "Routers MUST be manageable by SNMP." = (see=20 [RFC1157] Section 8) In general, for post-association phase, = most=20 external management tasks (including SNMP) should be done = through=20 interaction with the CE in order to support the appearance of a = single functional device. Therefore, it is recommended that = SNMP=20 management agent be implemented by CEs and the SNMP messages=20 received by FEs be redirected to their CEs. AgentX framework = defined=20 in RFC2741 may be applied here such that CEs act in the role of = master agent to process SNMP protocol messages while FEs act in = the=20 role of subagent to provide access to the MIB objects residing = on=20 FEs. AgentX protocol messages between the master agent (CE) and = the=20 subagent (FE) are encapsulated and transported via ForCES, just = like=20 data packets from any other application layer protocols. =20 =20 6. Summary=20 =20 This document defines an architectural framework for ForCES. It = identifies the relevant components for a ForCES network = element,=20 including (one or more) FEs, (one or more) CEs, one optional FE = manager, and one optional CE manager. It also identifies the=20 =20 Yang, et. al. Expires April 2003 [Page 22] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 interaction among these components and discusses all the major=20 reference points. It is important to point out that, among all = the=20 reference points, only the interface between CEs and FEs is = within=20 the scope of ForCES. ForCES alone may not be enough to support = all=20 desirablet NE configurations. However, we believe that ForCES = is the=20 most important element in realizing physical separation and=20 interoperability of CEs and FEs, and hence the first interface = that=20 ought to be standardized. Simple and useful configurations can = still=20 be implemented with only CE-FE interface being standardized, = e.g.,=20 single CE with full-meshed FEs and static configuration without = the=20 need for CE/FE managers.=20 =20 7. Security Considerations=20 =20 The security necessary across each reference point except Fp is = discussed throughout the document. In general, the physical=20 separation of two entities usually requires much stricter = security=20 measurement in place. For example, we pointed out in Section = 5.1=20 that authentication becomes necessary between CE manager and FE = manager, between CE and CE manager, between FE and FE manager = in=20 some configuration. The physical separation of CE and FE also=20 imposes serious security requirement for ForCES protocol. The=20 security requirements for reference point Fp (i.e., ForCES = protocol)=20 are discussed in detail in [FORCES-REQ] Section 8.=20 =20 8. Intellectual Property Right=20 =20 The authors are not aware of any intellectual property right = issues=20 pertaining to this document.=20 =20 9. References=20 =20 9.1. Normative References=20 =20 [RFC1157] J. Case, et. al., "A Simple Network Management = Protocol=20 (SNMP)", RFC1157, May 1990.=20 =20 [RFC1812] F. Baker, "Requirements for IP Version 4 Routers",=20 RFC1812, June 1995.=20 =20 [RFC2741] M. Daniele, et. al., "Agent Extensibility (AgentX)=20 Protocol Version 1", RFC2741, Jan 2000.=20 =20 [RFC2914] S. Floyd, "Congestion Control Principles", RFC2914,=20 September 2000.=20 =20 [RFC3084] K. Chan, et. al., "COPS Usage for Policy Provisioning = (COPS-PR)", RFC3084, March 2001.=20 =20 9.2. Informative References=20 =20 =20 Yang, et. al. Expires April 2003 [Page 23] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 [FORCES-REQ] T. Anderson, et. al., "Requirements for Separation = of=20 IP Control and Forwarding", work in progress, July 2002, = .=20 =20 [FORCES-APP] A. Crouch, et. al., "ForCES Applicability = Statement",=20 work in progress, June 2002, .=20 =20 10. Acknowledgments=20 =20 Joel M. Halpern gave us many insightful comments and = suggestions and=20 pointed out several major issues. Many of our colleagues and = people=20 in the ForCES mailing list also provided valuable feedback.=20 =20 11. Authors' Addresses=20 =20 Lily L. Yang=20 Intel Corp., MS JF3-206,=20 2111 NE 25th Avenue=20 Hillsboro, OR 97124 USA=20 Phone: +1 503 264 8813=20 Email: lily.l.yang@intel.com=20 =20 Ram Dantu=20 University of Texas Dallas=20 2601 North Flyod Road=20 Richardson Texas 75082=20 Phone: +1 972 883 4653=20 Email: ram.dantu@utdallas.edu=20 =20 Todd A. Anderson=20 Intel Corp.=20 2111 NE 25th Avenue=20 Hillsboro, OR 97124 USA=20 Phone: +1 503 712 1760=20 Email: todd.a.anderson@intel.com=20 =20 =20 =20 Yang, et. al. Expires April 2003 [Page 24] = =0C ------_=_NextPart_000_01C276E2.C76C86A0-- From owner-forces@PEACH.EASE.LSOFT.COM Mon Oct 21 13:31:58 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA07258 for ; Mon, 21 Oct 2002 13:31:57 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <9.00784EC1@cherry.ease.lsoft.com>; Mon, 21 Oct 2002 13:34:11 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 29658411 for FORCES@PEACH.EASE.LSOFT.COM; Mon, 21 Oct 2002 13:34:11 -0400 Received: from 143.183.152.8 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Mon, 21 Oct 2002 13:34:05 -0400 Received: from fmsmsxvs042.fm.intel.com (fmsmsxvs042.fm.intel.com [132.233.42.128]) by momus.sc.intel.com (8.11.6/8.11.6/d: solo.mc,v 1.48 2002/10/16 23:47:34 dmccart Exp $) with SMTP id g9LHXeH00215 for ; Mon, 21 Oct 2002 17:33:50 GMT Received: from fmsmsx29.FM.INTEL.COM ([132.233.42.29]) by fmsmsxvs042.fm.intel.com (NAVGW 2.5.2.11) with SMTP id M2002102110353230948 ; Mon, 21 Oct 2002 10:35:32 -0700 Received: by fmsmsx29.fm.intel.com with Internet Mail Service (5.5.2653.19) id ; Mon, 21 Oct 2002 10:33:40 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain Approved-By: "Putzolu, David" Message-ID: Date: Mon, 21 Oct 2002 10:33:39 -0700 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: "Putzolu, David" Subject: ForCES @ IETF 55 - Call for Agenda Items Comments: cc: "Patrick Droz (dro@zurich.ibm.com)" To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list All, The ForCES working group will be meeting at IETF 55. The current draft IETF 55 agenda has listed ForCES for Monday, November 18, 0900-1130. Please send any requests for agenda time to this list, myself, or Patrick. Cheers, David (co-chair) From owner-forces@PEACH.EASE.LSOFT.COM Thu Oct 24 19:25:51 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA26883 for ; Thu, 24 Oct 2002 19:25:48 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <6.0078EC86@cherry.ease.lsoft.com>; Thu, 24 Oct 2002 19:28:05 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 30046560 for FORCES@PEACH.EASE.LSOFT.COM; Thu, 24 Oct 2002 19:28:04 -0400 Received: from 143.183.152.8 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Thu, 24 Oct 2002 19:28:04 -0400 Received: from fmsmsxvs042.fm.intel.com (fmsmsxvs042.fm.intel.com [132.233.42.128]) by momus.sc.intel.com (8.11.6/8.11.6/d: solo.mc,v 1.48 2002/10/16 23:47:34 dmccart Exp $) with SMTP id g9ONRc818632 for ; Thu, 24 Oct 2002 23:27:48 GMT Received: from FMSMSX018.fm.intel.com ([132.233.42.197]) by fmsmsxvs042.fm.intel.com (NAVGW 2.5.2.11) with SMTP id M2002102416272612130 for ; Thu, 24 Oct 2002 16:27:26 -0700 Received: by fmsmsx018.fm.intel.com with Internet Mail Service (5.5.2653.19) id ; Thu, 24 Oct 2002 16:27:22 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/mixed; boundary="----_=_NextPart_000_01C27BB4.E4526510" Approved-By: "Khosravi, Hormuzd M" Message-ID: Date: Thu, 24 Oct 2002 16:27:17 -0700 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: "Khosravi, Hormuzd M" Subject: Re: ForCES Requirements Status Update, 10/09/2002 To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_000_01C27BB4.E4526510 Content-Type: text/plain; charset="ISO-8859-1" Hi All I have attached the latest version of the Requirements draft which I submitted to the IETF after incorporating all the comments below. Thanks Hormuzd -----Original Message----- From: Putzolu, David [mailto:david.putzolu@INTEL.COM] Sent: Wednesday, October 09, 2002 5:16 PM To: FORCES@PEACH.EASE.LSOFT.COM Subject: [FORCES] ForCES Requirements Status Update, 10/09/2002 All, Our area directors have provided feedback on the requirements draft - see below for a summary. The requirements design team is going to respond to the feedback and post a new revision for the WG to review. This new rev will have a brief last call comment period (one week), and then will be returned to the IESG for review for publication. Cheers, David & Patrick, co-chairs, ForCES WG --- AD Feedback --- Section 6.5 Requirement 5) Vendor-Specific Functions The ADs believe that the IESG will be very hesitant to approve this part of the draft as worded. This is due to a concern that vendors would simply use the extensibility mechanisms in the protocol to do everything in a proprietary fashion, including "common" functions. The ADs suggested rewording the section to talk about extensibility to support new, currently unknown functions. Make sure that it is clear that one is *not* compliant with ForCES if one controls all the standard/common functions in a proprietary manner. Section 7 Requirement 6) Reliability The ADs where confused by the mention of non-IP networks (hey, they are IETF ADs, IP is all they think of - just kidding!!!). Recommend breaking up the one long reliability paragraph into three paragraphs: Para 1: Talks about IP networks. Essentially the same content. Para 2: Elaborates what non-IP networks are (FastEth, GigE, cell fabric between control and forwarding elements). Para 3: Take the last sentence and expand on it to make clear that the ability to acknowledge the *content* and execution of messages, rather than just their delivery. I.e., TCP may guarantee delivery, but it tells you nothing about whether the forwarding entry add operation succeeded or not. It is the latter function that is required for all types of networks (IP and non-IP). Section 7 Requirement 9) Packet Redirection The ADs where concerned that as written this seems more like a requirement on the FEs/CEs rather than on the protocol. Suggest rewriting it to say that ForCES must define a way to deliver packets from FE's that are destined to the CE (i.e. RIP packets from other routers) and vice versa. This makes it a protocol requirement instead of a CE or FE requirement. --- Other Feedback --- Other feedback based on http://www.ietf.org/internet-drafts/draft-rfc-editor-rfc2223bis-02.txt * Looks like 58 or more lines per page. Reduce this to 55 lines per page as per section 3.1 item 3. * Table of contents should go directly after abstract per section 4. * References should appear before security considerations per section 4 * Make normative and informative references subsections rather than top level sections. --- end --- ------_=_NextPart_000_01C27BB4.E4526510 Content-Type: text/plain; name="draft-ietf-forces-requirements-07.txt" Content-Disposition: attachment; filename="draft-ietf-forces-requirements-07.txt" Content-Transfer-Encoding: quoted-printable Internet Draft H. Khosravi,=20 Expiration: April 2003 T. Anderson (Editors)=20 File: draft-ietf-forces-requirements-07.txt Intel=20 Working Group: ForCES October 2002=20 =20 =20 =20 =20 Requirements for Separation of IP Control and Forwarding=20 =20 =20 =20 draft-ietf-forces-requirements-07.txt=20 =20 =20 =20 =20 Status of this Memo=20 =20 This document is an Internet-Draft and is in full conformance with =20 all provisions of Section 10 of RFC2026. Internet-Drafts are =20 working documents of the Internet Engineering Task Force (IETF), =20 its areas, and its working groups. Note that other groups may =20 also distribute working documents as Internet-Drafts.=20 =20 Internet-Drafts are draft documents valid for a maximum of six =20 months and may be updated, replaced, or obsoleted by other =20 documents at any time. It is inappropriate to use Internet-Drafts =20 as reference material or to cite them other than as ``work in =20 progress.''=20 =20 The list of current Internet-Drafts can be accessed at=20 http://www.ietf.org/ietf/1id-abstracts.txt.=20 =20 The list of Internet-Draft Shadow Directories can be accessed at =20 http://www.ietf.org/shadow.html.=20 =20 Conventions used in this document =20 =20 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", = "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in =20 this document are to be interpreted as described in [RFC-2119].=20 =20 1. Abstract=20 =20 This document introduces the ForCES architecture and defines a set=20 of associated terminology. This document also defines a set of=20 architectural, modeling, and protocol requirements to logically=20 separate the control and data forwarding planes of an IP (IPv4,=20 IPv6, etc.) networking device.=20 =20 Khosravi, Anderson, et. al. Expires April 2003 [Page 1]=20 Internet Draft ForCES Requirements October 2002=20 =20 1. Abstract........................................................1 = 2. Definitions.....................................................2 = 3. Introduction....................................................4 = 4. Architecture....................................................5 = 5. Architectural Requirements......................................6 = 6. FE Model Requirements...........................................8 = 6.1. Types of Logical Functions......................................8 = 6.2. Variations of Logical Functions.................................8 = 6.3. Ordering of Logical Functions...................................8 = 6.4. Flexibility.....................................................9 = 6.5. Minimal Set of Logical Functions................................9 = 7. ForCES Protocol Requirements...................................10 = 8. References.....................................................13 = 8.1. Normative References...........................................13 = 8.2. Informative References.........................................13 = 9. Security Considerations........................................13 = 10. Authors' Addresses & Acknowledgments..........................13 = 11. Editors' Contact Information..................................15 = =20 =20 2. Definitions=20 =20 Addressable Entity (AE) - A physical device that is directly=20 addressable given some interconnect technology. For example, on=20 Ethernet, an AE is a device to which we can communicate using an=20 Ethernet MAC address; on IP networks, it is a device to which we can = communicate using an IP address; and on a switch fabric, it is a=20 device to which we can communicate using a switch fabric port=20 number.=20 =20 Physical Forwarding Element (PFE) - An AE that includes hardware=20 used to provide per-packet processing and handling. This hardware=20 may consist of (but is not limited to) network processors, ASIC's,=20 line cards with multiple chips or stand alone box with general- purpose processors. =20 =20 PFE Partition - A logical partition of a PFE consisting of some=20 subset of each of the resources (e.g., ports, memory, forwarding=20 table entries) available on the PFE. This concept is analogous to=20 that of the resources assigned to a virtual switching element as=20 described in [REQ-PART].=20 =20 Physical Control Element (PCE) - An AE that includes hardware used=20 to provide control functionality. This hardware typically includes=20 a general-purpose processor.=20 =20 PCE Partition - A logical partition of a PCE consisting of some=20 subset of each of the resources available on the PCE.=20 =20 =20 Khosravi, Anderson, et. al. Expires April 2003 [Page 2]=20 Internet Draft ForCES Requirements October 2002=20 Forwarding Element (FE) - A logical entity that implements the=20 ForCES protocol. FEs use the underlying hardware to provide per- packet processing and handling as directed by a CE via the ForCES=20 protocol. FEs may use PFE partitions, whole PFEs, or multiple PFEs. = =20 Proxy FE - A name for a type of FE that cannot directly modify its=20 underlying hardware but instead manipulates that hardware using some = intermediate form of communication (e.g., a non-ForCES protocol or=20 DMA). A proxy FE will typically be used in the case where a PFE=20 cannot implement the ForCES protocol directly (e.g., due to the lack = of a general purpose CPU).=20 =20 Control Element (CE) - A logical entity that implements the ForCES=20 protocol and uses it to instruct one or more FEs how to process=20 packets. CEs handle functionality such as the execution of control=20 and signaling protocols. CEs may consist of PCE partitions or whole = PCEs.=20 =20 Pre-association Phase - The period of time during which a FE Manager = (see below) and a CE Manager (see below) are determining which FE=20 and CE should be part of the same network element. Any partitioning=20 of PFEs and PCEs occurs during this phase.=20 =20 Post-association Phase - The period of time during which a FE does=20 know which CE is to control it and vice versa, including the time=20 during which the CE and FE are establishing communication with one=20 another.=20 =20 ForCES Protocol - While there may be multiple protocols used within=20 the overall ForCES architecture, the term "ForCES protocol" refers=20 only to the ForCES post-association phase protocol (see below).=20 =20 ForCES Post-Association Phase Protocol - The protocol used for post- association phase communication between CEs and FEs. This protocol=20 does not apply to CE-to-CE communication, FE-to-FE communication, or = to communication between FE and CE managers. The ForCES protocol is = a master-slave protocol in which FEs are slaves and CEs are masters. = =20 This protocol includes both the management of the communication=20 channel (e.g., connection establishment, heartbeats) and the control = messages themselves. This protocol could be a single protocol or=20 could consist of multiple protocols working together.=20 =20 FE Model - A model that describes the logical processing functions=20 of a FE.=20 =20 FE Manager - A logical entity that operates in the pre-association=20 phase and is responsible for determining to which CE(s) a FE should=20 communicate. This process is called CE discovery and may involve=20 the FE manager learning the capabilities of available CEs. A FE=20 =20 Khosravi, Anderson, et. al. Expires April 2003 [Page 3]=20 Internet Draft ForCES Requirements October 2002=20 manager may use anything from a static configuration to a pre- association phase protocol (see below) to determine which CE(s) to=20 use. Being a logical entity, a FE manager might be physically=20 combined with any of the other logical entities mentioned in this=20 section.=20 =20 CE Manager - A logical entity that operates in the pre-association=20 phase and is responsible for determining to which FE(s) a CE should=20 communicate. This process is called FE discovery and may involve=20 the CE manager learning the capabilities of available FEs. A CE=20 manager may use anything from a static configuration to a pre- association phase protocol (see below) to determine which FE to use. = =20 Being a logical entity, a CE manager might be physically combined=20 with any of the other logical entities mentioned in this section.=20 =20 Pre-association Phase Protocol - A protocol between FE managers and=20 CE managers that is used to determine which CEs or FEs to use. A=20 pre-association phase protocol may include a CE and/or FE capability = discovery mechanism. Note that this capability discovery process is = wholly separate from (and does not replace) that used within the=20 ForCES protocol (see Section 7, requirement #1). However, the two=20 capability discovery mechanisms may utilize the same FE model (see=20 Section 6). Pre-association phase protocols are not discussed=20 further in this document.=20 =20 ForCES Network Element (NE) - An entity composed of one or more CEs=20 and one or more FEs. To entities outside a NE, the NE represents a=20 single point of management. Similarly, a NE usually hides its=20 internal organization from external entities.=20 =20 ForCES Protocol Element - A FE or CE.=20 =20 High Touch Capability - This term will be used to apply to the=20 capabilities found in some forwarders to take action on the contents = or headers of a packet based on content other than what is found in=20 the IP header. Examples of these capabilities include NAT-PT,=20 firewall, and L7 content recognition.=20 =20 3. Introduction=20 =20 An IP network element is composed of numerous logically separate=20 entities that cooperate to provide a given functionality (such as a=20 routing or IP switching) and yet appear as a normal integrated=20 network element to external entities. Two primary types of network=20 element components exist: control-plane components and forwarding- plane components. In general, forwarding-plane components are ASIC, = network-processor, or general-purpose processor-based devices that=20 handle all data path operations. Conversely, control-plane=20 components are typically based on general-purpose processors that=20 =20 Khosravi, Anderson, et. al. Expires April 2003 [Page 4]=20 Internet Draft ForCES Requirements October 2002=20 provide control functionality such as the processing of routing or=20 signaling protocols. A standard set of mechanisms for connecting=20 these components provides increased scalability and allows the=20 control and forwarding planes to evolve independently, thus=20 promoting faster innovation.=20 =20 For the purpose of illustration, let us consider the architecture of = a router to illustrate the concept of separate control and=20 forwarding planes. The architecture of a router is composed of two=20 main parts. These components, while inter-related, perform=20 functions that are largely independent of each other. At the bottom = is the forwarding path that operates in the data-forwarding plane=20 and is responsible for per-packet processing and forwarding. Above=20 the forwarding plane is the network operating system that is=20 responsible for operations in the control plane. In the case of a=20 router or switch, the network operating system runs routing,=20 signaling and control protocols (e.g., RIP, OSPF and RSVP) and=20 dictates the forwarding behavior by manipulating forwarding tables,=20 per-flow QoS tables and access control lists. Typically, the=20 architecture of these devices combines all of this functionality=20 into a single functional whole with respect to external entities.=20 =20 4. Architecture=20 =20 The chief components of a NE architecture are the CE, the FE, and=20 the interconnect protocol. The CE is responsible for operations=20 such as signaling and control protocol processing and the=20 implementation of management protocols. Based on the information=20 acquired through control processing, the CE(s) dictates the packet- forwarding behavior of the FE(s) via the interconnect protocol. For = example, the CE might control a FE by manipulating its forwarding=20 tables, the state of its interfaces, or by adding or removing a NAT=20 binding.=20 =20 The FE operates in the forwarding plane and is responsible for per- packet processing and handling. By allowing the control and=20 forwarding planes to evolve independently, different types of FEs=20 can be developed - some general purpose and others more specialized. = =20 Some functions that FEs could perform include layer 3 forwarding,=20 metering, shaping, firewall, NAT, encapsulation (e.g., tunneling),=20 decapsulation, encryption, accounting, etc. Nearly all combinations = of these functions may be present in practical FEs.=20 =20 Below is a diagram illustrating an example NE composed of a CE and=20 two FEs. Both FEs and CE require minimal configuration as part of=20 the pre-configuration process and this may be done by FE Manager and = CE Manager respectively. Apart from this, there is no defined role=20 for FE Manager and CE Manager. The Proxy FE has also been defined=20 for clarification purpose. These components are out of scope of the=20 =20 Khosravi, Anderson, et. al. Expires April 2003 [Page 5]=20 Internet Draft ForCES Requirements October 2002=20 architecture and requirements for the ForCES protocol, which only=20 involves CEs and FEs.=20 =20 --------------------------------=20 | NE |=20 | ------------- |=20 | | CE | |=20 | ------------- |=20 | / \ |=20 | / \ |=20 | / \ |=20 | / \ |=20 | ----------- ----------- |=20 | | FE | | FE | |=20 | ----------- ----------- |=20 | | | | | | | | | |=20 | | | | | | | | | |=20 | | | | | | | | | |=20 | | | | | | | | | |=20 --------------------------------=20 | | | | | | | |=20 | | | | | | | |=20 =20 5. Architectural Requirements=20 =20 The following are the architectural requirements:=20 =20 1) CEs and FEs MUST be able to connect by a variety of interconnect=20 technologies. Examples of interconnect technologies used in current = architectures include Ethernet,bus backplanes, and ATM (cell)=20 fabrics. FEs MAY be connected to each other via a different=20 technology than that used for CE/FE communication.=20 =20 2) FEs MUST support a minimal set of capabilities necessary for=20 establishing network connectivity (e.g., interface discovery, port=20 up/down functions). Beyond this minimal set, the ForCES=20 architecture MUST NOT restrict the types or numbers of capabilities=20 that FEs may contain.=20 =20 3) Packets MUST be able to arrive at the NE by one FE and leave the=20 NE via a different FE.=20 =20 4) A NE MUST support the appearance of a single functional device. =20 For example, in a router, the TTL of the packet should be=20 decremented only once as it traverses the NE regardless of how many=20 FEs through which it passes. However, external entities (e.g., FE=20 managers and CE managers) MAY have direct access to individual=20 ForCES protocol elements for providing information to transition=20 them from the pre-association to post-association phase.=20 =20 Khosravi, Anderson, et. al. Expires April 2003 [Page 6]=20 Internet Draft ForCES Requirements October 2002=20 =20 5) The architecture MUST provide a way to prevent unauthorized=20 ForCES protocol elements from joining a NE.(For more protocol=20 details, refer to section 7 requirement# 2)=20 =20 6) A FE MUST be able to asynchronously inform the CE of a failure or = increase/decrease in available resources or capabilities on the FE.=20 Thus the FE MUST support error monitoring and reporting. (Since=20 there is not a strict 1-to-1 mapping between FEs and PFEs, it is=20 possible for the relationship between a FE and its physical=20 resources to change over time). For example, the number of physical = ports or the amount of memory allocated to a FE may vary over time. = The CE needs to be informed of such changes so that it can control=20 the FE in an accurate way.=20 =20 7) The architecture MUST support mechanisms for CE redundancy or CE=20 failover. This includes the ability for CEs and FEs to determine=20 when there is a loss of association between them, ability to restore = association and efficient state (re)synchronization mechanisms. This = also includes the ability to preset the actions an FE will take in=20 reaction to loss of association to its CE e.g., whether the FE will=20 continue to forward packets or whether it will halt operations.=20 =20 8) FEs MUST be able to redirect control packets (such as RIP, OSPF=20 messages) addressed to their interfaces to the CE. They MUST also=20 redirect other relevant packets (e.g., such as those with Router=20 Alert Option set) to their CE. The CEs MUST be able to configure the = packet redirection information/filters on the FEs. The CEs MUST also = be able to create packets and have its FEs deliver them.=20 =20 9) Any proposed ForCES architectures MUST explain how that=20 architecture supports all of the router functions as defined in=20 [RFC1812].=20 =20 10) In a ForCES NE, the FEs MUST be able to provide their topology=20 information (topology by which the FEs in the NE are connected) to=20 the CE(s).=20 =20 11) The ForCES NE architecture MUST be capable of supporting (i.e.,=20 must scale to) at least hundreds of FEs and tens of thousands of=20 ports.=20 =20 12) The ForCES architecture MUST allow FEs AND CEs to join and leave = NEs dynamically.=20 =20 13) The ForCES NE architecture MUST support multiple CEs and FEs.=20 =20 14) For pre-association phase setup, monitoring, configuration=20 issues, it MAY be useful to use standard management mechanisms for=20 =20 Khosravi, Anderson, et. al. Expires April 2003 [Page 7]=20 Internet Draft ForCES Requirements October 2002=20 CEs and FEs. The ForCES architecture and requirements do not=20 preclude this. In general, for post-association phase, most=20 management tasks SHOULD be done through interaction with the CE. In=20 certain conditions (e.g. CE/FE disconnection), it may be useful to=20 allow management tools (e.g. SNMP) to be used to diagnose and repair = problems. The following guidelines MUST be observed: =20 1. The ability for a management tool (e.g. SNMP) to be used to read=20 (but not change) the state of FE SHOULD NOT be precluded.=20 2. It MUST NOT be possible for management tools (e.g. SNMP, etc) to=20 change the state of a FE in a manner that affects overall NE=20 behavior without the CE being notified.=20 =20 6. FE Model Requirements=20 =20 The variety of FE functionality that the ForCES architecture allows=20 poses a potential problem for CEs. In order for a CE to effectively = control a FE, the CE must understand how the FE processes packets. =20 We therefore REQUIRE that a FE model be created that can express the = logical packet processing capabilities of a FE. This model will be=20 used in the ForCES protocol to describe FE capabilities (see Section = 7, requirement #1). The FE model MUST define both a capability model = and a state model, which expresses the current configuration of the=20 device. The FE model MUST also support multiple FEs in the NE=20 architecture.=20 =20 6.1. Types of Logical Functions=20 =20 The FE model MUST express what logical functions can be applied to=20 packets as they pass through a FE. =20 Logical functions are the packet processing functions that are=20 applied to the packets as they are forwarded through a FE. Examples=20 of logical functions are layer 3 forwarding, firewall, NAT, shaping. = Section 6.5 defines the minimal set of logical functions that the FE = Model MUST support.=20 =20 6.2. Variations of Logical Functions=20 The FE model MUST be capable of supporting/allowing variations in=20 the way logical functions are implemented on a FE. For example, on a = certain FE the forwarding logical function might have information=20 about both the next hop IP address and the next hop MAC address,=20 while on another FE these might be implemented as separate logical=20 functions. Another example would be NAT functionality that can have=20 several flavors such as Traditional/Outbound NAT, Bi-directional=20 NAT, Twice NAT, Multihomed NAT [RFC2663]. The model must be flexible = enough to allow such variations in functions.=20 =20 6.3. Ordering of Logical Functions=20 The model MUST be capable of describing the order in which these=20 logical functions are applied in a FE. The ordering of logical=20 =20 Khosravi, Anderson, et. al. Expires April 2003 [Page 8]=20 Internet Draft ForCES Requirements October 2002=20 functions is important in many cases. For example, a NAT function=20 may change a packet's source or destination IP address. Any number=20 of other logical functions (e.g., layer 3 forwarding, ingress/egress = firewall, shaping, accounting) may make use of the source or=20 destination IP address when making decisions. The CE needs to know=20 whether to configure these logical functions with the pre-NAT or=20 post-NAT IP address. Furthermore, the model MUST be capable of=20 expressing multiple instances of the same logical function in a FE's = processing path. Using NAT again as an example, one NAT function is = typically performed before the forwarding decision (packets arriving = externally have their public addresses replaced with private=20 addresses) and one NAT function is performed after the forwarding=20 decision (for packets exiting the domain, their private addresses=20 are replaced by public ones).=20 =20 6.4. Flexibility=20 =20 Finally, the FE model SHOULD provide a flexible infrastructure in=20 which new logical functions and new classification, action, and=20 parameterization data can be easily added. In addition, the FE=20 model MUST be capable of describing the types of statistics gathered = by each logical function. =20 =20 6.5. Minimal Set of Logical Functions=20 =20 The rest of this section defines a minimal set of logical functions=20 that any FE model MUST support. This minimal set DOES NOT imply=20 that all FEs must provide this functionality. Instead, these=20 requirements only specify that the model must be capable of=20 expressing the capabilities that FEs may choose to provide.=20 =20 1)Port Functions=20 The FE model MUST be capable of expressing the number of ports on=20 the device, the static attributes of each port (e.g., port type,=20 link speed), and the configurable attributes of each port (e.g., IP=20 address, administrative status).=20 =20 2)Forwarding Functions=20 The FE model MUST be capable of expressing the data that can be used = by the forwarding function to make a forwarding decision. Support=20 for IPv4 and IPv6 unicast and multicast forwarding functions MUST be = provided by the model.=20 =20 3)QoS Functions=20 The FE model MUST allow a FE to express its QoS capabilities in=20 terms of, e.g., metering, policing, shaping, and queuing functions. = The FE model MUST be capable of expressing the use of these=20 functions to provide IntServ or DiffServ functionality as described=20 in [RFC2211], [RFC2212], [RFC2215], [RFC2475], and [RFC3290].=20 =20 Khosravi, Anderson, et. al. Expires April 2003 [Page 9]=20 Internet Draft ForCES Requirements October 2002=20 =20 4)Generic Filtering Functions=20 The FE model MUST be capable of expressing complex sets of filtering = functions. The model MUST be able to express the existence of these = functions at arbitrary points in the sequence of a FE's packet=20 processing functions. The FE model MUST be capable of expressing a=20 wide range of classification abilities from single fields (e.g.,=20 destination address) to arbitrary n-tuples. Similarly, the FE model = MUST be capable of expressing what actions these filtering functions = can perform on packets that the classifier matches.=20 =20 5)Vendor-Specific Functions=20 The FE model SHOULD be extensible so that new, currently unknown FE = functionality can be expressed. The FE Model SHOULD NOT be extended=20 to express standard/common functions in a proprietary manner. This=20 would NOT be ForCES compliant.=20 =20 6)High-Touch Functions=20 The FE model MUST be capable of expressing the encapsulation and=20 tunneling capabilities of a FE. The FE model MUST support functions=20 that mark the class of service that a packet should receive (i.e.=20 IPv4 header TOS octet or the IPv6 Traffic Class octet). The FE=20 model MAY support other high touch functions (e.g., NAT, ALG).=20 =20 7)Security Functions=20 The FE model MUST be capable of expressing the types of encryption=20 that may be applied to packets in the forwarding path.=20 =20 8)Off-loaded Functions=20 Per-packet processing can leave state in the FE, so that logical=20 functions executed during packet processing can perform in a=20 consistent manner (for instance, each packet may update the state of = the token bucket occupancy of a give policer). In addition, FEs MUST = allow logical functions to execute asynchronously from packet=20 processing, according to a certain finite-state machine, in order to = perform functions that are, for instance, off-loaded from the CE to=20 the FE. The FE model MUST be capable of expressing these=20 asynchronous functions. Examples of such functions include the=20 finite-state machine execution required by TCP termination or OSPF=20 Hello processing, triggered not only by packet events, but by timer=20 events as well.=20 =20 7. ForCES Protocol Requirements=20 =20 This section specifies some of the requirements that the ForCES=20 protocol MUST meet.=20 =20 1)Configuration of Modeled Elements=20 =20 Khosravi, Anderson, et. al. Expires April 2003 [Page 10]=20 Internet Draft ForCES Requirements October 2002=20 The ForCES protocol MUST allow the CEs to determine the capabilities = of each FE. These capabilities SHALL be expressed using the FE=20 model whose requirements are defined in Section 6. Furthermore, the = protocol MUST provide a means for the CEs to control all the FE=20 capabilities that are discovered through the FE model. The protocol=20 MUST be able to add/remove classification/action entries, set/delete = parameters, query statistics, and register for and receive events.=20 =20 2)Support for Secure Communication=20 a) FE configuration will contain information critical to the=20 functioning of a network (e.g. IP Forwarding Tables). As such, it = MUST be possible to ensure the authenticity and integrity of=20 ForCES protocol messages.=20 b) FE configuration information may also contain information derived = from business relationships (e.g. service level agreements).=20 Therefore, it MUST be possible to secure (make private) ForCES=20 protocol messages.=20 c) In order to provide authentication, integrity and privacy for a=20 proposed ForCES protocol, existing security mechanisms such as=20 IPSec/IKE, TLS, etc. MUST be used/leveraged if applicable.=20 =20 3)Scalability=20 The ForCES protocol MUST be capable of supporting (i.e., must scale=20 to) at least hundreds of FEs and tens of thousands of ports. For=20 example, the ForCES protocol field sizes corresponding to FE or port = numbers SHALL be large enough to support the minimum required=20 numbers. This requirement does not relate to the performance of a=20 NE as the number of FEs or ports in the NE grows.=20 =20 4)Multihop=20 When the CEs and FEs are separated beyond a single hop, the ForCES=20 protocol will make use of an existing RFC2914 compliant L4 protocol=20 with adequate reliability, security and congestion control (e.g.=20 TCP, SCTP) for transport purposes.=20 =20 5)Message Priority=20 The ForCES protocol MUST provide a means to express the protocol=20 message priorities.=20 =20 6)Reliability=20 The ForCES protocol SHALL assume that it runs on top of an=20 unreliable, datagram service. For IP networks, an encapsulation of=20 the ForCES protocol SHALL be defined that uses a [RFC2914]-compliant = transport protocol and provides a datagram service (that could be=20 unreliable).=20 For non-IP networks such as FastEth, GigE, cell fabric between=20 control and forwarding elements, additional encapsulations MAY be=20 defined so long as they provide a datagram service to the ForCES=20 protocol. =20 =20 Khosravi, Anderson, et. al. Expires April 2003 [Page 11]=20 Internet Draft ForCES Requirements October 2002=20 However, reliability in the ForCES context means the ability to=20 acknowledge the content and execution of messages, rather than just=20 their delivery. For this reason, the ForCES protocol MUST provide=20 internal support for reliability mechanisms such as message=20 acknowledgements and/or state change confirmations. This is required = for both IP and non-IP networks.=20 =20 7)Interconnect Independence=20 The ForCES protocol MUST support a variety of interconnect=20 technologies. (refer to section 5, requirement# 1)=20 =20 8)CE redundancy or CE failover=20 The ForCES protocol MUST support mechanisms for CE redundancy or CE=20 failover. This includes the ability for CEs and FEs to determine=20 when there is a loss of association between them, ability to restore = association and efficient state (re)synchronization mechanisms. This = also includes the ability to preset the actions an FE will take in=20 reaction to loss of association to its CE e.g., whether the FE will=20 continue to forward packets or whether it will halt operations.=20 (refer to section 5, requirement# 7)=20 =20 9)Packet Redirection=20 The ForCES protocol MUST define a way for FEs to be able to redirect = control packets (such as RIP, OSPF messages) addressed to their=20 interfaces to the CE and vice-versa. It MUST also allow redirection=20 of other relevant packets (e.g., such as those with Router Alert=20 Option set) to the CE. The ForCES protocol MUST also allow CEs to=20 configure the packet redirection information/filters on the FEs.=20 (refer to section 5, requirement# 8)=20 =20 10)Topology Exchange=20 The ForCES protocol MUST allow the FEs to provide their topology=20 information (topology by which the FEs in the NE are connected) to=20 the CE(s). (refer to section 5, requirement# 10)=20 =20 11)Dynamic Association=20 The ForCES protocol MUST allow CEs and FEs to join and leave a NE=20 dynamically. (refer to section 5, requirement# 12)=20 =20 12)Command Bundling=20 The ForCES protocol MUST be able to group an ordered set of commands = to a FE. Each such group of commands SHOULD be sent to the FE in as=20 few messages as possible. Furthermore, the protocol MUST support the = ability to specify if a command group MUST have all-or-nothing=20 semantics.=20 =20 13)Asynchronous Event Notification=20 =20 Khosravi, Anderson, et. al. Expires April 2003 [Page 12]=20 Internet Draft ForCES Requirements October 2002=20 The ForCES protocol MUST be able to asynchronously notify the CE of=20 events on the FE such as failures or change in available resources=20 or capabilities. (refer to section 5, requirement# 6)=20 =20 14)Query Statistics=20 The ForCES protocol MUST provide a means for the CE to be able to=20 query statistics (monitor performance) from the FE.=20 =20 8. References=20 8.1.Normative References=20 =20 [RFC3290] Y. Bernet, et. al., "An Informal Management Model for=20 DiffServ Routers", , May 2002.=20 =20 [RFC1812] F. Baker, "Requirements for IP Version 4 Routers",=20 RFC1812, June 1995.=20 =20 [RFC2211] J. Wroclawski, "Specification of the Controlled-Load=20 Network Element Service", RFC2211, September 1997.=20 =20 [RFC2212] S. Shenker, C. Partridge, R. Guerin, "Specification of=20 Guaranteed Quality of Service", RFC2212, September 1997.=20 =20 [RFC2215] S. Shenker, J. Wroclawski, "General Characterization=20 Parameters for Integrated Service Network Elements", RFC2215,=20 September 1997.=20 =20 [RFC2475] S. Blake, et. Al., "An Architecture for Differentiated=20 Service", RFC2475, December 1998.=20 =20 [RFC2914] S. Floyd, "Congestion Control Principles", RFC2914,=20 September 2000.=20 =20 [RFC2663] P. Srisuresh & M. Holdrege, "IP Network Address=20 Translator (NAT) Terminology and Considerations", RFC2663, August=20 1999.=20 =20 8.2.Informative References=20 =20 [REQ-PART] T. Anderson, J. Buerkle, "Requirements for the Dynamic=20 Partitioning of Switching Elements", work in progress, July 2002,=20 .=20 =20 9. Security Considerations=20 =20 See architecture requirement #5 and protocol requirement #2.=20 =20 =20 10. Authors' Addresses & Acknowledgments=20 =20 Khosravi, Anderson, et. al. Expires April 2003 [Page 13]=20 Internet Draft ForCES Requirements October 2002=20 =20 This document was written by the ForCES Requirements design team:=20 =20 Todd A. Anderson (Editor)=20 =20 Ed Bowen=20 IBM Zurich Research Laboratory=20 Saumerstrasse 4=20 CH-8803 Rueschlikon Switzerland=20 Phone: +41 1 724 83 68=20 Email: edbowen@us.ibm.com=20 =20 Ram Dantu=20 Netrake Corporation=20 3000 Technology Drive, #100,=20 Plano, Texas, 75074=20 Email: rdantu@netrake.com=20 Phone: 214 291 1111=20 =20 Avri Doria=20 Institute for System Technology=20 Lulea University of Technology=20 SE-971 87, Lulea, Sweden=20 Phone: +46 (0)920 49 3030=20 Email: avri@sm.luth.se=20 =20 Ram Gopal=20 Nokia Research Center=20 5, Wayside Road,=20 Burlington, MA 01803=20 Phone: 1-781-993-3685=20 Email: ram.gopal@nokia.com=20 =20 Jamal Hadi Salim=20 Znyx Networks=20 Ottawa, Ontario=20 Canada=20 Email: hadi@znyx.com=20 =20 Hormuzd Khosravi (Editor)=20 =20 Muneyb Minhazuddin=20 Avaya Inc.=20 123, Epping road,=20 North Ryde, NSW 2113, Australia=20 Phone: +61 2 9352 8620=20 email: muneyb@avaya.com=20 =20 Margaret Wasserman=20 =20 Khosravi, Anderson, et. al. Expires April 2003 [Page 14]=20 Internet Draft ForCES Requirements October 2002=20 Wind River=20 10 Tara Blvd., Suite 330=20 Nashua, NH 03062=20 Phone: +1 603 897 2067=20 Email: mrw@windriver.com=20 =20 The authors would like to thank Vip Sharma and Lily Yang for their=20 valuable contributions.=20 =20 11. Editors' Contact Information=20 =20 Hormuzd Khosravi=20 Intel=20 2111 NE 25th Avenue=20 Hillsboro, OR 97124 USA=20 Phone: +1 503 264 0334=20 Email: hormuzd.m.khosravi@intel.com=20 =20 Todd A. Anderson=20 Intel=20 2111 NE 25th Avenue=20 Hillsboro, OR 97124 USA=20 Phone: +1 503 712 1760=20 Email: todd.a.anderson@intel.com=20 =20 =20 =20 Khosravi, Anderson, et. al. Expires April 2003 [Page 15]=20 ------_=_NextPart_000_01C27BB4.E4526510-- From owner-forces@PEACH.EASE.LSOFT.COM Fri Oct 25 17:16:50 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA12336 for ; Fri, 25 Oct 2002 17:16:44 -0400 (EDT) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <11.00791643@cherry.ease.lsoft.com>; Fri, 25 Oct 2002 17:18:59 -0400 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 30133795 for FORCES@PEACH.EASE.LSOFT.COM; Fri, 25 Oct 2002 17:18:58 -0400 Received: from 143.183.152.8 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Fri, 25 Oct 2002 17:18:37 -0400 Received: from fmsmsxvs042.fm.intel.com (fmsmsxvs042.fm.intel.com [132.233.42.128]) by momus.sc.intel.com (8.11.6/8.11.6/d: solo.mc,v 1.48 2002/10/16 23:47:34 dmccart Exp $) with SMTP id g9PLI9j17105 for ; Fri, 25 Oct 2002 21:18:20 GMT Received: from fmsmsx29.FM.INTEL.COM ([132.233.42.29]) by fmsmsxvs042.fm.intel.com (NAVGW 2.5.2.11) with SMTP id M2002102514180006221 for ; Fri, 25 Oct 2002 14:18:00 -0700 Received: by fmsmsx29.fm.intel.com with Internet Mail Service (5.5.2653.19) id ; Fri, 25 Oct 2002 14:17:54 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/mixed; boundary="----_=_NextPart_000_01C27C6B.F9B44100" Approved-By: "Yang, Lily L" Message-ID: Date: Fri, 25 Oct 2002 14:17:51 -0700 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: "Yang, Lily L" Subject: framework draft v03 submitted To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_000_01C27C6B.F9B44100 Content-Type: text/plain; charset="iso-8859-1" Hi, all I am posting this version (03) of the framework draft to IETF. On behalf of all co-authors, I would like to request the work group chairs last call on it. Best regards, Lily <> ------_=_NextPart_000_01C27C6B.F9B44100 Content-Type: text/plain; name="draft-ietf-forces-framework-03.txt" Content-Disposition: attachment; filename="draft-ietf-forces-framework-03.txt" Content-Transfer-Encoding: quoted-printable Internet Draft L. Yang=20 Expiration: April 2003 Intel Corp.=20 File: draft-ietf-forces-framework-03.txt R. Dantu=20 Working Group: ForCES Univ. of Texas = Dallas=20 T. Anderson=20 Intel Corp. =20 Oct 2002 = =20 =20 =20 ForCES Architectural Framework=20 =20 =20 =20 draft-ietf-forces-framework-03.txt=20 =20 =20 =20 =20 Status of this Memo=20 =20 This document is an Internet-Draft and is in full conformance = with=20 all provisions of Section 10 of RFC2026. Internet-Drafts are=20 working documents of the Internet Engineering Task Force = (IETF), its=20 areas, and its working groups. Note that other groups may also = distribute working documents as Internet-Drafts.=20 =20 Internet-Drafts are draft documents valid for a maximum of six=20 months and may be updated, replaced, or obsoleted by other = documents=20 at any time. It is inappropriate to use Internet-Drafts as=20 reference material or to cite them other than as ``work in=20 progress.''=20 =20 The list of current Internet-Drafts can be accessed at=20 http://www.ietf.org/ietf/1id-abstracts.txt.=20 =20 The list of Internet-Draft Shadow Directories can be accessed = at =20 http://www.ietf.org/shadow.html.=20 =20 Abstract=20 =20 This document defines the architectural framework for the = ForCES=20 (Forwarding and Control Element Separation) network elements = (NE),=20 and identifies the associated entities and the interaction = among=20 them. This framework is intended to satisfy the requirements=20 specified in the ForCES requirements draft [FORCES-REQ].=20 =20 Table of Contents=20 =20 1. = Definitions.....................................................2=20 2. Introduction to Forwarding and Control Element Separation=20 = (ForCES)...........................................................4=20 3. = Architecture....................................................7=20 3.1. Control Elements and Fr Reference = Point....................8=20 =20 =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 3.2. Forwarding Elements and Fi reference = point.................9=20 3.3. CE = Managers...............................................11=20 3.4. FE = Managers...............................................11=20 4. Operational = Phases.............................................12=20 4.1. Pre-association = Phase.....................................12=20 4.1.1. Fl Reference = Point...................................12=20 4.1.2. Ff Reference = Point...................................13=20 4.1.3. Fc Reference = Point...................................13=20 4.2. Post-association Phase and Fp reference = point.............14=20 4.2.1. Proximity and Interconnect between CEs and = FEs.......14=20 4.2.2. Association = Establishment............................14=20 4.2.3. Steady-state = Communication...........................15=20 4.2.4. Data Packets across Fp reference = point...............16=20 4.2.5. Proxy = FE.............................................17=20 4.3. Association = Re-establishment..............................17=20 5. Applicability to = RFC1812.......................................18=20 5.1. General Router = Requirements...............................19=20 5.2. Link = Layer................................................20=20 5.3. Internet Layer = Protocols..................................20=20 5.4. Internet Layer = Forwarding.................................21=20 5.5. Transport = Layer...........................................21=20 5.6. Application Layer -- Routing = Protocols....................22=20 5.7. Application Layer -- Network Management = Protocol..........22=20 6. = Summary........................................................22=20 7. Security = Considerations........................................23=20 8. Intellectual Property = Right....................................23=20 9. = References.....................................................23=20 9.1. Normative = References......................................23=20 9.2. Informative = References....................................23=20 10. = Acknowledgments...............................................24=20 11. Authors' = Addresses............................................24=20 =20 Conventions used in this document =20 =20 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL = NOT", =20 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" = in=20 this document are to be interpreted as described in [RFC-2119]. = =20 1. Definitions=20 =20 A set of terminology associated with the ForCES requirements is = defined in [FORCES-REQ] and we only include the ones that are = most=20 relevant to this document here. =20 =20 Forwarding Element (FE) - A logical entity that implements the=20 ForCES protocol. FEs use the underlying hardware to provide = per- packet processing and handling as directed by a CE via the = ForCES=20 protocol. =20 =20 Control Element (CE) - A logical entity that implements the = ForCES=20 protocol and uses it to instruct one or more FEs how to process = packets. CEs handle functionality such as the execution of = control =20 and signaling protocols. =20 =20 Yang, et. al. Expires April 2003 [Page 2] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 =20 ForCES Network Element (NE) - An entity composed of one or more = CEs=20 and one or more FEs. To entities outside a NE, the NE = represents a=20 single point of management. Similarly, a NE usually hides its=20 internal organization from external entities. =20 =20 Pre-association Phase - The period of time during which a FE = Manager=20 (see below) and a CE Manager (see below) are determining which = FE=20 and CE should be part of the same network element.=20 =20 Post-association Phase - The period of time during which a FE = does=20 know which CE is to control it and vice versa, including the = time=20 during which the CE and FE are establishing communication with = one=20 another. =20 =20 ForCES Protocol - While there may be multiple protocols used = within=20 the overall ForCES architecture, the term "ForCES protocol" = refers=20 only to the ForCES post-association phase protocol (see below). = =20 =20 ForCES Post-Association Phase Protocol - The protocol used for = post- association phase communication between CEs and FEs. This = protocol=20 does not apply to CE-to-CE communication, FE-to-FE = communication, or=20 to communication between FE and CE managers. The ForCES = protocol is=20 a master-slave protocol in which FEs are slaves and CEs are = masters. =20 This protocol includes both the management of the communication = channel (e.g., connection establishment, heartbeats) and the = control=20 messages themselves. This protocol could be a single protocol = or=20 could consist of multiple protocols working together. =20 =20 FE Manager - A logical entity that operates in the = pre-association=20 phase and is responsible for determining to which CE(s) a FE = should=20 communicate. This process is called CE discovery and may = involve=20 the FE manager learning the capabilities of available CEs. A = FE=20 manager may use anything from a static configuration to a pre- association phase protocol (see below) to determine which CE(s) = to=20 use. Being a logical entity, a FE manager might be physically=20 combined with any of the other logical entities mentioned in = this=20 section. =20 =20 CE Manager - A logical entity that operates in the = pre-association =20 phase and is responsible for determining to which FE(s) a CE = should =20 communicate. This process is called FE discovery and may = involve=20 the CE manager learning the capabilities of available FEs. A = CE=20 manager may use anything from a static configuration to a pre- association phase protocol (see below) to determine which FE to = use. =20 Being a logical entity, a CE manager might be physically = combined=20 with any of the other logical entities mentioned in this = section. =20 =20 Pre-association Phase Protocol - A protocol between FE managers = and=20 CE managers that is used to determine which CEs or FEs to use. = A=20 pre-association phase protocol may include a CE and/or FE = capability=20 discovery mechanism. Note that this capability discovery = process is=20 wholly separate from (and does not replace) that used within = the=20 =20 Yang, et. al. Expires April 2003 [Page 3] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 ForCES protocol. However, the two capability discovery = mechanisms=20 may utilize the same FE model. =20 =20 FE Model - A model that describes the logical processing = functions=20 of a FE. =20 =20 ForCES Protocol Element - A FE or CE. =20 =20 2. Introduction to Forwarding and Control Element Separation = (ForCES) =20 =20 An IP network element (NE) appears to external entities as a=20 monolithic piece of network equipment, e.g., a router, NAT,=20 firewall, or load balancer. Internally, however, an IP network=20 element (NE) (such as a router) is composed of numerous = logically=20 separated entities that cooperate to provide a given = functionality=20 (such as routing). Two types of network element components = exist:=20 control element (CE) in control plane and forwarding element = (FE) in=20 forwarding plane (or data plane). Forwarding elements = typically are=20 ASIC, network-processor, or general-purpose processor-based = devices=20 that handle data path operations for each packet. Control = elements=20 are typically based on general-purpose processors that provide=20 control functionality like routing and signaling protocols. =20 =20 ForCES aims to define a framework and associated protocol(s) to = standardize information exchange between the control and = forwarding=20 plane. Having standard mechanisms allows CEs and FEs to become=20 physically separated standard components. This physical = separation=20 accrues several benefits to the ForCES architecture. Separate=20 components would allow component vendors to specialize in one=20 component without having to become experts in all components.=20 Standard protocol also allows the CEs and FEs from different=20 component vendors to interoperate with each other and hence it=20 becomes possible for system vendors to integrate together the = CEs=20 and FEs from different component suppliers. This = interoperability=20 translates into a lot more design choices and flexibility to = the=20 system vendors. Overall, ForCES will enable rapid innovation in = both=20 the control and forwarding planes while maintaining=20 interoperability. Scalability is also easily provided by this=20 architecture in that additional forwarding or control capacity = can=20 be added to existing network elements without the need for = forklift=20 upgrades.=20 =20 One example of such physical separation is at the blade level.=20 Figure 1 shows such an example configuration of a router, with = two=20 control blades and multiple router (forwarding) blades, all=20 interconnected into a switch fabric backplane. In such chassis=20 configuration, the control blades are the CEs while the router=20 blades are FEs, and the switch fabric backplane provides the=20 physical interconnect for all the blades. Control blade A may = be the=20 primary CE while control blade B is the backup CE providing=20 redundancy. It is also possible to have a redundant switch = fabric=20 for high availability support. Routers today with this kind of=20 configuration use proprietary interface for messaging between = CEs=20 =20 Yang, et. al. Expires April 2003 [Page 4] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 and FEs. The goal of ForCES is to replace such proprietary = interface=20 with a standard protocol. With a standard protocol like ForCES=20 implemented on all blades, it becomes possible for control = blades=20 from vendor X and routing blades from vendor Y to work = seamlessly=20 together in one chassis. =20 =20 ------------------------- -------------------------=20 | Control Blade A | | Control Blade B |=20 | (CE) | | (CE) |=20 ------------------------- -------------------------=20 ^ | ^ |=20 | | | |=20 | V | V=20 ---------------------------------------------------------=20 | Switch Fabric Backplane |=20 ---------------------------------------------------------=20 ^ | ^ | ^ | = | | | | =85 | | = =20 | V | V | V = ------------ ------------ ------------ = |Router | |Router | |Router |=20 |Blade #1 | |Blade #2 | |Blade #N |=20 | (FE) | | (FE) | | (FE) |=20 ------------ ------------ ------------ = ^ | ^ | ^ | = | | | | =85 | | = =20 | V | V | V = =20 Figure 1. A router configuration example with separate = blades.=20 =20 =20 ------- -------=20 | CE1 | | CE2 |=20 ------- -------=20 ^ ^=20 | |=20 V V=20 = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Ethernet=20 ^ ^ =85 ^=20 | | |=20 V V V=20 ------- ------- --------=20 | FE#1| | FE#2| | FE#n |=20 ------- ------- --------=20 ^ | ^ | ^ | =20 | | | | | | =20 | V | V | V =20 =20 Figure 2. A router configuration example with separate = boxes.=20 =20 Another level of physical separation between the CEs and FEs = can be=20 at the box level. In such configuration, all the CEs and FEs = are=20 physically separated boxes, interconnected with some kind of = high=20 =20 Yang, et. al. Expires April 2003 [Page 5] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 speed LAN connection (like Gigabit Ethernet). These separated = CEs=20 and FEs are only one hop away from each other within a local = area=20 network. The CEs and FEs communicate to each other by running=20 ForCES, and the collection of these CEs and FEs together become = one=20 routing unit to the external world. Figure 2 shows such an = example.=20 =20 In both examples shown here, the same physical interconnect is = used=20 for both CE-to-FE and FE-to-FE communication. However, that = does not=20 have to be the case. One reason to use different interconnect = is=20 that CE-to-FE interconnect does not have to be as fast as the = FE-to- FE interconnect, so the more expensive fast connections can be = saved=20 for FE-to-FE only. The separate interconnects may also provide = reliability and redundancy benefits for the NE.=20 =20 -------------------------------------------------=20 | | | | | | |=20 |OSPF |RIP |BGP |CAC |LDP |=85 |=20 | | | | | | |=20 -------------------------------------------------=20 | ForCES Interface |=20 -------------------------------------------------=20 ^ ^ =20 ForCES | |data =20 control | |packets =20 messages| |(e.g., routing packets) =20 v v=20 -------------------------------------------------=20 | ForCES Interface |=20 -------------------------------------------------=20 | | | | | | |=20 |LPM Fwd|Meter |Shaper |NAT |Classi-|=85 |=20 | | | | |fier | |=20 -------------------------------------------------=20 | FE resources |=20 -------------------------------------------------=20 =20 Figure 3. Examples of CE and FE functions=20 =20 Some examples of control functions that can be implemented in = the CE=20 include routing protocols like RIP, OSPF and BGP, control and=20 signaling protocols like CAC (Call Admission Control), LDP = (Label=20 Distribution Protocol) for MPLS, etc. Examples of forwarding=20 functions in FE include LPM (longest prefix match) forwarder,=20 classifiers, traffic shaper, meter, NAT, etc. Figure 3 shows a=20 diagram with examples in both CE and FE. Any given NE may = contain=20 one or many of these CE and FE functions in it. The diagram = also=20 shows that ForCES protocol is used to transport both the = control=20 messages for ForCES itself and the data packets that are=20 originated/destined from/to the control functions in CE (e.g.,=20 routing packets). Section 4.2.4 provides more detail on this. =20 =20 A set of requirements for control and forwarding separation is=20 identified in [FORCES-REQ]. This document describes a ForCES=20 =20 Yang, et. al. Expires April 2003 [Page 6] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 architecture that satisfies the architectural requirements of = that=20 document and defines a framework for ForCES network elements = and the=20 associated entities to facilitate protocol definition. Whenever = necessary, this document uses many examples to illustrate the = issues=20 and/or possible solutions in ForCES. These examples are = intended to=20 be just examples, and should not be taken as the only or = definite=20 ways of doing certain things. It is expected that separate = document=20 will be produced by the ForCES working group to specify the = ForCES=20 protocol(s).=20 =20 3. Architecture=20 =20 This section defines the ForCES architectural framework and the = associated logical components. This ForCES framework defines=20 components of ForCES NEs including several ancillary = components. =20 These components may be connected in different kinds of = topologies=20 for flexible packet processing. =20 =20 --------------------------------------- = | ForCES Network Element | = -------------- Fc | -------------- -------------- | = | CE Manager |---------+-| CE 1 |------| CE 2 | | = -------------- | | | Fr | | | = | | -------------- -------------- | = | Fl | | | Fp / | = | | Fp| |----------| / | = | | | |/ | = | | | | | = | | | Fp /|----| | = | | | /--------/ | | = -------------- Ff | -------------- -------------- | = | FE Manager |---------+-| FE 1 | Fi | FE 2 | | = -------------- | | |------| | | = =20 | -------------- -------------- | = | | | | | | | | | | = ----+--+--+--+----------+--+--+--+----- = | | | | | | | |=20 | | | | | | | |=20 Fi/f Fi/f=20 =20 Figure 4. ForCES Architectural Diagram=20 =20 The diagram in Figure 4 shows the logical components of the = ForCES=20 architecture and their relationships. There are two kinds of=20 components inside a ForCES network element: control element = (CE) and=20 forwarding element (FE). The framework allows multiple = instances of=20 CE and FE inside one NE. Each FE contains one or more physical = media=20 interfaces for receiving and transmitting packets from/to the=20 external world. The aggregation of these FE interfaces becomes = the=20 NE=92s external interfaces. In addition to the external = interfaces,=20 there must also exist some kind of interconnect within the NE = so=20 that the CE and FE can communicate with each other, and one FE = can=20 forward packets to another FE. The diagram also shows two = entities=20 =20 Yang, et. al. Expires April 2003 [Page 7] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 outside of the ForCES NE: CE Manager and FE Manager. These two=20 entities provide configuration to the corresponding CE or FE in = the=20 pre-association phase (see Section 5.1). There is no defined = role=20 for FE Manager and CE Manager in post-association phase, thus = these=20 logical components are not considered part of the ForCES NE. =20 =20 For convenience, the logical interactions between these = components=20 are labeled by reference points Fp, Fc, Ff, Fr, Fl, and Fi, as = shown=20 in Figure 4. The FE external interfaces are labeled as Fi/f. = More=20 detail is provided in Section 4 and 5 for each of these = reference=20 points. All these reference points are important in = understanding=20 the ForCES architecture, however, the ForCES protocol is only=20 defined over one reference point -- Fp. =20 =20 The interface between two ForCES NEs is identical to the = interface=20 between two conventional routers and these two NEs exchange the = protocol packets through the external interfaces at Fi/f. = ForCES NEs=20 connect to existing routers transparently.=20 =20 3.1. Control Elements and Fr Reference Point=20 =20 It is not necessary to define any protocols across the Fr = reference=20 point to enable control and forwarding separation for simple=20 configurations like single CE and multiple FEs. However, this=20 architecture permits multiple CEs to be present in a network=20 element. In cases where an implementation uses multiple CEs, = the=20 invariant that the CEs and FEs together appear as a single NE = must=20 be maintained.=20 =20 Multiple CEs may be used for redundancy, load sharing, = distributed=20 control, or other purposes. Redundancy is the case where one = or=20 more CEs are prepared to take over should an active CE fail. = Load=20 sharing is the case where two or more CEs are concurrently = active=20 and where any request that can be serviced by one of the CEs = can=20 also be serviced by any of the other CEs. For both redundancy = and=20 load sharing, the CEs involved are equivalently capable. The = only=20 difference between these two cases is in terms of how many = active=20 CEs there are. Distributed control is the case where two or = more=20 CEs are concurrently active but where certain requests can only = be=20 serviced by certain CEs.=20 =20 When multiple CEs are employed in a ForCES NE, their internal=20 organization is considered an implementation issue that is = beyond=20 the scope of ForCES. CEs are wholly responsible for = coordinating=20 amongst themselves via the Fr reference point to provide = consistency=20 and synchronization. However, ForCES does not define the=20 implementation or protocols used between CEs, nor does it = define how=20 to distribute functionality among CEs. Nevertheless, ForCES = will=20 support mechanisms for CE redundancy or fail over, and it is=20 expected that vendors will provide redundancy or fail over = solutions=20 within this framework.=20 =20 =20 =20 Yang, et. al. Expires April 2003 [Page 8] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 3.2. Forwarding Elements and Fi reference point=20 =20 FEs perform per-packet processing and handling as directed by = CEs.=20 FEs have no initiative of their own. Instead, FEs are slaves = and=20 only do as they are told. FEs may communicate with one or more = CEs=20 concurrently across reference point Fp. FEs have no notion of = CE=20 redundancy, load sharing, or distributed control. Instead, FEs = accept commands from any CE authorized to control them, and it = is up=20 to the CEs to coordinate among themselves to achieve = redundancy,=20 load sharing or distributed control. The idea is to keep FEs as = simple and dumb as possible so that FEs can focus its resource = on=20 the packet processing functions. =20 =20 =20 ------- Fr -------=20 | CE1 | ------| CE2 |=20 ------- -------=20 | \ / |=20 | \ / |=20 | \ / |=20 | \/Fp |=20 | /\ |=20 | / \ |=20 | / \ |=20 ------- Fi -------=20 | FE1 |<----->| FE2 |=20 ------- -------=20 =20 Figure 5. CE redundancy example.=20 =20 For example, in Figure 5, FE1 and FE2 can be configured to = accept=20 commands from both the primary CE (CE1) and the backup CE = (CE2).=20 Upon detection of CE1 failure, perhaps across the Fr reference=20 point, CE2 is configured to take over activities of CE1. This = is=20 beyond the scope of ForCES and is not discussed further.=20 =20 Distributed control can be achieved in the similar fashion, = without=20 much intelligence on the part of FEs. For example, FEs can be=20 configured to detect RSVP and BGP protocol packets, and forward = RSVP=20 packets to one CE and BGP packets to another CE. Hence, FEs may = need=20 to do packet filtering for forwarding packets to specific CEs.=20 =20 This architecture permits multiple FEs to be present in a NE.=20 [FORCES-REQ] dictates that the ForCES protocol must be able to = scale=20 to at least hundreds of FEs (see [FORCES-REQ] Section 5, = requirement=20 #11). Each of these FEs may potentially have a different set of = packet processing functions, with different media interfaces. = FEs=20 are responsible for basic maintenance of layer-2 connectivity = with=20 other FEs and with external entities. Many layer-2 media = include=20 sophisticated control protocols. The FORCES protocol (over the = Fp=20 reference point) will be able to carry messages for such = protools so=20 =20 Yang, et. al. Expires April 2003 [Page 9] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 that, in keeping with the "dumb FE=92=92 model, the CE can = provide=20 appropriate intelligence and control over these media.=20 =20 When multiple FEs are present, ForCES requires that packets = must be=20 able to arrive at the NE by one FE and leave the NE via a = different=20 FE (See [FORCES-REQ], Section 5, Requirement #3). Packets that = enter the NE via one FE and leave the NE via a different FE are = transferred between FEs across the Fi reference point. The Fi=20 reference point is a separate protocol from the Fp reference = point=20 and is not currently defined by the ForCES architecture.=20 =20 =20 =20 ----------------- =20 | CE | =20 ----------------- =20 ^ ^ ^=20 / | \=20 / v \=20 / ------- \=20 / +->| FE3 |<-+ \=20 / | | | | \=20 v | ------- | v=20 ------- | | -------=20 | FE1 |<-+ +->| FE2 |=20 | |<--------------->| |=20 ------- -------=20 ^ | ^ |=20 | | | |=20 | v | v=20 =20 (a) Full mesh among FE1, FE2 and FE3.=20 =20 =20 ----------- =20 | CE | =20 ----------- =20 ^ ^ ^ ^=20 / | | \=20 /------ | | ------\=20 v v v v=20 ------- ------- ------- -------=20 | FE1 |<->| FE2 |<->| FE3 |<->| FE4 |=20 ------- ------- ------- -------=20 ^ | ^ | ^ | ^ |=20 | | | | | | | |=20 | v | v | v | v=20 =20 (b) Multiple FEs in a daisy chain=20 =20 =20 =20 ^ |=20 =20 Yang, et. al. Expires April 2003 [Page 10] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 | v=20 ----------- =20 | FE1 |<-----------------------| =20 ----------- |=20 ^ ^ |=20 / \ | =20 | ^ / \ ^ | V=20 v | v v | v ---------- = --------- --------- | | = | FE2 | | FE3 |<------------>| CE | = --------- --------- | | = ^ ^ ^ ---------- = | \ / ^ ^=20 | \ / | |=20 | v v | |=20 | ----------- | |=20 | | FE4 |<----------------------| | =20 | ----------- |=20 | | ^ |=20 | v | |=20 | |=20 |----------------------------------------|=20 =20 (c) Multiple FEs connected by a ring=20 =20 Figure 6. Some examples of FE topology.=20 =20 FEs could be connected in different kinds of topologies and = packet=20 processing may spread across several FEs in the topology. = Hence,=20 logical packet flow may be different from physical FE topology. = Figure 6 provides some topology examples. When it is necessary = to=20 forward packets between FEs, the CE needs to understand the FE=20 topology. The FE topology can be queried from the FEs by CEs. =20 =20 3.3. CE Managers=20 =20 CE managers are responsible for determining which FEs a CE = should=20 control. It is legitimate for CE managers to be hard-coded = with the=20 knowledge of with which FEs its CEs should communicate. A CE = manager=20 may also be physically embedded into a CE and be implemented as = a=20 simple keypad or other direct configuration mechanism on the = CE. =20 Finally, CE managers may be physically and logically separate=20 entities that configure the CE with FE information via such=20 mechanisms as COPS-PR [RFC3084] or SNMP [RFC1157].=20 =20 3.4. FE Managers=20 =20 FE managers are responsible for determining to which CE any=20 particular FE should initially communicate. Like CE managers, = no=20 restrictions are placed on how a FE manager decides to which CE = its=20 FEs should communicate, nor are restrictions placed on how FE=20 managers are implemented.=20 =20 =20 Yang, et. al. Expires April 2003 [Page 11] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 =20 4. Operational Phases=20 =20 Both FEs and CEs require some configuration in place before = they can=20 start information exchange and function as a coherent network=20 element. Two operational phases are identified in this = framework --=20 pre-association and post-association. =20 =20 4.1.Pre-association Phase=20 =20 Pre-association phase is the period of time during which a FE=20 Manager and a CE Manager are determining which FE and CE should = be=20 part of the same network element. The protocols used during = this=20 phase may include all or some of the message exchange over Fl, = Ff=20 and Fc reference points. However, all these may be optional and = none=20 of this is within the scope of ForCES protocol.=20 =20 4.1.1. Fl Reference Point=20 =20 =20 FE Manager FE CE Manager CE=20 | | | |=20 | | | |=20 |(security exchange) | |=20 1|<------------------------------>| |=20 | | | |=20 |(a list of CEs and their attributes) |=20 2|<-------------------------------| |=20 | | | |=20 |(a list of FEs and their attributes) |=20 3|------------------------------->| |=20 | | | |=20 | | | |=20 |<----------------Fl------------>| |=20 =20 Figure 7. An example of message exchange over Fl reference = point=20 =20 CE managers and FE managers may communicate across the Fl = reference=20 point in the pre-association phase in order to determine which = CEs=20 and FEs should communicate with each other. Communication = across=20 the Fl reference point is optional in this architecture. No=20 requirements are placed on this reference point.=20 =20 CE managers and FE managers may be operated by different = entities. =20 The operator of the CE manager may not want to divulge, except = to=20 specified FE managers, any characteristics of the CEs it = manages. =20 Similarly, the operator of the FE manager may not want to = divulge FE=20 characteristics, except to authorized entities. As such, CE=20 managers and FE managers may need to authenticate one another. = Subsequent communication between CE managers and FE managers = may=20 require other security functions such as privacy, = non-repudiation,=20 freshness, and integrity.=20 =20 =20 Yang, et. al. Expires April 2003 [Page 12] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 Once the necessary security functions have been performed, the = CE=20 and FE managers communicate to determine which CEs and FEs = should=20 communicate with each other. At the very minimum, the CE and = FE=20 managers need to learn of the existence of available FEs and = CEs=20 respectively. This discovery process may or may not entail one = or=20 both managers learning the capabilities of the discovered = ForCES=20 protocol elements. Figure 7 shows an example of possible = message=20 exchange between CE manager and FE manager over Fl reference = point.=20 =20 4.1.2. Ff Reference Point=20 =20 FE Manager FE CE Manager CE=20 | | | |=20 | | | |=20 |(security exchange) |(security exchange)=20 1|<------------>|authentication = 1|<----------->|authentication | | | |=20 |(FE ID, attributes) |(CE ID, attributes)=20 2|<-------------|request 2|<------------|request=20 | | | |=20 3|------------->|response 3|------------>|response=20 |(corresponding CE ID) |(corresponding FE ID)=20 | | | |=20 | | | |=20 |<-----Ff----->| |<-----Fc---->|=20 =20 Figure 8. Examples of message exchange=20 over Ff and Fc reference points.=20 =20 The Ff reference point is used to inform forwarding elements of = the=20 association decisions made by FE managers in pre-association = phase. =20 Only authorized entities may instruct a FE with respect to = which CE=20 should control it. Therefore, privacy, integrity, freshness, = and=20 authentication are necessary between FE manager and FEs when = the FE=20 manager is remote to the FE. Once the appropriate security has = been=20 established, FE manager instructs FEs across this reference = point to=20 join a new NE or to disconnect from an existing NE. Figure 8 = shows=20 example of message exchange over Ff reference point.=20 =20 Note that the FE manager function may be co-located with the FE = (such as by manual keypad entry of the CE IP address), in which = case=20 this reference point is reduced to a built-in function.=20 =20 4.1.3. Fc Reference Point=20 =20 The Fc reference point is used to inform control elements of = the=20 association decisions made by CE managers in pre-association = phase. =20 When the CE manager is remote, only authorized entities may = instruct=20 a CE to control certain FEs. Privacy, integrity, freshness and = authentication are also required across this reference point in = such=20 a configuration. Once appropriate security has been = established,=20 the CE manager instructs CEs as to which FEs they should = control and=20 =20 Yang, et. al. Expires April 2003 [Page 13] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 how they should control them. Figure 7 shows example of message = exchange over Fc reference point.=20 =20 As with the FE manager and FEs, configurations are possible = where=20 the CE manager and CE are co-located and no protocol is used = for=20 this function.=20 =20 4.2. Post-association Phase and Fp reference point=20 =20 Post-association phase is the period of time during which a FE = and=20 CE have been configured with information necessary to contact = each=20 other and includes both association establishment and = steady-state=20 communication. The communication between CE and FE is performed = across the Fp ("p" meaning protocol) reference point. ForCES=20 protocol is exclusively used for all communication across the = Fp=20 reference point. =20 =20 4.2.1. Proximity and Interconnect between CEs and FEs=20 =20 The ForCES Working Group has made a conscious decision that the = first version of ForCES will not be designed to support=20 configurations where the CE and FE are located arbitrarily in = the=20 network. In particular, ForCES is intended for "very close" = CE/FE=20 localities in IP networks, as defined by ForCES Applicability=20 Statement ([FORCES-APP]). Very Close localities consist of = control=20 and forwarding elements that either are components in the same=20 physical box, or are separated at most by one local network = hop. =20 =20 CEs and FEs can be connected by a variety of interconnect=20 technologies, including Ethernet connections, backplanes, ATM = (cell)=20 fabrics, etc. ForCES should be able to support each of these=20 interconnects (see [FORCES-REQ] Section 5, requirement #1). = ForCES=20 will make use of an existing RFC2914 compliant L4 protocol with = adequate reliability, security and congestion control (e.g. = TCP,=20 SCTP) for transport purposes.=20 =20 4.2.2. Association Establishment=20 =20 As an example, figure 9 shows some of the message exchange that = may=20 happen before the association between the CE and FE is fully=20 established. Either the CE or FE can initiate the connection. = The FE=20 needs to inform the CE of its own capability and its topology = in=20 relation to other FEs. The capability of the FE is represented = by=20 the FE model, described in another separate document. The model = would allow a FE to describe what kind of packet processing=20 functions it contains, in what order the processing happens, = what=20 kinds of configurable parameters it allows, what statistics it=20 collects and what events it might throw, etc. Once such = information=20 is available to the CE, the CE sends all necessary = configuration to=20 the FE so that the FE can start receiving and processing = packets=20 correctly. For example, the CE might need to send a snapshot of = the=20 current forwarding table to the FE so that the FE can start = routing=20 packets correctly. Once FE starts accepting packets for = processing,=20 =20 Yang, et. al. Expires April 2003 [Page 14] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 we say the association of this FE with its CE is now = established.=20 From then on, the CE and FE enter steady-state communication.=20 =20 =20 FE CE=20 | |=20 |(Hello, are you there?)|=20 1|<----------------------|=20 | |=20 |(Yes. let me join the NE please.) =20 2|---------------------->|=20 | |=20 |(What kind of FE are you? -- capability query)=20 3|<----------------------|=20 | |=20 |(Here is my FE functions/state: use model to = describe)=20 4|---------------------->|=20 | |=20 |(How are you connected with others? -- topology = query)=20 5|<----------------------|=20 | |=20 |(Here is the topology info)=20 6|---------------------->|=20 | |=20 |(Config for FE, e.g. forwarding table)=20 7|<----------------------|=20 | |=20 |(I am ready to go. Shall I?) =20 8|---------------------->|=20 | |=20 |(Go ahead!) |=20 9|<----------------------|=20 | |=20 =20 Figure 9. Example of message exchange between CE and FE=20 over Fp to establish NE association=20 =20 =20 4.2.3. Steady-state Communication=20 =20 Once an association is established between the CE and FE, the = ForCES=20 protocol is used by the CE and FE over Fp reference point to=20 exchange information to facilitate packet processing. =20 =20 FE CE=20 | |=20 |(Add these new routes.)|=20 1|<----------------------|=20 | |=20 |(Successful.) |=20 2|---------------------->|=20 | |=20 | |=20 =20 Yang, et. al. Expires April 2003 [Page 15] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 |(Query some stats.) |=20 1|<----------------------|=20 | |=20 |(Reply with stats collected.) =20 2|---------------------->|=20 | |=20 | |=20 |(My port is down, with port #.) = =20 1|---------------------->|=20 | |=20 |(Route to this port instead...) =20 2|<----------------------|=20 | |=20 | |=20 =20 Figure 10. Examples of message exchange between CE and FE = over Fp during steady-state communication=20 =20 Based on the information acquired through CEs' control = processing,=20 CEs will frequently need to manipulate the packet-forwarding=20 behaviors of their FE(s) by sending instructions to FEs. For=20 example, Figure 10 shows message exchange examples in which the = CE=20 sends new routes to the FE so that the FE can add them to its=20 forwarding table. The CE may query the FE for statistics = collected=20 by the FE and the FE may notify the CE of important events such = as=20 port failure.=20 =20 4.2.4. Data Packets across Fp reference point=20 =20 Control plane protocol packets (such as RIP, OSPF messages)=20 addressed to any of NE's interfaces are typically redirected by = the=20 receiving FE to its CE, and CE may originate packets and have = its FE=20 deliver them to other NEs. Therefore, ForCES protocol over Fp = not=20 only transports the ForCES protocol messages between CEs and = FEs,=20 but also encapsulates the data packets from control plane = protocols.=20 Moreover, one FE may be controlled by multiple CEs for = distributed=20 control. In this configuration, the control protocols supported = by=20 the FORCES NEs may spread across multiple CEs. For example, one = CE=20 may support routing protocols like OSPF and BGP, while = signaling and=20 admission control protocol like RSVP is supported in another = CE. FEs=20 are configured to recognize and filter these protocol packets = and=20 forward them to the corresponding CE. =20 =20 Figure 11 shows one example of how the BGP packets originated = by=20 router A are passed to router B. In this example, the ForCES=20 protocol is used to transport the packets from the CE to the FE = inside router A, and then from the FE to the CE inside router = B. In=20 light of the fact that the ForCES protocol is responsible to=20 transport both the control messages and the data packets = between the=20 CE and FE over Fp reference point, it is possible to use either = a=20 single protocol or multiple protocols to achieve that.=20 =20 --------------------- ----------------------=20 =20 Yang, et. al. Expires April 2003 [Page 16] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 | | | |=20 | +--------+ | | +--------+ |=20 | |CE(BGP) | | | |CE(BGP) | |=20 | +--------+ | | +--------+ |=20 | | | | ^ |=20 | |Fp | | |Fp |=20 | v | | | |=20 | +--------+ | | +--------+ |=20 | | FE | | | | FE | |=20 | +--------+ | | +--------+ |=20 | | | | ^ |=20 | Router | | | Router | |=20 | A | | | B | |=20 ---------+----------- -----------+----------=20 v ^=20 | |=20 | |=20 ------------------->--------------- =20 =20 Figure 11. Example to show data packet flow between two = NEs.=20 =20 =20 4.2.5. Proxy FE=20 =20 In the case where a physical FE cannot implement (e.g., due to = the=20 lack of a general purpose CPU) the ForCES protocol directly, a = proxy=20 FE can be used in the middle of Fp reference point. This allows = the=20 CE communicate to the physical FE via the proxy by using = ForCES,=20 while the proxy manipulates the physical FE using some = intermediary=20 form of communication (e.g., a non-ForCES protocol or DMA). In = such=20 an implementation, the combination of the proxy and the = physical FE=20 becomes one logical FE entity. =20 =20 4.3. Association Re-establishment=20 =20 FEs and CEs may join and leave NEs dynamically (see = [FORCES-REQ]=20 Section 5, requirements #12). When a FE or CE leaves the NE, = the=20 association with the NE is broken. If the leaving party rejoins = a NE=20 later, to re-establish the association, it may or may not need = to=20 re-enter the pre-association phase. Loss of association can = also=20 happen unexpectedly due to loss of connection between the CE = and the=20 FE. Therefore, the framework allows the bi-directional = transition=20 between these two phases, but the ForCES protocol is only = applicable=20 for the post-association phase. However, the protocol should = provide=20 mechanisms to support association re-establishment (see = [FORCES-REQ]=20 Section 5, requirement #7).=20 =20 The example in Figure 5 is used to illustrate what happens when = the=20 association is broken and later re-established again. Section = 4.2=20 already explains what happens if CE1 fails and how CE2 can take = over. If no CE redundancy is provided, at the association=20 establishment phase FEs need to be told what to do in the case = of CE=20 failure. FEs may be told to stop packet processing all together = if=20 =20 Yang, et. al. Expires April 2003 [Page 17] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 its CE fails. Or, FEs may be told to continue forwarding = packets for=20 a limited time even in the face of CE failure. No matter what = the=20 instruction is, it needs to be part of the configuration when = the=20 association is established. =20 =20 In the same example in Figure 5, assuming CE1 is the working CE = for=20 the moment, what would happen if one of the FEs, say FE1, = leaves the=20 NE temporarily? FE1 may voluntarily decide to leave the = association.=20 Or, FE1 may stop functioning simply due to unexpected failure. = In=20 former case, CE1 receives a "leave-association request" from = FE1. In=20 the latter, CE1 detects the failure of FE1 by some other mean. = In=20 both cases, CE1 keeps a note of such event from FE1 while = continue=20 commanding FE2. When FE1 decides to rejoin again, or when it is = back=20 up again from the failure, FE1 needs to re-discover its master = (CE).=20 This can be achieved by several means. It may re-enter the pre- association phase and get that information from its FE manager. = It=20 may retrieve the previous CE information from its cache, if it = can=20 validate the information freshness. Once it discovers its CE, = it=20 starts message exchange with CE to re-establish the association = just=20 as outlined in Figure 9, with the possible exception that it = might=20 be able to bypass the transport of the complete initialization=20 information. Suppose that FE1 still has its routing table and = other=20 state information from the last association, instead of sending = all=20 the information again from scratch, it can choose to use more=20 efficient mechanism to re-sync up the state with its CE. For=20 example, a checksum of the state might give a quick indication = of=20 whether or not the state is in-sync with its CE. By comparing = its=20 state with CE first, it sends information update only if it is=20 needed.=20 =20 =20 5. Applicability to RFC1812 =20 =20 [FORCES-REQ] Section 5, requirement #9 dictates that "All = proposed=20 ForCES architecture must explain how that architecture supports = all=20 of the router functions as defined in [RFC1812]." RFC1812 = discusses=20 many important requirements for IPv4 routers from the link = layer to=20 the application layer. This section addresses the relevant=20 requirements in RFC1812 for implementing IPv4 routers based on=20 ForCES architecture and explains how ForCES satisfies these=20 requirements by providing guidelines on how to separate the=20 functionalities required into forwarding plane and control = plane. =20 =20 In general, the forwarding plane carries out the bulk of the = per- packet processing that is required at line speed, while the = control=20 plane carries most of the computationally complex operations = that=20 are typical of the control and signaling protocols. However, it = is=20 not practical to draw a rigid line to divide the processing = into CEs=20 and FEs cleanly. Nor should the ForCES architecture limits the=20 innovative approaches in control and forwarding plane = separation. As=20 more and more processing power is available in the FEs, some of = the=20 control functions that traditionally are performed by CEs may = now be=20 moved to FEs for better performance and scalability. Such = offloaded=20 =20 Yang, et. al. Expires April 2003 [Page 18] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 functions may include part of ICMP or TCP processing, or part = of=20 routing protocols. Once off-loaded onto the forwarding plane, = such=20 CE functions, even though logically belong to the control = plane, now=20 become part of the FE logical functions. Just like the other = logical=20 functions performed by FEs, such off-loaded functions must be=20 expressed as part of the FE model so that the CEs can decide = how to=20 best take advantage of these off-loaded functions when present = on=20 the FEs.=20 =20 =20 5.1. General Router Requirements=20 =20 Routers have at least two or more logical interfaces. When CEs = and=20 FEs are separated by ForCES within a single NE, some additional = interfaces are needed for intra-NE communications. Figure 12 = shows=20 an example to illustrate that. This NE contains one CE and two = FEs.=20 Each FE has four interfaces; two of them are used for receiving = and=20 transmitting packets to the external world, while the other two = are=20 for intra-NE connections. CE has two logical interfaces #9 and = #10,=20 connected to interfaces #3 and #6 from FE1 and FE2, = respectively.=20 Interface #4 and #5 are connected for FE1-FE2 communication.=20 Therefore, this router NE provides four external interfaces = (#1, 2,=20 7 and 8). =20 =20 IPv4 routers must implement IP to support its packet forwarding = function, which is driven by its FIB (Forwarding Information = Base).=20 This Internet layer forwarding (see [RFC1812] Section 5)=20 functionality naturally belongs to FEs in the ForCES = architecture. =20 =20 ---------------------------------=20 | router NE |=20 | ----------- ----------- |=20 | | FE1 | | FE2 | |=20 | ----------- ----------- |=20 | 1| 2| 3| 4| 5| 6| 7| 8| |=20 | | | | | | | | | |=20 | | | | +----+ | | | |=20 | | | | | | | |=20 | | | 9| 10| | | |=20 | | | -------------- | | |=20 | | | | CE | | | |=20 | | | -------------- | | |=20 | | | | | |=20 -----+--+----------------+--+----=20 | | | | =20 | | | | =20 =20 Figure 12. A router NE example with four = interfaces.=20 =20 =20 A router may implement transport layer protocols (like TCP and = UDP)=20 that are required to support application layer protocols (see=20 [RFC1812] Section 6). One important class of application = protocols=20 =20 Yang, et. al. Expires April 2003 [Page 19] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 is routing protocols (see [RFC1812] Section 7). In ForCES=20 architecture, routing protocols are naturally implemented by = CEs.=20 Routing protocols require routers communicate with each other. = This=20 communication between CEs in different routers is supported in=20 ForCES by FEs' ability to redirect data packets addressed to = routers=20 (i.e., NEs) and CEs' ability to originate packets and have them = delivered by their FEs. This communication occurs across Fp=20 reference point inside each router and between neighboring = routers'=20 external interfaces, as illustrated in Figure 11. =20 =20 5.2.Link Layer=20 =20 Since FEs own all the external interfaces for the router, FEs = need=20 to conform to the link layer requirements in RFC1812. Arguably, = ARP=20 support may be implemented in either CEs or FEs. As we will see = later, a number of behaviors that RFC1812 mandates fall into = this=20 category -- they may be performed by the FE and may be = performed by=20 the CE. A general guideline is needed to ensure = interoperability=20 between separated control and forwarding planes. The guideline = we=20 offer here is that in general CEs are required to be capable of = these kind of operations while FEs may or may not choose to=20 implement them. FE model should indicate its capabilities in = this=20 regard so that CEs can decide where these functions are = implemented.=20 =20 Interface parameters, including MTU, IP address, etc., must be=20 configurable by CEs via ForCES. CEs must be able to determine=20 whether a physical interface in an FE is available to send = packets=20 or not. FEs must also inform CEs the status change of the = interfaces=20 (like link up/down) via ForCES.=20 =20 5.3.Internet Layer Protocols=20 =20 Both FEs and CEs must implement IP protocol and all mandatory=20 extensions as RFC1812 specified. CEs should implement IP = options=20 like source route and record route while FEs may choose to = implement=20 those as well. Timestamp option should be implemented by FEs to = insert the timestamp most accurately. FE must interpret the IP=20 options that it understands and preserve the rest unchanged for = use=20 by CEs. Both FEs and CEs might choose to silently discard = packets=20 without sending ICMP errors, but such events should be logged = and=20 counted. FEs may report statistics for such events to CEs via=20 ForCES. =20 =20 When multiple FEs are involved to process packets, the = appearance of=20 single NE must be strictly maintained. For example, = Time-To-Live=20 (TTL) must be decremented only once within a single NE. For = example,=20 it can be always decremented by the last FE with egress = function. =20 =20 FEs must receive and process normally any packets with a = broadcast=20 destination address or a multicast destination address that the = router has asked to receive. When IP multicast is supported in=20 routers, IGMP is implemented in CEs. CEs are also required of = ICMP=20 support, while it is optional for FEs to support ICMP. Such an=20 =20 Yang, et. al. Expires April 2003 [Page 20] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 option can be communicated to CEs as part of the FE model.=20 Therefore, FEs can always rely upon CEs to send out ICMP error=20 messages, but FEs also have the option to generate ICMP error=20 messages themselves. =20 =20 5.4.Internet Layer Forwarding=20 =20 IP forwarding is implemented by FEs. When the routing table is=20 updated at CEs, ForCES is used to send the new route entries = from=20 CEs to FEs. Each FE has its own forwarding table and uses this = table=20 to direct packets to the next hop interface. =20 =20 Upon receiving IP packets, FE verifies the IP header and = processes=20 most of the IP options. Some options can't be processed until = the=20 routing decision has been made. Routing decision is made after=20 examining the destination IP address. If the destination = address=20 belongs to the router itself, the packets are filtered and = either=20 processed locally or forwarded to CE, depending upon the=20 instructions set-up by CE. Otherwise, FE determines the next = hop IP=20 address by looking up in its forwarding table. FE also = determines=20 the network interface it uses to send the packets. Sometimes FE = may=20 need to forward the packets to another FE before packets can be = forwarded out to the next hop. Right before packets are = forwarded=20 out to the next hop, FE decrements TTL by 1 and processes any = IP=20 options that cannot be processed before. FE performs any IP=20 fragmentation if necessary, determines link layer address = (e.g., by=20 ARP), and encapsulates the IP datagram (or each of the = fragments=20 thereof) in an appropriate link layer frame and queues it for = output=20 on the interface selected.=20 =20 Other options mentioned in RFC1812 for IP forwarding may also = be=20 implemented at FEs, for example, packet filtering.=20 =20 FEs typically forward packets destined locally to CEs. FEs may = also=20 forward exceptional packets (packets that FEs don't know how to = handle) to CEs. CEs are required to handle packets forwarded by = FEs=20 for whatever different reasons. It might be necessary for = ForCES to=20 attach some meta-data with the packets to indicate the reasons = of=20 forwarding from FEs to CEs. Upon receiving packets with = meta-data=20 from FEs, CEs can decide to either process the packets = themselves,=20 or pass the packets to the upper layer protocols including = routing=20 and management protocols. If CEs are to process the packets by=20 themselves, CEs may choose to discard the packets, or modify = and re- send the packets. CEs may also originate new packets and = deliver=20 them to FEs for further forwarding. =20 =20 Any state change during router operation must also be handled=20 correctly according to RFC1812. For example, when an FE ceases=20 forwarding, the entire NE may continue forwarding packets, but = it=20 needs to stop advertising routes that are affected by the = failed FE. =20 =20 5.5.Transport Layer =20 =20 =20 Yang, et. al. Expires April 2003 [Page 21] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 Transport layer is typically implemented at CEs to support = higher=20 layer application protocols like routing protocols. In = practice,=20 this means that most CEs implement both the Transmission = Control=20 Protocol (TCP) and the User Datagram Protocol (UDP).=20 =20 Both CEs and FEs need to implement ForCES protocol. If some = layer-4=20 transport is used to support ForCES, then both CEs and FEs need = to=20 implement the L4 transport and ForCES protocols. It is possible = that=20 all FEs inside an NE implements only one such protocol entity. = =20 5.6. Application Layer -- Routing Protocols=20 =20 Both interior routing protocols and exterior routing protocols = are=20 implemented on CEs. The routing packets originated by CEs are=20 forwarded to FEs for delivery. The results of such protocols = (like=20 forwarding table update) are communicated to FEs via ForCES.=20 =20 For performance or scalability reasons, portions of the control = plane functions that need faster response may be moved from the = CEs=20 and off-loaded onto the FEs. For example in OSPF, the Hello = protocol=20 packets are generated and processed periodically. When done at = CEs,=20 the inbound Hello packets have to traverse from the external=20 interfaces at the FEs to the CEs via the internal CE-FE = channel.=20 Similarly, the outbound Hello packets have to go from the CEs = to the=20 FEs and to the external interfaces. Frequent Hello updates = place=20 heavy processing overhead on the CEs and can overwhelm the = CE-FE=20 channel as well. Since typically there are far more FEs than = CEs in=20 a router, when off-loaded, the Hello packets are processed in a = much=20 more distributed and scalable fashion. Interoperability is = ensured=20 by expressing such off-loaded functions in the FE model.=20 =20 5.7. Application Layer -- Network Management Protocol=20 =20 RFC1812 also dictates "Routers MUST be manageable by SNMP." = (see=20 [RFC1157] Section 8) In general, for post-association phase, = most=20 external management tasks (including SNMP) should be done = through=20 interaction with the CE in order to support the appearance of a = single functional device. Therefore, it is recommended that = SNMP=20 management agent be implemented by CEs and the SNMP messages=20 received by FEs be redirected to their CEs. AgentX framework = defined=20 in RFC2741 may be applied here such that CEs act in the role of = master agent to process SNMP protocol messages while FEs act in = the=20 role of subagent to provide access to the MIB objects residing = on=20 FEs. AgentX protocol messages between the master agent (CE) and = the=20 subagent (FE) are encapsulated and transported via ForCES, just = like=20 data packets from any other application layer protocols. =20 =20 6. Summary=20 =20 This document defines an architectural framework for ForCES. It = identifies the relevant components for a ForCES network = element,=20 including (one or more) FEs, (one or more) CEs, one optional FE = manager, and one optional CE manager. It also identifies the=20 =20 Yang, et. al. Expires April 2003 [Page 22] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 interaction among these components and discusses all the major=20 reference points. It is important to point out that, among all = the=20 reference points, only the interface between CEs and FEs is = within=20 the scope of ForCES. ForCES alone may not be enough to support = all=20 desirablet NE configurations. However, we believe that ForCES = is the=20 most important element in realizing physical separation and=20 interoperability of CEs and FEs, and hence the first interface = that=20 ought to be standardized. Simple and useful configurations can = still=20 be implemented with only CE-FE interface being standardized, = e.g.,=20 single CE with full-meshed FEs and static configuration without = the=20 need for CE/FE managers.=20 =20 7. Security Considerations=20 =20 The security necessary across each reference point except Fp is = discussed throughout the document. In general, the physical=20 separation of two entities usually requires much stricter = security=20 measurement in place. For example, we pointed out in Section = 5.1=20 that authentication becomes necessary between CE manager and FE = manager, between CE and CE manager, between FE and FE manager = in=20 some configuration. The physical separation of CE and FE also=20 imposes serious security requirement for ForCES protocol. The=20 security requirements for reference point Fp (i.e., ForCES = protocol)=20 are discussed in detail in [FORCES-REQ] Section 8.=20 =20 8. Intellectual Property Right=20 =20 The authors are not aware of any intellectual property right = issues=20 pertaining to this document.=20 =20 9. References=20 =20 9.1. Normative References=20 =20 [RFC1157] J. Case, et. al., "A Simple Network Management = Protocol=20 (SNMP)", RFC1157, May 1990.=20 =20 [RFC1812] F. Baker, "Requirements for IP Version 4 Routers",=20 RFC1812, June 1995.=20 =20 [RFC2741] M. Daniele, et. al., "Agent Extensibility (AgentX)=20 Protocol Version 1", RFC2741, Jan 2000.=20 =20 [RFC2914] S. Floyd, "Congestion Control Principles", RFC2914,=20 September 2000.=20 =20 [RFC3084] K. Chan, et. al., "COPS Usage for Policy Provisioning = (COPS-PR)", RFC3084, March 2001.=20 =20 9.2. Informative References=20 =20 =20 Yang, et. al. Expires April 2003 [Page 23] = =0C Internet Draft ForCES Framework Oct. 2002 = =20 =20 [FORCES-REQ] T. Anderson, et. al., "Requirements for Separation = of=20 IP Control and Forwarding", work in progress, July 2002, = .=20 =20 [FORCES-APP] A. Crouch, et. al., "ForCES Applicability = Statement",=20 work in progress, June 2002, .=20 =20 10. Acknowledgments=20 =20 Joel M. Halpern gave us many insightful comments and = suggestions and=20 pointed out several major issues. Many of our colleagues and = people=20 in the ForCES mailing list also provided valuable feedback.=20 =20 11. Authors' Addresses=20 =20 Lily L. Yang=20 Intel Corp., MS JF3-206,=20 2111 NE 25th Avenue=20 Hillsboro, OR 97124 USA=20 Phone: +1 503 264 8813=20 Email: lily.l.yang@intel.com=20 =20 Ram Dantu=20 University of Texas Dallas=20 2601 North Flyod Road=20 Richardson Texas 75082=20 Phone: +1 972 883 4653=20 Email: ram.dantu@utdallas.edu=20 =20 Todd A. Anderson=20 Intel Corp.=20 2111 NE 25th Avenue=20 Hillsboro, OR 97124 USA=20 Phone: +1 503 712 1760=20 Email: todd.a.anderson@intel.com=20 =20 =20 =20 Yang, et. al. Expires April 2003 [Page 24] = =0C ------_=_NextPart_000_01C27C6B.F9B44100-- From owner-forces@PEACH.EASE.LSOFT.COM Mon Oct 28 06:37:38 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA03990 for ; Mon, 28 Oct 2002 06:37:37 -0500 (EST) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <22.0079655D@cherry.ease.lsoft.com>; Mon, 28 Oct 2002 6:39:58 -0500 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 30250634 for FORCES@PEACH.EASE.LSOFT.COM; Mon, 28 Oct 2002 06:39:58 -0500 Received: from 132.151.1.176 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Mon, 28 Oct 2002 06:29:58 -0400 Received: from CNRI.Reston.VA.US (localhost [127.0.0.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA03170; Mon, 28 Oct 2002 06:27:35 -0500 (EST) Mime-Version: 1.0 Content-Type: Multipart/Mixed; Boundary="NextPart" Approved-By: nsyracus@CNRI.RESTON.VA.US Message-ID: <200210281127.GAA03170@ietf.org> Date: Mon, 28 Oct 2002 06:27:34 -0500 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation Comments: RFC822 error: Incorrect or incomplete address field found and ignored. From: Internet-Drafts@ietf.org Subject: I-D ACTION:draft-ietf-forces-requirements-07.txt To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list --NextPart A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Forwarding and Control Element Separation Working Group of the IETF. Title : Requirements for Separation of IP Control and Forwarding Author(s) : H. Khosravi, T. Anderson Filename : draft-ietf-forces-requirements-07.txt Pages : 15 Date : 2002-10-25 This document introduces the ForCES architecture and defines a set of associated terminology. This document also defines a set of architectural, modeling, and protocol requirements to logically separate the control and data forwarding planes of an IP (IPv4, IPv6, etc.) networking device. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-forces-requirements-07.txt To remove yourself from the IETF Announcement list, send a message to ietf-announce-request with the word unsubscribe in the body of the message. Internet-Drafts are also available by anonymous FTP. Login with the username "anonymous" and a password of your e-mail address. After logging in, type "cd internet-drafts" and then "get draft-ietf-forces-requirements-07.txt". A list of Internet-Drafts directories can be found in http://www.ietf.org/shadow.html or ftp://ftp.ietf.org/ietf/1shadow-sites.txt Internet-Drafts can also be obtained by e-mail. Send a message to: mailserv@ietf.org. In the body type: "FILE /internet-drafts/draft-ietf-forces-requirements-07.txt". NOTE: The mail server at ietf.org can return the document in MIME-encoded form by using the "mpack" utility. To use this feature, insert the command "ENCODING mime" before the "FILE" command. To decode the response(s), you will need "munpack" or a MIME-compliant mail reader. Different MIME-compliant mail readers exhibit different behavior, especially when dealing with "multipart" MIME messages (i.e. documents which have been split up into multiple messages), so check your local documentation on how to manipulate these messages. Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. --NextPart Content-Type: Multipart/Alternative; Boundary="OtherAccess" --OtherAccess Content-Type: Message/External-body; access-type="mail-server"; server="mailserv@ietf.org" Content-Type: text/plain Content-ID: <2002-10-25110307.I-D@ietf.org> ENCODING mime FILE /internet-drafts/draft-ietf-forces-requirements-07.txt --OtherAccess Content-Type: Message/External-body; name="draft-ietf-forces-requirements-07.txt"; site="ftp.ietf.org"; access-type="anon-ftp"; directory="internet-drafts" Content-Type: text/plain Content-ID: <2002-10-25110307.I-D@ietf.org> --OtherAccess-- --NextPart-- From owner-forces@PEACH.EASE.LSOFT.COM Tue Oct 29 06:26:38 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA28735 for ; Tue, 29 Oct 2002 06:26:37 -0500 (EST) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <4.00798EE4@cherry.ease.lsoft.com>; Tue, 29 Oct 2002 6:28:59 -0500 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 30345328 for FORCES@PEACH.EASE.LSOFT.COM; Tue, 29 Oct 2002 06:28:57 -0500 Received: from 132.151.1.176 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Tue, 29 Oct 2002 06:18:57 -0400 Received: from CNRI.Reston.VA.US (localhost [127.0.0.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA27565; Tue, 29 Oct 2002 06:16:33 -0500 (EST) Mime-Version: 1.0 Content-Type: Multipart/Mixed; Boundary="NextPart" Approved-By: nsyracus@CNRI.RESTON.VA.US Message-ID: <200210291116.GAA27565@ietf.org> Date: Tue, 29 Oct 2002 06:16:33 -0500 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation Comments: RFC822 error: Incorrect or incomplete address field found and ignored. From: Internet-Drafts@ietf.org Subject: I-D ACTION:draft-ietf-forces-framework-03.txt To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list --NextPart A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Forwarding and Control Element Separation Working Group of the IETF. Title : ForCES Architectural Framework Author(s) : L. Yang et al. Filename : draft-ietf-forces-framework-03.txt Pages : 24 Date : 2002-10-28 This document defines the architectural framework for ForCES (Forwarding and Control Element Separation) network elements (NE), and identifies the associated entities and the interaction among them. This framework is intended to satisfy the requirements specified in the ForCES requirements draft [FORCES-REQ]. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-forces-framework-03.txt To remove yourself from the IETF Announcement list, send a message to ietf-announce-request with the word unsubscribe in the body of the message. Internet-Drafts are also available by anonymous FTP. Login with the username "anonymous" and a password of your e-mail address. After logging in, type "cd internet-drafts" and then "get draft-ietf-forces-framework-03.txt". A list of Internet-Drafts directories can be found in http://www.ietf.org/shadow.html or ftp://ftp.ietf.org/ietf/1shadow-sites.txt Internet-Drafts can also be obtained by e-mail. Send a message to: mailserv@ietf.org. In the body type: "FILE /internet-drafts/draft-ietf-forces-framework-03.txt". NOTE: The mail server at ietf.org can return the document in MIME-encoded form by using the "mpack" utility. To use this feature, insert the command "ENCODING mime" before the "FILE" command. To decode the response(s), you will need "munpack" or a MIME-compliant mail reader. Different MIME-compliant mail readers exhibit different behavior, especially when dealing with "multipart" MIME messages (i.e. documents which have been split up into multiple messages), so check your local documentation on how to manipulate these messages. Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. --NextPart Content-Type: Multipart/Alternative; Boundary="OtherAccess" --OtherAccess Content-Type: Message/External-body; access-type="mail-server"; server="mailserv@ietf.org" Content-Type: text/plain Content-ID: <2002-10-28162943.I-D@ietf.org> ENCODING mime FILE /internet-drafts/draft-ietf-forces-framework-03.txt --OtherAccess Content-Type: Message/External-body; name="draft-ietf-forces-framework-03.txt"; site="ftp.ietf.org"; access-type="anon-ftp"; directory="internet-drafts" Content-Type: text/plain Content-ID: <2002-10-28162943.I-D@ietf.org> --OtherAccess-- --NextPart-- From owner-forces*forces-archive**IETF*-ORG@PEACH.EASE.LSOFT.COM Tue Oct 29 17:26:41 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA29706 for ; Tue, 29 Oct 2002 17:26:37 -0500 (EST) Received: from PEACH.EASE.LSOFT.COM (209.119.0.61) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <10.0079A30A@cherry.ease.lsoft.com>; Tue, 29 Oct 2002 17:28:59 -0500 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 30391939 for FORCES@PEACH.EASE.LSOFT.COM; Tue, 29 Oct 2002 17:28:58 -0500 Received: from 134.134.136.6 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Tue, 29 Oct 2002 17:28:58 -0400 Received: from talaria.jf.intel.com (talaria.jf.intel.com [10.7.209.7]) by hermes.jf.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.51 2002/09/23 20:43:23 dmccart Exp $) with ESMTP id g9TMR4Y12792 for ; Tue, 29 Oct 2002 22:27:05 GMT Received: from orsmsxvs040.jf.intel.com (orsmsxvs040.jf.intel.com [192.168.65.206]) by talaria.jf.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.27 2002/10/16 23:46:59 dmccart Exp $) with SMTP id g9TMHL902813 for ; Tue, 29 Oct 2002 22:17:24 GMT Received: from orsmsx26.jf.intel.com ([192.168.65.26]) by orsmsxvs040.jf.intel.com (NAVGW 2.5.2.11) with SMTP id M2002102914323823623 for ; Tue, 29 Oct 2002 14:32:38 -0800 Received: by orsmsx26.jf.intel.com with Internet Mail Service (5.5.2653.19) id ; Tue, 29 Oct 2002 14:28:53 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/mixed; boundary="----_=_NextPart_000_01C27F9A.8EAED9B0" Approved-By: "Yang, Lily L" Message-ID: Date: Tue, 29 Oct 2002 14:28:51 -0800 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: "Yang, Lily L" Subject: individual contribution on ForCES FE model, please comment To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_000_01C27F9A.8EAED9B0 Content-Type: text/plain; charset="iso-8859-1" Hi, all Joel Halpern, Ram Gopal, Ram Dantu and myself would like to submit a newer version of the individual contribution on ForCES FE model for the upcoming IETF meeting. We plan to submit it by this weekend before the deadline. We would like to post to the list even while we are still working on it and and get some feedback if possible. Please take a look and provide comments. Thanks, Lily <> ------_=_NextPart_000_01C27F9A.8EAED9B0 Content-Type: text/plain; name="draft-yang-forces-model-01.txt" Content-Disposition: attachment; filename="draft-yang-forces-model-01.txt" Content-Transfer-Encoding: quoted-printable Internet Draft L. Yang=20 Expiration: May 2003 Intel Labs=20 File: draft-yang-forces-model-01.txt J. Halpern=20 Working Group: ForCES =20 R. Gopal=20 Nokia=20 R. Dantu=20 Univ. of = Texas=20 Nov 2002=20 = =20 =20 =20 ForCES Forwarding Element Functional Model=20 =20 =20 =20 draft-yang-forces-model-01.txt=20 =20 =20 =20 =20 Status of this Memo=20 =20 This document is an Internet-Draft and is in full conformance = with=20 all provisions of Section 10 of RFC2026. Internet-Drafts are=20 working documents of the Internet Engineering Task Force = (IETF), its=20 areas, and its working groups. Note that other groups may also = distribute working documents as Internet-Drafts.=20 =20 Internet-Drafts are draft documents valid for a maximum of six=20 months and may be updated, replaced, or obsoleted by other = documents=20 at any time. It is inappropriate to use Internet-Drafts as=20 reference material or to cite them other than as ``work in=20 progress.''=20 =20 The list of current Internet-Drafts can be accessed at=20 http://www.ietf.org/ietf/1id-abstracts.txt.=20 =20 The list of Internet-Draft Shadow Directories can be accessed = at =20 http://www.ietf.org/shadow.html.=20 =20 Abstract=20 =20 This document defines a functional model for forwarding = elements=20 (FEs) used in the Forwarding and Control Plane Separation = (ForCES)=20 protocol. This model is used to describe the capabilities and = state=20 of ForCES forwarding elements within the context of the ForCES=20 protocol, so that ForCES control elements (CEs) can control the = FEs=20 accordingly. The model is to specify what logical functions are = present in the FEs, what capabilities these functions support, = and=20 in what order these functions are or can be performed. The=20 forwarding element model defined herein is intended to satisfy = the=20 requirements specified in the ForCES requirements draft = [FORCES- REQ]. Using this model, predefined or vendor specific logical=20 =20 =0C Internet Draft ForCES FE Functional Model Nov = 2002=20 =20 =20 functions can be expressed and configured. However, the = definition=20 of these components are not described and defined in this = document. =20 =20 Table of Contents =20 =20 = Abstract...........................................................1=20 1. = Definitions.....................................................2=20 2. Motivation and Requirements of FE = model.........................3=20 3. Capability (Configuration) Model versus State = Model.............3=20 4. FE = Model........................................................6=20 4.1. FE = Blocks..................................................6=20 4.2. FE Block = Library...........................................7=20 4.2.1. QoS = Functions.........................................8=20 4.2.2. Generic Filtering = Functions...........................9=20 4.2.3. Vendor Specific = Functions.............................9=20 4.2.4. Port = Functions........................................9=20 4.2.5. Forwarding = Functions..................................9=20 4.2.6. High-Touch Functions...............................= ..10=20 4.2.7. Security = Functions...................................10=20 4.2.8. Off-loaded = Functions.................................10=20 4.3. FE Stage and Directed Graph of = FE.........................10=20 4.3.1. Basic = Concepts.......................................10=20 4.3.2. Topological versus Encoded State = Approaches..........11=20 4.3.3. Cascading FE = Blocks..................................13=20 5. Data Modeling and = Representation...............................14=20 6. Security = Considerations........................................15=20 7. Intellectual Property = Right....................................15=20 8. IANA = consideration.............................................15=20 9. Normative = References...........................................15=20 10. Informative = References........................................16=20 11. = Acknowledgments...............................................16=20 12. Authors' = Addresses............................................16=20 =20 Conventions used in this document =20 =20 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL = NOT", =20 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" = in=20 this document are to be interpreted as described in [RFC-2119]. = =20 =20 1. Definitions=20 =20 A set of terminology associated with the ForCES requirements is = defined in [FORCES-REQ] and is not copied here. The following = list=20 of terminology is relevant to the FE model defined in this = document.=20 =20 Datapath -- A conceptual path taken by packets within the = forwarding=20 plane, inside an FE. There might exist more than one datapath = within=20 an FE.=20 =20 Forwarding Element (FE) Block -- An abstraction of the basic = packet=20 processing logical functions in the datapath. It is the = building=20 block of FE functionality. This concept abstracts away=20 =20 Yang, et. al. Expires May 2003 [Page 2] = =0C Internet Draft ForCES FE Functional Model Nov = 2002=20 =20 =20 implementation details from the parameters of interest for=20 configuration, control and management by CE. =20 =20 Forwarding Element (FE) Stage -- Representation of an FE block=20 instance in a FE's datapath. As a packet flows through an FE = along a=20 datapath, it flows through one or multiple distinct stages, = with=20 each stage implementing an instance of a certain logical = function=20 block. There may be multiple instances of the same functional = block=20 in a FE's datapath. =20 =20 2. Motivation and Requirements of FE model=20 =20 The ForCES architecture allows Forwarding Elements (FEs) of = varying=20 functionality to participate in a ForCES network element (NE). = The=20 implication of this varying functionality is that CEs can make = only=20 minimal assumptions about the functionality provided by its = FEs. =20 Before CEs can configure and control the forwarding behavior of = FEs,=20 CEs need to query and discover the capabilities and state of = their=20 FEs. [FORCES-REQ] mandates that this capabilities and state=20 information be expressed in the form of an FE model, and this = model=20 will be used as the basis for CEs to control and manipulate = FEs'=20 behavior via ForCES protocol. =20 =20 [FORCES-REQ] describes all the requirements placed on the FE = model=20 in detail. We provide a brief summary here to highlight some of = the=20 design issues we face. =20 . The FE model MUST express what logical functions can be = applied=20 to packets as they pass through an FE.=20 . The FE model MUST be capable of supporting/allowing = variations=20 in the way logical functions are implemented on an FE. =20 . The model MUST be capable of describing the order in which=20 these logical functions are applied in a FE. =20 . The FE model SHOULD be extendable and should have provision = to=20 express new or vendor specific logical functions.=20 . The FE model SHOULD be able to support minimal set of = logical=20 functions that are already identified, such as port = functions,=20 forwarding functions, QoS functions, filtering functions, = high- touch functions, security functions, vendor-specific = functions=20 and off-loaded functions. =20 =20 3. Capability (Configuration) Model versus State Model=20 =20 Since the motivation of an FE model is to allow CEs later to = control=20 and configure FEs' behavior via ForCES protocol, it becomes=20 essential to examine and understand what kind of control and=20 configuration CEs might do to FEs. It is also equally essential = to=20 understand how configurable or programmable FEs are today and = will=20 be in the near future. To understand the issue better, it is = helpful=20 to make a distinction between two different kinds of FE models = =96 FE=20 state model and FE capability (or configuration) model. =20 =20 FE state model describes the current state of the FE, that is, = the=20 instantaneous values or operational behavior of the FE. FE = state=20 =20 Yang, et. al. Expires May 2003 [Page 3] = =0C Internet Draft ForCES FE Functional Model Nov = 2002=20 =20 =20 model presents the snapshot view of the FE to the CE. FE = capability=20 model describes the configurable capabilities of an FE in terms = of=20 variations of functions supported or limitations contained.=20 Conceptually FE capability model presents the many possible = states=20 allowed on an FE. FE capability model allows for more dynamic = and=20 intelligent control of an FE by the CE. =20 =20 For example, using the FE state model, an FE may be described = to its=20 CE as the following: =20 - on a given port the packets are classified using a given=20 classification filter;=20 - the given classifier results in packets being metered in a = certain=20 way, and then marked in a certain way;=20 - the packets coming from specific markers are delivered into a = shared queue for handling, while other packets are delivered to = a=20 different queue;=20 - a specific scheduler with specific behavior and parameters = will=20 service these collected queues.=20 =20 On the other hand, the capabilities model describes the FE at = the=20 coarsest level such aspects as:=20 - this FE can handle IPv4 and IPv6 forwarding;=20 - this FE can perform classification on the following fields: = source=20 IP address, destination IP address, source port number, = destination=20 port number, etc;=20 - this FE can perform metering;=20 - this FE can handle up to N queues;=20 - this FE can add and remove encapsulating headers of types=20 including IPSec, GRE, L2TP.=20 =20 Where it gets more complicated is for the capability model to = cope=20 with the detailed limits, issues such as how many classifiers = the FE=20 can handle, how many queues, and how many buffer pools the FE = can=20 support, how many meters the FE can provide? There is also the = issue=20 as how flexibly these various functions can be interconnected = within=20 the FE? In another word, how programmable the FE really can be = and=20 how the FE capability model can reflect that?=20 =20 While one could try to build an object model for representing=20 capabilities in full, other efforts have found this to be a=20 significant undertaking. A middle of the road approach is to = define=20 coarse-grained capabilities and simple capacity measures. = Then, if=20 the CE attempts to instruct the FE to set up some specific = behavior=20 it is not capable of, the FE will return an error indicating = the=20 problem.=20 =20 It is clear that in the context of ForCES, a state model is=20 definitely necessary. The question is how much of the = capability=20 model is needed in addition to the state model. A simple state = model=20 without any capability flavor will severely limit ForCES=92s = ability=20 to take advantage of the flexibility offered by programmable = FEs. On=20 the other hand, an all too powerful capability model is = difficult to=20 =20 Yang, et. al. Expires May 2003 [Page 4] = =0C Internet Draft ForCES FE Functional Model Nov = 2002=20 =20 =20 develop and may impose unnecessary overhead for most of the FEs = that=20 only offer static functionalities. =20 =20 In order to strike a good balance, it is necessary to examine = the=20 kinds of control and configuration that CEs may do to FEs. =20 =20 The first kind of control and configuration is the simplest of = all.=20 It assumes that the logical functions that an FE supports are=20 already given and the interconnection of these functions = remains=20 static in its lifetime. Therefore, the CE can only control = FE=92s=20 behavior by manipulating the parameters for each individual=20 function, but it cannot change either the datapath or the = functions=20 along each datapath. We call this "static FE" control and=20 configuration. For example, Figure 2 and 3 each shows an FE=20 configuration example by representing the processing steps in a = directed graph interconnecting all the functional stages that=20 packets can possibly traverse. If such a configuration remains=20 static during FE's lifetime, then all CE can control is the=20 parameters associated with each stage in the graph, for = example, the=20 routing table in the LPM forwarder in Figure 2, or the token = bucket=20 parameters associated with meter1 in Figure 3. However, CE = cannot=20 reconfigure the graph topology dynamically, such as adding = another=20 meter or queue onto the FE in Figure 3 on the fly. For this = kind of=20 static control and configuration purpose, the useful FE model = should=20 describe how the graph is connected and what are the =93dials = and=20 knobs=94 (i.e., the parameters or attributes) each function = allows CE=20 to manipulate. It should also include the statistics and events = that=20 FEs can collect and report to CEs. Even for such a =93static = FE=94, some=20 capability model at the individual functions level may be = desirable=20 to convey the flexibility of each function. However, a lot of = other=20 information may not be necessary, like the packet formats = supported=20 between meter1 and counter1 in Figure 3 as an example, because = such=20 information is only useful when the graph can be re-configured=20 dynamically on the fly. =20 =20 The second kinds of control and configuration builds on top of = the=20 first kind just described above. Using Figure 3 as an example,=20 instead of presenting the static FE graph to the CE, the FE can = convey its capabilities to the CE by telling "this FE can = support=20 one classifier with up to N filters. This FE can also support M = meters, X queues, etc." We call this dynamic FE control and=20 configuration. For such dynamic control and configuration, a = more=20 powerful and flexible FE capability model is required. For = example,=20 it becomes necessary to model not only the capability of the=20 building blocks like classifiers, filters, meters etc., but = also the=20 linkage flexibility and constraints between the blocks, so that = CE=20 can have the intelligence to build a dynamic FE graph that = makes=20 sense.=20 =20 The third level of control and configuration is even more = powerful=20 and future looking. In addition to dynamic configuration, CEs = might=20 even be allowed to download a given functionality onto FEs at = run=20 time. This is similar to the active network concept and so we = call=20 =20 Yang, et. al. Expires May 2003 [Page 5] = =0C Internet Draft ForCES FE Functional Model Nov = 2002=20 =20 =20 it active FE control and configuration. Like active network, = this=20 still requires lots of research and experiment. =20 =20 The FE model proposed in this document intends to fully support = the=20 static FE control and configuration at the minimum. It is also = our=20 intention to allow dynamic FE control and configuration at a = certain=20 degree when it makes sense. This FE model currently makes no = attempt=20 to address issues beyond the first two kinds of control and=20 configuration scenarios. =20 =20 4. FE Model=20 =20 This section proposes a ForCES FE model to satisfy all the=20 requirements in [FORCES-REQ] for FE control and configuration. = The=20 approach taken is to model the FE datapath(s) and its packet=20 treatment behavior via a directional graph where each node in = the=20 graph is an instance of a well-defined logical function block. = =20 The FE model defines a generic FE block akin to that of an = abstract=20 base class in object-oriented terminology. The generic FE block = contains basic information like block type and textual = description=20 of the block function. Based on this generic FE block, a set of = well-known FE logical functions are defined with additional = state=20 and capability information pertinent to each specific function. = A=20 name space is used to associate a unique name or ID with each = type=20 of FE block. New logical functions can also be added later to=20 accommodate future innovation in forwarding plane, as long as = the=20 new functions are modeled as an FE block. With such a set of = basic=20 building blocks defined, any FE can be modeled by a directional = graph where each node is an instance of an FE block, = representing a=20 processing stage in the packet datapath. Each node contains=20 information like block name or ID (indicating the block type), = stage=20 ID (local to FE), number of downstream blocks and a list of the = stage IDs of those downstream blocks. =20 =20 The rest of this section is devoted to describe the informal = data=20 model of FE. The description here is intended to be abstract = and=20 conceptual, and examples are used for illustration purpose = only.=20 Separate document(s) will serve as specifications by using a = formal=20 data modeling language and those specifications should be = consistent=20 with the conceptual model described here. =20 =20 4.1. FE Blocks=20 =20 The concept of a generic FE block is the basic building block = of the=20 FE model, like an abstract base class in object-oriented=20 terminology. Actual FE logical functions like classifiers, IPv4 = forwarders and meters are examples of real FE blocks derived = from=20 the generic FE block concept. =20 =20 A well-defined block has a well-defined packet processing = behavior=20 and a well-defined set of state and capabilities that CE can=20 potentially configure or control via ForCES. Sometimes it is=20 =20 Yang, et. al. Expires May 2003 [Page 6] = =0C Internet Draft ForCES FE Functional Model Nov = 2002=20 =20 =20 difficult to decide the granularity of the FE blocks. The = general=20 rule of thumb is that FE blocks should be coarse grained, = stateful=20 and focus on a particular domain. If a logical function does = not=20 have anything to configure, does not keep any kind of = inter-packet=20 state information (tables, etc), then it probably is not a FE = block.=20 For example, CRC check and IP checksum are not considered FE = blocks=20 by themselves, instead should be part of another FE block (i.e. = RFC=20 1812 IP forwarder).=20 =20 Obviously, a namespace is needed to specify different types of=20 blocks. The namespace assigns either a unique ID or label to = each=20 distinct block type. Such a namespace must be extensible so = that new=20 functions can be easily added later. =20 =20 Therefore, the following defines a generic FE Block:=20 - block ID or label which uniquely identifies the block type;=20 - textual description of block function. =20 =20 4.2. FE Block Library=20 =20 We expect a small set of well-understood FE functional blocks = to be=20 defined initially. Such a set of blocks can be viewed as a FE = block=20 library. The minimum set of FE functions required in = [FORCES-REQ]=20 must be part of this library. It is expected that new FE blocks = would be defined and added into this library over time.=20 =20 The actual model for each functional block may differ and = contains=20 information pertinent to the semantics of the function itself.=20 However, some general guideline is still useful. For example,=20 typically it is important to specify information such as:=20 - how many inputs it takes and what kinds of packets and meta = data=20 it takes for each input;=20 - how many outputs it produces and what kind of packets and = meta=20 data it emits for each output;=20 - the packet processing (such as modification) behavior;=20 - what information is programmed into it (e.g., LPM list, next = hop=20 list, WRED parameters, etc.) and what parameters among them are = configurable; =20 - what statistics it keeps (e.g., drop count, CRC error count,=20 etc.); =20 - what events it can throw (e.g., table miss, port down, etc.). = =20 This document only intends to describe the conceptual FE model = and=20 illustrate it with some examples. However, it is not the = intention=20 of this document to define any specific block or the library = itself.=20 Separate document(s) would be written to do that. The minimum = set of=20 FE functions required in [FORCES-REQ] is listed and discussed=20 briefly in the following subsections. The IETF DiffServ=20 (Differentiated Services) and RAP (Resource Allocation = Protocols)=20 working groups have done some relevant work in modeling the=20 provisioning policy data for QoS functions and filtering = functions.=20 Therefore, we will start our discussion from these related = models.=20 =20 =20 Yang, et. al. Expires May 2003 [Page 7] = =0C Internet Draft ForCES FE Functional Model Nov = 2002=20 =20 =20 4.2.1. QoS Functions=20 =20 The IETF DiffServ working group has defined an informal data = model=20 [RFC3290] for QoS-related functions like classification, = metering,=20 marking, actions of marking, dropping, counting and = multiplexing,=20 queueing, etc. The latest work on DiffServ PIB (Policy = Information=20 Base) [DS-PIB] defines a set of provisioning classes to provide = policy control of resources implementing the Diferentiated = Services=20 Architecture. DiffServ PIB also has an element of capability = flavor=20 in it that can potentially enable more dynamic and intelligent=20 configuration of individual functions and the interconnection = of the=20 functions. =20 =20 We use the classifier defined in [RFC3290] as an example to=20 illustrate the DiffServ model. "Classifiers are 1:N (fan-out)=20 devices: they take a single traffic stream as input and = generate N=20 logically separate traffic streams as output. Classifiers are=20 parameterized by filters and output streams. Packets from the = input=20 stream are sorted into various output streams by filters which = match=20 the contents of the packet or possibly match other attributes=20 associated with the packet." To further define filters: "A = filter=20 consists of a set of conditions on the component values of a=20 packet's classification key (the header values, contents, and=20 attributes relevant for classification)." Figure 1 illustrates = an=20 example classifier. =20 =20 Unclassified classified=20 traffic traffic=20 +------------+=20 | |--> match Filter1 --> OutputA=20 ------->| classifier |--> match Filter2 --> OutputB=20 | |--> no match --> OutputC=20 +------------+=20 =20 Figure 1. An Example Classifier Using DiffServ Model=20 =20 Based on this conceptual model, [DS-PIB] specifies a classifier = of=20 1:N by N classifier elements. Each classifier element specifies = the=20 following:=20 - element ID which identifies the particular output out of N;=20 - classifier instance ID which identifies the classifier = instance=20 (all the N classifier elements belong to the same classifier = have=20 the same classifier instance ID);=20 - precedence which is an unsigned integer value to represent = the=20 relative order in which classifier elements are applied (the=20 classifier element with the highest precedence will be matched=20 first);=20 - next datapath element which provides a pointer to the next=20 function along this branch out of N fan-out;=20 - filter ID which points to the filter used for this branch = (Note=20 that filter is defined independent of the classifier and used = here=20 as a parameter to the classifier).=20 =20 Yang, et. al. Expires May 2003 [Page 8] = =0C Internet Draft ForCES FE Functional Model Nov = 2002=20 =20 =20 =20 It is clear from the example above that DiffServ model uses a=20 topological approach to capture the multiple datapath a packet = can=20 potentially take. Graphically, a classifier of 1:N has N output = branches leading to the next N datapath elements. This has=20 significant implication when we consider the interconnected = graph of=20 the functions on FE (see Section 4.3). The alternative is to = use an=20 encoded state approach where each packet gets some state = information=20 associated with it that indicates the datapath it takes next. = For=20 example, using the encoded state approach, a classifier of 1:N = may=20 be represented by just one output branch, if all N of the next=20 datapath elements are of the same block function, say, shaper.=20 Section 4.3 has more discussion on this topic. =20 =20 [DS-PIB] also defines a capability model for classifiers by=20 specifying a bit set to indicate the ability to classify based = on IP=20 source address, IP destination address, IP protocol numbers, IP = DSCP=20 field, layer 4 port number for UDP and TCP, and Ipv6 flow ID. = The=20 capability is thus made known by simply setting the bits=20 accordingly. Similar technique is also used to indicate = capabilities=20 of other functions like meters, droppers, etc.=20 =20 4.2.2. Generic Filtering Functions=20 =20 The framework PIB ([FRMWK-PIB]) from the IETF RAP (Resource=20 Allocation Protocol) working group defines four groups of PRCs=20 (Provisioning Classes) that are expected to be common to all = clients=20 that provision policy using COPS-PR ([RFC3084]). One of the = four PRC=20 groups is classifier group, which contains the Base Filter = Class and=20 the other extended filters including the IP Filter, the IEEE = 802=20 Filter and the Internal Label Filter. Even if SPPI ([RFC3159]) = is=20 not the final chosen data model for our FE model, it may still = be=20 valuable to use the work done here as a starting point for the=20 generic filter functions modeling.=20 =20 4.2.3. Vendor Specific Functions=20 =20 New and currently unknown FE functionality can be derived = (i.e.,=20 extended) based on the generic FE Block. The name space used to = identify the FE block type must be extensible such that new = logical=20 functions can be defined and added later to accommodate future=20 innovation in forwarding plane, as long as the new functions = are=20 modeled as an FE block. =20 =20 4.2.4. Port Functions=20 =20 (TBW)=20 =20 4.2.5. Forwarding Functions=20 =20 (TBW)=20 =20 Yang, et. al. Expires May 2003 [Page 9] = =0C Internet Draft ForCES FE Functional Model Nov = 2002=20 =20 =20 =20 =20 4.2.6. High-Touch Functions=20 =20 (TBW)=20 =20 4.2.7. Security Functions=20 =20 (TBW)=20 =20 4.2.8. Off-loaded Functions=20 =20 (TBW)=20 =20 4.3. FE Stage and Directed Graph of FE =20 =20 With a set of basic FE functions defined in the block library, = we=20 are ready to model any FE=92s packet processing behavior by a=20 directional graph where each node is an instance of an FE = block,=20 representing a processing stage in the packet datapath. This = section=20 describes the details behind such a =93directed graph=94 FE = model.=20 =20 4.3.1. Basic Concepts=20 =20 An FE stage is simply an instance of an FE block within an FE's = datapath. As a packet flows through an FE along a datapath, it = flows=20 through one or multiple distinct stages, with each stage=20 instantiating a certain FE logical function. Each FE allocates = an=20 FE-unique stage ID to each of its stages and passes the stage = ID=20 along with the corresponding block type as part of the FE stage = information. This allows multiple instances of the same block=20 present in a FE's datapath. Using NAT as an example, one NAT=20 function is typically performed before the forwarding stage = (packets=20 arriving externally have their public addresses replaced with=20 private addresses) and one NAT function is performed after (for = packets exiting the domain, their private addresses are = replaced by=20 public ones). So there are three stages (NAT, forwarding, and = NAT=20 again) in this example datapath, with two NAT instances present = in=20 two different stages. =20 =20 A static FE can be modeled by a directed graph interconnecting = all=20 the stages present in the FE. Each node in the graph = corresponds to=20 a stage. In order to represent the directed interconnection = between=20 two consecutive stages along a datapath, each stage contains a = =93next=20 stage=94 pointer that is simply the stage ID of its next stage = in the=20 graph. Therefore, the following defines an FE stage (i.e., a = node in=20 the FE gragh):=20 - stage identifier which uniquely identifies the node within = this FE=20 graph;=20 - block type which identifies the block function that this = stage is=20 an instance of;=20 - number of downstream stages which corresponds to the number = of=20 downstream nodes connected to this stage;=20 =20 Yang, et. al. Expires May 2003 [Page 10] = =0C Internet Draft ForCES FE Functional Model Nov = 2002=20 =20 =20 - downstream stage identifiers which corresponds to the set of=20 downstream nodes connected to this stage.=20 =20 With such information defined for each FE stage, it is now = possible=20 for CE to query the state of the static FE graph by querying = for the=20 initial (ingress) stages of the graph and then traversing the = whole=20 graph in a node-by-node fashion. =20 =20 4.3.2. Topological versus Encoded State Approaches=20 =20 As pointed out in Section 4.2.1, there are potentially two = different=20 approaches to model the nodes and the connections between the = nodes=20 in the FE graph. Using the topological approach as exemplified = by=20 DiffServ model, there are N connections between a fan-out node = of=20 1:N (e.g., a classifier) and its next stages. Using the encoded = state approach, fewer connections are typically needed between = the=20 same fan-out node and its next stages, because each packet = carries=20 some state information as metadata that the next stage nodes = can=20 interpret and invoke different packet treatment. Pure = topological=20 approaches can be overly complex to represent because they = force on=20 to build elaborate topologies with a lot more connections. An=20 encoded state approach is nicer in that it allows one to = simplify=20 the graph and represent the functional blocks with more = clarity. =20 =20 For example in Figure 2, stage #2 (IPv4 L3 LPM Forwarder) = generates=20 some metadata at its output to carry information on which port = the=20 packets should go to, and #3 (Enet-Egress-port-Manager) uses = this=20 meta data to direct the packets to the right egress port. = Figure 3=20 shows how the FE graph looks like when using the topological=20 approach instead, assuming 6 ingress and 4 egress ports. It is = clear=20 that Figure 3 is unwieldy comparing to Figure 2.=20 =20 Note that the FE graph can represent largely arbitrary = topologies of=20 the stages, regardless which approach (topological or encoded = state)=20 is taken. The only restrictions on topology relate to the = source and=20 sink nature of ingress and egress port functions respectively. = For=20 example, egress port functions must not have any downstream = stages=20 whereas no other stage may refer to an ingress port function as = one=20 of its downstream stages. =20 =20 +------------+ +------------+ +------------+=20 input | Ethernet | | | | Ethernet = |output=20 ------->| Ingress |-->| IPv4 L3 LPM|-->| Egress = |----->=20 | Port Mgr | | Forwarder | | Port Mgr |=20 +------------+ +------------+ +------------+=20 =20 {stage ID=3D1, {stage ID=3D2, {stage = ID=3D3,=20 type=3D type=3D type=3D=20 Enet-IngP-Mgr, IPv4-L3-LPM-fwd, = Enet-EgP-Mgr,=20 #downstream=3D1, #downstream=3D1, = #downstream=3D1,=20 downstream=3D{2} downstream=3D{3} = downstream=3Dnone=20 } } }=20 =20 =20 Yang, et. al. Expires May 2003 [Page 11] = =0C Internet Draft ForCES FE Functional Model Nov = 2002=20 =20 =20 Figure 2. A simple example of an FE graph using encoded = state=20 approach.=20 =20 =20 Input +------------+ +------------+ = output=20 ------->|Ingr-Port #1|-->| |=20 +------------+ | | +------------+=20 ------->|Ingr-Port #2|-->| = |-->|EgressPort#1|----->=20 +------------+ | | +------------+=20 ------->|Ingr-Port #3|-->|IPv4 L3 LPM = |-->|EgressPort#2|----->=20 +------------+ |Forwarder | +------------+=20 ------->|Ingr-Port #4|-->| = |-->|EgressPort#3|----->=20 +------------+ | | +------------+=20 ------->|Ingr-Port #5|-->| = |-->|EgressPort#4|----->=20 +------------+ | | +------------+=20 ------->|Ingr-Port #6|-->| |=20 +------------+ +------------+ =20 =20 {stage ID=3D1 {stage ID=3D7, {stage = ID=3D8,=20 type=3D type=3D type=3D=20 Enet-Ing-port, IPv4-L3-LPM-fwd, = Enet-Eg-port,=20 #downstream=3D1, #downstream=3D4, = #downstream=3D1,=20 downstream=3D{7} downstream=3D = downstream=3Dnone=20 } {8,9,10,11} }=20 . . . } . . .=20 {stage ID=3D6 {stage = ID=3D11,=20 type=3D type=3D=20 Enet-Ing-port, = Enet-Eg-port-Mgr,=20 #downstream=3D1, = #downstream=3D1,=20 downstream=3D{7} = downstream=3Dnone=20 } }=20 =20 Figure 3. The same example as in Figure 2 using = topological=20 approach.=20 =20 =20 Even with the state encoded approach, there might still be more = than=20 one datapath within an FE gragh. Figure 4 shows one such FE = block=20 example (based on an example in [RFC3290]), using the DiffServ=20 model. This FE implements QoS functions via combination of one = or=20 multiple instances of logical functions like classifier, meter, = marker, queue, scheduler, etc. The stage labeled "Classifier1" = is a=20 function with four downstream stages, namely, Meter1, Queue2,=20 Meter2, Queue4. If the encoded state approach is taken, it is=20 obvious that the two output branches to Queue2 and Queue4 can = be=20 collapsed into one. However, it is best to leave the branches = to=20 Meter1 and Meter2 as it is, because the packet processing = functions=20 following these two stages are different enough to make them = truly=20 two different datapath. =20 =20 =20 =20 =20 =20 Yang, et. al. Expires May 2003 [Page 12] = =0C Internet Draft ForCES FE Functional Model Nov = 2002=20 =20 =20 Queue1=20 +---+ +--+=20 | A|------------------->| |--+=20 +->| | | | |=20 | | B|--+ +--+ +--+ +--+ |=20 | +---+ | | | | | |=20 | Meter1 +->| |-->| | |=20 | | | | | |=20 | +--+ +--+ |=20 | Counter1 Absolute Queue2| +--+=20 +---+ | Dropper1 +--+ +--->|A |=20 | A|---+ | |------>|B |=20 -------->| B|------------------------------>| | +--->|C = |------>=20 | C|---+ +--+ | +->|D |=20 | X|-+ | | | +--+=20 +---+ | | +---+ +---+ Queue3| | = Scheduler=20 Classifier1 | | | A|------------>|A | +--+ | |=20 | +->| | | |->| |--+ |=20 | | B|--+ +--+ +->|B | | | |=20 | +---+ | | | | +---+ +--+ |=20 | Meter2 +->| |-+ Mux1 |=20 | | | |=20 | +--+ Queue4 |=20 | Marker1 +--+ |=20 +---------------------------->| |----+ =20 | |=20 +--+=20 =20 Figure 4. An FE example with multiple datapath.=20 =20 4.3.3. Cascading FE Blocks=20 =20 An FE block may contain zero, one or more ingress port stages.=20 Similarly, an FE block may contain zero, one or more egress = port=20 stages. In another word, not every FE block has to contain any=20 ingress port or egress port stages. For example, Figure 5 shows = two=20 cascading FE blocks. Block #1 contains one ingress port = function but=20 no egress port function, while block #2 contains one egress = port=20 function but no ingress port function. It is possible to = connect=20 these two FE blocks together to achieve the complete = ingress-to- egress packet processing function. This provides the = flexibility to=20 spread the functions across multiple FEs and interconnect them=20 together later for certain applications. =20 =20 =20 -------------------------------------------------------=20 | +---------+ +------------+ +---------+ |=20 input| | | | | | output |=20 ---+->| Ingress |-->|Header |-->|IPv4 = |---------+--->--+=20 | | port | |Decompressor| |Forwarder| FE | | = | +---------+ +------------+ +---------+ Block #1| | = ------------------------------------------------------| V = | = =20 Yang, et. al. Expires May 2003 [Page 13] = =0C Internet Draft ForCES FE Functional Model Nov = 2002=20 =20 =20 +-----------------------<-----------------------------+ = | =20 | |-----------------------------------------=20 V | +------------+ +----------+ |=20 | input | | | | output | = =20 +->--+->|Header |-->| Egress |---------+-->=20 | |Compressor | | port | FE |=20 | +------------+ +----------+ Block #2|=20 -----------------------------------------|=20 =20 =20 Figure 5. An example of two different FE blocks connected = together.=20 =20 =20 5. Data Modeling and Representation=20 =20 A formal data modeling language is needed to represent the=20 conceptual FE model described in this document and a full=20 specification will be written using such a data modeling = language.=20 It is also necessary to identify a data representation method = for=20 over-the-wire transport of the FE model data. =20 =20 The following is a list of some potential candidates for=20 consideration. For the moment, we intend to leave this as an = open=20 issue and much debate is needed in the ForCES WG before a = decision=20 can be made. Therefore, we only provide the candidate list and = some=20 initial discussion here without drawing a conclusion yet. =20 =20 - XML (Extensible Markup Language) Schema=20 - ASN.1 (Abstract Syntax Notation One)=20 - SMI (Structure of Management Information) [RFC1155]=20 - SPPI (Structure of Policy Provisioning Information) [RFC3159] = - UML (Universal Modeling Language)=20 =20 Most of the candidates here, with the notable exception of UML, = are=20 capable of representing the model in the document and = over-the-wire.=20 Of course, it is also possible to choose one data model = language for=20 specification in the document and later allow several = over-the-wire=20 representations to map the model into different = implementations. =20 =20 XML has the advantage of being human and machine readable with=20 widely available tools support. However, it is very verbose and = hence less efficient for over-the-wire transport. It also = requires=20 XML parsing functions in both the CE and FE and hence may = impose=20 large footprint esp. for FEs. Currently XML is not yet widely=20 deployed and used in network elements. XML for network = configuration=20 in general remains an open area that still requires substantial = investigation and experiment in IETF. =20 =20 ASN.1 format is human readable and widely used in network = protocols.=20 SMI is based on a subset of ASN.1 and used to define Management = Information Base (MIB) for SNMP. SPPI is the adapted subset of = SMI=20 used to define Policy Information Base (PIB) for COPS. = Substantial=20 =20 Yang, et. al. Expires May 2003 [Page 14] = =0C Internet Draft ForCES FE Functional Model Nov = 2002=20 =20 =20 investment has been made in SMI/MIBs/SNMP by IETF and the = Internet=20 community collectively has had many years of design and = operation=20 experience with SMI/MIBs/SNMP. However it is also well = recognized=20 that SMI/MIBs/SNMP is not well suited for configuration and so=20 SPPI/PIBs/COPS-PR attempts to optimize for network provisioning = and=20 configuration. =20 =20 UML is the software industry=92s standard language for = specifying,=20 visualizing, constructing and documenting the artifacts of = software=20 systems. It is a powerful tool for data modeling. However, it = does=20 not provide a data representation format for over-the-wire=20 transport. =20 =20 6. Security Considerations=20 =20 The FE model just describes the representation and organization = of=20 data sets and attributes in the forwarding plane. The = associated=20 communication protocol (i.e., ForCES protocol) will be defined = in=20 separate documents and so the security issues will be addressed = there.=20 =20 7. Intellectual Property Right=20 =20 The authors are not aware of any intellectual property right = issues=20 pertaining to this document.=20 =20 8. IANA consideration=20 =20 =20 If we are going to identify and name a new logical components = we may=20 need to standardize those components and the attributes. =20 =20 =20 9. Normative References=20 =20 [RFC1812] F. Baker, =93Requirements for IP Version 4 Routers", = June=20 1995.=20 =20 [RFC1155] M. Rose, et. al., =93Structure and Identification of=20 Management Informationfor TCP/IP-based Internets", = May=20 1990.=20 =20 [RFC3084] K. Chan, et. al., =93COPS Usage for Policy = Provisioning,=94=20 March 2001.=20 =20 [RFC3159] K. McCloghrie, et. al., =93Structure of Policy = Provisioning=20 Information (SPPI)", August 2001.=20 =20 [RFC3290] Y. Bernet, et. al., =93An Informal Management Model = for=20 Diffserv Routers=94, May 2002.=20 =20 =20 Yang, et. al. Expires May 2003 [Page 15] = =0C Internet Draft ForCES FE Functional Model Nov = 2002=20 =20 =20 10. Informative References=20 =20 [FORCES-REQ] H. Khosravi, et. al., =93Requirements for = Separation of=20 IP Control and Forwarding", work in progress, Oct = 2002,=20 .=20 =20 [DS-PIB] M. Fine, et. al., =93Differentiated Services Quality = of=20 Service Policy Information Base=94, work in = progress, June=20 2002, .=20 =20 [FRMWK-PIB] M. Fine, et. al., =93Framework Policy Information = Base=94,=20 work in progress, June 2002, .=20 =20 11. Acknowledgments=20 =20 The authors would also like to thank the following individuals = for=20 their invaluable technical input: David Putzolu, Hormuzd = Khosravi,=20 Eric Johnson, David Durham, Andrzej Matejko.=20 =20 12. Authors' Addresses=20 =20 Lily L. Yang=20 Intel Labs=20 2111 NE 25th Avenue=20 Hillsboro, OR 97124 USA=20 Phone: +1 503 264 8813=20 Email: lily.l.yang@intel.com=20 =20 Joel Halpern=20 P.O.Box 6049=20 Leesburg, VA 20178=20 Phone: +1 703 371 3043=20 Email: jmh@joelhalpern.com=20 =20 Ram Gopal=20 Nokia Research Center=20 5, Wayside Road,=20 Burlington, MA 01803=20 Phone: +1 781 993 3685=20 Email: ram.gopal@nokia.com=20 =20 Ram Dantu=20 University of Texas Dallas=20 2601 North Flyod Road=20 Richardson Texas 75082=20 Phone: +1 972 883 4653=20 Email: ram.dantu@utdallas.edu=20 =20 Yang, et. al. Expires May 2003 [Page 16] = =0C ------_=_NextPart_000_01C27F9A.8EAED9B0-- From owner-forces@PEACH.EASE.LSOFT.COM Tue Oct 29 17:54:13 2002 Received: from cherry.ease.lsoft.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA00391 for ; Tue, 29 Oct 2002 17:54:12 -0500 (EST) Received: from PEAR.EASE.LSOFT.COM (209.119.0.19) by cherry.ease.lsoft.com (LSMTP for Digital Unix v1.1b) with SMTP id <13.0079A272@cherry.ease.lsoft.com>; Tue, 29 Oct 2002 17:56:34 -0500 Received: from PEACH.EASE.LSOFT.COM by PEACH.EASE.LSOFT.COM (LISTSERV-TCP/IP release 1.8e) with spool id 30392590 for FORCES@PEACH.EASE.LSOFT.COM; Tue, 29 Oct 2002 17:56:33 -0500 Received: from 192.55.52.18 by WALNUT.EASE.LSOFT.COM (SMTPL release 1.0f) with TCP; Tue, 29 Oct 2002 17:56:33 -0400 Received: from talaria.fm.intel.com (talaria.fm.intel.com [10.1.192.39]) by hermes.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.51 2002/09/23 20:43:23 dmccart Exp $) with ESMTP id g9TMsxg08804 for ; Tue, 29 Oct 2002 22:54:59 GMT Received: from fmsmsxv040-1.fm.intel.com (fmsmsxvs040.fm.intel.com [132.233.42.124]) by talaria.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.27 2002/10/16 23:46:59 dmccart Exp $) with SMTP id g9TMxJ308896 for ; Tue, 29 Oct 2002 22:59:19 GMT Received: from FMSMSX016.fm.intel.com ([132.233.42.195]) by fmsmsxv040-1.fm.intel.com (NAVGW 2.5.2.11) with SMTP id M2002102914570428798 for ; Tue, 29 Oct 2002 14:57:04 -0800 Received: by fmsmsx016.fm.intel.com with Internet Mail Service (5.5.2653.19) id ; Tue, 29 Oct 2002 14:56:31 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain Approved-By: "Putzolu, David" Message-ID: Date: Tue, 29 Oct 2002 14:56:26 -0800 Reply-To: Forwarding and Control Element Separation Sender: Forwarding and Control Element Separation From: "Putzolu, David" Subject: ForCES Requirements Last Call - ends Tuesday, November 5, 2002 To: FORCES@PEACH.EASE.LSOFT.COM Precedence: list All, In response to AD & other feedback (appended to this email) the ForCES requirements document has been updated. A new version, draft-ietf-forces-requirements-07.txt, has been uploaded to the drafts repository. URL: http://www.ietf.org/internet-drafts/draft-ietf-forces-requirements-07.txt This email begins a last call for the ForCES requirements document. Given that the changes are focused and localized and that this draft has previously passed last call, this last call is one week in length and ends on Tuesday, November 5th. Please send to the list any suggested changes and/or comments on this document. Thanks, David & Patrick --- AD Feedback --- Section 6.5 Requirement 5) Vendor-Specific Functions The ADs believe that the IESG will be very hesitant to approve this part of the draft as worded. This is due to a concern that vendors would simply use the extensibility mechanisms in the protocol to do everything in a proprietary fashion, including "common" functions. The ADs suggested rewording the section to talk about extensibility to support new, currently unknown functions. Make sure that it is clear that one is *not* compliant with ForCES if one controls all the standard/common functions in a proprietary manner. Section 7 Requirement 6) Reliability The ADs where confused by the mention of non-IP networks. Recommend breaking up the one long reliability paragraph into three paragraphs: Para 1: Talks about IP networks. Essentially the same content. Para 2: Elaborates what non-IP networks are (FastEth, GigE, cell fabric between control and forwarding elements). Para 3: Take the last sentence and expand on it to make clear that the ability to acknowledge the *content* and execution of messages, rather than just their delivery. I.e., TCP may guarantee delivery, but it tells you nothing about whether the forwarding entry add operation succeeded or not. It is the latter function that is required for all types of networks (IP and non-IP). Section 7 Requirement 9) Packet Redirection The ADs where concerned that as written this seems more like a requirement on the FEs/CEs rather than on the protocol. Suggest rewriting it to say that ForCES must define a way to deliver packets from FE's that are destined to the CE (i.e. RIP packets from other routers) and vice versa. This makes it a protocol requirement instead of a CE or FE requirement. --- Other Feedback --- Other feedback based on http://www.ietf.org/internet-drafts/draft-rfc-editor-rfc2223bis-02.txt * Looks like 58 or more lines per page. Reduce this to 55 lines per page as per section 3.1 item 3. * Table of contents should go directly after abstract per section 4. * References should appear before security considerations per section 4 * Make normative and informative references subsections rather than top level sections. --- end ---