From nobody Mon Aug 4 10:32:00 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 89A7D1A0059 for ; Mon, 4 Aug 2014 10:31:59 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -14.502 X-Spam-Level: X-Spam-Status: No, score=-14.502 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9_DJq_rl0tWJ for ; Mon, 4 Aug 2014 10:31:58 -0700 (PDT) Received: from aer-iport-3.cisco.com (aer-iport-3.cisco.com [173.38.203.53]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B9D351A004D for ; Mon, 4 Aug 2014 10:31:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=149; q=dns/txt; s=iport; t=1407173519; x=1408383119; h=message-id:date:from:reply-to:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=dGuJPgr4hRCOYnaVl15KSC3K+FiAv8IT+k9ZOSdQ9Nw=; b=gdyE2JQE/sGPOp9RN4FyQmkNUrkpeTX+I6rogYD8ZvIuvrA9bRdB9RIa SDf/bhE+neJ1MuydC93Tm2/tJonF/IjxsF13/rwo8bDuRdOJYiGCrRI+p WXja1GmYr7wKeDSrSOuzFPGI2dWqPjFKgRHTSp1dQRlRtTEc+oJ0/LVim c=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AlwGAOPC31OtJssW/2dsb2JhbABakWTGVgGBJ3eEBAEBBB0bQAEQCw4TFg8JAwIBAgFFBgEMAQcBAYg+tTGQCBePTAeESwEEnAaHI408g04 X-IronPort-AV: E=Sophos;i="5.01,799,1400025600"; d="scan'208";a="128266547" Received: from aer-iport-nat.cisco.com (HELO aer-core-1.cisco.com) ([173.38.203.22]) by aer-iport-3.cisco.com with ESMTP; 04 Aug 2014 17:31:57 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by aer-core-1.cisco.com (8.14.5/8.14.5) with ESMTP id s74HVt0C029069 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 4 Aug 2014 17:31:55 GMT Received: from [127.0.0.1] (localhost [127.0.0.1]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id s74HVoST007977; Mon, 4 Aug 2014 18:31:51 +0100 (BST) Message-ID: <53DFC387.9020607@cisco.com> Date: Mon, 04 Aug 2014 18:31:51 +0100 From: Stewart Bryant User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Yimin Shen , Alexander Vainshtein References: <201407251524.s6PFOPn92012@magenta.juniper.net> <53D79504.4050904@cisco.com> <04f15bedf2be4c7480d3e9ca01bdfd7f@BY2PR05MB728.namprd05.prod.outlook.com> <2597291f7b074922b690e4b06999cf1a@AM3PR03MB612.eurprd03.prod.outlook.com> <30e13900814f41a681151971cbe9ebef@BY2PR05MB728.namprd05.prod.outlook.com> In-Reply-To: <30e13900814f41a681151971cbe9ebef@BY2PR05MB728.namprd05.prod.outlook.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/czovaD5aFwS_vbWiTKOHJFzMiHs Cc: Yakov Rekhter , "pwe3@ietf.org" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: stbryant@cisco.com List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Aug 2014 17:31:59 -0000 Yimin A question before I reply in detail Does this work over both an LDP signaled MPLS layer and an RSVP-TE signaled MPLS layer? Stewart From nobody Mon Aug 4 10:47:02 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4FCDD1A004D for ; Mon, 4 Aug 2014 10:47:00 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.602 X-Spam-Level: X-Spam-Status: No, score=-2.602 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qKo6BE1XKUJ5 for ; Mon, 4 Aug 2014 10:46:56 -0700 (PDT) Received: from na01-by2-obe.outbound.protection.outlook.com (mail-by2lp0240.outbound.protection.outlook.com [207.46.163.240]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B79CA1A008B for ; Mon, 4 Aug 2014 10:46:55 -0700 (PDT) Received: from BY2PR05MB728.namprd05.prod.outlook.com (10.141.223.25) by BY2PR05MB221.namprd05.prod.outlook.com (10.242.41.154) with Microsoft SMTP Server (TLS) id 15.0.995.14; Mon, 4 Aug 2014 17:46:53 +0000 Received: from BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) by BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) with mapi id 15.00.0995.014; Mon, 4 Aug 2014 17:46:53 +0000 From: Yimin Shen To: "stbryant@cisco.com" , Alexander Vainshtein Thread-Topic: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 Thread-Index: AQHPqByPzRsIO8QlbkK+kbMBZ9MDf5u3AuAAgAAZ0oCAABjewIABFRcAgADf/TCAB5kWgIAAAOTg Date: Mon, 4 Aug 2014 17:46:53 +0000 Message-ID: References: <201407251524.s6PFOPn92012@magenta.juniper.net> <53D79504.4050904@cisco.com> <04f15bedf2be4c7480d3e9ca01bdfd7f@BY2PR05MB728.namprd05.prod.outlook.com> <2597291f7b074922b690e4b06999cf1a@AM3PR03MB612.eurprd03.prod.outlook.com> <30e13900814f41a681151971cbe9ebef@BY2PR05MB728.namprd05.prod.outlook.com> <53DFC387.9020607@cisco.com> In-Reply-To: <53DFC387.9020607@cisco.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [66.129.241.10] x-microsoft-antispam: BCL:0;PCL:0;RULEID: x-forefront-prvs: 0293D40691 x-forefront-antispam-report: SFV:NSPM; SFS:(6009001)(37854004)(377454003)(164054003)(189002)(199002)(13464003)(19580405001)(19580395003)(83322001)(107046002)(33646002)(31966008)(87936001)(77982001)(79102001)(83072002)(85852003)(74502001)(74662001)(92566001)(76482001)(74316001)(2656002)(85306004)(93886004)(76576001)(99396002)(21056001)(50986999)(46102001)(54356999)(76176999)(106356001)(106116001)(20776003)(80022001)(4396001)(81342001)(95666004)(105586002)(99286002)(86362001)(101416001)(64706001)(66066001)(81542001)(24736002)(108616003); DIR:OUT; SFP:; SCL:1; SRVR:BY2PR05MB221; H:BY2PR05MB728.namprd05.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; LANG:en; Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: juniper.net Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/X6C0KyqIm7wKnpVaX-qxS1Xn7MM Cc: Yakov Rekhter , "pwe3@ietf.org" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Aug 2014 17:47:00 -0000 Hi Stewart, Yes, it works with both LDP and RSVP-TE (or other signaling protocols). Thi= s is based on the PWE3 architecture where PWE3 layer and PSN layer are deco= upled. If PSN is MPLS, all we need is an MPLS tunnel to the primary (or pro= tected) PE, and an MPLS bypass tunnel from the PLR to the protector. The tw= o tunnels may be LDP or RSVP-TE (or another protocol). The context ID defin= ed in this draft can be used to facilitate the setup of these tunnels. Thanks, /Yimin -----Original Message----- From: Stewart Bryant [mailto:stbryant@cisco.com]=20 Sent: Monday, August 04, 2014 1:32 PM To: Yimin Shen; Alexander Vainshtein Cc: Yakov Rekhter; pwe3@ietf.org Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin A question before I reply in detail Does this work over both an LDP signaled MPLS layer and an RSVP-TE signaled= MPLS layer? Stewart From nobody Mon Aug 4 21:48:03 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E332B1B28AF for ; Mon, 4 Aug 2014 21:48:00 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.602 X-Spam-Level: X-Spam-Status: No, score=-2.602 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yH3vvH_QInyY for ; Mon, 4 Aug 2014 21:47:58 -0700 (PDT) Received: from emea01-am1-obe.outbound.protection.outlook.com (mail-am1lrp0017.outbound.protection.outlook.com [213.199.154.17]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B87E71B2870 for ; Mon, 4 Aug 2014 21:47:57 -0700 (PDT) Received: from AM3PR03MB612.eurprd03.prod.outlook.com (10.242.110.144) by AM3PR03MB625.eurprd03.prod.outlook.com (10.242.110.156) with Microsoft SMTP Server (TLS) id 15.0.995.14; Tue, 5 Aug 2014 04:47:56 +0000 Received: from AM3PR03MB612.eurprd03.prod.outlook.com (10.242.110.144) by AM3PR03MB612.eurprd03.prod.outlook.com (10.242.110.144) with Microsoft SMTP Server (TLS) id 15.0.995.14; Tue, 5 Aug 2014 04:47:55 +0000 Received: from AM3PR03MB612.eurprd03.prod.outlook.com ([10.242.110.144]) by AM3PR03MB612.eurprd03.prod.outlook.com ([10.242.110.144]) with mapi id 15.00.0995.014; Tue, 5 Aug 2014 04:47:55 +0000 From: Alexander Vainshtein To: Yimin Shen , "stbryant@cisco.com" Thread-Topic: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 Thread-Index: AQHPqymWveYGLBru1kCsLuzIRj9tdJu3CWCAgAA8XgCAAPc/IIABBtiAgAd5y4CAAAQzgIAArHr4 Date: Tue, 5 Aug 2014 04:47:54 +0000 Message-ID: <1407214069986.86488@ecitele.com> References: <201407251524.s6PFOPn92012@magenta.juniper.net> <53D79504.4050904@cisco.com> <04f15bedf2be4c7480d3e9ca01bdfd7f@BY2PR05MB728.namprd05.prod.outlook.com> <2597291f7b074922b690e4b06999cf1a@AM3PR03MB612.eurprd03.prod.outlook.com> <30e13900814f41a681151971cbe9ebef@BY2PR05MB728.namprd05.prod.outlook.com> <53DFC387.9020607@cisco.com>, In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [164.40.145.154] x-microsoft-antispam: BCL:0;PCL:0;RULEID: x-forefront-prvs: 02945962BD x-forefront-antispam-report: SFV:NSPM; SFS:(6009001)(37854004)(377454003)(164054003)(13464003)(199002)(189002)(54356999)(50986999)(99396002)(19580405001)(1941001)(77982001)(4396001)(92566001)(15202345003)(76176999)(83322001)(76482001)(19580395003)(31966008)(2656002)(92726001)(95666004)(79102001)(80022001)(83072002)(106356001)(74502001)(101416001)(86362001)(74662001)(15975445006)(36756003)(20776003)(87936001)(64706001)(81342001)(107046002)(46102001)(21056001)(106116001)(66066001)(85852003)(105586002)(81542001)(85306004); DIR:OUT; SFP:; SCL:1; SRVR:AM3PR03MB612; H:AM3PR03MB612.eurprd03.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; LANG:en; Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Microsoft-Antispam: BCL:0;PCL:0;RULEID: X-OriginatorOrg: ecitele.com Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/X5199cRZJBJJsIX5ICw4W1b825c Cc: Yakov Rekhter , "pwe3@ietf.org" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Aug 2014 04:48:01 -0000 Yimin, Stewart and all,=0A= I have serious doubts regarding the possibility to combine the mechanism de= fined in this draft with LDP-instantiated LSPs as tunnels for PWs.=0A= =0A= Please consider the following use case:=0A= =0A= - The PSN is an MPLS network with a full mesh of tunnel LSPs between all th= e PE that is instantiated between the PEs=0A= - ECMP exists in this network so that actually there are several equal cost= LSPs between PE1 and PE2, possibly with different penultimate hops, say, P= 1 and P2=0A= - The user sets up a PW between PE1 and PE2 and defines PE3 as the "protect= or PE" for PE2 for this PW with some CE dual-homed to PE2 and PE3. =0A= - The user defines the PW in question as a "fat"(a.k.a. flow-aware) PW. As= a consequence some flows in this PW pass thru P1 and some other flows - t= hru P2. However, initially all these flows reach the CE via its link to P= E2=0A= - The link between P1 and PE2 fails. As a consequence the flows within the = fat PW passing thru this link are redirected redirected to PE2. At the same= time, the link between P2 and PE2 remains intact, so that the flows passin= g thru this link are not redirected=0A= - The result is that some of the flows in the PW in question now reach the = CE via its link to PE2 and some - via its link to PE3. Since you've explai= ned (see http://www.ietf.org/mail-archive/web/pwe3/current/msg13879.html) t= hat "CE is dual-home to 2 PEs over 2 distinct ACs", the results could be di= scouraging: e.g., consider the case when you are running an fat Ethernet PW= between two IP-callable devices, and, as the result of the split, IP traff= ic reaches the CE via one AC but ARP - via the other AC.=0A= =0A= I also have doubts regarding ability of LDP to take into account the relati= onship between the initial destination PE and the protector PE. To the best= of my understanding, the only thing LDP would recognize when setting up tu= nnel LSP is the ""Context ID IP address" - presumably an additional loopbac= k address of the original destination PE. I do not see how, without suitabl= e extensions it could identify Protector PE. =0A= =0A= I also suspect that LDP could recognize multiple Context ID addresses as be= longing to the same PE router and as a consequence merge LSPs that use this= IP address as a /32 IPv4 prefix FEC. If this happens, it would break the e= ntire mechanism in the scenarios when the same destination PE participates = in multiple contexts - quite a reasonable situation in the case of MS-PWs w= ith S-PE protection IMO, but also possible in the case of a T-PE that use d= ifferent Protectors for different PWs.=0A= =0A= Did I miss something substantial? =0A= =0A= Regards, =0A= Sasha=0A= ________________________________________=0A= From: Yimin Shen =0A= Sent: Monday, August 4, 2014 8:46 PM=0A= To: stbryant@cisco.com; Alexander Vainshtein=0A= Cc: Yakov Rekhter; pwe3@ietf.org=0A= Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01=0A= =0A= Hi Stewart,=0A= =0A= Yes, it works with both LDP and RSVP-TE (or other signaling protocols). Thi= s is based on the PWE3 architecture where PWE3 layer and PSN layer are deco= upled. If PSN is MPLS, all we need is an MPLS tunnel to the primary (or pro= tected) PE, and an MPLS bypass tunnel from the PLR to the protector. The tw= o tunnels may be LDP or RSVP-TE (or another protocol). The context ID defin= ed in this draft can be used to facilitate the setup of these tunnels.=0A= =0A= =0A= Thanks,=0A= =0A= /Yimin=0A= =0A= =0A= =0A= -----Original Message-----=0A= From: Stewart Bryant [mailto:stbryant@cisco.com]=0A= Sent: Monday, August 04, 2014 1:32 PM=0A= To: Yimin Shen; Alexander Vainshtein=0A= Cc: Yakov Rekhter; pwe3@ietf.org=0A= Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01=0A= =0A= Yimin=0A= =0A= A question before I reply in detail=0A= =0A= Does this work over both an LDP signaled MPLS layer and an RSVP-TE signaled= MPLS layer?=0A= =0A= Stewart=0A= From nobody Mon Aug 4 22:01:27 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 992901B28AF for ; Mon, 4 Aug 2014 22:01:26 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.901 X-Spam-Level: X-Spam-Status: No, score=-1.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6Eo2M_a7JyPI for ; Mon, 4 Aug 2014 22:01:22 -0700 (PDT) Received: from smtp-fr.alcatel-lucent.com (fr-hpida-esg-02.alcatel-lucent.com [135.245.210.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8ED921B2893 for ; Mon, 4 Aug 2014 22:01:22 -0700 (PDT) Received: from fr712usmtp2.zeu.alcatel-lucent.com (unknown [135.239.2.42]) by Websense Email Security Gateway with ESMTPS id 32F71EE572CDF; Tue, 5 Aug 2014 05:01:18 +0000 (GMT) Received: from FR712WXCHHUB03.zeu.alcatel-lucent.com (fr712wxchhub03.zeu.alcatel-lucent.com [135.239.2.74]) by fr712usmtp2.zeu.alcatel-lucent.com (GMO) with ESMTP id s7551Jwc019039 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Tue, 5 Aug 2014 07:01:19 +0200 Received: from FR711WXCHMBA07.zeu.alcatel-lucent.com ([169.254.3.52]) by FR712WXCHHUB03.zeu.alcatel-lucent.com ([135.239.2.74]) with mapi id 14.02.0247.003; Tue, 5 Aug 2014 07:01:19 +0200 From: "Henderickx, Wim (Wim)" To: Alexander Vainshtein , Yimin Shen , "stbryant@cisco.com" Thread-Topic: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 Thread-Index: AQHPqB3e90888u8Qmkyvzr9sufuEaZu3CWCAgAA8XgCAAPc/IIAA62iAgAd5y4CAAAQ0gIAAuK8AgAAlPwA= Date: Tue, 5 Aug 2014 05:01:19 +0000 Message-ID: References: <201407251524.s6PFOPn92012@magenta.juniper.net> <53D79504.4050904@cisco.com> <04f15bedf2be4c7480d3e9ca01bdfd7f@BY2PR05MB728.namprd05.prod.outlook.com> <2597291f7b074922b690e4b06999cf1a@AM3PR03MB612.eurprd03.prod.outlook.com> <30e13900814f41a681151971cbe9ebef@BY2PR05MB728.namprd05.prod.outlook.com> <53DFC387.9020607@cisco.com> <1407214069986.86488@ecitele.com> In-Reply-To: <1407214069986.86488@ecitele.com> Accept-Language: nl-BE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.3.140616 x-originating-ip: [135.239.27.39] Content-Type: text/plain; charset="us-ascii" Content-ID: <951E51B921FD6946A1D8105130467C8C@exchange.lucent.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/o9yqrD3X5cICr6So0sGM90LqL0w Cc: Yakov Rekhter , "pwe3@ietf.org" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Aug 2014 05:01:27 -0000 This solution relies on Ctxt label lookups which means PHP is not supported. The Ctxt Label will determine on which table to lookup the PW label and handle the appropriate action related to this PW. Hope this clarifies On 05/08/14 06:47, "Alexander Vainshtein" wrote: >Yimin, Stewart and all, >I have serious doubts regarding the possibility to combine the mechanism >defined in this draft with LDP-instantiated LSPs as tunnels for PWs. > >Please consider the following use case: > >- The PSN is an MPLS network with a full mesh of tunnel LSPs between all >the PE that is instantiated between the PEs >- ECMP exists in this network so that actually there are several equal >cost LSPs between PE1 and PE2, possibly with different penultimate hops, >say, P1 and P2 >- The user sets up a PW between PE1 and PE2 and defines PE3 as the >"protector PE" for PE2 for this PW with some CE dual-homed to PE2 and >PE3.=20 > - The user defines the PW in question as a "fat"(a.k.a. flow-aware) PW. >As a consequence some flows in this PW pass thru P1 and some other flows >- thru P2. However, initially all these flows reach the CE via its link >to PE2 >- The link between P1 and PE2 fails. As a consequence the flows within >the fat PW passing thru this link are redirected redirected to PE2. At >the same time, the link between P2 and PE2 remains intact, so that the >flows passing thru this link are not redirected >- The result is that some of the flows in the PW in question now reach >the CE via its link to PE2 and some - via its link to PE3. Since you've >explained (see=20 >http://www.ietf.org/mail-archive/web/pwe3/current/msg13879.html) that "CE >is dual-home to 2 PEs over 2 distinct ACs", the results could be >discouraging: e.g., consider the case when you are running an fat >Ethernet PW between two IP-callable devices, and, as the result of the >split, IP traffic reaches the CE via one AC but ARP - via the other AC. > >I also have doubts regarding ability of LDP to take into account the >relationship between the initial destination PE and the protector PE. To >the best of my understanding, the only thing LDP would recognize when >setting up tunnel LSP is the ""Context ID IP address" - presumably an >additional loopback address of the original destination PE. I do not see >how, without suitable extensions it could identify Protector PE. > >I also suspect that LDP could recognize multiple Context ID addresses as >belonging to the same PE router and as a consequence merge LSPs that use >this IP address as a /32 IPv4 prefix FEC. If this happens, it would break >the entire mechanism in the scenarios when the same destination PE >participates in multiple contexts - quite a reasonable situation in the >case of MS-PWs with S-PE protection IMO, but also possible in the case of >a T-PE that use different Protectors for different PWs. > > Did I miss something substantial? > >Regards,=20 >Sasha >________________________________________ >From: Yimin Shen >Sent: Monday, August 4, 2014 8:46 PM >To: stbryant@cisco.com; Alexander Vainshtein >Cc: Yakov Rekhter; pwe3@ietf.org >Subject: RE: [PWE3] WG Last Call for >draft-ietf-pwe3-endpoint-fast-protection-01 > >Hi Stewart, > >Yes, it works with both LDP and RSVP-TE (or other signaling protocols). >This is based on the PWE3 architecture where PWE3 layer and PSN layer are >decoupled. If PSN is MPLS, all we need is an MPLS tunnel to the primary >(or protected) PE, and an MPLS bypass tunnel from the PLR to the >protector. The two tunnels may be LDP or RSVP-TE (or another protocol). >The context ID defined in this draft can be used to facilitate the setup >of these tunnels. > > >Thanks, > >/Yimin > > > >-----Original Message----- >From: Stewart Bryant [mailto:stbryant@cisco.com] >Sent: Monday, August 04, 2014 1:32 PM >To: Yimin Shen; Alexander Vainshtein >Cc: Yakov Rekhter; pwe3@ietf.org >Subject: Re: [PWE3] WG Last Call for >draft-ietf-pwe3-endpoint-fast-protection-01 > >Yimin > >A question before I reply in detail > >Does this work over both an LDP signaled MPLS layer and an RSVP-TE >signaled MPLS layer? > >Stewart > >_______________________________________________ >pwe3 mailing list >pwe3@ietf.org >https://www.ietf.org/mailman/listinfo/pwe3 From nobody Mon Aug 4 22:13:15 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 725CC1A0117 for ; Mon, 4 Aug 2014 22:13:12 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.902 X-Spam-Level: X-Spam-Status: No, score=-1.902 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id D-_CpyK16WbE for ; Mon, 4 Aug 2014 22:13:08 -0700 (PDT) Received: from emea01-db3-obe.outbound.protection.outlook.com (mail-db3lrp0079.outbound.protection.outlook.com [213.199.154.79]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 315B11B2842 for ; Mon, 4 Aug 2014 22:13:07 -0700 (PDT) Received: from AM3PR03MB612.eurprd03.prod.outlook.com (10.242.110.144) by AM3PR03MB611.eurprd03.prod.outlook.com (10.242.109.28) with Microsoft SMTP Server (TLS) id 15.0.995.14; Tue, 5 Aug 2014 05:13:04 +0000 Received: from AM3PR03MB612.eurprd03.prod.outlook.com ([10.242.110.144]) by AM3PR03MB612.eurprd03.prod.outlook.com ([10.242.110.144]) with mapi id 15.00.0995.014; Tue, 5 Aug 2014 05:13:04 +0000 From: Alexander Vainshtein To: Yimin Shen Thread-Topic: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 Thread-Index: AQHPqymWveYGLBru1kCsLuzIRj9tdJu3CWCAgAA8XgCAAPc/IIABBtiAgAg4EiY= Date: Tue, 5 Aug 2014 05:13:03 +0000 Message-ID: <1407215578702.64387@ecitele.com> References: <201407251524.s6PFOPn92012@magenta.juniper.net> <53D79504.4050904@cisco.com> <04f15bedf2be4c7480d3e9ca01bdfd7f@BY2PR05MB728.namprd05.prod.outlook.com> <2597291f7b074922b690e4b06999cf1a@AM3PR03MB612.eurprd03.prod.outlook.com>, <30e13900814f41a681151971cbe9ebef@BY2PR05MB728.namprd05.prod.outlook.com> In-Reply-To: <30e13900814f41a681151971cbe9ebef@BY2PR05MB728.namprd05.prod.outlook.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [164.40.145.154] x-microsoft-antispam: BCL:0;PCL:0;RULEID: x-forefront-prvs: 02945962BD x-forefront-antispam-report: SFV:NSPM; SFS:(6009001)(37854004)(189002)(13464003)(252514010)(199002)(24454002)(51704005)(164054003)(377454003)(52604005)(43784003)(479174003)(20776003)(2656002)(81542001)(110136001)(74502001)(92566001)(19580405001)(4396001)(81342001)(86362001)(46102001)(99396002)(87936001)(74662001)(31966008)(92726001)(36756003)(64706001)(80022001)(77982001)(83072002)(21056001)(106116001)(85852003)(15202345003)(19580395003)(54356999)(79102001)(50986999)(95666004)(106356001)(1941001)(76176999)(101416001)(105586002)(83322001)(561944003)(107046002)(15975445006)(76482001)(66066001)(85306004); DIR:OUT; SFP:; SCL:1; SRVR:AM3PR03MB611; H:AM3PR03MB612.eurprd03.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; LANG:en; Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: ecitele.com Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/AhjojdtvpkIimfHuhUNXI7qMfH8 Cc: Yakov Rekhter , "pwe3@ietf.org" , "stbryant@cisco.com" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Aug 2014 05:13:12 -0000 Yimin hi,=0A= I apologize for missing your response when it has been sent.=0A= =0A= I will try to address some of your comments without yet one more level of i= n-lining:-).=0A= =0A= [Yimin] This draft assumes active-active PWs and bidirectional ACs.=0A= [Sasha] It would be nice if this were explicitly stated in the draft.=0A= =0A= [Yimin] I think you are talking about how to run Layer 3 between CEs with a= n overlay model, which I don't think is in scope of PWE3 architecture=0A= [Sasha] To the best of my understanding PWE3 is about an emulation of a nat= ive service. It does not have to be perfect, but limitations are supposed t= o be explicitly presented in Applicability Statements. On the practical le= vel, if a PW mechanism is not good enough to carry L3 traffic between a pai= r of CEs, what is it good for?=0A= And, last but not least, if CE1 and CE2 in my example were L2 switches runn= ing some flavor of STP, the situation would be no less disruptive.=0A= =0A= [Yimin] In case of egress AC failure, CE2 will detect the failure and start= to send traffic over PW2 (i.e. the backup PW). =0A= [Sasha] This is an assumption about CE2 which is definitely out of scope of= the PWE3 architecture. Actually what you say is that CE2 should treat its = failure to receive traffic over an AC as a bi-directional AC failure. =0A= [Yimin] CE1 should also start to send traffic over the backup PW2. This may= be triggered by many events, e.g. [1] PE1's notification to CE1 based on t= he PW status you mentioned above (but it is not mandatory); =0A= [Sasha] This is possible of course, but, as you've said, not mandatory=0A= [Yimin] or [2] CE1 receiving traffic from PE3. =0A= [Sasha] But you've said that ACs are active-active. So why receiving someth= ing from PE3 would be of any interest?=0A= And of course, there is a case when only CE2 is dual-homed but CE1 is not.= .. Is the draft relevant in this scenario?=0A= [Yimin] In any case, it may take some time for CE1 to do so. Before this ha= ppens on CE1, the traffic is still going over PW1 (primary PW) which is bei= ng locally repaired by PE2.=0A= [Sasha] I'd say that the draft relies on some "dual-homing protocol" betwee= n a dual-homed CE and its PEs to provide for a bi-directional switch, and t= hat such a switch could take some time. I am not sure this counts for "fast= protection".=0A= =0A= Regards,=0A= Sasha=0A= =0A= =0A= =0A= ________________________________________=0A= From: Yimin Shen =0A= Sent: Thursday, July 31, 2014 2:22 AM=0A= To: Alexander Vainshtein=0A= Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org=0A= Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01=0A= =0A= Hi Sasha,=0A= =0A= Thanks again for your comments. Please see inline...=0A= =0A= =0A= Thanks,=0A= =0A= /Yimin=0A= =0A= =0A= =0A= -----Original Message-----=0A= From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com]=0A= Sent: Wednesday, July 30, 2014 4:08 AM=0A= To: Yimin Shen=0A= Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org=0A= Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01=0A= =0A= Yimin hi,=0A= Lots of thanks for a prompt response.=0A= =0A= Unfortunately, from my point of view, the issues I've raised still remain u= nresolved. Please see additional questions inline below.=0A= =0A= Regards,=0A= Sasha=0A= Email: Alexander.Vainshtein@ecitele.com=0A= Mobile: 054-9266302=0A= =0A= > -----Original Message-----=0A= > From: Yimin Shen [mailto:yshen@juniper.net]=0A= > Sent: Tuesday, July 29, 2014 7:56 PM=0A= > To: Alexander Vainshtein; stbryant@cisco.com; Yakov Rekhter;=0A= > pwe3@ietf.org=0A= > Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-=0A= > protection-01=0A= >=0A= > Hi Sasha,=0A= >=0A= > Thanks very much for the review and comments. Please see my responses=0A= > inline. Hope they are helpful.=0A= >=0A= >=0A= > Thanks,=0A= >=0A= > /Yimin=0A= >=0A= >=0A= >=0A= > -----Original Message-----=0A= > From: pwe3 [mailto:pwe3-bounces@ietf.org] On Behalf Of Alexander=0A= > Vainshtein=0A= > Sent: Tuesday, July 29, 2014 10:08 AM=0A= > To: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org=0A= > Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-=0A= > protection-01=0A= >=0A= > Stewart, Yakov and all,=0A= > I have started reading the draft, and it seems that it breaks quite a=0A= > few common architectural assumptions.=0A= >=0A= > [yshen] This draft is fully based on the PWE3 architecture and the=0A= > MS-PW architecture.=0A= >=0A= > The following text from Section 3.1 "Single-Segment PW" looks highly=0A= > problematic to me:=0A= > =0A= > At any given time, each CE sends traffic via only one AC and receives= =0A= > traffic via only one AC. The two ACs MAY or MAY NOT be the same.=0A= > The AC used to send traffic is determined by the CE, and MAY rely on= =0A= > an end-to-end OAM mechanism between the CEs. The AC used for the CE= =0A= > to receive traffic is determined by the state of the network and the= =0A= > protection mechanism in use, as described later in this document.=0A= > .=0A= >=0A= > To the best of my order for the pair of interconnected CEs to be able=0A= > to operate in this way each of them should treat the pair of ACs shown=0A= > in Figure=0A= > 1 as a single logical entity.=0A= > E.g., if we are speaking about Ethernet PWs, each CE could treat=0A= > Ethernet links supporting these two ACs as a single Link Aggregation=0A= > Group. However, the draft does not mention any such thing (or at least I = did not find it).=0A= >=0A= > Do I miss something important here?=0A= >=0A= > [yshen] It is a logical view that the CE is dual-home to 2 PEs over 2=0A= > distinct ACs. AC is a concept well defined in RFC 3985. In terms of=0A= > the actual PE-CE connection in the case of Ethernet, the CE could have=0A= > 2 distinct Ethernet links or 2 VLANs over a single LAG.=0A= >=0A= [[Sasha]] If I read your response correctly, each dual-homed CE treats each= if its two ACs as an individual entity.=0A= I see this is as highly problematic. E.g., consider the case when CE1 and C= E2 are two CE routers running some link-state routing protocol between them= .=0A= - In the normal situation, CE1 probably would see its two ACs as two paral= lel bi-directional links.=0A= - With PW redundancy, one of these links (one that corresponds to the curre= nt Active PW) would be Up and the other one - Down=0A= - With your proposal, the typical link-state routing protocol between CE1 a= nd CE2 would not be able to form any adjacencies across any of these links,= because this process requires bi-directional link connectivity; so regardl= ess of whatever happens in the PW domain CE1 and CE2 would consider each ot= her as unreachable across this domain.=0A= Did I miss something? (I intentionally leave aside protocols like OLSR that= can handle unidirectional links).=0A= =0A= [yshen] I'm lost in the last 2 bullets. This draft assumes active-active PW= s and bidirectional ACs. I think you are talking about how to run Layer 3 b= etween CEs with an overlay model, which I don't think is in scope of PWE3 a= rchitecture.=0A= =0A= =0A= =0A= =0A= =0A= =0A= > I also wonder whether, in the single segment PW scenario, the same=0A= > level of protection could not be achieved with ICCP (RFC 7275)=0A= > running between each pair of PEs and, possibly with 4 (instead of 2)=0A= > PWs. For the reference, ICCP (or RFC 7275) is not mentioned in the=0A= > draft, even if a possible alternative solution to some of the problems=0A= > it solves. (=0A= >=0A= > [yshen] Thanks for pointing out ICCP. We will mention it in the next revi= sion.=0A= > In this draft we broadly refer to such kind of mechanism as "control=0A= > plane repair". The purpose of this draft is to locally repair traffic=0A= > at the failure point, in order to reduce traffic loss before ICCP or=0A= > other control plane repair mechanism kicks in.=0A= =0A= [[Sasha]] I am not sure ICCP in this case would mean slow protection. Its f= unction here would be just to synchronize the state of PWs between PE1 and = PE3/PE2 and PE4, and this can be reasonably fast IMO. The Protection State = Coordination (PSC) protocol in RFC 6378 looks like an analog (with some str= etch of course), and it is supposed to provide pretty fast protection switc= hing.=0A= =0A= [yshen] The draft doesn't comment specifically on ICCP performance. General= ly speaking, local repair mechanism tends to have the least overhead when r= estoring traffic. This is because both failure detection and traffic restor= ation are done right in the dataplane. It doesn't rely on dataplane to cont= rol plane notification (for failure event) or control plane protocol. For t= his exact reason there have been various IP/MPLS fast-reroute mechanisms de= fined by IETF. Of course, in a given scenario where the delay in global rep= air or control plane repair is tolerable for users, they don't have to use = local repair. Local repair is for the case where it is desirable to reduce = traffic loss even further. Note that local repair is a temporary repair, an= d it doesn't prevent global repair or control plane repair to perform a per= manent repair. I believe the draft has specifically mentioned this.=0A= =0A= =0A= =0A= =0A= =0A= >=0A= > Stewart did not specify in his email which MPLS architecture=0A= > principles he thinks are broken by this draft.=0A= >=0A= > I think, however, that some PWE3 architectural principles are=0A= > definitely are broken, and this includes:=0A= > - Bi-directional nature of the PW - it seems that the draft allows=0A= > each a PW be Up in one direction and Down in the other one=0A= >=0A= > [yshen] No, the draft does not allow that. It only says that we can=0A= > send traffic of one direction over one PW, while sending traffic of=0A= > the opposite direction over another PW (i.e. backup PW). Both PWs should = be up bidirectionally.=0A= >=0A= >=0A= [[Sasha]] Sorry but I do not understand that. The draft discusses something= like the "egress AC failure". If, say. PE1 detects that, I expect it woul= d send a PW Status with "PW not Forwarding" and "Local AC Transmit Fault" b= its set to PE2. Would this not mean that the PW between PE1 and PE2 is Down= ?=0A= =0A= [yshen] In case of egress AC failure, CE2 will detect the failure and start= to send traffic over PW2 (i.e. the backup PW). CE1 should also start to se= nd traffic over the backup PW2. This may be triggered by many events, e.g. = [1] PE1's notification to CE1 based on the PW status you mentioned above (b= ut it is not mandatory); or [2] CE1 receiving traffic from PE3. In any case= , it may take some time for CE1 to do so. Before this happens on CE1, the t= raffic is still going over PW1 (primary PW) which is being locally repaired= by PE2.=0A= =0A= =0A= =0A= =0A= =0A= =0A= > - Lack of dependencies between the PSN tunnels and the PWs they carry.=0A= > * Consider the network diagram shown in Figure 5 in Section 4.1=0A= > * The text suggests that the tunnel between PE1 and PE2 must=0A= > recognize PE4 as the "protector" node for the PW it carries=0A= > * What is supposed to happen if the tunnel between PE1 and PE2=0A= > carries, say, 10 PWs, each with its own backup PE PE4-1, ... PE4-10)?=0A= >=0A= > [yshen] In your example, the 10 PWs are protected by 10 different=0A= > protectors. In this case, you would need 10 different tunnels,=0A= > destined to 10 different context IDs (one context ID per PE4-x>). If multiple PWs are protected by the same protector, they can be= carried by the same tunnel.=0A= >=0A= [[Sasha]] This looks like a non-scalable solution to me.=0A= =0A= [yshen] I'm not sure there is any scalability issue here. We basically need= one tunnel per context ID, i.e. per . In reality, t= he number of should not be high. The draft also spe= cifies the centralized model for reducing the number of context IDs.=0A= =0A= =0A= =0A= =0A= =0A= >=0A= > - Lack of dependencies between the mechanism used for the tunnel setup=0A= > (LDP, RSVP-TE, manual configuration etc.) and the PWs this tunnel carries= .=0A= > * Is the protection model described in Section 4.1 compatible with= =0A= > Tunnel LSPs being set up by LDP? With, say, GRE tunnels carrying the=0A= > PWs using MPLS-in-GRE?=0A= >=0A= > [yshen] Yes, it is compatible. The draft is generic on transport=0A= > layer, and it allows tunnels to be LDP, RSVP-TE, GRE, etc. These=0A= > protocols already provide sufficient machineries for setting up transport= tunnels and bypass tunnels.=0A= > Therefore, the draft doesn't require new extension to these LSP=0A= > signaling protocols.=0A= >=0A= [[Sasha]] Sorry but this does not match your response to my previous questi= on/issue where you have explained that you effectively need a tunnel per PW= between PE1 and PE2. RSVP-TE can provide that; but how is LDP supposed to = do that?=0A= =0A= [yshen] Again, a tunnel can carry multiple PWs, if these PWs are protected = by the same protector. You could use 10 different protectors to protect 10 = PWs, but in reality when SP design their network, they should ask why they = have to do so. Regarding LDP, it is one FEC per context ID, and nothing spe= cial.=0A= =0A= =0A= =0A= =0A= > Last but not least, I would like to understand better how the draft is=0A= > protecting MS-PWs against S-PE failures.=0A= > Specifically, I'd like to understand whether it is applicable to the=0A= > case when each pair of T-PEs resides in its own AS, with two pairs of=0A= > directly connected ASBRs between these two ASs acting as S-PEs. There=0A= > would be no need to set up any tunnel LSPs between each pair of S-PEs in = this scenario.=0A= > I would appreciate an analysis of this scenario regarding ability to=0A= > protect about the ASBR/S-PE failure. IMO it is a relevant scenario=0A= > because MS-PWs are one of the methods to provide inter-AS services.=0A= >=0A= > [yshen] The MS-PW topology in this draft is based on the Figure-4 of=0A= > RFC 5659, which is slightly different than the scenario you mentioned abo= ve.=0A= >=0A= [[Sasha]] Figure 4 in RFC 5659 does not say anything about inter-AS.=0A= >=0A= > However, the mechanism in this draft does apply to this scenario as=0A= > well. It should work equally if S-PEs are one-hop or multi-hops away=0A= > from each other. Please see the picture below. In this case, SPE1 ~ SPE4 = are ASBRs.=0A= > When SPE4 detects the failure of SPE3, it will do a local repair and=0A= > redirect PW traffic to SPE4 which is the protector. The bypass between=0A= > SPE1 and SPE4 may be a one-hop tunnel or a multi-hop tunnel, depending=0A= > on the actual topology between the 2 AS'.=0A= >=0A= [[Sasha]] This approach can work if SPE1 is directly connected to SPE4 beca= use in this case there would be no need for a tunnel LSP between them.=0A= I do not see a simple way to set up an LSP between SPE1 (in AS-1) and SPE4(= in AS-2) if such connectivity is not present; could you possibly expand on = how this is supposed to happen?=0A= =0A= [yshen] Sasha, if you don't have a bypass tunnel (due to specific topology = or policy or whatever), you won't get any IP/MPLS fast-reroute mechanism to= work :) However it doesn't mean the mechanism is broken. In your case, I d= on't see any technique issue why you cannot set up a inter-AS bypass from S= PE1-SPE2-SPE4.=0A= =0A= =0A= =0A= >=0A= > In any case, SPE4 will send PW traffic over a=0A= > tunnel to TPE4.=0A= >=0A= > |<--------------- PW1 ----------->|=0A= >=0A= > - TPE1 -------- SPE1 - SPE3 --------- TPE2 -=0A= > / PLR \ \=0A= > / \ \=0A= > CE1 bypass\ CE2=0A= > \ \ /=0A= > \ \ /=0A= > - TPE3 -------- SPE2 - SPE4 --------- TPE4 -=0A= > protector=0A= >=0A= > |<--------------- PW2 ----------->|=0A= >=0A= > My 2c,=0A= > Sasha=0A= > Email: Alexander.Vainshtein@ecitele.com=0A= > Mobile: 054-9266302=0A= >=0A= > > -----Original Message-----=0A= > > From: pwe3 [mailto:pwe3-bounces@ietf.org] On Behalf Of Stewart=0A= > > Bryant=0A= > > Sent: Tuesday, July 29, 2014 3:35 PM=0A= > > To: Yakov Rekhter; pwe3@ietf.org=0A= > > Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-=0A= > > protection-01=0A= > >=0A= > > On 25/07/2014 16:24, Yakov Rekhter wrote:=0A= > > > Hi,=0A= > > >=0A= > > >> This email begins a second two week working group last call for=0A= > > >> http://tools.ietf.org/html/draft-ietf-pwe3-endpoint-fast-protection-= 01 .=0A= > > >>=0A= > > >> Please review the draft and post any comments to the PWE3 list.=0A= > > >>=0A= > > >> If you have read the draft, even if you don't have any comments,=0A= > > >> but would like to see the draft progressed, please can you=0A= > > >> indicate this to the=0A= > > list.=0A= > > >>=0A= > > >> This working group last call will end on Friday August 8th, 2014.=0A= > > > I would like to see the draft progressed.=0A= > > >=0A= > > > Yakov.=0A= > > >=0A= > > > _______________________________________________=0A= > > > pwe3 mailing list=0A= > > > pwe3@ietf.org=0A= > > > https://www.ietf.org/mailman/listinfo/pwe3=0A= > > >=0A= > > Yakov=0A= > >=0A= > > Doesn't this draft implicitly make a number of extensions to the=0A= > > MPLS architecture?=0A= > >=0A= > > Stewart=0A= > >=0A= > > _______________________________________________=0A= > > pwe3 mailing list=0A= > > pwe3@ietf.org=0A= > > https://www.ietf.org/mailman/listinfo/pwe3=0A= >=0A= > _______________________________________________=0A= > pwe3 mailing list=0A= > pwe3@ietf.org=0A= > https://www.ietf.org/mailman/listinfo/pwe3=0A= From nobody Tue Aug 5 01:34:58 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6604D1B28B4 for ; Tue, 5 Aug 2014 01:34:57 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.903 X-Spam-Level: X-Spam-Status: No, score=-106.903 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WYiINIm7ScZp for ; Tue, 5 Aug 2014 01:34:55 -0700 (PDT) Received: from rfc-editor.org (rfc-editor.org [4.31.198.49]) by ietfa.amsl.com (Postfix) with ESMTP id 78E391B289D for ; Tue, 5 Aug 2014 01:34:55 -0700 (PDT) Received: by rfc-editor.org (Postfix, from userid 30) id 4515C180095; Tue, 5 Aug 2014 01:33:31 -0700 (PDT) To: thomas.nadeau@bt.com, davidz@oversi.com, brian@innovationslab.net, ted.lemon@nominum.com, matthew.bocci@alcatel-lucent.com, agmalis@gmail.com X-PHP-Originating-Script: 6000:errata_mail_lib.php From: RFC Errata System Message-Id: <20140805083331.4515C180095@rfc-editor.org> Date: Tue, 5 Aug 2014 01:33:31 -0700 (PDT) Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/wRGqmSfKXXM8Se6zzfkyDpbG0Vw Cc: pwe3@ietf.org, rfc-editor@rfc-editor.org Subject: [PWE3] [Technical Errata Reported] RFC5601 (4069) X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Aug 2014 08:34:57 -0000 The following errata report has been submitted for RFC5601, "Pseudowire (PW) Management Information Base (MIB)". -------------------------------------- You may review the report below and at: http://www.rfc-editor.org/errata_search.php?rfc=5601&eid=4069 -------------------------------------- Type: Technical Reported by: Alexander ("Sasha") Vainshtein Section: 12 Original Text ------------- pwAttachedPwIndex OBJECT-TYPE SYNTAX PwIndexOrZeroType MAX-ACCESS read-create STATUS current DESCRIPTION "If the PW is attached to another PW instead of a local native service, this item indicates the pwIndex of the attached PW. Otherwise, this object MUST be set to zero. Attachment to another PW will have no PW specific entry in any of the service MIB modules." DEFVAL { 0 } ::= { pwEntry 10 } Corrected Text -------------- pwAttachedPwIndex OBJECT-TYPE SYNTAX PwIndexOrZeroType MAX-ACCESS read-create STATUS current DESCRIPTION "If the PW is attached to another PW instead of a local native service, this item indicates the pwIndex of the attached PW. Otherwise, this object MUST be set to zero. Attachment to another PW will have no PW specific entry in any of the service MIB modules. With regard to this attribute read-create means that its value can be changed when the containing row in the pwTable represents an inactive PW." DEFVAL { 0 } ::= { pwEntry 10 } Notes ----- Description of the pwEntry object in the same RFC states that "The read-create objects in this table are divided into three categories: 1) Objects that MUST NOT be changed after row activation. These are objects that define basic properties of the PW (for example type, destination, etc.). 2) Objects that MAY be changed when the PW is defined as not active. A change of these objects involves re-signaling of the PW or it might be traffic affecting. PW not active is defined as one of the following conditions: a) The pwRowStatus is notInService(2). b) The pwRowStatus is notReady(3). c) The pwAdminStatus is down(2). If the operator needs to change one of the values for an active row, the operator can either set the pwRowStatus to notInService(2) or set pwAdminStatus to down(2). Signaling (or traffic) is initiated again upon setting the pwRowStatus to active(1) or setting the pwAdminStatus to up(1) or testing(3), respectively. 3) Objects that MAY be changed at any time." In further states (in tthe same description) that "By default, all the read-create objects MUST NOT be changed after row activation, unless specifically indicated in the individual object description." pwAttachedPwIndex object is used to stitch a couple of PWs represented by two different rows in the pwTable, with the pwAttachedPwIndex value in the row representing one of them set to the pwIndex of the other one and vice versa. Since there is no way in the SMIv2 paradigm to create two rows in the same table in a single atomic operation, setting a this attribute in a pair of rows is only possible when both are created. In order to do that, read-create access mode of the pwAttachedPwIndex object has to be interpreted as ability to set its value when the row represents an inactive PW. In accordance with the quoted description, such an interpretation must be explicitly specified in the description of this object. Such a specification is missing in the current text, hence the default interpretation of the read-create access mode is holds. Proposed text fixes this problem. Instructions: ------------- This erratum is currently posted as "Reported". If necessary, please use "Reply All" to discuss whether it should be verified or rejected. When a decision is reached, the verifying party (IESG) can log in to change the status and edit the report, if necessary. -------------------------------------- RFC5601 (draft-ietf-pwe3-pw-mib-14) -------------------------------------- Title : Pseudowire (PW) Management Information Base (MIB) Publication Date : July 2009 Author(s) : T. Nadeau, Ed., D. Zelig, Ed. Category : PROPOSED STANDARD Source : Pseudo Wire Emulation Edge to Edge INT Area : Internet Stream : IETF Verifying Party : IESG From nobody Tue Aug 5 03:56:49 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3C2391B2951; Tue, 5 Aug 2014 03:56:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -14.502 X-Spam-Level: X-Spam-Status: No, score=-14.502 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Rako0jRyObZ7; Tue, 5 Aug 2014 03:56:38 -0700 (PDT) Received: from aer-iport-4.cisco.com (aer-iport-4.cisco.com [173.38.203.54]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 820001B2949; Tue, 5 Aug 2014 03:56:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=77147; q=dns/txt; s=iport; t=1407236193; x=1408445793; h=message-id:date:from:reply-to:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=uUWtCVdFp0m1LcucDwNkBmPFfMoE6tMl4GXjl/OSTGw=; b=DtGmlUg3tuAYIxtFwl8MVDPf8t/9mVg7KoENuaPwBavm2Cnx0B9mEb7M S7tb1jcibGKUJbNzPvth66reMRl58nEIbyK4i492HzIRkRoDwhcAQIvqs N436Fj8dtHH64g+mHGvtWIfPvqhhLDGYJYWFh/RMa/ucBv1pWK4NRutkL k=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AtgEAG+34FOtJssW/2dsb2JhbABSCYNfV4J4yHQKh0oBgSp3hAMBAQEDAQEBARcBAgYEES8GAQoBBQcECw4DAQMBAQECAgUWCAMCAgkDAgECARUfAwYIBgEMAQUCAQEXiB8IDawFhn+QIheBLIhThGIQAwEkER0FBwaCc4FSAQSGCZFUhCyHI408g05rAYEDAQEeBAI X-IronPort-AV: E=Sophos;i="5.01,804,1400025600"; d="scan'208";a="128958708" Received: from aer-iport-nat.cisco.com (HELO aer-core-4.cisco.com) ([173.38.203.22]) by aer-iport-4.cisco.com with ESMTP; 05 Aug 2014 10:56:28 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by aer-core-4.cisco.com (8.14.5/8.14.5) with ESMTP id s75AuRt3020736 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 5 Aug 2014 10:56:27 GMT Received: from [127.0.0.1] (localhost [127.0.0.1]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id s75AuQkf010009; Tue, 5 Aug 2014 11:56:26 +0100 (BST) Message-ID: <53E0B85B.8060707@cisco.com> Date: Tue, 05 Aug 2014 11:56:27 +0100 From: Stewart Bryant User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Yimin Shen , pwe3 , "pwe3-chairs@tools.ietf.org" References: <53D7B569.60400@cisco.com> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/-fCyHTCXfmYrZVId4erXf1rsBLM Cc: "mpls@ietf.org" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: stbryant@cisco.com List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Aug 2014 10:56:48 -0000 On 30/07/2014 20:07, Yimin Shen wrote: > Hi Stewart, > > Thanks very much for the detailed review and comments. I'd like to highlight a few things below about this draft, and then respond to your comments inline. Hope you could give the draft another closer look. Yimin, Having read your reply and having looked at the text again, and talked to a few folks I think I now understand the top level view of what is proposed. However that is in itself an indication that the draft needs considerable work before publication. Note that it was not initially obvious that you supported both LDP and RSVP-TE PSNs, nor was it initially obvious that you were using the PSN to protect the PWs and not looking further down the stack. Indeed I think there is some text in the draft that positively leads you in that direction since someone else picked it it during an MPLS review. To summarize: what you are doing is construct repairs at the PSN layer that map to each combination of PW + PW repair, which I think multiples the number of PSN tunnels by three times the dispersion factor where the dispersion factor is some factor defining the average number of PEs that the PWs addressed to a single PE need to be dispersed to. This leads to two questions that the WG needs to think about: 1) This is trading increased PSN state for response time. Given that PSN state was a critical factor in the choice of the Martini design over the CCC design this needs to be made clear up front in the text and the WG needs to decide that this is acceptable. 2) This increased state and the complexity of introducing context labels and protection routers to the PWE3 architecture is directly traceable to the need to do xPE node protection, and is not required for link protection including the egress AC. This desire to build the design around node protection (which always adds significant complexity to FRR) is in contrast to the IPFRR situation where link protection is of overwhelming importance and node protection seems to be of secondary importance. An alternative design, which would require less change to the existing PWE3 design would be to provide redundant connection to the xPEs and allow FRR link protection in the PSN to deal with link failures and then to protect the egress AC by making the T-PE a special type of S-PE that preferred to act as a T-PE but if the AC was down forwarded to an alternate T-PE for that PW. If the WG prefers to introduce the cross layer scheme you have proposed, then fine, we need to do the full set of updates needed. However first I think that we need to have rough consensus that the use cases and requirements justify the increased complexity compared of node protection compared to the alternative of providing link and AC protection. > > [1] This draft is completely based on the PWE3 and the MS-PW architecture. It is also based on RFC 5331 " MPLS Upstream Label Assignment and Context-Specific Label Space". So ideally readers should be familiar with that RFC. As far as I can see context labels have not been introduced into the PWE3 architecture. There is some reference to their use for P2MP PWs, but not in the P2P case. Indeed I think that RFC4447 notes explicitly that in the case where the configuration of PWs is signaled by LDP the platform label space must be used. The least that you need to do is to update RFC4447. Something that I noticed in a second pass through the text is that you seem to have changed the PW FEC data structure from the one specified by PWE3 to a variant of one used in LSP Ping. This seems unnecessary and may lead to confusion and difficulty if these get changed at some time in the future. A better design is surely to leave the PWE3 signalling system completely unchanged except to add an additional TLV specifying the context information if indeed we continue on that path. Apropos this, it may be the way that the document was written, but I did not see anything about the need to signal the PW interface parameters to the alternative PE. Also I did not see anything specifying the error handling as a result of parameter issues. A comment below that was not picked up is that the signalling design supports IPv4 only, and as I recall there is an IESG policy of not standardizing any IPv4 only solutions, so perhaps we should introduce IPv6 at the same time as harmonizing the normal and context label data structures. The LDP signalling of PWs with context labels need a more complete review for both completeness and correctness and to ensure that this text provides a description that is sufficiently complete that this new feature can be implemented solely from this text and the associated references. > > [2] The draft mentions two important roles of routers, PLR (point of local repair) and protector. You need a clearer definition of a PLR and protector. I think that a protector is a new type of SPE and needs a clearer architectural definition. The concept of a PLR taking action at one layer within the context of another is new in PWE3. Perhaps it is defined in MPLS in which case a reference would be appropriate. > > [3] PLR performs local repair in dataplane, based on pre-installed backup forwarding state pointing to a bypass tunnel. There is no control-plane involvement in local repair. Specifically in the initiation of the local repair, it is in the configuration of course. > This is similar to the existing IP/MPLS fast-reroute mechanisms. Therefore, it can achieve restore traffic with a similar performance to those mechanisms. That is the main goal of this draft. IP/MPLS FRR normally operates solely for the benefit of its own dataplane. You have this configured to act for the benefit of the client dataplane. > > [4] When doing local repair, the PLR only swaps the top label (i.e. tunnel label) to a bypass tunnel's label, and keeps the PW label (allocated by primary/protected PE) intact. The PLR doesn't need to know or understand the PW label. Indeed this is clearer now, and needs to be made much clearer in the text. However so do the implications for the system as a whole of this mode of operation. > > [5] The protector is the router at the other end of the bypass tunnel. It is responsible for forwarding the traffic further, so that the traffic can eventually reach the target CE. The protector must understand the PW label allocated by the primary PE. The use of the work primary PE is slightly confusing. Presumably you mean the ingress PE of the segment rather than the ingress PE from the point of view of the ingress AC. > This is achieved by using the context label forwarding paradigm specified in RFC 5331. This draft defines some LDP extensions to allow the primary PE to advertise PW labels to the protector. The protector locally maintains a label space for these PW labels of the protected PE, and looks up the PW label in this label space. The bypass tunnel serves as the pointer to this label space. Now I understand that. Please make this much clearer earlier in the text. Note my earlier concern about the design of those extensions. > > [6] This draft also introduces the concept of "context ID". A context ID is an IP address identifying a pair of . The use of an IP address rather than some identifier taken from the PW context is a design assumption not properly shared with the reader and needs to be clarified in the text. > It serves as the destination of transport tunnel. PWs carried by the tunnel will be redirected to the protector X during local repair. PWs to the same primary PE but protected by a different protector Y cannot be carried by this tunnel. Rather, they must be carried by a tunnel destined for context ID of . Indeed, but that needs to be made clearer, particularly as this seemingly has scaling consequences. > > [7] A PLR may be any router in the network, including P or PE router. The draft assumes that it doesn't know or care about PW info. > > [8] A protector is a PE router Within a PE context I am not sure that the properties of a PE router are defined. > in the "co-located" model. It may be a P router in the "centralized" model. For the later case, the draft describes how to use the new LDP extensions for the protector to learn and assoicate primary PW and backup PW and install forwarding entries in the context label table. So when you run the centralized protector, how does the PW parameter signaling and interface parameter verification work? Surely the centralized protector needs to be some form of PE rather than a variant of an MPLS P router? I have a bunch of comments on your answer below, put perhaps we can start by working through the comments above and the cycle back to the comments below within the resultant context. I will comment on our exchange below when we have reached some level of consensus on the preamble. Given the concerns that I have raised and those raised by Sasha I think that the chairs need consider taking this back to the WG for further work. - Stewart > > Thanks, > > /Yimin > > > -----Original Message----- > From: pwe3 [mailto:pwe3-bounces@ietf.org] On Behalf Of Stewart Bryant > Sent: Tuesday, July 29, 2014 10:53 AM > To: pwe3; pwe3-chairs@tools.ietf.org > Cc: mpls@ietf.org > Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 > > > > I do not support the publication of this draft without: > > 1) An assurance that MPLS WG has reviewed it and is happy > with the extensions that seem to be made to the MPLS architecture > > 2) A formal description of those extensions or a pointer > to those extensions in another RFC. > > 3) Significant clarification of how the detail of this > mechanism works. > > For this to work you need to have co-ordinated label spaces > in multiple PEs which is something that was investigated > by the SPRING WG and found not to work, particularly when > different manufacturers equipments were involved. > > [yshen] This draft does not rely on label space coordination. Rather, it is based on context label forwarding paradigm specified in RFC 5331. > > > As far as I can see you also need to peek below the top > of stack to do a fast reroute operation which is not > a currently defined MPLS operation. > > There seem to be a lot of details and clarifications missing > that need to be provided, and I am not entirely convinced > that all of the items declared out of scope are legitimately > out of scope and not needed in order for a multi-vendor > solution to be implemented. > > It would be useful to me (and I suspect to other readers) > to see what the dataplane looks like at various points > in the normal and repair path to verify that all parties > have the right information for this to work under normal > and various fault conditions. > > I have a number of comments inline that need to be addressed > before this can proceed. > > > - Stewart > > > PW Endpoint Fast Failure Protection > draft-ietf-pwe3-endpoint-fast-protection-01 > > Abstract > > This document specifies a fast mechanism for protecting pseudowires > against egress endpoint failures, including egress attachment circuit > failure, egress PE failure, multi-segment PW terminating PE failure, > and multi-segment PW switching PE failure. Designed on the basis of > multi-homed CE, PW redundancy, upstream label assignment and context > specific label switching, the mechanism enables local repair to be > performed by the router upstream adjacent to a failure. > > In > particular, the router can restore PW traffic in the order of tens of > milliseconds, > > SB> Not sure it's wise to specify a performance claim in a standards > SB> track RFC. > > [yshen] If this is viewed as inappropriate, we can remove it. > > by transmitting the traffic to a protector through a > pre-established bypass tunnel. Therefore, the mechanism can reduce > traffic loss before global repair reacts to the failure and the > network converges on the topology changes due to the failure. > > > > === > > > 1. Introduction > > Per RFC 3985, RFC 4447 and RFC 5659, a pseudowire (PW) or PW segment > can be thought of as a connection between a pair of forwarders hosted > by two PEs, carrying an emulated layer-2 service over a packet > switched network (PSN). In the single-segment PW (SS-PW) case, a > forwarder binds a PW to an attachment circuit (AC). In the multi- > segment PW (MS-PW) case, a forwarder on a terminating PE (T-PE) binds > a PW segment to an AC, while a forwarder on a switching PE (S-PE) > binds one PW segment to another PW segment. In each direction > between the PEs, PW packets are transported by a PSN tunnel, which is > called a transport tunnel. > > SB> Did we ever your the term transport tunnel before? > > [yshen] This is mainly for clarity. We could change it to " which is > called a transport tunnel *in this document*". Or we can completely replace "transport tunnel" with just "tunnel". > > 3. Reference Models and Failure Cases > > The mechanism in this document intends to > use these topologies for local repair purposes. > > SB> The toplology is not a mechanism. Are you saying that the mechanims > SB> only works in this topology? > > [yshen] No, these topologies are only examples for use by subsequent sections to describe the mechanisms. The mechanisms should equally work for other topologies as well. > > This SHALL enable > local repair and global repair to work in tandem to achieve broader > coverage of protection for services. > > SB> I have no idea how a SHALL in capitals applies here. > > [yshen] Will fix this. > > 3.1. Single-Segment PW > > > > |<-------------- PW1 --------------->| > > - PE1 -------------- P1 ---------------- PE2 - > / \ > / \ > CE1 CE2 > \ / > \ / > - PE3 -------------- P2 ---------------- PE4 - > > |<-------------- PW2 --------------->| > > Figure 1 > > > Each CE is multi-homed to two PEs. Hence, there are two divergent > paths between the CEs. > > SB> It does not follow that the paths are divergent! Who knows > SB> what happens in the IP layer and the underlying transport > SB> network. > > > [yshen] We could rephrase this and remove "divergent". All we want to say here is that there are two distinct sets of AC-PW-AC between the CEs. > > At any given time, each CE sends traffic via only one AC and receives > traffic via only one AC. The two ACs MAY or MAY NOT be the same. > > SB> May not be the same what? If they are actually the same AC you > SB> have no redundancy. > > [yshen] Will rephrase it. All we want to say is that at a given time, the CE may send traffic over one AC and receive traffic over another AC. > > The AC used to send traffic is determined by the CE, and MAY rely on > an end-to-end OAM mechanism between the CEs. The AC used for the CE > to receive traffic is determined by the state of the network and the > protection mechanism in use, as described later in this document. > > From the perspective of traffic flowing towards a given CE, the set > of PWs, PEs and ACs involved can be viewed to serve primary and > backup (or active and standby) roles. When the network is in a > steady state, the PW that is intended to carry the traffic is > referred to as a primary PW. The PE at the egress of the primary PW > is a primary PE. The AC connecting the CE and the primary PE is a > primary AC. The other PW may be used to carry the traffic upon a > network failure, and is referred to as a backup PW. The PE at the > egress of the backup PW is a backup PE. The AC connecting the CE and > the backup PE is a backup AC. > > In this document, the following primary and backup roles are assigned > for the traffic going from CE1 to CE2: > > Primary PW: PW1 > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 5] > > > Internet-Draft PW Endpoint Fast Failure Protection July 2014 > > > Primary PE: PE2 > > Primary AC: CE2-PE2 > > Backup PW: PW2 > > Backup PE: PE4 > > Backup AC: CE2-PE4 > > In this case, an egress AC failure refers to the failure of the AC > CE2-PE2. An egress node failure refers to the failure of PE2. > > The backup PE, backup PW and backup AC may be used to carry traffic > after a PW endpoint failure, when CE1 and CE2 switches traffic to PW2 > in local repair or global repair, as described later in this > document. > > |<-------------- PW1 --------------->| > > ------------- P1 ---------------- PE2 - > / \ > / \ > CE1 -- PE1 CE2 > \ / > \ / > ------------- P2 ---------------- PE4 - > > |<-------------- PW2 --------------->| > > Figure 2 > > Figure 2 shows another possible scenario, where CE1 is single-homed > to PE1, while CE2 remains multi-homed to PE2 and PE4. From the > perspective of egress protection for the traffic from CE1 to CE2, > this topology is not much different than Figure 1. However, for the > > SB> "not much different than Figure 1" is not helpful, either it is the > same, or you need to explain the difference. > > [yshen] Will rephrase. The difference is actually described after "However...". > > > traffic in the direction from CE2 to CE1, PE1 must anticipate traffic > on both PW1 and PW2, and sends it to CE1 over the AC CE1-PE1. > > SB> What does it mean to anticipate traffic. > > [yshen] To anticipate incoming traffic over either PW1 or PW2. Will rephrase. > > 3.2. Multi-Segment PW > > > > > > > > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 6] > > > Internet-Draft PW Endpoint Fast Failure Protection July 2014 > > > |<--------------- PW1 --------------->| > |<----- SEG1 ----->|<----- SEG2 ----->| > > - TPE1 -------------- SPE1 --------------- TPE2 - > / \ > / \ > CE1 CE2 > \ / > \ / > - TPE3 -------------- SPE2 --------------- TPE4 - > > |<----- SEG3 ----->|<----- SEG4 ----->| > |<--------------- PW2 --------------->| > > Figure 3 > > Figure 3 shows a topology that is similar to Figure 1 but in an MS-PW > environment. PW1 and PW2 are both MS-PWs. PW1 is established > between TPE1 and TPE2, and switched between segments SEG1 and SEG2 at > SPE1. PW2 is established between TPE3 and TPE4, and switched between > segments SEG3 and SEG4 at SPE2. CE1 is multi-homed to TPE1 and TPE3. > CE2 is multi-homed to TPE2 and TPE4. The transport tunnels of the PW > segments are not shown in this figure for clarity. > > In this document, the following primary and backup roles are assigned > for the traffic going from CE1 to CE2: > > Primary PW: PW1 > > Primary T-PE: TPE2 > > Primary S-PE: SPE1 > > Primary AC: CE2-TPE2 > > Backup PW: PW2 > > Backup T-PE: TPE4 > > Backup S-PE: SPE2 > > Backup AC: CE2-TPE4 > > In this case, an egress AC failure refers to the failure of the AC > CE2-TPE2. An egress node failure refers to the failure of TPE2. An > S-PE failure refers to the failure of SPE1. > > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 7] > > > Internet-Draft PW Endpoint Fast Failure Protection July 2014 > > > The backup T-PE, backup PW and backup AC are used for protecting the > primary PW against egress AC failure and egress node failure. The > backup S-PE and the backup PW are used for protecting the primary PW > against S-PE failure, as described later in this document. > > For consistency with the SS-PW scenario, primary T-PEs and a primary > S-PEs may simply be referred to as primary PEs in this document, > where specifics is not required. Similarly, backup T-PEs and backup > S-PEs may be referred to as backup PEs. > > 4. Theory of Operation > > The fast protection mechanism in this document provides three types > of protection for PWs, corresponding to the three types of failures > described in Section 1. > > a. Egress AC protection > > b. Egress (T-)PE node protection > > c. S-PE node protection > > The mechanism assumes a multi-homed CE with connectivity to a primary > PE and a backup PE, and the existence of a backup PW in the network. > In S-PE node protection, it also assumes the existence of a backup > S-PE on the backup PW. > > > SB> Do the PWs need to be symmetric - could you have more > S-PEs on one that you have on the other? > > [yshen] You could. For a given S-PE, as long as you can find another PE to serve as protector, it will work. > > 4.1. Local Repair and Protector > > The mechanism relies on local repair to be performed by routers > SB> Do you mean routers or PEs? Routers of more likely LSRs > operate at a different layer and have no business knowing about > PWs. > > [yshen] routers (LSRs). Yes, they have no idea about PWs. This draft doesn't assume they know or care about PWs. > > SB> Has this work been reviewed my MPLS WG? I am concerned > that you are doing a bunch of layer violation without explaining > the how this is achieved or whether it is acceptable within the > MPLS architecture. > > [yshen] There is no layer violation. The PLR simply redirect traffic from the original tunnel to a bypass tunnel, by doing a top label swap, without touching, changing or knowing the PW label. Please take a close look at the text. > > upstream adjacent to failures. Each of these routers is referred to > as a "point of local repair" (PLR). A PLR MUST be able to detect a > failure by using a rapid mechanism, such as physical layer failure > detection, Bidirectional Failure Detection (BFD) (RFC 5880), etc. In > anticipation of the failure, the PLR MUST also pre-establish a bypass > PSN tunnel to a "protector", and pre-install a bypass route in the > data plane. The bypass tunnel MUST have the property that it will > not be affected by the topology changes in the event of the failure. > > SB> This is far too imprecise - you need to separate operations and > topologies in the PW layer from operations and topologies in the > PSN layer > > [yshen] Similar to existing IP/MPLS fast-reroute RFCs, this draft doesn't restrict local repair trigger to any specific failure detection mechanisms. Any mechanism will work. > > SB> In each case you need to be clear how you are detecting failure > by specifying the MEPs and MIPs you are testing. > > Upon detecting the failure, the PLR MUST invoke the bypass route in > the data plane, and reroute PW traffic to the protector through the > bypass tunnel. The protector MUST in turn send the traffic to the > target CE. > > SB> No the protector knows nothing about CEs - they are in the client > layer, and you can only protect at the server layer or the server's > server layer. > > [yshen] In collocated model, the protector is a PE. In the centralized model, the protector may be an LSR , and it needs to participate in LDP PW signaling to understand PWs. See the related sections and LDP extensions in this draft. This centralized model is optional. > > This procedure is referred to as local repair. > > Different routers may serve as PLR and protector in different > scenarios. > > SB> Remember they are not routers - they are xPEs or LSRs. > > [yshen] Again, in collocated model, the protector is a PE. In the centralized model, the protector may be an LSR , and it needs to participate in LDP PW signaling to understand PWs. See the related sections and LDP extensions in this draft. But this centralized model is optional. > [yshen] PLRs may be LSRs, but they don't need to know PWs, because all they do is to swap top label (tunnel label) to a bypass tunnel's label. > > SB> In each of the following we need to note that these are > not RFC3985 PEs, even if they take no part in the protection > such as the PEs on the left. Indeed I am wondering if you need > to update RFC3985 and RFC5659 > > [yshen] There is no need to update RFC3985 and RFC5659. > > Yimin Shen, et al. Expires January 25, 2015 [Page 8] > > > Internet-Draft PW Endpoint Fast Failure Protection July 2014 > > > o In egress AC protection, the PLR is the primary PE that terminates > the primary PW and hosts the primary AC, and the protector is the > backup PE (Figure 4). > > |<-------------- PW1 --------------->| > > - PE1 -------------- P1 ---------------- PE2 - > / PLR \ > / | \ > CE1 bypass| CE2 > \ | / > \ | / > - PE3 -------------- P2 ---------------- PE4 - > protector > > |<-------------- PW2 --------------->| > > Figure 4 > > > o In egress PE node protection, the PLR is the penultimate hop > router of the transport tunnel of the primary PW, and the > protector is the backup PE (Figure 5). > > |<-------------- PW1 --------------->| > > - PE1 -------------- P1 ------- P3 ----- PE2 - > / PLR \ \ > / \ \ > CE1 bypass\ CE2 > \ \ / > \ \ / > - PE3 -------------- P2 ---------------- PE4 - > protector > > |<-------------- PW2 --------------->| > > Figure 5 > > SB> Note that it is nothing like as clean as this, if as you later have, > some of P3's traffic needs to go to PE4 and some to PE5. > Also what about the > > [yshen] Note that we are talking about a single PW, rather than multiple PWs carried by a single tunnel. > > o In S-PE node protection, the PLR is the penultimate hop router of > the transport tunnel of the primary PW segment, and the protector > is the backup S-PE (Figure 6). > > > > > > > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 9] > > > Internet-Draft PW Endpoint Fast Failure Protection July 2014 > > > |<--------------- PW1 --------------->| > |<----- SEG1 ----->|<----- SEG2 ----->| > > - TPE1 ----- P4 ----- SPE1 -------------- TPE2 - > / PLR \ \ > / \ \ > CE1 bypass\ CE2 > \ \ / > \ \ / > - TPE3 --------------- SPE2 -------------- TPE4 - > protector > > |<----- SEG3 ----->|<----- SEG4 ----->| > |<--------------- PW2 --------------->| > > Figure 6 > > A PLR can realize its role based on configuration or the signaling of > transport tunnel. For example, in the case where the transport > tunnel is signaled by RSVP, the penultimate hop router could realize > > SB> Could realize is not sufficient. The mechanism needs to be > unconditional. > > [yshen] Will rephrase. > > that it is the PLR for egress (T-)PE or S-PE failure based on the RRO > in Resv message, which should indicate that the router is one hop > away from the PE. The detail of how this could be achieved on a per- > protocol basis is out of the scope of this document. > > SB> So where is it in scope? How do I find out how to do it? > > In all scenarios, when a PLR reroutes traffic through a bypass tunnel > to a protector during local repair, it MUST keep the label of the > primary PW intact in the packets. This obviates the need for the PLR > to maintain bypass routes on a per-PW basis, and allows a single > bypass tunnel to carry traffic for multiple PWs. > > SB> OK, but didn't I read something about PE protecting subsets of the > egress traffic of other PEs? How would that work without per PW information? > > [yshen] PWs to a given PE may be protected by multiple protectors. A set of PWs sharing the same protector X can be carried by the same tunnel (destined to the context ID of ). PWs protected by a different protector Y must be carried by different tunnel (destined to the context ID of ). The PLR of each tunnel doesn't need to know PW info. All it does is to swap tunnel label to a bypass tunnel's label, and the bypass tunnel will guide the traffic to the intended protector. > > The procedure also requires that the protector SHOULD be able to > forward the traffic based on a PW label that is assigned by the > primary PE, and ensure the traffic to eventually reach the target CE. > From the protector's perspective, this PW label is an upstream > assigned label (RFC 5331). > > SB> .. but the protector may be a P router and know nothing about > PWs. > > [yshen] Again, in collocated model, the protector is a PE. In the centralized model, the protector may be an LSR , and it needs to participate in LDP PW signaling to understand PWs. See the related sections and LDP extensions in this draft. But this centralized model is optional. > > > To accomplish this, the protector SHOULD > learning the PW label from the primary PE prior to the failure, and > install proper forwarding state for the PW label in a dedicated label > space for the primary PE. > > SB> This is a new PE to P router protocol > > [yshen] The LDP extension is defined in a subsequent section. > > During local repair, the protector SHOULD > perform PW label lookup in this label space. > > SB> This has to be signed off by MPLS WG > SB> How can this only be a SHOULD - why is it not a MUST? > > [yshen] Will rephrase. > > SB> What do you do to ensure that they even have compatible label managers? > > [yshen] The LDP extensions defined in this draft covers how to negotiate the capability. > > > SB> Doesn't this mean a P router looking up two labels to do > the repair? > > [yshen] PLR only swaps the top label (i.e. tunnel label) to bypass label table, when doing the local repair. > > The above examples have shown the scenarios where the protectors are > backup (T/S-)PEs. In other scenarios, a protector may be a dedicated > router that assumes such role, separate from the backup (T/S-)PE of a > primary PW. During local repair, the PLR MUST still reroute traffic > to the protector through a bypass tunnel. The protector MUST in turn > send the traffic to the backup (T/S-)PE, which MUST then send the > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 10] > > > Internet-Draft PW Endpoint Fast Failure Protection July 2014 > > > traffic to the target CE via a backup AC or a backup PW segment. > More detail will be described in Section 4.3. > > 4.2. Context Identifier > > A protector MAY protect multiple primary PEs. > > SB> This requires that a P router knows about PWs - how does > it do that? > > [yshen] The protector will perform context label forwarding. The bypass tunnel points to a context label table, and the protector looks up the PW label in that table. > > The protector MUST > maintain a separate label space for each primary PE. Likewise, the > PWs terminated on a primary PE MAY be protected by multiple > protectors, each for a subset of the PWs. In any case, a given > primary PW SHOULD be associated with one and only one pair of > {primary PE, protector}. > > An IPv4/v6 address is assigned to each ordered pair of {primary PE, > protector} to facilitate protection establishment. This address is > referred to as a "context identifier". It MUST be globally unique, > or unique in the address space of the network where the primary PE > and the protector reside. > > SB> I think you need to make clear that this is not backwards compatible > with existing PW PEs. > > [yshen] PEs will need to support the extensions defined by this draft. > > SB> You also need to explain how the IP based context identifier > mechanism works. Is it in the control plane or the data plane? > > 4.2.1. Semantics > > The semantics of a context identifier is twofold. > > o It identifies a primary PE and an associated protector. In other > words, it identifies a primary PE on a per protector basis. A > given primary PE may be protected by multiple protectors, each for > a subset of the primary PWs terminated on the primary PE. A > distinct context identifier MUST be assigned to the primary PE and > each protector. > > For each primary PW, its ingress PE MUST set up or resolve a > transport tunnel with destination as the context identifier of the > {primary PE, protector}, rather than a private IP address of the > primary PE. This not only allows the transport tunnel to reach > the primary PE, but also conveys the identity of the protector to > the PLR(s) along the transport tunnel. Each PLR can in turn use > this information to set up a bypass tunnel to the protector > without relying on local configuration. > > o It indentifies the primary PE's label space on the protector. The > protector may protect PWs for multiple primary PEs. For each > primary PE, it MUST maintain a separate label space to store the > PW labels assigned by that primary PE. It MUST associate a PW > label with a label space via the context identifier of the > {primary PE, protector}, as described below. > > SB> Is this context identifier an IP address or an MPLS label? Where > is it carried? > > [yshen] a context identifier is an IP address. If the tunnel is IP tunnel, it is carried in IP header as dest address. If the tunnel is MPLS tunnel, it is indicated by the label. > > In addition to the normal LDP PW signaling, the primary PE MUST > have a targeted LDP session with the protector, and advertise PW > labels to the protector via LDP Label Mapping messages (See > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 11] > > > Internet-Draft PW Endpoint Fast Failure Protection July 2014 > > > Section 5 for detail). The primary PE MUST attach the context > identifier to each message. Upon receiving the message, the > protector MUST install the advertised PW label in the label space > identified by the context identifier. > > SB> I am currently confused about how a data packet carries the PW > label provided by another PE. > > When a PLR sets up or resolve a bypass tunnel to the protector, it > MUST set the destination to the context identifier, rather than a > private IP address of the protector. Once established, the bypass > tunnel, with either its MPLS label or IP tunnel destination > address, is used as the identifier of label space. On the > protector, all PW packets received on the bypass tunnel MUST be > forwarded based on a label lookup in that label space. > > SB> I do not understand the data plane structure that the above sentence > is describing. > > [yshen] This is basically the context label forwarding paradigm (RFC 5331). From the protector's perspective, the bypass tunnel serves as a context, and indicates the label space of the primary PE where the PW label must be looked up. > > > 4.2.2. Advertisement and Path Computation > > Using a context identifier as destination for both transport tunnel > and bypass tunnel requires both the primary PE and the protector to > advertise the context identifier via IGP as an IP address reachable > through both routers in routing domain and/or TE domain. This > imposes the following requirements on path computation for these > tunnels. > > o For the transport tunnel, the ingress PE MUST choose the primary > PE as the actual endpoint. > > o For the bypass tunnel, the PLR MUST choose the protector as the > actual endpoint. In egress (T-)PE node protection and S-PE node > protection, the bypass tunnel MUST avoid the primary (S-)PE. > > The detail of how the primary PE and the protector may advertise a > context identifier is independent of this mechanism and out of the > scope of this document. > > SB> Given that you give signalling data structures later, I am confused > as to how this can be out of scope of this text. Surely this is needed > to implement an interoperable solution? > > [yshen] There is no new extension needed for IGP. Both PEs advertise the context ID. That's it. Will rephrase it. > > One approach would be to advertise it as a > virtual proxy node connected to both routers, with the link between > the proxy node and the primary PE having a more preferable IGP or TE > metric than the link between the proxy node and the protector. The > ultimate goal is for a path computation algorithm, such as CSPF > (constrained shortest path first), LFA (RFC 5286) and MRT ([IP-LDP- > FRR-MRT]), to be able to compute the paths that meet the above > requirements. > > 4.3. Protection Models > > There are two protection models based on the location of a protector. > A network MAY use either model, or a combination of both. > > > > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 12] > > > Internet-Draft PW Endpoint Fast Failure Protection July 2014 > > > 4.3.1. Co-located Protector > > In this model, the protector is a backup PE that is directly > connected to the target CE via a backup AC, or it is a backup S-PE on > a backup PW. That is, the protector is co-located with the backup > (S-)PE. Examples of this model have been introduced in Figure 4, > Figure 5 and Figure 6 in Section 4.1. > > In egress AC protection and egress PE node protection, when a > protector receives traffic from the PLR, it forwards the traffic to > the CE via the backup AC. This is shown in Figure 7, where PE2 is > the PLR for egress AC failure, P3 is the PLR for PE2 failure, and PE4 > (the backup PE) is the protector. > > |<-------------- PW1 --------------->| > > - PE1 -------------- P1 ------- P3 ----- PE2 ---- > / PLR \ PLR \ > / \ | \ > CE1 bypass\ |bypass CE2 > \ \ | / > \ \ | / > - PE3 -------------- P2 ---------------- PE4 ---- > protector > > |<-------------- PW2 --------------->| > > Figure 7 > > In S-PE node protection, when a protector receives traffic from the > PLR, it MUST forward the traffic via the next segment of the backup > PW. The T-PE of the backup PW MUST in turn forward the traffic to > the CE via a backup AC. This is shown in Figure 8, where P4 is the > PLR for SPE1 failure, and SPE2 (the backup S-PE) is the protector for > SPE1. > > > > > > > > > > > > > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 13] > > > Internet-Draft PW Endpoint Fast Failure Protection July 2014 > > > |<--------------- PW1 --------------->| > |<----- SEG1 ----->|<----- SEG2 ----->| > > - TPE1 ----- P4 ----- SPE1 -------------- TPE2 - > / PLR \ \ > / \ \ > CE1 bypass\ CE2 > \ \ / > \ \ / > - TPE3 --------------- SPE2 -------------- TPE4 - > protector > > |<----- SEG3 ----->|<----- SEG4 ----->| > |<--------------- PW2 --------------->| > > Figure 8 > > In the co-located protector model, the number of context identifiers > needed by a network is the number of distinct {primary PE, backup PE} > pairs. From the perspective of scalability, the model is suitable > for networks where the number of backup PEs for any given primary PE > is relatively small. > > 4.3.2. Centralized Protector > > In this model, the protector is a dedicated P router or PE router > that serves the role. In egress AC protection and egress PE node > protection, the protector MAY or MAY NOT be a backup PE with a direct > connection to the target CE. In S-PE node protection, the protector > MAY or MAY NOT be a backup S-PE on the backup PW. > > In egress AC protection and egress PE node protection, when the > protector receives traffic from the PLR, if the protector has a > direct connection (i.e. backup AC) to the CE, it MUST forward the > traffic to the CE via the backup AC, which is similar to Figure 7. > Otherwise, it MUST forward the traffic to a backup PE, which MUST > then forward the traffic to the CE via a backup AC. This is shown in > Figure 9, where the protector receives traffic from P3 (the PLR for > egress PE failure) or PE2 (the PLR for egress AC failure) and > forwards the traffic to PE4 (the backup PE). The protector may be > protecting other PWs as well, which is not shown in this figure for > clarity. > > SB> If you are going to cover the shared case, I think you need to > make it much clearer how the PWs are identified from the received > data packets. > > > > > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 14] > > > Internet-Draft PW Endpoint Fast Failure Protection July 2014 > > > |<------------- PW1 --------------->| > > - PE1 ------------- P1 ------- P3 ----- PE2 -- > / PLR \ PLR \ > / \ / \ > / bypass\ /bypass \ > / \ / \ > CE1 protector CE2 > \ \ / > \ \ / > \ \ / > \ \ / > - PE3 ------------- P2 -----------------PE4 -- > > |<------------- PW2 --------------->| > > Figure 9 > > In S-PE node protection, when the protector receives traffic from the > PLR, if the protector is a backup S-PE of the backup PW, it MUST > forward the traffic via the next segment of the backup PW, and the > T-PE of the backup PW MUST forward the traffic to the CE via a backup > AC, which is similar to Figure 8. Otherwise, the protector MUST > first forward the traffic to the backup S-PE, which MUST then forward > the traffic via the next segment of the backup PW. Finally, the T-PE > of the backup PW MUST forward the traffic to the CE via a backup AC. > This is shown in Figure 10, where the protector forwards traffic to > SPE2 (the backup S-PE), SPE2 forwards the traffic to TPE4 via SEG4, > and TPE4 finally forwards traffic to CE2. The protector may be > protecting other PW segments as well, which is not shown in this > figure for clarity. > > > > > > > > > > > > > > > > > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 15] > > > Internet-Draft PW Endpoint Fast Failure Protection July 2014 > > > |<--------------- PW1 --------------->| > |<----- SEG1 ----->|<----- SEG2 ----->| > > - TPE1 ----- P4 ----- SPE1 -------------- TPE2 - > / PLR \ \ > / \ \ > / bypass\ \ > / \ \ > CE1 protector CE2 > \ \ / > \ \ / > \ \ / > \ \ / > - TPE3 --------------- SPE2 -------------- TPE4 - > > |<----- SEG3 ----->|<----- SEG4 ----->| > |<--------------- PW2 --------------->| > > Figure 10 > > The centralized protector model provides the convenience for multiple > primary PEs to share one protector. Each primary PE MAY only need > the one protector to protect all of its PWs. From the perspective of > scalability, the number of context identifiers needed by a network > MAY be bound to the number of primary PEs. > > 4.4. Transport Tunnel > > The ingress PE of a primary PW associates the PW with the primary > egress PE through LDP signaling. In addition, as mentioned in > Section 4.2.1, the ingress PE MUST associate the transport tunnel of > the PW with the context identifier of the {primary PE, protector}, > and set up or resolve the transport tunnel by using the context > identifier as destination. This not only ensures that PW traffic be > transported to the primary PE, but also facilitates bypass tunnel > establishment at PLR(s), as the context identifier implies the > identity of the protector as well. This is also the case for a > multi-segment PW, where the ingress PE and egress PE are T/S-PEs. > > The association between the transport tunnel and the context > identifier at the ingress PE MAY be achieved by configuration or an > auto-discovery mechanism. In the later case, the ingress PE MAY > learn the context identifier from the primary (egress) PE, if the > primary PE advertises the context identifier as "third party next > hop" in IPv4/v6 Interface_ID (RFC 3471, RFC 3472) in the LDP > Label Mapping message of the primary PW. > > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 16] > > > Internet-Draft PW Endpoint Fast Failure Protection July 2014 > > > 4.5. Bypass Tunnel > > A PLR may protect multiple PWs associated with one or multiple pairs > of {primary PE, protector}. The PLR MUST establish a bypass tunnel to > each protector for each distinct context identifier associated with > that protector. The destination of the bypass tunnel MUST be the > context identifier (Section 4.2.1). The PLR may derive the context > identifier from the destination of the transport tunnel that > traverses it. > > For examples, in Figure 7 and Figure 9, a bypass tunnel is > established from PE2 (PLR for egress AC failure) to the protector, > and another bypass tunnel is established from P3 (PLR for egress node > failure) to the protector. In Figure 8 and Figure 10, a bypass > tunnel is established from P4 (PLR for S-PE failure) to the > protector. > > During local repair, the PLR reroutes traffic to the protector > through a bypass tunnel with PW label intact in the packets. This > normally involves pushing a label to the label stack, if the bypass > tunnel is an MPLS tunnel, or pushing an IP header to the packets, if > the bypass tunnel is an IP tunnel. > > SB> Surely it is normally a swap? > > [yshen] yes, this is just a label swap of the top label, from tunnel label to bypass tunnel label. > > SB> I am somewhat confused about what this all looks like when this is > IP. The only PS/IP other than for SATOP are defined by L2TPv3 WG. > There is no IP MS-PW definition that I am aware of. I think you need > to be much clearer about what is going on when you discuss PW over IP > egress repair. > > The protector MUST in turn > forward the traffic based on the PW label. To achieve such kind of > forwarding, the protector MUST rely on the bypass tunnel as a context > to determine the primary PE's label space. If the bypass tunnel is > an MPLS tunnel, the protector MUST have assigned a non-reserved label > to the bypass tunnel during the establishment of the bypass tunnel, > and hence this label can serve as the context. If the bypass tunnel > is an IP tunnel, the protector can simply rely on the context > identifier carried as the destination address in IP header. > > A bypass tunnel MUST have the property that it is not affected by the > topology changes caused by the failure. Therefore, it can be used to > transmit traffic for local repair. It SHOULD remain effective, until > the traffic is moved to another fully functional egress AC, PW and/or > transport tunnel. > > 4.6. Forwarding State on Protector > > A protector MUST learn PW labels from all the primary PEs that it > protects (Section 5.2), and maintain the PW labels in respective > label spaces of the primary PEs. In the control plane, a label space > is identified by the context identifier of a pair of {primary PE, > protector}. In the forwarding plane, it is indicated by the bypass > tunnel(s) destined for the context identifier. > > SB> Are you saying that this mechanism mandates no PHP? > > [yshen] It mandates PHP for bypass tunnels only, as their labels serve as contexts on protectors. This is not mandatory for normal tunnels. > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 17] > > > Internet-Draft PW Endpoint Fast Failure Protection July 2014 > > > 4.6.1. Examples of Co-located Protector > > In Figure 7, PE4 is a co-located protector that protects PW1 against > egress AC failure and egress node failure. It maintains a label > space for PE2, which is identified by the context identifier of {PE2, > PE4}. It learns PW1's label from PE2, and installs an forwarding > entry for the label in that label space. The nexthop of the > forwarding entry indicates a label pop with outgoing interface > pointing to the backup AC CE2-PE4. > > SB> I am not sure what you are describing in the above. Is this an > MPLS operation or a PW operation. If the latter it is not a simple > label pop. > > [yshen] It is PW label pop. > > In Figure 8, SPE2 is a co-located protector that protects PW1 against > S-PE failure. It maintains a label space for SPE1, which is > identified by the context identifier of {SPE1, SPE2}. It learns > SEG1's label from SPE1, and installs a forwarding entry in the label > space. The nexthop of the forwarding entry indicates a label swap to > SEG4's label. > > 4.6.2. Examples of Centralized Protector > > In the centralized protector model, for each primary PW of which the > protector is not a backup (S-)PE, the protector MUST also learn the > label of the backup PW from the backup (S-)PE (Section 5.3). This is > the backup (S-)PE that the protector will forward traffic to. The > protector MUST install a forwarding entry with label swap from the > primary PW's label to the backup PW's label. > > In Figure 9, the protector is a centralized protector that protects > PW1 against egress AC failure and egress node failure. It maintains > a label space for PE2, which is identified by the context identifier > of {PE2, protector}. It learns PW1's label from PE2, and PW2's label > from PE4. It installs a forwarding entry for PW1's label in the > label space. The nexthop of the forwarding entry indicates a label > swap to PW2's label. > > In Figure 10, the protector is a centralized protector that protects > the PW segment SEG1 of PW1 against the node failure of SPE1. It > maintains a label space for SPE1, which is identified by the context > identifier of {SPE1, protector}. It learns SEG1's label from SPE1, > and learns SEG3's label from SPE2. It installs a forwarding entry > for SEG1's label in the label space. The nexthop of the forwarding > entry indicates a label swap to SEG3's label. > > 5. LDP Extensions > > As described in previous sections, a targeted LDP session MUST be > established between each pair of primary PE and protector. The > primary PE sends Label Mapping message over this session to advertise > primary PW labels to the protector. In the centralized protector > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 18] > > > Internet-Draft PW Endpoint Fast Failure Protection July 2014 > > > model, a targeted LDP session MUST also be established between a > backup (S-)PE and a protector. The backup PE sends Label Mapping > message over this session to advertise backup PW labels to the > protector. > > To facilitate the procedures, this document defines a new "Protection > FEC Element" TLV. The Label Mapping messages of both the LDP > sessions above MUST carry this TLV to indicate the identity of a > primary PW. Specifically, in the centralized protector model, the > Protection FEC Element TLV advertised by a backup (S-)PE MUST match > the one advertised by the primary PE, so that the protector can > associate the primary PW's label with the backup PW's label, and > perform a label swap. > > This document also defines the encoding of Capability Parameter TLV > (RFC 5561) for a new "Egress Protection Capability", to allow a > protector to announce its capability of processing the above > Protection FEC Element TLV and performing context specific label > switching for PW labels. > > The procedures in this section are only applicable, if the protector > advertises the Egress Protection Capability, the primary PE supports > the advertisement of the Protection FEC Element TLV, and in the > centralized protector model, the backup PE also supports the > advertisement of the Protection FEC Element TLV. > > 5.1. Egress Protection Capability TLV > > A protector MUST advertise the Egress Protection Capability TLV in > its Initialization message and Capability message, over the LDP > session with a primary PE. In the centralized protector model, the > protector MUST also advertise the TLV over the LDP session with a > backup PE. The TLV carries one or multiple context identifiers. To > the primary PE, the TLV SHOULD carry the context identifier of the > {primary PE, protector}. In the centralized protector model, the TLV > SHOULD carry to the backup PE multiple context identifiers, one for > each {primary PE, protector} where the backup PE serves as a backup > for the primary PE. This TLV SHOULD NOT be advertised by the primary > PE or the backup PE to the protector. > > The processing of the Egress Protection Capability TLV by a receiving > router SHOULD follow the procedures defined in RFC 5561. In > particular, the router SHOULD advertise PW information to the > protector by using the Protection FEC Element TLV, only after it has > received the Egress Protection Capability TLV from the protector. It > SHOULD validate each context identifier included in the TLV, and > advertise the information of only those PWs that are associated with > the context identifier. It SHOULD withdraw previously advertised > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 19] > > > Internet-Draft PW Endpoint Fast Failure Protection July 2014 > > > Protection FEC TLVs, when the protector has withdrawn a previously > advertised context identifier or the entire Egress Protection > Capability TLV via Capability message. > > The encoding of the Egress Protection Capability TLV is defined as > below. It conforms to the format of Capability Parameter TLV > specified in RFC 5561. > > 0 1 2 3 > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |U|F| Egress Protection (TBD) | Length | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |S| Reserved | | > +-+-+-+-+-+-+-+-+ | > | | > ~ Capability Data = context identifier(s) ~ > | | > | +-+-+-+-+-+-+-+-+ > | | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > Figure 11 > > The U-bit MUST be set to 1 so that a receiver MUST silently ignore > this TLV if unknown to it, and continue processing the rest of the > message. > > The F-bit MUST be set to 0 since this TLV is sent only in > Initialization and Capability messages, which are not forwarded. > > The TLV Code Point is TBD. It needs to be assigned by IANA. > > The S-bit indicates whether the sender is advertising (S=1) or > withdrawing (S=0) the capability. > > The "Capability Data" is encoded with the context identifier of the > {primary PE, protector}. > > SB> where is the structure of {primary PE, protector} defined? > > [yshen] The Protection FEC Element TLV carries the context ID of {primary PE, protector}. The {primary PE, protector} tuple itself is not encoded. > > 5.2. PW Label Distribution from Primary PE to Protector > > A primary PE SHOULD advertise a primary PW's label to a protector by > sending a Label Mapping message. The message includes a Protection > FEC Element TLV (see Section 5.4 for encoding), and an Upstream- > Assigned Label TLV (RFC 6389) encoded with the PW's label. The > combination of the Protection FEC Element TLV and the PW label > represents the primary PE's forwarding state for the PW. The Label > Mapping message SHOULD also carry an IPv4/v6 Interface_ID TLV (RFC > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 20] > > > Internet-Draft PW Endpoint Fast Failure Protection July 2014 > > > 6389, RFC 3471) encoded with the context identifier of the {primary > PE, protector}. > > The protector that receives this Label Mapping message SHOULD install > a forwarding entry for the PW label in the label space identified by > the context identifier. The nexthop of the forwarding entry SHOULD > ensure packets to be sent towards the target CE via a backup AC or a > backup (S-)PE, depending on the protection scenario. The protector > SHOULD silently discard a Label Mapping message if the included > context identifier is unknown to it. > > 5.3. PW Label Distribution from Backup PE to Protector > > In the centralized protector model, a backup PE SHOULD advertise a > backup PW's label to a protector by sending a Label Mapping message. > The message includes a Protection FEC Element TLV and a Generic Label > TLV encoded with the backup PW's label. This Protection FEC Element > MUST be identical to the Protection FEC Element TLV that the primary > PE advertises to the protector (Section 5.2). The context identifier > SHOULD NOT be encoded in Interface_ID TLV in this message. > > The protector that receives this Label Mapping message SHOULD > associate the backup PW with the primary PW, based on the common > Protection FEC Element TLV. It SHOULD distinguish between the Label > Mapping message from the primary PE and the Label Mapping message > from the backup PE based on the respective presence and absence of > context identifier in Interface_ID TLV. It SHOULD install a > forwarding entry for the primary PW's label in the label space > identified by the context identifier. The nexthop of the forwarding > entry SHOULD indicate a label swap to the backup PW's label, followed > by a label push or IP header push for a transport tunnel to the > backup PE. > > 5.4. Protection FEC Element TLV > > The Protection FEC Element TLV has type 0x83. Its format is defined > as below: > > > > > > > > > > > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 21] > > > Internet-Draft PW Endpoint Fast Failure Protection July 2014 > > > 0 1 2 3 > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Type(0x83) | Reserved | Encoding Type | Length | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | | > | | > ~ PW Information ~ > | | > | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > Figure 12 > > - Encoding Type > > Type of format that PW Information field is encoded. > > - Length > > Length of PW Information field in octets. > > - PW Information > > Field of variable length that specifies a PW > > For Encoding Type, 1 is defined for the PWid FEC Element format, and > 2 is defined for the Generalized PWid FEC Element format (RFC 4447). > > > 5.4.1. Encoding Format for PWid > > > > > > > > > > > > > > > > > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 22] > > > Internet-Draft PW Endpoint Fast Failure Protection July 2014 > > > 0 1 2 3 > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Type(0x83) | Reserved | Enc Type(1) | Length(16) | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Ingress PE Address | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Egress PE Address | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Group ID | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | PW ID | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |C| PW Type | Reserved | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > Figure 13 > > - Ingress PE Address > > IP address of the ingress PE of PW. > > - Egress PE Address > > IP address of the egress PE of PW. > > SB> How does this work for IPv6? > > - Group ID > > An arbitrary 32-bit value that represents a group of PWs and that > is used to create groups in the PW space. > > - PW ID > > A non-zero 32-bit connection ID that, together with the PW Type > field, identifies a particular PW. > > - Control word bit (C) > > A bit that flags the presence of a control word on this PW. If C > = 1, control word is present; If C = 0, control word is not > present. > > - PW Type > > A 15-bit quantity that represents the type of PW. > > SB> Needs a ref > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 23] > > > Internet-Draft PW Endpoint Fast Failure Protection July 2014 > > > 5.4.2. Encoding Format for Generalized PWid > > 0 1 2 3 > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Type(0x83) | Reserved | Enc Type(2) | Length | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Ingress PE Address | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Egress PE Address | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |C| PW Type | Reserved | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | AGI Type | Length | Value | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > ~ AGI Value (contd.) ~ > | | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | AII Type | Length | Value | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > ~ SAII Value (contd.) ~ > | | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | AII Type | Length | Value | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > ~ TAII Value (contd.) ~ > | | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > Figure 14 > > - Ingress PE Address > > IP address of the ingress PE of PW. > > - Egress PE Address > > IP address of the egress PE of PW. > > SB> IPv6? > > - Control word bit (C) > > A bit that flags the presence of a control word on this PW. If C > = 1, control word is present; If C = 0, control word is not > present. > > - PW Type > > A 15-bit quantity that represents the type of PW. > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 24] > > > Internet-Draft PW Endpoint Fast Failure Protection July 2014 > > > - AGI Type, Length, Value, AGI Value > > Attachment Group Identifier of PW. > > - SAII Type, Length, Value, SAII Value > > Source Attachment Individual Identifier of PW. > > - TAII Type, Length, Value, TAII Value > > Target Attachment Individual Identifier of PW. > > 6. Revertive Behavior > > Subsequent to local repair, there are three strategies for the > network to restore traffic to a fully functional PW. > > o Global revertive mode > > If the ingress CE is multi-homed (Figure 1), it MAY switch the > traffic to a backup AC which is bound to a backup PW. > Alternatively, if the ingress PE hosts a backup PW (Figure 2), the > ingress PE MAY switch the traffic to the backup PW. These > procedures are referred to as global repair. Possible triggers of > a global repair include PW status, OAM, and BFD. > > o Control plane revertive mode > > In egress PE node protection and S-PE node protection, it is > possible that the failure is limited to the link between the PLR > and the primary (S-)PE, whereas the primary (S-)PE is still up. > In this case, the PLR or an upstream router along the transport > tunnel MAY reroute the tunnel around the failed link via an > alternative path. Thus, the transport tunnel can continue to be > used to carry the PW traffic to the primary (S-)PE. This > procedure is driven by control plane convergence, and is referred > to as control plane repair. > > o Local revertive mode > > The PLR MAY move traffic back to the primary PW, after the failure > is resolved. In egress AC protection, upon detecting that the > primary AC is restored, the PLR MAY start forwarding traffic over > the AC again. Likewise, in egress PE node protection and S-PE > node protection, upon detecting that the primary PE is restored, > the PLR MAY re-establish the primary transport tunnel through the > primary PE, and move the traffic from the bypass tunnel back to > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 25] > > > Internet-Draft PW Endpoint Fast Failure Protection July 2014 > > > the transport tunnel. These procedures are referred to as local > reversion. > > The fast protection mechanism in this document SHOULD be used in > tandem with the global revertive mode. Particularly in the case of > egress (S-)PE failure, if the ingress PE or the protector loses > communication with the (S-)PE for an extensive period of time, the > LDP session between them may go down. Consequently, the ingress PE > may bring down the primary PW, or the protector may remove the > forwarding entry of the primary PW label. In either case, the > service will be disrupted. In other words, although the fast > protection can temporarily repair traffic, control plane state may > eventually be timed out if the failure persists. Therefore, it is > recommended that the global revertive mode SHOULD be set up in > advance, so that traffic can be moved to a fully functional backup PW > shortly after the local repair. > > The control plane revertive mode may always happen as part of the > convergence of control plane protocols. However, it is only > applicable to the specific scenarios described above. > > The local revertive mode is optional. In the circumstances where the > failure is caused by resource flapping, local reversion MAY be > dampened to limit potential disruptions. Local revertive mode MAY be > disabled completely by configuration. > > 7. IANA Considerations > > This document defines the encoding of the Capability Parameter TLV > for the new "Egress Protection Capability" in Section 5. This would > require IANA to assign a TLV Code Point to it. > > This document defines a new LDP Protection FEC Element TLV in > Section 5. IANA has assigned the type value 0x83 to it. > > SB> I am very confused about what explicit actions you are asking IANA > to take. Are you saying that there are no IANA actions? > > [yshen] Need IANA to assign a TLV code to "Egress Protection Capability", as said above. > > 8. Security Considerations > > The security considerations discussed in RFC 5036, RFC 5331, RFC > 3209, and RFC 4090 apply to this document. > > SB> Are there any PW security considerations that apply? > > SB> This is a new PW operational mode, so I am surprised that there are > no new security considerations. An early security review might be useful. > > > [yshen] Not sure about any specific security issues. > > > _______________________________________________ > pwe3 mailing list > pwe3@ietf.org > https://www.ietf.org/mailman/listinfo/pwe3 > . > -- For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/index.html From nobody Tue Aug 5 04:15:04 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CD8711B29A3; Tue, 5 Aug 2014 04:14:23 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.902 X-Spam-Level: X-Spam-Status: No, score=-1.902 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id a1VumMyoGbJw; Tue, 5 Aug 2014 04:14:11 -0700 (PDT) Received: from emea01-db3-obe.outbound.protection.outlook.com (mail-db3lrp0082.outbound.protection.outlook.com [213.199.154.82]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E62A21B2AA1; Tue, 5 Aug 2014 04:14:09 -0700 (PDT) Received: from AM3PR03MB612.eurprd03.prod.outlook.com (10.242.110.144) by AM3PR03MB610.eurprd03.prod.outlook.com (10.242.109.27) with Microsoft SMTP Server (TLS) id 15.0.995.14; Tue, 5 Aug 2014 11:14:06 +0000 Received: from AM3PR03MB612.eurprd03.prod.outlook.com ([10.242.110.144]) by AM3PR03MB612.eurprd03.prod.outlook.com ([10.242.110.144]) with mapi id 15.00.0995.014; Tue, 5 Aug 2014 11:14:06 +0000 From: Alexander Vainshtein To: "stbryant@cisco.com" , Yimin Shen Thread-Topic: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 Thread-Index: AQHPqzzbveYGLBru1kCsLuzIRj9tdJu4/J4AgAjkt4CAAAF6sA== Date: Tue, 5 Aug 2014 11:14:05 +0000 Message-ID: References: <53D7B569.60400@cisco.com> <53E0B85B.8060707@cisco.com> In-Reply-To: <53E0B85B.8060707@cisco.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [147.234.56.21] x-microsoft-antispam: BCL:0;PCL:0;RULEID: x-forefront-prvs: 02945962BD x-forefront-antispam-report: SFV:NSPM; SFS:(6009001)(13464003)(51704005)(24454002)(199002)(252514010)(189002)(164054003)(37854004)(479174003)(377454003)(106356001)(77982001)(80022001)(81342001)(74316001)(19580395003)(79102001)(81542001)(15202345003)(85852003)(46102001)(31966008)(105586002)(4396001)(50986999)(106116001)(64706001)(92566001)(19580405001)(66066001)(54356999)(95666004)(74502001)(83072002)(99396002)(86362001)(2656002)(20776003)(74662001)(101416001)(83322001)(33646002)(76482001)(87936001)(76176999)(76576001)(107046002)(21056001)(15975445006)(85306004)(24736002)(108616003)(559001)(579004); DIR:OUT; SFP:; SCL:1; SRVR:AM3PR03MB610; H:AM3PR03MB612.eurprd03.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; LANG:en; Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: ecitele.com Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/hY8kdfmT8MYs2l5omr4L_Y7Mp_4 Cc: "mpls@ietf.org" , pwe3 , "pwe3-chairs@tools.ietf.org" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Aug 2014 11:14:24 -0000 Stewart, Yimin and all, A couple of comments/questions inline below. Regards, Sasha=20 Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 > -----Original Message----- > From: pwe3 [mailto:pwe3-bounces@ietf.org] On Behalf Of Stewart Bryant > Sent: Tuesday, August 05, 2014 1:56 PM > To: Yimin Shen; pwe3; pwe3-chairs@tools.ietf.org > Cc: mpls@ietf.org > Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast- > protection-01 >=20 > On 30/07/2014 20:07, Yimin Shen wrote: > > Hi Stewart, > > > > Thanks very much for the detailed review and comments. I'd like to > highlight a few things below about this draft, and then respond to your > comments inline. Hope you could give the draft another closer look. > Yimin, >=20 > Having read your reply and having looked at the text again, > and talked to a few folks I think I now understand the top > level view of what is proposed. However that is in itself > an indication that the draft needs considerable work > before publication. Note that it was not initially obvious > that you supported both LDP and RSVP-TE PSNs, nor > was it initially obvious that you were using the PSN to > protect the PWs and not looking further down the stack. [[Sasha]] I still have serious doubts regarding LDP-based MPLS PSN - at lea= st without some extensions to LDP. > Indeed I think there is some text in the draft that positively > leads you in that direction since someone else picked > it it during an MPLS review. >=20 > To summarize: what you are doing is construct repairs > at the PSN layer that map to each combination of PW + PW repair, > which I think multiples the number of PSN tunnels by three times > the dispersion factor where the dispersion factor is some factor > defining the average number of PEs that the PWs addressed to > a single PE need to be dispersed to. >=20 > This leads to two questions that the WG needs to think about: >=20 > 1) This is trading increased PSN state for response time. > Given that PSN state was a critical factor in the choice of the Martini > design over the CCC design this needs to be made clear up front > in the text and the WG needs to decide that this is acceptable. >=20 > 2) This increased state and the complexity of introducing > context labels and protection routers to the PWE3 architecture > is directly traceable to the need to do xPE node protection, > and is not required for link protection including the egress AC. >=20 > This desire to build the design around node protection (which > always adds significant complexity to FRR) is in contrast to > the IPFRR situation where link protection is of overwhelming > importance and node protection seems to be of secondary > importance. >=20 > An alternative design, which would require less change to the > existing PWE3 design would be to provide redundant connection > to the xPEs and allow FRR link protection in the PSN to > deal with link failures and then to protect the egress AC by > making the T-PE a special type of S-PE that preferred to > act as a T-PE but if the AC was down forwarded to an alternate > T-PE for that PW. >=20 > If the WG prefers to introduce the cross layer scheme you have > proposed, then fine, we need to do the full set of updates needed. > However first I think that we need to have rough consensus that > the use cases and requirements justify the increased complexity > compared of node protection compared to the alternative of > providing link and AC protection. > > > > [1] This draft is completely based on the PWE3 and the MS-PW > architecture. It is also based on RFC 5331 " MPLS Upstream Label Assignme= nt > and Context-Specific Label Space". So ideally readers should be familiar = with > that RFC. > As far as I can see context labels have not been introduced > into the PWE3 architecture. There is some reference to their use for > P2MP PWs, but not in the P2P case. Indeed I think that RFC4447 > notes explicitly that in the case where the configuration of PWs > is signaled by LDP the platform label space must be used. The > least that you need to do is to update RFC4447. > [[Sasha]] yes, RFC 4447 explicitly requires the PW labels to be allocated f= rom the per-platform label space.=20 And according to RFC 5331 per-platform label space is a special case of the= label context. =20 >=20 > Something that I noticed in a second pass through the text is that > you seem to have changed the PW FEC data structure from the > one specified by PWE3 to a variant of one used in LSP Ping. This > seems unnecessary and may lead to confusion and difficulty if > these get changed at some time in the future. A better design > is surely to leave the PWE3 signalling system completely unchanged > except to add an additional TLV specifying the context information > if indeed we continue on that path. Apropos this, it may be the > way that the document was written, but I did not see anything about > the need to signal the PW interface parameters to the alternative > PE. Also I did not see anything specifying the error handling > as a result of parameter issues. >=20 > A comment below that was not picked up is that the signalling > design supports IPv4 only, and as I recall there is an IESG policy > of not standardizing any IPv4 only solutions, so perhaps we should > introduce IPv6 at the same time as harmonizing the normal and > context label data structures. >=20 > The LDP signalling of PWs with context labels need a more > complete review for both completeness and correctness and > to ensure that this text provides a description that is sufficiently > complete that this new feature can be implemented solely from > this text and the associated references. >=20 [[Sasha]] How is the router supposed to differentiate between an IP address= used as the context ID and an IP address used as the PE ID? > > > [2] The draft mentions two important roles of routers, PLR (point of lo= cal > repair) and protector. > You need a clearer definition of a PLR and protector. I think that > a protector is a new type of SPE and needs a clearer architectural > definition. >=20 > The concept of a PLR taking action at one layer within the context > of another is new in PWE3. Perhaps it is defined in MPLS in which > case a reference would be appropriate. > > > > [3] PLR performs local repair in dataplane, based on pre-installed back= up > forwarding state pointing to a bypass tunnel. There is no control-plane > involvement in local repair. > Specifically in the initiation of the local repair, it is in the > configuration > of course. > > This is similar to the existing IP/MPLS fast-reroute mechanisms. Theref= ore, > it can achieve restore traffic with a similar performance to those > mechanisms. That is the main goal of this draft. > IP/MPLS FRR normally operates solely for the benefit of its own > dataplane. You have this configured to act for the benefit of the > client dataplane. > > > > [4] When doing local repair, the PLR only swaps the top label (i.e. tun= nel > label) to a bypass tunnel's label, and keeps the PW label (allocated by > primary/protected PE) intact. The PLR doesn't need to know or understand > the PW label. > Indeed this is clearer now, and needs to be made much clearer in > the text. >=20 > However so do the implications for the system as a whole of this > mode of operation. > > > > [5] The protector is the router at the other end of the bypass tunnel. = It is > responsible for forwarding the traffic further, so that the traffic can > eventually reach the target CE. The protector must understand the PW labe= l > allocated by the primary PE. > The use of the work primary PE is slightly confusing. Presumably > you mean the ingress PE of the segment rather than the ingress > PE from the point of view of the ingress AC. > > This is achieved by using the context label forwarding paradigm specifi= ed in > RFC 5331. This draft defines some LDP extensions to allow the primary PE = to > advertise PW labels to the protector. The protector locally maintains a l= abel > space for these PW labels of the protected PE, and looks up the PW label = in > this label space. The bypass tunnel serves as the pointer to this label s= pace. > Now I understand that. Please make this much clearer > earlier in the text. >=20 > Note my earlier concern about the design of those extensions. > > > > [6] This draft also introduces the concept of "context ID". A context I= D is an > IP address identifying a pair of . > The use of an IP address rather than some identifier taken from > the PW context is a design assumption not properly shared with the > reader and needs to be clarified in the text. >=20 > > It serves as the destination of transport tunnel. PWs carried by the tu= nnel > will be redirected to the protector X during local repair. PWs to the sam= e > primary PE but protected by a different protector Y cannot be carried by = this > tunnel. Rather, they must be carried by a tunnel destined for context ID = of > . > Indeed, but that needs to be made clearer, particularly as this > seemingly has scaling consequences. > > > > [7] A PLR may be any router in the network, including P or PE router. T= he > draft assumes that it doesn't know or care about PW info. > > > > [8] A protector is a PE router > Within a PE context I am not sure that the properties of a PE router > are defined. > > in the "co-located" model. It may be a P router in the "centralized" mo= del. > For the later case, the draft describes how to use the new LDP extensions= for > the protector to learn and assoicate primary PW and backup PW and install > forwarding entries in the context label table. > So when you run the centralized protector, how does the > PW parameter signaling and interface parameter verification > work? Surely the centralized protector needs to be some > form of PE rather than a variant of an MPLS P router? >=20 > I have a bunch of comments on your answer below, put perhaps we > can start by working through the comments above and the cycle > back to the comments below within the resultant context. >=20 > I will comment on our exchange below when we have reached > some level of consensus on the preamble. >=20 > Given the concerns that I have raised and those raised by Sasha > I think that the chairs need consider taking this back to the WG > for further work. >=20 > - Stewart > > > > Thanks, > > > > /Yimin > > > > > > -----Original Message----- > > From: pwe3 [mailto:pwe3-bounces@ietf.org] On Behalf Of Stewart Bryant > > Sent: Tuesday, July 29, 2014 10:53 AM > > To: pwe3; pwe3-chairs@tools.ietf.org > > Cc: mpls@ietf.org > > Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast- > protection-01 > > > > > > > > I do not support the publication of this draft without: > > > > 1) An assurance that MPLS WG has reviewed it and is happy > > with the extensions that seem to be made to the MPLS architecture > > > > 2) A formal description of those extensions or a pointer > > to those extensions in another RFC. > > > > 3) Significant clarification of how the detail of this > > mechanism works. > > > > For this to work you need to have co-ordinated label spaces > > in multiple PEs which is something that was investigated > > by the SPRING WG and found not to work, particularly when > > different manufacturers equipments were involved. > > > > [yshen] This draft does not rely on label space coordination. Rather, i= t is > based on context label forwarding paradigm specified in RFC 5331. > > > > > > As far as I can see you also need to peek below the top > > of stack to do a fast reroute operation which is not > > a currently defined MPLS operation. > > > > There seem to be a lot of details and clarifications missing > > that need to be provided, and I am not entirely convinced > > that all of the items declared out of scope are legitimately > > out of scope and not needed in order for a multi-vendor > > solution to be implemented. > > > > It would be useful to me (and I suspect to other readers) > > to see what the dataplane looks like at various points > > in the normal and repair path to verify that all parties > > have the right information for this to work under normal > > and various fault conditions. > > > > I have a number of comments inline that need to be addressed > > before this can proceed. > > > > > > - Stewart > > > > > > PW Endpoint Fast Failure Protection > > draft-ietf-pwe3-endpoint-fast-protection-01 > > > > Abstract > > > > This document specifies a fast mechanism for protecting pseudowire= s > > against egress endpoint failures, including egress attachment circ= uit > > failure, egress PE failure, multi-segment PW terminating PE failur= e, > > and multi-segment PW switching PE failure. Designed on the basis = of > > multi-homed CE, PW redundancy, upstream label assignment and > context > > specific label switching, the mechanism enables local repair to be > > performed by the router upstream adjacent to a failure. > > > > In > > particular, the router can restore PW traffic in the order of tens= of > > milliseconds, > > > > SB> Not sure it's wise to specify a performance claim in a standards > > SB> track RFC. > > > > [yshen] If this is viewed as inappropriate, we can remove it. > > > > by transmitting the traffic to a protector through a > > pre-established bypass tunnel. Therefore, the mechanism can reduc= e > > traffic loss before global repair reacts to the failure and the > > network converges on the topology changes due to the failure. > > > > > > > > =3D=3D=3D > > > > > > 1. Introduction > > > > Per RFC 3985, RFC 4447 and RFC 5659, a pseudowire (PW) or PW segme= nt > > can be thought of as a connection between a pair of forwarders hos= ted > > by two PEs, carrying an emulated layer-2 service over a packet > > switched network (PSN). In the single-segment PW (SS-PW) case, a > > forwarder binds a PW to an attachment circuit (AC). In the multi- > > segment PW (MS-PW) case, a forwarder on a terminating PE (T-PE) bi= nds > > a PW segment to an AC, while a forwarder on a switching PE (S-PE) > > binds one PW segment to another PW segment. In each direction > > between the PEs, PW packets are transported by a PSN tunnel, which= is > > called a transport tunnel. > > > > SB> Did we ever your the term transport tunnel before? > > > > [yshen] This is mainly for clarity. We could change it to " which is > > called a transport tunnel *in this document*". Or we can completel= y > replace "transport tunnel" with just "tunnel". > > > > 3. Reference Models and Failure Cases > > > > The mechanism in this document intends to > > use these topologies for local repair purposes. > > > > SB> The toplology is not a mechanism. Are you saying that the mechanims > > SB> only works in this topology? > > > > [yshen] No, these topologies are only examples for use by subsequent > sections to describe the mechanisms. The mechanisms should equally work > for other topologies as well. > > > > This SHALL enable > > local repair and global repair to work in tandem to achieve broade= r > > coverage of protection for services. > > > > SB> I have no idea how a SHALL in capitals applies here. > > > > [yshen] Will fix this. > > > > 3.1. Single-Segment PW > > > > > > > > |<-------------- PW1 --------------->| > > > > - PE1 -------------- P1 ---------------- PE2 - > > / \ > > / \ > > CE1 CE2 > > \ / > > \ / > > - PE3 -------------- P2 ---------------- PE4 - > > > > |<-------------- PW2 --------------->| > > > > Figure 1 > > > > > > Each CE is multi-homed to two PEs. Hence, there are two divergent > > paths between the CEs. > > > > SB> It does not follow that the paths are divergent! Who knows > > SB> what happens in the IP layer and the underlying transport > > SB> network. > > > > > > [yshen] We could rephrase this and remove "divergent". All we want to s= ay > here is that there are two distinct sets of AC-PW-AC between the CEs. > > > > At any given time, each CE sends traffic via only one AC and recei= ves > > traffic via only one AC. The two ACs MAY or MAY NOT be the same. > > > > SB> May not be the same what? If they are actually the same AC you > > SB> have no redundancy. > > > > [yshen] Will rephrase it. All we want to say is that at a given time, t= he CE > may send traffic over one AC and receive traffic over another AC. > > > > The AC used to send traffic is determined by the CE, and MAY rely = on > > an end-to-end OAM mechanism between the CEs. The AC used for the > CE > > to receive traffic is determined by the state of the network and t= he > > protection mechanism in use, as described later in this document. > > > > From the perspective of traffic flowing towards a given CE, the se= t > > of PWs, PEs and ACs involved can be viewed to serve primary and > > backup (or active and standby) roles. When the network is in a > > steady state, the PW that is intended to carry the traffic is > > referred to as a primary PW. The PE at the egress of the primary = PW > > is a primary PE. The AC connecting the CE and the primary PE is a > > primary AC. The other PW may be used to carry the traffic upon a > > network failure, and is referred to as a backup PW. The PE at the > > egress of the backup PW is a backup PE. The AC connecting the CE = and > > the backup PE is a backup AC. > > > > In this document, the following primary and backup roles are assig= ned > > for the traffic going from CE1 to CE2: > > > > Primary PW: PW1 > > > > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 5= ] > > > > > > Internet-Draft PW Endpoint Fast Failure Protection July 201= 4 > > > > > > Primary PE: PE2 > > > > Primary AC: CE2-PE2 > > > > Backup PW: PW2 > > > > Backup PE: PE4 > > > > Backup AC: CE2-PE4 > > > > In this case, an egress AC failure refers to the failure of the AC > > CE2-PE2. An egress node failure refers to the failure of PE2. > > > > The backup PE, backup PW and backup AC may be used to carry traffi= c > > after a PW endpoint failure, when CE1 and CE2 switches traffic to = PW2 > > in local repair or global repair, as described later in this > > document. > > > > |<-------------- PW1 --------------->| > > > > ------------- P1 ---------------- PE2 - > > / \ > > / \ > > CE1 -- PE1 CE2 > > \ / > > \ / > > ------------- P2 ---------------- PE4 - > > > > |<-------------- PW2 --------------->| > > > > Figure 2 > > > > Figure 2 shows another possible scenario, where CE1 is single-home= d > > to PE1, while CE2 remains multi-homed to PE2 and PE4. From the > > perspective of egress protection for the traffic from CE1 to CE2, > > this topology is not much different than Figure 1. However, for t= he > > > > SB> "not much different than Figure 1" is not helpful, either it is the > > same, or you need to explain the difference. > > > > [yshen] Will rephrase. The difference is actually described after > "However...". > > > > > > traffic in the direction from CE2 to CE1, PE1 must anticipate traf= fic > > on both PW1 and PW2, and sends it to CE1 over the AC CE1-PE1. > > > > SB> What does it mean to anticipate traffic. > > > > [yshen] To anticipate incoming traffic over either PW1 or PW2. Will > rephrase. > > > > 3.2. Multi-Segment PW > > > > > > > > > > > > > > > > > > > > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 6= ] > > > > > > Internet-Draft PW Endpoint Fast Failure Protection July 201= 4 > > > > > > |<--------------- PW1 --------------->| > > |<----- SEG1 ----->|<----- SEG2 ----->| > > > > - TPE1 -------------- SPE1 --------------- TPE2 - > > / \ > > / \ > > CE1 CE2 > > \ / > > \ / > > - TPE3 -------------- SPE2 --------------- TPE4 - > > > > |<----- SEG3 ----->|<----- SEG4 ----->| > > |<--------------- PW2 --------------->| > > > > Figure 3 > > > > Figure 3 shows a topology that is similar to Figure 1 but in an MS= -PW > > environment. PW1 and PW2 are both MS-PWs. PW1 is established > > between TPE1 and TPE2, and switched between segments SEG1 and > SEG2 at > > SPE1. PW2 is established between TPE3 and TPE4, and switched > between > > segments SEG3 and SEG4 at SPE2. CE1 is multi-homed to TPE1 and TP= E3. > > CE2 is multi-homed to TPE2 and TPE4. The transport tunnels of the= PW > > segments are not shown in this figure for clarity. > > > > In this document, the following primary and backup roles are assig= ned > > for the traffic going from CE1 to CE2: > > > > Primary PW: PW1 > > > > Primary T-PE: TPE2 > > > > Primary S-PE: SPE1 > > > > Primary AC: CE2-TPE2 > > > > Backup PW: PW2 > > > > Backup T-PE: TPE4 > > > > Backup S-PE: SPE2 > > > > Backup AC: CE2-TPE4 > > > > In this case, an egress AC failure refers to the failure of the AC > > CE2-TPE2. An egress node failure refers to the failure of TPE2. = An > > S-PE failure refers to the failure of SPE1. > > > > > > > > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 7= ] > > > > > > Internet-Draft PW Endpoint Fast Failure Protection July 201= 4 > > > > > > The backup T-PE, backup PW and backup AC are used for protecting t= he > > primary PW against egress AC failure and egress node failure. The > > backup S-PE and the backup PW are used for protecting the primary = PW > > against S-PE failure, as described later in this document. > > > > For consistency with the SS-PW scenario, primary T-PEs and a prima= ry > > S-PEs may simply be referred to as primary PEs in this document, > > where specifics is not required. Similarly, backup T-PEs and back= up > > S-PEs may be referred to as backup PEs. > > > > 4. Theory of Operation > > > > The fast protection mechanism in this document provides three type= s > > of protection for PWs, corresponding to the three types of failure= s > > described in Section 1. > > > > a. Egress AC protection > > > > b. Egress (T-)PE node protection > > > > c. S-PE node protection > > > > The mechanism assumes a multi-homed CE with connectivity to a > primary > > PE and a backup PE, and the existence of a backup PW in the networ= k. > > In S-PE node protection, it also assumes the existence of a backup > > S-PE on the backup PW. > > > > > > SB> Do the PWs need to be symmetric - could you have more > > S-PEs on one that you have on the other? > > > > [yshen] You could. For a given S-PE, as long as you can find another PE= to > serve as protector, it will work. > > > > 4.1. Local Repair and Protector > > > > The mechanism relies on local repair to be performed by routers > > SB> Do you mean routers or PEs? Routers of more likely LSRs > > operate at a different layer and have no business knowing about > > PWs. > > > > [yshen] routers (LSRs). Yes, they have no idea about PWs. This draft > doesn't assume they know or care about PWs. > > > > SB> Has this work been reviewed my MPLS WG? I am concerned > > that you are doing a bunch of layer violation without explaining > > the how this is achieved or whether it is acceptable within the > > MPLS architecture. > > > > [yshen] There is no layer violation. The PLR simply redirect traffic fr= om the > original tunnel to a bypass tunnel, by doing a top label swap, without > touching, changing or knowing the PW label. Please take a close look at t= he > text. > > > > upstream adjacent to failures. Each of these routers is referred = to > > as a "point of local repair" (PLR). A PLR MUST be able to detect = a > > failure by using a rapid mechanism, such as physical layer failure > > detection, Bidirectional Failure Detection (BFD) (RFC 5880), etc. = In > > anticipation of the failure, the PLR MUST also pre-establish a byp= ass > > PSN tunnel to a "protector", and pre-install a bypass route in the > > data plane. The bypass tunnel MUST have the property that it will > > not be affected by the topology changes in the event of the failur= e. [[Sasha]] Does this mean that the bypass tunnel MUST NOT use the link betwe= en the PLR and the protected PE? If yes, then how can this be done by normal LDP? Or do you intend to use so= me parts of CR-LDP for this? > > > > SB> This is far too imprecise - you need to separate operations and > > topologies in the PW layer from operations and topologies in the > > PSN layer > > > > [yshen] Similar to existing IP/MPLS fast-reroute RFCs, this draft doesn= 't > restrict local repair trigger to any specific failure detection mechanism= s. Any > mechanism will work. > > > > SB> In each case you need to be clear how you are detecting failure > > by specifying the MEPs and MIPs you are testing. > > > > Upon detecting the failure, the PLR MUST invoke the bypass route i= n > > the data plane, and reroute PW traffic to the protector through th= e > > bypass tunnel. The protector MUST in turn send the traffic to the > > target CE. > > > > SB> No the protector knows nothing about CEs - they are in the client > > layer, and you can only protect at the server layer or the server's > > server layer. > > > > [yshen] In collocated model, the protector is a PE. In the centralized = model, > the protector may be an LSR , and it needs to participate in LDP PW signa= ling > to understand PWs. See the related sections and LDP extensions in this dr= aft. > This centralized model is optional. > > > > This procedure is referred to as local repair. > > > > Different routers may serve as PLR and protector in different > > scenarios. > > > > SB> Remember they are not routers - they are xPEs or LSRs. > > > > [yshen] Again, in collocated model, the protector is a PE. In the centr= alized > model, the protector may be an LSR , and it needs to participate in LDP P= W > signaling to understand PWs. See the related sections and LDP extensions = in > this draft. But this centralized model is optional. > > [yshen] PLRs may be LSRs, but they don't need to know PWs, because all > they do is to swap top label (tunnel label) to a bypass tunnel's label. > > > > SB> In each of the following we need to note that these are > > not RFC3985 PEs, even if they take no part in the protection > > such as the PEs on the left. Indeed I am wondering if you need > > to update RFC3985 and RFC5659 > > > > [yshen] There is no need to update RFC3985 and RFC5659. > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 8= ] > > > > > > Internet-Draft PW Endpoint Fast Failure Protection July 201= 4 > > > > > > o In egress AC protection, the PLR is the primary PE that termina= tes > > the primary PW and hosts the primary AC, and the protector is t= he > > backup PE (Figure 4). > > > > |<-------------- PW1 --------------->| > > > > - PE1 -------------- P1 ---------------- PE2 - > > / PLR \ > > / | \ > > CE1 bypass| CE2 > > \ | / > > \ | / > > - PE3 -------------- P2 ---------------- PE4 - > > protector > > > > |<-------------- PW2 --------------->| > > > > Figure 4 > > > > > > o In egress PE node protection, the PLR is the penultimate hop > > router of the transport tunnel of the primary PW, and the > > protector is the backup PE (Figure 5). > > > > |<-------------- PW1 --------------->| > > > > - PE1 -------------- P1 ------- P3 ----- PE2 - > > / PLR \ \ > > / \ \ > > CE1 bypass\ CE2 > > \ \ / > > \ \ / > > - PE3 -------------- P2 ---------------- PE4 - > > protector > > > > |<-------------- PW2 --------------->| > > > > Figure 5 > > > > SB> Note that it is nothing like as clean as this, if as you later have= , > > some of P3's traffic needs to go to PE4 and some to PE5. > > Also what about the > > > > [yshen] Note that we are talking about a single PW, rather than multipl= e > PWs carried by a single tunnel. > > > > o In S-PE node protection, the PLR is the penultimate hop router = of > > the transport tunnel of the primary PW segment, and the protect= or > > is the backup S-PE (Figure 6). > > > > > > > > > > > > > > > > > > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 9= ] > > > > > > Internet-Draft PW Endpoint Fast Failure Protection July 201= 4 > > > > > > |<--------------- PW1 --------------->| > > |<----- SEG1 ----->|<----- SEG2 ----->| > > > > - TPE1 ----- P4 ----- SPE1 -------------- TPE2 - > > / PLR \ \ > > / \ \ > > CE1 bypass\ CE2 > > \ \ / > > \ \ / > > - TPE3 --------------- SPE2 -------------- TPE4 - > > protector > > > > |<----- SEG3 ----->|<----- SEG4 ----->| > > |<--------------- PW2 --------------->| > > > > Figure 6 > > > > A PLR can realize its role based on configuration or the signaling= of > > transport tunnel. For example, in the case where the transport > > tunnel is signaled by RSVP, the penultimate hop router could reali= ze > > > > SB> Could realize is not sufficient. The mechanism needs to be > > unconditional. > > > > [yshen] Will rephrase. > > > > that it is the PLR for egress (T-)PE or S-PE failure based on the = RRO > > in Resv message, which should indicate that the router is one hop > > away from the PE. The detail of how this could be achieved on a p= er- > > protocol basis is out of the scope of this document. > > > > SB> So where is it in scope? How do I find out how to do it? > > > > In all scenarios, when a PLR reroutes traffic through a bypass tun= nel > > to a protector during local repair, it MUST keep the label of the > > primary PW intact in the packets. This obviates the need for the = PLR > > to maintain bypass routes on a per-PW basis, and allows a single > > bypass tunnel to carry traffic for multiple PWs. > > > > SB> OK, but didn't I read something about PE protecting subsets of the > > egress traffic of other PEs? How would that work without per PW > information? > > > > [yshen] PWs to a given PE may be protected by multiple protectors. A se= t > of PWs sharing the same protector X can be carried by the same tunnel > (destined to the context ID of ). PWs protected by a diffe= rent > protector Y must be carried by different tunnel (destined to the context = ID of > ). The PLR of each tunnel doesn't need to know PW info. Al= l it > does is to swap tunnel label to a bypass tunnel's label, and the bypass t= unnel > will guide the traffic to the intended protector. > > > > The procedure also requires that the protector SHOULD be able to > > forward the traffic based on a PW label that is assigned by the > > primary PE, and ensure the traffic to eventually reach the target = CE. > > From the protector's perspective, this PW label is an upstream > > assigned label (RFC 5331). > > > > SB> .. but the protector may be a P router and know nothing about > > PWs. > > > > [yshen] Again, in collocated model, the protector is a PE. In the centr= alized > model, the protector may be an LSR , and it needs to participate in LDP P= W > signaling to understand PWs. See the related sections and LDP extensions = in > this draft. But this centralized model is optional. > > > > > > To accomplish this, the protector SHOULD > > learning the PW label from the primary PE prior to the failure, an= d > > install proper forwarding state for the PW label in a dedicated la= bel > > space for the primary PE. > > > > SB> This is a new PE to P router protocol > > > > [yshen] The LDP extension is defined in a subsequent section. > > > > During local repair, the protector SHOULD > > perform PW label lookup in this label space. > > > > SB> This has to be signed off by MPLS WG > > SB> How can this only be a SHOULD - why is it not a MUST? > > > > [yshen] Will rephrase. > > > > SB> What do you do to ensure that they even have compatible label > managers? > > > > [yshen] The LDP extensions defined in this draft covers how to negotiat= e > the capability. > > > > > > SB> Doesn't this mean a P router looking up two labels to do > > the repair? > > > > [yshen] PLR only swaps the top label (i.e. tunnel label) to bypass labe= l > table, when doing the local repair. > > > > The above examples have shown the scenarios where the protectors a= re > > backup (T/S-)PEs. In other scenarios, a protector may be a dedica= ted > > router that assumes such role, separate from the backup (T/S-)PE o= f a > > primary PW. During local repair, the PLR MUST still reroute traff= ic > > to the protector through a bypass tunnel. The protector MUST in t= urn > > send the traffic to the backup (T/S-)PE, which MUST then send the > > > > > > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 10= ] > > > > > > Internet-Draft PW Endpoint Fast Failure Protection July 201= 4 > > > > > > traffic to the target CE via a backup AC or a backup PW segment. > > More detail will be described in Section 4.3. > > > > 4.2. Context Identifier > > > > A protector MAY protect multiple primary PEs. > > > > SB> This requires that a P router knows about PWs - how does > > it do that? > > > > [yshen] The protector will perform context label forwarding. The bypass > tunnel points to a context label table, and the protector looks up the PW > label in that table. > > > > The protector MUST > > maintain a separate label space for each primary PE. Likewise, th= e > > PWs terminated on a primary PE MAY be protected by multiple > > protectors, each for a subset of the PWs. In any case, a given > > primary PW SHOULD be associated with one and only one pair of > > {primary PE, protector}. > > > > An IPv4/v6 address is assigned to each ordered pair of {primary PE= , > > protector} to facilitate protection establishment. This address i= s > > referred to as a "context identifier". It MUST be globally unique= , > > or unique in the address space of the network where the primary PE > > and the protector reside. > > > > SB> I think you need to make clear that this is not backwards compatibl= e > > with existing PW PEs. > > > > [yshen] PEs will need to support the extensions defined by this draft. > > > > SB> You also need to explain how the IP based context identifier > > mechanism works. Is it in the control plane or the data plane? > > > > 4.2.1. Semantics > > > > The semantics of a context identifier is twofold. > > > > o It identifies a primary PE and an associated protector. In oth= er > > words, it identifies a primary PE on a per protector basis. A > > given primary PE may be protected by multiple protectors, each = for > > a subset of the primary PWs terminated on the primary PE. A > > distinct context identifier MUST be assigned to the primary PE = and > > each protector. > > > > For each primary PW, its ingress PE MUST set up or resolve a > > transport tunnel with destination as the context identifier of = the > > {primary PE, protector}, rather than a private IP address of th= e > > primary PE. This not only allows the transport tunnel to reach > > the primary PE, but also conveys the identity of the protector = to > > the PLR(s) along the transport tunnel. Each PLR can in turn us= e > > this information to set up a bypass tunnel to the protector > > without relying on local configuration. > > > > o It indentifies the primary PE's label space on the protector. = The > > protector may protect PWs for multiple primary PEs. For each > > primary PE, it MUST maintain a separate label space to store th= e > > PW labels assigned by that primary PE. It MUST associate a PW > > label with a label space via the context identifier of the > > {primary PE, protector}, as described below. > > > > SB> Is this context identifier an IP address or an MPLS label? Where > > is it carried? > > > > [yshen] a context identifier is an IP address. If the tunnel is IP tunn= el, it is > carried in IP header as dest address. If the tunnel is MPLS tunnel, it is > indicated by the label. > > > > In addition to the normal LDP PW signaling, the primary PE MUST > > have a targeted LDP session with the protector, and advertise P= W > > labels to the protector via LDP Label Mapping messages (See > > > > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 11= ] > > > > > > Internet-Draft PW Endpoint Fast Failure Protection July 201= 4 > > > > > > Section 5 for detail). The primary PE MUST attach the context > > identifier to each message. Upon receiving the message, the > > protector MUST install the advertised PW label in the label spa= ce > > identified by the context identifier. > > > > SB> I am currently confused about how a data packet carries the PW > > label provided by another PE. > > > > When a PLR sets up or resolve a bypass tunnel to the protector,= it > > MUST set the destination to the context identifier, rather than= a > > private IP address of the protector. Once established, the byp= ass > > tunnel, with either its MPLS label or IP tunnel destination > > address, is used as the identifier of label space. On the > > protector, all PW packets received on the bypass tunnel MUST be > > forwarded based on a label lookup in that label space. > > > > SB> I do not understand the data plane structure that the above sentenc= e > > is describing. > > > > [yshen] This is basically the context label forwarding paradigm (RFC 53= 31). > From the protector's perspective, the bypass tunnel serves as a context, = and > indicates the label space of the primary PE where the PW label must be > looked up. > > > > > > 4.2.2. Advertisement and Path Computation > > > > Using a context identifier as destination for both transport tunne= l > > and bypass tunnel requires both the primary PE and the protector t= o > > advertise the context identifier via IGP as an IP address reachabl= e > > through both routers in routing domain and/or TE domain. This > > imposes the following requirements on path computation for these > > tunnels. > > > > o For the transport tunnel, the ingress PE MUST choose the primar= y > > PE as the actual endpoint. > > > > o For the bypass tunnel, the PLR MUST choose the protector as the > > actual endpoint. In egress (T-)PE node protection and S-PE nod= e > > protection, the bypass tunnel MUST avoid the primary (S-)PE. > > > > The detail of how the primary PE and the protector may advertise a > > context identifier is independent of this mechanism and out of the > > scope of this document. > > > > SB> Given that you give signalling data structures later, I am confused > > as to how this can be out of scope of this text. Surely this is needed > > to implement an interoperable solution? > > > > [yshen] There is no new extension needed for IGP. Both PEs advertise th= e > context ID. That's it. Will rephrase it. > > > > One approach would be to advertise it as a > > virtual proxy node connected to both routers, with the link betwee= n > > the proxy node and the primary PE having a more preferable IGP or = TE > > metric than the link between the proxy node and the protector. Th= e > > ultimate goal is for a path computation algorithm, such as CSPF > > (constrained shortest path first), LFA (RFC 5286) and MRT ([IP-LDP= - > > FRR-MRT]), to be able to compute the paths that meet the above > > requirements. > > > > 4.3. Protection Models > > > > There are two protection models based on the location of a protect= or. > > A network MAY use either model, or a combination of both. > > > > > > > > > > > > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 12= ] > > > > > > Internet-Draft PW Endpoint Fast Failure Protection July 201= 4 > > > > > > 4.3.1. Co-located Protector > > > > In this model, the protector is a backup PE that is directly > > connected to the target CE via a backup AC, or it is a backup S-PE= on > > a backup PW. That is, the protector is co-located with the backup > > (S-)PE. Examples of this model have been introduced in Figure 4, > > Figure 5 and Figure 6 in Section 4.1. > > > > In egress AC protection and egress PE node protection, when a > > protector receives traffic from the PLR, it forwards the traffic t= o > > the CE via the backup AC. This is shown in Figure 7, where PE2 is > > the PLR for egress AC failure, P3 is the PLR for PE2 failure, and = PE4 > > (the backup PE) is the protector. > > > > |<-------------- PW1 --------------->| > > > > - PE1 -------------- P1 ------- P3 ----- PE2 ---- > > / PLR \ PLR \ > > / \ | \ > > CE1 bypass\ |bypass CE2 > > \ \ | / > > \ \ | / > > - PE3 -------------- P2 ---------------- PE4 ---- > > protector > > > > |<-------------- PW2 --------------->| > > > > Figure 7 > > > > In S-PE node protection, when a protector receives traffic from th= e > > PLR, it MUST forward the traffic via the next segment of the backu= p > > PW. The T-PE of the backup PW MUST in turn forward the traffic to > > the CE via a backup AC. This is shown in Figure 8, where P4 is th= e > > PLR for SPE1 failure, and SPE2 (the backup S-PE) is the protector = for > > SPE1. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 13= ] > > > > > > Internet-Draft PW Endpoint Fast Failure Protection July 201= 4 > > > > > > |<--------------- PW1 --------------->| > > |<----- SEG1 ----->|<----- SEG2 ----->| > > > > - TPE1 ----- P4 ----- SPE1 -------------- TPE2 - > > / PLR \ \ > > / \ \ > > CE1 bypass\ CE2 > > \ \ / > > \ \ / > > - TPE3 --------------- SPE2 -------------- TPE4 - > > protector > > > > |<----- SEG3 ----->|<----- SEG4 ----->| > > |<--------------- PW2 --------------->| > > > > Figure 8 > > > > In the co-located protector model, the number of context identifie= rs > > needed by a network is the number of distinct {primary PE, backup = PE} > > pairs. From the perspective of scalability, the model is suitable > > for networks where the number of backup PEs for any given primary = PE > > is relatively small. > > > > 4.3.2. Centralized Protector > > > > In this model, the protector is a dedicated P router or PE router > > that serves the role. In egress AC protection and egress PE node > > protection, the protector MAY or MAY NOT be a backup PE with a dir= ect > > connection to the target CE. In S-PE node protection, the protect= or > > MAY or MAY NOT be a backup S-PE on the backup PW. > > > > In egress AC protection and egress PE node protection, when the > > protector receives traffic from the PLR, if the protector has a > > direct connection (i.e. backup AC) to the CE, it MUST forward the > > traffic to the CE via the backup AC, which is similar to Figure 7. > > Otherwise, it MUST forward the traffic to a backup PE, which MUST > > then forward the traffic to the CE via a backup AC. This is shown= in > > Figure 9, where the protector receives traffic from P3 (the PLR fo= r > > egress PE failure) or PE2 (the PLR for egress AC failure) and > > forwards the traffic to PE4 (the backup PE). The protector may be > > protecting other PWs as well, which is not shown in this figure fo= r > > clarity. > > > > SB> If you are going to cover the shared case, I think you need to > > make it much clearer how the PWs are identified from the received > > data packets. > > > > > > > > > > > > > > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 14= ] > > > > > > Internet-Draft PW Endpoint Fast Failure Protection July 201= 4 > > > > > > |<------------- PW1 --------------->| > > > > - PE1 ------------- P1 ------- P3 ----- PE2 -- > > / PLR \ PLR \ > > / \ / \ > > / bypass\ /bypass \ > > / \ / \ > > CE1 protector CE2 > > \ \ / > > \ \ / > > \ \ / > > \ \ / > > - PE3 ------------- P2 -----------------PE4 -- > > > > |<------------- PW2 --------------->| > > > > Figure 9 > > > > In S-PE node protection, when the protector receives traffic from = the > > PLR, if the protector is a backup S-PE of the backup PW, it MUST > > forward the traffic via the next segment of the backup PW, and the > > T-PE of the backup PW MUST forward the traffic to the CE via a bac= kup > > AC, which is similar to Figure 8. Otherwise, the protector MUST > > first forward the traffic to the backup S-PE, which MUST then forw= ard > > the traffic via the next segment of the backup PW. Finally, the T= -PE > > of the backup PW MUST forward the traffic to the CE via a backup A= C. > > This is shown in Figure 10, where the protector forwards traffic t= o > > SPE2 (the backup S-PE), SPE2 forwards the traffic to TPE4 via SEG4= , > > and TPE4 finally forwards traffic to CE2. The protector may be > > protecting other PW segments as well, which is not shown in this > > figure for clarity. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 15= ] > > > > > > Internet-Draft PW Endpoint Fast Failure Protection July 201= 4 > > > > > > |<--------------- PW1 --------------->| > > |<----- SEG1 ----->|<----- SEG2 ----->| > > > > - TPE1 ----- P4 ----- SPE1 -------------- TPE2 - > > / PLR \ \ > > / \ \ > > / bypass\ \ > > / \ \ > > CE1 protector CE2 > > \ \ / > > \ \ / > > \ \ / > > \ \ / > > - TPE3 --------------- SPE2 -------------- TPE4 - > > > > |<----- SEG3 ----->|<----- SEG4 ----->| > > |<--------------- PW2 --------------->| > > > > Figure 10 > > > > The centralized protector model provides the convenience for multi= ple > > primary PEs to share one protector. Each primary PE MAY only need > > the one protector to protect all of its PWs. From the perspective= of > > scalability, the number of context identifiers needed by a network > > MAY be bound to the number of primary PEs. > > > > 4.4. Transport Tunnel > > > > The ingress PE of a primary PW associates the PW with the primary > > egress PE through LDP signaling. In addition, as mentioned in > > Section 4.2.1, the ingress PE MUST associate the transport tunnel = of > > the PW with the context identifier of the {primary PE, protector}, > > and set up or resolve the transport tunnel by using the context > > identifier as destination. This not only ensures that PW traffic = be > > transported to the primary PE, but also facilitates bypass tunnel > > establishment at PLR(s), as the context identifier implies the > > identity of the protector as well. This is also the case for a > > multi-segment PW, where the ingress PE and egress PE are T/S-PEs. > > > > The association between the transport tunnel and the context > > identifier at the ingress PE MAY be achieved by configuration or a= n > > auto-discovery mechanism. In the later case, the ingress PE MAY > > learn the context identifier from the primary (egress) PE, if the > > primary PE advertises the context identifier as "third party next > > hop" in IPv4/v6 Interface_ID (RFC 3471, RFC 3472) in the LDP > > Label Mapping message of the primary PW. > > > > > > > > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 16= ] > > > > > > Internet-Draft PW Endpoint Fast Failure Protection July 201= 4 > > > > > > 4.5. Bypass Tunnel > > > > A PLR may protect multiple PWs associated with one or multiple pai= rs > > of {primary PE, protector}. The PLR MUST establish a bypass tunnel= to > > each protector for each distinct context identifier associated wit= h > > that protector. The destination of the bypass tunnel MUST be the > > context identifier (Section 4.2.1). The PLR may derive the contex= t > > identifier from the destination of the transport tunnel that > > traverses it. > > > > For examples, in Figure 7 and Figure 9, a bypass tunnel is > > established from PE2 (PLR for egress AC failure) to the protector, > > and another bypass tunnel is established from P3 (PLR for egress n= ode > > failure) to the protector. In Figure 8 and Figure 10, a bypass > > tunnel is established from P4 (PLR for S-PE failure) to the > > protector. > > > > During local repair, the PLR reroutes traffic to the protector > > through a bypass tunnel with PW label intact in the packets. This > > normally involves pushing a label to the label stack, if the bypas= s > > tunnel is an MPLS tunnel, or pushing an IP header to the packets, = if > > the bypass tunnel is an IP tunnel. > > > > SB> Surely it is normally a swap? > > > > [yshen] yes, this is just a label swap of the top label, from tunnel la= bel to > bypass tunnel label. > > > > SB> I am somewhat confused about what this all looks like when this is > > IP. The only PS/IP other than for SATOP are defined by L2TPv3 WG. > > There is no IP MS-PW definition that I am aware of. I think you need > > to be much clearer about what is going on when you discuss PW over IP > > egress repair. > > > > The protector MUST in turn > > forward the traffic based on the PW label. To achieve such kind o= f > > forwarding, the protector MUST rely on the bypass tunnel as a cont= ext > > to determine the primary PE's label space. If the bypass tunnel i= s > > an MPLS tunnel, the protector MUST have assigned a non-reserved la= bel > > to the bypass tunnel during the establishment of the bypass tunnel= , > > and hence this label can serve as the context. If the bypass tunn= el > > is an IP tunnel, the protector can simply rely on the context > > identifier carried as the destination address in IP header. > > > > A bypass tunnel MUST have the property that it is not affected by = the > > topology changes caused by the failure. Therefore, it can be used= to > > transmit traffic for local repair. It SHOULD remain effective, un= til > > the traffic is moved to another fully functional egress AC, PW and= /or > > transport tunnel. > > > > 4.6. Forwarding State on Protector > > > > A protector MUST learn PW labels from all the primary PEs that it > > protects (Section 5.2), and maintain the PW labels in respective > > label spaces of the primary PEs. In the control plane, a label sp= ace > > is identified by the context identifier of a pair of {primary PE, > > protector}. In the forwarding plane, it is indicated by the bypass > > tunnel(s) destined for the context identifier. > > > > SB> Are you saying that this mechanism mandates no PHP? > > > > [yshen] It mandates PHP for bypass tunnels only, as their labels serve = as > contexts on protectors. This is not mandatory for normal tunnels. > > > > > > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 17= ] > > > > > > Internet-Draft PW Endpoint Fast Failure Protection July 201= 4 > > > > > > 4.6.1. Examples of Co-located Protector > > > > In Figure 7, PE4 is a co-located protector that protects PW1 again= st > > egress AC failure and egress node failure. It maintains a label > > space for PE2, which is identified by the context identifier of {P= E2, > > PE4}. It learns PW1's label from PE2, and installs an forwarding > > entry for the label in that label space. The nexthop of the > > forwarding entry indicates a label pop with outgoing interface > > pointing to the backup AC CE2-PE4. > > > > SB> I am not sure what you are describing in the above. Is this an > > MPLS operation or a PW operation. If the latter it is not a simple > > label pop. > > > > [yshen] It is PW label pop. > > > > In Figure 8, SPE2 is a co-located protector that protects PW1 agai= nst > > S-PE failure. It maintains a label space for SPE1, which is > > identified by the context identifier of {SPE1, SPE2}. It learns > > SEG1's label from SPE1, and installs a forwarding entry in the lab= el > > space. The nexthop of the forwarding entry indicates a label swap= to > > SEG4's label. > > > > 4.6.2. Examples of Centralized Protector > > > > In the centralized protector model, for each primary PW of which t= he > > protector is not a backup (S-)PE, the protector MUST also learn th= e > > label of the backup PW from the backup (S-)PE (Section 5.3). This= is > > the backup (S-)PE that the protector will forward traffic to. The > > protector MUST install a forwarding entry with label swap from the > > primary PW's label to the backup PW's label. > > > > In Figure 9, the protector is a centralized protector that protect= s > > PW1 against egress AC failure and egress node failure. It maintai= ns > > a label space for PE2, which is identified by the context identifi= er > > of {PE2, protector}. It learns PW1's label from PE2, and PW2's lab= el > > from PE4. It installs a forwarding entry for PW1's label in the > > label space. The nexthop of the forwarding entry indicates a labe= l > > swap to PW2's label. > > > > In Figure 10, the protector is a centralized protector that protec= ts > > the PW segment SEG1 of PW1 against the node failure of SPE1. It > > maintains a label space for SPE1, which is identified by the conte= xt > > identifier of {SPE1, protector}. It learns SEG1's label from SPE1, > > and learns SEG3's label from SPE2. It installs a forwarding entry > > for SEG1's label in the label space. The nexthop of the forwardin= g > > entry indicates a label swap to SEG3's label. > > > > 5. LDP Extensions > > > > As described in previous sections, a targeted LDP session MUST be > > established between each pair of primary PE and protector. The > > primary PE sends Label Mapping message over this session to advert= ise > > primary PW labels to the protector. In the centralized protector > > > > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 18= ] > > > > > > Internet-Draft PW Endpoint Fast Failure Protection July 201= 4 > > > > > > model, a targeted LDP session MUST also be established between a > > backup (S-)PE and a protector. The backup PE sends Label Mapping > > message over this session to advertise backup PW labels to the > > protector. > > > > To facilitate the procedures, this document defines a new "Protect= ion > > FEC Element" TLV. The Label Mapping messages of both the LDP > > sessions above MUST carry this TLV to indicate the identity of a > > primary PW. Specifically, in the centralized protector model, the > > Protection FEC Element TLV advertised by a backup (S-)PE MUST matc= h > > the one advertised by the primary PE, so that the protector can > > associate the primary PW's label with the backup PW's label, and > > perform a label swap. > > > > This document also defines the encoding of Capability Parameter TL= V > > (RFC 5561) for a new "Egress Protection Capability", to allow a > > protector to announce its capability of processing the above > > Protection FEC Element TLV and performing context specific label > > switching for PW labels. > > > > The procedures in this section are only applicable, if the protect= or > > advertises the Egress Protection Capability, the primary PE suppor= ts > > the advertisement of the Protection FEC Element TLV, and in the > > centralized protector model, the backup PE also supports the > > advertisement of the Protection FEC Element TLV. > > > > 5.1. Egress Protection Capability TLV > > > > A protector MUST advertise the Egress Protection Capability TLV in > > its Initialization message and Capability message, over the LDP > > session with a primary PE. In the centralized protector model, th= e > > protector MUST also advertise the TLV over the LDP session with a > > backup PE. The TLV carries one or multiple context identifiers. = To > > the primary PE, the TLV SHOULD carry the context identifier of the > > {primary PE, protector}. In the centralized protector model, the T= LV > > SHOULD carry to the backup PE multiple context identifiers, one fo= r > > each {primary PE, protector} where the backup PE serves as a backu= p > > for the primary PE. This TLV SHOULD NOT be advertised by the prim= ary > > PE or the backup PE to the protector. > > > > The processing of the Egress Protection Capability TLV by a receiv= ing > > router SHOULD follow the procedures defined in RFC 5561. In > > particular, the router SHOULD advertise PW information to the > > protector by using the Protection FEC Element TLV, only after it h= as > > received the Egress Protection Capability TLV from the protector. = It > > SHOULD validate each context identifier included in the TLV, and > > advertise the information of only those PWs that are associated wi= th > > the context identifier. It SHOULD withdraw previously advertised > > > > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 19= ] > > > > > > Internet-Draft PW Endpoint Fast Failure Protection July 201= 4 > > > > > > Protection FEC TLVs, when the protector has withdrawn a previously > > advertised context identifier or the entire Egress Protection > > Capability TLV via Capability message. > > > > The encoding of the Egress Protection Capability TLV is defined as > > below. It conforms to the format of Capability Parameter TLV > > specified in RFC 5561. > > > > 0 1 2 3 > > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-= + > > |U|F| Egress Protection (TBD) | Length = | > > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-= + > > |S| Reserved | = | > > +-+-+-+-+-+-+-+-+ = | > > | = | > > ~ Capability Data =3D context identifier(s) = ~ > > | = | > > | +-+-+-+-+-+-+-+-= + > > | | > > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > > > Figure 11 > > > > The U-bit MUST be set to 1 so that a receiver MUST silently ignore > > this TLV if unknown to it, and continue processing the rest of the > > message. > > > > The F-bit MUST be set to 0 since this TLV is sent only in > > Initialization and Capability messages, which are not forwarded. > > > > The TLV Code Point is TBD. It needs to be assigned by IANA. > > > > The S-bit indicates whether the sender is advertising (S=3D1) or > > withdrawing (S=3D0) the capability. > > > > The "Capability Data" is encoded with the context identifier of th= e > > {primary PE, protector}. > > > > SB> where is the structure of {primary PE, protector} defined? > > > > [yshen] The Protection FEC Element TLV carries the context ID of {prima= ry > PE, protector}. The {primary PE, protector} tuple itself is not encoded. > > > > 5.2. PW Label Distribution from Primary PE to Protector > > > > A primary PE SHOULD advertise a primary PW's label to a protector = by > > sending a Label Mapping message. The message includes a Protectio= n > > FEC Element TLV (see Section 5.4 for encoding), and an Upstream- > > Assigned Label TLV (RFC 6389) encoded with the PW's label. The > > combination of the Protection FEC Element TLV and the PW label > > represents the primary PE's forwarding state for the PW. The Labe= l > > Mapping message SHOULD also carry an IPv4/v6 Interface_ID TLV (RFC > > > > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 20= ] > > > > > > Internet-Draft PW Endpoint Fast Failure Protection July 201= 4 > > > > > > 6389, RFC 3471) encoded with the context identifier of the {primar= y > > PE, protector}. > > > > The protector that receives this Label Mapping message SHOULD inst= all > > a forwarding entry for the PW label in the label space identified = by > > the context identifier. The nexthop of the forwarding entry SHOUL= D > > ensure packets to be sent towards the target CE via a backup AC or= a > > backup (S-)PE, depending on the protection scenario. The protecto= r > > SHOULD silently discard a Label Mapping message if the included > > context identifier is unknown to it. > > > > 5.3. PW Label Distribution from Backup PE to Protector > > > > In the centralized protector model, a backup PE SHOULD advertise a > > backup PW's label to a protector by sending a Label Mapping messag= e. > > The message includes a Protection FEC Element TLV and a Generic La= bel > > TLV encoded with the backup PW's label. This Protection FEC Eleme= nt > > MUST be identical to the Protection FEC Element TLV that the prima= ry > > PE advertises to the protector (Section 5.2). The context identif= ier > > SHOULD NOT be encoded in Interface_ID TLV in this message. > > > > The protector that receives this Label Mapping message SHOULD > > associate the backup PW with the primary PW, based on the common > > Protection FEC Element TLV. It SHOULD distinguish between the Lab= el > > Mapping message from the primary PE and the Label Mapping message > > from the backup PE based on the respective presence and absence of > > context identifier in Interface_ID TLV. It SHOULD install a > > forwarding entry for the primary PW's label in the label space > > identified by the context identifier. The nexthop of the forwardi= ng > > entry SHOULD indicate a label swap to the backup PW's label, follo= wed > > by a label push or IP header push for a transport tunnel to the > > backup PE. > > > > 5.4. Protection FEC Element TLV > > > > The Protection FEC Element TLV has type 0x83. Its format is defin= ed > > as below: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 21= ] > > > > > > Internet-Draft PW Endpoint Fast Failure Protection July 201= 4 > > > > > > 0 1 2 3 > > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-= + > > | Type(0x83) | Reserved | Encoding Type | Length = | > > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-= + > > | = | > > | = | > > ~ PW Information = ~ > > | = | > > | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-= + > > | | > > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > > > Figure 12 > > > > - Encoding Type > > > > Type of format that PW Information field is encoded. > > > > - Length > > > > Length of PW Information field in octets. > > > > - PW Information > > > > Field of variable length that specifies a PW > > > > For Encoding Type, 1 is defined for the PWid FEC Element format, a= nd > > 2 is defined for the Generalized PWid FEC Element format (RFC 4447= ). > > > > > > 5.4.1. Encoding Format for PWid > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 22= ] > > > > > > Internet-Draft PW Endpoint Fast Failure Protection July 201= 4 > > > > > > 0 1 2 3 > > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-= + > > | Type(0x83) | Reserved | Enc Type(1) | Length(16) = | > > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-= + > > | Ingress PE Address = | > > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-= + > > | Egress PE Address = | > > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-= + > > | Group ID = | > > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-= + > > | PW ID = | > > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-= + > > |C| PW Type | Reserved = | > > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-= + > > > > Figure 13 > > > > - Ingress PE Address > > > > IP address of the ingress PE of PW. > > > > - Egress PE Address > > > > IP address of the egress PE of PW. > > > > SB> How does this work for IPv6? > > > > - Group ID > > > > An arbitrary 32-bit value that represents a group of PWs and th= at > > is used to create groups in the PW space. > > > > - PW ID > > > > A non-zero 32-bit connection ID that, together with the PW Type > > field, identifies a particular PW. > > > > - Control word bit (C) > > > > A bit that flags the presence of a control word on this PW. If= C > > =3D 1, control word is present; If C =3D 0, control word is not > > present. > > > > - PW Type > > > > A 15-bit quantity that represents the type of PW. > > > > SB> Needs a ref > > > > > > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 23= ] > > > > > > Internet-Draft PW Endpoint Fast Failure Protection July 201= 4 > > > > > > 5.4.2. Encoding Format for Generalized PWid > > > > 0 1 2 3 > > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-= + > > | Type(0x83) | Reserved | Enc Type(2) | Length = | > > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-= + > > | Ingress PE Address = | > > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-= + > > | Egress PE Address = | > > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-= + > > |C| PW Type | Reserved = | > > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-= + > > | AGI Type | Length | Value = | > > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-= + > > ~ AGI Value (contd.) = ~ > > | = | > > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-= + > > | AII Type | Length | Value = | > > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-= + > > ~ SAII Value (contd.) = ~ > > | = | > > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-= + > > | AII Type | Length | Value = | > > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-= + > > ~ TAII Value (contd.) = ~ > > | = | > > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-= + > > > > Figure 14 > > > > - Ingress PE Address > > > > IP address of the ingress PE of PW. > > > > - Egress PE Address > > > > IP address of the egress PE of PW. > > > > SB> IPv6? > > > > - Control word bit (C) > > > > A bit that flags the presence of a control word on this PW. If= C > > =3D 1, control word is present; If C =3D 0, control word is not > > present. > > > > - PW Type > > > > A 15-bit quantity that represents the type of PW. > > > > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 24= ] > > > > > > Internet-Draft PW Endpoint Fast Failure Protection July 201= 4 > > > > > > - AGI Type, Length, Value, AGI Value > > > > Attachment Group Identifier of PW. > > > > - SAII Type, Length, Value, SAII Value > > > > Source Attachment Individual Identifier of PW. > > > > - TAII Type, Length, Value, TAII Value > > > > Target Attachment Individual Identifier of PW. > > > > 6. Revertive Behavior > > > > Subsequent to local repair, there are three strategies for the > > network to restore traffic to a fully functional PW. > > > > o Global revertive mode > > > > If the ingress CE is multi-homed (Figure 1), it MAY switch the > > traffic to a backup AC which is bound to a backup PW. > > Alternatively, if the ingress PE hosts a backup PW (Figure 2), = the > > ingress PE MAY switch the traffic to the backup PW. These > > procedures are referred to as global repair. Possible triggers= of > > a global repair include PW status, OAM, and BFD. > > > > o Control plane revertive mode > > > > In egress PE node protection and S-PE node protection, it is > > possible that the failure is limited to the link between the PL= R > > and the primary (S-)PE, whereas the primary (S-)PE is still up. > > In this case, the PLR or an upstream router along the transport > > tunnel MAY reroute the tunnel around the failed link via an > > alternative path. Thus, the transport tunnel can continue to b= e > > used to carry the PW traffic to the primary (S-)PE. This > > procedure is driven by control plane convergence, and is referr= ed > > to as control plane repair. > > > > o Local revertive mode > > > > The PLR MAY move traffic back to the primary PW, after the fail= ure > > is resolved. In egress AC protection, upon detecting that the > > primary AC is restored, the PLR MAY start forwarding traffic ov= er > > the AC again. Likewise, in egress PE node protection and S-PE > > node protection, upon detecting that the primary PE is restored= , > > the PLR MAY re-establish the primary transport tunnel through t= he > > primary PE, and move the traffic from the bypass tunnel back to > > > > > > > > > > Yimin Shen, et al. Expires January 25, 2015 [Page 25= ] > > > > > > Internet-Draft PW Endpoint Fast Failure Protection July 201= 4 > > > > > > the transport tunnel. These procedures are referred to as loca= l > > reversion. > > > > The fast protection mechanism in this document SHOULD be used in > > tandem with the global revertive mode. Particularly in the case o= f > > egress (S-)PE failure, if the ingress PE or the protector loses > > communication with the (S-)PE for an extensive period of time, the > > LDP session between them may go down. Consequently, the ingress P= E > > may bring down the primary PW, or the protector may remove the > > forwarding entry of the primary PW label. In either case, the > > service will be disrupted. In other words, although the fast > > protection can temporarily repair traffic, control plane state may > > eventually be timed out if the failure persists. Therefore, it is > > recommended that the global revertive mode SHOULD be set up in > > advance, so that traffic can be moved to a fully functional backup= PW > > shortly after the local repair. > > > > The control plane revertive mode may always happen as part of the > > convergence of control plane protocols. However, it is only > > applicable to the specific scenarios described above. > > > > The local revertive mode is optional. In the circumstances where = the > > failure is caused by resource flapping, local reversion MAY be > > dampened to limit potential disruptions. Local revertive mode MAY= be > > disabled completely by configuration. > > > > 7. IANA Considerations > > > > This document defines the encoding of the Capability Parameter TLV > > for the new "Egress Protection Capability" in Section 5. This wou= ld > > require IANA to assign a TLV Code Point to it. > > > > This document defines a new LDP Protection FEC Element TLV in > > Section 5. IANA has assigned the type value 0x83 to it. > > > > SB> I am very confused about what explicit actions you are asking IANA > > to take. Are you saying that there are no IANA actions? > > > > [yshen] Need IANA to assign a TLV code to "Egress Protection Capability= ", > as said above. > > > > 8. Security Considerations > > > > The security considerations discussed in RFC 5036, RFC 5331, RFC > > 3209, and RFC 4090 apply to this document. > > > > SB> Are there any PW security considerations that apply? > > > > SB> This is a new PW operational mode, so I am surprised that there are > > no new security considerations. An early security review might be usefu= l. > > > > > > [yshen] Not sure about any specific security issues. > > > > > > _______________________________________________ > > pwe3 mailing list > > pwe3@ietf.org > > https://www.ietf.org/mailman/listinfo/pwe3 > > . > > >=20 >=20 > -- > For corporate legal information go to: >=20 > http://www.cisco.com/web/about/doing_business/legal/cri/index.html >=20 > _______________________________________________ > pwe3 mailing list > pwe3@ietf.org > https://www.ietf.org/mailman/listinfo/pwe3 From nobody Tue Aug 5 06:38:20 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 490471B27D1; Tue, 5 Aug 2014 06:38:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -14.502 X-Spam-Level: X-Spam-Status: No, score=-14.502 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YwloADJvZ1WZ; Tue, 5 Aug 2014 06:38:10 -0700 (PDT) Received: from aer-iport-3.cisco.com (aer-iport-3.cisco.com [173.38.203.53]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 636721A8BB5; Tue, 5 Aug 2014 06:38:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=997; q=dns/txt; s=iport; t=1407245891; x=1408455491; h=message-id:date:from:reply-to:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=157zYB1qYYIXEXD7Szi0I4m9qtzFzogede/B9V6gN1c=; b=Md9bM+5Yn4wcJu5RJsWHlaHtxICZB6mIhdBzodiVI1aXXqbI2uxclkz/ qBJ3dbVouxcAtXWgkBAR0r02jZCAi7vojXuHWe4NJaQ5uhAEImD8akwFD H20chkT6nch2z25AXkfaF852hwFKB8i992K61WlbMBfrRFcTwr/1+FUYw 8=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqEEACrd4FOtJssW/2dsb2JhbABb13EBgSl3hAQBAQQdG0ABEAsSDxYPCQMCAQIBNw4GAQwBBwEBiD6zB5AeF4wdgy8HhEsBBJwNhyONPoIHgUc X-IronPort-AV: E=Sophos;i="5.01,804,1400025600"; d="scan'208";a="129128835" Received: from aer-iport-nat.cisco.com (HELO aer-core-1.cisco.com) ([173.38.203.22]) by aer-iport-3.cisco.com with ESMTP; 05 Aug 2014 13:38:08 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by aer-core-1.cisco.com (8.14.5/8.14.5) with ESMTP id s75Dc6Ag004210 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 5 Aug 2014 13:38:06 GMT Received: from [127.0.0.1] (localhost [127.0.0.1]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id s75Dc412009507; Tue, 5 Aug 2014 14:38:05 +0100 (BST) Message-ID: <53E0DE3E.5030901@cisco.com> Date: Tue, 05 Aug 2014 14:38:06 +0100 From: Stewart Bryant User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Alexander Vainshtein , Yimin Shen References: <53D7B569.60400@cisco.com> <53E0B85B.8060707@cisco.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/isFkdVchFcDOsySObnT8SQsdZ58 Cc: "mpls@ietf.org" , pwe3 , "pwe3-chairs@tools.ietf.org" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: stbryant@cisco.com List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Aug 2014 13:38:16 -0000 >>> [1] This draft is completely based on the PWE3 and the MS-PW >> architecture. It is also based on RFC 5331 " MPLS Upstream Label Assignment >> and Context-Specific Label Space". So ideally readers should be familiar with >> that RFC. >> As far as I can see context labels have not been introduced >> into the PWE3 architecture. There is some reference to their use for >> P2MP PWs, but not in the P2P case. Indeed I think that RFC4447 >> notes explicitly that in the case where the configuration of PWs >> is signaled by LDP the platform label space must be used. The >> least that you need to do is to update RFC4447. >> > [[Sasha]] yes, RFC 4447 explicitly requires the PW labels to be allocated from the per-platform label space. > And according to RFC 5331 per-platform label space is a special case of the label context. However RRC5331 does not update RFC4447, so it must be assumed that the original definition of per-platform label space applies to PWs. - Stewart From nobody Tue Aug 5 07:00:08 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1B77D1B27C6; Tue, 5 Aug 2014 06:59:57 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.902 X-Spam-Level: X-Spam-Status: No, score=-1.902 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id G-_8fT1Ea7xE; Tue, 5 Aug 2014 06:59:54 -0700 (PDT) Received: from emea01-db3-obe.outbound.protection.outlook.com (mail-db3lrp0082.outbound.protection.outlook.com [213.199.154.82]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0141B1B27BC; Tue, 5 Aug 2014 06:59:53 -0700 (PDT) Received: from AM3PR03MB612.eurprd03.prod.outlook.com (10.242.110.144) by AM3PR03MB419.eurprd03.prod.outlook.com (10.242.110.148) with Microsoft SMTP Server (TLS) id 15.0.995.14; Tue, 5 Aug 2014 13:59:52 +0000 Received: from AM3PR03MB612.eurprd03.prod.outlook.com (10.242.110.144) by AM3PR03MB612.eurprd03.prod.outlook.com (10.242.110.144) with Microsoft SMTP Server (TLS) id 15.0.995.14; Tue, 5 Aug 2014 13:59:50 +0000 Received: from AM3PR03MB612.eurprd03.prod.outlook.com ([10.242.110.144]) by AM3PR03MB612.eurprd03.prod.outlook.com ([10.242.110.144]) with mapi id 15.00.0995.014; Tue, 5 Aug 2014 13:59:50 +0000 From: Alexander Vainshtein To: "stbryant@cisco.com" Thread-Topic: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 Thread-Index: AQHPsLKATtmDXehXFE2xGhx0gc5WeZvCCNcA Date: Tue, 5 Aug 2014 13:59:49 +0000 Message-ID: <9696d0db139d46ffaad7be11340215e8@AM3PR03MB612.eurprd03.prod.outlook.com> References: <53D7B569.60400@cisco.com> <53E0B85B.8060707@cisco.com> <53E0DE3E.5030901@cisco.com> In-Reply-To: <53E0DE3E.5030901@cisco.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [147.234.56.21] x-microsoft-antispam: BCL:0;PCL:0;RULEID: x-forefront-prvs: 02945962BD x-forefront-antispam-report: SFV:NSPM; SFS:(6009001)(199002)(51694002)(189002)(51704005)(13464003)(252514010)(377454003)(101416001)(76576001)(66066001)(74316001)(86362001)(74662001)(83072002)(80022001)(74502001)(20776003)(87936001)(106356001)(105586002)(81542001)(85306004)(106116001)(21056001)(64706001)(46102001)(81342001)(107046002)(85852003)(79102001)(33646002)(4396001)(77982001)(92566001)(110136001)(54356999)(99396002)(50986999)(19580405001)(76176999)(2656002)(2351001)(95666004)(93886004)(83322001)(19580395003)(76482001)(31966008)(24736002)(108616003); DIR:OUT; SFP:; SCL:1; SRVR:AM3PR03MB612; H:AM3PR03MB612.eurprd03.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; LANG:en; Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Microsoft-Antispam: BCL:0;PCL:0;RULEID: X-OriginatorOrg: ecitele.com Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/qqP6sOYXKdrfEg-2-TL0fPzGcGY Cc: "mpls@ietf.org" , pwe3 , "pwe3-chairs@tools.ietf.org" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Aug 2014 13:59:57 -0000 Stewart, I fully agree with you that using PW labels from a label space that is not= a per-platform one requires an explicit update to RFC 4447. My comment on this point has been triggered by your saying that you *think*= that RFC 4447 explicitly requires per-platform label space. Just wanted to clarify that this is indeed the case. Regards, Sasha=20 Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 > -----Original Message----- > From: Stewart Bryant [mailto:stbryant@cisco.com] > Sent: Tuesday, August 05, 2014 4:38 PM > To: Alexander Vainshtein; Yimin Shen > Cc: mpls@ietf.org; pwe3; pwe3-chairs@tools.ietf.org > Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast- > protection-01 - RFC4447 >=20 >=20 > >>> [1] This draft is completely based on the PWE3 and the MS-PW > >> architecture. It is also based on RFC 5331 " MPLS Upstream Label > >> Assignment and Context-Specific Label Space". So ideally readers > >> should be familiar with that RFC. > >> As far as I can see context labels have not been introduced into the > >> PWE3 architecture. There is some reference to their use for P2MP PWs, > >> but not in the P2P case. Indeed I think that RFC4447 notes explicitly > >> that in the case where the configuration of PWs is signaled by LDP > >> the platform label space must be used. The least that you need to do > >> is to update RFC4447. > >> > > [[Sasha]] yes, RFC 4447 explicitly requires the PW labels to be allocat= ed > from the per-platform label space. > > And according to RFC 5331 per-platform label space is a special case of= the > label context. > However RRC5331 does not update RFC4447, so it must be assumed that the > original definition of per-platform label space applies to PWs. >=20 > - Stewart >=20 From nobody Tue Aug 5 07:54:01 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7915E1B2852; Tue, 5 Aug 2014 07:53:57 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.902 X-Spam-Level: X-Spam-Status: No, score=-1.902 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nwQABhmy5u-L; Tue, 5 Aug 2014 07:53:52 -0700 (PDT) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1blp0183.outbound.protection.outlook.com [207.46.163.183]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2215E1B27F6; Tue, 5 Aug 2014 07:53:51 -0700 (PDT) Received: from BY2PR05MB728.namprd05.prod.outlook.com (10.141.223.25) by BY2PR05MB728.namprd05.prod.outlook.com (10.141.223.25) with Microsoft SMTP Server (TLS) id 15.0.995.14; Tue, 5 Aug 2014 14:53:49 +0000 Received: from BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) by BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) with mapi id 15.00.0995.014; Tue, 5 Aug 2014 14:53:49 +0000 From: Yimin Shen To: Alexander Vainshtein , "stbryant@cisco.com" Thread-Topic: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 Thread-Index: AQHPsLKAX28iK6GR00WfEHEZOVjN8pvCCaWAgAAAMWA= Date: Tue, 5 Aug 2014 14:53:49 +0000 Message-ID: <35ea5a386fa84b498eaf682370f7c6d4@BY2PR05MB728.namprd05.prod.outlook.com> References: <53D7B569.60400@cisco.com> <53E0B85B.8060707@cisco.com> <53E0DE3E.5030901@cisco.com> <9696d0db139d46ffaad7be11340215e8@AM3PR03MB612.eurprd03.prod.outlook.com> In-Reply-To: <9696d0db139d46ffaad7be11340215e8@AM3PR03MB612.eurprd03.prod.outlook.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [66.129.241.10] x-microsoft-antispam: BCL:0;PCL:0;RULEID: x-forefront-prvs: 02945962BD x-forefront-antispam-report: SFV:NSPM; SFS:(6009001)(51694002)(252514010)(189002)(199002)(164054003)(51704005)(377454003)(13464003)(2656002)(77982001)(79102001)(80022001)(105586002)(95666004)(101416001)(66066001)(64706001)(106116001)(99286002)(106356001)(4396001)(81542001)(20776003)(99396002)(74662001)(74316001)(74502001)(85306004)(92566001)(46102001)(21056001)(76576001)(76176999)(31966008)(87936001)(19580395003)(93886004)(81342001)(83322001)(50986999)(83072002)(54356999)(33646002)(85852003)(107046002)(86362001)(76482001)(19580405001)(24736002)(108616003); DIR:OUT; SFP:; SCL:1; SRVR:BY2PR05MB728; H:BY2PR05MB728.namprd05.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; LANG:en; Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: juniper.net Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/FGoNvnhrzKtLj7DMbCUcnjv0SUI Cc: "mpls@ietf.org" , pwe3 , "pwe3-chairs@tools.ietf.org" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Aug 2014 14:53:57 -0000 Hi Stewart, Sasha, Every PE allocates PW labels from per-platform label space. This draft does= n't change this. In addition, a protector maintains a separate label table = for each primary/protected PE that it protects. This label table contains t= he PW labels allocated by the primary/protected PE from its own "per-platfo= rm" label space. Hope this clarifies it. Thanks, /Yimin -----Original Message----- From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com]=20 Sent: Tuesday, August 05, 2014 10:00 AM To: stbryant@cisco.com Cc: mpls@ietf.org; pwe3; pwe3-chairs@tools.ietf.org; Yimin Shen Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 - RFC4447 Stewart, I fully agree with you that using PW labels from a label space that is not= a per-platform one requires an explicit update to RFC 4447. My comment on this point has been triggered by your saying that you *think*= that RFC 4447 explicitly requires per-platform label space. Just wanted to clarify that this is indeed the case. Regards, Sasha=20 Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 > -----Original Message----- > From: Stewart Bryant [mailto:stbryant@cisco.com] > Sent: Tuesday, August 05, 2014 4:38 PM > To: Alexander Vainshtein; Yimin Shen > Cc: mpls@ietf.org; pwe3; pwe3-chairs@tools.ietf.org > Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast- > protection-01 - RFC4447 >=20 >=20 > >>> [1] This draft is completely based on the PWE3 and the MS-PW > >> architecture. It is also based on RFC 5331 " MPLS Upstream Label > >> Assignment and Context-Specific Label Space". So ideally readers > >> should be familiar with that RFC. > >> As far as I can see context labels have not been introduced into the > >> PWE3 architecture. There is some reference to their use for P2MP PWs, > >> but not in the P2P case. Indeed I think that RFC4447 notes explicitly > >> that in the case where the configuration of PWs is signaled by LDP > >> the platform label space must be used. The least that you need to do > >> is to update RFC4447. > >> > > [[Sasha]] yes, RFC 4447 explicitly requires the PW labels to be allocat= ed > from the per-platform label space. > > And according to RFC 5331 per-platform label space is a special case of= the > label context. > However RRC5331 does not update RFC4447, so it must be assumed that the > original definition of per-platform label space applies to PWs. >=20 > - Stewart >=20 From nobody Tue Aug 5 09:12:32 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D66491B2A66; Tue, 5 Aug 2014 09:12:30 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -14.502 X-Spam-Level: X-Spam-Status: No, score=-14.502 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JLhYG2WCcMXA; Tue, 5 Aug 2014 09:12:22 -0700 (PDT) Received: from aer-iport-3.cisco.com (aer-iport-3.cisco.com [173.38.203.53]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id AD5981B2A4C; Tue, 5 Aug 2014 09:12:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=2873; q=dns/txt; s=iport; t=1407255142; x=1408464742; h=message-id:date:from:reply-to:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=ymRrSg6cUOTCj8PEjinba2PTql7JnrDqg43mtDLneD4=; b=HyW7lr5KXtHaJh4SRXc/q5l+/lkwp4vtvRIumQZ61XLG1Y8aP1FwdzY8 bltLbnzpxPBhvoAnZdCHbvKJTa4OoC+LiucAXBa8cextyWyt1qPDPNDHy 5rvogytSjadw7jgcWa3QJPRdtfTCeqCvgSHuIS/aQVjKzKvG1Eb6W22ae A=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AswEAGkB4VOtJssW/2dsb2JhbABbg19Xy3OHSAGBK3eEAwEBAQQdG0ABDAQLDgMBAwEBAQkWCAcJAwIBAgE0AwYIBgEMAQUCAQGIPg2zKpAhF4wdgk0RAS4iBwaERQEEnA2HI40+ggeBR2sBgQw X-IronPort-AV: E=Sophos;i="5.01,805,1400025600"; d="scan'208";a="129262628" Received: from aer-iport-nat.cisco.com (HELO aer-core-1.cisco.com) ([173.38.203.22]) by aer-iport-3.cisco.com with ESMTP; 05 Aug 2014 16:12:19 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by aer-core-1.cisco.com (8.14.5/8.14.5) with ESMTP id s75GCJxT020864 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 5 Aug 2014 16:12:19 GMT Received: from [127.0.0.1] (localhost [127.0.0.1]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id s75GCGLN011020; Tue, 5 Aug 2014 17:12:16 +0100 (BST) Message-ID: <53E10263.6040905@cisco.com> Date: Tue, 05 Aug 2014 17:12:19 +0100 From: Stewart Bryant User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Yimin Shen , Alexander Vainshtein References: <53D7B569.60400@cisco.com> <53E0B85B.8060707@cisco.com> <53E0DE3E.5030901@cisco.com> <9696d0db139d46ffaad7be11340215e8@AM3PR03MB612.eurprd03.prod.outlook.com> <35ea5a386fa84b498eaf682370f7c6d4@BY2PR05MB728.namprd05.prod.outlook.com> In-Reply-To: <35ea5a386fa84b498eaf682370f7c6d4@BY2PR05MB728.namprd05.prod.outlook.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/u4uQrh7y-h3VMay9MVdHpi3UZyo Cc: "mpls@ietf.org" , pwe3 , "pwe3-chairs@tools.ietf.org" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: stbryant@cisco.com List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Aug 2014 16:12:31 -0000 On 05/08/2014 15:53, Yimin Shen wrote: > Hi Stewart, Sasha, > > Every PE allocates PW labels from per-platform label space. This draft doesn't change this. In addition, a protector maintains a separate label table for each primary/protected PE that it protects. This label table contains the PW labels allocated by the primary/protected PE from its own "per-platform" label space. Hope this clarifies it. > > > Thanks, > > /Yimin This all needs to be much clearer in the draft. The protector in this context is I assume the receiving PE, not the protecting PE? S > > -----Original Message----- > From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] > Sent: Tuesday, August 05, 2014 10:00 AM > To: stbryant@cisco.com > Cc: mpls@ietf.org; pwe3; pwe3-chairs@tools.ietf.org; Yimin Shen > Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 > > Stewart, > I fully agree with you that using PW labels from a label space that is not a per-platform one requires an explicit update to RFC 4447. > My comment on this point has been triggered by your saying that you *think* that RFC 4447 explicitly requires per-platform label space. > Just wanted to clarify that this is indeed the case. > > Regards, > Sasha > Email: Alexander.Vainshtein@ecitele.com > Mobile: 054-9266302 > > >> -----Original Message----- >> From: Stewart Bryant [mailto:stbryant@cisco.com] >> Sent: Tuesday, August 05, 2014 4:38 PM >> To: Alexander Vainshtein; Yimin Shen >> Cc: mpls@ietf.org; pwe3; pwe3-chairs@tools.ietf.org >> Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast- >> protection-01 - RFC4447 >> >> >>>>> [1] This draft is completely based on the PWE3 and the MS-PW >>>> architecture. It is also based on RFC 5331 " MPLS Upstream Label >>>> Assignment and Context-Specific Label Space". So ideally readers >>>> should be familiar with that RFC. >>>> As far as I can see context labels have not been introduced into the >>>> PWE3 architecture. There is some reference to their use for P2MP PWs, >>>> but not in the P2P case. Indeed I think that RFC4447 notes explicitly >>>> that in the case where the configuration of PWs is signaled by LDP >>>> the platform label space must be used. The least that you need to do >>>> is to update RFC4447. >>>> >>> [[Sasha]] yes, RFC 4447 explicitly requires the PW labels to be allocated >> from the per-platform label space. >>> And according to RFC 5331 per-platform label space is a special case of the >> label context. >> However RRC5331 does not update RFC4447, so it must be assumed that the >> original definition of per-platform label space applies to PWs. >> >> - Stewart >> > . > -- For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/index.html From nobody Tue Aug 5 11:55:35 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4CAF31A0126 for ; Tue, 5 Aug 2014 11:55:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.902 X-Spam-Level: X-Spam-Status: No, score=-1.902 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wx2N437kLzxI for ; Tue, 5 Aug 2014 11:55:30 -0700 (PDT) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1lp0140.outbound.protection.outlook.com [207.46.163.140]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 68B231A0101 for ; Tue, 5 Aug 2014 11:55:29 -0700 (PDT) Received: from BY2PR05MB728.namprd05.prod.outlook.com (10.141.223.25) by BY2PR05MB221.namprd05.prod.outlook.com (10.242.41.154) with Microsoft SMTP Server (TLS) id 15.0.995.14; Tue, 5 Aug 2014 18:55:24 +0000 Received: from BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) by BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) with mapi id 15.00.0995.014; Tue, 5 Aug 2014 18:55:24 +0000 From: Yimin Shen To: Alexander Vainshtein Thread-Topic: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 Thread-Index: AQHPqByPzRsIO8QlbkK+kbMBZ9MDf5u3AuAAgAAZ0oCAABjewIABFRcAgADf/TCACF0AgIAA3TgQ Date: Tue, 5 Aug 2014 18:55:23 +0000 Message-ID: <2280b8e0d5f94a60badab91a2237181b@BY2PR05MB728.namprd05.prod.outlook.com> References: <201407251524.s6PFOPn92012@magenta.juniper.net> <53D79504.4050904@cisco.com> <04f15bedf2be4c7480d3e9ca01bdfd7f@BY2PR05MB728.namprd05.prod.outlook.com> <2597291f7b074922b690e4b06999cf1a@AM3PR03MB612.eurprd03.prod.outlook.com>, <30e13900814f41a681151971cbe9ebef@BY2PR05MB728.namprd05.prod.outlook.com> <1407215578702.64387@ecitele.com> In-Reply-To: <1407215578702.64387@ecitele.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [66.129.241.10] x-microsoft-antispam: BCL:0;PCL:0;RULEID: x-forefront-prvs: 02945962BD x-forefront-antispam-report: SFV:NSPM; SFS:(6009001)(37854004)(24454002)(377454003)(479174003)(252514010)(52604005)(43784003)(199002)(189002)(164054003)(13464003)(51704005)(19580405001)(19580395003)(33646002)(87936001)(77982001)(76482001)(2656002)(74502001)(15975445006)(74662001)(110136001)(107046002)(31966008)(83322001)(92566001)(74316001)(83072002)(85852003)(76576001)(99396002)(80022001)(66066001)(20776003)(50986999)(76176999)(54356999)(106356001)(21056001)(106116001)(46102001)(15202345003)(4396001)(85306004)(79102001)(81542001)(101416001)(105586002)(561944003)(64706001)(86362001)(95666004)(81342001)(99286002)(24736002)(108616003); DIR:OUT; SFP:; SCL:1; SRVR:BY2PR05MB221; H:BY2PR05MB728.namprd05.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; LANG:en; Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: juniper.net Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/x6s_TQXYIfTcY_j1GPALhXoozqw Cc: Yakov Rekhter , "pwe3@ietf.org" , "stbryant@cisco.com" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Aug 2014 18:55:34 -0000 Hi Sasha, All ACs and PWs in this draft are active at individual level. One set of AC= -PW-AC is used as backup to protect the other set of AC-PW-AC (i.e. primary= ). I think we both agree that CE behavior is out of the scope of PWE3. If you = want to run L3 between CEs, you could bundle the two ACs on CE as a single = logical L3 interface. I'm sure there are other approaches depending on the = type of CE. But again, this is out scope of PWE3. The "fast protection" refers to the ability to locally detect failure, rest= ore traffic, and reduce loss. CEs and PEs may go through any negotiation or= coordination to move traffic from the repaired PW to anther PW, at any pac= e. Before that happens, traffic has already been repaired by this mechanism= . Thanks, /Yimin -----Original Message----- From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com]=20 Sent: Tuesday, August 05, 2014 1:13 AM To: Yimin Shen Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin hi, I apologize for missing your response when it has been sent. I will try to address some of your comments without yet one more level of i= n-lining:-). [Yimin] This draft assumes active-active PWs and bidirectional ACs. [Sasha] It would be nice if this were explicitly stated in the draft. [Yimin] I think you are talking about how to run Layer 3 between CEs with a= n overlay model, which I don't think is in scope of PWE3 architecture [Sash= a] To the best of my understanding PWE3 is about an emulation of a native s= ervice. It does not have to be perfect, but limitations are supposed to be = explicitly presented in Applicability Statements. On the practical level, = if a PW mechanism is not good enough to carry L3 traffic between a pair of = CEs, what is it good for? And, last but not least, if CE1 and CE2 in my example were L2 switches runn= ing some flavor of STP, the situation would be no less disruptive. [Yimin] In case of egress AC failure, CE2 will detect the failure and start= to send traffic over PW2 (i.e. the backup PW).=20 [Sasha] This is an assumption about CE2 which is definitely out of scope of= the PWE3 architecture. Actually what you say is that CE2 should treat its = failure to receive traffic over an AC as a bi-directional AC failure.=20 [Yimin] CE1 should also start to send traffic over the backup PW2. This may= be triggered by many events, e.g. [1] PE1's notification to CE1 based on t= he PW status you mentioned above (but it is not mandatory); [Sasha] This is= possible of course, but, as you've said, not mandatory [Yimin] or [2] CE1 = receiving traffic from PE3.=20 [Sasha] But you've said that ACs are active-active. So why receiving someth= ing from PE3 would be of any interest? And of course, there is a case when only CE2 is dual-homed but CE1 is not.= .. Is the draft relevant in this scenario? [Yimin] In any case, it may take some time for CE1 to do so. Before this ha= ppens on CE1, the traffic is still going over PW1 (primary PW) which is bei= ng locally repaired by PE2. [Sasha] I'd say that the draft relies on some "dual-homing protocol" betwee= n a dual-homed CE and its PEs to provide for a bi-directional switch, and t= hat such a switch could take some time. I am not sure this counts for "fast= protection". Regards, Sasha ________________________________________ From: Yimin Shen Sent: Thursday, July 31, 2014 2:22 AM To: Alexander Vainshtein Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Hi Sasha, Thanks again for your comments. Please see inline... Thanks, /Yimin -----Original Message----- From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Wednesday, July 30, 2014 4:08 AM To: Yimin Shen Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin hi, Lots of thanks for a prompt response. Unfortunately, from my point of view, the issues I've raised still remain u= nresolved. Please see additional questions inline below. Regards, Sasha Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 > -----Original Message----- > From: Yimin Shen [mailto:yshen@juniper.net] > Sent: Tuesday, July 29, 2014 7:56 PM > To: Alexander Vainshtein; stbryant@cisco.com; Yakov Rekhter;=20 > pwe3@ietf.org > Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast- > protection-01 > > Hi Sasha, > > Thanks very much for the review and comments. Please see my responses=20 > inline. Hope they are helpful. > > > Thanks, > > /Yimin > > > > -----Original Message----- > From: pwe3 [mailto:pwe3-bounces@ietf.org] On Behalf Of Alexander=20 > Vainshtein > Sent: Tuesday, July 29, 2014 10:08 AM > To: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org > Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast- > protection-01 > > Stewart, Yakov and all, > I have started reading the draft, and it seems that it breaks quite a=20 > few common architectural assumptions. > > [yshen] This draft is fully based on the PWE3 architecture and the=20 > MS-PW architecture. > > The following text from Section 3.1 "Single-Segment PW" looks highly=20 > problematic to me: > > At any given time, each CE sends traffic via only one AC and receives > traffic via only one AC. The two ACs MAY or MAY NOT be the same. > The AC used to send traffic is determined by the CE, and MAY rely on > an end-to-end OAM mechanism between the CEs. The AC used for the CE > to receive traffic is determined by the state of the network and the > protection mechanism in use, as described later in this document. > . > > To the best of my order for the pair of interconnected CEs to be able=20 > to operate in this way each of them should treat the pair of ACs shown=20 > in Figure > 1 as a single logical entity. > E.g., if we are speaking about Ethernet PWs, each CE could treat=20 > Ethernet links supporting these two ACs as a single Link Aggregation=20 > Group. However, the draft does not mention any such thing (or at least I = did not find it). > > Do I miss something important here? > > [yshen] It is a logical view that the CE is dual-home to 2 PEs over 2=20 > distinct ACs. AC is a concept well defined in RFC 3985. In terms of=20 > the actual PE-CE connection in the case of Ethernet, the CE could have > 2 distinct Ethernet links or 2 VLANs over a single LAG. > [[Sasha]] If I read your response correctly, each dual-homed CE treats each= if its two ACs as an individual entity. I see this is as highly problematic. E.g., consider the case when CE1 and C= E2 are two CE routers running some link-state routing protocol between them= . - In the normal situation, CE1 probably would see its two ACs as two paral= lel bi-directional links. - With PW redundancy, one of these links (one that corresponds to the curre= nt Active PW) would be Up and the other one - Down - With your proposal, the typical link-state routing protocol between CE1 a= nd CE2 would not be able to form any adjacencies across any of these links,= because this process requires bi-directional link connectivity; so regardl= ess of whatever happens in the PW domain CE1 and CE2 would consider each ot= her as unreachable across this domain. Did I miss something? (I intentionally leave aside protocols like OLSR that= can handle unidirectional links). [yshen] I'm lost in the last 2 bullets. This draft assumes active-active PW= s and bidirectional ACs. I think you are talking about how to run Layer 3 b= etween CEs with an overlay model, which I don't think is in scope of PWE3 a= rchitecture. > I also wonder whether, in the single segment PW scenario, the same=20 > level of protection could not be achieved with ICCP (RFC 7275)=20 > running between each pair of PEs and, possibly with 4 (instead of 2)=20 > PWs. For the reference, ICCP (or RFC 7275) is not mentioned in the=20 > draft, even if a possible alternative solution to some of the problems=20 > it solves. ( > > [yshen] Thanks for pointing out ICCP. We will mention it in the next revi= sion. > In this draft we broadly refer to such kind of mechanism as "control=20 > plane repair". The purpose of this draft is to locally repair traffic=20 > at the failure point, in order to reduce traffic loss before ICCP or=20 > other control plane repair mechanism kicks in. [[Sasha]] I am not sure ICCP in this case would mean slow protection. Its f= unction here would be just to synchronize the state of PWs between PE1 and = PE3/PE2 and PE4, and this can be reasonably fast IMO. The Protection State = Coordination (PSC) protocol in RFC 6378 looks like an analog (with some str= etch of course), and it is supposed to provide pretty fast protection switc= hing. [yshen] The draft doesn't comment specifically on ICCP performance. General= ly speaking, local repair mechanism tends to have the least overhead when r= estoring traffic. This is because both failure detection and traffic restor= ation are done right in the dataplane. It doesn't rely on dataplane to cont= rol plane notification (for failure event) or control plane protocol. For t= his exact reason there have been various IP/MPLS fast-reroute mechanisms de= fined by IETF. Of course, in a given scenario where the delay in global rep= air or control plane repair is tolerable for users, they don't have to use = local repair. Local repair is for the case where it is desirable to reduce = traffic loss even further. Note that local repair is a temporary repair, an= d it doesn't prevent global repair or control plane repair to perform a per= manent repair. I believe the draft has specifically mentioned this. > > Stewart did not specify in his email which MPLS architecture=20 > principles he thinks are broken by this draft. > > I think, however, that some PWE3 architectural principles are=20 > definitely are broken, and this includes: > - Bi-directional nature of the PW - it seems that the draft allows=20 > each a PW be Up in one direction and Down in the other one > > [yshen] No, the draft does not allow that. It only says that we can=20 > send traffic of one direction over one PW, while sending traffic of=20 > the opposite direction over another PW (i.e. backup PW). Both PWs should = be up bidirectionally. > > [[Sasha]] Sorry but I do not understand that. The draft discusses something= like the "egress AC failure". If, say. PE1 detects that, I expect it woul= d send a PW Status with "PW not Forwarding" and "Local AC Transmit Fault" b= its set to PE2. Would this not mean that the PW between PE1 and PE2 is Down= ? [yshen] In case of egress AC failure, CE2 will detect the failure and start= to send traffic over PW2 (i.e. the backup PW). CE1 should also start to se= nd traffic over the backup PW2. This may be triggered by many events, e.g. = [1] PE1's notification to CE1 based on the PW status you mentioned above (b= ut it is not mandatory); or [2] CE1 receiving traffic from PE3. In any case= , it may take some time for CE1 to do so. Before this happens on CE1, the t= raffic is still going over PW1 (primary PW) which is being locally repaired= by PE2. > - Lack of dependencies between the PSN tunnels and the PWs they carry. > * Consider the network diagram shown in Figure 5 in Section 4.1 > * The text suggests that the tunnel between PE1 and PE2 must=20 > recognize PE4 as the "protector" node for the PW it carries > * What is supposed to happen if the tunnel between PE1 and PE2=20 > carries, say, 10 PWs, each with its own backup PE PE4-1, ... PE4-10)? > > [yshen] In your example, the 10 PWs are protected by 10 different=20 > protectors. In this case, you would need 10 different tunnels,=20 > destined to 10 different context IDs (one context ID per PE4-x>). If multiple PWs are protected by the same protector, they can be= carried by the same tunnel. > [[Sasha]] This looks like a non-scalable solution to me. [yshen] I'm not sure there is any scalability issue here. We basically need= one tunnel per context ID, i.e. per . In reality, t= he number of should not be high. The draft also spe= cifies the centralized model for reducing the number of context IDs. > > - Lack of dependencies between the mechanism used for the tunnel setup=20 > (LDP, RSVP-TE, manual configuration etc.) and the PWs this tunnel carries= . > * Is the protection model described in Section 4.1 compatible=20 > with Tunnel LSPs being set up by LDP? With, say, GRE tunnels carrying=20 > the PWs using MPLS-in-GRE? > > [yshen] Yes, it is compatible. The draft is generic on transport=20 > layer, and it allows tunnels to be LDP, RSVP-TE, GRE, etc. These=20 > protocols already provide sufficient machineries for setting up transport= tunnels and bypass tunnels. > Therefore, the draft doesn't require new extension to these LSP=20 > signaling protocols. > [[Sasha]] Sorry but this does not match your response to my previous questi= on/issue where you have explained that you effectively need a tunnel per PW= between PE1 and PE2. RSVP-TE can provide that; but how is LDP supposed to = do that? [yshen] Again, a tunnel can carry multiple PWs, if these PWs are protected = by the same protector. You could use 10 different protectors to protect 10 = PWs, but in reality when SP design their network, they should ask why they = have to do so. Regarding LDP, it is one FEC per context ID, and nothing spe= cial. > Last but not least, I would like to understand better how the draft is=20 > protecting MS-PWs against S-PE failures. > Specifically, I'd like to understand whether it is applicable to the=20 > case when each pair of T-PEs resides in its own AS, with two pairs of=20 > directly connected ASBRs between these two ASs acting as S-PEs. There=20 > would be no need to set up any tunnel LSPs between each pair of S-PEs in = this scenario. > I would appreciate an analysis of this scenario regarding ability to=20 > protect about the ASBR/S-PE failure. IMO it is a relevant scenario=20 > because MS-PWs are one of the methods to provide inter-AS services. > > [yshen] The MS-PW topology in this draft is based on the Figure-4 of=20 > RFC 5659, which is slightly different than the scenario you mentioned abo= ve. > [[Sasha]] Figure 4 in RFC 5659 does not say anything about inter-AS. > > However, the mechanism in this draft does apply to this scenario as=20 > well. It should work equally if S-PEs are one-hop or multi-hops away=20 > from each other. Please see the picture below. In this case, SPE1 ~ SPE4 = are ASBRs. > When SPE4 detects the failure of SPE3, it will do a local repair and=20 > redirect PW traffic to SPE4 which is the protector. The bypass between > SPE1 and SPE4 may be a one-hop tunnel or a multi-hop tunnel, depending=20 > on the actual topology between the 2 AS'. > [[Sasha]] This approach can work if SPE1 is directly connected to SPE4 beca= use in this case there would be no need for a tunnel LSP between them. I do not see a simple way to set up an LSP between SPE1 (in AS-1) and SPE4(= in AS-2) if such connectivity is not present; could you possibly expand on = how this is supposed to happen? [yshen] Sasha, if you don't have a bypass tunnel (due to specific topology = or policy or whatever), you won't get any IP/MPLS fast-reroute mechanism to= work :) However it doesn't mean the mechanism is broken. In your case, I d= on't see any technique issue why you cannot set up a inter-AS bypass from S= PE1-SPE2-SPE4. > > In any case, SPE4 will send PW traffic over a > tunnel to TPE4. > > |<--------------- PW1 ----------->| > > - TPE1 -------- SPE1 - SPE3 --------- TPE2 - > / PLR \ \ > / \ \ > CE1 bypass\ CE2 > \ \ / > \ \ / > - TPE3 -------- SPE2 - SPE4 --------- TPE4 - > protector > > |<--------------- PW2 ----------->| > > My 2c, > Sasha > Email: Alexander.Vainshtein@ecitele.com > Mobile: 054-9266302 > > > -----Original Message----- > > From: pwe3 [mailto:pwe3-bounces@ietf.org] On Behalf Of Stewart=20 > > Bryant > > Sent: Tuesday, July 29, 2014 3:35 PM > > To: Yakov Rekhter; pwe3@ietf.org > > Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast- > > protection-01 > > > > On 25/07/2014 16:24, Yakov Rekhter wrote: > > > Hi, > > > > > >> This email begins a second two week working group last call for > > >> http://tools.ietf.org/html/draft-ietf-pwe3-endpoint-fast-protection-= 01 . > > >> > > >> Please review the draft and post any comments to the PWE3 list. > > >> > > >> If you have read the draft, even if you don't have any comments,=20 > > >> but would like to see the draft progressed, please can you=20 > > >> indicate this to the > > list. > > >> > > >> This working group last call will end on Friday August 8th, 2014. > > > I would like to see the draft progressed. > > > > > > Yakov. > > > > > > _______________________________________________ > > > pwe3 mailing list > > > pwe3@ietf.org > > > https://www.ietf.org/mailman/listinfo/pwe3 > > > > > Yakov > > > > Doesn't this draft implicitly make a number of extensions to the=20 > > MPLS architecture? > > > > Stewart > > > > _______________________________________________ > > pwe3 mailing list > > pwe3@ietf.org > > https://www.ietf.org/mailman/listinfo/pwe3 > > _______________________________________________ > pwe3 mailing list > pwe3@ietf.org > https://www.ietf.org/mailman/listinfo/pwe3 From nobody Tue Aug 5 22:55:21 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8C48C1B2841 for ; Tue, 5 Aug 2014 22:55:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.602 X-Spam-Level: X-Spam-Status: No, score=-2.602 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7vgxmjRGkJNV for ; Tue, 5 Aug 2014 22:55:15 -0700 (PDT) Received: from emea01-am1-obe.outbound.protection.outlook.com (mail-am1lrp0014.outbound.protection.outlook.com [213.199.154.14]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A01301B2C7A for ; Tue, 5 Aug 2014 22:55:13 -0700 (PDT) Received: from AM3PR03MB609.eurprd03.prod.outlook.com (10.242.109.149) by AM3PR03MB579.eurprd03.prod.outlook.com (10.242.114.22) with Microsoft SMTP Server (TLS) id 15.0.995.14; Wed, 6 Aug 2014 05:55:11 +0000 Received: from AM3PR03MB612.eurprd03.prod.outlook.com (10.242.110.144) by AM3PR03MB609.eurprd03.prod.outlook.com (10.242.109.149) with Microsoft SMTP Server (TLS) id 15.0.995.14; Wed, 6 Aug 2014 05:55:09 +0000 Received: from AM3PR03MB612.eurprd03.prod.outlook.com ([10.242.110.144]) by AM3PR03MB612.eurprd03.prod.outlook.com ([10.242.110.144]) with mapi id 15.00.0995.014; Wed, 6 Aug 2014 05:55:09 +0000 From: Alexander Vainshtein To: Yimin Shen Thread-Topic: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 Thread-Index: AQHPqymWveYGLBru1kCsLuzIRj9tdJu3CWCAgAA8XgCAAPc/IIABBtiAgAg4EiaAAOtlgIAAs2DL Date: Wed, 6 Aug 2014 05:55:09 +0000 Message-ID: <1407304503657.72312@ecitele.com> References: <201407251524.s6PFOPn92012@magenta.juniper.net> <53D79504.4050904@cisco.com> <04f15bedf2be4c7480d3e9ca01bdfd7f@BY2PR05MB728.namprd05.prod.outlook.com> <2597291f7b074922b690e4b06999cf1a@AM3PR03MB612.eurprd03.prod.outlook.com>, <30e13900814f41a681151971cbe9ebef@BY2PR05MB728.namprd05.prod.outlook.com> <1407215578702.64387@ecitele.com>, <2280b8e0d5f94a60badab91a2237181b@BY2PR05MB728.namprd05.prod.outlook.com> In-Reply-To: <2280b8e0d5f94a60badab91a2237181b@BY2PR05MB728.namprd05.prod.outlook.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [164.40.145.153] x-microsoft-antispam: BCL:0;PCL:0;RULEID: x-forefront-prvs: 02951C14DC x-forefront-antispam-report: SFV:NSPM; SFS:(6009001)(43784003)(24454002)(51704005)(377454003)(252514010)(164054003)(199002)(189002)(13464003)(52604005)(479174003)(37854004)(93886004)(85306004)(80022001)(66066001)(15975445006)(21056001)(19580405001)(1941001)(31966008)(74502001)(19580395003)(83322001)(110136001)(15202345003)(64706001)(54356999)(50986999)(79102001)(76176999)(20776003)(74662001)(2656002)(87936001)(101416001)(76482001)(561944003)(77982001)(4396001)(99396002)(81342001)(92726001)(106116001)(46102001)(92566001)(107046002)(105586002)(95666004)(85852003)(86362001)(83072002)(36756003)(81542001)(106356001); DIR:OUT; SFP:; SCL:1; SRVR:AM3PR03MB609; H:AM3PR03MB612.eurprd03.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; LANG:en; Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Microsoft-Antispam: BCL:0;PCL:0;RULEID: X-OriginatorOrg: ecitele.com Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/2aFUHi_f1A9zYYbRR9O4xYSkuEM Cc: Yakov Rekhter , "pwe3@ietf.org" , "stbryant@cisco.com" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2014 05:55:20 -0000 Yimin hi!=0A= =0A= Please see a couple of comments inline below (marked [[Sasha]].=0A= =0A= Regards,=0A= Sasha=0A= ________________________________________=0A= From: Yimin Shen =0A= Sent: Tuesday, August 5, 2014 9:55 PM=0A= To: Alexander Vainshtein=0A= Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org=0A= Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01=0A= =0A= Hi Sasha,=0A= =0A= All ACs and PWs in this draft are active at individual level. One set of AC= -PW-AC is used as backup to protect the other set of AC-PW-AC (i.e. primary= ).=0A= =0A= I think we both agree that CE behavior is out of the scope of PWE3. =0A= =0A= [[Sasha]] Yes - provided that PWE3 matches reasonable CE expectations. If i= t is not so, it becomes very much in scope IMO.=0A= =0A= If you want to run L3 between CEs, you could bundle the two ACs on CE as a = single logical L3 interface. I'm sure there are other approaches depending = on the type of CE. But again, this is out scope of PWE3.=0A= =0A= [[Sasha]] Well, not really IMO. E.g., Section 7.2 of RFC 7225 specifically = defines a PWE3 solution for PW redundancy with multi-Chassis LAG. This solu= tion addresses the case when a dual-homed CE perceives a pair of Ethernet l= inks between itself and a pair of redundant PEs as a single Link Aggregatio= n Group, and effectively uses LACP as the dual-homing protocol. I'd say tha= t if you decide that CE should use some standard (or widely deployed) mecha= nism between itself and the PEs in order for PWE3 to provide a meaningful s= ervice, PWE3 has both to specify its expectations regarding the CE behavior= and the methods it uses to accommodate this behavior. =0A= An example that looks highly relevant to me with regard to your draft is li= ke following:=0A= =0A= The draft explicitly differentiates between "ingress" and "egress" AC failu= res and states that "Today, fast protection against ingress AC failure and = ingress (T-)PE failure is achievable by using a multi-homed CE and redunda= nt PWs.". Suppose that CE and PE employ some native mechanism (out of scope= of PWE3, of course) that would revert any unidirectional AC failure to a b= i-directional one in a very fast manner. (One example that comes to mind is= the LF/RF exchange mechanism oft 10-Gigabit Ethernet.) Would such a mechan= ism eliminate the need to protect against egress AC failures which seems t= o be one of the major drives for the draft?=0A= =0A= The "fast protection" refers to the ability to locally detect failure, rest= ore traffic, and reduce loss. CEs and PEs may go through any negotiation or= coordination to move traffic from the repaired PW to anther PW, at any pac= e. Before that happens, traffic has already been repaired by this mechanism= .=0A= =0A= =0A= Thanks,=0A= =0A= /Yimin=0A= =0A= =0A= =0A= -----Original Message-----=0A= From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com]=0A= Sent: Tuesday, August 05, 2014 1:13 AM=0A= To: Yimin Shen=0A= Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org=0A= Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01=0A= =0A= Yimin hi,=0A= I apologize for missing your response when it has been sent.=0A= =0A= I will try to address some of your comments without yet one more level of i= n-lining:-).=0A= =0A= [Yimin] This draft assumes active-active PWs and bidirectional ACs.=0A= [Sasha] It would be nice if this were explicitly stated in the draft.=0A= =0A= [Yimin] I think you are talking about how to run Layer 3 between CEs with a= n overlay model, which I don't think is in scope of PWE3 architecture [Sash= a] To the best of my understanding PWE3 is about an emulation of a native s= ervice. It does not have to be perfect, but limitations are supposed to be = explicitly presented in Applicability Statements. On the practical level, = if a PW mechanism is not good enough to carry L3 traffic between a pair of = CEs, what is it good for?=0A= And, last but not least, if CE1 and CE2 in my example were L2 switches runn= ing some flavor of STP, the situation would be no less disruptive.=0A= =0A= [Yimin] In case of egress AC failure, CE2 will detect the failure and start= to send traffic over PW2 (i.e. the backup PW).=0A= [Sasha] This is an assumption about CE2 which is definitely out of scope of= the PWE3 architecture. Actually what you say is that CE2 should treat its = failure to receive traffic over an AC as a bi-directional AC failure.=0A= [Yimin] CE1 should also start to send traffic over the backup PW2. This may= be triggered by many events, e.g. [1] PE1's notification to CE1 based on t= he PW status you mentioned above (but it is not mandatory); [Sasha] This is= possible of course, but, as you've said, not mandatory [Yimin] or [2] CE1 = receiving traffic from PE3.=0A= [Sasha] But you've said that ACs are active-active. So why receiving someth= ing from PE3 would be of any interest?=0A= And of course, there is a case when only CE2 is dual-homed but CE1 is not.= .. Is the draft relevant in this scenario?=0A= [Yimin] In any case, it may take some time for CE1 to do so. Before this ha= ppens on CE1, the traffic is still going over PW1 (primary PW) which is bei= ng locally repaired by PE2.=0A= [Sasha] I'd say that the draft relies on some "dual-homing protocol" betwee= n a dual-homed CE and its PEs to provide for a bi-directional switch, and t= hat such a switch could take some time. I am not sure this counts for "fast= protection".=0A= =0A= Regards,=0A= Sasha=0A= =0A= =0A= =0A= ________________________________________=0A= From: Yimin Shen =0A= Sent: Thursday, July 31, 2014 2:22 AM=0A= To: Alexander Vainshtein=0A= Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org=0A= Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01=0A= =0A= Hi Sasha,=0A= =0A= Thanks again for your comments. Please see inline...=0A= =0A= =0A= Thanks,=0A= =0A= /Yimin=0A= =0A= =0A= =0A= -----Original Message-----=0A= From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com]=0A= Sent: Wednesday, July 30, 2014 4:08 AM=0A= To: Yimin Shen=0A= Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org=0A= Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01=0A= =0A= Yimin hi,=0A= Lots of thanks for a prompt response.=0A= =0A= Unfortunately, from my point of view, the issues I've raised still remain u= nresolved. Please see additional questions inline below.=0A= =0A= Regards,=0A= Sasha=0A= Email: Alexander.Vainshtein@ecitele.com=0A= Mobile: 054-9266302=0A= =0A= > -----Original Message-----=0A= > From: Yimin Shen [mailto:yshen@juniper.net]=0A= > Sent: Tuesday, July 29, 2014 7:56 PM=0A= > To: Alexander Vainshtein; stbryant@cisco.com; Yakov Rekhter;=0A= > pwe3@ietf.org=0A= > Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-=0A= > protection-01=0A= >=0A= > Hi Sasha,=0A= >=0A= > Thanks very much for the review and comments. Please see my responses=0A= > inline. Hope they are helpful.=0A= >=0A= >=0A= > Thanks,=0A= >=0A= > /Yimin=0A= >=0A= >=0A= >=0A= > -----Original Message-----=0A= > From: pwe3 [mailto:pwe3-bounces@ietf.org] On Behalf Of Alexander=0A= > Vainshtein=0A= > Sent: Tuesday, July 29, 2014 10:08 AM=0A= > To: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org=0A= > Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-=0A= > protection-01=0A= >=0A= > Stewart, Yakov and all,=0A= > I have started reading the draft, and it seems that it breaks quite a=0A= > few common architectural assumptions.=0A= >=0A= > [yshen] This draft is fully based on the PWE3 architecture and the=0A= > MS-PW architecture.=0A= >=0A= > The following text from Section 3.1 "Single-Segment PW" looks highly=0A= > problematic to me:=0A= > =0A= > At any given time, each CE sends traffic via only one AC and receives= =0A= > traffic via only one AC. The two ACs MAY or MAY NOT be the same.=0A= > The AC used to send traffic is determined by the CE, and MAY rely on= =0A= > an end-to-end OAM mechanism between the CEs. The AC used for the CE= =0A= > to receive traffic is determined by the state of the network and the= =0A= > protection mechanism in use, as described later in this document.=0A= > .=0A= >=0A= > To the best of my order for the pair of interconnected CEs to be able=0A= > to operate in this way each of them should treat the pair of ACs shown=0A= > in Figure=0A= > 1 as a single logical entity.=0A= > E.g., if we are speaking about Ethernet PWs, each CE could treat=0A= > Ethernet links supporting these two ACs as a single Link Aggregation=0A= > Group. However, the draft does not mention any such thing (or at least I = did not find it).=0A= >=0A= > Do I miss something important here?=0A= >=0A= > [yshen] It is a logical view that the CE is dual-home to 2 PEs over 2=0A= > distinct ACs. AC is a concept well defined in RFC 3985. In terms of=0A= > the actual PE-CE connection in the case of Ethernet, the CE could have=0A= > 2 distinct Ethernet links or 2 VLANs over a single LAG.=0A= >=0A= [[Sasha]] If I read your response correctly, each dual-homed CE treats each= if its two ACs as an individual entity.=0A= I see this is as highly problematic. E.g., consider the case when CE1 and C= E2 are two CE routers running some link-state routing protocol between them= .=0A= - In the normal situation, CE1 probably would see its two ACs as two paral= lel bi-directional links.=0A= - With PW redundancy, one of these links (one that corresponds to the curre= nt Active PW) would be Up and the other one - Down=0A= - With your proposal, the typical link-state routing protocol between CE1 a= nd CE2 would not be able to form any adjacencies across any of these links,= because this process requires bi-directional link connectivity; so regardl= ess of whatever happens in the PW domain CE1 and CE2 would consider each ot= her as unreachable across this domain.=0A= Did I miss something? (I intentionally leave aside protocols like OLSR that= can handle unidirectional links).=0A= =0A= [yshen] I'm lost in the last 2 bullets. This draft assumes active-active PW= s and bidirectional ACs. I think you are talking about how to run Layer 3 b= etween CEs with an overlay model, which I don't think is in scope of PWE3 a= rchitecture.=0A= =0A= =0A= =0A= =0A= =0A= =0A= > I also wonder whether, in the single segment PW scenario, the same=0A= > level of protection could not be achieved with ICCP (RFC 7275)=0A= > running between each pair of PEs and, possibly with 4 (instead of 2)=0A= > PWs. For the reference, ICCP (or RFC 7275) is not mentioned in the=0A= > draft, even if a possible alternative solution to some of the problems=0A= > it solves. (=0A= >=0A= > [yshen] Thanks for pointing out ICCP. We will mention it in the next revi= sion.=0A= > In this draft we broadly refer to such kind of mechanism as "control=0A= > plane repair". The purpose of this draft is to locally repair traffic=0A= > at the failure point, in order to reduce traffic loss before ICCP or=0A= > other control plane repair mechanism kicks in.=0A= =0A= [[Sasha]] I am not sure ICCP in this case would mean slow protection. Its f= unction here would be just to synchronize the state of PWs between PE1 and = PE3/PE2 and PE4, and this can be reasonably fast IMO. The Protection State = Coordination (PSC) protocol in RFC 6378 looks like an analog (with some str= etch of course), and it is supposed to provide pretty fast protection switc= hing.=0A= =0A= [yshen] The draft doesn't comment specifically on ICCP performance. General= ly speaking, local repair mechanism tends to have the least overhead when r= estoring traffic. This is because both failure detection and traffic restor= ation are done right in the dataplane. It doesn't rely on dataplane to cont= rol plane notification (for failure event) or control plane protocol. For t= his exact reason there have been various IP/MPLS fast-reroute mechanisms de= fined by IETF. Of course, in a given scenario where the delay in global rep= air or control plane repair is tolerable for users, they don't have to use = local repair. Local repair is for the case where it is desirable to reduce = traffic loss even further. Note that local repair is a temporary repair, an= d it doesn't prevent global repair or control plane repair to perform a per= manent repair. I believe the draft has specifically mentioned this.=0A= =0A= =0A= =0A= =0A= =0A= >=0A= > Stewart did not specify in his email which MPLS architecture=0A= > principles he thinks are broken by this draft.=0A= >=0A= > I think, however, that some PWE3 architectural principles are=0A= > definitely are broken, and this includes:=0A= > - Bi-directional nature of the PW - it seems that the draft allows=0A= > each a PW be Up in one direction and Down in the other one=0A= >=0A= > [yshen] No, the draft does not allow that. It only says that we can=0A= > send traffic of one direction over one PW, while sending traffic of=0A= > the opposite direction over another PW (i.e. backup PW). Both PWs should = be up bidirectionally.=0A= >=0A= >=0A= [[Sasha]] Sorry but I do not understand that. The draft discusses something= like the "egress AC failure". If, say. PE1 detects that, I expect it woul= d send a PW Status with "PW not Forwarding" and "Local AC Transmit Fault" b= its set to PE2. Would this not mean that the PW between PE1 and PE2 is Down= ?=0A= =0A= [yshen] In case of egress AC failure, CE2 will detect the failure and start= to send traffic over PW2 (i.e. the backup PW). CE1 should also start to se= nd traffic over the backup PW2. This may be triggered by many events, e.g. = [1] PE1's notification to CE1 based on the PW status you mentioned above (b= ut it is not mandatory); or [2] CE1 receiving traffic from PE3. In any case= , it may take some time for CE1 to do so. Before this happens on CE1, the t= raffic is still going over PW1 (primary PW) which is being locally repaired= by PE2.=0A= =0A= =0A= =0A= =0A= =0A= =0A= > - Lack of dependencies between the PSN tunnels and the PWs they carry.=0A= > * Consider the network diagram shown in Figure 5 in Section 4.1=0A= > * The text suggests that the tunnel between PE1 and PE2 must=0A= > recognize PE4 as the "protector" node for the PW it carries=0A= > * What is supposed to happen if the tunnel between PE1 and PE2=0A= > carries, say, 10 PWs, each with its own backup PE PE4-1, ... PE4-10)?=0A= >=0A= > [yshen] In your example, the 10 PWs are protected by 10 different=0A= > protectors. In this case, you would need 10 different tunnels,=0A= > destined to 10 different context IDs (one context ID per PE4-x>). If multiple PWs are protected by the same protector, they can be= carried by the same tunnel.=0A= >=0A= [[Sasha]] This looks like a non-scalable solution to me.=0A= =0A= [yshen] I'm not sure there is any scalability issue here. We basically need= one tunnel per context ID, i.e. per . In reality, t= he number of should not be high. The draft also spe= cifies the centralized model for reducing the number of context IDs.=0A= =0A= =0A= =0A= =0A= =0A= >=0A= > - Lack of dependencies between the mechanism used for the tunnel setup=0A= > (LDP, RSVP-TE, manual configuration etc.) and the PWs this tunnel carries= .=0A= > * Is the protection model described in Section 4.1 compatible=0A= > with Tunnel LSPs being set up by LDP? With, say, GRE tunnels carrying=0A= > the PWs using MPLS-in-GRE?=0A= >=0A= > [yshen] Yes, it is compatible. The draft is generic on transport=0A= > layer, and it allows tunnels to be LDP, RSVP-TE, GRE, etc. These=0A= > protocols already provide sufficient machineries for setting up transport= tunnels and bypass tunnels.=0A= > Therefore, the draft doesn't require new extension to these LSP=0A= > signaling protocols.=0A= >=0A= [[Sasha]] Sorry but this does not match your response to my previous questi= on/issue where you have explained that you effectively need a tunnel per PW= between PE1 and PE2. RSVP-TE can provide that; but how is LDP supposed to = do that?=0A= =0A= [yshen] Again, a tunnel can carry multiple PWs, if these PWs are protected = by the same protector. You could use 10 different protectors to protect 10 = PWs, but in reality when SP design their network, they should ask why they = have to do so. Regarding LDP, it is one FEC per context ID, and nothing spe= cial.=0A= =0A= =0A= =0A= =0A= > Last but not least, I would like to understand better how the draft is=0A= > protecting MS-PWs against S-PE failures.=0A= > Specifically, I'd like to understand whether it is applicable to the=0A= > case when each pair of T-PEs resides in its own AS, with two pairs of=0A= > directly connected ASBRs between these two ASs acting as S-PEs. There=0A= > would be no need to set up any tunnel LSPs between each pair of S-PEs in = this scenario.=0A= > I would appreciate an analysis of this scenario regarding ability to=0A= > protect about the ASBR/S-PE failure. IMO it is a relevant scenario=0A= > because MS-PWs are one of the methods to provide inter-AS services.=0A= >=0A= > [yshen] The MS-PW topology in this draft is based on the Figure-4 of=0A= > RFC 5659, which is slightly different than the scenario you mentioned abo= ve.=0A= >=0A= [[Sasha]] Figure 4 in RFC 5659 does not say anything about inter-AS.=0A= >=0A= > However, the mechanism in this draft does apply to this scenario as=0A= > well. It should work equally if S-PEs are one-hop or multi-hops away=0A= > from each other. Please see the picture below. In this case, SPE1 ~ SPE4 = are ASBRs.=0A= > When SPE4 detects the failure of SPE3, it will do a local repair and=0A= > redirect PW traffic to SPE4 which is the protector. The bypass between=0A= > SPE1 and SPE4 may be a one-hop tunnel or a multi-hop tunnel, depending=0A= > on the actual topology between the 2 AS'.=0A= >=0A= [[Sasha]] This approach can work if SPE1 is directly connected to SPE4 beca= use in this case there would be no need for a tunnel LSP between them.=0A= I do not see a simple way to set up an LSP between SPE1 (in AS-1) and SPE4(= in AS-2) if such connectivity is not present; could you possibly expand on = how this is supposed to happen?=0A= =0A= [yshen] Sasha, if you don't have a bypass tunnel (due to specific topology = or policy or whatever), you won't get any IP/MPLS fast-reroute mechanism to= work :) However it doesn't mean the mechanism is broken. In your case, I d= on't see any technique issue why you cannot set up a inter-AS bypass from S= PE1-SPE2-SPE4.=0A= =0A= =0A= =0A= >=0A= > In any case, SPE4 will send PW traffic over a=0A= > tunnel to TPE4.=0A= >=0A= > |<--------------- PW1 ----------->|=0A= >=0A= > - TPE1 -------- SPE1 - SPE3 --------- TPE2 -=0A= > / PLR \ \=0A= > / \ \=0A= > CE1 bypass\ CE2=0A= > \ \ /=0A= > \ \ /=0A= > - TPE3 -------- SPE2 - SPE4 --------- TPE4 -=0A= > protector=0A= >=0A= > |<--------------- PW2 ----------->|=0A= >=0A= > My 2c,=0A= > Sasha=0A= > Email: Alexander.Vainshtein@ecitele.com=0A= > Mobile: 054-9266302=0A= >=0A= > > -----Original Message-----=0A= > > From: pwe3 [mailto:pwe3-bounces@ietf.org] On Behalf Of Stewart=0A= > > Bryant=0A= > > Sent: Tuesday, July 29, 2014 3:35 PM=0A= > > To: Yakov Rekhter; pwe3@ietf.org=0A= > > Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-=0A= > > protection-01=0A= > >=0A= > > On 25/07/2014 16:24, Yakov Rekhter wrote:=0A= > > > Hi,=0A= > > >=0A= > > >> This email begins a second two week working group last call for=0A= > > >> http://tools.ietf.org/html/draft-ietf-pwe3-endpoint-fast-protection-= 01 .=0A= > > >>=0A= > > >> Please review the draft and post any comments to the PWE3 list.=0A= > > >>=0A= > > >> If you have read the draft, even if you don't have any comments,=0A= > > >> but would like to see the draft progressed, please can you=0A= > > >> indicate this to the=0A= > > list.=0A= > > >>=0A= > > >> This working group last call will end on Friday August 8th, 2014.=0A= > > > I would like to see the draft progressed.=0A= > > >=0A= > > > Yakov.=0A= > > >=0A= > > > _______________________________________________=0A= > > > pwe3 mailing list=0A= > > > pwe3@ietf.org=0A= > > > https://www.ietf.org/mailman/listinfo/pwe3=0A= > > >=0A= > > Yakov=0A= > >=0A= > > Doesn't this draft implicitly make a number of extensions to the=0A= > > MPLS architecture?=0A= > >=0A= > > Stewart=0A= > >=0A= > > _______________________________________________=0A= > > pwe3 mailing list=0A= > > pwe3@ietf.org=0A= > > https://www.ietf.org/mailman/listinfo/pwe3=0A= >=0A= > _______________________________________________=0A= > pwe3 mailing list=0A= > pwe3@ietf.org=0A= > https://www.ietf.org/mailman/listinfo/pwe3=0A= From nobody Wed Aug 6 02:12:12 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 11D631A01F9 for ; Wed, 6 Aug 2014 02:12:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.601 X-Spam-Level: X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bl78tSNhUBiz for ; Wed, 6 Aug 2014 02:11:54 -0700 (PDT) Received: from emea01-am1-obe.outbound.protection.outlook.com (mail-am1lrp0011.outbound.protection.outlook.com [213.199.154.11]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 67BE51A0233 for ; Wed, 6 Aug 2014 02:11:22 -0700 (PDT) Received: from AM3PR03MB612.eurprd03.prod.outlook.com (10.242.110.144) by AM3PR03MB611.eurprd03.prod.outlook.com (10.242.109.28) with Microsoft SMTP Server (TLS) id 15.0.995.14; Wed, 6 Aug 2014 09:11:19 +0000 Received: from AM3PR03MB612.eurprd03.prod.outlook.com ([10.242.110.144]) by AM3PR03MB612.eurprd03.prod.outlook.com ([10.242.110.144]) with mapi id 15.00.0995.014; Wed, 6 Aug 2014 09:11:19 +0000 From: Alexander Vainshtein To: Yimin Shen Thread-Topic: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 Thread-Index: AQHPqymWveYGLBru1kCsLuzIRj9tdJu3CWCAgAA8XgCAAPc/IIABBtiAgAg4EiaAAOtlgIAAs2DLgAA1qVA= Date: Wed, 6 Aug 2014 09:11:18 +0000 Message-ID: References: <201407251524.s6PFOPn92012@magenta.juniper.net> <53D79504.4050904@cisco.com> <04f15bedf2be4c7480d3e9ca01bdfd7f@BY2PR05MB728.namprd05.prod.outlook.com> <2597291f7b074922b690e4b06999cf1a@AM3PR03MB612.eurprd03.prod.outlook.com>, <30e13900814f41a681151971cbe9ebef@BY2PR05MB728.namprd05.prod.outlook.com> <1407215578702.64387@ecitele.com>, <2280b8e0d5f94a60badab91a2237181b@BY2PR05MB728.namprd05.prod.outlook.com> <1407304503657.72312@ecitele.com> In-Reply-To: <1407304503657.72312@ecitele.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [147.234.56.21] x-microsoft-antispam: BCL:0;PCL:0;RULEID: x-forefront-prvs: 02951C14DC x-forefront-antispam-report: SFV:NSPM; SFS:(37854004)(24454002)(51704005)(13464003)(189002)(199002)(52604005)(252514010)(377454003)(43784003)(479174003)(164054003)(80022001)(85306004)(561944003)(66066001)(93886004)(76482001)(87936001)(46102001)(15202345003)(54356999)(76176999)(4396001)(19300405004)(81342001)(50986999)(16236675004)(1941001)(99396002)(31966008)(83322001)(19580395003)(19580405001)(83072002)(77982001)(81542001)(74316001)(92566001)(74662001)(20776003)(74502001)(64706001)(106116001)(19617315012)(33646002)(105586002)(76576001)(2656002)(79102001)(101416001)(86362001)(15975445006)(107046002)(106356001)(19625215002)(21056001)(110136001)(24736002)(108616003)(579004); DIR:OUT; SFP:; SCL:1; SRVR:AM3PR03MB611; H:AM3PR03MB612.eurprd03.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; LANG:en; Content-Type: multipart/alternative; boundary="_000_f8c4d4382dda4f7d94ce61b4b421fe4fAM3PR03MB612eurprd03pro_" MIME-Version: 1.0 X-OriginatorOrg: ecitele.com Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/AMHGjQHhdHNMCYPpxygJtjkj3QU Cc: Yakov Rekhter , "pwe3@ietf.org" , "stbryant@cisco.com" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2014 09:12:08 -0000 --_000_f8c4d4382dda4f7d94ce61b4b421fe4fAM3PR03MB612eurprd03pro_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Yimin and all, I would like to summarize the issues that I have with this draft. Digging f= or them in a very long email thread with multiple levels of in-lining is no= t very effective IMO. 1. Protection against unidirectional AC failures looks problematic to= me: a. The draft explicitly differentiates between "ingress" and "egress"= AC failures b. The outcome can be using a different AC for each direction of CE-to= -CE traffic with potentially destructive effect that cannot be mitigated at= the PWE3 level c. IMHO and FWIW a viable alternative could be to employ native AC me= chanisms converting unidirectional AC failures into bi-directional ones. Th= is would eliminate the differentiation between ingress and egress AC failur= es and facilitate using the PW redundancy mechanism for bi-directional prot= ection 2. Dependency upon specific mechanisms for setting up PSN tunnels see= ms to break some common PWE3 architectural principles: a. I am pretty well sure that the mechanisms specified in the draft c= annot be used with the PSN tunnel LSPs set up by LDP in DU mode b. If the authors, refer to LDP in the DoD mode augmented with some CR= -LDP-like mechanisms (e.g., for exclusion of specific links when setting up= the bypass LSP between the PLR and the Protector), this should, at the ver= y least, be discussed with and approved by the MPLS WG 3. Scalability of the proposed solution should be, at least, explicit= ly presented. In particular, the draft assumes that a dedicated LSP would b= e used between PE1 PE2 for each combination of PE2 with a specific Protect= or LSR. 4. The Protector LSR could be either a kind of S-PE (if different fro= m the secondary T-PE) or the secondary T-PE itself. In both cases it should= be capable of handling labels from specific context spaces as if they were= PW labels. This requires (at the very least) an update of RFC 4447 and sho= uld be explicitly presented and discussed. Hopefully this summary will be useful. Regards, Sasha Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 > -----Original Message----- > From: Alexander Vainshtein > Sent: Wednesday, August 06, 2014 8:55 AM > To: Yimin Shen > Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org > Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast- > protection-01 > > Yimin hi! > > Please see a couple of comments inline below (marked [[Sasha]]. > > Regards, > Sasha > ________________________________________ > From: Yimin Shen > > Sent: Tuesday, August 5, 2014 9:55 PM > To: Alexander Vainshtein > Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ie= tf.org > Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast- > protection-01 > > Hi Sasha, > > All ACs and PWs in this draft are active at individual level. One set of = AC-PW- > AC is used as backup to protect the other set of AC-PW-AC (i.e. primary). > > I think we both agree that CE behavior is out of the scope of PWE3. > > [[Sasha]] Yes - provided that PWE3 matches reasonable CE expectations. If= it > is not so, it becomes very much in scope IMO. > > If you want to run L3 between CEs, you could bundle the two ACs on CE as = a > single logical L3 interface. I'm sure there are other approaches dependin= g on > the type of CE. But again, this is out scope of PWE3. > > [[Sasha]] Well, not really IMO. E.g., Section 7.2 of RFC 7225 specificall= y > defines a PWE3 solution for PW redundancy with multi-Chassis LAG. This > solution addresses the case when a dual-homed CE perceives a pair of > Ethernet links between itself and a pair of redundant PEs as a single Lin= k > Aggregation Group, and effectively uses LACP as the dual-homing protocol. > I'd say that if you decide that CE should use some standard (or widely > deployed) mechanism between itself and the PEs in order for PWE3 to > provide a meaningful service, PWE3 has both to specify its expectations > regarding the CE behavior and the methods it uses to accommodate this > behavior. > An example that looks highly relevant to me with regard to your draft is = like > following: > > The draft explicitly differentiates between "ingress" and "egress" AC fai= lures > and states that "Today, fast protection against ingress AC failure and in= gress > (T-)PE failure is achievable by using a multi-homed CE and redundant PWs= .". > Suppose that CE and PE employ some native mechanism (out of scope of > PWE3, of course) that would revert any unidirectional AC failure to a bi- > directional one in a very fast manner. (One example that comes to mind is > the LF/RF exchange mechanism oft 10-Gigabit Ethernet.) Would such a > mechanism eliminate the need to protect against egress AC failures which > seems to be one of the major drives for the draft? > > The "fast protection" refers to the ability to locally detect failure, re= store > traffic, and reduce loss. CEs and PEs may go through any negotiation or > coordination to move traffic from the repaired PW to anther PW, at any pa= ce. > Before that happens, traffic has already been repaired by this mechanism. > > > Thanks, > > /Yimin > > > > -----Original Message----- > From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] > Sent: Tuesday, August 05, 2014 1:13 AM > To: Yimin Shen > Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ie= tf.org > Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast- > protection-01 > > Yimin hi, > I apologize for missing your response when it has been sent. > > I will try to address some of your comments without yet one more level of= in- > lining:-). > > [Yimin] This draft assumes active-active PWs and bidirectional ACs. > [Sasha] It would be nice if this were explicitly stated in the draft. > > [Yimin] I think you are talking about how to run Layer 3 between CEs with= an > overlay model, which I don't think is in scope of PWE3 architecture [Sash= a] To > the best of my understanding PWE3 is about an emulation of a native servi= ce. > It does not have to be perfect, but limitations are supposed to be explic= itly > presented in Applicability Statements. On the practical level, if a PW > mechanism is not good enough to carry L3 traffic between a pair of CEs, w= hat > is it good for? > And, last but not least, if CE1 and CE2 in my example were L2 switches > running some flavor of STP, the situation would be no less disruptive. > > [Yimin] In case of egress AC failure, CE2 will detect the failure and sta= rt to > send traffic over PW2 (i.e. the backup PW). > [Sasha] This is an assumption about CE2 which is definitely out of scope = of > the PWE3 architecture. Actually what you say is that CE2 should treat its > failure to receive traffic over an AC as a bi-directional AC failure. > [Yimin] CE1 should also start to send traffic over the backup PW2. This m= ay be > triggered by many events, e.g. [1] PE1's notification to CE1 based on the= PW > status you mentioned above (but it is not mandatory); [Sasha] This is pos= sible > of course, but, as you've said, not mandatory [Yimin] or [2] CE1 receivin= g > traffic from PE3. > [Sasha] But you've said that ACs are active-active. So why receiving > something from PE3 would be of any interest? > And of course, there is a case when only CE2 is dual-homed but CE1 is no= t... > Is the draft relevant in this scenario? > [Yimin] In any case, it may take some time for CE1 to do so. Before this > happens on CE1, the traffic is still going over PW1 (primary PW) which is= being > locally repaired by PE2. > [Sasha] I'd say that the draft relies on some "dual-homing protocol" betw= een > a dual-homed CE and its PEs to provide for a bi-directional switch, and t= hat > such a switch could take some time. I am not sure this counts for "fast > protection". > > Regards, > Sasha > > > > ________________________________________ > From: Yimin Shen > > Sent: Thursday, July 31, 2014 2:22 AM > To: Alexander Vainshtein > Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ie= tf.org > Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast- > protection-01 > > Hi Sasha, > > Thanks again for your comments. Please see inline... > > > Thanks, > > /Yimin > > > > -----Original Message----- > From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] > Sent: Wednesday, July 30, 2014 4:08 AM > To: Yimin Shen > Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ie= tf.org > Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast- > protection-01 > > Yimin hi, > Lots of thanks for a prompt response. > > Unfortunately, from my point of view, the issues I've raised still remain > unresolved. Please see additional questions inline below. > > Regards, > Sasha > Email: Alexander.Vainshtein@ecitele.com > Mobile: 054-9266302 > > > -----Original Message----- > > From: Yimin Shen [mailto:yshen@juniper.net] > > Sent: Tuesday, July 29, 2014 7:56 PM > > To: Alexander Vainshtein; stbryant@cisco.com= ; Yakov Rekhter; > > pwe3@ietf.org > > Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast- > > protection-01 > > > > Hi Sasha, > > > > Thanks very much for the review and comments. Please see my responses > > inline. Hope they are helpful. > > > > > > Thanks, > > > > /Yimin > > > > > > > > -----Original Message----- > > From: pwe3 [mailto:pwe3-bounces@ietf.org] On Behalf Of Alexander > > Vainshtein > > Sent: Tuesday, July 29, 2014 10:08 AM > > To: stbryant@cisco.com; Yakov Rekhter; pwe3@= ietf.org > > Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast- > > protection-01 > > > > Stewart, Yakov and all, > > I have started reading the draft, and it seems that it breaks quite a > > few common architectural assumptions. > > > > [yshen] This draft is fully based on the PWE3 architecture and the > > MS-PW architecture. > > > > The following text from Section 3.1 "Single-Segment PW" looks highly > > problematic to me: > > > > At any given time, each CE sends traffic via only one AC and receives > > traffic via only one AC. The two ACs MAY or MAY NOT be the same. > > The AC used to send traffic is determined by the CE, and MAY rely on > > an end-to-end OAM mechanism between the CEs. The AC used for the > CE > > to receive traffic is determined by the state of the network and the > > protection mechanism in use, as described later in this document. > > . > > > > To the best of my order for the pair of interconnected CEs to be able > > to operate in this way each of them should treat the pair of ACs shown > > in Figure > > 1 as a single logical entity. > > E.g., if we are speaking about Ethernet PWs, each CE could treat > > Ethernet links supporting these two ACs as a single Link Aggregation > > Group. However, the draft does not mention any such thing (or at least = I > did not find it). > > > > Do I miss something important here? > > > > [yshen] It is a logical view that the CE is dual-home to 2 PEs over 2 > > distinct ACs. AC is a concept well defined in RFC 3985. In terms of > > the actual PE-CE connection in the case of Ethernet, the CE could have > > 2 distinct Ethernet links or 2 VLANs over a single LAG. > > > [[Sasha]] If I read your response correctly, each dual-homed CE treats ea= ch if > its two ACs as an individual entity. > I see this is as highly problematic. E.g., consider the case when CE1 and= CE2 > are two CE routers running some link-state routing protocol between them. > - In the normal situation, CE1 probably would see its two ACs as two par= allel > bi-directional links. > - With PW redundancy, one of these links (one that corresponds to the > current Active PW) would be Up and the other one - Down > - With your proposal, the typical link-state routing protocol between CE1= and > CE2 would not be able to form any adjacencies across any of these links, > because this process requires bi-directional link connectivity; so regard= less of > whatever happens in the PW domain CE1 and CE2 would consider each other > as unreachable across this domain. > Did I miss something? (I intentionally leave aside protocols like OLSR th= at can > handle unidirectional links). > > [yshen] I'm lost in the last 2 bullets. This draft assumes active-active = PWs and > bidirectional ACs. I think you are talking about how to run Layer 3 betwe= en > CEs with an overlay model, which I don't think is in scope of PWE3 > architecture. > > > > > > > > I also wonder whether, in the single segment PW scenario, the same > > level of protection could not be achieved with ICCP (RFC 7275) > > running between each pair of PEs and, possibly with 4 (instead of 2) > > PWs. For the reference, ICCP (or RFC 7275) is not mentioned in the > > draft, even if a possible alternative solution to some of the problems > > it solves. ( > > > > [yshen] Thanks for pointing out ICCP. We will mention it in the next > revision. > > In this draft we broadly refer to such kind of mechanism as "control > > plane repair". The purpose of this draft is to locally repair traffic > > at the failure point, in order to reduce traffic loss before ICCP or > > other control plane repair mechanism kicks in. > > [[Sasha]] I am not sure ICCP in this case would mean slow protection. Its > function here would be just to synchronize the state of PWs between PE1 > and PE3/PE2 and PE4, and this can be reasonably fast IMO. The Protection > State Coordination (PSC) protocol in RFC 6378 looks like an analog (with = some > stretch of course), and it is supposed to provide pretty fast protection > switching. > > [yshen] The draft doesn't comment specifically on ICCP performance. > Generally speaking, local repair mechanism tends to have the least overhe= ad > when restoring traffic. This is because both failure detection and traffi= c > restoration are done right in the dataplane. It doesn't rely on dataplane= to > control plane notification (for failure event) or control plane protocol.= For this > exact reason there have been various IP/MPLS fast-reroute mechanisms > defined by IETF. Of course, in a given scenario where the delay in global > repair or control plane repair is tolerable for users, they don't have to= use > local repair. Local repair is for the case where it is desirable to reduc= e traffic > loss even further. Note that local repair is a temporary repair, and it d= oesn't > prevent global repair or control plane repair to perform a permanent repa= ir. I > believe the draft has specifically mentioned this. > > > > > > > > > Stewart did not specify in his email which MPLS architecture > > principles he thinks are broken by this draft. > > > > I think, however, that some PWE3 architectural principles are > > definitely are broken, and this includes: > > - Bi-directional nature of the PW - it seems that the draft allows > > each a PW be Up in one direction and Down in the other one > > > > [yshen] No, the draft does not allow that. It only says that we can > > send traffic of one direction over one PW, while sending traffic of > > the opposite direction over another PW (i.e. backup PW). Both PWs shoul= d > be up bidirectionally. > > > > > [[Sasha]] Sorry but I do not understand that. The draft discusses somethi= ng > like the "egress AC failure". If, say. PE1 detects that, I expect it wou= ld send a > PW Status with "PW not Forwarding" and "Local AC Transmit Fault" bits set= to > PE2. Would this not mean that the PW between PE1 and PE2 is Down? > > [yshen] In case of egress AC failure, CE2 will detect the failure and sta= rt to > send traffic over PW2 (i.e. the backup PW). CE1 should also start to send > traffic over the backup PW2. This may be triggered by many events, e.g. [= 1] > PE1's notification to CE1 based on the PW status you mentioned above (but= it > is not mandatory); or [2] CE1 receiving traffic from PE3. In any case, it= may > take some time for CE1 to do so. Before this happens on CE1, the traffic = is still > going over PW1 (primary PW) which is being locally repaired by PE2. > > > > > > > > - Lack of dependencies between the PSN tunnels and the PWs they carry. > > * Consider the network diagram shown in Figure 5 in Section 4.1 > > * The text suggests that the tunnel between PE1 and PE2 must > > recognize PE4 as the "protector" node for the PW it carries > > * What is supposed to happen if the tunnel between PE1 and PE2 > > carries, say, 10 PWs, each with its own backup PE PE4-1, ... PE4-10)? > > > > [yshen] In your example, the 10 PWs are protected by 10 different > > protectors. In this case, you would need 10 different tunnels, > > destined to 10 different context IDs (one context ID per > PE4-x>). If multiple PWs are protected by the same protector, they can = be > carried by the same tunnel. > > > [[Sasha]] This looks like a non-scalable solution to me. > > [yshen] I'm not sure there is any scalability issue here. We basically ne= ed one > tunnel per context ID, i.e. per . In reality, the = number > of should not be high. The draft also specifies t= he > centralized model for reducing the number of context IDs. > > > > > > > > > - Lack of dependencies between the mechanism used for the tunnel setup > > (LDP, RSVP-TE, manual configuration etc.) and the PWs this tunnel carri= es. > > * Is the protection model described in Section 4.1 compatible > > with Tunnel LSPs being set up by LDP? With, say, GRE tunnels carrying > > the PWs using MPLS-in-GRE? > > > > [yshen] Yes, it is compatible. The draft is generic on transport > > layer, and it allows tunnels to be LDP, RSVP-TE, GRE, etc. These > > protocols already provide sufficient machineries for setting up transpo= rt > tunnels and bypass tunnels. > > Therefore, the draft doesn't require new extension to these LSP > > signaling protocols. > > > [[Sasha]] Sorry but this does not match your response to my previous > question/issue where you have explained that you effectively need a tunne= l > per PW between PE1 and PE2. RSVP-TE can provide that; but how is LDP > supposed to do that? > > [yshen] Again, a tunnel can carry multiple PWs, if these PWs are protecte= d by > the same protector. You could use 10 different protectors to protect 10 P= Ws, > but in reality when SP design their network, they should ask why they hav= e > to do so. Regarding LDP, it is one FEC per context ID, and nothing specia= l. > > > > > > Last but not least, I would like to understand better how the draft is > > protecting MS-PWs against S-PE failures. > > Specifically, I'd like to understand whether it is applicable to the > > case when each pair of T-PEs resides in its own AS, with two pairs of > > directly connected ASBRs between these two ASs acting as S-PEs. There > > would be no need to set up any tunnel LSPs between each pair of S-PEs i= n > this scenario. > > I would appreciate an analysis of this scenario regarding ability to > > protect about the ASBR/S-PE failure. IMO it is a relevant scenario > > because MS-PWs are one of the methods to provide inter-AS services. > > > > [yshen] The MS-PW topology in this draft is based on the Figure-4 of > > RFC 5659, which is slightly different than the scenario you mentioned > above. > > > [[Sasha]] Figure 4 in RFC 5659 does not say anything about inter-AS. > > > > However, the mechanism in this draft does apply to this scenario as > > well. It should work equally if S-PEs are one-hop or multi-hops away > > from each other. Please see the picture below. In this case, SPE1 ~ SPE= 4 are > ASBRs. > > When SPE4 detects the failure of SPE3, it will do a local repair and > > redirect PW traffic to SPE4 which is the protector. The bypass between > > SPE1 and SPE4 may be a one-hop tunnel or a multi-hop tunnel, depending > > on the actual topology between the 2 AS'. > > > [[Sasha]] This approach can work if SPE1 is directly connected to SPE4 > because in this case there would be no need for a tunnel LSP between them= . > I do not see a simple way to set up an LSP between SPE1 (in AS-1) and > SPE4(in AS-2) if such connectivity is not present; could you possibly exp= and > on how this is supposed to happen? > > [yshen] Sasha, if you don't have a bypass tunnel (due to specific topolog= y or > policy or whatever), you won't get any IP/MPLS fast-reroute mechanism to > work :) However it doesn't mean the mechanism is broken. In your case, I > don't see any technique issue why you cannot set up a inter-AS bypass fro= m > SPE1-SPE2-SPE4. > > > > > > > In any case, SPE4 will send PW traffic over a > > tunnel to TPE4. > > > > |<--------------- PW1 ----------->| > > > > - TPE1 -------- SPE1 - SPE3 --------- TPE2 - > > / PLR \ \ > > / \ \ > > CE1 bypass\ CE2 > > \ \ / > > \ \ / > > - TPE3 -------- SPE2 - SPE4 --------- TPE4 - > > protector > > > > |<--------------- PW2 ----------->| > > > > My 2c, > > Sasha > > Email: Alexander.Vainshtein@ecitele.com > > Mobile: 054-9266302 > > > > > -----Original Message----- > > > From: pwe3 [mailto:pwe3-bounces@ietf.org] On Behalf Of Stewart > > > Bryant > > > Sent: Tuesday, July 29, 2014 3:35 PM > > > To: Yakov Rekhter; pwe3@ietf.org > > > Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast- > > > protection-01 > > > > > > On 25/07/2014 16:24, Yakov Rekhter wrote: > > > > Hi, > > > > > > > >> This email begins a second two week working group last call for > > > >> http://tools.ietf.org/html/draft-ietf-pwe3-endpoint-fast-protectio= n- > 01 . > > > >> > > > >> Please review the draft and post any comments to the PWE3 list. > > > >> > > > >> If you have read the draft, even if you don't have any comments, > > > >> but would like to see the draft progressed, please can you > > > >> indicate this to the > > > list. > > > >> > > > >> This working group last call will end on Friday August 8th, 2014. > > > > I would like to see the draft progressed. > > > > > > > > Yakov. > > > > > > > > _______________________________________________ > > > > pwe3 mailing list > > > > pwe3@ietf.org > > > > https://www.ietf.org/mailman/listinfo/pwe3 > > > > > > > Yakov > > > > > > Doesn't this draft implicitly make a number of extensions to the > > > MPLS architecture? > > > > > > Stewart > > > > > > _______________________________________________ > > > pwe3 mailing list > > > pwe3@ietf.org > > > https://www.ietf.org/mailman/listinfo/pwe3 > > > > _______________________________________________ > > pwe3 mailing list > > pwe3@ietf.org > > https://www.ietf.org/mailman/listinfo/pwe3 --_000_f8c4d4382dda4f7d94ce61b4b421fe4fAM3PR03MB612eurprd03pro_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Yimin and all,

I would like to summarize the issues that I have = with this draft. Digging for them in a very long email thread with multiple= levels of in-lining is not very effective IMO.

 

1.       Protection against unid= irectional AC failures looks problematic to me:

a.       The draft explicitly diffe= rentiates between “ingress” and “egress” AC failure= s

b.      The outcome can be using a= different AC for each direction of CE-to-CE traffic with potentially destr= uctive effect that cannot be mitigated at the PWE3 level

c.       IMHO and FWIW a viable alt= ernative could be to employ native AC mechanisms converting unidirectional = AC failures into bi-directional ones. This would eliminate the differentiat= ion between ingress and egress AC failures and facilitate using the PW redundancy mechanism for bi-direction= al protection

2.       Dependency upon specifi= c mechanisms for setting up PSN tunnels seems to break some common PWE3= architectural principles:

a.       I am pretty well sure that= the mechanisms specified in the draft cannot be used with the PSN tunnel L= SPs set up by LDP in DU mode

b.      If the authors, refer to L= DP in the DoD mode augmented with some CR-LDP-like mechanisms (e.g., for exclu= sion of specific links when setting up the bypass LSP between the PLR and t= he Protector), this should, at the very least, be discussed with and approv= ed by the MPLS WG

3.       Scalability of the = proposed solution should be, at least, explicitly presented. In particular,= the draft assumes that a dedicated LSP would be used between PE1  PE2= for each combination of PE2 with a specific Protector LSR.

4.       The Protector LSR could be= either a kind of S-PE (if different from the secondary T-PE) or the second= ary T-PE itself. In both cases it should be capable of handling labels from= specific context spaces as if they were PW labels. This requires (at the very least) an update of RFC 4447 an= d should be explicitly presented and discussed.

Hopefully this summary will be useful.=

 

 

Regards,

       Sasha <= /o:p>

Email: Alexander.Vainshtein@ecitele.com

Mobile: 054-9266302

 

> -----Original Message-----

> From: Alexander Vainshtein

> Sent: Wednesday, August 06, 2014 8:55 AM

> To: Yimin Shen

> Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@= ietf.org

> Subject: RE: [PWE3] WG Last Call for draft-i= etf-pwe3-endpoint-fast-

> protection-01

>

> Yimin hi!

>

> Please see a couple of comments inline below= (marked [[Sasha]].

>

> Regards,

> Sasha

> ________________________________________

> From: Yimin Shen <yshen@j= uniper.net>

> Sent: Tuesday, August 5, 2014 9:55 PM

> To: Alexander Vainshtein

> Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org

> Subject: RE: [PWE3] WG Last Call for draft-i= etf-pwe3-endpoint-fast-

> protection-01

>

> Hi Sasha,

>

> All ACs and PWs in this draft are active at = individual level. One set of AC-PW-

> AC is used as backup to protect the other se= t of AC-PW-AC (i.e. primary).

>

> I think we both agree that CE behavior is ou= t of the scope of PWE3.

>

> [[Sasha]] Yes - provided that PWE3 matches r= easonable CE expectations. If it

> is not so, it becomes very much in scope IMO= .

>

> If you want to run L3 between CEs, you could= bundle the two ACs on CE as a

> single logical L3 interface. I'm sure there = are other approaches depending on

> the type of CE. But again, this is out scope= of PWE3.

>

> [[Sasha]] Well, not really IMO. E.g., Sectio= n 7.2 of RFC 7225 specifically

> defines a PWE3 solution for PW redundancy wi= th multi-Chassis LAG. This

> solution addresses the case when a dual-home= d CE perceives a pair of

> Ethernet links between itself and a pair of = redundant PEs as a single Link

> Aggregation Group, and effectively uses LACP= as the dual-homing protocol.

> I'd say that if you decide that CE should us= e some standard (or widely

> deployed) mechanism between itself and the P= Es in order for PWE3 to

> provide a meaningful service, PWE3 has both = to specify its expectations

> regarding the CE behavior and the methods it= uses to accommodate this

> behavior.

> An example that looks highly relevant to me = with regard to your draft is like

> following:

>

> The draft explicitly differentiates between = "ingress" and "egress" AC failures

> and states that "Today, fast protection= against ingress AC failure and ingress

> (T-)PE  failure is achievable by using = a multi-homed CE and redundant PWs.".

> Suppose that CE and PE employ some native me= chanism (out of scope of

> PWE3, of course) that would revert any unidi= rectional AC failure to a bi-

> directional one in a very fast manner. (One = example that comes to mind is

> the LF/RF exchange mechanism oft 10-Gigabit = Ethernet.) Would such a

> mechanism eliminate the need  to protec= t against egress AC failures which

> seems to be one of the major drives for the = draft?

>

> The "fast protection" refers to th= e ability to locally detect failure, restore

> traffic, and reduce loss. CEs and PEs may go= through any negotiation or

> coordination to move traffic from the repair= ed PW to anther PW, at any pace.

> Before that happens, traffic has already bee= n repaired by this mechanism.

>

>

> Thanks,

>

> /Yimin

>

>

>

> -----Original Message-----

> From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com]

> Sent: Tuesday, August 05, 2014 1:13 AM

> To: Yimin Shen

> Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org

> Subject: RE: [PWE3] WG Last Call for draft-i= etf-pwe3-endpoint-fast-

> protection-01

>

> Yimin hi,

> I apologize for missing your response when i= t has been sent.

>

> I will try to address some of your comments = without yet one more level of in-

> lining:-).

>

> [Yimin] This draft assumes active-active PWs= and bidirectional ACs.

> [Sasha] It would be nice if this were explic= itly stated in the draft.

>

> [Yimin] I think you are talking about how to= run Layer 3 between CEs with an

> overlay model, which I don't think is in sco= pe of PWE3 architecture [Sasha] To

> the best of my understanding PWE3 is about a= n emulation of a native service.

> It does not have to be perfect, but limitati= ons are supposed to be explicitly

> presented in Applicability Statements. = On the practical level, if a PW

> mechanism is not good enough to carry L3 tra= ffic between a pair of CEs, what

> is it good for?

> And, last but not least, if CE1 and CE2 in m= y example were L2 switches

> running some flavor of STP, the situation wo= uld be no less disruptive.

>

> [Yimin] In case of egress AC failure, CE2 wi= ll detect the failure and start to

> send traffic over PW2 (i.e. the backup PW).<= /p>

> [Sasha] This is an assumption about CE2 whic= h is definitely out of scope of

> the PWE3 architecture. Actually what you say= is that CE2 should treat its

> failure to receive traffic over an AC as a b= i-directional AC failure.

> [Yimin] CE1 should also start to send traffi= c over the backup PW2. This may be

> triggered by many events, e.g. [1] PE1's not= ification to CE1 based on the PW

> status you mentioned above (but it is not ma= ndatory); [Sasha] This is possible

> of course, but, as you've said, not mandator= y [Yimin] or [2] CE1 receiving

> traffic from PE3.

> [Sasha] But you've said that ACs are active-= active. So why receiving

> something from PE3 would be of any interest?=

> And of course, there is a case when only CE2= is dual-homed but CE1  is not...

> Is the draft relevant in this scenario?

> [Yimin] In any case, it may take some time f= or CE1 to do so. Before this

> happens on CE1, the traffic is still going o= ver PW1 (primary PW) which is being

> locally repaired by PE2.

> [Sasha] I'd say that the draft relies on som= e "dual-homing protocol" between

> a dual-homed CE and its PEs to provide for a= bi-directional switch, and that

> such a switch could take some time. I am not= sure this counts for "fast

> protection".

>

> Regards,

> Sasha

>

>

>

> ________________________________________

> From: Yimin Shen <yshen@j= uniper.net>

> Sent: Thursday, July 31, 2014 2:22 AM

> To: Alexander Vainshtein

> Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org

> Subject: RE: [PWE3] WG Last Call for draft-i= etf-pwe3-endpoint-fast-

> protection-01

>

> Hi Sasha,

>

> Thanks again for your comments. Please see i= nline...

>

>

> Thanks,

>

> /Yimin

>

>

>

> -----Original Message-----

> From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com]

> Sent: Wednesday, July 30, 2014 4:08 AM

> To: Yimin Shen

> Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org

> Subject: RE: [PWE3] WG Last Call for draft-i= etf-pwe3-endpoint-fast-

> protection-01

>

> Yimin hi,

> Lots of thanks for a prompt response.

>

> Unfortunately, from my point of view, the is= sues I've raised still remain

> unresolved. Please see additional questions = inline below.

>

> Regards,

>        Sa= sha

> Email: Alexander.Vainshtein@= ecitele.com

> Mobile: 054-9266302

>

> > -----Original Message-----

> > From: Yimin Shen [mailt= o:yshen@juniper.net]

> > Sent: Tuesday, July 29, 2014 7:56 PM

> > To: Alexander Vainshtein; stbryant@cisco.com; Yakov Rekhter;

> > pwe3@ietf.org

> > Subject: RE: [PWE3] WG Last Call for dr= aft-ietf-pwe3-endpoint-fast-

> > protection-01

> >

> > Hi Sasha,

> >

> > Thanks very much for the review and com= ments. Please see my responses

> > inline. Hope they are helpful.

> >

> >

> > Thanks,

> >

> > /Yimin

> >

> >

> >

> > -----Original Message-----

> > From: pwe3 [mailto:= pwe3-bounces@ietf.org] On Behalf Of Alexander

> > Vainshtein

> > Sent: Tuesday, July 29, 2014 10:08 AM

> > To: stbryant@cisco.com= ; Yakov Rekhter; pwe3@ietf.org

> > Subject: Re: [PWE3] WG Last Call for dr= aft-ietf-pwe3-endpoint-fast-

> > protection-01

> >

> > Stewart, Yakov and all,

> > I have started reading the draft, and i= t seems that it breaks quite a

> > few common architectural assumptions.

> >

> > [yshen] This draft is fully based on th= e PWE3 architecture and the

> > MS-PW architecture.

> >

> > The following text from Section 3.1 &qu= ot;Single-Segment PW" looks highly

> > problematic to me:

> > <quote>

> >   At any given time, each CE = sends traffic via only one AC and receives

> >    traffic via only one = AC.  The two ACs MAY or MAY NOT be the same.

> >    The AC used to send t= raffic is determined by the CE, and MAY rely on

> >    an end-to-end OAM mec= hanism between the CEs.  The AC used for the

> CE

> >    to receive traffic is= determined by the state of the network and the

> >    protection mechanism = in use, as described later in this document.

> > <end quote>.

> >

> > To the best of my order for the pair of= interconnected CEs to be able

> > to operate in this way each of them sho= uld treat the pair of ACs shown

> > in Figure

> > 1 as a single logical entity.

> > E.g., if we are speaking about Ethernet= PWs, each CE could treat

> > Ethernet links supporting these two ACs= as a single Link Aggregation

> > Group. However, the draft does not ment= ion any such thing (or at least I

> did not find it).

> >

> > Do I miss something important here?

> >

> > [yshen] It is a logical view that the C= E is dual-home to 2 PEs over 2

> > distinct ACs. AC is a concept well defi= ned in RFC 3985. In terms of

> > the actual PE-CE connection in the case= of Ethernet, the CE could have

> > 2 distinct Ethernet links or 2 VLANs ov= er a single LAG.

> >

> [[Sasha]] If I read your response correctly,= each dual-homed CE treats each if

> its two ACs as an individual entity.

> I see this is as highly problematic. E.g., c= onsider the case when CE1 and CE2

> are two CE routers running some link-state r= outing protocol between them.

> -  In the normal situation, CE1 probabl= y would see its two ACs as two parallel

> bi-directional links.

> - With PW redundancy, one of these links (on= e that corresponds to the

> current Active PW) would be Up and the other= one - Down

> - With your proposal, the typical link-state= routing protocol between CE1 and

> CE2 would not be able to form any adjacencie= s across any of these links,

> because this process requires bi-directional= link connectivity; so regardless of

> whatever happens in the PW domain CE1 and CE= 2 would consider each other

> as unreachable across this domain.

> Did I miss something? (I intentionally leave= aside protocols like OLSR that can

> handle unidirectional links).

>

> [yshen] I'm lost in the last 2 bullets. This= draft assumes active-active PWs and

> bidirectional ACs. I think you are talking a= bout how to run Layer 3 between

> CEs with an overlay model, which I don't thi= nk is in scope of PWE3

> architecture.

>

>

>

>

>

>

> > I also wonder whether, in the single se= gment PW scenario, the same

> > level of protection could not be achiev= ed with ICCP  (RFC 7275)

> > running between each pair of PEs and, p= ossibly with 4 (instead of 2)

> > PWs.  For the reference, ICCP (or = RFC 7275) is not mentioned in the

> > draft, even if a possible alternative s= olution to some of the problems

> > it solves. (

> >

> > [yshen] Thanks for pointing out ICCP. W= e will mention it in the next

> revision.

> > In this draft we broadly refer to such = kind of mechanism as "control

> > plane repair". The purpose of this= draft is to locally repair traffic

> > at the failure point, in order to reduc= e traffic loss before ICCP or

> > other control plane repair mechanism ki= cks in.

>

> [[Sasha]] I am not sure ICCP in this case wo= uld mean slow protection. Its

> function here would be just to synchronize t= he state of PWs between PE1

> and PE3/PE2 and PE4, and this can be reasona= bly fast IMO. The Protection

> State Coordination (PSC) protocol in RFC 637= 8 looks like an analog (with some

> stretch of course), and it is supposed to pr= ovide pretty fast protection

> switching.

>

> [yshen] The draft doesn't comment specifical= ly on ICCP performance.

> Generally speaking, local repair mechanism t= ends to have the least overhead

> when restoring traffic. This is because both= failure detection and traffic

> restoration are done right in the dataplane.= It doesn't rely on dataplane to

> control plane notification (for failure even= t) or control plane protocol. For this

> exact reason there have been various IP/MPLS= fast-reroute mechanisms

> defined by IETF. Of course, in a given scena= rio where the delay in global

> repair or control plane repair is tolerable = for users, they don't have to use

> local repair. Local repair is for the case w= here it is desirable to reduce traffic

> loss even further. Note that local repair is= a temporary repair, and it doesn't

> prevent global repair or control plane repai= r to perform a permanent repair. I

> believe the draft has specifically mentioned= this.

>

>

>

>

>

> >

> > Stewart did not specify in his email wh= ich MPLS architecture

> > principles he thinks are broken by this= draft.

> >

> > I think, however, that some PWE3 archit= ectural principles are

> > definitely are broken, and this include= s:

> > - Bi-directional nature of the PW - it = seems that the draft allows

> > each a PW be Up in one direction and Do= wn in the other one

> >

> > [yshen] No, the draft does not allow th= at. It only says that we can

> > send traffic of one direction over one = PW, while sending traffic of

> > the opposite direction over another PW = (i.e. backup PW). Both PWs should

> be up bidirectionally.

> >

> >

> [[Sasha]] Sorry but I do not understand that= . The draft discusses something

> like  the "egress AC failure"= . If, say. PE1 detects that, I expect it would send a

> PW Status with "PW not Forwarding"= and "Local AC Transmit Fault" bits set to

> PE2. Would this not mean that the PW between= PE1 and PE2 is Down?

>

> [yshen] In case of egress AC failure, CE2 wi= ll detect the failure and start to

> send traffic over PW2 (i.e. the backup PW). = CE1 should also start to send

> traffic over the backup PW2. This may be tri= ggered by many events, e.g. [1]

> PE1's notification to CE1 based on the PW st= atus you mentioned above (but it

> is not mandatory); or [2] CE1 receiving traf= fic from PE3. In any case, it may

> take some time for CE1 to do so. Before this= happens on CE1, the traffic is still

> going over PW1 (primary PW) which is being l= ocally repaired by PE2.

>

>

>

>

>

>

> > - Lack of dependencies between the PSN = tunnels and the PWs they carry.

> >       * C= onsider the network diagram shown in Figure 5 in Section 4.1

> >       * T= he text suggests that the tunnel between PE1 and PE2 must

> > recognize PE4 as the "protector&qu= ot; node for the PW it carries

> >       * W= hat is supposed to happen if the tunnel between PE1 and PE2

> > carries, say, 10 PWs, each with its own= backup PE PE4-1, ... PE4-10)?

> >

> > [yshen] In your example, the 10 PWs are= protected by 10 different

> > protectors. In this case, you would nee= d 10 different tunnels,

> > destined to 10 different context IDs (o= ne context ID per <PE2,

> > PE4-x>). If multiple PWs are protect= ed by the same protector, they can be

> carried by the same tunnel.

> >

> [[Sasha]] This looks like a non-scalable sol= ution to me.

>

> [yshen] I'm not sure there is any scalabilit= y issue here. We basically need one

> tunnel per context ID, i.e. per <primary = PE, protector>. In reality, the number

> of <primary PE, protector> should not = be high. The draft also specifies the

> centralized model for reducing the number of= context IDs.

>

>

>

>

>

> >

> > - Lack of dependencies between the mech= anism used for the tunnel setup

> > (LDP, RSVP-TE, manual configuration etc= .) and the PWs this tunnel carries.

> >       * I= s the protection model described in Section 4.1 compatible

> > with Tunnel LSPs being set up by LDP? W= ith, say, GRE tunnels carrying

> > the PWs using MPLS-in-GRE?

> >

> > [yshen] Yes, it is compatible. The draf= t is generic on transport

> > layer, and it allows tunnels to be LDP,= RSVP-TE, GRE, etc. These

> > protocols already provide sufficient ma= chineries for setting up transport

> tunnels and bypass tunnels.

> > Therefore, the draft doesn't require ne= w extension to these LSP

> > signaling protocols.

> >

> [[Sasha]] Sorry but this does not match your= response to my previous

> question/issue where you have explained that= you effectively need a tunnel

> per PW between PE1 and PE2. RSVP-TE can prov= ide that; but how is LDP

> supposed to do that?

>

> [yshen] Again, a tunnel can carry multiple P= Ws, if these PWs are protected by

> the same protector. You could use 10 differe= nt protectors to protect 10 PWs,

> but in reality when SP design their network,= they should ask why they have

> to do so. Regarding LDP, it is one FEC per c= ontext ID, and nothing special.

>

>

>

>

> > Last but not least, I would like to und= erstand better how the draft is

> > protecting MS-PWs against S-PE failures= .

> > Specifically, I'd like to understand wh= ether it is applicable to the

> > case when each pair of T-PEs resides in= its own AS, with two pairs of

> > directly connected ASBRs between these = two ASs acting as S-PEs.  There

> > would be no need to set up any tunnel L= SPs between each pair of S-PEs in

> this scenario.

> > I would appreciate an analysis of this = scenario regarding ability to

> > protect about the ASBR/S-PE failure. IM= O it is a relevant scenario

> > because MS-PWs are one of the methods t= o provide inter-AS services.

> >

> > [yshen] The MS-PW topology in this draf= t is based on the Figure-4 of

> > RFC 5659, which is slightly different t= han the scenario you mentioned

> above.

> >

> [[Sasha]] Figure 4 in RFC 5659 does not say = anything about inter-AS.

> >

> > However, the mechanism in this draft do= es apply to this scenario as

> > well. It should work equally if S-PEs a= re one-hop or multi-hops away

> > from each other. Please see the picture= below. In this case, SPE1 ~ SPE4 are

> ASBRs.

> > When SPE4 detects the failure of SPE3, = it will do a local repair and

> > redirect PW traffic to SPE4 which is th= e protector. The bypass between

> > SPE1 and SPE4 may be a one-hop tunnel o= r a multi-hop tunnel, depending

> > on the actual topology between the 2 AS= '.

>  >

> [[Sasha]] This approach can work if SPE1 is = directly connected to SPE4

> because in this case there would be no need = for a tunnel LSP between them.

> I do not see a simple way to set up an LSP b= etween SPE1 (in AS-1) and

> SPE4(in AS-2) if such connectivity is not pr= esent; could you possibly expand

> on how this is supposed to happen?

>

> [yshen] Sasha, if you don't have a bypass tu= nnel (due to specific topology or

> policy or whatever), you won't get any IP/MP= LS fast-reroute mechanism to

> work :) However it doesn't mean the mechanis= m is broken. In your case, I

> don't see any technique issue why you cannot= set up a inter-AS bypass from

> SPE1-SPE2-SPE4.

>

>

>

> >

>  >  In any case, SPE4 will send= PW traffic over a

> > tunnel to TPE4.

> >

> >      &nbs= p;            |<-= -------------- PW1 ----------->|

> >

> >      &nbs= p;       - TPE1 -------- SPE1 - SPE3 --------= - TPE2 -

> >      &nbs= p;      /       = ;         PLR \   &n= bsp;            = ;       \

> >      &nbs= p;     /        = ;             &= nbsp;\           &nb= sp;           \

> >      &nbs= p;  CE1          &nb= sp;       bypass\    &nbs= p;            &= nbsp;      CE2

> >      &nbs= p;     \        = ;            &n= bsp;   \         &nb= sp;           /

> >      &nbs= p;      \       = ;            &n= bsp;    \        &nb= sp;          /

> >      &nbs= p;       - TPE3 -------- SPE2 - SPE4 --------= - TPE4 -

> >      &nbs= p;            &= nbsp;           &nbs= p;   protector

> >

> >      &nbs= p;            |<-= -------------- PW2 ----------->|

> >

> > My 2c,

> >      &nbs= p; Sasha

> > Email: Alexander.Vainshtein@= ecitele.com

> > Mobile: 054-9266302

> >

> > > -----Original Message-----

> > > From: pwe3 [ma= ilto:pwe3-bounces@ietf.org] On Behalf Of Stewart

> > > Bryant

> > > Sent: Tuesday, July 29, 2014 3:35 = PM

> > > To: Yakov Rekhter; pwe= 3@ietf.org

> > > Subject: Re: [PWE3] WG Last Call f= or draft-ietf-pwe3-endpoint-fast-

> > > protection-01

> > >

> > > On 25/07/2014 16:24, Yakov Rekhter= wrote:

> > > > Hi,

> > > >

> > > >> This email begins a secon= d two week working group last call for

> > > >> http://tools.ietf.org= /html/draft-ietf-pwe3-endpoint-fast-protection-

> 01 .

> > > >>

> > > >> Please review the draft a= nd post any comments to the PWE3 list.

> > > >>

> > > >> If you have read the draf= t, even if you don't have any comments,

> > > >> but would like to see the= draft progressed,  please can you

> > > >> indicate this to the

> > > list.

> > > >>

> > > >> This working group last c= all will end on Friday August 8th, 2014.

> > > > I would like to see the draft= progressed.

> > > >

> > > > Yakov.

> > > >

> > > > _____________________________= __________________

> > > > pwe3 mailing list

> > > > pwe3@ietf.org

> > > > https://www.ietf.org/= mailman/listinfo/pwe3

> > > >

> > > Yakov

> > >

> > > Doesn't this draft implicitly make= a number of extensions to the

> > > MPLS architecture?

> > >

> > > Stewart

> > >

> > > __________________________________= _____________

> > > pwe3 mailing list

> > > <= span style=3D"color:windowtext;text-decoration:none">pwe3@ietf.org

> > > https://www.ietf.org/= mailman/listinfo/pwe3

> >

> > _______________________________________= ________

> > pwe3 mailing list

> > pwe3@ietf.org

> > https://www.ietf.org/= mailman/listinfo/pwe3

--_000_f8c4d4382dda4f7d94ce61b4b421fe4fAM3PR03MB612eurprd03pro_-- From nobody Wed Aug 6 06:01:04 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7DC4A1B29C6; Wed, 6 Aug 2014 06:01:00 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.901 X-Spam-Level: X-Spam-Status: No, score=-1.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FO969kM4_m6A; Wed, 6 Aug 2014 06:00:47 -0700 (PDT) Received: from pipi.pi.nu (pipi.pi.nu [83.168.239.141]) (using TLSv1.1 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5C4511A0AF6; Wed, 6 Aug 2014 06:00:47 -0700 (PDT) Received: from [192.168.0.100] (81-236-221-144-no93.tbcn.telia.com [81.236.221.144]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: loa@pi.nu) by pipi.pi.nu (Postfix) with ESMTPSA id CF8701801586; Wed, 6 Aug 2014 15:00:44 +0200 (CEST) Message-ID: <53E226FC.7020302@pi.nu> Date: Wed, 06 Aug 2014 15:00:44 +0200 From: Loa Andersson User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: "Bocci, Matthew (Matthew)" , "pwe3@ietf.org" , "mpls@ietf.org" References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/3v1uv4-2j8elIDW52X0aQ_rfek8 Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2014 13:01:01 -0000 PWE3-chairs and authors, I must confess I find this document hard to read, I had to have some help to do so. After going over it carefully I'm convinced that it does not mess with MPLS Architecture. However I also not convinced that this is ready to go, and I'm not really sure what we need to do progress it. In its current form we are probably pushing critical review to ADs, IESG and the RFC Editor. Seems to b e the wrong thing to do. What I think we could do is to invoke the rtg area language review team and do a careful language review. When that is done (hoping that most of the problems disappear) we should run an MPLS architecture / PWE3 concept check up and finish of with a short wg last call. /Loa On 2014-07-25 16:56, Bocci, Matthew (Matthew) wrote: > This email begins a second two week working group last call for > http://tools.ietf.org/html/draft-ietf-pwe3-endpoint-fast-protection-01 . > > Please review the draft and post any comments to the PWE3 list. > > If you have read the draft, even if you don¹t have any comments, but would > like to see the draft progressed, please can you indicate this to the > list. > > This working group last call will end on Friday August 8th, 2014. > > > Best regards > > > Matthew and Andy > > _______________________________________________ > pwe3 mailing list > pwe3@ietf.org > https://www.ietf.org/mailman/listinfo/pwe3 > -- Loa Andersson email: loa@mail01.huawei.com Senior MPLS Expert loa@pi.nu Huawei Technologies (consultant) phone: +46 739 81 21 64 From nobody Wed Aug 6 08:54:26 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 701801B287F; Wed, 6 Aug 2014 08:54:23 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -14.502 X-Spam-Level: X-Spam-Status: No, score=-14.502 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aQXv2XDEJJum; Wed, 6 Aug 2014 08:54:21 -0700 (PDT) Received: from rcdn-iport-3.cisco.com (rcdn-iport-3.cisco.com [173.37.86.74]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 452171B2869; Wed, 6 Aug 2014 08:54:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=6873; q=dns/txt; s=iport; t=1407340462; x=1408550062; h=from:to:cc:subject:in-reply-to:reply-to:mime-version: content-id:date:message-id; bh=sXA0E92JZmqAf67kU5Hf9up5T3nCij+BU4X2SVh8j7c=; b=fCTqFa7i+RJaemDqC/rBf3y8OvfXX4YIfTUSG2Nn3isKqLsE1zRbFoZX KFFpKWH3st/XcQcAHt8nfVM3/rwpNez2YAKYuH2jj+8ZfL07LMeETe9C1 laT2JRFdT4hmAuALtP5JSYkV7loXDoS4zk6t6FIdeVLWJgN4thegGZpZb M=; X-IronPort-AV: E=Sophos;i="5.01,812,1400025600"; d="scan'208";a="345546750" Received: from rcdn-core-8.cisco.com ([173.37.93.144]) by rcdn-iport-3.cisco.com with ESMTP; 06 Aug 2014 15:54:21 +0000 Received: from erosen-lnx.cisco.com (erosen-lnx.cisco.com [161.44.71.19]) by rcdn-core-8.cisco.com (8.14.5/8.14.5) with ESMTP id s76FsKiJ000624; Wed, 6 Aug 2014 15:54:20 GMT Received: from erosen-lnx (localhost [127.0.0.1]) by erosen-lnx.cisco.com (Postfix) with ESMTP id E1B74788; Wed, 6 Aug 2014 11:54:19 -0400 (EDT) From: Eric Rosen To: Alexander Vainshtein In-reply-to: Your message of Tue, 05 Aug 2014 13:59:49 -0000. <9696d0db139d46ffaad7be11340215e8@AM3PR03MB612.eurprd03.prod.outlook.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <16166.1407340459.1@erosen-lnx> Date: Wed, 06 Aug 2014 11:54:19 -0400 Message-ID: <16167.1407340459@erosen-lnx> Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/0Uy2lF4DqnByawdezW6pXc-kxjY Cc: "mpls@ietf.org" , pwe3 , "pwe3-chairs@tools.ietf.org" , "stbryant@cisco.com" Subject: Re: [PWE3] [mpls] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: erosen@cisco.com List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2014 15:54:23 -0000 I've looked at this draft, and it seems to me that it does work in LDP-based networks (even with DU mode), and that it does not violate any architectural principles. However, I have made certain assumptions that aren't explicitly stated in the draft. (I am assuming that the draft is intended to be patterned after the "anycast BGP" scheme that I've heard about about, but that is not written up anywhere as far as I know.) Let me say how I think it is intended to work, and the authors can correct me if I'm wrong. For protection against the failure of the "primary" egress PE: - A "primary egress PE" partitions its set of PWs into n sets, where each set is associated with a given "protector". The "protector" is the backup egress PE for those PWs. - Each such set is associated with an IP address. This IP address must be known both to the primary PE and to the protector. * One can think of this address as an "anycast" address that, to routing, denotes both the primary PE and the protector. At the primary PE and the protector, the address denotes a particular set of PWs. * These anycast addresses are distributed by routing, and good old fashioned MPLS labels get assigned to them by LDP. The primary PE MAY bind implicit null to that address, but the protector MUST NOT bind implicit null to that address. * Routing is set up so that (a) if the primary PE is up, traffic to the anycast address always goes to the primary PE, but (b) if the primary PE is down, traffic to the anycast address goes to the protector instead. This can be achieved by messing with the metrics, e.g. - For a given PW, the primary PE informs both the protector and the ingress PE of (a) the label that it (the primary PE) has assigned to that PW, and the "anycast address" that corresponds to that PW. This requires some new signaling (beyond that specified in RFC 4447), but the requisite signaling is specified in the draft. - When sending a packet on the given PW, the ingress PE pushes on the label assigned to the PW by the egress PE. Then the ingress PE pushes on whatever LDP-assigned label it need to use to get the packet to the anycast address that is associated with the PW. - When the protector learns that the egress PE has assigned a given label and anycast address to the PW, it stores that label in a context-specific label space associated with that anycast address. Whenever the protector receives a packet whose top label is the label that the protector has bound to the given anycast address, the protector looks up the next label in the corresponding label space. Let's look at an example: CE1---PE1----PLR----PE2----CE2 | | |-----PE3-----| where PE3 is the protector for a PW connecting CE1 to CE2 (via PE1 and PE2 respectively). Call this pseudowire PW1. Thus there is some IP address, call it A, that PE2 and PE3 both advertise to routing. The metrics are set up so that as long as PE2 is up, it appears to be a better path to A than does PE3. So PE2 assigns label L1 to PW1, and uses T-LDP to inform both PE1 and PE3 of this mapping. To PE1, these are just outgoing labels. To PE3, however, they are incoming labels in a context-specific label space. PE2 and PE3 both assign labels to A (let's call these labels L2 and L3 respectively), and distribute the corresponding label bindings to PLR. PLR also assigns a label to A, and distributes that label binding (L4) to PE1. When PE1 has a packet to send on PW1, it pushes L1, then pushes L4, then sends the packet to PLR. PLR swaps L4 with L2 and sends the packet to PE2. (Note that L2 could be implicit NULL.) PE2 sees label L1, and sends the packet to CE2. Now if PE2 goes down, things happen a bit differently. When PLR gets a packet with L4 on top, it swaps L4 with L3, and sends the packet to PE3. (In this case, of course, L3 cannot be implicit NULL.) PE3 gets the packet, sees L3 on top, looks it up, and then says "L3 is a context label, so I have to pop it and then look up the following label in the L3-specific label table." This context-specific label table contains L1, which has been bound to the PW that leads to CE2. So PE2 pops the label and sends the packet to CE2. I think that's basically it, and the necessary mechanisms do seem to be described in the draft. The draft doesn't actually use the notion of "anycast address" explicitly, but I think the intention is that the "context labels" are definitely bound to anycast addresses. I hope the authors will correct me if I'm wrong. Although the above example shows a PLR that is a neighbor of both PE2 and PE3, this isn't really necessary, one could easily construct an example where PE2 and PE3 have no neighbors in common. The example above doesn't rely on LDP DoD at all, and doesn't make use of any RSVP-TE backup tunnels. However, when protecting against the failure of the egress AC, packets would go first to PE2 then to PE3. This scenario is a bit different, in that: - I think this scenario may require an RSVP-TE backup tunnel (on which PHP is not used) from PE2 to PE3. - PE2 would have to signal PE3 to associate the given PW with the given backup tunnel. - The label that PE3 assigns to that backup tunnel would identify the context in which the PW label is looked up. This is perhaps more detail than is actually in the draft ;-) but if it is an accurate description of the authors' intentions, the scheme seems to work. I don't think this modifies the MPLS architecture at all. It is true that the PE3 needs to maintain a context-specific label table that contains labels assigned by PE2. While one may say that that is a "coordinated label space", it is no more than what is required by the nature of "upstream-assigned labels". I.e., if you can't support this level of coordination, you just can't support upstream-assigned labels. Also, I don't think this draft is an "update" to RFC 4447. You don't need to read this draft to implement RFC 4447, nor does it modify the procedures of RFC 4447. It just introduces some new optional signaling and procedures. I don't think RFC 4447's requirement to use the "platform label space" should be interpreted as prohibiting the use of upstream-assigned labels. The intention of RFC 4447 was to prohibit the use of the interface-specific label space for PW labels. This prohibition was necessary because the ingress PE doesn't know which of the egress PE's network-facing interfaces will actually receive a given packet. This draft doesn't have that problem, as it provides clear procedures for determining the context in which the upstream-assigned labels are looked up. From nobody Wed Aug 6 09:58:44 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B6EF71A03EC; Wed, 6 Aug 2014 09:58:38 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.602 X-Spam-Level: X-Spam-Status: No, score=-2.602 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1mgOf8mPSg72; Wed, 6 Aug 2014 09:58:35 -0700 (PDT) Received: from na01-bl2-obe.outbound.protection.outlook.com (mail-bl2lp0205.outbound.protection.outlook.com [207.46.163.205]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id ED22F1A03F9; Wed, 6 Aug 2014 09:58:34 -0700 (PDT) Received: from BY2PR05MB728.namprd05.prod.outlook.com (10.141.223.25) by BY2PR05MB727.namprd05.prod.outlook.com (10.141.223.23) with Microsoft SMTP Server (TLS) id 15.0.995.14; Wed, 6 Aug 2014 16:58:25 +0000 Received: from BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) by BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) with mapi id 15.00.0995.014; Wed, 6 Aug 2014 16:58:25 +0000 From: Yimin Shen To: "erosen@cisco.com" , Alexander Vainshtein Thread-Topic: [mpls] [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 Thread-Index: AQHPsY61gq4Nx047Wka0XW5Tq3yfmZvDxD8Q Date: Wed, 6 Aug 2014 16:58:24 +0000 Message-ID: References: Your message of Tue, 05 Aug 2014 13:59:49 -0000. <9696d0db139d46ffaad7be11340215e8@AM3PR03MB612.eurprd03.prod.outlook.com> <16167.1407340459@erosen-lnx> In-Reply-To: <16167.1407340459@erosen-lnx> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [66.129.241.10] x-microsoft-antispam: BCL:0;PCL:0;RULEID: x-forefront-prvs: 02951C14DC x-forefront-antispam-report: SFV:NSPM; SFS:(6009001)(13464003)(164054003)(377454003)(189002)(199002)(37854004)(20776003)(85852003)(92566001)(15975445006)(19580405001)(76482001)(83072002)(99286002)(79102001)(76576001)(21056001)(77982001)(33646002)(99396002)(19580395003)(74502001)(74662001)(64706001)(83322001)(2656002)(87936001)(76176999)(54356999)(50986999)(106356001)(74316001)(106116001)(46102001)(31966008)(107046002)(86362001)(95666004)(4396001)(105586002)(81542001)(81342001)(85306004)(101416001)(66066001)(80022001)(24736002)(108616003); DIR:OUT; SFP:; SCL:1; SRVR:BY2PR05MB727; H:BY2PR05MB728.namprd05.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; LANG:en; Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: juniper.net Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/v0ulkOoGZv4ti3XmDJ7rDuv3W58 Cc: "mpls@ietf.org" , pwe3 , "pwe3-chairs@tools.ietf.org" Subject: Re: [PWE3] [mpls] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2014 16:58:39 -0000 Hi Eric, Thanks very much for your review and comments! What you have described below is very accurate and indeed what this draft i= s basically about -- [1] Primary PE tells protector about PW labels; [2] Pr= otector stores the PW labels in a special place called context label table,= and knows when to look up label in this table; [3] PLR sets up a bypass tu= nnel to protector, and the bypass tunnel (UHP) serves as a hint (i.e. conte= xt) for protector to look up PW label in the context label table. These 3 pieces are glued together by context ID and RFC 5331. Only for [1] = the draft needs to define some LDP extensions. [2] is based on RFC 5331. [3= ] relies on existing IP/MPLS signaling and distribution protocols which are= sufficient, and there doesn't seem to be any difficulty that tunnels and b= ypass tunnels cannot be set up in a scalable way. Regarding why this draft calls the IP address as context ID rather than "an= ycast address", the main reason is that we wanted to emphasize its role of = "context" in context label switching, from forwarding's point of view. A co= ntext ID is owned by both primary PE and protector. When it is advertised b= y both routers via IGP/BGP, yes, it could be viewed as an anycast address i= n terms of reachability. Thanks, /Yimin -----Original Message----- From: mpls [mailto:mpls-bounces@ietf.org] On Behalf Of Eric Rosen Sent: Wednesday, August 06, 2014 11:54 AM To: Alexander Vainshtein Cc: mpls@ietf.org; pwe3; pwe3-chairs@tools.ietf.org Subject: Re: [mpls] [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01 - RFC4447 I've looked at this draft, and it seems to me that it does work in LDP-base= d networks (even with DU mode), and that it does not violate any architectu= ral principles. However, I have made certain assumptions that aren't expli= citly stated in the draft. (I am assuming that the draft is intended to be= patterned after the "anycast BGP" scheme that I've heard about about, but = that is not written up anywhere as far as I know.) Let me say how I think = it is intended to work, and the authors can correct me if I'm wrong. For protection against the failure of the "primary" egress PE: - A "primary egress PE" partitions its set of PWs into n sets, where each set is associated with a given "protector". The "protector" is the backu= p egress PE for those PWs. - Each such set is associated with an IP address. This IP address must be known both to the primary PE and to the protector. * One can think of this address as an "anycast" address that, to routing, denotes both the primary PE and the protector. At the primary PE and the protector, the address denotes a particular set of PWs. * These anycast addresses are distributed by routing, and good old fashioned MPLS labels get assigned to them by LDP. The primary PE MAY bind implicit null to that address, but the protector MUST NOT bind implicit null to that address. * Routing is set up so that (a) if the primary PE is up, traffic to the anycast address always goes to the primary PE, but (b) if the primary P= E is down, traffic to the anycast address goes to the protector instead. This can be achieved by messing with the metrics, e.g. - For a given PW, the primary PE informs both the protector and the ingress PE of (a) the label that it (the primary PE) has assigned to that PW, an= d the "anycast address" that corresponds to that PW. =20 This requires some new signaling (beyond that specified in RFC 4447), but the requisite signaling is specified in the draft. - When sending a packet on the given PW, the ingress PE pushes on the label assigned to the PW by the egress PE. Then the ingress PE pushes on whatever LDP-assigned label it need to use to get the packet to the anycast address that is associated with the PW. - When the protector learns that the egress PE has assigned a given label and anycast address to the PW, it stores that label in a context-specific label space associated with that anycast address. Whenever the protector receives a packet whose top label is the label that the protector has bound to the given anycast address, the protector looks up the next label in the corresponding label space. =20 Let's look at an example: CE1---PE1----PLR----PE2----CE2 | | |-----PE3-----| where PE3 is the protector for a PW connecting CE1 to CE2 (via PE1 and PE2 respectively). Call this pseudowire PW1. Thus there is some IP address, call it A, that PE2 and PE3 both advertise t= o routing. The metrics are set up so that as long as PE2 is up, it appears= to be a better path to A than does PE3. So PE2 assigns label L1 to PW1, and uses T-LDP to inform both PE1 and PE3 o= f this mapping. To PE1, these are just outgoing labels. To PE3, however, = they are incoming labels in a context-specific label space. PE2 and PE3 both assign labels to A (let's call these labels L2 and L3 resp= ectively), and distribute the corresponding label bindings to PLR. PLR als= o assigns a label to A, and distributes that label binding (L4) to PE1. When PE1 has a packet to send on PW1, it pushes L1, then pushes L4, then se= nds the packet to PLR. PLR swaps L4 with L2 and sends the packet to PE2. (Note that L2 could be implicit NULL.) PE2 sees label L1, and sends the pa= cket to CE2. Now if PE2 goes down, things happen a bit differently. When PLR gets a pac= ket with L4 on top, it swaps L4 with L3, and sends the packet to PE3. (In = this case, of course, L3 cannot be implicit NULL.) PE3 gets the packet, sees L3 on top, looks it up, and then says "L3 is a co= ntext label, so I have to pop it and then look up the following label in th= e L3-specific label table." This context-specific label table contains L1,= which has been bound to the PW that leads to CE2. So PE2 pops the label a= nd sends the packet to CE2. I think that's basically it, and the necessary mechanisms do seem to be des= cribed in the draft. The draft doesn't actually use the notion of "anycast= address" explicitly, but I think the intention is that the "context labels= " are definitely bound to anycast addresses. I hope the authors will corre= ct me if I'm wrong. Although the above example shows a PLR that is a neighbor of both PE2 and P= E3, this isn't really necessary, one could easily construct an example wher= e PE2 and PE3 have no neighbors in common. The example above doesn't rely on LDP DoD at all, and doesn't make use of a= ny RSVP-TE backup tunnels. However, when protecting against the failure of the egress AC, packets woul= d go first to PE2 then to PE3. This scenario is a bit different, in that: - I think this scenario may require an RSVP-TE backup tunnel (on which PHP is not used) from PE2 to PE3. - PE2 would have to signal PE3 to associate the given PW with the given backup tunnel. - The label that PE3 assigns to that backup tunnel would identify the context in which the PW label is looked up. This is perhaps more detail than is actually in the draft ;-) but if it is = an accurate description of the authors' intentions, the scheme seems to wor= k. I don't think this modifies the MPLS architecture at all. It is true that = the PE3 needs to maintain a context-specific label table that contains labe= ls assigned by PE2. While one may say that that is a "coordinated label sp= ace", it is no more than what is required by the nature of "upstream-assign= ed labels". I.e., if you can't support this level of coordination, you jus= t can't support upstream-assigned labels. Also, I don't think this draft is an "update" to RFC 4447. You don't need = to read this draft to implement RFC 4447, nor does it modify the procedures= of RFC 4447. It just introduces some new optional signaling and procedure= s. I don't think RFC 4447's requirement to use the "platform label space" should be interpreted as prohibiting the use of upstream-assigned labels. The intention of RFC 4447 was to prohibit the use of the interface-specific= label space for PW labels. This prohibition was necessary because the ing= ress PE doesn't know which of the egress PE's network-facing interfaces wil= l actually receive a given packet. This draft doesn't have that problem, a= s it provides clear procedures for determining the context in which the ups= tream-assigned labels are looked up. _______________________________________________ mpls mailing list mpls@ietf.org https://www.ietf.org/mailman/listinfo/mpls From nobody Wed Aug 6 10:37:36 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3440D1A0417; Wed, 6 Aug 2014 10:37:32 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.903 X-Spam-Level: X-Spam-Status: No, score=-106.903 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7eCbwwA60ylo; Wed, 6 Aug 2014 10:37:30 -0700 (PDT) Received: from rfc-editor.org (rfc-editor.org [4.31.198.49]) by ietfa.amsl.com (Postfix) with ESMTP id EAD031A03EC; Wed, 6 Aug 2014 10:37:29 -0700 (PDT) Received: by rfc-editor.org (Postfix, from userid 30) id 7195B18001B; Wed, 6 Aug 2014 10:36:01 -0700 (PDT) To: Alexander.Vainshtein@ecitele.com, thomas.nadeau@bt.com, davidz@oversi.com X-PHP-Originating-Script: 1005:errata_mail_lib.php From: RFC Errata System Message-Id: <20140806173601.7195B18001B@rfc-editor.org> Date: Wed, 6 Aug 2014 10:36:01 -0700 (PDT) Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/86OvVFlQqV5Tr9Y84rOU9WJnigc Cc: brian@innovationslab.net, pwe3@ietf.org, iesg@ietf.org, rfc-editor@rfc-editor.org Subject: [PWE3] [Errata Verified] RFC5601 (4069) X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2014 17:37:32 -0000 The following errata report has been verified for RFC5601, "Pseudowire (PW) Management Information Base (MIB)". -------------------------------------- You may review the report below and at: http://www.rfc-editor.org/errata_search.php?rfc=5601&eid=4069 -------------------------------------- Status: Verified Type: Technical Reported by: Alexander ("Sasha") Vainshtein Date Reported: 2014-08-05 Verified by: Brian Haberman (IESG) Section: 12 Original Text ------------- pwAttachedPwIndex OBJECT-TYPE SYNTAX PwIndexOrZeroType MAX-ACCESS read-create STATUS current DESCRIPTION "If the PW is attached to another PW instead of a local native service, this item indicates the pwIndex of the attached PW. Otherwise, this object MUST be set to zero. Attachment to another PW will have no PW specific entry in any of the service MIB modules." DEFVAL { 0 } ::= { pwEntry 10 } Corrected Text -------------- pwAttachedPwIndex OBJECT-TYPE SYNTAX PwIndexOrZeroType MAX-ACCESS read-create STATUS current DESCRIPTION "If the PW is attached to another PW instead of a local native service, this item indicates the pwIndex of the attached PW. Otherwise, this object MUST be set to zero. Attachment to another PW will have no PW specific entry in any of the service MIB modules. This object may be modified only when the value of the associated pwAdminStatus object is down(2), and the associated pwOperStatus object has value down(2) or notPresent(5) such that the row in the pwTable represents an inactive PW." DEFVAL { 0 } ::= { pwEntry 10 } Notes ----- Description of the pwEntry object in the same RFC states that "The read-create objects in this table are divided into three categories: 1) Objects that MUST NOT be changed after row activation. These are objects that define basic properties of the PW (for example type, destination, etc.). 2) Objects that MAY be changed when the PW is defined as not active. A change of these objects involves re-signaling of the PW or it might be traffic affecting. PW not active is defined as one of the following conditions: a) The pwRowStatus is notInService(2). b) The pwRowStatus is notReady(3). c) The pwAdminStatus is down(2). If the operator needs to change one of the values for an active row, the operator can either set the pwRowStatus to notInService(2) or set pwAdminStatus to down(2). Signaling (or traffic) is initiated again upon setting the pwRowStatus to active(1) or setting the pwAdminStatus to up(1) or testing(3), respectively. 3) Objects that MAY be changed at any time." In further states (in tthe same description) that "By default, all the read-create objects MUST NOT be changed after row activation, unless specifically indicated in the individual object description." pwAttachedPwIndex object is used to stitch a couple of PWs represented by two different rows in the pwTable, with the pwAttachedPwIndex value in the row representing one of them set to the pwIndex of the other one and vice versa. Since there is no way in the SMIv2 paradigm to create two rows in the same table in a single atomic operation, setting a this attribute in a pair of rows is only possible when both are created. In order to do that, read-create access mode of the pwAttachedPwIndex object has to be interpreted as ability to set its value when the row represents an inactive PW. In accordance with the quoted description, such an interpretation must be explicitly specified in the description of this object. Such a specification is missing in the current text, hence the default interpretation of the read-create access mode is holds. Proposed text fixes this problem. -------------------------------------- RFC5601 (draft-ietf-pwe3-pw-mib-14) -------------------------------------- Title : Pseudowire (PW) Management Information Base (MIB) Publication Date : July 2009 Author(s) : T. Nadeau, Ed., D. Zelig, Ed. Category : PROPOSED STANDARD Source : Pseudo Wire Emulation Edge to Edge INT Area : Internet Stream : IETF Verifying Party : IESG From nobody Wed Aug 6 12:29:34 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E90E71A0115; Wed, 6 Aug 2014 12:29:33 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.902 X-Spam-Level: X-Spam-Status: No, score=-1.902 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xwuuH63zZfhx; Wed, 6 Aug 2014 12:29:26 -0700 (PDT) Received: from emea01-db3-obe.outbound.protection.outlook.com (mail-db3lrp0084.outbound.protection.outlook.com [213.199.154.84]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2770C1A043D; Wed, 6 Aug 2014 12:29:26 -0700 (PDT) Received: from AM3PR03MB612.eurprd03.prod.outlook.com (10.242.110.144) by AM3PR03MB419.eurprd03.prod.outlook.com (10.242.110.148) with Microsoft SMTP Server (TLS) id 15.0.995.14; Wed, 6 Aug 2014 19:29:24 +0000 Received: from AM3PR03MB612.eurprd03.prod.outlook.com (10.242.110.144) by AM3PR03MB612.eurprd03.prod.outlook.com (10.242.110.144) with Microsoft SMTP Server (TLS) id 15.0.995.14; Wed, 6 Aug 2014 19:29:23 +0000 Received: from AM3PR03MB612.eurprd03.prod.outlook.com ([10.242.110.144]) by AM3PR03MB612.eurprd03.prod.outlook.com ([10.242.110.144]) with mapi id 15.00.0995.014; Wed, 6 Aug 2014 19:29:23 +0000 From: Alexander Vainshtein To: "brian@innovationslab.net" Thread-Topic: [Errata Verified] RFC5601 (4069) Thread-Index: AQHPsZ0cTt9B/tK/qE6RM/UugAdBVZvD9ctA Date: Wed, 6 Aug 2014 19:29:22 +0000 Message-ID: <01d71ed3a97243f386a0a7a9247e91df@AM3PR03MB612.eurprd03.prod.outlook.com> References: <20140806173601.7195B18001B@rfc-editor.org> In-Reply-To: <20140806173601.7195B18001B@rfc-editor.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [79.178.122.151] x-microsoft-antispam: BCL:0;PCL:0;RULEID: x-forefront-prvs: 02951C14DC x-forefront-antispam-report: SFV:NSPM; SFS:(6009001)(377454003)(51704005)(13464003)(252514010)(189002)(199002)(377424004)(92566001)(105586002)(21056001)(46102001)(76176999)(54356999)(81542001)(99396002)(15202345003)(86362001)(95666004)(76482001)(16799955002)(50986999)(33646002)(107046002)(110136001)(77982001)(15975445006)(85306004)(64706001)(87936001)(31966008)(74316001)(2656002)(74662001)(74502001)(4396001)(83072002)(83322001)(106356001)(106116001)(15188155005)(81342001)(2351001)(80022001)(76576001)(66066001)(20776003)(19580405001)(19580395003)(101416001)(79102001)(85852003)(2501001)(24736002)(108616003); DIR:OUT; SFP:; SCL:1; SRVR:AM3PR03MB612; H:AM3PR03MB612.eurprd03.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; LANG:en; Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Microsoft-Antispam: BCL:0;PCL:0;RULEID: X-OriginatorOrg: ecitele.com Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/pOvryy38VQh71U4RSiKhrW5rYJE Cc: "davidz@oversi.com" , "pwe3@ietf.org" , "iesg@ietf.org" , "thomas.nadeau@bt.com" Subject: Re: [PWE3] [Errata Verified] RFC5601 (4069) X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2014 19:29:34 -0000 Brian, Lots of thanks for a prompt verification and for the much better corrected = text. Regards, Sasha=20 Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 > -----Original Message----- > From: RFC Errata System [mailto:rfc-editor@rfc-editor.org] > Sent: Wednesday, August 06, 2014 8:36 PM > To: Alexander Vainshtein; thomas.nadeau@bt.com; davidz@oversi.com > Cc: brian@innovationslab.net; iesg@ietf.org; pwe3@ietf.org; rfc-editor@rf= c- > editor.org > Subject: [Errata Verified] RFC5601 (4069) >=20 > The following errata report has been verified for RFC5601, "Pseudowire (P= W) > Management Information Base (MIB)". >=20 > -------------------------------------- > You may review the report below and at: > http://www.rfc-editor.org/errata_search.php?rfc=3D5601&eid=3D4069 >=20 > -------------------------------------- > Status: Verified > Type: Technical >=20 > Reported by: Alexander ("Sasha") Vainshtein > > Date Reported: 2014-08-05 > Verified by: Brian Haberman (IESG) >=20 > Section: 12 >=20 > Original Text > ------------- > pwAttachedPwIndex OBJECT-TYPE > SYNTAX PwIndexOrZeroType > MAX-ACCESS read-create > STATUS current > DESCRIPTION > "If the PW is attached to another PW instead of a local > native service, this item indicates the pwIndex of the > attached PW. Otherwise, this object MUST > be set to zero. Attachment to another PW will have no > PW specific entry in any of the service MIB modules." > DEFVAL { 0 } > ::=3D { pwEntry 10 } >=20 > Corrected Text > -------------- > pwAttachedPwIndex OBJECT-TYPE > SYNTAX PwIndexOrZeroType > MAX-ACCESS read-create > STATUS current > DESCRIPTION > "If the PW is attached to another PW instead of a local > native service, this item indicates the pwIndex of the > attached PW. Otherwise, this object MUST > be set to zero. Attachment to another PW will have no > PW specific entry in any of the service MIB modules. > This object may be modified only when the value of > the associated pwAdminStatus object is down(2), and > the associated pwOperStatus object has value down(2) > or notPresent(5) such that the row in the pwTable > represents an inactive PW." > DEFVAL { 0 } > ::=3D { pwEntry 10 } >=20 > Notes > ----- > Description of the pwEntry object in the same RFC states that "The read- > create objects in this table are divided into > three categories: > 1) Objects that MUST NOT be changed after row activation. > These are objects that define basic properties of the > PW (for example type, destination, etc.). > 2) Objects that MAY be changed when the PW is > defined as not active. A change of these objects involves > re-signaling of the PW or it might be traffic affecting. > PW not active is defined as one of the following > conditions: > a) The pwRowStatus is notInService(2). > b) The pwRowStatus is notReady(3). > c) The pwAdminStatus is down(2). > If the operator needs to change one of the values for an > active row, the operator can either set the pwRowStatus to > notInService(2) or set pwAdminStatus to down(2). > Signaling (or traffic) is initiated again upon setting > the pwRowStatus to active(1) or setting the pwAdminStatus > to up(1) or testing(3), respectively. > 3) Objects that MAY be changed at any time." >=20 > In further states (in tthe same description) that "By default, all the re= ad- > create objects MUST NOT be changed after row activation, unless specifica= lly > indicated in the individual object description." >=20 > pwAttachedPwIndex object is used to stitch a couple of PWs represented by > two different rows in the pwTable, with the pwAttachedPwIndex value in > the row representing one of them set to the pwIndex of the other one and > vice versa. > Since there is no way in the SMIv2 paradigm to create two rows in the sam= e > table in a single atomic operation, setting a this attribute in a pair of= rows is > only possible when both are created. In order to do that, read-create acc= ess > mode of the pwAttachedPwIndex object has to be interpreted as ability to > set its value when the row represents an inactive PW. > In accordance with the quoted description, such an interpretation must be > explicitly specified in the description of this object. >=20 > Such a specification is missing in the current text, hence the default > interpretation of the read-create access mode is holds. > Proposed text fixes this problem. >=20 > -------------------------------------- > RFC5601 (draft-ietf-pwe3-pw-mib-14) > -------------------------------------- > Title : Pseudowire (PW) Management Information Base (MIB) > Publication Date : July 2009 > Author(s) : T. Nadeau, Ed., D. Zelig, Ed. > Category : PROPOSED STANDARD > Source : Pseudo Wire Emulation Edge to Edge INT > Area : Internet > Stream : IETF > Verifying Party : IESG From nobody Wed Aug 6 12:35:46 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2F7F31B27B3; Wed, 6 Aug 2014 12:35:44 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TAVqTw5dYYHr; Wed, 6 Aug 2014 12:35:40 -0700 (PDT) Received: from mail-qa0-x22b.google.com (mail-qa0-x22b.google.com [IPv6:2607:f8b0:400d:c00::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8BC141B27AA; Wed, 6 Aug 2014 12:35:40 -0700 (PDT) Received: by mail-qa0-f43.google.com with SMTP id w8so2999634qac.30 for ; Wed, 06 Aug 2014 12:35:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=A5IlMpTMGplnHDQr+yD27HfG5jCGOuv+HLVnSskVkqU=; b=MXryko9gfYpdJwb3b+y7r5WIVXYc0luI5Pps8XY+R6YjXkF5ycm3dx+OEHSxwifdwv 6e1+Azq7iStLGACTgIVBz8tzfIj3mVcNAk+MErRAJOzyuSLb0b8a7bv2Iu4V3FlwLMa3 1vhBHPYoUGiMODYvcLIxpnex1HQmiJCofSsg+qcW+q3Pnyq0P5EjZyAxN4crWiT4MxC9 3NvRLtyCpdbN8g8rXWmA3xuN3VZ/W3Uhpsr4sW+vBOUm+ABefw9HLTNFeUBNlXw7BfeQ H+GeJj6NheQhAU3kge34qgfh8IAQ7YV07hneMN2cDI5duNYf8I/1ZTnCQOoVpvNaWntG s8DA== X-Received: by 10.140.41.102 with SMTP id y93mr6292755qgy.90.1407353739839; Wed, 06 Aug 2014 12:35:39 -0700 (PDT) MIME-Version: 1.0 Received: by 10.140.16.22 with HTTP; Wed, 6 Aug 2014 12:35:19 -0700 (PDT) In-Reply-To: <01d71ed3a97243f386a0a7a9247e91df@AM3PR03MB612.eurprd03.prod.outlook.com> References: <20140806173601.7195B18001B@rfc-editor.org> <01d71ed3a97243f386a0a7a9247e91df@AM3PR03MB612.eurprd03.prod.outlook.com> From: "Andrew G. Malis" Date: Wed, 6 Aug 2014 15:35:19 -0400 Message-ID: To: Alexander Vainshtein Content-Type: multipart/alternative; boundary=001a11c11e2cbc8f3f04fffb1338 Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/xzJEkFvWmZdUfhIU1ogIQEe74BI Cc: "brian@innovationslab.net" , "pwe3@ietf.org" , "davidz@oversi.com" , "thomas.nadeau@bt.com" , "iesg@ietf.org" Subject: Re: [PWE3] [Errata Verified] RFC5601 (4069) X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2014 19:35:44 -0000 --001a11c11e2cbc8f3f04fffb1338 Content-Type: text/plain; charset=UTF-8 Sasha, Just to give credit where it's due, Adrian was the source of the improved text. Cheers, Andy On Wed, Aug 6, 2014 at 3:29 PM, Alexander Vainshtein < Alexander.Vainshtein@ecitele.com> wrote: > Brian, > Lots of thanks for a prompt verification and for the much better corrected > text. > > Regards, > Sasha > Email: Alexander.Vainshtein@ecitele.com > Mobile: 054-9266302 > > > > -----Original Message----- > > From: RFC Errata System [mailto:rfc-editor@rfc-editor.org] > > Sent: Wednesday, August 06, 2014 8:36 PM > > To: Alexander Vainshtein; thomas.nadeau@bt.com; davidz@oversi.com > > Cc: brian@innovationslab.net; iesg@ietf.org; pwe3@ietf.org; > rfc-editor@rfc- > > editor.org > > Subject: [Errata Verified] RFC5601 (4069) > > > > The following errata report has been verified for RFC5601, "Pseudowire > (PW) > > Management Information Base (MIB)". > > > > -------------------------------------- > > You may review the report below and at: > > http://www.rfc-editor.org/errata_search.php?rfc=5601&eid=4069 > > > > -------------------------------------- > > Status: Verified > > Type: Technical > > > > Reported by: Alexander ("Sasha") Vainshtein > > > > Date Reported: 2014-08-05 > > Verified by: Brian Haberman (IESG) > > > > Section: 12 > > > > Original Text > > ------------- > > pwAttachedPwIndex OBJECT-TYPE > > SYNTAX PwIndexOrZeroType > > MAX-ACCESS read-create > > STATUS current > > DESCRIPTION > > "If the PW is attached to another PW instead of a local > > native service, this item indicates the pwIndex of the > > attached PW. Otherwise, this object MUST > > be set to zero. Attachment to another PW will have no > > PW specific entry in any of the service MIB modules." > > DEFVAL { 0 } > > ::= { pwEntry 10 } > > > > Corrected Text > > -------------- > > pwAttachedPwIndex OBJECT-TYPE > > SYNTAX PwIndexOrZeroType > > MAX-ACCESS read-create > > STATUS current > > DESCRIPTION > > "If the PW is attached to another PW instead of a local > > native service, this item indicates the pwIndex of the > > attached PW. Otherwise, this object MUST > > be set to zero. Attachment to another PW will have no > > PW specific entry in any of the service MIB modules. > > This object may be modified only when the value of > > the associated pwAdminStatus object is down(2), and > > the associated pwOperStatus object has value down(2) > > or notPresent(5) such that the row in the pwTable > > represents an inactive PW." > > DEFVAL { 0 } > > ::= { pwEntry 10 } > > > > Notes > > ----- > > Description of the pwEntry object in the same RFC states that "The read- > > create objects in this table are divided into > > three categories: > > 1) Objects that MUST NOT be changed after row activation. > > These are objects that define basic properties of the > > PW (for example type, destination, etc.). > > 2) Objects that MAY be changed when the PW is > > defined as not active. A change of these objects involves > > re-signaling of the PW or it might be traffic affecting. > > PW not active is defined as one of the following > > conditions: > > a) The pwRowStatus is notInService(2). > > b) The pwRowStatus is notReady(3). > > c) The pwAdminStatus is down(2). > > If the operator needs to change one of the values for an > > active row, the operator can either set the pwRowStatus to > > notInService(2) or set pwAdminStatus to down(2). > > Signaling (or traffic) is initiated again upon setting > > the pwRowStatus to active(1) or setting the pwAdminStatus > > to up(1) or testing(3), respectively. > > 3) Objects that MAY be changed at any time." > > > > In further states (in tthe same description) that "By default, all the > read- > > create objects MUST NOT be changed after row activation, unless > specifically > > indicated in the individual object description." > > > > pwAttachedPwIndex object is used to stitch a couple of PWs represented by > > two different rows in the pwTable, with the pwAttachedPwIndex value in > > the row representing one of them set to the pwIndex of the other one and > > vice versa. > > Since there is no way in the SMIv2 paradigm to create two rows in the > same > > table in a single atomic operation, setting a this attribute in a pair > of rows is > > only possible when both are created. In order to do that, read-create > access > > mode of the pwAttachedPwIndex object has to be interpreted as ability to > > set its value when the row represents an inactive PW. > > In accordance with the quoted description, such an interpretation must be > > explicitly specified in the description of this object. > > > > Such a specification is missing in the current text, hence the default > > interpretation of the read-create access mode is holds. > > Proposed text fixes this problem. > > > > -------------------------------------- > > RFC5601 (draft-ietf-pwe3-pw-mib-14) > > -------------------------------------- > > Title : Pseudowire (PW) Management Information Base (MIB) > > Publication Date : July 2009 > > Author(s) : T. Nadeau, Ed., D. Zelig, Ed. > > Category : PROPOSED STANDARD > > Source : Pseudo Wire Emulation Edge to Edge INT > > Area : Internet > > Stream : IETF > > Verifying Party : IESG > > _______________________________________________ > pwe3 mailing list > pwe3@ietf.org > https://www.ietf.org/mailman/listinfo/pwe3 > --001a11c11e2cbc8f3f04fffb1338 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Sasha,

Just to give credit where it'= ;s due, Adrian was the source of the improved text.

Cheers,
Andy


On Wed, Aug 6, 2014 at 3:29 PM, Alexander Vainshtein <= Alexa= nder.Vainshtein@ecitele.com> wrote:
Brian,
Lots of thanks for a prompt verification and for the much better corrected = text.

Regards,
=C2=A0 =C2=A0 =C2=A0 =C2=A0Sasha
Email: Alexander.Vainsh= tein@ecitele.com
Mobile: 054-9266302


> -----Original Message-----
> From: RFC Errata System [mailto:rfc-editor@rfc-editor.org]
> Sent: Wednesday, August 06, 2014 8:36 PM
> To: Alexander Vainshtein; thom= as.nadeau@bt.com; davidz@oversi.co= m
> Cc: brian@innovationslab.n= et; iesg@ietf.org; pwe3@ietf.org; rfc-editor@rfc-
> editor.org
> Subject: [Errata Verified] RFC5601 (4069)
>
> The following errata report has been verified for RFC5601, "Pseud= owire (PW)
> Management Information Base (MIB)".
>
> --------------------------------------
> You may review the report below and at:
> http://www.rfc-editor.org/errata_search.php?r= fc=3D5601&eid=3D4069
>
> --------------------------------------
> Status: Verified
> Type: Technical
>
> Reported by: Alexander ("Sasha") Vainshtein
> <Alexander.Vain= shtein@ecitele.com>
> Date Reported: 2014-08-05
> Verified by: Brian Haberman (IESG)
>
> Section: 12
>
> Original Text
> -------------
> =C2=A0pwAttachedPwIndex OBJECT-TYPE
> =C2=A0 =C2=A0 =C2=A0SYNTAX =C2=A0 =C2=A0 =C2=A0 =C2=A0PwIndexOrZeroTyp= e
> =C2=A0 =C2=A0 =C2=A0MAX-ACCESS =C2=A0 =C2=A0read-create
> =C2=A0 =C2=A0 =C2=A0STATUS =C2=A0 =C2=A0 =C2=A0 =C2=A0current
> =C2=A0 =C2=A0 =C2=A0DESCRIPTION
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0"If the PW is attached to anoth= er PW instead of a local
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 native service, this item indicates= the pwIndex of the
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 attached PW. =C2=A0Otherwise, this = object MUST
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 be set to zero. =C2=A0Attachment to= another PW will have no
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 PW specific entry in any of the ser= vice MIB modules."
> =C2=A0 =C2=A0 =C2=A0DEFVAL { 0 }
> =C2=A0 =C2=A0 =C2=A0::=3D { pwEntry 10 }
>
> Corrected Text
> --------------
> =C2=A0pwAttachedPwIndex OBJECT-TYPE
> =C2=A0 =C2=A0 =C2=A0SYNTAX =C2=A0 =C2=A0 =C2=A0 =C2=A0PwIndexOrZeroTyp= e
> =C2=A0 =C2=A0 =C2=A0MAX-ACCESS =C2=A0 =C2=A0read-create
> =C2=A0 =C2=A0 =C2=A0STATUS =C2=A0 =C2=A0 =C2=A0 =C2=A0current
> =C2=A0 =C2=A0 =C2=A0DESCRIPTION
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 "If the PW is attached to anot= her PW instead of a local
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0native service, this item ind= icates the pwIndex of the
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0attached PW. =C2=A0Otherwise,= this object MUST
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0be set to zero. =C2=A0Attachm= ent to another PW will have no
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0PW specific entry in any of t= he service MIB modules.
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0This object may be modified o= nly when the value of
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0the associated pwAdminStatus = object is down(2), and
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0the associated pwOperStatus o= bject has value down(2)
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0or notPresent(5) such that th= e row in the pwTable
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0represents an inactive PW.&qu= ot;
> =C2=A0 =C2=A0 =C2=A0DEFVAL { 0 }
> =C2=A0 =C2=A0 =C2=A0::=3D { pwEntry 10 }
>
> Notes
> -----
> Description of the pwEntry object in the same RFC states that "Th= e read-
> create objects in this table are divided into
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0three categories:
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A01) Objects that MUST NOT be c= hanged after row activation.
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 These are objects tha= t define basic properties of the
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 PW (for example type,= destination, etc.).
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A02) Objects that MAY be change= d when the PW is
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 defined as not active= . =C2=A0A change of these objects involves
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 re-signaling of the P= W or it might be traffic affecting.
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 PW not active is defi= ned as one of the following
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 conditions:
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 a) The = pwRowStatus is notInService(2).
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 b) The = pwRowStatus is notReady(3).
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 c) The = pwAdminStatus is down(2).
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0If the operator needs to chan= ge one of the values for an
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0active row, the operator can = either set the pwRowStatus to
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0notInService(2) or set pwAdmi= nStatus to down(2).
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Signaling (or traffic) is ini= tiated again upon setting
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0the pwRowStatus to active(1) = or setting the pwAdminStatus
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0to up(1) or testing(3), respe= ctively.
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A03) Objects that MAY be change= d at any time."
>
> In further states (in tthe same description) that "By default, al= l the read-
> create objects MUST NOT be changed after row activation, unless specif= ically
> indicated in the individual object description."
>
> pwAttachedPwIndex object is used to stitch a couple of PWs represented= by
> two different rows in the pwTable, with the pwAttachedPwIndex value in=
> the row representing one of them set to the pwIndex of the other one a= nd
> vice versa.
> Since there is no way in the SMIv2 paradigm to create two rows in the = same
> table in a single atomic operation, setting a this attribute in a pair= of rows is
> only possible when both are created. In order to do that, read-create = access
> mode of the pwAttachedPwIndex object has to be interpreted as ability = to
> set its value when the row represents an inactive PW.
> In accordance with the quoted description, such an interpretation must= be
> explicitly specified in the description of this object.
>
> Such a specification is missing in the current text, hence the default=
> interpretation of the read-create access mode is holds.
> Proposed text fixes this problem.
>
> --------------------------------------
> RFC5601 (draft-ietf-pwe3-pw-mib-14)
> --------------------------------------
> Title =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 : Pseudowire (P= W) Management Information Base (MIB)
> Publication Date =C2=A0 =C2=A0: July 2009
> Author(s) =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 : T. Nadeau, Ed., D. Zeli= g, Ed.
> Category =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0: PROPOSED STANDARD<= br> > Source =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0: Pseudo Wire E= mulation Edge to Edge INT
> Area =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0: Internet=
> Stream =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0: IETF
> Verifying Party =C2=A0 =C2=A0 : IESG

_______________________________________________
pwe3 mailing list
pwe3@ietf.org
ht= tps://www.ietf.org/mailman/listinfo/pwe3

--001a11c11e2cbc8f3f04fffb1338-- From nobody Wed Aug 6 17:44:55 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E4FFE1A02D6 for ; Wed, 6 Aug 2014 17:44:53 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.201 X-Spam-Level: X-Spam-Status: No, score=-4.201 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FAhcldWg0U9N for ; Wed, 6 Aug 2014 17:44:52 -0700 (PDT) Received: from lhrrgout.huawei.com (lhrrgout.huawei.com [194.213.3.17]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BAB971A0294 for ; Wed, 6 Aug 2014 17:44:49 -0700 (PDT) Received: from 172.18.7.190 (EHLO lhreml405-hub.china.huawei.com) ([172.18.7.190]) by lhrrg02-dlp.huawei.com (MOS 4.3.7-GA FastPath queued) with ESMTP id BHZ38591; Thu, 07 Aug 2014 00:44:47 +0000 (GMT) Received: from SZXEMA402-HUB.china.huawei.com (10.82.72.34) by lhreml405-hub.china.huawei.com (10.201.5.242) with Microsoft SMTP Server (TLS) id 14.3.158.1; Thu, 7 Aug 2014 01:44:47 +0100 Received: from SZXEMA501-MBS.china.huawei.com ([169.254.2.128]) by SZXEMA402-HUB.china.huawei.com ([10.82.72.34]) with mapi id 14.03.0158.001; Thu, 7 Aug 2014 08:44:44 +0800 From: Puyun To: "yaakov_s@rad.com" Thread-Topic: Re: [PWE3] draft-jiang-pwe3-mc-pon Thread-Index: Ac+x2McJ2ujzgSFjT0mKBjvwUD831A== Date: Thu, 7 Aug 2014 00:44:43 +0000 Message-ID: Accept-Language: zh-CN, en-US Content-Language: zh-CN X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.146.78.176] Content-Type: multipart/alternative; boundary="_000_A811AC6DCC5A5540A74766974361DEEA86F4C285SZXEMA501MBSchi_" MIME-Version: 1.0 X-CFilter-Loop: Reflected Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/rTx3BejatkRYTXXcMSCqStHagKw Cc: "pwe3@ietf.org" Subject: Re: [PWE3] draft-jiang-pwe3-mc-pon X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Aug 2014 00:44:54 -0000 --_000_A811AC6DCC5A5540A74766974361DEEA86F4C285SZXEMA501MBSchi_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Yaakov, Type B PON protection (both single-homing and dual-homing) has been deploye= d by more than 10 service providers across China, East Europe, South Americ= a, West Asia, and Africa. We can't provide more details of the SPs but some= of them may be willing to confirm it in this mailing list. Yun PU From: pwe3 [mailto:pwe3-bounces@ietf.org] On Behalf Of Yaakov Stein Sent: Sunday, July 27, 2014 11:49 PM To: pwe3@ietf.org Subject: [PWE3] draft-jiang-pwe3-mc-pon All, As I asked at the session last week, I am interested in hearing if anyone k= nows of deployment of Type B PON protection, which is the case this draft is trying to facilit= ate. If this is heavily deployed, then I support adoption of this draft. If it is not deployed, then I think we don't need it. Y(J)S --_000_A811AC6DCC5A5540A74766974361DEEA86F4C285SZXEMA501MBSchi_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi Yaakov,

 

Type B PON protection (both sin= gle-homing and dual-homing) has been deployed by more than 10 service provi= ders across China, East Europe, South America, West Asia, and Africa. We ca= n't provide more details of the SPs but some of them may be willing to confirm it in this mailing list.

 

Yun PU

 

 

From: pwe3 [mailto= :pwe3-bounces@ietf.org] On Behalf Of Yaakov Stein
Sent: Sunday, July 27, 2014 11:49 PM
To: pwe3@ietf.org
Subject: [PWE3] draft-jiang-pwe3-mc-pon

 

All,

 

As I asked at the session last = week, I am interested in hearing if anyone knows of deployment

of Type B PON protection, which= is the case this draft is trying to facilitate.

 

If this is heavily deployed, th= en I support adoption of this draft.

If it is not deployed, then I t= hink we don’t need it.

 

Y(J)S

 

--_000_A811AC6DCC5A5540A74766974361DEEA86F4C285SZXEMA501MBSchi_-- From nobody Wed Aug 6 19:33:45 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AA1481A0652 for ; Wed, 6 Aug 2014 19:33:44 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.601 X-Spam-Level: X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id w0nw_UdV9SIe for ; Wed, 6 Aug 2014 19:33:41 -0700 (PDT) Received: from na01-bl2-obe.outbound.protection.outlook.com (mail-bl2lp0208.outbound.protection.outlook.com [207.46.163.208]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DA6161A0645 for ; Wed, 6 Aug 2014 19:33:40 -0700 (PDT) Received: from BY2PR05MB728.namprd05.prod.outlook.com (10.141.223.25) by BY2PR05MB221.namprd05.prod.outlook.com (10.242.41.154) with Microsoft SMTP Server (TLS) id 15.0.995.14; Thu, 7 Aug 2014 02:33:37 +0000 Received: from BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) by BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) with mapi id 15.00.0995.014; Thu, 7 Aug 2014 02:33:37 +0000 From: Yimin Shen To: Alexander Vainshtein Thread-Topic: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 Thread-Index: AQHPqByPzRsIO8QlbkK+kbMBZ9MDf5u3AuAAgAAZ0oCAABjewIABFRcAgADf/TCACF0AgIAA3TgQgADA4ICAADbOAIAAzcgw Date: Thu, 7 Aug 2014 02:33:37 +0000 Message-ID: <046050cb2d254a139b4650b2d7f93075@BY2PR05MB728.namprd05.prod.outlook.com> References: <201407251524.s6PFOPn92012@magenta.juniper.net> <53D79504.4050904@cisco.com> <04f15bedf2be4c7480d3e9ca01bdfd7f@BY2PR05MB728.namprd05.prod.outlook.com> <2597291f7b074922b690e4b06999cf1a@AM3PR03MB612.eurprd03.prod.outlook.com>, <30e13900814f41a681151971cbe9ebef@BY2PR05MB728.namprd05.prod.outlook.com> <1407215578702.64387@ecitele.com>, <2280b8e0d5f94a60badab91a2237181b@BY2PR05MB728.namprd05.prod.outlook.com> <1407304503657.72312@ecitele.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [66.129.241.10] x-microsoft-antispam: BCL:0;PCL:0;RULEID: x-forefront-prvs: 029651C7A1 x-forefront-antispam-report: SFV:NSPM; SFS:(252514010)(377454003)(199002)(60444003)(189002)(43784003)(164054003)(80022001)(18717965001)(2656002)(76482001)(106356001)(19300405004)(15202345003)(99396002)(87936001)(66066001)(20776003)(64706001)(19625215002)(19609705001)(31966008)(110136001)(21056001)(74502001)(93886004)(85306004)(15975445006)(74316001)(107046002)(33646002)(95666004)(86362001)(19580405001)(101416001)(83322001)(4396001)(83072002)(77982001)(19580395003)(92566001)(74662001)(16236675004)(54356999)(46102001)(76576001)(81342001)(85852003)(99286002)(81542001)(50986999)(76176999)(105586002)(79102001)(106116001)(24736002)(108616003); DIR:OUT; SFP:; SCL:1; SRVR:BY2PR05MB221; H:BY2PR05MB728.namprd05.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; LANG:en; Content-Type: multipart/alternative; boundary="_000_046050cb2d254a139b4650b2d7f93075BY2PR05MB728namprd05pro_" MIME-Version: 1.0 X-OriginatorOrg: juniper.net Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/KuxyIXDoyWlFLdcM4xKqq0fUg88 Cc: Yakov Rekhter , "pwe3@ietf.org" , "stbryant@cisco.com" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Aug 2014 02:33:44 -0000 --_000_046050cb2d254a139b4650b2d7f93075BY2PR05MB728namprd05pro_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Sasha, Thanks again for the comments. Please see the following 4 items in response= to the 4 comments in your email. 1. Today when an egress PE detects an egress AC failure, it will simply hav= e to drop packets, until the remote PE stops sending traffic over the PW. T= his draft is intended to improve this behavior. By no means the local repai= r on egress PE will prevent the AC failure from being converted to and hand= led as a bi-directional AC failure. Note that bi-directional AC failure han= dling must involve the CE, but the CE may detect the failure at a slower pa= ce, or it may not be able to detect it in a timely manner, depending on the= actual situation. E.g. there may be other L2 devices between the PE and CE= ; the AC may itself be a logical wire across another network; the type and = location of the failure may make it easier for the PE to detect it; and so = on. In any case, before the AC failure is handled more gracefully (or bi-di= rectionally), the egress traffic has been locally repaired by the egress PE= . 2. As mentioned before, this draft doesn't have a dependency on a specific = PSN tunnel technology. 3. The draft already talks about scalability when describing the centralize= d protector model vs the co-located protector model. We can improve the tex= t to explicitly mention that the requirement for setting up LSPs on a per <= primary PE, protector> basis rather than per primary PE basis can potential= ly increase the number of LSPs in a network. This is a tradeoff. As mention= ed before, this does not necessarily cause a scaling issue. 4. We will leave this to PWE3 WG to decide whether this is really necessary= . Thanks, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Wednesday, August 06, 2014 5:11 AM To: Yimin Shen Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin and all, I would like to summarize the issues that I have with this draft. Digging f= or them in a very long email thread with multiple levels of in-lining is no= t very effective IMO. 1. Protection against unidirectional AC failures looks problematic to= me: a. The draft explicitly differentiates between "ingress" and "egress"= AC failures b. The outcome can be using a different AC for each direction of CE-to= -CE traffic with potentially destructive effect that cannot be mitigated at= the PWE3 level c. IMHO and FWIW a viable alternative could be to employ native AC me= chanisms converting unidirectional AC failures into bi-directional ones. Th= is would eliminate the differentiation between ingress and egress AC failur= es and facilitate using the PW redundancy mechanism for bi-directional prot= ection 2. Dependency upon specific mechanisms for setting up PSN tunnels see= ms to break some common PWE3 architectural principles: a. I am pretty well sure that the mechanisms specified in the draft c= annot be used with the PSN tunnel LSPs set up by LDP in DU mode b. If the authors, refer to LDP in the DoD mode augmented with some CR= -LDP-like mechanisms (e.g., for exclusion of specific links when setting up= the bypass LSP between the PLR and the Protector), this should, at the ver= y least, be discussed with and approved by the MPLS WG 3. Scalability of the proposed solution should be, at least, explicit= ly presented. In particular, the draft assumes that a dedicated LSP would b= e used between PE1 PE2 for each combination of PE2 with a specific Protect= or LSR. 4. The Protector LSR could be either a kind of S-PE (if different fro= m the secondary T-PE) or the secondary T-PE itself. In both cases it should= be capable of handling labels from specific context spaces as if they were= PW labels. This requires (at the very least) an update of RFC 4447 and sho= uld be explicitly presented and discussed. Hopefully this summary will be useful. Regards, Sasha Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 --_000_046050cb2d254a139b4650b2d7f93075BY2PR05MB728namprd05pro_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi Sasha,

 

Thanks again for the c= omments. Please see the following 4 items in response to the 4 comments in = your email.

 

1. Today when an egres= s PE detects an egress AC failure, it will simply have to drop packets, unt= il the remote PE stops sending traffic over the PW. This draft is intended = to improve this behavior. By no means the local repair on egress PE will prevent the AC failure from being conve= rted to and handled as a bi-directional AC failure. Note that bi-directiona= l AC failure handling must involve the CE, but the CE may detect the failur= e at a slower pace, or it may not be able to detect it in a timely manner, depending on the actual situation= . E.g. there may be other L2 devices between the PE and CE; the AC may itse= lf be a logical wire across another network; the type and location of the f= ailure may make it easier for the PE to detect it; and so on. In any case, before the AC failure is handled = more gracefully (or bi-directionally), the egress traffic has been locally = repaired by the egress PE.

 

2. As mentioned before= , this draft doesn’t have a dependency on a specific PSN tunnel techn= ology.

 

3. The draft already t= alks about scalability when describing the centralized protector model vs t= he co-located protector model. We can improve the text to explicitly mentio= n that the requirement for setting up LSPs on a per <primary PE, protector> basis rather than per primary = PE basis can potentially increase the number of LSPs in a network. This is = a tradeoff. As mentioned before, this does not necessarily cause a scaling = issue.

 

4. We will leave this = to PWE3 WG to decide whether this is really necessary.

 

Thanks,

 

/Yimin

 

 

From: Alexande= r Vainshtein [mailto:Al= exander.Vainshtein@ecitele.com]
Sent: Wednesday, August 06, 2014 5:11 AM
To: Yimin Shen
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Yimin and all,

I would like to summarize the issues that I have = with this draft. Digging for them in a very long email thread with multiple= levels of in-lining is not very effective IMO.

 

1.       Protection against unidirectional AC failures looks problematic to me:

a.       The draft explicitly differentiates between “= ingress” and “egress” AC failures

b.      The outcome can be using a different AC for each di= rection of CE-to-CE traffic with potentially destructive effect that cannot= be mitigated at the PWE3 level

c.       IMHO and FWIW a viable alternative could be to empl= oy native AC mechanisms converting unidirectional AC failures into bi-direc= tional ones. This would eliminate the differentiation between ingress and e= gress AC failures and facilitate using the PW redundancy mechanism for bi-directional protection

2.       Dependency upon specific mechanisms for setting = up PSN tunnels seems to break some common PWE3 architectural principles= :

a.       I am pretty well sure that the mechanisms specified= in the draft cannot be used with the PSN tunnel LSPs set up by LDP in DU mode

b.      If the authors, refer to LDP in the DoD mode= augmented with some CR-LDP-like mechanisms (e.g., for exclusion of specifi= c links when setting up the bypass LSP between the PLR and the Protector), = this should, at the very least, be discussed with and approved by the MPLS WG

3.       Scalability of the proposed solution should = be, at least, explicitly presented. In particular, the draft assumes that a= dedicated LSP would be used between PE1  PE2 for each combination of = PE2 with a specific Protector LSR.

4.       The Protector LSR could be either a kind of S-PE (i= f different from the secondary T-PE) or the secondary T-PE itself. In both = cases it should be capable of handling labels from specific context spaces = as if they were PW labels. This requires (at the very least) an update of RFC 4447 and should be explicitl= y presented and discussed.

Hopefully this summary will be useful.=

 

 

Regards,

       Sasha <= /o:p>

Email: Alexander.Vainshtein@ecitele.com

Mobile: 054-9266302

--_000_046050cb2d254a139b4650b2d7f93075BY2PR05MB728namprd05pro_-- From nobody Wed Aug 6 20:57:15 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CE3831A0A96; Wed, 6 Aug 2014 20:57:13 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.202 X-Spam-Level: X-Spam-Status: No, score=-4.202 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 69_fecizhvne; Wed, 6 Aug 2014 20:57:08 -0700 (PDT) Received: from lhrrgout.huawei.com (lhrrgout.huawei.com [194.213.3.17]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 898081A0A86; Wed, 6 Aug 2014 20:57:07 -0700 (PDT) Received: from 172.18.7.190 (EHLO lhreml402-hub.china.huawei.com) ([172.18.7.190]) by lhrrg01-dlp.huawei.com (MOS 4.3.7-GA FastPath queued) with ESMTP id BKY99311; Thu, 07 Aug 2014 03:57:05 +0000 (GMT) Received: from nkgeml405-hub.china.huawei.com (10.98.56.36) by lhreml402-hub.china.huawei.com (10.201.5.241) with Microsoft SMTP Server (TLS) id 14.3.158.1; Thu, 7 Aug 2014 04:57:05 +0100 Received: from NKGEML512-MBX.china.huawei.com ([169.254.7.78]) by nkgeml405-hub.china.huawei.com ([10.98.56.36]) with mapi id 14.03.0158.001; Thu, 7 Aug 2014 11:56:58 +0800 From: Mingui Zhang To: "erosen@cisco.com" , Alexander Vainshtein Thread-Topic: [PWE3] [mpls] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 Thread-Index: AQHPsY7DDiJw7y3ZEk+Q1kSqiialjpvEXftQ Date: Thu, 7 Aug 2014 03:56:57 +0000 Message-ID: <4552F0907735844E9204A62BBDD325E76AAAA7F4@nkgeml512-mbx.china.huawei.com> References: Your message of Tue, 05 Aug 2014 13:59:49 -0000. <9696d0db139d46ffaad7be11340215e8@AM3PR03MB612.eurprd03.prod.outlook.com> <16167.1407340459@erosen-lnx> In-Reply-To: <16167.1407340459@erosen-lnx> Accept-Language: en-US, zh-CN Content-Language: zh-CN X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.111.102.175] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-CFilter-Loop: Reflected Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/fEc7Yrq6B80yBmdGz1OcGOjarIc Cc: "mpls@ietf.org" , pwe3 , "pwe3-chairs@tools.ietf.org" , "stbryant@cisco.com" Subject: Re: [PWE3] [mpls] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Aug 2014 03:57:14 -0000 Hi Eric,=20 The explanation of the story is very clear. Let me complement a bit. >- A "primary egress PE" partitions its set of PWs into n sets, where each > set is associated with a given "protector". The "protector" is the back= up > egress PE for those PWs. When the backup PE acts as the protector, it is the 'co-located' model. Acc= ording to the text in the draft, the protector can also be any other PEs or= Ps. That is the 'centralized' model. In this model, operators are allowed = to designate the protector. Authors can correct me if I am wrong. >However, when protecting against the failure of the egress AC, packets wou= ld >go first to PE2 then to PE3. This scenario is a bit different, in that: > >- I think this scenario may require an RSVP-TE backup tunnel (on which PHP > is not used) from PE2 to PE3. > >- PE2 would have to signal PE3 to associate the given PW with the given > backup tunnel. > >- The label that PE3 assigns to that backup tunnel would identify the > context in which the PW label is looked up. > >This is perhaps more detail than is actually in the draft ;-) but if it is >an accurate description of the authors' intentions, the scheme seems to >work. I think the failure of the egress AC can't be handled as normal because the= failure does not break the anycast IP address. Since this IP address used = to be set to the loopback IP address of the PE. We know it's alive even tho= se physical interfaces are broken. So I understand why you advise to handle= the AC failures separately using the RSVP-TE backup tunnel. I think we still have the chance to incorporate it into the single LDP base= d scheme: We can replace the anycast IP address with an IP address of a virtual route= r. A link between the primary PE and the virtual router will appear in the = routing. When the primary PE detect the AC failure. It acts as the PLR. It = just translates it into a link failure of the routing and starts to use the= backup tunnel that is prepared in advance for the link failure. One concern is that "do we have scalability issue with these backup tunnels= "? IOW, do we have to set up backup tunnels per AC? The answer is no. Actua= lly, a set of ACs can multiplex the same backup tunnel if their correspondi= ng CEs are multi-homed to the same pair of . Thanks, Mingui >-----Original Message----- >From: pwe3 [mailto:pwe3-bounces@ietf.org] On Behalf Of Eric Rosen >Sent: Wednesday, August 06, 2014 11:54 PM >To: Alexander Vainshtein >Cc: mpls@ietf.org; pwe3; pwe3-chairs@tools.ietf.org; stbryant@cisco.com >Subject: Re: [PWE3] [mpls] WG Last Call for >draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 > >I've looked at this draft, and it seems to me that it does work in LDP-bas= ed >networks (even with DU mode), and that it does not violate any architectur= al >principles. However, I have made certain assumptions that aren't explicit= ly >stated in the draft. (I am assuming that the draft is intended to be >patterned after the "anycast BGP" scheme that I've heard about about, but >that is not written up anywhere as far as I know.) Let me say how I think >it is intended to work, and the authors can correct me if I'm wrong. > >For protection against the failure of the "primary" egress PE: > >- A "primary egress PE" partitions its set of PWs into n sets, where each > set is associated with a given "protector". The "protector" is the back= up > egress PE for those PWs. > >- Each such set is associated with an IP address. This IP address must be > known both to the primary PE and to the protector. > > * One can think of this address as an "anycast" address that, to routing= , > denotes both the primary PE and the protector. At the primary PE and > the protector, the address denotes a particular set of PWs. > > * These anycast addresses are distributed by routing, and good old > fashioned MPLS labels get assigned to them by LDP. The primary PE MAY > bind implicit null to that address, but the protector MUST NOT bind > implicit null to that address. > > * Routing is set up so that (a) if the primary PE is up, traffic to the > anycast address always goes to the primary PE, but (b) if the primary = PE > is down, traffic to the anycast address goes to the protector instead. > This can be achieved by messing with the metrics, e.g. > >- For a given PW, the primary PE informs both the protector and the ingres= s > PE of (a) the label that it (the primary PE) has assigned to that PW, a= nd > the "anycast address" that corresponds to that PW. > > This requires some new signaling (beyond that specified in RFC 4447), bu= t > the requisite signaling is specified in the draft. > >- When sending a packet on the given PW, the ingress PE pushes on the labe= l > assigned to the PW by the egress PE. Then the ingress PE pushes on > whatever LDP-assigned label it need to use to get the packet to the > anycast address that is associated with the PW. > >- When the protector learns that the egress PE has assigned a given label > and anycast address to the PW, it stores that label in a context-specifi= c > label space associated with that anycast address. Whenever the protecto= r > receives a packet whose top label is the label that the protector has > bound to the given anycast address, the protector looks up the next labe= l > in the corresponding label space. > >Let's look at an example: > >CE1---PE1----PLR----PE2----CE2 > | | > |-----PE3-----| > >where PE3 is the protector for a PW connecting CE1 to CE2 (via PE1 and PE2 >respectively). Call this pseudowire PW1. > >Thus there is some IP address, call it A, that PE2 and PE3 both advertise = to >routing. The metrics are set up so that as long as PE2 is up, it appears = to >be a better path to A than does PE3. > >So PE2 assigns label L1 to PW1, and uses T-LDP to inform both PE1 and PE3 = of >this mapping. To PE1, these are just outgoing labels. To PE3, however, >they are incoming labels in a context-specific label space. > >PE2 and PE3 both assign labels to A (let's call these labels L2 and L3 >respectively), and distribute the corresponding label bindings to PLR. PL= R >also assigns a label to A, and distributes that label binding (L4) to PE1. > >When PE1 has a packet to send on PW1, it pushes L1, then pushes L4, then >sends the packet to PLR. PLR swaps L4 with L2 and sends the packet to PE2= . >(Note that L2 could be implicit NULL.) PE2 sees label L1, and sends the >packet to CE2. > >Now if PE2 goes down, things happen a bit differently. When PLR gets a >packet with L4 on top, it swaps L4 with L3, and sends the packet to PE3. = (In >this case, of course, L3 cannot be implicit NULL.) > >PE3 gets the packet, sees L3 on top, looks it up, and then says "L3 is a >context label, so I have to pop it and then look up the following label in >the L3-specific label table." This context-specific label table contains >L1, which has been bound to the PW that leads to CE2. So PE2 pops the lab= el >and sends the packet to CE2. > >I think that's basically it, and the necessary mechanisms do seem to be >described in the draft. The draft doesn't actually use the notion of >"anycast address" explicitly, but I think the intention is that the "conte= xt >labels" are definitely bound to anycast addresses. I hope the authors wil= l >correct me if I'm wrong. > >Although the above example shows a PLR that is a neighbor of both PE2 and >PE3, this isn't really necessary, one could easily construct an example >where PE2 and PE3 have no neighbors in common. > >The example above doesn't rely on LDP DoD at all, and doesn't make use of >any RSVP-TE backup tunnels. > >However, when protecting against the failure of the egress AC, packets wou= ld >go first to PE2 then to PE3. This scenario is a bit different, in that: > >- I think this scenario may require an RSVP-TE backup tunnel (on which PHP > is not used) from PE2 to PE3. > >- PE2 would have to signal PE3 to associate the given PW with the given > backup tunnel. > >- The label that PE3 assigns to that backup tunnel would identify the > context in which the PW label is looked up. > >This is perhaps more detail than is actually in the draft ;-) but if it is >an accurate description of the authors' intentions, the scheme seems to >work. > >I don't think this modifies the MPLS architecture at all. It is true that >the PE3 needs to maintain a context-specific label table that contains >labels assigned by PE2. While one may say that that is a "coordinated lab= el >space", it is no more than what is required by the nature of >"upstream-assigned labels". I.e., if you can't support this level of >coordination, you just can't support upstream-assigned labels. > >Also, I don't think this draft is an "update" to RFC 4447. You don't need >to read this draft to implement RFC 4447, nor does it modify the procedure= s >of RFC 4447. It just introduces some new optional signaling and procedure= s. >I don't think RFC 4447's requirement to use the "platform label space" >should be interpreted as prohibiting the use of upstream-assigned labels. >The intention of RFC 4447 was to prohibit the use of the interface-specifi= c >label space for PW labels. This prohibition was necessary because the >ingress PE doesn't know which of the egress PE's network-facing interfaces >will actually receive a given packet. This draft doesn't have that proble= m, >as it provides clear procedures for determining the context in which the >upstream-assigned labels are looked up. > > > >_______________________________________________ >pwe3 mailing list >pwe3@ietf.org >https://www.ietf.org/mailman/listinfo/pwe3 From nobody Wed Aug 6 21:01:10 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D65D51A0A99; Wed, 6 Aug 2014 21:01:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.601 X-Spam-Level: X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dBV6M_OOihbP; Wed, 6 Aug 2014 21:01:05 -0700 (PDT) Received: from emea01-am1-obe.outbound.protection.outlook.com (mail-am1lrp0012.outbound.protection.outlook.com [213.199.154.12]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7FE041A0A96; Wed, 6 Aug 2014 21:01:04 -0700 (PDT) Received: from DB4PR03MB623.eurprd03.prod.outlook.com (10.141.235.14) by DB4PR03MB622.eurprd03.prod.outlook.com (10.141.234.28) with Microsoft SMTP Server (TLS) id 15.0.995.14; Thu, 7 Aug 2014 04:01:01 +0000 Received: from DB4PR03MB623.eurprd03.prod.outlook.com ([10.141.235.14]) by DB4PR03MB623.eurprd03.prod.outlook.com ([10.141.235.14]) with mapi id 15.00.0995.014; Thu, 7 Aug 2014 04:01:01 +0000 From: Alexander Vainshtein To: "Andrew G. Malis" , Adrian Farrel Thread-Topic: [PWE3] [Errata Verified] RFC5601 (4069) Thread-Index: AQHPsZ0cTt9B/tK/qE6RM/UugAdBVZvD9ctAgAACF4CAAIxtGA== Date: Thu, 7 Aug 2014 04:01:00 +0000 Message-ID: <1407384054140.37254@ecitele.com> References: <20140806173601.7195B18001B@rfc-editor.org> <01d71ed3a97243f386a0a7a9247e91df@AM3PR03MB612.eurprd03.prod.outlook.com>, In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [164.40.145.154] x-microsoft-antispam: BCL:0;PCL:0;RULEID: x-forefront-prvs: 029651C7A1 x-forefront-antispam-report: SFV:NSPM; SFS:(189002)(199002)(377424004)(377454003)(24454002)(51704005)(13464003)(252514010)(92566001)(74502001)(95666004)(31966008)(85306004)(74662001)(19617315012)(106116001)(99396002)(16236675004)(15975445006)(46102001)(64706001)(79102001)(86362001)(20776003)(81542001)(92726001)(87936001)(36756003)(106356001)(2656002)(105586002)(85852003)(4396001)(83072002)(16601075003)(80022001)(101416001)(76176999)(76482001)(19580395003)(19580405001)(83322001)(19625215002)(15202345003)(16297215004)(50986999)(107046002)(16799955002)(15188155005)(66066001)(19627405001)(81342001)(77982001)(21056001)(54356999); DIR:OUT; SFP:; SCL:1; SRVR:DB4PR03MB622; H:DB4PR03MB623.eurprd03.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; LANG:en; Content-Type: multipart/alternative; boundary="_000_140738405414037254ecitelecom_" MIME-Version: 1.0 X-OriginatorOrg: ecitele.com Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/bR8d3bJGbV3RJfFceXG79fNuR2s Cc: "brian@innovationslab.net" , "pwe3@ietf.org" , "davidz@oversi.com" , "thomas.nadeau@bt.com" , "iesg@ietf.org" Subject: Re: [PWE3] [Errata Verified] RFC5601 (4069) X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Aug 2014 04:01:09 -0000 --_000_140738405414037254ecitelecom_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Andy, Lots of thanks for an important input. There was no way I could guess that = from the Errata System report as it only mentions Brian as the verifier. Adrian, Brian Lots of thanks to both of you! Regards, Sasha ________________________________ From: Andrew G. Malis Sent: Wednesday, August 6, 2014 10:35 PM To: Alexander Vainshtein Cc: brian@innovationslab.net; davidz@oversi.com; pwe3@ietf.org; iesg@ietf.o= rg; thomas.nadeau@bt.com Subject: Re: [PWE3] [Errata Verified] RFC5601 (4069) Sasha, Just to give credit where it's due, Adrian was the source of the improved t= ext. Cheers, Andy On Wed, Aug 6, 2014 at 3:29 PM, Alexander Vainshtein > wrote: Brian, Lots of thanks for a prompt verification and for the much better corrected = text. Regards, Sasha Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 > -----Original Message----- > From: RFC Errata System [mailto:rfc-editor@rfc-editor.org] > Sent: Wednesday, August 06, 2014 8:36 PM > To: Alexander Vainshtein; thomas.nadeau@bt.com; davidz@oversi.com > Cc: brian@innovationslab.net; iesg@ietf.= org; pwe3@ietf.org; rfc-editor@= rfc- > editor.org > Subject: [Errata Verified] RFC5601 (4069) > > The following errata report has been verified for RFC5601, "Pseudowire (P= W) > Management Information Base (MIB)". > > -------------------------------------- > You may review the report below and at: > http://www.rfc-editor.org/errata_search.php?rfc=3D5601&eid=3D4069 > > -------------------------------------- > Status: Verified > Type: Technical > > Reported by: Alexander ("Sasha") Vainshtein > > > Date Reported: 2014-08-05 > Verified by: Brian Haberman (IESG) > > Section: 12 > > Original Text > ------------- > pwAttachedPwIndex OBJECT-TYPE > SYNTAX PwIndexOrZeroType > MAX-ACCESS read-create > STATUS current > DESCRIPTION > "If the PW is attached to another PW instead of a local > native service, this item indicates the pwIndex of the > attached PW. Otherwise, this object MUST > be set to zero. Attachment to another PW will have no > PW specific entry in any of the service MIB modules." > DEFVAL { 0 } > ::=3D { pwEntry 10 } > > Corrected Text > -------------- > pwAttachedPwIndex OBJECT-TYPE > SYNTAX PwIndexOrZeroType > MAX-ACCESS read-create > STATUS current > DESCRIPTION > "If the PW is attached to another PW instead of a local > native service, this item indicates the pwIndex of the > attached PW. Otherwise, this object MUST > be set to zero. Attachment to another PW will have no > PW specific entry in any of the service MIB modules. > This object may be modified only when the value of > the associated pwAdminStatus object is down(2), and > the associated pwOperStatus object has value down(2) > or notPresent(5) such that the row in the pwTable > represents an inactive PW." > DEFVAL { 0 } > ::=3D { pwEntry 10 } > > Notes > ----- > Description of the pwEntry object in the same RFC states that "The read- > create objects in this table are divided into > three categories: > 1) Objects that MUST NOT be changed after row activation. > These are objects that define basic properties of the > PW (for example type, destination, etc.). > 2) Objects that MAY be changed when the PW is > defined as not active. A change of these objects involves > re-signaling of the PW or it might be traffic affecting. > PW not active is defined as one of the following > conditions: > a) The pwRowStatus is notInService(2). > b) The pwRowStatus is notReady(3). > c) The pwAdminStatus is down(2). > If the operator needs to change one of the values for an > active row, the operator can either set the pwRowStatus to > notInService(2) or set pwAdminStatus to down(2). > Signaling (or traffic) is initiated again upon setting > the pwRowStatus to active(1) or setting the pwAdminStatus > to up(1) or testing(3), respectively. > 3) Objects that MAY be changed at any time." > > In further states (in tthe same description) that "By default, all the re= ad- > create objects MUST NOT be changed after row activation, unless specifica= lly > indicated in the individual object description." > > pwAttachedPwIndex object is used to stitch a couple of PWs represented by > two different rows in the pwTable, with the pwAttachedPwIndex value in > the row representing one of them set to the pwIndex of the other one and > vice versa. > Since there is no way in the SMIv2 paradigm to create two rows in the sam= e > table in a single atomic operation, setting a this attribute in a pair of= rows is > only possible when both are created. In order to do that, read-create acc= ess > mode of the pwAttachedPwIndex object has to be interpreted as ability to > set its value when the row represents an inactive PW. > In accordance with the quoted description, such an interpretation must be > explicitly specified in the description of this object. > > Such a specification is missing in the current text, hence the default > interpretation of the read-create access mode is holds. > Proposed text fixes this problem. > > -------------------------------------- > RFC5601 (draft-ietf-pwe3-pw-mib-14) > -------------------------------------- > Title : Pseudowire (PW) Management Information Base (MIB) > Publication Date : July 2009 > Author(s) : T. Nadeau, Ed., D. Zelig, Ed. > Category : PROPOSED STANDARD > Source : Pseudo Wire Emulation Edge to Edge INT > Area : Internet > Stream : IETF > Verifying Party : IESG _______________________________________________ pwe3 mailing list pwe3@ietf.org https://www.ietf.org/mailman/listinfo/pwe3 --_000_140738405414037254ecitelecom_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Andy,

Lots of thanks for an important input. There was no way I coul= d guess that from the Errata System report as it only mentions Brian as the= verifier.

 

Adrian, Brian

Lots of thanks to both of you!

 

Regards,

Sasha

From: Andrew G. Malis <= ;agmalis@gmail.com>
Sent: Wednesday, August 6, 2014 10:35 PM
To: Alexander Vainshtein
Cc: brian@innovationslab.net; davidz@oversi.com; pwe3@ietf.org; iesg= @ietf.org; thomas.nadeau@bt.com
Subject: Re: [PWE3] [Errata Verified] RFC5601 (4069)
 
Sasha,

Just to give credit where it's due, Adrian was the source of the impro= ved text.

Cheers,
Andy


On Wed, Aug 6, 2014 at 3:29 PM, Alexander Vainsh= tein <A= lexander.Vainshtein@ecitele.com> wrote:
Brian,
Lots of thanks for a prompt verification and for the much better corrected = text.

Regards,
       Sasha
Email: Alexander.Vainsh= tein@ecitele.com
Mobile: 054-9266302


> -----Original Message-----
> From: RFC Errata System [mailto:rfc-editor@rfc-editor.org]
> Sent: Wednesday, August 06, 2014 8:36 PM
> To: Alexander Vainshtein; thom= as.nadeau@bt.com; davidz@oversi.com
> Cc: brian@innovationslab.n= et; iesg@ietf.org; pwe3@ietf.org; rfc-= editor@rfc-
> editor.org
> Subject: [Errata Verified] RFC5601 (4069)
>
> The following errata report has been verified for RFC5601, "Pseud= owire (PW)
> Management Information Base (MIB)".
>
> --------------------------------------
> You may review the report below and at:
> http://www.rfc-editor.org/errata_search.php?rfc=3D5601&eid=3D4069 >
> --------------------------------------
> Status: Verified
> Type: Technical
>
> Reported by: Alexander ("Sasha") Vainshtein
> <Alexander.Vain= shtein@ecitele.com>
> Date Reported: 2014-08-05
> Verified by: Brian Haberman (IESG)
>
> Section: 12
>
> Original Text
> -------------
>  pwAttachedPwIndex OBJECT-TYPE
>      SYNTAX        PwIndexOrZeroTyp= e
>      MAX-ACCESS    read-create
>      STATUS        current
>      DESCRIPTION
>          "If the PW is attached to anoth= er PW instead of a local
>           native service, this item indicates= the pwIndex of the
>           attached PW.  Otherwise, this = object MUST
>           be set to zero.  Attachment to= another PW will have no
>           PW specific entry in any of the ser= vice MIB modules."
>      DEFVAL { 0 }
>      ::=3D { pwEntry 10 }
>
> Corrected Text
> --------------
>  pwAttachedPwIndex OBJECT-TYPE
>      SYNTAX        PwIndexOrZeroTyp= e
>      MAX-ACCESS    read-create
>      STATUS        current
>      DESCRIPTION
>           "If the PW is attached to anot= her PW instead of a local
>            native service, this item ind= icates the pwIndex of the
>            attached PW.  Otherwise,= this object MUST
>            be set to zero.  Attachm= ent to another PW will have no
>            PW specific entry in any of t= he service MIB modules.
>            This object may be modified o= nly when the value of
>            the associated pwAdminStatus = object is down(2), and
>            the associated pwOperStatus o= bject has value down(2)
>            or notPresent(5) such that th= e row in the pwTable
>            represents an inactive PW.&qu= ot;
>      DEFVAL { 0 }
>      ::=3D { pwEntry 10 }
>
> Notes
> -----
> Description of the pwEntry object in the same RFC states that "Th= e read-
> create objects in this table are divided into
>            three categories:
>            1) Objects that MUST NOT be c= hanged after row activation.
>               These are objects tha= t define basic properties of the
>               PW (for example type,= destination, etc.).
>            2) Objects that MAY be change= d when the PW is
>               defined as not active= .  A change of these objects involves
>               re-signaling of the P= W or it might be traffic affecting.
>               PW not active is defi= ned as one of the following
>               conditions:
>                   a) The = pwRowStatus is notInService(2).
>                   b) The = pwRowStatus is notReady(3).
>                   c) The = pwAdminStatus is down(2).
>            If the operator needs to chan= ge one of the values for an
>            active row, the operator can = either set the pwRowStatus to
>            notInService(2) or set pwAdmi= nStatus to down(2).
>            Signaling (or traffic) is ini= tiated again upon setting
>            the pwRowStatus to active(1) = or setting the pwAdminStatus
>            to up(1) or testing(3), respe= ctively.
>            3) Objects that MAY be change= d at any time."
>
> In further states (in tthe same description) that "By default, al= l the read-
> create objects MUST NOT be changed after row activation, unless specif= ically
> indicated in the individual object description."
>
> pwAttachedPwIndex object is used to stitch a couple of PWs represented= by
> two different rows in the pwTable, with the pwAttachedPwIndex value in=
> the row representing one of them set to the pwIndex of the other one a= nd
> vice versa.
> Since there is no way in the SMIv2 paradigm to create two rows in the = same
> table in a single atomic operation, setting a this attribute in a pair= of rows is
> only possible when both are created. In order to do that, read-create = access
> mode of the pwAttachedPwIndex object has to be interpreted as ability = to
> set its value when the row represents an inactive PW.
> In accordance with the quoted description, such an interpretation must= be
> explicitly specified in the description of this object.
>
> Such a specification is missing in the current text, hence the default=
> interpretation of the read-create access mode is holds.
> Proposed text fixes this problem.
>
> --------------------------------------
> RFC5601 (draft-ietf-pwe3-pw-mib-14)
> --------------------------------------
> Title               : Pseudowire (P= W) Management Information Base (MIB)
> Publication Date    : July 2009
> Author(s)           : T. Nadeau, Ed., D. Zeli= g, Ed.
> Category            : PROPOSED STANDARD<= br> > Source              : Pseudo Wire E= mulation Edge to Edge INT
> Area                : Internet=
> Stream              : IETF
> Verifying Party     : IESG

_______________________________________________
pwe3 mailing list
pwe3@ietf.org
ht= tps://www.ietf.org/mailman/listinfo/pwe3

--_000_140738405414037254ecitelecom_-- From nobody Thu Aug 7 00:27:03 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 894E31B2855; Thu, 7 Aug 2014 00:27:00 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -14.502 X-Spam-Level: X-Spam-Status: No, score=-14.502 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id S86jMkb95Nbb; Thu, 7 Aug 2014 00:26:59 -0700 (PDT) Received: from alln-iport-1.cisco.com (alln-iport-1.cisco.com [173.37.142.88]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D0B4D1B27E0; Thu, 7 Aug 2014 00:26:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=634; q=dns/txt; s=iport; t=1407396418; x=1408606018; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=5KZ22G/ouSAJcq5vd2Bti2HyrxM1KiVw8+pPWdbK6ZE=; b=mi/BbSWmcYZJ+vABruFWDqfszVGgCeeKSaENvbvdaV3pFIMHTbSihl3O i+EJKwxRzWkXhsRZnNCrirpWTNGscURmLGhzb+XS8qwn8lGD3dQwpCd5a 0rwvSHRhrftqt9sIR0h71RojWpqB1Fa1EAwlKxoQQfHZw0Bgt7z2130pg g=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AikFABIq41OtJA2B/2dsb2JhbABagw2BKa8KpREBgRMWd4QDAQEBAwEdHT8FCwIBCBgeEDIlAgQOBYg6CMNuF4wdgkQ4MweDL4EcAQScGJRtghGBRmw X-IronPort-AV: E=Sophos;i="5.01,816,1400025600"; d="scan'208";a="67168641" Received: from alln-core-9.cisco.com ([173.36.13.129]) by alln-iport-1.cisco.com with ESMTP; 07 Aug 2014 07:26:57 +0000 Received: from xhc-rcd-x05.cisco.com (xhc-rcd-x05.cisco.com [173.37.183.79]) by alln-core-9.cisco.com (8.14.5/8.14.5) with ESMTP id s777Qvfx015491 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Thu, 7 Aug 2014 07:26:57 GMT Received: from xmb-rcd-x04.cisco.com ([169.254.8.176]) by xhc-rcd-x05.cisco.com ([173.37.183.79]) with mapi id 14.03.0123.003; Thu, 7 Aug 2014 02:26:57 -0500 From: "Stewart Bryant (stbryant)" To: Mingui Zhang Thread-Topic: [PWE3] [mpls] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 Thread-Index: AQHPsY7DDiJw7y3ZEk+Q1kSqiialjpvEXftQgABg1hg= Date: Thu, 7 Aug 2014 07:26:56 +0000 Message-ID: <22D4AECA-2D36-4F79-98CB-96E4B9BDC126@cisco.com> References: Your message of Tue, 05 Aug 2014 13:59:49 -0000. <9696d0db139d46ffaad7be11340215e8@AM3PR03MB612.eurprd03.prod.outlook.com> <16167.1407340459@erosen-lnx>, <4552F0907735844E9204A62BBDD325E76AAAA7F4@nkgeml512-mbx.china.huawei.com> In-Reply-To: <4552F0907735844E9204A62BBDD325E76AAAA7F4@nkgeml512-mbx.china.huawei.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/TmEV5EAVE9C30fZrEVraH28kDmQ Cc: "mpls@ietf.org" , "Eric Rosen \(erosen\)" , pwe3 , "pwe3-chairs@tools.ietf.org" Subject: Re: [PWE3] [mpls] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Aug 2014 07:27:00 -0000 Sent from my iPad > On 7 Aug 2014, at 04:57, "Mingui Zhang" wrote: >=20 > Hi Eric,=20 >=20 > The explanation of the story is very clear. Let me complement a bit. >=20 >> - A "primary egress PE" partitions its set of PWs into n sets, where eac= h >> set is associated with a given "protector". The "protector" is the back= up >> egress PE for those PWs. >=20 > When the backup PE acts as the protector, it is the 'co-located' model.=20 This case does not however need context labels. It is a form of S-PE and ca= n stitch the PWs just like every existing S-PE already does. Stewart From nobody Thu Aug 7 02:58:14 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 548E91B2A3B; Thu, 7 Aug 2014 02:58:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zQE5VLTRqSaW; Thu, 7 Aug 2014 02:58:10 -0700 (PDT) Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id EA14E1B2A30; Thu, 7 Aug 2014 02:58:09 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: internet-drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 5.6.2.p5 Auto-Submitted: auto-generated Precedence: bulk Message-ID: <20140807095809.32627.78638.idtracker@ietfa.amsl.com> Date: Thu, 07 Aug 2014 02:58:09 -0700 Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/EwzjLX6W7YjeJ-4At2nRaQ5e3Bs Cc: pwe3@ietf.org Subject: [PWE3] I-D Action: draft-ietf-pwe3-mspw-er-05.txt X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Aug 2014 09:58:11 -0000 A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Pseudowire Emulation Edge to Edge Working Group of the IETF. Title : Explicit Path Routing for Dynamic Multi-Segment Pseudowires Authors : Pranjal Kumar Dutta Matthew Bocci Luca Martini Filename : draft-ietf-pwe3-mspw-er-05.txt Pages : 9 Date : 2014-08-07 Abstract: Dynamic Multi-Segment Pseudowire (MS-PW) setup through an explicit path may be required to provide a simple solution for 1:1 protection with diverse primary and backup MS-PWs for a service, or to enable controlled signaling (strict or loose) for special MS-PWs. This document specifies the extensions and procedures required to enable dynamic MS-PWs to be established along explicit paths. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-pwe3-mspw-er/ There's also a htmlized version available at: http://tools.ietf.org/html/draft-ietf-pwe3-mspw-er-05 A diff from the previous version is available at: http://www.ietf.org/rfcdiff?url2=draft-ietf-pwe3-mspw-er-05 Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From nobody Thu Aug 7 06:52:38 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D19F61B2B32; Thu, 7 Aug 2014 06:52:33 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.002 X-Spam-Level: X-Spam-Status: No, score=-2.002 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, J_CHICKENPOX_35=0.6, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id e53Ajc-MODFj; Thu, 7 Aug 2014 06:52:30 -0700 (PDT) Received: from emea01-am1-obe.outbound.protection.outlook.com (mail-am1lrp0016.outbound.protection.outlook.com [213.199.154.16]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6584F1B2B1D; Thu, 7 Aug 2014 06:52:29 -0700 (PDT) Received: from AM3PR03MB612.eurprd03.prod.outlook.com (10.242.110.144) by AM3PR03MB610.eurprd03.prod.outlook.com (10.242.109.27) with Microsoft SMTP Server (TLS) id 15.0.995.14; Thu, 7 Aug 2014 13:52:27 +0000 Received: from AM3PR03MB612.eurprd03.prod.outlook.com ([10.242.110.144]) by AM3PR03MB612.eurprd03.prod.outlook.com ([10.242.110.144]) with mapi id 15.00.1005.008; Thu, 7 Aug 2014 13:52:27 +0000 From: Alexander Vainshtein To: "erosen@cisco.com" Thread-Topic: [mpls] [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 Thread-Index: AQHPsY6z5lHYoEzJm0ODJoEdFDIfx5vFBHcg Date: Thu, 7 Aug 2014 13:52:26 +0000 Message-ID: <3be1532ff6644212b98eed535fccce9b@AM3PR03MB612.eurprd03.prod.outlook.com> References: Your message of Tue, 05 Aug 2014 13:59:49 -0000. <9696d0db139d46ffaad7be11340215e8@AM3PR03MB612.eurprd03.prod.outlook.com> <16167.1407340459@erosen-lnx> In-Reply-To: <16167.1407340459@erosen-lnx> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [147.234.56.21] x-microsoft-antispam: BCL:0;PCL:0;RULEID: x-forefront-prvs: 029651C7A1 x-forefront-antispam-report: SFV:NSPM; SFS:(6009001)(37854004)(13464003)(189002)(199002)(377454003)(252514010)(51444003)(76482001)(81342001)(2351001)(74502001)(83322001)(66066001)(4396001)(21056001)(77982001)(74316001)(33646002)(2656002)(87936001)(101416001)(74662001)(106356001)(110136001)(81542001)(86362001)(80022001)(76576001)(106116001)(83072002)(76176999)(46102001)(19580405001)(19580395003)(107046002)(20776003)(64706001)(85306004)(95666004)(99396002)(54356999)(50986999)(105586002)(85852003)(2501001)(24736002)(108616003); DIR:OUT; SFP:; SCL:1; SRVR:AM3PR03MB610; H:AM3PR03MB612.eurprd03.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; LANG:en; Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: ecitele.com Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/XCUiVe5YX-RCOxaa57NMqcb_qCc Cc: "mpls@ietf.org" , pwe3 , "pwe3-chairs@tools.ietf.org" , "stbryant@cisco.com" Subject: Re: [PWE3] [mpls] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Aug 2014 13:52:34 -0000 Eric, Lots of thanks for a very detailed and clear response. It is by far more re= adable (from my point of view) than the original text in the draft. It has = really helped me to understand the intention of the authors and some new po= tential pitfalls with this draft. I agree with most of your comments about fundamental differences between pr= otection against the failure of the Primary PE and protection against the "= egress AC failure". =20 I have some LDP-related questions that stem directly from your responses (I= have added Loa to this thread). You have explained (and the draft also say= s that!) that the "context identifier" IP addresses must be known to both t= he primary PE and the Protector, LDP assigning labels for them and distribu= ting corresponding Label Mappings. 1. I assume that this means that both the Primary PE and the Protector "own= " this IP address, and, as a consequence, include it in the LDP Address mes= sages they send to their respective adjacencies. Is this correct?=20 2. If the answer to the previous question is "Yes", what is supposed to hap= pen if the Primary PE and the Protector have a common adjacent P router? (This would be the case shown in Figure 5 in the draft if P3 were physi= cally connected both to PE2 and PE4). Such a router would see the same IP address in the LDP Address message= s received from two different adjacencies. Can LDP handle such a situation? Is it covered by RFC 5036? 3. If the answer for to my first question is "No", what would cause LDP in = the Primary PE and the Protector to allocate a terminated label (i.e.,=20 a label with action "Pop" in the ILM) to the FEC represented by the co= ntext identifier IP address? And how would the PLR understand that it is th= e penultimate hop towards the Primary PE? =20 I also have some doubts regarding ability to tweak the metrics in the netwo= rk in such a way that, as you've explained: - As long as the Primary PE is up, the PLR sees it as the best Next Hop fo= r the "context identifier" IP address - Once the Primary PE is Down, the PLR sees the Protector as best Next Hop = for the "context identifier" IP address. These doubts are based on the following concerns: 1. Potential impact of ECMP.=20 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Suppose that ECMP is employed in the network, and that, as a consequenc= e, there are multiple penultimate Next Hops=20 On multiple equal cost LSPs between PE1 and the Primary PE2. Is it poss= ible by just playing with the metrics to guarantee that: - As long as the Primary PE2 is Up, each potential PLR would see it as = the best next Hop towards the "context identifier" - Once PE2 is Down, each potential PLR (which would only know that its = own link to the Primary PE has failed ) would treat the Protector as its be= st Next Hop towards the "context identifier"? 2. Potential impact of changes in the network topology.=20 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Suppose that the metrics have been initially tweaked as required. Then = some links between the PLR and the Protector fail so that the cost of the available path betwe= en them increases. Now, when the Primary PE goes Down, the PLR only sees th= at Its link to the Primary PE goes down, but the cost of some alternative = path towards the "context identifier" could now still lead to the Primary P= E.=20 Due to these concerns, my personal bottom line is that operating the propos= ed mechanism over an MPLS network with LSPs instantiated using LDP in DU mo= de is sometimes possible but always tricky. =20 Regards, and,again, lots of thanks, Sasha=20 Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 > -----Original Message----- > From: Eric Rosen [mailto:erosen@cisco.com] > Sent: Wednesday, August 06, 2014 6:54 PM > To: Alexander Vainshtein > Cc: stbryant@cisco.com; mpls@ietf.org; pwe3; pwe3-chairs@tools.ietf.org > Subject: Re: [mpls] [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast= - > protection-01 - RFC4447 >=20 > I've looked at this draft, and it seems to me that it does work in LDP-ba= sed > networks (even with DU mode), and that it does not violate any architectu= ral > principles. However, I have made certain assumptions that aren't explici= tly > stated in the draft. (I am assuming that the draft is intended to be pat= terned > after the "anycast BGP" scheme that I've heard about about, but that is n= ot > written up anywhere as far as I know.) Let me say how I think it is inte= nded > to work, and the authors can correct me if I'm wrong. >=20 > For protection against the failure of the "primary" egress PE: >=20 > - A "primary egress PE" partitions its set of PWs into n sets, where each > set is associated with a given "protector". The "protector" is the bac= kup > egress PE for those PWs. >=20 > - Each such set is associated with an IP address. This IP address must b= e > known both to the primary PE and to the protector. >=20 > * One can think of this address as an "anycast" address that, to routin= g, > denotes both the primary PE and the protector. At the primary PE and > the protector, the address denotes a particular set of PWs. >=20 > * These anycast addresses are distributed by routing, and good old > fashioned MPLS labels get assigned to them by LDP. The primary PE MA= Y > bind implicit null to that address, but the protector MUST NOT bind > implicit null to that address. >=20 > * Routing is set up so that (a) if the primary PE is up, traffic to the > anycast address always goes to the primary PE, but (b) if the primary= PE > is down, traffic to the anycast address goes to the protector instead= . > This can be achieved by messing with the metrics, e.g. >=20 > - For a given PW, the primary PE informs both the protector and the ingre= ss > PE of (a) the label that it (the primary PE) has assigned to that PW, = and > the "anycast address" that corresponds to that PW. >=20 > This requires some new signaling (beyond that specified in RFC 4447), b= ut > the requisite signaling is specified in the draft. >=20 > - When sending a packet on the given PW, the ingress PE pushes on the lab= el > assigned to the PW by the egress PE. Then the ingress PE pushes on > whatever LDP-assigned label it need to use to get the packet to the > anycast address that is associated with the PW. >=20 > - When the protector learns that the egress PE has assigned a given label > and anycast address to the PW, it stores that label in a context-specif= ic > label space associated with that anycast address. Whenever the protect= or > receives a packet whose top label is the label that the protector has > bound to the given anycast address, the protector looks up the next lab= el > in the corresponding label space. >=20 > Let's look at an example: >=20 > CE1---PE1----PLR----PE2----CE2 > | | > |-----PE3-----| >=20 > where PE3 is the protector for a PW connecting CE1 to CE2 (via PE1 and PE= 2 > respectively). Call this pseudowire PW1. >=20 > Thus there is some IP address, call it A, that PE2 and PE3 both advertise= to > routing. The metrics are set up so that as long as PE2 is up, it appears= to be a > better path to A than does PE3. >=20 > So PE2 assigns label L1 to PW1, and uses T-LDP to inform both PE1 and PE3= of > this mapping. To PE1, these are just outgoing labels. To PE3, however, = they > are incoming labels in a context-specific label space. >=20 > PE2 and PE3 both assign labels to A (let's call these labels L2 and L3 > respectively), and distribute the corresponding label bindings to PLR. P= LR > also assigns a label to A, and distributes that label binding (L4) to PE1= . >=20 > When PE1 has a packet to send on PW1, it pushes L1, then pushes L4, then > sends the packet to PLR. PLR swaps L4 with L2 and sends the packet to PE= 2. > (Note that L2 could be implicit NULL.) PE2 sees label L1, and sends the = packet > to CE2. >=20 > Now if PE2 goes down, things happen a bit differently. When PLR gets a > packet with L4 on top, it swaps L4 with L3, and sends the packet to PE3. = (In > this case, of course, L3 cannot be implicit NULL.) >=20 > PE3 gets the packet, sees L3 on top, looks it up, and then says "L3 is a = context > label, so I have to pop it and then look up the following label in the L3= -specific > label table." This context-specific label table contains L1, which has b= een > bound to the PW that leads to CE2. So PE2 pops the label and sends the > packet to CE2. >=20 > I think that's basically it, and the necessary mechanisms do seem to be > described in the draft. The draft doesn't actually use the notion of "an= ycast > address" explicitly, but I think the intention is that the "context label= s" are > definitely bound to anycast addresses. I hope the authors will correct m= e if > I'm wrong. >=20 > Although the above example shows a PLR that is a neighbor of both PE2 and > PE3, this isn't really necessary, one could easily construct an example w= here > PE2 and PE3 have no neighbors in common. >=20 > The example above doesn't rely on LDP DoD at all, and doesn't make use of > any RSVP-TE backup tunnels. >=20 > However, when protecting against the failure of the egress AC, packets > would go first to PE2 then to PE3. This scenario is a bit different, in = that: >=20 > - I think this scenario may require an RSVP-TE backup tunnel (on which PH= P > is not used) from PE2 to PE3. >=20 > - PE2 would have to signal PE3 to associate the given PW with the given > backup tunnel. >=20 > - The label that PE3 assigns to that backup tunnel would identify the > context in which the PW label is looked up. >=20 > This is perhaps more detail than is actually in the draft ;-) but if it i= s an > accurate description of the authors' intentions, the scheme seems to work= . >=20 > I don't think this modifies the MPLS architecture at all. It is true tha= t the PE3 > needs to maintain a context-specific label table that contains labels ass= igned > by PE2. While one may say that that is a "coordinated label space", it i= s no > more than what is required by the nature of "upstream-assigned labels". > I.e., if you can't support this level of coordination, you just can't sup= port > upstream-assigned labels. >=20 > Also, I don't think this draft is an "update" to RFC 4447. You don't nee= d to > read this draft to implement RFC 4447, nor does it modify the procedures = of > RFC 4447. It just introduces some new optional signaling and procedures. > I don't think RFC 4447's requirement to use the "platform label space" > should be interpreted as prohibiting the use of upstream-assigned labels. > The intention of RFC 4447 was to prohibit the use of the interface-specif= ic > label space for PW labels. This prohibition was necessary because the in= gress > PE doesn't know which of the egress PE's network-facing interfaces will > actually receive a given packet. This draft doesn't have that problem, a= s it > provides clear procedures for determining the context in which the > upstream-assigned labels are looked up. >=20 >=20 From nobody Thu Aug 7 06:56:46 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C9DEF1B2B33; Thu, 7 Aug 2014 06:56:43 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.902 X-Spam-Level: X-Spam-Status: No, score=-1.902 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id v5w5T7gUqXHe; Thu, 7 Aug 2014 06:56:41 -0700 (PDT) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1blp0188.outbound.protection.outlook.com [207.46.163.188]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 605661B2AF8; Thu, 7 Aug 2014 06:56:40 -0700 (PDT) Received: from BY2PR05MB728.namprd05.prod.outlook.com (10.141.223.25) by BY2PR05MB726.namprd05.prod.outlook.com (10.141.223.19) with Microsoft SMTP Server (TLS) id 15.0.995.14; Thu, 7 Aug 2014 13:56:37 +0000 Received: from BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) by BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) with mapi id 15.00.0995.014; Thu, 7 Aug 2014 13:56:37 +0000 From: Yimin Shen To: Loa Andersson , "Bocci, Matthew (Matthew)" , "pwe3@ietf.org" , "mpls@ietf.org" Thread-Topic: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 Thread-Index: AQHPqBiOFwGKWlhnR0Cl4HRHJXR0fZvDnKwAgAGhqCA= Date: Thu, 7 Aug 2014 13:56:37 +0000 Message-ID: <3fe4383993b8486da7a6a6bb164d492f@BY2PR05MB728.namprd05.prod.outlook.com> References: <53E226FC.7020302@pi.nu> In-Reply-To: <53E226FC.7020302@pi.nu> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [66.129.241.10] x-microsoft-antispam: BCL:0;PCL:0;RULEID: x-forefront-prvs: 029651C7A1 x-forefront-antispam-report: SFV:NSPM; SFS:(6009001)(13464003)(189002)(24454002)(164054003)(252514010)(377454003)(377424004)(199002)(85306004)(66066001)(19580395003)(33646002)(20776003)(15202345003)(74316001)(76176999)(81342001)(64706001)(54356999)(81542001)(99396002)(50986999)(80022001)(86362001)(2201001)(101416001)(95666004)(106356001)(87936001)(2656002)(105586002)(106116001)(46102001)(77982001)(107046002)(83072002)(76482001)(19580405001)(4396001)(99286002)(74502001)(74662001)(21056001)(107886001)(15975445006)(76576001)(83322001)(85852003)(2501001)(24736002)(108616003); DIR:OUT; SFP:; SCL:1; SRVR:BY2PR05MB726; H:BY2PR05MB728.namprd05.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; LANG:en; Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: juniper.net Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/Boj_0Qm03cnS-psz6qpV8RKuWtw Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Aug 2014 13:56:44 -0000 Hi Loa, Thanks very much for your review and comments. Thanks, /Yimin -----Original Message----- From: pwe3 [mailto:pwe3-bounces@ietf.org] On Behalf Of Loa Andersson Sent: Wednesday, August 06, 2014 9:01 AM To: Bocci, Matthew (Matthew); pwe3@ietf.org; mpls@ietf.org Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 PWE3-chairs and authors, I must confess I find this document hard to read, I had to have some help t= o do so. After going over it carefully I'm convinced that it does not mess with MPLS= Architecture. However I also not convinced that this is ready to go, and I'm not really s= ure what we need to do progress it. In its current form we are probably pus= hing critical review to ADs, IESG and the RFC Editor. Seems to b e the wron= g thing to do. What I think we could do is to invoke the rtg area language review team and= do a careful language review. When that is done (hoping that most of the p= roblems disappear) we should run an MPLS architecture / PWE3 concept check = up and finish of with a short wg last call. /Loa On 2014-07-25 16:56, Bocci, Matthew (Matthew) wrote: > This email begins a second two week working group last call for > http://tools.ietf.org/html/draft-ietf-pwe3-endpoint-fast-protection-01 . > > Please review the draft and post any comments to the PWE3 list. > > If you have read the draft, even if you don=B9t have any comments, but=20 > would like to see the draft progressed, please can you indicate this=20 > to the list. > > This working group last call will end on Friday August 8th, 2014. > > > Best regards > > > Matthew and Andy > > _______________________________________________ > pwe3 mailing list > pwe3@ietf.org > https://www.ietf.org/mailman/listinfo/pwe3 > --=20 Loa Andersson email: loa@mail01.huawei.com Senior MPLS Expert loa@pi.nu Huawei Technologies (consultant) phone: +46 739 81 21 64 _______________________________________________ pwe3 mailing list pwe3@ietf.org https://www.ietf.org/mailman/listinfo/pwe3 From nobody Thu Aug 7 07:37:28 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 27E601B2BC3 for ; Thu, 7 Aug 2014 07:37:26 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.601 X-Spam-Level: X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1XpBMeElgRpQ for ; Thu, 7 Aug 2014 07:37:19 -0700 (PDT) Received: from emea01-am1-obe.outbound.protection.outlook.com (mail-am1lrp0011.outbound.protection.outlook.com [213.199.154.11]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8C7CC1B2B94 for ; Thu, 7 Aug 2014 07:36:42 -0700 (PDT) Received: from AM3PR03MB612.eurprd03.prod.outlook.com (10.242.110.144) by AM3PR03MB610.eurprd03.prod.outlook.com (10.242.109.27) with Microsoft SMTP Server (TLS) id 15.0.995.14; Thu, 7 Aug 2014 14:36:40 +0000 Received: from AM3PR03MB612.eurprd03.prod.outlook.com ([10.242.110.144]) by AM3PR03MB612.eurprd03.prod.outlook.com ([10.242.110.144]) with mapi id 15.00.1005.008; Thu, 7 Aug 2014 14:36:39 +0000 From: Alexander Vainshtein To: Yimin Shen Thread-Topic: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 Thread-Index: AQHPqymWveYGLBru1kCsLuzIRj9tdJu3CWCAgAA8XgCAAPc/IIABBtiAgAg4EiaAAOtlgIAAs2DLgAA1qVCAASlTgIAAxW1Q Date: Thu, 7 Aug 2014 14:36:39 +0000 Message-ID: <09ab595ab24a46a0a4e87ebeb2cd185a@AM3PR03MB612.eurprd03.prod.outlook.com> References: <201407251524.s6PFOPn92012@magenta.juniper.net> <53D79504.4050904@cisco.com> <04f15bedf2be4c7480d3e9ca01bdfd7f@BY2PR05MB728.namprd05.prod.outlook.com> <2597291f7b074922b690e4b06999cf1a@AM3PR03MB612.eurprd03.prod.outlook.com>, <30e13900814f41a681151971cbe9ebef@BY2PR05MB728.namprd05.prod.outlook.com> <1407215578702.64387@ecitele.com>, <2280b8e0d5f94a60badab91a2237181b@BY2PR05MB728.namprd05.prod.outlook.com> <1407304503657.72312@ecitele.com> <046050cb2d254a139b4650b2d7f93075@BY2PR05MB728.namprd05.prod.outlook.com> In-Reply-To: <046050cb2d254a139b4650b2d7f93075@BY2PR05MB728.namprd05.prod.outlook.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [147.234.56.21] x-microsoft-antispam: BCL:0;PCL:0;RULEID: x-forefront-prvs: 029651C7A1 x-forefront-antispam-report: SFV:NSPM; SFS:(45624003)(60444003)(189002)(199002)(164054003)(43784003)(377454003)(252514010)(15202345003)(15975445006)(107046002)(20776003)(19580395003)(19580405001)(85306004)(93886004)(92566001)(64706001)(106116001)(76576001)(80022001)(76176999)(46102001)(83072002)(50986999)(99396002)(54356999)(105586002)(85852003)(95666004)(16236675004)(21056001)(79102001)(77982001)(66066001)(4396001)(74316001)(76482001)(81342001)(19625215002)(74502001)(83322001)(106356001)(101416001)(31966008)(86362001)(81542001)(110136001)(19300405004)(33646002)(87936001)(2656002)(24736002)(108616003); DIR:OUT; SFP:; SCL:1; SRVR:AM3PR03MB610; H:AM3PR03MB612.eurprd03.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; LANG:en; Content-Type: multipart/alternative; boundary="_000_09ab595ab24a46a0a4e87ebeb2cd185aAM3PR03MB612eurprd03pro_" MIME-Version: 1.0 X-OriginatorOrg: ecitele.com Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/eD6MAtNR95TZttzqUD8CTnECQjw Cc: Yakov Rekhter , "pwe3@ietf.org" , "stbryant@cisco.com" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Aug 2014 14:37:26 -0000 --_000_09ab595ab24a46a0a4e87ebeb2cd185aAM3PR03MB612eurprd03pro_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Yimin hi, Please see some comments inline below. Regards, Sasha Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 From: Yimin Shen [mailto:yshen@juniper.net] Sent: Thursday, August 07, 2014 5:34 AM To: Alexander Vainshtein Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Hi Sasha, Thanks again for the comments. Please see the following 4 items in response= to the 4 comments in your email. 1. Today when an egress PE detects an egress AC failure [[Sasha]] Could you please explain how the egress PE detects an egress AC f= ailure? I have always thought that egress AC failure has to be detected fir= st by corresponding CE which would then notify the egress PE using some sui= table "Remote Failure"/BDI/RDI mechanism. Specific examples would really he= lp. it will simply have to drop packets, until the remote PE stops sending traf= fic over the PW. This draft is intended to improve this behavior. By no mea= ns the local repair on egress PE will prevent the AC failure from being con= verted to and handled as a bi-directional AC failure. Note that bi-directio= nal AC failure handling must involve the CE, but the CE may detect the fail= ure at a slower pace, or it may not be able to detect it in a timely manner= , depending on the actual situation. [[Sasha]] Please see above. IMHO and FWIW if the CE cannot detect its "Rece= ive Failure", the PE would not never know about the "Transmit Failure. E.g. there may be other L2 devices between the PE and CE; the AC may itself= be a logical wire across another network; the type and location of the fai= lure may make it easier for the PE to detect it; and so on. In any case, be= fore the AC failure is handled more gracefully (or bi-directionally), the e= gress traffic has been locally repaired by the egress PE. 2. As mentioned before, this draft doesn't have a dependency on a specific = PSN tunnel technology. [[Sasha]] I have provided my comments on this in my response to Eric. From = my point of view some LDP-specific issues still require clarification, but = my bottom line is that operating the draft over an MPLS network that uses L= DP in DU mode for setting up tunnel LSPs may sometimes be possible but woul= d always be tricky. This does not match my understanding of "no dependency = on a specific PSN tunnel technology", but this may be a matter of taste. 3. The draft already talks about scalability when describing the centralize= d protector model vs the co-located protector model. We can improve the tex= t to explicitly mention that the requirement for setting up LSPs on a per <= primary PE, protector> basis rather than per primary PE basis can potential= ly increase the number of LSPs in a network. This is a tradeoff. As mention= ed before, this does not necessarily cause a scaling issue. [[Sasha]] Yes, I have found the corresponding text in the draft. From my p= oint of view this issue is resolved. 4. We will leave this to PWE3 WG to decide whether this is really necessary= .[[Sasha]] I tend to agree with Eric that usage of context-specific labels = in the draft does not mean update to 4447. From my point of view this issue= is resolved as well. Thanks, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Wednesday, August 06, 2014 5:11 AM To: Yimin Shen Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin and all, I would like to summarize the issues that I have with this draft. Digging f= or them in a very long email thread with multiple levels of in-lining is no= t very effective IMO. 1. Protection against unidirectional AC failures looks problematic to= me: a. The draft explicitly differentiates between "ingress" and "egress"= AC failures b. The outcome can be using a different AC for each direction of CE-to= -CE traffic with potentially destructive effect that cannot be mitigated at= the PWE3 level c. IMHO and FWIW a viable alternative could be to employ native AC me= chanisms converting unidirectional AC failures into bi-directional ones. Th= is would eliminate the differentiation between ingress and egress AC failur= es and facilitate using the PW redundancy mechanism for bi-directional prot= ection 2. Dependency upon specific mechanisms for setting up PSN tunnels see= ms to break some common PWE3 architectural principles: a. I am pretty well sure that the mechanisms specified in the draft c= annot be used with the PSN tunnel LSPs set up by LDP in DU mode b. If the authors, refer to LDP in the DoD mode augmented with some CR= -LDP-like mechanisms (e.g., for exclusion of specific links when setting up= the bypass LSP between the PLR and the Protector), this should, at the ver= y least, be discussed with and approved by the MPLS WG 3. Scalability of the proposed solution should be, at least, explicit= ly presented. In particular, the draft assumes that a dedicated LSP would b= e used between PE1 PE2 for each combination of PE2 with a specific Protect= or LSR. 4. The Protector LSR could be either a kind of S-PE (if different fro= m the secondary T-PE) or the secondary T-PE itself. In both cases it should= be capable of handling labels from specific context spaces as if they were= PW labels. This requires (at the very least) an update of RFC 4447 and sho= uld be explicitly presented and discussed. Hopefully this summary will be useful. Regards, Sasha Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 --_000_09ab595ab24a46a0a4e87ebeb2cd185aAM3PR03MB612eurprd03pro_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Yimin hi,

Please see some commen= ts inline below.

 

Regards,

   &nbs= p;   Sasha

Email: Alexander.Vains= htein@ecitele.com

Mobile: 054-9266302

 

From: Yimin Sh= en [mailto:yshen@juniper.net]
Sent: Thursday, August 07, 2014 5:34 AM
To: Alexander Vainshtein
Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Hi Sasha,

 

Thanks again for the c= omments. Please see the following 4 items in response to the 4 comments in = your email.

 

1. Today when an egres= s PE detects an egress AC failure=

 

[[Sasha]] Could you please explain how the egress PE d= etects an egress AC failure? I have always thought that egress AC failure h= as to be detected first by corresponding CE which would then notify the egress PE using some suitable “Remote= Failure”/BDI/RDI mechanism. Specific examples would really help.

 

it will simply have to= drop packets, until the remote PE stops sending traffic over the PW. This = draft is intended to improve this behavior. By no means the local repair on= egress PE will prevent the AC failure from being converted to and handled as a bi-directional AC failure. Note t= hat bi-directional AC failure handling must involve the CE, but the CE may = detect the failure at a slower pace, or it may not be able to detect it in = a timely manner, depending on the actual situation. <= /p>

[[Sasha]] Please= see above. IMHO and FWIW if the CE cannot detect its “Receive Failur= e”, the PE would not never know about the “Transmit Failure.

 

E.g. there may be othe= r L2 devices between the PE and CE; the AC may itself be a logical wire acr= oss another network; the type and location of the failure may make it easie= r for the PE to detect it; and so on. In any case, before the AC failure is handled more gracefully (or bi-direc= tionally), the egress traffic has been locally repaired by the egress PE.

 

2. As mentioned before= , this draft doesn’t have a dependency on a specific PSN tunnel techn= ology.

[[Sasha]] I have= provided my comments on this in my response to Eric. From my point of view= some LDP-specific issues still require clarification, but my bottom line i= s that operating the draft = over an MPLS network that uses LDP in DU mode for setting up tunnel LSPs ma= y sometimes be possible but would always be tricky. This does not ma= tch my understanding of “no dependency on a specific PSN tunnel technology”, but this may be a matter of ta= ste.

 

3. The draft already t= alks about scalability when describing the centralized protector model vs t= he co-located protector model. We can improve the text to explicitly mentio= n that the requirement for setting up LSPs on a per <primary PE, protector> basis rather than per primary = PE basis can potentially increase the number of LSPs in a network. This is = a tradeoff. As mentioned before, this does not necessarily cause a scaling = issue.

[[Sasha]] Yes, I= have found the corresponding text in the draft.  From my point of vie= w this issue is resolved.=

 

4. We will leave this = to PWE3 WG to decide whether this is really necessary.[[Sasha]] I tend to agree with Eric that usage of con= text-specific labels in the draft does not mean update to 4447. From my point of view this issue is resolved as w= ell.

 

Thanks,

 

/Yimin

 

 

From: Alexande= r Vainshtein [mailto:Al= exander.Vainshtein@ecitele.com]
Sent: Wednesday, August 06, 2014 5:11 AM
To: Yimin Shen
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Yimin and all,

I would like to summarize the issues that I have = with this draft. Digging for them in a very long email thread with multiple= levels of in-lining is not very effective IMO.

 

1.       Protection against unid= irectional AC failures looks problematic to me:

a.       The draft explicitly diffe= rentiates between “ingress” and “egress” AC failure= s

b.      The outcome can be using a= different AC for each direction of CE-to-CE traffic with potentially destr= uctive effect that cannot be mitigated at the PWE3 level

c.       IMHO and FWIW a viable alt= ernative could be to employ native AC mechanisms converting unidirectional = AC failures into bi-directional ones. This would eliminate the differentiat= ion between ingress and egress AC failures and facilitate using the PW redundancy mechanism for bi-direction= al protection

2.       Dependency upon specifi= c mechanisms for setting up PSN tunnels seems to break some common PWE3= architectural principles:

a.       I am pretty well sure that= the mechanisms specified in the draft cannot be used with the PSN tunnel L= SPs set up by LDP in DU mode

b.      If the authors, refer to L= DP in the DoD mode augmented with some CR-LDP-like mechanisms (e.g., for exclu= sion of specific links when setting up the bypass LSP between the PLR and t= he Protector), this should, at the very least, be discussed with and approv= ed by the MPLS WG

3.       Scalability of the = proposed solution should be, at least, explicitly presented. In particular,= the draft assumes that a dedicated LSP would be used between PE1  PE2= for each combination of PE2 with a specific Protector LSR.

4.       The Protector LSR could be= either a kind of S-PE (if different from the secondary T-PE) or the second= ary T-PE itself. In both cases it should be capable of handling labels from= specific context spaces as if they were PW labels. This requires (at the very least) an update of RFC 4447 an= d should be explicitly presented and discussed.

Hopefully this summary will be useful.=

 

 

Regards,

       Sasha <= /o:p>

Email: Alexander.Vainshtein@ecitele.com

Mobile: 054-9266302

--_000_09ab595ab24a46a0a4e87ebeb2cd185aAM3PR03MB612eurprd03pro_-- From nobody Thu Aug 7 08:02:38 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 932531B2BF1; Thu, 7 Aug 2014 08:02:15 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.602 X-Spam-Level: X-Spam-Status: No, score=-2.602 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rHLJn6GUxROP; Thu, 7 Aug 2014 08:01:55 -0700 (PDT) Received: from na01-by2-obe.outbound.protection.outlook.com (mail-by2lp0239.outbound.protection.outlook.com [207.46.163.239]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 48AB01B2BE1; Thu, 7 Aug 2014 08:01:54 -0700 (PDT) Received: from BY2PR05MB728.namprd05.prod.outlook.com (10.141.223.25) by BY2PR05MB726.namprd05.prod.outlook.com (10.141.223.19) with Microsoft SMTP Server (TLS) id 15.0.995.14; Thu, 7 Aug 2014 15:01:51 +0000 Received: from BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) by BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) with mapi id 15.00.0995.014; Thu, 7 Aug 2014 15:01:51 +0000 From: Yimin Shen To: "stbryant@cisco.com" , pwe3 , "pwe3-chairs@tools.ietf.org" Thread-Topic: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 Thread-Index: AQHPqzzbzRsIO8QlbkK+kbMBZ9MDf5u4teQwgAkrcYCAA1mgIA== Date: Thu, 7 Aug 2014 15:01:51 +0000 Message-ID: <3fd1f1049aff43c99cbd3ff14e1f6c30@BY2PR05MB728.namprd05.prod.outlook.com> References: <53D7B569.60400@cisco.com> <53E0B85B.8060707@cisco.com> In-Reply-To: <53E0B85B.8060707@cisco.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [66.129.241.10] x-microsoft-antispam: BCL:0;PCL:0;RULEID: x-forefront-prvs: 029651C7A1 x-forefront-antispam-report: SFV:NSPM; SFS:(6009001)(13464003)(189002)(24454002)(51444003)(164054003)(51704005)(37854004)(377454003)(479174003)(199002)(85306004)(66066001)(19580395003)(33646002)(20776003)(15202345003)(74316001)(76176999)(81342001)(64706001)(54356999)(81542001)(99396002)(50986999)(80022001)(92566001)(86362001)(101416001)(95666004)(106356001)(79102001)(87936001)(2656002)(105586002)(106116001)(46102001)(77982001)(107046002)(83072002)(19580405001)(4396001)(76482001)(99286002)(74502001)(31966008)(21056001)(15975445006)(83322001)(76576001)(85852003)(2501001)(24736002)(108616003)(579004)(559001); DIR:OUT; SFP:; SCL:1; SRVR:BY2PR05MB726; H:BY2PR05MB728.namprd05.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; LANG:en; Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-OriginatorOrg: juniper.net Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/5GUWmon22Ttj0Y-6nPBKgYuGOME Cc: "mpls@ietf.org" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Aug 2014 15:02:15 -0000 SGkgU3Rld2FydCwNCg0KVGhhbmtzIHZlcnkgbXVjaCBhZ2FpbiBmb3IgdGhlIHJldmlldyBhbmQg ZGV0YWlsZWQgY29tbWVudHMuDQoNCldlIHdpbGwgY29uc2lkZXIgeW91ciBzdWdnZXN0aW9ucyBv biB0aGUgTERQIFRMViBzdHJ1Y3R1cmVzIGRlZmluZWQgaW4gdGhpcyBkcmFmdC4NCg0KUmVnYXJk aW5nIHRoZSB0cmFkZW9mZiBiZXR3ZWVuIHRoZSBuZWVkIGZvciBlZ3Jlc3MgUEUgbm9kZSBwcm90 ZWN0aW9uIGFuZCB0aGUgaW5jcmVhc2VkIHN0YXRlIGFuZCBjb21wbGV4aXR5IGluIFBTTiwgdGhp cyBpcyBhIGNvbW1vbiBjb25zaWRyYXRpb24gZm9yIGFsbCBJUC9NUExTIG5vZGUgcHJvdGVjdGlv biwgbm90IFBXRTMgc3BlY2lmaWMuIFdlIGRvIHNlZSB0aGUgcmVxdWlyZW1lbnQgZnJvbSBzZXJ2 aWNlIHByb3ZpZGVycy4gSSB0aGluayB0aGlzIHNob3VsZCBiZSB0aGVpciBkZWNpc2lvbiBvbiBh IHBlciBuZXR3b3JrIGJhc2lzLiBXaGVuIHRoZXkgZGVjaWRlIHRvIGRvIGl0LCB3ZSBoYXZlIGEg c29sdXRpb24gYXZhaWxhYmxlIGZvciB0aGVtLg0KDQpUaGF0IHNhaWQsIHRoaXMgZHJhZnQgZG9l c24ndCBuZWNlc3NhcmlseSBtdWx0aXBseSB0aGUgbnVtYmVyIG9mIFBTTiB0dW5uZWxzLiBJdCBk b2VzIHJlcXVpcmUgb25lIHR1bm5lbCBwZXIgPHByaW1hcnkgUEUsIHByb3RlY3Rvcj4uIEhvd2V2 ZXIsIGlmIGVhY2ggcHJpbWFyeSBQRSBpcyBwcm90ZWN0ZWQgYnkgb25seSAxIHByb3RlY3Rvciwg dGhlIG51bWJlciBvZiB0dW5uZWwgd2lsbCBiZSB0aGUgbnVtYmVyIG9mIHByaW1hcnkgUEVzLCB3 aGljaCBpcyBiYXNpY2FsbHkgdGhlIHNhbWUgYXMgdGhlIGNhc2Ugd2hlcmUgdGhpcyBtZWNoYW5p c20gaXMgbm90IHVzZWQuIFRoaXMgMToxIHJlbGF0aW9uc2hpcCBpcyBhY2hpZXZhYmxlIHdpdGgg dGhlIGNvLWxvY2F0ZWQgbW9kZWwsIGFuZCBpcyBlc3BlY2lhbGx5IGFjaGlldmFibGUgd2l0aCB0 aGUgY2VudHJhbGl6ZWQgbW9kZWwuIE9mIGNvdXJzZSwgaW4gYSBtb3JlIGdlbmVyYWwgY2FzZSwg aWYgdGhlIGF2YXJhZ2UgUEUtdG8tcHJvdGVjdG9yIHJhdGlvIGlzIDE6TiwgdGhlIG51bWJlciBv ZiB0dW5uZWxzIHdpbGwgYmUgbXVsdGlwbGllZCBieSBOLiBCdXQgb3BlcmF0b3JzIGNhbiBhbHdh eXMgZGVzaWduIHRoZWlyIG5ldHdvcmtzIHRob3VnaHRmdWxseSBhbmQgbWFuYWdlIHRoaXMgTiB0 byBiZSBsb3dldmVyIHRoYW4gdGhlIHRocmVzaG9sZCB0aGF0IG1heSBsZWFkIHRvIHNjYWxhYmls aXR5IHByb2JsZW1zLiAoSGVyZSwgSSdtIG5vdCBjb3VudGluZyBieXBhc3MgdHVubmVscyBpbiwg YmVjYXVzZSBieXBhc3MgdHVubmVscyBleGlzdCBpbiB0aGUgZXhpc3RpbmcgSVAvTVBMUyBmYXN0 LXJlcm91dGUgYXMgd2VsbC4pDQoNClRoYW5rcywNCg0KL1lpbWluDQoNCg0KDQotLS0tLU9yaWdp bmFsIE1lc3NhZ2UtLS0tLQ0KRnJvbTogU3Rld2FydCBCcnlhbnQgW21haWx0bzpzdGJyeWFudEBj aXNjby5jb21dIA0KU2VudDogVHVlc2RheSwgQXVndXN0IDA1LCAyMDE0IDY6NTYgQU0NClRvOiBZ aW1pbiBTaGVuOyBwd2UzOyBwd2UzLWNoYWlyc0B0b29scy5pZXRmLm9yZw0KQ2M6IG1wbHNAaWV0 Zi5vcmcNClN1YmplY3Q6IFJlOiBbUFdFM10gV0cgTGFzdCBDYWxsIGZvciBkcmFmdC1pZXRmLXB3 ZTMtZW5kcG9pbnQtZmFzdC1wcm90ZWN0aW9uLTAxDQoNCk9uIDMwLzA3LzIwMTQgMjA6MDcsIFlp bWluIFNoZW4gd3JvdGU6DQo+IEhpIFN0ZXdhcnQsDQo+DQo+IFRoYW5rcyB2ZXJ5IG11Y2ggZm9y IHRoZSBkZXRhaWxlZCByZXZpZXcgYW5kIGNvbW1lbnRzLiBJJ2QgbGlrZSB0byBoaWdobGlnaHQg YSBmZXcgdGhpbmdzIGJlbG93IGFib3V0IHRoaXMgZHJhZnQsIGFuZCB0aGVuIHJlc3BvbmQgdG8g eW91ciBjb21tZW50cyBpbmxpbmUuIEhvcGUgeW91IGNvdWxkIGdpdmUgdGhlIGRyYWZ0IGFub3Ro ZXIgY2xvc2VyIGxvb2suDQpZaW1pbiwNCg0KSGF2aW5nIHJlYWQgeW91ciByZXBseSBhbmQgaGF2 aW5nIGxvb2tlZCBhdCB0aGUgdGV4dCBhZ2FpbiwNCmFuZCB0YWxrZWQgdG8gYSBmZXcgZm9sa3Mg SSB0aGluayBJIG5vdyB1bmRlcnN0YW5kIHRoZSB0b3ANCmxldmVsIHZpZXcgb2Ygd2hhdCBpcyBw cm9wb3NlZC4gSG93ZXZlciB0aGF0IGlzIGluIGl0c2VsZg0KYW4gaW5kaWNhdGlvbiB0aGF0IHRo ZSBkcmFmdCBuZWVkcyBjb25zaWRlcmFibGUgd29yaw0KYmVmb3JlIHB1YmxpY2F0aW9uLiBOb3Rl IHRoYXQgaXQgd2FzIG5vdCBpbml0aWFsbHkgb2J2aW91cw0KdGhhdCB5b3Ugc3VwcG9ydGVkIGJv dGggTERQIGFuZCBSU1ZQLVRFIFBTTnMsIG5vcg0Kd2FzIGl0IGluaXRpYWxseSBvYnZpb3VzIHRo YXQgeW91IHdlcmUgdXNpbmcgdGhlIFBTTiB0bw0KcHJvdGVjdCB0aGUgUFdzIGFuZCBub3QgbG9v a2luZyBmdXJ0aGVyIGRvd24gdGhlIHN0YWNrLg0KSW5kZWVkIEkgdGhpbmsgdGhlcmUgaXMgc29t ZSB0ZXh0IGluIHRoZSBkcmFmdCB0aGF0IHBvc2l0aXZlbHkNCmxlYWRzIHlvdSBpbiB0aGF0IGRp cmVjdGlvbiBzaW5jZSBzb21lb25lIGVsc2UgcGlja2VkDQppdCBpdCBkdXJpbmcgYW4gTVBMUyBy ZXZpZXcuDQoNClRvIHN1bW1hcml6ZTogd2hhdCB5b3UgYXJlIGRvaW5nIGlzIGNvbnN0cnVjdCAg cmVwYWlycw0KYXQgdGhlIFBTTiBsYXllciB0aGF0IG1hcCB0byBlYWNoIGNvbWJpbmF0aW9uIG9m IFBXICsgUFcgcmVwYWlyLA0Kd2hpY2ggSSB0aGluayBtdWx0aXBsZXMgdGhlIG51bWJlciBvZiBQ U04gdHVubmVscyBieSB0aHJlZSB0aW1lcw0KdGhlIGRpc3BlcnNpb24gZmFjdG9yIHdoZXJlIHRo ZSBkaXNwZXJzaW9uIGZhY3RvciBpcyBzb21lIGZhY3Rvcg0KZGVmaW5pbmcgdGhlIGF2ZXJhZ2Ug bnVtYmVyIG9mIFBFcyB0aGF0IHRoZSBQV3MgYWRkcmVzc2VkIHRvDQphIHNpbmdsZSBQRSBuZWVk IHRvIGJlIGRpc3BlcnNlZCB0by4NCg0KVGhpcyBsZWFkcyB0byB0d28gcXVlc3Rpb25zIHRoYXQg dGhlIFdHIG5lZWRzIHRvIHRoaW5rIGFib3V0Og0KDQoxKSBUaGlzIGlzIHRyYWRpbmcgaW5jcmVh c2VkIFBTTiBzdGF0ZSBmb3IgcmVzcG9uc2UgdGltZS4NCkdpdmVuIHRoYXQgUFNOIHN0YXRlIHdh cyBhIGNyaXRpY2FsIGZhY3RvciBpbiB0aGUgY2hvaWNlIG9mIHRoZSBNYXJ0aW5pDQpkZXNpZ24g b3ZlciB0aGUgQ0NDIGRlc2lnbiB0aGlzIG5lZWRzIHRvIGJlIG1hZGUgY2xlYXIgdXAgZnJvbnQN CmluIHRoZSB0ZXh0IGFuZCB0aGUgV0cgbmVlZHMgdG8gZGVjaWRlIHRoYXQgdGhpcyBpcyBhY2Nl cHRhYmxlLg0KDQoyKSBUaGlzIGluY3JlYXNlZCBzdGF0ZSBhbmQgdGhlIGNvbXBsZXhpdHkgb2Yg aW50cm9kdWNpbmcNCmNvbnRleHQgbGFiZWxzIGFuZCBwcm90ZWN0aW9uIHJvdXRlcnMgdG8gdGhl IFBXRTMgYXJjaGl0ZWN0dXJlDQppcyBkaXJlY3RseSB0cmFjZWFibGUgdG8gdGhlIG5lZWQgdG8g ZG8geFBFIG5vZGUgcHJvdGVjdGlvbiwNCmFuZCBpcyBub3QgcmVxdWlyZWQgZm9yIGxpbmsgcHJv dGVjdGlvbiBpbmNsdWRpbmcgdGhlIGVncmVzcyBBQy4NCg0KVGhpcyBkZXNpcmUgdG8gYnVpbGQg dGhlIGRlc2lnbiBhcm91bmQgbm9kZSBwcm90ZWN0aW9uICh3aGljaA0KYWx3YXlzIGFkZHMgc2ln bmlmaWNhbnQgY29tcGxleGl0eSB0byBGUlIpIGlzIGluIGNvbnRyYXN0IHRvDQp0aGUgSVBGUlIg c2l0dWF0aW9uIHdoZXJlIGxpbmsgcHJvdGVjdGlvbiBpcyBvZiBvdmVyd2hlbG1pbmcNCmltcG9y dGFuY2UgYW5kIG5vZGUgcHJvdGVjdGlvbiBzZWVtcyB0byBiZSBvZiBzZWNvbmRhcnkNCmltcG9y dGFuY2UuDQoNCkFuIGFsdGVybmF0aXZlIGRlc2lnbiwgd2hpY2ggd291bGQgcmVxdWlyZSBsZXNz IGNoYW5nZSB0byB0aGUNCmV4aXN0aW5nIFBXRTMgZGVzaWduIHdvdWxkIGJlIHRvIHByb3ZpZGUg cmVkdW5kYW50IGNvbm5lY3Rpb24NCnRvIHRoZSB4UEVzIGFuZCBhbGxvdyBGUlIgbGluayBwcm90 ZWN0aW9uIGluIHRoZSBQU04gdG8NCmRlYWwgd2l0aCBsaW5rIGZhaWx1cmVzIGFuZCB0aGVuIHRv IHByb3RlY3QgdGhlIGVncmVzcyBBQyBieQ0KbWFraW5nIHRoZSBULVBFIGEgc3BlY2lhbCB0eXBl IG9mIFMtUEUgdGhhdCBwcmVmZXJyZWQgdG8NCmFjdCBhcyBhIFQtUEUgYnV0IGlmIHRoZSBBQyB3 YXMgZG93biBmb3J3YXJkZWQgdG8gYW4gYWx0ZXJuYXRlDQpULVBFIGZvciB0aGF0IFBXLg0KDQpJ ZiB0aGUgV0cgcHJlZmVycyB0byBpbnRyb2R1Y2UgdGhlIGNyb3NzIGxheWVyIHNjaGVtZSB5b3Ug aGF2ZQ0KcHJvcG9zZWQsIHRoZW4gZmluZSwgd2UgbmVlZCB0byBkbyB0aGUgZnVsbCBzZXQgb2Yg dXBkYXRlcyBuZWVkZWQuDQpIb3dldmVyIGZpcnN0IEkgdGhpbmsgdGhhdCB3ZSBuZWVkIHRvIGhh dmUgcm91Z2ggY29uc2Vuc3VzIHRoYXQNCnRoZSB1c2UgY2FzZXMgYW5kIHJlcXVpcmVtZW50cyBq dXN0aWZ5IHRoZSBpbmNyZWFzZWQgY29tcGxleGl0eQ0KY29tcGFyZWQgb2Ygbm9kZSBwcm90ZWN0 aW9uIGNvbXBhcmVkIHRvIHRoZSBhbHRlcm5hdGl2ZSBvZg0KcHJvdmlkaW5nIGxpbmsgYW5kIEFD IHByb3RlY3Rpb24uDQo+DQo+IFsxXSBUaGlzIGRyYWZ0IGlzIGNvbXBsZXRlbHkgYmFzZWQgb24g dGhlIFBXRTMgYW5kIHRoZSBNUy1QVyBhcmNoaXRlY3R1cmUuIEl0IGlzIGFsc28gYmFzZWQgb24g UkZDIDUzMzEgIiBNUExTIFVwc3RyZWFtIExhYmVsIEFzc2lnbm1lbnQgYW5kIENvbnRleHQtU3Bl Y2lmaWMgTGFiZWwgU3BhY2UiLiBTbyBpZGVhbGx5IHJlYWRlcnMgc2hvdWxkIGJlIGZhbWlsaWFy IHdpdGggdGhhdCBSRkMuDQpBcyBmYXIgYXMgSSBjYW4gc2VlIGNvbnRleHQgbGFiZWxzIGhhdmUg bm90IGJlZW4gaW50cm9kdWNlZA0KaW50byB0aGUgUFdFMyBhcmNoaXRlY3R1cmUuIFRoZXJlIGlz IHNvbWUgcmVmZXJlbmNlIHRvIHRoZWlyIHVzZSBmb3INClAyTVAgUFdzLCBidXQgbm90IGluIHRo ZSBQMlAgY2FzZS4gSW5kZWVkIEkgdGhpbmsgdGhhdCBSRkM0NDQ3DQpub3RlcyBleHBsaWNpdGx5 IHRoYXQgaW4gdGhlIGNhc2Ugd2hlcmUgdGhlIGNvbmZpZ3VyYXRpb24gb2YgUFdzDQppcyBzaWdu YWxlZCBieSBMRFAgdGhlIHBsYXRmb3JtIGxhYmVsIHNwYWNlIG11c3QgYmUgdXNlZC4gVGhlDQps ZWFzdCB0aGF0IHlvdSBuZWVkIHRvIGRvIGlzIHRvIHVwZGF0ZSBSRkM0NDQ3Lg0KDQpTb21ldGhp bmcgdGhhdCBJIG5vdGljZWQgaW4gYSBzZWNvbmQgcGFzcyB0aHJvdWdoIHRoZSB0ZXh0IGlzIHRo YXQNCnlvdSBzZWVtIHRvIGhhdmUgY2hhbmdlZCB0aGUgUFcgRkVDIGRhdGEgc3RydWN0dXJlIGZy b20gdGhlDQpvbmUgc3BlY2lmaWVkIGJ5IFBXRTMgdG8gYSB2YXJpYW50IG9mIG9uZSB1c2VkIGlu IExTUCBQaW5nLiBUaGlzDQpzZWVtcyB1bm5lY2Vzc2FyeSBhbmQgbWF5IGxlYWQgdG8gY29uZnVz aW9uIGFuZCBkaWZmaWN1bHR5IGlmDQp0aGVzZSBnZXQgY2hhbmdlZCBhdCBzb21lIHRpbWUgaW4g dGhlIGZ1dHVyZS4gQSBiZXR0ZXIgZGVzaWduDQppcyBzdXJlbHkgdG8gbGVhdmUgdGhlIFBXRTMg c2lnbmFsbGluZyBzeXN0ZW0gY29tcGxldGVseSB1bmNoYW5nZWQNCmV4Y2VwdCB0byBhZGQgYW4g YWRkaXRpb25hbCBUTFYgc3BlY2lmeWluZyB0aGUgY29udGV4dCBpbmZvcm1hdGlvbg0KaWYgaW5k ZWVkIHdlIGNvbnRpbnVlIG9uIHRoYXQgcGF0aC4gQXByb3BvcyB0aGlzLCBpdCBtYXkgYmUgdGhl DQp3YXkgdGhhdCB0aGUgZG9jdW1lbnQgd2FzIHdyaXR0ZW4sIGJ1dCBJIGRpZCBub3Qgc2VlIGFu eXRoaW5nIGFib3V0DQp0aGUgbmVlZCB0byBzaWduYWwgdGhlIFBXIGludGVyZmFjZSBwYXJhbWV0 ZXJzIHRvIHRoZSBhbHRlcm5hdGl2ZQ0KUEUuIEFsc28gSSBkaWQgbm90IHNlZSBhbnl0aGluZyBz cGVjaWZ5aW5nIHRoZSBlcnJvciBoYW5kbGluZw0KYXMgYSByZXN1bHQgb2YgcGFyYW1ldGVyIGlz c3Vlcy4NCg0KQSBjb21tZW50IGJlbG93IHRoYXQgd2FzIG5vdCBwaWNrZWQgdXAgaXMgdGhhdCB0 aGUgc2lnbmFsbGluZw0KZGVzaWduIHN1cHBvcnRzIElQdjQgb25seSwgYW5kIGFzIEkgcmVjYWxs IHRoZXJlIGlzIGFuIElFU0cgcG9saWN5DQpvZiBub3Qgc3RhbmRhcmRpemluZyBhbnkgSVB2NCBv bmx5IHNvbHV0aW9ucywgc28gcGVyaGFwcyB3ZSBzaG91bGQNCmludHJvZHVjZSBJUHY2IGF0IHRo ZSBzYW1lIHRpbWUgYXMgaGFybW9uaXppbmcgdGhlIG5vcm1hbCBhbmQNCmNvbnRleHQgbGFiZWwg ZGF0YSBzdHJ1Y3R1cmVzLg0KDQpUaGUgTERQIHNpZ25hbGxpbmcgb2YgUFdzIHdpdGggY29udGV4 dCBsYWJlbHMgbmVlZCBhIG1vcmUNCmNvbXBsZXRlIHJldmlldyBmb3IgYm90aCBjb21wbGV0ZW5l c3MgYW5kIGNvcnJlY3RuZXNzIGFuZA0KdG8gZW5zdXJlIHRoYXQgdGhpcyB0ZXh0IHByb3ZpZGVz IGEgZGVzY3JpcHRpb24gdGhhdCBpcyBzdWZmaWNpZW50bHkNCmNvbXBsZXRlIHRoYXQgdGhpcyBu ZXcgZmVhdHVyZSBjYW4gYmUgaW1wbGVtZW50ZWQgc29sZWx5IGZyb20NCnRoaXMgdGV4dCBhbmQg dGhlIGFzc29jaWF0ZWQgcmVmZXJlbmNlcy4NCj4NCj4gWzJdIFRoZSBkcmFmdCBtZW50aW9ucyB0 d28gaW1wb3J0YW50IHJvbGVzIG9mIHJvdXRlcnMsIFBMUiAocG9pbnQgb2YgbG9jYWwgcmVwYWly KSBhbmQgcHJvdGVjdG9yLg0KWW91IG5lZWQgYSBjbGVhcmVyIGRlZmluaXRpb24gb2YgYSBQTFIg YW5kIHByb3RlY3Rvci4gSSB0aGluayB0aGF0DQphIHByb3RlY3RvciBpcyBhIG5ldyB0eXBlIG9m IFNQRSBhbmQgbmVlZHMgYSBjbGVhcmVyIGFyY2hpdGVjdHVyYWwNCmRlZmluaXRpb24uDQoNClRo ZSBjb25jZXB0IG9mIGEgUExSIHRha2luZyBhY3Rpb24gYXQgb25lIGxheWVyIHdpdGhpbiB0aGUg Y29udGV4dA0Kb2YgYW5vdGhlciBpcyBuZXcgaW4gUFdFMy4gUGVyaGFwcyBpdCBpcyBkZWZpbmVk IGluIE1QTFMgaW4gd2hpY2gNCmNhc2UgYSByZWZlcmVuY2Ugd291bGQgYmUgYXBwcm9wcmlhdGUu DQo+DQo+IFszXSBQTFIgcGVyZm9ybXMgbG9jYWwgcmVwYWlyIGluIGRhdGFwbGFuZSwgYmFzZWQg b24gcHJlLWluc3RhbGxlZCBiYWNrdXAgZm9yd2FyZGluZyBzdGF0ZSBwb2ludGluZyB0byBhIGJ5 cGFzcyB0dW5uZWwuIFRoZXJlIGlzIG5vIGNvbnRyb2wtcGxhbmUgaW52b2x2ZW1lbnQgaW4gbG9j YWwgcmVwYWlyLg0KU3BlY2lmaWNhbGx5IGluIHRoZSBpbml0aWF0aW9uIG9mIHRoZSBsb2NhbCBy ZXBhaXIsIGl0IGlzIGluIHRoZSANCmNvbmZpZ3VyYXRpb24NCm9mIGNvdXJzZS4NCj4gVGhpcyBp cyBzaW1pbGFyIHRvIHRoZSBleGlzdGluZyBJUC9NUExTIGZhc3QtcmVyb3V0ZSBtZWNoYW5pc21z LiBUaGVyZWZvcmUsIGl0IGNhbiBhY2hpZXZlIHJlc3RvcmUgdHJhZmZpYyB3aXRoIGEgc2ltaWxh ciBwZXJmb3JtYW5jZSB0byB0aG9zZSBtZWNoYW5pc21zLiBUaGF0IGlzIHRoZSBtYWluIGdvYWwg b2YgdGhpcyBkcmFmdC4NCklQL01QTFMgRlJSIG5vcm1hbGx5IG9wZXJhdGVzIHNvbGVseSBmb3Ig dGhlIGJlbmVmaXQgb2YgaXRzIG93bg0KZGF0YXBsYW5lLiBZb3UgaGF2ZSB0aGlzIGNvbmZpZ3Vy ZWQgdG8gYWN0IGZvciB0aGUgYmVuZWZpdCBvZiB0aGUNCmNsaWVudCBkYXRhcGxhbmUuDQo+DQo+ IFs0XSBXaGVuIGRvaW5nIGxvY2FsIHJlcGFpciwgdGhlIFBMUiBvbmx5IHN3YXBzIHRoZSB0b3Ag bGFiZWwgKGkuZS4gdHVubmVsIGxhYmVsKSB0byBhIGJ5cGFzcyB0dW5uZWwncyBsYWJlbCwgYW5k IGtlZXBzIHRoZSBQVyBsYWJlbCAoYWxsb2NhdGVkIGJ5IHByaW1hcnkvcHJvdGVjdGVkIFBFKSBp bnRhY3QuIFRoZSBQTFIgZG9lc24ndCBuZWVkIHRvIGtub3cgb3IgdW5kZXJzdGFuZCB0aGUgUFcg bGFiZWwuDQpJbmRlZWQgdGhpcyBpcyBjbGVhcmVyIG5vdywgYW5kIG5lZWRzIHRvIGJlIG1hZGUg bXVjaCBjbGVhcmVyIGluDQp0aGUgdGV4dC4NCg0KSG93ZXZlciBzbyBkbyB0aGUgaW1wbGljYXRp b25zIGZvciB0aGUgc3lzdGVtIGFzIGEgd2hvbGUgb2YgdGhpcw0KbW9kZSBvZiBvcGVyYXRpb24u DQo+DQo+IFs1XSBUaGUgcHJvdGVjdG9yIGlzIHRoZSByb3V0ZXIgYXQgdGhlIG90aGVyIGVuZCBv ZiB0aGUgYnlwYXNzIHR1bm5lbC4gSXQgaXMgcmVzcG9uc2libGUgZm9yIGZvcndhcmRpbmcgdGhl IHRyYWZmaWMgZnVydGhlciwgc28gdGhhdCB0aGUgdHJhZmZpYyBjYW4gZXZlbnR1YWxseSByZWFj aCB0aGUgdGFyZ2V0IENFLiBUaGUgcHJvdGVjdG9yIG11c3QgdW5kZXJzdGFuZCB0aGUgUFcgbGFi ZWwgYWxsb2NhdGVkIGJ5IHRoZSBwcmltYXJ5IFBFLg0KVGhlIHVzZSBvZiB0aGUgd29yayBwcmlt YXJ5IFBFIGlzIHNsaWdodGx5IGNvbmZ1c2luZy4gUHJlc3VtYWJseQ0KeW91IG1lYW4gdGhlIGlu Z3Jlc3MgUEUgb2YgdGhlIHNlZ21lbnQgcmF0aGVyIHRoYW4gdGhlIGluZ3Jlc3MNClBFIGZyb20g dGhlIHBvaW50IG9mIHZpZXcgb2YgdGhlIGluZ3Jlc3MgQUMuDQo+IFRoaXMgaXMgYWNoaWV2ZWQg YnkgdXNpbmcgdGhlIGNvbnRleHQgbGFiZWwgZm9yd2FyZGluZyBwYXJhZGlnbSBzcGVjaWZpZWQg aW4gUkZDIDUzMzEuIFRoaXMgZHJhZnQgZGVmaW5lcyBzb21lIExEUCBleHRlbnNpb25zIHRvIGFs bG93IHRoZSBwcmltYXJ5IFBFIHRvIGFkdmVydGlzZSBQVyBsYWJlbHMgdG8gdGhlIHByb3RlY3Rv ci4gVGhlIHByb3RlY3RvciBsb2NhbGx5IG1haW50YWlucyBhIGxhYmVsIHNwYWNlIGZvciB0aGVz ZSBQVyBsYWJlbHMgb2YgdGhlIHByb3RlY3RlZCBQRSwgYW5kIGxvb2tzIHVwIHRoZSBQVyBsYWJl bCBpbiB0aGlzIGxhYmVsIHNwYWNlLiBUaGUgYnlwYXNzIHR1bm5lbCBzZXJ2ZXMgYXMgdGhlIHBv aW50ZXIgdG8gdGhpcyBsYWJlbCBzcGFjZS4NCk5vdyBJIHVuZGVyc3RhbmQgdGhhdC4gUGxlYXNl IG1ha2UgdGhpcyBtdWNoIGNsZWFyZXINCmVhcmxpZXIgaW4gdGhlIHRleHQuDQoNCk5vdGUgbXkg ZWFybGllciBjb25jZXJuIGFib3V0IHRoZSBkZXNpZ24gb2YgdGhvc2UgZXh0ZW5zaW9ucy4NCj4N Cj4gWzZdIFRoaXMgZHJhZnQgYWxzbyBpbnRyb2R1Y2VzIHRoZSBjb25jZXB0IG9mICJjb250ZXh0 IElEIi4gQSBjb250ZXh0IElEIGlzIGFuIElQIGFkZHJlc3MgaWRlbnRpZnlpbmcgYSBwYWlyIG9m IDxwcmltYXJ5IFBFLCBwcm90ZWN0b3IgWD4uDQpUaGUgdXNlIG9mIGFuIElQIGFkZHJlc3MgcmF0 aGVyIHRoYW4gc29tZSBpZGVudGlmaWVyIHRha2VuIGZyb20NCnRoZSBQVyBjb250ZXh0IGlzIGEg ZGVzaWduIGFzc3VtcHRpb24gbm90IHByb3Blcmx5IHNoYXJlZCB3aXRoIHRoZQ0KcmVhZGVyIGFu ZCBuZWVkcyB0byBiZSBjbGFyaWZpZWQgaW4gdGhlIHRleHQuDQoNCj4gSXQgc2VydmVzIGFzIHRo ZSBkZXN0aW5hdGlvbiBvZiB0cmFuc3BvcnQgdHVubmVsLiBQV3MgY2FycmllZCBieSB0aGUgdHVu bmVsIHdpbGwgYmUgcmVkaXJlY3RlZCB0byB0aGUgcHJvdGVjdG9yIFggZHVyaW5nIGxvY2FsIHJl cGFpci4gUFdzIHRvIHRoZSBzYW1lIHByaW1hcnkgUEUgYnV0IHByb3RlY3RlZCBieSBhIGRpZmZl cmVudCBwcm90ZWN0b3IgWSBjYW5ub3QgYmUgY2FycmllZCBieSB0aGlzIHR1bm5lbC4gUmF0aGVy LCB0aGV5IG11c3QgYmUgY2FycmllZCBieSBhIHR1bm5lbCBkZXN0aW5lZCBmb3IgY29udGV4dCBJ RCBvZiA8cHJpbWFyeSBQRSwgcHJvdGVjdG9yIFk+Lg0KSW5kZWVkLCBidXQgdGhhdCBuZWVkcyB0 byBiZSBtYWRlIGNsZWFyZXIsIHBhcnRpY3VsYXJseSBhcyB0aGlzDQpzZWVtaW5nbHkgaGFzIHNj YWxpbmcgY29uc2VxdWVuY2VzLg0KPg0KPiBbN10gQSBQTFIgbWF5IGJlIGFueSByb3V0ZXIgaW4g dGhlIG5ldHdvcmssIGluY2x1ZGluZyBQIG9yIFBFIHJvdXRlci4gVGhlIGRyYWZ0IGFzc3VtZXMg dGhhdCBpdCBkb2Vzbid0IGtub3cgb3IgY2FyZSBhYm91dCBQVyBpbmZvLg0KPg0KPiBbOF0gQSBw cm90ZWN0b3IgaXMgYSBQRSByb3V0ZXINCldpdGhpbiBhIFBFIGNvbnRleHQgSSBhbSBub3Qgc3Vy ZSB0aGF0IHRoZSBwcm9wZXJ0aWVzIG9mIGEgUEUgcm91dGVyDQphcmUgZGVmaW5lZC4NCj4gaW4g dGhlICJjby1sb2NhdGVkIiBtb2RlbC4gSXQgbWF5IGJlIGEgUCByb3V0ZXIgaW4gdGhlICJjZW50 cmFsaXplZCIgbW9kZWwuIEZvciB0aGUgbGF0ZXIgY2FzZSwgdGhlIGRyYWZ0IGRlc2NyaWJlcyBo b3cgdG8gdXNlIHRoZSBuZXcgTERQIGV4dGVuc2lvbnMgZm9yIHRoZSBwcm90ZWN0b3IgdG8gbGVh cm4gYW5kIGFzc29pY2F0ZSBwcmltYXJ5IFBXIGFuZCBiYWNrdXAgUFcgYW5kIGluc3RhbGwgZm9y d2FyZGluZyBlbnRyaWVzIGluIHRoZSBjb250ZXh0IGxhYmVsIHRhYmxlLg0KU28gd2hlbiB5b3Ug cnVuIHRoZSBjZW50cmFsaXplZCBwcm90ZWN0b3IsIGhvdyBkb2VzIHRoZQ0KUFcgcGFyYW1ldGVy IHNpZ25hbGluZyBhbmQgaW50ZXJmYWNlIHBhcmFtZXRlciB2ZXJpZmljYXRpb24NCndvcms/IFN1 cmVseSB0aGUgY2VudHJhbGl6ZWQgcHJvdGVjdG9yIG5lZWRzIHRvIGJlIHNvbWUNCmZvcm0gb2Yg UEUgcmF0aGVyIHRoYW4gYSB2YXJpYW50IG9mIGFuIE1QTFMgUCByb3V0ZXI/DQoNCkkgaGF2ZSBh IGJ1bmNoIG9mIGNvbW1lbnRzIG9uIHlvdXIgYW5zd2VyIGJlbG93LCBwdXQgcGVyaGFwcyB3ZQ0K Y2FuIHN0YXJ0IGJ5IHdvcmtpbmcgdGhyb3VnaCB0aGUgY29tbWVudHMgYWJvdmUgYW5kIHRoZSBj eWNsZQ0KYmFjayB0byB0aGUgY29tbWVudHMgYmVsb3cgd2l0aGluIHRoZSByZXN1bHRhbnQgY29u dGV4dC4NCg0KSSB3aWxsIGNvbW1lbnQgb24gb3VyIGV4Y2hhbmdlIGJlbG93IHdoZW4gd2UgaGF2 ZSByZWFjaGVkDQpzb21lIGxldmVsIG9mIGNvbnNlbnN1cyBvbiB0aGUgcHJlYW1ibGUuDQoNCkdp dmVuIHRoZSBjb25jZXJucyB0aGF0IEkgaGF2ZSByYWlzZWQgYW5kIHRob3NlIHJhaXNlZCBieSBT YXNoYQ0KSSB0aGluayB0aGF0IHRoZSBjaGFpcnMgbmVlZCBjb25zaWRlciB0YWtpbmcgdGhpcyBi YWNrIHRvIHRoZSBXRw0KZm9yIGZ1cnRoZXIgd29yay4NCg0KLSBTdGV3YXJ0DQo+DQo+IFRoYW5r cywNCj4NCj4gL1lpbWluDQo+DQo+DQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZy b206IHB3ZTMgW21haWx0bzpwd2UzLWJvdW5jZXNAaWV0Zi5vcmddIE9uIEJlaGFsZiBPZiBTdGV3 YXJ0IEJyeWFudA0KPiBTZW50OiBUdWVzZGF5LCBKdWx5IDI5LCAyMDE0IDEwOjUzIEFNDQo+IFRv OiBwd2UzOyBwd2UzLWNoYWlyc0B0b29scy5pZXRmLm9yZw0KPiBDYzogbXBsc0BpZXRmLm9yZw0K PiBTdWJqZWN0OiBSZTogW1BXRTNdIFdHIExhc3QgQ2FsbCBmb3IgZHJhZnQtaWV0Zi1wd2UzLWVu ZHBvaW50LWZhc3QtcHJvdGVjdGlvbi0wMQ0KPg0KPg0KPg0KPiBJIGRvIG5vdCBzdXBwb3J0IHRo ZSBwdWJsaWNhdGlvbiBvZiB0aGlzIGRyYWZ0IHdpdGhvdXQ6DQo+DQo+IDEpIEFuIGFzc3VyYW5j ZSB0aGF0IE1QTFMgV0cgaGFzIHJldmlld2VkIGl0IGFuZCBpcyBoYXBweQ0KPiB3aXRoIHRoZSBl eHRlbnNpb25zIHRoYXQgc2VlbSB0byBiZSBtYWRlIHRvIHRoZSBNUExTIGFyY2hpdGVjdHVyZQ0K Pg0KPiAyKSBBIGZvcm1hbCBkZXNjcmlwdGlvbiBvZiB0aG9zZSBleHRlbnNpb25zIG9yIGEgcG9p bnRlcg0KPiB0byB0aG9zZSBleHRlbnNpb25zIGluIGFub3RoZXIgUkZDLg0KPg0KPiAzKSBTaWdu aWZpY2FudCBjbGFyaWZpY2F0aW9uIG9mIGhvdyB0aGUgZGV0YWlsIG9mIHRoaXMNCj4gbWVjaGFu aXNtIHdvcmtzLg0KPg0KPiBGb3IgdGhpcyB0byB3b3JrIHlvdSBuZWVkIHRvIGhhdmUgY28tb3Jk aW5hdGVkIGxhYmVsIHNwYWNlcw0KPiBpbiBtdWx0aXBsZSBQRXMgd2hpY2ggaXMgc29tZXRoaW5n IHRoYXQgd2FzIGludmVzdGlnYXRlZA0KPiBieSB0aGUgU1BSSU5HIFdHIGFuZCBmb3VuZCBub3Qg dG8gd29yaywgcGFydGljdWxhcmx5IHdoZW4NCj4gZGlmZmVyZW50IG1hbnVmYWN0dXJlcnMgZXF1 aXBtZW50cyB3ZXJlIGludm9sdmVkLg0KPg0KPiBbeXNoZW5dIFRoaXMgZHJhZnQgZG9lcyBub3Qg cmVseSBvbiBsYWJlbCBzcGFjZSBjb29yZGluYXRpb24uIFJhdGhlciwgaXQgaXMgYmFzZWQgb24g Y29udGV4dCBsYWJlbCBmb3J3YXJkaW5nIHBhcmFkaWdtIHNwZWNpZmllZCBpbiBSRkMgNTMzMS4N Cj4NCj4NCj4gQXMgZmFyIGFzIEkgY2FuIHNlZSB5b3UgYWxzbyBuZWVkIHRvIHBlZWsgYmVsb3cg dGhlIHRvcA0KPiBvZiBzdGFjayB0byBkbyBhIGZhc3QgcmVyb3V0ZSBvcGVyYXRpb24gd2hpY2gg aXMgbm90DQo+IGEgY3VycmVudGx5IGRlZmluZWQgTVBMUyBvcGVyYXRpb24uDQo+DQo+IFRoZXJl IHNlZW0gdG8gYmUgYSBsb3Qgb2YgZGV0YWlscyBhbmQgY2xhcmlmaWNhdGlvbnMgbWlzc2luZw0K PiB0aGF0IG5lZWQgdG8gYmUgcHJvdmlkZWQsIGFuZCBJIGFtIG5vdCBlbnRpcmVseSBjb252aW5j ZWQNCj4gdGhhdCBhbGwgb2YgdGhlIGl0ZW1zIGRlY2xhcmVkIG91dCBvZiBzY29wZSBhcmUgbGVn aXRpbWF0ZWx5DQo+IG91dCBvZiBzY29wZSBhbmQgbm90IG5lZWRlZCBpbiBvcmRlciBmb3IgYSBt dWx0aS12ZW5kb3INCj4gc29sdXRpb24gdG8gYmUgaW1wbGVtZW50ZWQuDQo+DQo+IEl0IHdvdWxk IGJlIHVzZWZ1bCB0byBtZSAoYW5kIEkgc3VzcGVjdCB0byBvdGhlciByZWFkZXJzKQ0KPiB0byBz ZWUgd2hhdCB0aGUgZGF0YXBsYW5lIGxvb2tzIGxpa2UgYXQgdmFyaW91cyBwb2ludHMNCj4gaW4g dGhlIG5vcm1hbCBhbmQgcmVwYWlyIHBhdGggdG8gdmVyaWZ5IHRoYXQgYWxsIHBhcnRpZXMNCj4g aGF2ZSB0aGUgcmlnaHQgaW5mb3JtYXRpb24gZm9yIHRoaXMgdG8gd29yayB1bmRlciBub3JtYWwN Cj4gYW5kIHZhcmlvdXMgZmF1bHQgY29uZGl0aW9ucy4NCj4NCj4gSSBoYXZlIGEgbnVtYmVyIG9m IGNvbW1lbnRzIGlubGluZSB0aGF0IG5lZWQgdG8gYmUgYWRkcmVzc2VkDQo+IGJlZm9yZSB0aGlz IGNhbiBwcm9jZWVkLg0KPg0KPg0KPiAtIFN0ZXdhcnQNCj4NCj4NCj4gICAgICAgICAgICAgICAg ICAgICBQVyBFbmRwb2ludCBGYXN0IEZhaWx1cmUgUHJvdGVjdGlvbg0KPiAgICAgICAgICAgICAg ICAgZHJhZnQtaWV0Zi1wd2UzLWVuZHBvaW50LWZhc3QtcHJvdGVjdGlvbi0wMQ0KPg0KPiBBYnN0 cmFjdA0KPg0KPiAgICAgIFRoaXMgZG9jdW1lbnQgc3BlY2lmaWVzIGEgZmFzdCBtZWNoYW5pc20g Zm9yIHByb3RlY3RpbmcgcHNldWRvd2lyZXMNCj4gICAgICBhZ2FpbnN0IGVncmVzcyBlbmRwb2lu dCBmYWlsdXJlcywgaW5jbHVkaW5nIGVncmVzcyBhdHRhY2htZW50IGNpcmN1aXQNCj4gICAgICBm YWlsdXJlLCBlZ3Jlc3MgUEUgZmFpbHVyZSwgbXVsdGktc2VnbWVudCBQVyB0ZXJtaW5hdGluZyBQ RSBmYWlsdXJlLA0KPiAgICAgIGFuZCBtdWx0aS1zZWdtZW50IFBXIHN3aXRjaGluZyBQRSBmYWls dXJlLiAgRGVzaWduZWQgb24gdGhlIGJhc2lzIG9mDQo+ICAgICAgbXVsdGktaG9tZWQgQ0UsIFBX IHJlZHVuZGFuY3ksIHVwc3RyZWFtIGxhYmVsIGFzc2lnbm1lbnQgYW5kIGNvbnRleHQNCj4gICAg ICBzcGVjaWZpYyBsYWJlbCBzd2l0Y2hpbmcsIHRoZSBtZWNoYW5pc20gZW5hYmxlcyBsb2NhbCBy ZXBhaXIgdG8gYmUNCj4gICAgICBwZXJmb3JtZWQgYnkgdGhlIHJvdXRlciB1cHN0cmVhbSBhZGph Y2VudCB0byBhIGZhaWx1cmUuDQo+DQo+ICAgICAgSW4NCj4gICAgICBwYXJ0aWN1bGFyLCB0aGUg cm91dGVyIGNhbiByZXN0b3JlIFBXIHRyYWZmaWMgaW4gdGhlIG9yZGVyIG9mIHRlbnMgb2YNCj4g ICAgICBtaWxsaXNlY29uZHMsDQo+DQo+IFNCPiBOb3Qgc3VyZSBpdCdzIHdpc2UgdG8gc3BlY2lm eSBhIHBlcmZvcm1hbmNlIGNsYWltIGluIGEgc3RhbmRhcmRzDQo+IFNCPiB0cmFjayBSRkMuDQo+ DQo+IFt5c2hlbl0gSWYgdGhpcyBpcyB2aWV3ZWQgYXMgaW5hcHByb3ByaWF0ZSwgd2UgY2FuIHJl bW92ZSBpdC4NCj4NCj4gICAgICBieSB0cmFuc21pdHRpbmcgdGhlIHRyYWZmaWMgdG8gYSBwcm90 ZWN0b3IgdGhyb3VnaCBhDQo+ICAgICAgcHJlLWVzdGFibGlzaGVkIGJ5cGFzcyB0dW5uZWwuICBU aGVyZWZvcmUsIHRoZSBtZWNoYW5pc20gY2FuIHJlZHVjZQ0KPiAgICAgIHRyYWZmaWMgbG9zcyBi ZWZvcmUgZ2xvYmFsIHJlcGFpciByZWFjdHMgdG8gdGhlIGZhaWx1cmUgYW5kIHRoZQ0KPiAgICAg IG5ldHdvcmsgY29udmVyZ2VzIG9uIHRoZSB0b3BvbG9neSBjaGFuZ2VzIGR1ZSB0byB0aGUgZmFp bHVyZS4NCj4NCj4NCj4NCj4gPT09DQo+DQo+DQo+IDEuICBJbnRyb2R1Y3Rpb24NCj4NCj4gICAg ICBQZXIgUkZDIDM5ODUsIFJGQyA0NDQ3IGFuZCBSRkMgNTY1OSwgYSBwc2V1ZG93aXJlIChQVykg b3IgUFcgc2VnbWVudA0KPiAgICAgIGNhbiBiZSB0aG91Z2h0IG9mIGFzIGEgY29ubmVjdGlvbiBi ZXR3ZWVuIGEgcGFpciBvZiBmb3J3YXJkZXJzIGhvc3RlZA0KPiAgICAgIGJ5IHR3byBQRXMsIGNh cnJ5aW5nIGFuIGVtdWxhdGVkIGxheWVyLTIgc2VydmljZSBvdmVyIGEgcGFja2V0DQo+ICAgICAg c3dpdGNoZWQgbmV0d29yayAoUFNOKS4gIEluIHRoZSBzaW5nbGUtc2VnbWVudCBQVyAoU1MtUFcp IGNhc2UsIGENCj4gICAgICBmb3J3YXJkZXIgYmluZHMgYSBQVyB0byBhbiBhdHRhY2htZW50IGNp cmN1aXQgKEFDKS4gIEluIHRoZSBtdWx0aS0NCj4gICAgICBzZWdtZW50IFBXIChNUy1QVykgY2Fz ZSwgYSBmb3J3YXJkZXIgb24gYSB0ZXJtaW5hdGluZyBQRSAoVC1QRSkgYmluZHMNCj4gICAgICBh IFBXIHNlZ21lbnQgdG8gYW4gQUMsIHdoaWxlIGEgZm9yd2FyZGVyIG9uIGEgc3dpdGNoaW5nIFBF IChTLVBFKQ0KPiAgICAgIGJpbmRzIG9uZSBQVyBzZWdtZW50IHRvIGFub3RoZXIgUFcgc2VnbWVu dC4gIEluIGVhY2ggZGlyZWN0aW9uDQo+ICAgICAgYmV0d2VlbiB0aGUgUEVzLCBQVyBwYWNrZXRz IGFyZSB0cmFuc3BvcnRlZCBieSBhIFBTTiB0dW5uZWwsIHdoaWNoIGlzDQo+ICAgICAgY2FsbGVk IGEgdHJhbnNwb3J0IHR1bm5lbC4NCj4NCj4gU0I+IERpZCB3ZSBldmVyIHlvdXIgdGhlIHRlcm0g dHJhbnNwb3J0IHR1bm5lbCBiZWZvcmU/DQo+DQo+IFt5c2hlbl0gVGhpcyBpcyBtYWlubHkgZm9y IGNsYXJpdHkuIFdlIGNvdWxkIGNoYW5nZSBpdCB0byAiIHdoaWNoIGlzDQo+ICAgICAgY2FsbGVk IGEgdHJhbnNwb3J0IHR1bm5lbCAqaW4gdGhpcyBkb2N1bWVudCoiLiBPciB3ZSBjYW4gY29tcGxl dGVseSByZXBsYWNlICJ0cmFuc3BvcnQgdHVubmVsIiB3aXRoIGp1c3QgInR1bm5lbCIuDQo+DQo+ IDMuICBSZWZlcmVuY2UgTW9kZWxzIGFuZCBGYWlsdXJlIENhc2VzDQo+DQo+ICAgICAgVGhlIG1l Y2hhbmlzbSBpbiB0aGlzIGRvY3VtZW50IGludGVuZHMgdG8NCj4gICAgICB1c2UgdGhlc2UgdG9w b2xvZ2llcyBmb3IgbG9jYWwgcmVwYWlyIHB1cnBvc2VzLg0KPg0KPiBTQj4gVGhlIHRvcGxvbG9n eSBpcyBub3QgYSBtZWNoYW5pc20uIEFyZSB5b3Ugc2F5aW5nIHRoYXQgdGhlIG1lY2hhbmltcw0K PiBTQj4gb25seSB3b3JrcyBpbiB0aGlzIHRvcG9sb2d5Pw0KPg0KPiBbeXNoZW5dIE5vLCB0aGVz ZSB0b3BvbG9naWVzIGFyZSBvbmx5IGV4YW1wbGVzIGZvciB1c2UgYnkgc3Vic2VxdWVudCBzZWN0 aW9ucyB0byBkZXNjcmliZSB0aGUgbWVjaGFuaXNtcy4gVGhlIG1lY2hhbmlzbXMgc2hvdWxkIGVx dWFsbHkgd29yayBmb3Igb3RoZXIgdG9wb2xvZ2llcyBhcyB3ZWxsLg0KPg0KPiAgICAgIFRoaXMg U0hBTEwgZW5hYmxlDQo+ICAgICAgbG9jYWwgcmVwYWlyIGFuZCBnbG9iYWwgcmVwYWlyIHRvIHdv cmsgaW4gdGFuZGVtIHRvIGFjaGlldmUgYnJvYWRlcg0KPiAgICAgIGNvdmVyYWdlIG9mIHByb3Rl Y3Rpb24gZm9yIHNlcnZpY2VzLg0KPg0KPiBTQj4gSSBoYXZlIG5vIGlkZWEgaG93IGEgU0hBTEwg aW4gY2FwaXRhbHMgYXBwbGllcyBoZXJlLg0KPg0KPiBbeXNoZW5dIFdpbGwgZml4IHRoaXMuDQo+ DQo+IDMuMS4gIFNpbmdsZS1TZWdtZW50IFBXDQo+DQo+DQo+DQo+ICAgICAgICAgICAgICAgICAg ICAgfDwtLS0tLS0tLS0tLS0tLSBQVzEgLS0tLS0tLS0tLS0tLS0tPnwNCj4NCj4gICAgICAgICAg ICAgICAgIC0gUEUxIC0tLS0tLS0tLS0tLS0tIFAxIC0tLS0tLS0tLS0tLS0tLS0gUEUyIC0NCj4g ICAgICAgICAgICAgICAgLyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICBcDQo+ICAgICAgICAgICAgICAgLyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgIFwNCj4gICAgICAgICAgICBDRTEgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgIENFMg0KPiAgICAgICAgICAgICAgIFwgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAvDQo+ICAgICAgICAgICAg ICAgIFwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgLw0KPiAg ICAgICAgICAgICAgICAgLSBQRTMgLS0tLS0tLS0tLS0tLS0gUDIgLS0tLS0tLS0tLS0tLS0tLSBQ RTQgLQ0KPg0KPiAgICAgICAgICAgICAgICAgICAgIHw8LS0tLS0tLS0tLS0tLS0gUFcyIC0tLS0t LS0tLS0tLS0tLT58DQo+DQo+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgRmln dXJlIDENCj4NCj4NCj4gICAgICBFYWNoIENFIGlzIG11bHRpLWhvbWVkIHRvIHR3byBQRXMuICBI ZW5jZSwgdGhlcmUgYXJlIHR3byBkaXZlcmdlbnQNCj4gICAgICBwYXRocyBiZXR3ZWVuIHRoZSBD RXMuDQo+DQo+IFNCPiBJdCBkb2VzIG5vdCBmb2xsb3cgdGhhdCB0aGUgcGF0aHMgYXJlIGRpdmVy Z2VudCEgV2hvIGtub3dzDQo+IFNCPiB3aGF0IGhhcHBlbnMgaW4gdGhlIElQIGxheWVyIGFuZCB0 aGUgdW5kZXJseWluZyB0cmFuc3BvcnQNCj4gU0I+IG5ldHdvcmsuDQo+DQo+DQo+IFt5c2hlbl0g V2UgY291bGQgcmVwaHJhc2UgdGhpcyBhbmQgcmVtb3ZlICJkaXZlcmdlbnQiLiBBbGwgd2Ugd2Fu dCB0byBzYXkgaGVyZSBpcyB0aGF0IHRoZXJlIGFyZSB0d28gZGlzdGluY3Qgc2V0cyBvZiBBQy1Q Vy1BQyBiZXR3ZWVuIHRoZSBDRXMuDQo+DQo+ICAgICAgQXQgYW55IGdpdmVuIHRpbWUsIGVhY2gg Q0Ugc2VuZHMgdHJhZmZpYyB2aWEgb25seSBvbmUgQUMgYW5kIHJlY2VpdmVzDQo+ICAgICAgdHJh ZmZpYyB2aWEgb25seSBvbmUgQUMuICBUaGUgdHdvIEFDcyBNQVkgb3IgTUFZIE5PVCBiZSB0aGUg c2FtZS4NCj4NCj4gU0I+IE1heSBub3QgYmUgdGhlIHNhbWUgd2hhdD8gSWYgdGhleSBhcmUgYWN0 dWFsbHkgdGhlIHNhbWUgQUMgeW91DQo+IFNCPiBoYXZlIG5vIHJlZHVuZGFuY3kuDQo+DQo+IFt5 c2hlbl0gV2lsbCByZXBocmFzZSBpdC4gQWxsIHdlIHdhbnQgdG8gc2F5IGlzIHRoYXQgYXQgYSBn aXZlbiB0aW1lLCB0aGUgQ0UgbWF5IHNlbmQgdHJhZmZpYyBvdmVyIG9uZSBBQyBhbmQgcmVjZWl2 ZSB0cmFmZmljIG92ZXIgYW5vdGhlciBBQy4NCj4NCj4gICAgICBUaGUgQUMgdXNlZCB0byBzZW5k IHRyYWZmaWMgaXMgZGV0ZXJtaW5lZCBieSB0aGUgQ0UsIGFuZCBNQVkgcmVseSBvbg0KPiAgICAg IGFuIGVuZC10by1lbmQgT0FNIG1lY2hhbmlzbSBiZXR3ZWVuIHRoZSBDRXMuICBUaGUgQUMgdXNl ZCBmb3IgdGhlIENFDQo+ICAgICAgdG8gcmVjZWl2ZSB0cmFmZmljIGlzIGRldGVybWluZWQgYnkg dGhlIHN0YXRlIG9mIHRoZSBuZXR3b3JrIGFuZCB0aGUNCj4gICAgICBwcm90ZWN0aW9uIG1lY2hh bmlzbSBpbiB1c2UsIGFzIGRlc2NyaWJlZCBsYXRlciBpbiB0aGlzIGRvY3VtZW50Lg0KPg0KPiAg ICAgIEZyb20gdGhlIHBlcnNwZWN0aXZlIG9mIHRyYWZmaWMgZmxvd2luZyB0b3dhcmRzIGEgZ2l2 ZW4gQ0UsIHRoZSBzZXQNCj4gICAgICBvZiBQV3MsIFBFcyBhbmQgQUNzIGludm9sdmVkIGNhbiBi ZSB2aWV3ZWQgdG8gc2VydmUgcHJpbWFyeSBhbmQNCj4gICAgICBiYWNrdXAgKG9yIGFjdGl2ZSBh bmQgc3RhbmRieSkgcm9sZXMuICBXaGVuIHRoZSBuZXR3b3JrIGlzIGluIGENCj4gICAgICBzdGVh ZHkgc3RhdGUsIHRoZSBQVyB0aGF0IGlzIGludGVuZGVkIHRvIGNhcnJ5IHRoZSB0cmFmZmljIGlz DQo+ICAgICAgcmVmZXJyZWQgdG8gYXMgYSBwcmltYXJ5IFBXLiAgVGhlIFBFIGF0IHRoZSBlZ3Jl c3Mgb2YgdGhlIHByaW1hcnkgUFcNCj4gICAgICBpcyBhIHByaW1hcnkgUEUuICBUaGUgQUMgY29u bmVjdGluZyB0aGUgQ0UgYW5kIHRoZSBwcmltYXJ5IFBFIGlzIGENCj4gICAgICBwcmltYXJ5IEFD LiAgVGhlIG90aGVyIFBXIG1heSBiZSB1c2VkIHRvIGNhcnJ5IHRoZSB0cmFmZmljIHVwb24gYQ0K PiAgICAgIG5ldHdvcmsgZmFpbHVyZSwgYW5kIGlzIHJlZmVycmVkIHRvIGFzIGEgYmFja3VwIFBX LiAgVGhlIFBFIGF0IHRoZQ0KPiAgICAgIGVncmVzcyBvZiB0aGUgYmFja3VwIFBXIGlzIGEgYmFj a3VwIFBFLiAgVGhlIEFDIGNvbm5lY3RpbmcgdGhlIENFIGFuZA0KPiAgICAgIHRoZSBiYWNrdXAg UEUgaXMgYSBiYWNrdXAgQUMuDQo+DQo+ICAgICAgSW4gdGhpcyBkb2N1bWVudCwgdGhlIGZvbGxv d2luZyBwcmltYXJ5IGFuZCBiYWNrdXAgcm9sZXMgYXJlIGFzc2lnbmVkDQo+ICAgICAgZm9yIHRo ZSB0cmFmZmljIGdvaW5nIGZyb20gQ0UxIHRvIENFMjoNCj4NCj4gICAgICAgICBQcmltYXJ5IFBX OiBQVzENCj4NCj4NCj4NCj4gWWltaW4gU2hlbiwgZXQgYWwuICAgICAgRXhwaXJlcyBKYW51YXJ5 IDI1LCAyMDE1ICAgICAgICAgICAgICAgIFtQYWdlIDVdDQo+DQo+DQo+IEludGVybmV0LURyYWZ0 ICAgICBQVyBFbmRwb2ludCBGYXN0IEZhaWx1cmUgUHJvdGVjdGlvbiAgICAgICAgIEp1bHkgMjAx NA0KPg0KPg0KPiAgICAgICAgIFByaW1hcnkgUEU6IFBFMg0KPg0KPiAgICAgICAgIFByaW1hcnkg QUM6IENFMi1QRTINCj4NCj4gICAgICAgICBCYWNrdXAgUFc6IFBXMg0KPg0KPiAgICAgICAgIEJh Y2t1cCBQRTogUEU0DQo+DQo+ICAgICAgICAgQmFja3VwIEFDOiBDRTItUEU0DQo+DQo+ICAgICAg SW4gdGhpcyBjYXNlLCBhbiBlZ3Jlc3MgQUMgZmFpbHVyZSByZWZlcnMgdG8gdGhlIGZhaWx1cmUg b2YgdGhlIEFDDQo+ICAgICAgQ0UyLVBFMi4gIEFuIGVncmVzcyBub2RlIGZhaWx1cmUgcmVmZXJz IHRvIHRoZSBmYWlsdXJlIG9mIFBFMi4NCj4NCj4gICAgICBUaGUgYmFja3VwIFBFLCBiYWNrdXAg UFcgYW5kIGJhY2t1cCBBQyBtYXkgYmUgdXNlZCB0byBjYXJyeSB0cmFmZmljDQo+ICAgICAgYWZ0 ZXIgYSBQVyBlbmRwb2ludCBmYWlsdXJlLCB3aGVuIENFMSBhbmQgQ0UyIHN3aXRjaGVzIHRyYWZm aWMgdG8gUFcyDQo+ICAgICAgaW4gbG9jYWwgcmVwYWlyIG9yIGdsb2JhbCByZXBhaXIsIGFzIGRl c2NyaWJlZCBsYXRlciBpbiB0aGlzDQo+ICAgICAgZG9jdW1lbnQuDQo+DQo+ICAgICAgICAgICAg ICAgICAgICAgIHw8LS0tLS0tLS0tLS0tLS0gUFcxIC0tLS0tLS0tLS0tLS0tLT58DQo+DQo+ICAg ICAgICAgICAgICAgICAgICAgICAgIC0tLS0tLS0tLS0tLS0gUDEgLS0tLS0tLS0tLS0tLS0tLSBQ RTIgLQ0KPiAgICAgICAgICAgICAgICAgICAgICAgIC8gICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICBcDQo+ICAgICAgICAgICAgICAgICAgICAgICAvICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICBcDQo+ICAgICAgICAgICAgIENFMSAtLSBQRTEgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBDRTINCj4gICAgICAgICAgICAg ICAgICAgICAgIFwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIC8NCj4g ICAgICAgICAgICAgICAgICAgICAgICBcICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgLw0KPiAgICAgICAgICAgICAgICAgICAgICAgICAtLS0tLS0tLS0tLS0tIFAyIC0tLS0t LS0tLS0tLS0tLS0gUEU0IC0NCj4NCj4gICAgICAgICAgICAgICAgICAgICAgIHw8LS0tLS0tLS0t LS0tLS0gUFcyIC0tLS0tLS0tLS0tLS0tLT58DQo+DQo+ICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgRmlndXJlIDINCj4NCj4gICAgICBGaWd1cmUgMiBzaG93cyBhbm90aGVyIHBv c3NpYmxlIHNjZW5hcmlvLCB3aGVyZSBDRTEgaXMgc2luZ2xlLWhvbWVkDQo+ICAgICAgdG8gUEUx LCB3aGlsZSBDRTIgcmVtYWlucyBtdWx0aS1ob21lZCB0byBQRTIgYW5kIFBFNC4gIEZyb20gdGhl DQo+ICAgICAgcGVyc3BlY3RpdmUgb2YgZWdyZXNzIHByb3RlY3Rpb24gZm9yIHRoZSB0cmFmZmlj IGZyb20gQ0UxIHRvIENFMiwNCj4gICAgICB0aGlzIHRvcG9sb2d5IGlzIG5vdCBtdWNoIGRpZmZl cmVudCB0aGFuIEZpZ3VyZSAxLiAgSG93ZXZlciwgZm9yIHRoZQ0KPg0KPiBTQj4gIm5vdCBtdWNo IGRpZmZlcmVudCB0aGFuIEZpZ3VyZSAxIiBpcyBub3QgaGVscGZ1bCwgZWl0aGVyIGl0IGlzIHRo ZQ0KPiBzYW1lLCBvciB5b3UgbmVlZCB0byBleHBsYWluIHRoZSBkaWZmZXJlbmNlLg0KPg0KPiBb eXNoZW5dIFdpbGwgcmVwaHJhc2UuIFRoZSBkaWZmZXJlbmNlIGlzIGFjdHVhbGx5IGRlc2NyaWJl ZCBhZnRlciAiSG93ZXZlci4uLiIuDQo+DQo+DQo+ICAgICAgdHJhZmZpYyBpbiB0aGUgZGlyZWN0 aW9uIGZyb20gQ0UyIHRvIENFMSwgUEUxIG11c3QgYW50aWNpcGF0ZSB0cmFmZmljDQo+ICAgICAg b24gYm90aCBQVzEgYW5kIFBXMiwgYW5kIHNlbmRzIGl0IHRvIENFMSBvdmVyIHRoZSBBQyBDRTEt UEUxLg0KPg0KPiBTQj4gV2hhdCBkb2VzIGl0IG1lYW4gdG8gYW50aWNpcGF0ZSB0cmFmZmljLg0K Pg0KPiBbeXNoZW5dIFRvIGFudGljaXBhdGUgaW5jb21pbmcgdHJhZmZpYyBvdmVyIGVpdGhlciBQ VzEgb3IgUFcyLiBXaWxsIHJlcGhyYXNlLg0KPg0KPiAzLjIuICBNdWx0aS1TZWdtZW50IFBXDQo+ DQo+DQo+DQo+DQo+DQo+DQo+DQo+DQo+DQo+DQo+DQo+IFlpbWluIFNoZW4sIGV0IGFsLiAgICAg IEV4cGlyZXMgSmFudWFyeSAyNSwgMjAxNSAgICAgICAgICAgICAgICBbUGFnZSA2XQ0KPg0KPg0K PiBJbnRlcm5ldC1EcmFmdCAgICAgUFcgRW5kcG9pbnQgRmFzdCBGYWlsdXJlIFByb3RlY3Rpb24g ICAgICAgICBKdWx5IDIwMTQNCj4NCj4NCj4gICAgICAgICAgICAgICAgICAgICB8PC0tLS0tLS0t LS0tLS0tLSBQVzEgLS0tLS0tLS0tLS0tLS0tPnwNCj4gICAgICAgICAgICAgICAgICAgICB8PC0t LS0tIFNFRzEgLS0tLS0+fDwtLS0tLSBTRUcyIC0tLS0tPnwNCj4NCj4gICAgICAgICAgICAgICAg LSBUUEUxIC0tLS0tLS0tLS0tLS0tIFNQRTEgLS0tLS0tLS0tLS0tLS0tIFRQRTIgLQ0KPiAgICAg ICAgICAgICAgIC8gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgXA0KPiAgICAgICAgICAgICAgLyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgIFwNCj4gICAgICAgICAgIENFMSAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgQ0UyDQo+ICAgICAgICAgICAgICBcICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgLw0KPiAgICAgICAg ICAgICAgIFwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg Lw0KPiAgICAgICAgICAgICAgICAtIFRQRTMgLS0tLS0tLS0tLS0tLS0gU1BFMiAtLS0tLS0tLS0t LS0tLS0gVFBFNCAtDQo+DQo+ICAgICAgICAgICAgICAgICAgICAgfDwtLS0tLSBTRUczIC0tLS0t Pnw8LS0tLS0gU0VHNCAtLS0tLT58DQo+ICAgICAgICAgICAgICAgICAgICAgfDwtLS0tLS0tLS0t LS0tLS0gUFcyIC0tLS0tLS0tLS0tLS0tLT58DQo+DQo+ICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgRmlndXJlIDMNCj4NCj4gICAgICBGaWd1cmUgMyBzaG93cyBhIHRvcG9sb2d5 IHRoYXQgaXMgc2ltaWxhciB0byBGaWd1cmUgMSBidXQgaW4gYW4gTVMtUFcNCj4gICAgICBlbnZp cm9ubWVudC4gIFBXMSBhbmQgUFcyIGFyZSBib3RoIE1TLVBXcy4gIFBXMSBpcyBlc3RhYmxpc2hl ZA0KPiAgICAgIGJldHdlZW4gVFBFMSBhbmQgVFBFMiwgYW5kIHN3aXRjaGVkIGJldHdlZW4gc2Vn bWVudHMgU0VHMSBhbmQgU0VHMiBhdA0KPiAgICAgIFNQRTEuICBQVzIgaXMgZXN0YWJsaXNoZWQg YmV0d2VlbiBUUEUzIGFuZCBUUEU0LCBhbmQgc3dpdGNoZWQgYmV0d2Vlbg0KPiAgICAgIHNlZ21l bnRzIFNFRzMgYW5kIFNFRzQgYXQgU1BFMi4gIENFMSBpcyBtdWx0aS1ob21lZCB0byBUUEUxIGFu ZCBUUEUzLg0KPiAgICAgIENFMiBpcyBtdWx0aS1ob21lZCB0byBUUEUyIGFuZCBUUEU0LiAgVGhl IHRyYW5zcG9ydCB0dW5uZWxzIG9mIHRoZSBQVw0KPiAgICAgIHNlZ21lbnRzIGFyZSBub3Qgc2hv d24gaW4gdGhpcyBmaWd1cmUgZm9yIGNsYXJpdHkuDQo+DQo+ICAgICAgSW4gdGhpcyBkb2N1bWVu dCwgdGhlIGZvbGxvd2luZyBwcmltYXJ5IGFuZCBiYWNrdXAgcm9sZXMgYXJlIGFzc2lnbmVkDQo+ ICAgICAgZm9yIHRoZSB0cmFmZmljIGdvaW5nIGZyb20gQ0UxIHRvIENFMjoNCj4NCj4gICAgICAg ICBQcmltYXJ5IFBXOiBQVzENCj4NCj4gICAgICAgICBQcmltYXJ5IFQtUEU6IFRQRTINCj4NCj4g ICAgICAgICBQcmltYXJ5IFMtUEU6IFNQRTENCj4NCj4gICAgICAgICBQcmltYXJ5IEFDOiBDRTIt VFBFMg0KPg0KPiAgICAgICAgIEJhY2t1cCBQVzogUFcyDQo+DQo+ICAgICAgICAgQmFja3VwIFQt UEU6IFRQRTQNCj4NCj4gICAgICAgICBCYWNrdXAgUy1QRTogU1BFMg0KPg0KPiAgICAgICAgIEJh Y2t1cCBBQzogQ0UyLVRQRTQNCj4NCj4gICAgICBJbiB0aGlzIGNhc2UsIGFuIGVncmVzcyBBQyBm YWlsdXJlIHJlZmVycyB0byB0aGUgZmFpbHVyZSBvZiB0aGUgQUMNCj4gICAgICBDRTItVFBFMi4g IEFuIGVncmVzcyBub2RlIGZhaWx1cmUgcmVmZXJzIHRvIHRoZSBmYWlsdXJlIG9mIFRQRTIuICBB bg0KPiAgICAgIFMtUEUgZmFpbHVyZSByZWZlcnMgdG8gdGhlIGZhaWx1cmUgb2YgU1BFMS4NCj4N Cj4NCj4NCj4NCj4NCj4gWWltaW4gU2hlbiwgZXQgYWwuICAgICAgRXhwaXJlcyBKYW51YXJ5IDI1 LCAyMDE1ICAgICAgICAgICAgICAgIFtQYWdlIDddDQo+DQo+DQo+IEludGVybmV0LURyYWZ0ICAg ICBQVyBFbmRwb2ludCBGYXN0IEZhaWx1cmUgUHJvdGVjdGlvbiAgICAgICAgIEp1bHkgMjAxNA0K Pg0KPg0KPiAgICAgIFRoZSBiYWNrdXAgVC1QRSwgYmFja3VwIFBXIGFuZCBiYWNrdXAgQUMgYXJl IHVzZWQgZm9yIHByb3RlY3RpbmcgdGhlDQo+ICAgICAgcHJpbWFyeSBQVyBhZ2FpbnN0IGVncmVz cyBBQyBmYWlsdXJlIGFuZCBlZ3Jlc3Mgbm9kZSBmYWlsdXJlLiAgVGhlDQo+ICAgICAgYmFja3Vw IFMtUEUgYW5kIHRoZSBiYWNrdXAgUFcgYXJlIHVzZWQgZm9yIHByb3RlY3RpbmcgdGhlIHByaW1h cnkgUFcNCj4gICAgICBhZ2FpbnN0IFMtUEUgZmFpbHVyZSwgYXMgZGVzY3JpYmVkIGxhdGVyIGlu IHRoaXMgZG9jdW1lbnQuDQo+DQo+ICAgICAgRm9yIGNvbnNpc3RlbmN5IHdpdGggdGhlIFNTLVBX IHNjZW5hcmlvLCBwcmltYXJ5IFQtUEVzIGFuZCBhIHByaW1hcnkNCj4gICAgICBTLVBFcyBtYXkg c2ltcGx5IGJlIHJlZmVycmVkIHRvIGFzIHByaW1hcnkgUEVzIGluIHRoaXMgZG9jdW1lbnQsDQo+ ICAgICAgd2hlcmUgc3BlY2lmaWNzIGlzIG5vdCByZXF1aXJlZC4gIFNpbWlsYXJseSwgYmFja3Vw IFQtUEVzIGFuZCBiYWNrdXANCj4gICAgICBTLVBFcyBtYXkgYmUgcmVmZXJyZWQgdG8gYXMgYmFj a3VwIFBFcy4NCj4NCj4gNC4gIFRoZW9yeSBvZiBPcGVyYXRpb24NCj4NCj4gICAgICBUaGUgZmFz dCBwcm90ZWN0aW9uIG1lY2hhbmlzbSBpbiB0aGlzIGRvY3VtZW50IHByb3ZpZGVzIHRocmVlIHR5 cGVzDQo+ICAgICAgb2YgcHJvdGVjdGlvbiBmb3IgUFdzLCBjb3JyZXNwb25kaW5nIHRvIHRoZSB0 aHJlZSB0eXBlcyBvZiBmYWlsdXJlcw0KPiAgICAgIGRlc2NyaWJlZCBpbiBTZWN0aW9uIDEuDQo+ DQo+ICAgICAgYS4gIEVncmVzcyBBQyBwcm90ZWN0aW9uDQo+DQo+ICAgICAgYi4gIEVncmVzcyAo VC0pUEUgbm9kZSBwcm90ZWN0aW9uDQo+DQo+ICAgICAgYy4gIFMtUEUgbm9kZSBwcm90ZWN0aW9u DQo+DQo+ICAgICAgVGhlIG1lY2hhbmlzbSBhc3N1bWVzIGEgbXVsdGktaG9tZWQgQ0Ugd2l0aCBj b25uZWN0aXZpdHkgdG8gYSBwcmltYXJ5DQo+ICAgICAgUEUgYW5kIGEgYmFja3VwIFBFLCBhbmQg dGhlIGV4aXN0ZW5jZSBvZiBhIGJhY2t1cCBQVyBpbiB0aGUgbmV0d29yay4NCj4gICAgICBJbiBT LVBFIG5vZGUgcHJvdGVjdGlvbiwgaXQgYWxzbyBhc3N1bWVzIHRoZSBleGlzdGVuY2Ugb2YgYSBi YWNrdXANCj4gICAgICBTLVBFIG9uIHRoZSBiYWNrdXAgUFcuDQo+DQo+DQo+IFNCPiBEbyB0aGUg UFdzIG5lZWQgdG8gYmUgc3ltbWV0cmljIC0gY291bGQgeW91IGhhdmUgbW9yZQ0KPiBTLVBFcyBv biBvbmUgdGhhdCB5b3UgaGF2ZSBvbiB0aGUgb3RoZXI/DQo+DQo+IFt5c2hlbl0gWW91IGNvdWxk LiBGb3IgYSBnaXZlbiBTLVBFLCBhcyBsb25nIGFzIHlvdSBjYW4gZmluZCBhbm90aGVyIFBFIHRv IHNlcnZlIGFzIHByb3RlY3RvciwgaXQgd2lsbCB3b3JrLg0KPg0KPiA0LjEuICBMb2NhbCBSZXBh aXIgYW5kIFByb3RlY3Rvcg0KPg0KPiAgICAgIFRoZSBtZWNoYW5pc20gcmVsaWVzIG9uIGxvY2Fs IHJlcGFpciB0byBiZSBwZXJmb3JtZWQgYnkgcm91dGVycw0KPiBTQj4gRG8geW91IG1lYW4gcm91 dGVycyBvciBQRXM/IFJvdXRlcnMgb2YgbW9yZSBsaWtlbHkgTFNScw0KPiBvcGVyYXRlIGF0IGEg ZGlmZmVyZW50IGxheWVyIGFuZCBoYXZlIG5vIGJ1c2luZXNzIGtub3dpbmcgYWJvdXQNCj4gUFdz Lg0KPg0KPiBbeXNoZW5dIHJvdXRlcnMgKExTUnMpLiBZZXMsIHRoZXkgaGF2ZSBubyBpZGVhIGFi b3V0IFBXcy4gVGhpcyBkcmFmdCBkb2Vzbid0IGFzc3VtZSB0aGV5IGtub3cgb3IgY2FyZSBhYm91 dCBQV3MuDQo+DQo+IFNCPiBIYXMgdGhpcyB3b3JrIGJlZW4gcmV2aWV3ZWQgbXkgTVBMUyBXRz8g SSBhbSBjb25jZXJuZWQNCj4gdGhhdCB5b3UgYXJlIGRvaW5nIGEgYnVuY2ggb2YgbGF5ZXIgdmlv bGF0aW9uIHdpdGhvdXQgZXhwbGFpbmluZw0KPiB0aGUgaG93IHRoaXMgaXMgYWNoaWV2ZWQgb3Ig d2hldGhlciBpdCBpcyBhY2NlcHRhYmxlIHdpdGhpbiB0aGUNCj4gTVBMUyBhcmNoaXRlY3R1cmUu DQo+DQo+IFt5c2hlbl0gVGhlcmUgaXMgbm8gbGF5ZXIgdmlvbGF0aW9uLiBUaGUgUExSIHNpbXBs eSByZWRpcmVjdCB0cmFmZmljIGZyb20gdGhlIG9yaWdpbmFsIHR1bm5lbCB0byBhIGJ5cGFzcyB0 dW5uZWwsIGJ5IGRvaW5nIGEgdG9wIGxhYmVsIHN3YXAsIHdpdGhvdXQgdG91Y2hpbmcsIGNoYW5n aW5nIG9yIGtub3dpbmcgdGhlIFBXIGxhYmVsLiBQbGVhc2UgdGFrZSBhIGNsb3NlIGxvb2sgYXQg dGhlIHRleHQuDQo+DQo+ICAgICAgdXBzdHJlYW0gYWRqYWNlbnQgdG8gZmFpbHVyZXMuICBFYWNo IG9mIHRoZXNlIHJvdXRlcnMgaXMgcmVmZXJyZWQgdG8NCj4gICAgICBhcyBhICJwb2ludCBvZiBs b2NhbCByZXBhaXIiIChQTFIpLiAgQSBQTFIgTVVTVCBiZSBhYmxlIHRvIGRldGVjdCBhDQo+ICAg ICAgZmFpbHVyZSBieSB1c2luZyBhIHJhcGlkIG1lY2hhbmlzbSwgc3VjaCBhcyBwaHlzaWNhbCBs YXllciBmYWlsdXJlDQo+ICAgICAgZGV0ZWN0aW9uLCBCaWRpcmVjdGlvbmFsIEZhaWx1cmUgRGV0 ZWN0aW9uIChCRkQpIChSRkMgNTg4MCksIGV0Yy4gIEluDQo+ICAgICAgYW50aWNpcGF0aW9uIG9m IHRoZSBmYWlsdXJlLCB0aGUgUExSIE1VU1QgYWxzbyBwcmUtZXN0YWJsaXNoIGEgYnlwYXNzDQo+ ICAgICAgUFNOIHR1bm5lbCB0byBhICJwcm90ZWN0b3IiLCBhbmQgcHJlLWluc3RhbGwgYSBieXBh c3Mgcm91dGUgaW4gdGhlDQo+ICAgICAgZGF0YSBwbGFuZS4gIFRoZSBieXBhc3MgdHVubmVsIE1V U1QgaGF2ZSB0aGUgcHJvcGVydHkgdGhhdCBpdCB3aWxsDQo+ICAgICAgbm90IGJlIGFmZmVjdGVk IGJ5IHRoZSB0b3BvbG9neSBjaGFuZ2VzIGluIHRoZSBldmVudCBvZiB0aGUgZmFpbHVyZS4NCj4N Cj4gU0I+IFRoaXMgaXMgZmFyIHRvbyBpbXByZWNpc2UgLSB5b3UgbmVlZCB0byBzZXBhcmF0ZSBv cGVyYXRpb25zIGFuZA0KPiB0b3BvbG9naWVzIGluIHRoZSBQVyBsYXllciBmcm9tIG9wZXJhdGlv bnMgYW5kIHRvcG9sb2dpZXMgaW4gdGhlDQo+IFBTTiBsYXllcg0KPg0KPiBbeXNoZW5dIFNpbWls YXIgdG8gZXhpc3RpbmcgSVAvTVBMUyBmYXN0LXJlcm91dGUgUkZDcywgdGhpcyBkcmFmdCBkb2Vz bid0IHJlc3RyaWN0IGxvY2FsIHJlcGFpciB0cmlnZ2VyIHRvIGFueSBzcGVjaWZpYyBmYWlsdXJl IGRldGVjdGlvbiBtZWNoYW5pc21zLiBBbnkgbWVjaGFuaXNtIHdpbGwgd29yay4NCj4NCj4gU0I+ IEluIGVhY2ggY2FzZSB5b3UgbmVlZCB0byBiZSBjbGVhciBob3cgeW91IGFyZSBkZXRlY3Rpbmcg ZmFpbHVyZQ0KPiBieSBzcGVjaWZ5aW5nIHRoZSBNRVBzIGFuZCBNSVBzIHlvdSBhcmUgdGVzdGlu Zy4NCj4NCj4gICAgICBVcG9uIGRldGVjdGluZyB0aGUgZmFpbHVyZSwgdGhlIFBMUiBNVVNUIGlu dm9rZSB0aGUgYnlwYXNzIHJvdXRlIGluDQo+ICAgICAgdGhlIGRhdGEgcGxhbmUsIGFuZCByZXJv dXRlIFBXIHRyYWZmaWMgdG8gdGhlIHByb3RlY3RvciB0aHJvdWdoIHRoZQ0KPiAgICAgIGJ5cGFz cyB0dW5uZWwuICBUaGUgcHJvdGVjdG9yIE1VU1QgaW4gdHVybiBzZW5kIHRoZSB0cmFmZmljIHRv IHRoZQ0KPiAgICAgIHRhcmdldCBDRS4NCj4NCj4gU0I+IE5vIHRoZSBwcm90ZWN0b3Iga25vd3Mg bm90aGluZyBhYm91dCBDRXMgLSB0aGV5IGFyZSBpbiB0aGUgY2xpZW50DQo+IGxheWVyLCBhbmQg eW91IGNhbiBvbmx5IHByb3RlY3QgYXQgdGhlIHNlcnZlciBsYXllciBvciB0aGUgc2VydmVyJ3MN Cj4gc2VydmVyIGxheWVyLg0KPg0KPiBbeXNoZW5dIEluIGNvbGxvY2F0ZWQgbW9kZWwsIHRoZSBw cm90ZWN0b3IgaXMgYSBQRS4gSW4gdGhlIGNlbnRyYWxpemVkIG1vZGVsLCB0aGUgcHJvdGVjdG9y IG1heSBiZSBhbiBMU1IgLCBhbmQgaXQgbmVlZHMgdG8gcGFydGljaXBhdGUgaW4gTERQIFBXIHNp Z25hbGluZyB0byB1bmRlcnN0YW5kIFBXcy4gU2VlIHRoZSByZWxhdGVkIHNlY3Rpb25zIGFuZCBM RFAgZXh0ZW5zaW9ucyBpbiB0aGlzIGRyYWZ0LiBUaGlzIGNlbnRyYWxpemVkIG1vZGVsIGlzIG9w dGlvbmFsLg0KPg0KPiAgICAgIFRoaXMgcHJvY2VkdXJlIGlzIHJlZmVycmVkIHRvIGFzIGxvY2Fs IHJlcGFpci4NCj4NCj4gICAgICBEaWZmZXJlbnQgcm91dGVycyBtYXkgc2VydmUgYXMgUExSIGFu ZCBwcm90ZWN0b3IgaW4gZGlmZmVyZW50DQo+ICAgICAgc2NlbmFyaW9zLg0KPg0KPiBTQj4gUmVt ZW1iZXIgdGhleSBhcmUgbm90IHJvdXRlcnMgLSB0aGV5IGFyZSB4UEVzIG9yIExTUnMuDQo+DQo+ IFt5c2hlbl0gQWdhaW4sIGluIGNvbGxvY2F0ZWQgbW9kZWwsIHRoZSBwcm90ZWN0b3IgaXMgYSBQ RS4gSW4gdGhlIGNlbnRyYWxpemVkIG1vZGVsLCB0aGUgcHJvdGVjdG9yIG1heSBiZSBhbiBMU1Ig LCBhbmQgaXQgbmVlZHMgdG8gcGFydGljaXBhdGUgaW4gTERQIFBXIHNpZ25hbGluZyB0byB1bmRl cnN0YW5kIFBXcy4gU2VlIHRoZSByZWxhdGVkIHNlY3Rpb25zIGFuZCBMRFAgZXh0ZW5zaW9ucyBp biB0aGlzIGRyYWZ0LiBCdXQgdGhpcyBjZW50cmFsaXplZCBtb2RlbCBpcyBvcHRpb25hbC4NCj4g W3lzaGVuXSBQTFJzIG1heSBiZSBMU1JzLCBidXQgdGhleSBkb24ndCBuZWVkIHRvIGtub3cgUFdz LCBiZWNhdXNlIGFsbCB0aGV5IGRvIGlzIHRvIHN3YXAgdG9wIGxhYmVsICh0dW5uZWwgbGFiZWwp IHRvIGEgYnlwYXNzIHR1bm5lbCdzIGxhYmVsLg0KPg0KPiBTQj4gSW4gZWFjaCBvZiB0aGUgZm9s bG93aW5nIHdlIG5lZWQgdG8gbm90ZSB0aGF0IHRoZXNlIGFyZQ0KPiBub3QgUkZDMzk4NSBQRXMs IGV2ZW4gaWYgdGhleSB0YWtlIG5vIHBhcnQgaW4gdGhlIHByb3RlY3Rpb24NCj4gc3VjaCBhcyB0 aGUgUEVzIG9uIHRoZSBsZWZ0LiBJbmRlZWQgSSBhbSB3b25kZXJpbmcgaWYgeW91IG5lZWQNCj4g dG8gdXBkYXRlIFJGQzM5ODUgYW5kIFJGQzU2NTkNCj4NCj4gW3lzaGVuXSBUaGVyZSBpcyBubyBu ZWVkIHRvIHVwZGF0ZSBSRkMzOTg1IGFuZCBSRkM1NjU5Lg0KPg0KPiBZaW1pbiBTaGVuLCBldCBh bC4gICAgICBFeHBpcmVzIEphbnVhcnkgMjUsIDIwMTUgICAgICAgICAgICAgICAgW1BhZ2UgOF0N Cj4NCj4NCj4gSW50ZXJuZXQtRHJhZnQgICAgIFBXIEVuZHBvaW50IEZhc3QgRmFpbHVyZSBQcm90 ZWN0aW9uICAgICAgICAgSnVseSAyMDE0DQo+DQo+DQo+ICAgICAgbyAgSW4gZWdyZXNzIEFDIHBy b3RlY3Rpb24sIHRoZSBQTFIgaXMgdGhlIHByaW1hcnkgUEUgdGhhdCB0ZXJtaW5hdGVzDQo+ICAg ICAgICAgdGhlIHByaW1hcnkgUFcgYW5kIGhvc3RzIHRoZSBwcmltYXJ5IEFDLCBhbmQgdGhlIHBy b3RlY3RvciBpcyB0aGUNCj4gICAgICAgICBiYWNrdXAgUEUgKEZpZ3VyZSA0KS4NCj4NCj4gICAg ICAgICAgICAgICAgICAgICB8PC0tLS0tLS0tLS0tLS0tIFBXMSAtLS0tLS0tLS0tLS0tLS0+fA0K Pg0KPiAgICAgICAgICAgICAgICAgLSBQRTEgLS0tLS0tLS0tLS0tLS0gUDEgLS0tLS0tLS0tLS0t LS0tLSBQRTIgLQ0KPiAgICAgICAgICAgICAgICAvICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICBQTFIgIFwNCj4gICAgICAgICAgICAgICAvICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgIHwgICAgXA0KPiAgICAgICAgICAgIENFMSAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgYnlwYXNzfCAgICAgQ0UyDQo+ICAgICAgICAg ICAgICAgXCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB8ICAgIC8N Cj4gICAgICAgICAgICAgICAgXCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgIHwgICAvDQo+ICAgICAgICAgICAgICAgICAtIFBFMyAtLS0tLS0tLS0tLS0tLSBQMiAtLS0t LS0tLS0tLS0tLS0tIFBFNCAtDQo+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgIHByb3RlY3Rvcg0KPg0KPiAgICAgICAgICAgICAgICAgICAgIHw8 LS0tLS0tLS0tLS0tLS0gUFcyIC0tLS0tLS0tLS0tLS0tLT58DQo+DQo+ICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgRmlndXJlIDQNCj4NCj4NCj4gICAgICBvICBJbiBlZ3Jlc3Mg UEUgbm9kZSBwcm90ZWN0aW9uLCB0aGUgUExSIGlzIHRoZSBwZW51bHRpbWF0ZSBob3ANCj4gICAg ICAgICByb3V0ZXIgb2YgdGhlIHRyYW5zcG9ydCB0dW5uZWwgb2YgdGhlIHByaW1hcnkgUFcsIGFu ZCB0aGUNCj4gICAgICAgICBwcm90ZWN0b3IgaXMgdGhlIGJhY2t1cCBQRSAoRmlndXJlIDUpLg0K Pg0KPiAgICAgICAgICAgICAgICAgICAgIHw8LS0tLS0tLS0tLS0tLS0gUFcxIC0tLS0tLS0tLS0t LS0tLT58DQo+DQo+ICAgICAgICAgICAgICAgICAtIFBFMSAtLS0tLS0tLS0tLS0tLSBQMSAtLS0t LS0tIFAzIC0tLS0tIFBFMiAtDQo+ICAgICAgICAgICAgICAgIC8gICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgUExSIFwgICAgICAgICAgXA0KPiAgICAgICAgICAgICAgIC8gICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgXCAgICAgICAgICBcDQo+ICAgICAgICAgICAgQ0Ux ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgYnlwYXNzXCAgICAgICAgICBDRTINCj4g ICAgICAgICAgICAgICBcICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXCAg ICAgICAgLw0KPiAgICAgICAgICAgICAgICBcICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgXCAgICAgIC8NCj4gICAgICAgICAgICAgICAgIC0gUEUzIC0tLS0tLS0tLS0tLS0t IFAyIC0tLS0tLS0tLS0tLS0tLS0gUEU0IC0NCj4gICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgcHJvdGVjdG9yDQo+DQo+ICAgICAgICAgICAgICAg ICAgICAgfDwtLS0tLS0tLS0tLS0tLSBQVzIgLS0tLS0tLS0tLS0tLS0tPnwNCj4NCj4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICBGaWd1cmUgNQ0KPg0KPiBTQj4gTm90ZSB0aGF0 IGl0IGlzIG5vdGhpbmcgbGlrZSBhcyBjbGVhbiBhcyB0aGlzLCBpZiBhcyB5b3UgbGF0ZXIgaGF2 ZSwNCj4gc29tZSBvZiBQMydzIHRyYWZmaWMgbmVlZHMgdG8gZ28gdG8gUEU0IGFuZCBzb21lIHRv IFBFNS4NCj4gQWxzbyB3aGF0IGFib3V0IHRoZQ0KPg0KPiBbeXNoZW5dIE5vdGUgdGhhdCB3ZSBh cmUgdGFsa2luZyBhYm91dCBhIHNpbmdsZSBQVywgcmF0aGVyIHRoYW4gbXVsdGlwbGUgUFdzIGNh cnJpZWQgYnkgYSBzaW5nbGUgdHVubmVsLg0KPg0KPiAgICAgIG8gIEluIFMtUEUgbm9kZSBwcm90 ZWN0aW9uLCB0aGUgUExSIGlzIHRoZSBwZW51bHRpbWF0ZSBob3Agcm91dGVyIG9mDQo+ICAgICAg ICAgdGhlIHRyYW5zcG9ydCB0dW5uZWwgb2YgdGhlIHByaW1hcnkgUFcgc2VnbWVudCwgYW5kIHRo ZSBwcm90ZWN0b3INCj4gICAgICAgICBpcyB0aGUgYmFja3VwIFMtUEUgKEZpZ3VyZSA2KS4NCj4N Cj4NCj4NCj4NCj4NCj4NCj4NCj4NCj4NCj4NCj4gWWltaW4gU2hlbiwgZXQgYWwuICAgICAgRXhw aXJlcyBKYW51YXJ5IDI1LCAyMDE1ICAgICAgICAgICAgICAgIFtQYWdlIDldDQo+DQo+DQo+IElu dGVybmV0LURyYWZ0ICAgICBQVyBFbmRwb2ludCBGYXN0IEZhaWx1cmUgUHJvdGVjdGlvbiAgICAg ICAgIEp1bHkgMjAxNA0KPg0KPg0KPiAgICAgICAgICAgICAgICAgICAgIHw8LS0tLS0tLS0tLS0t LS0tIFBXMSAtLS0tLS0tLS0tLS0tLS0+fA0KPiAgICAgICAgICAgICAgICAgICAgIHw8LS0tLS0g U0VHMSAtLS0tLT58PC0tLS0tIFNFRzIgLS0tLS0+fA0KPg0KPiAgICAgICAgICAgICAgICAtIFRQ RTEgLS0tLS0gUDQgIC0tLS0tIFNQRTEgLS0tLS0tLS0tLS0tLS0gVFBFMiAtDQo+ICAgICAgICAg ICAgICAgLyAgICAgICAgICAgICBQTFIgXCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBc DQo+ICAgICAgICAgICAgICAvICAgICAgICAgICAgICAgICAgIFwgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgXA0KPiAgICAgICAgICAgQ0UxICAgICAgICAgICAgICAgYnlwYXNzXCAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICBDRTINCj4gICAgICAgICAgICAgIFwgICAgICAgICAg ICAgICAgICAgICBcICAgICAgICAgICAgICAgICAgICAgICAgICAgICAvDQo+ICAgICAgICAgICAg ICAgXCAgICAgICAgICAgICAgICAgICAgIFwgICAgICAgICAgICAgICAgICAgICAgICAgICAvDQo+ ICAgICAgICAgICAgICAgIC0gVFBFMyAtLS0tLS0tLS0tLS0tLS0gU1BFMiAtLS0tLS0tLS0tLS0t LSBUUEU0IC0NCj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBwcm90ZWN0b3IN Cj4NCj4gICAgICAgICAgICAgICAgICAgICB8PC0tLS0tIFNFRzMgLS0tLS0+fDwtLS0tLSBTRUc0 IC0tLS0tPnwNCj4gICAgICAgICAgICAgICAgICAgICB8PC0tLS0tLS0tLS0tLS0tLSBQVzIgLS0t LS0tLS0tLS0tLS0tPnwNCj4NCj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBG aWd1cmUgNg0KPg0KPiAgICAgIEEgUExSIGNhbiByZWFsaXplIGl0cyByb2xlIGJhc2VkIG9uIGNv bmZpZ3VyYXRpb24gb3IgdGhlIHNpZ25hbGluZyBvZg0KPiAgICAgIHRyYW5zcG9ydCB0dW5uZWwu ICBGb3IgZXhhbXBsZSwgaW4gdGhlIGNhc2Ugd2hlcmUgdGhlIHRyYW5zcG9ydA0KPiAgICAgIHR1 bm5lbCBpcyBzaWduYWxlZCBieSBSU1ZQLCB0aGUgcGVudWx0aW1hdGUgaG9wIHJvdXRlciBjb3Vs ZCByZWFsaXplDQo+DQo+IFNCPiBDb3VsZCByZWFsaXplIGlzIG5vdCBzdWZmaWNpZW50LiBUaGUg bWVjaGFuaXNtIG5lZWRzIHRvIGJlDQo+IHVuY29uZGl0aW9uYWwuDQo+DQo+IFt5c2hlbl0gV2ls bCByZXBocmFzZS4NCj4NCj4gICAgICB0aGF0IGl0IGlzIHRoZSBQTFIgZm9yIGVncmVzcyAoVC0p UEUgb3IgUy1QRSBmYWlsdXJlIGJhc2VkIG9uIHRoZSBSUk8NCj4gICAgICBpbiBSZXN2IG1lc3Nh Z2UsIHdoaWNoIHNob3VsZCBpbmRpY2F0ZSB0aGF0IHRoZSByb3V0ZXIgaXMgb25lIGhvcA0KPiAg ICAgIGF3YXkgZnJvbSB0aGUgUEUuICBUaGUgZGV0YWlsIG9mIGhvdyB0aGlzIGNvdWxkIGJlIGFj aGlldmVkIG9uIGEgcGVyLQ0KPiAgICAgIHByb3RvY29sIGJhc2lzIGlzIG91dCBvZiB0aGUgc2Nv cGUgb2YgdGhpcyBkb2N1bWVudC4NCj4NCj4gU0I+IFNvIHdoZXJlIGlzIGl0IGluIHNjb3BlPyBI b3cgZG8gSSBmaW5kIG91dCBob3cgdG8gZG8gaXQ/DQo+DQo+ICAgICAgSW4gYWxsIHNjZW5hcmlv cywgd2hlbiBhIFBMUiByZXJvdXRlcyB0cmFmZmljIHRocm91Z2ggYSBieXBhc3MgdHVubmVsDQo+ ICAgICAgdG8gYSBwcm90ZWN0b3IgZHVyaW5nIGxvY2FsIHJlcGFpciwgaXQgTVVTVCBrZWVwIHRo ZSBsYWJlbCBvZiB0aGUNCj4gICAgICBwcmltYXJ5IFBXIGludGFjdCBpbiB0aGUgcGFja2V0cy4g IFRoaXMgb2J2aWF0ZXMgdGhlIG5lZWQgZm9yIHRoZSBQTFINCj4gICAgICB0byBtYWludGFpbiBi eXBhc3Mgcm91dGVzIG9uIGEgcGVyLVBXIGJhc2lzLCBhbmQgYWxsb3dzIGEgc2luZ2xlDQo+ICAg ICAgYnlwYXNzIHR1bm5lbCB0byBjYXJyeSB0cmFmZmljIGZvciBtdWx0aXBsZSBQV3MuDQo+DQo+ IFNCPiBPSywgYnV0IGRpZG4ndCBJIHJlYWQgc29tZXRoaW5nIGFib3V0IFBFIHByb3RlY3Rpbmcg c3Vic2V0cyBvZiB0aGUNCj4gZWdyZXNzIHRyYWZmaWMgb2Ygb3RoZXIgUEVzPyBIb3cgd291bGQg dGhhdCB3b3JrIHdpdGhvdXQgcGVyIFBXIGluZm9ybWF0aW9uPw0KPg0KPiBbeXNoZW5dIFBXcyB0 byBhIGdpdmVuIFBFIG1heSBiZSBwcm90ZWN0ZWQgYnkgbXVsdGlwbGUgcHJvdGVjdG9ycy4gQSBz ZXQgb2YgUFdzIHNoYXJpbmcgdGhlIHNhbWUgcHJvdGVjdG9yIFggY2FuIGJlIGNhcnJpZWQgYnkg dGhlIHNhbWUgdHVubmVsIChkZXN0aW5lZCB0byB0aGUgY29udGV4dCBJRCBvZiA8cHJpbWFyeSBQ RSwgWD4pLiBQV3MgcHJvdGVjdGVkIGJ5IGEgZGlmZmVyZW50IHByb3RlY3RvciBZIG11c3QgYmUg Y2FycmllZCBieSBkaWZmZXJlbnQgdHVubmVsIChkZXN0aW5lZCB0byB0aGUgY29udGV4dCBJRCBv ZiA8cHJpbWFyeSBQRSwgWT4pLiBUaGUgUExSIG9mIGVhY2ggdHVubmVsIGRvZXNuJ3QgbmVlZCB0 byBrbm93IFBXIGluZm8uIEFsbCBpdCBkb2VzIGlzIHRvIHN3YXAgdHVubmVsIGxhYmVsIHRvIGEg YnlwYXNzIHR1bm5lbCdzIGxhYmVsLCBhbmQgdGhlIGJ5cGFzcyB0dW5uZWwgd2lsbCBndWlkZSB0 aGUgdHJhZmZpYyB0byB0aGUgaW50ZW5kZWQgcHJvdGVjdG9yLg0KPg0KPiAgICAgIFRoZSBwcm9j ZWR1cmUgYWxzbyByZXF1aXJlcyB0aGF0IHRoZSBwcm90ZWN0b3IgU0hPVUxEIGJlIGFibGUgdG8N Cj4gICAgICBmb3J3YXJkIHRoZSB0cmFmZmljIGJhc2VkIG9uIGEgUFcgbGFiZWwgdGhhdCBpcyBh c3NpZ25lZCBieSB0aGUNCj4gICAgICBwcmltYXJ5IFBFLCBhbmQgZW5zdXJlIHRoZSB0cmFmZmlj IHRvIGV2ZW50dWFsbHkgcmVhY2ggdGhlIHRhcmdldCBDRS4NCj4gICAgICBGcm9tIHRoZSBwcm90 ZWN0b3IncyBwZXJzcGVjdGl2ZSwgdGhpcyBQVyBsYWJlbCBpcyBhbiB1cHN0cmVhbQ0KPiAgICAg IGFzc2lnbmVkIGxhYmVsIChSRkMgNTMzMSkuDQo+DQo+IFNCPiAuLiBidXQgdGhlIHByb3RlY3Rv ciBtYXkgYmUgYSBQIHJvdXRlciBhbmQga25vdyBub3RoaW5nIGFib3V0DQo+IFBXcy4NCj4NCj4g W3lzaGVuXSBBZ2FpbiwgaW4gY29sbG9jYXRlZCBtb2RlbCwgdGhlIHByb3RlY3RvciBpcyBhIFBF LiBJbiB0aGUgY2VudHJhbGl6ZWQgbW9kZWwsIHRoZSBwcm90ZWN0b3IgbWF5IGJlIGFuIExTUiAs IGFuZCBpdCBuZWVkcyB0byBwYXJ0aWNpcGF0ZSBpbiBMRFAgUFcgc2lnbmFsaW5nIHRvIHVuZGVy c3RhbmQgUFdzLiBTZWUgdGhlIHJlbGF0ZWQgc2VjdGlvbnMgYW5kIExEUCBleHRlbnNpb25zIGlu IHRoaXMgZHJhZnQuIEJ1dCB0aGlzIGNlbnRyYWxpemVkIG1vZGVsIGlzIG9wdGlvbmFsLg0KPg0K Pg0KPiAgICAgIFRvIGFjY29tcGxpc2ggdGhpcywgdGhlIHByb3RlY3RvciBTSE9VTEQNCj4gICAg ICBsZWFybmluZyB0aGUgUFcgbGFiZWwgZnJvbSB0aGUgcHJpbWFyeSBQRSBwcmlvciB0byB0aGUg ZmFpbHVyZSwgYW5kDQo+ICAgICAgaW5zdGFsbCBwcm9wZXIgZm9yd2FyZGluZyBzdGF0ZSBmb3Ig dGhlIFBXIGxhYmVsIGluIGEgZGVkaWNhdGVkIGxhYmVsDQo+ICAgICAgc3BhY2UgZm9yIHRoZSBw cmltYXJ5IFBFLg0KPg0KPiBTQj4gVGhpcyBpcyBhIG5ldyBQRSB0byBQIHJvdXRlciBwcm90b2Nv bA0KPg0KPiBbeXNoZW5dIFRoZSBMRFAgZXh0ZW5zaW9uIGlzIGRlZmluZWQgaW4gYSBzdWJzZXF1 ZW50IHNlY3Rpb24uDQo+DQo+ICAgICAgRHVyaW5nIGxvY2FsIHJlcGFpciwgdGhlIHByb3RlY3Rv ciBTSE9VTEQNCj4gICAgICBwZXJmb3JtIFBXIGxhYmVsIGxvb2t1cCBpbiB0aGlzIGxhYmVsIHNw YWNlLg0KPg0KPiBTQj4gVGhpcyBoYXMgdG8gYmUgc2lnbmVkIG9mZiBieSBNUExTIFdHDQo+IFNC PiBIb3cgY2FuIHRoaXMgb25seSBiZSBhIFNIT1VMRCAtIHdoeSBpcyBpdCBub3QgYSBNVVNUPw0K Pg0KPiBbeXNoZW5dIFdpbGwgcmVwaHJhc2UuDQo+DQo+IFNCPiBXaGF0IGRvIHlvdSBkbyB0byBl bnN1cmUgdGhhdCB0aGV5IGV2ZW4gaGF2ZSBjb21wYXRpYmxlIGxhYmVsIG1hbmFnZXJzPw0KPg0K PiBbeXNoZW5dIFRoZSBMRFAgZXh0ZW5zaW9ucyBkZWZpbmVkIGluIHRoaXMgZHJhZnQgY292ZXJz IGhvdyB0byBuZWdvdGlhdGUgdGhlIGNhcGFiaWxpdHkuDQo+DQo+DQo+IFNCPiBEb2Vzbid0IHRo aXMgbWVhbiBhIFAgcm91dGVyIGxvb2tpbmcgdXAgdHdvIGxhYmVscyB0byBkbw0KPiB0aGUgcmVw YWlyPw0KPg0KPiBbeXNoZW5dIFBMUiBvbmx5IHN3YXBzIHRoZSB0b3AgbGFiZWwgKGkuZS4gdHVu bmVsIGxhYmVsKSB0byBieXBhc3MgbGFiZWwgdGFibGUsIHdoZW4gZG9pbmcgdGhlIGxvY2FsIHJl cGFpci4NCj4NCj4gICAgICBUaGUgYWJvdmUgZXhhbXBsZXMgaGF2ZSBzaG93biB0aGUgc2NlbmFy aW9zIHdoZXJlIHRoZSBwcm90ZWN0b3JzIGFyZQ0KPiAgICAgIGJhY2t1cCAoVC9TLSlQRXMuICBJ biBvdGhlciBzY2VuYXJpb3MsIGEgcHJvdGVjdG9yIG1heSBiZSBhIGRlZGljYXRlZA0KPiAgICAg IHJvdXRlciB0aGF0IGFzc3VtZXMgc3VjaCByb2xlLCBzZXBhcmF0ZSBmcm9tIHRoZSBiYWNrdXAg KFQvUy0pUEUgb2YgYQ0KPiAgICAgIHByaW1hcnkgUFcuICBEdXJpbmcgbG9jYWwgcmVwYWlyLCB0 aGUgUExSIE1VU1Qgc3RpbGwgcmVyb3V0ZSB0cmFmZmljDQo+ICAgICAgdG8gdGhlIHByb3RlY3Rv ciB0aHJvdWdoIGEgYnlwYXNzIHR1bm5lbC4gIFRoZSBwcm90ZWN0b3IgTVVTVCBpbiB0dXJuDQo+ ICAgICAgc2VuZCB0aGUgdHJhZmZpYyB0byB0aGUgYmFja3VwIChUL1MtKVBFLCB3aGljaCBNVVNU IHRoZW4gc2VuZCB0aGUNCj4NCj4NCj4NCj4NCj4gWWltaW4gU2hlbiwgZXQgYWwuICAgICAgRXhw aXJlcyBKYW51YXJ5IDI1LCAyMDE1ICAgICAgICAgICAgICAgW1BhZ2UgMTBdDQo+DQo+DQo+IElu dGVybmV0LURyYWZ0ICAgICBQVyBFbmRwb2ludCBGYXN0IEZhaWx1cmUgUHJvdGVjdGlvbiAgICAg ICAgIEp1bHkgMjAxNA0KPg0KPg0KPiAgICAgIHRyYWZmaWMgdG8gdGhlIHRhcmdldCBDRSB2aWEg YSBiYWNrdXAgQUMgb3IgYSBiYWNrdXAgUFcgc2VnbWVudC4NCj4gICAgICBNb3JlIGRldGFpbCB3 aWxsIGJlIGRlc2NyaWJlZCBpbiBTZWN0aW9uIDQuMy4NCj4NCj4gNC4yLiAgQ29udGV4dCBJZGVu dGlmaWVyDQo+DQo+ICAgICAgQSBwcm90ZWN0b3IgTUFZIHByb3RlY3QgbXVsdGlwbGUgcHJpbWFy eSBQRXMuDQo+DQo+IFNCPiBUaGlzIHJlcXVpcmVzIHRoYXQgYSBQIHJvdXRlciBrbm93cyBhYm91 dCBQV3MgLSBob3cgZG9lcw0KPiBpdCBkbyB0aGF0Pw0KPg0KPiBbeXNoZW5dIFRoZSBwcm90ZWN0 b3Igd2lsbCBwZXJmb3JtIGNvbnRleHQgbGFiZWwgZm9yd2FyZGluZy4gVGhlIGJ5cGFzcyB0dW5u ZWwgcG9pbnRzIHRvIGEgY29udGV4dCBsYWJlbCB0YWJsZSwgYW5kIHRoZSBwcm90ZWN0b3IgbG9v a3MgdXAgdGhlIFBXIGxhYmVsIGluIHRoYXQgdGFibGUuDQo+DQo+ICAgICAgVGhlIHByb3RlY3Rv ciBNVVNUDQo+ICAgICAgbWFpbnRhaW4gYSBzZXBhcmF0ZSBsYWJlbCBzcGFjZSBmb3IgZWFjaCBw cmltYXJ5IFBFLiAgTGlrZXdpc2UsIHRoZQ0KPiAgICAgIFBXcyB0ZXJtaW5hdGVkIG9uIGEgcHJp bWFyeSBQRSBNQVkgYmUgcHJvdGVjdGVkIGJ5IG11bHRpcGxlDQo+ICAgICAgcHJvdGVjdG9ycywg ZWFjaCBmb3IgYSBzdWJzZXQgb2YgdGhlIFBXcy4gIEluIGFueSBjYXNlLCBhIGdpdmVuDQo+ICAg ICAgcHJpbWFyeSBQVyBTSE9VTEQgYmUgYXNzb2NpYXRlZCB3aXRoIG9uZSBhbmQgb25seSBvbmUg cGFpciBvZg0KPiAgICAgIHtwcmltYXJ5IFBFLCBwcm90ZWN0b3J9Lg0KPg0KPiAgICAgIEFuIElQ djQvdjYgYWRkcmVzcyBpcyBhc3NpZ25lZCB0byBlYWNoIG9yZGVyZWQgcGFpciBvZiB7cHJpbWFy eSBQRSwNCj4gICAgICBwcm90ZWN0b3J9IHRvIGZhY2lsaXRhdGUgcHJvdGVjdGlvbiBlc3RhYmxp c2htZW50LiAgVGhpcyBhZGRyZXNzIGlzDQo+ICAgICAgcmVmZXJyZWQgdG8gYXMgYSAiY29udGV4 dCBpZGVudGlmaWVyIi4gIEl0IE1VU1QgYmUgZ2xvYmFsbHkgdW5pcXVlLA0KPiAgICAgIG9yIHVu aXF1ZSBpbiB0aGUgYWRkcmVzcyBzcGFjZSBvZiB0aGUgbmV0d29yayB3aGVyZSB0aGUgcHJpbWFy eSBQRQ0KPiAgICAgIGFuZCB0aGUgcHJvdGVjdG9yIHJlc2lkZS4NCj4NCj4gU0I+IEkgdGhpbmsg eW91IG5lZWQgdG8gbWFrZSBjbGVhciB0aGF0IHRoaXMgaXMgbm90IGJhY2t3YXJkcyBjb21wYXRp YmxlDQo+IHdpdGggZXhpc3RpbmcgUFcgUEVzLg0KPg0KPiBbeXNoZW5dIFBFcyB3aWxsIG5lZWQg dG8gc3VwcG9ydCB0aGUgZXh0ZW5zaW9ucyBkZWZpbmVkIGJ5IHRoaXMgZHJhZnQuDQo+DQo+IFNC PiBZb3UgYWxzbyBuZWVkIHRvIGV4cGxhaW4gaG93IHRoZSBJUCBiYXNlZCBjb250ZXh0IGlkZW50 aWZpZXINCj4gbWVjaGFuaXNtIHdvcmtzLiBJcyBpdCBpbiB0aGUgY29udHJvbCBwbGFuZSBvciB0 aGUgZGF0YSBwbGFuZT8NCj4NCj4gNC4yLjEuICBTZW1hbnRpY3MNCj4NCj4gICAgICBUaGUgc2Vt YW50aWNzIG9mIGEgY29udGV4dCBpZGVudGlmaWVyIGlzIHR3b2ZvbGQuDQo+DQo+ICAgICAgbyAg SXQgaWRlbnRpZmllcyBhIHByaW1hcnkgUEUgYW5kIGFuIGFzc29jaWF0ZWQgcHJvdGVjdG9yLiAg SW4gb3RoZXINCj4gICAgICAgICB3b3JkcywgaXQgaWRlbnRpZmllcyBhIHByaW1hcnkgUEUgb24g YSBwZXIgcHJvdGVjdG9yIGJhc2lzLiAgQQ0KPiAgICAgICAgIGdpdmVuIHByaW1hcnkgUEUgbWF5 IGJlIHByb3RlY3RlZCBieSBtdWx0aXBsZSBwcm90ZWN0b3JzLCBlYWNoIGZvcg0KPiAgICAgICAg IGEgc3Vic2V0IG9mIHRoZSBwcmltYXJ5IFBXcyB0ZXJtaW5hdGVkIG9uIHRoZSBwcmltYXJ5IFBF LiAgQQ0KPiAgICAgICAgIGRpc3RpbmN0IGNvbnRleHQgaWRlbnRpZmllciBNVVNUIGJlIGFzc2ln bmVkIHRvIHRoZSBwcmltYXJ5IFBFIGFuZA0KPiAgICAgICAgIGVhY2ggcHJvdGVjdG9yLg0KPg0K PiAgICAgICAgIEZvciBlYWNoIHByaW1hcnkgUFcsIGl0cyBpbmdyZXNzIFBFIE1VU1Qgc2V0IHVw IG9yIHJlc29sdmUgYQ0KPiAgICAgICAgIHRyYW5zcG9ydCB0dW5uZWwgd2l0aCBkZXN0aW5hdGlv biBhcyB0aGUgY29udGV4dCBpZGVudGlmaWVyIG9mIHRoZQ0KPiAgICAgICAgIHtwcmltYXJ5IFBF LCBwcm90ZWN0b3J9LCByYXRoZXIgdGhhbiBhIHByaXZhdGUgSVAgYWRkcmVzcyBvZiB0aGUNCj4g ICAgICAgICBwcmltYXJ5IFBFLiAgVGhpcyBub3Qgb25seSBhbGxvd3MgdGhlIHRyYW5zcG9ydCB0 dW5uZWwgdG8gcmVhY2gNCj4gICAgICAgICB0aGUgcHJpbWFyeSBQRSwgYnV0IGFsc28gY29udmV5 cyB0aGUgaWRlbnRpdHkgb2YgdGhlIHByb3RlY3RvciB0bw0KPiAgICAgICAgIHRoZSBQTFIocykg YWxvbmcgdGhlIHRyYW5zcG9ydCB0dW5uZWwuICBFYWNoIFBMUiBjYW4gaW4gdHVybiB1c2UNCj4g ICAgICAgICB0aGlzIGluZm9ybWF0aW9uIHRvIHNldCB1cCBhIGJ5cGFzcyB0dW5uZWwgdG8gdGhl IHByb3RlY3Rvcg0KPiAgICAgICAgIHdpdGhvdXQgcmVseWluZyBvbiBsb2NhbCBjb25maWd1cmF0 aW9uLg0KPg0KPiAgICAgIG8gIEl0IGluZGVudGlmaWVzIHRoZSBwcmltYXJ5IFBFJ3MgbGFiZWwg c3BhY2Ugb24gdGhlIHByb3RlY3Rvci4gIFRoZQ0KPiAgICAgICAgIHByb3RlY3RvciBtYXkgcHJv dGVjdCBQV3MgZm9yIG11bHRpcGxlIHByaW1hcnkgUEVzLiAgRm9yIGVhY2gNCj4gICAgICAgICBw cmltYXJ5IFBFLCBpdCBNVVNUIG1haW50YWluIGEgc2VwYXJhdGUgbGFiZWwgc3BhY2UgdG8gc3Rv cmUgdGhlDQo+ICAgICAgICAgUFcgbGFiZWxzIGFzc2lnbmVkIGJ5IHRoYXQgcHJpbWFyeSBQRS4g IEl0IE1VU1QgYXNzb2NpYXRlIGEgUFcNCj4gICAgICAgICBsYWJlbCB3aXRoIGEgbGFiZWwgc3Bh Y2UgdmlhIHRoZSBjb250ZXh0IGlkZW50aWZpZXIgb2YgdGhlDQo+ICAgICAgICAge3ByaW1hcnkg UEUsIHByb3RlY3Rvcn0sIGFzIGRlc2NyaWJlZCBiZWxvdy4NCj4NCj4gU0I+IElzIHRoaXMgY29u dGV4dCBpZGVudGlmaWVyIGFuIElQIGFkZHJlc3Mgb3IgYW4gTVBMUyBsYWJlbD8gV2hlcmUNCj4g aXMgaXQgY2FycmllZD8NCj4NCj4gW3lzaGVuXSBhIGNvbnRleHQgaWRlbnRpZmllciBpcyBhbiBJ UCBhZGRyZXNzLiBJZiB0aGUgdHVubmVsIGlzIElQIHR1bm5lbCwgaXQgaXMgY2FycmllZCBpbiBJ UCBoZWFkZXIgYXMgZGVzdCBhZGRyZXNzLiBJZiB0aGUgdHVubmVsIGlzIE1QTFMgdHVubmVsLCBp dCBpcyBpbmRpY2F0ZWQgYnkgdGhlIGxhYmVsLg0KPg0KPiAgICAgICAgIEluIGFkZGl0aW9uIHRv IHRoZSBub3JtYWwgTERQIFBXIHNpZ25hbGluZywgdGhlIHByaW1hcnkgUEUgTVVTVA0KPiAgICAg ICAgIGhhdmUgYSB0YXJnZXRlZCBMRFAgc2Vzc2lvbiB3aXRoIHRoZSBwcm90ZWN0b3IsIGFuZCBh ZHZlcnRpc2UgUFcNCj4gICAgICAgICBsYWJlbHMgdG8gdGhlIHByb3RlY3RvciB2aWEgTERQIExh YmVsIE1hcHBpbmcgbWVzc2FnZXMgKFNlZQ0KPg0KPg0KPg0KPiBZaW1pbiBTaGVuLCBldCBhbC4g ICAgICBFeHBpcmVzIEphbnVhcnkgMjUsIDIwMTUgICAgICAgICAgICAgICBbUGFnZSAxMV0NCj4N Cj4NCj4gSW50ZXJuZXQtRHJhZnQgICAgIFBXIEVuZHBvaW50IEZhc3QgRmFpbHVyZSBQcm90ZWN0 aW9uICAgICAgICAgSnVseSAyMDE0DQo+DQo+DQo+ICAgICAgICAgU2VjdGlvbiA1IGZvciBkZXRh aWwpLiAgVGhlIHByaW1hcnkgUEUgTVVTVCBhdHRhY2ggdGhlIGNvbnRleHQNCj4gICAgICAgICBp ZGVudGlmaWVyIHRvIGVhY2ggbWVzc2FnZS4gIFVwb24gcmVjZWl2aW5nIHRoZSBtZXNzYWdlLCB0 aGUNCj4gICAgICAgICBwcm90ZWN0b3IgTVVTVCBpbnN0YWxsIHRoZSBhZHZlcnRpc2VkIFBXIGxh YmVsIGluIHRoZSBsYWJlbCBzcGFjZQ0KPiAgICAgICAgIGlkZW50aWZpZWQgYnkgdGhlIGNvbnRl eHQgaWRlbnRpZmllci4NCj4NCj4gU0I+IEkgYW0gY3VycmVudGx5IGNvbmZ1c2VkIGFib3V0IGhv dyBhIGRhdGEgcGFja2V0IGNhcnJpZXMgdGhlIFBXDQo+IGxhYmVsIHByb3ZpZGVkIGJ5IGFub3Ro ZXIgUEUuDQo+DQo+ICAgICAgICAgV2hlbiBhIFBMUiBzZXRzIHVwIG9yIHJlc29sdmUgYSBieXBh c3MgdHVubmVsIHRvIHRoZSBwcm90ZWN0b3IsIGl0DQo+ICAgICAgICAgTVVTVCBzZXQgdGhlIGRl c3RpbmF0aW9uIHRvIHRoZSBjb250ZXh0IGlkZW50aWZpZXIsIHJhdGhlciB0aGFuIGENCj4gICAg ICAgICBwcml2YXRlIElQIGFkZHJlc3Mgb2YgdGhlIHByb3RlY3Rvci4gIE9uY2UgZXN0YWJsaXNo ZWQsIHRoZSBieXBhc3MNCj4gICAgICAgICB0dW5uZWwsIHdpdGggZWl0aGVyIGl0cyBNUExTIGxh YmVsIG9yIElQIHR1bm5lbCBkZXN0aW5hdGlvbg0KPiAgICAgICAgIGFkZHJlc3MsIGlzIHVzZWQg YXMgdGhlIGlkZW50aWZpZXIgb2YgbGFiZWwgc3BhY2UuICBPbiB0aGUNCj4gICAgICAgICBwcm90 ZWN0b3IsIGFsbCBQVyBwYWNrZXRzIHJlY2VpdmVkIG9uIHRoZSBieXBhc3MgdHVubmVsIE1VU1Qg YmUNCj4gICAgICAgICBmb3J3YXJkZWQgYmFzZWQgb24gYSBsYWJlbCBsb29rdXAgaW4gdGhhdCBs YWJlbCBzcGFjZS4NCj4NCj4gU0I+IEkgZG8gbm90IHVuZGVyc3RhbmQgdGhlIGRhdGEgcGxhbmUg c3RydWN0dXJlIHRoYXQgdGhlIGFib3ZlIHNlbnRlbmNlDQo+IGlzIGRlc2NyaWJpbmcuDQo+DQo+ IFt5c2hlbl0gVGhpcyBpcyBiYXNpY2FsbHkgdGhlIGNvbnRleHQgbGFiZWwgZm9yd2FyZGluZyBw YXJhZGlnbSAoUkZDIDUzMzEpLiBGcm9tIHRoZSBwcm90ZWN0b3IncyBwZXJzcGVjdGl2ZSwgdGhl IGJ5cGFzcyB0dW5uZWwgc2VydmVzIGFzIGEgY29udGV4dCwgYW5kIGluZGljYXRlcyB0aGUgbGFi ZWwgc3BhY2Ugb2YgdGhlIHByaW1hcnkgUEUgd2hlcmUgdGhlIFBXIGxhYmVsIG11c3QgYmUgbG9v a2VkIHVwLg0KPg0KPg0KPiA0LjIuMi4gIEFkdmVydGlzZW1lbnQgYW5kIFBhdGggQ29tcHV0YXRp b24NCj4NCj4gICAgICBVc2luZyBhIGNvbnRleHQgaWRlbnRpZmllciBhcyBkZXN0aW5hdGlvbiBm b3IgYm90aCB0cmFuc3BvcnQgdHVubmVsDQo+ICAgICAgYW5kIGJ5cGFzcyB0dW5uZWwgcmVxdWly ZXMgYm90aCB0aGUgcHJpbWFyeSBQRSBhbmQgdGhlIHByb3RlY3RvciB0bw0KPiAgICAgIGFkdmVy dGlzZSB0aGUgY29udGV4dCBpZGVudGlmaWVyIHZpYSBJR1AgYXMgYW4gSVAgYWRkcmVzcyByZWFj aGFibGUNCj4gICAgICB0aHJvdWdoIGJvdGggcm91dGVycyBpbiByb3V0aW5nIGRvbWFpbiBhbmQv b3IgVEUgZG9tYWluLiAgVGhpcw0KPiAgICAgIGltcG9zZXMgdGhlIGZvbGxvd2luZyByZXF1aXJl bWVudHMgb24gcGF0aCBjb21wdXRhdGlvbiBmb3IgdGhlc2UNCj4gICAgICB0dW5uZWxzLg0KPg0K PiAgICAgIG8gIEZvciB0aGUgdHJhbnNwb3J0IHR1bm5lbCwgdGhlIGluZ3Jlc3MgUEUgTVVTVCBj aG9vc2UgdGhlIHByaW1hcnkNCj4gICAgICAgICBQRSBhcyB0aGUgYWN0dWFsIGVuZHBvaW50Lg0K Pg0KPiAgICAgIG8gIEZvciB0aGUgYnlwYXNzIHR1bm5lbCwgdGhlIFBMUiBNVVNUIGNob29zZSB0 aGUgcHJvdGVjdG9yIGFzIHRoZQ0KPiAgICAgICAgIGFjdHVhbCBlbmRwb2ludC4gIEluIGVncmVz cyAoVC0pUEUgbm9kZSBwcm90ZWN0aW9uIGFuZCBTLVBFIG5vZGUNCj4gICAgICAgICBwcm90ZWN0 aW9uLCB0aGUgYnlwYXNzIHR1bm5lbCBNVVNUIGF2b2lkIHRoZSBwcmltYXJ5IChTLSlQRS4NCj4N Cj4gICAgICBUaGUgZGV0YWlsIG9mIGhvdyB0aGUgcHJpbWFyeSBQRSBhbmQgdGhlIHByb3RlY3Rv ciBtYXkgYWR2ZXJ0aXNlIGENCj4gICAgICBjb250ZXh0IGlkZW50aWZpZXIgaXMgaW5kZXBlbmRl bnQgb2YgdGhpcyBtZWNoYW5pc20gYW5kIG91dCBvZiB0aGUNCj4gICAgICBzY29wZSBvZiB0aGlz IGRvY3VtZW50Lg0KPg0KPiBTQj4gR2l2ZW4gdGhhdCB5b3UgZ2l2ZSBzaWduYWxsaW5nIGRhdGEg c3RydWN0dXJlcyBsYXRlciwgSSBhbSBjb25mdXNlZA0KPiBhcyB0byBob3cgdGhpcyBjYW4gYmUg b3V0IG9mIHNjb3BlIG9mIHRoaXMgdGV4dC4gU3VyZWx5IHRoaXMgaXMgbmVlZGVkDQo+IHRvIGlt cGxlbWVudCBhbiBpbnRlcm9wZXJhYmxlIHNvbHV0aW9uPw0KPg0KPiBbeXNoZW5dIFRoZXJlIGlz IG5vIG5ldyBleHRlbnNpb24gbmVlZGVkIGZvciBJR1AuIEJvdGggUEVzIGFkdmVydGlzZSB0aGUg Y29udGV4dCBJRC4gVGhhdCdzIGl0LiBXaWxsIHJlcGhyYXNlIGl0Lg0KPg0KPiAgICAgIE9uZSBh cHByb2FjaCB3b3VsZCBiZSB0byBhZHZlcnRpc2UgaXQgYXMgYQ0KPiAgICAgIHZpcnR1YWwgcHJv eHkgbm9kZSBjb25uZWN0ZWQgdG8gYm90aCByb3V0ZXJzLCB3aXRoIHRoZSBsaW5rIGJldHdlZW4N Cj4gICAgICB0aGUgcHJveHkgbm9kZSBhbmQgdGhlIHByaW1hcnkgUEUgaGF2aW5nIGEgbW9yZSBw cmVmZXJhYmxlIElHUCBvciBURQ0KPiAgICAgIG1ldHJpYyB0aGFuIHRoZSBsaW5rIGJldHdlZW4g dGhlIHByb3h5IG5vZGUgYW5kIHRoZSBwcm90ZWN0b3IuICBUaGUNCj4gICAgICB1bHRpbWF0ZSBn b2FsIGlzIGZvciBhIHBhdGggY29tcHV0YXRpb24gYWxnb3JpdGhtLCBzdWNoIGFzIENTUEYNCj4g ICAgICAoY29uc3RyYWluZWQgc2hvcnRlc3QgcGF0aCBmaXJzdCksIExGQSAoUkZDIDUyODYpIGFu ZCBNUlQgKFtJUC1MRFAtDQo+ICAgICAgRlJSLU1SVF0pLCB0byBiZSBhYmxlIHRvIGNvbXB1dGUg dGhlIHBhdGhzIHRoYXQgbWVldCB0aGUgYWJvdmUNCj4gICAgICByZXF1aXJlbWVudHMuDQo+DQo+ IDQuMy4gIFByb3RlY3Rpb24gTW9kZWxzDQo+DQo+ICAgICAgVGhlcmUgYXJlIHR3byBwcm90ZWN0 aW9uIG1vZGVscyBiYXNlZCBvbiB0aGUgbG9jYXRpb24gb2YgYSBwcm90ZWN0b3IuDQo+ICAgICAg QSBuZXR3b3JrIE1BWSB1c2UgZWl0aGVyIG1vZGVsLCBvciBhIGNvbWJpbmF0aW9uIG9mIGJvdGgu DQo+DQo+DQo+DQo+DQo+DQo+DQo+DQo+IFlpbWluIFNoZW4sIGV0IGFsLiAgICAgIEV4cGlyZXMg SmFudWFyeSAyNSwgMjAxNSAgICAgICAgICAgICAgIFtQYWdlIDEyXQ0KPg0KPg0KPiBJbnRlcm5l dC1EcmFmdCAgICAgUFcgRW5kcG9pbnQgRmFzdCBGYWlsdXJlIFByb3RlY3Rpb24gICAgICAgICBK dWx5IDIwMTQNCj4NCj4NCj4gNC4zLjEuICBDby1sb2NhdGVkIFByb3RlY3Rvcg0KPg0KPiAgICAg IEluIHRoaXMgbW9kZWwsIHRoZSBwcm90ZWN0b3IgaXMgYSBiYWNrdXAgUEUgdGhhdCBpcyBkaXJl Y3RseQ0KPiAgICAgIGNvbm5lY3RlZCB0byB0aGUgdGFyZ2V0IENFIHZpYSBhIGJhY2t1cCBBQywg b3IgaXQgaXMgYSBiYWNrdXAgUy1QRSBvbg0KPiAgICAgIGEgYmFja3VwIFBXLiAgVGhhdCBpcywg dGhlIHByb3RlY3RvciBpcyBjby1sb2NhdGVkIHdpdGggdGhlIGJhY2t1cA0KPiAgICAgIChTLSlQ RS4gIEV4YW1wbGVzIG9mIHRoaXMgbW9kZWwgaGF2ZSBiZWVuIGludHJvZHVjZWQgaW4gRmlndXJl IDQsDQo+ICAgICAgRmlndXJlIDUgYW5kIEZpZ3VyZSA2IGluIFNlY3Rpb24gNC4xLg0KPg0KPiAg ICAgIEluIGVncmVzcyBBQyBwcm90ZWN0aW9uIGFuZCBlZ3Jlc3MgUEUgbm9kZSBwcm90ZWN0aW9u LCB3aGVuIGENCj4gICAgICBwcm90ZWN0b3IgcmVjZWl2ZXMgdHJhZmZpYyBmcm9tIHRoZSBQTFIs IGl0IGZvcndhcmRzIHRoZSB0cmFmZmljIHRvDQo+ICAgICAgdGhlIENFIHZpYSB0aGUgYmFja3Vw IEFDLiAgVGhpcyBpcyBzaG93biBpbiBGaWd1cmUgNywgd2hlcmUgUEUyIGlzDQo+ICAgICAgdGhl IFBMUiBmb3IgZWdyZXNzIEFDIGZhaWx1cmUsIFAzIGlzIHRoZSBQTFIgZm9yIFBFMiBmYWlsdXJl LCBhbmQgUEU0DQo+ICAgICAgKHRoZSBiYWNrdXAgUEUpIGlzIHRoZSBwcm90ZWN0b3IuDQo+DQo+ ICAgICAgICAgICAgICAgICAgICB8PC0tLS0tLS0tLS0tLS0tIFBXMSAtLS0tLS0tLS0tLS0tLS0+ fA0KPg0KPiAgICAgICAgICAgICAgICAtIFBFMSAtLS0tLS0tLS0tLS0tLSBQMSAtLS0tLS0tIFAz IC0tLS0tIFBFMiAtLS0tDQo+ICAgICAgICAgICAgICAgLyAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICBQTFIgXCAgICAgUExSICAgICBcDQo+ICAgICAgICAgICAgICAvICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgIFwgICAgIHwgICAgICAgXA0KPiAgICAgICAgICAgQ0Ux ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgYnlwYXNzXCAgICB8YnlwYXNzICBDRTIN Cj4gICAgICAgICAgICAgIFwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBc ICAgfCAgICAgICAvDQo+ICAgICAgICAgICAgICAgXCAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgIFwgIHwgICAgICAvDQo+ICAgICAgICAgICAgICAgIC0gUEUzIC0tLS0tLS0t LS0tLS0tIFAyIC0tLS0tLS0tLS0tLS0tLS0gUEU0IC0tLS0NCj4gICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHByb3RlY3Rvcg0KPg0KPiAgICAgICAgICAg ICAgICAgICAgfDwtLS0tLS0tLS0tLS0tLSBQVzIgLS0tLS0tLS0tLS0tLS0tPnwNCj4NCj4gICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBGaWd1cmUgNw0KPg0KPiAgICAgIEluIFMt UEUgbm9kZSBwcm90ZWN0aW9uLCB3aGVuIGEgcHJvdGVjdG9yIHJlY2VpdmVzIHRyYWZmaWMgZnJv bSB0aGUNCj4gICAgICBQTFIsIGl0IE1VU1QgZm9yd2FyZCB0aGUgdHJhZmZpYyB2aWEgdGhlIG5l eHQgc2VnbWVudCBvZiB0aGUgYmFja3VwDQo+ICAgICAgUFcuICBUaGUgVC1QRSBvZiB0aGUgYmFj a3VwIFBXIE1VU1QgaW4gdHVybiBmb3J3YXJkIHRoZSB0cmFmZmljIHRvDQo+ICAgICAgdGhlIENF IHZpYSBhIGJhY2t1cCBBQy4gIFRoaXMgaXMgc2hvd24gaW4gRmlndXJlIDgsIHdoZXJlIFA0IGlz IHRoZQ0KPiAgICAgIFBMUiBmb3IgU1BFMSBmYWlsdXJlLCBhbmQgU1BFMiAodGhlIGJhY2t1cCBT LVBFKSBpcyB0aGUgcHJvdGVjdG9yIGZvcg0KPiAgICAgIFNQRTEuDQo+DQo+DQo+DQo+DQo+DQo+ DQo+DQo+DQo+DQo+DQo+DQo+DQo+DQo+DQo+DQo+DQo+IFlpbWluIFNoZW4sIGV0IGFsLiAgICAg IEV4cGlyZXMgSmFudWFyeSAyNSwgMjAxNSAgICAgICAgICAgICAgIFtQYWdlIDEzXQ0KPg0KPg0K PiBJbnRlcm5ldC1EcmFmdCAgICAgUFcgRW5kcG9pbnQgRmFzdCBGYWlsdXJlIFByb3RlY3Rpb24g ICAgICAgICBKdWx5IDIwMTQNCj4NCj4NCj4gICAgICAgICAgICAgICAgICAgICB8PC0tLS0tLS0t LS0tLS0tLSBQVzEgLS0tLS0tLS0tLS0tLS0tPnwNCj4gICAgICAgICAgICAgICAgICAgICB8PC0t LS0tIFNFRzEgLS0tLS0+fDwtLS0tLSBTRUcyIC0tLS0tPnwNCj4NCj4gICAgICAgICAgICAgICAg LSBUUEUxIC0tLS0tIFA0ICAtLS0tLSBTUEUxIC0tLS0tLS0tLS0tLS0tIFRQRTIgLQ0KPiAgICAg ICAgICAgICAgIC8gICAgICAgICAgICAgUExSIFwgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgXA0KPiAgICAgICAgICAgICAgLyAgICAgICAgICAgICAgICAgICBcICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgIFwNCj4gICAgICAgICAgIENFMSAgICAgICAgICAgICAgIGJ5cGFzc1wg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgQ0UyDQo+ICAgICAgICAgICAgICBcICAgICAg ICAgICAgICAgICAgICAgXCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgLw0KPiAgICAgICAg ICAgICAgIFwgICAgICAgICAgICAgICAgICAgICBcICAgICAgICAgICAgICAgICAgICAgICAgICAg Lw0KPiAgICAgICAgICAgICAgICAtIFRQRTMgLS0tLS0tLS0tLS0tLS0tIFNQRTIgLS0tLS0tLS0t LS0tLS0gVFBFNCAtDQo+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBwcm90ZWN0b3IN Cj4NCj4gICAgICAgICAgICAgICAgICAgICB8PC0tLS0tIFNFRzMgLS0tLS0+fDwtLS0tLSBTRUc0 IC0tLS0tPnwNCj4gICAgICAgICAgICAgICAgICAgICB8PC0tLS0tLS0tLS0tLS0tLSBQVzIgLS0t LS0tLS0tLS0tLS0tPnwNCj4NCj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBG aWd1cmUgOA0KPg0KPiAgICAgIEluIHRoZSBjby1sb2NhdGVkIHByb3RlY3RvciBtb2RlbCwgdGhl IG51bWJlciBvZiBjb250ZXh0IGlkZW50aWZpZXJzDQo+ICAgICAgbmVlZGVkIGJ5IGEgbmV0d29y ayBpcyB0aGUgbnVtYmVyIG9mIGRpc3RpbmN0IHtwcmltYXJ5IFBFLCBiYWNrdXAgUEV9DQo+ICAg ICAgcGFpcnMuICBGcm9tIHRoZSBwZXJzcGVjdGl2ZSBvZiBzY2FsYWJpbGl0eSwgdGhlIG1vZGVs IGlzIHN1aXRhYmxlDQo+ICAgICAgZm9yIG5ldHdvcmtzIHdoZXJlIHRoZSBudW1iZXIgb2YgYmFj a3VwIFBFcyBmb3IgYW55IGdpdmVuIHByaW1hcnkgUEUNCj4gICAgICBpcyByZWxhdGl2ZWx5IHNt YWxsLg0KPg0KPiA0LjMuMi4gIENlbnRyYWxpemVkIFByb3RlY3Rvcg0KPg0KPiAgICAgIEluIHRo aXMgbW9kZWwsIHRoZSBwcm90ZWN0b3IgaXMgYSBkZWRpY2F0ZWQgUCByb3V0ZXIgb3IgUEUgcm91 dGVyDQo+ICAgICAgdGhhdCBzZXJ2ZXMgdGhlIHJvbGUuICBJbiBlZ3Jlc3MgQUMgcHJvdGVjdGlv biBhbmQgZWdyZXNzIFBFIG5vZGUNCj4gICAgICBwcm90ZWN0aW9uLCB0aGUgcHJvdGVjdG9yIE1B WSBvciBNQVkgTk9UIGJlIGEgYmFja3VwIFBFIHdpdGggYSBkaXJlY3QNCj4gICAgICBjb25uZWN0 aW9uIHRvIHRoZSB0YXJnZXQgQ0UuICBJbiBTLVBFIG5vZGUgcHJvdGVjdGlvbiwgdGhlIHByb3Rl Y3Rvcg0KPiAgICAgIE1BWSBvciBNQVkgTk9UIGJlIGEgYmFja3VwIFMtUEUgb24gdGhlIGJhY2t1 cCBQVy4NCj4NCj4gICAgICBJbiBlZ3Jlc3MgQUMgcHJvdGVjdGlvbiBhbmQgZWdyZXNzIFBFIG5v ZGUgcHJvdGVjdGlvbiwgd2hlbiB0aGUNCj4gICAgICBwcm90ZWN0b3IgcmVjZWl2ZXMgdHJhZmZp YyBmcm9tIHRoZSBQTFIsIGlmIHRoZSBwcm90ZWN0b3IgaGFzIGENCj4gICAgICBkaXJlY3QgY29u bmVjdGlvbiAoaS5lLiBiYWNrdXAgQUMpIHRvIHRoZSBDRSwgaXQgTVVTVCBmb3J3YXJkIHRoZQ0K PiAgICAgIHRyYWZmaWMgdG8gdGhlIENFIHZpYSB0aGUgYmFja3VwIEFDLCB3aGljaCBpcyBzaW1p bGFyIHRvIEZpZ3VyZSA3Lg0KPiAgICAgIE90aGVyd2lzZSwgaXQgTVVTVCBmb3J3YXJkIHRoZSB0 cmFmZmljIHRvIGEgYmFja3VwIFBFLCB3aGljaCBNVVNUDQo+ICAgICAgdGhlbiBmb3J3YXJkIHRo ZSB0cmFmZmljIHRvIHRoZSBDRSB2aWEgYSBiYWNrdXAgQUMuICBUaGlzIGlzIHNob3duIGluDQo+ ICAgICAgRmlndXJlIDksIHdoZXJlIHRoZSBwcm90ZWN0b3IgcmVjZWl2ZXMgdHJhZmZpYyBmcm9t IFAzICh0aGUgUExSIGZvcg0KPiAgICAgIGVncmVzcyBQRSBmYWlsdXJlKSBvciBQRTIgKHRoZSBQ TFIgZm9yIGVncmVzcyBBQyBmYWlsdXJlKSBhbmQNCj4gICAgICBmb3J3YXJkcyB0aGUgdHJhZmZp YyB0byBQRTQgKHRoZSBiYWNrdXAgUEUpLiAgVGhlIHByb3RlY3RvciBtYXkgYmUNCj4gICAgICBw cm90ZWN0aW5nIG90aGVyIFBXcyBhcyB3ZWxsLCB3aGljaCBpcyBub3Qgc2hvd24gaW4gdGhpcyBm aWd1cmUgZm9yDQo+ICAgICAgY2xhcml0eS4NCj4NCj4gU0I+IElmIHlvdSBhcmUgZ29pbmcgdG8g Y292ZXIgdGhlIHNoYXJlZCBjYXNlLCBJIHRoaW5rIHlvdSBuZWVkIHRvDQo+IG1ha2UgaXQgbXVj aCBjbGVhcmVyIGhvdyB0aGUgUFdzIGFyZSBpZGVudGlmaWVkIGZyb20gdGhlIHJlY2VpdmVkDQo+ IGRhdGEgcGFja2V0cy4NCj4NCj4NCj4NCj4NCj4NCj4NCj4NCj4NCj4gWWltaW4gU2hlbiwgZXQg YWwuICAgICAgRXhwaXJlcyBKYW51YXJ5IDI1LCAyMDE1ICAgICAgICAgICAgICAgW1BhZ2UgMTRd DQo+DQo+DQo+IEludGVybmV0LURyYWZ0ICAgICBQVyBFbmRwb2ludCBGYXN0IEZhaWx1cmUgUHJv dGVjdGlvbiAgICAgICAgIEp1bHkgMjAxNA0KPg0KPg0KPiAgICAgICAgICAgICAgICAgICAgIHw8 LS0tLS0tLS0tLS0tLSBQVzEgLS0tLS0tLS0tLS0tLS0tPnwNCj4NCj4gICAgICAgICAgICAgICAg IC0gUEUxIC0tLS0tLS0tLS0tLS0gUDEgLS0tLS0tLSBQMyAtLS0tLSBQRTIgLS0NCj4gICAgICAg ICAgICAgICAgLyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIFBMUiBcICAgICBQTFIgICBc DQo+ICAgICAgICAgICAgICAgLyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIFwg ICAgIC8gICAgIFwNCj4gICAgICAgICAgICAgIC8gICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgIGJ5cGFzc1wgICAvYnlwYXNzIFwNCj4gICAgICAgICAgICAgLyAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICBcIC8gICAgICAgICBcDQo+ICAgICAgICAgIENFMSAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgcHJvdGVjdG9yICAgICAgIENFMg0KPiAg ICAgICAgICAgICBcICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBcICAg ICAgICAgIC8NCj4gICAgICAgICAgICAgIFwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgIFwgICAgICAgIC8NCj4gICAgICAgICAgICAgICBcICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICBcICAgICAgLw0KPiAgICAgICAgICAgICAgICBcICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBcICAgIC8NCj4gICAgICAgICAgICAg ICAgIC0gUEUzIC0tLS0tLS0tLS0tLS0gUDIgLS0tLS0tLS0tLS0tLS0tLS1QRTQgLS0NCj4NCj4g ICAgICAgICAgICAgICAgICAgICB8PC0tLS0tLS0tLS0tLS0gUFcyIC0tLS0tLS0tLS0tLS0tLT58 DQo+DQo+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgRmlndXJlIDkNCj4NCj4g ICAgICBJbiBTLVBFIG5vZGUgcHJvdGVjdGlvbiwgd2hlbiB0aGUgcHJvdGVjdG9yIHJlY2VpdmVz IHRyYWZmaWMgZnJvbSB0aGUNCj4gICAgICBQTFIsIGlmIHRoZSBwcm90ZWN0b3IgaXMgYSBiYWNr dXAgUy1QRSBvZiB0aGUgYmFja3VwIFBXLCBpdCBNVVNUDQo+ICAgICAgZm9yd2FyZCB0aGUgdHJh ZmZpYyB2aWEgdGhlIG5leHQgc2VnbWVudCBvZiB0aGUgYmFja3VwIFBXLCBhbmQgdGhlDQo+ICAg ICAgVC1QRSBvZiB0aGUgYmFja3VwIFBXIE1VU1QgZm9yd2FyZCB0aGUgdHJhZmZpYyB0byB0aGUg Q0UgdmlhIGEgYmFja3VwDQo+ICAgICAgQUMsIHdoaWNoIGlzIHNpbWlsYXIgdG8gRmlndXJlIDgu ICBPdGhlcndpc2UsIHRoZSBwcm90ZWN0b3IgTVVTVA0KPiAgICAgIGZpcnN0IGZvcndhcmQgdGhl IHRyYWZmaWMgdG8gdGhlIGJhY2t1cCBTLVBFLCB3aGljaCBNVVNUIHRoZW4gZm9yd2FyZA0KPiAg ICAgIHRoZSB0cmFmZmljIHZpYSB0aGUgbmV4dCBzZWdtZW50IG9mIHRoZSBiYWNrdXAgUFcuICBG aW5hbGx5LCB0aGUgVC1QRQ0KPiAgICAgIG9mIHRoZSBiYWNrdXAgUFcgTVVTVCBmb3J3YXJkIHRo ZSB0cmFmZmljIHRvIHRoZSBDRSB2aWEgYSBiYWNrdXAgQUMuDQo+ICAgICAgVGhpcyBpcyBzaG93 biBpbiBGaWd1cmUgMTAsIHdoZXJlIHRoZSBwcm90ZWN0b3IgZm9yd2FyZHMgdHJhZmZpYyB0bw0K PiAgICAgIFNQRTIgKHRoZSBiYWNrdXAgUy1QRSksIFNQRTIgZm9yd2FyZHMgdGhlIHRyYWZmaWMg dG8gVFBFNCB2aWEgU0VHNCwNCj4gICAgICBhbmQgVFBFNCBmaW5hbGx5IGZvcndhcmRzIHRyYWZm aWMgdG8gQ0UyLiAgVGhlIHByb3RlY3RvciBtYXkgYmUNCj4gICAgICBwcm90ZWN0aW5nIG90aGVy IFBXIHNlZ21lbnRzIGFzIHdlbGwsIHdoaWNoIGlzIG5vdCBzaG93biBpbiB0aGlzDQo+ICAgICAg ZmlndXJlIGZvciBjbGFyaXR5Lg0KPg0KPg0KPg0KPg0KPg0KPg0KPg0KPg0KPg0KPg0KPg0KPg0K Pg0KPg0KPg0KPg0KPg0KPg0KPg0KPg0KPiBZaW1pbiBTaGVuLCBldCBhbC4gICAgICBFeHBpcmVz IEphbnVhcnkgMjUsIDIwMTUgICAgICAgICAgICAgICBbUGFnZSAxNV0NCj4NCj4NCj4gSW50ZXJu ZXQtRHJhZnQgICAgIFBXIEVuZHBvaW50IEZhc3QgRmFpbHVyZSBQcm90ZWN0aW9uICAgICAgICAg SnVseSAyMDE0DQo+DQo+DQo+ICAgICAgICAgICAgICAgICAgICAgfDwtLS0tLS0tLS0tLS0tLS0g UFcxIC0tLS0tLS0tLS0tLS0tLT58DQo+ICAgICAgICAgICAgICAgICAgICAgfDwtLS0tLSBTRUcx IC0tLS0tPnw8LS0tLS0gU0VHMiAtLS0tLT58DQo+DQo+ICAgICAgICAgICAgICAgIC0gVFBFMSAt LS0tLSBQNCAgLS0tLS0gU1BFMSAtLS0tLS0tLS0tLS0tLSBUUEUyIC0NCj4gICAgICAgICAgICAg ICAvICAgICAgICAgICAgIFBMUiBcICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIFwNCj4g ICAgICAgICAgICAgIC8gICAgICAgICAgICAgICAgICAgXCAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICBcDQo+ICAgICAgICAgICAgIC8gICAgICAgICAgICAgICBieXBhc3NcICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgIFwNCj4gICAgICAgICAgICAvICAgICAgICAgICAgICAgICAg ICAgICBcICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIFwNCj4gICAgICAgICBDRTEgICAg ICAgICAgICAgICAgICAgICBwcm90ZWN0b3IgICAgICAgICAgICAgICAgICAgICAgICAgICBDRTIN Cj4gICAgICAgICAgICBcICAgICAgICAgICAgICAgICAgICAgICAgXCAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgIC8NCj4gICAgICAgICAgICAgXCAgICAgICAgICAgICAgICAgICAgICAgIFwg ICAgICAgICAgICAgICAgICAgICAgICAgICAgLw0KPiAgICAgICAgICAgICAgXCAgICAgICAgICAg ICAgICAgICAgICAgIFwgICAgICAgICAgICAgICAgICAgICAgICAgIC8NCj4gICAgICAgICAgICAg ICBcICAgICAgICAgICAgICAgICAgICAgICAgXCAgICAgICAgICAgICAgICAgICAgICAgIC8NCj4g ICAgICAgICAgICAgICAgLSBUUEUzIC0tLS0tLS0tLS0tLS0tLSBTUEUyIC0tLS0tLS0tLS0tLS0t IFRQRTQgLQ0KPg0KPiAgICAgICAgICAgICAgICAgICAgIHw8LS0tLS0gU0VHMyAtLS0tLT58PC0t LS0tIFNFRzQgLS0tLS0+fA0KPiAgICAgICAgICAgICAgICAgICAgIHw8LS0tLS0tLS0tLS0tLS0t IFBXMiAtLS0tLS0tLS0tLS0tLS0+fA0KPg0KPiAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgIEZpZ3VyZSAxMA0KPg0KPiAgICAgIFRoZSBjZW50cmFsaXplZCBwcm90ZWN0b3IgbW9k ZWwgcHJvdmlkZXMgdGhlIGNvbnZlbmllbmNlIGZvciBtdWx0aXBsZQ0KPiAgICAgIHByaW1hcnkg UEVzIHRvIHNoYXJlIG9uZSBwcm90ZWN0b3IuICBFYWNoIHByaW1hcnkgUEUgTUFZIG9ubHkgbmVl ZA0KPiAgICAgIHRoZSBvbmUgcHJvdGVjdG9yIHRvIHByb3RlY3QgYWxsIG9mIGl0cyBQV3MuICBG cm9tIHRoZSBwZXJzcGVjdGl2ZSBvZg0KPiAgICAgIHNjYWxhYmlsaXR5LCB0aGUgbnVtYmVyIG9m IGNvbnRleHQgaWRlbnRpZmllcnMgbmVlZGVkIGJ5IGEgbmV0d29yaw0KPiAgICAgIE1BWSBiZSBi b3VuZCB0byB0aGUgbnVtYmVyIG9mIHByaW1hcnkgUEVzLg0KPg0KPiA0LjQuICBUcmFuc3BvcnQg VHVubmVsDQo+DQo+ICAgICAgVGhlIGluZ3Jlc3MgUEUgb2YgYSBwcmltYXJ5IFBXIGFzc29jaWF0 ZXMgdGhlIFBXIHdpdGggdGhlIHByaW1hcnkNCj4gICAgICBlZ3Jlc3MgUEUgdGhyb3VnaCBMRFAg c2lnbmFsaW5nLiAgSW4gYWRkaXRpb24sIGFzIG1lbnRpb25lZCBpbg0KPiAgICAgIFNlY3Rpb24g NC4yLjEsIHRoZSBpbmdyZXNzIFBFIE1VU1QgYXNzb2NpYXRlIHRoZSB0cmFuc3BvcnQgdHVubmVs IG9mDQo+ICAgICAgdGhlIFBXIHdpdGggdGhlIGNvbnRleHQgaWRlbnRpZmllciBvZiB0aGUge3By aW1hcnkgUEUsIHByb3RlY3Rvcn0sDQo+ICAgICAgYW5kIHNldCB1cCBvciByZXNvbHZlIHRoZSB0 cmFuc3BvcnQgdHVubmVsIGJ5IHVzaW5nIHRoZSBjb250ZXh0DQo+ICAgICAgaWRlbnRpZmllciBh cyBkZXN0aW5hdGlvbi4gIFRoaXMgbm90IG9ubHkgZW5zdXJlcyB0aGF0IFBXIHRyYWZmaWMgYmUN Cj4gICAgICB0cmFuc3BvcnRlZCB0byB0aGUgcHJpbWFyeSBQRSwgYnV0IGFsc28gZmFjaWxpdGF0 ZXMgYnlwYXNzIHR1bm5lbA0KPiAgICAgIGVzdGFibGlzaG1lbnQgYXQgUExSKHMpLCBhcyB0aGUg Y29udGV4dCBpZGVudGlmaWVyIGltcGxpZXMgdGhlDQo+ICAgICAgaWRlbnRpdHkgb2YgdGhlIHBy b3RlY3RvciBhcyB3ZWxsLiAgVGhpcyBpcyBhbHNvIHRoZSBjYXNlIGZvciBhDQo+ICAgICAgbXVs dGktc2VnbWVudCBQVywgd2hlcmUgdGhlIGluZ3Jlc3MgUEUgYW5kIGVncmVzcyBQRSBhcmUgVC9T LVBFcy4NCj4NCj4gICAgICBUaGUgYXNzb2NpYXRpb24gYmV0d2VlbiB0aGUgdHJhbnNwb3J0IHR1 bm5lbCBhbmQgdGhlIGNvbnRleHQNCj4gICAgICBpZGVudGlmaWVyIGF0IHRoZSBpbmdyZXNzIFBF IE1BWSBiZSBhY2hpZXZlZCBieSBjb25maWd1cmF0aW9uIG9yIGFuDQo+ICAgICAgYXV0by1kaXNj b3ZlcnkgbWVjaGFuaXNtLiAgSW4gdGhlIGxhdGVyIGNhc2UsIHRoZSBpbmdyZXNzIFBFIE1BWQ0K PiAgICAgIGxlYXJuIHRoZSBjb250ZXh0IGlkZW50aWZpZXIgZnJvbSB0aGUgcHJpbWFyeSAoZWdy ZXNzKSBQRSwgaWYgdGhlDQo+ICAgICAgcHJpbWFyeSBQRSBhZHZlcnRpc2VzIHRoZSBjb250ZXh0 IGlkZW50aWZpZXIgYXMgInRoaXJkIHBhcnR5IG5leHQNCj4gICAgICBob3AiIGluIElQdjQvdjYg SW50ZXJmYWNlX0lEICAoUkZDIDM0NzEsIFJGQyAzNDcyKSBpbiB0aGUgTERQDQo+ICAgICAgTGFi ZWwgTWFwcGluZyBtZXNzYWdlIG9mIHRoZSBwcmltYXJ5IFBXLg0KPg0KPg0KPg0KPg0KPg0KPiBZ aW1pbiBTaGVuLCBldCBhbC4gICAgICBFeHBpcmVzIEphbnVhcnkgMjUsIDIwMTUgICAgICAgICAg ICAgICBbUGFnZSAxNl0NCj4NCj4NCj4gSW50ZXJuZXQtRHJhZnQgICAgIFBXIEVuZHBvaW50IEZh c3QgRmFpbHVyZSBQcm90ZWN0aW9uICAgICAgICAgSnVseSAyMDE0DQo+DQo+DQo+IDQuNS4gIEJ5 cGFzcyBUdW5uZWwNCj4NCj4gICAgICBBIFBMUiBtYXkgcHJvdGVjdCBtdWx0aXBsZSBQV3MgYXNz b2NpYXRlZCB3aXRoIG9uZSBvciBtdWx0aXBsZSBwYWlycw0KPiAgICAgIG9mIHtwcmltYXJ5IFBF LCBwcm90ZWN0b3J9LiBUaGUgUExSIE1VU1QgZXN0YWJsaXNoIGEgYnlwYXNzIHR1bm5lbCB0bw0K PiAgICAgIGVhY2ggcHJvdGVjdG9yIGZvciBlYWNoIGRpc3RpbmN0IGNvbnRleHQgaWRlbnRpZmll ciBhc3NvY2lhdGVkIHdpdGgNCj4gICAgICB0aGF0IHByb3RlY3Rvci4gIFRoZSBkZXN0aW5hdGlv biBvZiB0aGUgYnlwYXNzIHR1bm5lbCBNVVNUIGJlIHRoZQ0KPiAgICAgIGNvbnRleHQgaWRlbnRp ZmllciAoU2VjdGlvbiA0LjIuMSkuICBUaGUgUExSIG1heSBkZXJpdmUgdGhlIGNvbnRleHQNCj4g ICAgICBpZGVudGlmaWVyIGZyb20gdGhlIGRlc3RpbmF0aW9uIG9mIHRoZSB0cmFuc3BvcnQgdHVu bmVsIHRoYXQNCj4gICAgICB0cmF2ZXJzZXMgaXQuDQo+DQo+ICAgICAgRm9yIGV4YW1wbGVzLCBp biBGaWd1cmUgNyBhbmQgRmlndXJlIDksIGEgYnlwYXNzIHR1bm5lbCBpcw0KPiAgICAgIGVzdGFi bGlzaGVkIGZyb20gUEUyIChQTFIgZm9yIGVncmVzcyBBQyBmYWlsdXJlKSB0byB0aGUgcHJvdGVj dG9yLA0KPiAgICAgIGFuZCBhbm90aGVyIGJ5cGFzcyB0dW5uZWwgaXMgZXN0YWJsaXNoZWQgZnJv bSBQMyAoUExSIGZvciBlZ3Jlc3Mgbm9kZQ0KPiAgICAgIGZhaWx1cmUpIHRvIHRoZSBwcm90ZWN0 b3IuICBJbiBGaWd1cmUgOCBhbmQgRmlndXJlIDEwLCBhIGJ5cGFzcw0KPiAgICAgIHR1bm5lbCBp cyBlc3RhYmxpc2hlZCBmcm9tIFA0IChQTFIgZm9yIFMtUEUgZmFpbHVyZSkgdG8gdGhlDQo+ICAg ICAgcHJvdGVjdG9yLg0KPg0KPiAgICAgIER1cmluZyBsb2NhbCByZXBhaXIsIHRoZSBQTFIgcmVy b3V0ZXMgdHJhZmZpYyB0byB0aGUgcHJvdGVjdG9yDQo+ICAgICAgdGhyb3VnaCBhIGJ5cGFzcyB0 dW5uZWwgd2l0aCBQVyBsYWJlbCBpbnRhY3QgaW4gdGhlIHBhY2tldHMuICBUaGlzDQo+ICAgICAg bm9ybWFsbHkgaW52b2x2ZXMgcHVzaGluZyBhIGxhYmVsIHRvIHRoZSBsYWJlbCBzdGFjaywgaWYg dGhlIGJ5cGFzcw0KPiAgICAgIHR1bm5lbCBpcyBhbiBNUExTIHR1bm5lbCwgb3IgcHVzaGluZyBh biBJUCBoZWFkZXIgdG8gdGhlIHBhY2tldHMsIGlmDQo+ICAgICAgdGhlIGJ5cGFzcyB0dW5uZWwg aXMgYW4gSVAgdHVubmVsLg0KPg0KPiBTQj4gU3VyZWx5IGl0IGlzIG5vcm1hbGx5IGEgc3dhcD8N Cj4NCj4gW3lzaGVuXSB5ZXMsIHRoaXMgaXMganVzdCBhIGxhYmVsIHN3YXAgb2YgdGhlIHRvcCBs YWJlbCwgZnJvbSB0dW5uZWwgbGFiZWwgdG8gYnlwYXNzIHR1bm5lbCBsYWJlbC4NCj4NCj4gU0I+ IEkgYW0gc29tZXdoYXQgY29uZnVzZWQgYWJvdXQgd2hhdCB0aGlzIGFsbCBsb29rcyBsaWtlIHdo ZW4gdGhpcyBpcw0KPiBJUC4gVGhlIG9ubHkgUFMvSVAgb3RoZXIgdGhhbiBmb3IgU0FUT1AgYXJl IGRlZmluZWQgYnkgTDJUUHYzIFdHLg0KPiBUaGVyZSBpcyBubyBJUCBNUy1QVyBkZWZpbml0aW9u IHRoYXQgSSBhbSBhd2FyZSBvZi4gSSB0aGluayB5b3UgbmVlZA0KPiB0byBiZSBtdWNoIGNsZWFy ZXIgYWJvdXQgd2hhdCBpcyBnb2luZyBvbiB3aGVuIHlvdSBkaXNjdXNzIFBXIG92ZXIgSVANCj4g ZWdyZXNzIHJlcGFpci4NCj4NCj4gICAgICBUaGUgcHJvdGVjdG9yIE1VU1QgaW4gdHVybg0KPiAg ICAgIGZvcndhcmQgdGhlIHRyYWZmaWMgYmFzZWQgb24gdGhlIFBXIGxhYmVsLiAgVG8gYWNoaWV2 ZSBzdWNoIGtpbmQgb2YNCj4gICAgICBmb3J3YXJkaW5nLCB0aGUgcHJvdGVjdG9yIE1VU1QgcmVs eSBvbiB0aGUgYnlwYXNzIHR1bm5lbCBhcyBhIGNvbnRleHQNCj4gICAgICB0byBkZXRlcm1pbmUg dGhlIHByaW1hcnkgUEUncyBsYWJlbCBzcGFjZS4gIElmIHRoZSBieXBhc3MgdHVubmVsIGlzDQo+ ICAgICAgYW4gTVBMUyB0dW5uZWwsIHRoZSBwcm90ZWN0b3IgTVVTVCBoYXZlIGFzc2lnbmVkIGEg bm9uLXJlc2VydmVkIGxhYmVsDQo+ICAgICAgdG8gdGhlIGJ5cGFzcyB0dW5uZWwgZHVyaW5nIHRo ZSBlc3RhYmxpc2htZW50IG9mIHRoZSBieXBhc3MgdHVubmVsLA0KPiAgICAgIGFuZCBoZW5jZSB0 aGlzIGxhYmVsIGNhbiBzZXJ2ZSBhcyB0aGUgY29udGV4dC4gIElmIHRoZSBieXBhc3MgdHVubmVs DQo+ICAgICAgaXMgYW4gSVAgdHVubmVsLCB0aGUgcHJvdGVjdG9yIGNhbiBzaW1wbHkgcmVseSBv biB0aGUgY29udGV4dA0KPiAgICAgIGlkZW50aWZpZXIgY2FycmllZCBhcyB0aGUgZGVzdGluYXRp b24gYWRkcmVzcyBpbiBJUCBoZWFkZXIuDQo+DQo+ICAgICAgQSBieXBhc3MgdHVubmVsIE1VU1Qg aGF2ZSB0aGUgcHJvcGVydHkgdGhhdCBpdCBpcyBub3QgYWZmZWN0ZWQgYnkgdGhlDQo+ICAgICAg dG9wb2xvZ3kgY2hhbmdlcyBjYXVzZWQgYnkgdGhlIGZhaWx1cmUuICBUaGVyZWZvcmUsIGl0IGNh biBiZSB1c2VkIHRvDQo+ICAgICAgdHJhbnNtaXQgdHJhZmZpYyBmb3IgbG9jYWwgcmVwYWlyLiAg SXQgU0hPVUxEIHJlbWFpbiBlZmZlY3RpdmUsIHVudGlsDQo+ICAgICAgdGhlIHRyYWZmaWMgaXMg bW92ZWQgdG8gYW5vdGhlciBmdWxseSBmdW5jdGlvbmFsIGVncmVzcyBBQywgUFcgYW5kL29yDQo+ ICAgICAgdHJhbnNwb3J0IHR1bm5lbC4NCj4NCj4gNC42LiAgRm9yd2FyZGluZyBTdGF0ZSBvbiBQ cm90ZWN0b3INCj4NCj4gICAgICBBIHByb3RlY3RvciBNVVNUIGxlYXJuIFBXIGxhYmVscyBmcm9t IGFsbCB0aGUgcHJpbWFyeSBQRXMgdGhhdCBpdA0KPiAgICAgIHByb3RlY3RzIChTZWN0aW9uIDUu MiksIGFuZCBtYWludGFpbiB0aGUgUFcgbGFiZWxzIGluIHJlc3BlY3RpdmUNCj4gICAgICBsYWJl bCBzcGFjZXMgb2YgdGhlIHByaW1hcnkgUEVzLiAgSW4gdGhlIGNvbnRyb2wgcGxhbmUsIGEgbGFi ZWwgc3BhY2UNCj4gICAgICBpcyBpZGVudGlmaWVkIGJ5IHRoZSBjb250ZXh0IGlkZW50aWZpZXIg b2YgYSBwYWlyIG9mIHtwcmltYXJ5IFBFLA0KPiAgICAgIHByb3RlY3Rvcn0uIEluIHRoZSBmb3J3 YXJkaW5nIHBsYW5lLCBpdCBpcyBpbmRpY2F0ZWQgYnkgdGhlIGJ5cGFzcw0KPiAgICAgIHR1bm5l bChzKSBkZXN0aW5lZCBmb3IgdGhlIGNvbnRleHQgaWRlbnRpZmllci4NCj4NCj4gU0I+IEFyZSB5 b3Ugc2F5aW5nIHRoYXQgdGhpcyBtZWNoYW5pc20gbWFuZGF0ZXMgbm8gUEhQPw0KPg0KPiBbeXNo ZW5dIEl0IG1hbmRhdGVzIFBIUCBmb3IgYnlwYXNzIHR1bm5lbHMgb25seSwgYXMgdGhlaXIgbGFi ZWxzIHNlcnZlIGFzIGNvbnRleHRzIG9uIHByb3RlY3RvcnMuIFRoaXMgaXMgbm90IG1hbmRhdG9y eSBmb3Igbm9ybWFsIHR1bm5lbHMuDQo+DQo+DQo+DQo+DQo+IFlpbWluIFNoZW4sIGV0IGFsLiAg ICAgIEV4cGlyZXMgSmFudWFyeSAyNSwgMjAxNSAgICAgICAgICAgICAgIFtQYWdlIDE3XQ0KPg0K Pg0KPiBJbnRlcm5ldC1EcmFmdCAgICAgUFcgRW5kcG9pbnQgRmFzdCBGYWlsdXJlIFByb3RlY3Rp b24gICAgICAgICBKdWx5IDIwMTQNCj4NCj4NCj4gNC42LjEuICBFeGFtcGxlcyBvZiBDby1sb2Nh dGVkIFByb3RlY3Rvcg0KPg0KPiAgICAgIEluIEZpZ3VyZSA3LCBQRTQgaXMgYSBjby1sb2NhdGVk IHByb3RlY3RvciB0aGF0IHByb3RlY3RzIFBXMSBhZ2FpbnN0DQo+ICAgICAgZWdyZXNzIEFDIGZh aWx1cmUgYW5kIGVncmVzcyBub2RlIGZhaWx1cmUuICBJdCBtYWludGFpbnMgYSBsYWJlbA0KPiAg ICAgIHNwYWNlIGZvciBQRTIsIHdoaWNoIGlzIGlkZW50aWZpZWQgYnkgdGhlIGNvbnRleHQgaWRl bnRpZmllciBvZiB7UEUyLA0KPiAgICAgIFBFNH0uIEl0IGxlYXJucyBQVzEncyBsYWJlbCBmcm9t IFBFMiwgYW5kIGluc3RhbGxzIGFuIGZvcndhcmRpbmcNCj4gICAgICBlbnRyeSBmb3IgdGhlIGxh YmVsIGluIHRoYXQgbGFiZWwgc3BhY2UuICBUaGUgbmV4dGhvcCBvZiB0aGUNCj4gICAgICBmb3J3 YXJkaW5nIGVudHJ5IGluZGljYXRlcyBhIGxhYmVsIHBvcCB3aXRoIG91dGdvaW5nIGludGVyZmFj ZQ0KPiAgICAgIHBvaW50aW5nIHRvIHRoZSBiYWNrdXAgQUMgQ0UyLVBFNC4NCj4NCj4gU0I+IEkg YW0gbm90IHN1cmUgd2hhdCB5b3UgYXJlIGRlc2NyaWJpbmcgaW4gdGhlIGFib3ZlLiBJcyB0aGlz IGFuDQo+IE1QTFMgb3BlcmF0aW9uIG9yIGEgUFcgb3BlcmF0aW9uLiBJZiB0aGUgbGF0dGVyIGl0 IGlzIG5vdCBhIHNpbXBsZQ0KPiBsYWJlbCBwb3AuDQo+DQo+IFt5c2hlbl0gSXQgaXMgUFcgbGFi ZWwgcG9wLg0KPg0KPiAgICAgIEluIEZpZ3VyZSA4LCBTUEUyIGlzIGEgY28tbG9jYXRlZCBwcm90 ZWN0b3IgdGhhdCBwcm90ZWN0cyBQVzEgYWdhaW5zdA0KPiAgICAgIFMtUEUgZmFpbHVyZS4gIEl0 IG1haW50YWlucyBhIGxhYmVsIHNwYWNlIGZvciBTUEUxLCB3aGljaCBpcw0KPiAgICAgIGlkZW50 aWZpZWQgYnkgdGhlIGNvbnRleHQgaWRlbnRpZmllciBvZiB7U1BFMSwgU1BFMn0uIEl0IGxlYXJu cw0KPiAgICAgIFNFRzEncyBsYWJlbCBmcm9tIFNQRTEsIGFuZCBpbnN0YWxscyBhIGZvcndhcmRp bmcgZW50cnkgaW4gdGhlIGxhYmVsDQo+ICAgICAgc3BhY2UuICBUaGUgbmV4dGhvcCBvZiB0aGUg Zm9yd2FyZGluZyBlbnRyeSBpbmRpY2F0ZXMgYSBsYWJlbCBzd2FwIHRvDQo+ICAgICAgU0VHNCdz IGxhYmVsLg0KPg0KPiA0LjYuMi4gIEV4YW1wbGVzIG9mIENlbnRyYWxpemVkIFByb3RlY3Rvcg0K Pg0KPiAgICAgIEluIHRoZSBjZW50cmFsaXplZCBwcm90ZWN0b3IgbW9kZWwsIGZvciBlYWNoIHBy aW1hcnkgUFcgb2Ygd2hpY2ggdGhlDQo+ICAgICAgcHJvdGVjdG9yIGlzIG5vdCBhIGJhY2t1cCAo Uy0pUEUsIHRoZSBwcm90ZWN0b3IgTVVTVCBhbHNvIGxlYXJuIHRoZQ0KPiAgICAgIGxhYmVsIG9m IHRoZSBiYWNrdXAgUFcgZnJvbSB0aGUgYmFja3VwIChTLSlQRSAoU2VjdGlvbiA1LjMpLiAgVGhp cyBpcw0KPiAgICAgIHRoZSBiYWNrdXAgKFMtKVBFIHRoYXQgdGhlIHByb3RlY3RvciB3aWxsIGZv cndhcmQgdHJhZmZpYyB0by4gIFRoZQ0KPiAgICAgIHByb3RlY3RvciBNVVNUIGluc3RhbGwgYSBm b3J3YXJkaW5nIGVudHJ5IHdpdGggbGFiZWwgc3dhcCBmcm9tIHRoZQ0KPiAgICAgIHByaW1hcnkg UFcncyBsYWJlbCB0byB0aGUgYmFja3VwIFBXJ3MgbGFiZWwuDQo+DQo+ICAgICAgSW4gRmlndXJl IDksIHRoZSBwcm90ZWN0b3IgaXMgYSBjZW50cmFsaXplZCBwcm90ZWN0b3IgdGhhdCBwcm90ZWN0 cw0KPiAgICAgIFBXMSBhZ2FpbnN0IGVncmVzcyBBQyBmYWlsdXJlIGFuZCBlZ3Jlc3Mgbm9kZSBm YWlsdXJlLiAgSXQgbWFpbnRhaW5zDQo+ICAgICAgYSBsYWJlbCBzcGFjZSBmb3IgUEUyLCB3aGlj aCBpcyBpZGVudGlmaWVkIGJ5IHRoZSBjb250ZXh0IGlkZW50aWZpZXINCj4gICAgICBvZiB7UEUy LCBwcm90ZWN0b3J9LiBJdCBsZWFybnMgUFcxJ3MgbGFiZWwgZnJvbSBQRTIsIGFuZCBQVzIncyBs YWJlbA0KPiAgICAgIGZyb20gUEU0LiAgSXQgaW5zdGFsbHMgYSBmb3J3YXJkaW5nIGVudHJ5IGZv ciBQVzEncyBsYWJlbCBpbiB0aGUNCj4gICAgICBsYWJlbCBzcGFjZS4gIFRoZSBuZXh0aG9wIG9m IHRoZSBmb3J3YXJkaW5nIGVudHJ5IGluZGljYXRlcyBhIGxhYmVsDQo+ICAgICAgc3dhcCB0byBQ VzIncyBsYWJlbC4NCj4NCj4gICAgICBJbiBGaWd1cmUgMTAsIHRoZSBwcm90ZWN0b3IgaXMgYSBj ZW50cmFsaXplZCBwcm90ZWN0b3IgdGhhdCBwcm90ZWN0cw0KPiAgICAgIHRoZSBQVyBzZWdtZW50 IFNFRzEgb2YgUFcxIGFnYWluc3QgdGhlIG5vZGUgZmFpbHVyZSBvZiBTUEUxLiAgSXQNCj4gICAg ICBtYWludGFpbnMgYSBsYWJlbCBzcGFjZSBmb3IgU1BFMSwgd2hpY2ggaXMgaWRlbnRpZmllZCBi eSB0aGUgY29udGV4dA0KPiAgICAgIGlkZW50aWZpZXIgb2Yge1NQRTEsIHByb3RlY3Rvcn0uIEl0 IGxlYXJucyBTRUcxJ3MgbGFiZWwgZnJvbSBTUEUxLA0KPiAgICAgIGFuZCBsZWFybnMgU0VHMydz IGxhYmVsIGZyb20gU1BFMi4gIEl0IGluc3RhbGxzIGEgZm9yd2FyZGluZyBlbnRyeQ0KPiAgICAg IGZvciBTRUcxJ3MgbGFiZWwgaW4gdGhlIGxhYmVsIHNwYWNlLiAgVGhlIG5leHRob3Agb2YgdGhl IGZvcndhcmRpbmcNCj4gICAgICBlbnRyeSBpbmRpY2F0ZXMgYSBsYWJlbCBzd2FwIHRvIFNFRzMn cyBsYWJlbC4NCj4NCj4gNS4gIExEUCBFeHRlbnNpb25zDQo+DQo+ICAgICAgQXMgZGVzY3JpYmVk IGluIHByZXZpb3VzIHNlY3Rpb25zLCBhIHRhcmdldGVkIExEUCBzZXNzaW9uIE1VU1QgYmUNCj4g ICAgICBlc3RhYmxpc2hlZCBiZXR3ZWVuIGVhY2ggcGFpciBvZiBwcmltYXJ5IFBFIGFuZCBwcm90 ZWN0b3IuICBUaGUNCj4gICAgICBwcmltYXJ5IFBFIHNlbmRzIExhYmVsIE1hcHBpbmcgbWVzc2Fn ZSBvdmVyIHRoaXMgc2Vzc2lvbiB0byBhZHZlcnRpc2UNCj4gICAgICBwcmltYXJ5IFBXIGxhYmVs cyB0byB0aGUgcHJvdGVjdG9yLiAgSW4gdGhlIGNlbnRyYWxpemVkIHByb3RlY3Rvcg0KPg0KPg0K Pg0KPiBZaW1pbiBTaGVuLCBldCBhbC4gICAgICBFeHBpcmVzIEphbnVhcnkgMjUsIDIwMTUgICAg ICAgICAgICAgICBbUGFnZSAxOF0NCj4NCj4NCj4gSW50ZXJuZXQtRHJhZnQgICAgIFBXIEVuZHBv aW50IEZhc3QgRmFpbHVyZSBQcm90ZWN0aW9uICAgICAgICAgSnVseSAyMDE0DQo+DQo+DQo+ICAg ICAgbW9kZWwsIGEgdGFyZ2V0ZWQgTERQIHNlc3Npb24gTVVTVCBhbHNvIGJlIGVzdGFibGlzaGVk IGJldHdlZW4gYQ0KPiAgICAgIGJhY2t1cCAoUy0pUEUgYW5kIGEgcHJvdGVjdG9yLiAgVGhlIGJh Y2t1cCBQRSBzZW5kcyBMYWJlbCBNYXBwaW5nDQo+ICAgICAgbWVzc2FnZSBvdmVyIHRoaXMgc2Vz c2lvbiB0byBhZHZlcnRpc2UgYmFja3VwIFBXIGxhYmVscyB0byB0aGUNCj4gICAgICBwcm90ZWN0 b3IuDQo+DQo+ICAgICAgVG8gZmFjaWxpdGF0ZSB0aGUgcHJvY2VkdXJlcywgdGhpcyBkb2N1bWVu dCBkZWZpbmVzIGEgbmV3ICJQcm90ZWN0aW9uDQo+ICAgICAgRkVDIEVsZW1lbnQiIFRMVi4gIFRo ZSBMYWJlbCBNYXBwaW5nIG1lc3NhZ2VzIG9mIGJvdGggdGhlIExEUA0KPiAgICAgIHNlc3Npb25z IGFib3ZlIE1VU1QgY2FycnkgdGhpcyBUTFYgdG8gaW5kaWNhdGUgdGhlIGlkZW50aXR5IG9mIGEN Cj4gICAgICBwcmltYXJ5IFBXLiAgU3BlY2lmaWNhbGx5LCBpbiB0aGUgY2VudHJhbGl6ZWQgcHJv dGVjdG9yIG1vZGVsLCB0aGUNCj4gICAgICBQcm90ZWN0aW9uIEZFQyBFbGVtZW50IFRMViBhZHZl cnRpc2VkIGJ5IGEgYmFja3VwIChTLSlQRSBNVVNUIG1hdGNoDQo+ICAgICAgdGhlIG9uZSBhZHZl cnRpc2VkIGJ5IHRoZSBwcmltYXJ5IFBFLCBzbyB0aGF0IHRoZSBwcm90ZWN0b3IgY2FuDQo+ICAg ICAgYXNzb2NpYXRlIHRoZSBwcmltYXJ5IFBXJ3MgbGFiZWwgd2l0aCB0aGUgYmFja3VwIFBXJ3Mg bGFiZWwsIGFuZA0KPiAgICAgIHBlcmZvcm0gYSBsYWJlbCBzd2FwLg0KPg0KPiAgICAgIFRoaXMg ZG9jdW1lbnQgYWxzbyBkZWZpbmVzIHRoZSBlbmNvZGluZyBvZiBDYXBhYmlsaXR5IFBhcmFtZXRl ciBUTFYNCj4gICAgICAoUkZDIDU1NjEpIGZvciBhIG5ldyAiRWdyZXNzIFByb3RlY3Rpb24gQ2Fw YWJpbGl0eSIsIHRvIGFsbG93IGENCj4gICAgICBwcm90ZWN0b3IgdG8gYW5ub3VuY2UgaXRzIGNh cGFiaWxpdHkgb2YgcHJvY2Vzc2luZyB0aGUgYWJvdmUNCj4gICAgICBQcm90ZWN0aW9uIEZFQyBF bGVtZW50IFRMViBhbmQgcGVyZm9ybWluZyBjb250ZXh0IHNwZWNpZmljIGxhYmVsDQo+ICAgICAg c3dpdGNoaW5nIGZvciBQVyBsYWJlbHMuDQo+DQo+ICAgICAgVGhlIHByb2NlZHVyZXMgaW4gdGhp cyBzZWN0aW9uIGFyZSBvbmx5IGFwcGxpY2FibGUsIGlmIHRoZSBwcm90ZWN0b3INCj4gICAgICBh ZHZlcnRpc2VzIHRoZSBFZ3Jlc3MgUHJvdGVjdGlvbiBDYXBhYmlsaXR5LCB0aGUgcHJpbWFyeSBQ RSBzdXBwb3J0cw0KPiAgICAgIHRoZSBhZHZlcnRpc2VtZW50IG9mIHRoZSBQcm90ZWN0aW9uIEZF QyBFbGVtZW50IFRMViwgYW5kIGluIHRoZQ0KPiAgICAgIGNlbnRyYWxpemVkIHByb3RlY3RvciBt b2RlbCwgdGhlIGJhY2t1cCBQRSBhbHNvIHN1cHBvcnRzIHRoZQ0KPiAgICAgIGFkdmVydGlzZW1l bnQgb2YgdGhlIFByb3RlY3Rpb24gRkVDIEVsZW1lbnQgVExWLg0KPg0KPiA1LjEuICBFZ3Jlc3Mg UHJvdGVjdGlvbiBDYXBhYmlsaXR5IFRMVg0KPg0KPiAgICAgIEEgcHJvdGVjdG9yIE1VU1QgYWR2 ZXJ0aXNlIHRoZSBFZ3Jlc3MgUHJvdGVjdGlvbiBDYXBhYmlsaXR5IFRMViBpbg0KPiAgICAgIGl0 cyBJbml0aWFsaXphdGlvbiBtZXNzYWdlIGFuZCBDYXBhYmlsaXR5IG1lc3NhZ2UsIG92ZXIgdGhl IExEUA0KPiAgICAgIHNlc3Npb24gd2l0aCBhIHByaW1hcnkgUEUuICBJbiB0aGUgY2VudHJhbGl6 ZWQgcHJvdGVjdG9yIG1vZGVsLCB0aGUNCj4gICAgICBwcm90ZWN0b3IgTVVTVCBhbHNvIGFkdmVy dGlzZSB0aGUgVExWIG92ZXIgdGhlIExEUCBzZXNzaW9uIHdpdGggYQ0KPiAgICAgIGJhY2t1cCBQ RS4gIFRoZSBUTFYgY2FycmllcyBvbmUgb3IgbXVsdGlwbGUgY29udGV4dCBpZGVudGlmaWVycy4g IFRvDQo+ICAgICAgdGhlIHByaW1hcnkgUEUsIHRoZSBUTFYgU0hPVUxEIGNhcnJ5IHRoZSBjb250 ZXh0IGlkZW50aWZpZXIgb2YgdGhlDQo+ICAgICAge3ByaW1hcnkgUEUsIHByb3RlY3Rvcn0uIElu IHRoZSBjZW50cmFsaXplZCBwcm90ZWN0b3IgbW9kZWwsIHRoZSBUTFYNCj4gICAgICBTSE9VTEQg Y2FycnkgdG8gdGhlIGJhY2t1cCBQRSBtdWx0aXBsZSBjb250ZXh0IGlkZW50aWZpZXJzLCBvbmUg Zm9yDQo+ICAgICAgZWFjaCB7cHJpbWFyeSBQRSwgcHJvdGVjdG9yfSB3aGVyZSB0aGUgYmFja3Vw IFBFIHNlcnZlcyBhcyBhIGJhY2t1cA0KPiAgICAgIGZvciB0aGUgcHJpbWFyeSBQRS4gIFRoaXMg VExWIFNIT1VMRCBOT1QgYmUgYWR2ZXJ0aXNlZCBieSB0aGUgcHJpbWFyeQ0KPiAgICAgIFBFIG9y IHRoZSBiYWNrdXAgUEUgdG8gdGhlIHByb3RlY3Rvci4NCj4NCj4gICAgICBUaGUgcHJvY2Vzc2lu ZyBvZiB0aGUgRWdyZXNzIFByb3RlY3Rpb24gQ2FwYWJpbGl0eSBUTFYgYnkgYSByZWNlaXZpbmcN Cj4gICAgICByb3V0ZXIgU0hPVUxEIGZvbGxvdyB0aGUgcHJvY2VkdXJlcyBkZWZpbmVkIGluIFJG QyA1NTYxLiAgSW4NCj4gICAgICBwYXJ0aWN1bGFyLCB0aGUgcm91dGVyIFNIT1VMRCBhZHZlcnRp c2UgUFcgaW5mb3JtYXRpb24gdG8gdGhlDQo+ICAgICAgcHJvdGVjdG9yIGJ5IHVzaW5nIHRoZSBQ cm90ZWN0aW9uIEZFQyBFbGVtZW50IFRMViwgb25seSBhZnRlciBpdCBoYXMNCj4gICAgICByZWNl aXZlZCB0aGUgRWdyZXNzIFByb3RlY3Rpb24gQ2FwYWJpbGl0eSBUTFYgZnJvbSB0aGUgcHJvdGVj dG9yLiAgSXQNCj4gICAgICBTSE9VTEQgdmFsaWRhdGUgZWFjaCBjb250ZXh0IGlkZW50aWZpZXIg aW5jbHVkZWQgaW4gdGhlIFRMViwgYW5kDQo+ICAgICAgYWR2ZXJ0aXNlIHRoZSBpbmZvcm1hdGlv biBvZiBvbmx5IHRob3NlIFBXcyB0aGF0IGFyZSBhc3NvY2lhdGVkIHdpdGgNCj4gICAgICB0aGUg Y29udGV4dCBpZGVudGlmaWVyLiAgSXQgU0hPVUxEIHdpdGhkcmF3IHByZXZpb3VzbHkgYWR2ZXJ0 aXNlZA0KPg0KPg0KPg0KPiBZaW1pbiBTaGVuLCBldCBhbC4gICAgICBFeHBpcmVzIEphbnVhcnkg MjUsIDIwMTUgICAgICAgICAgICAgICBbUGFnZSAxOV0NCj4NCj4NCj4gSW50ZXJuZXQtRHJhZnQg ICAgIFBXIEVuZHBvaW50IEZhc3QgRmFpbHVyZSBQcm90ZWN0aW9uICAgICAgICAgSnVseSAyMDE0 DQo+DQo+DQo+ICAgICAgUHJvdGVjdGlvbiBGRUMgVExWcywgd2hlbiB0aGUgcHJvdGVjdG9yIGhh cyB3aXRoZHJhd24gYSBwcmV2aW91c2x5DQo+ICAgICAgYWR2ZXJ0aXNlZCBjb250ZXh0IGlkZW50 aWZpZXIgb3IgdGhlIGVudGlyZSBFZ3Jlc3MgUHJvdGVjdGlvbg0KPiAgICAgIENhcGFiaWxpdHkg VExWIHZpYSBDYXBhYmlsaXR5IG1lc3NhZ2UuDQo+DQo+ICAgICAgVGhlIGVuY29kaW5nIG9mIHRo ZSBFZ3Jlc3MgUHJvdGVjdGlvbiBDYXBhYmlsaXR5IFRMViBpcyBkZWZpbmVkIGFzDQo+ICAgICAg YmVsb3cuICBJdCBjb25mb3JtcyB0byB0aGUgZm9ybWF0IG9mIENhcGFiaWxpdHkgUGFyYW1ldGVy IFRMVg0KPiAgICAgIHNwZWNpZmllZCBpbiBSRkMgNTU2MS4NCj4NCj4gICAgICAgICAwICAgICAg ICAgICAgICAgICAgIDEgICAgICAgICAgICAgICAgICAgMiAgICAgICAgICAgICAgICAgICAzDQo+ ICAgICAgICAgMCAxIDIgMyA0IDUgNiA3IDggOSAwIDEgMiAzIDQgNSA2IDcgOCA5IDAgMSAyIDMg NCA1IDYgNyA4IDkgMCAxDQo+ICAgICAgICArLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0r LSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKw0KPiAgICAgICAgfFV8RnwgIEVncmVz cyBQcm90ZWN0aW9uIChUQkQpICB8ICAgICAgICAgICAgICBMZW5ndGggICAgICAgICAgIHwNCj4g ICAgICAgICstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSst Ky0rLSstKy0rLSstKy0rDQo+ICAgICAgICB8U3wgUmVzZXJ2ZWQgICAgfCAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfA0KPiAgICAgICAgKy0rLSstKy0rLSst Ky0rLSsgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHwNCj4g ICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICB8DQo+ICAgICAgICB+ICAgICAgICAgICAgICAgIENhcGFiaWxpdHkgRGF0 YSA9IGNvbnRleHQgaWRlbnRpZmllcihzKSAgICAgICAgfg0KPiAgICAgICAgfCAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHwNCj4g ICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICst Ky0rLSstKy0rLSstKy0rDQo+ICAgICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICB8DQo+ICAgICAgICArLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSst Ky0rLSstKy0rLSstKy0rLSstKy0rDQo+DQo+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgRmlndXJlIDExDQo+DQo+ICAgICAgVGhlIFUtYml0IE1VU1QgYmUgc2V0IHRvIDEgc28g dGhhdCBhIHJlY2VpdmVyIE1VU1Qgc2lsZW50bHkgaWdub3JlDQo+ICAgICAgdGhpcyBUTFYgaWYg dW5rbm93biB0byBpdCwgYW5kIGNvbnRpbnVlIHByb2Nlc3NpbmcgdGhlIHJlc3Qgb2YgdGhlDQo+ ICAgICAgbWVzc2FnZS4NCj4NCj4gICAgICBUaGUgRi1iaXQgTVVTVCBiZSBzZXQgdG8gMCBzaW5j ZSB0aGlzIFRMViBpcyBzZW50IG9ubHkgaW4NCj4gICAgICBJbml0aWFsaXphdGlvbiBhbmQgQ2Fw YWJpbGl0eSBtZXNzYWdlcywgd2hpY2ggYXJlIG5vdCBmb3J3YXJkZWQuDQo+DQo+ICAgICAgVGhl IFRMViBDb2RlIFBvaW50IGlzIFRCRC4gIEl0IG5lZWRzIHRvIGJlIGFzc2lnbmVkIGJ5IElBTkEu DQo+DQo+ICAgICAgVGhlIFMtYml0IGluZGljYXRlcyB3aGV0aGVyIHRoZSBzZW5kZXIgaXMgYWR2 ZXJ0aXNpbmcgKFM9MSkgb3INCj4gICAgICB3aXRoZHJhd2luZyAoUz0wKSB0aGUgY2FwYWJpbGl0 eS4NCj4NCj4gICAgICBUaGUgIkNhcGFiaWxpdHkgRGF0YSIgaXMgZW5jb2RlZCB3aXRoIHRoZSBj b250ZXh0IGlkZW50aWZpZXIgb2YgdGhlDQo+ICAgICAge3ByaW1hcnkgUEUsIHByb3RlY3Rvcn0u DQo+DQo+IFNCPiB3aGVyZSBpcyB0aGUgc3RydWN0dXJlIG9mIHtwcmltYXJ5IFBFLCBwcm90ZWN0 b3J9IGRlZmluZWQ/DQo+DQo+IFt5c2hlbl0gVGhlIFByb3RlY3Rpb24gRkVDIEVsZW1lbnQgVExW IGNhcnJpZXMgdGhlIGNvbnRleHQgSUQgb2Yge3ByaW1hcnkgUEUsIHByb3RlY3Rvcn0uIFRoZSB7 cHJpbWFyeSBQRSwgcHJvdGVjdG9yfSB0dXBsZSBpdHNlbGYgaXMgbm90IGVuY29kZWQuDQo+DQo+ IDUuMi4gIFBXIExhYmVsIERpc3RyaWJ1dGlvbiBmcm9tIFByaW1hcnkgUEUgdG8gUHJvdGVjdG9y DQo+DQo+ICAgICAgQSBwcmltYXJ5IFBFIFNIT1VMRCBhZHZlcnRpc2UgYSBwcmltYXJ5IFBXJ3Mg bGFiZWwgdG8gYSBwcm90ZWN0b3IgYnkNCj4gICAgICBzZW5kaW5nIGEgTGFiZWwgTWFwcGluZyBt ZXNzYWdlLiAgVGhlIG1lc3NhZ2UgaW5jbHVkZXMgYSBQcm90ZWN0aW9uDQo+ICAgICAgRkVDIEVs ZW1lbnQgVExWIChzZWUgU2VjdGlvbiA1LjQgZm9yIGVuY29kaW5nKSwgYW5kIGFuIFVwc3RyZWFt LQ0KPiAgICAgIEFzc2lnbmVkIExhYmVsIFRMViAoUkZDIDYzODkpIGVuY29kZWQgd2l0aCB0aGUg UFcncyBsYWJlbC4gIFRoZQ0KPiAgICAgIGNvbWJpbmF0aW9uIG9mIHRoZSBQcm90ZWN0aW9uIEZF QyBFbGVtZW50IFRMViBhbmQgdGhlIFBXIGxhYmVsDQo+ICAgICAgcmVwcmVzZW50cyB0aGUgcHJp bWFyeSBQRSdzIGZvcndhcmRpbmcgc3RhdGUgZm9yIHRoZSBQVy4gIFRoZSBMYWJlbA0KPiAgICAg IE1hcHBpbmcgbWVzc2FnZSBTSE9VTEQgYWxzbyBjYXJyeSBhbiBJUHY0L3Y2IEludGVyZmFjZV9J RCBUTFYgKFJGQw0KPg0KPg0KPg0KPiBZaW1pbiBTaGVuLCBldCBhbC4gICAgICBFeHBpcmVzIEph bnVhcnkgMjUsIDIwMTUgICAgICAgICAgICAgICBbUGFnZSAyMF0NCj4NCj4NCj4gSW50ZXJuZXQt RHJhZnQgICAgIFBXIEVuZHBvaW50IEZhc3QgRmFpbHVyZSBQcm90ZWN0aW9uICAgICAgICAgSnVs eSAyMDE0DQo+DQo+DQo+ICAgICAgNjM4OSwgUkZDIDM0NzEpIGVuY29kZWQgd2l0aCB0aGUgY29u dGV4dCBpZGVudGlmaWVyIG9mIHRoZSB7cHJpbWFyeQ0KPiAgICAgIFBFLCBwcm90ZWN0b3J9Lg0K Pg0KPiAgICAgIFRoZSBwcm90ZWN0b3IgdGhhdCByZWNlaXZlcyB0aGlzIExhYmVsIE1hcHBpbmcg bWVzc2FnZSBTSE9VTEQgaW5zdGFsbA0KPiAgICAgIGEgZm9yd2FyZGluZyBlbnRyeSBmb3IgdGhl IFBXIGxhYmVsIGluIHRoZSBsYWJlbCBzcGFjZSBpZGVudGlmaWVkIGJ5DQo+ICAgICAgdGhlIGNv bnRleHQgaWRlbnRpZmllci4gIFRoZSBuZXh0aG9wIG9mIHRoZSBmb3J3YXJkaW5nIGVudHJ5IFNI T1VMRA0KPiAgICAgIGVuc3VyZSBwYWNrZXRzIHRvIGJlIHNlbnQgdG93YXJkcyB0aGUgdGFyZ2V0 IENFIHZpYSBhIGJhY2t1cCBBQyBvciBhDQo+ICAgICAgYmFja3VwIChTLSlQRSwgZGVwZW5kaW5n IG9uIHRoZSBwcm90ZWN0aW9uIHNjZW5hcmlvLiAgVGhlIHByb3RlY3Rvcg0KPiAgICAgIFNIT1VM RCBzaWxlbnRseSBkaXNjYXJkIGEgTGFiZWwgTWFwcGluZyBtZXNzYWdlIGlmIHRoZSBpbmNsdWRl ZA0KPiAgICAgIGNvbnRleHQgaWRlbnRpZmllciBpcyB1bmtub3duIHRvIGl0Lg0KPg0KPiA1LjMu ICBQVyBMYWJlbCBEaXN0cmlidXRpb24gZnJvbSBCYWNrdXAgUEUgdG8gUHJvdGVjdG9yDQo+DQo+ ICAgICAgSW4gdGhlIGNlbnRyYWxpemVkIHByb3RlY3RvciBtb2RlbCwgYSBiYWNrdXAgUEUgU0hP VUxEIGFkdmVydGlzZSBhDQo+ICAgICAgYmFja3VwIFBXJ3MgbGFiZWwgdG8gYSBwcm90ZWN0b3Ig Ynkgc2VuZGluZyBhIExhYmVsIE1hcHBpbmcgbWVzc2FnZS4NCj4gICAgICBUaGUgbWVzc2FnZSBp bmNsdWRlcyBhIFByb3RlY3Rpb24gRkVDIEVsZW1lbnQgVExWIGFuZCBhIEdlbmVyaWMgTGFiZWwN Cj4gICAgICBUTFYgZW5jb2RlZCB3aXRoIHRoZSBiYWNrdXAgUFcncyBsYWJlbC4gIFRoaXMgUHJv dGVjdGlvbiBGRUMgRWxlbWVudA0KPiAgICAgIE1VU1QgYmUgaWRlbnRpY2FsIHRvIHRoZSBQcm90 ZWN0aW9uIEZFQyBFbGVtZW50IFRMViB0aGF0IHRoZSBwcmltYXJ5DQo+ICAgICAgUEUgYWR2ZXJ0 aXNlcyB0byB0aGUgcHJvdGVjdG9yIChTZWN0aW9uIDUuMikuICBUaGUgY29udGV4dCBpZGVudGlm aWVyDQo+ICAgICAgU0hPVUxEIE5PVCBiZSBlbmNvZGVkIGluIEludGVyZmFjZV9JRCBUTFYgaW4g dGhpcyBtZXNzYWdlLg0KPg0KPiAgICAgIFRoZSBwcm90ZWN0b3IgdGhhdCByZWNlaXZlcyB0aGlz IExhYmVsIE1hcHBpbmcgbWVzc2FnZSBTSE9VTEQNCj4gICAgICBhc3NvY2lhdGUgdGhlIGJhY2t1 cCBQVyB3aXRoIHRoZSBwcmltYXJ5IFBXLCBiYXNlZCBvbiB0aGUgY29tbW9uDQo+ICAgICAgUHJv dGVjdGlvbiBGRUMgRWxlbWVudCBUTFYuICBJdCBTSE9VTEQgZGlzdGluZ3Vpc2ggYmV0d2VlbiB0 aGUgTGFiZWwNCj4gICAgICBNYXBwaW5nIG1lc3NhZ2UgZnJvbSB0aGUgcHJpbWFyeSBQRSBhbmQg dGhlIExhYmVsIE1hcHBpbmcgbWVzc2FnZQ0KPiAgICAgIGZyb20gdGhlIGJhY2t1cCBQRSBiYXNl ZCBvbiB0aGUgcmVzcGVjdGl2ZSBwcmVzZW5jZSBhbmQgYWJzZW5jZSBvZg0KPiAgICAgIGNvbnRl eHQgaWRlbnRpZmllciBpbiBJbnRlcmZhY2VfSUQgVExWLiAgSXQgU0hPVUxEIGluc3RhbGwgYQ0K PiAgICAgIGZvcndhcmRpbmcgZW50cnkgZm9yIHRoZSBwcmltYXJ5IFBXJ3MgbGFiZWwgaW4gdGhl IGxhYmVsIHNwYWNlDQo+ICAgICAgaWRlbnRpZmllZCBieSB0aGUgY29udGV4dCBpZGVudGlmaWVy LiAgVGhlIG5leHRob3Agb2YgdGhlIGZvcndhcmRpbmcNCj4gICAgICBlbnRyeSBTSE9VTEQgaW5k aWNhdGUgYSBsYWJlbCBzd2FwIHRvIHRoZSBiYWNrdXAgUFcncyBsYWJlbCwgZm9sbG93ZWQNCj4g ICAgICBieSBhIGxhYmVsIHB1c2ggb3IgSVAgaGVhZGVyIHB1c2ggZm9yIGEgdHJhbnNwb3J0IHR1 bm5lbCB0byB0aGUNCj4gICAgICBiYWNrdXAgUEUuDQo+DQo+IDUuNC4gIFByb3RlY3Rpb24gRkVD IEVsZW1lbnQgVExWDQo+DQo+ICAgICAgVGhlIFByb3RlY3Rpb24gRkVDIEVsZW1lbnQgVExWIGhh cyB0eXBlIDB4ODMuICBJdHMgZm9ybWF0IGlzIGRlZmluZWQNCj4gICAgICBhcyBiZWxvdzoNCj4N Cj4NCj4NCj4NCj4NCj4NCj4NCj4NCj4NCj4NCj4NCj4NCj4NCj4NCj4gWWltaW4gU2hlbiwgZXQg YWwuICAgICAgRXhwaXJlcyBKYW51YXJ5IDI1LCAyMDE1ICAgICAgICAgICAgICAgW1BhZ2UgMjFd DQo+DQo+DQo+IEludGVybmV0LURyYWZ0ICAgICBQVyBFbmRwb2ludCBGYXN0IEZhaWx1cmUgUHJv dGVjdGlvbiAgICAgICAgIEp1bHkgMjAxNA0KPg0KPg0KPiAgICAgICAgIDAgICAgICAgICAgICAg ICAgICAgMSAgICAgICAgICAgICAgICAgICAyICAgICAgICAgICAgICAgICAgIDMNCj4gICAgICAg ICAwIDEgMiAzIDQgNSA2IDcgOCA5IDAgMSAyIDMgNCA1IDYgNyA4IDkgMCAxIDIgMyA0IDUgNiA3 IDggOSAwIDENCj4gICAgICAgICstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSst Ky0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rDQo+ICAgICAgICB8ICAgVHlwZSgweDgzKSAgfCAg ICBSZXNlcnZlZCAgIHwgRW5jb2RpbmcgVHlwZSB8ICAgIExlbmd0aCAgICAgfA0KPiAgICAgICAg Ky0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0r LSstKy0rLSsNCj4gICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICB8DQo+ICAgICAgICB8ICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfA0KPiAgICAgICAg fiAgICAgICAgICAgICAgICAgICAgICAgICBQVyBJbmZvcm1hdGlvbiAgICAgICAgICAgICAgICAg ICAgICAgIH4NCj4gICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICB8DQo+ICAgICAgICB8ICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKw0KPiAgICAgICAg fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB8DQo+ICAgICAgICArLSstKy0rLSstKy0r LSstKy0rLSstKy0rLSstKy0rLSsNCj4NCj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICBGaWd1cmUgMTINCj4NCj4gICAgICAtIEVuY29kaW5nIFR5cGUNCj4NCj4gICAgICAgICBU eXBlIG9mIGZvcm1hdCB0aGF0IFBXIEluZm9ybWF0aW9uIGZpZWxkIGlzIGVuY29kZWQuDQo+DQo+ ICAgICAgLSBMZW5ndGgNCj4NCj4gICAgICAgICBMZW5ndGggb2YgUFcgSW5mb3JtYXRpb24gZmll bGQgaW4gb2N0ZXRzLg0KPg0KPiAgICAgIC0gUFcgSW5mb3JtYXRpb24NCj4NCj4gICAgICAgICBG aWVsZCBvZiB2YXJpYWJsZSBsZW5ndGggdGhhdCBzcGVjaWZpZXMgYSBQVw0KPg0KPiAgICAgIEZv ciBFbmNvZGluZyBUeXBlLCAxIGlzIGRlZmluZWQgZm9yIHRoZSBQV2lkIEZFQyBFbGVtZW50IGZv cm1hdCwgYW5kDQo+ICAgICAgMiBpcyBkZWZpbmVkIGZvciB0aGUgR2VuZXJhbGl6ZWQgUFdpZCBG RUMgRWxlbWVudCBmb3JtYXQgKFJGQyA0NDQ3KS4NCj4NCj4NCj4gNS40LjEuICBFbmNvZGluZyBG b3JtYXQgZm9yIFBXaWQNCj4NCj4NCj4NCj4NCj4NCj4NCj4NCj4NCj4NCj4NCj4NCj4NCj4NCj4N Cj4NCj4NCj4NCj4NCj4NCj4NCj4gWWltaW4gU2hlbiwgZXQgYWwuICAgICAgRXhwaXJlcyBKYW51 YXJ5IDI1LCAyMDE1ICAgICAgICAgICAgICAgW1BhZ2UgMjJdDQo+DQo+DQo+IEludGVybmV0LURy YWZ0ICAgICBQVyBFbmRwb2ludCBGYXN0IEZhaWx1cmUgUHJvdGVjdGlvbiAgICAgICAgIEp1bHkg MjAxNA0KPg0KPg0KPiAgICAgICAgIDAgICAgICAgICAgICAgICAgICAgMSAgICAgICAgICAgICAg ICAgICAyICAgICAgICAgICAgICAgICAgIDMNCj4gICAgICAgICAwIDEgMiAzIDQgNSA2IDcgOCA5 IDAgMSAyIDMgNCA1IDYgNyA4IDkgMCAxIDIgMyA0IDUgNiA3IDggOSAwIDENCj4gICAgICAgICst Ky0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0r LSstKy0rDQo+ICAgICAgICB8ICAgVHlwZSgweDgzKSAgfCAgICBSZXNlcnZlZCAgIHwgIEVuYyBU eXBlKDEpICB8ICAgTGVuZ3RoKDE2KSAgfA0KPiAgICAgICAgKy0rLSstKy0rLSstKy0rLSstKy0r LSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSsNCj4gICAgICAgIHwg ICAgICAgICAgICAgICAgICAgICAgSW5ncmVzcyBQRSBBZGRyZXNzICAgICAgICAgICAgICAgICAg ICAgICB8DQo+ICAgICAgICArLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSst Ky0rLSstKy0rLSstKy0rLSstKy0rLSstKw0KPiAgICAgICAgfCAgICAgICAgICAgICAgICAgICAg ICAgRWdyZXNzIFBFIEFkZHJlc3MgICAgICAgICAgICAgICAgICAgICAgIHwNCj4gICAgICAgICst Ky0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0r LSstKy0rDQo+ICAgICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgIEdyb3VwIElEICAg ICAgICAgICAgICAgICAgICAgICAgICAgfA0KPiAgICAgICAgKy0rLSstKy0rLSstKy0rLSstKy0r LSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSsNCj4gICAgICAgIHwg ICAgICAgICAgICAgICAgICAgICAgICAgICAgIFBXIElEICAgICAgICAgICAgICAgICAgICAgICAg ICAgICB8DQo+ICAgICAgICArLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSst Ky0rLSstKy0rLSstKy0rLSstKy0rLSstKw0KPiAgICAgICAgfEN8ICAgICAgICAgICBQVyBUeXBl ICAgICAgICAgICB8ICAgICAgICAgICBSZXNlcnZlZCAgICAgICAgICAgIHwNCj4gICAgICAgICst Ky0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0r LSstKy0rDQo+DQo+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgRmlndXJlIDEz DQo+DQo+ICAgICAgLSBJbmdyZXNzIFBFIEFkZHJlc3MNCj4NCj4gICAgICAgICBJUCBhZGRyZXNz IG9mIHRoZSBpbmdyZXNzIFBFIG9mIFBXLg0KPg0KPiAgICAgIC0gRWdyZXNzIFBFIEFkZHJlc3MN Cj4NCj4gICAgICAgICBJUCBhZGRyZXNzIG9mIHRoZSBlZ3Jlc3MgUEUgb2YgUFcuDQo+DQo+IFNC PiBIb3cgZG9lcyB0aGlzIHdvcmsgZm9yIElQdjY/DQo+DQo+ICAgICAgLSBHcm91cCBJRA0KPg0K PiAgICAgICAgIEFuIGFyYml0cmFyeSAzMi1iaXQgdmFsdWUgdGhhdCByZXByZXNlbnRzIGEgZ3Jv dXAgb2YgUFdzIGFuZCB0aGF0DQo+ICAgICAgICAgaXMgdXNlZCB0byBjcmVhdGUgZ3JvdXBzIGlu IHRoZSBQVyBzcGFjZS4NCj4NCj4gICAgICAtIFBXIElEDQo+DQo+ICAgICAgICAgQSBub24temVy byAzMi1iaXQgY29ubmVjdGlvbiBJRCB0aGF0LCB0b2dldGhlciB3aXRoIHRoZSBQVyBUeXBlDQo+ ICAgICAgICAgZmllbGQsIGlkZW50aWZpZXMgYSBwYXJ0aWN1bGFyIFBXLg0KPg0KPiAgICAgIC0g Q29udHJvbCB3b3JkIGJpdCAoQykNCj4NCj4gICAgICAgICBBIGJpdCB0aGF0IGZsYWdzIHRoZSBw cmVzZW5jZSBvZiBhIGNvbnRyb2wgd29yZCBvbiB0aGlzIFBXLiAgSWYgQw0KPiAgICAgICAgID0g MSwgY29udHJvbCB3b3JkIGlzIHByZXNlbnQ7IElmIEMgPSAwLCBjb250cm9sIHdvcmQgaXMgbm90 DQo+ICAgICAgICAgcHJlc2VudC4NCj4NCj4gICAgICAtIFBXIFR5cGUNCj4NCj4gICAgICAgICBB IDE1LWJpdCBxdWFudGl0eSB0aGF0IHJlcHJlc2VudHMgdGhlIHR5cGUgb2YgUFcuDQo+DQo+IFNC PiBOZWVkcyBhIHJlZg0KPg0KPg0KPg0KPg0KPiBZaW1pbiBTaGVuLCBldCBhbC4gICAgICBFeHBp cmVzIEphbnVhcnkgMjUsIDIwMTUgICAgICAgICAgICAgICBbUGFnZSAyM10NCj4NCj4NCj4gSW50 ZXJuZXQtRHJhZnQgICAgIFBXIEVuZHBvaW50IEZhc3QgRmFpbHVyZSBQcm90ZWN0aW9uICAgICAg ICAgSnVseSAyMDE0DQo+DQo+DQo+IDUuNC4yLiAgRW5jb2RpbmcgRm9ybWF0IGZvciBHZW5lcmFs aXplZCBQV2lkDQo+DQo+ICAgICAgICAgMCAgICAgICAgICAgICAgICAgICAxICAgICAgICAgICAg ICAgICAgIDIgICAgICAgICAgICAgICAgICAgMw0KPiAgICAgICAgIDAgMSAyIDMgNCA1IDYgNyA4 IDkgMCAxIDIgMyA0IDUgNiA3IDggOSAwIDEgMiAzIDQgNSA2IDcgOCA5IDAgMQ0KPiAgICAgICAg Ky0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0r LSstKy0rLSsNCj4gICAgICAgIHwgICBUeXBlKDB4ODMpICB8ICAgIFJlc2VydmVkICAgfCAgRW5j IFR5cGUoMikgIHwgICBMZW5ndGggICAgICB8DQo+ICAgICAgICArLSstKy0rLSstKy0rLSstKy0r LSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKw0KPiAgICAgICAg fCAgICAgICAgICAgICAgICAgICAgICBJbmdyZXNzIFBFIEFkZHJlc3MgICAgICAgICAgICAgICAg ICAgICAgIHwNCj4gICAgICAgICstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSst Ky0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rDQo+ICAgICAgICB8ICAgICAgICAgICAgICAgICAg ICAgICBFZ3Jlc3MgUEUgQWRkcmVzcyAgICAgICAgICAgICAgICAgICAgICAgfA0KPiAgICAgICAg Ky0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0r LSstKy0rLSsNCj4gICAgICAgIHxDfCAgICAgICAgICAgUFcgVHlwZSAgICAgICAgICAgfCAgICAg ICAgICAgUmVzZXJ2ZWQgICAgICAgICAgICB8DQo+ICAgICAgICArLSstKy0rLSstKy0rLSstKy0r LSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKw0KPiAgICAgICAg fCAgIEFHSSBUeXBlICAgIHwgICAgTGVuZ3RoICAgICB8ICAgICAgVmFsdWUgICAgICAgICAgICAg ICAgICAgIHwNCj4gICAgICAgICstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSst Ky0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rDQo+ICAgICAgICB+ICAgICAgICAgICAgICAgICAg ICBBR0kgIFZhbHVlIChjb250ZC4pICAgICAgICAgICAgICAgICAgICAgICAgfg0KPiAgICAgICAg fCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgIHwNCj4gICAgICAgICstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSst Ky0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rDQo+ICAgICAgICB8ICAgQUlJIFR5cGUgICAgfCAg ICBMZW5ndGggICAgIHwgICAgICBWYWx1ZSAgICAgICAgICAgICAgICAgICAgfA0KPiAgICAgICAg Ky0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0r LSstKy0rLSsNCj4gICAgICAgIH4gICAgICAgICAgICAgICAgICAgU0FJSSAgVmFsdWUgKGNvbnRk LikgICAgICAgICAgICAgICAgICAgICAgICB+DQo+ICAgICAgICB8ICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfA0KPiAgICAgICAg Ky0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0r LSstKy0rLSsNCj4gICAgICAgIHwgICBBSUkgVHlwZSAgICB8ICAgIExlbmd0aCAgICAgfCAgICAg IFZhbHVlICAgICAgICAgICAgICAgICAgICB8DQo+ICAgICAgICArLSstKy0rLSstKy0rLSstKy0r LSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKw0KPiAgICAgICAg fiAgICAgICAgICAgICAgICAgICBUQUlJIFZhbHVlIChjb250ZC4pICAgICAgICAgICAgICAgICAg ICAgICAgIH4NCj4gICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICB8DQo+ICAgICAgICArLSstKy0rLSstKy0rLSstKy0r LSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKy0rLSstKw0KPg0KPiAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIEZpZ3VyZSAxNA0KPg0KPiAgICAgIC0gSW5n cmVzcyBQRSBBZGRyZXNzDQo+DQo+ICAgICAgICAgSVAgYWRkcmVzcyBvZiB0aGUgaW5ncmVzcyBQ RSBvZiBQVy4NCj4NCj4gICAgICAtIEVncmVzcyBQRSBBZGRyZXNzDQo+DQo+ICAgICAgICAgSVAg YWRkcmVzcyBvZiB0aGUgZWdyZXNzIFBFIG9mIFBXLg0KPg0KPiBTQj4gSVB2Nj8NCj4NCj4gICAg ICAtIENvbnRyb2wgd29yZCBiaXQgKEMpDQo+DQo+ICAgICAgICAgQSBiaXQgdGhhdCBmbGFncyB0 aGUgcHJlc2VuY2Ugb2YgYSBjb250cm9sIHdvcmQgb24gdGhpcyBQVy4gIElmIEMNCj4gICAgICAg ICA9IDEsIGNvbnRyb2wgd29yZCBpcyBwcmVzZW50OyBJZiBDID0gMCwgY29udHJvbCB3b3JkIGlz IG5vdA0KPiAgICAgICAgIHByZXNlbnQuDQo+DQo+ICAgICAgLSBQVyBUeXBlDQo+DQo+ICAgICAg ICAgQSAxNS1iaXQgcXVhbnRpdHkgdGhhdCByZXByZXNlbnRzIHRoZSB0eXBlIG9mIFBXLg0KPg0K Pg0KPg0KPiBZaW1pbiBTaGVuLCBldCBhbC4gICAgICBFeHBpcmVzIEphbnVhcnkgMjUsIDIwMTUg ICAgICAgICAgICAgICBbUGFnZSAyNF0NCj4NCj4NCj4gSW50ZXJuZXQtRHJhZnQgICAgIFBXIEVu ZHBvaW50IEZhc3QgRmFpbHVyZSBQcm90ZWN0aW9uICAgICAgICAgSnVseSAyMDE0DQo+DQo+DQo+ ICAgICAgLSBBR0kgVHlwZSwgTGVuZ3RoLCBWYWx1ZSwgQUdJIFZhbHVlDQo+DQo+ICAgICAgICAg QXR0YWNobWVudCBHcm91cCBJZGVudGlmaWVyIG9mIFBXLg0KPg0KPiAgICAgIC0gU0FJSSBUeXBl LCBMZW5ndGgsIFZhbHVlLCBTQUlJIFZhbHVlDQo+DQo+ICAgICAgICAgU291cmNlIEF0dGFjaG1l bnQgSW5kaXZpZHVhbCBJZGVudGlmaWVyIG9mIFBXLg0KPg0KPiAgICAgIC0gVEFJSSBUeXBlLCBM ZW5ndGgsIFZhbHVlLCBUQUlJIFZhbHVlDQo+DQo+ICAgICAgICAgVGFyZ2V0IEF0dGFjaG1lbnQg SW5kaXZpZHVhbCBJZGVudGlmaWVyIG9mIFBXLg0KPg0KPiA2LiAgUmV2ZXJ0aXZlIEJlaGF2aW9y DQo+DQo+ICAgICAgU3Vic2VxdWVudCB0byBsb2NhbCByZXBhaXIsIHRoZXJlIGFyZSB0aHJlZSBz dHJhdGVnaWVzIGZvciB0aGUNCj4gICAgICBuZXR3b3JrIHRvIHJlc3RvcmUgdHJhZmZpYyB0byBh IGZ1bGx5IGZ1bmN0aW9uYWwgUFcuDQo+DQo+ICAgICAgbyAgR2xvYmFsIHJldmVydGl2ZSBtb2Rl DQo+DQo+ICAgICAgICAgSWYgdGhlIGluZ3Jlc3MgQ0UgaXMgbXVsdGktaG9tZWQgKEZpZ3VyZSAx KSwgaXQgTUFZIHN3aXRjaCB0aGUNCj4gICAgICAgICB0cmFmZmljIHRvIGEgYmFja3VwIEFDIHdo aWNoIGlzIGJvdW5kIHRvIGEgYmFja3VwIFBXLg0KPiAgICAgICAgIEFsdGVybmF0aXZlbHksIGlm IHRoZSBpbmdyZXNzIFBFIGhvc3RzIGEgYmFja3VwIFBXIChGaWd1cmUgMiksIHRoZQ0KPiAgICAg ICAgIGluZ3Jlc3MgUEUgTUFZIHN3aXRjaCB0aGUgdHJhZmZpYyB0byB0aGUgYmFja3VwIFBXLiAg VGhlc2UNCj4gICAgICAgICBwcm9jZWR1cmVzIGFyZSByZWZlcnJlZCB0byBhcyBnbG9iYWwgcmVw YWlyLiAgUG9zc2libGUgdHJpZ2dlcnMgb2YNCj4gICAgICAgICBhIGdsb2JhbCByZXBhaXIgaW5j bHVkZSBQVyBzdGF0dXMsIE9BTSwgYW5kIEJGRC4NCj4NCj4gICAgICBvICBDb250cm9sIHBsYW5l IHJldmVydGl2ZSBtb2RlDQo+DQo+ICAgICAgICAgSW4gZWdyZXNzIFBFIG5vZGUgcHJvdGVjdGlv biBhbmQgUy1QRSBub2RlIHByb3RlY3Rpb24sIGl0IGlzDQo+ICAgICAgICAgcG9zc2libGUgdGhh dCB0aGUgZmFpbHVyZSBpcyBsaW1pdGVkIHRvIHRoZSBsaW5rIGJldHdlZW4gdGhlIFBMUg0KPiAg ICAgICAgIGFuZCB0aGUgcHJpbWFyeSAoUy0pUEUsIHdoZXJlYXMgdGhlIHByaW1hcnkgKFMtKVBF IGlzIHN0aWxsIHVwLg0KPiAgICAgICAgIEluIHRoaXMgY2FzZSwgdGhlIFBMUiBvciBhbiB1cHN0 cmVhbSByb3V0ZXIgYWxvbmcgdGhlIHRyYW5zcG9ydA0KPiAgICAgICAgIHR1bm5lbCBNQVkgcmVy b3V0ZSB0aGUgdHVubmVsIGFyb3VuZCB0aGUgZmFpbGVkIGxpbmsgdmlhIGFuDQo+ICAgICAgICAg YWx0ZXJuYXRpdmUgcGF0aC4gIFRodXMsIHRoZSB0cmFuc3BvcnQgdHVubmVsIGNhbiBjb250aW51 ZSB0byBiZQ0KPiAgICAgICAgIHVzZWQgdG8gY2FycnkgdGhlIFBXIHRyYWZmaWMgdG8gdGhlIHBy aW1hcnkgKFMtKVBFLiAgVGhpcw0KPiAgICAgICAgIHByb2NlZHVyZSBpcyBkcml2ZW4gYnkgY29u dHJvbCBwbGFuZSBjb252ZXJnZW5jZSwgYW5kIGlzIHJlZmVycmVkDQo+ICAgICAgICAgdG8gYXMg Y29udHJvbCBwbGFuZSByZXBhaXIuDQo+DQo+ICAgICAgbyAgTG9jYWwgcmV2ZXJ0aXZlIG1vZGUN Cj4NCj4gICAgICAgICBUaGUgUExSIE1BWSBtb3ZlIHRyYWZmaWMgYmFjayB0byB0aGUgcHJpbWFy eSBQVywgYWZ0ZXIgdGhlIGZhaWx1cmUNCj4gICAgICAgICBpcyByZXNvbHZlZC4gIEluIGVncmVz cyBBQyBwcm90ZWN0aW9uLCB1cG9uIGRldGVjdGluZyB0aGF0IHRoZQ0KPiAgICAgICAgIHByaW1h cnkgQUMgaXMgcmVzdG9yZWQsIHRoZSBQTFIgTUFZIHN0YXJ0IGZvcndhcmRpbmcgdHJhZmZpYyBv dmVyDQo+ICAgICAgICAgdGhlIEFDIGFnYWluLiAgTGlrZXdpc2UsIGluIGVncmVzcyBQRSBub2Rl IHByb3RlY3Rpb24gYW5kIFMtUEUNCj4gICAgICAgICBub2RlIHByb3RlY3Rpb24sIHVwb24gZGV0 ZWN0aW5nIHRoYXQgdGhlIHByaW1hcnkgUEUgaXMgcmVzdG9yZWQsDQo+ICAgICAgICAgdGhlIFBM UiBNQVkgcmUtZXN0YWJsaXNoIHRoZSBwcmltYXJ5IHRyYW5zcG9ydCB0dW5uZWwgdGhyb3VnaCB0 aGUNCj4gICAgICAgICBwcmltYXJ5IFBFLCBhbmQgbW92ZSB0aGUgdHJhZmZpYyBmcm9tIHRoZSBi eXBhc3MgdHVubmVsIGJhY2sgdG8NCj4NCj4NCj4NCj4NCj4gWWltaW4gU2hlbiwgZXQgYWwuICAg ICAgRXhwaXJlcyBKYW51YXJ5IDI1LCAyMDE1ICAgICAgICAgICAgICAgW1BhZ2UgMjVdDQo+DQo+ DQo+IEludGVybmV0LURyYWZ0ICAgICBQVyBFbmRwb2ludCBGYXN0IEZhaWx1cmUgUHJvdGVjdGlv biAgICAgICAgIEp1bHkgMjAxNA0KPg0KPg0KPiAgICAgICAgIHRoZSB0cmFuc3BvcnQgdHVubmVs LiAgVGhlc2UgcHJvY2VkdXJlcyBhcmUgcmVmZXJyZWQgdG8gYXMgbG9jYWwNCj4gICAgICAgICBy ZXZlcnNpb24uDQo+DQo+ICAgICAgVGhlIGZhc3QgcHJvdGVjdGlvbiBtZWNoYW5pc20gaW4gdGhp cyBkb2N1bWVudCBTSE9VTEQgYmUgdXNlZCBpbg0KPiAgICAgIHRhbmRlbSB3aXRoIHRoZSBnbG9i YWwgcmV2ZXJ0aXZlIG1vZGUuICBQYXJ0aWN1bGFybHkgaW4gdGhlIGNhc2Ugb2YNCj4gICAgICBl Z3Jlc3MgKFMtKVBFIGZhaWx1cmUsIGlmIHRoZSBpbmdyZXNzIFBFIG9yIHRoZSBwcm90ZWN0b3Ig bG9zZXMNCj4gICAgICBjb21tdW5pY2F0aW9uIHdpdGggdGhlIChTLSlQRSBmb3IgYW4gZXh0ZW5z aXZlIHBlcmlvZCBvZiB0aW1lLCB0aGUNCj4gICAgICBMRFAgc2Vzc2lvbiBiZXR3ZWVuIHRoZW0g bWF5IGdvIGRvd24uICBDb25zZXF1ZW50bHksIHRoZSBpbmdyZXNzIFBFDQo+ICAgICAgbWF5IGJy aW5nIGRvd24gdGhlIHByaW1hcnkgUFcsIG9yIHRoZSBwcm90ZWN0b3IgbWF5IHJlbW92ZSB0aGUN Cj4gICAgICBmb3J3YXJkaW5nIGVudHJ5IG9mIHRoZSBwcmltYXJ5IFBXIGxhYmVsLiAgSW4gZWl0 aGVyIGNhc2UsIHRoZQ0KPiAgICAgIHNlcnZpY2Ugd2lsbCBiZSBkaXNydXB0ZWQuICBJbiBvdGhl ciB3b3JkcywgYWx0aG91Z2ggdGhlIGZhc3QNCj4gICAgICBwcm90ZWN0aW9uIGNhbiB0ZW1wb3Jh cmlseSByZXBhaXIgdHJhZmZpYywgY29udHJvbCBwbGFuZSBzdGF0ZSBtYXkNCj4gICAgICBldmVu dHVhbGx5IGJlIHRpbWVkIG91dCBpZiB0aGUgZmFpbHVyZSBwZXJzaXN0cy4gIFRoZXJlZm9yZSwg aXQgaXMNCj4gICAgICByZWNvbW1lbmRlZCB0aGF0IHRoZSBnbG9iYWwgcmV2ZXJ0aXZlIG1vZGUg U0hPVUxEIGJlIHNldCB1cCBpbg0KPiAgICAgIGFkdmFuY2UsIHNvIHRoYXQgdHJhZmZpYyBjYW4g YmUgbW92ZWQgdG8gYSBmdWxseSBmdW5jdGlvbmFsIGJhY2t1cCBQVw0KPiAgICAgIHNob3J0bHkg YWZ0ZXIgdGhlIGxvY2FsIHJlcGFpci4NCj4NCj4gICAgICBUaGUgY29udHJvbCBwbGFuZSByZXZl cnRpdmUgbW9kZSBtYXkgYWx3YXlzIGhhcHBlbiBhcyBwYXJ0IG9mIHRoZQ0KPiAgICAgIGNvbnZl cmdlbmNlIG9mIGNvbnRyb2wgcGxhbmUgcHJvdG9jb2xzLiAgSG93ZXZlciwgaXQgaXMgb25seQ0K PiAgICAgIGFwcGxpY2FibGUgdG8gdGhlIHNwZWNpZmljIHNjZW5hcmlvcyBkZXNjcmliZWQgYWJv dmUuDQo+DQo+ICAgICAgVGhlIGxvY2FsIHJldmVydGl2ZSBtb2RlIGlzIG9wdGlvbmFsLiAgSW4g dGhlIGNpcmN1bXN0YW5jZXMgd2hlcmUgdGhlDQo+ICAgICAgZmFpbHVyZSBpcyBjYXVzZWQgYnkg cmVzb3VyY2UgZmxhcHBpbmcsIGxvY2FsIHJldmVyc2lvbiBNQVkgYmUNCj4gICAgICBkYW1wZW5l ZCB0byBsaW1pdCBwb3RlbnRpYWwgZGlzcnVwdGlvbnMuICBMb2NhbCByZXZlcnRpdmUgbW9kZSBN QVkgYmUNCj4gICAgICBkaXNhYmxlZCBjb21wbGV0ZWx5IGJ5IGNvbmZpZ3VyYXRpb24uDQo+DQo+ IDcuICBJQU5BIENvbnNpZGVyYXRpb25zDQo+DQo+ICAgICAgVGhpcyBkb2N1bWVudCBkZWZpbmVz IHRoZSBlbmNvZGluZyBvZiB0aGUgQ2FwYWJpbGl0eSBQYXJhbWV0ZXIgVExWDQo+ICAgICAgZm9y IHRoZSBuZXcgIkVncmVzcyBQcm90ZWN0aW9uIENhcGFiaWxpdHkiIGluIFNlY3Rpb24gNS4gIFRo aXMgd291bGQNCj4gICAgICByZXF1aXJlIElBTkEgdG8gYXNzaWduIGEgVExWIENvZGUgUG9pbnQg dG8gaXQuDQo+DQo+ICAgICAgVGhpcyBkb2N1bWVudCBkZWZpbmVzIGEgbmV3IExEUCBQcm90ZWN0 aW9uIEZFQyBFbGVtZW50IFRMViBpbg0KPiAgICAgIFNlY3Rpb24gNS4gIElBTkEgaGFzIGFzc2ln bmVkIHRoZSB0eXBlIHZhbHVlIDB4ODMgdG8gaXQuDQo+DQo+IFNCPiBJIGFtIHZlcnkgY29uZnVz ZWQgYWJvdXQgd2hhdCBleHBsaWNpdCBhY3Rpb25zIHlvdSBhcmUgYXNraW5nIElBTkENCj4gdG8g dGFrZS4gQXJlIHlvdSBzYXlpbmcgdGhhdCB0aGVyZSBhcmUgbm8gSUFOQSBhY3Rpb25zPw0KPg0K PiBbeXNoZW5dIE5lZWQgSUFOQSB0byBhc3NpZ24gYSBUTFYgY29kZSB0byAiRWdyZXNzIFByb3Rl Y3Rpb24gQ2FwYWJpbGl0eSIsIGFzIHNhaWQgYWJvdmUuDQo+DQo+IDguICBTZWN1cml0eSBDb25z aWRlcmF0aW9ucw0KPg0KPiAgICAgIFRoZSBzZWN1cml0eSBjb25zaWRlcmF0aW9ucyBkaXNjdXNz ZWQgaW4gUkZDIDUwMzYsIFJGQyA1MzMxLCBSRkMNCj4gICAgICAzMjA5LCBhbmQgUkZDIDQwOTAg YXBwbHkgdG8gdGhpcyBkb2N1bWVudC4NCj4NCj4gU0I+IEFyZSB0aGVyZSBhbnkgUFcgc2VjdXJp dHkgY29uc2lkZXJhdGlvbnMgdGhhdCBhcHBseT8NCj4NCj4gU0I+IFRoaXMgaXMgYSBuZXcgUFcg b3BlcmF0aW9uYWwgbW9kZSwgc28gSSBhbSBzdXJwcmlzZWQgdGhhdCB0aGVyZSBhcmUNCj4gbm8g bmV3IHNlY3VyaXR5IGNvbnNpZGVyYXRpb25zLiBBbiBlYXJseSBzZWN1cml0eSByZXZpZXcgbWln aHQgYmUgdXNlZnVsLg0KPg0KPg0KPiBbeXNoZW5dIE5vdCBzdXJlIGFib3V0IGFueSBzcGVjaWZp YyBzZWN1cml0eSBpc3N1ZXMuDQo+DQo+DQo+IF9fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fDQo+IHB3ZTMgbWFpbGluZyBsaXN0DQo+IHB3ZTNAaWV0Zi5vcmcN Cj4gaHR0cHM6Ly93d3cuaWV0Zi5vcmcvbWFpbG1hbi9saXN0aW5mby9wd2UzDQo+IC4NCj4NCg0K DQotLSANCkZvciBjb3Jwb3JhdGUgbGVnYWwgaW5mb3JtYXRpb24gZ28gdG86DQoNCmh0dHA6Ly93 d3cuY2lzY28uY29tL3dlYi9hYm91dC9kb2luZ19idXNpbmVzcy9sZWdhbC9jcmkvaW5kZXguaHRt bA0KDQo= From nobody Thu Aug 7 08:43:46 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 679751B2C02; Thu, 7 Aug 2014 08:43:43 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -14.502 X-Spam-Level: X-Spam-Status: No, score=-14.502 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nYmqA4lgq77b; Thu, 7 Aug 2014 08:43:41 -0700 (PDT) Received: from aer-iport-1.cisco.com (aer-iport-1.cisco.com [173.38.203.51]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C51701B2C1A; Thu, 7 Aug 2014 08:43:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=2343; q=dns/txt; s=iport; t=1407426219; x=1408635819; h=message-id:date:from:reply-to:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=iIXm7Ix4K/tBHxgqMAENNyzNdviMk7AWyA3v9cT/T9M=; b=EVOU9GzPtcZuH6dQZFIIVT5ggYCMc5sOgVMSFKtFZDfsiCNjVjIXiLgT bylq8j7Q8j2rvi19Q3MPPbb11b/OAxoWobZS0CSyPkTQZO9HEqHfSbVpG SIC3BkEqdIiPwYtoquZ1CTJFwnLODh7hiezqf4S6/v5tKCPD515fNEwdM w=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqQEAOCd41OtJssW/2dsb2JhbABahy3ODIMbAYEpd4QDAQEBAwEdBhU6BgEFCwsOCgICBRYLAgIJAwIBAgFFBgEMAQcBAReIHwitZ4Z/j0AXgSyIU4VNB4J5gVIBBJwblHCDWA X-IronPort-AV: E=Sophos;i="5.01,818,1400025600"; d="scan'208";a="136179009" Received: from aer-iport-nat.cisco.com (HELO aer-core-4.cisco.com) ([173.38.203.22]) by aer-iport-1.cisco.com with ESMTP; 07 Aug 2014 15:43:36 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by aer-core-4.cisco.com (8.14.5/8.14.5) with ESMTP id s77FhaEh013514 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 7 Aug 2014 15:43:36 GMT Received: from [IPv6:::1] (localhost [127.0.0.1]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id s77FhZXq010368; Thu, 7 Aug 2014 16:43:35 +0100 (BST) Message-ID: <53E39EA8.1040905@cisco.com> Date: Thu, 07 Aug 2014 16:43:36 +0100 From: Stewart Bryant User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Yimin Shen , pwe3 , "pwe3-chairs@tools.ietf.org" References: <53D7B569.60400@cisco.com> <53E0B85B.8060707@cisco.com> <3fd1f1049aff43c99cbd3ff14e1f6c30@BY2PR05MB728.namprd05.prod.outlook.com> In-Reply-To: <3fd1f1049aff43c99cbd3ff14e1f6c30@BY2PR05MB728.namprd05.prod.outlook.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/zvbQQBU-YrSqrrmfUrKtXCJ7NCM Cc: "mpls@ietf.org" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: stbryant@cisco.com List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Aug 2014 15:43:43 -0000 On 07/08/2014 16:01, Yimin Shen wrote: > Hi Stewart, > > Thanks very much again for the review and detailed comments. > > We will consider your suggestions on the LDP TLV structures defined in this draft. > > Regarding the tradeoff between the need for egress PE node protection and the increased state and complexity in PSN, this is a common considration for all IP/MPLS node protection, not PWE3 specific. SB> Agree > We do see the requirement from service providers. I think this should be their decision on a per network basis. When they decide to do it, we have a solution available for them. SB> Sure but what is the view of the WG. Do we want to recommend this approach as the best approach to the industry, or should we recommend a simpler approach that allows better layer separation. Perhaps some operators could comment on what their requirements are so we can better a better feel for how to make the complexity+scale trade off against protection coverage. SB> Remember a standards track RFC is a serious recommendation of good practice. > > That said, this draft doesn't necessarily multiply the number of PSN tunnels. It does require one tunnel per . SB> Surely it requires on per protection group? > However, if each primary PE is protected by only 1 protector, the number of tunnel will be the number of primary PEs, which is basically the same as the case where this mechanism is not used. SB> Not necessarily. This is a consequence of your method. Without context labels you deliver the traffic on the ordinary IP path to the alternative T-PE > This 1:1 relationship is achievable with the co-located model, and is especially achievable with the centralized model. Of course, in a more general case, if the avarage PE-to-protector ratio is 1:N, the number of tunnels will be multiplied by N. SB> With the centralized model - is this a PSN function, or is the centralized protector a type of S-PE? > But operators can always design their networks thoughtfully and manage this N to be lowever than the threshold that may lead to scalability problems. (Here, I'm not counting bypass tunnels in, because bypass tunnels exist in the existing IP/MPLS fast-reroute as well.) SB>Yes, but with some sort PW stitching model this simply does not apply. - Stewart From nobody Thu Aug 7 19:47:00 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B3B0E1A0023; Thu, 7 Aug 2014 19:46:56 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.202 X-Spam-Level: X-Spam-Status: No, score=-4.202 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yjbsao9Prby6; Thu, 7 Aug 2014 19:46:55 -0700 (PDT) Received: from lhrrgout.huawei.com (lhrrgout.huawei.com [194.213.3.17]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 67D321A002E; Thu, 7 Aug 2014 19:46:52 -0700 (PDT) Received: from 172.18.7.190 (EHLO lhreml406-hub.china.huawei.com) ([172.18.7.190]) by lhrrg02-dlp.huawei.com (MOS 4.3.7-GA FastPath queued) with ESMTP id BIA43629; Fri, 08 Aug 2014 02:46:50 +0000 (GMT) Received: from NKGEML404-HUB.china.huawei.com (10.98.56.35) by lhreml406-hub.china.huawei.com (10.201.5.243) with Microsoft SMTP Server (TLS) id 14.3.158.1; Fri, 8 Aug 2014 03:46:49 +0100 Received: from NKGEML512-MBX.china.huawei.com ([169.254.7.78]) by nkgeml404-hub.china.huawei.com ([10.98.56.35]) with mapi id 14.03.0158.001; Fri, 8 Aug 2014 10:46:45 +0800 From: Mingui Zhang To: "Stewart Bryant (stbryant)" Thread-Topic: [PWE3] [mpls] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 Thread-Index: AQHPsY7DDiJw7y3ZEk+Q1kSqiialjpvEXftQgABg1hiAACg8oA== Date: Fri, 8 Aug 2014 02:46:44 +0000 Message-ID: <4552F0907735844E9204A62BBDD325E76AAAB6F9@nkgeml512-mbx.china.huawei.com> References: Your message of Tue, 05 Aug 2014 13:59:49 -0000. <9696d0db139d46ffaad7be11340215e8@AM3PR03MB612.eurprd03.prod.outlook.com> <16167.1407340459@erosen-lnx>, <4552F0907735844E9204A62BBDD325E76AAAA7F4@nkgeml512-mbx.china.huawei.com> <22D4AECA-2D36-4F79-98CB-96E4B9BDC126@cisco.com> In-Reply-To: <22D4AECA-2D36-4F79-98CB-96E4B9BDC126@cisco.com> Accept-Language: en-US, zh-CN Content-Language: zh-CN X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.111.102.175] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-CFilter-Loop: Reflected Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/6b9Wb-ITT1d1QzPHNDgWz1VdxnI Cc: "mpls@ietf.org" , "Eric Rosen \(erosen\)" , pwe3 , "pwe3-chairs@tools.ietf.org" Subject: Re: [PWE3] [mpls] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Aug 2014 02:46:56 -0000 Hi Stewart, I think authors would say the S-PE stitching method involves the control pl= ane processing during the repair procedure. They emphasized their method us= es data plane & local repair, which can be faster. Here, I want to raise one issue:=20 If the primary PE fails and the PLR redirects the traffic during the flying= of the packets, hoping the backup PE delivers the packets immediately. Thi= s means the AC at the backup PE side is also ACTIVE. So the CE has active-a= ctive connections to both egress PEs. This is obviously different from the = PW-RED's active-standby mechanism [RFC6718]. Will this difference bring us = the frame duplication issue? I think we need to address this in the updated= version.=20 Thanks, Mingui >-----Original Message----- >From: Stewart Bryant (stbryant) [mailto:stbryant@cisco.com] >Sent: Thursday, August 07, 2014 3:27 PM >To: Mingui Zhang >Cc: Eric Rosen (erosen); Alexander Vainshtein; mpls@ietf.org; pwe3; >pwe3-chairs@tools.ietf.org >Subject: Re: [PWE3] [mpls] WG Last Call for >draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 > > > >Sent from my iPad > >> On 7 Aug 2014, at 04:57, "Mingui Zhang" wrote: >> >> Hi Eric, >> >> The explanation of the story is very clear. Let me complement a bit. >> >>> - A "primary egress PE" partitions its set of PWs into n sets, where ea= ch >>> set is associated with a given "protector". The "protector" is the bac= kup >>> egress PE for those PWs. >> >> When the backup PE acts as the protector, it is the 'co-located' model. > >This case does not however need context labels. It is a form of S-PE and c= an >stitch the PWs just like every existing S-PE already does. > >Stewart > From nobody Fri Aug 8 02:31:06 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id ECF1C1A0391; Fri, 8 Aug 2014 02:31:02 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.901 X-Spam-Level: X-Spam-Status: No, score=-1.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id T67Kow9_5ZB1; Fri, 8 Aug 2014 02:31:00 -0700 (PDT) Received: from pipi.pi.nu (pipi.pi.nu [83.168.239.141]) (using TLSv1.1 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5D5D21A0344; Fri, 8 Aug 2014 02:31:00 -0700 (PDT) Received: from [192.168.0.100] (81-236-221-144-no93.tbcn.telia.com [81.236.221.144]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: loa@pi.nu) by pipi.pi.nu (Postfix) with ESMTPSA id BD1FB18013DA; Fri, 8 Aug 2014 11:30:58 +0200 (CEST) Message-ID: <53E498D4.6000107@pi.nu> Date: Fri, 08 Aug 2014 11:31:00 +0200 From: Loa Andersson User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Mingui Zhang , "Stewart Bryant (stbryant)" References: Your message of Tue, 05 Aug 2014 13:59:49 -0000. <9696d0db139d46ffaad7be11340215e8@AM3PR03MB612.eurprd03.prod.outlook.com> <16167.1407340459@erosen-lnx>, <4552F0907735844E9204A62BBDD325E76AAAA7F4@nkgeml512-mbx.china.huawei.com> <22D4AECA-2D36-4F79-98CB-96E4B9BDC126@cisco.com> <4552F0907735844E9204A62BBDD325E76AAAB6F9@nkgeml512-mbx.china.huawei.com> In-Reply-To: <4552F0907735844E9204A62BBDD325E76AAAB6F9@nkgeml512-mbx.china.huawei.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/Y1ag46aycEnHh-_SzRhiPv8zRHE Cc: "mpls@ietf.org" , "Eric Rosen \(erosen\)" , pwe3 , "pwe3-chairs@tools.ietf.org" Subject: Re: [PWE3] [mpls] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Aug 2014 09:31:03 -0000 Authors, Mingui, Stewart On 2014-08-08 04:46, Mingui Zhang wrote: > Hi Stewart, > > I think authors would say the S-PE stitching method involves the control plane processing during the repair procedure. They emphasized their method uses data plane & local repair, which can be faster. > Here, I want to raise one issue: It seems that we take it for granted that the method proposed on this draft is faster than e.g. the e2e protection a la mpls-tp. Why is that? If we have implementations of both, do we have any real measurements? /Loa -- Loa Andersson email: loa@mail01.huawei.com Senior MPLS Expert loa@pi.nu Huawei Technologies (consultant) phone: +46 739 81 21 64 From nobody Fri Aug 8 02:54:30 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 34B191B29DA; Fri, 8 Aug 2014 02:54:26 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -14.502 X-Spam-Level: X-Spam-Status: No, score=-14.502 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KH9sSgYd1CgG; Fri, 8 Aug 2014 02:54:25 -0700 (PDT) Received: from aer-iport-1.cisco.com (aer-iport-1.cisco.com [173.38.203.51]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 852221B29D5; Fri, 8 Aug 2014 02:54:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=673; q=dns/txt; s=iport; t=1407491665; x=1408701265; h=message-id:date:from:reply-to:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=QesHiyeJrUPGN5bK6eIQIouA8kDFY5XiyldwV3R4slI=; b=fccJI7foOtoLOI/yXIH9PIPhnOfNfzi6nzWZp4rfwn3aI9EB410PlY9J T8+Av2xhZmqQl+6BgjBffsD61okpnZxSw0E9twXEwMZA5CxwzhIAvm1m2 Ga2Wxz/e0hlbSlx2DLdxKn9bDSsvczMlWWMofE1YwTGaO3amlNiReC7rH Q=; X-IronPort-AV: E=Sophos;i="5.01,824,1400025600"; d="scan'208";a="136958178" Received: from aer-iport-nat.cisco.com (HELO aer-core-4.cisco.com) ([173.38.203.22]) by aer-iport-1.cisco.com with ESMTP; 08 Aug 2014 09:54:22 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by aer-core-4.cisco.com (8.14.5/8.14.5) with ESMTP id s789sLHe028305 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 8 Aug 2014 09:54:22 GMT Received: from [IPv6:::1] (localhost [127.0.0.1]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id s789sJZw014864; Fri, 8 Aug 2014 10:54:19 +0100 (BST) Message-ID: <53E49E4D.4040905@cisco.com> Date: Fri, 08 Aug 2014 10:54:21 +0100 From: Stewart Bryant User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Mingui Zhang References: Your message of Tue, 05 Aug 2014 13:59:49 -0000. <9696d0db139d46ffaad7be11340215e8@AM3PR03MB612.eurprd03.prod.outlook.com> <16167.1407340459@erosen-lnx>, <4552F0907735844E9204A62BBDD325E76AAAA7F4@nkgeml512-mbx.china.huawei.com> <22D4AECA-2D36-4F79-98CB-96E4B9BDC126@cisco.com> <4552F0907735844E9204A62BBDD325E76AAAB6F9@nkgeml512-mbx.china.huawei.com> In-Reply-To: <4552F0907735844E9204A62BBDD325E76AAAB6F9@nkgeml512-mbx.china.huawei.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/vDC7D0REpO7j4F3gm4PU-3_LxWw Cc: "mpls@ietf.org" , "Eric Rosen \(erosen\)" , pwe3 , "pwe3-chairs@tools.ietf.org" Subject: Re: [PWE3] [mpls] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: stbryant@cisco.com List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Aug 2014 09:54:26 -0000 On 08/08/2014 03:46, Mingui Zhang wrote: > Hi Stewart, > > I think authors would say the S-PE stitching method involves the control plane processing during the repair procedure. Then they would be incorrect. The update of the PW entry in the FIB of an SPE is identical to the operation needed to update the FIB for any other FIB entry when FRR is triggered. Now there are more entries to update, but the degree of difficult presented will be implementation specific, but then so is the availability of context labels. > They emphasized their method uses data plane & local repair, which can be faster. So would an FRR repair in the PW plane. Stewart From nobody Fri Aug 8 06:59:58 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B637F1B2A11 for ; Fri, 8 Aug 2014 06:59:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.302 X-Spam-Level: X-Spam-Status: No, score=-4.302 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id P7uUicGIyclR for ; Fri, 8 Aug 2014 06:59:53 -0700 (PDT) Received: from mailuogwdur.emc.com (mailuogwdur.emc.com [128.221.224.79]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A0AD51B29E1 for ; Fri, 8 Aug 2014 06:59:53 -0700 (PDT) Received: from maildlpprd54.lss.emc.com (maildlpprd54.lss.emc.com [10.106.48.158]) by mailuogwprd51.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s78DxpIn019862 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 8 Aug 2014 09:59:52 -0400 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd51.lss.emc.com s78DxpIn019862 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=emc.com; s=jan2013; t=1407506392; bh=6mUv1JzDbywFmX4n2NX0e2PRTNk=; h=From:To:Date:Subject:Message-ID:Content-Type: Content-Transfer-Encoding:MIME-Version; b=CN+H7mIGtt0orI7nSMTRwsZSC0LiMp/mgTPVsBawsJA6fsPzeXkyQqhuvkD/X0oha 9H56oXz8oIv66D7ffiDx/2AsxpWUOvXIssPf5kspBRkvCFRh6W/d7dleHVdlfBSYFc 2HU/fVVRkDRPoPRUOfzY5fYlbmcCSq4Cw/HZxZnw= X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd51.lss.emc.com s78DxpIn019862 Received: from mailusrhubprd52.lss.emc.com (mailusrhubprd52.lss.emc.com [10.106.48.25]) by maildlpprd54.lss.emc.com (RSA Interceptor); Fri, 8 Aug 2014 09:59:45 -0400 Received: from mxhub07.corp.emc.com (mxhub07.corp.emc.com [128.222.70.204]) by mailusrhubprd52.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s78DxidE028258 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Fri, 8 Aug 2014 09:59:44 -0400 Received: from mx15a.corp.emc.com ([169.254.1.175]) by mxhub07.corp.emc.com ([128.222.70.204]) with mapi; Fri, 8 Aug 2014 09:59:43 -0400 From: "Black, David" To: Yaakov Stein , "pwe3@ietf.org" Date: Fri, 8 Aug 2014 09:59:43 -0400 Thread-Topic: WGLC comments on: draft-ietf-pwe3-congcons-02.txt Thread-Index: Ac+zEP27iWho+8uAQLKBEFcFeq8EIg== Message-ID: <8D3D17ACE214DC429325B2B98F3AE712077B9517F4@MX15A.corp.emc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd52.lss.emc.com X-RSA-Classifications: public Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/cow7WBz-oDv0N6gtiYK7S12Yyqk Subject: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Aug 2014 13:59:55 -0000 As we're now moving quickly to try to complete this draft , I agreed w/Yaakov that my author comments would be sent in as WG Last Call comments = ... ... so here they are: Major issues: The reference to a transport circuit breaker in section 3 is a bit optimist= ic and high level: We can accomplish this by employing a transport circuit breaker, by which we mean an automatic mechanism for terminating a flow to prevent negative impact on other flows and on the stability of the network [I-D.fairhurst-tsvwg-circuit-breaker]. This should be accompanied by at least an example of how to do this - past discussion has suggested carrier monitoring of delivered service quality in order to shut down PWs that are experiencing loss rates that result in unacceptable service quality (as defined for us by the ITU ...). I'd also like to add some text to suggest that operators use this sort of circuit breaker practice when TDM PWs compete with elastic traffic. Figure 1 and Figure 2 indicate that based on using the TCP throughput equation as a proxy for potential impact on elastic traffic (which is what we're doing in this draft), for the same fixed bandwidth PW, a larger segme= nt size is less likely to cause impacts. I'd like to add a suggestion to use the larger PW segment sizes as long as they're consistent with delivery of the service. There's been a related (long-to-the-point-of-ratholed) debate over in the Transport Area about whether packet-rate-limited/congestible forwarding is still relevant. My view is that such equipment exists, and hence that this is a relevant consideration for this draft, and would not mention that issue here. Minor issues: Section 2 is about Ethernet PWs that carry elastic traffic. Should others be mentioned as also covered by this analysis - e.g., ATM and Frame Relay PWs carrying elastic traffic? Section 2 does not cover Ethernet PWs carrying inelastic traffic - that wou= ld be unusual, but should it be explicitly noted as out of scope, or just not discussed (as is the case for the current text)? Section 3 should say that the reason the draft focuses on TDM PWs is that they're the inelastic PWs that are likely to be carried over IP in practice= . My understanding is that other potentially inelastic PWs are MPLS-only in practice. A terminology section to roll up all the acronym expansions (e.g., TDM, CBR= ) in one place seems like a good idea. This text on p.11 could be incendiary: Under this condition it is safe to place the TDM PW along with congestion - responsive traffic such as TCP, without causing additional congestion. First of all to reduce the flammability, "safe" -> "reasonable" and there are a number of other occurrences of "safe" that deserve similar treatment = ;-). Beyond that, this assertion is dependent on the operating conditions stayin= g within that reasonable operating envelope, which entails use of something like a circuit breaker - I'd say that here, and reference the earlier circuit breaker discussion. Figures 7 and 8 could use some shading or lines to indicate the typical deployed operating regions for the pseudo-wires to visibly show that they fall under the curves, particularly for larger segment sizes. I'd also repeat or refer to the suggestion to use larger=20 Nits: References aren't allowed in abstracts, so remove reference to [I-D.fairhurst-tsvwg-circuit-breaker] Introduction, first paragraph, delete "PWE3" towards end of paragraph - preferably the 4 byte PWE3 control word. In addition to its current general statements, the Security Considerations section should point to the security considerations sections in a few particularly relevant RFCs - I'd suggest 3985, 4553, 5086 and 5087. Thanks, --David > -----Original Message----- > From: pwe3 [mailto:pwe3-bounces@ietf.org] On Behalf Of Yaakov Stein > Sent: Thursday, July 24, 2014 4:52 PM > To: pwe3@ietf.org > Subject: Re: [PWE3] I-D Action: draft-ietf-pwe3-congcons-02.txt >=20 > Please look at the PDF version only > (the txt version does not have the graphs). >=20 > Y(J)S >=20 >=20 From nobody Fri Aug 8 07:00:44 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E61791B2A31; Fri, 8 Aug 2014 07:00:41 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.602 X-Spam-Level: X-Spam-Status: No, score=-2.602 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dZtueEZFWAf6; Fri, 8 Aug 2014 07:00:39 -0700 (PDT) Received: from na01-bl2-obe.outbound.protection.outlook.com (mail-bl2lp0211.outbound.protection.outlook.com [207.46.163.211]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CEBC71B29CD; Fri, 8 Aug 2014 07:00:38 -0700 (PDT) Received: from BY2PR05MB728.namprd05.prod.outlook.com (10.141.223.25) by BY2PR05MB728.namprd05.prod.outlook.com (10.141.223.25) with Microsoft SMTP Server (TLS) id 15.0.995.14; Fri, 8 Aug 2014 14:00:29 +0000 Received: from BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) by BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) with mapi id 15.00.0995.014; Fri, 8 Aug 2014 14:00:29 +0000 From: Yimin Shen To: Loa Andersson , Mingui Zhang , "Stewart Bryant (stbryant)" Thread-Topic: [PWE3] [mpls] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 Thread-Index: AQHPsut9xMpTT3UKUUu7y8CePcLxcpvGtyQQ Date: Fri, 8 Aug 2014 14:00:28 +0000 Message-ID: <496a2c137775477c8c78c7e8d5463c6e@BY2PR05MB728.namprd05.prod.outlook.com> References: Your message of Tue, 05 Aug 2014 13:59:49 -0000. <9696d0db139d46ffaad7be11340215e8@AM3PR03MB612.eurprd03.prod.outlook.com> <16167.1407340459@erosen-lnx>, <4552F0907735844E9204A62BBDD325E76AAAA7F4@nkgeml512-mbx.china.huawei.com> <22D4AECA-2D36-4F79-98CB-96E4B9BDC126@cisco.com> <4552F0907735844E9204A62BBDD325E76AAAB6F9@nkgeml512-mbx.china.huawei.com> <53E498D4.6000107@pi.nu> In-Reply-To: <53E498D4.6000107@pi.nu> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [66.129.241.10] x-microsoft-antispam: BCL:0;PCL:0;RULEID: x-forefront-prvs: 02973C87BC x-forefront-antispam-report: SFV:NSPM; SFS:(6009001)(377424004)(252514010)(164054003)(51704005)(13464003)(189002)(199002)(24454002)(377454003)(76576001)(50986999)(99396002)(74662001)(81342001)(106356001)(106116001)(76176999)(54356999)(33646002)(93886004)(74316001)(80022001)(85306004)(79102001)(66066001)(105586002)(81542001)(99286002)(95666004)(64706001)(20776003)(4396001)(83322001)(76482001)(19580405001)(15975445006)(74502001)(86362001)(2656002)(46102001)(107046002)(21056001)(19580395003)(85852003)(87936001)(77982001)(101416001)(83072002)(24736002)(108616003); DIR:OUT; SFP:; SCL:1; SRVR:BY2PR05MB728; H:BY2PR05MB728.namprd05.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; LANG:en; Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: juniper.net Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/8YEHJp7ftHEKkBzM2lSlwTJqyKg Cc: "mpls@ietf.org" , "Eric Rosen \(erosen\)" , pwe3 , "pwe3-chairs@tools.ietf.org" Subject: Re: [PWE3] [mpls] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Aug 2014 14:00:42 -0000 Hi Loa, This draft assumes decoupled control plane and dataplane architecture of ro= uters, and it requires that the backup path (or backup forwarding state) be= pre-programmed in the dataplane on PLR. Hence when the dataplane (of PLR) = detects a failure, it is able to restore traffic locally and independently = by redirecting traffic over the preprogrammed backup path. This is generall= y assumed to be faster that mechanisms that involve control plane protocols= , or end-to-end messaging or OAM, because there is no event propagation del= ay. I believe this is a common assumption for all IP/MPLS FRR mechanisms. It will not be a problem if this mechanism and an e2e mechanism are both im= plemented in a network. As described in the draft, this mechanism provides = local repair, and it is viewed as a temporary repair. The draft strongly re= commend this mechanism to be used in tandem with an e2e or global repair me= chanism, so that the later can eventually move traffic from the locally rep= aired PW over to a fully functional PW, as a permanent repair. Thanks, /Yimin -----Original Message----- From: pwe3 [mailto:pwe3-bounces@ietf.org] On Behalf Of Loa Andersson Sent: Friday, August 08, 2014 5:31 AM To: Mingui Zhang; Stewart Bryant (stbryant) Cc: mpls@ietf.org; Eric Rosen (erosen); pwe3; pwe3-chairs@tools.ietf.org Subject: Re: [PWE3] [mpls] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01 - RFC4447 Authors, Mingui, Stewart On 2014-08-08 04:46, Mingui Zhang wrote: > Hi Stewart, > > I think authors would say the S-PE stitching method involves the control = plane processing during the repair procedure. They emphasized their method = uses data plane & local repair, which can be faster. > Here, I want to raise one issue: It seems that we take it for granted that the method proposed on this draft= is faster than e.g. the e2e protection a la mpls-tp. Why is that? If we have implementations of both, do we have any real measurements? /Loa --=20 Loa Andersson email: loa@mail01.huawei.com Senior MPLS Expert loa@pi.nu Huawei Technologies (consultant) phone: +46 739 81 21 64 _______________________________________________ pwe3 mailing list pwe3@ietf.org https://www.ietf.org/mailman/listinfo/pwe3 From nobody Fri Aug 8 07:02:57 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BC0AE1B29E1; Fri, 8 Aug 2014 07:02:53 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.602 X-Spam-Level: X-Spam-Status: No, score=-2.602 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ynowV0AuPLsV; Fri, 8 Aug 2014 07:02:52 -0700 (PDT) Received: from na01-by2-obe.outbound.protection.outlook.com (mail-by2lp0235.outbound.protection.outlook.com [207.46.163.235]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 22E2A1B2A54; Fri, 8 Aug 2014 07:02:52 -0700 (PDT) Received: from BLUPR05CA0074.namprd05.prod.outlook.com (10.141.20.44) by DM2PR05MB733.namprd05.prod.outlook.com (10.141.178.18) with Microsoft SMTP Server (TLS) id 15.0.995.14; Fri, 8 Aug 2014 14:02:50 +0000 Received: from BL2FFO11FD044.protection.gbl (2a01:111:f400:7c09::130) by BLUPR05CA0074.outlook.office365.com (2a01:111:e400:855::44) with Microsoft SMTP Server (TLS) id 15.0.1005.10 via Frontend Transport; Fri, 8 Aug 2014 14:02:49 +0000 Received: from P-EMF01-SAC.jnpr.net (66.129.239.15) by BL2FFO11FD044.mail.protection.outlook.com (10.173.161.140) with Microsoft SMTP Server (TLS) id 15.0.990.10 via Frontend Transport; Fri, 8 Aug 2014 14:02:49 +0000 Received: from magenta.juniper.net (172.17.27.123) by P-EMF01-SAC.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.146.0; Fri, 8 Aug 2014 07:02:47 -0700 Received: from juniper.net (sapphire.juniper.net [172.17.28.108]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id s78E2kn63116; Fri, 8 Aug 2014 07:02:46 -0700 (PDT) (envelope-from yakov@juniper.net) Message-ID: <201408081402.s78E2kn63116@magenta.juniper.net> To: Loa Andersson In-Reply-To: <53E498D4.6000107@pi.nu> References: Your message of Tue, 05 Aug 2014 13:59:49 -0000. <9696d0db139d46ffaad7be11340215e8@AM3PR03MB612.eurprd03.prod.outlook.com> <16167.1407340459@erosen-lnx>, <4552F0907735844E9204A62BBDD325E76AAAA7F4@nkgeml512-mbx.china.huawei.com> <22D4AECA-2D36-4F79-98CB-96E4B9BDC126@cisco.com> <4552F0907735844E9204A62BBDD325E76AAAB6F9@nkgeml512-mbx.china.huawei.com> <53E498D4.6000107@pi.nu> X-MH-In-Reply-To: Loa Andersson message dated "Fri, 08 Aug 2014 11:31:00 +0200." MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <95677.1407506566.1@juniper.net> Date: Fri, 8 Aug 2014 07:02:46 -0700 From: Yakov Rekhter X-EOPAttributedMessage: 0 X-Forefront-Antispam-Report: CIP:66.129.239.15; CTRY:US; IPV:NLI; IPV:NLI; EFV:NLI; SFV:NSPM; SFS:(6009001)(377424004)(199002)(189002)(24454002)(51704005)(81156004)(50986999)(110136001)(99396002)(102836001)(76482001)(77982001)(16796002)(107046002)(95666004)(54356999)(76176999)(93886004)(106466001)(105596002)(21056001)(85306004)(74662001)(44976005)(46406003)(97756001)(74502001)(97736001)(79102001)(81542001)(46102001)(64706001)(20776003)(68736004)(69596002)(4396001)(81342001)(80022001)(86362001)(83322001)(47776003)(23726002)(87936001)(6806004)(83072002)(50466002)(92726001)(84676001)(85852003); DIR:OUT; SFP:; SCL:1; SRVR:DM2PR05MB733; H:P-EMF01-SAC.jnpr.net; FPR:; MLV:sfv; PTR:InfoDomainNonexistent; MX:1; LANG:en; X-Microsoft-Antispam: BCL:0;PCL:0;RULEID: X-Forefront-PRVS: 02973C87BC Received-SPF: SoftFail (protection.outlook.com: domain of transitioning juniper.net discourages use of 66.129.239.15 as permitted sender) Authentication-Results: spf=softfail (sender IP is 66.129.239.15) smtp.mailfrom=yakov@juniper.net; X-OriginatorOrg: juniper.net Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/kL4GLk_Wwd0dCT0sde2CE6-Azz0 Cc: "mpls@ietf.org" , "pwe3-chairs@tools.ietf.org" , pwe3 , "Eric Rosen \(erosen\)" , "Stewart Bryant \(stbryant\)" Subject: Re: [PWE3] [mpls] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Aug 2014 14:02:53 -0000 Loa, > Authors, Mingui, Stewart > > On 2014-08-08 04:46, Mingui Zhang wrote: > > Hi Stewart, > > > > I think authors would say the S-PE stitching method involves > > the control plane processing during the repair procedure. They > > emphasized their method uses data plane & local repair, which > > can be faster. > > Here, I want to raise one issue: > > It seems that we take it for granted that the method proposed on this > draft is faster than e.g. the e2e protection a la mpls-tp. Why is that? To answer your question let me quote from "Network Recovery" by JP Vasseur, Mario Pickavet, Piet Demeester (Section 5.8.1, page 336): With global protection, rerouting is performed by the head-end LSR, which means that this requires for the head-end LSR to receive the failure indication to reroute the affected traffic onto their respective backup paths (whose paths have been precomputed and signaled). So in terms of recovery time, the delta between global and local protection is the failure indication signal propagation time to the head-end LSR. Yakov. From nobody Fri Aug 8 07:34:32 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EB2FE1B2B23 for ; Fri, 8 Aug 2014 07:34:30 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.901 X-Spam-Level: X-Spam-Status: No, score=-1.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QQzBKILqSDML for ; Fri, 8 Aug 2014 07:34:25 -0700 (PDT) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1blp0184.outbound.protection.outlook.com [207.46.163.184]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7AEFB1B2C34 for ; Fri, 8 Aug 2014 07:34:24 -0700 (PDT) Received: from BY2PR05MB728.namprd05.prod.outlook.com (10.141.223.25) by BY2PR05MB221.namprd05.prod.outlook.com (10.242.41.154) with Microsoft SMTP Server (TLS) id 15.0.995.14; Fri, 8 Aug 2014 14:34:20 +0000 Received: from BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) by BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) with mapi id 15.00.0995.014; Fri, 8 Aug 2014 14:34:20 +0000 From: Yimin Shen To: Alexander Vainshtein Thread-Topic: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 Thread-Index: AQHPqByPzRsIO8QlbkK+kbMBZ9MDf5u3AuAAgAAZ0oCAABjewIABFRcAgADf/TCACF0AgIAA3TgQgADA4ICAADbOAIAAzcgwgAEfc4CAAYjQ4A== Date: Fri, 8 Aug 2014 14:34:19 +0000 Message-ID: <2afb5c963e054aef929f38dcb3defa07@BY2PR05MB728.namprd05.prod.outlook.com> References: <201407251524.s6PFOPn92012@magenta.juniper.net> <53D79504.4050904@cisco.com> <04f15bedf2be4c7480d3e9ca01bdfd7f@BY2PR05MB728.namprd05.prod.outlook.com> <2597291f7b074922b690e4b06999cf1a@AM3PR03MB612.eurprd03.prod.outlook.com>, <30e13900814f41a681151971cbe9ebef@BY2PR05MB728.namprd05.prod.outlook.com> <1407215578702.64387@ecitele.com>, <2280b8e0d5f94a60badab91a2237181b@BY2PR05MB728.namprd05.prod.outlook.com> <1407304503657.72312@ecitele.com> <046050cb2d254a139b4650b2d7f93075@BY2PR05MB728.namprd05.prod.outlook.com> <09ab595ab24a46a0a4e87ebeb2cd185a@AM3PR03MB612.eurprd03.prod.outlook.com> In-Reply-To: <09ab595ab24a46a0a4e87ebeb2cd185a@AM3PR03MB612.eurprd03.prod.outlook.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [66.129.241.10] x-microsoft-antispam: BCL:0;PCL:0;RULEID: x-forefront-prvs: 02973C87BC x-forefront-antispam-report: SFV:NSPM; SFS:(45624003)(252514010)(37854004)(377454003)(199002)(189002)(60444003)(43784003)(164054003)(76482001)(80022001)(2656002)(21056001)(110136001)(64706001)(66066001)(18717965001)(99396002)(31966008)(19609705001)(15975445006)(74316001)(19625215002)(74502001)(93886004)(107046002)(33646002)(95666004)(86362001)(19580405001)(101416001)(83322001)(19580395003)(46102001)(106116001)(81542001)(83072002)(81342001)(19300405004)(16236675004)(92566001)(15202345003)(87936001)(74662001)(4396001)(106356001)(20776003)(85306004)(76176999)(99286002)(76576001)(50986999)(54356999)(105586002)(85852003)(79102001)(77982001)(24736002)(108616003); DIR:OUT; SFP:; SCL:1; SRVR:BY2PR05MB221; H:BY2PR05MB728.namprd05.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; LANG:en; Content-Type: multipart/alternative; boundary="_000_2afb5c963e054aef929f38dcb3defa07BY2PR05MB728namprd05pro_" MIME-Version: 1.0 X-OriginatorOrg: juniper.net Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/SyGvSCt3HgrO_WLWZlH4TtKB9wo Cc: Yakov Rekhter , "pwe3@ietf.org" , "stbryant@cisco.com" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Aug 2014 14:34:31 -0000 --_000_2afb5c963e054aef929f38dcb3defa07BY2PR05MB728namprd05pro_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Sasha, Thanks again for the comments and questions! Regarding AC failure detection on PE, in the typical cases, if there is a f= iber cut, or a hardware failure affecting the port on the PE, or a hardware= failure affecting the port on the CE (assume the CE is directed attached t= o the PE), the PE should be able to directly and independently detect the f= ailure at physical or link layer. Regarding LDP, I agree with you that LDP by its nature is not as straight-f= orward as RSVP-TE in this scenario. Since a context ID are advertised by bo= th primary PE and protector, we should avoid creating an ECMP scenario. Tha= t's why section 4.2.2 provides a suggestion to advertise the context ID as = a proxy node (i.e. a virtual node attached to the two routers), and make it= s reachability via the primary PE more favorable than that via the protecto= r. MRT and other mechanisms could be used to help LDP improve protection co= verage. Also, nothing would prevent LDP from using RSVP bypass tunnels. As = you can see, there may be many different approaches, and there may new appr= oaches coming in the future. In this PWE3 document, we just want to stay at= PW level and be generic with PSN level. In any case, this mechanism is ach= ievable with LDP, and LDP as a tunnel signaling protocol really doesn't nee= d any new extension. Hope this clarifies it. Thanks, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Thursday, August 07, 2014 10:37 AM To: Yimin Shen Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin hi, Please see some comments inline below. Regards, Sasha Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 From: Yimin Shen [mailto:yshen@juniper.net] Sent: Thursday, August 07, 2014 5:34 AM To: Alexander Vainshtein Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Hi Sasha, Thanks again for the comments. Please see the following 4 items in response= to the 4 comments in your email. 1. Today when an egress PE detects an egress AC failure [[Sasha]] Could you please explain how the egress PE detects an egress AC f= ailure? I have always thought that egress AC failure has to be detected fir= st by corresponding CE which would then notify the egress PE using some sui= table "Remote Failure"/BDI/RDI mechanism. Specific examples would really he= lp. it will simply have to drop packets, until the remote PE stops sending traf= fic over the PW. This draft is intended to improve this behavior. By no mea= ns the local repair on egress PE will prevent the AC failure from being con= verted to and handled as a bi-directional AC failure. Note that bi-directio= nal AC failure handling must involve the CE, but the CE may detect the fail= ure at a slower pace, or it may not be able to detect it in a timely manner= , depending on the actual situation. [[Sasha]] Please see above. IMHO and FWIW if the CE cannot detect its "Rece= ive Failure", the PE would not never know about the "Transmit Failure. E.g. there may be other L2 devices between the PE and CE; the AC may itself= be a logical wire across another network; the type and location of the fai= lure may make it easier for the PE to detect it; and so on. In any case, be= fore the AC failure is handled more gracefully (or bi-directionally), the e= gress traffic has been locally repaired by the egress PE. 2. As mentioned before, this draft doesn't have a dependency on a specific = PSN tunnel technology. [[Sasha]] I have provided my comments on this in my response to Eric. From = my point of view some LDP-specific issues still require clarification, but = my bottom line is that operating the draft over an MPLS network that uses L= DP in DU mode for setting up tunnel LSPs may sometimes be possible but woul= d always be tricky. This does not match my understanding of "no dependency = on a specific PSN tunnel technology", but this may be a matter of taste. 3. The draft already talks about scalability when describing the centralize= d protector model vs the co-located protector model. We can improve the tex= t to explicitly mention that the requirement for setting up LSPs on a per <= primary PE, protector> basis rather than per primary PE basis can potential= ly increase the number of LSPs in a network. This is a tradeoff. As mention= ed before, this does not necessarily cause a scaling issue. [[Sasha]] Yes, I have found the corresponding text in the draft. From my p= oint of view this issue is resolved. 4. We will leave this to PWE3 WG to decide whether this is really necessary= .[[Sasha]] I tend to agree with Eric that usage of context-specific labels = in the draft does not mean update to 4447. From my point of view this issue= is resolved as well. Thanks, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Wednesday, August 06, 2014 5:11 AM To: Yimin Shen Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin and all, I would like to summarize the issues that I have with this draft. Digging f= or them in a very long email thread with multiple levels of in-lining is no= t very effective IMO. 1. Protection against unidirectional AC failures looks problematic to= me: a. The draft explicitly differentiates between "ingress" and "egress"= AC failures b. The outcome can be using a different AC for each direction of CE-to= -CE traffic with potentially destructive effect that cannot be mitigated at= the PWE3 level c. IMHO and FWIW a viable alternative could be to employ native AC me= chanisms converting unidirectional AC failures into bi-directional ones. Th= is would eliminate the differentiation between ingress and egress AC failur= es and facilitate using the PW redundancy mechanism for bi-directional prot= ection 2. Dependency upon specific mechanisms for setting up PSN tunnels see= ms to break some common PWE3 architectural principles: a. I am pretty well sure that the mechanisms specified in the draft c= annot be used with the PSN tunnel LSPs set up by LDP in DU mode b. If the authors, refer to LDP in the DoD mode augmented with some CR= -LDP-like mechanisms (e.g., for exclusion of specific links when setting up= the bypass LSP between the PLR and the Protector), this should, at the ver= y least, be discussed with and approved by the MPLS WG 3. Scalability of the proposed solution should be, at least, explicit= ly presented. In particular, the draft assumes that a dedicated LSP would b= e used between PE1 PE2 for each combination of PE2 with a specific Protect= or LSR. 4. The Protector LSR could be either a kind of S-PE (if different fro= m the secondary T-PE) or the secondary T-PE itself. In both cases it should= be capable of handling labels from specific context spaces as if they were= PW labels. This requires (at the very least) an update of RFC 4447 and sho= uld be explicitly presented and discussed. Hopefully this summary will be useful. Regards, Sasha Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 --_000_2afb5c963e054aef929f38dcb3defa07BY2PR05MB728namprd05pro_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi Sasha,

 

Thanks again for the c= omments and questions!

 

Regarding AC failure d= etection on PE, in the typical cases, if there is a fiber cut, or a hardwar= e failure affecting the port on the PE, or a hardware failure affecting the= port on the CE (assume the CE is directed attached to the PE), the PE should be able to directly and independently d= etect the failure at physical or link layer.

 

Regarding LDP, I agree= with you that LDP by its nature is not as straight-forward as RSVP-TE in t= his scenario. Since a context ID are advertised by both primary PE and prot= ector, we should avoid creating an ECMP scenario. That’s why section 4.2.2 provides a suggestion to advertis= e the context ID as a proxy node (i.e. a virtual node attached to the two r= outers), and make its reachability via the primary PE more favorable than t= hat via the protector. MRT and other mechanisms could be used to help LDP improve protection coverage. Also, nothing would= prevent LDP from using RSVP bypass tunnels. As you can see, there may be m= any different approaches, and there may new approaches coming in the future= . In this PWE3 document, we just want to stay at PW level and be generic with PSN level. In any case, this = mechanism is achievable with LDP, and LDP as a tunnel signaling protocol re= ally doesn’t need any new extension.  

 

Hope this clarifies it= .

 

Thanks,

 

/Yimin

 

 

From: Alexande= r Vainshtein [mailto:Alexander.Vainshtein@ecitele.com]
Sent: Thursday, August 07, 2014 10:37 AM
To: Yimin Shen
Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Yimin hi,

Please see some commen= ts inline below.

 

Regards,

   &nbs= p;   Sasha

Email: Alexander.Vainshtein@ecitele.com

Mobile: 054-9266302

 

From: Yimin Sh= en [mailto:yshen@juniper.net]
Sent: Thursday, August 07, 2014 5:34 AM
To: Alexander Vainshtein
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Hi Sasha,

 

Thanks again for the c= omments. Please see the following 4 items in response to the 4 comments in = your email.

 

1. Today when an egres= s PE detects an egress AC failure=

 

[[Sasha]] Could = you please explain how the egress PE detects an egress AC failure? I have a= lways thought that egress AC failure has to be detected first by correspond= ing CE which would then notify the egress PE using some suitable “Remote Failure”/BDI/RDI mechanism. Spe= cific examples would really help.

 

it will simply have to= drop packets, until the remote PE stops sending traffic over the PW. This = draft is intended to improve this behavior. By no means the local repair on= egress PE will prevent the AC failure from being converted to and handled as a bi-directional AC failure. Note t= hat bi-directional AC failure handling must involve the CE, but the CE may = detect the failure at a slower pace, or it may not be able to detect it in = a timely manner, depending on the actual situation. <= /p>

[[Sasha]] Please= see above. IMHO and FWIW if the CE cannot detect its “Receive Failur= e”, the PE would not never know about the “Transmit Failure.

 

E.g. there may be othe= r L2 devices between the PE and CE; the AC may itself be a logical wire acr= oss another network; the type and location of the failure may make it easie= r for the PE to detect it; and so on. In any case, before the AC failure is handled more gracefully (or bi-direc= tionally), the egress traffic has been locally repaired by the egress PE.

 

2. As mentioned before= , this draft doesn’t have a dependency on a specific PSN tunnel techn= ology.

[[Sasha]] I have= provided my comments on this in my response to Eric. From my point of view= some LDP-specific issues still require clarification, but my bottom line i= s that operating the draft = over an MPLS network that uses LDP in DU mode for setting up tunnel LSPs ma= y sometimes be possible but would always be tricky. This does not ma= tch my understanding of “no dependency on a specific PSN tunnel technology”, but this may be a matter of ta= ste.

 

3. The draft already t= alks about scalability when describing the centralized protector model vs t= he co-located protector model. We can improve the text to explicitly mentio= n that the requirement for setting up LSPs on a per <primary PE, protector> basis rather than per primary = PE basis can potentially increase the number of LSPs in a network. This is = a tradeoff. As mentioned before, this does not necessarily cause a scaling = issue.

[[Sasha]] Yes, I= have found the corresponding text in the draft.  From my point of vie= w this issue is resolved.=

 

4. We will leave this = to PWE3 WG to decide whether this is really necessary.[[Sasha]] I tend to agree with Eric that usage of con= text-specific labels in the draft does not mean update to 4447. From my point of view this issue is resolved as w= ell.

 

Thanks,

 

/Yimin

 

 

From: Alexande= r Vainshtein [mailto:Al= exander.Vainshtein@ecitele.com]
Sent: Wednesday, August 06, 2014 5:11 AM
To: Yimin Shen
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Yimin and all,

I would like to summarize the issues that I have = with this draft. Digging for them in a very long email thread with multiple= levels of in-lining is not very effective IMO.

 

1.       Protection against unidirectional AC failures looks problematic to me:

a.       The draft explicitly differentiates between “= ingress” and “egress” AC failures

b.      The outcome can be using a different AC for each di= rection of CE-to-CE traffic with potentially destructive effect that cannot= be mitigated at the PWE3 level

c.       IMHO and FWIW a viable alternative could be to empl= oy native AC mechanisms converting unidirectional AC failures into bi-direc= tional ones. This would eliminate the differentiation between ingress and e= gress AC failures and facilitate using the PW redundancy mechanism for bi-directional protection

2.       Dependency upon specific mechanisms for setting = up PSN tunnels seems to break some common PWE3 architectural principles= :

a.       I am pretty well sure that the mechanisms specified= in the draft cannot be used with the PSN tunnel LSPs set up by LDP in DU mode

b.      If the authors, refer to LDP in the DoD mode= augmented with some CR-LDP-like mechanisms (e.g., for exclusion of specifi= c links when setting up the bypass LSP between the PLR and the Protector), = this should, at the very least, be discussed with and approved by the MPLS WG

3.       Scalability of the proposed solution should = be, at least, explicitly presented. In particular, the draft assumes that a= dedicated LSP would be used between PE1  PE2 for each combination of = PE2 with a specific Protector LSR.

4.       The Protector LSR could be either a kind of S-PE (i= f different from the secondary T-PE) or the secondary T-PE itself. In both = cases it should be capable of handling labels from specific context spaces = as if they were PW labels. This requires (at the very least) an update of RFC 4447 and should be explicitl= y presented and discussed.

Hopefully this summary will be useful.=

 

 

Regards,

       Sasha <= /o:p>

Email: Alexander.Vainshtein@ecitele.com

Mobile: 054-9266302

--_000_2afb5c963e054aef929f38dcb3defa07BY2PR05MB728namprd05pro_-- From nobody Fri Aug 8 07:40:50 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C0F021B2ACA; Fri, 8 Aug 2014 07:40:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.901 X-Spam-Level: X-Spam-Status: No, score=-1.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MFdm5kTJm4ho; Fri, 8 Aug 2014 07:40:46 -0700 (PDT) Received: from pipi.pi.nu (pipi.pi.nu [83.168.239.141]) (using TLSv1.1 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id F30481B2C08; Fri, 8 Aug 2014 07:40:45 -0700 (PDT) Received: from [192.168.0.100] (81-236-221-144-no93.tbcn.telia.com [81.236.221.144]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: loa@pi.nu) by pipi.pi.nu (Postfix) with ESMTPSA id 890211801586; Fri, 8 Aug 2014 16:40:44 +0200 (CEST) Message-ID: <53E4E16E.7050206@pi.nu> Date: Fri, 08 Aug 2014 16:40:46 +0200 From: Loa Andersson User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Yakov Rekhter References: Your message of Tue, 05 Aug 2014 13:59:49 -0000. <9696d0db139d46ffaad7be11340215e8@AM3PR03MB612.eurprd03.prod.outlook.com> <16167.1407340459@erosen-lnx>, <4552F0907735844E9204A62BBDD325E76AAAA7F4@nkgeml512-mbx.china.huawei.com> <22D4AECA-2D36-4F79-98CB-96E4B9BDC126@cisco.com> <4552F0907735844E9204A62BBDD325E76AAAB6F9@nkgeml512-mbx.china.huawei.com> <53E498D4.6000107@pi.nu> <201408081402.s78E2kn63116@magenta.juniper.net> In-Reply-To: <201408081402.s78E2kn63116@magenta.juniper.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/FbQ-QyEtDWRBHYndzIkBEAG8Mpc Cc: "mpls@ietf.org" , "pwe3-chairs@tools.ietf.org" , pwe3 , "Eric Rosen \(erosen\)" , "Stewart Bryant \(stbryant\)" Subject: Re: [PWE3] [mpls] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Aug 2014 14:40:48 -0000 Yakov, On 2014-08-08 16:02, Yakov Rekhter wrote: > Loa, > >> Authors, Mingui, Stewart >> >> On 2014-08-08 04:46, Mingui Zhang wrote: >>> Hi Stewart, >>> >>> I think authors would say the S-PE stitching method involves >>> the control plane processing during the repair procedure. They >>> emphasized their method uses data plane & local repair, which >>> can be faster. >>> Here, I want to raise one issue: >> >> It seems that we take it for granted that the method proposed on this >> draft is faster than e.g. the e2e protection a la mpls-tp. Why is that? > > To answer your question let me quote from "Network Recovery" by JP > Vasseur, Mario Pickavet, Piet Demeester (Section 5.8.1, page 336): > > With global protection, rerouting is performed by the head-end > LSR, which means that this requires for the head-end LSR to receive > the failure indication to reroute the affected traffic onto their > respective backup paths (whose paths have been precomputed and > signaled). So in terms of recovery time, the delta between global > and local protection is the failure indication signal propagation > time to the head-end LSR. > > Yakov. > Well, I won't argue that this is what is said in the book, but it is not what I asked about. mpls-tp runs e.g. 1:1 or 1:n, i.e. 1 working lsp (or pw) is protected by one or more pre-established lsp's or pw's. detection time can be as low as 10ms and the switch over is far shorter. While I understand that the draft is much more resource conservative, what is the foundation to claim that is faster? /Loa -- Loa Andersson email: loa@mail01.huawei.com Senior MPLS Expert loa@pi.nu Huawei Technologies (consultant) phone: +46 739 81 21 64 From nobody Fri Aug 8 08:12:00 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F3C751B2B3C; Fri, 8 Aug 2014 08:11:56 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id r6WMGS7w6TgY; Fri, 8 Aug 2014 08:11:55 -0700 (PDT) Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 49CD21B2B26; Fri, 8 Aug 2014 08:11:14 -0700 (PDT) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: The IESG To: IETF-Announce X-Test-IDTracker: no X-IETF-IDTracker: 5.6.2.p5 Auto-Submitted: auto-generated Precedence: bulk Sender: Message-ID: <20140808151114.11366.18692.idtracker@ietfa.amsl.com> Date: Fri, 08 Aug 2014 08:11:14 -0700 Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/MiLLM-B7nwf0brR7vvZBfQojVg4 Cc: pwe3@ietf.org Subject: [PWE3] Last Call: (Explicit Path Routing for Dynamic Multi-Segment Pseudowires) to Proposed Standard X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Reply-To: ietf@ietf.org List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Aug 2014 15:11:57 -0000 The IESG has received a request from the Pseudowire Emulation Edge to Edge WG (pwe3) to consider the following document: - 'Explicit Path Routing for Dynamic Multi-Segment Pseudowires' as Proposed Standard The IESG plans to make a decision in the next few weeks, and solicits final comments on this action. Please send substantive comments to the ietf@ietf.org mailing lists by 2014-08-22. Exceptionally, comments may be sent to iesg@ietf.org instead. In either case, please retain the beginning of the Subject line to allow automated sorting. Abstract Dynamic Multi-Segment Pseudowire (MS-PW) setup through an explicit path may be required to provide a simple solution for 1:1 protection with diverse primary and backup MS-PWs for a service, or to enable controlled signaling (strict or loose) for special MS-PWs. This document specifies the extensions and procedures required to enable dynamic MS-PWs to be established along explicit paths. The file can be obtained via http://datatracker.ietf.org/doc/draft-ietf-pwe3-mspw-er/ IESG discussion can be tracked via http://datatracker.ietf.org/doc/draft-ietf-pwe3-mspw-er/ballot/ No IPR declarations have been submitted directly on this I-D. From nobody Fri Aug 8 08:37:30 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2C1011B2BA6; Fri, 8 Aug 2014 08:37:29 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.602 X-Spam-Level: X-Spam-Status: No, score=-2.602 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RqrPOp7kYTeF; Fri, 8 Aug 2014 08:37:27 -0700 (PDT) Received: from na01-by2-obe.outbound.protection.outlook.com (mail-by2lp0237.outbound.protection.outlook.com [207.46.163.237]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4073A1B2AF2; Fri, 8 Aug 2014 08:37:27 -0700 (PDT) Received: from BY2PR05MB728.namprd05.prod.outlook.com (10.141.223.25) by BY2PR05MB725.namprd05.prod.outlook.com (10.141.223.13) with Microsoft SMTP Server (TLS) id 15.0.1005.10; Fri, 8 Aug 2014 15:37:25 +0000 Received: from BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) by BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) with mapi id 15.00.0995.014; Fri, 8 Aug 2014 15:37:25 +0000 From: Yimin Shen To: Mingui Zhang , "Stewart Bryant (stbryant)" Thread-Topic: [mpls] [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 Thread-Index: AQHPshD/uYiweJIOHUOm1CPA/rtgVpvGAdgAgADUDSA= Date: Fri, 8 Aug 2014 15:37:24 +0000 Message-ID: <35b809ce6fa34dd09844dddce13a6203@BY2PR05MB728.namprd05.prod.outlook.com> References: Your message of Tue, 05 Aug 2014 13:59:49 -0000. <9696d0db139d46ffaad7be11340215e8@AM3PR03MB612.eurprd03.prod.outlook.com> <16167.1407340459@erosen-lnx>, <4552F0907735844E9204A62BBDD325E76AAAA7F4@nkgeml512-mbx.china.huawei.com> <22D4AECA-2D36-4F79-98CB-96E4B9BDC126@cisco.com> <4552F0907735844E9204A62BBDD325E76AAAB6F9@nkgeml512-mbx.china.huawei.com> In-Reply-To: <4552F0907735844E9204A62BBDD325E76AAAB6F9@nkgeml512-mbx.china.huawei.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [66.129.241.10] x-microsoft-antispam: BCL:0;PCL:0;RULEID:;UriScan:; x-forefront-prvs: 02973C87BC x-forefront-antispam-report: SFV:NSPM; SFS:(6009001)(164054003)(13464003)(377454003)(51704005)(189002)(24454002)(199002)(20776003)(15975445006)(95666004)(107046002)(76482001)(105586002)(33646002)(93886004)(2656002)(83072002)(87936001)(81342001)(85852003)(80022001)(92566001)(64706001)(86362001)(106356001)(74316001)(106116001)(79102001)(85306004)(77982001)(101416001)(99286002)(46102001)(81542001)(21056001)(74662001)(31966008)(74502001)(50986999)(99396002)(76176999)(54356999)(76576001)(19580405001)(83322001)(66066001)(19580395003)(4396001)(24736002)(108616003); DIR:OUT; SFP:; SCL:1; SRVR:BY2PR05MB725; H:BY2PR05MB728.namprd05.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; LANG:en; Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: juniper.net Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/bB5-j0xhnCw7yfrNhMAl4kJhD98 Cc: "mpls@ietf.org" , pwe3 , "pwe3-chairs@tools.ietf.org" Subject: Re: [PWE3] [mpls] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Aug 2014 15:37:29 -0000 Hi Mingui, Thanks for the comments. Please see inline... Thanks, /Yimin -----Original Message----- From: mpls [mailto:mpls-bounces@ietf.org] On Behalf Of Mingui Zhang Sent: Thursday, August 07, 2014 10:47 PM To: Stewart Bryant (stbryant) Cc: mpls@ietf.org; pwe3; pwe3-chairs@tools.ietf.org Subject: Re: [mpls] [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01 - RFC4447 Hi Stewart, I think authors would say the S-PE stitching method involves the control pl= ane processing during the repair procedure. They emphasized their method us= es data plane & local repair, which can be faster. Here, I want to raise one issue:=20 If the primary PE fails and the PLR redirects the traffic during the flying= of the packets, hoping the backup PE delivers the packets immediately. Thi= s means the AC at the backup PE side is also ACTIVE. So the CE has active-a= ctive connections to both egress PEs. This is obviously different from the = PW-RED's active-standby mechanism [RFC6718]. Will this difference bring us = the frame duplication issue? I think we need to address this in the updated= version.=20 [yshen] Don't confuse the backup PW in this document with the standby PW in= RFC 6718. Each active or standby PW in RFC 6718 could be protected by a ba= ckup PW specified in this draft, if that's intended. There shouldn't be any= packet duplication during local repair, because PLR only sends packets to = protector. Thanks, Mingui >-----Original Message----- >From: Stewart Bryant (stbryant) [mailto:stbryant@cisco.com] >Sent: Thursday, August 07, 2014 3:27 PM >To: Mingui Zhang >Cc: Eric Rosen (erosen); Alexander Vainshtein; mpls@ietf.org; pwe3;=20 >pwe3-chairs@tools.ietf.org >Subject: Re: [PWE3] [mpls] WG Last Call for >draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 > > > >Sent from my iPad > >> On 7 Aug 2014, at 04:57, "Mingui Zhang" wrote: >> >> Hi Eric, >> >> The explanation of the story is very clear. Let me complement a bit. >> >>> - A "primary egress PE" partitions its set of PWs into n sets, where=20 >>> each set is associated with a given "protector". The "protector" is=20 >>> the backup egress PE for those PWs. >> >> When the backup PE acts as the protector, it is the 'co-located' model. > >This case does not however need context labels. It is a form of S-PE=20 >and can stitch the PWs just like every existing S-PE already does. > >Stewart > _______________________________________________ mpls mailing list mpls@ietf.org https://www.ietf.org/mailman/listinfo/mpls From nobody Fri Aug 8 12:28:19 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CCE001A038C; Fri, 8 Aug 2014 12:28:15 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.902 X-Spam-Level: X-Spam-Status: No, score=-1.902 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6UBwx2DhBaGB; Fri, 8 Aug 2014 12:28:14 -0700 (PDT) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1lp0139.outbound.protection.outlook.com [207.46.163.139]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 628421A0380; Fri, 8 Aug 2014 12:28:13 -0700 (PDT) Received: from BL2PR05CA0013.namprd05.prod.outlook.com (10.255.226.13) by BY2PR05MB727.namprd05.prod.outlook.com (10.141.223.23) with Microsoft SMTP Server (TLS) id 15.0.1005.10; Fri, 8 Aug 2014 19:28:10 +0000 Received: from BY2FFO11FD023.protection.gbl (2a01:111:f400:7c0c::189) by BL2PR05CA0013.outlook.office365.com (2a01:111:e400:c04::13) with Microsoft SMTP Server (TLS) id 15.0.1005.10 via Frontend Transport; Fri, 8 Aug 2014 19:28:09 +0000 Received: from P-EMF01-SAC.jnpr.net (66.129.239.15) by BY2FFO11FD023.mail.protection.outlook.com (10.1.15.212) with Microsoft SMTP Server (TLS) id 15.0.990.10 via Frontend Transport; Fri, 8 Aug 2014 19:28:09 +0000 Received: from magenta.juniper.net (172.17.27.123) by P-EMF01-SAC.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.146.0; Fri, 8 Aug 2014 12:27:34 -0700 Received: from juniper.net (sapphire.juniper.net [172.17.28.108]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id s78JRXn08948; Fri, 8 Aug 2014 12:27:33 -0700 (PDT) (envelope-from yakov@juniper.net) Message-ID: <201408081927.s78JRXn08948@magenta.juniper.net> To: Loa Andersson In-Reply-To: <53E4E16E.7050206@pi.nu> References: Your message of Tue, 05 Aug 2014 13:59:49 -0000. <9696d0db139d46ffaad7be11340215e8@AM3PR03MB612.eurprd03.prod.outlook.com> <16167.1407340459@erosen-lnx>, <4552F0907735844E9204A62BBDD325E76AAAA7F4@nkgeml512-mbx.china.huawei.com> <22D4AECA-2D36-4F79-98CB-96E4B9BDC126@cisco.com> <4552F0907735844E9204A62BBDD325E76AAAB6F9@nkgeml512-mbx.china.huawei.com> <53E498D4.6000107@pi.nu> <201408081402.s78E2kn63116@magenta.juniper.net> <53E4E16E.7050206@pi.nu> X-MH-In-Reply-To: Loa Andersson message dated "Fri, 08 Aug 2014 16:40:46 +0200." MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <4085.1407526053.1@juniper.net> Date: Fri, 8 Aug 2014 12:27:33 -0700 From: Yakov Rekhter X-EOPAttributedMessage: 0 X-Forefront-Antispam-Report: CIP:66.129.239.15; CTRY:US; IPV:NLI; IPV:NLI; EFV:NLI; SFV:NSPM; SFS:(6009001)(51704005)(377424004)(24454002)(199002)(189002)(4396001)(102836001)(81156004)(106466001)(105596002)(95666004)(6806004)(107046002)(69596002)(110136001)(80022001)(81342001)(44976005)(76482001)(83322001)(77982001)(92566001)(68736004)(83072002)(84676001)(85852003)(21056001)(23726002)(31966008)(16796002)(87936001)(46406003)(97736001)(97756001)(79102001)(81542001)(50986999)(47776003)(54356999)(76176999)(99396002)(92726001)(46102001)(74502001)(74662001)(86362001)(85306004)(93886004)(64706001)(20776003)(50466002); DIR:OUT; SFP:; SCL:1; SRVR:BY2PR05MB727; H:P-EMF01-SAC.jnpr.net; FPR:; MLV:sfv; PTR:InfoDomainNonexistent; MX:1; LANG:en; X-Microsoft-Antispam: BCL:0;PCL:0;RULEID:;UriScan:; X-Forefront-PRVS: 02973C87BC Received-SPF: SoftFail (protection.outlook.com: domain of transitioning juniper.net discourages use of 66.129.239.15 as permitted sender) Authentication-Results: spf=softfail (sender IP is 66.129.239.15) smtp.mailfrom=yakov@juniper.net; X-OriginatorOrg: juniper.net Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/SQaQHKj7zjyE1QMKHimCNsMt9G0 Cc: "mpls@ietf.org" , "Eric Rosen \(erosen\)" , pwe3 , "pwe3-chairs@tools.ietf.org" , "Stewart Bryant \(stbryant\)" Subject: Re: [PWE3] [mpls] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Aug 2014 19:28:16 -0000 Loa, > Yakov, > > On 2014-08-08 16:02, Yakov Rekhter wrote: > > Loa, > > > >> Authors, Mingui, Stewart > >> > >> On 2014-08-08 04:46, Mingui Zhang wrote: > >>> Hi Stewart, > >>> > >>> I think authors would say the S-PE stitching method involves > >>> the control plane processing during the repair procedure. They > >>> emphasized their method uses data plane & local repair, which > >>> can be faster. > >>> Here, I want to raise one issue: > >> > >> It seems that we take it for granted that the method proposed on this > >> draft is faster than e.g. the e2e protection a la mpls-tp. Why is that? > > > > To answer your question let me quote from "Network Recovery" by JP > > Vasseur, Mario Pickavet, Piet Demeester (Section 5.8.1, page 336): > > > > With global protection, rerouting is performed by the head-end > > LSR, which means that this requires for the head-end LSR to receive > > the failure indication to reroute the affected traffic onto their > > respective backup paths (whose paths have been precomputed and > > signaled). So in terms of recovery time, the delta between global > > and local protection is the failure indication signal propagation > > time to the head-end LSR. > > > > Yakov. > > > > Well, I won't argue that this is what is said in the book, but it is > not what I asked about. > > mpls-tp runs e.g. 1:1 or 1:n, i.e. 1 working lsp (or pw) is protected > by one or more pre-established lsp's or pw's. detection time can be > as low as 10ms and the switch over is far shorter. > > While I understand that the draft is much more resource conservative, > what is the foundation to claim that is faster? Local protection does not involve propagating failure indication beyond the point of failure. Do you think that the detection time at the head-end can be less than the time it takes to propagate failure indication from the point of failure to the head-end ? Furhermore, even if we assume that for some topologies the propagation time could be 10ms, could it be 10ms irrespective of the distance and the number of hops spanned by a PW ? Yakov. From nobody Sat Aug 9 04:22:18 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7B7421B27B7; Sat, 9 Aug 2014 04:22:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.901 X-Spam-Level: X-Spam-Status: No, score=-1.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MqYPAMrS9X8c; Sat, 9 Aug 2014 04:22:14 -0700 (PDT) Received: from pipi.pi.nu (pipi.pi.nu [83.168.239.141]) (using TLSv1.1 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3A1ED1B27B2; Sat, 9 Aug 2014 04:22:14 -0700 (PDT) Received: from [192.168.0.123] (81-229-83-119-no65.business.telia.com [81.229.83.119]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: loa@pi.nu) by pipi.pi.nu (Postfix) with ESMTPSA id C456A1801586; Sat, 9 Aug 2014 13:22:12 +0200 (CEST) Message-ID: <53E60466.9000008@pi.nu> Date: Sat, 09 Aug 2014 13:22:14 +0200 From: Loa Andersson User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Yakov Rekhter References: Your message of Tue, 05 Aug 2014 13:59:49 -0000. <9696d0db139d46ffaad7be11340215e8@AM3PR03MB612.eurprd03.prod.outlook.com> <16167.1407340459@erosen-lnx>, <4552F0907735844E9204A62BBDD325E76AAAA7F4@nkgeml512-mbx.china.huawei.com> <22D4AECA-2D36-4F79-98CB-96E4B9BDC126@cisco.com> <4552F0907735844E9204A62BBDD325E76AAAB6F9@nkgeml512-mbx.china.huawei.com> <53E498D4.6000107@pi.nu> <201408081402.s78E2kn63116@magenta.juniper.net> <53E4E16E.7050206@pi.nu> <201408081927.s78JRXn08948@magenta.juniper.net> In-Reply-To: <201408081927.s78JRXn08948@magenta.juniper.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/pgL79G2Fn2DKVKfDRsmTFb2SRBM Cc: "mpls@ietf.org" , "Eric Rosen \(erosen\)" , pwe3 , "pwe3-chairs@tools.ietf.org" , "Stewart Bryant \(stbryant\)" Subject: Re: [PWE3] [mpls] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Aug 2014 11:22:16 -0000 Yakov, On 2014-08-08 21:27, Yakov Rekhter wrote: > Loa, > >> Yakov, >> >> On 2014-08-08 16:02, Yakov Rekhter wrote: >>> Loa, >>> >>>> Authors, Mingui, Stewart >>>> >>>> On 2014-08-08 04:46, Mingui Zhang wrote: >>>>> Hi Stewart, >>>>> >>>>> I think authors would say the S-PE stitching method involves >>>>> the control plane processing during the repair procedure. They >>>>> emphasized their method uses data plane & local repair, which >>>>> can be faster. >>>>> Here, I want to raise one issue: >>>> >>>> It seems that we take it for granted that the method proposed on this >>>> draft is faster than e.g. the e2e protection a la mpls-tp. Why is that? >>> >>> To answer your question let me quote from "Network Recovery" by JP >>> Vasseur, Mario Pickavet, Piet Demeester (Section 5.8.1, page 336): >>> >>> With global protection, rerouting is performed by the head-end >>> LSR, which means that this requires for the head-end LSR to receive >>> the failure indication to reroute the affected traffic onto their >>> respective backup paths (whose paths have been precomputed and >>> signaled). So in terms of recovery time, the delta between global >>> and local protection is the failure indication signal propagation >>> time to the head-end LSR. >>> >>> Yakov. >>> >> >> Well, I won't argue that this is what is said in the book, but it is >> not what I asked about. >> >> mpls-tp runs e.g. 1:1 or 1:n, i.e. 1 working lsp (or pw) is protected >> by one or more pre-established lsp's or pw's. detection time can be >> as low as 10ms and the switch over is far shorter. >> >> While I understand that the draft is much more resource conservative, >> what is the foundation to claim that is faster? > > Local protection does not involve propagating failure indication > beyond the point of failure. > > Do you think that the detection time at the head-end can be less > than the time it takes to propagate failure indication from the > point of failure to the head-end ? > > Furhermore, even if we assume that for some topologies the propagation > time could be 10ms, could it be 10ms irrespective of the distance > and the number of hops spanned by a PW ? > > Yakov. > MPLS-TP style protection does not involve propagating (i.e. does not involve sending a a failure indication from the point of detection to the head end). The failure at the head end is e.g. detected if you loose three consecutive keep alive message traveling in the data plane. If you send one keep-alive message every 3.3 msec, you can reach a time to repair this is - detection time (approximately 10 ms) - transfer time (typically usec's) - switchover time (good guess is approximately 10 ms) MPLS-TP protection was designed to met the 50 msec requirement, it is my guess that is about the same as what this draft achieves. Note that I'm not say that the protection mechanism described is "bad" or "slow", to the contrary I think it is both useful and fast, only that we are talking of time to repair for the to mechanisms. /Loa -- Loa Andersson email: loa@mail01.huawei.com Senior MPLS Expert loa@pi.nu Huawei Technologies (consultant) phone: +46 739 81 21 64 From nobody Sat Aug 9 04:40:49 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D86DB1B27C4; Sat, 9 Aug 2014 04:40:47 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id O1y-oKcmJiig; Sat, 9 Aug 2014 04:40:46 -0700 (PDT) Received: from mail-we0-x230.google.com (mail-we0-x230.google.com [IPv6:2a00:1450:400c:c03::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D145D1B27C2; Sat, 9 Aug 2014 04:40:45 -0700 (PDT) Received: by mail-we0-f176.google.com with SMTP id q58so6612912wes.35 for ; Sat, 09 Aug 2014 04:40:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:disposition-notification-to:date:from:reply-to :user-agent:mime-version:to:cc:subject:references:in-reply-to :content-type:content-transfer-encoding; bh=t+A3WbV2KaW3YExBhGJ/005Ye8CexWp7rz5HGveB6s0=; b=Ly2BTjTL9/AQGLB7/Wm0ZvIx2w7vNnbrI5aCu+7l5mbhfsuZ4iRLVjg96GnbztfSWn GM3MkyMBnIqXsxwQjxJhy8IiiuX4HRspqVwt2bEhG5YlOHIy/Is13xRlTQBEOuwxW4re Zr7X8RNi52L/D09yjuBSRqgv6FK+0ZAtzus7O/UgLpWay/6PHG3rbtdBiJtQ1JttEhCz G+cw6U11vbrwoo30CQ0u59sGK6v2hry7QsEBff8hORKspo28GNNOb97knBgqJZc9EAP3 xhktBuF1HBnaHAo1cV+ikt+49RVTi49eu5MmMmQDPlWrivOPSSVK9acgO1F1B0mc4jeq 5c7A== X-Received: by 10.194.237.194 with SMTP id ve2mr38930416wjc.89.1407584444349; Sat, 09 Aug 2014 04:40:44 -0700 (PDT) Received: from McAsterix.local (g215085.upc-g.chello.nl. [80.57.215.85]) by mx.google.com with ESMTPSA id w14sm1016678wij.2.2014.08.09.04.40.41 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 09 Aug 2014 04:40:43 -0700 (PDT) Message-ID: <53E608B8.3060203@gmail.com> Date: Sat, 09 Aug 2014 13:40:40 +0200 From: Huub van Helvoort User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Yimin Shen References: Your message of Tue, 05 Aug 2014 13:59:49 -0000. <9696d0db139d46ffaad7be11340215e8@AM3PR03MB612.eurprd03.prod.outlook.com> <16167.1407340459@erosen-lnx>, <4552F0907735844E9204A62BBDD325E76AAAA7F4@nkgeml512-mbx.china.huawei.com> <22D4AECA-2D36-4F79-98CB-96E4B9BDC126@cisco.com> <4552F0907735844E9204A62BBDD325E76AAAB6F9@nkgeml512-mbx.china.huawei.com> <53E498D4.6000107@pi.nu> <496a2c137775477c8c78c7e8d5463c6e@BY2PR05MB728.namprd05.prod.outlook.com> In-Reply-To: <496a2c137775477c8c78c7e8d5463c6e@BY2PR05MB728.namprd05.prod.outlook.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/So_qTlnQVApV7bxtZVIqm9OviHM Cc: "mpls@ietf.org" , pwe3 , "pwe3-chairs@tools.ietf.org" Subject: Re: [PWE3] [mpls] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: huubatwork@gmail.com List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Aug 2014 11:40:48 -0000 Hello Yimin, You wrote: > It will not be a problem if this mechanism and an e2e mechanism > are both implemented in a network. As described in the draft, > this mechanism provides local repair, and it is viewed as a > temporary repair. The draft strongly recommend this mechanism > to be used in tandem with an e2e or global repair mechanism, > so that the later can eventually move traffic from the locally > repaired PW over to a fully functional PW, as a permanent repair. How do you warrant that all the repair mechanisms present do not decide to switch at the same time? This would result in unpredictable repairs. So some coordination between the repair mechanisms is required. regards, Huub. > -----Original Message----- > From: pwe3 [mailto:pwe3-bounces@ietf.org] On Behalf Of Loa Andersson > Sent: Friday, August 08, 2014 5:31 AM > To: Mingui Zhang; Stewart Bryant (stbryant) > Cc: mpls@ietf.org; Eric Rosen (erosen); pwe3; pwe3-chairs@tools.ietf.org > Subject: Re: [PWE3] [mpls] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 - RFC4447 > > Authors, Mingui, Stewart > > On 2014-08-08 04:46, Mingui Zhang wrote: >> Hi Stewart, >> >> I think authors would say the S-PE stitching method involves the control plane processing during the repair procedure. They emphasized their method uses data plane & local repair, which can be faster. >> Here, I want to raise one issue: > > It seems that we take it for granted that the method proposed on this draft is faster than e.g. the e2e protection a la mpls-tp. Why is that? > If we have implementations of both, do we have any real measurements? > > /Loa > -- ***************************************************************** 请记ä½ï¼Œä½ æ˜¯ç‹¬ä¸€æ— äºŒçš„，就åƒå…¶ä»–æ¯ä¸€ä¸ªäººä¸€æ · From nobody Sat Aug 9 19:34:18 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 269E31A03FE for ; Sat, 9 Aug 2014 19:34:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.669 X-Spam-Level: X-Spam-Status: No, score=-2.669 tagged_above=-999 required=5 tests=[BAYES_50=0.8, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, GB_I_INVITATION=-2, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-0.668, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id P93fsuQWEP02 for ; Sat, 9 Aug 2014 19:34:13 -0700 (PDT) Received: from mailuogwhop.emc.com (mailuogwhop.emc.com [168.159.213.141]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 57F3D1A03F8 for ; Sat, 9 Aug 2014 19:34:12 -0700 (PDT) Received: from maildlpprd06.lss.emc.com (maildlpprd06.lss.emc.com [10.253.24.38]) by mailuogwprd03.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s7A2YAoN018704 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 9 Aug 2014 22:34:11 -0400 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd03.lss.emc.com s7A2YAoN018704 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=emc.com; s=jan2013; t=1407638051; bh=X8Vy/7OtsPqDTMrKPNH2WCblbrM=; h=From:To:Date:Subject:Message-ID:References:In-Reply-To: Content-Type:Content-Transfer-Encoding:MIME-Version; b=Co3vrC6i3ibNg+1uKlR5T4h22qtMebW+smhc4yZlJhnd6V9gqWX+0Jv1ShFeTLb81 YRHU8klY4W6ZzdFkbp8+V5J7/LSDBYyJgSamFuHdjjf8mseoxQhZ6K6YnjQbMkuvbr tXW/z5uYEFS+zebXhDN5C2YaOGrATeYSc7RWEjKg= X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd03.lss.emc.com s7A2YAoN018704 Received: from mailusrhubprd54.lss.emc.com (mailusrhubprd54.lss.emc.com [10.106.48.19]) by maildlpprd06.lss.emc.com (RSA Interceptor); Sat, 9 Aug 2014 19:33:57 -0700 Received: from mxhub14.corp.emc.com (mxhub14.corp.emc.com [128.222.70.235]) by mailusrhubprd54.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s7A2Xu5E001855 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Sat, 9 Aug 2014 22:33:57 -0400 Received: from mx15a.corp.emc.com ([169.254.1.175]) by mxhub14.corp.emc.com ([128.222.70.235]) with mapi; Sat, 9 Aug 2014 22:33:55 -0400 From: "Black, David" To: Yaakov Stein , "pwe3@ietf.org" Date: Sat, 9 Aug 2014 22:32:24 -0400 Thread-Topic: WGLC comments on: draft-ietf-pwe3-congcons-02.txt Thread-Index: Ac+zEP27iWho+8uAQLKBEFcFeq8EIgBLGLOA Message-ID: <8D3D17ACE214DC429325B2B98F3AE712077B951928@MX15A.corp.emc.com> References: <8D3D17ACE214DC429325B2B98F3AE712077B9517F4@MX15A.corp.emc.com> In-Reply-To: <8D3D17ACE214DC429325B2B98F3AE712077B9517F4@MX15A.corp.emc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd54.lss.emc.com X-RSA-Classifications: public Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/SeaP1ReCtxiSFDaJvfCtVV3a5Ig Subject: Re: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Aug 2014 02:34:16 -0000 Following up on this ... > As we're now moving quickly to try to complete this draft , I agreed > w/Yaakov that my author comments would be sent in as WG Last Call comment= s ... > ... so here they are: ... with some suggested new text ... =20 > Major issues: >=20 > The reference to a transport circuit breaker in section 3 is a bit optimi= stic > and high level: >=20 > We can accomplish this by > employing a transport circuit breaker, by which we mean an automatic > mechanism for terminating a flow to prevent negative impact on other > flows and on the stability of the network > [I-D.fairhurst-tsvwg-circuit-breaker]. >=20 > This should be accompanied by at least an example of how to do this - pas= t > discussion has suggested carrier monitoring of delivered service quality > in order to shut down PWs that are experiencing loss rates that result in > unacceptable service quality (as defined for us by the ITU ...). I'd als= o > like to add some text to suggest that operators use this sort of circuit > breaker practice when TDM PWs compete with elastic traffic. After this text in Section 3: We can accomplish this by employing a transport circuit breaker, by which we mean an automatic mechanism for terminating a flow to prevent negative impact on other flows and on the stability of the network [I-D.fairhurst-tsvwg-circuit-breaker]. Note that a transport circuit breaker is intended as a protection mechanism of last resort, just as an electrical circuit breaker is only triggered when absolutely necessary. =20 I suggest adding: For TDM pseudowires, such a circuit breaker can be realized via operator monitoring of drop rates and consequent on the quality of TDM service delivered and shutting down TDM pseudowires that persistentl= y fail to deliver acceptable TDM service (as defined by [G775] and [G826]). This document explains why a TDM PW that is causing congestion problems via interference with elastic traffic cannot be delivering acceptable TDM service (see the rest of this Section, plus Appendices A and B). And breaking the paragraph after the added text. > Figure 1 and Figure 2 indicate that based on using the TCP throughput > equation as a proxy for potential impact on elastic traffic (which is wha= t > we're doing in this draft), for the same fixed bandwidth PW, a larger seg= ment > size is less likely to cause impacts. I'd like to add a suggestion to us= e > the larger PW segment sizes as long as they're consistent with delivery o= f > the service. There's been a related (long-to-the-point-of-ratholed) deba= te > over in the Transport Area about whether packet-rate-limited/congestible > forwarding is still relevant. My view is that such equipment exists, and > hence that this is a relevant consideration for this draft, and would not > mention that issue here. Ignoring the rathole invitation ;-). This is still in section 3, after: We see that in general for large packet sizes, short delays, and low packet loss rates, the TDM pseudowires consume much less bandwidth than TCP would under identical conditions. Only for small packets, long delays, and high packet loss ratios, do TDM PWs potentially consume more bandwidth, and even then only marginally. I suggest adding: Comparing figures 1 and 2, use of a larger segment size for a T1 or E1 TDM PW moves the operational behavior of the PW away from the boundary at which the TCP throughput equation suggests that undue interference wi= th elastic traffic may start to occur. We suggest that larger segment sizes be preferred for T1 and E1 TDM PWs, but note that other considerations may favor smaller segment sizes (e.g., jitter buffer size, bounding the impact of an isolated packet drop event). and again, breaking the paragraph after that added text. I hope that latter sentence ("We suggest that ...") is acceptable to the WG= . > Minor issues: >=20 > Section 2 is about Ethernet PWs that carry elastic traffic. Should other= s > be mentioned as also covered by this analysis - e.g., ATM and Frame Relay > PWs carrying elastic traffic? After: In this section we consider Ethernet PWs that primarily carry congestion-responsive traffic. We show that we automatically obtain the desired congestion avoidance behavior, and that additional mechanisms are not needed. I suggest adding: This discussion is applicable to any PW that carries congestion- responsive traffic (e.g., ATM and Frame Relay PWs); Ethernet PWs are used here for clarity, as they are a common example. =20 > Section 2 does not cover Ethernet PWs carrying inelastic traffic - that w= ould > be unusual, but should it be explicitly noted as out of scope, or just no= t > discussed (as is the case for the current text)? I suggest doing nothing here, and planning to deal with this concern if it should arise later (e.g., IETF Last Call or IESG Evaluation). =20 > Section 3 should say that the reason the draft focuses on TDM PWs is that > they're the inelastic PWs that are likely to be carried over IP in practi= ce. > My understanding is that other potentially inelastic PWs are MPLS-only > in practice. After: Inelastic PWs, such as TDM PWs ([RFC4553][RFC5086][RFC5087]), are potentially more problematic than the elastic PWs of the previous section. Being constant bit-rate (CBR), TDM PWs can not be made responsive to congestion. On the other hand, being CBR, they also do not attempt to capture additional bandwidth when neighboring TCP flows back off. I suggest adding: We focus on TDM PWs here as they are the only inelastic PWs that are likely to be IP-encapsulated in practice. Most other inelastic PWs tend to only be used with MPLS. [I'd ask Yaakov and other PW experts to double-check the latter sentence, please.] > A terminology section to roll up all the acronym expansions (e.g., TDM, C= BR) > in one place seems like a good idea. These four acronyms would be good to put into a terminology section: CBR Constant Bit Rate PSN Packet Switched Network PW Pseudowire TDM Time Division Multiplexing >=20 > This text on p.11 could be incendiary: >=20 > Under this condition it is safe to place the TDM PW along with > congestion - responsive traffic such as TCP, without causing > additional congestion. >=20 > First of all to reduce the flammability, "safe" -> "reasonable" and there > are a number of other occurrences of "safe" that deserve similar treatmen= t ;- > ). > Beyond that, this assertion is dependent on the operating conditions stay= ing > within that reasonable operating envelope, which entails use of something > like a circuit breaker - I'd say that here, and reference the earlier > circuit breaker discussion. Global substitution: "safe" -> "reasonable", starting with the abstract. Then for this specific text on p.11: OLD Under this condition it is safe to place the TDM PW along with congestion - responsive traffic such as TCP, without causing additional congestion. NEW Under this condition it is reasonable to place the TDM PW in competition with congestion-responsive traffic such as TCP, as the result should = not produce unacceptable congestion effects. That assumes that the opera= ting conditions remain within the reasonable operating envelope - as discu= ssed above, a transport circuit breaker is a suggested means of managing a= TDM PW so that it is shut down if operating conditions remain persistentl= y outside the reasonable operating envelope for the PW (e.g., due to a high packet loss rate). > Figures 7 and 8 could use some shading or lines to indicate the typical > deployed operating regions for the pseudo-wires to visibly show that they > fall under the curves, particularly for larger segment sizes. I'd also > repeat or refer to the suggestion to use larger >=20 > Nits: >=20 > References aren't allowed in abstracts, so remove reference to > [I-D.fairhurst-tsvwg-circuit-breaker] >=20 > Introduction, first paragraph, delete "PWE3" towards end of paragraph - >=20 > preferably the 4 byte PWE3 control word. >=20 > In addition to its current general statements, the Security Consideration= s > section should point to the security considerations sections in a few > particularly relevant RFCs - I'd suggest 3985, 4553, 5086 and 5087. After: This document does not introduce any new congestion-specific mechanisms and thus does not introduce any new security considerations above those present for PWs in general. I suggest adding a new paragraph: For PW security considerations, see the security considerations sections of the cited RFCS; the security considerations in [RFC3985], [RFC4553], [RFC5086] and [RFC5087] are particularly applicable to this document. >=20 > Thanks, > --David >=20 >=20 > > -----Original Message----- > > From: pwe3 [mailto:pwe3-bounces@ietf.org] On Behalf Of Yaakov Stein > > Sent: Thursday, July 24, 2014 4:52 PM > > To: pwe3@ietf.org > > Subject: Re: [PWE3] I-D Action: draft-ietf-pwe3-congcons-02.txt > > > > Please look at the PDF version only > > (the txt version does not have the graphs). > > > > Y(J)S > > > > >=20 > _______________________________________________ > pwe3 mailing list > pwe3@ietf.org > https://www.ietf.org/mailman/listinfo/pwe3 From nobody Sat Aug 9 22:15:42 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 248321A0515 for ; Sat, 9 Aug 2014 22:15:39 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.099 X-Spam-Level: X-Spam-Status: No, score=0.099 tagged_above=-999 required=5 tests=[BAYES_50=0.8, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mPCoNpjkP6tA for ; Sat, 9 Aug 2014 22:15:34 -0700 (PDT) Received: from emea01-am1-obe.outbound.protection.outlook.com (mail-am1lrp0017.outbound.protection.outlook.com [213.199.154.17]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0A3FB1A050E for ; Sat, 9 Aug 2014 22:15:31 -0700 (PDT) Received: from AM3PR03MB612.eurprd03.prod.outlook.com (10.242.110.144) by AM3PR03MB611.eurprd03.prod.outlook.com (10.242.109.28) with Microsoft SMTP Server (TLS) id 15.0.995.14; Sun, 10 Aug 2014 05:15:29 +0000 Received: from AM3PR03MB612.eurprd03.prod.outlook.com ([10.242.110.144]) by AM3PR03MB612.eurprd03.prod.outlook.com ([10.242.110.144]) with mapi id 15.00.1005.008; Sun, 10 Aug 2014 05:15:28 +0000 From: Alexander Vainshtein To: Yimin Shen Thread-Topic: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 Thread-Index: AQHPqymWveYGLBru1kCsLuzIRj9tdJu3CWCAgAA8XgCAAPc/IIABBtiAgAg4EiaAAOtlgIAAs2DLgAA1qVCAASlTgIAAxW1QgAGWRICAAnyxrg== Date: Sun, 10 Aug 2014 05:15:28 +0000 Message-ID: <1407647722694.18724@ecitele.com> References: <201407251524.s6PFOPn92012@magenta.juniper.net> <53D79504.4050904@cisco.com> <04f15bedf2be4c7480d3e9ca01bdfd7f@BY2PR05MB728.namprd05.prod.outlook.com> <2597291f7b074922b690e4b06999cf1a@AM3PR03MB612.eurprd03.prod.outlook.com>, <30e13900814f41a681151971cbe9ebef@BY2PR05MB728.namprd05.prod.outlook.com> <1407215578702.64387@ecitele.com>, <2280b8e0d5f94a60badab91a2237181b@BY2PR05MB728.namprd05.prod.outlook.com> <1407304503657.72312@ecitele.com> <046050cb2d254a139b4650b2d7f93075@BY2PR05MB728.namprd05.prod.outlook.com> <09ab595ab24a46a0a4e87ebeb2cd185a@AM3PR03MB612.eurprd03.prod.outlook.com>, <2afb5c963e054aef929f38dcb3defa07@BY2PR05MB728.namprd05.prod.outlook.com> In-Reply-To: <2afb5c963e054aef929f38dcb3defa07@BY2PR05MB728.namprd05.prod.outlook.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [5.153.9.202] x-microsoft-antispam: BCL:0;PCL:0;RULEID: x-forefront-prvs: 029976C540 x-forefront-antispam-report: SFV:NSPM; SFS:(10019004)(164054003)(43784003)(51444003)(377454003)(37854004)(199002)(189002)(252514010)(16236675004)(95666004)(83322001)(81542001)(81342001)(50986999)(54356999)(1941001)(46102001)(74502001)(74662001)(76176999)(110136001)(101416001)(19580405001)(19627405001)(19580395003)(79102001)(76482001)(99396002)(106356001)(19625215002)(105586002)(4396001)(21056001)(107046002)(2656002)(86362001)(92566001)(31966008)(87936001)(20776003)(106116001)(64706001)(93886004)(85306004)(92726001)(561944003)(83072002)(36756003)(77982001)(71446004)(66066001)(80022001)(85852003)(19607625011)(19627315001); DIR:OUT; SFP:1102; SCL:1; SRVR:AM3PR03MB611; H:AM3PR03MB612.eurprd03.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; LANG:en; Content-Type: multipart/alternative; boundary="_000_140764772269418724ecitelecom_" MIME-Version: 1.0 X-OriginatorOrg: ecitele.com Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/SZaeeEcRiFCrfBt9i1txiPta4yw Cc: Yakov Rekhter , "erosen@cisco.com" , "pwe3@ietf.org" , "stbryant@cisco.com" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Aug 2014 05:15:39 -0000 --_000_140764772269418724ecitelecom_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Yimin hi, Lots of thanks for a prompt response. I must admit that I have missed the proposal in Section 4.2.2 to treat "con= text ID" IP address as a proxy node that is adjacent to both the Primary PE= and the Protector NEs. (It also has not been mentioned in Eri=E7's clarifi= cation email). I think that this approach would require certain manipulatio= n of both the IGP and LDP, but this requires more thought on my side. One o= f the issues that come to mind, is the need for LDP in both these nodes to = allocate a locally terminated label for the corresponding FEC while not adv= ertising context IP address as one of the IP addresses it owns. I think that your statement about "avoiding ECMP scenario" is problematic,= because, from my point of view, ECMP is one of the important characterist= ics of LDP-based MPLS PSN. And in any case the draft does not say anything= about that (or, at least neither ECMP nor "equal cost multipath" is mentio= ned in the text). If you see ECMP avoidance as a pre-condition for using t= he mechanism defined in draft with LDP LSPs, this should be quite explicit = and detailed IMO. I still fail to understand what constitutes an egress AC failure in the dr= aft and how it is different from the ingress AC failure. Unfortunately the = examples you've provided in your response do not help much: e.g., a fiber c= ut close to would be most probably recognized by the PE as a bi-directiona= l AC failure, while a fiber cut that is close to CE but remote from PE woul= d most probably not detected by the PE. (Of course if PE and CE are termina= ting the same fiber span, the fiber cut would be recognized by both). My guess (FWIW) that the draft actually discusses protection of traffic in = the PE --> CE direction triggered by a failure detected by the PE while lea= ving protection of traffic in the CE --> PE direction some other mechanism = that would be triggered by CE detecting the failure. This mechanism presuma= bly would be different from the PW redundancy mechanism because it would re= quire that an active-active scheme for Primary and Backup PWs. If this guess is correct (or close to reality), I would expect some explici= t text explaining it. In particular, this text should explain that "redunda= nt PWs" mentioned in the draft are not related to "PW redundancy". The curr= ent text makes a mix of these two notions, and this definitely does not hel= p the reader (or at least this reader :-). Hopefully these comments would be useful. Regards, Sasha ________________________________ From: Yimin Shen Sent: Friday, August 8, 2014 5:34 PM To: Alexander Vainshtein Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Hi Sasha, Thanks again for the comments and questions! Regarding AC failure detection on PE, in the typical cases, if there is a f= iber cut, or a hardware failure affecting the port on the PE, or a hardware= failure affecting the port on the CE (assume the CE is directed attached t= o the PE), the PE should be able to directly and independently detect the f= ailure at physical or link layer. Regarding LDP, I agree with you that LDP by its nature is not as straight-f= orward as RSVP-TE in this scenario. Since a context ID are advertised by bo= th primary PE and protector, we should avoid creating an ECMP scenario. Tha= t=92s why section 4.2.2 provides a suggestion to advertise the context ID a= s a proxy node (i.e. a virtual node attached to the two routers), and make = its reachability via the primary PE more favorable than that via the protec= tor. MRT and other mechanisms could be used to help LDP improve protection = coverage. Also, nothing would prevent LDP from using RSVP bypass tunnels. A= s you can see, there may be many different approaches, and there may new ap= proaches coming in the future. In this PWE3 document, we just want to stay = at PW level and be generic with PSN level. In any case, this mechanism is a= chievable with LDP, and LDP as a tunnel signaling protocol really doesn=92t= need any new extension. Hope this clarifies it. Thanks, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Thursday, August 07, 2014 10:37 AM To: Yimin Shen Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin hi, Please see some comments inline below. Regards, Sasha Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 From: Yimin Shen [mailto:yshen@juniper.net] Sent: Thursday, August 07, 2014 5:34 AM To: Alexander Vainshtein Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Hi Sasha, Thanks again for the comments. Please see the following 4 items in response= to the 4 comments in your email. 1. Today when an egress PE detects an egress AC failure [[Sasha]] Could you please explain how the egress PE detects an egress AC f= ailure? I have always thought that egress AC failure has to be detected fir= st by corresponding CE which would then notify the egress PE using some sui= table =93Remote Failure=94/BDI/RDI mechanism. Specific examples would reall= y help. it will simply have to drop packets, until the remote PE stops sending traf= fic over the PW. This draft is intended to improve this behavior. By no mea= ns the local repair on egress PE will prevent the AC failure from being con= verted to and handled as a bi-directional AC failure. Note that bi-directio= nal AC failure handling must involve the CE, but the CE may detect the fail= ure at a slower pace, or it may not be able to detect it in a timely manner= , depending on the actual situation. [[Sasha]] Please see above. IMHO and FWIW if the CE cannot detect its =93Re= ceive Failure=94, the PE would not never know about the =93Transmit Failure= . E.g. there may be other L2 devices between the PE and CE; the AC may itself= be a logical wire across another network; the type and location of the fai= lure may make it easier for the PE to detect it; and so on. In any case, be= fore the AC failure is handled more gracefully (or bi-directionally), the e= gress traffic has been locally repaired by the egress PE. 2. As mentioned before, this draft doesn=92t have a dependency on a specifi= c PSN tunnel technology. [[Sasha]] I have provided my comments on this in my response to Eric. From = my point of view some LDP-specific issues still require clarification, but = my bottom line is that operating the draft over an MPLS network that uses L= DP in DU mode for setting up tunnel LSPs may sometimes be possible but woul= d always be tricky. This does not match my understanding of =93no dependenc= y on a specific PSN tunnel technology=94, but this may be a matter of taste= . 3. The draft already talks about scalability when describing the centralize= d protector model vs the co-located protector model. We can improve the tex= t to explicitly mention that the requirement for setting up LSPs on a per <= primary PE, protector> basis rather than per primary PE basis can potential= ly increase the number of LSPs in a network. This is a tradeoff. As mention= ed before, this does not necessarily cause a scaling issue. [[Sasha]] Yes, I have found the corresponding text in the draft. From my p= oint of view this issue is resolved. 4. We will leave this to PWE3 WG to decide whether this is really necessary= .[[Sasha]] I tend to agree with Eric that usage of context-specific labels = in the draft does not mean update to 4447. From my point of view this issue= is resolved as well. Thanks, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Wednesday, August 06, 2014 5:11 AM To: Yimin Shen Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin and all, I would like to summarize the issues that I have with this draft. Digging f= or them in a very long email thread with multiple levels of in-lining is no= t very effective IMO. 1. Protection against unidirectional AC failures looks problematic to= me: a. The draft explicitly differentiates between =93ingress=94 and =93e= gress=94 AC failures b. The outcome can be using a different AC for each direction of CE-to= -CE traffic with potentially destructive effect that cannot be mitigated at= the PWE3 level c. IMHO and FWIW a viable alternative could be to employ native AC me= chanisms converting unidirectional AC failures into bi-directional ones. Th= is would eliminate the differentiation between ingress and egress AC failur= es and facilitate using the PW redundancy mechanism for bi-directional prot= ection 2. Dependency upon specific mechanisms for setting up PSN tunnels see= ms to break some common PWE3 architectural principles: a. I am pretty well sure that the mechanisms specified in the draft c= annot be used with the PSN tunnel LSPs set up by LDP in DU mode b. If the authors, refer to LDP in the DoD mode augmented with some CR= -LDP-like mechanisms (e.g., for exclusion of specific links when setting up= the bypass LSP between the PLR and the Protector), this should, at the ver= y least, be discussed with and approved by the MPLS WG 3. Scalability of the proposed solution should be, at least, explicit= ly presented. In particular, the draft assumes that a dedicated LSP would b= e used between PE1 PE2 for each combination of PE2 with a specific Protect= or LSR. 4. The Protector LSR could be either a kind of S-PE (if different fro= m the secondary T-PE) or the secondary T-PE itself. In both cases it should= be capable of handling labels from specific context spaces as if they were= PW labels. This requires (at the very least) an update of RFC 4447 and sho= uld be explicitly presented and discussed. Hopefully this summary will be useful. Regards, Sasha Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 --_000_140764772269418724ecitelecom_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable

Yimin hi,

Lots of thanks for a prompt response.

 

I must admit that I have missed the proposal in Section 4.2.2 to treat &= quot;context ID" IP address as a proxy node that is adjacent to both t= he Primary PE and the Protector NEs. (It also has not been mentioned in Eri= =E7's clarification email). I think that this approach would require certain manipulation of both the IGP and LDP, but t= his requires more thought on my side. One of the issues that come to m= ind, is the need for LDP in both these nodes to allocate a l= ocally terminated label for the corresponding FEC while not advertising context IP address as one of the IP addresses it owns.

 

I think that your statement about "avoiding ECMP  scenario&quo= t; is problematic, because, from my point of view,  ECMP is = one of the important characteristics of LDP-based MPLS PSN.  = ;And in any case the draft does not say anything about that (or, at least&n= bsp;neither ECMP nor "equal cost multipath" is mentioned in the te= xt).  If you see ECMP avoidance as a pre-condition for using the mecha= nism defined in draft with LDP LSPs, this should be quite explicit and deta= iled IMO.

 

I still fail to understand  what constitutes an egress AC failure i= n the draft and how it is different from the ingress AC failure. Unfortunat= ely the examples you've provided in your response do not help much: e.g., a= fiber cut close to  would be most probably recognized by the PE as a bi-directional AC failure, while a fiber cut tha= t is close to CE but remote from PE would most probably not detected by the= PE. (Of course if PE and CE are terminating the same fiber span, the fiber= cut would be recognized by both).

 

My guess (FWIW) that the draft actually discusses protection of traffic = in the PE --> CE direction triggered by a failure detected by the PE whi= le leaving protection of traffic in the CE --> PE direction some other m= echanism that would be triggered by CE detecting the failure. This mechanism presumably would be different from t= he PW redundancy mechanism because it would require that an active-active s= cheme for Primary and Backup  PWs.

 

If this guess is correct (or close to reality), I would expect some expl= icit text explaining it. In particular, this text should explain that "= ;redundant PWs" mentioned in the draft are not related to "PW red= undancy". The current text makes a mix of these two notions, and this definitely does not help the reader (or at least thi= s reader :-).

 

Hopefully these comments would be useful.

 

Regards,

Sasha

From: Yimin Shen <yshe= n@juniper.net>
Sent: Friday, August 8, 2014 5:34 PM
To: Alexander Vainshtein
Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01
 

Hi Sasha,

 

Thanks agai= n for the comments and questions!

 

Regarding A= C failure detection on PE, in the typical cases, if there is a fiber cut, o= r a hardware failure affecting the port on the PE, or a hardware failure af= fecting the port on the CE (assume the CE is directed attached to the PE), the PE should be able to directly and = independently detect the failure at physical or link layer.

 

Regarding L= DP, I agree with you that LDP by its nature is not as straight-forward as R= SVP-TE in this scenario. Since a context ID are advertised by both primary = PE and protector, we should avoid creating an ECMP scenario. That=92s why section 4.2.2 provides a suggestion to adve= rtise the context ID as a proxy node (i.e. a virtual node attached to the t= wo routers), and make its reachability via the primary PE more favorable th= an that via the protector. MRT and other mechanisms could be used to help LDP improve protection coverage. Al= so, nothing would prevent LDP from using RSVP bypass tunnels. As you can se= e, there may be many different approaches, and there may new approaches com= ing in the future. In this PWE3 document, we just want to stay at PW level and be generic with PSN level. = In any case, this mechanism is achievable with LDP, and LDP as a tunnel sig= naling protocol really doesn=92t need any new extension.  

 

Hope this c= larifies it.

 

Thanks,

 

/Yimin

 

 

From: Alex= ander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com]
Sent: Thursday, August 07, 2014 10:37 AM
To: Yimin Shen
Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Yimin hi,<= /span>

Please see= some comments inline below.

 

Regards,

 &nbs= p;     Sasha

Email: Alexander.Vainshtein@ecitele.com

Mobile: 05= 4-9266302

 

From: Yimi= n Shen [mailto:yshen@juniper.net]
Sent: Thursday, August 07, 2014 5:34 AM
To: Alexander Vainshtein
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Hi Sasha,

 

Thanks agai= n for the comments. Please see the following 4 items in response to the 4 c= omments in your email.

 

1. Today wh= en an egress PE detects an egress AC failure

&nbs= p;

[[Sa= sha]] Could you please explain how the egress PE detects an egress AC failu= re? I have always thought that egress AC failure has to be detected first b= y corresponding CE which would then notify the egress PE using some suitable =93Remote Failure=94/BDI/RDI mech= anism. Specific examples would really help.

&nbs= p;

it will sim= ply have to drop packets, until the remote PE stops sending traffic over th= e PW. This draft is intended to improve this behavior. By no means the loca= l repair on egress PE will prevent the AC failure from being converted to and handled as a bi-directional AC fail= ure. Note that bi-directional AC failure handling must involve the CE, but = the CE may detect the failure at a slower pace, or it may not be able to de= tect it in a timely manner, depending on the actual situation. =

[[Sa= sha]] Please see above. IMHO and FWIW if the CE cannot detect its =93Receiv= e Failure=94, the PE would not never know about the =93Transmit Failure.

&nbs= p;

E.g. there = may be other L2 devices between the PE and CE; the AC may itself be a logic= al wire across another network; the type and location of the failure may ma= ke it easier for the PE to detect it; and so on. In any case, before the AC failure is handled more gracefully (= or bi-directionally), the egress traffic has been locally repaired by the e= gress PE.

 

2. As menti= oned before, this draft doesn=92t have a dependency on a specific PSN tunne= l technology.

[[Sa= sha]] I have provided my comments on this in my response to Eric. From my p= oint of view some LDP-specific issues still require clarification, but my b= ottom line is that operating the draft over an MPLS networ= k that uses LDP in DU mode for setting up tunnel LSPs may sometimes be poss= ible but would always be tricky. This does not match my understandin= g of =93no dependency on a specific PSN tunnel technology=94, but this may be a matter of taste.

 

3. The draf= t already talks about scalability when describing the centralized protector= model vs the co-located protector model. We can improve the text to explic= itly mention that the requirement for setting up LSPs on a per <primary PE, protector> basis rather than p= er primary PE basis can potentially increase the number of LSPs in a networ= k. This is a tradeoff. As mentioned before, this does not necessarily cause= a scaling issue.

[[Sa= sha]] Yes, I have found the corresponding text in the draft.  From my = point of view this issue is resolved.

 

4. We will = leave this to PWE3 WG to decide whether this is really necessary.= [[Sasha]] I tend to agree with= Eric that usage of context-specific labels in the draft does not mean update to 4447. From my point of view this issu= e is resolved as well.

 

Thanks,

 

/Yimin

 

 

From: Alex= ander Vainshtein [mailt= o:Alexander.Vainshtein@ecitele.com]
Sent: Wednesday, August 06, 2014 5:11 AM
To: Yimin Shen
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Yimin and all,

I would like to summarize the issues that I have = with this draft. Digging for them in a very long email thread with multiple= levels of in-lining is not very effective IMO.

 

1.     &= nbsp; Protection against unidirectional AC failures looks pr= oblematic to me:

a.     &nb= sp; The draft explicitly differentiates between =93ingress=94 and= =93egress=94 AC failures

b.      The outcome can be using a different AC for each direction of= CE-to-CE traffic with potentially destructive effect that cannot be mitiga= ted at the PWE3 level

c.     &nb= sp; IMHO and FWIW a viable alternative could be to employ native = AC mechanisms converting unidirectional AC failures into bi-directional one= s. This would eliminate the differentiation between ingress and egress AC f= ailures and facilitate using the PW redundancy mechanism for bi-directional protection

2.     &= nbsp; Dependency upon specific mechanisms for setting up PSN tun= nels seems to break some common PWE3 architectural principles:

a.     &nb= sp; I am pretty well sure that the mechanisms specified in the dr= aft cannot be used with the PSN tunnel LSPs set up by LDP in DU mode

b.      If the authors, refer to LDP in the DoD mode augmented= with some CR-LDP-like mechanisms (e.g., for exclusion of specific links wh= en setting up the bypass LSP between the PLR and the Protector), this shoul= d, at the very least, be discussed with and approved by the MPLS WG

3.     &= nbsp; Scalability of the proposed solution should be, at lea= st, explicitly presented. In particular, the draft assumes that a dedicated= LSP would be used between PE1  PE2 for each combination of PE2 with a= specific Protector LSR.

4.     &= nbsp; The Protector LSR could be either a kind of S-PE (if differen= t from the secondary T-PE) or the secondary T-PE itself. In both cases it s= hould be capable of handling labels from specific context spaces as if they= were PW labels. This requires (at the very least) an update of RFC 4447 and should be explicitly presented a= nd discussed.

Hopefully this summary will be useful.

 

 

Regards,

       Sasha

Email: Alexander.Vainshtein@ecitele.com

Mobile: 054-9266302

--_000_140764772269418724ecitelecom_-- From nobody Sun Aug 10 02:24:09 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 494181A06A8 for ; Sun, 10 Aug 2014 02:24:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -15.168 X-Spam-Level: X-Spam-Status: No, score=-15.168 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-0.668, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EFEu7nEn8GhW for ; Sun, 10 Aug 2014 02:24:06 -0700 (PDT) Received: from rcdn-iport-8.cisco.com (rcdn-iport-8.cisco.com [173.37.86.79]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E52281A06A2 for ; Sun, 10 Aug 2014 02:24:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=1946; q=dns/txt; s=iport; t=1407662646; x=1408872246; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=vYJ8pPnVx3wnMakVLrLsEf3dTbTzYuARh87O95xXOYA=; b=d8uBUrFYtsxE4Ud6PAHmzG6mEsWov2db+hA/YP9pnEu7knn3eOtFURHH 15wN9GViZg9o85eFHkGbMXdx8davuaI00qcXo3Hvsiro9HCHGMF+IfygZ DxARWFxgmnOpOibEHdYeZcfOOKdTkja5NP1ees8lSyX0iASlBR3EcBoId Q=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgUFAEw551OtJV2Z/2dsb2JhbABZgw3WFAGBBBZ3hAQBAQQdXBACAQgEAUEyJQIEDohHwRQXj0wHgy+BHQEEkRmLFpR7g1w X-IronPort-AV: E=Sophos;i="5.01,835,1400025600"; d="scan'208,217";a="346402930" Received: from rcdn-core-2.cisco.com ([173.37.93.153]) by rcdn-iport-8.cisco.com with ESMTP; 10 Aug 2014 09:24:04 +0000 Received: from xhc-aln-x11.cisco.com (xhc-aln-x11.cisco.com [173.36.12.85]) by rcdn-core-2.cisco.com (8.14.5/8.14.5) with ESMTP id s7A9O4FV022634 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Sun, 10 Aug 2014 09:24:04 GMT Received: from xmb-rcd-x04.cisco.com ([169.254.8.74]) by xhc-aln-x11.cisco.com ([173.36.12.85]) with mapi id 14.03.0195.001; Sun, 10 Aug 2014 04:24:04 -0500 From: "Stewart Bryant (stbryant)" To: Alexander Vainshtein Thread-Topic: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 Thread-Index: AQHPqymWveYGLBru1kCsLuzIRj9tdJu3CWCAgAA8XgCAAPc/IIABBtiAgAg4EiaAAOtlgIAAs2DLgAA1qVCAASlTgIAAxW1QgAGWRICAAnyxroAAUUlr Date: Sun, 10 Aug 2014 09:24:03 +0000 Message-ID: <004289D0-C473-4C1D-8A76-B42757F599A2@cisco.com> References: <201407251524.s6PFOPn92012@magenta.juniper.net> <53D79504.4050904@cisco.com> <04f15bedf2be4c7480d3e9ca01bdfd7f@BY2PR05MB728.namprd05.prod.outlook.com> <2597291f7b074922b690e4b06999cf1a@AM3PR03MB612.eurprd03.prod.outlook.com>, <30e13900814f41a681151971cbe9ebef@BY2PR05MB728.namprd05.prod.outlook.com> <1407215578702.64387@ecitele.com>, <2280b8e0d5f94a60badab91a2237181b@BY2PR05MB728.namprd05.prod.outlook.com> <1407304503657.72312@ecitele.com> <046050cb2d254a139b4650b2d7f93075@BY2PR05MB728.namprd05.prod.outlook.com> <09ab595ab24a46a0a4e87ebeb2cd185a@AM3PR03MB612.eurprd03.prod.outlook.com>, <2afb5c963e054aef929f38dcb3defa07@BY2PR05MB728.namprd05.prod.outlook.com>, <1407647722694.18724@ecitele.com> In-Reply-To: <1407647722694.18724@ecitele.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: multipart/alternative; boundary="_000_004289D0C4734C1D8A76B42757F599A2ciscocom_" MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/vMZKY3hAMJBsGgCetkgsPtr1emw Cc: Yakov Rekhter , "pwe3@ietf.org" , "Eric Rosen \(erosen\)" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Aug 2014 09:24:07 -0000 --_000_004289D0C4734C1D8A76B42757F599A2ciscocom_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable ECMP paths to xPEs naturally form and disappear in LDP PSNs (they are a fun= damental property of that PSN type) and it is entirely normal to find an E= CMP to an xPE. So, if you have a PW that is load balancing (again quite normal) and one of= the attachments to the xPE fails, you can legitimately end up with some t= raffic on one egress AC and some on another. please you confirm that this design works correctly when a PE gets its traf= fic from both ACs? Stewart --_000_004289D0C4734C1D8A76B42757F599A2ciscocom_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

ECMP paths to xPEs naturally form and disappear in LDP PSNs (they are = a fundamental property of that PSN type) and it is entirely normal  to= find an ECMP to an xPE.

So, if you have a PW that is load balancing (again quite normal) and o= ne of the attachments to the xPE fails, you can legitimately end  up w= ith some traffic on one egress AC and some on another.

please you confirm that this design works correctly when a PE gets its= traffic from both ACs?

Stewart

--_000_004289D0C4734C1D8A76B42757F599A2ciscocom_-- From nobody Sun Aug 10 03:22:58 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DF2041A06C3 for ; Sun, 10 Aug 2014 03:22:56 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.901 X-Spam-Level: X-Spam-Status: No, score=-1.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Z6MYsT2qgVZv for ; Sun, 10 Aug 2014 03:22:54 -0700 (PDT) Received: from emea01-am1-obe.outbound.protection.outlook.com (mail-am1lrp0010.outbound.protection.outlook.com [213.199.154.10]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7025C1A06C2 for ; Sun, 10 Aug 2014 03:22:53 -0700 (PDT) Received: from AM3PR03MB612.eurprd03.prod.outlook.com (10.242.110.144) by AM3PR03MB609.eurprd03.prod.outlook.com (10.242.109.149) with Microsoft SMTP Server (TLS) id 15.0.1005.10; Sun, 10 Aug 2014 10:22:51 +0000 Received: from AM3PR03MB612.eurprd03.prod.outlook.com ([10.242.110.144]) by AM3PR03MB612.eurprd03.prod.outlook.com ([10.242.110.144]) with mapi id 15.00.1005.008; Sun, 10 Aug 2014 10:22:50 +0000 From: Alexander Vainshtein To: "Stewart Bryant (stbryant)" Thread-Topic: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 Thread-Index: AQHPqymWveYGLBru1kCsLuzIRj9tdJu3CWCAgAA8XgCAAPc/IIABBtiAgAg4EiaAAOtlgIAAs2DLgAA1qVCAASlTgIAAxW1QgAGWRICAAnyxroAAUUlrgAAN14A= Date: Sun, 10 Aug 2014 10:22:50 +0000 Message-ID: References: <201407251524.s6PFOPn92012@magenta.juniper.net> <53D79504.4050904@cisco.com> <04f15bedf2be4c7480d3e9ca01bdfd7f@BY2PR05MB728.namprd05.prod.outlook.com> <2597291f7b074922b690e4b06999cf1a@AM3PR03MB612.eurprd03.prod.outlook.com>, <30e13900814f41a681151971cbe9ebef@BY2PR05MB728.namprd05.prod.outlook.com> <1407215578702.64387@ecitele.com>, <2280b8e0d5f94a60badab91a2237181b@BY2PR05MB728.namprd05.prod.outlook.com> <1407304503657.72312@ecitele.com> <046050cb2d254a139b4650b2d7f93075@BY2PR05MB728.namprd05.prod.outlook.com> <09ab595ab24a46a0a4e87ebeb2cd185a@AM3PR03MB612.eurprd03.prod.outlook.com>, <2afb5c963e054aef929f38dcb3defa07@BY2PR05MB728.namprd05.prod.outlook.com>, <1407647722694.18724@ecitele.com> <004289D0-C473-4C1D-8A76-B42757F599A2@cisco.com> In-Reply-To: <004289D0-C473-4C1D-8A76-B42757F599A2@cisco.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [147.234.56.21] x-microsoft-antispam: BCL:0;PCL:0;RULEID:;UriScan:; x-forefront-prvs: 029976C540 x-forefront-antispam-report: SFV:NSPM; SFS:(10019004)(6009001)(252514010)(377454003)(199002)(189002)(106356001)(2656002)(92566001)(74502001)(74662001)(19625215002)(46102001)(85852003)(20776003)(31966008)(77982001)(101416001)(87936001)(76576001)(54356999)(83072002)(4396001)(110136001)(76482001)(99396002)(16236675004)(79102001)(15202345003)(33646002)(19580395003)(105586002)(19300405004)(81542001)(83322001)(81342001)(107046002)(66066001)(85306004)(50986999)(64706001)(76176999)(93886004)(19580405001)(21056001)(80022001)(86362001)(74316001)(15975445006)(95666004)(24736002)(108616003); DIR:OUT; SFP:1102; SCL:1; SRVR:AM3PR03MB609; H:AM3PR03MB612.eurprd03.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; LANG:en; Content-Type: multipart/alternative; boundary="_000_ee672591563c40fda0cd2debd238db26AM3PR03MB612eurprd03pro_" MIME-Version: 1.0 X-OriginatorOrg: ecitele.com Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/KeXDvhOPa_hpQ4cD7CvNb81wjrI Cc: Yakov Rekhter , "pwe3@ietf.org" , "Eric Rosen \(erosen\)" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Aug 2014 10:22:57 -0000 --_000_ee672591563c40fda0cd2debd238db26AM3PR03MB612eurprd03pro_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Stewart, My reading of your email is like following: 1. Within the scope of the draft the PLR is always the penultimate P = LSR on the LSP terminated by the Primary PE. I believe that the draft state= s that explicitly 2. The PLR cannot differentiate between the failure of the link conne= cting it to the Primary PE and failure of the Primary PE in a timely manner= ; hence every time such a link fails, the PLR assumes failure of the Primar= y PE to be on the safe side. T 3. With ECMP in the PSN, there typically would be several LSPs termin= ated by the Primary PE, each with its own PLR. So if the link between one o= f these PLRs and the Primary PE fails, it would apply the mechanisms define= d in the draft, while the rest of the PLRs would not do anything. 4. As a consequence, the dual-homed PE would receive some flows of th= e traffic crossing a flow-aware pw from one AC and some - from the other AC= . The draft does not specify that this is possible and that the CE should = be able to cope with this situation. Is my interpretation correct? Regards, Sasha Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 From: Stewart Bryant (stbryant) [mailto:stbryant@cisco.com] Sent: Sunday, August 10, 2014 12:24 PM To: Alexander Vainshtein Cc: Yimin Shen; Yakov Rekhter; pwe3@ietf.org; Eric Rosen (erosen) Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 ECMP paths to xPEs naturally form and disappear in LDP PSNs (they are a fun= damental property of that PSN type) and it is entirely normal to find an E= CMP to an xPE. So, if you have a PW that is load balancing (again quite normal) and one of= the attachments to the xPE fails, you can legitimately end up with some t= raffic on one egress AC and some on another. please you confirm that this design works correctly when a PE gets its traf= fic from both ACs? Stewart --_000_ee672591563c40fda0cd2debd238db26AM3PR03MB612eurprd03pro_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Stewart,

 <= /p>

My reading of your email = is like following:

1.&nbs= p;      Within the scope of the draft the PLR is always the penultimate P L= SR on the LSP terminated by the Primary PE. I believe that the draft states that explicitly

2.&nbs= p;      The PLR cannot differentiate between the failure of the link connec= ting it to the Primary PE and failure of the Primary PE in a timely manner; hence every time such a link fails, the PLR assumes fa= ilure of the Primary PE to be on the safe side. T

3.&nbs= p;      With ECMP in the PSN, there typically would be several LSPs termina= ted by the Primary PE, each with its own PLR. So if the link between one of these PLRs and the Primary PE fails, it would apply th= e mechanisms defined in the draft, while the rest of the PLRs would not do = anything.

4.&nbs= p;      As a consequence, the dual-homed PE would receive some flows of the= traffic crossing a flow-aware pw from one AC and some – from the other AC.  The draft does not specify that this is possible = and that the CE should be able to cope with this situation.

 <= /p>

Is my interpretation corr= ect?

 <= /p>

Regards,

    &= nbsp;  Sasha

Email: Alexander.Vainshte= in@ecitele.com

Mobile: 054-9266302<= /o:p>

 <= /p>

From: Stewart = Bryant (stbryant) [mailto:stbryant@cisco.com]
Sent: Sunday, August 10, 2014 12:24 PM
To: Alexander Vainshtein
Cc: Yimin Shen; Yakov Rekhter; pwe3@ietf.org; Eric Rosen (erosen) Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

 

ECMP paths to xPEs naturally form and disappear in L= DP PSNs (they are a fundamental property of that PSN type) and it is entire= ly normal  to find an ECMP to an xPE.

 

So, if you have a PW that is load balancing (again q= uite normal) and one of the attachments to the xPE fails, you can legitimat= ely end  up with some traffic on one egress AC and some on another.

 

please you confirm that this design works correctly = when a PE gets its traffic from both ACs?

 

Stewart

 

--_000_ee672591563c40fda0cd2debd238db26AM3PR03MB612eurprd03pro_-- From nobody Sun Aug 10 06:38:33 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A7E7A1A071B for ; Sun, 10 Aug 2014 06:38:31 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -15.168 X-Spam-Level: X-Spam-Status: No, score=-15.168 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-0.668, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9Lx-tIJT_1S1 for ; Sun, 10 Aug 2014 06:38:28 -0700 (PDT) Received: from alln-iport-2.cisco.com (alln-iport-2.cisco.com [173.37.142.89]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6AD5D1A0718 for ; Sun, 10 Aug 2014 06:38:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=13526; q=dns/txt; s=iport; t=1407677908; x=1408887508; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=DhXLfdDs+MVEF9iWVm968R6/eX61BEjbqii7OllzfB0=; b=VWV2j8HGlWFYtE9RRPHJ+xEZyQqP3vl5YRGb6U4U1w8u3lal8v8UBNZo vdDMtZP6NfVxY41YRf0ISwfeo5NK317Zcz5DCiwtoXaxhbRUs217VxOpu voU5h1zfEYCmdjNgd/GvIP35pxAaZwX8xN3xPsKnCqglMdOFd3hFjJ+71 A=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AiYFABN151OtJA2H/2dsb2JhbABagkdGgSnUcAGBBRZ3hAMBAQEEHRBMEAIBCBEEAQEoBzIUCQgCBA4FiELBGBeOahEBLCQGAYMvgR0FjUiBPoITixaUe4NcbIEO X-IronPort-AV: E=Sophos; i="5.01,836,1400025600"; d="scan'208,217"; a="68014843" Received: from alln-core-2.cisco.com ([173.36.13.135]) by alln-iport-2.cisco.com with ESMTP; 10 Aug 2014 13:38:27 +0000 Received: from xhc-rcd-x15.cisco.com (xhc-rcd-x15.cisco.com [173.37.183.89]) by alln-core-2.cisco.com (8.14.5/8.14.5) with ESMTP id s7ADcR6i008354 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Sun, 10 Aug 2014 13:38:27 GMT Received: from xmb-rcd-x04.cisco.com ([169.254.8.74]) by xhc-rcd-x15.cisco.com ([173.37.183.89]) with mapi id 14.03.0195.001; Sun, 10 Aug 2014 08:38:27 -0500 From: "Stewart Bryant (stbryant)" To: Alexander Vainshtein Thread-Topic: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 Thread-Index: AQHPqymWveYGLBru1kCsLuzIRj9tdJu3CWCAgAA8XgCAAPc/IIABBtiAgAg4EiaAAOtlgIAAs2DLgAA1qVCAASlTgIAAxW1QgAGWRICAAnyxroAAUUlrgAAN14CAADk9+Q== Date: Sun, 10 Aug 2014 13:38:26 +0000 Message-ID: References: <201407251524.s6PFOPn92012@magenta.juniper.net> <53D79504.4050904@cisco.com> <04f15bedf2be4c7480d3e9ca01bdfd7f@BY2PR05MB728.namprd05.prod.outlook.com> <2597291f7b074922b690e4b06999cf1a@AM3PR03MB612.eurprd03.prod.outlook.com>, <30e13900814f41a681151971cbe9ebef@BY2PR05MB728.namprd05.prod.outlook.com> <1407215578702.64387@ecitele.com>, <2280b8e0d5f94a60badab91a2237181b@BY2PR05MB728.namprd05.prod.outlook.com> <1407304503657.72312@ecitele.com> <046050cb2d254a139b4650b2d7f93075@BY2PR05MB728.namprd05.prod.outlook.com> <09ab595ab24a46a0a4e87ebeb2cd185a@AM3PR03MB612.eurprd03.prod.outlook.com>, <2afb5c963e054aef929f38dcb3defa07@BY2PR05MB728.namprd05.prod.outlook.com>, <1407647722694.18724@ecitele.com> <004289D0-C473-4C1D-8A76-B42757F599A2@cisco.com>, In-Reply-To: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: multipart/alternative; boundary="_000_CE90DCBD03CB4E6EB6A0AC35C1D0D0B7ciscocom_" MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/_Coh2PxPOqzt7Cdcr5kEvOWmgTE Cc: Yakov Rekhter , "pwe3@ietf.org" , "Eric Rosen \(erosen\)" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Aug 2014 13:38:31 -0000 --_000_CE90DCBD03CB4E6EB6A0AC35C1D0D0B7ciscocom_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable That is my interpretation. Now they could specify no PW ecmp but that restricts the scope of the draft= and must be both stated and tested against the requirements. Stewart On 10 Aug 2014, at 11:23, "Alexander Vainshtein" > wrote: Stewart, My reading of your email is like following: 1. Within the scope of the draft the PLR is always the penultimate P = LSR on the LSP terminated by the Primary PE. I believe that the draft state= s that explicitly 2. The PLR cannot differentiate between the failure of the link conne= cting it to the Primary PE and failure of the Primary PE in a timely manner= ; hence every time such a link fails, the PLR assumes failure of the Primar= y PE to be on the safe side. T 3. With ECMP in the PSN, there typically would be several LSPs termin= ated by the Primary PE, each with its own PLR. So if the link between one o= f these PLRs and the Primary PE fails, it would apply the mechanisms define= d in the draft, while the rest of the PLRs would not do anything. 4. As a consequence, the dual-homed PE would receive some flows of th= e traffic crossing a flow-aware pw from one AC and some =96 from the other = AC. The draft does not specify that this is possible and that the CE shoul= d be able to cope with this situation. Is my interpretation correct? Regards, Sasha Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 From: Stewart Bryant (stbryant) [mailto:stbryant@cisco.com] Sent: Sunday, August 10, 2014 12:24 PM To: Alexander Vainshtein Cc: Yimin Shen; Yakov Rekhter; pwe3@ietf.org; Eric Ro= sen (erosen) Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 ECMP paths to xPEs naturally form and disappear in LDP PSNs (they are a fun= damental property of that PSN type) and it is entirely normal to find an E= CMP to an xPE. So, if you have a PW that is load balancing (again quite normal) and one of= the attachments to the xPE fails, you can legitimately end up with some t= raffic on one egress AC and some on another. please you confirm that this design works correctly when a PE gets its traf= fic from both ACs? Stewart --_000_CE90DCBD03CB4E6EB6A0AC35C1D0D0B7ciscocom_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable
That is my interpretation.

Now they could specify no PW ecmp but that restricts the scope of the = draft and must be both stated and tested against the requirements.

Stewart



On 10 Aug 2014, at 11:23, "Alexander Vainshtein" <Alexander.Vainshtein@ecitele.com> wrote:

Stewart,

 <= /p>

My reading of your email = is like following:

1.=        Within the scope of the draft the PLR is always the penultimate= P LSR on the LSP terminated by the Primary PE. I believe that the draft states that explicitly

2.=        The PLR cannot differentiate between the failure of the link co= nnecting it to the Primary PE and failure of the Primary PE in a timely manner; hence every time such a link fails, the PLR assumes= failure of the Primary PE to be on the safe side. T

3.=        With ECMP in the PSN, there typically would be several LSPs ter= minated by the Primary PE, each with its own PLR. So if the link between one of these PLRs and the Primary PE fails, it would appl= y the mechanisms defined in the draft, while the rest of the PLRs would not= do anything.

4.=        As a consequence, the dual-homed PE would receive some flows of= the traffic crossing a flow-aware pw from one AC and some =96 from the other AC.  The draft does not specify that this is possi= ble and that the CE should be able to cope with this situation.<= /span>

 <= /p>

Is my interpretation corr= ect?

 <= /p>

 <= /p>

From: Stewart = Bryant (stbryant) [mailto:stbryant@ci= sco.com]
Sent: Sunday, August 10, 2014 12:24 PM
To: Alexander Vainshtein
Cc: Yimin Shen; Yakov Rekhter; pwe3= @ietf.org; Eric Rosen (erosen)
Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

 

ECMP paths to xPEs naturally form and disappear in L= DP PSNs (they are a fundamental property of that PSN type) and it is entire= ly normal  to find an ECMP to an xPE.

 

So, if you have a PW that is load balancing (again q= uite normal) and one of the attachments to the xPE fails, you can legitimat= ely end  up with some traffic on one egress AC and some on another.

 

please you confirm that this design works correctly = when a PE gets its traffic from both ACs?

 

Stewart

 

--_000_CE90DCBD03CB4E6EB6A0AC35C1D0D0B7ciscocom_-- From nobody Sun Aug 10 11:45:47 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 46BBB1A00DF for ; Sun, 10 Aug 2014 11:45:46 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.969 X-Spam-Level: X-Spam-Status: No, score=-6.969 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, GB_I_INVITATION=-2, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.668, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id X5Q-5xpMGCkq for ; Sun, 10 Aug 2014 11:45:44 -0700 (PDT) Received: from mailuogwdur.emc.com (mailuogwdur.emc.com [128.221.224.79]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B66AB1A00BA for ; Sun, 10 Aug 2014 11:45:43 -0700 (PDT) Received: from maildlpprd56.lss.emc.com (maildlpprd56.lss.emc.com [10.106.48.160]) by mailuogwprd53.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s7AIjfMT029195 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Sun, 10 Aug 2014 14:45:41 -0400 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd53.lss.emc.com s7AIjfMT029195 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=emc.com; s=jan2013; t=1407696342; bh=C7XkTAdmRD37sq54JCwg6SRvtXI=; h=From:To:Date:Subject:Message-ID:References:In-Reply-To: Content-Type:Content-Transfer-Encoding:MIME-Version; b=lkPx7LJi86pIZ7FavcNvDkLC4rpTH5V/kgBtbf7651VIQ04Rl8I+7ZixbvTrDHt5b HcFiIA0fQ1app0OLg7d4TZS30g0bFroBXTw68nprmp8HH3OcuSYg/IrkKG5K2bfTVK QkYOeoMyWt4E6HU6tP/tY0r4iuHQ0vgoZulll3N0= X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd53.lss.emc.com s7AIjfMT029195 Received: from mailusrhubprd53.lss.emc.com (mailusrhubprd53.lss.emc.com [10.106.48.18]) by maildlpprd56.lss.emc.com (RSA Interceptor) for ; Sun, 10 Aug 2014 14:45:31 -0400 Received: from mxhub08.corp.emc.com (mxhub08.corp.emc.com [128.222.70.205]) by mailusrhubprd53.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s7AIjUnA012182 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL) for ; Sun, 10 Aug 2014 14:45:31 -0400 Received: from mx15a.corp.emc.com ([169.254.1.175]) by mxhub08.corp.emc.com ([128.222.70.205]) with mapi; Sun, 10 Aug 2014 14:45:30 -0400 From: "Black, David" To: "Black, David" , "pwe3@ietf.org" Date: Sun, 10 Aug 2014 14:45:28 -0400 Thread-Topic: WGLC comments on: draft-ietf-pwe3-congcons-02.txt Thread-Index: Ac+zEP27iWho+8uAQLKBEFcFeq8EIgBLGLOAACM+uQA= Message-ID: <8D3D17ACE214DC429325B2B98F3AE712077B951940@MX15A.corp.emc.com> References: <8D3D17ACE214DC429325B2B98F3AE712077B9517F4@MX15A.corp.emc.com> <8D3D17ACE214DC429325B2B98F3AE712077B951928@MX15A.corp.emc.com> In-Reply-To: <8D3D17ACE214DC429325B2B98F3AE712077B951928@MX15A.corp.emc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd53.lss.emc.com X-RSA-Classifications: public Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/IIZenokSevpjjemhpDT_Br0dl0U Subject: Re: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Aug 2014 18:45:46 -0000 Ugh, the spelling checker "helped" - "consequent on the quality" :-(. Instead of this proposed new text from my original message: For TDM pseudowires, such a circuit breaker can be realized via operator monitoring of drop rates and consequent on the quality of TDM service delivered and shutting down TDM pseudowires that persistentl= y fail to deliver acceptable TDM service (as defined by [G775] and [G826]). This document explains why a TDM PW that is causing congestion problems via interference with elastic traffic cannot be delivering acceptable TDM service (see the rest of this Section, plus Appendices A and B). This would be better (also reformatted as a list for clarity): For TDM pseudowires, such a circuit breaker can be realized via: 1) network operator monitoring of drop rates and the resulting quality of TDM service delivered; and 2) a network operational practice of shutting down TDM pseudowires that persistently fail to deliver acceptable TDM service (as defined by [G775] and [G826]). This document explains why a TDM PW that is causing congestion problems via interference with elastic traffic is not delivering acceptable TDM service (see the rest of this section, plus Appendices A and B). Thanks, --David > -----Original Message----- > From: pwe3 [mailto:pwe3-bounces@ietf.org] On Behalf Of Black, David > Sent: Saturday, August 09, 2014 10:32 PM > To: Yaakov Stein; pwe3@ietf.org > Subject: Re: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt >=20 > Following up on this ... >=20 > > As we're now moving quickly to try to complete this draft , I agreed > > w/Yaakov that my author comments would be sent in as WG Last Call comme= nts > ... > > ... so here they are: >=20 > ... with some suggested new text ... >=20 > > Major issues: > > > > The reference to a transport circuit breaker in section 3 is a bit > optimistic > > and high level: > > > > We can accomplish this by > > employing a transport circuit breaker, by which we mean an automatic > > mechanism for terminating a flow to prevent negative impact on other > > flows and on the stability of the network > > [I-D.fairhurst-tsvwg-circuit-breaker]. > > > > This should be accompanied by at least an example of how to do this - p= ast > > discussion has suggested carrier monitoring of delivered service qualit= y > > in order to shut down PWs that are experiencing loss rates that result = in > > unacceptable service quality (as defined for us by the ITU ...). I'd a= lso > > like to add some text to suggest that operators use this sort of circui= t > > breaker practice when TDM PWs compete with elastic traffic. >=20 > After this text in Section 3: >=20 > We can accomplish this by > employing a transport circuit breaker, by which we mean an automatic > mechanism for terminating a flow to prevent negative impact on other > flows and on the stability of the network > [I-D.fairhurst-tsvwg-circuit-breaker]. Note that a transport circuit > breaker is intended as a protection mechanism of last resort, just as > an electrical circuit breaker is only triggered when absolutely > necessary. >=20 > I suggest adding: >=20 > For TDM pseudowires, such a circuit breaker can be realized via > operator monitoring of drop rates and consequent on the quality of > TDM service delivered and shutting down TDM pseudowires that persisten= tly > fail to deliver acceptable TDM service (as defined by [G775] and > [G826]). This document explains why a TDM PW that is causing congesti= on > problems via interference with elastic traffic cannot be delivering > acceptable TDM service (see the rest of this Section, plus Appendices > A and B). >=20 > And breaking the paragraph after the added text. >=20 > > Figure 1 and Figure 2 indicate that based on using the TCP throughput > > equation as a proxy for potential impact on elastic traffic (which is w= hat > > we're doing in this draft), for the same fixed bandwidth PW, a larger > segment > > size is less likely to cause impacts. I'd like to add a suggestion to = use > > the larger PW segment sizes as long as they're consistent with delivery= of > > the service. There's been a related (long-to-the-point-of-ratholed) de= bate > > over in the Transport Area about whether packet-rate-limited/congestibl= e > > forwarding is still relevant. My view is that such equipment exists, a= nd > > hence that this is a relevant consideration for this draft, and would n= ot > > mention that issue here. >=20 > Ignoring the rathole invitation ;-). This is still in section 3, after: >=20 > We see that in general for large packet sizes, short delays, and low > packet loss rates, the TDM pseudowires consume much less bandwidth > than TCP would under identical conditions. Only for small packets, > long delays, and high packet loss ratios, do TDM PWs potentially > consume more bandwidth, and even then only marginally. >=20 > I suggest adding: >=20 > Comparing figures 1 and 2, use of a larger segment size for a T1 or E1 > TDM PW moves the operational behavior of the PW away from the boundary > at which the TCP throughput equation suggests that undue interference = with > elastic traffic may start to occur. We suggest that larger segment > sizes be preferred for T1 and E1 TDM PWs, but note that other > considerations may favor smaller segment sizes (e.g., jitter buffer > size, bounding the impact of an isolated packet drop event). >=20 > and again, breaking the paragraph after that added text. >=20 > I hope that latter sentence ("We suggest that ...") is acceptable to the = WG. >=20 > > Minor issues: > > > > Section 2 is about Ethernet PWs that carry elastic traffic. Should oth= ers > > be mentioned as also covered by this analysis - e.g., ATM and Frame Rel= ay > > PWs carrying elastic traffic? >=20 > After: >=20 > In this section we consider Ethernet PWs that primarily carry > congestion-responsive traffic. We show that we automatically obtain > the desired congestion avoidance behavior, and that additional > mechanisms are not needed. >=20 > I suggest adding: >=20 > This discussion is applicable to any PW that carries congestion- > responsive traffic (e.g., ATM and Frame Relay PWs); Ethernet PWs are > used here for clarity, as they are a common example. >=20 > > Section 2 does not cover Ethernet PWs carrying inelastic traffic - that > would > > be unusual, but should it be explicitly noted as out of scope, or just = not > > discussed (as is the case for the current text)? >=20 > I suggest doing nothing here, and planning to deal with this concern > if it should arise later (e.g., IETF Last Call or IESG Evaluation). >=20 > > Section 3 should say that the reason the draft focuses on TDM PWs is th= at > > they're the inelastic PWs that are likely to be carried over IP in prac= tice. > > My understanding is that other potentially inelastic PWs are MPLS-only > > in practice. >=20 > After: >=20 > Inelastic PWs, such as TDM PWs ([RFC4553][RFC5086][RFC5087]), are > potentially more problematic than the elastic PWs of the previous > section. Being constant bit-rate (CBR), TDM PWs can not be made > responsive to congestion. On the other hand, being CBR, they also do > not attempt to capture additional bandwidth when neighboring TCP > flows back off. >=20 > I suggest adding: >=20 > We focus on TDM PWs here as they are the only inelastic PWs that are > likely to be IP-encapsulated in practice. Most other inelastic PWs > tend to only be used with MPLS. >=20 > [I'd ask Yaakov and other PW experts to double-check the latter sentence, > please.] >=20 > > A terminology section to roll up all the acronym expansions (e.g., TDM,= CBR) > > in one place seems like a good idea. >=20 > These four acronyms would be good to put into a terminology section: >=20 > CBR Constant Bit Rate > PSN Packet Switched Network > PW Pseudowire > TDM Time Division Multiplexing >=20 > > > > This text on p.11 could be incendiary: > > > > Under this condition it is safe to place the TDM PW along with > > congestion - responsive traffic such as TCP, without causing > > additional congestion. > > > > First of all to reduce the flammability, "safe" -> "reasonable" and the= re > > are a number of other occurrences of "safe" that deserve similar treatm= ent > ;- > > ). > > Beyond that, this assertion is dependent on the operating conditions st= aying > > within that reasonable operating envelope, which entails use of somethi= ng > > like a circuit breaker - I'd say that here, and reference the earlier > > circuit breaker discussion. >=20 > Global substitution: "safe" -> "reasonable", starting with the abstract. >=20 > Then for this specific text on p.11: >=20 > OLD > Under this condition it is safe to place the TDM PW along with > congestion - responsive traffic such as TCP, without causing > additional congestion. > NEW > Under this condition it is reasonable to place the TDM PW in competition > with congestion-responsive traffic such as TCP, as the result shoul= d not > produce unacceptable congestion effects. That assumes that the > operating > conditions remain within the reasonable operating envelope - as > discussed > above, a transport circuit breaker is a suggested means of managing= a > TDM > PW so that it is shut down if operating conditions remain persisten= tly > outside the reasonable operating envelope for the PW (e.g., due to = a > high packet loss rate). >=20 > > Figures 7 and 8 could use some shading or lines to indicate the typical > > deployed operating regions for the pseudo-wires to visibly show that th= ey > > fall under the curves, particularly for larger segment sizes. I'd also > > repeat or refer to the suggestion to use larger > > > > Nits: > > > > References aren't allowed in abstracts, so remove reference to > > [I-D.fairhurst-tsvwg-circuit-breaker] > > > > Introduction, first paragraph, delete "PWE3" towards end of paragraph - > > > > preferably the 4 byte PWE3 control word. > > > > In addition to its current general statements, the Security Considerati= ons > > section should point to the security considerations sections in a few > > particularly relevant RFCs - I'd suggest 3985, 4553, 5086 and 5087. >=20 > After: >=20 > This document does not introduce any new congestion-specific > mechanisms and thus does not introduce any new security > considerations above those present for PWs in general. >=20 > I suggest adding a new paragraph: >=20 > For PW security considerations, see the security considerations sectio= ns > of the cited RFCS; the security considerations in [RFC3985], [RFC4553]= , > [RFC5086] and [RFC5087] are particularly applicable to this document. >=20 > > > > Thanks, > > --David > > > > > > > -----Original Message----- > > > From: pwe3 [mailto:pwe3-bounces@ietf.org] On Behalf Of Yaakov Stein > > > Sent: Thursday, July 24, 2014 4:52 PM > > > To: pwe3@ietf.org > > > Subject: Re: [PWE3] I-D Action: draft-ietf-pwe3-congcons-02.txt > > > > > > Please look at the PDF version only > > > (the txt version does not have the graphs). > > > > > > Y(J)S > > > > > > > > > > _______________________________________________ > > pwe3 mailing list > > pwe3@ietf.org > > https://www.ietf.org/mailman/listinfo/pwe3 >=20 > _______________________________________________ > pwe3 mailing list > pwe3@ietf.org > https://www.ietf.org/mailman/listinfo/pwe3 From nobody Mon Aug 11 08:03:42 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2627A1A0451 for ; Mon, 11 Aug 2014 08:03:37 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.568 X-Spam-Level: X-Spam-Status: No, score=-2.568 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.668] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rzZq9pADee4w for ; Mon, 11 Aug 2014 08:03:30 -0700 (PDT) Received: from smtp-fr.alcatel-lucent.com (fr-hpgre-esg-02.alcatel-lucent.com [135.245.210.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 47AFF1A03E3 for ; Mon, 11 Aug 2014 08:03:30 -0700 (PDT) Received: from fr711usmtp1.zeu.alcatel-lucent.com (unknown [135.239.2.122]) by Websense Email Security Gateway with ESMTPS id BB0EF44829EB4 for ; Mon, 11 Aug 2014 15:03:24 +0000 (GMT) Received: from FR711WXCHHUB01.zeu.alcatel-lucent.com (fr711wxchhub01.zeu.alcatel-lucent.com [135.239.2.111]) by fr711usmtp1.zeu.alcatel-lucent.com (GMO) with ESMTP id s7BF2WY1029477 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL) for ; Mon, 11 Aug 2014 17:03:25 +0200 Received: from FR711WXCHMBA05.zeu.alcatel-lucent.com ([169.254.1.23]) by FR711WXCHHUB01.zeu.alcatel-lucent.com ([135.239.2.111]) with mapi id 14.02.0247.003; Mon, 11 Aug 2014 17:02:55 +0200 From: "Bocci, Matthew (Matthew)" To: "Bocci, Matthew (Matthew)" , "pwe3@ietf.org" Thread-Topic: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 Thread-Index: AQHPtXVU8ehNX8WWMUSGw5QQaKCCHA== Date: Mon, 11 Aug 2014 15:02:54 +0000 Message-ID: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.3.140616 x-originating-ip: [135.239.27.39] Content-Type: text/plain; charset="utf-8" Content-ID: <2884F2408BDAA0438DD6D8EC2F659B13@exchange.lucent.com> Content-Transfer-Encoding: base64 MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/Kmc87uv7oVmwEafiPY1xSu5CpfM Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Aug 2014 15:03:39 -0000 VGhpcyBlbWFpbCBlbmRzIHRoZSBXRyBMYXN0IENhbGwgZm9yDQpkcmFmdC1pZXRmLXB3ZTMtZW5k cG9pbnQtZmFzdC1wcm90ZWN0aW9uLTAxLiBUaGFua3MgdG8gZXZlcnkgb25lIHdobw0KcmV2aWV3 ZWQgYW5kIGNvbW1lbnRlZCBvbiB0aGUgZHJhZnQuIEFsdGhvdWdoIGl0IHNlZW1zIHRoYXQgdGhl cmUgaXMNCmludGVyZXN0LCB0aGUgZG9jdW1lbnQgYWxzbyBuZWVkcyB0byBiZSBzdWJzdGFudGlh bGx5IHJldmlzZWQsDQpwYXJ0aWN1bGFybHkgd2l0aCByZXNwZWN0IHRvIHRoZSBjbGFyaXR5IHdp dGggd2hpY2ggdGhlIHByb3Bvc2VkIG1lY2hhbmlzbQ0KaXMgZGVzY3JpYmVkLg0KDQpQbGVhc2Ug Y291bGQgdGhlIGF1dGhvcnMgd29yayB0byByZXNvbHZlIHRoZSBjb21tZW50cyBvbiB0aGUgbGlz dCBpbiBhIG5ldw0KdmVyc2lvbiBvZiB0aGUgZHJhZnQuIFdlIHdpbGwgbm90IHByb2dyZXNzIHRo ZSBkcmFmdCB0byBhbm90aGVyIFdHIGxhc3QNCmNhbGwgdW50aWwgaXQgaGFzIHJlYWNoZWQgYSBn cmVhdGVyIGxldmVsIG9mIG1hdHVyaXR5Lg0KDQpCZXN0IHJlZ2FyZHMNCg0KTWF0dGhldw0KDQpP biAyNS8wNy8yMDE0IDE1OjU2LCAiQm9jY2ksIE1hdHRoZXcgKE1hdHRoZXcpIg0KPG1hdHRoZXcu Ym9jY2lAYWxjYXRlbC1sdWNlbnQuY29tPiB3cm90ZToNCg0KPlRoaXMgZW1haWwgYmVnaW5zIGEg c2Vjb25kIHR3byB3ZWVrIHdvcmtpbmcgZ3JvdXAgbGFzdCBjYWxsIGZvcg0KPmh0dHA6Ly90b29s cy5pZXRmLm9yZy9odG1sL2RyYWZ0LWlldGYtcHdlMy1lbmRwb2ludC1mYXN0LXByb3RlY3Rpb24t MDEgLg0KPg0KPlBsZWFzZSByZXZpZXcgdGhlIGRyYWZ0IGFuZCBwb3N0IGFueSBjb21tZW50cyB0 byB0aGUgUFdFMyBsaXN0Lg0KPg0KPklmIHlvdSBoYXZlIHJlYWQgdGhlIGRyYWZ0LCBldmVuIGlm IHlvdSBkb27CuXQgaGF2ZSBhbnkgY29tbWVudHMsIGJ1dCB3b3VsZA0KPmxpa2UgdG8gc2VlIHRo ZSBkcmFmdCBwcm9ncmVzc2VkLCAgcGxlYXNlIGNhbiB5b3UgaW5kaWNhdGUgdGhpcyB0byB0aGUN Cj5saXN0Lg0KPg0KPlRoaXMgd29ya2luZyBncm91cCBsYXN0IGNhbGwgd2lsbCBlbmQgb24gRnJp ZGF5IEF1Z3VzdCA4dGgsIDIwMTQuDQo+DQo+DQo+QmVzdCByZWdhcmRzDQo+DQo+DQo+TWF0dGhl dyBhbmQgQW5keQ0KPg0KPl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fDQo+cHdlMyBtYWlsaW5nIGxpc3QNCj5wd2UzQGlldGYub3JnDQo+aHR0cHM6Ly93d3cu aWV0Zi5vcmcvbWFpbG1hbi9saXN0aW5mby9wd2UzDQoNCg== From nobody Mon Aug 11 10:06:33 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3E5301A06AB for ; Mon, 11 Aug 2014 10:06:30 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.902 X-Spam-Level: X-Spam-Status: No, score=-1.902 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4W5RbIcHSGGT for ; Mon, 11 Aug 2014 10:06:28 -0700 (PDT) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1blp0185.outbound.protection.outlook.com [207.46.163.185]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 10F081A069A for ; Mon, 11 Aug 2014 10:06:27 -0700 (PDT) Received: from BY2PR05MB728.namprd05.prod.outlook.com (10.141.223.25) by BY2PR05MB728.namprd05.prod.outlook.com (10.141.223.25) with Microsoft SMTP Server (TLS) id 15.0.1005.10; Mon, 11 Aug 2014 17:06:24 +0000 Received: from BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) by BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) with mapi id 15.00.1005.008; Mon, 11 Aug 2014 17:06:24 +0000 From: Yimin Shen To: "Bocci, Matthew (Matthew)" , "pwe3@ietf.org" Thread-Topic: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 Thread-Index: AQHPtXVUzRsIO8QlbkK+kbMBZ9MDf5vLoalw Date: Mon, 11 Aug 2014 17:06:24 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [66.129.241.10] x-microsoft-antispam: BCL:0;PCL:0;RULEID:;UriScan:; x-forefront-prvs: 03008837BD x-forefront-antispam-report: SFV:NSPM; SFS:(6009001)(164054003)(51704005)(24454002)(377454003)(199002)(189002)(13464003)(479174003)(54356999)(50986999)(106116001)(106356001)(105586002)(15202345003)(92566001)(76176999)(101416001)(79102001)(15975445006)(81342001)(83072002)(85852003)(4396001)(107046002)(77982001)(107886001)(76482001)(46102001)(21056001)(86362001)(85306004)(20776003)(95666004)(74662001)(19580395003)(81542001)(99286002)(99396002)(19580405001)(31966008)(66066001)(80022001)(83322001)(76576001)(2656002)(74502001)(87936001)(64706001)(74316001)(33646002)(2501001)(24736002)(108616003); DIR:OUT; SFP:; SCL:1; SRVR:BY2PR05MB728; H:BY2PR05MB728.namprd05.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; LANG:en; Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-OriginatorOrg: juniper.net Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/EU9hpe0bGj_dQJtzLsD-mYJ9ARI Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Aug 2014 17:06:30 -0000 VGhhbmtzIE1hdHRoZXchDQoNCldlIHdpbGwgY29tbXVuaWNhdGUgd2l0aCB0aGUgcmV2aWV3ZXJz IGFuZCBhZGRyZXNzIGFsbCB0aGUgY29tbWVudHMgaW4gdGhlIG5leHQgcmV2LiANCg0KDQpUaGFu a3MsDQoNCi9ZaW1pbg0KDQoNCg0KLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCkZyb206IHB3 ZTMgW21haWx0bzpwd2UzLWJvdW5jZXNAaWV0Zi5vcmddIE9uIEJlaGFsZiBPZiBCb2NjaSwgTWF0 dGhldyAoTWF0dGhldykNClNlbnQ6IE1vbmRheSwgQXVndXN0IDExLCAyMDE0IDExOjAzIEFNDQpU bzogQm9jY2ksIE1hdHRoZXcgKE1hdHRoZXcpOyBwd2UzQGlldGYub3JnDQpTdWJqZWN0OiBSZTog W1BXRTNdIFdHIExhc3QgQ2FsbCBmb3IgZHJhZnQtaWV0Zi1wd2UzLWVuZHBvaW50LWZhc3QtcHJv dGVjdGlvbi0wMQ0KDQpUaGlzIGVtYWlsIGVuZHMgdGhlIFdHIExhc3QgQ2FsbCBmb3INCmRyYWZ0 LWlldGYtcHdlMy1lbmRwb2ludC1mYXN0LXByb3RlY3Rpb24tMDEuIFRoYW5rcyB0byBldmVyeSBv bmUgd2hvIHJldmlld2VkIGFuZCBjb21tZW50ZWQgb24gdGhlIGRyYWZ0LiBBbHRob3VnaCBpdCBz ZWVtcyB0aGF0IHRoZXJlIGlzIGludGVyZXN0LCB0aGUgZG9jdW1lbnQgYWxzbyBuZWVkcyB0byBi ZSBzdWJzdGFudGlhbGx5IHJldmlzZWQsIHBhcnRpY3VsYXJseSB3aXRoIHJlc3BlY3QgdG8gdGhl IGNsYXJpdHkgd2l0aCB3aGljaCB0aGUgcHJvcG9zZWQgbWVjaGFuaXNtIGlzIGRlc2NyaWJlZC4N Cg0KUGxlYXNlIGNvdWxkIHRoZSBhdXRob3JzIHdvcmsgdG8gcmVzb2x2ZSB0aGUgY29tbWVudHMg b24gdGhlIGxpc3QgaW4gYSBuZXcgdmVyc2lvbiBvZiB0aGUgZHJhZnQuIFdlIHdpbGwgbm90IHBy b2dyZXNzIHRoZSBkcmFmdCB0byBhbm90aGVyIFdHIGxhc3QgY2FsbCB1bnRpbCBpdCBoYXMgcmVh Y2hlZCBhIGdyZWF0ZXIgbGV2ZWwgb2YgbWF0dXJpdHkuDQoNCkJlc3QgcmVnYXJkcw0KDQpNYXR0 aGV3DQoNCk9uIDI1LzA3LzIwMTQgMTU6NTYsICJCb2NjaSwgTWF0dGhldyAoTWF0dGhldykiDQo8 bWF0dGhldy5ib2NjaUBhbGNhdGVsLWx1Y2VudC5jb20+IHdyb3RlOg0KDQo+VGhpcyBlbWFpbCBi ZWdpbnMgYSBzZWNvbmQgdHdvIHdlZWsgd29ya2luZyBncm91cCBsYXN0IGNhbGwgZm9yDQo+aHR0 cDovL3Rvb2xzLmlldGYub3JnL2h0bWwvZHJhZnQtaWV0Zi1wd2UzLWVuZHBvaW50LWZhc3QtcHJv dGVjdGlvbi0wMSAuDQo+DQo+UGxlYXNlIHJldmlldyB0aGUgZHJhZnQgYW5kIHBvc3QgYW55IGNv bW1lbnRzIHRvIHRoZSBQV0UzIGxpc3QuDQo+DQo+SWYgeW91IGhhdmUgcmVhZCB0aGUgZHJhZnQs IGV2ZW4gaWYgeW91IGRvbsK5dCBoYXZlIGFueSBjb21tZW50cywgYnV0IA0KPndvdWxkIGxpa2Ug dG8gc2VlIHRoZSBkcmFmdCBwcm9ncmVzc2VkLCAgcGxlYXNlIGNhbiB5b3UgaW5kaWNhdGUgdGhp cyANCj50byB0aGUgbGlzdC4NCj4NCj5UaGlzIHdvcmtpbmcgZ3JvdXAgbGFzdCBjYWxsIHdpbGwg ZW5kIG9uIEZyaWRheSBBdWd1c3QgOHRoLCAyMDE0Lg0KPg0KPg0KPkJlc3QgcmVnYXJkcw0KPg0K Pg0KPk1hdHRoZXcgYW5kIEFuZHkNCj4NCj5fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fXw0KPnB3ZTMgbWFpbGluZyBsaXN0DQo+cHdlM0BpZXRmLm9yZw0KPmh0 dHBzOi8vd3d3LmlldGYub3JnL21haWxtYW4vbGlzdGluZm8vcHdlMw0KDQpfX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXw0KcHdlMyBtYWlsaW5nIGxpc3QNCnB3 ZTNAaWV0Zi5vcmcNCmh0dHBzOi8vd3d3LmlldGYub3JnL21haWxtYW4vbGlzdGluZm8vcHdlMw0K From nobody Mon Aug 11 13:16:01 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3A5081A0008 for ; Mon, 11 Aug 2014 13:15:56 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.601 X-Spam-Level: X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jxD27PcNTiAS for ; Mon, 11 Aug 2014 13:15:50 -0700 (PDT) Received: from na01-by2-obe.outbound.protection.outlook.com (mail-by2lp0243.outbound.protection.outlook.com [207.46.163.243]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 14D871A0002 for ; Mon, 11 Aug 2014 13:15:50 -0700 (PDT) Received: from BY2PR05MB222.namprd05.prod.outlook.com (10.242.41.151) by BY2PR05MB045.namprd05.prod.outlook.com (10.242.34.143) with Microsoft SMTP Server (TLS) id 15.0.1005.10; Mon, 11 Aug 2014 20:15:43 +0000 Received: from BY2PR05MB728.namprd05.prod.outlook.com (10.141.223.25) by BY2PR05MB222.namprd05.prod.outlook.com (10.242.41.151) with Microsoft SMTP Server (TLS) id 15.0.1005.10; Mon, 11 Aug 2014 20:15:38 +0000 Received: from BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) by BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) with mapi id 15.00.1005.008; Mon, 11 Aug 2014 20:15:38 +0000 From: Yimin Shen To: Alexander Vainshtein Thread-Topic: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 Thread-Index: AQHPqByPzRsIO8QlbkK+kbMBZ9MDf5u3AuAAgAAZ0oCAABjewIABFRcAgADf/TCACF0AgIAA3TgQgADA4ICAADbOAIAAzcgwgAEfc4CAAYjQ4IACkWQAgAKGg+A= Date: Mon, 11 Aug 2014 20:15:38 +0000 Message-ID: <6b0fa5df117f43d69c31efab88161e70@BY2PR05MB728.namprd05.prod.outlook.com> References: <201407251524.s6PFOPn92012@magenta.juniper.net> <53D79504.4050904@cisco.com> <04f15bedf2be4c7480d3e9ca01bdfd7f@BY2PR05MB728.namprd05.prod.outlook.com> <2597291f7b074922b690e4b06999cf1a@AM3PR03MB612.eurprd03.prod.outlook.com>, <30e13900814f41a681151971cbe9ebef@BY2PR05MB728.namprd05.prod.outlook.com> <1407215578702.64387@ecitele.com>, <2280b8e0d5f94a60badab91a2237181b@BY2PR05MB728.namprd05.prod.outlook.com> <1407304503657.72312@ecitele.com> <046050cb2d254a139b4650b2d7f93075@BY2PR05MB728.namprd05.prod.outlook.com> <09ab595ab24a46a0a4e87ebeb2cd185a@AM3PR03MB612.eurprd03.prod.outlook.com>, <2afb5c963e054aef929f38dcb3defa07@BY2PR05MB728.namprd05.prod.outlook.com> <1407647722694.18724@ecitele.com> In-Reply-To: <1407647722694.18724@ecitele.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [66.129.241.10] x-microsoft-antispam: BCL:0;PCL:0;RULEID:;UriScan:;UriScan:; x-forefront-prvs: 03008837BD x-forefront-antispam-report: SFV:NSPM; SFS:(164054003)(43784003)(37854004)(377454003)(199002)(189002)(252514010)(85306004)(93886004)(19300405004)(21056001)(110136001)(15975445006)(19625215002)(561944003)(80022001)(66066001)(99396002)(99286002)(95666004)(106116001)(105586002)(106356001)(64706001)(33646002)(85852003)(4396001)(101416001)(18717965001)(20776003)(74502001)(87936001)(86362001)(31966008)(74662001)(15202345003)(2656002)(79102001)(77982001)(76482001)(16236675004)(46102001)(92566001)(74316001)(76176999)(19580395003)(107046002)(81342001)(54356999)(50986999)(83072002)(19580405001)(83322001)(81542001)(76576001)(24736002)(108616003); DIR:OUT; SFP:; SCL:1; SRVR:BY2PR05MB222; H:BY2PR05MB728.namprd05.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; LANG:en; Content-Type: multipart/alternative; boundary="_000_6b0fa5df117f43d69c31efab88161e70BY2PR05MB728namprd05pro_" MIME-Version: 1.0 X-Microsoft-Antispam: BCL:0;PCL:0;RULEID:; X-OriginatorOrg: juniper.net Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/EN6T2w9u1MGdHbJNTW5O6jUPmAI Cc: Yakov Rekhter , "erosen@cisco.com" , "pwe3@ietf.org" , "stbryant@cisco.com" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Aug 2014 20:15:56 -0000 --_000_6b0fa5df117f43d69c31efab88161e70BY2PR05MB728namprd05pro_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi Sasha, By "avoiding ECMP", I was strictly talking about the context ID FEC from th= e PLR's perspective, rather than avoiding ECMP broadly for other FECs on th= e PLR or any FECs on other routers. Specifically, if the PLR has a set of ECMP to the context ID FEC, all the E= CMP paths must traverse the primary PE only. This will ensure that all PWs = be carried to the primary PE. And this can be achieved by using the proxy n= ode approach suggested in this draft. The path(s) traversing the protector = to reach the context ID FEC can be used as bypass LSP(s). We can improve the text on this to make it clearer. Thanks, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Sunday, August 10, 2014 1:15 AM To: Yimin Shen Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org; erosen@cisco.com Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin hi, Lots of thanks for a prompt response. I must admit that I have missed the proposal in Section 4.2.2 to treat "con= text ID" IP address as a proxy node that is adjacent to both the Primary PE= and the Protector NEs. (It also has not been mentioned in Eri=E7's clarifi= cation email). I think that this approach would require certain manipulatio= n of both the IGP and LDP, but this requires more thought on my side. One o= f the issues that come to mind, is the need for LDP in both these nodes to = allocate a locally terminated label for the corresponding FEC while not adv= ertising context IP address as one of the IP addresses it owns. I think that your statement about "avoiding ECMP scenario" is problematic,= because, from my point of view, ECMP is one of the important characterist= ics of LDP-based MPLS PSN. And in any case the draft does not say anything= about that (or, at least neither ECMP nor "equal cost multipath" is mentio= ned in the text). If you see ECMP avoidance as a pre-condition for using t= he mechanism defined in draft with LDP LSPs, this should be quite explicit = and detailed IMO. I still fail to understand what constitutes an egress AC failure in the dr= aft and how it is different from the ingress AC failure. Unfortunately the = examples you've provided in your response do not help much: e.g., a fiber c= ut close to would be most probably recognized by the PE as a bi-directiona= l AC failure, while a fiber cut that is close to CE but remote from PE woul= d most probably not detected by the PE. (Of course if PE and CE are termina= ting the same fiber span, the fiber cut would be recognized by both). My guess (FWIW) that the draft actually discusses protection of traffic in = the PE --> CE direction triggered by a failure detected by the PE while lea= ving protection of traffic in the CE --> PE direction some other mechanism = that would be triggered by CE detecting the failure. This mechanism presuma= bly would be different from the PW redundancy mechanism because it would re= quire that an active-active scheme for Primary and Backup PWs. If this guess is correct (or close to reality), I would expect some explici= t text explaining it. In particular, this text should explain that "redunda= nt PWs" mentioned in the draft are not related to "PW redundancy". The curr= ent text makes a mix of these two notions, and this definitely does not hel= p the reader (or at least this reader :-). Hopefully these comments would be useful. Regards, Sasha ________________________________ From: Yimin Shen > Sent: Friday, August 8, 2014 5:34 PM To: Alexander Vainshtein Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Hi Sasha, Thanks again for the comments and questions! Regarding AC failure detection on PE, in the typical cases, if there is a f= iber cut, or a hardware failure affecting the port on the PE, or a hardware= failure affecting the port on the CE (assume the CE is directed attached t= o the PE), the PE should be able to directly and independently detect the f= ailure at physical or link layer. Regarding LDP, I agree with you that LDP by its nature is not as straight-f= orward as RSVP-TE in this scenario. Since a context ID are advertised by bo= th primary PE and protector, we should avoid creating an ECMP scenario. Tha= t's why section 4.2.2 provides a suggestion to advertise the context ID as = a proxy node (i.e. a virtual node attached to the two routers), and make it= s reachability via the primary PE more favorable than that via the protecto= r. MRT and other mechanisms could be used to help LDP improve protection co= verage. Also, nothing would prevent LDP from using RSVP bypass tunnels. As = you can see, there may be many different approaches, and there may new appr= oaches coming in the future. In this PWE3 document, we just want to stay at= PW level and be generic with PSN level. In any case, this mechanism is ach= ievable with LDP, and LDP as a tunnel signaling protocol really doesn't nee= d any new extension. Hope this clarifies it. Thanks, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Thursday, August 07, 2014 10:37 AM To: Yimin Shen Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin hi, Please see some comments inline below. Regards, Sasha Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 From: Yimin Shen [mailto:yshen@juniper.net] Sent: Thursday, August 07, 2014 5:34 AM To: Alexander Vainshtein Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Hi Sasha, Thanks again for the comments. Please see the following 4 items in response= to the 4 comments in your email. 1. Today when an egress PE detects an egress AC failure [[Sasha]] Could you please explain how the egress PE detects an egress AC f= ailure? I have always thought that egress AC failure has to be detected fir= st by corresponding CE which would then notify the egress PE using some sui= table "Remote Failure"/BDI/RDI mechanism. Specific examples would really he= lp. it will simply have to drop packets, until the remote PE stops sending traf= fic over the PW. This draft is intended to improve this behavior. By no mea= ns the local repair on egress PE will prevent the AC failure from being con= verted to and handled as a bi-directional AC failure. Note that bi-directio= nal AC failure handling must involve the CE, but the CE may detect the fail= ure at a slower pace, or it may not be able to detect it in a timely manner= , depending on the actual situation. [[Sasha]] Please see above. IMHO and FWIW if the CE cannot detect its "Rece= ive Failure", the PE would not never know about the "Transmit Failure. E.g. there may be other L2 devices between the PE and CE; the AC may itself= be a logical wire across another network; the type and location of the fai= lure may make it easier for the PE to detect it; and so on. In any case, be= fore the AC failure is handled more gracefully (or bi-directionally), the e= gress traffic has been locally repaired by the egress PE. 2. As mentioned before, this draft doesn't have a dependency on a specific = PSN tunnel technology. [[Sasha]] I have provided my comments on this in my response to Eric. From = my point of view some LDP-specific issues still require clarification, but = my bottom line is that operating the draft over an MPLS network that uses L= DP in DU mode for setting up tunnel LSPs may sometimes be possible but woul= d always be tricky. This does not match my understanding of "no dependency = on a specific PSN tunnel technology", but this may be a matter of taste. 3. The draft already talks about scalability when describing the centralize= d protector model vs the co-located protector model. We can improve the tex= t to explicitly mention that the requirement for setting up LSPs on a per <= primary PE, protector> basis rather than per primary PE basis can potential= ly increase the number of LSPs in a network. This is a tradeoff. As mention= ed before, this does not necessarily cause a scaling issue. [[Sasha]] Yes, I have found the corresponding text in the draft. From my p= oint of view this issue is resolved. 4. We will leave this to PWE3 WG to decide whether this is really necessary= .[[Sasha]] I tend to agree with Eric that usage of context-specific labels = in the draft does not mean update to 4447. From my point of view this issue= is resolved as well. Thanks, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Wednesday, August 06, 2014 5:11 AM To: Yimin Shen Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin and all, I would like to summarize the issues that I have with this draft. Digging f= or them in a very long email thread with multiple levels of in-lining is no= t very effective IMO. 1. Protection against unidirectional AC failures looks problematic to= me: a. The draft explicitly differentiates between "ingress" and "egress"= AC failures b. The outcome can be using a different AC for each direction of CE-to= -CE traffic with potentially destructive effect that cannot be mitigated at= the PWE3 level c. IMHO and FWIW a viable alternative could be to employ native AC me= chanisms converting unidirectional AC failures into bi-directional ones. Th= is would eliminate the differentiation between ingress and egress AC failur= es and facilitate using the PW redundancy mechanism for bi-directional prot= ection 2. Dependency upon specific mechanisms for setting up PSN tunnels see= ms to break some common PWE3 architectural principles: a. I am pretty well sure that the mechanisms specified in the draft c= annot be used with the PSN tunnel LSPs set up by LDP in DU mode b. If the authors, refer to LDP in the DoD mode augmented with some CR= -LDP-like mechanisms (e.g., for exclusion of specific links when setting up= the bypass LSP between the PLR and the Protector), this should, at the ver= y least, be discussed with and approved by the MPLS WG 3. Scalability of the proposed solution should be, at least, explicit= ly presented. In particular, the draft assumes that a dedicated LSP would b= e used between PE1 PE2 for each combination of PE2 with a specific Protect= or LSR. 4. The Protector LSR could be either a kind of S-PE (if different fro= m the secondary T-PE) or the secondary T-PE itself. In both cases it should= be capable of handling labels from specific context spaces as if they were= PW labels. This requires (at the very least) an update of RFC 4447 and sho= uld be explicitly presented and discussed. Hopefully this summary will be useful. Regards, Sasha Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 --_000_6b0fa5df117f43d69c31efab88161e70BY2PR05MB728namprd05pro_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Hi Sasha,

 

By “avoiding ECM= P”, I was strictly talking about the context ID FEC from the PLR̵= 7;s perspective, rather than avoiding ECMP broadly for other FECs on the PL= R or any FECs on other routers.

 

Specifically, if the P= LR has a set of ECMP to the context ID FEC, all the ECMP paths must travers= e the primary PE only. This will ensure that all PWs be carried to the prim= ary PE. And this can be achieved by using the proxy node approach suggested in this draft. The path(s) travers= ing the protector to reach the context ID FEC can be used as bypass LSP(s).=

 

We can improve the tex= t on this to make it clearer.

 

Thanks,

 

/Yimin

 

 

From: Alexande= r Vainshtein [mailto:Alexander.Vainshtein@ecitele.com]
Sent: Sunday, August 10, 2014 1:15 AM
To: Yimin Shen
Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org; erosen@cisco.c= om
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Yimin hi,

Lots of thanks for a prompt response.

 

I must admit that I have missed the proposal= in Section 4.2.2 to treat "context ID" IP address as a proxy nod= e that is adjacent to both the Primary PE and the Protector NEs. (It also h= as not been mentioned in Eri=E7's clarification email). I think that this approach would require certain manipulation of b= oth the IGP and LDP, but this requires more thought on my side. One of the&= nbsp;issues that come to mind, is the need for LDP in both these = nodes to allocate a locally terminated label for the corresponding FEC while not advertising context IP address as one of t= he IP addresses it owns.

 

I think that your statement about "avoi= ding ECMP  scenario" is problematic, because, from my p= oint of view,  ECMP is one of the important characteristics of LD= P-based MPLS PSN.  And in any case the draft does not say anythin= g about that (or, at least neither ECMP nor "equal cost multipath&q= uot; is mentioned in the text).  If you see ECMP avoidance as a p= re-condition for using the mechanism defined in draft with LDP LSPs, this s= hould be quite explicit and detailed IMO.

 

I still fail to understand  what consti= tutes an egress AC failure in the draft and how it is different from the in= gress AC failure. Unfortunately the examples you've provided in your respon= se do not help much: e.g., a fiber cut close to  would be most probably recognized by the PE as a bi-directi= onal AC failure, while a fiber cut that is close to CE but remote from PE w= ould most probably not detected by the PE. (Of course if PE and CE are term= inating the same fiber span, the fiber cut would be recognized by both).

 

My guess (FWIW) that the draft actually disc= usses protection of traffic in the PE --> CE direction triggered by a fa= ilure detected by the PE while leaving protection of traffic in the CE --&g= t; PE direction some other mechanism that would be triggered by CE detecting the failure. This mechanism presumably = would be different from the PW redundancy mechanism because it would requir= e that an active-active scheme for Primary and Backup  PWs.=

 

If this guess is correct (or close to realit= y), I would expect some explicit text explaining it. In particular, this te= xt should explain that "redundant PWs" mentioned in the draft are= not related to "PW redundancy". The current text makes a mix of these two notions, and this definitely does not help t= he reader (or at least this reader :-).

 

Hopefully these comments would be useful.

 

Regards,

Sasha

From: Yimin Shen <yshen@juniper.net>
Sent: Friday, August 8, 2014 5:34 PM
To: Alexander Vainshtein
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Hi Sasha,

 

Thanks again for the c= omments and questions!

 

Regarding AC failure d= etection on PE, in the typical cases, if there is a fiber cut, or a hardwar= e failure affecting the port on the PE, or a hardware failure affecting the= port on the CE (assume the CE is directed attached to the PE), the PE should be able to directly and independently d= etect the failure at physical or link layer.

 

Regarding LDP, I agree= with you that LDP by its nature is not as straight-forward as RSVP-TE in t= his scenario. Since a context ID are advertised by both primary PE and prot= ector, we should avoid creating an ECMP scenario. That’s why section 4.2.2 provides a suggestion to advertis= e the context ID as a proxy node (i.e. a virtual node attached to the two r= outers), and make its reachability via the primary PE more favorable than t= hat via the protector. MRT and other mechanisms could be used to help LDP improve protection coverage. Also, nothing would= prevent LDP from using RSVP bypass tunnels. As you can see, there may be m= any different approaches, and there may new approaches coming in the future= . In this PWE3 document, we just want to stay at PW level and be generic with PSN level. In any case, this = mechanism is achievable with LDP, and LDP as a tunnel signaling protocol re= ally doesn’t need any new extension.  

 

Hope this clarifies it= .

 

Thanks,

 

/Yimin

 

 

From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com]
Sent: Thursday, August 07, 2014 10:37 AM
To: Yimin Shen
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Yimin hi,

Please see some commen= ts inline below.

 

Regards,

   &nbs= p;   Sasha

Email: Alexander.Vainshtein@ecitele.com

Mobile: 054-9266302

 

From: Yimin Shen [mailto= :yshen@juniper.net]
Sent: Thursday, August 07, 2014 5:34 AM
To: Alexander Vainshtein
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Hi Sasha,

 

Thanks again for the c= omments. Please see the following 4 items in response to the 4 comments in = your email.

 

1. Today when an egres= s PE detects an egress AC failure=

 

[[Sasha]] Could = you please explain how the egress PE detects an egress AC failure? I have a= lways thought that egress AC failure has to be detected first by correspond= ing CE which would then notify the egress PE using some suitable “Remote Failure”/BDI/RDI mechanism. Spe= cific examples would really help.

 

it will simply have to= drop packets, until the remote PE stops sending traffic over the PW. This = draft is intended to improve this behavior. By no means the local repair on= egress PE will prevent the AC failure from being converted to and handled as a bi-directional AC failure. Note t= hat bi-directional AC failure handling must involve the CE, but the CE may = detect the failure at a slower pace, or it may not be able to detect it in = a timely manner, depending on the actual situation. <= /p>

[[Sasha]] Please= see above. IMHO and FWIW if the CE cannot detect its “Receive Failur= e”, the PE would not never know about the “Transmit Failure.

 

E.g. there may be othe= r L2 devices between the PE and CE; the AC may itself be a logical wire acr= oss another network; the type and location of the failure may make it easie= r for the PE to detect it; and so on. In any case, before the AC failure is handled more gracefully (or bi-direc= tionally), the egress traffic has been locally repaired by the egress PE.

 

2. As mentioned before= , this draft doesn’t have a dependency on a specific PSN tunnel techn= ology.

[[Sasha]] I have= provided my comments on this in my response to Eric. From my point of view= some LDP-specific issues still require clarification, but my bottom line i= s that operating the draft over an MPLS network = that uses LDP in DU mode for setting up tunnel LSPs may sometimes be possib= le but would always be tricky. This does not match my understanding = of “no dependency on a specific PSN tunnel technology”, but this may be a matter of taste.

 

3. The draft already t= alks about scalability when describing the centralized protector model vs t= he co-located protector model. We can improve the text to explicitly mentio= n that the requirement for setting up LSPs on a per <primary PE, protector> basis rather than per primary = PE basis can potentially increase the number of LSPs in a network. This is = a tradeoff. As mentioned before, this does not necessarily cause a scaling = issue.

[[Sasha]] Yes, I= have found the corresponding text in the draft.  From my point of vie= w this issue is resolved.=

 

4. We will leave this = to PWE3 WG to decide whether this is really necessary.[[Sasha]] I tend to agree with Eric that usage of con= text-specific labels in the draft does not mean update to 4447. From my point of view this issue is resolved as w= ell.

 

Thanks,

 

/Yimin

 

 

From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com]
Sent: Wednesday, August 06, 2014 5:11 AM
To: Yimin Shen
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Yimin and all,=

I would like to sum= marize the issues that I have with this draft. Digging for them in a very l= ong email thread with multiple levels of in-lining is not very effective IM= O.

 

1. &nbs= p;     Protection against unidirectional A= C failures looks problematic to me= :

a. &nb= sp;     The draft explicitly differentiates be= tween “ingress” and “egress” AC failures=

b. &nb= sp;    The outcome can be using a different A= C for each direction of CE-to-CE traffic with potentially destructive effec= t that cannot be mitigated at the PWE3 level

c. &nb= sp;     IMHO and FWIW a viable alternative cou= ld be to employ native AC mechanisms converting unidirectional AC failures = into bi-directional ones. This would eliminate the differentiation between = ingress and egress AC failures and facilitate using the PW redundancy mechanism for bi-directional protection=

2. &nbs= p;     Dependency upon specific mechanisms= for setting up PSN tunnels seems = to break some common PWE3 architectural principles:

a. &nb= sp;     I am pretty well sure that the mechani= sms specified in the draft cannot be used with the PSN tunnel LSPs set up b= y LDP in DU mode

b. &nb= sp;    If the authors, refer to LDP in the DoD mode augmented with some CR-LDP-like mechanisms (e.g., for exclusi= on of specific links when setting up the bypass LSP between the PLR and the= Protector), this should, at the very least, be discussed with and approved by the MPLS WG

3. &nbs= p;     Scalability of the proposed solution should be, at least, explicitl= y presented. In particular, the draft assumes that a dedicated LSP would be= used between PE1  PE2 for each combination of PE2 with a specific Protector LSR.

4. &nbs= p;     The Protector LSR could be either a ki= nd of S-PE (if different from the secondary T-PE) or the secondary T-PE its= elf. In both cases it should be capable of handling labels from specific co= ntext spaces as if they were PW labels. This requires (at the very least) an update of RFC 4447 and should be expl= icitly presented and discussed.

Hopefully this summ= ary will be useful.

 

 

Regards,=

   &= nbsp;   Sasha

Email: Alexander.Vainshtein@ecitele.com

Mobile: 054-9266302=

--_000_6b0fa5df117f43d69c31efab88161e70BY2PR05MB728namprd05pro_-- From nobody Mon Aug 11 13:34:19 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0A7201A005B for ; Mon, 11 Aug 2014 13:34:18 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -15.168 X-Spam-Level: X-Spam-Status: No, score=-15.168 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-0.668, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id I1c2cnB3gAug for ; Mon, 11 Aug 2014 13:34:14 -0700 (PDT) Received: from alln-iport-1.cisco.com (alln-iport-1.cisco.com [173.37.142.88]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CB66F1A0010 for ; Mon, 11 Aug 2014 13:34:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=43949; q=dns/txt; s=iport; t=1407789253; x=1408998853; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=lnIJUVxQr3biwwaLJjbc8jIDZgPbL1Z+CGadLZUaFBA=; b=cbZRZNfyoeVtZQnjQSXES4RTrD1bo+Va7cINg0GEIAXgQtVzSMvvVcqi JNdKq8QyTSHjWriKFTZhTu+lz6qmvTddNvKHlL7w1Q/+Ssb1uRhsvzb+Y 3U40/2kT/HNHNXrYB5SC/6hoJuBbV4JE6Ee4sVCqnoK/8dr+JB9Z3FteZ k=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgkFAKQn6VOtJA2F/2dsb2JhbABagkdGsGKlGAGBGxZ3hAMBAQEEGgMKBjoSEAIBCA4DBAEBIQEGBzIUCQgCBA4FG4gnwyYXjmEJEQEjGxIGAYMvgR0FjUiBPoITixaUe4NcbIEO X-IronPort-AV: E=Sophos; i="5.01,844,1400025600"; d="scan'208,217"; a="68284614" Received: from alln-core-11.cisco.com ([173.36.13.133]) by alln-iport-1.cisco.com with ESMTP; 11 Aug 2014 20:34:10 +0000 Received: from xhc-aln-x08.cisco.com (xhc-aln-x08.cisco.com [173.36.12.82]) by alln-core-11.cisco.com (8.14.5/8.14.5) with ESMTP id s7BKYAE4007330 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Mon, 11 Aug 2014 20:34:10 GMT Received: from xmb-rcd-x04.cisco.com ([169.254.8.74]) by xhc-aln-x08.cisco.com ([173.36.12.82]) with mapi id 14.03.0195.001; Mon, 11 Aug 2014 15:34:10 -0500 From: "Stewart Bryant (stbryant)" To: Yimin Shen Thread-Topic: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 Thread-Index: AQHPqymWveYGLBru1kCsLuzIRj9tdJu3CWCAgAA8XgCAAPc/IIABBtiAgAg4EiaAAOtlgIAAs2DLgAA1qVCAASlTgIAAxW1QgAGWRICAAnyxroAC7XwA//+xXCQ= Date: Mon, 11 Aug 2014 20:34:09 +0000 Message-ID: References: <201407251524.s6PFOPn92012@magenta.juniper.net> <53D79504.4050904@cisco.com> <04f15bedf2be4c7480d3e9ca01bdfd7f@BY2PR05MB728.namprd05.prod.outlook.com> <2597291f7b074922b690e4b06999cf1a@AM3PR03MB612.eurprd03.prod.outlook.com>, <30e13900814f41a681151971cbe9ebef@BY2PR05MB728.namprd05.prod.outlook.com> <1407215578702.64387@ecitele.com>, <2280b8e0d5f94a60badab91a2237181b@BY2PR05MB728.namprd05.prod.outlook.com> <1407304503657.72312@ecitele.com> <046050cb2d254a139b4650b2d7f93075@BY2PR05MB728.namprd05.prod.outlook.com> <09ab595ab24a46a0a4e87ebeb2cd185a@AM3PR03MB612.eurprd03.prod.outlook.com>, <2afb5c963e054aef929f38dcb3defa07@BY2PR05MB728.namprd05.prod.outlook.com> <1407647722694.18724@ecitele.com>, <6b0fa5df117f43d69c31efab88161e70@BY2PR05MB728.namprd05.prod.outlook.com> In-Reply-To: <6b0fa5df117f43d69c31efab88161e70@BY2PR05MB728.namprd05.prod.outlook.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: multipart/alternative; boundary="_000_E0A00E4FF9A34D1A9F9282BDFF493B89ciscocom_" MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/5NNJTzAECym8fXrwx0eXyC8Wv7I Cc: Yakov Rekhter , "Eric Rosen \(erosen\)" , "pwe3@ietf.org" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Aug 2014 20:34:18 -0000 --_000_E0A00E4FF9A34D1A9F9282BDFF493B89ciscocom_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Yimin How does that address the case I described? ECMPs may form further out than the PLR, and not all May route through the = PLR. Stewart Sent from my iPad On 11 Aug 2014, at 22:16, "Yimin Shen" > wrote: Hi Sasha, By =93avoiding ECMP=94, I was strictly talking about the context ID FEC fro= m the PLR=92s perspective, rather than avoiding ECMP broadly for other FECs= on the PLR or any FECs on other routers. Specifically, if the PLR has a set of ECMP to the context ID FEC, all the E= CMP paths must traverse the primary PE only. This will ensure that all PWs = be carried to the primary PE. And this can be achieved by using the proxy n= ode approach suggested in this draft. The path(s) traversing the protector = to reach the context ID FEC can be used as bypass LSP(s). We can improve the text on this to make it clearer Thanksf, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Sunday, August 10, 2014 1:15 AM To: Yimin Shen Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org; erosen@cisco.com Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin hi, Lots of thanks for a prompt response. I must admit that I have missed the proposal in Section 4.2.2 to treat "con= text ID" IP address as a proxy node that is adjacent to both the Primary PE= and the Protector NEs. (It also has not been mentioned in Eri=E7's clarifi= cation email). I think that this approach would require certain manipulatio= n of both the IGP and LDP, but this requires more thought on my side. One o= f the issues that come to mind, is the need for LDP in both these nodes to = allocate a locally terminated label for the corresponding FEC while not adv= ertising context IP address as one of the IP addresses it owns. I think that your statement about "avoiding ECMP scenario" is problematic,= because, from my point of view, ECMP is one of the important characterist= ics of LDP-based MPLS PSN. And in any case the draft does not say anything= about that (or, at least neither ECMP nor "equal cost multipath" is mentio= ned in the text). If you see ECMP avoidance as a pre-condition for using t= he mechanism defined in draft with LDP LSPs, this should be quite explicit = and detailed IMO. I still fail to understand what constitutes an egress AC failure in the dr= aft and how it is different from the ingress AC failure. Unfortunately the = examples you've provided in your response do not help much: e.g., a fiber c= ut close to would be most probably recognized by the PE as a bi-directiona= l AC failure, while a fiber cut that is close to CE but remote from PE woul= d most probably not detected by the PE. (Of course if PE and CE are termina= ting the same fiber span, the fiber cut would be recognized by both). My guess (FWIW) that the draft actually discusses protection of traffic in = the PE --> CE direction triggered by a failure detected by the PE while lea= ving protection of traffic in the CE --> PE direction some other mechanism = that would be triggered by CE detecting the failure. This mechanism presuma= bly would be different from the PW redundancy mechanism because it would re= quire that an active-active scheme for Primary and Backup PWs. If this guess is correct (or close to reality), I would expect some explici= t text explaining it. In particular, this text should explain that "redunda= nt PWs" mentioned in the draft are not related to "PW redundancy". The curr= ent text makes a mix of these two notions, and this definitely does not hel= p the reader (or at least this reader :-). Hopefully these comments would be useful. Regards, Sasha ________________________________ From: Yimin Shen > Sent: Friday, August 8, 2014 5:34 PM To: Alexander Vainshtein Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Hi Sasha, Thanks again for the comments and questions! Regarding AC failure detection on PE, in the typical cases, if there is a f= iber cut, or a hardware failure affecting the port on the PE, or a hardware= failure affecting the port on the CE (assume the CE is directed attached t= o the PE), the PE should be able to directly and independently detect the f= ailure at physical or link layer. Regarding LDP, I agree with you that LDP by its nature is not as straight-f= orward as RSVP-TE in this scenario. Since a context ID are advertised by bo= th primary PE and protector, we should avoid creating an ECMP scenario. Tha= t=92s why section 4.2.2 provides a suggestion to advertise the context ID a= s a proxy node (i.e. a virtual node attached to the two routers), and make = its reachability via the primary PE more favorable than that via the protec= tor. MRT and other mechanisms could be used to help LDP improve protection = coverage. Also, nothing would prevent LDP from using RSVP bypass tunnels. A= s you can see, there may be many different approaches, and there may new ap= proaches coming in the future. In this PWE3 document, we just want to stay = at PW level and be generic with PSN level. In any case, this mechanism is a= chievable with LDP, and LDP as a tunnel signaling protocol really doesn=92t= need any new extension. Hope this clarifies it. Thanks, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Thursday, August 07, 2014 10:37 AM To: Yimin Shen Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin hi, Please see some comments inline below. Regards, Sasha Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 From: Yimin Shen [mailto:yshen@juniper.net] Sent: Thursday, August 07, 2014 5:34 AM To: Alexander Vainshtein Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Hi Sasha, Thanks again for the comments. Please see the following 4 items in response= to the 4 comments in your email. 1. Today when an egress PE detects an egress AC failure [[Sasha]] Could you please explain how the egress PE detects an egress AC f= ailure? I have always thought that egress AC failure has to be detected fir= st by corresponding CE which would then notify the egress PE using some sui= table =93Remote Failure=94/BDI/RDI mechanism. Specific examples would reall= y help. it will simply have to drop packets, until the remote PE stops sending traf= fic over the PW. This draft is intended to improve this behavior. By no mea= ns the local repair on egress PE will prevent the AC failure from being con= verted to and handled as a bi-directional AC failure. Note that bi-directio= nal AC failure handling must involve the CE, but the CE may detect the fail= ure at a slower pace, or it may not be able to detect it in a timely manner= , depending on the actual situation. [[Sasha]] Please see above. IMHO and FWIW if the CE cannot detect its =93Re= ceive Failure=94, the PE would not never know about the =93Transmit Failure= . E.g. there may be other L2 devices between the PE and CE; the AC may itself= be a logical wire across another network; the type and location of the fai= lure may make it easier for the PE to detect it; and so on. In any case, be= fore the AC failure is handled more gracefully (or bi-directionally), the e= gress traffic has been locally repaired by the egress PE. 2. As mentioned before, this draft doesn=92t have a dependency on a specifi= c PSN tunnel technology. [[Sasha]] I have provided my comments on this in my response to Eric. From = my point of view some LDP-specific issues still require clarification, but = my bottom line is that operating the draft over an MPLS network that uses L= DP in DU mode for setting up tunnel LSPs may sometimes be possible but woul= d always be tricky. This does not match my understanding of =93no dependenc= y on a specific PSN tunnel technology=94, but this may be a matter of taste= . 3. The draft already talks about scalability when describing the centralize= d protector model vs the co-located protector model. We can improve the tex= t to explicitly mention that the requirement for setting up LSPs on a per <= primary PE, protector> basis rather than per primary PE basis can potential= ly increase the number of LSPs in a network. This is a tradeoff. As mention= ed before, this does not necessarily cause a scaling issue. [[Sasha]] Yes, I have found the corresponding text in the draft. From my p= oint of view this issue is resolved. 4. We will leave this to PWE3 WG to decide whether this is really necessary= .[[Sasha]] I tend to agree with Eric that usage of context-specific labels = in the draft does not mean update to 4447. From my point of view this issue= is resolved as well. Thanks, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Wednesday, August 06, 2014 5:11 AM To: Yimin Shen Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin and all, I would like to summarize the issues that I have with this draft. Digging f= or them in a very long email thread with multiple levels of in-lining is no= t very effective IMO. 1. Protection against unidirectional AC failures looks problematic to= me: a. The draft explicitly differentiates between =93ingress=94 and =93e= gress=94 AC failures b. The outcome can be using a different AC for each direction of CE-to= -CE traffic with potentially destructive effect that cannot be mitigated at= the PWE3 level c. IMHO and FWIW a viable alternative could be to employ native AC me= chanisms converting unidirectional AC failures into bi-directional ones. Th= is would eliminate the differentiation between ingress and egress AC failur= es and facilitate using the PW redundancy mechanism for bi-directional prot= ection 2. Dependency upon specific mechanisms for setting up PSN tunnels see= ms to break some common PWE3 architectural principles: a. I am pretty well sure that the mechanisms specified in the draft c= annot be used with the PSN tunnel LSPs set up by LDP in DU mode b. If the authors, refer to LDP in the DoD mode augmented with some CR= -LDP-like mechanisms (e.g., for exclusion of specific links when setting up= the bypass LSP between the PLR and the Protector), this should, at the ver= y least, be discussed with and approved by the MPLS WG 3. Scalability of the proposed solution should be, at least, explicit= ly presented. In particular, the draft assumes that a dedicated LSP would b= e used between PE1 PE2 for each combination of PE2 with a specific Protect= or LSR. 4. The Protector LSR could be either a kind of S-PE (if different fro= m the secondary T-PE) or the secondary T-PE itself. In both cases it should= be capable of handling labels from specific context spaces as if they were= PW labels. This requires (at the very least) an update of RFC 4447 and sho= uld be explicitly presented and discussed. Hopefully this summary will be useful. Regards, Sasha Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 --_000_E0A00E4FF9A34D1A9F9282BDFF493B89ciscocom_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable
Yimin

How does that address the case I described? 

ECMPs may form further out than the PLR, and not all May route through= the PLR.

Stewart

Sent from my iPad

On 11 Aug 2014, at 22:16, "Yimin Shen" <yshen@juniper.net> wrote:

Hi Sasha,

 

By =93avoiding ECMP=94= , I was strictly talking about the context ID FEC from the PLR=92s perspect= ive, rather than avoiding ECMP broadly for other FECs on the PLR or any FEC= s on other routers.

 

Specifically, if the P= LR has a set of ECMP to the context ID FEC, all the ECMP paths must travers= e the primary PE only. This will ensure that all PWs be carried to the prim= ary PE. And this can be achieved by using the proxy node approach suggested in this draft. The path(s) travers= ing the protector to reach the context ID FEC can be used as bypass LSP(s).=

 

We can improve the tex= t on this to make it clearer

Thanksf,

 

/Yimin

 

 

From: Alexande= r Vainshtein [mailto:Al= exander.Vainshtein@ecitele.com]
Sent: Sunday, August 10, 2014 1:15 AM
To: Yimin Shen
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org; erosen@cisco.com
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Yimin hi,

Lots of thanks for a prompt response.

 

I must admit that I have missed the proposal= in Section 4.2.2 to treat "context ID" IP address as a proxy nod= e that is adjacent to both the Primary PE and the Protector NEs. (It also h= as not been mentioned in Eri=E7's clarification email). I think that this approach would require certain manipulation of b= oth the IGP and LDP, but this requires more thought on my side. One of the&= nbsp;issues that come to mind, is the need for LDP in both these = nodes to allocate a locally terminated label for the corresponding FEC while not advertising context IP address as one of t= he IP addresses it owns.

 

I think that your statement about "avoi= ding ECMP  scenario" is problematic, because, from my p= oint of view,  ECMP is one of the important characteristics of LD= P-based MPLS PSN.  And in any case the draft does not say anythin= g about that (or, at least neither ECMP nor "equal cost multipath&q= uot; is mentioned in the text).  If you see ECMP avoidance as a p= re-condition for using the mechanism defined in draft with LDP LSPs, this s= hould be quite explicit and detailed IMO.

 

I still fail to understand  what consti= tutes an egress AC failure in the draft and how it is different from the in= gress AC failure. Unfortunately the examples you've provided in your respon= se do not help much: e.g., a fiber cut close to  would be most probably recognized by the PE as a bi-directi= onal AC failure, while a fiber cut that is close to CE but remote from PE w= ould most probably not detected by the PE. (Of course if PE and CE are term= inating the same fiber span, the fiber cut would be recognized by both).

 

My guess (FWIW) that the draft actually disc= usses protection of traffic in the PE --> CE direction triggered by a fa= ilure detected by the PE while leaving protection of traffic in the CE --&g= t; PE direction some other mechanism that would be triggered by CE detecting the failure. This mechanism presumably = would be different from the PW redundancy mechanism because it would requir= e that an active-active scheme for Primary and Backup  PWs.=

 

If this guess is correct (or close to realit= y), I would expect some explicit text explaining it. In particular, this te= xt should explain that "redundant PWs" mentioned in the draft are= not related to "PW redundancy". The current text makes a mix of these two notions, and this definitely does not help t= he reader (or at least this reader :-).

 

Hopefully these comments would be useful.

 

Regards,

Sasha

From: Yimin Shen <yshen@juniper.net>
Sent: Friday, August 8, 2014 5:34 PM
To: Alexander Vainshtein
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Hi Sasha,

 

Thanks again for the c= omments and questions!

 

Regarding AC failure d= etection on PE, in the typical cases, if there is a fiber cut, or a hardwar= e failure affecting the port on the PE, or a hardware failure affecting the= port on the CE (assume the CE is directed attached to the PE), the PE should be able to directly and independently d= etect the failure at physical or link layer.

 

Regarding LDP, I agree= with you that LDP by its nature is not as straight-forward as RSVP-TE in t= his scenario. Since a context ID are advertised by both primary PE and prot= ector, we should avoid creating an ECMP scenario. That=92s why section 4.2.2 provides a suggestion to advertise th= e context ID as a proxy node (i.e. a virtual node attached to the two route= rs), and make its reachability via the primary PE more favorable than that = via the protector. MRT and other mechanisms could be used to help LDP improve protection coverage. Also, nothing would= prevent LDP from using RSVP bypass tunnels. As you can see, there may be m= any different approaches, and there may new approaches coming in the future= . In this PWE3 document, we just want to stay at PW level and be generic with PSN level. In any case, this = mechanism is achievable with LDP, and LDP as a tunnel signaling protocol re= ally doesn=92t need any new extension.  

 

Hope this clarifies it= .

 

Thanks,

 

/Yimin

 

 

From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com]
Sent: Thursday, August 07, 2014 10:37 AM
To: Yimin Shen
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Yimin hi,

Please see some commen= ts inline below.

 

Regards,

   &nbs= p;   Sasha

Email: Alexander.Vainshtein@ecitele.com

Mobile: 054-9266302

 

From: Yimin Shen [mailto= :yshen@juniper.net]
Sent: Thursday, August 07, 2014 5:34 AM
To: Alexander Vainshtein
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Hi Sasha,

 

Thanks again for the c= omments. Please see the following 4 items in response to the 4 comments in = your email.

 

1. Today when an egres= s PE detects an egress AC failure=

 

[[Sasha]] Could = you please explain how the egress PE detects an egress AC failure? I have a= lways thought that egress AC failure has to be detected first by correspond= ing CE which would then notify the egress PE using some suitable =93Remote Failure=94/BDI/RDI mechanism. Specific ex= amples would really help.=

 

it will simply have to= drop packets, until the remote PE stops sending traffic over the PW. This = draft is intended to improve this behavior. By no means the local repair on= egress PE will prevent the AC failure from being converted to and handled as a bi-directional AC failure. Note t= hat bi-directional AC failure handling must involve the CE, but the CE may = detect the failure at a slower pace, or it may not be able to detect it in = a timely manner, depending on the actual situation. <= /p>

[[Sasha]] Please= see above. IMHO and FWIW if the CE cannot detect its =93Receive Failure=94= , the PE would not never know about the =93Transmit Failure.

 

E.g. there may be othe= r L2 devices between the PE and CE; the AC may itself be a logical wire acr= oss another network; the type and location of the failure may make it easie= r for the PE to detect it; and so on. In any case, before the AC failure is handled more gracefully (or bi-direc= tionally), the egress traffic has been locally repaired by the egress PE.

 

2. As mentioned before= , this draft doesn=92t have a dependency on a specific PSN tunnel technolog= y.

[[Sasha]] I have= provided my comments on this in my response to Eric. From my point of view= some LDP-specific issues still require clarification, but my bottom line i= s that operating the draft over an MPLS network = that uses LDP in DU mode for setting up tunnel LSPs may sometimes be possib= le but would always be tricky. This does not match my understanding = of =93no dependency on a specific PSN tunnel technology=94, but this may be a matter of taste.

 

3. The draft already t= alks about scalability when describing the centralized protector model vs t= he co-located protector model. We can improve the text to explicitly mentio= n that the requirement for setting up LSPs on a per <primary PE, protector> basis rather than per primary = PE basis can potentially increase the number of LSPs in a network. This is = a tradeoff. As mentioned before, this does not necessarily cause a scaling = issue.

[[Sasha]] Yes, I= have found the corresponding text in the draft.  From my point of vie= w this issue is resolved.=

 

4. We will leave this = to PWE3 WG to decide whether this is really necessary.[[Sasha]] I tend to agree with Eric that usage of con= text-specific labels in the draft does not mean update to 4447. From my point of view this issue is resolved as w= ell.

 

Thanks,

 

/Yimin

 

 

From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com]
Sent: Wednesday, August 06, 2014 5:11 AM
To: Yimin Shen
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Yimin and all,=

I would like to sum= marize the issues that I have with this draft. Digging for them in a very l= ong email thread with multiple levels of in-lining is not very effective IM= O.

 

1. &nbs= p;     Protection against unidirectional A= C failures looks problematic to me= :

a. &nb= sp;     The draft explicitly differentiates be= tween =93ingress=94 and =93egress=94 AC failures

b. &nb= sp;    The outcome can be using a different A= C for each direction of CE-to-CE traffic with potentially destructive effec= t that cannot be mitigated at the PWE3 level

c. &nb= sp;     IMHO and FWIW a viable alternative cou= ld be to employ native AC mechanisms converting unidirectional AC failures = into bi-directional ones. This would eliminate the differentiation between = ingress and egress AC failures and facilitate using the PW redundancy mechanism for bi-directional protection=

2. &nbs= p;     Dependency upon specific mechanisms= for setting up PSN tunnels seems = to break some common PWE3 architectural principles:

a. &nb= sp;     I am pretty well sure that the mechani= sms specified in the draft cannot be used with the PSN tunnel LSPs set up b= y LDP in DU mode

b. &nb= sp;    If the authors, refer to LDP in the DoD mode augmented with some CR-LDP-like mechanisms (e.g., for exclusi= on of specific links when setting up the bypass LSP between the PLR and the= Protector), this should, at the very least, be discussed with and approved by the MPLS WG

3. &nbs= p;     Scalability of the proposed solution should be, at least, explicitl= y presented. In particular, the draft assumes that a dedicated LSP would be= used between PE1  PE2 for each combination of PE2 with a specific Protector LSR.

4. &nbs= p;     The Protector LSR could be either a ki= nd of S-PE (if different from the secondary T-PE) or the secondary T-PE its= elf. In both cases it should be capable of handling labels from specific co= ntext spaces as if they were PW labels. This requires (at the very least) an update of RFC 4447 and should be expl= icitly presented and discussed.

Hopefully this summ= ary will be useful.

 

 

Regards,=

   &= nbsp;   Sasha

Email: Alexander.Vainshtein@ecitele.com

Mobile: 054-9266302=

--_000_E0A00E4FF9A34D1A9F9282BDFF493B89ciscocom_-- From nobody Mon Aug 11 13:54:01 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B4A141A00E2 for ; Mon, 11 Aug 2014 13:53:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.601 X-Spam-Level: X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jRrDFen0G7By for ; Mon, 11 Aug 2014 13:53:55 -0700 (PDT) Received: from na01-by2-obe.outbound.protection.outlook.com (mail-by2lp0241.outbound.protection.outlook.com [207.46.163.241]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CCE291A00D8 for ; Mon, 11 Aug 2014 13:53:55 -0700 (PDT) Received: from BY2PR05MB728.namprd05.prod.outlook.com (10.141.223.25) by BY2PR05MB223.namprd05.prod.outlook.com (10.242.41.141) with Microsoft SMTP Server (TLS) id 15.0.1005.10; Mon, 11 Aug 2014 20:53:52 +0000 Received: from BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) by BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) with mapi id 15.00.1005.008; Mon, 11 Aug 2014 20:53:52 +0000 From: Yimin Shen To: Alexander Vainshtein , "Stewart Bryant (stbryant)" Thread-Topic: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 Thread-Index: AQHPqByPzRsIO8QlbkK+kbMBZ9MDf5u3AuAAgAAZ0oCAABjewIABFRcAgADf/TCACF0AgIAA3TgQgADA4ICAADbOAIAAzcgwgAEfc4CAAYjQ4IACkWQAgABFdICAABBsAIACOATw Date: Mon, 11 Aug 2014 20:53:51 +0000 Message-ID: <379439d133a548a29a8d5193c9e2620f@BY2PR05MB728.namprd05.prod.outlook.com> References: <201407251524.s6PFOPn92012@magenta.juniper.net> <53D79504.4050904@cisco.com> <04f15bedf2be4c7480d3e9ca01bdfd7f@BY2PR05MB728.namprd05.prod.outlook.com> <2597291f7b074922b690e4b06999cf1a@AM3PR03MB612.eurprd03.prod.outlook.com>, <30e13900814f41a681151971cbe9ebef@BY2PR05MB728.namprd05.prod.outlook.com> <1407215578702.64387@ecitele.com>, <2280b8e0d5f94a60badab91a2237181b@BY2PR05MB728.namprd05.prod.outlook.com> <1407304503657.72312@ecitele.com> <046050cb2d254a139b4650b2d7f93075@BY2PR05MB728.namprd05.prod.outlook.com> <09ab595ab24a46a0a4e87ebeb2cd185a@AM3PR03MB612.eurprd03.prod.outlook.com>, <2afb5c963e054aef929f38dcb3defa07@BY2PR05MB728.namprd05.prod.outlook.com>, <1407647722694.18724@ecitele.com> <004289D0-C473-4C1D-8A76-B42757F599A2@cisco.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [66.129.241.10] x-microsoft-antispam: BCL:0;PCL:0;RULEID:;UriScan:; x-forefront-prvs: 03008837BD x-forefront-antispam-report: SFV:NSPM; SFS:(377454003)(164054003)(189002)(199002)(252514010)(74316001)(76176999)(20776003)(15975445006)(50986999)(16236675004)(106116001)(18717965001)(64706001)(76482001)(101416001)(81342001)(74502001)(74662001)(31966008)(80022001)(81542001)(19609705001)(66066001)(105586002)(85306004)(106356001)(99396002)(92566001)(107046002)(93886004)(86362001)(85852003)(77982001)(19625215002)(15202345003)(76576001)(19300405004)(95666004)(33646002)(19580405001)(99286002)(2656002)(21056001)(19580395003)(54356999)(46102001)(83072002)(87936001)(83322001)(79102001)(4396001)(24736002)(108616003); DIR:OUT; SFP:; SCL:1; SRVR:BY2PR05MB223; H:BY2PR05MB728.namprd05.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; LANG:en; Content-Type: multipart/alternative; boundary="_000_379439d133a548a29a8d5193c9e2620fBY2PR05MB728namprd05pro_" MIME-Version: 1.0 X-OriginatorOrg: juniper.net Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/1ZJ4g9hcKa7SSUV1dfBEDJ5PXs0 Cc: Yakov Rekhter , "Eric Rosen \(erosen\)" , "pwe3@ietf.org" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Aug 2014 20:53:59 -0000 --_000_379439d133a548a29a8d5193c9e2620fBY2PR05MB728namprd05pro_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Stewart, Sasha, In LDP, a transit router may have ECMP to a give FEC (of egress PE), as you= mentioned. If this LSP carries multiple PWs, the hash algorithm on the rou= ter should map a given PW to one and only one path, in order to preserve pa= cket order on the egress PE. Let's look at an example in the context of this draft. Suppose this transit= router has 3 ECMP to the primary egress PE. On path_1 and path_2, the rout= er is the penultimate hop, most likely because this router has 2 parallel l= inks to the primary PE. On path_3, it is not the penultimate hop, because t= here are other downstream routers. Also, suppose there are 3 PWs to the egr= ess PE, and the hash algorithm has mapped PW_1 to path_1, PW_2 to path_2, a= nd PW_3 to path_3. Hence, this router serves as PLR for PW_1 and PW_2, but = not for PW_3. The PLR will set up a bypass tunnel to protect PW_1 and PW_2. (Theoretical= ly it could also set up 2 bypass tunnels to protect PW_1 and PW_2 separatel= y, but it doesn't really matter here). The PLR will then attach the bypass = as a backup nexthop to Path_1, and a backup nexthop to Path_2. Now, the PLR detects a link failure over Path_1. In the dataplane, the PLR = reacts to this event by moving PW_1's packets to the bypass. It will not do= anything for PW_2 or PW_3, because their paths are still up. IOW, PW_2 and= PW_3 are not affected. From CE's perspective, the CE will be receiving PW_= 1's traffic over the backup AC, while continuing to receive PW_2 and PW_3's= traffic over the primary AC. Likewise, if the router detects a failure of PW_2 or PW_3, the other 2 PWs = will not be affected. Thanks, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Sunday, August 10, 2014 6:23 AM To: Stewart Bryant (stbryant) Cc: Yimin Shen; Yakov Rekhter; pwe3@ietf.org; Eric Rosen (erosen) Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Stewart, My reading of your email is like following: 1. Within the scope of the draft the PLR is always the penultimate P = LSR on the LSP terminated by the Primary PE. I believe that the draft state= s that explicitly 2. The PLR cannot differentiate between the failure of the link conne= cting it to the Primary PE and failure of the Primary PE in a timely manner= ; hence every time such a link fails, the PLR assumes failure of the Primar= y PE to be on the safe side. T 3. With ECMP in the PSN, there typically would be several LSPs termin= ated by the Primary PE, each with its own PLR. So if the link between one o= f these PLRs and the Primary PE fails, it would apply the mechanisms define= d in the draft, while the rest of the PLRs would not do anything. 4. As a consequence, the dual-homed PE would receive some flows of th= e traffic crossing a flow-aware pw from one AC and some - from the other AC= . The draft does not specify that this is possible and that the CE should = be able to cope with this situation. Is my interpretation correct? Regards, Sasha Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 From: Stewart Bryant (stbryant) [mailto:stbryant@cisco.com] Sent: Sunday, August 10, 2014 12:24 PM To: Alexander Vainshtein Cc: Yimin Shen; Yakov Rekhter; pwe3@ietf.org; Eric Ro= sen (erosen) Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 ECMP paths to xPEs naturally form and disappear in LDP PSNs (they are a fun= damental property of that PSN type) and it is entirely normal to find an E= CMP to an xPE. So, if you have a PW that is load balancing (again quite normal) and one of= the attachments to the xPE fails, you can legitimately end up with some t= raffic on one egress AC and some on another. please you confirm that this design works correctly when a PE gets its traf= fic from both ACs? Stewart --_000_379439d133a548a29a8d5193c9e2620fBY2PR05MB728namprd05pro_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi Stewart, Sasha,

 <= /p>

In LDP, a transit router = may have ECMP to a give FEC (of egress PE), as you mentioned. If this LSP c= arries multiple PWs, the hash algorithm on the router should map a given PW to one and only one path, in order to preserve packet order= on the egress PE.

 <= /p>

Let’s look at an ex= ample in the context of this draft. Suppose this transit router has 3 ECMP = to the primary egress PE. On path_1 and path_2, the router is the penultimate hop, most likely because this router has 2 parallel links = to the primary PE. On path_3, it is not the penultimate hop, because there = are other downstream routers. Also, suppose there are 3 PWs to the egress P= E, and the hash algorithm has mapped PW_1 to path_1, PW_2 to path_2, and PW_3 to path_3. Hence, this router ser= ves as PLR for PW_1 and PW_2, but not for PW_3.

 <= /p>

The PLR will set up a byp= ass tunnel to protect PW_1 and PW_2.  (Theoretically it could also set= up 2 bypass tunnels to protect PW_1 and PW_2 separately, but it doesn’t really matter here). The PLR will then attach the bypass = as a backup nexthop to Path_1, and a backup nexthop to Path_2.

 <= /p>

Now, the PLR detects a li= nk failure over Path_1. In the dataplane, the PLR reacts to this event by m= oving PW_1’s packets to the bypass. It will not do anything for PW_2 or PW_3, because their paths are still up. IOW, PW_2 and PW_3 are= not affected. From CE’s perspective, the CE will be receiving PW_1&#= 8217;s traffic over the backup AC, while continuing to receive PW_2 and PW_= 3’s traffic over the primary AC.

 <= /p>

Likewise, if the router d= etects a failure of PW_2 or PW_3, the other 2 PWs will not be affected.

 <= /p>

Thanks,=

 <= /p>

/Yimin<= /p>

 <= /p>

 <= /p>

From: Alexande= r Vainshtein [mailto:Alexander.Vainshtein@ecitele.com]
Sent: Sunday, August 10, 2014 6:23 AM
To: Stewart Bryant (stbryant)
Cc: Yimin Shen; Yakov Rekhter; pwe3@ietf.org; Eric Rosen (erosen) Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Stewart,

 <= /p>

My reading of your email = is like following:

1. = ;      Within the scope = of the draft the PLR is always the penultimate P LSR on the LSP terminated = by the Primary PE. I believe that the draft states that explicitly

2. = ;      The PLR cannot di= fferentiate between the failure of the link connecting it to the Primary PE= and failure of the Primary PE in a timely manner; hence every time such a link fails, the PLR assumes failure of the Primary PE to= be on the safe side. T

3. = ;      With ECMP in the = PSN, there typically would be several LSPs terminated by the Primary PE, ea= ch with its own PLR. So if the link between one of these PLRs and the Primary PE fails, it would apply the mechanisms defined in th= e draft, while the rest of the PLRs would not do anything.

4. = ;      As a consequence,= the dual-homed PE would receive some flows of the traffic crossing a flow-= aware pw from one AC and some – from the other AC.  The draft does not specify that this is possible and that the CE should be abl= e to cope with this situation.

 <= /p>

Is my interpretation corr= ect?

 <= /p>

Regards,

    &= nbsp;  Sasha

Email: Alexander.Vainshtein@ec= itele.com

Mobile: 054-9266302<= /o:p>

 <= /p>

From: Stewart = Bryant (stbryant) [mailto:stbryant@ci= sco.com]
Sent: Sunday, August 10, 2014 12:24 PM
To: Alexander Vainshtein
Cc: Yimin Shen; Yakov Rekhter; pwe3= @ietf.org; Eric Rosen (erosen)
Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

 

ECMP paths to xPEs naturally form and disappear in L= DP PSNs (they are a fundamental property of that PSN type) and it is entire= ly normal  to find an ECMP to an xPE.

 

So, if you have a PW that is load balancing (again q= uite normal) and one of the attachments to the xPE fails, you can legitimat= ely end  up with some traffic on one egress AC and some on another.

 

please you confirm that this design works correctly = when a PE gets its traffic from both ACs?

 

Stewart

 

--_000_379439d133a548a29a8d5193c9e2620fBY2PR05MB728namprd05pro_-- From nobody Mon Aug 11 14:28:41 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A954A1A00EF for ; Mon, 11 Aug 2014 14:28:38 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -15.168 X-Spam-Level: X-Spam-Status: No, score=-15.168 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-0.668, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id c9s7Skeu8X8D for ; Mon, 11 Aug 2014 14:28:36 -0700 (PDT) Received: from alln-iport-7.cisco.com (alln-iport-7.cisco.com [173.37.142.94]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2BDB11A00D7 for ; Mon, 11 Aug 2014 14:28:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=22149; q=dns/txt; s=iport; t=1407792516; x=1409002116; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=9R82U65MOTIBQXhGWydaFd+8gaQ7kPQuteFtBHu3zag=; b=aE+OZ00eHlreG0IldJLioCULj+5D0pz8wPDYHi4+kfFHxTNa4XuUJ+GL v7Z+ONoJL13DfBTCMNy4U0flJarZ18PGQUCU1YzOyA854nazJzIaRxo6n 53vqzDHwfMyM0Q3XBIec3VAnen5SszrbV+T0ehYn1VbpXowmd+zJ/mZfY M=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgkFAH006VOtJA2B/2dsb2JhbABagkdGsGOlGAGBEhZ3hAMBAQEEHRA6EhACAQgOAwQBASEHBzIUCQgCBA4FiELDSheOYQkRAVAGAYMvgR0FjUiDUYZehDiUe4FlG4FcbIEO X-IronPort-AV: E=Sophos; i="5.01,844,1400025600"; d="scan'208,217"; a="68352597" Received: from alln-core-9.cisco.com ([173.36.13.129]) by alln-iport-7.cisco.com with ESMTP; 11 Aug 2014 21:28:35 +0000 Received: from xhc-aln-x03.cisco.com (xhc-aln-x03.cisco.com [173.36.12.77]) by alln-core-9.cisco.com (8.14.5/8.14.5) with ESMTP id s7BLSZhb012559 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Mon, 11 Aug 2014 21:28:35 GMT Received: from xmb-rcd-x04.cisco.com ([169.254.8.74]) by xhc-aln-x03.cisco.com ([173.36.12.77]) with mapi id 14.03.0195.001; Mon, 11 Aug 2014 16:28:35 -0500 From: "Stewart Bryant (stbryant)" To: Yimin Shen Thread-Topic: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 Thread-Index: AQHPqymWveYGLBru1kCsLuzIRj9tdJu3CWCAgAA8XgCAAPc/IIABBtiAgAg4EiaAAOtlgIAAs2DLgAA1qVCAASlTgIAAxW1QgAGWRICAAnyxroAAUUlrgAAN14CAApkKgP//tcn4 Date: Mon, 11 Aug 2014 21:28:13 +0000 Message-ID: <5C6C292D-0DC7-4B96-A24C-026BA778EE27@cisco.com> References: <201407251524.s6PFOPn92012@magenta.juniper.net> <53D79504.4050904@cisco.com> <04f15bedf2be4c7480d3e9ca01bdfd7f@BY2PR05MB728.namprd05.prod.outlook.com> <2597291f7b074922b690e4b06999cf1a@AM3PR03MB612.eurprd03.prod.outlook.com>, <30e13900814f41a681151971cbe9ebef@BY2PR05MB728.namprd05.prod.outlook.com> <1407215578702.64387@ecitele.com>, <2280b8e0d5f94a60badab91a2237181b@BY2PR05MB728.namprd05.prod.outlook.com> <1407304503657.72312@ecitele.com> <046050cb2d254a139b4650b2d7f93075@BY2PR05MB728.namprd05.prod.outlook.com> <09ab595ab24a46a0a4e87ebeb2cd185a@AM3PR03MB612.eurprd03.prod.outlook.com>, <2afb5c963e054aef929f38dcb3defa07@BY2PR05MB728.namprd05.prod.outlook.com>, <1407647722694.18724@ecitele.com> <004289D0-C473-4C1D-8A76-B42757F599A2@cisco.com> , <379439d133a548a29a8d5193c9e2620f@BY2PR05MB728.namprd05.prod.outlook.com> In-Reply-To: <379439d133a548a29a8d5193c9e2620f@BY2PR05MB728.namprd05.prod.outlook.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: multipart/alternative; boundary="_000_5C6C292D0DC74B96A24C026BA778EE27ciscocom_" MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/fs3SbGNlz_6ByhfxZdvlzs6nDfo Cc: Yakov Rekhter , "Eric Rosen \(erosen\)" , "pwe3@ietf.org" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Aug 2014 21:28:38 -0000 --_000_5C6C292D0DC74B96A24C026BA778EE27ciscocom_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Sent from my iPad On 11 Aug 2014, at 22:53, "Yimin Shen" > wrote: Hi Stewart, Sasha, In LDP, a transit router may have ECMP to a give FEC (of egress PE), as you= mentioned. If this LSP carries multiple PWs, the hash algorithm on the rou= ter should map a given PW to one and only one path, in order to preserve pa= cket order on the egress PE. Only when A) the CW is present and B) there is no FAT label, or I suppose no EL if you chose to do it that way= . The point is that with some PEs you only care about flow order not absolute= order. If you want to put them out of scope then fine, but you have to either expl= icitly state the restriction, or fix the problem, and without taking either= of those approaches you get traffic split over both ACs. Stewart Let=92s look at an example in the context of this draft. Suppose this trans= it router has 3 ECMP to the primary egress PE. On path_1 and path_2, the ro= uter is the penultimate hop, most likely because this router has 2 parallel= links to the primary PE. On path_3, it is not the penultimate hop, because= there are other downstream routers. Also, suppose there are 3 PWs to the e= gress PE, and the hash algorithm has mapped PW_1 to path_1, PW_2 to path_2,= and PW_3 to path_3. Hence, this router serves as PLR for PW_1 and PW_2, bu= t not for PW_3. The PLR will set up a bypass tunnel to protect PW_1 and PW_2. (Theoretical= ly it could also set up 2 bypass tunnels to protect PW_1 and PW_2 separatel= y, but it doesn=92t really matter here). The PLR will then attach the bypas= s as a backup nexthop to Path_1, and a backup nexthop to Path_2. Now, the PLR detects a link failure over Path_1. In the dataplane, the PLR = reacts to this event by moving PW_1=92s packets to the bypass. It will not = do anything for PW_2 or PW_3, because their paths are still up. IOW, PW_2 a= nd PW_3 are not affected. From CE=92s perspective, the CE will be receiving= PW_1=92s traffic over the backup AC, while continuing to receive PW_2 and = PW_3=92s traffic over the primary AC. Likewise, if the router detects a failure of PW_2 or PW_3, the other 2 PWs = will not be affected. Thanks, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Sunday, August 10, 2014 6:23 AM To: Stewart Bryant (stbryant) Cc: Yimin Shen; Yakov Rekhter; pwe3@ietf.org; Eric Ro= sen (erosen) Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Stewart, My reading of your email is like following: 1. Within the scope of the draft the PLR is always the penultimate P = LSR on the LSP terminated by the Primary PE. I believe that the draft state= s that explicitly 2. The PLR cannot differentiate between the failure of the link conne= cting it to the Primary PE and failure of the Primary PE in a timely manner= ; hence every time such a link fails, the PLR assumes failure of the Primar= y PE to be on the safe side. T 3. With ECMP in the PSN, there typically would be several LSPs termin= ated by the Primary PE, each with its own PLR. So if the link between one o= f these PLRs and the Primary PE fails, it would apply the mechanisms define= d in the draft, while the rest of the PLRs would not do anything. 4. As a consequence, the dual-homed PE would receive some flows of th= e traffic crossing a flow-aware pw from one AC and some =96 from the other = AC. The draft does not specify that this is possible and that the CE shoul= d be able to cope with this situation. Is my interpretation correct? Regards, Sasha Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 From: Stewart Bryant (stbryant) [mailto:stbryant@cisco.com] Sent: Sunday, August 10, 2014 12:24 PM To: Alexander Vainshtein Cc: Yimin Shen; Yakov Rekhter; pwe3@ietf.org; Eric Ro= sen (erosen) Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 ECMP paths to xPEs naturally form and disappear in LDP PSNs (they are a fun= damental property of that PSN type) and it is entirely normal to find an E= CMP to an xPE. So, if you have a PW that is load balancing (again quite normal) and one of= the attachments to the xPE fails, you can legitimately end up with some t= raffic on one egress AC and some on another. please you confirm that this design works correctly when a PE gets its traf= fic from both ACs? Stewart --_000_5C6C292D0DC74B96A24C026BA778EE27ciscocom_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable


Sent from my iPad

On 11 Aug 2014, at 22:53, "Yimin Shen" <yshen@juniper.net> wrote:

Hi Stewart, Sasha,

 <= /p>

In LDP, a transit router = may have ECMP to a give FEC (of egress PE), as you mentioned. If this LSP c= arries multiple PWs, the hash algorithm on the router should map a given PW to one and only one path, in order to preserve packet order= on the egress PE.


Only when 

A) the CW is present

and 

B) there is no FAT label, or I suppose no EL if you chose to do it tha= t way.

The point is that with some PEs you only care about flow order not abs= olute order.

If you want to put them out of scope then fine, but you have to either= explicitly state the restriction, or fix the problem, and without taking e= ither of those approaches you get traffic split over both ACs.

Stewart

 <= /p>

Let=92s look at an exampl= e in the context of this draft. Suppose this transit router has 3 ECMP to t= he primary egress PE. On path_1 and path_2, the router is the penultimate hop, most likely because this router has 2 parallel links = to the primary PE. On path_3, it is not the penultimate hop, because there = are other downstream routers. Also, suppose there are 3 PWs to the egress P= E, and the hash algorithm has mapped PW_1 to path_1, PW_2 to path_2, and PW_3 to path_3. Hence, this router ser= ves as PLR for PW_1 and PW_2, but not for PW_3.

 <= /p>

The PLR will set up a byp= ass tunnel to protect PW_1 and PW_2.  (Theoretically it could also set= up 2 bypass tunnels to protect PW_1 and PW_2 separately, but it doesn=92t really matter here). The PLR will then attach the bypass as a= backup nexthop to Path_1, and a backup nexthop to Path_2.

 <= /p>

Now, the PLR detects a li= nk failure over Path_1. In the dataplane, the PLR reacts to this event by m= oving PW_1=92s packets to the bypass. It will not do anything for PW_2 or PW_3, because their paths are still up. IOW, PW_2 and PW_3 are= not affected. From CE=92s perspective, the CE will be receiving PW_1=92s t= raffic over the backup AC, while continuing to receive PW_2 and PW_3=92s tr= affic over the primary AC.

 <= /p>

Likewise, if the router d= etects a failure of PW_2 or PW_3, the other 2 PWs will not be affected.

 <= /p>

Thanks,=

 <= /p>

/Yimin<= /p>

 <= /p>

 <= /p>

From: Alexande= r Vainshtein [mailto:Al= exander.Vainshtein@ecitele.com]
Sent: Sunday, August 10, 2014 6:23 AM
To: Stewart Bryant (stbryant)
Cc: Yimin Shen; Yakov Rekhter; pwe3= @ietf.org; Eric Rosen (erosen)
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Stewart,

 <= /p>

My reading of your email = is like following:

1.&= nbsp;      Within the sc= ope of the draft the PLR is always the penultimate P LSR on the LSP termina= ted by the Primary PE. I believe that the draft states that explicitly

2.&= nbsp;      The PLR canno= t differentiate between the failure of the link connecting it to the Primar= y PE and failure of the Primary PE in a timely manner; hence every time such a link fails, the PLR assumes failure of the Primary= PE to be on the safe side. T

3.&= nbsp;      With ECMP in = the PSN, there typically would be several LSPs terminated by the Primary PE= , each with its own PLR. So if the link between one of these PLRs and the Primary PE fails, it would apply the mechanisms defined= in the draft, while the rest of the PLRs would not do anything.=

4.&= nbsp;      As a conseque= nce, the dual-homed PE would receive some flows of the traffic crossing a f= low-aware pw from one AC and some =96 from the other AC.  The draft does not specify that this is possible and that the CE sho= uld be able to cope with this situation.

 <= /p>

Is my interpretation corr= ect?

 <= /p>

Regards,

    &= nbsp;  Sasha

Email: Alexander.Vainshtein@ec= itele.com

Mobile: 054-9266302<= /o:p>

 <= /p>

From: Stewart = Bryant (stbryant) [mailto:stbryant@ci= sco.com]
Sent: Sunday, August 10, 2014 12:24 PM
To: Alexander Vainshtein
Cc: Yimin Shen; Yakov Rekhter; pwe3= @ietf.org; Eric Rosen (erosen)
Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

 

ECMP paths to xPEs naturally form and disappear in L= DP PSNs (they are a fundamental property of that PSN type) and it is entire= ly normal  to find an ECMP to an xPE.

 

So, if you have a PW that is load balancing (again q= uite normal) and one of the attachments to the xPE fails, you can legitimat= ely end  up with some traffic on one egress AC and some on another.

 

please you confirm that this design works correctly = when a PE gets its traffic from both ACs?

 

Stewart

 

--_000_5C6C292D0DC74B96A24C026BA778EE27ciscocom_-- From nobody Mon Aug 11 21:15:20 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 29DEB1A015D for ; Mon, 11 Aug 2014 21:15:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.601 X-Spam-Level: X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QgO2-ttWk5ax for ; Mon, 11 Aug 2014 21:15:15 -0700 (PDT) Received: from emea01-db3-obe.outbound.protection.outlook.com (mail-db3lrp0083.outbound.protection.outlook.com [213.199.154.83]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 671311A014C for ; Mon, 11 Aug 2014 21:15:11 -0700 (PDT) Received: from AM3PR03MB612.eurprd03.prod.outlook.com (10.242.110.144) by AM3PR03MB609.eurprd03.prod.outlook.com (10.242.109.149) with Microsoft SMTP Server (TLS) id 15.0.1005.10; Tue, 12 Aug 2014 04:15:08 +0000 Received: from AM3PR03MB612.eurprd03.prod.outlook.com ([10.242.110.144]) by AM3PR03MB612.eurprd03.prod.outlook.com ([10.242.110.144]) with mapi id 15.00.1005.008; Tue, 12 Aug 2014 04:15:08 +0000 From: Alexander Vainshtein To: Yimin Shen Thread-Topic: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 Thread-Index: AQHPqymWveYGLBru1kCsLuzIRj9tdJu3CWCAgAA8XgCAAPc/IIABBtiAgAg4EiaAAOtlgIAAs2DLgAA1qVCAASlTgIAAxW1QgAGWRICAAnyxroACmasAgACEMUE= Date: Tue, 12 Aug 2014 04:15:08 +0000 Message-ID: <1407816900919.37823@ecitele.com> References: <201407251524.s6PFOPn92012@magenta.juniper.net> <53D79504.4050904@cisco.com> <04f15bedf2be4c7480d3e9ca01bdfd7f@BY2PR05MB728.namprd05.prod.outlook.com> <2597291f7b074922b690e4b06999cf1a@AM3PR03MB612.eurprd03.prod.outlook.com>, <30e13900814f41a681151971cbe9ebef@BY2PR05MB728.namprd05.prod.outlook.com> <1407215578702.64387@ecitele.com>, <2280b8e0d5f94a60badab91a2237181b@BY2PR05MB728.namprd05.prod.outlook.com> <1407304503657.72312@ecitele.com> <046050cb2d254a139b4650b2d7f93075@BY2PR05MB728.namprd05.prod.outlook.com> <09ab595ab24a46a0a4e87ebeb2cd185a@AM3PR03MB612.eurprd03.prod.outlook.com>, <2afb5c963e054aef929f38dcb3defa07@BY2PR05MB728.namprd05.prod.outlook.com> <1407647722694.18724@ecitele.com>, <6b0fa5df117f43d69c31efab88161e70@BY2PR05MB728.namprd05.prod.outlook.com> In-Reply-To: <6b0fa5df117f43d69c31efab88161e70@BY2PR05MB728.namprd05.prod.outlook.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [5.153.9.202] x-microsoft-antispam: BCL:0;PCL:0;RULEID:;UriScan:; x-forefront-prvs: 0301360BF5 x-forefront-antispam-report: SFV:NSPM; SFS:(10019004)(189002)(37854004)(252514010)(377454003)(45624003)(51444003)(164054003)(43784003)(199003)(80022001)(19580405001)(85852003)(79102001)(92726001)(87936001)(86362001)(20776003)(561944003)(4396001)(64706001)(83072002)(2656002)(92566001)(36756003)(81542001)(110136001)(85306004)(74502001)(19580395003)(54356999)(76176999)(74662001)(19627405001)(76482001)(46102001)(106116001)(105586002)(106356001)(107046002)(95666004)(66066001)(99396002)(19625215002)(101416001)(83322001)(16236675004)(50986999)(77982001)(81342001)(1941001)(31966008)(21056001)(19607625011)(19627315001); DIR:OUT; SFP:1102; SCL:1; SRVR:AM3PR03MB609; H:AM3PR03MB612.eurprd03.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Content-Type: multipart/alternative; boundary="_000_140781690091937823ecitelecom_" MIME-Version: 1.0 X-OriginatorOrg: ecitele.com Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/FdIdieVWTCl3Pf8rkh1D6i1E8Ds Cc: Yakov Rekhter , "erosen@cisco.com" , "pwe3@ietf.org" , "stbryant@cisco.com" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Aug 2014 04:15:19 -0000 --_000_140781690091937823ecitelecom_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Yimin, Lots of thanks for a prompt response. My concerns related to ECMP have been mainly about multiple potential PLRs = on equal cost LSPs between the ingress PE" and "Primary egress PE". I do not see how this can be easily avoided and my reading of your response= is that you do not consider it either. One of the aspects of multiple potential PLRs has been raised by Stewart in= his email: if the link between one of these and the Primary PE fails while= Primary PE remains OK, the traffic would be split. Another aspect of this scenario is that, in the case of the Primary PE fail= ure, each of these multiple PLRs would select the Protector as its best Nex= t Hop to the Context ID. Regards, Sasha ________________________________ From: Yimin Shen Sent: Monday, August 11, 2014 11:15 PM To: Alexander Vainshtein Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org; erosen@cisco.com Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Hi Sasha, By =93avoiding ECMP=94, I was strictly talking about the context ID FEC fro= m the PLR=92s perspective, rather than avoiding ECMP broadly for other FECs= on the PLR or any FECs on other routers. Specifically, if the PLR has a set of ECMP to the context ID FEC, all the E= CMP paths must traverse the primary PE only. This will ensure that all PWs = be carried to the primary PE. And this can be achieved by using the proxy n= ode approach suggested in this draft. The path(s) traversing the protector = to reach the context ID FEC can be used as bypass LSP(s). We can improve the text on this to make it clearer. Thanks, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Sunday, August 10, 2014 1:15 AM To: Yimin Shen Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org; erosen@cisco.com Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin hi, Lots of thanks for a prompt response. I must admit that I have missed the proposal in Section 4.2.2 to treat "con= text ID" IP address as a proxy node that is adjacent to both the Primary PE= and the Protector NEs. (It also has not been mentioned in Eri=E7's clarifi= cation email). I think that this approach would require certain manipulatio= n of both the IGP and LDP, but this requires more thought on my side. One o= f the issues that come to mind, is the need for LDP in both these nodes to = allocate a locally terminated label for the corresponding FEC while not adv= ertising context IP address as one of the IP addresses it owns. I think that your statement about "avoiding ECMP scenario" is problematic,= because, from my point of view, ECMP is one of the important characterist= ics of LDP-based MPLS PSN. And in any case the draft does not say anything= about that (or, at least neither ECMP nor "equal cost multipath" is mentio= ned in the text). If you see ECMP avoidance as a pre-condition for using t= he mechanism defined in draft with LDP LSPs, this should be quite explicit = and detailed IMO. I still fail to understand what constitutes an egress AC failure in the dr= aft and how it is different from the ingress AC failure. Unfortunately the = examples you've provided in your response do not help much: e.g., a fiber c= ut close to would be most probably recognized by the PE as a bi-directiona= l AC failure, while a fiber cut that is close to CE but remote from PE woul= d most probably not detected by the PE. (Of course if PE and CE are termina= ting the same fiber span, the fiber cut would be recognized by both). My guess (FWIW) that the draft actually discusses protection of traffic in = the PE --> CE direction triggered by a failure detected by the PE while lea= ving protection of traffic in the CE --> PE direction some other mechanism = that would be triggered by CE detecting the failure. This mechanism presuma= bly would be different from the PW redundancy mechanism because it would re= quire that an active-active scheme for Primary and Backup PWs. If this guess is correct (or close to reality), I would expect some explici= t text explaining it. In particular, this text should explain that "redunda= nt PWs" mentioned in the draft are not related to "PW redundancy". The curr= ent text makes a mix of these two notions, and this definitely does not hel= p the reader (or at least this reader :-). Hopefully these comments would be useful. Regards, Sasha ________________________________ From: Yimin Shen > Sent: Friday, August 8, 2014 5:34 PM To: Alexander Vainshtein Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Hi Sasha, Thanks again for the comments and questions! Regarding AC failure detection on PE, in the typical cases, if there is a f= iber cut, or a hardware failure affecting the port on the PE, or a hardware= failure affecting the port on the CE (assume the CE is directed attached t= o the PE), the PE should be able to directly and independently detect the f= ailure at physical or link layer. Regarding LDP, I agree with you that LDP by its nature is not as straight-f= orward as RSVP-TE in this scenario. Since a context ID are advertised by bo= th primary PE and protector, we should avoid creating an ECMP scenario. Tha= t=92s why section 4.2.2 provides a suggestion to advertise the context ID a= s a proxy node (i.e. a virtual node attached to the two routers), and make = its reachability via the primary PE more favorable than that via the protec= tor. MRT and other mechanisms could be used to help LDP improve protection = coverage. Also, nothing would prevent LDP from using RSVP bypass tunnels. A= s you can see, there may be many different approaches, and there may new ap= proaches coming in the future. In this PWE3 document, we just want to stay = at PW level and be generic with PSN level. In any case, this mechanism is a= chievable with LDP, and LDP as a tunnel signaling protocol really doesn=92t= need any new extension. Hope this clarifies it. Thanks, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Thursday, August 07, 2014 10:37 AM To: Yimin Shen Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin hi, Please see some comments inline below. Regards, Sasha Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 From: Yimin Shen [mailto:yshen@juniper.net] Sent: Thursday, August 07, 2014 5:34 AM To: Alexander Vainshtein Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Hi Sasha, Thanks again for the comments. Please see the following 4 items in response= to the 4 comments in your email. 1. Today when an egress PE detects an egress AC failure [[Sasha]] Could you please explain how the egress PE detects an egress AC f= ailure? I have always thought that egress AC failure has to be detected fir= st by corresponding CE which would then notify the egress PE using some sui= table =93Remote Failure=94/BDI/RDI mechanism. Specific examples would reall= y help. it will simply have to drop packets, until the remote PE stops sending traf= fic over the PW. This draft is intended to improve this behavior. By no mea= ns the local repair on egress PE will prevent the AC failure from being con= verted to and handled as a bi-directional AC failure. Note that bi-directio= nal AC failure handling must involve the CE, but the CE may detect the fail= ure at a slower pace, or it may not be able to detect it in a timely manner= , depending on the actual situation. [[Sasha]] Please see above. IMHO and FWIW if the CE cannot detect its =93Re= ceive Failure=94, the PE would not never know about the =93Transmit Failure= . E.g. there may be other L2 devices between the PE and CE; the AC may itself= be a logical wire across another network; the type and location of the fai= lure may make it easier for the PE to detect it; and so on. In any case, be= fore the AC failure is handled more gracefully (or bi-directionally), the e= gress traffic has been locally repaired by the egress PE. 2. As mentioned before, this draft doesn=92t have a dependency on a specifi= c PSN tunnel technology. [[Sasha]] I have provided my comments on this in my response to Eric. From = my point of view some LDP-specific issues still require clarification, but = my bottom line is that operating the draft over an MPLS network that uses L= DP in DU mode for setting up tunnel LSPs may sometimes be possible but woul= d always be tricky. This does not match my understanding of =93no dependenc= y on a specific PSN tunnel technology=94, but this may be a matter of taste= . 3. The draft already talks about scalability when describing the centralize= d protector model vs the co-located protector model. We can improve the tex= t to explicitly mention that the requirement for setting up LSPs on a per <= primary PE, protector> basis rather than per primary PE basis can potential= ly increase the number of LSPs in a network. This is a tradeoff. As mention= ed before, this does not necessarily cause a scaling issue. [[Sasha]] Yes, I have found the corresponding text in the draft. From my p= oint of view this issue is resolved. 4. We will leave this to PWE3 WG to decide whether this is really necessary= .[[Sasha]] I tend to agree with Eric that usage of context-specific labels = in the draft does not mean update to 4447. From my point of view this issue= is resolved as well. Thanks, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Wednesday, August 06, 2014 5:11 AM To: Yimin Shen Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin and all, I would like to summarize the issues that I have with this draft. Digging f= or them in a very long email thread with multiple levels of in-lining is no= t very effective IMO. 1. Protection against unidirectional AC failures looks problematic to= me: a. The draft explicitly differentiates between =93ingress=94 and =93e= gress=94 AC failures b. The outcome can be using a different AC for each direction of CE-to= -CE traffic with potentially destructive effect that cannot be mitigated at= the PWE3 level c. IMHO and FWIW a viable alternative could be to employ native AC me= chanisms converting unidirectional AC failures into bi-directional ones. Th= is would eliminate the differentiation between ingress and egress AC failur= es and facilitate using the PW redundancy mechanism for bi-directional prot= ection 2. Dependency upon specific mechanisms for setting up PSN tunnels see= ms to break some common PWE3 architectural principles: a. I am pretty well sure that the mechanisms specified in the draft c= annot be used with the PSN tunnel LSPs set up by LDP in DU mode b. If the authors, refer to LDP in the DoD mode augmented with some CR= -LDP-like mechanisms (e.g., for exclusion of specific links when setting up= the bypass LSP between the PLR and the Protector), this should, at the ver= y least, be discussed with and approved by the MPLS WG 3. Scalability of the proposed solution should be, at least, explicit= ly presented. In particular, the draft assumes that a dedicated LSP would b= e used between PE1 PE2 for each combination of PE2 with a specific Protect= or LSR. 4. The Protector LSR could be either a kind of S-PE (if different fro= m the secondary T-PE) or the secondary T-PE itself. In both cases it should= be capable of handling labels from specific context spaces as if they were= PW labels. This requires (at the very least) an update of RFC 4447 and sho= uld be explicitly presented and discussed. Hopefully this summary will be useful. Regards, Sasha Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 --_000_140781690091937823ecitelecom_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable

Yimin,

Lots of thanks for a prompt response.

 

My concerns related to ECMP have been mainly about multiple potential PL= Rs on equal cost LSPs between the ingress PE" and "Primary e= gress PE".

 

I do not see how this can be easily avoided and my reading of your respo= nse is that you do not consider it either.

 

One of the aspects of multiple potential PLRs has been raised by Stewart= in his email: if the link between one of these and the Primary PE fai= ls while Primary PE remains OK, the traffic would be split.

 

Another aspect of this scenario is that, in the case of the Primary= PE failure, each of these multiple PLRs would select the Protector as= its best Next Hop to the Context ID.

 

Regards,

Sasha

From: Yimin Shen <yshe= n@juniper.net>
Sent: Monday, August 11, 2014 11:15 PM
To: Alexander Vainshtein
Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org; erosen@cisco.c= om
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01
 

Hi Sasha,

 

By =93avoid= ing ECMP=94, I was strictly talking about the context ID FEC from the PLR= =92s perspective, rather than avoiding ECMP broadly for other FECs on the P= LR or any FECs on other routers.

 

Specificall= y, if the PLR has a set of ECMP to the context ID FEC, all the ECMP paths m= ust traverse the primary PE only. This will ensure that all PWs be carried = to the primary PE. And this can be achieved by using the proxy node approach suggested in this draft. The path(s) trav= ersing the protector to reach the context ID FEC can be used as bypass LSP(= s).

 

We can impr= ove the text on this to make it clearer.

 

Thanks,

 

/Yimin

 

 

From: Alex= ander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com]
Sent: Sunday, August 10, 2014 1:15 AM
To: Yimin Shen
Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org; erosen@cisco.c= om
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Yimin hi,

Lots of thanks for a prompt response.

 

I must admit that I have missed the propos= al in Section 4.2.2 to treat "context ID" IP address as a proxy n= ode that is adjacent to both the Primary PE and the Protector NEs. (It also= has not been mentioned in Eri=E7's clarification email). I think that this approach would require certain manipulation of b= oth the IGP and LDP, but this requires more thought on my side. One of the&= nbsp;issues that come to mind, is the need for LDP in both these = nodes to allocate a locally terminated label for the corresponding FEC while not advertising context IP address as one of t= he IP addresses it owns.

 

I think that your statement about "av= oiding ECMP  scenario" is problematic, because, from my= point of view,  ECMP is one of the important characteristics of = LDP-based MPLS PSN.  And in any case the draft does not say anyth= ing about that (or, at least neither ECMP nor "equal cost multi= path" is mentioned in the text).  If you see ECMP avoidance = as a pre-condition for using the mechanism defined in draft with LDP LSPs, = this should be quite explicit and detailed IMO.

 

I still fail to understand  what cons= titutes an egress AC failure in the draft and how it is different from the = ingress AC failure. Unfortunately the examples you've provided in your resp= onse do not help much: e.g., a fiber cut close to  would be most probably recognized by the PE as a bi-directi= onal AC failure, while a fiber cut that is close to CE but remote from PE w= ould most probably not detected by the PE. (Of course if PE and CE are term= inating the same fiber span, the fiber cut would be recognized by both).

 

My guess (FWIW) that the draft actually di= scusses protection of traffic in the PE --> CE direction triggered by a = failure detected by the PE while leaving protection of traffic in the CE --= > PE direction some other mechanism that would be triggered by CE detecting the failure. This mechanism presumably = would be different from the PW redundancy mechanism because it would requir= e that an active-active scheme for Primary and Backup  PWs.

 

If this guess is correct (or close to real= ity), I would expect some explicit text explaining it. In particular, this = text should explain that "redundant PWs" mentioned in the draft a= re not related to "PW redundancy". The current text makes a mix of these two notions, and this definitely does not help t= he reader (or at least this reader :-).

 

Hopefully these comments would be useful.<= /span>

 

Regards,

Sasha

From: Yimin Shen <yshen@juniper.net>
Sent: Friday, August 8, 2014 5:34 PM
To: Alexander Vainshtein
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Hi Sasha,

 

Thanks agai= n for the comments and questions!

 

Regarding A= C failure detection on PE, in the typical cases, if there is a fiber cut, o= r a hardware failure affecting the port on the PE, or a hardware failure af= fecting the port on the CE (assume the CE is directed attached to the PE), the PE should be able to directly and = independently detect the failure at physical or link layer.

 

Regarding L= DP, I agree with you that LDP by its nature is not as straight-forward as R= SVP-TE in this scenario. Since a context ID are advertised by both primary = PE and protector, we should avoid creating an ECMP scenario. That=92s why section 4.2.2 provides a suggestion to adve= rtise the context ID as a proxy node (i.e. a virtual node attached to the t= wo routers), and make its reachability via the primary PE more favorable th= an that via the protector. MRT and other mechanisms could be used to help LDP improve protection coverage. Al= so, nothing would prevent LDP from using RSVP bypass tunnels. As you can se= e, there may be many different approaches, and there may new approaches com= ing in the future. In this PWE3 document, we just want to stay at PW level and be generic with PSN level. = In any case, this mechanism is achievable with LDP, and LDP as a tunnel sig= naling protocol really doesn=92t need any new extension.  

 

Hope this c= larifies it.

 

Thanks,

 

/Yimin

 

 

From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ec= itele.com]
Sent: Thursday, August 07, 2014 10:37 AM
To: Yimin Shen
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Yimin hi,<= /span>

Please see= some comments inline below.<= /span>

 

Regards,

 &nbs= p;     Sasha

Email: Alexander.Vainshtein@ecitele.com

Mobile: 05= 4-9266302

 

From: Yimin Shen [mailto:yshen@juniper.net]
Sent: Thursday, August 07, 2014 5:34 AM
To: Alexander Vainshtein
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Hi Sasha,

 

Thanks agai= n for the comments. Please see the following 4 items in response to the 4 c= omments in your email.

 

1. Today wh= en an egress PE detects an egress AC failure

&nbs= p;

[[Sa= sha]] Could you please explain how the egress PE detects an egress AC failu= re? I have always thought that egress AC failure has to be detected first b= y corresponding CE which would then notify the egress PE using some suitable =93Remote Failure=94/BDI/RDI mech= anism. Specific examples would really help.

&nbs= p;

it will sim= ply have to drop packets, until the remote PE stops sending traffic over th= e PW. This draft is intended to improve this behavior. By no means the loca= l repair on egress PE will prevent the AC failure from being converted to and handled as a bi-directional AC fail= ure. Note that bi-directional AC failure handling must involve the CE, but = the CE may detect the failure at a slower pace, or it may not be able to de= tect it in a timely manner, depending on the actual situation.

[[Sa= sha]] Please see above. IMHO and FWIW if the CE cannot detect its =93Receiv= e Failure=94, the PE would not never know about the =93Transmit Failure.

&nbs= p;

E.g. there = may be other L2 devices between the PE and CE; the AC may itself be a logic= al wire across another network; the type and location of the failure may ma= ke it easier for the PE to detect it; and so on. In any case, before the AC failure is handled more gracefully (= or bi-directionally), the egress traffic has been locally repaired by the e= gress PE.

 

2. As menti= oned before, this draft doesn=92t have a dependency on a specific PSN tunne= l technology.

[[Sa= sha]] I have provided my comments on this in my response to Eric. From my p= oint of view some LDP-specific issues still require clarification, but my b= ottom line is that operating the draft over an MPLS networ= k that uses LDP in DU mode for setting up tunnel LSPs may sometimes be poss= ible but would always be tricky. This does not match my understandin= g of =93no dependency on a specific PSN tunnel technology=94, but this may be a matter of taste.

 

3. The draf= t already talks about scalability when describing the centralized protector= model vs the co-located protector model. We can improve the text to explic= itly mention that the requirement for setting up LSPs on a per <primary PE, protector> basis rather than p= er primary PE basis can potentially increase the number of LSPs in a networ= k. This is a tradeoff. As mentioned before, this does not necessarily cause= a scaling issue.

[[Sa= sha]] Yes, I have found the corresponding text in the draft.  From my = point of view this issue is resolved.

 

4. We will = leave this to PWE3 WG to decide whether this is really necessary.= [[Sasha]] I tend to agree with= Eric that usage of context-specific labels in the draft does not mean update to 4447. From my point of view this issu= e is resolved as well.

 

Thanks,

 

/Yimin

 

 

From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ec= itele.com]
Sent: Wednesday, August 06, 2014 5:11 AM
To: Yimin Shen
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Yimin and= all,

I would l= ike to summarize the issues that I have with this draft. Digging for them i= n a very long email thread with multiple levels of in-lining is not very ef= fective IMO.

 

1.       Protection against unidir= ectional AC failures loo= ks problematic to me:

a.       The draft explicitly differe= ntiates between =93ingress=94 and =93egress=94 AC failures

b.      The outcome can be using a d= ifferent AC for each direction of CE-to-CE traffic with potentially destruc= tive effect that cannot be mitigated at the PWE3 level

c.       IMHO and FWIW a viable alter= native could be to employ native AC mechanisms converting unidirectional AC= failures into bi-directional ones. This would eliminate the differentiatio= n between ingress and egress AC failures and facilitate using the PW redundancy mechanism for bi-directional protec= tion

2.       Dependency upon specific = mechanisms for setting up PSN tunnels seems to break some common PWE3 architectural principles:

a.       I am pretty well sure that t= he mechanisms specified in the draft cannot be used with the PSN tunnel LSP= s set up by LDP in DU mode

b.      If the authors, refer to LDP= in the DoD mode augmented with some CR-LDP-like mechanisms (e.g., for exclusio= n of specific links when setting up the bypass LSP between the PLR and the = Protector), this should, at the very least, be discussed with and approved = by the MPLS WG

3.       Scalability of the proposed solution should be, a= t least, explicitly presented. In particular, the draft assumes that a dedi= cated LSP would be used between PE1  PE2 for each combination of PE2 with a specific Protector LSR.

4.       The Protector LSR could be e= ither a kind of S-PE (if different from the secondary T-PE) or the secondar= y T-PE itself. In both cases it should be capable of handling labels from s= pecific context spaces as if they were PW labels. This requires (at the very least) an update of RFC 4447 an= d should be explicitly presented and discussed.

Hopefully= this summary will be useful.

 

 

Regards,<= /span>

 &nb= sp;     Sasha

Email: Alexander.Vainshtein@ecitele.com

Mobile: 0= 54-9266302

--_000_140781690091937823ecitelecom_-- From nobody Tue Aug 12 08:24:42 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 579B61A03F1 for ; Tue, 12 Aug 2014 08:24:40 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.901 X-Spam-Level: X-Spam-Status: No, score=-1.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id klXpuqIzlgHE for ; Tue, 12 Aug 2014 08:24:32 -0700 (PDT) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1lp0145.outbound.protection.outlook.com [207.46.163.145]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0EE061A03E4 for ; Tue, 12 Aug 2014 08:24:31 -0700 (PDT) Received: from BY2PR05MB222.namprd05.prod.outlook.com (10.242.41.151) by BY2PR05MB744.namprd05.prod.outlook.com (10.141.223.148) with Microsoft SMTP Server (TLS) id 15.0.1005.10; Tue, 12 Aug 2014 15:24:30 +0000 Received: from BY2PR05MB728.namprd05.prod.outlook.com (10.141.223.25) by BY2PR05MB222.namprd05.prod.outlook.com (10.242.41.151) with Microsoft SMTP Server (TLS) id 15.0.1005.10; Tue, 12 Aug 2014 15:24:27 +0000 Received: from BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) by BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) with mapi id 15.00.1005.008; Tue, 12 Aug 2014 15:24:27 +0000 From: Yimin Shen To: "Stewart Bryant (stbryant)" Thread-Topic: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 Thread-Index: AQHPqByPzRsIO8QlbkK+kbMBZ9MDf5u3AuAAgAAZ0oCAABjewIABFRcAgADf/TCACF0AgIAA3TgQgADA4ICAADbOAIAAzcgwgAEfc4CAAYjQ4IACkWQAgABFdICAABBsAIACOATwgAAUOYCAASozwA== Date: Tue, 12 Aug 2014 15:24:27 +0000 Message-ID: References: <201407251524.s6PFOPn92012@magenta.juniper.net> <53D79504.4050904@cisco.com> <04f15bedf2be4c7480d3e9ca01bdfd7f@BY2PR05MB728.namprd05.prod.outlook.com> <2597291f7b074922b690e4b06999cf1a@AM3PR03MB612.eurprd03.prod.outlook.com>, <30e13900814f41a681151971cbe9ebef@BY2PR05MB728.namprd05.prod.outlook.com> <1407215578702.64387@ecitele.com>, <2280b8e0d5f94a60badab91a2237181b@BY2PR05MB728.namprd05.prod.outlook.com> <1407304503657.72312@ecitele.com> <046050cb2d254a139b4650b2d7f93075@BY2PR05MB728.namprd05.prod.outlook.com> <09ab595ab24a46a0a4e87ebeb2cd185a@AM3PR03MB612.eurprd03.prod.outlook.com>, <2afb5c963e054aef929f38dcb3defa07@BY2PR05MB728.namprd05.prod.outlook.com>, <1407647722694.18724@ecitele.com> <004289D0-C473-4C1D-8A76-B42757F599A2@cisco.com> , <379439d133a548a29a8d5193c9e2620f@BY2PR05MB728.namprd05.prod.outlook.com> <5C6C292D-0DC7-4B96-A24C-026BA778EE27@cisco.com> In-Reply-To: <5C6C292D-0DC7-4B96-A24C-026BA778EE27@cisco.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [66.129.241.10] x-microsoft-antispam: BCL:0;PCL:0;RULEID:;UriScan:;UriScan:; x-forefront-prvs: 0301360BF5 x-forefront-antispam-report: SFV:NSPM; SFS:(24454002)(164054003)(52604005)(377454003)(189002)(252514010)(199003)(110136001)(21056001)(93886004)(85306004)(4396001)(95666004)(66066001)(99286002)(19300405004)(106116001)(105586002)(106356001)(64706001)(85852003)(33646002)(19609705001)(20776003)(18717965001)(101416001)(15202345003)(86362001)(2656002)(31966008)(74662001)(79102001)(77982001)(76482001)(46102001)(92566001)(16236675004)(76176999)(50986999)(87936001)(19625215002)(83072002)(19580405001)(81342001)(15975445006)(80022001)(99396002)(74502001)(54356999)(74316001)(107046002)(83322001)(81542001)(76576001)(19580395003)(24736002)(108616003); DIR:OUT; SFP:; SCL:1; SRVR:BY2PR05MB222; H:BY2PR05MB728.namprd05.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Content-Type: multipart/alternative; boundary="_000_fbe4066b005742d2a5ab10cd6cc3af91BY2PR05MB728namprd05pro_" MIME-Version: 1.0 X-Microsoft-Antispam: BCL:0;PCL:0;RULEID:; X-OriginatorOrg: juniper.net Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/B56gW27xIvOk82Pf4ZoPQvaOkSo Cc: Yakov Rekhter , "Eric Rosen \(erosen\)" , "pwe3@ietf.org" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Aug 2014 15:24:40 -0000 --_000_fbe4066b005742d2a5ab10cd6cc3af91BY2PR05MB728namprd05pro_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Stewart, Thanks for pointing this out. We will explicitly describe these cases in th= e draft. Even if traffic may arrive at CE on both ACs, it should not cause = an issue for the CE, because either the CE doesn't care about packet order = (case A), or packet order of sub-flow associated with a FAT label is still = preserved (case B). Thanks, /Yimin From: Stewart Bryant (stbryant) [mailto:stbryant@cisco.com] Sent: Monday, August 11, 2014 5:28 PM To: Yimin Shen Cc: Alexander Vainshtein; Yakov Rekhter; pwe3@ietf.org; Eric Rosen (erosen) Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Sent from my iPad On 11 Aug 2014, at 22:53, "Yimin Shen" > wrote: Hi Stewart, Sasha, In LDP, a transit router may have ECMP to a give FEC (of egress PE), as you= mentioned. If this LSP carries multiple PWs, the hash algorithm on the rou= ter should map a given PW to one and only one path, in order to preserve pa= cket order on the egress PE. Only when A) the CW is present and B) there is no FAT label, or I suppose no EL if you chose to do it that way= . The point is that with some PEs you only care about flow order not absolute= order. If you want to put them out of scope then fine, but you have to either expl= icitly state the restriction, or fix the problem, and without taking either= of those approaches you get traffic split over both ACs. Stewart Let's look at an example in the context of this draft. Suppose this transit= router has 3 ECMP to the primary egress PE. On path_1 and path_2, the rout= er is the penultimate hop, most likely because this router has 2 parallel l= inks to the primary PE. On path_3, it is not the penultimate hop, because t= here are other downstream routers. Also, suppose there are 3 PWs to the egr= ess PE, and the hash algorithm has mapped PW_1 to path_1, PW_2 to path_2, a= nd PW_3 to path_3. Hence, this router serves as PLR for PW_1 and PW_2, but = not for PW_3. The PLR will set up a bypass tunnel to protect PW_1 and PW_2. (Theoretical= ly it could also set up 2 bypass tunnels to protect PW_1 and PW_2 separatel= y, but it doesn't really matter here). The PLR will then attach the bypass = as a backup nexthop to Path_1, and a backup nexthop to Path_2. Now, the PLR detects a link failure over Path_1. In the dataplane, the PLR = reacts to this event by moving PW_1's packets to the bypass. It will not do= anything for PW_2 or PW_3, because their paths are still up. IOW, PW_2 and= PW_3 are not affected. From CE's perspective, the CE will be receiving PW_= 1's traffic over the backup AC, while continuing to receive PW_2 and PW_3's= traffic over the primary AC. Likewise, if the router detects a failure of PW_2 or PW_3, the other 2 PWs = will not be affected. Thanks, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Sunday, August 10, 2014 6:23 AM To: Stewart Bryant (stbryant) Cc: Yimin Shen; Yakov Rekhter; pwe3@ietf.org; Eric Ro= sen (erosen) Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Stewart, My reading of your email is like following: 1. Within the scope of the draft the PLR is always the penultimate P L= SR on the LSP terminated by the Primary PE. I believe that the draft states= that explicitly 2. The PLR cannot differentiate between the failure of the link connec= ting it to the Primary PE and failure of the Primary PE in a timely manner;= hence every time such a link fails, the PLR assumes failure of the Primary= PE to be on the safe side. T 3. With ECMP in the PSN, there typically would be several LSPs termina= ted by the Primary PE, each with its own PLR. So if the link between one of= these PLRs and the Primary PE fails, it would apply the mechanisms defined= in the draft, while the rest of the PLRs would not do anything. 4. As a consequence, the dual-homed PE would receive some flows of the= traffic crossing a flow-aware pw from one AC and some - from the other AC.= The draft does not specify that this is possible and that the CE should b= e able to cope with this situation. Is my interpretation correct? Regards, Sasha Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 From: Stewart Bryant (stbryant) [mailto:stbryant@cisco.com] Sent: Sunday, August 10, 2014 12:24 PM To: Alexander Vainshtein Cc: Yimin Shen; Yakov Rekhter; pwe3@ietf.org; Eric Ro= sen (erosen) Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 ECMP paths to xPEs naturally form and disappear in LDP PSNs (they are a fun= damental property of that PSN type) and it is entirely normal to find an E= CMP to an xPE. So, if you have a PW that is load balancing (again quite normal) and one of= the attachments to the xPE fails, you can legitimately end up with some t= raffic on one egress AC and some on another. please you confirm that this design works correctly when a PE gets its traf= fic from both ACs? Stewart --_000_fbe4066b005742d2a5ab10cd6cc3af91BY2PR05MB728namprd05pro_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi Stewart,

 <= /p>

Thanks for pointing this = out. We will explicitly describe these cases in the draft. Even if traffic = may arrive at CE on both ACs, it should not cause an issue for the CE, because either the CE doesn’t care about packet order (c= ase A), or packet order of sub-flow associated with a FAT label is still pr= eserved (case B).

 <= /p>

 <= /p>

Thanks,=

 <= /p>

/Yimin<= /p>

 <= /p>

 <= /p>

From: Stewart = Bryant (stbryant) [mailto:stbryant@cisco.com]
Sent: Monday, August 11, 2014 5:28 PM
To: Yimin Shen
Cc: Alexander Vainshtein; Yakov Rekhter; pwe3@ietf.org; Eric Rosen (= erosen)
Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 



Sent from my iPad


On 11 Aug 2014, at 22:53, "Yimin Shen" <yshen@juniper.net> wrote:

Hi Stewart, Sasha,=

 <= /p>

In LDP, a transit router = may have ECMP to a give FEC (of egress PE), as you mentioned. If this LSP c= arries multiple PWs, the hash algorithm on the router should map a given PW to one and only one path, in order to preserve packet order= on the egress PE.

 

Only when 

 

A) the CW is present

 

and 

 

B) there is no FAT label, or I suppose no EL if you = chose to do it that way.

 

The point is that with some PEs you only care about = flow order not absolute order.

 

If you want to put them out of scope then fine, but = you have to either explicitly state the restriction, or fix the problem, an= d without taking either of those approaches you get traffic split over both= ACs.

 

Stewart

 

 <= /p>

Let’s look at an ex= ample in the context of this draft. Suppose this transit router has 3 ECMP = to the primary egress PE. On path_1 and path_2, the router is the penultimate hop, most likely because this router has 2 parallel links = to the primary PE. On path_3, it is not the penultimate hop, because there = are other downstream routers. Also, suppose there are 3 PWs to the egress P= E, and the hash algorithm has mapped PW_1 to path_1, PW_2 to path_2, and PW_3 to path_3. Hence, this router ser= ves as PLR for PW_1 and PW_2, but not for PW_3.

 <= /p>

The PLR will set up a byp= ass tunnel to protect PW_1 and PW_2.  (Theoretically it could also set= up 2 bypass tunnels to protect PW_1 and PW_2 separately, but it doesn’t really matter here). The PLR will then attach the bypass = as a backup nexthop to Path_1, and a backup nexthop to Path_2.<= /o:p>

 <= /p>

Now, the PLR detects a li= nk failure over Path_1. In the dataplane, the PLR reacts to this event by m= oving PW_1’s packets to the bypass. It will not do anything for PW_2 or PW_3, because their paths are still up. IOW, PW_2 and PW_3 are= not affected. From CE’s perspective, the CE will be receiving PW_1&#= 8217;s traffic over the backup AC, while continuing to receive PW_2 and PW_= 3’s traffic over the primary AC.

 <= /p>

Likewise, if the router d= etects a failure of PW_2 or PW_3, the other 2 PWs will not be affected.

 <= /p>

Thanks,=

 <= /p>

/Yimin<= /p>

 <= /p>

 <= /p>

From: Alexande= r Vainshtein [mailto:Al= exander.Vainshtein@ecitele.com]
Sent: Sunday, August 10, 2014 6:23 AM
To: Stewart Bryant (stbryant)
Cc: Yimin Shen; Yakov Rekhter; pwe3= @ietf.org; Eric Rosen (erosen)
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Stewart,

 <= /p>

My reading of your email = is like following:

1.      Within the scope of the = draft the PLR is always the penultimate P LSR on the LSP terminated by the = Primary PE. I believe that the draft states that explicitly

2.      The PLR cannot different= iate between the failure of the link connecting it to the Primary PE and fa= ilure of the Primary PE in a timely manner; hence every time such a link fails, the PLR assumes failure of the Primary PE to be on= the safe side. T

3.      With ECMP in the PSN, th= ere typically would be several LSPs terminated by the Primary PE, each with= its own PLR. So if the link between one of these PLRs and the Primary PE fails, it would apply the mechanisms defined in the dra= ft, while the rest of the PLRs would not do anything.

4.      As a consequence, the du= al-homed PE would receive some flows of the traffic crossing a flow-aware p= w from one AC and some – from the other AC.  The draft does not specify that this is possible and that the CE should be able to c= ope with this situation.

 <= /p>

Is my interpretation corr= ect?

 <= /p>

Regards,

    &= nbsp;  Sasha

Email: Alexander.Vainshtein@ec= itele.com

Mobile: 054-9266302

 <= /p>

From: Stewart = Bryant (stbryant) [mailto:stbryant@ci= sco.com]
Sent: Sunday, August 10, 2014 12:24 PM
To: Alexander Vainshtein
Cc: Yimin Shen; Yakov Rekhter; pwe3= @ietf.org; Eric Rosen (erosen)
Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

 

ECMP paths to xPEs naturally form and disappear in L= DP PSNs (they are a fundamental property of that PSN type) and it is entire= ly normal  to find an ECMP to an xPE.

 

So, if you have a PW that is load balancing (again q= uite normal) and one of the attachments to the xPE fails, you can legitimat= ely end  up with some traffic on one egress AC and some on another.

 

please you confirm that this design works correctly = when a PE gets its traffic from both ACs?

 

Stewart

 

--_000_fbe4066b005742d2a5ab10cd6cc3af91BY2PR05MB728namprd05pro_-- From nobody Tue Aug 12 08:26:12 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BD5781A0428 for ; Tue, 12 Aug 2014 08:26:10 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.601 X-Spam-Level: X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fYhvin_icKQ1 for ; Tue, 12 Aug 2014 08:26:00 -0700 (PDT) Received: from na01-bl2-obe.outbound.protection.outlook.com (mail-bl2lp0204.outbound.protection.outlook.com [207.46.163.204]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 339A31A0947 for ; Tue, 12 Aug 2014 08:25:56 -0700 (PDT) Received: from BY2PR05MB728.namprd05.prod.outlook.com (10.141.223.25) by BY2PR05MB223.namprd05.prod.outlook.com (10.242.41.141) with Microsoft SMTP Server (TLS) id 15.0.1005.10; Tue, 12 Aug 2014 15:25:52 +0000 Received: from BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) by BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) with mapi id 15.00.1005.008; Tue, 12 Aug 2014 15:25:52 +0000 From: Yimin Shen To: Alexander Vainshtein Thread-Topic: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 Thread-Index: AQHPqByPzRsIO8QlbkK+kbMBZ9MDf5u3AuAAgAAZ0oCAABjewIABFRcAgADf/TCACF0AgIAA3TgQgADA4ICAADbOAIAAzcgwgAEfc4CAAYjQ4IACkWQAgAKGg+CAAI1LAIAArafg Date: Tue, 12 Aug 2014 15:25:51 +0000 Message-ID: References: <201407251524.s6PFOPn92012@magenta.juniper.net> <53D79504.4050904@cisco.com> <04f15bedf2be4c7480d3e9ca01bdfd7f@BY2PR05MB728.namprd05.prod.outlook.com> <2597291f7b074922b690e4b06999cf1a@AM3PR03MB612.eurprd03.prod.outlook.com>, <30e13900814f41a681151971cbe9ebef@BY2PR05MB728.namprd05.prod.outlook.com> <1407215578702.64387@ecitele.com>, <2280b8e0d5f94a60badab91a2237181b@BY2PR05MB728.namprd05.prod.outlook.com> <1407304503657.72312@ecitele.com> <046050cb2d254a139b4650b2d7f93075@BY2PR05MB728.namprd05.prod.outlook.com> <09ab595ab24a46a0a4e87ebeb2cd185a@AM3PR03MB612.eurprd03.prod.outlook.com>, <2afb5c963e054aef929f38dcb3defa07@BY2PR05MB728.namprd05.prod.outlook.com> <1407647722694.18724@ecitele.com>, <6b0fa5df117f43d69c31efab88161e70@BY2PR05MB728.namprd05.prod.outlook.com> <1407816900919.37823@ecitele.com> In-Reply-To: <1407816900919.37823@ecitele.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [66.129.241.10] x-microsoft-antispam: BCL:0;PCL:0;RULEID:;UriScan:; x-forefront-prvs: 0301360BF5 x-forefront-antispam-report: SFV:NSPM; SFS:(60444003)(37854004)(377454003)(45624003)(189002)(43784003)(164054003)(199003)(252514010)(80022001)(66066001)(20776003)(18717965001)(50986999)(31966008)(76176999)(74316001)(15975445006)(76482001)(561944003)(106116001)(101416001)(81342001)(64706001)(74662001)(74502001)(81542001)(105586002)(106356001)(99396002)(107046002)(85306004)(83072002)(92566001)(86362001)(15202345003)(85852003)(77982001)(93886004)(110136001)(19625215002)(76576001)(95666004)(19300405004)(33646002)(99286002)(19580405001)(46102001)(54356999)(2656002)(21056001)(16236675004)(19580395003)(87936001)(4396001)(79102001)(83322001)(24736002)(108616003); DIR:OUT; SFP:; SCL:1; SRVR:BY2PR05MB223; H:BY2PR05MB728.namprd05.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Content-Type: multipart/alternative; boundary="_000_e7b0e541e0ff4d6aa93258713017e8b1BY2PR05MB728namprd05pro_" MIME-Version: 1.0 X-OriginatorOrg: juniper.net Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/3p3Jmf6GXb1gxy-pqO9k315y8yU Cc: Yakov Rekhter , "erosen@cisco.com" , "pwe3@ietf.org" , "stbryant@cisco.com" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Aug 2014 15:26:10 -0000 --_000_e7b0e541e0ff4d6aa93258713017e8b1BY2PR05MB728namprd05pro_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi Sasha, Please see inline... Thanks, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Tuesday, August 12, 2014 12:15 AM To: Yimin Shen Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org; erosen@cisco.com Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin, Lots of thanks for a prompt response. My concerns related to ECMP have been mainly about multiple potential PLRs = on equal cost LSPs between the ingress PE" and "Primary egress PE". I do not see how this can be easily avoided and my reading of your response= is that you do not consider it either. [yshen] This draft doesn't preclude or avoid it, as I explained below. One of the aspects of multiple potential PLRs has been raised by Stewart in= his email: if the link between one of these and the Primary PE fails while= Primary PE remains OK, the traffic would be split. [yshen] Packets of "one given PW" may be load-balanced over ECMP to the pri= mary PE, due to no-CW or FAT label. In the above failure case, we only move= the affected "branch of flow" over to a bypass tunnel. This shouldn't caus= e any issue for the CE, because either [1] the CE doesn't care about packet= order (in the case where the PW has no CW), or [2] packet order of the sub= -flow associated with a FAT label is still preserved. Another aspect of this scenario is that, in the case of the Primary PE fail= ure, each of these multiple PLRs would select the Protector as its best Nex= t Hop to the Context ID. [yshen] Yes, and the PLR should move packets from the bypass tunnel back to= this LSP, which is now going over the new "best next hop" to the protector= . From the protector's perspective, this LSP will actually be the same as = the bypass tunnel, because it's associated with the context ID FEC and henc= e its label still points to the context label table where PW label should b= e looked up. We can improve the text of the draft to explicitly clarify thi= s. Regards, Sasha ________________________________ From: Yimin Shen > Sent: Monday, August 11, 2014 11:15 PM To: Alexander Vainshtein Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org; erosen@cisco.com Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Hi Sasha, By "avoiding ECMP", I was strictly talking about the context ID FEC from th= e PLR's perspective, rather than avoiding ECMP broadly for other FECs on th= e PLR or any FECs on other routers. Specifically, if the PLR has a set of ECMP to the context ID FEC, all the E= CMP paths must traverse the primary PE only. This will ensure that all PWs = be carried to the primary PE. And this can be achieved by using the proxy n= ode approach suggested in this draft. The path(s) traversing the protector = to reach the context ID FEC can be used as bypass LSP(s). We can improve the text on this to make it clearer. Thanks, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Sunday, August 10, 2014 1:15 AM To: Yimin Shen Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org; erosen@cisco.com Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin hi, Lots of thanks for a prompt response. I must admit that I have missed the proposal in Section 4.2.2 to treat "con= text ID" IP address as a proxy node that is adjacent to both the Primary PE= and the Protector NEs. (It also has not been mentioned in Eri=E7's clarifi= cation email). I think that this approach would require certain manipulatio= n of both the IGP and LDP, but this requires more thought on my side. One o= f the issues that come to mind, is the need for LDP in both these nodes to = allocate a locally terminated label for the corresponding FEC while not adv= ertising context IP address as one of the IP addresses it owns. I think that your statement about "avoiding ECMP scenario" is problematic,= because, from my point of view, ECMP is one of the important characterist= ics of LDP-based MPLS PSN. And in any case the draft does not say anything= about that (or, at least neither ECMP nor "equal cost multipath" is mentio= ned in the text). If you see ECMP avoidance as a pre-condition for using t= he mechanism defined in draft with LDP LSPs, this should be quite explicit = and detailed IMO. I still fail to understand what constitutes an egress AC failure in the dr= aft and how it is different from the ingress AC failure. Unfortunately the = examples you've provided in your response do not help much: e.g., a fiber c= ut close to would be most probably recognized by the PE as a bi-directiona= l AC failure, while a fiber cut that is close to CE but remote from PE woul= d most probably not detected by the PE. (Of course if PE and CE are termina= ting the same fiber span, the fiber cut would be recognized by both). My guess (FWIW) that the draft actually discusses protection of traffic in = the PE --> CE direction triggered by a failure detected by the PE while lea= ving protection of traffic in the CE --> PE direction some other mechanism = that would be triggered by CE detecting the failure. This mechanism presuma= bly would be different from the PW redundancy mechanism because it would re= quire that an active-active scheme for Primary and Backup PWs. If this guess is correct (or close to reality), I would expect some explici= t text explaining it. In particular, this text should explain that "redunda= nt PWs" mentioned in the draft are not related to "PW redundancy". The curr= ent text makes a mix of these two notions, and this definitely does not hel= p the reader (or at least this reader :-). Hopefully these comments would be useful. Regards, Sasha ________________________________ From: Yimin Shen > Sent: Friday, August 8, 2014 5:34 PM To: Alexander Vainshtein Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Hi Sasha, Thanks again for the comments and questions! Regarding AC failure detection on PE, in the typical cases, if there is a f= iber cut, or a hardware failure affecting the port on the PE, or a hardware= failure affecting the port on the CE (assume the CE is directed attached t= o the PE), the PE should be able to directly and independently detect the f= ailure at physical or link layer. Regarding LDP, I agree with you that LDP by its nature is not as straight-f= orward as RSVP-TE in this scenario. Since a context ID are advertised by bo= th primary PE and protector, we should avoid creating an ECMP scenario. Tha= t's why section 4.2.2 provides a suggestion to advertise the context ID as = a proxy node (i.e. a virtual node attached to the two routers), and make it= s reachability via the primary PE more favorable than that via the protecto= r. MRT and other mechanisms could be used to help LDP improve protection co= verage. Also, nothing would prevent LDP from using RSVP bypass tunnels. As = you can see, there may be many different approaches, and there may new appr= oaches coming in the future. In this PWE3 document, we just want to stay at= PW level and be generic with PSN level. In any case, this mechanism is ach= ievable with LDP, and LDP as a tunnel signaling protocol really doesn't nee= d any new extension. Hope this clarifies it. Thanks, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Thursday, August 07, 2014 10:37 AM To: Yimin Shen Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin hi, Please see some comments inline below. Regards, Sasha Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 From: Yimin Shen [mailto:yshen@juniper.net] Sent: Thursday, August 07, 2014 5:34 AM To: Alexander Vainshtein Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Hi Sasha, Thanks again for the comments. Please see the following 4 items in response= to the 4 comments in your email. 1. Today when an egress PE detects an egress AC failure [[Sasha]] Could you please explain how the egress PE detects an egress AC f= ailure? I have always thought that egress AC failure has to be detected fir= st by corresponding CE which would then notify the egress PE using some sui= table "Remote Failure"/BDI/RDI mechanism. Specific examples would really he= lp. it will simply have to drop packets, until the remote PE stops sending traf= fic over the PW. This draft is intended to improve this behavior. By no mea= ns the local repair on egress PE will prevent the AC failure from being con= verted to and handled as a bi-directional AC failure. Note that bi-directio= nal AC failure handling must involve the CE, but the CE may detect the fail= ure at a slower pace, or it may not be able to detect it in a timely manner= , depending on the actual situation. [[Sasha]] Please see above. IMHO and FWIW if the CE cannot detect its "Rece= ive Failure", the PE would not never know about the "Transmit Failure. E.g. there may be other L2 devices between the PE and CE; the AC may itself= be a logical wire across another network; the type and location of the fai= lure may make it easier for the PE to detect it; and so on. In any case, be= fore the AC failure is handled more gracefully (or bi-directionally), the e= gress traffic has been locally repaired by the egress PE. 2. As mentioned before, this draft doesn't have a dependency on a specific = PSN tunnel technology. [[Sasha]] I have provided my comments on this in my response to Eric. From = my point of view some LDP-specific issues still require clarification, but = my bottom line is that operating the draft over an MPLS network that uses L= DP in DU mode for setting up tunnel LSPs may sometimes be possible but woul= d always be tricky. This does not match my understanding of "no dependency = on a specific PSN tunnel technology", but this may be a matter of taste. 3. The draft already talks about scalability when describing the centralize= d protector model vs the co-located protector model. We can improve the tex= t to explicitly mention that the requirement for setting up LSPs on a per <= primary PE, protector> basis rather than per primary PE basis can potential= ly increase the number of LSPs in a network. This is a tradeoff. As mention= ed before, this does not necessarily cause a scaling issue. [[Sasha]] Yes, I have found the corresponding text in the draft. From my p= oint of view this issue is resolved. 4. We will leave this to PWE3 WG to decide whether this is really necessary= .[[Sasha]] I tend to agree with Eric that usage of context-specific labels = in the draft does not mean update to 4447. From my point of view this issue= is resolved as well. Thanks, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Wednesday, August 06, 2014 5:11 AM To: Yimin Shen Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin and all, I would like to summarize the issues that I have with this draft. Digging f= or them in a very long email thread with multiple levels of in-lining is no= t very effective IMO. 1. Protection against unidirectional AC failures looks problematic to= me: a. The draft explicitly differentiates between "ingress" and "egress"= AC failures b. The outcome can be using a different AC for each direction of CE-to= -CE traffic with potentially destructive effect that cannot be mitigated at= the PWE3 level c. IMHO and FWIW a viable alternative could be to employ native AC me= chanisms converting unidirectional AC failures into bi-directional ones. Th= is would eliminate the differentiation between ingress and egress AC failur= es and facilitate using the PW redundancy mechanism for bi-directional prot= ection 2. Dependency upon specific mechanisms for setting up PSN tunnels see= ms to break some common PWE3 architectural principles: a. I am pretty well sure that the mechanisms specified in the draft c= annot be used with the PSN tunnel LSPs set up by LDP in DU mode b. If the authors, refer to LDP in the DoD mode augmented with some CR= -LDP-like mechanisms (e.g., for exclusion of specific links when setting up= the bypass LSP between the PLR and the Protector), this should, at the ver= y least, be discussed with and approved by the MPLS WG 3. Scalability of the proposed solution should be, at least, explicit= ly presented. In particular, the draft assumes that a dedicated LSP would b= e used between PE1 PE2 for each combination of PE2 with a specific Protect= or LSR. 4. The Protector LSR could be either a kind of S-PE (if different fro= m the secondary T-PE) or the secondary T-PE itself. In both cases it should= be capable of handling labels from specific context spaces as if they were= PW labels. This requires (at the very least) an update of RFC 4447 and sho= uld be explicitly presented and discussed. Hopefully this summary will be useful. Regards, Sasha Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 --_000_e7b0e541e0ff4d6aa93258713017e8b1BY2PR05MB728namprd05pro_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Hi Sasha,

 

Please see inline̷= 0;

 

 

Thanks,

 

/Yimin

 

 

From: Alexande= r Vainshtein [mailto:Alexander.Vainshtein@ecitele.com]
Sent: Tuesday, August 12, 2014 12:15 AM
To: Yimin Shen
Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org; erosen@cisco.c= om
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Yimin,

Lots of thanks for a prompt response.

 

My concerns related to ECMP have been mainly= about multiple potential PLRs on equal cost LSPs between the ingress = PE" and "Primary egress PE".

 

I do not see how this can be easily avoided = and my reading of your response is that you do not consider it either.=

 

[yshen] This draft doesn’t preclu= de or avoid it, as I explained below.

 

One of the aspects of multiple potential PLR= s has been raised by Stewart in his email: if the link between one of = these and the Primary PE fails while Primary PE remains OK, the traffic wou= ld be split.

 

[yshen] Packets of “one given PW&= #8221; may be load-balanced over ECMP to the primary PE, due to no-CW or FA= T label. In the above failure case, we only move the affected “branch of flow” over to a bypass tunnel. This shouldn’t cause any iss= ue for the CE, because either [1] the CE doesn’t care about packet or= der (in the case where the PW has no CW), or [2] packet order of the sub-fl= ow associated with a FAT label is still preserved.

 

Another aspect of this scenario is that= , in the case of the Primary PE failure, each of these multiple PLRs w= ould select the Protector as its best Next Hop to the Context ID.

 

[yshen] Yes, and the PLR should move pa= ckets from the bypass tunnel back to this LSP, which is now going over the = new “best next hop” to the protector.  From the protector&= #8217;s perspective, this LSP will actually be the same as the bypass tunnel, beca= use it’s associated with the context ID FEC and hence its label still= points to the context label table where PW label should be looked up. We c= an improve the text of the draft to explicitly clarify this.

 

Regards,

Sasha

From: Yimin Shen <yshen@juniper.net>
Sent: Monday, August 11, 2014 11:15 PM
To: Alexander Vainshtein
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org; erosen@cisco.com
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Hi Sasha,

 

By “avoiding ECM= P”, I was strictly talking about the context ID FEC from the PLR̵= 7;s perspective, rather than avoiding ECMP broadly for other FECs on the PL= R or any FECs on other routers.

 

Specifically, if the P= LR has a set of ECMP to the context ID FEC, all the ECMP paths must travers= e the primary PE only. This will ensure that all PWs be carried to the prim= ary PE. And this can be achieved by using the proxy node approach suggested in this draft. The path(s) travers= ing the protector to reach the context ID FEC can be used as bypass LSP(s).=

 

We can improve the tex= t on this to make it clearer.

 

Thanks,

 

/Yimin

 

 

From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com]
Sent: Sunday, August 10, 2014 1:15 AM
To: Yimin Shen
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org; erosen@cisco.com
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Yimin hi,

Lots of thanks for a prompt response.=

 <= o:p>

I must admit that I have missed the proposal= in Section 4.2.2 to treat "context ID" IP address as a proxy nod= e that is adjacent to both the Primary PE and the Protector NEs. (It also h= as not been mentioned in Eri=E7's clarification email). I think that this approach would require certain manipulation of b= oth the IGP and LDP, but this requires more thought on my side. One of the&= nbsp;issues that come to mind, is the need for LDP in both these = nodes to allocate a locally terminated label for the corresponding FEC while not advertising context IP address as one of t= he IP addresses it owns.

 <= o:p>

I think that your statement about "avoi= ding ECMP  scenario" is problematic, because, from my p= oint of view,  ECMP is one of the important characteristics of LD= P-based MPLS PSN.  And in any case the draft does not say anythin= g about that (or, at least neither ECMP nor "equal cost multipath&q= uot; is mentioned in the text).  If you see ECMP avoidance as a p= re-condition for using the mechanism defined in draft with LDP LSPs, this s= hould be quite explicit and detailed IMO.

 <= o:p>

I still fail to understand  what consti= tutes an egress AC failure in the draft and how it is different from the in= gress AC failure. Unfortunately the examples you've provided in your respon= se do not help much: e.g., a fiber cut close to  would be most probably recognized by the PE as a bi-directi= onal AC failure, while a fiber cut that is close to CE but remote from PE w= ould most probably not detected by the PE. (Of course if PE and CE are term= inating the same fiber span, the fiber cut would be recognized by both).

 <= o:p>

My guess (FWIW) that the draft actually disc= usses protection of traffic in the PE --> CE direction triggered by a fa= ilure detected by the PE while leaving protection of traffic in the CE --&g= t; PE direction some other mechanism that would be triggered by CE detecting the failure. This mechanism presumably = would be different from the PW redundancy mechanism because it would requir= e that an active-active scheme for Primary and Backup  PWs.

 <= o:p>

If this guess is correct (or close to realit= y), I would expect some explicit text explaining it. In particular, this te= xt should explain that "redundant PWs" mentioned in the draft are= not related to "PW redundancy". The current text makes a mix of these two notions, and this definitely does not help t= he reader (or at least this reader :-).

 <= o:p>

Hopefully these comments would be useful.

 <= o:p>

Regards,

Sasha

From: Yimin Shen <yshen@juniper.net>
Sent: Friday, August 8, 2014 5:34 PM
To: Alexander Vainshtein
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Hi Sasha,

 

Thanks again for the c= omments and questions!

 

Regarding AC failure d= etection on PE, in the typical cases, if there is a fiber cut, or a hardwar= e failure affecting the port on the PE, or a hardware failure affecting the= port on the CE (assume the CE is directed attached to the PE), the PE should be able to directly and independently d= etect the failure at physical or link layer.

 

Regarding LDP, I agree= with you that LDP by its nature is not as straight-forward as RSVP-TE in t= his scenario. Since a context ID are advertised by both primary PE and prot= ector, we should avoid creating an ECMP scenario. That’s why section 4.2.2 provides a suggestion to advertis= e the context ID as a proxy node (i.e. a virtual node attached to the two r= outers), and make its reachability via the primary PE more favorable than t= hat via the protector. MRT and other mechanisms could be used to help LDP improve protection coverage. Also, nothing would= prevent LDP from using RSVP bypass tunnels. As you can see, there may be m= any different approaches, and there may new approaches coming in the future= . In this PWE3 document, we just want to stay at PW level and be generic with PSN level. In any case, this = mechanism is achievable with LDP, and LDP as a tunnel signaling protocol re= ally doesn’t need any new extension.  

 

Hope this clarifies it= .

 

Thanks,

 

/Yimin

 

 

From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com]
Sent: Thursday, August 07, 2014 10:37 AM
To: Yimin Shen
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Yimin hi,

Please see some commen= ts inline below.

 

Regards,

   &nbs= p;   Sasha

Email: Alexander.Vainshtein@ecitele.com

Mobile: 054-9266302

 

From: Yimin Shen [mailto= :yshen@juniper.net]
Sent: Thursday, August 07, 2014 5:34 AM
To: Alexander Vainshtein
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Hi Sasha,

 

Thanks again for the c= omments. Please see the following 4 items in response to the 4 comments in = your email.

 

1. Today when an egres= s PE detects an egress AC failure=

 

[[Sasha]] Could = you please explain how the egress PE detects an egress AC failure? I have a= lways thought that egress AC failure has to be detected first by correspond= ing CE which would then notify the egress PE using some suitable “Remote Failure”/BDI/RDI mechanism. Spe= cific examples would really help.

 

it will simply have to= drop packets, until the remote PE stops sending traffic over the PW. This = draft is intended to improve this behavior. By no means the local repair on= egress PE will prevent the AC failure from being converted to and handled as a bi-directional AC failure. Note t= hat bi-directional AC failure handling must involve the CE, but the CE may = detect the failure at a slower pace, or it may not be able to detect it in = a timely manner, depending on the actual situation. <= /p>

[[Sasha]] Please= see above. IMHO and FWIW if the CE cannot detect its “Receive Failur= e”, the PE would not never know about the “Transmit Failure.

 

E.g. there may be othe= r L2 devices between the PE and CE; the AC may itself be a logical wire acr= oss another network; the type and location of the failure may make it easie= r for the PE to detect it; and so on. In any case, before the AC failure is handled more gracefully (or bi-direc= tionally), the egress traffic has been locally repaired by the egress PE.

 

2. As mentioned before= , this draft doesn’t have a dependency on a specific PSN tunnel techn= ology.

[[Sasha]] I have= provided my comments on this in my response to Eric. From my point of view= some LDP-specific issues still require clarification, but my bottom line i= s that operating the draft over an MPLS network = that uses LDP in DU mode for setting up tunnel LSPs may sometimes be possib= le but would always be tricky. This does not match my understanding = of “no dependency on a specific PSN tunnel technology”, but this may be a matter of taste.

 

3. The draft already t= alks about scalability when describing the centralized protector model vs t= he co-located protector model. We can improve the text to explicitly mentio= n that the requirement for setting up LSPs on a per <primary PE, protector> basis rather than per primary = PE basis can potentially increase the number of LSPs in a network. This is = a tradeoff. As mentioned before, this does not necessarily cause a scaling = issue.

[[Sasha]] Yes, I= have found the corresponding text in the draft.  From my point of vie= w this issue is resolved.=

 

4. We will leave this = to PWE3 WG to decide whether this is really necessary.[[Sasha]] I tend to agree with Eric that usage of con= text-specific labels in the draft does not mean update to 4447. From my point of view this issue is resolved as w= ell.

 

Thanks,

 

/Yimin

 

 

From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com]
Sent: Wednesday, August 06, 2014 5:11 AM
To: Yimin Shen
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Yimin and all,=

I would like to sum= marize the issues that I have with this draft. Digging for them in a very l= ong email thread with multiple levels of in-lining is not very effective IM= O.

 

1. &nbs= p;     Protection against unidirectional A= C failures looks problematic to me= :

a. &nb= sp;     The draft explicitly differentiates be= tween “ingress” and “egress” AC failures=

b. &nb= sp;    The outcome can be using a different A= C for each direction of CE-to-CE traffic with potentially destructive effec= t that cannot be mitigated at the PWE3 level

c. &nb= sp;     IMHO and FWIW a viable alternative cou= ld be to employ native AC mechanisms converting unidirectional AC failures = into bi-directional ones. This would eliminate the differentiation between = ingress and egress AC failures and facilitate using the PW redundancy mechanism for bi-directional protection=

2. &nbs= p;     Dependency upon specific mechanisms= for setting up PSN tunnels seems = to break some common PWE3 architectural principles:

a. &nb= sp;     I am pretty well sure that the mechani= sms specified in the draft cannot be used with the PSN tunnel LSPs set up b= y LDP in DU mode

b. &nb= sp;    If the authors, refer to LDP in the DoD mode augmented with some CR-LDP-like mechanisms (e.g., for exclusi= on of specific links when setting up the bypass LSP between the PLR and the= Protector), this should, at the very least, be discussed with and approved by the MPLS WG

3. &nbs= p;     Scalability of the proposed solution should be, at least, explicitl= y presented. In particular, the draft assumes that a dedicated LSP would be= used between PE1  PE2 for each combination of PE2 with a specific Protector LSR.

4. &nbs= p;     The Protector LSR could be either a ki= nd of S-PE (if different from the secondary T-PE) or the secondary T-PE its= elf. In both cases it should be capable of handling labels from specific co= ntext spaces as if they were PW labels. This requires (at the very least) an update of RFC 4447 and should be expl= icitly presented and discussed.

Hopefully this summ= ary will be useful.

 

 

Regards,=

   &= nbsp;   Sasha

Email: Alexander.Vainshtein@ecitele.com

Mobile: 054-9266302=

--_000_e7b0e541e0ff4d6aa93258713017e8b1BY2PR05MB728namprd05pro_-- From nobody Tue Aug 12 08:42:56 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7E9BF1A098D for ; Tue, 12 Aug 2014 08:42:54 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.001 X-Spam-Level: X-Spam-Status: No, score=-2.001 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, J_CHICKENPOX_27=0.6, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RwlQOaCmwR7o for ; Tue, 12 Aug 2014 08:42:45 -0700 (PDT) Received: from emea01-am1-obe.outbound.protection.outlook.com (mail-am1lrp0012.outbound.protection.outlook.com [213.199.154.12]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9410E1A0983 for ; Tue, 12 Aug 2014 08:42:44 -0700 (PDT) Received: from AM3PR03MB612.eurprd03.prod.outlook.com (10.242.110.144) by AM3PR03MB612.eurprd03.prod.outlook.com (10.242.110.144) with Microsoft SMTP Server (TLS) id 15.0.1005.10; Tue, 12 Aug 2014 15:42:40 +0000 Received: from AM3PR03MB612.eurprd03.prod.outlook.com ([10.242.110.144]) by AM3PR03MB612.eurprd03.prod.outlook.com ([10.242.110.144]) with mapi id 15.00.1005.008; Tue, 12 Aug 2014 15:42:40 +0000 From: Alexander Vainshtein To: Yimin Shen Thread-Topic: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 Thread-Index: AQHPqymWveYGLBru1kCsLuzIRj9tdJu3CWCAgAA8XgCAAPc/IIABBtiAgAg4EiaAAOtlgIAAs2DLgAA1qVCAASlTgIAAxW1QgAGWRICAAnyxroACmasAgACEMUGAAL0sgIAAAmKA Date: Tue, 12 Aug 2014 15:42:40 +0000 Message-ID: <808559abe80a4cf99368a16c33d1e066@AM3PR03MB612.eurprd03.prod.outlook.com> References: <201407251524.s6PFOPn92012@magenta.juniper.net> <53D79504.4050904@cisco.com> <04f15bedf2be4c7480d3e9ca01bdfd7f@BY2PR05MB728.namprd05.prod.outlook.com> <2597291f7b074922b690e4b06999cf1a@AM3PR03MB612.eurprd03.prod.outlook.com>, <30e13900814f41a681151971cbe9ebef@BY2PR05MB728.namprd05.prod.outlook.com> <1407215578702.64387@ecitele.com>, <2280b8e0d5f94a60badab91a2237181b@BY2PR05MB728.namprd05.prod.outlook.com> <1407304503657.72312@ecitele.com> <046050cb2d254a139b4650b2d7f93075@BY2PR05MB728.namprd05.prod.outlook.com> <09ab595ab24a46a0a4e87ebeb2cd185a@AM3PR03MB612.eurprd03.prod.outlook.com>, <2afb5c963e054aef929f38dcb3defa07@BY2PR05MB728.namprd05.prod.outlook.com> <1407647722694.18724@ecitele.com>, <6b0fa5df117f43d69c31efab88161e70@BY2PR05MB728.namprd05.prod.outlook.com> <1407816900919.37823@ecitele.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [147.234.56.21] x-microsoft-antispam: BCL:0;PCL:0;RULEID:;UriScan:; x-forefront-prvs: 0301360BF5 x-forefront-antispam-report: SFV:NSPM; SFS:(10019004)(189002)(45624003)(199003)(164054003)(43784003)(37854004)(252514010)(377454003)(60444003)(66066001)(110136001)(77982001)(76176999)(93886004)(74502001)(2656002)(54356999)(76482001)(83072002)(85852003)(87936001)(74662001)(4396001)(86362001)(92566001)(31966008)(79102001)(575784001)(80022001)(46102001)(15202345003)(19580405001)(74316001)(105586002)(95666004)(106116001)(106356001)(15975445006)(33646002)(19300405004)(16236675004)(64706001)(81342001)(1941001)(19580395003)(81542001)(107046002)(50986999)(99396002)(76576001)(101416001)(19625215002)(85306004)(20776003)(561944003)(83322001)(21056001)(24736002)(108616003); DIR:OUT; SFP:1102; SCL:1; SRVR:AM3PR03MB612; H:AM3PR03MB612.eurprd03.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Content-Type: multipart/alternative; boundary="_000_808559abe80a4cf99368a16c33d1e066AM3PR03MB612eurprd03pro_" MIME-Version: 1.0 X-OriginatorOrg: ecitele.com Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/5TLoraqsyzobNlEZ3IJw5wX8Dk4 Cc: Yakov Rekhter , "erosen@cisco.com" , "pwe3@ietf.org" , "stbryant@cisco.com" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Aug 2014 15:42:54 -0000 --_000_808559abe80a4cf99368a16c33d1e066AM3PR03MB612eurprd03pro_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Yimin hi! Lots of thanks for a prompt response. The problem with splitting the PW traffic is not limited to re0rdering. In the simplest case, consider the case when the PW s have been set between= two IEEE802.1 bridges that perform MAC learning. Split traffic could easily result in frequent re-learning of a certain MAC = address... The bottom line is that if split traffic cannot be precluded, the dual-home= d CE must be aware of this fact and behave accordingly (e.g., by grouping a= pair of ports that support its connections to Primary PE and the Protector= PE as a LAG:)). >From my point of view these dependencies should be explicitly defined (say,= in the Applicability Statement). To the best of my recollection, this has = been a long-established PWE3 WG policy from its early days. Regards, Sasha Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 From: Yimin Shen [mailto:yshen@juniper.net] Sent: Tuesday, August 12, 2014 6:26 PM To: Alexander Vainshtein Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org; erosen@cisco.com Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Hi Sasha, Please see inline... Thanks, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Tuesday, August 12, 2014 12:15 AM To: Yimin Shen Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org; erosen@cisco.com Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin, Lots of thanks for a prompt response. My concerns related to ECMP have been mainly about multiple potential PLRs = on equal cost LSPs between the ingress PE" and "Primary egress PE". I do not see how this can be easily avoided and my reading of your response= is that you do not consider it either. [yshen] This draft doesn't preclude or avoid it, as I explained below. One of the aspects of multiple potential PLRs has been raised by Stewart in= his email: if the link between one of these and the Primary PE fails while= Primary PE remains OK, the traffic would be split. [yshen] Packets of "one given PW" may be load-balanced over ECMP to the pri= mary PE, due to no-CW or FAT label. In the above failure case, we only move= the affected "branch of flow" over to a bypass tunnel. This shouldn't caus= e any issue for the CE, because either [1] the CE doesn't care about packet= order (in the case where the PW has no CW), or [2] packet order of the sub= -flow associated with a FAT label is still preserved. Another aspect of this scenario is that, in the case of the Primary PE fail= ure, each of these multiple PLRs would select the Protector as its best Nex= t Hop to the Context ID. [yshen] Yes, and the PLR should move packets from the bypass tunnel back to= this LSP, which is now going over the new "best next hop" to the protector= . From the protector's perspective, this LSP will actually be the same as = the bypass tunnel, because it's associated with the context ID FEC and henc= e its label still points to the context label table where PW label should b= e looked up. We can improve the text of the draft to explicitly clarify thi= s. Regards, Sasha ________________________________ From: Yimin Shen > Sent: Monday, August 11, 2014 11:15 PM To: Alexander Vainshtein Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org; erosen@cisco.com Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Hi Sasha, By "avoiding ECMP", I was strictly talking about the context ID FEC from th= e PLR's perspective, rather than avoiding ECMP broadly for other FECs on th= e PLR or any FECs on other routers. Specifically, if the PLR has a set of ECMP to the context ID FEC, all the E= CMP paths must traverse the primary PE only. This will ensure that all PWs = be carried to the primary PE. And this can be achieved by using the proxy n= ode approach suggested in this draft. The path(s) traversing the protector = to reach the context ID FEC can be used as bypass LSP(s). We can improve the text on this to make it clearer. Thanks, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Sunday, August 10, 2014 1:15 AM To: Yimin Shen Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org; erosen@cisco.com Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin hi, Lots of thanks for a prompt response. I must admit that I have missed the proposal in Section 4.2.2 to treat "con= text ID" IP address as a proxy node that is adjacent to both the Primary PE= and the Protector NEs. (It also has not been mentioned in Eri=E7's clarifi= cation email). I think that this approach would require certain manipulatio= n of both the IGP and LDP, but this requires more thought on my side. One o= f the issues that come to mind, is the need for LDP in both these nodes to = allocate a locally terminated label for the corresponding FEC while not adv= ertising context IP address as one of the IP addresses it owns. I think that your statement about "avoiding ECMP scenario" is problematic,= because, from my point of view, ECMP is one of the important characterist= ics of LDP-based MPLS PSN. And in any case the draft does not say anything= about that (or, at least neither ECMP nor "equal cost multipath" is mentio= ned in the text). If you see ECMP avoidance as a pre-condition for using t= he mechanism defined in draft with LDP LSPs, this should be quite explicit = and detailed IMO. I still fail to understand what constitutes an egress AC failure in the dr= aft and how it is different from the ingress AC failure. Unfortunately the = examples you've provided in your response do not help much: e.g., a fiber c= ut close to would be most probably recognized by the PE as a bi-directiona= l AC failure, while a fiber cut that is close to CE but remote from PE woul= d most probably not detected by the PE. (Of course if PE and CE are termina= ting the same fiber span, the fiber cut would be recognized by both). My guess (FWIW) that the draft actually discusses protection of traffic in = the PE --> CE direction triggered by a failure detected by the PE while lea= ving protection of traffic in the CE --> PE direction some other mechanism = that would be triggered by CE detecting the failure. This mechanism presuma= bly would be different from the PW redundancy mechanism because it would re= quire that an active-active scheme for Primary and Backup PWs. If this guess is correct (or close to reality), I would expect some explici= t text explaining it. In particular, this text should explain that "redunda= nt PWs" mentioned in the draft are not related to "PW redundancy". The curr= ent text makes a mix of these two notions, and this definitely does not hel= p the reader (or at least this reader :-). Hopefully these comments would be useful. Regards, Sasha ________________________________ From: Yimin Shen > Sent: Friday, August 8, 2014 5:34 PM To: Alexander Vainshtein Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Hi Sasha, Thanks again for the comments and questions! Regarding AC failure detection on PE, in the typical cases, if there is a f= iber cut, or a hardware failure affecting the port on the PE, or a hardware= failure affecting the port on the CE (assume the CE is directed attached t= o the PE), the PE should be able to directly and independently detect the f= ailure at physical or link layer. Regarding LDP, I agree with you that LDP by its nature is not as straight-f= orward as RSVP-TE in this scenario. Since a context ID are advertised by bo= th primary PE and protector, we should avoid creating an ECMP scenario. Tha= t's why section 4.2.2 provides a suggestion to advertise the context ID as = a proxy node (i.e. a virtual node attached to the two routers), and make it= s reachability via the primary PE more favorable than that via the protecto= r. MRT and other mechanisms could be used to help LDP improve protection co= verage. Also, nothing would prevent LDP from using RSVP bypass tunnels. As = you can see, there may be many different approaches, and there may new appr= oaches coming in the future. In this PWE3 document, we just want to stay at= PW level and be generic with PSN level. In any case, this mechanism is ach= ievable with LDP, and LDP as a tunnel signaling protocol really doesn't nee= d any new extension. Hope this clarifies it. Thanks, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Thursday, August 07, 2014 10:37 AM To: Yimin Shen Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin hi, Please see some comments inline below. Regards, Sasha Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 From: Yimin Shen [mailto:yshen@juniper.net] Sent: Thursday, August 07, 2014 5:34 AM To: Alexander Vainshtein Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Hi Sasha, Thanks again for the comments. Please see the following 4 items in response= to the 4 comments in your email. 1. Today when an egress PE detects an egress AC failure [[Sasha]] Could you please explain how the egress PE detects an egress AC f= ailure? I have always thought that egress AC failure has to be detected fir= st by corresponding CE which would then notify the egress PE using some sui= table "Remote Failure"/BDI/RDI mechanism. Specific examples would really he= lp. it will simply have to drop packets, until the remote PE stops sending traf= fic over the PW. This draft is intended to improve this behavior. By no mea= ns the local repair on egress PE will prevent the AC failure from being con= verted to and handled as a bi-directional AC failure. Note that bi-directio= nal AC failure handling must involve the CE, but the CE may detect the fail= ure at a slower pace, or it may not be able to detect it in a timely manner= , depending on the actual situation. [[Sasha]] Please see above. IMHO and FWIW if the CE cannot detect its "Rece= ive Failure", the PE would not never know about the "Transmit Failure. E.g. there may be other L2 devices between the PE and CE; the AC may itself= be a logical wire across another network; the type and location of the fai= lure may make it easier for the PE to detect it; and so on. In any case, be= fore the AC failure is handled more gracefully (or bi-directionally), the e= gress traffic has been locally repaired by the egress PE. 2. As mentioned before, this draft doesn't have a dependency on a specific = PSN tunnel technology. [[Sasha]] I have provided my comments on this in my response to Eric. From = my point of view some LDP-specific issues still require clarification, but = my bottom line is that operating the draft over an MPLS network that uses L= DP in DU mode for setting up tunnel LSPs may sometimes be possible but woul= d always be tricky. This does not match my understanding of "no dependency = on a specific PSN tunnel technology", but this may be a matter of taste. 3. The draft already talks about scalability when describing the centralize= d protector model vs the co-located protector model. We can improve the tex= t to explicitly mention that the requirement for setting up LSPs on a per <= primary PE, protector> basis rather than per primary PE basis can potential= ly increase the number of LSPs in a network. This is a tradeoff. As mention= ed before, this does not necessarily cause a scaling issue. [[Sasha]] Yes, I have found the corresponding text in the draft. From my p= oint of view this issue is resolved. 4. We will leave this to PWE3 WG to decide whether this is really necessary= .[[Sasha]] I tend to agree with Eric that usage of context-specific labels = in the draft does not mean update to 4447. From my point of view this issue= is resolved as well. Thanks, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Wednesday, August 06, 2014 5:11 AM To: Yimin Shen Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin and all, I would like to summarize the issues that I have with this draft. Digging f= or them in a very long email thread with multiple levels of in-lining is no= t very effective IMO. 1. Protection against unidirectional AC failures looks problematic to= me: a. The draft explicitly differentiates between "ingress" and "egress"= AC failures b. The outcome can be using a different AC for each direction of CE-to= -CE traffic with potentially destructive effect that cannot be mitigated at= the PWE3 level c. IMHO and FWIW a viable alternative could be to employ native AC me= chanisms converting unidirectional AC failures into bi-directional ones. Th= is would eliminate the differentiation between ingress and egress AC failur= es and facilitate using the PW redundancy mechanism for bi-directional prot= ection 2. Dependency upon specific mechanisms for setting up PSN tunnels see= ms to break some common PWE3 architectural principles: a. I am pretty well sure that the mechanisms specified in the draft c= annot be used with the PSN tunnel LSPs set up by LDP in DU mode b. If the authors, refer to LDP in the DoD mode augmented with some CR= -LDP-like mechanisms (e.g., for exclusion of specific links when setting up= the bypass LSP between the PLR and the Protector), this should, at the ver= y least, be discussed with and approved by the MPLS WG 3. Scalability of the proposed solution should be, at least, explicit= ly presented. In particular, the draft assumes that a dedicated LSP would b= e used between PE1 PE2 for each combination of PE2 with a specific Protect= or LSR. 4. The Protector LSR could be either a kind of S-PE (if different fro= m the secondary T-PE) or the secondary T-PE itself. In both cases it should= be capable of handling labels from specific context spaces as if they were= PW labels. This requires (at the very least) an update of RFC 4447 and sho= uld be explicitly presented and discussed. Hopefully this summary will be useful. Regards, Sasha Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 --_000_808559abe80a4cf99368a16c33d1e066AM3PR03MB612eurprd03pro_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Yimin hi!

Lots of thanks for a p= rompt response.

 

 

The problem with split= ting the PW traffic is not limited to re0rdering.

 

In the simplest case, = consider the case when the PW s have been set between two IEEE802.1 bridges= that perform MAC learning.

Split traffic could ea= sily result in frequent re-learning of a certain MAC address…

 

 

The bottom line is tha= t if split traffic cannot be precluded, the dual-homed CE must be aware of = this fact and behave accordingly (e.g., by grouping a pair of ports that su= pport its connections to Primary PE and the Protector PE as a LAGJ).

 

From my point of view = these dependencies should be explicitly defined (say, in the Applicability = Statement). To the best of my recollection, this has been a long-establishe= d PWE3 WG policy from its early days.

Regards,

   &nbs= p;   Sasha

Email: Alexander.Vains= htein@ecitele.com

Mobile: 054-9266302

 

From: Yimin Sh= en [mailto:yshen@juniper.net]
Sent: Tuesday, August 12, 2014 6:26 PM
To: Alexander Vainshtein
Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org; erosen@cisco.c= om
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Hi Sasha,

 

Please see inline̷= 0;

 

 

Thanks,

 

/Yimin

 

 

From: Alexande= r Vainshtein [mailto:Al= exander.Vainshtein@ecitele.com]
Sent: Tuesday, August 12, 2014 12:15 AM
To: Yimin Shen
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org; erosen@cisco.com
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Yimin,

Lots of thanks for a prompt response.

 

My concerns related to ECMP have been mainly= about multiple potential PLRs on equal cost LSPs between the ingress = PE" and "Primary egress PE".

 

I do not see how this can be easily avoided = and my reading of your response is that you do not consider it either.=

 

[yshen] This draft doesn’t preclu= de or avoid it, as I explained below.

 

One of the aspects of multiple potential PLR= s has been raised by Stewart in his email: if the link between one of = these and the Primary PE fails while Primary PE remains OK, the traffic wou= ld be split.

 

[yshen] Packets of “one given PW&= #8221; may be load-balanced over ECMP to the primary PE, due to no-CW or FA= T label. In the above failure case, we only move the affected “branch of flow” over to a bypass tunnel. This shouldn’t cause any iss= ue for the CE, because either [1] the CE doesn’t care about packet or= der (in the case where the PW has no CW), or [2] packet order of the sub-fl= ow associated with a FAT label is still preserved.

 

Another aspect of this scenario is that= , in the case of the Primary PE failure, each of these multiple PLRs w= ould select the Protector as its best Next Hop to the Context ID.

 

[yshen] Yes, and the PLR should move pa= ckets from the bypass tunnel back to this LSP, which is now going over the = new “best next hop” to the protector.  From the protector&= #8217;s perspective, this LSP will actually be the same as the bypass tunnel, beca= use it’s associated with the context ID FEC and hence its label still= points to the context label table where PW label should be looked up. We c= an improve the text of the draft to explicitly clarify this.

 

Regards,

Sasha

From: Yimin Shen <yshen@juniper.net>
Sent: Monday, August 11, 2014 11:15 PM
To: Alexander Vainshtein
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org; erosen@cisco.com
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Hi Sasha,

 

By “avoiding ECM= P”, I was strictly talking about the context ID FEC from the PLR̵= 7;s perspective, rather than avoiding ECMP broadly for other FECs on the PL= R or any FECs on other routers.

 

Specifically, if the P= LR has a set of ECMP to the context ID FEC, all the ECMP paths must travers= e the primary PE only. This will ensure that all PWs be carried to the prim= ary PE. And this can be achieved by using the proxy node approach suggested in this draft. The path(s) travers= ing the protector to reach the context ID FEC can be used as bypass LSP(s).=

 

We can improve the tex= t on this to make it clearer.

 

Thanks,

 

/Yimin

 

 

From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com]
Sent: Sunday, August 10, 2014 1:15 AM
To: Yimin Shen
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org; erosen@cisco.com
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Yimin hi,

Lots of thanks for a prompt response.=

 <= o:p>

I must admit that I have missed the proposal= in Section 4.2.2 to treat "context ID" IP address as a proxy nod= e that is adjacent to both the Primary PE and the Protector NEs. (It also h= as not been mentioned in Eri=E7's clarification email). I think that this approach would require certain manipulation of b= oth the IGP and LDP, but this requires more thought on my side. One of the&= nbsp;issues that come to mind, is the need for LDP in both these = nodes to allocate a locally terminated label for the corresponding FEC while not advertising context IP address as one of t= he IP addresses it owns.

 <= o:p>

I think that your statement about "avoi= ding ECMP  scenario" is problematic, because, from my p= oint of view,  ECMP is one of the important characteristics of LD= P-based MPLS PSN.  And in any case the draft does not say anythin= g about that (or, at least neither ECMP nor "equal cost multipath&q= uot; is mentioned in the text).  If you see ECMP avoidance as a p= re-condition for using the mechanism defined in draft with LDP LSPs, this s= hould be quite explicit and detailed IMO.

 <= o:p>

I still fail to understand  what consti= tutes an egress AC failure in the draft and how it is different from the in= gress AC failure. Unfortunately the examples you've provided in your respon= se do not help much: e.g., a fiber cut close to  would be most probably recognized by the PE as a bi-directi= onal AC failure, while a fiber cut that is close to CE but remote from PE w= ould most probably not detected by the PE. (Of course if PE and CE are term= inating the same fiber span, the fiber cut would be recognized by both).

 <= o:p>

My guess (FWIW) that the draft actually disc= usses protection of traffic in the PE --> CE direction triggered by a fa= ilure detected by the PE while leaving protection of traffic in the CE --&g= t; PE direction some other mechanism that would be triggered by CE detecting the failure. This mechanism presumably = would be different from the PW redundancy mechanism because it would requir= e that an active-active scheme for Primary and Backup  PWs.

 <= o:p>

If this guess is correct (or close to realit= y), I would expect some explicit text explaining it. In particular, this te= xt should explain that "redundant PWs" mentioned in the draft are= not related to "PW redundancy". The current text makes a mix of these two notions, and this definitely does not help t= he reader (or at least this reader :-).

 <= o:p>

Hopefully these comments would be useful.

 <= o:p>

Regards,

Sasha

From: Yimin Shen <yshen@juniper.net>
Sent: Friday, August 8, 2014 5:34 PM
To: Alexander Vainshtein
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Hi Sasha,

 

Thanks again for the c= omments and questions!

 

Regarding AC failure d= etection on PE, in the typical cases, if there is a fiber cut, or a hardwar= e failure affecting the port on the PE, or a hardware failure affecting the= port on the CE (assume the CE is directed attached to the PE), the PE should be able to directly and independently d= etect the failure at physical or link layer.

 

Regarding LDP, I agree= with you that LDP by its nature is not as straight-forward as RSVP-TE in t= his scenario. Since a context ID are advertised by both primary PE and prot= ector, we should avoid creating an ECMP scenario. That’s why section 4.2.2 provides a suggestion to advertis= e the context ID as a proxy node (i.e. a virtual node attached to the two r= outers), and make its reachability via the primary PE more favorable than t= hat via the protector. MRT and other mechanisms could be used to help LDP improve protection coverage. Also, nothing would= prevent LDP from using RSVP bypass tunnels. As you can see, there may be m= any different approaches, and there may new approaches coming in the future= . In this PWE3 document, we just want to stay at PW level and be generic with PSN level. In any case, this = mechanism is achievable with LDP, and LDP as a tunnel signaling protocol re= ally doesn’t need any new extension.  

 

Hope this clarifies it= .

 

Thanks,

 

/Yimin

 

 

From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com]
Sent: Thursday, August 07, 2014 10:37 AM
To: Yimin Shen
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Yimin hi,

Please see some commen= ts inline below.

 

Regards,

   &nbs= p;   Sasha

Email: Alexander.Vainshtein@ecitele.com

Mobile: 054-9266302

 

From: Yimin Shen [mailto= :yshen@juniper.net]
Sent: Thursday, August 07, 2014 5:34 AM
To: Alexander Vainshtein
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Hi Sasha,

 

Thanks again for the c= omments. Please see the following 4 items in response to the 4 comments in = your email.

 

1. Today when an egres= s PE detects an egress AC failure=

 

[[Sasha]] Could = you please explain how the egress PE detects an egress AC failure? I have a= lways thought that egress AC failure has to be detected first by correspond= ing CE which would then notify the egress PE using some suitable “Remote Failure”/BDI/RDI mechanism. Spe= cific examples would really help.

 

it will simply have to= drop packets, until the remote PE stops sending traffic over the PW. This = draft is intended to improve this behavior. By no means the local repair on= egress PE will prevent the AC failure from being converted to and handled as a bi-directional AC failure. Note t= hat bi-directional AC failure handling must involve the CE, but the CE may = detect the failure at a slower pace, or it may not be able to detect it in = a timely manner, depending on the actual situation. <= /p>

[[Sasha]] Please= see above. IMHO and FWIW if the CE cannot detect its “Receive Failur= e”, the PE would not never know about the “Transmit Failure.

 

E.g. there may be othe= r L2 devices between the PE and CE; the AC may itself be a logical wire acr= oss another network; the type and location of the failure may make it easie= r for the PE to detect it; and so on. In any case, before the AC failure is handled more gracefully (or bi-direc= tionally), the egress traffic has been locally repaired by the egress PE.

 

2. As mentioned before= , this draft doesn’t have a dependency on a specific PSN tunnel techn= ology.

[[Sasha]] I have= provided my comments on this in my response to Eric. From my point of view= some LDP-specific issues still require clarification, but my bottom line i= s that operating the draft over an MPLS network = that uses LDP in DU mode for setting up tunnel LSPs may sometimes be possib= le but would always be tricky. This does not match my understanding = of “no dependency on a specific PSN tunnel technology”, but this may be a matter of taste.

 

3. The draft already t= alks about scalability when describing the centralized protector model vs t= he co-located protector model. We can improve the text to explicitly mentio= n that the requirement for setting up LSPs on a per <primary PE, protector> basis rather than per primary = PE basis can potentially increase the number of LSPs in a network. This is = a tradeoff. As mentioned before, this does not necessarily cause a scaling = issue.

[[Sasha]] Yes, I= have found the corresponding text in the draft.  From my point of vie= w this issue is resolved.=

 

4. We will leave this = to PWE3 WG to decide whether this is really necessary.[[Sasha]] I tend to agree with Eric that usage of con= text-specific labels in the draft does not mean update to 4447. From my point of view this issue is resolved as w= ell.

 

Thanks,

 

/Yimin

 

 

From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com]
Sent: Wednesday, August 06, 2014 5:11 AM
To: Yimin Shen
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Yimin and all,=

I would like to sum= marize the issues that I have with this draft. Digging for them in a very l= ong email thread with multiple levels of in-lining is not very effective IM= O.

 

= 1. &= nbsp;     Protection against unidirectional A= C failures looks problematic to me= :

= a. &= nbsp;     The draft explicitly differentiates be= tween “ingress” and “egress” AC failures=

= b. &= nbsp;    The outcome can be using a different A= C for each direction of CE-to-CE traffic with potentially destructive effec= t that cannot be mitigated at the PWE3 level

= c. &= nbsp;     IMHO and FWIW a viable alternative cou= ld be to employ native AC mechanisms converting unidirectional AC failures = into bi-directional ones. This would eliminate the differentiation between = ingress and egress AC failures and facilitate using the PW redundancy mechanism for bi-directional protection=

= 2. &= nbsp;     Dependency upon specific mechanisms= for setting up PSN tunnels seems = to break some common PWE3 architectural principles:

= a. &= nbsp;     I am pretty well sure that the mechani= sms specified in the draft cannot be used with the PSN tunnel LSPs set up b= y LDP in DU mode

= b. &= nbsp;    If the authors, refer to LDP in the DoD mode augmented with some CR-LDP-like mechanisms (e.g., for exclusi= on of specific links when setting up the bypass LSP between the PLR and the= Protector), this should, at the very least, be discussed with and approved by the MPLS WG

= 3. &= nbsp;     Scalability of the proposed solution should be, at least, explicitl= y presented. In particular, the draft assumes that a dedicated LSP would be= used between PE1  PE2 for each combination of PE2 with a specific Protector LSR.

= 4. &= nbsp;     The Protector LSR could be either a ki= nd of S-PE (if different from the secondary T-PE) or the secondary T-PE its= elf. In both cases it should be capable of handling labels from specific co= ntext spaces as if they were PW labels. This requires (at the very least) an update of RFC 4447 and should be expl= icitly presented and discussed.

Hopefully this summ= ary will be useful.

 

 

Regards,=

   &= nbsp;   Sasha

Email: Alexander.Vainshtein@ecitele.com

Mobile: 054-9266302=

--_000_808559abe80a4cf99368a16c33d1e066AM3PR03MB612eurprd03pro_-- From nobody Wed Aug 13 14:10:02 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 39DBD1A06FC for ; Wed, 13 Aug 2014 14:10:00 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.001 X-Spam-Level: X-Spam-Status: No, score=-2.001 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, J_CHICKENPOX_27=0.6, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id U6nI9yg85sQ9 for ; Wed, 13 Aug 2014 14:09:53 -0700 (PDT) Received: from na01-bl2-obe.outbound.protection.outlook.com (mail-bl2lp0209.outbound.protection.outlook.com [207.46.163.209]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8672C1A070B for ; Wed, 13 Aug 2014 14:09:49 -0700 (PDT) Received: from BY2PR05MB728.namprd05.prod.outlook.com (10.141.223.25) by BY2PR05MB223.namprd05.prod.outlook.com (10.242.41.141) with Microsoft SMTP Server (TLS) id 15.0.1005.10; Wed, 13 Aug 2014 21:09:45 +0000 Received: from BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) by BY2PR05MB728.namprd05.prod.outlook.com ([10.141.223.25]) with mapi id 15.00.1005.008; Wed, 13 Aug 2014 21:09:45 +0000 From: Yimin Shen To: Alexander Vainshtein Thread-Topic: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 Thread-Index: AQHPqByPzRsIO8QlbkK+kbMBZ9MDf5u3AuAAgAAZ0oCAABjewIABFRcAgADf/TCACF0AgIAA3TgQgADA4ICAADbOAIAAzcgwgAEfc4CAAYjQ4IACkWQAgAKGg+CAAI1LAIAArafggAAScQCAAezqUA== Date: Wed, 13 Aug 2014 21:09:44 +0000 Message-ID: References: <201407251524.s6PFOPn92012@magenta.juniper.net> <53D79504.4050904@cisco.com> <04f15bedf2be4c7480d3e9ca01bdfd7f@BY2PR05MB728.namprd05.prod.outlook.com> <2597291f7b074922b690e4b06999cf1a@AM3PR03MB612.eurprd03.prod.outlook.com>, <30e13900814f41a681151971cbe9ebef@BY2PR05MB728.namprd05.prod.outlook.com> <1407215578702.64387@ecitele.com>, <2280b8e0d5f94a60badab91a2237181b@BY2PR05MB728.namprd05.prod.outlook.com> <1407304503657.72312@ecitele.com> <046050cb2d254a139b4650b2d7f93075@BY2PR05MB728.namprd05.prod.outlook.com> <09ab595ab24a46a0a4e87ebeb2cd185a@AM3PR03MB612.eurprd03.prod.outlook.com>, <2afb5c963e054aef929f38dcb3defa07@BY2PR05MB728.namprd05.prod.outlook.com> <1407647722694.18724@ecitele.com>, <6b0fa5df117f43d69c31efab88161e70@BY2PR05MB728.namprd05.prod.outlook.com> <1407816900919.37823@ecitele.com> <808559abe80a4cf99368a16c33d1e066@AM3PR03MB612.eurprd03.prod.outlook.com> In-Reply-To: <808559abe80a4cf99368a16c33d1e066@AM3PR03MB612.eurprd03.prod.outlook.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [66.129.241.10] x-microsoft-antispam: BCL:0;PCL:0;RULEID:;UriScan:; x-forefront-prvs: 0302D4F392 x-forefront-antispam-report: SFV:NSPM; SFS:(252514010)(199003)(51914003)(164054003)(189002)(51444003)(37854004)(43784003)(377454003)(45624003)(19580405001)(46102001)(33646002)(99286002)(54356999)(87936001)(2656002)(575784001)(15202345003)(85852003)(86362001)(110136001)(19625215002)(95666004)(76576001)(19300405004)(77982001)(108616004)(4396001)(79102001)(83322001)(16236675004)(21056001)(74316001)(19580395003)(76176999)(85306004)(50986999)(101416001)(15975445006)(18717965001)(20776003)(66066001)(80022001)(107046002)(99396002)(83072002)(106356001)(561944003)(105586002)(106116001)(76482001)(81342001)(74502001)(81542001)(74662001)(24736002)(579004); DIR:OUT; SFP:; SCL:1; SRVR:BY2PR05MB223; H:BY2PR05MB728.namprd05.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Content-Type: multipart/alternative; boundary="_000_ca3322b9794e43c299525ea59c517e6aBY2PR05MB728namprd05pro_" MIME-Version: 1.0 X-OriginatorOrg: juniper.net Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/zOjOD9c9CnHjaKuNK6mg4qP1HwI Cc: Yakov Rekhter , "erosen@cisco.com" , "pwe3@ietf.org" , "stbryant@cisco.com" Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01 X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Aug 2014 21:10:00 -0000 --_000_ca3322b9794e43c299525ea59c517e6aBY2PR05MB728namprd05pro_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi Sasha, Thanks for these comments! I agree with your points on these scenarios. Wil= l describe and clarify them in the next rev. Thanks, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Tuesday, August 12, 2014 11:43 AM To: Yimin Shen Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org; erosen@cisco.com Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin hi! Lots of thanks for a prompt response. The problem with splitting the PW traffic is not limited to re0rdering. In the simplest case, consider the case when the PW s have been set between= two IEEE802.1 bridges that perform MAC learning. Split traffic could easily result in frequent re-learning of a certain MAC = address... The bottom line is that if split traffic cannot be precluded, the dual-home= d CE must be aware of this fact and behave accordingly (e.g., by grouping a= pair of ports that support its connections to Primary PE and the Protector= PE as a LAG:)). >From my point of view these dependencies should be explicitly defined (say,= in the Applicability Statement). To the best of my recollection, this has = been a long-established PWE3 WG policy from its early days. Regards, Sasha Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 From: Yimin Shen [mailto:yshen@juniper.net] Sent: Tuesday, August 12, 2014 6:26 PM To: Alexander Vainshtein Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org; erosen@cisco.com Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Hi Sasha, Please see inline... Thanks, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Tuesday, August 12, 2014 12:15 AM To: Yimin Shen Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org; erosen@cisco.com Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin, Lots of thanks for a prompt response. My concerns related to ECMP have been mainly about multiple potential PLRs = on equal cost LSPs between the ingress PE" and "Primary egress PE". I do not see how this can be easily avoided and my reading of your response= is that you do not consider it either. [yshen] This draft doesn't preclude or avoid it, as I explained below. One of the aspects of multiple potential PLRs has been raised by Stewart in= his email: if the link between one of these and the Primary PE fails while= Primary PE remains OK, the traffic would be split. [yshen] Packets of "one given PW" may be load-balanced over ECMP to the pri= mary PE, due to no-CW or FAT label. In the above failure case, we only move= the affected "branch of flow" over to a bypass tunnel. This shouldn't caus= e any issue for the CE, because either [1] the CE doesn't care about packet= order (in the case where the PW has no CW), or [2] packet order of the sub= -flow associated with a FAT label is still preserved. Another aspect of this scenario is that, in the case of the Primary PE fail= ure, each of these multiple PLRs would select the Protector as its best Nex= t Hop to the Context ID. [yshen] Yes, and the PLR should move packets from the bypass tunnel back to= this LSP, which is now going over the new "best next hop" to the protector= . From the protector's perspective, this LSP will actually be the same as = the bypass tunnel, because it's associated with the context ID FEC and henc= e its label still points to the context label table where PW label should b= e looked up. We can improve the text of the draft to explicitly clarify thi= s. Regards, Sasha ________________________________ From: Yimin Shen > Sent: Monday, August 11, 2014 11:15 PM To: Alexander Vainshtein Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org; erosen@cisco.com Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Hi Sasha, By "avoiding ECMP", I was strictly talking about the context ID FEC from th= e PLR's perspective, rather than avoiding ECMP broadly for other FECs on th= e PLR or any FECs on other routers. Specifically, if the PLR has a set of ECMP to the context ID FEC, all the E= CMP paths must traverse the primary PE only. This will ensure that all PWs = be carried to the primary PE. And this can be achieved by using the proxy n= ode approach suggested in this draft. The path(s) traversing the protector = to reach the context ID FEC can be used as bypass LSP(s). We can improve the text on this to make it clearer. Thanks, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Sunday, August 10, 2014 1:15 AM To: Yimin Shen Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org; erosen@cisco.com Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin hi, Lots of thanks for a prompt response. I must admit that I have missed the proposal in Section 4.2.2 to treat "con= text ID" IP address as a proxy node that is adjacent to both the Primary PE= and the Protector NEs. (It also has not been mentioned in Eri=E7's clarifi= cation email). I think that this approach would require certain manipulatio= n of both the IGP and LDP, but this requires more thought on my side. One o= f the issues that come to mind, is the need for LDP in both these nodes to = allocate a locally terminated label for the corresponding FEC while not adv= ertising context IP address as one of the IP addresses it owns. I think that your statement about "avoiding ECMP scenario" is problematic,= because, from my point of view, ECMP is one of the important characterist= ics of LDP-based MPLS PSN. And in any case the draft does not say anything= about that (or, at least neither ECMP nor "equal cost multipath" is mentio= ned in the text). If you see ECMP avoidance as a pre-condition for using t= he mechanism defined in draft with LDP LSPs, this should be quite explicit = and detailed IMO. I still fail to understand what constitutes an egress AC failure in the dr= aft and how it is different from the ingress AC failure. Unfortunately the = examples you've provided in your response do not help much: e.g., a fiber c= ut close to would be most probably recognized by the PE as a bi-directiona= l AC failure, while a fiber cut that is close to CE but remote from PE woul= d most probably not detected by the PE. (Of course if PE and CE are termina= ting the same fiber span, the fiber cut would be recognized by both). My guess (FWIW) that the draft actually discusses protection of traffic in = the PE --> CE direction triggered by a failure detected by the PE while lea= ving protection of traffic in the CE --> PE direction some other mechanism = that would be triggered by CE detecting the failure. This mechanism presuma= bly would be different from the PW redundancy mechanism because it would re= quire that an active-active scheme for Primary and Backup PWs. If this guess is correct (or close to reality), I would expect some explici= t text explaining it. In particular, this text should explain that "redunda= nt PWs" mentioned in the draft are not related to "PW redundancy". The curr= ent text makes a mix of these two notions, and this definitely does not hel= p the reader (or at least this reader :-). Hopefully these comments would be useful. Regards, Sasha ________________________________ From: Yimin Shen > Sent: Friday, August 8, 2014 5:34 PM To: Alexander Vainshtein Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Hi Sasha, Thanks again for the comments and questions! Regarding AC failure detection on PE, in the typical cases, if there is a f= iber cut, or a hardware failure affecting the port on the PE, or a hardware= failure affecting the port on the CE (assume the CE is directed attached t= o the PE), the PE should be able to directly and independently detect the f= ailure at physical or link layer. Regarding LDP, I agree with you that LDP by its nature is not as straight-f= orward as RSVP-TE in this scenario. Since a context ID are advertised by bo= th primary PE and protector, we should avoid creating an ECMP scenario. Tha= t's why section 4.2.2 provides a suggestion to advertise the context ID as = a proxy node (i.e. a virtual node attached to the two routers), and make it= s reachability via the primary PE more favorable than that via the protecto= r. MRT and other mechanisms could be used to help LDP improve protection co= verage. Also, nothing would prevent LDP from using RSVP bypass tunnels. As = you can see, there may be many different approaches, and there may new appr= oaches coming in the future. In this PWE3 document, we just want to stay at= PW level and be generic with PSN level. In any case, this mechanism is ach= ievable with LDP, and LDP as a tunnel signaling protocol really doesn't nee= d any new extension. Hope this clarifies it. Thanks, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Thursday, August 07, 2014 10:37 AM To: Yimin Shen Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin hi, Please see some comments inline below. Regards, Sasha Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 From: Yimin Shen [mailto:yshen@juniper.net] Sent: Thursday, August 07, 2014 5:34 AM To: Alexander Vainshtein Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Hi Sasha, Thanks again for the comments. Please see the following 4 items in response= to the 4 comments in your email. 1. Today when an egress PE detects an egress AC failure [[Sasha]] Could you please explain how the egress PE detects an egress AC f= ailure? I have always thought that egress AC failure has to be detected fir= st by corresponding CE which would then notify the egress PE using some sui= table "Remote Failure"/BDI/RDI mechanism. Specific examples would really he= lp. it will simply have to drop packets, until the remote PE stops sending traf= fic over the PW. This draft is intended to improve this behavior. By no mea= ns the local repair on egress PE will prevent the AC failure from being con= verted to and handled as a bi-directional AC failure. Note that bi-directio= nal AC failure handling must involve the CE, but the CE may detect the fail= ure at a slower pace, or it may not be able to detect it in a timely manner= , depending on the actual situation. [[Sasha]] Please see above. IMHO and FWIW if the CE cannot detect its "Rece= ive Failure", the PE would not never know about the "Transmit Failure. E.g. there may be other L2 devices between the PE and CE; the AC may itself= be a logical wire across another network; the type and location of the fai= lure may make it easier for the PE to detect it; and so on. In any case, be= fore the AC failure is handled more gracefully (or bi-directionally), the e= gress traffic has been locally repaired by the egress PE. 2. As mentioned before, this draft doesn't have a dependency on a specific = PSN tunnel technology. [[Sasha]] I have provided my comments on this in my response to Eric. From = my point of view some LDP-specific issues still require clarification, but = my bottom line is that operating the draft over an MPLS network that uses L= DP in DU mode for setting up tunnel LSPs may sometimes be possible but woul= d always be tricky. This does not match my understanding of "no dependency = on a specific PSN tunnel technology", but this may be a matter of taste. 3. The draft already talks about scalability when describing the centralize= d protector model vs the co-located protector model. We can improve the tex= t to explicitly mention that the requirement for setting up LSPs on a per <= primary PE, protector> basis rather than per primary PE basis can potential= ly increase the number of LSPs in a network. This is a tradeoff. As mention= ed before, this does not necessarily cause a scaling issue. [[Sasha]] Yes, I have found the corresponding text in the draft. From my p= oint of view this issue is resolved. 4. We will leave this to PWE3 WG to decide whether this is really necessary= .[[Sasha]] I tend to agree with Eric that usage of context-specific labels = in the draft does not mean update to 4447. From my point of view this issue= is resolved as well. Thanks, /Yimin From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com] Sent: Wednesday, August 06, 2014 5:11 AM To: Yimin Shen Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf= .org Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protecti= on-01 Yimin and all, I would like to summarize the issues that I have with this draft. Digging f= or them in a very long email thread with multiple levels of in-lining is no= t very effective IMO. 1. Protection against unidirectional AC failures looks problematic to= me: a. The draft explicitly differentiates between "ingress" and "egress"= AC failures b. The outcome can be using a different AC for each direction of CE-to= -CE traffic with potentially destructive effect that cannot be mitigated at= the PWE3 level c. IMHO and FWIW a viable alternative could be to employ native AC me= chanisms converting unidirectional AC failures into bi-directional ones. Th= is would eliminate the differentiation between ingress and egress AC failur= es and facilitate using the PW redundancy mechanism for bi-directional prot= ection 2. Dependency upon specific mechanisms for setting up PSN tunnels see= ms to break some common PWE3 architectural principles: a. I am pretty well sure that the mechanisms specified in the draft c= annot be used with the PSN tunnel LSPs set up by LDP in DU mode b. If the authors, refer to LDP in the DoD mode augmented with some CR= -LDP-like mechanisms (e.g., for exclusion of specific links when setting up= the bypass LSP between the PLR and the Protector), this should, at the ver= y least, be discussed with and approved by the MPLS WG 3. Scalability of the proposed solution should be, at least, explicit= ly presented. In particular, the draft assumes that a dedicated LSP would b= e used between PE1 PE2 for each combination of PE2 with a specific Protect= or LSR. 4. The Protector LSR could be either a kind of S-PE (if different fro= m the secondary T-PE) or the secondary T-PE itself. In both cases it should= be capable of handling labels from specific context spaces as if they were= PW labels. This requires (at the very least) an update of RFC 4447 and sho= uld be explicitly presented and discussed. Hopefully this summary will be useful. Regards, Sasha Email: Alexander.Vainshtein@ecitele.com Mobile: 054-9266302 --_000_ca3322b9794e43c299525ea59c517e6aBY2PR05MB728namprd05pro_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Hi Sasha,

 

Thanks for these comme= nts! I agree with your points on these scenarios. Will describe and clarify= them in the next rev.

 

 

Thanks,

 

/Yimin

 

 

From: Alexande= r Vainshtein [mailto:Alexander.Vainshtein@ecitele.com]
Sent: Tuesday, August 12, 2014 11:43 AM
To: Yimin Shen
Cc: stbryant@cisco.com; Yakov Rekhter; pwe3@ietf.org; erosen@cisco.c= om
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Yimin hi!

Lots of thanks for a p= rompt response.

 

 

The problem with split= ting the PW traffic is not limited to re0rdering.

 

In the simplest case, = consider the case when the PW s have been set between two IEEE802.1 bridges= that perform MAC learning.

Split traffic could ea= sily result in frequent re-learning of a certain MAC address…

 

 

The bottom line is tha= t if split traffic cannot be precluded, the dual-homed CE must be aware of = this fact and behave accordingly (e.g., by grouping a pair of ports that su= pport its connections to Primary PE and the Protector PE as a LAGJ).

 

From my point of view = these dependencies should be explicitly defined (say, in the Applicability = Statement). To the best of my recollection, this has been a long-establishe= d PWE3 WG policy from its early days.

 

Regards,

   &nbs= p;   Sasha

Email: Alexander.Vainshtein@ecitele.com

Mobile: 054-9266302

 

From: Yimin Sh= en [mailto:yshen@juniper.net]
Sent: Tuesday, August 12, 2014 6:26 PM
To: Alexander Vainshtein
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org; erosen@cisco.com
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Hi Sasha,

 

Please see inline̷= 0;

 

 

Thanks,

 

/Yimin

 

 

From: Alexande= r Vainshtein [mailto:Al= exander.Vainshtein@ecitele.com]
Sent: Tuesday, August 12, 2014 12:15 AM
To: Yimin Shen
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org; erosen@cisco.com
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Yimin,

Lots of thanks for a prompt response.

 

My concerns related to ECMP have been mainly= about multiple potential PLRs on equal cost LSPs between the ingress = PE" and "Primary egress PE".

 

I do not see how this can be easily avoided = and my reading of your response is that you do not consider it either.=

 

[yshen] This draft doesn’t preclu= de or avoid it, as I explained below.

 

One of the aspects of multiple potential PLR= s has been raised by Stewart in his email: if the link between one of = these and the Primary PE fails while Primary PE remains OK, the traffic wou= ld be split.

 

[yshen] Packets of “one given PW&= #8221; may be load-balanced over ECMP to the primary PE, due to no-CW or FA= T label. In the above failure case, we only move the affected “branch of flow” over to a bypass tunnel. This shouldn’t cause any iss= ue for the CE, because either [1] the CE doesn’t care about packet or= der (in the case where the PW has no CW), or [2] packet order of the sub-fl= ow associated with a FAT label is still preserved.

 

Another aspect of this scenario is that= , in the case of the Primary PE failure, each of these multiple PLRs w= ould select the Protector as its best Next Hop to the Context ID.

 

[yshen] Yes, and the PLR should move pa= ckets from the bypass tunnel back to this LSP, which is now going over the = new “best next hop” to the protector.  From the protector&= #8217;s perspective, this LSP will actually be the same as the bypass tunnel, beca= use it’s associated with the context ID FEC and hence its label still= points to the context label table where PW label should be looked up. We c= an improve the text of the draft to explicitly clarify this.

 

Regards,

Sasha

From: Yimin Shen <yshen@juniper.net>
Sent: Monday, August 11, 2014 11:15 PM
To: Alexander Vainshtein
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org; erosen@cisco.com
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Hi Sasha,

 

By “avoiding ECM= P”, I was strictly talking about the context ID FEC from the PLR̵= 7;s perspective, rather than avoiding ECMP broadly for other FECs on the PL= R or any FECs on other routers.

 

Specifically, if the P= LR has a set of ECMP to the context ID FEC, all the ECMP paths must travers= e the primary PE only. This will ensure that all PWs be carried to the prim= ary PE. And this can be achieved by using the proxy node approach suggested in this draft. The path(s) travers= ing the protector to reach the context ID FEC can be used as bypass LSP(s).=

 

We can improve the tex= t on this to make it clearer.

 

Thanks,

 

/Yimin

 

 

From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com]
Sent: Sunday, August 10, 2014 1:15 AM
To: Yimin Shen
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org; erosen@cisco.com
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Yimin hi,

Lots of thanks for a prompt response.=

 <= o:p>

I must admit that I have missed the proposal= in Section 4.2.2 to treat "context ID" IP address as a proxy nod= e that is adjacent to both the Primary PE and the Protector NEs. (It also h= as not been mentioned in Eri=E7's clarification email). I think that this approach would require certain manipulation of b= oth the IGP and LDP, but this requires more thought on my side. One of the&= nbsp;issues that come to mind, is the need for LDP in both these = nodes to allocate a locally terminated label for the corresponding FEC while not advertising context IP address as one of t= he IP addresses it owns.

 <= o:p>

I think that your statement about "avoi= ding ECMP  scenario" is problematic, because, from my p= oint of view,  ECMP is one of the important characteristics of LD= P-based MPLS PSN.  And in any case the draft does not say anythin= g about that (or, at least neither ECMP nor "equal cost multipath&q= uot; is mentioned in the text).  If you see ECMP avoidance as a p= re-condition for using the mechanism defined in draft with LDP LSPs, this s= hould be quite explicit and detailed IMO.

 <= o:p>

I still fail to understand  what consti= tutes an egress AC failure in the draft and how it is different from the in= gress AC failure. Unfortunately the examples you've provided in your respon= se do not help much: e.g., a fiber cut close to  would be most probably recognized by the PE as a bi-directi= onal AC failure, while a fiber cut that is close to CE but remote from PE w= ould most probably not detected by the PE. (Of course if PE and CE are term= inating the same fiber span, the fiber cut would be recognized by both).

 <= o:p>

My guess (FWIW) that the draft actually disc= usses protection of traffic in the PE --> CE direction triggered by a fa= ilure detected by the PE while leaving protection of traffic in the CE --&g= t; PE direction some other mechanism that would be triggered by CE detecting the failure. This mechanism presumably = would be different from the PW redundancy mechanism because it would requir= e that an active-active scheme for Primary and Backup  PWs.

 <= o:p>

If this guess is correct (or close to realit= y), I would expect some explicit text explaining it. In particular, this te= xt should explain that "redundant PWs" mentioned in the draft are= not related to "PW redundancy". The current text makes a mix of these two notions, and this definitely does not help t= he reader (or at least this reader :-).

 <= o:p>

Hopefully these comments would be useful.

 <= o:p>

Regards,

Sasha

From: Yimin Shen <yshen@juniper.net>
Sent: Friday, August 8, 2014 5:34 PM
To: Alexander Vainshtein
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Hi Sasha,

 

Thanks again for the c= omments and questions!

 

Regarding AC failure d= etection on PE, in the typical cases, if there is a fiber cut, or a hardwar= e failure affecting the port on the PE, or a hardware failure affecting the= port on the CE (assume the CE is directed attached to the PE), the PE should be able to directly and independently d= etect the failure at physical or link layer.

 

Regarding LDP, I agree= with you that LDP by its nature is not as straight-forward as RSVP-TE in t= his scenario. Since a context ID are advertised by both primary PE and prot= ector, we should avoid creating an ECMP scenario. That’s why section 4.2.2 provides a suggestion to advertis= e the context ID as a proxy node (i.e. a virtual node attached to the two r= outers), and make its reachability via the primary PE more favorable than t= hat via the protector. MRT and other mechanisms could be used to help LDP improve protection coverage. Also, nothing would= prevent LDP from using RSVP bypass tunnels. As you can see, there may be m= any different approaches, and there may new approaches coming in the future= . In this PWE3 document, we just want to stay at PW level and be generic with PSN level. In any case, this = mechanism is achievable with LDP, and LDP as a tunnel signaling protocol re= ally doesn’t need any new extension.  

 

Hope this clarifies it= .

 

Thanks,

 

/Yimin

 

 

From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com]
Sent: Thursday, August 07, 2014 10:37 AM
To: Yimin Shen
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Yimin hi,

Please see some commen= ts inline below.

 

Regards,

   &nbs= p;   Sasha

Email: Alexander.Vainshtein@ecitele.com

Mobile: 054-9266302

 

From: Yimin Shen [mailto= :yshen@juniper.net]
Sent: Thursday, August 07, 2014 5:34 AM
To: Alexander Vainshtein
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Hi Sasha,

 

Thanks again for the c= omments. Please see the following 4 items in response to the 4 comments in = your email.

 

1. Today when an egres= s PE detects an egress AC failure=

 

[[Sasha]] Could = you please explain how the egress PE detects an egress AC failure? I have a= lways thought that egress AC failure has to be detected first by correspond= ing CE which would then notify the egress PE using some suitable “Remote Failure”/BDI/RDI mechanism. Spe= cific examples would really help.

 

it will simply have to= drop packets, until the remote PE stops sending traffic over the PW. This = draft is intended to improve this behavior. By no means the local repair on= egress PE will prevent the AC failure from being converted to and handled as a bi-directional AC failure. Note t= hat bi-directional AC failure handling must involve the CE, but the CE may = detect the failure at a slower pace, or it may not be able to detect it in = a timely manner, depending on the actual situation. <= /p>

[[Sasha]] Please= see above. IMHO and FWIW if the CE cannot detect its “Receive Failur= e”, the PE would not never know about the “Transmit Failure.

 

E.g. there may be othe= r L2 devices between the PE and CE; the AC may itself be a logical wire acr= oss another network; the type and location of the failure may make it easie= r for the PE to detect it; and so on. In any case, before the AC failure is handled more gracefully (or bi-direc= tionally), the egress traffic has been locally repaired by the egress PE.

 

2. As mentioned before= , this draft doesn’t have a dependency on a specific PSN tunnel techn= ology.

[[Sasha]] I have= provided my comments on this in my response to Eric. From my point of view= some LDP-specific issues still require clarification, but my bottom line i= s that operating the draft over an MPLS network = that uses LDP in DU mode for setting up tunnel LSPs may sometimes be possib= le but would always be tricky. This does not match my understanding = of “no dependency on a specific PSN tunnel technology”, but this may be a matter of taste.

 

3. The draft already t= alks about scalability when describing the centralized protector model vs t= he co-located protector model. We can improve the text to explicitly mentio= n that the requirement for setting up LSPs on a per <primary PE, protector> basis rather than per primary = PE basis can potentially increase the number of LSPs in a network. This is = a tradeoff. As mentioned before, this does not necessarily cause a scaling = issue.

[[Sasha]] Yes, I= have found the corresponding text in the draft.  From my point of vie= w this issue is resolved.=

 

4. We will leave this = to PWE3 WG to decide whether this is really necessary.[[Sasha]] I tend to agree with Eric that usage of con= text-specific labels in the draft does not mean update to 4447. From my point of view this issue is resolved as w= ell.

 

Thanks,

 

/Yimin

 

 

From: Alexander Vainshtein [mailto:Alexander.Vainshtein@ecitele.com]
Sent: Wednesday, August 06, 2014 5:11 AM
To: Yimin Shen
Cc: stbryant@cisco.com; Ya= kov Rekhter; pwe3@ietf.org
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-p= rotection-01

 

Yimin and all,=

I would like to sum= marize the issues that I have with this draft. Digging for them in a very l= ong email thread with multiple levels of in-lining is not very effective IM= O.

 

1. &nbs= p;     Protection against unidirectional A= C failures looks problematic to me= :

a. &nb= sp;     The draft explicitly differentiates be= tween “ingress” and “egress” AC failures=

b. &nb= sp;    The outcome can be using a different A= C for each direction of CE-to-CE traffic with potentially destructive effec= t that cannot be mitigated at the PWE3 level

c. &nb= sp;     IMHO and FWIW a viable alternative cou= ld be to employ native AC mechanisms converting unidirectional AC failures = into bi-directional ones. This would eliminate the differentiation between = ingress and egress AC failures and facilitate using the PW redundancy mechanism for bi-directional protection=

2. &nbs= p;     Dependency upon specific mechanisms= for setting up PSN tunnels seems = to break some common PWE3 architectural principles:

a. &nb= sp;     I am pretty well sure that the mechani= sms specified in the draft cannot be used with the PSN tunnel LSPs set up b= y LDP in DU mode

b. &nb= sp;    If the authors, refer to LDP in the DoD mode augmented with some CR-LDP-like mechanisms (e.g., for exclusi= on of specific links when setting up the bypass LSP between the PLR and the= Protector), this should, at the very least, be discussed with and approved by the MPLS WG

3. &nbs= p;     Scalability of the proposed solution should be, at least, explicitl= y presented. In particular, the draft assumes that a dedicated LSP would be= used between PE1  PE2 for each combination of PE2 with a specific Protector LSR.

4. &nbs= p;     The Protector LSR could be either a ki= nd of S-PE (if different from the secondary T-PE) or the secondary T-PE its= elf. In both cases it should be capable of handling labels from specific co= ntext spaces as if they were PW labels. This requires (at the very least) an update of RFC 4447 and should be expl= icitly presented and discussed.

Hopefully this summ= ary will be useful.

 

 

Regards,=

   &= nbsp;   Sasha

Email: Alexander.Vainshtein@ecitele.com

Mobile: 054-9266302=

--_000_ca3322b9794e43c299525ea59c517e6aBY2PR05MB728namprd05pro_-- From nobody Thu Aug 14 10:02:01 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4A0751A6F44 for ; Thu, 14 Aug 2014 10:01:59 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.268 X-Spam-Level: X-Spam-Status: No, score=-5.268 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, GB_I_INVITATION=-2, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-0.668, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vSfqRsigwa5C for ; Thu, 14 Aug 2014 10:01:52 -0700 (PDT) Received: from hubrelay-by-03.bt.com (hubrelay-by-03.bt.com [62.7.242.139]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 363421A093D for ; Thu, 14 Aug 2014 10:01:50 -0700 (PDT) Received: from EVMHR01-UKBR.domain1.systemhost.net (193.113.108.40) by EVMHR03-UKBR.bt.com (10.216.161.35) with Microsoft SMTP Server (TLS) id 8.3.348.2; Thu, 14 Aug 2014 18:01:47 +0100 Received: from EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) by EVMHR01-UKBR.domain1.systemhost.net (193.113.108.40) with Microsoft SMTP Server (TLS) id 8.3.348.2; Thu, 14 Aug 2014 18:01:47 +0100 Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) with Microsoft SMTP Server id 14.3.181.6; Thu, 14 Aug 2014 18:01:47 +0100 Received: from BTP075694.jungle.bt.co.uk ([10.109.26.130]) by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id s7EH1hbH000406; Thu, 14 Aug 2014 18:01:43 +0100 Message-ID: <201408141701.s7EH1hbH000406@bagheera.jungle.bt.co.uk> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Thu, 14 Aug 2014 18:01:41 +0100 To: Yaakov Stein From: Bob Briscoe In-Reply-To: <7.1.0.9.2.20140814120558.108e7b30@bt.com> References: <8D3D17ACE214DC429325B2B98F3AE712077B9517F4@MX15A.corp.emc.com> <8D3D17ACE214DC429325B2B98F3AE712077B951928@MX15A.corp.emc.com> <7.1.0.9.2.20140814120558.108e7b30@bt.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="=====================_794198531==.ALT" X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158 Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/AGM0P2GhRi1n4MvSSXtRl6uD7ug Cc: "pwe3@ietf.org" , "Andrew G. Malis" Subject: Re: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Aug 2014 17:01:59 -0000 --=====================_794198531==.ALT Content-Type: text/plain; charset="iso-8859-1"; format=flowed Content-Transfer-Encoding: quoted-printable Yaakov, This time adding pointer to all my nits and=20 including the list, which I hadn't noticed wasn't=20 cc'd when I replied to all. More inline within my own previous email.. At 14:02 14/08/2014, Bob Briscoe wrote: >To: Yaakov Stein >From: Bob Briscoe >Subject: RE: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt >Cc: "Andrew G. Malis" , BOCCI=20 >Matthew ,=20 >"Black, David" > >Yaakov, > >I have just re-read it thoroughly. I have=20 >noticed about two dozen editorial nits where=20 >correcting them would not change the intended=20 >sense (altho sometimes the sense needs to change=20 >to say what we intended!). Examples: >- Unexpanded abbreviations (David's suggestion would help, but there are= more) >- Missing words, tautologies, random punctuation problems etc >- Uses of the wrong word, even tho we know what you mean >- The caveat about dynamics vs steady state=20 >needs to shift down a para, so that it follows=20 >into the para that says what to do about dynamics. > >Should I take the token, and edit the XML=20 >directly, then you can see a diff to check if=20 >you want to accept my suggestions? I've provided xml source of my suggested edits (and a diff) here: I'm not implying that this is now the master=20 version - it's just an easy way to describe edits=20 (and you can use it as the basis for a new master=20 if you want). Similarly to David, pls take my=20 comments as co-author into consideration in parallel with the WGLC= responses. I couldn't alter the figures. I would ask for the following changes: * All Figs: s/PLR/PLP/ * Figs 1-4: - s/delay/one-way delay. * Fig 1: max at 10 not 50 on y axis * Figs 5-8: - s/safe/undeniably safe/ * Figs 7-8: max at 0.5% on x axis * Fig 9: s/Pd/PLP/ I incorporated those nits I could see on the list=20 (Sasha, Stewart & David), some of which I had noticed myself. >Below are six thoughts that go slightly beyond=20 >'editorial'. 1-4 could be considered=20 >'editorial'. But 5&6 are the more significant "techno-political" points. Suggested resolutions of these six bigger issues=20 are now also in my xml suggestion above too. >1. The phrase "mechanisms for provisioning PWs=20 >in service provider networks" is too opaque (the=20 >PWE audience might understand this scoping=20 >sentence, but a L4 reader might not understand=20 >what we're excluding here, which could make them suspicious). >CURRENT: > Mechanisms for provisioning PWs > in service provider networks are well understood and will not be > discussed further here. >SUGGESTED: > The case where the service provider network=20 > arranges sufficient capacity for a PW is well=20 > understood and will not be discussed further=20 > here. The concern is over PWs deployed=20 > independently of the service provider network's=20 > traffic engineering or capacity planning. > >2. The justification for using OWD of a few milliseconds misses the point. >" The one-way delay > of a native TDM service consists of the physical time-of-flight plus > 125 microseconds for each TDM switch traversed; and is thus very > small as compared to typical PSN network-crossing latencies. >" >The 125us per switch is irrelevant. The point=20 >that needs making is that PWs are generally=20 >deployed with a short physical time of flight. > >3. Related point: the captions on the graphs do=20 >not clearly indicate that they are OWD, not RTT.=20 >This could be clarified in the text, or on the graphs. > >4. Change packet loss rate (and PLR) to packet=20 >loss probability throughout, including on the=20 >graph axes. PLR is an ambiguous term, even=20 >though it is widely used in the industry. Packet=20 >loss rate could be measured in packet per=20 >second, whereas packet loss probability is=20 >unambiguously dimensionless. Indeed, in the=20 >appendices, packet loss rates with respect to time are calculated. > >5. Rather than David's "reasonable" I suggest we=20 >use the phrase "unambiguously safe" not just=20 >"safe" for the headings of Figs 5-8 and in the=20 >text, as below. Otherwise, we are implying that=20 >it is unsafe outside the region. I thought of a better phrase: "undeniably safe"=20 instead of "unambiguously safe". I slightly prefer this over these other possibilities: - unarguably safe - indisputably safe - unconditionally safe >CURRENT: > One may view this condition as defining an operating envelope for a > TDM PW, as a TDM PW that occupies no more bandwidth than a TCP flow > causes no more congestion than that TCP flow would. Under this > condition it is safe to place the TDM PW along with congestion- > responsive traffic such as TCP, without causing additional > congestion. on the other hand, were the TDM PW to consume > significantly more bandwidth a TCP flow, it could contribute > disproportionately to congestion, and its mixture with congestion- > responsive traffic might be inappropriate. >SUGGESTED: > One may view this condition as defining an=20 > unambiguously safe operating envelope for a > TDM PW, as a TDM PW that occupies no more bandwidth than a TCP flow > causes no more congestion than that TCP flow=20 > would. This does not draw us into a debate=20 > over whether TCP-friendliness is a valid=20 > constraint; it simply says that there can be no=20 > question that a PW is safe if it does no more=20 > harm than a single TCP flow. Under this > condition it is unambiguously safe to place=20 > the TDM PW along with congestion- > responsive traffic such as TCP. On the=20 > other hand, were the TDM PW to consume > significantly more bandwidth a TCP flow, it could contribute > disproportionately to congestion, and its mixture with congestion- > responsive traffic might start to cause concern. > >6. The analysis in Appendices A & B needs to=20 >mention that loss bursts arise primarily from=20 >tail drop buffers, and the introduction of=20 >active queue management (AQM) makes losses more=20 >independent of each other (less bursty). (I have=20 >references showing evidence of this.) In the=20 >comparisons with TCP, we have assumed=20 >independent losses anyway, because they are the=20 >worst case (and fortunately they are easier to=20 >analyse). On a similar point, in appendix B,=20 >where we say "(bursty packet loss was also=20 >simulated but is not reported here)", we ought=20 >to give a reason otherwise readers will be=20 >suspicious. A good reason would be that uniformly random is the worst case. > >I've also responded to a couple of David's=20 >suggestions inline (I agree with the others)... I did /not/ incorporate the more extensive text=20 requested/suggested by Stewart/David. I didn't=20 want to step all over your call as editor on this. As you'll see my responses to David's points are=20 still inline below. I agree with them all except: * I don't think we should go into any detail on a=20 circuit-breaker mechanism in this draft, which=20 should stay on the purely informational high ground. * Also I don't think we need to recommend a PW=20 should prefer larger packets; given we've just=20 shown there isn't a problem, we don't need to propose any rememdy. * I'm not convinced refs for Security=20 Considerations are necessary, but I wouldn't mind if you added them either. Cheers Bob >At 17:44 13/08/2014, Yaakov Stein wrote: >>Bob told me on Friday of the IETF that he was=20 >>going to look over the final version and comment. >> >>Bob =96 ping ? >> >>Y(J)S >> >>From: Andrew G. Malis [mailto:agmalis@gmail.com] >>Sent: 13 August, 2014 17:02 >>To: Yaakov Stein >>Cc: Bob Briscoe; rbriscoe@jungle.bt.co.uk; BOCCI Matthew; Black, David >>Subject: Re: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt >> >>Yaakov, >> >>I still haven't heard from Bob, but you should=20 >>incorporate David's updates into a new=20 >>revision, and I'll use that version for my final Shepherd's review. >> >>Thanks, >>Andy >> >>On Mon, Aug 11, 2014 at 11:16 AM, Andrew G.=20 >>Malis <agmalis@gmail.com> wrote: >>Actually, I'm going to end the LC as soon as I=20 >>hear back from Bob on the IPR call. I sent him a reminder earlier this= morning. >> >>Cheers, >>Andy >> >>On Sun, Aug 10, 2014 at 9:53 AM, Andrew G.=20 >>Malis <agmalis@gmail.com> wrote: >>David, >> >>That's perfect, thanks! I'm going to end the LC tomorrow. >> >>Cheers, >>Andy >> >>On Sat, Aug 9, 2014 at 10:32 PM, Black, David=20 >><david.black@emc.com> wrote: >>Following up on this ... >> >> > As we're now moving quickly to try to complete this draft , I agreed >> > w/Yaakov that my author comments would be=20 >> sent in as WG Last Call comments ... >> > ... so here they are: >>... with some suggested new text ... >> >> > Major issues: >> > >> > The reference to a transport circuit breaker=20 >> in section 3 is a bit optimistic >> > and high level: >> > >> > We can accomplish this by >> > employing a transport circuit breaker, by which we mean an= automatic >> > mechanism for terminating a flow to prevent negative impact on= other >> > flows and on the stability of the network >> > [I-D.fairhurst-tsvwg-circuit-breaker]. >> > >> > This should be accompanied by at least an example of how to do this -= past >> > discussion has suggested carrier monitoring of delivered service= quality >> > in order to shut down PWs that are experiencing loss rates that result= in >> > unacceptable service quality (as defined for us by the ITU ...). I'd= also >> > like to add some text to suggest that operators use this sort of= circuit >> > breaker practice when TDM PWs compete with elastic traffic. >>After this text in Section 3: >> >> We can accomplish this by >> employing a transport circuit breaker, by which we mean an automatic >> mechanism for terminating a flow to prevent negative impact on other >> flows and on the stability of the network >> [I-D.fairhurst-tsvwg-circuit-breaker]. Note that a transport circuit >> breaker is intended as a protection mechanism of last resort, just as >> an electrical circuit breaker is only triggered when absolutely >> necessary. >> >>I suggest adding: >> >> For TDM pseudowires, such a circuit breaker can be realized via >> operator monitoring of drop rates and consequent on the quality of >> TDM service delivered and shutting down TDM pseudowires that= persistently >> fail to deliver acceptable TDM service (as defined by [G775] and >> [G826]). This document explains why a TDM PW that is causing= congestion >> problems via interference with elastic traffic cannot be delivering >> acceptable TDM service (see the rest of this Section, plus Appendices >> A and B). > >I don't think we should go there. We need to=20 >keep this draft firmly as informational. If=20 >someone thinks a mechanism is needed because of=20 >the information in this draft, then they can write about a mechanism. > > >>And breaking the paragraph after the added text. >> >> > Figure 1 and Figure 2 indicate that based on using the TCP throughput >> > equation as a proxy for potential impact on elastic traffic (which is= what >> > we're doing in this draft), for the same=20 >> fixed bandwidth PW, a larger segment >> > size is less likely to cause impacts. I'd like to add a suggestion to= use >> > the larger PW segment sizes as long as they're consistent with delivery= of >> > the service. There's been a related=20 >> (long-to-the-point-of-ratholed) debate >> > over in the Transport Area about whether= packet-rate-limited/congestible >> > forwarding is still relevant. My view is that such equipment exists,= and >> > hence that this is a relevant consideration for this draft, and would= not >> > mention that issue here. >>Ignoring the rathole invitation ;-). This is still in section 3, after: >> >> We see that in general for large packet sizes, short delays, and low >> packet loss rates, the TDM pseudowires consume much less bandwidth >> than TCP would under identical conditions. Only for small packets, >> long delays, and high packet loss ratios, do TDM PWs potentially >> consume more bandwidth, and even then only marginally. >> >>I suggest adding: >> >> Comparing figures 1 and 2, use of a larger segment size for a T1 or E1 >> TDM PW moves the operational behavior of the PW away from the boundary >> at which the TCP throughput equation=20 >> suggests that undue interference with >> elastic traffic may start to occur. We suggest that larger segment >> sizes be preferred for T1 and E1 TDM PWs, but note that other >> considerations may favor smaller segment sizes (e.g., jitter buffer >> size, bounding the impact of an isolated packet drop event). > >There is no need to recommend any remedial=20 >action, when we have just shown there is not a problem. > >Further, the "marginal problem" only applies=20 >when the TCP traffic uses packets smaller than=20 >128B, which we only used as a hypothetical=20 >worst-case for the analysis. But in practice=20 >this doesn't happen with long-running TCP flows=20 >- they use packets that are as large as possible=20 >(irrespective of what the PW uses). So there really is /no/ problem to= remedy. > > >>and again, breaking the paragraph after that added text. >> >>I hope that latter sentence ("We suggest that ...") is acceptable to the= WG. >> >> > Minor issues: >> > >> > Section 2 is about Ethernet PWs that carry elastic traffic. Should= others >> > be mentioned as also covered by this analysis - e.g., ATM and Frame= Relay >> > PWs carrying elastic traffic? >>After: >> >> In this section we consider Ethernet PWs that primarily carry >> congestion-responsive traffic. We show that we automatically obtain >> the desired congestion avoidance behavior, and that additional >> mechanisms are not needed. >> >>I suggest adding: >> >> This discussion is applicable to any PW that carries congestion- >> responsive traffic (e.g., ATM and Frame Relay PWs); Ethernet PWs are >> used here for clarity, as they are a common example. >> >> > Section 2 does not cover Ethernet PWs=20 >> carrying inelastic traffic - that would >> > be unusual, but should it be explicitly noted as out of scope, or just= not >> > discussed (as is the case for the current text)? >>I suggest doing nothing here, and planning to deal with this concern >>if it should arise later (e.g., IETF Last Call or IESG Evaluation). >> >> > Section 3 should say that the reason the draft focuses on TDM PWs is= that >> > they're the inelastic PWs that are likely to=20 >> be carried over IP in practice. >> > My understanding is that other potentially inelastic PWs are MPLS-only >> > in practice. >>After: >> >> Inelastic PWs, such as TDM PWs ([RFC4553][RFC5086][RFC5087]), are >> potentially more problematic than the elastic PWs of the previous >> section. Being constant bit-rate (CBR), TDM PWs can not be made >> responsive to congestion. On the other hand, being CBR, they also do >> not attempt to capture additional bandwidth when neighboring TCP >> flows back off. >> >>I suggest adding: >> >> We focus on TDM PWs here as they are the only inelastic PWs that are >> likely to be IP-encapsulated in practice. Most other inelastic PWs >> tend to only be used with MPLS. >> >>[I'd ask Yaakov and other PW experts to double-check the latter sentence, >>please.] >> >> > A terminology section to roll up all the=20 >> acronym expansions (e.g., TDM, CBR) >> > in one place seems like a good idea. >>These four acronyms would be good to put into a terminology section: >> >>CBR Constant Bit Rate >>PSN Packet Switched Network >>PW Pseudowire >>TDM Time Division Multiplexing >> >> > >> > This text on p.11 could be incendiary: >> > >> > Under this condition it is safe to place the TDM PW along with >> > congestion - responsive traffic such as TCP, without causing >> > additional congestion. >> > >> > First of all to reduce the flammability, "safe" -> "reasonable" and= there >> > are a number of other occurrences of "safe"=20 >> that deserve similar treatment ;- >> > ). >> > Beyond that, this assertion is dependent on=20 >> the operating conditions staying >> > within that reasonable operating envelope, which entails use of= something >> > like a circuit breaker - I'd say that here, and reference the earlier >> > circuit breaker discussion. >>Global substitution: "safe" -> "reasonable", starting with the abstract. >> >>Then for this specific text on p.11: >> >>OLD >> Under this condition it is safe to place the TDM PW along with >> congestion - responsive traffic such as TCP, without causing >> additional congestion. >>NEW >> Under this condition it is reasonable=20 >> to place the TDM PW in competition >> with congestion-responsive traffic such=20 >> as TCP, as the result should not >> produce unacceptable congestion=20 >> effects. That assumes that the operating >> conditions remain within the reasonable=20 >> operating envelope - as discussed >> above, a transport circuit breaker is a=20 >> suggested means of managing a TDM >> PW so that it is shut down if operating conditions remain= persistently >> outside the reasonable operating envelope for the PW (e.g., due to= a >> high packet loss rate). > >No need to go there. See my suggestions earlier. > > >> > Figures 7 and 8 could use some shading or lines to indicate the typical >> > deployed operating regions for the pseudo-wires to visibly show that= they >> > fall under the curves, particularly for larger segment sizes. I'd also >> > repeat or refer to the suggestion to use larger > >I wanted to suggest this myself, but it is hard=20 >to depict fuzzy boundaries. If you can do this, pls do. > >Given the PW operating regions are bounded by v=20 >low loss probability, the x axis in Figs 7&8=20 >ought to be scaled much smaller as well. > >Related point: the y axis in Fig 1 ought to max at 10, not 50. > >Regards > > > >Bob > >> > >> > Nits: >> > >> > References aren't allowed in abstracts, so remove reference to >> > [I-D.fairhurst-tsvwg-circuit-breaker] >> > >> > Introduction, first paragraph, delete "PWE3" towards end of paragraph - >> > >> > preferably the 4 byte PWE3 control word. >> > >> > In addition to its current general statements, the Security= Considerations >> > section should point to the security considerations sections in a few >> > particularly relevant RFCs - I'd suggest 3985, 4553, 5086 and 5087. >>After: >> >> This document does not introduce any new congestion-specific >> mechanisms and thus does not introduce any new security >> considerations above those present for PWs in general. >> >>I suggest adding a new paragraph: >> >> For PW security considerations, see the security considerations= sections >> of the cited RFCS; the security considerations in [RFC3985],= [RFC4553], >> [RFC5086] and [RFC5087] are particularly applicable to this document. >> >> > >> > Thanks, >> > --David >> > >> > >> > > -----Original Message----- >> > > From: pwe3 [mailto:pwe3-bounces@ietf.org] On Behalf Of Yaakov Stein >> > > Sent: Thursday, July 24, 2014 4:52 PM >> > > To: pwe3@ietf.org >> > > Subject: Re: [PWE3] I-D Action: draft-ietf-pwe3-congcons-02.txt >> > > >> > > Please look at the PDF version only >> > > (the txt version does not have the graphs). >> > > >> > > Y(J)S >> > > >> > > >> > >> > _______________________________________________ >> > pwe3 mailing list >> > pwe3@ietf.org >> > https://www.ietf.org/mailman/listinfo/pwe3 >> >>_______________________________________________ >>pwe3 mailing list >>pwe3@ietf.org >>https://www.ietf.org/mailman/listinfo/pwe3 >> >> >> > >________________________________________________________________ >Bob Briscoe, BT ________________________________________________________________ Bob Briscoe, BT=20 --=====================_794198531==.ALT Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Yaakov,

This time adding pointer to all my nits and including the list, which I hadn't noticed wasn't cc'd when I replied to all. More inline within my own previous email..

At 14:02 14/08/2014, Bob Briscoe wrote:
To: Yaakov Stein <yaakov_s@rad.com>
From: Bob Briscoe <bob.briscoe@bt.com>
Subject: RE: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt
Cc: "Andrew G. Malis" <agmalis@gmail.com>, BOCCI Matthew <Matthew.Bocci@alcatel-lucent.com>, "Black, David" <david.black@emc.com>

Yaakov,

I have just re-read it thoroughly. I have noticed about two dozen editorial nits where correcting them would not change the intended sense (altho sometimes the sense needs to change to say what we intended!). Examples:
- Unexpanded abbreviations (David's suggestion would help, but there are more)
- Missing words, tautologies, random punctuation problems etc
- Uses of the wrong word, even tho we know what you mean
- The caveat about dynamics vs steady state needs to shift down a para, so that it follows into the para that says what to do about dynamics.

Should I take the token, and edit the XML directly, then you can see a diff to check if you want to accept my suggestions?

I've provided xml source of my suggested edits (and a diff) here:
< http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-pwe3-c= ongcons-DIFF03a-02.html >
< http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-pwe3-c= ongcons-03a.txt >
< http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-pwe3-c= ongcons-03a.xml >

I'm not implying that this is now the master version - it's just an easy way to describe edits (and you can use it as the basis for a new master if you want). Similarly to David, pls take my comments as co-author into consideration in parallel with the WGLC responses.

I couldn't alter the figures. I would ask for the following changes:
* All Figs: s/PLR/PLP/
* Figs 1-4:
  - s/delay/one-way delay.
* Fig 1: max at 10 not 50 on y axis
* Figs 5-8:
  - s/safe/undeniably safe/
* Figs 7-8: max at 0.5% on x axis
* Fig 9: s/Pd/PLP/

I incorporated those nits I could see on the list (Sasha, Stewart & David), some of which I had noticed myself.

Below are six thoughts that g= o slightly beyond 'editorial'. 1-4 could be considered 'editorial'. But 5&6 are the more significant "techno-political" points.

Suggested resolutions of these six bigger issues are now also in my xml suggestion above too.


1. The phrase "mechanism= s for provisioning PWs in service provider networks" is too opaque (the PWE audience might understand this scoping sentence, but a L4 reader might not understand what we're excluding here, which could make them suspicious).
CURRENT:
   Mechanisms for provisioning PWs
   in service provider networks are well understood and will not be
   discussed further here.
SUGGESTED:
   The case where the service provider network arranges sufficient capacity for a PW is well understood and will not be discussed further here. The concern is over PWs deployed independently of the service provider network's traffic engineering or capacity planning.

2. The justification for using OWD of a few milliseconds misses the point.
"  The one-way delay
   of a native TDM service consists of the physical time-of-flight plus
   125 microseconds for each TDM switch traversed; and is thus very
   small as compared to typical PSN network-crossing latencies.
"
The 125us per switch is irrelevant. The point that needs making is that PWs are generally deployed with a short physical time of flight.

3. Related point: the captions on the graphs do not clearly indicate that they are OWD, not RTT. This could be clarified in the text, or on the graphs.

4. Change packet loss rate (and PLR) to packet loss probability throughout, including on the graph axes. PLR is an ambiguous term, even though it is widely used in the industry. Packet loss rate could be measured in packet per second, whereas packet loss probability is unambiguously dimensionless. Indeed, in the appendices, packet loss rates with respect to time are calculated.

5. Rather than David's "reasonable" I suggest we use the phrase "unambiguously safe" not just "safe" for the headings of Figs 5-8 and in the text, as below. Otherwise, we are implying that it is unsafe outside the region.

I thought of a better phrase: "undeniably safe" instead of "unambiguously safe".
I slightly prefer this over these other possibilities:
- unarguably safe
- indisputably safe
- unconditionally safe

CURRENT:
   One may view this condition as defining an operating envelope for a
   TDM PW, as a TDM PW that occupies no more bandwidth than a TCP flow
   causes no more congestion than that TCP flow would.  Under this
   condition it is safe to place the TDM PW along with congestion-
   responsive traffic such as TCP, without causing additional
   congestion.  on the other hand, were the TDM PW to consume
   significantly more bandwidth a TCP flow, it could contribute
   disproportionately to congestion, and its mixture with congestion-
   responsive traffic might be inappropriate.
SUGGESTED:
   One may view this condition as defining an unambiguously safe operating envelope for a
   TDM PW, as a TDM PW that occupies no more bandwidth than a TCP flow
   causes no more congestion than that TCP flow would.  This does not draw us into a debate over whether TCP-friendliness is a valid constraint; it simply says that there can be no question that a PW is safe if it does no more harm than a single TCP flow. Under this
   condition it is unambiguously safe to place the TDM PW along with congestion-
   responsive traffic such as TCP.  On the other hand, were the TDM PW to consume
   significantly more bandwidth a TCP flow, it could contribute
   disproportionately to congestion, and its mixture with congestion-
   responsive traffic might start to cause concern.

6. The analysis in Appendices A & B needs to mention that loss bursts arise primarily from tail drop buffers, and the introduction of active queue management (AQM) makes losses more independent of each other (less bursty). (I have references showing evidence of this.) In the comparisons with TCP, we have assumed independent losses anyway, because they are the worst case (and fortunately they are easier to analyse). On a similar point, in appendix B, where we say "(bursty packet loss was also simulated but is not reported here)", we ought to give a reason otherwise readers will be suspicious. A good reason would be that uniformly random is the worst case.

I've also responded to a couple of David's suggestions inline (I agree with the others)...

I did /not/ incorporate the more extensive text requested/suggested by Stewart/David. I didn't want to step all over your call as editor on this.

As you'll see my responses to David's points are still inline below. I agree with them all except:
* I don't think we should go into any detail on a circuit-breaker mechanism in this draft, which should stay on the purely informational high ground.
* Also I don't think we need to recommend a PW should prefer larger packets; given we've just shown there isn't a problem, we don't need to propose any rememdy.
* I'm not convinced refs for Security Considerations are necessary, but I wouldn't mind if you added them either.


Cheers



Bob




At 17:44 13/08/2014, Yaakov Stein wrote:
Bob told me on Friday of the IETF that he was going to look over the final version and comment.
 
Bob =96 ping ?
 
Y(J)S
 
From: Andrew G. Malis [ mailto:agmalis@gmail.com]
Sent: 13 August, 2014 17:02
To: Yaakov Stein
Cc: Bob Briscoe; rbriscoe@jungle.bt.co.uk; BOCCI Matthew; Black, David
Subject: Re: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt
 
Yaakov,
 
I still haven't heard from Bob, but you should incorporate David's updates into a new revision, and I'll use that version for my final Shepherd's review.
 
Thanks,
Andy
 
On Mon, Aug 11, 2014 at 11:16 AM, Andrew G. Malis <agmalis@gmail.com> wrote:
Actually, I'm going to end the LC as soon as I hear back from Bob on the IPR call. I sent him a reminder earlier this morning.
 
Cheers,
Andy
 
On Sun, Aug 10, 2014 at 9:53 AM, Andrew G. Malis <agmalis@gmail.com> wrote:
David,
 
That's perfect, thanks! I'm going to end the LC tomorrow.
 
Cheers,
Andy
 
On Sat, Aug 9, 2014 at 10:32 PM, Black, David <david.black@emc.com> wrote:
Following up on this ...

> As we're now moving quickly to try to complete this draft , I agreed
> w/Yaakov that my author comments would be sent in as WG Last Call comments ...
> ... so here they are:
... with some suggested new text ...

> Major issues:
>
> The reference to a transport circuit breaker in section 3 is a bit optimistic
> and high level:
>
>       We can accomplish this by
>       employing a transport circuit breaker, by which we mean an automatic
>       mechanism for terminating a flow to prevent negative impact on other
>       flows and on the stability of the network
>       [I-D.fairhurst-tsvwg-circuit-breaker].
>
> This should be accompanied by at least an example of how to do this - past
> discussion has suggested carrier monitoring of delivered service quality
> in order to shut down PWs that are experiencing loss rates that result in
> unacceptable service quality (as defined for us by the ITU ...).  I'd also
> like to add some text to suggest that operators use this sort of circuit
> breaker practice when TDM PWs compete with elastic traffic.
After this text in Section 3:

   We can accomplish this by
   employing a transport circuit breaker, by which we mean an automatic
   mechanism for terminating a flow to prevent negative impact on other
   flows and on the stability of the network
   [I-D.fairhurst-tsvwg-circuit-breaker].  Note that a transport circuit
   breaker is intended as a protection mechanism of last resort, just as
   an electrical circuit breaker is only triggered when absolutely
   necessary.

I suggest adding:

   For TDM pseudowires, such a circuit breaker can be realized via
   operator monitoring of drop rates and consequent on the quality of
   TDM service delivered and shutting down TDM pseudowires that persistently
   fail to deliver acceptable TDM service (as defined by [G775] and
   [G826]).  This document explains why a TDM PW that is causing congestion
   problems via interference with elastic traffic cannot be delivering
   acceptable TDM service (see the rest of this Section, plus Appendices
   A and B).

I don't think we should go there. We need to keep this draft firmly as informational. If someone thinks a mechanism is needed because of the information in this draft, then they can write about a mechanism.


And breaking the paragraph af= ter the added text.

> Figure 1 and Figure 2 indicate that based on using the TCP throughput
> equation as a proxy for potential impact on elastic traffic (which is what
> we're doing in this draft), for the same fixed bandwidth PW, a larger segment
> size is less likely to cause impacts.  I'd like to add a suggestion to use
> the larger PW segment sizes as long as they're consistent with delivery of
> the service.  There's been a related (long-to-the-point-of-ratholed) debate
> over in the Transport Area about whether packet-rate-limited/congestible
> forwarding is still relevant.  My view is that such equipment exists, and
> hence that this is a relevant consideration for this draft, and would not
> mention that issue here.
Ignoring the rathole invitation ;-).  This is still in section 3, after:

   We see that in general for large packet sizes, short delays, and low
   packet loss rates, the TDM pseudowires consume much less bandwidth
   than TCP would under identical conditions.  Only for small packets,
   long delays, and high packet loss ratios, do TDM PWs potentially
   consume more bandwidth, and even then only marginally.

I suggest adding:

   Comparing figures 1 and 2, use of a larger segment size for a T1 or E1
   TDM PW moves the operational behavior of the PW away from the boundary
   at which the TCP throughput equation suggests that undue interference with
   elastic traffic may start to occur.  We suggest that larger segment
   sizes be preferred for T1 and E1 TDM PWs, but note that other
   considerations may favor smaller segment sizes (e.g., jitter buffer
   size, bounding the impact of an isolated packet drop event).

There is no need to recommend any remedial action, when we have just shown there is not a problem.

Further, the "marginal problem" only applies when the TCP traffic uses packets smaller than 128B, which we only used as a hypothetical worst-case for the analysis. But in practice this doesn't happen with long-running TCP flows - they use packets that are as large as possible (irrespective of what the PW uses). So there really is /no/ problem to remedy.


and again, breaking the paragraph after that added text.

I hope that latter sentence ("We suggest that ...") is acceptable to the WG.

> Minor issues:
>
> Section 2 is about Ethernet PWs that carry elastic traffic.  Should others
> be mentioned as also covered by this analysis - e.g., ATM and Frame Relay
> PWs carrying elastic traffic?
After:

   In this section we consider Ethernet PWs that primarily carry
   congestion-responsive traffic.  We show that we automatically obtain
   the desired congestion avoidance behavior, and that additional
   mechanisms are not needed.

I suggest adding:

   This discussion is applicable to any PW that carries congestion-
   responsive traffic (e.g., ATM and Frame Relay PWs); Ethernet PWs are
   used here for clarity, as they are a common example.

> Section 2 does not cover Ethernet PWs carrying inelastic traffic - that would
> be unusual, but should it be explicitly noted as out of scope, or just not
> discussed (as is the case for the current text)?
I suggest doing nothing here, and planning to deal with this concern
if it should arise later (e.g., IETF Last Call or IESG Evaluation).

> Section 3 should say that the reason the draft focuses on TDM PWs is that
> they're the inelastic PWs that are likely to be carried over IP in practice.
> My understanding is that other potentially inelastic PWs are MPLS-only
> in practice.
After:

   Inelastic PWs, such as TDM PWs ([RFC4553][RFC5086][RFC5087]), are
   potentially more problematic than the elastic PWs of the previous
   section.  Being constant bit-rate (CBR), TDM PWs can not be made
   responsive to congestion.  On the other hand, being CBR, they also do
   not attempt to capture additional bandwidth when neighboring TCP
   flows back off.

I suggest adding:

   We focus on TDM PWs here as they are the only inelastic PWs that are
   likely to be IP-encapsulated in practice.  Most other inelastic PWs
   tend to only be used with MPLS.

[I'd ask Yaakov and other PW experts to double-check the latter sentence,
please.]

> A terminology section to roll up all the acronym expansions (e.g., TDM, CBR)
> in one place seems like a good idea.
These four acronyms would be good to put into a terminology section:

CBR Constant Bit Rate
PSN Packet Switched Network
PW Pseudowire
TDM Time Division Multiplexing

>
> This text on p.11 could be incendiary:
>
>       Under this condition it is safe to place the TDM PW along with
>       congestion - responsive traffic such as TCP, without causing
>       additional congestion.
>
> First of all to reduce the flammability, "safe" -> "reasonable" and there
> are a number of other occurrences of "safe" that deserve similar treatment ;-
> ).
> Beyond that, this assertion is dependent on the operating conditions staying
> within that reasonable operating envelope, which entails use of something
> like a circuit breaker - I'd say that here, and reference the earlier
> circuit breaker discussion.
Global substitution: "safe" -> "reasonable", starting with the abstract.

Then for this specific text on p.11:

OLD
        Under this condition it is safe to place the TDM PW along with
        congestion - responsive traffic such as TCP, without causing
        additional congestion.
NEW
        Under this condition it is reasonable to place the TDM PW in competition
      with congestion-responsive traffic such as TCP, as the result should not
      produce unacceptable congestion effects.  That assumes that the operating
      conditions remain within the reasonable operating envelope - as discussed
      above, a transport circuit breaker is a suggested means of managing a TDM
      PW so that it is shut down if operating conditions remain persistently
      outside the reasonable operating envelope for the PW (e.g., due to a
      high packet loss rate).

No need to go there. See my suggestions earlier.


> Figures 7 and 8 could us= e some shading or lines to indicate the typical
> deployed operating regions for the pseudo-wires to visibly show that they
> fall under the curves, particularly for larger segment sizes.  I'd also
> repeat or refer to the suggestion to use larger

I wanted to suggest this myself, but it is hard to depict fuzzy boundaries. If you can do this, pls do.

Given the PW operating regions are bounded by v low loss probability, the x axis in Figs 7&8 ought to be scaled much smaller as well.

Related point: the y axis in Fig 1 ought to max at 10, not 50.

Regards



Bob

>
> Nits:
>
> References aren't allowed in abstracts, so remove reference to
> [I-D.fairhurst-tsvwg-circuit-breaker]
>
> Introduction, first paragraph, delete "PWE3" towards end of paragraph -
>
>    preferably the 4 byte PWE3 control word.
>
> In addition to its current general statements, the Security Considerations
> section should point to the security considerations sections in a few
> particularly relevant RFCs - I'd suggest 3985, 4553, 5086 and 5087.
After:

   This document does not introduce any new congestion-specific
   mechanisms and thus does not introduce any new security
   considerations above those present for PWs in general.

I suggest adding a new paragraph:

   For PW security considerations, see the security considerations sections
   of the cited RFCS; the security considerations in [RFC3985], [RFC4553],
   [RFC5086] and [RFC5087] are particularly applicable to this document.

>
> Thanks,
> --David
>
>
> > -----Original Message-----
> > From: pwe3 [ mailto:pwe3-bounces@ietf.org] On Behalf Of Yaakov Stein
> > Sent: Thursday, July 24, 2014 4:52 PM
> > To: pwe3@ietf.org
> > Subject: Re: [PWE3] I-D Action: draft-ietf-pwe3-congcons-02.txt
> >
> > Please look at the PDF version only
> > (the txt version does not have the graphs).
> >
> > Y(J)S
> >
> >
>
> _______________________________________________
> pwe3 mailing list
> pwe3@ietf.org
> https://www.ietf.org/mailman/listinfo/pwe3

_______________________________________________
pwe3 mailing list
pwe3@ietf.org
https://www.ietf.org/mailman/listinfo/pwe3
 
 
 

________________________________________________________________
Bob Briscoe,           &n= bsp;            =             &nbs= p;             BT

________________________________________________________________
Bob Briscoe,           &n= bsp;            =             &nbs= p;             BT --=====================_794198531==.ALT-- From nobody Thu Aug 14 11:40:16 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6CEC41A8033 for ; Thu, 14 Aug 2014 11:40:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -7.368 X-Spam-Level: X-Spam-Status: No, score=-7.368 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, GB_I_INVITATION=-2, GB_I_LETTER=-2, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-0.668, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xEkzOp1uUsyB for ; Thu, 14 Aug 2014 11:39:58 -0700 (PDT) Received: from mailuogwhop.emc.com (mailuogwhop.emc.com [168.159.213.141]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 43D4C1A035B for ; Thu, 14 Aug 2014 11:39:50 -0700 (PDT) Received: from maildlpprd01.lss.emc.com (maildlpprd01.lss.emc.com [10.253.24.33]) by mailuogwprd04.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s7EIdmhm022918 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 14 Aug 2014 14:39:49 -0400 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd04.lss.emc.com s7EIdmhm022918 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=emc.com; s=jan2013; t=1408041589; bh=2OJG+kXVBNYnuyWuxl00Ao61K7g=; h=From:To:Date:Subject:Message-ID:References:In-Reply-To: Content-Type:MIME-Version; b=R8AS54kTm6jdI0LEjXox5FuuQ3TyhbI+edi4ZCErT2HOW6FTVyGSe7UsBomMH5f17 BcehPzpDG7iGzFPQu430irHxVL1i6ckv4AmYW4N236ZUBDr2ypP5gD++nOIUmA6LYM CBi1UAexMEH6MHPnE1e9lh9g5/eWO0rJSrLM1dmI= X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd04.lss.emc.com s7EIdmhm022918 Received: from mailusrhubprd03.lss.emc.com (mailusrhubprd03.lss.emc.com [10.253.24.21]) by maildlpprd01.lss.emc.com (RSA Interceptor) for ; Thu, 14 Aug 2014 14:39:37 -0400 Received: from mxhub09.corp.emc.com (mxhub09.corp.emc.com [10.254.92.104]) by mailusrhubprd03.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s7EIdarx003246 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL) for ; Thu, 14 Aug 2014 14:39:36 -0400 Received: from mx15a.corp.emc.com ([169.254.1.175]) by mxhub09.corp.emc.com ([10.254.92.104]) with mapi; Thu, 14 Aug 2014 14:39:36 -0400 From: "Black, David" To: "pwe3@ietf.org" Date: Thu, 14 Aug 2014 14:39:34 -0400 Thread-Topic: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt Thread-Index: Ac+3wBXFXMAxOLLcSy+DvdqN9xfFmAAHaodAAARIp6A= Message-ID: <8D3D17ACE214DC429325B2B98F3AE712077B95212C@MX15A.corp.emc.com> References: <8D3D17ACE214DC429325B2B98F3AE712077B9517F4@MX15A.corp.emc.com> <8D3D17ACE214DC429325B2B98F3AE712077B951928@MX15A.corp.emc.com> <201408141302.s7ED2km0032238@bagheera.jungle.bt.co.uk> <8D3D17ACE214DC429325B2B98F3AE712077B952113@MX15A.corp.emc.com> In-Reply-To: <8D3D17ACE214DC429325B2B98F3AE712077B952113@MX15A.corp.emc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_8D3D17ACE214DC429325B2B98F3AE712077B95212CMX15Acorpemcc_" MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd03.lss.emc.com X-RSA-Classifications: DLM_1, public Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/gjVbjmycwWlzgj21eatpkWAYR_Y Subject: [PWE3] FW: WGLC comments on: draft-ietf-pwe3-congcons-02.txt X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Aug 2014 18:40:11 -0000 --_000_8D3D17ACE214DC429325B2B98F3AE712077B95212CMX15Acorpemcc_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable And forwarding my views to the list ... ... On Bob's six thoughts (I have problems with 2 & 5 - the other 4 are bas= ically ok w/me) plus follow-ups on Bob's concerns w/my two major issues, he= re goes: -- 1 -- I agree in principle, but I think Bob's suggested new text is too b= lack/white and should be generalized. Bob suggests: > The case where the service provider network arranges sufficient capacity = for a PW is well understood and > will not be discussed further here. The concern is over PWs deployed inde= pendently of the service provider > network's traffic engineering or capacity planning. I want to include the case where capacity has been arranged, but may not be= sufficient, I suggest rewriting Bob's second sentence as: The concern is over PWs that share network capacity with elastic or congest= ion-responsive traffic, independent of whether that sharing is a result of = the service provider network's traffic engineering and capacity planning or= independent deployment of PWs by entities other than network operators. -- 2 -- Here, I disagree. One of the significant contributions of this dra= ft is that TDM PWs don't cause congestion problems in practice, and one of = the major reasons for that is the low latency requirements for acceptable T= DM service delivery, both in ITU-T standards and in actual TDM PW deploymen= t. The 125us behavior of native TDM switches is relevant to the latter, an= d hence I prefer to include mention of it. -- 3 -- I agree; clear captions are always a feature, not a bug ;-). -- 4 -- I agree with changing "packet loss rate" to "packet loss probabilit= y". -- 5 -- I firmly disagree with Bob here, although he's right that this is "= techno-political" ;-). I am concerned that S-A-F-E is likely to be a 4-letter word in wider Transp= ort Area review, and hence, I want to use a word that does not imply anythi= ng remotely resembling an absolute guarantee that nothing can possibly ever= go wrong (yes, I know Bob didn't mean to go to quite that extreme an inter= pretation, but S-A-F-E is liable to be read that way by those with an "axe = to grind"), even though there's lots of solid operational experience that s= uggests all is well here. -- 6 -- I think I'm ok with what Bob proposes to better characterize loss b= urstiness, but want to see specific text. -- Circuit Breaker Mechanism -- >> [David] I suggest adding: >> >> For TDM pseudowires, such a circuit breaker can be realized via >> operator monitoring of drop rates and consequent on the quality of >> TDM service delivered and shutting down TDM pseudowires that persiste= ntly >> fail to deliver acceptable TDM service (as defined by [G775] and >> [G826]). This document explains why a TDM PW that is causing congest= ion >> problems via interference with elastic traffic cannot be delivering >> acceptable TDM service (see the rest of this Section, plus Appendices >> A and B). >> > [Bob] I don't think we should go there. We need to keep this draft firmly= as informational. If someone > thinks a mechanism is needed because of the information in this draft, th= en they can write about a mechanism. [David] I firmly disagree, but Bob and I might be talking past each other, = as I do want to stay informational ;-). This text is intended to describe an example or possibility, not to specify= an implementation requirement. The new text was intended as an OAM/OSS pr= actice recommendation to the operator, not as a normative protocol spec. I'm ok with inserting "example", "possibility" or something similar if that= helps w/Bob's concern, but I strongly oppose doing nothing here, and note = that this is also needed to respond to one of Stewart Bryant's WG LC commen= ts. -- PW segment size -- >> [David] Ignoring the rathole invitation ;-). This is still in section 3= , after: >> >> We see that in general for large packet sizes, short delays, and low >> packet loss rates, the TDM pseudowires consume much less bandwidth >> than TCP would under identical conditions. Only for small packets, >> long delays, and high packet loss ratios, do TDM PWs potentially >> consume more bandwidth, and even then only marginally. >> >> I suggest adding: >> >> Comparing figures 1 and 2, use of a larger segment size for a T1 or E= 1 >> TDM PW moves the operational behavior of the PW away from the boundar= y >> at which the TCP throughput equation suggests that undue interference= with >> elastic traffic may start to occur. We suggest that larger segment >> sizes be preferred for T1 and E1 TDM PWs, but note that other >> considerations may favor smaller segment sizes (e.g., jitter buffer >> size, bounding the impact of an isolated packet drop event). >> > [Bob] There is no need to recommend any remedial action, when we have jus= t shown there is not a problem. [David] Not exactly - Figure 1 shows that there is a potential concern at a= combination of the high end of the range of TDM PW latencies (4ms), and hi= gher packet loss ratios. > Further, the "marginal problem" only applies when the TCP traffic uses pa= ckets smaller than 128B, which we only used > as a hypothetical worst-case for the analysis. But in practice this doesn= 't happen with long-running TCP flows - > they use packets that are as large as possible (irrespective of what the = PW uses). So there really is /no/ problem to remedy. Ok, Bob can relax - his transport credentials are intact, as this is an ent= ry point to the bit-rate-congestible vs. packet-rate-congestible resource r= athole that I want to ignore :-). I believe that larger TDM PW segment sizes are a good thing for both this o= perating envelope boundary concern as well as per-packet processing overhea= ds in TDM PW functionality, modulo other considerations, e.g., jitter buffe= rs, and so I prefer to keep this addition. Thanks, --David From: Bob Briscoe [mailto:bob.briscoe@bt.com] Sent: Thursday, August 14, 2014 9:03 AM To: Yaakov Stein Cc: Andrew G. Malis; BOCCI Matthew; Black, David Subject: RE: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt Yaakov, I have just re-read it thoroughly. I have noticed about two dozen editorial= nits where correcting them would not change the intended sense (altho some= times the sense needs to change to say what we intended!). Examples: - Unexpanded abbreviations (David's suggestion would help, but there are mo= re) - Missing words, tautologies, random punctuation problems etc - Uses of the wrong word, even tho we know what you mean - The caveat about dynamics vs steady state needs to shift down a para, so = that it follows into the para that says what to do about dynamics. Should I take the token, and edit the XML directly, then you can see a diff= to check if you want to accept my suggestions? Below are six thoughts that go slightly beyond 'editorial'. 1-4 could be co= nsidered 'editorial'. But 5&6 are the more significant "techno-political" p= oints. 1. The phrase "mechanisms for provisioning PWs in service provider networks= " is too opaque (the PWE audience might understand this scoping sentence, b= ut a L4 reader won't understand what we're excluding here, which will make = them suspicious). CURRENT: Mechanisms for provisioning PWs in service provider networks are well understood and will not be discussed further here. SUGGESTED: The case where the service provider network arranges sufficient capacity= for a PW is well understood and will not be discussed further here. The co= ncern is over PWs deployed independently of the service provider network's = traffic engineering or capacity planning. 2. The justification for using OWD of a few milliseconds misses the point. " The one-way delay of a native TDM service consists of the physical time-of-flight plus 125 microseconds for each TDM switch traversed; and is thus very small as compared to typical PSN network-crossing latencies. " The 125us per switch is irrelevant. The point that needs making is that PWs= are generally deployed with a short physical time of flight. 3. Related point: the captions on the graphs do not clearly indicate that t= hey are OWD, not RTT. This could be clarified in the text, or on the graphs= . 4. Change packet loss rate (and PLR) to packet loss probability throughout,= including on the graph axes. PLR is an ambiguous term, even though it is w= idely used in the industry. Packet loss rate could be measured in packet pe= r second, whereas packet loss probability is unambiguously dimensionless. I= ndeed, in the appendices, packet loss rates with respect to time are calcul= ated. 5. Rather than David's "reasonable" I suggest we use the phrase "unambiguou= sly safe" not just "safe" for the headings of Figs 5-8 and in the text, as = below. Otherwise, we are implying that it is unsafe outside the region. CURRENT: One may view this condition as defining an operating envelope for a TDM PW, as a TDM PW that occupies no more bandwidth than a TCP flow causes no more congestion than that TCP flow would. Under this condition it is safe to place the TDM PW along with congestion- responsive traffic such as TCP, without causing additional congestion. on the other hand, were the TDM PW to consume significantly more bandwidth a TCP flow, it could contribute disproportionately to congestion, and its mixture with congestion- responsive traffic might be inappropriate. SUGGESTED: One may view this condition as defining an unambiguously safe operating = envelope for a TDM PW, as a TDM PW that occupies no more bandwidth than a TCP flow causes no more congestion than that TCP flow would. This does not draw = us into a debate over whether TCP-friendliness is a valid constraint; it si= mply says that there can be no question that a PW is safe if it does no mor= e harm than a single TCP flow. Under this condition it is unambiguously safe to place the TDM PW along with conges= tion- responsive traffic such as TCP. On the other hand, were the TDM PW to c= onsume significantly more bandwidth a TCP flow, it could contribute disproportionately to congestion, and its mixture with congestion- responsive traffic might start to cause concern. 6. The analysis in Appendices A & B needs to mention that loss bursts arise= primarily from tail drop buffers, and the introduction of active queue man= agement (AQM) makes losses more independent of each other (less bursty). (I= have references showing evidence of this.) In the comparisons with TCP, we= have assumed independent losses anyway, because they are the worst case (a= nd fortunately they are easier to analyse). On a similar point, in appendix= B, where we say "(bursty packet loss was also simulated but is not reporte= d here)", we ought to give a reason otherwise readers will be suspicious. A= good reason would be that uniformly random is the worst case. I've also responded to a couple of David's suggestions inline (I agree with= the others)... At 17:44 13/08/2014, Yaakov Stein wrote: Bob told me on Friday of the IETF that he was going to look over the final = version and comment. Bob - ping ? Y(J)S From: Andrew G. Malis [ mailto:agmalis@gmail.com] Sent: 13 August, 2014 17:02 To: Yaakov Stein Cc: Bob Briscoe; rbriscoe@jungle.bt.co.uk;= BOCCI Matthew; Black, David Subject: Re: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt Yaakov, I still haven't heard from Bob, but you should incorporate David's updates = into a new revision, and I'll use that version for my final Shepherd's revi= ew. Thanks, Andy On Mon, Aug 11, 2014 at 11:16 AM, Andrew G. Malis > wrote: Actually, I'm going to end the LC as soon as I hear back from Bob on the IP= R call. I sent him a reminder earlier this morning. Cheers, Andy On Sun, Aug 10, 2014 at 9:53 AM, Andrew G. Malis > wrote: David, That's perfect, thanks! I'm going to end the LC tomorrow. Cheers, Andy On Sat, Aug 9, 2014 at 10:32 PM, Black, David > wrote: Following up on this ... > As we're now moving quickly to try to complete this draft , I agreed > w/Yaakov that my author comments would be sent in as WG Last Call comment= s ... > ... so here they are: ... with some suggested new text ... > Major issues: > > The reference to a transport circuit breaker in section 3 is a bit optimi= stic > and high level: > > We can accomplish this by > employing a transport circuit breaker, by which we mean an automati= c > mechanism for terminating a flow to prevent negative impact on othe= r > flows and on the stability of the network > [I-D.fairhurst-tsvwg-circuit-breaker]. > > This should be accompanied by at least an example of how to do this - pas= t > discussion has suggested carrier monitoring of delivered service quality > in order to shut down PWs that are experiencing loss rates that result in > unacceptable service quality (as defined for us by the ITU ...). I'd als= o > like to add some text to suggest that operators use this sort of circuit > breaker practice when TDM PWs compete with elastic traffic. After this text in Section 3: We can accomplish this by employing a transport circuit breaker, by which we mean an automatic mechanism for terminating a flow to prevent negative impact on other flows and on the stability of the network [I-D.fairhurst-tsvwg-circuit-breaker]. Note that a transport circuit breaker is intended as a protection mechanism of last resort, just as an electrical circuit breaker is only triggered when absolutely necessary. I suggest adding: For TDM pseudowires, such a circuit breaker can be realized via operator monitoring of drop rates and consequent on the quality of TDM service delivered and shutting down TDM pseudowires that persistentl= y fail to deliver acceptable TDM service (as defined by [G775] and [G826]). This document explains why a TDM PW that is causing congestion problems via interference with elastic traffic cannot be delivering acceptable TDM service (see the rest of this Section, plus Appendices A and B). I don't think we should go there. We need to keep this draft firmly as info= rmational. If someone thinks a mechanism is needed because of the informati= on in this draft, then they can write about a mechanism. And breaking the paragraph after the added text. > Figure 1 and Figure 2 indicate that based on using the TCP throughput > equation as a proxy for potential impact on elastic traffic (which is wha= t > we're doing in this draft), for the same fixed bandwidth PW, a larger seg= ment > size is less likely to cause impacts. I'd like to add a suggestion to us= e > the larger PW segment sizes as long as they're consistent with delivery o= f > the service. There's been a related (long-to-the-point-of-ratholed) deba= te > over in the Transport Area about whether packet-rate-limited/congestible > forwarding is still relevant. My view is that such equipment exists, and > hence that this is a relevant consideration for this draft, and would not > mention that issue here. Ignoring the rathole invitation ;-). This is still in section 3, after: We see that in general for large packet sizes, short delays, and low packet loss rates, the TDM pseudowires consume much less bandwidth than TCP would under identical conditions. Only for small packets, long delays, and high packet loss ratios, do TDM PWs potentially consume more bandwidth, and even then only marginally. I suggest adding: Comparing figures 1 and 2, use of a larger segment size for a T1 or E1 TDM PW moves the operational behavior of the PW away from the boundary at which the TCP throughput equation suggests that undue interference wi= th elastic traffic may start to occur. We suggest that larger segment sizes be preferred for T1 and E1 TDM PWs, but note that other considerations may favor smaller segment sizes (e.g., jitter buffer size, bounding the impact of an isolated packet drop event). There is no need to recommend any remedial action, when we have just shown = there is not a problem. Further, the "marginal problem" only applies when the TCP traffic uses pack= ets smaller than 128B, which we only used as a hypothetical worst-case for = the analysis. But in practice this doesn't happen with long-running TCP flo= ws - they use packets that are as large as possible (irrespective of what t= he PW uses). So there really is /no/ problem to remedy. and again, breaking the paragraph after that added text. I hope that latter sentence ("We suggest that ...") is acceptable to the WG= . > Minor issues: > > Section 2 is about Ethernet PWs that carry elastic traffic. Should other= s > be mentioned as also covered by this analysis - e.g., ATM and Frame Relay > PWs carrying elastic traffic? After: In this section we consider Ethernet PWs that primarily carry congestion-responsive traffic. We show that we automatically obtain the desired congestion avoidance behavior, and that additional mechanisms are not needed. I suggest adding: This discussion is applicable to any PW that carries congestion- responsive traffic (e.g., ATM and Frame Relay PWs); Ethernet PWs are used here for clarity, as they are a common example. > Section 2 does not cover Ethernet PWs carrying inelastic traffic - that w= ould > be unusual, but should it be explicitly noted as out of scope, or just no= t > discussed (as is the case for the current text)? I suggest doing nothing here, and planning to deal with this concern if it should arise later (e.g., IETF Last Call or IESG Evaluation). > Section 3 should say that the reason the draft focuses on TDM PWs is that > they're the inelastic PWs that are likely to be carried over IP in practi= ce. > My understanding is that other potentially inelastic PWs are MPLS-only > in practice. After: Inelastic PWs, such as TDM PWs ([RFC4553][RFC5086][RFC5087]), are potentially more problematic than the elastic PWs of the previous section. Being constant bit-rate (CBR), TDM PWs can not be made responsive to congestion. On the other hand, being CBR, they also do not attempt to capture additional bandwidth when neighboring TCP flows back off. I suggest adding: We focus on TDM PWs here as they are the only inelastic PWs that are likely to be IP-encapsulated in practice. Most other inelastic PWs tend to only be used with MPLS. [I'd ask Yaakov and other PW experts to double-check the latter sentence, please.] > A terminology section to roll up all the acronym expansions (e.g., TDM, C= BR) > in one place seems like a good idea. These four acronyms would be good to put into a terminology section: CBR Constant Bit Rate PSN Packet Switched Network PW Pseudowire TDM Time Division Multiplexing > > This text on p.11 could be incendiary: > > Under this condition it is safe to place the TDM PW along with > congestion - responsive traffic such as TCP, without causing > additional congestion. > > First of all to reduce the flammability, "safe" -> "reasonable" and there > are a number of other occurrences of "safe" that deserve similar treatmen= t ;- > ). > Beyond that, this assertion is dependent on the operating conditions stay= ing > within that reasonable operating envelope, which entails use of something > like a circuit breaker - I'd say that here, and reference the earlier > circuit breaker discussion. Global substitution: "safe" -> "reasonable", starting with the abstract. Then for this specific text on p.11: OLD Under this condition it is safe to place the TDM PW along with congestion - responsive traffic such as TCP, without causing additional congestion. NEW Under this condition it is reasonable to place the TDM PW in compet= ition with congestion-responsive traffic such as TCP, as the result should = not produce unacceptable congestion effects. That assumes that the opera= ting conditions remain within the reasonable operating envelope - as discu= ssed above, a transport circuit breaker is a suggested means of managing a= TDM PW so that it is shut down if operating conditions remain persistentl= y outside the reasonable operating envelope for the PW (e.g., due to a high packet loss rate). No need to go there. See my suggestions earlier. > Figures 7 and 8 could use some shading or lines to indicate the typical > deployed operating regions for the pseudo-wires to visibly show that they > fall under the curves, particularly for larger segment sizes. I'd also > repeat or refer to the suggestion to use larger I wanted to suggest this myself, but it is hard to depict fuzzy boundaries.= If you can do this, pls do. Given the PW operating regions are bounded by v low loss probability, the x= axis in Figs 7&8 ought to be scaled much smaller as well. Related point: the y axis in Fig 1 ought to max at 10, not 50. Regards Bob > > Nits: > > References aren't allowed in abstracts, so remove reference to > [I-D.fairhurst-tsvwg-circuit-breaker] > > Introduction, first paragraph, delete "PWE3" towards end of paragraph - > > preferably the 4 byte PWE3 control word. > > In addition to its current general statements, the Security Consideration= s > section should point to the security considerations sections in a few > particularly relevant RFCs - I'd suggest 3985, 4553, 5086 and 5087. After: This document does not introduce any new congestion-specific mechanisms and thus does not introduce any new security considerations above those present for PWs in general. I suggest adding a new paragraph: For PW security considerations, see the security considerations sections of the cited RFCS; the security considerations in [RFC3985], [RFC4553], [RFC5086] and [RFC5087] are particularly applicable to this document. > > Thanks, > --David > > > > -----Original Message----- > > From: pwe3 [ mailto:pwe3-bounces@ietf.org] On Behalf Of Yaakov Stein > > Sent: Thursday, July 24, 2014 4:52 PM > > To: pwe3@ietf.org > > Subject: Re: [PWE3] I-D Action: draft-ietf-pwe3-congcons-02.txt > > > > Please look at the PDF version only > > (the txt version does not have the graphs). > > > > Y(J)S > > > > > > _______________________________________________ > pwe3 mailing list > pwe3@ietf.org > https://www.ietf.org/mailman/listinfo/pwe3 _______________________________________________ pwe3 mailing list pwe3@ietf.org https://www.ietf.org/mailman/listinfo/pwe3 ________________________________________________________________ Bob Briscoe, BT --_000_8D3D17ACE214DC429325B2B98F3AE712077B95212CMX15Acorpemcc_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

And forwarding my views to the list ...

 

... On Bob’s six thoughts (I have= problems with 2 & 5 - the other 4 are basically ok w/me) plus follow-u= ps on Bob’s concerns w/my two major issues, here goes:

 =

-- 1 -- I agree in principl= e, but I think Bob’s suggested new text is too black/white and should= be generalized.

 

Bob suggests:

 

> The case where the service provider network arranges sufficie= nt capacity for a PW is well understood and

> will not be discussed further here. The= concern is over PWs deployed independently of the service provider

> network's traff= ic engineering or capacity planning.

 

I want to include the case where capacity has been arranged, = but may not be sufficient, I suggest rewriting Bob’s second sentence = as:

&= nbsp;

The c= oncern is over PWs that share network capacity with elastic or congestion-r= esponsive traffic, independent of whether that sharing is a result of the s= ervice provider network's traffic engineering and capacity planning or inde= pendent deployment of PWs by entities other than network operators.

 =

-- 2 -- Here, I disagree.&n= bsp; One of the significant contributions of this draft is that TDM PWs don= ’t cause congestion problems in practice, and one of the major reason= s for that is the low latency requirements for acceptable TDM service deliv= ery, both in ITU-T standards and in actual TDM PW deployment.  The 125= us behavior of native TDM switches is relevant to the latter, and hence I p= refer to include mention of it.

 

-- 3 -- I agree; clear captions are always a feature, no= t a bug ;-).

 

= -- 4 -- I agree with changing “packet loss rate” to “pack= et loss probability”.

 

 

<= p class=3DMsoNormal style=3D'margin-left:.5in'>I am concerned that S-A-F-E is l= ikely to be a 4-letter word in wider Transport Area review, and hence, I wa= nt to use a word that does not imply anything remotely resembling an absolu= te guarantee that nothing can possibly ever go wrong (yes, I know Bob didn&= #8217;t mean to go to quite that extreme an interpretation, but S-A-F-E is = liable to be read that way by those with an “axe to grind”), ev= en though there’s lots of solid operational experience that suggests = all is well here.

 

-- 6 -- I think I’m ok with what Bob proposes to better characte= rize loss burstiness, but want to see specific text.

<= p class=3DMsoNormal style=3D'margin-left:.5in'> 

-- Circuit Breaker Mechanism --

 

>> [Da= vid] I suggest adding:
>>
>>    For TDM pseudo= wires, such a circuit breaker can be realized via
>>    = operator monitoring of drop rates and consequent on the quality of
>&= gt;    TDM service delivered and shutting down TDM pseudowires th= at persistently
>>    fail to deliver acceptable TDM ser= vice (as defined by [G775] and
>>    [G826]).  This= document explains why a TDM PW that is causing congestion
>> &nbs= p;  problems via interference with elastic traffic cannot be deliverin= g
>>    acceptable TDM service (see the rest of this Sec= tion, plus Appendices
>>    A and B).

>>
> [Bob] I don't= think we should go there. We need to keep this draft firmly as information= al. If someone

> thinks a mechanism is needed because of the information in this draf= t, then they can write about a mechanism.

 

[David] I firmly disagree, but Bob and I might be = talking past each other, as I do want to stay informational ;-).=

 

This text is intended t= o describe an example or possibility, not to specify an implementation requ= irement.  The new text was intended as an OAM/OSS practice recommendat= ion to the operator, not as a normative protocol spec.

 

I’m ok with inserting ̶= 0;example”, “possibility” or something similar if that he= lps w/Bob’s concern, but I strongly oppose doing nothing here, and no= te that this is also needed to respond to one of Stewart Bryant’s WG = LC comments.

 

= -- PW segment size --

 

>> [David] Ignoring the rathole invitation ;-). = This is still in section 3, after:
>>
>>   We = see that in general for large packet sizes, short delays, and low
>&g= t;   packet loss rates, the TDM pseudowires consume much less ban= dwidth
>>    than TCP would under identical conditions.&= nbsp; Only for small packets,
>>    long delays, and hig= h packet loss ratios, do TDM PWs potentially
>>   consum= e more bandwidth, and even then only marginally.
>>
>> I= suggest adding:
>>
>>    Comparing figures 1 a= nd 2, use of a larger segment size for a T1 or E1
>>    = TDM PW moves the operational behavior of the PW away from the boundary
&= gt;>    at which the TCP throughput equation suggests that und= ue interference with
>>    elastic traffic may start to = occur.  We suggest that larger segment
>>    sizes = be preferred for T1 and E1 TDM PWs, but note that other
>>  &= nbsp; considerations may favor smaller segment sizes (e.g., jitter buffer>>    size, bounding the impact of an isolated packet dro= p event).

>= ;>
> [Bob] There is no need to recommend any remedial action, when= we have just shown there is not a problem.

 

[David] Not exactly - Figure 1 shows that there is a p= otential concern at a combination of the high end of the range of TDM PW la= tencies (4ms), and higher packet loss ratios. 

> Fur= ther, the "marginal problem" only applies when the TCP traffic us= es packets smaller than 128B, which we only used

> as a hypothetical worst-case for t= he analysis. But in practice this doesn't happen with long-running TCP flow= s -

> they use packets t= hat are as large as possible (irrespective of what the PW uses). So there r= eally is /no/ problem to remedy.

Ok, Bob can relax - his transport credentials are intact= , as this is an entry point to the bit-rate-congestible vs. packet-rate-con= gestible resource rathole that I want to ignore :-).

<= p class=3DMsoNormal style=3D'margin-left:.5in'> 

I believe that larger TDM PW segmen= t sizes are a good thing for both this operating envelope boundary concern = as well as per-packet processing overheads in TDM PW functionality, modulo = other considerations, e.g., jitter buffers, and so I prefer to keep this ad= dition.

 

Thanks,
--David

 

From: Bob Briscoe [mailto:bob.briscoe@bt.com]
Sent= : Thursday, August 14, 2014 9:03 AM
To: Yaakov Stein
Cc= : Andrew G. Malis; BOCCI Matthew; Black, David
Subject: RE: [= PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt

 

Yaakov,

I have just re-rea= d it thoroughly. I have noticed about two dozen editorial nits where correc= ting them would not change the intended sense (altho sometimes the sense ne= eds to change to say what we intended!). Examples:
- Unexpanded abbrevia= tions (David's suggestion would help, but there are more)
- Missing word= s, tautologies, random punctuation problems etc
- Uses of the wrong word= , even tho we know what you mean
- The caveat about dynamics vs steady s= tate needs to shift down a para, so that it follows into the para that says= what to do about dynamics.

Should I take the token, and edit the XM= L directly, then you can see a diff to check if you want to accept my sugge= stions?

Below are six thoughts that go slightly beyond 'editorial'. = 1-4 could be considered 'editorial'. But 5&6 are the more significant &= quot;techno-political" points.

1. The phrase "mechanisms = for provisioning PWs in service provider networks" is too opaque (the = PWE audience might understand this scoping sentence, but a L4 reader won't = understand what we're excluding here, which will make them suspicious).
= CURRENT:
   Mechanisms for provisioning PWs
   in= service provider networks are well understood and will not be
 &nb= sp; discussed further here.
SUGGESTED:
   The case where th= e service provider network arranges sufficient capacity for a PW is well un= derstood and will not be discussed further here. The concern is over PWs de= ployed independently of the service provider network's traffic engineering = or capacity planning.

2. The justification for using OWD of a few mi= lliseconds misses the point.
"  The one-way delay
 &nb= sp; of a native TDM service consists of the physical time-of-flight plus   125 microseconds for each TDM switch traversed; and is thus v= ery
   small as compared to typical PSN network-crossing laten= cies.
"
The 125us per switch is irrelevant. The point that needs= making is that PWs are generally deployed with a short physical time of fl= ight.

3. Related point: the captions on the graphs do not clearly in= dicate that they are OWD, not RTT. This could be clarified in the text, or = on the graphs.

4. Change packet loss rate (and PLR) to packet loss p= robability throughout, including on the graph axes. PLR is an ambiguous ter= m, even though it is widely used in the industry. Packet loss rate could be= measured in packet per second, whereas packet loss probability is unambigu= ously dimensionless. Indeed, in the appendices, packet loss rates with resp= ect to time are calculated.

5. Rather than David's "reasonable&= quot; I suggest we use the phrase "unambiguously safe" not just &= quot;safe" for the headings of Figs 5-8 and in the text, as below. Oth= erwise, we are implying that it is unsafe outside the region.

CURREN= T:
   One may view this condition as defining an operating env= elope for a
   TDM PW, as a TDM PW that occupies no more bandw= idth than a TCP flow
   causes no more congestion than that TC= P flow would.  Under this
   condition it is safe to plac= e the TDM PW along with congestion-
   responsive traffic such= as TCP, without causing additional
   congestion.  on th= e other hand, were the TDM PW to consume
   significantly more= bandwidth a TCP flow, it could contribute
   disproportionate= ly to congestion, and its mixture with congestion-
   responsi= ve traffic might be inappropriate.
SUGGESTED:
   One may vi= ew this condition as defining an unambiguously safe operating envelope for = a
   TDM PW, as a TDM PW that occupies no more bandwidth than = a TCP flow
   causes no more congestion than that TCP flow wou= ld.  This does not draw us into a debate over whether TCP-friendliness= is a valid constraint; it simply says that there can be no question that a= PW is safe if it does no more harm than a single TCP flow. Under this
&= nbsp;  condition it is unambiguously safe to place the TDM PW along wi= th congestion-
   responsive traffic such as TCP.  On the= other hand, were the TDM PW to consume
   significantly more = bandwidth a TCP flow, it could contribute
   disproportionatel= y to congestion, and its mixture with congestion-
   responsiv= e traffic might start to cause concern.

6. The analysis in Appendice= s A & B needs to mention that loss bursts arise primarily from tail dro= p buffers, and the introduction of active queue management (AQM) makes loss= es more independent of each other (less bursty). (I have references showing= evidence of this.) In the comparisons with TCP, we have assumed independen= t losses anyway, because they are the worst case (and fortunately they are = easier to analyse). On a similar point, in appendix B, where we say "(= bursty packet loss was also simulated but is not reported here)", we o= ught to give a reason otherwise readers will be suspicious. A good reason w= ould be that uniformly random is the worst case.

I've also responded= to a couple of David's suggestions inline (I agree with the others)...
=

At 17:44 13/08/2014, Yaakov Stein wrote:

Bob told me on Friday of the IETF that = he was going to look over the final version and comment.
 
Bob &= #8211; ping ?
 
Y(J)S
 
From: Andrew G. Malis = [ mailto:agmalis@gmail.com]
Sent: 13 August, 2014 17:02
To: Yaakov Stein
Cc: B= ob Briscoe; rbriscoe@jungle.bt.= co.uk; BOCCI Matthew; Black, David
Subject: Re: [PWE3] WGLC c= omments on: draft-ietf-pwe3-congcons-02.txt
 
Yaakov,
 <= br>I still haven't heard from Bob, but you should incorporate David's updat= es into a new revision, and I'll use that version for my final Shepherd's r= eview.
 
Thanks,
Andy
 
On Mon, Aug 11, 2014 at 11= :16 AM, Andrew G. Malis <agmalis@gm= ail.com> wrote:
Actually, I'm going to end the LC as soon as I he= ar back from Bob on the IPR call. I sent him a reminder earlier this mornin= g.
 
Cheers,
Andy
 
On Sun, Aug 10, 2014 at 9:53 A= M, Andrew G. Malis <agmalis@gmail.c= om> wrote:
David,
 
That's perfect, thanks! I'm going = to end the LC tomorrow.
 
Cheers,
Andy
 
On Sat, A= ug 9, 2014 at 10:32 PM, Black, David <david.black@emc.com> wrote:
Following up on this ...

= > As we're now moving quickly to try to complete this draft , I agreed> w/Yaakov that my author comments would be sent in as WG Last Call co= mments ...
> ... so here they are:
... with some suggested new tex= t ...

> Major issues:
>
> The reference to a transpor= t circuit breaker in section 3 is a bit optimistic
> and high level:<= br>>
>       We can accomplish this = by
>       employing a transport circui= t breaker, by which we mean an automatic
>    &nb= sp;  mechanism for terminating a flow to prevent negative impact on ot= her
>       flows and on the stability = of the network
>       [I-D.fairhurst-t= svwg-circuit-breaker].
>
> This should be accompanied by at lea= st an example of how to do this - past
> discussion has suggested car= rier monitoring of delivered service quality
> in order to shut down = PWs that are experiencing loss rates that result in
> unacceptable se= rvice quality (as defined for us by the ITU ...).  I'd also
> li= ke to add some text to suggest that operators use this sort of circuit
&= gt; breaker practice when TDM PWs compete with elastic traffic.
After th= is text in Section 3:

   We can accomplish this by
&nbs= p;  employing a transport circuit breaker, by which we mean an automat= ic
   mechanism for terminating a flow to prevent negative imp= act on other
   flows and on the stability of the network
&= nbsp;  [I-D.fairhurst-tsvwg-circuit-breaker].  Note that a transp= ort circuit
   breaker is intended as a protection mechanism o= f last resort, just as
   an electrical circuit breaker is onl= y triggered when absolutely
   necessary.

I suggest add= ing:

   For TDM pseudowires, such a circuit breaker can be= realized via
   operator monitoring of drop rates and consequ= ent on the quality of
   TDM service delivered and shutting do= wn TDM pseudowires that persistently
   fail to deliver accept= able TDM service (as defined by [G775] and
   [G826]).  T= his document explains why a TDM PW that is causing congestion
 &nbs= p; problems via interference with elastic traffic cannot be delivering
&= nbsp;  acceptable TDM service (see the rest of this Section, plus Appe= ndices
   A and B).


I don't think we should go there. We need to keep this draft fi= rmly as informational. If someone thinks a mechanism is needed because of t= he information in this draft, then they can write about a mechanism.

And bre= aking the paragraph after the added text.

> Figure 1 and Figure 2= indicate that based on using the TCP throughput
> equation as a prox= y for potential impact on elastic traffic (which is what
> we're doin= g in this draft), for the same fixed bandwidth PW, a larger segment
>= size is less likely to cause impacts.  I'd like to add a suggestion t= o use
> the larger PW segment sizes as long as they're consistent wit= h delivery of
> the service.  There's been a related (long-to-th= e-point-of-ratholed) debate
> over in the Transport Area about whethe= r packet-rate-limited/congestible
> forwarding is still relevant.&nbs= p; My view is that such equipment exists, and
> hence that this is a = relevant consideration for this draft, and would not
> mention that i= ssue here.
Ignoring the rathole invitation ;-).  This is still in s= ection 3, after:

   We see that in general for large packe= t sizes, short delays, and low
   packet loss rates, the TDM p= seudowires consume much less bandwidth
   than TCP would under= identical conditions.  Only for small packets,
   long d= elays, and high packet loss ratios, do TDM PWs potentially
   = consume more bandwidth, and even then only marginally.

I suggest add= ing:

   Comparing figures 1 and 2, use of a larger segment= size for a T1 or E1
   TDM PW moves the operational behavior = of the PW away from the boundary
   at which the TCP throughpu= t equation suggests that undue interference with
   elastic tr= affic may start to occur.  We suggest that larger segment
 &nb= sp; sizes be preferred for T1 and E1 TDM PWs, but note that other
 =   considerations may favor smaller segment sizes (e.g., jitter buffer<= br>   size, bounding the impact of an isolated packet drop event)= .


There is no need to r= ecommend any remedial action, when we have just shown there is not a proble= m.

Further, the "marginal problem" only applies when the T= CP traffic uses packets smaller than 128B, which we only used as a hypothet= ical worst-case for the analysis. But in practice this doesn't happen with = long-running TCP flows - they use packets that are as large as possible (ir= respective of what the PW uses). So there really is /no/ problem to remedy.=


= and again, breaking the paragraph after that added text.

I hope that= latter sentence ("We suggest that ...") is acceptable to the WG.=

> Minor issues:
>
> Section 2 is about Ethernet PWs = that carry elastic traffic.  Should others
> be mentioned as als= o covered by this analysis - e.g., ATM and Frame Relay
> PWs carrying= elastic traffic?
After:

   In this section we consider= Ethernet PWs that primarily carry
   congestion-responsive tr= affic.  We show that we automatically obtain
   the desir= ed congestion avoidance behavior, and that additional
   mecha= nisms are not needed.

I suggest adding:

   This dis= cussion is applicable to any PW that carries congestion-
   re= sponsive traffic (e.g., ATM and Frame Relay PWs); Ethernet PWs are
 = ;  used here for clarity, as they are a common example.

> Se= ction 2 does not cover Ethernet PWs carrying inelastic traffic - that would=
> be unusual, but should it be explicitly noted as out of scope, or = just not
> discussed (as is the case for the current text)?
I sugg= est doing nothing here, and planning to deal with this concern
if it sho= uld arise later (e.g., IETF Last Call or IESG Evaluation).

> Sect= ion 3 should say that the reason the draft focuses on TDM PWs is that
&g= t; they're the inelastic PWs that are likely to be carried over IP in pract= ice.
> My understanding is that other potentially inelastic PWs are M= PLS-only
> in practice.
After:

   Inelastic PWs, = such as TDM PWs ([RFC4553][RFC5086][RFC5087]), are
   potentia= lly more problematic than the elastic PWs of the previous
   s= ection.  Being constant bit-rate (CBR), TDM PWs can not be made
&nb= sp;  responsive to congestion.  On the other hand, being CBR, the= y also do
   not attempt to capture additional bandwidth when = neighboring TCP
   flows back off.

I suggest adding:
   We focus on TDM PWs here as they are the only inelastic P= Ws that are
   likely to be IP-encapsulated in practice. = Most other inelastic PWs
   tend to only be used with MPLS.
[I'd ask Yaakov and other PW experts to double-check the latter sente= nce,
please.]

> A terminology section to roll up all the acron= ym expansions (e.g., TDM, CBR)
> in one place seems like a good idea.=
These four acronyms would be good to put into a terminology section:
CBR Constant Bit Rate
PSN Packet Switched Network
PW Pseudowire<= br>TDM Time Division Multiplexing

>
> This text on p.11 cou= ld be incendiary:
>
>       Under= this condition it is safe to place the TDM PW along with
> &nbs= p;     congestion - responsive traffic such as TCP, wit= hout causing
>       additional congest= ion.
>
> First of all to reduce the flammability, "safe&qu= ot; -> "reasonable" and there
> are a number of other oc= currences of "safe" that deserve similar treatment ;-
> ).<= br>> Beyond that, this assertion is dependent on the operating condition= s staying
> within that reasonable operating envelope, which entails = use of something
> like a circuit breaker - I'd say that here, and re= ference the earlier
> circuit breaker discussion.
Global substitut= ion: "safe" -> "reasonable", starting with the abstr= act.

Then for this specific text on p.11:

OLD
  =       Under this condition it is safe to place the= TDM PW along with
        congestion= - responsive traffic such as TCP, without causing
   &nb= sp;    additional congestion.
NEW
   &n= bsp;    Under this condition it is reasonable to place the T= DM PW in competition
      with congestion-resp= onsive traffic such as TCP, as the result should not
   &= nbsp;  produce unacceptable congestion effects.  That assumes tha= t the operating
      conditions remain within = the reasonable operating envelope - as discussed
    = ;  above, a transport circuit breaker is a suggested means of managing= a TDM
      PW so that it is shut down if oper= ating conditions remain persistently
      outs= ide the reasonable operating envelope for the PW (e.g., due to a
 &= nbsp;    high packet loss rate).


No need to go there. See my suggestions earlier.

&g= t; Figures 7 and 8 could use some shading or lines to indicate the typical<= br>> deployed operating regions for the pseudo-wires to visibly show tha= t they
> fall under the curves, particularly for larger segment sizes= .  I'd also
> repeat or refer to the suggestion to use larger


I wanted to suggest this = myself, but it is hard to depict fuzzy boundaries. If you can do this, pls = do.

Given the PW operating regions are bounded by v low loss probabi= lity, the x axis in Figs 7&8 ought to be scaled much smaller as well. <= br>
Related point: the y axis in Fig 1 ought to max at 10, not 50.
Regards



Bob

>
> Nits:
>
> References aren'= t allowed in abstracts, so remove reference to
> [I-D.fairhurst-tsvwg= -circuit-breaker]
>
> Introduction, first paragraph, delete &qu= ot;PWE3" towards end of paragraph -
>
>    = preferably the 4 byte PWE3 control word.
>
> In addition to its= current general statements, the Security Considerations
> section sh= ould point to the security considerations sections in a few
> particu= larly relevant RFCs - I'd suggest 3985, 4553, 5086 and 5087.
After:
<= br>   This document does not introduce any new congestion-specifi= c
   mechanisms and thus does not introduce any new security   considerations above those present for PWs in general.
I suggest adding a new paragraph:

   For PW security con= siderations, see the security considerations sections
   of th= e cited RFCS; the security considerations in [RFC3985], [RFC4553],
 = ;  [RFC5086] and [RFC5087] are particularly applicable to this documen= t.

>
> Thanks,
> --David
>
>
> >= -----Original Message-----
> > From: pwe3 [ mailto:pwe3-bounces@ietf.org] On Behalf Of Yaakov S= tein
> > Sent: Thursday, July 24, 2014 4:52 PM
> > To: pwe3@ietf.org
> > Subject: Re: = [PWE3] I-D Action: draft-ietf-pwe3-congcons-02.txt
> >
> >= ; Please look at the PDF version only
> > (the txt version does no= t have the graphs).
> >
> > Y(J)S
> >
> &g= t;
>
> _______________________________________________
> = pwe3 mailing list
> pwe3@ietf.org
> https://ww= w.ietf.org/mailman/listinfo/pwe3

_______________________________= ________________
pwe3 mailing list
p= we3@ietf.org
= https://www.ietf.org/mailman/listinfo/pwe3
 
 
 = ;

_____________________________= ___________________________________
Bob Briscoe,    =             &nb= sp;            =             &nb= sp;        BT

= = --_000_8D3D17ACE214DC429325B2B98F3AE712077B95212CMX15Acorpemcc_-- From nobody Fri Aug 15 03:07:17 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5490A1A8A2F for ; Fri, 15 Aug 2014 03:07:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.268 X-Spam-Level: X-Spam-Status: No, score=-6.268 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, GB_I_INVITATION=-2, GB_I_LETTER=-2, GB_SUMOF=1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-0.668, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 94AGXM2Go4VD for ; Fri, 15 Aug 2014 03:07:07 -0700 (PDT) Received: from hubrelay-rd.bt.com (hubrelay-rd.bt.com [62.239.224.99]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2D35D1A8A0D for ; Fri, 15 Aug 2014 03:07:05 -0700 (PDT) Received: from EVMHR01-UKBR.domain1.systemhost.net (193.113.108.40) by EVMHR68-UKRD.bt.com (10.187.101.23) with Microsoft SMTP Server (TLS) id 8.3.348.2; Fri, 15 Aug 2014 11:07:03 +0100 Received: from EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) by EVMHR01-UKBR.domain1.systemhost.net (193.113.108.40) with Microsoft SMTP Server (TLS) id 8.3.348.2; Fri, 15 Aug 2014 11:07:02 +0100 Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) with Microsoft SMTP Server id 14.3.181.6; Fri, 15 Aug 2014 11:06:57 +0100 Received: from BTP075694.jungle.bt.co.uk ([10.111.229.151]) by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id s7FA6nWG003570; Fri, 15 Aug 2014 11:06:51 +0100 Message-ID: <201408151006.s7FA6nWG003570@bagheera.jungle.bt.co.uk> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Fri, 15 Aug 2014 11:03:14 +0100 To: "Black, David" From: Bob Briscoe In-Reply-To: <8D3D17ACE214DC429325B2B98F3AE712077B95212C@MX15A.corp.emc. com> References: <8D3D17ACE214DC429325B2B98F3AE712077B9517F4@MX15A.corp.emc.com> <8D3D17ACE214DC429325B2B98F3AE712077B951928@MX15A.corp.emc.com> <201408141302.s7ED2km0032238@bagheera.jungle.bt.co.uk> <8D3D17ACE214DC429325B2B98F3AE712077B952113@MX15A.corp.emc.com> <8D3D17ACE214DC429325B2B98F3AE712077B95212C@MX15A.corp.emc.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="=====================_855705482==.ALT" X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158 Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/VD90qgnm7X26p0wvGqnLaNAfVsk Cc: "pwe3@ietf.org" Subject: Re: [PWE3] FW: WGLC comments on: draft-ietf-pwe3-congcons-02.txt X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Aug 2014 10:07:16 -0000 --=====================_855705482==.ALT Content-Type: text/plain; charset="iso-8859-1"; format=flowed Content-Transfer-Encoding: quoted-printable David, I decided to think on a better word for 'safe'=20 overnight, and allow others to comment. But=20 no-one has, so here's my responses to all your outstanding points... I've provided slightly updated xml source of my=20 suggested edits (and a diff) here: Again note: xml is purely for precision, not=20 implying I have taken over editorship. 1. Happy with your new scoping text, except the=20 the first 'independent' clangs with the second, and it's unnecessary anyway: " The concern is over PWs that share network=20 capacity with elastic or congestion-responsive=20 traffic, [independent of] whether that sharing is=20 a result of the service provider network's=20 traffic engineering and capacity planning or=20 independent deployment of PWs by entities other than network operators. " 2. My point wasn't to remove the reference to the=20 125us (in my suggested text I didn't). It was=20 that the 125us stuff /alone/ doesn't warrant the=20 'thus' in the next clause; the logic needs to be=20 connected by talking about the sum of the flight=20 time /and/ the per-node processing time. CURRENT: The one-way delay of a native TDM service consists of the physical time-of-flight plus 125 microseconds for each TDM switch traversed; and is thus very small as compared to typical PSN network-crossing latencies. SUGGESTED: The one-way delay of a native TDM service consists of the physical time-of-flight plus 125 microseconds for each TDM switch=20 traversed; so the total one-way delay for most=20 TDM PWs has to be small compared to typical PSN network-crossing latencies. 5. I understand and agree - 'S-A-F-E region' is=20 incendiary. But I don't like 'reasonable region'=20 - it's slightly /too/ meaningless. Other possibilities are: - admissable region (problem: implies outside the region is inadmissable) - harmonious region (problem: too Californian) So I suggest two words: - friendly or acceptable, each used depending on context. On the graphs, I suggest 'operating region=20 (friendly under curve)' because that is precisely=20 where the curves come from - 'friendly' does what=20 it says on the tin, and the implied 'unfriendly'=20 above the curve correctly avoids implying that the sky will fall. Elsewhere, I suggest the following replacements for the S-word: "It is thus worthwhile specifying under what=20 conditions such competition is acceptable" "One may view this condition as defining a=20 'friendly' operating envelope for a TDM PW" "Under this condition it is acceptable to place=20 the TDM PW along with ... TCP. " "comparison with something widely considered to=20 offer an acceptable response to congestion" 7. Circuit breaker Scope creep. Rat-hole. Stop digging. Yes, I know Stewart has asked us to dig out=20 another rat as well, but the appropriate response is "Out of scope". You might think your text is uncontroversial, but=20 if we are tempted to take Stewart's bait (rat=20 metaphor intended), it will probably start new=20 debates that this doc doesn't deserve: "What does=20 persistent mean?", "Which traffic is more important/ valuable?", yada yada. I suggest the text merely makes it clear that=20 this debate is in scope for elsewhere, not here.=20 I've also just realised that Yaakov has written=20 the sentences around circuit breaker with the=20 wrong logical sequence. The circuit breaker needs=20 to be introduced as something the PW uses when=20 its /own/ quality has failed, which should avoid=20 any need to shut down because it is threatening elastic services: CURRENT: If the bandwidth consumed by a TDM PW is considered detrimental, the only available remedy is to completely shut down the PW, by using a transport circuit breaker mechanism. However, we will show that even before such an action is warranted, the PW will become unable to faithfully emulate the native TDM service; for example, when a TDM service is carrying voice grade telephony channels, the voice quality will degrade to below useful levels. SUGGESTED: We will show that, before a PW becomes=20 deterimental to elastic traffic, it will become=20 unable to faithfully emulate the native TDM service; for example, when a TDM service is carrying voice grade telephony channels, the voice quality will degrade to below useful levels. Once the quality of=20 the emulated service is persistently=20 unacceptable, if a PW shuts down selected flows=20 or shuts itself down completely, it will=20 automatically avoid detriment to elastic traffic.=20 A transport circuit breaker=20 [I-D.fairhurst-tsvwg-circuit-breaker] could be=20 used, but mechanism is beyond the scope of the present document. 8. Segment size. Everyone's in a big group-hug because we've=20 written a draft showing there's is no need to=20 change anything. Then just at the end of an=20 uneventful WGLC, we hear David's voice from the=20 bottom of a deep hole shouting, "avoid the=20 rat-hole invitation", while furiously digging for a rat. There is no need to add a recommendation that PWs=20 need to be changed to use larger packets. No=20 matter how hedged and soft this recommendation=20 would be, it's not needed anyway. Stop digging. Yaakov's new text is fine as it stands. "Only for=20 small packets, long delays, and high packet loss=20 ratios, do TDM PWs potentially consume more=20 bandwidth, and even then only marginally. " In order to justify your sentence, we would have=20 to include graphs with packet-rate (not bit-rate)=20 on the vertical axis and go into a whole treatise=20 as to why packet-rate comparison might be=20 important as well. Then the graph would show one=20 plot for all TCP segment sizes and multiple plots=20 for PWs with different packet sizes. Then we'd=20 see that the corner of the two smallest packet=20 PWs just clipped the TCP curve at loss=20 probabilities that are stupidly large anyway. So=20 even after all this, it would hardly justify a need for your sentence. All that digging would not be worth it, just to=20 discover (again) that there is no rat. Bob At 19:39 14/08/2014, Black, David wrote: >Content-Language: en-US >Content-Type: multipart/alternative; >=20 >boundary=3D"_000_8D3D17ACE214DC429325B2B98F3AE712077B95212CMX15Acorpemcc_" > >And forwarding my views to the list ... > >... On Bob=92s six thoughts (I have problems with=20 >2 & 5 - the other 4 are basically ok w/me) plus=20 >follow-ups on Bob=92s concerns w/my two major issues, here goes: > >-- 1 -- I agree in principle, but I think Bob=92s=20 >suggested new text is too black/white and should be generalized. > >Bob suggests: > > > The case where the service provider network=20 > arranges sufficient capacity for a PW is well understood and > > will not be discussed further here. The=20 > concern is over PWs deployed independently of the service provider > > network's traffic engineering or capacity planning. > >I want to include the case where capacity has=20 >been arranged, but may not be sufficient, I=20 >suggest rewriting Bob=92s second sentence as: > >The concern is over PWs that share network=20 >capacity with elastic or congestion-responsive=20 >traffic, independent of whether that sharing is=20 >a result of the service provider network's=20 >traffic engineering and capacity planning or=20 >independent deployment of PWs by entities other than network operators. > >-- 2 -- Here, I disagree. One of the=20 >significant contributions of this draft is that=20 >TDM PWs don=92t cause congestion problems in=20 >practice, and one of the major reasons for that=20 >is the low latency requirements for acceptable=20 >TDM service delivery, both in ITU-T standards=20 >and in actual TDM PW deployment. The 125us=20 >behavior of native TDM switches is relevant to=20 >the latter, and hence I prefer to include mention of it. > >-- 3 -- I agree; clear captions are always a feature, not a bug ;-). > >-- 4 -- I agree with changing =93packet loss rate=94 to =93packet loss= probability=94. > >-- 5 -- I firmly disagree with Bob here,=20 >although he=92s right that this is =93techno-political=94 ;-). > >I am concerned that S-A-F-E is likely to be a=20 >4-letter word in wider Transport Area review,=20 >and hence, I want to use a word that does not=20 >imply anything remotely resembling an absolute=20 >guarantee that nothing can possibly ever go=20 >wrong (yes, I know Bob didn=92t mean to go to=20 >quite that extreme an interpretation, but=20 >S-A-F-E is liable to be read that way by those=20 >with an =93axe to grind=94), even though there=92s=20 >lots of solid operational experience that suggests all is well here. > >-- 6 -- I think I=92m ok with what Bob proposes to=20 >better characterize loss burstiness, but want to see specific text. > >-- Circuit Breaker Mechanism -- > > >> [David] I suggest adding: > >> > >> For TDM pseudowires, such a circuit breaker can be realized via > >> operator monitoring of drop rates and consequent on the quality of > >> TDM service delivered and shutting down=20 > TDM pseudowires that persistently > >> fail to deliver acceptable TDM service (as defined by [G775] and > >> [G826]). This document explains why a=20 > TDM PW that is causing congestion > >> problems via interference with elastic traffic cannot be delivering > >> acceptable TDM service (see the rest of this Section, plus= Appendices > >> A and B). > >> > > [Bob] I don't think we should go there. We=20 > need to keep this draft firmly as informational. If someone > > thinks a mechanism is needed because of the=20 > information in this draft, then they can write about a mechanism. > >[David] I firmly disagree, but Bob and I might=20 >be talking past each other, as I do want to stay informational ;-). > >This text is intended to describe an example or=20 >possibility, not to specify an implementation=20 >requirement. The new text was intended as an=20 >OAM/OSS practice recommendation to the operator,=20 >not as a normative protocol spec. > >I=92m ok with inserting =93example=94, =93possibility=94=20 >or something similar if that helps w/Bob=92s=20 >concern, but I strongly oppose doing nothing=20 >here, and note that this is also needed to=20 >respond to one of Stewart Bryant=92s WG LC comments. > >-- PW segment size -- > > >> [David] Ignoring the rathole invitation=20 > ;-). This is still in section 3, after: > >> > >> We see that in general for large packet sizes, short delays, and low > >> packet loss rates, the TDM pseudowires consume much less bandwidth > >> than TCP would under identical conditions. Only for small packets, > >> long delays, and high packet loss ratios, do TDM PWs potentially > >> consume more bandwidth, and even then only marginally. > >> > >> I suggest adding: > >> > >> Comparing figures 1 and 2, use of a larger segment size for a T1 or= E1 > >> TDM PW moves the operational behavior of the PW away from the= boundary > >> at which the TCP throughput equation=20 > suggests that undue interference with > >> elastic traffic may start to occur. We suggest that larger segment > >> sizes be preferred for T1 and E1 TDM PWs, but note that other > >> considerations may favor smaller segment sizes (e.g., jitter buffer > >> size, bounding the impact of an isolated packet drop event). > >> > > [Bob] There is no need to recommend any=20 > remedial action, when we have just shown there is not a problem. > >[David] Not exactly - Figure 1 shows that there=20 >is a potential concern at a combination of the=20 >high end of the range of TDM PW latencies (4ms),=20 >and higher packet loss ratios. > > > Further, the "marginal problem" only applies=20 > when the TCP traffic uses packets smaller than 128B, which we only used > > as a hypothetical worst-case for the=20 > analysis. But in practice this doesn't happen with long-running TCP flows= - > > they use packets that are as large as=20 > possible (irrespective of what the PW uses). So=20 > there really is /no/ problem to remedy. >Ok, Bob can relax - his transport credentials=20 >are intact, as this is an entry point to the=20 >bit-rate-congestible vs. packet-rate-congestible=20 >resource rathole that I want to ignore :-). > >I believe that larger TDM PW segment sizes are a=20 >good thing for both this operating envelope=20 >boundary concern as well as per-packet=20 >processing overheads in TDM PW functionality,=20 >modulo other considerations, e.g., jitter=20 >buffers, and so I prefer to keep this addition. > >Thanks, >--David > >From: Bob Briscoe [mailto:bob.briscoe@bt.com] >Sent: Thursday, August 14, 2014 9:03 AM >To: Yaakov Stein >Cc: Andrew G. Malis; BOCCI Matthew; Black, David >Subject: RE: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt > >Yaakov, > >I have just re-read it thoroughly. I have=20 >noticed about two dozen editorial nits where=20 >correcting them would not change the intended=20 >sense (altho sometimes the sense needs to change=20 >to say what we intended!). Examples: >- Unexpanded abbreviations (David's suggestion would help, but there are= more) >- Missing words, tautologies, random punctuation problems etc >- Uses of the wrong word, even tho we know what you mean >- The caveat about dynamics vs steady state=20 >needs to shift down a para, so that it follows=20 >into the para that says what to do about dynamics. > >Should I take the token, and edit the XML=20 >directly, then you can see a diff to check if=20 >you want to accept my suggestions? > >Below are six thoughts that go slightly beyond=20 >'editorial'. 1-4 could be considered=20 >'editorial'. But 5&6 are the more significant "techno-political" points. > >1. The phrase "mechanisms for provisioning PWs=20 >in service provider networks" is too opaque (the=20 >PWE audience might understand this scoping=20 >sentence, but a L4 reader won't understand what=20 >we're excluding here, which will make them suspicious). >CURRENT: > Mechanisms for provisioning PWs > in service provider networks are well understood and will not be > discussed further here. >SUGGESTED: > The case where the service provider network=20 > arranges sufficient capacity for a PW is well=20 > understood and will not be discussed further=20 > here. The concern is over PWs deployed=20 > independently of the service provider network's=20 > traffic engineering or capacity planning. > >2. The justification for using OWD of a few milliseconds misses the point. >" The one-way delay > of a native TDM service consists of the physical time-of-flight plus > 125 microseconds for each TDM switch traversed; and is thus very > small as compared to typical PSN network-crossing latencies. >" >The 125us per switch is irrelevant. The point=20 >that needs making is that PWs are generally=20 >deployed with a short physical time of flight. > >3. Related point: the captions on the graphs do=20 >not clearly indicate that they are OWD, not RTT.=20 >This could be clarified in the text, or on the graphs. > >4. Change packet loss rate (and PLR) to packet=20 >loss probability throughout, including on the=20 >graph axes. PLR is an ambiguous term, even=20 >though it is widely used in the industry. Packet=20 >loss rate could be measured in packet per=20 >second, whereas packet loss probability is=20 >unambiguously dimensionless. Indeed, in the=20 >appendices, packet loss rates with respect to time are calculated. > >5. Rather than David's "reasonable" I suggest we=20 >use the phrase "unambiguously safe" not just=20 >"safe" for the headings of Figs 5-8 and in the=20 >text, as below. Otherwise, we are implying that=20 >it is unsafe outside the region. > >CURRENT: > One may view this condition as defining an operating envelope for a > TDM PW, as a TDM PW that occupies no more bandwidth than a TCP flow > causes no more congestion than that TCP flow would. Under this > condition it is safe to place the TDM PW along with congestion- > responsive traffic such as TCP, without causing additional > congestion. on the other hand, were the TDM PW to consume > significantly more bandwidth a TCP flow, it could contribute > disproportionately to congestion, and its mixture with congestion- > responsive traffic might be inappropriate. >SUGGESTED: > One may view this condition as defining an=20 > unambiguously safe operating envelope for a > TDM PW, as a TDM PW that occupies no more bandwidth than a TCP flow > causes no more congestion than that TCP flow=20 > would. This does not draw us into a debate=20 > over whether TCP-friendliness is a valid=20 > constraint; it simply says that there can be no=20 > question that a PW is safe if it does no more=20 > harm than a single TCP flow. Under this > condition it is unambiguously safe to place=20 > the TDM PW along with congestion- > responsive traffic such as TCP. On the=20 > other hand, were the TDM PW to consume > significantly more bandwidth a TCP flow, it could contribute > disproportionately to congestion, and its mixture with congestion- > responsive traffic might start to cause concern. > >6. The analysis in Appendices A & B needs to=20 >mention that loss bursts arise primarily from=20 >tail drop buffers, and the introduction of=20 >active queue management (AQM) makes losses more=20 >independent of each other (less bursty). (I have=20 >references showing evidence of this.) In the=20 >comparisons with TCP, we have assumed=20 >independent losses anyway, because they are the=20 >worst case (and fortunately they are easier to=20 >analyse). On a similar point, in appendix B,=20 >where we say "(bursty packet loss was also=20 >simulated but is not reported here)", we ought=20 >to give a reason otherwise readers will be=20 >suspicious. A good reason would be that uniformly random is the worst case. > >I've also responded to a couple of David's=20 >suggestions inline (I agree with the others)... > > >At 17:44 13/08/2014, Yaakov Stein wrote: >Bob told me on Friday of the IETF that he was=20 >going to look over the final version and comment. > >Bob =96 ping ? > >Y(J)S > >From: Andrew G. Malis [ mailto:agmalis@gmail.com] >Sent: 13 August, 2014 17:02 >To: Yaakov Stein >Cc: Bob Briscoe;=20 >rbriscoe@jungle.bt.co.uk;=20 >BOCCI Matthew; Black, David >Subject: Re: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt > >Yaakov, > >I still haven't heard from Bob, but you should=20 >incorporate David's updates into a new revision,=20 >and I'll use that version for my final Shepherd's review. > >Thanks, >Andy > >On Mon, Aug 11, 2014 at 11:16 AM, Andrew G.=20 >Malis <agmalis@gmail.com> wrote: >Actually, I'm going to end the LC as soon as I=20 >hear back from Bob on the IPR call. I sent him a reminder earlier this= morning. > >Cheers, >Andy > >On Sun, Aug 10, 2014 at 9:53 AM, Andrew G. Malis=20 ><agmalis@gmail.com> wrote: >David, > >That's perfect, thanks! I'm going to end the LC tomorrow. > >Cheers, >Andy > >On Sat, Aug 9, 2014 at 10:32 PM, Black, David=20 ><david.black@emc.com> wrote: >Following up on this ... > > > As we're now moving quickly to try to complete this draft , I agreed > > w/Yaakov that my author comments would be=20 > sent in as WG Last Call comments ... > > ... so here they are: >... with some suggested new text ... > > > Major issues: > > > > The reference to a transport circuit breaker=20 > in section 3 is a bit optimistic > > and high level: > > > > We can accomplish this by > > employing a transport circuit breaker, by which we mean an= automatic > > mechanism for terminating a flow to prevent negative impact on= other > > flows and on the stability of the network > > [I-D.fairhurst-tsvwg-circuit-breaker]. > > > > This should be accompanied by at least an example of how to do this -= past > > discussion has suggested carrier monitoring of delivered service quality > > in order to shut down PWs that are experiencing loss rates that result= in > > unacceptable service quality (as defined for us by the ITU ...). I'd= also > > like to add some text to suggest that operators use this sort of circuit > > breaker practice when TDM PWs compete with elastic traffic. >After this text in Section 3: > > We can accomplish this by > employing a transport circuit breaker, by which we mean an automatic > mechanism for terminating a flow to prevent negative impact on other > flows and on the stability of the network > [I-D.fairhurst-tsvwg-circuit-breaker]. Note that a transport circuit > breaker is intended as a protection mechanism of last resort, just as > an electrical circuit breaker is only triggered when absolutely > necessary. > >I suggest adding: > > For TDM pseudowires, such a circuit breaker can be realized via > operator monitoring of drop rates and consequent on the quality of > TDM service delivered and shutting down TDM pseudowires that= persistently > fail to deliver acceptable TDM service (as defined by [G775] and > [G826]). This document explains why a TDM PW that is causing= congestion > problems via interference with elastic traffic cannot be delivering > acceptable TDM service (see the rest of this Section, plus Appendices > A and B). > >I don't think we should go there. We need to=20 >keep this draft firmly as informational. If=20 >someone thinks a mechanism is needed because of=20 >the information in this draft, then they can write about a mechanism. > > >And breaking the paragraph after the added text. > > > Figure 1 and Figure 2 indicate that based on using the TCP throughput > > equation as a proxy for potential impact on elastic traffic (which is= what > > we're doing in this draft), for the same=20 > fixed bandwidth PW, a larger segment > > size is less likely to cause impacts. I'd like to add a suggestion to= use > > the larger PW segment sizes as long as they're consistent with delivery= of > > the service. There's been a related (long-to-the-point-of-ratholed)= debate > > over in the Transport Area about whether packet-rate-limited/congestible > > forwarding is still relevant. My view is that such equipment exists,= and > > hence that this is a relevant consideration for this draft, and would= not > > mention that issue here. >Ignoring the rathole invitation ;-). This is still in section 3, after: > > We see that in general for large packet sizes, short delays, and low > packet loss rates, the TDM pseudowires consume much less bandwidth > than TCP would under identical conditions. Only for small packets, > long delays, and high packet loss ratios, do TDM PWs potentially > consume more bandwidth, and even then only marginally. > >I suggest adding: > > Comparing figures 1 and 2, use of a larger segment size for a T1 or E1 > TDM PW moves the operational behavior of the PW away from the boundary > at which the TCP throughput equation suggests that undue interference= with > elastic traffic may start to occur. We suggest that larger segment > sizes be preferred for T1 and E1 TDM PWs, but note that other > considerations may favor smaller segment sizes (e.g., jitter buffer > size, bounding the impact of an isolated packet drop event). > >There is no need to recommend any remedial=20 >action, when we have just shown there is not a problem. > >Further, the "marginal problem" only applies=20 >when the TCP traffic uses packets smaller than=20 >128B, which we only used as a hypothetical=20 >worst-case for the analysis. But in practice=20 >this doesn't happen with long-running TCP flows=20 >- they use packets that are as large as possible=20 >(irrespective of what the PW uses). So there really is /no/ problem to= remedy. > > >and again, breaking the paragraph after that added text. > >I hope that latter sentence ("We suggest that ...") is acceptable to the= WG. > > > Minor issues: > > > > Section 2 is about Ethernet PWs that carry elastic traffic. Should= others > > be mentioned as also covered by this analysis - e.g., ATM and Frame= Relay > > PWs carrying elastic traffic? >After: > > In this section we consider Ethernet PWs that primarily carry > congestion-responsive traffic. We show that we automatically obtain > the desired congestion avoidance behavior, and that additional > mechanisms are not needed. > >I suggest adding: > > This discussion is applicable to any PW that carries congestion- > responsive traffic (e.g., ATM and Frame Relay PWs); Ethernet PWs are > used here for clarity, as they are a common example. > > > Section 2 does not cover Ethernet PWs=20 > carrying inelastic traffic - that would > > be unusual, but should it be explicitly noted as out of scope, or just= not > > discussed (as is the case for the current text)? >I suggest doing nothing here, and planning to deal with this concern >if it should arise later (e.g., IETF Last Call or IESG Evaluation). > > > Section 3 should say that the reason the draft focuses on TDM PWs is= that > > they're the inelastic PWs that are likely to=20 > be carried over IP in practice. > > My understanding is that other potentially inelastic PWs are MPLS-only > > in practice. >After: > > Inelastic PWs, such as TDM PWs ([RFC4553][RFC5086][RFC5087]), are > potentially more problematic than the elastic PWs of the previous > section. Being constant bit-rate (CBR), TDM PWs can not be made > responsive to congestion. On the other hand, being CBR, they also do > not attempt to capture additional bandwidth when neighboring TCP > flows back off. > >I suggest adding: > > We focus on TDM PWs here as they are the only inelastic PWs that are > likely to be IP-encapsulated in practice. Most other inelastic PWs > tend to only be used with MPLS. > >[I'd ask Yaakov and other PW experts to double-check the latter sentence, >please.] > > > A terminology section to roll up all the=20 > acronym expansions (e.g., TDM, CBR) > > in one place seems like a good idea. >These four acronyms would be good to put into a terminology section: > >CBR Constant Bit Rate >PSN Packet Switched Network >PW Pseudowire >TDM Time Division Multiplexing > > > > > This text on p.11 could be incendiary: > > > > Under this condition it is safe to place the TDM PW along with > > congestion - responsive traffic such as TCP, without causing > > additional congestion. > > > > First of all to reduce the flammability, "safe" -> "reasonable" and= there > > are a number of other occurrences of "safe"=20 > that deserve similar treatment ;- > > ). > > Beyond that, this assertion is dependent on=20 > the operating conditions staying > > within that reasonable operating envelope, which entails use of= something > > like a circuit breaker - I'd say that here, and reference the earlier > > circuit breaker discussion. >Global substitution: "safe" -> "reasonable", starting with the abstract. > >Then for this specific text on p.11: > >OLD > Under this condition it is safe to place the TDM PW along with > congestion - responsive traffic such as TCP, without causing > additional congestion. >NEW > Under this condition it is reasonable=20 > to place the TDM PW in competition > with congestion-responsive traffic such=20 > as TCP, as the result should not > produce unacceptable congestion=20 > effects. That assumes that the operating > conditions remain within the reasonable=20 > operating envelope - as discussed > above, a transport circuit breaker is a=20 > suggested means of managing a TDM > PW so that it is shut down if operating conditions remain= persistently > outside the reasonable operating envelope for the PW (e.g., due to a > high packet loss rate). > >No need to go there. See my suggestions earlier. > > > > Figures 7 and 8 could use some shading or lines to indicate the typical > > deployed operating regions for the pseudo-wires to visibly show that= they > > fall under the curves, particularly for larger segment sizes. I'd also > > repeat or refer to the suggestion to use larger > >I wanted to suggest this myself, but it is hard=20 >to depict fuzzy boundaries. If you can do this, pls do. > >Given the PW operating regions are bounded by v=20 >low loss probability, the x axis in Figs 7&8=20 >ought to be scaled much smaller as well. > >Related point: the y axis in Fig 1 ought to max at 10, not 50. > >Regards > > > >Bob > > > > > Nits: > > > > References aren't allowed in abstracts, so remove reference to > > [I-D.fairhurst-tsvwg-circuit-breaker] > > > > Introduction, first paragraph, delete "PWE3" towards end of paragraph - > > > > preferably the 4 byte PWE3 control word. > > > > In addition to its current general statements, the Security= Considerations > > section should point to the security considerations sections in a few > > particularly relevant RFCs - I'd suggest 3985, 4553, 5086 and 5087. >After: > > This document does not introduce any new congestion-specific > mechanisms and thus does not introduce any new security > considerations above those present for PWs in general. > >I suggest adding a new paragraph: > > For PW security considerations, see the security considerations= sections > of the cited RFCS; the security considerations in [RFC3985], [RFC4553], > [RFC5086] and [RFC5087] are particularly applicable to this document. > > > > > Thanks, > > --David > > > > > > > -----Original Message----- > > > From: pwe3 [ mailto:pwe3-bounces@ietf.org] On Behalf Of Yaakov Stein > > > Sent: Thursday, July 24, 2014 4:52 PM > > > To: pwe3@ietf.org > > > Subject: Re: [PWE3] I-D Action: draft-ietf-pwe3-congcons-02.txt > > > > > > Please look at the PDF version only > > > (the txt version does not have the graphs). > > > > > > Y(J)S > > > > > > > > > > _______________________________________________ > > pwe3 mailing list > > pwe3@ietf.org > > https://www.ietf.org/mailman/listinfo/pwe3 > >_______________________________________________ >pwe3 mailing list >pwe3@ietf.org >https://www.ietf.org/mailman/listinfo/pwe3 > > > > >________________________________________________________________ >Bob Briscoe, BT >_______________________________________________ >pwe3 mailing list >pwe3@ietf.org >https://www.ietf.org/mailman/listinfo/pwe3 ________________________________________________________________ Bob Briscoe, BT=20 --=====================_855705482==.ALT Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable David,

I decided to think on a better word for 'safe' overnight, and allow others to comment. But no-one has, so here's my responses to all your outstanding points...

I've provided slightly updated xml source of my suggested edits (and a diff) here:
< http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-pwe3-c= ongcons-DIFF03b-02.html >
< http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-pwe3-c= ongcons-03b.txt >
< http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-pwe3-c= ongcons-03b.xml >
Again note: xml is purely for precision, not implying I have taken over editorship.

1. Happy with your new scoping text, except the the first 'independent' clangs with the second, and it's unnecessary anyway:
"  The concern is over PWs that share network capacity with elastic or congestion-responsive traffic, [independent of] whether that sharing is a result of the service provider network's traffic engineering and capacity planning or independent deployment of PWs by entities other than network operators.
"

2. My point wasn't to remove the reference to the 125us (in my suggested text I didn't). It was that the 125us stuff /alone/ doesn't warrant the 'thus' in the next clause; the logic needs to be connected by talking about the sum of the flight time /and/ the per-node processing time.
CURRENT:
   The one-way delay
   of a native TDM service consists of the physical time-of-flight plus
   125 microseconds for each TDM switch traversed; and is thus very
   small as compared to typical PSN network-crossing latencies.
SUGGESTED:
   The one-way delay
   of a native TDM service consists of the physical time-of-flight plus
   125 microseconds for each TDM switch traversed; so the total one-way delay for most TDM PWs has to be small compared to typical PSN network-crossing latencies.

5. I understand and agree - 'S-A-F-E region' is incendiary. But I don't like 'reasonable region' - it's slightly /too/ meaningless.

Other possibilities are:
- admissable region (problem: implies outside the region is inadmissable)
- harmonious region (problem: too Californian)
So I suggest two words:
- friendly or acceptable, each used depending on context.

On the graphs, I suggest 'operating region (friendly under curve)' because that is precisely where the curves come from - 'friendly' does what it says on the tin, and the implied 'unfriendly' above the curve correctly avoids implying that the sky will fall.

Elsewhere, I suggest the following replacements for the S-word:
"It is thus worthwhile specifying under what conditions such competition is acceptable"
"One may view this condition as defining a 'friendly' operating envelope for a TDM PW"
"Under this condition it is acceptable to place the TDM PW along with ... TCP. "
"comparison with something widely considered to offer an acceptable response to congestion"


7. Circuit breaker
Scope creep. Rat-hole. Stop digging.
Yes, I know Stewart has asked us to dig out another rat as well, but the appropriate response is "Out of scope".

You might think your text is uncontroversial, but if we are tempted to take Stewart's bait (rat metaphor intended), it will probably start new debates that this doc doesn't deserve: "What does persistent mean?", "Which traffic is more important/ valuable?", yada yada.

I suggest the text merely makes it clear that this debate is in scope for elsewhere, not here. I've also just realised that Yaakov has written the sentences around circuit breaker with the wrong logical sequence. The circuit breaker needs to be introduced as something the PW uses when its /own/ quality has failed, which should avoid any need to shut down because it is threatening elastic services:

CURRENT:
      If the bandwidth consumed by a TDM PW is
      considered detrimental, the only available remedy is to completely
      shut down the PW, by using a transport circuit breaker mechanism.
      However, we will show that even before such an action is
      warranted, the PW will become unable to faithfully emulate the
      native TDM service; for example, when a TDM service is carrying
      voice grade telephony channels, the voice quality will degrade to
      below useful levels.
SUGGESTED:
      We will show that, before a PW becomes deterimental to elastic traffic, it will become unable to faithfully emulate the
      native TDM service; for example, when a TDM service is carrying
      voice grade telephony channels, the voice quality will degrade to
      below useful levels. Once the quality of the emulated service is persistently unacceptable, if a PW shuts down selected flows or shuts itself down completely, it will automatically avoid detriment to elastic traffic. A transport circuit breaker [I-D.fairhurst-tsvwg-circuit-breaker] could be used, but mechanism is beyond the scope of the present document.


8. Segment size.
Everyone's in a big group-hug because we've written a draft showing there's is no need to change anything. Then just at the end of an uneventful WGLC, we hear David's voice from the bottom of a deep hole shouting, "avoid the rat-hole invitation", while furiously digging for a rat.

There is no need to add a recommendation that PWs need to be changed to use larger packets. No matter how hedged and soft this recommendation would be, it's not needed anyway. Stop digging.

Yaakov's new text is fine as it stands. "Only for small packets, long delays, and high packet loss ratios, do TDM PWs potentially consume more bandwidth, and even then only marginally. "

In order to justify your sentence, we would have to include graphs with packet-rate (not bit-rate) on the vertical axis and go into a whole treatise as to why packet-rate comparison might be important as well. Then the graph would show one plot for all TCP segment sizes and multiple plots for PWs with different packet sizes. Then we'd see that the corner of the two smallest packet PWs just clipped the TCP curve at loss probabilities that are stupidly large anyway. So even after all this, it would hardly justify a need for your sentence.

All that digging would not be worth it, just to discover (again) that there is no rat.


Bob


At 19:39 14/08/2014, Black, David wrote:
Content-Language: en-US
Content-Type: multipart/alternative;
         boundary=3D"_000_8D3D17ACE214DC429325B2B98F3AE712077B95212CMX15Acorpemc= c_"

And forwarding my views to the list ...
 
... On Bob=92s six thoughts (I have problems with 2 & 5 - the other 4 are basically ok w/me) plus follow-ups on Bob=92s concerns w/my two major issues, here goes:
 
-- 1 -- I agree in principle, but I think Bob=92s suggested new text is too black/white and should be generalized.
 
Bob suggests:
 
> The case where the service provider network arranges sufficient capacity for a PW is well understood and
> will not be discussed further here. The concern is over PWs deployed independently of the service provider
> network's traffic engineering or capacity planning.
 
I want to include the case where capacity has been arranged, but may not be sufficient, I suggest rewriting Bob=92s second sentence as:
 
The concern is over PWs that share network capacity with elastic or congestion-responsive traffic, independent of whether that sharing is a result of the service provider network's traffic engineering and capacity planning or independent deployment of PWs by entities other than network operators.
 
-- 2 -- Here, I disagree.  One of the significant contributions of this draft is that TDM PWs don=92t cause congestion problems in practice, and one of the major reasons for that is the low latency requirements for acceptable TDM service delivery, both in ITU-T standards and in actual TDM PW deployment.  The 125us behavior of native TDM switches is relevant to the latter, and hence I prefer to include mention of it.
 
-- 3 -- I agree; clear captions are always a feature, not a bug ;-).
 
-- 4 -- I agree with changing =93packet loss rate=94 to =93packet loss probability=94.
 
-- 5 -- I firmly disagree with Bob here, although he=92s right that this is =93techno-political=94 ;-).
 
I am concerned that S-A-F-E is likely to be a 4-letter word in wider Transport Area review, and hence, I want to use a word that does not imply anything remotely resembling an absolute guarantee that nothing can possibly ever go wrong (yes, I know Bob didn=92t mean to go to quite that extreme an interpretation, but S-A-F-E is liable to be read that way by those with an =93axe to grind=94), even though there=92s lots of solid operational experience that suggests all is well here.
 
-- 6 -- I think I=92m ok with what Bob proposes to better characterize loss burstiness, but want to see specific text.
 
-- Circuit Breaker Mechanism --
 
>> [David] I suggest adding:
>>
>>    For TDM pseudowires, such a circuit breaker can be realized via
>>    operator monitoring of drop rates and consequent on the quality of
>>    TDM service delivered and shutting down TDM pseudowires that persistently
>>    fail to deliver acceptable TDM service (as defined by [G775] and
>>    [G826]).  This document explains why a TDM PW that is causing congestion
>>    problems via interference with elastic traffic cannot be delivering
>>    acceptable TDM service (see the rest of this Section, plus Appendices
>>    A and B).
>>
> [Bob] I don't think we should go there. We need to keep this draft firmly as informational. If someone
> thinks a mechanism is needed because of the information in this draft, then they can write about a mechanism.
 
[David] I firmly disagree, but Bob and I might be talking past each other, as I do want to stay informational ;-).
 
This text is intended to describe an example or possibility, not to specify an implementation requirement.  The new text was intended as an OAM/OSS practice recommendation to the operator, not as a normative protocol spec.
 
I=92m ok with inserting =93example=94, =93possibility=94 or something= similar if that helps w/Bob=92s concern, but I strongly oppose doing nothing here, and note that this is also needed to respond to one of Stewart Bryant=92s WG LC comments.
 
-- PW segment size --
 
>> [David] Ignoring the rathole invitation ;-).  This is still in section 3, after:
>>
>>   We see that in general for large packet sizes, short delays, and low
>>   packet loss rates, the TDM pseudowires consume much less bandwidth
>>    than TCP would under identical conditions.  Only for small packets,
>>    long delays, and high packet loss ratios, do TDM PWs potentially
>>   consume more bandwidth, and even then only marginally.
>>
>> I suggest adding:
>>
>>    Comparing figures 1 and 2, use of a larger segment size for a T1 or E1
>>    TDM PW moves the operational behavior of the PW away from the boundary
>>    at which the TCP throughput equation suggests that undue interference with
>>    elastic traffic may start to occur.  We suggest that larger segment
>>    sizes be preferred for T1 and E1 TDM PWs, but note that other
>>    considerations may favor smaller segment sizes (e.g., jitter buffer
>>    size, bounding the impact of an isolated packet drop event).
>>
> [Bob] There is no need to recommend any remedial action, when we have just shown there is not a problem.
 
[David] Not exactly - Figure 1 shows that there is a potential concern at a combination of the high end of the range of TDM PW latencies (4ms), and higher packet loss ratios. 

> Further, the "marginal problem" only applies when the TCP traffic uses packets smaller than 128B, which we only used
> as a hypothetical worst-case for the analysis. But in practice this doesn't happen with long-running TCP flows -
> they use packets that are as large as possible (irrespective of what the PW uses). So there really is /no/ problem to remedy.
Ok, Bob can relax - his transport credentials are intact, as this is an entry point to the bit-rate-congestible vs. packet-rate-congestible resource rathole that I want to ignore :-).
 
I believe that larger TDM PW segment sizes are a good thing for both this operating envelope boundary concern as well as per-packet processing overheads in TDM PW functionality, modulo other considerations, e.g., jitter buffers, and so I prefer to keep this addition.
 
Thanks,
--David
 
From: Bob Briscoe [mailto:bob.briscoe@bt.com]
Sent: Thursday, August 14, 2014 9:03 AM
To: Yaakov Stein
Cc: Andrew G. Malis; BOCCI Matthew; Black, David
Subject: RE: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt
 
Yaakov,

I have just re-read it thoroughly. I have noticed about two dozen editorial nits where correcting them would not change the intended sense (altho sometimes the sense needs to change to say what we intended!). Examples:
- Unexpanded abbreviations (David's suggestion would help, but there are more)
- Missing words, tautologies, random punctuation problems etc
- Uses of the wrong word, even tho we know what you mean
- The caveat about dynamics vs steady state needs to shift down a para, so that it follows into the para that says what to do about dynamics.

Should I take the token, and edit the XML directly, then you can see a diff to check if you want to accept my suggestions?

Below are six thoughts that go slightly beyond 'editorial'. 1-4 could be considered 'editorial'. But 5&6 are the more significant "techno-political" points.

1. The phrase "mechanisms for provisioning PWs in service provider networks" is too opaque (the PWE audience might understand this scoping sentence, but a L4 reader won't understand what we're excluding here, which will make them suspicious).
CURRENT:
   Mechanisms for provisioning PWs
   in service provider networks are well understood and will not be
   discussed further here.
SUGGESTED:
   The case where the service provider network arranges sufficient capacity for a PW is well understood and will not be discussed further here. The concern is over PWs deployed independently of the service provider network's traffic engineering or capacity planning.

2. The justification for using OWD of a few milliseconds misses the point.
"  The one-way delay
   of a native TDM service consists of the physical time-of-flight plus
   125 microseconds for each TDM switch traversed; and is thus very
   small as compared to typical PSN network-crossing latencies.
"
The 125us per switch is irrelevant. The point that needs making is that PWs are generally deployed with a short physical time of flight.

3. Related point: the captions on the graphs do not clearly indicate that they are OWD, not RTT. This could be clarified in the text, or on the graphs.

4. Change packet loss rate (and PLR) to packet loss probability throughout, including on the graph axes. PLR is an ambiguous term, even though it is widely used in the industry. Packet loss rate could be measured in packet per second, whereas packet loss probability is unambiguously dimensionless. Indeed, in the appendices, packet loss rates with respect to time are calculated.

5. Rather than David's "reasonable" I suggest we use the phrase "unambiguously safe" not just "safe" for the headings of Figs 5-8 and in the text, as below. Otherwise, we are implying that it is unsafe outside the region.

CURRENT:
   One may view this condition as defining an operating envelope for a
   TDM PW, as a TDM PW that occupies no more bandwidth than a TCP flow
   causes no more congestion than that TCP flow would.  Under this
   condition it is safe to place the TDM PW along with congestion-
   responsive traffic such as TCP, without causing additional
   congestion.  on the other hand, were the TDM PW to consume
   significantly more bandwidth a TCP flow, it could contribute
   disproportionately to congestion, and its mixture with congestion-
   responsive traffic might be inappropriate.
SUGGESTED:
   One may view this condition as defining an unambiguously safe operating envelope for a
   TDM PW, as a TDM PW that occupies no more bandwidth than a TCP flow
   causes no more congestion than that TCP flow would.  This does not draw us into a debate over whether TCP-friendliness is a valid constraint; it simply says that there can be no question that a PW is safe if it does no more harm than a single TCP flow. Under this
   condition it is unambiguously safe to place the TDM PW along with congestion-
   responsive traffic such as TCP.  On the other hand, were the TDM PW to consume
   significantly more bandwidth a TCP flow, it could contribute
   disproportionately to congestion, and its mixture with congestion-
   responsive traffic might start to cause concern.

6. The analysis in Appendices A & B needs to mention that loss bursts arise primarily from tail drop buffers, and the introduction of active queue management (AQM) makes losses more independent of each other (less bursty). (I have references showing evidence of this.) In the comparisons with TCP, we have assumed independent losses anyway, because they are the worst case (and fortunately they are easier to analyse). On a similar point, in appendix B, where we say "(bursty packet loss was also simulated but is not reported here)", we ought to give a reason otherwise readers will be suspicious. A good reason would be that uniformly random is the worst case.

I've also responded to a couple of David's suggestions inline (I agree with the others)...


At 17:44 13/08/2014, Yaakov Stein wrote:
Bob told me on Friday of the IETF that he was going to look over the final version and comment.
 
Bob =96 ping ?
 
Y(J)S
 
From: Andrew G. Malis [ mailto:agmalis@gmail.com]
Sent: 13 August, 2014 17:02
To: Yaakov Stein
Cc: Bob Briscoe; rbriscoe@jungle.bt.co.uk; BOCCI Matthew; Black, David
Subject: Re: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt
 
Yaakov,
 
I still haven't heard from Bob, but you should incorporate David's updates into a new revision, and I'll use that version for my final Shepherd's review.
 
Thanks,
Andy
 
On Mon, Aug 11, 2014 at 11:16 AM, Andrew G. Malis <agmalis@gmail.com> wrote:
Actually, I'm going to end the LC as soon as I hear back from Bob on the IPR call. I sent him a reminder earlier this morning.
 
Cheers,
Andy
 
On Sun, Aug 10, 2014 at 9:53 AM, Andrew G. Malis <agmalis@gmail.com> wrote:
David,
 
That's perfect, thanks! I'm going to end the LC tomorrow.
 
Cheers,
Andy
 
On Sat, Aug 9, 2014 at 10:32 PM, Black, David <david.black@emc.com> wrote:
Following up on this ...

> As we're now moving quickly to try to complete this draft , I agreed
> w/Yaakov that my author comments would be sent in as WG Last Call comments ...
> ... so here they are:
... with some suggested new text ...

> Major issues:
>
> The reference to a transport circuit breaker in section 3 is a bit optimistic
> and high level:
>
>       We can accomplish this by
>       employing a transport circuit breaker, by which we mean an automatic
>       mechanism for terminating a flow to prevent negative impact on other
>       flows and on the stability of the network
>       [I-D.fairhurst-tsvwg-circuit-breaker].
>
> This should be accompanied by at least an example of how to do this - past
> discussion has suggested carrier monitoring of delivered service quality
> in order to shut down PWs that are experiencing loss rates that result in
> unacceptable service quality (as defined for us by the ITU ...).  I'd also
> like to add some text to suggest that operators use this sort of circuit
> breaker practice when TDM PWs compete with elastic traffic.
After this text in Section 3:

   We can accomplish this by
   employing a transport circuit breaker, by which we mean an automatic
   mechanism for terminating a flow to prevent negative impact on other
   flows and on the stability of the network
   [I-D.fairhurst-tsvwg-circuit-breaker].  Note that a transport circuit
   breaker is intended as a protection mechanism of last resort, just as
   an electrical circuit breaker is only triggered when absolutely
   necessary.

I suggest adding:

   For TDM pseudowires, such a circuit breaker can be realized via
   operator monitoring of drop rates and consequent on the quality of
   TDM service delivered and shutting down TDM pseudowires that persistently
   fail to deliver acceptable TDM service (as defined by [G775] and
   [G826]).  This document explains why a TDM PW that is causing congestion
   problems via interference with elastic traffic cannot be delivering
   acceptable TDM service (see the rest of this Section, plus Appendices
   A and B).

I don't think we should go there. We need to keep this draft firmly as informational. If someone thinks a mechanism is needed because of the information in this draft, then they can write about a mechanism.


And breaking the paragraph after the added text.

> Figure 1 and Figure 2 indicate that based on using the TCP throughput
> equation as a proxy for potential impact on elastic traffic (which is what
> we're doing in this draft), for the same fixed bandwidth PW, a larger segment
> size is less likely to cause impacts.  I'd like to add a suggestion to use
> the larger PW segment sizes as long as they're consistent with delivery of
> the service.  There's been a related (long-to-the-point-of-ratholed) debate
> over in the Transport Area about whether packet-rate-limited/congestible
> forwarding is still relevant.  My view is that such equipment exists, and
> hence that this is a relevant consideration for this draft, and would not
> mention that issue here.
Ignoring the rathole invitation ;-).  This is still in section 3, after:

   We see that in general for large packet sizes, short delays, and low
   packet loss rates, the TDM pseudowires consume much less bandwidth
   than TCP would under identical conditions.  Only for small packets,
   long delays, and high packet loss ratios, do TDM PWs potentially
   consume more bandwidth, and even then only marginally.

I suggest adding:

   Comparing figures 1 and 2, use of a larger segment size for a T1 or E1
   TDM PW moves the operational behavior of the PW away from the boundary
   at which the TCP throughput equation suggests that undue interference with
   elastic traffic may start to occur.  We suggest that larger segment
   sizes be preferred for T1 and E1 TDM PWs, but note that other
   considerations may favor smaller segment sizes (e.g., jitter buffer
   size, bounding the impact of an isolated packet drop event).

There is no need to recommend any remedial action, when we have just shown there is not a problem.

Further, the "marginal problem" only applies when the TCP traffic uses packets smaller than 128B, which we only used as a hypothetical worst-case for the analysis. But in practice this doesn't happen with long-running TCP flows - they use packets that are as large as possible (irrespective of what the PW uses). So there really is /no/ problem to remedy.


and again, breaking the paragraph after that added text.

I hope that latter sentence ("We suggest that ...") is acceptable to the WG.

> Minor issues:
>
> Section 2 is about Ethernet PWs that carry elastic traffic.  Should others
> be mentioned as also covered by this analysis - e.g., ATM and Frame Relay
> PWs carrying elastic traffic?
After:

   In this section we consider Ethernet PWs that primarily carry
   congestion-responsive traffic.  We show that we automatically obtain
   the desired congestion avoidance behavior, and that additional
   mechanisms are not needed.

I suggest adding:

   This discussion is applicable to any PW that carries congestion-
   responsive traffic (e.g., ATM and Frame Relay PWs); Ethernet PWs are
   used here for clarity, as they are a common example.

> Section 2 does not cover Ethernet PWs carrying inelastic traffic - that would
> be unusual, but should it be explicitly noted as out of scope, or just not
> discussed (as is the case for the current text)?
I suggest doing nothing here, and planning to deal with this concern
if it should arise later (e.g., IETF Last Call or IESG Evaluation).

> Section 3 should say that the reason the draft focuses on TDM PWs is that
> they're the inelastic PWs that are likely to be carried over IP in practice.
> My understanding is that other potentially inelastic PWs are MPLS-only
> in practice.
After:

   Inelastic PWs, such as TDM PWs ([RFC4553][RFC5086][RFC5087]), are
   potentially more problematic than the elastic PWs of the previous
   section.  Being constant bit-rate (CBR), TDM PWs can not be made
   responsive to congestion.  On the other hand, being CBR, they also do
   not attempt to capture additional bandwidth when neighboring TCP
   flows back off.

I suggest adding:

   We focus on TDM PWs here as they are the only inelastic PWs that are
   likely to be IP-encapsulated in practice.  Most other inelastic PWs
   tend to only be used with MPLS.

[I'd ask Yaakov and other PW experts to double-check the latter sentence,
please.]

> A terminology section to roll up all the acronym expansions (e.g., TDM, CBR)
> in one place seems like a good idea.
These four acronyms would be good to put into a terminology section:

CBR Constant Bit Rate
PSN Packet Switched Network
PW Pseudowire
TDM Time Division Multiplexing

>
> This text on p.11 could be incendiary:
>
>       Under this condition it is safe to place the TDM PW along with
>       congestion - responsive traffic such as TCP, without causing
>       additional congestion.
>
> First of all to reduce the flammability, "safe" -> "reasonable" and there
> are a number of other occurrences of "safe" that deserve similar treatment ;-
> ).
> Beyond that, this assertion is dependent on the operating conditions staying
> within that reasonable operating envelope, which entails use of something
> like a circuit breaker - I'd say that here, and reference the earlier
> circuit breaker discussion.
Global substitution: "safe" -> "reasonable", starting with the abstract.

Then for this specific text on p.11:

OLD
        Under this condition it is safe to place the TDM PW along with
        congestion - responsive traffic such as TCP, without causing
        additional congestion.
NEW
        Under this condition it is reasonable to place the TDM PW in competition
      with congestion-responsive traffic such as TCP, as the result should not
      produce unacceptable congestion effects.  That assumes that the operating
      conditions remain within the reasonable operating envelope - as discussed
      above, a transport circuit breaker is a suggested means of managing a TDM
      PW so that it is shut down if operating conditions remain persistently
      outside the reasonable operating envelope for the PW (e.g., due to a
      high packet loss rate).

No need to go there. See my suggestions earlier.


> Figures 7 and 8 could use some shading or lines to indicate the typical
> deployed operating regions for the pseudo-wires to visibly show that they
> fall under the curves, particularly for larger segment sizes.  I'd also
> repeat or refer to the suggestion to use larger

I wanted to suggest this myself, but it is hard to depict fuzzy boundaries. If you can do this, pls do.

Given the PW operating regions are bounded by v low loss probability, the x axis in Figs 7&8 ought to be scaled much smaller as well.

Related point: the y axis in Fig 1 ought to max at 10, not 50.

Regards



Bob

>
> Nits:
>
> References aren't allowed in abstracts, so remove reference to
> [I-D.fairhurst-tsvwg-circuit-breaker]
>
> Introduction, first paragraph, delete "PWE3" towards end of paragraph -
>
>    preferably the 4 byte PWE3 control word.
>
> In addition to its current general statements, the Security Considerations
> section should point to the security considerations sections in a few
> particularly relevant RFCs - I'd suggest 3985, 4553, 5086 and 5087.
After:

   This document does not introduce any new congestion-specific
   mechanisms and thus does not introduce any new security
   considerations above those present for PWs in general.

I suggest adding a new paragraph:

   For PW security considerations, see the security considerations sections
   of the cited RFCS; the security considerations in [RFC3985], [RFC4553],
   [RFC5086] and [RFC5087] are particularly applicable to this document.

>
> Thanks,
> --David
>
>
> > -----Original Message-----
> > From: pwe3 [ mailto:pwe3-bounces@ietf.org] On Behalf Of Yaakov Stein
> > Sent: Thursday, July 24, 2014 4:52 PM
> > To: pwe3@ietf.org
> > Subject: Re: [PWE3] I-D Action: draft-ietf-pwe3-congcons-02.txt
> >
> > Please look at the PDF version only
> > (the txt version does not have the graphs).
> >
> > Y(J)S
> >
> >
>
> _______________________________________________
> pwe3 mailing list
> pwe3@ietf.org
> https://www.ietf.org/mailman/listinfo/pwe3

_______________________________________________
pwe3 mailing list
pwe3@ietf.org
https://www.ietf.org/mailman/listinfo/pwe3
 
 
 

________________________________________________________________
Bob Briscoe,           &n= bsp;            =             &nbs= p;             BT
_______________________________________________
pwe3 mailing list
pwe3@ietf.org
https://www.ietf.org/mailman/listinfo/pwe3

________________________________________________________________
Bob Briscoe,           &n= bsp;            =             &nbs= p;             BT --=====================_855705482==.ALT-- From nobody Mon Aug 18 05:45:13 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D2B1A1A01D2 for ; Mon, 18 Aug 2014 05:45:12 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.268 X-Spam-Level: X-Spam-Status: No, score=-5.268 tagged_above=-999 required=5 tests=[BAYES_50=0.8, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, GB_I_INVITATION=-2, GB_I_LETTER=-2, GB_SUMOF=1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.668, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ayWfkdSZgTwj for ; Mon, 18 Aug 2014 05:45:01 -0700 (PDT) Received: from mailuogwdur.emc.com (mailuogwdur.emc.com [128.221.224.79]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id EFDC41A01BD for ; Mon, 18 Aug 2014 05:45:00 -0700 (PDT) Received: from maildlpprd52.lss.emc.com (maildlpprd52.lss.emc.com [10.106.48.156]) by mailuogwprd52.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s7ICirDe031998 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 18 Aug 2014 08:44:54 -0400 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd52.lss.emc.com s7ICirDe031998 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=emc.com; s=jan2013; t=1408365894; bh=v8gUVV4RslESWP6Bvp29SAiJGZ0=; h=From:To:CC:Date:Subject:Message-ID:References:In-Reply-To: Content-Type:MIME-Version; b=mifgFUtvIcWdlKZU4InQ/Lygaa0EJe21wwJYi2w+wQ7h+vjmIomsE3lQ5dToAR8T5 QgFUERG8pe9826ZlgEDXqBgbsL63bHbn98ne80WUxqhlm7D7RcXKxln2LqyAVLef0a kEqsX5tC3kC2rI66nMNyFtzNoasd/352OjVHE1Dk= X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd52.lss.emc.com s7ICirDe031998 Received: from mailusrhubprd02.lss.emc.com (mailusrhubprd02.lss.emc.com [10.253.24.20]) by maildlpprd52.lss.emc.com (RSA Interceptor); Mon, 18 Aug 2014 08:44:40 -0400 Received: from mxhub32.corp.emc.com (mxhub32.corp.emc.com [128.222.70.172]) by mailusrhubprd02.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s7ICieFU028082 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Mon, 18 Aug 2014 08:44:40 -0400 Received: from mx15a.corp.emc.com ([169.254.1.175]) by mxhub32.corp.emc.com ([128.222.70.172]) with mapi; Mon, 18 Aug 2014 08:44:40 -0400 From: "Black, David" To: Bob Briscoe Date: Mon, 18 Aug 2014 08:44:39 -0400 Thread-Topic: [PWE3] FW: WGLC comments on: draft-ietf-pwe3-congcons-02.txt Thread-Index: Ac+4cK08ao1hnXAQQ/2ogxRmGCvqXACbcseg Message-ID: <8D3D17ACE214DC429325B2B98F3AE712077BB42180@MX15A.corp.emc.com> References: <8D3D17ACE214DC429325B2B98F3AE712077B9517F4@MX15A.corp.emc.com> <8D3D17ACE214DC429325B2B98F3AE712077B951928@MX15A.corp.emc.com> <201408141302.s7ED2km0032238@bagheera.jungle.bt.co.uk> <8D3D17ACE214DC429325B2B98F3AE712077B952113@MX15A.corp.emc.com> <8D3D17ACE214DC429325B2B98F3AE712077B95212C@MX15A.corp.emc.com> <201408151006.s7FA6nWG003570@bagheera.jungle.bt.co.uk> In-Reply-To: <201408151006.s7FA6nWG003570@bagheera.jungle.bt.co.uk> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_8D3D17ACE214DC429325B2B98F3AE712077BB42180MX15Acorpemcc_" MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd02.lss.emc.com X-RSA-Classifications: DLM_1, public, GIS Solicitation Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/lX_Yu3j1a3HJ9bzB5anWKuNeyCU Cc: "pwe3@ietf.org" Subject: Re: [PWE3] FW: WGLC comments on: draft-ietf-pwe3-congcons-02.txt X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Aug 2014 12:45:13 -0000 --_000_8D3D17ACE214DC429325B2B98F3AE712077BB42180MX15Acorpemcc_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Bob, 1-4 & 6: ok 5: "acceptable" is a good replacement for S-A-F-E, but I'm not sure about "= friendly". The key concept here is assurance that there are no problems. = How about "clearly acceptable"? That may need some additional text to say = that acceptable network and service behavior may be delivered outside the c= learly acceptable region, but one cannot rely on the analysis in this draft= (based on the TCP throughput formula) to reach such a conclusion. 7: I disagree - the tsvwg circuit breaker draft is rather general, and we n= ow have two WG LC comments on this draft asking how to apply a circuit brea= ker to PWs. I think we're on solid ground with the references to the ITU-T= service quality standards and expecting operator monitoring of service qua= lity. OTOH, the word "persistently" seemed to be a concern, and I have no = problem removing it (anyone who asks about that can be directed to G.775 an= d G.826). New suggested text: For TDM pseudowires, such a circuit breaker can be realized via operator monitoring of drop rates and the quality of TDM service delivered in order to shut down TDM pseudowires that fail to deliver acceptable TDM service (as defined by [G775] and [G826]). This document explains why a TDM PW that is causing congestion problems via interference with elastic traffic is also not delivering acceptable TDM service (see the rest of this Section, plus Appendices A and B). 8: I believe that figures 1 and 2 suffice to make the point, but I see that= a recommendation may be a bit of a reach for this draft. How about the fo= llowing modified form of my first sentence, which just states the inference= that can be drawn from Figures 1 and 2? - Figures 1 and 2 indicate that a larger segment size for a T1 or E1 TDM PW moves the operational behavior of the PW away from the boundary at which the TCP throughput equation suggests that undue interference wi= th elastic traffic may start to occur. Thanks, --David From: Bob Briscoe [mailto:bob.briscoe@bt.com] Sent: Friday, August 15, 2014 6:03 AM To: Black, David Cc: pwe3@ietf.org Subject: Re: [PWE3] FW: WGLC comments on: draft-ietf-pwe3-congcons-02.txt David, I decided to think on a better word for 'safe' overnight, and allow others = to comment. But no-one has, so here's my responses to all your outstanding = points... I've provided slightly updated xml source of my suggested edits (and a diff= ) here: < http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-pwe= 3-congcons-DIFF03b-02.html > < http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-pwe= 3-congcons-03b.txt > < http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-pwe= 3-congcons-03b.xml > Again note: xml is purely for precision, not implying I have taken over edi= torship. 1. Happy with your new scoping text, except the the first 'independent' cla= ngs with the second, and it's unnecessary anyway: " The concern is over PWs that share network capacity with elastic or cong= estion-responsive traffic, [independent of] whether that sharing is a resul= t of the service provider network's traffic engineering and capacity planni= ng or independent deployment of PWs by entities other than network operator= s. " 2. My point wasn't to remove the reference to the 125us (in my suggested te= xt I didn't). It was that the 125us stuff /alone/ doesn't warrant the 'thus= ' in the next clause; the logic needs to be connected by talking about the = sum of the flight time /and/ the per-node processing time. CURRENT: The one-way delay of a native TDM service consists of the physical time-of-flight plus 125 microseconds for each TDM switch traversed; and is thus very small as compared to typical PSN network-crossing latencies. SUGGESTED: The one-way delay of a native TDM service consists of the physical time-of-flight plus 125 microseconds for each TDM switch traversed; so the total one-way del= ay for most TDM PWs has to be small compared to typical PSN network-crossin= g latencies. 5. I understand and agree - 'S-A-F-E region' is incendiary. But I don't lik= e 'reasonable region' - it's slightly /too/ meaningless. Other possibilities are: - admissable region (problem: implies outside the region is inadmissable) - harmonious region (problem: too Californian) So I suggest two words: - friendly or acceptable, each used depending on context. On the graphs, I suggest 'operating region (friendly under curve)' because = that is precisely where the curves come from - 'friendly' does what it says= on the tin, and the implied 'unfriendly' above the curve correctly avoids = implying that the sky will fall. Elsewhere, I suggest the following replacements for the S-word: "It is thus worthwhile specifying under what conditions such competition is= acceptable" "One may view this condition as defining a 'friendly' operating envelope fo= r a TDM PW" "Under this condition it is acceptable to place the TDM PW along with ... T= CP. " "comparison with something widely considered to offer an acceptable respons= e to congestion" 7. Circuit breaker Scope creep. Rat-hole. Stop digging. Yes, I know Stewart has asked us to dig out another rat as well, but the ap= propriate response is "Out of scope". You might think your text is uncontroversial, but if we are tempted to take= Stewart's bait (rat metaphor intended), it will probably start new debates= that this doc doesn't deserve: "What does persistent mean?", "Which traffi= c is more important/ valuable?", yada yada. I suggest the text merely makes it clear that this debate is in scope for e= lsewhere, not here. I've also just realised that Yaakov has written the sen= tences around circuit breaker with the wrong logical sequence. The circuit = breaker needs to be introduced as something the PW uses when its /own/ qual= ity has failed, which should avoid any need to shut down because it is thre= atening elastic services: CURRENT: If the bandwidth consumed by a TDM PW is considered detrimental, the only available remedy is to completely shut down the PW, by using a transport circuit breaker mechanism. However, we will show that even before such an action is warranted, the PW will become unable to faithfully emulate the native TDM service; for example, when a TDM service is carrying voice grade telephony channels, the voice quality will degrade to below useful levels. SUGGESTED: We will show that, before a PW becomes deterimental to elastic traffi= c, it will become unable to faithfully emulate the native TDM service; for example, when a TDM service is carrying voice grade telephony channels, the voice quality will degrade to below useful levels. Once the quality of the emulated service is pers= istently unacceptable, if a PW shuts down selected flows or shuts itself do= wn completely, it will automatically avoid detriment to elastic traffic. A = transport circuit breaker [I-D.fairhurst-tsvwg-circuit-breaker] could be us= ed, but mechanism is beyond the scope of the present document. 8. Segment size. Everyone's in a big group-hug because we've written a draft showing there's= is no need to change anything. Then just at the end of an uneventful WGLC,= we hear David's voice from the bottom of a deep hole shouting, "avoid the = rat-hole invitation", while furiously digging for a rat. There is no need to add a recommendation that PWs need to be changed to use= larger packets. No matter how hedged and soft this recommendation would be= , it's not needed anyway. Stop digging. Yaakov's new text is fine as it stands. "Only for small packets, long delay= s, and high packet loss ratios, do TDM PWs potentially consume more bandwid= th, and even then only marginally. " In order to justify your sentence, we would have to include graphs with pac= ket-rate (not bit-rate) on the vertical axis and go into a whole treatise a= s to why packet-rate comparison might be important as well. Then the graph = would show one plot for all TCP segment sizes and multiple plots for PWs wi= th different packet sizes. Then we'd see that the corner of the two smalles= t packet PWs just clipped the TCP curve at loss probabilities that are stup= idly large anyway. So even after all this, it would hardly justify a need f= or your sentence. All that digging would not be worth it, just to discover (again) that there= is no rat. Bob At 19:39 14/08/2014, Black, David wrote: Content-Language: en-US Content-Type: multipart/alternative; boundary=3D"_000_8D3D17ACE214DC429325B2B98F3AE712077B95212CMX15Aco= rpemcc_" And forwarding my views to the list ... ... On Bob's six thoughts (I have problems with 2 & 5 - the other 4 are bas= ically ok w/me) plus follow-ups on Bob's concerns w/my two major issues, he= re goes: -- 1 -- I agree in principle, but I think Bob's suggested new text is too b= lack/white and should be generalized. Bob suggests: > The case where the service provider network arranges sufficient capacity = for a PW is well understood and > will not be discussed further here. The concern is over PWs deployed inde= pendently of the service provider > network's traffic engineering or capacity planning. I want to include the case where capacity has been arranged, but may not be= sufficient, I suggest rewriting Bob's second sentence as: The concern is over PWs that share network capacity with elastic or congest= ion-responsive traffic, independent of whether that sharing is a result of = the service provider network's traffic engineering and capacity planning or= independent deployment of PWs by entities other than network operators. -- 2 -- Here, I disagree. One of the significant contributions of this dra= ft is that TDM PWs don't cause congestion problems in practice, and one of = the major reasons for that is the low latency requirements for acceptable T= DM service delivery, both in ITU-T standards and in actual TDM PW deploymen= t. The 125us behavior of native TDM switches is relevant to the latter, an= d hence I prefer to include mention of it. -- 3 -- I agree; clear captions are always a feature, not a bug ;-). -- 4 -- I agree with changing "packet loss rate" to "packet loss probabilit= y". -- 5 -- I firmly disagree with Bob here, although he's right that this is "= techno-political" ;-). I am concerned that S-A-F-E is likely to be a 4-letter word in wider Transp= ort Area review, and hence, I want to use a word that does not imply anythi= ng remotely resembling an absolute guarantee that nothing can possibly ever= go wrong (yes, I know Bob didn't mean to go to quite that extreme an inter= pretation, but S-A-F-E is liable to be read that way by those with an "axe = to grind"), even though there's lots of solid operational experience that s= uggests all is well here. -- 6 -- I think I'm ok with what Bob proposes to better characterize loss b= urstiness, but want to see specific text. -- Circuit Breaker Mechanism -- >> [David] I suggest adding: >> >> For TDM pseudowires, such a circuit breaker can be realized via >> operator monitoring of drop rates and consequent on the quality of >> TDM service delivered and shutting down TDM pseudowires that persiste= ntly >> fail to deliver acceptable TDM service (as defined by [G775] and >> [G826]). This document explains why a TDM PW that is causing congest= ion >> problems via interference with elastic traffic cannot be delivering >> acceptable TDM service (see the rest of this Section, plus Appendices >> A and B). >> > [Bob] I don't think we should go there. We need to keep this draft firmly= as informational. If someone > thinks a mechanism is needed because of the information in this draft, th= en they can write about a mechanism. [David] I firmly disagree, but Bob and I might be talking past each other, = as I do want to stay informational ;-). This text is intended to describe an example or possibility, not to specify= an implementation requirement. The new text was intended as an OAM/OSS pr= actice recommendation to the operator, not as a normative protocol spec. I'm ok with inserting "example", "possibility" or something similar if that= helps w/Bob's concern, but I strongly oppose doing nothing here, and note = that this is also needed to respond to one of Stewart Bryant's WG LC commen= ts. -- PW segment size -- >> [David] Ignoring the rathole invitation ;-). This is still in section 3= , after: >> >> We see that in general for large packet sizes, short delays, and low >> packet loss rates, the TDM pseudowires consume much less bandwidth >> than TCP would under identical conditions. Only for small packets, >> long delays, and high packet loss ratios, do TDM PWs potentially >> consume more bandwidth, and even then only marginally. >> >> I suggest adding: >> >> Comparing figures 1 and 2, use of a larger segment size for a T1 or E= 1 >> TDM PW moves the operational behavior of the PW away from the boundar= y >> at which the TCP throughput equation suggests that undue interference= with >> elastic traffic may start to occur. We suggest that larger segment >> sizes be preferred for T1 and E1 TDM PWs, but note that other >> considerations may favor smaller segment sizes (e.g., jitter buffer >> size, bounding the impact of an isolated packet drop event). >> > [Bob] There is no need to recommend any remedial action, when we have jus= t shown there is not a problem. [David] Not exactly - Figure 1 shows that there is a potential concern at a= combination of the high end of the range of TDM PW latencies (4ms), and hi= gher packet loss ratios. > Further, the "marginal problem" only applies when the TCP traffic uses pa= ckets smaller than 128B, which we only used > as a hypothetical worst-case for the analysis. But in practice this doesn= 't happen with long-running TCP flows - > they use packets that are as large as possible (irrespective of what the = PW uses). So there really is /no/ problem to remedy. Ok, Bob can relax - his transport credentials are intact, as this is an ent= ry point to the bit-rate-congestible vs. packet-rate-congestible resource r= athole that I want to ignore :-). I believe that larger TDM PW segment sizes are a good thing for both this o= perating envelope boundary concern as well as per-packet processing overhea= ds in TDM PW functionality, modulo other considerations, e.g., jitter buffe= rs, and so I prefer to keep this addition. Thanks, --David From: Bob Briscoe [mailto:bob.briscoe@bt.com] Sent: Thursday, August 14, 2014 9:03 AM To: Yaakov Stein Cc: Andrew G. Malis; BOCCI Matthew; Black, David Subject: RE: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt Yaakov, I have just re-read it thoroughly. I have noticed about two dozen editorial= nits where correcting them would not change the intended sense (altho some= times the sense needs to change to say what we intended!). Examples: - Unexpanded abbreviations (David's suggestion would help, but there are mo= re) - Missing words, tautologies, random punctuation problems etc - Uses of the wrong word, even tho we know what you mean - The caveat about dynamics vs steady state needs to shift down a para, so = that it follows into the para that says what to do about dynamics. Should I take the token, and edit the XML directly, then you can see a diff= to check if you want to accept my suggestions? Below are six thoughts that go slightly beyond 'editorial'. 1-4 could be co= nsidered 'editorial'. But 5&6 are the more significant "techno-political" p= oints. 1. The phrase "mechanisms for provisioning PWs in service provider networks= " is too opaque (the PWE audience might understand this scoping sentence, b= ut a L4 reader won't understand what we're excluding here, which will make = them suspicious). CURRENT: Mechanisms for provisioning PWs in service provider networks are well understood and will not be discussed further here. SUGGESTED: The case where the service provider network arranges sufficient capacity= for a PW is well understood and will not be discussed further here. The co= ncern is over PWs deployed independently of the service provider network's = traffic engineering or capacity planning. 2. The justification for using OWD of a few milliseconds misses the point. " The one-way delay of a native TDM service consists of the physical time-of-flight plus 125 microseconds for each TDM switch traversed; and is thus very small as compared to typical PSN network-crossing latencies. " The 125us per switch is irrelevant. The point that needs making is that PWs= are generally deployed with a short physical time of flight. 3. Related point: the captions on the graphs do not clearly indicate that t= hey are OWD, not RTT. This could be clarified in the text, or on the graphs= . 4. Change packet loss rate (and PLR) to packet loss probability throughout,= including on the graph axes. PLR is an ambiguous term, even though it is w= idely used in the industry. Packet loss rate could be measured in packet pe= r second, whereas packet loss probability is unambiguously dimensionless. I= ndeed, in the appendices, packet loss rates with respect to time are calcul= ated. 5. Rather than David's "reasonable" I suggest we use the phrase "unambiguou= sly safe" not just "safe" for the headings of Figs 5-8 and in the text, as = below. Otherwise, we are implying that it is unsafe outside the region. CURRENT: One may view this condition as defining an operating envelope for a TDM PW, as a TDM PW that occupies no more bandwidth than a TCP flow causes no more congestion than that TCP flow would. Under this condition it is safe to place the TDM PW along with congestion- responsive traffic such as TCP, without causing additional congestion. on the other hand, were the TDM PW to consume significantly more bandwidth a TCP flow, it could contribute disproportionately to congestion, and its mixture with congestion- responsive traffic might be inappropriate. SUGGESTED: One may view this condition as defining an unambiguously safe operating = envelope for a TDM PW, as a TDM PW that occupies no more bandwidth than a TCP flow causes no more congestion than that TCP flow would. This does not draw = us into a debate over whether TCP-friendliness is a valid constraint; it si= mply says that there can be no question that a PW is safe if it does no mor= e harm than a single TCP flow. Under this condition it is unambiguously safe to place the TDM PW along with conges= tion- responsive traffic such as TCP. On the other hand, were the TDM PW to c= onsume significantly more bandwidth a TCP flow, it could contribute disproportionately to congestion, and its mixture with congestion- responsive traffic might start to cause concern. 6. The analysis in Appendices A & B needs to mention that loss bursts arise= primarily from tail drop buffers, and the introduction of active queue man= agement (AQM) makes losses more independent of each other (less bursty). (I= have references showing evidence of this.) In the comparisons with TCP, we= have assumed independent losses anyway, because they are the worst case (a= nd fortunately they are easier to analyse). On a similar point, in appendix= B, where we say "(bursty packet loss was also simulated but is not reporte= d here)", we ought to give a reason otherwise readers will be suspicious. A= good reason would be that uniformly random is the worst case. I've also responded to a couple of David's suggestions inline (I agree with= the others)... At 17:44 13/08/2014, Yaakov Stein wrote: Bob told me on Friday of the IETF that he was going to look over the final = version and comment. Bob - ping ? Y(J)S From: Andrew G. Malis [ mailto:agmalis@gmail.com] Sent: 13 August, 2014 17:02 To: Yaakov Stein Cc: Bob Briscoe; rbriscoe@jungle.bt.co.uk;= BOCCI Matthew; Black, David Subject: Re: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt Yaakov, I still haven't heard from Bob, but you should incorporate David's updates = into a new revision, and I'll use that version for my final Shepherd's revi= ew. Thanks, Andy On Mon, Aug 11, 2014 at 11:16 AM, Andrew G. Malis > wrote: Actually, I'm going to end the LC as soon as I hear back from Bob on the IP= R call. I sent him a reminder earlier this morning. Cheers, Andy On Sun, Aug 10, 2014 at 9:53 AM, Andrew G. Malis > wrote: David, That's perfect, thanks! I'm going to end the LC tomorrow. Cheers, Andy On Sat, Aug 9, 2014 at 10:32 PM, Black, David > wrote: Following up on this ... > As we're now moving quickly to try to complete this draft , I agreed > w/Yaakov that my author comments would be sent in as WG Last Call comment= s ... > ... so here they are: ... with some suggested new text ... > Major issues: > > The reference to a transport circuit breaker in section 3 is a bit optimi= stic > and high level: > > We can accomplish this by > employing a transport circuit breaker, by which we mean an automati= c > mechanism for terminating a flow to prevent negative impact on othe= r > flows and on the stability of the network > [I-D.fairhurst-tsvwg-circuit-breaker]. > > This should be accompanied by at least an example of how to do this - pas= t > discussion has suggested carrier monitoring of delivered service quality > in order to shut down PWs that are experiencing loss rates that result in > unacceptable service quality (as defined for us by the ITU ...). I'd als= o > like to add some text to suggest that operators use this sort of circuit > breaker practice when TDM PWs compete with elastic traffic. After this text in Section 3: We can accomplish this by employing a transport circuit breaker, by which we mean an automatic mechanism for terminating a flow to prevent negative impact on other flows and on the stability of the network [I-D.fairhurst-tsvwg-circuit-breaker]. Note that a transport circuit breaker is intended as a protection mechanism of last resort, just as an electrical circuit breaker is only triggered when absolutely necessary. I suggest adding: For TDM pseudowires, such a circuit breaker can be realized via operator monitoring of drop rates and consequent on the quality of TDM service delivered and shutting down TDM pseudowires that persistentl= y fail to deliver acceptable TDM service (as defined by [G775] and [G826]). This document explains why a TDM PW that is causing congestion problems via interference with elastic traffic cannot be delivering acceptable TDM service (see the rest of this Section, plus Appendices A and B). I don't think we should go there. We need to keep this draft firmly as info= rmational. If someone thinks a mechanism is needed because of the informati= on in this draft, then they can write about a mechanism. And breaking the paragraph after the added text. > Figure 1 and Figure 2 indicate that based on using the TCP throughput > equation as a proxy for potential impact on elastic traffic (which is wha= t > we're doing in this draft), for the same fixed bandwidth PW, a larger seg= ment > size is less likely to cause impacts. I'd like to add a suggestion to us= e > the larger PW segment sizes as long as they're consistent with delivery o= f > the service. There's been a related (long-to-the-point-of-ratholed) deba= te > over in the Transport Area about whether packet-rate-limited/congestible > forwarding is still relevant. My view is that such equipment exists, and > hence that this is a relevant consideration for this draft, and would not > mention that issue here. Ignoring the rathole invitation ;-). This is still in section 3, after: We see that in general for large packet sizes, short delays, and low packet loss rates, the TDM pseudowires consume much less bandwidth than TCP would under identical conditions. Only for small packets, long delays, and high packet loss ratios, do TDM PWs potentially consume more bandwidth, and even then only marginally. I suggest adding: Comparing figures 1 and 2, use of a larger segment size for a T1 or E1 TDM PW moves the operational behavior of the PW away from the boundary at which the TCP throughput equation suggests that undue interference wi= th elastic traffic may start to occur. We suggest that larger segment sizes be preferred for T1 and E1 TDM PWs, but note that other considerations may favor smaller segment sizes (e.g., jitter buffer size, bounding the impact of an isolated packet drop event). There is no need to recommend any remedial action, when we have just shown = there is not a problem. Further, the "marginal problem" only applies when the TCP traffic uses pack= ets smaller than 128B, which we only used as a hypothetical worst-case for = the analysis. But in practice this doesn't happen with long-running TCP flo= ws - they use packets that are as large as possible (irrespective of what t= he PW uses). So there really is /no/ problem to remedy. and again, breaking the paragraph after that added text. I hope that latter sentence ("We suggest that ...") is acceptable to the WG= . > Minor issues: > > Section 2 is about Ethernet PWs that carry elastic traffic. Should other= s > be mentioned as also covered by this analysis - e.g., ATM and Frame Relay > PWs carrying elastic traffic? After: In this section we consider Ethernet PWs that primarily carry congestion-responsive traffic. We show that we automatically obtain the desired congestion avoidance behavior, and that additional mechanisms are not needed. I suggest adding: This discussion is applicable to any PW that carries congestion- responsive traffic (e.g., ATM and Frame Relay PWs); Ethernet PWs are used here for clarity, as they are a common example. > Section 2 does not cover Ethernet PWs carrying inelastic traffic - that w= ould > be unusual, but should it be explicitly noted as out of scope, or just no= t > discussed (as is the case for the current text)? I suggest doing nothing here, and planning to deal with this concern if it should arise later (e.g., IETF Last Call or IESG Evaluation). > Section 3 should say that the reason the draft focuses on TDM PWs is that > they're the inelastic PWs that are likely to be carried over IP in practi= ce. > My understanding is that other potentially inelastic PWs are MPLS-only > in practice. After: Inelastic PWs, such as TDM PWs ([RFC4553][RFC5086][RFC5087]), are potentially more problematic than the elastic PWs of the previous section. Being constant bit-rate (CBR), TDM PWs can not be made responsive to congestion. On the other hand, being CBR, they also do not attempt to capture additional bandwidth when neighboring TCP flows back off. I suggest adding: We focus on TDM PWs here as they are the only inelastic PWs that are likely to be IP-encapsulated in practice. Most other inelastic PWs tend to only be used with MPLS. [I'd ask Yaakov and other PW experts to double-check the latter sentence, please.] > A terminology section to roll up all the acronym expansions (e.g., TDM, C= BR) > in one place seems like a good idea. These four acronyms would be good to put into a terminology section: CBR Constant Bit Rate PSN Packet Switched Network PW Pseudowire TDM Time Division Multiplexing > > This text on p.11 could be incendiary: > > Under this condition it is safe to place the TDM PW along with > congestion - responsive traffic such as TCP, without causing > additional congestion. > > First of all to reduce the flammability, "safe" -> "reasonable" and there > are a number of other occurrences of "safe" that deserve similar treatmen= t ;- > ). > Beyond that, this assertion is dependent on the operating conditions stay= ing > within that reasonable operating envelope, which entails use of something > like a circuit breaker - I'd say that here, and reference the earlier > circuit breaker discussion. Global substitution: "safe" -> "reasonable", starting with the abstract. Then for this specific text on p.11: OLD Under this condition it is safe to place the TDM PW along with congestion - responsive traffic such as TCP, without causing additional congestion. NEW Under this condition it is reasonable to place the TDM PW in compet= ition with congestion-responsive traffic such as TCP, as the result should = not produce unacceptable congestion effects. That assumes that the opera= ting conditions remain within the reasonable operating envelope - as discu= ssed above, a transport circuit breaker is a suggested means of managing a= TDM PW so that it is shut down if operating conditions remain persistentl= y outside the reasonable operating envelope for the PW (e.g., due to a high packet loss rate). No need to go there. See my suggestions earlier. > Figures 7 and 8 could use some shading or lines to indicate the typical > deployed operating regions for the pseudo-wires to visibly show that they > fall under the curves, particularly for larger segment sizes. I'd also > repeat or refer to the suggestion to use larger I wanted to suggest this myself, but it is hard to depict fuzzy boundaries.= If you can do this, pls do. Given the PW operating regions are bounded by v low loss probability, the x= axis in Figs 7&8 ought to be scaled much smaller as well. Related point: the y axis in Fig 1 ought to max at 10, not 50. Regards Bob > > Nits: > > References aren't allowed in abstracts, so remove reference to > [I-D.fairhurst-tsvwg-circuit-breaker] > > Introduction, first paragraph, delete "PWE3" towards end of paragraph - > > preferably the 4 byte PWE3 control word. > > In addition to its current general statements, the Security Consideration= s > section should point to the security considerations sections in a few > particularly relevant RFCs - I'd suggest 3985, 4553, 5086 and 5087. After: This document does not introduce any new congestion-specific mechanisms and thus does not introduce any new security considerations above those present for PWs in general. I suggest adding a new paragraph: For PW security considerations, see the security considerations sections of the cited RFCS; the security considerations in [RFC3985], [RFC4553], [RFC5086] and [RFC5087] are particularly applicable to this document. > > Thanks, > --David > > > > -----Original Message----- > > From: pwe3 [ mailto:pwe3-bounces@ietf.org] On Behalf Of Yaakov Stein > > Sent: Thursday, July 24, 2014 4:52 PM > > To: pwe3@ietf.org > > Subject: Re: [PWE3] I-D Action: draft-ietf-pwe3-congcons-02.txt > > > > Please look at the PDF version only > > (the txt version does not have the graphs). > > > > Y(J)S > > > > > > _______________________________________________ > pwe3 mailing list > pwe3@ietf.org > https://www.ietf.org/mailman/listinfo/pwe3 _______________________________________________ pwe3 mailing list pwe3@ietf.org https://www.ietf.org/mailman/listinfo/pwe3 ________________________________________________________________ Bob Briscoe, BT _______________________________________________ pwe3 mailing list pwe3@ietf.org https://www.ietf.org/mailman/listinfo/pwe3 ________________________________________________________________ Bob Briscoe, BT --_000_8D3D17ACE214DC429325B2B98F3AE712077BB42180MX15Acorpemcc_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Bob,

 

1-4 &am= p; 6: ok

 

=

5: “acceptable” is a good replacement for S-A-= F-E, but I’m not sure about “friendly”.  The key con= cept here is assurance that there are no problems.  How about “c= learly acceptable”?  That may need some additional text to say t= hat acceptable network and service behavior may be delivered outside the cl= early acceptable region, but one cannot rely on the analysis in this draft = (based on the TCP throughput formula) to reach such a conclusion.

 

7= : I disagree - the tsvwg circuit breaker draft is rather general, and we no= w have two WG LC comments on this draft asking how to apply a circuit break= er to PWs.  I think we’re on solid ground with the references to= the ITU-T service quality standards and expecting operator monitoring of s= ervice quality.  OTOH, the word “persistently” seemed to b= e a concern, and I have no problem removing it (anyone who asks about that = can be directed to G.775 and G.826).  New suggested text:

 

= &nbs= p;  For TDM pseudowires, such a circuit breaker can be realized via

   operator monitoring of dro= p rates and the quality of

 &nbs= p; TDM service delivered in order to shut down TDM pseudowires that

   fail to deliver acceptable TDM = service (as defined by [G775] and

&nb= sp;  [G826]).  This document explains why a TDM PW that is causin= g congestion

   problems vi= a interference with elastic traffic is also not delivering

   acceptable TDM service (see the rest of = this Section, plus Appendices

 &= nbsp; A and B).

 

8: I believe that figures 1 and 2 suffice to make t= he point, but I see that a recommendation may be a bit of a reach for this = draft.  How about the following modified form of my first sentence, wh= ich just states the inference that can be drawn from Figures 1 and 2? -

  

   Figures 1 and 2 indicate that a larger segmen= t size for a T1 or E1
   TDM PW moves the operational behavior= of the PW away from the boundary
   at which the TCP throughp= ut equation suggests that undue interference with
   elastic t= raffic may start to occur. 

=  

Thanks,
--David

 

From: Bob Briscoe [mailto:bob.bris= coe@bt.com]
Sent: Friday, August 15, 2014 6:03 AM
To: = Black, David
Cc: pwe3@ietf.org
Subject: Re: [PWE3] FW: = WGLC comments on: draft-ietf-pwe3-congcons-02.txt

 

Davi= d,

I decided to think on a better word for 'safe' overnight, and all= ow others to comment. But no-one has, so here's my responses to all your ou= tstanding points...

I've provided slightly updated xml source of my = suggested edits (and a diff) here:
< http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-= pwe3-congcons-DIFF03b-02.html >
<= http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-pwe3= -congcons-03b.txt >
< http://www.= bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-pwe3-congcons-03= b.xml >
Again note: xml is purely for precision, not implying I h= ave taken over editorship.

1. Happy with your new scoping text, exce= pt the the first 'independent' clangs with the second, and it's unnecessary= anyway:
"  The concern is over PWs that share network capacit= y with elastic or congestion-responsive traffic, [independent of] whether t= hat sharing is a result of the service provider network's traffic engineeri= ng and capacity planning or independent deployment of PWs by entities other= than network operators.
"

2. My point wasn't to remove the = reference to the 125us (in my suggested text I didn't). It was that the 125= us stuff /alone/ doesn't warrant the 'thus' in the next clause; the logic n= eeds to be connected by talking about the sum of the flight time /and/ the = per-node processing time.
CURRENT:
   The one-way delay
=    of a native TDM service consists of the physical time-of-fligh= t plus
   125 microseconds for each TDM switch traversed; and = is thus very
   small as compared to typical PSN network-cross= ing latencies.
SUGGESTED:
   The one-way delay
 &n= bsp; of a native TDM service consists of the physical time-of-flight plus <= br>   125 microseconds for each TDM switch traversed; so the tota= l one-way delay for most TDM PWs has to be small compared to typical PSN ne= twork-crossing latencies.

5. I understand and agree - 'S-A-F-E regi= on' is incendiary. But I don't like 'reasonable region' - it's slightly /to= o/ meaningless.

Other possibilities are:
- admissable region (pro= blem: implies outside the region is inadmissable)
- harmonious region (p= roblem: too Californian)
So I suggest two words:
- friendly or accept= able, each used depending on context.

On the graphs, I suggest 'ope= rating region (friendly under curve)' because that is precisely where the c= urves come from - 'friendly' does what it says on the tin, and the implied = 'unfriendly' above the curve correctly avoids implying that the sky will fa= ll.

Elsewhere, I suggest the following replacements for the S-word:<= br>"It is thus worthwhile specifying under what conditions such compet= ition is acceptable"
"One may view this condition as defining = a 'friendly' operating envelope for a TDM PW"
"Under this cond= ition it is acceptable to place the TDM PW along with ... TCP. "
&q= uot;comparison with something widely considered to offer an acceptable resp= onse to congestion"


7. Circuit breaker
Scope creep. Rat-= hole. Stop digging.
Yes, I know Stewart has asked us to dig out another= rat as well, but the appropriate response is "Out of scope".
=
You might think your text is uncontroversial, but if we are tempted to = take Stewart's bait (rat metaphor intended), it will probably start new deb= ates that this doc doesn't deserve: "What does persistent mean?",= "Which traffic is more important/ valuable?", yada yada.

= I suggest the text merely makes it clear that this debate is in scope for e= lsewhere, not here. I've also just realised that Yaakov has written the sen= tences around circuit breaker with the wrong logical sequence. The circuit = breaker needs to be introduced as something the PW uses when its /own/ qual= ity has failed, which should avoid any need to shut down because it is thre= atening elastic services:

CURRENT:
     = If the bandwidth consumed by a TDM PW is
     = considered detrimental, the only available remedy is to completely
&nbs= p;     shut down the PW, by using a transport circuit b= reaker mechanism.
      However, we will show t= hat even before such an action is
      warrant= ed, the PW will become unable to faithfully emulate the
  &nbs= p;   native TDM service; for example, when a TDM service is carry= ing
      voice grade telephony channels, the v= oice quality will degrade to
      below useful= levels.
SUGGESTED:
      We will show that= , before a PW becomes deterimental to elastic traffic, it will become unabl= e to faithfully emulate the
      native TDM se= rvice; for example, when a TDM service is carrying
   &nb= sp;  voice grade telephony channels, the voice quality will degrade to=
      below useful levels. Once the quality of= the emulated service is persistently unacceptable, if a PW shuts down sele= cted flows or shuts itself down completely, it will automatically avoid det= riment to elastic traffic. A transport circuit breaker [I-D.fairhurst-tsvwg= -circuit-breaker] could be used, but mechanism is beyond the scope of the p= resent document.


8. Segment size.
Everyone's in a big group-h= ug because we've written a draft showing there's is no need to change anyth= ing. Then just at the end of an uneventful WGLC, we hear David's voice from= the bottom of a deep hole shouting, "avoid the rat-hole invitation&qu= ot;, while furiously digging for a rat.

There is no need to add a re= commendation that PWs need to be changed to use larger packets. No matter h= ow hedged and soft this recommendation would be, it's not needed anyway. St= op digging.

Yaakov's new text is fine as it stands. "Only for s= mall packets, long delays, and high packet loss ratios, do TDM PWs potentia= lly consume more bandwidth, and even then only marginally. "

I= n order to justify your sentence, we would have to include graphs with pack= et-rate (not bit-rate) on the vertical axis and go into a whole treatise as= to why packet-rate comparison might be important as well. Then the graph w= ould show one plot for all TCP segment sizes and multiple plots for PWs wit= h different packet sizes. Then we'd see that the corner of the two smallest= packet PWs just clipped the TCP curve at loss probabilities that are stupi= dly large anyway. So even after all this, it would hardly justify a need fo= r your sentence.

All that digging would not be worth it, just to di= scover (again) that there is no rat.


Bob


At 19:39 14= /08/2014, Black, David wrote:

Co= ntent-Language: en-US
Content-Type: multipart/alternative;
 &nbs= p;       boundary=3D"_000_8D3D17ACE214DC= 429325B2B98F3AE712077B95212CMX15Acorpemcc_"

And forwarding my v= iews to the list ...
 
... On Bob’s six thoughts (I have p= roblems with 2 & 5 - the other 4 are basically ok w/me) plus follow-ups= on Bob’s concerns w/my two major issues, here goes:
 
-- = 1 -- I agree in principle, but I think Bob’s suggested new text is to= o black/white and should be generalized.
 
Bob suggests:
&nbs= p;
> The case where the service provider network arranges sufficient = capacity for a PW is well understood and
> will not be discussed furt= her here. The concern is over PWs deployed independently of the service pro= vider
> network's traffic engineering or capacity planning.
 =
I want to include the case where capacity has been arranged, but may no= t be sufficient, I suggest rewriting Bob’s second sentence as:
&nb= sp;
The concern is over PWs that share network capacity with elastic or = congestion-responsive traffic, independent of whether that sharing is a res= ult of the service provider network's traffic engineering and capacity plan= ning or independent deployment of PWs by entities other than network operat= ors.
 
-- 2 -- Here, I disagree.  One of the significant co= ntributions of this draft is that TDM PWs don’t cause congestion prob= lems in practice, and one of the major reasons for that is the low latency = requirements for acceptable TDM service delivery, both in ITU-T standards a= nd in actual TDM PW deployment.  The 125us behavior of native TDM swit= ches is relevant to the latter, and hence I prefer to include mention of it= .
 
-- 3 -- I agree; clear captions are always a feature, not a = bug ;-).
 
-- 4 -- I agree with changing “packet loss rate= ” to “packet loss probability”.
 
-- 5 -- I fi= rmly disagree with Bob here, although he’s right that this is “= techno-political” ;-).
 
I am concerned that S-A-F-E is li= kely to be a 4-letter word in wider Transport Area review, and hence, I wan= t to use a word that does not imply anything remotely resembling an absolut= e guarantee that nothing can possibly ever go wrong (yes, I know Bob didn&#= 8217;t mean to go to quite that extreme an interpretation, but S-A-F-E is l= iable to be read that way by those with an “axe to grind”), eve= n though there’s lots of solid operational experience that suggests a= ll is well here.
 
-- 6 -- I think I’m ok with what Bob pr= oposes to better characterize loss burstiness, but want to see specific tex= t.
 
-- Circuit Breaker Mechanism --
 
>> [Davi= d] I suggest adding:
>>
>>    For TDM pse= udowires, such a circuit breaker can be realized via
>>  = ;  operator monitoring of drop rates and consequent on the quality of<= br>>>    TDM service delivered and shutting down TDM p= seudowires that persistently
>>    fail to deliver = acceptable TDM service (as defined by [G775] and
>>  &nb= sp; [G826]).  This document explains why a TDM PW that is causing cong= estion
>>    problems via interference with elastic= traffic cannot be delivering
>>    acceptable TDM = service (see the rest of this Section, plus Appendices
>> &nb= sp;  A and B).
>>
> [Bob] I don't think we should go t= here. We need to keep this draft firmly as informational. If someone
>= ; thinks a mechanism is needed because of the information in this draft, th= en they can write about a mechanism.
 
[David] I firmly disagree= , but Bob and I might be talking past each other, as I do want to stay info= rmational ;-).
 
This text is intended to describe an example or= possibility, not to specify an implementation requirement.  The new t= ext was intended as an OAM/OSS practice recommendation to the operator, not= as a normative protocol spec.
 
I’m ok with inserting = 220;example”, “possibility” or something similar if that = helps w/Bob’s concern, but I strongly oppose doing nothing here, and = note that this is also needed to respond to one of Stewart Bryant’s W= G LC comments.
 
-- PW segment size --
 
>> [Da= vid] Ignoring the rathole invitation ;-).  This is still in section 3,= after:
>>
>>   We see that in general for larg= e packet sizes, short delays, and low
>>   packet loss r= ates, the TDM pseudowires consume much less bandwidth
>> &nbs= p;  than TCP would under identical conditions.  Only for small pa= ckets,
>>    long delays, and high packet loss rati= os, do TDM PWs potentially
>>   consume more bandwidth, = and even then only marginally.
>>
>> I suggest adding:>>
>>    Comparing figures 1 and 2, use of = a larger segment size for a T1 or E1
>>    TDM PW m= oves the operational behavior of the PW away from the boundary
>>&= nbsp;   at which the TCP throughput equation suggests that undue = interference with
>>    elastic traffic may start t= o occur.  We suggest that larger segment
>>   = sizes be preferred for T1 and E1 TDM PWs, but note that other
>>&= nbsp;   considerations may favor smaller segment sizes (e.g., jit= ter buffer
>>    size, bounding the impact of an is= olated packet drop event).
>>
> [Bob] There is no need to re= commend any remedial action, when we have just shown there is not a problem= .
 
[David] Not exactly - Figure 1 shows that there is a potenti= al concern at a combination of the high end of the range of TDM PW latencie= s (4ms), and higher packet loss ratios. 

> Further, the &qu= ot;marginal problem" only applies when the TCP traffic uses packets sm= aller than 128B, which we only used
> as a hypothetical worst-case fo= r the analysis. But in practice this doesn't happen with long-running TCP f= lows -
> they use packets that are as large as possible (irrespective= of what the PW uses). So there really is /no/ problem to remedy.
Ok, Bo= b can relax - his transport credentials are intact, as this is an entry poi= nt to the bit-rate-congestible vs. packet-rate-congestible resource rathole= that I want to ignore :-).
 
I believe that larger TDM PW segme= nt sizes are a good thing for both this operating envelope boundary concern= as well as per-packet processing overheads in TDM PW functionality, modulo= other considerations, e.g., jitter buffers, and so I prefer to keep this a= ddition.
 
Thanks,
--David
 
From: Bob Bris= coe [mailto:bob.briscoe@bt.com] <= br>Sent: Thursday, August 14, 2014 9:03 AM
To: Yaakov Stei= n
Cc: Andrew G. Malis; BOCCI Matthew; Black, David
Subject:= RE: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt
 =
Yaakov,

I have just re-read it thoroughly. I have noticed about = two dozen editorial nits where correcting them would not change the intende= d sense (altho sometimes the sense needs to change to say what we intended!= ). Examples:
- Unexpanded abbreviations (David's suggestion would help, = but there are more)
- Missing words, tautologies, random punctuation pro= blems etc
- Uses of the wrong word, even tho we know what you mean
- = The caveat about dynamics vs steady state needs to shift down a para, so th= at it follows into the para that says what to do about dynamics.

Sho= uld I take the token, and edit the XML directly, then you can see a diff to= check if you want to accept my suggestions?

Below are six thoughts = that go slightly beyond 'editorial'. 1-4 could be considered 'editorial'. B= ut 5&6 are the more significant "techno-political" points.
1. The phrase "mechanisms for provisioning PWs in service provid= er networks" is too opaque (the PWE audience might understand this sco= ping sentence, but a L4 reader won't understand what we're excluding here, = which will make them suspicious).
CURRENT:
   Mechanisms fo= r provisioning PWs
   in service provider networks are well un= derstood and will not be
   discussed further here.
SUGGEST= ED:
   The case where the service provider network arranges su= fficient capacity for a PW is well understood and will not be discussed fur= ther here. The concern is over PWs deployed independently of the service pr= ovider network's traffic engineering or capacity planning.

2. The ju= stification for using OWD of a few milliseconds misses the point.
"=   The one-way delay
   of a native TDM service consists o= f the physical time-of-flight plus
   125 microseconds for eac= h TDM switch traversed; and is thus very
   small as compared = to typical PSN network-crossing latencies.
"
The 125us per switc= h is irrelevant. The point that needs making is that PWs are generally depl= oyed with a short physical time of flight.

3. Related point: the cap= tions on the graphs do not clearly indicate that they are OWD, not RTT. Thi= s could be clarified in the text, or on the graphs.

4. Change packet= loss rate (and PLR) to packet loss probability throughout, including on th= e graph axes. PLR is an ambiguous term, even though it is widely used in th= e industry. Packet loss rate could be measured in packet per second, wherea= s packet loss probability is unambiguously dimensionless. Indeed, in the ap= pendices, packet loss rates with respect to time are calculated.

5. = Rather than David's "reasonable" I suggest we use the phrase &quo= t;unambiguously safe" not just "safe" for the headings of Fi= gs 5-8 and in the text, as below. Otherwise, we are implying that it is uns= afe outside the region.

CURRENT:
   One may view this c= ondition as defining an operating envelope for a
   TDM PW, as= a TDM PW that occupies no more bandwidth than a TCP flow
   c= auses no more congestion than that TCP flow would.  Under this
&nbs= p;  condition it is safe to place the TDM PW along with congestion-   responsive traffic such as TCP, without causing additional   congestion.  on the other hand, were the TDM PW to consu= me
   significantly more bandwidth a TCP flow, it could contri= bute
   disproportionately to congestion, and its mixture with= congestion-
   responsive traffic might be inappropriate.
= SUGGESTED:
   One may view this condition as defining an unamb= iguously safe operating envelope for a
   TDM PW, as a TDM PW = that occupies no more bandwidth than a TCP flow
   causes no m= ore congestion than that TCP flow would.  This does not draw us into a= debate over whether TCP-friendliness is a valid constraint; it simply says= that there can be no question that a PW is safe if it does no more harm th= an a single TCP flow. Under this
   condition it is unambiguou= sly safe to place the TDM PW along with congestion-
   respons= ive traffic such as TCP.  On the other hand, were the TDM PW to consum= e
   significantly more bandwidth a TCP flow, it could contrib= ute
   disproportionately to congestion, and its mixture with = congestion-
   responsive traffic might start to cause concern= .

6. The analysis in Appendices A & B needs to mention that loss= bursts arise primarily from tail drop buffers, and the introduction of act= ive queue management (AQM) makes losses more independent of each other (les= s bursty). (I have references showing evidence of this.) In the comparisons= with TCP, we have assumed independent losses anyway, because they are the = worst case (and fortunately they are easier to analyse). On a similar point= , in appendix B, where we say "(bursty packet loss was also simulated = but is not reported here)", we ought to give a reason otherwise reader= s will be suspicious. A good reason would be that uniformly random is the w= orst case.

I've also responded to a couple of David's suggestions in= line (I agree with the others)...


At 17:44 13/08/2014, Yaakov St= ein wrote:
Bob told me on Friday of the IETF that he was going to look o= ver the final version and comment.
 
Bob – ping ?
 = ;
Y(J)S
 
From: Andrew G. Malis [ mailto:agmalis@gmail.com]
Sent: 13 August, = 2014 17:02
To: Yaakov Stein
Cc: Bob Briscoe; rbriscoe@jungle.bt.co.uk; BOCCI Matth= ew; Black, David
Subject: Re: [PWE3] WGLC comments on: draft-ietf= -pwe3-congcons-02.txt
 
Yaakov,
 
I still haven't hea= rd from Bob, but you should incorporate David's updates into a new revision= , and I'll use that version for my final Shepherd's review.
 
Th= anks,
Andy
 
On Mon, Aug 11, 2014 at 11:16 AM, Andrew G. Mali= s <agmalis@gmail.com> wrote:=
Actually, I'm going to end the LC as soon as I hear back from Bob on th= e IPR call. I sent him a reminder earlier this morning.
 
Cheers= ,
Andy
 
On Sun, Aug 10, 2014 at 9:53 AM, Andrew G. Malis <= ;agmalis@gmail.com> wrote:
D= avid,
 
That's perfect, thanks! I'm going to end the LC tomorrow= .
 
Cheers,
Andy
 
On Sat, Aug 9, 2014 at 10:32 PM= , Black, David <david.black@emc.c= om> wrote:
Following up on this ...

> As we're now movi= ng quickly to try to complete this draft , I agreed
> w/Yaakov that m= y author comments would be sent in as WG Last Call comments ...
> ...= so here they are:
... with some suggested new text ...

> Majo= r issues:
>
> The reference to a transport circuit breaker in s= ection 3 is a bit optimistic
> and high level:
>
> &= nbsp;     We can accomplish this by
>  =      employing a transport circuit breaker, by which we= mean an automatic
>       mechanism fo= r terminating a flow to prevent negative impact on other
>  = ;     flows and on the stability of the network
>=        [I-D.fairhurst-tsvwg-circuit-breaker].=
>
> This should be accompanied by at least an example of how t= o do this - past
> discussion has suggested carrier monitoring of del= ivered service quality
> in order to shut down PWs that are experienc= ing loss rates that result in
> unacceptable service quality (as defi= ned for us by the ITU ...).  I'd also
> like to add some text to= suggest that operators use this sort of circuit
> breaker practice w= hen TDM PWs compete with elastic traffic.
After this text in Section 3:<= br>
   We can accomplish this by
   employing a t= ransport circuit breaker, by which we mean an automatic
   mec= hanism for terminating a flow to prevent negative impact on other
 =   flows and on the stability of the network
   [I-D.fairh= urst-tsvwg-circuit-breaker].  Note that a transport circuit
 &= nbsp; breaker is intended as a protection mechanism of last resort, just as=
   an electrical circuit breaker is only triggered when absol= utely
   necessary.

I suggest adding:

 &nbs= p; For TDM pseudowires, such a circuit breaker can be realized via
 = ;  operator monitoring of drop rates and consequent on the quality of<= br>   TDM service delivered and shutting down TDM pseudowires tha= t persistently
   fail to deliver acceptable TDM service (as d= efined by [G775] and
   [G826]).  This document explains = why a TDM PW that is causing congestion
   problems via interf= erence with elastic traffic cannot be delivering
   acceptable= TDM service (see the rest of this Section, plus Appendices
  = A and B).

I don't think we should go there. We need to keep this dr= aft firmly as informational. If someone thinks a mechanism is needed becaus= e of the information in this draft, then they can write about a mechanism.<= br>

And breaking the paragraph after the added text.

> Fig= ure 1 and Figure 2 indicate that based on using the TCP throughput
> = equation as a proxy for potential impact on elastic traffic (which is what<= br>> we're doing in this draft), for the same fixed bandwidth PW, a larg= er segment
> size is less likely to cause impacts.  I'd like to = add a suggestion to use
> the larger PW segment sizes as long as they= 're consistent with delivery of
> the service.  There's been a r= elated (long-to-the-point-of-ratholed) debate
> over in the Transport= Area about whether packet-rate-limited/congestible
> forwarding is s= till relevant.  My view is that such equipment exists, and
> hen= ce that this is a relevant consideration for this draft, and would not
&= gt; mention that issue here.
Ignoring the rathole invitation ;-).  = This is still in section 3, after:

   We see that in gener= al for large packet sizes, short delays, and low
   packet los= s rates, the TDM pseudowires consume much less bandwidth
   th= an TCP would under identical conditions.  Only for small packets,
&= nbsp;  long delays, and high packet loss ratios, do TDM PWs potentiall= y
   consume more bandwidth, and even then only marginally.
I suggest adding:

   Comparing figures 1 and 2, use o= f a larger segment size for a T1 or E1
   TDM PW moves the ope= rational behavior of the PW away from the boundary
   at which= the TCP throughput equation suggests that undue interference with
 = ;  elastic traffic may start to occur.  We suggest that larger se= gment
   sizes be preferred for T1 and E1 TDM PWs, but note th= at other
   considerations may favor smaller segment sizes (e.= g., jitter buffer
   size, bounding the impact of an isolated = packet drop event).

There is no need to recommend any remedial actio= n, when we have just shown there is not a problem.

Further, the &quo= t;marginal problem" only applies when the TCP traffic uses packets sma= ller than 128B, which we only used as a hypothetical worst-case for the ana= lysis. But in practice this doesn't happen with long-running TCP flows - th= ey use packets that are as large as possible (irrespective of what the PW u= ses). So there really is /no/ problem to remedy.


and again, brea= king the paragraph after that added text.

I hope that latter sentenc= e ("We suggest that ...") is acceptable to the WG.

> Mi= nor issues:
>
> Section 2 is about Ethernet PWs that carry elas= tic traffic.  Should others
> be mentioned as also covered by th= is analysis - e.g., ATM and Frame Relay
> PWs carrying elastic traffi= c?
After:

   In this section we consider Ethernet PWs t= hat primarily carry
   congestion-responsive traffic.  We= show that we automatically obtain
   the desired congestion a= voidance behavior, and that additional
   mechanisms are not n= eeded.

I suggest adding:

   This discussion is appl= icable to any PW that carries congestion-
   responsive traffi= c (e.g., ATM and Frame Relay PWs); Ethernet PWs are
   used he= re for clarity, as they are a common example.

> Section 2 does no= t cover Ethernet PWs carrying inelastic traffic - that would
> be unu= sual, but should it be explicitly noted as out of scope, or just not
>= ; discussed (as is the case for the current text)?
I suggest doing nothi= ng here, and planning to deal with this concern
if it should arise later= (e.g., IETF Last Call or IESG Evaluation).

> Section 3 should sa= y that the reason the draft focuses on TDM PWs is that
> they're the = inelastic PWs that are likely to be carried over IP in practice.
> My= understanding is that other potentially inelastic PWs are MPLS-only
>= ; in practice.
After:

   Inelastic PWs, such as TDM PWs= ([RFC4553][RFC5086][RFC5087]), are
   potentially more proble= matic than the elastic PWs of the previous
   section.  B= eing constant bit-rate (CBR), TDM PWs can not be made
   respo= nsive to congestion.  On the other hand, being CBR, they also do
&n= bsp;  not attempt to capture additional bandwidth when neighboring TCP=
   flows back off.

I suggest adding:

 &nbs= p; We focus on TDM PWs here as they are the only inelastic PWs that are
=    likely to be IP-encapsulated in practice.  Most other ine= lastic PWs
   tend to only be used with MPLS.

[I'd ask = Yaakov and other PW experts to double-check the latter sentence,
please.= ]

> A terminology section to roll up all the acronym expansions (= e.g., TDM, CBR)
> in one place seems like a good idea.
These four = acronyms would be good to put into a terminology section:

CBR Consta= nt Bit Rate
PSN Packet Switched Network
PW Pseudowire
TDM Time Div= ision Multiplexing

>
> This text on p.11 could be incendiar= y:
>
>       Under this condition= it is safe to place the TDM PW along with
>    &= nbsp;  congestion - responsive traffic such as TCP, without causing>       additional congestion.
>> First of all to reduce the flammability, "safe" -> "= ;reasonable" and there
> are a number of other occurrences of &q= uot;safe" that deserve similar treatment ;-
> ).
> Beyond = that, this assertion is dependent on the operating conditions staying
&g= t; within that reasonable operating envelope, which entails use of somethin= g
> like a circuit breaker - I'd say that here, and reference the ear= lier
> circuit breaker discussion.
Global substitution: "safe= " -> "reasonable", starting with the abstract.

The= n for this specific text on p.11:

OLD
    &nb= sp;   Under this condition it is safe to place the TDM PW along w= ith
        congestion - responsive t= raffic such as TCP, without causing
      =   additional congestion.
NEW
      = ;  Under this condition it is reasonable to place the TDM PW in compet= ition
      with congestion-responsive traffic = such as TCP, as the result should not
      pro= duce unacceptable congestion effects.  That assumes that the operating=
      conditions remain within the reasonable = operating envelope - as discussed
      above, = a transport circuit breaker is a suggested means of managing a TDM
 = ;     PW so that it is shut down if operating condition= s remain persistently
      outside the reasona= ble operating envelope for the PW (e.g., due to a
   &nbs= p;  high packet loss rate).

No need to go there. See my suggest= ions earlier.


> Figures 7 and 8 could use some shading or lin= es to indicate the typical
> deployed operating regions for the pseud= o-wires to visibly show that they
> fall under the curves, particular= ly for larger segment sizes.  I'd also
> repeat or refer to the = suggestion to use larger

I wanted to suggest this myself, but it is = hard to depict fuzzy boundaries. If you can do this, pls do.

Given t= he PW operating regions are bounded by v low loss probability, the x axis i= n Figs 7&8 ought to be scaled much smaller as well.

Related poi= nt: the y axis in Fig 1 ought to max at 10, not 50.

Regards

<= br>
Bob

>
> Nits:
>
> References aren't allo= wed in abstracts, so remove reference to
> [I-D.fairhurst-tsvwg-circu= it-breaker]
>
> Introduction, first paragraph, delete "PWE= 3" towards end of paragraph -
>
>    prefer= ably the 4 byte PWE3 control word.
>
> In addition to its curre= nt general statements, the Security Considerations
> section should p= oint to the security considerations sections in a few
> particularly = relevant RFCs - I'd suggest 3985, 4553, 5086 and 5087.
After:

&nb= sp;  This document does not introduce any new congestion-specific
&= nbsp;  mechanisms and thus does not introduce any new security
&nbs= p;  considerations above those present for PWs in general.

I su= ggest adding a new paragraph:

   For PW security considera= tions, see the security considerations sections
   of the cite= d RFCS; the security considerations in [RFC3985], [RFC4553],
  = ; [RFC5086] and [RFC5087] are particularly applicable to this document.
=
>
> Thanks,
> --David
>
>
> > -----= Original Message-----
> > From: pwe3 [ mailto:pwe3-bounces@ietf.org] On Behalf Of Yaakov Stein> > Sent: Thursday, July 24, 2014 4:52 PM
> > To: pwe3@ietf.org
> > Subject: Re: [PWE3= ] I-D Action: draft-ietf-pwe3-congcons-02.txt
> >
> > Ple= ase look at the PDF version only
> > (the txt version does not hav= e the graphs).
> >
> > Y(J)S
> >
> >>
> _______________________________________________
> pwe3 = mailing list
> pwe3@ietf.org
= > https://www.iet= f.org/mailman/listinfo/pwe3

____________________________________= ___________
pwe3 mailing list
pwe3@i= etf.org
https= ://www.ietf.org/mailman/listinfo/pwe3
 
 
 
=
________________________________________________________________
Bob= Briscoe,           =             &nb= sp;            =             &nb= sp; BT
_______________________________________________
pwe3 mailing l= ist
pwe3@ietf.org
https://www.ietf.org/mailman/listi= nfo/pwe3

_____________________________________________= ___________________
Bob Briscoe,      &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     BT

= --_000_8D3D17ACE214DC429325B2B98F3AE712077BB42180MX15Acorpemcc_-- From nobody Tue Aug 19 08:02:10 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4699A1A0376 for ; Tue, 19 Aug 2014 08:02:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.266 X-Spam-Level: X-Spam-Status: No, score=-2.266 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, IP_NOT_FRIENDLY=0.334, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HzHTSrOGNYVr for ; Tue, 19 Aug 2014 08:02:05 -0700 (PDT) Received: from mx0b-00103a01.pphosted.com (mx0b-00103a01.pphosted.com [67.231.152.227]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E2E921A03BD for ; Tue, 19 Aug 2014 08:01:43 -0700 (PDT) Received: from pps.filterd (m0002317.ppops.net [127.0.0.1]) by mx0b-00103a01.pphosted.com (8.14.5/8.14.5) with SMTP id s7JEdjIh027464; Tue, 19 Aug 2014 11:01:39 -0400 Received: from vawvcgsie2k1301.ciena.com (LIN1-118-36-35.ciena.com [63.118.36.35]) by mx0b-00103a01.pphosted.com with ESMTP id 1nug4fm41c-1 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NOT); Tue, 19 Aug 2014 11:01:39 -0400 Received: from VAWVE2K13MBX01.ciena.com (10.4.156.87) by VAWVCGSIE2K1301.ciena.com (10.4.62.15) with Microsoft SMTP Server (TLS) id 15.0.847.32; Tue, 19 Aug 2014 11:01:32 -0400 Received: from ONWVEXCHHT01.ciena.com (10.128.6.16) by VAWVE2K13MBX01.ciena.com (10.4.156.87) with Microsoft SMTP Server (TLS) id 15.0.847.32; Tue, 19 Aug 2014 11:01:30 -0400 Received: from ONWVEXCHMB04.ciena.com ([::1]) by ONWVEXCHHT01.ciena.com ([::1]) with mapi; Tue, 19 Aug 2014 10:58:59 -0400 From: "Shah, Himanshu" To: "pwe3@ietf.org" , Stewart Bryant Date: Tue, 19 Aug 2014 10:58:55 -0400 Thread-Topic: VCCV for gal draft Thread-Index: Ac+7vhjDmv0d7JtJT/mOrOP1+yoMSw== Message-ID: <40746B2300A8FC4AB04EE722A593182B772B1D99@ONWVEXCHMB04.ciena.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US x-tm-as-product-ver: SMEX-10.0.0.1412-7.000.1014-20892.005 x-tm-as-result: No--49.510200-8.000000-31 x-tm-as-user-approved-sender: No x-tm-as-user-blocked-sender: No Content-Type: multipart/alternative; boundary="_000_40746B2300A8FC4AB04EE722A593182B772B1D99ONWVEXCHMB04cie_" MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.12.52, 1.0.27, 0.0.0000 definitions=2014-08-19_03:2014-08-19,2014-08-19,1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1402240000 definitions=main-1408190174 Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/KMa0DETvxEBaZx5Nmb5TThE0N80 Cc: Somnath Ojha Subject: [PWE3] VCCV for gal draft X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Aug 2014 15:02:08 -0000 --_000_40746B2300A8FC4AB04EE722A593182B772B1D99ONWVEXCHMB04cie_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Stewart, In IETF 89 WG meeting we discussed to revive http://tools.ietf.org/html/dra= ft-ietf-pwe3-vccv-for-gal-01 Relevant minutes from the meeting below. Stewart - you agreed to refresh it (we had discussions on the emails after = the meeting as well). If you are busy and would like me to, let me know. I had talked to Tom N. a while back and he was busy with SDN and no time to= pursue this, so one of us have to. Please let me know. Thanks, himanshu =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D minutes from IETF 89 PWE3 WG meeting = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Matthew: anything to say on VCCV for GAL? (VCCV type 4) Luca: where's Tom N? He was heading it. 3 documens 4447 bis for last call refresh adoption keep alive draft, but not sure if interest. If not, willi= ng to drop it. VCCV Type 4, Tom lead author, we set out a stable draft but= probably expired. Nothing current, willing to drop. WG: No! Luca: then we need someone to drive. Himanshu: I can help. We need the draft and the IANA assignment. Matthew: please sort out refreshing the draft. Himanshu: Yes. Is WG draft. YJS: Proposal for this. 3 drafts around this. THere is far less interest bu= t want to close type 4. We should make one draft that says here are the typ= es, deprecate 2 and add 4. Luca: rfc4447bis is just an errata with no tech changes. The document you = are talking about is there. We didn't want to go down a rat hole. Don't w= ant to open RFC 5085. Just need to progress this, it is in good shape. YJS: WG wanted to clean up all documents, but no time. So we will just prog= ress the current draft. --_000_40746B2300A8FC4AB04EE722A593182B772B1D99ONWVEXCHMB04cie_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi Stewart,

In IETF 89 WG meeting we discussed to revive http://tools.ietf.org/html/draft-= ietf-pwe3-vccv-for-gal-01

Relevant minutes from the meeting below.<= /i>

 

=

Stewart – you agreed to refresh i= t (we had discussions on the emails after the meeting as well).<= /span>

If you are busy and would= like me to, let me know.

= I had talked to Tom N. a while back and he was busy with SDN an= d no time to pursue this, so one of us have to.

 

Please let me know.

 

Thanks,

himanshu

 

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D minutes from IETF 89 PWE3 WG meetin= g =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

 

 

Matthew:= anything to say on VCCV for GAL? (VCCV type 4)

L= uca: where's Tom N? He was heading it.  3 documens 4447 bis for last c= all

refresh adoption keep alive draft, but not su= re if interest.  If not, willing to drop it.  VCCV Type 4, Tom le= ad author, we set out a stable draft but probably expired.  Nothing cu= rrent, willing to drop.

WG: No!

Luca: then we need someone to drive.

Hi= manshu: I can help.  We need the draft and the IANA assignment.

Matthew: please sort out refreshing the draft.

Himanshu: Yes. Is WG draft.

YJS= : Proposal for this. 3 drafts around this. THere is far less interest but w= ant to close type 4. We should make one draft that says here are the types,= deprecate 2 and add 4.

Luca: rfc4447bis is just = an errata with no tech changes.  The document you are talking about is= there.  We didn't want to go down a rat hole.  Don't want to ope= n RFC 5085.  Just need to progress this, it is in good shape.

YJS: WG wanted to clean up all documents, but no time. So= we will just progress the current draft.

&n= bsp;

 <= /span>

= --_000_40746B2300A8FC4AB04EE722A593182B772B1D99ONWVEXCHMB04cie_-- From nobody Tue Aug 19 08:13:23 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BB2871A0390 for ; Tue, 19 Aug 2014 08:13:21 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.6 X-Spam-Level: X-Spam-Status: No, score=-102.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PYkI93rihc7u for ; Tue, 19 Aug 2014 08:13:19 -0700 (PDT) Received: from usevmg20.ericsson.net (usevmg20.ericsson.net [198.24.6.45]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 50B1C1A0040 for ; Tue, 19 Aug 2014 08:13:19 -0700 (PDT) X-AuditID: c618062d-f79206d0000014d2-c6-53f3152c7e55 Received: from EUSAAHC008.ericsson.se (Unknown_Domain [147.117.188.96]) by usevmg20.ericsson.net (Symantec Mail Security) with SMTP id 15.2F.05330.C2513F35; Tue, 19 Aug 2014 11:13:17 +0200 (CEST) Received: from EUSAAMB103.ericsson.se ([147.117.188.120]) by EUSAAHC008.ericsson.se ([147.117.188.96]) with mapi id 14.03.0174.001; Tue, 19 Aug 2014 11:13:12 -0400 From: Gregory Mirsky To: "Shah, Himanshu" , "pwe3@ietf.org" , Stewart Bryant Thread-Topic: VCCV for gal draft Thread-Index: Ac+7vhjDmv0d7JtJT/mOrOP1+yoMSwAAaLYg Date: Tue, 19 Aug 2014 15:13:11 +0000 Message-ID: <7347100B5761DC41A166AC17F22DF1121B8217F0@eusaamb103.ericsson.se> References: <40746B2300A8FC4AB04EE722A593182B772B1D99@ONWVEXCHMB04.ciena.com> In-Reply-To: <40746B2300A8FC4AB04EE722A593182B772B1D99@ONWVEXCHMB04.ciena.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [147.117.188.11] Content-Type: multipart/alternative; boundary="_000_7347100B5761DC41A166AC17F22DF1121B8217F0eusaamb103erics_" MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFupmkeLIzCtJLcpLzFFi42KZXLonQVdX9HOwwf8VIha7Nq5mtOj7tIXF YvPHt+wW557OYXRg8Th78x+Lx5TfG1k9liz5yeQx8cwNxgCWKC6blNSczLLUIn27BK6M86e8 Cy4lV9z9l9rA2BHRxcjJISFgInG55zMbhC0mceHeeiCbi0NI4CijxPU1U6Cc5YwSG478ZwKp YhMwknixsYcdxBYRyJN49ukIK4jNDBS/+/gdC4gtLCAvcfzFTKgaBYm1J98yQthGEq23t4Jt YxFQlTjY+x+sl1fAV+L0/Ulg9UJA9oZH88HmcAr4SXxZshJsLyPQdd9PrWGC2CUucevJfCaI qwUkluw5zwxhi0q8fPyPFcJWkvj4ez47RH2+xIu1+6F2CUqcnPmEZQKj6Cwko2YhKZuFpAwi riOxYPcnNghbW2LZwtfMMPaZA4+ZkMUXMLKvYuQoLU4ty003MtjECIy9YxJsujsY97y0PMQo wMGoxMO74NzHYCHWxLLiytxDjNIcLErivLNq5wULCaQnlqRmp6YWpBbFF5XmpBYfYmTi4JRq YBQ79Xn6k41dMT/Mvzq+mnbMKq7VYe+15PK1fyrNPrktuHzzbtP6s+/W+udof95Ud51zumxP iNfyj2yvnU8nZRwQ0KyrXtox8b+XfYpy71fBpX/7i+x8TtgytllcZZnu1TvPITPR1n7tri83 /87Y/6wgb6rYjIlMDAIHLiRHVtQ+ORPpw/zpfqcSS3FGoqEWc1FxIgC8vSxNngIAAA== Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/S9BEYpyMyCKlW5CZlh3-JOxbWKw Cc: Somnath Ojha Subject: Re: [PWE3] VCCV for gal draft X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Aug 2014 15:13:21 -0000 --_000_7347100B5761DC41A166AC17F22DF1121B8217F0eusaamb103erics_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Himanshu, I think that earlier mail http://www.ietf.org/mail-archive/web/pwe3/current= /msg13778.html captured WG agreement on VCCV Type 4 Control Channel. I'd be= glad to work on the final version. Regards, Greg From: pwe3 [mailto:pwe3-bounces@ietf.org] On Behalf Of Shah, Himanshu Sent: Tuesday, August 19, 2014 7:59 AM To: pwe3@ietf.org; Stewart Bryant Cc: Somnath Ojha Subject: [PWE3] VCCV for gal draft Hi Stewart, In IETF 89 WG meeting we discussed to revive http://tools.ietf.org/html/dra= ft-ietf-pwe3-vccv-for-gal-01 Relevant minutes from the meeting below. Stewart - you agreed to refresh it (we had discussions on the emails after = the meeting as well). If you are busy and would like me to, let me know. I had talked to Tom N. a while back and he was busy with SDN and no time to= pursue this, so one of us have to. Please let me know. Thanks, himanshu =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D minutes from IETF 89 PWE3 WG meeting = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Matthew: anything to say on VCCV for GAL? (VCCV type 4) Luca: where's Tom N? He was heading it. 3 documens 4447 bis for last call refresh adoption keep alive draft, but not sure if interest. If not, willi= ng to drop it. VCCV Type 4, Tom lead author, we set out a stable draft but= probably expired. Nothing current, willing to drop. WG: No! Luca: then we need someone to drive. Himanshu: I can help. We need the draft and the IANA assignment. Matthew: please sort out refreshing the draft. Himanshu: Yes. Is WG draft. YJS: Proposal for this. 3 drafts around this. THere is far less interest bu= t want to close type 4. We should make one draft that says here are the typ= es, deprecate 2 and add 4. Luca: rfc4447bis is just an errata with no tech changes. The document you = are talking about is there. We didn't want to go down a rat hole. Don't w= ant to open RFC 5085. Just need to progress this, it is in good shape. YJS: WG wanted to clean up all documents, but no time. So we will just prog= ress the current draft. --_000_7347100B5761DC41A166AC17F22DF1121B8217F0eusaamb103erics_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi Himanshu,

I think that earlier m= ail http://www.ietf.org/mail-archive/web/pwe3/current/msg13778.html capture= d WG agreement on VCCV Type 4 Control Channel. I’d be glad to work on= the final version.

 

   &nbs= p;            Regard= s,

   &nbs= p;            &= nbsp;           &nbs= p;   Greg

 

From: pwe3 [ma= ilto:pwe3-bounces@ietf.org] On Behalf Of Shah, Himanshu
Sent: Tuesday, August 19, 2014 7:59 AM
To: pwe3@ietf.org; Stewart Bryant
Cc: Somnath Ojha
Subject: [PWE3] VCCV for gal draft

 

Hi Stewart,<= /o:p>

In IETF 89 WG mee= ting we discussed to revive http= ://tools.ietf.org/html/draft-ietf-pwe3-vccv-for-gal-01

Relevant minutes = from the meeting below.

 =

Stewart – y= ou agreed to refresh it (we had discussions on the emails after the meeting= as well).

If you are busy a= nd would like me to, let me know.

I had talked to T= om N. a while back and he was busy with SDN and no time to pursue this, so = one of us have to.

 =

Please let me kno= w.

 =

Thanks,

himanshu

 =

 =

=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D minutes from IETF 89 PWE3 WG meeting =3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

 =

 =

Matthew: anything to say on VCCV for GAL? (VCCV type 4)

Luca: where's Tom N? He was heading it.  3 documens 4= 447 bis for last call

refresh adoption keep alive draft, but not sure if interes= t.  If not, willing to drop it.  VCCV Type 4, Tom lead author, we= set out a stable draft but probably expired.  Nothing current, willing to drop.

WG: No!

Luca: then we need someone to drive.

Himanshu: I can help.  We need the draft and the IANA= assignment.

Matthew: please sort out refreshing the draft.<= /span>

Himanshu: Yes. Is WG draft.

YJS: Proposal for this. 3 drafts around this. THere is far= less interest but want to close type 4. We should make one draft that says= here are the types, deprecate 2 and add 4.

Luca: rfc4447bis is just an errata with no tech changes.&n= bsp; The document you are talking about is there.  We didn't want to g= o down a rat hole.  Don't want to open RFC 5085.  Just need to progress this, it is in good shape.

YJS: WG wanted to clean up all documents, but no time. So = we will just progress the current draft.

 

 =

--_000_7347100B5761DC41A166AC17F22DF1121B8217F0eusaamb103erics_-- From nobody Tue Aug 19 12:37:41 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DECA51A0ADF for ; Tue, 19 Aug 2014 12:37:39 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -15.168 X-Spam-Level: X-Spam-Status: No, score=-15.168 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-0.668, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pl-SmOTbTFZX for ; Tue, 19 Aug 2014 12:37:38 -0700 (PDT) Received: from rcdn-iport-8.cisco.com (rcdn-iport-8.cisco.com [173.37.86.79]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0AE3B1A0B00 for ; Tue, 19 Aug 2014 12:37:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=12700; q=dns/txt; s=iport; t=1408477058; x=1409686658; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=d5/4wZN0gnqxCVxvzscJekyS6Uj43KQ4I9Pho9GGejI=; b=cYlhsT30bT5DBLZABfG0pm+ZLgz3BG0S4lNcAuUHkHdmMmsty8b0ljMa AibbZAPlo7KcuaDzyZlODXkavGJikr7jLJaANHWUL9X6yMAPmd40JSZuq WKYaHYm3+A5E/WyymlMI2eofR/0SSAYtt7e50c+TvKtFUt2MVnP1KnhPG 0=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqMLAHum81OtJV2S/2dsb2JhbABagkcjI1NXBIQUxnSBWQEJh1kBgQwWd4QEAQEEAQEBawsQAgEIPwcnCxQRAQEEDgWIQgEMwHcXj0gEB4MvgR0FjxKCE4QmhneBV5Msg11sgUiBBwEBAQ X-IronPort-AV: E=Sophos;i="5.01,897,1400025600"; d="scan'208,217";a="348742262" Received: from rcdn-core-10.cisco.com ([173.37.93.146]) by rcdn-iport-8.cisco.com with ESMTP; 19 Aug 2014 19:37:37 +0000 Received: from xhc-aln-x01.cisco.com (xhc-aln-x01.cisco.com [173.36.12.75]) by rcdn-core-10.cisco.com (8.14.5/8.14.5) with ESMTP id s7JJbaFd000793 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Tue, 19 Aug 2014 19:37:37 GMT Received: from xmb-aln-x02.cisco.com ([fe80::8c1c:7b85:56de:ffd1]) by xhc-aln-x01.cisco.com ([173.36.12.75]) with mapi id 14.03.0195.001; Tue, 19 Aug 2014 14:37:36 -0500 From: "Carlos Pignataro (cpignata)" To: "Shah, Himanshu" Thread-Topic: [PWE3] VCCV for gal draft Thread-Index: Ac+7vhjDmv0d7JtJT/mOrOP1+yoMSwAUN1AA Date: Tue, 19 Aug 2014 19:37:35 +0000 Message-ID: <141D7BE3-A795-4F6B-A562-41AE80EFBB64@cisco.com> References: <40746B2300A8FC4AB04EE722A593182B772B1D99@ONWVEXCHMB04.ciena.com> In-Reply-To: <40746B2300A8FC4AB04EE722A593182B772B1D99@ONWVEXCHMB04.ciena.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.117.115.59] Content-Type: multipart/alternative; boundary="_000_141D7BE3A7954F6BA56241AE80EFBB64ciscocom_" MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/PdL3ypKVsI7iKDnDwqv9rccjGcU Cc: "pwe3@ietf.org" , Somnath Ojha , "Stewart Bryant \(stbryant\)" Subject: Re: [PWE3] VCCV for gal draft X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Aug 2014 19:37:40 -0000 --_000_141D7BE3A7954F6BA56241AE80EFBB64ciscocom_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Hi, Himanshu, Going from memory and not checking the history breadcrums (so I might be to= tally wrong), but -- wasn't the idea to have: http://tools.ietf.org/html/draft-nadeau-pwe3-vccv2-00 Replace http://tools.ietf.org/html/draft-ietf-pwe3-vccv-for-gal-01 ? Thanks, Carlos. On Aug 19, 2014, at 10:58 AM, Shah, Himanshu > wrote: Hi Stewart, In IETF 89 WG meeting we discussed to revive http://tools.ietf.org/html/dra= ft-ietf-pwe3-vccv-for-gal-01 Relevant minutes from the meeting below. Stewart =96 you agreed to refresh it (we had discussions on the emails afte= r the meeting as well). If you are busy and would like me to, let me know. I had talked to Tom N. a while back and he was busy with SDN and no time to= pursue this, so one of us have to. Please let me know. Thanks, himanshu =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D minutes from IETF 89 PWE3 WG meeting = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Matthew: anything to say on VCCV for GAL? (VCCV type 4) Luca: where's Tom N? He was heading it. 3 documens 4447 bis for last call refresh adoption keep alive draft, but not sure if interest. If not, willi= ng to drop it. VCCV Type 4, Tom lead author, we set out a stable draft but= probably expired. Nothing current, willing to drop. WG: No! Luca: then we need someone to drive. Himanshu: I can help. We need the draft and the IANA assignment. Matthew: please sort out refreshing the draft. Himanshu: Yes. Is WG draft. YJS: Proposal for this. 3 drafts around this. THere is far less interest bu= t want to close type 4. We should make one draft that says here are the typ= es, deprecate 2 and add 4. Luca: rfc4447bis is just an errata with no tech changes. The document you = are talking about is there. We didn't want to go down a rat hole. Don't w= ant to open RFC 5085. Just need to progress this, it is in good shape. YJS: WG wanted to clean up all documents, but no time. So we will just prog= ress the current draft. _______________________________________________ pwe3 mailing list pwe3@ietf.org https://www.ietf.org/mailman/listinfo/pwe3 --_000_141D7BE3A7954F6BA56241AE80EFBB64ciscocom_ Content-Type: text/html; charset="Windows-1252" Content-ID: Content-Transfer-Encoding: quoted-printable Hi, Himanshu,

Going from memory and not checking the history breadcrums (so I might = be totally wrong), but -- wasn't the idea to have:
Replace
?

Thanks,

Carlos.

On Aug 19, 2014, at 10:58 AM, Shah, Himanshu <hshah@ciena.com> wrote:

Hi Stewart,
In IETF 89 WG meeting we discussed to revive http://tools.ietf.org/html/draft-ietf-pwe3-vccv-for-gal-= 01
Relevant minutes from the meeting below.
 
Stewart =96 you agreed to refresh it (we had discu= ssions on the emails after the meeting as well).
If you are busy and would like me to, let me know.=
I had talked to Tom N. a while back and he was bus= y with SDN and no time to pursue this, so one of us have to.
 
Please let me know.
 
Thanks,
himanshu
 
 
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D minutes from = IETF 89 PWE3 WG meeting =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
 
 
Matthew: anyth= ing to say on VCCV for GAL? (VCCV type 4)
Luca: where's = Tom N? He was heading it.  3 documens 4447 bis for last call
refresh adopti= on keep alive draft, but not sure if interest.  If not, willing to dro= p it.  VCCV Type 4, Tom lead author, we set out a stable draft but pro= bably expired.  Nothing current, willing to drop.
WG: No!
Luca: then we = need someone to drive.
Himanshu: I ca= n help.  We need the draft and the IANA assignment.<= /div>
Matthew: pleas= e sort out refreshing the draft.
Himanshu: Yes.= Is WG draft.
YJS: Proposal = for this. 3 drafts around this. THere is far less interest but want to clos= e type 4. We should make one draft that says here are the types, deprecate = 2 and add 4.
Luca: rfc4447b= is is just an errata with no tech changes.  The document you are talki= ng about is there.  We didn't want to go down a rat hole.  Don't = want to open RFC 5085.  Just need to progress this, it is in good shape.
YJS: WG wanted= to clean up all documents, but no time. So we will just progress the curre= nt draft.
 <= /div>
 
_______________________________________________
pwe3 mailing list
pwe3@ietf.org
https://www.ietf.org/mailman/listinfo/pwe3=

--_000_141D7BE3A7954F6BA56241AE80EFBB64ciscocom_-- From nobody Tue Aug 19 13:11:06 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F28551A6EF1 for ; Tue, 19 Aug 2014 13:11:03 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.266 X-Spam-Level: X-Spam-Status: No, score=-2.266 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, IP_NOT_FRIENDLY=0.334, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hsMwNHUT5MvM for ; Tue, 19 Aug 2014 13:10:56 -0700 (PDT) Received: from mx0b-00103a01.pphosted.com (mx0b-00103a01.pphosted.com [67.231.152.227]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A1F901A6EE5 for ; Tue, 19 Aug 2014 13:10:56 -0700 (PDT) Received: from pps.filterd (m0002317.ppops.net [127.0.0.1]) by mx0b-00103a01.pphosted.com (8.14.5/8.14.5) with SMTP id s7JKATbg028888; Tue, 19 Aug 2014 16:10:53 -0400 Received: from mdwexght02.ciena.com (LIN1-118-36-29.ciena.com [63.118.36.29]) by mx0b-00103a01.pphosted.com with ESMTP id 1nvadwrw43-2 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Tue, 19 Aug 2014 16:10:53 -0400 Received: from MDWEXCHCGSIHT01.ciena.com (10.4.140.106) by MDWEXGHT02.ciena.com (10.4.140.213) with Microsoft SMTP Server (TLS) id 8.3.298.1; Tue, 19 Aug 2014 16:10:53 -0400 Received: from ONWVEXCHHT04.ciena.com (10.128.6.44) by MDWEXCHCGSIHT01.ciena.com (10.4.140.106) with Microsoft SMTP Server (TLS) id 8.3.298.1; Tue, 19 Aug 2014 16:10:52 -0400 Received: from ONWVEXCHMB04.ciena.com ([::1]) by ONWVEXCHHT04.ciena.com ([::1]) with mapi; Tue, 19 Aug 2014 16:10:51 -0400 From: "Shah, Himanshu" To: "Carlos Pignataro (cpignata)" Date: Tue, 19 Aug 2014 16:10:50 -0400 Thread-Topic: [PWE3] VCCV for gal draft Thread-Index: Ac+7vhjDmv0d7JtJT/mOrOP1+yoMSwAUN1AAAAliNVA= Message-ID: <40746B2300A8FC4AB04EE722A593182B772B2030@ONWVEXCHMB04.ciena.com> References: <40746B2300A8FC4AB04EE722A593182B772B1D99@ONWVEXCHMB04.ciena.com> <141D7BE3-A795-4F6B-A562-41AE80EFBB64@cisco.com> In-Reply-To: <141D7BE3-A795-4F6B-A562-41AE80EFBB64@cisco.com> Accept-Language: en-US, en-CA Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-CA X-TM-AS-Product-Ver: SMEX-10.0.0.1412-7.000.1014-20894.002 X-TM-AS-Result: No--40.224000-0.000000-31 X-TM-AS-User-Approved-Sender: Yes X-TM-AS-User-Blocked-Sender: No Content-Type: multipart/alternative; boundary="_000_40746B2300A8FC4AB04EE722A593182B772B2030ONWVEXCHMB04cie_" MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.12.52, 1.0.27, 0.0.0000 definitions=2014-08-19_05:2014-08-19,2014-08-19,1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1402240000 definitions=main-1408190232 Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/teC_xdWGBj0oB8RMSR3DCyLIs_c Cc: "pwe3@ietf.org" , Somnath Ojha , "Stewart Bryant \(stbryant\)" Subject: Re: [PWE3] VCCV for gal draft X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Aug 2014 20:11:04 -0000 --_000_40746B2300A8FC4AB04EE722A593182B772B2030ONWVEXCHMB04cie_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Carlos - Was discussed. See the last 4 lines of the minutes (below). Consensus was to progress the "vccv-for-gal" draft. Thanks, himanshu From: Carlos Pignataro (cpignata) [mailto:cpignata@cisco.com] Sent: Tuesday, August 19, 2014 3:38 PM To: Shah, Himanshu Cc: pwe3@ietf.org; Stewart Bryant (stbryant); Somnath Ojha Subject: Re: [PWE3] VCCV for gal draft Hi, Himanshu, Going from memory and not checking the history breadcrums (so I might be to= tally wrong), but -- wasn't the idea to have: http://tools.ietf.org/html/draft-nadeau-pwe3-vccv2-00 Replace http://tools.ietf.org/html/draft-ietf-pwe3-vccv-for-gal-01 ? Thanks, Carlos. On Aug 19, 2014, at 10:58 AM, Shah, Himanshu > wrote: Hi Stewart, In IETF 89 WG meeting we discussed to revive http://tools.ietf.org/html/dra= ft-ietf-pwe3-vccv-for-gal-01 Relevant minutes from the meeting below. Stewart - you agreed to refresh it (we had discussions on the emails after = the meeting as well). If you are busy and would like me to, let me know. I had talked to Tom N. a while back and he was busy with SDN and no time to= pursue this, so one of us have to. Please let me know. Thanks, himanshu =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D minutes from IETF 89 PWE3 WG meeting = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Matthew: anything to say on VCCV for GAL? (VCCV type 4) Luca: where's Tom N? He was heading it. 3 documens 4447 bis for last call refresh adoption keep alive draft, but not sure if interest. If not, willi= ng to drop it. VCCV Type 4, Tom lead author, we set out a stable draft but= probably expired. Nothing current, willing to drop. WG: No! Luca: then we need someone to drive. Himanshu: I can help. We need the draft and the IANA assignment. Matthew: please sort out refreshing the draft. Himanshu: Yes. Is WG draft. YJS: Proposal for this. 3 drafts around this. THere is far less interest bu= t want to close type 4. We should make one draft that says here are the typ= es, deprecate 2 and add 4. Luca: rfc4447bis is just an errata with no tech changes. The document you = are talking about is there. We didn't want to go down a rat hole. Don't w= ant to open RFC 5085. Just need to progress this, it is in good shape. YJS: WG wanted to clean up all documents, but no time. So we will just prog= ress the current draft. _______________________________________________ pwe3 mailing list pwe3@ietf.org https://www.ietf.org/mailman/listinfo/pwe3 --_000_40746B2300A8FC4AB04EE722A593182B772B2030ONWVEXCHMB04cie_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi Carlos= –

Was discussed. =

See the last 4 lines of= the minutes (below).

C= onsensus was to progress the “vccv-for-gal” draft.

 

Thanks,

himanshu

&nbs= p;

From: Carlos Pignat= aro (cpignata) [mailto:cpignata@cisco.com]
Sent: Tuesday, August= 19, 2014 3:38 PM
To: Shah, Himanshu
Cc: pwe3@ietf.org;= Stewart Bryant (stbryant); Somnath Ojha
Subject: Re: [PWE3] VCCV= for gal draft

&= nbsp;

Hi, Himanshu,

 

Going = from memory and not checking the history breadcrums (so I might be totally = wrong), but -- wasn't the idea to have:

Replace

?

 

Thanks= ,

 

=

Carlos.

 

On Aug 19, 2014, a= t 10:58 AM, Shah, Himanshu <hshah@cie= na.com> wrote:



Hi Stewart,=

In IETF 8= 9 WG meeting we discussed to revive&nbs= p;http://tools.ietf.org/html/draft-ietf-pwe= 3-vccv-for-gal-01

Relevant minutes from the meeting below.=

 

Stewa= rt – you agreed to refresh it (we had discussions on the emails after= the meeting as well).

If you are busy and would like me to, let me know.<= o:p>

I had t= alked to Tom N. a while back and he was busy with SDN and no time to pursue= this, so one of us have to.

 

Please let me know.=

 

Thanks,

himanshu=

 

 = ;

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D minutes from IETF 89 PWE3 WG meeting = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

<= /div>

 =

 

Matthew: anything to say on VCCV for GAL? (VCCV typ= e 4)

Luca: where's Tom N? He was headi= ng it.  3 documens 4447 bis for last call

<= div>

refresh adoption keep alive draft, but not sure if interest. = If not, willing to drop it.  VCCV Type 4, Tom lead author, we set out= a stable draft but probably expired.  Nothing current, willing to dro= p.

WG: No!

=

Luca: then we need someone to drive.

<= div>

Himanshu: I can help.  We need the draft and the IANA assignm= ent.

Matthew: please sort out refreshi= ng the draft.

Himanshu: Yes. Is WG dra= ft.

YJS: Proposal for this. 3 drafts a= round this. THere is far less interest but want to close type 4. We should = make one draft that says here are the types, deprecate 2 and add 4.<= span style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"'>

Luca: rfc4447bis is just an errata with no te= ch changes.  The document you are talking about is there.  We did= n't want to go down a rat hole.  Don't want to open RFC 5085.  Ju= st need to progress this, it is in good shape.

<= div>

YJS: WG wanted to clean up all documents, but no time. So we will = just progress the current draft.

&nb= sp;

&nb= sp;

_________________________= ______________________
pwe3 mailing list
pwe3@ietf.org
http= s://www.ietf.org/mailman/listinfo/pwe3

 

= --_000_40746B2300A8FC4AB04EE722A593182B772B2030ONWVEXCHMB04cie_-- From nobody Tue Aug 19 13:13:48 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 883341A6ED8 for ; Tue, 19 Aug 2014 13:13:46 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.266 X-Spam-Level: X-Spam-Status: No, score=-2.266 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, IP_NOT_FRIENDLY=0.334, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IoJqFtxvlKof for ; Tue, 19 Aug 2014 13:13:44 -0700 (PDT) Received: from mx0b-00103a01.pphosted.com (mx0b-00103a01.pphosted.com [67.231.152.227]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 467F71A0B73 for ; Tue, 19 Aug 2014 13:13:44 -0700 (PDT) Received: from pps.filterd (m0002317.ppops.net [127.0.0.1]) by mx0b-00103a01.pphosted.com (8.14.5/8.14.5) with SMTP id s7JKAFJW028827; Tue, 19 Aug 2014 16:13:39 -0400 Received: from mdwvexchht01.ciena.com (LIN1-118-36-28.ciena.com [63.118.36.28]) by mx0b-00103a01.pphosted.com with ESMTP id 1nvadwrwb4-1 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Tue, 19 Aug 2014 16:13:39 -0400 Received: from ONWVEXCHHT04.ciena.com (10.128.6.44) by MDWVEXCHHT01.ciena.com (10.4.156.175) with Microsoft SMTP Server (TLS) id 8.3.298.1; Tue, 19 Aug 2014 16:13:38 -0400 Received: from ONWVEXCHMB04.ciena.com ([::1]) by ONWVEXCHHT04.ciena.com ([::1]) with mapi; Tue, 19 Aug 2014 16:13:37 -0400 From: "Shah, Himanshu" To: Gregory Mirsky , "pwe3@ietf.org" , Stewart Bryant Date: Tue, 19 Aug 2014 16:13:36 -0400 Thread-Topic: VCCV for gal draft Thread-Index: Ac+7vhjDmv0d7JtJT/mOrOP1+yoMSwAAaLYgAAqKWgA= Message-ID: <40746B2300A8FC4AB04EE722A593182B772B2032@ONWVEXCHMB04.ciena.com> References: <40746B2300A8FC4AB04EE722A593182B772B1D99@ONWVEXCHMB04.ciena.com> <7347100B5761DC41A166AC17F22DF1121B8217F0@eusaamb103.ericsson.se> In-Reply-To: <7347100B5761DC41A166AC17F22DF1121B8217F0@eusaamb103.ericsson.se> Accept-Language: en-US, en-CA Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-CA X-TM-AS-Product-Ver: SMEX-10.0.0.1412-7.000.1014-20894.002 X-TM-AS-Result: No--32.836600-8.000000-31 X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No Content-Type: multipart/alternative; boundary="_000_40746B2300A8FC4AB04EE722A593182B772B2032ONWVEXCHMB04cie_" MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.12.52, 1.0.27, 0.0.0000 definitions=2014-08-19_05:2014-08-19,2014-08-19,1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1402240000 definitions=main-1408190232 Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/NpDkhSbyYEL1M2NNvEqkgsPl_98 Cc: Somnath Ojha Subject: Re: [PWE3] VCCV for gal draft X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Aug 2014 20:13:47 -0000 --_000_40746B2300A8FC4AB04EE722A593182B772B2032ONWVEXCHMB04cie_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Greg - Lets wait for Stewart to respond. If he is busy, you and I will take up to refresh.. Thanks, himanshu From: Gregory Mirsky [mailto:gregory.mirsky@ericsson.com] Sent: Tuesday, August 19, 2014 11:13 AM To: Shah, Himanshu; pwe3@ietf.org; Stewart Bryant Cc: Somnath Ojha Subject: RE: VCCV for gal draft Hi Himanshu, I think that earlier mail http://www.ietf.org/mail-archive/web/pwe3/current= /msg13778.html captured WG agreement on VCCV Type 4 Control Channel. I'd be= glad to work on the final version. Regards, Greg From: pwe3 [mailto:pwe3-bounces@ietf.org] On Behalf Of Shah, Himanshu Sent: Tuesday, August 19, 2014 7:59 AM To: pwe3@ietf.org; Stewart Bryant Cc: Somnath Ojha Subject: [PWE3] VCCV for gal draft Hi Stewart, In IETF 89 WG meeting we discussed to revive http://tools.ietf.org/html/dra= ft-ietf-pwe3-vccv-for-gal-01 Relevant minutes from the meeting below. Stewart - you agreed to refresh it (we had discussions on the emails after = the meeting as well). If you are busy and would like me to, let me know. I had talked to Tom N. a while back and he was busy with SDN and no time to= pursue this, so one of us have to. Please let me know. Thanks, himanshu =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D minutes from IETF 89 PWE3 WG meeting = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Matthew: anything to say on VCCV for GAL? (VCCV type 4) Luca: where's Tom N? He was heading it. 3 documens 4447 bis for last call refresh adoption keep alive draft, but not sure if interest. If not, willi= ng to drop it. VCCV Type 4, Tom lead author, we set out a stable draft but= probably expired. Nothing current, willing to drop. WG: No! Luca: then we need someone to drive. Himanshu: I can help. We need the draft and the IANA assignment. Matthew: please sort out refreshing the draft. Himanshu: Yes. Is WG draft. YJS: Proposal for this. 3 drafts around this. THere is far less interest bu= t want to close type 4. We should make one draft that says here are the typ= es, deprecate 2 and add 4. Luca: rfc4447bis is just an errata with no tech changes. The document you = are talking about is there. We didn't want to go down a rat hole. Don't w= ant to open RFC 5085. Just need to progress this, it is in good shape. YJS: WG wanted to clean up all documents, but no time. So we will just prog= ress the current draft. --_000_40746B2300A8FC4AB04EE722A593182B772B2032ONWVEXCHMB04cie_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable = --_000_40746B2300A8FC4AB04EE722A593182B772B2032ONWVEXCHMB04cie_-- From nobody Tue Aug 19 14:10:07 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 859A41A891A for ; Tue, 19 Aug 2014 14:10:00 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -15.168 X-Spam-Level: X-Spam-Status: No, score=-15.168 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-0.668, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MECwxV1tQEHH for ; Tue, 19 Aug 2014 14:09:56 -0700 (PDT) Received: from alln-iport-3.cisco.com (alln-iport-3.cisco.com [173.37.142.90]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9F5BB1A012D for ; Tue, 19 Aug 2014 14:09:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=20277; q=dns/txt; s=iport; t=1408482596; x=1409692196; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=J/DOoDQR8UhvTtW3zR5+ynKRGdKxDheIKeZC1IKpy3U=; b=CGudZCuIgR5SdyjIXv61JDv3LGgweAgxrXJrV1kipy6Tui0l8HHr4IiH uWMwBQ5lL9vxQKSeDipfJnTJu+r71L2iFvuhljoLJ0SNOdAYSCcZ7DVT7 B+Z0f6vBzXrCBRU7BT04nbLtVA7ZTDm3LaMh1NQIqnrc21GmQ+rDYL+Qa c=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AjcFAOi781OtJA2J/2dsb2JhbABagkcjI1NXBMsMgVkBCYdZAYEMFneEAwEBAQQBAQEqQQsQAgEIEQMBAQEhBwcnCxQJCAEBBA4FiEIBDMEFF48oEw0EBgGETAWPEoIThCaGd4FXkyyDXWyBSIEHAQEB X-IronPort-AV: E=Sophos; i="5.01,897,1400025600"; d="scan'208,217"; a="70638064" Received: from alln-core-4.cisco.com ([173.36.13.137]) by alln-iport-3.cisco.com with ESMTP; 19 Aug 2014 21:09:55 +0000 Received: from xhc-rcd-x08.cisco.com (xhc-rcd-x08.cisco.com [173.37.183.82]) by alln-core-4.cisco.com (8.14.5/8.14.5) with ESMTP id s7JL9t89001990 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Tue, 19 Aug 2014 21:09:55 GMT Received: from xmb-aln-x02.cisco.com ([fe80::8c1c:7b85:56de:ffd1]) by xhc-rcd-x08.cisco.com ([173.37.183.82]) with mapi id 14.03.0195.001; Tue, 19 Aug 2014 16:09:55 -0500 From: "Carlos Pignataro (cpignata)" To: "Shah, Himanshu" Thread-Topic: [PWE3] VCCV for gal draft Thread-Index: Ac+7vhjDmv0d7JtJT/mOrOP1+yoMSwAUN1AAAAliNVD//4unAA== Date: Tue, 19 Aug 2014 21:09:55 +0000 Message-ID: References: <40746B2300A8FC4AB04EE722A593182B772B1D99@ONWVEXCHMB04.ciena.com> <141D7BE3-A795-4F6B-A562-41AE80EFBB64@cisco.com> <40746B2300A8FC4AB04EE722A593182B772B2030@ONWVEXCHMB04.ciena.com> In-Reply-To: <40746B2300A8FC4AB04EE722A593182B772B2030@ONWVEXCHMB04.ciena.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.3.140616 x-originating-ip: [10.82.226.2] Content-Type: multipart/alternative; boundary="_000_D01934B164C3Bcpignataciscocom_" MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/oHjtztHwVGt167MfKXuXwaJ6t9k Cc: "pwe3@ietf.org" , Somnath Ojha , "Stewart Bryant \(stbryant\)" Subject: Re: [PWE3] VCCV for gal draft X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Aug 2014 21:10:00 -0000 --_000_D01934B164C3Bcpignataciscocom_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Hi, Himanshu, Looking at draft-nadeau-pwe3-vccv2-00, refreshing my memory, it tries to ra= tionalize the use of CC Types, including the CC Type 4 defined in draft-iet= f-pwe3-vccv-for-gal-01. So CC Type 4 is just like you said defined in draft= -ietf-pwe3-vccv-for-gal. Thanks, Carlos. From: , Himanshu > Date: Tuesday, August 19, 2014 at 4:10 PM To: Carlos Pignataro > Cc: "pwe3@ietf.org" >, Stewart Bryant >, Somna= th Ojha > Subject: RE: [PWE3] VCCV for gal draft Hi Carlos =96 Was discussed. See the last 4 lines of the minutes (below). Consensus was to progress the =93vccv-for-gal=94 draft. Thanks, himanshu From: Carlos Pignataro (cpignata) [mailto:cpignata@cisco.com] Sent: Tuesday, August 19, 2014 3:38 PM To: Shah, Himanshu Cc: pwe3@ietf.org; Stewart Bryant (stbryant); Somnath= Ojha Subject: Re: [PWE3] VCCV for gal draft Hi, Himanshu, Going from memory and not checking the history breadcrums (so I might be to= tally wrong), but -- wasn't the idea to have: http://tools.ietf.org/html/draft-nadeau-pwe3-vccv2-00 Replace http://tools.ietf.org/html/draft-ietf-pwe3-vccv-for-gal-01 ? Thanks, Carlos. On Aug 19, 2014, at 10:58 AM, Shah, Himanshu > wrote: Hi Stewart, In IETF 89 WG meeting we discussed to revive http://tools.ietf.org/html/dra= ft-ietf-pwe3-vccv-for-gal-01 Relevant minutes from the meeting below. Stewart =96 you agreed to refresh it (we had discussions on the emails afte= r the meeting as well). If you are busy and would like me to, let me know. I had talked to Tom N. a while back and he was busy with SDN and no time to= pursue this, so one of us have to. Please let me know. Thanks, himanshu =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D minutes from IETF 89 PWE3 WG meeting = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Matthew: anything to say on VCCV for GAL? (VCCV type 4) Luca: where's Tom N? He was heading it. 3 documens 4447 bis for last call refresh adoption keep alive draft, but not sure if interest. If not, willi= ng to drop it. VCCV Type 4, Tom lead author, we set out a stable draft but= probably expired. Nothing current, willing to drop. WG: No! Luca: then we need someone to drive. Himanshu: I can help. We need the draft and the IANA assignment. Matthew: please sort out refreshing the draft. Himanshu: Yes. Is WG draft. YJS: Proposal for this. 3 drafts around this. THere is far less interest bu= t want to close type 4. We should make one draft that says here are the typ= es, deprecate 2 and add 4. Luca: rfc4447bis is just an errata with no tech changes. The document you = are talking about is there. We didn't want to go down a rat hole. Don't w= ant to open RFC 5085. Just need to progress this, it is in good shape. YJS: WG wanted to clean up all documents, but no time. So we will just prog= ress the current draft. _______________________________________________ pwe3 mailing list pwe3@ietf.org https://www.ietf.org/mailman/listinfo/pwe3 --_000_D01934B164C3Bcpignataciscocom_ Content-Type: text/html; charset="Windows-1252" Content-ID: <3CBD856D5FF0E24794CB69D7B4EED915@emea.cisco.com> Content-Transfer-Encoding: quoted-printable
Hi, Himanshu,

Looking at draft-nadeau-pwe3-vccv2-00, refreshing my memory, it tries = to rationalize the use of CC Types, including the CC Type 4 defined in draf= t-ietf-pwe3-vccv-for-gal-01. So CC Type 4 is just like you said defined in = draft-ietf-pwe3-vccv-for-gal.

Thanks,

Carlos.

From: <Shah>, Himanshu <hshah@ciena.com>
Date: Tuesday, August 19, 2014 at 4= :10 PM
To: Carlos Pignataro <cpignata@cisco.com>
Cc: "pwe3@ietf.org" <pwe3@i= etf.org>, Stewart Bryant <s= tbryant@cisco.com>, Somnath Ojha <somnath@india.tejasnetworks.com>
Subject: RE: [PWE3] VCCV for gal dr= aft

Hi Carlos =96

Was discussed.

See the last 4 lines of the minutes (b= elow).

Consensus was to progress the =93vccv-= for-gal=94 draft.

 

Thanks,

himanshu

 

From: Carlos Pignataro (cpignata) [mailto:cpignata@cisco.com]
Sent: Tuesday, August 19, 2014 3:38 PM
To: Shah, Himanshu
Cc: pwe3@ietf.org; Stewart Brya= nt (stbryant); Somnath Ojha
Subject: Re: [PWE3] VCCV for gal draft

 

Hi, Himanshu,

 

Going from memory and not checking the history bread= crums (so I might be totally wrong), but -- wasn't the idea to have:

Replace

?

 

Thanks,

 

Carlos.

 

On Aug 19, 2014, at 10:58 AM, Shah, Himanshu <hshah@ciena.com> wrote:



Hi Stewart,

In IETF 89 WG meeting we di= scussed to revive http://tools.ietf.org/html/draft-ietf-pwe3-vccv-for-gal-0= 1

Relevant minutes from the m= eeting below.

 =

Stewart =96 you agreed to r= efresh it (we had discussions on the emails after the meeting as well).<= o:p>

If you are busy and would l= ike me to, let me know.

I had talked to Tom N. a wh= ile back and he was busy with SDN and no time to pursue this, so one of us = have to.

 =

Please let me know.<= /i>=

 =

Thanks,

himanshu

 =

 =

=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D minutes from IETF 89 PWE3 WG meeting =3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D

 =

 =

Matthew: anything to say on VCCV for GAL? (VCCV type 4)<= /span>

Luca: where's Tom N? He was heading it.  3 documens 4447 bis = for last call

refresh adoption keep alive draft, but not sure if interest. = If not, willing to drop it.  VCCV Type 4, Tom lead author, we set out= a stable draft but probably expired.  Nothing current, willing to drop.

WG: No!

Luca: then we need someone to drive.

Himanshu: I can help.  We need the draft and the IANA assignm= ent.

Matthew: please sort out refreshing the draft.<= /p>

Himanshu: Yes. Is WG draft.

YJS: Proposal for this. 3 drafts around this. THere is far less in= terest but want to close type 4. We should make one draft that says here ar= e the types, deprecate 2 and add 4.

Luca: rfc4447bis is just an errata with no tech changes.  The= document you are talking about is there.  We didn't want to go down a= rat hole.  Don't want to open RFC 5085.  Just need to progress this, it is in good shape.

YJS: WG wanted to clean up all documents, but no time. So we will = just progress the current draft.

 

 =

_______________________________________________
pwe3 mailing list
pwe3@ietf.org=
https://www.ietf.org/mailman/listinfo/pwe3

 

 --_000_D01934B164C3Bcpignataciscocom_-- From nobody Wed Aug 20 08:32:32 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8ACB71A06D1 for ; Wed, 20 Aug 2014 08:32:29 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.701 X-Spam-Level: X-Spam-Status: No, score=0.701 tagged_above=-999 required=5 tests=[BAYES_50=0.8, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7o5m5wDy2GHj for ; Wed, 20 Aug 2014 08:32:25 -0700 (PDT) Received: from mail-qc0-x231.google.com (mail-qc0-x231.google.com [IPv6:2607:f8b0:400d:c01::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 01A201A06E4 for ; Wed, 20 Aug 2014 08:32:15 -0700 (PDT) Received: by mail-qc0-f177.google.com with SMTP id x13so7958596qcv.36 for ; Wed, 20 Aug 2014 08:32:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type; bh=ZgWjepWUU4O/410kyPHRmTAeKwhuqiuSRjng5mNT8cY=; b=BGRjpmo6bFOQ7Nco3/AymAYSE2q18kKiT04i8PK6+a318LwSx70/kWC80IS0EJn4f7 Pt3x8vmFFBqAsSI6enjK/N/XJ/6H1YIs+gZPSZSrElj7/5cAUZ4Ef7tn1ZjZEAEddxEA K9o98pc7WOI0KE27zw/VtCigYrNi6mgFrJrFZPy2ksGxNYu5+XLqHkxcD7JEbGSto42O VU36dDUd7Rhi101/2RNW1QbJy9kv2wERalIk1otvDZVXeLeOsxpklFliJm0jg5BVzXlb bYG0Aj31oOepeBGNzU4/xs8RjaOkGM3jT8kDbdmjnMUgnBRmQRrtI7dHBfW/qEhdrdVv 3/vA== X-Received: by 10.140.18.211 with SMTP id 77mr73841645qgf.57.1408548734233; Wed, 20 Aug 2014 08:32:14 -0700 (PDT) MIME-Version: 1.0 Received: by 10.140.91.99 with HTTP; Wed, 20 Aug 2014 08:31:54 -0700 (PDT) From: "Andrew G. Malis" Date: Wed, 20 Aug 2014 11:31:54 -0400 Message-ID: To: "pwe3@ietf.org" Content-Type: multipart/alternative; boundary=001a11353dd0f424e30501114e5e Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/EJ4F6Ecv6h_WzBADFjH5vg6aXxA Subject: [PWE3] Ending WG last call for draft-ietf-pwe3-congcons-02.txt X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Aug 2014 15:32:29 -0000 --001a11353dd0f424e30501114e5e Content-Type: text/plain; charset=UTF-8 It looks like the comments on draft-ietf-pwe3-congcons-02.txt have quieted down, so I'm ending WG last call. I'll put the draft into the "update requested" state, and will forward it to the IESG once the revision is available. Thanks, Andy --001a11353dd0f424e30501114e5e Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
It looks like the comments on=C2=A0draft-ietf-pwe3-congcon= s-02.txt have quieted down, so I'm ending WG last call. I'll put th= e draft into the "update requested" state, and will forward it to= the IESG once the revision is available.

Thanks,
Andy

--001a11353dd0f424e30501114e5e-- From nobody Wed Aug 20 11:36:03 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8B4B31A0668 for ; Wed, 20 Aug 2014 11:35:59 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.568 X-Spam-Level: X-Spam-Status: No, score=-3.568 tagged_above=-999 required=5 tests=[BAYES_50=0.8, GB_I_INVITATION=-2, GB_I_LETTER=-2, GB_SUMOF=1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-0.668, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wWazI1-eaSs1 for ; Wed, 20 Aug 2014 11:35:51 -0700 (PDT) Received: from hubrelay-rd.bt.com (hubrelay-rd.bt.com [62.239.224.98]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1E7DB1A04E9 for ; Wed, 20 Aug 2014 11:35:49 -0700 (PDT) Received: from EVMHR72-UKRD.domain1.systemhost.net (10.36.3.110) by EVMHR66-UKRD.bt.com (10.187.101.21) with Microsoft SMTP Server (TLS) id 14.3.195.1; Wed, 20 Aug 2014 19:35:41 +0100 Received: from EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) by EVMHR72-UKRD.domain1.systemhost.net (10.36.3.110) with Microsoft SMTP Server (TLS) id 8.3.348.2; Wed, 20 Aug 2014 19:35:45 +0100 Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) with Microsoft SMTP Server id 14.3.181.6; Wed, 20 Aug 2014 19:35:45 +0100 Received: from BTP075694.jungle.bt.co.uk ([10.215.130.93]) by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id s7KIZerA027879; Wed, 20 Aug 2014 19:35:41 +0100 Message-ID: <201408201835.s7KIZerA027879@bagheera.jungle.bt.co.uk> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Wed, 20 Aug 2014 19:35:35 +0100 To: "Black, David" From: Bob Briscoe In-Reply-To: <8D3D17ACE214DC429325B2B98F3AE712077BB42180@MX15A.corp.emc. com> References: <8D3D17ACE214DC429325B2B98F3AE712077B9517F4@MX15A.corp.emc.com> <8D3D17ACE214DC429325B2B98F3AE712077B951928@MX15A.corp.emc.com> <201408141302.s7ED2km0032238@bagheera.jungle.bt.co.uk> <8D3D17ACE214DC429325B2B98F3AE712077B952113@MX15A.corp.emc.com> <8D3D17ACE214DC429325B2B98F3AE712077B95212C@MX15A.corp.emc.com> <201408151006.s7FA6nWG003570@bagheera.jungle.bt.co.uk> <8D3D17ACE214DC429325B2B98F3AE712077BB42180@MX15A.corp.emc.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="=====================_1318234672==.ALT" X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158 Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/DQmkK0S_eAXL0n66_Zc5x6u_nY0 Cc: "pwe3@ietf.org" Subject: Re: [PWE3] FW: WGLC comments on: draft-ietf-pwe3-congcons-02.txt X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Aug 2014 18:36:00 -0000 --=====================_1318234672==.ALT Content-Type: text/plain; charset="iso-8859-1"; format=flowed Content-Transfer-Encoding: quoted-printable David, Sry for delay. inline... (Despite end of WGLC, we can still continue our=20 public intra-author ping-pong :) At 13:44 18/08/2014, Black, David wrote: >Bob, > >1-4 & 6: ok > >5: =93acceptable=94 is a good replacement for=20 >S-A-F-E, but I=92m not sure about =93friendly=94. The=20 >key concept here is assurance that there are no=20 >problems. How about =93clearly acceptable=94? That=20 >may need some additional text to say that=20 >acceptable network and service behavior may be=20 >delivered outside the clearly acceptable region,=20 >but one cannot rely on the analysis in this=20 >draft (based on the TCP throughput formula) to reach such a conclusion. I wanted a different word from 'acceptable' for=20 the adjective in the captions above the Figures=20 describing the regions, because there's no=20 opportunity for more text there to explain that=20 we do not mean that outside the region is=20 unacceptable. I thought about various qualifiers=20 with acceptable (like clearly, unarguably, etc),=20 but I don't think these are right either. I figured that the curves that delineate the=20 region are defined by the TCP-friendly equation,=20 so 'friendly region' says on the tin what it does=20 (and it is OK to imply that outside the region is=20 unfriendly, because unfriendly doesn't imply=20 "Don't go there" like 'unacceptable' would). Let's stop batting back & forth until we see what Yaakov (or others) think. >7: I disagree - the tsvwg circuit breaker draft=20 >is rather general, and we now have two WG LC=20 >comments on this draft asking how to apply a=20 >circuit breaker to PWs. I think we=92re on solid=20 >ground with the references to the ITU-T service=20 >quality standards and expecting operator=20 >monitoring of service quality. OTOH, the word=20 >=93persistently=94 seemed to be a concern, and I=20 >have no problem removing it (anyone who asks=20 >about that can be directed to G.775 and G.826). New suggested text: > > For TDM pseudowires, such a circuit breaker can be realized via > operator monitoring of drop rates and the quality of > TDM service delivered in order to shut down TDM pseudowires that > fail to deliver acceptable TDM service (as defined by [G775] and > [G826]). This document explains why a TDM PW that is causing= congestion > problems via interference with elastic traffic is also not delivering > acceptable TDM service (see the rest of this Section, plus Appendices > A and B). If we're going to say a bit about how to realize=20 a circuit-breaker, I'd be happier with the word=20 persistently than without. I'd still rather we=20 didn't allow this draft to mission creep into mechanism mode. In fact, on reflection, whatever we say about=20 circuit-breakers, I'd prefer not to mention=20 circuit-breakers in the abstract, which implies=20 this draft says stuff about mechanism, which was never the intent. > >8: I believe that figures 1 and 2 suffice to=20 >make the point, but I see that a recommendation=20 >may be a bit of a reach for this draft. How=20 >about the following modified form of my first=20 >sentence, which just states the inference that=20 >can be drawn from Figures 1 and 2? - > > Figures 1 and 2 indicate that a larger segment size for a T1 or E1 > TDM PW moves the operational behavior of the PW away from the boundary > at which the TCP throughput equation suggests that undue interference= with > elastic traffic may start to occur. Not so - that's the problem. On these graphs, the=20 op behaviour plots won't move at all if the PW=20 uses larger packets. These are bit-rate plots, so=20 the only way to move anything is to change the=20 TCP segment size (which moves the TCP plots). To=20 justify what you want to say, either: a) these would have to be packet-rate plots, or b) we would have to turn the assumption we've=20 made about equal packet sizes into an operational=20 requirement, not just an analytical assumption. To go in either of these directions would require a spade and vermin. Bob PS. GOing offline until next Tue. Might check email, but no guarantee. > >Thanks, >--David > >From: Bob Briscoe [mailto:bob.briscoe@bt.com] >Sent: Friday, August 15, 2014 6:03 AM >To: Black, David >Cc: pwe3@ietf.org >Subject: Re: [PWE3] FW: WGLC comments on: draft-ietf-pwe3-congcons-02.txt > >David, > >I decided to think on a better word for 'safe'=20 >overnight, and allow others to comment. But=20 >no-one has, so here's my responses to all your outstanding points... > >I've provided slightly updated xml source of my=20 >suggested edits (and a diff) here: ><=20 >http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-pwe3-= congcons-DIFF03b-02.html =20 > > ><=20 >http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-pwe3-= congcons-03b.txt =20 > > ><=20 >http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-pwe3-= congcons-03b.xml =20 > > >Again note: xml is purely for precision, not=20 >implying I have taken over editorship. > >1. Happy with your new scoping text, except the=20 >the first 'independent' clangs with the second, and it's unnecessary= anyway: >" The concern is over PWs that share network=20 >capacity with elastic or congestion-responsive=20 >traffic, [independent of] whether that sharing=20 >is a result of the service provider network's=20 >traffic engineering and capacity planning or=20 >independent deployment of PWs by entities other than network operators. >" > >2. My point wasn't to remove the reference to=20 >the 125us (in my suggested text I didn't). It=20 >was that the 125us stuff /alone/ doesn't warrant=20 >the 'thus' in the next clause; the logic needs=20 >to be connected by talking about the sum of the=20 >flight time /and/ the per-node processing time. >CURRENT: > The one-way delay > of a native TDM service consists of the physical time-of-flight plus > 125 microseconds for each TDM switch traversed; and is thus very > small as compared to typical PSN network-crossing latencies. >SUGGESTED: > The one-way delay > of a native TDM service consists of the physical time-of-flight plus > 125 microseconds for each TDM switch=20 > traversed; so the total one-way delay for most=20 > TDM PWs has to be small compared to typical PSN network-crossing= latencies. > >5. I understand and agree - 'S-A-F-E region' is=20 >incendiary. But I don't like 'reasonable region'=20 >- it's slightly /too/ meaningless. > >Other possibilities are: >- admissable region (problem: implies outside the region is inadmissable) >- harmonious region (problem: too Californian) >So I suggest two words: >- friendly or acceptable, each used depending on context. > >On the graphs, I suggest 'operating region=20 >(friendly under curve)' because that is=20 >precisely where the curves come from -=20 >'friendly' does what it says on the tin, and the=20 >implied 'unfriendly' above the curve correctly=20 >avoids implying that the sky will fall. > >Elsewhere, I suggest the following replacements for the S-word: >"It is thus worthwhile specifying under what=20 >conditions such competition is acceptable" >"One may view this condition as defining a=20 >'friendly' operating envelope for a TDM PW" >"Under this condition it is acceptable to place=20 >the TDM PW along with ... TCP. " >"comparison with something widely considered to=20 >offer an acceptable response to congestion" > > >7. Circuit breaker >Scope creep. Rat-hole. Stop digging. >Yes, I know Stewart has asked us to dig out=20 >another rat as well, but the appropriate response is "Out of scope". > >You might think your text is uncontroversial,=20 >but if we are tempted to take Stewart's bait=20 >(rat metaphor intended), it will probably start=20 >new debates that this doc doesn't deserve: "What=20 >does persistent mean?", "Which traffic is more=20 >important/ valuable?", yada yada. > >I suggest the text merely makes it clear that=20 >this debate is in scope for elsewhere, not here.=20 >I've also just realised that Yaakov has written=20 >the sentences around circuit breaker with the=20 >wrong logical sequence. The circuit breaker=20 >needs to be introduced as something the PW uses=20 >when its /own/ quality has failed, which should=20 >avoid any need to shut down because it is threatening elastic services: > >CURRENT: > If the bandwidth consumed by a TDM PW is > considered detrimental, the only available remedy is to completely > shut down the PW, by using a transport circuit breaker mechanism. > However, we will show that even before such an action is > warranted, the PW will become unable to faithfully emulate the > native TDM service; for example, when a TDM service is carrying > voice grade telephony channels, the voice quality will degrade to > below useful levels. >SUGGESTED: > We will show that, before a PW becomes=20 > deterimental to elastic traffic, it will become=20 > unable to faithfully emulate the > native TDM service; for example, when a TDM service is carrying > voice grade telephony channels, the voice quality will degrade to > below useful levels. Once the quality of=20 > the emulated service is persistently=20 > unacceptable, if a PW shuts down selected flows=20 > or shuts itself down completely, it will=20 > automatically avoid detriment to elastic=20 > traffic. A transport circuit breaker=20 > [I-D.fairhurst-tsvwg-circuit-breaker] could be=20 > used, but mechanism is beyond the scope of the present document. > > >8. Segment size. >Everyone's in a big group-hug because we've=20 >written a draft showing there's is no need to=20 >change anything. Then just at the end of an=20 >uneventful WGLC, we hear David's voice from the=20 >bottom of a deep hole shouting, "avoid the=20 >rat-hole invitation", while furiously digging for a rat. > >There is no need to add a recommendation that=20 >PWs need to be changed to use larger packets. No=20 >matter how hedged and soft this recommendation=20 >would be, it's not needed anyway. Stop digging. > >Yaakov's new text is fine as it stands. "Only=20 >for small packets, long delays, and high packet=20 >loss ratios, do TDM PWs potentially consume more=20 >bandwidth, and even then only marginally. " > >In order to justify your sentence, we would have=20 >to include graphs with packet-rate (not=20 >bit-rate) on the vertical axis and go into a=20 >whole treatise as to why packet-rate comparison=20 >might be important as well. Then the graph would=20 >show one plot for all TCP segment sizes and=20 >multiple plots for PWs with different packet=20 >sizes. Then we'd see that the corner of the two=20 >smallest packet PWs just clipped the TCP curve=20 >at loss probabilities that are stupidly large=20 >anyway. So even after all this, it would hardly=20 >justify a need for your sentence. > >All that digging would not be worth it, just to=20 >discover (again) that there is no rat. > > >Bob > > >At 19:39 14/08/2014, Black, David wrote: > >Content-Language: en-US >Content-Type: multipart/alternative; >=20 >boundary=3D"_000_8D3D17ACE214DC429325B2B98F3AE712077B95212CMX15Acorpemcc_" > >And forwarding my views to the list ... > >... On Bob=92s six thoughts (I have problems with=20 >2 & 5 - the other 4 are basically ok w/me) plus=20 >follow-ups on Bob=92s concerns w/my two major issues, here goes: > >-- 1 -- I agree in principle, but I think Bob=92s=20 >suggested new text is too black/white and should be generalized. > >Bob suggests: > > > The case where the service provider network=20 > arranges sufficient capacity for a PW is well understood and > > will not be discussed further here. The=20 > concern is over PWs deployed independently of the service provider > > network's traffic engineering or capacity planning. > >I want to include the case where capacity has=20 >been arranged, but may not be sufficient, I=20 >suggest rewriting Bob=92s second sentence as: > >The concern is over PWs that share network=20 >capacity with elastic or congestion-responsive=20 >traffic, independent of whether that sharing is=20 >a result of the service provider network's=20 >traffic engineering and capacity planning or=20 >independent deployment of PWs by entities other than network operators. > >-- 2 -- Here, I disagree. One of the=20 >significant contributions of this draft is that=20 >TDM PWs don=92t cause congestion problems in=20 >practice, and one of the major reasons for that=20 >is the low latency requirements for acceptable=20 >TDM service delivery, both in ITU-T standards=20 >and in actual TDM PW deployment. The 125us=20 >behavior of native TDM switches is relevant to=20 >the latter, and hence I prefer to include mention of it. > >-- 3 -- I agree; clear captions are always a feature, not a bug ;-). > >-- 4 -- I agree with changing =93packet loss rate=94 to =93packet loss= probability=94. > >-- 5 -- I firmly disagree with Bob here,=20 >although he=92s right that this is =93techno-political=94 ;-). > >I am concerned that S-A-F-E is likely to be a=20 >4-letter word in wider Transport Area review,=20 >and hence, I want to use a word that does not=20 >imply anything remotely resembling an absolute=20 >guarantee that nothing can possibly ever go=20 >wrong (yes, I know Bob didn=92t mean to go to=20 >quite that extreme an interpretation, but=20 >S-A-F-E is liable to be read that way by those=20 >with an =93axe to grind=94), even though there=92s=20 >lots of solid operational experience that suggests all is well here. > >-- 6 -- I think I=92m ok with what Bob proposes to=20 >better characterize loss burstiness, but want to see specific text. > >-- Circuit Breaker Mechanism -- > > >> [David] I suggest adding: > >> > >> For TDM pseudowires, such a circuit breaker can be realized via > >> operator monitoring of drop rates and consequent on the quality of > >> TDM service delivered and shutting down=20 > TDM pseudowires that persistently > >> fail to deliver acceptable TDM service (as defined by [G775] and > >> [G826]). This document explains why a=20 > TDM PW that is causing congestion > >> problems via interference with elastic traffic cannot be delivering > >> acceptable TDM service (see the rest of this Section, plus= Appendices > >> A and B). > >> > > [Bob] I don't think we should go there. We=20 > need to keep this draft firmly as informational. If someone > > thinks a mechanism is needed because of the=20 > information in this draft, then they can write about a mechanism. > >[David] I firmly disagree, but Bob and I might=20 >be talking past each other, as I do want to stay informational ;-). > >This text is intended to describe an example or=20 >possibility, not to specify an implementation=20 >requirement. The new text was intended as an=20 >OAM/OSS practice recommendation to the operator,=20 >not as a normative protocol spec. > >I=92m ok with inserting =93example=94, =93possibility=94=20 >or something similar if that helps w/Bob=92s=20 >concern, but I strongly oppose doing nothing=20 >here, and note that this is also needed to=20 >respond to one of Stewart Bryant=92s WG LC comments. > >-- PW segment size -- > > >> [David] Ignoring the rathole invitation=20 > ;-). This is still in section 3, after: > >> > >> We see that in general for large packet sizes, short delays, and low > >> packet loss rates, the TDM pseudowires consume much less bandwidth > >> than TCP would under identical conditions. Only for small packets, > >> long delays, and high packet loss ratios, do TDM PWs potentially > >> consume more bandwidth, and even then only marginally. > >> > >> I suggest adding: > >> > >> Comparing figures 1 and 2, use of a larger segment size for a T1 or= E1 > >> TDM PW moves the operational behavior of the PW away from the= boundary > >> at which the TCP throughput equation=20 > suggests that undue interference with > >> elastic traffic may start to occur. We suggest that larger segment > >> sizes be preferred for T1 and E1 TDM PWs, but note that other > >> considerations may favor smaller segment sizes (e.g., jitter buffer > >> size, bounding the impact of an isolated packet drop event). > >> > > [Bob] There is no need to recommend any=20 > remedial action, when we have just shown there is not a problem. > >[David] Not exactly - Figure 1 shows that there=20 >is a potential concern at a combination of the=20 >high end of the range of TDM PW latencies (4ms),=20 >and higher packet loss ratios. > > > Further, the "marginal problem" only applies=20 > when the TCP traffic uses packets smaller than 128B, which we only used > > as a hypothetical worst-case for the=20 > analysis. But in practice this doesn't happen with long-running TCP flows= - > > they use packets that are as large as=20 > possible (irrespective of what the PW uses). So=20 > there really is /no/ problem to remedy. >Ok, Bob can relax - his transport credentials=20 >are intact, as this is an entry point to the=20 >bit-rate-congestible vs. packet-rate-congestible=20 >resource rathole that I want to ignore :-). > >I believe that larger TDM PW segment sizes are a=20 >good thing for both this operating envelope=20 >boundary concern as well as per-packet=20 >processing overheads in TDM PW functionality,=20 >modulo other considerations, e.g., jitter=20 >buffers, and so I prefer to keep this addition. > >Thanks, >--David > >From: Bob Briscoe [mailto:bob.briscoe@bt.com] >Sent: Thursday, August 14, 2014 9:03 AM >To: Yaakov Stein >Cc: Andrew G. Malis; BOCCI Matthew; Black, David >Subject: RE: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt > >Yaakov, > >I have just re-read it thoroughly. I have=20 >noticed about two dozen editorial nits where=20 >correcting them would not change the intended=20 >sense (altho sometimes the sense needs to change=20 >to say what we intended!). Examples: >- Unexpanded abbreviations (David's suggestion would help, but there are= more) >- Missing words, tautologies, random punctuation problems etc >- Uses of the wrong word, even tho we know what you mean >- The caveat about dynamics vs steady state=20 >needs to shift down a para, so that it follows=20 >into the para that says what to do about dynamics. > >Should I take the token, and edit the XML=20 >directly, then you can see a diff to check if=20 >you want to accept my suggestions? > >Below are six thoughts that go slightly beyond=20 >'editorial'. 1-4 could be considered=20 >'editorial'. But 5&6 are the more significant "techno-political" points. > >1. The phrase "mechanisms for provisioning PWs=20 >in service provider networks" is too opaque (the=20 >PWE audience might understand this scoping=20 >sentence, but a L4 reader won't understand what=20 >we're excluding here, which will make them suspicious). >CURRENT: > Mechanisms for provisioning PWs > in service provider networks are well understood and will not be > discussed further here. >SUGGESTED: > The case where the service provider network=20 > arranges sufficient capacity for a PW is well=20 > understood and will not be discussed further=20 > here. The concern is over PWs deployed=20 > independently of the service provider network's=20 > traffic engineering or capacity planning. > >2. The justification for using OWD of a few milliseconds misses the point. >" The one-way delay > of a native TDM service consists of the physical time-of-flight plus > 125 microseconds for each TDM switch traversed; and is thus very > small as compared to typical PSN network-crossing latencies. >" >The 125us per switch is irrelevant. The point=20 >that needs making is that PWs are generally=20 >deployed with a short physical time of flight. > >3. Related point: the captions on the graphs do=20 >not clearly indicate that they are OWD, not RTT.=20 >This could be clarified in the text, or on the graphs. > >4. Change packet loss rate (and PLR) to packet=20 >loss probability throughout, including on the=20 >graph axes. PLR is an ambiguous term, even=20 >though it is widely used in the industry. Packet=20 >loss rate could be measured in packet per=20 >second, whereas packet loss probability is=20 >unambiguously dimensionless. Indeed, in the=20 >appendices, packet loss rates with respect to time are calculated. > >5. Rather than David's "reasonable" I suggest we=20 >use the phrase "unambiguously safe" not just=20 >"safe" for the headings of Figs 5-8 and in the=20 >text, as below. Otherwise, we are implying that=20 >it is unsafe outside the region. > >CURRENT: > One may view this condition as defining an operating envelope for a > TDM PW, as a TDM PW that occupies no more bandwidth than a TCP flow > causes no more congestion than that TCP flow would. Under this > condition it is safe to place the TDM PW along with congestion- > responsive traffic such as TCP, without causing additional > congestion. on the other hand, were the TDM PW to consume > significantly more bandwidth a TCP flow, it could contribute > disproportionately to congestion, and its mixture with congestion- > responsive traffic might be inappropriate. >SUGGESTED: > One may view this condition as defining an=20 > unambiguously safe operating envelope for a > TDM PW, as a TDM PW that occupies no more bandwidth than a TCP flow > causes no more congestion than that TCP flow=20 > would. This does not draw us into a debate=20 > over whether TCP-friendliness is a valid=20 > constraint; it simply says that there can be no=20 > question that a PW is safe if it does no more=20 > harm than a single TCP flow. Under this > condition it is unambiguously safe to place=20 > the TDM PW along with congestion- > responsive traffic such as TCP. On the=20 > other hand, were the TDM PW to consume > significantly more bandwidth a TCP flow, it could contribute > disproportionately to congestion, and its mixture with congestion- > responsive traffic might start to cause concern. > >6. The analysis in Appendices A & B needs to=20 >mention that loss bursts arise primarily from=20 >tail drop buffers, and the introduction of=20 >active queue management (AQM) makes losses more=20 >independent of each other (less bursty). (I have=20 >references showing evidence of this.) In the=20 >comparisons with TCP, we have assumed=20 >independent losses anyway, because they are the=20 >worst case (and fortunately they are easier to=20 >analyse). On a similar point, in appendix B,=20 >where we say "(bursty packet loss was also=20 >simulated but is not reported here)", we ought=20 >to give a reason otherwise readers will be=20 >suspicious. A good reason would be that uniformly random is the worst case. > >I've also responded to a couple of David's=20 >suggestions inline (I agree with the others)... > > >At 17:44 13/08/2014, Yaakov Stein wrote: >Bob told me on Friday of the IETF that he was=20 >going to look over the final version and comment. > >Bob =96 ping ? > >Y(J)S > >From: Andrew G. Malis [ mailto:agmalis@gmail.com] >Sent: 13 August, 2014 17:02 >To: Yaakov Stein >Cc: Bob Briscoe;=20 >rbriscoe@jungle.bt.co.uk;=20 >BOCCI Matthew; Black, David >Subject: Re: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt > >Yaakov, > >I still haven't heard from Bob, but you should=20 >incorporate David's updates into a new revision,=20 >and I'll use that version for my final Shepherd's review. > >Thanks, >Andy > >On Mon, Aug 11, 2014 at 11:16 AM, Andrew G.=20 >Malis <agmalis@gmail.com> wrote: >Actually, I'm going to end the LC as soon as I=20 >hear back from Bob on the IPR call. I sent him a reminder earlier this= morning. > >Cheers, >Andy > >On Sun, Aug 10, 2014 at 9:53 AM, Andrew G. Malis=20 ><agmalis@gmail.com> wrote: >David, > >That's perfect, thanks! I'm going to end the LC tomorrow. > >Cheers, >Andy > >On Sat, Aug 9, 2014 at 10:32 PM, Black, David=20 ><david.black@emc.com> wrote: >Following up on this ... > > > As we're now moving quickly to try to complete this draft , I agreed > > w/Yaakov that my author comments would be=20 > sent in as WG Last Call comments ... > > ... so here they are: >... with some suggested new text ... > > > Major issues: > > > > The reference to a transport circuit breaker=20 > in section 3 is a bit optimistic > > and high level: > > > > We can accomplish this by > > employing a transport circuit breaker, by which we mean an= automatic > > mechanism for terminating a flow to prevent negative impact on= other > > flows and on the stability of the network > > [I-D.fairhurst-tsvwg-circuit-breaker]. > > > > This should be accompanied by at least an example of how to do this -= past > > discussion has suggested carrier monitoring of delivered service quality > > in order to shut down PWs that are experiencing loss rates that result= in > > unacceptable service quality (as defined for us by the ITU ...). I'd= also > > like to add some text to suggest that operators use this sort of circuit > > breaker practice when TDM PWs compete with elastic traffic. >After this text in Section 3: > > We can accomplish this by > employing a transport circuit breaker, by which we mean an automatic > mechanism for terminating a flow to prevent negative impact on other > flows and on the stability of the network > [I-D.fairhurst-tsvwg-circuit-breaker]. Note that a transport circuit > breaker is intended as a protection mechanism of last resort, just as > an electrical circuit breaker is only triggered when absolutely > necessary. > >I suggest adding: > > For TDM pseudowires, such a circuit breaker can be realized via > operator monitoring of drop rates and consequent on the quality of > TDM service delivered and shutting down TDM pseudowires that= persistently > fail to deliver acceptable TDM service (as defined by [G775] and > [G826]). This document explains why a TDM PW that is causing= congestion > problems via interference with elastic traffic cannot be delivering > acceptable TDM service (see the rest of this Section, plus Appendices > A and B). > >I don't think we should go there. We need to=20 >keep this draft firmly as informational. If=20 >someone thinks a mechanism is needed because of=20 >the information in this draft, then they can write about a mechanism. > > >And breaking the paragraph after the added text. > > > Figure 1 and Figure 2 indicate that based on using the TCP throughput > > equation as a proxy for potential impact on elastic traffic (which is= what > > we're doing in this draft), for the same=20 > fixed bandwidth PW, a larger segment > > size is less likely to cause impacts. I'd like to add a suggestion to= use > > the larger PW segment sizes as long as they're consistent with delivery= of > > the service. There's been a related (long-to-the-point-of-ratholed)= debate > > over in the Transport Area about whether packet-rate-limited/congestible > > forwarding is still relevant. My view is that such equipment exists,= and > > hence that this is a relevant consideration for this draft, and would= not > > mention that issue here. >Ignoring the rathole invitation ;-). This is still in section 3, after: > > We see that in general for large packet sizes, short delays, and low > packet loss rates, the TDM pseudowires consume much less bandwidth > than TCP would under identical conditions. Only for small packets, > long delays, and high packet loss ratios, do TDM PWs potentially > consume more bandwidth, and even then only marginally. > >I suggest adding: > > Comparing figures 1 and 2, use of a larger segment size for a T1 or E1 > TDM PW moves the operational behavior of the PW away from the boundary > at which the TCP throughput equation suggests that undue interference= with > elastic traffic may start to occur. We suggest that larger segment > sizes be preferred for T1 and E1 TDM PWs, but note that other > considerations may favor smaller segment sizes (e.g., jitter buffer > size, bounding the impact of an isolated packet drop event). > >There is no need to recommend any remedial=20 >action, when we have just shown there is not a problem. > >Further, the "marginal problem" only applies=20 >when the TCP traffic uses packets smaller than=20 >128B, which we only used as a hypothetical=20 >worst-case for the analysis. But in practice=20 >this doesn't happen with long-running TCP flows=20 >- they use packets that are as large as possible=20 >(irrespective of what the PW uses). So there really is /no/ problem to= remedy. > > >and again, breaking the paragraph after that added text. > >I hope that latter sentence ("We suggest that ...") is acceptable to the= WG. > > > Minor issues: > > > > Section 2 is about Ethernet PWs that carry elastic traffic. Should= others > > be mentioned as also covered by this analysis - e.g., ATM and Frame= Relay > > PWs carrying elastic traffic? >After: > > In this section we consider Ethernet PWs that primarily carry > congestion-responsive traffic. We show that we automatically obtain > the desired congestion avoidance behavior, and that additional > mechanisms are not needed. > >I suggest adding: > > This discussion is applicable to any PW that carries congestion- > responsive traffic (e.g., ATM and Frame Relay PWs); Ethernet PWs are > used here for clarity, as they are a common example. > > > Section 2 does not cover Ethernet PWs=20 > carrying inelastic traffic - that would > > be unusual, but should it be explicitly noted as out of scope, or just= not > > discussed (as is the case for the current text)? >I suggest doing nothing here, and planning to deal with this concern >if it should arise later (e.g., IETF Last Call or IESG Evaluation). > > > Section 3 should say that the reason the draft focuses on TDM PWs is= that > > they're the inelastic PWs that are likely to=20 > be carried over IP in practice. > > My understanding is that other potentially inelastic PWs are MPLS-only > > in practice. >After: > > Inelastic PWs, such as TDM PWs ([RFC4553][RFC5086][RFC5087]), are > potentially more problematic than the elastic PWs of the previous > section. Being constant bit-rate (CBR), TDM PWs can not be made > responsive to congestion. On the other hand, being CBR, they also do > not attempt to capture additional bandwidth when neighboring TCP > flows back off. > >I suggest adding: > > We focus on TDM PWs here as they are the only inelastic PWs that are > likely to be IP-encapsulated in practice. Most other inelastic PWs > tend to only be used with MPLS. > >[I'd ask Yaakov and other PW experts to double-check the latter sentence, >please.] > > > A terminology section to roll up all the=20 > acronym expansions (e.g., TDM, CBR) > > in one place seems like a good idea. >These four acronyms would be good to put into a terminology section: > >CBR Constant Bit Rate >PSN Packet Switched Network >PW Pseudowire >TDM Time Division Multiplexing > > > > > This text on p.11 could be incendiary: > > > > Under this condition it is safe to place the TDM PW along with > > congestion - responsive traffic such as TCP, without causing > > additional congestion. > > > > First of all to reduce the flammability, "safe" -> "reasonable" and= there > > are a number of other occurrences of "safe"=20 > that deserve similar treatment ;- > > ). > > Beyond that, this assertion is dependent on=20 > the operating conditions staying > > within that reasonable operating envelope, which entails use of= something > > like a circuit breaker - I'd say that here, and reference the earlier > > circuit breaker discussion. >Global substitution: "safe" -> "reasonable", starting with the abstract. > >Then for this specific text on p.11: > >OLD > Under this condition it is safe to place the TDM PW along with > congestion - responsive traffic such as TCP, without causing > additional congestion. >NEW > Under this condition it is reasonable=20 > to place the TDM PW in competition > with congestion-responsive traffic such=20 > as TCP, as the result should not > produce unacceptable congestion=20 > effects. That assumes that the operating > conditions remain within the reasonable=20 > operating envelope - as discussed > above, a transport circuit breaker is a=20 > suggested means of managing a TDM > PW so that it is shut down if operating conditions remain= persistently > outside the reasonable operating envelope for the PW (e.g., due to a > high packet loss rate). > >No need to go there. See my suggestions earlier. > > > > Figures 7 and 8 could use some shading or lines to indicate the typical > > deployed operating regions for the pseudo-wires to visibly show that= they > > fall under the curves, particularly for larger segment sizes. I'd also > > repeat or refer to the suggestion to use larger > >I wanted to suggest this myself, but it is hard=20 >to depict fuzzy boundaries. If you can do this, pls do. > >Given the PW operating regions are bounded by v=20 >low loss probability, the x axis in Figs 7&8=20 >ought to be scaled much smaller as well. > >Related point: the y axis in Fig 1 ought to max at 10, not 50. > >Regards > > > >Bob > > > > > Nits: > > > > References aren't allowed in abstracts, so remove reference to > > [I-D.fairhurst-tsvwg-circuit-breaker] > > > > Introduction, first paragraph, delete "PWE3" towards end of paragraph - > > > > preferably the 4 byte PWE3 control word. > > > > In addition to its current general statements, the Security= Considerations > > section should point to the security considerations sections in a few > > particularly relevant RFCs - I'd suggest 3985, 4553, 5086 and 5087. >After: > > This document does not introduce any new congestion-specific > mechanisms and thus does not introduce any new security > considerations above those present for PWs in general. > >I suggest adding a new paragraph: > > For PW security considerations, see the security considerations= sections > of the cited RFCS; the security considerations in [RFC3985], [RFC4553], > [RFC5086] and [RFC5087] are particularly applicable to this document. > > > > > Thanks, > > --David > > > > > > > -----Original Message----- > > > From: pwe3 [=20 > mailto:pwe3-bounces@ietf.org]=20 > On Behalf Of Yaakov Stein > > > Sent: Thursday, July 24, 2014 4:52 PM > > > To: pwe3@ietf.org > > > Subject: Re: [PWE3] I-D Action: draft-ietf-pwe3-congcons-02.txt > > > > > > Please look at the PDF version only > > > (the txt version does not have the graphs). > > > > > > Y(J)S > > > > > > > > > > _______________________________________________ > > pwe3 mailing list > > pwe3@ietf.org > > https://www.ietf.org/mailman/listinfo/pwe3 > >_______________________________________________ >pwe3 mailing list >pwe3@ietf.org >https://www.ietf.org/mailman/listinfo/pwe3 > > > > >________________________________________________________________ >Bob Briscoe, BT >_______________________________________________ >pwe3 mailing list >pwe3@ietf.org >https://www.ietf.org/mailman/listinfo/pwe3 > >________________________________________________________________ >Bob Briscoe, BT ________________________________________________________________ Bob Briscoe, BT=20 --=====================_1318234672==.ALT Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable David,

Sry for delay. inline...

(Despite end of WGLC, we can still continue our public intra-author ping-pong :)

At 13:44 18/08/2014, Black, David wrote:
Bob,
 
1-4 & 6: ok
 
5: =93acceptable=94 is a good replacement for S-A-F-E, but I=92m not sure ab= out =93friendly=94.  The key concept here is assurance that there are no problems.  How about =93clearly acceptable=94?  That may need some additional text to say that acceptable network and service behavior may be delivered outside the clearly acceptable region, but one cannot rely on the analysis in this draft (based on the TCP throughput formula) to reach such a conclusion.

I wanted a different word from 'acceptable' for the adjective in the captions above the Figures describing the regions, because there's no opportunity for more text there to explain that we do not mean that outside the region is unacceptable. I thought about various qualifiers with acceptable (like clearly, unarguably, etc), but I don't think these are right either.

I figured that the curves that delineate the region are defined by the TCP-friendly equation, so 'friendly region' says on the tin what it does (and it is OK to imply that outside the region is unfriendly, because unfriendly doesn't imply "Don't go there" like 'unacceptable' would).

Let's stop batting back & forth until we see what Yaakov (or others) think.


7: I disagree - the tsvwg circuit breaker draft is rather general, and we now have two WG LC comments on this draft asking how to apply a circuit breaker to PWs.  I think we=92re on solid ground with the references to the ITU-T service quality standards and expecting operator monitoring of service quality.  OTOH, the word =93persistently=94 seemed to be a concern, and I have no problem removing it (anyone who asks about that can be directed to G.775 and G.826).  New suggested text:
 
   For TDM pseudowires, such a circuit breaker can be realized via
   operator monitoring of drop rates and the quality of
   TDM service delivered in order to shut down TDM pseudowires that
   fail to deliver acceptable TDM service (as defined by [G775] and
   [G826]).  This document explains why a TDM PW that is causing congestion
   problems via interference with elastic traffic is also not delivering
   acceptable TDM service (see the rest of this Section, plus Appendices
   A and B).

If we're going to say a bit about how to realize a circuit-breaker, I'd be happier with the word persistently than without. I'd still rather we didn't allow this draft to mission creep into mechanism mode.

In fact, on reflection, whatever we say about circuit-breakers, I'd prefer not to mention circuit-breakers in the abstract, which implies this draft says stuff about mechanism, which was never the intent.

 
8: I believe that figures 1 and 2 suffice to make the point, but I see that a recommendation may be a bit of a reach for this draft.  How about the following modified form of my first sentence, which just states the inference that can be drawn from Figures 1 and 2? -
  
   Figures 1 and 2 indicate that a larger segment size for a T1 or E1
   TDM PW moves the operational behavior of the PW away from the boundary
   at which the TCP throughput equation suggests that undue interference with
   elastic traffic may start to occur.

Not so - that's the problem. On these graphs, the op behaviour plots won't move at all if the PW uses larger packets. These are bit-rate plots, so the only way to move anything is to change the TCP segment size (which moves the TCP plots). To justify what you want to say, either:
a) these would have to be packet-rate plots, or
b) we would have to turn the assumption we've made about equal packet sizes into an operational requirement, not just an analytical assumption.

To go in either of these directions would require a spade and vermin.


Bob

PS. GOing offline until next Tue. Might check email, but no guarantee.

 
Thanks,
--David
 
From: Bob Briscoe [ mailto:bob.briscoe@bt.com]
Sent: Friday, August 15, 2014 6:03 AM
To: Black, David
Cc: pwe3@ietf.org
Subject: Re: [PWE3] FW: WGLC comments on: draft-ietf-pwe3-congcons-02.txt
 
David,

I decided to think on a better word for 'safe' overnight, and allow others to comment. But no-one has, so here's my responses to all your outstanding points...

I've provided slightly updated xml source of my suggested edits (and a diff) here:
< http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-pwe3-c= ongcons-DIFF03b-02.html >
< http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-pwe3-c= ongcons-03b.txt >
< http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-pwe3-c= ongcons-03b.xml >
Again note: xml is purely for precision, not implying I have taken over editorship.

1. Happy with your new scoping text, except the the first 'independent' clangs with the second, and it's unnecessary anyway:
"  The concern is over PWs that share network capacity with elastic or congestion-responsive traffic, [independent of] whether that sharing is a result of the service provider network's traffic engineering and capacity planning or independent deployment of PWs by entities other than network operators.
"

2. My point wasn't to remove the reference to the 125us (in my suggested text I didn't). It was that the 125us stuff /alone/ doesn't warrant the 'thus' in the next clause; the logic needs to be connected by talking about the sum of the flight time /and/ the per-node processing time.
CURRENT:
   The one-way delay
   of a native TDM service consists of the physical time-of-flight plus
   125 microseconds for each TDM switch traversed; and is thus very
   small as compared to typical PSN network-crossing latencies.
SUGGESTED:
   The one-way delay
   of a native TDM service consists of the physical time-of-flight plus
   125 microseconds for each TDM switch traversed; so the total one-way delay for most TDM PWs has to be small compared to typical PSN network-crossing latencies.

5. I understand and agree - 'S-A-F-E region' is incendiary. But I don't like 'reasonable region' - it's slightly /too/ meaningless.

Other possibilities are:
- admissable region (problem: implies outside the region is inadmissable)
- harmonious region (problem: too Californian)
So I suggest two words:
- friendly or acceptable, each used depending on context.

On the graphs, I suggest 'operating region (friendly under curve)' because that is precisely where the curves come from - 'friendly' does what it says on the tin, and the implied 'unfriendly' above the curve correctly avoids implying that the sky will fall.

Elsewhere, I suggest the following replacements for the S-word:
"It is thus worthwhile specifying under what conditions such competition is acceptable"
"One may view this condition as defining a 'friendly' operating envelope for a TDM PW"
"Under this condition it is acceptable to place the TDM PW along with ... TCP. "
"comparison with something widely considered to offer an acceptable response to congestion"


7. Circuit breaker
Scope creep. Rat-hole. Stop digging.
Yes, I know Stewart has asked us to dig out another rat as well, but the appropriate response is "Out of scope".

You might think your text is uncontroversial, but if we are tempted to take Stewart's bait (rat metaphor intended), it will probably start new debates that this doc doesn't deserve: "What does persistent mean?", "Which traffic is more important/ valuable?", yada yada.

I suggest the text merely makes it clear that this debate is in scope for elsewhere, not here. I've also just realised that Yaakov has written the sentences around circuit breaker with the wrong logical sequence. The circuit breaker needs to be introduced as something the PW uses when its /own/ quality has failed, which should avoid any need to shut down because it is threatening elastic services:

CURRENT:
      If the bandwidth consumed by a TDM PW is
      considered detrimental, the only available remedy is to completely
      shut down the PW, by using a transport circuit breaker mechanism.
      However, we will show that even before such an action is
      warranted, the PW will become unable to faithfully emulate the
      native TDM service; for example, when a TDM service is carrying
      voice grade telephony channels, the voice quality will degrade to
      below useful levels.
SUGGESTED:
      We will show that, before a PW becomes deterimental to elastic traffic, it will become unable to faithfully emulate the
      native TDM service; for example, when a TDM service is carrying
      voice grade telephony channels, the voice quality will degrade to
      below useful levels. Once the quality of the emulated service is persistently unacceptable, if a PW shuts down selected flows or shuts itself down completely, it will automatically avoid detriment to elastic traffic. A transport circuit breaker [I-D.fairhurst-tsvwg-circuit-breaker] could be used, but mechanism is beyond the scope of the present document.


8. Segment size.
Everyone's in a big group-hug because we've written a draft showing there's is no need to change anything. Then just at the end of an uneventful WGLC, we hear David's voice from the bottom of a deep hole shouting, "avoid the rat-hole invitation", while furiously digging for a rat.

There is no need to add a recommendation that PWs need to be changed to use larger packets. No matter how hedged and soft this recommendation would be, it's not needed anyway. Stop digging.

Yaakov's new text is fine as it stands. "Only for small packets, long delays, and high packet loss ratios, do TDM PWs potentially consume more bandwidth, and even then only marginally. "

In order to justify your sentence, we would have to include graphs with packet-rate (not bit-rate) on the vertical axis and go into a whole treatise as to why packet-rate comparison might be important as well. Then the graph would show one plot for all TCP segment sizes and multiple plots for PWs with different packet sizes. Then we'd see that the corner of the two smallest packet PWs just clipped the TCP curve at loss probabilities that are stupidly large anyway. So even after all this, it would hardly justify a need for your sentence.

All that digging would not be worth it, just to discover (again) that there is no rat.


Bob


At 19:39 14/08/2014, Black, David wrote:

Content-Language: en-US
Content-Type: multipart/alternative;
         boundary=3D"_000_8D3D17ACE214DC429325B2B98F3AE712077B95212CMX15Acorpemc= c_"

And forwarding my views to the list ...
 
... On Bob=92s six thoughts (I have problems with 2 & 5 - the other 4 are basically ok w/me) plus follow-ups on Bob=92s concerns w/my two major issues, here goes:
 
-- 1 -- I agree in principle, but I think Bob=92s suggested new text is too black/white and should be generalized.
 
Bob suggests:
 
> The case where the service provider network arranges sufficient capacity for a PW is well understood and
> will not be discussed further here. The concern is over PWs deployed independently of the service provider
> network's traffic engineering or capacity planning.
 
I want to include the case where capacity has been arranged, but may not be sufficient, I suggest rewriting Bob=92s second sentence as:
 
The concern is over PWs that share network capacity with elastic or congestion-responsive traffic, independent of whether that sharing is a result of the service provider network's traffic engineering and capacity planning or independent deployment of PWs by entities other than network operators.
 
-- 2 -- Here, I disagree.  One of the significant contributions of this draft is that TDM PWs don=92t cause congestion problems in practice, and one of the major reasons for that is the low latency requirements for acceptable TDM service delivery, both in ITU-T standards and in actual TDM PW deployment.  The 125us behavior of native TDM switches is relevant to the latter, and hence I prefer to include mention of it.
 
-- 3 -- I agree; clear captions are always a feature, not a bug ;-).
 
-- 4 -- I agree with changing =93packet loss rate=94 to =93packet loss probability=94.
 
-- 5 -- I firmly disagree with Bob here, although he=92s right that this is =93techno-political=94 ;-).
 
I am concerned that S-A-F-E is likely to be a 4-letter word in wider Transport Area review, and hence, I want to use a word that does not imply anything remotely resembling an absolute guarantee that nothing can possibly ever go wrong (yes, I know Bob didn=92t mean to go to quite that extreme an interpretation, but S-A-F-E is liable to be read that way by those with an =93axe to grind=94), even though there=92s lots of solid operational experience that suggests all is well here.
 
-- 6 -- I think I=92m ok with what Bob proposes to better characterize loss burstiness, but want to see specific text.
 
-- Circuit Breaker Mechanism --
 
>> [David] I suggest adding:
>>
>>    For TDM pseudowires, such a circuit breaker can be realized via
>>    operator monitoring of drop rates and consequent on the quality of
>>    TDM service delivered and shutting down TDM pseudowires that persistently
>>    fail to deliver acceptable TDM service (as defined by [G775] and
>>    [G826]).  This document explains why a TDM PW that is causing congestion
>>    problems via interference with elastic traffic cannot be delivering
>>    acceptable TDM service (see the rest of this Section, plus Appendices
>>    A and B).
>>
> [Bob] I don't think we should go there. We need to keep this draft firmly as informational. If someone
> thinks a mechanism is needed because of the information in this draft, then they can write about a mechanism.
 
[David] I firmly disagree, but Bob and I might be talking past each other, as I do want to stay informational ;-).
 
This text is intended to describe an example or possibility, not to specify an implementation requirement.  The new text was intended as an OAM/OSS practice recommendation to the operator, not as a normative protocol spec.
 
I=92m ok with inserting =93example=94, =93possibility=94 or something= similar if that helps w/Bob=92s concern, but I strongly oppose doing nothing here, and note that this is also needed to respond to one of Stewart Bryant=92s WG LC comments.
 
-- PW segment size --
 
>> [David] Ignoring the rathole invitation ;-).  This is still in section 3, after:
>>
>>   We see that in general for large packet sizes, short delays, and low
>>   packet loss rates, the TDM pseudowires consume much less bandwidth
>>    than TCP would under identical conditions.  Only for small packets,
>>    long delays, and high packet loss ratios, do TDM PWs potentially
>>   consume more bandwidth, and even then only marginally.
>>
>> I suggest adding:
>>
>>    Comparing figures 1 and 2, use of a larger segment size for a T1 or E1
>>    TDM PW moves the operational behavior of the PW away from the boundary
>>    at which the TCP throughput equation suggests that undue interference with
>>    elastic traffic may start to occur.  We suggest that larger segment
>>    sizes be preferred for T1 and E1 TDM PWs, but note that other
>>    considerations may favor smaller segment sizes (e.g., jitter buffer
>>    size, bounding the impact of an isolated packet drop event).
>>
> [Bob] There is no need to recommend any remedial action, when we have just shown there is not a problem.
 
[David] Not exactly - Figure 1 shows that there is a potential concern at a combination of the high end of the range of TDM PW latencies (4ms), and higher packet loss ratios. 

> Further, the "marginal problem" only applies when the TCP traffic uses packets smaller than 128B, which we only used
> as a hypothetical worst-case for the analysis. But in practice this doesn't happen with long-running TCP flows -
> they use packets that are as large as possible (irrespective of what the PW uses). So there really is /no/ problem to remedy.
Ok, Bob can relax - his transport credentials are intact, as this is an entry point to the bit-rate-congestible vs. packet-rate-congestible resource rathole that I want to ignore :-).
 
I believe that larger TDM PW segment sizes are a good thing for both this operating envelope boundary concern as well as per-packet processing overheads in TDM PW functionality, modulo other considerations, e.g., jitter buffers, and so I prefer to keep this addition.
 
Thanks,
--David
 
From: Bob Briscoe [mailto:bob.briscoe@bt.com]
Sent: Thursday, August 14, 2014 9:03 AM
To: Yaakov Stein
Cc: Andrew G. Malis; BOCCI Matthew; Black, David
Subject: RE: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt
 
Yaakov,

I have just re-read it thoroughly. I have noticed about two dozen editorial nits where correcting them would not change the intended sense (altho sometimes the sense needs to change to say what we intended!). Examples:
- Unexpanded abbreviations (David's suggestion would help, but there are more)
- Missing words, tautologies, random punctuation problems etc
- Uses of the wrong word, even tho we know what you mean
- The caveat about dynamics vs steady state needs to shift down a para, so that it follows into the para that says what to do about dynamics.

Should I take the token, and edit the XML directly, then you can see a diff to check if you want to accept my suggestions?

Below are six thoughts that go slightly beyond 'editorial'. 1-4 could be considered 'editorial'. But 5&6 are the more significant "techno-political" points.

1. The phrase "mechanisms for provisioning PWs in service provider networks" is too opaque (the PWE audience might understand this scoping sentence, but a L4 reader won't understand what we're excluding here, which will make them suspicious).
CURRENT:
   Mechanisms for provisioning PWs
   in service provider networks are well understood and will not be
   discussed further here.
SUGGESTED:
   The case where the service provider network arranges sufficient capacity for a PW is well understood and will not be discussed further here. The concern is over PWs deployed independently of the service provider network's traffic engineering or capacity planning.

2. The justification for using OWD of a few milliseconds misses the point.
"  The one-way delay
   of a native TDM service consists of the physical time-of-flight plus
   125 microseconds for each TDM switch traversed; and is thus very
   small as compared to typical PSN network-crossing latencies.
"
The 125us per switch is irrelevant. The point that needs making is that PWs are generally deployed with a short physical time of flight.

3. Related point: the captions on the graphs do not clearly indicate that they are OWD, not RTT. This could be clarified in the text, or on the graphs.

4. Change packet loss rate (and PLR) to packet loss probability throughout, including on the graph axes. PLR is an ambiguous term, even though it is widely used in the industry. Packet loss rate could be measured in packet per second, whereas packet loss probability is unambiguously dimensionless. Indeed, in the appendices, packet loss rates with respect to time are calculated.

5. Rather than David's "reasonable" I suggest we use the phrase "unambiguously safe" not just "safe" for the headings of Figs 5-8 and in the text, as below. Otherwise, we are implying that it is unsafe outside the region.

CURRENT:
   One may view this condition as defining an operating envelope for a
   TDM PW, as a TDM PW that occupies no more bandwidth than a TCP flow
   causes no more congestion than that TCP flow would.  Under this
   condition it is safe to place the TDM PW along with congestion-
   responsive traffic such as TCP, without causing additional
   congestion.  on the other hand, were the TDM PW to consume
   significantly more bandwidth a TCP flow, it could contribute
   disproportionately to congestion, and its mixture with congestion-
   responsive traffic might be inappropriate.
SUGGESTED:
   One may view this condition as defining an unambiguously safe operating envelope for a
   TDM PW, as a TDM PW that occupies no more bandwidth than a TCP flow
   causes no more congestion than that TCP flow would.  This does not draw us into a debate over whether TCP-friendliness is a valid constraint; it simply says that there can be no question that a PW is safe if it does no more harm than a single TCP flow. Under this
   condition it is unambiguously safe to place the TDM PW along with congestion-
   responsive traffic such as TCP.  On the other hand, were the TDM PW to consume
   significantly more bandwidth a TCP flow, it could contribute
   disproportionately to congestion, and its mixture with congestion-
   responsive traffic might start to cause concern.

6. The analysis in Appendices A & B needs to mention that loss bursts arise primarily from tail drop buffers, and the introduction of active queue management (AQM) makes losses more independent of each other (less bursty). (I have references showing evidence of this.) In the comparisons with TCP, we have assumed independent losses anyway, because they are the worst case (and fortunately they are easier to analyse). On a similar point, in appendix B, where we say "(bursty packet loss was also simulated but is not reported here)", we ought to give a reason otherwise readers will be suspicious. A good reason would be that uniformly random is the worst case.

I've also responded to a couple of David's suggestions inline (I agree with the others)...


At 17:44 13/08/2014, Yaakov Stein wrote:
Bob told me on Friday of the IETF that he was going to look over the final version and comment.
 
Bob =96 ping ?
 
Y(J)S
 
From: Andrew G. Malis [ mailto:agmalis@gmail.com]
Sent: 13 August, 2014 17:02
To: Yaakov Stein
Cc: Bob Briscoe; rbriscoe@jungle.bt.co.uk; BOCCI Matthew; Black, David
Subject: Re: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt
 
Yaakov,
 
I still haven't heard from Bob, but you should incorporate David's updates into a new revision, and I'll use that version for my final Shepherd's review.
 
Thanks,
Andy
 
On Mon, Aug 11, 2014 at 11:16 AM, Andrew G. Malis <agmalis@gmail.com> wrote:
Actually, I'm going to end the LC as soon as I hear back from Bob on the IPR call. I sent him a reminder earlier this morning.
 
Cheers,
Andy
 
On Sun, Aug 10, 2014 at 9:53 AM, Andrew G. Malis <agmalis@gmail.com> wrote:
David,
 
That's perfect, thanks! I'm going to end the LC tomorrow.
 
Cheers,
Andy
 
On Sat, Aug 9, 2014 at 10:32 PM, Black, David <david.black@emc.com> wrote:
Following up on this ...

> As we're now moving quickly to try to complete this draft , I agreed
> w/Yaakov that my author comments would be sent in as WG Last Call comments ...
> ... so here they are:
... with some suggested new text ...

> Major issues:
>
> The reference to a transport circuit breaker in section 3 is a bit optimistic
> and high level:
>
>       We can accomplish this by
>       employing a transport circuit breaker, by which we mean an automatic
>       mechanism for terminating a flow to prevent negative impact on other
>       flows and on the stability of the network
>       [I-D.fairhurst-tsvwg-circuit-breaker].
>
> This should be accompanied by at least an example of how to do this - past
> discussion has suggested carrier monitoring of delivered service quality
> in order to shut down PWs that are experiencing loss rates that result in
> unacceptable service quality (as defined for us by the ITU ...).  I'd also
> like to add some text to suggest that operators use this sort of circuit
> breaker practice when TDM PWs compete with elastic traffic.
After this text in Section 3:

   We can accomplish this by
   employing a transport circuit breaker, by which we mean an automatic
   mechanism for terminating a flow to prevent negative impact on other
   flows and on the stability of the network
   [I-D.fairhurst-tsvwg-circuit-breaker].  Note that a transport circuit
   breaker is intended as a protection mechanism of last resort, just as
   an electrical circuit breaker is only triggered when absolutely
   necessary.

I suggest adding:

   For TDM pseudowires, such a circuit breaker can be realized via
   operator monitoring of drop rates and consequent on the quality of
   TDM service delivered and shutting down TDM pseudowires that persistently
   fail to deliver acceptable TDM service (as defined by [G775] and
   [G826]).  This document explains why a TDM PW that is causing congestion
   problems via interference with elastic traffic cannot be delivering
   acceptable TDM service (see the rest of this Section, plus Appendices
   A and B).

I don't think we should go there. We need to keep this draft firmly as informational. If someone thinks a mechanism is needed because of the information in this draft, then they can write about a mechanism.


And breaking the paragraph after the added text.

> Figure 1 and Figure 2 indicate that based on using the TCP throughput
> equation as a proxy for potential impact on elastic traffic (which is what
> we're doing in this draft), for the same fixed bandwidth PW, a larger segment
> size is less likely to cause impacts.  I'd like to add a suggestion to use
> the larger PW segment sizes as long as they're consistent with delivery of
> the service.  There's been a related (long-to-the-point-of-ratholed) debate
> over in the Transport Area about whether packet-rate-limited/congestible
> forwarding is still relevant.  My view is that such equipment exists, and
> hence that this is a relevant consideration for this draft, and would not
> mention that issue here.
Ignoring the rathole invitation ;-).  This is still in section 3, after:

   We see that in general for large packet sizes, short delays, and low
   packet loss rates, the TDM pseudowires consume much less bandwidth
   than TCP would under identical conditions.  Only for small packets,
   long delays, and high packet loss ratios, do TDM PWs potentially
   consume more bandwidth, and even then only marginally.

I suggest adding:

   Comparing figures 1 and 2, use of a larger segment size for a T1 or E1
   TDM PW moves the operational behavior of the PW away from the boundary
   at which the TCP throughput equation suggests that undue interference with
   elastic traffic may start to occur.  We suggest that larger segment
   sizes be preferred for T1 and E1 TDM PWs, but note that other
   considerations may favor smaller segment sizes (e.g., jitter buffer
   size, bounding the impact of an isolated packet drop event).

There is no need to recommend any remedial action, when we have just shown there is not a problem.

Further, the "marginal problem" only applies when the TCP traffic uses packets smaller than 128B, which we only used as a hypothetical worst-case for the analysis. But in practice this doesn't happen with long-running TCP flows - they use packets that are as large as possible (irrespective of what the PW uses). So there really is /no/ problem to remedy.


and again, breaking the paragraph after that added text.

I hope that latter sentence ("We suggest that ...") is acceptable to the WG.

> Minor issues:
>
> Section 2 is about Ethernet PWs that carry elastic traffic.  Should others
> be mentioned as also covered by this analysis - e.g., ATM and Frame Relay
> PWs carrying elastic traffic?
After:

   In this section we consider Ethernet PWs that primarily carry
   congestion-responsive traffic.  We show that we automatically obtain
   the desired congestion avoidance behavior, and that additional
   mechanisms are not needed.

I suggest adding:

   This discussion is applicable to any PW that carries congestion-
   responsive traffic (e.g., ATM and Frame Relay PWs); Ethernet PWs are
   used here for clarity, as they are a common example.

> Section 2 does not cover Ethernet PWs carrying inelastic traffic - that would
> be unusual, but should it be explicitly noted as out of scope, or just not
> discussed (as is the case for the current text)?
I suggest doing nothing here, and planning to deal with this concern
if it should arise later (e.g., IETF Last Call or IESG Evaluation).

> Section 3 should say that the reason the draft focuses on TDM PWs is that
> they're the inelastic PWs that are likely to be carried over IP in practice.
> My understanding is that other potentially inelastic PWs are MPLS-only
> in practice.
After:

   Inelastic PWs, such as TDM PWs ([RFC4553][RFC5086][RFC5087]), are
   potentially more problematic than the elastic PWs of the previous
   section.  Being constant bit-rate (CBR), TDM PWs can not be made
   responsive to congestion.  On the other hand, being CBR, they also do
   not attempt to capture additional bandwidth when neighboring TCP
   flows back off.

I suggest adding:

   We focus on TDM PWs here as they are the only inelastic PWs that are
   likely to be IP-encapsulated in practice.  Most other inelastic PWs
   tend to only be used with MPLS.

[I'd ask Yaakov and other PW experts to double-check the latter sentence,
please.]

> A terminology section to roll up all the acronym expansions (e.g., TDM, CBR)
> in one place seems like a good idea.
These four acronyms would be good to put into a terminology section:

CBR Constant Bit Rate
PSN Packet Switched Network
PW Pseudowire
TDM Time Division Multiplexing

>
> This text on p.11 could be incendiary:
>
>       Under this condition it is safe to place the TDM PW along with
>       congestion - responsive traffic such as TCP, without causing
>       additional congestion.
>
> First of all to reduce the flammability, "safe" -> "reasonable" and there
> are a number of other occurrences of "safe" that deserve similar treatment ;-
> ).
> Beyond that, this assertion is dependent on the operating conditions staying
> within that reasonable operating envelope, which entails use of something
> like a circuit breaker - I'd say that here, and reference the earlier
> circuit breaker discussion.
Global substitution: "safe" -> "reasonable", starting with the abstract.

Then for this specific text on p.11:

OLD
        Under this condition it is safe to place the TDM PW along with
        congestion - responsive traffic such as TCP, without causing
        additional congestion.
NEW
        Under this condition it is reasonable to place the TDM PW in competition
      with congestion-responsive traffic such as TCP, as the result should not
      produce unacceptable congestion effects.  That assumes that the operating
      conditions remain within the reasonable operating envelope - as discussed
      above, a transport circuit breaker is a suggested means of managing a TDM
      PW so that it is shut down if operating conditions remain persistently
      outside the reasonable operating envelope for the PW (e.g., due to a
      high packet loss rate).

No need to go there. See my suggestions earlier.


> Figures 7 and 8 could use some shading or lines to indicate the typical
> deployed operating regions for the pseudo-wires to visibly show that they
> fall under the curves, particularly for larger segment sizes.  I'd also
> repeat or refer to the suggestion to use larger

I wanted to suggest this myself, but it is hard to depict fuzzy boundaries. If you can do this, pls do.

Given the PW operating regions are bounded by v low loss probability, the x axis in Figs 7&8 ought to be scaled much smaller as well.

Related point: the y axis in Fig 1 ought to max at 10, not 50.

Regards



Bob

>
> Nits:
>
> References aren't allowed in abstracts, so remove reference to
> [I-D.fairhurst-tsvwg-circuit-breaker]
>
> Introduction, first paragraph, delete "PWE3" towards end of paragraph -
>
>    preferably the 4 byte PWE3 control word.
>
> In addition to its current general statements, the Security Considerations
> section should point to the security considerations sections in a few
> particularly relevant RFCs - I'd suggest 3985, 4553, 5086 and 5087.
After:

   This document does not introduce any new congestion-specific
   mechanisms and thus does not introduce any new security
   considerations above those present for PWs in general.

I suggest adding a new paragraph:

   For PW security considerations, see the security considerations sections
   of the cited RFCS; the security considerations in [RFC3985], [RFC4553],
   [RFC5086] and [RFC5087] are particularly applicable to this document.

>
> Thanks,
> --David
>
>
> > -----Original Message-----
> > From: pwe3 [ mailto:pwe3-bounces@ietf.org] On Behalf Of Yaakov Stein
> > Sent: Thursday, July 24, 2014 4:52 PM
> > To: pwe3@ietf.org
> > Subject: Re: [PWE3] I-D Action: draft-ietf-pwe3-congcons-02.txt
> >
> > Please look at the PDF version only
> > (the txt version does not have the graphs).
> >
> > Y(J)S
> >
> >
>
> _______________________________________________
> pwe3 mailing list
> pwe3@ietf.org
> https://www.ietf.org/mailman/listinfo/pwe3

_______________________________________________
pwe3 mailing list
pwe3@ietf.org
https://www.ietf.org/mailman/listinfo/pwe3
 
 
 

________________________________________________________________
Bob Briscoe,           &n= bsp;            =             &nbs= p;             BT
_______________________________________________
pwe3 mailing list
pwe3@ietf.org
https://www.ietf.org/mailman/listinfo/pwe3

________________________________________________________________
Bob Briscoe,           &n= bsp;            =             &nbs= p;             BT

________________________________________________________________
Bob Briscoe,           &n= bsp;            =             &nbs= p;             BT --=====================_1318234672==.ALT-- From nobody Wed Aug 20 11:59:40 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0CBA71A06D6 for ; Wed, 20 Aug 2014 11:59:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -7.968 X-Spam-Level: X-Spam-Status: No, score=-7.968 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, GB_I_INVITATION=-2, GB_I_LETTER=-2, GB_SUMOF=1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.668, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id q_FiRamz4KsL for ; Wed, 20 Aug 2014 11:59:21 -0700 (PDT) Received: from mailuogwdur.emc.com (mailuogwdur.emc.com [128.221.224.79]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CAAE11A065F for ; Wed, 20 Aug 2014 11:59:20 -0700 (PDT) Received: from maildlpprd56.lss.emc.com (maildlpprd56.lss.emc.com [10.106.48.160]) by mailuogwprd54.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s7KIxBMf020794 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 20 Aug 2014 14:59:13 -0400 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd54.lss.emc.com s7KIxBMf020794 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=emc.com; s=jan2013; t=1408561153; bh=saqBs2MylNyDhci5Q1koFOU+u7I=; h=From:To:CC:Date:Subject:Message-ID:References:In-Reply-To: Content-Type:MIME-Version; b=Jh0ZR8XdIaqYWCKBkAa0/bCuC24SuZNNxZpQnOIg5DyK526bsVRI3FoQBQw2Jc/KR gMRkHCqGV7WARLA5WJ96QttDhkR9sjIYseth7gPjiIy4zwcVxEgkUz+Xq89YXtDlIU kjOFOs46+xfjS/IJj5GjSHaYlUF57fsfOiuwrxAM= X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd54.lss.emc.com s7KIxBMf020794 Received: from mailusrhubprd52.lss.emc.com (mailusrhubprd52.lss.emc.com [10.106.48.25]) by maildlpprd56.lss.emc.com (RSA Interceptor); Wed, 20 Aug 2014 14:59:02 -0400 Received: from mxhub17.corp.emc.com (mxhub17.corp.emc.com [10.254.93.46]) by mailusrhubprd52.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s7KIx4T7030217 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Wed, 20 Aug 2014 14:59:04 -0400 Received: from mx15a.corp.emc.com ([169.254.1.175]) by mxhub17.corp.emc.com ([10.254.93.46]) with mapi; Wed, 20 Aug 2014 14:59:03 -0400 From: "Black, David" To: Bob Briscoe Date: Wed, 20 Aug 2014 14:59:03 -0400 Thread-Topic: [PWE3] FW: WGLC comments on: draft-ietf-pwe3-congcons-02.txt Thread-Index: Ac+8pZHsslDyppi5QT2y5ido+r6a7gAAQJEQ Message-ID: <8D3D17ACE214DC429325B2B98F3AE712077BB426E5@MX15A.corp.emc.com> References: <8D3D17ACE214DC429325B2B98F3AE712077B9517F4@MX15A.corp.emc.com> <8D3D17ACE214DC429325B2B98F3AE712077B951928@MX15A.corp.emc.com> <201408141302.s7ED2km0032238@bagheera.jungle.bt.co.uk> <8D3D17ACE214DC429325B2B98F3AE712077B952113@MX15A.corp.emc.com> <8D3D17ACE214DC429325B2B98F3AE712077B95212C@MX15A.corp.emc.com> <201408151006.s7FA6nWG003570@bagheera.jungle.bt.co.uk> <8D3D17ACE214DC429325B2B98F3AE712077BB42180@MX15A.corp.emc.com> <201408201835.s7KIZerA027879@bagheera.jungle.bt.co.uk> In-Reply-To: <201408201835.s7KIZerA027879@bagheera.jungle.bt.co.uk> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_8D3D17ACE214DC429325B2B98F3AE712077BB426E5MX15Acorpemcc_" MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd52.lss.emc.com X-RSA-Classifications: DLM_1, public, GIS Solicitation Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/RBDh8vn5OC32eDVzwgdAlZOOTJA Cc: "pwe3@ietf.org" Subject: Re: [PWE3] FW: WGLC comments on: draft-ietf-pwe3-congcons-02.txt X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Aug 2014 18:59:36 -0000 --_000_8D3D17ACE214DC429325B2B98F3AE712077BB426E5MX15Acorpemcc_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Bob, I think we're close - with luck, this email provides resolutions in princip= le to 5 and 7, leaving 8, where I don't understand your argument. -- On 5: > I figured that the curves that delineate the region are defined by the TC= P-friendly equation, so 'friendly region' says on > the tin what it does (and it is OK to imply that outside the region is un= friendly, because unfriendly doesn't imply > "Don't go there" like 'unacceptable' would). "TCP-friendly" and "possibly TCP-unfriendly" would be fine with me - I'm just uncomfortable with use of vanilla unqualified "friendly." -- On 7, in reverse order: > In fact, on reflection, whatever we say about circuit-breakers, I'd prefe= r not to mention circuit-breakers in the > abstract, which implies this draft says stuff about mechanism, which was = never the intent. Removal of mention of circuit breakers from the abstract is fine with me. > If we're going to say a bit about how to realize a circuit-breaker, I'd b= e happier with the word persistently > than without. I'd still rather we didn't allow this draft to mission cree= p into mechanism mode. I continue to agree with Stewart that something needs to be said here - the= proposed text leaves enormous flexibility to the operator to decide when to shut dow= n TDM PWs. Here it is with "persistently" restored: For TDM pseudowires, such a circuit breaker can be realized via operator monitoring of drop rates and the quality of TDM service delivered in order to shut down TDM pseudowires that persistently fail to deliver acceptable TDM service (as defined by [G775= ] and [G826]). This document explains why a TDM PW that is causing congestion problems via interference with elastic traffic is also not delivering acceptable TDM service (see the rest of this Section, plus Appendices A and B). -- On 8, I think I see the start of our disagreement: > On these graphs, the op behaviour plots won't move at all if the PW uses = larger packets. These are > bit-rate plots, so the only way to move anything is to change the TCP seg= ment size (which moves the TCP plots). Actually, they're bit-rate plots on the assumption that both the PW and TCP= are using the same segment size, which is why Figure 2 looks better for the PWs than Figu= re 1 (the PW lines did not move, but the TCP curves did move due to the segment size cha= nge). Or in other words: - Figure 1 compares a PW using 64-byte segments to a TCP connection using 6= 4-byte segments. - Figure 2 compares a PW using 256-byte segments to a TCP connection using = 256-byte segments. - Nothing in the draft makes a comparison using different segment sizes. So, "if the PW uses larger packets" then the approach in this draft compare= s it to a TCP connection using larger segment, and the result looks like Figure 2 instead= of Figure 1. I do not like the idea of comparing a PW using 256-byte segments with a TCP= connection using 64-byte segments, which is what I understand that above "won't move a= t all" assertion to be about. The TCP curves do move between Figure 1 and Figure 2, based s= olely on a TCP segment size change. Thanks, --David From: Bob Briscoe [mailto:bob.briscoe@bt.com] Sent: Wednesday, August 20, 2014 2:36 PM To: Black, David Cc: pwe3@ietf.org Subject: RE: [PWE3] FW: WGLC comments on: draft-ietf-pwe3-congcons-02.txt David, Sry for delay. inline... (Despite end of WGLC, we can still continue our public intra-author ping-po= ng :) At 13:44 18/08/2014, Black, David wrote: Bob, 1-4 & 6: ok 5: "acceptable" is a good replacement for S-A-F-E, but I'm not sure about "= friendly". The key concept here is assurance that there are no problems. = How about "clearly acceptable"? That may need some additional text to say = that acceptable network and service behavior may be delivered outside the c= learly acceptable region, but one cannot rely on the analysis in this draft= (based on the TCP throughput formula) to reach such a conclusion. I wanted a different word from 'acceptable' for the adjective in the captio= ns above the Figures describing the regions, because there's no opportunity= for more text there to explain that we do not mean that outside the region= is unacceptable. I thought about various qualifiers with acceptable (like = clearly, unarguably, etc), but I don't think these are right either. I figured that the curves that delineate the region are defined by the TCP-= friendly equation, so 'friendly region' says on the tin what it does (and i= t is OK to imply that outside the region is unfriendly, because unfriendly = doesn't imply "Don't go there" like 'unacceptable' would). Let's stop batting back & forth until we see what Yaakov (or others) think. 7: I disagree - the tsvwg circuit breaker draft is rather general, and we n= ow have two WG LC comments on this draft asking how to apply a circuit brea= ker to PWs. I think we're on solid ground with the references to the ITU-T= service quality standards and expecting operator monitoring of service qua= lity. OTOH, the word "persistently" seemed to be a concern, and I have no = problem removing it (anyone who asks about that can be directed to G.775 an= d G.826). New suggested text: For TDM pseudowires, such a circuit breaker can be realized via operator monitoring of drop rates and the quality of TDM service delivered in order to shut down TDM pseudowires that fail to deliver acceptable TDM service (as defined by [G775] and [G826]). This document explains why a TDM PW that is causing congestion problems via interference with elastic traffic is also not delivering acceptable TDM service (see the rest of this Section, plus Appendices A and B). If we're going to say a bit about how to realize a circuit-breaker, I'd be = happier with the word persistently than without. I'd still rather we didn't= allow this draft to mission creep into mechanism mode. In fact, on reflection, whatever we say about circuit-breakers, I'd prefer = not to mention circuit-breakers in the abstract, which implies this draft s= ays stuff about mechanism, which was never the intent. 8: I believe that figures 1 and 2 suffice to make the point, but I see that= a recommendation may be a bit of a reach for this draft. How about the fo= llowing modified form of my first sentence, which just states the inference= that can be drawn from Figures 1 and 2? - Figures 1 and 2 indicate that a larger segment size for a T1 or E1 TDM PW moves the operational behavior of the PW away from the boundary at which the TCP throughput equation suggests that undue interference wi= th elastic traffic may start to occur. Not so - that's the problem. On these graphs, the op behaviour plots won't = move at all if the PW uses larger packets. These are bit-rate plots, so the= only way to move anything is to change the TCP segment size (which moves t= he TCP plots). To justify what you want to say, either: a) these would have to be packet-rate plots, or b) we would have to turn the assumption we've made about equal packet sizes= into an operational requirement, not just an analytical assumption. To go in either of these directions would require a spade and vermin. Bob PS. GOing offline until next Tue. Might check email, but no guarantee. Thanks, --David From: Bob Briscoe [ mailto:bob.briscoe@bt.com] Sent: Friday, August 15, 2014 6:03 AM To: Black, David Cc: pwe3@ietf.org Subject: Re: [PWE3] FW: WGLC comments on: draft-ietf-pwe3-congcons-02.txt David, I decided to think on a better word for 'safe' overnight, and allow others = to comment. But no-one has, so here's my responses to all your outstanding = points... I've provided slightly updated xml source of my suggested edits (and a diff= ) here: < http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-pwe= 3-congcons-DIFF03b-02.html > < http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-pwe= 3-congcons-03b.txt > < http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-pwe= 3-congcons-03b.xml > Again note: xml is purely for precision, not implying I have taken over edi= torship. 1. Happy with your new scoping text, except the the first 'independent' cla= ngs with the second, and it's unnecessary anyway: " The concern is over PWs that share network capacity with elastic or cong= estion-responsive traffic, [independent of] whether that sharing is a resul= t of the service provider network's traffic engineering and capacity planni= ng or independent deployment of PWs by entities other than network operator= s. " 2. My point wasn't to remove the reference to the 125us (in my suggested te= xt I didn't). It was that the 125us stuff /alone/ doesn't warrant the 'thus= ' in the next clause; the logic needs to be connected by talking about the = sum of the flight time /and/ the per-node processing time. CURRENT: The one-way delay of a native TDM service consists of the physical time-of-flight plus 125 microseconds for each TDM switch traversed; and is thus very small as compared to typical PSN network-crossing latencies. SUGGESTED: The one-way delay of a native TDM service consists of the physical time-of-flight plus 125 microseconds for each TDM switch traversed; so the total one-way del= ay for most TDM PWs has to be small compared to typical PSN network-crossin= g latencies. 5. I understand and agree - 'S-A-F-E region' is incendiary. But I don't lik= e 'reasonable region' - it's slightly /too/ meaningless. Other possibilities are: - admissable region (problem: implies outside the region is inadmissable) - harmonious region (problem: too Californian) So I suggest two words: - friendly or acceptable, each used depending on context. On the graphs, I suggest 'operating region (friendly under curve)' because = that is precisely where the curves come from - 'friendly' does what it says= on the tin, and the implied 'unfriendly' above the curve correctly avoids = implying that the sky will fall. Elsewhere, I suggest the following replacements for the S-word: "It is thus worthwhile specifying under what conditions such competition is= acceptable" "One may view this condition as defining a 'friendly' operating envelope fo= r a TDM PW" "Under this condition it is acceptable to place the TDM PW along with ... T= CP. " "comparison with something widely considered to offer an acceptable respons= e to congestion" 7. Circuit breaker Scope creep. Rat-hole. Stop digging. Yes, I know Stewart has asked us to dig out another rat as well, but the ap= propriate response is "Out of scope". You might think your text is uncontroversial, but if we are tempted to take= Stewart's bait (rat metaphor intended), it will probably start new debates= that this doc doesn't deserve: "What does persistent mean?", "Which traffi= c is more important/ valuable?", yada yada. I suggest the text merely makes it clear that this debate is in scope for e= lsewhere, not here. I've also just realised that Yaakov has written the sen= tences around circuit breaker with the wrong logical sequence. The circuit = breaker needs to be introduced as something the PW uses when its /own/ qual= ity has failed, which should avoid any need to shut down because it is thre= atening elastic services: CURRENT: If the bandwidth consumed by a TDM PW is considered detrimental, the only available remedy is to completely shut down the PW, by using a transport circuit breaker mechanism. However, we will show that even before such an action is warranted, the PW will become unable to faithfully emulate the native TDM service; for example, when a TDM service is carrying voice grade telephony channels, the voice quality will degrade to below useful levels. SUGGESTED: We will show that, before a PW becomes deterimental to elastic traffi= c, it will become unable to faithfully emulate the native TDM service; for example, when a TDM service is carrying voice grade telephony channels, the voice quality will degrade to below useful levels. Once the quality of the emulated service is pers= istently unacceptable, if a PW shuts down selected flows or shuts itself do= wn completely, it will automatically avoid detriment to elastic traffic. A = transport circuit breaker [I-D.fairhurst-tsvwg-circuit-breaker] could be us= ed, but mechanism is beyond the scope of the present document. 8. Segment size. Everyone's in a big group-hug because we've written a draft showing there's= is no need to change anything. Then just at the end of an uneventful WGLC,= we hear David's voice from the bottom of a deep hole shouting, "avoid the = rat-hole invitation", while furiously digging for a rat. There is no need to add a recommendation that PWs need to be changed to use= larger packets. No matter how hedged and soft this recommendation would be= , it's not needed anyway. Stop digging. Yaakov's new text is fine as it stands. "Only for small packets, long delay= s, and high packet loss ratios, do TDM PWs potentially consume more bandwid= th, and even then only marginally. " In order to justify your sentence, we would have to include graphs with pac= ket-rate (not bit-rate) on the vertical axis and go into a whole treatise a= s to why packet-rate comparison might be important as well. Then the graph = would show one plot for all TCP segment sizes and multiple plots for PWs wi= th different packet sizes. Then we'd see that the corner of the two smalles= t packet PWs just clipped the TCP curve at loss probabilities that are stup= idly large anyway. So even after all this, it would hardly justify a need f= or your sentence. All that digging would not be worth it, just to discover (again) that there= is no rat. Bob At 19:39 14/08/2014, Black, David wrote: Content-Language: en-US Content-Type: multipart/alternative; boundary=3D"_000_8D3D17ACE214DC429325B2B98F3AE712077B95212CMX15Aco= rpemcc_" And forwarding my views to the list ... ... On Bob's six thoughts (I have problems with 2 & 5 - the other 4 are bas= ically ok w/me) plus follow-ups on Bob's concerns w/my two major issues, he= re goes: -- 1 -- I agree in principle, but I think Bob's suggested new text is too b= lack/white and should be generalized. Bob suggests: > The case where the service provider network arranges sufficient capacity = for a PW is well understood and > will not be discussed further here. The concern is over PWs deployed inde= pendently of the service provider > network's traffic engineering or capacity planning. I want to include the case where capacity has been arranged, but may not be= sufficient, I suggest rewriting Bob's second sentence as: The concern is over PWs that share network capacity with elastic or congest= ion-responsive traffic, independent of whether that sharing is a result of = the service provider network's traffic engineering and capacity planning or= independent deployment of PWs by entities other than network operators. -- 2 -- Here, I disagree. One of the significant contributions of this dra= ft is that TDM PWs don't cause congestion problems in practice, and one of = the major reasons for that is the low latency requirements for acceptable T= DM service delivery, both in ITU-T standards and in actual TDM PW deploymen= t. The 125us behavior of native TDM switches is relevant to the latter, an= d hence I prefer to include mention of it. -- 3 -- I agree; clear captions are always a feature, not a bug ;-). -- 4 -- I agree with changing "packet loss rate" to "packet loss probabilit= y". -- 5 -- I firmly disagree with Bob here, although he's right that this is "= techno-political" ;-). I am concerned that S-A-F-E is likely to be a 4-letter word in wider Transp= ort Area review, and hence, I want to use a word that does not imply anythi= ng remotely resembling an absolute guarantee that nothing can possibly ever= go wrong (yes, I know Bob didn't mean to go to quite that extreme an inter= pretation, but S-A-F-E is liable to be read that way by those with an "axe = to grind"), even though there's lots of solid operational experience that s= uggests all is well here. -- 6 -- I think I'm ok with what Bob proposes to better characterize loss b= urstiness, but want to see specific text. -- Circuit Breaker Mechanism -- >> [David] I suggest adding: >> >> For TDM pseudowires, such a circuit breaker can be realized via >> operator monitoring of drop rates and consequent on the quality of >> TDM service delivered and shutting down TDM pseudowires that persiste= ntly >> fail to deliver acceptable TDM service (as defined by [G775] and >> [G826]). This document explains why a TDM PW that is causing congest= ion >> problems via interference with elastic traffic cannot be delivering >> acceptable TDM service (see the rest of this Section, plus Appendices >> A and B). >> > [Bob] I don't think we should go there. We need to keep this draft firmly= as informational. If someone > thinks a mechanism is needed because of the information in this draft, th= en they can write about a mechanism. [David] I firmly disagree, but Bob and I might be talking past each other, = as I do want to stay informational ;-). This text is intended to describe an example or possibility, not to specify= an implementation requirement. The new text was intended as an OAM/OSS pr= actice recommendation to the operator, not as a normative protocol spec. I'm ok with inserting "example", "possibility" or something similar if that= helps w/Bob's concern, but I strongly oppose doing nothing here, and note = that this is also needed to respond to one of Stewart Bryant's WG LC commen= ts. -- PW segment size -- >> [David] Ignoring the rathole invitation ;-). This is still in section 3= , after: >> >> We see that in general for large packet sizes, short delays, and low >> packet loss rates, the TDM pseudowires consume much less bandwidth >> than TCP would under identical conditions. Only for small packets, >> long delays, and high packet loss ratios, do TDM PWs potentially >> consume more bandwidth, and even then only marginally. >> >> I suggest adding: >> >> Comparing figures 1 and 2, use of a larger segment size for a T1 or E= 1 >> TDM PW moves the operational behavior of the PW away from the boundar= y >> at which the TCP throughput equation suggests that undue interference= with >> elastic traffic may start to occur. We suggest that larger segment >> sizes be preferred for T1 and E1 TDM PWs, but note that other >> considerations may favor smaller segment sizes (e.g., jitter buffer >> size, bounding the impact of an isolated packet drop event). >> > [Bob] There is no need to recommend any remedial action, when we have jus= t shown there is not a problem. [David] Not exactly - Figure 1 shows that there is a potential concern at a= combination of the high end of the range of TDM PW latencies (4ms), and hi= gher packet loss ratios. > Further, the "marginal problem" only applies when the TCP traffic uses pa= ckets smaller than 128B, which we only used > as a hypothetical worst-case for the analysis. But in practice this doesn= 't happen with long-running TCP flows - > they use packets that are as large as possible (irrespective of what the = PW uses). So there really is /no/ problem to remedy. Ok, Bob can relax - his transport credentials are intact, as this is an ent= ry point to the bit-rate-congestible vs. packet-rate-congestible resource r= athole that I want to ignore :-). I believe that larger TDM PW segment sizes are a good thing for both this o= perating envelope boundary concern as well as per-packet processing overhea= ds in TDM PW functionality, modulo other considerations, e.g., jitter buffe= rs, and so I prefer to keep this addition. Thanks, --David From: Bob Briscoe [mailto:bob.briscoe@bt.com] Sent: Thursday, August 14, 2014 9:03 AM To: Yaakov Stein Cc: Andrew G. Malis; BOCCI Matthew; Black, David Subject: RE: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt Yaakov, I have just re-read it thoroughly. I have noticed about two dozen editorial= nits where correcting them would not change the intended sense (altho some= times the sense needs to change to say what we intended!). Examples: - Unexpanded abbreviations (David's suggestion would help, but there are mo= re) - Missing words, tautologies, random punctuation problems etc - Uses of the wrong word, even tho we know what you mean - The caveat about dynamics vs steady state needs to shift down a para, so = that it follows into the para that says what to do about dynamics. Should I take the token, and edit the XML directly, then you can see a diff= to check if you want to accept my suggestions? Below are six thoughts that go slightly beyond 'editorial'. 1-4 could be co= nsidered 'editorial'. But 5&6 are the more significant "techno-political" p= oints. 1. The phrase "mechanisms for provisioning PWs in service provider networks= " is too opaque (the PWE audience might understand this scoping sentence, b= ut a L4 reader won't understand what we're excluding here, which will make = them suspicious). CURRENT: Mechanisms for provisioning PWs in service provider networks are well understood and will not be discussed further here. SUGGESTED: The case where the service provider network arranges sufficient capacity= for a PW is well understood and will not be discussed further here. The co= ncern is over PWs deployed independently of the service provider network's = traffic engineering or capacity planning. 2. The justification for using OWD of a few milliseconds misses the point. " The one-way delay of a native TDM service consists of the physical time-of-flight plus 125 microseconds for each TDM switch traversed; and is thus very small as compared to typical PSN network-crossing latencies. " The 125us per switch is irrelevant. The point that needs making is that PWs= are generally deployed with a short physical time of flight. 3. Related point: the captions on the graphs do not clearly indicate that t= hey are OWD, not RTT. This could be clarified in the text, or on the graphs= . 4. Change packet loss rate (and PLR) to packet loss probability throughout,= including on the graph axes. PLR is an ambiguous term, even though it is w= idely used in the industry. Packet loss rate could be measured in packet pe= r second, whereas packet loss probability is unambiguously dimensionless. I= ndeed, in the appendices, packet loss rates with respect to time are calcul= ated. 5. Rather than David's "reasonable" I suggest we use the phrase "unambiguou= sly safe" not just "safe" for the headings of Figs 5-8 and in the text, as = below. Otherwise, we are implying that it is unsafe outside the region. CURRENT: One may view this condition as defining an operating envelope for a TDM PW, as a TDM PW that occupies no more bandwidth than a TCP flow causes no more congestion than that TCP flow would. Under this condition it is safe to place the TDM PW along with congestion- responsive traffic such as TCP, without causing additional congestion. on the other hand, were the TDM PW to consume significantly more bandwidth a TCP flow, it could contribute disproportionately to congestion, and its mixture with congestion- responsive traffic might be inappropriate. SUGGESTED: One may view this condition as defining an unambiguously safe operating = envelope for a TDM PW, as a TDM PW that occupies no more bandwidth than a TCP flow causes no more congestion than that TCP flow would. This does not draw = us into a debate over whether TCP-friendliness is a valid constraint; it si= mply says that there can be no question that a PW is safe if it does no mor= e harm than a single TCP flow. Under this condition it is unambiguously safe to place the TDM PW along with conges= tion- responsive traffic such as TCP. On the other hand, were the TDM PW to c= onsume significantly more bandwidth a TCP flow, it could contribute disproportionately to congestion, and its mixture with congestion- responsive traffic might start to cause concern. 6. The analysis in Appendices A & B needs to mention that loss bursts arise= primarily from tail drop buffers, and the introduction of active queue man= agement (AQM) makes losses more independent of each other (less bursty). (I= have references showing evidence of this.) In the comparisons with TCP, we= have assumed independent losses anyway, because they are the worst case (a= nd fortunately they are easier to analyse). On a similar point, in appendix= B, where we say "(bursty packet loss was also simulated but is not reporte= d here)", we ought to give a reason otherwise readers will be suspicious. A= good reason would be that uniformly random is the worst case. I've also responded to a couple of David's suggestions inline (I agree with= the others)... At 17:44 13/08/2014, Yaakov Stein wrote: Bob told me on Friday of the IETF that he was going to look over the final = version and comment. Bob - ping ? Y(J)S From: Andrew G. Malis [ mailto:agmalis@gmail.com] Sent: 13 August, 2014 17:02 To: Yaakov Stein Cc: Bob Briscoe; rbriscoe@jungle.bt.co.uk;= BOCCI Matthew; Black, David Subject: Re: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt Yaakov, I still haven't heard from Bob, but you should incorporate David's updates = into a new revision, and I'll use that version for my final Shepherd's revi= ew. Thanks, Andy On Mon, Aug 11, 2014 at 11:16 AM, Andrew G. Malis > wrote: Actually, I'm going to end the LC as soon as I hear back from Bob on the IP= R call. I sent him a reminder earlier this morning. Cheers, Andy On Sun, Aug 10, 2014 at 9:53 AM, Andrew G. Malis > wrote: David, That's perfect, thanks! I'm going to end the LC tomorrow. Cheers, Andy On Sat, Aug 9, 2014 at 10:32 PM, Black, David > wrote: Following up on this ... > As we're now moving quickly to try to complete this draft , I agreed > w/Yaakov that my author comments would be sent in as WG Last Call comment= s ... > ... so here they are: ... with some suggested new text ... > Major issues: > > The reference to a transport circuit breaker in section 3 is a bit optimi= stic > and high level: > > We can accomplish this by > employing a transport circuit breaker, by which we mean an automati= c > mechanism for terminating a flow to prevent negative impact on othe= r > flows and on the stability of the network > [I-D.fairhurst-tsvwg-circuit-breaker]. > > This should be accompanied by at least an example of how to do this - pas= t > discussion has suggested carrier monitoring of delivered service quality > in order to shut down PWs that are experiencing loss rates that result in > unacceptable service quality (as defined for us by the ITU ...). I'd als= o > like to add some text to suggest that operators use this sort of circuit > breaker practice when TDM PWs compete with elastic traffic. After this text in Section 3: We can accomplish this by employing a transport circuit breaker, by which we mean an automatic mechanism for terminating a flow to prevent negative impact on other flows and on the stability of the network [I-D.fairhurst-tsvwg-circuit-breaker]. Note that a transport circuit breaker is intended as a protection mechanism of last resort, just as an electrical circuit breaker is only triggered when absolutely necessary. I suggest adding: For TDM pseudowires, such a circuit breaker can be realized via operator monitoring of drop rates and consequent on the quality of TDM service delivered and shutting down TDM pseudowires that persistentl= y fail to deliver acceptable TDM service (as defined by [G775] and [G826]). This document explains why a TDM PW that is causing congestion problems via interference with elastic traffic cannot be delivering acceptable TDM service (see the rest of this Section, plus Appendices A and B). I don't think we should go there. We need to keep this draft firmly as info= rmational. If someone thinks a mechanism is needed because of the informati= on in this draft, then they can write about a mechanism. And breaking the paragraph after the added text. > Figure 1 and Figure 2 indicate that based on using the TCP throughput > equation as a proxy for potential impact on elastic traffic (which is wha= t > we're doing in this draft), for the same fixed bandwidth PW, a larger seg= ment > size is less likely to cause impacts. I'd like to add a suggestion to us= e > the larger PW segment sizes as long as they're consistent with delivery o= f > the service. There's been a related (long-to-the-point-of-ratholed) deba= te > over in the Transport Area about whether packet-rate-limited/congestible > forwarding is still relevant. My view is that such equipment exists, and > hence that this is a relevant consideration for this draft, and would not > mention that issue here. Ignoring the rathole invitation ;-). This is still in section 3, after: We see that in general for large packet sizes, short delays, and low packet loss rates, the TDM pseudowires consume much less bandwidth than TCP would under identical conditions. Only for small packets, long delays, and high packet loss ratios, do TDM PWs potentially consume more bandwidth, and even then only marginally. I suggest adding: Comparing figures 1 and 2, use of a larger segment size for a T1 or E1 TDM PW moves the operational behavior of the PW away from the boundary at which the TCP throughput equation suggests that undue interference wi= th elastic traffic may start to occur. We suggest that larger segment sizes be preferred for T1 and E1 TDM PWs, but note that other considerations may favor smaller segment sizes (e.g., jitter buffer size, bounding the impact of an isolated packet drop event). There is no need to recommend any remedial action, when we have just shown = there is not a problem. Further, the "marginal problem" only applies when the TCP traffic uses pack= ets smaller than 128B, which we only used as a hypothetical worst-case for = the analysis. But in practice this doesn't happen with long-running TCP flo= ws - they use packets that are as large as possible (irrespective of what t= he PW uses). So there really is /no/ problem to remedy. and again, breaking the paragraph after that added text. I hope that latter sentence ("We suggest that ...") is acceptable to the WG= . > Minor issues: > > Section 2 is about Ethernet PWs that carry elastic traffic. Should other= s > be mentioned as also covered by this analysis - e.g., ATM and Frame Relay > PWs carrying elastic traffic? After: In this section we consider Ethernet PWs that primarily carry congestion-responsive traffic. We show that we automatically obtain the desired congestion avoidance behavior, and that additional mechanisms are not needed. I suggest adding: This discussion is applicable to any PW that carries congestion- responsive traffic (e.g., ATM and Frame Relay PWs); Ethernet PWs are used here for clarity, as they are a common example. > Section 2 does not cover Ethernet PWs carrying inelastic traffic - that w= ould > be unusual, but should it be explicitly noted as out of scope, or just no= t > discussed (as is the case for the current text)? I suggest doing nothing here, and planning to deal with this concern if it should arise later (e.g., IETF Last Call or IESG Evaluation). > Section 3 should say that the reason the draft focuses on TDM PWs is that > they're the inelastic PWs that are likely to be carried over IP in practi= ce. > My understanding is that other potentially inelastic PWs are MPLS-only > in practice. After: Inelastic PWs, such as TDM PWs ([RFC4553][RFC5086][RFC5087]), are potentially more problematic than the elastic PWs of the previous section. Being constant bit-rate (CBR), TDM PWs can not be made responsive to congestion. On the other hand, being CBR, they also do not attempt to capture additional bandwidth when neighboring TCP flows back off. I suggest adding: We focus on TDM PWs here as they are the only inelastic PWs that are likely to be IP-encapsulated in practice. Most other inelastic PWs tend to only be used with MPLS. [I'd ask Yaakov and other PW experts to double-check the latter sentence, please.] > A terminology section to roll up all the acronym expansions (e.g., TDM, C= BR) > in one place seems like a good idea. These four acronyms would be good to put into a terminology section: CBR Constant Bit Rate PSN Packet Switched Network PW Pseudowire TDM Time Division Multiplexing > > This text on p.11 could be incendiary: > > Under this condition it is safe to place the TDM PW along with > congestion - responsive traffic such as TCP, without causing > additional congestion. > > First of all to reduce the flammability, "safe" -> "reasonable" and there > are a number of other occurrences of "safe" that deserve similar treatmen= t ;- > ). > Beyond that, this assertion is dependent on the operating conditions stay= ing > within that reasonable operating envelope, which entails use of something > like a circuit breaker - I'd say that here, and reference the earlier > circuit breaker discussion. Global substitution: "safe" -> "reasonable", starting with the abstract. Then for this specific text on p.11: OLD Under this condition it is safe to place the TDM PW along with congestion - responsive traffic such as TCP, without causing additional congestion. NEW Under this condition it is reasonable to place the TDM PW in compet= ition with congestion-responsive traffic such as TCP, as the result should = not produce unacceptable congestion effects. That assumes that the opera= ting conditions remain within the reasonable operating envelope - as discu= ssed above, a transport circuit breaker is a suggested means of managing a= TDM PW so that it is shut down if operating conditions remain persistentl= y outside the reasonable operating envelope for the PW (e.g., due to a high packet loss rate). No need to go there. See my suggestions earlier. > Figures 7 and 8 could use some shading or lines to indicate the typical > deployed operating regions for the pseudo-wires to visibly show that they > fall under the curves, particularly for larger segment sizes. I'd also > repeat or refer to the suggestion to use larger I wanted to suggest this myself, but it is hard to depict fuzzy boundaries.= If you can do this, pls do. Given the PW operating regions are bounded by v low loss probability, the x= axis in Figs 7&8 ought to be scaled much smaller as well. Related point: the y axis in Fig 1 ought to max at 10, not 50. Regards Bob > > Nits: > > References aren't allowed in abstracts, so remove reference to > [I-D.fairhurst-tsvwg-circuit-breaker] > > Introduction, first paragraph, delete "PWE3" towards end of paragraph - > > preferably the 4 byte PWE3 control word. > > In addition to its current general statements, the Security Consideration= s > section should point to the security considerations sections in a few > particularly relevant RFCs - I'd suggest 3985, 4553, 5086 and 5087. After: This document does not introduce any new congestion-specific mechanisms and thus does not introduce any new security considerations above those present for PWs in general. I suggest adding a new paragraph: For PW security considerations, see the security considerations sections of the cited RFCS; the security considerations in [RFC3985], [RFC4553], [RFC5086] and [RFC5087] are particularly applicable to this document. > > Thanks, > --David > > > > -----Original Message----- > > From: pwe3 [ mailto:pwe3-bounces@ietf.org] On Behalf Of Yaakov Stein > > Sent: Thursday, July 24, 2014 4:52 PM > > To: pwe3@ietf.org > > Subject: Re: [PWE3] I-D Action: draft-ietf-pwe3-congcons-02.txt > > > > Please look at the PDF version only > > (the txt version does not have the graphs). > > > > Y(J)S > > > > > > _______________________________________________ > pwe3 mailing list > pwe3@ietf.org > https://www.ietf.org/mailman/listinfo/pwe3 _______________________________________________ pwe3 mailing list pwe3@ietf.org https://www.ietf.org/mailman/listinfo/pwe3 ________________________________________________________________ Bob Briscoe, BT _______________________________________________ pwe3 mailing list pwe3@ietf.org https://www.ietf.org/mailman/listinfo/pwe3 ________________________________________________________________ Bob Briscoe, BT ________________________________________________________________ Bob Briscoe, BT --_000_8D3D17ACE214DC429325B2B98F3AE712077BB426E5MX15Acorpemcc_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Bob,

 

I think= we’re close - with luck, this email provides resolutions in principl= e to 5 and 7,

leaving 8, where I don&= #8217;t understand your argument.

 

-- On 5:

 

> I figur= ed that the curves that delineate the region are defined by the TCP-friendl= y equation, so 'friendly region' says on

> the tin what it does (and it is OK to imply that outside the region i= s unfriendly, because unfriendly doesn't imply

> "Don't go there" like 'unacceptable' would).

 

= R= 20;TCP-friendly” and “possibly TCP-unfriendly” would be f= ine with me -

I’m just uncomfor= table with use of vanilla unqualified “friendly.”

 

<= span style=3D'font-size:10.0pt;font-family:"Courier New";color:black'>-- On= 7, in reverse order:

 

> In fact, on reflection, whatever = we say about circuit-breakers, I'd prefer not to mention circuit-breakers i= n the

> abstract, which implies this = draft says stuff about mechanism, which was never the intent.

 

Removal of ment= ion of circuit breakers from the abstract is fine with me.

 

> If we're going to say a bit about how to realize a circuit-breaker, I'd= be happier with the word persistently

&= gt; than without. I'd still rather we didn't allow this draft to mission cr= eep into mechanism mode.

 

I continue to agree with Stewart that something need= s to be said here - the proposed

= text= leaves enormous flexibility to the operator to decide when to shut down TD= M PWs.

Here it is with “persist= ently” restored:

 

   For TDM pseudowires, such a= circuit breaker can be realized via
   operator monitoring of= drop rates and the quality of
   TDM service delivered in ord= er to shut down TDM pseudowires that
   persistently fail to d= eliver acceptable TDM service (as defined by [G775] and
   [G8= 26]).  This document explains why a TDM PW that is causing congestion<= br>   problems via interference with elastic traffic is also not = delivering
   acceptable TDM service (see the rest of this Sec= tion, plus Appendices
   A and B).

 

-- On 8, I think I see the= start of our disagreement:

&nbs= p;

> On these graphs, the op behavi= our plots won't move at all if the PW uses larger packets. These are

> bit-rate plots, so the only way to move a= nything is to change the TCP segment size (which moves the TCP plots).=

 

Actual= ly, they’re bit-rate plots on the assumption that both the PW and TCP= are using the

same segment size, whi= ch is why Figure 2 looks better for the PWs than Figure 1 (the PW

lines did not move, but the TCP curves did mov= e due to the segment size change).

 

Or in other words:

- Figure 1 compares a PW using 64-byte segments to= a TCP connection using 64-byte segments.

- Figure 2 compares a PW using 256-byte segments to a TCP connection u= sing 256-byte segments.

- Nothing in = the draft makes a comparison using different segment sizes.

 

So, = 220;if the PW uses larger packets” then the approach in this draft co= mpares it to a TCP

connection using l= arger segment, and the result looks like Figure 2 instead of Figure 1.=

 

I do not like the idea of comparing a PW using 256-byte segments with a= TCP connection

using 64-byte segment= s, which is what I understand that above “won’t move at allR= 21; assertion

to be about.  The = TCP curves do move between Figure 1 and Figure 2, based solely on a TCP

segment size change.

 

T= hanks,
--David

=  

From: Bob B= riscoe [mailto:bob.briscoe@bt.com]
Sent: Wednesday, August 20, 2= 014 2:36 PM
To: Black, David
Cc: pwe3@ietf.org
Su= bject: RE: [PWE3] FW: WGLC comments on: draft-ietf-pwe3-congcons-02.txt=

 

David,

Sry for delay. inline...

(Despit= e end of WGLC, we can still continue our public intra-author ping-pong :)
At 13:44 18/08/2014, Black, David wrote:

Bob,
 
1-4 & 6: ok
 
5: “acc= eptable” is a good replacement for S-A-F-E, but I’m not sure ab= out “friendly”.  The key concept here is assurance that th= ere are no problems.  How about “clearly acceptable”? = ; That may need some additional text to say that acceptable network and ser= vice behavior may be delivered outside the clearly acceptable region, but o= ne cannot rely on the analysis in this draft (based on the TCP throughput f= ormula) to reach such a conclusion.


= I wanted a different word from 'acceptable' for the adjective in the captio= ns above the Figures describing the regions, because there's no opportunity= for more text there to explain that we do not mean that outside the region= is unacceptable. I thought about various qualifiers with acceptable (like = clearly, unarguably, etc), but I don't think these are right either.
I figured that the curves that delineate the region are defined by the TCP= -friendly equation, so 'friendly region' says on the tin what it does (and = it is OK to imply that outside the region is unfriendly, because unfriendly= doesn't imply "Don't go there" like 'unacceptable' would).
Let's stop batting back & forth until we see what Yaakov (or others) = think.



7: I disagree - t= he tsvwg circuit breaker draft is rather general, and we now have two WG LC= comments on this draft asking how to apply a circuit breaker to PWs. = I think we’re on solid ground with the references to the ITU-T servi= ce quality standards and expecting operator monitoring of service quality.&= nbsp; OTOH, the word “persistently” seemed to be a concern, and= I have no problem removing it (anyone who asks about that can be directed = to G.775 and G.826).  New suggested text:
 
   Fo= r TDM pseudowires, such a circuit breaker can be realized via
 &nbs= p; operator monitoring of drop rates and the quality of
   TDM= service delivered in order to shut down TDM pseudowires that
 &nbs= p; fail to deliver acceptable TDM service (as defined by [G775] and
&nbs= p;  [G826]).  This document explains why a TDM PW that is causing= congestion
   problems via interference with elastic traffic = is also not delivering
   acceptable TDM service (see the rest= of this Section, plus Appendices
   A and B).

<= p class=3DMsoNormal>
If we're going to say a bit about how to realize a = circuit-breaker, I'd be happier with the word persistently than without. I'= d still rather we didn't allow this draft to mission creep into mechanism m= ode.

In fact, on reflection, whatever we say about circuit-breakers,= I'd prefer not to mention circuit-breakers in the abstract, which implies = this draft says stuff about mechanism, which was never the intent.

<= br>

 
8: I believe that figures = 1 and 2 suffice to make the point, but I see that a recommendation may be a= bit of a reach for this draft.  How about the following modified form= of my first sentence, which just states the inference that can be drawn fr= om Figures 1 and 2? -
  
   Figures 1 and 2 indi= cate that a larger segment size for a T1 or E1
   TDM PW moves= the operational behavior of the PW away from the boundary
   = at which the TCP throughput equation suggests that undue interference with<= br>   elastic traffic may start to occur.


Not so - that's the problem. On these graphs, the op behav= iour plots won't move at all if the PW uses larger packets. These are bit-r= ate plots, so the only way to move anything is to change the TCP segment si= ze (which moves the TCP plots). To justify what you want to say, either:a) these would have to be packet-rate plots, or
b) we would have to tur= n the assumption we've made about equal packet sizes into an operational re= quirement, not just an analytical assumption.

To go in either of the= se directions would require a spade and vermin.


Bob

PS. G= Oing offline until next Tue. Might check email, but no guarantee.

 
Thanks,
--David
 = ;
From: Bob Briscoe [ mailt= o:bob.briscoe@bt.com]
Sent: Friday, August 15, 2014 6:03 AM<= br>To: Black, David
Cc: p= we3@ietf.org
Subject: Re: [PWE3] FW: WGLC comments on: draft-= ietf-pwe3-congcons-02.txt
 
David,

I decided to think on = a better word for 'safe' overnight, and allow others to comment. But no-one= has, so here's my responses to all your outstanding points...

I've = provided slightly updated xml source of my suggested edits (and a diff) her= e:
< http://www.bobbriscoe.net/p= rojects/2020comms/pwe3-congcons/draft-ietf-pwe3-congcons-DIFF03b-02.html >
< http://www.bobbriscoe.net/proje= cts/2020comms/pwe3-congcons/draft-ietf-pwe3-congcons-03b.txt >
&l= t; http://www.bobbriscoe.net/projects/2020comm= s/pwe3-congcons/draft-ietf-pwe3-congcons-03b.xml >
Again note: xm= l is purely for precision, not implying I have taken over editorship.
1. Happy with your new scoping text, except the the first 'independent' c= langs with the second, and it's unnecessary anyway:
"  The con= cern is over PWs that share network capacity with elastic or congestion-res= ponsive traffic, [independent of] whether that sharing is a result of the s= ervice provider network's traffic engineering and capacity planning or inde= pendent deployment of PWs by entities other than network operators.
&quo= t;

2. My point wasn't to remove the reference to the 125us (in my su= ggested text I didn't). It was that the 125us stuff /alone/ doesn't warrant= the 'thus' in the next clause; the logic needs to be connected by talking = about the sum of the flight time /and/ the per-node processing time.
CUR= RENT:
   The one-way delay
   of a native TDM ser= vice consists of the physical time-of-flight plus
   125 micro= seconds for each TDM switch traversed; and is thus very
   sma= ll as compared to typical PSN network-crossing latencies.
SUGGESTED:
=    The one-way delay
   of a native TDM service con= sists of the physical time-of-flight plus
   125 microseconds= for each TDM switch traversed; so the total one-way delay for most TDM PWs= has to be small compared to typical PSN network-crossing latencies.
5. I understand and agree - 'S-A-F-E region' is incendiary. But I don't l= ike 'reasonable region' - it's slightly /too/ meaningless.

Other pos= sibilities are:
- admissable region (problem: implies outside the region= is inadmissable)
- harmonious region (problem: too Californian)
So I= suggest two words:
- friendly or acceptable, each used depending on con= text.

On the graphs, I suggest 'operating region (friendly under cu= rve)' because that is precisely where the curves come from - 'friendly' doe= s what it says on the tin, and the implied 'unfriendly' above the curve cor= rectly avoids implying that the sky will fall.

Elsewhere, I suggest = the following replacements for the S-word:
"It is thus worthwhile s= pecifying under what conditions such competition is acceptable"
&qu= ot;One may view this condition as defining a 'friendly' operating envelope = for a TDM PW"
"Under this condition it is acceptable to place = the TDM PW along with ... TCP. "
"comparison with something wi= dely considered to offer an acceptable response to congestion"

=
7. Circuit breaker
Scope creep. Rat-hole. Stop digging.
Yes, I k= now Stewart has asked us to dig out another rat as well, but the appropriat= e response is "Out of scope".

You might think your text is= uncontroversial, but if we are tempted to take Stewart's bait (rat metapho= r intended), it will probably start new debates that this doc doesn't deser= ve: "What does persistent mean?", "Which traffic is more imp= ortant/ valuable?", yada yada.

I suggest the text merely makes = it clear that this debate is in scope for elsewhere, not here. I've also ju= st realised that Yaakov has written the sentences around circuit breaker wi= th the wrong logical sequence. The circuit breaker needs to be introduced a= s something the PW uses when its /own/ quality has failed, which should avo= id any need to shut down because it is threatening elastic services:
CURRENT:
      If the bandwidth consumed by a = TDM PW is
      considered detrimental, the onl= y available remedy is to completely
      shut = down the PW, by using a transport circuit breaker mechanism.
  = ;    However, we will show that even before such an action i= s
      warranted, the PW will become unable to= faithfully emulate the
      native TDM servic= e; for example, when a TDM service is carrying
    &= nbsp; voice grade telephony channels, the voice quality will degrade to
=       below useful levels.
SUGGESTED:
 = ;     We will show that, before a PW becomes deteriment= al to elastic traffic, it will become unable to faithfully emulate the
&= nbsp;     native TDM service; for example, when a TDM s= ervice is carrying
      voice grade telephony = channels, the voice quality will degrade to
    &nbs= p; below useful levels. Once the quality of the emulated service is persist= ently unacceptable, if a PW shuts down selected flows or shuts itself down = completely, it will automatically avoid detriment to elastic traffic. A tra= nsport circuit breaker [I-D.fairhurst-tsvwg-circuit-breaker] could be used,= but mechanism is beyond the scope of the present document.


8. S= egment size.
Everyone's in a big group-hug because we've written a draft= showing there's is no need to change anything. Then just at the end of an = uneventful WGLC, we hear David's voice from the bottom of a deep hole shout= ing, "avoid the rat-hole invitation", while furiously digging for= a rat.

There is no need to add a recommendation that PWs need to be= changed to use larger packets. No matter how hedged and soft this recommen= dation would be, it's not needed anyway. Stop digging.

Yaakov's new = text is fine as it stands. "Only for small packets, long delays, and h= igh packet loss ratios, do TDM PWs potentially consume more bandwidth, and = even then only marginally. "

In order to justify your sentence= , we would have to include graphs with packet-rate (not bit-rate) on the ve= rtical axis and go into a whole treatise as to why packet-rate comparison m= ight be important as well. Then the graph would show one plot for all TCP s= egment sizes and multiple plots for PWs with different packet sizes. Then w= e'd see that the corner of the two smallest packet PWs just clipped the TCP= curve at loss probabilities that are stupidly large anyway. So even after = all this, it would hardly justify a need for your sentence.

All tha= t digging would not be worth it, just to discover (again) that there is no = rat.


Bob


At 19:39 14/08/2014, Black, David wrote:
Content-Language: en-US
Content-Type: multipart/alternative;
&nb= sp;        boundary=3D"_000_8D3D17A= CE214DC429325B2B98F3AE712077B95212CMX15Acorpemcc_"

And forwardi= ng my views to the list ...
 
... On Bob’s six thoughts (I= have problems with 2 & 5 - the other 4 are basically ok w/me) plus fol= low-ups on Bob’s concerns w/my two major issues, here goes:
 =
-- 1 -- I agree in principle, but I think Bob’s suggested new tex= t is too black/white and should be generalized.
 
Bob suggests:<= br> 
> The case where the service provider network arranges suff= icient capacity for a PW is well understood and
> will not be discuss= ed further here. The concern is over PWs deployed independently of the serv= ice provider
> network's traffic engineering or capacity planning. 
I want to include the case where capacity has been arranged, but= may not be sufficient, I suggest rewriting Bob’s second sentence as:=
 
The concern is over PWs that share network capacity with elas= tic or congestion-responsive traffic, independent of whether that sharing i= s a result of the service provider network's traffic engineering and capaci= ty planning or independent deployment of PWs by entities other than network= operators.
 
-- 2 -- Here, I disagree.  One of the signifi= cant contributions of this draft is that TDM PWs don’t cause congesti= on problems in practice, and one of the major reasons for that is the low l= atency requirements for acceptable TDM service delivery, both in ITU-T stan= dards and in actual TDM PW deployment.  The 125us behavior of native T= DM switches is relevant to the latter, and hence I prefer to include mentio= n of it.
 
-- 3 -- I agree; clear captions are always a feature,= not a bug ;-).
 
-- 4 -- I agree with changing “packet lo= ss rate” to “packet loss probability”.
 
-- 5 = -- I firmly disagree with Bob here, although he’s right that this is = “techno-political” ;-).
 
I am concerned that S-A-F-= E is likely to be a 4-letter word in wider Transport Area review, and hence= , I want to use a word that does not imply anything remotely resembling an = absolute guarantee that nothing can possibly ever go wrong (yes, I know Bob= didn’t mean to go to quite that extreme an interpretation, but S-A-F= -E is liable to be read that way by those with an “axe to grind”= ;), even though there’s lots of solid operational experience that sug= gests all is well here.
 
-- 6 -- I think I’m ok with what= Bob proposes to better characterize loss burstiness, but want to see speci= fic text.
 
-- Circuit Breaker Mechanism --
 
>>= ; [David] I suggest adding:
>>
>>    For = TDM pseudowires, such a circuit breaker can be realized via
>>&nbs= p;   operator monitoring of drop rates and consequent on the qual= ity of
>>    TDM service delivered and shutting dow= n TDM pseudowires that persistently
>>    fail to d= eliver acceptable TDM service (as defined by [G775] and
>> &n= bsp;  [G826]).  This document explains why a TDM PW that is causi= ng congestion
>>    problems via interference with = elastic traffic cannot be delivering
>>    acceptab= le TDM service (see the rest of this Section, plus Appendices
>>&n= bsp;   A and B).
>>
> [Bob] I don't think we shou= ld go there. We need to keep this draft firmly as informational. If someone=
> thinks a mechanism is needed because of the information in this dr= aft, then they can write about a mechanism.
 
[David] I firmly d= isagree, but Bob and I might be talking past each other, as I do want to st= ay informational ;-).
 
This text is intended to describe an exa= mple or possibility, not to specify an implementation requirement.  Th= e new text was intended as an OAM/OSS practice recommendation to the operat= or, not as a normative protocol spec.
 
I’m ok with insert= ing “example”, “possibility” or something similar i= f that helps w/Bob’s concern, but I strongly oppose doing nothing her= e, and note that this is also needed to respond to one of Stewart Bryant= 217;s WG LC comments.
 
-- PW segment size --
 
>&= gt; [David] Ignoring the rathole invitation ;-).  This is still in sec= tion 3, after:
>>
>>   We see that in general f= or large packet sizes, short delays, and low
>>   packet= loss rates, the TDM pseudowires consume much less bandwidth
>>&nb= sp;   than TCP would under identical conditions.  Only for s= mall packets,
>>    long delays, and high packet lo= ss ratios, do TDM PWs potentially
>>   consume more band= width, and even then only marginally.
>>
>> I suggest ad= ding:
>>
>>    Comparing figures 1 and 2, = use of a larger segment size for a T1 or E1
>>    T= DM PW moves the operational behavior of the PW away from the boundary
&g= t;>    at which the TCP throughput equation suggests that= undue interference with
>>    elastic traffic may = start to occur.  We suggest that larger segment
>>  = ;  sizes be preferred for T1 and E1 TDM PWs, but note that other
&g= t;>    considerations may favor smaller segment sizes (e.= g., jitter buffer
>>    size, bounding the impact o= f an isolated packet drop event).
>>
> [Bob] There is no nee= d to recommend any remedial action, when we have just shown there is not a = problem.
 
[David] Not exactly - Figure 1 shows that there is a = potential concern at a combination of the high end of the range of TDM PW l= atencies (4ms), and higher packet loss ratios. 

> Further, = the "marginal problem" only applies when the TCP traffic uses pac= kets smaller than 128B, which we only used
> as a hypothetical worst-= case for the analysis. But in practice this doesn't happen with long-runnin= g TCP flows -
> they use packets that are as large as possible (irres= pective of what the PW uses). So there really is /no/ problem to remedy.Ok, Bob can relax - his transport credentials are intact, as this is an en= try point to the bit-rate-congestible vs. packet-rate-congestible resource = rathole that I want to ignore :-).
 
I believe that larger TDM P= W segment sizes are a good thing for both this operating envelope boundary = concern as well as per-packet processing overheads in TDM PW functionality,= modulo other considerations, e.g., jitter buffers, and so I prefer to keep= this addition.
 
Thanks,
--David
 
From: B= ob Briscoe [mailto:bob.briscoe@bt.com= ]
Sent: Thursday, August 14, 2014 9:03 AM
To: Yaak= ov Stein
Cc: Andrew G. Malis; BOCCI Matthew; Black, David
S= ubject: RE: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt 
Yaakov,

I have just re-read it thoroughly. I have noticed= about two dozen editorial nits where correcting them would not change the = intended sense (altho sometimes the sense needs to change to say what we in= tended!). Examples:
- Unexpanded abbreviations (David's suggestion would= help, but there are more)
- Missing words, tautologies, random punctuat= ion problems etc
- Uses of the wrong word, even tho we know what you mea= n
- The caveat about dynamics vs steady state needs to shift down a para= , so that it follows into the para that says what to do about dynamics.
=
Should I take the token, and edit the XML directly, then you can see a = diff to check if you want to accept my suggestions?

Below are six th= oughts that go slightly beyond 'editorial'. 1-4 could be considered 'editor= ial'. But 5&6 are the more significant "techno-political" poi= nts.

1. The phrase "mechanisms for provisioning PWs in service= provider networks" is too opaque (the PWE audience might understand t= his scoping sentence, but a L4 reader won't understand what we're excluding= here, which will make them suspicious).
CURRENT:
   Mechan= isms for provisioning PWs
   in service provider networks are = well understood and will not be
   discussed further here.
= SUGGESTED:
   The case where the service provider network arra= nges sufficient capacity for a PW is well understood and will not be discus= sed further here. The concern is over PWs deployed independently of the ser= vice provider network's traffic engineering or capacity planning.

2.= The justification for using OWD of a few milliseconds misses the point."  The one-way delay
   of a native TDM service con= sists of the physical time-of-flight plus
   125 microseconds = for each TDM switch traversed; and is thus very
   small as co= mpared to typical PSN network-crossing latencies.
"
The 125us pe= r switch is irrelevant. The point that needs making is that PWs are general= ly deployed with a short physical time of flight.

3. Related point: = the captions on the graphs do not clearly indicate that they are OWD, not R= TT. This could be clarified in the text, or on the graphs.

4. Change= packet loss rate (and PLR) to packet loss probability throughout, includin= g on the graph axes. PLR is an ambiguous term, even though it is widely use= d in the industry. Packet loss rate could be measured in packet per second,= whereas packet loss probability is unambiguously dimensionless. Indeed, in= the appendices, packet loss rates with respect to time are calculated.
=
5. Rather than David's "reasonable" I suggest we use the phra= se "unambiguously safe" not just "safe" for the heading= s of Figs 5-8 and in the text, as below. Otherwise, we are implying that it= is unsafe outside the region.

CURRENT:
   One may view= this condition as defining an operating envelope for a
   TDM= PW, as a TDM PW that occupies no more bandwidth than a TCP flow
 &= nbsp; causes no more congestion than that TCP flow would.  Under this<= br>   condition it is safe to place the TDM PW along with congest= ion-
   responsive traffic such as TCP, without causing additi= onal
   congestion.  on the other hand, were the TDM PW t= o consume
   significantly more bandwidth a TCP flow, it could= contribute
   disproportionately to congestion, and its mixtu= re with congestion-
   responsive traffic might be inappropria= te.
SUGGESTED:
   One may view this condition as defining a= n unambiguously safe operating envelope for a
   TDM PW, as a = TDM PW that occupies no more bandwidth than a TCP flow
   caus= es no more congestion than that TCP flow would.  This does not draw us= into a debate over whether TCP-friendliness is a valid constraint; it simp= ly says that there can be no question that a PW is safe if it does no more = harm than a single TCP flow. Under this
   condition it is una= mbiguously safe to place the TDM PW along with congestion-
   = responsive traffic such as TCP.  On the other hand, were the TDM PW to= consume
   significantly more bandwidth a TCP flow, it could = contribute
   disproportionately to congestion, and its mixtur= e with congestion-
   responsive traffic might start to cause = concern.

6. The analysis in Appendices A & B needs to mention th= at loss bursts arise primarily from tail drop buffers, and the introduction= of active queue management (AQM) makes losses more independent of each oth= er (less bursty). (I have references showing evidence of this.) In the comp= arisons with TCP, we have assumed independent losses anyway, because they a= re the worst case (and fortunately they are easier to analyse). On a simila= r point, in appendix B, where we say "(bursty packet loss was also sim= ulated but is not reported here)", we ought to give a reason otherwise= readers will be suspicious. A good reason would be that uniformly random i= s the worst case.

I've also responded to a couple of David's suggest= ions inline (I agree with the others)...


At 17:44 13/08/2014, Ya= akov Stein wrote:
Bob told me on Friday of the IETF that he was going to= look over the final version and comment.
 
Bob – ping ? 
Y(J)S
 
From: Andrew G. Malis [ mailto:agmalis@gmail.com]
Sent: 13 A= ugust, 2014 17:02
To: Yaakov Stein
Cc: Bob Briscoe; rbriscoe@jungle.bt.co.uk; BOCC= I Matthew; Black, David
Subject: Re: [PWE3] WGLC comments on: dra= ft-ietf-pwe3-congcons-02.txt
 
Yaakov,
 
I still have= n't heard from Bob, but you should incorporate David's updates into a new r= evision, and I'll use that version for my final Shepherd's review.
 = ;
Thanks,
Andy
 
On Mon, Aug 11, 2014 at 11:16 AM, Andrew = G. Malis <agmalis@gmail.com>= wrote:
Actually, I'm going to end the LC as soon as I hear back from Bo= b on the IPR call. I sent him a reminder earlier this morning.
 Cheers,
Andy
 
On Sun, Aug 10, 2014 at 9:53 AM, Andrew G. Ma= lis <agmalis@gmail.com> wrot= e:
David,
 
That's perfect, thanks! I'm going to end the LC t= omorrow.
 
Cheers,
Andy
 
On Sat, Aug 9, 2014 at 1= 0:32 PM, Black, David <david.blac= k@emc.com> wrote:
Following up on this ...

> As we're n= ow moving quickly to try to complete this draft , I agreed
> w/Yaakov= that my author comments would be sent in as WG Last Call comments ...
&= gt; ... so here they are:
... with some suggested new text ...

&g= t; Major issues:
>
> The reference to a transport circuit break= er in section 3 is a bit optimistic
> and high level:
>
>=        We can accomplish this by
> = ;      employing a transport circuit breaker, by w= hich we mean an automatic
>       mecha= nism for terminating a flow to prevent negative impact on other
>&nbs= p;      flows and on the stability of the network<= br>>       [I-D.fairhurst-tsvwg-circuit-br= eaker].
>
> This should be accompanied by at least an example o= f how to do this - past
> discussion has suggested carrier monitoring= of delivered service quality
> in order to shut down PWs that are ex= periencing loss rates that result in
> unacceptable service quality (= as defined for us by the ITU ...).  I'd also
> like to add some = text to suggest that operators use this sort of circuit
> breaker pra= ctice when TDM PWs compete with elastic traffic.
After this text in Sect= ion 3:

   We can accomplish this by
   employ= ing a transport circuit breaker, by which we mean an automatic
 &nb= sp; mechanism for terminating a flow to prevent negative impact on other   flows and on the stability of the network
   [I-= D.fairhurst-tsvwg-circuit-breaker].  Note that a transport circuit
=    breaker is intended as a protection mechanism of last resort, = just as
   an electrical circuit breaker is only triggered whe= n absolutely
   necessary.

I suggest adding:

&nb= sp;  For TDM pseudowires, such a circuit breaker can be realized via   operator monitoring of drop rates and consequent on the qual= ity of
   TDM service delivered and shutting down TDM pseudowi= res that persistently
   fail to deliver acceptable TDM servic= e (as defined by [G775] and
   [G826]).  This document ex= plains why a TDM PW that is causing congestion
   problems via= interference with elastic traffic cannot be delivering
   acc= eptable TDM service (see the rest of this Section, plus Appendices
 = ;  A and B).

I don't think we should go there. We need to keep = this draft firmly as informational. If someone thinks a mechanism is needed= because of the information in this draft, then they can write about a mech= anism.


And breaking the paragraph after the added text.

&= gt; Figure 1 and Figure 2 indicate that based on using the TCP throughput> equation as a proxy for potential impact on elastic traffic (which i= s what
> we're doing in this draft), for the same fixed bandwidth PW,= a larger segment
> size is less likely to cause impacts.  I'd l= ike to add a suggestion to use
> the larger PW segment sizes as long = as they're consistent with delivery of
> the service.  There's b= een a related (long-to-the-point-of-ratholed) debate
> over in the Tr= ansport Area about whether packet-rate-limited/congestible
> forwardi= ng is still relevant.  My view is that such equipment exists, and
&= gt; hence that this is a relevant consideration for this draft, and would n= ot
> mention that issue here.
Ignoring the rathole invitation ;-).=   This is still in section 3, after:

   We see that i= n general for large packet sizes, short delays, and low
   pac= ket loss rates, the TDM pseudowires consume much less bandwidth
 &n= bsp; than TCP would under identical conditions.  Only for small packet= s,
   long delays, and high packet loss ratios, do TDM PWs pot= entially
   consume more bandwidth, and even then only margina= lly.

I suggest adding:

   Comparing figures 1 and 2= , use of a larger segment size for a T1 or E1
   TDM PW moves = the operational behavior of the PW away from the boundary
   a= t which the TCP throughput equation suggests that undue interference with   elastic traffic may start to occur.  We suggest that la= rger segment
   sizes be preferred for T1 and E1 TDM PWs, but = note that other
   considerations may favor smaller segment si= zes (e.g., jitter buffer
   size, bounding the impact of an is= olated packet drop event).

There is no need to recommend any remedia= l action, when we have just shown there is not a problem.

Further, t= he "marginal problem" only applies when the TCP traffic uses pack= ets smaller than 128B, which we only used as a hypothetical worst-case for = the analysis. But in practice this doesn't happen with long-running TCP flo= ws - they use packets that are as large as possible (irrespective of what t= he PW uses). So there really is /no/ problem to remedy.


and agai= n, breaking the paragraph after that added text.

I hope that latter = sentence ("We suggest that ...") is acceptable to the WG.

= > Minor issues:
>
> Section 2 is about Ethernet PWs that car= ry elastic traffic.  Should others
> be mentioned as also covere= d by this analysis - e.g., ATM and Frame Relay
> PWs carrying elastic= traffic?
After:

   In this section we consider Etherne= t PWs that primarily carry
   congestion-responsive traffic.&n= bsp; We show that we automatically obtain
   the desired conge= stion avoidance behavior, and that additional
   mechanisms ar= e not needed.

I suggest adding:

   This discussion = is applicable to any PW that carries congestion-
   responsive= traffic (e.g., ATM and Frame Relay PWs); Ethernet PWs are
   = used here for clarity, as they are a common example.

> Section 2 = does not cover Ethernet PWs carrying inelastic traffic - that would
>= be unusual, but should it be explicitly noted as out of scope, or just not=
> discussed (as is the case for the current text)?
I suggest doin= g nothing here, and planning to deal with this concern
if it should aris= e later (e.g., IETF Last Call or IESG Evaluation).

> Section 3 sh= ould say that the reason the draft focuses on TDM PWs is that
> they'= re the inelastic PWs that are likely to be carried over IP in practice.
= > My understanding is that other potentially inelastic PWs are MPLS-only=
> in practice.
After:

   Inelastic PWs, such as = TDM PWs ([RFC4553][RFC5086][RFC5087]), are
   potentially more= problematic than the elastic PWs of the previous
   section.&= nbsp; Being constant bit-rate (CBR), TDM PWs can not be made
  = ; responsive to congestion.  On the other hand, being CBR, they also d= o
   not attempt to capture additional bandwidth when neighbor= ing TCP
   flows back off.

I suggest adding:

&nb= sp;  We focus on TDM PWs here as they are the only inelastic PWs that = are
   likely to be IP-encapsulated in practice.  Most ot= her inelastic PWs
   tend to only be used with MPLS.

[I= 'd ask Yaakov and other PW experts to double-check the latter sentence,
= please.]

> A terminology section to roll up all the acronym expan= sions (e.g., TDM, CBR)
> in one place seems like a good idea.
Thes= e four acronyms would be good to put into a terminology section:

CBR= Constant Bit Rate
PSN Packet Switched Network
PW Pseudowire
TDM T= ime Division Multiplexing

>
> This text on p.11 could be in= cendiary:
>
>       Under this co= ndition it is safe to place the TDM PW along with
>   =     congestion - responsive traffic such as TCP, without cau= sing
>       additional congestion.
= >
> First of all to reduce the flammability, "safe" ->= ; "reasonable" and there
> are a number of other occurrence= s of "safe" that deserve similar treatment ;-
> ).
> = Beyond that, this assertion is dependent on the operating conditions stayin= g
> within that reasonable operating envelope, which entails use of s= omething
> like a circuit breaker - I'd say that here, and reference = the earlier
> circuit breaker discussion.
Global substitution: &qu= ot;safe" -> "reasonable", starting with the abstract.
=
Then for this specific text on p.11:

OLD
   &n= bsp;    Under this condition it is safe to place the TDM PW = along with
        congestion - respo= nsive traffic such as TCP, without causing
     = ;   additional congestion.
NEW
    &nbs= p;   Under this condition it is reasonable to place the TDM PW in= competition
      with congestion-responsive t= raffic such as TCP, as the result should not
    &nb= sp; produce unacceptable congestion effects.  That assumes that the op= erating
      conditions remain within the reas= onable operating envelope - as discussed
      = above, a transport circuit breaker is a suggested means of managing a TDM      PW so that it is shut down if operating co= nditions remain persistently
      outside the = reasonable operating envelope for the PW (e.g., due to a
  &nb= sp;   high packet loss rate).

No need to go there. See my = suggestions earlier.


> Figures 7 and 8 could use some shading= or lines to indicate the typical
> deployed operating regions for th= e pseudo-wires to visibly show that they
> fall under the curves, par= ticularly for larger segment sizes.  I'd also
> repeat or refer = to the suggestion to use larger

I wanted to suggest this myself, but= it is hard to depict fuzzy boundaries. If you can do this, pls do.

= Given the PW operating regions are bounded by v low loss probability, the x= axis in Figs 7&8 ought to be scaled much smaller as well.

Rela= ted point: the y axis in Fig 1 ought to max at 10, not 50.

Regards


Bob

>
> Nits:
>
> References aren= 't allowed in abstracts, so remove reference to
> [I-D.fairhurst-tsvw= g-circuit-breaker]
>
> Introduction, first paragraph, delete &q= uot;PWE3" towards end of paragraph -
>
>   = preferably the 4 byte PWE3 control word.
>
> In addition to it= s current general statements, the Security Considerations
> section s= hould point to the security considerations sections in a few
> partic= ularly relevant RFCs - I'd suggest 3985, 4553, 5086 and 5087.
After:
=
   This document does not introduce any new congestion-specif= ic
   mechanisms and thus does not introduce any new security<= br>   considerations above those present for PWs in general.
<= br>I suggest adding a new paragraph:

   For PW security co= nsiderations, see the security considerations sections
   of t= he cited RFCS; the security considerations in [RFC3985], [RFC4553],
&nbs= p;  [RFC5086] and [RFC5087] are particularly applicable to this docume= nt.

>
> Thanks,
> --David
>
>
> >= ; -----Original Message-----
> > From: pwe3 [ mailto:pwe3-bounces@ietf.org] On Behalf Of Yaakov = Stein
> > Sent: Thursday, July 24, 2014 4:52 PM
> > To: <= a href=3D"mailto:pwe3@ietf.org">pwe3@ietf.org
> > Subject: Re:= [PWE3] I-D Action: draft-ietf-pwe3-congcons-02.txt
> >
> &g= t; Please look at the PDF version only
> > (the txt version does n= ot have the graphs).
> >
> > Y(J)S
> >
> &= gt;
>
> _______________________________________________
>= pwe3 mailing list
> pwe3@ietf.org
> https://w= ww.ietf.org/mailman/listinfo/pwe3

______________________________= _________________
pwe3 mailing list
= pwe3@ietf.org
https://www.ietf.org/mailman/listinfo/pwe3
 
 
&nbs= p;

________________________________________________________________<= br>Bob Briscoe,          =             &nb= sp;            =             &nb= sp;  BT
_______________________________________________
pwe3 mai= ling list
pwe3@ietf.org
https://www.ietf.org/mailma= n/listinfo/pwe3

________________________________________________= ________________
Bob Briscoe,       &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;     BT

____________________________= ____________________________________
Bob Briscoe,    = ;            &n= bsp;            = ;            &n= bsp;        BT

= --_000_8D3D17ACE214DC429325B2B98F3AE712077BB426E5MX15Acorpemcc_-- From nobody Tue Aug 26 11:32:21 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 038EB1A0198 for ; Tue, 26 Aug 2014 11:32:18 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.568 X-Spam-Level: X-Spam-Status: No, score=-3.568 tagged_above=-999 required=5 tests=[BAYES_50=0.8, GB_I_INVITATION=-2, GB_I_LETTER=-2, GB_SUMOF=1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-0.668, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qCjEL8giip2P for ; Tue, 26 Aug 2014 11:32:04 -0700 (PDT) Received: from hubrelay-rd.bt.com (hubrelay-rd.bt.com [62.239.224.98]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0D20E1A0166 for ; Tue, 26 Aug 2014 11:32:02 -0700 (PDT) Received: from EVMHR02-UKBR.domain1.systemhost.net (193.113.108.41) by EVMHR65-UKRD.bt.com (10.187.101.20) with Microsoft SMTP Server (TLS) id 14.3.181.6; Tue, 26 Aug 2014 19:32:00 +0100 Received: from EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) by EVMHR02-UKBR.domain1.systemhost.net (193.113.108.41) with Microsoft SMTP Server (TLS) id 8.3.348.2; Tue, 26 Aug 2014 19:31:55 +0100 Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) with Microsoft SMTP Server id 14.3.181.6; Tue, 26 Aug 2014 19:31:50 +0100 Received: from BTP075694.jungle.bt.co.uk ([10.109.159.142]) by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id s7QIVgVr027059; Tue, 26 Aug 2014 19:31:43 +0100 Message-ID: <201408261831.s7QIVgVr027059@bagheera.jungle.bt.co.uk> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Tue, 26 Aug 2014 19:31:41 +0100 To: "Black, David" From: Bob Briscoe In-Reply-To: <8D3D17ACE214DC429325B2B98F3AE712077BB426E5@MX15A.corp.emc. com> References: <8D3D17ACE214DC429325B2B98F3AE712077B9517F4@MX15A.corp.emc.com> <8D3D17ACE214DC429325B2B98F3AE712077B951928@MX15A.corp.emc.com> <201408141302.s7ED2km0032238@bagheera.jungle.bt.co.uk> <8D3D17ACE214DC429325B2B98F3AE712077B952113@MX15A.corp.emc.com> <8D3D17ACE214DC429325B2B98F3AE712077B95212C@MX15A.corp.emc.com> <201408151006.s7FA6nWG003570@bagheera.jungle.bt.co.uk> <8D3D17ACE214DC429325B2B98F3AE712077BB42180@MX15A.corp.emc.com> <201408201835.s7KIZerA027879@bagheera.jungle.bt.co.uk> <8D3D17ACE214DC429325B2B98F3AE712077BB426E5@MX15A.corp.emc.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="=====================_1836403412==.ALT" X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158 Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/bWYxAs4ejMc3hejvcZWpjNeDnFo Cc: "pwe3@ietf.org" Subject: Re: [PWE3] FW: WGLC comments on: draft-ietf-pwe3-congcons-02.txt X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Aug 2014 18:32:18 -0000 --=====================_1836403412==.ALT Content-Type: text/plain; charset="iso-8859-1"; format=flowed Content-Transfer-Encoding: quoted-printable David, At 19:59 20/08/2014, Black, David wrote: >Bob, > >I think we=92re close - with luck, this email=20 >provides resolutions in principle to 5 and 7, >leaving 8, where I don=92t understand your argument. ...I'd like to hear from Yaakov before we start=20 declaring consensus - it's been just two of us so far! > >-- On 5: > > > I figured that the curves that delineate the=20 > region are defined by the TCP-friendly equation, so 'friendly region' says= on > > the tin what it does (and it is OK to imply=20 > that outside the region is unfriendly, because unfriendly doesn't imply > > "Don't go there" like 'unacceptable' would). > >=93TCP-friendly=94 and =93possibly TCP-unfriendly=94 would be fine with me= - >I=92m just uncomfortable with use of vanilla unqualified =93friendly.=94 I'm sold on TCP-friendly region. > >-- On 7, in reverse order: > > > In fact, on reflection, whatever we say about=20 > circuit-breakers, I'd prefer not to mention circuit-breakers in the > > abstract, which implies this draft says stuff=20 > about mechanism, which was never the intent. > >Removal of mention of circuit breakers from the abstract is fine with me. > > > If we're going to say a bit about how to=20 > realize a circuit-breaker, I'd be happier with the word persistently > > than without. I'd still rather we didn't=20 > allow this draft to mission creep into mechanism mode. > >I continue to agree with Stewart that something=20 >needs to be said here - the proposed >text leaves enormous flexibility to the operator=20 >to decide when to shut down TDM PWs. That's my point. A draft that isn't about=20 mechanism shouldn't tell a developer or an=20 operator what they can or can't do about=20 mechanism. If we want to talk about that, we will=20 need another draft (perhaps a PWE example in the=20 circuit-breakers draft, which has an intended status of Standards Track). >Here it is with =93persistently=94 restored: > > For TDM pseudowires, such a circuit breaker can be realized via > operator monitoring of drop rates and the quality of > TDM service delivered in order to shut down TDM pseudowires that > persistently fail to deliver acceptable TDM=20 > service (as defined by [G775] and > [G826]). This document explains why a TDM PW that is causing= congestion > problems via interference with elastic traffic is also not delivering > acceptable TDM service (see the rest of this Section, plus Appendices > A and B). I can live with this much text. But don't blame=20 me if this causes the draft to get referred back=20 or questioned as it goes through subsequent phases of the RFC process. > >-- On 8, I think I see the start of our disagreement: > > > On these graphs, the op behaviour plots won't=20 > move at all if the PW uses larger packets. These are > > bit-rate plots, so the only way to move=20 > anything is to change the TCP segment size (which moves the TCP plots). > >Actually, they=92re bit-rate plots on the=20 >assumption that both the PW and TCP are using the >same segment size, which is why Figure 2 looks=20 >better for the PWs than Figure 1 (the PW >lines did not move, but the TCP curves did move=20 >due to the segment size change). > >Or in other words: >- Figure 1 compares a PW using 64-byte segments=20 >to a TCP connection using 64-byte segments. >- Figure 2 compares a PW using 256-byte segments=20 >to a TCP connection using 256-byte segments. >- Nothing in the draft makes a comparison using different segment sizes. > >So, =93if the PW uses larger packets=94 then the=20 >approach in this draft compares it to a TCP >connection using larger segment, and the result=20 >looks like Figure 2 instead of Figure 1. > >I do not like the idea of comparing a PW using=20 >256-byte segments with a TCP connection >using 64-byte segments, which is what I=20 >understand that above =93won=92t move at all=94 assertion >to be about. The TCP curves do move between=20 >Figure 1 and Figure 2, based solely on a TCP >segment size change. Correct. We're getting somewhere. The draft=20 actually says two sentences about this assumption: " In order to quantitatively compare TDM PWs to TCP flows, we will compare the effect of TDM PW packets with that of TCP packets of the same packet size and sent at the same rate. This is potentially an overly pessimistic comparison, as TDM PW packets are frequently configured to be short in order to minimize latency, while TCP packets are free to be much larger. " You are focusing solely on the first sentence,=20 which we only used to simplify presentation of the analysis. I am saying any recommendations should be based=20 on the analysis /after/ having adjusted the=20 outcome to reverse out any assumptions that=20 otherwise make the analysis unrealistic (ie. take=20 account of the second quoted sentence). But my case is even stronger than that: a) Even if we solely make the assumption of=20 comparable packet sizes (for the sake of=20 analysis) and ignore the second qualifying=20 sentence (the reality), then as we continue=20 through the whole doc we still get to the point=20 where we show the problem is virtually non-existent. Given: * TCP-friendliness is not an absolute=20 requirement - one bit-rate within half-a-dozen=20 times the other is fine (I recall Sally FLoyd=20 writing somewhere that x8 shouldn't be a problem) * the overlap only occurs because we've drawn=20 the upper bound of loss probability for a PWE=20 conservatively high (Appendix A says this). So we don't want to then move straight into=20 recommending a solution in the following=20 paragraph, which makes it look like we don't=20 believe our own conclusion - that PWEs are not a=20 significant capacity sharing problem - even when=20 both PWE and TCP use the same size packets. b) If we then take note of the second qualifying=20 sentence above, it really does make this an=20 unequivocal non-problem. So we really shouldn't=20 be proposing that operators change anything. c) Actually, if we did state your recommendation,=20 it wouldn't be correct to say the analysis=20 supports it, because packet size problems require=20 consideration of packet-congestion, not just=20 bit-congestion. So we would have to repeat the=20 full analysis for packet-congestion, not=20 bit-congestion, and show packet-rate graphs as=20 well as bit-rate graphs. Having done this, and=20 having similarly taken points (a) and (b) into=20 account, we would still come to the same=20 conclusion - that there is not a problem=20 significant enough to need to recommend any action. So, for all these reasons, let's not make this recommendation. Bob > >Thanks, >--David > >From: Bob Briscoe [mailto:bob.briscoe@bt.com] >Sent: Wednesday, August 20, 2014 2:36 PM >To: Black, David >Cc: pwe3@ietf.org >Subject: RE: [PWE3] FW: WGLC comments on: draft-ietf-pwe3-congcons-02.txt > >David, > >Sry for delay. inline... > >(Despite end of WGLC, we can still continue our=20 >public intra-author ping-pong :) > >At 13:44 18/08/2014, Black, David wrote: > >Bob, > >1-4 & 6: ok > >5: =93acceptable=94 is a good replacement for=20 >S-A-F-E, but I=92m not sure about =93friendly=94. The=20 >key concept here is assurance that there are no=20 >problems. How about =93clearly acceptable=94? That=20 >may need some additional text to say that=20 >acceptable network and service behavior may be=20 >delivered outside the clearly acceptable region,=20 >but one cannot rely on the analysis in this=20 >draft (based on the TCP throughput formula) to reach such a conclusion. > >I wanted a different word from 'acceptable' for=20 >the adjective in the captions above the Figures=20 >describing the regions, because there's no=20 >opportunity for more text there to explain that=20 >we do not mean that outside the region is=20 >unacceptable. I thought about various qualifiers=20 >with acceptable (like clearly, unarguably, etc),=20 >but I don't think these are right either. > >I figured that the curves that delineate the=20 >region are defined by the TCP-friendly equation,=20 >so 'friendly region' says on the tin what it=20 >does (and it is OK to imply that outside the=20 >region is unfriendly, because unfriendly doesn't=20 >imply "Don't go there" like 'unacceptable' would). > >Let's stop batting back & forth until we see what Yaakov (or others) think. > > > >7: I disagree - the tsvwg circuit breaker draft=20 >is rather general, and we now have two WG LC=20 >comments on this draft asking how to apply a=20 >circuit breaker to PWs. I think we=92re on solid=20 >ground with the references to the ITU-T service=20 >quality standards and expecting operator=20 >monitoring of service quality. OTOH, the word=20 >=93persistently=94 seemed to be a concern, and I=20 >have no problem removing it (anyone who asks=20 >about that can be directed to G.775 and G.826). New suggested text: > > For TDM pseudowires, such a circuit breaker can be realized via > operator monitoring of drop rates and the quality of > TDM service delivered in order to shut down TDM pseudowires that > fail to deliver acceptable TDM service (as defined by [G775] and > [G826]). This document explains why a TDM PW that is causing= congestion > problems via interference with elastic traffic is also not delivering > acceptable TDM service (see the rest of this Section, plus Appendices > A and B). > >If we're going to say a bit about how to realize=20 >a circuit-breaker, I'd be happier with the word=20 >persistently than without. I'd still rather we=20 >didn't allow this draft to mission creep into mechanism mode. > >In fact, on reflection, whatever we say about=20 >circuit-breakers, I'd prefer not to mention=20 >circuit-breakers in the abstract, which implies=20 >this draft says stuff about mechanism, which was never the intent. > > > >8: I believe that figures 1 and 2 suffice to=20 >make the point, but I see that a recommendation=20 >may be a bit of a reach for this draft. How=20 >about the following modified form of my first=20 >sentence, which just states the inference that=20 >can be drawn from Figures 1 and 2? - > > Figures 1 and 2 indicate that a larger segment size for a T1 or E1 > TDM PW moves the operational behavior of the PW away from the boundary > at which the TCP throughput equation suggests that undue interference= with > elastic traffic may start to occur. > >Not so - that's the problem. On these graphs,=20 >the op behaviour plots won't move at all if the=20 >PW uses larger packets. These are bit-rate=20 >plots, so the only way to move anything is to=20 >change the TCP segment size (which moves the TCP=20 >plots). To justify what you want to say, either: >a) these would have to be packet-rate plots, or >b) we would have to turn the assumption we've=20 >made about equal packet sizes into an=20 >operational requirement, not just an analytical assumption. > >To go in either of these directions would require a spade and vermin. > > >Bob > >PS. GOing offline until next Tue. Might check email, but no guarantee. > > > >Thanks, >--David > >From: Bob Briscoe [ mailto:bob.briscoe@bt.com] >Sent: Friday, August 15, 2014 6:03 AM >To: Black, David >Cc: pwe3@ietf.org >Subject: Re: [PWE3] FW: WGLC comments on: draft-ietf-pwe3-congcons-02.txt > >David, > >I decided to think on a better word for 'safe'=20 >overnight, and allow others to comment. But=20 >no-one has, so here's my responses to all your outstanding points... > >I've provided slightly updated xml source of my=20 >suggested edits (and a diff) here: ><=20 >http://www.bobbriscoe.net/projects/2020comms/pwe3-= congcons/draft-ietf-pwe3-congcons-DIFF03b-02.html =20 > > ><=20 >http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-pwe3-= congcons-03b.txt =20 > > ><=20 >http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons= /draft-ietf-pwe3-congcons-03b.xml =20 > > >Again note: xml is purely for precision, not=20 >implying I have taken over editorship. > >1. Happy with your new scoping text, except the=20 >the first 'independent' clangs with the second, and it's unnecessary= anyway: >" The concern is over PWs that share network=20 >capacity with elastic or congestion-responsive=20 >traffic, [independent of] whether that sharing=20 >is a result of the service provider network's=20 >traffic engineering and capacity planning or=20 >independent deployment of PWs by entities other than network operators. >" > >2. My point wasn't to remove the reference to=20 >the 125us (in my suggested text I didn't). It=20 >was that the 125us stuff /alone/ doesn't warrant=20 >the 'thus' in the next clause; the logic needs=20 >to be connected by talking about the sum of the=20 >flight time /and/ the per-node processing time. >CURRENT: > The one-way delay > of a native TDM service consists of the physical time-of-flight plus > 125 microseconds for each TDM switch traversed; and is thus very > small as compared to typical PSN network-crossing latencies. >SUGGESTED: > The one-way delay > of a native TDM service consists of the physical time-of-flight plus > 125 microseconds for each TDM switch=20 > traversed; so the total one-way delay for most=20 > TDM PWs has to be small compared to typical PSN network-crossing= latencies. > >5. I understand and agree - 'S-A-F-E region' is=20 >incendiary. But I don't like 'reasonable region'=20 >- it's slightly /too/ meaningless. > >Other possibilities are: >- admissable region (problem: implies outside the region is inadmissable) >- harmonious region (problem: too Californian) >So I suggest two words: >- friendly or acceptable, each used depending on context. > >On the graphs, I suggest 'operating region=20 >(friendly under curve)' because that is=20 >precisely where the curves come from -=20 >'friendly' does what it says on the tin, and the=20 >implied 'unfriendly' above the curve correctly=20 >avoids implying that the sky will fall. > >Elsewhere, I suggest the following replacements for the S-word: >"It is thus worthwhile specifying under what=20 >conditions such competition is acceptable" >"One may view this condition as defining a=20 >'friendly' operating envelope for a TDM PW" >"Under this condition it is acceptable to place=20 >the TDM PW along with ... TCP. " >"comparison with something widely considered to=20 >offer an acceptable response to congestion" > > >7. Circuit breaker >Scope creep. Rat-hole. Stop digging. >Yes, I know Stewart has asked us to dig out=20 >another rat as well, but the appropriate response is "Out of scope". > >You might think your text is uncontroversial,=20 >but if we are tempted to take Stewart's bait=20 >(rat metaphor intended), it will probably start=20 >new debates that this doc doesn't deserve: "What=20 >does persistent mean?", "Which traffic is more=20 >important/ valuable?", yada yada. > >I suggest the text merely makes it clear that=20 >this debate is in scope for elsewhere, not here.=20 >I've also just realised that Yaakov has written=20 >the sentences around circuit breaker with the=20 >wrong logical sequence. The circuit breaker=20 >needs to be introduced as something the PW uses=20 >when its /own/ quality has failed, which should=20 >avoid any need to shut down because it is threatening elastic services: > >CURRENT: > If the bandwidth consumed by a TDM PW is > considered detrimental, the only available remedy is to completely > shut down the PW, by using a transport circuit breaker mechanism. > However, we will show that even before such an action is > warranted, the PW will become unable to faithfully emulate the > native TDM service; for example, when a TDM service is carrying > voice grade telephony channels, the voice quality will degrade to > below useful levels. >SUGGESTED: > We will show that, before a PW becomes=20 > deterimental to elastic traffic, it will become=20 > unable to faithfully emulate the > native TDM service; for example, when a TDM service is carrying > voice grade telephony channels, the voice quality will degrade to > below useful levels. Once the quality of=20 > the emulated service is persistently=20 > unacceptable, if a PW shuts down selected flows=20 > or shuts itself down completely, it will=20 > automatically avoid detriment to elastic=20 > traffic. A transport circuit breaker=20 > [I-D.fairhurst-tsvwg-circuit-breaker] could be=20 > used, but mechanism is beyond the scope of the present document. > > >8. Segment size. >Everyone's in a big group-hug because we've=20 >written a draft showing there's is no need to=20 >change anything. Then just at the end of an=20 >uneventful WGLC, we hear David's voice from the=20 >bottom of a deep hole shouting, "avoid the=20 >rat-hole invitation", while furiously digging for a rat. > >There is no need to add a recommendation that=20 >PWs need to be changed to use larger packets. No=20 >matter how hedged and soft this recommendation=20 >would be, it's not needed anyway. Stop digging. > >Yaakov's new text is fine as it stands. "Only=20 >for small packets, long delays, and high packet=20 >loss ratios, do TDM PWs potentially consume more=20 >bandwidth, and even then only marginally. " > >In order to justify your sentence, we would have=20 >to include graphs with packet-rate (not=20 >bit-rate) on the vertical axis and go into a=20 >whole treatise as to why packet-rate comparison=20 >might be important as well. Then the graph would=20 >show one plot for all TCP segment sizes and=20 >multiple plots for PWs with different packet=20 >sizes. Then we'd see that the corner of the two=20 >smallest packet PWs just clipped the TCP curve=20 >at loss probabilities that are stupidly large=20 >anyway. So even after all this, it would hardly=20 >justify a need for your sentence. > >All that digging would not be worth it, just to=20 >discover (again) that there is no rat. > > >Bob > > >At 19:39 14/08/2014, Black, David wrote: > >Content-Language: en-US >Content-Type: multipart/alternative; >=20 >boundary=3D"_000_8D3D17ACE214DC429325B2B98F3AE712077B95212CMX15Acorpemcc_" > >And forwarding my views to the list ... > >... On Bob=92s six thoughts (I have problems with=20 >2 & 5 - the other 4 are basically ok w/me) plus=20 >follow-ups on Bob=92s concerns w/my two major issues, here goes: > >-- 1 -- I agree in principle, but I think Bob=92s=20 >suggested new text is too black/white and should be generalized. > >Bob suggests: > > > The case where the service provider network=20 > arranges sufficient capacity for a PW is well understood and > > will not be discussed further here. The=20 > concern is over PWs deployed independently of the service provider > > network's traffic engineering or capacity planning. > >I want to include the case where capacity has=20 >been arranged, but may not be sufficient, I=20 >suggest rewriting Bob=92s second sentence as: > >The concern is over PWs that share network=20 >capacity with elastic or congestion-responsive=20 >traffic, independent of whether that sharing is=20 >a result of the service provider network's=20 >traffic engineering and capacity planning or=20 >independent deployment of PWs by entities other than network operators. > >-- 2 -- Here, I disagree. One of the=20 >significant contributions of this draft is that=20 >TDM PWs don=92t cause congestion problems in=20 >practice, and one of the major reasons for that=20 >is the low latency requirements for acceptable=20 >TDM service delivery, both in ITU-T standards=20 >and in actual TDM PW deployment. The 125us=20 >behavior of native TDM switches is relevant to=20 >the latter, and hence I prefer to include mention of it. > >-- 3 -- I agree; clear captions are always a feature, not a bug ;-). > >-- 4 -- I agree with changing =93packet loss rate=94 to =93packet loss= probability=94. > >-- 5 -- I firmly disagree with Bob here,=20 >although he=92s right that this is =93techno-political=94 ;-). > >I am concerned that S-A-F-E is likely to be a=20 >4-letter word in wider Transport Area review,=20 >and hence, I want to use a word that does not=20 >imply anything remotely resembling an absolute=20 >guarantee that nothing can possibly ever go=20 >wrong (yes, I know Bob didn=92t mean to go to=20 >quite that extreme an interpretation, but=20 >S-A-F-E is liable to be read that way by those=20 >with an =93axe to grind=94), even though there=92s=20 >lots of solid operational experience that suggests all is well here. > >-- 6 -- I think I=92m ok with what Bob proposes to=20 >better characterize loss burstiness, but want to see specific text. > >-- Circuit Breaker Mechanism -- > > >> [David] I suggest adding: > >> > >> For TDM pseudowires, such a circuit breaker can be realized via > >> operator monitoring of drop rates and consequent on the quality of > >> TDM service delivered and shutting down=20 > TDM pseudowires that persistently > >> fail to deliver acceptable TDM service (as defined by [G775] and > >> [G826]). This document explains why a=20 > TDM PW that is causing congestion > >> problems via interference with elastic traffic cannot be delivering > >> acceptable TDM service (see the rest of this Section, plus= Appendices > >> A and B). > >> > > [Bob] I don't think we should go there. We=20 > need to keep this draft firmly as informational. If someone > > thinks a mechanism is needed because of the=20 > information in this draft, then they can write about a mechanism. > >[David] I firmly disagree, but Bob and I might=20 >be talking past each other, as I do want to stay informational ;-). > >This text is intended to describe an example or=20 >possibility, not to specify an implementation=20 >requirement. The new text was intended as an=20 >OAM/OSS practice recommendation to the operator,=20 >not as a normative protocol spec. > >I=92m ok with inserting =93example=94, =93possibility=94=20 >or something similar if that helps w/Bob=92s=20 >concern, but I strongly oppose doing nothing=20 >here, and note that this is also needed to=20 >respond to one of Stewart Bryant=92s WG LC comments. > >-- PW segment size -- > > >> [David] Ignoring the rathole invitation=20 > ;-). This is still in section 3, after: > >> > >> We see that in general for large packet sizes, short delays, and low > >> packet loss rates, the TDM pseudowires consume much less bandwidth > >> than TCP would under identical conditions. Only for small packets, > >> long delays, and high packet loss ratios, do TDM PWs potentially > >> consume more bandwidth, and even then only marginally. > >> > >> I suggest adding: > >> > >> Comparing figures 1 and 2, use of a larger segment size for a T1 or= E1 > >> TDM PW moves the operational behavior of the PW away from the= boundary > >> at which the TCP throughput equation=20 > suggests that undue interference with > >> elastic traffic may start to occur. We suggest that larger segment > >> sizes be preferred for T1 and E1 TDM PWs, but note that other > >> considerations may favor smaller segment sizes (e.g., jitter buffer > >> size, bounding the impact of an isolated packet drop event). > >> > > [Bob] There is no need to recommend any=20 > remedial action, when we have just shown there is not a problem. > >[David] Not exactly - Figure 1 shows that there=20 >is a potential concern at a combination of the=20 >high end of the range of TDM PW latencies (4ms),=20 >and higher packet loss ratios. > > > Further, the "marginal problem" only applies=20 > when the TCP traffic uses packets smaller than 128B, which we only used > > as a hypothetical worst-case for the=20 > analysis. But in practice this doesn't happen with long-running TCP flows= - > > they use packets that are as large as=20 > possible (irrespective of what the PW uses). So=20 > there really is /no/ problem to remedy. >Ok, Bob can relax - his transport credentials=20 >are intact, as this is an entry point to the=20 >bit-rate-congestible vs. packet-rate-congestible=20 >resource rathole that I want to ignore :-). > >I believe that larger TDM PW segment sizes are a=20 >good thing for both this operating envelope=20 >boundary concern as well as per-packet=20 >processing overheads in TDM PW functionality,=20 >modulo other considerations, e.g., jitter=20 >buffers, and so I prefer to keep this addition. > >Thanks, >--David > >From: Bob Briscoe [mailto:bob.briscoe@bt.com] >Sent: Thursday, August 14, 2014 9:03 AM >To: Yaakov Stein >Cc: Andrew G. Malis; BOCCI Matthew; Black, David >Subject: RE: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt > >Yaakov, > >I have just re-read it thoroughly. I have=20 >noticed about two dozen editorial nits where=20 >correcting them would not change the intended=20 >sense (altho sometimes the sense needs to change=20 >to say what we intended!). Examples: >- Unexpanded abbreviations (David's suggestion would help, but there are= more) >- Missing words, tautologies, random punctuation problems etc >- Uses of the wrong word, even tho we know what you mean >- The caveat about dynamics vs steady state=20 >needs to shift down a para, so that it follows=20 >into the para that says what to do about dynamics. > >Should I take the token, and edit the XML=20 >directly, then you can see a diff to check if=20 >you want to accept my suggestions? > >Below are six thoughts that go slightly beyond=20 >'editorial'. 1-4 could be considered=20 >'editorial'. But 5&6 are the more significant "techno-political" points. > >1. The phrase "mechanisms for provisioning PWs=20 >in service provider networks" is too opaque (the=20 >PWE audience might understand this scoping=20 >sentence, but a L4 reader won't understand what=20 >we're excluding here, which will make them suspicious). >CURRENT: > Mechanisms for provisioning PWs > in service provider networks are well understood and will not be > discussed further here. >SUGGESTED: > The case where the service provider network=20 > arranges sufficient capacity for a PW is well=20 > understood and will not be discussed further=20 > here. The concern is over PWs deployed=20 > independently of the service provider network's=20 > traffic engineering or capacity planning. > >2. The justification for using OWD of a few milliseconds misses the point. >" The one-way delay > of a native TDM service consists of the physical time-of-flight plus > 125 microseconds for each TDM switch traversed; and is thus very > small as compared to typical PSN network-crossing latencies. >" >The 125us per switch is irrelevant. The point=20 >that needs making is that PWs are generally=20 >deployed with a short physical time of flight. > >3. Related point: the captions on the graphs do=20 >not clearly indicate that they are OWD, not RTT.=20 >This could be clarified in the text, or on the graphs. > >4. Change packet loss rate (and PLR) to packet=20 >loss probability throughout, including on the=20 >graph axes. PLR is an ambiguous term, even=20 >though it is widely used in the industry. Packet=20 >loss rate could be measured in packet per=20 >second, whereas packet loss probability is=20 >unambiguously dimensionless. Indeed, in the=20 >appendices, packet loss rates with respect to time are calculated. > >5. Rather than David's "reasonable" I suggest we=20 >use the phrase "unambiguously safe" not just=20 >"safe" for the headings of Figs 5-8 and in the=20 >text, as below. Otherwise, we are implying that=20 >it is unsafe outside the region. > >CURRENT: > One may view this condition as defining an operating envelope for a > TDM PW, as a TDM PW that occupies no more bandwidth than a TCP flow > causes no more congestion than that TCP flow would. Under this > condition it is safe to place the TDM PW along with congestion- > responsive traffic such as TCP, without causing additional > congestion. on the other hand, were the TDM PW to consume > significantly more bandwidth a TCP flow, it could contribute > disproportionately to congestion, and its mixture with congestion- > responsive traffic might be inappropriate. >SUGGESTED: > One may view this condition as defining an=20 > unambiguously safe operating envelope for a > TDM PW, as a TDM PW that occupies no more bandwidth than a TCP flow > causes no more congestion than that TCP flow=20 > would. This does not draw us into a debate=20 > over whether TCP-friendliness is a valid=20 > constraint; it simply says that there can be no=20 > question that a PW is safe if it does no more=20 > harm than a single TCP flow. Under this > condition it is unambiguously safe to place=20 > the TDM PW along with congestion- > responsive traffic such as TCP. On the=20 > other hand, were the TDM PW to consume > significantly more bandwidth a TCP flow, it could contribute > disproportionately to congestion, and its mixture with congestion- > responsive traffic might start to cause concern. > >6. The analysis in Appendices A & B needs to=20 >mention that loss bursts arise primarily from=20 >tail drop buffers, and the introduction of=20 >active queue management (AQM) makes losses more=20 >independent of each other (less bursty). (I have=20 >references showing evidence of this.) In the=20 >comparisons with TCP, we have assumed=20 >independent losses anyway, because they are the=20 >worst case (and fortunately they are easier to=20 >analyse). On a similar point, in appendix B,=20 >where we say "(bursty packet loss was also=20 >simulated but is not reported here)", we ought=20 >to give a reason otherwise readers will be=20 >suspicious. A good reason would be that uniformly random is the worst case. > >I've also responded to a couple of David's=20 >suggestions inline (I agree with the others)... > > >At 17:44 13/08/2014, Yaakov Stein wrote: >Bob told me on Friday of the IETF that he was=20 >going to look over the final version and comment. > >Bob =96 ping ? > >Y(J)S > >From: Andrew G. Malis [ mailto:agmalis@gmail.com] >Sent: 13 August, 2014 17:02 >To: Yaakov Stein >Cc: Bob Briscoe;=20 >rbriscoe@jungle.bt.co.uk;=20 >BOCCI Matthew; Black, David >Subject: Re: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt > >Yaakov, > >I still haven't heard from Bob, but you should=20 >incorporate David's updates into a new revision,=20 >and I'll use that version for my final Shepherd's review. > >Thanks, >Andy > >On Mon, Aug 11, 2014 at 11:16 AM, Andrew G.=20 >Malis <agmalis@gmail.com> wrote: >Actually, I'm going to end the LC as soon as I=20 >hear back from Bob on the IPR call. I sent him a reminder earlier this= morning. > >Cheers, >Andy > >On Sun, Aug 10, 2014 at 9:53 AM, Andrew G. Malis=20 ><agmalis@gmail.com> wrote: >David, > >That's perfect, thanks! I'm going to end the LC tomorrow. > >Cheers, >Andy > >On Sat, Aug 9, 2014 at 10:32 PM, Black, David=20 ><david.black@emc.com> wrote: >Following up on this ... > > > As we're now moving quickly to try to complete this draft , I agreed > > w/Yaakov that my author comments would be=20 > sent in as WG Last Call comments ... > > ... so here they are: >... with some suggested new text ... > > > Major issues: > > > > The reference to a transport circuit breaker=20 > in section 3 is a bit optimistic > > and high level: > > > > We can accomplish this by > > employing a transport circuit breaker, by which we mean an= automatic > > mechanism for terminating a flow to prevent negative impact on= other > > flows and on the stability of the network > > [I-D.fairhurst-tsvwg-circuit-breaker]. > > > > This should be accompanied by at least an example of how to do this -= past > > discussion has suggested carrier monitoring of delivered service quality > > in order to shut down PWs that are experiencing loss rates that result= in > > unacceptable service quality (as defined for us by the ITU ...). I'd= also > > like to add some text to suggest that operators use this sort of circuit > > breaker practice when TDM PWs compete with elastic traffic. >After this text in Section 3: > > We can accomplish this by > employing a transport circuit breaker, by which we mean an automatic > mechanism for terminating a flow to prevent negative impact on other > flows and on the stability of the network > [I-D.fairhurst-tsvwg-circuit-breaker]. Note that a transport circuit > breaker is intended as a protection mechanism of last resort, just as > an electrical circuit breaker is only triggered when absolutely > necessary. > >I suggest adding: > > For TDM pseudowires, such a circuit breaker can be realized via > operator monitoring of drop rates and consequent on the quality of > TDM service delivered and shutting down TDM pseudowires that= persistently > fail to deliver acceptable TDM service (as defined by [G775] and > [G826]). This document explains why a TDM PW that is causing= congestion > problems via interference with elastic traffic cannot be delivering > acceptable TDM service (see the rest of this Section, plus Appendices > A and B). > >I don't think we should go there. We need to=20 >keep this draft firmly as informational. If=20 >someone thinks a mechanism is needed because of=20 >the information in this draft, then they can write about a mechanism. > > >And breaking the paragraph after the added text. > > > Figure 1 and Figure 2 indicate that based on using the TCP throughput > > equation as a proxy for potential impact on elastic traffic (which is= what > > we're doing in this draft), for the same=20 > fixed bandwidth PW, a larger segment > > size is less likely to cause impacts. I'd like to add a suggestion to= use > > the larger PW segment sizes as long as they're consistent with delivery= of > > the service. There's been a related (long-to-the-point-of-ratholed)= debate > > over in the Transport Area about whether packet-rate-limited/congestible > > forwarding is still relevant. My view is that such equipment exists,= and > > hence that this is a relevant consideration for this draft, and would= not > > mention that issue here. >Ignoring the rathole invitation ;-). This is still in section 3, after: > > We see that in general for large packet sizes, short delays, and low > packet loss rates, the TDM pseudowires consume much less bandwidth > than TCP would under identical conditions. Only for small packets, > long delays, and high packet loss ratios, do TDM PWs potentially > consume more bandwidth, and even then only marginally. > >I suggest adding: > > Comparing figures 1 and 2, use of a larger segment size for a T1 or E1 > TDM PW moves the operational behavior of the PW away from the boundary > at which the TCP throughput equation suggests that undue interference= with > elastic traffic may start to occur. We suggest that larger segment > sizes be preferred for T1 and E1 TDM PWs, but note that other > considerations may favor smaller segment sizes (e.g., jitter buffer > size, bounding the impact of an isolated packet drop event). > >There is no need to recommend any remedial=20 >action, when we have just shown there is not a problem. > >Further, the "marginal problem" only applies=20 >when the TCP traffic uses packets smaller than=20 >128B, which we only used as a hypothetical=20 >worst-case for the analysis. But in practice=20 >this doesn't happen with long-running TCP flows=20 >- they use packets that are as large as possible=20 >(irrespective of what the PW uses). So there really is /no/ problem to= remedy. > > >and again, breaking the paragraph after that added text. > >I hope that latter sentence ("We suggest that ...") is acceptable to the= WG. > > > Minor issues: > > > > Section 2 is about Ethernet PWs that carry elastic traffic. Should= others > > be mentioned as also covered by this analysis - e.g., ATM and Frame= Relay > > PWs carrying elastic traffic? >After: > > In this section we consider Ethernet PWs that primarily carry > congestion-responsive traffic. We show that we automatically obtain > the desired congestion avoidance behavior, and that additional > mechanisms are not needed. > >I suggest adding: > > This discussion is applicable to any PW that carries congestion- > responsive traffic (e.g., ATM and Frame Relay PWs); Ethernet PWs are > used here for clarity, as they are a common example. > > > Section 2 does not cover Ethernet PWs=20 > carrying inelastic traffic - that would > > be unusual, but should it be explicitly noted as out of scope, or just= not > > discussed (as is the case for the current text)? >I suggest doing nothing here, and planning to deal with this concern >if it should arise later (e.g., IETF Last Call or IESG Evaluation). > > > Section 3 should say that the reason the draft focuses on TDM PWs is= that > > they're the inelastic PWs that are likely to=20 > be carried over IP in practice. > > My understanding is that other potentially inelastic PWs are MPLS-only > > in practice. >After: > > Inelastic PWs, such as TDM PWs ([RFC4553][RFC5086][RFC5087]), are > potentially more problematic than the elastic PWs of the previous > section. Being constant bit-rate (CBR), TDM PWs can not be made > responsive to congestion. On the other hand, being CBR, they also do > not attempt to capture additional bandwidth when neighboring TCP > flows back off. > >I suggest adding: > > We focus on TDM PWs here as they are the only inelastic PWs that are > likely to be IP-encapsulated in practice. Most other inelastic PWs > tend to only be used with MPLS. > >[I'd ask Yaakov and other PW experts to double-check the latter sentence, >please.] > > > A terminology section to roll up all the=20 > acronym expansions (e.g., TDM, CBR) > > in one place seems like a good idea. >These four acronyms would be good to put into a terminology section: > >CBR Constant Bit Rate >PSN Packet Switched Network >PW Pseudowire >TDM Time Division Multiplexing > > > > > This text on p.11 could be incendiary: > > > > Under this condition it is safe to place the TDM PW along with > > congestion - responsive traffic such as TCP, without causing > > additional congestion. > > > > First of all to reduce the flammability, "safe" -> "reasonable" and= there > > are a number of other occurrences of "safe"=20 > that deserve similar treatment ;- > > ). > > Beyond that, this assertion is dependent on=20 > the operating conditions staying > > within that reasonable operating envelope, which entails use of= something > > like a circuit breaker - I'd say that here, and reference the earlier > > circuit breaker discussion. >Global substitution: "safe" -> "reasonable", starting with the abstract. > >Then for this specific text on p.11: > >OLD > Under this condition it is safe to place the TDM PW along with > congestion - responsive traffic such as TCP, without causing > additional congestion. >NEW > Under this condition it is reasonable=20 > to place the TDM PW in competition > with congestion-responsive traffic such=20 > as TCP, as the result should not > produce unacceptable congestion=20 > effects. That assumes that the operating > conditions remain within the reasonable=20 > operating envelope - as discussed > above, a transport circuit breaker is a=20 > suggested means of managing a TDM > PW so that it is shut down if operating conditions remain= persistently > outside the reasonable operating envelope for the PW (e.g., due to a > high packet loss rate). > >No need to go there. See my suggestions earlier. > > > > Figures 7 and 8 could use some shading or lines to indicate the typical > > deployed operating regions for the pseudo-wires to visibly show that= they > > fall under the curves, particularly for larger segment sizes. I'd also > > repeat or refer to the suggestion to use larger > >I wanted to suggest this myself, but it is hard=20 >to depict fuzzy boundaries. If you can do this, pls do. > >Given the PW operating regions are bounded by v=20 >low loss probability, the x axis in Figs 7&8=20 >ought to be scaled much smaller as well. > >Related point: the y axis in Fig 1 ought to max at 10, not 50. > >Regards > > > >Bob > > > > > Nits: > > > > References aren't allowed in abstracts, so remove reference to > > [I-D.fairhurst-tsvwg-circuit-breaker] > > > > Introduction, first paragraph, delete "PWE3" towards end of paragraph - > > > > preferably the 4 byte PWE3 control word. > > > > In addition to its current general statements, the Security= Considerations > > section should point to the security considerations sections in a few > > particularly relevant RFCs - I'd suggest 3985, 4553, 5086 and 5087. >After: > > This document does not introduce any new congestion-specific > mechanisms and thus does not introduce any new security > considerations above those present for PWs in general. > >I suggest adding a new paragraph: > > For PW security considerations, see the security considerations= sections > of the cited RFCS; the security considerations in [RFC3985], [RFC4553], > [RFC5086] and [RFC5087] are particularly applicable to this document. > > > > > Thanks, > > --David > > > > > > > -----Original Message----- > > > From: pwe3 [=20 > mailto:pwe3-bounces@ietf.org]=20 > On Behalf Of Yaakov Stein > > > Sent: Thursday, July 24, 2014 4:52 PM > > > To: pwe3@ietf.org > > > Subject: Re: [PWE3] I-D Action: draft-ietf-pwe3-congcons-02.txt > > > > > > Please look at the PDF version only > > > (the txt version does not have the graphs). > > > > > > Y(J)S > > > > > > > > > > _______________________________________________ > > pwe3 mailing list > > pwe3@ietf.org > > https://www.ietf.org/mailman/listinfo/pwe3 > >_______________________________________________ >pwe3 mailing list >pwe3@ietf.org >https://www.ietf.org/mailman/listinfo/pwe3 > > > > >________________________________________________________________ >Bob Briscoe, BT >_______________________________________________ >pwe3 mailing list >pwe3@ietf.org >https://www.ietf.org/mailman/listinfo/pwe3 > >________________________________________________________________ >Bob Briscoe, BT > >________________________________________________________________ >Bob Briscoe, BT ________________________________________________________________ Bob Briscoe, BT=20 --=====================_1836403412==.ALT Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable David,

At 19:59 20/08/2014, Black, David wrote:
Bob,
 
I think we=92re close - with luck, this email provides resolutions in principle to 5 and 7,
leaving 8, where I don=92t understand your argument.

...I'd like to hear from Yaakov before we start declaring consensus - it's been just two of us so far!

 
-- On 5:
 
> I figured that the curves that delineate the region are defined by the TCP-friendly equation, so 'friendly region' says on
> the tin what it does (and it is OK to imply that outside the region is unfriendly, because unfriendly doesn't imply
> "Don't go there" like 'unacceptable' would).
 
=93TCP-friendly=94 and =93possibly TCP-unfriendly=94 would be fine with me= -
I=92m just uncomfortable with use of vanilla unqualified =93friendly.=94

I'm sold on TCP-friendly region.

 
-- On 7, in reverse order:
 
> In fact, on reflection, whatever we say about circuit-breakers, I'd prefer not to mention circuit-breakers in the
> abstract, which implies this draft says stuff about mechanism, which was never the intent.
 
Removal of mention of circuit breakers from the abstract is fine with me.
 
> If we're going to say a bit about how to realize a circuit-breaker, I'd be happier with the word persistently
> than without. I'd still rather we didn't allow this draft to mission creep into mechanism mode.
 
I continue to agree with Stewart that something needs to be said here - the proposed
text leaves enormous flexibility to the operator to decide when to shut down TDM PWs.

That's my point. A draft that isn't about mechanism shouldn't tell a developer or an operator what they can or can't do about mechanism. If we want to talk about that, we will need another draft (perhaps a PWE example in the circuit-breakers draft, which has an intended status of Standards Track).

Here it is with =93persistent= ly=94 restored:
 
   For TDM pseudowires, such a circuit breaker can be realized via
   operator monitoring of drop rates and the quality of
   TDM service delivered in order to shut down TDM pseudowires that
   persistently fail to deliver acceptable TDM service (as defined by [G775] and
   [G826]).  This document explains why a TDM PW that is causing congestion
   problems via interference with elastic traffic is also not delivering
   acceptable TDM service (see the rest of this Section, plus Appendices
   A and B).

I can live with this much text. But don't blame me if this causes the draft to get referred back or questioned as it goes through subsequent phases of the RFC process.

 
-- On 8, I think I see the start of our disagreement:
 
> On these graphs, the op behaviour plots won't move at all if the PW uses larger packets. These are
> bit-rate plots, so the only way to move anything is to change the TCP segment size (which moves the TCP plots).
 
Actually, they=92re bit-rate plots on the assumption that both the PW and TCP are using the
same segment size, which is why Figure 2 looks better for the PWs than Figure 1 (the PW
lines did not move, but the TCP curves did move due to the segment size change).
 
Or in other words:
- Figure 1 compares a PW using 64-byte segments to a TCP connection using 64-byte segments.
- Figure 2 compares a PW using 256-byte segments to a TCP connection using 256-byte segments.
- Nothing in the draft makes a comparison using different segment sizes.
 
So, =93if the PW uses larger packets=94 then the approach in this draft compares it to a TCP
connection using larger segment, and the result looks like Figure 2 instead of Figure 1.
 
I do not like the idea of comparing a PW using 256-byte segments with a TCP connection
using 64-byte segments, which is what I understand that above =93won=92t mov= e at all=94 assertion
to be about.  The TCP curves do move between Figure 1 and Figure 2, based solely on a TCP
segment size change.

Correct. We're getting somewhere. The draft actually says two sentences about this assumption:
"   In order to quantitatively compare TDM PWs to TCP flows, we will
   compare the effect of TDM PW packets with that of TCP packets of the
   same packet size and sent at the same rate.  This is potentially an
   overly pessimistic comparison, as TDM PW packets are frequently
   configured to be short in order to minimize latency, while TCP
   packets are free to be much larger.
"

You are focusing solely on the first sentence, which we only used to simplify presentation of the analysis.

I am saying any recommendations should be based on the analysis /after/ having adjusted the outcome to reverse out any assumptions that otherwise make the analysis unrealistic (ie. take account of the second quoted sentence).

But my case is even stronger than that:

a) Even if we solely make the assumption of comparable packet sizes (for the sake of analysis) and ignore the second qualifying sentence (the reality), then as we continue through the whole doc we still get to the point where we show the problem is virtually non-existent. Given:
  * TCP-friendliness is not an absolute requirement - one bit-rate within half-a-dozen times the other is fine (I recall Sally FLoyd writing somewhere that x8 shouldn't be a problem)
  * the overlap only occurs because we've drawn the upper bound of loss probability for a PWE conservatively high (Appendix A says this).

So we don't want to then move straight into recommending a solution in the following paragraph, which makes it look like we don't believe our own conclusion - that PWEs are not a significant capacity sharing problem - even when both PWE and TCP use the same size packets.

b) If we then take note of the second qualifying sentence above, it really does make this an unequivocal non-problem. So we really shouldn't be proposing that operators change anything.

c) Actually, if we did state your recommendation, it wouldn't be correct to say the analysis supports it, because packet size problems require consideration of packet-congestion, not just bit-congestion. So we would have to repeat the full analysis for packet-congestion, not bit-congestion, and show packet-rate graphs as well as bit-rate graphs. Having done this, and having similarly taken points (a) and (b) into account, we would still come to the same conclusion - that there is not a problem significant enough to need to recommend any action.

So, for all these reasons, let's not make this recommendation.



Bob

 
Thanks,
--David
 
From: Bob Briscoe [ mailto:bob.briscoe@bt.com]
Sent: Wednesday, August 20, 2014 2:36 PM
To: Black, David
Cc: pwe3@ietf.org
Subject: RE: [PWE3] FW: WGLC comments on: draft-ietf-pwe3-congcons-02.txt
 
David,

Sry for delay. inline...

(Despite end of WGLC, we can still continue our public intra-author ping-pong :)

At 13:44 18/08/2014, Black, David wrote:

Bob,
 
1-4 & 6: ok
 
5: =93acceptable=94 is a good replacement for S-A-F-E, but I=92m not sure ab= out =93friendly=94.  The key concept here is assurance that there are no problems.  How about =93clearly acceptable=94?  That may need some additional text to say that acceptable network and service behavior may be delivered outside the clearly acceptable region, but one cannot rely on the analysis in this draft (based on the TCP throughput formula) to reach such a conclusion.

I wanted a different word from 'acceptable' for the adjective in the captions above the Figures describing the regions, because there's no opportunity for more text there to explain that we do not mean that outside the region is unacceptable. I thought about various qualifiers with acceptable (like clearly, unarguably, etc), but I don't think these are right either.

I figured that the curves that delineate the region are defined by the TCP-friendly equation, so 'friendly region' says on the tin what it does (and it is OK to imply that outside the region is unfriendly, because unfriendly doesn't imply "Don't go there" like 'unacceptable' would).

Let's stop batting back & forth until we see what Yaakov (or others) think.



7: I disagree - the tsvwg circuit breaker draft is rather general, and we now have two WG LC comments on this draft asking how to apply a circuit breaker to PWs.  I think we=92re on solid ground with the references to the ITU-T service quality standards and expecting operator monitoring of service quality.  OTOH, the word =93persistently=94 seemed to be a concern, and I have no problem removing it (anyone who asks about that can be directed to G.775 and G.826).  New suggested text:
 
   For TDM pseudowires, such a circuit breaker can be realized via
   operator monitoring of drop rates and the quality of
   TDM service delivered in order to shut down TDM pseudowires that
   fail to deliver acceptable TDM service (as defined by [G775] and
   [G826]).  This document explains why a TDM PW that is causing congestion
   problems via interference with elastic traffic is also not delivering
   acceptable TDM service (see the rest of this Section, plus Appendices
   A and B).

If we're going to say a bit about how to realize a circuit-breaker, I'd be happier with the word persistently than without. I'd still rather we didn't allow this draft to mission creep into mechanism mode.

In fact, on reflection, whatever we say about circuit-breakers, I'd prefer not to mention circuit-breakers in the abstract, which implies this draft says stuff about mechanism, which was never the intent.


 
8: I believe that figures 1 and 2 suffice to make the point, but I see that a recommendation may be a bit of a reach for this draft.  How about the following modified form of my first sentence, which just states the inference that can be drawn from Figures 1 and 2? -
  
   Figures 1 and 2 indicate that a larger segment size for a T1 or E1
   TDM PW moves the operational behavior of the PW away from the boundary
   at which the TCP throughput equation suggests that undue interference with
   elastic traffic may start to occur.

Not so - that's the problem. On these graphs, the op behaviour plots won't move at all if the PW uses larger packets. These are bit-rate plots, so the only way to move anything is to change the TCP segment size (which moves the TCP plots). To justify what you want to say, either:
a) these would have to be packet-rate plots, or
b) we would have to turn the assumption we've made about equal packet sizes into an operational requirement, not just an analytical assumption.

To go in either of these directions would require a spade and vermin.


Bob

PS. GOing offline until next Tue. Might check email, but no guarantee.


 
Thanks,
--David
 
From: Bob Briscoe [ mailto:bob.briscoe@bt.com]
Sent: Friday, August 15, 2014 6:03 AM
To: Black, David
Cc: pwe3@ietf.org
Subject: Re: [PWE3] FW: WGLC comments on: draft-ietf-pwe3-congcons-02.txt
 
David,

I decided to think on a better word for 'safe' overnight, and allow others to comment. But no-one has, so here's my responses to all your outstanding points...

I've provided slightly updated xml source of my suggested edits (and a diff) here:
< http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-pwe3-c= ongcons-DIFF03b-02.html >
< http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-pwe3-c= ongcons-03b.txt >
< http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-pwe3-c= ongcons-03b.xml >
Again note: xml is purely for precision, not implying I have taken over editorship.

1. Happy with your new scoping text, except the the first 'independent' clangs with the second, and it's unnecessary anyway:
"  The concern is over PWs that share network capacity with elastic or congestion-responsive traffic, [independent of] whether that sharing is a result of the service provider network's traffic engineering and capacity planning or independent deployment of PWs by entities other than network operators.
"

2. My point wasn't to remove the reference to the 125us (in my suggested text I didn't). It was that the 125us stuff /alone/ doesn't warrant the 'thus' in the next clause; the logic needs to be connected by talking about the sum of the flight time /and/ the per-node processing time.
CURRENT:
   The one-way delay
   of a native TDM service consists of the physical time-of-flight plus
   125 microseconds for each TDM switch traversed; and is thus very
   small as compared to typical PSN network-crossing latencies.
SUGGESTED:
   The one-way delay
   of a native TDM service consists of the physical time-of-flight plus
   125 microseconds for each TDM switch traversed; so the total one-way delay for most TDM PWs has to be small compared to typical PSN network-crossing latencies.

5. I understand and agree - 'S-A-F-E region' is incendiary. But I don't like 'reasonable region' - it's slightly /too/ meaningless.

Other possibilities are:
- admissable region (problem: implies outside the region is inadmissable)
- harmonious region (problem: too Californian)
So I suggest two words:
- friendly or acceptable, each used depending on context.

On the graphs, I suggest 'operating region (friendly under curve)' because that is precisely where the curves come from - 'friendly' does what it says on the tin, and the implied 'unfriendly' above the curve correctly avoids implying that the sky will fall.

Elsewhere, I suggest the following replacements for the S-word:
"It is thus worthwhile specifying under what conditions such competition is acceptable"
"One may view this condition as defining a 'friendly' operating envelope for a TDM PW"
"Under this condition it is acceptable to place the TDM PW along with ... TCP. "
"comparison with something widely considered to offer an acceptable response to congestion"


7. Circuit breaker
Scope creep. Rat-hole. Stop digging.
Yes, I know Stewart has asked us to dig out another rat as well, but the appropriate response is "Out of scope".

You might think your text is uncontroversial, but if we are tempted to take Stewart's bait (rat metaphor intended), it will probably start new debates that this doc doesn't deserve: "What does persistent mean?", "Which traffic is more important/ valuable?", yada yada.

I suggest the text merely makes it clear that this debate is in scope for elsewhere, not here. I've also just realised that Yaakov has written the sentences around circuit breaker with the wrong logical sequence. The circuit breaker needs to be introduced as something the PW uses when its /own/ quality has failed, which should avoid any need to shut down because it is threatening elastic services:

CURRENT:
      If the bandwidth consumed by a TDM PW is
      considered detrimental, the only available remedy is to completely
      shut down the PW, by using a transport circuit breaker mechanism.
      However, we will show that even before such an action is
      warranted, the PW will become unable to faithfully emulate the
      native TDM service; for example, when a TDM service is carrying
      voice grade telephony channels, the voice quality will degrade to
      below useful levels.
SUGGESTED:
      We will show that, before a PW becomes deterimental to elastic traffic, it will become unable to faithfully emulate the
      native TDM service; for example, when a TDM service is carrying
      voice grade telephony channels, the voice quality will degrade to
      below useful levels. Once the quality of the emulated service is persistently unacceptable, if a PW shuts down selected flows or shuts itself down completely, it will automatically avoid detriment to elastic traffic. A transport circuit breaker [I-D.fairhurst-tsvwg-circuit-breaker] could be used, but mechanism is beyond the scope of the present document.


8. Segment size.
Everyone's in a big group-hug because we've written a draft showing there's is no need to change anything. Then just at the end of an uneventful WGLC, we hear David's voice from the bottom of a deep hole shouting, "avoid the rat-hole invitation", while furiously digging for a rat.

There is no need to add a recommendation that PWs need to be changed to use larger packets. No matter how hedged and soft this recommendation would be, it's not needed anyway. Stop digging.

Yaakov's new text is fine as it stands. "Only for small packets, long delays, and high packet loss ratios, do TDM PWs potentially consume more bandwidth, and even then only marginally. "

In order to justify your sentence, we would have to include graphs with packet-rate (not bit-rate) on the vertical axis and go into a whole treatise as to why packet-rate comparison might be important as well. Then the graph would show one plot for all TCP segment sizes and multiple plots for PWs with different packet sizes. Then we'd see that the corner of the two smallest packet PWs just clipped the TCP curve at loss probabilities that are stupidly large anyway. So even after all this, it would hardly justify a need for your sentence.

All that digging would not be worth it, just to discover (again) that there is no rat.


Bob


At 19:39 14/08/2014, Black, David wrote:

Content-Language: en-US
Content-Type: multipart/alternative;
         boundary=3D"_000_8D3D17ACE214DC429325B2B98F3AE712077B95212CMX15Acorpemc= c_"

And forwarding my views to the list ...
 
... On Bob=92s six thoughts (I have problems with 2 & 5 - the other 4 are basically ok w/me) plus follow-ups on Bob=92s concerns w/my two major issues, here goes:
 
-- 1 -- I agree in principle, but I think Bob=92s suggested new text is too black/white and should be generalized.
 
Bob suggests:
 
> The case where the service provider network arranges sufficient capacity for a PW is well understood and
> will not be discussed further here. The concern is over PWs deployed independently of the service provider
> network's traffic engineering or capacity planning.
 
I want to include the case where capacity has been arranged, but may not be sufficient, I suggest rewriting Bob=92s second sentence as:
 
The concern is over PWs that share network capacity with elastic or congestion-responsive traffic, independent of whether that sharing is a result of the service provider network's traffic engineering and capacity planning or independent deployment of PWs by entities other than network operators.
 
-- 2 -- Here, I disagree.  One of the significant contributions of this draft is that TDM PWs don=92t cause congestion problems in practice, and one of the major reasons for that is the low latency requirements for acceptable TDM service delivery, both in ITU-T standards and in actual TDM PW deployment.  The 125us behavior of native TDM switches is relevant to the latter, and hence I prefer to include mention of it.
 
-- 3 -- I agree; clear captions are always a feature, not a bug ;-).
 
-- 4 -- I agree with changing =93packet loss rate=94 to =93packet loss probability=94.
 
-- 5 -- I firmly disagree with Bob here, although he=92s right that this is =93techno-political=94 ;-).
 
I am concerned that S-A-F-E is likely to be a 4-letter word in wider Transport Area review, and hence, I want to use a word that does not imply anything remotely resembling an absolute guarantee that nothing can possibly ever go wrong (yes, I know Bob didn=92t mean to go to quite that extreme an interpretation, but S-A-F-E is liable to be read that way by those with an =93axe to grind=94), even though there=92s lots of solid operational experience that suggests all is well here.
 
-- 6 -- I think I=92m ok with what Bob proposes to better characterize loss burstiness, but want to see specific text.
 
-- Circuit Breaker Mechanism --
 
>> [David] I suggest adding:
>>
>>    For TDM pseudowires, such a circuit breaker can be realized via
>>    operator monitoring of drop rates and consequent on the quality of
>>    TDM service delivered and shutting down TDM pseudowires that persistently
>>    fail to deliver acceptable TDM service (as defined by [G775] and
>>    [G826]).  This document explains why a TDM PW that is causing congestion
>>    problems via interference with elastic traffic cannot be delivering
>>    acceptable TDM service (see the rest of this Section, plus Appendices
>>    A and B).
>>
> [Bob] I don't think we should go there. We need to keep this draft firmly as informational. If someone
> thinks a mechanism is needed because of the information in this draft, then they can write about a mechanism.
 
[David] I firmly disagree, but Bob and I might be talking past each other, as I do want to stay informational ;-).
 
This text is intended to describe an example or possibility, not to specify an implementation requirement.  The new text was intended as an OAM/OSS practice recommendation to the operator, not as a normative protocol spec.
 
I=92m ok with inserting =93example=94, =93possibility=94 or something= similar if that helps w/Bob=92s concern, but I strongly oppose doing nothing here, and note that this is also needed to respond to one of Stewart Bryant=92s WG LC comments.
 
-- PW segment size --
 
>> [David] Ignoring the rathole invitation ;-).  This is still in section 3, after:
>>
>>   We see that in general for large packet sizes, short delays, and low
>>   packet loss rates, the TDM pseudowires consume much less bandwidth
>>    than TCP would under identical conditions.  Only for small packets,
>>    long delays, and high packet loss ratios, do TDM PWs potentially
>>   consume more bandwidth, and even then only marginally.
>>
>> I suggest adding:
>>
>>    Comparing figures 1 and 2, use of a larger segment size for a T1 or E1
>>    TDM PW moves the operational behavior of the PW away from the boundary
>>    at which the TCP throughput equation suggests that undue interference with
>>    elastic traffic may start to occur.  We suggest that larger segment
>>    sizes be preferred for T1 and E1 TDM PWs, but note that other
>>    considerations may favor smaller segment sizes (e.g., jitter buffer
>>    size, bounding the impact of an isolated packet drop event).
>>
> [Bob] There is no need to recommend any remedial action, when we have just shown there is not a problem.
 
[David] Not exactly - Figure 1 shows that there is a potential concern at a combination of the high end of the range of TDM PW latencies (4ms), and higher packet loss ratios. 

> Further, the "marginal problem" only applies when the TCP traffic uses packets smaller than 128B, which we only used
> as a hypothetical worst-case for the analysis. But in practice this doesn't happen with long-running TCP flows -
> they use packets that are as large as possible (irrespective of what the PW uses). So there really is /no/ problem to remedy.
Ok, Bob can relax - his transport credentials are intact, as this is an entry point to the bit-rate-congestible vs. packet-rate-congestible resource rathole that I want to ignore :-).
 
I believe that larger TDM PW segment sizes are a good thing for both this operating envelope boundary concern as well as per-packet processing overheads in TDM PW functionality, modulo other considerations, e.g., jitter buffers, and so I prefer to keep this addition.
 
Thanks,
--David
 
From: Bob Briscoe [mailto:bob.briscoe@bt.com]
Sent: Thursday, August 14, 2014 9:03 AM
To: Yaakov Stein
Cc: Andrew G. Malis; BOCCI Matthew; Black, David
Subject: RE: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt
 
Yaakov,

I have just re-read it thoroughly. I have noticed about two dozen editorial nits where correcting them would not change the intended sense (altho sometimes the sense needs to change to say what we intended!). Examples:
- Unexpanded abbreviations (David's suggestion would help, but there are more)
- Missing words, tautologies, random punctuation problems etc
- Uses of the wrong word, even tho we know what you mean
- The caveat about dynamics vs steady state needs to shift down a para, so that it follows into the para that says what to do about dynamics.

Should I take the token, and edit the XML directly, then you can see a diff to check if you want to accept my suggestions?

Below are six thoughts that go slightly beyond 'editorial'. 1-4 could be considered 'editorial'. But 5&6 are the more significant "techno-political" points.

1. The phrase "mechanisms for provisioning PWs in service provider networks" is too opaque (the PWE audience might understand this scoping sentence, but a L4 reader won't understand what we're excluding here, which will make them suspicious).
CURRENT:
   Mechanisms for provisioning PWs
   in service provider networks are well understood and will not be
   discussed further here.
SUGGESTED:
   The case where the service provider network arranges sufficient capacity for a PW is well understood and will not be discussed further here. The concern is over PWs deployed independently of the service provider network's traffic engineering or capacity planning.

2. The justification for using OWD of a few milliseconds misses the point.
"  The one-way delay
   of a native TDM service consists of the physical time-of-flight plus
   125 microseconds for each TDM switch traversed; and is thus very
   small as compared to typical PSN network-crossing latencies.
"
The 125us per switch is irrelevant. The point that needs making is that PWs are generally deployed with a short physical time of flight.

3. Related point: the captions on the graphs do not clearly indicate that they are OWD, not RTT. This could be clarified in the text, or on the graphs.

4. Change packet loss rate (and PLR) to packet loss probability throughout, including on the graph axes. PLR is an ambiguous term, even though it is widely used in the industry. Packet loss rate could be measured in packet per second, whereas packet loss probability is unambiguously dimensionless. Indeed, in the appendices, packet loss rates with respect to time are calculated.

5. Rather than David's "reasonable" I suggest we use the phrase "unambiguously safe" not just "safe" for the headings of Figs 5-8 and in the text, as below. Otherwise, we are implying that it is unsafe outside the region.

CURRENT:
   One may view this condition as defining an operating envelope for a
   TDM PW, as a TDM PW that occupies no more bandwidth than a TCP flow
   causes no more congestion than that TCP flow would.  Under this
   condition it is safe to place the TDM PW along with congestion-
   responsive traffic such as TCP, without causing additional
   congestion.  on the other hand, were the TDM PW to consume
   significantly more bandwidth a TCP flow, it could contribute
   disproportionately to congestion, and its mixture with congestion-
   responsive traffic might be inappropriate.
SUGGESTED:
   One may view this condition as defining an unambiguously safe operating envelope for a
   TDM PW, as a TDM PW that occupies no more bandwidth than a TCP flow
   causes no more congestion than that TCP flow would.  This does not draw us into a debate over whether TCP-friendliness is a valid constraint; it simply says that there can be no question that a PW is safe if it does no more harm than a single TCP flow. Under this
   condition it is unambiguously safe to place the TDM PW along with congestion-
   responsive traffic such as TCP.  On the other hand, were the TDM PW to consume
   significantly more bandwidth a TCP flow, it could contribute
   disproportionately to congestion, and its mixture with congestion-
   responsive traffic might start to cause concern.

6. The analysis in Appendices A & B needs to mention that loss bursts arise primarily from tail drop buffers, and the introduction of active queue management (AQM) makes losses more independent of each other (less bursty). (I have references showing evidence of this.) In the comparisons with TCP, we have assumed independent losses anyway, because they are the worst case (and fortunately they are easier to analyse). On a similar point, in appendix B, where we say "(bursty packet loss was also simulated but is not reported here)", we ought to give a reason otherwise readers will be suspicious. A good reason would be that uniformly random is the worst case.

I've also responded to a couple of David's suggestions inline (I agree with the others)...


At 17:44 13/08/2014, Yaakov Stein wrote:
Bob told me on Friday of the IETF that he was going to look over the final version and comment.
 
Bob =96 ping ?
 
Y(J)S
 
From: Andrew G. Malis [ mailto:agmalis@gmail.com]
Sent: 13 August, 2014 17:02
To: Yaakov Stein
Cc: Bob Briscoe; rbriscoe@jungle.bt.co.uk; BOCCI Matthew; Black, David
Subject: Re: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt
 
Yaakov,
 
I still haven't heard from Bob, but you should incorporate David's updates into a new revision, and I'll use that version for my final Shepherd's review.
 
Thanks,
Andy
 
On Mon, Aug 11, 2014 at 11:16 AM, Andrew G. Malis <agmalis@gmail.com> wrote:
Actually, I'm going to end the LC as soon as I hear back from Bob on the IPR call. I sent him a reminder earlier this morning.
 
Cheers,
Andy
 
On Sun, Aug 10, 2014 at 9:53 AM, Andrew G. Malis <agmalis@gmail.com> wrote:
David,
 
That's perfect, thanks! I'm going to end the LC tomorrow.
 
Cheers,
Andy
 
On Sat, Aug 9, 2014 at 10:32 PM, Black, David <david.black@emc.com> wrote:
Following up on this ...

> As we're now moving quickly to try to complete this draft , I agreed
> w/Yaakov that my author comments would be sent in as WG Last Call comments ...
> ... so here they are:
... with some suggested new text ...

> Major issues:
>
> The reference to a transport circuit breaker in section 3 is a bit optimistic
> and high level:
>
>       We can accomplish this by
>       employing a transport circuit breaker, by which we mean an automatic
>       mechanism for terminating a flow to prevent negative impact on other
>       flows and on the stability of the network
>       [I-D.fairhurst-tsvwg-circuit-breaker].
>
> This should be accompanied by at least an example of how to do this - past
> discussion has suggested carrier monitoring of delivered service quality
> in order to shut down PWs that are experiencing loss rates that result in
> unacceptable service quality (as defined for us by the ITU ...).  I'd also
> like to add some text to suggest that operators use this sort of circuit
> breaker practice when TDM PWs compete with elastic traffic.
After this text in Section 3:

   We can accomplish this by
   employing a transport circuit breaker, by which we mean an automatic
   mechanism for terminating a flow to prevent negative impact on other
   flows and on the stability of the network
   [I-D.fairhurst-tsvwg-circuit-breaker].  Note that a transport circuit
   breaker is intended as a protection mechanism of last resort, just as
   an electrical circuit breaker is only triggered when absolutely
   necessary.

I suggest adding:

   For TDM pseudowires, such a circuit breaker can be realized via
   operator monitoring of drop rates and consequent on the quality of
   TDM service delivered and shutting down TDM pseudowires that persistently
   fail to deliver acceptable TDM service (as defined by [G775] and
   [G826]).  This document explains why a TDM PW that is causing congestion
   problems via interference with elastic traffic cannot be delivering
   acceptable TDM service (see the rest of this Section, plus Appendices
   A and B).

I don't think we should go there. We need to keep this draft firmly as informational. If someone thinks a mechanism is needed because of the information in this draft, then they can write about a mechanism.


And breaking the paragraph after the added text.

> Figure 1 and Figure 2 indicate that based on using the TCP throughput
> equation as a proxy for potential impact on elastic traffic (which is what
> we're doing in this draft), for the same fixed bandwidth PW, a larger segment
> size is less likely to cause impacts.  I'd like to add a suggestion to use
> the larger PW segment sizes as long as they're consistent with delivery of
> the service.  There's been a related (long-to-the-point-of-ratholed) debate
> over in the Transport Area about whether packet-rate-limited/congestible
> forwarding is still relevant.  My view is that such equipment exists, and
> hence that this is a relevant consideration for this draft, and would not
> mention that issue here.
Ignoring the rathole invitation ;-).  This is still in section 3, after:

   We see that in general for large packet sizes, short delays, and low
   packet loss rates, the TDM pseudowires consume much less bandwidth
   than TCP would under identical conditions.  Only for small packets,
   long delays, and high packet loss ratios, do TDM PWs potentially
   consume more bandwidth, and even then only marginally.

I suggest adding:

   Comparing figures 1 and 2, use of a larger segment size for a T1 or E1
   TDM PW moves the operational behavior of the PW away from the boundary
   at which the TCP throughput equation suggests that undue interference with
   elastic traffic may start to occur.  We suggest that larger segment
   sizes be preferred for T1 and E1 TDM PWs, but note that other
   considerations may favor smaller segment sizes (e.g., jitter buffer
   size, bounding the impact of an isolated packet drop event).

There is no need to recommend any remedial action, when we have just shown there is not a problem.

Further, the "marginal problem" only applies when the TCP traffic uses packets smaller than 128B, which we only used as a hypothetical worst-case for the analysis. But in practice this doesn't happen with long-running TCP flows - they use packets that are as large as possible (irrespective of what the PW uses). So there really is /no/ problem to remedy.


and again, breaking the paragraph after that added text.

I hope that latter sentence ("We suggest that ...") is acceptable to the WG.

> Minor issues:
>
> Section 2 is about Ethernet PWs that carry elastic traffic.  Should others
> be mentioned as also covered by this analysis - e.g., ATM and Frame Relay
> PWs carrying elastic traffic?
After:

   In this section we consider Ethernet PWs that primarily carry
   congestion-responsive traffic.  We show that we automatically obtain
   the desired congestion avoidance behavior, and that additional
   mechanisms are not needed.

I suggest adding:

   This discussion is applicable to any PW that carries congestion-
   responsive traffic (e.g., ATM and Frame Relay PWs); Ethernet PWs are
   used here for clarity, as they are a common example.

> Section 2 does not cover Ethernet PWs carrying inelastic traffic - that would
> be unusual, but should it be explicitly noted as out of scope, or just not
> discussed (as is the case for the current text)?
I suggest doing nothing here, and planning to deal with this concern
if it should arise later (e.g., IETF Last Call or IESG Evaluation).

> Section 3 should say that the reason the draft focuses on TDM PWs is that
> they're the inelastic PWs that are likely to be carried over IP in practice.
> My understanding is that other potentially inelastic PWs are MPLS-only
> in practice.
After:

   Inelastic PWs, such as TDM PWs ([RFC4553][RFC5086][RFC5087]), are
   potentially more problematic than the elastic PWs of the previous
   section.  Being constant bit-rate (CBR), TDM PWs can not be made
   responsive to congestion.  On the other hand, being CBR, they also do
   not attempt to capture additional bandwidth when neighboring TCP
   flows back off.

I suggest adding:

   We focus on TDM PWs here as they are the only inelastic PWs that are
   likely to be IP-encapsulated in practice.  Most other inelastic PWs
   tend to only be used with MPLS.

[I'd ask Yaakov and other PW experts to double-check the latter sentence,
please.]

> A terminology section to roll up all the acronym expansions (e.g., TDM, CBR)
> in one place seems like a good idea.
These four acronyms would be good to put into a terminology section:

CBR Constant Bit Rate
PSN Packet Switched Network
PW Pseudowire
TDM Time Division Multiplexing

>
> This text on p.11 could be incendiary:
>
>       Under this condition it is safe to place the TDM PW along with
>       congestion - responsive traffic such as TCP, without causing
>       additional congestion.
>
> First of all to reduce the flammability, "safe" -> "reasonable" and there
> are a number of other occurrences of "safe" that deserve similar treatment ;-
> ).
> Beyond that, this assertion is dependent on the operating conditions staying
> within that reasonable operating envelope, which entails use of something
> like a circuit breaker - I'd say that here, and reference the earlier
> circuit breaker discussion.
Global substitution: "safe" -> "reasonable", starting with the abstract.

Then for this specific text on p.11:

OLD
        Under this condition it is safe to place the TDM PW along with
        congestion - responsive traffic such as TCP, without causing
        additional congestion.
NEW
        Under this condition it is reasonable to place the TDM PW in competition
      with congestion-responsive traffic such as TCP, as the result should not
      produce unacceptable congestion effects.  That assumes that the operating
      conditions remain within the reasonable operating envelope - as discussed
      above, a transport circuit breaker is a suggested means of managing a TDM
      PW so that it is shut down if operating conditions remain persistently
      outside the reasonable operating envelope for the PW (e.g., due to a
      high packet loss rate).

No need to go there. See my suggestions earlier.


> Figures 7 and 8 could use some shading or lines to indicate the typical
> deployed operating regions for the pseudo-wires to visibly show that they
> fall under the curves, particularly for larger segment sizes.  I'd also
> repeat or refer to the suggestion to use larger

I wanted to suggest this myself, but it is hard to depict fuzzy boundaries. If you can do this, pls do.

Given the PW operating regions are bounded by v low loss probability, the x axis in Figs 7&8 ought to be scaled much smaller as well.

Related point: the y axis in Fig 1 ought to max at 10, not 50.

Regards



Bob

>
> Nits:
>
> References aren't allowed in abstracts, so remove reference to
> [I-D.fairhurst-tsvwg-circuit-breaker]
>
> Introduction, first paragraph, delete "PWE3" towards end of paragraph -
>
>    preferably the 4 byte PWE3 control word.
>
> In addition to its current general statements, the Security Considerations
> section should point to the security considerations sections in a few
> particularly relevant RFCs - I'd suggest 3985, 4553, 5086 and 5087.
After:

   This document does not introduce any new congestion-specific
   mechanisms and thus does not introduce any new security
   considerations above those present for PWs in general.

I suggest adding a new paragraph:

   For PW security considerations, see the security considerations sections
   of the cited RFCS; the security considerations in [RFC3985], [RFC4553],
   [RFC5086] and [RFC5087] are particularly applicable to this document.

>
> Thanks,
> --David
>
>
> > -----Original Message-----
> > From: pwe3 [ mailto:pwe3-bounces@ietf.org] On Behalf Of Yaakov Stein
> > Sent: Thursday, July 24, 2014 4:52 PM
> > To: pwe3@ietf.org
> > Subject: Re: [PWE3] I-D Action: draft-ietf-pwe3-congcons-02.txt
> >
> > Please look at the PDF version only
> > (the txt version does not have the graphs).
> >
> > Y(J)S
> >
> >
>
> _______________________________________________
> pwe3 mailing list
> pwe3@ietf.org
> https://www.ietf.org/mailman/listinfo/pwe3

_______________________________________________
pwe3 mailing list
pwe3@ietf.org
https://www.ietf.org/mailman/listinfo/pwe3
 
 
 

________________________________________________________________
Bob Briscoe,           &n= bsp;            =             &nbs= p;             BT
_______________________________________________
pwe3 mailing list
pwe3@ietf.org
https://www.ietf.org/mailman/listinfo/pwe3

________________________________________________________________
Bob Briscoe,           &n= bsp;            =             &nbs= p;             BT

________________________________________________________________
Bob Briscoe,           &n= bsp;            =             &nbs= p;             BT

________________________________________________________________
Bob Briscoe,           &n= bsp;            =             &nbs= p;             BT --=====================_1836403412==.ALT-- From nobody Wed Aug 27 21:39:34 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 50B4C1A0348 for ; Wed, 27 Aug 2014 21:39:32 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.069 X-Spam-Level: X-Spam-Status: No, score=-6.069 tagged_above=-999 required=5 tests=[BAYES_20=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, GB_I_INVITATION=-2, GB_I_LETTER=-2, GB_SUMOF=1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.668, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OfZbTFpt6HYY for ; Wed, 27 Aug 2014 21:39:17 -0700 (PDT) Received: from mailuogwdur.emc.com (mailuogwdur.emc.com [128.221.224.79]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6680E1A0347 for ; Wed, 27 Aug 2014 21:39:16 -0700 (PDT) Received: from maildlpprd51.lss.emc.com (maildlpprd51.lss.emc.com [10.106.48.155]) by mailuogwprd52.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s7S4d8NH008631 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 28 Aug 2014 00:39:08 -0400 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd52.lss.emc.com s7S4d8NH008631 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=emc.com; s=jan2013; t=1409200749; bh=6RsW1iAB1nnC2wPNk1BYlfgoqek=; h=From:To:CC:Date:Subject:Message-ID:References:In-Reply-To: Content-Type:MIME-Version; b=ktg3k2QzgVeTaTalLVx5QBFTOtPybYW5Iiw2SvVMty4ZMFThp3cns5M/1FMG9t+aB QF9aLELKiIMMvny5m3+Fzuao7M1Y9czKnX2AzENJe3LSoX7QgovW7ofKpr7Y7JQ+RH +vqxC3kdw8uIArMKKgsxr3bL7PgEyQxAHHbraG/s= X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd52.lss.emc.com s7S4d8NH008631 Received: from mailusrhubprd54.lss.emc.com (mailusrhubprd54.lss.emc.com [10.106.48.19]) by maildlpprd51.lss.emc.com (RSA Interceptor); Thu, 28 Aug 2014 00:38:53 -0400 Received: from mxhub05.corp.emc.com (mxhub05.corp.emc.com [128.222.70.202]) by mailusrhubprd54.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s7S4cstT004314 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Thu, 28 Aug 2014 00:38:55 -0400 Received: from mx15a.corp.emc.com ([169.254.1.175]) by mxhub05.corp.emc.com ([128.222.70.202]) with mapi; Thu, 28 Aug 2014 00:38:54 -0400 From: "Black, David" To: Bob Briscoe Date: Thu, 28 Aug 2014 00:38:52 -0400 Thread-Topic: [PWE3] FW: WGLC comments on: draft-ietf-pwe3-congcons-02.txt Thread-Index: Ac/BXAw7lJBljsefTbC5z/Wl+DY8cABHYp0Q Message-ID: <8D3D17ACE214DC429325B2B98F3AE712077BC6669A@MX15A.corp.emc.com> References: <8D3D17ACE214DC429325B2B98F3AE712077B9517F4@MX15A.corp.emc.com> <8D3D17ACE214DC429325B2B98F3AE712077B951928@MX15A.corp.emc.com> <201408141302.s7ED2km0032238@bagheera.jungle.bt.co.uk> <8D3D17ACE214DC429325B2B98F3AE712077B952113@MX15A.corp.emc.com> <8D3D17ACE214DC429325B2B98F3AE712077B95212C@MX15A.corp.emc.com> <201408151006.s7FA6nWG003570@bagheera.jungle.bt.co.uk> <8D3D17ACE214DC429325B2B98F3AE712077BB42180@MX15A.corp.emc.com> <201408201835.s7KIZerA027879@bagheera.jungle.bt.co.uk> <8D3D17ACE214DC429325B2B98F3AE712077BB426E5@MX15A.corp.emc.com> <201408261831.s7QIVgVr027059@bagheera.jungle.bt.co.uk> In-Reply-To: <201408261831.s7QIVgVr027059@bagheera.jungle.bt.co.uk> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_8D3D17ACE214DC429325B2B98F3AE712077BC6669AMX15Acorpemcc_" MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd54.lss.emc.com X-RSA-Classifications: DLM_1, public, GIS Solicitation Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/32FZE6waV0yj-HCfr7voY3q7TXw Cc: "pwe3@ietf.org" Subject: Re: [PWE3] FW: WGLC comments on: draft-ietf-pwe3-congcons-02.txt X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Aug 2014 04:39:32 -0000 --_000_8D3D17ACE214DC429325B2B98F3AE712077BC6669AMX15Acorpemcc_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Bob, I'll yield to your views on point 8 to get the draft done. I continue to b= elieve that a recommendation for larger PW segment sizes is appropriate, bu= t it's at best secondary to the primary purpose of this draft, which is to = explain why TDM PWs do not pose congestion problems in practice (as you poi= nt out). Yaakov - when do you think you'll be able to submit a revised draft? I'm a= bout to vanish for 3 weeks of vacation w/little to no email access. Thanks, --David From: Bob Briscoe [mailto:bob.briscoe@bt.com] Sent: Tuesday, August 26, 2014 2:32 PM To: Black, David Cc: pwe3@ietf.org; Black, David Subject: RE: [PWE3] FW: WGLC comments on: draft-ietf-pwe3-congcons-02.txt David, At 19:59 20/08/2014, Black, David wrote: Bob, I think we're close - with luck, this email provides resolutions in princip= le to 5 and 7, leaving 8, where I don't understand your argument. ...I'd like to hear from Yaakov before we start declaring consensus - it's = been just two of us so far! -- On 5: > I figured that the curves that delineate the region are defined by the TC= P-friendly equation, so 'friendly region' says on > the tin what it does (and it is OK to imply that outside the region is un= friendly, because unfriendly doesn't imply > "Don't go there" like 'unacceptable' would). "TCP-friendly" and "possibly TCP-unfriendly" would be fine with me - I'm just uncomfortable with use of vanilla unqualified "friendly." I'm sold on TCP-friendly region. -- On 7, in reverse order: > In fact, on reflection, whatever we say about circuit-breakers, I'd prefe= r not to mention circuit-breakers in the > abstract, which implies this draft says stuff about mechanism, which was = never the intent. Removal of mention of circuit breakers from the abstract is fine with me. > If we're going to say a bit about how to realize a circuit-breaker, I'd b= e happier with the word persistently > than without. I'd still rather we didn't allow this draft to mission cree= p into mechanism mode. I continue to agree with Stewart that something needs to be said here - the= proposed text leaves enormous flexibility to the operator to decide when to shut dow= n TDM PWs. That's my point. A draft that isn't about mechanism shouldn't tell a develo= per or an operator what they can or can't do about mechanism. If we want to= talk about that, we will need another draft (perhaps a PWE example in the = circuit-breakers draft, which has an intended status of Standards Track). Here it is with "persistently" restored: For TDM pseudowires, such a circuit breaker can be realized via operator monitoring of drop rates and the quality of TDM service delivered in order to shut down TDM pseudowires that persistently fail to deliver acceptable TDM service (as defined by [G775= ] and [G826]). This document explains why a TDM PW that is causing congestion problems via interference with elastic traffic is also not delivering acceptable TDM service (see the rest of this Section, plus Appendices A and B). I can live with this much text. But don't blame me if this causes the draft= to get referred back or questioned as it goes through subsequent phases of= the RFC process. -- On 8, I think I see the start of our disagreement: > On these graphs, the op behaviour plots won't move at all if the PW uses = larger packets. These are > bit-rate plots, so the only way to move anything is to change the TCP seg= ment size (which moves the TCP plots). Actually, they're bit-rate plots on the assumption that both the PW and TCP= are using the same segment size, which is why Figure 2 looks better for the PWs than Figu= re 1 (the PW lines did not move, but the TCP curves did move due to the segment size cha= nge). Or in other words: - Figure 1 compares a PW using 64-byte segments to a TCP connection using 6= 4-byte segments. - Figure 2 compares a PW using 256-byte segments to a TCP connection using = 256-byte segments. - Nothing in the draft makes a comparison using different segment sizes. So, "if the PW uses larger packets" then the approach in this draft compare= s it to a TCP connection using larger segment, and the result looks like Figure 2 instead= of Figure 1. I do not like the idea of comparing a PW using 256-byte segments with a TCP= connection using 64-byte segments, which is what I understand that above "won't move a= t all" assertion to be about. The TCP curves do move between Figure 1 and Figure 2, based s= olely on a TCP segment size change. Correct. We're getting somewhere. The draft actually says two sentences abo= ut this assumption: " In order to quantitatively compare TDM PWs to TCP flows, we will compare the effect of TDM PW packets with that of TCP packets of the same packet size and sent at the same rate. This is potentially an overly pessimistic comparison, as TDM PW packets are frequently configured to be short in order to minimize latency, while TCP packets are free to be much larger. " You are focusing solely on the first sentence, which we only used to simpli= fy presentation of the analysis. I am saying any recommendations should be based on the analysis /after/ hav= ing adjusted the outcome to reverse out any assumptions that otherwise make= the analysis unrealistic (ie. take account of the second quoted sentence). But my case is even stronger than that: a) Even if we solely make the assumption of comparable packet sizes (for th= e sake of analysis) and ignore the second qualifying sentence (the reality)= , then as we continue through the whole doc we still get to the point where= we show the problem is virtually non-existent. Given: * TCP-friendliness is not an absolute requirement - one bit-rate within h= alf-a-dozen times the other is fine (I recall Sally FLoyd writing somewhere= that x8 shouldn't be a problem) * the overlap only occurs because we've drawn the upper bound of loss pro= bability for a PWE conservatively high (Appendix A says this). So we don't want to then move straight into recommending a solution in the = following paragraph, which makes it look like we don't believe our own conc= lusion - that PWEs are not a significant capacity sharing problem - even wh= en both PWE and TCP use the same size packets. b) If we then take note of the second qualifying sentence above, it really = does make this an unequivocal non-problem. So we really shouldn't be propos= ing that operators change anything. c) Actually, if we did state your recommendation, it wouldn't be correct to= say the analysis supports it, because packet size problems require conside= ration of packet-congestion, not just bit-congestion. So we would have to r= epeat the full analysis for packet-congestion, not bit-congestion, and show= packet-rate graphs as well as bit-rate graphs. Having done this, and havin= g similarly taken points (a) and (b) into account, we would still come to t= he same conclusion - that there is not a problem significant enough to need= to recommend any action. So, for all these reasons, let's not make this recommendation. Bob Thanks, --David From: Bob Briscoe [ mailto:bob.briscoe@bt.com] Sent: Wednesday, August 20, 2014 2:36 PM To: Black, David Cc: pwe3@ietf.org Subject: RE: [PWE3] FW: WGLC comments on: draft-ietf-pwe3-congcons-02.txt David, Sry for delay. inline... (Despite end of WGLC, we can still continue our public intra-author ping-po= ng :) At 13:44 18/08/2014, Black, David wrote: Bob, 1-4 & 6: ok 5: "acceptable" is a good replacement for S-A-F-E, but I'm not sure about "= friendly". The key concept here is assurance that there are no problems. = How about "clearly acceptable"? That may need some additional text to say = that acceptable network and service behavior may be delivered outside the c= learly acceptable region, but one cannot rely on the analysis in this draft= (based on the TCP throughput formula) to reach such a conclusion. I wanted a different word from 'acceptable' for the adjective in the captio= ns above the Figures describing the regions, because there's no opportunity= for more text there to explain that we do not mean that outside the region= is unacceptable. I thought about various qualifiers with acceptable (like = clearly, unarguably, etc), but I don't think these are right either. I figured that the curves that delineate the region are defined by the TCP-= friendly equation, so 'friendly region' says on the tin what it does (and i= t is OK to imply that outside the region is unfriendly, because unfriendly = doesn't imply "Don't go there" like 'unacceptable' would). Let's stop batting back & forth until we see what Yaakov (or others) think. 7: I disagree - the tsvwg circuit breaker draft is rather general, and we n= ow have two WG LC comments on this draft asking how to apply a circuit brea= ker to PWs. I think we're on solid ground with the references to the ITU-T= service quality standards and expecting operator monitoring of service qua= lity. OTOH, the word "persistently" seemed to be a concern, and I have no = problem removing it (anyone who asks about that can be directed to G.775 an= d G.826). New suggested text: For TDM pseudowires, such a circuit breaker can be realized via operator monitoring of drop rates and the quality of TDM service delivered in order to shut down TDM pseudowires that fail to deliver acceptable TDM service (as defined by [G775] and [G826]). This document explains why a TDM PW that is causing congestion problems via interference with elastic traffic is also not delivering acceptable TDM service (see the rest of this Section, plus Appendices A and B). If we're going to say a bit about how to realize a circuit-breaker, I'd be = happier with the word persistently than without. I'd still rather we didn't= allow this draft to mission creep into mechanism mode. In fact, on reflection, whatever we say about circuit-breakers, I'd prefer = not to mention circuit-breakers in the abstract, which implies this draft s= ays stuff about mechanism, which was never the intent. 8: I believe that figures 1 and 2 suffice to make the point, but I see that= a recommendation may be a bit of a reach for this draft. How about the fo= llowing modified form of my first sentence, which just states the inference= that can be drawn from Figures 1 and 2? - Figures 1 and 2 indicate that a larger segment size for a T1 or E1 TDM PW moves the operational behavior of the PW away from the boundary at which the TCP throughput equation suggests that undue interference wi= th elastic traffic may start to occur. Not so - that's the problem. On these graphs, the op behaviour plots won't = move at all if the PW uses larger packets. These are bit-rate plots, so the= only way to move anything is to change the TCP segment size (which moves t= he TCP plots). To justify what you want to say, either: a) these would have to be packet-rate plots, or b) we would have to turn the assumption we've made about equal packet sizes= into an operational requirement, not just an analytical assumption. To go in either of these directions would require a spade and vermin. Bob PS. GOing offline until next Tue. Might check email, but no guarantee. Thanks, --David From: Bob Briscoe [ mailto:bob.briscoe@bt.com] Sent: Friday, August 15, 2014 6:03 AM To: Black, David Cc: pwe3@ietf.org Subject: Re: [PWE3] FW: WGLC comments on: draft-ietf-pwe3-congcons-02.txt David, I decided to think on a better word for 'safe' overnight, and allow others = to comment. But no-one has, so here's my responses to all your outstanding = points... I've provided slightly updated xml source of my suggested edits (and a diff= ) here: < http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-pwe= 3-congcons-DIFF03b-02.html > < http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-pwe= 3-congcons-03b.txt > < http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-pwe= 3-congcons-03b.xml > Again note: xml is purely for precision, not implying I have taken over edi= torship. 1. Happy with your new scoping text, except the the first 'independent' cla= ngs with the second, and it's unnecessary anyway: " The concern is over PWs that share network capacity with elastic or cong= estion-responsive traffic, [independent of] whether that sharing is a resul= t of the service provider network's traffic engineering and capacity planni= ng or independent deployment of PWs by entities other than network operator= s. " 2. My point wasn't to remove the reference to the 125us (in my suggested te= xt I didn't). It was that the 125us stuff /alone/ doesn't warrant the 'thus= ' in the next clause; the logic needs to be connected by talking about the = sum of the flight time /and/ the per-node processing time. CURRENT: The one-way delay of a native TDM service consists of the physical time-of-flight plus 125 microseconds for each TDM switch traversed; and is thus very small as compared to typical PSN network-crossing latencies. SUGGESTED: The one-way delay of a native TDM service consists of the physical time-of-flight plus 125 microseconds for each TDM switch traversed; so the total one-way del= ay for most TDM PWs has to be small compared to typical PSN network-crossin= g latencies. 5. I understand and agree - 'S-A-F-E region' is incendiary. But I don't lik= e 'reasonable region' - it's slightly /too/ meaningless. Other possibilities are: - admissable region (problem: implies outside the region is inadmissable) - harmonious region (problem: too Californian) So I suggest two words: - friendly or acceptable, each used depending on context. On the graphs, I suggest 'operating region (friendly under curve)' because = that is precisely where the curves come from - 'friendly' does what it says= on the tin, and the implied 'unfriendly' above the curve correctly avoids = implying that the sky will fall. Elsewhere, I suggest the following replacements for the S-word: "It is thus worthwhile specifying under what conditions such competition is= acceptable" "One may view this condition as defining a 'friendly' operating envelope fo= r a TDM PW" "Under this condition it is acceptable to place the TDM PW along with ... T= CP. " "comparison with something widely considered to offer an acceptable respons= e to congestion" 7. Circuit breaker Scope creep. Rat-hole. Stop digging. Yes, I know Stewart has asked us to dig out another rat as well, but the ap= propriate response is "Out of scope". You might think your text is uncontroversial, but if we are tempted to take= Stewart's bait (rat metaphor intended), it will probably start new debates= that this doc doesn't deserve: "What does persistent mean?", "Which traffi= c is more important/ valuable?", yada yada. I suggest the text merely makes it clear that this debate is in scope for e= lsewhere, not here. I've also just realised that Yaakov has written the sen= tences around circuit breaker with the wrong logical sequence. The circuit = breaker needs to be introduced as something the PW uses when its /own/ qual= ity has failed, which should avoid any need to shut down because it is thre= atening elastic services: CURRENT: If the bandwidth consumed by a TDM PW is considered detrimental, the only available remedy is to completely shut down the PW, by using a transport circuit breaker mechanism. However, we will show that even before such an action is warranted, the PW will become unable to faithfully emulate the native TDM service; for example, when a TDM service is carrying voice grade telephony channels, the voice quality will degrade to below useful levels. SUGGESTED: We will show that, before a PW becomes deterimental to elastic traffi= c, it will become unable to faithfully emulate the native TDM service; for example, when a TDM service is carrying voice grade telephony channels, the voice quality will degrade to below useful levels. Once the quality of the emulated service is pers= istently unacceptable, if a PW shuts down selected flows or shuts itself do= wn completely, it will automatically avoid detriment to elastic traffic. A = transport circuit breaker [I-D.fairhurst-tsvwg-circuit-breaker] could be us= ed, but mechanism is beyond the scope of the present document. 8. Segment size. Everyone's in a big group-hug because we've written a draft showing there's= is no need to change anything. Then just at the end of an uneventful WGLC,= we hear David's voice from the bottom of a deep hole shouting, "avoid the = rat-hole invitation", while furiously digging for a rat. There is no need to add a recommendation that PWs need to be changed to use= larger packets. No matter how hedged and soft this recommendation would be= , it's not needed anyway. Stop digging. Yaakov's new text is fine as it stands. "Only for small packets, long delay= s, and high packet loss ratios, do TDM PWs potentially consume more bandwid= th, and even then only marginally. " In order to justify your sentence, we would have to include graphs with pac= ket-rate (not bit-rate) on the vertical axis and go into a whole treatise a= s to why packet-rate comparison might be important as well. Then the graph = would show one plot for all TCP segment sizes and multiple plots for PWs wi= th different packet sizes. Then we'd see that the corner of the two smalles= t packet PWs just clipped the TCP curve at loss probabilities that are stup= idly large anyway. So even after all this, it would hardly justify a need f= or your sentence. All that digging would not be worth it, just to discover (again) that there= is no rat. Bob At 19:39 14/08/2014, Black, David wrote: Content-Language: en-US Content-Type: multipart/alternative; boundary=3D"_000_8D3D17ACE214DC429325B2B98F3AE712077B95212CMX15Aco= rpemcc_" And forwarding my views to the list ... ... On Bob's six thoughts (I have problems with 2 & 5 - the other 4 are bas= ically ok w/me) plus follow-ups on Bob's concerns w/my two major issues, he= re goes: -- 1 -- I agree in principle, but I think Bob's suggested new text is too b= lack/white and should be generalized. Bob suggests: > The case where the service provider network arranges sufficient capacity = for a PW is well understood and > will not be discussed further here. The concern is over PWs deployed inde= pendently of the service provider > network's traffic engineering or capacity planning. I want to include the case where capacity has been arranged, but may not be= sufficient, I suggest rewriting Bob's second sentence as: The concern is over PWs that share network capacity with elastic or congest= ion-responsive traffic, independent of whether that sharing is a result of = the service provider network's traffic engineering and capacity planning or= independent deployment of PWs by entities other than network operators. -- 2 -- Here, I disagree. One of the significant contributions of this dra= ft is that TDM PWs don't cause congestion problems in practice, and one of = the major reasons for that is the low latency requirements for acceptable T= DM service delivery, both in ITU-T standards and in actual TDM PW deploymen= t. The 125us behavior of native TDM switches is relevant to the latter, an= d hence I prefer to include mention of it. -- 3 -- I agree; clear captions are always a feature, not a bug ;-). -- 4 -- I agree with changing "packet loss rate" to "packet loss probabilit= y". -- 5 -- I firmly disagree with Bob here, although he's right that this is "= techno-political" ;-). I am concerned that S-A-F-E is likely to be a 4-letter word in wider Transp= ort Area review, and hence, I want to use a word that does not imply anythi= ng remotely resembling an absolute guarantee that nothing can possibly ever= go wrong (yes, I know Bob didn't mean to go to quite that extreme an inter= pretation, but S-A-F-E is liable to be read that way by those with an "axe = to grind"), even though there's lots of solid operational experience that s= uggests all is well here. -- 6 -- I think I'm ok with what Bob proposes to better characterize loss b= urstiness, but want to see specific text. -- Circuit Breaker Mechanism -- >> [David] I suggest adding: >> >> For TDM pseudowires, such a circuit breaker can be realized via >> operator monitoring of drop rates and consequent on the quality of >> TDM service delivered and shutting down TDM pseudowires that persiste= ntly >> fail to deliver acceptable TDM service (as defined by [G775] and >> [G826]). This document explains why a TDM PW that is causing congest= ion >> problems via interference with elastic traffic cannot be delivering >> acceptable TDM service (see the rest of this Section, plus Appendices >> A and B). >> > [Bob] I don't think we should go there. We need to keep this draft firmly= as informational. If someone > thinks a mechanism is needed because of the information in this draft, th= en they can write about a mechanism. [David] I firmly disagree, but Bob and I might be talking past each other, = as I do want to stay informational ;-). This text is intended to describe an example or possibility, not to specify= an implementation requirement. The new text was intended as an OAM/OSS pr= actice recommendation to the operator, not as a normative protocol spec. I'm ok with inserting "example", "possibility" or something similar if that= helps w/Bob's concern, but I strongly oppose doing nothing here, and note = that this is also needed to respond to one of Stewart Bryant's WG LC commen= ts. -- PW segment size -- >> [David] Ignoring the rathole invitation ;-). This is still in section 3= , after: >> >> We see that in general for large packet sizes, short delays, and low >> packet loss rates, the TDM pseudowires consume much less bandwidth >> than TCP would under identical conditions. Only for small packets, >> long delays, and high packet loss ratios, do TDM PWs potentially >> consume more bandwidth, and even then only marginally. >> >> I suggest adding: >> >> Comparing figures 1 and 2, use of a larger segment size for a T1 or E= 1 >> TDM PW moves the operational behavior of the PW away from the boundar= y >> at which the TCP throughput equation suggests that undue interference= with >> elastic traffic may start to occur. We suggest that larger segment >> sizes be preferred for T1 and E1 TDM PWs, but note that other >> considerations may favor smaller segment sizes (e.g., jitter buffer >> size, bounding the impact of an isolated packet drop event). >> > [Bob] There is no need to recommend any remedial action, when we have jus= t shown there is not a problem. [David] Not exactly - Figure 1 shows that there is a potential concern at a= combination of the high end of the range of TDM PW latencies (4ms), and hi= gher packet loss ratios. > Further, the "marginal problem" only applies when the TCP traffic uses pa= ckets smaller than 128B, which we only used > as a hypothetical worst-case for the analysis. But in practice this doesn= 't happen with long-running TCP flows - > they use packets that are as large as possible (irrespective of what the = PW uses). So there really is /no/ problem to remedy. Ok, Bob can relax - his transport credentials are intact, as this is an ent= ry point to the bit-rate-congestible vs. packet-rate-congestible resource r= athole that I want to ignore :-). I believe that larger TDM PW segment sizes are a good thing for both this o= perating envelope boundary concern as well as per-packet processing overhea= ds in TDM PW functionality, modulo other considerations, e.g., jitter buffe= rs, and so I prefer to keep this addition. Thanks, --David From: Bob Briscoe [mailto:bob.briscoe@bt.com] Sent: Thursday, August 14, 2014 9:03 AM To: Yaakov Stein Cc: Andrew G. Malis; BOCCI Matthew; Black, David Subject: RE: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt Yaakov, I have just re-read it thoroughly. I have noticed about two dozen editorial= nits where correcting them would not change the intended sense (altho some= times the sense needs to change to say what we intended!). Examples: - Unexpanded abbreviations (David's suggestion would help, but there are mo= re) - Missing words, tautologies, random punctuation problems etc - Uses of the wrong word, even tho we know what you mean - The caveat about dynamics vs steady state needs to shift down a para, so = that it follows into the para that says what to do about dynamics. Should I take the token, and edit the XML directly, then you can see a diff= to check if you want to accept my suggestions? Below are six thoughts that go slightly beyond 'editorial'. 1-4 could be co= nsidered 'editorial'. But 5&6 are the more significant "techno-political" p= oints. 1. The phrase "mechanisms for provisioning PWs in service provider networks= " is too opaque (the PWE audience might understand this scoping sentence, b= ut a L4 reader won't understand what we're excluding here, which will make = them suspicious). CURRENT: Mechanisms for provisioning PWs in service provider networks are well understood and will not be discussed further here. SUGGESTED: The case where the service provider network arranges sufficient capacity= for a PW is well understood and will not be discussed further here. The co= ncern is over PWs deployed independently of the service provider network's = traffic engineering or capacity planning. 2. The justification for using OWD of a few milliseconds misses the point. " The one-way delay of a native TDM service consists of the physical time-of-flight plus 125 microseconds for each TDM switch traversed; and is thus very small as compared to typical PSN network-crossing latencies. " The 125us per switch is irrelevant. The point that needs making is that PWs= are generally deployed with a short physical time of flight. 3. Related point: the captions on the graphs do not clearly indicate that t= hey are OWD, not RTT. This could be clarified in the text, or on the graphs= . 4. Change packet loss rate (and PLR) to packet loss probability throughout,= including on the graph axes. PLR is an ambiguous term, even though it is w= idely used in the industry. Packet loss rate could be measured in packet pe= r second, whereas packet loss probability is unambiguously dimensionless. I= ndeed, in the appendices, packet loss rates with respect to time are calcul= ated. 5. Rather than David's "reasonable" I suggest we use the phrase "unambiguou= sly safe" not just "safe" for the headings of Figs 5-8 and in the text, as = below. Otherwise, we are implying that it is unsafe outside the region. CURRENT: One may view this condition as defining an operating envelope for a TDM PW, as a TDM PW that occupies no more bandwidth than a TCP flow causes no more congestion than that TCP flow would. Under this condition it is safe to place the TDM PW along with congestion- responsive traffic such as TCP, without causing additional congestion. on the other hand, were the TDM PW to consume significantly more bandwidth a TCP flow, it could contribute disproportionately to congestion, and its mixture with congestion- responsive traffic might be inappropriate. SUGGESTED: One may view this condition as defining an unambiguously safe operating = envelope for a TDM PW, as a TDM PW that occupies no more bandwidth than a TCP flow causes no more congestion than that TCP flow would. This does not draw = us into a debate over whether TCP-friendliness is a valid constraint; it si= mply says that there can be no question that a PW is safe if it does no mor= e harm than a single TCP flow. Under this condition it is unambiguously safe to place the TDM PW along with conges= tion- responsive traffic such as TCP. On the other hand, were the TDM PW to c= onsume significantly more bandwidth a TCP flow, it could contribute disproportionately to congestion, and its mixture with congestion- responsive traffic might start to cause concern. 6. The analysis in Appendices A & B needs to mention that loss bursts arise= primarily from tail drop buffers, and the introduction of active queue man= agement (AQM) makes losses more independent of each other (less bursty). (I= have references showing evidence of this.) In the comparisons with TCP, we= have assumed independent losses anyway, because they are the worst case (a= nd fortunately they are easier to analyse). On a similar point, in appendix= B, where we say "(bursty packet loss was also simulated but is not reporte= d here)", we ought to give a reason otherwise readers will be suspicious. A= good reason would be that uniformly random is the worst case. I've also responded to a couple of David's suggestions inline (I agree with= the others)... At 17:44 13/08/2014, Yaakov Stein wrote: Bob told me on Friday of the IETF that he was going to look over the final = version and comment. Bob - ping ? Y(J)S From: Andrew G. Malis [ mailto:agmalis@gmail.com] Sent: 13 August, 2014 17:02 To: Yaakov Stein Cc: Bob Briscoe; rbriscoe@jungle.bt.co.uk;= BOCCI Matthew; Black, David Subject: Re: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt Yaakov, I still haven't heard from Bob, but you should incorporate David's updates = into a new revision, and I'll use that version for my final Shepherd's revi= ew. Thanks, Andy On Mon, Aug 11, 2014 at 11:16 AM, Andrew G. Malis > wrote: Actually, I'm going to end the LC as soon as I hear back from Bob on the IP= R call. I sent him a reminder earlier this morning. Cheers, Andy On Sun, Aug 10, 2014 at 9:53 AM, Andrew G. Malis > wrote: David, That's perfect, thanks! I'm going to end the LC tomorrow. Cheers, Andy On Sat, Aug 9, 2014 at 10:32 PM, Black, David > wrote: Following up on this ... > As we're now moving quickly to try to complete this draft , I agreed > w/Yaakov that my author comments would be sent in as WG Last Call comment= s ... > ... so here they are: ... with some suggested new text ... > Major issues: > > The reference to a transport circuit breaker in section 3 is a bit optimi= stic > and high level: > > We can accomplish this by > employing a transport circuit breaker, by which we mean an automati= c > mechanism for terminating a flow to prevent negative impact on othe= r > flows and on the stability of the network > [I-D.fairhurst-tsvwg-circuit-breaker]. > > This should be accompanied by at least an example of how to do this - pas= t > discussion has suggested carrier monitoring of delivered service quality > in order to shut down PWs that are experiencing loss rates that result in > unacceptable service quality (as defined for us by the ITU ...). I'd als= o > like to add some text to suggest that operators use this sort of circuit > breaker practice when TDM PWs compete with elastic traffic. After this text in Section 3: We can accomplish this by employing a transport circuit breaker, by which we mean an automatic mechanism for terminating a flow to prevent negative impact on other flows and on the stability of the network [I-D.fairhurst-tsvwg-circuit-breaker]. Note that a transport circuit breaker is intended as a protection mechanism of last resort, just as an electrical circuit breaker is only triggered when absolutely necessary. I suggest adding: For TDM pseudowires, such a circuit breaker can be realized via operator monitoring of drop rates and consequent on the quality of TDM service delivered and shutting down TDM pseudowires that persistentl= y fail to deliver acceptable TDM service (as defined by [G775] and [G826]). This document explains why a TDM PW that is causing congestion problems via interference with elastic traffic cannot be delivering acceptable TDM service (see the rest of this Section, plus Appendices A and B). I don't think we should go there. We need to keep this draft firmly as info= rmational. If someone thinks a mechanism is needed because of the informati= on in this draft, then they can write about a mechanism. And breaking the paragraph after the added text. > Figure 1 and Figure 2 indicate that based on using the TCP throughput > equation as a proxy for potential impact on elastic traffic (which is wha= t > we're doing in this draft), for the same fixed bandwidth PW, a larger seg= ment > size is less likely to cause impacts. I'd like to add a suggestion to us= e > the larger PW segment sizes as long as they're consistent with delivery o= f > the service. There's been a related (long-to-the-point-of-ratholed) deba= te > over in the Transport Area about whether packet-rate-limited/congestible > forwarding is still relevant. My view is that such equipment exists, and > hence that this is a relevant consideration for this draft, and would not > mention that issue here. Ignoring the rathole invitation ;-). This is still in section 3, after: We see that in general for large packet sizes, short delays, and low packet loss rates, the TDM pseudowires consume much less bandwidth than TCP would under identical conditions. Only for small packets, long delays, and high packet loss ratios, do TDM PWs potentially consume more bandwidth, and even then only marginally. I suggest adding: Comparing figures 1 and 2, use of a larger segment size for a T1 or E1 TDM PW moves the operational behavior of the PW away from the boundary at which the TCP throughput equation suggests that undue interference wi= th elastic traffic may start to occur. We suggest that larger segment sizes be preferred for T1 and E1 TDM PWs, but note that other considerations may favor smaller segment sizes (e.g., jitter buffer size, bounding the impact of an isolated packet drop event). There is no need to recommend any remedial action, when we have just shown = there is not a problem. Further, the "marginal problem" only applies when the TCP traffic uses pack= ets smaller than 128B, which we only used as a hypothetical worst-case for = the analysis. But in practice this doesn't happen with long-running TCP flo= ws - they use packets that are as large as possible (irrespective of what t= he PW uses). So there really is /no/ problem to remedy. and again, breaking the paragraph after that added text. I hope that latter sentence ("We suggest that ...") is acceptable to the WG= . > Minor issues: > > Section 2 is about Ethernet PWs that carry elastic traffic. Should other= s > be mentioned as also covered by this analysis - e.g., ATM and Frame Relay > PWs carrying elastic traffic? After: In this section we consider Ethernet PWs that primarily carry congestion-responsive traffic. We show that we automatically obtain the desired congestion avoidance behavior, and that additional mechanisms are not needed. I suggest adding: This discussion is applicable to any PW that carries congestion- responsive traffic (e.g., ATM and Frame Relay PWs); Ethernet PWs are used here for clarity, as they are a common example. > Section 2 does not cover Ethernet PWs carrying inelastic traffic - that w= ould > be unusual, but should it be explicitly noted as out of scope, or just no= t > discussed (as is the case for the current text)? I suggest doing nothing here, and planning to deal with this concern if it should arise later (e.g., IETF Last Call or IESG Evaluation). > Section 3 should say that the reason the draft focuses on TDM PWs is that > they're the inelastic PWs that are likely to be carried over IP in practi= ce. > My understanding is that other potentially inelastic PWs are MPLS-only > in practice. After: Inelastic PWs, such as TDM PWs ([RFC4553][RFC5086][RFC5087]), are potentially more problematic than the elastic PWs of the previous section. Being constant bit-rate (CBR), TDM PWs can not be made responsive to congestion. On the other hand, being CBR, they also do not attempt to capture additional bandwidth when neighboring TCP flows back off. I suggest adding: We focus on TDM PWs here as they are the only inelastic PWs that are likely to be IP-encapsulated in practice. Most other inelastic PWs tend to only be used with MPLS. [I'd ask Yaakov and other PW experts to double-check the latter sentence, please.] > A terminology section to roll up all the acronym expansions (e.g., TDM, C= BR) > in one place seems like a good idea. These four acronyms would be good to put into a terminology section: CBR Constant Bit Rate PSN Packet Switched Network PW Pseudowire TDM Time Division Multiplexing > > This text on p.11 could be incendiary: > > Under this condition it is safe to place the TDM PW along with > congestion - responsive traffic such as TCP, without causing > additional congestion. > > First of all to reduce the flammability, "safe" -> "reasonable" and there > are a number of other occurrences of "safe" that deserve similar treatmen= t ;- > ). > Beyond that, this assertion is dependent on the operating conditions stay= ing > within that reasonable operating envelope, which entails use of something > like a circuit breaker - I'd say that here, and reference the earlier > circuit breaker discussion. Global substitution: "safe" -> "reasonable", starting with the abstract. Then for this specific text on p.11: OLD Under this condition it is safe to place the TDM PW along with congestion - responsive traffic such as TCP, without causing additional congestion. NEW Under this condition it is reasonable to place the TDM PW in compet= ition with congestion-responsive traffic such as TCP, as the result should = not produce unacceptable congestion effects. That assumes that the opera= ting conditions remain within the reasonable operating envelope - as discu= ssed above, a transport circuit breaker is a suggested means of managing a= TDM PW so that it is shut down if operating conditions remain persistentl= y outside the reasonable operating envelope for the PW (e.g., due to a high packet loss rate). No need to go there. See my suggestions earlier. > Figures 7 and 8 could use some shading or lines to indicate the typical > deployed operating regions for the pseudo-wires to visibly show that they > fall under the curves, particularly for larger segment sizes. I'd also > repeat or refer to the suggestion to use larger I wanted to suggest this myself, but it is hard to depict fuzzy boundaries.= If you can do this, pls do. Given the PW operating regions are bounded by v low loss probability, the x= axis in Figs 7&8 ought to be scaled much smaller as well. Related point: the y axis in Fig 1 ought to max at 10, not 50. Regards Bob > > Nits: > > References aren't allowed in abstracts, so remove reference to > [I-D.fairhurst-tsvwg-circuit-breaker] > > Introduction, first paragraph, delete "PWE3" towards end of paragraph - > > preferably the 4 byte PWE3 control word. > > In addition to its current general statements, the Security Consideration= s > section should point to the security considerations sections in a few > particularly relevant RFCs - I'd suggest 3985, 4553, 5086 and 5087. After: This document does not introduce any new congestion-specific mechanisms and thus does not introduce any new security considerations above those present for PWs in general. I suggest adding a new paragraph: For PW security considerations, see the security considerations sections of the cited RFCS; the security considerations in [RFC3985], [RFC4553], [RFC5086] and [RFC5087] are particularly applicable to this document. > > Thanks, > --David > > > > -----Original Message----- > > From: pwe3 [ mailto:pwe3-bounces@ietf.org] On Behalf Of Yaakov Stein > > Sent: Thursday, July 24, 2014 4:52 PM > > To: pwe3@ietf.org > > Subject: Re: [PWE3] I-D Action: draft-ietf-pwe3-congcons-02.txt > > > > Please look at the PDF version only > > (the txt version does not have the graphs). > > > > Y(J)S > > > > > > _______________________________________________ > pwe3 mailing list > pwe3@ietf.org > https://www.ietf.org/mailman/listinfo/pwe3 _______________________________________________ pwe3 mailing list pwe3@ietf.org https://www.ietf.org/mailman/listinfo/pwe3 ________________________________________________________________ Bob Briscoe, BT _______________________________________________ pwe3 mailing list pwe3@ietf.org https://www.ietf.org/mailman/listinfo/pwe3 ________________________________________________________________ Bob Briscoe, BT ________________________________________________________________ Bob Briscoe, BT ________________________________________________________________ Bob Briscoe, BT --_000_8D3D17ACE214DC429325B2B98F3AE712077BC6669AMX15Acorpemcc_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Bob,

 

I’= ;ll yield to your views on point 8 to get the draft done.  I continue = to believe that a recommendation for larger PW segment sizes is appropriate= , but it’s at best secondary to the primary purpose of this draft, wh= ich is to explain why TDM PWs do not pose congestion problems in practice (= as you point out).

 <= /span>

Yaakov - when do you think you’ll be able = to submit a revised draft?  I’m about to vanish for 3 weeks of v= acation w/little to no email access.

=  

Thanks,
--David

 

From: Bob Briscoe [mailto:bob.= briscoe@bt.com]
Sent: Tuesday, August 26, 2014 2:32 PM
To:= Black, David
Cc: pwe3@ietf.org; Black, David
Subject:<= /b> RE: [PWE3] FW: WGLC comments on: draft-ietf-pwe3-congcons-02.txt

 

David,

At 19:59 20/08/2014, Black, David wrote:

Bob,
 
I think we’re= close - with luck, this email provides resolutions in principle to 5 and 7= ,
leaving 8, where I don’t understand your argument.


...I'd like to hear from Yaakov before we start d= eclaring consensus - it's been just two of us so far!


 
-- On 5:
 
> I figured th= at the curves that delineate the region are defined by the TCP-friendly equ= ation, so 'friendly region' says on
> the tin what it does (and it is= OK to imply that outside the region is unfriendly, because unfriendly does= n't imply
> "Don't go there" like 'unacceptable' would). 
“TCP-friendly” and “possibly TCP-unfriendly= 221; would be fine with me -
I’m just uncomfortable with use of va= nilla unqualified “friendly.”


I'm sold on TCP-friendly region.


 
-- On 7, in reverse order:
 
> In fact= , on reflection, whatever we say about circuit-breakers, I'd prefer not to = mention circuit-breakers in the
> abstract, which implies this draft = says stuff about mechanism, which was never the intent.
 
Remova= l of mention of circuit breakers from the abstract is fine with me.
&nbs= p;
> If we're going to say a bit about how to realize a circuit-break= er, I'd be happier with the word persistently
> than without. I'd sti= ll rather we didn't allow this draft to mission creep into mechanism mode.<= br> 
I continue to agree with Stewart that something needs to be sa= id here - the proposed
text leaves enormous flexibility to the operator = to decide when to shut down TDM PWs.

That's my point. A draft that isn't about mechanism shouldn't tell a devel= oper or an operator what they can or can't do about mechanism. If we want t= o talk about that, we will need another draft (perhaps a PWE example in the= circuit-breakers draft, which has an intended status of Standards Track).<= br>

Here it is with “persi= stently” restored:
 
   For TDM pseudowires, suc= h a circuit breaker can be realized via
   operator monitoring= of drop rates and the quality of
   TDM service delivered in = order to shut down TDM pseudowires that
   persistently fail t= o deliver acceptable TDM service (as defined by [G775] and
   = [G826]).  This document explains why a TDM PW that is causing congesti= on
   problems via interference with elastic traffic is also n= ot delivering
   acceptable TDM service (see the rest of this = Section, plus Appendices
   A and B).


I can live with this much text. But don't blame me if this= causes the draft to get referred back or questioned as it goes through sub= sequent phases of the RFC process.


 
-- On 8, I think I see the start of our disagreement:
&= nbsp;
> On these graphs, the op behaviour plots won't move at all if = the PW uses larger packets. These are
> bit-rate plots, so the only w= ay to move anything is to change the TCP segment size (which moves the TCP = plots).
 
Actually, they’re bit-rate plots on the assumpti= on that both the PW and TCP are using the
same segment size, which is wh= y Figure 2 looks better for the PWs than Figure 1 (the PW
lines did not = move, but the TCP curves did move due to the segment size change).
 = ;
Or in other words:
- Figure 1 compares a PW using 64-byte segments = to a TCP connection using 64-byte segments.
- Figure 2 compares a PW usi= ng 256-byte segments to a TCP connection using 256-byte segments.
- Noth= ing in the draft makes a comparison using different segment sizes.
 = ;
So, “if the PW uses larger packets” then the approach in t= his draft compares it to a TCP
connection using larger segment, and the = result looks like Figure 2 instead of Figure 1.
 
I do not like = the idea of comparing a PW using 256-byte segments with a TCP connectionusing 64-byte segments, which is what I understand that above “won&#= 8217;t move at all” assertion
to be about.  The TCP curves do= move between Figure 1 and Figure 2, based solely on a TCP
segment size = change.


Correct. We're getting somew= here. The draft actually says two sentences about this assumption:
"= ;   In order to quantitatively compare TDM PWs to TCP flows, we w= ill
   compare the effect of TDM PW packets with that of TCP p= ackets of the
   same packet size and sent at the same rate.&n= bsp; This is potentially an
   overly pessimistic comparison, = as TDM PW packets are frequently
   configured to be short in = order to minimize latency, while TCP
   packets are free to be= much larger.
"

You are focusing solely on the first sentenc= e, which we only used to simplify presentation of the analysis.

I a= m saying any recommendations should be based on the analysis /after/ having= adjusted the outcome to reverse out any assumptions that otherwise make th= e analysis unrealistic (ie. take account of the second quoted sentence).
But my case is even stronger than that:

a) Even if we solely ma= ke the assumption of comparable packet sizes (for the sake of analysis) and= ignore the second qualifying sentence (the reality), then as we continue t= hrough the whole doc we still get to the point where we show the problem is= virtually non-existent. Given:
  * TCP-friendliness is not an abso= lute requirement - one bit-rate within half-a-dozen times the other is fine= (I recall Sally FLoyd writing somewhere that x8 shouldn't be a problem)   * the overlap only occurs because we've drawn the upper bound of l= oss probability for a PWE conservatively high (Appendix A says this).
So we don't want to then move straight into recommending a solution in th= e following paragraph, which makes it look like we don't believe our own co= nclusion - that PWEs are not a significant capacity sharing problem - even = when both PWE and TCP use the same size packets.

b) If we then take = note of the second qualifying sentence above, it really does make this an u= nequivocal non-problem. So we really shouldn't be proposing that operators = change anything.

c) Actually, if we did state your recommendation, i= t wouldn't be correct to say the analysis supports it, because packet size = problems require consideration of packet-congestion, not just bit-congestio= n. So we would have to repeat the full analysis for packet-congestion, not = bit-congestion, and show packet-rate graphs as well as bit-rate graphs. Hav= ing done this, and having similarly taken points (a) and (b) into account, = we would still come to the same conclusion - that there is not a problem si= gnificant enough to need to recommend any action.

So, for all these = reasons, let's not make this recommendation.



Bob


=

 
Thanks,
--David
 <= br>From: Bob Briscoe [ mailto:= bob.briscoe@bt.com]
Sent: Wednesday, August 20, 2014 2:36 PM=
To: Black, David
Cc: = pwe3@ietf.org
Subject: RE: [PWE3] FW: WGLC comments on: draft= -ietf-pwe3-congcons-02.txt
 
David,

Sry for delay. inline= ...

(Despite end of WGLC, we can still continue our public intra-aut= hor ping-pong :)

At 13:44 18/08/2014, Black, David wrote:

Bob= ,
 
1-4 & 6: ok
 
5: “acceptable” is = a good replacement for S-A-F-E, but I’m not sure about “friendl= y”.  The key concept here is assurance that there are no problem= s.  How about “clearly acceptable”?  That may need so= me additional text to say that acceptable network and service behavior may = be delivered outside the clearly acceptable region, but one cannot rely on = the analysis in this draft (based on the TCP throughput formula) to reach s= uch a conclusion.

I wanted a different word from 'acceptable' for th= e adjective in the captions above the Figures describing the regions, becau= se there's no opportunity for more text there to explain that we do not mea= n that outside the region is unacceptable. I thought about various qualifie= rs with acceptable (like clearly, unarguably, etc), but I don't think these= are right either.

I figured that the curves that delineate the regi= on are defined by the TCP-friendly equation, so 'friendly region' says on t= he tin what it does (and it is OK to imply that outside the region is unfri= endly, because unfriendly doesn't imply "Don't go there" like 'un= acceptable' would).

Let's stop batting back & forth until we see= what Yaakov (or others) think.



7: I disagree - the tsvwg ci= rcuit breaker draft is rather general, and we now have two WG LC comments o= n this draft asking how to apply a circuit breaker to PWs.  I think we= ’re on solid ground with the references to the ITU-T service quality = standards and expecting operator monitoring of service quality.  OTOH,= the word “persistently” seemed to be a concern, and I have no = problem removing it (anyone who asks about that can be directed to G.775 an= d G.826).  New suggested text:
 
   For TDM pseud= owires, such a circuit breaker can be realized via
   operator= monitoring of drop rates and the quality of
   TDM service de= livered in order to shut down TDM pseudowires that
   fail to = deliver acceptable TDM service (as defined by [G775] and
   [G= 826]).  This document explains why a TDM PW that is causing congestion=
   problems via interference with elastic traffic is also not= delivering
   acceptable TDM service (see the rest of this Se= ction, plus Appendices
   A and B).

If we're going to s= ay a bit about how to realize a circuit-breaker, I'd be happier with the wo= rd persistently than without. I'd still rather we didn't allow this draft t= o mission creep into mechanism mode.

In fact, on reflection, whateve= r we say about circuit-breakers, I'd prefer not to mention circuit-breakers= in the abstract, which implies this draft says stuff about mechanism, whic= h was never the intent.


 
8: I believe that figures 1 an= d 2 suffice to make the point, but I see that a recommendation may be a bit= of a reach for this draft.  How about the following modified form of = my first sentence, which just states the inference that can be drawn from F= igures 1 and 2? -
  
   Figures 1 and 2 indicate= that a larger segment size for a T1 or E1
   TDM PW moves the= operational behavior of the PW away from the boundary
   at w= hich the TCP throughput equation suggests that undue interference with
&= nbsp;  elastic traffic may start to occur.

Not so - that's the= problem. On these graphs, the op behaviour plots won't move at all if the = PW uses larger packets. These are bit-rate plots, so the only way to move a= nything is to change the TCP segment size (which moves the TCP plots). To j= ustify what you want to say, either:
a) these would have to be packet-ra= te plots, or
b) we would have to turn the assumption we've made about eq= ual packet sizes into an operational requirement, not just an analytical as= sumption.

To go in either of these directions would require a spade = and vermin.


Bob

PS. GOing offline until next Tue. Might c= heck email, but no guarantee.


 
Thanks,
--David
&n= bsp;
From: Bob Briscoe [ ma= ilto:bob.briscoe@bt.com]
Sent: Friday, August 15, 2014 6:03 = AM
To: Black, David
Cc: pwe3@ietf.org
Subject: Re: [PWE3] FW: WGLC comments on: dra= ft-ietf-pwe3-congcons-02.txt
 
David,

I decided to think = on a better word for 'safe' overnight, and allow others to comment. But no-= one has, so here's my responses to all your outstanding points...

I'= ve provided slightly updated xml source of my suggested edits (and a diff) = here:
< http://www.bobbriscoe.ne= t/projects/2020comms/pwe3-congcons/draft-ietf-pwe3-congcons-DIFF03b-02.html= >
< http://www.bobbriscoe.net/pr= ojects/2020comms/pwe3-congcons/draft-ietf-pwe3-congcons-03b.txt >< http://www.bobbriscoe.net/projects/2020c= omms/pwe3-congcons/draft-ietf-pwe3-congcons-03b.xml >
Again note:= xml is purely for precision, not implying I have taken over editorship.
1. Happy with your new scoping text, except the the first 'independent= ' clangs with the second, and it's unnecessary anyway:
"  The = concern is over PWs that share network capacity with elastic or congestion-= responsive traffic, [independent of] whether that sharing is a result of th= e service provider network's traffic engineering and capacity planning or i= ndependent deployment of PWs by entities other than network operators.
&= quot;

2. My point wasn't to remove the reference to the 125us (in my= suggested text I didn't). It was that the 125us stuff /alone/ doesn't warr= ant the 'thus' in the next clause; the logic needs to be connected by talki= ng about the sum of the flight time /and/ the per-node processing time.
= CURRENT:
   The one-way delay
   of a native TDM = service consists of the physical time-of-flight plus
   125 mi= croseconds for each TDM switch traversed; and is thus very
   = small as compared to typical PSN network-crossing latencies.
SUGGESTED:<= br>   The one-way delay
   of a native TDM service = consists of the physical time-of-flight plus
   125 microseco= nds for each TDM switch traversed; so the total one-way delay for most TDM = PWs has to be small compared to typical PSN network-crossing latencies.
5. I understand and agree - 'S-A-F-E region' is incendiary. But I don'= t like 'reasonable region' - it's slightly /too/ meaningless.

Other = possibilities are:
- admissable region (problem: implies outside the reg= ion is inadmissable)
- harmonious region (problem: too Californian)
S= o I suggest two words:
- friendly or acceptable, each used depending on = context.

On the graphs, I suggest 'operating region (friendly under= curve)' because that is precisely where the curves come from - 'friendly' = does what it says on the tin, and the implied 'unfriendly' above the curve = correctly avoids implying that the sky will fall.

Elsewhere, I sugge= st the following replacements for the S-word:
"It is thus worthwhil= e specifying under what conditions such competition is acceptable"
= "One may view this condition as defining a 'friendly' operating envelo= pe for a TDM PW"
"Under this condition it is acceptable to pla= ce the TDM PW along with ... TCP. "
"comparison with something= widely considered to offer an acceptable response to congestion"
<= br>
7. Circuit breaker
Scope creep. Rat-hole. Stop digging.
Yes, = I know Stewart has asked us to dig out another rat as well, but the appropr= iate response is "Out of scope".

You might think your text= is uncontroversial, but if we are tempted to take Stewart's bait (rat meta= phor intended), it will probably start new debates that this doc doesn't de= serve: "What does persistent mean?", "Which traffic is more = important/ valuable?", yada yada.

I suggest the text merely mak= es it clear that this debate is in scope for elsewhere, not here. I've also= just realised that Yaakov has written the sentences around circuit breaker= with the wrong logical sequence. The circuit breaker needs to be introduce= d as something the PW uses when its /own/ quality has failed, which should = avoid any need to shut down because it is threatening elastic services:
=
CURRENT:
      If the bandwidth consumed by= a TDM PW is
      considered detrimental, the = only available remedy is to completely
      sh= ut down the PW, by using a transport circuit breaker mechanism.
 &n= bsp;    However, we will show that even before such an actio= n is
      warranted, the PW will become unable= to faithfully emulate the
      native TDM ser= vice; for example, when a TDM service is carrying
   &nbs= p;  voice grade telephony channels, the voice quality will degrade to<= br>      below useful levels.
SUGGESTED:
&n= bsp;     We will show that, before a PW becomes deterim= ental to elastic traffic, it will become unable to faithfully emulate the      native TDM service; for example, when a TD= M service is carrying
      voice grade telepho= ny channels, the voice quality will degrade to
    &= nbsp; below useful levels. Once the quality of the emulated service is pers= istently unacceptable, if a PW shuts down selected flows or shuts itself do= wn completely, it will automatically avoid detriment to elastic traffic. A = transport circuit breaker [I-D.fairhurst-tsvwg-circuit-breaker] could be us= ed, but mechanism is beyond the scope of the present document.


8= . Segment size.
Everyone's in a big group-hug because we've written a dr= aft showing there's is no need to change anything. Then just at the end of = an uneventful WGLC, we hear David's voice from the bottom of a deep hole sh= outing, "avoid the rat-hole invitation", while furiously digging = for a rat.

There is no need to add a recommendation that PWs need to= be changed to use larger packets. No matter how hedged and soft this recom= mendation would be, it's not needed anyway. Stop digging.

Yaakov's n= ew text is fine as it stands. "Only for small packets, long delays, an= d high packet loss ratios, do TDM PWs potentially consume more bandwidth, a= nd even then only marginally. "

In order to justify your sente= nce, we would have to include graphs with packet-rate (not bit-rate) on the= vertical axis and go into a whole treatise as to why packet-rate compariso= n might be important as well. Then the graph would show one plot for all TC= P segment sizes and multiple plots for PWs with different packet sizes. The= n we'd see that the corner of the two smallest packet PWs just clipped the = TCP curve at loss probabilities that are stupidly large anyway. So even aft= er all this, it would hardly justify a need for your sentence.

All = that digging would not be worth it, just to discover (again) that there is = no rat.


Bob


At 19:39 14/08/2014, Black, David wrote:=

Content-Language: en-US
Content-Type: multipart/alternative;
=          boundary=3D"_000_8D3D= 17ACE214DC429325B2B98F3AE712077B95212CMX15Acorpemcc_"

And forwa= rding my views to the list ...
 
... On Bob’s six thoughts= (I have problems with 2 & 5 - the other 4 are basically ok w/me) plus = follow-ups on Bob’s concerns w/my two major issues, here goes:
&nb= sp;
-- 1 -- I agree in principle, but I think Bob’s suggested new = text is too black/white and should be generalized.
 
Bob suggest= s:
 
> The case where the service provider network arranges s= ufficient capacity for a PW is well understood and
> will not be disc= ussed further here. The concern is over PWs deployed independently of the s= ervice provider
> network's traffic engineering or capacity planning.=
 
I want to include the case where capacity has been arranged, = but may not be sufficient, I suggest rewriting Bob’s second sentence = as:
 
The concern is over PWs that share network capacity with e= lastic or congestion-responsive traffic, independent of whether that sharin= g is a result of the service provider network's traffic engineering and cap= acity planning or independent deployment of PWs by entities other than netw= ork operators.
 
-- 2 -- Here, I disagree.  One of the sign= ificant contributions of this draft is that TDM PWs don’t cause conge= stion problems in practice, and one of the major reasons for that is the lo= w latency requirements for acceptable TDM service delivery, both in ITU-T s= tandards and in actual TDM PW deployment.  The 125us behavior of nativ= e TDM switches is relevant to the latter, and hence I prefer to include men= tion of it.
 
-- 3 -- I agree; clear captions are always a featu= re, not a bug ;-).
 
-- 4 -- I agree with changing “packet= loss rate” to “packet loss probability”.
 
--= 5 -- I firmly disagree with Bob here, although he’s right that this = is “techno-political” ;-).
 
I am concerned that S-A= -F-E is likely to be a 4-letter word in wider Transport Area review, and he= nce, I want to use a word that does not imply anything remotely resembling = an absolute guarantee that nothing can possibly ever go wrong (yes, I know = Bob didn’t mean to go to quite that extreme an interpretation, but S-= A-F-E is liable to be read that way by those with an “axe to grind= 221;), even though there’s lots of solid operational experience that = suggests all is well here.
 
-- 6 -- I think I’m ok with w= hat Bob proposes to better characterize loss burstiness, but want to see sp= ecific text.
 
-- Circuit Breaker Mechanism --
 
>= > [David] I suggest adding:
>>
>>    F= or TDM pseudowires, such a circuit breaker can be realized via
>>&= nbsp;   operator monitoring of drop rates and consequent on the q= uality of
>>    TDM service delivered and shutting = down TDM pseudowires that persistently
>>    fail t= o deliver acceptable TDM service (as defined by [G775] and
>> = ;   [G826]).  This document explains why a TDM PW that is ca= using congestion
>>    problems via interference wi= th elastic traffic cannot be delivering
>>    accep= table TDM service (see the rest of this Section, plus Appendices
>>= ;    A and B).
>>
> [Bob] I don't think we s= hould go there. We need to keep this draft firmly as informational. If some= one
> thinks a mechanism is needed because of the information in this= draft, then they can write about a mechanism.
 
[David] I firml= y disagree, but Bob and I might be talking past each other, as I do want to= stay informational ;-).
 
This text is intended to describe an = example or possibility, not to specify an implementation requirement. = The new text was intended as an OAM/OSS practice recommendation to the ope= rator, not as a normative protocol spec.
 
I’m ok with ins= erting “example”, “possibility” or something simila= r if that helps w/Bob’s concern, but I strongly oppose doing nothing = here, and note that this is also needed to respond to one of Stewart Bryant= ’s WG LC comments.
 
-- PW segment size --
 
&g= t;> [David] Ignoring the rathole invitation ;-).  This is still in = section 3, after:
>>
>>   We see that in genera= l for large packet sizes, short delays, and low
>>   pac= ket loss rates, the TDM pseudowires consume much less bandwidth
>>=     than TCP would under identical conditions.  Only fo= r small packets,
>>    long delays, and high packet= loss ratios, do TDM PWs potentially
>>   consume more b= andwidth, and even then only marginally.
>>
>> I suggest= adding:
>>
>>    Comparing figures 1 and = 2, use of a larger segment size for a T1 or E1
>>   = ; TDM PW moves the operational behavior of the PW away from the boundary>>    at which the TCP throughput equation suggests t= hat undue interference with
>>    elastic traffic m= ay start to occur.  We suggest that larger segment
>> &n= bsp;  sizes be preferred for T1 and E1 TDM PWs, but note that other>>    considerations may favor smaller segment sizes = (e.g., jitter buffer
>>    size, bounding the impac= t of an isolated packet drop event).
>>
> [Bob] There is no = need to recommend any remedial action, when we have just shown there is not= a problem.
 
[David] Not exactly - Figure 1 shows that there is= a potential concern at a combination of the high end of the range of TDM P= W latencies (4ms), and higher packet loss ratios. 

> Furthe= r, the "marginal problem" only applies when the TCP traffic uses = packets smaller than 128B, which we only used
> as a hypothetical wor= st-case for the analysis. But in practice this doesn't happen with long-run= ning TCP flows -
> they use packets that are as large as possible (ir= respective of what the PW uses). So there really is /no/ problem to remedy.=
Ok, Bob can relax - his transport credentials are intact, as this is an= entry point to the bit-rate-congestible vs. packet-rate-congestible resour= ce rathole that I want to ignore :-).
 
I believe that larger TD= M PW segment sizes are a good thing for both this operating envelope bounda= ry concern as well as per-packet processing overheads in TDM PW functionali= ty, modulo other considerations, e.g., jitter buffers, and so I prefer to k= eep this addition.
 
Thanks,
--David
 
From: Bob Briscoe [mailto:bob.briscoe@bt.= com]
Sent: Thursday, August 14, 2014 9:03 AM
To: Y= aakov Stein
Cc: Andrew G. Malis; BOCCI Matthew; Black, David
<= b>Subject: RE: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt=
 
Yaakov,

I have just re-read it thoroughly. I have noti= ced about two dozen editorial nits where correcting them would not change t= he intended sense (altho sometimes the sense needs to change to say what we= intended!). Examples:
- Unexpanded abbreviations (David's suggestion wo= uld help, but there are more)
- Missing words, tautologies, random punct= uation problems etc
- Uses of the wrong word, even tho we know what you = mean
- The caveat about dynamics vs steady state needs to shift down a p= ara, so that it follows into the para that says what to do about dynamics.<= br>
Should I take the token, and edit the XML directly, then you can see= a diff to check if you want to accept my suggestions?

Below are six= thoughts that go slightly beyond 'editorial'. 1-4 could be considered 'edi= torial'. But 5&6 are the more significant "techno-political" = points.

1. The phrase "mechanisms for provisioning PWs in serv= ice provider networks" is too opaque (the PWE audience might understan= d this scoping sentence, but a L4 reader won't understand what we're exclud= ing here, which will make them suspicious).
CURRENT:
   Mec= hanisms for provisioning PWs
   in service provider networks a= re well understood and will not be
   discussed further here.<= br>SUGGESTED:
   The case where the service provider network a= rranges sufficient capacity for a PW is well understood and will not be dis= cussed further here. The concern is over PWs deployed independently of the = service provider network's traffic engineering or capacity planning.
2. The justification for using OWD of a few milliseconds misses the point.=
"  The one-way delay
   of a native TDM service = consists of the physical time-of-flight plus
   125 microsecon= ds for each TDM switch traversed; and is thus very
   small as= compared to typical PSN network-crossing latencies.
"
The 125us= per switch is irrelevant. The point that needs making is that PWs are gene= rally deployed with a short physical time of flight.

3. Related poin= t: the captions on the graphs do not clearly indicate that they are OWD, no= t RTT. This could be clarified in the text, or on the graphs.

4. Cha= nge packet loss rate (and PLR) to packet loss probability throughout, inclu= ding on the graph axes. PLR is an ambiguous term, even though it is widely = used in the industry. Packet loss rate could be measured in packet per seco= nd, whereas packet loss probability is unambiguously dimensionless. Indeed,= in the appendices, packet loss rates with respect to time are calculated.<= br>
5. Rather than David's "reasonable" I suggest we use the p= hrase "unambiguously safe" not just "safe" for the head= ings of Figs 5-8 and in the text, as below. Otherwise, we are implying that= it is unsafe outside the region.

CURRENT:
   One may v= iew this condition as defining an operating envelope for a
   = TDM PW, as a TDM PW that occupies no more bandwidth than a TCP flow
&nbs= p;  causes no more congestion than that TCP flow would.  Under th= is
   condition it is safe to place the TDM PW along with cong= estion-
   responsive traffic such as TCP, without causing add= itional
   congestion.  on the other hand, were the TDM P= W to consume
   significantly more bandwidth a TCP flow, it co= uld contribute
   disproportionately to congestion, and its mi= xture with congestion-
   responsive traffic might be inapprop= riate.
SUGGESTED:
   One may view this condition as definin= g an unambiguously safe operating envelope for a
   TDM PW, as= a TDM PW that occupies no more bandwidth than a TCP flow
   c= auses no more congestion than that TCP flow would.  This does not draw= us into a debate over whether TCP-friendliness is a valid constraint; it s= imply says that there can be no question that a PW is safe if it does no mo= re harm than a single TCP flow. Under this
   condition it is = unambiguously safe to place the TDM PW along with congestion-
 &nbs= p; responsive traffic such as TCP.  On the other hand, were the TDM PW= to consume
   significantly more bandwidth a TCP flow, it cou= ld contribute
   disproportionately to congestion, and its mix= ture with congestion-
   responsive traffic might start to cau= se concern.

6. The analysis in Appendices A & B needs to mention= that loss bursts arise primarily from tail drop buffers, and the introduct= ion of active queue management (AQM) makes losses more independent of each = other (less bursty). (I have references showing evidence of this.) In the c= omparisons with TCP, we have assumed independent losses anyway, because the= y are the worst case (and fortunately they are easier to analyse). On a sim= ilar point, in appendix B, where we say "(bursty packet loss was also = simulated but is not reported here)", we ought to give a reason otherw= ise readers will be suspicious. A good reason would be that uniformly rando= m is the worst case.

I've also responded to a couple of David's sugg= estions inline (I agree with the others)...


At 17:44 13/08/2014,= Yaakov Stein wrote:
Bob told me on Friday of the IETF that he was going= to look over the final version and comment.
 
Bob – ping = ?
 
Y(J)S
 
From: Andrew G. Malis [ mailto:agmalis@gmail.com]
Sent: 1= 3 August, 2014 17:02
To: Yaakov Stein
Cc: Bob Briscoe; = rbriscoe@jungle.bt.co.uk; B= OCCI Matthew; Black, David
Subject: Re: [PWE3] WGLC comments on: = draft-ietf-pwe3-congcons-02.txt
 
Yaakov,
 
I still h= aven't heard from Bob, but you should incorporate David's updates into a ne= w revision, and I'll use that version for my final Shepherd's review.
&n= bsp;
Thanks,
Andy
 
On Mon, Aug 11, 2014 at 11:16 AM, Andr= ew G. Malis <agmalis@gmail.com&= gt; wrote:
Actually, I'm going to end the LC as soon as I hear back from= Bob on the IPR call. I sent him a reminder earlier this morning.
 =
Cheers,
Andy
 
On Sun, Aug 10, 2014 at 9:53 AM, Andrew G.= Malis <agmalis@gmail.com> w= rote:
David,
 
That's perfect, thanks! I'm going to end the L= C tomorrow.
 
Cheers,
Andy
 
On Sat, Aug 9, 2014 a= t 10:32 PM, Black, David <david.b= lack@emc.com> wrote:
Following up on this ...

> As we'r= e now moving quickly to try to complete this draft , I agreed
> w/Yaa= kov that my author comments would be sent in as WG Last Call comments ...> ... so here they are:
... with some suggested new text ...
> Major issues:
>
> The reference to a transport circuit br= eaker in section 3 is a bit optimistic
> and high level:
>
&= gt;       We can accomplish this by
>&n= bsp;      employing a transport circuit breaker, b= y which we mean an automatic
>       me= chanism for terminating a flow to prevent negative impact on other
>&= nbsp;      flows and on the stability of the netwo= rk
>       [I-D.fairhurst-tsvwg-circuit= -breaker].
>
> This should be accompanied by at least an exampl= e of how to do this - past
> discussion has suggested carrier monitor= ing of delivered service quality
> in order to shut down PWs that are= experiencing loss rates that result in
> unacceptable service qualit= y (as defined for us by the ITU ...).  I'd also
> like to add so= me text to suggest that operators use this sort of circuit
> breaker = practice when TDM PWs compete with elastic traffic.
After this text in S= ection 3:

   We can accomplish this by
   emp= loying a transport circuit breaker, by which we mean an automatic
 =   mechanism for terminating a flow to prevent negative impact on other=
   flows and on the stability of the network
   = [I-D.fairhurst-tsvwg-circuit-breaker].  Note that a transport circuit<= br>   breaker is intended as a protection mechanism of last resor= t, just as
   an electrical circuit breaker is only triggered = when absolutely
   necessary.

I suggest adding:

=    For TDM pseudowires, such a circuit breaker can be realized vi= a
   operator monitoring of drop rates and consequent on the q= uality of
   TDM service delivered and shutting down TDM pseud= owires that persistently
   fail to deliver acceptable TDM ser= vice (as defined by [G775] and
   [G826]).  This document= explains why a TDM PW that is causing congestion
   problems = via interference with elastic traffic cannot be delivering
   = acceptable TDM service (see the rest of this Section, plus Appendices
&n= bsp;  A and B).

I don't think we should go there. We need to ke= ep this draft firmly as informational. If someone thinks a mechanism is nee= ded because of the information in this draft, then they can write about a m= echanism.


And breaking the paragraph after the added text.
> Figure 1 and Figure 2 indicate that based on using the TCP throughpu= t
> equation as a proxy for potential impact on elastic traffic (whic= h is what
> we're doing in this draft), for the same fixed bandwidth = PW, a larger segment
> size is less likely to cause impacts.  I'= d like to add a suggestion to use
> the larger PW segment sizes as lo= ng as they're consistent with delivery of
> the service.  There'= s been a related (long-to-the-point-of-ratholed) debate
> over in the= Transport Area about whether packet-rate-limited/congestible
> forwa= rding is still relevant.  My view is that such equipment exists, and> hence that this is a relevant consideration for this draft, and woul= d not
> mention that issue here.
Ignoring the rathole invitation ;= -).  This is still in section 3, after:

   We see tha= t in general for large packet sizes, short delays, and low
   = packet loss rates, the TDM pseudowires consume much less bandwidth
 = ;  than TCP would under identical conditions.  Only for small pac= kets,
   long delays, and high packet loss ratios, do TDM PWs = potentially
   consume more bandwidth, and even then only marg= inally.

I suggest adding:

   Comparing figures 1 an= d 2, use of a larger segment size for a T1 or E1
   TDM PW mov= es the operational behavior of the PW away from the boundary
  = ; at which the TCP throughput equation suggests that undue interference wit= h
   elastic traffic may start to occur.  We suggest that= larger segment
   sizes be preferred for T1 and E1 TDM PWs, b= ut note that other
   considerations may favor smaller segment= sizes (e.g., jitter buffer
   size, bounding the impact of an= isolated packet drop event).

There is no need to recommend any reme= dial action, when we have just shown there is not a problem.

Further= , the "marginal problem" only applies when the TCP traffic uses p= ackets smaller than 128B, which we only used as a hypothetical worst-case f= or the analysis. But in practice this doesn't happen with long-running TCP = flows - they use packets that are as large as possible (irrespective of wha= t the PW uses). So there really is /no/ problem to remedy.


and a= gain, breaking the paragraph after that added text.

I hope that latt= er sentence ("We suggest that ...") is acceptable to the WG.
<= br>> Minor issues:
>
> Section 2 is about Ethernet PWs that = carry elastic traffic.  Should others
> be mentioned as also cov= ered by this analysis - e.g., ATM and Frame Relay
> PWs carrying elas= tic traffic?
After:

   In this section we consider Ethe= rnet PWs that primarily carry
   congestion-responsive traffic= .  We show that we automatically obtain
   the desired co= ngestion avoidance behavior, and that additional
   mechanisms= are not needed.

I suggest adding:

   This discussi= on is applicable to any PW that carries congestion-
   respons= ive traffic (e.g., ATM and Frame Relay PWs); Ethernet PWs are
 &nbs= p; used here for clarity, as they are a common example.

> Section= 2 does not cover Ethernet PWs carrying inelastic traffic - that would
&= gt; be unusual, but should it be explicitly noted as out of scope, or just = not
> discussed (as is the case for the current text)?
I suggest d= oing nothing here, and planning to deal with this concern
if it should a= rise later (e.g., IETF Last Call or IESG Evaluation).

> Section 3= should say that the reason the draft focuses on TDM PWs is that
> th= ey're the inelastic PWs that are likely to be carried over IP in practice.<= br>> My understanding is that other potentially inelastic PWs are MPLS-o= nly
> in practice.
After:

   Inelastic PWs, such = as TDM PWs ([RFC4553][RFC5086][RFC5087]), are
   potentially m= ore problematic than the elastic PWs of the previous
   sectio= n.  Being constant bit-rate (CBR), TDM PWs can not be made
 &n= bsp; responsive to congestion.  On the other hand, being CBR, they als= o do
   not attempt to capture additional bandwidth when neigh= boring TCP
   flows back off.

I suggest adding:

=    We focus on TDM PWs here as they are the only inelastic PWs th= at are
   likely to be IP-encapsulated in practice.  Most= other inelastic PWs
   tend to only be used with MPLS.
[I'd ask Yaakov and other PW experts to double-check the latter sentence,<= br>please.]

> A terminology section to roll up all the acronym ex= pansions (e.g., TDM, CBR)
> in one place seems like a good idea.
T= hese four acronyms would be good to put into a terminology section:

= CBR Constant Bit Rate
PSN Packet Switched Network
PW Pseudowire
TD= M Time Division Multiplexing

>
> This text on p.11 could be= incendiary:
>
>       Under this= condition it is safe to place the TDM PW along with
>  &nb= sp;    congestion - responsive traffic such as TCP, without = causing
>       additional congestion.<= br>>
> First of all to reduce the flammability, "safe" -= > "reasonable" and there
> are a number of other occurre= nces of "safe" that deserve similar treatment ;-
> ).
&g= t; Beyond that, this assertion is dependent on the operating conditions sta= ying
> within that reasonable operating envelope, which entails use o= f something
> like a circuit breaker - I'd say that here, and referen= ce the earlier
> circuit breaker discussion.
Global substitution: = "safe" -> "reasonable", starting with the abstract.<= br>
Then for this specific text on p.11:

OLD
   = ;     Under this condition it is safe to place the TDM = PW along with
        congestion - re= sponsive traffic such as TCP, without causing
    &n= bsp;   additional congestion.
NEW
    &= nbsp;   Under this condition it is reasonable to place the TDM PW= in competition
      with congestion-responsiv= e traffic such as TCP, as the result should not
    =   produce unacceptable congestion effects.  That assumes that the= operating
      conditions remain within the r= easonable operating envelope - as discussed
    &nbs= p; above, a transport circuit breaker is a suggested means of managing a TD= M
      PW so that it is shut down if operating= conditions remain persistently
      outside t= he reasonable operating envelope for the PW (e.g., due to a
  =     high packet loss rate).

No need to go there. See = my suggestions earlier.


> Figures 7 and 8 could use some shad= ing or lines to indicate the typical
> deployed operating regions for= the pseudo-wires to visibly show that they
> fall under the curves, = particularly for larger segment sizes.  I'd also
> repeat or ref= er to the suggestion to use larger

I wanted to suggest this myself, = but it is hard to depict fuzzy boundaries. If you can do this, pls do.
<= br>Given the PW operating regions are bounded by v low loss probability, th= e x axis in Figs 7&8 ought to be scaled much smaller as well.

R= elated point: the y axis in Fig 1 ought to max at 10, not 50.

Regard= s



Bob

>
> Nits:
>
> References a= ren't allowed in abstracts, so remove reference to
> [I-D.fairhurst-t= svwg-circuit-breaker]
>
> Introduction, first paragraph, delete= "PWE3" towards end of paragraph -
>
>  &nb= sp; preferably the 4 byte PWE3 control word.
>
> In addition to= its current general statements, the Security Considerations
> sectio= n should point to the security considerations sections in a few
> par= ticularly relevant RFCs - I'd suggest 3985, 4553, 5086 and 5087.
After:<= br>
   This document does not introduce any new congestion-spe= cific
   mechanisms and thus does not introduce any new securi= ty
   considerations above those present for PWs in general.
I suggest adding a new paragraph:

   For PW security= considerations, see the security considerations sections
   o= f the cited RFCS; the security considerations in [RFC3985], [RFC4553],
&= nbsp;  [RFC5086] and [RFC5087] are particularly applicable to this doc= ument.

>
> Thanks,
> --David
>
>
> = > -----Original Message-----
> > From: pwe3 [ mailto:pwe3-bounces@ietf.org] On Behalf Of Yaak= ov Stein
> > Sent: Thursday, July 24, 2014 4:52 PM
> > To= : pwe3@ietf.org
> > Subject: = Re: [PWE3] I-D Action: draft-ietf-pwe3-congcons-02.txt
> >
>= > Please look at the PDF version only
> > (the txt version doe= s not have the graphs).
> >
> > Y(J)S
> >
>= ; >
>
> _______________________________________________
&= gt; pwe3 mailing list
> pwe3@ietf.or= g
> https:= //www.ietf.org/mailman/listinfo/pwe3

___________________________= ____________________
pwe3 mailing list
pwe3@ietf.org
https://www.ietf.org/mailman/listinfo/pwe3
 
 
&= nbsp;

______________________________________________________________= __
Bob Briscoe,         &nb= sp;            =             &nb= sp;            =    BT
_______________________________________________
pwe3 = mailing list
pwe3@ietf.org
https://www.ietf.org/mail= man/listinfo/pwe3

______________________________________________= __________________
Bob Briscoe,       = ;            &n= bsp;            = ;            &n= bsp;     BT

____________________________________= ____________________________
Bob Briscoe,     &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;       BT

________________= ________________________________________________
Bob Briscoe,  = ;            &n= bsp;            = ;            &n= bsp;          BT

= --_000_8D3D17ACE214DC429325B2B98F3AE712077BC6669AMX15Acorpemcc_-- From nobody Fri Aug 29 23:54:21 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A49911A8828 for ; Fri, 29 Aug 2014 23:54:18 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.869 X-Spam-Level: X-Spam-Status: No, score=-4.869 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.668, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tW2TRmCICNq4 for ; Fri, 29 Aug 2014 23:54:16 -0700 (PDT) Received: from lhrrgout.huawei.com (lhrrgout.huawei.com [194.213.3.17]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 87F251A8827 for ; Fri, 29 Aug 2014 23:54:16 -0700 (PDT) Received: from 172.18.7.190 (EHLO lhreml402-hub.china.huawei.com) ([172.18.7.190]) by lhrrg01-dlp.huawei.com (MOS 4.3.7-GA FastPath queued) with ESMTP id BLX89094; Sat, 30 Aug 2014 06:54:15 +0000 (GMT) Received: from SZXEMA402-HUB.china.huawei.com (10.82.72.34) by lhreml402-hub.china.huawei.com (10.201.5.241) with Microsoft SMTP Server (TLS) id 14.3.158.1; Sat, 30 Aug 2014 07:54:14 +0100 Received: from SZXEMA510-MBX.china.huawei.com ([169.254.3.57]) by SZXEMA402-HUB.china.huawei.com ([10.82.72.34]) with mapi id 14.03.0158.001; Sat, 30 Aug 2014 14:54:11 +0800 From: Mach Chen To: "pwe3@ietf.org" Thread-Topic: Mail regarding draft-ietf-pwe3-mpls-tp-oam-config Thread-Index: Ac/EHzNiM8L42u/HRiC1vK41AsrBRQ== Date: Sat, 30 Aug 2014 06:54:11 +0000 Message-ID: Accept-Language: en-US, zh-CN Content-Language: zh-CN X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.111.97.72] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-CFilter-Loop: Reflected Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/4uXEpYxlV2CfF2BQEaW_BVlzR7U Subject: [PWE3] Mail regarding draft-ietf-pwe3-mpls-tp-oam-config X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Aug 2014 06:54:18 -0000 Hi, We submitted an update before Toronto meeting, and are working an update to= refine and polish the document.=20 This draft intends to solve a requirement (Requirement 51) of MPLS-TP: "If the control plane is used, it MUST support the configuration and modification of OAM maintenance points as well as the activation/ deactivation of OAM when the transport path or transport service is established or modified (Requirement 51)[RFC5654]. " We'd really like more reviews and feedbacks from the WG, any comments (is t= his document useful? is the solution technical right? etc.) are welcome!=20 Thanks, Mach From nobody Fri Aug 29 23:59:36 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 88A181A8830; Fri, 29 Aug 2014 23:59:27 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.601 X-Spam-Level: X-Spam-Status: No, score=-102.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Dbi3A8bSZ0Qi; Fri, 29 Aug 2014 23:59:25 -0700 (PDT) Received: from usevmg20.ericsson.net (usevmg20.ericsson.net [198.24.6.45]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 02F751A8828; Fri, 29 Aug 2014 23:59:24 -0700 (PDT) X-AuditID: c618062d-f79206d0000014d2-fd-5401215f795d Received: from EUSAAHC003.ericsson.se (Unknown_Domain [147.117.188.81]) by usevmg20.ericsson.net (Symantec Mail Security) with SMTP id DC.CD.05330.F5121045; Sat, 30 Aug 2014 02:57:03 +0200 (CEST) Received: from EUSAAMB103.ericsson.se ([147.117.188.120]) by EUSAAHC003.ericsson.se ([147.117.188.81]) with mapi id 14.03.0174.001; Sat, 30 Aug 2014 02:59:18 -0400 From: Gregory Mirsky To: Mach Chen , "pwe3@ietf.org" Thread-Topic: Mail regarding draft-ietf-pwe3-mpls-tp-oam-config Thread-Index: Ac/EHzNiM8L42u/HRiC1vK41AsrBRQAADI0w Date: Sat, 30 Aug 2014 06:59:17 +0000 Message-ID: <7347100B5761DC41A166AC17F22DF1121B828BE1@eusaamb103.ericsson.se> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [147.117.188.9] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrCLMWRmVeSWpSXmKPExsUyuXRPoG68ImOIQfN/EYsLa4Utbi1dyWrR 92kLiwOzR8uRt6weS5b8ZApgiuKySUnNySxLLdK3S+DKmH/vPVPBIc6K9Ts3szUwvmTvYuTk kBAwkei8e44RwhaTuHBvPVsXIxeHkMBRRonbfd1MEM5yRol33/Yxg1SxCRhJvNjYA9YtIuAq 8WPCcqAiDg5mAWWJU3dlQMLCAnYSm96fYIQosZf43vuNCcI2kliw9z4biM0ioCrx4fdXsDG8 Ar4Sz1d8BRsvJBAq8XPlDrB6ToEwieVrFoDVMwId9/3UGrA4s4C4xK0n85kgjhaQWLLnPDOE LSrx8vE/VghbUWJf/3R2iHodiQW7P7FB2NoSyxa+ZobYKyhxcuYTlgmMYrOQjJ2FpGUWkpZZ SFoWMLKsYuQoLU4ty003MtjECIyWYxJsujsY97y0PMQowMGoxMP74DtDiBBrYllxZe4hRmkO FiVx3lm184KFBNITS1KzU1MLUovii0pzUosPMTJxcEo1MOqf3pDntETSYJX8vqtmwpteLD/8 S+ihw91v+dMfTpxVkhjDeXTXXC6pXTFBekbK0xbd/SS1sZ1x97RKJwmbRZouYhYiK6o46sst 7pRFLFS+FONlsogheHbkt9dBf18VGfb0b3CP/b71nyd3esLTM52d0QuXXcmUf7qncN6yRQ8M NTuk46zUbymxFGckGmoxFxUnAgCquJSWdwIAAA== Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/bAPt7QFRxnV_YgHkkLa4BKU343Q Cc: "mpls@ietf.org" Subject: Re: [PWE3] Mail regarding draft-ietf-pwe3-mpls-tp-oam-config X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Aug 2014 06:59:28 -0000 Hi Mach, I think that the draft-ietf-mpls-lsp-ping-mpls-tp-oam-conf intended to be u= sed to configure OAM on MPLS-TP PWs. It lapsed and expired but I'm working = on updating it will publish shortly. Regards, Greg -----Original Message----- From: pwe3 [mailto:pwe3-bounces@ietf.org] On Behalf Of Mach Chen Sent: Friday, August 29, 2014 11:54 PM To: pwe3@ietf.org Subject: [PWE3] Mail regarding draft-ietf-pwe3-mpls-tp-oam-config Hi, We submitted an update before Toronto meeting, and are working an update to= refine and polish the document.=20 This draft intends to solve a requirement (Requirement 51) of MPLS-TP: "If the control plane is used, it MUST support the configuration and modification of OAM maintenance points as well as the activation/ deactivation of OAM when the transport path or transport service is established or modified (Requirement 51)[RFC5654]. " We'd really like more reviews and feedbacks from the WG, any comments (is t= his document useful? is the solution technical right? etc.) are welcome!=20 Thanks, Mach _______________________________________________ pwe3 mailing list pwe3@ietf.org https://www.ietf.org/mailman/listinfo/pwe3 From nobody Sat Aug 30 00:22:05 2014 Return-Path: X-Original-To: pwe3@ietfa.amsl.com Delivered-To: pwe3@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7FB911A8844; Sat, 30 Aug 2014 00:22:01 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.869 X-Spam-Level: X-Spam-Status: No, score=-4.869 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.668, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IuPZXI5WBD36; Sat, 30 Aug 2014 00:21:57 -0700 (PDT) Received: from lhrrgout.huawei.com (lhrrgout.huawei.com [194.213.3.17]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id AA0121A8849; Sat, 30 Aug 2014 00:21:55 -0700 (PDT) Received: from 172.18.7.190 (EHLO lhreml402-hub.china.huawei.com) ([172.18.7.190]) by lhrrg02-dlp.huawei.com (MOS 4.3.7-GA FastPath queued) with ESMTP id BIW70471; Sat, 30 Aug 2014 07:21:53 +0000 (GMT) Received: from SZXEMA401-HUB.china.huawei.com (10.82.72.33) by lhreml402-hub.china.huawei.com (10.201.5.241) with Microsoft SMTP Server (TLS) id 14.3.158.1; Sat, 30 Aug 2014 08:21:53 +0100 Received: from SZXEMA510-MBX.china.huawei.com ([169.254.3.57]) by SZXEMA401-HUB.china.huawei.com ([10.82.72.33]) with mapi id 14.03.0158.001; Sat, 30 Aug 2014 15:21:50 +0800 From: Mach Chen To: Gregory Mirsky , "pwe3@ietf.org" Thread-Topic: Mail regarding draft-ietf-pwe3-mpls-tp-oam-config Thread-Index: Ac/EHzNiM8L42u/HRiC1vK41AsrBRQAADI0wAAA1eWA= Date: Sat, 30 Aug 2014 07:21:50 +0000 Message-ID: References: <7347100B5761DC41A166AC17F22DF1121B828BE1@eusaamb103.ericsson.se> In-Reply-To: <7347100B5761DC41A166AC17F22DF1121B828BE1@eusaamb103.ericsson.se> Accept-Language: en-US, zh-CN Content-Language: zh-CN X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.111.97.72] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-CFilter-Loop: Reflected Archived-At: http://mailarchive.ietf.org/arch/msg/pwe3/_NA6e3VMbCJzS0LrW1_2XkEqTCs Cc: "mpls@ietf.org" Subject: Re: [PWE3] Mail regarding draft-ietf-pwe3-mpls-tp-oam-config X-BeenThere: pwe3@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Pseudowire Emulation Edge to Edge List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Aug 2014 07:22:01 -0000 Hi Greg, Thanks for your prompt response! Yes, I know that draft. Actually, during the MPLS-TP era, there are three r= elevant drafts (this document, one is in ccamp and the one you mentioned) t= hat intend to realize OAM configuration. I guess, the reason for this is th= at there is a requirement (it requires that the control plane MUST support = this function) in RFC5654: "The MPLS-TP control plane MUST support the configuration and modification of OAM maintenance points as well as the activation/ deactivation of OAM when the transport path or transport service is established or modified." These three drafts have been there for a long time, and the draft (http://t= ools.ietf.org/html/draft-ietf-ccamp-rsvp-te-mpls-tp-oam-ext-12) has already= in the publish process.=20 I'd like hear the WG's opinions on how to progress this draft,=20 1. Is this document useful? 2. Is there still interest to progress this draft? Best regards, Mach=20 > -----Original Message----- > From: Gregory Mirsky [mailto:gregory.mirsky@ericsson.com] > Sent: Saturday, August 30, 2014 2:59 PM > To: Mach Chen; pwe3@ietf.org > Cc: mpls@ietf.org > Subject: RE: Mail regarding draft-ietf-pwe3-mpls-tp-oam-config >=20 > Hi Mach, > I think that the draft-ietf-mpls-lsp-ping-mpls-tp-oam-conf intended to be= used to > configure OAM on MPLS-TP PWs. It lapsed and expired but I'm working on > updating it will publish shortly. >=20 > Regards, > Greg >=20 > -----Original Message----- > From: pwe3 [mailto:pwe3-bounces@ietf.org] On Behalf Of Mach Chen > Sent: Friday, August 29, 2014 11:54 PM > To: pwe3@ietf.org > Subject: [PWE3] Mail regarding draft-ietf-pwe3-mpls-tp-oam-config >=20 > Hi, >=20 > We submitted an update before Toronto meeting, and are working an update = to > refine and polish the document. >=20 > This draft intends to solve a requirement (Requirement 51) of MPLS-TP: >=20 > "If the control plane is used, it MUST support the configuration and > modification of OAM maintenance points as well as the activation/ > deactivation of OAM when the transport path or transport service is > established or modified (Requirement 51)[RFC5654]. > " > We'd really like more reviews and feedbacks from the WG, any comments (is= this > document useful? is the solution technical right? etc.) are welcome! >=20 > Thanks, > Mach >=20 > _______________________________________________ > pwe3 mailing list > pwe3@ietf.org > https://www.ietf.org/mailman/listinfo/pwe3

Hi Greg -

Lets wait for Stewart to respond.

If he is busy, you and I will take up to refresh..

Thanks,

= himanshu

 

From: Gregory Mirsky [mailto:gregory.mirsky@ericsson.c= om]
Sent: Tuesday, August 19, 2014 11:13 AM
To: Shah, = Himanshu; pwe3@ietf.org; Stewart Bryant
Cc: Somnath Ojha
Su= bject: RE: VCCV for gal draft

 

Hi Himanshu,

I think that earlier mail http://www.ietf.org/mail-arc= hive/web/pwe3/current/msg13778.html captured WG agreement on VCCV Type = 4 Control Channel. I’d be glad to work on the final version.

 =

 &= nbsp;           &nbs= p;  Regards,

          =             &nb= sp;         Greg<= /p>

 

From: pwe3 [mailto:pwe3-bounces@ietf.org] On Behalf Of Sha= h, Himanshu
Sent: Tuesday, August 19, 2014 7:59 AM
To: = pwe3@ietf.org; Stewart Bryant
Cc= : Somnath Ojha
Subject: [PWE3] VCCV for gal draft<= /span>

 

Hi Stewart,

In IETF 89 WG meeting we discussed to revive http://tool= s.ietf.org/html/draft-ietf-pwe3-vccv-for-gal-01

Relevant minutes from the meeting bel= ow.

 = ;

Stewart – y= ou agreed to refresh it (we had discussions on the emails after the meeting= as well).

If = you are busy and would like me to, let me know.

I had talked to Tom N. a while back and h= e was busy with SDN and no time to pursue this, so one of us have to.<= /o:p>

 

Please let me know.

 

Thanks,

himanshu

 

 

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D minutes from IE= TF 89 PWE3 WG meeting =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=

 <= /span>

 <= /i>

Matthew: anything to say on VCCV for GAL? (VCCV type 4)

Luca: where's Tom N? He was heading it.  3 documens= 4447 bis for last call

refresh adoption keep ali= ve draft, but not sure if interest.  If not, willing to drop it. = VCCV Type 4, Tom lead author, we set out a stable draft but probably expir= ed.  Nothing current, willing to drop.

WG:= No!

Luca: then we need someone to drive.

Himanshu: I can help.  We need the draft and the IA= NA assignment.

Matthew: please sort out refreshin= g the draft.

Himanshu: Yes. Is WG draft.

YJS: Proposal for this. 3 drafts around this. THere is fa= r less interest but want to close type 4. We should make one draft that say= s here are the types, deprecate 2 and add 4.

Luc= a: rfc4447bis is just an errata with no tech changes.  The document yo= u are talking about is there.  We didn't want to go down a rat hole.&n= bsp; Don't want to open RFC 5085.  Just need to progress this, it is i= n good shape.

YJS: WG wanted to clean up all docu= ments, but no time. So we will just progress the current draft.<= /span>