From pcn-bounces@ietf.org Thu Nov 01 02:36:06 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1InTdi-0005Kg-Qs; Thu, 01 Nov 2007 02:34:50 -0400 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1InTdg-0005JV-Tt for pcn-confirm+ok@megatron.ietf.org; Thu, 01 Nov 2007 02:34:48 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1InTdg-0005JN-E2 for pcn@ietf.org; Thu, 01 Nov 2007 02:34:48 -0400 Received: from rotterdam.ewi.utwente.nl ([130.89.10.5]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1InTde-00023w-N0 for pcn@ietf.org; Thu, 01 Nov 2007 02:34:48 -0400 Received: from webmail.cs.utwente.nl (janus.ewi.utwente.nl [130.89.10.26]) by rotterdam.ewi.utwente.nl (8.13.6/8.13.6) with SMTP id lA16YfH5009696; Thu, 1 Nov 2007 07:34:41 +0100 (MET) Received: from 84.82.109.231 (auth. user karagian@imap1.ewi.utwente.nl) by webmail.cs.utwente.nl with HTTP; Thu, 01 Nov 2007 06:34:41 +0000 To: "Anna Charny (acharny)" , "anurag.bhargava@ericsson.com" , "Lars Westberg (KI/EAB)" Date: Thu, 01 Nov 2007 06:34:40 +0000 X-Mailer: IlohaMail/0.8.13 (On: webmail.cs.utwente.nl) Message-ID: In-Reply-To: From: "Georgios Karagiannis" Bounce-To: "Georgios Karagiannis" MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0 () X-Scanned-By: MIMEDefang 2.52 on 130.89.10.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0rc3 (rotterdam.ewi.utwente.nl [130.89.10.5]); Thu, 01 Nov 2007 07:34:43 +0100 (MET) X-Spam-Score: 0.0 (/) X-Scan-Signature: e274a7d5658fb8b0d6fbc93f042d014b Cc: "pcn@ietf.org" Subject: [PCN] Re: Questions on LC-PCN draft version 01 X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Anna Thank you very much for your comments! Before answering to your comments, see in line below, I would like to explain in an abstract way the LC-PCN algorithm. * Setting the thresholds at PCN_interior_nodes: ----------------------------------------------- In order to calculate the PCN_upper_rate we use two parameters: Maximum PHB capacity: that is the maximum capacity that can be supported by a PCN_interior_node Termination_offset_rate: that is an absolute rate value that should be set equal into all PCN_interior_nodes. Note that this value is used by PCN_interior_nodes to calculate their PCN_upper_rate and also during the situation that a PCN_interior_node is in flow termination state and it receives PCN_marked packets. Please see pseudo code on page 20. This value must be set equal into all PCN_interior_nodes such that all these nodes will know when to take into account the incoming PCN_marked packets and when not. The PCN_upper_rate is then found as: PCN_upper_rate =3D "Maximum PHB capacity" - Termination_offset_rate The PCN_lower_rate is configured in all PCN_interior-nodes and it can be calculated in the following way: PCN_lower_rate =3D PCN_upper_rate - Admission_offset_rate The Admission_offset_rate is an absolute rate value and it is equal in all PCN_interior_nodes and PCN_egress_nodes. Note that this value is used by PCN_interior_nodes to calculate their PCN_lower_rate and the PCN_egress_nodes to calculate their PCN_upper_rate_egress. Furthermore, this value is used by the PCN_interior_nodes also during the situation that a PCN_interior_node is in admission control state and it receives PCN_marked packets. Please see pseudo code on page 13. This value must be set equal into all PCN_interior_nodes such that all these nodes will know when to take into account the incoming PCN_marked packets and when not. Note that a PCN_interior_node can PCN_mark packets up to an excess rate equal to the Admission_offset_rate. If the exess rate in an PCN_interior_node is higher than the Admission_offset_rate, then the PCN_interior_node changes state from admission control state to flow termination state. * Setting the thresholds at PCN_egress_nodes: --------------------------------------------- The question is how to calculate the threshold that defines when a PCN_egress_node goes into the admission control state. One way to do that is to consider that when the PCN_egress_node receives a PCN_marked packet it will mean that at least one PCN_interior_node started to be admission control congested and therefore it will go from Normal state to admission control state. Of course this will somehow might provide some errors, because there might be situations that this consideration might be to conservative. Therefore, we use a percentage of received PCN_marking encoded packets in proportion to total rate of received packets. In this version of the draft we call this value as: PCN_lower_rate_egress. This is wrong. What we should say is: PCN_lower_rate_egress: is equal to incoming_PCN_marking_rate, when: (incoming_PCN_marking_rate/measured PHB rate) =3D to a preconfigured percentage, say 1%. From now on I denote this percentage as: PCN_lower_percentage_egress. Thus we can say that a PCN_egress_node changes from Normal state to admission control state when incoming_PCN_marking_rate/measured PHB rate > PCN_lower_percentage_egress. Where, incoming_PCN_marking_rate =3D N * input_PCN_marking_bytes/T Where, input_PCN_marking_bytes =3D received number of "PCN_marking" encoded packets during measurement period T. Now we defined the condition that the PCN_egress_node changes state from Normal state to admission control state. But how will the PCN_egress_node change state from admission control state to flow termination state. In order to explain this, it is imporatnt to note that each PCN_interior_node that is in admission control state it can PCN_mark packets up to a value equal to Admission_offset_rate. Furthermore, if a PCN_interior_node receives incoming PCN_marked packets and is in the addmission control state, it will not remark any packets if the excess rate is equal or lower than the incoming_PCN_marking_rate, see page 13. Furthermore, if we will consider as normal situations the situations that no ECMP occurs and that all flows belonging to the same ingress-egress aggregate will use the same path from PCN_ingress to PCN_egress, this will mean that when the PCN_egress_node receives, for the given ingress-egress aggregate an excess rate equal to Admission_offset_rate it will have to change from admission control state to flow termination state. Thus in this case the second threshold, that in this case is a rate and not a percentage, can be calculated as follows: PCN_upper_egress_rate =3D PCN_lower_egress_rate + Admission_offset_rate. However, there are some corner cases, that mainly occur when the different congestion points (admission control congested PCN_interior_nodes) on the same path are not simulataneously starting to be congested. Therefore we use the multicongestion_error parameter to identify the error bound that ocurs due to these corner cases. Note that this error bound can be e.g., predefined ones off line by the operator, by studying the network topology and/or studying how often such corner cases could occur and/or doing off line measurements. Therefore we use: PCN_upper_rate_egress =3D PCN_lower_rate_egress + Admission_offset_rate +/- multicongestion_error * How the states of operation in PCN_interior_nodes are being changed? --------------------------------------------------------------------- This is explained on page 17, 18, by using Figure 4. Change from Normal state to Admission control state: event A Occurs when: Measured PHB rate > PCN_lower_rate Change from Admission control state to Flow Termination state: event B Occurs when: Measured PHB rate > PCN_upper_rate * How the states of operation in PCN_egress_nodes are being changed? --------------------------------------------------------------------- This is explained on page 21, 22. Note that the description of event A on page 21 has to be modified to avoid the confusions that were caused up to now, see explanation given above. Change from Normal state to Admission control state: event A Occurs when: incoming_PCN_marking_rate/measured PHB rate) > PCN_lower_percentage_egress As explained above: IF ((incoming_PCN_marking_rate/measured PHB rate) =3D PCN_lower_percentage_egress) THEN PCN_lower_egress_rate =3D incoming_PCN_marking_rate Change from Admission control state to flow termination state: event B Occurs when: incoming_PCN_marking_rate > PCN_upper_rate_egress It is important to note that also the explanation of event C on page 21, has to be modified to avoid the confusions that were caused up to now, see explanation given above: Change from Admission control state to Normal state: Occurs when: incoming_PCN_marking_rate/measured PHB rate) =3D< PCN_lower_percentage_egress * Generated excess rate by an PCN_interior_node operating in admission control state. ------------------------------------------------------------- The excess rate =3D signaled_overload_rate. The maximum excess rate that a PCN_interior_node can calculate in admission control state is equal to Admission_offset_rate, see explanation above. The number of bytes that are remarked, signaled_remarked_bytes depend on the value of calculated excess rate (signaled_overload_rate), the value of the Admission_offset_rate and the value of the incoming_PCN_marking_rate, see pseudo code on page 13. Note that all packets that are passing through a congested PCN_interior_node an are not being PCN_marked by the PCN_interior_node have to be remarked using the PCN_Affected_marking. Regarding probes, the probe packets that are passing through a congested node are either PCN_marked or PCN_Affected_marked. * Generating excess rate by an PCN_interior_node operating in flow termination state: ---------------------------------------------------------------- The excess rate =3D signaled_overload_rate. Note that the calculation of the signaled_overload_rate is different than in the situation that the PCN_Interior_node operates in admission control state, see page 19 and 20. This is due to the fact that a sliding window is used to solve an undershooting problem, see discussion on page 19. The number of bytes that are remarked, signaled_remarked_bytes depend on the value of calculated excess rate (signaled_overload_rate), the value of the Termination_offset_rate and the value of the incoming_PCN_marking_rate, and the see pseudo code on page 20. Note that all packets that are passing through a congested PCN_interior_node an are not being PCN_marked by the PCN_interior_node have to be remarked using the PCN_Affected_marking. * Providing admission control at PCN_egress_nodes: -------------------------------------------------- When the PCN_egress_node is operating in admission control state than a flow that is requesting admission into the PCN domain can b etreated in the following way: If no probing is used, the request for admission can be accomplished by using an external to PCN signaling protocol. In this case when the request arrives at a PCN_egress_node that operates in admission control state then the request is rejected. If it operates in Normal state is accepted. If probing is used, the request for admission is accomplished by using probe packets. In this case when the probe arrives at a PCN_egress_node and it is either PCN_marking or PCN_Affected_marking encoded is rejected. Otherwise is accepted. Note that probes can only be used when PCN_Affected_marking is appled in whole PCN domain. Otherwise, the admission control procedure will work by using e.g., an external signaling protocol used between PCN_ingress_nodes and PCN_egress_nodes. * Providing flow termination at PCN_egress_nodes: -------------------------------------------------- When a PCN_egress_node operates on flow termination it calculates the incoming excess rate (incoming_PCN_marking_rate). By using the excess rate the PCN_egress_node calclates the numer of flows that have to be terminated using the pseudo code given on page 23. Note that the PCN_egress_node has to maintain per flow reservation states. The information contained in the per flow states is used for the calculation of the flow that have to be terminated, see pseudocode on page 23. Note also that this pseudo code uses priority clases, but it operates when also no priority clases are used by setting Maximum_priority =3D 0 Where, 0 =3D< priority_class =3D< Maximum_priority Below, I am providing some answers to your comments! > -----Original Message----- > From: Anna Charny (acharny) [mailto:acharny@cisco.com] > Sent: zondag 7 oktober 2007 21:40 > To: anurag.bhargava@ericsson.com; Georgios Karagiannis; Lars Westberg > (KI/EAB) > Cc: pcn@ietf.org > Subject: Questions on LC-PCN draft version 01 > > Dear authors of the LC-PCN draft, > > Am I right that there has been no response yet to Phil's questions > (attached at the end) on the new version of your draft below (I > checked the pcn list and it does not seem it went there)? If the > response was sent, can you resend it please? > > I have many of the same questions, and also a few additional ones. > Here are some of them. > > 1) I do not understand whether pcn-lower_rate_egress and > pcn_upper_rate ingress are expressed as a fraction of the total rate > (a unitless entity, analogous to CLE) or an absolute value (in bits or > bytes per second). The text seems to be contradictory on this point: > > on page 8 it is a fraction: > > "PCN_lower_rate_egress =3D predefined percentage of received > PCN_marking", > > while on page 13 it is an absolute rate: > > "If the incoming_PCN_marking_rate is higher than a preconfigured > PCN_lower_rate_egress, ..." where the > > (Incoming_PCN_marking_rate is clearly defined as abslolue rate on page > 12: "Where the "incoming_PCN_marking_rate" is calculated as follows: > incoming_PCN_marking_rate =3D (received number of "PCN_marking" > DSCP during T)* N)/T;) Georgios: You are right, please see discussion above on paragaph denoted as: "Setting the thresholds at PCN_egress_nodes". We use a percentage of received PCN_marking encoded packets in proportion to total rate of received packets to define when a PCN_egres_node goes from Normal state to admission control state. In this version of the draft we call this value as: PCN_lower_rate_egress. This is wrong. What we should say is: PCN_lower_rate_egress: is equal to incoming_PCN_marking_rate, when: (incoming_PCN_marking_rate/measured PHB rate) =3D to a preconfigured percentage, say 1%. From now on I denote this percentage as: PCN_lower_percentage_egress. Thus we can say that a PCN_egress_node changes from Normal state to admission control state when incoming_PCN_marking_rate/measured PHB rate > PCN_lower_percentage_egress. Where, incoming_PCN_marking_rate =3D N * input_PCN_marking_bytes/T Where, input_PCN_marking_bytes =3D received number of "PCN_marking" encoded packets during measurement period T. > > 2) Regarding the question above, > > * if pcn_lower_rate_eggress (and pcn_uppre-rate_egress) are > absolute rates, then how do you choose that abslolute rate value, > given that the rates of ingress-egress aggregates at the same ingress > may be vastly different in magnitude? Georgios: See explanation above! Hopefully this misundersatnding is also solved! > > *if, however, pcn_lower_rate_eggress (and > pcn_uppre-rate_egress) are ratios, then what is the guidance of > setting these ratios so that the egress can always tell admission > state from the termination state, given that the same marking is used > for admission and termination marking? > Specifically, suppose at the bottleneck pcn_lower_threshold is set to > X, and pcn_upper_threshold to aX with some a>1 (assume for example > a=3D1.5). > How does the egress node tell between the conditions when the total > pcn traffic on the bottleneck is 1.1X (which is the state wnen > admission is needed but termination is not, - in this case > (1.1X-X)/X=3D0.1 traffic is pcn-marked), and the case when the total > traffic on the bottleneck is 1.1aX, which is the case when termination > is needed, and (1.1ax-ax)/ax=3D0.1), if in both cases the ratio between > marked and total traffic is the same value 0.1? Georgios: In order to solve the above described problems, the thresholds used in the PCN_interior_nodes and PCN_egress_nodes are using: the Admission_offset_rate that is an absolute rate value which is set equal into whole PCN domain. Note that the maximum excess rate that a PCN_interior_node can calculate in admission control state is equal to Admission_offset_rate. If the excess rate is higher than the Admission_offset_rate then the node changes from admission control state to flow termination state. Please see all details that I have provided at the top of this email. In particular, check the paragraphs denoted above as: "Setting the thresholds at PCN_interior_nodes" "Setting the thresholds at PCN_egress_nodes" "Generated excess rate by an PCN_interior_node operating in admission control state" > > 3) On page 9, multicongestion error error is introduced, but the text > is silent on how this error is defined and what is it set to (if it is > a configuration parameter), or how it is measured (if it is something > that is being measured). Can you explain, please? Georgios: Please see the paragraph denoted above as "Setting the thresholds at PCN_egress_nodes" Best regards, Georgios > > I have some more questions, including sharing those asked by Phil in > the attached message), but I will stop now, as I hope clarifications > on the above (and Phil's) questions will help understand the draft > better... > > Thank you in advance for clarifying all this. > > Anna _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 01 03:04:29 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1InU65-0002NJ-F3; Thu, 01 Nov 2007 03:04:09 -0400 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1InU64-0002N4-Gp for pcn-confirm+ok@megatron.ietf.org; Thu, 01 Nov 2007 03:04:08 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1InU5x-0002Cd-7p for pcn@ietf.org; Thu, 01 Nov 2007 03:04:01 -0400 Received: from rotterdam.ewi.utwente.nl ([130.89.10.5]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1InU5q-00019A-T1 for pcn@ietf.org; Thu, 01 Nov 2007 03:04:01 -0400 Received: from webmail.cs.utwente.nl (janus.ewi.utwente.nl [130.89.10.26]) by rotterdam.ewi.utwente.nl (8.13.6/8.13.6) with SMTP id lA173dW1015297; Thu, 1 Nov 2007 08:03:39 +0100 (MET) Received: from 84.82.109.231 (auth. user karagian@imap1.ewi.utwente.nl) by webmail.cs.utwente.nl with HTTP; Thu, 01 Nov 2007 07:03:39 +0000 To: "philip.eardley@bt.com" , "anurag.bhargava@ericsson.com" , "pcn@ietf.org" Subject: RE: [PCN] LC-PCN version 01 uploaded Date: Thu, 01 Nov 2007 07:03:38 +0000 X-Mailer: IlohaMail/0.8.13 (On: webmail.cs.utwente.nl) Message-ID: In-Reply-To: <75A199C5D243C741BF3D3F1EBCEF9BA5019DC606@E03MVZ1-UKDY.domain1.systemhost.net> From: "Georgios Karagiannis" Bounce-To: "Georgios Karagiannis" MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0 () X-Scanned-By: MIMEDefang 2.52 on 130.89.10.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0rc3 (rotterdam.ewi.utwente.nl [130.89.10.5]); Thu, 01 Nov 2007 08:03:43 +0100 (MET) X-Spam-Score: 0.0 (/) X-Scan-Signature: fe105289edd72640d9f392da880eefa2 Cc: X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Phil Thank you very much! The answers to your questions included in this email are already given in the reply that I have just sent to Anna some minutes ago! <> But please see in line! On 10/26/2007, "philip.eardley@bt.com" wrote: >In-line >Thanks >Phil >> >> > Let me summarise to make sure I understand it right. >> > >> > - two encodings are used. one ('PCN-marking') is used for >> > both adm ctrl & termination to indicate the excess traffic. >> > If a PCN-interior-node is PCN-marking some pkts, then it >> > marks all other pkts with another encoding ('affected marking') >>=20 >> Georgios: Yes, you are right. All other packets are marked >> using the "PCN_Affected_marking" encoding. >>=20 >> > >> > - the 'PCN-marking' algorithm has several regimes: >> > * when traffic rate is below PCN-lower-rate, no pkts are marked >> > * as the traffic rate climbs above the PCN-lower-rate, an >> > increasing number of pkts are marked (marking rate =3D excess >> > rate divided by N) [adm ctrl state] >>=20 >> Georgios: It is more complicated than that! Please check the >> pseudocode on page 13. The packets are PCN-marking encoded >> only if the incoming_PCN_marking_rate is smaller or equal to >> the Admission_offset_rate. Where the incoming_PCN_marking_rate >> is actualy the rate of incoming packets that are PCN_marked. >>=20 >> > * at some rate (PCN-lower-rate + adm-offset-rate), then pkts >> > are marked at the same rate, even if the traffic rate climbs >> > higher [adm ctrl state] >>=20 >> Georgios: No, if the incoming_PCN_marking_rate is higher than the >> Admission_offset_rate than no anymore packets are PCN_marked. >> They are however marked as PCN_Affected_marking! >>=20 >> > * when the traffic rate reaches the PCN-upper-rate, >> > additional pkts are marked (additional marking rate =3D excess >> > rate above PCN-upper-rate divided by N) [termination ctrl state] >>=20 >> Georgios: It is again more complicated than that, please >> see the pseudocode on Page 20. Similar to the admission control state >> the rate of packets to be marked also depends on >> the incoming_PCN_marking_rate. Moreover, if the interior node is >> in termination ctrl state >> then the additional marking rate is using the sliding window >> algorithm to overcome the undershooting issue, see page 19 and page >20. >>=20 >>=20 >> > * if pkts arrive already PCN-marked, then the node measures >> > the rate of pkts already marked & works out what excess rate >> > this corresponds to, and only marks additional pkts if its >> > own excess rate is even higher. >>=20 >> Georgios: Well, it is more complicated than that. Please see pseudo >> code onn page 13 and page 20, the >> rate of the already PCN_marked packets is denoted in the pseudocode >> as incoming_PCN_marking_rate. >>=20 >>=20 >> > * the PCN-egress-node measures the rate of PCN-marked pkts >> > and determines whether the overloaded node is in adm ctrl >> > state or termination ctrl state >>=20 >> Georgios: Yes, but it is more complicated than that please see >> pseudo code on page 14 and on page 23. >>=20 > >[phil] I'm now confused whether I understand your algo. >I should have said: my first bullets were all assuming that no pkts >arrived already PCN-marked, ie incoming_PCN_marking_rate =3D 0. Is my >summary correct with this assumption? > >Also, the pcn-egres-node determining what state it's in. I was reading >the description on page 21-22. pges 14 & 23 don't seem to have relevant >pseudocode? Georgios: please see the abstract description of the LC-PCN algorithm that I gave on the reply that I have sent to Anna some minutes ago! > > >> > >> > Questions: >> > - I don't understand the point of doing PCN-marking in the adm ctrl >> > state. When it comes to making an adm decision, you send a (single) >> > probe pkt - if it's PCN-marked or affected-marked then you block the >> > call. So when a node's in the adm ctrl state, it could just do >> > affected-marking of all pkts. Wouldn't that work just as well? >>=20 >> Georgios: this algorithm can provide admission control even if >> probing is not used. However, by using probing we ensure that the >> flow that is requesting access is passing through the interior node >> that is either in admission control state or flow termination state. > >[phil] ok. >>=20 >> > - with you current algo, imagine there are two paths through the nw: >> > A-B-X-D >> > A-M-N-X-D >> > If B & N are both in adm ctrl state then they both will do >> > PCN-marking. >> > At the PCN-egress-node the PCN-marking rate may be high enough that >it >> > believes termination is needed. Are there topologies/scenarios where >> > this could happen? I think your multicongestion_error >> > parameter in S3.4 >> > is connected to this problem; I didn't find your discussion about it >> > convincing >>=20 >> Georgios: You are right, the multicongestion_error parameter in S3.4 >> is used to emphasize the multicongestion error bounds in this >> calculation. Note that >> the calculation of this error bound can only be estimated by off line >> tests in a predefined network scenario. >> Note that by using the dependency of the incoming_PCN_marking_rate >> the multicongestion error bound can be decreased. > >[phil] ok. I don't like this. the idea of PCN is to be a >measurement-based approach so don't have to do these kind of off-line >tests. In particular, there'll be wrong when the network is suffering >from failures - the strength of a measurement-based approach should >exactly be that it gracefully adapts despite network failures.=20 Georgios: please see the abstract description of the LC-PCN algorithm that I gave on the reply that I have sent to Anna some minutes ago! In particular, please check the paragraph denoted as "Setting the thresholds at PCN_egress_nodes" in the reply that I have sent to Annas email some minutes ago! Furthermore, your above comment is not fair, since the example that you gave can be considered as a corner case! Only in such cases the multicongestion_error bound is needed. Now, regarding your example: >> > A-B-X-D >> > A-M-N-X-D >> > If B & N are both in adm ctrl state then they both will do >> > PCN-marking. Are you assuming that ECMP occurs on the PCN domain and therefore flows leaving from A and going towards D will not follow the same path. If no ECMP is used I would expect that all flows belonging to the same A/D aggregate, and leave from A and go towards D will follow either A-B-X-D or will follow A-M-N-X-D, but they will not be separated. >>=20 >> > - the 'affected marking' is claimed to solve ECMP issues. However, >the >> > same affected marking is used in both adm ctrl state & >> > termination ctrl >> > state - I don't think this works. Imagine that a node [node-1] on >one >> > path is in adm ctrl state & a node [node-2] on another path is in >> > termination ctrl state. Therefore flows are terminated, and only >flows >> > that are being PCN-marked or affected-marked are terminated. However >> > this could lead to flows being terminated that go through >> > node-1; node-2 >> > will still be just as badly overloaded. >>=20 >> Georgios: In admission control state, probing is used to ensure that >> the packets associated to a requesting flow are passing through the >> congested PCN_interior node. ECMP is one >> of the issues that are associated with the fact that packets of a flow >can >> follow a different path than the path followed by an aggregate of >flows >> (starting from the same ingress and ending at the same egress). >> In order to accomplish this the probe packets should be marked using >> either PCN_marking encoding or the PCN_Affected_marking encoding. >> In flow termination state, both the PCN_marking and >PCN_Affected_marking >> encoded packets are used to ensure that the flows that are selected >for >> termination are indeed passing through a severely congested node. >> Note that the issue that you described above is associated to the >> multicongestion-error bounds. Therefore, this multicongestion-error >bound >> should be estimated as much as >> accurate as possible. > >[phil] ok, so you agree this is a problem.=20 Georgios: The issue that I have described here is not a problem that cannot be solved. Please see the abstract description of the LC-PCN algorithm that I gave on the reply that I have sent to Anna some minutes ago! >>=20 >> > - in the termination algorithm, S4.2.2, the >"termination_offset_rate" >> > factor makes no sense to me >>=20 >> Georgios: Why not? > >[phil] well I don't understand what the point of it is. I thought I >understood the point of the *admission* offset_rate and I couldn't see >why anything similar was needed for termination. But maybe I >misunderstood the adm-offset-rate as well! Georgios: This is explained in a detailed way in the reply that I have sent to Annas comments some minutes ago! >>=20 >> > - the parameters PCN_lower_rate_egress & PCN_upper_rate_egress are >> > expressed in the wrong units (you have conditions like >> > signalled_overload_rate > PCN_lower_rate_egress; the former is a >rate, >> > the latter a % so some tweaking is needed to convert the latter into >a >> > rate) >>=20 >> Georgios: You are right, a conversion is needed to convert the latter >> into a rate. > >[phil] ok. So everything is rate. Note this raises Anna's qu: > * if pcn_lower_rate_eggress (and pcn_uppre-rate_egress) are absolute >rates, then how do you choose that abslolute rate value, given that the >rates of ingress-egress aggregates at the same ingress may be vastly >different in magnitude? Georgios: No, not everything is rate, please see my reply to Annas comments and in particular the paragraph denoted as: "Setting the thresholds at PCN_egress_nodes" Best regards, Georgios >>=20 >>=20 >> > >> > best wishes >> > >> > phil/ >> > >> > > -----Original Message----- >> > > From: Anurag Bhargava (RL/TNT) >[mailto:anurag.bhargava@ericsson.com] >> > > Sent: 11 September 2007 12:36 >> > > To: pcn >> > > Subject: [PCN] LC-PCN version 01 uploaded >> > > >> > > Hello, >> > > FYI - We have uploaded a new version of the PCN draft. The major >> > > changes are some clarification and adopting to the >> > Architecture draft >> > > terminology. >> > > >> > > "LC-PCN: The Load Control PCN Solution", Lars Westberg, 5-Sep-07, >> > > >> > > >> > > Please let me know if you have any questions. >> > > >> > > Thanks, >> > > -Anurag Bhargava, Ph.D. >> > > >> > > >> > > _______________________________________________ >> > > PCN mailing list >> > > PCN@ietf.org >> > > https://www1.ietf.org/mailman/listinfo/pcn >> > >> > >> > _______________________________________________ >> > PCN mailing list >> > PCN@ietf.org >> > https://www1.ietf.org/mailman/listinfo/pcn _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 01 04:31:34 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1InVRl-0001aO-Ty; Thu, 01 Nov 2007 04:30:37 -0400 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1InVRk-0001YI-NN for pcn-confirm+ok@megatron.ietf.org; Thu, 01 Nov 2007 04:30:36 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1InVRk-0001UG-9T for pcn@ietf.org; Thu, 01 Nov 2007 04:30:36 -0400 Received: from wrzx28.rz.uni-wuerzburg.de ([132.187.3.28] helo=mailrelay.rz.uni-wuerzburg.de) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1InVRa-0004Ev-0Q for pcn@ietf.org; Thu, 01 Nov 2007 04:30:32 -0400 Received: from virusscan.mail (localhost [127.0.0.1]) by mailrelay.mail (Postfix) with ESMTP id 84F35D53C for ; Thu, 1 Nov 2007 09:29:43 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by virusscan.mail (Postfix) with ESMTP id 781D8D541 for ; Thu, 1 Nov 2007 09:29:43 +0100 (CET) Received: from europa.informatik.uni-wuerzburg.de (wicx01.informatik.uni-wuerzburg.de [132.187.11.1]) by mailmaster.uni-wuerzburg.de (Postfix) with ESMTP id 6566BD53C for ; Thu, 1 Nov 2007 09:29:43 +0100 (CET) Received: from nero.informatik.uni-wuerzburg.de (win3005.informatik.uni-wuerzburg.de [132.187.106.5]) by europa.informatik.uni-wuerzburg.de (8.11.3/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id lA18Thh07536 for ; Thu, 1 Nov 2007 09:29:43 +0100 Received: from [127.0.0.1] (nero.informatik.uni-wuerzburg.de [132.187.106.5]) by nero.informatik.uni-wuerzburg.de (Postfix) with ESMTP id A5F826F591 for ; Thu, 1 Nov 2007 09:22:19 +0100 (CET) Message-ID: <47298F65.3090508@informatik.uni-wuerzburg.de> Date: Thu, 01 Nov 2007 09:33:41 +0100 From: Michael Menth Organization: University of Wuerzburg User-Agent: Thunderbird 2.0.0.6 (Windows/20070728) MIME-Version: 1.0 To: pcn@ietf.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at uni-wuerzburg.de X-Spam-Score: 0.0 (/) X-Scan-Signature: 2409bba43e9c8d580670fda8b695204a Subject: [PCN] Study on PCN metering and marking algorithms X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: menth@informatik.uni-wuerzburg.de List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi, we recently had a discussion about threshold and ramp marking on the list that ended with lots of speculation concerning the marking results but without any conclusion. http://www1.ietf.org/mail-archive/web/pcn/current/msg00723.html We compared threshold and ramp marking for PCN-based admission control using simulations and provided some insights concerning suitable parameters settings and tradeoffs. Here is a link to the technical report: http://www3.informatik.uni-wuerzburg.de/TR/tr437.pdf We will put a summary of the report into the next version of the 3sm draft. http://www.ietf.org/internet-drafts/draft-babiarz-pcn-3sm-00.txt Regards, Michael -- Dr. Michael Menth, Assistant Professor University of Wuerzburg, Institute of Computer Science Am Hubland, D-97074 Wuerzburg, Germany, room B206 phone: (+49)-931/888-6644, fax: (+49)-931/888-6632 mailto:menth@informatik.uni-wuerzburg.de http://www3.informatik.uni-wuerzburg.de/research/ngn _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 01 07:15:36 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1InY16-0003vv-BU; Thu, 01 Nov 2007 07:15:16 -0400 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1InY15-0003vQ-3B for pcn-confirm+ok@megatron.ietf.org; Thu, 01 Nov 2007 07:15:15 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1InY14-0003vI-LR for pcn@ietf.org; Thu, 01 Nov 2007 07:15:14 -0400 Received: from smtp.nokia.com ([131.228.20.173] helo=mgw-ext14.nokia.com) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1InY13-0004VH-OU for pcn@ietf.org; Thu, 01 Nov 2007 07:15:14 -0400 Received: from esebh108.NOE.Nokia.com (esebh108.ntc.nokia.com [172.21.143.145]) by mgw-ext14.nokia.com (Switch-3.2.5/Switch-3.2.5) with ESMTP id lA1BF63b029054; Thu, 1 Nov 2007 13:15:07 +0200 Received: from esebh103.NOE.Nokia.com ([172.21.143.33]) by esebh108.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.1830); Thu, 1 Nov 2007 13:14:38 +0200 Received: from mgw-int01.ntc.nokia.com ([172.21.143.96]) by esebh103.NOE.Nokia.com over TLS secured channel with Microsoft SMTPSVC(6.0.3790.1830); Thu, 1 Nov 2007 13:14:38 +0200 Received: from [172.21.34.231] (esdhcp034231.research.nokia.com [172.21.34.231]) by mgw-int01.ntc.nokia.com (Switch-3.2.5/Switch-3.2.5) with ESMTP id lA1BEWdq022668; Thu, 1 Nov 2007 13:14:32 +0200 In-Reply-To: <66C55C26FA491C42A9C9BB62A376DAFF01631876@E03MVB1-UKBR.domain1.systemhost.net> References: <66C55C26FA491C42A9C9BB62A376DAFF01631876@E03MVB1-UKBR.domain1.systemhost.net> Mime-Version: 1.0 (Apple Message framework v752.3) Message-Id: <0BC591DC-4D67-49CD-B70C-321511F59913@nokia.com> From: Lars Eggert Subject: Re: [PCN] traffic matrix scenario Date: Thu, 1 Nov 2007 13:14:31 +0200 To: "ext ben.strulo@bt.com" X-Mailer: Apple Mail (2.752.3) X-OriginalArrivalTime: 01 Nov 2007 11:14:38.0278 (UTC) FILETIME=[64330E60:01C81C78] X-Nokia-AV: Clean X-Spam-Score: 0.0 (/) X-Scan-Signature: 8f374d0786b25a451ef87d82c076f593 Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============2088462447==" Errors-To: pcn-bounces@ietf.org --===============2088462447== Content-Type: multipart/signed; micalg=sha1; boundary=Apple-Mail-86-860106559; protocol="application/pkcs7-signature" --Apple-Mail-86-860106559 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Hi, On 2007-10-31, at 15:51, ext ben.strulo@bt.com wrote: > Broadly speaking I think our objective is that flow termination should > only be necessary in the case of an unexpected decrease in core > capacity. We generally do not expect an Admission Control system to > admit calls and then later terminate them in the absence of internal > network problems such as link failure. I think this (how "OK" flow termination is) is the root cause of our argument about PCN. I see PCN as a "best-effort" sort of QoS. A PCN has a load range in which it operates well. You increase load by admitting flows, you get signals about the load from the markings, and then you react to it. If you over-admitted a bit, you stop admission until load abates. If you over-admitted a lot, you need to terminate flows to get the domain back into a stable state. Flow admission is how you ramp up the load, stopping admission is the "light" mechanism to reduce load over time, flow termination is the "heavy" mechanism to reduce load quickly. You can be optimistic about admitting flows, because you can always recover from wrong admission decisions. If flow termination is to be avoided at all costs, the properties of the architecture fundamentally change. All of a sudden, you need to be a *lot* more careful about when and what you admit, because all you have left to react to overload is the "light" stop-admission mechanism. You must not over-admit, so other mechanisms are needed to make sure you don't over-admit, i.e., probing. > My scenario mentioned the failure of an exchange really only as an > example of the sort of extreme scenarios we consider. The real > examples > I had in mind were simply flash crowds: widespread and unexpected > increases in call request rates with an unusual (e.g. regionally > focussed) traffic matrix. > > The sort of scenario that might be particularly testing for this > particular probing issue, would be an initial anomalous traffic > pattern > consisting of very heavy traffic on a few aggregates causing > pre-congestion on just a few links, followed by a subsequent > widespread > increase in demand which also focuses on those links. It is easy to > construct reasonable (though not necessarily probable) sequences of > external events that could cause this sort of traffic pattern: for > example, news reporting initially being local and progressing to > national coverage. I believe that it will be very difficult to design a simple architecture that can handle such cases with just stop-admission as a permitted reaction mechanism. How likely are these scenarios? > We would expect an Admission Control system to do a good job of > rejecting requests in this scenario. Though this is not completely > cut > and dried: it's possible a very small amount of flow termination might > be acceptable. > >> If we aren't in agreement, then I wonder under what >> circumstances you'd consider flow termination to be appropriate? > > As I say, only really when there is a sudden and significant > decrease in > core capacity. Even then, we would normally expect to be > provisioned to > deal with all but the most serious failures. After reading through your scenario, I believe that PCN (as I understand it) is the wrong tool for the job. You really want firm guarantees that the domain can sustain a flow before admitting it, and you never want to terminate flows unless a catastrophic event occurs. In other words, you want path-signaled QoS reservations, maybe using RSVP or NSIS. We have protocols and an architecture for path-coupled QoS, and I see no need and no benefit in PCN attempting to solve the same problem in a different way. (And if there is no other problem for PCN to solve, maybe we're done...) Lars --Apple-Mail-86-860106559 Content-Transfer-Encoding: base64 Content-Type: application/pkcs7-signature; name=smime.p7s Content-Disposition: attachment; filename=smime.p7s MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIGQDCCAvkw ggJioAMCAQICEGtxkLFHQu1g67hlmYfwPuIwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UEBhMCWkEx JTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQ ZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA3MDEwNDE2MTE1NFoXDTA4MDEwNDE2MTE1 NFowXDEPMA0GA1UEBBMGRWdnZXJ0MQ0wCwYDVQQqEwRMYXJzMRQwEgYDVQQDEwtMYXJzIEVnZ2Vy dDEkMCIGCSqGSIb3DQEJARYVbGFycy5lZ2dlcnRAbm9raWEuY29tMIIBIjANBgkqhkiG9w0BAQEF AAOCAQ8AMIIBCgKCAQEA26hjntVPZMiwh4d8tyuk9KiucvG92BVUGArk5zO1jhmq7tLFgU1mqcIg VGumfQqjkAzIuh83kxIiqBrB05Wvxp85apwt4sUCvzMe8mQWzZZZYp0rBzwpOj+VC5pxsMRYtYW+ BaTFBEmvFr7D7C7roZUk0lDppS5SZFs4SQbEe9ykB9aeUf1iTiw2+ikyP2+JEYto14WYCoxFWbms FbQ1uaaK+yn1WH2YHhyMfi0IOxuT0jtbsngjHJMpWIqq334zBhj+rPSUug47YBHr41FCDMHbbbjv WbyTyDYneOW3WY8LY8bGWVlAknQGB7lgduShe6e0RI/7SL/VE6X27lM9LwIDAQABozIwMDAgBgNV HREEGTAXgRVsYXJzLmVnZ2VydEBub2tpYS5jb20wDAYDVR0TAQH/BAIwADANBgkqhkiG9w0BAQUF AAOBgQCx3nxxTg/WowobVTRXBiXUEEZj4LBiunWbuAnR0UbJIgijvq3JbiJvZo2gUGiW9LPaHSBz 4oxT1VR/S/6O0a4e87oV5cOQm6ReB68o9rDfUzxHTwoCfv2wwMwBrXyzQMjyQK4Lt8siV41gmZpm rSXMhjTJdd9uYYNoZKQLq+VM+TCCAz8wggKooAMCAQICAQ0wDQYJKoZIhvcNAQEFBQAwgdExCzAJ BgNVBAYTAlpBMRUwEwYDVQQIEwxXZXN0ZXJuIENhcGUxEjAQBgNVBAcTCUNhcGUgVG93bjEaMBgG A1UEChMRVGhhd3RlIENvbnN1bHRpbmcxKDAmBgNVBAsTH0NlcnRpZmljYXRpb24gU2VydmljZXMg RGl2aXNpb24xJDAiBgNVBAMTG1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBDQTErMCkGCSqGSIb3 DQEJARYccGVyc29uYWwtZnJlZW1haWxAdGhhd3RlLmNvbTAeFw0wMzA3MTcwMDAwMDBaFw0xMzA3 MTYyMzU5NTlaMGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5 KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQTCBnzAN BgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAxKY8VXNV+065yplaHmjAdQRwnd/p/6Me7L3N9VvyGna9 fww6YfK/Uc4B1OVQCjDXAmNaLIkVcI7dyfArhVqqP3FWy688Cwfn8R+RNiQqE88r1fOCdz0Dviv+ uxg+B79AgAJk16emu59l0cUqVIUPSAR/p7bRPGEEQB5kGXJgt/sCAwEAAaOBlDCBkTASBgNVHRMB Af8ECDAGAQH/AgEAMEMGA1UdHwQ8MDowOKA2oDSGMmh0dHA6Ly9jcmwudGhhd3RlLmNvbS9UaGF3 dGVQZXJzb25hbEZyZWVtYWlsQ0EuY3JsMAsGA1UdDwQEAwIBBjApBgNVHREEIjAgpB4wHDEaMBgG A1UEAxMRUHJpdmF0ZUxhYmVsMi0xMzgwDQYJKoZIhvcNAQEFBQADgYEASIzRUIPqCy7MDaNmrGcP f6+svsIXoUOWlJ1/TCG4+DYfqi2fNi/A9BxQIJNwPP2t4WFiw9k6GX6EsZkbAMUaC4J0niVQlGLH 2ydxVyWN3amcOY6MIE9lX5Xa9/eH1sYITq726jTlEBpbNU1341YheILcIRk13iSx0x1G/11fZU8x ggMQMIIDDAIBATB2MGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAo UHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQ a3GQsUdC7WDruGWZh/A+4jAJBgUrDgMCGgUAoIIBbzAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcB MBwGCSqGSIb3DQEJBTEPFw0wNzExMDExMTE0MzFaMCMGCSqGSIb3DQEJBDEWBBTwhYsJVaytmP/Y eVpYM1cNs1aWtDCBhQYJKwYBBAGCNxAEMXgwdjBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhh d3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVt YWlsIElzc3VpbmcgQ0ECEGtxkLFHQu1g67hlmYfwPuIwgYcGCyqGSIb3DQEJEAILMXigdjBiMQsw CQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UE AxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0ECEGtxkLFHQu1g67hlmYfwPuIw DQYJKoZIhvcNAQEBBQAEggEAI7IDMDAV/mDbFxzGTKCQex5QTVFIrTL2ER/IlfTy6j3ryh+e7ENm lOMV/qDisJxWKxUoDzCdGw98nZ7bbLh6Re02/ne11tle25QmBTkdyTtww9+0jApReygnoo9gGC9v cyTyLlkNrFTczQxLJ4AEpaGIXbptSR4nwyyjv/UN3nMPvMzXHizcavjvN5Wm4eY01TYIIgm+rNDi yngr/4FK1jT1wswmoujBmNCyyFDZZVJWAihZmTsNHjGQ9w6V2LmyFweqovz6sO6ASgZZxdBAbgul sUxWwROwl1tb45ULajPE1IDD8l8Ol6ZeR2ZkTIMz4g4/CzCrS1s5PZMWoa3bBAAAAAAAAA== --Apple-Mail-86-860106559-- --===============2088462447== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn --===============2088462447==-- From pcn-bounces@ietf.org Thu Nov 01 07:39:01 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1InYNd-00023I-5z; Thu, 01 Nov 2007 07:38:33 -0400 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1InYNb-000201-JF for pcn-confirm+ok@megatron.ietf.org; Thu, 01 Nov 2007 07:38:31 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1InYNb-0001zo-0R for pcn@ietf.org; Thu, 01 Nov 2007 07:38:31 -0400 Received: from smtp.nokia.com ([131.228.20.173] helo=mgw-ext14.nokia.com) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1InYNZ-0005Eh-RS for pcn@ietf.org; Thu, 01 Nov 2007 07:38:30 -0400 Received: from esebh106.NOE.Nokia.com (esebh106.ntc.nokia.com [172.21.138.213]) by mgw-ext14.nokia.com (Switch-3.2.5/Switch-3.2.5) with ESMTP id lA1BcLVW016843; Thu, 1 Nov 2007 13:38:26 +0200 Received: from esebh103.NOE.Nokia.com ([172.21.143.33]) by esebh106.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.1830); Thu, 1 Nov 2007 13:38:16 +0200 Received: from mgw-int01.ntc.nokia.com ([172.21.143.96]) by esebh103.NOE.Nokia.com over TLS secured channel with Microsoft SMTPSVC(6.0.3790.1830); Thu, 1 Nov 2007 13:38:16 +0200 Received: from [172.21.34.231] (esdhcp034231.research.nokia.com [172.21.34.231]) by mgw-int01.ntc.nokia.com (Switch-3.2.5/Switch-3.2.5) with ESMTP id lA1BcFx7011188; Thu, 1 Nov 2007 13:38:15 +0200 Mime-Version: 1.0 (Apple Message framework v752.3) To: pcn Message-Id: From: Lars Eggert Date: Thu, 1 Nov 2007 13:38:13 +0200 X-Mailer: Apple Mail (2.752.3) X-OriginalArrivalTime: 01 Nov 2007 11:38:16.0732 (UTC) FILETIME=[B1A9F5C0:01C81C7B] X-Nokia-AV: Clean X-Spam-Score: 0.0 (/) X-Scan-Signature: 17e5edc4dfd335965c1d21372171c01c Cc: Subject: [PCN] 5.5. Probing functions X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0969721259==" Errors-To: pcn-bounces@ietf.org --===============0969721259== Content-Type: multipart/signed; micalg=sha1; boundary=Apple-Mail-88-861529103; protocol="application/pkcs7-signature" --Apple-Mail-88-861529103 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Hi, I read through the section on probing in the architecture draft, and have some comments. Folks know that I'm not too thrilled with the idea of probing, so this should come as no surprise... > 5.5. Probing functions > > Probing functions are optional, and can be used for admission > control. > > PCN's admission control, as described so far, is essentially a > reactive mechanism where the PCN-egress-node monitors the pre- > congestion level for traffic from each PCN-ingress-node; if the > level > rises then it blocks new flows on that ingress-egress-aggregate. > However, it's possible that an ingress-egress-aggregate carries no > traffic, and so the PCN-egress-node can't make an admission > decision > using the usual method described earlier. > > One approach is to be "optimistic" and simply admit the new flow. > However it's possible to envisage a scenario where the traffic > levels > on other ingress-egress-aggregates are already so high that they're > blocking new PCN-flows and admitting a new flow onto this 'empty' > ingress-egress-aggregate would add extra traffic onto the link > that's > already pre-congested - which may 'tip the balance' so that PCN's > flow termination mechanism is activated or some packets are > dropped. > This risk could be lessened by configuring on each link sufficient > 'safety margin' above the PCN-lower-rate. > > An alternative approach is to make PCN a more proactive mechanism. > The PCN-ingress-node explicitly determines, before admitting the > prospective new flow, whether the ingress-egress-aggregate can > support it. This can be seen as a "pessimistic" approach, in > contrast to the "optimism" of the approach above. It involves > probing: a PCN-ingress-node generates and sends probe packets in > order to test the pre-congestion level that the flow would > experience. The above is a really good intro to the problem - nice job. > A probe packet is just a dummy data packet, generated by > the PCN-ingress-node and addressed to the PCN-egress-node. A > downside of probing is that it adds delay to the admission control > process. Also note that in the scenario described in the previous > paragraph (where traffic levels on other ingress-egress- > aggregates is > already very high), the probe packets may also 'tip the balance'. This discussion on the downsides of probing is too short. Let me list some issues: (1) The ingress needs to generate traffic in a pattern and at a rate that lets it draw conclusions about whether it is OK to admit the flow that is waiting for admission. How does it know what characteristics the probe traffic should have and how does it generate the probe traffic? (2) How long do you need to probe for before you declare it safe to admit the new flow? What's the delay before a new flow can be admitted, and can apps actually deal with this delay? (3) Since you're sending probe traffic at a rate that is likely not insignificant, how do you prevent the probe traffic itself from causing congestion and triggering stop-admission or flow-termination actions? If you're treating it differently (e.g., at a lower priority), how is what the probe traffic experiences still representative of what the real flow would experience? (How can you treat it differently and still make ECMP work?) Without at least some good ideas about what the answers to these questions will be I believe it is premature to declare that probing is an optional component of PCN. > However, the risk should be reduced because it should be > possible to > send probe packets for a shorter time and at a lower rate than a > typical data flow. This claim is related to (1) and (2) above. Is there any evidence that is it possible to "send probe packets for a shorter time and at a lower rate than a typical data flow"? I'm not aware of any research in this space. > The situation is more complicated if there is multipath routing > (ECMP) in the PCN-domain. It is then possible for some paths to be > pre-congested whilst other paths within the same ingress-egress- > aggregate aren't pre-congested. > > One approach essentially ignores ECMP: as usual, admit or block > a new > flow depending on the "measurements of PCN-traffic" on the ingress- > egress-aggregate. This is rather similar to the "optimistic" > approach above. > > An alternative ("pessimistic" or "proactive") approach is to probe > the ECMP path. The PCN-ingress-node generates and sends probe > packets (dummy data) that follow the specific ECMP path that the > new > flow would do, in order to test the pre-congestion level along it. > An ECMP algorithm typically examines: the source and destination IP > addresses and port numbers, the protocol ID and the DSCP. Hence > these fields must have the same values in the probe packets as the > future data packets would have. On the other hand, the PCN-egress- > node needs to consume the probe packets to ensure that they don't > travel beyond the PCN-domain (eg they might confuse the destination > end node). Hence somehow the PCN-egress-node has to be able to > disambiguate a probe packet from a data packet, via the > characteristic setting of particular bit(s) in the packet's > header or > body - but these bit(s) mustn't be used by any PCN-interior-node's > ECMP algorithm. This should be possible with a typical ECMP > algorithm, but isn't in the general case. Some things in this paragraph aren't specific to ECMP, such as that the egress needs to consume the probe packets. > The probing functions are: > > o Make decision that probing is needed. As described above, > this is > when the ingress-egress-aggregate or the ECMP path carries no > PCN- > traffic. An alternative is always to probe, ie probe before > admitting every PCN-flow. > > o (if required) Communicate the request that probing is needed > - the > PCN-egress-node signals to the PCN-ingress-node that probing is > needed > > o Generate probe traffic - the PCN-ingress-node generates the > probe > traffic. The appropriate number (or rate) of probe packets will > depend on the PCN-marking algorithm; for example an excess-rate- > marking algorithm generates fewer PCN-marks than a threshold- > marking algorithm, and so will need more probe packets. > > o Forward probe packets - as far as PCN-interior-nodes are > concerned, probe packets must be handled the same as (ordinary > data) PCN-packets, in terms of routing, scheduling and PCN- > marking. > > o Consume probe packets - the PCN-egress-node consumes probe > packets > to ensure that they don't travel beyond the PCN-domain. Nice summary of the required functions. Lars --Apple-Mail-88-861529103 Content-Transfer-Encoding: base64 Content-Type: application/pkcs7-signature; name=smime.p7s Content-Disposition: attachment; filename=smime.p7s MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIGQDCCAvkw ggJioAMCAQICEGtxkLFHQu1g67hlmYfwPuIwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UEBhMCWkEx JTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQ ZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA3MDEwNDE2MTE1NFoXDTA4MDEwNDE2MTE1 NFowXDEPMA0GA1UEBBMGRWdnZXJ0MQ0wCwYDVQQqEwRMYXJzMRQwEgYDVQQDEwtMYXJzIEVnZ2Vy dDEkMCIGCSqGSIb3DQEJARYVbGFycy5lZ2dlcnRAbm9raWEuY29tMIIBIjANBgkqhkiG9w0BAQEF AAOCAQ8AMIIBCgKCAQEA26hjntVPZMiwh4d8tyuk9KiucvG92BVUGArk5zO1jhmq7tLFgU1mqcIg VGumfQqjkAzIuh83kxIiqBrB05Wvxp85apwt4sUCvzMe8mQWzZZZYp0rBzwpOj+VC5pxsMRYtYW+ BaTFBEmvFr7D7C7roZUk0lDppS5SZFs4SQbEe9ykB9aeUf1iTiw2+ikyP2+JEYto14WYCoxFWbms FbQ1uaaK+yn1WH2YHhyMfi0IOxuT0jtbsngjHJMpWIqq334zBhj+rPSUug47YBHr41FCDMHbbbjv WbyTyDYneOW3WY8LY8bGWVlAknQGB7lgduShe6e0RI/7SL/VE6X27lM9LwIDAQABozIwMDAgBgNV HREEGTAXgRVsYXJzLmVnZ2VydEBub2tpYS5jb20wDAYDVR0TAQH/BAIwADANBgkqhkiG9w0BAQUF AAOBgQCx3nxxTg/WowobVTRXBiXUEEZj4LBiunWbuAnR0UbJIgijvq3JbiJvZo2gUGiW9LPaHSBz 4oxT1VR/S/6O0a4e87oV5cOQm6ReB68o9rDfUzxHTwoCfv2wwMwBrXyzQMjyQK4Lt8siV41gmZpm rSXMhjTJdd9uYYNoZKQLq+VM+TCCAz8wggKooAMCAQICAQ0wDQYJKoZIhvcNAQEFBQAwgdExCzAJ BgNVBAYTAlpBMRUwEwYDVQQIEwxXZXN0ZXJuIENhcGUxEjAQBgNVBAcTCUNhcGUgVG93bjEaMBgG A1UEChMRVGhhd3RlIENvbnN1bHRpbmcxKDAmBgNVBAsTH0NlcnRpZmljYXRpb24gU2VydmljZXMg RGl2aXNpb24xJDAiBgNVBAMTG1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBDQTErMCkGCSqGSIb3 DQEJARYccGVyc29uYWwtZnJlZW1haWxAdGhhd3RlLmNvbTAeFw0wMzA3MTcwMDAwMDBaFw0xMzA3 MTYyMzU5NTlaMGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5 KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQTCBnzAN BgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAxKY8VXNV+065yplaHmjAdQRwnd/p/6Me7L3N9VvyGna9 fww6YfK/Uc4B1OVQCjDXAmNaLIkVcI7dyfArhVqqP3FWy688Cwfn8R+RNiQqE88r1fOCdz0Dviv+ uxg+B79AgAJk16emu59l0cUqVIUPSAR/p7bRPGEEQB5kGXJgt/sCAwEAAaOBlDCBkTASBgNVHRMB Af8ECDAGAQH/AgEAMEMGA1UdHwQ8MDowOKA2oDSGMmh0dHA6Ly9jcmwudGhhd3RlLmNvbS9UaGF3 dGVQZXJzb25hbEZyZWVtYWlsQ0EuY3JsMAsGA1UdDwQEAwIBBjApBgNVHREEIjAgpB4wHDEaMBgG A1UEAxMRUHJpdmF0ZUxhYmVsMi0xMzgwDQYJKoZIhvcNAQEFBQADgYEASIzRUIPqCy7MDaNmrGcP f6+svsIXoUOWlJ1/TCG4+DYfqi2fNi/A9BxQIJNwPP2t4WFiw9k6GX6EsZkbAMUaC4J0niVQlGLH 2ydxVyWN3amcOY6MIE9lX5Xa9/eH1sYITq726jTlEBpbNU1341YheILcIRk13iSx0x1G/11fZU8x ggMQMIIDDAIBATB2MGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAo UHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQ a3GQsUdC7WDruGWZh/A+4jAJBgUrDgMCGgUAoIIBbzAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcB MBwGCSqGSIb3DQEJBTEPFw0wNzExMDExMTM4MTRaMCMGCSqGSIb3DQEJBDEWBBRHQYwF+2y7bsim kbpXmZ6dIjWgsjCBhQYJKwYBBAGCNxAEMXgwdjBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhh d3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVt YWlsIElzc3VpbmcgQ0ECEGtxkLFHQu1g67hlmYfwPuIwgYcGCyqGSIb3DQEJEAILMXigdjBiMQsw CQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UE AxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0ECEGtxkLFHQu1g67hlmYfwPuIw DQYJKoZIhvcNAQEBBQAEggEAr5X5qGxhfES1lSTtlG/ergxuhVHMdCF/gWeoGmJz2b9SIvM5JOhc X7SuQGseUZpKvgeS31qp+D24TWYneEiHuqHfNeoWvuJ9UlofYGbjcp25xsMQGhL3j0JMaNeVdLD5 TmIF0i2a72xBCoaZfUihvvp3c7Wil/DLS5PZlxaMWv+svRNfMRJwRRnny7VZPjKen+VztvZSwJ3p WtTwgeXkqAtC/mj/DVuOD7dowUQeTACeSbzbjS6NdjGA5xcXYodtlEk36CisJ+brKEZLXvC47MHY 7oQOweSYhTguQGFv9T4M2yDoE/8w9T7V6lkqsCEgyJnIFtcgxE/S2u/ywIc8pAAAAAAAAA== --Apple-Mail-88-861529103-- --===============0969721259== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn --===============0969721259==-- From pcn-bounces@ietf.org Thu Nov 01 08:25:35 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1InZ6l-000288-TB; Thu, 01 Nov 2007 08:25:11 -0400 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1InZ6k-00027N-JS for pcn-confirm+ok@megatron.ietf.org; Thu, 01 Nov 2007 08:25:10 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1InZ6j-00026C-SB for pcn@ietf.org; Thu, 01 Nov 2007 08:25:09 -0400 Received: from smtp4.smtp.bt.com ([217.32.164.151]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1InZ6j-0007Ng-1d for pcn@ietf.org; Thu, 01 Nov 2007 08:25:09 -0400 Received: from E03MVB1-UKBR.domain1.systemhost.net ([193.113.197.108]) by smtp4.smtp.bt.com with Microsoft SMTPSVC(6.0.3790.1830); Thu, 1 Nov 2007 12:25:07 +0000 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [PCN] traffic matrix scenario Date: Thu, 1 Nov 2007 12:25:07 -0000 Message-ID: <66C55C26FA491C42A9C9BB62A376DAFF01631DF6@E03MVB1-UKBR.domain1.systemhost.net> In-Reply-To: <0BC591DC-4D67-49CD-B70C-321511F59913@nokia.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] traffic matrix scenario Thread-Index: AcgceHifgxZOLA2FQsaMBLg6JPelmAACDV9A From: To: X-OriginalArrivalTime: 01 Nov 2007 12:25:07.0971 (UTC) FILETIME=[3D4AFD30:01C81C82] X-Spam-Score: 0.0 (/) X-Scan-Signature: ff03b0075c3fc728d7d60a15b4ee1ad2 Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi, Interestingly, I mostly agree with you. I am not particularly convinced that probing is a good route to go down, and I suspect that PCN will have a great deal of difficulty in working well with ECMP. On the other hand I think that there can be reasonable scenarios where PCN can be engineered to make flow termination (caused by over-admission) extremely unlikely without completely changing the architecture. I have in mind scenarios where the safety margin is large, and the admission process is carefully managed (and aggregation levels are large). The question that most interests me then is how well does PCN perform? How large does the safety margin need to be? How tightly capped does admission control need to be? I do not have complete answers, but the work we have done so far leads me to believe that there are credible scenarios in which (some version of) PCN is an appropriate and reasonably efficient solution. My chief concern then is to ensure that the PCN WG chooses options that work as well as possible in as many scenarios as possible. Ben > -----Original Message----- > From: Lars Eggert [mailto:lars.eggert@nokia.com]=20 > Sent: 01 November 2007 11:15 > To: Strulo,B,Ben,CXR9 R > Cc: pcn@ietf.org > Subject: Re: [PCN] traffic matrix scenario >=20 > Hi, >=20 > On 2007-10-31, at 15:51, ext ben.strulo@bt.com wrote: > > Broadly speaking I think our objective is that flow=20 > termination should=20 > > only be necessary in the case of an unexpected decrease in core=20 > > capacity. We generally do not expect an Admission Control=20 > system to=20 > > admit calls and then later terminate them in the absence of=20 > internal=20 > > network problems such as link failure. >=20 > I think this (how "OK" flow termination is) is the root cause=20 > of our argument about PCN. >=20 > I see PCN as a "best-effort" sort of QoS. A PCN has a load=20 > range in which it operates well. You increase load by=20 > admitting flows, you get signals about the load from the=20 > markings, and then you react to it. =20 > If you over-admitted a bit, you stop admission until load=20 > abates. If you over-admitted a lot, you need to terminate=20 > flows to get the domain back into a stable state. Flow=20 > admission is how you ramp up the load, stopping admission is=20 > the "light" mechanism to reduce load over time, flow=20 > termination is the "heavy" mechanism to reduce load quickly.=20 > You can be optimistic about admitting flows, because you can=20 > always recover from wrong admission decisions. >=20 > If flow termination is to be avoided at all costs, the=20 > properties of the architecture fundamentally change. All of a=20 > sudden, you need to be a *lot* more careful about when and=20 > what you admit, because all you have left to react to=20 > overload is the "light" stop-admission mechanism. You must=20 > not over-admit, so other mechanisms are needed to make sure=20 > you don't over-admit, i.e., probing. >=20 > > My scenario mentioned the failure of an exchange really only as an=20 > > example of the sort of extreme scenarios we consider. The real=20 > > examples I had in mind were simply flash crowds: widespread and=20 > > unexpected increases in call request rates with an unusual (e.g.=20 > > regionally > > focussed) traffic matrix. > > > > The sort of scenario that might be particularly testing for this=20 > > particular probing issue, would be an initial anomalous traffic=20 > > pattern consisting of very heavy traffic on a few=20 > aggregates causing=20 > > pre-congestion on just a few links, followed by a subsequent=20 > > widespread increase in demand which also focuses on those=20 > links. It=20 > > is easy to construct reasonable (though not necessarily probable)=20 > > sequences of external events that could cause this sort of traffic=20 > > pattern: for example, news reporting initially being local and=20 > > progressing to national coverage. >=20 > I believe that it will be very difficult to design a simple=20 > architecture that can handle such cases with just=20 > stop-admission as a permitted reaction mechanism. >=20 > How likely are these scenarios? >=20 > > We would expect an Admission Control system to do a good job of=20 > > rejecting requests in this scenario. Though this is not completely=20 > > cut and dried: it's possible a very small amount of flow=20 > termination=20 > > might be acceptable. > > > >> If we aren't in agreement, then I wonder under what circumstances=20 > >> you'd consider flow termination to be appropriate? > > > > As I say, only really when there is a sudden and=20 > significant decrease=20 > > in core capacity. Even then, we would normally expect to be=20 > > provisioned to deal with all but the most serious failures. >=20 > After reading through your scenario, I believe that PCN (as I=20 > understand it) is the wrong tool for the job. You really want=20 > firm guarantees that the domain can sustain a flow before=20 > admitting it, and you never want to terminate flows unless a=20 > catastrophic event occurs. In other words, you want=20 > path-signaled QoS reservations, maybe using RSVP or NSIS. We=20 > have protocols and an architecture for path-coupled QoS, and=20 > I see no need and no benefit in PCN attempting to solve the=20 > same problem in a different way. >=20 > (And if there is no other problem for PCN to solve, maybe=20 > we're done...) >=20 > Lars _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 01 08:40:54 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1InZLh-0006m1-75; Thu, 01 Nov 2007 08:40:37 -0400 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1InZLf-0006kk-Jc for pcn-confirm+ok@megatron.ietf.org; Thu, 01 Nov 2007 08:40:35 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1InZLf-0006kU-5v for pcn@ietf.org; Thu, 01 Nov 2007 08:40:35 -0400 Received: from rotterdam.ewi.utwente.nl ([130.89.10.5]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1InZLY-0008FK-Nv for pcn@ietf.org; Thu, 01 Nov 2007 08:40:35 -0400 Received: from utip105 (utip105.ewi.utwente.nl [130.89.13.76]) by rotterdam.ewi.utwente.nl (8.13.6/8.13.6) with ESMTP id lA1CdEun016742; Thu, 1 Nov 2007 13:40:24 +0100 (MET) From: "Georgios Karagiannis" To: "'Lars Eggert'" , "'pcn'" References: Subject: RE: [PCN] 5.5. Probing functions Date: Thu, 1 Nov 2007 13:39:07 +0100 Message-ID: <000f01c81c84$5635b400$4c0d5982@dynamic.ewi.utwente.nl> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 11 In-Reply-To: X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3198 Thread-Index: Acgce/ZeWG+LpeNgQzWdHjNX/OnIpQABxZNQ X-Spam-Score: 0 () X-Scanned-By: MIMEDefang 2.52 on 130.89.10.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0rc3 (rotterdam.ewi.utwente.nl [130.89.10.5]); Thu, 01 Nov 2007 13:40:25 +0100 (MET) X-Spam-Score: 0.0 (/) X-Scan-Signature: f2728948111f2edaaf8980b5b9de55af Cc: X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Lars Please receive some answers on the three questions! > (1) The ingress needs to generate traffic in a pattern and at > a rate that lets it draw conclusions about whether it is OK > to admit the flow that is waiting for admission. How does it > know what characteristics the probe traffic should have and > how does it generate the probe traffic? Georgios: The PCN_ingress_node generates one probe packet for each new incoming flow that requests access from the PCN_domain. > > (2) How long do you need to probe for before you declare it > safe to admit the new flow? Georgios: One probe packet per new incoming and flow is enough if the PCN_domain takes care that the probe packet will be marked PCN_marked or using an other type of encoding, if it is passing through a congested PCN_interior_node. > What's the delay before a new > flow can be admitted, and can apps actually deal with this delay? The delay = one RTT (within PCN_domain). > > (3) Since you're sending probe traffic at a rate that is > likely not insignificant, how do you prevent the probe > traffic itself from causing congestion and triggering > stop-admission or flow-termination actions? If you're > treating it differently (e.g., at a lower priority), how is > what the probe traffic experiences still representative of > what the real flow would experience? (How can you treat it > differently and still make ECMP work?) Georgios: By just sending one probe packet per new incoming flow that requests access into the PCN_domain. Best regards, Georgios > -----Original Message----- > From: Lars Eggert [mailto:lars.eggert@nokia.com] > Sent: donderdag 1 november 2007 12:38 > To: pcn > Subject: [PCN] 5.5. Probing functions > > Hi, > > I read through the section on probing in the architecture > draft, and have some comments. Folks know that I'm not too > thrilled with the idea of probing, so this should come as no > surprise... > > > 5.5. Probing functions > > > > Probing functions are optional, and can be used for admission > > control. > > > > PCN's admission control, as described so far, is essentially a > > reactive mechanism where the PCN-egress-node monitors the pre- > > congestion level for traffic from each PCN-ingress-node; if the > > level > > rises then it blocks new flows on that ingress-egress-aggregate. > > However, it's possible that an ingress-egress-aggregate > carries no > > traffic, and so the PCN-egress-node can't make an admission > > decision > > using the usual method described earlier. > > > > One approach is to be "optimistic" and simply admit the new flow. > > However it's possible to envisage a scenario where the traffic > > levels > > on other ingress-egress-aggregates are already so high > that they're > > blocking new PCN-flows and admitting a new flow onto this 'empty' > > ingress-egress-aggregate would add extra traffic onto the link > > that's > > already pre-congested - which may 'tip the balance' so that PCN's > > flow termination mechanism is activated or some packets are > > dropped. > > This risk could be lessened by configuring on each link > sufficient > > 'safety margin' above the PCN-lower-rate. > > > > An alternative approach is to make PCN a more proactive > mechanism. > > The PCN-ingress-node explicitly determines, before admitting the > > prospective new flow, whether the ingress-egress-aggregate can > > support it. This can be seen as a "pessimistic" approach, in > > contrast to the "optimism" of the approach above. It involves > > probing: a PCN-ingress-node generates and sends probe packets in > > order to test the pre-congestion level that the flow would > > experience. > > The above is a really good intro to the problem - nice job. > > > A probe packet is just a dummy data packet, generated by > > the PCN-ingress-node and addressed to the PCN-egress-node. A > > downside of probing is that it adds delay to the > admission control > > process. Also note that in the scenario described in > the previous > > paragraph (where traffic levels on other ingress-egress- > aggregates > > is > > already very high), the probe packets may also 'tip the balance'. > > This discussion on the downsides of probing is too short. Let > me list some issues: > > (1) The ingress needs to generate traffic in a pattern and at > a rate that lets it draw conclusions about whether it is OK > to admit the flow that is waiting for admission. How does it > know what characteristics the probe traffic should have and > how does it generate the probe traffic? > > (2) How long do you need to probe for before you declare it > safe to admit the new flow? What's the delay before a new > flow can be admitted, and can apps actually deal with this delay? > > (3) Since you're sending probe traffic at a rate that is > likely not insignificant, how do you prevent the probe > traffic itself from causing congestion and triggering > stop-admission or flow-termination actions? If you're > treating it differently (e.g., at a lower priority), how is > what the probe traffic experiences still representative of > what the real flow would experience? (How can you treat it > differently and still make ECMP work?) > > Without at least some good ideas about what the answers to > these questions will be I believe it is premature to declare > that probing is an optional component of PCN. > > > However, the risk should be reduced because it should be > possible > > to > > send probe packets for a shorter time and at a lower rate than a > > typical data flow. > > This claim is related to (1) and (2) above. Is there any > evidence that is it possible to "send probe packets for a > shorter time and at a lower rate than a typical data flow"? > I'm not aware of any research in this space. > > > The situation is more complicated if there is multipath routing > > (ECMP) in the PCN-domain. It is then possible for some > paths to be > > pre-congested whilst other paths within the same ingress-egress- > > aggregate aren't pre-congested. > > > > One approach essentially ignores ECMP: as usual, admit > or block a > > new > > flow depending on the "measurements of PCN-traffic" on > the ingress- > > egress-aggregate. This is rather similar to the "optimistic" > > approach above. > > > > An alternative ("pessimistic" or "proactive") approach > is to probe > > the ECMP path. The PCN-ingress-node generates and sends probe > > packets (dummy data) that follow the specific ECMP path that the > > new > > flow would do, in order to test the pre-congestion level > along it. > > An ECMP algorithm typically examines: the source and > destination IP > > addresses and port numbers, the protocol ID and the DSCP. Hence > > these fields must have the same values in the probe > packets as the > > future data packets would have. On the other hand, the > PCN-egress- > > node needs to consume the probe packets to ensure that they don't > > travel beyond the PCN-domain (eg they might confuse the > destination > > end node). Hence somehow the PCN-egress-node has to be able to > > disambiguate a probe packet from a data packet, via the > > characteristic setting of particular bit(s) in the > packet's header > > or > > body - but these bit(s) mustn't be used by any > PCN-interior-node's > > ECMP algorithm. This should be possible with a typical ECMP > > algorithm, but isn't in the general case. > > Some things in this paragraph aren't specific to ECMP, such > as that the egress needs to consume the probe packets. > > > The probing functions are: > > > > o Make decision that probing is needed. As described > above, this > > is > > when the ingress-egress-aggregate or the ECMP path carries no > > PCN- > > traffic. An alternative is always to probe, ie probe before > > admitting every PCN-flow. > > > > o (if required) Communicate the request that probing is needed > > - the > > PCN-egress-node signals to the PCN-ingress-node that > probing is > > needed > > > > o Generate probe traffic - the PCN-ingress-node generates the > > probe > > traffic. The appropriate number (or rate) of probe > packets will > > depend on the PCN-marking algorithm; for example an > excess-rate- > > marking algorithm generates fewer PCN-marks than a threshold- > > marking algorithm, and so will need more probe packets. > > > > o Forward probe packets - as far as PCN-interior-nodes are > > concerned, probe packets must be handled the same as (ordinary > > data) PCN-packets, in terms of routing, scheduling and PCN- > > marking. > > > > o Consume probe packets - the PCN-egress-node consumes probe > > packets > > to ensure that they don't travel beyond the PCN-domain. > > Nice summary of the required functions. > > Lars > _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 01 08:49:34 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1InZTn-0006tx-Vr; Thu, 01 Nov 2007 08:48:59 -0400 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1InZTm-0006sp-AJ for pcn-confirm+ok@megatron.ietf.org; Thu, 01 Nov 2007 08:48:58 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1InZTl-0006sh-SM for pcn@ietf.org; Thu, 01 Nov 2007 08:48:57 -0400 Received: from smtp.nokia.com ([131.228.20.173] helo=mgw-ext14.nokia.com) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1InZTe-00006R-LS for pcn@ietf.org; Thu, 01 Nov 2007 08:48:57 -0400 Received: from esebh107.NOE.Nokia.com (esebh107.ntc.nokia.com [172.21.143.143]) by mgw-ext14.nokia.com (Switch-3.2.5/Switch-3.2.5) with ESMTP id lA1CmcUc013729; Thu, 1 Nov 2007 14:48:47 +0200 Received: from esebh104.NOE.Nokia.com ([172.21.143.34]) by esebh107.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.1830); Thu, 1 Nov 2007 14:48:46 +0200 Received: from esebh102.NOE.Nokia.com ([172.21.138.183]) by esebh104.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.1830); Thu, 1 Nov 2007 14:48:46 +0200 Received: from mgw-int01.ntc.nokia.com ([172.21.143.96]) by esebh102.NOE.Nokia.com over TLS secured channel with Microsoft SMTPSVC(6.0.3790.1830); Thu, 1 Nov 2007 14:48:45 +0200 Received: from [172.21.34.231] (esdhcp034231.research.nokia.com [172.21.34.231]) by mgw-int01.ntc.nokia.com (Switch-3.2.5/Switch-3.2.5) with ESMTP id lA1Cmi9A008022; Thu, 1 Nov 2007 14:48:44 +0200 In-Reply-To: <66C55C26FA491C42A9C9BB62A376DAFF01631DF6@E03MVB1-UKBR.domain1.systemhost.net> References: <66C55C26FA491C42A9C9BB62A376DAFF01631DF6@E03MVB1-UKBR.domain1.systemhost.net> Mime-Version: 1.0 (Apple Message framework v752.3) Message-Id: From: Lars Eggert Subject: Re: [PCN] traffic matrix scenario Date: Thu, 1 Nov 2007 14:48:43 +0200 To: "ext ben.strulo@bt.com" X-Mailer: Apple Mail (2.752.3) X-OriginalArrivalTime: 01 Nov 2007 12:48:45.0768 (UTC) FILETIME=[8A5DA480:01C81C85] X-Nokia-AV: Clean X-Spam-Score: 0.0 (/) X-Scan-Signature: a87a9cdae4ac5d3fbeee75cd0026d632 Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1951931746==" Errors-To: pcn-bounces@ietf.org --===============1951931746== Content-Type: multipart/signed; micalg=sha1; boundary=Apple-Mail-92-865758512; protocol="application/pkcs7-signature" --Apple-Mail-92-865758512 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed On 2007-11-1, at 14:25, ext ben.strulo@bt.com wrote: > I am not particularly convinced that probing is a good route to go > down, > and I suspect that PCN will have a great deal of difficulty in working > well with ECMP. I'm happy to hear I'm not alone on this :-) > On the other hand I think that there can be reasonable scenarios where > PCN can be engineered to make flow termination (caused by > over-admission) extremely unlikely without completely changing the > architecture. I have in mind scenarios where the safety margin is > large, and the admission process is carefully managed (and aggregation > levels are large). The question that most interests me then is how > well > does PCN perform? How large does the safety margin need to be? How > tightly capped does admission control need to be? These are exactly the right questions to ask. When we chartered PCN, the intuition was that flow admission and termination combined with a sufficient degree of over-provisioning and the right marking behavior result in a simple and useful system that would establish a degree of QoS. The question is now how exactly we need to define the parameters to derive such a system for a given network scenario. > I do not have complete answers, but the work we have done so far leads > me to believe that there are credible scenarios in which (some version > of) PCN is an appropriate and reasonably efficient solution. I'm happy to hear folks are working on this - I believe that understanding which scenarios are appropriate for solving with PCN techniques (vs. other techniques such as path-coupled reservations) is important. > My chief concern then is to ensure that the PCN WG chooses options > that > work as well as possible in as many scenarios as possible. If you add "while remaining architecturally simple and robust", I completely agree. Lars --Apple-Mail-92-865758512 Content-Transfer-Encoding: base64 Content-Type: application/pkcs7-signature; name=smime.p7s Content-Disposition: attachment; filename=smime.p7s MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIGQDCCAvkw ggJioAMCAQICEGtxkLFHQu1g67hlmYfwPuIwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UEBhMCWkEx JTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQ ZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA3MDEwNDE2MTE1NFoXDTA4MDEwNDE2MTE1 NFowXDEPMA0GA1UEBBMGRWdnZXJ0MQ0wCwYDVQQqEwRMYXJzMRQwEgYDVQQDEwtMYXJzIEVnZ2Vy dDEkMCIGCSqGSIb3DQEJARYVbGFycy5lZ2dlcnRAbm9raWEuY29tMIIBIjANBgkqhkiG9w0BAQEF AAOCAQ8AMIIBCgKCAQEA26hjntVPZMiwh4d8tyuk9KiucvG92BVUGArk5zO1jhmq7tLFgU1mqcIg VGumfQqjkAzIuh83kxIiqBrB05Wvxp85apwt4sUCvzMe8mQWzZZZYp0rBzwpOj+VC5pxsMRYtYW+ BaTFBEmvFr7D7C7roZUk0lDppS5SZFs4SQbEe9ykB9aeUf1iTiw2+ikyP2+JEYto14WYCoxFWbms FbQ1uaaK+yn1WH2YHhyMfi0IOxuT0jtbsngjHJMpWIqq334zBhj+rPSUug47YBHr41FCDMHbbbjv WbyTyDYneOW3WY8LY8bGWVlAknQGB7lgduShe6e0RI/7SL/VE6X27lM9LwIDAQABozIwMDAgBgNV HREEGTAXgRVsYXJzLmVnZ2VydEBub2tpYS5jb20wDAYDVR0TAQH/BAIwADANBgkqhkiG9w0BAQUF AAOBgQCx3nxxTg/WowobVTRXBiXUEEZj4LBiunWbuAnR0UbJIgijvq3JbiJvZo2gUGiW9LPaHSBz 4oxT1VR/S/6O0a4e87oV5cOQm6ReB68o9rDfUzxHTwoCfv2wwMwBrXyzQMjyQK4Lt8siV41gmZpm rSXMhjTJdd9uYYNoZKQLq+VM+TCCAz8wggKooAMCAQICAQ0wDQYJKoZIhvcNAQEFBQAwgdExCzAJ BgNVBAYTAlpBMRUwEwYDVQQIEwxXZXN0ZXJuIENhcGUxEjAQBgNVBAcTCUNhcGUgVG93bjEaMBgG A1UEChMRVGhhd3RlIENvbnN1bHRpbmcxKDAmBgNVBAsTH0NlcnRpZmljYXRpb24gU2VydmljZXMg RGl2aXNpb24xJDAiBgNVBAMTG1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBDQTErMCkGCSqGSIb3 DQEJARYccGVyc29uYWwtZnJlZW1haWxAdGhhd3RlLmNvbTAeFw0wMzA3MTcwMDAwMDBaFw0xMzA3 MTYyMzU5NTlaMGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5 KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQTCBnzAN BgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAxKY8VXNV+065yplaHmjAdQRwnd/p/6Me7L3N9VvyGna9 fww6YfK/Uc4B1OVQCjDXAmNaLIkVcI7dyfArhVqqP3FWy688Cwfn8R+RNiQqE88r1fOCdz0Dviv+ uxg+B79AgAJk16emu59l0cUqVIUPSAR/p7bRPGEEQB5kGXJgt/sCAwEAAaOBlDCBkTASBgNVHRMB Af8ECDAGAQH/AgEAMEMGA1UdHwQ8MDowOKA2oDSGMmh0dHA6Ly9jcmwudGhhd3RlLmNvbS9UaGF3 dGVQZXJzb25hbEZyZWVtYWlsQ0EuY3JsMAsGA1UdDwQEAwIBBjApBgNVHREEIjAgpB4wHDEaMBgG A1UEAxMRUHJpdmF0ZUxhYmVsMi0xMzgwDQYJKoZIhvcNAQEFBQADgYEASIzRUIPqCy7MDaNmrGcP f6+svsIXoUOWlJ1/TCG4+DYfqi2fNi/A9BxQIJNwPP2t4WFiw9k6GX6EsZkbAMUaC4J0niVQlGLH 2ydxVyWN3amcOY6MIE9lX5Xa9/eH1sYITq726jTlEBpbNU1341YheILcIRk13iSx0x1G/11fZU8x ggMQMIIDDAIBATB2MGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAo UHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQ a3GQsUdC7WDruGWZh/A+4jAJBgUrDgMCGgUAoIIBbzAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcB MBwGCSqGSIb3DQEJBTEPFw0wNzExMDExMjQ4NDNaMCMGCSqGSIb3DQEJBDEWBBQTDMMu0PfJz8F5 3geYNiejTlPiKzCBhQYJKwYBBAGCNxAEMXgwdjBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhh d3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVt YWlsIElzc3VpbmcgQ0ECEGtxkLFHQu1g67hlmYfwPuIwgYcGCyqGSIb3DQEJEAILMXigdjBiMQsw CQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UE AxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0ECEGtxkLFHQu1g67hlmYfwPuIw DQYJKoZIhvcNAQEBBQAEggEAkuCD6BiJXJEBwfJRC9P5Vc0xbdBl1Xlz23X8k+93cSjQZz3kh1GR YVySdKpj4LUd3wdxc3IbyBQAHHPIc7tCyt3PEvL53US8g/BGJWmhl7H9+BJ6/Czd5U0K1nvbMGF9 VtAAUPdKU/6mSQdVfRex5c4msM6QP3OzIqFgria8hsAnhhN+lK7grQ2f4NiiVM+ol9IMePhHnP8r kWolRkDo8VE4eHdg1YAc7pb/zj4qEQxzoCgicPHHylW79oIw4q2TC56KZUaBRFrZXDDVTJjGK9FN tfWs6kExRGFWqr6E1F2BevIkmT2W11iHKx+TzqOeIby002ruYKKfkzV4AKx4LAAAAAAAAA== --Apple-Mail-92-865758512-- --===============1951931746== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn --===============1951931746==-- From pcn-bounces@ietf.org Thu Nov 01 09:11:04 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1InZow-0005o1-9n; Thu, 01 Nov 2007 09:10:50 -0400 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1InZov-0005n0-28 for pcn-confirm+ok@megatron.ietf.org; Thu, 01 Nov 2007 09:10:49 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1InZou-0005mQ-N5 for pcn@ietf.org; Thu, 01 Nov 2007 09:10:48 -0400 Received: from smtp.nokia.com ([131.228.20.172] helo=mgw-ext13.nokia.com) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1InZot-0000vr-5A for pcn@ietf.org; Thu, 01 Nov 2007 09:10:48 -0400 Received: from esebh107.NOE.Nokia.com (esebh107.ntc.nokia.com [172.21.143.143]) by mgw-ext13.nokia.com (Switch-3.2.5/Switch-3.2.5) with ESMTP id lA1DAOZP022469; Thu, 1 Nov 2007 15:10:43 +0200 Received: from esebh104.NOE.Nokia.com ([172.21.143.34]) by esebh107.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.1830); Thu, 1 Nov 2007 15:10:26 +0200 Received: from esebh101.NOE.Nokia.com ([172.21.138.177]) by esebh104.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.1830); Thu, 1 Nov 2007 15:10:25 +0200 Received: from mgw-int01.ntc.nokia.com ([172.21.143.96]) by esebh101.NOE.Nokia.com over TLS secured channel with Microsoft SMTPSVC(6.0.3790.1830); Thu, 1 Nov 2007 15:10:24 +0200 Received: from [172.21.34.231] (esdhcp034231.research.nokia.com [172.21.34.231]) by mgw-int01.ntc.nokia.com (Switch-3.2.5/Switch-3.2.5) with ESMTP id lA1DAMvG027633; Thu, 1 Nov 2007 15:10:23 +0200 In-Reply-To: <000f01c81c84$5635b400$4c0d5982@dynamic.ewi.utwente.nl> References: <000f01c81c84$5635b400$4c0d5982@dynamic.ewi.utwente.nl> Mime-Version: 1.0 (Apple Message framework v752.3) Message-Id: <2723F111-5919-49CB-AC22-4A5A795F4852@nokia.com> From: Lars Eggert Subject: Re: [PCN] 5.5. Probing functions Date: Thu, 1 Nov 2007 15:10:21 +0200 To: ext Georgios Karagiannis X-Mailer: Apple Mail (2.752.3) X-OriginalArrivalTime: 01 Nov 2007 13:10:24.0557 (UTC) FILETIME=[90811DD0:01C81C88] X-Nokia-AV: Clean X-Spam-Score: 0.0 (/) X-Scan-Signature: 5d7a7e767f20255fce80fa0b77fb2433 Cc: 'pcn' X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0851146161==" Errors-To: pcn-bounces@ietf.org --===============0851146161== Content-Type: multipart/signed; micalg=sha1; boundary=Apple-Mail-93-867056519; protocol="application/pkcs7-signature" --Apple-Mail-93-867056519 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Hi, On 2007-11-1, at 14:39, ext Georgios Karagiannis wrote: >> (1) The ingress needs to generate traffic in a pattern and at >> a rate that lets it draw conclusions about whether it is OK >> to admit the flow that is waiting for admission. How does it >> know what characteristics the probe traffic should have and >> how does it generate the probe traffic? > > Georgios: The PCN_ingress_node generates one probe packet for each new > incoming flow that requests access from the PCN_domain. how is a single packet probing the path? It's not obvious to me how one would derive bottleneck load based on a single packet, i.e., based on one or zero markings. >> (2) How long do you need to probe for before you declare it >> safe to admit the new flow? > Georgios: One probe packet per new incoming and flow is enough > if the PCN_domain takes care that the probe packet will be marked > PCN_marked or using an other type of encoding, if it is passing > through a congested PCN_interior_node. Ah - so this requires the marking scheme to mark all packets once the defined pre-congestion load level is reached, instead of marking proportionally with an increase in load, correct? This eliminates some of the marking options that have been discussed. And you need to ensure that - after having gotten an indication of "not loaded" from the path - the single flow you're about to admit cannot by itself push the bottleneck straight into overload. This is, because the "not loaded" piece of information doesn't tell you how close to pre-congestion the bottleneck is, and if it can actually sustain the rate that the flow is going to send at. You could do this through appropriate configuration, i.e., ensuring that the level at which you start marking is sufficiently below the overload state that you can sustain one new flow. Or two. Or three. How much is enough? And if you did this through configuration, i.e., making sure that 1,2,3,... flows can be admitted, what's the benefit of probing? Why not simply admit the flow? >> What's the delay before a new >> flow can be admitted, and can apps actually deal with this delay? > > The delay = one RTT (within PCN_domain). > >> (3) Since you're sending probe traffic at a rate that is >> likely not insignificant, how do you prevent the probe >> traffic itself from causing congestion and triggering >> stop-admission or flow-termination actions? If you're >> treating it differently (e.g., at a lower priority), how is >> what the probe traffic experiences still representative of >> what the real flow would experience? (How can you treat it >> differently and still make ECMP work?) > > Georgios: By just sending one probe packet per new incoming flow that > requests access into the PCN_domain. Lars --Apple-Mail-93-867056519 Content-Transfer-Encoding: base64 Content-Type: application/pkcs7-signature; name=smime.p7s Content-Disposition: attachment; filename=smime.p7s MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIGQDCCAvkw ggJioAMCAQICEGtxkLFHQu1g67hlmYfwPuIwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UEBhMCWkEx JTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQ ZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA3MDEwNDE2MTE1NFoXDTA4MDEwNDE2MTE1 NFowXDEPMA0GA1UEBBMGRWdnZXJ0MQ0wCwYDVQQqEwRMYXJzMRQwEgYDVQQDEwtMYXJzIEVnZ2Vy dDEkMCIGCSqGSIb3DQEJARYVbGFycy5lZ2dlcnRAbm9raWEuY29tMIIBIjANBgkqhkiG9w0BAQEF AAOCAQ8AMIIBCgKCAQEA26hjntVPZMiwh4d8tyuk9KiucvG92BVUGArk5zO1jhmq7tLFgU1mqcIg VGumfQqjkAzIuh83kxIiqBrB05Wvxp85apwt4sUCvzMe8mQWzZZZYp0rBzwpOj+VC5pxsMRYtYW+ BaTFBEmvFr7D7C7roZUk0lDppS5SZFs4SQbEe9ykB9aeUf1iTiw2+ikyP2+JEYto14WYCoxFWbms FbQ1uaaK+yn1WH2YHhyMfi0IOxuT0jtbsngjHJMpWIqq334zBhj+rPSUug47YBHr41FCDMHbbbjv WbyTyDYneOW3WY8LY8bGWVlAknQGB7lgduShe6e0RI/7SL/VE6X27lM9LwIDAQABozIwMDAgBgNV HREEGTAXgRVsYXJzLmVnZ2VydEBub2tpYS5jb20wDAYDVR0TAQH/BAIwADANBgkqhkiG9w0BAQUF AAOBgQCx3nxxTg/WowobVTRXBiXUEEZj4LBiunWbuAnR0UbJIgijvq3JbiJvZo2gUGiW9LPaHSBz 4oxT1VR/S/6O0a4e87oV5cOQm6ReB68o9rDfUzxHTwoCfv2wwMwBrXyzQMjyQK4Lt8siV41gmZpm rSXMhjTJdd9uYYNoZKQLq+VM+TCCAz8wggKooAMCAQICAQ0wDQYJKoZIhvcNAQEFBQAwgdExCzAJ BgNVBAYTAlpBMRUwEwYDVQQIEwxXZXN0ZXJuIENhcGUxEjAQBgNVBAcTCUNhcGUgVG93bjEaMBgG A1UEChMRVGhhd3RlIENvbnN1bHRpbmcxKDAmBgNVBAsTH0NlcnRpZmljYXRpb24gU2VydmljZXMg RGl2aXNpb24xJDAiBgNVBAMTG1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBDQTErMCkGCSqGSIb3 DQEJARYccGVyc29uYWwtZnJlZW1haWxAdGhhd3RlLmNvbTAeFw0wMzA3MTcwMDAwMDBaFw0xMzA3 MTYyMzU5NTlaMGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5 KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQTCBnzAN BgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAxKY8VXNV+065yplaHmjAdQRwnd/p/6Me7L3N9VvyGna9 fww6YfK/Uc4B1OVQCjDXAmNaLIkVcI7dyfArhVqqP3FWy688Cwfn8R+RNiQqE88r1fOCdz0Dviv+ uxg+B79AgAJk16emu59l0cUqVIUPSAR/p7bRPGEEQB5kGXJgt/sCAwEAAaOBlDCBkTASBgNVHRMB Af8ECDAGAQH/AgEAMEMGA1UdHwQ8MDowOKA2oDSGMmh0dHA6Ly9jcmwudGhhd3RlLmNvbS9UaGF3 dGVQZXJzb25hbEZyZWVtYWlsQ0EuY3JsMAsGA1UdDwQEAwIBBjApBgNVHREEIjAgpB4wHDEaMBgG A1UEAxMRUHJpdmF0ZUxhYmVsMi0xMzgwDQYJKoZIhvcNAQEFBQADgYEASIzRUIPqCy7MDaNmrGcP f6+svsIXoUOWlJ1/TCG4+DYfqi2fNi/A9BxQIJNwPP2t4WFiw9k6GX6EsZkbAMUaC4J0niVQlGLH 2ydxVyWN3amcOY6MIE9lX5Xa9/eH1sYITq726jTlEBpbNU1341YheILcIRk13iSx0x1G/11fZU8x ggMQMIIDDAIBATB2MGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAo UHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQ a3GQsUdC7WDruGWZh/A+4jAJBgUrDgMCGgUAoIIBbzAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcB MBwGCSqGSIb3DQEJBTEPFw0wNzExMDExMzEwMjFaMCMGCSqGSIb3DQEJBDEWBBRQ1kUXYwabCJfX IsmmadT4/soV+jCBhQYJKwYBBAGCNxAEMXgwdjBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhh d3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVt YWlsIElzc3VpbmcgQ0ECEGtxkLFHQu1g67hlmYfwPuIwgYcGCyqGSIb3DQEJEAILMXigdjBiMQsw CQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UE AxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0ECEGtxkLFHQu1g67hlmYfwPuIw DQYJKoZIhvcNAQEBBQAEggEAEC142iyp+Mw/hjlQ1FHhBHk+GiSVgfxoCmMa60m1PRd7rHrgHJN0 mHIeCnjzYT/SI/zDYGqPhgSUhHgW0+SmUjQCOD804wDOdmWIgzlBpdnzsKgIRZlGlxsSURh7C0Vs 9/1OOOUuyGREQKVajY0krks5Eb2ICcdy8wqkhAQdsGZpH3/f+RhrFkfZHX6ds75Ey0Wg9nhL3eSZ S2d+q91OsXtB1f8mrA3UwD56uRcfdltdO7xq7gqyaoLaPPs5FZEhk6QteqJTMSIucuIyKdQx3ORP pxN3GWKDBEw6BfIIuihrxjFek9ZbkxDAS7KNG8cxwBz2cQO9yhZBdvSL7DV3iAAAAAAAAA== --Apple-Mail-93-867056519-- --===============0851146161== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn --===============0851146161==-- From pcn-bounces@ietf.org Thu Nov 01 09:53:06 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1InaTX-00060r-Kr; Thu, 01 Nov 2007 09:52:47 -0400 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1InaTX-00060l-0J for pcn-confirm+ok@megatron.ietf.org; Thu, 01 Nov 2007 09:52:47 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1InaTW-0005yL-Mf for pcn@ietf.org; Thu, 01 Nov 2007 09:52:46 -0400 Received: from imr2.ericy.com ([198.24.6.3]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1InaTL-0002Lw-Ap for pcn@ietf.org; Thu, 01 Nov 2007 09:52:41 -0400 Received: from eusrcmw750.eamcs.ericsson.se (eusrcmw750.exu.ericsson.se [138.85.77.50]) by imr2.ericy.com (8.13.1/8.13.1) with ESMTP id lA1Dq3b3018882; Thu, 1 Nov 2007 07:52:03 -0600 Received: from eusrcmw721.eamcs.ericsson.se ([138.85.77.21]) by eusrcmw750.eamcs.ericsson.se with Microsoft SMTPSVC(6.0.3790.1830); Thu, 1 Nov 2007 08:52:02 -0500 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [PCN] LC-PCN version 01 uploaded Date: Thu, 1 Nov 2007 08:52:02 -0500 Message-ID: In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] LC-PCN version 01 uploaded Thread-Index: AcgcVW8fHTe4VVURQqujuhANOp4G9wAOIbpg References: <75A199C5D243C741BF3D3F1EBCEF9BA5019DC606@E03MVZ1-UKDY.domain1.systemhost.net> From: "Anurag Bhargava" To: "Georgios Karagiannis" , , X-OriginalArrivalTime: 01 Nov 2007 13:52:02.0793 (UTC) FILETIME=[6191AD90:01C81C8E] X-Spam-Score: 0.0 (/) X-Scan-Signature: fac892abe0c719c7bb99f6e7c710cdae Cc: X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Phil, Anna and others, We are also in the process of re-editing the draft to make things more clear and less confusing. We will upload the next 02 version ASAP (and definitely by Nov 19, 2007). BR, -Anurag Bhargava, Ph.D. =20 >-----Original Message----- >From: Georgios Karagiannis [mailto:karagian@cs.utwente.nl]=20 >Sent: Thursday, November 01, 2007 3:04 AM >To: philip.eardley@bt.com; Anurag Bhargava; pcn@ietf.org >Subject: RE: [PCN] LC-PCN version 01 uploaded > >Hi Phil > >Thank you very much! >The answers to your questions included in this email are=20 >already given in the reply that I have just sent to Anna some=20 >minutes ago! ><> > >But please see in line! > > >On 10/26/2007, "philip.eardley@bt.com" wrote: > >>In-line >>Thanks >>Phil >>> >>> > Let me summarise to make sure I understand it right. >>> > >>> > - two encodings are used. one ('PCN-marking') is used for=20 >both adm=20 >>> > ctrl & termination to indicate the excess traffic. >>> > If a PCN-interior-node is PCN-marking some pkts, then it=20 >marks all=20 >>> > other pkts with another encoding ('affected marking') >>>=20 >>> Georgios: Yes, you are right. All other packets are marked=20 >using the=20 >>> "PCN_Affected_marking" encoding. >>>=20 >>> > >>> > - the 'PCN-marking' algorithm has several regimes: >>> > * when traffic rate is below PCN-lower-rate, no pkts are marked >>> > * as the traffic rate climbs above the PCN-lower-rate, an=20 >>> > increasing number of pkts are marked (marking rate =3D excess rate = >>> > divided by N) [adm ctrl state] >>>=20 >>> Georgios: It is more complicated than that! Please check the=20 >>> pseudocode on page 13. The packets are PCN-marking encoded only if=20 >>> the incoming_PCN_marking_rate is smaller or equal to the=20 >>> Admission_offset_rate. Where the incoming_PCN_marking_rate=20 >is actualy=20 >>> the rate of incoming packets that are PCN_marked. >>>=20 >>> > * at some rate (PCN-lower-rate + adm-offset-rate), then pkts are=20 >>> > marked at the same rate, even if the traffic rate climbs higher=20 >>> > [adm ctrl state] >>>=20 >>> Georgios: No, if the incoming_PCN_marking_rate is higher than the=20 >>> Admission_offset_rate than no anymore packets are PCN_marked. >>> They are however marked as PCN_Affected_marking! >>>=20 >>> > * when the traffic rate reaches the PCN-upper-rate,=20 >additional pkts=20 >>> > are marked (additional marking rate =3D excess rate above=20 >>> > PCN-upper-rate divided by N) [termination ctrl state] >>>=20 >>> Georgios: It is again more complicated than that, please see the=20 >>> pseudocode on Page 20. Similar to the admission control state the=20 >>> rate of packets to be marked also depends on the=20 >>> incoming_PCN_marking_rate. Moreover, if the interior node is in=20 >>> termination ctrl state then the additional marking rate is=20 >using the=20 >>> sliding window algorithm to overcome the undershooting issue, see=20 >>> page 19 and page >>20. >>>=20 >>>=20 >>> > * if pkts arrive already PCN-marked, then the node measures the=20 >>> > rate of pkts already marked & works out what excess rate this=20 >>> > corresponds to, and only marks additional pkts if its own excess=20 >>> > rate is even higher. >>>=20 >>> Georgios: Well, it is more complicated than that. Please see pseudo=20 >>> code onn page 13 and page 20, the rate of the already PCN_marked=20 >>> packets is denoted in the pseudocode as incoming_PCN_marking_rate. >>>=20 >>>=20 >>> > * the PCN-egress-node measures the rate of PCN-marked pkts and=20 >>> > determines whether the overloaded node is in adm ctrl state or=20 >>> > termination ctrl state >>>=20 >>> Georgios: Yes, but it is more complicated than that please=20 >see pseudo=20 >>> code on page 14 and on page 23. >>>=20 >> >>[phil] I'm now confused whether I understand your algo. >>I should have said: my first bullets were all assuming that no pkts=20 >>arrived already PCN-marked, ie incoming_PCN_marking_rate =3D 0. Is my=20 >>summary correct with this assumption? >> >>Also, the pcn-egres-node determining what state it's in. I=20 >was reading=20 >>the description on page 21-22. pges 14 & 23 don't seem to=20 >have relevant=20 >>pseudocode? > >Georgios: please see the abstract description of the LC-PCN=20 >algorithm that I gave on the reply that I have sent to Anna=20 >some minutes ago! > > >> >> >>> > >>> > Questions: >>> > - I don't understand the point of doing PCN-marking in=20 >the adm ctrl=20 >>> > state. When it comes to making an adm decision, you send=20 >a (single)=20 >>> > probe pkt - if it's PCN-marked or affected-marked then you block=20 >>> > the call. So when a node's in the adm ctrl state, it=20 >could just do=20 >>> > affected-marking of all pkts. Wouldn't that work just as well? >>>=20 >>> Georgios: this algorithm can provide admission control even if=20 >>> probing is not used. However, by using probing we ensure that the=20 >>> flow that is requesting access is passing through the interior node=20 >>> that is either in admission control state or flow termination state. >> >>[phil] ok. >>>=20 >>> > - with you current algo, imagine there are two paths=20 >through the nw: >>> > A-B-X-D >>> > A-M-N-X-D >>> > If B & N are both in adm ctrl state then they both will do=20 >>> > PCN-marking. >>> > At the PCN-egress-node the PCN-marking rate may be high=20 >enough that >>it >>> > believes termination is needed. Are there topologies/scenarios=20 >>> > where this could happen? I think your multicongestion_error=20 >>> > parameter in S3.4 is connected to this problem; I didn't=20 >find your=20 >>> > discussion about it convincing >>>=20 >>> Georgios: You are right, the multicongestion_error=20 >parameter in S3.4=20 >>> is used to emphasize the multicongestion error bounds in this=20 >>> calculation. Note that the calculation of this error bound can only=20 >>> be estimated by off line tests in a predefined network scenario. >>> Note that by using the dependency of the incoming_PCN_marking_rate=20 >>> the multicongestion error bound can be decreased. >> >>[phil] ok. I don't like this. the idea of PCN is to be a=20 >>measurement-based approach so don't have to do these kind of off-line=20 >>tests. In particular, there'll be wrong when the network is suffering=20 >>from failures - the strength of a measurement-based approach should=20 >>exactly be that it gracefully adapts despite network failures. > > >Georgios: please see the abstract description of the LC-PCN=20 >algorithm that I gave on the reply that I have sent to Anna=20 >some minutes ago! >In particular, please check the paragraph denoted as "Setting=20 >the thresholds at PCN_egress_nodes" in the reply that I have=20 >sent to Annas email some minutes ago! > >Furthermore, your above comment is not fair, since the example=20 >that you gave can be considered as a corner case! Only in such=20 >cases the multicongestion_error bound is needed. > >Now, regarding your example: >>> > A-B-X-D >>> > A-M-N-X-D >>> > If B & N are both in adm ctrl state then they both will do=20 >>> > PCN-marking. > >Are you assuming that ECMP occurs on the PCN domain and=20 >therefore flows leaving from A and going towards D will not=20 >follow the same path. >If no ECMP is used I would expect that all flows belonging to=20 >the same A/D aggregate, and leave from A and go towards D will=20 >follow either A-B-X-D or will follow A-M-N-X-D, but they will=20 >not be separated. > > > >>>=20 >>> > - the 'affected marking' is claimed to solve ECMP issues. However, >>the >>> > same affected marking is used in both adm ctrl state &=20 >termination=20 >>> > ctrl state - I don't think this works. Imagine that a=20 >node [node-1]=20 >>> > on >>one >>> > path is in adm ctrl state & a node [node-2] on another path is in=20 >>> > termination ctrl state. Therefore flows are terminated, and only >>flows >>> > that are being PCN-marked or affected-marked are terminated.=20 >>> > However this could lead to flows being terminated that go through=20 >>> > node-1; node-2 will still be just as badly overloaded. >>>=20 >>> Georgios: In admission control state, probing is used to=20 >ensure that=20 >>> the packets associated to a requesting flow are passing through the=20 >>> congested PCN_interior node. ECMP is one of the issues that are=20 >>> associated with the fact that packets of a flow >>can >>> follow a different path than the path followed by an aggregate of >>flows >>> (starting from the same ingress and ending at the same egress). >>> In order to accomplish this the probe packets should be=20 >marked using=20 >>> either PCN_marking encoding or the PCN_Affected_marking encoding. >>> In flow termination state, both the PCN_marking and >>PCN_Affected_marking >>> encoded packets are used to ensure that the flows that are selected >>for >>> termination are indeed passing through a severely congested node. >>> Note that the issue that you described above is associated to the=20 >>> multicongestion-error bounds. Therefore, this multicongestion-error >>bound >>> should be estimated as much as >>> accurate as possible. >> >>[phil] ok, so you agree this is a problem.=20 > >Georgios: The issue that I have described here is not a=20 >problem that cannot be solved. Please see the abstract=20 >description of the LC-PCN algorithm that I gave on the reply=20 >that I have sent to Anna some minutes ago! > >>>=20 >>> > - in the termination algorithm, S4.2.2, the >>"termination_offset_rate" >>> > factor makes no sense to me >>>=20 >>> Georgios: Why not? >> >>[phil] well I don't understand what the point of it is. I thought I=20 >>understood the point of the *admission* offset_rate and I=20 >couldn't see=20 >>why anything similar was needed for termination. But maybe I=20 >>misunderstood the adm-offset-rate as well! > >Georgios: This is explained in a detailed way in the reply=20 >that I have sent to Annas comments some minutes ago! > >>>=20 >>> > - the parameters PCN_lower_rate_egress &=20 >PCN_upper_rate_egress are=20 >>> > expressed in the wrong units (you have conditions like=20 >>> > signalled_overload_rate > PCN_lower_rate_egress; the former is a >>rate, >>> > the latter a % so some tweaking is needed to convert the latter=20 >>> > into >>a >>> > rate) >>>=20 >>> Georgios: You are right, a conversion is needed to convert=20 >the latter=20 >>> into a rate. >> >>[phil] ok. So everything is rate. Note this raises Anna's qu: >> * if pcn_lower_rate_eggress (and pcn_uppre-rate_egress)=20 >are absolute=20 >>rates, then how do you choose that abslolute rate value,=20 >given that the=20 >>rates of ingress-egress aggregates at the same ingress may be vastly=20 >>different in magnitude? > >Georgios: No, not everything is rate, please see my reply to=20 >Annas comments and in particular the paragraph denoted as: >"Setting the thresholds at PCN_egress_nodes" > >Best regards, >Georgios > >>>=20 >>>=20 >>> > >>> > best wishes >>> > >>> > phil/ >>> > >>> > > -----Original Message----- >>> > > From: Anurag Bhargava (RL/TNT) >>[mailto:anurag.bhargava@ericsson.com] >>> > > Sent: 11 September 2007 12:36 >>> > > To: pcn >>> > > Subject: [PCN] LC-PCN version 01 uploaded >>> > > >>> > > Hello, >>> > > FYI - We have uploaded a new version of the PCN draft.=20 >The major=20 >>> > > changes are some clarification and adopting to the >>> > Architecture draft >>> > > terminology. >>> > > >>> > > "LC-PCN: The Load Control PCN Solution", Lars Westberg,=20 >5-Sep-07,=20 >>> > > >>> > > >>> > > Please let me know if you have any questions. >>> > > >>> > > Thanks, >>> > > -Anurag Bhargava, Ph.D. >>> > > >>> > > >>> > > _______________________________________________ >>> > > PCN mailing list >>> > > PCN@ietf.org >>> > > https://www1.ietf.org/mailman/listinfo/pcn >>> > >>> > >>> > _______________________________________________ >>> > PCN mailing list >>> > PCN@ietf.org >>> > https://www1.ietf.org/mailman/listinfo/pcn > _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 01 11:13:44 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Inbjj-0000DP-1U; Thu, 01 Nov 2007 11:13:35 -0400 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Inbji-0000DK-En for pcn-confirm+ok@megatron.ietf.org; Thu, 01 Nov 2007 11:13:34 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Inbji-0000Cq-4R for pcn@ietf.org; Thu, 01 Nov 2007 11:13:34 -0400 Received: from rotterdam.ewi.utwente.nl ([130.89.10.5]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1Inbjg-00053g-QI for pcn@ietf.org; Thu, 01 Nov 2007 11:13:34 -0400 Received: from utip105 (utip105.ewi.utwente.nl [130.89.13.76]) by rotterdam.ewi.utwente.nl (8.13.6/8.13.6) with ESMTP id lA1FCwZY015936; Thu, 1 Nov 2007 16:13:29 +0100 (MET) From: "Georgios Karagiannis" To: "'Lars Eggert'" References: <000f01c81c84$5635b400$4c0d5982@dynamic.ewi.utwente.nl> <2723F111-5919-49CB-AC22-4A5A795F4852@nokia.com> Subject: RE: [PCN] 5.5. Probing functions Date: Thu, 1 Nov 2007 16:12:51 +0100 Message-ID: <001a01c81c99$b4735350$4c0d5982@dynamic.ewi.utwente.nl> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 11 In-Reply-To: <2723F111-5919-49CB-AC22-4A5A795F4852@nokia.com> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3198 Thread-Index: AcgcjOMH1cGJAyzwS+6qIkWWHgBz2QACPUtQ X-Spam-Score: 0 () X-Scanned-By: MIMEDefang 2.52 on 130.89.10.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0rc3 (rotterdam.ewi.utwente.nl [130.89.10.5]); Thu, 01 Nov 2007 16:13:30 +0100 (MET) X-Spam-Score: 0.0 (/) X-Scan-Signature: 87a3f533bb300b99e2a18357f3c1563d Cc: 'pcn' X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Lars Please see in line! Best regards, Georgios > -----Original Message----- > From: Lars Eggert [mailto:lars.eggert@nokia.com] > Sent: donderdag 1 november 2007 14:10 > To: ext Georgios Karagiannis > Cc: 'pcn' > Subject: Re: [PCN] 5.5. Probing functions > > Hi, > > On 2007-11-1, at 14:39, ext Georgios Karagiannis wrote: > >> (1) The ingress needs to generate traffic in a pattern and > at a rate > >> that lets it draw conclusions about whether it is OK to admit the > >> flow that is waiting for admission. How does it know what > >> characteristics the probe traffic should have and how does it > >> generate the probe traffic? > > > > Georgios: The PCN_ingress_node generates one probe packet > for each new > > incoming flow that requests access from the PCN_domain. > > how is a single packet probing the path? It's not obvious to > me how one would derive bottleneck load based on a single > packet, i.e., based on one or zero markings. Georgios: The PCN_ingress_node sends a probe packet and it waits for a response from the PCN_egress_node. If a PCN_interior_node is congested (is in admission control state), then the probe will be marked, either using the PCN_marking or an additional encoding, say PCN_Affected_marking. If the probe packet is dropped on the way, then the PCN_ingress_node has to know this such that the probe packet is retransmitted. This can be done by using retransmission timers at the PCN_ingress_node. > > >> (2) How long do you need to probe for before you declare > it safe to > >> admit the new flow? > > Georgios: One probe packet per new incoming and flow is > enough if the > > PCN_domain takes care that the probe packet will be marked > PCN_marked > > or using an other type of encoding, if it is passing through a > > congested PCN_interior_node. > > Ah - so this requires the marking scheme to mark all packets > once the defined pre-congestion load level is reached, > instead of marking proportionally with an increase in load, > correct? Georgios: No! Marking proportionally is still needed, since it is also needed to encode the flow termination state of the PCN_interior nodes. However, all packets that are passing through the congested node and are not PCN_marked they will have to be remarked with this additional encoding, denoted as PCN_Affected_marking. > This eliminates some of the marking options that > have been discussed. Georgios: No, please see above! > > And you need to ensure that - after having gotten an > indication of "not loaded" from the path - the single flow > you're about to admit cannot by itself push the bottleneck > straight into overload. This is, because the "not loaded" > piece of information doesn't tell you how close to > pre-congestion the bottleneck is, and if it can actually > sustain the rate that the flow is going to send at. Georgios: Yes, you are right, this is an issue that occurs in a similar way on all MBAC mechanisms. > > You could do this through appropriate configuration, i.e., > ensuring that the level at which you start marking is > sufficiently below the overload state that you can sustain > one new flow. Or two. Or three. > How much is enough? > > And if you did this through configuration, i.e., making sure > that 1,2,3,... flows can be admitted, what's the benefit of > probing? Why not simply admit the flow? Georgios: This preconfiguration might not be easy, but it is needed when any type of precongesion mechanism (that is not using probing) is used. When probing is used, then the PCN_egress_node is certain that when a received probe is marked that this probe (and thus all that packets that are associated with the flow that generated the proobe packet) is/will be passing through the congested node. In this situation the requesting flow is rejected. If the probe is not marked then the requesting flow is admitted. If probing is not used, then the PCN_egress_node cannot be certain about such decissions. Furthermore, when probing is used, the admission control mechanism also works when the ingress/egress aggregate is not yet available/created at the PCN_egress_node. This cannot be done if probing is not used. > > >> What's the delay before a new > >> flow can be admitted, and can apps actually deal with this delay? > > > > The delay = one RTT (within PCN_domain). > > > >> (3) Since you're sending probe traffic at a rate that is > likely not > >> insignificant, how do you prevent the probe traffic itself from > >> causing congestion and triggering stop-admission or > flow-termination > >> actions? If you're treating it differently (e.g., at a lower > >> priority), how is what the probe traffic experiences still > >> representative of what the real flow would experience? > (How can you > >> treat it differently and still make ECMP work?) > > > > Georgios: By just sending one probe packet per new incoming > flow that > > requests access into the PCN_domain. > > Lars > _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 01 11:20:11 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Inbps-0006Tw-OS; Thu, 01 Nov 2007 11:19:56 -0400 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Inbpr-0006QL-2g for pcn-confirm+ok@megatron.ietf.org; Thu, 01 Nov 2007 11:19:55 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Inbpq-0006QD-Ox for pcn@ietf.org; Thu, 01 Nov 2007 11:19:54 -0400 Received: from tcmail31.telekom.de ([217.6.95.238]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1Inbpk-0005Hj-E6 for pcn@ietf.org; Thu, 01 Nov 2007 11:19:54 -0400 Received: from S4DE8PSAANQ.mitte.t-com.de by tcmail31.telekom.de with ESMTP; Thu, 1 Nov 2007 16:19:32 +0100 Received: from S4DE8PSAANK.mitte.t-com.de ([10.151.229.10]) by S4DE8PSAANQ.mitte.t-com.de with Microsoft SMTPSVC(6.0.3790.3959); Thu, 1 Nov 2007 16:19:32 +0100 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: quoted-printable Subject: RE: [PCN] traffic matrix scenario Date: Thu, 1 Nov 2007 16:19:31 +0100 Message-Id: <1B6169C658325341A3B8066E23919E1C4C13D7@S4DE8PSAANK.mitte.t-com.de> In-Reply-To: <66C55C26FA491C42A9C9BB62A376DAFF01631DF6@E03MVB1-UKBR.domain1.systemhost.net> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] traffic matrix scenario Thread-Index: AcgceHifgxZOLA2FQsaMBLg6JPelmAACDV9AAAW4U9A= From: "Geib, Ruediger" To: X-OriginalArrivalTime: 01 Nov 2007 15:19:32.0008 (UTC) FILETIME=[9A585280:01C81C9A] X-Spam-Score: -1.0 (-) X-Scan-Signature: cd3fc8e909678b38737fc606dec187f0 Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Ben, I appreciate your work on scenarios and safety margins for PCN=20 solutions working without probing. I further share your views=20 on ECMP.=20 Lars, having some text in a document describing different=20 scenarios where probing isn't required, where probing could=20 make sense and when reservations are useful could make=20 sense. I assume, investigations like those of Ben are=20 sufficient, i.e it is not expected that the standards must=20 define quantified operational and planning conditions under=20 which probing free PCN works? If we want to keep the WGs=20 Milestone schedule, a qualitative scenario description as=20 part of standards should do. Regards, Rudiger |-----Original Message----- |From: ben.strulo@bt.com [mailto:ben.strulo@bt.com] |Sent: Thursday, November 01, 2007 1:25 PM |To: lars.eggert@nokia.com |Cc: pcn@ietf.org |Subject: RE: [PCN] traffic matrix scenario | | |Hi, | |Interestingly, I mostly agree with you. | |I am not particularly convinced that probing is a good route=20 |to go down, |and I suspect that PCN will have a great deal of difficulty in working |well with ECMP. | |On the other hand I think that there can be reasonable scenarios where |PCN can be engineered to make flow termination (caused by |over-admission) extremely unlikely without completely changing the |architecture. I have in mind scenarios where the safety margin is |large, and the admission process is carefully managed (and aggregation |levels are large). The question that most interests me then=20 |is how well |does PCN perform? How large does the safety margin need to be? How |tightly capped does admission control need to be? | |I do not have complete answers, but the work we have done so far leads |me to believe that there are credible scenarios in which (some version |of) PCN is an appropriate and reasonably efficient solution. | |My chief concern then is to ensure that the PCN WG chooses options that |work as well as possible in as many scenarios as possible. | |Ben | | |> -----Original Message----- |> From: Lars Eggert [mailto:lars.eggert@nokia.com]=20 |> Sent: 01 November 2007 11:15 |> To: Strulo,B,Ben,CXR9 R |> Cc: pcn@ietf.org |> Subject: Re: [PCN] traffic matrix scenario |>=20 |> Hi, |>=20 |> On 2007-10-31, at 15:51, ext ben.strulo@bt.com wrote: |> > Broadly speaking I think our objective is that flow=20 |> termination should=20 |> > only be necessary in the case of an unexpected decrease in core=20 |> > capacity. We generally do not expect an Admission Control=20 |> system to=20 |> > admit calls and then later terminate them in the absence of=20 |> internal=20 |> > network problems such as link failure. |>=20 |> I think this (how "OK" flow termination is) is the root cause=20 |> of our argument about PCN. |>=20 |> I see PCN as a "best-effort" sort of QoS. A PCN has a load=20 |> range in which it operates well. You increase load by=20 |> admitting flows, you get signals about the load from the=20 |> markings, and then you react to it. =20 |> If you over-admitted a bit, you stop admission until load=20 |> abates. If you over-admitted a lot, you need to terminate=20 |> flows to get the domain back into a stable state. Flow=20 |> admission is how you ramp up the load, stopping admission is=20 |> the "light" mechanism to reduce load over time, flow=20 |> termination is the "heavy" mechanism to reduce load quickly.=20 |> You can be optimistic about admitting flows, because you can=20 |> always recover from wrong admission decisions. |>=20 |> If flow termination is to be avoided at all costs, the=20 |> properties of the architecture fundamentally change. All of a=20 |> sudden, you need to be a *lot* more careful about when and=20 |> what you admit, because all you have left to react to=20 |> overload is the "light" stop-admission mechanism. You must=20 |> not over-admit, so other mechanisms are needed to make sure=20 |> you don't over-admit, i.e., probing. |>=20 |> > My scenario mentioned the failure of an exchange really only as an=20 |> > example of the sort of extreme scenarios we consider. The real=20 |> > examples I had in mind were simply flash crowds: widespread and=20 |> > unexpected increases in call request rates with an unusual (e.g.=20 |> > regionally |> > focussed) traffic matrix. |> > |> > The sort of scenario that might be particularly testing for this=20 |> > particular probing issue, would be an initial anomalous traffic=20 |> > pattern consisting of very heavy traffic on a few=20 |> aggregates causing=20 |> > pre-congestion on just a few links, followed by a subsequent=20 |> > widespread increase in demand which also focuses on those=20 |> links. It=20 |> > is easy to construct reasonable (though not necessarily probable)=20 |> > sequences of external events that could cause this sort of traffic=20 |> > pattern: for example, news reporting initially being local and=20 |> > progressing to national coverage. |>=20 |> I believe that it will be very difficult to design a simple=20 |> architecture that can handle such cases with just=20 |> stop-admission as a permitted reaction mechanism. |>=20 |> How likely are these scenarios? |>=20 |> > We would expect an Admission Control system to do a good job of=20 |> > rejecting requests in this scenario. Though this is not=20 |completely=20 |> > cut and dried: it's possible a very small amount of flow=20 |> termination=20 |> > might be acceptable. |> > |> >> If we aren't in agreement, then I wonder under what circumstances=20 |> >> you'd consider flow termination to be appropriate? |> > |> > As I say, only really when there is a sudden and=20 |> significant decrease=20 |> > in core capacity. Even then, we would normally expect to be=20 |> > provisioned to deal with all but the most serious failures. |>=20 |> After reading through your scenario, I believe that PCN (as I=20 |> understand it) is the wrong tool for the job. You really want=20 |> firm guarantees that the domain can sustain a flow before=20 |> admitting it, and you never want to terminate flows unless a=20 |> catastrophic event occurs. In other words, you want=20 |> path-signaled QoS reservations, maybe using RSVP or NSIS. We=20 |> have protocols and an architecture for path-coupled QoS, and=20 |> I see no need and no benefit in PCN attempting to solve the=20 |> same problem in a different way. |>=20 |> (And if there is no other problem for PCN to solve, maybe=20 |> we're done...) |>=20 |> Lars | | |_______________________________________________ |PCN mailing list |PCN@ietf.org |https://www1.ietf.org/mailman/listinfo/pcn | _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 01 12:00:56 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IncTE-0001Hs-UW; Thu, 01 Nov 2007 12:00:37 -0400 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IncTD-0001GZ-71 for pcn-confirm+ok@megatron.ietf.org; Thu, 01 Nov 2007 12:00:35 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IncTC-0001Eb-ME for pcn@ietf.org; Thu, 01 Nov 2007 12:00:34 -0400 Received: from sj-iport-5.cisco.com ([171.68.10.87]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IncTA-0005M6-9Q for pcn@ietf.org; Thu, 01 Nov 2007 12:00:34 -0400 X-IronPort-AV: E=Sophos;i="4.21,359,1188802800"; d="scan'208";a="184211742" Received: from sj-dkim-4.cisco.com ([171.71.179.196]) by sj-iport-5.cisco.com with ESMTP; 01 Nov 2007 09:00:31 -0700 Received: from sj-core-1.cisco.com (sj-core-1.cisco.com [171.71.177.237]) by sj-dkim-4.cisco.com (8.12.11/8.12.11) with ESMTP id lA1G0Vfh031065; Thu, 1 Nov 2007 09:00:31 -0700 Received: from xbh-rtp-211.amer.cisco.com (xbh-rtp-211.cisco.com [64.102.31.102]) by sj-core-1.cisco.com (8.12.10/8.12.6) with ESMTP id lA1G0PY3006836; Thu, 1 Nov 2007 16:00:31 GMT Received: from xmb-rtp-203.amer.cisco.com ([64.102.31.20]) by xbh-rtp-211.amer.cisco.com with Microsoft SMTPSVC(6.0.3790.1830); Thu, 1 Nov 2007 12:00:27 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Thu, 1 Nov 2007 12:00:28 -0400 Message-ID: In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Questions on LC-PCN draft version 01 Thread-Index: AcgcUUwkumZGNuM5Tiqy93VpjbujkgAQb1Dg From: "Anna Charny (acharny)" To: "Georgios Karagiannis" , , "Lars Westberg (KI/EAB)" X-OriginalArrivalTime: 01 Nov 2007 16:00:27.0755 (UTC) FILETIME=[521593B0:01C81CA0] X-TM-AS-Product-Ver: SMEX-8.0.0.1181-5.000.1023-15518.002 X-TM-AS-Result: No--38.262300-8.000000-2 X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No DKIM-Signature: v=0.5; a=rsa-sha256; q=dns/txt; l=21859; t=1193932831; x=1194796831; c=relaxed/simple; s=sjdkim4002; h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version; d=cisco.com; i=acharny@cisco.com; z=From:=20=22Anna=20Charny=20(acharny)=22=20 |Subject:=20RE=3A=20Questions=20on=20=20LC-PCN=20draft=20version=2001 |Sender:=20; bh=cXTWgJr5L8JUApS/O98H3wZn2DZHQpGeO02lnwfpHMQ=; b=pi9JKErFd2F7V+sz40yA4wW3osTNA7aTIHgRDlfy2cdCBfD8mXRNN1aycgwXFMS+3SbW6ent 2K8ffljM3uWK7amb05+gpueoLEw88bCXGKZSs2AdnfrSRfpK1CLeKMAd; Authentication-Results: sj-dkim-4; header.From=acharny@cisco.com; dkim=pass ( sig from cisco.com/sjdkim4002 verified; ); X-Spam-Score: 0.0 (/) X-Scan-Signature: f251f249fc9a04067ce354aa0943ab98 Cc: pcn@ietf.org Subject: [PCN] RE: Questions on LC-PCN draft version 01 X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org =20 Hi Georgios, A few more clarification questions: > -----Original Message----- > From: Georgios Karagiannis [mailto:karagian@cs.utwente.nl]=20 > Sent: Thursday, November 01, 2007 2:35 AM > To: Anna Charny (acharny); anurag.bhargava@ericsson.com; Lars=20 > Westberg (KI/EAB) > Cc: pcn@ietf.org > Subject: Re: Questions on LC-PCN draft version 01 >=20 > Hi Anna >=20 > Thank you very much for your comments! > Before answering to your comments, see in line below, I would=20 > like to explain in an abstract way the LC-PCN algorithm. >=20 > * Setting the thresholds at PCN_interior_nodes: > ----------------------------------------------- > In order to calculate the PCN_upper_rate we use two parameters: > Maximum PHB capacity: that is the maximum capacity that can=20 > be supported by a PCN_interior_node > Termination_offset_rate: that is an absolute rate value that=20 > should be set equal into all PCN_interior_nodes. Note that=20 > this value is used by PCN_interior_nodes to calculate their=20 > PCN_upper_rate and also during the situation that a=20 > PCN_interior_node is in flow termination state and it=20 > receives PCN_marked packets. Please see pseudo code on page 20. > This value must be set equal into all PCN_interior_nodes such=20 > that all these nodes will know when to take into account the=20 > incoming PCN_marked packets and when not. >=20 > The PCN_upper_rate is then found as: > PCN_upper_rate =3D "Maximum PHB capacity" - Termination_offset_rate >=20 So if you have a link of 10 Mbps and a link of 40 Mbps (say each is allowed to use all bandwidth for PCN, for simplicity), then they both use the same *absolote* value of the termination-offset-rate?. So If I wanted to have PCN-upper-rate 20% below my link capacity on all links, I would not be able to do so with this approach? A larger link would have a smaller relative safety margin, then... Same comment for the difference between admission and termination - the relative difference between admission and termination threshods (Pcn-lower-rate and pcn-upper-rate) is global, so the larger links have a smaller relative difference between admission and termination thresholds. Right? > The PCN_lower_rate is configured in all PCN_interior-nodes=20 > and it can be calculated in the following way: > PCN_lower_rate =3D PCN_upper_rate - Admission_offset_rate >=20 > The Admission_offset_rate is an absolute rate value and it is=20 > equal in all PCN_interior_nodes and PCN_egress_nodes. Note=20 > that this value is used by PCN_interior_nodes to calculate=20 > their PCN_lower_rate and the PCN_egress_nodes to calculate=20 > their PCN_upper_rate_egress. Furthermore, this value is used=20 > by the PCN_interior_nodes also during the situation that a=20 > PCN_interior_node is in admission control state and it=20 > receives PCN_marked packets. Please see pseudo code on page 13. > This value must be set equal into all PCN_interior_nodes such=20 > that all these nodes will know when to take into account the=20 > incoming PCN_marked packets and when not. Note that a=20 > PCN_interior_node can PCN_mark packets up to an excess rate=20 > equal to the Admission_offset_rate. If the exess rate in an=20 > PCN_interior_node is higher than the Admission_offset_rate,=20 > then the PCN_interior_node changes state from admission=20 > control state to flow termination state. OK. >=20 > * Setting the thresholds at PCN_egress_nodes: > --------------------------------------------- > The question is how to calculate the threshold that defines=20 > when a PCN_egress_node goes into the admission control state. > One way to do that is to consider that when the=20 > PCN_egress_node receives a PCN_marked packet it will mean=20 > that at least one PCN_interior_node started to be admission=20 > control congested and therefore it will go from Normal state=20 > to admission control state. Of course this will somehow might=20 > provide some errors, because there might be situations that=20 > this consideration might be to conservative. Therefore, we=20 > use a percentage of received PCN_marking encoded packets in=20 > proportion to total rate of received packets. In this version=20 > of the draft we call this value as: > PCN_lower_rate_egress. This is wrong. > What we should say is: > PCN_lower_rate_egress: is equal to incoming_PCN_marking_rate, when: > (incoming_PCN_marking_rate/measured PHB rate) =3D to a=20 > preconfigured percentage, say 1%. From now on I denote this=20 > percentage as: > PCN_lower_percentage_egress. In the draft, pcn_lower- and -upper_rate_egress seem to be defined on a per *ingress-egress-pair basis*. Is that correct?=20 > Thus we can say that a PCN_egress_node changes from Normal=20 > state to admission control state when=20 > incoming_PCN_marking_rate/measured PHB rate >=20 > PCN_lower_percentage_egress. > Where, > incoming_PCN_marking_rate =3D N * input_PCN_marking_bytes/T=20 > Where, input_PCN_marking_bytes =3D received number of=20 > "PCN_marking" encoded packets during measurement period T. Again, is it per ingress-egress pair, or not? >=20 > Now we defined the condition that the PCN_egress_node changes=20 > state from Normal state to admission control state. But how=20 > will the PCN_egress_node change state from admission control=20 > state to flow termination state. > In order to explain this, it is imporatnt to note that each=20 > PCN_interior_node that is in admission control state it can=20 > PCN_mark packets up to a value equal to=20 > Admission_offset_rate. Furthermore, if a PCN_interior_node=20 > receives incoming PCN_marked packets and is in the addmission=20 > control state, it will not remark any packets if the excess=20 > rate is equal or lower than the incoming_PCN_marking_rate,=20 > see page 13. > Furthermore, if we will consider as normal situations the=20 > situations that no ECMP occurs and that all flows belonging=20 > to the same ingress-egress aggregate will use the same path=20 > from PCN_ingress to PCN_egress, this will mean that when the=20 > PCN_egress_node receives, for the given ingress-egress=20 > aggregate an excess rate equal to Admission_offset_rate it=20 > will have to change from admission control state to flow=20 > termination state. > Thus in this case the second threshold, that in this case is=20 > a rate and not a percentage, can be calculated as follows: > PCN_upper_egress_rate =3D PCN_lower_egress_rate + = Admission_offset_rate. Here is where I am afraid I am fundamentally confused. Pcn_lower-egress_rate seems to have beed redefined above as a percentage threshold, but it is=20 an absolute rate again here. Perhaps you just mean that Pcn-lower-egress-rate =3D=20 pcn_lower_egress_percentage * total-measured-rate-of-this-ingress-egrees-aggregate/100? So pcn-lower-egress-rate then is just the fraction of the total rate of the ingress-egress aggregate corresponding to the configured percentage threshold. Right? Assuming the above is correct, I cannot see how the system will work correctly. Consider the following example. A bottleneck link of capacity 100 mbps , pcn-termination-offset of 40mbps and pcn-admission-offset of 30 mbps is shared by 100 ingress-egress aggregates, each with the total pcn rate of 1 mbps. On this link pcn-upper-threshold is 60mbps, and pcn-lower-threshold is 30mbps. Suppose further that all 100 ingress-egress aggregates go to different egress nodes (and suppose for simplicity there is no other traffic going to these egresses). The bottleneck link is clearly in the termination state, as the total rate of pcn traffic on this link is 100 mbps which is above the pcn-upper-threshold. This means we want the egress nodes to somehow recognise this state and get into the termination mode. But it seems in this example the egresses can't ever recognize that the system is in the termination mode. See below. Suppose pcn-lower-egress-percentage is set to w% (0=3D 30 mbps (for all possible settings of w). So our pcn-upper-egress rate is always greater than 30 Mbps, while the entire rate of pcn traffic the egress will ever see is *at most* 1 Mbps (because it is plainly the total rate of the single ingress-egress aggreate that goes to this egress). So, regardless of the actual amount of marked traffic in this egress-egress aggregate, the absolute excess rate of this ingress-egress aggregate will never be above 1mbps either. In turn that means that the pcn-upper-egress-rate will NEVER be exceeded (because pcn-upper-egress-rate is so much larger than the total rate of the pcn traffic at the egress). In turn, that appears to mean that in this scenario none of the egresses will ever end up in termination state, and so termination will never occur. That does not seem right.=20 I would appreciate any help in clarifying this.=20 Anna > However, there are some corner cases, that mainly occur when=20 > the different congestion points (admission control congested > PCN_interior_nodes) on the same path are not simulataneously=20 > starting to be congested. Therefore we use the=20 > multicongestion_error parameter to identify the error bound=20 > that ocurs due to these corner cases. Note that this error=20 > bound can be e.g., predefined ones off line by the operator,=20 > by studying the network topology and/or studying how often=20 > such corner cases could occur and/or doing off line=20 > measurements. Therefore we use: > PCN_upper_rate_egress =3D PCN_lower_rate_egress +=20 > Admission_offset_rate +/- multicongestion_error >=20 > * How the states of operation in PCN_interior_nodes are being changed? > --------------------------------------------------------------------- > This is explained on page 17, 18, by using Figure 4. >=20 > Change from Normal state to Admission control state: event A=20 > Occurs when: > Measured PHB rate > PCN_lower_rate >=20 > Change from Admission control state to Flow Termination =20 > state: event B Occurs when: > Measured PHB rate > PCN_upper_rate >=20 > * How the states of operation in PCN_egress_nodes are being changed? > --------------------------------------------------------------------- > This is explained on page 21, 22. Note that the description=20 > of event A on page 21 has to be modified to avoid the=20 > confusions that were caused up to now, see explanation given above. >=20 > Change from Normal state to Admission control state: event A=20 > Occurs when: > incoming_PCN_marking_rate/measured PHB rate) >=20 > PCN_lower_percentage_egress As explained above: > IF ((incoming_PCN_marking_rate/measured PHB rate) =3D > PCN_lower_percentage_egress) > THEN PCN_lower_egress_rate =3D incoming_PCN_marking_rate >=20 > Change from Admission control state to flow termination=20 > state: event B Occurs when: > incoming_PCN_marking_rate > PCN_upper_rate_egress >=20 > It is important to note that also the explanation of event C=20 > on page 21, has to be modified to avoid the confusions that=20 > were caused up to now, see explanation given above: > Change from Admission control state to Normal state: > Occurs when: > incoming_PCN_marking_rate/measured PHB rate) =3D<=20 > PCN_lower_percentage_egress >=20 >=20 > * Generated excess rate by an PCN_interior_node operating in=20 > admission control state. > ------------------------------------------------------------- >=20 > The excess rate =3D signaled_overload_rate. > The maximum excess rate that a PCN_interior_node can=20 > calculate in admission control state is equal to=20 > Admission_offset_rate, see explanation above. > The number of bytes that are remarked,=20 > signaled_remarked_bytes depend on the value of calculated=20 > excess rate (signaled_overload_rate), the value of the=20 > Admission_offset_rate and the value of the=20 > incoming_PCN_marking_rate, see pseudo code on page 13. >=20 > Note that all packets that are passing through a congested=20 > PCN_interior_node an are not being PCN_marked by the=20 > PCN_interior_node have to be remarked using the PCN_Affected_marking. >=20 >=20 > Regarding probes, the probe packets that are passing through=20 > a congested node are either PCN_marked or PCN_Affected_marked. >=20 > * Generating excess rate by an PCN_interior_node operating in=20 > flow termination state: > ---------------------------------------------------------------- > The excess rate =3D signaled_overload_rate. > Note that the calculation of the signaled_overload_rate is=20 > different than in the situation that the PCN_Interior_node=20 > operates in admission control state, see page 19 and 20. This=20 > is due to the fact that a sliding window is used to solve an=20 > undershooting problem, see discussion on page 19. > The number of bytes that are remarked,=20 > signaled_remarked_bytes depend on the value of calculated=20 > excess rate (signaled_overload_rate), the value of the=20 > Termination_offset_rate and the value of the=20 > incoming_PCN_marking_rate, and the see pseudo code on page 20. >=20 > Note that all packets that are passing through a congested=20 > PCN_interior_node an are not being PCN_marked by the=20 > PCN_interior_node have to be remarked using the PCN_Affected_marking. >=20 > * Providing admission control at PCN_egress_nodes: > -------------------------------------------------- > When the PCN_egress_node is operating in admission control=20 > state than a flow that is requesting admission into the PCN=20 > domain can b etreated in the following way: >=20 > If no probing is used, the request for admission can be=20 > accomplished by using an external to PCN signaling protocol.=20 > In this case when the request arrives at a PCN_egress_node=20 > that operates in admission control state then the request is=20 > rejected. If it operates in Normal state is accepted. >=20 > If probing is used, the request for admission is accomplished=20 > by using probe packets. In this case when the probe arrives=20 > at a PCN_egress_node and it is either PCN_marking or=20 > PCN_Affected_marking encoded is rejected. Otherwise is accepted. > Note that probes can only be used when PCN_Affected_marking=20 > is appled in whole PCN domain. Otherwise, the admission=20 > control procedure will work by using e.g., an external=20 > signaling protocol used between PCN_ingress_nodes and=20 > PCN_egress_nodes. >=20 > * Providing flow termination at PCN_egress_nodes: > -------------------------------------------------- >=20 > When a PCN_egress_node operates on flow termination it=20 > calculates the incoming excess rate (incoming_PCN_marking_rate). > By using the excess rate the PCN_egress_node calclates the=20 > numer of flows that have to be terminated using the pseudo=20 > code given on page 23. Note that the PCN_egress_node has to=20 > maintain per flow reservation states. > The information contained in the per flow states is used for=20 > the calculation of the flow that have to be terminated, see=20 > pseudocode on page 23. Note also that this pseudo code uses=20 > priority clases, but it operates when also no priority clases=20 > are used by setting Maximum_priority =3D 0 Where, 0 =3D<=20 > priority_class =3D< Maximum_priority >=20 >=20 >=20 > Below, I am providing some answers to your comments! >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > > -----Original Message----- > > From: Anna Charny (acharny) [mailto:acharny@cisco.com] > > Sent: zondag 7 oktober 2007 21:40 > > To: anurag.bhargava@ericsson.com; Georgios Karagiannis;=20 > Lars Westberg > > (KI/EAB) > > Cc: pcn@ietf.org > > Subject: Questions on LC-PCN draft version 01 > > > > Dear authors of the LC-PCN draft, > > > > Am I right that there has been no response yet to Phil's questions=20 > > (attached at the end) on the new version of your draft below (I=20 > > checked the pcn list and it does not seem it went there)? If the=20 > > response was sent, can you resend it please? > > > > I have many of the same questions, and also a few additional ones. > > Here are some of them. > > > > 1) I do not understand whether pcn-lower_rate_egress and=20 > > pcn_upper_rate ingress are expressed as a fraction of the=20 > total rate=20 > > (a unitless entity, analogous to CLE) or an absolute value=20 > (in bits or=20 > > bytes per second). The text seems to be contradictory on this point: > > > > on page 8 it is a fraction: > > > > "PCN_lower_rate_egress =3D predefined percentage of received=20 > > PCN_marking", > > > > while on page 13 it is an absolute rate: > > > > "If the incoming_PCN_marking_rate is higher than a preconfigured=20 > > PCN_lower_rate_egress, ..." where the > > > > (Incoming_PCN_marking_rate is clearly defined as abslolue=20 > rate on page > > 12: "Where the "incoming_PCN_marking_rate" is calculated as follows: > > incoming_PCN_marking_rate =3D (received number of "PCN_marking" > > DSCP during T)* N)/T;) >=20 > Georgios: You are right, please see discussion above on=20 > paragaph denoted > as: > "Setting the thresholds at PCN_egress_nodes". >=20 >=20 > We use a percentage of received PCN_marking encoded packets=20 > in proportion to total rate of received packets to define=20 > when a PCN_egres_node goes from Normal state to admission=20 > control state. In this version of the draft we call this=20 > value as: PCN_lower_rate_egress. This is wrong. > What we should say is: > PCN_lower_rate_egress: is equal to incoming_PCN_marking_rate, when: > (incoming_PCN_marking_rate/measured PHB rate) =3D to a=20 > preconfigured percentage, say 1%. From now on I denote this=20 > percentage as: > PCN_lower_percentage_egress. > Thus we can say that a PCN_egress_node changes from Normal=20 > state to admission control state when=20 > incoming_PCN_marking_rate/measured PHB rate >=20 > PCN_lower_percentage_egress. > Where, > incoming_PCN_marking_rate =3D N * input_PCN_marking_bytes/T=20 > Where, input_PCN_marking_bytes =3D received number of=20 > "PCN_marking" encoded packets during measurement period T. >=20 >=20 > > > > 2) Regarding the question above, > > > > * if pcn_lower_rate_eggress (and pcn_uppre-rate_egress) are=20 > absolute=20 > > rates, then how do you choose that abslolute rate value, given that=20 > > the rates of ingress-egress aggregates at the same ingress may be=20 > > vastly different in magnitude? >=20 > Georgios: See explanation above! Hopefully this=20 > misundersatnding is also solved! >=20 > > > > *if, however, pcn_lower_rate_eggress (and > > pcn_uppre-rate_egress) are ratios, then what is the guidance of=20 > > setting these ratios so that the egress can always tell admission=20 > > state from the termination state, given that the same=20 > marking is used=20 > > for admission and termination marking? > > Specifically, suppose at the bottleneck pcn_lower_threshold=20 > is set to=20 > > X, and pcn_upper_threshold to aX with some a>1 (assume for example=20 > > a=3D1.5). > > How does the egress node tell between the conditions when the total=20 > > pcn traffic on the bottleneck is 1.1X (which is the state wnen=20 > > admission is needed but termination is not, - in this case > > (1.1X-X)/X=3D0.1 traffic is pcn-marked), and the case when the total = > > traffic on the bottleneck is 1.1aX, which is the case when=20 > termination=20 > > is needed, and (1.1ax-ax)/ax=3D0.1), if in both cases the=20 > ratio between=20 > > marked and total traffic is the same value 0.1? >=20 > Georgios: In order to solve the above described problems, the=20 > thresholds used in the PCN_interior_nodes and PCN_egress_nodes are > using: > the Admission_offset_rate that is an absolute rate value=20 > which is set equal into whole PCN domain. >=20 > Note that the maximum excess rate that a PCN_interior_node=20 > can calculate in admission control state is equal to=20 > Admission_offset_rate. If the excess rate is higher than the=20 > Admission_offset_rate then the node changes from admission=20 > control state to flow termination state. >=20 > Please see all details that I have provided at the top of=20 > this email. In particular, check the paragraphs denoted above as: > "Setting the thresholds at PCN_interior_nodes" > "Setting the thresholds at PCN_egress_nodes" > "Generated excess rate by an PCN_interior_node operating in=20 > admission control state" >=20 >=20 >=20 > > > > 3) On page 9, multicongestion error error is introduced,=20 > but the text=20 > > is silent on how this error is defined and what is it set=20 > to (if it is=20 > > a configuration parameter), or how it is measured (if it is=20 > something=20 > > that is being measured). Can you explain, please? >=20 > Georgios: > Please see the paragraph denoted above as "Setting the=20 > thresholds at PCN_egress_nodes" >=20 >=20 > Best regards, > Georgios >=20 >=20 > > > > I have some more questions, including sharing those asked=20 > by Phil in=20 > > the attached message), but I will stop now, as I hope=20 > clarifications=20 > > on the above (and Phil's) questions will help understand the draft=20 > > better... > > > > Thank you in advance for clarifying all this. > > > > Anna >=20 _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Fri Nov 02 11:58:09 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Inyu4-0002mq-5j; Fri, 02 Nov 2007 11:57:48 -0400 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Inyu2-0002m2-PD for pcn-confirm+ok@megatron.ietf.org; Fri, 02 Nov 2007 11:57:46 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Inyu2-0002ib-CC for pcn@ietf.org; Fri, 02 Nov 2007 11:57:46 -0400 Received: from smtp1.smtp.bt.com ([217.32.164.137]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1Inytw-0003X0-K0 for pcn@ietf.org; Fri, 02 Nov 2007 11:57:41 -0400 Received: from E03MVZ1-UKDY.domain1.systemhost.net ([193.113.30.61]) by smtp1.smtp.bt.com with Microsoft SMTPSVC(6.0.3790.1830); Fri, 2 Nov 2007 15:57:39 +0000 x-mimeole: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Date: Fri, 2 Nov 2007 15:57:39 -0000 Message-ID: <75A199C5D243C741BF3D3F1EBCEF9BA503B342C7@E03MVZ1-UKDY.domain1.systemhost.net> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Comments /questions on LC-PCN draft Thread-Index: AcgdaRfcltW5tnJLSLyJtQdPEU+OVw== From: To: X-OriginalArrivalTime: 02 Nov 2007 15:57:39.0287 (UTC) FILETIME=[18152670:01C81D69] X-Spam-Score: 0.0 (/) X-Scan-Signature: b38aee91eedbacb27d28d558bc16c035 Subject: [PCN] Comments /questions on LC-PCN draft X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1695423055==" Errors-To: pcn-bounces@ietf.org This is a multi-part message in MIME format. --===============1695423055== Content-class: urn:content-classes:message Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C81D69.180EDF76" This is a multi-part message in MIME format. ------_=_NextPart_001_01C81D69.180EDF76 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable I've re-started the email thread. =20 Thanks Georgios for your description of LC-PCN in recent email. I'm going to try & re-phrase how it works, partly to make sure I've understood & partly to try and re-use the PCN terminology (I admit to finding the translations required hard work). Hence it slightly repeats some of my earlier email, but some of its different where I now understand your scheme better =20 First, let me summarise to make sure I understand it right [fingers crossed I do!]. =20 - two encodings are used. one ('PCN-marking') is used for both adm ctrl & termination to indicate the excess traffic. If a PCN-interior-node is PCN-marking some pkts, then it marks all other pkts with another encoding ('affected marking') =20 - the 'PCN-marking' algorithm has several regimes: [let us assume that none of the packets arriving at an interface are PCN-marked] * when traffic rate is below PCN-lower-rate, no pkts are marked=20 * as the traffic rate climbs above the PCN-lower-rate, an increasing number of pkts are marked (marking rate =3D excess rate divided by N). note, this applies whatever the traffic rate (the same formula, excess rate divided by N, is used when the traffic rate is above the PCN-upper-rate =20 * Fiddle factor 1: [applies to marking when rate is below PCN-lower-rate] if pkts arrive already PCN-marked, then the node measures the rate of pkts already marked & works out what excess rate this corresponds to, and only marks additional pkts if its own excess rate is even higher. * Fiddle factor 2: [applies to marking when rate is above PCN-upper-rate] assume that any rate above PCN-upper-rate, as measured in the previous measurement window period T, will be in the process of being terminated. So can ignore this much excess (above PCN-upper-rate) in this measurement period. =20 * the PCN-egress-node measures the rate of PCN-marked pkts and determines whether the overloaded node is in adm ctrl state or termination ctrl state * PCN-egress-node calculates the % of PCN-bytes that are PCN-marked, multiplying by the N factor (to compensate for the PCN-interior-nodes only marking 1/N of their excess rate). If above a certain % then new calls are blocked.=20 * PCN-egress-node also calculates the amount (bit rate) of PCN-bytes that are PCN-marked (again multiplying by the N factor). If this is above the PCN-upper-rate then it terminates some flows =20 * for a variety of reasons the following 4 factors have to be the same on every node in the PCN-domain: N, T (sliding window measurement period), [PCN-upper-rate - PCN-lower-rate], [(policed rate for the PCN class) - PCN-upper-rate]. =20 =20 Questions: * imagine there are 2 independent ECMP paths for a particular ingress-egress-aggregate, and that there's a PCN-interior-node on each path that is somewhat pre-congested (traffic between PCN-lower-rate and PCN-upper-rate - in 'adm ctrl state'). These add together at the PCN-egress-node so it appears that some PCN-interior-node is highly pre-congested (traffic above PCN-upper-rate). From your replies I believe you agree. To try to get round this you've introduced a fiddle factor, basically the PCN-egress-node has to see even more PCN-marks before it really believes that flows need terminate. * the same 'affected marking' is used in both adm ctrl state & termination ctrl state. Imagine that a node [node-1] on one ECMP path is in adm ctrl state & a node [node-2] on another ECMP path is in termination ctrl state. Therefore flows are terminated, and only flows that are being PCN-marked or affected-marked are terminated. However this could lead to flows being terminated that go through node-1; node-2 will still be just as badly overloaded=20 * one reading of these two bullets is that you haven't really solved ECMP. * there are a lot of parameters that need to be configured with the same value everywhere in the PCN-domain. It would need careful thought about what goes wrong if some of them are mis-configured on some PCN-nodes * it's particularly foul if T (the measurement period) needs to be the same on every PCN-node. I'm not sure this is really the case, but it seemed to me to come out of "Fiddle factor 2" * looking at the 2 rate parameters, [PCN-upper-rate - PCN-lower-rate], [(policed rate for the PCN class) - PCN-upper-rate]. It seems quite restrictive that these have to be the same on all PCN-nodes in the PCN-domain. They might have very different capacities for instance.=20 * the "fiddle factors" make the algorithm quite complicated I think for PCN-interior-nodes.=20 * "fiddle factor 2" seems the wrong approach to me - I think it would be better to keep marking at the same rate and for the PCN-egress-node to terminate more slowly (in order to avoid the problem of terminating too much). At least 2 reasons: if marked pkts are lost, then at best calls take longer to be terminated (another measurement period before get PCN-marks again) & at worst it might be possible to devise scenarios where termination never happens; secondly, I think it makes it easier for the PCN-egress-node to take account of policy [operator experience etc] to decide how fast to react to overload, whereas your approach fixes the speed of reaction as a parameter of the PCN-interior-node. =20 That's it for now! =20 best wishes =20 phil/ =20 =20 ------_=_NextPart_001_01C81D69.180EDF76 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

I’ve re-started the email = thread.

 

Thanks Georgios for your description of = LC-PCN in recent email. I’m going to try & re-phrase how it works, = partly to make sure I’ve understood & partly to try and re-use the PCN terminology (I admit to finding the translations required hard work). = Hence it slightly repeats some of my earlier email, but some of its different = where I now understand your scheme better

 

First, let me = summarise to make sure I understand it right [fingers crossed I = do!].

 

- two encodings are = used. one ('PCN-marking') is used for both adm ctrl & termination to indicate = the excess traffic. If a PCN-interior-node is PCN-marking some pkts, then it = marks all other pkts with another encoding ('affected = marking')

 

- the 'PCN-marking' algorithm has several regimes:

[let us assume that = none of the packets arriving at an interface are PCN-marked]

* when traffic rate = is below PCN-lower-rate, no pkts are marked

* as the traffic = rate climbs above the PCN-lower-rate, an increasing number of pkts are marked = (marking rate =3D excess rate divided by N). note, this applies whatever the traffic = rate (the same formula, excess rate divided by N, is used when the traffic rate is = above the PCN-upper-rate

 

* Fiddle factor 1: = [applies to marking when rate is below PCN-lower-rate] if pkts arrive already PCN-marked, then the node measures the rate of pkts already marked & = works out what excess rate this corresponds to, and only marks additional pkts = if its own excess rate is even higher.

* Fiddle factor 2: = [applies to marking when rate is above PCN-upper-rate] assume that any rate above PCN-upper-rate, as measured in the previous measurement window period T, = will be in the process of being terminated. So can ignore this much excess = (above PCN-upper-rate) in this measurement period.

 

* the = PCN-egress-node measures the rate of PCN-marked pkts and determines whether the = overloaded node is in adm ctrl state or termination ctrl state

* PCN-egress-node = calculates the % of PCN-bytes that are PCN-marked, multiplying by the N factor (to compensate for the PCN-interior-nodes only marking 1/N of their excess = rate). If above a certain % then new calls are blocked.

* PCN-egress-node = also calculates the amount (bit rate) of PCN-bytes that are PCN-marked (again multiplying by the N factor). If this is above the PCN-upper-rate then = it terminates some flows

 

* for a variety of = reasons the following 4 factors have to be the same on every node in the PCN-domain: = N, T (sliding window measurement period), [PCN-upper-rate – = PCN-lower-rate], [(policed rate for the PCN class) – = PCN-upper-rate].

 

 

Questions:

* imagine there are = 2 independent ECMP paths for a particular ingress-egress-aggregate, and that = there’s a PCN-interior-node on each path that is somewhat pre-congested (traffic = between PCN-lower-rate and PCN-upper-rate – in ‘adm ctrl = state’). These add together at the PCN-egress-node so it appears that some = PCN-interior-node is highly pre-congested (traffic above PCN-upper-rate). From your = replies I believe you agree. To try to get round this you’ve introduced a fiddle = factor, basically the PCN-egress-node has to see even more PCN-marks before it = really believes that flows need terminate. * the same 'affected marking' is = used in both adm ctrl state & termination ctrl state. Imagine that a node = [node-1] on one ECMP path is in adm ctrl state & a node [node-2] on another = ECMP path is in termination ctrl state. Therefore flows are terminated, and only = flows that are being PCN-marked or affected-marked are terminated. However = this could lead to flows being terminated that go through node-1; node-2 will still = be just as badly overloaded

* one reading of = these two bullets is that you haven’t really solved ECMP.

* there are a lot = of parameters that need to be configured with the same value everywhere in = the PCN-domain. It would need careful thought about what goes wrong if some = of them are mis-configured on some PCN-nodes

* it’s = particularly foul if T (the measurement period) needs to be the same on every = PCN-node. I’m not sure this is really the case, but it seemed to me to come out of = “Fiddle factor 2”

* looking at the 2 = rate parameters, [PCN-upper-rate – PCN-lower-rate], [(policed rate for the PCN = class) – PCN-upper-rate]. It seems quite restrictive that these have to be the = same on all PCN-nodes in the PCN-domain. They might have very different = capacities for instance.

* the “fiddle = factors” make the algorithm quite complicated I think for PCN-interior-nodes. =

* “fiddle = factor 2” seems the wrong approach to me – I think it would be better to = keep marking at the same rate and for the PCN-egress-node to terminate more = slowly (in order to avoid the problem of terminating too much). At least 2 = reasons: if marked pkts are lost, then at best calls take longer to be terminated = (another measurement period before get PCN-marks again) & at worst it might = be possible to devise scenarios where termination never happens; secondly, I think = it makes it easier for the PCN-egress-node to take account of policy [operator experience etc] to decide how fast to react to overload, whereas your = approach fixes the speed of reaction as a parameter of the = PCN-interior-node.

 

That’s it for = now!

 

best = wishes

 

phil/

 

 

=00 ------_=_NextPart_001_01C81D69.180EDF76-- --===============1695423055== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn --===============1695423055==-- From pcn-bounces@ietf.org Fri Nov 02 12:22:22 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1InzGK-0007YT-8W; Fri, 02 Nov 2007 12:20:48 -0400 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1InzGI-0007YN-VC for pcn-confirm+ok@megatron.ietf.org; Fri, 02 Nov 2007 12:20:46 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1InzGI-0007Y9-LP for pcn@ietf.org; Fri, 02 Nov 2007 12:20:46 -0400 Received: from rotterdam.ewi.utwente.nl ([130.89.10.5]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1InzGC-0003zu-71 for pcn@ietf.org; Fri, 02 Nov 2007 12:20:46 -0400 Received: from webmail.cs.utwente.nl (janus.ewi.utwente.nl [130.89.10.26]) by rotterdam.ewi.utwente.nl (8.13.6/8.13.6) with SMTP id lA2GKQOH003242; Fri, 2 Nov 2007 17:20:26 +0100 (MET) Received: from 193.192.232.84 (auth. user karagian@imap1.ewi.utwente.nl) by webmail.cs.utwente.nl with HTTP; Fri, 02 Nov 2007 16:20:25 +0000 To: "Anna Charny (acharny)" , "anurag.bhargava@ericsson.com" , "Lars Westberg (KI/EAB)" Date: Fri, 02 Nov 2007 16:20:23 +0000 X-Mailer: IlohaMail/0.8.13 (On: webmail.cs.utwente.nl) Message-ID: In-Reply-To: From: "Georgios Karagiannis" Bounce-To: "Georgios Karagiannis" MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0 () X-Scanned-By: MIMEDefang 2.52 on 130.89.10.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0rc3 (rotterdam.ewi.utwente.nl [130.89.10.5]); Fri, 02 Nov 2007 17:20:29 +0100 (MET) X-Spam-Score: 0.0 (/) X-Scan-Signature: d424907374faffed8e9e11e94f671eb2 Cc: "pcn@ietf.org" Subject: [PCN] RE: Questions on LC-PCN draft version 01 X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Anna Thank you very much for your comments! Please see in line! On 11/1/2007, "Anna Charny (acharny)" wrote: > >Hi Georgios, > >A few more clarification questions: > >> -----Original Message----- >> From: Georgios Karagiannis [mailto:karagian@cs.utwente.nl] >> Sent: Thursday, November 01, 2007 2:35 AM >> To: Anna Charny (acharny); anurag.bhargava@ericsson.com; Lars >> Westberg (KI/EAB) >> Cc: pcn@ietf.org >> Subject: Re: Questions on LC-PCN draft version 01 >> >> Hi Anna >> >> Thank you very much for your comments! >> Before answering to your comments, see in line below, I would >> like to explain in an abstract way the LC-PCN algorithm. >> >> * Setting the thresholds at PCN_interior_nodes: >> ----------------------------------------------- >> In order to calculate the PCN_upper_rate we use two parameters: >> Maximum PHB capacity: that is the maximum capacity that can >> be supported by a PCN_interior_node >> Termination_offset_rate: that is an absolute rate value that >> should be set equal into all PCN_interior_nodes. Note that >> this value is used by PCN_interior_nodes to calculate their >> PCN_upper_rate and also during the situation that a >> PCN_interior_node is in flow termination state and it >> receives PCN_marked packets. Please see pseudo code on page 20. >> This value must be set equal into all PCN_interior_nodes such >> that all these nodes will know when to take into account the >> incoming PCN_marked packets and when not. >> >> The PCN_upper_rate is then found as: >> PCN_upper_rate =3D "Maximum PHB capacity" - Termination_offset_rate >> > >So if you have a link of 10 Mbps and a link of 40 Mbps (say each is >allowed to use all bandwidth for PCN, for simplicity), then they both >use the same *absolote* value of the termination-offset-rate?. Georgios: Yes, you are right! This has to do with the importance of using the Termination_offset_rate. We will motivate this in the following version of the draft. > So If I >wanted to have PCN-upper-rate 20% below my link capacity on all links, >I would not be able to do so with this approach? A larger link would >have a smaller relative safety margin, then... Georgios: Yes, this can be considered as a disadvantage. However, there might be other ways of preconfiguring each PCN_interior_node to know the termination_offset rate used by each neighbour PCN_interior_node. Then the pseudocode on page 20 will have to use the Termination_offset_rate associated with the incoming link from where the incoming PCN_marked packets are arriving. Then this constraint can be avoided. >Same comment for the >difference between admission and termination - the relative difference >between admission and termination threshods (Pcn-lower-rate and >pcn-upper-rate) is global, so the larger links have a smaller relative >difference between admission and termination thresholds. Right? Georgios: No, because the Admission_offset_rate is an absolute value! So this difference is globally equal. > >> The PCN_lower_rate is configured in all PCN_interior-nodes >> and it can be calculated in the following way: >> PCN_lower_rate =3D PCN_upper_rate - Admission_offset_rate >> >> The Admission_offset_rate is an absolute rate value and it is >> equal in all PCN_interior_nodes and PCN_egress_nodes. Note >> that this value is used by PCN_interior_nodes to calculate >> their PCN_lower_rate and the PCN_egress_nodes to calculate >> their PCN_upper_rate_egress. Furthermore, this value is used >> by the PCN_interior_nodes also during the situation that a >> PCN_interior_node is in admission control state and it >> receives PCN_marked packets. Please see pseudo code on page 13. >> This value must be set equal into all PCN_interior_nodes such >> that all these nodes will know when to take into account the >> incoming PCN_marked packets and when not. Note that a >> PCN_interior_node can PCN_mark packets up to an excess rate >> equal to the Admission_offset_rate. If the exess rate in an >> PCN_interior_node is higher than the Admission_offset_rate, >> then the PCN_interior_node changes state from admission >> control state to flow termination state. > >OK. > >> >> * Setting the thresholds at PCN_egress_nodes: >> --------------------------------------------- >> The question is how to calculate the threshold that defines >> when a PCN_egress_node goes into the admission control state. >> One way to do that is to consider that when the >> PCN_egress_node receives a PCN_marked packet it will mean >> that at least one PCN_interior_node started to be admission >> control congested and therefore it will go from Normal state >> to admission control state. Of course this will somehow might >> provide some errors, because there might be situations that >> this consideration might be to conservative. Therefore, we >> use a percentage of received PCN_marking encoded packets in >> proportion to total rate of received packets. In this version >> of the draft we call this value as: >> PCN_lower_rate_egress. This is wrong. >> What we should say is: >> PCN_lower_rate_egress: is equal to incoming_PCN_marking_rate, when: >> (incoming_PCN_marking_rate/measured PHB rate) =3D to a >> preconfigured percentage, say 1%. From now on I denote this >> percentage as: >> PCN_lower_percentage_egress. > >In the draft, pcn_lower- and -upper_rate_egress seem to be defined on a >per *ingress-egress-pair basis*. >Is that correct? georgios: Yes, it is correct! > >> Thus we can say that a PCN_egress_node changes from Normal >> state to admission control state when >> incoming_PCN_marking_rate/measured PHB rate > >> PCN_lower_percentage_egress. >> Where, >> incoming_PCN_marking_rate =3D N * input_PCN_marking_bytes/T >> Where, input_PCN_marking_bytes =3D received number of >> "PCN_marking" encoded packets during measurement period T. > >Again, is it per ingress-egress pair, or not? georgios: Yes, it is correct! > >> >> Now we defined the condition that the PCN_egress_node changes >> state from Normal state to admission control state. But how >> will the PCN_egress_node change state from admission control >> state to flow termination state. >> In order to explain this, it is imporatnt to note that each >> PCN_interior_node that is in admission control state it can >> PCN_mark packets up to a value equal to >> Admission_offset_rate. Furthermore, if a PCN_interior_node >> receives incoming PCN_marked packets and is in the addmission >> control state, it will not remark any packets if the excess >> rate is equal or lower than the incoming_PCN_marking_rate, >> see page 13. >> Furthermore, if we will consider as normal situations the >> situations that no ECMP occurs and that all flows belonging >> to the same ingress-egress aggregate will use the same path >> from PCN_ingress to PCN_egress, this will mean that when the >> PCN_egress_node receives, for the given ingress-egress >> aggregate an excess rate equal to Admission_offset_rate it >> will have to change from admission control state to flow >> termination state. >> Thus in this case the second threshold, that in this case is >> a rate and not a percentage, can be calculated as follows: >> PCN_upper_egress_rate =3D PCN_lower_egress_rate + Admission_offset_rate. > > >Here is where I am afraid I am fundamentally confused. >Pcn_lower-egress_rate seems to have beed redefined above as a percentage >threshold, but it is >an absolute rate again here. Perhaps you just mean that >Pcn-lower-egress-rate =3D >pcn_lower_egress_percentage * >total-measured-rate-of-this-ingress-egrees-aggregate/100? >So pcn-lower-egress-rate then is just the fraction of the total rate of >the ingress-egress aggregate corresponding to the configured percentage >threshold. Right? Georgios: I think I could not explain and translate what I wanted to do in the right way! A better way of expressing the need of using a rate of percentage of the received PCN_marking encoded packets, would be the following: Pcn-lower-egress-rate =3D pcn_lower_egress_percentage * Admission_offset_rate This means that event_A in Section 4.2.3 will be activated when incoming_PCN_marking_rate > pcn_lower_egress_percentage * Admission_offset_rate or when: incoming_PCN_marking_rate > Pcn-lower-egress-rate This means that the PCN_egress_ node does not change to admission control state when at least one PCN_marked packet arrives, but it changes when the incoming_PHB_marking rate equals a percentage of the Admission_offset_rate. This is due to the fact that Admission_offset_rate is used by the interioe nodes to change from admission control state to flow termination state. > >Assuming the above is correct, I cannot see how the system will work >correctly. Consider the following example. A bottleneck link of >capacity 100 mbps , pcn-termination-offset of 40mbps and >pcn-admission-offset of 30 mbps is shared by 100 ingress-egress >aggregates, each with the total pcn rate of 1 mbps. On this link >pcn-upper-threshold is 60mbps, and pcn-lower-threshold is 30mbps. >Suppose further that all 100 ingress-egress aggregates go to different >egress nodes (and suppose for simplicity there is no other traffic going >to these egresses). The bottleneck link is clearly in the termination >state, as the total rate of pcn traffic on this link is 100 mbps which >is above the pcn-upper-threshold. This means we want the egress nodes to >somehow recognise this state and get into the termination mode. But it >seems in this example the egresses can't ever recognize that the system >is in the termination mode. See below. > >Suppose pcn-lower-egress-percentage is set to w% (0of these egresses will compute the (absolute) pcn-lower-egress-rate as >total pcn traffic of the ingress-egress aggregate it sees (which is 1 >mbps) times the pcn_lower_egress_percentage/100, so it will get >pcn-lower-egress-rate=3D w* 0.01 Mbps. >According to your explanation above, it will then compute >Pcn-upper-egress-rate=3Dpcn-lower-egress-rate+Admission-offset-rate=3D >w*0.01 mbps+30mbps >=3D 30 mbps (for all possible settings of w). > >So our pcn-upper-egress rate is always greater than 30 Mbps, while the >entire rate of pcn traffic the egress will ever see is *at most* 1 Mbps >(because it is plainly the total rate of the single ingress-egress >aggreate that goes to this egress). So, regardless of the actual amount >of marked traffic in this egress-egress aggregate, the absolute excess >rate of this ingress-egress aggregate will never be above 1mbps either. >In turn that means that the pcn-upper-egress-rate will NEVER be exceeded >(because pcn-upper-egress-rate is so much larger than the total rate of >the pcn traffic at the egress). In turn, that appears to mean that in >this scenario none of the egresses will ever end up in termination >state, and so termination will never occur. That does not seem right. Georgios: Thank you for the given example, which stimulated me to better translate what I wanted to specify and what I have written down. Please see above that I defned: Pcn-lower-egress-rate =3D pcn_lower_egress_percentage * Admission_offset_rate The situation that you described cannot (often) occur because the difference between the admission control threshold and flow termination threshold at the egress is equal to Admission_offset_rate +/- multicongestion_error. Note that the Admission_offset_rate is an absolute rate value. The multicongestion_error is used to identify the bounds that certain situations occur where the PCN_egreess_ node should have been in flow termination state but it is not. Please see previous discussions on this. > >I would appreciate any help in clarifying this. > >Anna > > >> However, there are some corner cases, that mainly occur when >> the different congestion points (admission control congested >> PCN_interior_nodes) on the same path are not simulataneously >> starting to be congested. Therefore we use the >> multicongestion_error parameter to identify the error bound >> that ocurs due to these corner cases. Note that this error >> bound can be e.g., predefined ones off line by the operator, >> by studying the network topology and/or studying how often >> such corner cases could occur and/or doing off line >> measurements. Therefore we use: >> PCN_upper_rate_egress =3D PCN_lower_rate_egress + >> Admission_offset_rate +/- multicongestion_error >> >> * How the states of operation in PCN_interior_nodes are being changed? >> --------------------------------------------------------------------- >> This is explained on page 17, 18, by using Figure 4. >> >> Change from Normal state to Admission control state: event A >> Occurs when: >> Measured PHB rate > PCN_lower_rate >> >> Change from Admission control state to Flow Termination >> state: event B Occurs when: >> Measured PHB rate > PCN_upper_rate >> >> * How the states of operation in PCN_egress_nodes are being changed? >> --------------------------------------------------------------------- >> This is explained on page 21, 22. Note that the description >> of event A on page 21 has to be modified to avoid the >> confusions that were caused up to now, see explanation given above. >> >> Change from Normal state to Admission control state: event A >> Occurs when: >> incoming_PCN_marking_rate/measured PHB rate) > >> PCN_lower_percentage_egress As explained above: >> IF ((incoming_PCN_marking_rate/measured PHB rate) =3D >> PCN_lower_percentage_egress) >> THEN PCN_lower_egress_rate =3D incoming_PCN_marking_rate >> >> Change from Admission control state to flow termination >> state: event B Occurs when: >> incoming_PCN_marking_rate > PCN_upper_rate_egress >> >> It is important to note that also the explanation of event C >> on page 21, has to be modified to avoid the confusions that >> were caused up to now, see explanation given above: >> Change from Admission control state to Normal state: >> Occurs when: >> incoming_PCN_marking_rate/measured PHB rate) =3D< >> PCN_lower_percentage_egress >> >> >> * Generated excess rate by an PCN_interior_node operating in >> admission control state. >> ------------------------------------------------------------- >> >> The excess rate =3D signaled_overload_rate. >> The maximum excess rate that a PCN_interior_node can >> calculate in admission control state is equal to >> Admission_offset_rate, see explanation above. >> The number of bytes that are remarked, >> signaled_remarked_bytes depend on the value of calculated >> excess rate (signaled_overload_rate), the value of the >> Admission_offset_rate and the value of the >> incoming_PCN_marking_rate, see pseudo code on page 13. >> >> Note that all packets that are passing through a congested >> PCN_interior_node an are not being PCN_marked by the >> PCN_interior_node have to be remarked using the PCN_Affected_marking. >> >> >> Regarding probes, the probe packets that are passing through >> a congested node are either PCN_marked or PCN_Affected_marked. >> >> * Generating excess rate by an PCN_interior_node operating in >> flow termination state: >> ---------------------------------------------------------------- >> The excess rate =3D signaled_overload_rate. >> Note that the calculation of the signaled_overload_rate is >> different than in the situation that the PCN_Interior_node >> operates in admission control state, see page 19 and 20. This >> is due to the fact that a sliding window is used to solve an >> undershooting problem, see discussion on page 19. >> The number of bytes that are remarked, >> signaled_remarked_bytes depend on the value of calculated >> excess rate (signaled_overload_rate), the value of the >> Termination_offset_rate and the value of the >> incoming_PCN_marking_rate, and the see pseudo code on page 20. >> >> Note that all packets that are passing through a congested >> PCN_interior_node an are not being PCN_marked by the >> PCN_interior_node have to be remarked using the PCN_Affected_marking. >> >> * Providing admission control at PCN_egress_nodes: >> -------------------------------------------------- >> When the PCN_egress_node is operating in admission control >> state than a flow that is requesting admission into the PCN >> domain can b etreated in the following way: >> >> If no probing is used, the request for admission can be >> accomplished by using an external to PCN signaling protocol. >> In this case when the request arrives at a PCN_egress_node >> that operates in admission control state then the request is >> rejected. If it operates in Normal state is accepted. >> >> If probing is used, the request for admission is accomplished >> by using probe packets. In this case when the probe arrives >> at a PCN_egress_node and it is either PCN_marking or >> PCN_Affected_marking encoded is rejected. Otherwise is accepted. >> Note that probes can only be used when PCN_Affected_marking >> is appled in whole PCN domain. Otherwise, the admission >> control procedure will work by using e.g., an external >> signaling protocol used between PCN_ingress_nodes and >> PCN_egress_nodes. >> >> * Providing flow termination at PCN_egress_nodes: >> -------------------------------------------------- >> >> When a PCN_egress_node operates on flow termination it >> calculates the incoming excess rate (incoming_PCN_marking_rate). >> By using the excess rate the PCN_egress_node calclates the >> numer of flows that have to be terminated using the pseudo >> code given on page 23. Note that the PCN_egress_node has to >> maintain per flow reservation states. >> The information contained in the per flow states is used for >> the calculation of the flow that have to be terminated, see >> pseudocode on page 23. Note also that this pseudo code uses >> priority clases, but it operates when also no priority clases >> are used by setting Maximum_priority =3D 0 Where, 0 =3D< >> priority_class =3D< Maximum_priority >> >> >> >> Below, I am providing some answers to your comments! >> >> >> >> >> >> >> >> >> > -----Original Message----- >> > From: Anna Charny (acharny) [mailto:acharny@cisco.com] >> > Sent: zondag 7 oktober 2007 21:40 >> > To: anurag.bhargava@ericsson.com; Georgios Karagiannis; >> Lars Westberg >> > (KI/EAB) >> > Cc: pcn@ietf.org >> > Subject: Questions on LC-PCN draft version 01 >> > >> > Dear authors of the LC-PCN draft, >> > >> > Am I right that there has been no response yet to Phil's questions >> > (attached at the end) on the new version of your draft below (I >> > checked the pcn list and it does not seem it went there)? If the >> > response was sent, can you resend it please? >> > >> > I have many of the same questions, and also a few additional ones. >> > Here are some of them. >> > >> > 1) I do not understand whether pcn-lower_rate_egress and >> > pcn_upper_rate ingress are expressed as a fraction of the >> total rate >> > (a unitless entity, analogous to CLE) or an absolute value >> (in bits or >> > bytes per second). The text seems to be contradictory on this point: >> > >> > on page 8 it is a fraction: >> > >> > "PCN_lower_rate_egress =3D predefined percentage of received >> > PCN_marking", >> > >> > while on page 13 it is an absolute rate: >> > >> > "If the incoming_PCN_marking_rate is higher than a preconfigured >> > PCN_lower_rate_egress, ..." where the >> > >> > (Incoming_PCN_marking_rate is clearly defined as abslolue >> rate on page >> > 12: "Where the "incoming_PCN_marking_rate" is calculated as follows: >> > incoming_PCN_marking_rate =3D (received number of "PCN_marking" >> > DSCP during T)* N)/T;) >> >> Georgios: You are right, please see discussion above on >> paragaph denoted >> as: >> "Setting the thresholds at PCN_egress_nodes". >> >> >> We use a percentage of received PCN_marking encoded packets >> in proportion to total rate of received packets to define >> when a PCN_egres_node goes from Normal state to admission >> control state. In this version of the draft we call this >> value as: PCN_lower_rate_egress. This is wrong. >> What we should say is: >> PCN_lower_rate_egress: is equal to incoming_PCN_marking_rate, when: >> (incoming_PCN_marking_rate/measured PHB rate) =3D to a >> preconfigured percentage, say 1%. From now on I denote this >> percentage as: >> PCN_lower_percentage_egress. >> Thus we can say that a PCN_egress_node changes from Normal >> state to admission control state when >> incoming_PCN_marking_rate/measured PHB rate > >> PCN_lower_percentage_egress. >> Where, >> incoming_PCN_marking_rate =3D N * input_PCN_marking_bytes/T >> Where, input_PCN_marking_bytes =3D received number of >> "PCN_marking" encoded packets during measurement period T. >> >> >> > >> > 2) Regarding the question above, >> > >> > * if pcn_lower_rate_eggress (and pcn_uppre-rate_egress) are >> absolute >> > rates, then how do you choose that abslolute rate value, given that >> > the rates of ingress-egress aggregates at the same ingress may be >> > vastly different in magnitude? >> >> Georgios: See explanation above! Hopefully this >> misundersatnding is also solved! >> >> > >> > *if, however, pcn_lower_rate_eggress (and >> > pcn_uppre-rate_egress) are ratios, then what is the guidance of >> > setting these ratios so that the egress can always tell admission >> > state from the termination state, given that the same >> marking is used >> > for admission and termination marking? >> > Specifically, suppose at the bottleneck pcn_lower_threshold >> is set to >> > X, and pcn_upper_threshold to aX with some a>1 (assume for example >> > a=3D1.5). >> > How does the egress node tell between the conditions when the total >> > pcn traffic on the bottleneck is 1.1X (which is the state wnen >> > admission is needed but termination is not, - in this case >> > (1.1X-X)/X=3D0.1 traffic is pcn-marked), and the case when the total >> > traffic on the bottleneck is 1.1aX, which is the case when >> termination >> > is needed, and (1.1ax-ax)/ax=3D0.1), if in both cases the >> ratio between >> > marked and total traffic is the same value 0.1? >> >> Georgios: In order to solve the above described problems, the >> thresholds used in the PCN_interior_nodes and PCN_egress_nodes are >> using: >> the Admission_offset_rate that is an absolute rate value >> which is set equal into whole PCN domain. >> >> Note that the maximum excess rate that a PCN_interior_node >> can calculate in admission control state is equal to >> Admission_offset_rate. If the excess rate is higher than the >> Admission_offset_rate then the node changes from admission >> control state to flow termination state. >> >> Please see all details that I have provided at the top of >> this email. In particular, check the paragraphs denoted above as: >> "Setting the thresholds at PCN_interior_nodes" >> "Setting the thresholds at PCN_egress_nodes" >> "Generated excess rate by an PCN_interior_node operating in >> admission control state" >> >> >> >> > >> > 3) On page 9, multicongestion error error is introduced, >> but the text >> > is silent on how this error is defined and what is it set >> to (if it is >> > a configuration parameter), or how it is measured (if it is >> something >> > that is being measured). Can you explain, please? >> >> Georgios: >> Please see the paragraph denoted above as "Setting the >> thresholds at PCN_egress_nodes" >> >> >> Best regards, >> Georgios >> >> >> > >> > I have some more questions, including sharing those asked >> by Phil in >> > the attached message), but I will stop now, as I hope >> clarifications >> > on the above (and Phil's) questions will help understand the draft >> > better... >> > >> > Thank you in advance for clarifying all this. >> > >> > Anna >> _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Fri Nov 02 13:14:39 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Io068-0006AG-RY; Fri, 02 Nov 2007 13:14:20 -0400 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Io068-00069d-3b for pcn-confirm+ok@megatron.ietf.org; Fri, 02 Nov 2007 13:14:20 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Io067-00069V-Po for pcn@ietf.org; Fri, 02 Nov 2007 13:14:19 -0400 Received: from rotterdam.ewi.utwente.nl ([130.89.10.5]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1Io066-0005w6-FP for pcn@ietf.org; Fri, 02 Nov 2007 13:14:19 -0400 Received: from webmail.cs.utwente.nl (janus.ewi.utwente.nl [130.89.10.26]) by rotterdam.ewi.utwente.nl (8.13.6/8.13.6) with SMTP id lA2HE8Dd017619; Fri, 2 Nov 2007 18:14:12 +0100 (MET) Received: from 193.192.232.84 (auth. user karagian@imap1.ewi.utwente.nl) by webmail.cs.utwente.nl with HTTP; Fri, 02 Nov 2007 17:14:07 +0000 To: "philip.eardley@bt.com" , "pcn@ietf.org" Subject: Re: [PCN] Comments /questions on LC-PCN draft Date: Fri, 02 Nov 2007 17:14:06 +0000 X-Mailer: IlohaMail/0.8.13 (On: webmail.cs.utwente.nl) Message-ID: In-Reply-To: <75A199C5D243C741BF3D3F1EBCEF9BA503B342C7@E03MVZ1-UKDY.domain1.systemhost.net> From: "Georgios Karagiannis" Bounce-To: "Georgios Karagiannis" MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0 () X-Scanned-By: MIMEDefang 2.52 on 130.89.10.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0rc3 (rotterdam.ewi.utwente.nl [130.89.10.5]); Fri, 02 Nov 2007 18:14:15 +0100 (MET) X-Spam-Score: 0.0 (/) X-Scan-Signature: a4a24b484706be629f915bfb1a3e4771 Cc: X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Phil Thank you very much! Please see in line! On 11/2/2007, "philip.eardley@bt.com" wrote: >I've re-started the email thread. > > > >Thanks Georgios for your description of LC-PCN in recent email. I'm >going to try & re-phrase how it works, partly to make sure I've >understood & partly to try and re-use the PCN terminology (I admit to >finding the translations required hard work). Hence it slightly repeats >some of my earlier email, but some of its different where I now >understand your scheme better > > > >First, let me summarise to make sure I understand it right [fingers >crossed I do!]. > > > >- two encodings are used. one ('PCN-marking') is used for both adm ctrl >& termination to indicate the excess traffic. If a PCN-interior-node is >PCN-marking some pkts, then it marks all other pkts with another >encoding ('affected marking') > > > >- the 'PCN-marking' algorithm has several regimes: > >[let us assume that none of the packets arriving at an interface are >PCN-marked] > >* when traffic rate is below PCN-lower-rate, no pkts are marked Georgios: Yes, right > >* as the traffic rate climbs above the PCN-lower-rate, an increasing >number of pkts are marked (marking rate =3D excess rate divided by N). >note, this applies whatever the traffic rate (the same formula, excess >rate divided by N, is used when the traffic rate is above the >PCN-upper-rate Georgios: when the traffic rate is above the PCN-upper-rate then the excess rate is calculated the same, but the sliding window mechanism is used to avoid undershoot. > > > >* Fiddle factor 1: [applies to marking when rate is below >PCN-lower-rate] if pkts arrive already PCN-marked, then the node >measures the rate of pkts already marked & works out what excess rate >this corresponds to, and only marks additional pkts if its own excess >rate is even higher. Georgios: As I said earlier this is only partially true, see pseudo-code on page 13, where the remarking depends on the incoming_PHR_marking_rate but also on teh Admission_offset_rate. > >* Fiddle factor 2: [applies to marking when rate is above >PCN-upper-rate] assume that any rate above PCN-upper-rate, as measured >in the previous measurement window period T, will be in the process of >being terminated. So can ignore this much excess (above PCN-upper-rate) >in this measurement period. Georgios: As I mentioned earlier it is more complex than that, please see pseudocode on page 20. The remarking also depends on the incoming_PCN_marking_rate and on the Termination_offset_rate. > > > >* the PCN-egress-node measures the rate of PCN-marked pkts and >determines whether the overloaded node is in adm ctrl state or >termination ctrl state Georgios: Yes, right > >* PCN-egress-node calculates the % of PCN-bytes that are PCN-marked, >multiplying by the N factor (to compensate for the PCN-interior-nodes >only marking 1/N of their excess rate). If above a certain % then new >calls are blocked. Georgios:Yes, but when probing is used then this somehow different, because the egress checks if the probe is marked or not. > >* PCN-egress-node also calculates the amount (bit rate) of PCN-bytes >that are PCN-marked (again multiplying by the N factor). If this is >above the PCN-upper-rate then it terminates some flows. Georgios: Yes, right. > > > >* for a variety of reasons the following 4 factors have to be the same >on every node in the PCN-domain: N, T (sliding window measurement >period), [PCN-upper-rate - PCN-lower-rate], [(policed rate for the PCN >class) - PCN-upper-rate]. Georgios: Yes. Note however, that the last one that represents Termination_offset_rate could be released from this constraint if each PCN_interior_node is able to know the Termination_offset_rate used by each of its neighbouring PCN_interior_nodes. > > > > > >Questions: > >* imagine there are 2 independent ECMP paths for a particular >ingress-egress-aggregate, and that there's a PCN-interior-node on each >path that is somewhat pre-congested (traffic between PCN-lower-rate and >PCN-upper-rate - in 'adm ctrl state'). These add together at the >PCN-egress-node so it appears that some PCN-interior-node is highly >pre-congested (traffic above PCN-upper-rate). From your replies I >believe you agree.=20 Georgios: Yes, I agree, but do you agree that this is a corner case? >To try to get round this you've introduced a fiddle >factor, basically the PCN-egress-node has to see even more PCN-marks >before it really believes that flows need terminate.=20 Georgios: Yes, you are right * the same >'affected marking' is used in both adm ctrl state & termination ctrl >state. Imagine that a node [node-1] on one ECMP path is in adm ctrl >state & a node [node-2] on another ECMP path is in termination ctrl >state. Therefore flows are terminated, and only flows that are being >PCN-marked or affected-marked are terminated. However this could lead to >flows being terminated that go through node-1; node-2 will still be just >as badly overloaded Georgios: Not really! Node 2 will not be just as badly overloaded. Flows will be terminated that go through node-2 and only few flows that go throgh node-1 could be terminated. Without using affected_marking the severe congestion on node-2 will be in a smaller proportion solved. Furthermore, note that affected marking in admission control state is only required when probing is required, in order to accomplish the case that the probe always get marked when it passes through a congested node. > >* one reading of these two bullets is that you haven't really solved >ECMP.=20 Georgios: Agree that when Affected_marking is used during flow termination state and is also used during admission control state, then the ECMP solution used during flow termination state works less optimal than the situation that the affected_marking is only used during flow termination state and not in admission control state. However, it works better than if you would not use Affected_marking at all. Note that if probing is not used, then affected_marking is not used during the admisison control state. In this way the admission control state does not solve the ECMP case, but during flow termination, the ECMP case can be competely solved. > >* there are a lot of parameters that need to be configured with the same >value everywhere in the PCN-domain. It would need careful thought about >what goes wrong if some of them are mis-configured on some PCN-nodes Georgios: Agree, > >* it's particularly foul if T (the measurement period) needs to be the >same on every PCN-node. I'm not sure this is really the case, but it >seemed to me to come out of "Fiddle factor 2" > >* looking at the 2 rate parameters, [PCN-upper-rate - PCN-lower-rate], >[(policed rate for the PCN class) - PCN-upper-rate]. It seems quite >restrictive that these have to be the same on all PCN-nodes in the >PCN-domain. They might have very different capacities for instance. Georgios: I think that the main problem is related to setting the [(policed rate for the PCN class) - PCN-upper-rate] constant. Please see above for the explanation. > >* the "fiddle factors" make the algorithm quite complicated I think for >PCN-interior-nodes. georgios: Quite complicated compared to what? > >* "fiddle factor 2" seems the wrong approach to me - I think it would be >better to keep marking at the same rate and for the PCN-egress-node to >terminate more slowly (in order to avoid the problem of terminating too >much). At least 2 reasons: if marked pkts are lost, then at best calls >take longer to be terminated (another measurement period before get >PCN-marks again) & at worst it might be possible to devise scenarios >where termination never happens; secondly, I think it makes it easier >for the PCN-egress-node to take account of policy [operator experience >etc] to decide how fast to react to overload, whereas your approach >fixes the speed of reaction as a parameter of the PCN-interior-node. > Georgios: Please note that the PCN_egress_node changes state from admission control to termination state if the value of the excess rate exceeds the value of Admission_offset_rate +/- multicongestion_error. By using worst case scenarios the value of the multicongestion_error can be a priori estimated. > >That's it for now! > > > >best wishes > > > >phil/ _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Fri Nov 02 14:28:17 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Io1Fa-0000iz-LV; Fri, 02 Nov 2007 14:28:10 -0400 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Io1FZ-0000hV-Et for pcn-confirm+ok@megatron.ietf.org; Fri, 02 Nov 2007 14:28:09 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Io1FZ-0000gX-1R for pcn@ietf.org; Fri, 02 Nov 2007 14:28:09 -0400 Received: from sj-iport-5.cisco.com ([171.68.10.87]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1Io1FW-0000pS-6v for pcn@ietf.org; Fri, 02 Nov 2007 14:28:08 -0400 X-IronPort-AV: E=Sophos;i="4.21,363,1188802800"; d="scan'208";a="184535016" Received: from sj-dkim-4.cisco.com ([171.71.179.196]) by sj-iport-5.cisco.com with ESMTP; 02 Nov 2007 11:28:05 -0700 Received: from sj-core-2.cisco.com (sj-core-2.cisco.com [171.71.177.254]) by sj-dkim-4.cisco.com (8.12.11/8.12.11) with ESMTP id lA2IS58s016837; Fri, 2 Nov 2007 11:28:05 -0700 Received: from xbh-rtp-201.amer.cisco.com (xbh-rtp-201.cisco.com [64.102.31.12]) by sj-core-2.cisco.com (8.12.10/8.12.6) with ESMTP id lA2IRrZe026972; Fri, 2 Nov 2007 18:28:01 GMT Received: from xmb-rtp-203.amer.cisco.com ([64.102.31.20]) by xbh-rtp-201.amer.cisco.com with Microsoft SMTPSVC(6.0.3790.1830); Fri, 2 Nov 2007 14:27:56 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Fri, 2 Nov 2007 14:27:55 -0400 Message-ID: In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Questions on LC-PCN draft version 01 Thread-Index: AcgdbFJQYoieAMH+Tsy5xnVOydp6mAAEFphg From: "Anna Charny (acharny)" To: "Georgios Karagiannis" , , "Lars Westberg (KI/EAB)" X-OriginalArrivalTime: 02 Nov 2007 18:27:56.0664 (UTC) FILETIME=[16DBAF80:01C81D7E] X-TM-AS-Product-Ver: SMEX-8.0.0.1181-5.000.1023-15520.002 X-TM-AS-Result: No--39.362300-8.000000-2 X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No DKIM-Signature: v=0.5; a=rsa-sha256; q=dns/txt; l=27235; t=1194028085; x=1194892085; c=relaxed/simple; s=sjdkim4002; h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version; d=cisco.com; i=acharny@cisco.com; z=From:=20=22Anna=20Charny=20(acharny)=22=20 |Subject:=20RE=3A=20Questions=20on=20=20LC-PCN=20draft=20version=2001 |Sender:=20; bh=UcNxijUFS1PUfGOgjnhMEZEBSXYou/F6sG6e8I0rJI8=; b=lpcQf8MMY5VNDHu7GCSaxnoqdz9yXpc4rFq4BCFd4+RUfdkRGUxKEMtQBps4BOxjPfMmeEwQ B9v+u8mE2S2gMG9eGkZbpWvAmswqmaBgh7EmXVE5tzZSDu7sOgc+C93i; Authentication-Results: sj-dkim-4; header.From=acharny@cisco.com; dkim=pass ( sig from cisco.com/sjdkim4002 verified; ); X-Spam-Score: 0.0 (/) X-Scan-Signature: 06321bb70e4329e24fb56a67c5eca3a0 Cc: pcn@ietf.org Subject: [PCN] RE: Questions on LC-PCN draft version 01 X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Georgios, Still something does not seem right. So let me restate my current understanding: At the internal node: Pcn-upper-rate =3D max pcn capacity - termination-offset-rate Pcn-lower-rate =3D pcn-upper rate - admission-offset rate (both offsetas are global values) At the egress: Pcn-lower-egress-percentage - a configured percentage Pcn-lower-egress-rate =3D=20 > -----Original Message----- > From: Georgios Karagiannis [mailto:karagian@cs.utwente.nl]=20 > Sent: Friday, November 02, 2007 12:20 PM > To: Anna Charny (acharny); anurag.bhargava@ericsson.com; Lars=20 > Westberg (KI/EAB) > Cc: pcn@ietf.org > Subject: RE: Questions on LC-PCN draft version 01 >=20 > Hi Anna >=20 > Thank you very much for your comments! >=20 > Please see in line! >=20 >=20 > On 11/1/2007, "Anna Charny (acharny)" wrote: >=20 > > > >Hi Georgios, > > > >A few more clarification questions: > > > >> -----Original Message----- > >> From: Georgios Karagiannis [mailto:karagian@cs.utwente.nl] > >> Sent: Thursday, November 01, 2007 2:35 AM > >> To: Anna Charny (acharny); anurag.bhargava@ericsson.com; Lars=20 > >> Westberg (KI/EAB) > >> Cc: pcn@ietf.org > >> Subject: Re: Questions on LC-PCN draft version 01 > >> > >> Hi Anna > >> > >> Thank you very much for your comments! > >> Before answering to your comments, see in line below, I=20 > would like to=20 > >> explain in an abstract way the LC-PCN algorithm. > >> > >> * Setting the thresholds at PCN_interior_nodes: > >> ----------------------------------------------- > >> In order to calculate the PCN_upper_rate we use two parameters: > >> Maximum PHB capacity: that is the maximum capacity that can be=20 > >> supported by a PCN_interior_node > >> Termination_offset_rate: that is an absolute rate value=20 > that should=20 > >> be set equal into all PCN_interior_nodes. Note that this value is=20 > >> used by PCN_interior_nodes to calculate their=20 > PCN_upper_rate and also=20 > >> during the situation that a PCN_interior_node is in flow=20 > termination=20 > >> state and it receives PCN_marked packets. Please see=20 > pseudo code on=20 > >> page 20. > >> This value must be set equal into all PCN_interior_nodes such that=20 > >> all these nodes will know when to take into account the incoming=20 > >> PCN_marked packets and when not. > >> > >> The PCN_upper_rate is then found as: > >> PCN_upper_rate =3D "Maximum PHB capacity" - Termination_offset_rate > >> > > > >So if you have a link of 10 Mbps and a link of 40 Mbps (say each is=20 > >allowed to use all bandwidth for PCN, for simplicity), then=20 > they both=20 > >use the same *absolote* value of the termination-offset-rate?. >=20 > Georgios: Yes, you are right! This has to do with the=20 > importance of using the Termination_offset_rate. We will=20 > motivate this in the following version of the draft. >=20 > > So If I > >wanted to have PCN-upper-rate 20% below my link capacity on=20 > all links,=20 > >I would not be able to do so with this approach? A larger link would=20 > >have a smaller relative safety margin, then... >=20 > Georgios: Yes, this can be considered as a disadvantage.=20 > However, there might be other ways of preconfiguring each=20 > PCN_interior_node to know the termination_offset rate used by=20 > each neighbour PCN_interior_node. Then the pseudocode on page=20 > 20 will have to use the Termination_offset_rate associated=20 > with the incoming link from where the incoming PCN_marked=20 > packets are arriving. Then this constraint can be avoided. >=20 >=20 > >Same comment for the > >difference between admission and termination - the relative=20 > difference=20 > >between admission and termination threshods (Pcn-lower-rate and > >pcn-upper-rate) is global, so the larger links have a=20 > smaller relative=20 > >difference between admission and termination thresholds. Right? >=20 > Georgios: No, because the Admission_offset_rate is an absolute value! > So this difference is globally equal. >=20 > > > >> The PCN_lower_rate is configured in all PCN_interior-nodes=20 > and it can=20 > >> be calculated in the following way: > >> PCN_lower_rate =3D PCN_upper_rate - Admission_offset_rate > >> > >> The Admission_offset_rate is an absolute rate value and it=20 > is equal=20 > >> in all PCN_interior_nodes and PCN_egress_nodes. Note that=20 > this value=20 > >> is used by PCN_interior_nodes to calculate their=20 > PCN_lower_rate and=20 > >> the PCN_egress_nodes to calculate their PCN_upper_rate_egress.=20 > >> Furthermore, this value is used by the PCN_interior_nodes=20 > also during=20 > >> the situation that a PCN_interior_node is in admission=20 > control state=20 > >> and it receives PCN_marked packets. Please see pseudo code on page=20 > >> 13. > >> This value must be set equal into all PCN_interior_nodes such that=20 > >> all these nodes will know when to take into account the incoming=20 > >> PCN_marked packets and when not. Note that a PCN_interior_node can=20 > >> PCN_mark packets up to an excess rate equal to the=20 > >> Admission_offset_rate. If the exess rate in an=20 > PCN_interior_node is=20 > >> higher than the Admission_offset_rate, then the PCN_interior_node=20 > >> changes state from admission control state to flow=20 > termination state. > > > >OK. > > > >> > >> * Setting the thresholds at PCN_egress_nodes: > >> --------------------------------------------- > >> The question is how to calculate the threshold that defines when a=20 > >> PCN_egress_node goes into the admission control state. > >> One way to do that is to consider that when the PCN_egress_node=20 > >> receives a PCN_marked packet it will mean that at least one=20 > >> PCN_interior_node started to be admission control congested and=20 > >> therefore it will go from Normal state to admission=20 > control state. Of=20 > >> course this will somehow might provide some errors, because there=20 > >> might be situations that this consideration might be to=20 > conservative.=20 > >> Therefore, we use a percentage of received PCN_marking encoded=20 > >> packets in proportion to total rate of received packets. In this=20 > >> version of the draft we call this value as: > >> PCN_lower_rate_egress. This is wrong. > >> What we should say is: > >> PCN_lower_rate_egress: is equal to incoming_PCN_marking_rate, when: > >> (incoming_PCN_marking_rate/measured PHB rate) =3D to a = preconfigured=20 > >> percentage, say 1%. From now on I denote this percentage as: > >> PCN_lower_percentage_egress. > > > >In the draft, pcn_lower- and -upper_rate_egress seem to be=20 > defined on a=20 > >per *ingress-egress-pair basis*. > >Is that correct? >=20 > georgios: Yes, it is correct! > > > >> Thus we can say that a PCN_egress_node changes from Normal=20 > state to=20 > >> admission control state when=20 > incoming_PCN_marking_rate/measured PHB=20 > >> rate > PCN_lower_percentage_egress. > >> Where, > >> incoming_PCN_marking_rate =3D N * input_PCN_marking_bytes/T Where,=20 > >> input_PCN_marking_bytes =3D received number of "PCN_marking" = encoded=20 > >> packets during measurement period T. > > > >Again, is it per ingress-egress pair, or not? >=20 > georgios: Yes, it is correct! >=20 > > > >> > >> Now we defined the condition that the PCN_egress_node=20 > changes state=20 > >> from Normal state to admission control state. But how will the=20 > >> PCN_egress_node change state from admission control state to flow=20 > >> termination state. > >> In order to explain this, it is imporatnt to note that each=20 > >> PCN_interior_node that is in admission control state it=20 > can PCN_mark=20 > >> packets up to a value equal to Admission_offset_rate.=20 > Furthermore, if=20 > >> a PCN_interior_node receives incoming PCN_marked packets and is in=20 > >> the addmission control state, it will not remark any=20 > packets if the=20 > >> excess rate is equal or lower than the=20 > incoming_PCN_marking_rate, see=20 > >> page 13. > >> Furthermore, if we will consider as normal situations the=20 > situations=20 > >> that no ECMP occurs and that all flows belonging to the same=20 > >> ingress-egress aggregate will use the same path from=20 > PCN_ingress to=20 > >> PCN_egress, this will mean that when the PCN_egress_node receives,=20 > >> for the given ingress-egress aggregate an excess rate equal to=20 > >> Admission_offset_rate it will have to change from=20 > admission control=20 > >> state to flow termination state. > >> Thus in this case the second threshold, that in this case=20 > is a rate=20 > >> and not a percentage, can be calculated as follows: > >> PCN_upper_egress_rate =3D PCN_lower_egress_rate +=20 > Admission_offset_rate. > > > > > >Here is where I am afraid I am fundamentally confused. > >Pcn_lower-egress_rate seems to have beed redefined above as a=20 > >percentage threshold, but it is an absolute rate again here. Perhaps=20 > >you just mean that Pcn-lower-egress-rate =3D=20 > pcn_lower_egress_percentage=20 > >* total-measured-rate-of-this-ingress-egrees-aggregate/100? > >So pcn-lower-egress-rate then is just the fraction of the=20 > total rate of=20 > >the ingress-egress aggregate corresponding to the configured=20 > percentage=20 > >threshold. Right? >=20 > Georgios: I think I could not explain and translate what I=20 > wanted to do in the right way! > A better way of expressing the need of using a rate of=20 > percentage of the received PCN_marking encoded packets, would=20 > be the following: >=20 > Pcn-lower-egress-rate =3D pcn_lower_egress_percentage *=20 > Admission_offset_rate >=20 >=20 > This means that event_A in Section 4.2.3 will be activated=20 > when incoming_PCN_marking_rate > pcn_lower_egress_percentage=20 > * Admission_offset_rate or when: > incoming_PCN_marking_rate > Pcn-lower-egress-rate >=20 > This means that the PCN_egress_ node does not change to=20 > admission control state when at least one PCN_marked packet=20 > arrives, but it changes when the incoming_PHB_marking rate=20 > equals a percentage of the Admission_offset_rate. This is due=20 > to the fact that Admission_offset_rate is used by the=20 > interioe nodes to change from admission control state to flow=20 > termination state. >=20 > > > >Assuming the above is correct, I cannot see how the system will work=20 > >correctly. Consider the following example. A bottleneck link of=20 > >capacity 100 mbps , pcn-termination-offset of 40mbps and=20 > >pcn-admission-offset of 30 mbps is shared by 100 ingress-egress=20 > >aggregates, each with the total pcn rate of 1 mbps. On this link=20 > >pcn-upper-threshold is 60mbps, and pcn-lower-threshold is 30mbps. > >Suppose further that all 100 ingress-egress aggregates go to=20 > different=20 > >egress nodes (and suppose for simplicity there is no other traffic=20 > >going to these egresses). The bottleneck link is clearly in the=20 > >termination state, as the total rate of pcn traffic on this=20 > link is 100=20 > >mbps which is above the pcn-upper-threshold. This means we want the=20 > >egress nodes to somehow recognise this state and get into the=20 > >termination mode. But it seems in this example the egresses=20 > can't ever=20 > >recognize that the system is in the termination mode. See below. >=20 >=20 > > > >Suppose pcn-lower-egress-percentage is set to w% (0 Then, each=20 > >of these egresses will compute the (absolute)=20 > pcn-lower-egress-rate as=20 > >total pcn traffic of the ingress-egress aggregate it sees (which is 1 > >mbps) times the pcn_lower_egress_percentage/100, so it will get=20 > >pcn-lower-egress-rate=3D w* 0.01 Mbps. > >According to your explanation above, it will then compute=20 > = >Pcn-upper-egress-rate=3Dpcn-lower-egress-rate+Admission-offset-rate=3D > >w*0.01 mbps+30mbps >=3D 30 mbps (for all possible settings of w). > > > >So our pcn-upper-egress rate is always greater than 30 Mbps,=20 > while the=20 > >entire rate of pcn traffic the egress will ever see is *at=20 > most* 1 Mbps=20 > >(because it is plainly the total rate of the single ingress-egress=20 > >aggreate that goes to this egress). So, regardless of the=20 > actual amount=20 > >of marked traffic in this egress-egress aggregate, the=20 > absolute excess=20 > >rate of this ingress-egress aggregate will never be above=20 > 1mbps either. > >In turn that means that the pcn-upper-egress-rate will NEVER be=20 > >exceeded (because pcn-upper-egress-rate is so much larger than the=20 > >total rate of the pcn traffic at the egress). In turn, that=20 > appears to=20 > >mean that in this scenario none of the egresses will ever end up in=20 > >termination state, and so termination will never occur. That=20 > does not seem right. >=20 >=20 > Georgios: Thank you for the given example, which stimulated=20 > me to better translate what I wanted to specify and what I=20 > have written down. > Please see above that I defned: > Pcn-lower-egress-rate =3D pcn_lower_egress_percentage *=20 > Admission_offset_rate >=20 > The situation that you described cannot (often) occur because=20 > the difference between the admission control threshold and=20 > flow termination threshold at the egress is equal to=20 > Admission_offset_rate +/- multicongestion_error. Note that=20 > the Admission_offset_rate is an absolute rate value. The=20 > multicongestion_error is used to identify the bounds that=20 > certain situations occur where the PCN_egreess_ node should=20 > have been in flow termination state but it is not. Please see=20 > previous discussions on this. >=20 > > > >I would appreciate any help in clarifying this. > > > >Anna > > > > > >> However, there are some corner cases, that mainly occur when the=20 > >> different congestion points (admission control congested > >> PCN_interior_nodes) on the same path are not=20 > simulataneously starting=20 > >> to be congested. Therefore we use the=20 > multicongestion_error parameter=20 > >> to identify the error bound that ocurs due to these corner cases.=20 > >> Note that this error bound can be e.g., predefined ones=20 > off line by=20 > >> the operator, by studying the network topology and/or studying how=20 > >> often such corner cases could occur and/or doing off line=20 > >> measurements. Therefore we use: > >> PCN_upper_rate_egress =3D PCN_lower_rate_egress +=20 > Admission_offset_rate=20 > >> +/- multicongestion_error > >> > >> * How the states of operation in PCN_interior_nodes are=20 > being changed? > >>=20 > --------------------------------------------------------------------- > >> This is explained on page 17, 18, by using Figure 4. > >> > >> Change from Normal state to Admission control state: event=20 > A Occurs=20 > >> when: > >> Measured PHB rate > PCN_lower_rate > >> > >> Change from Admission control state to Flow Termination > >> state: event B Occurs when: > >> Measured PHB rate > PCN_upper_rate > >> > >> * How the states of operation in PCN_egress_nodes are=20 > being changed? > >>=20 > --------------------------------------------------------------------- > >> This is explained on page 21, 22. Note that the=20 > description of event=20 > >> A on page 21 has to be modified to avoid the confusions that were=20 > >> caused up to now, see explanation given above. > >> > >> Change from Normal state to Admission control state: event=20 > A Occurs=20 > >> when: > >> incoming_PCN_marking_rate/measured PHB rate) >=20 > >> PCN_lower_percentage_egress As explained above: > >> IF ((incoming_PCN_marking_rate/measured PHB rate) =3D > >> PCN_lower_percentage_egress) > >> THEN PCN_lower_egress_rate =3D incoming_PCN_marking_rate > >> > >> Change from Admission control state to flow termination > >> state: event B Occurs when: > >> incoming_PCN_marking_rate > PCN_upper_rate_egress > >> > >> It is important to note that also the explanation of event=20 > C on page=20 > >> 21, has to be modified to avoid the confusions that were=20 > caused up to=20 > >> now, see explanation given above: > >> Change from Admission control state to Normal state: > >> Occurs when: > >> incoming_PCN_marking_rate/measured PHB rate) =3D<=20 > >> PCN_lower_percentage_egress > >> > >> > >> * Generated excess rate by an PCN_interior_node operating in=20 > >> admission control state. > >> ------------------------------------------------------------- > >> > >> The excess rate =3D signaled_overload_rate. > >> The maximum excess rate that a PCN_interior_node can calculate in=20 > >> admission control state is equal to Admission_offset_rate, see=20 > >> explanation above. > >> The number of bytes that are remarked,=20 > signaled_remarked_bytes depend=20 > >> on the value of calculated excess rate=20 > (signaled_overload_rate), the=20 > >> value of the Admission_offset_rate and the value of the=20 > >> incoming_PCN_marking_rate, see pseudo code on page 13. > >> > >> Note that all packets that are passing through a congested=20 > >> PCN_interior_node an are not being PCN_marked by the=20 > >> PCN_interior_node have to be remarked using the=20 > PCN_Affected_marking. > >> > >> > >> Regarding probes, the probe packets that are passing through a=20 > >> congested node are either PCN_marked or PCN_Affected_marked. > >> > >> * Generating excess rate by an PCN_interior_node operating in flow=20 > >> termination state: > >> ---------------------------------------------------------------- > >> The excess rate =3D signaled_overload_rate. > >> Note that the calculation of the signaled_overload_rate is=20 > different=20 > >> than in the situation that the PCN_Interior_node operates in=20 > >> admission control state, see page 19 and 20. This is due=20 > to the fact=20 > >> that a sliding window is used to solve an undershooting=20 > problem, see=20 > >> discussion on page 19. > >> The number of bytes that are remarked,=20 > signaled_remarked_bytes depend=20 > >> on the value of calculated excess rate=20 > (signaled_overload_rate), the=20 > >> value of the Termination_offset_rate and the value of the=20 > >> incoming_PCN_marking_rate, and the see pseudo code on page 20. > >> > >> Note that all packets that are passing through a congested=20 > >> PCN_interior_node an are not being PCN_marked by the=20 > >> PCN_interior_node have to be remarked using the=20 > PCN_Affected_marking. > >> > >> * Providing admission control at PCN_egress_nodes: > >> -------------------------------------------------- > >> When the PCN_egress_node is operating in admission control=20 > state than=20 > >> a flow that is requesting admission into the PCN domain can b=20 > >> etreated in the following way: > >> > >> If no probing is used, the request for admission can be=20 > accomplished=20 > >> by using an external to PCN signaling protocol. > >> In this case when the request arrives at a PCN_egress_node that=20 > >> operates in admission control state then the request is=20 > rejected. If=20 > >> it operates in Normal state is accepted. > >> > >> If probing is used, the request for admission is accomplished by=20 > >> using probe packets. In this case when the probe arrives at a=20 > >> PCN_egress_node and it is either PCN_marking or=20 > PCN_Affected_marking=20 > >> encoded is rejected. Otherwise is accepted. > >> Note that probes can only be used when=20 > PCN_Affected_marking is appled=20 > >> in whole PCN domain. Otherwise, the admission control=20 > procedure will=20 > >> work by using e.g., an external signaling protocol used between=20 > >> PCN_ingress_nodes and PCN_egress_nodes. > >> > >> * Providing flow termination at PCN_egress_nodes: > >> -------------------------------------------------- > >> > >> When a PCN_egress_node operates on flow termination it=20 > calculates the=20 > >> incoming excess rate (incoming_PCN_marking_rate). > >> By using the excess rate the PCN_egress_node calclates the=20 > numer of=20 > >> flows that have to be terminated using the pseudo code=20 > given on page=20 > >> 23. Note that the PCN_egress_node has to maintain per flow=20 > >> reservation states. > >> The information contained in the per flow states is used for the=20 > >> calculation of the flow that have to be terminated, see=20 > pseudocode on=20 > >> page 23. Note also that this pseudo code uses priority=20 > clases, but it=20 > >> operates when also no priority clases are used by setting=20 > >> Maximum_priority =3D 0 Where, 0 =3D< priority_class =3D< = Maximum_priority > >> > >> > >> > >> Below, I am providing some answers to your comments! > >> > >> > >> > >> > >> > >> > >> > >> > >> > -----Original Message----- > >> > From: Anna Charny (acharny) [mailto:acharny@cisco.com] > >> > Sent: zondag 7 oktober 2007 21:40 > >> > To: anurag.bhargava@ericsson.com; Georgios Karagiannis; > >> Lars Westberg > >> > (KI/EAB) > >> > Cc: pcn@ietf.org > >> > Subject: Questions on LC-PCN draft version 01 > >> > > >> > Dear authors of the LC-PCN draft, > >> > > >> > Am I right that there has been no response yet to Phil's=20 > questions=20 > >> > (attached at the end) on the new version of your draft below (I=20 > >> > checked the pcn list and it does not seem it went there)? If the=20 > >> > response was sent, can you resend it please? > >> > > >> > I have many of the same questions, and also a few=20 > additional ones. > >> > Here are some of them. > >> > > >> > 1) I do not understand whether pcn-lower_rate_egress and=20 > >> > pcn_upper_rate ingress are expressed as a fraction of the > >> total rate > >> > (a unitless entity, analogous to CLE) or an absolute value > >> (in bits or > >> > bytes per second). The text seems to be contradictory on=20 > this point: > >> > > >> > on page 8 it is a fraction: > >> > > >> > "PCN_lower_rate_egress =3D predefined percentage of received=20 > >> > PCN_marking", > >> > > >> > while on page 13 it is an absolute rate: > >> > > >> > "If the incoming_PCN_marking_rate is higher than a preconfigured=20 > >> > PCN_lower_rate_egress, ..." where the > >> > > >> > (Incoming_PCN_marking_rate is clearly defined as abslolue > >> rate on page > >> > 12: "Where the "incoming_PCN_marking_rate" is calculated=20 > as follows: > >> > incoming_PCN_marking_rate =3D (received number of "PCN_marking" > >> > DSCP during T)* N)/T;) > >> > >> Georgios: You are right, please see discussion above on paragaph=20 > >> denoted > >> as: > >> "Setting the thresholds at PCN_egress_nodes". > >> > >> > >> We use a percentage of received PCN_marking encoded packets in=20 > >> proportion to total rate of received packets to define when a=20 > >> PCN_egres_node goes from Normal state to admission control=20 > state. In=20 > >> this version of the draft we call this value as:=20 > >> PCN_lower_rate_egress. This is wrong. > >> What we should say is: > >> PCN_lower_rate_egress: is equal to incoming_PCN_marking_rate, when: > >> (incoming_PCN_marking_rate/measured PHB rate) =3D to a = preconfigured=20 > >> percentage, say 1%. From now on I denote this percentage as: > >> PCN_lower_percentage_egress. > >> Thus we can say that a PCN_egress_node changes from Normal=20 > state to=20 > >> admission control state when=20 > incoming_PCN_marking_rate/measured PHB=20 > >> rate > PCN_lower_percentage_egress. > >> Where, > >> incoming_PCN_marking_rate =3D N * input_PCN_marking_bytes/T Where,=20 > >> input_PCN_marking_bytes =3D received number of "PCN_marking" = encoded=20 > >> packets during measurement period T. > >> > >> > >> > > >> > 2) Regarding the question above, > >> > > >> > * if pcn_lower_rate_eggress (and pcn_uppre-rate_egress) are > >> absolute > >> > rates, then how do you choose that abslolute rate value,=20 > given that=20 > >> > the rates of ingress-egress aggregates at the same=20 > ingress may be=20 > >> > vastly different in magnitude? > >> > >> Georgios: See explanation above! Hopefully this=20 > misundersatnding is=20 > >> also solved! > >> > >> > > >> > *if, however, pcn_lower_rate_eggress (and > >> > pcn_uppre-rate_egress) are ratios, then what is the guidance of=20 > >> > setting these ratios so that the egress can always tell=20 > admission=20 > >> > state from the termination state, given that the same > >> marking is used > >> > for admission and termination marking? > >> > Specifically, suppose at the bottleneck pcn_lower_threshold > >> is set to > >> > X, and pcn_upper_threshold to aX with some a>1 (assume=20 > for example=20 > >> > a=3D1.5). > >> > How does the egress node tell between the conditions=20 > when the total=20 > >> > pcn traffic on the bottleneck is 1.1X (which is the state wnen=20 > >> > admission is needed but termination is not, - in this case > >> > (1.1X-X)/X=3D0.1 traffic is pcn-marked), and the case when=20 > the total=20 > >> > traffic on the bottleneck is 1.1aX, which is the case when > >> termination > >> > is needed, and (1.1ax-ax)/ax=3D0.1), if in both cases the > >> ratio between > >> > marked and total traffic is the same value 0.1? > >> > >> Georgios: In order to solve the above described problems, the=20 > >> thresholds used in the PCN_interior_nodes and PCN_egress_nodes are > >> using: > >> the Admission_offset_rate that is an absolute rate value=20 > which is set=20 > >> equal into whole PCN domain. > >> > >> Note that the maximum excess rate that a PCN_interior_node can=20 > >> calculate in admission control state is equal to=20 > >> Admission_offset_rate. If the excess rate is higher than the=20 > >> Admission_offset_rate then the node changes from admission control=20 > >> state to flow termination state. > >> > >> Please see all details that I have provided at the top of=20 > this email.=20 > >> In particular, check the paragraphs denoted above as: > >> "Setting the thresholds at PCN_interior_nodes" > >> "Setting the thresholds at PCN_egress_nodes" > >> "Generated excess rate by an PCN_interior_node operating=20 > in admission=20 > >> control state" > >> > >> > >> > >> > > >> > 3) On page 9, multicongestion error error is introduced, > >> but the text > >> > is silent on how this error is defined and what is it set > >> to (if it is > >> > a configuration parameter), or how it is measured (if it is > >> something > >> > that is being measured). Can you explain, please? > >> > >> Georgios: > >> Please see the paragraph denoted above as "Setting the=20 > thresholds at=20 > >> PCN_egress_nodes" > >> > >> > >> Best regards, > >> Georgios > >> > >> > >> > > >> > I have some more questions, including sharing those asked > >> by Phil in > >> > the attached message), but I will stop now, as I hope > >> clarifications > >> > on the above (and Phil's) questions will help understand=20 > the draft=20 > >> > better... > >> > > >> > Thank you in advance for clarifying all this. > >> > > >> > Anna > >> >=20 _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Fri Nov 02 14:32:26 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Io1Ji-0003eZ-PH; Fri, 02 Nov 2007 14:32:26 -0400 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Io1Jh-0003do-Ip for pcn-confirm+ok@megatron.ietf.org; Fri, 02 Nov 2007 14:32:25 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Io1Jh-0003de-8s for pcn@ietf.org; Fri, 02 Nov 2007 14:32:25 -0400 Received: from sj-iport-4.cisco.com ([171.68.10.86]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1Io1Ja-0000tU-HZ for pcn@ietf.org; Fri, 02 Nov 2007 14:32:25 -0400 X-IronPort-AV: E=Sophos;i="4.21,363,1188802800"; d="scan'208";a="14407307" Received: from sj-dkim-4.cisco.com ([171.71.179.196]) by sj-iport-4.cisco.com with ESMTP; 02 Nov 2007 11:32:11 -0700 Received: from sj-core-1.cisco.com (sj-core-1.cisco.com [171.71.177.237]) by sj-dkim-4.cisco.com (8.12.11/8.12.11) with ESMTP id lA2IWAi4023751; Fri, 2 Nov 2007 11:32:10 -0700 Received: from xbh-rtp-201.amer.cisco.com (xbh-rtp-201.cisco.com [64.102.31.12]) by sj-core-1.cisco.com (8.12.10/8.12.6) with ESMTP id lA2IViYB016612; Fri, 2 Nov 2007 18:32:09 GMT Received: from xmb-rtp-203.amer.cisco.com ([64.102.31.20]) by xbh-rtp-201.amer.cisco.com with Microsoft SMTPSVC(6.0.3790.1830); Fri, 2 Nov 2007 14:31:48 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: PLEASE IGNORE RE: [PCN] RE: Questions on LC-PCN draft version 01 Date: Fri, 2 Nov 2007 14:31:47 -0400 Message-ID: In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: PLEASE IGNORE RE: [PCN] RE: Questions on LC-PCN draft version 01 Thread-Index: AcgdbFJQYoieAMH+Tsy5xnVOydp6mAAEFphgAABzwfA= From: "Anna Charny (acharny)" To: "Anna Charny (acharny)" , "Georgios Karagiannis" , , "Lars Westberg (KI/EAB)" X-OriginalArrivalTime: 02 Nov 2007 18:31:48.0541 (UTC) FILETIME=[A1114ED0:01C81D7E] X-TM-AS-Product-Ver: SMEX-8.0.0.1181-5.000.1023-15520.002 X-TM-AS-Result: No--38.225800-8.000000-2 X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No DKIM-Signature: v=0.5; a=rsa-sha256; q=dns/txt; l=28847; t=1194028330; x=1194892330; c=relaxed/simple; s=sjdkim4002; h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version; d=cisco.com; i=acharny@cisco.com; z=From:=20=22Anna=20Charny=20(acharny)=22=20 |Subject:=20PLEASE=20IGNORE=20RE=3A=20[PCN]=20RE=3A=20Questions=20on=20=2 0LC-PCN=20draft=20version=2001 |Sender:=20; bh=6x+p/Y55mVhJajCHXV6FlK2KIUpoGISQAaiJ5TvM2o4=; b=EfjKL8uFeZI385vlqQ58TXG3vrYmxgvuBSJZPwhBkbVoIffHb5NPnUxrRR9VgKxnV305O3Yy BeYynxqY3Nk3ly1cAeSSFOyvXjOxrdgttcUr/+q0txHVXYfr99TMPZAr; Authentication-Results: sj-dkim-4; header.From=acharny@cisco.com; dkim=pass ( sig from cisco.com/sjdkim4002 verified; ); X-Spam-Score: -4.0 (----) X-Scan-Signature: bc3b4a0e61e6c2546ab80acca7fc79df Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Sorry, this went out unfinished (cat walked on keyboard :-)) PLEASE IGNORE Ann a > -----Original Message----- > From: Anna Charny (acharny)=20 > Sent: Friday, November 02, 2007 2:28 PM > To: Georgios Karagiannis; anurag.bhargava@ericsson.com; Lars=20 > Westberg (KI/EAB) > Cc: pcn@ietf.org > Subject: [PCN] RE: Questions on LC-PCN draft version 01 >=20 > Hi Georgios, >=20 > Still something does not seem right. So let me restate my current > understanding: >=20 > At the internal node: >=20 > Pcn-upper-rate =3D max pcn capacity - termination-offset-rate=20 > Pcn-lower-rate =3D pcn-upper rate - admission-offset rate >=20 > (both offsetas are global values) >=20 > At the egress: >=20 > Pcn-lower-egress-percentage - a configured percentage=20 > Pcn-lower-egress-rate =3D=20 >=20 >=20 > > -----Original Message----- > > From: Georgios Karagiannis [mailto:karagian@cs.utwente.nl] > > Sent: Friday, November 02, 2007 12:20 PM > > To: Anna Charny (acharny); anurag.bhargava@ericsson.com;=20 > Lars Westberg=20 > > (KI/EAB) > > Cc: pcn@ietf.org > > Subject: RE: Questions on LC-PCN draft version 01 > >=20 > > Hi Anna > >=20 > > Thank you very much for your comments! > >=20 > > Please see in line! > >=20 > >=20 > > On 11/1/2007, "Anna Charny (acharny)" wrote: > >=20 > > > > > >Hi Georgios, > > > > > >A few more clarification questions: > > > > > >> -----Original Message----- > > >> From: Georgios Karagiannis [mailto:karagian@cs.utwente.nl] > > >> Sent: Thursday, November 01, 2007 2:35 AM > > >> To: Anna Charny (acharny); anurag.bhargava@ericsson.com; Lars=20 > > >> Westberg (KI/EAB) > > >> Cc: pcn@ietf.org > > >> Subject: Re: Questions on LC-PCN draft version 01 > > >> > > >> Hi Anna > > >> > > >> Thank you very much for your comments! > > >> Before answering to your comments, see in line below, I > > would like to > > >> explain in an abstract way the LC-PCN algorithm. > > >> > > >> * Setting the thresholds at PCN_interior_nodes: > > >> ----------------------------------------------- > > >> In order to calculate the PCN_upper_rate we use two parameters: > > >> Maximum PHB capacity: that is the maximum capacity that can be=20 > > >> supported by a PCN_interior_node > > >> Termination_offset_rate: that is an absolute rate value > > that should > > >> be set equal into all PCN_interior_nodes. Note that this=20 > value is=20 > > >> used by PCN_interior_nodes to calculate their > > PCN_upper_rate and also > > >> during the situation that a PCN_interior_node is in flow > > termination > > >> state and it receives PCN_marked packets. Please see > > pseudo code on > > >> page 20. > > >> This value must be set equal into all PCN_interior_nodes=20 > such that=20 > > >> all these nodes will know when to take into account the incoming=20 > > >> PCN_marked packets and when not. > > >> > > >> The PCN_upper_rate is then found as: > > >> PCN_upper_rate =3D "Maximum PHB capacity" - = Termination_offset_rate > > >> > > > > > >So if you have a link of 10 Mbps and a link of 40 Mbps=20 > (say each is=20 > > >allowed to use all bandwidth for PCN, for simplicity), then > > they both > > >use the same *absolote* value of the termination-offset-rate?. > >=20 > > Georgios: Yes, you are right! This has to do with the importance of=20 > > using the Termination_offset_rate. We will motivate this in the=20 > > following version of the draft. > >=20 > > > So If I > > >wanted to have PCN-upper-rate 20% below my link capacity on > > all links, > > >I would not be able to do so with this approach? A larger=20 > link would=20 > > >have a smaller relative safety margin, then... > >=20 > > Georgios: Yes, this can be considered as a disadvantage.=20 > > However, there might be other ways of preconfiguring each=20 > > PCN_interior_node to know the termination_offset rate used by each=20 > > neighbour PCN_interior_node. Then the pseudocode on page 20=20 > will have=20 > > to use the Termination_offset_rate associated with the=20 > incoming link=20 > > from where the incoming PCN_marked packets are arriving. Then this=20 > > constraint can be avoided. > >=20 > >=20 > > >Same comment for the > > >difference between admission and termination - the relative > > difference > > >between admission and termination threshods (Pcn-lower-rate and > > >pcn-upper-rate) is global, so the larger links have a > > smaller relative > > >difference between admission and termination thresholds. Right? > >=20 > > Georgios: No, because the Admission_offset_rate is an=20 > absolute value! > > So this difference is globally equal. > >=20 > > > > > >> The PCN_lower_rate is configured in all PCN_interior-nodes > > and it can > > >> be calculated in the following way: > > >> PCN_lower_rate =3D PCN_upper_rate - Admission_offset_rate > > >> > > >> The Admission_offset_rate is an absolute rate value and it > > is equal > > >> in all PCN_interior_nodes and PCN_egress_nodes. Note that > > this value > > >> is used by PCN_interior_nodes to calculate their > > PCN_lower_rate and > > >> the PCN_egress_nodes to calculate their PCN_upper_rate_egress.=20 > > >> Furthermore, this value is used by the PCN_interior_nodes > > also during > > >> the situation that a PCN_interior_node is in admission > > control state > > >> and it receives PCN_marked packets. Please see pseudo=20 > code on page=20 > > >> 13. > > >> This value must be set equal into all PCN_interior_nodes=20 > such that=20 > > >> all these nodes will know when to take into account the incoming=20 > > >> PCN_marked packets and when not. Note that a=20 > PCN_interior_node can=20 > > >> PCN_mark packets up to an excess rate equal to the=20 > > >> Admission_offset_rate. If the exess rate in an > > PCN_interior_node is > > >> higher than the Admission_offset_rate, then the=20 > PCN_interior_node=20 > > >> changes state from admission control state to flow > > termination state. > > > > > >OK. > > > > > >> > > >> * Setting the thresholds at PCN_egress_nodes: > > >> --------------------------------------------- > > >> The question is how to calculate the threshold that=20 > defines when a=20 > > >> PCN_egress_node goes into the admission control state. > > >> One way to do that is to consider that when the PCN_egress_node=20 > > >> receives a PCN_marked packet it will mean that at least one=20 > > >> PCN_interior_node started to be admission control congested and=20 > > >> therefore it will go from Normal state to admission > > control state. Of > > >> course this will somehow might provide some errors,=20 > because there=20 > > >> might be situations that this consideration might be to > > conservative.=20 > > >> Therefore, we use a percentage of received PCN_marking encoded=20 > > >> packets in proportion to total rate of received packets. In this=20 > > >> version of the draft we call this value as: > > >> PCN_lower_rate_egress. This is wrong. > > >> What we should say is: > > >> PCN_lower_rate_egress: is equal to=20 > incoming_PCN_marking_rate, when: > > >> (incoming_PCN_marking_rate/measured PHB rate) =3D to a=20 > preconfigured=20 > > >> percentage, say 1%. From now on I denote this percentage as: > > >> PCN_lower_percentage_egress. > > > > > >In the draft, pcn_lower- and -upper_rate_egress seem to be > > defined on a > > >per *ingress-egress-pair basis*. > > >Is that correct? > >=20 > > georgios: Yes, it is correct! > > > > > >> Thus we can say that a PCN_egress_node changes from Normal > > state to > > >> admission control state when > > incoming_PCN_marking_rate/measured PHB > > >> rate > PCN_lower_percentage_egress. > > >> Where, > > >> incoming_PCN_marking_rate =3D N * input_PCN_marking_bytes/T = Where,=20 > > >> input_PCN_marking_bytes =3D received number of=20 > "PCN_marking" encoded=20 > > >> packets during measurement period T. > > > > > >Again, is it per ingress-egress pair, or not? > >=20 > > georgios: Yes, it is correct! > >=20 > > > > > >> > > >> Now we defined the condition that the PCN_egress_node > > changes state > > >> from Normal state to admission control state. But how will the=20 > > >> PCN_egress_node change state from admission control=20 > state to flow=20 > > >> termination state. > > >> In order to explain this, it is imporatnt to note that each=20 > > >> PCN_interior_node that is in admission control state it > > can PCN_mark > > >> packets up to a value equal to Admission_offset_rate.=20 > > Furthermore, if > > >> a PCN_interior_node receives incoming PCN_marked packets=20 > and is in=20 > > >> the addmission control state, it will not remark any > > packets if the > > >> excess rate is equal or lower than the > > incoming_PCN_marking_rate, see > > >> page 13. > > >> Furthermore, if we will consider as normal situations the > > situations > > >> that no ECMP occurs and that all flows belonging to the same=20 > > >> ingress-egress aggregate will use the same path from > > PCN_ingress to > > >> PCN_egress, this will mean that when the PCN_egress_node=20 > receives,=20 > > >> for the given ingress-egress aggregate an excess rate equal to=20 > > >> Admission_offset_rate it will have to change from > > admission control > > >> state to flow termination state. > > >> Thus in this case the second threshold, that in this case > > is a rate > > >> and not a percentage, can be calculated as follows: > > >> PCN_upper_egress_rate =3D PCN_lower_egress_rate + > > Admission_offset_rate. > > > > > > > > >Here is where I am afraid I am fundamentally confused. > > >Pcn_lower-egress_rate seems to have beed redefined above as a=20 > > >percentage threshold, but it is an absolute rate again=20 > here. Perhaps=20 > > >you just mean that Pcn-lower-egress-rate =3D > > pcn_lower_egress_percentage > > >* total-measured-rate-of-this-ingress-egrees-aggregate/100? > > >So pcn-lower-egress-rate then is just the fraction of the > > total rate of > > >the ingress-egress aggregate corresponding to the configured > > percentage > > >threshold. Right? > >=20 > > Georgios: I think I could not explain and translate what I=20 > wanted to=20 > > do in the right way! > > A better way of expressing the need of using a rate of=20 > percentage of=20 > > the received PCN_marking encoded packets, would be the following: > >=20 > > Pcn-lower-egress-rate =3D pcn_lower_egress_percentage *=20 > > Admission_offset_rate > >=20 > >=20 > > This means that event_A in Section 4.2.3 will be activated when=20 > > incoming_PCN_marking_rate > pcn_lower_egress_percentage > > * Admission_offset_rate or when: > > incoming_PCN_marking_rate > Pcn-lower-egress-rate > >=20 > > This means that the PCN_egress_ node does not change to admission=20 > > control state when at least one PCN_marked packet arrives, but it=20 > > changes when the incoming_PHB_marking rate equals a=20 > percentage of the=20 > > Admission_offset_rate. This is due to the fact that=20 > > Admission_offset_rate is used by the interioe nodes to change from=20 > > admission control state to flow termination state. > >=20 > > > > > >Assuming the above is correct, I cannot see how the system=20 > will work=20 > > >correctly. Consider the following example. A bottleneck link of=20 > > >capacity 100 mbps , pcn-termination-offset of 40mbps and=20 > > >pcn-admission-offset of 30 mbps is shared by 100 ingress-egress=20 > > >aggregates, each with the total pcn rate of 1 mbps. On this link=20 > > >pcn-upper-threshold is 60mbps, and pcn-lower-threshold is 30mbps. > > >Suppose further that all 100 ingress-egress aggregates go to > > different > > >egress nodes (and suppose for simplicity there is no other traffic=20 > > >going to these egresses). The bottleneck link is clearly in the=20 > > >termination state, as the total rate of pcn traffic on this > > link is 100 > > >mbps which is above the pcn-upper-threshold. This means we=20 > want the=20 > > >egress nodes to somehow recognise this state and get into the=20 > > >termination mode. But it seems in this example the egresses > > can't ever > > >recognize that the system is in the termination mode. See below. > >=20 > >=20 > > > > > >Suppose pcn-lower-egress-percentage is set to w% (0 > Then, each > > >of these egresses will compute the (absolute) > > pcn-lower-egress-rate as > > >total pcn traffic of the ingress-egress aggregate it sees=20 > (which is 1 > > >mbps) times the pcn_lower_egress_percentage/100, so it will get=20 > > >pcn-lower-egress-rate=3D w* 0.01 Mbps. > > >According to your explanation above, it will then compute=20 > > = >Pcn-upper-egress-rate=3Dpcn-lower-egress-rate+Admission-offset-rate=3D > > >w*0.01 mbps+30mbps >=3D 30 mbps (for all possible settings of w). > > > > > >So our pcn-upper-egress rate is always greater than 30 Mbps, > > while the > > >entire rate of pcn traffic the egress will ever see is *at > > most* 1 Mbps > > >(because it is plainly the total rate of the single ingress-egress=20 > > >aggreate that goes to this egress). So, regardless of the > > actual amount > > >of marked traffic in this egress-egress aggregate, the > > absolute excess > > >rate of this ingress-egress aggregate will never be above > > 1mbps either. > > >In turn that means that the pcn-upper-egress-rate will NEVER be=20 > > >exceeded (because pcn-upper-egress-rate is so much larger than the=20 > > >total rate of the pcn traffic at the egress). In turn, that > > appears to > > >mean that in this scenario none of the egresses will ever=20 > end up in=20 > > >termination state, and so termination will never occur. That > > does not seem right. > >=20 > >=20 > > Georgios: Thank you for the given example, which stimulated me to=20 > > better translate what I wanted to specify and what I have written=20 > > down. > > Please see above that I defned: > > Pcn-lower-egress-rate =3D pcn_lower_egress_percentage *=20 > > Admission_offset_rate > >=20 > > The situation that you described cannot (often) occur because the=20 > > difference between the admission control threshold and flow=20 > > termination threshold at the egress is equal to=20 > Admission_offset_rate=20 > > +/- multicongestion_error. Note that the=20 > Admission_offset_rate is an=20 > > absolute rate value. The multicongestion_error is used to=20 > identify the=20 > > bounds that certain situations occur where the PCN_egreess_ node=20 > > should have been in flow termination state but it is not.=20 > Please see=20 > > previous discussions on this. > >=20 > > > > > >I would appreciate any help in clarifying this. > > > > > >Anna > > > > > > > > >> However, there are some corner cases, that mainly occur when the=20 > > >> different congestion points (admission control congested > > >> PCN_interior_nodes) on the same path are not > > simulataneously starting > > >> to be congested. Therefore we use the > > multicongestion_error parameter > > >> to identify the error bound that ocurs due to these=20 > corner cases.=20 > > >> Note that this error bound can be e.g., predefined ones > > off line by > > >> the operator, by studying the network topology and/or=20 > studying how=20 > > >> often such corner cases could occur and/or doing off line=20 > > >> measurements. Therefore we use: > > >> PCN_upper_rate_egress =3D PCN_lower_rate_egress + > > Admission_offset_rate > > >> +/- multicongestion_error > > >> > > >> * How the states of operation in PCN_interior_nodes are > > being changed? > > >>=20 > >=20 > --------------------------------------------------------------------- > > >> This is explained on page 17, 18, by using Figure 4. > > >> > > >> Change from Normal state to Admission control state: event > > A Occurs > > >> when: > > >> Measured PHB rate > PCN_lower_rate > > >> > > >> Change from Admission control state to Flow Termination > > >> state: event B Occurs when: > > >> Measured PHB rate > PCN_upper_rate > > >> > > >> * How the states of operation in PCN_egress_nodes are > > being changed? > > >>=20 > >=20 > --------------------------------------------------------------------- > > >> This is explained on page 21, 22. Note that the > > description of event > > >> A on page 21 has to be modified to avoid the confusions=20 > that were=20 > > >> caused up to now, see explanation given above. > > >> > > >> Change from Normal state to Admission control state: event > > A Occurs > > >> when: > > >> incoming_PCN_marking_rate/measured PHB rate) >=20 > > >> PCN_lower_percentage_egress As explained above: > > >> IF ((incoming_PCN_marking_rate/measured PHB rate) =3D > > >> PCN_lower_percentage_egress) > > >> THEN PCN_lower_egress_rate =3D incoming_PCN_marking_rate > > >> > > >> Change from Admission control state to flow termination > > >> state: event B Occurs when: > > >> incoming_PCN_marking_rate > PCN_upper_rate_egress > > >> > > >> It is important to note that also the explanation of event > > C on page > > >> 21, has to be modified to avoid the confusions that were > > caused up to > > >> now, see explanation given above: > > >> Change from Admission control state to Normal state: > > >> Occurs when: > > >> incoming_PCN_marking_rate/measured PHB rate) =3D<=20 > > >> PCN_lower_percentage_egress > > >> > > >> > > >> * Generated excess rate by an PCN_interior_node operating in=20 > > >> admission control state. > > >> ------------------------------------------------------------- > > >> > > >> The excess rate =3D signaled_overload_rate. > > >> The maximum excess rate that a PCN_interior_node can=20 > calculate in=20 > > >> admission control state is equal to Admission_offset_rate, see=20 > > >> explanation above. > > >> The number of bytes that are remarked, > > signaled_remarked_bytes depend > > >> on the value of calculated excess rate > > (signaled_overload_rate), the > > >> value of the Admission_offset_rate and the value of the=20 > > >> incoming_PCN_marking_rate, see pseudo code on page 13. > > >> > > >> Note that all packets that are passing through a congested=20 > > >> PCN_interior_node an are not being PCN_marked by the=20 > > >> PCN_interior_node have to be remarked using the > > PCN_Affected_marking. > > >> > > >> > > >> Regarding probes, the probe packets that are passing through a=20 > > >> congested node are either PCN_marked or PCN_Affected_marked. > > >> > > >> * Generating excess rate by an PCN_interior_node=20 > operating in flow=20 > > >> termination state: > > >> ---------------------------------------------------------------- > > >> The excess rate =3D signaled_overload_rate. > > >> Note that the calculation of the signaled_overload_rate is > > different > > >> than in the situation that the PCN_Interior_node operates in=20 > > >> admission control state, see page 19 and 20. This is due > > to the fact > > >> that a sliding window is used to solve an undershooting > > problem, see > > >> discussion on page 19. > > >> The number of bytes that are remarked, > > signaled_remarked_bytes depend > > >> on the value of calculated excess rate > > (signaled_overload_rate), the > > >> value of the Termination_offset_rate and the value of the=20 > > >> incoming_PCN_marking_rate, and the see pseudo code on page 20. > > >> > > >> Note that all packets that are passing through a congested=20 > > >> PCN_interior_node an are not being PCN_marked by the=20 > > >> PCN_interior_node have to be remarked using the > > PCN_Affected_marking. > > >> > > >> * Providing admission control at PCN_egress_nodes: > > >> -------------------------------------------------- > > >> When the PCN_egress_node is operating in admission control > > state than > > >> a flow that is requesting admission into the PCN domain can b=20 > > >> etreated in the following way: > > >> > > >> If no probing is used, the request for admission can be > > accomplished > > >> by using an external to PCN signaling protocol. > > >> In this case when the request arrives at a PCN_egress_node that=20 > > >> operates in admission control state then the request is > > rejected. If > > >> it operates in Normal state is accepted. > > >> > > >> If probing is used, the request for admission is accomplished by=20 > > >> using probe packets. In this case when the probe arrives at a=20 > > >> PCN_egress_node and it is either PCN_marking or > > PCN_Affected_marking > > >> encoded is rejected. Otherwise is accepted. > > >> Note that probes can only be used when > > PCN_Affected_marking is appled > > >> in whole PCN domain. Otherwise, the admission control > > procedure will > > >> work by using e.g., an external signaling protocol used between=20 > > >> PCN_ingress_nodes and PCN_egress_nodes. > > >> > > >> * Providing flow termination at PCN_egress_nodes: > > >> -------------------------------------------------- > > >> > > >> When a PCN_egress_node operates on flow termination it > > calculates the > > >> incoming excess rate (incoming_PCN_marking_rate). > > >> By using the excess rate the PCN_egress_node calclates the > > numer of > > >> flows that have to be terminated using the pseudo code > > given on page > > >> 23. Note that the PCN_egress_node has to maintain per flow=20 > > >> reservation states. > > >> The information contained in the per flow states is used for the=20 > > >> calculation of the flow that have to be terminated, see > > pseudocode on > > >> page 23. Note also that this pseudo code uses priority > > clases, but it > > >> operates when also no priority clases are used by setting=20 > > >> Maximum_priority =3D 0 Where, 0 =3D< priority_class =3D<=20 > Maximum_priority > > >> > > >> > > >> > > >> Below, I am providing some answers to your comments! > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > -----Original Message----- > > >> > From: Anna Charny (acharny) [mailto:acharny@cisco.com] > > >> > Sent: zondag 7 oktober 2007 21:40 > > >> > To: anurag.bhargava@ericsson.com; Georgios Karagiannis; > > >> Lars Westberg > > >> > (KI/EAB) > > >> > Cc: pcn@ietf.org > > >> > Subject: Questions on LC-PCN draft version 01 > > >> > > > >> > Dear authors of the LC-PCN draft, > > >> > > > >> > Am I right that there has been no response yet to Phil's > > questions > > >> > (attached at the end) on the new version of your draft=20 > below (I=20 > > >> > checked the pcn list and it does not seem it went=20 > there)? If the=20 > > >> > response was sent, can you resend it please? > > >> > > > >> > I have many of the same questions, and also a few > > additional ones. > > >> > Here are some of them. > > >> > > > >> > 1) I do not understand whether pcn-lower_rate_egress and=20 > > >> > pcn_upper_rate ingress are expressed as a fraction of the > > >> total rate > > >> > (a unitless entity, analogous to CLE) or an absolute value > > >> (in bits or > > >> > bytes per second). The text seems to be contradictory on > > this point: > > >> > > > >> > on page 8 it is a fraction: > > >> > > > >> > "PCN_lower_rate_egress =3D predefined percentage of received=20 > > >> > PCN_marking", > > >> > > > >> > while on page 13 it is an absolute rate: > > >> > > > >> > "If the incoming_PCN_marking_rate is higher than a=20 > preconfigured=20 > > >> > PCN_lower_rate_egress, ..." where the > > >> > > > >> > (Incoming_PCN_marking_rate is clearly defined as abslolue > > >> rate on page > > >> > 12: "Where the "incoming_PCN_marking_rate" is calculated > > as follows: > > >> > incoming_PCN_marking_rate =3D (received number of "PCN_marking" > > >> > DSCP during T)* N)/T;) > > >> > > >> Georgios: You are right, please see discussion above on paragaph=20 > > >> denoted > > >> as: > > >> "Setting the thresholds at PCN_egress_nodes". > > >> > > >> > > >> We use a percentage of received PCN_marking encoded packets in=20 > > >> proportion to total rate of received packets to define when a=20 > > >> PCN_egres_node goes from Normal state to admission control > > state. In > > >> this version of the draft we call this value as:=20 > > >> PCN_lower_rate_egress. This is wrong. > > >> What we should say is: > > >> PCN_lower_rate_egress: is equal to=20 > incoming_PCN_marking_rate, when: > > >> (incoming_PCN_marking_rate/measured PHB rate) =3D to a=20 > preconfigured=20 > > >> percentage, say 1%. From now on I denote this percentage as: > > >> PCN_lower_percentage_egress. > > >> Thus we can say that a PCN_egress_node changes from Normal > > state to > > >> admission control state when > > incoming_PCN_marking_rate/measured PHB > > >> rate > PCN_lower_percentage_egress. > > >> Where, > > >> incoming_PCN_marking_rate =3D N * input_PCN_marking_bytes/T = Where,=20 > > >> input_PCN_marking_bytes =3D received number of=20 > "PCN_marking" encoded=20 > > >> packets during measurement period T. > > >> > > >> > > >> > > > >> > 2) Regarding the question above, > > >> > > > >> > * if pcn_lower_rate_eggress (and pcn_uppre-rate_egress) are > > >> absolute > > >> > rates, then how do you choose that abslolute rate value, > > given that > > >> > the rates of ingress-egress aggregates at the same > > ingress may be > > >> > vastly different in magnitude? > > >> > > >> Georgios: See explanation above! Hopefully this > > misundersatnding is > > >> also solved! > > >> > > >> > > > >> > *if, however, pcn_lower_rate_eggress (and > > >> > pcn_uppre-rate_egress) are ratios, then what is the=20 > guidance of=20 > > >> > setting these ratios so that the egress can always tell > > admission > > >> > state from the termination state, given that the same > > >> marking is used > > >> > for admission and termination marking? > > >> > Specifically, suppose at the bottleneck pcn_lower_threshold > > >> is set to > > >> > X, and pcn_upper_threshold to aX with some a>1 (assume > > for example > > >> > a=3D1.5). > > >> > How does the egress node tell between the conditions > > when the total > > >> > pcn traffic on the bottleneck is 1.1X (which is the state wnen=20 > > >> > admission is needed but termination is not, - in this case > > >> > (1.1X-X)/X=3D0.1 traffic is pcn-marked), and the case when > > the total > > >> > traffic on the bottleneck is 1.1aX, which is the case when > > >> termination > > >> > is needed, and (1.1ax-ax)/ax=3D0.1), if in both cases the > > >> ratio between > > >> > marked and total traffic is the same value 0.1? > > >> > > >> Georgios: In order to solve the above described problems, the=20 > > >> thresholds used in the PCN_interior_nodes and=20 > PCN_egress_nodes are > > >> using: > > >> the Admission_offset_rate that is an absolute rate value > > which is set > > >> equal into whole PCN domain. > > >> > > >> Note that the maximum excess rate that a PCN_interior_node can=20 > > >> calculate in admission control state is equal to=20 > > >> Admission_offset_rate. If the excess rate is higher than the=20 > > >> Admission_offset_rate then the node changes from=20 > admission control=20 > > >> state to flow termination state. > > >> > > >> Please see all details that I have provided at the top of > > this email.=20 > > >> In particular, check the paragraphs denoted above as: > > >> "Setting the thresholds at PCN_interior_nodes" > > >> "Setting the thresholds at PCN_egress_nodes" > > >> "Generated excess rate by an PCN_interior_node operating > > in admission > > >> control state" > > >> > > >> > > >> > > >> > > > >> > 3) On page 9, multicongestion error error is introduced, > > >> but the text > > >> > is silent on how this error is defined and what is it set > > >> to (if it is > > >> > a configuration parameter), or how it is measured (if it is > > >> something > > >> > that is being measured). Can you explain, please? > > >> > > >> Georgios: > > >> Please see the paragraph denoted above as "Setting the > > thresholds at > > >> PCN_egress_nodes" > > >> > > >> > > >> Best regards, > > >> Georgios > > >> > > >> > > >> > > > >> > I have some more questions, including sharing those asked > > >> by Phil in > > >> > the attached message), but I will stop now, as I hope > > >> clarifications > > >> > on the above (and Phil's) questions will help understand > > the draft > > >> > better... > > >> > > > >> > Thank you in advance for clarifying all this. > > >> > > > >> > Anna > > >> > >=20 >=20 >=20 > _______________________________________________ > PCN mailing list > PCN@ietf.org > https://www1.ietf.org/mailman/listinfo/pcn >=20 _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Fri Nov 02 14:36:35 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Io1Nb-0005xI-Kj; Fri, 02 Nov 2007 14:36:27 -0400 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Io1NZ-0005wg-TC for pcn-confirm+ok@megatron.ietf.org; Fri, 02 Nov 2007 14:36:25 -0400 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Io1NZ-0005wY-JP for pcn@ietf.org; Fri, 02 Nov 2007 14:36:25 -0400 Received: from rotterdam.ewi.utwente.nl ([130.89.10.5]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1Io1NY-00013a-4F for pcn@ietf.org; Fri, 02 Nov 2007 14:36:25 -0400 Received: from webmail.cs.utwente.nl (janus.ewi.utwente.nl [130.89.10.26]) by rotterdam.ewi.utwente.nl (8.13.6/8.13.6) with SMTP id lA2IaITE007670; Fri, 2 Nov 2007 19:36:18 +0100 (MET) Received: from 193.192.232.84 (auth. user karagian@imap1.ewi.utwente.nl) by webmail.cs.utwente.nl with HTTP; Fri, 02 Nov 2007 18:36:17 +0000 To: "Anna Charny (acharny)" , "anurag.bhargava@ericsson.com" , "Lars Westberg (KI/EAB)" Date: Fri, 02 Nov 2007 18:36:15 +0000 X-Mailer: IlohaMail/0.8.13 (On: webmail.cs.utwente.nl) Message-ID: In-Reply-To: From: "Georgios Karagiannis" Bounce-To: "Georgios Karagiannis" MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0 () X-Scanned-By: MIMEDefang 2.52 on 130.89.10.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0rc3 (rotterdam.ewi.utwente.nl [130.89.10.5]); Fri, 02 Nov 2007 19:36:21 +0100 (MET) X-Spam-Score: 0.0 (/) X-Scan-Signature: aafd3813f49c1dfb11e9623a3ab5d812 Cc: "pcn@ietf.org" Subject: [PCN] RE: Questions on LC-PCN draft version 01 X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Anna Please see in line On 11/2/2007, "Anna Charny (acharny)" wrote: >Hi Georgios, > >Still something does not seem right. So let me restate my current >understanding: > >At the internal node: > >Pcn-upper-rate =3D max pcn capacity - termination-offset-rate >Pcn-lower-rate =3D pcn-upper rate - admission-offset rate > >(both offsetas are global values) Georgios: Yes, but the termination_offset_rate could be released from this constraint if each PCN_interior_node would be configured to know the Termination_offset_rate of each neighbour interior node. > >At the egress: > >Pcn-lower-egress-percentage - a configured percentage >Pcn-lower-egress-rate =3D Pcn-lower-egress-percentage * Admission_offset_ra= te PCN_upper_egress_rate=3D Pcn-lower-egress-rate + admission_offset_rate +/- multicongestion_error Best regards, Georgios > > >> -----Original Message----- >> From: Georgios Karagiannis [mailto:karagian@cs.utwente.nl]=20 >> Sent: Friday, November 02, 2007 12:20 PM >> To: Anna Charny (acharny); anurag.bhargava@ericsson.com; Lars=20 >> Westberg (KI/EAB) >> Cc: pcn@ietf.org >> Subject: RE: Questions on LC-PCN draft version 01 >>=20 >> Hi Anna >>=20 >> Thank you very much for your comments! >>=20 >> Please see in line! >>=20 >>=20 >> On 11/1/2007, "Anna Charny (acharny)" wrote: >>=20 >> > >> >Hi Georgios, >> > >> >A few more clarification questions: >> > >> >> -----Original Message----- >> >> From: Georgios Karagiannis [mailto:karagian@cs.utwente.nl] >> >> Sent: Thursday, November 01, 2007 2:35 AM >> >> To: Anna Charny (acharny); anurag.bhargava@ericsson.com; Lars=20 >> >> Westberg (KI/EAB) >> >> Cc: pcn@ietf.org >> >> Subject: Re: Questions on LC-PCN draft version 01 >> >> >> >> Hi Anna >> >> >> >> Thank you very much for your comments! >> >> Before answering to your comments, see in line below, I=20 >> would like to=20 >> >> explain in an abstract way the LC-PCN algorithm. >> >> >> >> * Setting the thresholds at PCN_interior_nodes: >> >> ----------------------------------------------- >> >> In order to calculate the PCN_upper_rate we use two parameters: >> >> Maximum PHB capacity: that is the maximum capacity that can be=20 >> >> supported by a PCN_interior_node >> >> Termination_offset_rate: that is an absolute rate value=20 >> that should=20 >> >> be set equal into all PCN_interior_nodes. Note that this value is=20 >> >> used by PCN_interior_nodes to calculate their=20 >> PCN_upper_rate and also=20 >> >> during the situation that a PCN_interior_node is in flow=20 >> termination=20 >> >> state and it receives PCN_marked packets. Please see=20 >> pseudo code on=20 >> >> page 20. >> >> This value must be set equal into all PCN_interior_nodes such that=20 >> >> all these nodes will know when to take into account the incoming=20 >> >> PCN_marked packets and when not. >> >> >> >> The PCN_upper_rate is then found as: >> >> PCN_upper_rate =3D "Maximum PHB capacity" - Termination_offset_rate >> >> >> > >> >So if you have a link of 10 Mbps and a link of 40 Mbps (say each is=20 >> >allowed to use all bandwidth for PCN, for simplicity), then=20 >> they both=20 >> >use the same *absolote* value of the termination-offset-rate?. >>=20 >> Georgios: Yes, you are right! This has to do with the=20 >> importance of using the Termination_offset_rate. We will=20 >> motivate this in the following version of the draft. >>=20 >> > So If I >> >wanted to have PCN-upper-rate 20% below my link capacity on=20 >> all links,=20 >> >I would not be able to do so with this approach? A larger link would=20 >> >have a smaller relative safety margin, then... >>=20 >> Georgios: Yes, this can be considered as a disadvantage.=20 >> However, there might be other ways of preconfiguring each=20 >> PCN_interior_node to know the termination_offset rate used by=20 >> each neighbour PCN_interior_node. Then the pseudocode on page=20 >> 20 will have to use the Termination_offset_rate associated=20 >> with the incoming link from where the incoming PCN_marked=20 >> packets are arriving. Then this constraint can be avoided. >>=20 >>=20 >> >Same comment for the >> >difference between admission and termination - the relative=20 >> difference=20 >> >between admission and termination threshods (Pcn-lower-rate and >> >pcn-upper-rate) is global, so the larger links have a=20 >> smaller relative=20 >> >difference between admission and termination thresholds. Right? >>=20 >> Georgios: No, because the Admission_offset_rate is an absolute value! >> So this difference is globally equal. >>=20 >> > >> >> The PCN_lower_rate is configured in all PCN_interior-nodes=20 >> and it can=20 >> >> be calculated in the following way: >> >> PCN_lower_rate =3D PCN_upper_rate - Admission_offset_rate >> >> >> >> The Admission_offset_rate is an absolute rate value and it=20 >> is equal=20 >> >> in all PCN_interior_nodes and PCN_egress_nodes. Note that=20 >> this value=20 >> >> is used by PCN_interior_nodes to calculate their=20 >> PCN_lower_rate and=20 >> >> the PCN_egress_nodes to calculate their PCN_upper_rate_egress.=20 >> >> Furthermore, this value is used by the PCN_interior_nodes=20 >> also during=20 >> >> the situation that a PCN_interior_node is in admission=20 >> control state=20 >> >> and it receives PCN_marked packets. Please see pseudo code on page=20 >> >> 13. >> >> This value must be set equal into all PCN_interior_nodes such that=20 >> >> all these nodes will know when to take into account the incoming=20 >> >> PCN_marked packets and when not. Note that a PCN_interior_node can=20 >> >> PCN_mark packets up to an excess rate equal to the=20 >> >> Admission_offset_rate. If the exess rate in an=20 >> PCN_interior_node is=20 >> >> higher than the Admission_offset_rate, then the PCN_interior_node=20 >> >> changes state from admission control state to flow=20 >> termination state. >> > >> >OK. >> > >> >> >> >> * Setting the thresholds at PCN_egress_nodes: >> >> --------------------------------------------- >> >> The question is how to calculate the threshold that defines when a=20 >> >> PCN_egress_node goes into the admission control state. >> >> One way to do that is to consider that when the PCN_egress_node=20 >> >> receives a PCN_marked packet it will mean that at least one=20 >> >> PCN_interior_node started to be admission control congested and=20 >> >> therefore it will go from Normal state to admission=20 >> control state. Of=20 >> >> course this will somehow might provide some errors, because there=20 >> >> might be situations that this consideration might be to=20 >> conservative.=20 >> >> Therefore, we use a percentage of received PCN_marking encoded=20 >> >> packets in proportion to total rate of received packets. In this=20 >> >> version of the draft we call this value as: >> >> PCN_lower_rate_egress. This is wrong. >> >> What we should say is: >> >> PCN_lower_rate_egress: is equal to incoming_PCN_marking_rate, when: >> >> (incoming_PCN_marking_rate/measured PHB rate) =3D to a preconfigured=20 >> >> percentage, say 1%. From now on I denote this percentage as: >> >> PCN_lower_percentage_egress. >> > >> >In the draft, pcn_lower- and -upper_rate_egress seem to be=20 >> defined on a=20 >> >per *ingress-egress-pair basis*. >> >Is that correct? >>=20 >> georgios: Yes, it is correct! >> > >> >> Thus we can say that a PCN_egress_node changes from Normal=20 >> state to=20 >> >> admission control state when=20 >> incoming_PCN_marking_rate/measured PHB=20 >> >> rate > PCN_lower_percentage_egress. >> >> Where, >> >> incoming_PCN_marking_rate =3D N * input_PCN_marking_bytes/T Where,=20 >> >> input_PCN_marking_bytes =3D received number of "PCN_marking" encoded=20 >> >> packets during measurement period T. >> > >> >Again, is it per ingress-egress pair, or not? >>=20 >> georgios: Yes, it is correct! >>=20 >> > >> >> >> >> Now we defined the condition that the PCN_egress_node=20 >> changes state=20 >> >> from Normal state to admission control state. But how will the=20 >> >> PCN_egress_node change state from admission control state to flow=20 >> >> termination state. >> >> In order to explain this, it is imporatnt to note that each=20 >> >> PCN_interior_node that is in admission control state it=20 >> can PCN_mark=20 >> >> packets up to a value equal to Admission_offset_rate.=20 >> Furthermore, if=20 >> >> a PCN_interior_node receives incoming PCN_marked packets and is in=20 >> >> the addmission control state, it will not remark any=20 >> packets if the=20 >> >> excess rate is equal or lower than the=20 >> incoming_PCN_marking_rate, see=20 >> >> page 13. >> >> Furthermore, if we will consider as normal situations the=20 >> situations=20 >> >> that no ECMP occurs and that all flows belonging to the same=20 >> >> ingress-egress aggregate will use the same path from=20 >> PCN_ingress to=20 >> >> PCN_egress, this will mean that when the PCN_egress_node receives,=20 >> >> for the given ingress-egress aggregate an excess rate equal to=20 >> >> Admission_offset_rate it will have to change from=20 >> admission control=20 >> >> state to flow termination state. >> >> Thus in this case the second threshold, that in this case=20 >> is a rate=20 >> >> and not a percentage, can be calculated as follows: >> >> PCN_upper_egress_rate =3D PCN_lower_egress_rate +=20 >> Admission_offset_rate. >> > >> > >> >Here is where I am afraid I am fundamentally confused. >> >Pcn_lower-egress_rate seems to have beed redefined above as a=20 >> >percentage threshold, but it is an absolute rate again here. Perhaps=20 >> >you just mean that Pcn-lower-egress-rate =3D=20 >> pcn_lower_egress_percentage=20 >> >* total-measured-rate-of-this-ingress-egrees-aggregate/100? >> >So pcn-lower-egress-rate then is just the fraction of the=20 >> total rate of=20 >> >the ingress-egress aggregate corresponding to the configured=20 >> percentage=20 >> >threshold. Right? >>=20 >> Georgios: I think I could not explain and translate what I=20 >> wanted to do in the right way! >> A better way of expressing the need of using a rate of=20 >> percentage of the received PCN_marking encoded packets, would=20 >> be the following: >>=20 >> Pcn-lower-egress-rate =3D pcn_lower_egress_percentage *=20 >> Admission_offset_rate >>=20 >>=20 >> This means that event_A in Section 4.2.3 will be activated=20 >> when incoming_PCN_marking_rate > pcn_lower_egress_percentage=20 >> * Admission_offset_rate or when: >> incoming_PCN_marking_rate > Pcn-lower-egress-rate >>=20 >> This means that the PCN_egress_ node does not change to=20 >> admission control state when at least one PCN_marked packet=20 >> arrives, but it changes when the incoming_PHB_marking rate=20 >> equals a percentage of the Admission_offset_rate. This is due=20 >> to the fact that Admission_offset_rate is used by the=20 >> interioe nodes to change from admission control state to flow=20 >> termination state. >>=20 >> > >> >Assuming the above is correct, I cannot see how the system will work=20 >> >correctly. Consider the following example. A bottleneck link of=20 >> >capacity 100 mbps , pcn-termination-offset of 40mbps and=20 >> >pcn-admission-offset of 30 mbps is shared by 100 ingress-egress=20 >> >aggregates, each with the total pcn rate of 1 mbps. On this link=20 >> >pcn-upper-threshold is 60mbps, and pcn-lower-threshold is 30mbps. >> >Suppose further that all 100 ingress-egress aggregates go to=20 >> different=20 >> >egress nodes (and suppose for simplicity there is no other traffic=20 >> >going to these egresses). The bottleneck link is clearly in the=20 >> >termination state, as the total rate of pcn traffic on this=20 >> link is 100=20 >> >mbps which is above the pcn-upper-threshold. This means we want the=20 >> >egress nodes to somehow recognise this state and get into the=20 >> >termination mode. But it seems in this example the egresses=20 >> can't ever=20 >> >recognize that the system is in the termination mode. See below. >>=20 >>=20 >> > >> >Suppose pcn-lower-egress-percentage is set to w% (0> Then, each=20 >> >of these egresses will compute the (absolute)=20 >> pcn-lower-egress-rate as=20 >> >total pcn traffic of the ingress-egress aggregate it sees (which is 1 >> >mbps) times the pcn_lower_egress_percentage/100, so it will get=20 >> >pcn-lower-egress-rate=3D w* 0.01 Mbps. >> >According to your explanation above, it will then compute=20 >> >Pcn-upper-egress-rate=3Dpcn-lower-egress-rate+Admission-offset-rate=3D >> >w*0.01 mbps+30mbps >=3D 30 mbps (for all possible settings of w). >> > >> >So our pcn-upper-egress rate is always greater than 30 Mbps,=20 >> while the=20 >> >entire rate of pcn traffic the egress will ever see is *at=20 >> most* 1 Mbps=20 >> >(because it is plainly the total rate of the single ingress-egress=20 >> >aggreate that goes to this egress). So, regardless of the=20 >> actual amount=20 >> >of marked traffic in this egress-egress aggregate, the=20 >> absolute excess=20 >> >rate of this ingress-egress aggregate will never be above=20 >> 1mbps either. >> >In turn that means that the pcn-upper-egress-rate will NEVER be=20 >> >exceeded (because pcn-upper-egress-rate is so much larger than the=20 >> >total rate of the pcn traffic at the egress). In turn, that=20 >> appears to=20 >> >mean that in this scenario none of the egresses will ever end up in=20 >> >termination state, and so termination will never occur. That=20 >> does not seem right. >>=20 >>=20 >> Georgios: Thank you for the given example, which stimulated=20 >> me to better translate what I wanted to specify and what I=20 >> have written down. >> Please see above that I defned: >> Pcn-lower-egress-rate =3D pcn_lower_egress_percentage *=20 >> Admission_offset_rate >>=20 >> The situation that you described cannot (often) occur because=20 >> the difference between the admission control threshold and=20 >> flow termination threshold at the egress is equal to=20 >> Admission_offset_rate +/- multicongestion_error. Note that=20 >> the Admission_offset_rate is an absolute rate value. The=20 >> multicongestion_error is used to identify the bounds that=20 >> certain situations occur where the PCN_egreess_ node should=20 >> have been in flow termination state but it is not. Please see=20 >> previous discussions on this. >>=20 >> > >> >I would appreciate any help in clarifying this. >> > >> >Anna >> > >> > >> >> However, there are some corner cases, that mainly occur when the=20 >> >> different congestion points (admission control congested >> >> PCN_interior_nodes) on the same path are not=20 >> simulataneously starting=20 >> >> to be congested. Therefore we use the=20 >> multicongestion_error parameter=20 >> >> to identify the error bound that ocurs due to these corner cases.=20 >> >> Note that this error bound can be e.g., predefined ones=20 >> off line by=20 >> >> the operator, by studying the network topology and/or studying how=20 >> >> often such corner cases could occur and/or doing off line=20 >> >> measurements. Therefore we use: >> >> PCN_upper_rate_egress =3D PCN_lower_rate_egress +=20 >> Admission_offset_rate=20 >> >> +/- multicongestion_error >> >> >> >> * How the states of operation in PCN_interior_nodes are=20 >> being changed? >> >>=20 >> --------------------------------------------------------------------- >> >> This is explained on page 17, 18, by using Figure 4. >> >> >> >> Change from Normal state to Admission control state: event=20 >> A Occurs=20 >> >> when: >> >> Measured PHB rate > PCN_lower_rate >> >> >> >> Change from Admission control state to Flow Termination >> >> state: event B Occurs when: >> >> Measured PHB rate > PCN_upper_rate >> >> >> >> * How the states of operation in PCN_egress_nodes are=20 >> being changed? >> >>=20 >> --------------------------------------------------------------------- >> >> This is explained on page 21, 22. Note that the=20 >> description of event=20 >> >> A on page 21 has to be modified to avoid the confusions that were=20 >> >> caused up to now, see explanation given above. >> >> >> >> Change from Normal state to Admission control state: event=20 >> A Occurs=20 >> >> when: >> >> incoming_PCN_marking_rate/measured PHB rate) >=20 >> >> PCN_lower_percentage_egress As explained above: >> >> IF ((incoming_PCN_marking_rate/measured PHB rate) =3D >> >> PCN_lower_percentage_egress) >> >> THEN PCN_lower_egress_rate =3D incoming_PCN_marking_rate >> >> >> >> Change from Admission control state to flow termination >> >> state: event B Occurs when: >> >> incoming_PCN_marking_rate > PCN_upper_rate_egress >> >> >> >> It is important to note that also the explanation of event=20 >> C on page=20 >> >> 21, has to be modified to avoid the confusions that were=20 >> caused up to=20 >> >> now, see explanation given above: >> >> Change from Admission control state to Normal state: >> >> Occurs when: >> >> incoming_PCN_marking_rate/measured PHB rate) =3D<=20 >> >> PCN_lower_percentage_egress >> >> >> >> >> >> * Generated excess rate by an PCN_interior_node operating in=20 >> >> admission control state. >> >> ------------------------------------------------------------- >> >> >> >> The excess rate =3D signaled_overload_rate. >> >> The maximum excess rate that a PCN_interior_node can calculate in=20 >> >> admission control state is equal to Admission_offset_rate, see=20 >> >> explanation above. >> >> The number of bytes that are remarked,=20 >> signaled_remarked_bytes depend=20 >> >> on the value of calculated excess rate=20 >> (signaled_overload_rate), the=20 >> >> value of the Admission_offset_rate and the value of the=20 >> >> incoming_PCN_marking_rate, see pseudo code on page 13. >> >> >> >> Note that all packets that are passing through a congested=20 >> >> PCN_interior_node an are not being PCN_marked by the=20 >> >> PCN_interior_node have to be remarked using the=20 >> PCN_Affected_marking. >> >> >> >> >> >> Regarding probes, the probe packets that are passing through a=20 >> >> congested node are either PCN_marked or PCN_Affected_marked. >> >> >> >> * Generating excess rate by an PCN_interior_node operating in flow=20 >> >> termination state: >> >> ---------------------------------------------------------------- >> >> The excess rate =3D signaled_overload_rate. >> >> Note that the calculation of the signaled_overload_rate is=20 >> different=20 >> >> than in the situation that the PCN_Interior_node operates in=20 >> >> admission control state, see page 19 and 20. This is due=20 >> to the fact=20 >> >> that a sliding window is used to solve an undershooting=20 >> problem, see=20 >> >> discussion on page 19. >> >> The number of bytes that are remarked,=20 >> signaled_remarked_bytes depend=20 >> >> on the value of calculated excess rate=20 >> (signaled_overload_rate), the=20 >> >> value of the Termination_offset_rate and the value of the=20 >> >> incoming_PCN_marking_rate, and the see pseudo code on page 20. >> >> >> >> Note that all packets that are passing through a congested=20 >> >> PCN_interior_node an are not being PCN_marked by the=20 >> >> PCN_interior_node have to be remarked using the=20 >> PCN_Affected_marking. >> >> >> >> * Providing admission control at PCN_egress_nodes: >> >> -------------------------------------------------- >> >> When the PCN_egress_node is operating in admission control=20 >> state than=20 >> >> a flow that is requesting admission into the PCN domain can b=20 >> >> etreated in the following way: >> >> >> >> If no probing is used, the request for admission can be=20 >> accomplished=20 >> >> by using an external to PCN signaling protocol. >> >> In this case when the request arrives at a PCN_egress_node that=20 >> >> operates in admission control state then the request is=20 >> rejected. If=20 >> >> it operates in Normal state is accepted. >> >> >> >> If probing is used, the request for admission is accomplished by=20 >> >> using probe packets. In this case when the probe arrives at a=20 >> >> PCN_egress_node and it is either PCN_marking or=20 >> PCN_Affected_marking=20 >> >> encoded is rejected. Otherwise is accepted. >> >> Note that probes can only be used when=20 >> PCN_Affected_marking is appled=20 >> >> in whole PCN domain. Otherwise, the admission control=20 >> procedure will=20 >> >> work by using e.g., an external signaling protocol used between=20 >> >> PCN_ingress_nodes and PCN_egress_nodes. >> >> >> >> * Providing flow termination at PCN_egress_nodes: >> >> -------------------------------------------------- >> >> >> >> When a PCN_egress_node operates on flow termination it=20 >> calculates the=20 >> >> incoming excess rate (incoming_PCN_marking_rate). >> >> By using the excess rate the PCN_egress_node calclates the=20 >> numer of=20 >> >> flows that have to be terminated using the pseudo code=20 >> given on page=20 >> >> 23. Note that the PCN_egress_node has to maintain per flow=20 >> >> reservation states. >> >> The information contained in the per flow states is used for the=20 >> >> calculation of the flow that have to be terminated, see=20 >> pseudocode on=20 >> >> page 23. Note also that this pseudo code uses priority=20 >> clases, but it=20 >> >> operates when also no priority clases are used by setting=20 >> >> Maximum_priority =3D 0 Where, 0 =3D< priority_class =3D< Maximum_priori= ty >> >> >> >> >> >> >> >> Below, I am providing some answers to your comments! >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> > -----Original Message----- >> >> > From: Anna Charny (acharny) [mailto:acharny@cisco.com] >> >> > Sent: zondag 7 oktober 2007 21:40 >> >> > To: anurag.bhargava@ericsson.com; Georgios Karagiannis; >> >> Lars Westberg >> >> > (KI/EAB) >> >> > Cc: pcn@ietf.org >> >> > Subject: Questions on LC-PCN draft version 01 >> >> > >> >> > Dear authors of the LC-PCN draft, >> >> > >> >> > Am I right that there has been no response yet to Phil's=20 >> questions=20 >> >> > (attached at the end) on the new version of your draft below (I=20 >> >> > checked the pcn list and it does not seem it went there)? If the=20 >> >> > response was sent, can you resend it please? >> >> > >> >> > I have many of the same questions, and also a few=20 >> additional ones. >> >> > Here are some of them. >> >> > >> >> > 1) I do not understand whether pcn-lower_rate_egress and=20 >> >> > pcn_upper_rate ingress are expressed as a fraction of the >> >> total rate >> >> > (a unitless entity, analogous to CLE) or an absolute value >> >> (in bits or >> >> > bytes per second). The text seems to be contradictory on=20 >> this point: >> >> > >> >> > on page 8 it is a fraction: >> >> > >> >> > "PCN_lower_rate_egress =3D predefined percentage of received=20 >> >> > PCN_marking", >> >> > >> >> > while on page 13 it is an absolute rate: >> >> > >> >> > "If the incoming_PCN_marking_rate is higher than a preconfigured=20 >> >> > PCN_lower_rate_egress, ..." where the >> >> > >> >> > (Incoming_PCN_marking_rate is clearly defined as abslolue >> >> rate on page >> >> > 12: "Where the "incoming_PCN_marking_rate" is calculated=20 >> as follows: >> >> > incoming_PCN_marking_rate =3D (received number of "PCN_marking" >> >> > DSCP during T)* N)/T;) >> >> >> >> Georgios: You are right, please see discussion above on paragaph=20 >> >> denoted >> >> as: >> >> "Setting the thresholds at PCN_egress_nodes". >> >> >> >> >> >> We use a percentage of received PCN_marking encoded packets in=20 >> >> proportion to total rate of received packets to define when a=20 >> >> PCN_egres_node goes from Normal state to admission control=20 >> state. In=20 >> >> this version of the draft we call this value as:=20 >> >> PCN_lower_rate_egress. This is wrong. >> >> What we should say is: >> >> PCN_lower_rate_egress: is equal to incoming_PCN_marking_rate, when: >> >> (incoming_PCN_marking_rate/measured PHB rate) =3D to a preconfigured=20 >> >> percentage, say 1%. From now on I denote this percentage as: >> >> PCN_lower_percentage_egress. >> >> Thus we can say that a PCN_egress_node changes from Normal=20 >> state to=20 >> >> admission control state when=20 >> incoming_PCN_marking_rate/measured PHB=20 >> >> rate > PCN_lower_percentage_egress. >> >> Where, >> >> incoming_PCN_marking_rate =3D N * input_PCN_marking_bytes/T Where,=20 >> >> input_PCN_marking_bytes =3D received number of "PCN_marking" encoded=20 >> >> packets during measurement period T. >> >> >> >> >> >> > >> >> > 2) Regarding the question above, >> >> > >> >> > * if pcn_lower_rate_eggress (and pcn_uppre-rate_egress) are >> >> absolute >> >> > rates, then how do you choose that abslolute rate value,=20 >> given that=20 >> >> > the rates of ingress-egress aggregates at the same=20 >> ingress may be=20 >> >> > vastly different in magnitude? >> >> >> >> Georgios: See explanation above! Hopefully this=20 >> misundersatnding is=20 >> >> also solved! >> >> >> >> > >> >> > *if, however, pcn_lower_rate_eggress (and >> >> > pcn_uppre-rate_egress) are ratios, then what is the guidance of=20 >> >> > setting these ratios so that the egress can always tell=20 >> admission=20 >> >> > state from the termination state, given that the same >> >> marking is used >> >> > for admission and termination marking? >> >> > Specifically, suppose at the bottleneck pcn_lower_threshold >> >> is set to >> >> > X, and pcn_upper_threshold to aX with some a>1 (assume=20 >> for example=20 >> >> > a=3D1.5). >> >> > How does the egress node tell between the conditions=20 >> when the total=20 >> >> > pcn traffic on the bottleneck is 1.1X (which is the state wnen=20 >> >> > admission is needed but termination is not, - in this case >> >> > (1.1X-X)/X=3D0.1 traffic is pcn-marked), and the case when=20 >> the total=20 >> >> > traffic on the bottleneck is 1.1aX, which is the case when >> >> termination >> >> > is needed, and (1.1ax-ax)/ax=3D0.1), if in both cases the >> >> ratio between >> >> > marked and total traffic is the same value 0.1? >> >> >> >> Georgios: In order to solve the above described problems, the=20 >> >> thresholds used in the PCN_interior_nodes and PCN_egress_nodes are >> >> using: >> >> the Admission_offset_rate that is an absolute rate value=20 >> which is set=20 >> >> equal into whole PCN domain. >> >> >> >> Note that the maximum excess rate that a PCN_interior_node can=20 >> >> calculate in admission control state is equal to=20 >> >> Admission_offset_rate. If the excess rate is higher than the=20 >> >> Admission_offset_rate then the node changes from admission control=20 >> >> state to flow termination state. >> >> >> >> Please see all details that I have provided at the top of=20 >> this email.=20 >> >> In particular, check the paragraphs denoted above as: >> >> "Setting the thresholds at PCN_interior_nodes" >> >> "Setting the thresholds at PCN_egress_nodes" >> >> "Generated excess rate by an PCN_interior_node operating=20 >> in admission=20 >> >> control state" >> >> >> >> >> >> >> >> > >> >> > 3) On page 9, multicongestion error error is introduced, >> >> but the text >> >> > is silent on how this error is defined and what is it set >> >> to (if it is >> >> > a configuration parameter), or how it is measured (if it is >> >> something >> >> > that is being measured). Can you explain, please? >> >> >> >> Georgios: >> >> Please see the paragraph denoted above as "Setting the=20 >> thresholds at=20 >> >> PCN_egress_nodes" >> >> >> >> >> >> Best regards, >> >> Georgios >> >> >> >> >> >> > >> >> > I have some more questions, including sharing those asked >> >> by Phil in >> >> > the attached message), but I will stop now, as I hope >> >> clarifications >> >> > on the above (and Phil's) questions will help understand=20 >> the draft=20 >> >> > better... >> >> > >> >> > Thank you in advance for clarifying all this. >> >> > >> >> > Anna >> >> >>=20 _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Fri Nov 02 15:25:08 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Io28G-0006wN-NJ; Fri, 02 Nov 2007 15:24:40 -0400 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Io28F-0006vh-Da for pcn-confirm+ok@megatron.ietf.org; Fri, 02 Nov 2007 15:24:39 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Io28E-0006vU-Vb for pcn@ietf.org; Fri, 02 Nov 2007 15:24:39 -0400 Received: from sj-iport-2-in.cisco.com ([171.71.176.71] helo=sj-iport-2.cisco.com) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1Io28B-0003Go-PQ for pcn@ietf.org; Fri, 02 Nov 2007 15:24:38 -0400 X-IronPort-AV: E=Sophos;i="4.21,363,1188802800"; d="scan'208";a="412912890" Received: from rtp-dkim-2.cisco.com ([64.102.121.159]) by sj-iport-2.cisco.com with ESMTP; 02 Nov 2007 12:24:34 -0700 Received: from rtp-core-2.cisco.com (rtp-core-2.cisco.com [64.102.124.13]) by rtp-dkim-2.cisco.com (8.12.11/8.12.11) with ESMTP id lA2JOYi6009961; Fri, 2 Nov 2007 15:24:34 -0400 Received: from xbh-rtp-201.amer.cisco.com (xbh-rtp-201.cisco.com [64.102.31.12]) by rtp-core-2.cisco.com (8.12.10/8.12.6) with ESMTP id lA2JOXfo001863; Fri, 2 Nov 2007 19:24:34 GMT Received: from xmb-rtp-203.amer.cisco.com ([64.102.31.20]) by xbh-rtp-201.amer.cisco.com with Microsoft SMTPSVC(6.0.3790.1830); Fri, 2 Nov 2007 15:24:33 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Fri, 2 Nov 2007 15:24:32 -0400 Message-ID: In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Questions on LC-PCN draft version 01 Thread-Index: AcgdbFJQYoieAMH+Tsy5xnVOydp6mAAEdO9Q From: "Anna Charny (acharny)" To: "Georgios Karagiannis" , "Lars Westberg (KI/EAB)" , X-OriginalArrivalTime: 02 Nov 2007 19:24:33.0788 (UTC) FILETIME=[FFB3ABC0:01C81D85] X-TM-AS-Product-Ver: SMEX-8.0.0.1181-5.000.1023-15520.002 X-TM-AS-Result: No--34.911400-8.000000-2 X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No DKIM-Signature: v=0.5; a=rsa-sha256; q=dns/txt; l=31451; t=1194031474; x=1194895474; c=relaxed/simple; s=rtpdkim2001; h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version; d=cisco.com; i=acharny@cisco.com; z=From:=20=22Anna=20Charny=20(acharny)=22=20 |Subject:=20RE=3A=20Questions=20on=20=20LC-PCN=20draft=20version=2001 |Sender:=20 |To:=20=22Georgios=20Karagiannis=22=20, =0A=20=20= 20=20=20=20=20=20=22Lars=20Westberg=20(KI/EAB)=22=20,=0A=20=20=20=20=20=20=20=20; bh=UjY7OjfzgZbJXK7H7aiHQvTx9buXha+hhd2oMNKirEg=; b=PxTOYsCSvzE+0t/vkTsrulMBrP9ROXp4Y/P17N6/+lCZ07J8uiBlDDykiU4Dw0Y1Xb61hSg3 4yAqFUT6WFboKCBvodnHXXf7rW1ZUqpG2OfYbNJ98tmMn2XFRl3mbEEd; Authentication-Results: rtp-dkim-2; header.From=acharny@cisco.com; dkim=pass ( sig from cisco.com/rtpdkim2001 verified; ); X-Spam-Score: 0.0 (/) X-Scan-Signature: cadb9ba0ba1c1ba4f99ac017158fabc3 Cc: pcn@ietf.org Subject: [PCN] RE: Questions on LC-PCN draft version 01 X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Georgios, OK, trying again... Something still seems wrong.=20 First wanted to confirm that I now got the setting of the thresholds right (the following collected from your various responses): NOTE: I am ignoring multi-congestion error because I am still trying to figure out how it works in a single bottleneck case.=20 Pcn-lower-egress-percentage is some configured value (%) Pcn-lower-egress-rate =3D pcn-lower-egress-percentage * admission-termination-rate/100 (I assume the division by 100 is necessary?) Pcn-upper-egress-rate=3Dpcn-lower-egress-rate+Admission-offset-rate Admisison-offset-rate is a global parameter that has the meaning of the difference between the pcn-upper-rate and pcn-lower-rate at all interior nodes (which is assumed the same in all cases). Egress goes into termination mode when it sees the rate of marked packets of a given ingress exceeding the pcn-upper-egress-rate. So far so good? Assuming it is all correct, my example where none of the egresses can ever go into the termination mode remains valid, I believe. Let me restate the example. Consider the following example. A bottleneck link of capacity 100 mbps , pcn-termination-offset of 40mbps and pcn-admission-offset of 30 mbps is shared by 100 ingress-egress aggregates, each with the total pcn rate of 1 mbps. On this link pcn-upper-threshold is 60mbps, and pcn-lower-threshold is 30mbps. Suppose further that all 100 ingress-egress aggregates go to different egress nodes (and suppose for simplicity there is no other traffic going to these egresses; this last assumption is irrelevant because the egress measurement is per ingress-egress pair anyway, but makes it convenient to imagine). The bottleneck link is clearly in the termination state, as the total rate of pcn traffic on this link is 100 mbps which is above the pcn-upper-threshold.=20 This means we want the egress nodes to somehow recognise this state and get into the termination mode. But it seems in this example the egresses can't ever recognize that the system is in the termination mode. See below. Suppose pcn-lower-egress-percentage is set to w% (0 admission-offset-rate =3D 30Mbps So our pcn-upper-egress rate is always greater than 30 Mbps,=20 while the entire rate of pcn traffic the egress will ever see is *at=20 most* 1 Mbps (because it is plainly the total rate of the single ingress-egress aggreate that goes to this egress). So, regardless of the actual amount of marked traffic in this egress-egress aggregate, the=20 absolute excess rate of this ingress-egress aggregate will never be above 1mbps either.In turn that means that the pcn-upper-egress-rate will NEVER be exceeded (because pcn-upper-egress-rate is so much larger than the total rate of the pcn traffic at the egress). In turn, that=20 appears to mean that in this scenario none of the egresses will ever end up in termination state, and so termination will never occur. That=20 (still) does not seem right. So again, even with the corrected definition of pcn-lower-thershold, this example seems to imply a rather fundamental flaw in the algorithm. You say, in your response to the previous example, that=20 >the situation that you described cannot (often) occur because=20 > the difference between the admission control threshold and=20 > flow termination threshold at the egress is equal to=20 > Admission_offset_rate +/- multicongestion_error.=20 I believe that multi-congestion error is irrelevant here, because=20 in the considered example there is only a single congestion point. >Note that=20 > the Admission_offset_rate is an absolute rate value. The=20 > multicongestion_error is used to identify the bounds that=20 > certain situations occur where the PCN_egreess_ node should=20 > have been in flow termination state but it is not. Please see=20 > previous discussions on this. I do not understand the relevance of the multiocongestion error in this case, or rules for its setting at all, I am afraid. Do we need multi-congestion error (which I do not know how to set anyway) to deal with a single congestion point also? What should it be set to? Do you need to examine a configuration AND all ingress-egress aggregare rates AND all possible failures to set it correctly?=20 Best, Anna=20 > -----Original Message----- > From: Georgios Karagiannis [mailto:karagian@cs.utwente.nl]=20 > Sent: Friday, November 02, 2007 12:20 PM > To: Anna Charny (acharny); anurag.bhargava@ericsson.com; Lars=20 > Westberg (KI/EAB) > Cc: pcn@ietf.org > Subject: RE: Questions on LC-PCN draft version 01 >=20 > Hi Anna >=20 > Thank you very much for your comments! >=20 > Please see in line! >=20 >=20 > On 11/1/2007, "Anna Charny (acharny)" wrote: >=20 > > > >Hi Georgios, > > > >A few more clarification questions: > > > >> -----Original Message----- > >> From: Georgios Karagiannis [mailto:karagian@cs.utwente.nl] > >> Sent: Thursday, November 01, 2007 2:35 AM > >> To: Anna Charny (acharny); anurag.bhargava@ericsson.com; Lars=20 > >> Westberg (KI/EAB) > >> Cc: pcn@ietf.org > >> Subject: Re: Questions on LC-PCN draft version 01 > >> > >> Hi Anna > >> > >> Thank you very much for your comments! > >> Before answering to your comments, see in line below, I=20 > would like to=20 > >> explain in an abstract way the LC-PCN algorithm. > >> > >> * Setting the thresholds at PCN_interior_nodes: > >> ----------------------------------------------- > >> In order to calculate the PCN_upper_rate we use two parameters: > >> Maximum PHB capacity: that is the maximum capacity that can be=20 > >> supported by a PCN_interior_node > >> Termination_offset_rate: that is an absolute rate value=20 > that should=20 > >> be set equal into all PCN_interior_nodes. Note that this value is=20 > >> used by PCN_interior_nodes to calculate their=20 > PCN_upper_rate and also=20 > >> during the situation that a PCN_interior_node is in flow=20 > termination=20 > >> state and it receives PCN_marked packets. Please see=20 > pseudo code on=20 > >> page 20. > >> This value must be set equal into all PCN_interior_nodes such that=20 > >> all these nodes will know when to take into account the incoming=20 > >> PCN_marked packets and when not. > >> > >> The PCN_upper_rate is then found as: > >> PCN_upper_rate =3D "Maximum PHB capacity" - Termination_offset_rate > >> > > > >So if you have a link of 10 Mbps and a link of 40 Mbps (say each is=20 > >allowed to use all bandwidth for PCN, for simplicity), then=20 > they both=20 > >use the same *absolote* value of the termination-offset-rate?. >=20 > Georgios: Yes, you are right! This has to do with the=20 > importance of using the Termination_offset_rate. We will=20 > motivate this in the following version of the draft. >=20 > > So If I > >wanted to have PCN-upper-rate 20% below my link capacity on=20 > all links,=20 > >I would not be able to do so with this approach? A larger link would=20 > >have a smaller relative safety margin, then... >=20 > Georgios: Yes, this can be considered as a disadvantage.=20 > However, there might be other ways of preconfiguring each=20 > PCN_interior_node to know the termination_offset rate used by=20 > each neighbour PCN_interior_node. Then the pseudocode on page=20 > 20 will have to use the Termination_offset_rate associated=20 > with the incoming link from where the incoming PCN_marked=20 > packets are arriving. Then this constraint can be avoided. >=20 >=20 > >Same comment for the > >difference between admission and termination - the relative=20 > difference=20 > >between admission and termination threshods (Pcn-lower-rate and > >pcn-upper-rate) is global, so the larger links have a=20 > smaller relative=20 > >difference between admission and termination thresholds. Right? >=20 > Georgios: No, because the Admission_offset_rate is an absolute value! > So this difference is globally equal. >=20 > > > >> The PCN_lower_rate is configured in all PCN_interior-nodes=20 > and it can=20 > >> be calculated in the following way: > >> PCN_lower_rate =3D PCN_upper_rate - Admission_offset_rate > >> > >> The Admission_offset_rate is an absolute rate value and it=20 > is equal=20 > >> in all PCN_interior_nodes and PCN_egress_nodes. Note that=20 > this value=20 > >> is used by PCN_interior_nodes to calculate their=20 > PCN_lower_rate and=20 > >> the PCN_egress_nodes to calculate their PCN_upper_rate_egress.=20 > >> Furthermore, this value is used by the PCN_interior_nodes=20 > also during=20 > >> the situation that a PCN_interior_node is in admission=20 > control state=20 > >> and it receives PCN_marked packets. Please see pseudo code on page=20 > >> 13. > >> This value must be set equal into all PCN_interior_nodes such that=20 > >> all these nodes will know when to take into account the incoming=20 > >> PCN_marked packets and when not. Note that a PCN_interior_node can=20 > >> PCN_mark packets up to an excess rate equal to the=20 > >> Admission_offset_rate. If the exess rate in an=20 > PCN_interior_node is=20 > >> higher than the Admission_offset_rate, then the PCN_interior_node=20 > >> changes state from admission control state to flow=20 > termination state. > > > >OK. > > > >> > >> * Setting the thresholds at PCN_egress_nodes: > >> --------------------------------------------- > >> The question is how to calculate the threshold that defines when a=20 > >> PCN_egress_node goes into the admission control state. > >> One way to do that is to consider that when the PCN_egress_node=20 > >> receives a PCN_marked packet it will mean that at least one=20 > >> PCN_interior_node started to be admission control congested and=20 > >> therefore it will go from Normal state to admission=20 > control state. Of=20 > >> course this will somehow might provide some errors, because there=20 > >> might be situations that this consideration might be to=20 > conservative.=20 > >> Therefore, we use a percentage of received PCN_marking encoded=20 > >> packets in proportion to total rate of received packets. In this=20 > >> version of the draft we call this value as: > >> PCN_lower_rate_egress. This is wrong. > >> What we should say is: > >> PCN_lower_rate_egress: is equal to incoming_PCN_marking_rate, when: > >> (incoming_PCN_marking_rate/measured PHB rate) =3D to a = preconfigured=20 > >> percentage, say 1%. From now on I denote this percentage as: > >> PCN_lower_percentage_egress. > > > >In the draft, pcn_lower- and -upper_rate_egress seem to be=20 > defined on a=20 > >per *ingress-egress-pair basis*. > >Is that correct? >=20 > georgios: Yes, it is correct! > > > >> Thus we can say that a PCN_egress_node changes from Normal=20 > state to=20 > >> admission control state when=20 > incoming_PCN_marking_rate/measured PHB=20 > >> rate > PCN_lower_percentage_egress. > >> Where, > >> incoming_PCN_marking_rate =3D N * input_PCN_marking_bytes/T Where,=20 > >> input_PCN_marking_bytes =3D received number of "PCN_marking" = encoded=20 > >> packets during measurement period T. > > > >Again, is it per ingress-egress pair, or not? >=20 > georgios: Yes, it is correct! >=20 > > > >> > >> Now we defined the condition that the PCN_egress_node=20 > changes state=20 > >> from Normal state to admission control state. But how will the=20 > >> PCN_egress_node change state from admission control state to flow=20 > >> termination state. > >> In order to explain this, it is imporatnt to note that each=20 > >> PCN_interior_node that is in admission control state it=20 > can PCN_mark=20 > >> packets up to a value equal to Admission_offset_rate.=20 > Furthermore, if=20 > >> a PCN_interior_node receives incoming PCN_marked packets and is in=20 > >> the addmission control state, it will not remark any=20 > packets if the=20 > >> excess rate is equal or lower than the=20 > incoming_PCN_marking_rate, see=20 > >> page 13. > >> Furthermore, if we will consider as normal situations the=20 > situations=20 > >> that no ECMP occurs and that all flows belonging to the same=20 > >> ingress-egress aggregate will use the same path from=20 > PCN_ingress to=20 > >> PCN_egress, this will mean that when the PCN_egress_node receives,=20 > >> for the given ingress-egress aggregate an excess rate equal to=20 > >> Admission_offset_rate it will have to change from=20 > admission control=20 > >> state to flow termination state. > >> Thus in this case the second threshold, that in this case=20 > is a rate=20 > >> and not a percentage, can be calculated as follows: > >> PCN_upper_egress_rate =3D PCN_lower_egress_rate +=20 > Admission_offset_rate. > > > > > >Here is where I am afraid I am fundamentally confused. > >Pcn_lower-egress_rate seems to have beed redefined above as a=20 > >percentage threshold, but it is an absolute rate again here. Perhaps=20 > >you just mean that Pcn-lower-egress-rate =3D=20 > pcn_lower_egress_percentage=20 > >* total-measured-rate-of-this-ingress-egrees-aggregate/100? > >So pcn-lower-egress-rate then is just the fraction of the=20 > total rate of=20 > >the ingress-egress aggregate corresponding to the configured=20 > percentage=20 > >threshold. Right? >=20 > Georgios: I think I could not explain and translate what I=20 > wanted to do in the right way! > A better way of expressing the need of using a rate of=20 > percentage of the received PCN_marking encoded packets, would=20 > be the following: >=20 > Pcn-lower-egress-rate =3D pcn_lower_egress_percentage *=20 > Admission_offset_rate >=20 >=20 > This means that event_A in Section 4.2.3 will be activated=20 > when incoming_PCN_marking_rate > pcn_lower_egress_percentage=20 > * Admission_offset_rate or when: > incoming_PCN_marking_rate > Pcn-lower-egress-rate >=20 > This means that the PCN_egress_ node does not change to=20 > admission control state when at least one PCN_marked packet=20 > arrives, but it changes when the incoming_PHB_marking rate=20 > equals a percentage of the Admission_offset_rate. This is due=20 > to the fact that Admission_offset_rate is used by the=20 > interioe nodes to change from admission control state to flow=20 > termination state. >=20 > > > >Assuming the above is correct, I cannot see how the system will work=20 > >correctly. Consider the following example. A bottleneck link of=20 > >capacity 100 mbps , pcn-termination-offset of 40mbps and=20 > >pcn-admission-offset of 30 mbps is shared by 100 ingress-egress=20 > >aggregates, each with the total pcn rate of 1 mbps. On this link=20 > >pcn-upper-threshold is 60mbps, and pcn-lower-threshold is 30mbps. > >Suppose further that all 100 ingress-egress aggregates go to=20 > different=20 > >egress nodes (and suppose for simplicity there is no other traffic=20 > >going to these egresses). The bottleneck link is clearly in the=20 > >termination state, as the total rate of pcn traffic on this=20 > link is 100=20 > >mbps which is above the pcn-upper-threshold. This means we want the=20 > >egress nodes to somehow recognise this state and get into the=20 > >termination mode. But it seems in this example the egresses=20 > can't ever=20 > >recognize that the system is in the termination mode. See below. >=20 >=20 > > > >Suppose pcn-lower-egress-percentage is set to w% (0 Then, each=20 > >of these egresses will compute the (absolute)=20 > pcn-lower-egress-rate as=20 > >total pcn traffic of the ingress-egress aggregate it sees (which is 1 > >mbps) times the pcn_lower_egress_percentage/100, so it will get=20 > >pcn-lower-egress-rate=3D w* 0.01 Mbps. > >According to your explanation above, it will then compute=20 > = >Pcn-upper-egress-rate=3Dpcn-lower-egress-rate+Admission-offset-rate=3D > >w*0.01 mbps+30mbps >=3D 30 mbps (for all possible settings of w). > > > >So our pcn-upper-egress rate is always greater than 30 Mbps,=20 > while the=20 > >entire rate of pcn traffic the egress will ever see is *at=20 > most* 1 Mbps=20 > >(because it is plainly the total rate of the single ingress-egress=20 > >aggreate that goes to this egress). So, regardless of the=20 > actual amount=20 > >of marked traffic in this egress-egress aggregate, the=20 > absolute excess=20 > >rate of this ingress-egress aggregate will never be above=20 > 1mbps either. > >In turn that means that the pcn-upper-egress-rate will NEVER be=20 > >exceeded (because pcn-upper-egress-rate is so much larger than the=20 > >total rate of the pcn traffic at the egress). In turn, that=20 > appears to=20 > >mean that in this scenario none of the egresses will ever end up in=20 > >termination state, and so termination will never occur. That=20 > does not seem right. >=20 >=20 > Georgios: Thank you for the given example, which stimulated=20 > me to better translate what I wanted to specify and what I=20 > have written down. > Please see above that I defned: > Pcn-lower-egress-rate =3D pcn_lower_egress_percentage *=20 > Admission_offset_rate >=20 > The situation that you described cannot (often) occur because=20 > the difference between the admission control threshold and=20 > flow termination threshold at the egress is equal to=20 > Admission_offset_rate +/- multicongestion_error. Note that=20 > the Admission_offset_rate is an absolute rate value. The=20 > multicongestion_error is used to identify the bounds that=20 > certain situations occur where the PCN_egreess_ node should=20 > have been in flow termination state but it is not. Please see=20 > previous discussions on this. >=20 > > > >I would appreciate any help in clarifying this. > > > >Anna > > > > > >> However, there are some corner cases, that mainly occur when the=20 > >> different congestion points (admission control congested > >> PCN_interior_nodes) on the same path are not=20 > simulataneously starting=20 > >> to be congested. Therefore we use the=20 > multicongestion_error parameter=20 > >> to identify the error bound that ocurs due to these corner cases.=20 > >> Note that this error bound can be e.g., predefined ones=20 > off line by=20 > >> the operator, by studying the network topology and/or studying how=20 > >> often such corner cases could occur and/or doing off line=20 > >> measurements. Therefore we use: > >> PCN_upper_rate_egress =3D PCN_lower_rate_egress +=20 > Admission_offset_rate=20 > >> +/- multicongestion_error > >> > >> * How the states of operation in PCN_interior_nodes are=20 > being changed? > >>=20 > --------------------------------------------------------------------- > >> This is explained on page 17, 18, by using Figure 4. > >> > >> Change from Normal state to Admission control state: event=20 > A Occurs=20 > >> when: > >> Measured PHB rate > PCN_lower_rate > >> > >> Change from Admission control state to Flow Termination > >> state: event B Occurs when: > >> Measured PHB rate > PCN_upper_rate > >> > >> * How the states of operation in PCN_egress_nodes are=20 > being changed? > >>=20 > --------------------------------------------------------------------- > >> This is explained on page 21, 22. Note that the=20 > description of event=20 > >> A on page 21 has to be modified to avoid the confusions that were=20 > >> caused up to now, see explanation given above. > >> > >> Change from Normal state to Admission control state: event=20 > A Occurs=20 > >> when: > >> incoming_PCN_marking_rate/measured PHB rate) >=20 > >> PCN_lower_percentage_egress As explained above: > >> IF ((incoming_PCN_marking_rate/measured PHB rate) =3D > >> PCN_lower_percentage_egress) > >> THEN PCN_lower_egress_rate =3D incoming_PCN_marking_rate > >> > >> Change from Admission control state to flow termination > >> state: event B Occurs when: > >> incoming_PCN_marking_rate > PCN_upper_rate_egress > >> > >> It is important to note that also the explanation of event=20 > C on page=20 > >> 21, has to be modified to avoid the confusions that were=20 > caused up to=20 > >> now, see explanation given above: > >> Change from Admission control state to Normal state: > >> Occurs when: > >> incoming_PCN_marking_rate/measured PHB rate) =3D<=20 > >> PCN_lower_percentage_egress > >> > >> > >> * Generated excess rate by an PCN_interior_node operating in=20 > >> admission control state. > >> ------------------------------------------------------------- > >> > >> The excess rate =3D signaled_overload_rate. > >> The maximum excess rate that a PCN_interior_node can calculate in=20 > >> admission control state is equal to Admission_offset_rate, see=20 > >> explanation above. > >> The number of bytes that are remarked,=20 > signaled_remarked_bytes depend=20 > >> on the value of calculated excess rate=20 > (signaled_overload_rate), the=20 > >> value of the Admission_offset_rate and the value of the=20 > >> incoming_PCN_marking_rate, see pseudo code on page 13. > >> > >> Note that all packets that are passing through a congested=20 > >> PCN_interior_node an are not being PCN_marked by the=20 > >> PCN_interior_node have to be remarked using the=20 > PCN_Affected_marking. > >> > >> > >> Regarding probes, the probe packets that are passing through a=20 > >> congested node are either PCN_marked or PCN_Affected_marked. > >> > >> * Generating excess rate by an PCN_interior_node operating in flow=20 > >> termination state: > >> ---------------------------------------------------------------- > >> The excess rate =3D signaled_overload_rate. > >> Note that the calculation of the signaled_overload_rate is=20 > different=20 > >> than in the situation that the PCN_Interior_node operates in=20 > >> admission control state, see page 19 and 20. This is due=20 > to the fact=20 > >> that a sliding window is used to solve an undershooting=20 > problem, see=20 > >> discussion on page 19. > >> The number of bytes that are remarked,=20 > signaled_remarked_bytes depend=20 > >> on the value of calculated excess rate=20 > (signaled_overload_rate), the=20 > >> value of the Termination_offset_rate and the value of the=20 > >> incoming_PCN_marking_rate, and the see pseudo code on page 20. > >> > >> Note that all packets that are passing through a congested=20 > >> PCN_interior_node an are not being PCN_marked by the=20 > >> PCN_interior_node have to be remarked using the=20 > PCN_Affected_marking. > >> > >> * Providing admission control at PCN_egress_nodes: > >> -------------------------------------------------- > >> When the PCN_egress_node is operating in admission control=20 > state than=20 > >> a flow that is requesting admission into the PCN domain can b=20 > >> etreated in the following way: > >> > >> If no probing is used, the request for admission can be=20 > accomplished=20 > >> by using an external to PCN signaling protocol. > >> In this case when the request arrives at a PCN_egress_node that=20 > >> operates in admission control state then the request is=20 > rejected. If=20 > >> it operates in Normal state is accepted. > >> > >> If probing is used, the request for admission is accomplished by=20 > >> using probe packets. In this case when the probe arrives at a=20 > >> PCN_egress_node and it is either PCN_marking or=20 > PCN_Affected_marking=20 > >> encoded is rejected. Otherwise is accepted. > >> Note that probes can only be used when=20 > PCN_Affected_marking is appled=20 > >> in whole PCN domain. Otherwise, the admission control=20 > procedure will=20 > >> work by using e.g., an external signaling protocol used between=20 > >> PCN_ingress_nodes and PCN_egress_nodes. > >> > >> * Providing flow termination at PCN_egress_nodes: > >> -------------------------------------------------- > >> > >> When a PCN_egress_node operates on flow termination it=20 > calculates the=20 > >> incoming excess rate (incoming_PCN_marking_rate). > >> By using the excess rate the PCN_egress_node calclates the=20 > numer of=20 > >> flows that have to be terminated using the pseudo code=20 > given on page=20 > >> 23. Note that the PCN_egress_node has to maintain per flow=20 > >> reservation states. > >> The information contained in the per flow states is used for the=20 > >> calculation of the flow that have to be terminated, see=20 > pseudocode on=20 > >> page 23. Note also that this pseudo code uses priority=20 > clases, but it=20 > >> operates when also no priority clases are used by setting=20 > >> Maximum_priority =3D 0 Where, 0 =3D< priority_class =3D< = Maximum_priority > >> > >> > >> > >> Below, I am providing some answers to your comments! > >> > >> > >> > >> > >> > >> > >> > >> > >> > -----Original Message----- > >> > From: Anna Charny (acharny) [mailto:acharny@cisco.com] > >> > Sent: zondag 7 oktober 2007 21:40 > >> > To: anurag.bhargava@ericsson.com; Georgios Karagiannis; > >> Lars Westberg > >> > (KI/EAB) > >> > Cc: pcn@ietf.org > >> > Subject: Questions on LC-PCN draft version 01 > >> > > >> > Dear authors of the LC-PCN draft, > >> > > >> > Am I right that there has been no response yet to Phil's=20 > questions=20 > >> > (attached at the end) on the new version of your draft below (I=20 > >> > checked the pcn list and it does not seem it went there)? If the=20 > >> > response was sent, can you resend it please? > >> > > >> > I have many of the same questions, and also a few=20 > additional ones. > >> > Here are some of them. > >> > > >> > 1) I do not understand whether pcn-lower_rate_egress and=20 > >> > pcn_upper_rate ingress are expressed as a fraction of the > >> total rate > >> > (a unitless entity, analogous to CLE) or an absolute value > >> (in bits or > >> > bytes per second). The text seems to be contradictory on=20 > this point: > >> > > >> > on page 8 it is a fraction: > >> > > >> > "PCN_lower_rate_egress =3D predefined percentage of received=20 > >> > PCN_marking", > >> > > >> > while on page 13 it is an absolute rate: > >> > > >> > "If the incoming_PCN_marking_rate is higher than a preconfigured=20 > >> > PCN_lower_rate_egress, ..." where the > >> > > >> > (Incoming_PCN_marking_rate is clearly defined as abslolue > >> rate on page > >> > 12: "Where the "incoming_PCN_marking_rate" is calculated=20 > as follows: > >> > incoming_PCN_marking_rate =3D (received number of "PCN_marking" > >> > DSCP during T)* N)/T;) > >> > >> Georgios: You are right, please see discussion above on paragaph=20 > >> denoted > >> as: > >> "Setting the thresholds at PCN_egress_nodes". > >> > >> > >> We use a percentage of received PCN_marking encoded packets in=20 > >> proportion to total rate of received packets to define when a=20 > >> PCN_egres_node goes from Normal state to admission control=20 > state. In=20 > >> this version of the draft we call this value as:=20 > >> PCN_lower_rate_egress. This is wrong. > >> What we should say is: > >> PCN_lower_rate_egress: is equal to incoming_PCN_marking_rate, when: > >> (incoming_PCN_marking_rate/measured PHB rate) =3D to a = preconfigured=20 > >> percentage, say 1%. From now on I denote this percentage as: > >> PCN_lower_percentage_egress. > >> Thus we can say that a PCN_egress_node changes from Normal=20 > state to=20 > >> admission control state when=20 > incoming_PCN_marking_rate/measured PHB=20 > >> rate > PCN_lower_percentage_egress. > >> Where, > >> incoming_PCN_marking_rate =3D N * input_PCN_marking_bytes/T Where,=20 > >> input_PCN_marking_bytes =3D received number of "PCN_marking" = encoded=20 > >> packets during measurement period T. > >> > >> > >> > > >> > 2) Regarding the question above, > >> > > >> > * if pcn_lower_rate_eggress (and pcn_uppre-rate_egress) are > >> absolute > >> > rates, then how do you choose that abslolute rate value,=20 > given that=20 > >> > the rates of ingress-egress aggregates at the same=20 > ingress may be=20 > >> > vastly different in magnitude? > >> > >> Georgios: See explanation above! Hopefully this=20 > misundersatnding is=20 > >> also solved! > >> > >> > > >> > *if, however, pcn_lower_rate_eggress (and > >> > pcn_uppre-rate_egress) are ratios, then what is the guidance of=20 > >> > setting these ratios so that the egress can always tell=20 > admission=20 > >> > state from the termination state, given that the same > >> marking is used > >> > for admission and termination marking? > >> > Specifically, suppose at the bottleneck pcn_lower_threshold > >> is set to > >> > X, and pcn_upper_threshold to aX with some a>1 (assume=20 > for example=20 > >> > a=3D1.5). > >> > How does the egress node tell between the conditions=20 > when the total=20 > >> > pcn traffic on the bottleneck is 1.1X (which is the state wnen=20 > >> > admission is needed but termination is not, - in this case > >> > (1.1X-X)/X=3D0.1 traffic is pcn-marked), and the case when=20 > the total=20 > >> > traffic on the bottleneck is 1.1aX, which is the case when > >> termination > >> > is needed, and (1.1ax-ax)/ax=3D0.1), if in both cases the > >> ratio between > >> > marked and total traffic is the same value 0.1? > >> > >> Georgios: In order to solve the above described problems, the=20 > >> thresholds used in the PCN_interior_nodes and PCN_egress_nodes are > >> using: > >> the Admission_offset_rate that is an absolute rate value=20 > which is set=20 > >> equal into whole PCN domain. > >> > >> Note that the maximum excess rate that a PCN_interior_node can=20 > >> calculate in admission control state is equal to=20 > >> Admission_offset_rate. If the excess rate is higher than the=20 > >> Admission_offset_rate then the node changes from admission control=20 > >> state to flow termination state. > >> > >> Please see all details that I have provided at the top of=20 > this email.=20 > >> In particular, check the paragraphs denoted above as: > >> "Setting the thresholds at PCN_interior_nodes" > >> "Setting the thresholds at PCN_egress_nodes" > >> "Generated excess rate by an PCN_interior_node operating=20 > in admission=20 > >> control state" > >> > >> > >> > >> > > >> > 3) On page 9, multicongestion error error is introduced, > >> but the text > >> > is silent on how this error is defined and what is it set > >> to (if it is > >> > a configuration parameter), or how it is measured (if it is > >> something > >> > that is being measured). Can you explain, please? > >> > >> Georgios: > >> Please see the paragraph denoted above as "Setting the=20 > thresholds at=20 > >> PCN_egress_nodes" > >> > >> > >> Best regards, > >> Georgios > >> > >> > >> > > >> > I have some more questions, including sharing those asked > >> by Phil in > >> > the attached message), but I will stop now, as I hope > >> clarifications > >> > on the above (and Phil's) questions will help understand=20 > the draft=20 > >> > better... > >> > > >> > Thank you in advance for clarifying all this. > >> > > >> > Anna > >> >=20 _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Fri Nov 02 16:05:33 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Io2lb-0006Fn-00; Fri, 02 Nov 2007 16:05:19 -0400 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Io2la-0006FU-6n for pcn-confirm+ok@megatron.ietf.org; Fri, 02 Nov 2007 16:05:18 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Io2lZ-0006FJ-Pc for pcn@ietf.org; Fri, 02 Nov 2007 16:05:17 -0400 Received: from rotterdam.ewi.utwente.nl ([130.89.10.5]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1Io2lW-0005DS-O7 for pcn@ietf.org; Fri, 02 Nov 2007 16:05:17 -0400 Received: from webmail.cs.utwente.nl (janus.ewi.utwente.nl [130.89.10.26]) by rotterdam.ewi.utwente.nl (8.13.6/8.13.6) with SMTP id lA2K4lBD027863; Fri, 2 Nov 2007 21:04:47 +0100 (MET) Received: from 193.192.232.84 (auth. user karagian@imap1.ewi.utwente.nl) by webmail.cs.utwente.nl with HTTP; Fri, 02 Nov 2007 20:04:46 +0000 To: "Anna Charny (acharny)" , "Lars Westberg (KI/EAB)" , "anurag.bhargava@ericsson.com" Date: Fri, 02 Nov 2007 20:04:44 +0000 X-Mailer: IlohaMail/0.8.13 (On: webmail.cs.utwente.nl) Message-ID: In-Reply-To: From: "Georgios Karagiannis" Bounce-To: "Georgios Karagiannis" MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0 () X-Scanned-By: MIMEDefang 2.52 on 130.89.10.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0rc3 (rotterdam.ewi.utwente.nl [130.89.10.5]); Fri, 02 Nov 2007 21:04:51 +0100 (MET) X-Spam-Score: 1.8 (+) X-Scan-Signature: ad122f56a92d6ccd133117ee8a4b1ff3 Cc: "pcn@ietf.org" Subject: [PCN] RE: Questions on LC-PCN draft version 01 X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Anna Please see in line! Best regards, Georgios On 11/2/2007, "Anna Charny (acharny)" wrote: >Hi Georgios, > >OK, trying again... > >Something still seems wrong.=20 > >First wanted to confirm that I now got the setting of the thresholds >right (the following collected from your various responses): > >NOTE: I am ignoring multi-congestion error because I am still trying to >figure out how it works in a single bottleneck case.=20 > >Pcn-lower-egress-percentage is some configured value (%) > >Pcn-lower-egress-rate =3D pcn-lower-egress-percentage * >admission-termination-rate/100 (I assume the division by 100 is >necessary?) > >Pcn-upper-egress-rate=3Dpcn-lower-egress-rate+Admission-offset-rate > >Admisison-offset-rate is a global parameter that has the meaning of the >difference between the pcn-upper-rate and pcn-lower-rate at all interior >nodes (which is assumed the same in all cases). > >Egress goes into termination mode when it sees the rate of marked >packets of a given ingress exceeding the pcn-upper-egress-rate. > >So far so good? > >Assuming it is all correct, my example where none of the egresses can >ever go into the termination mode remains valid, I believe. Let me >restate the example. > >Consider the following example. A bottleneck link of capacity 100 mbps , >pcn-termination-offset of 40mbps and pcn-admission-offset of 30 mbps is >shared by 100 ingress-egress aggregates, each with the total pcn rate of >1 mbps. On this link pcn-upper-threshold is 60mbps, and >pcn-lower-threshold is 30mbps. Suppose further that all 100 >ingress-egress aggregates go to different egress nodes (and suppose for >simplicity there is no other traffic going to these egresses; this last >assumption is irrelevant because the egress measurement is per >ingress-egress pair anyway, but makes it convenient to imagine). The >bottleneck link is clearly in the termination state, as the total rate >of pcn traffic on this link is 100 mbps which is above the >pcn-upper-threshold.=20 > >This means we want the egress nodes to somehow recognise this state and >get into the termination mode. But it seems in this example the egresses > >can't ever recognize that the system is in the termination mode. See >below. Georgios: I think that the example that you provide is an wors case scenario. First of all the Admission_offset_rate is too high. This Admission_offset_rate should be set such that siuations as you describe will not occur. > >Suppose pcn-lower-egress-percentage is set to w% (0u=3Dw/100. Then, each of these egresses will compute the=20 >pcn-lower-egress-rate=3Du*admission-offset-rate > >and=20 > >pcn-upper-egress-rate=3Dpcn-lower-rate+admission-offset-rate =3D=20 >(u+1)*admission-offset-rate > admission-offset-rate =3D 30Mbps > > >So our pcn-upper-egress rate is always greater than 30 Mbps,=20 >while the entire rate of pcn traffic the egress will ever see is *at=20 >most* 1 Mbps (because it is plainly the total rate of the single >ingress-egress aggreate that goes to this egress). So, regardless of the > >actual amount of marked traffic in this egress-egress aggregate, the=20 >absolute excess rate of this ingress-egress aggregate will never be >above 1mbps either.In turn that means that the pcn-upper-egress-rate >will NEVER be exceeded (because pcn-upper-egress-rate is so much larger >than the total rate of the pcn traffic at the egress). In turn, that=20 >appears to mean that in this scenario none of the egresses will ever end >up in termination state, and so termination will never occur. That=20 >(still) does not seem right. > > >So again, even with the corrected definition of pcn-lower-thershold, >this example seems to imply a rather fundamental flaw in the algorithm. > >You say, in your response to the previous example, that=20 > >>the situation that you described cannot (often) occur because=20 >> the difference between the admission control threshold and=20 >> flow termination threshold at the egress is equal to=20 >> Admission_offset_rate +/- multicongestion_error.=20 > >I believe that multi-congestion error is irrelevant here, because=20 >in the considered example there is only a single congestion point. > >>Note that=20 >> the Admission_offset_rate is an absolute rate value. The=20 >> multicongestion_error is used to identify the bounds that=20 >> certain situations occur where the PCN_egreess_ node should=20 >> have been in flow termination state but it is not. Please see=20 >> previous discussions on this. > >I do not understand the relevance of the multiocongestion error in this >case, or rules for its setting at all, I am afraid. Do we need >multi-congestion error (which I do not know how to set anyway) to deal >with a single congestion point also? What should it be set to? Do you >need to examine a configuration AND all ingress-egress aggregare rates >AND all possible failures to set it correctly?=20 Georgios: You are right that the muli-congestion-error does not play a role, but the selection of the Admission_offset_rate plays in this situation a signifficat role and it should be selected such that wores case scenarios, such as the one described by you do not occur+ >Best, >Anna=20 >> -----Original Message----- >> From: Georgios Karagiannis [mailto:karagian@cs.utwente.nl]=20 >> Sent: Friday, November 02, 2007 12:20 PM >> To: Anna Charny (acharny); anurag.bhargava@ericsson.com; Lars=20 >> Westberg (KI/EAB) >> Cc: pcn@ietf.org >> Subject: RE: Questions on LC-PCN draft version 01 >>=20 >> Hi Anna >>=20 >> Thank you very much for your comments! >>=20 >> Please see in line! >>=20 >>=20 >> On 11/1/2007, "Anna Charny (acharny)" wrote: >>=20 >> > >> >Hi Georgios, >> > >> >A few more clarification questions: >> > >> >> -----Original Message----- >> >> From: Georgios Karagiannis [mailto:karagian@cs.utwente.nl] >> >> Sent: Thursday, November 01, 2007 2:35 AM >> >> To: Anna Charny (acharny); anurag.bhargava@ericsson.com; Lars=20 >> >> Westberg (KI/EAB) >> >> Cc: pcn@ietf.org >> >> Subject: Re: Questions on LC-PCN draft version 01 >> >> >> >> Hi Anna >> >> >> >> Thank you very much for your comments! >> >> Before answering to your comments, see in line below, I=20 >> would like to=20 >> >> explain in an abstract way the LC-PCN algorithm. >> >> >> >> * Setting the thresholds at PCN_interior_nodes: >> >> ----------------------------------------------- >> >> In order to calculate the PCN_upper_rate we use two parameters: >> >> Maximum PHB capacity: that is the maximum capacity that can be=20 >> >> supported by a PCN_interior_node >> >> Termination_offset_rate: that is an absolute rate value=20 >> that should=20 >> >> be set equal into all PCN_interior_nodes. Note that this value is=20 >> >> used by PCN_interior_nodes to calculate their=20 >> PCN_upper_rate and also=20 >> >> during the situation that a PCN_interior_node is in flow=20 >> termination=20 >> >> state and it receives PCN_marked packets. Please see=20 >> pseudo code on=20 >> >> page 20. >> >> This value must be set equal into all PCN_interior_nodes such that=20 >> >> all these nodes will know when to take into account the incoming=20 >> >> PCN_marked packets and when not. >> >> >> >> The PCN_upper_rate is then found as: >> >> PCN_upper_rate =3D "Maximum PHB capacity" - Termination_offset_rate >> >> >> > >> >So if you have a link of 10 Mbps and a link of 40 Mbps (say each is=20 >> >allowed to use all bandwidth for PCN, for simplicity), then=20 >> they both=20 >> >use the same *absolote* value of the termination-offset-rate?. >>=20 >> Georgios: Yes, you are right! This has to do with the=20 >> importance of using the Termination_offset_rate. We will=20 >> motivate this in the following version of the draft. >>=20 >> > So If I >> >wanted to have PCN-upper-rate 20% below my link capacity on=20 >> all links,=20 >> >I would not be able to do so with this approach? A larger link would=20 >> >have a smaller relative safety margin, then... >>=20 >> Georgios: Yes, this can be considered as a disadvantage.=20 >> However, there might be other ways of preconfiguring each=20 >> PCN_interior_node to know the termination_offset rate used by=20 >> each neighbour PCN_interior_node. Then the pseudocode on page=20 >> 20 will have to use the Termination_offset_rate associated=20 >> with the incoming link from where the incoming PCN_marked=20 >> packets are arriving. Then this constraint can be avoided. >>=20 >>=20 >> >Same comment for the >> >difference between admission and termination - the relative=20 >> difference=20 >> >between admission and termination threshods (Pcn-lower-rate and >> >pcn-upper-rate) is global, so the larger links have a=20 >> smaller relative=20 >> >difference between admission and termination thresholds. Right? >>=20 >> Georgios: No, because the Admission_offset_rate is an absolute value! >> So this difference is globally equal. >>=20 >> > >> >> The PCN_lower_rate is configured in all PCN_interior-nodes=20 >> and it can=20 >> >> be calculated in the following way: >> >> PCN_lower_rate =3D PCN_upper_rate - Admission_offset_rate >> >> >> >> The Admission_offset_rate is an absolute rate value and it=20 >> is equal=20 >> >> in all PCN_interior_nodes and PCN_egress_nodes. Note that=20 >> this value=20 >> >> is used by PCN_interior_nodes to calculate their=20 >> PCN_lower_rate and=20 >> >> the PCN_egress_nodes to calculate their PCN_upper_rate_egress.=20 >> >> Furthermore, this value is used by the PCN_interior_nodes=20 >> also during=20 >> >> the situation that a PCN_interior_node is in admission=20 >> control state=20 >> >> and it receives PCN_marked packets. Please see pseudo code on page=20 >> >> 13. >> >> This value must be set equal into all PCN_interior_nodes such that=20 >> >> all these nodes will know when to take into account the incoming=20 >> >> PCN_marked packets and when not. Note that a PCN_interior_node can=20 >> >> PCN_mark packets up to an excess rate equal to the=20 >> >> Admission_offset_rate. If the exess rate in an=20 >> PCN_interior_node is=20 >> >> higher than the Admission_offset_rate, then the PCN_interior_node=20 >> >> changes state from admission control state to flow=20 >> termination state. >> > >> >OK. >> > >> >> >> >> * Setting the thresholds at PCN_egress_nodes: >> >> --------------------------------------------- >> >> The question is how to calculate the threshold that defines when a=20 >> >> PCN_egress_node goes into the admission control state. >> >> One way to do that is to consider that when the PCN_egress_node=20 >> >> receives a PCN_marked packet it will mean that at least one=20 >> >> PCN_interior_node started to be admission control congested and=20 >> >> therefore it will go from Normal state to admission=20 >> control state. Of=20 >> >> course this will somehow might provide some errors, because there=20 >> >> might be situations that this consideration might be to=20 >> conservative.=20 >> >> Therefore, we use a percentage of received PCN_marking encoded=20 >> >> packets in proportion to total rate of received packets. In this=20 >> >> version of the draft we call this value as: >> >> PCN_lower_rate_egress. This is wrong. >> >> What we should say is: >> >> PCN_lower_rate_egress: is equal to incoming_PCN_marking_rate, when: >> >> (incoming_PCN_marking_rate/measured PHB rate) =3D to a preconfigured=20 >> >> percentage, say 1%. From now on I denote this percentage as: >> >> PCN_lower_percentage_egress. >> > >> >In the draft, pcn_lower- and -upper_rate_egress seem to be=20 >> defined on a=20 >> >per *ingress-egress-pair basis*. >> >Is that correct? >>=20 >> georgios: Yes, it is correct! >> > >> >> Thus we can say that a PCN_egress_node changes from Normal=20 >> state to=20 >> >> admission control state when=20 >> incoming_PCN_marking_rate/measured PHB=20 >> >> rate > PCN_lower_percentage_egress. >> >> Where, >> >> incoming_PCN_marking_rate =3D N * input_PCN_marking_bytes/T Where,=20 >> >> input_PCN_marking_bytes =3D received number of "PCN_marking" encoded=20 >> >> packets during measurement period T. >> > >> >Again, is it per ingress-egress pair, or not? >>=20 >> georgios: Yes, it is correct! >>=20 >> > >> >> >> >> Now we defined the condition that the PCN_egress_node=20 >> changes state=20 >> >> from Normal state to admission control state. But how will the=20 >> >> PCN_egress_node change state from admission control state to flow=20 >> >> termination state. >> >> In order to explain this, it is imporatnt to note that each=20 >> >> PCN_interior_node that is in admission control state it=20 >> can PCN_mark=20 >> >> packets up to a value equal to Admission_offset_rate.=20 >> Furthermore, if=20 >> >> a PCN_interior_node receives incoming PCN_marked packets and is in=20 >> >> the addmission control state, it will not remark any=20 >> packets if the=20 >> >> excess rate is equal or lower than the=20 >> incoming_PCN_marking_rate, see=20 >> >> page 13. >> >> Furthermore, if we will consider as normal situations the=20 >> situations=20 >> >> that no ECMP occurs and that all flows belonging to the same=20 >> >> ingress-egress aggregate will use the same path from=20 >> PCN_ingress to=20 >> >> PCN_egress, this will mean that when the PCN_egress_node receives,=20 >> >> for the given ingress-egress aggregate an excess rate equal to=20 >> >> Admission_offset_rate it will have to change from=20 >> admission control=20 >> >> state to flow termination state. >> >> Thus in this case the second threshold, that in this case=20 >> is a rate=20 >> >> and not a percentage, can be calculated as follows: >> >> PCN_upper_egress_rate =3D PCN_lower_egress_rate +=20 >> Admission_offset_rate. >> > >> > >> >Here is where I am afraid I am fundamentally confused. >> >Pcn_lower-egress_rate seems to have beed redefined above as a=20 >> >percentage threshold, but it is an absolute rate again here. Perhaps=20 >> >you just mean that Pcn-lower-egress-rate =3D=20 >> pcn_lower_egress_percentage=20 >> >* total-measured-rate-of-this-ingress-egrees-aggregate/100? >> >So pcn-lower-egress-rate then is just the fraction of the=20 >> total rate of=20 >> >the ingress-egress aggregate corresponding to the configured=20 >> percentage=20 >> >threshold. Right? >>=20 >> Georgios: I think I could not explain and translate what I=20 >> wanted to do in the right way! >> A better way of expressing the need of using a rate of=20 >> percentage of the received PCN_marking encoded packets, would=20 >> be the following: >>=20 >> Pcn-lower-egress-rate =3D pcn_lower_egress_percentage *=20 >> Admission_offset_rate >>=20 >>=20 >> This means that event_A in Section 4.2.3 will be activated=20 >> when incoming_PCN_marking_rate > pcn_lower_egress_percentage=20 >> * Admission_offset_rate or when: >> incoming_PCN_marking_rate > Pcn-lower-egress-rate >>=20 >> This means that the PCN_egress_ node does not change to=20 >> admission control state when at least one PCN_marked packet=20 >> arrives, but it changes when the incoming_PHB_marking rate=20 >> equals a percentage of the Admission_offset_rate. This is due=20 >> to the fact that Admission_offset_rate is used by the=20 >> interioe nodes to change from admission control state to flow=20 >> termination state. >>=20 >> > >> >Assuming the above is correct, I cannot see how the system will work=20 >> >correctly. Consider the following example. A bottleneck link of=20 >> >capacity 100 mbps , pcn-termination-offset of 40mbps and=20 >> >pcn-admission-offset of 30 mbps is shared by 100 ingress-egress=20 >> >aggregates, each with the total pcn rate of 1 mbps. On this link=20 >> >pcn-upper-threshold is 60mbps, and pcn-lower-threshold is 30mbps. >> >Suppose further that all 100 ingress-egress aggregates go to=20 >> different=20 >> >egress nodes (and suppose for simplicity there is no other traffic=20 >> >going to these egresses). The bottleneck link is clearly in the=20 >> >termination state, as the total rate of pcn traffic on this=20 >> link is 100=20 >> >mbps which is above the pcn-upper-threshold. This means we want the=20 >> >egress nodes to somehow recognise this state and get into the=20 >> >termination mode. But it seems in this example the egresses=20 >> can't ever=20 >> >recognize that the system is in the termination mode. See below. >>=20 >>=20 >> > >> >Suppose pcn-lower-egress-percentage is set to w% (0> Then, each=20 >> >of these egresses will compute the (absolute)=20 >> pcn-lower-egress-rate as=20 >> >total pcn traffic of the ingress-egress aggregate it sees (which is 1 >> >mbps) times the pcn_lower_egress_percentage/100, so it will get=20 >> >pcn-lower-egress-rate=3D w* 0.01 Mbps. >> >According to your explanation above, it will then compute=20 >> >Pcn-upper-egress-rate=3Dpcn-lower-egress-rate+Admission-offset-rate=3D >> >w*0.01 mbps+30mbps >=3D 30 mbps (for all possible settings of w). >> > >> >So our pcn-upper-egress rate is always greater than 30 Mbps,=20 >> while the=20 >> >entire rate of pcn traffic the egress will ever see is *at=20 >> most* 1 Mbps=20 >> >(because it is plainly the total rate of the single ingress-egress=20 >> >aggreate that goes to this egress). So, regardless of the=20 >> actual amount=20 >> >of marked traffic in this egress-egress aggregate, the=20 >> absolute excess=20 >> >rate of this ingress-egress aggregate will never be above=20 >> 1mbps either. >> >In turn that means that the pcn-upper-egress-rate will NEVER be=20 >> >exceeded (because pcn-upper-egress-rate is so much larger than the=20 >> >total rate of the pcn traffic at the egress). In turn, that=20 >> appears to=20 >> >mean that in this scenario none of the egresses will ever end up in=20 >> >termination state, and so termination will never occur. That=20 >> does not seem right. >>=20 >>=20 >> Georgios: Thank you for the given example, which stimulated=20 >> me to better translate what I wanted to specify and what I=20 >> have written down. >> Please see above that I defned: >> Pcn-lower-egress-rate =3D pcn_lower_egress_percentage *=20 >> Admission_offset_rate >>=20 >> The situation that you described cannot (often) occur because=20 >> the difference between the admission control threshold and=20 >> flow termination threshold at the egress is equal to=20 >> Admission_offset_rate +/- multicongestion_error. Note that=20 >> the Admission_offset_rate is an absolute rate value. The=20 >> multicongestion_error is used to identify the bounds that=20 >> certain situations occur where the PCN_egreess_ node should=20 >> have been in flow termination state but it is not. Please see=20 >> previous discussions on this. >>=20 >> > >> >I would appreciate any help in clarifying this. >> > >> >Anna >> > >> > >> >> However, there are some corner cases, that mainly occur when the=20 >> >> different congestion points (admission control congested >> >> PCN_interior_nodes) on the same path are not=20 >> simulataneously starting=20 >> >> to be congested. Therefore we use the=20 >> multicongestion_error parameter=20 >> >> to identify the error bound that ocurs due to these corner cases.=20 >> >> Note that this error bound can be e.g., predefined ones=20 >> off line by=20 >> >> the operator, by studying the network topology and/or studying how=20 >> >> often such corner cases could occur and/or doing off line=20 >> >> measurements. Therefore we use: >> >> PCN_upper_rate_egress =3D PCN_lower_rate_egress +=20 >> Admission_offset_rate=20 >> >> +/- multicongestion_error >> >> >> >> * How the states of operation in PCN_interior_nodes are=20 >> being changed? >> >>=20 >> --------------------------------------------------------------------- >> >> This is explained on page 17, 18, by using Figure 4. >> >> >> >> Change from Normal state to Admission control state: event=20 >> A Occurs=20 >> >> when: >> >> Measured PHB rate > PCN_lower_rate >> >> >> >> Change from Admission control state to Flow Termination >> >> state: event B Occurs when: >> >> Measured PHB rate > PCN_upper_rate >> >> >> >> * How the states of operation in PCN_egress_nodes are=20 >> being changed? >> >>=20 >> --------------------------------------------------------------------- >> >> This is explained on page 21, 22. Note that the=20 >> description of event=20 >> >> A on page 21 has to be modified to avoid the confusions that were=20 >> >> caused up to now, see explanation given above. >> >> >> >> Change from Normal state to Admission control state: event=20 >> A Occurs=20 >> >> when: >> >> incoming_PCN_marking_rate/measured PHB rate) >=20 >> >> PCN_lower_percentage_egress As explained above: >> >> IF ((incoming_PCN_marking_rate/measured PHB rate) =3D >> >> PCN_lower_percentage_egress) >> >> THEN PCN_lower_egress_rate =3D incoming_PCN_marking_rate >> >> >> >> Change from Admission control state to flow termination >> >> state: event B Occurs when: >> >> incoming_PCN_marking_rate > PCN_upper_rate_egress >> >> >> >> It is important to note that also the explanation of event=20 >> C on page=20 >> >> 21, has to be modified to avoid the confusions that were=20 >> caused up to=20 >> >> now, see explanation given above: >> >> Change from Admission control state to Normal state: >> >> Occurs when: >> >> incoming_PCN_marking_rate/measured PHB rate) =3D<=20 >> >> PCN_lower_percentage_egress >> >> >> >> >> >> * Generated excess rate by an PCN_interior_node operating in=20 >> >> admission control state. >> >> ------------------------------------------------------------- >> >> >> >> The excess rate =3D signaled_overload_rate. >> >> The maximum excess rate that a PCN_interior_node can calculate in=20 >> >> admission control state is equal to Admission_offset_rate, see=20 >> >> explanation above. >> >> The number of bytes that are remarked,=20 >> signaled_remarked_bytes depend=20 >> >> on the value of calculated excess rate=20 >> (signaled_overload_rate), the=20 >> >> value of the Admission_offset_rate and the value of the=20 >> >> incoming_PCN_marking_rate, see pseudo code on page 13. >> >> >> >> Note that all packets that are passing through a congested=20 >> >> PCN_interior_node an are not being PCN_marked by the=20 >> >> PCN_interior_node have to be remarked using the=20 >> PCN_Affected_marking. >> >> >> >> >> >> Regarding probes, the probe packets that are passing through a=20 >> >> congested node are either PCN_marked or PCN_Affected_marked. >> >> >> >> * Generating excess rate by an PCN_interior_node operating in flow=20 >> >> termination state: >> >> ---------------------------------------------------------------- >> >> The excess rate =3D signaled_overload_rate. >> >> Note that the calculation of the signaled_overload_rate is=20 >> different=20 >> >> than in the situation that the PCN_Interior_node operates in=20 >> >> admission control state, see page 19 and 20. This is due=20 >> to the fact=20 >> >> that a sliding window is used to solve an undershooting=20 >> problem, see=20 >> >> discussion on page 19. >> >> The number of bytes that are remarked,=20 >> signaled_remarked_bytes depend=20 >> >> on the value of calculated excess rate=20 >> (signaled_overload_rate), the=20 >> >> value of the Termination_offset_rate and the value of the=20 >> >> incoming_PCN_marking_rate, and the see pseudo code on page 20. >> >> >> >> Note that all packets that are passing through a congested=20 >> >> PCN_interior_node an are not being PCN_marked by the=20 >> >> PCN_interior_node have to be remarked using the=20 >> PCN_Affected_marking. >> >> >> >> * Providing admission control at PCN_egress_nodes: >> >> -------------------------------------------------- >> >> When the PCN_egress_node is operating in admission control=20 >> state than=20 >> >> a flow that is requesting admission into the PCN domain can b=20 >> >> etreated in the following way: >> >> >> >> If no probing is used, the request for admission can be=20 >> accomplished=20 >> >> by using an external to PCN signaling protocol. >> >> In this case when the request arrives at a PCN_egress_node that=20 >> >> operates in admission control state then the request is=20 >> rejected. If=20 >> >> it operates in Normal state is accepted. >> >> >> >> If probing is used, the request for admission is accomplished by=20 >> >> using probe packets. In this case when the probe arrives at a=20 >> >> PCN_egress_node and it is either PCN_marking or=20 >> PCN_Affected_marking=20 >> >> encoded is rejected. Otherwise is accepted. >> >> Note that probes can only be used when=20 >> PCN_Affected_marking is appled=20 >> >> in whole PCN domain. Otherwise, the admission control=20 >> procedure will=20 >> >> work by using e.g., an external signaling protocol used between=20 >> >> PCN_ingress_nodes and PCN_egress_nodes. >> >> >> >> * Providing flow termination at PCN_egress_nodes: >> >> -------------------------------------------------- >> >> >> >> When a PCN_egress_node operates on flow termination it=20 >> calculates the=20 >> >> incoming excess rate (incoming_PCN_marking_rate). >> >> By using the excess rate the PCN_egress_node calclates the=20 >> numer of=20 >> >> flows that have to be terminated using the pseudo code=20 >> given on page=20 >> >> 23. Note that the PCN_egress_node has to maintain per flow=20 >> >> reservation states. >> >> The information contained in the per flow states is used for the=20 >> >> calculation of the flow that have to be terminated, see=20 >> pseudocode on=20 >> >> page 23. Note also that this pseudo code uses priority=20 >> clases, but it=20 >> >> operates when also no priority clases are used by setting=20 >> >> Maximum_priority =3D 0 Where, 0 =3D< priority_class =3D< Maximum_priori= ty >> >> >> >> >> >> >> >> Below, I am providing some answers to your comments! >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> > -----Original Message----- >> >> > From: Anna Charny (acharny) [mailto:acharny@cisco.com] >> >> > Sent: zondag 7 oktober 2007 21:40 >> >> > To: anurag.bhargava@ericsson.com; Georgios Karagiannis; >> >> Lars Westberg >> >> > (KI/EAB) >> >> > Cc: pcn@ietf.org >> >> > Subject: Questions on LC-PCN draft version 01 >> >> > >> >> > Dear authors of the LC-PCN draft, >> >> > >> >> > Am I right that there has been no response yet to Phil's=20 >> questions=20 >> >> > (attached at the end) on the new version of your draft below (I=20 >> >> > checked the pcn list and it does not seem it went there)? If the=20 >> >> > response was sent, can you resend it please? >> >> > >> >> > I have many of the same questions, and also a few=20 >> additional ones. >> >> > Here are some of them. >> >> > >> >> > 1) I do not understand whether pcn-lower_rate_egress and=20 >> >> > pcn_upper_rate ingress are expressed as a fraction of the >> >> total rate >> >> > (a unitless entity, analogous to CLE) or an absolute value >> >> (in bits or >> >> > bytes per second). The text seems to be contradictory on=20 >> this point: >> >> > >> >> > on page 8 it is a fraction: >> >> > >> >> > "PCN_lower_rate_egress =3D predefined percentage of received=20 >> >> > PCN_marking", >> >> > >> >> > while on page 13 it is an absolute rate: >> >> > >> >> > "If the incoming_PCN_marking_rate is higher than a preconfigured=20 >> >> > PCN_lower_rate_egress, ..." where the >> >> > >> >> > (Incoming_PCN_marking_rate is clearly defined as abslolue >> >> rate on page >> >> > 12: "Where the "incoming_PCN_marking_rate" is calculated=20 >> as follows: >> >> > incoming_PCN_marking_rate =3D (received number of "PCN_marking" >> >> > DSCP during T)* N)/T;) >> >> >> >> Georgios: You are right, please see discussion above on paragaph=20 >> >> denoted >> >> as: >> >> "Setting the thresholds at PCN_egress_nodes". >> >> >> >> >> >> We use a percentage of received PCN_marking encoded packets in=20 >> >> proportion to total rate of received packets to define when a=20 >> >> PCN_egres_node goes from Normal state to admission control=20 >> state. In=20 >> >> this version of the draft we call this value as:=20 >> >> PCN_lower_rate_egress. This is wrong. >> >> What we should say is: >> >> PCN_lower_rate_egress: is equal to incoming_PCN_marking_rate, when: >> >> (incoming_PCN_marking_rate/measured PHB rate) =3D to a preconfigured=20 >> >> percentage, say 1%. From now on I denote this percentage as: >> >> PCN_lower_percentage_egress. >> >> Thus we can say that a PCN_egress_node changes from Normal=20 >> state to=20 >> >> admission control state when=20 >> incoming_PCN_marking_rate/measured PHB=20 >> >> rate > PCN_lower_percentage_egress. >> >> Where, >> >> incoming_PCN_marking_rate =3D N * input_PCN_marking_bytes/T Where,=20 >> >> input_PCN_marking_bytes =3D received number of "PCN_marking" encoded=20 >> >> packets during measurement period T. >> >> >> >> >> >> > >> >> > 2) Regarding the question above, >> >> > >> >> > * if pcn_lower_rate_eggress (and pcn_uppre-rate_egress) are >> >> absolute >> >> > rates, then how do you choose that abslolute rate value,=20 >> given that=20 >> >> > the rates of ingress-egress aggregates at the same=20 >> ingress may be=20 >> >> > vastly different in magnitude? >> >> >> >> Georgios: See explanation above! Hopefully this=20 >> misundersatnding is=20 >> >> also solved! >> >> >> >> > >> >> > *if, however, pcn_lower_rate_eggress (and >> >> > pcn_uppre-rate_egress) are ratios, then what is the guidance of=20 >> >> > setting these ratios so that the egress can always tell=20 >> admission=20 >> >> > state from the termination state, given that the same >> >> marking is used >> >> > for admission and termination marking? >> >> > Specifically, suppose at the bottleneck pcn_lower_threshold >> >> is set to >> >> > X, and pcn_upper_threshold to aX with some a>1 (assume=20 >> for example=20 >> >> > a=3D1.5). >> >> > How does the egress node tell between the conditions=20 >> when the total=20 >> >> > pcn traffic on the bottleneck is 1.1X (which is the state wnen=20 >> >> > admission is needed but termination is not, - in this case >> >> > (1.1X-X)/X=3D0.1 traffic is pcn-marked), and the case when=20 >> the total=20 >> >> > traffic on the bottleneck is 1.1aX, which is the case when >> >> termination >> >> > is needed, and (1.1ax-ax)/ax=3D0.1), if in both cases the >> >> ratio between >> >> > marked and total traffic is the same value 0.1? >> >> >> >> Georgios: In order to solve the above described problems, the=20 >> >> thresholds used in the PCN_interior_nodes and PCN_egress_nodes are >> >> using: >> >> the Admission_offset_rate that is an absolute rate value=20 >> which is set=20 >> >> equal into whole PCN domain. >> >> >> >> Note that the maximum excess rate that a PCN_interior_node can=20 >> >> calculate in admission control state is equal to=20 >> >> Admission_offset_rate. If the excess rate is higher than the=20 >> >> Admission_offset_rate then the node changes from admission control=20 >> >> state to flow termination state. >> >> >> >> Please see all details that I have provided at the top of=20 >> this email.=20 >> >> In particular, check the paragraphs denoted above as: >> >> "Setting the thresholds at PCN_interior_nodes" >> >> "Setting the thresholds at PCN_egress_nodes" >> >> "Generated excess rate by an PCN_interior_node operating=20 >> in admission=20 >> >> control state" >> >> >> >> >> >> >> >> > >> >> > 3) On page 9, multicongestion error error is introduced, >> >> but the text >> >> > is silent on how this error is defined and what is it set >> >> to (if it is >> >> > a configuration parameter), or how it is measured (if it is >> >> something >> >> > that is being measured). Can you explain, please? >> >> >> >> Georgios: >> >> Please see the paragraph denoted above as "Setting the=20 >> thresholds at=20 >> >> PCN_egress_nodes" >> >> >> >> >> >> Best regards, >> >> Georgios >> >> >> >> >> >> > >> >> > I have some more questions, including sharing those asked >> >> by Phil in >> >> > the attached message), but I will stop now, as I hope >> >> clarifications >> >> > on the above (and Phil's) questions will help understand=20 >> the draft=20 >> >> > better... >> >> > >> >> > Thank you in advance for clarifying all this. >> >> > >> >> > Anna >> >> >>=20 _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Fri Nov 02 17:41:09 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Io4Fy-0006F1-BM; Fri, 02 Nov 2007 17:40:46 -0400 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Io4Fx-0006Er-7K for pcn-confirm+ok@megatron.ietf.org; Fri, 02 Nov 2007 17:40:45 -0400 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Io4Fw-0006EV-Pd for pcn@ietf.org; Fri, 02 Nov 2007 17:40:44 -0400 Received: from sj-iport-3-in.cisco.com ([171.71.176.72] helo=sj-iport-3.cisco.com) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1Io4Ft-0002kQ-Bf for pcn@ietf.org; Fri, 02 Nov 2007 17:40:44 -0400 X-IronPort-AV: E=Sophos;i="4.21,364,1188802800"; d="scan'208";a="542730568" Received: from sj-dkim-1.cisco.com ([171.71.179.21]) by sj-iport-3.cisco.com with ESMTP; 02 Nov 2007 14:40:19 -0700 Received: from sj-core-2.cisco.com (sj-core-2.cisco.com [171.71.177.254]) by sj-dkim-1.cisco.com (8.12.11/8.12.11) with ESMTP id lA2LeIs3029614; Fri, 2 Nov 2007 14:40:18 -0700 Received: from xbh-rtp-201.amer.cisco.com (xbh-rtp-201.cisco.com [64.102.31.12]) by sj-core-2.cisco.com (8.12.10/8.12.6) with ESMTP id lA2Le2Zk019132; Fri, 2 Nov 2007 21:40:17 GMT Received: from xmb-rtp-203.amer.cisco.com ([64.102.31.20]) by xbh-rtp-201.amer.cisco.com with Microsoft SMTPSVC(6.0.3790.1830); Fri, 2 Nov 2007 17:40:16 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Fri, 2 Nov 2007 17:40:14 -0400 Message-ID: In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Questions on LC-PCN draft version 01 Thread-Index: Acgdi7YZ6QrHYYvMRceiiE0GjQ2pjAAC642w From: "Anna Charny (acharny)" To: "Georgios Karagiannis" , "Lars Westberg (KI/EAB)" , X-OriginalArrivalTime: 02 Nov 2007 21:40:16.0809 (UTC) FILETIME=[F5521190:01C81D98] X-TM-AS-Product-Ver: SMEX-8.0.0.1181-5.000.1023-15520.002 X-TM-AS-Result: No--29.242200-8.000000-2 X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No DKIM-Signature: v=0.5; a=rsa-sha256; q=dns/txt; l=35471; t=1194039618; x=1194903618; c=relaxed/simple; s=sjdkim1004; h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version; d=cisco.com; i=acharny@cisco.com; z=From:=20=22Anna=20Charny=20(acharny)=22=20 |Subject:=20RE=3A=20Questions=20on=20=20LC-PCN=20draft=20version=2001 |Sender:=20; bh=lP/PlygJ5GDSihCCB6UiL2k0W/HZHVSa8YHKtxFXNN0=; b=HeQk+4wg4zZlMSJAyV/1LzGg2GKjQioICdt3KPlUKXuoLoUo7OUhmahMpDGZN71S91OdLJn3 K8u5VvjNQ/7Q42iUl8zu6TepVeJh2b8x3ao9L6rSgP38V0DVDt1XzSt1y6PLMq4jbLphkvr/A5 oiPMk5igdPXxNqWwmi8OyTfyw=; Authentication-Results: sj-dkim-1; header.From=acharny@cisco.com; dkim=pass ( sig from cisco.com/sjdkim1004 verified; ); X-Spam-Score: 0.0 (/) X-Scan-Signature: 06f60f0744e48a3e4358b40a71e32d86 Cc: pcn@ietf.org Subject: [PCN] RE: Questions on LC-PCN draft version 01 X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Georgios, You are saying that the example I provided was a worst case scenario that cannot happen in practice. =20 Why is that? All that is required to construct a similar example is to have a bottleneck which multiplexes traffic from many ingresses to one (or a small number of) egresses, and these aggregates can together load the bottleneck link. This scenario will easily result in the situation when the admission-offset on the bottleneck (i.e. the difference between the admission and termination thresholds) is large compared to the individual ingress-egress aggregate. Why do you think that is a corner case?=20 Anna=20 =20 > -----Original Message----- > From: Georgios Karagiannis [mailto:karagian@cs.utwente.nl]=20 > Sent: Friday, November 02, 2007 4:05 PM > To: Anna Charny (acharny); Lars Westberg (KI/EAB);=20 > anurag.bhargava@ericsson.com > Cc: pcn@ietf.org > Subject: RE: Questions on LC-PCN draft version 01 >=20 > Hi Anna >=20 > Please see in line! >=20 > Best regards, > Georgios >=20 > On 11/2/2007, "Anna Charny (acharny)" wrote: >=20 > >Hi Georgios, > > > >OK, trying again... > > > >Something still seems wrong.=20 > > > >First wanted to confirm that I now got the setting of the thresholds=20 > >right (the following collected from your various responses): > > > >NOTE: I am ignoring multi-congestion error because I am=20 > still trying=20 > >to figure out how it works in a single bottleneck case. > > > >Pcn-lower-egress-percentage is some configured value (%) > > > >Pcn-lower-egress-rate =3D pcn-lower-egress-percentage *=20 > >admission-termination-rate/100 (I assume the division by 100 is > >necessary?) > > > >Pcn-upper-egress-rate=3Dpcn-lower-egress-rate+Admission-offset-rate > > > >Admisison-offset-rate is a global parameter that has the=20 > meaning of the=20 > >difference between the pcn-upper-rate and pcn-lower-rate at all=20 > >interior nodes (which is assumed the same in all cases). > > > >Egress goes into termination mode when it sees the rate of marked=20 > >packets of a given ingress exceeding the pcn-upper-egress-rate. > > > >So far so good? > > > >Assuming it is all correct, my example where none of the=20 > egresses can=20 > >ever go into the termination mode remains valid, I believe. Let me=20 > >restate the example. > > > >Consider the following example. A bottleneck link of=20 > capacity 100 mbps=20 > >, pcn-termination-offset of 40mbps and pcn-admission-offset=20 > of 30 mbps=20 > >is shared by 100 ingress-egress aggregates, each with the total pcn=20 > >rate of > >1 mbps. On this link pcn-upper-threshold is 60mbps, and=20 > >pcn-lower-threshold is 30mbps. Suppose further that all 100=20 > >ingress-egress aggregates go to different egress nodes (and=20 > suppose for=20 > >simplicity there is no other traffic going to these=20 > egresses; this last=20 > >assumption is irrelevant because the egress measurement is per=20 > >ingress-egress pair anyway, but makes it convenient to imagine). The=20 > >bottleneck link is clearly in the termination state, as the=20 > total rate=20 > >of pcn traffic on this link is 100 mbps which is above the=20 > >pcn-upper-threshold. > > > >This means we want the egress nodes to somehow recognise=20 > this state and=20 > >get into the termination mode. But it seems in this example the=20 > >egresses > > > >can't ever recognize that the system is in the termination mode. See=20 > >below. >=20 > Georgios: I think that the example that you provide is an=20 > wors case scenario. > First of all the Admission_offset_rate is too high. > This Admission_offset_rate should be set such that siuations=20 > as you describe will not occur. >=20 >=20 > > > >Suppose pcn-lower-egress-percentage is set to w% (0 >u=3Dw/100. Then, each of these egresses will compute the=20 > >pcn-lower-egress-rate=3Du*admission-offset-rate > > > >and > > > >pcn-upper-egress-rate=3Dpcn-lower-rate+admission-offset-rate =3D=20 > >(u+1)*admission-offset-rate > admission-offset-rate =3D 30Mbps > > > > > >So our pcn-upper-egress rate is always greater than 30 Mbps,=20 > while the=20 > >entire rate of pcn traffic the egress will ever see is *at > >most* 1 Mbps (because it is plainly the total rate of the single=20 > >ingress-egress aggreate that goes to this egress). So, regardless of=20 > >the > > > >actual amount of marked traffic in this egress-egress aggregate, the=20 > >absolute excess rate of this ingress-egress aggregate will never be=20 > >above 1mbps either.In turn that means that the pcn-upper-egress-rate=20 > >will NEVER be exceeded (because pcn-upper-egress-rate is so=20 > much larger=20 > >than the total rate of the pcn traffic at the egress). In=20 > turn, that=20 > >appears to mean that in this scenario none of the egresses will ever=20 > >end up in termination state, and so termination will never=20 > occur. That > >(still) does not seem right. > > > > > >So again, even with the corrected definition of pcn-lower-thershold,=20 > >this example seems to imply a rather fundamental flaw in the=20 > algorithm. > > > >You say, in your response to the previous example, that > > > >>the situation that you described cannot (often) occur because the=20 > >>difference between the admission control threshold and flow=20 > >>termination threshold at the egress is equal to =20 > Admission_offset_rate=20 > >>+/- multicongestion_error. > > > >I believe that multi-congestion error is irrelevant here, because in=20 > >the considered example there is only a single congestion point. > > > >>Note that > >> the Admission_offset_rate is an absolute rate value. The =20 > >>multicongestion_error is used to identify the bounds that certain=20 > >>situations occur where the PCN_egreess_ node should have=20 > been in flow=20 > >>termination state but it is not. Please see previous=20 > discussions on=20 > >>this. > > > >I do not understand the relevance of the multiocongestion=20 > error in this=20 > >case, or rules for its setting at all, I am afraid. Do we need=20 > >multi-congestion error (which I do not know how to set=20 > anyway) to deal=20 > >with a single congestion point also? What should it be set=20 > to? Do you=20 > >need to examine a configuration AND all ingress-egress=20 > aggregare rates=20 > >AND all possible failures to set it correctly? >=20 >=20 > Georgios: You are right that the muli-congestion-error does=20 > not play a role, but the selection of the=20 > Admission_offset_rate plays in this situation a signifficat=20 > role and it should be selected such that wores case=20 > scenarios, such as the one described by you do not occur+ >=20 >=20 > >Best, > >Anna > >> -----Original Message----- > >> From: Georgios Karagiannis [mailto:karagian@cs.utwente.nl] > >> Sent: Friday, November 02, 2007 12:20 PM > >> To: Anna Charny (acharny); anurag.bhargava@ericsson.com; Lars=20 > >> Westberg (KI/EAB) > >> Cc: pcn@ietf.org > >> Subject: RE: Questions on LC-PCN draft version 01 > >>=20 > >> Hi Anna > >>=20 > >> Thank you very much for your comments! > >>=20 > >> Please see in line! > >>=20 > >>=20 > >> On 11/1/2007, "Anna Charny (acharny)" wrote: > >>=20 > >> > > >> >Hi Georgios, > >> > > >> >A few more clarification questions: > >> > > >> >> -----Original Message----- > >> >> From: Georgios Karagiannis [mailto:karagian@cs.utwente.nl] > >> >> Sent: Thursday, November 01, 2007 2:35 AM > >> >> To: Anna Charny (acharny); anurag.bhargava@ericsson.com; Lars=20 > >> >> Westberg (KI/EAB) > >> >> Cc: pcn@ietf.org > >> >> Subject: Re: Questions on LC-PCN draft version 01 > >> >> > >> >> Hi Anna > >> >> > >> >> Thank you very much for your comments! > >> >> Before answering to your comments, see in line below, I > >> would like to > >> >> explain in an abstract way the LC-PCN algorithm. > >> >> > >> >> * Setting the thresholds at PCN_interior_nodes: > >> >> ----------------------------------------------- > >> >> In order to calculate the PCN_upper_rate we use two parameters: > >> >> Maximum PHB capacity: that is the maximum capacity that can be=20 > >> >> supported by a PCN_interior_node > >> >> Termination_offset_rate: that is an absolute rate value > >> that should > >> >> be set equal into all PCN_interior_nodes. Note that=20 > this value is=20 > >> >> used by PCN_interior_nodes to calculate their > >> PCN_upper_rate and also > >> >> during the situation that a PCN_interior_node is in flow > >> termination > >> >> state and it receives PCN_marked packets. Please see > >> pseudo code on > >> >> page 20. > >> >> This value must be set equal into all=20 > PCN_interior_nodes such that=20 > >> >> all these nodes will know when to take into account the=20 > incoming=20 > >> >> PCN_marked packets and when not. > >> >> > >> >> The PCN_upper_rate is then found as: > >> >> PCN_upper_rate =3D "Maximum PHB capacity" -=20 > Termination_offset_rate > >> >> > >> > > >> >So if you have a link of 10 Mbps and a link of 40 Mbps=20 > (say each is=20 > >> >allowed to use all bandwidth for PCN, for simplicity), then > >> they both > >> >use the same *absolote* value of the termination-offset-rate?. > >>=20 > >> Georgios: Yes, you are right! This has to do with the=20 > importance of=20 > >> using the Termination_offset_rate. We will motivate this in the=20 > >> following version of the draft. > >>=20 > >> > So If I > >> >wanted to have PCN-upper-rate 20% below my link capacity on > >> all links, > >> >I would not be able to do so with this approach? A larger=20 > link would=20 > >> >have a smaller relative safety margin, then... > >>=20 > >> Georgios: Yes, this can be considered as a disadvantage.=20 > >> However, there might be other ways of preconfiguring each=20 > >> PCN_interior_node to know the termination_offset rate used by each=20 > >> neighbour PCN_interior_node. Then the pseudocode on page=20 > 20 will have=20 > >> to use the Termination_offset_rate associated with the=20 > incoming link=20 > >> from where the incoming PCN_marked packets are arriving. Then this=20 > >> constraint can be avoided. > >>=20 > >>=20 > >> >Same comment for the > >> >difference between admission and termination - the relative > >> difference > >> >between admission and termination threshods (Pcn-lower-rate and > >> >pcn-upper-rate) is global, so the larger links have a > >> smaller relative > >> >difference between admission and termination thresholds. Right? > >>=20 > >> Georgios: No, because the Admission_offset_rate is an=20 > absolute value! > >> So this difference is globally equal. > >>=20 > >> > > >> >> The PCN_lower_rate is configured in all PCN_interior-nodes > >> and it can > >> >> be calculated in the following way: > >> >> PCN_lower_rate =3D PCN_upper_rate - Admission_offset_rate > >> >> > >> >> The Admission_offset_rate is an absolute rate value and it > >> is equal > >> >> in all PCN_interior_nodes and PCN_egress_nodes. Note that > >> this value > >> >> is used by PCN_interior_nodes to calculate their > >> PCN_lower_rate and > >> >> the PCN_egress_nodes to calculate their PCN_upper_rate_egress.=20 > >> >> Furthermore, this value is used by the PCN_interior_nodes > >> also during > >> >> the situation that a PCN_interior_node is in admission > >> control state > >> >> and it receives PCN_marked packets. Please see pseudo=20 > code on page=20 > >> >> 13. > >> >> This value must be set equal into all=20 > PCN_interior_nodes such that=20 > >> >> all these nodes will know when to take into account the=20 > incoming=20 > >> >> PCN_marked packets and when not. Note that a=20 > PCN_interior_node can=20 > >> >> PCN_mark packets up to an excess rate equal to the=20 > >> >> Admission_offset_rate. If the exess rate in an > >> PCN_interior_node is > >> >> higher than the Admission_offset_rate, then the=20 > PCN_interior_node=20 > >> >> changes state from admission control state to flow > >> termination state. > >> > > >> >OK. > >> > > >> >> > >> >> * Setting the thresholds at PCN_egress_nodes: > >> >> --------------------------------------------- > >> >> The question is how to calculate the threshold that=20 > defines when a=20 > >> >> PCN_egress_node goes into the admission control state. > >> >> One way to do that is to consider that when the PCN_egress_node=20 > >> >> receives a PCN_marked packet it will mean that at least one=20 > >> >> PCN_interior_node started to be admission control congested and=20 > >> >> therefore it will go from Normal state to admission > >> control state. Of > >> >> course this will somehow might provide some errors,=20 > because there=20 > >> >> might be situations that this consideration might be to > >> conservative.=20 > >> >> Therefore, we use a percentage of received PCN_marking encoded=20 > >> >> packets in proportion to total rate of received=20 > packets. In this=20 > >> >> version of the draft we call this value as: > >> >> PCN_lower_rate_egress. This is wrong. > >> >> What we should say is: > >> >> PCN_lower_rate_egress: is equal to=20 > incoming_PCN_marking_rate, when: > >> >> (incoming_PCN_marking_rate/measured PHB rate) =3D to a=20 > preconfigured=20 > >> >> percentage, say 1%. From now on I denote this percentage as: > >> >> PCN_lower_percentage_egress. > >> > > >> >In the draft, pcn_lower- and -upper_rate_egress seem to be > >> defined on a > >> >per *ingress-egress-pair basis*. > >> >Is that correct? > >>=20 > >> georgios: Yes, it is correct! > >> > > >> >> Thus we can say that a PCN_egress_node changes from Normal > >> state to > >> >> admission control state when > >> incoming_PCN_marking_rate/measured PHB > >> >> rate > PCN_lower_percentage_egress. > >> >> Where, > >> >> incoming_PCN_marking_rate =3D N *=20 > input_PCN_marking_bytes/T Where,=20 > >> >> input_PCN_marking_bytes =3D received number of=20 > "PCN_marking" encoded=20 > >> >> packets during measurement period T. > >> > > >> >Again, is it per ingress-egress pair, or not? > >>=20 > >> georgios: Yes, it is correct! > >>=20 > >> > > >> >> > >> >> Now we defined the condition that the PCN_egress_node > >> changes state > >> >> from Normal state to admission control state. But how will the=20 > >> >> PCN_egress_node change state from admission control=20 > state to flow=20 > >> >> termination state. > >> >> In order to explain this, it is imporatnt to note that each=20 > >> >> PCN_interior_node that is in admission control state it > >> can PCN_mark > >> >> packets up to a value equal to Admission_offset_rate.=20 > >> Furthermore, if > >> >> a PCN_interior_node receives incoming PCN_marked=20 > packets and is in=20 > >> >> the addmission control state, it will not remark any > >> packets if the > >> >> excess rate is equal or lower than the > >> incoming_PCN_marking_rate, see > >> >> page 13. > >> >> Furthermore, if we will consider as normal situations the > >> situations > >> >> that no ECMP occurs and that all flows belonging to the same=20 > >> >> ingress-egress aggregate will use the same path from > >> PCN_ingress to > >> >> PCN_egress, this will mean that when the=20 > PCN_egress_node receives,=20 > >> >> for the given ingress-egress aggregate an excess rate equal to=20 > >> >> Admission_offset_rate it will have to change from > >> admission control > >> >> state to flow termination state. > >> >> Thus in this case the second threshold, that in this case > >> is a rate > >> >> and not a percentage, can be calculated as follows: > >> >> PCN_upper_egress_rate =3D PCN_lower_egress_rate + > >> Admission_offset_rate. > >> > > >> > > >> >Here is where I am afraid I am fundamentally confused. > >> >Pcn_lower-egress_rate seems to have beed redefined above as a=20 > >> >percentage threshold, but it is an absolute rate again=20 > here. Perhaps=20 > >> >you just mean that Pcn-lower-egress-rate =3D > >> pcn_lower_egress_percentage > >> >* total-measured-rate-of-this-ingress-egrees-aggregate/100? > >> >So pcn-lower-egress-rate then is just the fraction of the > >> total rate of > >> >the ingress-egress aggregate corresponding to the configured > >> percentage > >> >threshold. Right? > >>=20 > >> Georgios: I think I could not explain and translate what I=20 > wanted to=20 > >> do in the right way! > >> A better way of expressing the need of using a rate of=20 > percentage of=20 > >> the received PCN_marking encoded packets, would be the following: > >>=20 > >> Pcn-lower-egress-rate =3D pcn_lower_egress_percentage *=20 > >> Admission_offset_rate > >>=20 > >>=20 > >> This means that event_A in Section 4.2.3 will be activated when=20 > >> incoming_PCN_marking_rate > pcn_lower_egress_percentage > >> * Admission_offset_rate or when: > >> incoming_PCN_marking_rate > Pcn-lower-egress-rate > >>=20 > >> This means that the PCN_egress_ node does not change to admission=20 > >> control state when at least one PCN_marked packet arrives, but it=20 > >> changes when the incoming_PHB_marking rate equals a=20 > percentage of the=20 > >> Admission_offset_rate. This is due to the fact that=20 > >> Admission_offset_rate is used by the interioe nodes to change from=20 > >> admission control state to flow termination state. > >>=20 > >> > > >> >Assuming the above is correct, I cannot see how the=20 > system will work=20 > >> >correctly. Consider the following example. A bottleneck link of=20 > >> >capacity 100 mbps , pcn-termination-offset of 40mbps and=20 > >> >pcn-admission-offset of 30 mbps is shared by 100 ingress-egress=20 > >> >aggregates, each with the total pcn rate of 1 mbps. On this link=20 > >> >pcn-upper-threshold is 60mbps, and pcn-lower-threshold is 30mbps. > >> >Suppose further that all 100 ingress-egress aggregates go to > >> different > >> >egress nodes (and suppose for simplicity there is no=20 > other traffic=20 > >> >going to these egresses). The bottleneck link is clearly in the=20 > >> >termination state, as the total rate of pcn traffic on this > >> link is 100 > >> >mbps which is above the pcn-upper-threshold. This means=20 > we want the=20 > >> >egress nodes to somehow recognise this state and get into the=20 > >> >termination mode. But it seems in this example the egresses > >> can't ever > >> >recognize that the system is in the termination mode. See below. > >>=20 > >>=20 > >> > > >> >Suppose pcn-lower-egress-percentage is set to w% (0 >> Then, each > >> >of these egresses will compute the (absolute) > >> pcn-lower-egress-rate as > >> >total pcn traffic of the ingress-egress aggregate it sees=20 > (which is=20 > >> >1 > >> >mbps) times the pcn_lower_egress_percentage/100, so it will get=20 > >> >pcn-lower-egress-rate=3D w* 0.01 Mbps. > >> >According to your explanation above, it will then compute=20 > >> = >Pcn-upper-egress-rate=3Dpcn-lower-egress-rate+Admission-offset-rate=3D > >> >w*0.01 mbps+30mbps >=3D 30 mbps (for all possible settings of w). > >> > > >> >So our pcn-upper-egress rate is always greater than 30 Mbps, > >> while the > >> >entire rate of pcn traffic the egress will ever see is *at > >> most* 1 Mbps > >> >(because it is plainly the total rate of the single=20 > ingress-egress=20 > >> >aggreate that goes to this egress). So, regardless of the > >> actual amount > >> >of marked traffic in this egress-egress aggregate, the > >> absolute excess > >> >rate of this ingress-egress aggregate will never be above > >> 1mbps either. > >> >In turn that means that the pcn-upper-egress-rate will NEVER be=20 > >> >exceeded (because pcn-upper-egress-rate is so much larger=20 > than the=20 > >> >total rate of the pcn traffic at the egress). In turn, that > >> appears to > >> >mean that in this scenario none of the egresses will ever=20 > end up in=20 > >> >termination state, and so termination will never occur. That > >> does not seem right. > >>=20 > >>=20 > >> Georgios: Thank you for the given example, which stimulated me to=20 > >> better translate what I wanted to specify and what I have written=20 > >> down. > >> Please see above that I defned: > >> Pcn-lower-egress-rate =3D pcn_lower_egress_percentage *=20 > >> Admission_offset_rate > >>=20 > >> The situation that you described cannot (often) occur because the=20 > >> difference between the admission control threshold and flow=20 > >> termination threshold at the egress is equal to=20 > Admission_offset_rate=20 > >> +/- multicongestion_error. Note that the=20 > Admission_offset_rate is an=20 > >> absolute rate value. The multicongestion_error is used to identify=20 > >> the bounds that certain situations occur where the=20 > PCN_egreess_ node=20 > >> should have been in flow termination state but it is not.=20 > Please see=20 > >> previous discussions on this. > >>=20 > >> > > >> >I would appreciate any help in clarifying this. > >> > > >> >Anna > >> > > >> > > >> >> However, there are some corner cases, that mainly occur=20 > when the=20 > >> >> different congestion points (admission control congested > >> >> PCN_interior_nodes) on the same path are not > >> simulataneously starting > >> >> to be congested. Therefore we use the > >> multicongestion_error parameter > >> >> to identify the error bound that ocurs due to these=20 > corner cases.=20 > >> >> Note that this error bound can be e.g., predefined ones > >> off line by > >> >> the operator, by studying the network topology and/or=20 > studying how=20 > >> >> often such corner cases could occur and/or doing off line=20 > >> >> measurements. Therefore we use: > >> >> PCN_upper_rate_egress =3D PCN_lower_rate_egress + > >> Admission_offset_rate > >> >> +/- multicongestion_error > >> >> > >> >> * How the states of operation in PCN_interior_nodes are > >> being changed? > >> >>=20 > >>=20 > --------------------------------------------------------------------- > >> >> This is explained on page 17, 18, by using Figure 4. > >> >> > >> >> Change from Normal state to Admission control state: event > >> A Occurs > >> >> when: > >> >> Measured PHB rate > PCN_lower_rate > >> >> > >> >> Change from Admission control state to Flow Termination > >> >> state: event B Occurs when: > >> >> Measured PHB rate > PCN_upper_rate > >> >> > >> >> * How the states of operation in PCN_egress_nodes are > >> being changed? > >> >>=20 > >>=20 > --------------------------------------------------------------------- > >> >> This is explained on page 21, 22. Note that the > >> description of event > >> >> A on page 21 has to be modified to avoid the confusions=20 > that were=20 > >> >> caused up to now, see explanation given above. > >> >> > >> >> Change from Normal state to Admission control state: event > >> A Occurs > >> >> when: > >> >> incoming_PCN_marking_rate/measured PHB rate) >=20 > >> >> PCN_lower_percentage_egress As explained above: > >> >> IF ((incoming_PCN_marking_rate/measured PHB rate) =3D > >> >> PCN_lower_percentage_egress) > >> >> THEN PCN_lower_egress_rate =3D incoming_PCN_marking_rate > >> >> > >> >> Change from Admission control state to flow termination > >> >> state: event B Occurs when: > >> >> incoming_PCN_marking_rate > PCN_upper_rate_egress > >> >> > >> >> It is important to note that also the explanation of event > >> C on page > >> >> 21, has to be modified to avoid the confusions that were > >> caused up to > >> >> now, see explanation given above: > >> >> Change from Admission control state to Normal state: > >> >> Occurs when: > >> >> incoming_PCN_marking_rate/measured PHB rate) =3D<=20 > >> >> PCN_lower_percentage_egress > >> >> > >> >> > >> >> * Generated excess rate by an PCN_interior_node operating in=20 > >> >> admission control state. > >> >> ------------------------------------------------------------- > >> >> > >> >> The excess rate =3D signaled_overload_rate. > >> >> The maximum excess rate that a PCN_interior_node can=20 > calculate in=20 > >> >> admission control state is equal to Admission_offset_rate, see=20 > >> >> explanation above. > >> >> The number of bytes that are remarked, > >> signaled_remarked_bytes depend > >> >> on the value of calculated excess rate > >> (signaled_overload_rate), the > >> >> value of the Admission_offset_rate and the value of the=20 > >> >> incoming_PCN_marking_rate, see pseudo code on page 13. > >> >> > >> >> Note that all packets that are passing through a congested=20 > >> >> PCN_interior_node an are not being PCN_marked by the=20 > >> >> PCN_interior_node have to be remarked using the > >> PCN_Affected_marking. > >> >> > >> >> > >> >> Regarding probes, the probe packets that are passing through a=20 > >> >> congested node are either PCN_marked or PCN_Affected_marked. > >> >> > >> >> * Generating excess rate by an PCN_interior_node=20 > operating in flow=20 > >> >> termination state: > >> >> ---------------------------------------------------------------- > >> >> The excess rate =3D signaled_overload_rate. > >> >> Note that the calculation of the signaled_overload_rate is > >> different > >> >> than in the situation that the PCN_Interior_node operates in=20 > >> >> admission control state, see page 19 and 20. This is due > >> to the fact > >> >> that a sliding window is used to solve an undershooting > >> problem, see > >> >> discussion on page 19. > >> >> The number of bytes that are remarked, > >> signaled_remarked_bytes depend > >> >> on the value of calculated excess rate > >> (signaled_overload_rate), the > >> >> value of the Termination_offset_rate and the value of the=20 > >> >> incoming_PCN_marking_rate, and the see pseudo code on page 20. > >> >> > >> >> Note that all packets that are passing through a congested=20 > >> >> PCN_interior_node an are not being PCN_marked by the=20 > >> >> PCN_interior_node have to be remarked using the > >> PCN_Affected_marking. > >> >> > >> >> * Providing admission control at PCN_egress_nodes: > >> >> -------------------------------------------------- > >> >> When the PCN_egress_node is operating in admission control > >> state than > >> >> a flow that is requesting admission into the PCN domain can b=20 > >> >> etreated in the following way: > >> >> > >> >> If no probing is used, the request for admission can be > >> accomplished > >> >> by using an external to PCN signaling protocol. > >> >> In this case when the request arrives at a PCN_egress_node that=20 > >> >> operates in admission control state then the request is > >> rejected. If > >> >> it operates in Normal state is accepted. > >> >> > >> >> If probing is used, the request for admission is=20 > accomplished by=20 > >> >> using probe packets. In this case when the probe arrives at a=20 > >> >> PCN_egress_node and it is either PCN_marking or > >> PCN_Affected_marking > >> >> encoded is rejected. Otherwise is accepted. > >> >> Note that probes can only be used when > >> PCN_Affected_marking is appled > >> >> in whole PCN domain. Otherwise, the admission control > >> procedure will > >> >> work by using e.g., an external signaling protocol used between=20 > >> >> PCN_ingress_nodes and PCN_egress_nodes. > >> >> > >> >> * Providing flow termination at PCN_egress_nodes: > >> >> -------------------------------------------------- > >> >> > >> >> When a PCN_egress_node operates on flow termination it > >> calculates the > >> >> incoming excess rate (incoming_PCN_marking_rate). > >> >> By using the excess rate the PCN_egress_node calclates the > >> numer of > >> >> flows that have to be terminated using the pseudo code > >> given on page > >> >> 23. Note that the PCN_egress_node has to maintain per flow=20 > >> >> reservation states. > >> >> The information contained in the per flow states is=20 > used for the=20 > >> >> calculation of the flow that have to be terminated, see > >> pseudocode on > >> >> page 23. Note also that this pseudo code uses priority > >> clases, but it > >> >> operates when also no priority clases are used by setting=20 > >> >> Maximum_priority =3D 0 Where, 0 =3D< priority_class =3D<=20 > >> >> Maximum_priority > >> >> > >> >> > >> >> > >> >> Below, I am providing some answers to your comments! > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > -----Original Message----- > >> >> > From: Anna Charny (acharny) [mailto:acharny@cisco.com] > >> >> > Sent: zondag 7 oktober 2007 21:40 > >> >> > To: anurag.bhargava@ericsson.com; Georgios Karagiannis; > >> >> Lars Westberg > >> >> > (KI/EAB) > >> >> > Cc: pcn@ietf.org > >> >> > Subject: Questions on LC-PCN draft version 01 > >> >> > > >> >> > Dear authors of the LC-PCN draft, > >> >> > > >> >> > Am I right that there has been no response yet to Phil's > >> questions > >> >> > (attached at the end) on the new version of your=20 > draft below (I=20 > >> >> > checked the pcn list and it does not seem it went=20 > there)? If the=20 > >> >> > response was sent, can you resend it please? > >> >> > > >> >> > I have many of the same questions, and also a few > >> additional ones. > >> >> > Here are some of them. > >> >> > > >> >> > 1) I do not understand whether pcn-lower_rate_egress and=20 > >> >> > pcn_upper_rate ingress are expressed as a fraction of the > >> >> total rate > >> >> > (a unitless entity, analogous to CLE) or an absolute value > >> >> (in bits or > >> >> > bytes per second). The text seems to be contradictory on > >> this point: > >> >> > > >> >> > on page 8 it is a fraction: > >> >> > > >> >> > "PCN_lower_rate_egress =3D predefined percentage of received=20 > >> >> > PCN_marking", > >> >> > > >> >> > while on page 13 it is an absolute rate: > >> >> > > >> >> > "If the incoming_PCN_marking_rate is higher than a=20 > preconfigured=20 > >> >> > PCN_lower_rate_egress, ..." where the > >> >> > > >> >> > (Incoming_PCN_marking_rate is clearly defined as abslolue > >> >> rate on page > >> >> > 12: "Where the "incoming_PCN_marking_rate" is calculated > >> as follows: > >> >> > incoming_PCN_marking_rate =3D (received number of = "PCN_marking" > >> >> > DSCP during T)* N)/T;) > >> >> > >> >> Georgios: You are right, please see discussion above on=20 > paragaph=20 > >> >> denoted > >> >> as: > >> >> "Setting the thresholds at PCN_egress_nodes". > >> >> > >> >> > >> >> We use a percentage of received PCN_marking encoded packets in=20 > >> >> proportion to total rate of received packets to define when a=20 > >> >> PCN_egres_node goes from Normal state to admission control > >> state. In > >> >> this version of the draft we call this value as:=20 > >> >> PCN_lower_rate_egress. This is wrong. > >> >> What we should say is: > >> >> PCN_lower_rate_egress: is equal to=20 > incoming_PCN_marking_rate, when: > >> >> (incoming_PCN_marking_rate/measured PHB rate) =3D to a=20 > preconfigured=20 > >> >> percentage, say 1%. From now on I denote this percentage as: > >> >> PCN_lower_percentage_egress. > >> >> Thus we can say that a PCN_egress_node changes from Normal > >> state to > >> >> admission control state when > >> incoming_PCN_marking_rate/measured PHB > >> >> rate > PCN_lower_percentage_egress. > >> >> Where, > >> >> incoming_PCN_marking_rate =3D N *=20 > input_PCN_marking_bytes/T Where,=20 > >> >> input_PCN_marking_bytes =3D received number of=20 > "PCN_marking" encoded=20 > >> >> packets during measurement period T. > >> >> > >> >> > >> >> > > >> >> > 2) Regarding the question above, > >> >> > > >> >> > * if pcn_lower_rate_eggress (and pcn_uppre-rate_egress) are > >> >> absolute > >> >> > rates, then how do you choose that abslolute rate value, > >> given that > >> >> > the rates of ingress-egress aggregates at the same > >> ingress may be > >> >> > vastly different in magnitude? > >> >> > >> >> Georgios: See explanation above! Hopefully this > >> misundersatnding is > >> >> also solved! > >> >> > >> >> > > >> >> > *if, however, pcn_lower_rate_eggress (and > >> >> > pcn_uppre-rate_egress) are ratios, then what is the=20 > guidance of=20 > >> >> > setting these ratios so that the egress can always tell > >> admission > >> >> > state from the termination state, given that the same > >> >> marking is used > >> >> > for admission and termination marking? > >> >> > Specifically, suppose at the bottleneck pcn_lower_threshold > >> >> is set to > >> >> > X, and pcn_upper_threshold to aX with some a>1 (assume > >> for example > >> >> > a=3D1.5). > >> >> > How does the egress node tell between the conditions > >> when the total > >> >> > pcn traffic on the bottleneck is 1.1X (which is the=20 > state wnen=20 > >> >> > admission is needed but termination is not, - in this case > >> >> > (1.1X-X)/X=3D0.1 traffic is pcn-marked), and the case when > >> the total > >> >> > traffic on the bottleneck is 1.1aX, which is the case when > >> >> termination > >> >> > is needed, and (1.1ax-ax)/ax=3D0.1), if in both cases the > >> >> ratio between > >> >> > marked and total traffic is the same value 0.1? > >> >> > >> >> Georgios: In order to solve the above described problems, the=20 > >> >> thresholds used in the PCN_interior_nodes and=20 > PCN_egress_nodes are > >> >> using: > >> >> the Admission_offset_rate that is an absolute rate value > >> which is set > >> >> equal into whole PCN domain. > >> >> > >> >> Note that the maximum excess rate that a PCN_interior_node can=20 > >> >> calculate in admission control state is equal to=20 > >> >> Admission_offset_rate. If the excess rate is higher than the=20 > >> >> Admission_offset_rate then the node changes from=20 > admission control=20 > >> >> state to flow termination state. > >> >> > >> >> Please see all details that I have provided at the top of > >> this email.=20 > >> >> In particular, check the paragraphs denoted above as: > >> >> "Setting the thresholds at PCN_interior_nodes" > >> >> "Setting the thresholds at PCN_egress_nodes" > >> >> "Generated excess rate by an PCN_interior_node operating > >> in admission > >> >> control state" > >> >> > >> >> > >> >> > >> >> > > >> >> > 3) On page 9, multicongestion error error is introduced, > >> >> but the text > >> >> > is silent on how this error is defined and what is it set > >> >> to (if it is > >> >> > a configuration parameter), or how it is measured (if it is > >> >> something > >> >> > that is being measured). Can you explain, please? > >> >> > >> >> Georgios: > >> >> Please see the paragraph denoted above as "Setting the > >> thresholds at > >> >> PCN_egress_nodes" > >> >> > >> >> > >> >> Best regards, > >> >> Georgios > >> >> > >> >> > >> >> > > >> >> > I have some more questions, including sharing those asked > >> >> by Phil in > >> >> > the attached message), but I will stop now, as I hope > >> >> clarifications > >> >> > on the above (and Phil's) questions will help understand > >> the draft > >> >> > better... > >> >> > > >> >> > Thank you in advance for clarifying all this. > >> >> > > >> >> > Anna > >> >> > >>=20 >=20 _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Mon Nov 05 01:09:35 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iov9O-0000id-E9; Mon, 05 Nov 2007 01:09:30 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Iov9M-0000hn-SE for pcn-confirm+ok@megatron.ietf.org; Mon, 05 Nov 2007 01:09:28 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iov9M-0000gf-5G for pcn@ietf.org; Mon, 05 Nov 2007 01:09:28 -0500 Received: from szxga04-in.huawei.com ([61.144.161.7]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1Iov9J-0004jl-Ej for pcn@ietf.org; Mon, 05 Nov 2007 01:09:27 -0500 Received: from huawei.com (szxga04-in [172.24.2.12]) by szxga04-in.huawei.com (iPlanet Messaging Server 5.2 HotFix 2.14 (built Aug 8 2006)) with ESMTP id <0JR0008P1RRI6S@szxga04-in.huawei.com> for pcn@ietf.org; Mon, 05 Nov 2007 14:09:18 +0800 (CST) Received: from huawei.com ([172.24.1.18]) by szxga04-in.huawei.com (iPlanet Messaging Server 5.2 HotFix 2.14 (built Aug 8 2006)) with ESMTP id <0JR0006VWRRF9H@szxga04-in.huawei.com> for pcn@ietf.org; Mon, 05 Nov 2007 14:09:18 +0800 (CST) Received: from z24109b ([10.70.76.84]) by szxml03-in.huawei.com (iPlanet Messaging Server 5.2 HotFix 2.14 (built Aug 8 2006)) with ESMTPA id <0JR0008CCRRE2L@szxml03-in.huawei.com> for pcn@ietf.org; Mon, 05 Nov 2007 14:09:15 +0800 (CST) Date: Mon, 05 Nov 2007 14:09:14 +0800 From: Tina TSOU Subject: Re: [PCN] RSVP and ECMP To: "Anna Charny (acharny)" , Hannes Tschofenig Message-id: <021c01c81f72$6468c620$544c460a@china.huawei.com> MIME-version: 1.0 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2900.3138 X-Mailer: Microsoft Outlook Express 6.00.2900.3138 X-Priority: 3 X-MSMail-priority: Normal References: <4720A16F.6050409@gmx.net> X-Spam-Score: 0.0 (/) X-Scan-Signature: 06d29b9b22258f4f07fe89b4d4a05a86 Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0330635671==" Errors-To: pcn-bounces@ietf.org This is a multi-part message in MIME format. --===============0330635671== Content-type: multipart/alternative; boundary="Boundary_(ID_eChq4ENo2JH2BhkMJieyXg)" This is a multi-part message in MIME format. --Boundary_(ID_eChq4ENo2JH2BhkMJieyXg) Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: 7BIT Hi Hannes and all, In my mind, in PCN domain RSVP PATH messages do not need to be carried over the same path as subsequent data packets. It does not impact on the PCN mechanism working. B. R. Tina ----- Original Message ----- From: Hannes Tschofenig To: Anna Charny (acharny) Cc: pcn@ietf.org Sent: Thursday, October 25, 2007 10:00 PM Subject: Re: [PCN] RSVP and ECMP Hi Anna, Anna Charny (acharny) wrote: > Hi Hannes, > > >> The problems obviously show up if the load balancing device >> does not inspect the RSVP traffic. >> Then, you cannot ensure that the signaling traffic follows >> the data traffic anymore. >> >> > > Sorry, I did not quite understand your point. Wouldn't a packet > carrying RSVP message be processes just like any other packet in the > node which is not RSVP-capable? Theoretically yes. It just depends on the behavior of the box. The RSVP packet is, however, not a data packet. It looks different. > So all that matters for the discussion > on using RSVP message as a probe is whether that packet goes the same > way as the corresponding data packets of the same flow under ecpm > decisions? Correct, and the question is: Is it possible that data traffic travels along a different path than the signaling traffic. The answer is "yes" (unless you have control over the box making the decisions). > Am I missing the point you are making? > > Ciao Hannes > Anna > > >> -----Original Message----- >> From: Hannes Tschofenig [mailto:Hannes.Tschofenig@gmx.net] >> Sent: Thursday, October 25, 2007 6:14 AM >> To: Georgios Karagiannis >> Cc: pcn@ietf.org >> Subject: Re: [PCN] RSVP and ECMP >> >> The problems obviously show up if the load balancing device >> does not inspect the RSVP traffic. >> Then, you cannot ensure that the signaling traffic follows >> the data traffic anymore. >> >> Also with tunnels you need to ensure that at least one of the >> tunnel endpoints is aware of the signaling protocol. This >> might seem to be a simple requirement but I assume in reality >> it is not that easy. This is the area where complexity says >> HELLO to path-coupled signaling. >> >> Ciao >> Hannes >> >> PS: When you still think that it is totally trivial then you >> might want to look at the many tunneling schemes folks use or >> want to use. >> >> Georgios Karagiannis wrote: >> >>> Hi Bruce, Hi all >>> >>> I agree with Bruce! If it is to use probing, then we have to ensure >>> that the probes should get the similar correct ECMP >>> >> treatment as the >> >>> correct ECMP treatment achieved by RSVP. >>> >>> This means that the probe messages have to have the same >>> >> addresses and >> >>> ports as the data. >>> I think tat this is not difficult and not complex to be achieved. >>> >>> Best regards, >>> Georgios >>> >>> >>> >>> On 10/23/2007, "Bruce Davie" wrote: >>> >>> >>> >>>> Michael, >>>> RSVP tries to forward the Path exactly the same way that it would >>>> forward the data. Since Path messages are processed outside of the >>>> fast path, it is possible for the router to look at >>>> >> anything that is >> >>>> in the path message, including source addr, dest addr, and port >>>> numbers, and figure out where a data packet that had those >>>> >> values in >> >>>> its IP header would get forwarded. It sends the Path >>>> >> message out that >> >>>> interface. In practice, this takes care of the vast >>>> >> majority of ECMP >> >>>> algorithms. This approach is implemented today. >>>> >>>> If ECMP was using something from the IP or higher layer >>>> >> header that >> >>>> was not in the Path message, then a problem would arise. >>>> >>>> So, practically speaking, if the probe messages have the same >>>> addresses and ports as the data, they should get correct ECMP >>>> treatment. Or at least, fare no worse than RSVP. >>>> >>>> Bruce >>>> >>>> On Oct 23, 2007, at 12:07 PM, Michael Menth wrote: >>>> >>>> >>>> >>>>> Hi, >>>>> >>>>> I've got a question regarding RSVP. PATH messages need to >>>>> >> be carried >> >>>>> over the same path as subsequent data packets. How is >>>>> >> this achieved >> >>>>> in the presence of ECMP routing? In theory, PATH messages >>>>> >> can take a >> >>>>> different route than data packets if the load balancer takes >>>>> arbitrary parts of the header(s) to calculate a suitable >>>>> >> hash value, >> >>>>> which decides which route the packet will take. >>>>> >>>>> This issue seems to be related to the problem of how to make sure >>>>> that PCN probe messages take the same path as subsequent PCN data >>>>> packets, provided that they have the same source and destination >>>>> ports and addresses. >>>>> >>>>> I am interested in how this problem is solved in practice >>>>> >> or whether >> >>>>> it is intentionally avoided. >>>>> >>>>> Regards, >>>>> >>>>> Michael >>>>> >>>>> -- >>>>> Dr. Michael Menth, Assistant Professor University of Wuerzburg, >>>>> Institute of Computer Science Am Hubland, D-97074 Wuerzburg, >>>>> Germany, room B206 >>>>> phone: (+49)-931/888-6644, fax: (+49)-931/888-6632 >>>>> mailto:menth@informatik.uni-wuerzburg.de >>>>> http://www3.informatik.uni-wuerzburg.de/research/ngn >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> PCN mailing list >>>>> PCN@ietf.org >>>>> https://www1.ietf.org/mailman/listinfo/pcn >>>>> >>>>> >>>> _______________________________________________ >>>> PCN mailing list >>>> PCN@ietf.org >>>> https://www1.ietf.org/mailman/listinfo/pcn >>>> >>>> >>> _______________________________________________ >>> PCN mailing list >>> PCN@ietf.org >>> https://www1.ietf.org/mailman/listinfo/pcn >>> >>> >> >> _______________________________________________ >> PCN mailing list >> PCN@ietf.org >> https://www1.ietf.org/mailman/listinfo/pcn >> >> _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn --Boundary_(ID_eChq4ENo2JH2BhkMJieyXg) Content-type: text/html; charset=ISO-8859-1 Content-transfer-encoding: 7BIT
Hi Hannes and all,
In my mind, in PCN domain RSVP PATH messages do not need to be carried over the same path as subsequent data packets. It does not impact on the PCN mechanism working.
 
B. R.
Tina
----- Original Message -----
Sent: Thursday, October 25, 2007 10:00 PM
Subject: Re: [PCN] RSVP and ECMP

Hi Anna,

Anna Charny (acharny) wrote:
> Hi Hannes,
>
>  
>> The problems obviously show up if the load balancing device
>> does not inspect the RSVP traffic.
>> Then, you cannot ensure that the signaling traffic follows
>> the data traffic anymore.
>>
>>    
>
> Sorry, I did not quite understand your point.  Wouldn't a packet
> carrying RSVP message be processes just like any other packet in the
> node which is not RSVP-capable?

Theoretically yes. It just depends on the behavior of the box. The RSVP
packet is, however, not a data packet.
It looks different.

>  So all that matters for the discussion
> on using RSVP message as a probe is whether that packet goes the same
> way as the corresponding data packets of the same flow under ecpm
> decisions?
Correct, and the question is: Is it possible that data traffic travels
along a different path than the signaling traffic.
The answer is "yes" (unless you have control over the box making the
decisions).

>  Am I missing the point you are making?
>
>  


Ciao
Hannes

> Anna
>
>  
>> -----Original Message-----
>> From: Hannes Tschofenig [mailto:Hannes.Tschofenig@gmx.net]
>> Sent: Thursday, October 25, 2007 6:14 AM
>> To: Georgios Karagiannis
>> Cc: pcn@ietf.org
>> Subject: Re: [PCN] RSVP and ECMP
>>
>> The problems obviously show up if the load balancing device
>> does not inspect the RSVP traffic.
>> Then, you cannot ensure that the signaling traffic follows
>> the data traffic anymore.
>>
>> Also with tunnels you need to ensure that at least one of the
>> tunnel endpoints is aware of the signaling protocol. This
>> might seem to be a simple requirement but I assume in reality
>> it is not that easy. This is the area where complexity says
>> HELLO to path-coupled signaling.
>>
>> Ciao
>> Hannes
>>
>> PS: When you still think that it is totally trivial then you
>> might want to look at the many tunneling schemes folks use or
>> want to use.
>>
>> Georgios Karagiannis wrote:
>>    
>>> Hi Bruce, Hi all
>>>
>>> I agree with Bruce! If it is to use probing, then we have to ensure
>>> that the probes should get the similar correct ECMP
>>>      
>> treatment as the
>>    
>>> correct ECMP treatment achieved by RSVP.
>>>
>>> This means that the probe messages have to have the same
>>>      
>> addresses and
>>    
>>> ports as the data.
>>> I think tat this is not difficult and not complex to be achieved.
>>>
>>> Best regards,
>>> Georgios
>>>
>>>
>>>
>>> On 10/23/2007, "Bruce Davie" <bdavie@cisco.com> wrote:
>>>
>>>  
>>>      
>>>> Michael,
>>>>  RSVP tries to forward the Path exactly the same way that it would
>>>> forward the data. Since Path messages are processed outside of the
>>>> fast path, it is possible for the router to look at
>>>>        
>> anything that is
>>    
>>>> in the path message, including source addr, dest addr, and port
>>>> numbers, and figure out where a data packet that had those
>>>>        
>> values in
>>    
>>>> its IP header would get forwarded. It sends the Path
>>>>        
>> message out that
>>    
>>>> interface. In practice, this takes care of the vast
>>>>        
>> majority of ECMP
>>    
>>>> algorithms. This approach is implemented today.
>>>>
>>>>  If ECMP was using something from the IP or higher layer
>>>>        
>> header that
>>    
>>>> was not in the Path message, then a problem would arise.
>>>>
>>>>  So, practically speaking, if the probe messages have the same
>>>> addresses and ports as the data, they should get correct ECMP
>>>> treatment. Or at least, fare no worse than RSVP.
>>>>
>>>> Bruce
>>>>
>>>> On Oct 23, 2007, at 12:07 PM, Michael Menth wrote:
>>>>
>>>>    
>>>>        
>>>>> Hi,
>>>>>
>>>>> I've got a question regarding RSVP. PATH messages need to
>>>>>          
>> be carried
>>    
>>>>> over the same path as subsequent data packets. How is
>>>>>          
>> this achieved
>>    
>>>>> in the presence of ECMP routing? In theory, PATH messages
>>>>>          
>> can take a
>>    
>>>>> different route than data packets if the load balancer takes
>>>>> arbitrary parts of the header(s) to calculate a suitable
>>>>>          
>> hash value,
>>    
>>>>> which decides which route the packet will take.
>>>>>
>>>>> This issue seems to be related to the problem of how to make sure
>>>>> that PCN probe messages take the same path as subsequent PCN data
>>>>> packets, provided that they have the same source and destination
>>>>> ports and addresses.
>>>>>
>>>>> I am interested in how this problem is solved in practice
>>>>>          
>> or whether
>>    
>>>>> it is intentionally avoided.
>>>>>
>>>>> Regards,
>>>>>
>>>>>    Michael
>>>>>
>>>>> --
>>>>> Dr. Michael Menth, Assistant Professor University of Wuerzburg,
>>>>> Institute of Computer Science Am Hubland, D-97074 Wuerzburg,
>>>>> Germany, room B206
>>>>> phone: (+49)-931/888-6644, fax: (+49)-931/888-6632
>>>>> mailto:menth@informatik.uni-wuerzburg.de
>>>>> http://www3.informatik.uni-wuerzburg.de/research/ngn
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> PCN mailing list
>>>>> PCN@ietf.org
>>>>> https://www1.ietf.org/mailman/listinfo/pcn
>>>>>      
>>>>>          
>>>> _______________________________________________
>>>> PCN mailing list
>>>> PCN@ietf.org
>>>> https://www1.ietf.org/mailman/listinfo/pcn
>>>>    
>>>>        
>>> _______________________________________________
>>> PCN mailing list
>>> PCN@ietf.org
>>> https://www1.ietf.org/mailman/listinfo/pcn
>>>  
>>>      
>>
>> _______________________________________________
>> PCN mailing list
>> PCN@ietf.org
>> https://www1.ietf.org/mailman/listinfo/pcn
>>
>>    



_______________________________________________
PCN mailing list
PCN@ietf.org
https://www1.ietf.org/mailman/listinfo/pcn
--Boundary_(ID_eChq4ENo2JH2BhkMJieyXg)-- --===============0330635671== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn --===============0330635671==-- From pcn-bounces@ietf.org Mon Nov 05 03:58:10 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Ioxmb-0002uI-DY; Mon, 05 Nov 2007 03:58:09 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IoxmZ-0002rk-Jn for pcn-confirm+ok@megatron.ietf.org; Mon, 05 Nov 2007 03:58:07 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IoxmU-0002mB-KP for pcn@ietf.org; Mon, 05 Nov 2007 03:58:02 -0500 Received: from smtp.nokia.com ([131.228.20.171] helo=mgw-ext12.nokia.com) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IoxmT-00083d-Qu for pcn@ietf.org; Mon, 05 Nov 2007 03:58:02 -0500 Received: from esebh106.NOE.Nokia.com (esebh106.ntc.nokia.com [172.21.138.213]) by mgw-ext12.nokia.com (Switch-3.2.5/Switch-3.2.5) with ESMTP id lA58vO0Y021761; Mon, 5 Nov 2007 10:57:41 +0200 Received: from esebh103.NOE.Nokia.com ([172.21.143.33]) by esebh106.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.1830); Mon, 5 Nov 2007 10:57:33 +0200 Received: from esebh101.NOE.Nokia.com ([172.21.138.177]) by esebh103.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.1830); Mon, 5 Nov 2007 10:57:34 +0200 Received: from mgw-int01.ntc.nokia.com ([172.21.143.96]) by esebh101.NOE.Nokia.com over TLS secured channel with Microsoft SMTPSVC(6.0.3790.1830); Mon, 5 Nov 2007 10:57:33 +0200 Received: from [172.21.37.232] (esdhcp037232.research.nokia.com [172.21.37.232]) by mgw-int01.ntc.nokia.com (Switch-3.2.5/Switch-3.2.5) with ESMTP id lA58vUum018131; Mon, 5 Nov 2007 10:57:30 +0200 In-Reply-To: <001a01c81c99$b4735350$4c0d5982@dynamic.ewi.utwente.nl> References: <000f01c81c84$5635b400$4c0d5982@dynamic.ewi.utwente.nl> <2723F111-5919-49CB-AC22-4A5A795F4852@nokia.com> <001a01c81c99$b4735350$4c0d5982@dynamic.ewi.utwente.nl> Mime-Version: 1.0 (Apple Message framework v752.3) Message-Id: <98501B20-1260-4EE0-87F1-DCCA0770230D@nokia.com> From: Lars Eggert Subject: Re: [PCN] 5.5. Probing functions Date: Mon, 5 Nov 2007 10:57:27 +0200 To: ext Georgios Karagiannis X-Mailer: Apple Mail (2.752.3) X-OriginalArrivalTime: 05 Nov 2007 08:57:33.0240 (UTC) FILETIME=[E758DB80:01C81F89] X-Nokia-AV: Clean X-Spam-Score: 0.0 (/) X-Scan-Signature: dbb8771284c7a36189745aa720dc20ab Cc: 'pcn' X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0609919827==" Errors-To: pcn-bounces@ietf.org --===============0609919827== Content-Type: multipart/signed; micalg=sha1; boundary=Apple-Mail-134--950000572; protocol="application/pkcs7-signature" --Apple-Mail-134--950000572 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Hi, On 2007-11-1, at 17:12, ext Georgios Karagiannis wrote: >> From: Lars Eggert [mailto:lars.eggert@nokia.com] >> >> how is a single packet probing the path? It's not obvious to >> me how one would derive bottleneck load based on a single >> packet, i.e., based on one or zero markings. > > Georgios: The PCN_ingress_node sends a probe packet and it waits > for a response from the PCN_egress_node. > If a PCN_interior_node is congested (is in admission control state), > then the probe will be marked, either using the PCN_marking or an > additional > encoding, say PCN_Affected_marking. ah - so this depends on the fact that a congested router uses a marking scheme that always tags the packet with _something_. In other words, this wouldn't work with Anna's single-bit marking scheme, right? (Because that does not mark all packets passing a congested router.) > If the probe packet is dropped on the way, then the > PCN_ingress_node has to > know this such that the probe packet is retransmitted. This can be > done by using > retransmission timers at the PCN_ingress_node. That adds delay, obviously. You also need to defined some retransmission scheme that is safe without knowing the path RTT. >>>> (2) How long do you need to probe for before you declare it safe to >>>> admit the new flow? >>> Georgios: One probe packet per new incoming and flow is enough if >>> the >>> PCN_domain takes care that the probe packet will be marked >>> PCN_marked >>> or using an other type of encoding, if it is passing through a >>> congested PCN_interior_node. >> >> Ah - so this requires the marking scheme to mark all packets >> once the defined pre-congestion load level is reached, >> instead of marking proportionally with an increase in load, >> correct? > > Georgios: No! Marking proportionally is still needed, since it is also > needed to encode the flow termination state of the PCN_interior nodes. > However, all packets that are passing through the congested node > and are not > PCN_marked they will have to be remarked with this additional > encoding, denoted as > PCN_Affected_marking. Thanks - I think I've understood this now. See above for my comments about single-bit marking - this wouldn't work with single-bit marking, would it? >> This eliminates some of the marking options that >> have been discussed. > > Georgios: No, please see above! I still believe this does eliminate single-bit marking. >>>> What's the delay before a new >>>> flow can be admitted, and can apps actually deal with this delay? >>> >>> The delay = one RTT (within PCN_domain). So what happens to the flow during the RTT needed for probing? Packets are dropped? That would seem to significantly affect various apps. Lars --Apple-Mail-134--950000572 Content-Transfer-Encoding: base64 Content-Type: application/pkcs7-signature; name=smime.p7s Content-Disposition: attachment; filename=smime.p7s MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIGQDCCAvkw ggJioAMCAQICEGtxkLFHQu1g67hlmYfwPuIwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UEBhMCWkEx JTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQ ZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA3MDEwNDE2MTE1NFoXDTA4MDEwNDE2MTE1 NFowXDEPMA0GA1UEBBMGRWdnZXJ0MQ0wCwYDVQQqEwRMYXJzMRQwEgYDVQQDEwtMYXJzIEVnZ2Vy dDEkMCIGCSqGSIb3DQEJARYVbGFycy5lZ2dlcnRAbm9raWEuY29tMIIBIjANBgkqhkiG9w0BAQEF AAOCAQ8AMIIBCgKCAQEA26hjntVPZMiwh4d8tyuk9KiucvG92BVUGArk5zO1jhmq7tLFgU1mqcIg VGumfQqjkAzIuh83kxIiqBrB05Wvxp85apwt4sUCvzMe8mQWzZZZYp0rBzwpOj+VC5pxsMRYtYW+ BaTFBEmvFr7D7C7roZUk0lDppS5SZFs4SQbEe9ykB9aeUf1iTiw2+ikyP2+JEYto14WYCoxFWbms FbQ1uaaK+yn1WH2YHhyMfi0IOxuT0jtbsngjHJMpWIqq334zBhj+rPSUug47YBHr41FCDMHbbbjv WbyTyDYneOW3WY8LY8bGWVlAknQGB7lgduShe6e0RI/7SL/VE6X27lM9LwIDAQABozIwMDAgBgNV HREEGTAXgRVsYXJzLmVnZ2VydEBub2tpYS5jb20wDAYDVR0TAQH/BAIwADANBgkqhkiG9w0BAQUF AAOBgQCx3nxxTg/WowobVTRXBiXUEEZj4LBiunWbuAnR0UbJIgijvq3JbiJvZo2gUGiW9LPaHSBz 4oxT1VR/S/6O0a4e87oV5cOQm6ReB68o9rDfUzxHTwoCfv2wwMwBrXyzQMjyQK4Lt8siV41gmZpm rSXMhjTJdd9uYYNoZKQLq+VM+TCCAz8wggKooAMCAQICAQ0wDQYJKoZIhvcNAQEFBQAwgdExCzAJ BgNVBAYTAlpBMRUwEwYDVQQIEwxXZXN0ZXJuIENhcGUxEjAQBgNVBAcTCUNhcGUgVG93bjEaMBgG A1UEChMRVGhhd3RlIENvbnN1bHRpbmcxKDAmBgNVBAsTH0NlcnRpZmljYXRpb24gU2VydmljZXMg RGl2aXNpb24xJDAiBgNVBAMTG1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBDQTErMCkGCSqGSIb3 DQEJARYccGVyc29uYWwtZnJlZW1haWxAdGhhd3RlLmNvbTAeFw0wMzA3MTcwMDAwMDBaFw0xMzA3 MTYyMzU5NTlaMGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5 KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQTCBnzAN BgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAxKY8VXNV+065yplaHmjAdQRwnd/p/6Me7L3N9VvyGna9 fww6YfK/Uc4B1OVQCjDXAmNaLIkVcI7dyfArhVqqP3FWy688Cwfn8R+RNiQqE88r1fOCdz0Dviv+ uxg+B79AgAJk16emu59l0cUqVIUPSAR/p7bRPGEEQB5kGXJgt/sCAwEAAaOBlDCBkTASBgNVHRMB Af8ECDAGAQH/AgEAMEMGA1UdHwQ8MDowOKA2oDSGMmh0dHA6Ly9jcmwudGhhd3RlLmNvbS9UaGF3 dGVQZXJzb25hbEZyZWVtYWlsQ0EuY3JsMAsGA1UdDwQEAwIBBjApBgNVHREEIjAgpB4wHDEaMBgG A1UEAxMRUHJpdmF0ZUxhYmVsMi0xMzgwDQYJKoZIhvcNAQEFBQADgYEASIzRUIPqCy7MDaNmrGcP f6+svsIXoUOWlJ1/TCG4+DYfqi2fNi/A9BxQIJNwPP2t4WFiw9k6GX6EsZkbAMUaC4J0niVQlGLH 2ydxVyWN3amcOY6MIE9lX5Xa9/eH1sYITq726jTlEBpbNU1341YheILcIRk13iSx0x1G/11fZU8x ggMQMIIDDAIBATB2MGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAo UHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQ a3GQsUdC7WDruGWZh/A+4jAJBgUrDgMCGgUAoIIBbzAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcB MBwGCSqGSIb3DQEJBTEPFw0wNzExMDUwODU3MjhaMCMGCSqGSIb3DQEJBDEWBBS6KgmHseyZ5kJC afPSRqNv0RNNszCBhQYJKwYBBAGCNxAEMXgwdjBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhh d3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVt YWlsIElzc3VpbmcgQ0ECEGtxkLFHQu1g67hlmYfwPuIwgYcGCyqGSIb3DQEJEAILMXigdjBiMQsw CQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UE AxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0ECEGtxkLFHQu1g67hlmYfwPuIw DQYJKoZIhvcNAQEBBQAEggEAQKyHCbaoLfGcAiswSQLOZ9n5fLom7Mx0mD5SlOzDQUZO8ooLXJZh Zs/tWvx0LGpSXhrHPIfF8Z0cfZd6ywZsVsHrkJEvTFZ4xTLo35KfBInFG0lhV+03j7ou+3SOMDEa O3N2KIqa7DAHjy2D4/IkUoEqUBcOZF5o7f6O+YOJcEhi6M1oP+H03rphneKhuKKWhVbc2BJHLN0l OJVoa1BOa/6iNeL3qeAWFVX+KyynfSV3aXfJ0jBaui12YXdJel2cGS/8a7PFi+wPm+u/ul9PIjXQ oeioeMHgaVlPz4DPsFD3OKt/JNqV6ZOZkxw6Mn073AY8z/EBmm11NrHs46/UqwAAAAAAAA== --Apple-Mail-134--950000572-- --===============0609919827== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn --===============0609919827==-- From pcn-bounces@ietf.org Mon Nov 05 07:26:48 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Ip12V-0002OT-Jt; Mon, 05 Nov 2007 07:26:47 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Ip12T-0002OG-R8 for pcn-confirm+ok@megatron.ietf.org; Mon, 05 Nov 2007 07:26:45 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Ip12T-0002O7-Fp for pcn@ietf.org; Mon, 05 Nov 2007 07:26:45 -0500 Received: from sj-iport-5.cisco.com ([171.68.10.87]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1Ip12S-0004aF-Qu for pcn@ietf.org; Mon, 05 Nov 2007 07:26:45 -0500 X-IronPort-AV: E=Sophos;i="4.21,372,1188802800"; d="scan'208";a="184919880" Received: from sj-dkim-3.cisco.com ([171.71.179.195]) by sj-iport-5.cisco.com with ESMTP; 05 Nov 2007 04:26:42 -0800 Received: from sj-core-1.cisco.com (sj-core-1.cisco.com [171.71.177.237]) by sj-dkim-3.cisco.com (8.12.11/8.12.11) with ESMTP id lA5CQdhm013648; Mon, 5 Nov 2007 04:26:39 -0800 Received: from xbh-rtp-201.amer.cisco.com (xbh-rtp-201.cisco.com [64.102.31.12]) by sj-core-1.cisco.com (8.12.10/8.12.6) with ESMTP id lA5CQVXn015853; Mon, 5 Nov 2007 12:26:34 GMT Received: from xmb-rtp-203.amer.cisco.com ([64.102.31.20]) by xbh-rtp-201.amer.cisco.com with Microsoft SMTPSVC(6.0.3790.1830); Mon, 5 Nov 2007 07:26:31 -0500 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [PCN] 5.5. Probing functions Date: Mon, 5 Nov 2007 07:26:30 -0500 Message-ID: In-Reply-To: <98501B20-1260-4EE0-87F1-DCCA0770230D@nokia.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] 5.5. Probing functions Thread-Index: AcgfiiiHCHCogjpWTb26i+sjfyw1+gAG5vug From: "Anna Charny (acharny)" To: "Lars Eggert" , "ext Georgios Karagiannis" X-OriginalArrivalTime: 05 Nov 2007 12:26:31.0648 (UTC) FILETIME=[18D21600:01C81FA7] X-TM-AS-Product-Ver: SMEX-8.0.0.1181-5.000.1023-15522.000 X-TM-AS-Result: No--15.366600-8.000000-31 X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No DKIM-Signature: v=0.5; a=rsa-sha256; q=dns/txt; l=3895; t=1194265599; x=1195129599; c=relaxed/simple; s=sjdkim3002; h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version; d=cisco.com; i=acharny@cisco.com; z=From:=20=22Anna=20Charny=20(acharny)=22=20 |Subject:=20RE=3A=20[PCN]=205.5.=20Probing=20functions=20 |Sender:=20; bh=ruF3QdEVNN2Teq8dl2oH5fj5HeCsBAaBJeaeNk4R9bY=; b=Vo/CWMOu+avZxvm5GOPonjdJxMrDqNMMnUnTk/Ss7UkSXkCqcdSqX/qeHhfTKWWsQ9O6Xx71 3oKHHZaiASZJnPl8OQNYpCtsi/HydXXdh17ybR8eHEXEVM6oUPi6GLks; Authentication-Results: sj-dkim-3; header.From=acharny@cisco.com; dkim=pass ( sig from cisco.com/sjdkim3002 verified; ); X-Spam-Score: 0.0 (/) X-Scan-Signature: a87a9cdae4ac5d3fbeee75cd0026d632 Cc: pcn X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Lars, Please see below:=20 > -----Original Message----- > From: Lars Eggert [mailto:lars.eggert@nokia.com]=20 > Sent: Monday, November 05, 2007 3:57 AM > To: ext Georgios Karagiannis > Cc: 'pcn' > Subject: Re: [PCN] 5.5. Probing functions=20 >=20 > Hi, >=20 > On 2007-11-1, at 17:12, ext Georgios Karagiannis wrote: > >> From: Lars Eggert [mailto:lars.eggert@nokia.com] > >> > >> how is a single packet probing the path? It's not obvious=20 > to me how=20 > >> one would derive bottleneck load based on a single packet, i.e.,=20 > >> based on one or zero markings. > > > > Georgios: The PCN_ingress_node sends a probe packet and it=20 > waits for a=20 > > response from the PCN_egress_node. > > If a PCN_interior_node is congested (is in admission=20 > control state),=20 > > then the probe will be marked, either using the PCN_marking or an=20 > > additional encoding, say PCN_Affected_marking. >=20 > ah - so this depends on the fact that a congested router uses=20 > a marking scheme that always tags the packet with _something_. >=20 > In other words, this wouldn't work with Anna's single-bit=20 > marking scheme, right? (Because that does not mark all=20 > packets passing a congested router.) >=20 You are quite right - single marking would not necessarily mark a=20 probe message - you would need to send several probes to notice=20 that the path is congested with high enough probability - and that introduces an additional delay. This does not happen with schemes that mark all packets with something when the path is congested (for which you need an additional codepoint). =20 All of this assumes that RSVP message/probe goes through the same path as the data - which happens for some ECMP hashes and not for others.=20 Anna=20 > > If the probe packet is dropped on the way, then the=20 > PCN_ingress_node=20 > > has to know this such that the probe packet is=20 > retransmitted. This can=20 > > be done by using retransmission timers at the PCN_ingress_node. >=20 > That adds delay, obviously. You also need to defined some=20 > retransmission scheme that is safe without knowing the path RTT. >=20 > >>>> (2) How long do you need to probe for before you declare=20 > it safe to=20 > >>>> admit the new flow? > >>> Georgios: One probe packet per new incoming and flow is enough if=20 > >>> the PCN_domain takes care that the probe packet will be marked=20 > >>> PCN_marked or using an other type of encoding, if it is passing=20 > >>> through a congested PCN_interior_node. > >> > >> Ah - so this requires the marking scheme to mark all=20 > packets once the=20 > >> defined pre-congestion load level is reached, instead of marking=20 > >> proportionally with an increase in load, correct? > > > > Georgios: No! Marking proportionally is still needed, since=20 > it is also=20 > > needed to encode the flow termination state of the=20 > PCN_interior nodes. > > However, all packets that are passing through the congested=20 > node and=20 > > are not PCN_marked they will have to be remarked with this=20 > additional=20 > > encoding, denoted as PCN_Affected_marking. >=20 > Thanks - I think I've understood this now. See above for my=20 > comments about single-bit marking - this wouldn't work with=20 > single-bit marking, would it? >=20 > >> This eliminates some of the marking options that have been=20 > discussed. > > > > Georgios: No, please see above! >=20 > I still believe this does eliminate single-bit marking. >=20 > >>>> What's the delay before a new > >>>> flow can be admitted, and can apps actually deal with this delay? > >>> > >>> The delay =3D one RTT (within PCN_domain). >=20 > So what happens to the flow during the RTT needed for probing? =20 > Packets are dropped? That would seem to significantly affect=20 > various apps. >=20 > Lars >=20 _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Mon Nov 05 08:05:57 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Ip1eM-0001KG-Ge; Mon, 05 Nov 2007 08:05:54 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Ip1eL-0001Hn-7M for pcn-confirm+ok@megatron.ietf.org; Mon, 05 Nov 2007 08:05:53 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Ip1eK-0001He-Tw for pcn@ietf.org; Mon, 05 Nov 2007 08:05:52 -0500 Received: from smtp3.smtp.bt.com ([217.32.164.138]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1Ip1eJ-00064s-9u for pcn@ietf.org; Mon, 05 Nov 2007 08:05:52 -0500 Received: from E03MVZ1-UKDY.domain1.systemhost.net ([193.113.30.65]) by smtp3.smtp.bt.com with Microsoft SMTPSVC(6.0.3790.1830); Mon, 5 Nov 2007 13:05:50 +0000 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [PCN] Comments /questions on LC-PCN draft Date: Mon, 5 Nov 2007 13:05:49 -0000 Message-ID: <75A199C5D243C741BF3D3F1EBCEF9BA503B342D4@E03MVZ1-UKDY.domain1.systemhost.net> In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] Comments /questions on LC-PCN draft Thread-Index: Acgdc82dwbFPdIH+S4qdTF89LDfIiwCNLOYw From: To: , X-OriginalArrivalTime: 05 Nov 2007 13:05:50.0663 (UTC) FILETIME=[96E73570:01C81FAC] X-Spam-Score: -1.0 (-) X-Scan-Signature: 90e8b0e368115979782f8b3d811b226b Cc: X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Georgios, thanks yet more in-line phil > -----Original Message----- > From: Georgios Karagiannis [mailto:karagian@cs.utwente.nl] > Sent: 02 November 2007 17:14 > To: Eardley,PL,Philip,CXR9 R; pcn@ietf.org > Subject: Re: [PCN] Comments /questions on LC-PCN draft >=20 > Hi Phil >=20 > Thank you very much! > Please see in line! >=20 >=20 > On 11/2/2007, "philip.eardley@bt.com" wrote: >=20 > >I've re-started the email thread. > > > > > > > >Thanks Georgios for your description of LC-PCN in recent email. I'm > >going to try & re-phrase how it works, partly to make sure I've > >understood & partly to try and re-use the PCN terminology (I admit to > >finding the translations required hard work). Hence it slightly repeats > >some of my earlier email, but some of its different where I now > >understand your scheme better > > > > > > > >First, let me summarise to make sure I understand it right [fingers > >crossed I do!]. > > > > > > > >- two encodings are used. one ('PCN-marking') is used for both adm ctrl > >& termination to indicate the excess traffic. If a PCN-interior-node is > >PCN-marking some pkts, then it marks all other pkts with another > >encoding ('affected marking') > > > > > > > >- the 'PCN-marking' algorithm has several regimes: > > > >[let us assume that none of the packets arriving at an interface are > >PCN-marked] > > > >* when traffic rate is below PCN-lower-rate, no pkts are marked >=20 > Georgios: Yes, right > > > >* as the traffic rate climbs above the PCN-lower-rate, an increasing > >number of pkts are marked (marking rate =3D excess rate divided by = N). > >note, this applies whatever the traffic rate (the same formula, excess > >rate divided by N, is used when the traffic rate is above the > >PCN-upper-rate >=20 > Georgios: when the traffic rate is above the PCN-upper-rate then the > excess rate is calculated the same, but the sliding window mechanism is > used to avoid undershoot. [phil] eh? The T factor appears whether you've above or below the PCN-upper-rate. >=20 > > > > > > > >* Fiddle factor 1: [applies to marking when rate is below > >PCN-lower-rate] if pkts arrive already PCN-marked, then the node > >measures the rate of pkts already marked & works out what excess rate > >this corresponds to, and only marks additional pkts if its own excess > >rate is even higher. >=20 > Georgios: As I said earlier this is only partially true, see pseudo-code > on page 13, where the remarking depends on the incoming_PHR_marking_rate > but also on teh Admission_offset_rate. [phil] the appearance of the admission-offset-rate in page 13 pseudocode seems to me to exactly fit the way I described fiddle factor 1. but I think this is too detailed to be worth discussing - there are more important points at issue. >=20 > > > >* Fiddle factor 2: [applies to marking when rate is above > >PCN-upper-rate] assume that any rate above PCN-upper-rate, as measured > >in the previous measurement window period T, will be in the process of > >being terminated. So can ignore this much excess (above PCN-upper-rate) > >in this measurement period. >=20 > Georgios: As I mentioned earlier it is more complex than that, please see > pseudocode on page 20. The remarking also depends on the > incoming_PCN_marking_rate and on the Termination_offset_rate. [phil] ditto previous comment. This is probably my brain not being able to cope with all the new terminology. >=20 > > > > > > > >* the PCN-egress-node measures the rate of PCN-marked pkts and > >determines whether the overloaded node is in adm ctrl state or > >termination ctrl state >=20 > Georgios: Yes, right > > > >* PCN-egress-node calculates the % of PCN-bytes that are PCN-marked, > >multiplying by the N factor (to compensate for the PCN-interior-nodes > >only marking 1/N of their excess rate). If above a certain % then new > >calls are blocked. >=20 > Georgios:Yes, but when probing is used then this somehow different, > because the egress checks if the probe is marked or not. [phil] probing is a completely different case.=20 > > > >* PCN-egress-node also calculates the amount (bit rate) of PCN-bytes > >that are PCN-marked (again multiplying by the N factor). If this is > >above the PCN-upper-rate then it terminates some flows. >=20 > Georgios: Yes, right. >=20 > > > > > > > >* for a variety of reasons the following 4 factors have to be the same > >on every node in the PCN-domain: N, T (sliding window measurement > >period), [PCN-upper-rate - PCN-lower-rate], [(policed rate for the PCN > >class) - PCN-upper-rate]. >=20 > Georgios: Yes. Note however, that the last one that represents > Termination_offset_rate could be released from this constraint if each > PCN_interior_node is able to know the Termination_offset_rate used by > each of its neighbouring PCN_interior_nodes. >=20 [phil] above answers =3D> at least I've more or less understood it!! > > > > > > > > > > > >Questions: > > > >* imagine there are 2 independent ECMP paths for a particular > >ingress-egress-aggregate, and that there's a PCN-interior-node on each > >path that is somewhat pre-congested (traffic between PCN-lower-rate and > >PCN-upper-rate - in 'adm ctrl state'). These add together at the > >PCN-egress-node so it appears that some PCN-interior-node is highly > >pre-congested (traffic above PCN-upper-rate). From your replies I > >believe you agree. >=20 > Georgios: Yes, I agree, but do you agree that this is a corner case? [phil] not really. If the network is operating with quite a lot of traffic so the adm ctrl mechanism is often blocking calls, it seems likely to me that there'll be several links that are doing pcn-marking. I don't know if there are any simulations that can confirm/deny this. anyway, it seems very bad news if this does happen with your algorithm - assuming that flow termination is a mechanism of last resort.=20 >=20 >=20 > >To try to get round this you've introduced a fiddle > >factor, basically the PCN-egress-node has to see even more PCN-marks > >before it really believes that flows need terminate. >=20 > Georgios: Yes, you are right [phil] which means, when there's one link in above PCN-upper-rate, that you might not terminate flows. Which reduced the effectiveness of the flow termination mechanism.=20 >=20 > * the same > >'affected marking' is used in both adm ctrl state & termination ctrl > >state. Imagine that a node [node-1] on one ECMP path is in adm ctrl > >state & a node [node-2] on another ECMP path is in termination ctrl > >state. Therefore flows are terminated, and only flows that are being > >PCN-marked or affected-marked are terminated. However this could lead to > >flows being terminated that go through node-1; node-2 will still be just > >as badly overloaded >=20 > Georgios: Not really! Node 2 will not be just as badly overloaded. Flows > will be terminated that go through node-2 and only few flows that go > throgh node-1 could be terminated. Without using affected_marking the > severe congestion on node-2 will be in a smaller proportion solved. [phil] I missed a line of explanation; "However, imagine that unluckily we happen to select for termination only flows that go through node-1 and none at all that go through node-2". Now you could say [1] this is unlikely [2] even if it happens then in the next round there'll still be marking and then flows through node-2 will be terminated. Two responses [1] in the first round you've terminated flows through node-1 that don't need to be terminated, which is annoying. [2] there may be nasty cases where the second round you still don't terminate any flows through node-2 - eg where all the flows are emergency calls. You keep terminating flows that don't actually help solve the problem.=20 Overall, to me this means that your solution to the ECMP problem isnt that good. (NB I fully accept that the ECMP problem is hard to solve and that we don't know of any really nice solutions (simple and effective).) =20 >=20 >=20 >=20 > Furthermore, note that affected marking in admission control state is > only required when probing is required, in order to accomplish the case > that the probe always get marked when it passes through a congested node. >=20 > > > >* one reading of these two bullets is that you haven't really solved > >ECMP. >=20 > Georgios: Agree that when Affected_marking is used during flow > termination state and is also used during admission control state, then > the ECMP solution used during flow termination state works less optimal > than the situation that the affected_marking is only used during flow > termination state and not in admission control state. However, it works > better than if you would not use Affected_marking at all. >=20 > Note that if probing is not used, then affected_marking is not used > during the admisison control state. In this way the admission control > state does not solve the ECMP case, but during flow termination, the > ECMP case can be competely solved. [phil] true. But then if the operator decides they do want to use probing they have to change all the routers. Maybe this is ok, not sure. >=20 > > > >* there are a lot of parameters that need to be configured with the same > >value everywhere in the PCN-domain. It would need careful thought about > >what goes wrong if some of them are mis-configured on some PCN-nodes >=20 > Georgios: Agree, > > > >* it's particularly foul if T (the measurement period) needs to be the > >same on every PCN-node. I'm not sure this is really the case, but it > >seemed to me to come out of "Fiddle factor 2" > > > >* looking at the 2 rate parameters, [PCN-upper-rate - PCN-lower-rate], > >[(policed rate for the PCN class) - PCN-upper-rate]. It seems quite > >restrictive that these have to be the same on all PCN-nodes in the > >PCN-domain. They might have very different capacities for instance. >=20 > Georgios: I think that the main problem is related to setting the > [(policed rate for the PCN class) - PCN-upper-rate] constant. Please see > above for the explanation. >=20 > > > >* the "fiddle factors" make the algorithm quite complicated I think for > >PCN-interior-nodes. >=20 > georgios: Quite complicated compared to what? [phil] compared with what they do at the moment [eg they have a token bucket].=20 >=20 > > > >* "fiddle factor 2" seems the wrong approach to me - I think it would be > >better to keep marking at the same rate and for the PCN-egress-node to > >terminate more slowly (in order to avoid the problem of terminating too > >much). At least 2 reasons: if marked pkts are lost, then at best calls > >take longer to be terminated (another measurement period before get > >PCN-marks again) & at worst it might be possible to devise scenarios > >where termination never happens; secondly, I think it makes it easier > >for the PCN-egress-node to take account of policy [operator experience > >etc] to decide how fast to react to overload, whereas your approach > >fixes the speed of reaction as a parameter of the PCN-interior-node. > > >=20 > Georgios: Please note that the PCN_egress_node changes state from > admission control to termination state if the value of the excess rate > exceeds the value of Admission_offset_rate +/- multicongestion_error. > By using worst case scenarios the value of the multicongestion_error can > be a priori estimated. [phil] maybe. But then in the case where only one link is pre-congested, then the PCN mechanism is less effective because the pre-congestion has to be higher before the mechanisms kick in. >=20 > > > >That's it for now! > > > > > > > >best wishes > > > > > > > >phil/ _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Mon Nov 05 13:57:28 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Ip78a-0002Qt-63; Mon, 05 Nov 2007 13:57:28 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Ip78Y-0002Qf-RV for pcn-confirm+ok@megatron.ietf.org; Mon, 05 Nov 2007 13:57:26 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Ip78Y-0002QX-Gh for pcn@ietf.org; Mon, 05 Nov 2007 13:57:26 -0500 Received: from smtp3.smtp.bt.com ([217.32.164.138]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1Ip78X-00081u-V3 for pcn@ietf.org; Mon, 05 Nov 2007 13:57:26 -0500 Received: from E03MVZ1-UKDY.domain1.systemhost.net ([193.113.30.65]) by smtp3.smtp.bt.com with Microsoft SMTPSVC(6.0.3790.1830); Mon, 5 Nov 2007 18:57:24 +0000 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [PCN] Architecture draft - probing section & general updates. Date: Mon, 5 Nov 2007 18:57:23 -0000 Message-ID: <75A199C5D243C741BF3D3F1EBCEF9BA503B342DC@E03MVZ1-UKDY.domain1.systemhost.net> In-Reply-To: <9671A92C3C8B5744BC97F855F7CB646512DFE32B@zcarhxm1.corp.nortel.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] Architecture draft - probing section & general updates. Thread-Index: AcgW7SC2PEVcOi1+Qg+vZ1UxgSPGoQDfIixAAVzgg4A= From: To: , X-OriginalArrivalTime: 05 Nov 2007 18:57:24.0858 (UTC) FILETIME=[B405EDA0:01C81FDD] X-Spam-Score: 0.0 (/) X-Scan-Signature: f66b12316365a3fe519e75911daf28a8 Cc: pcn@ietf.org, Ruediger.Geib@t-systems.com X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Joe - trying to clarify... what's the PCN-domain? Does it run from one *enterprise* LAN edge switch/router to another (there's a tunnelled link between them), or from the end terminals. Ie what are the pcn-boundary-nodes {&pcn-interior-nodes)? The scenario isnt captured I think in the draft-ietf-pcn-architecture, so something should probably be added... Thanks Phil.=20 > -----Original Message----- > From: Jozef Babiarz [mailto:babiarz@nortel.com] > Sent: 29 October 2007 21:06 > To: Lars Eggert > Cc: pcn@ietf.org; Geib, Ruediger > Subject: RE: [PCN] Architecture draft - probing section & general updates. >=20 > Hi Lars, > I did not explain my scenario that clearly, I confused people the word > "access node". So here goes a second try. >=20 > I'm thinking of a scenario that many enterprises face today. Large multi > location enterprises use multiple WAN links or VPS to interconnect their > locations (branch offices). PCN is run inside the enterprise network > including WAN links which may be tunneled across the carrier network. > Carrier network is not required to support PCN as the enterprise traffic > including PCN marking is tunneled, however it could. Network Access > Control with PCN admission control is done at the *enterprise* LAN edge > switch/router. New flows can be routed to any egress LAN edge > switch/router and some flows will (between different locations) be > routed over one of the WAN links which are normally bandwidth > constrained. There is a high probability that many of the edge LAN > switches/routers will have no flows setup between each other (no > ingress-egress aggregate) as there are a large number of edge LAN > switches/routers in the enterprise network and people calling patterns > change. They normal expect that they should be able to call anyone. >=20 >=20 > Regards, Joe > email:babiarz@nortel.com > Telephone:613-763-6098 >=20 > -----Original Message----- > From: Lars Eggert [mailto:lars.eggert@nokia.com] > Sent: October 25, 2007 5:54 AM > To: Babiarz, Jozef (CAR:0S03) > Cc: Geib, Ruediger; pcn@ietf.org > Subject: Re: [PCN] Architecture draft - probing section & general > updates. >=20 > On 2007-10-24, at 19:14, ext Jozef Babiarz wrote: > > I'm thinking of a scenario that many enterprises face today. Large > > multi > > location enterprises use multiple WAN links or VPS to interconnect > > their > > locations. PCN is run inside the enterprise network including WAN > > links > > which maybe tunneled across the carrier network. Network Access > > Control > > with PCN admission control is done at the enterprise access edge > > nodes. > > New flows can be routed to any egress access edge node and some flows > > will (between different locations) be routed over one of the WAN links > > which are normally bandwidth constrained. >=20 > I understand your scenario so far. >=20 > > There is a high probability > > that many access nodes will only have one flow between each other as > > there are a large number of them. >=20 > I don't see how this follows, however. It seems that if the sites > that are being interconnected aren't tiny, there should very likely > be multiple flows per ingress/egress pair, no? >=20 > Lars >=20 >=20 > _______________________________________________ > PCN mailing list > PCN@ietf.org > https://www1.ietf.org/mailman/listinfo/pcn _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Mon Nov 05 14:30:37 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Ip7ef-0005vF-0z; Mon, 05 Nov 2007 14:30:37 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Ip7ee-0005uR-Cg for pcn-confirm+ok@megatron.ietf.org; Mon, 05 Nov 2007 14:30:36 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Ip7ee-0005uJ-1P for pcn@ietf.org; Mon, 05 Nov 2007 14:30:36 -0500 Received: from smtp1.smtp.bt.com ([217.32.164.137]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1Ip7ed-0000SG-5n for pcn@ietf.org; Mon, 05 Nov 2007 14:30:35 -0500 Received: from E03MVZ1-UKDY.domain1.systemhost.net ([193.113.30.65]) by smtp1.smtp.bt.com with Microsoft SMTPSVC(6.0.3790.1830); Mon, 5 Nov 2007 19:30:33 +0000 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Mon, 5 Nov 2007 19:30:32 -0000 Message-ID: <75A199C5D243C741BF3D3F1EBCEF9BA503B342DD@E03MVZ1-UKDY.domain1.systemhost.net> In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: 5.5. Probing functions Thread-Index: Acgce7uAIUGeKCASRrCiFdfKcQLMsQDYoCjA From: To: , X-OriginalArrivalTime: 05 Nov 2007 19:30:33.0797 (UTC) FILETIME=[5585EF50:01C81FE2] X-Spam-Score: 0.0 (/) X-Scan-Signature: b1c41982e167b872076d0018e4e1dc3c Cc: Subject: [PCN] RE: 5.5. Probing functions X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org General conclusion is that next version of arch draft needs to include more on the downsides of probing. Also seems to be a feeling that should be out of initial scope [not sure if this needs saying in draft] Follow-ups in-line Phil > -----Original Message----- > From: Lars Eggert [mailto:lars.eggert@nokia.com] > Sent: 01 November 2007 11:38 > To: pcn > Cc: Eardley,PL,Philip,CXR9 R > Subject: 5.5. Probing functions >=20 [cut] > > A probe packet is just a dummy data packet, generated by > > the PCN-ingress-node and addressed to the PCN-egress-node. A > > downside of probing is that it adds delay to the admission control > > process. Also note that in the scenario described in the previous > > paragraph (where traffic levels on other ingress-egress- > > aggregates is > > already very high), the probe packets may also 'tip the balance'. >=20 > This discussion on the downsides of probing is too short. Let me list > some issues: >=20 > (1) The ingress needs to generate traffic in a pattern and at a rate > that lets it draw conclusions about whether it is OK to admit the > flow that is waiting for admission. How does it know what > characteristics the probe traffic should have and how does it > generate the probe traffic? >=20 > (2) How long do you need to probe for before you declare it safe to > admit the new flow? What's the delay before a new flow can be > admitted, and can apps actually deal with this delay? >=20 > (3) Since you're sending probe traffic at a rate that is likely not > insignificant, how do you prevent the probe traffic itself from > causing congestion and triggering stop-admission or flow-termination > actions? If you're treating it differently (e.g., at a lower > priority), how is what the probe traffic experiences still > representative of what the real flow would experience? (How can you > treat it differently and still make ECMP work?) >=20 > Without at least some good ideas about what the answers to these > questions will be I believe it is premature to declare that probing > is an optional component of PCN. I think these are fair points. My impression is that probing advocates have 2 reasons: - the on-going ingress-egress-aggregate measurement of marked pkts doesn't work. Because there's often no traffic on this ingress-egress-aggregate, or no traffic on the ECMP path; AND simply admitting the flow has a significant risk of leading to overload [pkts dropped &/or flows terminated] - there's no signalling state on the PCN-boundary-nodes for this ingress-egress-aggregate (and no traffic already on it); AND the router behaviour is such that every pkt is marked if the flow should be blocked; AND just admitting the call is dangerous because there's a good chance of overload in some parts of the nw (near the edge where the capacity is low). Hence probing creates no extra delay (since you have to send a signalling set-up msg anyway). Often this scenario is people who envisage PCN going to end terminals or at least further out than core nw [I think]. The first scenario will only happen in narrow conditions [flash crowds, many ingress-egress-aggregates without any traffic]. An alternative way of handling this is to limit the rate at which admit new flows [at least this makes the conditions when the problem occurs even narrower]. The second scenario seems to break the (link) aggregation assumption? presumably it could be solved by running different mechanism at the edge compared with at the core. [joe, sorry if you've already explained, the late hour =3D> even poorer memory than usual.] >=20 > > However, the risk should be reduced because it should be > > possible to > > send probe packets for a shorter time and at a lower rate than a > > typical data flow. >=20 > This claim is related to (1) and (2) above. Is there any evidence > that is it possible to "send probe packets for a shorter time and at > a lower rate than a typical data flow"? I'm not aware of any research > in this space. >=20 > > The situation is more complicated if there is multipath routing > > (ECMP) in the PCN-domain. It is then possible for some paths to be > > pre-congested whilst other paths within the same ingress-egress- > > aggregate aren't pre-congested. > > > > One approach essentially ignores ECMP: as usual, admit or block > > a new > > flow depending on the "measurements of PCN-traffic" on the ingress- > > egress-aggregate. This is rather similar to the "optimistic" > > approach above. > > > > An alternative ("pessimistic" or "proactive") approach is to probe > > the ECMP path. The PCN-ingress-node generates and sends probe > > packets (dummy data) that follow the specific ECMP path that the > > new > > flow would do, in order to test the pre-congestion level along it. > > An ECMP algorithm typically examines: the source and destination IP > > addresses and port numbers, the protocol ID and the DSCP. Hence > > these fields must have the same values in the probe packets as the > > future data packets would have. On the other hand, the PCN-egress- > > node needs to consume the probe packets to ensure that they don't > > travel beyond the PCN-domain (eg they might confuse the destination > > end node). Hence somehow the PCN-egress-node has to be able to > > disambiguate a probe packet from a data packet, via the > > characteristic setting of particular bit(s) in the packet's > > header or > > body - but these bit(s) mustn't be used by any PCN-interior-node's > > ECMP algorithm. This should be possible with a typical ECMP > > algorithm, but isn't in the general case. >=20 > Some things in this paragraph aren't specific to ECMP, such as that > the egress needs to consume the probe packets. in the non-ecmp scenario the pcn-ingress-node is sending probes addressed to the pcn-egress-node, so it's quite easy for the egress to consume them! _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Tue Nov 06 03:19:34 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpJem-0001vN-Ur; Tue, 06 Nov 2007 03:19:32 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IpJel-0001tb-4F for pcn-confirm+ok@megatron.ietf.org; Tue, 06 Nov 2007 03:19:31 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpJeg-0001lG-7S for pcn@ietf.org; Tue, 06 Nov 2007 03:19:26 -0500 Received: from smtp.nokia.com ([131.228.20.173] helo=mgw-ext14.nokia.com) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IpJed-0002pt-Fo for pcn@ietf.org; Tue, 06 Nov 2007 03:19:26 -0500 Received: from esebh106.NOE.Nokia.com (esebh106.ntc.nokia.com [172.21.138.213]) by mgw-ext14.nokia.com (Switch-3.2.5/Switch-3.2.5) with ESMTP id lA68IwF9008956; Tue, 6 Nov 2007 10:19:20 +0200 Received: from esebh103.NOE.Nokia.com ([172.21.143.33]) by esebh106.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.1830); Tue, 6 Nov 2007 10:19:17 +0200 Received: from mgw-int02.ntc.nokia.com ([172.21.143.97]) by esebh103.NOE.Nokia.com over TLS secured channel with Microsoft SMTPSVC(6.0.3790.1830); Tue, 6 Nov 2007 10:19:16 +0200 Received: from [172.21.35.2] (esdhcp0352.research.nokia.com [172.21.35.2]) by mgw-int02.ntc.nokia.com (Switch-3.2.5/Switch-3.2.5) with ESMTP id lA68JFB3012836; Tue, 6 Nov 2007 10:19:15 +0200 In-Reply-To: <75A199C5D243C741BF3D3F1EBCEF9BA503B342DD@E03MVZ1-UKDY.domain1.systemhost.net> References: <75A199C5D243C741BF3D3F1EBCEF9BA503B342DD@E03MVZ1-UKDY.domain1.systemhost.net> Mime-Version: 1.0 (Apple Message framework v752.3) Message-Id: From: Lars Eggert Date: Tue, 6 Nov 2007 10:19:11 +0200 To: "ext philip.eardley@bt.com" X-Mailer: Apple Mail (2.752.3) X-OriginalArrivalTime: 06 Nov 2007 08:19:16.0681 (UTC) FILETIME=[B8E78390:01C8204D] X-Nokia-AV: Clean X-Spam-Score: 0.0 (/) X-Scan-Signature: 2beba50d0fcdeee5f091c59f204d4365 Cc: pcn@ietf.org Subject: [PCN] Re: 5.5. Probing functions X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1031015123==" Errors-To: pcn-bounces@ietf.org --===============1031015123== Content-Type: multipart/signed; micalg=sha1; boundary=Apple-Mail-4--865896824; protocol="application/pkcs7-signature" --Apple-Mail-4--865896824 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Hi, On 2007-11-5, at 21:30, ext philip.eardley@bt.com wrote: > My impression is that probing advocates have 2 reasons: thanks for summarizing this! I think this does capture the main points made. > - the on-going ingress-egress-aggregate measurement of marked pkts > doesn't work. Because there's often no traffic on this > ingress-egress-aggregate, or no traffic on the ECMP path; AND simply > admitting the flow has a significant risk of leading to overload [pkts > dropped &/or flows terminated] > - there's no signalling state on the PCN-boundary-nodes for this > ingress-egress-aggregate (and no traffic already on it); AND the > router > behaviour is such that every pkt is marked if the flow should be > blocked; AND just admitting the call is dangerous because there's a > good > chance of overload in some parts of the nw (near the edge where the > capacity is low). Hence probing creates no extra delay (since you have > to send a signalling set-up msg anyway). Often this scenario is people > who envisage PCN going to end terminals or at least further out than > core nw [I think]. Although I agree that we can construct scenarios where "simply admitting the flow has a significant risk of leading to overload", I still remain unclear on how likely such cases are in whatever we envision the most likely deployment scenarios to be. Basically, I worry that we're making the architecture complex for what will provide little real benefit in the end. Also, don't forget that using probing will mean dropping the packets of the flow waiting for admission for an RTT *every time* probing is used. Whereas "just admitting" may only cause overload sometimes. There is a tradeoff here - I believe it's possible to construct a case where probing hurts more than "just admitting." And I'm still waiting for someone to convince me that dropping the initial packets of a flow at the ingress during the probing RTT will result in something that is acceptable for apps. > The first scenario will only happen in narrow conditions [flash > crowds, > many ingress-egress-aggregates without any traffic]. An alternative > way > of handling this is to limit the rate at which admit new flows [at > least > this makes the conditions when the problem occurs even narrower]. And we should try to determine how narrow these conditions are. If they're very narrow, we may be able to avoid them through configuration suggestions (sufficient overprovisioning, sufficiently broad marking regions, etc.) > The second scenario seems to break the (link) aggregation assumption? > presumably it could be solved by running different mechanism at the > edge > compared with at the core. [joe, sorry if you've already explained, > the > late hour => even poorer memory than usual.] Yes, we seem to be running into the "the number of flows across any potential aggregation bottleneck is sufficiently large for stateless, statistical mechanisms to be effective" restriction from the charter here. These scenarios seems to require a deterministic mechanism, while PCN is supposed to provide a probabilistic one. Lars --Apple-Mail-4--865896824 Content-Transfer-Encoding: base64 Content-Type: application/pkcs7-signature; name=smime.p7s Content-Disposition: attachment; filename=smime.p7s MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIGQDCCAvkw ggJioAMCAQICEGtxkLFHQu1g67hlmYfwPuIwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UEBhMCWkEx JTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQ ZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA3MDEwNDE2MTE1NFoXDTA4MDEwNDE2MTE1 NFowXDEPMA0GA1UEBBMGRWdnZXJ0MQ0wCwYDVQQqEwRMYXJzMRQwEgYDVQQDEwtMYXJzIEVnZ2Vy dDEkMCIGCSqGSIb3DQEJARYVbGFycy5lZ2dlcnRAbm9raWEuY29tMIIBIjANBgkqhkiG9w0BAQEF AAOCAQ8AMIIBCgKCAQEA26hjntVPZMiwh4d8tyuk9KiucvG92BVUGArk5zO1jhmq7tLFgU1mqcIg VGumfQqjkAzIuh83kxIiqBrB05Wvxp85apwt4sUCvzMe8mQWzZZZYp0rBzwpOj+VC5pxsMRYtYW+ BaTFBEmvFr7D7C7roZUk0lDppS5SZFs4SQbEe9ykB9aeUf1iTiw2+ikyP2+JEYto14WYCoxFWbms FbQ1uaaK+yn1WH2YHhyMfi0IOxuT0jtbsngjHJMpWIqq334zBhj+rPSUug47YBHr41FCDMHbbbjv WbyTyDYneOW3WY8LY8bGWVlAknQGB7lgduShe6e0RI/7SL/VE6X27lM9LwIDAQABozIwMDAgBgNV HREEGTAXgRVsYXJzLmVnZ2VydEBub2tpYS5jb20wDAYDVR0TAQH/BAIwADANBgkqhkiG9w0BAQUF AAOBgQCx3nxxTg/WowobVTRXBiXUEEZj4LBiunWbuAnR0UbJIgijvq3JbiJvZo2gUGiW9LPaHSBz 4oxT1VR/S/6O0a4e87oV5cOQm6ReB68o9rDfUzxHTwoCfv2wwMwBrXyzQMjyQK4Lt8siV41gmZpm rSXMhjTJdd9uYYNoZKQLq+VM+TCCAz8wggKooAMCAQICAQ0wDQYJKoZIhvcNAQEFBQAwgdExCzAJ BgNVBAYTAlpBMRUwEwYDVQQIEwxXZXN0ZXJuIENhcGUxEjAQBgNVBAcTCUNhcGUgVG93bjEaMBgG A1UEChMRVGhhd3RlIENvbnN1bHRpbmcxKDAmBgNVBAsTH0NlcnRpZmljYXRpb24gU2VydmljZXMg RGl2aXNpb24xJDAiBgNVBAMTG1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBDQTErMCkGCSqGSIb3 DQEJARYccGVyc29uYWwtZnJlZW1haWxAdGhhd3RlLmNvbTAeFw0wMzA3MTcwMDAwMDBaFw0xMzA3 MTYyMzU5NTlaMGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5 KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQTCBnzAN BgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAxKY8VXNV+065yplaHmjAdQRwnd/p/6Me7L3N9VvyGna9 fww6YfK/Uc4B1OVQCjDXAmNaLIkVcI7dyfArhVqqP3FWy688Cwfn8R+RNiQqE88r1fOCdz0Dviv+ uxg+B79AgAJk16emu59l0cUqVIUPSAR/p7bRPGEEQB5kGXJgt/sCAwEAAaOBlDCBkTASBgNVHRMB Af8ECDAGAQH/AgEAMEMGA1UdHwQ8MDowOKA2oDSGMmh0dHA6Ly9jcmwudGhhd3RlLmNvbS9UaGF3 dGVQZXJzb25hbEZyZWVtYWlsQ0EuY3JsMAsGA1UdDwQEAwIBBjApBgNVHREEIjAgpB4wHDEaMBgG A1UEAxMRUHJpdmF0ZUxhYmVsMi0xMzgwDQYJKoZIhvcNAQEFBQADgYEASIzRUIPqCy7MDaNmrGcP f6+svsIXoUOWlJ1/TCG4+DYfqi2fNi/A9BxQIJNwPP2t4WFiw9k6GX6EsZkbAMUaC4J0niVQlGLH 2ydxVyWN3amcOY6MIE9lX5Xa9/eH1sYITq726jTlEBpbNU1341YheILcIRk13iSx0x1G/11fZU8x ggMQMIIDDAIBATB2MGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAo UHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQ a3GQsUdC7WDruGWZh/A+4jAJBgUrDgMCGgUAoIIBbzAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcB MBwGCSqGSIb3DQEJBTEPFw0wNzExMDYwODE5MTFaMCMGCSqGSIb3DQEJBDEWBBSVxeC/4BMUlmkv 2d/v3OQFgT9yOzCBhQYJKwYBBAGCNxAEMXgwdjBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhh d3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVt YWlsIElzc3VpbmcgQ0ECEGtxkLFHQu1g67hlmYfwPuIwgYcGCyqGSIb3DQEJEAILMXigdjBiMQsw CQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UE AxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0ECEGtxkLFHQu1g67hlmYfwPuIw DQYJKoZIhvcNAQEBBQAEggEAmeaFmk3j89ebOU+qUhhb22qxfcn3WWhSD2PVNAd3Xse7ckLi5vgn UYFbw0a86aGiAVH/rmAKDV1EPAkaz+J+bTdz/luhAwWEL/vWRSCpDg+0TAiiYe9qgXzNFX1Pi1wy R6wjkOf7p0tJRz51pwCmXLEtd8PEwfBCh7CZdtFY7tgUQrJ1Q2ybif+c3+dq+Quu3DD1gO/3egAo azIPRJRgi8h6hxCdFavshaqDfxHV0+tFdyJJaHB/lek7TLsw7OszNXVB5TJH4ygz0+LtCdIr47r4 +NYKD0PWERjQoaFUGxPw/uaR5xwX8zYcQjWRU/gZ69fYjK/bbMaynwq6TgA+QAAAAAAAAA== --Apple-Mail-4--865896824-- --===============1031015123== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn --===============1031015123==-- From pcn-bounces@ietf.org Tue Nov 06 03:37:07 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpJvn-0005NG-Ih; Tue, 06 Nov 2007 03:37:07 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IpJvm-0005IU-2T for pcn-confirm+ok@megatron.ietf.org; Tue, 06 Nov 2007 03:37:06 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpJvl-0005Hp-OE for pcn@ietf.org; Tue, 06 Nov 2007 03:37:05 -0500 Received: from wrzx28.rz.uni-wuerzburg.de ([132.187.3.28] helo=mailrelay.rz.uni-wuerzburg.de) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IpJvh-0003P7-Q7 for pcn@ietf.org; Tue, 06 Nov 2007 03:37:05 -0500 Received: from virusscan.mail (localhost [127.0.0.1]) by mailrelay.mail (Postfix) with ESMTP id F3297C02A; Tue, 6 Nov 2007 09:36:58 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by virusscan.mail (Postfix) with ESMTP id E532BC0B9; Tue, 6 Nov 2007 09:36:58 +0100 (CET) Received: from europa.informatik.uni-wuerzburg.de (wicx01.informatik.uni-wuerzburg.de [132.187.11.1]) by mailmaster.uni-wuerzburg.de (Postfix) with ESMTP id 9818CC02A; Tue, 6 Nov 2007 09:36:56 +0100 (CET) Received: from nero.informatik.uni-wuerzburg.de (win3005.informatik.uni-wuerzburg.de [132.187.106.5]) by europa.informatik.uni-wuerzburg.de (8.11.3/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id lA68auh10325; Tue, 6 Nov 2007 09:36:56 +0100 Received: from [127.0.0.1] (nero.informatik.uni-wuerzburg.de [132.187.106.5]) by nero.informatik.uni-wuerzburg.de (Postfix) with ESMTP id B2D516F591; Tue, 6 Nov 2007 09:29:28 +0100 (CET) Message-ID: <473025CC.7020408@informatik.uni-wuerzburg.de> Date: Tue, 06 Nov 2007 09:29:00 +0100 From: Michael Menth Organization: University of Wuerzburg User-Agent: Thunderbird 2.0.0.6 (Windows/20070728) MIME-Version: 1.0 To: Lars Eggert Subject: Re: [PCN] Re: 5.5. Probing functions References: <75A199C5D243C741BF3D3F1EBCEF9BA503B342DD@E03MVZ1-UKDY.domain1.systemhost.net> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at uni-wuerzburg.de X-Spam-Score: 0.0 (/) X-Scan-Signature: f66b12316365a3fe519e75911daf28a8 Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: menth@informatik.uni-wuerzburg.de List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Lars and Phil, I think that one major motivation for probing is to make PCN capable to deal with multipath routing like ECMP. Some path are empty, others are full. Some flows must be rejected, others can proceed although they all belong to the same ingress-egress-aggregate. Regards, Michael Lars Eggert wrote: > Hi, > > On 2007-11-5, at 21:30, ext philip.eardley@bt.com wrote: >> My impression is that probing advocates have 2 reasons: > > thanks for summarizing this! I think this does capture the main points > made. > >> - the on-going ingress-egress-aggregate measurement of marked pkts >> doesn't work. Because there's often no traffic on this >> ingress-egress-aggregate, or no traffic on the ECMP path; AND simply >> admitting the flow has a significant risk of leading to overload [pkts >> dropped &/or flows terminated] >> - there's no signalling state on the PCN-boundary-nodes for this >> ingress-egress-aggregate (and no traffic already on it); AND the router >> behaviour is such that every pkt is marked if the flow should be >> blocked; AND just admitting the call is dangerous because there's a good >> chance of overload in some parts of the nw (near the edge where the >> capacity is low). Hence probing creates no extra delay (since you have >> to send a signalling set-up msg anyway). Often this scenario is people >> who envisage PCN going to end terminals or at least further out than >> core nw [I think]. > > Although I agree that we can construct scenarios where "simply > admitting the flow has a significant risk of leading to overload", I > still remain unclear on how likely such cases are in whatever we > envision the most likely deployment scenarios to be. Basically, I > worry that we're making the architecture complex for what will provide > little real benefit in the end. > > Also, don't forget that using probing will mean dropping the packets > of the flow waiting for admission for an RTT *every time* probing is > used. Whereas "just admitting" may only cause overload sometimes. > There is a tradeoff here - I believe it's possible to construct a case > where probing hurts more than "just admitting." > > And I'm still waiting for someone to convince me that dropping the > initial packets of a flow at the ingress during the probing RTT will > result in something that is acceptable for apps. > >> The first scenario will only happen in narrow conditions [flash crowds, >> many ingress-egress-aggregates without any traffic]. An alternative way >> of handling this is to limit the rate at which admit new flows [at least >> this makes the conditions when the problem occurs even narrower]. > > And we should try to determine how narrow these conditions are. If > they're very narrow, we may be able to avoid them through > configuration suggestions (sufficient overprovisioning, sufficiently > broad marking regions, etc.) > >> The second scenario seems to break the (link) aggregation assumption? >> presumably it could be solved by running different mechanism at the edge >> compared with at the core. [joe, sorry if you've already explained, the >> late hour => even poorer memory than usual.] > > Yes, we seem to be running into the "the number of flows across any > potential aggregation bottleneck is sufficiently large for stateless, > statistical mechanisms to be effective" restriction from the charter > here. These scenarios seems to require a deterministic mechanism, > while PCN is supposed to provide a probabilistic one. > > Lars > ------------------------------------------------------------------------ > > _______________________________________________ > PCN mailing list > PCN@ietf.org > https://www1.ietf.org/mailman/listinfo/pcn > -- Dr. Michael Menth, Assistant Professor University of Wuerzburg, Institute of Computer Science Am Hubland, D-97074 Wuerzburg, Germany, room B206 phone: (+49)-931/888-6644, fax: (+49)-931/888-6632 mailto:menth@informatik.uni-wuerzburg.de http://www3.informatik.uni-wuerzburg.de/research/ngn _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Tue Nov 06 04:13:16 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpKUm-00080K-Lv; Tue, 06 Nov 2007 04:13:16 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IpKUl-0007zo-Hm for pcn-confirm+ok@megatron.ietf.org; Tue, 06 Nov 2007 04:13:15 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpKUl-0007zg-8J for pcn@ietf.org; Tue, 06 Nov 2007 04:13:15 -0500 Received: from smtp3.smtp.bt.com ([217.32.164.138]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IpKUg-0004Uc-2e for pcn@ietf.org; Tue, 06 Nov 2007 04:13:15 -0500 Received: from E03MVZ1-UKDY.domain1.systemhost.net ([193.113.30.65]) by smtp3.smtp.bt.com with Microsoft SMTPSVC(6.0.3790.1830); Tue, 6 Nov 2007 09:13:09 +0000 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [PCN] Re: 5.5. Probing functions Date: Tue, 6 Nov 2007 09:13:08 -0000 Message-ID: <75A199C5D243C741BF3D3F1EBCEF9BA503B342E0@E03MVZ1-UKDY.domain1.systemhost.net> In-Reply-To: <473025CC.7020408@informatik.uni-wuerzburg.de> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] Re: 5.5. Probing functions Thread-Index: AcggUDKa4LUC41jsStq0PhNQgQOoOwABFx6A From: To: , X-OriginalArrivalTime: 06 Nov 2007 09:13:09.0619 (UTC) FILETIME=[3FE2B430:01C82055] X-Spam-Score: -1.0 (-) X-Scan-Signature: 6ffdee8af20de249c24731d8414917d3 Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Michael Do you have an idea of how likely this is? will it be the case that for a small increase in load all/most of the ECMP paths become full? Does it depend strongly on topology? If each link is shared by mnay ecmp paths from many different ingress-egress-aggregates is this less of a risk? phil > -----Original Message----- > From: Michael Menth [mailto:menth@informatik.uni-wuerzburg.de] > Sent: 06 November 2007 08:29 > To: Lars Eggert > Cc: Eardley,PL,Philip,CXR9 R; pcn@ietf.org > Subject: Re: [PCN] Re: 5.5. Probing functions >=20 > Hi Lars and Phil, >=20 > I think that one major motivation for probing is to make PCN capable to > deal with multipath routing like ECMP. Some path are empty, others are > full. Some flows must be rejected, others can proceed although they all > belong to the same ingress-egress-aggregate. >=20 > Regards, >=20 > Michael >=20 > Lars Eggert wrote: > > Hi, > > > > On 2007-11-5, at 21:30, ext philip.eardley@bt.com wrote: > >> My impression is that probing advocates have 2 reasons: > > > > thanks for summarizing this! I think this does capture the main points > > made. > > > >> - the on-going ingress-egress-aggregate measurement of marked pkts > >> doesn't work. Because there's often no traffic on this > >> ingress-egress-aggregate, or no traffic on the ECMP path; AND simply > >> admitting the flow has a significant risk of leading to overload [pkts > >> dropped &/or flows terminated] > >> - there's no signalling state on the PCN-boundary-nodes for this > >> ingress-egress-aggregate (and no traffic already on it); AND the router > >> behaviour is such that every pkt is marked if the flow should be > >> blocked; AND just admitting the call is dangerous because there's a > good > >> chance of overload in some parts of the nw (near the edge where the > >> capacity is low). Hence probing creates no extra delay (since you have > >> to send a signalling set-up msg anyway). Often this scenario is people > >> who envisage PCN going to end terminals or at least further out than > >> core nw [I think]. > > > > Although I agree that we can construct scenarios where "simply > > admitting the flow has a significant risk of leading to overload", I > > still remain unclear on how likely such cases are in whatever we > > envision the most likely deployment scenarios to be. Basically, I > > worry that we're making the architecture complex for what will provide > > little real benefit in the end. > > > > Also, don't forget that using probing will mean dropping the packets > > of the flow waiting for admission for an RTT *every time* probing is > > used. Whereas "just admitting" may only cause overload sometimes. > > There is a tradeoff here - I believe it's possible to construct a case > > where probing hurts more than "just admitting." > > > > And I'm still waiting for someone to convince me that dropping the > > initial packets of a flow at the ingress during the probing RTT will > > result in something that is acceptable for apps. > > > >> The first scenario will only happen in narrow conditions [flash crowds, > >> many ingress-egress-aggregates without any traffic]. An alternative way > >> of handling this is to limit the rate at which admit new flows [at > least > >> this makes the conditions when the problem occurs even narrower]. > > > > And we should try to determine how narrow these conditions are. If > > they're very narrow, we may be able to avoid them through > > configuration suggestions (sufficient overprovisioning, sufficiently > > broad marking regions, etc.) > > > >> The second scenario seems to break the (link) aggregation assumption? > >> presumably it could be solved by running different mechanism at the > edge > >> compared with at the core. [joe, sorry if you've already explained, the > >> late hour =3D> even poorer memory than usual.] > > > > Yes, we seem to be running into the "the number of flows across any > > potential aggregation bottleneck is sufficiently large for stateless, > > statistical mechanisms to be effective" restriction from the charter > > here. These scenarios seems to require a deterministic mechanism, > > while PCN is supposed to provide a probabilistic one. > > > > Lars > > ------------------------------------------------------------------------ > > > > _______________________________________________ > > PCN mailing list > > PCN@ietf.org > > https://www1.ietf.org/mailman/listinfo/pcn > > >=20 > -- > Dr. Michael Menth, Assistant Professor > University of Wuerzburg, Institute of Computer Science > Am Hubland, D-97074 Wuerzburg, Germany, room B206 > phone: (+49)-931/888-6644, fax: (+49)-931/888-6632 > mailto:menth@informatik.uni-wuerzburg.de > http://www3.informatik.uni-wuerzburg.de/research/ngn _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Tue Nov 06 05:33:39 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpLkZ-0007It-1h; Tue, 06 Nov 2007 05:33:39 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IpLkW-0007Ge-UB for pcn-confirm+ok@megatron.ietf.org; Tue, 06 Nov 2007 05:33:36 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpLkW-0007G2-KF for pcn@ietf.org; Tue, 06 Nov 2007 05:33:36 -0500 Received: from wrzx28.rz.uni-wuerzburg.de ([132.187.3.28] helo=mailrelay.rz.uni-wuerzburg.de) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IpLkT-0006Y5-IN for pcn@ietf.org; Tue, 06 Nov 2007 05:33:36 -0500 Received: from virusscan.mail (localhost [127.0.0.1]) by mailrelay.mail (Postfix) with ESMTP id 0DE32D5E6; Tue, 6 Nov 2007 11:33:33 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by virusscan.mail (Postfix) with ESMTP id 007B1DB0F; Tue, 6 Nov 2007 11:33:33 +0100 (CET) Received: from europa.informatik.uni-wuerzburg.de (wicx01.informatik.uni-wuerzburg.de [132.187.11.1]) by mailmaster.uni-wuerzburg.de (Postfix) with ESMTP id B06CCD5E6; Tue, 6 Nov 2007 11:33:30 +0100 (CET) Received: from nero.informatik.uni-wuerzburg.de (win3005.informatik.uni-wuerzburg.de [132.187.106.5]) by europa.informatik.uni-wuerzburg.de (8.11.3/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id lA6AXUh11396; Tue, 6 Nov 2007 11:33:30 +0100 Received: from [127.0.0.1] (nero.informatik.uni-wuerzburg.de [132.187.106.5]) by nero.informatik.uni-wuerzburg.de (Postfix) with ESMTP id 670FF6F591; Tue, 6 Nov 2007 11:26:02 +0100 (CET) Message-ID: <4730411D.9010604@informatik.uni-wuerzburg.de> Date: Tue, 06 Nov 2007 11:25:33 +0100 From: Michael Menth Organization: University of Wuerzburg User-Agent: Thunderbird 2.0.0.6 (Windows/20070728) MIME-Version: 1.0 To: philip.eardley@bt.com Subject: Re: [PCN] Re: 5.5. Probing functions References: <75A199C5D243C741BF3D3F1EBCEF9BA503B342E0@E03MVZ1-UKDY.domain1.systemhost.net> In-Reply-To: <75A199C5D243C741BF3D3F1EBCEF9BA503B342E0@E03MVZ1-UKDY.domain1.systemhost.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at uni-wuerzburg.de X-Spam-Score: 0.0 (/) X-Scan-Signature: 612a16ba5c5f570bfc42b3ac5606ac53 Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: menth@informatik.uni-wuerzburg.de List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Phil and others, I don't know how likely this is in operational networks, it certainly depends on topology, routing, and traffic dynamics. We did a study on load balancing algorithms with a different focus. It turned out that with static load balancing significant imbalances occur due to statistical effects even if a 50:50 balance is envisaged. http://www3.informatik.uni-wuerzburg.de/staff/menth/Publications/Menth06p.pdf This is just one source of traffic imbalance. Another and more important issue is that links of a network can have substantially different utilization, no matter due to with reason (different bandwidth, different traffic load, different amount of backup capacity, ...). Therefore, some alternative paths of a single ingress-egress-aggregate have high utilization, others have low utilization. If the load increases, some flows of that aggregate must be blocked while others can safely be admitted. If we do not consider this issue, we need to ask ourselves why we look at admission control at all for such networks instead of just doing reasonable overprovisioning (if it's possible and economically viable). Regards, Michael philip.eardley@bt.com wrote: > Michael > > Do you have an idea of how likely this is? will it be the case that for > a small increase in load all/most of the ECMP paths become full? Does it > depend strongly on topology? If each link is shared by mnay ecmp paths > from many different ingress-egress-aggregates is this less of a risk? > > phil > > >> -----Original Message----- >> From: Michael Menth [mailto:menth@informatik.uni-wuerzburg.de] >> Sent: 06 November 2007 08:29 >> To: Lars Eggert >> Cc: Eardley,PL,Philip,CXR9 R; pcn@ietf.org >> Subject: Re: [PCN] Re: 5.5. Probing functions >> >> Hi Lars and Phil, >> >> I think that one major motivation for probing is to make PCN capable >> > to > >> deal with multipath routing like ECMP. Some path are empty, others are >> full. Some flows must be rejected, others can proceed although they >> > all > >> belong to the same ingress-egress-aggregate. >> >> Regards, >> >> Michael >> >> Lars Eggert wrote: >> >>> Hi, >>> >>> On 2007-11-5, at 21:30, ext philip.eardley@bt.com wrote: >>> >>>> My impression is that probing advocates have 2 reasons: >>>> >>> thanks for summarizing this! I think this does capture the main >>> > points > >>> made. >>> >>> >>>> - the on-going ingress-egress-aggregate measurement of marked pkts >>>> doesn't work. Because there's often no traffic on this >>>> ingress-egress-aggregate, or no traffic on the ECMP path; AND >>>> > simply > >>>> admitting the flow has a significant risk of leading to overload >>>> > [pkts > >>>> dropped &/or flows terminated] >>>> - there's no signalling state on the PCN-boundary-nodes for this >>>> ingress-egress-aggregate (and no traffic already on it); AND the >>>> > router > >>>> behaviour is such that every pkt is marked if the flow should be >>>> blocked; AND just admitting the call is dangerous because there's a >>>> >> good >> >>>> chance of overload in some parts of the nw (near the edge where the >>>> capacity is low). Hence probing creates no extra delay (since you >>>> > have > >>>> to send a signalling set-up msg anyway). Often this scenario is >>>> > people > >>>> who envisage PCN going to end terminals or at least further out >>>> > than > >>>> core nw [I think]. >>>> >>> Although I agree that we can construct scenarios where "simply >>> admitting the flow has a significant risk of leading to overload", I >>> still remain unclear on how likely such cases are in whatever we >>> envision the most likely deployment scenarios to be. Basically, I >>> worry that we're making the architecture complex for what will >>> > provide > >>> little real benefit in the end. >>> >>> Also, don't forget that using probing will mean dropping the packets >>> of the flow waiting for admission for an RTT *every time* probing is >>> used. Whereas "just admitting" may only cause overload sometimes. >>> There is a tradeoff here - I believe it's possible to construct a >>> > case > >>> where probing hurts more than "just admitting." >>> >>> And I'm still waiting for someone to convince me that dropping the >>> initial packets of a flow at the ingress during the probing RTT will >>> result in something that is acceptable for apps. >>> >>> >>>> The first scenario will only happen in narrow conditions [flash >>>> > crowds, > >>>> many ingress-egress-aggregates without any traffic]. An alternative >>>> > way > >>>> of handling this is to limit the rate at which admit new flows [at >>>> >> least >> >>>> this makes the conditions when the problem occurs even narrower]. >>>> >>> And we should try to determine how narrow these conditions are. If >>> they're very narrow, we may be able to avoid them through >>> configuration suggestions (sufficient overprovisioning, sufficiently >>> broad marking regions, etc.) >>> >>> >>>> The second scenario seems to break the (link) aggregation >>>> > assumption? > >>>> presumably it could be solved by running different mechanism at the >>>> >> edge >> >>>> compared with at the core. [joe, sorry if you've already explained, >>>> > the > >>>> late hour => even poorer memory than usual.] >>>> >>> Yes, we seem to be running into the "the number of flows across any >>> potential aggregation bottleneck is sufficiently large for >>> > stateless, > >>> statistical mechanisms to be effective" restriction from the charter >>> here. These scenarios seems to require a deterministic mechanism, >>> while PCN is supposed to provide a probabilistic one. >>> >>> Lars >>> >>> > ------------------------------------------------------------------------ > >>> _______________________________________________ >>> PCN mailing list >>> PCN@ietf.org >>> https://www1.ietf.org/mailman/listinfo/pcn >>> >>> >> -- >> Dr. Michael Menth, Assistant Professor >> University of Wuerzburg, Institute of Computer Science >> Am Hubland, D-97074 Wuerzburg, Germany, room B206 >> phone: (+49)-931/888-6644, fax: (+49)-931/888-6632 >> mailto:menth@informatik.uni-wuerzburg.de >> http://www3.informatik.uni-wuerzburg.de/research/ngn >> -- Dr. Michael Menth, Assistant Professor University of Wuerzburg, Institute of Computer Science Am Hubland, D-97074 Wuerzburg, Germany, room B206 phone: (+49)-931/888-6644, fax: (+49)-931/888-6632 mailto:menth@informatik.uni-wuerzburg.de http://www3.informatik.uni-wuerzburg.de/research/ngn _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Tue Nov 06 09:31:51 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpPT5-0006vV-0X; Tue, 06 Nov 2007 09:31:51 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IpPT3-0006to-Ne for pcn-confirm+ok@megatron.ietf.org; Tue, 06 Nov 2007 09:31:49 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpPT3-0006tI-BU for pcn@ietf.org; Tue, 06 Nov 2007 09:31:49 -0500 Received: from sj-iport-2-in.cisco.com ([171.71.176.71] helo=sj-iport-2.cisco.com) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IpPT2-0000Wm-99 for pcn@ietf.org; Tue, 06 Nov 2007 09:31:49 -0500 X-IronPort-AV: E=Sophos;i="4.21,378,1188802800"; d="scan'208";a="413767043" Received: from rtp-dkim-2.cisco.com ([64.102.121.159]) by sj-iport-2.cisco.com with ESMTP; 06 Nov 2007 06:31:46 -0800 Received: from rtp-core-2.cisco.com (rtp-core-2.cisco.com [64.102.124.13]) by rtp-dkim-2.cisco.com (8.12.11/8.12.11) with ESMTP id lA6EVkHe012148; Tue, 6 Nov 2007 09:31:46 -0500 Received: from xbh-rtp-211.amer.cisco.com (xbh-rtp-211.cisco.com [64.102.31.102]) by rtp-core-2.cisco.com (8.12.10/8.12.6) with ESMTP id lA6EVVg4023358; Tue, 6 Nov 2007 14:31:45 GMT Received: from xmb-rtp-203.amer.cisco.com ([64.102.31.20]) by xbh-rtp-211.amer.cisco.com with Microsoft SMTPSVC(6.0.3790.1830); Tue, 6 Nov 2007 09:31:43 -0500 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [PCN] Re: 5.5. Probing functions Date: Tue, 6 Nov 2007 09:31:42 -0500 Message-ID: In-Reply-To: <75A199C5D243C741BF3D3F1EBCEF9BA503B342E0@E03MVZ1-UKDY.domain1.systemhost.net> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] Re: 5.5. Probing functions Thread-Index: AcggUDKa4LUC41jsStq0PhNQgQOoOwABFx6AAAnybOA= From: "Anna Charny (acharny)" To: , , X-OriginalArrivalTime: 06 Nov 2007 14:31:43.0792 (UTC) FILETIME=[C0D20B00:01C82081] X-TM-AS-Product-Ver: SMEX-8.0.0.1181-5.000.1023-15528.002 X-TM-AS-Result: No--21.024300-8.000000-31 X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No DKIM-Signature: v=0.5; a=rsa-sha256; q=dns/txt; l=7476; t=1194359506; x=1195223506; c=relaxed/simple; s=rtpdkim2001; h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version; d=cisco.com; i=acharny@cisco.com; z=From:=20=22Anna=20Charny=20(acharny)=22=20 |Subject:=20RE=3A=20[PCN]=20Re=3A=205.5.=20Probing=20functions |Sender:=20 |To:=20, =20, =0A =20=20=20=20=20=20=20=20; bh=AOajupsAv0hEjPE48maEMFH9u/giUyRuUjZz781QGFE=; b=e4O2vdWGnzewfhdpgkRigAn+vEsS+VWaU2hPSlV7hASzGRQA0HMI4ktGGTTP+jTHdmgOI1H2 Tq64BPQC1+w2QlYDx0ZGKf6vguT9wtSIBu34YV6Z7j7/pzpY16lXvrRR; Authentication-Results: rtp-dkim-2; header.From=acharny@cisco.com; dkim=pass ( sig from cisco.com/rtpdkim2001 verified; ); X-Spam-Score: 0.0 (/) X-Scan-Signature: 7a0494a0224ca59418dd8f92694c1fdb Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Phil, Michael, Based on our own internal study here are a few highlights that can shed light on the ECMP accuracy: 1) once the number of flows (at hash granularity) becomes large (~100,000), most hashes perform very well at a single hop for a wide range of hash inputs. 2) when the number of flows in a hash is smaller (not that small actually - on the order of 1000!) practically all hashes display large statistical variation for some distributions of the hash inputs (where large is on the order 20-40% or more of relative error per bucket). This appears to be true for a wide set of hashes, including those shown very robust in many academic studies. The above is for single hop only, and for heterogeneous traffic rates. If flow rates distribution is skewed (i.e. there are a few large flows), it can become much worse wrt actual bandwidth distribution. Some of the known ways to fix this issue is to employ hashes that look deeper into the packets so that large flows (e.g. based on (src, dst) hash) get split into smaller flows (e.g. look not only at src,dst, but also at layer 4 info) - but note that these same methods cause, for our PCN purposes, a problem that rsvp messages may no longer follow the same path as the data packets.=20 So I think realistically, for truly large aggregation at the core, ECMP imbalances are likely to be not much of an issue, but for relatively lower levels of aggregation one might expect that fairly large ECMP inaccuracies in some cases are quite possible. Note that "relatively low aggregation" may occur for tunneled traffic (i.e. lots of small flows are aggregated into a relatively small number of large tunnels).=20 Anna=20 =20 > -----Original Message----- > From: philip.eardley@bt.com [mailto:philip.eardley@bt.com]=20 > Sent: Tuesday, November 06, 2007 4:13 AM > To: menth@informatik.uni-wuerzburg.de; lars.eggert@nokia.com > Cc: pcn@ietf.org > Subject: RE: [PCN] Re: 5.5. Probing functions >=20 > Michael >=20 > Do you have an idea of how likely this is? will it be the=20 > case that for a small increase in load all/most of the ECMP=20 > paths become full? Does it depend strongly on topology? If=20 > each link is shared by mnay ecmp paths from many different=20 > ingress-egress-aggregates is this less of a risk? >=20 > phil >=20 > > -----Original Message----- > > From: Michael Menth [mailto:menth@informatik.uni-wuerzburg.de] > > Sent: 06 November 2007 08:29 > > To: Lars Eggert > > Cc: Eardley,PL,Philip,CXR9 R; pcn@ietf.org > > Subject: Re: [PCN] Re: 5.5. Probing functions > >=20 > > Hi Lars and Phil, > >=20 > > I think that one major motivation for probing is to make PCN capable > to > > deal with multipath routing like ECMP. Some path are empty,=20 > others are=20 > > full. Some flows must be rejected, others can proceed although they > all > > belong to the same ingress-egress-aggregate. > >=20 > > Regards, > >=20 > > Michael > >=20 > > Lars Eggert wrote: > > > Hi, > > > > > > On 2007-11-5, at 21:30, ext philip.eardley@bt.com wrote: > > >> My impression is that probing advocates have 2 reasons: > > > > > > thanks for summarizing this! I think this does capture the main > points > > > made. > > > > > >> - the on-going ingress-egress-aggregate measurement of=20 > marked pkts=20 > > >> doesn't work. Because there's often no traffic on this=20 > > >> ingress-egress-aggregate, or no traffic on the ECMP path; AND > simply > > >> admitting the flow has a significant risk of leading to overload > [pkts > > >> dropped &/or flows terminated] > > >> - there's no signalling state on the PCN-boundary-nodes for this=20 > > >> ingress-egress-aggregate (and no traffic already on it); AND the > router > > >> behaviour is such that every pkt is marked if the flow should be=20 > > >> blocked; AND just admitting the call is dangerous=20 > because there's a > > good > > >> chance of overload in some parts of the nw (near the=20 > edge where the=20 > > >> capacity is low). Hence probing creates no extra delay (since you > have > > >> to send a signalling set-up msg anyway). Often this scenario is > people > > >> who envisage PCN going to end terminals or at least further out > than > > >> core nw [I think]. > > > > > > Although I agree that we can construct scenarios where "simply=20 > > > admitting the flow has a significant risk of leading to=20 > overload", I=20 > > > still remain unclear on how likely such cases are in whatever we=20 > > > envision the most likely deployment scenarios to be. Basically, I=20 > > > worry that we're making the architecture complex for what will > provide > > > little real benefit in the end. > > > > > > Also, don't forget that using probing will mean dropping=20 > the packets=20 > > > of the flow waiting for admission for an RTT *every time*=20 > probing is=20 > > > used. Whereas "just admitting" may only cause overload sometimes. > > > There is a tradeoff here - I believe it's possible to construct a > case > > > where probing hurts more than "just admitting." > > > > > > And I'm still waiting for someone to convince me that=20 > dropping the=20 > > > initial packets of a flow at the ingress during the=20 > probing RTT will=20 > > > result in something that is acceptable for apps. > > > > > >> The first scenario will only happen in narrow conditions [flash > crowds, > > >> many ingress-egress-aggregates without any traffic]. An=20 > alternative > way > > >> of handling this is to limit the rate at which admit new=20 > flows [at > > least > > >> this makes the conditions when the problem occurs even narrower]. > > > > > > And we should try to determine how narrow these=20 > conditions are. If=20 > > > they're very narrow, we may be able to avoid them through=20 > > > configuration suggestions (sufficient overprovisioning,=20 > sufficiently=20 > > > broad marking regions, etc.) > > > > > >> The second scenario seems to break the (link) aggregation > assumption? > > >> presumably it could be solved by running different=20 > mechanism at the > > edge > > >> compared with at the core. [joe, sorry if you've already=20 > explained, > the > > >> late hour =3D> even poorer memory than usual.] > > > > > > Yes, we seem to be running into the "the number of flows=20 > across any=20 > > > potential aggregation bottleneck is sufficiently large for > stateless, > > > statistical mechanisms to be effective" restriction from=20 > the charter=20 > > > here. These scenarios seems to require a deterministic mechanism,=20 > > > while PCN is supposed to provide a probabilistic one. > > > > > > Lars > > > > -------------------------------------------------------------- > ---------- > > > > > > _______________________________________________ > > > PCN mailing list > > > PCN@ietf.org > > > https://www1.ietf.org/mailman/listinfo/pcn > > > > >=20 > > -- > > Dr. Michael Menth, Assistant Professor University of Wuerzburg,=20 > > Institute of Computer Science Am Hubland, D-97074=20 > Wuerzburg, Germany,=20 > > room B206 > > phone: (+49)-931/888-6644, fax: (+49)-931/888-6632=20 > > mailto:menth@informatik.uni-wuerzburg.de > > http://www3.informatik.uni-wuerzburg.de/research/ngn >=20 >=20 >=20 > _______________________________________________ > PCN mailing list > PCN@ietf.org > https://www1.ietf.org/mailman/listinfo/pcn >=20 _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Tue Nov 06 10:28:19 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpQLh-0005bb-5h; Tue, 06 Nov 2007 10:28:17 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IpQLg-0005ZW-6t for pcn-confirm+ok@megatron.ietf.org; Tue, 06 Nov 2007 10:28:16 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpQLf-0005Yw-T7 for pcn@ietf.org; Tue, 06 Nov 2007 10:28:15 -0500 Received: from wrzx28.rz.uni-wuerzburg.de ([132.187.3.28] helo=mailrelay.rz.uni-wuerzburg.de) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IpQLa-000760-Co for pcn@ietf.org; Tue, 06 Nov 2007 10:28:15 -0500 Received: from virusscan.mail (localhost [127.0.0.1]) by mailrelay.mail (Postfix) with ESMTP id AE071F5B7; Tue, 6 Nov 2007 16:28:09 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by virusscan.mail (Postfix) with ESMTP id A08BAF5BA; Tue, 6 Nov 2007 16:28:09 +0100 (CET) Received: from europa.informatik.uni-wuerzburg.de (wicx01.informatik.uni-wuerzburg.de [132.187.11.1]) by mailmaster.uni-wuerzburg.de (Postfix) with ESMTP id 45F17F5B7; Tue, 6 Nov 2007 16:28:06 +0100 (CET) Received: from nero.informatik.uni-wuerzburg.de (win3005.informatik.uni-wuerzburg.de [132.187.106.5]) by europa.informatik.uni-wuerzburg.de (8.11.3/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id lA6FS6h13871; Tue, 6 Nov 2007 16:28:06 +0100 Received: from [127.0.0.1] (nero.informatik.uni-wuerzburg.de [132.187.106.5]) by nero.informatik.uni-wuerzburg.de (Postfix) with ESMTP id 172906F591; Tue, 6 Nov 2007 16:20:38 +0100 (CET) Message-ID: <47308628.1010505@informatik.uni-wuerzburg.de> Date: Tue, 06 Nov 2007 16:20:08 +0100 From: Michael Menth Organization: University of Wuerzburg User-Agent: Thunderbird 2.0.0.6 (Windows/20070728) MIME-Version: 1.0 To: "Anna Charny (acharny)" Subject: Re: [PCN] Re: 5.5. Probing functions References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at uni-wuerzburg.de X-Spam-Score: 0.0 (/) X-Scan-Signature: 8a4bcf8f67063cac573319207fe3db35 Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: menth@informatik.uni-wuerzburg.de List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Anna, Phil, Anna, you addressed the impact of ECMP on possible imbalances of link utilization. I am with you in the sense that this is only a minor issue in networks with high traffic aggregation. However, the point I wanted to make is that different links can have different utilizations in their normal operational mode (e.g. because some of them are provisioned with backup capacity for rerouted traffic, others aren't). This effect is not caused by ECMP or load balancing in general. But the consequence of that effect is that alternative paths can have substantially different utilization which possibly becomes problematic when ECMP is used in the presence of PCN-based admission control. Regards, Michael Anna Charny (acharny) wrote: > Hi Phil, Michael, > > Based on our own internal study here are a few highlights that can shed > light on the ECMP accuracy: > > 1) once the number of flows (at hash granularity) becomes large > (~100,000), most hashes perform very well at a single hop for a wide > range of hash inputs. > 2) when the number of flows in a hash is smaller (not that small > actually - on the order of 1000!) practically all hashes display large > statistical variation for some distributions of the hash inputs (where > large is on the order 20-40% or more of relative error per bucket). > This appears to be true for a wide set of hashes, including those shown > very robust in many academic studies. > > The above is for single hop only, and for heterogeneous traffic rates. > If flow rates distribution is skewed (i.e. there are a few large flows), > it can become much worse wrt actual bandwidth distribution. Some of the > known ways to fix this issue is to employ hashes that look deeper into > the packets so that large flows (e.g. based on (src, dst) hash) get > split into smaller flows (e.g. look not only at src,dst, but also at > layer 4 info) - but note that these same methods cause, for our PCN > purposes, a problem that rsvp messages may no longer follow the same > path as the data packets. > > So I think realistically, for truly large aggregation at the core, ECMP > imbalances are likely to be not much of an issue, but for relatively > lower levels of aggregation one might expect that fairly large ECMP > inaccuracies in some cases are quite possible. Note that "relatively > low aggregation" may occur for tunneled traffic (i.e. lots of small > flows are aggregated into a relatively small number of large tunnels). > > Anna > > > >> -----Original Message----- >> From: philip.eardley@bt.com [mailto:philip.eardley@bt.com] >> Sent: Tuesday, November 06, 2007 4:13 AM >> To: menth@informatik.uni-wuerzburg.de; lars.eggert@nokia.com >> Cc: pcn@ietf.org >> Subject: RE: [PCN] Re: 5.5. Probing functions >> >> Michael >> >> Do you have an idea of how likely this is? will it be the >> case that for a small increase in load all/most of the ECMP >> paths become full? Does it depend strongly on topology? If >> each link is shared by mnay ecmp paths from many different >> ingress-egress-aggregates is this less of a risk? >> >> phil >> >> >>> -----Original Message----- >>> From: Michael Menth [mailto:menth@informatik.uni-wuerzburg.de] >>> Sent: 06 November 2007 08:29 >>> To: Lars Eggert >>> Cc: Eardley,PL,Philip,CXR9 R; pcn@ietf.org >>> Subject: Re: [PCN] Re: 5.5. Probing functions >>> >>> Hi Lars and Phil, >>> >>> I think that one major motivation for probing is to make PCN capable >>> >> to >> >>> deal with multipath routing like ECMP. Some path are empty, >>> >> others are >> >>> full. Some flows must be rejected, others can proceed although they >>> >> all >> >>> belong to the same ingress-egress-aggregate. >>> >>> Regards, >>> >>> Michael >>> >>> Lars Eggert wrote: >>> >>>> Hi, >>>> >>>> On 2007-11-5, at 21:30, ext philip.eardley@bt.com wrote: >>>> >>>>> My impression is that probing advocates have 2 reasons: >>>>> >>>> thanks for summarizing this! I think this does capture the main >>>> >> points >> >>>> made. >>>> >>>> >>>>> - the on-going ingress-egress-aggregate measurement of >>>>> >> marked pkts >> >>>>> doesn't work. Because there's often no traffic on this >>>>> ingress-egress-aggregate, or no traffic on the ECMP path; AND >>>>> >> simply >> >>>>> admitting the flow has a significant risk of leading to overload >>>>> >> [pkts >> >>>>> dropped &/or flows terminated] >>>>> - there's no signalling state on the PCN-boundary-nodes for this >>>>> ingress-egress-aggregate (and no traffic already on it); AND the >>>>> >> router >> >>>>> behaviour is such that every pkt is marked if the flow should be >>>>> blocked; AND just admitting the call is dangerous >>>>> >> because there's a >> >>> good >>> >>>>> chance of overload in some parts of the nw (near the >>>>> >> edge where the >> >>>>> capacity is low). Hence probing creates no extra delay (since you >>>>> >> have >> >>>>> to send a signalling set-up msg anyway). Often this scenario is >>>>> >> people >> >>>>> who envisage PCN going to end terminals or at least further out >>>>> >> than >> >>>>> core nw [I think]. >>>>> >>>> Although I agree that we can construct scenarios where "simply >>>> admitting the flow has a significant risk of leading to >>>> >> overload", I >> >>>> still remain unclear on how likely such cases are in whatever we >>>> envision the most likely deployment scenarios to be. Basically, I >>>> worry that we're making the architecture complex for what will >>>> >> provide >> >>>> little real benefit in the end. >>>> >>>> Also, don't forget that using probing will mean dropping >>>> >> the packets >> >>>> of the flow waiting for admission for an RTT *every time* >>>> >> probing is >> >>>> used. Whereas "just admitting" may only cause overload sometimes. >>>> There is a tradeoff here - I believe it's possible to construct a >>>> >> case >> >>>> where probing hurts more than "just admitting." >>>> >>>> And I'm still waiting for someone to convince me that >>>> >> dropping the >> >>>> initial packets of a flow at the ingress during the >>>> >> probing RTT will >> >>>> result in something that is acceptable for apps. >>>> >>>> >>>>> The first scenario will only happen in narrow conditions [flash >>>>> >> crowds, >> >>>>> many ingress-egress-aggregates without any traffic]. An >>>>> >> alternative >> way >> >>>>> of handling this is to limit the rate at which admit new >>>>> >> flows [at >> >>> least >>> >>>>> this makes the conditions when the problem occurs even narrower]. >>>>> >>>> And we should try to determine how narrow these >>>> >> conditions are. If >> >>>> they're very narrow, we may be able to avoid them through >>>> configuration suggestions (sufficient overprovisioning, >>>> >> sufficiently >> >>>> broad marking regions, etc.) >>>> >>>> >>>>> The second scenario seems to break the (link) aggregation >>>>> >> assumption? >> >>>>> presumably it could be solved by running different >>>>> >> mechanism at the >> >>> edge >>> >>>>> compared with at the core. [joe, sorry if you've already >>>>> >> explained, >> the >> >>>>> late hour => even poorer memory than usual.] >>>>> >>>> Yes, we seem to be running into the "the number of flows >>>> >> across any >> >>>> potential aggregation bottleneck is sufficiently large for >>>> >> stateless, >> >>>> statistical mechanisms to be effective" restriction from >>>> >> the charter >> >>>> here. These scenarios seems to require a deterministic mechanism, >>>> while PCN is supposed to provide a probabilistic one. >>>> >>>> Lars >>>> >>>> >> -------------------------------------------------------------- >> ---------- >> >>>> _______________________________________________ >>>> PCN mailing list >>>> PCN@ietf.org >>>> https://www1.ietf.org/mailman/listinfo/pcn >>>> >>>> >>> -- >>> Dr. Michael Menth, Assistant Professor University of Wuerzburg, >>> Institute of Computer Science Am Hubland, D-97074 >>> >> Wuerzburg, Germany, >> >>> room B206 >>> phone: (+49)-931/888-6644, fax: (+49)-931/888-6632 >>> mailto:menth@informatik.uni-wuerzburg.de >>> http://www3.informatik.uni-wuerzburg.de/research/ngn >>> >> >> _______________________________________________ >> PCN mailing list >> PCN@ietf.org >> https://www1.ietf.org/mailman/listinfo/pcn >> >> -- Dr. Michael Menth, Assistant Professor University of Wuerzburg, Institute of Computer Science Am Hubland, D-97074 Wuerzburg, Germany, room B206 phone: (+49)-931/888-6644, fax: (+49)-931/888-6632 mailto:menth@informatik.uni-wuerzburg.de http://www3.informatik.uni-wuerzburg.de/research/ngn _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Tue Nov 06 10:39:08 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpQWB-0002TO-UP; Tue, 06 Nov 2007 10:39:07 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IpQWA-0002Sa-Pb for pcn-confirm+ok@megatron.ietf.org; Tue, 06 Nov 2007 10:39:06 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpQWA-0002S1-DN for pcn@ietf.org; Tue, 06 Nov 2007 10:39:06 -0500 Received: from sj-iport-5.cisco.com ([171.68.10.87]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IpQW9-0002WF-2Z for pcn@ietf.org; Tue, 06 Nov 2007 10:39:06 -0500 X-IronPort-AV: E=Sophos;i="4.21,378,1188802800"; d="scan'208";a="185116256" Received: from sj-dkim-4.cisco.com ([171.71.179.196]) by sj-iport-5.cisco.com with ESMTP; 06 Nov 2007 07:39:04 -0800 Received: from sj-core-5.cisco.com (sj-core-5.cisco.com [171.71.177.238]) by sj-dkim-4.cisco.com (8.12.11/8.12.11) with ESMTP id lA6Fd4m5003111; Tue, 6 Nov 2007 07:39:04 -0800 Received: from xbh-rtp-201.amer.cisco.com (xbh-rtp-201.cisco.com [64.102.31.12]) by sj-core-5.cisco.com (8.12.10/8.12.6) with ESMTP id lA6Fcpvu005835; Tue, 6 Nov 2007 15:39:04 GMT Received: from xmb-rtp-203.amer.cisco.com ([64.102.31.20]) by xbh-rtp-201.amer.cisco.com with Microsoft SMTPSVC(6.0.3790.1830); Tue, 6 Nov 2007 10:38:56 -0500 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [PCN] Re: 5.5. Probing functions Date: Tue, 6 Nov 2007 10:38:55 -0500 Message-ID: In-Reply-To: <47308628.1010505@informatik.uni-wuerzburg.de> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] Re: 5.5. Probing functions Thread-Index: Acggiabo63PrGuJISkqoEAqaF1vYSQAAHYmA From: "Anna Charny (acharny)" To: X-OriginalArrivalTime: 06 Nov 2007 15:38:56.0864 (UTC) FILETIME=[24B7FE00:01C8208B] X-TM-AS-Product-Ver: SMEX-8.0.0.1181-5.000.1023-15528.002 X-TM-AS-Result: No--21.619600-8.000000-2 X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No DKIM-Signature: v=0.5; a=rsa-sha256; q=dns/txt; l=11553; t=1194363544; x=1195227544; c=relaxed/simple; s=sjdkim4002; h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version; d=cisco.com; i=acharny@cisco.com; z=From:=20=22Anna=20Charny=20(acharny)=22=20 |Subject:=20RE=3A=20[PCN]=20Re=3A=205.5.=20Probing=20functions |Sender:=20; bh=V3b1db3gKSOkcC6S5iamERNRS4EbsE2MvW0Me4shHtc=; b=pkEGjCa5qBhtWmf49twavO2KB8Msd0TMwBrlbl8K1PIjLfs11h3iMgIX/KvC28mvD8ybQr9D xYdXYkdOKriBfQSpmHVTgDNItpmnl+9PDmOL0zsaHdRNgTXrKwsexYQP; Authentication-Results: sj-dkim-4; header.From=acharny@cisco.com; dkim=pass ( sig from cisco.com/sjdkim4002 verified; ); X-Spam-Score: 0.0 (/) X-Scan-Signature: c119f9923e40f08a1d7f390ce651ea92 Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Michael,=20 > -----Original Message----- > From: Michael Menth [mailto:menth@informatik.uni-wuerzburg.de]=20 > Sent: Tuesday, November 06, 2007 10:20 AM > To: Anna Charny (acharny) > Cc: Philip Eardley; Lars Eggert; pcn@ietf.org > Subject: Re: [PCN] Re: 5.5. Probing functions >=20 > Hi Anna, Phil, >=20 > Anna, you addressed the impact of ECMP on possible imbalances=20 > of link utilization. I am with you in the sense that this is=20 > only a minor issue in networks with high traffic aggregation. >=20 Actually my point was not only that with sufficiently large aggregations levels ECMP imbalance is not an practical issue - it was also that there are realistic circumstances when ECMP imbalance may in fact be an issue (specifically, deployments with large tunnels) > However, the point I wanted to make is that different links=20 > can have different utilizations in their normal operational=20 > mode (e.g. because some of them are provisioned with backup=20 > capacity for rerouted traffic, others aren't). This effect is=20 > not caused by ECMP or load balancing in general. But the=20 > consequence of that effect is that alternative paths can have=20 > substantially different utilization which possibly becomes=20 > problematic when ECMP is used in the presence of PCN-based=20 > admission control. >=20 Would't we expect that in this case PCN thresholds will be chosen as part of traffic engineering/network planning to reflect the expected utilisation with and without "planned" failures - so setting these PCN thresholds (as well as the rest of the network planning tasks) will be chosen with the expected ECMP load-balancing in mind? So the issue of imbalances will only/typically occur for "unplanned" multiple failures or flash crowds for which this planning fails? =20 Anna=20 Anna=20 > Regards, >=20 > Michael >=20 > Anna Charny (acharny) wrote: > > Hi Phil, Michael, > > > > Based on our own internal study here are a few highlights that can=20 > > shed light on the ECMP accuracy: > > > > 1) once the number of flows (at hash granularity) becomes large=20 > > (~100,000), most hashes perform very well at a single hop=20 > for a wide=20 > > range of hash inputs. > > 2) when the number of flows in a hash is smaller (not that small=20 > > actually - on the order of 1000!) practically all hashes=20 > display large=20 > > statistical variation for some distributions of the hash=20 > inputs (where=20 > > large is on the order 20-40% or more of relative error per bucket). > > This appears to be true for a wide set of hashes, including those=20 > > shown very robust in many academic studies. > > > > The above is for single hop only, and for heterogeneous=20 > traffic rates. > > If flow rates distribution is skewed (i.e. there are a few large=20 > > flows), it can become much worse wrt actual bandwidth distribution.=20 > > Some of the known ways to fix this issue is to employ=20 > hashes that look=20 > > deeper into the packets so that large flows (e.g. based on=20 > (src, dst)=20 > > hash) get split into smaller flows (e.g. look not only at=20 > src,dst, but=20 > > also at layer 4 info) - but note that these same methods cause, for=20 > > our PCN purposes, a problem that rsvp messages may no longer follow=20 > > the same path as the data packets. > > > > So I think realistically, for truly large aggregation at the core,=20 > > ECMP imbalances are likely to be not much of an issue, but for=20 > > relatively lower levels of aggregation one might expect that fairly=20 > > large ECMP inaccuracies in some cases are quite possible. =20 > Note that=20 > > "relatively low aggregation" may occur for tunneled traffic=20 > (i.e. lots=20 > > of small flows are aggregated into a relatively small=20 > number of large tunnels). > > > > Anna > > =20 > > > > =20 > >> -----Original Message----- > >> From: philip.eardley@bt.com [mailto:philip.eardley@bt.com] > >> Sent: Tuesday, November 06, 2007 4:13 AM > >> To: menth@informatik.uni-wuerzburg.de; lars.eggert@nokia.com > >> Cc: pcn@ietf.org > >> Subject: RE: [PCN] Re: 5.5. Probing functions > >> > >> Michael > >> > >> Do you have an idea of how likely this is? will it be the=20 > case that=20 > >> for a small increase in load all/most of the ECMP paths=20 > become full?=20 > >> Does it depend strongly on topology? If each link is=20 > shared by mnay=20 > >> ecmp paths from many different ingress-egress-aggregates=20 > is this less=20 > >> of a risk? > >> > >> phil > >> > >> =20 > >>> -----Original Message----- > >>> From: Michael Menth [mailto:menth@informatik.uni-wuerzburg.de] > >>> Sent: 06 November 2007 08:29 > >>> To: Lars Eggert > >>> Cc: Eardley,PL,Philip,CXR9 R; pcn@ietf.org > >>> Subject: Re: [PCN] Re: 5.5. Probing functions > >>> > >>> Hi Lars and Phil, > >>> > >>> I think that one major motivation for probing is to make=20 > PCN capable > >>> =20 > >> to > >> =20 > >>> deal with multipath routing like ECMP. Some path are empty, > >>> =20 > >> others are > >> =20 > >>> full. Some flows must be rejected, others can proceed=20 > although they > >>> =20 > >> all > >> =20 > >>> belong to the same ingress-egress-aggregate. > >>> > >>> Regards, > >>> > >>> Michael > >>> > >>> Lars Eggert wrote: > >>> =20 > >>>> Hi, > >>>> > >>>> On 2007-11-5, at 21:30, ext philip.eardley@bt.com wrote: > >>>> =20 > >>>>> My impression is that probing advocates have 2 reasons: > >>>>> =20 > >>>> thanks for summarizing this! I think this does capture the main > >>>> =20 > >> points > >> =20 > >>>> made. > >>>> > >>>> =20 > >>>>> - the on-going ingress-egress-aggregate measurement of > >>>>> =20 > >> marked pkts > >> =20 > >>>>> doesn't work. Because there's often no traffic on this=20 > >>>>> ingress-egress-aggregate, or no traffic on the ECMP path; AND > >>>>> =20 > >> simply > >> =20 > >>>>> admitting the flow has a significant risk of leading to overload > >>>>> =20 > >> [pkts > >> =20 > >>>>> dropped &/or flows terminated] > >>>>> - there's no signalling state on the PCN-boundary-nodes=20 > for this=20 > >>>>> ingress-egress-aggregate (and no traffic already on it); AND the > >>>>> =20 > >> router > >> =20 > >>>>> behaviour is such that every pkt is marked if the flow=20 > should be=20 > >>>>> blocked; AND just admitting the call is dangerous > >>>>> =20 > >> because there's a > >> =20 > >>> good > >>> =20 > >>>>> chance of overload in some parts of the nw (near the > >>>>> =20 > >> edge where the > >> =20 > >>>>> capacity is low). Hence probing creates no extra delay=20 > (since you > >>>>> =20 > >> have > >> =20 > >>>>> to send a signalling set-up msg anyway). Often this scenario is > >>>>> =20 > >> people > >> =20 > >>>>> who envisage PCN going to end terminals or at least further out > >>>>> =20 > >> than > >> =20 > >>>>> core nw [I think]. > >>>>> =20 > >>>> Although I agree that we can construct scenarios where "simply=20 > >>>> admitting the flow has a significant risk of leading to > >>>> =20 > >> overload", I > >> =20 > >>>> still remain unclear on how likely such cases are in whatever we=20 > >>>> envision the most likely deployment scenarios to be.=20 > Basically, I=20 > >>>> worry that we're making the architecture complex for what will > >>>> =20 > >> provide > >> =20 > >>>> little real benefit in the end. > >>>> > >>>> Also, don't forget that using probing will mean dropping > >>>> =20 > >> the packets > >> =20 > >>>> of the flow waiting for admission for an RTT *every time* > >>>> =20 > >> probing is > >> =20 > >>>> used. Whereas "just admitting" may only cause overload sometimes. > >>>> There is a tradeoff here - I believe it's possible to construct a > >>>> =20 > >> case > >> =20 > >>>> where probing hurts more than "just admitting." > >>>> > >>>> And I'm still waiting for someone to convince me that > >>>> =20 > >> dropping the > >> =20 > >>>> initial packets of a flow at the ingress during the > >>>> =20 > >> probing RTT will > >> =20 > >>>> result in something that is acceptable for apps. > >>>> > >>>> =20 > >>>>> The first scenario will only happen in narrow conditions [flash > >>>>> =20 > >> crowds, > >> =20 > >>>>> many ingress-egress-aggregates without any traffic]. An > >>>>> =20 > >> alternative > >> way > >> =20 > >>>>> of handling this is to limit the rate at which admit new > >>>>> =20 > >> flows [at > >> =20 > >>> least > >>> =20 > >>>>> this makes the conditions when the problem occurs even=20 > narrower]. > >>>>> =20 > >>>> And we should try to determine how narrow these > >>>> =20 > >> conditions are. If > >> =20 > >>>> they're very narrow, we may be able to avoid them through=20 > >>>> configuration suggestions (sufficient overprovisioning, > >>>> =20 > >> sufficiently > >> =20 > >>>> broad marking regions, etc.) > >>>> > >>>> =20 > >>>>> The second scenario seems to break the (link) aggregation > >>>>> =20 > >> assumption? > >> =20 > >>>>> presumably it could be solved by running different > >>>>> =20 > >> mechanism at the > >> =20 > >>> edge > >>> =20 > >>>>> compared with at the core. [joe, sorry if you've already > >>>>> =20 > >> explained, > >> the > >> =20 > >>>>> late hour =3D> even poorer memory than usual.] > >>>>> =20 > >>>> Yes, we seem to be running into the "the number of flows > >>>> =20 > >> across any > >> =20 > >>>> potential aggregation bottleneck is sufficiently large for > >>>> =20 > >> stateless, > >> =20 > >>>> statistical mechanisms to be effective" restriction from > >>>> =20 > >> the charter > >> =20 > >>>> here. These scenarios seems to require a deterministic=20 > mechanism,=20 > >>>> while PCN is supposed to provide a probabilistic one. > >>>> > >>>> Lars > >>>> > >>>> =20 > >> -------------------------------------------------------------- > >> ---------- > >> =20 > >>>> _______________________________________________ > >>>> PCN mailing list > >>>> PCN@ietf.org > >>>> https://www1.ietf.org/mailman/listinfo/pcn > >>>> > >>>> =20 > >>> -- > >>> Dr. Michael Menth, Assistant Professor University of Wuerzburg,=20 > >>> Institute of Computer Science Am Hubland, D-97074 > >>> =20 > >> Wuerzburg, Germany, > >> =20 > >>> room B206 > >>> phone: (+49)-931/888-6644, fax: (+49)-931/888-6632=20 > >>> mailto:menth@informatik.uni-wuerzburg.de > >>> http://www3.informatik.uni-wuerzburg.de/research/ngn > >>> =20 > >> > >> _______________________________________________ > >> PCN mailing list > >> PCN@ietf.org > >> https://www1.ietf.org/mailman/listinfo/pcn > >> > >> =20 >=20 > -- > Dr. Michael Menth, Assistant Professor > University of Wuerzburg, Institute of Computer Science Am=20 > Hubland, D-97074 Wuerzburg, Germany, room B206 > phone: (+49)-931/888-6644, fax: (+49)-931/888-6632=20 > mailto:menth@informatik.uni-wuerzburg.de > http://www3.informatik.uni-wuerzburg.de/research/ngn >=20 _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Tue Nov 06 18:26:10 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpXo7-0004hH-PU; Tue, 06 Nov 2007 18:26:07 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IpXo7-0004hC-Aj for pcn-confirm+ok@megatron.ietf.org; Tue, 06 Nov 2007 18:26:07 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpXo6-0004h4-Vh for pcn@ietf.org; Tue, 06 Nov 2007 18:26:07 -0500 Received: from smtp4.smtp.bt.com ([217.32.164.151]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IpXo6-0000Ky-DM for pcn@ietf.org; Tue, 06 Nov 2007 18:26:06 -0500 Received: from i2kc06-ukbr.domain1.systemhost.net ([193.113.197.70]) by smtp4.smtp.bt.com with Microsoft SMTPSVC(6.0.3790.1830); Tue, 6 Nov 2007 23:26:05 +0000 Received: from cbibipnt05.iuser.iroot.adidom.com ([147.149.196.177]) by i2kc06-ukbr.domain1.systemhost.net with Microsoft SMTPSVC(6.0.3790.1830); Tue, 6 Nov 2007 23:26:05 +0000 Received: From bagheera.jungle.bt.co.uk ([132.146.168.158]) by cbibipnt05.iuser.iroot.adidom.com (WebShield SMTP v4.5 MR1a P0803.399); id 1194391563975; Tue, 6 Nov 2007 23:26:03 +0000 Received: from mut.jungle.bt.co.uk ([10.73.192.76]) by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id lA6NPae8013046; Tue, 6 Nov 2007 23:25:48 GMT Message-Id: <5.2.1.1.2.20071106200651.0269e130@pop3.jungle.bt.co.uk> X-Sender: rbriscoe@pop3.jungle.bt.co.uk X-Mailer: QUALCOMM Windows Eudora Version 5.2.1 Date: Tue, 06 Nov 2007 23:25:37 +0000 To: "EARDLEY, Phil" From: Bob Briscoe Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Spam-Score: -1.36 () ALL_TRUSTED X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158 X-OriginalArrivalTime: 06 Nov 2007 23:26:05.0052 (UTC) FILETIME=[66D213C0:01C820CC] X-Spam-Score: 0.0 (/) X-Scan-Signature: b5d20af10c334b36874c0264b10f59f1 Cc: PCN IETF list Subject: [PCN] draft-ietf-pcn-architecture-01.txt disambiguation suggestions X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Phil, Arch draft is looking v good now (I've just started the OAM text I promised). Thanks for making all the changes. A few review comments (none are substantial - they are either nits or ways to help readers avoid misinterpreting)... First, note the auto-generated nits off the IETF tools page: 1. Introduction > A possibility > after re-chartering is to consider that the PCN-domain encompasses > several DiffServ domains that don't trust each other Suggest change 'Diffserv domains' -> 'autonomous systems', as the term "Diffserv domain" carries extra baggage that isn't in PCN, particularly ingress traffic conditioning. There are many other occurrences of the term 'Diffserv domain' throughout, but this is the only one I would ask to be changed. However, preferably where "Diffserv domain" is cited as [RFC2475], I would like to see an explicit statement that we do NOT imply PCN needs or uses traffic conditioning like the majority of RFC2475 deployments (although we discuss a scenario where it does). A large part of RFC2475 is about traffic conditioning. So citing RFC2475 without this caveat could be misread as "Take a Diffserv domain with all its traffic conditioning, then add all the PCN stuff as well." From bitter experience, I know people reading IETF docs often make assumptions we didn't expect. 5.7. Addressing > Alternatively, if > PCN-traffic is always tunnelled across the PCN-domain, then the > PCN-ingress-node's address is simply the source address of the > outer packet header - but then the PCN-ingress-node needs to know > the address of the PCN-egress-node , using one of the many automated tunnel endpoint discovery mechanisms available (e.g. signalling or probing over the data route, interrogating routing, using a centralised broker) or by manual configuration >. {I figured that people will read this as an open issue unless we make it clear that common solutions are available.} > the centralised node > may need to know the addresses of the PCN-ingress-node and PCN- > egress-node, the PCN-egress-node may need to >know the address of the PCN- > ingress-node, and the PCN-ingress-node may need to >know the address of the > centralised node. NOTE: Consideration of the centralised case is > out of scope of the initial PCN WG Charter. 5.8. Tunnelling > PCN-marking is then > orthogonal to tunnel encapsulation /decapsulation. I think this contradicts the statement just before that decapsulation has to overwrite inner with outer if it is more severe. That requires the decapsulator to have knowledge of PCN, to know which marking is more severe. So PCN & encapsulation are orthogonal, but not PCN & decapsulation. Bob ____________________________________________________________________________ Bob Briscoe, Networks Research Centre, BT Research B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK. +44 1473 645196 _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Tue Nov 06 22:34:33 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpbgW-0008SY-7k; Tue, 06 Nov 2007 22:34:32 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IpbgV-0008Pu-MG for pcn-confirm+ok@megatron.ietf.org; Tue, 06 Nov 2007 22:34:31 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpbgV-0008Pe-BQ for pcn@ietf.org; Tue, 06 Nov 2007 22:34:31 -0500 Received: from sj-iport-6.cisco.com ([171.71.176.117]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IpbgU-0001lc-Sd for pcn@ietf.org; Tue, 06 Nov 2007 22:34:31 -0500 X-IronPort-AV: E=Sophos;i="4.21,381,1188802800"; d="scan'208";a="247702045" Received: from sj-dkim-1.cisco.com ([171.71.179.21]) by sj-iport-6.cisco.com with ESMTP; 06 Nov 2007 19:34:30 -0800 Received: from sj-core-2.cisco.com (sj-core-2.cisco.com [171.71.177.254]) by sj-dkim-1.cisco.com (8.12.11/8.12.11) with ESMTP id lA73YSTb020652; Tue, 6 Nov 2007 19:34:28 -0800 Received: from xbh-rtp-201.amer.cisco.com (xbh-rtp-201.cisco.com [64.102.31.12]) by sj-core-2.cisco.com (8.12.10/8.12.6) with ESMTP id lA73YNfd000098; Wed, 7 Nov 2007 03:34:23 GMT Received: from xmb-rtp-203.amer.cisco.com ([64.102.31.20]) by xbh-rtp-201.amer.cisco.com with Microsoft SMTPSVC(6.0.3790.1830); Tue, 6 Nov 2007 22:34:22 -0500 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Tue, 6 Nov 2007 22:34:21 -0500 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Comments on the architecture draft Thread-Index: Acgg7xV8BPMFDrf5Q4mvEJ9w7X1clw== From: "Anna Charny (acharny)" To: X-OriginalArrivalTime: 07 Nov 2007 03:34:22.0541 (UTC) FILETIME=[166A7FD0:01C820EF] X-TM-AS-Product-Ver: SMEX-8.0.0.1181-5.000.1023-15528.002 X-TM-AS-Result: No-3.224900-8.000000-31 X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No DKIM-Signature: v=0.5; a=rsa-sha256; q=dns/txt; l=2262; t=1194406468; x=1195270468; c=relaxed/simple; s=sjdkim1004; h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version; d=cisco.com; i=acharny@cisco.com; z=From:=20=22Anna=20Charny=20(acharny)=22=20 |Subject:=20Comments=20on=20the=20architecture=20draft |Sender:=20; bh=cnuN25gb+EahtouNi5G++U6rtIeV2/FMe79lHBRFsjg=; b=PxAhGdujV8AWR5S7lCLyeEGCj+/wccndUgss0gDyAOjNCJn/6wkh3sR92hdYJP5637/n9i/+ kuj8uyYRgoe8O5Jc+vUcRnhtgXVB6/YB6hekmrk/2oAU6auQ6ONcmeKcdEMR6moJaIZgiK5GaZ +5rOhRZo3FNXH9EufG6jtUHOQ=; Authentication-Results: sj-dkim-1; header.From=acharny@cisco.com; dkim=pass ( sig from cisco.com/sjdkim1004 verified; ); X-Spam-Score: 0.0 (/) X-Scan-Signature: 41c17b4b16d1eedaa8395c26e9a251c4 Cc: pcn@ietf.org Subject: [PCN] Comments on the architecture draft X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Phil, The draft is looking really good. Here are a few comments/questions though: Page 5: >Depending on the deployment scenario, the decision-making > functionality (about flow admission and termination) could reside > at the PCN-ingress-nodes or PCN-egress-nodes or at some central > control node in the PCN-domain. NOTE: The Charter restricts us: > the decision-making functionality is at the PCN-boundary-nodes Does it mean that we should first define the specific deployment scenarios of interest and then define the boundary-node functionality, or the other way around?=20 Page 6: >The PCN-domain extends to the end users. NOTE: This is outside >the Charter because it breaks Assumption 3 (aggregation, see >later; .... This does not necessarily break the aggregation assumption because the aggregation assumption is at the *bottleneck*. Current charter does not preclude this at the moment (although ongoing discussion on the list questions whether explicit assumptions on ingress-egress aggregation should be made) Page 9: (trust discussion) >Similarly, a PCN-boundary-node has to trust that all the PCN-nodes >are doing PCN-marking.=20 Do we really need to require that *all* nodes do marking? If some node does not mark, then only its adjacent links are not handled. Is the above requirement too strict? (We do need to say there are no rogue ones) Page 12: >the PCN-egress-node measures (possibly as a moving average) the >fraction of the PCN-traffic that is PCN-marked. Note that current 3sm proposal (at least as simulated) does not compute CLE but acts on receipt of one or several marked packets to make an admission-stop decision. Page 18: >A probe packet is just a dummy data packet, generated by >the PCN-ingress-node and addressed to the PCN-egress-node This seems to exclude using RSVP messages as probes Page 23: >ECMP and signaling: It is possible that, in a PCN-domain running >ECMP, the signaling packets (eg RSVP, NSIS) follow a different >path than the data packets. This depends on which fields the ECMP >algorithm uses. The above seems of no consequence unless signaling messages are used as probes - right? Hope this is helpful, Anna=20 _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Tue Nov 06 22:41:10 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Ipbmw-00062N-Kk; Tue, 06 Nov 2007 22:41:10 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Ipbmv-00061W-Ld for pcn-confirm+ok@megatron.ietf.org; Tue, 06 Nov 2007 22:41:09 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Ipbmv-00061O-Bh for pcn@ietf.org; Tue, 06 Nov 2007 22:41:09 -0500 Received: from newdev.eecs.harvard.edu ([140.247.60.212]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1Ipbms-0005lZ-59 for pcn@ietf.org; Tue, 06 Nov 2007 22:41:09 -0500 Received: by newdev.eecs.harvard.edu (Postfix, from userid 501) id 610D261B673; Tue, 6 Nov 2007 22:38:18 -0500 (EST) To: pcn@ietf.org Message-Id: <20071107033818.610D261B673@newdev.eecs.harvard.edu> Date: Tue, 6 Nov 2007 22:38:18 -0500 (EST) From: sob@harvard.edu (Scott O. Bradner) X-Spam-Score: 0.0 (/) X-Scan-Signature: 7bac9cb154eb5790ae3b2913587a40de Subject: [PCN] 1st call - agenda requests for Vancouver X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org this is a first call for timeslots on the PCN agenda in Vancouver we are currently scheduled for wed afternoon from 1300-1610 we expect that a good chunk of the time will be spent on discussions of the Architecture draft but if others have other (in-charter) topics please let us know Scott & Steve _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Tue Nov 06 22:46:22 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Ipbry-00037l-1V; Tue, 06 Nov 2007 22:46:22 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Ipbrw-000378-Sp for pcn-confirm+ok@megatron.ietf.org; Tue, 06 Nov 2007 22:46:20 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Ipbrw-00036u-Iz for pcn@ietf.org; Tue, 06 Nov 2007 22:46:20 -0500 Received: from sj-iport-6.cisco.com ([171.71.176.117]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1Ipbrt-0005uv-7O for pcn@ietf.org; Tue, 06 Nov 2007 22:46:20 -0500 X-IronPort-AV: E=Sophos;i="4.21,381,1188802800"; d="scan'208";a="247705290" Received: from sj-dkim-1.cisco.com ([171.71.179.21]) by sj-iport-6.cisco.com with ESMTP; 06 Nov 2007 19:46:16 -0800 Received: from sj-core-5.cisco.com (sj-core-5.cisco.com [171.71.177.238]) by sj-dkim-1.cisco.com (8.12.11/8.12.11) with ESMTP id lA73kGDF029861; Tue, 6 Nov 2007 19:46:16 -0800 Received: from xbh-rtp-211.amer.cisco.com (xbh-rtp-211.cisco.com [64.102.31.102]) by sj-core-5.cisco.com (8.12.10/8.12.6) with ESMTP id lA73kCQ8000305; Wed, 7 Nov 2007 03:46:16 GMT Received: from xmb-rtp-203.amer.cisco.com ([64.102.31.20]) by xbh-rtp-211.amer.cisco.com with Microsoft SMTPSVC(6.0.3790.1830); Tue, 6 Nov 2007 22:46:14 -0500 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [PCN] 1st call - agenda requests for Vancouver Date: Tue, 6 Nov 2007 22:46:13 -0500 Message-ID: In-Reply-To: <20071107033818.610D261B673@newdev.eecs.harvard.edu> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] 1st call - agenda requests for Vancouver Thread-Index: Acgg8A53TsUM4+zhQkOHI3ejE1kldgAAEyuQ From: "Anna Charny (acharny)" To: "Scott O. Bradner" , X-OriginalArrivalTime: 07 Nov 2007 03:46:14.0772 (UTC) FILETIME=[BEF05340:01C820F0] X-TM-AS-Product-Ver: SMEX-8.0.0.1181-5.000.1023-15528.002 X-TM-AS-Result: No--20.466400-8.000000-2 X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No DKIM-Signature: v=0.5; a=rsa-sha256; q=dns/txt; l=907; t=1194407176; x=1195271176; c=relaxed/simple; s=sjdkim1004; h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version; d=cisco.com; i=acharny@cisco.com; z=From:=20=22Anna=20Charny=20(acharny)=22=20 |Subject:=20RE=3A=20[PCN]=201st=20call=20-=20agenda=20requests=20for=20Va ncouver |Sender:=20; bh=FzLuvORlkBPcKis0/VaZfOoo6rdImE+uQwyFAprG4Aw=; b=JtLdmrDZx5wI5+utifZEOlBWFYJIkzO8hArARG2vSaprkTHtYYAVU3hk8KbT5Ofs3HcnaR0F awkmNL2dbmu9G09s4/2+SjP11xtioNrId4+28rq4nBX3hajyypP598rb6yDnHhMthlGZgmVk9M 6r0Y7vAFzEBz8wYQZrcHT+BUI=; Authentication-Results: sj-dkim-1; header.From=acharny@cisco.com; dkim=pass ( sig from cisco.com/sjdkim1004 verified; ); X-Spam-Score: -4.0 (----) X-Scan-Signature: b19722fc8d3865b147c75ae2495625f2 Cc: X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hello Scott, I would like to request=20 1) a slot for comparison draft we are finally about to send out=20 2) a slot for an update on the single marking draft=20 Thank you, Anna=20 > -----Original Message----- > From: Scott O. Bradner [mailto:sob@harvard.edu]=20 > Sent: Tuesday, November 06, 2007 10:38 PM > To: pcn@ietf.org > Subject: [PCN] 1st call - agenda requests for Vancouver >=20 >=20 > this is a first call for timeslots on the PCN agenda in Vancouver >=20 > we are currently scheduled for wed afternoon from 1300-1610 >=20 > we expect that a good chunk of the time will be spent on=20 > discussions of the Architecture draft but if others have=20 > other (in-charter) topics please let us know >=20 > Scott & Steve >=20 >=20 > _______________________________________________ > PCN mailing list > PCN@ietf.org > https://www1.ietf.org/mailman/listinfo/pcn >=20 _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Wed Nov 07 04:50:18 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IphY9-0004lc-EH; Wed, 07 Nov 2007 04:50:17 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IphY7-0004kA-69 for pcn-confirm+ok@megatron.ietf.org; Wed, 07 Nov 2007 04:50:16 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IphY1-0004h2-Qz for pcn@ietf.org; Wed, 07 Nov 2007 04:50:09 -0500 Received: from szxga04-in.huawei.com ([61.144.161.7]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IphXx-0000rj-As for pcn@ietf.org; Wed, 07 Nov 2007 04:50:09 -0500 Received: from huawei.com (szxga04-in [172.24.2.12]) by szxga04-in.huawei.com (iPlanet Messaging Server 5.2 HotFix 2.14 (built Aug 8 2006)) with ESMTP id <0JR400620RB1L3@szxga04-in.huawei.com> for pcn@ietf.org; Wed, 07 Nov 2007 17:49:49 +0800 (CST) Received: from huawei.com ([172.24.1.24]) by szxga04-in.huawei.com (iPlanet Messaging Server 5.2 HotFix 2.14 (built Aug 8 2006)) with ESMTP id <0JR400526RB0JH@szxga04-in.huawei.com> for pcn@ietf.org; Wed, 07 Nov 2007 17:49:49 +0800 (CST) Received: from z24109b ([10.70.76.84]) by szxml04-in.huawei.com (iPlanet Messaging Server 5.2 HotFix 2.14 (built Aug 8 2006)) with ESMTPA id <0JR400ITQRB0FF@szxml04-in.huawei.com> for pcn@ietf.org; Wed, 07 Nov 2007 17:49:48 +0800 (CST) Date: Wed, 07 Nov 2007 17:49:48 +0800 From: Tina TSOU Subject: Re: [PCN] 1st call - agenda requests for Vancouver To: pcn@ietf.org, "Scott O. Bradner" , "Anna Charny (acharny)" Message-id: <02ad01c82123$88f15b30$544c460a@china.huawei.com> MIME-version: 1.0 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2900.3138 X-Mailer: Microsoft Outlook Express 6.00.2900.3138 X-Priority: 3 X-MSMail-priority: Normal References: X-Spam-Score: 0.0 (/) X-Scan-Signature: 37af5f8fbf6f013c5b771388e24b09e7 Cc: X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0341224265==" Errors-To: pcn-bounces@ietf.org This is a multi-part message in MIME format. --===============0341224265== Content-type: multipart/alternative; boundary="Boundary_(ID_Z6oWlEdaqI2zbMFwQfWd5A)" This is a multi-part message in MIME format. --Boundary_(ID_Z6oWlEdaqI2zbMFwQfWd5A) Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: 7BIT Hi Scott, We would like to request 3) a slot for PCN Boundary Node Behaviour Thanks a lot:) B. R. Tina ----- Original Message ----- From: Anna Charny (acharny) To: Scott O. Bradner ; pcn@ietf.org Sent: Wednesday, November 07, 2007 11:46 AM Subject: RE: [PCN] 1st call - agenda requests for Vancouver Hello Scott, I would like to request 1) a slot for comparison draft we are finally about to send out 2) a slot for an update on the single marking draft Thank you, Anna > -----Original Message----- > From: Scott O. Bradner [mailto:sob@harvard.edu] > Sent: Tuesday, November 06, 2007 10:38 PM > To: pcn@ietf.org > Subject: [PCN] 1st call - agenda requests for Vancouver > > > this is a first call for timeslots on the PCN agenda in Vancouver > > we are currently scheduled for wed afternoon from 1300-1610 > > we expect that a good chunk of the time will be spent on > discussions of the Architecture draft but if others have > other (in-charter) topics please let us know > > Scott & Steve > > > _______________________________________________ > PCN mailing list > PCN@ietf.org > https://www1.ietf.org/mailman/listinfo/pcn > _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn --Boundary_(ID_Z6oWlEdaqI2zbMFwQfWd5A) Content-type: text/html; charset=ISO-8859-1 Content-transfer-encoding: 7BIT
Hi Scott,
We would like to request
 
3) a slot for PCN Boundary Node Behaviour
 
Thanks a lot:)
 
B. R.
Tina
----- Original Message -----
Sent: Wednesday, November 07, 2007 11:46 AM
Subject: RE: [PCN] 1st call - agenda requests for Vancouver

Hello Scott,

I would like to request

1) a slot for comparison draft we are finally about to send out
2) a slot for an update on the single marking draft

Thank you,
Anna

> -----Original Message-----
> From: Scott O. Bradner [mailto:sob@harvard.edu]
> Sent: Tuesday, November 06, 2007 10:38 PM
> To: pcn@ietf.org
> Subject: [PCN] 1st call - agenda requests for Vancouver
>
>
> this is a first call for timeslots on the PCN agenda in Vancouver
>
> we are currently scheduled for wed afternoon from 1300-1610
>
> we expect that a good chunk of the time will be spent on
> discussions of the Architecture draft but if others have
> other (in-charter) topics please let us know
>
> Scott & Steve
>
>
> _______________________________________________
> PCN mailing list
> PCN@ietf.org
> https://www1.ietf.org/mailman/listinfo/pcn
>


_______________________________________________
PCN mailing list
PCN@ietf.org
https://www1.ietf.org/mailman/listinfo/pcn
--Boundary_(ID_Z6oWlEdaqI2zbMFwQfWd5A)-- --===============0341224265== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn --===============0341224265==-- From pcn-bounces@ietf.org Wed Nov 07 06:12:21 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpipZ-000650-9r; Wed, 07 Nov 2007 06:12:21 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IpipX-00063F-J9 for pcn-confirm+ok@megatron.ietf.org; Wed, 07 Nov 2007 06:12:19 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpipX-00061i-0J for pcn@ietf.org; Wed, 07 Nov 2007 06:12:19 -0500 Received: from smtp4.smtp.bt.com ([217.32.164.151]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IpipW-0001P7-CC for pcn@ietf.org; Wed, 07 Nov 2007 06:12:18 -0500 Received: from E03MVZ1-UKDY.domain1.systemhost.net ([193.113.30.65]) by smtp4.smtp.bt.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 7 Nov 2007 11:12:17 +0000 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Wed, 7 Nov 2007 11:10:59 -0000 Message-ID: <75A199C5D243C741BF3D3F1EBCEF9BA503B342F3@E03MVZ1-UKDY.domain1.systemhost.net> In-Reply-To: <5.2.1.1.2.20071106200651.0269e130@pop3.jungle.bt.co.uk> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: draft-ietf-pcn-architecture-01.txt disambiguation suggestions Thread-Index: AcggzGaRxsTCpKs9StKAVkabh140CAAYmPLw From: To: X-OriginalArrivalTime: 07 Nov 2007 11:12:17.0330 (UTC) FILETIME=[0EAA9920:01C8212F] X-Spam-Score: 0.0 (/) X-Scan-Signature: b1c41982e167b872076d0018e4e1dc3c Cc: pcn@ietf.org Subject: [PCN] RE: draft-ietf-pcn-architecture-01.txt disambiguation suggestions X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Thanks bob, will add these into the next version phil > -----Original Message----- > From: Briscoe,RJ,Bob,XVR9 BRISCORJ R > Sent: 06 November 2007 23:26 > To: Eardley,PL,Philip,CXR9 R > Cc: PCN IETF list > Subject: draft-ietf-pcn-architecture-01.txt disambiguation suggestions >=20 > Phil, >=20 > Arch draft is looking v good now (I've just started the OAM text I > promised). Thanks for making all the changes. >=20 > A few review comments (none are substantial - they are either nits or ways > to help readers avoid misinterpreting)... >=20 > First, note the auto-generated nits off the IETF tools page: > architecture-01.nits.txt> >=20 >=20 > 1. Introduction >=20 >=20 > > A possibility > > after re-chartering is to consider that the PCN-domain encompasses > > several DiffServ domains that don't trust each other >=20 > Suggest change 'Diffserv domains' -> 'autonomous systems', as the term > "Diffserv domain" carries extra baggage that isn't in PCN, particularly > ingress traffic conditioning. >=20 > There are many other occurrences of the term 'Diffserv domain' throughout, > but this is the only one I would ask to be changed. >=20 > However, preferably where "Diffserv domain" is cited as [RFC2475], I would > like to see an explicit statement that we do NOT imply PCN needs or uses > traffic conditioning like the majority of RFC2475 deployments (although we > discuss a scenario where it does). >=20 > A large part of RFC2475 is about traffic conditioning. So citing RFC2475 > without this caveat could be misread as "Take a Diffserv domain with all > its traffic conditioning, then add all the PCN stuff as well." From bitter > experience, I know people reading IETF docs often make assumptions we > didn't expect. >=20 >=20 > 5.7. Addressing >=20 >=20 > > Alternatively, if > > PCN-traffic is always tunnelled across the PCN-domain, then the > > PCN-ingress-node's address is simply the source address of the > > outer packet header - but then the PCN-ingress-node needs to know > > the address of the PCN-egress-node >=20 > , using one of the many automated tunnel endpoint discovery mechanisms > available (e.g. signalling or probing over the data route, interrogating > routing, using a centralised broker) or by manual configuration >=20 > >. >=20 > {I figured that people will read this as an open issue unless we make it > clear that common solutions are available.} >=20 >=20 >=20 > > the centralised node > > may need to know the addresses of the PCN-ingress-node and PCN- > > egress-node, the PCN-egress-node >=20 > may need to >=20 > >know the address of the PCN- > > ingress-node, and the PCN-ingress-node >=20 > may need to >=20 > >know the address of the > > centralised node. NOTE: Consideration of the centralised case is > > out of scope of the initial PCN WG Charter. >=20 > 5.8. Tunnelling >=20 >=20 > > PCN-marking is then > > orthogonal to tunnel encapsulation /decapsulation. >=20 > I think this contradicts the statement just before that decapsulation has > to overwrite inner with outer if it is more severe. That requires the > decapsulator to have knowledge of PCN, to know which marking is more > severe. So PCN & encapsulation are orthogonal, but not PCN & > decapsulation. >=20 >=20 > Bob >=20 >=20 > ________________________________________________________________________ __ > __ > Bob Briscoe, Networks Research Centre, BT > Research > B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK. +44 1473 > 645196 >=20 _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Wed Nov 07 06:33:53 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpjAP-0003Ix-Cl; Wed, 07 Nov 2007 06:33:53 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IpjAP-0003Is-2D for pcn-confirm+ok@megatron.ietf.org; Wed, 07 Nov 2007 06:33:53 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpjAO-0003I3-NP for pcn@ietf.org; Wed, 07 Nov 2007 06:33:52 -0500 Received: from smtp4.smtp.bt.com ([217.32.164.151]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IpjAK-0003xo-U6 for pcn@ietf.org; Wed, 07 Nov 2007 06:33:52 -0500 Received: from E03MVZ1-UKDY.domain1.systemhost.net ([193.113.30.65]) by smtp4.smtp.bt.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 7 Nov 2007 11:33:46 +0000 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Wed, 7 Nov 2007 11:32:45 -0000 Message-ID: <75A199C5D243C741BF3D3F1EBCEF9BA503B342F4@E03MVZ1-UKDY.domain1.systemhost.net> In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Comments on the architecture draft Thread-Index: Acgg7xV8BPMFDrf5Q4mvEJ9w7X1clwAQD2sQ From: To: X-OriginalArrivalTime: 07 Nov 2007 11:33:46.0497 (UTC) FILETIME=[0F11DF10:01C82132] X-Spam-Score: -1.0 (-) X-Scan-Signature: 5ebbf074524e58e662bc8209a6235027 Cc: pcn@ietf.org Subject: [PCN] RE: Comments on the architecture draft X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Thanks anna, In-line phil > -----Original Message----- > From: Anna Charny (acharny) [mailto:acharny@cisco.com] > Sent: 07 November 2007 03:34 > To: Eardley,PL,Philip,CXR9 R > Cc: pcn@ietf.org > Subject: Comments on the architecture draft >=20 > Hi Phil, >=20 > The draft is looking really good. > Here are a few comments/questions though: >=20 > Page 5: >=20 > >Depending on the deployment scenario, the decision-making > > functionality (about flow admission and termination) could reside > > at the PCN-ingress-nodes or PCN-egress-nodes or at some central > > control node in the PCN-domain. NOTE: The Charter restricts us: > > the decision-making functionality is at the PCN-boundary-nodes >=20 > Does it mean that we should first define the specific deployment > scenarios of interest and then define the boundary-node functionality, > or the other way around? [phil] I think personally I tend to think about it more along your "first" lines, though I don't think it matters if others think on the "second" lines. (is this a cop out?!) The para was just supposed to say that we'll only define decision-making at boundary nodes and only consider such scenarios.=20 >=20 > Page 6: >=20 > >The PCN-domain extends to the end users. NOTE: This is outside > >the Charter because it breaks Assumption 3 (aggregation, see > >later; .... >=20 > This does not necessarily break the aggregation assumption because the > aggregation assumption is at the *bottleneck*. Current charter does not > preclude this at the moment (although ongoing discussion on the list > questions whether explicit assumptions on ingress-egress aggregation > should be made) ok, will re-phrase >=20 >=20 > Page 9: >=20 > (trust discussion) > >Similarly, a PCN-boundary-node has to trust that all the PCN-nodes > >are doing PCN-marking. >=20 > Do we really need to require that *all* nodes do marking? If some node > does not mark, then only its adjacent links are not handled. Is the > above requirement too strict? (We do need to say there are no rogue > ones) ok, I could add a caveat that if it's known that an interface cannot become pre-congested then PCN-marking on it will never be done & so the functionality isn't strictly necessary. Eg it's possibly to design by topology to ensure this - however, note the danger: if the topology changes (upgrade network; some links have failed elsewhere in PCN-domain) then it can no longer be guaranteed. =20 >=20 > Page 12: >=20 > >the PCN-egress-node measures (possibly as a moving average) the > >fraction of the PCN-traffic that is PCN-marked. >=20 > Note that current 3sm proposal (at least as simulated) does not compute > CLE but acts on receipt of one or several marked packets to make an > admission-stop decision. Arrgh. I'll try and re-word.=20 >=20 > Page 18: >=20 > >A probe packet is just a dummy data packet, generated by > >the PCN-ingress-node and addressed to the PCN-egress-node >=20 > This seems to exclude using RSVP messages as probes will add. >=20 > Page 23: >=20 > >ECMP and signaling: It is possible that, in a PCN-domain running > >ECMP, the signaling packets (eg RSVP, NSIS) follow a different > >path than the data packets. This depends on which fields the ECMP > >algorithm uses. >=20 > The above seems of no consequence unless signaling messages are used as > probes - right? think this is true. >=20 > Hope this is helpful, yes, thanks! > Anna _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Wed Nov 07 06:34:45 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpjBF-0003u2-FL; Wed, 07 Nov 2007 06:34:45 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IpjBE-0003ts-DE for pcn-confirm+ok@megatron.ietf.org; Wed, 07 Nov 2007 06:34:44 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpjBE-0003tX-12 for pcn@ietf.org; Wed, 07 Nov 2007 06:34:44 -0500 Received: from wrzx28.rz.uni-wuerzburg.de ([132.187.3.28] helo=mailrelay.rz.uni-wuerzburg.de) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IpjBD-0002DQ-Lb for pcn@ietf.org; Wed, 07 Nov 2007 06:34:43 -0500 Received: from virusscan.mail (localhost [127.0.0.1]) by mailrelay.mail (Postfix) with ESMTP id A6224BFE2 for ; Wed, 7 Nov 2007 12:34:42 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by virusscan.mail (Postfix) with ESMTP id 982CBF570 for ; Wed, 7 Nov 2007 12:34:42 +0100 (CET) Received: from europa.informatik.uni-wuerzburg.de (wicx01.informatik.uni-wuerzburg.de [132.187.11.1]) by mailmaster.uni-wuerzburg.de (Postfix) with ESMTP id 7B845F559 for ; Wed, 7 Nov 2007 12:34:42 +0100 (CET) Received: from nero.informatik.uni-wuerzburg.de (win3005.informatik.uni-wuerzburg.de [132.187.106.5]) by europa.informatik.uni-wuerzburg.de (8.11.3/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id lA7BYgh21789 for ; Wed, 7 Nov 2007 12:34:42 +0100 Received: from [127.0.0.1] (nero.informatik.uni-wuerzburg.de [132.187.106.5]) by nero.informatik.uni-wuerzburg.de (Postfix) with ESMTP id 22E466F591 for ; Wed, 7 Nov 2007 12:27:13 +0100 (CET) Message-ID: <4731A0F3.20603@informatik.uni-wuerzburg.de> Date: Wed, 07 Nov 2007 12:26:43 +0100 From: Michael Menth Organization: University of Wuerzburg User-Agent: Thunderbird 2.0.0.6 (Windows/20070728) MIME-Version: 1.0 To: pcn@ietf.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at uni-wuerzburg.de X-Spam-Score: 0.0 (/) X-Scan-Signature: 7655788c23eb79e336f5f8ba8bce7906 Subject: [PCN] Question on RSVP state information X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: menth@informatik.uni-wuerzburg.de List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi, I've got a question for RSVP experts. The RSVP PATH state stores the previous RSVP-capable hop of the future data flow. This information is needed that RESV messages can establish the reservation on the correct path. My question: Is the next RSVP-capable hop of the future data communication also available in the RESV or PATH state of the reservation? At first spot, I don't see why RSVP would need it. Regards, Michael -- Dr. Michael Menth, Assistant Professor University of Wuerzburg, Institute of Computer Science Am Hubland, D-97074 Wuerzburg, Germany, room B206 phone: (+49)-931/888-6644, fax: (+49)-931/888-6632 mailto:menth@informatik.uni-wuerzburg.de http://www3.informatik.uni-wuerzburg.de/research/ngn _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Wed Nov 07 08:02:47 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpkYN-0003oY-5x; Wed, 07 Nov 2007 08:02:43 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IpkYK-0003lN-KF for pcn-confirm+ok@megatron.ietf.org; Wed, 07 Nov 2007 08:02:40 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpkYJ-0003kp-UF for pcn@ietf.org; Wed, 07 Nov 2007 08:02:39 -0500 Received: from ams-iport-1.cisco.com ([144.254.224.140]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IpkYF-000765-Mm for pcn@ietf.org; Wed, 07 Nov 2007 08:02:39 -0500 X-IronPort-AV: E=Sophos;i="4.21,384,1188770400"; d="scan'208";a="157063763" Received: from ams-dkim-1.cisco.com ([144.254.224.138]) by ams-iport-1.cisco.com with ESMTP; 07 Nov 2007 14:02:35 +0100 Received: from ams-core-1.cisco.com (ams-core-1.cisco.com [144.254.224.150]) by ams-dkim-1.cisco.com (8.12.11/8.12.11) with ESMTP id lA7D2Y9T025072; Wed, 7 Nov 2007 14:02:34 +0100 Received: from xbh-ams-331.emea.cisco.com (xbh-ams-331.cisco.com [144.254.231.71]) by ams-core-1.cisco.com (8.12.10/8.12.6) with ESMTP id lA7D2S8i006933; Wed, 7 Nov 2007 13:02:30 GMT Received: from xfe-ams-331.emea.cisco.com ([144.254.231.72]) by xbh-ams-331.emea.cisco.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 7 Nov 2007 14:02:28 +0100 Received: from [144.254.53.188] ([144.254.53.188]) by xfe-ams-331.emea.cisco.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 7 Nov 2007 14:02:28 +0100 In-Reply-To: <4731A0F3.20603@informatik.uni-wuerzburg.de> References: <4731A0F3.20603@informatik.uni-wuerzburg.de> Mime-Version: 1.0 (Apple Message framework v752.3) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <24C402B4-411C-4D60-84EA-56737B4EA072@cisco.com> Content-Transfer-Encoding: 7bit From: Francois Le Faucheur IMAP Subject: Re: [PCN] Question on RSVP state information Date: Wed, 7 Nov 2007 14:02:27 +0100 To: menth@informatik.uni-wuerzburg.de X-Mailer: Apple Mail (2.752.3) X-OriginalArrivalTime: 07 Nov 2007 13:02:28.0415 (UTC) FILETIME=[732E1CF0:01C8213E] X-TM-AS-Product-Ver: SMEX-8.0.0.1181-5.000.1023-15530.001 X-TM-AS-Result: No--16.506100-8.000000-31 X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No DKIM-Signature: v=0.5; a=rsa-sha256; q=dns/txt; l=1354; t=1194440554; x=1195304554; c=relaxed/simple; s=amsdkim1002; h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version; d=cisco.com; i=flefauch@cisco.com; z=From:=20Francois=20Le=20Faucheur=20IMAP=20 |Subject:=20Re=3A=20[PCN]=20Question=20on=20RSVP=20state=20information |Sender:=20; bh=DbXEl0N7gnLZKH6NJ8uO/hjkTyNhDyQoN+AmLV5fZjo=; b=wm/Y0XyQvijwlHvLrspIosEUw9jiSLNJyi0Abenzvn8oFgCrARJMmZSoKEKjKNscmkaggRSE P0GeD19hx8JCNs3u3WuvO5aZXuSPIr/AqbUa4T3CcGL+RYytFABM0amv; Authentication-Results: ams-dkim-1; header.From=flefauch@cisco.com; dkim=pass ( sig from cisco.com/amsdkim1002 verified; ); X-Spam-Score: -4.0 (----) X-Scan-Signature: a7d6aff76b15f3f56fcb94490e1052e4 Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Michael, On 7 Nov 2007, at 12:26, Michael Menth wrote: > Hi, > > I've got a question for RSVP experts. The RSVP PATH state stores > the previous RSVP-capable hop of the future data flow. This > information is needed that RESV messages can establish the > reservation on the correct path. > > My question: Is the next RSVP-capable hop of the future data > communication also available in the RESV or PATH state of the > reservation? At first spot, I don't see why RSVP would need it. The Resv message contains the address of the RSVP router that issued the Resv message. It is kept as part of the Resv state by the upstream RSVP router. On this upstream RSVP router, this effectively identifies the current "RSVP Next Hop". This is useful for a number of features (like RSVP errors, Refresh Reduction,..). Francois > > Regards, > > Michael > > -- > Dr. Michael Menth, Assistant Professor > University of Wuerzburg, Institute of Computer Science > Am Hubland, D-97074 Wuerzburg, Germany, room B206 > phone: (+49)-931/888-6644, fax: (+49)-931/888-6632 > mailto:menth@informatik.uni-wuerzburg.de > http://www3.informatik.uni-wuerzburg.de/research/ngn > > > > _______________________________________________ > PCN mailing list > PCN@ietf.org > https://www1.ietf.org/mailman/listinfo/pcn _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Wed Nov 07 12:19:14 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpoYc-0007JY-Ol; Wed, 07 Nov 2007 12:19:14 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IpoYb-0007J9-7x for pcn-confirm+ok@megatron.ietf.org; Wed, 07 Nov 2007 12:19:13 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IpoYa-0007Ip-S4; Wed, 07 Nov 2007 12:19:12 -0500 Received: from rtp-iport-2.cisco.com ([64.102.122.149]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IpoYR-0000bo-Se; Wed, 07 Nov 2007 12:19:12 -0500 X-IronPort-AV: E=Sophos;i="4.21,385,1188792000"; d="txt'?scan'208";a="136263530" Received: from rtp-dkim-2.cisco.com ([64.102.121.159]) by rtp-iport-2.cisco.com with ESMTP; 07 Nov 2007 12:19:03 -0500 Received: from rtp-core-1.cisco.com (rtp-core-1.cisco.com [64.102.124.12]) by rtp-dkim-2.cisco.com (8.12.11/8.12.11) with ESMTP id lA7HJ3co007717; Wed, 7 Nov 2007 12:19:03 -0500 Received: from xbh-rtp-201.amer.cisco.com (xbh-rtp-201.cisco.com [64.102.31.12]) by rtp-core-1.cisco.com (8.12.10/8.12.6) with ESMTP id lA7HIrWT006859; Wed, 7 Nov 2007 17:19:03 GMT Received: from xmb-rtp-203.amer.cisco.com ([64.102.31.20]) by xbh-rtp-201.amer.cisco.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 7 Nov 2007 12:18:58 -0500 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----_=_NextPart_001_01C82162.48452C1D" Date: Wed, 7 Nov 2007 12:18:57 -0500 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: PCN Comparison draft submission Thread-Index: AcghYkeE/OGLHP1zSrCgsJFlV+Xf1w== From: "Anna Charny (acharny)" To: X-OriginalArrivalTime: 07 Nov 2007 17:18:58.0663 (UTC) FILETIME=[487B9770:01C82162] X-TM-AS-Product-Ver: SMEX-8.0.0.1181-5.000.1023-15530.002 X-TM-AS-Result: No--11.442300-8.000000-31 X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No DKIM-Signature: v=0.5; a=rsa-sha256; q=dns/txt; l=115528; t=1194455943; x=1195319943; c=relaxed/simple; s=rtpdkim2001; h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version; d=cisco.com; i=acharny@cisco.com; z=From:=20=22Anna=20Charny=20(acharny)=22=20 |Subject:=20PCN=20Comparison=20draft=20submission |Sender:=20 |To:=20; bh=hsc7CsR+MFj24W4U6nwjZHvDfc6XsVdn1vgkHjoZUes=; b=nNfEqNwKMZOOF5CpkwI7R48+AtFfsWyyMR8DwzT5LLJQseCUIFqw31LyITcsUukPXd3MfCSI DTmOpIKCwKWCZet9rVAX5/+FZtfvNwReZjJcsjYIzEkoVjXie+UtiM8m; Authentication-Results: rtp-dkim-2; header.From=acharny@cisco.com; dkim=pass ( sig from cisco.com/rtpdkim2001 verified; ); X-Spam-Score: -4.0 (----) X-Scan-Signature: 66f400f61005d4d0a97f265bc4a7a9ca Cc: pcn@ietf.org Subject: [PCN] PCN Comparison draft submission X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org This is a multi-part message in MIME format. ------_=_NextPart_001_01C82162.48452C1D Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi, Please post the attached draft-charny-pcn-comparison-00. Thank you, Anna Charny ------_=_NextPart_001_01C82162.48452C1D Content-Type: text/plain; name="draft-charny-pcn-comparison-00.txt" Content-Transfer-Encoding: base64 Content-Description: draft-charny-pcn-comparison-00.txt Content-Disposition: attachment; filename="draft-charny-pcn-comparison-00.txt" DQoNCk5ldHdvcmsgV29ya2luZyBHcm91cCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgIEEuIENoYXJueQ0KSW50ZXJuZXQtRHJhZnQgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICBDaXNjbyBTeXN0ZW1zLCBJbmMuDQpJbnRlbmRlZCBzdGF0dXM6IElu Zm9ybWF0aW9uYWwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIEouIEJhYmlhcnoNCkV4 cGlyZXM6IE1heSAxNCwgMjAwOCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgIE5vcnRlbA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgIE0uIE1lbnRoDQogICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgVW5pdmVyc2l0eSBvZiBXdWVyemJ1cmcNCiAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBK LiBaaGFuZw0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBDaXNj byBTeXN0ZW1zLCBJbmMgJiBDb3JuZWxsDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIFVuaXZlcnNpdHkNCiAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBOb3ZlbWJlciAxMSwgMjAw Nw0KDQoNCiAgICAgICAgICAgICAgICAgQ29tcGFyaXNvbiBvZiBQcm9wb3NlZCBQQ04gQXBwcm9h Y2hlcw0KICAgICAgICAgICAgICAgICAgIGRyYWZ0LWNoYXJueS1wY24tY29tcGFyaXNvbi0wMC50 eHQNCg0KU3RhdHVzIG9mIHRoaXMgTWVtbw0KDQogICBCeSBzdWJtaXR0aW5nIHRoaXMgSW50ZXJu ZXQtRHJhZnQsIGVhY2ggYXV0aG9yIHJlcHJlc2VudHMgdGhhdCBhbnkNCiAgIGFwcGxpY2FibGUg cGF0ZW50IG9yIG90aGVyIElQUiBjbGFpbXMgb2Ygd2hpY2ggaGUgb3Igc2hlIGlzIGF3YXJlDQog ICBoYXZlIGJlZW4gb3Igd2lsbCBiZSBkaXNjbG9zZWQsIGFuZCBhbnkgb2Ygd2hpY2ggaGUgb3Ig c2hlIGJlY29tZXMNCiAgIGF3YXJlIHdpbGwgYmUgZGlzY2xvc2VkLCBpbiBhY2NvcmRhbmNlIHdp dGggU2VjdGlvbiA2IG9mIEJDUCA3OS4NCg0KICAgSW50ZXJuZXQtRHJhZnRzIGFyZSB3b3JraW5n IGRvY3VtZW50cyBvZiB0aGUgSW50ZXJuZXQgRW5naW5lZXJpbmcNCiAgIFRhc2sgRm9yY2UgKElF VEYpLCBpdHMgYXJlYXMsIGFuZCBpdHMgd29ya2luZyBncm91cHMuICBOb3RlIHRoYXQNCiAgIG90 aGVyIGdyb3VwcyBtYXkgYWxzbyBkaXN0cmlidXRlIHdvcmtpbmcgZG9jdW1lbnRzIGFzIEludGVy bmV0LQ0KICAgRHJhZnRzLg0KDQogICBJbnRlcm5ldC1EcmFmdHMgYXJlIGRyYWZ0IGRvY3VtZW50 cyB2YWxpZCBmb3IgYSBtYXhpbXVtIG9mIHNpeCBtb250aHMNCiAgIGFuZCBtYXkgYmUgdXBkYXRl ZCwgcmVwbGFjZWQsIG9yIG9ic29sZXRlZCBieSBvdGhlciBkb2N1bWVudHMgYXQgYW55DQogICB0 aW1lLiAgSXQgaXMgaW5hcHByb3ByaWF0ZSB0byB1c2UgSW50ZXJuZXQtRHJhZnRzIGFzIHJlZmVy ZW5jZQ0KICAgbWF0ZXJpYWwgb3IgdG8gY2l0ZSB0aGVtIG90aGVyIHRoYW4gYXMgIndvcmsgaW4g cHJvZ3Jlc3MuIg0KDQogICBUaGUgbGlzdCBvZiBjdXJyZW50IEludGVybmV0LURyYWZ0cyBjYW4g YmUgYWNjZXNzZWQgYXQNCiAgIGh0dHA6Ly93d3cuaWV0Zi5vcmcvaWV0Zi8xaWQtYWJzdHJhY3Rz LnR4dC4NCg0KICAgVGhlIGxpc3Qgb2YgSW50ZXJuZXQtRHJhZnQgU2hhZG93IERpcmVjdG9yaWVz IGNhbiBiZSBhY2Nlc3NlZCBhdA0KICAgaHR0cDovL3d3dy5pZXRmLm9yZy9zaGFkb3cuaHRtbC4N Cg0KICAgVGhpcyBJbnRlcm5ldC1EcmFmdCB3aWxsIGV4cGlyZSBvbiBNYXkgMTQsIDIwMDguDQoN CkNvcHlyaWdodCBOb3RpY2UNCg0KICAgQ29weXJpZ2h0IChDKSBUaGUgSUVURiBUcnVzdCAoMjAw NykuDQoNCkFic3RyYWN0DQoNCiAgIFNldmVyYWwsIHNvbWV0aW1lcyBjb25mbGljdGluZyBwcm9w b3NhbHMgaGF2ZSBiZWVuIG9mZmVyZWQgZm9yIHRoZQ0KICAgY29uc2lkZXJhdGlvbiBvZiB0aGUg UENOIFdHIHJlZ2FyZGluZyBQQ04gaW50ZXJuYWwgbm9kZSBhbmQgUENOIGVkZ2UNCg0KDQoNCkNo YXJueSwgZXQgYWwuICAgICAgICAgICAgRXhwaXJlcyBNYXkgMTQsIDIwMDggICAgICAgICAgICAg ICAgICBbUGFnZSAxXQ0KDA0KSW50ZXJuZXQtRHJhZnQgICAgICAgICAgICAgIENvbXBhcmlzb24g RHJhZnQgICAgICAgICAgICAgICBOb3ZlbWJlciAyMDA3DQoNCg0KICAgbm9kZSBiZWhhdmlvcnMu ICBCYXNlZCBvbiB0aGUgV0cgY2hhcnRlciwgdGhlIFdHIG5lZWRzIHRvIG1ha2UgYQ0KICAgZGVj aXNpb24gb24gd2hpY2ggb2YgdGhlIHByb3Bvc2VkIFBDTi1pbnRlcmlvci1ub2RlIGFuZCBQQ04t Ym91bmRhcnktDQogICBub2RlcyBiZWhhdmlvcnMgdG8gZW5kb3JzZS4gIFRoZSBwcmltYXJ5IGdv YWwgb2YgdGhpcyBkcmFmdCBpcw0KICAgdHdvZm9sZC4gIEZpcnN0LCB3ZSBhdHRlbXB0IHRvIHN1 bW1hcml6ZSB0aGUgZnVuY3Rpb25hbCBkaWZmZXJlbmNlcw0KICAgYmV0d2VlbiB0aGUgcHJvcG9z ZWQgYWx0ZXJuYXRpdmVzLiAgU2Vjb25kLCB3ZSBwcm92aWRlIGEgYnJpZWYNCiAgIHN1bW1hcnkg b2YgcGVyZm9ybWFuY2UgZXZhbHVhdGlvbiByZXN1bHRzLiAgRmluYWxseSB3ZSBwcm9wb3NlIGEg dmlldw0KICAgb24gaG93IGEgKHBhcmFtZXRlcml6ZWQpIHNwZWNpZmljYXRpb24gb2YgdGhlIFBD Ti1pbnRlcmlvci1ub2RlDQogICBtZXRlcmluZyBhbmQgbWFya2luZyBmdW5jdGlvbiBjYW4gYmUg ZGVzY3JpYmVkIHRvIGVuYWJsZSBzZXZlcmFsIG9mDQogICB0aGUgcHJvcG9zZWQgYmVoYXZpb3Jz LiAgV2UgYXJndWUgdGhhdCBpZiB0aGlzIHBhcmFtZXRlcml6ZWQNCiAgIHNwZWNpZmljYXRpb24g aXMgdXNlZCBmb3Igc3BlY2lmeWluZyB0aGUgUENOLWludGVyaW9yLW5vZGUgYmVoYXZpb3IsDQog ICB0aGVuIGl0IGNhbiBzdXBwb3J0IGEgcmFuZ2Ugb2YgYmVoYXZpb3JzIGF0IHRoZSBQQ04tYm91 bmRhcnktbm9kZS4NCiAgIFRoZSBkZWNpc2lvbiBvbiB3aGljaCBvZiB0aGUgUENOLWJvdW5kYXJ5 LW5vZGUgYmVoYXZpb3JzIHRvIGNob29zZQ0KICAgY2FuIHRoZW4gYmUgY29uc2lkZXJlZCBzZXBh cmF0ZWx5LiAgV2UgYWxzbyBkaXNjdXNzIGNvbXBsZXhpdGllcw0KICAgYXNzb2NpYXRlZCB3aXRo IGNob29zaW5nIHN1Y2ggdW5pZm9ybSBhcHByb2FjaC4NCg0KUmVxdWlyZW1lbnRzIExhbmd1YWdl DQoNCiAgIFRoZSBrZXkgd29yZHMgIk1VU1QiLCAiTVVTVCBOT1QiLCAiUkVRVUlSRUQiLCAiU0hB TEwiLCAiU0hBTEwgTk9UIiwNCiAgICJTSE9VTEQiLCAiU0hPVUxEIE5PVCIsICJSRUNPTU1FTkRF RCIsICJNQVkiLCBhbmQgIk9QVElPTkFMIiBpbiB0aGlzDQogICBkb2N1bWVudCBhcmUgdG8gYmUg aW50ZXJwcmV0ZWQgYXMgZGVzY3JpYmVkIGluIFJGQyAyMTE5IFtSRkMyMTE5XS4NCg0KDQoNCg0K DQoNCg0KDQoNCg0KDQoNCg0KDQoNCg0KDQoNCg0KDQoNCg0KDQoNCg0KDQoNCg0KDQoNCg0KQ2hh cm55LCBldCBhbC4gICAgICAgICAgICBFeHBpcmVzIE1heSAxNCwgMjAwOCAgICAgICAgICAgICAg ICAgIFtQYWdlIDJdDQoMDQpJbnRlcm5ldC1EcmFmdCAgICAgICAgICAgICAgQ29tcGFyaXNvbiBE cmFmdCAgICAgICAgICAgICAgIE5vdmVtYmVyIDIwMDcNCg0KDQpUYWJsZSBvZiBDb250ZW50cw0K DQogICAxLiAgSW50cm9kdWN0aW9uIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAu IC4gLiAuIC4gLiAuIC4gIDQNCiAgICAgMS4xLiAgVGVybWlub2xvZ3kgIC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAgNA0KICAgICAxLjIuICBJbnRyb2R1Y3Rp b24gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuICA0DQogICAy LiAgUENOLUludGVyaW9yLU5vZGUgTWV0ZXJpbmcgYW5kIE1hcmtpbmcgRnVuY3Rpb25zIC4gLiAu IC4gLiAuIC4gIDUNCiAgICAgMi4xLiAgTWV0ZXJpbmcgYW5kIE1hcmtpbmcgVHlwZXMgLiAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAgNQ0KICAgICAyLjIuICBNZXRlcmluZyBhbmQgUmUt TWFya2luZyBvZiBQcmV2aW91c2x5IE1hcmtlZCBQYWNrZXRzIC4gLiAuICA2DQogICAgICAgMi4y LjEuICBBZG1pc3Npb24gTWFya2luZyAgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAu IC4gIDcNCiAgICAgICAyLjIuMi4gIFRlcm1pbmF0aW9uIE1hcmtpbmcgIC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAgNw0KICAgICAyLjMuICBOdW1iZXIgb2YgTWV0ZXJpbmcgRnVu Y3Rpb25zIGFuZCBNYXJraW5nIENvZGVwb2ludHMgIC4gLiAuICA3DQogICAzLiAgRHJvcHBpbmcg UG9saWNpZXMgIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gIDcN CiAgIDQuICBQQ04tQm91bmRhcnktTm9kZSBCZWhhdmlvcnMgIC4gLiAuIC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAgOA0KICAgICA0LjEuICBDTCBCb3VuZGFyeSBCZWhhdmlvciAuIC4gLiAu IC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuICA4DQogICAgIDQuMi4gIFNNIEJvdW5kYXJ5 IEJlaGF2aW9yIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gIDkNCiAgICAg NC4zLiAgM1NNIEJvdW5kYXJ5IEJlaGF2aW9yICAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAgOQ0KICAgICA0LjQuICBOb3RlcyBvbiBVc2luZyBEaWZmZXJlbnQgQm91bmRhcnkg QmVoYXZpb3JzIHdpdGgNCiAgICAgICAgICAgRGlmZmVyZW50IE1hcmtpbmcvTWV0ZXJpbmcgQmVo YXZpb3JzIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAxMA0KICAgNS4gIEluZm9ybWF0aW9uZWQgU2ln bmFsZWQgQmV0d2VlbiB0aGUgQm91bmRhcnkgTm9kZXMgIC4gLiAuIC4gLiAuIDEwDQogICA2LiAg RGVhbGluZyB3aXRoIEVDTVAgIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4g LiAuIC4gMTENCiAgIDcuICBTdWl0YWJpbGl0eSBmb3IgUHJvYmluZyBmb3IgQWRtaXNzaW9uIENv bnRyb2wgIC4gLiAuIC4gLiAuIC4gLiAxMQ0KICAgOC4gIENvbmZpZ3VyYXRpb24gQ29tcGxleGl0 eSBhbmQgQ29uZmlndXJhdGlvIFJlc3RyaWN0aW9ucyAuIC4gLiAuIDEyDQogICA5LiAgRnVuY3Rp b25hbCBDb21wYXJpc29uIFN1bW1hcnkgIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4g MTMNCiAgIDEwLiBPdGhlciBDb21wYXJpc29uIENyaXRlcmlhICAuIC4gLiAuIC4gLiAuIC4gLiAu IC4gLiAuIC4gLiAuIC4gLiAxNw0KICAgICAxMC4xLiBQZXJmb3JtYW5jZSBDb21wYXJpc29uIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIDE3DQogICAxMS4gVW5pZmllZCBEZXNj cmlwcmlvbiBvZiBNYXJraW5nIGFuZCBNZXRlcmluZyBGdW5jdGlvbnMgIC4gLiAuIC4gMTkNCiAg IDEyLiBEaWZmaWN1bHRpZXMgd2l0aCBBbGxvd2luZyBNdWx0aXBsZSBNYXJraW5nIEJlaGF2aW9y cyAgLiAuIC4gLiAyMw0KICAgMTMuIFNlY3VyaXR5IENvbnNpZGVyYXRpb25zICAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIDI0DQogICAxNC4gSUFOQSBDb25zaWRlcmF0aW9u cyAgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gMjQNCiAgIDE1LiBB cHBlbmRpeCAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAu IC4gLiAyNA0KICAgICAxNS4xLiBGb3JtdWxhdGlvbiBvZiB0aGUgU2ltcGxlIEdlbmVyYWxpemVk IE1ldGVyaW5nIGFuZA0KICAgICAgICAgICBNYXJraW5nIEFsZ29yaXRobSAgLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIDI0DQogICAgIDE1LjIuIFZRIEZvcm11bGF0aW9u IG9mIHRoZSBDb21wbGV4IEdlbmVyYWxpemVkIE1ldGVyaW5nIGFuZA0KICAgICAgICAgICBNYXJr aW5nIEFsZ29yaXRobSAgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIDI2 DQogICAxNi4gUmVmZXJlbmNlcyAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAu IC4gLiAuIC4gLiAuIC4gMjkNCiAgICAgMTYuMS4gTm9ybWF0aXZlIFJlZmVyZW5jZXMgLiAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAyOQ0KICAgICAxNi4yLiBJbmZvcm1hdGl2 ZSBSZWZlcmVuY2VzIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIDI5DQogICAg IDE2LjMuIFJlZmVyZW5jZXMgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAu IC4gLiAuIC4gMzENCiAgIEF1dGhvcnMnIEFkZHJlc3NlcyAuIC4gLiAuIC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAzMQ0KICAgSW50ZWxsZWN0dWFsIFByb3BlcnR5IGFu ZCBDb3B5cmlnaHQgU3RhdGVtZW50cyAuIC4gLiAuIC4gLiAuIC4gLiAuIDMzDQoNCg0KDQoNCg0K DQoNCg0KDQoNCg0KQ2hhcm55LCBldCBhbC4gICAgICAgICAgICBFeHBpcmVzIE1heSAxNCwgMjAw OCAgICAgICAgICAgICAgICAgIFtQYWdlIDNdDQoMDQpJbnRlcm5ldC1EcmFmdCAgICAgICAgICAg ICAgQ29tcGFyaXNvbiBEcmFmdCAgICAgICAgICAgICAgIE5vdmVtYmVyIDIwMDcNCg0KDQoxLiAg SW50cm9kdWN0aW9uDQoNCjEuMS4gIFRlcm1pbm9sb2d5DQoNCiAgIFRoaXMgZHJhZnQgdXNlcyB0 aGUgdGVybWlub2xvZ3kgZGVmaW5lZCBpbg0KICAgW0ktRC5pZXRmLXBjbi1hcmNoaXRlY3R1cmVd DQoNCjEuMi4gIEludHJvZHVjdGlvbg0KDQogICBBIG51bWJlciBvZiBzZWVtaW5nbHkgZGl2ZXJz ZSBhcHByb2FjaGVzIGhhdmUgYmVlbiBwcmVzZW50ZWQgaW4gdGhlDQogICBjb250ZXh0IG9mIHRo ZSB3b3JrIGluIHRoZSBQQ04gV0csIGVuY29tcGFzc2luZyBib3RoIHRoZSBQQ04tDQogICBpbnRl cmlvci1ub2RlIGFuZCBQQ04tYm91bmRhcnktbm9kZSBub2RlIGJlaGF2aW9ycy4gIFRoZSBnb2Fs IG9mIHRoaXMNCiAgIGluZm9ybWF0aW9uYWwgZHJhZnQgaXMgdG8gcHJvdmlkZSBhIGZ1bmN0aW9u YWwgY29tcGFyaXNvbiBvZiBzZXZlcmFsDQogICBjYW5kaWRhdGUgYXBwcm9hY2hlcyBmb3IgUENO IGJlaGF2aW9ycy4gIFdlIHJlZmVyIHRoZSByZWFkZXIgdG8NCiAgIFtJLUQuaWV0Zi1wY24tYXJj aGl0ZWN0dXJlXSBmb3IgYW4gYXJjaGl0ZWN0dXJhbCBvdmVydmlldyBvZiBQQ04uDQoNCiAgIFRo ZSBmaXJzdCBkcmFmdCBvZiB0aGlzIGRvY3VtZW50IGNvbmNlbnRyYXRlcyBvbiB0aGUgQ0wgYXBw cm9hY2gNCiAgIHByb3Bvc2VkIGluIFtJLUQuYnJpc2NvZS10c3Z3Zy1jbC1hcmNoaXRlY3R1cmVd ICwgdGhlIDNTTSBhcHByb2FjaA0KICAgZGVzY3JpYmVkIGluIFtJLUQuYmFiaWFyei1wY24tM3Nt XSAsIGFuZCB0aGUgU2luZ2xlLU1hcmtpbmcgKFNNKQ0KICAgYXBwcm9hY2ggcHJvcG9zZWQgaW4g W0ktRC5jaGFybnktcGNuLXNpbmdsZS1tYXJraW5nXS4gIFRoZSBhcHByb2FjaA0KICAgcHJvcG9z ZWQgaW4gW0ktRC53ZXN0YmVyZy1wY24tbG9hZC1jb250cm9sXSBpcyBub3QgY292ZXJlZCBpbiB0 aGUNCiAgIGluaXRpYWwgdmVyc2lvbiBvZiB0aGlzIGRyYWZ0LCBwZW5kaW5nIGNsYXJpZmljYXRp b25zIG9uIHRoZSBvcGVuDQogICBxdWVzdGlvbnMgcmVnYXJkaW5nIHRoZSBkZXRhaWxzIG9mIHRo ZSBhbGdvcml0aG1zIGRlc2NyaWJlZCBpbiB0aGF0DQogICBkcmFmdC4NCg0KICAgQXQgdGhlIHRp bWUgb2Ygd3JpdGluZyBvZiB0aGlzIGRyYWZ0LCBzZXZlcmFsIHBlcmZvcm1hbmNlIHN0dWRpZXMN CiAgIGhhdmUgYmVlbiB1bmRlcnRha2VuIGFuZCBhcmUgb24tZ29pbmcuICBUaGUgc3R1ZGllcyBw ZXJmb3JtZWQgaW4NCiAgIFtJLUQuemhhbmctcGNuLXBlcmZvcm1hbmNlLWV2YWx1YXRpb25dIGFu ZA0KICAgW0ktRC5jaGFybnktcGNuLXNpbmdsZS1tYXJraW5nXSBwcm92aWRlIGEgc2lkZS1ieS1z aWRlIGNvbXBhcmlzb24gb2YNCiAgIHBlcmZvcm1hbmNlIG9mIHRoZSBDTCBhbmQgU00gcHJvcG9z YWxzLiAgQSBwZXJmb3JtYW5jZSBldmFsdWF0aW9uIG9mDQogICB0aGUgM1NNIGFwcHJvYWNoIHdh cyByZXBvcnRlZCBpbiBbSS1ELmJhYmlhcnotcGNuLWV4cGxpY2l0LW1hcmtpbmddDQogICBhbmQg W1RSNDM3XS4gIEhvd2V2ZXIsIGR1ZSB0byBhIG51bWJlciBvZiBkaWZmZXJlbmNlcyBpbiB0aGUN CiAgIGV4cGVyaW1lbnRhbCBzZXR1cHMgaW4gZGlmZmVyZW50IHNpbXVsYXRpb24gc3R1ZGllcywg YSBzaWRlLWJ5LXNpZGUNCiAgIHBlcmZvcm1hbmNlIGNvbXBhcmlzb24gYmV0d2VlbiAzU00gYW5k IHRoZSBvdGhlciB0d28gYXBwcm9hY2hlcyBpcw0KICAgbm90IGZ1bGx5IHBvc3NpYmxlIGF0IHRo aXMgcG9pbnQuICBUaGVyZWZvcmUsIHdlIHByZXNlbnQgb25seSBhIHNob3J0DQogICBvdmVydmll dyBvZiBzb21lIG9mIHRoZSBwZXJmb3JtYW5jZSBldmFsdWF0aW9uIHJlc3VsdHMgd2hlcmUNCiAg IHBvc3NpYmxlLg0KDQogICBXZSB0aGVuIGFyZ3VlIHRoYXQgYSB1bmlmaWVkIChwYXJhbWV0ZXJp emVkKSBmb3JtdWxhdGlvbiBvZiB0aGUNCiAgIG1ldGVyaW5nIGFuZCBtYXJraW5nIGJlaGF2aW9y IGF0IHRoZSBQQ04taW50ZXJpb3Itbm9kZXMgY2FuIGJlDQogICBkZWZpbmVkLiAgVGh1cyBkZWZp bmVkIHVuaWZpZWQgUENOLWludGVyaW9yLW5vZGUgYmVoYXZpb3IgbWF5IHN1cHBvcnQNCiAgIG11 bHRpcGxlIFBDTi1ib3VuZGFyeS1ub2RlIGJlaGF2aW9ycywgYW5kIGhlbmNlIGluIHByaW5jaXBs ZSBjYW4gYmUNCiAgIHVzZWQgaW4gYSB2YXJpZXR5IG9mIGVudmlyb25tZW50cy4gIFdlIGFsc28g ZGlzY3VzcyBzb21lIG9mIHRoZQ0KICAgdHJhZGVvZmZzIGFuZCBhZGRpdGlvbmFsIGNvbXBsZXhp dGllcyBhc3NvY2lhdGVkIHdpdGggc3VjaCB1bmlmaWVkDQogICBQQ04taW50ZXJpb3Itbm9kZSBi ZWhhdmlvciBkZWZpbml0aW9uLg0KDQogICBTZWN0aW9ucyAyLTggcHJvdmlkZSBhbiBvdmVydmll dyBvZiB0aGUgdGhyZWUgcHJvcG9zYWxzLCBmb2xsb3dlZCBieQ0KICAgYSBzdW1tYXJ5IG9mIGZ1 bmN0aW9uYWwgZGlmZmVyZW5jZXMgYmV0d2VlbiB0aGUgcHJvcG9zYWxzIGluIFNlY3Rpb24NCg0K DQoNCkNoYXJueSwgZXQgYWwuICAgICAgICAgICAgRXhwaXJlcyBNYXkgMTQsIDIwMDggICAgICAg ICAgICAgICAgICBbUGFnZSA0XQ0KDA0KSW50ZXJuZXQtRHJhZnQgICAgICAgICAgICAgIENvbXBh cmlzb24gRHJhZnQgICAgICAgICAgICAgICBOb3ZlbWJlciAyMDA3DQoNCg0KICAgOS4gIFNlY3Rp b24gMTAgYnJpZWZseSBjb3ZlcnMgb3RoZXIgY29tcGFyaXNvbiBjcml0ZXJpYSBub3QgY292ZXJl ZA0KICAgaW4gdGhpcyBkcmFmdCwgaW5jbHVkaW5nIGEgYnJpZWYgc3VtbWFyeSBvZiBwZXJmb3Jt YW5jZSBldmFsdWF0aW9uDQogICBlZmZvcnRzIGFzIG9mIHRoZSB0aW1lIG9mIHdyaXRpbmcgb2Yg dGhpcyBkb2N1bWVudC4gIFNlY3Rpb24gMTENCiAgIHByb3ZpZGVzIGEgdW5pZmllZCBkZXNjcmlw dGlvbiBvZiBhZG1pc3Npb24gYW5kIHRlcm1pbmF0aW9uIGZ1bmN0aW9ucw0KICAgb2YgdGhlIFBD Ti1pbnRlcmlvci1ub2RlIGNvdmVyaW5nIGFsbCB0aHJlZSBwcm9wb3NhbHMgb2YgQ0wsIFNNIGFu ZA0KICAgM1NNLCBmb2xsb3dlZCBieSBhIGJyaWVmIGRpc2N1c3Npb24gb2YgdGhlIHRyYWRlb2Zm cyBhc3NvY2lhdGVkIHdpdGgNCiAgIHN1Y2ggdW5pZmllZCBkZWZpbml0aW9uIGluIFNlY3Rpb24g MTIuDQoNCg0KMi4gIFBDTi1JbnRlcmlvci1Ob2RlIE1ldGVyaW5nIGFuZCBNYXJraW5nIEZ1bmN0 aW9ucw0KDQogICBUaGlzIHNlY3Rpb24gcHJvdmlkZXMgYSBoaWdoIGxldmVsIGZ1bmN0aW9uYWwg Y29tcGFyaXNvbiBvZiB0aGUNCiAgIG1ldGVyaW5nIGFuZCBtYXJraW5nIGZ1bmN0aW9ucyBhdCB0 aGUgUENOLWludGVyaW9yLW5vZGUuICBGb3IgbW9yZQ0KICAgZGV0YWlsLCBwbGVhc2Ugc2VlIFtJ LUQuYnJpc2NvZS10c3Z3Zy1jbC1waGJdLA0KICAgW0ktRC5jaGFybnktcGNuLXNpbmdsZS1tYXJr aW5nXSwgYW5kIFtJLUQuYmFiaWFyei1wY24tM3NtXS4NCg0KMi4xLiAgTWV0ZXJpbmcgYW5kIE1h cmtpbmcgVHlwZXMNCg0KICAgTWV0ZXJpbmcgZnVuY3Rpb25zIGFyZSBkZWZpbmVkIGluIGRpZmZl cmVudCBwcm9wb3NhbHMgdmlhIHRoZSBub3Rpb25zDQogICBvZiBUb2tlbiBCdWNrZXQgKFRCKSBv ciBWaXJ0dWFsIFF1ZXVlIChWUSkuICBUaGVzZSB0d28gZm9ybXVsYXRpb25zDQogICBhcmUgZXF1 aXZhbGVudCBpbiB0aGUgc2Vuc2UgdGhhdCBlYWNoIG9uZSBjYW4gYmUgaW1wbGVtZW50ZWQgdmlh IHRoZQ0KICAgb3RoZXIgd2l0aCBhcHByb3ByaWF0ZSBzZXR0aW5ncy4gIFRoZXkgbWF5IGNvdW50 IHBhY2tldCBvciBieXRlcy4NCiAgIFRoZSBtYXJraW5nIGZ1bmN0aW9ucyBkaWZmZXIgd2l0aCBy ZXNwZWN0IHRvIGhvdyB0aGUgcXVldWUgbGVuZ3RoIG9mDQogICB0aGUgUSBvciB0aGUgZmlsbCBz dGF0ZSBvZiB0aGUgVEIgaXMgdXNlZCB3aGljaCBoYXMgYSBkaXJlY3QNCiAgIGluZmx1ZW5jZSBv biB0aGUgbWFya2luZyByZXN1bHQuICBUaGUgZm9sbG93aW5nIGFyZSB0aGUgbWFya2luZw0KICAg YmVoYXZpb3VycyB1c2VkIGluIFtJLUQuYnJpc2NvZS10c3Z3Zy1jbC1waGJdLA0KICAgW0ktRC5j aGFybnktcGNuLXNpbmdsZS1tYXJraW5nXSBhbmQgW0ktRC5iYWJpYXJ6LXBjbi0zc21dLg0KDQog ICBvICBFeGNlc3MtcmF0ZS1tYXJraW5nIG1hcmtzIHBhY2tldHMgd2hpY2ggZXhjZWVkIHRoZSBj b25maWd1cmVkDQogICAgICByYXRlLiAgSW4gdGhlIFRCIGZvcm11bGF0aW9uLCB0aGUgcGFja2V0 cyBhcmUgbWFya2VkIHdoZW4gdGhlDQogICAgICBhcnJpdmluZyBwYWNrZXQgZG9lcyBub3QgZmlu ZCBlbm91Z2ggdG9rZW5zIGluIHRoZSB0b2tlbiBidWNrZXQNCiAgICAgIChhbmQgdGhlIG1hcmtl ZCBwYWNrZXQgZG9lcyBub3QgY29uc3VtZSB0b2tlbnMgZnJvbSB0aGUgVEIpLiAgSW4NCiAgICAg IHRoZSBWUSBmb3JtdWxhdGlvbiwgdGhlIHBhY2tldHMgYXJlIG1hcmtlZCB3aGVuIHRoZSBhcnJp dmluZw0KICAgICAgcGFja2V0IHdvdWxkIGV4Y2VlZCB0aGUgY29uZmlndXJlZCBtYXhpbXVtIHNp emUgb2YgdGhlIFZRIChhbmQgdGhlDQogICAgICBtYXJrZWQgcGFja2V0IGlzIG5vdCBhZGRlZCB0 byB0aGUgUSkuICBUaGVzZSB0d28gZm9ybXVsYXRpb25zIGFyZQ0KICAgICAgZXF1aXZhbGVudCB3 aXRoIHRoZSBzYW1lIHJhdGVzIG9mIHRoZSBUQiBhbmQgVlEsIGFuZCB0aGUgZGVwdGggb2YNCiAg ICAgIHRoZSBUQiBudW1lcmljYWxseSBlcXVhbCB0byB0aGUgbWF4aW11bSBzaXplIG9mIHRoZSBR LiBUaGlzIGlzIHRoZQ0KICAgICAgbWFya2luZyB1c2VkIGZvciB0ZXJtaW5hdGlvbiBpbiBDTCwg YW5kIGZvciBib3RoIGFkbWlzc2lvbiBhbmQNCiAgICAgIHRlcm1pbmF0aW9uIGluIFNNLg0KDQog ICBvICBFeGNlc3MtcmF0ZS1tYXJraW5nLXdpdGgtbWFya2luZy1mcmVxdWVuY3ktcmVkdWN0aW9u IGlzIHNpbWlsYXIgdG8NCiAgICAgIHRoZSBFeGNlc3MtcmF0ZS1tYXJraW5nIGluIHRoZSBzZW5z ZSB0aGF0IGl0IGFsc28gbWFya3MgcGFja2V0cw0KICAgICAgd2hpY2ggZG8gbm90IGZpbmQgdG9r ZW5zIGluIHRoZSBUQiAoaW4gdGhlIFRCIGZvcm11bGF0aW9uKSwgb3INCiAgICAgIHdvdWxkIGV4 Y2VlZCB0aGUgbWF4aW11bSBzaXplIG9mIHRoZSBWUSAoaW4gdGhlIFZRIGZvcm11bGF0aW9uKS4N CiAgICAgIEhvd2V2ZXIsIHRvIHJlZHVjZSB0aGUgbnVtYmVyIG9mIG1hcmtlZCBwYWNrZXRzLCB3 aGVuZXZlciBhIHBhY2tldA0KICAgICAgaXMgbWFya2VkIGEgY2VydGFpbiBhbW91bnQgb2YgdG9r ZW5zIGFyZSBhZGRlZCB0byB0aGUgVEIgKGluIHRoZQ0KICAgICAgVEIgZm9ybXVsYXRpb24pIG9y IHRoZSBzYW1lIG51bWJlciBvZiBieXRlcyBpcyByZW1vdmVkIGZyb20gdGhlDQogICAgICBxdWV1 ZSBvZiB0aGUgVlEgKGluIHRoZSBWUSBmb3JtdWxhdGlvbikuICBUaGlzIGlzIHRoZSBtYXJraW5n IHVzZWQNCg0KDQoNCkNoYXJueSwgZXQgYWwuICAgICAgICAgICAgRXhwaXJlcyBNYXkgMTQsIDIw MDggICAgICAgICAgICAgICAgICBbUGFnZSA1XQ0KDA0KSW50ZXJuZXQtRHJhZnQgICAgICAgICAg ICAgIENvbXBhcmlzb24gRHJhZnQgICAgICAgICAgICAgICBOb3ZlbWJlciAyMDA3DQoNCg0KICAg ICAgZm9yIHRlcm1pbmF0aW9uIGluIDNTTS4NCg0KICAgbyAgUmFtcC1tYXJraW5nIGlzIGRlZmlu ZWQgYXMgZm9sbG93cy4gIEluIHRoZSBWUSBmb3JtdWxhdGlvbiwgdHdvIFZRDQogICAgICB0aHJl c2hvbGRzIGFyZSBkZWZpbmVkIChiZWxvdyB0aGUgbWF4aW11bSBWUSBzaXplKS4gIFBhY2tldHMg YXJlDQogICAgICBtYXJrZWQgd2l0aCBhIGNlcnRhaW4gcHJvYmFiaWxpdHkgZGVwZW5kaW5nIG9u IHRoZSBWUSBzaXplIGF0IHRoZQ0KICAgICAgdGltZSBvZiB0aGUgcGFja2V0IGFycml2YWwuICBU aGlzIHByb2JhYmlsaXR5IGlzIDAgZm9yIFZRIHF1ZXVlDQogICAgICBsZW5ndGggZnJvbSAwIHRv IGEgbG93ZXIgVlEgdGhyZXNob2xkLCBpdCByaXNlcyBsaW5lYXJseSBmcm9tIDAgdG8NCiAgICAg IDEgYmV0d2VlbiB0aGUgbG93ZXIgYW5kIGEgdXBwZXIgVlEgdGhyZXNob2xkLCBhbmQgaXQgaXMg MSBhYm92ZQ0KICAgICAgdGhlIHVwcGVyIFZRIHRocmVzaG9sZC4gIEFzIGEgcmVzdWx0LCBubyBw YWNrZXRzIGFyZSBtYXJrZWQgd2hlbg0KICAgICAgdGhlIHF1ZXVlIGxlbmd0aCBpcyBiZWxvdyB0 aGUgbG93ZXIgVlEgdGhyZXNob2xkLCBhIGZldyBwYWNrZXRzDQogICAgICBhcmUgbWFya2VkIHdo ZW4gdGhlIHF1ZXVlIGxlbmd0aCBpcyBiZXR3ZWVuIHRoZSBsb3dlciBhbmQgdGhlDQogICAgICB1 cHBlciBWUSB0aHJlc2hvbGRzLCBhbmQgYWxsIHBhY2tldHMgYXJlIG1hcmtlZCB3aGVuIHRoZSBx dWV1ZQ0KICAgICAgbGVuZ3RoIGlzIGFib3ZlIHRoZSB1cHBlciBWUSB0aHJlc2hvbGQuICBJbiB0 aGUgZXF1aXZhbGVudCBUQg0KICAgICAgZm9ybXVsYXRpb24sIHR3byBhZGRpdGlvbmFsIFRCIGZp bGwgdGhyZXNob2xkcyAoY2FsbGVkIHRoZSBsb3dlcg0KICAgICAgYW5kIHVwcGVyIFRCIHRocmVz aG9sZHMsIGJvdGggbm90IGV4Y2VlZGluZyB0aGUgVEIgZGVwdGgpIGFyZQ0KICAgICAgZGVmaW5l ZCAuICBUaGUgcGFja2VycyBhcmUgbWFya2VkIHdpdGggdGhlIHByb2JhYmlsaXR5IDEgZm9yIGEg VEINCiAgICAgIGZpbGwgc3RhdGUgZnJvbSAwIHRvIGEgbG93ZXIgVEIgdGhyZXNob2xkLiAgVGhl IHByb2JhYmlsaXR5DQogICAgICBkZWNyZWFzZXMgbGluZWFybHkgZnJvbSAxIHRvIDAgYmV0d2Vl biB0aGUgbG93ZXIgYW5kIHVwcGVyIFRCDQogICAgICB0aHJlc2hvbGRzLCBhbmQgaXQgaXMgMCBh Ym92ZSB0aGUgdXBwZXIgVEIgdGhyZXNob2xkLiAgQXMgYQ0KICAgICAgcmVzdWx0LCBhbGwgcGFj a2V0cyBhcmUgbWFya2VkIHdoZW4gdGhlIGZpbGwgc3RhdGUgaXMgYmVsb3cgdGhlDQogICAgICBs b3dlciB0aHJlc2hvbGQsIGEgZmV3IHBhY2tldHMgYXJlIG1hcmtlZCB3aGVuIHRoZSBmaWxsIHN0 YXRlIGlzDQogICAgICBiZXR3ZWVuIHRoZSBsb3dlciBhbmQgdGhlIHVwcGVyIHRocmVzaG9sZCwg YW5kIG5vIHBhY2tldHMgYXJlDQogICAgICBtYXJrZWQgYWJvdmUgdGhlIHVwcGVyIHRocmVzaG9s ZC4gIFRoaXMgaXMgdGhlIG1hcmtpbmcgZGVzY3JpYmVkDQogICAgICBpbiBbSS1ELmJyaXNjb2Ut dHN2d2ctY2wtcGhiXSAoaW4gdGhlIFZRIGZvcm11bGF0aW9uKSB3aGVyZSBpdCBpcw0KICAgICAg dXNlZCBmb3IgYWRtaXNzaW9uLg0KDQogICBvICBUaHJlc2hvbGQtbWFya2luZyBtYXJrcyBwYWNr ZXRzIHRoYXQgbWFrZSB0aGUgcXVldWUgbGVuZ3RoIG9mIHRoZQ0KICAgICAgVlEgZXhjZWVkIGEg Y2VydGFpbiB0aHJlc2hvbGQgd2hpY2ggaXMgbG93ZXIgdGhhbiB0aGUgcXVldWUgc2l6ZS4NCiAg ICAgIEFzIGEgcmVzdWx0LCBhbGwgcGFja2V0cyBhcmUgbWFya2VkIHdoZW4gdGhlIG1ldGVyZWQg dHJhZmZpYw0KICAgICAgZXhjZWVkcyB0aGUgVlEgcmF0ZS4gIEluIHRoZSBlcXVpdmFsZW50IFRC IGZvcm11bGF0aW9uLCBUaHJlc2hvbGQtDQogICAgICBtYXJraW5nIG1hcmtzIHBhY2tldHMgdGhh dCBtYWtlIHRoZSBudW1iZXIgb2YgdG9rZW5zIGluIHRoZSBUQg0KICAgICAgZmFsbCBiZWxvdyBh IGNlcnRhaW4gdGhyZXNob2xkIHdoaWNoIGlzIGxhcmdlciB0aGFuIHplcm8uICBBcyBhDQogICAg ICByZXN1bHQsIGFsbCBwYWNrZXRzIGFyZSBtYXJrZWQgd2hlbiB0aGUgbWV0ZXJlZCB0cmFmZmlj IGV4Y2VlZHMNCiAgICAgIHRoZSBUQiByYXRlLiAgVGhlIFZRIGFuZCBUQiBmb3JtdWxhdGlvbnMg YXJlIGVxdWl2YWxlbnQgd2l0aCB0aGUNCiAgICAgIFZRIG9mIHJhdGUgUiwgbWF4aW11bSBzaXpl IFMgYW5kIFZRIHRocmVzaG9sZCBULCBhbmQgVEIgd2l0aCByYXRlDQogICAgICBSLCBkZXB0aCBC IGFuZCB0aHJlc2hvbGQgVCwgYW5kIFRCIHRocmVzaG9sZCBCLVQuICBUaHJlc2hvbGQtDQogICAg ICBtYXJraW5nIHMgYSBzcGVjaWFsIGNhc2Ugb2YgUmFtcCBtYXJraW5nIHdoZW4gdGhlIGxvd2Vy IGFuZCB0aGUNCiAgICAgIHVwcGVyIChUQiBvciBRKSB0aHJlc2hvbGRzIGFyZSBpZGVudGljYWwu ICBJdCBpcyB1c2VkIGZvcg0KICAgICAgYWRtaXNzaW9uIGluIENMIGFuZCAzU00uDQoNCjIuMi4g IE1ldGVyaW5nIGFuZCBSZS1NYXJraW5nIG9mIFByZXZpb3VzbHkgTWFya2VkIFBhY2tldHMNCg0K ICAgV2hlbiBwYWNrZXRzIHRyYXZlbCBvdmVyIHNldmVyYWwgbGlua3Mgd2l0aGluIGEgUENOIGRv bWFpbiwgdGhleSBhcmUNCiAgIHBvc3NpYmx5IG1hcmtlZC4gIFRoaXMgc2VjdGlvbiBjbGFyaWZp ZXMgdGhlIHF1ZXN0aW9uIGNvbmNlcm5pbmcNCiAgIHBhY2tldHMgb2Ygd2hpY2ggbWFya2luZ3Mg c2hvdWxkIGJlIHRha2VuIGludG8gYWNjb3VudCBmb3IgdGhlDQogICBtZXRlcmluZyBwcm9jZXNz IG9uIHN1YnNlcXVlbnQgbGlua3MgYW5kIGlmIHNvIHdoaWNoIG1hcmtpbmdzIGFyZSByZS0NCiAg IG1hcmtlZC4NCg0KDQoNCg0KQ2hhcm55LCBldCBhbC4gICAgICAgICAgICBFeHBpcmVzIE1heSAx NCwgMjAwOCAgICAgICAgICAgICAgICAgIFtQYWdlIDZdDQoMDQpJbnRlcm5ldC1EcmFmdCAgICAg ICAgICAgICAgQ29tcGFyaXNvbiBEcmFmdCAgICAgICAgICAgICAgIE5vdmVtYmVyIDIwMDcNCg0K DQoyLjIuMS4gIEFkbWlzc2lvbiBNYXJraW5nDQoNCiAgIDNTTSBhbmQgQ0wgY29uc2lkZXIgcGFj a2V0cyBvZiBhbGwgbWFya2luZ3MgZm9yIG1ldGVyaW5nLCBidXQgVE0tDQogICBtYXJrZWQgcGFj a2V0cyBtdXN0IG5vdCBiZSByZS1tYXJrZWQgdG8gQU0uICBTTSByZXF1aXJlcyB0aGF0DQogICBw cmV2aW91c2x5IG1hcmtlZCBwYWNrZXRzIGFyZSBleGNsdWRlZCBmcm9tIG1ldGVyaW5nLCBhcyBu b3QgZG9pbmcgc28NCiAgIHdvdWxkIHJlc3VsdCBpbiB1bmRlcmVzdGltYXRpb24gb2Ygc3VzdGFp bmFibGUgYWRtaXNzaW9uIHJhdGUgaW4gdGhlDQogICBtdWx0aXBsZSBib3R0bGVuZWNrIHNjZW5h cmlvcywgYW5kIGNvbnNlcXVlbnRseSB3aWxsIHJlc3VsdCBpbiB0aGUNCiAgIHVuZGVyLWVzdGlt YXRpb24gb2YgdGhlIHN1c3RhaW5hYmxlIHRlcm1pbmF0aW9uIHJhdGUgYXQgdGhlIFBDTi0NCiAg IGluZ3Jlc3Mtbm9kZSwgaW4gdHVybiBjYXVzaW5nIG92ZXItdGVybWluYXRpb24gaW4gdGhlIG11 bHRpcGxlDQogICBib3R0bGVuZWNrcyBzY2VuYXJpb3MuDQoNCjIuMi4yLiAgVGVybWluYXRpb24g TWFya2luZw0KDQogICBDTCBuZWVkcyB0byBleGNsdWRlIGFsbCBwcmV2aW91c2x5IHRlcm1pbmF0 aW9uLW1hcmtlZCBwYWNrZXRzIGZyb20NCiAgIG1ldGVyaW5nIGluIG9yZGVyIHRvIHByZXZlbnQg dW5kZXJlc3RpbWF0aW9uIG9mIHN1c3RhaW5hYmxlDQogICB0ZXJtaW5hdGlvbiByYXRlLiAgSWYg cHJldmlvdXNseSB0ZXJtaW5hdGlvbi1tYXJrZWQgcGFja2V0cyBhcmUgbm90DQogICBleGNsdWRl ZCBmcm9tIG1ldGVyaW5nLCBzdWJzdGFudGlhbCBvdmVyLXRlcm1pbmF0aW9uIGluIHRoZSBtdWx0 aXBsZQ0KICAgYm90dGxlbmVjayBzY2VuYXJpb3MgbWlnaHQgb2NjdXIuIDNTTSBjYW4gYWNjb21t b2RhdGUgcHJldmlvdXNseQ0KICAgdGVybWluYXRpb24tIG1hcmtlZCBwYWNrZXRzIGJlaW5nIGlu Y2x1ZGVkIGZvciB0ZXJtaW5hdGlvbiBtZXRlcmluZw0KICAgKGFsdGhvdWdoIHRoZSBleGFjdCBp bXBhY3Qgb2YgZG9pbmcgc28gbmVlZHMgdG8gYmUgZnVydGhlcg0KICAgZXZhbHVhdGVkKS4gIEJv dGggaW4gQ0wgYW5kIDNTTSwgQU0tbWFya2VkIHBhY2tldHMgbWF5IGJlIHJlbWFya2VkIHRvDQog ICBUTS4gIE5vdGUgdGhhdCBTTSBkb2VzIG5vdCBlbXBsb3kgdGVybWluYXRpb24gbWFya2luZyBh dCB0aGUgUENOLQ0KICAgaW5ncmVzcy1ub2RlLCBhbiBoZW5jZSBkb2VzIG5vdCBoYXZlIGFueSB0 ZXJtaW5hdGlvbi1tYXJrZWQgcGFja2V0cw0KICAgYXQgYWxsLg0KDQoyLjMuICBOdW1iZXIgb2Yg TWV0ZXJpbmcgRnVuY3Rpb25zIGFuZCBNYXJraW5nIENvZGVwb2ludHMNCg0KICAgQm90aCBDTCBh bmQgM1NNIHJlcXVpcmUgMiBkaWZmZXJlbnQgbWV0ZXJpbmcgZnVuY3Rpb25zIC0gb25lIChwY24t DQogICBsb3dlci10aHJlc2hvbGQpIGZvciBhZG1pc3Npb24gYW5kIG9uZSAocGNuLXVwcGVyLXRo cmVzaG9sZCkgZm9yDQogICB0ZXJtaW5hdGlvbi4gIFRocmVlIG1hcmtpbmcgY29kZXBvaW50cyBh cmUgbmVlZGVkIGZvciBib3RoOiB1bm1hcmtlZCwNCiAgIGFkbWlzc2lvbi1tYXJrZWQgKGZpcnN0 IFBDTiBlbmNvZGluZykgZm9yIGFkbWlzc2lvbiwgYW5kIHRlcm1pbmF0aW9uLQ0KICAgbWFya2Vk IChzZWNvbmQgUENOIGVuY29kaW5nKSB0byB0ZXJtaW5hdGUgZmxvd3MuDQoNCiAgIEluIGNvbnRy YXN0LCBTTSByZXF1aXJlcyBhIHNpbmdsZSBtZXRlcmluZyB0aHJlc2hvbGQgYW5kIHR3bw0KICAg ZGlmZmVyZW50IG1hcmtpbmcgY29kZXBvaW50czogbWFya2VkIGFuZCB1bm1hcmtlZC4gIChOb3Rl OiBUaGVyZSBpcyBhDQogICBjaG9pY2UgdG8gYmUgbWFkZSB3aGV0aGVyIHRoZSAibWFya2VkIiBj b2RlcG9pbnQgc2hvdWxkIHVzZSB0aGUgZmlyc3QNCiAgIG9yIHRoZSBzZWNvbmQgUENOIGVuY29k aW5nIHN0YXRlLCBpZiBib3RoIGFyZSBkZWZpbmVkLikNCg0KDQozLiAgRHJvcHBpbmcgUG9saWNp ZXMNCg0KICAgQSBzdWJ0bGUgZGlmZmVyZW5jZSBpbiBleGlzdGluZyBwcm9wb3NhbHMgZm9yIHRo ZSBQQ04taW50ZXJpb3Itbm9kZQ0KICAgYmVoYXZpb3IgaXMgcmVsYXRlZCB0byBob3cgYWxyZWFk eSBtYXJrZWQgcGFja2V0cyBuZWVkIHRvIGJlIHRyZWF0ZWQNCiAgIGluIHRoZSBwcmVzZW5jZSBv ZiBsb3NzLiAgSXQgdHVybnMgb3V0IHRoYXQgYWxsIHRocmVlIGFwcHJvYWNoZXMNCiAgIGFzc3Vt ZSBkaWZmZXJlbnQgZHJvcCBwcmVmZXJlbmNlcy4NCg0KICAgQ0wgbmVlZHMgcHJlZmVyZW50aWFs IGRyb3BwaW5nIG9mIHRlcm1pbmF0aW9uLW1hcmtlZCBwYWNrZXRzLiAgSWYNCiAgIHN1Y2ggcHJl ZmVyZW50aWFsIGRyb3BwaW5nIGlzIG5vdCBpbXBsZW1lbnRlZCwgdGhlbiBwb3NzaWJsZSBvdmVy LQ0KDQoNCg0KQ2hhcm55LCBldCBhbC4gICAgICAgICAgICBFeHBpcmVzIE1heSAxNCwgMjAwOCAg ICAgICAgICAgICAgICAgIFtQYWdlIDddDQoMDQpJbnRlcm5ldC1EcmFmdCAgICAgICAgICAgICAg Q29tcGFyaXNvbiBEcmFmdCAgICAgICAgICAgICAgIE5vdmVtYmVyIDIwMDcNCg0KDQogICB0ZXJt aW5hdGlvbiBtYXkgb2NjdXIuICBJdCBjYW4gYmUgYXJndWVkIHRoYXQgdGhlIGFkbWlzc2lvbiBm dW5jdGlvbg0KICAgb2YgQ0wgaXMgbm90IHNlbnNpdGl2ZSB0byB3aGV0aGVyIG9yIG5vdCBhZG1p c3Npb24tbWFya2VkIHBhY2tldHMgYXJlDQogICBwcmVmZXJlbnRpYWxseSBkcm9wcGVkIG9yIG5v dC4NCg0KICAgU00gcmVsaWVzIG9uIHByZWZlcmVudGlhbCBkcm9wIG9mIG1hcmtlZCBwYWNrZXRz LiAgV2hpbGUgYWRtaXNzaW9uDQogICBmdW5jdGlvbiBvZiB0aGlzIGFwcHJvYWNoIGFwcGVhcnMg dG8gYmUgaW5zZW5zaXRpdmUgdG8gdGhlIGRyb3ANCiAgIHByZWZlcmVuY2UganVzdCBhcyBDTCBh ZG1pc3Npb24gZnVuY3Rpb24sIHRoZSB0ZXJtaW5hdGlvbiBmdW5jdGlvbiBvZg0KICAgU00gd2ls bCByZXN1bHQgaW4gb3Zlci10ZXJtaW5hdGlvbiBpZiBwcmVmZXJlbnRpYWwgZHJvcHBpbmcgb2YN CiAgIGFscmVhZHkgbWFya2VkIHBhY2tldHMgaXMgbm90IGltcGxlbWVudGVkLiAgV2hpbGUsIGlu c2Vuc2l0aXZpdHkgb2YNCiAgIENMIGFkbWlzc2lvbiBmdW5jdGlvbiB0byBtYXJrZWQgcGFja2V0 IGRyb3AgcmVtYWlucyB0byBiZSBzdHVkaWVkLA0KICAgZXNwZWNpYWxseSBpbiB0aGUgcHJlc2Vu Y2Ugb2YgbGFyZ2UgZGlmZmVyZW5jZXMgaW4gcGFja2V0IHNpemVzLg0KDQogICBJbiBjb250cmFz dCwgdGhlIHByb3Bvc2FsIGluIDNTTSBjYW4gYmVuZWZpdCBmcm9tIHByZWZlcmVudGlhbA0KICAg ZHJvcHBpbmcgb2YgdW5tYXJrZWQgZmxvdy10ZXJtaW5hdGlvbiBwYWNrZXRzLCBidXQgaXQgY2Fu IGZ1bmN0aW9uDQogICB3aXRob3V0IGF0IHRoZSBleHBlbnNlIG9mIGxvbmdlciB0ZXJtaW5hdGlv biB0aW1lLiAgSXQgY2FuIGJlIGFyZ3VlZA0KICAgdGhhdCB0aGUgYWRtaXNzaW9uIGZ1bmN0aW9u IG9mIDNTTSBpcyBub3Qgc2Vuc2l0aXZlIHRvIHdoZXRoZXIgb3Igbm90DQogICBhZG1pc3Npb24t bWFya2VkIHBhY2tldHMgYXJlIHByZWZlcmVudGlhbGx5IGRyb3BwZWQgb3Igbm90Lg0KDQoNCjQu ICBQQ04tQm91bmRhcnktTm9kZSBCZWhhdmlvcnMNCg0KNC4xLiAgQ0wgQm91bmRhcnkgQmVoYXZp b3INCg0KICAgSW4gdGhlIENMIGFwcHJvYWNoLCB0aGUgUENOLWVncmVzcy1ub2RlIG1lYXN1cmVz IHRoZSByYXRlIG9mDQogICBhZG1pc3Npb24tbWFya2VkLCB0ZXJtaW5hdGlvbi1tYXJrZWQsIGFu ZCB1bm1hcmtlZCBQQ04tdHJhZmZpYyBwZXINCiAgIGluZ3Jlc3MtZWdyZXNzLWFnZ3JlZ2F0ZS4g IEluIGFkZGl0aW9uLCB0aGUgUENOLWluZ3Jlc3Mtbm9kZSBtZWFzdXJlcw0KICAgdGhlIHJhdGUg b2Ygc2VudCBQQ04tdHJhZmZpYyBwZXIgaW5ncmVzcy1lZ3Jlc3MtYWdncmVnYXRlLiAgVG8NCiAg IHN1cHBvcnQgYWRtaXNzaW9uIGNvbnRyb2wsIHRoZSBQQ04tZWdyZXNzLW5vZGUgY2FsY3VsYXRl cyB0aGUNCiAgIENvbmdlc3Rpb24gTGV2ZWwgRXN0aW1hdGUgKENMRSkgZGVmaW5lZCBhcyBhIGZy YWN0aW9uIG9mIChhZG1pc3Npb24tDQogICBvciB0ZXJtaW5hdGlvbi0pIG1hcmtlZCB0cmFmZmlj IGFuZCB0aGUgb3ZlcmFsbCB0cmFmZmljLCBhbmQgc2lnbmFscw0KICAgdGhlIENMRSB0byB0aGUg UENOLWluZ3Jlc3Mtbm9kZS4gIFRoZSBQQ04taW5ncmVzcy1ub2RlIGFjY2VwdHMgb3INCiAgIHJl amVjdHMgZmxvd3MgYmFzZWQgb24gd2hldGhlciB0aGlzIENMRSB2YWx1ZSBmb3IgYSBwYXJ0aWN1 bGFyDQogICBpbmdyZXNzLWVncmVzcyBhZ2dyZWdhdGUgZXhjZWVkcyBhIHByZS1kZWZpbmVkIHRo cmVzaG9sZC4NCg0KICAgVG8gc3VwcG9ydCBmbG93IHRlcm1pbmF0aW9uLCB0aGUgUENOLWVncmVz cy1ub2RlIGNhbGN1bGF0ZXMgKG9uIGENCiAgIHBlci1pbmdyZXNzLWVncmVzcyBiYXNpcykgdGhl IFN1c3RhaW5hYmxlIChUZXJtaW5hdGlvbikgUmF0ZSwgZGVmaW5lZA0KICAgYXMgdGhlIGNvbWJp bmVkIHJhdGUgb2YgdW5tYXJrZWQgYW5kIGFkbWlzc2lvbi1tYXJrZWQgcGFja2V0cywgYW5kDQog ICBzaWduYWxzIHRoZSBTdXN0YWluYWJsZSBSYXRlIHRvIHRoZSBQQ04taW5ncmVzcy1ub2RlLiAg VGhlIFBDTi0NCiAgIGluZ3Jlc3Mtbm9kZSBtZWFzdXJlcyB0aGUgaW5ncmVzcyBzZW5kaW5nIHJh dGUgKGFnYWluIG9uIGEgcGVyLQ0KICAgaW5ncmVzcy1lZ3Jlc3MgYmFzaXMpIGFuZCBjYWxjdWxh dGVzIHRoZSBkaWZmZXJlbmNlIHRvIHRoZQ0KICAgc3VzdGFpbmFibGUgdGVybWluYXRpb24gcmF0 ZS4gIEl0IGNob29zZXMgYW4gYXBwcm9wcmlhdGUgc2V0IG9mIGZsb3dzDQogICB3aG9zZSBjb21i aW5lZCByYXRlIGNvcnJlc3BvbmRzIHRvIHRoYXQgZGlmZmVyZW5jZSBhbmQgdGVybWluYXRlcw0K ICAgdGhlc2UgZmxvd3MuICAoTm90ZTogdGhpcyBjaG9pY2UgaXMgZG9uZSB3aXRob3V0IGFueSBr bm93bGVkZ2Ugb2YNCiAgIHdoaWNoIGZsb3dzIGhhZCB0ZXJtaW5hdGlvbi1tYXJrZWQgcGFja2V0 cy4pDQoNCg0KDQoNCg0KDQoNCkNoYXJueSwgZXQgYWwuICAgICAgICAgICAgRXhwaXJlcyBNYXkg MTQsIDIwMDggICAgICAgICAgICAgICAgICBbUGFnZSA4XQ0KDA0KSW50ZXJuZXQtRHJhZnQgICAg ICAgICAgICAgIENvbXBhcmlzb24gRHJhZnQgICAgICAgICAgICAgICBOb3ZlbWJlciAyMDA3DQoN Cg0KNC4yLiAgU00gQm91bmRhcnkgQmVoYXZpb3INCg0KICAgSW4gdGhlIFNNIGFwcHJvYWNoLCB0 aGUgUENOLWVncmVzcy1ub2RlIG1lYXN1cmVzIHRoZSByYXRlIG9mIG1hcmtlZA0KICAgYW5kIHVu bWFya2VkIHRyYWZmaWMgcGVyIGluZ3Jlc3MtZWdyZXNzLWFnZ3JlZ2F0ZS4gIEluIGFkZGl0aW9u LCB0aGUNCiAgIFBDTi1pbmdyZXNzLW5vZGUgbWVhc3VyZXMgdGhlIHJhdGUgb2Ygc2VudCBQQ04t dHJhZmZpYyBwZXIgaW5ncmVzcy0NCiAgIGVncmVzcy1hZ2dyZWdhdGUuDQoNCiAgIFRvIHN1cHBv cnQgYWRtaXNzaW9uIGNvbnRyb2wsIHRoZSBQQ04tZWdyZXNzLW5vZGUgY2FsY3VsYXRlcyB0aGUN CiAgIGNvbmdlc3Rpb24gbGV2ZWwgZXN0aW1hdGUgKENMRSkgYXMgdGhlIGZyYWN0aW9uIG9mIG1h cmtlZCB0cmFmZmljIGFuZA0KICAgdGhlIG92ZXJhbGwgdHJhZmZpYywgYW5kIHNpZ25hbHMgdGhl IENMRSB0byB0aGUgUENOLWluZ3Jlc3Mtbm9kZS4NCiAgIFRoZSBQQ04taW5ncmVzcy1ub2RlIGFj Y2VwdHMgb3IgcmVqZWN0cyBmbG93cyBiYXNlZCBvbiB3aGV0aGVyIHRoaXMNCiAgIENMRSB2YWx1 ZSBleGNlZWRzIGEgcHJlLWRlZmluZWQgdGhyZXNob2xkLg0KDQogICBBbHRob3VnaCB0aGUgYm91 bmRhcnkgZnVuY3Rpb25zIG5lY2Vzc2FyeSB0byBzdXBwb3J0IGFkbWlzc2lvbg0KICAgY29udHJv bCBhcmUgc2ltaWxhciBmb3IgQ0wgYW5kIFNNLCBhbiBpbXBvcnRhbnQgZGlmZmVyZW5jZSBiZXR3 ZWVuDQogICB0aGUgdHdvIGFsZ29yaXRobXMgc3RlbXMgZnJvbSB0aGUgZmFjdCB0aGF0IENMIGlz IHVzaW5nIHRocmVzaG9sZCAob3INCiAgIHJhbXApIG1hcmtpbmcsIHdoaWxlIFNNIHVzZXMgZXhj ZXNzLXJhdGUtbWFya2luZy4gIEFzIGEgcmVzdWx0LCB0aGUNCiAgIG1lYW5pbmdmdWwgdmFsdWUg b2YgQ0xFIGlzIGFsc28gZGlmZmVyZW50LiAgRm9yIGV4YW1wbGUsIGluIGNhc2Ugb2YNCiAgIHNt YWxsIG92ZXJsb2FkcywgU00gd2lsbCBoYXZlIG9ubHkgYSBzbWFsbCBmcmFjdGlvbiBvZiBwYWNr ZXRzIG1hcmtlZA0KICAgKGFuZCBoZW5jZSB0aGUgYXBwcm9wcmlhdGUgQ0xFIHZhbHVlIG5lZWRl ZCB0byBkZXRlY3QgdGhlIG92ZXJsb2FkDQogICB3aXRob3V0IG92ZXIgYWRtaXNzaW9uIGlzIHNt YWxsKSwgd2hpbGUgYSBzbWFsbCAoYnV0IGNvbnNpc3RlbnQpDQogICBvdmVybG9hZCB3aXRoIENM IHJlc3VsdHMgaW4gdGhlIG1ham9yaXR5IG9mIHBhY2tldHMgYmVpbmcgbWFya2VkLA0KICAgcmVz dWx0aW5nIGluIENMIGJlaW5nIHF1aXRlIHJvYnVzdCBmb3IgYSByYW5nZSBvZiBDTEUgdmFsdWVz Lg0KDQogICBUbyBzdXBwb3J0IGZsb3cgdGVybWluYXRpb24sIHRoZSBQQ04tZWdyZXNzLW5vZGUg c2lnbmFscyB0aGUgcmF0ZSBvZg0KICAgdW5tYXJrZWQgcGFja2V0cyBhcyB0aGUgc28tY2FsbGVk IFN1c3RhaW5hYmxlIChBZG1pc3Npb24pIFJhdGUgdG8gdGhlDQogICBQQ04taW5ncmVzcy1ub2Rl LiAgVGhlIFBDTi1pbmdyZXNzLW5vZGUgbXVsdGlwbGllcyBpdCBieSBhIHN5c3RlbS0NCiAgIHdp ZGUgY29uc3RhbnQgdG8gZ2V0IHRoZSBTdXN0YWluYWJsZSBUZXJtaW5hdGlvbiBSYXRlLiAgVGhl IHJlc3Qgb2YNCiAgIHRoZSB0ZXJtaW5hdGlvbiBpcyBkb25lIGxpa2UgaW4gdGhlIENMIGFwcHJv YWNoLiAgVGhlIFBDTi1pbmdyZXNzLQ0KICAgbm9kZSBtZWFzdXJlcyB0aGUgaW5ncmVzcyByYXRl IGFuZCBjYWxjdWxhdGVzIHRoZSBkaWZmZXJlbmNlIHRvIHRoZQ0KICAgc3VzdGFpbmFibGUgdGVy bWluYXRpb24gcmF0ZS4gIEl0IGNob29zZXMgYW4gYXBwcm9wcmlhdGUgc2V0IG9mIGZsb3dzDQog ICB3aG9zZSBvdmVyYWxsIHJhdGUgY29ycmVzcG9uZHMgdG8gdGhhdCBkaWZmZXJlbmNlIGFuZCB0 ZXJtaW5hdGVzDQogICB0aGVzZXMgZmxvd3MuICAoTk9URTogVGhpcyBjaG9pY2UgaXMgZG9uZSB3 aXRob3V0IGFueSBrbm93bGVkZ2Ugb2YNCiAgIHdoaWNoIGZsb3dzIGhhZCBtYXJrZWQgcGFja2V0 cy4pDQoNCjQuMy4gIDNTTSBCb3VuZGFyeSBCZWhhdmlvcg0KDQogICBUbyBzdXBwb3J0IGZsb3cg YWRtaXNzaW9uLCBhIFBDTi1lZ3Jlc3Mtbm9kZSBhbmFseXplcyB0aGUgcGFja2V0DQogICBtYXJr aW5ncyBwZXIgaW5ncmVzcy1lZ3Jlc3MtYWdncmVnYXRlLiAgRGVwZW5kaW5nIG9uIHRoZSBtYXJr aW5ncyBpdA0KICAgc2VuZHMgZnJvbSB0aW1lIHRvIHRpbWUgImFkbWlzc2lvbi1zdG9wIiBvciAi YWRtaXNzaW9uLWNvbnRpbnVlIg0KICAgbWVzc2FnZXMgdG8gdGhlIGNvcnJlc3BvbmRpbmcgUENO LWluZ3Jlc3Mtbm9kZXMgdG8gY29udHJvbCB0aGVpcg0KICAgYWRtaXNzaW9uIG9mIG5ldyBmbG93 cy4gIFRoZSBQQ04taW5ncmVzcy1ub2RlIGFkbWl0cyBuZXcgZmxvd3Mgd2hlbg0KICAgdGhlIGxh c3QgY29udHJvbCBtZXNzYWdlIHdhcyAiYWRtaXNzaW9uLWNvbnRpbnVlIiBhbmQgaXQgcmVqZWN0 cyB0aGVtDQogICB3aGVuIGl0IHdhcyAiYWRtaXNzaW9uLXN0b3AiLiAzU00gbGVhdmVzIGRlbGli ZXJhdGVseSBvcGVuIHRoZSB3YXkNCiAgIGhvdyB0aGUgUENOLWVncmVzcy1ub2RlIGRlY2lkZXMg d2hlbiB0byBzZW5kIGEgc3BlY2lmaWMgY29udHJvbA0KICAgbWVzc2FnZS4gIEEgdmVyeSBzaW1w bGUgb3B0aW9uIGZvciB0aGUgUENOLWVncmVzcy1ub2RlIGlzIHRvIHNlbmQgYW4NCiAgICJhZG1p c3Npb24tc3RvcCIgbWVzc2FnZSB3aGVuIGEgc2luZ2xlIHBhY2tldCB3aXRoIGFkbWlzc2lvbi0g b3INCiAgIHRlcm1pbmF0aW9uLW1hcmtpbmcgaXMgb2JzZXJ2ZWQgYW5kIHRvIHNlbmQgYW4gImFk bWlzc2lvbi1jb250aW51ZSINCg0KDQoNCkNoYXJueSwgZXQgYWwuICAgICAgICAgICAgRXhwaXJl cyBNYXkgMTQsIDIwMDggICAgICAgICAgICAgICAgICBbUGFnZSA5XQ0KDA0KSW50ZXJuZXQtRHJh ZnQgICAgICAgICAgICAgIENvbXBhcmlzb24gRHJhZnQgICAgICAgICAgICAgICBOb3ZlbWJlciAy MDA3DQoNCg0KICAgbWVzc2FnZSBzb21lIHRpbWUgYWZ0ZXIgdGhlIGxhc3QgbWFya2VkIHBhY2tl dCBoYXMgYmVlbiBvYnNlcnZlZC4NCiAgIFRoaXMgaXMgdGhlIG1ldGhvZCB1c2VkIGZvciBwZXJm b3JtYW5jZSBldmFsdWF0aW9uIG9mIDNTTSByZXBvcnRlZCBpbg0KICAgW0ktRC5iYWJpYXJ6LXBj bi1leHBsaWNpdC1tYXJraW5nXS4gIEEgbW9yZSBzb3BoaXN0aWNhdGVkIG9wdGlvbiBpcw0KICAg dG8gY2FsY3VsYXRlIGEgQ0xFIGJhc2VkIG9uIGFuIGV4cG9uZW50aWFsbHkgd2VpZ2h0ZWQgbW92 aW5nIGF2ZXJhZ2UNCiAgIChFV01BKSBjb3VudGluZyBtYXJrZWQgbWVzc2FnZXMgYXMgMSBhbmQg dW1hcmtlZCBtZXNzYWdlcyBhcyAwLiAgV2hlbg0KICAgdGhlIENMRSBleGNlZWRzIGFuIHVwcGVy IHRocmVzaG9sZCwgYW4gImFkbWlzc2lvbi1zdG9wIiBtZXNzYWdlIGlzDQogICBzZW50LCBhbmQg d2hlbiB0aGUgQ0xFIGZhbGxzIGJlbG93IGEgbG93ZXIgdGhyZXNob2xkLCBhbiAiYWRtaXNzaW9u LQ0KICAgY29udGludWUiIG1lc3NhZ2UgaXMgc2VudC4gIE90aGVyIGltcGxlbWVudGF0aW9ucyBh cmUgcG9zc2libGUgc2luY2UNCiAgIHRoZSBkZWNpc2lvbiBsb2dpYyBpcyBsb2NhbCB0byB0aGUg UENOLWVncmVzcy1ub2RlLg0KDQogICBUbyBzdXBwb3J0IGZsb3cgdGVybWluYXRpb24sIHRoZSBQ Q04tZWdyZXNzLW5vZGUgaW4gdGhlIDNTTSBhcHByb2FjaA0KICAgYWdhaW4gbW9uaXRvcnMgdGhl IHBhY2tldCBtYXJraW5ncyBhbmQgc2lnbmFscyB0aGUgZmxvdyBJRCBvZg0KICAgdGVybWluYXRp b24tbWFya2VkIHBhY2tldHMgdG8gdGhlIFBDTi1pbmdyZXNzLW5vZGUgd2hlcmVieSBzZXZlcmFs DQogICBmbG93IElEcyBtYXkgYmUgc2VudCBpbiBhIHNpbmdsZSBtZXNzYWdlLiAgVGhlIFBDTi1p bmdyZXNzLW5vZGUNCiAgIHRlcm1pbmF0ZXMgdGhlc2UgZmxvd3MuICBOb3RlIHRoYXQgbmVpdGhl ciB0aGUgUENOLWluZ3Jlc3Mtbm9kZSBub3INCiAgIHRoZSBQQ04tZWdyZXNzLW5vZGUgYXJlIHJl cXVpcmVkIHRvIHBlcmZvcm0gcmF0ZSBtZWFzdXJlbWVudHMgaW4gM1NNLg0KDQo0LjQuICBOb3Rl cyBvbiBVc2luZyBEaWZmZXJlbnQgQm91bmRhcnkgQmVoYXZpb3JzIHdpdGggRGlmZmVyZW50DQog ICAgICBNYXJraW5nL01ldGVyaW5nIEJlaGF2aW9ycw0KDQogICBUaGUgcHJldmlvdXMgdGhyZWUg U3Vic2VjdGlvbnMgZGVzY3JpYmUgdGhlIFBDTi1ib3VuZGFyeS1ub2RlDQogICBiZWhhdmlvcnMg aW4gY29uanVuY3Rpb24gd2l0aCBzcGVjaWZpYyBwcm9wb3NhbHMgd2hlcmUgdGhlc2UNCiAgIGJl aGF2aW9ycyB3ZXJlIGRlZmluZWQuICBJdCBzaG91bGQgYmUgbm90ZWQsIGhvd2V2ZXIsIHRoYXQg dmFyaW91cw0KICAgZmVhdHVyZXMgb2Ygc3BlY2lmaWMgYm91bmRhcnkgYmVoYXZpb3JzIGRlc2Ny aWJlZCBpbiB0aGUgcHJldmlvdXMNCiAgIHNlY3Rpb25zIG1heSBiZSB1c2VkIHdpdGggZGlmZmVy ZW50IG1hcmtpbmcvbWV0ZXJpbmcgc3RyYXRlZ2llcy4gIEZvcg0KICAgZXhhbXBsZSwgYXMgaW5k aWNhdGVkIGluIFNlY3Rpb24gNC4zLCB0aGUgYm91bmRhcnkgYmVoYXZpb3IgdGhhdCBDTA0KICAg dXNlcyBmb3IgYWRtaXNzaW9uIGNhbiBhbHNvIGJlIHVzZWQgd2l0aCAzU00uICBMaWtld2lzZSwg dGhlDQogICBUZXJtaW5hdGlvbiBmdW5jdGlvbiBvZiBDTCBtYXkgY2hvb3NlIHRvIG9ubHkgdGVy bWluYXRlIHRob3NlIGZsb3dzDQogICB3aG9zZSBwYWNrZXRzIGFyZSBUTS1tYXJrZWQgLSBzZWUg U2VjdGlvbiA2IGZvciBkaXNjdXNzaW9uIG9uIGhvdw0KICAgdGhpcyBhZGRpdGlvbmFsIGZ1bmN0 aW9uYWxpdHkgY2FuIGJlIHVzZWQgdG8gYWRkcmVzcyBFQ01QIGlzc3Vlcy4NCg0KICAgSW4gZ2Vu ZXJhbCwgbmV3IGJvdW5kYXJ5IGJlaGF2aW9ycyBtYXkgYmUgZGVzaWduZWQgdG8gd29yayB3aXRo DQogICBwcm9wb3NlZCBtZXRlcmluZyBhbmQgbWFya2luZyBtZWNoYW5pc20uICBOZXZlcnRoZWxl c3MsIGluIHRoZQ0KICAgcmVtYWluaW5nIHBvcnRpb24gb2YgdGhpcyBkb2N1bWVudCB3ZSB3aWxs IGFzc3VtZSBzcGVjaWZpYyBib3VuZGFyeQ0KICAgbWVjaGFuaXNtcyBhcyBkZXNjcmliZWQgaW4g c2VjdGlvbnMgNC4xIC0gNC4zIHVubGVzcyBzdGF0ZWQNCiAgIG90aGVyd2lzZS4NCg0KDQo1LiAg SW5mb3JtYXRpb25lZCBTaWduYWxlZCBCZXR3ZWVuIHRoZSBCb3VuZGFyeSBOb2Rlcw0KDQogICBJ biBDTCwgdGhlIFBDTi1lZ3Jlc3Mtbm9kZSBtdXN0IHNlbmQgdG8gdGhlIFBDTi1pbmdyZXNzLW5v ZGUgdHdvDQogICB2YWx1ZXM6IGEgQ0xFIGZvciBBZG1pc3Npb24gYW5kIFN1c3RhaW5hYmxlIChU ZXJtaW5hdGlvbikgcmF0ZSBmb3INCiAgIFRlcm1pbmF0aW9uDQoNCiAgIEluIFNNLCB0aGUgUENO LWVncmVzcy1ub2RlIHNlbmRzIHRvIHRoZSBQQ04taW5ncmVzcy1ub2RlIHRoZSB0d28NCiAgIHZh bHVlcyBhcyBpbiBDTDogdGhlIENMRSBhbmQgU3VzdGFpbmFibGUgKEFkbWlzc2lvbikgUmF0ZS4g IChOb3RlDQogICB0aGF0IHdoaWxlIHRoZSBmb3JtYXQgb2YgdGhlIHNpZ25hbGluZyBpbmZvcm1h dGlvbiBpcyB0aGUgc2FtZSBmb3IgQ0wNCiAgIGFuZCBTTSwgdGhlIG1lYW5pbmcgb2YgU3VzdGFp bmFibGUgUmF0ZSBpcyBkaWZmZXJlbnQgaW4gdGhlIHR3bw0KDQoNCg0KQ2hhcm55LCBldCBhbC4g ICAgICAgICAgICBFeHBpcmVzIE1heSAxNCwgMjAwOCAgICAgICAgICAgICAgICAgW1BhZ2UgMTBd DQoMDQpJbnRlcm5ldC1EcmFmdCAgICAgICAgICAgICAgQ29tcGFyaXNvbiBEcmFmdCAgICAgICAg ICAgICAgIE5vdmVtYmVyIDIwMDcNCg0KDQogICBjYXNlcykuDQoNCiAgIEluIDNTTSwgdGhlIFBD Ti1lZ3Jlc3Mtbm9kZSBzaWduYWxzIHRvIHRoZSBQQ04taW5ncmVzcy1ub2RlIHVzaW5nDQogICBj b250cm9sIG1lc3NhZ2VzIGZvciB0aGUgQWRtaXNzaW9uIHByb2Nlc3MgKGUuZy4gYWRtaXNzaW9u LXN0b3Agb3INCiAgIGFkbWlzc2lvbi1jb250aW51ZSBtZXNzYWdlcykuICBGb3IgVGVybWluYXRp b24sIHRoZSBQQ04tZWdyZXNzLW5vZGUNCiAgIHNpZ25hbHMgdG8gdGhlIGluZ3Jlc3MgdGhlIGZs b3cgSUQgb2YgZWFjaCBmbG93IHRvIGJlIHRlcm1pbmF0ZWQuDQogICBOb3RlIHRoYXQgc2V2ZXJh bCBJRHMgY2FuIGJlIGNvbW11bmljYXRlZCBpbiBhIHNpbmdsZSBzaWduYWxsaW5nDQogICAoaS5l LiAgUlNWUCkgbWVzc2FnZS4NCg0KDQo2LiAgRGVhbGluZyB3aXRoIEVDTVANCg0KICAgRm9yIGFk bWlzc2lvbiBjb250cm9sLCBuZWl0aGVyIG9mIHRoZSB0aHJlZSBhbGdvcml0aG1zIGNvbnNpZGVy ZWQgaW4NCiAgIHRoaXMgZHJhZnQgYWRkcmVzcyBFQ01QIGlzc3VlIGluIHRoZSBhYnNlbmNlIG9m IHByb2Jpbmcgb2Ygc29tZSBzb3J0Lg0KICAgVGhlIHByb2JpbmcgZGlzY3Vzc2lvbiBpcyBkZWZl cnJlZCB0byB0aGUgbmV4dCBzZWN0aW9uLg0KDQogICBXaXRob3V0IHByb2JpbmcsIGluIHRoZSBj YXNlIHdoZW4gY29uZ2VzdGlvbiBzdGF0ZSBvZiBkaWZmZXJlbnQgcGF0aHMNCiAgIGluIHRoZSBu ZXR3b3JrIGRpZmZlciwgZmxvd3MgbWF5IGJlIGFkbWl0dGVkIG9uIGEgY29uZ2VzdGVkIHBhdGgN CiAgIHdoaWxlIHRoZSBvdGhlciBwYXRoIGNhbiByZW1haW4gdW5jb25nZXN0ZWQsIG9yLCBjb252 ZXJzZWx5LA0KICAgYWRtaXNzaW9uIG1heSBzdG9wIG9uIGFsbCBwYXRocywgd2hlbiBvbmx5IG9u ZSBwYXRoIGJlY29tZXMgcHJlLQ0KICAgY29uZ2VzdGVkLg0KDQogICBGb3IgdGVybWluYXRpb24s IDNTTSB3aWxsIGNvcnJlY3RseSBpZGVudGlmeSBmb3IgdGVybWluYXRpb24gb25seQ0KICAgdGhv c2UgZmxvd3Mgd2hpY2ggcGFzcyBjb25nZXN0ZWQgcGF0aHMgZXZlbiBpbiB0aGUgcHJlc2VuY2Ug b2YgRUNNUC4NCiAgIEluIGNvbnRyYXN0LCBib3RoIENMIGFuZCBTTSBtYXkgZXJyb25lb3VzbHkg dGVybWluYXRlIGZsb3dzIHRoYXQgZG8NCiAgIG5vdCB0cmF2ZXJzZSBjb25nZXN0ZWQgcGF0aHMu ICBGb3IgQ0wsIGFuIG9wdGlvbiBpcyBhdmFpbGFibGUgdG8NCiAgIGNob29zZSBmb3IgdGVybWlu YXRpb24gb25seSB0aG9zZSBmbG93cyB0aGF0IGFyZSB0ZXJtaW5hdGlvbi1tYXJrZWQuDQogICBI b3dldmVyLCBkb2luZyBzbyB0aGVuIHJlcXVpcmVzIHRoYXQgdGhlIGVncmVzcyBuZWVkcyB0byBh ZGRpdGlvbmFsbHkNCiAgIHNpZ25hbCB0byB0aGUgaW5ncmVzcyB3aGljaCBmbG93cyBoYXZlIGJl ZW4gbWFya2VkLCBvbiB0b3Agb2YgdGhlDQogICBvdGhlciBzaWduYWxpbmcgaW5mb3JtYXRpb24g ZGVzY3JpYmVkIGluIFNlY3Rpb24gNS4NCg0KICAgU2luZ2xlLW1hcmtpbmcgZG9lcyBub3QgZXhw bGljaXRseSBtYXJrIHRyYWZmaWMgYnkgdGVybWluYXRpb24tDQogICBtYXJraW5nIGFuZCBzbyB0 aGUgYWJvdmUgb3B0aW9uIHRvIGlkZW50aWZ5IGZsb3dzIG9mIGNvbmdlc3RlZCBwYXRoDQogICBk b2VzIG5vdCB3b3JrLiAgU00gZG9lcyBoYXZlIGFuIG9wdGlvbiB0byB0ZXJtaW5hdGUgb25seSB0 aG9zZSBmbG93cw0KICAgdGhhdCBhcmUgbWFya2VkIGZvciBhZG1pc3Npb24uICBIb3dldmVyLCB0 aGlzIG1heSByZXN1bHQgaW4gZXJyb25lb3VzDQogICB0ZXJtaW5hdGlvbiBvZiBmbG93cyBvbiBw YXRocyB3aGVyZSB0cmFmZmljIGlzIGFib3ZlIHRoZSBBZG1pc3Npb24NCiAgIHRocmVzaG9sZCBi dXQgYmVsb3cgdGhlIGxldmVsIHRoYXQgc2hvdWxkIGNhdXNlIFRlcm1pbmF0aW9uLiAgSnVzdCBh cw0KICAgaW4gdGhlIGNhc2Ugb2YgQ0wsIGRvaW5nIHNvIGFsc28gaW52b2x2ZXMgdGhlIG5lY2Vz c2l0eSB0byBzaWduYWwNCiAgIGFkZGl0aW9uYWwgaW5mb3JtYXRpb24gKGZsb3cgSURzKSBiZXR3 ZWVuIHRoZSBQQ04tZWdyZXNzLW5vZGUgYW5kDQogICBQQ04taW5ncmVzcyBub2RlLg0KDQoNCjcu ICBTdWl0YWJpbGl0eSBmb3IgUHJvYmluZyBmb3IgQWRtaXNzaW9uIENvbnRyb2wNCg0KICAgQWx0 aG91Z2ggcHJvYmluZyBpcyBjdXJyZW50bHkgb3V0LW9mLXNjb3BlIG9mIHRoZSBQQ04gV0cgY2hh cnRlciwgaXQNCiAgIG1heSBiZSB1c2VmdWwgaW4gYSBudW1iZXIgb2Ygc2l0dWF0aW9ucyAoaW4g cGFydGljdWxhciwgZGVhbGluZyB3aXRoDQogICBFQ01QIGZvciBhZG1pc3Npb24sIG9yIGFkZHJl c3NpbmcgZmxhc2ggY3Jvd2Qgc2l0dWF0aW9ucyBpbiB0aGUNCiAgIHByZXNlbmNlIG9mIG1hbnkg c21hbGwgaW5ncmVzcy1lZ3Jlc3MgYWdncmVnYXRlcykuICBXZSBkbyBub3QgYXR0ZW1wdA0KDQoN Cg0KQ2hhcm55LCBldCBhbC4gICAgICAgICAgICBFeHBpcmVzIE1heSAxNCwgMjAwOCAgICAgICAg ICAgICAgICAgW1BhZ2UgMTFdDQoMDQpJbnRlcm5ldC1EcmFmdCAgICAgICAgICAgICAgQ29tcGFy aXNvbiBEcmFmdCAgICAgICAgICAgICAgIE5vdmVtYmVyIDIwMDcNCg0KDQogICBoZXJlIHRvIGFk ZHJlc3MgdGhlIGlzc3VlIG9mIHdoZXRoZXIgb3Igbm90IGFuZCBleGFjdGx5IGhvdyB3ZWxsDQog ICBwcm9iaW5nIG1pZ2h0IHdvcmssIG5vciBkbyB3ZSBkaXNjdXNzIGFueSBwcm90b2NvbCBpc3N1 ZXMgYW5kDQogICBjb21wbGV4aXRpZXMgYXNzb2NpYXRlZCB3aXRoIHByb2JpbmcuICBJbnN0ZWFk IHdlIHRvdWNoIHVwb24gb25seSBvbmUNCiAgIGFzcGVjdCBvZiBpdCAtIGhvdyB3ZWxsIHRoZSBQ Q04gYWRtaXNzaW9uIG1hcmtpbmcgb2YgYnkgdGhlIHRocmVlDQogICBwcm9wb3NhbHMgaW4gcXVl c3Rpb24gbWlnaHQgd29yayB3aXRoIHByb2JpbmcuDQoNCiAgIFRocmVzaG9sZC1tYXJraW5nIGVt cGxveWVkIGJ5IGJvdGggQ0wgYW5kIDNTTSByZXN1bHRzIGluIGFsbCBwYWNrZXRzDQogICBiZWlu ZyBtYXJrZWQgb25jZSB0aGUgdHJhZmZpYyByYXRlIGV4Y2VlZHMgUENOLWxvd2VyLXRocmVzaG9s ZCAoaS5lDQogICB0aGUgYWRtaXNzaWJsZSByYXRlIHRocmVzaG9sZCkuICBUaGVyZWZvcmUsIGEg c2luZ2xlIHByb2JlIHBhY2tldCBpcw0KICAgZ3VhcmFudGVlZCB0byBiZSBtYXJrZWQgaWYgdGhl IFBDTi10cmFmZmljIHJhdGUgZXhjZWVkcyB0aGUgUENOLQ0KICAgbG93ZXItdGhyZXNob2xkIHRo cmVzaG9sZC4gIFRoZXJlZm9yZSwgdGhlIGFkbWlzc2lvbiBkZWNpc2lvbiBjYW4gYmUNCiAgIHJl bGlhYmx5IG1hZGUgYmFzZWQgb24gYSBzaW5nbGUgcHJvYmUuICBPbmUgcG9zc2liaWxpdHkgbWF5 IGJlIHRvIHVzZQ0KICAgc2lnbmFsaW5nIChlLmcuICBSU1ZQKSBtZXNzYWdlcyBhcyBwcm9iZXMg aW4gdGhpcyBjYXNlLg0KDQogICBJbiBTTSwgYWRtaXNzaW9uIG1ldGVyaW5nIGFuZCBtYXJraW5n IGlzIGJhc2VkIG9uIEV4Y2Vzcy1yYXRlLQ0KICAgbWFya2luZy4gIEluIHRoaXMgY2FzZSwgb25s eSBhIGZyYWN0aW9uIG9mIHRyYWZmaWMgaXMgbWFya2VkIHdoZW4NCiAgIHRyYWZmaWMgZXhjZWVk cyB0aGUgY29uZmlndXJlZCAoYWRtaXNzaWJsZSByYXRlKSB0aHJlc2hvbGQuDQogICBUaGVyZWZv cmUsIHdoZW4gdGhlIFBDTi10cmFmZmljIHJhdGUgZXhjZWVkcyB0aGlzIHRocmVzaG9sZCwgYSBz aW5nbGUNCiAgIHByb2JlIHdpbGwgb25seSBiZSBtYXJrZWQgd2l0aCBhIGNlcnRhaW4gcHJvYmFi aWxpdHksIGFuZCBzbyBhIHNlcmllcw0KICAgb2YgcHJvYmVzIG5lZWQgdG8gYmUgc2VudCB0byBk ZXRlY3QgY29uZ2VzdGlvbiB3aXRoIGhpZ2ggcHJvYmFiaWxpdHkuDQoNCiAgIFRodXMsIFRocmVz aG9sZC1tYXJraW5nIG9mIENMIGFuZCAzU00gYWxsb3dzIGZhc3RlciBhbmQgc2ltcGxlcg0KICAg YWRtaXNzaW9uIGRlY2lzaW9ucyB0aGFuIEV4Y2Vzcy1yYXRlLW1hcmtpbmcgb2YgU00uDQoNCiAg IEluIGFkZGl0aW9uIGl0IHNob3VsZCBiZSBub3RlZCB0aGF0IHRoZSBuZWNlc3NpdHkgdG8gc2Vu ZCBtdWx0aXBsZQ0KICAgcHJvYmVzIGZvciBTTSBhZGRzIGEgcG90ZW50aWFsIHByb2JsZW0gd2l0 aCB1c2luZyBSU1ZQIG1lc3NhZ2VzIGFzDQogICBwcm9iZXMuICBFeHRlbnNpb25zIG5lY2Vzc2Fy eSB0byBkbyBzbyBoYXZlIG5vdCBiZWVuIGNvbnNpZGVyZWQgYXQNCiAgIHRoaXMgdGltZS4NCg0K DQo4LiAgQ29uZmlndXJhdGlvbiBDb21wbGV4aXR5IGFuZCBDb25maWd1cmF0aW8gUmVzdHJpY3Rp b25zDQoNCiAgIFRoZSBhcHByb2FjaCBpbiBTTSByZXF1aXJlcyBhIGdsb2JhbCBjb25maWd1cmF0 aW9uIHBhcmFtZXRlciBhdCB0aGUNCiAgIGVkZ2VzIHJlZmxlY3RpbmcgdGhlIGFzc3VtZWQgcmF0 aW8gYmV0d2VlbiB0aGUgKGltcGxpY2l0KSB0ZXJtaW5hdGlvbg0KICAgdGhyZXNob2xkIGFuZCAo ZXhwbGljaXQpIGFkbWlzc2lvbiB0aHJlc2hvbGQsIHdoaWNoIGlzIGFzc3VtZWQgdG8gYmUNCiAg IGdsb2JhbCBvbiBhbGwgbGlua3MgaW4gdGhlIFBDTiBkb21haW4uICBUaGlzIG5lY2Vzc2l0YXRl cyB0aGUgdXNlIG9mDQogICBhIGdsb2JhbCBwYXJhbWV0ZXIgdGhhdCBuZWVkcyB0byBiZSBjb25m aWd1cmVkIHRvIHRoZSBzYW1lIHZhbHVlIG9uDQogICBhbGwgUENOLWJvdW5kYXJ5LW5vZGVzIGlu IHRoZSBuZXR3b3JrLiAgVGhpcyBjbGVhcmx5IGxlYWRzIHRvIGFuDQogICBhZGRpdGlvbmFsIGNv bmZpZ3VyYXRpb24gY29tcGxleGl0eSBvZiBTTSBjb21wYXJlZCB0byBDTCBhbmQgM1NNLg0KDQog ICBBbiBhZGRpdGlvbmFsIGlzc3VlIGNhdXNlZCBieSB0aGUgYXNzdW1wdGlvbiBvZiB0aGUgZ2xv YmFsIHJhdGlvDQogICBiZXR3ZWVuIChpbXBsaWNpdCkgdGVybWluYXRpb24gdGhyZXNob2xkIGFu ZCAoZXhwbGljaXQpIGFkbWlzc2lvbg0KICAgdGhyZXNob2xkIG9mIFNNIGlzIHJlbGF0ZWQgdG8g dGhlIGZsZXhpYmlsaXR5IG9mIHJlc2lsaWVuY3kgcGxhbm5pbmcuDQogICBJbiB0aGlzIGNvbnRl eHQsIGEgbmF0dXJhbCBhcHByb2FjaCBpcyB0byB1c2UgUENOLWxvd2VyLXRocmVzaG9sZCB0bw0K ICAgcmVwcmVzZW50IHRoZSBleHBlY3RlZCB1dGlsaXNhdGlvbiBvbiBkaWZmZXJlbnQgbGlua3Mg Zm9yIGV4cGVjdGVkDQogICB0cmFmZmljIG1hdHJpeCB1bmRlciBub3JtYWwsIG5vbi1mYWlsdXJl LCBjb25kaXRpb25zLCBhbmQgdG8gdmlldw0KICAgUENOLXVwcGVyLXRocmVzaG9sZCB0byByZXBy ZXNlbnQgZXhwZWN0ZWQgd29yc3QgY2FzZSB1dGlsaXphdGlvbg0KICAgdW5kZXIgYW55IG9mIHRo ZSAicGxhbm5lZCIgZmFpbHVyZXMuDQoNCg0KDQpDaGFybnksIGV0IGFsLiAgICAgICAgICAgIEV4 cGlyZXMgTWF5IDE0LCAyMDA4ICAgICAgICAgICAgICAgICBbUGFnZSAxMl0NCgwNCkludGVybmV0 LURyYWZ0ICAgICAgICAgICAgICBDb21wYXJpc29uIERyYWZ0ICAgICAgICAgICAgICAgTm92ZW1i ZXIgMjAwNw0KDQoNCiAgIEFzIGRpc2N1c3NlZCBpbiBbTWVudGhdIGFuZCBbSS1ELmNoYXJueS1w Y24tc2luZ2xlLW1hcmtpbmddICwgdGhlDQogICBnbG9iYWwgcmF0aW8gYmV0d2VlbiB0aGUgdGVy bWluYXRpb24gYW5kIGFkbWlzc2lvbiB1dGlsaXNhdGlvbiBsZXZlbHMNCiAgIGFzc3VtZWQgaW4g U00gbGltaXRzIHRoZSBmbGV4aWJpbGl0eSBvZiB0cmFmZmljIGVuZ2luZWVyaW5nIGZvcg0KICAg cmVzaWxpZW5jeSB0byBhIGNlcnRhaW4gZXh0ZW50LiAgU3BlY2lmaWNhbGx5LCBXaGlsZSBhbGwg dGhyZWUNCiAgIGFwcHJvYWNoZXMgY2FuIGJlIGNvbmZpZ3VyZWQgdG8gZW5zdXJlIHRoYXQgdGhl IHBsYW5uZWQgbWF0cml4IGNhbiBiZQ0KICAgcHJvdGVjdGVkIGFnYWluc3QgYWxsIHRoZSBwbGFu bmVkIGZhaWx1cmUgY29uZGl0aW9ucywgdGhlIG5hdHVyZSBvZg0KICAgdGhlIGd1YXJhbnRlZSBm b3IgdGhlIGFkbWl0dGVkIHRyYWZmaWMgaXMgZGlmZmVyZW50LiAgQm90aCBDTCBhbmQgM1NNDQog ICBjYW4gYmUgY29uZmlndXJlZCB0byBwcm90ZWN0IGFsbCBhZG1pdHRlZCB0cmFmZmljIChidXQg d291bGQgbm90DQogICBhZG1pdCBtb3JlIHRoYW4gdGhlIHBsYW5uZWQgdHJhZmZpYyBtYXRyaXgp LCB3aGlsZSBTTSBjYW4gYmUNCiAgIGNvbmZpZ3VyZWQgdG8gYWRtaXQgbW9yZSB0cmFmZmljIHRo YW4gcGxhbm5lZCwgYnV0IHdpbGwgbm90IGd1YXJhbnRlZQ0KICAgcHJvdGVjdGlvbiBhZ2FpbnN0 IHBsYW5uZWQgZmFpbHVyZXMgZm9yIHRyYWZmaWMgZXhjZWVkaW5nIHBsYW5uZWQNCiAgIHV0aWxp emF0aW9uIGZvciB0aGUgcGxhbm5lZCB0cmFmZmljIG1hdHJpeC4NCg0KICAgSXQgc2hvdWxkIGJl IG5vdGVkIHRoYXQgaW4gcHJpbmNpcGxlLCBhbGwgdGhyZWUgYWxnb3JpdGhtcyBjYW4gYmUNCiAg IGNvbmZpZ3VyZWQgdG8gcHJvdGVjdCBhbGwgYWRtaXR0ZWQgdHJhZmZpYyAod2hldGhlciBvciBu b3QgdGhpcw0KICAgYWRtaXR0ZWQgdHJhZmZpYyBleGNlZWRzIHRoZSBwbGFubmVkIHRyYWZmaWMg bWF0cml4IG9yIG5vdCkuDQogICBIb3dldmVyLCBhcyBhcmd1ZWQgaW4gW01lbnRoXSwgaW4gdGhp cyBjYXNlIFNNIGNhbiBnZW5lcmFsbHkgYWRtaXQNCiAgIGxlc3MgdHJhZmZpYyB0aGFuIENMIG9y IDNTTS4NCg0KICAgV2UgcmVmZXIgdGhlIHJlYWRlciB0byBbTWVudGhdIGFuZCBbSS1ELmNoYXJu eS1wY24tc2luZ2xlLW1hcmtpbmddDQogICBmb3IgYSBtb3JlIGRldGFpbGVkIGRpc2N1c3Npb24g b24gdGhpcyBpc3N1ZS4NCg0KDQo5LiAgRnVuY3Rpb25hbCBDb21wYXJpc29uIFN1bW1hcnkNCg0K ICAgVGhlIGZvbGxvd2luZyBUYWJsZSBzdW1tYXJpemVzIHRoZSBkaWZmZXJlbmNlcyBiZXR3ZWVu IHRoZSB0aHJlZQ0KICAgYXBwcm9hY2hlcyBkaXNjdXNzZWQgc28gZmFyLg0KDQogICAocHJlYW1i bGUpDQogICAgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLXwNCiAgIHxDb21wYXJpc29uICAgIHwgICAgICBTTSAgICAgICB8 ICAgICAgIDNTTSAgICAgICAgfCAgICAgIENMICAgICAgICAgfA0KICAgfGNyaXRlcmlhICAgICAg fCAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgICB8DQog ICB8LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0NCiAgIHwjIG9mIGVuY29kaW5nIHwgICAgICAgMiAgICAgICB8ICAgICAg ICAzICAgICAgICAgfCAgICAgICAzICAgICAgICAgfA0KICAgfGVuY29kaW5nICAgICAgfCAgICAg ICAgICAgICAgIHwgICAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgICB8DQogICB8c3Rh dGVzICAgICAgICB8ICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAgIHwgICAgICAgICAg ICAgICAgIHwNCiAgIHwtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tfA0KICAgfCMgb2YgbWV0ZXJpbmcgfCAgICAgICAgICAg ICAgIHwgICAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgICB8DQogICB8bWVjaGFuaXNt cyAgICB8ICAgICAgIDEgICAgICAgfCAgICAgICAgMiAgICAgICAgIHwgICAgICAgMiAgICAgICAg IHwNCiAgIHxpbiBmb3J3YXJkaW5nIHwgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgICAg fCAgICAgICAgICAgICAgICAgfA0KICAgfHBhdGggb2YgICAgICAgfCAgICAgICAgICAgICAgIHwg ICAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgICB8DQogICB8aW50ZXJpb3Igbm9kZXN8 ICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgIHwNCiAg IHwtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tfA0KICAgfFR5cGUgb2YgICAgICAgfCAgIGV4Y2Vzcy1yYXRlIHwgICAgIHRo cmVzaG9sZCAgICB8ICB0aHJlc2hvbGQgb3IgICB8DQogICB8bWV0ZXJpbmcgICYgICB8ICAgICBt YXJraW5nICAgfCAgICAgbWFya2luZyAgICAgIHwgIHJhbXAgbWFya2luZyAgIHwNCiAgIHxtYXJr aW5nIGZvciAgIHwgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgICAgfCAgICAgICAgICAg ICAgICAgfA0KICAgfGFkbWlzc2lvbiAgICAgfCAgICAgICAgICAgICAgIHwgICAgICAgICAgICAg ICAgICB8ICAgICAgICAgICAgICAgICB8DQogICB8LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLXwNCg0KDQoNCkNoYXJueSwg ZXQgYWwuICAgICAgICAgICAgRXhwaXJlcyBNYXkgMTQsIDIwMDggICAgICAgICAgICAgICAgIFtQ YWdlIDEzXQ0KDA0KSW50ZXJuZXQtRHJhZnQgICAgICAgICAgICAgIENvbXBhcmlzb24gRHJhZnQg ICAgICAgICAgICAgICBOb3ZlbWJlciAyMDA3DQoNCg0KICAgfFR5cGUgb2YgICAgICAgfCAgICAg Ti9BICAgICAgIHwgZXhjZXNzLXJhdGUgd2l0aCB8ICBleGNlc3MtcmF0ZSAgICB8DQogICB8bWV0 ZXJpbmcgYW5kICB8ICAgICAgICAgICAgICAgfCBtYXJraW5nIGZyZXF1ZW5jeXwgICAgIG1hcmtp bmcgICAgIHwNCiAgIHxtYXJraW5nIGZvciAgIHwgICAgICAgICAgICAgICB8ICAgICByZWR1Y3Rp b24gICAgfCAgICAgICAgICAgICAgICAgfA0KICAgfHRlcm1pbmF0aW9uICAgfCAgICAgICAgICAg ICAgIHwgICAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgICB8DQogICB8LS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LXwNCiAgIHxNZXRlcmluZyBhbmQgIHwgZG8gbm90IG1ldGVyICB8ICBtZXRlciBhbGwgcGt0cyAg fCBkbyBub3QgbWV0ZXIgICAgfA0KICAgfHJlbWFya2luZyBvZiAgfCBBTS1tYXJrZWQgICAgIHwg IGZvciBhZG1pc3Npb24gICB8IFRNLW1hcmtlZCBwa3RzICB8DQogICB8cHJldmlvdXNseSAgICB8 IHBhY2tldHMgICAgICAgfCAgYW5kIHRlcm1pbmF0aW9uO3wgZm9yIHRlcm1pbmF0aW9uIHwNCiAg IHxtYXJrZWQgICAgICAgIHwgICAgICAgICAgICAgICB8ICBkbyBub3QgcmUtbWFyayAgfCBtZXRl ciBhbGwgcGt0czt8DQogICB8cGFja2V0cyAgICAgICB8ICAgICAgICAgICAgICAgfCAgVE0tbWFy a2VkIHBrdHMgIHwgZm9yIGFkbWlzc2lvbiwgIHwNCiAgIHwgICAgICAgICAgICAgIHwgICAgICAg ICAgICAgICB8ICAgICAgICAgICAgICAgICAgfCBkbyBub3QgcmUtbWFyayAgfA0KICAgfCAgICAg ICAgICAgICAgfCAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICB8IFRNLW1hcmtlZCBw a3RzICB8DQogICB8LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLXwNCiAgIHxQYWNrZXQgRHJvcCAgIHxwcmVmZXJlbnRpYWxs eSB8IG5vIHNlbnNpdGl2aXR5ICAgfCBubyBzZW5zaXRpdml0eSAgfA0KICAgfHByZWZlcmVuY2Ug ICAgfCBkcm9wIEFNLW1hcmtlZHwgdG8gbW9kZXJhdGUgZHJvcCB8IHRvIG1vZGVyYXRlIGRyb3B8 DQogICB8ICAgICAgICAgICAgICB8IHBhY2tldHM7IG92ZXItfCBvZiBBTS1tYXJrZWQgcGt0c3wg b2YgQU0tbWFya2VkICAgIHwNCiAgIHwgICAgICAgICAgICAgIHx0ZXJtaW5hdGlvbiAgICB8IHBy ZWZlcmVudGlhbGx5ICAgfCBwYWNrZXRzOyAgICAgICAgfA0KICAgfCAgICAgICAgICAgICAgfCBv dGhlcndpc2UgICAgIHwgZHJvcCBub24tVE0tICAgICB8IHByZWZlcmVudGlhbGx5ICB8DQogICB8 ICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgfCBtYXJrZWQgcGFja2V0cy0gIHwgZHJvcCBU TS1tYXJrZWQgIHwNCiAgIHwgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICB8IG92ZXItdGVy bWluYXRpb24gfCBwYWNrZXRzIC0gICAgICAgfA0KICAgfCAgICAgICAgICAgICAgfCAgICAgICAg ICAgICAgIHwgb3RoZXJ3aXNlLCBlc3BlLSB8b3Zlci10ZXJtaW5hdGlvbiB8DQogICB8ICAgICAg ICAgICAgICB8ICAgICAgICAgICAgICAgfCBjaWFsbHkgdW5kZXIgICAgIHwgb3RoZXJ3aXNlICAg ICAgIHwNCiAgIHwgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICB8IGhpZ2ggbG9zcyAgICAg ICAgfCAgICAgICAgICAgICAgICAgfA0KICAgfC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLSAtLS0tLS0tLS0tLS0tLS0tLS18DQogICB8RWdyZXNzICAgICAg ICB8IG1lYXN1cmUgcmF0ZXMgfCBvYnNlcnZlIHBhY2tldCAgIHwgbWVhc3VyZSByYXRlcyBvZnwN CiAgIHxGdW5jdGlvbiAgICAgIHwgb2YgbWFya2VkIGFuZCB8IG1hcmtpbmdzLCBzZW5kICAgfCBB TS9UTSBtYXJrZWQgYW5kfA0KICAgfCAgICAgICAgICAgICAgfCB1bm1hcmtlZCBwa3RzLHwgY29u dHJvbCBtZXNzYWdlcyB8IHVubWFya2VkIHBhY2tldHN8DQogICB8ICAgICAgICAgICAgICB8IGNv bXB1dGUgQ0xFLCAgfCB0byBjb250cm9sIGFkbWlzLXwgY29tcHV0ZSBDTEUsc2VuZHwNCiAgIHwg ICAgICAgICAgICAgIHwgc2VuZCBib3RoIHRvICB8IHNpb24gYXQgaW5ncmVzczsgfCBDTEUgYW5k IHRoZSByYXRlfA0KICAgfCAgICAgICAgICAgICAgfCBpbmdyZXNzICAgICAgIHwgc2VuZCBJRHMg b2YgVE0tICB8IG9mIHVubWFya2VkIHBrdHN8DQogICB8ICAgICAgICAgICAgICB8ICAgICAgICAg ICAgICAgfCBtYXJrZWQgcGFja2V0cyB0b3wgdG8gaW5ncmVzcyAgICAgIHwNCiAgIHwgICAgICAg ICAgICAgIHwgICAgICAgICAgICAgICB8IGluZ3Jlc3MgICAgICAgICAgfCAgICAgICAgICAgICAg ICAgfA0KICAgfC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS18DQogICB8SW5ncmVzcyAgICAgICB8Y29tcHV0ZSAgc3VzLSAg fCB0ZXJtaW5hdGUgdGhvc2UgIHwgbWVhc3VyZSBzZW5kaW5nIHwNCiAgIHxUZXJtaW5hdGlvbiAg IHx0YWluYWJsZSByYXRlOyB8IGZsb3dzIHdob3NlIGlkcyAgfCByYXRlOyBjb21wdXRlICAgfA0K ICAgfGZ1bmN0aW9uICAgICAgfG1lYXN1cmUgc2VuZGluZ3wgd2VyZSBzaWduYWxsZWQgICB8IHJh dGUgdG8gdGVybWktICB8DQogICB8ICAgICAgICAgICAgICB8cmF0ZTsgY29tcHV0ZSAgfCBieSB0 aGUgZWdyZXNzICAgIHwgbmF0ZSwgc2VsZWN0ICAgIHwNCiAgIHwgICAgICAgICAgICAgIHxyYXRl IHRvICAgICAgICB8ICAgICAgICAgICAgICAgICAgfCBmbG93cyB0byAgICAgICAgfA0KICAgfCAg ICAgICAgICAgICAgfHRlcm1pbmF0ZSwgc2UtIHwgICAgICAgICAgICAgICAgICB8IHRlcm1pbmF0 ZSAgICAgICB8DQogICB8ICAgICAgICAgICAgICB8bGVjdCBmbG93cyB0byAgfCAgICAgICAgICAg ICAgICAgIHwgICAgICAgICAgICAgICAgIHwNCiAgIHwgICAgICAgICAgICAgIHx0ZXJtaW5hdGUg ICAgICB8ICAgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAgfA0KICAgfC0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS18DQogICB8SW5ncmVzcyAgICAgICB8c3RvcCBhZG1pdHRpbmcgfCBzdG9wIGFkbWl0dGluZyAg IHxzdG9wIGFkbWl0dGluZyAgIHwNCiAgIHxBZG1pc3Npb24gICAgIHx3aGVuIENMRSAgICAgICB8 IHdoZW4gbm90aWZpZWQgYnkgfHdoZW4gQ0xFIGV4Y2VlZHMgfA0KICAgfEZ1bmN0aW9uICAgICAg fGV4Y2VlZHMgY29uZmktIHwgZWdyZXNzOyByZXN1bWUgICB8Y29uZmlndXJlZCB2YWx1ZTt8DQog ICB8ICAgICAgICAgICAgICB8Z3VyZWQgdmFsdWU7ICAgfCBhZnRlciBhIHRpbWVvdXQgIHxyZXN0 YXJ0IGFkbWl0dGluZ3wNCiAgIHwgICAgICAgICAgICAgIHxyZXN0YXJ0IGFkbWl0LSB8IG9yIHdo ZW4gbm90aWZpZWQgfHdoZW4gQ0xFIHJlZHVjZXMgfA0KICAgfCAgICAgICAgICAgICAgfHRpbmcg d2hlbiBDTEUgIHwgYnkgaW5ncmVzcyAgICAgICB8YmVsb3cgY29uZmlndXJlZCB8DQoNCg0KDQpD aGFybnksIGV0IGFsLiAgICAgICAgICAgIEV4cGlyZXMgTWF5IDE0LCAyMDA4ICAgICAgICAgICAg ICAgICBbUGFnZSAxNF0NCgwNCkludGVybmV0LURyYWZ0ICAgICAgICAgICAgICBDb21wYXJpc29u IERyYWZ0ICAgICAgICAgICAgICAgTm92ZW1iZXIgMjAwNw0KDQoNCiAgIHwgICAgICAgICAgICAg IHxmYWxscyBiZWxvdyAgICB8ICAgICAgICAgICAgICAgICAgfHZhbHVlICAgICAgICAgICAgfA0K ICAgfCAgICAgICAgICAgICAgfGNvbmZpZ3VyZWQgICAgIHwgICAgICAgICAgICAgICAgICB8ICAg ICAgICAgICAgICAgICB8DQogICB8ICAgICAgICAgICAgICB8dmFsdWUgICAgICAgICAgfCAgICAg ICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgIHwNCiAgIHwtLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tfA0KICAgfHJh dGUgICAgICAgICAgfCByZXF1aXJlZCAgICAgIHwgICAgb3B0aW9uYWwgICAgICB8ICAgcmVxdWly ZWQgICAgICB8DQogICB8bWVhc3VyZW1lbnQgICB8IDEgYXQgaW5ncmVzcyAgfCAgIDEgYXQgZWdy ZXNzICAgIHwgICAxIGF0IGluZ3Jlc3MgIHwNCiAgIHxhdCBib3VuZGFyeSAgIHwgMiBhdCBlZ3Jl c3MgICB8ICAgICAgICAgICAgICAgICAgfCAgIDIgYXQgZWdyZXNzICAgfA0KICAgfG5vZGVzICAg ICAgICAgfCAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAg ICB8DQogICB8LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLXwNCiAgIHxJbmZvcm1hdGlvbiAgIHxDTEUsIHN1c3RhaW5hLSB8 Y29udHJvbCBtZXNzYWdlcyAgfENMRSwgc3VzdGFpbmFibGUgfA0KICAgfHNpZ25hbGxlZCAgICAg fGJsZShhZG1pc3Npb24pIHx0byBzdG9wICYgcG9zc2libHl8KHRlcm1pbmF0aW9uKSAgICB8DQog ICB8ZnJvbSBlZ3Jlc3MgICB8cmF0ZSAgICAgICAgICAgfHJlc3RhcnQgYWRtaXNzaW9uO3xyYXRl ICAgICAgICAgICAgIHwNCiAgIHx0byBpbmdyZXNzICAgIHwgICAgICAgICAgICAgICB8aWRzIG9m IGZsb3dzIHdpdGggfCAgICAgICAgICAgICAgICAgfA0KICAgfCAgICAgICAgICAgICAgfCAgICAg ICAgICAgICAgIHwgVE0tbWFya2VkIHBhY2tldHN8ICAgICAgICAgICAgICAgICB8DQogICB8LS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLXwNCiAgIHxFQ01QIHN1cHBvcnQgIHxubzsgICBvbmx5ICAgICB8ICAgICB5ZXMgICAg ICAgICAgfCBubzsgIGJ1dCBmdWxsICAgfA0KICAgfGZvciAgICAgICAgICAgfHBhcnRpYWwgc3Vw cG9ydHwgICAgICAgICAgICAgICAgICB8IHN1cHBvcnQgd2l0aCAgICB8DQogICB8VGVybWluYXRp b24gICB8d2l0aCBhZGRpdGlvbmFsfCAgICAgICAgICAgICAgICAgIHwgYWRkaXRpb25hbCAgICAg IHwNCiAgIHwgICAgICAgICAgICAgIHwgY29tcGxleGl0eSBhdCB8ICAgICAgICAgICAgICAgICAg fCBjb21wbGV4aXR5IGF0ICAgfA0KICAgfCAgICAgICAgICAgICAgfCB0aGUgZWRnZSArICAgIHwg ICAgICAgICAgICAgICAgICB8IHRoZSBlZGdlICsgICAgICB8DQogICB8ICAgICAgICAgICAgICB8 IHNpZ25hbGluZyBmbG93fCAgICAgICAgICAgICAgICAgIHwgcGx1cyBzaWduYWxsaW5nIHwNCiAg IHwgICAgICAgICAgICAgIHwgZmxvdyBJRHMgZnJvbSB8ICAgICAgICAgICAgICAgICAgfCBmbG93 IElEcyBmcm9tICAgfA0KICAgfCAgICAgICAgICAgICAgfCBlZ3Jlc3MgdG8gICAgIHwgICAgICAg ICAgICAgICAgICB8IGVncmVzcyB0byAgICAgICB8DQogICB8ICAgICAgICAgICAgICB8IGluZ3Jl c3MgICAgICAgfCAgICAgICAgICAgICAgICAgIHwgaW5ncmVzcyAgICAgICAgIHwNCiAgIHwtLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tfA0KICAgfEVDTVAgc3VwcG9ydCAgfG5vIHcvb3V0IHByb2Jlc3wgbm8gdy9vdXQgcHJv YmluZyB8IG5vIHcvb3V0IHByb2Jpbmd8DQogICB8Zm9yIEFkbWlzc2lvbiB8eWVzIHdpdGggcHJv YmVzfCB5ZXMgd2l0aCBwcm9iaW5nIHwgeWVzIHdpdGggcHJvYmluZ3wNCiAgIHwgICAgICAgICAg ICAgIHxidXQgbmVlZHMgbWFueSB8KG5lZWRzIG9uZSBwcm9iZSwgfChuZWVkcyBvbmUgcHJvYmUs fA0KICAgfCAgICAgICAgICAgICAgfHByb2JlczsgdXNlIG9mIHxjYW4gdXNlIFJTVlAgYXMgICB8 IGNhbiB1c2UgUlNWUCAgICB8DQogICB8ICAgICAgICAgICAgICB8UlNWUCBhcyBwcm9iZXMgfHBy b2JlKSAgICAgICAgICAgIHwgYXMgcHJvYmUpICAgICAgIHwNCiAgIHwgICAgICAgICAgICAgIHxu b3QgdW5kZXJzdG9vZCB8ICAgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAgfA0KICAg fC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS18DQogICB8TmV0d29yay13aWRlICB8ICAgICAgeWVzICAgICAgfCAgICAgICAg bm8gICAgICAgIHwgICAgICBubyAgICAgICAgIHwNCiAgIHxwYXJhbWV0ZXIgICAgIHwgIChvbmUg Z2xvYmFsICB8ICAgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAgfA0KICAgfGNvbmZp Z3VyYXRpb24gfCAgIHBhcmFtZXRlciAgIHwgICAgICAgICAgICAgICAgICB8ICAgICAgICAgICAg ICAgICB8DQogICB8Y29vcmRpbmF0aW9uICB8ICBhdCB0aGUgZWRnZXMpfCAgICAgICAgICAgICAg ICAgIHwgICAgICAgICAgICAgICAgIHwNCiAgIHwtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tfA0KICAgfCBTdXBwb3J0IGZv ciAgfCAgICAgIHllcyAgICAgIHwgICAgICB5ZXMgICAgICAgICB8ICAgICAgeWVzICAgICAgICB8 DQogICB8IHJlc2lsaWVuY3kgICB8IGJ1dCB3ZWFrZXIgICAgfCAgICAgICAgICAgICAgICAgIHwg ICAgICAgICAgICAgICAgIHwNCiAgIHwgcGxhbm5pbmcgICAgIHwgZ3VhcmFudGVlcyAgICB8ICAg ICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAgfA0KICAgfCAgICAgICAgICAgICAgfCB1 bmRlciBwbGFubmVkIHwgICAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgICB8DQogICB8 ICAgICAgICAgICAgICB8IGZhaWx1cmVzIGZvciAgfCAgICAgICAgICAgICAgICAgIHwgICAgICAg ICAgICAgICAgIHwNCiAgIHwgICAgICAgICAgICAgIHwgdHJhZmZpYyBleGNlZS18ICAgICAgICAg ICAgICAgICAgfCAgICAgICAgICAgICAgICAgfA0KICAgfCAgICAgICAgICAgICAgfCBkaW5nIHBs YW5uZWQgIHwgICAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgICB8DQogICB8ICAgICAg ICAgICAgICB8dHJhZmZpYyBtYXRyaXg7fCAgICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAg ICAgIHwNCiAgIHwgICAgICAgICAgICAgIHxpZiBhbGwgYWRtaXR0ZWR8ICAgICAgICAgICAgICAg ICAgfCAgICAgICAgICAgICAgICAgfA0KICAgfCAgICAgICAgICAgICAgfHRyYWZmaWMgaXMgdG8g IHwgICAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgICB8DQogICB8ICAgICAgICAgICAg ICB8YmUgcHJvdGVjdGVkLCAgfCAgICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgIHwN Cg0KDQoNCkNoYXJueSwgZXQgYWwuICAgICAgICAgICAgRXhwaXJlcyBNYXkgMTQsIDIwMDggICAg ICAgICAgICAgICAgIFtQYWdlIDE1XQ0KDA0KSW50ZXJuZXQtRHJhZnQgICAgICAgICAgICAgIENv bXBhcmlzb24gRHJhZnQgICAgICAgICAgICAgICBOb3ZlbWJlciAyMDA3DQoNCg0KICAgfCAgICAg ICAgICAgICAgfFNNIGNhbiBhZG1pdCAgIHwgICAgICAgICAgICAgICAgICB8ICAgICAgICAgICAg ICAgICB8DQogICB8ICAgICAgICAgICAgICB8bGVzcyB0cmFmZmljICAgfCAgICAgICAgICAgICAg ICAgIHwgICAgICAgICAgICAgICAgIHwNCiAgIHwgICAgICAgICAgICAgIHx0aGFuIENMIG9yIDNT TSB8ICAgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAgfA0KICAgfC0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS18 DQoNCg0KICAgKFRhYmxlIDguMS4gIEZ1bmN0aW9uYWwgQ29tcGFyaXNvbiBvZiBDTCwgU00gYW5k IDNTTSkNCg0KICAgRmluYWxseSwgYSBmZXcgd29yZHMgYXJlIGR1ZSBvbiByZWxhdGl2ZSBjb21w bGV4aXRpZXMgb2YgdGhlIHRocmVlDQogICBzY2hlbWVzLiAgSXQgc2hvdWxkIGJlIGNsZWFyIGZy b20gdGhlIGFib3ZlIHRhYmxlIHRoYXQgcmVsYXRpdmUNCiAgIGNvbXBsZXhpdHkgaGFzIGEgbnVt YmVyIG9mIGRpbWVuc2lvbnMuICBTcGVjaWZpY2FsbHk6DQoNCiAgIG8gIENvbXBsZXhpdHkgb2Yg dGhlIGZvcndhcmRpbmcgcGF0aC4gIEZyb20gdGhhdCBzdGFuZHBvaW50IFNNDQogICAgICBhcHBl YXJzIHRvIGJlIHRoZSBzaW1wbGVzdCwgYXMgaXQgcmVxdWlyZXMgb25seSBhIHNpbmdsZSBtZXRl cmluZw0KICAgICAgbWVjaGFuaXNtLCBDTCBhcHBlYXJzIHRvIGNvbWUgbmV4dCwgY2xvc2VseSBm b2xsb3dlZCBieSAzU00uDQoNCiAgIG8gIENvbXBsZXhpdHkgb2YgY29uZGl0aW9uYWwgbWV0ZXJp bmcgKGkuZSBjaGVja2luZyB3aGV0aGVyIGENCiAgICAgIHBhcnRpY3VsYXJseSBtYXJrZWQgcGFj a2V0IG5lZWRzIHRvIGJlIG1ldGVyZWQgKS4gIEZyb20gdGhhdA0KICAgICAgc3RhbmRwb2ludCwg YWxsIHRocmVlIGFwcHJvYWNoZXMgYXBwZWFyIHRvIGJlIGNvbXBhcmFibGUuICAoTm90ZQ0KICAg ICAgdGhhdCB3aGlsZSBTTSBhZG1pc3Npb24gZnVuY3Rpb24gaXMgbW9yZSBjb21wbGV4IGZyb20g dGhpcyBwb2ludA0KICAgICAgb2YgdmlldyB0aGFuIGFkbWlzc2lvbiBmdW5jdGlvbnMgb2YgQ0wg YW5kIDNTTSwgdGhlIGFkZGl0aW9uYWwNCiAgICAgIGNvbXBsZXhpdHkgb2Ygbm90IG1ldGVyaW5n IEFNLW1hcmtlZCBwYWNrZXRzIGhhcyB0byBiZSBiYWxhbmNlZA0KICAgICAgYWdhaW5zdCBzaW1p bGFyIGNvbXBsZXhpdHkgb2YgdGhlIHRlcm1pbmF0aW9uIGZ1bmN0aW9ucyBvZiBDTCBhbmQNCiAg ICAgIDNTTSkuDQoNCiAgIG8gIENvbXBsZXhpdHkgb2YgaW1wbGVtZW50aW5nIHByZWZlcmVudGlh bCBwYWNrZXQgbG9zcy4gIEZyb20gdGhhdA0KICAgICAgc3RhbmRwb2ludCBhbGwgdGhyZWUgYXBw cm9hY2hlcyBhcHBlYXIgY29tcGFyYWJsZSwgd2hlbiBib3RoDQogICAgICBhZG1pc3Npb24gYW5k IHRlcm1pbmF0aW9uIGZ1bmN0aW9ucyBhcmUgY29uc2lkZXJlZC4NCg0KICAgbyAgQ29tcGxleGl0 eSBvZiBib3VuZGFyeSBiZWhhdmlvcnMuICBGcm9tIHRoYXQgc3RhbmRwb2ludCAzU00NCiAgICAg IGFwcGVhcnMgdG8gYmUgdGhlIGxlYXN0IGNvbXBsZXgsIENMIGNvbWluZyBuZXh0IGFuZCBTTSBm b2xsb3dpbmcNCiAgICAgIGNsb3NlbHkuDQoNCiAgIG8gIENvbXBsZXhpdHkgb2Ygc3VwcG9ydCBm b3IgRUNNUC4gIEZyb20gdGhhdCBzdGFuZHBvaW50IDNTTSByZXF1aXJlcw0KICAgICAgbm8gYWRk aXRpb25hbCBmdW5jdGlvbmFsaXR5LCB3aGlsZSBDTCBhbmQgU00gcmVxdWlyZSBleHRyYQ0KICAg ICAgY29tcGxleGl0eSBhdCB0aGUgYm91bmRhcnkgbm9kZXMgYW5kIGV4dHJhIHNpZ25hbGluZyBp bmZvcm1hdGlvbg0KICAgICAgZXhjaGFuZ2UgYmV0d2VlbiB0aGUgZWdyZXNzIGFuZCB0aGUgaW5n cmVzcyAoYW5kIGluIGFkZGl0aW9uIFNNDQogICAgICBoYXMgb25seSBwYXJ0aWFsIHN1cHBvcnQg LSBzZWUgc2VjdGlvbiA2IGFuZCB0YWJsZSA4LjEgZm9yIGl0cw0KICAgICAgbGltaXRhdGlvbnMp Lg0KDQogICBvICBHbG9iYWwgY29uZmlndXJhdGlvbiBjb21wbGV4aXR5LiAgRnJvbSB0aGF0IHN0 YW5kcG9pbnQgQ2wgYW5kIDNTTQ0KICAgICAgY29tZSBmaXJzdCwgd2l0aCBTTSBiZWluZyB0aGUg bW9yZSBjb21wbGV4IGFzIHRoZSBvbmx5IG9uZQ0KICAgICAgcmVxdWlyaW5nIGdsb2JhbCBwYXJh bWV0ZXIgc2V0dGluZy4NCg0KICAgVGhlc2UgY29tcGFyaXNvbnMgYXJlIHZlcnkgY3J1ZGUgYW5k IGFyZSBvbmx5IGludGVuZGVkIHRvIHJvdWdobHkNCiAgIHN1bW1hcml6ZSB0aGUgZGV0YWlsZWQg ZGlmZmVyZW5jZXMgZGVzY3JpYmVkIGluIFRhYmxlIDguMS4NCg0KDQoNCg0KDQpDaGFybnksIGV0 IGFsLiAgICAgICAgICAgIEV4cGlyZXMgTWF5IDE0LCAyMDA4ICAgICAgICAgICAgICAgICBbUGFn ZSAxNl0NCgwNCkludGVybmV0LURyYWZ0ICAgICAgICAgICAgICBDb21wYXJpc29uIERyYWZ0ICAg ICAgICAgICAgICAgTm92ZW1iZXIgMjAwNw0KDQoNCjEwLiAgT3RoZXIgQ29tcGFyaXNvbiBDcml0 ZXJpYQ0KDQogICBUaGlzIHNlY3Rpb24gZGlzY3Vzc2VzIGEgbnVtYmVyIG9mIG90aGVyIGNvbXBh cmlzb24gY3JpdGVyaWEgdGhhdA0KICAgaGF2ZSBub3QgYmVlbiBzdHVkaWVkIGluIGRldGFpbCBv ciBhcmUgY3VycmVudGx5IHVuZGVyIGNvbnNpZGVyYXRpb24NCg0KICAgMS4gIENvLWV4aXN0ZW5j ZSB3aXRoIFJGQzMxNjggKHdvcmsgaW4gcHJvZ3Jlc3MpOw0KDQogICAyLiAgTXVsdGljYXN0IHN1 cHBvcnQgKFRCRCwgTk9URTogdGhpcyBpcyBvdXQtb2Ytc2NvcGUgZm9yIHRoZQ0KICAgICAgIGN1 cnJlbnQgY2hhcnRlcikNCg0KICAgMy4gIEZ1dHVyZSBleHRlbnNpb25zIHRvIG11bHRpcGxlIGRv bWFpbnMNCiAgICAgICAoW0ktRC5icmlzY29lLXRzdndnLXJlLWVjbi1ib3JkZXItY2hlYXRdIGZv ciBDTCwgbm90IGNsYXJpdHkgb24NCiAgICAgICBvdGhlciBhcHByb2FjaGVzOyBOT1RFOiB0aGlz IGlzIG91dC1vZi1zY29wZSBvZiB0aGUgY3VycmVudA0KICAgICAgIGNoYXJ0ZXIpDQoNCiAgIDQu ICBFeHRlbnNpYmlsaXR5IHRvIGhvc3QtaW5pdGlhdGVkIFBDTiAoM1NNIGRlc2lnbmVkIHRvIHN1 cHBvcnQNCiAgICAgICBob3N0LWluaXRpYXRlZCBQQ04gYnV0IHBlcmZvcm1hbmNlIGFuYWx5c2lz IGlzIG9uZ29pbmc7IE5PVEU6DQogICAgICAgdGhpcyBpcyBvdXQtb2Ytc2NvcGUgZm9yIHRoZSBj dXJyZW50IGNoYXJ0ZXIuDQoNCiAgIDUuICBFeHRlbnNpYmlsaXR5IHRvIHJhdGUtYWRhcHRpdmUg dHJhZmZpYyAoVEJEOyBOT1RFOiB0aGlzIGlzIG91dC0NCiAgICAgICBvZi1zY29wZSBvZiB0aGUg Y3VycmVudCBjaGFydGVyKQ0KDQogICA2LiAgU3VwcG9ydCBvZiBtdWx0aXBsZSBwcmVjZWRlbmNl IGxldmVscyAoVEJEOyBOT1RFOiB0aGlzIGlzIG91dC1vZi0NCiAgICAgICBzY29wZSBvZiB0aGUg Y3VycmVudCBjaGFydGVyKQ0KDQogICA3LiAgUGVyZm9ybWFuY2UgY29tcGFyaXNvbiAob25nb2lu Zywgc2VlIG5leHQgU3Vic2VjdGlvbikuDQoNCjEwLjEuICBQZXJmb3JtYW5jZSBDb21wYXJpc29u DQoNCiAgIEFzIG1lbnRpb25lZCBpbiB0aGUgSW50cm9kdWN0aW9uLCBhdCB0aGUgdGltZSBvZiB3 cml0aW5nIHRoaXMNCiAgIGRvY3VtZW50IHNldmVyYWwgcGVyZm9ybWFuY2Ugc3R1ZGllcyBoYXZl IGJlZW4gcmVwb3J0ZWQgaW4NCiAgIFtJLUQuemhhbmctcGNuLXBlcmZvcm1hbmNlLWV2YWx1YXRp b25dLA0KICAgW0ktRC5jaGFybnktcGNuLXNpbmdsZS1tYXJraW5nXVtJLUQuYmFiaWFyei1wY24t ZXhwbGljaXQtbWFya2luZ10sDQogICBhbmQgW1RSNDM3XS4gIFdoaWxlIHRoZSBmaXJzdCB0d28g c3R1ZGllcyBoYXZlIGF0dGVtcHRlZCBhIGNhcmVmdWwNCiAgIHNpZGUtYnktc2lkZSBjb21wYXJp c29ucyBvZiBDTCBhbmQgU00sIHRoZSBzZXQgb2YgZXhwZXJpbWVudHMNCiAgIHJlcG9ydGVkIGlz IGxlc3MgZXh0ZW5zaXZlLCBhbmQgYSBudW1iZXIgb2YgZGlmZmVyZW5jZXMgZXhpc3RpbmcgaW4N CiAgIHRoZSBzaW11bGF0aW9uIG1vZGVscyBhbmQgZXhwZXJpbWVudCBzZXR1cCBtYWtlIGEgc2lk ZS1ieS1zaWRlDQogICBhcHBsZXMtdG8tYXBwbGVzIGNvbXBhcmlzb24gb2YgdGhlIDMgc2NoZW1l cyBkaWZmaWN1bHQuICBJbiB0aGlzIHdlDQogICB3aWxsIGF0dGVtcHQgdG8gc3VtbWFyaXplIHBl cmZvcm1hbmNlIGV2YWx1YXRpb24gY3JpdGVyaWEgdGhhdCBjb3VsZA0KICAgYmUgdXNlZCBmb3Ig Y29tcGFyaXNvbiBvZiB0aGUgZGlmZmVyZW50IGFwcHJvYWNoZXMsIGFzIHdlbGwgYXMNCiAgIHBy b3ZpZGUgaGlnaCBsZXZlbCBjb25jbHVzaW9ucyBvbiBzb21lIG9mIHRoZW0sIHdoZXJlIGF2YWls YWJsZQ0KICAgc3R1ZGllcyBhbGxvdyB0aG9zZSBjb25jbHVzaW9ucy4NCg0KICAgV2UgcmVmZXIg dGhlIHJlYWRlciB0byBbSS1ELnpoYW5nLXBjbi1wZXJmb3JtYW5jZS1ldmFsdWF0aW9uXSwNCiAg IFtJLUQuY2hhcm55LXBjbi1zaW5nbGUtbWFya2luZ10sIFtJLUQuYmFiaWFyei1wY24tZXhwbGlj aXQtbWFya2luZ10sDQogICBhbmQgW1RSNDM3XSBmb3IgZGV0YWlscyB0aGF0IGNvdWxkIG5vdCBi ZSBjYXB0dXJlZCB3aXRoaW4gdGhlIGZvcm1hdA0KICAgb2YgdGhlIHRhYmxlIGJlbG93Lg0KDQoN Cg0KDQpDaGFybnksIGV0IGFsLiAgICAgICAgICAgIEV4cGlyZXMgTWF5IDE0LCAyMDA4ICAgICAg ICAgICAgICAgICBbUGFnZSAxN10NCgwNCkludGVybmV0LURyYWZ0ICAgICAgICAgICAgICBDb21w YXJpc29uIERyYWZ0ICAgICAgICAgICAgICAgTm92ZW1iZXIgMjAwNw0KDQoNCiAgIERJU0NMQUlN RVI6IHN0YXRlbWVudHMgaW4gdGhlIHRhYmxlIGJlbG93IHNob3VsZCBiZSB1bmRlcnN0b29kIG9u bHkNCiAgIGluIHRlcm1zIHJlbGF0aXZlIHRvIHRoZSB0aHJlZSBjb25zaWRlcmVkIGFsZ29yaXRo bXMgYW5kIHdpdGggcmVzcGVjdA0KICAgdG8gc3BlY2lmaWMgZXhwZXJpbWVudHMgcGVyZm9ybWVk OyB0aGV5IGFuZCBhcmUgaW50ZW5kZWQgZm9yIGNydWRlDQogICBxdWFsaXRhdGl2ZSBjb21wYXJp c29uIG9ubHkgYmFzZWQgb24gKG9uZ29pbmcpIHNpbXVsYXRpb24gc3R1ZGllcyBhcw0KICAgb2Yg dGhlIHRpbWUgb2Ygd3JpdGluZyBvZiB0aGlzIGRvY3VtZW50IC4gIFF1YW50aWZpY2F0aW9uIG9m IHRoZXNlDQogICBzdGF0ZW1lbnRzIGNhbiBiZSBmb3VuZCBpbiB0aGUgY29ycmVzcG9uZGluZyBw ZXJmb3JtYW5jZSBzdHVkaWVzDQogICByZWZlcmVuY2VkIGVhcmxpZXIgaW4gdGhpcyBzZWN0aW9u Lg0KDQogICAocHJlYW1ibGUpDQogICAgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQogICB8Q29tcGFyaXNvbiAgIHwgICAg ICBTTSAgICAgIHwgICAgICAgM1NNICAgICAgICB8ICAgICAgICBDTCAgICAgICB8DQogICB8Q3Jp dGVyaWEgICAgIHwgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICB8ICAgICAgICAgICAg ICAgICB8DQogICB8LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS18DQogICB8U2Vuc2l0aXZpdHkgIHwgcG9vciBwZXJmb3ItIHwg Z29vZCBwZXJmb3JtYW5jZSB8IHBvb3IgcGVyZm9yLSAgICB8DQogICB8dG8gbG93ICAgICAgIHwg bWFuY2UgYXQgbG93IHwgYXQgbG93IGFnZ3JlZ2EtICB8IG1hbmNlIGF0IGxvdyAgICB8DQogICB8 Ym90dGxlbmVjayAgIHwgYm90dGxlbmVjayAgIHwgdGlvbiByZXBvcnRlZCAgICB8IGJvdHRsZW5l Y2sgICAgICB8DQogICB8YWdncmVnYXRpb24gIHwgYWdncmVnYXRpb24gIHwgKGV2YWx1YXRpb24g ICAgICB8IGFnZ3JlZ2F0aW9uICAgICB8DQogICB8ICAgICAgICAgICAgIHwgICAgICAgICAgICAg IHwgIG9uZ29pbmcpICAgICAgICB8ICAgICAgICAgICAgICAgICB8DQogICB8LS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS18DQog ICB8U2Vuc2l0aXZpdHkgIHwgcGVyZm9ybWFuY2UgIHwgZ29vZCBpbiByZXBvcnRlZCB8IGFkbWlz c2lvbiAgICAgICB8DQogICB8dG8gbG93ICAgICAgIHxkZWdyYWRhdGlvbiAgIHwgZXhwZXJpbWVu dHM7ICAgICB8IGluc2Vuc2l0aXZlOyAgICB8DQogICB8aW5ncmVzcy0gICAgIHxmb3IgYm90aCAg ICAgIHwgdHJhZmZpYyBhbmQgICAgICB8IHRlcm1pbmF0aW9uICAgICB8DQogICB8ZWdyZXNzICAg ICAgIHx0ZXJtaW5hdGlvbiAgIHwgdG9wb2xvZ3kgICAgICAgICB8IHNlbnNpdGl2ZSBmb3IgICB8 DQogICB8YWdncmVnYXRpb24gIHxhbmQgYWRtaXNzaW9uIHwgc2Vuc2l0aXZpdHkgc3R1ZHl8IHNv bWUgdHJhZmZpYyAgICB8DQogICB8ICAgICAgICAgICAgIHx1bmRlciBzb21lICAgIHwgbmVlZGVk ICAgICAgICAgICB8IHR5cGVzICAgICAgICAgICB8DQogICB8ICAgICAgICAgICAgIHx0cmFmZmlj IHR5cGVzIHwgICAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgICB8DQogICB8LS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS18DQogICB8U2Vuc2l0aXZpdHkgIHwgYSByYW5nZSBvZiAgIHxldmFsdWF0aW9uIG9uZ29pbmd8 ICByZWxhdGl2ZWx5ICAgICB8DQogICB8dG8gbWFya2luZyAmIHwgcGFyYW1zIGV4aXN0IHxzbG93 LWRvd24gcGFyYW0uIFN8ICBpbnNlbnNpdGl2ZSB0byB8DQogICB8bWVhc3VyZW1lbnQgIHwgd2l0 aCBjb25zaXMtIHxoYXMgdGhlIGJpZ2dlc3QgICB8ICBtYXJraW5nIGFuZCAgICB8DQogICB8cGFy YW1ldGVycyAgIHwgdGVudCBwZXJmb3ItIHxpbXBhY3Qgb24gZmxvdyAgICB8ICBtZWFzdXJlbWVu dCAgICB8DQogICB8ICAgICAgICAgICAgIHwgbWFuY2UgYWNyb3NzIHx0ZXJtaW5hdGlvbiBzcGVl ZCB8ICBwYXJhbXMgYWNyb3NzICB8DQogICB8ICAgICAgICAgICAgIHwgYSByYW5nZSBvZiAgIHwo aXRzIGNob2ljZSBkZS0gICB8ICBhIHJhbmdlIG9mICAgICB8DQogICB8ICAgICAgICAgICAgIHwg dHJhZmZpYyB0eXBlc3xwZW5kcyBvbiB0b3BvbG9neSB8ICB0cmFmZmljIHR5cGVzICB8DQogICB8 ICAgICAgICAgICAgIHwgJiB0b3BvbG9naWVzIHxhbmQgdHJhZmZpYyByYXRlKSB8ICBhbmQgdG9w b2xvZ2llcyB8DQogICB8LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS18DQogICB8QWNjdXJhY3kgb2YgIHxnb29kIGZvciBsYXJn ZXwgICAgICBnb29kICAgICAgICB8ICAgIGdvb2QgICAgICAgICB8DQogICB8YWRtaXNzaW9uICAg IHxpbmdyZXNzLWVncmVzc3woYnV0IHNlbnNpdGl2aXR5ICB8ICAgICAgICAgICAgICAgICB8DQog ICB8YWNyb3NzIHRyYWYtIHxhZ2dyZWdhdGlvbiAgIHx0byBwYXJhbWV0ZXJzIFRCRCl8ICAgICAg ICAgICAgICAgICB8DQogICB8ZmljIHR5cGVzIGFuZHwgICAgICAgICAgICAgIHwgICAgICAgICAg ICAgICAgICB8ICAgICAgICAgICAgICAgICB8DQogICB8dG9wb2xvZ2llcyAgIHwgICAgICAgICAg ICAgIHwgICAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgICB8DQogICB8LS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS18 DQogICB8QWNjdXJhY3kgb2YgIHxvdmVyLSAgICAgICAgIHwgICAgICAgIGdvb2QgICAgICB8ICAg ICBnb29kICAgICAgICB8DQogICB8dGVybWluYXRpb24gIHx0ZXJtaW5hdGlvbiAgIHwgKGJ1dCBz ZW5zaXRpdml0eSB8ICAgICAgICAgICAgICAgICB8DQogICB8KHNpbmdsZSAgICAgIHxjb21wYXJl ZCB0byBDTHwgIHRvIHBhcmFtcyBUQkQpICB8ICAgICAgICAgICAgICAgICB8DQogICB8IGJvdHRs ZW5lY2spIHxpZiB0ZXJtaW5hdGlvbnwgICAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAg ICB8DQogICB8ICAgICAgICAgICAgIHx0cmlnZ2VyIGlzIG5vdHwgICAgICAgICAgICAgICAgICB8 ICAgICAgICAgICAgICAgICB8DQogICB8ICAgICAgICAgICAgIHxzbW9vdGhlZDsgICAgIHwgICAg ICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgICB8DQoNCg0KDQpDaGFybnksIGV0IGFsLiAg ICAgICAgICAgIEV4cGlyZXMgTWF5IDE0LCAyMDA4ICAgICAgICAgICAgICAgICBbUGFnZSAxOF0N CgwNCkludGVybmV0LURyYWZ0ICAgICAgICAgICAgICBDb21wYXJpc29uIERyYWZ0ICAgICAgICAg ICAgICAgTm92ZW1iZXIgMjAwNw0KDQoNCiAgIHwgICAgICAgICAgICAgfG90aGVyd2lzZSAgICAg fCAgICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgIHwNCiAgIHwgICAgICAgICAgICAg fHNsb3dlciByZWFjLSAgfCAgICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgIHwNCiAg IHwgICAgICAgICAgICAgfHRpb24gdGhhbiBDTCAgfCAgICAgICAgICAgICAgICAgIHwgICAgICAg ICAgICAgICAgIHwNCiAgIHwtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLXwNCiAgIHxBY2N1cmFjeSBvZiAgfCBtb3JlIG92ZXIt ICAgfCAgIG5vdCByZXBvcnRlZCAgIHwgICAgICBnb29kICAgICAgIHwNCiAgIHx0ZXJtaW5hdGlv biAgfCB0ZXJtaW5hdGlvbiAgfCAgICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgIHwN CiAgIHx3ZXQgYm90dGxlLSAgfCB0aGFuIENMICAgICAgfCAgICAgICAgICAgICAgICAgIHwgICAg ICAgICAgICAgICAgIHwNCiAgIHxuZWNrIHV0aWxlLiAgIHwgICAgICAgICAgICAgIHwgICAgICAg ICAgICAgICAgICB8ICAgICAgICAgICAgICAgICB8DQogICB8KG11bHRpLWJvdC0gIHwgICAgICAg ICAgICAgIHwgICAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgICB8DQogICB8bmVjayBj YXNlKSAgIHwgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAg ICB8DQogICB8LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS18DQogICB8UmVhY3Rpb24gdGltZXwgc2xvd2VyIHRoYW4gIHwgc2xv d2VyIHRoYW4gQ0wsICB8ICAgICAgZmFzdCAgICAgICB8DQogICB8dGltZSB0byAgICAgIHwgQ0wg d2l0aCAgICAgIHwgY29tcGFyaXNvbiB3aXRoICB8ICAgICAgICAgICAgICAgICB8DQogICB8dGVy bWluYXRpb24gIHwgc21vb3RoaW5nIG9mIHwgU00gVEJEOyBwYXJhbWV0ZXJ8ICAgICAgICAgICAg ICAgICB8DQogICB8ICAgICAgICAgICAgIHwgdGVybWluYXRpb24gIHwgc2Vuc2l0aXZpdHkgVEJE ICB8ICAgICAgICAgICAgICAgICB8DQogICB8ICAgICAgICAgICAgIHwgdHJpZ2dlciwgZmFzdHwg ICAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgICB8DQogICB8ICAgICAgICAgICAgIHwg b3RoZXJ3aXNlICAgIHwgICAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgICB8DQogICB8 ICAgICAgICAgICAgIHwgdG8gM1NNICAgICAgIHwgICAgICAgICAgICAgICAgICB8ICAgICAgICAg ICAgICAgICB8DQogICB8LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS18DQogICB8U2Vuc2l0aXZpdHkgIHxub3QgcmVwb3J0ZWQ7 IHxwcmVmZXJlbnRpYWxseSAgICB8IG5vdCByZXBvcnRlZCA7ICB8DQogICB8dG8gbGFyZ2UgYW5k IHxmbG93IHNlbGVjdGlvbnxzZWxlY3RzIGxhcmdlICAgICB8IGZsb3cgc2VsZWN0aW9uICB8DQog ICB8c21hbGwgZmxvd3MgIHxhdCBpbmdyZXNzICAgIHxmbG93czsgc2xvdyBkb3duICB8IGF0IGlu Z3Jlc3MgbW9yZSB8DQogICB8KHRlcm1pbmF0aW9uKXxtb3JlIGNvbXBsaS0gIHxwYXJhbWV0ZXIg aGFyZCB0byB8IGNvbXBsaWNhdGVkIChvciB8DQogICB8ICAgICAgICAgICAgIHxjYXRlZCAob3Ig bGVzc3xzZWxlY3QgZm9yIG1peCBvZiB8IGxlc3MgYWNjdXJhdGUpICB8DQogICB8ICAgICAgICAg ICAgIHxhY2N1cmF0ZSB3aXRoIHx0cmFmZmljIHJhdGVzICAgICB8IHdpdGggd2lkZWx5IGRpZi18 DQogICB8ICAgICAgICAgICAgIHx3aWRlbHkgZGlmZmVyLSB8KG92ZXItdGVybWluYXRpb24gfCBm ZXJyZW50IHJhdGVzOyAgfA0KICAgfCAgICAgICAgICAgICB8cmVudCByYXRlczsgICB8b3Igc2xv d2VyIHJlYWN0LSAgfCAoc3R1ZHkgb25nb2luZykgfA0KICAgfCAgICAgICAgICAgICB8c3R1ZHkg b25nb2luZyB8dGlvbiBpZiBzbG93ZG93biAgfCAgICAgICAgICAgICAgICAgfA0KICAgfCAgICAg ICAgICAgICB8ICAgICAgICAgICAgICB8cGFyYW1ldGVyIGRvZXMgICAgfCAgICAgICAgICAgICAg ICAgfA0KICAgfCAgICAgICAgICAgICB8ICAgICAgICAgICAgICB8bm90IHJlZmxlY3QgICAgICAg fCAgICAgICAgICAgICAgICAgfA0KICAgfCAgICAgICAgICAgICB8ICAgICAgICAgICAgICB8YXZl cmFnZSByYXRlOyAgICAgfCAgICAgICAgICAgICAgICAgfA0KICAgfCAgICAgICAgICAgICB8ICAg ICAgICAgICAgICB8ZXZhbHVhdGlvbiBvbmdvaW5nfCAgICAgICAgICAgICAgICAgfA0KICAgfC0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tfA0KICAgfEJlYXQtZG93biAgICB8IG1vcmUgYmVhdGRvd258IG5vdCByZXBvcnRlZCAg ICAgfCBiZWF0LWRvd24gb2YgICAgfA0KICAgfGVmZmVjdCAgICAgICB8IG9mIGxvbmctaGF1bCB8 ICAgICAgICAgICAgICAgICAgfCBmbG93cyB0cmF2ZXJzaW5nfA0KICAgfG11bHRpLWJvdHRsZS18 IGFnZ3JlZ2F0ZXMgICB8ICAgICAgICAgICAgICAgICAgfCBtb3JlIGJvdHRsZW5lY2tzfA0KICAg fG5lY2sgY2FzZSkgICB8IHRoYW4gQ0wgICAgICB8ICAgICAgICAgICAgICAgICAgfCAgICAgICAg ICAgICAgICAgfA0KICAgIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KICAgKFRhYmxlIDkuMS4gIFN1bW1hcnkgb2YgKHVw LXRvLWRhdGUpIHBlcmZvcm1hbmNlIGNvbXBhcmlzb24uICkNCg0KDQoxMS4gIFVuaWZpZWQgRGVz Y3JpcHJpb24gb2YgTWFya2luZyBhbmQgTWV0ZXJpbmcgRnVuY3Rpb25zDQoNCiAgIEluIHRoaXMg c2VjdGlvbiB3ZSBhZGRyZXNzIHRoZSBxdWVzdGlvbiBvZiBob3cgdGhlIG1ldGVyaW5nIGFuZA0K ICAgbWFya2luZyBmdW5jdGlvbnMgb2YgdGhlIHRocmVlIGFwcHJvYWNoZXMgY2FuIGJlIGRlc2Ny aWJlZCBpbiBhDQogICB1bmlmaWVkIHdheSBzbyB0aGF0IGRpZmZlcmVudCBtYXJraW5nIGJlaGF2 aW9ycyBjb25zaWRlcmVkIGluIHRoaXMNCiAgIGRyYWZ0IGNhbiBiZSBvYnRhaW5lZCBieSBkaWZm ZXJlbnQgcGFyYW1ldHJpemF0aW9uIGNob2ljZXMuICBBDQogICBwb3RlbnRpYWwgYmVuZWZpdCBv ZiBzdWNoIHVuaWZpZWQgZGVzY3JpcHRpb24gaXMgdGhhdCBhIHNpbmdsZSBQQ04tDQoNCg0KDQpD aGFybnksIGV0IGFsLiAgICAgICAgICAgIEV4cGlyZXMgTWF5IDE0LCAyMDA4ICAgICAgICAgICAg ICAgICBbUGFnZSAxOV0NCgwNCkludGVybmV0LURyYWZ0ICAgICAgICAgICAgICBDb21wYXJpc29u IERyYWZ0ICAgICAgICAgICAgICAgTm92ZW1iZXIgMjAwNw0KDQoNCiAgIGludGVybmFsLW5vZGUg YmVoYXZpb3IgY291bGQgc3VwcG9ydCBhIHdpZGVyIHJhbmdlIG9mIGRpZmZlcmVudCBQQ04tDQog ICBib3VuZGFyeS1ub2RlIGJlaGF2aW9ycywgYW5kIGhlbmNlLCBwb3RlbnRpYWxseSwgY2FuIGJl IG9mIHVzZSB1bmRlcg0KICAgZGlmZmVyZW50IGRlcGxveW1lbnQgc2NlbmFyaW9zLiAgV2UgZGlz Y3VzcyB0aGUgZGlmZmljdWx0aWVzDQogICBhc3NvY2lhdGVkIHdpdGggc3VjaCB1bmlmaWVkIGRl c2NyaXB0aW9uIGluIHRoZSBmb2xsb3dpbmcgc2VjdGlvbiwNCiAgIHdoZXJlIHdlIGFsc28gcmFp c2UgdGhlIHF1ZXN0aW9uIG9mIHdoZXRoZXIgdGhlIGJlbmVmaXRzIG9mIHN1Y2gNCiAgIHVuaWZp ZWQgYmVoYXZpb3Igb3V0d2VpZ2ggYSBudW1iZXIgb2YgZHJhd2JhY2tzIGRpc2N1c3NlZCBpbiB0 aGF0DQogICBzZWN0aW9uLg0KDQogICBJbiBGaWd1cmUgMTAuMSwgd2Ugc2hvdyBhIHJlbGF0aXZl bHkgY29tcGFjdCB2ZXJzaW9uIG9mIHN1Y2ggdW5pZmllZA0KICAgYWxnb3JpdGhtIHVzaW5nIHRo ZSBub3Rpb24gb2YgVG9rZW4gQnVja2V0IChUQikgLiAgVGhpcyBwc2V1ZG9jb2RlDQogICBkb2Vz IG5vdCBzdXBwb3J0IHJhbXAtbWFya2luZywgYXMgdGhlIGN1cnJlbnQgcGVyZm9ybWFuY2UgZXZh bHVhdGlvbg0KICAgc3R1ZGllcyBpbmRpY2F0ZSBsaXR0bGUgYWRkaXRpb25hbCBiZW5lZml0IG9m IHJhbXAtbWFya2luZyBjb21wYXJlZA0KICAgdG8gdGhyZXNob2xkLW1hcmtpbmcuICBIb3dldmVy LCBpbiB0aGUgQXBwZW5kaXggd2UgcHJlc2VudCBhIChtb3JlDQogICBjb21wbGV4KSB2ZXJzaW9u IG9mIHRoZSBtYXJraW5nIGFsZ29yaXRobSB0aGF0IGRvZXMgc3VwcG9ydCByYW1wLQ0KICAgbWFy a2luZyBhcyB3ZWxsLg0KDQogICBXZSBub3RlIHRoYXQgdGhlIGFsZ29yaXRobSBiZWxvdyAoYXMg d2VsbCBhcyBhIG1vcmUgY29tcGxleCBjb21wbGV0ZQ0KICAgdmVyc2lvbiBpbiB0aGUgQXBwZW5k aXgpIGNhbiBiZSBmdXJ0aGVyIG9wdGltaXplZC4gIFdlIG1ha2Ugbm8NCiAgIG9wdGltaXphdGlv biBhdHRlbXB0IGluIHRoZSBpbnRlcmVzdCBvZiBjbGFyaXR5Lg0KDQogICBUaGUgVEIgaGFzIGEg cmF0ZSBvZiBUQi5yYXRlIGFuZCBhIGRlcHRoIG9mIFRCLnNpemUuICBJdCBoYXMgb25lDQogICBt YXJraW5nIHRocmVzaG9sZHMgVEIudGhyZXNob2xkIHRvIHN1cHBvcnQgdGhyZXNob2xkIG1hcmtp bmcuICBJZg0KICAgVEIudGhyZXNob2xkIGlzIGNvbmZpZ3VyZWQgdG8gYmUgZ3JlYXRlciB0aGFu IHplcm8sIHRoZW4gcGFja2V0cyBhcmUNCiAgIG1hcmtlZCBpZiB0aGUgVEIgZmlsbCBzdGF0ZSBp cyBiZWxvdyBUQi50aHJlc2hvbGQgYWZ0ZXIgdGhlaXIgYXJyaXZhbA0KICAgYW5kIHJlbW92YWwg b2YgdG9rZW5zIGZyb20gdGhlIHF1ZXVlOyBvdGhlcndpc2UgcGFja2V0cyBhcmUgbm90DQogICBt YXJrZWQuICBUaGUgImNsYXNzaWMiIHRva2VuIGJ1Y2tldCB1c2VkIGJ5IFNNIGFuZCB0aGUgdGVy bWluYXRpb24NCiAgIGZ1bmN0aW9uIG9mIENMIGlzIG9idGFpbmVkIGJ5IHNldHRpbmcgVEIudGhl c2hvbGQgPSAwLg0KDQogICBJbiBhZGRpdGlvbiwgdGhlIHNsb3dkb3duIGZhY3RvciBUQi5zbG93 ZG93biBpcyB1c2VkIHRvIGltcGxlbWVudA0KICAgbWFya2luZyBmcmVxdWVuY3kgcmVkdWN0aW9u OiBUQi5zbG93ZG93biB0b2tlbnMgYXJlIGFkZGVkIHRvIHRoZQ0KICAgdG9rZW4gYnVja2V0IHdo ZW4gYSBwYWNrZXQgaXMgbWFya2VkLiAgVGhlIG1ldGVyaW5nIGlzIGFwcGxpZWQgb25seQ0KICAg dG8gcGFja2V0cyB3aG9zZSBtYXJraW5nIGlzIHBhcnQgb2YgYSBzcGVjaWZpYyBzdWJzZXQgdGhh dCB3ZSBjYWxsDQogICBUQi5tZXRlcmVkTWFya2luZy4gIFRoZSBUQi5tYXJraW5nVHlwZSBpbmRp Y2F0ZXMgdGhlIHR5cGUgb2YNCiAgIGNvZGVwb2ludCB0aGF0IGlzIHVzZWQgZm9yIG1hcmtpbmcu ICBJbiBhZGRpdGlvbiwgdGhlIFRCIGhhcyBhDQogICB2YXJpYWJsZSBUQi5maWxsIHRoYXQgcmVj b3JkcyB0aGUgbnVtYmVyIG9mIHRva2VucyBpbiB0aGUgYnVja2V0IGFuZA0KICAgdGhlIHZhcmlh YmxlIFRCLmxhc3RVcGRhdGUgcmVjb3JkcyB0aGUgbGFzdCB1cGRhdGUgaW5zdGFudCBvZiB0aGUN CiAgIGJ1Y2tldC4gIFRoZSBnbG9iYWwgdmFyaWFibGUgIm5vdyIgaW5kaWNhdGVzIHRoZSBjdXJy ZW50IHRpbWUuDQogICBQYWNrZXRzIGhhdmUgc2l6ZSBwYWNrZXQuc2l6ZSAoaW4gYnl0ZXMpIGFu ZCBtYXJraW5nIHBhY2tldC5tYXJrLg0KICAgVGhlIGFsZ29yaXRobSBpcyBmb2xsb3dlZCBieSBh IHRhYmxlIGRlc2NyaWJpbmcgdGhlIHBhcmFtZXRlcml6YXRpb24NCiAgIG5lY2Vzc2FyeSB0byBp bXBsZW1lbnQgQWRtaXNzaW9uIGFuZCBUZXJtaW5hdGlvbiBGdW5jdGlvbnMgZm9yDQogICBkaWZm ZXJlbnQgcHJvcG9zYWxzLg0KDQoNCg0KDQoNCg0KDQoNCg0KDQpDaGFybnksIGV0IGFsLiAgICAg ICAgICAgIEV4cGlyZXMgTWF5IDE0LCAyMDA4ICAgICAgICAgICAgICAgICBbUGFnZSAyMF0NCgwN CkludGVybmV0LURyYWZ0ICAgICAgICAgICAgICBDb21wYXJpc29uIERyYWZ0ICAgICAgICAgICAg ICAgTm92ZW1iZXIgMjAwNw0KDQoNCiAgIChwcmVhbWJsZSkNClBhcmFtZXRlcnM6DQpUQi5yYXRl OiB0b2tlbiByYXRlIG9mIFRCIGluIGJ5dGVzL3MNClRCLnNpemU6IGRlcHRoIG9mIFRCIGluIHRv a2VucyAoYnl0ZXMpDQpUQi50aHJlc2hvbGQ6IG1hcmtpbmcgdGhyZXNob2xkIG9mIFRCIGluIGJ5 dGVzDQpUQi5zbG93ZG93bjogc2xvd2Rvd24gZmFjdG9yIGZvciBtYXJraW5nIGZyZXF1ZW5jeSBy ZWR1Y3Rpb24gb2YgVEIgaW4gYnl0ZXMNClRCLm1hcmtpbmdUeXBlOiBQQ04tZmlyc3QtZW5jb2Rp bmcgKCJhZG1pc3Npb24iKSAgb3INCiAgICAgICAgICAgICAgICBQQ04tc2Vjb25kLWVuY29kaW5n ICgidGVybWluYXRpb24iKS4NClRCLm1ldGVyZWRNYXJraW5nczogc2V0IG9mIHBhY2tldCBtYXJr aW5ncyB0aGF0IGFyZSBlbGlnaWJsZSBmb3IgbWV0ZXJpbmcgYnkgVEIsDQogICAgICAgICAgICAg ICAgICAgIGl0IGlzIGEgc3Vic2V0IG9mICgidW5tYXJrZWQiLCAiYWRtaXNzaW9uIiwgInRlcm1p bmF0aW9uIikuDQogICAgICAgICAgICAgICAgICAgIE5PVEU6IHRoaXMgc2V0IGRlcGVuZHMgb24g d2hldGhlciBpdCBpcyBhZG1pc3Npb24gb3INCiAgICAgICAgICAgICAgICAgICAgdGVybWluYXRp b24gdGhhdCB0aGUgZnVuY3Rpb24gYmVsb3cgaXMgdXNlZCBmb3IuDQpOT1RFOiBzZXR0aW5ncyBv ZiB0aGVzZSBwYXJhbWV0ZXJzIGZvciBkaWZmZXJlbnQgYXBwcm9hY2hlcyBhcmUgc2hvd24NCmlu IFRhYmxlcyAxMC4yIGFuZCAxMC4zDQoNCklucHV0OiBwYWNrZXQNCiAgICAvLyB0YWtlIHBhc3Nl ZCB0aW1lIHNpbmNlIGxhc3QgdXBkYXRlIGludG8gYWNjb3VudA0KICAgIFRCLmZpbGwgPSBtaW4o VEIuc2l6ZSwgVEIuZmlsbCsobm93LVRCLmxhc3RVcGRhdGUpICogVEIucmF0ZSk7DQogICAgVEIu bGFzdFVwZGF0ZSA9IG5vdzsNCg0KICAgIC8vIG1ldGVyIGFuZCBtYXJrDQogICAgSWYgKHBhY2tl dC5tYXJrIGluIFRCLm1ldGVyZWRNYXJraW5ncykNCiAgICAgICAgaWYgKFRCLmZpbGwgPCBwYWNr ZXQuc2l6ZSkNCiAgICAgICAgICAgIC8vIHJlLW1hcmtpbmcgb2YgVE0tbWFya2VkIHBhY2tldHMg dG8gQU0gbm90IGFsbG93ZWQNCiAgICAgICAgICAgIGlmICghKHBhY2tldC5tYXJrID09ICJ0ZXJt aW5hdGlvbiIgYW5kIFRCLm1hcmtpbmdUeXBlID09ICJhZG1pc3Npb24iKSkNCiAgICAgICAgICAg ICAgICBwYWNrZXQubWFyayA9IFRCLm1hcmtpbmdUeXBlOw0KICAgICAgICAgICAgZW5kaWYNCiAg ICAgICAgZWxzZQ0KICAgICAgICAgICAgVEIuZmlsbCA9IFRCLmZpbGwgLSBwYWNrZXQuc2l6ZTsN CiAgICAgICAgICAgIGlmIChUQi5maWxsIDwgVEIudGhyZXNob2xkKQ0KICAgICAgICAgICAgICAg IC8vIHJlLW1hcmtpbmcgb2YgVE0tbWFya2VkIHBhY2tldHMgdG8gQU0gbm90IGFsbG93ZWQNCiAg ICAgICAgICAgICAgICBpZiAoIShwYWNrZXQubWFyayA9PSAidGVybWluYXRpb24iIGFuZCBUQi5t YXJraW5nVHlwZSA9PSAiYWRtaXNzaW9uIikpDQogICAgICAgICAgICAgICAgICAgIHBhY2tldC5t YXJrID0gVEIubWFya2luZ1R5cGU7DQogICAgICAgICAgICAgICAgZW5kaWYNCiAgICAgICAgICAg IGVuZGlmDQogICAgICAgIGVuZGlmDQogICAgZW5kaWYNCg0KICAgIC8vIG1hcmtpbmcgZnJlcXVl bmN5IHJlZHVjdGlvbg0KICAgIGlmIChwYWNrZXQubWFyayA9PSAidGVybWluYXRpb24iKQ0KICAg ICAgICBUQi5maWxsID0gbWluKFRCLnNpemUsIFRCLmZpbGwgKyBUQi5zbG93ZG93bik7DQogICAg ZW5kaWYNCg0KT3V0cHV0OiB2b2lkDQoNCiAgIChGaWd1cmUgMTAuMS4gIFNpbXBsZSBnZW5lcmFs aXplZCBtZXRlcmluZyBhbmQgbWFya2luZyBhbGdvcml0aG0NCiAgIGJhc2VkIG9uIFRCLWZvcm11 bGF0aW9uKQ0KDQoNCg0KDQpDaGFybnksIGV0IGFsLiAgICAgICAgICAgIEV4cGlyZXMgTWF5IDE0 LCAyMDA4ICAgICAgICAgICAgICAgICBbUGFnZSAyMV0NCgwNCkludGVybmV0LURyYWZ0ICAgICAg ICAgICAgICBDb21wYXJpc29uIERyYWZ0ICAgICAgICAgICAgICAgTm92ZW1iZXIgMjAwNw0KDQoN CiAgIChwcmVhbWJsZSkNCnwtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tfA0KfCAgICAgICAgICAgICAgICB8ICBDTCBB ZG1pc3Npb24gICB8ICAgICAgU00gICAgICAgICB8IDNTTSBBZG1pc3Npb24gICB8DQp8LS0tLS0t LS0tLS0tLS0tLXwtLS0tLS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0t LS0tLS0tLXwNCnwgVEIucmF0ZSAgICAgICAgfCAgICAgIFBDTi0gICAgICAgfCAgICAgIFBDTi0g ICAgICAgfCAgICAgIFBDTi0gICAgICAgfA0KfCAgICAgICAgICAgICAgICB8IGxvd2VyLXRocmVz aG9sZCB8IGxvd2VyLXRocmVzaG9sZCB8IGxvd2VyLXRocmVzaG9sZCB8DQp8LS0tLS0tLS0tLS0t LS0tLXwtLS0tLS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tLS0tLS0t LXwNCnwgVEIuc2l6ZSAgICAgICAgfCAgIGNvbmZpZ3VyZWQgICAgfCAgIGNvbmZpZ3VyZWQgICAg fCAgIGNvbmZpZ3VyZWQgICAgfA0KfC0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS0tLS18 LS0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS0tLS18DQp8IFRCLnRocmVzaG9sZCAgIHwg ICBjb25maWd1cmVkICAgIHwgICAgICAgMCAgICAgICAgIHwgICBjb25maWd1cmVkICAgIHwNCnwt LS0tLS0tLS0tLS0tLS0tfC0tLS0tLS0tLS0tLS0tLS0tfC0tLS0tLS0tLS0tLS0tLS0tfC0tLS0t LS0tLS0tLS0tLS0tfA0KfCBUQi5zbG93ZG93biAgICB8ICAgICAgIDAgICAgICAgICB8ICAgICAg IDAgICAgICAgICB8ICAgICAgIDAgICAgICAgICB8DQp8LS0tLS0tLS0tLS0tLS0tLXwtLS0tLS0t LS0tLS0tLS0tLXwtLS0tLS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tLS0tLS0tLXwNCnwgVEIubWV0 ZXJlZC0gICAgfCAgInVubWFya2VkIiAgICAgfCAgInVubWFya2VkIiAgICAgfCAgInVubWFya2Vk IiAgICAgfA0KfCBNYXJraW5ncyAgICAgICB8ICAiYWRtaXNzaW9uIiAgICB8ICAgICAgICAgICAg ICAgICB8ICAiYWRtaXNzaW9uIiAgICB8DQp8ICAgICAgICAgICAgICAgIHwgICJ0ZXJtaW5hdGlv biIgIHwgICAgICAgICAgICAgICAgIHwgICJ0ZXJtaW5hdGlvbiIgIHwNCnwtLS0tLS0tLS0tLS0t LS0tfC0tLS0tLS0tLS0tLS0tLS0tfC0tLS0tLS0tLS0tLS0tLS0tfC0tLS0tLS0tLS0tLS0tLS0t fA0KfCBUQi5tYXJraW5nVHlwZSB8ICAiYWRtaXNzaW9uIiAgICB8ICAiYWRtaXNzaW9uIiAgICB8 ICAiYWRtaXNzaW9uIiAgICB8DQogLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLXwNCiAgIChGaWd1cmUgMTAuMiBBZG1p c3Npb24gU2V0dGluZ3MgZm9yIHRoZSBUaHJlZSBBbGdvcml0aG1zKQ0KDQogICAocHJlYW1ibGUp DQogLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLXwNCnwgICAgICAgICAgICAgICAgfCBDTCBUZXJtaW5hdGlvbiAgfCAg ICAgIFNNICAgICAgICAgfCAzU00gVGVybWluYXRpb24gfA0KfC0tLS0tLS0tLS0tLS0tLS18LS0t LS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS0tLS18DQp8IFRC LnJhdGUgICAgICAgIHwgIFBDTi11cHBlci0gICAgIHwgICAgIE4vQSAgICAgICAgIHwgUENOLXVw cGVyLSAgICAgIHwNCnwgICAgICAgICAgICAgICAgfCAgdGhyZXNob2xkICAgICAgfCAgICAgICAg ICAgICAgICAgfCB0aHJlc2hvbGQgICAgICAgfA0KfC0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0t LS0tLS0tLS18LS0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS0tLS18DQp8IFRCLnNpemUg ICAgICAgIHwgICBjb25maWd1cmVkICAgIHwgICAgIE4vQSAgICAgICAgIHwgICBjb25maWd1cmVk ICAgIHwNCnwtLS0tLS0tLS0tLS0tLS0tfC0tLS0tLS0tLS0tLS0tLS0tfC0tLS0tLS0tLS0tLS0t LS0tfC0tLS0tLS0tLS0tLS0tLS0tfA0KfCBUQi50aHJlc2hvbGQgICB8ICAgICAgIDAgICAgICAg ICB8ICAgICBOL0EgICAgICAgICB8ICAgICAgIDAgICAgICAgICB8DQp8LS0tLS0tLS0tLS0tLS0t LXwtLS0tLS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tLS0tLS0tLXwN CnwgVEIuc2xvd2Rvd24gICAgfCAgICAgICAwICAgICAgICAgfCAgICAgTi9BICAgICAgICAgfCAg IGNvbmZpZ3VyZWQgICAgfA0KfC0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS0tLS18LS0t LS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS0tLS18DQp8IFRCLm1ldGVyZWQtICAgIHwgICJ1 bm1hcmtlZCIgICAgIHwgICAgIE4vQSAgICAgICAgIHwgICJ1bm1hcmtlZCIgICAgIHwNCnwgTWFy a2luZ3MgICAgICAgfCAgImFkbWlzc2lvbiIgICAgfCAgICAgICAgICAgICAgICAgfCAgImFkbWlz c2lvbiIgICAgfA0KfCAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgICB8ICAgICAgICAg ICAgICAgICB8ICAidGVybWluYXRpb24iICB8DQp8LS0tLS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0t LS0tLS0tLXwtLS0tLS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tLS0tLS0tLXwNCnwgVEIubWFya2lu Z1R5cGUgfCAgInRlcm1pbmF0aW9uIiAgfCAgICAgTi9BICAgICAgICAgfCAgInRlcm1pbmF0aW9u IiAgfA0KIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS18DQogICAoRmlndXJlIDEwLjMgVGVybWluYXRpb24gU2V0dGlu Z3MgZm9yIHRoZSBUaHJlZSBBbGdvcml0aG1zKQ0KDQogICBGaWd1cmVzIDEwLjIgYW5kIDEwLjMg cHJvdmlkZSBwYXJhbWV0ZXIgc2V0dGluZ3MgY29ycmVzcG9uZGluZyB0bw0KICAgZGlmZmVyZW50 IG1ldGVyaW5nIGFuZCBtYXJraW5nIGZ1bmN0aW9ucy4NCg0KICAgSXQgc2hvdWxkIGJlIG5vdGVk LCB0aGF0IHdoZW4gdGhlIGFsZ29yaXRobSBkZXNjcmliZWQgYnkgdGhlDQogICBwc2V1ZG9jb2Rl IGluIEZpZ3VyZSAxMC4xIGlzIHVzZWQgdG8gcGVyZm9ybSB0aHJlc2hvbGQtbWFya2luZywgaXQN CiAgIGhhcyBhIHNsaWdodGx5IGRpZmZlcmVudCBiZWhhdmlvciB0aGFuIHRoZSBhbGdvcml0aG0g ZGVzY3JpYmVkIGluIENMDQoNCg0KDQpDaGFybnksIGV0IGFsLiAgICAgICAgICAgIEV4cGlyZXMg TWF5IDE0LCAyMDA4ICAgICAgICAgICAgICAgICBbUGFnZSAyMl0NCgwNCkludGVybmV0LURyYWZ0 ICAgICAgICAgICAgICBDb21wYXJpc29uIERyYWZ0ICAgICAgICAgICAgICAgTm92ZW1iZXIgMjAw Nw0KDQoNCiAgIGFuZCAzU00uICBUaGUgcGFja2V0IG1hcmtpbmcgYWxnb3JpdGhtIG9mIDNTTSBi YXNlcyBpdHMgbWFya2luZw0KICAgZGVjaXNpb24gb24gVEIuZmlsbCBiZWZvcmUgdG9rZW5zIGFy ZSByZW1vdmVkIHdoaWxlIG91ciBhbGdvcml0aG0NCiAgIG1ha2VzIHRoZSBkZWNpc2lvbiBhZnRl cndhcmRzLiAgSW4gYWRkaXRpb24sIHRoZSBhZG1pc3Npb24gbWFya2luZw0KICAgYWxnb3JpdGht cyBvZiBib3RoIDNTTSBhbmQgQ0wgc2V0IFRCLmZpbGw9MCB3aGVuIFRCLmZpbGw8cGFja2V0LnNp emUNCiAgIHdoaWxlIG91ciBhbGdvcml0aG0gbGVhdmVzIFRCLmZpbGwgdW5jaGFuZ2VkIGluIHRo aXMgY2FzZS4gIFRoaXMgaXMNCiAgIGRvbmUgaW4gdGhlIGludGVyZXN0IG9mIHNpbXBsaWZpY2F0 aW9uLCBhcyB3ZSBiZWxpZXZlIHRoYXQgdGhlIGltcGFjdA0KICAgb2YgdGhpcyBjaGFuZ2Ugb24g cGVyZm9ybWFuY2Ugd2lsbCBiZSBtaW5pbWFsIGluIHByYWN0aWNlLiAgVGhlDQogICBwc2V1ZG9j b2RlIHdlIHByb3ZpZGUgaW4gdGhlIEFwcGVuZGl4IGlzIGEgY29tcGxldGUgKGFuZCBtb3JlDQog ICBjb21wbGV4KSB2ZXJzaW9uIG9mIHRoZSB1bmlmaWVkIGFsZ29yaXRobSB0byBhZGRyZXNzIHRo aXMgaXNzdWUuICBBcw0KICAgbWVudGlvbmVkIGVhcmxpZXIsIHRoaXMgY29tcGxldGUgYWxnb3Jp dGhtIGFsc28gc3VwcG9ydHMgdGhlIHJhbXANCiAgIG1hcmtpbmcuICBJbiBvdXIganVkZ2VtZW50 LCBob3dldmVyLCBhIHNpbXBsZXIgdmVyc2lvbiBvZiBhbGdvcml0aG0NCiAgIDEwLjEgd291bGQg c3VmZmljZSBpbiBwcmFjdGljZS4NCg0KICAgV2UgYWxzbyBwcmVzZW50IGFuIGVxdWl2YWxlbnQg VlEgZm9ybXVsYXRpb24gb2YgdGhlIHNhbWUgYWxnb3JpdGhtIGluDQogICB0aGUgQXBwZW5kaXgu DQoNCg0KMTIuICBEaWZmaWN1bHRpZXMgd2l0aCBBbGxvd2luZyBNdWx0aXBsZSBNYXJraW5nIEJl aGF2aW9ycw0KDQogICBUaGVyZSBhcmUgYSBudW1iZXIgb2YgZGlmZmljdWx0aWVzIGFzc29jaWF0 ZWQgd2l0aCBhbGxvd2luZyBtdWx0aXBsZQ0KICAgZWRnZSBiZWhhdmlvcnMgaW4gdGhlIFBDTiBm cmFtZXdvcmsuICBCZWxvdyBpcyBhIGxpc3Qgb2Ygc29tZSBvZg0KICAgdGhlc2UgZGlmZmljdWx0 aWVzLg0KDQogICBvICBBZGRpdGlvbmFsIGltcGxlbWVudGF0aW9uIGNvbXBsZXhpdHkgaW4gdGhl IGNvcmUgZGV2aWNlcyBuZWVkZWQgdG8NCiAgICAgIHN1cHBvcnQgbXVsdGlwbGUgb3B0aW9ucy4N Cg0KICAgbyAgQWRkaXRpb25hbCBjb25maWd1cmF0aW9uIGNvbXBsZXhpdHkgbmVlZGVkIHRvIHN1 cHBvcnQgdGhlc2UNCiAgICAgIGRpZmZlcmVudCBvcHRpb25zIChlc3BlY2lhbGx5IGltcG9ydGFu dCBpbiB0aGUgY2FzZSB3aGVuIGRpZmZlcmVudA0KICAgICAgUENOIGRvbWFpbnMgY29uZmlndXJl ZCBmb3IgZGlmZmVyZW50IG9wdGlvbnMgbWVyZ2UpDQoNCiAgIG8gIERpZmZlcmVuY2VzIGluIHBh Y2tldCBkcm9wcGluZyBwcmVmZXJlbmNlcyByZXByZXNlbnQgYWRkaXRpb25hbA0KICAgICAgY29t cGxleGl0eSBpZiBkaWZmZXJlbnQgcG9saWNpZXMgbmVlZCB0byBiZSB1c2VkIHdpdGggZGlmZmVy ZW50DQogICAgICBvcHRpb25zIChhbHRob3VnaCB0aGUgZXhhY3QgaW1wYWN0IG9mIG5vdCBpbXBs ZW1lbnRpbmcgYW55DQogICAgICBkcm9wcGluZyBwcmVmZXJlbmNlcyBhdCBhbGwgZm9yIGRpZmZl cmVudCBhbGdvcml0aG1zIGlzIHVuZGVyDQogICAgICBzdHVkeSkuDQoNCiAgIG8gIERpZmZlcmVu Y2UgaW4gdGhlIFBDTiBpbmZvcm1hdGlvbiBzaWduYWxsZWQgYmV0d2VlbiB0aGUgZWdyZXNzIGFu ZA0KICAgICAgaW5ncmVzcyByZXF1aXJlIGRlZmluaXRpb24gYW5kIGltcGxlbWVudGF0aW9uIG9m IGRpZmZlcmVudA0KICAgICAgc2lnbmFsbGluZyBvcHRpb25zDQoNCiAgIG8gIEFkZGl0aW9uYWwg Y29tcGxleGl0eSBvZiBzdGFuZGFyZGl6YXRpb24gcHJvY2Vzcw0KDQogICBvICBUaGUgdW5pZmll ZCBzcGVjaWZpY2F0aW9uIGlzIGxpbWl0ZWQgdG8gdGhlIHRocmVlIGFwcHJvYWNoZXMNCiAgICAg IGNvbnNpZGVyZWQgaW4gdGhpcyBkcmFmdCBhbmQgZG9lcyBub3QgY29uc2lkZXIgYW55IG90aGVy IHBvc3NpYmxlDQogICAgICBhcHByb2FjaGVzIGluY2x1ZGluZyB0aGF0IHN1Z2dlc3RlZA0KICAg ICAgaW5bSS1ELndlc3RiZXJnLXBjbi1sb2FkLWNvbnRyb2xdDQoNCiAgIEEgcXVlc3Rpb24gdGhl biBhcmlzZXMgd2hldGhlciB0aGUgYmVuZWZpdHMgb2Ygc3BlY2lmeWluZyBhIG1vcmUNCg0KDQoN CkNoYXJueSwgZXQgYWwuICAgICAgICAgICAgRXhwaXJlcyBNYXkgMTQsIDIwMDggICAgICAgICAg ICAgICAgIFtQYWdlIDIzXQ0KDA0KSW50ZXJuZXQtRHJhZnQgICAgICAgICAgICAgIENvbXBhcmlz b24gRHJhZnQgICAgICAgICAgICAgICBOb3ZlbWJlciAyMDA3DQoNCg0KICAgZmxleGlibGUgdW5p ZmllZCBjb3JlIGJlaGF2aW9yIG91dHdlaWdoIHRoZSBhYm92ZSBkcmF3YmFja3MuICBXZSBkbw0K ICAgbm90IGF0dGVtcHQgdG8gYW5zd2VyIHRoaXMgcXVlc3Rpb24gaW4gdGhpcyBkcmFmdCwgYnV0 IHJhdGhlciBwb3NlIGl0DQogICBmb3IgYSBnZW5lcmFsIGRpc2N1c3Npb24gb2YgdGhlIFdHLg0K DQogICBGaW5hbGx5LCBpdCBzaG91bGQgYmUgbm90ZWQgdGhhdCBub3QgYWxsIG9mIHRoZSBhYm92 ZSBkaWZmaWN1bHRpZXMNCiAgIHBlcnRhaW4gdG8gZGlmZmVyZW50IGNvbWJpbmF0aW9uIG9mIHRo ZSBhcHByb2FjaGVzIGluIHRoZSBzYW1lDQogICBkZWdyZWUuICBGb3IgZXhhbXBsZSwgdGhlIGRp ZmZlcmVuY2VzIGJldHdlZW4gU00gYW5kIENMIHdpdGggcmVzcGVjdA0KICAgdG8gdGhlIFBDTi1i b3VuZGFyeS1ub2RlIGZ1bmN0aW9ucyBhbmQgdGhlIGluZm9ybWF0aW9uIHNpZ25hbGxlZA0KICAg YmV0d2VlbiB0aGUgUENOLWVncmVzcy1ub2RlIGFuZCBQQ04taW5ncmVzcy1ub2RlIGFyZSByZWxh dGl2ZWx5IHNtYWxsDQogICBjb21wYXJlZCB0byB0aGUgc3Vic3RhbnRpYWwgZGlmZmVyZW5jZXMg YmV0d2VlbiB0aGUgYm91bmRhcnkgYmVoYXZpb3INCiAgIGFuZCBpbmZvcm1hdGlvbiB0cmFuc3Bv cnQgb2YgM1NNIGNvbXBhcmVkIHRvIGJvdGggQ0wgYW5kIFNNLg0KICAgVGhlcmVmb3JlLCBhcyBk ZXNjcmliZWQgaW4gW0ktRC5jaGFybnktcGNuLXNpbmdsZS1tYXJraW5nXSwgU00gY291bGQNCiAg IGJlIGRlZmluZWQgYXMgYSAic3RlcHBpbmcgc3RvbmUiIHRvIENMIHdpdGggcmVsYXRpdmVseSBz bWFsbA0KICAgYWRkaXRpb25hbCBjb21wbGV4aXR5IGluIHN1Y2ggYSB3YXkgdGhhdCB0aGUgaW5m b3JtYXRpb24gdHJhbnNwb3J0IGlzDQogICBpZGVudGljYWwsIGFuZCB0aGUgYm91bmRhcnkgYmVo YXZpb3IgaXMgYWxtb3N0IGlkZW50aWNhbCAoYW5kIGNhbiBiZQ0KICAgc3dpdGNoZWQgYmV0d2Vl biB0aGUgdHdvIHNjaGVtZXMgYnkgYSB0b2dnbGUgb2YgYSBjb25maWd1cmF0aW9uDQogICBwYXJh bWV0ZXIpLiAgVGh1cywgdHJhbnNpdGlvbiBmcm9tIFNNIHRvIENMIGVzc2VudGlhbGx5IGFtb3Vu dHMgdG8NCiAgIHVwZ3JhZGluZyB0aGUgY29yZSBub2RlcyBvbmx5LiAgSW4gY29udHJhc3QsIHRy YW5zaXRpb24gZnJvbSBTTSBvciBDTA0KICAgdG8gM1NNIChvciB2aWNlIGEgdmVyc2EpIHJlcXVp cmVzIG5vdCBvbmx5IGEgY2hhbmdlIGluIHRoZSBjb3JlDQogICBiZWhhdmlvciwgYnV0IGEgc3Vi c3RhbnRpYWwgY2hhbmdlIGluIHRoZSBib3VuZGFyeSBiZWhhdmlvciBhbmQNCiAgIGluZm9ybWF0 aW9uIHRyYW5zcG9ydC4NCg0KDQoxMy4gIFNlY3VyaXR5IENvbnNpZGVyYXRpb25zDQoNCiAgIFRC RA0KDQoNCjE0LiAgSUFOQSBDb25zaWRlcmF0aW9ucw0KDQogICBUQkQNCg0KDQoxNS4gIEFwcGVu ZGl4DQoNCjE1LjEuICBGb3JtdWxhdGlvbiBvZiB0aGUgU2ltcGxlIEdlbmVyYWxpemVkIE1ldGVy aW5nIGFuZCBNYXJraW5nDQogICAgICAgQWxnb3JpdGhtDQoNCiAgIFRoZSBzaW1wbGUgZ2VuZXJh bGl6ZWQgbWV0ZXJpbmcgYW5kIG1hcmtpbmcgYWxnb3JpdGhtIGNhbiBhbHNvIGJlDQogICBkZXNj cmliZWQgYmFzZWQgb24gYSB2aXJ0dWFsIHF1ZXVlIChWUSkuICBUaGUgdHJhbnNmb3JtYXRpb24g aXMNCiAgIHN0cmFpZ2h0Zm9yd2FyZCBhbmQgdGhlIHJlc3VsdCBpcyBnaXZlbiBpbiB0aGUgYWxn b3JpdGhtIHNob3duIGluDQogICBGaWd1cmUgMTQuMS4NCg0KDQoNCg0KDQoNCg0KDQoNCkNoYXJu eSwgZXQgYWwuICAgICAgICAgICAgRXhwaXJlcyBNYXkgMTQsIDIwMDggICAgICAgICAgICAgICAg IFtQYWdlIDI0XQ0KDA0KSW50ZXJuZXQtRHJhZnQgICAgICAgICAgICAgIENvbXBhcmlzb24gRHJh ZnQgICAgICAgICAgICAgICBOb3ZlbWJlciAyMDA3DQoNCg0KICAgKHByZWFtYmxlKQ0KUGFyYW1l dGVyczoNClZRLnJhdGU6IHNlcnZpY2UgcmF0ZSBvZiBWUSBpbiBieXRlcy9zDQpWUS5zaXplOiBx dWV1ZSBzaXplIG9mIFZRIGluIGJ5dGVzDQpWUS50aHJlc2hvbGQ6IG1hcmtpbmcgdGhyZXNob2xk IG9mIFZRIGluIGJ5dGVzDQpWUS5zbG93ZG93bjogc2xvd2Rvd24gZmFjdG9yIGZvciBtYXJraW5n IGZyZXF1ZW5jeSByZWR1Y3Rpb24gb2YgVlEgaW4gYnl0ZXMNClZRLm1hcmtpbmdUeXBlOiBQQ04t Zmlyc3QtZW5jb2RpbmcgKCJhZG1pc3Npb24iKSAgb3IgIFBDTi1zZWNvbmQtZW5jb2RpbmcgKCJ0 ZXJtaW5hdGlvbiIpLg0KVlEubWV0ZXJlZE1hcmtpbmdzOiBzZXQgb2YgcGFja2V0IG1hcmtpbmdz IHRoYXQgYXJlIGVsaWdpYmxlIGZvciBtZXRlcmluZyBieSBWUSwNCiAgICAgICAgICAgICAgICAg ICAgaXQgaXMgYSBzdWJzZXQgb2YgKCJ1bm1hcmtlZCIsICJhZG1pc3Npb24iLCAidGVybWluYXRp b24iKS4NCg0KVlEubGVuZ3RoOiBudW1iZXIgb2YgYnl0ZXMgY3VycmVudGx5IGluIHRoZSBxdWV1 ZSBvZiB0aGUgVlENCg0KSW5wdXQ6IHBhY2tldA0KICAgIC8vIHRha2UgcGFzc2VkIHRpbWUgc2lu Y2UgbGFzdCB1cGRhdGUgaW50byBhY2NvdW50DQogICAgVlEubGVuZ3RoID0gbWF4KDAsIFZRLmxl bmd0aC0obm93LVZRLmxhc3RVcGRhdGUpICogVlEucmF0ZSk7DQogICAgVlEubGFzdFVwZGF0ZSA9 IG5vdzsNCg0KICAgIC8vIG1ldGVyIGFuZCBtYXJrDQogICAgSWYgKHBhY2tldC5tYXJrIGluIFZR Lm1ldGVyZWRNYXJraW5ncykNCiAgICAgICAgaWYgKFZRLmxlbmd0aCtwYWNrZXQuc2l6ZSA+IFZR LnNpemUpDQogICAgICAgICAgICBpZiAoIShwYWNrZXQubWFyayA9PSAidGVybWluYXRpb24iIGFu ZCBWUS5tYXJraW5nVHlwZSA9PSAiYWRtaXNzaW9uIikpDQogICAgICAgICAgICAgICAgcGFja2V0 Lm1hcmsgPSBWUS5tYXJraW5nVHlwZTsNCiAgICAgICAgICAgIGVuZGlmDQogICAgICAgIGVsc2UN CiAgICAgICAgICAgIFZRLmxlbmd0aCA9IFZRLmxlbmd0aCArIHBhY2tldC5zaXplOw0KICAgICAg ICAgICAgaWYgKFZRLmxlbmd0aCA+IFZRLnRocmVzaG9sZCkNCiAgICAgICAgICAgICAgICAvLyBy ZS1tYXJraW5nIG9mIFRNLW1hcmtlZCBwYWNrZXRzIHRvIEFNIG5vdCBhbGxvd2VkDQogICAgICAg ICAgICAgICAgaWYgKCEocGFja2V0Lm1hcmsgPT0gInRlcm1pbmF0aW9uIiBhbmQgVlEubWFya2lu Z1R5cGUgPT0gImFkbWlzc2lvbiIpKQ0KICAgICAgICAgICAgICAgICAgICBwYWNrZXQubWFyayA9 IFZRLm1hcmtpbmdUeXBlOw0KICAgICAgICAgICAgICAgIGVuZGlmDQogICAgICAgICAgICBlbmRp Zg0KICAgICAgICBlbmRpZg0KICAgIGVuZGlmDQoNCiAgICAvLyBtYXJraW5nIGZyZXF1ZW5jeSBy ZWR1Y3Rpb24NCiAgICBpZiAocGFja2V0Lm1hcmsgPT0gInRlcm1pbmF0aW9uIikNCiAgICAgICAg VlEubGVuZ3RoID0gbWF4KDAsIFZRLmxlbmd0aCAtIFZRLnNsb3dkb3duKTsNCiAgICBlbmRpZg0K DQpPdXRwdXQ6IHZvaWQNCg0KICAgKEZpZ3VyZSAxNC4xKQ0KDQogICBUaGUgdGFibGVzIGluIEZp Z3MgMTQuMyBhbmQgMTQuNCBhdCB0aGUgZW5kIG9mIHRoaXMgU2VjdGlvbiBnaXZlIHRoZQ0KICAg Y29ycmVzcG9uZGluZyBwYXJhbWV0ZXIgc2V0dGluZyBmb3IgdGhlIHRocmVlIHByb3Bvc2Fscy4N Cg0KICAgTk9URTogVGhlcmUgYXJlIHR3byBmdXJ0aGVyIG9wdGltaXphdGlvbiBvcHRpb25zIHRo YXQgYXJlIHBvc3NpYmxlDQogICBmb3IgdGhpcyAoYW5kIHRoZSBUQi1iYXNlZCkgbWV0ZXJpbmcg YW5kIG1hcmtpbmcgYWxnb3JpdGhtOg0KDQoNCg0KQ2hhcm55LCBldCBhbC4gICAgICAgICAgICBF eHBpcmVzIE1heSAxNCwgMjAwOCAgICAgICAgICAgICAgICAgW1BhZ2UgMjVdDQoMDQpJbnRlcm5l dC1EcmFmdCAgICAgICAgICAgICAgQ29tcGFyaXNvbiBEcmFmdCAgICAgICAgICAgICAgIE5vdmVt YmVyIDIwMDcNCg0KDQogICAxLiAgSWYgVE0tcGFja2V0cyBhcmUgbm90IG1ldGVyZWQgZm9yIGFk bWlzc2lvbi1tYXJraW5nLCB0aGUgaW5uZXINCiAgICAgICBpZi1jbGF1c2UgdG8gYXZvaWQgdGhl IHJlLW1hcmtpbmcgb2YgVE0tbWFya2VkIHBhY2tldHMgdG8gQU0gY2FuDQogICAgICAgYmUgcmVt b3ZlZC4gIEN1cnJlbnQgcGVyZm9ybWFuY2UgcmVzdWx0cyBzdWdnZXN0IHRoYXQgdGhpcyBjb3Vs ZA0KICAgICAgIGJlIGRvbmUsIGJ1dCBmb3IgdGhlIHNha2Ugb2YgZXh0ZW5zaWJpbGl0eSB0b3dh cmRzIGZ1dHVyZSByYXRlDQogICAgICAgYWRhcHRhdGlvbiBpdCBpcyBiZXR0ZXIgdG8ga2VlcCBv cGVuIHRoZSBvcHRpb24gb2YgbWV0ZXJpbmcgYWxsDQogICAgICAgcGFja2V0cyBhbHNvIGZvciBh ZG1pc3Npb24gbWFya2luZy4NCg0KICAgMi4gIFRoZSBtYXJraW5nIGZyZXF1ZW5jeSByZWR1Y3Rp b24gbWF5IGJlIGFwcGxpZWQgd2hlbiBwYWNrZXRzIGFyZQ0KICAgICAgIHJlLW1hcmtlZCB0byBU TS4gIFRoaXMgaXMgYXQgdGhlIGV4cGVuc2Ugb2YgaW5jcmVhc2VkIHVuZmFpcm5lc3MNCiAgICAg ICBhbmQgb3ZlciB0ZXJtaW5hdGlvbiBpbiB0aGUgcHJlc2VuY2Ugb2Ygc2V2ZXJhbCBzaW11bHRh bmVvdXNseQ0KICAgICAgIG92ZXJsb2FkZWQgbGlua3MuICBUaGUgYWR2YW50YWdlIGlzIGFuIGlt cHJvdmVkIHJ1bnRpbWUgb2YgdGhlDQogICAgICAgYWxnb3JpdGhtIGFuZCBhIHBvc3NpYmx5IHNp bXBsZXIgaW1wbGVtZW50YXRpb24uDQoNCjE1LjIuICBWUSBGb3JtdWxhdGlvbiBvZiB0aGUgQ29t cGxleCBHZW5lcmFsaXplZCBNZXRlcmluZyBhbmQgTWFya2luZw0KICAgICAgIEFsZ29yaXRobQ0K DQogICBUaGUgc2ltcGxlIGdlbmVyYWxpemVkIG1ldGVyaW5nIGFuZCBtYXJraW5nIGFsZ29yaXRo bSBpbiAxNC4xIGhhcyB0aGUNCiAgIGZvbGxvd2luZyBzaG9ydGNvbWluZ3M6DQoNCiAgIDEuICBJ dCBkb2VzIG5vdCBzdXBwb3J0IHJhbXAgbWFya2luZyB3aGljaCBtYXkgYmUgZGVzaXJhYmxlIGZv ciBDTDsNCg0KICAgMi4gIElmIHRoZSBwYWNrZXQgZG9lcyBub3QgZml0IGludG8gdGhlIHF1ZXVl LCB0aGUgcXVldWUgY2Fubm90IGJlDQogICAgICAgZmlsbGVkIHVwIHRvIGl0cyBzaXplLiAgVGhp cyBpcyBhIHByb3BlcnR5IGZvciBhZG1pc3Npb24gbWFya2luZw0KICAgICAgIGluIENMIGFuZCAz U00gdGhhdCBpcyBub3QgaW1wbGVtZW50ZWQgYnkgYSBzaW1wbGlmaWVkIGFsZ29yaXRobQ0KICAg ICAgIChhbHRob3VnaCB0aGUgaW1wYWN0IG9mIHRoaXMgY2hhbmdlIGFwcGVhcnMgdG8gYmUgbWlu aW1hbCk7DQoNCiAgIDMuICBJZiB0aGUgcGFja2V0IGZpdHMgaW50byB0aGUgcXVldWUsIHRoZSBt YXJraW5nIGRlY2lzaW9uIGlzIGFsd2F5cw0KICAgICAgIGJhc2VkIG9uIHRoZSBxdWV1ZSBzaXpl IGluY2x1ZGluZyB0aGUgc2l6ZSBvZiB0aGUgbmV3IHBhY2tldC4NCiAgICAgICBUaGlzIGxlYWRz IHRvIGEgbmljZXIgZm9ybXVsYXRpb24gb2YgdGhyZXNob2xkIG9yIHJhbXAgbWFya2luZw0KICAg ICAgIChob3dldmVyLCB0aGUgaW1wYWN0IG9mIHRoaXMgY2hhbmdlIHNlZW1zIG1pbmltYWwpIElm IHRoZQ0KICAgICAgIGdlbmVyYWxpemVkIGFsZ29yaXRobSB0YWtlcyBhbHNvIHRoZXNlIDMgaXNz dWVzIGludG8gYWNjb3VudCwgaXQNCiAgICAgICBiZWNvbWVzIHNpZ25pZmljYW50bHkgbW9yZSBj b21wbGV4Lg0KDQogICBUbyBpbXByb3ZlIHNob3J0Y29taW5nIDEsIHRoZSBhbGdvcml0aG0gaGFz IG5vdyB0d28gbWFya2luZw0KICAgdGhyZXNob2xkcyB0byBzdXBwb3J0IHJhbXAgYW5kIHRocmVz aG9sZCBtYXJraW5nIGluc3RlYWQgb2YgYSBzaW5nbGUNCiAgIG9uZTogVlEubG93ZXJUaHJlc2hv bGQgYW5kIFZRLnVwcGVyVGhyZXNob2xkLiAgUGFja2V0cyBhcmUgbm90IG1hcmtlZA0KICAgaWYg dGhlIHF1ZXVlIGxlbmd0aCBpcyBiZWxvdyBWUS5sb3dlclRocmVzaG9sZCwgdGhlIG1hcmtpbmcN CiAgIHByb2JhYmlsaXR5IGxpbmVhcmx5IGluY3JlYXNlcyBmcm9tIDAgdG8gMSBiZXR3ZWVuIFZR Lmxvd2VyVGhyZXNob2xkDQogICBhbmQgVlEudXBwZXJUaHJlc2hvbGQsIGFuZCBwYWNrZXRzIGFy ZSBkZWZpbml0ZWx5IG1hcmtlZCBpZiB0aGUgcXVldWUNCiAgIGxlbmd0aCBpcyBWUS51cHBlclRo cmVzaG9sZCBhbmQgYWJvdmUuICBJZiBib3RoIHRocmVzaG9sZHMgYXJlIHNldCB0bw0KICAgdGhl IHNhbWUgcG9zaXRpdmUgdmFsdWUsIHRocmVzaG9sZCBtYXJraW5nIGlzIHBlcmZvcm1lZDogcGFj a2V0cyBhcmUNCiAgIG5vdCBtYXJrZWQgaWYgdGhlIHF1ZXVlIGxlbmd0aCBpcyBiZWxvdyBvciBl cXVhbCB0byB0aGF0IHRocmVzaG9sZCwNCiAgIG90aGVyd2lzZSB0aGV5IGFyZSBtYXJrZWQuDQoN CiAgIFRvIGltcHJvdmUgc2hvcnRjb21pbmcgMiBhbmQgMywgdGhlIEJvb2xlYW4gdmFyaWFibGUN CiAgIFZRLmFsd2F5c1VwZGF0ZVN0YXRlIGlzIGludHJvZHVjZWQuICBJZiBpdCBpcyBzZXQgdG8g dHJ1ZSwgdGhlIHF1ZXVlDQogICBsZW5ndGggc2hvdWxkIGFsd2F5cyBiZSBpbmNyZWFzZWQgYnkg dGhlIHBhY2tldCBzaXplOyB0aGlzIGlzIHRoZQ0KICAgZGVzaXJlZCBiZWhhdmlvdXIgZm9yIGFk bWlzc2lvbiBtYXJraW5nIHdoZW4gYWxsIHBhY2tldHMgYXJlIGV4cGVjdGVkDQoNCg0KDQpDaGFy bnksIGV0IGFsLiAgICAgICAgICAgIEV4cGlyZXMgTWF5IDE0LCAyMDA4ICAgICAgICAgICAgICAg ICBbUGFnZSAyNl0NCgwNCkludGVybmV0LURyYWZ0ICAgICAgICAgICAgICBDb21wYXJpc29uIERy YWZ0ICAgICAgICAgICAgICAgTm92ZW1iZXIgMjAwNw0KDQoNCiAgIHRvIGJlIG1hcmtlZCB3aGVu IHRoZSBhZG1pc3NpYmxlIHJhdGUgaXMgZXhjZWVkZWQgYnkgdGhlIFBDTiByYXRlLg0KICAgSWYg aXQgaXMgc2V0IHRvIGZhbHNlLCB0aGUgcXVldWUgbGVuZ3RoIGlzIG9ubHkgaW5jcmVhc2VkIGlm IHRoZQ0KICAgcGFja2V0IGlzIG5vdCBtYXJrZWQ7IHRoaXMgaXMgdGhlIGRlc2lyZWQgYmVoYXZp b3VyIGZvciBleGNlc3MtcmF0ZS0NCiAgIG1hcmtpbmcuDQoNCiAgIChwcmVhbWJsZSkNCkFkZGl0 aW9uYWwgcGFyYW1ldGVyczoNClZRLmxvd2VyVGhyZXNob2xkOiBsb3dlciBtYXJraW5nIHRocmVz aG9sZCBvZiBWUSBpbiBieXRlcw0KVlEudXBwZXJUaHJlc2hvbGQ6IHVwcGVyIG1hcmtpbmcgdGhy ZXNob2xkIG9mIFZRIGluIGJ5dGVzDQpWUS5hbHdheXNVcGRhdGVTdGF0ZTogVlEgbGVuZ3RoIHN0 YXRlIGFsd2F5cyBpbmNyZWFzZWQgb3Igbm90DQoNCklucHV0OiBwYWNrZXQNCg0KICAgIC8vIHRh a2UgcGFzc2VkIHRpbWUgc2luY2UgbGFzdCB1cGRhdGUgaW50byBhY2NvdW50DQogICAgVlEubGVu Z3RoID0gbWF4KDAsIFZRLmxlbmd0aC0obm93LVZRLmxhc3RVcGRhdGUpKlZRLnJhdGUpOw0KICAg IFZRLmxhc3RVcGRhdGUgPSBub3c7DQoNCiAgICAvLyBtZXRlciBhbmQgbWFyaw0KICAgIGlmIChw YWNrZXQubWFyayBpbiBWUS5tZXRlcmVkTWFya2luZ3MpDQogICAgICAgIGlmIChWUS5hbHdheXNV cGRhdGVTdGF0ZSA9PSB0cnVlKQ0KICAgICAgICAgICAgLy8gdGhyZXNob2xkIG9yIHJhbXAtbWFy a2luZw0KICAgICAgICAgICAgaWYgKFZRLmxlbmd0aCA+IFZRLnVwcGVyVGhyZXNob2xkKQ0KICAg ICAgICAgICAgICAgIC8vIHJlLW1hcmtpbmcgb2YgVE0tbWFya2VkIHBhY2tldHMgdG8gQU0gbm90 IGFsbG93ZWQNCiAgICAgICAgICAgICAgICBpZiAoIShwYWNrZXQubWFyayA9PSAidGVybWluYXRp b24iIGFuZCBWUS5tYXJraW5nVHlwZSA9PSAiYWRtaXNzaW9uIikpDQogICAgICAgICAgICAgICAg ICAgIHBhY2tldC5tYXJrID0gVlEubWFya2luZ1R5cGU7DQogICAgICAgICAgICAgICAgZW5kaWYN CiAgICAgICAgICAgIGVsc2VpZiAoVlEubGVuZ3RoID4gVlEubG93ZXJUaHJlc2hvbGQpDQogICAg ICAgICAgICAgICAgY2hvb3NlIHJhbmRvbSBudW1iZXIgdSAoMCA8IHUgPCAxKTsNCiAgICAgICAg ICAgICAgICBpZiAodSA8IChWUS5sZW5ndGgtVlEubG93ZXJUaHJlc2hvbGQpLyhWUS51cHBlclRo cmVzaG9sZC1WUS5sb3dlclRocmVzaG9sZCkpDQogICAgICAgICAgICAgICAgICAgIC8vIHJlLW1h cmtpbmcgb2YgVE0tbWFya2VkIHBhY2tldHMgdG8gQU0gbm90IGFsbG93ZWQNCiAgICAgICAgICAg ICAgICAgICAgaWYgKCEocGFja2V0Lm1hcmsgPT0gInRlcm1pbmF0aW9uIiBhbmQgVlEubWFya2lu Z1R5cGUgPT0gImFkbWlzc2lvbiIpKQ0KICAgICAgICAgICAgICAgICAgICAgICAgcGFja2V0Lm1h cmsgPSBWUS5tYXJraW5nVHlwZTsNCiAgICAgICAgICAgICAgICAgICAgZW5kaWYNCiAgICAgICAg ICAgICAgICBlbmRpZg0KICAgICAgICAgICAgZW5kaWYNCiAgICAgICAgICAgIFZRLmxlbmd0aCA9 IG1pbihWUS5zaXplLCBWUS5sZW5ndGgrcGFja2V0LnNpemUpOw0KICAgICAgICBlbHNlDQogICAg ICAgICAgICAvLyBleGNlc3MtcmF0ZS1tYXJraW5nDQogICAgICAgICAgICBpZiAoVlEubGVuZ3Ro ICsgcGFja2V0LnNpemUgPiBWUS5zaXplKQ0KICAgICAgICAgICAgICAgIC8vIHJlLW1hcmtpbmcg b2YgVE0tbWFya2VkIHBhY2tldHMgdG8gQU0gbm90IGFsbG93ZWQNCiAgICAgICAgICAgICAgICBp ZiAoIShwYWNrZXQubWFyayA9PSAidGVybWluYXRpb24iIGFuZCBWUS5tYXJraW5nVHlwZSA9PSAi YWRtaXNzaW9uIikpDQogICAgICAgICAgICAgICAgICAgIHBhY2tldC5tYXJrID0gVlEubWFya2lu Z1R5cGU7DQogICAgICAgICAgICAgICAgZW5kaWYNCiAgICAgICAgICAgIGVsc2UNCiAgICAgICAg ICAgICAgICBWUS5sZW5ndGggPSBWUS5sZW5ndGggKyBwYWNrZXQuc2l6ZTsNCiAgICAgICAgICAg IGVuZGlmDQogICAgICAgIGVuZGlmDQogICAgZW5kaWYNCg0KDQoNCkNoYXJueSwgZXQgYWwuICAg ICAgICAgICAgRXhwaXJlcyBNYXkgMTQsIDIwMDggICAgICAgICAgICAgICAgIFtQYWdlIDI3XQ0K DA0KSW50ZXJuZXQtRHJhZnQgICAgICAgICAgICAgIENvbXBhcmlzb24gRHJhZnQgICAgICAgICAg ICAgICBOb3ZlbWJlciAyMDA3DQoNCg0KICAgIC8vIG1hcmtpbmcgZnJlcXVlbmN5IHJlZHVjdGlv bg0KICAgIGlmIChwYWNrZXQubWFyayA9PSAidGVybWluYXRpb24iKQ0KICAgICAgICBWUS5sZW5n dGggPSBtYXgoMCwgVlEubGVuZ3RoIC0gVlEuc2xvd2Rvd24pOw0KICAgIGVuZGlmDQoNCk91dHB1 dDogdm9pZA0KDQogICAoRmlndXJlIDE0LjIpDQoNCiAgIChwcmVhbWJsZSkNCnwtLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tfA0KfCAgICAgICAgICAgICAgICB8ICBDTCBBZG1pc3Npb24gICB8ICAgICAgIFNNICAgICAg ICB8IDNTTSBBZG1pc3Npb24gICB8DQp8LS0tLS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tLS0tLS0t LXwtLS0tLS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tLS0tLS0tLXwNCnwgICAgVlEucmF0ZSAgICAg fCAgICAgIFBDTi0gICAgICAgfCAgICAgIFBDTi0gICAgICAgfCAgICAgIFBDTi0gICAgICAgfA0K fCAgICAgICAgICAgICAgICB8IGxvd2VyLXRocmVzaG9sZCB8IGxvd2VyLXRocmVzaG9sZCB8IGxv d2VyLXRocmVzaG9sZCB8DQp8LS0tLS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tLS0tLS0tLXwtLS0t LS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tLS0tLS0tLXwNCnwgICAgVlEuc2l6ZSAgICAgfCAgIGNv bmZpZ3VyZWQgICAgfCAgIGNvbmZpZ3VyZWQgICAgfCAgIGNvbmZpZ3VyZWQgICAgfA0KfC0tLS0t LS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0t LS0tLS0tLS18DQp8ICAgVlEuYWx3YXlzLSAgIHwgICAgIHRydWUgICAgICAgIHwgICAgICBmYWxz ZSAgICAgIHwgICAgICB0cnVlICAgICAgIHwNCnwgIFVwZGF0ZVN0YXRlICAgfCAgICAgICAgICAg ICAgICAgfCAgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAgfA0KfC0tLS0tLS0tLS0t LS0tLS18LS0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS0t LS18DQp8ICAgICAgVlEuICAgICAgIHwgICBjb25maWd1cmVkICAgIHwgICAgICAgMCAgICAgICAg IHwgICBjb25maWd1cmVkICAgIHwNCnwgbG93ZXJUaHJlc2hvbGQgfCAgICAgICAgICAgICAgICAg fCAgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAgfA0KfC0tLS0tLS0tLS0tLS0tLS18 LS0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS0tLS18DQp8 ICAgICAgVlEuICAgICAgIHwgICBjb25maWd1cmVkICAgIHwgICAgICAgMCAgICAgICAgIHwgICBj b25maWd1cmVkICAgIHwNCnwgdXBwZXJUaHJlc2hvbGQgfCAgICAgICAgICAgICAgICAgfCAgICAg ICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAgfA0KfC0tLS0tLS0tLS0tLS0tLS18LS0tLS0t LS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS0tLS18DQp8ICBWUS5z bG93ZG93biAgIHwgICAgICAgMCAgICAgICAgIHwgICAgICAgMCAgICAgICAgIHwgICAgICAgMCAg ICAgICAgIHwNCnwtLS0tLS0tLS0tLS0tLS0tfC0tLS0tLS0tLS0tLS0tLS0tfC0tLS0tLS0tLS0t LS0tLS0tfC0tLS0tLS0tLS0tLS0tLS0tfA0KfCAgICAgIFZRLiAgICAgICB8ICAidW5tYXJrZWQi ICAgICB8ICAidW5tYXJrZWQiICAgICB8ICAidW5tYXJrZWQiICAgICB8DQp8IG1ldGVyZWRNYXJr aW5nc3wgICJhZG1pc3Npb24iICAgIHwgICAgICAgICAgICAgICAgIHwgICJhZG1pc3Npb24iICAg IHwNCnwgICAgICAgICAgICAgICAgfCAgInRlcm1pbmF0aW9uIiAgfCAgICAgICAgICAgICAgICAg fCAgInRlcm1pbmF0aW9uIiAgfA0KfC0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS0tLS18 LS0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS0tLS18DQp8IFZRLm1hcmtpbmdUeXBlIHwg ICJhZG1pc3Npb24iICAgIHwgICJhZG1pc3Npb24iICAgIHwgICJhZG1pc3Npb24iICAgIHwNCiAt LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tfA0KICAgKEZpZ3VyZSAxNC4zLiAgQWRtaXNzaW9uIHNldHRpbmdzIGZvciB0 aGUgdGhyZWUgYWxnb3JpdGhtcyAoVlENCiAgIGZvcm11bGF0aW9uKSkNCg0KDQoNCg0KDQoNCg0K DQoNCg0KDQoNCg0KDQpDaGFybnksIGV0IGFsLiAgICAgICAgICAgIEV4cGlyZXMgTWF5IDE0LCAy MDA4ICAgICAgICAgICAgICAgICBbUGFnZSAyOF0NCgwNCkludGVybmV0LURyYWZ0ICAgICAgICAg ICAgICBDb21wYXJpc29uIERyYWZ0ICAgICAgICAgICAgICAgTm92ZW1iZXIgMjAwNw0KDQoNCiAg IChwcmVhbWJsZSkNCiAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KfCAgICAgICAgICAgICAgICB8IENMIFRlcm1p bmF0aW9uICB8ICAgICAgIFNNICAgICAgICB8IDNTTSBUZXJtaW5hdGlvbiB8DQp8LS0tLS0tLS0t LS0tLS0tLXwtLS0tLS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tLS0t LS0tLXwNCnwgICAgVlEucmF0ZSAgICAgfCAgICAgUENOLSAgICAgICAgfCAgICAgIE4vQSAgICAg ICAgfCAgICAgIFBDTi0gICAgICAgfA0KfCAgICAgICAgICAgICAgICB8IGxvd2VyLXRocmVzaG9s ZCB8ICAgICAgICAgICAgICAgICB8IHVwcGVyLXRocnNlaG9sZCB8DQp8LS0tLS0tLS0tLS0tLS0t LXwtLS0tLS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tLS0tLS0tLXwN CnwgICAgVlEuc2l6ZSAgICAgfCAgIGNvbmZpZ3VyZWQgICAgfCAgICAgIE4vQSAgICAgICAgfCAg IGNvbmZpZ3VyZWQgICAgfA0KfC0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS0tLS18LS0t LS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS0tLS18DQp8ICAgVlEuYWx3YXlzLSAgIHwgICAg IGZhbHNlICAgICAgIHwgICAgICBOL0EgICAgICAgIHwgICAgICBmYWxzZSAgICAgIHwNCnwgIFVw ZGF0ZVN0YXRlICAgfCAgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAgfCAgICAgICAg ICAgICAgICAgfA0KfC0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0t LS0tLS0tLS18LS0tLS0tLS0tLS0tLS0tLS18DQp8ICAgICAgVlEuICAgICAgIHwgICAgICAgMCAg ICAgICAgIHwgICAgICBOL0EgICAgICAgIHwgICAgICAgMCAgICAgICAgIHwNCnwgbG93ZXJUaHJl c2hvbGQgfCAgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAg ICAgfA0KfC0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS0t LS18LS0tLS0tLS0tLS0tLS0tLS18DQp8ICAgICAgVlEuICAgICAgIHwgICAgICAgMCAgICAgICAg IHwgICAgICBOL0EgICAgICAgIHwgICAgICAgMCAgICAgICAgIHwNCnwgdXBwZXJUaHJlc2hvbGQg fCAgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAgfA0K fC0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS0tLS18LS0t LS0tLS0tLS0tLS0tLS18DQp8ICBWUS5zbG93ZG93biAgIHwgICAgICAgMCAgICAgICAgIHwgICAg ICBOL0EgICAgICAgIHwgICBjb25maWd1cmVkICAgIHwNCnwtLS0tLS0tLS0tLS0tLS0tfC0tLS0t LS0tLS0tLS0tLS0tfC0tLS0tLS0tLS0tLS0tLS0tfC0tLS0tLS0tLS0tLS0tLS0tfA0KfCAgICAg IFZRLiAgICAgICB8ICAidW5tYXJrZWQiICAgICB8ICAgICAgTi9BICAgICAgICB8ICAidW5tYXJr ZWQiICAgICB8DQp8IG1ldGVyZWRNYXJraW5nc3wgICJhZG1pc3Npb24iICAgIHwgICAgICAgICAg ICAgICAgIHwgICJhZG1pc3Npb24iICAgIHwNCnwgICAgICAgICAgICAgICAgfCAgICAgICAgICAg ICAgICAgfCAgICAgICAgICAgICAgICAgfCAgInRlcm1pbmF0aW9uIiAgfA0KfC0tLS0tLS0tLS0t LS0tLSB8LS0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS0t LS18DQp8IFZRLm1hcmtpbmdUeXBlIHwgICJ0ZXJtaW5hdGlvbiIgIHwgICAgICBOL0EgICAgICAg IHwgInRlcm1pbmF0aW9uIiAgIHwNCiAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQoNCiAgIChGaWd1cmUgMTQuNC4g IFRlcm1pbmF0aW9uIHNldHRpbmdzIGZvciB0aGUgdGhyZWUgYWxnb3JpdGhtcyAoVlENCiAgIGZv cm11bGF0aW9uKSkNCg0KDQoxNi4gIFJlZmVyZW5jZXMNCg0KMTYuMS4gIE5vcm1hdGl2ZSBSZWZl cmVuY2VzDQoNCiAgIFtSRkMyMTE5XSAgQnJhZG5lciwgUy4sICJLZXkgd29yZHMgZm9yIHVzZSBp biBSRkNzIHRvIEluZGljYXRlDQogICAgICAgICAgICAgIFJlcXVpcmVtZW50IExldmVscyIsIEJD UCAxNCwgUkZDIDIxMTksIE1hcmNoIDE5OTcuDQoNCjE2LjIuICBJbmZvcm1hdGl2ZSBSZWZlcmVu Y2VzDQoNCiAgIFtJLUQuYmFiaWFyei1wY24tM3NtXQ0KICAgICAgICAgICAgICBCYWJpYXJ6LCBK LiwgIlRocmVlIFN0YXRlIFBDTiBNYXJraW5nIiwNCiAgICAgICAgICAgICAgZHJhZnQtYmFiaWFy ei1wY24tM3NtLTAwICh3b3JrIGluIHByb2dyZXNzKSwgSnVseSAyMDA3Lg0KDQogICBbSS1ELmJh YmlhcnotcGNuLWV4cGxpY2l0LW1hcmtpbmddDQogICAgICAgICAgICAgIExpdSwgWC4gYW5kIEou IEJhYmlhcnosICJTaW11bGF0aW9ucyBSZXN1bHRzIGZvciAzc00iLA0KICAgICAgICAgICAgICBk cmFmdC1iYWJpYXJ6LXBjbi1leHBsaWNpdC1tYXJraW5nLTAxICh3b3JrIGluIHByb2dyZXNzKSwN CiAgICAgICAgICAgICAgSnVseSAyMDA3Lg0KDQoNCg0KQ2hhcm55LCBldCBhbC4gICAgICAgICAg ICBFeHBpcmVzIE1heSAxNCwgMjAwOCAgICAgICAgICAgICAgICAgW1BhZ2UgMjldDQoMDQpJbnRl cm5ldC1EcmFmdCAgICAgICAgICAgICAgQ29tcGFyaXNvbiBEcmFmdCAgICAgICAgICAgICAgIE5v dmVtYmVyIDIwMDcNCg0KDQogICBbSS1ELmJyaXNjb2UtdHN2d2ctY2wtYXJjaGl0ZWN0dXJlXQ0K ICAgICAgICAgICAgICBCcmlzY29lLCBCLiwgIkFuIGVkZ2UtdG8tZWRnZSBEZXBsb3ltZW50IE1v ZGVsIGZvciBQcmUtDQogICAgICAgICAgICAgIENvbmdlc3Rpb24gTm90aWZpY2F0aW9uOiBBZG1p c3Npb24gIENvbnRyb2wgb3ZlciBhDQogICAgICAgICAgICAgIERpZmZTZXJ2IFJlZ2lvbiIsIGRy YWZ0LWJyaXNjb2UtdHN2d2ctY2wtYXJjaGl0ZWN0dXJlLTA0DQogICAgICAgICAgICAgICh3b3Jr IGluIHByb2dyZXNzKSwgT2N0b2JlciAyMDA2Lg0KDQogICBbSS1ELmJyaXNjb2UtdHN2d2ctY2wt cGhiXQ0KICAgICAgICAgICAgICBCcmlzY29lLCBCLiwgIlByZS1Db25nZXN0aW9uIE5vdGlmaWNh dGlvbiBtYXJraW5nIiwNCiAgICAgICAgICAgICAgZHJhZnQtYnJpc2NvZS10c3Z3Zy1jbC1waGIt MDMgKHdvcmsgaW4gcHJvZ3Jlc3MpLA0KICAgICAgICAgICAgICBPY3RvYmVyIDIwMDYuDQoNCiAg IFtJLUQuYnJpc2NvZS10c3Z3Zy1yZS1lY24tYm9yZGVyLWNoZWF0XQ0KICAgICAgICAgICAgICBC cmlzY29lLCBCLiwgIkVtdWxhdGluZyBCb3JkZXIgRmxvdyBQb2xpY2luZyB1c2luZyBSZS1FQ04N CiAgICAgICAgICAgICAgb24gQnVsayBEYXRhIiwgZHJhZnQtYnJpc2NvZS10c3Z3Zy1yZS1lY24t Ym9yZGVyLWNoZWF0LTAxDQogICAgICAgICAgICAgICh3b3JrIGluIHByb2dyZXNzKSwgSnVuZSAy MDA2Lg0KDQogICBbSS1ELmJyaXNjb2UtdHN2d2ctcmUtZWNuLXRjcF0NCiAgICAgICAgICAgICAg QnJpc2NvZSwgQi4sICJSZS1FQ046IEFkZGluZyBBY2NvdW50YWJpbGl0eSBmb3IgQ2F1c2luZw0K ICAgICAgICAgICAgICBDb25nZXN0aW9uIHRvIFRDUC9JUCIsIGRyYWZ0LWJyaXNjb2UtdHN2d2ct cmUtZWNuLXRjcC0wNA0KICAgICAgICAgICAgICAod29yayBpbiBwcm9ncmVzcyksIEp1bHkgMjAw Ny4NCg0KICAgW0ktRC5jaGFybnktcGNuLXNpbmdsZS1tYXJraW5nXQ0KICAgICAgICAgICAgICBD aGFybnksIEEuLCAiUHJlLUNvbmdlc3Rpb24gTm90aWZpY2F0aW9uIFVzaW5nIFNpbmdsZQ0KICAg ICAgICAgICAgICBNYXJraW5nIGZvciBBZG1pc3Npb24gYW5kICBUZXJtaW5hdGlvbiIsDQogICAg ICAgICAgICAgIGRyYWZ0LWNoYXJueS1wY24tc2luZ2xlLW1hcmtpbmctMDIgKHdvcmsgaW4gcHJv Z3Jlc3MpLA0KICAgICAgICAgICAgICBKdWx5IDIwMDcuDQoNCiAgIFtJLUQuZGF2aWUtZWNuLW1w bHNdDQogICAgICAgICAgICAgIERhdmllLCBCLiwgIkV4cGxpY2l0IENvbmdlc3Rpb24gTWFya2lu ZyBpbiBNUExTIiwNCiAgICAgICAgICAgICAgZHJhZnQtZGF2aWUtZWNuLW1wbHMtMDEgKHdvcmsg aW4gcHJvZ3Jlc3MpLCBPY3RvYmVyIDIwMDYuDQoNCiAgIFtJLUQuaWV0Zi1wY24tYXJjaGl0ZWN0 dXJlXQ0KICAgICAgICAgICAgICBFYXJkbGV5LCBQLiwgIlByZS1Db25nZXN0aW9uIE5vdGlmaWNh dGlvbiBBcmNoaXRlY3R1cmUiLA0KICAgICAgICAgICAgICBkcmFmdC1pZXRmLXBjbi1hcmNoaXRl Y3R1cmUtMDEgKHdvcmsgaW4gcHJvZ3Jlc3MpLA0KICAgICAgICAgICAgICBPY3RvYmVyIDIwMDcu DQoNCiAgIFtJLUQubGVmYXVjaGV1ci1lbWVyZ2VuY3ktcnN2cF0NCiAgICAgICAgICAgICAgRmF1 Y2hldXIsIEYuLCAiUlNWUCBFeHRlbnNpb25zIGZvciBFbWVyZ2VuY3kgU2VydmljZXMiLA0KICAg ICAgICAgICAgICBkcmFmdC1sZWZhdWNoZXVyLWVtZXJnZW5jeS1yc3ZwLTAyICh3b3JrIGluIHBy b2dyZXNzKSwNCiAgICAgICAgICAgICAgSnVuZSAyMDA2Lg0KDQogICBbSS1ELndlc3RiZXJnLXBj bi1sb2FkLWNvbnRyb2xdDQogICAgICAgICAgICAgIFdlc3RiZXJnLCBMLiwgIkxDLVBDTjogVGhl IExvYWQgQ29udHJvbCBQQ04gU29sdXRpb24iLA0KICAgICAgICAgICAgICBkcmFmdC13ZXN0YmVy Zy1wY24tbG9hZC1jb250cm9sLTAxICh3b3JrIGluIHByb2dyZXNzKSwNCiAgICAgICAgICAgICAg U2VwdGVtYmVyIDIwMDcuDQoNCiAgIFtJLUQuemhhbmctcGNuLXBlcmZvcm1hbmNlLWV2YWx1YXRp b25dDQogICAgICAgICAgICAgIFpoYW5nLCBYLiwgIlBlcmZvcm1hbmNlIEV2YWx1YXRpb24gb2Yg Q0wtUEhCIEFkbWlzc2lvbiBhbmQNCg0KDQoNCkNoYXJueSwgZXQgYWwuICAgICAgICAgICAgRXhw aXJlcyBNYXkgMTQsIDIwMDggICAgICAgICAgICAgICAgIFtQYWdlIDMwXQ0KDA0KSW50ZXJuZXQt RHJhZnQgICAgICAgICAgICAgIENvbXBhcmlzb24gRHJhZnQgICAgICAgICAgICAgICBOb3ZlbWJl ciAyMDA3DQoNCg0KICAgICAgICAgICAgICBUZXJtaW5hdGlvbiBBbGdvcml0aG1zIiwNCiAgICAg ICAgICAgICAgZHJhZnQtemhhbmctcGNuLXBlcmZvcm1hbmNlLWV2YWx1YXRpb24tMDIgKHdvcmsg aW4NCiAgICAgICAgICAgICAgcHJvZ3Jlc3MpLCBKdWx5IDIwMDcuDQoNCjE2LjMuICBSZWZlcmVu Y2VzDQoNCiAgIFtNZW50aF0gICAgIlBDTi1CYXNlZCBSZXNpbGllbnQgTmV0d29yayBBZG1pc3Np b24gQ29udHJvbDogVGhlIEltcGFjdA0KICAgICAgICAgICAgICBvZiBhIFNpbmdsZSBCaXQiLCAy MDA3Lg0KDQogICBbVFI0MzddICAgICJDb21wYXJpc29uIG9mIE1hcmtpbmcgQWxnb3JpdGhtcyBm b3IgUENOLUJhc2VkIEFkbWlzc2lvbg0KICAgICAgICAgICAgICBDb250cm9sLCBUZWNobmljYWwg UmVwb3J0IE5vLiA0MzcsIFVuaXZlcnNpdHkgb2YNCiAgICAgICAgICAgICAgV3VlcnpidXJnIiwg T2N0b2JlciAyMDA3Lg0KDQoNCkF1dGhvcnMnIEFkZHJlc3Nlcw0KDQogICBBbm5hIENoYXJueQ0K ICAgQ2lzY28gU3lzdGVtcywgSW5jLg0KICAgMTQxNCBNYXNzLiBBdmUuDQogICBCb3hib3JvdWdo LCBNQSAgMDE3MTkNCiAgIFVTQQ0KDQogICBFbWFpbDogYWNoYXJueUBjaXNjby5jb20NCg0KDQog ICBKb3NlcGggQmFiaWFyeg0KICAgTm9ydGVsDQogICAzNTAwIENhcmxpbmcgQXZlbnVlDQogICBP dHRhd2EsIE9udGFyaW8gIEsySCA4RTkNCiAgIENhbmFkYQ0KDQogICBFbWFpbDogYmFiaWFyekBu b3J0ZWwuY29tDQoNCg0KICAgTWljaGFlbCBNZW50aA0KICAgVW5pdmVyc2l0eSBvZiBXdWVyemJ1 cmcNCiAgIEluZm9ybWF0aWsgSUlJIEFtIEh1YmxhbmQNCiAgIFd1ZXJ6YnVyZywgICA5NzA3NA0K ICAgR2VybWFueQ0KDQogICBFbWFpbDogbWVudGhAbWVudGhAaW5mb3JtYXRpay51bmktd3Vlcnpi dXJnLmRlDQoNCg0KDQoNCg0KDQoNCg0KDQoNCkNoYXJueSwgZXQgYWwuICAgICAgICAgICAgRXhw aXJlcyBNYXkgMTQsIDIwMDggICAgICAgICAgICAgICAgIFtQYWdlIDMxXQ0KDA0KSW50ZXJuZXQt RHJhZnQgICAgICAgICAgICAgIENvbXBhcmlzb24gRHJhZnQgICAgICAgICAgICAgICBOb3ZlbWJl ciAyMDA3DQoNCg0KICAgSm95IFpoYW5nDQogICBDaXNjbyBTeXN0ZW1zLCBJbmMgJiBDb3JuZWxs IFVuaXZlcnNpdHkNCiAgIDE0MTQgTWFzcy4gQXZlLg0KICAgQm94Ym9yb3VnaCwgTUEgIDAxNzE5 DQogICBVU0ENCg0KICAgRW1haWw6IGFjaGFybnlAY2lzY28uY29tDQoNCg0KDQoNCg0KDQoNCg0K DQoNCg0KDQoNCg0KDQoNCg0KDQoNCg0KDQoNCg0KDQoNCg0KDQoNCg0KDQoNCg0KDQoNCg0KDQoN Cg0KDQoNCg0KDQoNCg0KQ2hhcm55LCBldCBhbC4gICAgICAgICAgICBFeHBpcmVzIE1heSAxNCwg MjAwOCAgICAgICAgICAgICAgICAgW1BhZ2UgMzJdDQoMDQpJbnRlcm5ldC1EcmFmdCAgICAgICAg ICAgICAgQ29tcGFyaXNvbiBEcmFmdCAgICAgICAgICAgICAgIE5vdmVtYmVyIDIwMDcNCg0KDQpG dWxsIENvcHlyaWdodCBTdGF0ZW1lbnQNCg0KICAgQ29weXJpZ2h0IChDKSBUaGUgSUVURiBUcnVz dCAoMjAwNykuDQoNCiAgIFRoaXMgZG9jdW1lbnQgaXMgc3ViamVjdCB0byB0aGUgcmlnaHRzLCBs aWNlbnNlcyBhbmQgcmVzdHJpY3Rpb25zDQogICBjb250YWluZWQgaW4gQkNQIDc4LCBhbmQgZXhj ZXB0IGFzIHNldCBmb3J0aCB0aGVyZWluLCB0aGUgYXV0aG9ycw0KICAgcmV0YWluIGFsbCB0aGVp ciByaWdodHMuDQoNCiAgIFRoaXMgZG9jdW1lbnQgYW5kIHRoZSBpbmZvcm1hdGlvbiBjb250YWlu ZWQgaGVyZWluIGFyZSBwcm92aWRlZCBvbiBhbg0KICAgIkFTIElTIiBiYXNpcyBhbmQgVEhFIENP TlRSSUJVVE9SLCBUSEUgT1JHQU5JWkFUSU9OIEhFL1NIRSBSRVBSRVNFTlRTDQogICBPUiBJUyBT UE9OU09SRUQgQlkgKElGIEFOWSksIFRIRSBJTlRFUk5FVCBTT0NJRVRZLCBUSEUgSUVURiBUUlVT VCBBTkQNCiAgIFRIRSBJTlRFUk5FVCBFTkdJTkVFUklORyBUQVNLIEZPUkNFIERJU0NMQUlNIEFM TCBXQVJSQU5USUVTLCBFWFBSRVNTDQogICBPUiBJTVBMSUVELCBJTkNMVURJTkcgQlVUIE5PVCBM SU1JVEVEIFRPIEFOWSBXQVJSQU5UWSBUSEFUIFRIRSBVU0UgT0YNCiAgIFRIRSBJTkZPUk1BVElP TiBIRVJFSU4gV0lMTCBOT1QgSU5GUklOR0UgQU5ZIFJJR0hUUyBPUiBBTlkgSU1QTElFRA0KICAg V0FSUkFOVElFUyBPRiBNRVJDSEFOVEFCSUxJVFkgT1IgRklUTkVTUyBGT1IgQSBQQVJUSUNVTEFS IFBVUlBPU0UuDQoNCg0KSW50ZWxsZWN0dWFsIFByb3BlcnR5DQoNCiAgIFRoZSBJRVRGIHRha2Vz IG5vIHBvc2l0aW9uIHJlZ2FyZGluZyB0aGUgdmFsaWRpdHkgb3Igc2NvcGUgb2YgYW55DQogICBJ bnRlbGxlY3R1YWwgUHJvcGVydHkgUmlnaHRzIG9yIG90aGVyIHJpZ2h0cyB0aGF0IG1pZ2h0IGJl IGNsYWltZWQgdG8NCiAgIHBlcnRhaW4gdG8gdGhlIGltcGxlbWVudGF0aW9uIG9yIHVzZSBvZiB0 aGUgdGVjaG5vbG9neSBkZXNjcmliZWQgaW4NCiAgIHRoaXMgZG9jdW1lbnQgb3IgdGhlIGV4dGVu dCB0byB3aGljaCBhbnkgbGljZW5zZSB1bmRlciBzdWNoIHJpZ2h0cw0KICAgbWlnaHQgb3IgbWln aHQgbm90IGJlIGF2YWlsYWJsZTsgbm9yIGRvZXMgaXQgcmVwcmVzZW50IHRoYXQgaXQgaGFzDQog ICBtYWRlIGFueSBpbmRlcGVuZGVudCBlZmZvcnQgdG8gaWRlbnRpZnkgYW55IHN1Y2ggcmlnaHRz LiAgSW5mb3JtYXRpb24NCiAgIG9uIHRoZSBwcm9jZWR1cmVzIHdpdGggcmVzcGVjdCB0byByaWdo dHMgaW4gUkZDIGRvY3VtZW50cyBjYW4gYmUNCiAgIGZvdW5kIGluIEJDUCA3OCBhbmQgQkNQIDc5 Lg0KDQogICBDb3BpZXMgb2YgSVBSIGRpc2Nsb3N1cmVzIG1hZGUgdG8gdGhlIElFVEYgU2VjcmV0 YXJpYXQgYW5kIGFueQ0KICAgYXNzdXJhbmNlcyBvZiBsaWNlbnNlcyB0byBiZSBtYWRlIGF2YWls YWJsZSwgb3IgdGhlIHJlc3VsdCBvZiBhbg0KICAgYXR0ZW1wdCBtYWRlIHRvIG9idGFpbiBhIGdl bmVyYWwgbGljZW5zZSBvciBwZXJtaXNzaW9uIGZvciB0aGUgdXNlIG9mDQogICBzdWNoIHByb3By aWV0YXJ5IHJpZ2h0cyBieSBpbXBsZW1lbnRlcnMgb3IgdXNlcnMgb2YgdGhpcw0KICAgc3BlY2lm aWNhdGlvbiBjYW4gYmUgb2J0YWluZWQgZnJvbSB0aGUgSUVURiBvbi1saW5lIElQUiByZXBvc2l0 b3J5IGF0DQogICBodHRwOi8vd3d3LmlldGYub3JnL2lwci4NCg0KICAgVGhlIElFVEYgaW52aXRl cyBhbnkgaW50ZXJlc3RlZCBwYXJ0eSB0byBicmluZyB0byBpdHMgYXR0ZW50aW9uIGFueQ0KICAg Y29weXJpZ2h0cywgcGF0ZW50cyBvciBwYXRlbnQgYXBwbGljYXRpb25zLCBvciBvdGhlciBwcm9w cmlldGFyeQ0KICAgcmlnaHRzIHRoYXQgbWF5IGNvdmVyIHRlY2hub2xvZ3kgdGhhdCBtYXkgYmUg cmVxdWlyZWQgdG8gaW1wbGVtZW50DQogICB0aGlzIHN0YW5kYXJkLiAgUGxlYXNlIGFkZHJlc3Mg dGhlIGluZm9ybWF0aW9uIHRvIHRoZSBJRVRGIGF0DQogICBpZXRmLWlwckBpZXRmLm9yZy4NCg0K DQpBY2tub3dsZWRnbWVudA0KDQogICBGdW5kaW5nIGZvciB0aGUgUkZDIEVkaXRvciBmdW5jdGlv biBpcyBwcm92aWRlZCBieSB0aGUgSUVURg0KICAgQWRtaW5pc3RyYXRpdmUgU3VwcG9ydCBBY3Rp dml0eSAoSUFTQSkuDQoNCg0KDQoNCg0KQ2hhcm55LCBldCBhbC4gICAgICAgICAgICBFeHBpcmVz IE1heSAxNCwgMjAwOCAgICAgICAgICAgICAgICAgW1BhZ2UgMzNdDQoMDQoNCg0K ------_=_NextPart_001_01C82162.48452C1D Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn ------_=_NextPart_001_01C82162.48452C1D-- From pcn-bounces@ietf.org Wed Nov 07 12:35:46 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Ipooc-0008JJ-1q; Wed, 07 Nov 2007 12:35:46 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Ipooa-0008HM-VL for pcn-confirm+ok@megatron.ietf.org; Wed, 07 Nov 2007 12:35:44 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Ipooa-0008HE-Lt for pcn@ietf.org; Wed, 07 Nov 2007 12:35:44 -0500 Received: from sj-iport-1-in.cisco.com ([171.71.176.70] helo=sj-iport-1.cisco.com) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IpooX-0001Ie-DM for pcn@ietf.org; Wed, 07 Nov 2007 12:35:44 -0500 X-IronPort-AV: E=Sophos;i="4.21,385,1188802800"; d="scan'208";a="30885686" Received: from sj-dkim-2.cisco.com ([171.71.179.186]) by sj-iport-1.cisco.com with ESMTP; 07 Nov 2007 09:35:37 -0800 Received: from sj-core-3.cisco.com (sj-core-3.cisco.com [171.68.223.137]) by sj-dkim-2.cisco.com (8.12.11/8.12.11) with ESMTP id lA7HZef5022904; Wed, 7 Nov 2007 09:35:40 -0800 Received: from xbh-rtp-201.amer.cisco.com (xbh-rtp-201.cisco.com [64.102.31.12]) by sj-core-3.cisco.com (8.12.10/8.12.6) with ESMTP id lA7HZFeb006332; Wed, 7 Nov 2007 17:35:36 GMT Received: from xmb-rtp-203.amer.cisco.com ([64.102.31.20]) by xbh-rtp-201.amer.cisco.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 7 Nov 2007 12:35:26 -0500 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Wed, 7 Nov 2007 12:35:25 -0500 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Please comment on the comparison draft Thread-Index: AcghZJS+OPLOt7yQS0ybQh33Wft4kg== From: "Anna Charny (acharny)" To: X-OriginalArrivalTime: 07 Nov 2007 17:35:26.0869 (UTC) FILETIME=[957FDC50:01C82164] X-TM-AS-Product-Ver: SMEX-8.0.0.1181-5.000.1023-15530.002 X-TM-AS-Result: No--11.820400-8.000000-31 X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No DKIM-Signature: v=0.5; a=rsa-sha256; q=dns/txt; l=1445; t=1194456940; x=1195320940; c=relaxed/simple; s=sjdkim2002; h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version; d=cisco.com; i=acharny@cisco.com; z=From:=20=22Anna=20Charny=20(acharny)=22=20 |Subject:=20Please=20comment=20on=20the=20comparison=20draft |Sender:=20; bh=9urazwRaOqegxfrVy+Ug2VbidGs9ApDDDFTPMwxSKdM=; b=NERvYc7on//BcNeHm6On3J1nmx1WR/EdDN9sy7wXGeU3VHVVhXdVsYrIdsec/JD4MHdtLjcs 3zLYXzCFnrfXmUTVf6OzdxkJwwB8HJujEeAYPFV8RyWCsUt7jP80vVu7; Authentication-Results: sj-dkim-2; header.From=acharny@cisco.com; dkim=pass ( sig from cisco.com/sjdkim2002 verified; ); X-Spam-Score: -4.0 (----) X-Scan-Signature: b19722fc8d3865b147c75ae2495625f2 Cc: "Joy Zhang \(joyzhang\)" Subject: [PCN] Please comment on the comparison draft X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Dear all, We have just submitted the PCN Comparison draft. Please take a look and send us comments/suggestions. We may still have a chance to send the next version before the 01 deadline if necessary. We would like to specifically ask the WG group to comment on the specific issue, discussed below. In Sections 11-14 we discuss the possibility of specifying a unified (and so more complex) description of the pcn-internal-node behavior that could, with an appropriate parameterization, support several of the proposed approaches, and support several edge behaviors. We would like to poll the WG opinion on whether the benefits do (or do not) outweigh the complexity, and - consequently - whether the WG does need to make a choice between the different metering/marking behaviors or can go for the unified behavior. If we do end up wanting to make the choice, we would like to request feedback on whether the choice can be made based on the available data so far, or suggestions on additional criteria/evaluation. Finally we do apologize that the draft does not currently discuss draft-westberg-. We will have to address this when the questions/concerns on its operation raised on the list are resolved (hopefully in the next version of draft-westberg). Also, thanks to Phil Eardley for his useful comments to an early version of the draft.=20 Thank you, Anna (on behalf of the rest of the authors) =20 _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 08 11:08:43 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iq9vu-0002j6-Rf; Thu, 08 Nov 2007 11:08:42 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Iq9vt-0002iq-Vh for pcn-confirm+ok@megatron.ietf.org; Thu, 08 Nov 2007 11:08:41 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iq9vt-0002ht-5r for pcn@ietf.org; Thu, 08 Nov 2007 11:08:41 -0500 Received: from zrtps0kn.nortel.com ([47.140.192.55]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1Iq9vs-0003eK-OE for pcn@ietf.org; Thu, 08 Nov 2007 11:08:40 -0500 Received: from zcarhxm1.corp.nortel.com (zcarhxm1.corp.nortel.com [47.129.230.97]) by zrtps0kn.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id lA8G8ai28078; Thu, 8 Nov 2007 16:08:36 GMT X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [PCN] 1st call - agenda requests for Vancouver Date: Thu, 8 Nov 2007 11:08:26 -0500 Message-ID: <9671A92C3C8B5744BC97F855F7CB6465130CCF86@zcarhxm1.corp.nortel.com> In-Reply-To: <20071107033818.610D261B673@newdev.eecs.harvard.edu> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] 1st call - agenda requests for Vancouver Thread-Index: Acgg8A6E/0Dl7dlzReepgyPQ2wvAVgBL/fjg References: <20071107033818.610D261B673@newdev.eecs.harvard.edu> From: "Jozef Babiarz" To: "Scott O. Bradner" , X-Spam-Score: 0.0 (/) X-Scan-Signature: 69a74e02bbee44ab4f8eafdbcedd94a1 Cc: X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Scott, I would like to request=20 1) a slot for provide an update to Three State PCN Marking; darft-babiarz-pcn-3sm 2) a slot to provide additional Simulation Results for 3sm; draft-babiarz-pcn-explicit-marking=20 Updates to the above drafts will be posted before the cutoff date. Regards, Joe email:babiarz@nortel.com Telephone:613-763-6098 -----Original Message----- From: Scott O. Bradner [mailto:sob@harvard.edu]=20 Sent: November 6, 2007 10:38 PM To: pcn@ietf.org Subject: [PCN] 1st call - agenda requests for Vancouver this is a first call for timeslots on the PCN agenda in Vancouver we are currently scheduled for wed afternoon from 1300-1610 we expect that a good chunk of the time will be spent on discussions of the Architecture draft but if others have other (in-charter) topics please let us know Scott & Steve _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Fri Nov 09 07:38:49 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IqT8K-0006mK-UK; Fri, 09 Nov 2007 07:38:48 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IqT8J-0006kq-FS for pcn-confirm+ok@megatron.ietf.org; Fri, 09 Nov 2007 07:38:47 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IqT8J-0006ki-1z for pcn@ietf.org; Fri, 09 Nov 2007 07:38:47 -0500 Received: from rotterdam.ewi.utwente.nl ([130.89.10.5]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IqT8F-0000dC-SE for pcn@ietf.org; Fri, 09 Nov 2007 07:38:46 -0500 Received: from webmail.cs.utwente.nl (janus.ewi.utwente.nl [130.89.10.26]) by rotterdam.ewi.utwente.nl (8.13.6/8.13.6) with SMTP id lA9CcVjG024322; Fri, 9 Nov 2007 13:38:35 +0100 (MET) Received: from 213.16.183.10 (auth. user karagian@imap1.ewi.utwente.nl) by webmail.cs.utwente.nl with HTTP; Fri, 09 Nov 2007 12:38:31 +0000 To: "Anna Charny (acharny)" , "Lars Westberg (KI/EAB)" , "anurag.bhargava@ericsson.com" Date: Fri, 09 Nov 2007 12:38:29 +0000 X-Mailer: IlohaMail/0.8.13 (On: webmail.cs.utwente.nl) Message-ID: <4Rsu8mxG.1194611909.3868340.karagian@ewi.utwente.nl> In-Reply-To: From: "Georgios Karagiannis" Bounce-To: "Georgios Karagiannis" MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0 () X-Scanned-By: MIMEDefang 2.52 on 130.89.10.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0rc3 (rotterdam.ewi.utwente.nl [130.89.10.5]); Fri, 09 Nov 2007 13:38:40 +0100 (MET) X-Spam-Score: 1.8 (+) X-Scan-Signature: 946b615681bce2b72aace5af8d0d8a1d Cc: "pcn@ietf.org" Subject: [PCN] RE: Questions on LC-PCN draft version 01 X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Anna Sorry for the quick response! I think that you are right! What we had to emphasize is that the Admission_offset_rate at PCN_egress_nodes is different than the Admission_offset_rate used at the PCN_interior_nodes. If we will consider as normal situations the situations that no ECMP occurs and that all flows belonging to the same ingress-egress aggregate will use the same path from PCN_ingress to PCN_egress, this will mean that when the PCN_egress_node receives, for the given ingress-egress aggregate an excess rate equal to a fraction of the Admission_offset_rate, say fraction F * Admission_offset_rate, where "fraction used for admission control" > F >=3D 1, it will have to change from admission control state to flow termination state. Note that F can be preconfigured and depends on the network topology. Best regards, Georgios On 11/2/2007, "Anna Charny (acharny)" wrote: >Hi Georgios, > >You are saying that the example I provided was a worst case scenario >that cannot happen in practice. =20 > >Why is that? All that is required to construct a similar example is to >have a bottleneck which multiplexes traffic from many ingresses to one >(or a small number of) egresses, and these aggregates can together load >the bottleneck link. This scenario will easily result in the situation >when the admission-offset on the bottleneck (i.e. the difference between >the admission and termination thresholds) is large compared to the >individual ingress-egress aggregate. Why do you think that is a corner >case?=20 > >Anna=20 >=20 > >> -----Original Message----- >> From: Georgios Karagiannis [mailto:karagian@cs.utwente.nl]=20 >> Sent: Friday, November 02, 2007 4:05 PM >> To: Anna Charny (acharny); Lars Westberg (KI/EAB);=20 >> anurag.bhargava@ericsson.com >> Cc: pcn@ietf.org >> Subject: RE: Questions on LC-PCN draft version 01 >>=20 >> Hi Anna >>=20 >> Please see in line! >>=20 >> Best regards, >> Georgios >>=20 >> On 11/2/2007, "Anna Charny (acharny)" wrote: >>=20 >> >Hi Georgios, >> > >> >OK, trying again... >> > >> >Something still seems wrong.=20 >> > >> >First wanted to confirm that I now got the setting of the thresholds=20 >> >right (the following collected from your various responses): >> > >> >NOTE: I am ignoring multi-congestion error because I am=20 >> still trying=20 >> >to figure out how it works in a single bottleneck case. >> > >> >Pcn-lower-egress-percentage is some configured value (%) >> > >> >Pcn-lower-egress-rate =3D pcn-lower-egress-percentage *=20 >> >admission-termination-rate/100 (I assume the division by 100 is >> >necessary?) >> > >> >Pcn-upper-egress-rate=3Dpcn-lower-egress-rate+Admission-offset-rate >> > >> >Admisison-offset-rate is a global parameter that has the=20 >> meaning of the=20 >> >difference between the pcn-upper-rate and pcn-lower-rate at all=20 >> >interior nodes (which is assumed the same in all cases). >> > >> >Egress goes into termination mode when it sees the rate of marked=20 >> >packets of a given ingress exceeding the pcn-upper-egress-rate. >> > >> >So far so good? >> > >> >Assuming it is all correct, my example where none of the=20 >> egresses can=20 >> >ever go into the termination mode remains valid, I believe. Let me=20 >> >restate the example. >> > >> >Consider the following example. A bottleneck link of=20 >> capacity 100 mbps=20 >> >, pcn-termination-offset of 40mbps and pcn-admission-offset=20 >> of 30 mbps=20 >> >is shared by 100 ingress-egress aggregates, each with the total pcn=20 >> >rate of >> >1 mbps. On this link pcn-upper-threshold is 60mbps, and=20 >> >pcn-lower-threshold is 30mbps. Suppose further that all 100=20 >> >ingress-egress aggregates go to different egress nodes (and=20 >> suppose for=20 >> >simplicity there is no other traffic going to these=20 >> egresses; this last=20 >> >assumption is irrelevant because the egress measurement is per=20 >> >ingress-egress pair anyway, but makes it convenient to imagine). The=20 >> >bottleneck link is clearly in the termination state, as the=20 >> total rate=20 >> >of pcn traffic on this link is 100 mbps which is above the=20 >> >pcn-upper-threshold. >> > >> >This means we want the egress nodes to somehow recognise=20 >> this state and=20 >> >get into the termination mode. But it seems in this example the=20 >> >egresses >> > >> >can't ever recognize that the system is in the termination mode. See=20 >> >below. >>=20 >> Georgios: I think that the example that you provide is an=20 >> wors case scenario. >> First of all the Admission_offset_rate is too high. >> This Admission_offset_rate should be set such that siuations=20 >> as you describe will not occur. >>=20 >>=20 >> > >> >Suppose pcn-lower-egress-percentage is set to w% (0> >u=3Dw/100. Then, each of these egresses will compute the=20 >> >pcn-lower-egress-rate=3Du*admission-offset-rate >> > >> >and >> > >> >pcn-upper-egress-rate=3Dpcn-lower-rate+admission-offset-rate =3D=20 >> >(u+1)*admission-offset-rate > admission-offset-rate =3D 30Mbps >> > >> > >> >So our pcn-upper-egress rate is always greater than 30 Mbps,=20 >> while the=20 >> >entire rate of pcn traffic the egress will ever see is *at >> >most* 1 Mbps (because it is plainly the total rate of the single=20 >> >ingress-egress aggreate that goes to this egress). So, regardless of=20 >> >the >> > >> >actual amount of marked traffic in this egress-egress aggregate, the=20 >> >absolute excess rate of this ingress-egress aggregate will never be=20 >> >above 1mbps either.In turn that means that the pcn-upper-egress-rate=20 >> >will NEVER be exceeded (because pcn-upper-egress-rate is so=20 >> much larger=20 >> >than the total rate of the pcn traffic at the egress). In=20 >> turn, that=20 >> >appears to mean that in this scenario none of the egresses will ever=20 >> >end up in termination state, and so termination will never=20 >> occur. That >> >(still) does not seem right. >> > >> > >> >So again, even with the corrected definition of pcn-lower-thershold,=20 >> >this example seems to imply a rather fundamental flaw in the=20 >> algorithm. >> > >> >You say, in your response to the previous example, that >> > >> >>the situation that you described cannot (often) occur because the=20 >> >>difference between the admission control threshold and flow=20 >> >>termination threshold at the egress is equal to =20 >> Admission_offset_rate=20 >> >>+/- multicongestion_error. >> > >> >I believe that multi-congestion error is irrelevant here, because in=20 >> >the considered example there is only a single congestion point. >> > >> >>Note that >> >> the Admission_offset_rate is an absolute rate value. The =20 >> >>multicongestion_error is used to identify the bounds that certain=20 >> >>situations occur where the PCN_egreess_ node should have=20 >> been in flow=20 >> >>termination state but it is not. Please see previous=20 >> discussions on=20 >> >>this. >> > >> >I do not understand the relevance of the multiocongestion=20 >> error in this=20 >> >case, or rules for its setting at all, I am afraid. Do we need=20 >> >multi-congestion error (which I do not know how to set=20 >> anyway) to deal=20 >> >with a single congestion point also? What should it be set=20 >> to? Do you=20 >> >need to examine a configuration AND all ingress-egress=20 >> aggregare rates=20 >> >AND all possible failures to set it correctly? >>=20 >>=20 >> Georgios: You are right that the muli-congestion-error does=20 >> not play a role, but the selection of the=20 >> Admission_offset_rate plays in this situation a signifficat=20 >> role and it should be selected such that wores case=20 >> scenarios, such as the one described by you do not occur+ >>=20 >>=20 >> >Best, >> >Anna >> >> -----Original Message----- >> >> From: Georgios Karagiannis [mailto:karagian@cs.utwente.nl] >> >> Sent: Friday, November 02, 2007 12:20 PM >> >> To: Anna Charny (acharny); anurag.bhargava@ericsson.com; Lars=20 >> >> Westberg (KI/EAB) >> >> Cc: pcn@ietf.org >> >> Subject: RE: Questions on LC-PCN draft version 01 >> >>=20 >> >> Hi Anna >> >>=20 >> >> Thank you very much for your comments! >> >>=20 >> >> Please see in line! >> >>=20 >> >>=20 >> >> On 11/1/2007, "Anna Charny (acharny)" wrote: >> >>=20 >> >> > >> >> >Hi Georgios, >> >> > >> >> >A few more clarification questions: >> >> > >> >> >> -----Original Message----- >> >> >> From: Georgios Karagiannis [mailto:karagian@cs.utwente.nl] >> >> >> Sent: Thursday, November 01, 2007 2:35 AM >> >> >> To: Anna Charny (acharny); anurag.bhargava@ericsson.com; Lars=20 >> >> >> Westberg (KI/EAB) >> >> >> Cc: pcn@ietf.org >> >> >> Subject: Re: Questions on LC-PCN draft version 01 >> >> >> >> >> >> Hi Anna >> >> >> >> >> >> Thank you very much for your comments! >> >> >> Before answering to your comments, see in line below, I >> >> would like to >> >> >> explain in an abstract way the LC-PCN algorithm. >> >> >> >> >> >> * Setting the thresholds at PCN_interior_nodes: >> >> >> ----------------------------------------------- >> >> >> In order to calculate the PCN_upper_rate we use two parameters: >> >> >> Maximum PHB capacity: that is the maximum capacity that can be=20 >> >> >> supported by a PCN_interior_node >> >> >> Termination_offset_rate: that is an absolute rate value >> >> that should >> >> >> be set equal into all PCN_interior_nodes. Note that=20 >> this value is=20 >> >> >> used by PCN_interior_nodes to calculate their >> >> PCN_upper_rate and also >> >> >> during the situation that a PCN_interior_node is in flow >> >> termination >> >> >> state and it receives PCN_marked packets. Please see >> >> pseudo code on >> >> >> page 20. >> >> >> This value must be set equal into all=20 >> PCN_interior_nodes such that=20 >> >> >> all these nodes will know when to take into account the=20 >> incoming=20 >> >> >> PCN_marked packets and when not. >> >> >> >> >> >> The PCN_upper_rate is then found as: >> >> >> PCN_upper_rate =3D "Maximum PHB capacity" -=20 >> Termination_offset_rate >> >> >> >> >> > >> >> >So if you have a link of 10 Mbps and a link of 40 Mbps=20 >> (say each is=20 >> >> >allowed to use all bandwidth for PCN, for simplicity), then >> >> they both >> >> >use the same *absolote* value of the termination-offset-rate?. >> >>=20 >> >> Georgios: Yes, you are right! This has to do with the=20 >> importance of=20 >> >> using the Termination_offset_rate. We will motivate this in the=20 >> >> following version of the draft. >> >>=20 >> >> > So If I >> >> >wanted to have PCN-upper-rate 20% below my link capacity on >> >> all links, >> >> >I would not be able to do so with this approach? A larger=20 >> link would=20 >> >> >have a smaller relative safety margin, then... >> >>=20 >> >> Georgios: Yes, this can be considered as a disadvantage.=20 >> >> However, there might be other ways of preconfiguring each=20 >> >> PCN_interior_node to know the termination_offset rate used by each=20 >> >> neighbour PCN_interior_node. Then the pseudocode on page=20 >> 20 will have=20 >> >> to use the Termination_offset_rate associated with the=20 >> incoming link=20 >> >> from where the incoming PCN_marked packets are arriving. Then this=20 >> >> constraint can be avoided. >> >>=20 >> >>=20 >> >> >Same comment for the >> >> >difference between admission and termination - the relative >> >> difference >> >> >between admission and termination threshods (Pcn-lower-rate and >> >> >pcn-upper-rate) is global, so the larger links have a >> >> smaller relative >> >> >difference between admission and termination thresholds. Right? >> >>=20 >> >> Georgios: No, because the Admission_offset_rate is an=20 >> absolute value! >> >> So this difference is globally equal. >> >>=20 >> >> > >> >> >> The PCN_lower_rate is configured in all PCN_interior-nodes >> >> and it can >> >> >> be calculated in the following way: >> >> >> PCN_lower_rate =3D PCN_upper_rate - Admission_offset_rate >> >> >> >> >> >> The Admission_offset_rate is an absolute rate value and it >> >> is equal >> >> >> in all PCN_interior_nodes and PCN_egress_nodes. Note that >> >> this value >> >> >> is used by PCN_interior_nodes to calculate their >> >> PCN_lower_rate and >> >> >> the PCN_egress_nodes to calculate their PCN_upper_rate_egress.=20 >> >> >> Furthermore, this value is used by the PCN_interior_nodes >> >> also during >> >> >> the situation that a PCN_interior_node is in admission >> >> control state >> >> >> and it receives PCN_marked packets. Please see pseudo=20 >> code on page=20 >> >> >> 13. >> >> >> This value must be set equal into all=20 >> PCN_interior_nodes such that=20 >> >> >> all these nodes will know when to take into account the=20 >> incoming=20 >> >> >> PCN_marked packets and when not. Note that a=20 >> PCN_interior_node can=20 >> >> >> PCN_mark packets up to an excess rate equal to the=20 >> >> >> Admission_offset_rate. If the exess rate in an >> >> PCN_interior_node is >> >> >> higher than the Admission_offset_rate, then the=20 >> PCN_interior_node=20 >> >> >> changes state from admission control state to flow >> >> termination state. >> >> > >> >> >OK. >> >> > >> >> >> >> >> >> * Setting the thresholds at PCN_egress_nodes: >> >> >> --------------------------------------------- >> >> >> The question is how to calculate the threshold that=20 >> defines when a=20 >> >> >> PCN_egress_node goes into the admission control state. >> >> >> One way to do that is to consider that when the PCN_egress_node=20 >> >> >> receives a PCN_marked packet it will mean that at least one=20 >> >> >> PCN_interior_node started to be admission control congested and=20 >> >> >> therefore it will go from Normal state to admission >> >> control state. Of >> >> >> course this will somehow might provide some errors,=20 >> because there=20 >> >> >> might be situations that this consideration might be to >> >> conservative.=20 >> >> >> Therefore, we use a percentage of received PCN_marking encoded=20 >> >> >> packets in proportion to total rate of received=20 >> packets. In this=20 >> >> >> version of the draft we call this value as: >> >> >> PCN_lower_rate_egress. This is wrong. >> >> >> What we should say is: >> >> >> PCN_lower_rate_egress: is equal to=20 >> incoming_PCN_marking_rate, when: >> >> >> (incoming_PCN_marking_rate/measured PHB rate) =3D to a=20 >> preconfigured=20 >> >> >> percentage, say 1%. From now on I denote this percentage as: >> >> >> PCN_lower_percentage_egress. >> >> > >> >> >In the draft, pcn_lower- and -upper_rate_egress seem to be >> >> defined on a >> >> >per *ingress-egress-pair basis*. >> >> >Is that correct? >> >>=20 >> >> georgios: Yes, it is correct! >> >> > >> >> >> Thus we can say that a PCN_egress_node changes from Normal >> >> state to >> >> >> admission control state when >> >> incoming_PCN_marking_rate/measured PHB >> >> >> rate > PCN_lower_percentage_egress. >> >> >> Where, >> >> >> incoming_PCN_marking_rate =3D N *=20 >> input_PCN_marking_bytes/T Where,=20 >> >> >> input_PCN_marking_bytes =3D received number of=20 >> "PCN_marking" encoded=20 >> >> >> packets during measurement period T. >> >> > >> >> >Again, is it per ingress-egress pair, or not? >> >>=20 >> >> georgios: Yes, it is correct! >> >>=20 >> >> > >> >> >> >> >> >> Now we defined the condition that the PCN_egress_node >> >> changes state >> >> >> from Normal state to admission control state. But how will the=20 >> >> >> PCN_egress_node change state from admission control=20 >> state to flow=20 >> >> >> termination state. >> >> >> In order to explain this, it is imporatnt to note that each=20 >> >> >> PCN_interior_node that is in admission control state it >> >> can PCN_mark >> >> >> packets up to a value equal to Admission_offset_rate.=20 >> >> Furthermore, if >> >> >> a PCN_interior_node receives incoming PCN_marked=20 >> packets and is in=20 >> >> >> the addmission control state, it will not remark any >> >> packets if the >> >> >> excess rate is equal or lower than the >> >> incoming_PCN_marking_rate, see >> >> >> page 13. >> >> >> Furthermore, if we will consider as normal situations the >> >> situations >> >> >> that no ECMP occurs and that all flows belonging to the same=20 >> >> >> ingress-egress aggregate will use the same path from >> >> PCN_ingress to >> >> >> PCN_egress, this will mean that when the=20 >> PCN_egress_node receives,=20 >> >> >> for the given ingress-egress aggregate an excess rate equal to=20 >> >> >> Admission_offset_rate it will have to change from >> >> admission control >> >> >> state to flow termination state. >> >> >> Thus in this case the second threshold, that in this case >> >> is a rate >> >> >> and not a percentage, can be calculated as follows: >> >> >> PCN_upper_egress_rate =3D PCN_lower_egress_rate + >> >> Admission_offset_rate. >> >> > >> >> > >> >> >Here is where I am afraid I am fundamentally confused. >> >> >Pcn_lower-egress_rate seems to have beed redefined above as a=20 >> >> >percentage threshold, but it is an absolute rate again=20 >> here. Perhaps=20 >> >> >you just mean that Pcn-lower-egress-rate =3D >> >> pcn_lower_egress_percentage >> >> >* total-measured-rate-of-this-ingress-egrees-aggregate/100? >> >> >So pcn-lower-egress-rate then is just the fraction of the >> >> total rate of >> >> >the ingress-egress aggregate corresponding to the configured >> >> percentage >> >> >threshold. Right? >> >>=20 >> >> Georgios: I think I could not explain and translate what I=20 >> wanted to=20 >> >> do in the right way! >> >> A better way of expressing the need of using a rate of=20 >> percentage of=20 >> >> the received PCN_marking encoded packets, would be the following: >> >>=20 >> >> Pcn-lower-egress-rate =3D pcn_lower_egress_percentage *=20 >> >> Admission_offset_rate >> >>=20 >> >>=20 >> >> This means that event_A in Section 4.2.3 will be activated when=20 >> >> incoming_PCN_marking_rate > pcn_lower_egress_percentage >> >> * Admission_offset_rate or when: >> >> incoming_PCN_marking_rate > Pcn-lower-egress-rate >> >>=20 >> >> This means that the PCN_egress_ node does not change to admission=20 >> >> control state when at least one PCN_marked packet arrives, but it=20 >> >> changes when the incoming_PHB_marking rate equals a=20 >> percentage of the=20 >> >> Admission_offset_rate. This is due to the fact that=20 >> >> Admission_offset_rate is used by the interioe nodes to change from=20 >> >> admission control state to flow termination state. >> >>=20 >> >> > >> >> >Assuming the above is correct, I cannot see how the=20 >> system will work=20 >> >> >correctly. Consider the following example. A bottleneck link of=20 >> >> >capacity 100 mbps , pcn-termination-offset of 40mbps and=20 >> >> >pcn-admission-offset of 30 mbps is shared by 100 ingress-egress=20 >> >> >aggregates, each with the total pcn rate of 1 mbps. On this link=20 >> >> >pcn-upper-threshold is 60mbps, and pcn-lower-threshold is 30mbps. >> >> >Suppose further that all 100 ingress-egress aggregates go to >> >> different >> >> >egress nodes (and suppose for simplicity there is no=20 >> other traffic=20 >> >> >going to these egresses). The bottleneck link is clearly in the=20 >> >> >termination state, as the total rate of pcn traffic on this >> >> link is 100 >> >> >mbps which is above the pcn-upper-threshold. This means=20 >> we want the=20 >> >> >egress nodes to somehow recognise this state and get into the=20 >> >> >termination mode. But it seems in this example the egresses >> >> can't ever >> >> >recognize that the system is in the termination mode. See below. >> >>=20 >> >>=20 >> >> > >> >> >Suppose pcn-lower-egress-percentage is set to w% (0> >> Then, each >> >> >of these egresses will compute the (absolute) >> >> pcn-lower-egress-rate as >> >> >total pcn traffic of the ingress-egress aggregate it sees=20 >> (which is=20 >> >> >1 >> >> >mbps) times the pcn_lower_egress_percentage/100, so it will get=20 >> >> >pcn-lower-egress-rate=3D w* 0.01 Mbps. >> >> >According to your explanation above, it will then compute=20 >> >> >Pcn-upper-egress-rate=3Dpcn-lower-egress-rate+Admission-offset-rate=3D >> >> >w*0.01 mbps+30mbps >=3D 30 mbps (for all possible settings of w). >> >> > >> >> >So our pcn-upper-egress rate is always greater than 30 Mbps, >> >> while the >> >> >entire rate of pcn traffic the egress will ever see is *at >> >> most* 1 Mbps >> >> >(because it is plainly the total rate of the single=20 >> ingress-egress=20 >> >> >aggreate that goes to this egress). So, regardless of the >> >> actual amount >> >> >of marked traffic in this egress-egress aggregate, the >> >> absolute excess >> >> >rate of this ingress-egress aggregate will never be above >> >> 1mbps either. >> >> >In turn that means that the pcn-upper-egress-rate will NEVER be=20 >> >> >exceeded (because pcn-upper-egress-rate is so much larger=20 >> than the=20 >> >> >total rate of the pcn traffic at the egress). In turn, that >> >> appears to >> >> >mean that in this scenario none of the egresses will ever=20 >> end up in=20 >> >> >termination state, and so termination will never occur. That >> >> does not seem right. >> >>=20 >> >>=20 >> >> Georgios: Thank you for the given example, which stimulated me to=20 >> >> better translate what I wanted to specify and what I have written=20 >> >> down. >> >> Please see above that I defned: >> >> Pcn-lower-egress-rate =3D pcn_lower_egress_percentage *=20 >> >> Admission_offset_rate >> >>=20 >> >> The situation that you described cannot (often) occur because the=20 >> >> difference between the admission control threshold and flow=20 >> >> termination threshold at the egress is equal to=20 >> Admission_offset_rate=20 >> >> +/- multicongestion_error. Note that the=20 >> Admission_offset_rate is an=20 >> >> absolute rate value. The multicongestion_error is used to identify=20 >> >> the bounds that certain situations occur where the=20 >> PCN_egreess_ node=20 >> >> should have been in flow termination state but it is not.=20 >> Please see=20 >> >> previous discussions on this. >> >>=20 >> >> > >> >> >I would appreciate any help in clarifying this. >> >> > >> >> >Anna >> >> > >> >> > >> >> >> However, there are some corner cases, that mainly occur=20 >> when the=20 >> >> >> different congestion points (admission control congested >> >> >> PCN_interior_nodes) on the same path are not >> >> simulataneously starting >> >> >> to be congested. Therefore we use the >> >> multicongestion_error parameter >> >> >> to identify the error bound that ocurs due to these=20 >> corner cases.=20 >> >> >> Note that this error bound can be e.g., predefined ones >> >> off line by >> >> >> the operator, by studying the network topology and/or=20 >> studying how=20 >> >> >> often such corner cases could occur and/or doing off line=20 >> >> >> measurements. Therefore we use: >> >> >> PCN_upper_rate_egress =3D PCN_lower_rate_egress + >> >> Admission_offset_rate >> >> >> +/- multicongestion_error >> >> >> >> >> >> * How the states of operation in PCN_interior_nodes are >> >> being changed? >> >> >>=20 >> >>=20 >> --------------------------------------------------------------------- >> >> >> This is explained on page 17, 18, by using Figure 4. >> >> >> >> >> >> Change from Normal state to Admission control state: event >> >> A Occurs >> >> >> when: >> >> >> Measured PHB rate > PCN_lower_rate >> >> >> >> >> >> Change from Admission control state to Flow Termination >> >> >> state: event B Occurs when: >> >> >> Measured PHB rate > PCN_upper_rate >> >> >> >> >> >> * How the states of operation in PCN_egress_nodes are >> >> being changed? >> >> >>=20 >> >>=20 >> --------------------------------------------------------------------- >> >> >> This is explained on page 21, 22. Note that the >> >> description of event >> >> >> A on page 21 has to be modified to avoid the confusions=20 >> that were=20 >> >> >> caused up to now, see explanation given above. >> >> >> >> >> >> Change from Normal state to Admission control state: event >> >> A Occurs >> >> >> when: >> >> >> incoming_PCN_marking_rate/measured PHB rate) >=20 >> >> >> PCN_lower_percentage_egress As explained above: >> >> >> IF ((incoming_PCN_marking_rate/measured PHB rate) =3D >> >> >> PCN_lower_percentage_egress) >> >> >> THEN PCN_lower_egress_rate =3D incoming_PCN_marking_rate >> >> >> >> >> >> Change from Admission control state to flow termination >> >> >> state: event B Occurs when: >> >> >> incoming_PCN_marking_rate > PCN_upper_rate_egress >> >> >> >> >> >> It is important to note that also the explanation of event >> >> C on page >> >> >> 21, has to be modified to avoid the confusions that were >> >> caused up to >> >> >> now, see explanation given above: >> >> >> Change from Admission control state to Normal state: >> >> >> Occurs when: >> >> >> incoming_PCN_marking_rate/measured PHB rate) =3D<=20 >> >> >> PCN_lower_percentage_egress >> >> >> >> >> >> >> >> >> * Generated excess rate by an PCN_interior_node operating in=20 >> >> >> admission control state. >> >> >> ------------------------------------------------------------- >> >> >> >> >> >> The excess rate =3D signaled_overload_rate. >> >> >> The maximum excess rate that a PCN_interior_node can=20 >> calculate in=20 >> >> >> admission control state is equal to Admission_offset_rate, see=20 >> >> >> explanation above. >> >> >> The number of bytes that are remarked, >> >> signaled_remarked_bytes depend >> >> >> on the value of calculated excess rate >> >> (signaled_overload_rate), the >> >> >> value of the Admission_offset_rate and the value of the=20 >> >> >> incoming_PCN_marking_rate, see pseudo code on page 13. >> >> >> >> >> >> Note that all packets that are passing through a congested=20 >> >> >> PCN_interior_node an are not being PCN_marked by the=20 >> >> >> PCN_interior_node have to be remarked using the >> >> PCN_Affected_marking. >> >> >> >> >> >> >> >> >> Regarding probes, the probe packets that are passing through a=20 >> >> >> congested node are either PCN_marked or PCN_Affected_marked. >> >> >> >> >> >> * Generating excess rate by an PCN_interior_node=20 >> operating in flow=20 >> >> >> termination state: >> >> >> ---------------------------------------------------------------- >> >> >> The excess rate =3D signaled_overload_rate. >> >> >> Note that the calculation of the signaled_overload_rate is >> >> different >> >> >> than in the situation that the PCN_Interior_node operates in=20 >> >> >> admission control state, see page 19 and 20. This is due >> >> to the fact >> >> >> that a sliding window is used to solve an undershooting >> >> problem, see >> >> >> discussion on page 19. >> >> >> The number of bytes that are remarked, >> >> signaled_remarked_bytes depend >> >> >> on the value of calculated excess rate >> >> (signaled_overload_rate), the >> >> >> value of the Termination_offset_rate and the value of the=20 >> >> >> incoming_PCN_marking_rate, and the see pseudo code on page 20. >> >> >> >> >> >> Note that all packets that are passing through a congested=20 >> >> >> PCN_interior_node an are not being PCN_marked by the=20 >> >> >> PCN_interior_node have to be remarked using the >> >> PCN_Affected_marking. >> >> >> >> >> >> * Providing admission control at PCN_egress_nodes: >> >> >> -------------------------------------------------- >> >> >> When the PCN_egress_node is operating in admission control >> >> state than >> >> >> a flow that is requesting admission into the PCN domain can b=20 >> >> >> etreated in the following way: >> >> >> >> >> >> If no probing is used, the request for admission can be >> >> accomplished >> >> >> by using an external to PCN signaling protocol. >> >> >> In this case when the request arrives at a PCN_egress_node that=20 >> >> >> operates in admission control state then the request is >> >> rejected. If >> >> >> it operates in Normal state is accepted. >> >> >> >> >> >> If probing is used, the request for admission is=20 >> accomplished by=20 >> >> >> using probe packets. In this case when the probe arrives at a=20 >> >> >> PCN_egress_node and it is either PCN_marking or >> >> PCN_Affected_marking >> >> >> encoded is rejected. Otherwise is accepted. >> >> >> Note that probes can only be used when >> >> PCN_Affected_marking is appled >> >> >> in whole PCN domain. Otherwise, the admission control >> >> procedure will >> >> >> work by using e.g., an external signaling protocol used between=20 >> >> >> PCN_ingress_nodes and PCN_egress_nodes. >> >> >> >> >> >> * Providing flow termination at PCN_egress_nodes: >> >> >> -------------------------------------------------- >> >> >> >> >> >> When a PCN_egress_node operates on flow termination it >> >> calculates the >> >> >> incoming excess rate (incoming_PCN_marking_rate). >> >> >> By using the excess rate the PCN_egress_node calclates the >> >> numer of >> >> >> flows that have to be terminated using the pseudo code >> >> given on page >> >> >> 23. Note that the PCN_egress_node has to maintain per flow=20 >> >> >> reservation states. >> >> >> The information contained in the per flow states is=20 >> used for the=20 >> >> >> calculation of the flow that have to be terminated, see >> >> pseudocode on >> >> >> page 23. Note also that this pseudo code uses priority >> >> clases, but it >> >> >> operates when also no priority clases are used by setting=20 >> >> >> Maximum_priority =3D 0 Where, 0 =3D< priority_class =3D<=20 >> >> >> Maximum_priority >> >> >> >> >> >> >> >> >> >> >> >> Below, I am providing some answers to your comments! >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> > -----Original Message----- >> >> >> > From: Anna Charny (acharny) [mailto:acharny@cisco.com] >> >> >> > Sent: zondag 7 oktober 2007 21:40 >> >> >> > To: anurag.bhargava@ericsson.com; Georgios Karagiannis; >> >> >> Lars Westberg >> >> >> > (KI/EAB) >> >> >> > Cc: pcn@ietf.org >> >> >> > Subject: Questions on LC-PCN draft version 01 >> >> >> > >> >> >> > Dear authors of the LC-PCN draft, >> >> >> > >> >> >> > Am I right that there has been no response yet to Phil's >> >> questions >> >> >> > (attached at the end) on the new version of your=20 >> draft below (I=20 >> >> >> > checked the pcn list and it does not seem it went=20 >> there)? If the=20 >> >> >> > response was sent, can you resend it please? >> >> >> > >> >> >> > I have many of the same questions, and also a few >> >> additional ones. >> >> >> > Here are some of them. >> >> >> > >> >> >> > 1) I do not understand whether pcn-lower_rate_egress and=20 >> >> >> > pcn_upper_rate ingress are expressed as a fraction of the >> >> >> total rate >> >> >> > (a unitless entity, analogous to CLE) or an absolute value >> >> >> (in bits or >> >> >> > bytes per second). The text seems to be contradictory on >> >> this point: >> >> >> > >> >> >> > on page 8 it is a fraction: >> >> >> > >> >> >> > "PCN_lower_rate_egress =3D predefined percentage of received=20 >> >> >> > PCN_marking", >> >> >> > >> >> >> > while on page 13 it is an absolute rate: >> >> >> > >> >> >> > "If the incoming_PCN_marking_rate is higher than a=20 >> preconfigured=20 >> >> >> > PCN_lower_rate_egress, ..." where the >> >> >> > >> >> >> > (Incoming_PCN_marking_rate is clearly defined as abslolue >> >> >> rate on page >> >> >> > 12: "Where the "incoming_PCN_marking_rate" is calculated >> >> as follows: >> >> >> > incoming_PCN_marking_rate =3D (received number of "PCN_marking" >> >> >> > DSCP during T)* N)/T;) >> >> >> >> >> >> Georgios: You are right, please see discussion above on=20 >> paragaph=20 >> >> >> denoted >> >> >> as: >> >> >> "Setting the thresholds at PCN_egress_nodes". >> >> >> >> >> >> >> >> >> We use a percentage of received PCN_marking encoded packets in=20 >> >> >> proportion to total rate of received packets to define when a=20 >> >> >> PCN_egres_node goes from Normal state to admission control >> >> state. In >> >> >> this version of the draft we call this value as:=20 >> >> >> PCN_lower_rate_egress. This is wrong. >> >> >> What we should say is: >> >> >> PCN_lower_rate_egress: is equal to=20 >> incoming_PCN_marking_rate, when: >> >> >> (incoming_PCN_marking_rate/measured PHB rate) =3D to a=20 >> preconfigured=20 >> >> >> percentage, say 1%. From now on I denote this percentage as: >> >> >> PCN_lower_percentage_egress. >> >> >> Thus we can say that a PCN_egress_node changes from Normal >> >> state to >> >> >> admission control state when >> >> incoming_PCN_marking_rate/measured PHB >> >> >> rate > PCN_lower_percentage_egress. >> >> >> Where, >> >> >> incoming_PCN_marking_rate =3D N *=20 >> input_PCN_marking_bytes/T Where,=20 >> >> >> input_PCN_marking_bytes =3D received number of=20 >> "PCN_marking" encoded=20 >> >> >> packets during measurement period T. >> >> >> >> >> >> >> >> >> > >> >> >> > 2) Regarding the question above, >> >> >> > >> >> >> > * if pcn_lower_rate_eggress (and pcn_uppre-rate_egress) are >> >> >> absolute >> >> >> > rates, then how do you choose that abslolute rate value, >> >> given that >> >> >> > the rates of ingress-egress aggregates at the same >> >> ingress may be >> >> >> > vastly different in magnitude? >> >> >> >> >> >> Georgios: See explanation above! Hopefully this >> >> misundersatnding is >> >> >> also solved! >> >> >> >> >> >> > >> >> >> > *if, however, pcn_lower_rate_eggress (and >> >> >> > pcn_uppre-rate_egress) are ratios, then what is the=20 >> guidance of=20 >> >> >> > setting these ratios so that the egress can always tell >> >> admission >> >> >> > state from the termination state, given that the same >> >> >> marking is used >> >> >> > for admission and termination marking? >> >> >> > Specifically, suppose at the bottleneck pcn_lower_threshold >> >> >> is set to >> >> >> > X, and pcn_upper_threshold to aX with some a>1 (assume >> >> for example >> >> >> > a=3D1.5). >> >> >> > How does the egress node tell between the conditions >> >> when the total >> >> >> > pcn traffic on the bottleneck is 1.1X (which is the=20 >> state wnen=20 >> >> >> > admission is needed but termination is not, - in this case >> >> >> > (1.1X-X)/X=3D0.1 traffic is pcn-marked), and the case when >> >> the total >> >> >> > traffic on the bottleneck is 1.1aX, which is the case when >> >> >> termination >> >> >> > is needed, and (1.1ax-ax)/ax=3D0.1), if in both cases the >> >> >> ratio between >> >> >> > marked and total traffic is the same value 0.1? >> >> >> >> >> >> Georgios: In order to solve the above described problems, the=20 >> >> >> thresholds used in the PCN_interior_nodes and=20 >> PCN_egress_nodes are >> >> >> using: >> >> >> the Admission_offset_rate that is an absolute rate value >> >> which is set >> >> >> equal into whole PCN domain. >> >> >> >> >> >> Note that the maximum excess rate that a PCN_interior_node can=20 >> >> >> calculate in admission control state is equal to=20 >> >> >> Admission_offset_rate. If the excess rate is higher than the=20 >> >> >> Admission_offset_rate then the node changes from=20 >> admission control=20 >> >> >> state to flow termination state. >> >> >> >> >> >> Please see all details that I have provided at the top of >> >> this email.=20 >> >> >> In particular, check the paragraphs denoted above as: >> >> >> "Setting the thresholds at PCN_interior_nodes" >> >> >> "Setting the thresholds at PCN_egress_nodes" >> >> >> "Generated excess rate by an PCN_interior_node operating >> >> in admission >> >> >> control state" >> >> >> >> >> >> >> >> >> >> >> >> > >> >> >> > 3) On page 9, multicongestion error error is introduced, >> >> >> but the text >> >> >> > is silent on how this error is defined and what is it set >> >> >> to (if it is >> >> >> > a configuration parameter), or how it is measured (if it is >> >> >> something >> >> >> > that is being measured). Can you explain, please? >> >> >> >> >> >> Georgios: >> >> >> Please see the paragraph denoted above as "Setting the >> >> thresholds at >> >> >> PCN_egress_nodes" >> >> >> >> >> >> >> >> >> Best regards, >> >> >> Georgios >> >> >> >> >> >> >> >> >> > >> >> >> > I have some more questions, including sharing those asked >> >> >> by Phil in >> >> >> > the attached message), but I will stop now, as I hope >> >> >> clarifications >> >> >> > on the above (and Phil's) questions will help understand >> >> the draft >> >> >> > better... >> >> >> > >> >> >> > Thank you in advance for clarifying all this. >> >> >> > >> >> >> > Anna >> >> >> >> >>=20 >>=20 _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Fri Nov 09 07:41:23 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IqTAp-0000rE-0M; Fri, 09 Nov 2007 07:41:23 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IqTAo-0000o4-0Q for pcn-confirm+ok@megatron.ietf.org; Fri, 09 Nov 2007 07:41:22 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IqTAn-0000nG-Ms for pcn@ietf.org; Fri, 09 Nov 2007 07:41:21 -0500 Received: from rotterdam.ewi.utwente.nl ([130.89.10.5]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IqTAk-0004R4-58 for pcn@ietf.org; Fri, 09 Nov 2007 07:41:21 -0500 Received: from webmail.cs.utwente.nl (janus.ewi.utwente.nl [130.89.10.26]) by rotterdam.ewi.utwente.nl (8.13.6/8.13.6) with SMTP id lA9CfHFn024961 for ; Fri, 9 Nov 2007 13:41:17 +0100 (MET) Received: from 213.16.183.10 (auth. user karagian@imap1.ewi.utwente.nl) by webmail.cs.utwente.nl with HTTP; Fri, 09 Nov 2007 12:41:16 +0000 To: "pcn@ietf.org" Date: Fri, 09 Nov 2007 12:41:16 +0000 X-Mailer: IlohaMail/0.8.13 (On: webmail.cs.utwente.nl) Message-ID: <7d6XOXRK.1194612076.8325830.karagian@ewi.utwente.nl> From: "Georgios Karagiannis" Bounce-To: "Georgios Karagiannis" MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0 () X-Scanned-By: MIMEDefang 2.52 on 130.89.10.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0rc3 (rotterdam.ewi.utwente.nl [130.89.10.5]); Fri, 09 Nov 2007 13:41:17 +0100 (MET) X-Spam-Score: 0.0 (/) X-Scan-Signature: 79899194edc4f33a41f49410777972f8 Subject: [PCN] probing functionality X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi all Another method that can be used for PCN probing can be applied as follows. The probe packets are generated by the PCN_ingress_nodes in the following way. A probe packet belonging to a certain micro-flow is generated using the same 5 touple (source, destination IP address, source, destination ports, protocol number) as the packets belonging to the same micro-flow. In addition to this the PCN_ingress_node has to set the Router Alert IP option on the probe packet. In this way all the PCN_interior_node will have to observe the received probe packets. Thus if a PCN_interior_node receives a probe packet then, due to the Router Alert option it has to handle it differently then the user packets. The PCN_interior_node has to PCN_mark the probe packet if it is operating in admission control state (or flow termination state). Otherwise the probe packet remains unmarked. Best regards, Georgios _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Fri Nov 09 07:44:53 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IqTED-00051T-7R; Fri, 09 Nov 2007 07:44:53 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IqTEB-00051O-D6 for pcn-confirm+ok@megatron.ietf.org; Fri, 09 Nov 2007 07:44:51 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IqTEB-00051G-3W for pcn@ietf.org; Fri, 09 Nov 2007 07:44:51 -0500 Received: from rotterdam.ewi.utwente.nl ([130.89.10.5]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IqTE8-0004XE-LY for pcn@ietf.org; Fri, 09 Nov 2007 07:44:51 -0500 Received: from webmail.cs.utwente.nl (janus.ewi.utwente.nl [130.89.10.26]) by rotterdam.ewi.utwente.nl (8.13.6/8.13.6) with SMTP id lA9CikVi025216 for ; Fri, 9 Nov 2007 13:44:46 +0100 (MET) Received: from 213.16.183.10 (auth. user karagian@imap1.ewi.utwente.nl) by webmail.cs.utwente.nl with HTTP; Fri, 09 Nov 2007 12:44:46 +0000 To: "pcn@ietf.org" Date: Fri, 09 Nov 2007 12:44:45 +0000 X-Mailer: IlohaMail/0.8.13 (On: webmail.cs.utwente.nl) Message-ID: From: "Georgios Karagiannis" Bounce-To: "Georgios Karagiannis" MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0 () X-Scanned-By: MIMEDefang 2.52 on 130.89.10.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0rc3 (rotterdam.ewi.utwente.nl [130.89.10.5]); Fri, 09 Nov 2007 13:44:48 +0100 (MET) X-Spam-Score: 0.0 (/) X-Scan-Signature: 5ba9b8496764663b12c333825fbf6b3d Subject: [PCN] new description of LC-PCN algorithm X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Anna, Hi Phil, Hi all Based on the comments that I have got so far I have tried to modify the LC-PCN algorithm. The main changes are: * PCN_affected_marking is only used during flow termination. In this way the ECMP solution during flow termination can be completely solved. * Probing is changed: Regarding probe generation, it is important to note that probe packets are generated by the PCN_ingress_nodes in the following way. A probe packet belonging to a certain micro-flow is generated using the same 5 touple (source, destination IP address, source, destination ports, protocol number) as the packets belonging to the same micro-flow. In addition to this the PCN_ingress_node has to set the Router Alert IP option on the probe packet. In this way all the PCN_interior_node will have to observe the received probe packets. Thus if a PCN_interior_node receives a probe packet then, due to the Router Alert option it has to handle it differently then the user packets. The PCN_interior_node has to PCN_mark the probe packet if it is operating in admission control state (or flow termination state). Otherwise the probe packet remains unmarked. * The PCN_egress_node changes to flow termination state if it recieves at least one PCN_Affected_marked packet. Below I am describing the new proposed LC-PCN operation: * Setting the thresholds at PCN_interior_nodes: ----------------------------------------------- In order to calculate the PCN_upper_rate we use two parameters: Maximum PHB capacity: that is the maximum capacity that can be supported by a PCN_interior_node Termination_offset_rate: that is an absolute rate value that should be set equal into all PCN_interior_nodes. Note that this value is used by PCN_interior_nodes to calculate their PCN_upper_rate and also during the situation that a PCN_interior_node is in flow termination state and it receives PCN_marked packets. Please see pseudo code on page 20. This value must be set equal into all PCN_interior_nodes such that all these nodes will know when to take into account the incoming PCN_marked packets and when not. The Termination_offset_rate is needed due to the following fact. Consider the fact that when the measured PHB rate exceeds the "Maximum PHB capacity" than the packets belonging to the given PHB will be either dropped or set to another PHB. In the situation of multiple severe congestion situations solving the severe congestion on a severe congestion point, further away than the PCN_egress_node, say severe_congestion_point_1, it could cause the situation that the severe congestion on a node located on the same path and closer to the PCN_egress_node, say severe_congestion_point_2, will be solved without marking the excess rate measured at severe_congestion_point_2. This is however true only if the measured PHB rate on severe_congestion_point_1 does not exceed the "Maximum PHB capacity". This is due to the fact that before the severe_congestion_point_1 goes into flow termination it generates a maximum PHB rate that it does not exceed its "Maximum PHB capacity" - termination_offset_rate and after severe congestion it generates a maximum PHB rate not higher than "Maximum PHB capacity". Thus if the excess rate on severe_congestion_point_1 is higher than "Maximum PHB capacity" is not seen by severe_congestion_point_2 but, due to the principle of marking, it will be seen by the PCN_egress_nodes. Therefore, the severe_congestion_point_2 has to consider the incoming_PCN_marked_rate from severe_congestion_point_1 in its marking algorithm only for the rate equal to "Maximum PHB capaciy" - PCN_upper_rate (associated with severe_congestion_point_1). Note that "Maximum PHB capacity" - PCN_upper_rate is equal to Termination_offset_rate. Note that the severe_congestion_point_2 can know the termination_offset_rate used by the previous severe congestion point by using a variable that is equal into whole the PCN domain. Note that the Termination_offset_rate can also be equal to 0. The PCN_upper_rate is then found as: PCN_upper_rate =3D "Maximum PHB capacity" - Termination_offset_rate The PCN_lower_rate is configured in all PCN_interior-nodes and it can be calculated in the following way: PCN_lower_rate =3D PCN_upper_rate - Admission_offset_rate The Admission_offset_rate is an absolute rate value and it is equal in all PCN_interior_nodes and PCN_egress_nodes. Note that this value is used by PCN_interior_nodes to calculate their PCN_lower_rate and the PCN_egress_nodes to calculate their PCN_upper_rate_egress. Furthermore, this value is used by the PCN_interior_nodes also during the situation that a PCN_interior_node is in admission control state and it receives PCN_marked packets. Please see pseudo code on page 13. This value must be set equal into all PCN_interior_nodes such that all these nodes will know when to take into account the incoming PCN_marked packets and when not. Note that a PCN_interior_node can PCN_mark packets up to an excess rate equal to the Admission_offset_rate. If the exess rate in an PCN_interior_node is higher than the Admission_offset_rate, then the PCN_interior_node changes state from admission control state to flow termination state. * Setting the thresholds at PCN_egress_nodes: --------------------------------------------- The question is how to calculate the threshold that defines when a PCN_egress_node goes into the admission control state. One way to do that is to consider that when the PCN_egress_node receives a PCN_marked packet it will mean that at least one PCN_interior_node started to be admission control congested and therefore it will go from Normal state to admission control state. Of course this will somehow might provide some errors, because there might be situations that this consideration might be to conservative. Therefore, we use a factor of received PCN_marking encoded packets. PCN_lower_rate_egress: is equal to a fraction of the Admission_offset_rate, say fraction A * Admission_offset_rate, where 0 PCN_lower_rate_egress. where, incoming_PCN_marking_rate =3D N * input_PCN_marking_bytes/T Where, input_PCN_marking_bytes =3D received number of "PCN_marking" encoded packets during measurement period T. Now we defined the condition that the PCN_egress_node changes state from Normal state to admission control state. But how will the PCN_egress_node change state from admission control state to flow termination state. There are two solutions for this: First solution: if the PCN_interior_nodes use the PCN_Affected_marking encoding only during flow termination for the packets that are passing through the severe congested node, but without being PCN_marked, then the PCN_egress_node can change to flow termination state when it receives PCN_Affected_marked packets and change from flow termnation state to normal state when it does not receive PCN_Affected_marked packets. Second solution: In order to explain this, it is important to note that each PCN_interior_node that is in admission control state it can PCN_mark packets up to a value equal to the Admission_offset_rate. Furthermore, if a PCN_interior_node receives incoming PCN_marked packets and is in the addmission control state, it will not remark any packets if the excess rate is equal or lower than the incoming_PCN_marking_rate, see page 13. Furthermore, if we will consider as normal situations the situations that no ECMP occurs and that all flows belonging to the same ingress-egress aggregate will use the same path from PCN_ingress to PCN_egress, this will mean that when the PCN_egress_node receives, for the given ingress-egress aggregate an excess rate equal to a fraction of the Admission_offset_rate, say fraction F * Admission_offset_rate, where A>F>=3D1, it will have to change from admission control state to flow termination state. Note that F can be preconfigured and depends on the network topology. Thus in this case the second threshold, can be calculated as follows: PCN_upper_egress_rate =3D PCN_lower_egress_rate + F*Admission_offset_rate. However, there are some corner cases, that mainly occur when the different congestion points (admission control congested PCN_interior_nodes) on the same path are not simulataneously starting to be congested. Therefore we use the multicongestion_error parameter to identify the error bound that ocurs due to these corner cases. Note that this error bound can be e.g., predefined ones off line by the operator, by studying the network topology and/or studying how often such corner cases could occur and/or doing off line measurements. Therefore we use: PCN_upper_rate_egress =3D PCN_lower_rate_egress + F*Admission_offset_rate +/- multicongestion_error I prefer to use the first solution, because then the PCN_egress_node can solve completely the ECMP issue and the F factor and the multicongestion_error factor do not have to be preconfigured anymore. * How the states of operation in PCN_interior_nodes are being changed? --------------------------------------------------------------------- This is explained on page 17, 18, by using Figure 4. Change from Normal state to Admission control state: event A Occurs when: Measured PHB rate > PCN_lower_rate Change from Admission control state to Flow Termination state: event B Occurs when: Measured PHB rate > PCN_upper_rate * How the states of operation in PCN_egress_nodes are being changed? --------------------------------------------------------------------- This is explained on page 21, 22, but they have to be modified. Change from Normal state to Admission control state: event A Occurs when: incoming_PCN_marking_rate > PCN_lower_rate_egress Change from Admission control state to flow termination state: Using First solution mentioned above event B Occurs when the PCN_egress_node receives at least one PCN_Affected_marked packet. Using second solution: incoming_PCN_marking_rate > PCN_upper_rate_egress It is important to note that also the explanation of event C on page 21, has to be modified, see explanation given above: Change from Admission control state to Normal state: Occurs when: incoming_PCN_marking_rate =3D< PCN_lower_rate Change from flow termination to Normal state: Uisng first solution: The PCN_egress_node does not receive any PCN_Affected_marked packets Using second solution: * Generated excess rate by an PCN_interior_node operating in admission control state. ------------------------------------------------------------- The excess rate =3D signaled_overload_rate. The maximum excess rate that a PCN_interior_node can calculate in admission control state is equal to Admission_offset_rate, see explanation above. The number of bytes that are remarked, signaled_remarked_bytes depend on the value of calculated excess rate (signaled_overload_rate), the value of the Admission_offset_rate and the value of the incoming_PCN_marking_rate, see pseudo code on page 13. Regarding probes, and based on the received commenst by Phil and Anna I modified the operation of probes. Regarding probe generation, it is important to note that probe packets are generated by the PCN_ingress_nodes in the following way. A probe packet belonging to a certain micro-flow is generated using the same 5 touple (source, destination IP address, source, destination ports, protocol number) as the packets belonging to the same micro-flow. In addition to this the PCN_ingress_node has to set the Router Alert IP option on the probe packet. In this way all the PCN_interior_node will have to observe the received probe packets. Thus if a PCN_interior_node receives a probe packet then, due to the Router Alert option it has to handle it differently then the user packets. The PCN_interior_node has to PCN_mark the probe packet if it is operating in admission control state (or flow termination state). Otherwise the probe packet remains unmarked. * Generating excess rate by an PCN_interior_node operating in flow termination state: ---------------------------------------------------------------- The excess rate =3D signaled_overload_rate. Note that the calculation of the signaled_overload_rate is different than in the situation that the PCN_Interior_node operates in admission control state, see page 19 and 20. This is due to the fact that a sliding window is used to solve an undershooting problem, see discussion on page 19. The number of bytes that are remarked, signaled_remarked_bytes depend on the value of calculated excess rate (signaled_overload_rate), the value of the Termination_offset_rate and the value of the incoming_PCN_marking_rate, and the see pseudo code on page 20. Note that all packets that are passing through a congested PCN_interior_node and are not being PCN_marked by the PCN_interior_node have to be remarked using the PCN_Affected_marking. * Providing admission control at PCN_egress_nodes: -------------------------------------------------- When the PCN_egress_node is operating in admission control state than a flow that is requesting admission into the PCN domain can be treated in the following way: If no probing is used, the request for admission can be accomplished by using an external to PCN signaling protocol. In this case when the request arrives at a PCN_egress_node that operates in admission control state then the request is rejected. If it operates in Normal state is accepted. If probing is used, the request for admission is accomplished by using probe packets. In this case when the probe arrives at a PCN_egress_node and it is PCN_marking encoded is rejected. Otherwise is accepted. * Providing flow termination at PCN_egress_nodes: -------------------------------------------------- When a PCN_egress_node operates on flow termination it calculates the incoming excess rate (incoming_PCN_marking_rate). By using the excess rate the PCN_egress_node calclates the number of flows that have to be terminated using the pseudo code given on page 23. Note that the PCN_egress_node has to maintain per flow reservation states. The information contained in the per flow states is used for the calculation of the flow that have to be terminated, see pseudocode on page 23. Note also that this pseudo code uses priority clases, but it operates when also no priority clases are used by setting Maximum_priority =3D 0 Where, 0 =3D< priority_class =3D< Maximum_priority Best regards, Georgios _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Fri Nov 09 12:28:35 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IqXel-0004P0-Ew; Fri, 09 Nov 2007 12:28:35 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IqXek-0004NJ-Ct for pcn-confirm+ok@megatron.ietf.org; Fri, 09 Nov 2007 12:28:34 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IqXek-0004NB-0k for pcn@ietf.org; Fri, 09 Nov 2007 12:28:34 -0500 Received: from smtp1.smtp.bt.com ([217.32.164.137]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IqXeh-0003Ta-WC for pcn@ietf.org; Fri, 09 Nov 2007 12:28:33 -0500 Received: from E03MVZ1-UKDY.domain1.systemhost.net ([193.113.30.65]) by smtp1.smtp.bt.com with Microsoft SMTPSVC(6.0.3790.1830); Fri, 9 Nov 2007 17:28:30 +0000 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [PCN] new description of LC-PCN algorithm Date: Fri, 9 Nov 2007 17:28:29 -0000 Message-ID: <75A199C5D243C741BF3D3F1EBCEF9BA503B34324@E03MVZ1-UKDY.domain1.systemhost.net> In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] new description of LC-PCN algorithm Thread-Index: AcgizlYG4BTllksSR8iEau+pndg02AAIsXPw From: To: , X-OriginalArrivalTime: 09 Nov 2007 17:28:30.0637 (UTC) FILETIME=[F23B65D0:01C822F5] X-Spam-Score: 0.0 (/) X-Scan-Signature: 82b297dca242a35ee50ccecf5bf2e37f Cc: X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Thanks Georgios Rather than be swamped by detail on a Friday evening, I wanted to make some high level points. - the PCN-affected-marking change is (I think) sensible. I think this idea could be added (perhaps as an option for operator) to any of the other 3 schemes. So it might be valuable to have a short email /draft that only talked about this idea? an analysis of the benefit, so can judge whether it's worth the pain (the extra encoding point required). The benefit may also depend on what marking behaviour is standardised?=20 - at a high level your proposal seems gradually to be looking more similar to the single marking idea, ie rate based marking behaviour for any rate above PCN-lower-rate ("adm marking" & "termination marking") (yes I know your algorithm changes above PCN-upper-rate, but only slightly). Of course there are differences. However, I wonder how much they matter at the moment; we have to either select which high level approach to standardise or else standardise in a way that allows more than one approach through configurable parameters (see draft anna sent yesterday).=20 - I wonder if it would be worth looking at the differences one at a time and seeing if single marking could be improved by adding it? For example, one difference is that you swap back & forth between rates in bit/s and rates as % capacity (of PHB) - I admit I can't really see the benefit and the downside is that you have extra globally fixed parameters (eg {PCN-upper-rate =3D PCN-lower-rate}). There are some differences (I think fairly minor ones) in terms of handling already marked pkts that are worth looking at. The idea about adjusting the marking behaviour to account for some flows already being in the middle of being terminated - we should discuss in a wider context (not just applicable to lc-pcn) (personally at the moment I don't like the idea, as I said in previous email I think it's better adjusting at the PCN-boundary-node than at the PCN-interior-node). Especially important are any differences in the PCN-interior-node marking behaviour. Have a good weekend, Phil/ > -----Original Message----- > From: Georgios Karagiannis [mailto:karagian@cs.utwente.nl] > Sent: 09 November 2007 12:45 > To: pcn@ietf.org > Subject: [PCN] new description of LC-PCN algorithm >=20 > Hi Anna, Hi Phil, Hi all >=20 > Based on the comments that I have got so far I have tried to modify the > LC-PCN algorithm. >=20 > The main changes are: > * PCN_affected_marking is only used during flow termination. In this way > the ECMP solution during flow termination can be completely solved. >=20 > * Probing is changed: > Regarding probe generation, it is important to note that probe > packets are generated by the PCN_ingress_nodes in the following way. > A probe packet belonging to a certain micro-flow is generated using the > same 5 touple (source, destination IP address, source, destination ports, > protocol number) > as the packets belonging to the same micro-flow. In addition to this > the PCN_ingress_node has to set the Router Alert IP option on the probe > packet. In this way all > the PCN_interior_node will have to observe the received probe packets. >=20 > Thus if a PCN_interior_node receives a probe packet then, due to the > Router Alert option it has to handle it differently then the user packets. > The PCN_interior_node has to PCN_mark the probe packet if it is operating > in admission control > state (or flow termination state). Otherwise the probe packet remains > unmarked. >=20 > * The PCN_egress_node changes to flow termination state if it recieves > at least one PCN_Affected_marked packet. >=20 > Below I am describing the new proposed LC-PCN operation: >=20 > * Setting the thresholds at PCN_interior_nodes: > ----------------------------------------------- > In order to calculate the PCN_upper_rate we use two parameters: > Maximum PHB capacity: that is the maximum capacity that can be supported > by a PCN_interior_node > Termination_offset_rate: that is an absolute rate value that should be set > equal into all PCN_interior_nodes. Note that this value is used by > PCN_interior_nodes to calculate their PCN_upper_rate and also during > the situation that a PCN_interior_node is in flow termination state > and it receives PCN_marked packets. Please see pseudo code on page 20. > This value must be set equal into all PCN_interior_nodes such that all > these nodes will know when to take into account the incoming > PCN_marked packets and when not. The Termination_offset_rate is needed > due to the following fact. > Consider the fact that when the measured PHB rate exceeds the > "Maximum PHB capacity" than the packets belonging to the > given PHB will be either dropped or set to another PHB. > In the situation of multiple severe congestion situations solving the > severe congestion on a severe congestion point, further away than the > PCN_egress_node, say severe_congestion_point_1, it could cause the > situation > that the severe congestion on a node located on the same path and > closer to the PCN_egress_node, say severe_congestion_point_2, > will be solved without marking the excess rate measured at > severe_congestion_point_2. This is however true only if > the measured PHB rate on severe_congestion_point_1 does not > exceed the "Maximum PHB capacity". > This is due to the fact that before the severe_congestion_point_1 goes > into > flow termination it generates a maximum PHB rate that it does not exceed > its "Maximum PHB capacity" - termination_offset_rate and after severe > congestion it generates a > maximum PHB rate not higher than "Maximum PHB capacity". > Thus if the excess rate on severe_congestion_point_1 is higher > than "Maximum PHB capacity" is not seen by severe_congestion_point_2 > but, due to the principle of marking, it will be seen by the > PCN_egress_nodes. > Therefore, the severe_congestion_point_2 has to consider the > incoming_PCN_marked_rate from severe_congestion_point_1 > in its marking algorithm only for the rate equal to "Maximum PHB > capaciy" - PCN_upper_rate (associated with > severe_congestion_point_1). Note that "Maximum PHB capacity" - > PCN_upper_rate > is equal to Termination_offset_rate. > Note that the severe_congestion_point_2 can know the > termination_offset_rate used by the previous severe congestion point > by using a variable that is equal into whole the PCN domain. Note that > the Termination_offset_rate can also be equal to 0. >=20 > The PCN_upper_rate is then found as: > PCN_upper_rate =3D "Maximum PHB capacity" - Termination_offset_rate >=20 > The PCN_lower_rate is configured in all PCN_interior-nodes and it can be > calculated in the following way: > PCN_lower_rate =3D PCN_upper_rate - Admission_offset_rate >=20 > The Admission_offset_rate is an absolute rate value and it is equal in > all PCN_interior_nodes and PCN_egress_nodes. Note that this value is > used by >=20 > PCN_interior_nodes to calculate their PCN_lower_rate and the > PCN_egress_nodes to calculate their PCN_upper_rate_egress. Furthermore, > this value is used by >=20 > the PCN_interior_nodes also during the situation that a PCN_interior_node > is in admission control state and it receives PCN_marked packets. Please > see pseudo >=20 > code on page 13. > This value must be set equal into all PCN_interior_nodes such that all > these nodes will know when to take into account the incoming PCN_marked > packets and when not. Note that a PCN_interior_node can PCN_mark packets > up to an excess rate equal to the Admission_offset_rate. If the exess rate > in an PCN_interior_node is higher than the Admission_offset_rate, then the > PCN_interior_node changes state from admission control state to > flow termination state. >=20 > * Setting the thresholds at PCN_egress_nodes: > --------------------------------------------- > The question is how to calculate the threshold that defines when a > PCN_egress_node goes into the admission control state. > One way to do that is to consider that when the PCN_egress_node receives a > PCN_marked packet it will mean that at least one PCN_interior_node > started to be admission control congested and therefore it will go > from Normal state to admission control state. Of course this will > somehow might provide some errors, because there might be situations > that this consideration might be to conservative. > Therefore, we use a factor of received PCN_marking encoded packets. >=20 > PCN_lower_rate_egress: is equal to a fraction of the > Admission_offset_rate, say > fraction A * Admission_offset_rate, where 0 Thus: PCN_lower_rate_egress =3D A * Admission_offset_rate. >=20 >=20 > Thus we can say that a PCN_egress_node changes from Normal state to > admission control state when incoming_PCN_marking_rate > > PCN_lower_rate_egress. >=20 > where, incoming_PCN_marking_rate =3D N * input_PCN_marking_bytes/T > Where, input_PCN_marking_bytes =3D received number of "PCN_marking" > encoded > packets during measurement period T. >=20 > Now we defined the condition that the PCN_egress_node changes state from > Normal state to admission control state. >=20 >=20 > But how will the PCN_egress_node change state from admission control state > to flow termination state. >=20 > There are two solutions for this: > First solution: if the PCN_interior_nodes use the PCN_Affected_marking > encoding only during flow termination for the packets that are passing > through > the severe congested node, but without being PCN_marked, > then the PCN_egress_node can change to flow termination state when it > receives > PCN_Affected_marked packets and change from flow termnation state to > normal state > when it does not receive PCN_Affected_marked packets. >=20 > Second solution: > In order to explain this, it is important to note that each > PCN_interior_node > that is in admission control state it can PCN_mark packets up to a value > equal to the Admission_offset_rate. >=20 > Furthermore, if a PCN_interior_node receives incoming PCN_marked packets > and is in the addmission control state, it will not remark any packets if > the excess rate is equal or lower than the incoming_PCN_marking_rate, > see page 13. > Furthermore, if we will consider as normal situations the situations that > no ECMP occurs and that all flows belonging to the same ingress-egress > aggregate will use the same path from PCN_ingress to PCN_egress, > this will mean that when the PCN_egress_node receives, for the given > ingress-egress aggregate an excess rate equal to a fraction of the > Admission_offset_rate, say fraction F * Admission_offset_rate, where > A>F>=3D1, > it will have to change from admission control state to flow termination > state. > Note that F can be preconfigured and depends on > the network topology. > Thus in this case the second threshold, can be calculated as follows: > PCN_upper_egress_rate =3D PCN_lower_egress_rate + F*Admission_offset_rate. > However, there are some corner cases, that mainly occur when the > different congestion points (admission control congested > PCN_interior_nodes) on the same path are not simulataneously starting to > be congested. Therefore we use the multicongestion_error parameter to > identify the error bound that ocurs due to these corner cases. Note that > this error bound can be e.g., predefined ones off line by the > operator, by studying the network topology and/or studying how > often such corner cases could occur and/or doing off line measurements. > Therefore we use: > PCN_upper_rate_egress =3D PCN_lower_rate_egress + F*Admission_offset_rate > +/- multicongestion_error >=20 > I prefer to use the first solution, because then the PCN_egress_node can > solve completely the > ECMP issue and the F factor and the multicongestion_error factor do not > have > to be preconfigured anymore. > * How the states of operation in PCN_interior_nodes are being changed? > --------------------------------------------------------------------- > This is explained on page 17, 18, by using Figure 4. >=20 > Change from Normal state to Admission control state: event A Occurs when: > Measured PHB rate > PCN_lower_rate >=20 > Change from Admission control state to Flow Termination state: event B > Occurs when: > Measured PHB rate > PCN_upper_rate >=20 > * How the states of operation in PCN_egress_nodes are being changed? > --------------------------------------------------------------------- > This is explained on page 21, 22, but they have to be modified. >=20 > Change from Normal state to Admission control state: event A Occurs when: > incoming_PCN_marking_rate > PCN_lower_rate_egress >=20 > Change from Admission control state to flow termination state: > Using First solution mentioned above event B Occurs when the > PCN_egress_node receives > at least one PCN_Affected_marked packet. >=20 > Using second solution: > incoming_PCN_marking_rate > PCN_upper_rate_egress >=20 > It is important to note that also the explanation of event C on page 21, > has to be modified, > see explanation given above: > Change from Admission control state to Normal state: > Occurs when: >=20 > incoming_PCN_marking_rate =3D< PCN_lower_rate >=20 > Change from flow termination to Normal state: > Uisng first solution: > The PCN_egress_node does not receive any PCN_Affected_marked packets >=20 > Using second solution: >=20 >=20 > * Generated excess rate by an PCN_interior_node operating in admission > control state. > ------------------------------------------------------------- >=20 > The excess rate =3D signaled_overload_rate. > The maximum excess rate that a PCN_interior_node can calculate > in admission control state is equal to Admission_offset_rate, > see explanation above. > The number of bytes that are remarked, signaled_remarked_bytes > depend on the value of calculated excess rate (signaled_overload_rate), > the value of the Admission_offset_rate and the value of the > incoming_PCN_marking_rate, see pseudo code on page 13. >=20 >=20 >=20 > Regarding probes, and based on the received commenst by Phil and Anna > I modified the operation of probes. >=20 > Regarding probe generation, it is important to note that probe > packets are generated by the PCN_ingress_nodes in the following way. > A probe packet belonging to a certain micro-flow is generated using the > same 5 touple (source, destination IP address, source, destination ports, > protocol number) > as the packets belonging to the same micro-flow. In addition to this > the PCN_ingress_node has to set the Router Alert IP option on the probe > packet. In this way all > the PCN_interior_node will have to observe the received probe packets. >=20 > Thus if a PCN_interior_node receives a probe packet then, due to the > Router Alert option it has to handle it differently then the user packets. > The PCN_interior_node has to PCN_mark the probe packet if it is operating > in admission control > state (or flow termination state). Otherwise the probe packet remains > unmarked. >=20 >=20 >=20 > * Generating excess rate by an PCN_interior_node operating in flow > termination state: > ---------------------------------------------------------------- > The excess rate =3D signaled_overload_rate. > Note that the calculation of the signaled_overload_rate is different than > in the situation that the PCN_Interior_node operates in admission control > state, see page 19 and 20. This is due to the fact that a sliding window > is used to solve an undershooting problem, see discussion on page 19. > The number of bytes that are remarked, signaled_remarked_bytes depend on > the value of calculated excess rate (signaled_overload_rate), the value of > the Termination_offset_rate and the value of the > incoming_PCN_marking_rate, > and the see pseudo code on page 20. >=20 > Note that all packets that are passing through a congested > PCN_interior_node > and are not being PCN_marked by the PCN_interior_node have to be remarked > using the PCN_Affected_marking. >=20 > * Providing admission control at PCN_egress_nodes: > -------------------------------------------------- > When the PCN_egress_node is operating in admission control state > than a flow that is requesting admission into the PCN domain can be > treated > in the following way: >=20 > If no probing is used, the request for admission can be accomplished by > using an external to PCN signaling protocol. In this case when the > request arrives at a PCN_egress_node that operates in > admission control state then the request is rejected. > If it operates in Normal state is accepted. >=20 > If probing is used, the request for admission is accomplished by > using probe packets. In this case when the probe arrives at > a PCN_egress_node and it is PCN_marking encoded is rejected. > Otherwise is accepted. >=20 > * Providing flow termination at PCN_egress_nodes: > -------------------------------------------------- >=20 > When a PCN_egress_node operates on flow termination it calculates > the incoming excess rate (incoming_PCN_marking_rate). > By using the excess rate the PCN_egress_node calclates the number of > flows that have to be terminated using the pseudo code given on page 23. > Note that the PCN_egress_node has to maintain per flow reservation states. > The information contained in the per flow states is used for the > calculation of the flow that have to be terminated, see pseudocode > on page 23. Note also that this pseudo code uses priority clases, > but it operates when also no priority clases are used by setting > Maximum_priority =3D 0 Where, 0 =3D< priority_class =3D< = Maximum_priority >=20 > Best regards, > Georgios >=20 >=20 > _______________________________________________ > PCN mailing list > PCN@ietf.org > https://www1.ietf.org/mailman/listinfo/pcn _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Mon Nov 12 05:07:08 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IrWCB-0007lF-UE; Mon, 12 Nov 2007 05:07:07 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IrWCA-0007jY-3G for pcn-confirm+ok@megatron.ietf.org; Mon, 12 Nov 2007 05:07:06 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IrWC9-0007jP-Od for pcn@ietf.org; Mon, 12 Nov 2007 05:07:05 -0500 Received: from wrzx28.rz.uni-wuerzburg.de ([132.187.3.28] helo=mailrelay.rz.uni-wuerzburg.de) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IrWC9-0006xN-8E for pcn@ietf.org; Mon, 12 Nov 2007 05:07:05 -0500 Received: from virusscan.mail (localhost [127.0.0.1]) by mailrelay.mail (Postfix) with ESMTP id 4169EC0BB; Mon, 12 Nov 2007 11:07:02 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by virusscan.mail (Postfix) with ESMTP id 34257C0CB; Mon, 12 Nov 2007 11:07:02 +0100 (CET) Received: from europa.informatik.uni-wuerzburg.de (wicx01.informatik.uni-wuerzburg.de [132.187.11.1]) by mailmaster.uni-wuerzburg.de (Postfix) with ESMTP id 0AE5EC0BB; Mon, 12 Nov 2007 11:07:00 +0100 (CET) Received: from nero.informatik.uni-wuerzburg.de (win3005.informatik.uni-wuerzburg.de [132.187.106.5]) by europa.informatik.uni-wuerzburg.de (8.11.3/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id lACA70h27826; Mon, 12 Nov 2007 11:07:00 +0100 Received: from [127.0.0.1] (nero.informatik.uni-wuerzburg.de [132.187.106.5]) by nero.informatik.uni-wuerzburg.de (Postfix) with ESMTP id 8A8116F591; Mon, 12 Nov 2007 10:59:27 +0100 (CET) Message-ID: <473823E5.4000502@informatik.uni-wuerzburg.de> Date: Mon, 12 Nov 2007 10:59:01 +0100 From: Michael Menth Organization: University of Wuerzburg User-Agent: Thunderbird 2.0.0.6 (Windows/20070728) MIME-Version: 1.0 To: "Scott O. Bradner" Subject: Re: [PCN] 1st call - agenda requests for Vancouver References: <20071107033818.610D261B673@newdev.eecs.harvard.edu> In-Reply-To: <20071107033818.610D261B673@newdev.eecs.harvard.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at uni-wuerzburg.de X-Spam-Score: 0.0 (/) X-Scan-Signature: 7baded97d9887f7a0c7e8a33c2e3ea1b Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: menth@informatik.uni-wuerzburg.de List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Scott, I would like to request a slot to present the performance results documented in http://www.ietf.org/internet-drafts/draft-menth-pcn-performance-00.txt It currently covers insights regarding the behavior of threshold and ramp marking. I'll update the document with a summary of further results by next week. Regards, Michael Scott O. Bradner wrote: > this is a first call for timeslots on the PCN agenda in Vancouver > > we are currently scheduled for wed afternoon from 1300-1610 > > we expect that a good chunk of the time will be spent on discussions of > the Architecture draft but if others have other (in-charter) topics > please let us know > > Scott & Steve > > > _______________________________________________ > PCN mailing list > PCN@ietf.org > https://www1.ietf.org/mailman/listinfo/pcn > -- Dr. Michael Menth, Assistant Professor University of Wuerzburg, Institute of Computer Science Am Hubland, D-97074 Wuerzburg, Germany, room B206 phone: (+49)-931/888-6644, fax: (+49)-931/888-6632 mailto:menth@informatik.uni-wuerzburg.de http://www3.informatik.uni-wuerzburg.de/research/ngn _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Tue Nov 13 03:30:54 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IrrAb-0006K0-Jn; Tue, 13 Nov 2007 03:30:53 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IrrAa-0006Hb-76 for pcn-confirm+ok@megatron.ietf.org; Tue, 13 Nov 2007 03:30:52 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IrrAU-0006Cd-QG for pcn@ietf.org; Tue, 13 Nov 2007 03:30:46 -0500 Received: from tcmail31.telekom.de ([217.6.95.238]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IrrAU-0000QU-3w for pcn@ietf.org; Tue, 13 Nov 2007 03:30:46 -0500 Received: from s4de8psaans.mitte.t-com.de by tcmail31.telekom.de with ESMTP; Tue, 13 Nov 2007 09:30:40 +0100 Received: from S4DE8PSAANK.mitte.t-com.de ([10.151.229.10]) by s4de8psaans.mitte.t-com.de with Microsoft SMTPSVC(6.0.3790.3959); Tue, 13 Nov 2007 09:30:39 +0100 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Subject: RE: [PCN] probing functionality Date: Tue, 13 Nov 2007 09:30:33 +0100 Message-Id: <1B6169C658325341A3B8066E23919E1C4C142D@S4DE8PSAANK.mitte.t-com.de> In-Reply-To: <7d6XOXRK.1194612076.8325830.karagian@ewi.utwente.nl> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] probing functionality Thread-Index: AcgizgAMeZ8CADlfTkuLQLO3K+CKsAC/HMkw From: "Geib, Ruediger" To: X-OriginalArrivalTime: 13 Nov 2007 08:30:39.0982 (UTC) FILETIME=[791338E0:01C825CF] X-Spam-Score: 0.0 (/) X-Scan-Signature: 10d3e4e3c32e363f129e380e644649be Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Georgios, maybe I'm wrong, but I think you were the one to mention that=20 probing could be a single packet, which would be the RSVP=20 PATH message. The RSVP path message or a similar message of=20 another protocol is transmitted from PCN ingress node to PCN=20 egress node anyway prior to flow admission and it probably=20 has the RAO set. If we rely on ECMP being based on addresses,=20 it travels the way of the flow to be admitted. Couldn't we use this single signaling packet to indicate=20 pending congestion to an egress node? Two cases may apply: 1)on IP, the packet could be PCN marked by the congested=20 router once it notes, that the packet is to be forwarded=20 across a congested PCN interface. It is forwarded on=20 slow path due to RAO, special marking shouldn't be an=20 issue. 2)if RAO can't be applied, because slow path processing is=20 not desired within the PCN domain or it is impossible=20 (as in the MPLS case), then this signaling packet should=20 be marked once traffic approaches the termination=20 threshold. That means that once the termination threshold=20 is approached, all packets crossing a link must be=20 pre-congestion marked.=20 None of the above two requires special probings to be build=20 and allows PCN at least make entering the termination=20 mode due to over admission improbable, also if ECMP is present.=20 Ramp marking remains an applicable option in the admission=20 threshold region.=20 It is a requirement however, that source and destination=20 address of the flow to be admitted are used for signaling=20 between PCN ingress and egress node prior to flow admission. Your suggestion addresses point 1) above, but you don't=20 explicitely suggest to use RSVP below. Thus your packet always=20 travels the same path as the flow to be admitted, but it no=20 longer must be a signaling packet. I'd prefer to stick with=20 the RSVP/NSIS packet approach and suggest recommending to restrict ECMP to hash on addresses if PCN and ECMP are=20 operated in a single domain.=20 Regards, Rudiger |-----Original Message----- |From: Georgios Karagiannis [mailto:karagian@cs.utwente.nl] |Sent: Friday, November 09, 2007 1:41 PM |To: pcn@ietf.org |Subject: [PCN] probing functionality | | |Hi all | |Another method that can be used for PCN probing can be applied=20 |as follows. |The probe packets are generated by the PCN_ingress_nodes in=20 |the following |way. |A probe packet belonging to a certain micro-flow is generated using the |same 5 touple (source, destination IP address, source,=20 |destination ports, |protocol number) as the packets belonging to the same micro-flow. In |addition to this the PCN_ingress_node has to set the Router Alert IP |option on the probe packet. In this way all the PCN_interior_node will |have to observe the received probe packets. | |Thus if a PCN_interior_node receives a probe packet then, due to the |Router Alert option it has to handle it differently then the=20 |user packets. |The PCN_interior_node has to PCN_mark the probe packet if it=20 |is operating |in admission control state (or flow termination state). Otherwise the |probe packet remains unmarked. | |Best regards, |Georgios | | |_______________________________________________ |PCN mailing list |PCN@ietf.org |https://www1.ietf.org/mailman/listinfo/pcn | _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Tue Nov 13 13:22:57 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Is0PZ-0004GF-Ns; Tue, 13 Nov 2007 13:22:57 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Is0PY-0004G9-NM for pcn-confirm+ok@megatron.ietf.org; Tue, 13 Nov 2007 13:22:56 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Is0PY-0004G1-Bi for pcn@ietf.org; Tue, 13 Nov 2007 13:22:56 -0500 Received: from wrzx28.rz.uni-wuerzburg.de ([132.187.3.28] helo=mailrelay.rz.uni-wuerzburg.de) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1Is0PX-0001Fl-Hr for pcn@ietf.org; Tue, 13 Nov 2007 13:22:56 -0500 Received: from virusscan.mail (localhost [127.0.0.1]) by mailrelay.mail (Postfix) with ESMTP id 457D0F150; Tue, 13 Nov 2007 19:22:52 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by virusscan.mail (Postfix) with ESMTP id 37E81F1A5; Tue, 13 Nov 2007 19:22:52 +0100 (CET) Received: from europa.informatik.uni-wuerzburg.de (wicx01.informatik.uni-wuerzburg.de [132.187.11.1]) by mailmaster.uni-wuerzburg.de (Postfix) with ESMTP id E6000F150; Tue, 13 Nov 2007 19:22:49 +0100 (CET) Received: from nero.informatik.uni-wuerzburg.de (win3005.informatik.uni-wuerzburg.de [132.187.106.5]) by europa.informatik.uni-wuerzburg.de (8.11.3/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id lADIMnI09118; Tue, 13 Nov 2007 19:22:49 +0100 Received: from [127.0.0.1] (nero.informatik.uni-wuerzburg.de [132.187.106.5]) by nero.informatik.uni-wuerzburg.de (Postfix) with ESMTP id 1DE1E6F591; Tue, 13 Nov 2007 19:15:15 +0100 (CET) Message-ID: <4739E99C.6080305@informatik.uni-wuerzburg.de> Date: Tue, 13 Nov 2007 19:14:52 +0100 From: Michael Menth Organization: University of Wuerzburg User-Agent: Thunderbird 2.0.0.6 (Windows/20070728) MIME-Version: 1.0 To: "Geib, Ruediger" Subject: Re: [PCN] probing functionality References: <1B6169C658325341A3B8066E23919E1C4C142D@S4DE8PSAANK.mitte.t-com.de> In-Reply-To: <1B6169C658325341A3B8066E23919E1C4C142D@S4DE8PSAANK.mitte.t-com.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at uni-wuerzburg.de X-Spam-Score: 0.0 (/) X-Scan-Signature: 2beba50d0fcdeee5f091c59f204d4365 Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: menth@informatik.uni-wuerzburg.de List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Ruediger, what you describe is how we also think that probing could be realized when RSVP or some other signalling protocol is used end-to-end. The initial RSVP PATH message is used for probing: if the PATH message arrives at the PCN egress and is not marked, the PCN egress node proceeds as a usual RSVP node and forwards the message further downstream. But if the message is marked, the PCN egress node sends back a PATHERR message to reject the flow. Conversely, if a RESV message returns, this implies that the PATH message was not marked at the PCN egress node. Therefore, the PCN ingress node can accept the new flow when it receives a RESV message. In 3sm and CL, all packets are marked with "admission-stop" when the PCN traffic rate exceeds the admissible rate of a link. If the RSVP control messages are transmitted as PCN traffic, they are also marked with "admission-stop" in case of pre-congestion. Therefore, no special treatment of such probing traffic is needed inside the PCN domain. Regards, Michael Geib, Ruediger wrote: > Georgios, > > maybe I'm wrong, but I think you were the one to mention that > probing could be a single packet, which would be the RSVP > PATH message. The RSVP path message or a similar message of > another protocol is transmitted from PCN ingress node to PCN > egress node anyway prior to flow admission and it probably > has the RAO set. If we rely on ECMP being based on addresses, > it travels the way of the flow to be admitted. > > Couldn't we use this single signaling packet to indicate > pending congestion to an egress node? Two cases may apply: > > 1)on IP, the packet could be PCN marked by the congested > router once it notes, that the packet is to be forwarded > across a congested PCN interface. It is forwarded on > slow path due to RAO, special marking shouldn't be an > issue. > > 2)if RAO can't be applied, because slow path processing is > not desired within the PCN domain or it is impossible > (as in the MPLS case), then this signaling packet should > be marked once traffic approaches the termination > threshold. That means that once the termination threshold > is approached, all packets crossing a link must be > pre-congestion marked. > > None of the above two requires special probings to be build > and allows PCN at least make entering the termination > mode due to over admission improbable, also if ECMP is present. > Ramp marking remains an applicable option in the admission > threshold region. > > It is a requirement however, that source and destination > address of the flow to be admitted are used for signaling > between PCN ingress and egress node prior to flow admission. > > Your suggestion addresses point 1) above, but you don't > explicitely suggest to use RSVP below. Thus your packet always > travels the same path as the flow to be admitted, but it no > longer must be a signaling packet. I'd prefer to stick with > the RSVP/NSIS packet approach and suggest recommending to > restrict ECMP to hash on addresses if PCN and ECMP are > operated in a single domain. > > Regards, > > Rudiger > > |-----Original Message----- > |From: Georgios Karagiannis [mailto:karagian@cs.utwente.nl] > |Sent: Friday, November 09, 2007 1:41 PM > |To: pcn@ietf.org > |Subject: [PCN] probing functionality > | > | > |Hi all > | > |Another method that can be used for PCN probing can be applied > |as follows. > |The probe packets are generated by the PCN_ingress_nodes in > |the following > |way. > |A probe packet belonging to a certain micro-flow is generated using the > |same 5 touple (source, destination IP address, source, > |destination ports, > |protocol number) as the packets belonging to the same micro-flow. In > |addition to this the PCN_ingress_node has to set the Router Alert IP > |option on the probe packet. In this way all the PCN_interior_node will > |have to observe the received probe packets. > | > |Thus if a PCN_interior_node receives a probe packet then, due to the > |Router Alert option it has to handle it differently then the > |user packets. > |The PCN_interior_node has to PCN_mark the probe packet if it > |is operating > |in admission control state (or flow termination state). Otherwise the > |probe packet remains unmarked. > | > |Best regards, > |Georgios > | > | > |_______________________________________________ > |PCN mailing list > |PCN@ietf.org > |https://www1.ietf.org/mailman/listinfo/pcn > | > > > _______________________________________________ > PCN mailing list > PCN@ietf.org > https://www1.ietf.org/mailman/listinfo/pcn > -- Dr. Michael Menth, Assistant Professor University of Wuerzburg, Institute of Computer Science Am Hubland, D-97074 Wuerzburg, Germany, room B206 phone: (+49)-931/888-6644, fax: (+49)-931/888-6632 mailto:menth@informatik.uni-wuerzburg.de http://www3.informatik.uni-wuerzburg.de/research/ngn _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Tue Nov 13 21:36:50 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Is87U-0003Rx-3p; Tue, 13 Nov 2007 21:36:48 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Is87T-0003Ql-PJ for pcn-confirm+ok@megatron.ietf.org; Tue, 13 Nov 2007 21:36:47 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Is87T-0003Qd-FW for pcn@ietf.org; Tue, 13 Nov 2007 21:36:47 -0500 Received: from smtp103.rog.mail.re2.yahoo.com ([206.190.36.81]) by ietf-mx.ietf.org with smtp (Exim 4.43) id 1Is87Q-0002Mb-6x for pcn@ietf.org; Tue, 13 Nov 2007 21:36:47 -0500 Received: (qmail 22959 invoked from network); 14 Nov 2007 02:36:44 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=rogers.com; h=Received:X-YMail-OSG:Message-ID:Date:From:User-Agent:MIME-Version:To:Subject:Content-Type:Content-Transfer-Encoding; b=V0afdIsKB9522TfOTKe8tPkXVW1VaY6A9Bc75SQgQG71WdiSg3dU9KV1KGSoCM+A08NYF9sf4M/3+hlMO489NEz4TtMC0xTFoaQMizExxCF4Lia+wZSLq8evcZO3+haE0jR8mcqf22HcfhD9sx5totLM8+JjB5BCFSXGtmxV958= ; Received: from unknown (HELO ?192.168.0.100?) (tom.taylor@rogers.com@99.241.147.92 with plain) by smtp103.rog.mail.re2.yahoo.com with SMTP; 14 Nov 2007 02:36:44 -0000 X-YMail-OSG: 2IOZ7BMVM1m89Tz8FkyqpbWA6zOhlC0KZXFVO0IQGJ8XECigY1aFarRmWDzWAuS.DA-- Message-ID: <473A5F3D.3070605@rogers.com> Date: Tue, 13 Nov 2007 21:36:45 -0500 From: Tom Taylor User-Agent: Thunderbird 2.0.0.6 (Windows/20070728) MIME-Version: 1.0 To: pcn Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Scan-Signature: 9ed51c9d1356100bce94f1ae4ec616a9 Subject: [PCN] [Fwd: New Version Notification for draft-briscoe-pcn-boundary-behav-00] X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org This is a start on the work of defining boundary node behaviour. At the moment it is more of a survey than a prescription. It addresses a few issues, among them the question of how boundary nodes can get the information needed to classify flows into aggregates, and what strategy egress nodes should use to send congestion level information back to the ingress nodes. Comments are welcome. -------- Original Message -------- Subject: New Version Notification for draft-briscoe-pcn-boundary-behav-00 Date: Mon, 12 Nov 2007 08:47:32 -0500 From: IETF I-D Submission Tool To: tom.taylor@rogers.com CC: bob.briscoe@bt.com, tena@huawei.com A new version of I-D, draft-briscoe-pcn-boundary-behav-00.txt has been successfuly submitted by Tom Taylor and posted to the IETF repository. Filename: draft-briscoe-pcn-boundary-behav Revision: 00 Title: PCN Boundary Node Behaviour Creation_date: 2007-11-11 WG ID: Independent Submission Number_of_pages: 18 Abstract: The Pre-Congestion Notification Architecture document defines a PCN domain and the PCN-ingress and PCN-egress nodes that form its boundary. The present document is an attempt to describe the detailed behaviour of the PCN boundary nodes. It is a contribution toward the PCN WG milestone: "Suggested Flow Admission and Termination Boundary Mechanisms". This first version is expected to evolve with discussion and further thought toward a more precise and prescriptive view of boundary node behaviour. The IETF Secretariat. _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Wed Nov 14 02:57:55 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IsD8F-0007dL-5S; Wed, 14 Nov 2007 02:57:55 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IsD8D-0007c6-Pn for pcn-confirm+ok@megatron.ietf.org; Wed, 14 Nov 2007 02:57:54 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IsD88-0007XI-Sp for pcn@ietf.org; Wed, 14 Nov 2007 02:57:48 -0500 Received: from tcmail31.telekom.de ([217.6.95.238]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IsD85-0000Tc-Ja for pcn@ietf.org; Wed, 14 Nov 2007 02:57:48 -0500 Received: from s4de8psaans.mitte.t-com.de by tcmail31.telekom.de with ESMTP; Wed, 14 Nov 2007 08:57:44 +0100 Received: from S4DE8PSAANK.mitte.t-com.de ([10.151.229.10]) by s4de8psaans.mitte.t-com.de with Microsoft SMTPSVC(6.0.3790.3959); Wed, 14 Nov 2007 08:57:43 +0100 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Subject: RE: [PCN] probing functionality Date: Wed, 14 Nov 2007 08:57:43 +0100 Message-Id: <1B6169C658325341A3B8066E23919E1C4C143A@S4DE8PSAANK.mitte.t-com.de> In-Reply-To: <4739E99C.6080305@informatik.uni-wuerzburg.de> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] probing functionality Thread-Index: AcgmIltWJGG6mX6bTgCOvFrOCOkPigAb9IjQ From: "Geib, Ruediger" To: X-OriginalArrivalTime: 14 Nov 2007 07:57:43.0794 (UTC) FILETIME=[09969D20:01C82694] X-Spam-Score: -1.0 (-) X-Scan-Signature: e472ca43d56132790a46d9eefd95f0a5 Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hello Michael, to complete the statements on RSVP, if signaling messages carry=20 the addresses of the flows they reserve resources for, also the=20 refreshes travel the same path as the data. So they can be used=20 to identify the flows already admitted, should termination be=20 pending. This is a simple way to identify admitted flows in the presence of ECMP and pre-congestion. This won't work with=20 aggregated reservations or summary refreshes, obviously. I wouldn't call that "probing", if it is just based on standard=20 signaling messages, which are transmitted anyway.=20 Regards, Rudiger |-----Original Message----- |From: Michael Menth [mailto:menth@informatik.uni-wuerzburg.de] |Sent: Tuesday, November 13, 2007 7:15 PM |To: Geib, R=FCdiger |Cc: pcn@ietf.org |Subject: Re: [PCN] probing functionality | | |Hi Ruediger, | |what you describe is how we also think that probing could be realized=20 |when RSVP or some other signalling protocol is used end-to-end. | |The initial RSVP PATH message is used for probing: if the PATH message=20 |arrives at the PCN egress and is not marked, the PCN egress node=20 |proceeds as a usual RSVP node and forwards the message further=20 |downstream. But if the message is marked, the PCN egress node=20 |sends back=20 |a PATHERR message to reject the flow. Conversely, if a RESV message=20 |returns, this implies that the PATH message was not marked at the PCN=20 |egress node. Therefore, the PCN ingress node can accept the new flow=20 |when it receives a RESV message. | |In 3sm and CL, all packets are marked with "admission-stop"=20 |when the PCN=20 |traffic rate exceeds the admissible rate of a link. If the=20 |RSVP control=20 |messages are transmitted as PCN traffic, they are also marked with=20 |"admission-stop" in case of pre-congestion. Therefore, no special=20 |treatment of such probing traffic is needed inside the PCN domain. | |Regards, | | Michael | | |Geib, Ruediger wrote: |> Georgios, |> |> maybe I'm wrong, but I think you were the one to mention that=20 |> probing could be a single packet, which would be the RSVP=20 |> PATH message. The RSVP path message or a similar message of=20 |> another protocol is transmitted from PCN ingress node to PCN=20 |> egress node anyway prior to flow admission and it probably=20 |> has the RAO set. If we rely on ECMP being based on addresses,=20 |> it travels the way of the flow to be admitted. |> |> Couldn't we use this single signaling packet to indicate=20 |> pending congestion to an egress node? Two cases may apply: |> |> 1)on IP, the packet could be PCN marked by the congested=20 |> router once it notes, that the packet is to be forwarded=20 |> across a congested PCN interface. It is forwarded on=20 |> slow path due to RAO, special marking shouldn't be an=20 |> issue. |> |> 2)if RAO can't be applied, because slow path processing is=20 |> not desired within the PCN domain or it is impossible=20 |> (as in the MPLS case), then this signaling packet should=20 |> be marked once traffic approaches the termination=20 |> threshold. That means that once the termination threshold=20 |> is approached, all packets crossing a link must be=20 |> pre-congestion marked.=20 |> |> None of the above two requires special probings to be build=20 |> and allows PCN at least make entering the termination=20 |> mode due to over admission improbable, also if ECMP is present.=20 |> Ramp marking remains an applicable option in the admission=20 |> threshold region.=20 |> |> It is a requirement however, that source and destination=20 |> address of the flow to be admitted are used for signaling=20 |> between PCN ingress and egress node prior to flow admission. |> |> Your suggestion addresses point 1) above, but you don't=20 |> explicitely suggest to use RSVP below. Thus your packet always=20 |> travels the same path as the flow to be admitted, but it no=20 |> longer must be a signaling packet. I'd prefer to stick with=20 |> the RSVP/NSIS packet approach and suggest recommending to |> restrict ECMP to hash on addresses if PCN and ECMP are=20 |> operated in a single domain.=20 |> |> Regards, |> |> Rudiger |> |> |-----Original Message----- |> |From: Georgios Karagiannis [mailto:karagian@cs.utwente.nl] |> |Sent: Friday, November 09, 2007 1:41 PM |> |To: pcn@ietf.org |> |Subject: [PCN] probing functionality |> | |> | |> |Hi all |> | |> |Another method that can be used for PCN probing can be applied=20 |> |as follows. |> |The probe packets are generated by the PCN_ingress_nodes in=20 |> |the following |> |way. |> |A probe packet belonging to a certain micro-flow is=20 |generated using the |> |same 5 touple (source, destination IP address, source,=20 |> |destination ports, |> |protocol number) as the packets belonging to the same micro-flow. In |> |addition to this the PCN_ingress_node has to set the Router Alert IP |> |option on the probe packet. In this way all the=20 |PCN_interior_node will |> |have to observe the received probe packets. |> | |> |Thus if a PCN_interior_node receives a probe packet then, due to the |> |Router Alert option it has to handle it differently then the=20 |> |user packets. |> |The PCN_interior_node has to PCN_mark the probe packet if it=20 |> |is operating |> |in admission control state (or flow termination state).=20 |Otherwise the |> |probe packet remains unmarked. |> | |> |Best regards, |> |Georgios |> | |> | |> |_______________________________________________ |> |PCN mailing list |> |PCN@ietf.org |> |https://www1.ietf.org/mailman/listinfo/pcn |> | |> |> |> _______________________________________________ |> PCN mailing list |> PCN@ietf.org |> https://www1.ietf.org/mailman/listinfo/pcn |> =20 | |--=20 |Dr. Michael Menth, Assistant Professor |University of Wuerzburg, Institute of Computer Science |Am Hubland, D-97074 Wuerzburg, Germany, room B206 |phone: (+49)-931/888-6644, fax: (+49)-931/888-6632 |mailto:menth@informatik.uni-wuerzburg.de |http://www3.informatik.uni-wuerzburg.de/research/ngn | | | |_______________________________________________ |PCN mailing list |PCN@ietf.org |https://www1.ietf.org/mailman/listinfo/pcn | _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Wed Nov 14 03:11:36 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IsDLU-0002mW-Bj; Wed, 14 Nov 2007 03:11:36 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IsDLS-0002mM-TZ for pcn-confirm+ok@megatron.ietf.org; Wed, 14 Nov 2007 03:11:34 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IsDLS-0002mE-K1 for pcn@ietf.org; Wed, 14 Nov 2007 03:11:34 -0500 Received: from wrzx28.rz.uni-wuerzburg.de ([132.187.3.28] helo=mailrelay.rz.uni-wuerzburg.de) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IsDLO-0000qM-4x for pcn@ietf.org; Wed, 14 Nov 2007 03:11:34 -0500 Received: from virusscan.mail (localhost [127.0.0.1]) by mailrelay.mail (Postfix) with ESMTP id 48475C156; Wed, 14 Nov 2007 09:11:29 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by virusscan.mail (Postfix) with ESMTP id 60D8BC15B; Wed, 14 Nov 2007 09:11:28 +0100 (CET) Received: from europa.informatik.uni-wuerzburg.de (wicx01.informatik.uni-wuerzburg.de [132.187.11.1]) by mailmaster.uni-wuerzburg.de (Postfix) with ESMTP id 2BF91C14F; Wed, 14 Nov 2007 09:11:27 +0100 (CET) Received: from nero.informatik.uni-wuerzburg.de (win3005.informatik.uni-wuerzburg.de [132.187.106.5]) by europa.informatik.uni-wuerzburg.de (8.11.3/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id lAE8BRI13134; Wed, 14 Nov 2007 09:11:27 +0100 Received: from [127.0.0.1] (nero.informatik.uni-wuerzburg.de [132.187.106.5]) by nero.informatik.uni-wuerzburg.de (Postfix) with ESMTP id 1066A6F591; Wed, 14 Nov 2007 09:03:50 +0100 (CET) Message-ID: <473AABCB.1000608@informatik.uni-wuerzburg.de> Date: Wed, 14 Nov 2007 09:03:23 +0100 From: Michael Menth Organization: University of Wuerzburg User-Agent: Thunderbird 2.0.0.6 (Windows/20070728) MIME-Version: 1.0 To: "Geib, Ruediger" Subject: Re: [PCN] probing functionality References: <1B6169C658325341A3B8066E23919E1C4C143A@S4DE8PSAANK.mitte.t-com.de> In-Reply-To: <1B6169C658325341A3B8066E23919E1C4C143A@S4DE8PSAANK.mitte.t-com.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed X-Virus-Scanned: by amavisd-new at uni-wuerzburg.de Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Scan-Signature: 88b11fc64c1bfdb4425294ef5374ca07 Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: menth@informatik.uni-wuerzburg.de List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Ruediger, Geib, Ruediger wrote: > Hello Michael, > > to complete the statements on RSVP, if signaling messages carry=20 > the addresses of the flows they reserve resources for, also the=20 > refreshes travel the same path as the data. So they can be used=20 > to identify the flows already admitted, should termination be=20 > pending. This is a simple way to identify admitted flows in the > presence of ECMP and pre-congestion. This won't work with=20 > aggregated reservations=20 True - this is a solution only for end-to-end RSVP. What you mention=20 above would work with some kind of tunneling (MPLS, IP-in-IP).=20 Therefore, we probably need to look at different deployment scenarios. > or summary refreshes, obviously. > =20 As long as the first PATH message and RESV message are transmitted as=20 single messages I don't see a problem. > I wouldn't call that "probing", if it is just based on standard=20 > signaling messages, which are transmitted anyway.=20 > =20 Yes, this is rather lightweight in the sense that it doesn't require=20 extra messages. Therefore, one even hesitates to call it probing.=20 Nevertheless, the PATH message is reused for probing and fulfills this=20 function. Regards, Michael > Regards, > > Rudiger > > |-----Original Message----- > |From: Michael Menth [mailto:menth@informatik.uni-wuerzburg.de] > |Sent: Tuesday, November 13, 2007 7:15 PM > |To: Geib, R=FCdiger > |Cc: pcn@ietf.org > |Subject: Re: [PCN] probing functionality > | > | > |Hi Ruediger, > | > |what you describe is how we also think that probing could be realized=20 > |when RSVP or some other signalling protocol is used end-to-end. > | > |The initial RSVP PATH message is used for probing: if the PATH message= =20 > |arrives at the PCN egress and is not marked, the PCN egress node=20 > |proceeds as a usual RSVP node and forwards the message further=20 > |downstream. But if the message is marked, the PCN egress node=20 > |sends back=20 > |a PATHERR message to reject the flow. Conversely, if a RESV message=20 > |returns, this implies that the PATH message was not marked at the PCN=20 > |egress node. Therefore, the PCN ingress node can accept the new flow=20 > |when it receives a RESV message. > | > |In 3sm and CL, all packets are marked with "admission-stop"=20 > |when the PCN=20 > |traffic rate exceeds the admissible rate of a link. If the=20 > |RSVP control=20 > |messages are transmitted as PCN traffic, they are also marked with=20 > |"admission-stop" in case of pre-congestion. Therefore, no special=20 > |treatment of such probing traffic is needed inside the PCN domain. > | > |Regards, > | > | Michael > | > | > |Geib, Ruediger wrote: > |> Georgios, > |> > |> maybe I'm wrong, but I think you were the one to mention that=20 > |> probing could be a single packet, which would be the RSVP=20 > |> PATH message. The RSVP path message or a similar message of=20 > |> another protocol is transmitted from PCN ingress node to PCN=20 > |> egress node anyway prior to flow admission and it probably=20 > |> has the RAO set. If we rely on ECMP being based on addresses,=20 > |> it travels the way of the flow to be admitted. > |> > |> Couldn't we use this single signaling packet to indicate=20 > |> pending congestion to an egress node? Two cases may apply: > |> > |> 1)on IP, the packet could be PCN marked by the congested=20 > |> router once it notes, that the packet is to be forwarded=20 > |> across a congested PCN interface. It is forwarded on=20 > |> slow path due to RAO, special marking shouldn't be an=20 > |> issue. > |> > |> 2)if RAO can't be applied, because slow path processing is=20 > |> not desired within the PCN domain or it is impossible=20 > |> (as in the MPLS case), then this signaling packet should=20 > |> be marked once traffic approaches the termination=20 > |> threshold. That means that once the termination threshold=20 > |> is approached, all packets crossing a link must be=20 > |> pre-congestion marked.=20 > |> > |> None of the above two requires special probings to be build=20 > |> and allows PCN at least make entering the termination=20 > |> mode due to over admission improbable, also if ECMP is present.=20 > |> Ramp marking remains an applicable option in the admission=20 > |> threshold region.=20 > |> > |> It is a requirement however, that source and destination=20 > |> address of the flow to be admitted are used for signaling=20 > |> between PCN ingress and egress node prior to flow admission. > |> > |> Your suggestion addresses point 1) above, but you don't=20 > |> explicitely suggest to use RSVP below. Thus your packet always=20 > |> travels the same path as the flow to be admitted, but it no=20 > |> longer must be a signaling packet. I'd prefer to stick with=20 > |> the RSVP/NSIS packet approach and suggest recommending to > |> restrict ECMP to hash on addresses if PCN and ECMP are=20 > |> operated in a single domain.=20 > |> > |> Regards, > |> > |> Rudiger > |> > |> |-----Original Message----- > |> |From: Georgios Karagiannis [mailto:karagian@cs.utwente.nl] > |> |Sent: Friday, November 09, 2007 1:41 PM > |> |To: pcn@ietf.org > |> |Subject: [PCN] probing functionality > |> | > |> | > |> |Hi all > |> | > |> |Another method that can be used for PCN probing can be applied=20 > |> |as follows. > |> |The probe packets are generated by the PCN_ingress_nodes in=20 > |> |the following > |> |way. > |> |A probe packet belonging to a certain micro-flow is=20 > |generated using the > |> |same 5 touple (source, destination IP address, source,=20 > |> |destination ports, > |> |protocol number) as the packets belonging to the same micro-flow. I= n > |> |addition to this the PCN_ingress_node has to set the Router Alert I= P > |> |option on the probe packet. In this way all the=20 > |PCN_interior_node will > |> |have to observe the received probe packets. > |> | > |> |Thus if a PCN_interior_node receives a probe packet then, due to th= e > |> |Router Alert option it has to handle it differently then the=20 > |> |user packets. > |> |The PCN_interior_node has to PCN_mark the probe packet if it=20 > |> |is operating > |> |in admission control state (or flow termination state).=20 > |Otherwise the > |> |probe packet remains unmarked. > |> | > |> |Best regards, > |> |Georgios > |> | > |> | > |> |_______________________________________________ > |> |PCN mailing list > |> |PCN@ietf.org > |> |https://www1.ietf.org/mailman/listinfo/pcn > |> | > |> > |> > |> _______________________________________________ > |> PCN mailing list > |> PCN@ietf.org > |> https://www1.ietf.org/mailman/listinfo/pcn > |> =20 > | > |--=20 > |Dr. Michael Menth, Assistant Professor > |University of Wuerzburg, Institute of Computer Science > |Am Hubland, D-97074 Wuerzburg, Germany, room B206 > |phone: (+49)-931/888-6644, fax: (+49)-931/888-6632 > |mailto:menth@informatik.uni-wuerzburg.de > |http://www3.informatik.uni-wuerzburg.de/research/ngn > | > | > | > |_______________________________________________ > |PCN mailing list > |PCN@ietf.org > |https://www1.ietf.org/mailman/listinfo/pcn > | > =20 --=20 Dr. Michael Menth, Assistant Professor University of Wuerzburg, Institute of Computer Science Am Hubland, D-97074 Wuerzburg, Germany, room B206 phone: (+49)-931/888-6644, fax: (+49)-931/888-6632 mailto:menth@informatik.uni-wuerzburg.de http://www3.informatik.uni-wuerzburg.de/research/ngn _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Wed Nov 14 07:24:52 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IsHIZ-0003LU-Fx; Wed, 14 Nov 2007 07:24:51 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IsHIY-0003GW-F9 for pcn-confirm+ok@megatron.ietf.org; Wed, 14 Nov 2007 07:24:50 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IsHIX-0003EK-Sv for pcn@ietf.org; Wed, 14 Nov 2007 07:24:50 -0500 Received: from imr2.ericy.com ([198.24.6.3]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IsHIR-0007z8-Dz for pcn@ietf.org; Wed, 14 Nov 2007 07:24:49 -0500 Received: from eusrcmw750.eamcs.ericsson.se (eusrcmw750.exu.ericsson.se [138.85.77.50]) by imr2.ericy.com (8.13.1/8.13.1) with ESMTP id lAECNaNF029099 for ; Wed, 14 Nov 2007 06:24:43 -0600 Received: from eusrcmw721.eamcs.ericsson.se ([138.85.77.21]) by eusrcmw750.eamcs.ericsson.se with Microsoft SMTPSVC(6.0.3790.1830); Wed, 14 Nov 2007 06:24:22 -0600 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----_=_NextPart_001_01C826B9.49841AAA" Date: Wed, 14 Nov 2007 06:24:21 -0600 Message-ID: X-MS-Has-Attach: yes X-MS-TNEF-Correlator: Thread-Topic: PCN: Updating draft-westberg-pcn-load-control with 02 version Thread-Index: AcgmuHKXz6UILcUHSPqSP9NNJDch8wAAMkRw From: "Anurag Bhargava" To: X-OriginalArrivalTime: 14 Nov 2007 12:24:22.0476 (UTC) FILETIME=[498BC0C0:01C826B9] X-Spam-Score: 0.0 (/) X-Scan-Signature: 0983761b4f0fa8fd9edab7fc88282461 Subject: [PCN] FW: PCN: Updating draft-westberg-pcn-load-control with 02 version X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org This is a multi-part message in MIME format. ------_=_NextPart_001_01C826B9.49841AAA Content-Type: multipart/alternative; boundary="----_=_NextPart_002_01C826B9.49841AAA" ------_=_NextPart_002_01C826B9.49841AAA Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable FYI, Thanks -Anurag Bhargava, Ph.D. > ______________________________________________=20 > From: Anurag Bhargava =20 > Sent: Wednesday, November 14, 2007 7:18 AM > To: 'internet-drafts@ietf.org' > Cc: 'Georgios Karagiannis'; Lars Westberg > Subject: PCN: Updating draft-westberg-pcn-load-control with 02 > version >=20 >=20 >=20 > <>=20 > BR, > -Anurag Bhargava, Ph.D. >=20 ------_=_NextPart_002_01C826B9.49841AAA Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable FW: PCN: Updating draft-westberg-pcn-load-control with 02 = version

FYI,

Thanks
-Anurag Bhargava, Ph.D.


    ______________________________________________
    From:   Anurag Bhargava 
    Sent:   Wednesday, November 14, 2007 7:18 AM
    To:     'internet-drafts@ietf.org'
    Cc:     'Georgios Karagiannis'; Lars Westberg
    Subject:       = PCN: Updating = draft-westberg-pcn-load-control with 02 version



    = <<draft-westberg-pcn-load-control-02.txt>>
    BR,
    -Anurag Bhargava, Ph.D.

------_=_NextPart_002_01C826B9.49841AAA-- ------_=_NextPart_001_01C826B9.49841AAA Content-Type: text/plain; name="draft-westberg-pcn-load-control-02.txt" Content-Transfer-Encoding: base64 Content-Description: draft-westberg-pcn-load-control-02.txt Content-Disposition: attachment; filename="draft-westberg-pcn-load-control-02.txt" CgoKUENOICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgIEwuIFdlc3RiZXJnCkludGVybmV0LURyYWZ0ICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICBBLiBCaGFyZ2F2YQpJbnRlbmRlZCBzdGF0dXM6IFN0YW5k YXJkcyBUcmFjayAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgQS4gQmFkZXIKRXhwaXJl czogTWF5IDE3LCAyMDA4ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg IEVyaWNzc29uCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICBHLiBLYXJhZ2lhbm5pcwogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgVW5pdmVyc2l0eSBvZiBUd2VudGUKICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIE5vdmVtYmVyIDE0LCAyMDA3 CgoKICAgICAgICAgICAgICAgICBMQy1QQ046IFRoZSBMb2FkIENvbnRyb2wgUENOIFNvbHV0aW9u CiAgICAgICAgICAgICAgICAgICBkcmFmdC13ZXN0YmVyZy1wY24tbG9hZC1jb250cm9sLTAyCgpT dGF0dXMgb2YgdGhpcyBNZW1vCgogICBCeSBzdWJtaXR0aW5nIHRoaXMgSW50ZXJuZXQtRHJhZnQs IGVhY2ggYXV0aG9yIHJlcHJlc2VudHMgdGhhdCBhbnkKICAgYXBwbGljYWJsZSBwYXRlbnQgb3Ig b3RoZXIgSVBSIGNsYWltcyBvZiB3aGljaCBoZSBvciBzaGUgaXMgYXdhcmUKICAgaGF2ZSBiZWVu IG9yIHdpbGwgYmUgZGlzY2xvc2VkLCBhbmQgYW55IG9mIHdoaWNoIGhlIG9yIHNoZSBiZWNvbWVz CiAgIGF3YXJlIHdpbGwgYmUgZGlzY2xvc2VkLCBpbiBhY2NvcmRhbmNlIHdpdGggU2VjdGlvbiA2 IG9mIEJDUCA3OS4KCiAgIEludGVybmV0LURyYWZ0cyBhcmUgd29ya2luZyBkb2N1bWVudHMgb2Yg dGhlIEludGVybmV0IEVuZ2luZWVyaW5nCiAgIFRhc2sgRm9yY2UgKElFVEYpLCBpdHMgYXJlYXMs IGFuZCBpdHMgd29ya2luZyBncm91cHMuICBOb3RlIHRoYXQKICAgb3RoZXIgZ3JvdXBzIG1heSBh bHNvIGRpc3RyaWJ1dGUgd29ya2luZyBkb2N1bWVudHMgYXMgSW50ZXJuZXQtCiAgIERyYWZ0cy4K CiAgIEludGVybmV0LURyYWZ0cyBhcmUgZHJhZnQgZG9jdW1lbnRzIHZhbGlkIGZvciBhIG1heGlt dW0gb2Ygc2l4IG1vbnRocwogICBhbmQgbWF5IGJlIHVwZGF0ZWQsIHJlcGxhY2VkLCBvciBvYnNv bGV0ZWQgYnkgb3RoZXIgZG9jdW1lbnRzIGF0IGFueQogICB0aW1lLiAgSXQgaXMgaW5hcHByb3By aWF0ZSB0byB1c2UgSW50ZXJuZXQtRHJhZnRzIGFzIHJlZmVyZW5jZQogICBtYXRlcmlhbCBvciB0 byBjaXRlIHRoZW0gb3RoZXIgdGhhbiBhcyAid29yayBpbiBwcm9ncmVzcy4iCgogICBUaGUgbGlz dCBvZiBjdXJyZW50IEludGVybmV0LURyYWZ0cyBjYW4gYmUgYWNjZXNzZWQgYXQKICAgaHR0cDov L3d3dy5pZXRmLm9yZy9pZXRmLzFpZC1hYnN0cmFjdHMudHh0LgoKICAgVGhlIGxpc3Qgb2YgSW50 ZXJuZXQtRHJhZnQgU2hhZG93IERpcmVjdG9yaWVzIGNhbiBiZSBhY2Nlc3NlZCBhdAogICBodHRw Oi8vd3d3LmlldGYub3JnL3NoYWRvdy5odG1sLgoKICAgVGhpcyBJbnRlcm5ldC1EcmFmdCB3aWxs IGV4cGlyZSBvbiBNYXkgMTcsIDIwMDguCgpDb3B5cmlnaHQgTm90aWNlCgogICBDb3B5cmlnaHQg KEMpIFRoZSBJRVRGIFRydXN0ICgyMDA3KS4KCkFic3RyYWN0CgogICBUaGVyZSBpcyBhbiBpbmNy ZWFzZWQgaW50ZXJlc3Qgb2Ygc2ltcGxlIGFuZCBzY2FsYWJsZSByZXNvdXJjZQogICBwcm92aXNp b25pbmcgc29sdXRpb24gZm9yIERpZmZzZXJ2IG5ldHdvcmsuICBUaGUgTG9hZCBDb250cm9sIFBD TgogICAoTEMtUENOKSBhZGRyZXNzZXMgdGhlIGZvbGxvd2luZyBpc3N1ZXM6CgoKCgoKV2VzdGJl cmcsIGV0IGFsLiAgICAgICAgICBFeHBpcmVzIE1heSAxNywgMjAwOCAgICAgICAgICAgICAgICAg IFtQYWdlIDFdCgwKSW50ZXJuZXQtRHJhZnQgICAgICAgICAgICAgICAgICAgTEMtUENOICAgICAg ICAgICAgICAgICAgICBOb3ZlbWJlciAyMDA3CgoKICAgbyAgQWRtaXNzaW9uIENvbnRyb2wgZm9y IHJlYWwgdGltZSBkYXRhIGZsb3dzIGluIHN0YXRlbGVzcyBEaWZmc2VydgogICAgICBEb21haW5z CgogICBvICBGbG93IFRlcm1pbmF0aW9uOiBUZXJtaW5hdGlvbiBvZiBmbG93cyBpbiBjYXNlIG9m IGV4Y2VwdGlvbmFsCiAgICAgIGV2ZW50cywgc3VjaCBhcyBzZXZlcmUgY29uZ2VzdGlvbiBhZnRl ciByZS1yb3V0aW5nLgoKICAgQWRtaXNzaW9uIGNvbnRyb2wgaW4gYSBEaWZmc2VydiBzdGF0ZWxl c3MgZG9tYWluIGlzIGEgY29tYmluYXRpb24gb2Y6CgogICBvICBQcm9iaW5nLCB3aGVyZWJ5IGEg cHJvYmUgcGFja2V0IGlzIHNlbnQgYWxvbmcgdGhlIGZvcndhcmRpbmcgcGF0aAogICAgICBpbiBh IG5ldHdvcmsgdG8gZGV0ZXJtaW5lIHdoZXRoZXIgYSBmbG93IGNhbiBiZSBhZG1pdHRlZCBiYXNl ZAogICAgICB1cG9uIHRoZSBjdXJyZW50IGNvbmdlc3Rpb24gc3RhdGUgb2YgdGhlIG5ldHdvcmsK CiAgIG8gIEFkbWlzc2lvbiBDb250cm9sIGJhc2VkIG9uIGRhdGEgbWFya2luZywgd2hlcmVieSBp biBjb25nZXN0aW9uCiAgICAgIHNpdHVhdGlvbnMgdGhlIGRhdGEgcGFja2V0cyBhcmUgbWFya2Vk IHRvIG5vdGlmeSB0aGUgUENOLWVncmVzcy0KICAgICAgbm9kZSB0aGF0IGEgY29uZ2VzdGlvbiBv Y2N1cnJlZCBvbiBhIHBhcnRpY3VsYXIgUENOLWluZ3Jlc3Mtbm9kZQogICAgICB0byBQQ04tZWdy ZXNzLW5vZGUgcGF0aC4KCiAgIFRoZSBzY2hlbWUgcHJvdmlkZXMgdGhlIGNhcGFiaWxpdHkgb2Yg Y29udHJvbGxpbmcgdGhlIHRyYWZmaWMgbG9hZCBpbgogICB0aGUgbmV0d29yayB3aXRob3V0IHJl cXVpcmluZyBzaWduYWxpbmcgb3IgYW55IHBlci1mbG93IHByb2Nlc3NpbmcgaW4KICAgdGhlIFBD Ti1pbnRlcmlvci1ub2Rlcy4gIFRoZSBjb21wbGV4aXR5IG9mIExvYWQgQ29udHJvbCBpcyBrZXB0 IHRvIGEKICAgbWluaW11bSB0byBtYWtlIGltcGxlbWVudGF0aW9uIHNpbXBsZS4KCgoKCgoKCgoK CgoKCgoKCgoKCgoKCgoKCgoKCgoKV2VzdGJlcmcsIGV0IGFsLiAgICAgICAgICBFeHBpcmVzIE1h eSAxNywgMjAwOCAgICAgICAgICAgICAgICAgIFtQYWdlIDJdCgwKSW50ZXJuZXQtRHJhZnQgICAg ICAgICAgICAgICAgICAgTEMtUENOICAgICAgICAgICAgICAgICAgICBOb3ZlbWJlciAyMDA3CgoK VGFibGUgb2YgQ29udGVudHMKCiAgIDEuICBJbnRyb2R1Y3Rpb24gLiAuIC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAgNAogICAyLiAgVGVybWlub2xvZ3kgIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gIDUKICAgMy4g IExDLVBDTiBPdmVydmlldyAgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAu IC4gLiAuICA1CiAgICAgMy4xLiAgQWRtaXNzaW9uIGNvbnRyb2wgYmFzZWQgb24gcHJvYmluZyAu IC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAgNQogICAgIDMuMi4gIEZsb3cgVGVybWluYXRpb24gLiAu IC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gIDYKICAgICAzLjMuICBDb21t b24gUENOIG5vZGUgY29uZmlndXJhdGlvbnMgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuICA3 CiAgICAgMy40LiAgQ29uZmlndXJhdGlvbiBvZiBlZGdlIG5vZGVzICAuIC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAxMAogICA0LiAgTEMtUENOIGRldGFpbGVkIGRlc2NyaXB0aW9uICAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gMTMKICAgICA0LjEuICBBZG1pc3Npb24gY29u dHJvbCBiYXNlZCBvbiBwcm9iaW5nIGZvciB1bmlkaXJlY3Rpb25hbAogICAgICAgICAgIGZsb3dz ICAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gMTMK ICAgICAgIDQuMS4xLiAgT3BlcmF0aW9uIGluIFBDTi1pbmdyZXNzLW5vZGVzIC4gLiAuIC4gLiAu IC4gLiAuIC4gLiAuIDEzCiAgICAgICA0LjEuMi4gIE9wZXJhdGlvbiBpbiBQQ04taW50ZXJpb3It bm9kZXMgIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAxNAogICAgICAgNC4xLjMuICBPcGVyYXRpb24g aW4gUENOLWVncmVzcy1ub2RlcyAgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gMTYKICAgICA0LjIu ICBGbG93IFRlcm1pbmF0aW9uIGZvciB1bmlkaXJlY3Rpb25hbCBmbG93cyAgLiAuIC4gLiAuIC4g LiAuIDE3CiAgICAgICA0LjIuMS4gIE9wZXJhdGlvbiBpbiB0aGUgUENOLWluZ3Jlc3Mtbm9kZXMg LiAuIC4gLiAuIC4gLiAuIC4gLiAxNwogICAgICAgNC4yLjIuICBPcGVyYXRpb24gaW4gdGhlIFBD Ti1pbnRlcmlvci1ub2RlcyAgLiAuIC4gLiAuIC4gLiAuIC4gMTgKICAgICAgIDQuMi4zLiAgT3Bl cmF0aW9uIGluIHRoZSBQQ04tZWdyZXNzLW5vZGVzICAuIC4gLiAuIC4gLiAuIC4gLiAuIDIzCiAg ICAgNC4zLiAgQWRtaXNzaW9uIGNvbnRyb2wgYmFzZWQgb24gcHJvYmluZyBmb3IgYmktZGlyZWN0 aW9uYWwKICAgICAgICAgICBmbG93cyAgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAu IC4gLiAuIC4gLiAuIC4gLiAuIDI2CiAgICAgNC40LiAgRmxvdyBUZXJtaW5hdGlvbiBoYW5kbGlu ZyBmb3IgYmktZGlyZWN0aW9uYWwgZmxvd3MgLiAuIC4gLiAyNwogICA1LiAgU2VjdXJpdHkgQ29u c2lkZXJhdGlvbnMgIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gMzIKICAg Ni4gIElBTkEgQ29uc2lkZXJhdGlvbnMgIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIDMyCiAgIDcuICBBY2tub3dsZWRnZW1lbnRzIC4gLiAuIC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAzMgogICA4LiAgSW5mb3JtYXRpdmUgUmVmZXJlbmNl cyAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gMzIKICAgQXV0aG9ycycg QWRkcmVzc2VzIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAu IDM0CiAgIEludGVsbGVjdHVhbCBQcm9wZXJ0eSBhbmQgQ29weXJpZ2h0IFN0YXRlbWVudHMgLiAu IC4gLiAuIC4gLiAuIC4gLiAzNQoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCldlc3RiZXJnLCBldCBh bC4gICAgICAgICAgRXhwaXJlcyBNYXkgMTcsIDIwMDggICAgICAgICAgICAgICAgICBbUGFnZSAz XQoMCkludGVybmV0LURyYWZ0ICAgICAgICAgICAgICAgICAgIExDLVBDTiAgICAgICAgICAgICAg ICAgICAgTm92ZW1iZXIgMjAwNwoKCjEuICBJbnRyb2R1Y3Rpb24KCiAgIFRoZSBhbW91bnQgb2Yg dHJhZmZpYyBjYXJyaWVkIG9uIHRoZSBJbnRlcm5ldCBpcyBub3cgZ3JlYXRlciB0aGFuIHRoZQog ICB0cmFmZmljIG9uIHRoZSB3b3JsZCdzIHRlbGVwaG9ueSBuZXR3b3JrLiAgU3RpbGwsIEludGVy bmV0LWJhc2VkCiAgIGNvbW11bmljYXRpb24gc2VydmljZXMgZ2VuZXJhdGUgbGVzcyBpbmNvbWUg dGhhbiBwbGFpbiBvbGQgdGVsZXBob255CiAgIHNlcnZpY2VzLiAgRW5hYmxpbmcgdmFsdWUtYWRk ZWQgc2VydmljZXMgb3ZlciB0aGUgSW50ZXJuZXQgaXMKICAgdGhlcmVmb3JlIGNydWNpYWwgZm9y IHNlcnZpY2UgcHJvdmlkZXJzLiAgT25lIHNpZ25pZmljYW50IGNsYXNzIG9mCiAgIHN1Y2ggdmFs dWUtYWRkZWQgc2VydmljZXMgcmVxdWlyZXMgcmVhbC10aW1lIHBhY2tldCB0cmFuc3BvcnRhdGlv bi4KICAgSXQgY2FuIGJlIGV4cGVjdGVkIHRoYXQgdGhlc2UgcmVhbC10aW1lIHNlcnZpY2VzIHdp bGwgYmUgcG9wdWxhciBhcwogICB0aGV5IHJlcGxpY2F0ZSBvciBhcmUgbmF0dXJhbCBleHRlbnNp b25zIG9mIGV4aXN0aW5nIGNvbW11bmljYXRpb24KICAgc2VydmljZXMgbGlrZSB0ZWxlcGhvbnku ICBFeGFjdCBhbmQgcmVsaWFibGUgcmVzb3VyY2UgbWFuYWdlbWVudAogICAoZS5nLiwgYWRtaXNz aW9uIGNvbnRyb2wpIGlzIGVzc2VudGlhbCBmb3IgYWNoaWV2aW5nIGhpZ2ggdXRpbGl6YXRpb24K ICAgaW4gbmV0d29ya3Mgd2l0aCByZWFsLXRpbWUgdHJhbnNwb3J0YXRpb24gY2FwYWJpbGl0aWVz LiAgVGhlIHByb2JsZW0KICAgaXMgZGlmZmljdWx0IG1haW5seSBkdWUgdG8gc2NhbGFiaWxpdHkg aXNzdWVzLgoKICAgV2l0aCB0aGUgaW50cm9kdWN0aW9uIG9mIGRpZmZlcmVudGlhdGVkIHNlcnZp Y2VzIChEUykgW1JGQzI0NzVdLCBpdAogICBpcyBub3cgcG9zc2libGUgdG8gcHJvdmlkZSBsYXJn ZSBzY2FsZSwgcmVhbC10aW1lIHNlcnZpY2VzLiAgVGhlCiAgIGJhc2ljIGlkZWEgb2YgRGlmZlNl cnYgaXMgdGhhdCwgcmF0aGVyIHRoYW4gY2xhc3NpZnlpbmcgcGFja2V0cyBhdAogICBlYWNoIHJv dXRlciwgcGFja2V0cyBhcmUgb25seSBjbGFzc2lmaWVkIGF0IHRoZSBlZGdlIGRldmljZXMuICBU aGUKICAgcmVzdWx0IC0gdGhlIHJlcXVpcmVkIHBhY2tldCB0cmVhdG1lbnQgLSBpcyBzdG9yZWQg YW5kIGNhcnJpZWQgaW4gdGhlCiAgIHBhY2tldCBoZWFkZXJzLCBhbmQgY29yZSByb3V0ZXJzIGNh biBjYXJyeSBvdXQgYXBwcm9wcmlhdGUKICAgc2NoZWR1bGluZy4KCiAgIFRoZSBjdXJyZW50IGRl ZmluaXRpb24gb2YgRGlmZlNlcnYsIGhvd2V2ZXIsIGRvZXMgbm90IGNvbnRhaW4gYW55CiAgIHNp bXBsZSwgc2NhbGFibGUgc29sdXRpb24gdG8gdGhlIHByb2JsZW0gb2YgcmVzb3VyY2UgcHJvdmlz aW9uaW5nIGFuZAogICBjb250cm9sLiAgQSBudW1iZXIgb2YgYXBwcm9hY2hlcyB0byBzb2x2aW5n IHRoZSBwcm9ibGVtIGFscmVhZHkgZXhpc3QKICAgW1JGQzMxNzVdLCBbQmVyc29uOTddLCBbU3Rv aWNhOTldLCBbQmVybmV0OTldLiAgVGhlIHNjaGVtZSBwcmVzZW50ZWQKICAgaW4gdGhpcyBkb2N1 bWVudCBkb2VzIG5vdCByZXF1aXJlIGFueSBzdGF0ZSBhZ2dyZWdhdGlvbiBhbmQgYWltcyBhdAog ICBleHRyZW1lIHNpbXBsaWNpdHkgYW5kIGxvdyBjb3N0IG9mIGltcGxlbWVudGF0aW9uIGFsb25n IHdpdGggZ29vZAogICBzY2FsaW5nIHByb3BlcnRpZXMuICBMb2FkIGNvbnRyb2wgb3BlcmF0ZXMg ZWRnZS10by1lZGdlIGluIGEgRFMKICAgZG9tYWluLCBvciBiZXR3ZWVuIHR3byBSU1ZQIG9yIE5T SVMgY2FwYWJsZSByb3V0ZXJzLCB3aGVyZSBvbmx5IHRoZQogICBlZGdlIGRldmljZXMga2VlcCBm bG93IHN0YXRlIGFuZCBkbyBwZXItZmxvdyBwcm9jZXNzaW5nLiAgVGhlIG1haW4KICAgcHVycG9z ZSBvZiBMb2FkIENvbnRyb2wgaXMgdG8gcHJvdmlkZSBhIHNpbXBsZSBhbmQgc2NhbGFibGUgc29s dXRpb24KICAgdG8gdGhlIHJlc291cmNlIHByb3Zpc2lvbmluZyBwcm9ibGVtLgoKICAgVGhlIG9y aWdpbmFsIExvYWQgQ29udHJvbCBjb25jZXB0LCBzdWJtaXR0ZWQgaW4gQXByaWwgMjAwMCwKICAg W1dlc3RiZXJnMDBdLCBoYXMgYmVlbiBkZXZlbG9wZWQgZnVydGhlciB0byBhIHNpZ25hbGluZyBj b25jZXB0IG5hbWVkCiAgIFJlc291cmNlIE1hbmFnZW1lbnQgaW4gRGlmZnNlcnYuICBSTUQgd2Fz IGluY29ycG9yYXRlZCBieSBOU0lTCiAgIHdvcmtpbmcgZ3JvdXAsIHdoZXJlIHRoZSBwcm90b2Nv bCBkZXRhaWxzIHdlcmUgd29ya2VkIG91dCBmb3IgdXNpbmcKICAgTlNJUyBhcyBleHRlcm5hbCBw cm90b2NvbCBbUk1EXS4gIFJlY2VudGx5IG5ldyBkcmFmdHMgaGF2ZSBiZWVuCiAgIHN1Ym1pdHRl ZCBhaW1pbmcgdG8gc3RhbmRhcmRpemUgbmV3IERpZmZzZXJ2IFBIQiB0aGF0IHByb3ZpZGVzCiAg IGNvbnRyb2xsZWQgbG9hZCBzZXJ2aWNlcyBpbiBEaWZmc2VydiBkb21haW5zIFtDTC1QSEJdLCBb Q0wtQVJDSF0sCiAgIFtCYWJpMDddLCBbQ2hhcjA3XS4gIFRoZXNlIGNvbmNlcHRzIGFyZSB2ZXJ5 IHNpbWlsYXIgdG8gdGhlIG9yaWdpbmFsCiAgIHR3by1iaXQgbWFya2luZyBzY2hlbWUgb2YgTG9h ZCBDb250cm9sLgoKICAgVGhpcyBkb2N1bWVudCBhaW1zIHRvIGRldmVsb3AgYSBjb21tb24gZnJh bWV3b3JrIHRoYXQgY291bGQgYmUgdXNlZAogICBib3RoIHdpdGggUlNWUCBhbmQgTlNJUyBleHRl cm5hbCBwcm90b2NvbHMuCgoKCgpXZXN0YmVyZywgZXQgYWwuICAgICAgICAgIEV4cGlyZXMgTWF5 IDE3LCAyMDA4ICAgICAgICAgICAgICAgICAgW1BhZ2UgNF0KDApJbnRlcm5ldC1EcmFmdCAgICAg ICAgICAgICAgICAgICBMQy1QQ04gICAgICAgICAgICAgICAgICAgIE5vdmVtYmVyIDIwMDcKCgog ICBUaGUgcmVtYWluZGVyIG9mIHRoaXMgZHJhZnQgaXMgc3RydWN0dXJlZCBhcyBmb2xsb3dzLiAg QWZ0ZXIgdGhlCiAgIHRlcm1pbm9sb2d5IGluIFNlY3Rpb24gMiwgd2UgZ2l2ZSBhbiBvdmVydmll dyBvZiB0aGUgTEMtUENOIGluCiAgIFNlY3Rpb24gMy4gIEluIFNlY3Rpb24gNCB3ZSBnaXZlIGEg ZGV0YWlsZWQgZGVzY3JpcHRpb24gb2YgdGhlIExDLQogICBQQ04uICBTZWN0aW9uIDUgZGlzY3Vz c2VzIHNlY3VyaXR5IGlzc3Vlcy4KCgoyLiAgVGVybWlub2xvZ3kKCiAgIFRoZSBrZXkgd29yZHMg Ik1VU1QiLCAiTVVTVCBOT1QiLCAiUkVRVUlSRUQiLCAiU0hBTEwiLCAiU0hBTEwgTk9UIiwKICAg IlNIT1VMRCIsICJTSE9VTEQgTk9UIiwgIlJFQ09NTUVOREVEIiwgIk1BWSIsIGFuZCAiT1BUSU9O QUwiIGluIHRoaXMKICAgZG9jdW1lbnQgYXJlIHRvIGJlIGludGVycHJldGVkIGFzIGRlc2NyaWJl ZCBpbiBSRkMgMjExOS4gIFRoZSB0ZXJtcwogICBzcGVjaWZpZWQgaW4gW0VhcmQwN10gYXJlIHVz ZWQuCgoKMy4gIExDLVBDTiBPdmVydmlldwoKICAgTG9hZCBDb250cm9sIFBDTiAoTEMtUENOKSBp cyBhY2hpZXZlZCBieSB0d28gYWN0aW9uczogQWRtaXNzaW9uCiAgIENvbnRyb2wgYmFzZWQgb24g cHJvYmluZyBhbmQvb3IgRmxvdyBUZXJtaW5hdGlvbi4gIFRoZSBMQy1QQ04gY2FuIGJlCiAgIGFw cGxpZWQgd2l0aGluIGVpdGhlciBhIHNpbmdsZSBQQ04gZG9tYWluLCBzZWUgRmlndXJlIDEsIG9y IG11bHRpcGxlCiAgIG5laWdoYm9yaW5nIFBDTiBkb21haW5zLCB3aGVuIGEgdHJ1c3QgcmVsYXRp b25zaGlwIGV4aXN0cyBiZXR3ZWVuCiAgIHRoZXNlIG11bHRpcGxlIFBDTiBkb21haW5zLgoKICAg ICBQQ04tSW5ncmVzcy1Ob2RlICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIFBDTi1F Z3Jlc3MtTm9kZQogICAgICAgICAgICAgICAgICAgICAgICAoUENOLUludGVyaW9yLU5vZGVzOyBJ LU5vZGVzKQogICAgICAgICAgICAgICAgICAgICAgICAgICAgfCAgICAgICAgICB8ICAgICAgICAg ICAgfAogICAgICAgICAgICAgICAgICAgICAgICAgICAgfCAgICAgICAgICB8ICAgICAgICAgICAg fAogICAgICAgICAgICAgICAgICAgICAgICAgICAgViAgICAgICAgICBWICAgICAgICAgICAgVgog ICAgICstLS0tLS0tKyAgIERhdGEgKy0tLS0tLSsgICAgICArLS0tLS0tKyAgICAgICArLS0tLS0t KyAgICAgKy0tLS0tLSsKICAgICB8LS0tLS0tLXwtLS0tLS0tLXwtLS0tLS18LS0tLS0tfC0tLS0t LXwtLS0tLS0tfC0tLS0tLXwtLS0tPnwtLS0tLS18CiAgICAgfCAgICAgICB8ICAgRmxvdyB8ICAg ICAgfCAgICAgIHwgICAgICB8ICAgICAgIHwgICAgICB8ICAgICB8ICAgICAgfAogICAgIHxJbmdy ZXNzfCAgICAgICAgfEktTm9kZXwgICAgICB8SS1Ob2RlfCAgICAgICB8SS1Ob2RlfCAgICAgfEVn cmVzc3wKICAgICB8ICAgICAgIHwgICAgICAgIHwgICAgICB8ICAgICAgfCAgICAgIHwgICAgICAg fCAgICAgIHwgICAgIHwgICAgICB8CiAgICAgKy0tLS0tLS0rICAgICAgICArLS0tLS0tKyAgICAg ICstLS0tLS0rICAgICAgICstLS0tLS0rICAgICArLS0tLS0tKwogICAgICAgICAgICAgID09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0+CiAgICAgICAgICAg ICAgPD09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgU2lnbmFsaW5nCgogICAgICAgICAgICAg ICAgICAgICAgIEZpZ3VyZSAxOiBBY3RvcnMgaW4gdGhlIExDLVBDTgoKMy4xLiAgQWRtaXNzaW9u IGNvbnRyb2wgYmFzZWQgb24gcHJvYmluZwoKICAgVGhlIGFkbWlzc2lvbiBjb250cm9sIGZ1bmN0 aW9uIGJhc2VkIG9uIHByb2JpbmcgY2FuIGJlIHVzZWQgdG8KICAgaW1wbGVtZW50IGEgc2ltcGxl IG1lYXN1cmVtZW50LWJhc2VkIGFkbWlzc2lvbiBjb250cm9sIHdpdGhpbiBhIFBDTgogICBkb21h aW4uICBJbiB0aGUgUENOLWludGVyaW9yLW5vZGVzIHRocmVzaG9sZHMgYXJlIHNldCBmb3IgdGhl IHRyYWZmaWMKICAgYmVsb25naW5nIHRvIGRpZmZlcmVudCBQSEJzIGluIHRoZSBtZWFzdXJlbWVu dCBiYXNlZCBhZG1pc3Npb24KICAgY29udHJvbCBmdW5jdGlvbi4gIEluIHRoaXMgc2NlbmFyaW8g YW4gSVAgcGFja2V0IGlzIHVzZWQgYXMgYSBwcm9iZQogICBwYWNrZXQsIG1lYW5pbmcgdGhhdCB0 aGUgRFNDUCBmaWVsZCBpbiB0aGUgaGVhZGVyIG9mIHRoZSBJUCBwYWNrZXQgaXMKICAgcmUtbWFy a2VkIHdoZW4gdGhlIG1lYXN1cmVkIFBIQiB0aHJvdWdocHV0IHJhdGUgZXhjZWVkcyBhIHByZWRl ZmluZWQKCgoKV2VzdGJlcmcsIGV0IGFsLiAgICAgICAgICBFeHBpcmVzIE1heSAxNywgMjAwOCAg ICAgICAgICAgICAgICAgIFtQYWdlIDVdCgwKSW50ZXJuZXQtRHJhZnQgICAgICAgICAgICAgICAg ICAgTEMtUENOICAgICAgICAgICAgICAgICAgICBOb3ZlbWJlciAyMDA3CgoKICAgY29uZ2VzdGlv biB0aHJlc2hvbGQsIGkuZSwgUENOX2xvd2VyX3JhdGUuSW4gYWRkaXRpb24gdG8gdGhpcyB0aGUK ICAgUENOX2luZ3Jlc3Nfbm9kZSBoYXMgdG8gc2V0IHRoZSBSb3V0ZXIgQWxlcnQgSVAgb3B0aW9u IG9uIHRoZSBwcm9iZQogICBwYWNrZXQuICBJbiB0aGlzIHdheSBhbGwgdGhlIFBDTl9pbnRlcmlv cl9ub2RlIHdpbGwgaGF2ZSB0byBvYnNlcnZlCiAgIHRoZSByZWNlaXZlZCBwcm9iZSBwYWNrZXRz LiAgVGh1cyBpZiBhIFBDTl9pbnRlcmlvcl9ub2RlIHJlY2VpdmVzIGEKICAgcHJvYmUgcGFja2V0 IHRoZW4sIGR1ZSB0byB0aGUgUm91dGVyIEFsZXJ0IG9wdGlvbiBpdCBoYXMgdG8gaGFuZGxlIGl0 CiAgIGRpZmZlcmVudGx5IHRoZW4gdGhlIHVzZXIgcGFja2V0cy4KCiAgIFRoZSBQQ05faW50ZXJp b3Jfbm9kZSBoYXMgdG8gUENOX21hcmsgdGhlIHByb2JlIHBhY2tldCBpZiBpdCBpcwogICBvcGVy YXRpbmcgaW4gQWRtaXNzaW9uIENvbnRyb2wgc3RhdGUgKG9yIEZsb3cgVGVybWluYXRpb24gc3Rh dGUpLgogICBPdGhlcndpc2UgdGhlIHByb2JlIHBhY2tldCByZW1haW5zIHVubWFya2VkLgoKICAg SW4gdGhpcyB3YXkgdGhlIGRhdGEgcGFja2V0cyBhcmUgbWFya2VkIHRvIG5vdGlmeSB0aGUgUENO LWVncmVzcy1ub2RlCiAgIHRoYXQgYSBjb25nZXN0aW9uIGhhcyBvY2N1cnJlZCBvbiBhIHBhcnRp Y3VsYXIgUENOLWluZ3Jlc3Mtbm9kZSB0bwogICBQQ04tZWdyZXNzLW5vZGUgcGF0aC4KCiAgIElm IG5vIHByb2JpbmcgaXMgdXNlZCwgdGhlIHJlcXVlc3QgZm9yIGFkbWlzc2lvbiBjYW4gYmUgYWNj b21wbGlzaGVkCiAgIGJ5IHVzaW5nIGFuIGV4dGVybmFsIHRvIFBDTiwgc2lnbmFsaW5nIHByb3Rv Y29sLiAgSW4gdGhpcyBjYXNlIHdoZW4KICAgdGhlIHJlcXVlc3QsIGNhcnJpZWQgYnkgdGhlIGV4 dGVybmFsIHRvIFBDTiBzaWduYWxpbmcgcHJvdG9jb2wKICAgYXJyaXZlcyBhdCBhIFBDTl9lZ3Jl c3Nfbm9kZSB0aGF0IG9wZXJhdGVzIGluIGFkbWlzc2lvbiBjb250cm9sIHN0YXRlCiAgIHRoZW4g dGhlIHJlcXVlc3QgaXMgcmVqZWN0ZWQuICBJZiBpdCBvcGVyYXRlcyBpbiBOb3JtYWwgc3RhdGUg aXQgaXMKICAgYWNjZXB0ZWQuCgogICBJZiBwcm9iaW5nIGlzIHVzZWQsIHRoZSByZXF1ZXN0IGZv ciBhZG1pc3Npb24gaXMgYWNjb21wbGlzaGVkIGJ5CiAgIHVzaW5nIGEgcHJvYmUgcGFja2V0LiAg SW4gdGhpcyBjYXNlIHdoZW4gdGhlIHByb2JlIGFycml2ZXMgYXQgYQogICBQQ05fZWdyZXNzX25v ZGUgYW5kIGl0IGlzIFBDTl9tYXJraW5nIGVuY29kZWQgaXMgcmVqZWN0ZWQuICBPdGhlcndpc2UK ICAgaXMgYWNjZXB0ZWQuCgogICBOb3RlIHRoYXQgYnkgdXNpbmcgcHJvYmluZywgdGhlIEVDTVAg KEVxdWFsIENvc3QgTXVsdGkgUGF0aCkgcHJvYmxlbQogICB0aGF0IGlzIGFzc29jaWF0ZWQgd2l0 aCB0aGUgYWRtaXNzaW9uIGNvbnRyb2wgZmVhdHVyZSBjYW4gYmUsIHRvIGEKICAgY2VydGFpbiBk ZWdyZWUsIHNvbHZlZCBieSBiZWluZyBhYmxlIHRvIGlkZW50aWZ5IHdoaWNoIGZsb3dzIGFyZQog ICBwYXNzaW5nIHRocm91Z2ggdGhlIGNvbmdlc3RlZCBub2RlLiAgTm90ZSB0aGF0IHRoZSBFQ01Q IHByb2JsZW0gaXMKICAgcmVsYXRlZCB0byB0aGUgZmFjdCB0aGF0IGZsb3dzIHRoYXQgYXJlIG5v dCBwYXNzaW5nIHRocm91Z2ggYQogICBjb25nZXN0ZWQgUENOLWludGVyaW9yLW5vZGUgY2FuIGJl bG9uZyB0byBhbiBhZ2dyZWdhdGUgdGhhdCBkZXRlY3RzIGEKICAgY29uZ2VzdGlvbi4KCiAgIEFu eSBtZWFzdXJlcyB0aGF0IGFyZSB0YWtlbiBvbiBzdWNoIGZsb3dzIHdpbGwgbm90IHNvbHZlIHRo ZQogICBjb25nZXN0aW9uIHByb2JsZW0sIHNpbmNlIHN1Y2ggZmxvd3MgYXJlIG5vdCBjb250cmli dXRpbmcgYW5kIGNhdXNpbmcKICAgdGhlIGNvbmdlc3Rpb24gaW4gdGhlIFBDTi1pbnRlcmlvci1u b2RlLgoKMy4yLiAgRmxvdyBUZXJtaW5hdGlvbgoKICAgVGhlIEZsb3cgVGVybWluYXRpb24gZnVu Y3Rpb24gaXMgYWJsZSB0byB0ZXJtaW5hdGUgZmxvd3MgaW4gY2FzZSBvZgogICBleGNlcHRpb25h bCBldmVudHMsIHN1Y2ggYXMgc2V2ZXJlIGNvbmdlc3Rpb24gYWZ0ZXIgcmUtcm91dGluZy4gIFRo ZQogICBleGNlcHRpb25hbCBldmVudCwgb3Igc2V2ZXJlIGNvbmdlc3Rpb24gY2FuIGJlIGRldGVj dGVkIHVzaW5nIGEgRFNDUAogICByZW1hcmtpbmcgYXBwcm9hY2ggd2hlcmUgdGhlIFBDTl9tYXJr aW5nIGlzIHByb3BvcnRpb25hbCB0byB0aGUKICAgZXhjZXNzIHJhdGUuICBJbiBwYXJ0aWN1bGFy LCB0aGUgUENOLWludGVyaW9yLW5vZGVzIHBhY2tldHMgdXNpbmcgdGhlCiAgIFBDTl9tYXJraW5n IERTQ1AsIHdoZW5ldmVyIHRoZSBtZWFzdXJlZCBQSEIgdGhyb3VnaHB1dCByYXRlIGV4Y2VlZHMg YQogICBwcmUtY29uZmlndXJlZCB0aHJvdWdocHV0IHRocmVzaG9sZCBkZW5vdGVkIGFzIFBDTl91 cHBlcl9yYXRlLgoKCgpXZXN0YmVyZywgZXQgYWwuICAgICAgICAgIEV4cGlyZXMgTWF5IDE3LCAy MDA4ICAgICAgICAgICAgICAgICAgW1BhZ2UgNl0KDApJbnRlcm5ldC1EcmFmdCAgICAgICAgICAg ICAgICAgICBMQy1QQ04gICAgICAgICAgICAgICAgICAgIE5vdmVtYmVyIDIwMDcKCgogICBUaGUg UENOLWVncmVzcy1ub2RlcyBjYW4gdXNlIHRoZSByZW1hcmtlZCBQQ05fbWFya2luZyBEU0NQIHBh Y2tldHMgdG8KICAgY2FsY3VsYXRlIHRoZSBmcmFjdGlvbiBvZiB0aHJvdWdocHV0IG9yIGJhbmR3 aWR0aCB0aGF0IGRvZXMgZXhjZWVkCiAgIFBDTl91cHBlcl9yYXRlX2VncmVzcy4gIFRoZSBQQ05f QWZmZWN0ZWRfbWFya2luZyBEU0NQIGlzIHVzZWQgdG8gbWFyawogICBhbGwgcGFja2V0cyB0aGF0 IGFyZSBwYXNzaW5nIHRocm91Z2ggYW4gUENOLWludGVyaW9yLW5vZGUgdGhhdCBpcwogICBlaXRo ZXIgaW4gRmxvdyBUZXJtaW5hdGlvbiBzdGF0ZSBhbmQgYXJlIG5vdCBQQ05fbWFya2luZyBEU0NQ CiAgIGVuY29kZWQuICBJbiB0aGlzIHdheSBhbiBFQ01QIHNvbHV0aW9uIGNhbiBiZSBwcm92aWRl ZCBmb3IgdGhlIEZsb3cKICAgVGVybWluYXRpb24gc3RhdGUuICBUaGUgUENOLWVncmVzcy1ub2Rl IGNhbiB0aGVuLCBpbiBjb21iaW5hdGlvbiB3aXRoCiAgIHRoZSBQQ04taW5ncmVzcy1ub2RlLCBz ZW5kZXIgb2YgdGhlIHRyYWZmaWMgYW5kIHRoZSBzdXBwb3J0IG9mIHRoZQogICBQQ04gZG9tYWlu KHMpLCByZWR1Y2UgdGhlIGdlbmVyYXRlZCByYXRlLCBieSB0ZXJtaW5hdGluZyBvbmdvaW5nCiAg IGZsb3dzLCB1bnRpbCB0aGUgZXhjZXNzIHJhdGUgZHJvcHMgYmVsb3cgUENOX3VwcGVyX3JhdGVf ZWdyZXNzLgoKMy4zLiAgQ29tbW9uIFBDTiBub2RlIGNvbmZpZ3VyYXRpb25zCgogICBUaGUgUENO LWludGVyaW9yLW5vZGVzLCBzZWUgRmlndXJlIDEsIHdoaWNoIGFyZSBzdXBwb3J0aW5nIHRoZSBM Qy0KICAgUENOLCBtdXN0IHBlcmZvcm0gdGhlIGZvbGxvd2luZyBmdW5jdGlvbmFsaXRpZXM6Cgog ICAoMSkgTWV0ZXIgKyAoMikgTWFya2luZyBBY3Rpb246IHRoZSBQQ04taW50ZXJpb3Itbm9kZXMg bXVzdCBiZQogICBjb25maWd1cmVkIHdpdGggYSBtZXRlciBhbmQgbWFya2luZyBmdW5jdGlvbiB0 aGF0IG1lYXN1cmVzIGFuZAogICByZW1hcmtzIGJ5dGVzIHRoYXQgYXJlIG91dCBvZiBhIGNvbmZp Z3VyZWQgdHJhZmZpYyBwcm9maWxlIChlLmcuLAogICBiYW5kd2lkdGggdGhyZXNob2xkKSBmb3Ig YSBjb3JyZXNwb25kaW5nIFBIQiB0cmFmZmljIGNsYXNzLCB0bwogICBwcm92aWRlIGFuIGluZGlj YXRpb24gb2YgYSBwb3RlbnRpYWwgcmVzb3VyY2UgbGltaXRhdGlvbiB0byBhIFBDTi0KICAgZWdy ZXNzLW5vZGUuICBUaGUgdHJhZmZpYyBwcm9maWxlIGNhbiBiZSBzZXQgYWNjb3JkaW5nIHRvIGFu CiAgIGVuZ2luZWVyZWQgYmFuZHdpZHRoIGxpbWl0YXRpb24gYmFzZWQgb24gcHJlLWNvbmZpZ3Vy ZWQgdGhyZXNob2xkcyBvcgogICBiYXNlZCBvbiBhIGNhcGFjaXR5IGxpbWl0YXRpb24gb2Ygc3Bl Y2lmaWMgUEhCcy4gIEJ5IHVzaW5nIGFuCiAgIGFsZ29yaXRobSB0aGF0IGNhbGN1bGF0ZXMgdGhl IHJhdGUgb2YgYnl0ZXMgdGhhdCBhcmUgb3V0IG9mIHByb2ZpbGUsCiAgIHNheSBzaWduYWxlZF9y ZW1hcmtlZF9ieXRlczsgYSBzcGVjaWFsIG51bWJlciBvZiBieXRlcywgaS5lLiwKICAgc2lnbmFs ZWRfcmVtYXJrZWRfYnl0ZXMvTiwgYXJlIHJlbWFya2VkIHRvIGEgc2Vjb25kIERTQ1AsIGRlbm90 ZWQgaW4KICAgdGhpcyBleGFtcGxlIGFzIFBDTl9tYXJraW5nIERTQ1AsIHRoYXQgcmVjZWl2ZXMg dGhlIHNhbWUgUEhCIGFzIHRoZQogICBvcmlnaW5hbCBEU0NQICh3aGVyZSBOIGlzIGVxdWFsIG9y IGdyZWF0ZXIgdGhhbiAxKS4gIEFub3RoZXIgdHlwZSBvZgogICBlbmNvZGluZyB0aGF0IGlzIHVz ZWQsIGlzIHRoZSBQQ05fQWZmZWN0ZWRfbWFya2luZyBEU0NQLCB3aGljaCBpcwogICB1c2VkIHRv IG1hcmsgYWxsIHBhY2tldHMgdGhhdCBhcmUgcGFzc2luZyB0aHJvdWdoIGFuIFBDTi1pbnRlcmlv ci0KICAgbm9kZSBpbiBGbG93IFRlcm1pbmF0aW9uIHN0YXRlIGFuZCB0aGUgYXJyaXZpbmcgcGFj a2V0cyBhcmUgbm90CiAgIFBDTl9tYXJraW5nIERTQ1AgZW5jb2RlZC4KCiAgIFRoZSBQQ05fbWFy a2luZyBEU0NQIGFuZCBQQ05fQWZmZWN0ZWRfbWFya2luZyBEU0NQIGFyZSBkZWZpbmVkIHRvIGJl CiAgIHVzZWQgb25seSBsb2NhbGx5IHdpdGhpbiB0aGUgUENOIGRvbWFpbi4gICJOIiBpcyBhIHBy ZS1jb25maWd1cmVkCiAgIHBhcmFtZXRlciB1c2VkIHRvIGluZGljYXRlIHRoZSBwcm9wb3J0aW9u YWxpdHkgYmV0d2VlbiB0aGUgbWVhc3VyZWQKICAgb3V0IG9mIHByb2ZpbGUgYnl0ZXMgYW5kIHRo ZSByZW1hcmtlZCBieXRlcy4gIElmICJOIiBpcyB1c2VkIGluIHRoZQogICBhbGdvcml0aG0sIHRo ZW4gaXQgbXVzdCBoYXZlIHRoZSBzYW1lIHZhbHVlIGluIGFsbCBEaWZmc2VydiBub2RlcwogICB0 aGF0IHVzZSB0aGlzIG1lY2hhbmlzbS4gIEFzIHByZXZpb3VzbHkgbWVudGlvbmVkLCBOIGlzIGhp Z2hlciBvcgogICBlcXVhbCB0byAxIChOID49IDEpLgoKICAgKDMpIFBhY2tldCBDbGFzc2lmaWNh dGlvbiArICg0KSBTY2hlZHVsaW5nOiBUaGUgUENOLWludGVyaW9yLW5vZGUKICAgU0hPVUxEIGJl IGNvbmZpZ3VyZWQgdG8gY29uc2lkZXIgdGhhdCB0aGUgcGFja2V0cyBtYXJrZWQgZWl0aGVyIHdp dGgKICAgdGhlIG9yaWdpbmFsIERTQ1Agb3Igd2l0aCB0aGUgUENOX21hcmtpbmcgRFNDUCBvciBB ZmZlY3RlZF8gbWFya2luZwogICBEU0NQIFNIT1VMRCByZWNlaXZlIHRoZSBzYW1lIHBlciBob3Ag YmVoYXZpb3IgdHJlYXRtZW50LiAgSG93ZXZlciwKICAgcGFja2V0cyB0aGF0IGFyZSBtYXJrZWQg d2l0aCB0aGUgUENOX21hcmtpbmcgRFNDUCwgbWF5IGJlIGNsYXNzaWZpZWQKICAgdG8gZW50ZXIg YSBkaWZmZXJlbnQgYW5kIGxhcmdlciB2aXJ0dWFsIHF1ZXVlIHRoYW4gdGhlIHBhY2tldHMgbWFy a2VkCgoKCldlc3RiZXJnLCBldCBhbC4gICAgICAgICAgRXhwaXJlcyBNYXkgMTcsIDIwMDggICAg ICAgICAgICAgICAgICBbUGFnZSA3XQoMCkludGVybmV0LURyYWZ0ICAgICAgICAgICAgICAgICAg IExDLVBDTiAgICAgICAgICAgICAgICAgICAgTm92ZW1iZXIgMjAwNwoKCiAgIHdpdGggZWl0aGVy IHRoZSBvcmlnaW5hbCBEU0NQIG9yIFBDTl9BZmZlY3RlZF9tYXJraW5nIERTQ1AuICBUaGlzIGNh bgogICBlbnN1cmUgdGhhdCB0aGUgZHJvcHBpbmcgcHJvYmFiaWxpdHkgb2YgUENOX21hcmtpbmcg RFNDUCByZW1hcmtlZAogICBwYWNrZXRzIGlzIGxvd2VyIHRoYW4gdGhlIGRyb3BwaW5nIHByb2Jh YmlsaXR5IG9mIG9yaWdpbmFsIERTQ1AKICAgcmVtYXJrZWQgcGFja2V0cy4gIFRoaXMgY2xhc3Np ZmljYXRpb24gY2FuIGJlIGFjY29tcGxpc2hlZCBieSB1c2luZwogICB0aGUgcGFja2V0IGNsYXNz aWZpY2F0aW9uIGZ1bmN0aW9uLCB3aGlsZSB0aGUgd2F5IG9mIGhvdyB0aGUgcGFja2V0cwogICBh cmUgdHJlYXRlZCBpbiB0aGUgdmlydHVhbCBxdWV1ZXMgaXMgYWNjb21wbGlzaGVkIHVzaW5nIHRo ZQogICBzY2hlZHVsaW5nIGZ1bmN0aW9uLiAgTm90ZSB0aGF0IHRoZSBvcmlnaW5hbCBEU0NQIG1h cmtlZCBwYWNrZXRzIGFuZAogICB0aGVpciBhc3NvY2lhdGVkIFBDTl9tYXJraW5nIERTQ1AgcGFj a2V0cyBnZXQgdGhlIHNhbWUgZm9yd2FyZGluZwogICBiZWhhdmlvci4gIFRoZSBtYWluIGRpZmZl cmVuY2UgaXMgcmVsYXRlZCB0byB0aGUgZmFjdCB0aGF0IHRoZQogICBQQ05fbWFya2luZyBEU0NQ IHBhY2tldHMgZ2V0IGEgbG93ZXIgZHJvcHBpbmcgcHJvYmFiaWxpdHkgY29tcGFyZWQgdG8KICAg dGhlIG9yaWdpbmFsX0RTQ1AgcGFja2V0cy4gIFRoaXMgaXMgYmVjYXVzZSB0aGUgbWFya2luZyBp bmZvcm1hdGlvbgogICBjYXJyaWVkIGJ5IHRoZSBQQ05fbWFya2luZyBEU0NQIHBhY2tldHMgaGFz IGEgaGlnaGVyIHNpZ25pZmljYW5jZSBmb3IKICAgdGhlIG9wZXJhdGlvbiBvZiB0aGUgcmVzb3Vy Y2UgdW5hdmFpbGFiaWxpdHkgYWxnb3JpdGhtIGNvbXBhcmVkIHRvCiAgIHRoZSBtYXJraW5nIGlu Zm9ybWF0aW9uIGNhcnJpZWQgYnkgdGhlIG9yaWdpbmFsX0RTQ1AgcGFja2V0cy4KCiAgIFRoZSB0 d28gdmlydHVhbCBxdWV1ZXMsIG9uZSBmb3IgdGhlIG9yaWdpbmFsX0RTQ1AgYW5kIGFub3RoZXIg b25lIGZvcgogICBQQ05fbWFya2luZyBEU0NQIG1hcmtlZCBwYWNrZXRzIGNhbiwgZm9yIGV4YW1w bGUsIGJlIGltcGxlbWVudGVkIGJ5CiAgIHVzaW5nIG9uZSBEcm9wIFRhaWwgcGh5c2ljYWwgcXVl dWUgYW5kIGJ5IG1haW50YWluaW5nIHF1ZXVpbmcKICAgaW5mb3JtYXRpb24gYW5kIGFsc28gb25l IHF1ZXVpbmcgdGhyZXNob2xkIGZvciBlYWNoIG9mIHRoZSB2aXJ0dWFsCiAgIHF1ZXVlcy4gIFRo ZSBwaHlzaWNhbCBxdWV1ZSB1c2VzIHRoZSBzYW1lIHNjaGVkdWxpbmcgYWxnb3JpdGhtLCBidXQK ICAgdGhlIGxlbmd0aCBvZiBlYWNoIG9mIHRoZSB2aXJ0dWFsIHF1ZXVlIGRlZmluZXMgdGhlIHBh Y2tldCBkcm9wcGluZwogICBwcm9iYWJpbGl0eSBvZiBhIHZpcnR1YWwgcXVldWUuICBUaGUgY2xh c3NpZmljYXRpb24gb2YgcGFja2V0cyBTSE9VTEQKICAgYmUgYmFzZWQgb24gZWl0aGVyIHRoZSBE U0NQIG9yIG9uIGEgY29tYmluYXRpb24gb2YgSVAgaGVhZGVyIGZpZWxkcwogICBpbmNsdWRpbmcg dGhlIERTQ1AuCgogICBXaGVuIHRoZSBMQy1QQ04gaXMgYXBwbGllZCBpbiBtdWx0aXBsZSBuZWln aGJvcmluZyBQQ04gZG9tYWlucyB3aGVyZQogICBhIHRydXN0IHJlbGF0aW9uc2hpcCBleGlzdHMg YmV0d2VlbiB0aGVzZSBtdWx0aXBsZSBQQ04gZG9tYWlucyBhbmQgYQogICBwYWNrZXQgaXMgcmVj ZWl2ZWQgYnkgdGhlIGVkZ2Ugcm91dGVyIG9mIGFub3RoZXIgdHJ1c3RlZCBkb21haW4gKG5ldwog ICBQQ04gZG9tYWluLCB0aGF0IG1pZ2h0IGJlIG1hbmFnZWQgYnkgYW5vdGhlciBvcGVyYXRvciks IHJlbWFya2luZyBvZgogICB0aGUgb3JpZ2luYWwgRFNDUCwgUENOX21hcmtpbmcgRFNDUCBhbmQg UENOX0FmZmVjdGVkX21hcmtpbmcgRFNDUCB0bwogICBvdGhlciBEU0NQcywgc2F5IG9yaWdpbmFs IG5ld19EU0NQLCBQQ05fbWFya2luZyBuZXdfRFNDUCBhbmQKICAgUENOX0FmZmVjdGVkX21hcmtp bmcgbmV3X0RTQ1AgbWlnaHQgYmUgbmVjZXNzYXJ5LiAgVGhpcyBpcyBiZWNhdXNlCiAgIHRoZSBu ZWlnaGJvciBQQ04gb3BlcmF0b3IgbWF5IHVzZSBkaWZmZXJlbnQgRGlmZnNlcnYgTWFwcGluZyBz Y2hlbWVzLgoKICAgUENOX3VwcGVyX3JhdGUgaXMgY29uZmlndXJlZCBpbiBhbGwgUENOLWludGVy aW9yLW5vZGVzIGFuZCBpdCBjYW4gYmUKICAgY2FsY3VsYXRlZCBpbiB0aGUgZm9sbG93aW5nIHdh eToKCiAgIFBDTl91cHBlcl9yYXRlID0gTWF4aW11bSBQSEIgY2FwYWNpdHkgLSBUZXJtaW5hdGlv bl9vZmZzZXRfcmF0ZQoKICAgTWF4aW11bSBQSEIgY2FwYWNpdHkgaXMgdGhlIG1heGltdW0gbGlu ayBjYXBhY2l0eSB0aGF0IGlzIHN1cHBvcnRlZAogICBieSBhIFBDTiBub2RlLgoKICAgVGhlIFRl cm1pbmF0aW9uX29mZnNldF9yYXRlIGlzIGFuIGFic29sdXRlIHJhdGUgdmFsdWUgdGhhdCBzaG91 bGQgYmUKICAgc2V0IGVxdWFsIGludG8gYWxsIFBDTl9pbnRlcmlvcl9ub2Rlcy4gIFRoZSBUZXJt aW5hdGlvbl9vZmZzZXRfcmF0ZQogICBjYW4gYWxzbyBiZSBlcXVhbCB0byAwLgoKICAgTm90ZSB0 aGF0IHRoaXMgdmFsdWUgaXMgdXNlZCBieSBQQ05faW50ZXJpb3Jfbm9kZXMgdG8gY2FsY3VsYXRl IHRoZWlyCiAgIFBDTl91cHBlcl9yYXRlIGFuZCBpcyBhbHNvIHVzZWQgZHVyaW5nIHRoZSBzaXR1 YXRpb24gdGhhdCBhCgoKCldlc3RiZXJnLCBldCBhbC4gICAgICAgICAgRXhwaXJlcyBNYXkgMTcs IDIwMDggICAgICAgICAgICAgICAgICBbUGFnZSA4XQoMCkludGVybmV0LURyYWZ0ICAgICAgICAg ICAgICAgICAgIExDLVBDTiAgICAgICAgICAgICAgICAgICAgTm92ZW1iZXIgMjAwNwoKCiAgIFBD Tl9pbnRlcmlvcl9ub2RlIGlzIGluIGZsb3cgdGVybWluYXRpb24gc3RhdGUgYW5kIGl0IHJlY2Vp dmVzCiAgIFBDTl9tYXJrZWQgcGFja2V0cy4gIFRoaXMgc2l0dWF0aW9uIG9jY3VycyB3aGVuIG1v cmUgdGhhbiBvbmUgUENOLQogICBpbnRlcmlvci1ub2RlcyBsb2NhdGVkIG9uIHNhbWUgY29tbXVu aWNhdGlvbiBwYXRoLCBhcmUgc2ltdWx0YW5lb3VzbHkKICAgb3BlcmF0aW5nIGluIHRoZSBhZG1p c3Npb24gY29udHJvbCBzdGF0ZSBvciBmbG93IHRlcm1pbmF0aW9uIHN0YXRlLgogICBUaGUgVGVy bWluYXRpb25fb2Zmc2V0X3JhdGUgaXMgbmVlZGVkIGR1ZSB0byB0aGUgZm9sbG93aW5nIGZhY3Qu CiAgIENvbnNpZGVyIHRoZSBmYWN0IHRoYXQgd2hlbiB0aGUgbWVhc3VyZWQgUEhCIHJhdGUgZXhj ZWVkcyB0aGUKICAgIk1heGltdW0gUEhCIGNhcGFjaXR5IiB0aGVuIHRoZSBwYWNrZXRzIGJlbG9u Z2luZyB0byB0aGUgZ2l2ZW4gUEhCCiAgIHdpbGwgYmUgZWl0aGVyIGRyb3BwZWQgb3Igc2V0IHRv IGFub3RoZXIgUEhCLiAgSW4gbXVsdGlwbGUgc2V2ZXJlCiAgIGNvbmdlc3Rpb24gc2l0dWF0aW9u cyBzb2x2aW5nIHRoZSBzZXZlcmUgY29uZ2VzdGlvbiBvbiBhIHNldmVyZQogICBjb25nZXN0aW9u IFBDTl9JbnRlcmlvcl9ub2RlLCBmdXJ0aGVyIGF3YXkgdGhhbiB0aGUgUENOX2VncmVzc19ub2Rl LAogICBzYXkgc2V2ZXJlX2Nvbmdlc3Rpb25fcG9pbnRfMSwgaXQgY291bGQgY2F1c2UgdGhlIHNp dHVhdGlvbiB0aGF0IHRoZQogICBzZXZlcmUgY29uZ2VzdGlvbiBvbiBhIFBDTl9JbnRlcmlvcl9u b2RlIGxvY2F0ZWQgb24gdGhlIHNhbWUgcGF0aCBhbmQKICAgY2xvc2VyIHRvIHRoZSBQQ05fZWdy ZXNzX25vZGUsIHNheSBzZXZlcmVfY29uZ2VzdGlvbl9wb2ludF8yLCB3aWxsIGJlCiAgIHNvbHZl ZCB3aXRob3V0IG1hcmtpbmcgdGhlIGV4Y2VzcyByYXRlIG1lYXN1cmVkIGF0CiAgIHNldmVyZV9j b25nZXN0aW9uX3BvaW50XzIuICBUaGlzIGlzIGhvd2V2ZXIgdHJ1ZSBvbmx5IGlmIHRoZSBtZWFz dXJlZAogICBQSEIgcmF0ZSBvbiBzZXZlcmVfY29uZ2VzdGlvbl9wb2ludF8xIGRvZXMgbm90IGV4 Y2VlZCB0aGUgIk1heGltdW0KICAgUEhCIGNhcGFjaXR5Ii4gIFRoaXMgaXMgZHVlIHRvIHRoZSBm YWN0IHRoYXQgYmVmb3JlIHRoZQogICBzZXZlcmVfY29uZ2VzdGlvbl9wb2ludF8xIGdvZXMgaW50 byBmbG93IHRlcm1pbmF0aW9uIGl0IGdlbmVyYXRlcyBhCiAgIG1lYXN1cmVkIFBIQiByYXRlIHRo YXQgaXQgZG9lcyBub3QgZXhjZWVkIHRoZSB2YWx1ZSBlcXVhbCB0bwogICAoIk1heGltdW0gUEhC IGNhcGFjaXR5Ii0gVGVybWluYXRpb25fb2Zmc2V0X3JhdGUpIGFuZCBpbiBmbG93CiAgIHRlcm1p bmF0aW9uIHN0YXRlIGl0IGdlbmVyYXRlcyBhIG1lYXN1cmVkIFBIQiByYXRlIG5vdCBoaWdoZXIg dGhhbgogICAiTWF4aW11bSBQSEIgY2FwYWNpdHkiLiAgVGh1cyBpZiB0aGUgZXhjZXNzIHJhdGUg b24KICAgc2V2ZXJlX2Nvbmdlc3Rpb25fcG9pbnRfMSBpcyBoaWdoZXIgdGhhbiAiTWF4aW11bSBQ SEIgY2FwYWNpdHkiIHRoZW4KICAgdGhpcyBpdCBpcyBub3Qgc2VlbiBieSBzZXZlcmVfY29uZ2Vz dGlvbl9wb2ludF8yIGJ1dCwgZHVlIHRvIHRoZQogICBwcmluY2lwbGUgb2YgbWFya2luZywgaXQg d2lsbCBiZSBzZWVuIGJ5IHRoZSBQQ05fZWdyZXNzX25vZGVzLgoKICAgVGhlcmVmb3JlLCB0aGUg c2V2ZXJlX2Nvbmdlc3Rpb25fcG9pbnRfMiBoYXMgdG8gY29uc2lkZXIgdGhlCiAgIGluY29taW5n X1BDTl9tYXJrZWRfcmF0ZSBmcm9tIHNldmVyZV9jb25nZXN0aW9uX3BvaW50XzEgaW4gaXRzCiAg IG1hcmtpbmcgYWxnb3JpdGhtIG9ubHkgZm9yIG1lYXN1cmVkIFBIQiByYXRlcyBoaWdoZXIgdGhh biB0aGUKICAgUENOX3VwcGVyX3JhdGUgKGFzc29jaWF0ZWQgd2l0aCBzZXZlcmVfY29uZ2VzdGlv bl9wb2ludF8xKSBhbmQgbG93ZXIKICAgb3IgZXF1YWwgdG8gdGhlIFBDTl91cHBlcl9yYXRlICsg VGVybWluYXRpb25fb2Zmc2V0X3JhdGUuICBUaGUKICAgc2V2ZXJlX2Nvbmdlc3Rpb25fcG9pbnRf MiBjYW4gY29tcHV0ZSB0aGUgVGVybWluYXRpb25fb2Zmc2V0X3JhdGUKICAgdXNlZCBieSB0aGUg cHJldmlvdXMgc2V2ZXJlIGNvbmdlc3Rpb24gcG9pbnQgYnkgdXNpbmcgYSB2YXJpYWJsZSB0aGF0 CiAgIGlzIHRoZSBzYW1lIGluIHRoZSB3aG9sZSBQQ04gZG9tYWluLgoKICAgUENOX2xvd2VyX3Jh dGUgaXMgY29uZmlndXJlZCBpbiBhbGwgUENOLWludGVyaW9yLW5vZGVzIGFuZCBpcwogICBjYWxj dWxhdGVkIGluIHRoZSBmb2xsb3dpbmcgd2F5OgoKICAgUENOX2xvd2VyX3JhdGUgPSBQQ05fdXBw ZXJfcmF0ZSAtIEFkbWlzc2lvbl9vZmZzZXRfcmF0ZQoKICAgVGhlIEFkbWlzc2lvbl9vZmZzZXRf cmF0ZSBpcyBhbiBhYnNvbHV0ZSByYXRlIHZhbHVlIGFuZCBpdCBpcyBlcXVhbAogICBpbiBhbGwg UENOX2ludGVyaW9yX25vZGVzIGFuZCBQQ05fZWdyZXNzX25vZGVzLgoKICAgVGhlIEFkbWlzc2lv bl9vZmZzZXRfcmF0ZSBhbmQgVGVybWluYXRpb25fb2Zmc2V0X3JhdGUgYXJlIHJlcXVpcmVkIGlu CiAgIG9yZGVyIHRvIHByb3ZpZGUgYSBzb2x1dGlvbiBmb3IgdGhlIHNpdHVhdGlvbiB0aGF0IG1v cmUgdGhhbiBvbmUgUENOLQogICBpbnRlcmlvci1ub2RlcyBsb2NhdGVkIG9uIHNhbWUgY29tbXVu aWNhdGlvbiBwYXRoLCBhcmUgc2ltdWx0YW5lb3VzbHkKICAgb3BlcmF0aW5nIGluIHRoZSBBZG1p c3Npb24gQ29udHJvbCBvciBGbG93IFRlcm1pbmF0aW9uIHN0YXRlLAogICByZXNwZWN0aXZlbGx5 LgoKCgpXZXN0YmVyZywgZXQgYWwuICAgICAgICAgIEV4cGlyZXMgTWF5IDE3LCAyMDA4ICAgICAg ICAgICAgICAgICAgW1BhZ2UgOV0KDApJbnRlcm5ldC1EcmFmdCAgICAgICAgICAgICAgICAgICBM Qy1QQ04gICAgICAgICAgICAgICAgICAgIE5vdmVtYmVyIDIwMDcKCgogICBUaGUgQWRtaXNzaW9u X29mZnNldF9yYXRlIGFuZCBUZXJtaW5hdGlvbl9vZmZzZXRfcmF0ZSBhcmUgcmVxdWlyZWQgaW4K ICAgb3JkZXIgdG8gcHJvdmlkZSBhIHNvbHV0aW9uIGZvciB0aGUgc2l0dWF0aW9uIHRoYXQgbW9y ZSB0aGFuIG9uZSBQQ04tCiAgIGludGVyaW9yLW5vZGVzIGxvY2F0ZWQgb24gc2FtZSBjb21tdW5p Y2F0aW9uIHBhdGgsIGFyZSBzaW11bHRhbmVvdXNseQogICBvcGVyYXRpbmcgaW4gdGhlIGFkbWlz c2lvbiBjb250cm9sIHN0YXRlIG9yIGZsb3cgdGVybWluYXRpb24gc3RhdGUsCiAgIHJlc3BlY3Rp dmVseS4KCiAgIEl0IGlzIGhvd2V2ZXIsIGNvbnNpZGVyZWQgdGhhdCBTTEEgYWdyZWVtZW50cyBl eGlzdCBiZXR3ZWVuIHRoZQogICBvcGVyYXRvcihzKSBvZiB0aGVzZSBQQ04gZG9tYWlucywgdGh1 cyBhbHNvIHRoZSByZW1hcmtpbmcgcnVsZXMKICAgZm9sbG93ZWQgaW4gZWFjaCBQQ04gZG9tYWlu IGFyZSBrbm93bi4gIE5vdGUgdGhhdCB0aGUgUENOIG5vZGVzIHVzZWQKICAgaW4gdGhlIG5laWdi b3VyaW5nIFBDTiBkb21haW5zIHNob3VsZCB1c2UgdGhlIHNhbWUgY2xhc3NpZmljYXRpb24sCiAg IG1ldGVyICYgbWFya2luZyBhY3Rpb25zIGFzIGRlc2NyaWJlZCBhYm92ZS4KCjMuNC4gIENvbmZp Z3VyYXRpb24gb2YgZWRnZSBub2RlcwoKICAgVGhlIGVkZ2VzIG11c3QgbWFpbnRhaW5zIGFnZ3Jl Z2F0ZWQgc3RhdGVzIHRoYXQgZW5jb21wYXNzIHNldmVyYWwKICAgZmxvd3MvY2FsbHMuICBUaGUg c2l6ZSBvZiB0aGUgYWdncmVnYXRlcyBzaG91bGQgYmUgbGFyZ2UgZW5vdWdoIHRvCiAgIGVuc3Vy ZSB0aGF0IG5ldyBmbG93cy9jYWxscyBiZWxvbmcgdG8gYWdncmVnYXRlcyB3aGVyZSBvbmdvaW5n IGNhbGxzCiAgIHByb3ZpZGUgZmVlZGJhY2sgZm9yIGFkbWlzc2lvbiBjb250cm9sIGRlY2lzaW9u cy4gIEluIGFkZGl0aW9uIHRvCiAgIHRoaXMgdGhlIGVkZ2VzIG11c3QgbWFpbnRhaW4gcGVyIGZs b3cgc3RhdGVzLgoKICAgV2hlbiB0aGUgUENOLWVncmVzcy1ub2RlcywgcmVjZWl2ZSB0aGUgcmVt YXJrZWQgUENOX21hcmtpbmcgRFNDUAogICBwYWNrZXRzLCB0aGUgcmF0ZSBvZiB0aGUgcmVjZWl2 ZWQgUENOX21hcmtpbmcgRFNDUCBieXRlcywgcGVyIGVhY2gKICAgZmxvdyBhZ2dyZWdhdGUsIGlz IG1lYXN1cmVkLiAgTm90ZSB0aGF0IHRoZSBjYWxjdWxhdGVkIHJhdGUgaGFzIHRvIGJlCiAgIG11 bHRpcGxpZWQgd2l0aCB0aGUgcGFyYW1ldGVyICJOIiwgYWJvdmUsIGluIG9yZGVyIHRvIGNhbGN1 bGF0ZSB0aGUKICAgcmVhbCByYXRlIG9mIG92ZXJsb2FkLCBzYXkgc2lnbmFsZWRfb3ZlcmxvYWRf cmF0ZS4gIFRoaXMgcmF0ZSBjYW4gYmUKICAgdXNlZCB0byBwcm92aWRlIGhhbmRsaW5nIGRlY2lz aW9ucyBvbiB0aGUgQWRtaXNzaW9uIENvbnRyb2wgYW5kIEZsb3cKICAgVGVybWluYXRpb24gZnVu Y3Rpb25hbGl0eS4gIFR3byB0eXBlcyBvZiBoYW5kbGluZyBkZWNpc2lvbnMgY291bGQgYmUKICAg c3VwcG9ydGVkLgoKICAgRm9yIGFkbWlzc2lvbiBjb250cm9sLCB0aGUgUENOLWVncmVzcy1ub2Rl IGNhbiBtYWludGFpbiBhdCBsZWFzdCBvbmUKICAgdGhyZXNob2xkLCBzYXkgUENOX2xvd2VyX3Jh dGVfZWdyZXNzLiAgVGhlbiBpZiB0aGUgY2FsY3VsYXRlZCByYXRlIG9mCiAgIHJlbWFya2VkIFBD Tl9tYXJraW5nIERTQ1AgYnl0ZXMgaXMgaGlnaGVyIHRoYW4gUENOX2xvd2VyX3JhdGVfZWdyZXNz LAogICBpLmUuLCBzaWduYWxlZF9vdmVybG9hZF9yYXRlID4gUENOX2xvd2VyX3JhdGVfZWdyZXNz LCB0aGVuIHRoZSBQQ04tCiAgIGVncmVzcy1ub2RlIGNhbiB1c2UgdGhpcyBpbmZvcm1hdGlvbiB0 byBwcm92aWRlIHRoZSBiYXNpcyBvZiBjYWxsCiAgIGFkbWlzc2lvbiBkZWNpc2lvbnMgZm9yIG5l dyBmbG93cy4gIFRoZSBkZXRhaWxlZCBzcGVjaWZpY2F0aW9uIG9mCiAgIHRoaXMgYWxnb3JpdGht IGlzIGdpdmVuIGluIFNlY3Rpb24gNC4xLjQuCgogICBPbmUgd2F5IHRvIGNhbGN1bGF0ZSB0aGUg UENOX2xvd2VyX3JhdGVfZWdyZXNzIHRocmVzaG9sZCB0aGF0IGRlZmluZXMKICAgd2hlbiBhIFBD Tl9lZ3Jlc3Nfbm9kZSBnb2VzIGludG8gdGhlIGFkbWlzc2lvbiBjb250cm9sIHN0YXRlIHRoYXQg aXMKICAgdG8gbW9uaXRvciB3aGVuIHRoZSBQQ05fZWdyZXNzX25vZGUgcmVjZWl2ZXMgYSBQQ05f bWFya2VkIHBhY2tldC4KICAgVGhhdCB3aWxsIG1lYW4gdGhhdCBhdCBsZWFzdCBvbmUgaW50ZXJt ZWRpYXRlIFBDTl9pbnRlcmlvcl9ub2RlCiAgIHN0YXJ0ZWQgdG8gYmUgaW4gY29uZ2VzdGVkIHN0 YXRlIGFuZCB0aHVzIHRoZSBlZ3Jlc3Mgbm9kZSB0cmFuc2l0aW9uCiAgIGZyb20gTm9ybWFsIHN0 YXRlIHRvIGFkbWlzc2lvbiBjb250cm9sIHN0YXRlLiAgV2UgdXNlIGEgZnJhY3Rpb24gb2YKICAg dGhlIHJlY2VpdmVkIFBDTl9tYXJraW5nIGVuY29kZWQgcGFja2V0cyB0byBiZSByZWFsaXN0aWMu ICBUaGUgdmFsdWUKICAgb2YgUENOX2xvd2VyX3JhdGVfZWdyZXNzIGlzIGNhbGN1bGF0ZWQgYXMg Zm9sbG93czoKCiAgIFBDTl9sb3dlcl9yYXRlX2VncmVzcyA9IEEgKiBBZG1pc3Npb25fb2Zmc2V0 X3JhdGUsIHdoZXJlIDAgPCBBIDwgMQogICBUeXBpY2FsbHksIGZhY3RvciBBIHNob3VsZCBiZSBz ZXQgbG93IGFyb3VuZCAxJS4KCgoKV2VzdGJlcmcsIGV0IGFsLiAgICAgICAgICBFeHBpcmVzIE1h eSAxNywgMjAwOCAgICAgICAgICAgICAgICAgW1BhZ2UgMTBdCgwKSW50ZXJuZXQtRHJhZnQgICAg ICAgICAgICAgICAgICAgTEMtUENOICAgICAgICAgICAgICAgICAgICBOb3ZlbWJlciAyMDA3CgoK ICAgSWYgdGhlIFBDTiBkb21haW4gc3VwcG9ydHMgcHJvYmluZyB0aGVuIHRoZSBQQ04taW5ncmVz cy1ub2RlIGlzCiAgIGNvbmZpZ3VyZWQgc3VjaCB0aGF0IHdoZW4gaXQgcmVjZWl2ZXMgYSByZXF1 ZXN0IGZvciByZXNlcnZhdGlvbgogICBtZXNzYWdlLCBpdCBnZW5lcmF0ZXMgYSBwcm9iZSBwYWNr ZXQgdGhhdCBpcyBzZW50IHdpdGhpbiB0aGUgUENOCiAgIGRvbWFpbi4gIFRoZSBwcm9iZSBwYWNr ZXQgc2hvdWxkIHVzZSB0aGUgc2FtZSBmbG93IElEIGFuZCBEU0NQIHZhbHVlCiAgIGFzIHRoZSBv bmVzIHVzZWQgYnkgdGhlIGRhdGEgcGFja2V0cyBhc3NvY2lhdGVkIHdpdGggdGhlIHJlcXVlc3Qg Zm9yCiAgIHJlc2VydmF0aW9uIG1lc3NhZ2UuICBGdXJ0aGVybW9yZSwgdGhlIHByb2JlIHBhY2tl dCBNVVNUIGVuYWJsZSB0aGUKICAgUm91dGVyIEFsZXJ0IE9wdGlvbi4KCiAgIElmIHRoZSBQQ04t aW5ncmVzcy1ub2RlIHJlY2VpdmVzIGEgcmVzcG9uc2UgdGhhdCBub3RpZmllcyB0aGF0IHRoZQog ICBwcm9iZSB3YXMgc3VjY2Vzc2Z1bGx5IHByb2Nlc3NlZCwgdGhlbiB0aGUgcmVzZXJ2YXRpb24g cmVxdWVzdCBpcwogICBhZG1pdHRlZC4gIE90aGVyd2lzZSBpdCBpcyByZWplY3RlZC4gIEJvdGgg c2l0dWF0aW9ucyBhcmUgbm90aWZpZWQgdG8KICAgdGhlIHNlbmRlciBvZiB0aGUgZmxvdy4KCiAg IElmIG5vIHByb2JpbmcgaXMgdXNlZCB3aXRoaW4gdGhlIFBDTiBkb21haW4sIHRoZSByZXF1ZXN0 IGZvcgogICBhZG1pc3Npb24gY2FuIGJlIGFjY29tcGxpc2hlZCBieSB1c2luZyBhbiBleHRlcm5h bCB0byBQQ04gc2lnbmFsaW5nCiAgIHByb3RvY29sLiAgSW4gdGhpcyBjYXNlIHdoZW4gdGhlIHJl cXVlc3QgYXJyaXZlcyBhdCBhIFBDTl9lZ3Jlc3Nfbm9kZQogICB0aGF0IG9wZXJhdGVzIGluIGFk bWlzc2lvbiBjb250cm9sIG9wZXJhdGlvbi9zdGF0ZSB0aGVuIHRoZSByZXF1ZXN0CiAgIGlzIHJl amVjdGVkLiAgSWYgaXQgb3BlcmF0ZXMgaW4gTm9ybWFsIG9wZXJhdGlvbi9zdGF0ZSBpcyBhY2Nl cHRlZC4KCiAgIFdoZW4gdGhlIEZsb3cgVGVybWluYXRpb24gcHJvY2VkdXJlIGlzIGFsc28gc3Vw cG9ydGVkLCB0aGVuIGF0IGxlYXN0CiAgIHR3byBwcmUtY29uZmlndXJlZCBiYW5kd2lkdGggdGhy ZXNob2xkcyBhcmUgdXNlZCwgaS5lLiwKICAgUENOX2xvd2VyX3JhdGVfZWdyZXNzIGFuZCBQQ05f dXBwZXJfcmF0ZV9lZ3Jlc3MsIHdpdGgKICAgUENOX3VwcGVyX3JhdGVfZWdyZXNzID4gUENOX2xv d2VyX3JhdGVfZWdyZXNzLgoKICAgQnV0IGhvdyB3aWxsIHRoZSBQQ05fZWdyZXNzX25vZGUgY2hh bmdlIHN0YXRlIGZyb20gQWRtaXNzaW9uIENvbnRyb2wKICAgc3RhdGUgdG8gRmxvdyBUZXJtaW5h dGlvbiBzdGF0ZS4gIFR3byBzb2x1dGlvbnMgYXJlIHByb3ZpZGVkIGJlbG93CiAgIHRoYXQgc3Bl Y2lmeSBob3cgdGhlIFBDTl9lZ3Jlc3Nfbm9kZSBjYW4gdHJhbnNpdGlvbiBmcm9tIEFkbWlzc2lv bgogICBjb250cm9sIHN0YXRlIHRvIEZsb3cgVGVybWluYXRpb24gc3RhdGUuICBGaXJzdCBzb2x1 dGlvbjogaWYgdGhlCiAgIFBDTl9pbnRlcmlvcl9ub2RlcyB1c2UgdGhlIFBDTl9BZmZlY3RlZF9t YXJraW5nIGVuY29kaW5nIG9ubHkgZHVyaW5nCiAgIGZsb3cgdGVybWluYXRpb24gZm9yIHRoZSBw YWNrZXRzIHRoYXQgYXJlIHBhc3NpbmcgdGhyb3VnaCB0aGUgc2V2ZXJlCiAgIGNvbmdlc3RlZCBu b2RlLCBidXQgd2l0aG91dCBiZWluZyBQQ05fbWFya2VkLCB0aGVuIHRoZQogICBQQ05fZWdyZXNz X25vZGUgY2FuIGNoYW5nZSB0byBmbG93IHRlcm1pbmF0aW9uIHN0YXRlIHdoZW4gaXQgcmVjZWl2 ZXMKICAgUENOX0FmZmVjdGVkX21hcmtlZCBwYWNrZXRzLiAgVGhlIHRyYW5zaXRpb24gZnJvbSBm bG93IHRlcm1pbmF0aW9uCiAgIHN0YXRlIHRvIG5vcm1hbCBzdGF0ZSBvY2N1cnMgd2hlbiB0aGUg UENOX2VncmVzc19ub2RlIGRvZXMgbm90CiAgIHJlY2VpdmUgYW55IFBDTl9BZmZlY3RlZF9tYXJr ZWQgcGFja2V0cy4gIFNlY29uZCBzb2x1dGlvbjogSW4gb3JkZXIKICAgdG8gZXhwbGFpbiB0aGlz LCBpdCBpcyBpbXBvcnRhbnQgdG8gbm90ZSB0aGF0IGVhY2ggUENOX2ludGVyaW9yX25vZGUsCiAg IHRoYXQgaXMgaW4gQWRtaXNzaW9uIENvbnRyb2wgc3RhdGUsIGNhbiBQQ05fbWFyayBwYWNrZXRz IHVwIHRvCiAgIEFkbWlzc2lvbl9vZmZzZXRfcmF0ZS4gIEZ1cnRoZXJtb3JlLCBpZiBhIFBDTl9p bnRlcmlvcl9ub2RlIHJlY2VpdmVzCiAgIGluY29taW5nIFBDTl9tYXJrZWQgcGFja2V0cyBhbmQg aXMgaW4gdGhlIEFkbWlzc2lvbiBDb250cm9sIHN0YXRlLAogICB3aWxsIG5vdCByZW1hcmsgYW55 IHBhY2tldHMgaWYgdGhlIGV4Y2VzcyByYXRlIGlzIGVxdWFsIG9yIGxvd2VyIHRoYW4KICAgdGhl IGluY29taW5nX1BDTl9tYXJraW5nX3JhdGUuICBJZiB3ZSBjb25zaWRlciB0aGUgc2l0dWF0aW9u IHdoZXJlIG5vCiAgIEVDTVAgb2NjdXJzIGFuZCB0aGF0IGFsbCBmbG93cyBiZWxvbmdpbmcgdG8g dGhlIHNhbWUgaW5ncmVzcy1lZ3Jlc3MKICAgcGFpciB3aWxsIHVzZSB0aGUgc2FtZSBwYXRoIGZy b20gUENOX2luZ3Jlc3MgdG8gUENOX2VncmVzcywgdGhpcwogICB3b3VsZCBtZWFuIHRoYXQgd2hl biB0aGUgUENOX2VncmVzc19ub2RlIHJlY2VpdmVzIGFuIGV4Y2VzcyByYXRlCiAgIGVxdWFsIHRv IGEgZnJhY3Rpb24gb2YgdGhlIEFkbWlzc2lvbl9vZmZzZXRfcmF0ZSBpLmUuICBGICoKICAgQWRt aXNzaW9uX29mZnNldF9yYXRlLCB3aGVyZSAxID49IEYgPiBBLCBpdCB3b3VsZCB0cmFuc2l0aW9u IGZyb20KICAgQWRtaXNzaW9uIENvbnRyb2wgc3RhdGUgdG8gRmxvdyBUZXJtaW5hdGlvbiBzdGF0 ZS4gIE5vdGUgdGhhdCBGIGNhbgogICBiZSBwcmVjb25maWd1cmVkIGFuZCBkZXBlbmRzIG9uIHRo ZSBuZXR3b3JrIHRvcG9sb2d5LiAgVGh1cyBpbiB0aGlzCgoKCldlc3RiZXJnLCBldCBhbC4gICAg ICAgICAgRXhwaXJlcyBNYXkgMTcsIDIwMDggICAgICAgICAgICAgICAgIFtQYWdlIDExXQoMCklu dGVybmV0LURyYWZ0ICAgICAgICAgICAgICAgICAgIExDLVBDTiAgICAgICAgICAgICAgICAgICAg Tm92ZW1iZXIgMjAwNwoKCiAgIGNhc2UgdGhlIHNlY29uZCB0aHJlc2hvbGQsIGlzIGNhbGN1bGF0 ZWQgYXMgZm9sbG93czoKCiAgIFBDTl91cHBlcl9lZ3Jlc3NfcmF0ZSA9IFBDTl9sb3dlcl9lZ3Jl c3NfcmF0ZSArIEYgKgogICBBZG1pc3Npb25fb2Zmc2V0X3JhdGUuICBIb3dldmVyLCB0aGVyZSBh cmUgc29tZSBzcGVjaWFsL2Nvcm5lciBjYXNlcywKICAgdGhhdCBtYWlubHkgb2NjdXIgd2hlbiBk aWZmZXJlbnQgY29uZ2VzdGlvbiBwb2ludHMgKGFkbWlzc2lvbiBjb250cm9sCiAgIGNvbmdlc3Rl ZCBQQ05faW50ZXJpb3Jfbm9kZXMpIG9uIHRoZSBzYW1lIHBhdGggYXJlIG5vdCBzaW11bHRhbmVv dXNseQogICBzdGFydGluZyB0byBiZSBjb25nZXN0ZWQuICBUaGVyZWZvcmUgd2UgdXNlIHRoZSBt dWx0aWNvbmdlc3Rpb25fZXJyb3IKICAgcGFyYW1ldGVyIHRvIGlkZW50aWZ5IHRoZSBlcnJvciBi b3VuZCB0aGF0IG9jY3VycyBkdWUgdG8gdGhlc2UKICAgc3BlY2lhbCBjYXNlcy4gIE5vdGUgdGhh dCB0aGlzIGVycm9yIGJvdW5kIGNhbiBiZSBlLmcuLCBwcmVkZWZpbmVkCiAgIG9uZXMgb2ZmIGxp bmUgYnkgdGhlIG9wZXJhdG9yLCBieSBzdHVkeWluZyB0aGUgbmV0d29yayB0b3BvbG9neQogICBh bmQvb3Igc3R1ZHlpbmcgaG93IG9mdGVuIHN1Y2ggY29ybmVyIGNhc2VzIGNvdWxkIG9jY3VyIGFu ZC9vciBkb2luZwogICBvZmYgbGluZSBtZWFzdXJlbWVudHMuICBUaGVyZWZvcmUsIHRoZSBQQ05f dXBwZXJfcmF0ZV9lZ3Jlc3MgY2FuIGJlCiAgIGNhbGN1bGF0ZWQgYXMgZm9sbG93czoKCiAgICAg ICAgICAgICAgIFBDTl91cHBlcl9yYXRlX2VncmVzcyA9IFBDTl9sb3dlcl9yYXRlX2VncmVzcyAr CiAgICAgICAgICAgICAgICAgICAgRiAqIEFkbWlzc2lvbl9vZmZzZXRfcmF0ZSArLy0gbXVsdGlj b25nZXN0aW9uX2Vycm9yCgogICBOb3RlIHRoYXQgd2hlbiB0aGUgUENOX0FmZmVjdGVkX21hcmtp bmcgaXMgYXBwbGllZCBpbiB3aG9sZSBQQ04KICAgZG9tYWluLCB0aGVuIHRoZSBmaXJzdCBzb2x1 dGlvbiBkZXNjcmliZWQgYWJvdmUgU0hPVUxEIGJlIHNlbGVjdGVkLAogICBvdGhlcndpc2UgdGhl IHNlY29uZCBzb2x1dGlvbiBkZXNjcmliZWQgYWJvdmUgU0hPVUxEIGJlIHNlbGVjdGVkLgoKICAg VGhlIFBDTi1lZ3Jlc3Mtbm9kZSBzaG91bGQgb3BlcmF0ZSBpbiB0aGUgZm9sbG93aW5nIHdheS4K CiAgIFdoZW4gdGhlIFBDTi1lZ3Jlc3Mtbm9kZSBvcGVyYXRlcyBpbiBmbG93IHRlcm1pbmF0aW9u IHN0YXRlLCB0aGVuIHRoZQogICBQQ04tIGVncmVzcy1ub2RlIGNhbiBjYWxjdWxhdGUgdGhlIGFt b3VudCBvZiBleGNlc3MgcmF0ZSBhYm92ZSB0aGlzCiAgIHRocmVzaG9sZCwgc2VlIFNlY3Rpb24g NC4yLjMuCgogICBCeSB1c2luZyB0aGlzIGV4Y2VzcyByYXRlLCB0aGUgUENOLWVncmVzcy1ub2Rl IGNhbiBzdXBwb3J0IHRoZSBiZWxvdwogICBvcHRpb25zOgoKICAgbyAgaWRlbnRpZnkgb25nb2lu ZyBmbG93cywgdGhhdCBhcmUgcGFydCBvZiB0aGUgYWdncmVnYXRlLCB0byBiZQogICAgICB0ZXJt aW5hdGVkIGFuZCBzZW5kIEZsb3cgVGVybWluYXRpb24gbm90aWZpY2F0aW9ucyB0byB0aGVzZQog ICAgICBvbmdvaW5nIHNlc3Npb25zIHRvd2FyZHMgdGhlIFBDTi1pbmdyZXNzLW5vZGUKCiAgIG8g IHNlbmQgdGhlIG1lYXN1cmVkIHZhbHVlKHMpIG9mIHRoZSBleGNlc3MgcmF0ZSB0b3dhcmRzIHRo ZSBQQ04tCiAgICAgIGluZ3Jlc3Mtbm9kZQoKICAgVGhlICJQQ05fQWZmZWN0ZWRfbWFya2luZyBE U0NQIiBlbmNvZGluZyBpcyB1c2VkIHRvIG1hcmsgYWxsIHBhY2tldHMKICAgdGhhdCBhcmUgcGFz c2luZyB0aHJvdWdoIGFuIFBDTi1pbnRlcmlvci1ub2RlIHRoYXQgaXMgb3BlcmF0aW5nIGluCiAg IEZsb3cgVGVybWluYXRpb24gc3RhdGUgYW5kIGFyZSBub3QgIlBDTl9tYXJraW5nIERTQ1AiIGVu Y29kZWQuICBUaGUKICAgUENOLWVncmVzcy1ub2RlIHVzZXMgdGhlIHJlY2VpdmVkICJQQ05fQWZm ZWN0ZWRfbWFya2luZyBEU0NQIiBwYWNrZXRzCiAgIHRvIGlkZW50aWZ5IHdoaWNoIGZsb3dzIGhh dmUgcGFzc2VkIHRocm91Z2ggb25lIG9yIG1vcmUgUENOLUludGVyaW9yLQogICBOb2RlcyB0aGF0 IG9wZXJhdGUgaW4gRmxvdyBUZXJtaW5hdGlvbiBzdGF0ZS4gIEluIHRoaXMgd2F5IGFuIEVDTVAK ICAgc29sdXRpb24gY2FuIGJlIHByb3ZpZGVkIGZvciB0aGUgRmxvdyBUZXJtaW5hdGlvbiBzdGF0 ZS4KCiAgIElmIHRoZSBQQ04taW5ncmVzcy1ub2RlLCBkdWUgdG8gdGhlIEZsb3cgVGVybWluYXRp b24gY29uZ2VzdGlvbgogICBzaXR1YXRpb24sIHJlY2VpdmVzIGZsb3cgdGVybWluYXRpb24gbm90 aWZpY2F0aW9ucyBmb3IgY2VydGFpbiBmbG93cywKICAgaXQgd2lsbCBoYXZlIHRvIHRlcm1pbmF0 ZSB0aGVzZSBmbG93cyB3aXRoaW4gdGhlIFBDTiBkb21haW4gYW5kIHNlbmQKCgoKV2VzdGJlcmcs IGV0IGFsLiAgICAgICAgICBFeHBpcmVzIE1heSAxNywgMjAwOCAgICAgICAgICAgICAgICAgW1Bh Z2UgMTJdCgwKSW50ZXJuZXQtRHJhZnQgICAgICAgICAgICAgICAgICAgTEMtUENOICAgICAgICAg ICAgICAgICAgICBOb3ZlbWJlciAyMDA3CgoKICAgZmxvdyB0ZXJtaW5hdGlvbiBub3RpZmljYXRp b25zIHRvd2FyZHMgdGhlIHNlbmRlciBvZiB0aGVzZSBmbG93cy4KICAgVGhlIFBDTi1pbmdyZXNz LW5vZGUsIHVwIHRvIHRoZSBtb21lbnQgdGhhdCB0aGUgc2V2ZXJlIGNvbmdlc3Rpb24KICAgc2l0 dWF0aW9uIGlzIHNvbHZlZCwgaXQgd2lsbCBhbHNvIGhhdmUgdG8gc3RvcCBhZG1pdHRpbmcgbmV3 IGZsb3dzCiAgIHRoYXQgY291bGQgYmUgaW5jb3Jwb3JhdGVkIHdpdGhpbiB0aGUgYWdncmVnYXRl ZCBzdGF0ZSB0aGF0IGlzCiAgIGFmZmVjdGVkIGJ5IHRoZSBzZXZlcmUgY29uZ2VzdGlvbiBzaXR1 YXRpb24uICBGdXJ0aGVybW9yZSwgdGhlIFBDTi0KICAgaW5ncmVzcy1ub2RlIHVzZXMgdGhlIHJl Y2VpdmVkIG1lYXN1cmVkIGV4Y2VzcyByYXRlIHRvIHJlc2l6ZSB0aGUKICAgYWdncmVnYXRlZCBy ZXNlcnZhdGlvbiBzdGF0ZS4KCgo0LiAgTEMtUENOIGRldGFpbGVkIGRlc2NyaXB0aW9uCgogICBU aGlzIHNlY3Rpb24gZGVzY3JpYmVzIHRoZSBkZXRhaWxzIG9mIHRoZSB1c2VkIExDLVBDTiBhbGdv cml0aG1zLgogICBTZWN0aW9uIDQuMSBhbmQgNC4yIGRlc2NyaWJlIHRoZSAiQWRtaXNzaW9uIGNv bnRyb2wgYmFzZWQgb24gcHJvYmluZyIKICAgYW5kICJGbG93IFRlcm1pbmF0aW9uIiBzY2VuYXJp bywgcmVzcGVjdGl2ZWx5LCBmb3IgdGhlIHNpdHVhdGlvbiB0aGF0CiAgIHRoZSBlbmQtdG8tZW5k IHNlc3Npb25zIGFyZSB1c2luZyB1bmlkaXJlY3Rpb25hbCByZXNlcnZhdGlvbnMuCiAgIFNlY3Rp b25zIDQuMyBhbmQgNC40IGFyZSBkZXNjcmliaW5nIHRoZSB0d28gYWxnb3JpdGhtcyBmb3IgdGhl CiAgIHNpdHVhdGlvbiB0aGF0IHRoZSBlbmQtdG8tZW5kIHNlc3Npb25zIGFyZSB1c2luZyBiaS1k aXJlY3Rpb25hbAogICByZXNlcnZhdGlvbnMuCgo0LjEuICBBZG1pc3Npb24gY29udHJvbCBiYXNl ZCBvbiBwcm9iaW5nIGZvciB1bmlkaXJlY3Rpb25hbCBmbG93cwoKICAgVGhlIGFkbWlzc2lvbiBj b250cm9sIGZ1bmN0aW9uIGJhc2VkIG9uIHByb2JpbmcgY2FuIGJlIHVzZWQgdG8KICAgaW1wbGVt ZW50IGEgc2ltcGxlIG1lYXN1cmVtZW50LWJhc2VkIGFkbWlzc2lvbiBjb250cm9sIHdpdGhpbiBh IFBDTgogICBkb21haW4uICBBdCBQQ04taW50ZXJpb3Itbm9kZXMgYWxvbmcgdGhlIGRhdGEgcGF0 aCBQQ05fbG93ZXJfcmF0ZSBhcmUKICAgc2V0IGluIHRoZSBtZWFzdXJlbWVudCBiYXNlZCBhZG1p c3Npb24gY29udHJvbCBmdW5jdGlvbiBmb3IgdGhlCiAgIHRyYWZmaWMgYmVsb25naW5nIHRvIGRp ZmZlcmVudCBQSEJzLgoKNC4xLjEuICBPcGVyYXRpb24gaW4gUENOLWluZ3Jlc3Mtbm9kZXMKCiAg IEFmdGVyIGEgdHJpZ2dlciBldmVudCwgZS5nLiwgdGhlIFBDTi1pbmdyZXNzLW5vZGUgcmVjZWl2 ZXMgYQogICByZXNlcnZhdGlvbiByZXF1ZXN0IG1lc3NhZ2UsIHRoZSBQQ04taW5ncmVzcy1ub2Rl IGNhbiBkbyB0aGUKICAgZm9sbG93aW5nOgoKICAgSWYgdGhlIFBDTiBkb21haW4gc3VwcG9ydHMg cHJvYmluZywgdGhlbiB0aGUgUENOX2luZ3Jlc3Nfbm9kZSBzZW5kcyBhCiAgIHByb2JlIHBhY2tl dCwgc2VlIEZpZ3VyZSAyLCB0b3dhcmRzIHRoZSBQQ04tZWdyZXNzLW5vZGUuICBOb3RlIHRoYXQK ICAgdGhlIHByb2JlIHBhY2tldCBzaG91bGQgdXNlIHRoZSBzYW1lIGZsb3cgSUQgaW5mb3JtYXRp b24gYW5kIERTQ1AKICAgdmFsdWUgYXMgdGhlIGRhdGEgcGFja2V0cyBhc3NvY2lhdGVkIHdpdGgg dGhlIHJlY2VpdmVkIHJlc2VydmF0aW9uCiAgIHJlcXVlc3QgbWVzc2FnZS4gIFRoZSBwcm9iZSBw YWNrZXQgU0hPVUxEIHNldCBhIFJvdXRlciBBbGVydCBPcHRpb24uCiAgIElmIHRoZSBQQ04taW5n cmVzcy1ub2RlIHJlY2VpdmVzIGEgcmVzcG9uc2UgdGhhdCBub3RpZmllcyB0aGF0IHRoZQogICBw cm9iZSB3YXMgc3VjY2Vzc2Z1bGx5IHByb2Nlc3NlZCwgdGhlbiB0aGUgcmVzZXJ2YXRpb24gcmVx dWVzdCBpcwogICBhZG1pdHRlZC4gIE90aGVyd2lzZSBpdCBpcyByZWplY3RlZC4gIEJvdGggc2l0 dWF0aW9ucyBoYXZlIHRvIGJlCiAgIG5vdGlmaWVkIHRvIHRoZSBzZW5kZXIgb2YgdGhlIGZsb3cu CgogICBJZiB0aGUgUENOIGRvbWFpbiBkb2VzIG5vdCBzdXBwb3J0IHByb2JpbmcsIHRoZW4gdGhl IHJlc2VydmF0aW9uCiAgIHJlcXVlc3QgbWVzc2FnZSBiZWxvbmdpbmcgdG8gdGhlIGV4dGVybmFs IHNpZ25hbGluZyBwcm90b2NvbCBjYW4gYmUKICAgdXNlZCBkdXJpbmcgdGhlIGFkbWlzc2lvbiBj b250cm9sIHByb2Nlc3MuICBJZiB0aGUgUENOLWluZ3Jlc3Mtbm9kZQogICByZWNlaXZlcyBhIHJl c3BvbnNlIHRoYXQgbm90aWZpZXMgdGhhdCB0aGUgcmVzZXJ2YXRpb24gcmVxdWVzdAogICBtZXNz YWdlIGJlbG9uZ2luZyB0byB0aGUgZXh0ZXJuYWwgc2lnbmFsaW5nIHByb3RvY29sIHdhcyBzdWNj ZXNzZnVsbHkKCgoKV2VzdGJlcmcsIGV0IGFsLiAgICAgICAgICBFeHBpcmVzIE1heSAxNywgMjAw OCAgICAgICAgICAgICAgICAgW1BhZ2UgMTNdCgwKSW50ZXJuZXQtRHJhZnQgICAgICAgICAgICAg ICAgICAgTEMtUENOICAgICAgICAgICAgICAgICAgICBOb3ZlbWJlciAyMDA3CgoKICAgcHJvY2Vz c2VkLCB0aGVuIHRoZSByZXNlcnZhdGlvbiByZXF1ZXN0IGlzIGFkbWl0dGVkLiAgT3RoZXJ3aXNl IGl0IGlzCiAgIHJlamVjdGVkLkJvdGggc2l0dWF0aW9ucyBoYXZlIHRvIGJlIG5vdGlmaWVkIHRv IHRoZSBzZW5kZXIgb2YgdGhlCiAgIGZsb3cuCgo0LjEuMi4gIE9wZXJhdGlvbiBpbiBQQ04taW50 ZXJpb3Itbm9kZXMKCiAgIFVzaW5nIHN0YW5kYXJkIGZ1bmN0aW9uYWxpdGllcyBhZG1pc3Npb24g Y29udHJvbCB0aHJlc2hvbGRzLCBpLmUuLAogICBQQ05fbG93ZXJfcmF0ZSwgYXJlIHNldCBmb3Ig dGhlIHRyYWZmaWMgYmVsb25naW5nIHRvIGRpZmZlcmVudCBQSEJzLAogICBzZWUgU2VjdGlvbiAz LgoKICAgV2hlbiB0aGUgUENOX2ludGVyaW9yX25vZGUgb3BlcmF0ZXMgaW4gQWRtaXNzaW9uIENv bnRyb2wgc3RhdGUgYW5kCiAgIHRoZSBQQ05fbG93ZXJfcmF0ZSBpcyBleGNlZWRlZCB0aGVuIHRo ZSBEU0NQIGZpZWxkIG9mIGRhdGEgcGFja2V0cwogICBhcmUgcHJvcG9ydGlvbmFsbHkgdG8gdGhl IGV4Y2VzcyByYXRlIHJlLW1hcmtlZCwgdXNpbmcgdGhlCiAgIFBDTl9tYXJraW5nIERTQ1AsIHNl ZSBldmVudCBBLCBpbiBGaWd1cmUgNC4gIEZ1cnRoZXJtb3JlLCB3aGVuCiAgIHByb2JpbmcgaXMg dXNlZCBhbmQgd2hlbiB0aGUgUENOX2ludGVyaW9yX25vZGUgb3BlcmF0ZXMgaW4gYWRtaXNzaW9u CiAgIGNvbnRyb2wgc3RhdGUgYW5kIGl0IHJlY2VpdmVzIGEgcHJvYmUgcGFja2V0LCB0aGlzIHBy b2JlIHBhY2tldCBNVVNUCiAgIGJlIHJlbWFya2VkIHVzaW5nIHRoZSBQQ05fbWFyayBEU0NQIGVu Y29kaW5nLiAgTm90ZSB0aGF0IHRoZSBwcm9iZQogICBwYWNrZXQgd2lsbCBiZSBwcm9jZXNzZWQg YnkgdGhlIFBDTl9pbnRlcmlvcl9ub2RlIHNpbmNlIGl0IGNhcnJpZXMgYQogICBSb3V0ZXIgQWxl cnQgT3B0aW9uLgoKICAgQW4gZXhhbXBsZSBvZiB0aGUgZGV0YWlsZWQgb3BlcmF0aW9uIG9mIHRo aXMgcHJvY2VkdXJlIGlzIGRlc2NyaWJlZAogICBiZWxvdy4KCiAgIFRoZSBwcmVkZWZpbmVkIFBD Tl9sb3dlcl9yYXRlLCBzZWUgU2VjdGlvbiAzLjMgYW5kIFNlY3Rpb24gNC4yLjIgaXMKICAgc2V0 IGFjY29yZGluZyB0bywgYW5kIHVzdWFsbHkgbGVzcyB0aGFuLCBhbiBlbmdpbmVlcmVkIGJhbmR3 aWR0aAogICBsaW1pdGF0aW9uLCBpLmUuLCByZWFsIGFkbWlzc2lvbiB0aHJlc2hvbGQsIGJhc2Vk IG9uIGUuZy4gYWdyZWVkCiAgIFNlcnZpY2UgTGV2ZWwgQWdyZWVtZW50IG9yIGEgY2FwYWNpdHkg bGltaXRhdGlvbiBvZiBzcGVjaWZpYyBsaW5rcy4KICAgVGhlIGRpZmZlcmVuY2UgYmV0d2VlbiB0 aGUgUENOX2xvd2VyX3JhdGUgYW5kIHRoZSBlbmdpbmVlcmVkCiAgIGJhbmR3aWR0aCBsaW1pdGF0 aW9uLCBpLmUuLCByZWFsIGFkbWlzc2lvbiB0aHJlc2hvbGQsIHByb3ZpZGVzIGFuCiAgIGludGVy dmFsIHdoZXJlIHRoZSBzaWduYWxpbmcgaW5mb3JtYXRpb24gb24gcmVzb3VyY2UgbGltaXRhdGlv biBpcwogICBhbHJlYWR5IHNlbnQgYnkgYSBub2RlIGJ1dCB0aGUgYWN0dWFsIHJlc291cmNlIGxp bWl0YXRpb24gaXMgbm90CiAgIHJlYWNoZWQuICBOb3RlIHRoYXQgdGhpcyBkaWZmZXJlbmNlIGlz IHVzZWQgYXQgdGhlIFBDTi1lZ3Jlc3Mtbm9kZSB0bwogICB0cmlnZ2VyIHRoZSBzaXR1YXRpb24g dGhhdCB0aGUgUENOLWVncmVzcy1ub2RlIG9wZXJhdGVzIGluIHRoZQogICBhZG1pc3Npb24gY29u dHJvbCBzdGF0ZS4gIFRoaXMgaXMgZHVlIHRvIHRoZSBmYWN0IHRoYXQgZGF0YSBwYWNrZXRzCiAg IGFzc29jaWF0ZWQgd2l0aCBhbiBhZG1pdHRlZCBzZXNzaW9uIGhhdmUgbm90IHlldCBhcnJpdmVk LCB3aGlsZQogICBhbGxvd3MgdGhlIGFkbWlzc2lvbiBjb250cm9sIHByb2Nlc3MgYXZhaWxhYmxl IGF0IHRoZSBQQ04tZWdyZXNzLW5vZGUKICAgdG8gaW50ZXJwcmV0IHRoZSBzaWduYWxpbmcgaW5m b3JtYXRpb24gYW5kIHJlamVjdCBuZXcgY2FsbHMgYmVmb3JlCiAgIHJlYWNoaW5nIGNvbmdlc3Rp b24uICBOb3RlIHRoYXQgaW4gdGhlIHNpdHVhdGlvbiB3aGVuIHRoZSBkYXRhIHJhdGUKICAgaXMg aGlnaGVyIHRoYW4gdGhlIHByZWNvbmZpZ3VyZWQgY29uZ2VzdGlvbiBub3RpZmljYXRpb24gcmF0 ZSwgYWxzbwogICBkYXRhIHBhY2tldHMgYXJlIHJlLW1hcmtlZCB0byBQQ05fbWFya2luZyBEU0NQ LgoKICAgRHVyaW5nIGFkbWlzc2lvbiBjb250cm9sIHRoZSBpbnRlcmlvciBub2RlIGNhbGN1bGF0 ZXMsIHBlciB0cmFmZmljCiAgIGNsYXNzIChQSEIpLCB0aGUgaW5jb21pbmcgcmF0ZSB0aGF0IGlz IGFib3ZlIFBDTl9sb3dlcl9yYXRlLCBkZW5vdGVkCiAgIGFzIHNpZ25hbGVkX292ZXJsb2FkX3Jh dGUsIGluIHRoZSBmb2xsb3dpbmcgd2F5OgoKICAgbyAgYmVmb3JlIHF1ZXVpbmcgYW5kIGV2ZW50 dWFsbHkgZHJvcHBpbmcgdGhlIHBhY2tldHMsIGF0IHRoZSBlbmQgb2YKICAgICAgZWFjaCBtZWFz dXJlbWVudCBpbnRlcnZhbCBvZiBUIHNlY29uZHMsIHRoZSBQQ04taW50ZXJpb3Itbm9kZQogICAg ICBzaG91bGQgY291bnQgdGhlIHRvdGFsIG51bWJlciBvZiBvcmlnaW5hbCBEU0NQLCBQQ05fbWFy a2luZyBEU0NQCgoKCldlc3RiZXJnLCBldCBhbC4gICAgICAgICAgRXhwaXJlcyBNYXkgMTcsIDIw MDggICAgICAgICAgICAgICAgIFtQYWdlIDE0XQoMCkludGVybmV0LURyYWZ0ICAgICAgICAgICAg ICAgICAgIExDLVBDTiAgICAgICAgICAgICAgICAgICAgTm92ZW1iZXIgMjAwNwoKCiAgICAgIGFu ZCBQQ05fQWZmZWN0ZWRfbWFya2luZyBEU0NQIGJ5dGVzIHJlY2VpdmVkLCBkZW5vdGUgdGhpcyBu dW1iZXIKICAgICAgYXMgdG90YWxfcmVjZWl2ZWRfYnl0ZXMuICBOb3RlIHRoYXQgdGhlcmUgYXJl IHNpdHVhdGlvbnMgd2hlbiBtb3JlCiAgICAgIHRoYW4gb25lIFBDTi1pbnRlcmlvci1ub2RlcyBp biB0aGUgc2FtZSBjb21tdW5pY2F0aW9uIHBhdGggYmVjb21lCiAgICAgIGFkbWlzc2lvbiBjb250 cm9sIGNvbmdlc3RlZCBhbmQgb3BlcmF0ZSBpbiBBZG1pc3Npb24gQ29udHJvbAogICAgICBzdGF0 ZS4gIFRoZXJlZm9yZSwgYW55IFBDTi1pbnRlcmlvci1ub2RlIGxvY2F0ZWQgYmVoaW5kIGEgUENO LQogICAgICBpbnRlcmlvci1ub2RlIHRoYXQgb3BlcmF0ZXMgaW4gQWRtaXNzaW9uIENvbnRyb2wg c3RhdGUgbWF5IHJlY2VpdmUKICAgICAgUENOX21hcmtpbmcgRFNDUCBhbmQgUENOX0FmZmVjdGVk X21hcmtpbmcgRFNDUCBieXRlcy4KCiAgIFRoZW4gdGhlIFBDTi1pbnRlcmlvci1ub2RlIGNhbGN1 bGF0ZXMgdGhlIGN1cnJlbnQgZXN0aW1hdGVkCiAgIG92ZXJsb2FkZWQgcmF0ZSwgc2F5IHNpZ25h bGVkX292ZXJsb2FkX3JhdGUsIGJ5IHVzaW5nIHRoZSBmb2xsb3dpbmcKICAgZXF1YXRpb246Cgog ICAgIHNpZ25hbGVkX292ZXJsb2FkX3JhdGUgPQogICAgICAgICgodG90YWxfcmVjZWl2ZWRfYnl0 ZXMpIC8gVCkgLSBQQ05fbG93ZXJfcmF0ZSkKCiAgIFRvIHByb3ZpZGUgcmVsaWFibGUgZXN0aW1h dGlvbiBvZiB0aGUgZW5jb2RlZCBpbmZvcm1hdGlvbiBzZXZlcmFsCiAgIHRlY2huaXF1ZXMgY2Fu IGJlIHVzZWQsIHNlZSBbQXRMaTAxXSwgW0FkQ2EwM10sIFtUaENvMDRdLCBbQW5IYTA2XS4KCiAg IFRoZSBieXRlcyB0aGF0IGhhdmUgdG8gYmUgcmVtYXJrZWQgdG8gc2F0aXNmeSB0aGUgc2lnbmFs ZWQgb3ZlcmxvYWQKICAgcmF0ZSwgZS5nLiwgc2lnbmFsZWRfcmVtYXJrZWRfYnl0ZXMsIGFyZSBj YWxjdWxhdGVkIGFzIGZvbGxvd3M6CgogICAgIElGIChtZWFzdXJlZCBQSEIgcmF0ZSA+IFBDTl9s b3dlcl9yYXRlKSBBTkQKICAgICAgICAobWVhc3VyZWQgUEhCIHJhdGUgPTwgUENOX3VwcGVyX3Jh dGUpCiAgICAgVEhFTgogICAgICB7CiAgICAgICAgSUYgKGluY29taW5nX1BDTl9tYXJraW5nX3Jh dGUgPD4gMCkgQU5ECiAgICAgICAgICAgKGluY29taW5nX1BDTl9tYXJraW5nX3JhdGUgPD0gQWRt aXNzaW9uX29mZnNldF9yYXRlKQogICAgICAgIFRIRU4KICAgICAgICAgeyBzaWduYWxlZF9yZW1h cmtlZF9ieXRlcyA9CiAgICAgICAgICAgICAoKHNpZ25hbGVkX292ZXJsb2FkX3JhdGUgLQogICAg ICAgICAgICAgIGluY29taW5nX1BDTl9tYXJraW5nX3JhdGUpICogVCkgLyBOCiAgICAgICAgIH0K ICAgICAgICBFTFNFIElGIChpbmNvbWluZ19QQ05fbWFya2luZ19yYXRlID0gMCkKICAgICAgICBU SEVOIHNpZ25hbGVkX3JlbWFya2VkX2J5dGVzID0KICAgICAgICAgICAgICAgc2lnbmFsZWRfb3Zl cmxvYWRfcmF0ZSAqIFQgLyBOCiAgICAgICAgRUxTRSBJRiAoaW5jb21pbmdfUENOX21hcmtpbmdf cmF0ZSA+CiAgICAgICAgICAgICAgICAgIEFkbWlzc2lvbl9vZmZzZXRfcmF0ZSkKICAgICAgICBU SEVOIHNpZ25hbGVkX3JlbWFya2VkX2J5dGVzID0gMAogICAgICAgfQoKICAgV2hlcmUgdGhlICJp bmNvbWluZ19QQ05fbWFya2luZ19yYXRlIiBpcyBjYWxjdWxhdGVkIGFzIGZvbGxvd3M6CgogICAg IGluY29taW5nX1BDTl9tYXJraW5nX3JhdGUgPQogICAgICAgIChyZWNlaXZlZCBudW1iZXIgb2Yg IlBDTl9tYXJraW5nIiBEU0NQIGR1cmluZyBUKSAqIE4pL1QKCiAgIFdoZW4gaW5jb21pbmcgcmVt YXJrZWQgYnl0ZXMgYXJlIGRyb3BwZWQsIHRoZSBvcGVyYXRpb24gb2YgdGhlCiAgIGFkbWlzc2lv biBjb250cm9sIGFsZ29yaXRobSBtYXkgYmUgYWZmZWN0ZWQsIGUuZy4sIHRoZSBhbGdvcml0aG0g bWF5CiAgIGJlY29tZSBpbiBjZXJ0YWluIHNpdHVhdGlvbnMgc2xvd2VyLiAgQW4gaW1wbGVtZW50 YXRpb24gb2YgdGhlCgoKCldlc3RiZXJnLCBldCBhbC4gICAgICAgICAgRXhwaXJlcyBNYXkgMTcs IDIwMDggICAgICAgICAgICAgICAgIFtQYWdlIDE1XQoMCkludGVybmV0LURyYWZ0ICAgICAgICAg ICAgICAgICAgIExDLVBDTiAgICAgICAgICAgICAgICAgICAgTm92ZW1iZXIgMjAwNwoKCiAgIGFs Z29yaXRobSBtYXkgYXNzdXJlIGFzIG11Y2ggYXMgcG9zc2libGUgdGhhdCB0aGUgaW5jb21pbmcg bWFya2VkCiAgIGJ5dGVzIGFyZSBub3QgZHJvcHBlZC4gIFRoaXMgY291bGQgZm9yIGV4YW1wbGUg YmUgYWNjb21wbGlzaGVkIGJ5CiAgIHVzaW5nIGRpZmZlcmVudCBkcm9wcGluZyByYXRlIHRocmVz aG9sZHMgZm9yIFBDTl9tYXJraW5nIERTQ1AgYW5kCiAgIHVubWFya2VkIChvcmlnaW5hbCBEU0NQ IGFuZCBQQ05fQWZmZWN0ZWRfbWFya2luZyBEU0NQKSBieXRlcywgc2VlCiAgIFNlY3Rpb24gMy4z LgoKICAgV2hlbiB0aGUgbWVhc3VyZWQgUEhCIHRocm91Z2hwdXQgcmF0ZSBpcyBoaWdoZXIgdGhh biBQQ05fdXBwZXJfcmF0ZSwKICAgc2VlIEZpZ3VyZSA0LCB0aGVuIGl0IGlzIGNvbnNpZGVyZWQg dGhhdCB0aGUgb3BlcmF0aW9uIFBDTi1pbnRlcmlvci0KICAgbm9kZSBoYXMgbW92ZWQgdG8gdGhl IEZsb3cgVGVybWluYXRpb24gc3RhdGUuCgo0LjEuMy4gIE9wZXJhdGlvbiBpbiBQQ04tZWdyZXNz LW5vZGVzCgogICBXaGVuIHRoZSBvcGVyYXRpb24gc3RhdGUgb2YgdGhlIGluZ3Jlc3MvZWdyZXNz IHBhaXIgYWdncmVnYXRlIGluIHRoZQogICBQQ05fZWdyZXNzX25vZGUgaXMgaW4gdGhlIEFkbWlz c2lvbiBDb250cm9sIHN0YXRlIChzZWUgRmlndXJlIDQgYW5kCiAgIFNlY3Rpb24gNC4yLjMpLCB0 aGVuIHRoZSBpbXBsZW1lbnRhdGlvbiBvZiB0aGlzIGFsZ29yaXRobSBpcwogICBhY2NvbXBsaXNo ZWQgdXNpbmcgdGhlIHJlY2VpdmVkIGRhdGEgcGFja2V0cyB0aGF0IGFyZSBtYXJrZWQgdXNpbmcK ICAgdGhlIFBDTl9tYXJraW5nIERTQ1AgZW5jb2RpbmcuICBJbiB0aGlzIGNhc2UsIGR1cmluZyBh IG1lYXN1cmVtZW50CiAgIGludGVydmFsIFQsIHRoZSBQQ04tZWdyZXNzLW5vZGUgbWVhc3VyZXMg dGhlIGlucHV0X1BDTl9tYXJraW5nX2J5dGVzCiAgIGJ5IGNvdW50aW5nLCBkdXJpbmcgdGhlIGlu dGVydmFsIFQsIHRoZSBQQ05fbWFya2luZyBieXRlcy4KCiAgIFRoZSBpbmNvbWluZ19QQ05fbWFy a2luZ19yYXRlIGNhbiBiZSB0aGVuIGNhbGN1bGF0ZWQgYXMgZm9sbG93czoKCiAgICAgaW5jb21p bmdfUENOX21hcmtpbmdfcmF0ZSA9CiAgICAgICAgTiAqIGlucHV0X1BDTl9tYXJraW5nX2J5dGVz IC8gVAoKICAgVG8gcHJvdmlkZSByZWxpYWJsZSBlc3RpbWF0aW9uIG9mIHRoZSBlbmNvZGVkIGlu Zm9ybWF0aW9uIHNldmVyYWwKICAgdGVjaG5pcXVlcyBjYW4gYmUgdXNlZCwgc2VlIFtBdExpMDFd LCBbQWRDYTAzXSwgW1RoQ28wNF0sIFtBbkhhMDZdLgoKICAgSWYgdGhlIGluY29taW5nX1BDTl9t YXJraW5nX3JhdGUgaXMgaGlnaGVyIHRoYW4gYSBwcmVjb25maWd1cmVkCiAgIFBDTl9sb3dlcl9y YXRlX2VncmVzcyAoc2VlIFNlY3Rpb24gMy40IGFuZCBGaWd1cmUgNCksIHRoZW4gdGhlCiAgIGNv bW11bmljYXRpb24gcGF0aCBiZXR3ZWVuIFBDTi1pbmdyZXNzLW5vZGUgYW5kIFBDTi1lZ3Jlc3Mt bm9kZSBpcwogICBjb25zaWRlcmVkIHRvIGJlIHByZS1jb25nZXN0ZWQuCgogICBJZiBwcm9iaW5n IGlzIHVzZWQgd2l0aGluIHRoZSB3aG9sZSBQQ04gZG9tYWluLCBhbmQgd2hlbiB0aGUgcHJvYmUK ICAgYXJyaXZlcyBhdCBhIFBDTl9lZ3Jlc3Nfbm9kZSB3aXRoIFBDTiBtYXJraW5nIERTQ1AgZW5j b2RlZCB0aGVuIGl0CiAgIFNIT1VMRCBiZSByZWplY3RlZC4gIElmIHRoZSByZXF1ZXN0aW5nIHBy b2JlIHBhY2tldCBpcyBub3QgbWFya2VkCiAgIHVzaW5nIHRoZSBQQ05fbWFya2luZyBEU0NQIHRo ZW4gdGhpcyByZXF1ZXN0aW5nIHByb2JlIFNIT1VMRCBiZQogICBhZG1pdHRlZC4gIEluIHRoaXMg d2F5IGl0IGlzIGVuc3VyZWQgdGhhdCB0aGUgcHJvYmUgcGFja2V0IHBhc3NlZAogICB0aHJvdWdo IHRoZSBub2RlIHRoYXQgaXQgaXMgY29uZ2VzdGVkLiAgVGhpcyBmZWF0dXJlIGlzIHZlcnkgdXNl ZnVsCiAgIHdoZW4gRUNNUCBiYXNlZCByb3V0aW5nIGlzIHVzZWQgdG8gZGV0ZWN0IG9ubHkgZmxv d3MgdGhhdCBhcmUgcGFzc2luZwogICB0aHJvdWdoIHRoZSBwcmUtIGNvbmdlc3RlZCByb3V0ZXIu ICBOb3RlIHRoYXQgaWYgYW4gaW5ncmVzcy9lZ3Jlc3MKICAgcGFpciBhZ2dyZWdhdGVkIHN0YXRl IGlzIG5vdCBhdmFpbGFibGUgYXQgdGhlIFBDTl9lZ3Jlc3Nfbm9kZSwgdGhlbgogICB0aGUgUENO X2VncmVzcyBub2RlIGNhbm5vdCBkZXRlcm1pbmUgd2hldGhlciBhIFBDTl9lZ3Jlc3Nfbm9kZQog ICBhc3NvY2lhdGVkIHdpdGggdGhlIGluZ3Jlc3MtZWdyZXNzIGFnZ3JlZ2F0ZSBvcGVyYXRlcyBp biBub3JtYWwKICAgc3RhdGUsIGFkbWlzc2lvbiBjb250cm9sIHN0YXRlIG9yIGZsb3cgdGVybWlu YXRpb24gc3RhdGUuICBIb3dldmVyLAogICBldmVuIGluIHRoaXMgY2FzZSwgd2hlbiBhIHByb2Jl IHBhY2tldCBhcnJpdmVzIGF0IHRoZSBQQ04tZWdyZXNzLQogICBub2RlLCB0aGVuIHRoaXMgcmVx dWVzdCBpcyByZWplY3RlZCBpZiB0aGUgcHJvYmUgcGFja2V0IGlzCiAgIFBDTl9tYXJrZWQuICBP dGhlcndpc2UgKGlmIGl0IGlzIG5vdCBQQ05fbWFya2VkKSBpdCBpcyBhY2NlcHRlZC4KCgoKV2Vz dGJlcmcsIGV0IGFsLiAgICAgICAgICBFeHBpcmVzIE1heSAxNywgMjAwOCAgICAgICAgICAgICAg ICAgW1BhZ2UgMTZdCgwKSW50ZXJuZXQtRHJhZnQgICAgICAgICAgICAgICAgICAgTEMtUENOICAg ICAgICAgICAgICAgICAgICBOb3ZlbWJlciAyMDA3CgoKICAgSWYgcHJvYmluZyBpcyBub3QgdXNl ZCB3aXRoaW4gdGhlIHdob2xlIFBDTiBkb21haW4gYW5kIHRoZSByZXF1ZXN0CiAgIGZvciBhZG1p c3Npb24gY2FuIGJlIGFjY29tcGxpc2hlZCBieSB1c2luZyBhbiBleHRlcm5hbCB0byBQQ04sCiAg IHNpZ25hbGluZyBwcm90b2NvbC4gIEluIHRoaXMgY2FzZSB3aGVuIHRoZSByZXF1ZXN0IGFycml2 ZXMgYXQgYQogICBQQ05fZWdyZXNzX25vZGUgdGhhdCBvcGVyYXRlcyBpbiBhZG1pc3Npb24gY29u dHJvbCBzdGF0ZSB0aGVuIHRoZQogICByZXF1ZXN0IGlzIHJlamVjdGVkLiAgSWYgaXQgb3BlcmF0 ZXMgaW4gTm9ybWFsIHN0YXRlIGl0IGlzIGFjY2VwdGVkLgoKICAgSW4gYW55IG9mIHRoZSBzaXR1 YXRpb25zIHRoZSBQQ04tZWdyZXNzLW5vZGUgd2lsbCBoYXZlIHRvIG5vdGlmeSB0aGUKICAgUENO LWluZ3Jlc3Mtbm9kZSB3aGV0aGVyIHRoZSByZXF1ZXN0IGZvciByZXNlcnZhdGlvbiBpcyBhZG1p dHRlZCBvcgogICByZWplY3RlZC4KClBDTi1pbmdyZXNzLW5vZGUgIFBDTi1pbnRlcmlvci1ub2Rl ICBQQ04taW50ZXJpb3Itbm9kZSAgIFBDTi1lZ3Jlc3Mtbm9kZQoKICB1c2VyICB8ICAgICAgICAg ICAgICAgICAgfCAgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAgIHwKICBkYXRhICB8 ICB1c2VyIGRhdGEgICAgICAgfCAgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAgIHwK IC0tLS0tLT58LS0tLS0tLS0tLS0tLS0tLS0+fCAgICAgdXNlciBkYXRhICAgfCAgICAgICAgICAg ICAgICAgIHwKICAgICAgICB8ICAgICAgICAgICAgICAgICAgfC0tLS0tLS0tLS0tLS0tLS0+fCB1 c2VyIGRhdGEgICAgICAgIHwKICAgICAgICB8ICAgICAgICAgICAgICAgICAgfCAgICAgICAgICAg ICAgICAgfC0tLS0tLS0tLS0tLS0tLS0tPnwKICB1c2VyICB8ICAgICAgICAgICAgICAgICAgfCAg ICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAgIHwKICBkYXRhICB8ICB1c2VyIGRhdGEg ICAgICAgfCAgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAgIHwKIC0tLS0tLT58LS0t LS0tLS0tLS0tLS0tLS0+fCAgICAgdXNlciBkYXRhICAgfCB1c2VyIGRhdGEgICAgICAgIHwKICAg ICAgICB8ICAgICAgICAgICAgICAgICAgfC0tLS0tLS0tLS0tLS0tLS0+UygjIG1hcmtlZCBieXRl cykgIHwKICAgICAgICB8ICAgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAgUy0tLS0t LS0tLS0tLS0tLS0tPnwKICAgICAgICB8ICAgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAg ICAgUygjIHVubWFya2VkIGJ5dGVzKXwKICAgICAgICB8ICAgICAgICAgICAgICAgICAgfCAgICAg ICAgICAgICAgICAgUy0tLS0tLS0tLS0tLS0tLS0tPnwKICAgICAgICB8ICAgICAgICAgICAgICAg ICAgfCAgICAgICAgICAgICAgICAgUyAgICAgICAgICAgICAgICAgIHwKcmVxdWVzdCBmb3IgcmVz ZXJ2YXRpb24gICAgfCAgICAgICAgICAgICAgICAgUyAgICAgICAgICAgICAgICAgIHwKLS0tLS0t LT58ICAgICAgICAgICBwcm9iZSBwYWNrZXQgICAgICAgICAgICAgUyAgICAgICAgICAgICAgICAg IHwKICAgICAgICB8LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0+UyAgICAgICAg ICAgICAgICAgIHwKICAgICAgICB8ICAgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAg UyAgcHJvYmUgcGFja2V0ICAgIHwKICAgICAgICB8ICAgICAgICAgICAgICAgICAgfCAgICAgICAg ICAgICAgICAgUy0tLS0tLS0tLS0tLS0tLS0tPnwKICAgICAgICB8ICAgICAgICAgICAgICAgICAg fHJlc3BvbnNlICAgICAgICAgICAgICAgICAgICAgICAgICAgIHwKICAgICAgICB8PC0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLXwKIHJlc3BvbnNl ICAgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAgIHwK IDwtLS0tLS18ICAgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAgfCAgICAgICAgICAg ICAgICAgIHwKCiAgICAgICAgICAgICAgRmlndXJlOiAyICBBZG1pc3Npb24gY29udHJvbCBiYXNl ZCBvbiBwcm9iaW5nCgo0LjIuICBGbG93IFRlcm1pbmF0aW9uIGZvciB1bmlkaXJlY3Rpb25hbCBm bG93cwoKICAgVGhlIEZsb3cgVGVybWluYXRpb24gaGFuZGxpbmcgbWV0aG9kIHJlcXVpcmVzIHRo ZSBmb2xsb3dpbmcKICAgZnVuY3Rpb25hbGl0aWVzLgoKNC4yLjEuICBPcGVyYXRpb24gaW4gdGhl IFBDTi1pbmdyZXNzLW5vZGVzCgogICBVcG9uIHJlY2VpdmluZyB0aGUgbm90aWZpY2F0aW9uIG1l c3NhZ2Ugc2VudCBieSB0aGUgUENOLWVncmVzcy1ub2RlLAogICB0aGUgUENOLWluZ3Jlc3Mtbm9k ZSByZXNvbHZlcyB0aGUgZmxvdyB0ZXJtaW5hdGlvbiBjb25nZXN0aW9uIGJ5IGEKICAgcHJlZGVm aW5lZCBwb2xpY3ksIGUuZy4sIGJ5IHJlZnVzaW5nIG5ldyBpbmNvbWluZyBmbG93cyAoc2Vzc2lv bnMpLAogICB0ZXJtaW5hdGluZyB0aGUgYWZmZWN0ZWQgYW5kIG5vdGlmaWVkIGZsb3dzIChzZXNz aW9ucyksIGFuZCBibG9ja2luZwoKCgpXZXN0YmVyZywgZXQgYWwuICAgICAgICAgIEV4cGlyZXMg TWF5IDE3LCAyMDA4ICAgICAgICAgICAgICAgICBbUGFnZSAxN10KDApJbnRlcm5ldC1EcmFmdCAg ICAgICAgICAgICAgICAgICBMQy1QQ04gICAgICAgICAgICAgICAgICAgIE5vdmVtYmVyIDIwMDcK CgogICB0aGVpciBwYWNrZXRzIG9yIHNoaWZ0aW5nIHRoZW0gdG8gYW4gYWx0ZXJuYXRpdmUgTEMt UENOIHRyYWZmaWMgY2xhc3MKICAgKFBIQikuICBUaGlzIG9wZXJhdGlvbiBpcyBkZXBpY3RlZCBp biBGaWd1cmUgMywgd2hlcmUgdGhlIFBDTi0KICAgaW5ncmVzcy0gbm9kZSwgZm9yIGVhY2ggZmxv dyAoc2Vzc2lvbikgdG8gYmUgdGVybWluYXRlZCwgcmVjZWl2ZXMgYQogICBub3RpZmljYXRpb24g bWVzc2FnZS4KCiAgIFdoZW4gdGhlIFBDTi1pbmdyZXNzLW5vZGUgcmVjZWl2ZXMgdGhlIG5vdGlm aWNhdGlvbiBtZXNzYWdlLCBpdAogICBzdGFydHMgdGhlIHRlcm1pbmF0aW9uIG9mIHRoZSBmbG93 cyB3aXRoaW4gdGhlIExDLVBDTiBkb21haW4gYnkKICAgc2VuZGluZyByZWxlYXNlIG1lc3NhZ2Vz LgoKUENOLWluZ3Jlc3Mtbm9kZSAgUENOLWludGVyaW9yLW5vZGUgIFBDTi1pbnRlcmlvci1ub2Rl ICAgUENOLWVncmVzcy1ub2RlCgogIHVzZXIgIHwgICAgICAgICAgICAgICAgICB8ICAgICAgICAg ICAgICAgICB8ICAgICAgICAgICAgICAgICAgfAogIGRhdGEgIHwgIHVzZXIgZGF0YSAgICAgICB8 ICAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgICAgfAogLS0tLS0tPnwtLS0tLS0tLS0t LS0tLS0tLT58ICAgICB1c2VyIGRhdGEgICB8IHVzZXIgZGF0YSAgICAgICAgfAogICAgICAgIHwg ICAgICAgICAgICAgICAgICB8LS0tLS0tLS0tLS0tLS0tLT5TKCMgbWFya2VkIGJ5dGVzKSAgfAog ICAgICAgIHwgICAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgICBTLS0tLS0tLS0tLS0t LS0tLS0+fAogICAgICAgIHwgICAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgICBTKCMg dW5tYXJrZWQgYnl0ZXMpfAogICAgICAgIHwgICAgICAgICAgICAgICAgICB8ICAgICAgICAgICAg ICAgICBTLS0tLS0tLS0tLS0tLS0tLS0+fFRlcm0uCiAgICAgICAgfCAgICAgICAgICAgICAgIG5v dGlmaWNhdGlvbiBmb3IgdGVybWluYXRpb24gICAgICAgICAgICB8Zmxvdz8KICAgICAgICB8PC0t LS0tLS0tLS0tLS0tLS0tfC0tLS0tLS0tLS0tLS0tLS0tUy0tLS0tLS0tLS0tLS0tLS0tLXxZRVMK ICAgICAgICAgICByZWxlYXNlICAgICAgICAgfCAgICAgICAgICAgICAgICAgUyAgICAgICAgICAg ICAgICAgIHwKICAgICAgICB8IC0tLS0tLS0tLS0tLS0tLS0tfC0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tPnwKICAgICAgICB8ICAgICAgICAgICAgICAgICAgfCAgICAgICAgICAg ICAgICAgfCAgICAgICAgICAgICAgICAgIHwKCiAgICAgICAgICAgICAgRmlndXJlOiAzICBMQy1Q Q04gRmxvdyBUZXJtaW5hdGlvbiBoYW5kbGluZwoKICAgV2hlbiB0aGUgUENOLWluZ3Jlc3Mtbm9k ZSByZWNlaXZlcyB0aGUgbm90aWZpY2F0aW9uIG1lc3NhZ2UgdGhhdAogICBjb250YWlucyB0aGUg dG8gYmUgcmVsZWFzZWQgYWdncmVnYXRpb24gYmFuZHdpZHRoLCBpdCBjYW4gdXNlIGl0IHRvCiAg IHJlc2l6ZSB0aGUgc2l6ZSBvZiB0aGUgYWdncmVnYXRpb24gc2l6ZSBhY2NvcmRpbmdseS4KCjQu Mi4yLiAgT3BlcmF0aW9uIGluIHRoZSBQQ04taW50ZXJpb3Itbm9kZXMKCiAgIFRoZSBQQ04taW50 ZXJpb3Itbm9kZSB0aGF0IG9wZXJhdGVzIGluIGEgRmxvdyBUZXJtaW5hdGlvbiBzdGF0ZQogICBy ZW1hcmtzIGRhdGEgcGFja2V0cyBwYXNzaW5nIHRoZSBub2RlLiAgRm9yIHRoaXMgcmVtYXJraW5n LCB0d28KICAgYWRkaXRpb25hbCBEU0NQcyBjYW4gYmUgYWxsb2NhdGVkIGZvciBlYWNoIHRyYWZm aWMgY2xhc3MuICBPbmUgRFNDUAogICBjYW4gYmUgdXNlZCB0byBpbmRpY2F0ZSB0aGF0IHRoZSBw YWNrZXQgcGFzc2VkIGEgbm9kZSB0aGF0IG9wZXJhdGVzCiAgIGluIHRoZSBGbG93IFRlcm1pbmF0 aW9uIHN0YXRlLiAgVGhpcyB0eXBlIG9mIERTQ1AgaXMgZGVub3RlZCBpbiB0aGlzCiAgIGRvY3Vt ZW50IGFzIFBDTl9BZmZlY3RlZF9tYXJraW5nIERTQ1AuCgogICBUaGUgdXNlIG9mIHRoaXMgRFND UCB0eXBlIGVsaW1pbmF0ZXMgdGhlIHBvc3NpYmlsaXR5IHRoYXQsIGR1ZSB0bwogICBlLmcuICBF Q01QIChFcXVhbCBDb3N0IE11bHRpcGxlIFBhdGhzKSBlbmFibGVkIHJvdXRpbmcsIHRoZSBQQ04t CiAgIGVncmVzcy1ub2RlIGVpdGhlciBkb2VzIG5vdCBkZXRlY3QgcGFja2V0cyBwYXNzZWQgYSBu b2RlIHRoYXQgb3BlcmF0cwogICBpbiB0aGUgRmxvdyBUZXJtaW5hdGlvbiBzdGF0ZSBvciBlcnJv bmVvdXNseSBkZXRlY3RzIHBhY2tldHMgdGhhdAogICBhY3R1YWxseSBkaWQgbm90IHBhc3MgdGhl IHNldmVyZSBjb25nZXN0ZWQgbm9kZS4gIE5vdGUgdGhhdCB0aGlzIHR5cGUKICAgb2YgRFNDUCBN VVNUIG9ubHkgYmUgdXNlZCBpZiBhbGwgdGhlIG5vZGVzIHdpdGhpbiB0aGUgUENOIGRvbWFpbiBh cmUKICAgY29uZmlndXJlZCB0byB1c2UgaXQuICBPdGhlcndpc2UsIHRoaXMgdHlwZSBvZiBEU0NQ IE1VU1QgTk9UIGJlCiAgIGFwcGxpZWQuICBUaGUgb3RoZXIgRFNDUCBNVVNUIGJlIHVzZWQgdG8g aW5kaWNhdGUgdGhlIGRlZ3JlZSBvZgogICBjb25nZXN0aW9uIGJ5IG1hcmtpbmcgdGhlIGJ5dGVz IHByb3BvcnRpb25hbGx5IHRvIHRoZSBkZWdyZWUgb2YKCgoKV2VzdGJlcmcsIGV0IGFsLiAgICAg ICAgICBFeHBpcmVzIE1heSAxNywgMjAwOCAgICAgICAgICAgICAgICAgW1BhZ2UgMThdCgwKSW50 ZXJuZXQtRHJhZnQgICAgICAgICAgICAgICAgICAgTEMtUENOICAgICAgICAgICAgICAgICAgICBO b3ZlbWJlciAyMDA3CgoKICAgY29uZ2VzdGlvbi4gIFRoaXMgdHlwZSBvZiBEU0NQIGlzIGRlbm90 ZWQgaW4gdGhpcyBkb2N1bWVudCBhcwogICBQQ05fbWFya2luZy4KCiAgIE5vdGUgdGhhdCBpbiB0 aGlzIGRvY3VtZW50IHRoZSB0ZXJtcyBtYXJrZWQgcGFja2V0cyBvciBtYXJrZWQgYnl0ZXMKICAg cmVmZXIgdG8gdGhlIFBDTl9tYXJraW5nIERTQ1AuICBUaGUgdGVybXMgdW5tYXJrZWQgcGFja2V0 cyBvcgogICB1bm1hcmtlZCBieXRlcyBhcmUgcmVwcmVzZW50aW5nIHRoZSBwYWNrZXRzIG9yIHRo ZSBieXRlcyBiZWxvbmdpbmcgdG8KICAgdGhlc2UgcGFja2V0cyB0aGF0IHRoZWlyIERTQ1AgaXMg ZWl0aGVyIHRoZSBQQ05fQWZmZWN0ZWRfbWFya2luZyBEU0NQCiAgIG9yIHRoZSBvcmlnaW5hbCBE U0NQLiAgRnVydGhlcm1vcmUsIGluIHRoZSBhbGdvcml0aG0gZGVzY3JpYmVkIGJlbG93CiAgIGl0 IGlzIGNvbnNpZGVyZWQgdGhhdCB0aGUgcm91dGVyIG1heSBkcm9wIHJlY2VpdmVkIHBhY2tldHMu ICBUaGUKICAgY291bnRpbmcvbWVhc3VyaW5nIG9mIG1hcmtlZCBvciB1bm1hcmtlZCBieXRlcyBk ZXNjcmliZWQgaW4gdGhpcwogICBzZWN0aW9uIGlzIGFjY29tcGxpc2hlZCB3aXRoaW4gbWVhc3Vy ZW1lbnQgcGVyaW9kcy4gIEFsbCBub2RlcyB3aXRoaW4KICAgYSBQQ04gZG9tYWluIHVzZSBhIG1l YXN1cmVtZW50IGludGVydmFsLCBzYXkgVCBzZWNvbmRzLCB3aGljaCBNVVNUIGJlCiAgIHByZS1j b25maWd1cmVkLgoKICAgVG8gcHJvdmlkZSByZWxpYWJsZSBlc3RpbWF0aW9uIG9mIHRoZSBlbmNv ZGVkIGluZm9ybWF0aW9uIHNldmVyYWwKICAgdGVjaG5pcXVlcyBjYW4gYmUgdXNlZCwgc2VlIFtB dExpMDFdLCBbQWRDYTAzXSwgW1RoQ28wNF0sIFtBbkhhMDZdLgoKICAgSXQgaXMgUkVDT01NRU5E RUQgdGhhdCB0aGUgdG90YWwgbnVtYmVyIG9mIGFkZGl0aW9uYWwgKGxvY2FsIGFuZAogICBleHBl cmltZW50YWwpIERTQ1BzIG5lZWRlZCBmb3IgZmxvdyB0ZXJtaW5hdGlvbiBoYW5kbGluZyB3aXRo aW4gYW4KICAgUENOIGRvbWFpbiBzaG91bGQgYmUgYXMgbG93IGFzIHBvc3NpYmxlIGFuZCBpdCBz aG91bGQgbm90IGV4Y2VlZCB0aGUKICAgbGltaXQgb2YgOC4KCiAgIEFuIGV4YW1wbGUgb2YgYSBy ZW1hcmtpbmcgcHJvY2VkdXJlIGlzIGdpdmVuIGJlbG93LiAgUGVyIHN1cHBvcnRlZAogICBQSEIs IHRoZSBQQ04taW50ZXJpb3Itbm9kZSBjYW4gc3VwcG9ydCB0aGUgb3BlcmF0aW9uIFN0YXRlcyBk ZXBpY3RlZAogICBpbiBGaWd1cmUgNCwgd2hlbiB0aGUgYWRtaXNzaW9uIGNvbnRyb2wgYmFzZWQg b24gcHJvYmluZyBzaWduYWxpbmcKICAgc2NoZW1lIGlzIHVzZWQgaW4gY29tYmluYXRpb24gd2l0 aCB0aGlzIGZsb3cgdGVybWluYXRpb24gdHlwZS4KCiAgICAgICAgICAgICAgICAgLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCiAgICAgICAgICAgICAgICB8ICAg ICAgICBldmVudCBCICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfAogICAgICAgICAgICAg ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIFYKICAgICAg ICAgICAgIC0tLS0tLS0tLS0gICAgICAgICAgICAgLS0tLS0tLS0tLS0tLSAgICAgICAgICAgLS0t LS0tLS0tLQogICAgICAgICAgICB8IE5vcm1hbCAgIHwgIGV2ZW50IEEgIHwgQWRtaXNzaW9uICAg fCBldmVudCBCIHwgRmxvdyAgICAgIHwKICAgICAgICAgICAgfCAgc3RhdGUgICB8LS0tLS0tLS0t LT58IENvbnRyb2wgICAgIHwtLS0tLS0tLT58VGVybWluYXRpb258CiAgICAgICAgICAgIHwgICAg ICAgICAgfCAgICAgICAgICAgfCAgc3RhdGUgICAgICB8ICAgICAgICAgfCAgc3RhdGUgICAgfAog ICAgICAgICAgICAgLS0tLS0tLS0tLSAgICAgICAgICAgICAtLS0tLS0tLS0tLS0tICAgICAgICAg ICAtLS0tLS0tLS0tCiAgICAgICAgICAgICAgXiAgXiAgICAgICAgICAgICAgICAgICAgICAgfCAg ICAgICAgICAgICAgICAgICAgIHwKICAgICAgICAgICAgICB8ICB8ICAgICAgZXZlbnQgQyAgICAg ICAgICB8ICAgICAgICAgICAgICAgICAgICAgfAogICAgICAgICAgICAgIHwgICAtLS0tLS0tLS0t LS0tLS0tLS0tLS0tLSAgICAgICAgICAgICAgICAgICAgICB8CiAgICAgICAgICAgICAgfCAgICAg ICAgIGV2ZW50IEQgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHwKICAgICAgICAgICAg ICAgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCgogICAg ICAgICAgICAgICBGaWd1cmUgNDogU3RhdGVzIG9mIG9wZXJhdGlvbiwgZmxvdyB0ZXJtaW5hdGlv biB3aXRoCiAgICAgICAgICAgICAgIGNvbmdlc3Rpb24gbm90aWZpY2F0aW9uIGJhc2VkIG9uIHBy b2JpbmcKCiAgIFRoZSB0ZXJtcyB1c2VkIGluIEZpZ3VyZSA0IGFyZToKCiAgIE5vcm1hbCBzdGF0 ZTogcmVwcmVzZW50cyB0aGUgbm9ybWFsIG9wZXJhdGlvbiBjb25kaXRpb25zIG9mIHRoZSBub2Rl LAogICBpLmUuIG5vIGNvbmdlc3Rpb24KCgoKV2VzdGJlcmcsIGV0IGFsLiAgICAgICAgICBFeHBp cmVzIE1heSAxNywgMjAwOCAgICAgICAgICAgICAgICAgW1BhZ2UgMTldCgwKSW50ZXJuZXQtRHJh ZnQgICAgICAgICAgICAgICAgICAgTEMtUENOICAgICAgICAgICAgICAgICAgICBOb3ZlbWJlciAy MDA3CgoKICAgRmxvdyBUZXJtaW5hdGlvbiBzdGF0ZTogaXQgcmVwcmVzZW50cyB0aGUgc3RhdGUg cmVsYXRlZCB0byBhIGNlcnRhaW4KICAgUEhCIHdoZW4gdGhlIFBDTi1pbnRlcmlvci1ub2RlIGlz IHNldmVyZWx5IGNvbmdlc3RlZCBhbmQgb25nb2luZwogICBmbG93cyBuZWVkIHRvIGJlIHRlcm1p bmF0ZWQgaW4gb3JkZXIgdG8gc29sdmUgdGhpcyBjb25nZXN0aW9uLgoKICAgQWRtaXNzaW9uIENv bnRyb2wgc3RhdGU6IHN0YXRlIHdoZXJlIHRoZSBsb2FkIGlzIHJlbGF0aXZlbHkgaGlnaCwKICAg Y2xvc2UgdG8gdGhlIGxldmVsIHdoZW4gcHJlLWNvbmdlc3Rpb24gY2FuIG9jY3VyCgogICBldmVu dCBBOiB0aGlzIGV2ZW50IG9jY3VycyB3aGVuIHRoZSBpbmNvbWluZyBtZWFzdXJlZCBQSEIgcmF0 ZSBpcwogICBoaWdoZXIgdGhhbiB0aGUgYWRtaXNzaW9uIGNvbnRyb2wgdGhyZXNob2xkLCBpLmUu LCBQQ05fbG93ZXJfcmF0ZSwKICAgc2VlIFNlY3Rpb24gNC4xLCA0LjMuCgogICBldmVudCBCOiB0 aGlzIGV2ZW50IG9jY3VycyB3aGVuIHRoZSBpbmNvbWluZyBtZWFzdXJlZCBQSEIgcmF0ZSBpcwog ICBoaWdoZXIgdGhhbiB0aGUgZmxvdyB0ZXJtaW5hdGlvbiB0aHJlc2hvbGQsIGkuZS4sIFBDTl91 cHBlcl9yYXRlLgoKICAgZXZlbnQgQzogdGhpcyBldmVudCBvY2N1cnMgd2hlbiB0aGUgaW5jb21p bmcgbWVhc3VyZWQgUEhCIHJhdGUgaXMKICAgbG93ZXIgb3IgZXF1YWwgdG8gdGhlIGFkbWlzc2lv biBjb250cm9sIHRocmVzaG9sZCwgaS5lLiwKICAgUENOX2xvd2VyX3JhdGUuCgogICBldmVudCBE OiB0aGlzIGV2ZW50IG9jY3VycyB3aGVuIHRoZSBpbmNvbWluZyBtZWFzdXJlZCBQSEIgcmF0ZSBp cwogICBsb3dlciBvciBlcXVhbCB0byB0aGUgZmxvdyB0ZXJtaW5hdGlvbiB0aHJlc2hvbGQsIFBD Tl91cHBlcl9yYXRlLgoKICAgRHVyaW5nIGZsb3cgdGVybWluYXRpb24gdGhlIFBDTi1pbnRlcmlv ci1ub2RlIGNhbGN1bGF0ZXMsIHBlciB0cmFmZmljCiAgIGNsYXNzIChQSEIpLCB0aGUgaW5jb21p bmcgbWVhc3VyZWQgUEhCIHJhdGUgdGhhdCBpcyBhYm92ZSB0aGUgZmxvdwogICB0ZXJtaW5hdGlv biB0aHJlc2hvbGQsIGkuZS4sIGRlbm90ZWQgaW4gU2VjdGlvbiAzLjMgYXMKICAgUENOX3VwcGVy X3JhdGUsIGRlbm90ZWQgYXMgc2lnbmFsZWRfb3ZlcmxvYWRfcmF0ZSwgaW4gdGhlIGZvbGxvd2lu ZwogICB3YXk6CgogICBvICBBIFBDTi1pbnRlcmlvci1ub2RlIHRoYXQgb3BlcmF0ZXMgaW4gRmxv dyBUZXJtaW5hdGlvbiBzdGF0ZSBzaG91bGQKICAgICAgdGFrZSBpbnRvIGFjY291bnQgdGhhdCBw YWNrZXRzIG1pZ2h0IGJlIGRyb3BwZWQuICBUaGVyZWZvcmUsCiAgICAgIGJlZm9yZSBxdWV1aW5n IGFuZCBldmVudHVhbGx5IGRyb3BwaW5nIHBhY2tldHMsIHRoZSBQQ04taW50ZXJpb3ItCiAgICAg IG5vZGUgc2hvdWxkIGNvdW50LCBwZXIgaW50ZXJ2YWwgVCwgdGhlIHRvdGFsIG51bWJlciBvZiBv cmlnaW5hbAogICAgICBEU0NQLCBQQ05fbWFya2luZyBEU0NQIGFuZCBQQ05fQWZmZWN0ZWRfbWFy a2luZyBEU0NQIGJ5dGVzCiAgICAgIHJlY2VpdmVkIGJ5IHRoZSBQQ04taW50ZXJpb3Itbm9kZSB0 aGF0IG9wZXJhdGVzIGluIEZsb3cKICAgICAgVGVybWluYXRpb24gc3RhdGUuICBEZW5vdGUgdGhp cyBudW1iZXIgYXMgdG90YWxfcmVjZWl2ZWRfYnl0ZXMuCiAgICAgIE5vdGUgdGhhdCB0aGVyZSBh cmUgc2l0dWF0aW9ucyB3aGVuIG1vcmUgdGhhbiBvbmUgUENOLWludGVyaW9yLQogICAgICBub2Rl cyBpbiB0aGUgc2FtZSBjb21tdW5pY2F0aW9uIHBhdGggYmVjb21lIHNldmVyZSBjb25nZXN0ZWQg YW5kCiAgICAgIGNhbiBvcGVyYXRlIGluIEZsb3cgVGVybWluYXRpb24gc3RhdGUuICBUaGVyZWZv cmUsIGFueSBQQ04tCiAgICAgIGludGVyaW9yLW5vZGUgbG9jYXRlZCBiZWhpbmQgYSBQQ04taW50 ZXJpb3Itbm9kZSB0aGF0IG9wZXJhdGVzIGluCiAgICAgIEZsb3cgVGVybWluYXRpb24gc3RhdGUs IG1heSByZWNlaXZlIFBDTl9tYXJraW5nIERTQ1AgYW5kCiAgICAgIFBDTl9BZmZlY3RlZF9tYXJr aW5nIERTQ1AgbWFya2VkIGJ5dGVzLgoKICAgbyAgYmVmb3JlIHF1ZXVpbmcgYW5kIGV2ZW50dWFs bHkgZHJvcHBpbmcgdGhlIHBhY2tldHMsIGF0IHRoZSBlbmQgb2YKICAgICAgZWFjaCBtZWFzdXJl bWVudCBpbnRlcnZhbCBvZiBUIHNlY29uZHMsIGNhbGN1bGF0ZSB0aGUgY3VycmVudAogICAgICBl c3RpbWF0ZWQgb3ZlcmxvYWRlZCByYXRlLCBzYXkgbWVhc3VyZWRfb3ZlcmxvYWRfcmF0ZSwgYnkg dXNpbmcKICAgICAgdGhlIHNhbWUgbWV0aG9kIGFzIGRlc3JpYmVkIGluIFNlY3Rpb24gNC4xLjIu LCBzZWUgYmVsb3c6CiAgICAgIG1lYXN1cmVkX292ZXJsb2FkX3JhdGUgPSAoKHRvdGFsX3JlY2Vp dmVkX2J5dGVzKSAvIFQpIC0KICAgICAgUENOX3VwcGVyX3JhdGUpCgoKCgpXZXN0YmVyZywgZXQg YWwuICAgICAgICAgIEV4cGlyZXMgTWF5IDE3LCAyMDA4ICAgICAgICAgICAgICAgICBbUGFnZSAy MF0KDApJbnRlcm5ldC1EcmFmdCAgICAgICAgICAgICAgICAgICBMQy1QQ04gICAgICAgICAgICAg ICAgICAgIE5vdmVtYmVyIDIwMDcKCgogICBIb3dldmVyLCB0aGUgbWFpbiBkaWZmZXJlbmNlIGJl dHdlZW4gY2FsY3VsYXRpbmcgdGhlIHNpZ25hbGVkCiAgIG92ZXJsb2FkX3JhdGUgZHVyaW5nIEFk bWlzc2lvbiBDb250cm9sIGFuZCBGbG93IFRlcm1pbmF0aW9uIGlzIHRoYXQKICAgZHVyaW5nIHRo ZSBmbG93IHRlcm1pbmF0aW9uIHNpdHVhdGlvbiBzaW5jZSBtYXJraW5nIGlzIGRvbmUgaW4gUENO LQogICBpbnRlcmlvci1ub2RlcywgdGhlIGRlY2lzaW9ucyBhcmUgbWFkZSBhdCBQQ04tZWdyZXNz LW5vZGVzLCBhbmQKICAgdGVybWluYXRpb24gb2YgZmxvd3MgYXJlIHBlcmZvcm1lZCBieSBQQ04t aW5ncmVzcy1ub2RlcywgdGhlcmUgaXMgYQogICBzaWduaWZpY2FudCBkZWxheSB1bnRpbCB0aGUg b3ZlcmxvYWQgaW5mb3JtYXRpb24gaXMgbGVhcm5lZCBieSB0aGUKICAgUENOLWluZ3Jlc3Mtbm9k ZXMsIHNlZSBTZWN0aW9uIDYgb2YgW0NzVGEwNV0uICBUaGUgZGVsYXkgY29uc2lzdHMgb2YKICAg dGhlIHRyaXAgdGltZSBvZiBkYXRhIHBhY2tldHMgZnJvbSB0aGUgUENOLWludGVyaW9yLW5vZGUg dGhhdAogICBvcGVyYXRlcyBpbiBGbG93IFRlcm1pbmF0aW9uIHN0YXRlIHRvIHRoZSBQQ04tZWdy ZXNzLW5vZGUsIHRoZQogICBtZWFzdXJlbWVudCBpbnRlcnZhbCwgaS5lLiwgVCwgYW5kIHRoZSB0 cmlwIHRpbWUgb2YgdGhlIG5vdGlmaWNhdGlvbgogICBzaWduYWxpbmcgbWVzc2FnZXMgZnJvbSBQ Q04tZWdyZXNzLW5vZGUgdG8gUENOLWluZ3Jlc3Mtbm9kZS4KICAgTW9yZW92ZXIsIHVudGlsIHRo ZSBvdmVybG9hZCBkZWNyZWFzZXMgYXQgdGhlIFBDTi1pbnRlcmlvci1ub2RlIHRoYXQKICAgb3Bl cmF0ZXMgaW4gRmxvdyBUZXJtaW5hdGlvbiBzdGF0ZSwgYW4gYWRkaXRpb25hbCB0cmlwIHRpbWUg ZnJvbSB0aGUKICAgUENOLWluZ3Jlc3Mtbm9kZSB0byB0aGlzIFBDTi1pbnRlcmlvci1ub2RlIG11 c3QgZXhwaXJlLiAgVGhpcyBpcwogICBiZWNhdXNlIGltbWVkaWF0ZWx5IGJlZm9yZSByZWNlaXZp bmcgdGhlIGZsb3cgdGVybWluYXRpb24KICAgbm90aWZpY2F0aW9uLCB0aGUgUENOLWluZ3Jlc3Mt bm9kZSBtYXkgaGF2ZSBzZW50IG91dCBwYWNrZXRzIGluIHRoZQogICBmbG93cyB0aGF0IHdlcmUg c2VsZWN0ZWQgZm9yIHRlcm1pbmF0aW9uLiAgVGhhdCBpcywgYSB0ZXJtaW5hdGVkIGZsb3cKICAg bWF5IGNvbnRyaWJ1dGUgdG8gY29uZ2VzdGlvbiBmb3IgYSB0aW1lIGxvbmdlciB0aGF0IGlzIHRh a2VuIGZyb20gdGhlCiAgIFBDTi1pbmdyZXNzLW5vZGUgdG8gdGhlIFBDTi1pbnRlcmlvci1ub2Rl LiAgV2l0aG91dCBjb25zaWRlcmluZyB0aGUKICAgYWJvdmUsIFBDTi1pbnRlcmlvci1ub2RlcyB3 b3VsZCBjb250aW51ZSBtYXJraW5nIHRoZSBwYWNrZXRzIHVudGlsCiAgIHRoZSBtZWFzdXJlZCB1 dGlsaXphdGlvbiBmYWxscyBiZWxvdyB0aGUgZmxvdyB0ZXJtaW5hdGlvbiB0aHJlc2hvbGQuCiAg IEluIHRoaXMgd2F5LCBhdCB0aGUgZW5kIG1vcmUgZmxvd3Mgd2lsbCBiZSB0ZXJtaW5hdGVkIHRo YW4gbmVjZXNzYXJ5LAogICBpLmUuLCBhbiBvdmVyLXJlYWN0aW9uIHRha2VzIHBsYWNlLiAgW0Nz VGEwNV0gcHJvdmlkZXMgYSBzb2x1dGlvbiB0bwogICB0aGlzIHByb2JsZW0sIHdoZXJlIHRoZSBQ Q04taW50ZXJpb3Itbm9kZXMgdXNlIGEgc2xpZGluZyB3aW5kb3cKICAgbWVtb3J5IHRvIGtlZXAg dHJhY2sgb2YgdGhlIHNpZ25hbGluZyBvdmVybG9hZCBpbiBhIGNvdXBsZSBvZgogICBwcmV2aW91 cyBtZWFzdXJlbWVudCBpbnRlcnZhbHMuICBBdCB0aGUgZW5kIG9mIGEgbWVhc3VyZW1lbnQKICAg aW50ZXJ2YWxzLCBULCBiZWZvcmUgZW5jb2RpbmcgYW5kIHNpZ25hbGluZyB0aGUgb3ZlcmxvYWRl ZCByYXRlIGFzCiAgIFBDTl9tYXJraW5nIERTQ1AgcGFja2V0cywgdGhlIGFjdHVhbCBvdmVybG9h ZCBpcyBkZWNyZWFzZWQgd2l0aCB0aGUKICAgc3VtIG9mIGFscmVhZHkgc2lnbmFsZWQgb3Zlcmxv YWQgc3RvcmVkIGluIHRoZSBzbGlkaW5nIHdpbmRvdyBtZW1vcnksCiAgIHNpbmNlIHRoYXQgb3Zl cmxvYWQgaXMgYWxyZWFkeSBiZWluZyBoYW5kbGVkIGluIHRoZSBmbG93IHRlcm1pbmF0aW9uCiAg IGhhbmRsaW5nIGNvbnRyb2wgbG9vcC4gIFRoZSBzbGlkaW5nIHdpbmRvdyBtZW1vcnkgY29uc2lz dHMgb2YgYW4KICAgaW50ZWdlciBudW1iZXIgb2YgY2VsbHMsIGkuZSwgbiA9IG1heGltdW0gbnVt YmVyIG9mIGNlbGxzLgogICBHdWlkZWxpbmVzIGZvciBjb25maWd1cmluZyB0aGUgc2xpZGluZyB3 aW5kb3cgcGFyYW1ldGVycyBhcmUgZ2l2ZW4gaW4KICAgW0NzVGEwNV0uCgogICBBdCB0aGUgZW5k IG9mIGVhY2ggbWVhc3VyZW1lbnQgaW50ZXJ2YWwsIHRoZSBuZXdlc3QgY2FsY3VsYXRlZAogICBv dmVybG9hZCBpcyBwdXNoZWQgaW50byB0aGUgbWVtb3J5LCBhbmQgdGhlIG9sZGVzdCBjZWxsIGlz IGRyb3BwZWQuCgogICBJZiBNaSBpcyB0aGUgb3ZlcmxvYWRfcmF0ZSBzdG9yZWQgaW4gaXRoIG1l bW9yeSBjZWxsIChpID0gWzEuLm5dKSwKICAgdGhlbiBhdCB0aGUgZW5kIG9mIGV2ZXJ5IG1lYXN1 cmVtZW50IGludGVydmFsLCB0aGUgb3ZlcmxvYWQgcmF0ZSB0aGF0CiAgIGlzIHNpZ25hbGVkIHRv IHRoZSBQQ04tZWdyZXNzLW5vZGUsIGkuZS4sIHNpZ25hbGVkX292ZXJsb2FkX3JhdGUgaXMKICAg Y2FsY3VsYXRlZCBhcyBmb2xsb3dzOgoKCgoKCgoKCgpXZXN0YmVyZywgZXQgYWwuICAgICAgICAg IEV4cGlyZXMgTWF5IDE3LCAyMDA4ICAgICAgICAgICAgICAgICBbUGFnZSAyMV0KDApJbnRlcm5l dC1EcmFmdCAgICAgICAgICAgICAgICAgICBMQy1QQ04gICAgICAgICAgICAgICAgICAgIE5vdmVt YmVyIDIwMDcKCgogICAgU3VtX01pID0wCiAgICBGb3IgaSA9MSB0byBuCiAgICAgewogICAgICBT dW1fTWkgPSBTdW1fTWkgKyBNaQogICAgIH0KCiAgICBzaWduYWxlZF9vdmVybG9hZF9yYXRlID0g bWVhc3VyZWRfb3ZlcmxvYWRfcmF0ZSAtIFN1bV9NaSwKCiAgICB3aGVyZSBTdW1fTWkgaXMgY2Fs Y3VsYXRlZCBhcyBhYm92ZS4KCiAgTmV4dCwgdGhlIHNsaWRpbmcgbWVtb3J5IGlzIHVwZGF0ZWQg YXMgZm9sbG93czoKCiAgICBmb3IgaSA9IDEuLihuLTEpOiBNaSA8IC0gTWkrMQogICAgICBNbiA8 IC0gc2lnbmFsZWRfb3ZlcmxvYWRfcmF0ZQoKICBUaGUgYnl0ZXMgdGhhdCBoYXZlIHRvIGJlIHJl bWFya2VkIHRvIHNhdGlzZnkgdGhlIHNpZ25hbGVkIG92ZXJsb2FkCiAgcmF0ZTogc2lnbmFsZWRf cmVtYXJrZWRfYnl0ZXMsIGFyZSBjYWxjdWxhdGVkIGFzIGZvbGxvd3M6CgogICAgSUYgKG1lYXN1 cmVkIFBIQiByYXRlID4gUENOX3VwcGVyX3JhdGUpCiAgICBUSEVOCiAgICB7CiAgICAgIElGIChp bmNvbWluZ19QQ05fbWFya2luZ19yYXRlIDw+IDApIEFORAogICAgICAgICAgKGluY29taW5nX1BD Tl9tYXJraW5nX3JhdGUgPTwgVGVybWluYXRpb25fb2Zmc2V0X3JhdGUpCiAgICAgIFRIRU4KICAg ICAgICB7IHNpZ25hbGVkX3JlbWFya2VkX2J5dGVzID0KICAgICAgICAgICAgICAoKHNpZ25hbGVk X292ZXJsb2FkX3JhdGUgLQogICAgICAgICAgICAgICBpbmNvbWluZ19QQ05fbWFya2luZ19yYXRl KSAqIFQpIC8gTgogICAgICAgIH0KICAgICAgRUxTRSBJRiAoaW5jb21pbmdfUENOX21hcmtpbmdf cmF0ZSA9MCkKICAgICAgVEhFTiBzaWduYWxlZF9yZW1hcmtlZF9ieXRlcyA9IHNpZ25hbGVkX292 ZXJsb2FkX3JhdGUgKiBUIC8gTgogICAgICBFTFNFIElGIChpbmNvbWluZ19QQ05fbWFya2luZ19y YXRlID4KICAgICAgICAgICAgICAgICAgVGVybWluYXRpb25fb2Zmc2V0X3JhdGUpCiAgICAgIFRI RU4gc2lnbmFsZWRfcmVtYXJrZWRfYnl0ZXMgPQogICAgICAgICAgICAgICAgKChzaWduYWxlZF9v dmVybG9hZF9yYXRlIC0gVGVybWluYXRpb25fb2Zmc2V0X3JhdGUpKlQpL04KICAgIH0KCiAgIFRo ZSBzaWduYWxfcmVtYXJrZWRfYnl0ZXMgcmVwcmVzZW50cyBhbHNvIHRoZSBudW1iZXIgb2YgdGhl IG91dGdvaW5nCiAgIHBhY2tldHMgKGFmdGVyIHRoZSBkcm9wcGluZyBzdGFnZSkgdGhhdCBtdXN0 IGJlIHJlbWFya2VkLCBkdXJpbmcgZWFjaAogICBtZWFzdXJlbWVudCBpbnRlcnZhbCBULCBieSBh IG5vZGUgd2hlbiBvcGVyYXRlcyBpbiBmbG93IHRlcm1pbmF0aW9uCiAgIHN0YXRlLgoKICAgTm90 ZSB0aGF0IGluIG9yZGVyIHRvIHByb2Nlc3MgYW4gb3ZlcmxvYWQgc2l0dWF0aW9uIGhpZ2hlciB0 aGFuIDEwMCUKICAgb2YgdGhlIG1haW50YWluZWQgUENOX3VwcGVyX3JhdGUgYWxsIHRoZSBub2Rl cyB3aXRoaW4gdGhlIFBDTiBkb21haW4KICAgbXVzdCBiZSBjb25maWd1cmVkIGFuZCBtYWludGFp biBhIHNjYWxpbmcgcGFyYW1ldGVyLCBlLmcuLCBOIHVzZWQgaW4KICAgdGhlIGFib3ZlIGVxdWF0 aW9uLCB3aGljaCBpbiBjb21iaW5hdGlvbiB3aXRoIHRoZSBQQ05fbWFya2luZyBEU0NQCiAgIGVu Y29kZWQgYnl0ZXMsIGUuZy4sIHNpZ25hbGVkX3JlbWFya2VkX2J5dGVzLCBzdWNoIGEgaGlnaCBv dmVybG9hZAogICBzaXR1YXRpb24gY2FuIGJlIGNhbGN1bGF0ZWQgYW5kIHJlcHJlc2VudGVkLiAg TiBjYW4gYmUgZXF1YWwgb3IKICAgaGlnaGVyIHRoYW4gMS4KCgoKV2VzdGJlcmcsIGV0IGFsLiAg ICAgICAgICBFeHBpcmVzIE1heSAxNywgMjAwOCAgICAgICAgICAgICAgICAgW1BhZ2UgMjJdCgwK SW50ZXJuZXQtRHJhZnQgICAgICAgICAgICAgICAgICAgTEMtUENOICAgICAgICAgICAgICAgICAg ICBOb3ZlbWJlciAyMDA3CgoKICAgTm90ZSB0aGF0IHdoZW4gaW5jb21pbmcgcmVtYXJrZWQgYnl0 ZXMgYXJlIGRyb3BwZWQsIHRoZSBvcGVyYXRpb24gb2YKICAgdGhlIGZsb3cgdGVybWluYXRpb24g YWxnb3JpdGhtIG1heSBiZSBhZmZlY3RlZCwgZS5nLiwgdGhlIGFsZ29yaXRobQogICBtYXkgYmVj b21lIGluIGNlcnRhaW4gc2l0dWF0aW9ucyBzbG93ZXIuICBBbiBpbXBsZW1lbnRhdGlvbiBvZiB0 aGUKICAgYWxnb3JpdGhtIG1heSBhc3N1cmUgYXMgbXVjaCBhcyBwb3NzaWJsZSB0aGF0IHRoZSBp bmNvbWluZyBtYXJrZWQKICAgYnl0ZXMgYXJlIG5vdCBkcm9wcGVkLiAgVGhpcyBjb3VsZCBmb3Ig ZXhhbXBsZSBiZSBhY2NvbXBsaXNoZWQgYnkKICAgdXNpbmcgZGlmZmVyZW50IGRyb3BwaW5nIHJh dGUgdGhyZXNob2xkcyBmb3IgbWFya2VkIGFuZCB1bm1hcmtlZAogICBieXRlcywgc2VlIFNlY3Rp b24gMy4zLgoKICAgQWxsIHRoZSBvdXRnb2luZyBwYWNrZXRzIHRoYXQgYXJlIG5vdCBtYXJrZWQg KGkuZS4sIGJ5IHVzaW5nIHRoZQogICBQQ05fbWFya2luZyBEU0NQKSBoYXZlIHRvIGJlIHJlbWFy a2VkIHVzaW5nIHRoZSBQQ05fQWZmZWN0ZWRfbWFya2luZwogICBEU0NQLgoKNC4yLjMuICBPcGVy YXRpb24gaW4gdGhlIFBDTi1lZ3Jlc3Mtbm9kZXMKCiAgIFdoZW4gdGhlIG9wZXJhdGlvbiBzdGF0 ZSBvZiB0aGUgaW5ncmVzcy9lZ3Jlc3MgcGFpciBhZ2dyZWdhdGUgaW4gdGhlCiAgIFBDTl9lZ3Jl c3Nfbm9kZSBpcyB0aGUgZmxvdyB0ZXJtaW5hdGlvbiwgc2VlIEZpZ3VyZSA0LCB0aGVuIHRoZQog ICBpbXBsZW1lbnRhdGlvbiBvZiB0aGlzIGFsZ29yaXRobSBpcyBhY2NvbXBsaXNoZWQgaW4gdGhl IGZvbGxvd2luZwogICB3YXkuCgogICBUaGUgUENOLWVncmVzcy1ub2RlIG5vZGUgYXBwbGllcyBh IHByZWRlZmluZWQgcG9saWN5IHRvIHNvbHZlIHRoZQogICBmbG93IHRlcm1pbmF0aW9uIHNpdHVh dGlvbiwgYnkgc2VsZWN0aW5nIGEgbnVtYmVyIG9mIGludGVyLWRvbWFpbgogICAoZW5kLXRvLWVu ZCkgZmxvd3MgdGhhdCBzaG91bGQgYmUgdGVybWluYXRlZCwgb3IgZm9yd2FyZGVkIGluIGEgbG93 ZXIKICAgcHJpb3JpdHkgcXVldWUuCgogICBTb21lIGZsb3dzLCBiZWxvbmdpbmcgdG8gdGhlIHNh bWUgUEhCIHRyYWZmaWMgY2xhc3MgbWlnaHQgZ2V0IG90aGVyCiAgIHByaW9yaXR5IHRoYW4gb3Ro ZXIgZmxvd3MgYmVsb25naW5nIHRvIHRoZSBzYW1lIFBIQiB0cmFmZmljIGNsYXNzLgogICBJdCBp cyBjb25zaWRlcmVkIHRoYXQgdGhpcyBkaWZmZXJlbmNlIGluIHByaW9yaXR5IGNhbiBiZSBub3Rp ZmllZCBieQogICBhIHNpZ25hbGxpbmcgcHJvdG9jb2wgYW5kIHRoYXQgdGhlIGVkZ2VzIGNhbiBz dG9yZSBhbmQgbWFpbnRhaW4gdGhlCiAgIHByaW9yaXR5IGluZm9ybWF0aW9uIHJlbGV0ZWQgdG8g ZWFjaCBvZiB0aGUgZW5kLXRvLWVuZCBmbG93cy4gIFRoZQogICB0ZXJtaW5hdGVkIGZsb3dzIGFy ZSBzZWxlY3RlZCBmcm9tIHRoZSBmbG93cyBoYXZpbmcgdGhlIHNhbWUgUEhCCiAgIHRyYWZmaWMg Y2xhc3MgYXMgdGhlIFBIQiBvZiB0aGUgbWFya2VkIChhcyBQQ05fbWFya2luZyBEU0NQKSBhbmQK ICAgUENOX0FmZmVjdGVkX21hcmtpbmcgRFNDUCAod2hlbiBhcHBsaWVkIGluIHRoZSBjb21wbGV0 ZSBQQ04gZG9tYWluKQogICBwYWNrZXRzIGFuZCB0aGF0IGFyZSBiZWxvbmdpbmcgdG8gdGhlIHNh bWUgaW5ncmVzcy9lZ3Jlc3MgcGFpcgogICBhZ2dyZWdhdGUuCgogICBGb3IgZmxvd3MgYXNzb2Np YXRlZCB3aXRoIHRoZSBzYW1lIFBIQiB0cmFmZmljIGNsYXNzIHRoZSBwcmlvcml0eSBvZgogICB0 aGUgZmxvdyBwbGF5cyBhIHNpZ25pZmljYW50IHJvbGUuICBBbiBleGFtcGxlIG9mIGNhbGN1bGF0 aW5nIHRoZQogICBudW1iZXIgb2YgZmxvd3MgYXNzb2NpYXRlZCB3aXRoIGVhY2ggcHJpb3JpdHkg Y2xhc3MgdGhhdCBoYXZlIHRvIGJlCiAgIHRlcm1pbmF0ZWQgaXMgZGVzY3JpYmVkIGJlbG93LgoK ICAgVGhlIHN0YXRlcyBvZiBvcGVyYXRpb24gaW4gUENOLWVncmVzcy1ub2RlcyBhcmUgc2ltaWxh ciB0byB0aGUgb25lcwogICBkZXNjcmliZWQgaW4gU2VjdGlvbiA0LjIuMi4gIFRoZSBkZWZpbml0 aW9uIG9mIHRoZSBldmVudHMsIHNlZSBiZWxvdywKICAgaXMgaG93ZXZlciBkaWZmZXJlbnQgdGhh biB0aGUgZGVmaW5pdGlvbiBvZiB0aGUgZXZlbnRzIGdpdmVuIGluCiAgIEZpZ3VyZSA0LgoKICAg byAgZXZlbnQgQTogdGhlIFBDTi1lZ3Jlc3Mtbm9kZSBtZWFzdXJlcyB0aGUgcmF0ZSBvZiB0aGUg aW5jb21pbmcKICAgICAgIlBDTl9tYXJraW5nIiBlbmNvZGVkIHBhY2tldHMsIGkuZS4sIGluY29t aW5nX1BDTl9tYXJraW5nX3JhdGUsCiAgICAgIGFuZCBjb21wYXJlIGl0IHdpdGggYSBwcmVkZWZp bmVkIFBDTl9sb3dlcl9yYXRlX2VncmVzcyBhbmQgdG8gYQoKCgpXZXN0YmVyZywgZXQgYWwuICAg ICAgICAgIEV4cGlyZXMgTWF5IDE3LCAyMDA4ICAgICAgICAgICAgICAgICBbUGFnZSAyM10KDApJ bnRlcm5ldC1EcmFmdCAgICAgICAgICAgICAgICAgICBMQy1QQ04gICAgICAgICAgICAgICAgICAg IE5vdmVtYmVyIDIwMDcKCgogICAgICBQQ05fdXBwZXJfcmF0ZV9lZ3Jlc3MgaW4gdGhlIFBDTi0g ZWdyZXNzLW5vZGUsIHNlZSBTZWN0aW9uIDMuNC4KICAgICAgV2hlbiB0aGUgaW5jb21pbmdfUENO X21hcmtpbmdfcmF0ZSwgaXMgaGlnaGVyIHRoYW4gdGhlCiAgICAgIFBDTl9sb3dlcl9yYXRlX2Vn cmVzcyBidXQgbG93ZXIgb3IgZXF1YWwgdG8gdGhlIGZsb3cgdGVybWluYXRpb24KICAgICAgdGhy ZXNob2xkLCBpLmUuLCBQQ05fdXBwZXJfcmF0ZV9lZ3Jlc3MgdGhlbiBldmVudF9BIGlzIGFjdGl2 YXRlZC4KCiAgIG8gIGV2ZW50IEI6IHRoaXMgZXZlbnQgaXMgYWN0aXZhdGVkIGRlcGVuZGluZyBv biB3aGljaCBvZiB0aGUKICAgICAgc29sdXRpb25zIGRlc2NyaWJlZCBpbiBTZWN0aW9uIDMuNCBh cmUgYXBwbGllZCBhdCB0aGUKICAgICAgUENOX2VncmVzc19ub2RlLiAgSWYgdGhlIFBDTl9BZmZl Y3RlZF9tYXJraW5nIGlzIHVzZWQgd2l0aGluIHdob2xlCiAgICAgIFBDTiBkb21haW4sIHRoZW4g ZXZlbnQgQiBvY2N1cnMgd2hlbiB0aGUgUENOX2VncmVzc19ub2RlIHJlY2VpdmVzCiAgICAgIGF0 IGxlYXN0IG9uZSBwYWNrZXQgdGhhdCBpcyBhc3NvY2lhdGVkIHdpdGggdGhlIGluZ3Jlc3MvZWdy ZXNzCiAgICAgIGFnZ3JlZ2F0ZSBhbmQgaXMgUENOX0FmZmVjdGVkX21hcmtpbmcgZW5jb2RlZC4g IElmIHRoZQogICAgICBQQ05fQWZmZWN0ZWRfbWFya2luZyBpcyBub3QgdXNlZCB3aXRoaW4gd2hv bGUgUENOIGRvbWFpbiB0aGVuCiAgICAgIGV2ZW50IEIgaXMgYWN0aXZhdGVkIHdoZW4gdGhlIGlu Y29taW5nX1BDTl9tYXJraW5nX3JhdGUgcmVjZWl2ZWQKICAgICAgYnkgdGhlIFBDTi1lZ3Jlc3Mt IG5vZGUgaXMgaGlnaGVyIHRoYW4gdGhlIFBDTl91cHBlcl9yYXRlX2VncmVzcywKICAgICAgc2Vl IFNlY3Rpb24gMy40LgoKICAgbyAgZXZlbnQgQzogdGhpcyBldmVudCBvY2N1cnMgd2hlbiB0aGUg aW5jb21pbmdfUENOX21hcmtpbmdfcmF0ZQogICAgICByZWNlaXZlZCBieSB0aGUgUENOLWVncmVz cy1ub2RlIGlzIGxvd2VyIG9yIGVxdWFsIHRvCiAgICAgIFBDTl9sb3dlcl9yYXRlX2VncmVzcywg c2VlIFNlY3Rpb24gMy40LgoKICAgbyAgZXZlbnQgRDogdGhpcyBldmVudCBpcyBhY3RpdmF0ZWQg ZGVwZW5kaW5nIG9uIHdoaWNoIG9mIHRoZQogICAgICBzb2x1dGlvbnMgZGVzY3JpYmVkIGluIFNl Y3Rpb24gMy40IGFyZSBhcHBsaWVkIGF0IHRoZQogICAgICBQQ05fZWdyZXNzX25vZGUuICBJZiB0 aGUgUENOX0FmZmVjdGVkX21hcmtpbmcgaXMgdXNlZCB3aXRoaW4gd2hvbGUKICAgICAgUENOIGRv bWFpbiwgdGhlbiBldmVudCBEIG9jY3VycyB3aGVuIHRoZSBQQ05fZWdyZXNzX25vZGUgZG9lcyBu b3QKICAgICAgcmVjZWl2ZXMgYW55IFBDTl9hZmZlY3RlZF9tYXJrZWQgcGFja2V0cyB3aXRoaW4g YSBwcmVkZWZpbmVkCiAgICAgIGFtb3VudCBvZiB0aW1lLCBlLmcuLCBvbmUgbWVhc3VyZW1lbnQg cGVyaW9kLiAgSWYgdGhlCiAgICAgIFBDTl9BZmZlY3RlZF9tYXJraW5nIGlzIG5vdCB1c2VkIHdp dGhpbiB3aG9sZSBQQ04gZG9tYWluIHRoZW4KICAgICAgZXZlbnQgRCBvY2N1cnMgd2hlbiB0aGUg aW5jb21pbmdfUENOX21hcmtpbmdfcmF0ZSByZWNlaXZlZCBieSB0aGUKICAgICAgUENOLSBlZ3Jl c3Mtbm9kZSBpcyBsb3dlciBvciBlcXVhbCB0byBQQ05fdXBwZXJfcmF0ZV9lZ3Jlc3MsIHNlZQog ICAgICBTZWN0aW9uIDMuNC4KCiAgIEFuIGV4YW1wbGUgb2YgdGhlIGFsZ29yaXRobSBmb3IgY2Fs Y3VsYXRpb24gb2YgdGhlIG51bWJlciBvZiBmbG93cwogICBhc3NvY2lhdGVkIHdpdGggZWFjaCBw cmlvcml0eSBjbGFzcyB0aGF0IGhhdmUgdG8gYmUgdGVybWluYXRlZCBpcwogICBleHBsYWluZWQg YnkgdGhlIHBzZXVkb2NvZGUgYmVsb3cuICBGaXJzdCwgd2hlbiB0aGUgUENOLWVncmVzcy1ub2Rl CiAgIG9wZXJhdGVzIGluIHRoZSBmbG93IHRlcm1pbmF0aW9uIHN0YXRlIHRoZW4gdGhlIHRvdGFs IGFtb3VudCBvZgogICByZW1hcmtlZCAoUENOX21hcmtpbmcgRFNDUCBtYXJrZWQpIHJhdGUsIHBl ciBpbmdyZXNzL2VncmVzcyBwYWlyCiAgIHJlc2VydmF0aW9uIGFnZ3JlZ2F0ZSwgYXNzb2NpYXRl ZCB3aXRoIHRoZSBQSEIgdHJhZmZpYyBjbGFzcywgc2F5CiAgIGluY29taW5nX1BDTl9tYXJraW5n X3JhdGUsIGlzIGNhbGN1bGF0ZWQuICBUaGlzIHJhdGUgcmVwcmVzZW50cyB0aGUKICAgZmxvdyB0 ZXJtaW5hdGlvbiBiYW5kd2lkdGgsIHBlciBpbmdyZXNzL2VncmVzcyBwYWlyLCB0aGF0IHNob3Vs ZCBiZQogICB0ZXJtaW5hdGVkLiAgTm90ZSB0aGF0IHRoZSBiZWxvdyBhbGdvcml0aG0gaXMgcGVy Zm9ybWVkIGZvciBlYWNoCiAgIGluZ3Jlc3MvZWdyZXNzIHBhaXIgcmVzZXJ2YXRpb24gYWdncmVn YXRlLiAgVGhlCiAgIGluY29taW5nX1BDTl9tYXJraW5nX3JhdGUgY2FuIGJlIHRoZW4gY2FsY3Vs YXRlZCBhcyBmb2xsb3dzOgoKICAgICBpbmNvbWluZ19QQ05fbWFya2luZ19yYXRlID0KICAgICAg IE4gKiBpbnB1dF9QQ05fbWFya2luZ19ieXRlcyAvIFQKCiAgIFRvIHByb3ZpZGUgcmVsaWFibGUg ZXN0aW1hdGlvbiBvZiB0aGUgZW5jb2RlZCBpbmZvcm1hdGlvbiBzZXZlcmFsCiAgIHRlY2huaXF1 ZXMgY2FuIGJlIHVzZWQsIHNlZSBbQXRMaTAxXSwgW0FkQ2EwM10sW1RoQ28wNF0sIFtBbkhhMDZd LgoKCgpXZXN0YmVyZywgZXQgYWwuICAgICAgICAgIEV4cGlyZXMgTWF5IDE3LCAyMDA4ICAgICAg ICAgICAgICAgICBbUGFnZSAyNF0KDApJbnRlcm5ldC1EcmFmdCAgICAgICAgICAgICAgICAgICBM Qy1QQ04gICAgICAgICAgICAgICAgICAgIE5vdmVtYmVyIDIwMDcKCgogICBJZiB0aGUgaW5jb21p bmdfY29uZ2VzdGlvbl9yYXRlIGlzIGhpZ2hlciB0aGFuIGEgcHJlY29uZmlndXJlZAogICBQQ05f dXBwZXJfcmF0ZV9lZ3Jlc3MsIHNlZSBTZWN0aW9uIDMuNCBhbmQgRmlndXJlIDQsIHRoZW4gaXQg aXMKICAgY29uc2lkZXJlZCB0aGF0IGF0IGxlYXN0IG9uZSBQQ04taW50ZXJpb3Itbm9kZSBsb2Nh dGVkIG9uIGEKICAgY29tbXVuaWNhdGlvbiBwYXRoIGJldHdlZW4gUENOLWluZ3Jlc3Mtbm9kZSBh bmQgUENOLWVncmVzcy1ub2RlIGlzCiAgIGNvbnNpZGVyZWQgdG8gb3BlcmF0ZSBpbiB0aGUgRmxv dyBUZXJtaW5hdGlvbiBzdGF0ZS4gIFRoZQogICBpbmNvbWluZ19QQ05fbWFya2luZ19yYXRlIGNh biBiZSBjYWxjdWxhdGVkIGFzIGZvbGxvd3M6CgogICAgIGluY29taW5nX1BDTl9tYXJraW5nX3Jh dGUgPQogICAgICAgTiAqIGlucHV0X1BDTl9tYXJraW5nX2J5dGVzIC8gVAoKICAgV2hlcmUsIGlu cHV0X1BDTl9tYXJraW5nX2J5dGVzIHJlcHJlc2VudHMgdGhlIG51bWJlciBvZiBtYXJrZWQgYnl0 ZXMKICAgdGhhdCBhcnJpdmUgYXQgdGhlIFBDTi1lZ3Jlc3Mtbm9kZSwgZHVyaW5nIG9uZSBtZWFz dXJlbWVudCBpbnRlcnZhbAogICBULCBOIGlzIGRlZmluZWQgYXMgaW4gU2VjdGlvbiAzLjMgYW5k IDQuMi4xLiAgVGhlIHRlcm0gZGVub3RlZCBhcwogICB0ZXJtaW5hdGVkX2JhbmR3aWR0aCBpcyBh IHRlbXBvcmFsIHZhcmlhYmxlIHJlcHJlc2VudGluZyB0aGUgdG90YWwKICAgYmFuZHdpZHRoIHRo YXQgaGF2ZSB0byBiZSB0ZXJtaW5hdGVkLCBiZWxvbmdpbmcgdG8gdGhlIHNhbWUgUEhCCiAgIHRy YWZmaWMgY2xhc3MuICBUaGUgdGVybWluYXRlX2Zsb3dfYmFuZHdpZHRoKHByaW9yaXR5X2NsYXNz KSBpcyB0aGUKICAgdG90YWwgb2YgYmFuZHdpZHRoIGFzc29jaWF0ZWQgd2l0aCBmbG93cyBvZiBw cmlvcml0eSBjbGFzcyBlcXVhbCB0bwogICBwcmlvcml0eV9jbGFzcy4gIFRoZSBwYXJhbWV0ZXIg cHJpb3JpdHlfY2xhc3MgaXMgYW4gaW50ZWdlcgogICBmdWxmaWxsaW5nCgogICAwIDwgcHJpb3Jp dHlfY2xhc3MgPTwgTWF4aW11bV9wcmlvcml0eS4KCiAgIE5vdGUgdGhhdCBpZiB0aGUgUENOIGRv bWFpbiBkb2VzIG5vdCBzdXBwb3J0IHByaW9yaXR5IGRpZmZlcmVudGlhdGlvbgogICB0aGVuIHRo ZSB2YXJpYWJsZSBNYXhpbXVtX3ByaW9yaXR5IFNIT1VMRCBiZSBlcXVhbCB0byAwLgoKICAgVGhl IGNhbGN1bGF0ZV90ZXJtaW5hdGVfZmxvd3MocHJpb3JpdHlfY2xhc3MpIGZ1bmN0aW9uIGRldGVy bWluZXMgdGhlCiAgIGZsb3dzIGZvciBhIGdpdmVuIHByaW9yaXR5IGNsYXNzIGFuZCBwZXIgUEhC IHRoYXQgaGFzIHRvIGJlCiAgIHRlcm1pbmF0ZWQuICBUaGlzIGZ1bmN0aW9uIGFsc28gY2FsY3Vs YXRlcyB0aGUgdGVybQogICBzdW1fYmFuZHdpZHRoX3Rlcm1pbmF0ZShwcmlvcml0eV9jbGFzcyks IHdoaWNoIGlzIHRoZSBzdW0gb2YgdGhlCiAgIGJhbmR3aXRoIGFzc29jaWF0ZWQgd2l0aCB0aGUg Zmxvd3MgdGhhdCB3aWxsIGJlIHRlcm1pbmF0ZWQuICBUaGUKICAgY29uc3RyYWludCBvZiBmaW5k aW5nIHRoZSB0b3RhbCBudW1iZXIgb2YgZmxvd3MgdGhhdCBoYXZlIHRvIGJlCiAgIHRlcm1pbmF0 ZWQgaXMgdGhhdCBzdW1fYmFuZHdpZHRoX3Rlcm1pbmF0ZShwcmlvcml0eV9jbGFzcyksIHNob3Vs ZCBiZQogICBzbWFsbGVyIG9yIGFwcHJveGltYXRlbGx5IGVxdWFsIHRvIHRoZSB2YXJpYWJsZQog ICB0ZXJtaW5hdGVfYmFuZHdpZHRoKHByaW9yaXR5X2NsYXNzKS4KCiAgICB0ZXJtaW5hdGVkX2Jh bmR3aWR0aCA9IDA7CiAgICBwcmlvcml0eV9jbGFzcyA9IDA7CiAgICB3aGlsZSB0ZXJtaW5hdGVk X2JhbmR3aWR0aCA8IGluY29taW5nX1BDTl9tYXJraW5nX3JhdGUKICAgIHsKICAgICAgdGVybWlu YXRlX2JhbmR3aWR0aChwcmlvcml0eV9jbGFzcykgPQogICAgICAgICBpbmNvbWluZ19QQ05fbWFy a2luZ19yYXRlIC0gdGVybWluYXRlZF9iYW5kd2lkdGgKICAgICAgY2FsY3VsYXRlX3Rlcm1pbmF0 ZV9mbG93cyhwcmlvcml0eV9jbGFzcyk7CiAgICAgIHRlcm1pbmF0ZWRfYmFuZHdpZHRoID0KICAg ICAgICAgc3VtX2JhbmR3aWR0aF90ZXJtaW5hdGUocHJpb3JpdHlfY2xhc3MpICsgdGVybWluYXRl ZF9iYW5kd2lkdGg7CiAgICAgIHByaW9yaXR5X2NsYXNzID0gcHJpb3JpdHlfY2xhc3MgKyAxOwog ICAgfQoKICAgRm9yIHRoZSBlbmQtdG8tZW5kIGZsb3dzIChzZXNzaW9ucykgdGhhdCBoYXZlIHRv IGJlIHRlcm1pbmF0ZWQsIHRoZQoKCgpXZXN0YmVyZywgZXQgYWwuICAgICAgICAgIEV4cGlyZXMg TWF5IDE3LCAyMDA4ICAgICAgICAgICAgICAgICBbUGFnZSAyNV0KDApJbnRlcm5ldC1EcmFmdCAg ICAgICAgICAgICAgICAgICBMQy1QQ04gICAgICAgICAgICAgICAgICAgIE5vdmVtYmVyIDIwMDcK CgogICBQQ04tZWdyZXNzLW5vZGUgZ2VuZXJhdGVzIGFuZCBzZW5kcyBub3RpZmljYXRpb24gbWVz c2FnZSB0byB0aGUgUENOLQogICBpbmdyZXNzLW5vZGUgdG8gaW5kaWNhdGUgdGhlIGZsb3cgdGVy bWluYXRpb24gaW4gdGhlIGNvbW11bmljYXRpb24KICAgcGF0aC4gIEZ1cnRoZXJtb3JlLCBmb3Ig dGhlIGFnZ3JlZ2F0ZWQgc2Vzc2lvbnMgdGhhdCBhcmUgYWZmZWN0ZWQsCiAgIHRoZSBQQ04tZWdy ZXNzLW5vZGUgc2VuZHMgd2l0aGluIGEgbm90aWZ5IG1lc3NhZ2UgdGhhdCBjb250YWlucyB0aGUK ICAgVG8gYmUgcmVsZWFzZWQgYmFuZHdpZHRoLCBhc3NvY2lhdGVkIHdpdGggdGhlIGFnZ3JlZ2F0 ZWQgcmVzZXJ2YXRpb24KICAgc3RhdGUuICBOb3RlIHRoYXQgUENOLWVncmVzcy1ub2RlIHNob3Vs ZCByZXN0b3JlIHRoZSBvcmlnaW5hbCBEU0NQCiAgIHZhbHVlcyBvZiB0aGUgcmVtYXJrZWQgcGFj a2V0cywgb3RoZXJ3aXNlIG11bHRpcGxlIGFjdGlvbnMgZm9yIHRoZQogICBzYW1lIGV2ZW50IG1p Z2h0IG9jY3VyLiAgSG93ZXZlciwgdGhpcyB2YWx1ZSBNQVkgYmUgbGVmdCBpbiBpdHMKICAgcmVt YXJraW5nIGZvcm0gaWYgdGhlcmUgaXMgYW4gU0xBIGFncmVlbWVudCBiZXR3ZWVuIGRvbWFpbnMg dGhhdCBhCiAgIGRvd25zdHJlYW0gZG9tYWluIGhhbmRsZXMgdGhlIHJlbWFya2luZyBwcm9ibGVt LgoKNC4zLiAgQWRtaXNzaW9uIGNvbnRyb2wgYmFzZWQgb24gcHJvYmluZyBmb3IgYmktZGlyZWN0 aW9uYWwgZmxvd3MKCiAgIFRoaXMgc2VjdGlvbiBkZXNjcmliZXMgdGhlIGFkbWlzc2lvbiBjb250 cm9sIHNjaGVtZSB0aGF0IHVzZXMgdGhlCiAgIGFkbWlzc2lvbiBjb250cm9sIGZ1bmN0aW9uIGJh c2VkIG9uIHByb2Jpbmcgd2hlbiBiaS1kaXJlY3Rpb25hbAogICByZXNlcnZhdGlvbnMgYXJlIHN1 cHBvcnRlZC4KClBDTi1pbmdyZXNzLW5vZGUgIFBDTi1pbnRlcmlvci1ub2RlICBQQ04taW50ZXJp b3Itbm9kZSAgIFBDTi1lZ3Jlc3Mtbm9kZQoKdXNlcnwgICAgICAgICAgICAgICAgfCAgICAgICAg ICAgICB8ICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgfApkYXRhfCAgICAgICAgICAgICAg ICB8ICAgICAgICAgICAgIHwgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICB8Ci0tLT58ICAg ICAgICAgICAgICAgIHwgdXNlciBkYXRhICAgfCAgICAgICAgICAgICAgfHVzZXIgZGF0YSAgICAg IHwKICAgIHwtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLT5TICgj bWFya2VkIGJ5dGVzKQogICAgfCAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgIHwgICAgICAg ICAgICAgIFMtLS0tLS0tLS0tLS0tLT58CiAgICB8ICAgICAgICAgICAgICAgIHwgICAgICAgICAg ICAgfCAgICAgICAgICAgICAgUygjdW5tYXJrZWQgYnl0ZXMpCiAgICB8ICAgICAgICAgICAgICAg IHwgICAgICAgICAgICAgfCAgICAgICAgICAgICAgUy0tLS0tLS0tLS0tLS0tPnwKICAgIHwgICAg ICAgICAgICAgICAgfCAgICAgICAgICAgICB8ICAgICAgICAgICAgICBTICAgICAgICAgICAgICAg fAogICAgfCAgICAgICAgICAgICAgICB8ICAgICAgICAgICBwcm9iZShyZS1tYXJrZWQgRFNDUCkg ICAgICAgICAgICB8CiAgICB8ICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgfCAgICAgICAg ICAgICAgUyAgICAgICAgICAgICAgIHwKICAgIHwtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLT5TICAgICAgICAgICAgICAgfAogICAgfCAgICAgICAgICAgICAgICB8 ICAgICAgICAgICAgIHwgICAgICAgICAgICAgIFMtLS0tLS0tLS0tLS0tLT58CiAgICB8ICAgICAg ICAgICAgICAgIHwgICAgICAgICAgICAgfCAgICAgICAgICAgICAgUyAgICAgICAgICAgICAgIHwK ICAgIHwgICAgICAgICAgICAgICAgfCAgICAgICAgICByZXNwb25zZSh1bnN1Y2Nlc3NmdWwpICAg ICAgICAgICB8CiAgICB8PC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLXwKICAgIHwgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICB8 ICAgICAgICAgICAgICBTICAgICAgICAgICAgICAgfAoKCiAgICAgICAgICAgIEZpZ3VyZSA1OiBB ZG1pc3Npb24gY29udHJvbCBiYXNlZCBvbiBwcm9iaW5nCiAgICAgICAgICAgIGZvciBiaS1kaXJl Y3Rpb25hbCBhZG1pc3Npb24gY29udHJvbCAocHJlLWNvbmdlc3Rpb24gb24KICAgICAgICAgICAg cGF0aCBmcm9tIFBDTi1pbmdyZXNzLW5vZGUgdG93YXJkcyBQQ04tZWdyZXNzLW5vZGUpCgogICBU aGlzIHByb2NlZHVyZSBpcyBzaW1pbGFyIHRvIHRoZSBhZG1pc3Npb24gY29udHJvbCBwcm9jZWR1 cmUKICAgZGVzY3JpYmVkIGluIFNlY3Rpb24gNC4xLCBmb3IgdGhlIHNpdHVhdGlvbiB0aGF0IHRo ZSBQQ04gZG9tYWluCiAgIHN1cHBvcnRzIHByb2JpbmcuICBUaGUgbWFpbiBkaWZmZXJlbmNlIGlz IHJlbGF0ZWQgdG8gdGhlIGxvY2F0aW9uIG9mCiAgIHRoZSBQQ04taW50ZXJpb3ItbmRvZSB0aGF0 IG9wZXJhdGVzIGluIGFkbWlzc2lvbiBjb250cm9sIHN0YXRlLCBpLmUuLAogICAiZm9yd2FyZCIg cGF0aCAoaS5lLiwgcGF0aCBiZXR3ZWVuIFBDTi1pbmdyZXNzLW5vZGUgdG93YXJkcyBQQ04tCiAg IGVncmVzcy1ub2RlKSBvciAicmV2ZXJzZSIgcGF0aCAoaS5lLiwgcGF0aCBiZXR3ZWVuIFBDTi0g ZWdyZXNzLW5vZGUKICAgdG93YXJkcyBQQ04taW5ncmVzcy1ub2RlKS4gIEZpZ3VyZSA1IHNob3dz IHRoZSBzY2VuYXJpbyB3aGVyZSB0aGUKCgoKV2VzdGJlcmcsIGV0IGFsLiAgICAgICAgICBFeHBp cmVzIE1heSAxNywgMjAwOCAgICAgICAgICAgICAgICAgW1BhZ2UgMjZdCgwKSW50ZXJuZXQtRHJh ZnQgICAgICAgICAgICAgICAgICAgTEMtUENOICAgICAgICAgICAgICAgICAgICBOb3ZlbWJlciAy MDA3CgoKICAgcHJlLWNvbmdlc3RlZCBQQ04taW50ZXJpb3Itbm9kZSBpcyBsb2NhdGVkIGluIHRo ZSAiZm9yd2FyZCIgcGF0aC4KICAgVGhlIGZ1bmN0aW9uYWxpdHkgb2YgcHJvdmlkaW5nIGFkbWlz c2lvbiBjb250cm9sIGlzIHRoZSBzYW1lIGFzIHRoZQogICBvbmUgZGVzY3JpYmVkIGluIFNlY3Rp b24gNC4xLCBGaWd1cmUgMi4gIEZpZ3VyZSA2IHNob3dzIHRoZSBzY2VuYXJpbwogICB3aGVyZSB0 aGUgcHJlLWNvbmdlc3RlZCBQQ04taW50ZXJpb3Itbm9kZSBpcyBsb2NhdGVkIGluIHRoZSAicmV2 ZXJzZSIKICAgcGF0aC4gIFRoZSBwcm9iZSBwYWNrZXQgc2VudCBpbiB0aGUgImZvcndhcmQiIGRp cmVjdGlvbiB3aWxsIG5vdCBiZQogICBhZmZlY3RlZCBieSB0aGUgcHJlLWNvbmdlc3RlZCBQQ04t aW50ZXJpb3Itbm9kZSwgd2hpbGUgdGhlIERTQ1AgdmFsdWUKICAgaW4gdGhlIElQIGhlYWRlciBv ZiBhbnkgcGFja2V0IG9mIHRoZSAicmV2ZXJzZSIgZGlyZWN0aW9uIGZsb3cgYW5kCiAgIGFsc28g b2YgdGhlIHByb2JlIHBhY2tldCB0aGF0IGNhcnJpZXMgdGhlIHNlbnQgaW4gdGhlICJyZXZlcnNl IgogICBkaXJlY3Rpb24gd2lsbCBiZSByZW1hcmtlZCBieSB0aGUgcHJlLWNvbmdlc3RlZCBub2Rl LiAgVGhlIFBDTi0KICAgaW5ncmVzcy1ub2RlIGlzIGluIHRoaXMgd2F5IG5vdGlmaWVkIHRoYXQg YSBwcmUtY29uZ2VzdGlvbiBzaXR1YXRpb24KICAgb2NjdXJyZWQgaW4gdGhlIG5ldHdvcmsgYW5k IHRoZXJlZm9yZSBpdCBpcyBhYmxlIHRvIHJlamVjdCB0aGUgbmV3CiAgIGluaXRpYXRpb24gb2Yg dGhlIHJlc2VydmF0aW9uLgoKUENOLWluZ3Jlc3Mtbm9kZSAgUENOLWludGVyaW9yLW5vZGUgIFBD Ti1pbnRlcmlvci1ub2RlICAgUENOLWVncmVzcy1ub2RlCgp1c2VyfCAgICAgICAgICAgICAgICB8 ICAgICAgICAgICAgICAgIHwgICAgICAgICAgIHwgICAgICAgICAgICAgICB8CmRhdGF8ICAgICAg ICAgICAgICAgIHwgICAgICAgICAgICAgICAgfCAgICAgICAgICAgfCAgICAgICAgICAgICAgIHwK LS0tPnwgICAgICAgICAgICAgICAgfCB1c2VyIGRhdGEgICAgICB8ICAgICAgICAgICB8ICAgICAg ICAgICAgICAgfAogICAgfC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tPnx1c2VyIGRhdGEgICAgICB8dXNlcgogICAgfCAgICAgICAgICAgICAgICB8ICAgICAgICAg ICAgICAgIHwgICAgICAgICAgIHwtLS0tLS0tLS0tLS0tLT58ZGF0YQogICAgfCAgICAgICAgICAg ICAgICB8ICAgICAgICAgICAgICAgIHwgICAgICAgICAgIHwgICAgICAgICAgICAgICB8LS0tPgog ICAgfCAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgIHwgICAgICAgICAgIHwgICAgICAg ICAgICAgICB8dXNlcgogICAgfCAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgIHwgICAg ICAgICAgIHwgICAgICAgICAgICAgICB8ZGF0YQogICAgfCAgICAgICAgICAgICAgICB8ICAgICAg ICAgICAgICAgIHwgICAgICAgICAgIHwgICAgICAgICAgICAgICB8PC0tLQogICAgfCAgICAgICAg ICAgICAgICBTICAgICAgICAgICAgICAgIHwgdXNlciBkYXRhIHwgICAgICAgICAgICAgICB8CiAg ICB8ICAgICAgICAgICAgICAgIFMgIHVzZXIgZGF0YSAgICAgfDwtLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLXwKICAgIHwgICB1c2VyIGRhdGEgICAgUzwtLS0tLS0tLS0tLS0tLS18ICAgICAgICAg ICB8ICAgICAgICAgICAgICAgfAogICAgfDwtLS0tLS0tLS0tLS0tLS1TICAgICAgICAgICAgICAg IHwgICAgICAgICAgIHwgICAgICAgICAgICAgICB8CiAgICB8ICB1c2VyIGRhdGEgICAgIFMgICAg ICAgICAgICAgICAgfCAgICAgICAgICAgfCAgICAgICAgICAgICAgIHwKICAgIHwgKCNtYXJrZWQg Ynl0ZXMpUyAgICAgICAgICAgICAgICB8ICAgICAgICAgICB8ICAgICAgICAgICAgICAgfAogICAg fDwtLS0tLS0tLS0tLS0tLS1TICAgICAgICAgICAgICAgIHwgICAgICAgICAgIHwgICAgICAgICAg ICAgICB8CiAgICB8ICAgICAgICAgICAgICAgIFMgICAgICAgICAgIHByb2JlKHVubWFya2VkIERT Q1ApICAgICAgICAgICAgIHwKICAgIHwgICAgICAgICAgICAgICAgUyAgICAgICAgICAgICB8ICAg ICAgICAgICAgICB8ICAgICAgICAgICAgICAgfAogICAgfC0tLS0tLS0tLS0tLS0tLS1TLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLT58CiAgICB8ICAgICAgICAgICAg ICAgIFMgICAgICAgICAgcHJvYmUocmUtbWFya2VkIERTQ1ApICAgICAgICAgICAgIHwKICAgIHwg ICAgICAgICAgICAgICAgUzwtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tfAogICAgfDwtLS0tLS0tLS0tLS0tLS1TICAgICAgICAgICAgIHwgICAgICAgICAgICAgIHwg ICAgICAgICAgICAgICB8CgoKICAgICAgICAgICAgRmlndXJlIDY6IEFkbWlzc2lvbiBjb250cm9s IGJhc2VkIG9uIHByb2JpbmcgZm9yCiAgICAgICAgICAgIGJpLWRpcmVjdGlvbmFsIGFkbWlzc2lv biBjb250cm9sIChwcmUtY29uZ2VzdGlvbiBvbiBwYXRoCiAgICAgICAgICAgIFBDTi1lZ3Jlc3Mt bm9kZSB0b3dhcmRzIFBDTi1pbmdyZXNzLW5vZGUpCgo0LjQuICBGbG93IFRlcm1pbmF0aW9uIGhh bmRsaW5nIGZvciBiaS1kaXJlY3Rpb25hbCBmbG93cwoKICAgVGhpcyBzZWN0aW9uIGRlc2NyaWJl cyB0aGUgZmxvdyB0ZXJtaW5hdGlvbiBoYW5kbGluZyBvcGVyYXRpb24gZm9yCiAgIGJpLWRpcmVj dGlvbmFsIGZsb3dzLiAgVGhpcyBmbG93IHRlcm1pbmF0aW9uIGhhbmRsaW5nIG9wZXJhdGlvbiBp cwogICBzaW1pbGFyIHRvIHRoZSBvbmUgZGVzY3JpYmVkIGluIFNlY3Rpb24gNC4yLgoKCgpXZXN0 YmVyZywgZXQgYWwuICAgICAgICAgIEV4cGlyZXMgTWF5IDE3LCAyMDA4ICAgICAgICAgICAgICAg ICBbUGFnZSAyN10KDApJbnRlcm5ldC1EcmFmdCAgICAgICAgICAgICAgICAgICBMQy1QQ04gICAg ICAgICAgICAgICAgICAgIE5vdmVtYmVyIDIwMDcKCgpQQ04taW5ncmVzcy1ub2RlICBQQ04taW50 ZXJpb3Itbm9kZSAgUENOLWludGVyaW9yLW5vZGUgICBQQ04tZWdyZXNzLW5vZGUKCnVzZXJ8ICAg ICAgICAgICAgICAgIHwgICAgICAgICAgICAgfCAgICAgICAgICAgICAgfCAgICAgICAgICAgICAg IHwKZGF0YXwgICAgdXNlciAgICAgICAgfCAgICAgICAgICAgICB8ICAgICAgICAgICAgICB8ICAg ICAgICAgICAgICAgfAotLS0+fCAgICBkYXRhICAgICAgICB8IHVzZXIgZGF0YSAgIHwgICAgICAg ICAgICAgIHx1c2VyIGRhdGEgICAgICB8CiAgICB8LS0tLS0tLS0tLS0tLS0tPnwgICAgICAgICAg ICAgfCAgICAgICAgICAgICAgUyAgICAgICAgICAgICAgIHwKICAgIHwgICAgICAgICAgICAgICAg fC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLT5TICgjbWFya2VkIGJ5dGVzKQogICAgfCAgICAg ICAgICAgICAgICB8ICAgICAgICAgICAgIHwgICAgICAgICAgICAgIFMtLS0tLS0tLS0tLS0tLT58 CiAgICB8ICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgfCAgICAgICAgICAgICAgUygjdW5t YXJrZWQgYnl0ZXMpCiAgICB8ICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgfCAgICAgICAg ICAgICAgUy0tLS0tLS0tLS0tLS0tPnxUZXJtCiAgICB8ICAgICAgICAgICAgICAgIHwgICAgICAg ICAgICAgfCAgICAgICAgICAgICAgUyAgICAgICAgICAgICAgIHxmbG93PwogICAgfCAgICAgICAg ICAgICAgICB8ICAgICAgICAgIG5vdGlmaWNhdGlvbiAodGVybWluYXRlKSAgICAgICAgICB8WUVT CiAgICB8PC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLXwKICAgIHxyZWxlYXNlIChmb3J3YXJkKSAgICAgICAgICAgICB8ICAgICAgICAg ICAgICBTICAgICAgICAgICAgICAgfAogICAgfC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLT58CiAgICB8ICAgICAgICByZWxlYXNlIChy ZXZlcmVzZSkgICAgfCAgICAgICAgICAgICAgUyAgICAgICAgICAgICAgIHwKICAgIHw8LS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tfAog ICAgfCAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgIHwgICAgICAgICAgICAgIFMgICAgICAg ICAgICAgICB8CgogICAgICAgICAgICBGaWd1cmUgNzogRmxvdyB0ZXJtaW5hdGlvbiBoYW5kbGlu ZyBmb3IgYmktZGlyZWN0aW9uYWwKICAgICAgICAgICAgcmVzZXJ2YXRpb24gKGNvbmdlc3Rpb24g b24gcGF0aCBQQ04taW5ncmVzcy1ub2RlCiAgICAgICAgICAgIHRvd2FyZHMgUENOLWVncmVzcy1u b2RlKQoKICAgVGhpcyBwcm9jZWR1cmUgaXMgc2ltaWxhciB0byB0aGUgZmxvdyB0ZXJtaW5hdGlv biBoYW5kbGluZyBwcm9jZWR1cmUKICAgZGVzY3JpYmVkIGluIFNlY3Rpb24gNC4yLiAgVGhlIG1h aW4gZGlmZmVyZW5jZSBpcyByZWxhdGVkIHRvIHRoZQogICBsb2NhdGlvbiBvZiB0aGUgdGhlIFBD Ti1pbnRlcmlvci1uZG9lIHRoYXQgb3BlcmF0ZXMgaW4gRmxvdwogICBUZXJtaW5hdGlvbiBzdGF0 ZSwgLCBpLmUuICJmb3J3YXJkIiBvciAicmV2ZXJzZSIgcGF0aC4gIFdoZW4gYSBmbG93CiAgIHRl cm1pbmF0aW9uIGNvbmdlc3Rpb24gb2NjdXJzIG9uIGUuZy4sIGluIHRoZSBmb3J3YXJkIHBhdGgs IGFuZCB3aGVuCiAgIHRoZSBhbGdvcml0aG0gdGVybWluYXRlcyBmbG93cyB0byBzb2x2ZSB0aGUg ZmxvdyB0ZXJtaW5hdGlvbiBpbiB0aGUKICAgZm9yd2FyZCBwYXRoLCB0aGVuIHRoZSByZXNlcnZl ZCBiYW5kd2lkdGggYXNzb2NpYXRlZCB3aXRoIHRoZQogICB0ZXJtaW5hdGVkIGJpZGlyZWN0aW9u YWwgZmxvd3MgaXMgYWxzbyByZWxlYXNlZC4gIFRoZXJlZm9yZSwgYQogICBjYXJlZnVsIHNlbGVj dGlvbiBvZiB0aGUgZmxvd3MgdGhhdCBoYXZlIHRvIGJlIHRlcm1pbmF0ZWQgc2hvdWxkIHRha2UK ICAgcGxhY2UuICBBIHBvc3NpYmxlIG1ldGhvZCBvZiBzZWxlY3RpbmcgdGhlIGZsb3dzIGJlbG9u Z2luZyB0byB0aGUKICAgc2FtZSBwcmlvcml0eSB0eXBlIHBhc3NpbmcgdGhyb3VnaCB0aGUgZmxv dyB0ZXJtaW5hdGlvbiBjb25nZXN0aW9uCiAgIHBvaW50IG9uIGEgdW5pZGlyZWN0aW9uYWwgcGF0 aCBjYW4gYmUgdGhlIGZvbGxvd2luZzoKCiAgIG8gIHRoZSBQQ04tZWdyZXNzLW5vZGUgc2hvdWxk IHNlbGVjdCwgaWYgcG9zc2libGUsIGZpcnN0CiAgICAgIHVuaWRpcmVjdGlvbmFsIGZsb3dzIGlu c3RlYWQgb2YgYmlkaXJlY3Rpb25hbCBmbG93cwoKICAgbyAgdGhlIFBDTi1lZ3Jlc3Mtbm9kZSBz aG91bGQgc2VsZWN0LCBpZiBwb3NzaWJsZSwgYmlkaXJlY3Rpb25hbAogICAgICBmbG93cyB0aGF0 IHJlc2VydmVkIGEgcmVsYXRpdmVseSBzbWFsbCBhbW91bnQgb2YgcmVzb3VyY2VzIG9uIHRoZQog ICAgICBwYXRoIHJldmVyc2VkIHRvIHRoZSBwYXRoIG9mIGNvbmdlc3Rpb24uCgoKCgoKCgoKCldl c3RiZXJnLCBldCBhbC4gICAgICAgICAgRXhwaXJlcyBNYXkgMTcsIDIwMDggICAgICAgICAgICAg ICAgIFtQYWdlIDI4XQoMCkludGVybmV0LURyYWZ0ICAgICAgICAgICAgICAgICAgIExDLVBDTiAg ICAgICAgICAgICAgICAgICAgTm92ZW1iZXIgMjAwNwoKClBDTi1pbmdyZXNzLW5vZGUgIFBDTi1p bnRlcmlvci1ub2RlICBQQ04taW50ZXJpb3Itbm9kZSAgIFBDTi1lZ3Jlc3Mtbm9kZQoKdXNlcnwg ICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICB8ICAgICAgICAgICB8ICAgICAgICAgICAg ICAgfApkYXRhfCAgICB1c2VyICAgICAgICB8ICAgICAgICAgICAgICAgIHwgICAgICAgICAgIHwg ICAgICAgICAgICAgICB8Ci0tLT58ICAgIGRhdGEgICAgICAgIHwgdXNlciBkYXRhICAgICAgfCAg ICAgICAgICAgfHVzZXIgZGF0YSAgICAgIHwKICAgIHwtLS0tLS0tLS0tLS0tLS0+fCAgICAgICAg ICAgICAgICB8ICAgICAgICAgICB8ICAgICAgICAgICAgICAgfAogICAgfCAgICAgICAgICAgICAg ICB8LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tPnx1c2VyIGRhdGEgICAgICB8dXNlcgogICAg fCAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgIHwgICAgICAgICAgIHwtLS0tLS0tLS0t LS0tLT58ZGF0YQogICAgfCAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgIHwgICAgICAg ICAgIHwgICAgICAgICAgICAgICB8LS0tPgogICAgfCAgICAgICAgICAgICAgICB8ICAgICAgICAg ICAgICAgIHwgIHVzZXIgICAgIHwgICAgICAgICAgICAgICB8PC0tLQogICAgfCAgIHVzZXIgZGF0 YSAgICB8ICAgICAgICAgICAgICAgIHwgIGRhdGEgICAgIHw8LS0tLS0tLS0tLS0tLS18CiAgICB8 ICgjbWFya2VkIGJ5dGVzKXwgICAgICAgICAgICAgICAgUzwtLS0tLS0tLS0tfCAgICAgICAgICAg ICAgIHwKICAgIHw8LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS1TICAgICAgICAgICB8 ICAgICAgICAgICAgICAgfAogICAgfCAoI3VubWFya2VkIGJ5dGVzKSAgICAgICAgICAgICAgIFMg ICAgICAgICAgIHwgICAgICAgICAgICAgICB8ClRlcm18PC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tUyAgICAgICAgICAgfCAgICAgICAgICAgICAgIHwKRmxvdz8gICAgICAgICAgICAg ICAgfCAgICAgICAgICAgICAgICBTICAgICAgICAgICB8ICAgICAgICAgICAgICAgfApZRVMgfCAg ICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgIFMgICAgICAgICAgIHwgICAgICAgICAgICAg ICB8CiAgICB8cmVsZWFzZSAoZm9yd2FyZCkgICAgICAgICAgICAgICAgUyAgICAgICAgICAgfCAg ICAgICAgICAgICAgIHwKICAgIHwtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0+fAogICAgfCAgICAgICAgcmVsZWFzZSAocmV2ZXJzZSkg ICAgICAgIFMgICAgICAgICAgIHwgICAgICAgICAgICAgICB8CiAgICB8PC0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLXwKICAgIHwgICAg ICAgICAgICAgICAgfCAgICAgICAgICAgICAgICBTICAgICAgICAgICB8ICAgICAgICAgICAgICAg fAoKICAgICAgICAgICAgICBGaWd1cmUgODogRmxvdyB0ZXJtaW5hdGlvbiBoYW5kbGluZyBmb3IK ICAgICAgICAgICAgICBiaS1kaXJlY3Rpb25hbCByZXNlcnZhdGlvbiAoZmxvdyB0ZXJtaW5hdGlv biBjb25nZXN0aW9uIG9uCiAgICAgICAgICAgICAgcGF0aCBQQ04tZWdyZXNzLW5vZGUgdG93YXJk cyBQQ04taW5ncmVzcy1ub2RlKQoKICAgRnVydGhlcm1vcmUsIGEgc3BlY2lhbCBjYXNlIG9mIHRo aXMgb3BlcmF0aW9uIGlzIGFzc29jaWF0ZWQgdG8gdGhlCiAgIEZsb3cgVGVybWluYXRpb24gc2l0 dWF0aW9uIG9jY3VycmluZyBzaW11bHRhbmVvdXNseSBvbiB0aGUgZm9yd2FyZAogICBhbmQgcmV2 ZXJzZSBwYXRocy4gIEFuIGV4YW1wbGUgb2YgdGhpcyBvcGVyYXRpb24gaXMgZ2l2ZW4gYmVsb3cu CiAgIENvbnNpZGVyIHRoYXQgdGhlIFBDTi1lZ3Jlc3Mtbm9kZSBzZWxlY3RzIGEgbnVtYmVyIG9m IGJpLWRpcmVjdGlvbmFsCiAgIGZsb3dzIHRvIGJlIHRlcm1pbmF0ZWQsIHNlZSBGaWd1cmUgOS4g IEluIHRoaXMgY2FzZSB0aGUgUENOLWVncmVzcy0KICAgbm9kZSB3aWxsIHNlbmQgZm9yIGVhY2gg YmktZGlyZWN0aW9uYWwgZmxvd3MgYSBub3RpZmljYXRpb24gbWVzc2FnZQogICB0byBQQ04taW5n cmVzcy1ub2RlLiAgSWYgdGhlIFBDTi1pbmdyZXNzLW5vZGUgcmVjZWl2ZXMgdGhlc2UKICAgbm90 aWZpY2F0aW9uIG1lc3NhZ2VzIGFuZCBpdHMgb3BlcmF0aW9uYWwgc3RhdGUgKGFzc29jaWF0ZWQg d2l0aAogICByZXZlcnNlIHBhdGgpIGlzIGluIHRoZSBGbG93IFRlcm1pbmF0aW9uIHN0YXRlIChz ZWUgRmlndXJlIDQpLCB0aGVuCiAgIHRoZSBQQ04taW5ncmVzcy1ub2RlIG9wZXJhdGVzIGluIHRo ZSBmb2xsb3dpbmcgd2F5OgoKCgoKCgoKCgoKCgoKCldlc3RiZXJnLCBldCBhbC4gICAgICAgICAg RXhwaXJlcyBNYXkgMTcsIDIwMDggICAgICAgICAgICAgICAgIFtQYWdlIDI5XQoMCkludGVybmV0 LURyYWZ0ICAgICAgICAgICAgICAgICAgIExDLVBDTiAgICAgICAgICAgICAgICAgICAgTm92ZW1i ZXIgMjAwNwoKClBDTi1pbmdyZXNzLW5vZGUgIFBDTi1pbnRlcmlvci1ub2RlICBQQ04taW50ZXJp b3Itbm9kZSAgIFBDTi1lZ3Jlc3Mtbm9kZQoKdXNlcnwgICAgICAgICAgICAgICAgfCAgICAgICAg ICAgICAgICB8ICAgICAgICAgICB8ICAgICAgICAgICAgICAgfApkYXRhfCAgICB1c2VyICAgICAg ICB8ICAgICAgICAgICAgICAgIHwgICAgICAgICAgIHwgICAgICAgICAgICAgICB8Ci0tLT58ICAg IGRhdGEgICAgICAgIHwgI3VubWFya2VkIGJ5dGVzfCAgICAgICAgICAgfCAgICAgICAgICAgICAg IHwKICAgIHwtLS0tLS0tLS0tLS0tLS0+UyAjbWFya2VkIGJ5dGVzICB8ICAgICAgICAgICB8ICAg ICAgICAgICAgICAgfAogICAgfCAgICAgICAgICAgICAgICBTLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tPnwgICAgICAgICAgICAgICB8CiAgICB8ICAgICAgICAgICAgICAgIHwgICAgICAgICAg ICAgICAgfCAgICAgICAgICAgfC0tLS0tLS0tLS0tLS0tPnxkYXRhCiAgICB8ICAgICAgICAgICAg ICAgIHwgICAgICAgICAgICAgICAgfCAgICAgICAgICAgfCAgICAgICAgICAgICAgIHwtLS0+CiAg ICB8ICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgfCAgICAgICAgICAgfCAgICAgICAg ICAgICAgVGVybS4/CiAgICB8ICAgICAgICAgICAgTk9USUZZICAgICAgICAgICAgICAgfCAgICAg ICAgICAgfCAgICAgICAgICAgICAgIHxZZXMKICAgIHw8LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tfAogICAgfCAgICAgICAgICAgICAg ICB8ICAgICAgICAgICAgICAgIHwgICAgICAgICAgIHwgICAgICAgICAgICAgICB8ZGF0YQogICAg fCAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgIHwgIHVzZXIgICAgIHwgICAgICAgICAg ICAgICB8PC0tLQogICAgfCAgIHVzZXIgZGF0YSAgICB8ICAgICAgICAgICAgICAgIHwgIGRhdGEg ICAgIHw8LS0tLS0tLS0tLS0tLS18CiAgICB8ICgjbWFya2VkIGJ5dGVzKXwgICAgICAgICAgICAg ICAgUzwtLS0tLS0tLS0tfCAgICAgICAgICAgICAgIHwKICAgIHw8LS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS1TICAgICAgICAgICB8ICAgICAgICAgICAgICAgfAogICAgfCAoI3VubWFy a2VkIGJ5dGVzKSAgICAgICAgICAgICAgIFMgICAgICAgICAgIHwgICAgICAgICAgICAgICB8ClRl cm18PC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tUyAgICAgICAgICAgfCAgICAgICAg ICAgICAgIHwKRmxvdz8gICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICBTICAgICAgICAg ICB8ICAgICAgICAgICAgICAgfApZRVMgfCAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAg IFMgICAgICAgICAgIHwgICAgICAgICAgICAgICB8CiAgICB8cmVsZWFzZSAoZm9yd2FyZCkgICAg ICAgICAgICAgICAgUyAgICAgICAgICAgfCAgICAgICAgICAgICAgIHwKICAgIHwtLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0+fAogICAg fCAgICAgICAgcmVsZWFzZSAocmV2ZXJzZSkgICAgICAgIFMgICAgICAgICAgIHwgICAgICAgICAg ICAgICB8CiAgICB8PC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLXwKCgogICAgICAgICAgICAgIEZpZ3VyZSA5OiBGbG93IHRlcm1pbmF0 aW9uIGhhbmRsaW5nIGZvcgogICAgICAgICAgICAgIGJpLWRpcmVjdGlvbmFsIHJlc2VydmF0aW9u IChmbG93IHRlcm1pbmF0aW9uIGNvbmdlc3Rpb24gb24KICAgICAgICAgICAgICBib3RoIGZvcndh cmQgYW5kIHJldmVyc2UgZGlyZWN0aW9uKQoKICAgbyAgRm9yIGVhY2ggbm90aWZpY2F0aW9uIG1l c3NhZ2UsIHRoZSBQQ04taW5ncmVzcy1ub2RlIHNob3VsZAogICAgICBpZGVudGlmeSB0aGUgYmlk aXJlY3Rpb25hbCBmbG93cyB0aGF0IGhhdmUgdG8gYmUgdGVybWluYXRlZC4KCiAgIG8gIFRoZSBQ Q04taW5ncmVzcy1ub2RlIHRoZW4gY2FsY3VsYXRlcyB0aGUgdG90YWwgYmFuZHdpZHRoIHRoYXQK ICAgICAgc2hvdWxkIGJlIHJlbGVhc2VkIGluIHRoZSByZXZlcnNlIGRpcmVjdGlvbiAodGh1cyBu b3QgaW4gZm9yd2FyZAogICAgICBkaXJlY3Rpb24pIGlmIHRoZSBiaWRpcmVjdGlvbmFsIGZsb3dz IHdpbGwgYmUgdGVybWluYXRlZAogICAgICAocHJlZW1wdGVkKSwgc2F5ICJub3RpZnlfcmV2ZXJz ZV9iYW5kd2lkdGgiLiAgVGhpcyBiYW5kd2lkdGggY2FuCiAgICAgIGJlIGNhbGN1bGF0ZWQgYnkg dGhlIHN1bSBvZiB0aGUgYmFuZHdpZHRoIHZhbHVlcyBhc3NvY2lhdGVkIHdpdGgKICAgICAgYWxs IHRoZSBlbmQtdG8tZW5kIGZsb3dzIHRoYXQgcmVjZWl2ZWQgYSAoZmxvdyB0ZXJtaW5hdGlvbikK ICAgICAgbm90aWZpY2F0aW9uIG1lc3NhZ2UuCgogICBvICBGdXJ0aGVybW9yZSwgdXNpbmcgdGhl IHJlY2VpdmVkIG1hcmtlZCBwYWNrZXRzIChmcm9tIHRoZSByZXZlcnNlCiAgICAgIHBhdGgpIHRo ZSBQQ04taW5ncmVzcy1ub2RlIHdpbGwgY2FsY3VsYXRlLCB1c2luZyB0aGUgYWxnb3JpdGhtCiAg ICAgIHVzZWQgYnkgYW4gUENOLWVncmVzcy1ub2RlIGFuZCBkZXNjcmliZWQgaW4gU2VjdGlvbiA0 LjIuMywgdGhlCiAgICAgIHRvdGFsIGJhbmR3aWR0aCB0aGF0IGhhcyB0byBiZSB0ZXJtaW5hdGVk IGluIG9yZGVyIHRvIHNvbHZlIHRoZQogICAgICBmbG93IHRlcm1pbmF0aW9uIGNvbmdlc3Rpb24g aW4gdGhlIHJldmVyc2UgcGF0aCBkaXJlY3Rpb24sIHNheQogICAgICAibWFya2VkX3JldmVyc2Vf YmFuZHdpZHRoIi4KCgoKV2VzdGJlcmcsIGV0IGFsLiAgICAgICAgICBFeHBpcmVzIE1heSAxNywg MjAwOCAgICAgICAgICAgICAgICAgW1BhZ2UgMzBdCgwKSW50ZXJuZXQtRHJhZnQgICAgICAgICAg ICAgICAgICAgTEMtUENOICAgICAgICAgICAgICAgICAgICBOb3ZlbWJlciAyMDA3CgoKICAgbyAg VGhlIFBDTi1pbmdyZXNzLW5vZGUgdGhlbiBjYWxjdWxhdGVzIHRoZSBiYW5kd2lkdGggb2YgdGhl CiAgICAgIGFkZGl0aW9uYWwgZmxvd3MgdGhhdCBoYXZlIHRvIGJlIHRlcm1pbmF0ZWQsIHNheQog ICAgICAiYWRkaXRpb25hbF9yZXZlcnNlX2JhbmR3aWR0aCIsIGluIG9yZGVyIHRvIHNvbHZlIHRo ZSBmbG93CiAgICAgIHRlcm1pbmF0aW9uIGNvbmdlc3Rpb24gaW4gdGhlIHJldmVyc2UgZGlyZWN0 aW9uLCBieSB0YWtpbmcgaW50bwogICAgICBhY2NvdW50OgoKICAgICAgKiAgdGhlIGJhbmR3aWR0 aCBpbiB0aGUgcmV2ZXJzZSBkaXJlY3Rpb24gb2YgdGhlIGJpZGlyZWN0aW9uYWwKICAgICAgICAg Zmxvd3MgdGhhdCB3ZXJlIGFwcG9pbnRlZCBieSB0aGUgUENOLWVncmVzcy1ub2RlICh0aGUgb25l cyB0aGF0CiAgICAgICAgIHJlY2VpdmVkIGEgbm90aWZpY2F0aW9uIG1lc3NhZ2UpIHRvIGJlIHBy ZWVtcHRlZCwgaS5lLiwKICAgICAgICAgIm5vdGlmeV9yZXZlcnNlX2JhbmR3aWR0aCIKCiAgICAg ICogIHRoZSB0b3RhbCBhbW91bnQgb2YgYmFuZHdpZHRoIGluIHRoZSByZXZlcnNlIGRpcmVjdGlv biB0aGF0IGhhcwogICAgICAgICBiZWVuIGNhbGN1bGF0ZWQgYnkgdXNpbmcgdGhlIHJlY2VpdmVk IG1hcmtlZCBwYWNrZXRzLCBpLmUuLAogICAgICAgICAibWFya2VkX3JldmVyc2VfYmFuZHdpZHRo Ii4gIFRoaXMgYWRkaXRpb25hbCBiYW5kd2lkdGggY2FuIGJlCiAgICAgICAgIGNhbGN1bGF0ZWQg dXNpbmcgdGhlIGZvbGxvd2luZyBhbGdvcml0aG06CgoKICAgICAgIElGICgibWFya2VkX3JldmVy c2VfYmFuZHdpZHRoIiA+ICJub3RpZnlfcmV2ZXJzZV9iYW5kd2lkdGgiKSBUSEVOCiAgICAgICAg ICAgImFkZGl0aW9uYWxfcmV2ZXJzZV9iYW5kd2lkdGgiID0KICAgICAgICAgICAgICAibWFya2Vk X3JldmVyc2VfYmFuZHdpZHRoIi0gIm5vdGlmeV9yZXZlcnNlX2JhbmR3aWR0aCI7CiAgICAgICBF TFNFCiAgICAgICAgICAgImFkZGl0aW9uYWxfcmV2ZXJzZV9iYW5kd2lkdGgiID0gMAoKICAgbyAg UENOLWluZ3Jlc3Mtbm9kZSB0ZXJtaW5hdGVzIHRoZSBmbG93cyB0aGF0IGV4cGVyaWVuY2VkIGEg c2V2ZXJlCiAgICAgIGNvbmdlc3Rpb24gaW4gdGhlICJmb3J3YXJkIiBwYXRoIGFuZCByZWNlaXZl ZCBhIChmbG93IHRlcm1pbmF0aW9uKQogICAgICBub3RpZmljYXRpb24gbWVzc2FnZQoKICAgbyAg SWYgcG9zc2libGUgdGhlIFBDTi1pbmdyZXNzLW5vZGUgc2hvdWxkIHRlcm1pbmF0ZSB1bmlkaXJl Y3Rpb25hbAogICAgICBmbG93cyB0aGF0IGFyZSB1c2luZyB0aGUgc2FtZSBlZ3Jlc3MtaW5ncmVz cyByZXZlcnNlIGRpcmVjdGlvbgogICAgICBjb21tdW5pY2F0aW9uIHBhdGggdG8gc2F0aXNmeSB0 aGUgcmVsZWFzZSBvZiBhIHRvdGFsIGJhbmRpd3RkaCB1cAogICAgICBlcXVhbCB0byB0aGU6ICJh ZGRpdGlvbmFsX3JldmVyc2VfYmFuZHdpZHRoIi4KCiAgIG8gIElmIHRoZSBudW1iZXIgb2YgcmVx dWlyZWQgdW5pLWRpcmVjdGlvbmFsIGZsb3dzICh0byBzYXRpc2Z5IHRoZQogICAgICBhYm92ZSBp c3N1ZSkgaXMgbm90IGF2YWlsYWJsZSwgdGhlbiBhIG51bWJlciBvZiBiaS1kaXJlY3Rpb25hbAog ICAgICBmbG93cyB0aGF0IGFyZSB1c2luZyB0aGUgc2FtZSBlZ3Jlc3MtaW5ncmVzcyByZXZlcnNl IGRpcmVjdGlvbgogICAgICBjb21tdW5pY2F0aW9uIHBhdGggbWF5IGJlIHNlbGVjdGVkIGZvciBm bG93IHRlcm1pbmF0aW9uIGluIG9yZGVyCiAgICAgIHRvIHNhdGlzZnkgdGhlIHJlbGVhc2Ugb2Yg YSB0b3RhbCBiYW5kaXd0ZGggZXF1YWwgdXAgdG8gdGhlOgogICAgICAiYWRkaXRpb25hbF9yZXZl cnNlX2JhbmR3aWR0aCIuICBOb3RlIHRoYXQgdXNpbmcgdGhlIGd1aWRlbGluZXMKICAgICAgZ2l2 ZW4gaW4gYWJvdmUsIGZpcnN0IHRoZSBiaWRpcmVjdGlvbmFsIGZsb3dzIHRoYXQgcmVzZXJ2ZWQg YQogICAgICByZWxhdGl2ZWx5IHNtYWxsIGFtb3VudCBvZiByZXNvdXJjZXMgb24gdGhlIHBhdGgg cmV2ZXJzZWQgdG8gdGhlCiAgICAgIHBhdGggb2YgY29uZ2VzdGlvbiBzaG91bGQgYmUgc2VsZWN0 ZWQgZm9yIHRlcm1pbmF0aW9uLgoKICAgbyAgRnVydGhlcm1vcmUsIHRoZSBQQ04tZWdyZXNzLW5v ZGUgaW5jbHVkZXMgdGhlIHRvIGJlIHJlbGVhc2VkCiAgICAgIGFnZ3JlZ2F0ZWQgYmFuZHdpZHRo IHZhbHVlIGluIG9uZSBvZiB0aGUgbm90aWZpY2F0aW9uIG1lc3NhZ2VzLgoKICAgbyAgVGhlIFBD Ti1pbmdyZXNzLW5vZGUgcmVjZWl2ZXMgdGhpcyBub3RpZmljYXRpb24gbWVzc2FnZSBhbmQgcmVh ZHMKICAgICAgdGhlIHZhbHVlIG9mIHRoZSBjYXJyaWVkIHRvIGJlIHJlbGVhc2VkIGFnZ3JlZ2F0 ZWQgYmFuZHdpZHRoLgoKCgoKV2VzdGJlcmcsIGV0IGFsLiAgICAgICAgICBFeHBpcmVzIE1heSAx NywgMjAwOCAgICAgICAgICAgICAgICAgW1BhZ2UgMzFdCgwKSW50ZXJuZXQtRHJhZnQgICAgICAg ICAgICAgICAgICAgTEMtUENOICAgICAgICAgICAgICAgICAgICBOb3ZlbWJlciAyMDA3CgoKICAg VGhlIHNpemUgb2YgdGhlIGFnZ3JlZ2F0ZWQgcmVzZXJ2YXRpb24gc3RhdGUgY2FuIGJlIHJlZHVj ZWQgaW4gdGhlCiAgICJmb3J3YXJkIiBhbmQgInJldmVyc2UiIGJ5IHVzaW5nIHRoZSByZWNlaXZl ZCB0byBiZSByZWR1Y2VkIHZhbHVlcwogICB0aGUgYWdncmVnYXRlZCBiYW5kd2lkdGggaW4gImZv cndhcmQiIGFuZCAicmV2ZXJlc2UiIGRpcmVjdGlvbnMuCiAgIEZpZ3VyZSA3IHNob3dzIHRoZSBz Y2VuYXJpbyB3aGVyZSB0aGUgc2V2ZXJlIGNvbmdlc3RlZCBub2RlIGlzCiAgIGxvY2F0ZWQgaW4g dGhlICJmb3J3YXJkIiBwYXRoLiAgVGhpcyBzY2VuYXJpbyBpcyB2ZXJ5IHNpbWlsYXIgdG8gdGhl CiAgIGZsb3cgdGVybWluYXRpb24gaGFuZGxpbmcgc2NlbmFyaW8gZGVzY3JpYmVkIGluIFNlY3Rp b24gNC4yLiAgVGhlCiAgIGRpZmZlcmVuY2UgaXMgcmVsYXRlZCB0byB0aGUgcmVsZWFzZSBwcm9j ZWR1cmUsIHdoaWNoIGlzIGFjY29tcGxpc2hlZAogICBpbiBib3RoIGRpcmVjdGlvbnMgImZvcndh cmQiIGFuZCAicmV2ZXJzZSIuICBGaWd1cmUgOCBzaG93cyB0aGUKICAgc2NlbmFyaW8gd2hlcmUg dGhlIHNldmVyZSBjb25nZXN0ZWQgbm9kZSBpcyBsb2NhdGVkIGluIHRoZSAicmV2ZXJzZSIKICAg cGF0aC4gIFRoZSBtYWluIGRpZmZlcmVuY2UgYmV0d2VlbiB0aGlzIHNjZW5hcmlvIGFuZCB0aGUg c2NlbmFyaW8KICAgc2hvd24gaW4gRmlndXJlIDcgaXMgdGhhdCBubyBub3RpZmljYXRpb24gbWVz c2FnZXMgaGF2ZSB0byBiZQogICBnZW5lcmF0ZWQgYnkgdGhlIFBDTi1lZ3Jlc3Mtbm9kZS4gIFRo aXMgaXMgYmVjYXVzZSB0aGUgKCNtYXJrZWQgYW5kCiAgICN1bm1hcmtlZCkgdXNlciBkYXRhIGlz IGFycml2aW5nIGF0IHRoZSBQQ04taW5ncmVzcy1ub2RlLiAgVGhlIFBDTi0KICAgaW5ncmVzcy1u b2RlIHdpbGwgYmUgYWJsZSB0byBjYWxjdWxhdGUgdGhlIG51bWJlciBvZiBmbG93cyB0aGF0IGhh dmUKICAgdG8gYmUgdGVybWluYXRlZCBvciBmb3J3YXJkZWQgaW4gYSBsb3dlciBwcmlvcml0eSBx dWV1ZS4KCgo1LiAgU2VjdXJpdHkgQ29uc2lkZXJhdGlvbnMKCiAgIFRoZSBzZWN1cml0eSBjb25z aWRlcmF0aW9ucyBhc3NvY2lhdGVkIHdpdGggdGhpcyBkb2N1bWVudCBhcmUgc2ltaWxhcgogICB0 byB0aGUgb25lIGRlc2NyaWJlZCBpbiBbRWFyZDA3XS4KCgo2LiAgSUFOQSBDb25zaWRlcmF0aW9u cwoKICAgVG8gYmUgQWRkZWQKCgo3LiAgQWNrbm93bGVkZ2VtZW50cwoKICAgVG8gYmUgQWRkZWQK Cgo4LiAgSW5mb3JtYXRpdmUgUmVmZXJlbmNlcwoKICAgW0FkQ2EwM10gICBBZGxlciwgTS4sIENh aSwgSi4sIFNoYXBpcm8sIEouLCBhbmQgRC4gVG93c2xleSwKICAgICAgICAgICAgICAiRXN0aW1h dGlvbiBvZiBjb25nZXN0aW9uIHByaWNlIHVzaW5nIHByb2JhYmlsaXN0aWMgcGFja2V0CiAgICAg ICAgICAgICAgbWFya2luZyIsIFByb2MuIElFRUUgSU5GT0NPTSwgcHAuIDIwNjgtMjA3OCwgMjAw My4KCiAgIFtBbkhhMDZdICAgTGFjaGxhbiwgQS4gYW5kIFMuIEhhbmx5LCAiVGhlIEVzdGltYXRp b24gRXJyb3Igb2YKICAgICAgICAgICAgICBBZGFwdGl2ZSBEZXRlcm1pbmlzdGljIFBhY2tldCBN YXJraW5nIiwgNDR0aCBBbm51YWwKICAgICAgICAgICAgICBBbGxlcnRvbiBDb25mZXJlbmNlIG9u IENvbW11bmljYXRpb24sICBDb250cm9sIGFuZAogICAgICAgICAgICAgIENvbXB1dGluZywgLCAy MDA2LgoKICAgW0F0TGkwMV0gICBBdGh1cmFsaXlhLCBTLiwgTGksIFYuLCBMb3csIFMuLCBhbmQg US4gWWluLCAiUkVNOiBhY3RpdmUKICAgICAgICAgICAgICBxdWV1ZSBtYW5hZ2VtZW50IiwgSUVF RSBOZXR3b3JrLCB2b2wuIDE1LCBwcC4gNDgtNTMsIE1heS8KICAgICAgICAgICAgICBKdW5lIDIw MDEuCgoKCgpXZXN0YmVyZywgZXQgYWwuICAgICAgICAgIEV4cGlyZXMgTWF5IDE3LCAyMDA4ICAg ICAgICAgICAgICAgICBbUGFnZSAzMl0KDApJbnRlcm5ldC1EcmFmdCAgICAgICAgICAgICAgICAg ICBMQy1QQ04gICAgICAgICAgICAgICAgICAgIE5vdmVtYmVyIDIwMDcKCgogICBbQmFiaTA3XSAg IEJhYmlhcnosIEouIGFuZCBldC4gYWwuLCAiVGhyZWUgU3RhdGUgUENOIE1hcmtpbmciLAogICAg ICAgICAgICAgIGRyYWZ0LWJhYmlhcnotcGNuLTNzbS0wMCAod29yayBpbiBwcm9ncmVzcyksICwg SnVuZSAyMDA3LgoKICAgW0Jlcm5ldDk5XQogICAgICAgICAgICAgIEJlcm5ldHQsIFkuLCBZYXZh dGthciwgUi4sIEZvcmQsIFAuLCBCYWtlciwgRi4sIFpoYW5nLCBMLiwKICAgICAgICAgICAgICBT cGVlciwgTS4sIGFuZCBSLiBCcmFkZW4sICJJbnRlcm9wZXJhdGlvbiBvZiBSU1ZQL0ludHNlcnYK ICAgICAgICAgICAgICBhbmQgRGlmZnNlcnYgTmV0d29ya3MiLCBXb3JrIGluIFByb2dyZXNzICwg TWFyY2ggMTk5OS4KCiAgIFtCZXJzb245N10KICAgICAgICAgICAgICBCZXJzb24sIFMuIGFuZCBS LiBWaW5jZW50LCAiQWdncmVnYXRpb24gb2YgSW50ZXJuZXQKICAgICAgICAgICAgICBJbnRlZ3Jh dGVkIFNlcnZpY2VzIFN0YXRlIiwgV29yayBpbiBQcm9ncmVzcywgLAogICAgICAgICAgICAgIERl Y2VtYmVyIDE5OTcuCgogICBbQ0wtQVJDSF0gIEJyaXNjb2UsIEIuIGFuZCBldC4gYWwuLCAiQW4g ZWRnZS10by1lZGdlIERlcGxveW1lbnQgbW9kZWwKICAgICAgICAgICAgICBmb3IgcHJlLWNvbmdl c3Rpb24gbm90aWZpY2F0aW9uOiBBZG1pc3Npb24gY29udHJvbCBvdmVyIGEKICAgICAgICAgICAg ICBEaWZmc2VydiByZWdpb24iLCAgLCBPY3RvYmVyIDIwMDYuCgogICBbQ0wtUEhCXSAgIEJyaXNj b2UsIEIuIGFuZCBldC4gYWwuLCAiUHJlLWNvbmdlc3Rpb24gbm90aWZpY2F0aW9uCiAgICAgICAg ICAgICAgbWFya2luZyIsICAsIE9jdG9iZXIgMjAwNi4KCiAgIFtDaGFyMDddICAgQ2hhcm55LCBB LiBhbmQgZXQuIGFsLiwgIlByZS1Db25nZXN0aW9uIE5vdGlmaWNhdGlvbiBVc2luZwogICAgICAg ICAgICAgIFNpbmdsZSBNYXJraW5nIGZvciBBZG1pc3Npb24gYW5kIFRlcm1pbmF0aW9uIiwKICAg ICAgICAgICAgICBkcmFmdC1jaGFybnktcGNuLXNpbmdsZS1tYXJraW5nLTAyICh3b3JrIGluIHBy b2dyZXNzKSwgLAogICAgICAgICAgICAgIEp1bHkgMjAwNy4KCiAgIFtDc1RhMDVdICAgQ3Nhc3ph ciwgQS4sIFRha2FjcywgQS4sIFN6YWJvLCBSLiwgYW5kIFQuIEhlbmssCiAgICAgICAgICAgICAg IlJlc2lsaWVudCBSZWR1Y2VkLVN0YXRlIFJlc291cmNlIFJlc2VydmF0aW9uIiwgSm91cm5hbCBv ZgogICAgICAgICAgICAgIENvbW11bmljYXRpb24gYW5kICBOZXR3b3JrcyBWb2wuIDcsIE51bS4g NCwgRGVjZW1iZXIgMjAwNS4KCiAgIFtFYXJkMDddICAgRWFyZGxleSwgUC4sICJQcmUtQ29uZ2Vz dGlvbiBOb3RpZmljYXRpb24gQXJjaGl0ZWN0dXJlIiwKICAgICAgICAgICAgICBkcmFmdC1pZXRm LXBjbi1hcmNoaXRlY3R1cmUtMDEgKHdvcmsgaW4gcHJvZ3Jlc3MpLCAsCiAgICAgICAgICAgICAg T2N0b2JlciAyMDA3LgoKICAgW1JGQzI0NzVdICBCbGFrZSwgUy4sIEJsYWNrLCBELiwgQ2FybHNv biwgTS4sIERhdmllcywgRS4sIFdhbmcsIFouLAogICAgICAgICAgICAgIGFuZCBXLiBXZWlzcywg IkFuIEFyY2hpdGVjdHVyZSBmb3IgRGlmZmVyZW50aWF0ZWQKICAgICAgICAgICAgICBTZXJ2aWNl cyIsIFJGQyAyNDc1LCBEZWNlbWJlciAxOTk4LgoKICAgW1JGQzMxNzVdICBCYWtlciwgRi4sIEl0 dXJyYWxkZSwgQy4sIExlIEZhdWNoZXVyLCBGLiwgYW5kIEIuIERhdmllLAogICAgICAgICAgICAg ICJBZ2dyZWdhdGlvbiBvZiBSU1ZQIGZvciBJUHY0IGFuZCBJUHY2IFJlc2VydmF0aW9ucyIsCiAg ICAgICAgICAgICAgUkZDIDMxNzUsIFNlcHRlbWJlciAyMDAxLgoKICAgW1JNRF0gICAgICBCYWRl ciwgQS4sICJSTUQtUU9TTTogVGhlIHJlc291cmNlIG1hbmFnZW1lbnQgaW4gRGlmZnNlcnYKICAg ICAgICAgICAgICBRb1MgTW9kZWwiLCBkcmFmdC1pZXRmLW5zaXMtcm1kLTExLnR4dCAod29yayBp bgogICAgICAgICAgICAgIHByb2dyZXNzKSwgLCBNYXJjaCAyMDA3LgoKICAgW1N0b2ljYTk5XQog ICAgICAgICAgICAgIFN0b2ljYSwgSS4gYW5kIGV0LiBhbC4sICJQZXIgSG9wIEJlaGF2aW9ycyBC YXNlZCBvbgogICAgICAgICAgICAgIER5bmFtaWMgIFBhY2tldCBTdGF0ZXMiLCBXb3JrIGluIFBy b2dyZXNzICwgRmVicnVhcnkgMTk5OS4KCgoKV2VzdGJlcmcsIGV0IGFsLiAgICAgICAgICBFeHBp cmVzIE1heSAxNywgMjAwOCAgICAgICAgICAgICAgICAgW1BhZ2UgMzNdCgwKSW50ZXJuZXQtRHJh ZnQgICAgICAgICAgICAgICAgICAgTEMtUENOICAgICAgICAgICAgICAgICAgICBOb3ZlbWJlciAy MDA3CgoKICAgW1RoQ28wNF0gICBUaG9tbWVzLCBSLiBhbmQgTS4gQ29hdGVzLCAiRGV0ZXJtaW5p c3RpYyBwYWNrZXQgbWFya2luZwogICAgICAgICAgICAgIGZvciBjb25nZXN0aW9uIHBhY2tldCBl c3RpbWF0aW9uIiwgUHJvYy4gSUVFRSBJbmZvY29tICwKICAgICAgICAgICAgICAyMDA0LgoKICAg W1dlc3RiZXJnMDBdCiAgICAgICAgICAgICAgV2VzdGJlcmcsIEwuIGFuZCBldC4gYWwuLCAiTG9h ZCBDb250cm9sIG9mIFJlYWwtVGltZQogICAgICAgICAgICAgIFRyYWZmaWMiLCBJRVRGIFdvcmsg aW4gUHJvZ3Jlc3MgLCBBcHJpbCAyMDAwLgoKCkF1dGhvcnMnIEFkZHJlc3NlcwoKICAgTGFycyBX ZXN0YmVyZwogICBFcmljc3NvbgogICBUb3JzaGFtbnNnYXRhbiAyMwogICBTRS0xNjQgODAgU3Rv Y2tob2xtCiAgIFN3ZWRlbgoKICAgRW1haWw6IExhcnMud2VzdGJlcmdAZXJpY3Nzb24uY29tCgoK ICAgQW51cmFnIEJoYXJnYXZhCiAgIEVyaWNzc29uCiAgIDkyMCBNYWluIENhbXB1cyBEci4sIFN1 aXRlIDUwMAogICBSYWxlaWdoLCBOQyAgMjc2MDYKICAgVVNBCgogICBQaG9uZTogKzEgOTE5IDQ3 MiA5OTY0CiAgIEVtYWlsOiBhbnVyYWcuYmhhcmdhdmFAZXJpY3Nzb24uY29tCgoKICAgQXR0aWxh IEJhZGVyCiAgIEVyaWNzc29uCiAgIExhYm9yYyAxCiAgIEJ1ZGFwZXN0CiAgIEh1bmdhcnkKCiAg IEVtYWlsOiBBdHRpbGEuQmFkZXJAZXJpY3Nzb24uY29tCgoKICAgR2Vvcmdpb3MgS2FyYWdpYW5u aXMKICAgVW5pdmVyc2l0eSBvZiBUd2VudGUKICAgUC5PLiBCb3ggMjE3CiAgIDc1MDAgQUUgRW5z Y2VkZQogICBOZXRoZXJsYW5kcwoKICAgRW1haWw6IGcua2FyYWdpYW5uaXNAZXdpLnV0d2VudGUu bmwKCgoKCgpXZXN0YmVyZywgZXQgYWwuICAgICAgICAgIEV4cGlyZXMgTWF5IDE3LCAyMDA4ICAg ICAgICAgICAgICAgICBbUGFnZSAzNF0KDApJbnRlcm5ldC1EcmFmdCAgICAgICAgICAgICAgICAg ICBMQy1QQ04gICAgICAgICAgICAgICAgICAgIE5vdmVtYmVyIDIwMDcKCgpGdWxsIENvcHlyaWdo dCBTdGF0ZW1lbnQKCiAgIENvcHlyaWdodCAoQykgVGhlIElFVEYgVHJ1c3QgKDIwMDcpLgoKICAg VGhpcyBkb2N1bWVudCBpcyBzdWJqZWN0IHRvIHRoZSByaWdodHMsIGxpY2Vuc2VzIGFuZCByZXN0 cmljdGlvbnMKICAgY29udGFpbmVkIGluIEJDUCA3OCwgYW5kIGV4Y2VwdCBhcyBzZXQgZm9ydGgg dGhlcmVpbiwgdGhlIGF1dGhvcnMKICAgcmV0YWluIGFsbCB0aGVpciByaWdodHMuCgogICBUaGlz IGRvY3VtZW50IGFuZCB0aGUgaW5mb3JtYXRpb24gY29udGFpbmVkIGhlcmVpbiBhcmUgcHJvdmlk ZWQgb24gYW4KICAgIkFTIElTIiBiYXNpcyBhbmQgVEhFIENPTlRSSUJVVE9SLCBUSEUgT1JHQU5J WkFUSU9OIEhFL1NIRSBSRVBSRVNFTlRTCiAgIE9SIElTIFNQT05TT1JFRCBCWSAoSUYgQU5ZKSwg VEhFIElOVEVSTkVUIFNPQ0lFVFksIFRIRSBJRVRGIFRSVVNUIEFORAogICBUSEUgSU5URVJORVQg RU5HSU5FRVJJTkcgVEFTSyBGT1JDRSBESVNDTEFJTSBBTEwgV0FSUkFOVElFUywgRVhQUkVTUwog ICBPUiBJTVBMSUVELCBJTkNMVURJTkcgQlVUIE5PVCBMSU1JVEVEIFRPIEFOWSBXQVJSQU5UWSBU SEFUIFRIRSBVU0UgT0YKICAgVEhFIElORk9STUFUSU9OIEhFUkVJTiBXSUxMIE5PVCBJTkZSSU5H RSBBTlkgUklHSFRTIE9SIEFOWSBJTVBMSUVECiAgIFdBUlJBTlRJRVMgT0YgTUVSQ0hBTlRBQklM SVRZIE9SIEZJVE5FU1MgRk9SIEEgUEFSVElDVUxBUiBQVVJQT1NFLgoKCkludGVsbGVjdHVhbCBQ cm9wZXJ0eQoKICAgVGhlIElFVEYgdGFrZXMgbm8gcG9zaXRpb24gcmVnYXJkaW5nIHRoZSB2YWxp ZGl0eSBvciBzY29wZSBvZiBhbnkKICAgSW50ZWxsZWN0dWFsIFByb3BlcnR5IFJpZ2h0cyBvciBv dGhlciByaWdodHMgdGhhdCBtaWdodCBiZSBjbGFpbWVkIHRvCiAgIHBlcnRhaW4gdG8gdGhlIGlt cGxlbWVudGF0aW9uIG9yIHVzZSBvZiB0aGUgdGVjaG5vbG9neSBkZXNjcmliZWQgaW4KICAgdGhp cyBkb2N1bWVudCBvciB0aGUgZXh0ZW50IHRvIHdoaWNoIGFueSBsaWNlbnNlIHVuZGVyIHN1Y2gg cmlnaHRzCiAgIG1pZ2h0IG9yIG1pZ2h0IG5vdCBiZSBhdmFpbGFibGU7IG5vciBkb2VzIGl0IHJl cHJlc2VudCB0aGF0IGl0IGhhcwogICBtYWRlIGFueSBpbmRlcGVuZGVudCBlZmZvcnQgdG8gaWRl bnRpZnkgYW55IHN1Y2ggcmlnaHRzLiAgSW5mb3JtYXRpb24KICAgb24gdGhlIHByb2NlZHVyZXMg d2l0aCByZXNwZWN0IHRvIHJpZ2h0cyBpbiBSRkMgZG9jdW1lbnRzIGNhbiBiZQogICBmb3VuZCBp biBCQ1AgNzggYW5kIEJDUCA3OS4KCiAgIENvcGllcyBvZiBJUFIgZGlzY2xvc3VyZXMgbWFkZSB0 byB0aGUgSUVURiBTZWNyZXRhcmlhdCBhbmQgYW55CiAgIGFzc3VyYW5jZXMgb2YgbGljZW5zZXMg dG8gYmUgbWFkZSBhdmFpbGFibGUsIG9yIHRoZSByZXN1bHQgb2YgYW4KICAgYXR0ZW1wdCBtYWRl IHRvIG9idGFpbiBhIGdlbmVyYWwgbGljZW5zZSBvciBwZXJtaXNzaW9uIGZvciB0aGUgdXNlIG9m CiAgIHN1Y2ggcHJvcHJpZXRhcnkgcmlnaHRzIGJ5IGltcGxlbWVudGVycyBvciB1c2VycyBvZiB0 aGlzCiAgIHNwZWNpZmljYXRpb24gY2FuIGJlIG9idGFpbmVkIGZyb20gdGhlIElFVEYgb24tbGlu ZSBJUFIgcmVwb3NpdG9yeSBhdAogICBodHRwOi8vd3d3LmlldGYub3JnL2lwci4KCiAgIFRoZSBJ RVRGIGludml0ZXMgYW55IGludGVyZXN0ZWQgcGFydHkgdG8gYnJpbmcgdG8gaXRzIGF0dGVudGlv biBhbnkKICAgY29weXJpZ2h0cywgcGF0ZW50cyBvciBwYXRlbnQgYXBwbGljYXRpb25zLCBvciBv dGhlciBwcm9wcmlldGFyeQogICByaWdodHMgdGhhdCBtYXkgY292ZXIgdGVjaG5vbG9neSB0aGF0 IG1heSBiZSByZXF1aXJlZCB0byBpbXBsZW1lbnQKICAgdGhpcyBzdGFuZGFyZC4gIFBsZWFzZSBh ZGRyZXNzIHRoZSBpbmZvcm1hdGlvbiB0byB0aGUgSUVURiBhdAogICBpZXRmLWlwckBpZXRmLm9y Zy4KCgpBY2tub3dsZWRnbWVudAoKICAgRnVuZGluZyBmb3IgdGhlIFJGQyBFZGl0b3IgZnVuY3Rp b24gaXMgcHJvdmlkZWQgYnkgdGhlIElFVEYKICAgQWRtaW5pc3RyYXRpdmUgU3VwcG9ydCBBY3Rp dml0eSAoSUFTQSkuCgoKCgoKV2VzdGJlcmcsIGV0IGFsLiAgICAgICAgICBFeHBpcmVzIE1heSAx NywgMjAwOCAgICAgICAgICAgICAgICAgW1BhZ2UgMzVdCgwKCg== ------_=_NextPart_001_01C826B9.49841AAA Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn ------_=_NextPart_001_01C826B9.49841AAA-- From pcn-bounces@ietf.org Wed Nov 14 07:27:32 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IsHLA-0007PN-K1; Wed, 14 Nov 2007 07:27:32 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IsHL9-0007KE-4v for pcn-confirm+ok@megatron.ietf.org; Wed, 14 Nov 2007 07:27:31 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IsHL8-0007IA-PU for pcn@ietf.org; Wed, 14 Nov 2007 07:27:30 -0500 Received: from imr1.ericy.com ([198.24.6.9]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IsHL5-0007mS-CV for pcn@ietf.org; Wed, 14 Nov 2007 07:27:30 -0500 Received: from eusrcmw750.eamcs.ericsson.se (eusrcmw750.exu.ericsson.se [138.85.77.50]) by imr1.ericy.com (8.13.1/8.13.1) with ESMTP id lAECS4gA018564; Wed, 14 Nov 2007 06:28:04 -0600 Received: from eusrcmw721.eamcs.ericsson.se ([138.85.77.21]) by eusrcmw750.eamcs.ericsson.se with Microsoft SMTPSVC(6.0.3790.1830); Wed, 14 Nov 2007 06:27:23 -0600 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [PCN] 1st call - agenda requests for Vancouver Date: Wed, 14 Nov 2007 06:27:23 -0600 Message-ID: In-Reply-To: <20071107033818.610D261B673@newdev.eecs.harvard.edu> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] 1st call - agenda requests for Vancouver Thread-Index: Acgg8A577NBAKxRCQTWX0rqdtzpQnAFyVuvg References: <20071107033818.610D261B673@newdev.eecs.harvard.edu> From: "Anurag Bhargava" To: "Scott O. Bradner" , X-OriginalArrivalTime: 14 Nov 2007 12:27:23.0237 (UTC) FILETIME=[B549B150:01C826B9] X-Spam-Score: 0.0 (/) X-Scan-Signature: 8abaac9e10c826e8252866cbe6766464 Cc: X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Scott/Steve, We would also like to request for a slot to present 02 version of draft-westberg-pcn-load-control. Thanks -Anurag Bhargava, Ph.D. =20 >-----Original Message----- >From: Scott O. Bradner [mailto:sob@harvard.edu]=20 >Sent: Tuesday, November 06, 2007 10:38 PM >To: pcn@ietf.org >Subject: [PCN] 1st call - agenda requests for Vancouver > > >this is a first call for timeslots on the PCN agenda in Vancouver > >we are currently scheduled for wed afternoon from 1300-1610 > >we expect that a good chunk of the time will be spent on=20 >discussions of the Architecture draft but if others have other=20 >(in-charter) topics please let us know > >Scott & Steve > > >_______________________________________________ >PCN mailing list >PCN@ietf.org >https://www1.ietf.org/mailman/listinfo/pcn > _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 15 09:56:00 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Isg8O-0006a3-4z; Thu, 15 Nov 2007 09:56:00 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Isg8N-0006Xi-K6 for pcn-confirm+ok@megatron.ietf.org; Thu, 15 Nov 2007 09:55:59 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Isg8N-0006Uz-5e for pcn@ietf.org; Thu, 15 Nov 2007 09:55:59 -0500 Received: from rotterdam.ewi.utwente.nl ([130.89.10.5]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1Isg8M-0007u1-3U for pcn@ietf.org; Thu, 15 Nov 2007 09:55:59 -0500 Received: from utip105 (utip105.ewi.utwente.nl [130.89.13.76]) by rotterdam.ewi.utwente.nl (8.13.6/8.13.6) with ESMTP id lAFEtofk008736; Thu, 15 Nov 2007 15:55:55 +0100 (MET) From: "Georgios Karagiannis" To: , "'Geib, Ruediger'" References: <1B6169C658325341A3B8066E23919E1C4C143A@S4DE8PSAANK.mitte.t-com.de> <473AABCB.1000608@informatik.uni-wuerzburg.de> Subject: RE: [PCN] probing functionality Date: Thu, 15 Nov 2007 15:55:45 +0100 Message-ID: <001a01c82797$9ceb9950$4c0d5982@dynamic.ewi.utwente.nl> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Office Outlook 11 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3198 In-Reply-To: <473AABCB.1000608@informatik.uni-wuerzburg.de> thread-index: AcgmlhuYlQNhvbLbSjGytAXTXZqXoABAKtfg X-Spam-Score: 0.087 () AWL X-Scanned-By: MIMEDefang 2.52 on 130.89.10.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0rc3 (rotterdam.ewi.utwente.nl [130.89.10.5]); Thu, 15 Nov 2007 15:55:56 +0100 (MET) X-Spam-Score: 0.0 (/) X-Scan-Signature: 2a9ffb6f997442a3b543bcdaf483b990 Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Michael and Rudieger You canb use the method that I have proposed not only on user packets, = but also on signaling messages, e.g., RSVP PATH or NSIS RESERVE, as long=20 as: * they follow as much as possible the path followed by the user = packets, * they can be associated with a certain flow and * these signaling messages will be=20 used by a a router that does not support the RSVP or NSIS protocol = suite. This could be done if e.g., these signaling messages carry a Router = Alert option! Best Regards, Georgios =20 > -----Original Message----- > From: Michael Menth [mailto:menth@informatik.uni-wuerzburg.de]=20 > Sent: woensdag 14 november 2007 9:03 > To: Geib, Ruediger > Cc: pcn@ietf.org > Subject: Re: [PCN] probing functionality >=20 > Hi Ruediger, >=20 > Geib, Ruediger wrote: > > Hello Michael, > > > > to complete the statements on RSVP, if signaling messages carry the=20 > > addresses of the flows they reserve resources for, also the=20 > refreshes=20 > > travel the same path as the data. So they can be used to=20 > identify the=20 > > flows already admitted, should termination be pending. This is a=20 > > simple way to identify admitted flows in the presence of ECMP and=20 > > pre-congestion. This won't work with aggregated reservations > True - this is a solution only for end-to-end RSVP. What you=20 > mention above would work with some kind of tunneling (MPLS,=20 > IP-in-IP).=20 > Therefore, we probably need to look at different deployment scenarios. >=20 > > or summary refreshes, obviously. > > =20 > As long as the first PATH message and RESV message are=20 > transmitted as single messages I don't see a problem. >=20 > > I wouldn't call that "probing", if it is just based on standard=20 > > signaling messages, which are transmitted anyway. > > =20 > Yes, this is rather lightweight in the sense that it doesn't=20 > require extra messages. Therefore, one even hesitates to call=20 > it probing.=20 > Nevertheless, the PATH message is reused for probing and=20 > fulfills this function. >=20 > Regards, >=20 > Michael >=20 > > Regards, > > > > Rudiger > > > > |-----Original Message----- > > |From: Michael Menth [mailto:menth@informatik.uni-wuerzburg.de] > > |Sent: Tuesday, November 13, 2007 7:15 PM > > |To: Geib, R=FCdiger > > |Cc: pcn@ietf.org > > |Subject: Re: [PCN] probing functionality > > | > > | > > |Hi Ruediger, > > | > > |what you describe is how we also think that probing could=20 > be realized=20 > > |when RSVP or some other signalling protocol is used end-to-end. > > | > > |The initial RSVP PATH message is used for probing: if the PATH=20 > > |message arrives at the PCN egress and is not marked, the=20 > PCN egress=20 > > |node proceeds as a usual RSVP node and forwards the=20 > message further=20 > > |downstream. But if the message is marked, the PCN egress=20 > node sends=20 > > |back a PATHERR message to reject the flow. Conversely, if a RESV=20 > > |message returns, this implies that the PATH message was=20 > not marked at=20 > > |the PCN egress node. Therefore, the PCN ingress node can=20 > accept the=20 > > |new flow when it receives a RESV message. > > | > > |In 3sm and CL, all packets are marked with "admission-stop"=20 > > |when the PCN > > |traffic rate exceeds the admissible rate of a link. If the RSVP=20 > > |control messages are transmitted as PCN traffic, they are=20 > also marked=20 > > |with "admission-stop" in case of pre-congestion. Therefore, no=20 > > |special treatment of such probing traffic is needed inside the PCN=20 > > |domain. > > | > > |Regards, > > | > > | Michael > > | > > | > > |Geib, Ruediger wrote: > > |> Georgios, > > |> > > |> maybe I'm wrong, but I think you were the one to mention that=20 > > |> probing could be a single packet, which would be the RSVP PATH=20 > > |> message. The RSVP path message or a similar message of another=20 > > |> protocol is transmitted from PCN ingress node to PCN egress node=20 > > |> anyway prior to flow admission and it probably has the=20 > RAO set. If=20 > > |> we rely on ECMP being based on addresses, it travels the=20 > way of the=20 > > |> flow to be admitted. > > |> > > |> Couldn't we use this single signaling packet to indicate pending=20 > > |> congestion to an egress node? Two cases may apply: > > |> > > |> 1)on IP, the packet could be PCN marked by the congested=20 > > |> router once it notes, that the packet is to be forwarded=20 > > |> across a congested PCN interface. It is forwarded on=20 > > |> slow path due to RAO, special marking shouldn't be an=20 > > |> issue. > > |> > > |> 2)if RAO can't be applied, because slow path processing is=20 > > |> not desired within the PCN domain or it is impossible=20 > > |> (as in the MPLS case), then this signaling packet should=20 > > |> be marked once traffic approaches the termination=20 > > |> threshold. That means that once the termination threshold=20 > > |> is approached, all packets crossing a link must be=20 > > |> pre-congestion marked.=20 > > |> > > |> None of the above two requires special probings to be build and=20 > > |> allows PCN at least make entering the termination mode=20 > due to over=20 > > |> admission improbable, also if ECMP is present. > > |> Ramp marking remains an applicable option in the admission=20 > > |> threshold region. > > |> > > |> It is a requirement however, that source and destination=20 > address of=20 > > |> the flow to be admitted are used for signaling between=20 > PCN ingress=20 > > |> and egress node prior to flow admission. > > |> > > |> Your suggestion addresses point 1) above, but you don't=20 > explicitely=20 > > |> suggest to use RSVP below. Thus your packet always=20 > travels the same=20 > > |> path as the flow to be admitted, but it no longer must be a=20 > > |> signaling packet. I'd prefer to stick with the RSVP/NSIS packet=20 > > |> approach and suggest recommending to restrict ECMP to hash on=20 > > |> addresses if PCN and ECMP are operated in a single domain. > > |> > > |> Regards, > > |> > > |> Rudiger > > |> > > |> |-----Original Message----- > > |> |From: Georgios Karagiannis [mailto:karagian@cs.utwente.nl] > > |> |Sent: Friday, November 09, 2007 1:41 PM > > |> |To: pcn@ietf.org > > |> |Subject: [PCN] probing functionality > > |> | > > |> | > > |> |Hi all > > |> | > > |> |Another method that can be used for PCN probing can be=20 > applied as=20 > > |> |follows. > > |> |The probe packets are generated by the PCN_ingress_nodes in the=20 > > |> |following way. > > |> |A probe packet belonging to a certain micro-flow is > > |generated using the > > |> |same 5 touple (source, destination IP address, source,=20 > destination=20 > > |> |ports, protocol number) as the packets belonging to the same=20 > > |> |micro-flow. In addition to this the PCN_ingress_node has to set=20 > > |> |the Router Alert IP option on the probe packet. In this way all=20 > > |> |the > > |PCN_interior_node will > > |> |have to observe the received probe packets. > > |> | > > |> |Thus if a PCN_interior_node receives a probe packet=20 > then, due to=20 > > |> |the Router Alert option it has to handle it differently=20 > then the=20 > > |> |user packets. > > |> |The PCN_interior_node has to PCN_mark the probe packet if it is=20 > > |> |operating in admission control state (or flow=20 > termination state). > > |Otherwise the > > |> |probe packet remains unmarked. > > |> | > > |> |Best regards, > > |> |Georgios > > |> | > > |> | > > |> |_______________________________________________ > > |> |PCN mailing list > > |> |PCN@ietf.org > > |> |https://www1.ietf.org/mailman/listinfo/pcn > > |> | > > |> > > |> > > |> _______________________________________________ > > |> PCN mailing list > > |> PCN@ietf.org > > |> https://www1.ietf.org/mailman/listinfo/pcn > > |> =20 > > | > > |-- > > |Dr. Michael Menth, Assistant Professor University of Wuerzburg,=20 > > |Institute of Computer Science Am Hubland, D-97074=20 > Wuerzburg, Germany,=20 > > |room B206 > > |phone: (+49)-931/888-6644, fax: (+49)-931/888-6632=20 > > |mailto:menth@informatik.uni-wuerzburg.de > > |http://www3.informatik.uni-wuerzburg.de/research/ngn > > | > > | > > | > > |_______________________________________________ > > |PCN mailing list > > |PCN@ietf.org > > |https://www1.ietf.org/mailman/listinfo/pcn > > | > > =20 >=20 > -- > Dr. Michael Menth, Assistant Professor > University of Wuerzburg, Institute of Computer Science Am=20 > Hubland, D-97074 Wuerzburg, Germany, room B206 > phone: (+49)-931/888-6644, fax: (+49)-931/888-6632=20 > mailto:menth@informatik.uni-wuerzburg.de > http://www3.informatik.uni-wuerzburg.de/research/ngn >=20 >=20 >=20 > _______________________________________________ > PCN mailing list > PCN@ietf.org > https://www1.ietf.org/mailman/listinfo/pcn >=20 _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 15 10:06:31 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IsgIY-0002aa-Vx; Thu, 15 Nov 2007 10:06:30 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IsgIX-0002ZL-7C for pcn-confirm+ok@megatron.ietf.org; Thu, 15 Nov 2007 10:06:29 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IsgIW-0002ZD-Qf for pcn@ietf.org; Thu, 15 Nov 2007 10:06:28 -0500 Received: from rotterdam.ewi.utwente.nl ([130.89.10.5]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IsgIU-0008Pj-SQ for pcn@ietf.org; Thu, 15 Nov 2007 10:06:28 -0500 Received: from utip105 (utip105.ewi.utwente.nl [130.89.13.76]) by rotterdam.ewi.utwente.nl (8.13.6/8.13.6) with ESMTP id lAFF6Ef5011225; Thu, 15 Nov 2007 16:06:23 +0100 (MET) From: "Georgios Karagiannis" To: , References: <75A199C5D243C741BF3D3F1EBCEF9BA503B34324@E03MVZ1-UKDY.domain1.systemhost.net> Subject: RE: [PCN] new description of LC-PCN algorithm Date: Thu, 15 Nov 2007 16:06:09 +0100 Message-ID: <001b01c82799$13128700$4c0d5982@dynamic.ewi.utwente.nl> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 11 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3198 In-Reply-To: <75A199C5D243C741BF3D3F1EBCEF9BA503B34324@E03MVZ1-UKDY.domain1.systemhost.net> thread-index: AcgizlYG4BTllksSR8iEau+pndg02AAIsXPwASnkNdA= X-Spam-Score: 0.087 () AWL X-Scanned-By: MIMEDefang 2.52 on 130.89.10.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0rc3 (rotterdam.ewi.utwente.nl [130.89.10.5]); Thu, 15 Nov 2007 16:06:26 +0100 (MET) X-Spam-Score: 0.0 (/) X-Scan-Signature: 8e140a89d08e89747ee196e282ac2228 Cc: X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Phil Regarding your comments please note that I will be happy to participate in the algorithm comparison draft! I think that your questions (which are not related to LC-PCN but also to other proposals) can be asnwered within this algorithm comparison draft! Best regards, Georgios > -----Original Message----- > From: philip.eardley@bt.com [mailto:philip.eardley@bt.com] > Sent: vrijdag 9 november 2007 18:28 > To: karagian@cs.utwente.nl; pcn@ietf.org > Subject: RE: [PCN] new description of LC-PCN algorithm > > Thanks Georgios > > Rather than be swamped by detail on a Friday evening, I > wanted to make some high level points. > > - the PCN-affected-marking change is (I think) sensible. I > think this idea could be added (perhaps as an option for > operator) to any of the other 3 schemes. So it might be > valuable to have a short email /draft that only talked about > this idea? an analysis of the benefit, so can judge whether > it's worth the pain (the extra encoding point required). > The benefit may also depend on what marking behaviour is > standardised? > > - at a high level your proposal seems gradually to be looking > more similar to the single marking idea, ie rate based > marking behaviour for any rate above PCN-lower-rate ("adm > marking" & "termination marking") (yes I know your algorithm > changes above PCN-upper-rate, but only slightly). Of course > there are differences. However, I wonder how much they matter > at the moment; we have to either select which high level > approach to standardise or else standardise in a way that > allows more than one approach through configurable parameters > (see draft anna sent yesterday). > > - I wonder if it would be worth looking at the differences > one at a time and seeing if single marking could be improved > by adding it? For example, one difference is that you swap > back & forth between rates in bit/s and rates as % capacity > (of PHB) - I admit I can't really see the benefit and the > downside is that you have extra globally fixed parameters (eg > {PCN-upper-rate = PCN-lower-rate}). There are some > differences (I think fairly minor ones) in terms of handling > already marked pkts that are worth looking at. The idea about > adjusting the marking behaviour to account for some flows > already being in the middle of being terminated - we should > discuss in a wider context (not just applicable to lc-pcn) > (personally at the moment I don't like the idea, as I said in > previous email I think it's better adjusting at the > PCN-boundary-node than at the PCN-interior-node). Especially > important are any differences in the PCN-interior-node > marking behaviour. > > Have a good weekend, > Phil/ > > > -----Original Message----- > > From: Georgios Karagiannis [mailto:karagian@cs.utwente.nl] > > Sent: 09 November 2007 12:45 > > To: pcn@ietf.org > > Subject: [PCN] new description of LC-PCN algorithm > > > > Hi Anna, Hi Phil, Hi all > > > > Based on the comments that I have got so far I have tried to modify > the > > LC-PCN algorithm. > > > > The main changes are: > > * PCN_affected_marking is only used during flow termination. In this > way > > the ECMP solution during flow termination can be completely solved. > > > > * Probing is changed: > > Regarding probe generation, it is important to note that > probe packets > > are generated by the PCN_ingress_nodes in the following way. > > A probe packet belonging to a certain micro-flow is generated using > the > > same 5 touple (source, destination IP address, source, destination > ports, > > protocol number) > > as the packets belonging to the same micro-flow. In > addition to this > > the PCN_ingress_node has to set the Router Alert IP option on the > probe > > packet. In this way all > > the PCN_interior_node will have to observe the received > probe packets. > > > > Thus if a PCN_interior_node receives a probe packet then, > due to the > > Router Alert option it has to handle it differently then the user > packets. > > The PCN_interior_node has to PCN_mark the probe packet if it is > operating > > in admission control > > state (or flow termination state). Otherwise the probe > packet remains > > unmarked. > > > > * The PCN_egress_node changes to flow termination state if > it recieves > > at least one PCN_Affected_marked packet. > > > > Below I am describing the new proposed LC-PCN operation: > > > > * Setting the thresholds at PCN_interior_nodes: > > ----------------------------------------------- > > In order to calculate the PCN_upper_rate we use two parameters: > > Maximum PHB capacity: that is the maximum capacity that can be > supported > > by a PCN_interior_node > > Termination_offset_rate: that is an absolute rate value > that should be > set > > equal into all PCN_interior_nodes. Note that this value is used by > > PCN_interior_nodes to calculate their PCN_upper_rate and > also during > > the situation that a PCN_interior_node is in flow > termination state > > and it receives PCN_marked packets. Please see pseudo code on page > 20. > > This value must be set equal into all PCN_interior_nodes > such that all > > these nodes will know when to take into account the incoming > > PCN_marked packets and when not. The > Termination_offset_rate is needed > > due to the following fact. > > Consider the fact that when the measured PHB rate exceeds > the "Maximum > > PHB capacity" than the packets belonging to the given PHB will be > > either dropped or set to another PHB. > > In the situation of multiple severe congestion situations > solving the > > severe congestion on a severe congestion point, further > away than the > > PCN_egress_node, say severe_congestion_point_1, it could cause the > > situation that the severe congestion on a node located on the same > > path and closer to the PCN_egress_node, say > severe_congestion_point_2, > > will be solved without marking the excess rate measured at > > severe_congestion_point_2. This is however true only if the > measured > > PHB rate on severe_congestion_point_1 does not exceed the > "Maximum PHB > > capacity". > > This is due to the fact that before the > severe_congestion_point_1 goes > > into flow termination it generates a maximum PHB rate that > it does not > exceed > > its "Maximum PHB capacity" - termination_offset_rate and > after severe > > congestion it generates a maximum PHB rate not higher than "Maximum > > PHB capacity". > > Thus if the excess rate on severe_congestion_point_1 is higher than > > "Maximum PHB capacity" is not seen by > severe_congestion_point_2 but, > > due to the principle of marking, it will be seen by the > > PCN_egress_nodes. > > Therefore, the severe_congestion_point_2 has to consider the > > incoming_PCN_marked_rate from severe_congestion_point_1 in > its marking > > algorithm only for the rate equal to "Maximum PHB capaciy" - > > PCN_upper_rate (associated with severe_congestion_point_1). > Note that > > "Maximum PHB capacity" - PCN_upper_rate is equal to > > Termination_offset_rate. > > Note that the severe_congestion_point_2 can know the > > termination_offset_rate used by the previous severe > congestion point > > by using a variable that is equal into whole the PCN > domain. Note that > > the Termination_offset_rate can also be equal to 0. > > > > The PCN_upper_rate is then found as: > > PCN_upper_rate = "Maximum PHB capacity" - Termination_offset_rate > > > > The PCN_lower_rate is configured in all PCN_interior-nodes > and it can > be > > calculated in the following way: > > PCN_lower_rate = PCN_upper_rate - Admission_offset_rate > > > > The Admission_offset_rate is an absolute rate value and it > is equal in > > all PCN_interior_nodes and PCN_egress_nodes. Note that this > value is > > used by > > > > PCN_interior_nodes to calculate their PCN_lower_rate and the > > PCN_egress_nodes to calculate their PCN_upper_rate_egress. > Furthermore, > > this value is used by > > > > the PCN_interior_nodes also during the situation that a > PCN_interior_node > > is in admission control state and it receives PCN_marked packets. > Please > > see pseudo > > > > code on page 13. > > This value must be set equal into all PCN_interior_nodes > such that all > > these nodes will know when to take into account the incoming > PCN_marked > > packets and when not. Note that a PCN_interior_node can PCN_mark > packets > > up to an excess rate equal to the Admission_offset_rate. If > the exess > rate > > in an PCN_interior_node is higher than the > Admission_offset_rate, then > the > > PCN_interior_node changes state from admission control > state to flow > > termination state. > > > > * Setting the thresholds at PCN_egress_nodes: > > --------------------------------------------- > > The question is how to calculate the threshold that defines when a > > PCN_egress_node goes into the admission control state. > > One way to do that is to consider that when the PCN_egress_node > receives a > > PCN_marked packet it will mean that at least one PCN_interior_node > > started to be admission control congested and therefore it will go > > from Normal state to admission control state. Of course this will > > somehow might provide some errors, because there might be > situations > > that this consideration might be to conservative. > > Therefore, we use a factor of received PCN_marking encoded packets. > > > > PCN_lower_rate_egress: is equal to a fraction of the > > Admission_offset_rate, say fraction A * > Admission_offset_rate, where > > 0 preconfigured. > > Thus: PCN_lower_rate_egress = A * Admission_offset_rate. > > > > > > Thus we can say that a PCN_egress_node changes from Normal state to > > admission control state when incoming_PCN_marking_rate > > > PCN_lower_rate_egress. > > > > where, incoming_PCN_marking_rate = N * input_PCN_marking_bytes/T > > Where, input_PCN_marking_bytes = received number of "PCN_marking" > > encoded > > packets during measurement period T. > > > > Now we defined the condition that the PCN_egress_node changes state > from > > Normal state to admission control state. > > > > > > But how will the PCN_egress_node change state from admission control > state > > to flow termination state. > > > > There are two solutions for this: > > First solution: if the PCN_interior_nodes use the > PCN_Affected_marking > > encoding only during flow termination for the packets that > are passing > > through the severe congested node, but without being > PCN_marked, then > > the PCN_egress_node can change to flow termination state when it > > receives PCN_Affected_marked packets and change from flow > termnation > > state to normal state when it does not receive PCN_Affected_marked > > packets. > > > > Second solution: > > In order to explain this, it is important to note that each > > PCN_interior_node that is in admission control state it can > PCN_mark > > packets up to a > value > > equal to the Admission_offset_rate. > > > > Furthermore, if a PCN_interior_node receives incoming PCN_marked > packets > > and is in the addmission control state, it will not remark > any packets > if > > the excess rate is equal or lower than the > incoming_PCN_marking_rate, > > see page 13. > > Furthermore, if we will consider as normal situations the situations > that > > no ECMP occurs and that all flows belonging to the same > ingress-egress > > aggregate will use the same path from PCN_ingress to > PCN_egress, this > > will mean that when the PCN_egress_node receives, for the given > > ingress-egress aggregate an excess rate equal to a fraction of the > > Admission_offset_rate, say fraction F * Admission_offset_rate, where > > A>F>=1, > > it will have to change from admission control state to flow > termination > > state. > > Note that F can be preconfigured and depends on the network > topology. > > Thus in this case the second threshold, can be calculated > as follows: > > PCN_upper_egress_rate = PCN_lower_egress_rate + > F*Admission_offset_rate. > > However, there are some corner cases, that mainly occur when the > > different congestion points (admission control congested > > PCN_interior_nodes) on the same path are not > simulataneously starting > to > > be congested. Therefore we use the multicongestion_error > parameter to > > identify the error bound that ocurs due to these corner cases. Note > that > > this error bound can be e.g., predefined ones off line by the > > operator, by studying the network topology and/or studying > how often > > such corner cases could occur and/or doing off line > measurements. > > Therefore we use: > > PCN_upper_rate_egress = PCN_lower_rate_egress + > F*Admission_offset_rate > > +/- multicongestion_error > > > > I prefer to use the first solution, because then the PCN_egress_node > can > > solve completely the > > ECMP issue and the F factor and the multicongestion_error factor do > not > > have > > to be preconfigured anymore. > > * How the states of operation in PCN_interior_nodes are > being changed? > > > --------------------------------------------------------------------- > > This is explained on page 17, 18, by using Figure 4. > > > > Change from Normal state to Admission control state: event A Occurs > when: > > Measured PHB rate > PCN_lower_rate > > > > Change from Admission control state to Flow Termination > state: event > B > > Occurs when: > > Measured PHB rate > PCN_upper_rate > > > > * How the states of operation in PCN_egress_nodes are being changed? > > > --------------------------------------------------------------------- > > This is explained on page 21, 22, but they have to be modified. > > > > Change from Normal state to Admission control state: event A Occurs > when: > > incoming_PCN_marking_rate > PCN_lower_rate_egress > > > > Change from Admission control state to flow termination state: > > Using First solution mentioned above event B Occurs when the > > PCN_egress_node receives at least one PCN_Affected_marked packet. > > > > Using second solution: > > incoming_PCN_marking_rate > PCN_upper_rate_egress > > > > It is important to note that also the explanation of event C on page > 21, > > has to be modified, > > see explanation given above: > > Change from Admission control state to Normal state: > > Occurs when: > > > > incoming_PCN_marking_rate =< PCN_lower_rate > > > > Change from flow termination to Normal state: > > Uisng first solution: > > The PCN_egress_node does not receive any PCN_Affected_marked packets > > > > Using second solution: > > > > > > * Generated excess rate by an PCN_interior_node operating > in admission > > control state. > > ------------------------------------------------------------- > > > > The excess rate = signaled_overload_rate. > > The maximum excess rate that a PCN_interior_node can calculate in > > admission control state is equal to Admission_offset_rate, see > > explanation above. > > The number of bytes that are remarked, > signaled_remarked_bytes depend > > on the value of calculated excess rate > (signaled_overload_rate), > > the value of the Admission_offset_rate and the value of the > > incoming_PCN_marking_rate, see pseudo code on page 13. > > > > > > > > Regarding probes, and based on the received commenst by > Phil and Anna > > I modified the operation of probes. > > > > Regarding probe generation, it is important to note that > probe packets > > are generated by the PCN_ingress_nodes in the following way. > > A probe packet belonging to a certain micro-flow is generated using > the > > same 5 touple (source, destination IP address, source, destination > ports, > > protocol number) > > as the packets belonging to the same micro-flow. In > addition to this > > the PCN_ingress_node has to set the Router Alert IP option on the > probe > > packet. In this way all > > the PCN_interior_node will have to observe the received > probe packets. > > > > Thus if a PCN_interior_node receives a probe packet then, > due to the > > Router Alert option it has to handle it differently then the user > packets. > > The PCN_interior_node has to PCN_mark the probe packet if it is > operating > > in admission control > > state (or flow termination state). Otherwise the probe > packet remains > > unmarked. > > > > > > > > * Generating excess rate by an PCN_interior_node operating in flow > > termination state: > > ---------------------------------------------------------------- > > The excess rate = signaled_overload_rate. > > Note that the calculation of the signaled_overload_rate is different > than > > in the situation that the PCN_Interior_node operates in admission > control > > state, see page 19 and 20. This is due to the fact that a sliding > window > > is used to solve an undershooting problem, see discussion > on page 19. > > The number of bytes that are remarked, > signaled_remarked_bytes depend > on > > the value of calculated excess rate (signaled_overload_rate), the > value of > > the Termination_offset_rate and the value of the > > incoming_PCN_marking_rate, and the see pseudo code on page 20. > > > > Note that all packets that are passing through a congested > > PCN_interior_node and are not being PCN_marked by the > > PCN_interior_node have to be > remarked > > using the PCN_Affected_marking. > > > > * Providing admission control at PCN_egress_nodes: > > -------------------------------------------------- > > When the PCN_egress_node is operating in admission control > state than > > a flow that is requesting admission into the PCN domain can > be treated > > in the following way: > > > > If no probing is used, the request for admission can be accomplished > by > > using an external to PCN signaling protocol. In this case when the > > request arrives at a PCN_egress_node that operates in admission > > control state then the request is rejected. > > If it operates in Normal state is accepted. > > > > If probing is used, the request for admission is > accomplished by using > > probe packets. In this case when the probe arrives at a > > PCN_egress_node and it is PCN_marking encoded is rejected. > > Otherwise is accepted. > > > > * Providing flow termination at PCN_egress_nodes: > > -------------------------------------------------- > > > > When a PCN_egress_node operates on flow termination it > calculates the > > incoming excess rate (incoming_PCN_marking_rate). > > By using the excess rate the PCN_egress_node calclates the > number of > > flows that have to be terminated using the pseudo code given on page > 23. > > Note that the PCN_egress_node has to maintain per flow reservation > states. > > The information contained in the per flow states is used for the > > calculation of the flow that have to be terminated, see > pseudocode on > > page 23. Note also that this pseudo code uses priority > clases, but it > > operates when also no priority clases are used by setting > > Maximum_priority = 0 Where, 0 =< priority_class =< Maximum_priority > > > > Best regards, > > Georgios > > > > > > _______________________________________________ > > PCN mailing list > > PCN@ietf.org > > https://www1.ietf.org/mailman/listinfo/pcn > _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Sun Nov 18 21:44:07 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1ItwcJ-0002Fp-0Y; Sun, 18 Nov 2007 21:44:07 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1ItwcH-0002Fe-Nm for pcn-confirm+ok@megatron.ietf.org; Sun, 18 Nov 2007 21:44:05 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1ItwcH-0002FT-Ck for pcn@ietf.org; Sun, 18 Nov 2007 21:44:05 -0500 Received: from zcars04f.nortel.com ([47.129.242.57]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1ItwcF-0001io-3N for pcn@ietf.org; Sun, 18 Nov 2007 21:44:05 -0500 Received: from zcarhxm1.corp.nortel.com (zcarhxm1.corp.nortel.com [47.129.230.97]) by zcars04f.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id lAJ2i0e13313 for ; Mon, 19 Nov 2007 02:44:01 GMT X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Sun, 18 Nov 2007 21:43:57 -0500 Message-ID: <9671A92C3C8B5744BC97F855F7CB646513355439@zcarhxm1.corp.nortel.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: New Version Notification for draft-babiarz-pcn-3sm-01 Thread-Index: AcgqVHB+ZRFMCGSSSH2kSn80isr/UwAAQnMQ From: "Jozef Babiarz" To: X-Spam-Score: 0.0 (/) X-Scan-Signature: 9466e0365fc95844abaf7c3f15a05c7d Subject: [PCN] FW: New Version Notification for draft-babiarz-pcn-3sm-01 X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org A New Internet-Draft is available from the on-line Internet-Drafts directories. Title : Three State PCN Marking Author(s) : J. Babiarz, et al. Filename : draft-babiarz-pcn-3sm-01.txt Pages : 31 Date : 2007-11-18 This document proposes a mechanism for admission control and flow termination. It is based on the concept of pre-congestion notification (PCN) using three different codepoints: "no pre- congestion", "admission-stop", and "excess-traffic" for packet marking. Therefore, the proposal is called three state marking (3sm). The behaviour of edge nodes is presented which distinguishes from other proposals through little complexity and its ability to cope with multipath routing. Algorithms required for packet metering and marking are explained in detail. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-babiarz-pcn-3sm-01.txt=20 =20 Regards, Joe email:babiarz@nortel.com Telephone:613-763-6098 _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Mon Nov 19 03:50:45 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iu2L7-00034c-S8; Mon, 19 Nov 2007 03:50:45 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Iu2Ky-0002lw-9G for pcn-confirm+ok@megatron.ietf.org; Mon, 19 Nov 2007 03:50:36 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iu2Kq-0002YI-G8 for pcn@ietf.org; Mon, 19 Nov 2007 03:50:28 -0500 Received: from smtp.nokia.com ([192.100.105.134] helo=mgw-mx09.nokia.com) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1Iu2Kq-0006Tn-5G for pcn@ietf.org; Mon, 19 Nov 2007 03:50:28 -0500 Received: from esebh107.NOE.Nokia.com (esebh107.ntc.nokia.com [172.21.143.143]) by mgw-mx09.nokia.com (Switch-3.2.6/Switch-3.2.6) with ESMTP id lAJ8njPC004649; Mon, 19 Nov 2007 02:50:14 -0600 Received: from esebh102.NOE.Nokia.com ([172.21.138.183]) by esebh107.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.1830); Mon, 19 Nov 2007 10:49:52 +0200 Received: from mgw-int01.ntc.nokia.com ([172.21.143.96]) by esebh102.NOE.Nokia.com over TLS secured channel with Microsoft SMTPSVC(6.0.3790.1830); Mon, 19 Nov 2007 10:49:51 +0200 Received: from esdhcp03989.research.nokia.com (esdhcp03989.research.nokia.com [172.21.39.89]) by mgw-int01.ntc.nokia.com (Switch-3.2.5/Switch-3.2.5) with ESMTP id lAJ8nnEJ030816; Mon, 19 Nov 2007 10:49:50 +0200 Message-Id: <838171B8-DEE5-4F12-9F18-910533478F1D@nokia.com> From: Lars Eggert To: ext Georgios Karagiannis In-Reply-To: <7d6XOXRK.1194612076.8325830.karagian@ewi.utwente.nl> Mime-Version: 1.0 (Apple Message framework v915) Subject: Re: [PCN] probing functionality Date: Mon, 19 Nov 2007 10:49:49 +0200 References: <7d6XOXRK.1194612076.8325830.karagian@ewi.utwente.nl> X-Mailer: Apple Mail (2.915) X-OriginalArrivalTime: 19 Nov 2007 08:49:51.0182 (UTC) FILETIME=[25B8E6E0:01C82A89] X-Nokia-AV: Clean X-Spam-Score: 0.0 (/) X-Scan-Signature: 082a9cbf4d599f360ac7f815372a6a15 Cc: "pcn@ietf.org" X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1370297640==" Errors-To: pcn-bounces@ietf.org --===============1370297640== Content-Type: multipart/signed; boundary=Apple-Mail-20-259141511; micalg=sha1; protocol="application/pkcs7-signature" --Apple-Mail-20-259141511 Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit On 2007-11-9, at 14:41, ext Georgios Karagiannis wrote: > A probe packet belonging to a certain micro-flow is generated using > the > same 5 touple (source, destination IP address, source, destination > ports, > protocol number) as the packets belonging to the same micro-flow. In > addition to this the PCN_ingress_node has to set the Router Alert IP > option on the probe packet. In this way all the PCN_interior_node will > have to observe the received probe packets. FYI, during the IETF last call and the following IESG discussion around NSIS GIST, routing area folks claimed that NSIS' use of RAO was problematic. You probably want to ask some routing folks what they think of using RAO for this purpose. Lars --Apple-Mail-20-259141511 Content-Disposition: attachment; filename=smime.p7s Content-Type: application/pkcs7-signature; name=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIC/TCCAvkw ggJioAMCAQICEGtxkLFHQu1g67hlmYfwPuIwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UEBhMCWkEx JTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQ ZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA3MDEwNDE2MTE1NFoXDTA4MDEwNDE2MTE1 NFowXDEPMA0GA1UEBBMGRWdnZXJ0MQ0wCwYDVQQqEwRMYXJzMRQwEgYDVQQDEwtMYXJzIEVnZ2Vy dDEkMCIGCSqGSIb3DQEJARYVbGFycy5lZ2dlcnRAbm9raWEuY29tMIIBIjANBgkqhkiG9w0BAQEF AAOCAQ8AMIIBCgKCAQEA26hjntVPZMiwh4d8tyuk9KiucvG92BVUGArk5zO1jhmq7tLFgU1mqcIg VGumfQqjkAzIuh83kxIiqBrB05Wvxp85apwt4sUCvzMe8mQWzZZZYp0rBzwpOj+VC5pxsMRYtYW+ BaTFBEmvFr7D7C7roZUk0lDppS5SZFs4SQbEe9ykB9aeUf1iTiw2+ikyP2+JEYto14WYCoxFWbms FbQ1uaaK+yn1WH2YHhyMfi0IOxuT0jtbsngjHJMpWIqq334zBhj+rPSUug47YBHr41FCDMHbbbjv WbyTyDYneOW3WY8LY8bGWVlAknQGB7lgduShe6e0RI/7SL/VE6X27lM9LwIDAQABozIwMDAgBgNV HREEGTAXgRVsYXJzLmVnZ2VydEBub2tpYS5jb20wDAYDVR0TAQH/BAIwADANBgkqhkiG9w0BAQUF AAOBgQCx3nxxTg/WowobVTRXBiXUEEZj4LBiunWbuAnR0UbJIgijvq3JbiJvZo2gUGiW9LPaHSBz 4oxT1VR/S/6O0a4e87oV5cOQm6ReB68o9rDfUzxHTwoCfv2wwMwBrXyzQMjyQK4Lt8siV41gmZpm rSXMhjTJdd9uYYNoZKQLq+VM+TGCAxAwggMMAgEBMHYwYjELMAkGA1UEBhMCWkExJTAjBgNVBAoT HFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBG cmVlbWFpbCBJc3N1aW5nIENBAhBrcZCxR0LtYOu4ZZmH8D7iMAkGBSsOAwIaBQCgggFvMBgGCSqG SIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTA3MTExOTA4NDk1MFowIwYJKoZI hvcNAQkEMRYEFHT3jEJELPyoJnYyWP9Z8wvSPUUcMIGFBgkrBgEEAYI3EAQxeDB2MGIxCzAJBgNV BAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQDEyNU aGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQa3GQsUdC7WDruGWZh/A+4jCBhwYL KoZIhvcNAQkQAgsxeKB2MGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGlu ZyAoUHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBD QQIQa3GQsUdC7WDruGWZh/A+4jANBgkqhkiG9w0BAQEFAASCAQBTx9eWLW7MhTFmYUUpoQ7xXRJZ PkmbuXAAnpRKYNFyWducj+Bvz3C4BE1vFyvOGUBk3RjlQrVUhwxGCVmDCV51hVlAUSgOuByp0mFQ fyXpfY09QoB0Ithv/AKxnoPHBN62aWnMO38uyxspAi7KnueA40CXsAN27XT+rZTm8siy7LsnPYNm GZh9OUv0ty3PpJGbnFMuKTqonPeuBWBLI7I/+TTLwkQARtd3jJf2hSSrYmBlVtXbiYIHAP9qKLgx ufLOCg14BBynczJvMfqvLJVdfAJa8D0EqRKPc4C8Zr5JfWaoNMV1iuPMhuJJ35oI2RjHKdky8c6o zjLPfplOBOWgAAAAAAAA --Apple-Mail-20-259141511-- --===============1370297640== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn --===============1370297640==-- From pcn-bounces@ietf.org Mon Nov 19 09:00:29 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iu7Aq-0005cN-Mj; Mon, 19 Nov 2007 09:00:28 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Iu7Ap-0005c1-F4 for pcn-confirm+ok@megatron.ietf.org; Mon, 19 Nov 2007 09:00:27 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iu7Ao-0005bc-Us for pcn@ietf.org; Mon, 19 Nov 2007 09:00:26 -0500 Received: from zrtps0kn.nortel.com ([47.140.192.55]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1Iu7Al-0002rt-NB for pcn@ietf.org; Mon, 19 Nov 2007 09:00:26 -0500 Received: from zrtphxm1.corp.nortel.com (zrtphxm1.corp.nortel.com [47.140.202.50]) by zrtps0kn.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id lAJE0J306607 for ; Mon, 19 Nov 2007 14:00:19 GMT Received: from KCHAN-2K3.nortel.com ([47.130.16.174] RDNS failed) by zrtphxm1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.1830); Mon, 19 Nov 2007 08:59:52 -0500 X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Mon, 19 Nov 2007 09:00:16 -0500 To: pcn@ietf.org From: "Kwok-Ho Chan" Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Message-ID: X-OriginalArrivalTime: 19 Nov 2007 13:59:53.0178 (UTC) FILETIME=[75603BA0:01C82AB4] X-Spam-Score: 0.0 (/) X-Scan-Signature: 2857c5c041d6c02d7181d602c22822c8 Subject: [PCN] Fwd: I-D Action:draft-chan-pcn-encoding-comparison-01.txt X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org FYI, Updated PCN Encoding Comparison draft. -- Kwok -- >To: i-d-announce@ietf.org >Cc: >From: Internet-Drafts@ietf.org >Date: Mon, 19 Nov 2007 08:50:01 -0500 >X-Scan-Signature: 386e0819b1192672467565a524848168 >Subject: I-D Action:draft-chan-pcn-encoding-comparison-01.txt >X-BeenThere: i-d-announce@ietf.org >X-Mailman-Version: 2.1.5 >Reply-To: internet-drafts@ietf.org >List-Id: i-d-announce.ietf.org >List-Unsubscribe: , > >List-Archive: >List-Post: >List-Help: >List-Subscribe: , > >X-Spam-Score: 1.3 >X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on ertph000.nortel.com >X-Spam-Tests: FORGED_RCVD_HELO=0.05,MIME_BOUND_NEXTPART=0.106, > NO_REAL_NAME=0.178,TO_CC_NOT_NORTEL=1 >X-DNSBL-Score: -50 >X-DNSBL-Servers: bl.nortel.com >X-SMTP-HELO: megatron.ietf.org >X-SMTP-MAIL-FROM: i-d-announce-bounces@ietf.org >X-SMTP-RCPT-TO: >kboyle@nortel.com,bstucker@nortel.com,balles@nortel.com,bharrath@nortel.com,babiarz@nortel.com,aceler@nortel.com,huiwc@nortel.com,amuhanna@nortel.com,mchen@nortel.com,khchan@nortel.com >X-SMTP-PEER-INFO: odin.ietf.ORG [156.154.16.145] >X-SMTP-REASON: PASSED >X-SMTP-ID: 1195480342.03028435 >X-OriginalArrivalTime: 19 Nov 2007 13:51:59.0066 (UTC) >FILETIME=[5AC87BA0:01C82AB3] > >A New Internet-Draft is available from the on-line Internet-Drafts >directories. > > Title : Pre-Congestion Notification Encoding Comparison > Author(s) : K. Chan, G. Karagiannis > Filename : draft-chan-pcn-encoding-comparison-01.txt > Pages : 18 > Date : 2007-11-19 > >DiffServ mechanisms have been developed to support Quality of Service >(QoS). However, the level of assurance that can be provided with >DiffServ without substantial over-provisioning is limited. Pre- >Congestion Notification (PCN) investigates the use of per-flow >admission control to provide the required service guarantees for the >admitted traffic. While admission control will protect the QoS under >normal operating conditions, an additional flow termination mechanism >is necessary in the times of heavy congestion (e.g. caused by route >changes due to link or node failure). > >Encoding and their transport are required to carry the congestion and >pre-congestion information from the congestion and pre-congestion >points to the decision points. This document provides a survey of >several encoding methods, using comparisons amongst them as a way to >explain their strengths and weaknesses. > >A URL for this Internet-Draft is: >http://www.ietf.org/internet-drafts/draft-chan-pcn-encoding-comparison-01.txt > >To remove yourself from the I-D Announcement list, send a message to >i-d-announce-request@ietf.org with the word unsubscribe in the body of >the message. >You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce >to change your subscription settings. > >Internet-Drafts are also available by anonymous FTP. Login with the >username "anonymous" and a password of your e-mail address. After >logging in, type "cd internet-drafts" and then > "get draft-chan-pcn-encoding-comparison-01.txt". > >A list of Internet-Drafts directories can be found in >http://www.ietf.org/shadow.html >or ftp://ftp.ietf.org/ietf/1shadow-sites.txt > >Internet-Drafts can also be obtained by e-mail. > >Send a message to: > mailserv@ietf.org. >In the body type: > "FILE /internet-drafts/draft-chan-pcn-encoding-comparison-01.txt". > >NOTE: The mail server at ietf.org can return the document in > MIME-encoded form by using the "mpack" utility. To use this > feature, insert the command "ENCODING mime" before the "FILE" > command. To decode the response(s), you will need "munpack" or > a MIME-compliant mail reader. Different MIME-compliant mail readers > exhibit different behavior, especially when dealing with > "multipart" MIME messages (i.e. documents which have been split > up into multiple messages), so check your local documentation on > how to manipulate these messages. > >Below is the data which will enable a MIME compliant mail reader >implementation to automatically retrieve the ASCII version of the >Internet-Draft. > >Content-Type: text/plain >Content-ID: <2007-11-19084912.I-D@ietf.org> > >ENCODING mime >FILE /internet-drafts/draft-chan-pcn-encoding-comparison-01.txt > > > >_______________________________________________ >I-D-Announce mailing list >I-D-Announce@ietf.org >https://www1.ietf.org/mailman/listinfo/i-d-announce _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Mon Nov 19 09:14:36 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iu7OW-0007hv-KH; Mon, 19 Nov 2007 09:14:36 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Iu7OV-0007hV-Kh for pcn-confirm+ok@megatron.ietf.org; Mon, 19 Nov 2007 09:14:35 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iu7OV-0007hN-B5 for pcn@ietf.org; Mon, 19 Nov 2007 09:14:35 -0500 Received: from zcars04f.nortel.com ([47.129.242.57]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1Iu7OS-0004Ie-1y for pcn@ietf.org; Mon, 19 Nov 2007 09:14:35 -0500 Received: from zrtphxm1.corp.nortel.com (zrtphxm1.corp.nortel.com [47.140.202.50]) by zcars04f.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id lAJEETp19541; Mon, 19 Nov 2007 14:14:29 GMT Received: from KCHAN-2K3.nortel.com ([47.130.16.174] RDNS failed) by zrtphxm1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.1830); Mon, 19 Nov 2007 09:13:34 -0500 X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Mon, 19 Nov 2007 09:13:57 -0500 To: sob@harvard.edu (Scott O. Bradner), steven.blake@ericsson.com From: "Kwok-Ho Chan" Subject: Re: [PCN] 1st call - agenda requests for Vancouver In-Reply-To: <20071107033818.610D261B673@newdev.eecs.harvard.edu> References: <20071107033818.610D261B673@newdev.eecs.harvard.edu> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Message-ID: X-OriginalArrivalTime: 19 Nov 2007 14:13:34.0980 (UTC) FILETIME=[5F354440:01C82AB6] X-Spam-Score: 0.0 (/) X-Scan-Signature: 9182cfff02fae4f1b6e9349e01d62f32 Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Scott & Steve: I would like to request a time slot for: - the updated Encoding Comparison draft. Thanks! -- Kwok -- At 10:38 PM 11/6/2007, Scott O. Bradner wrote: >this is a first call for timeslots on the PCN agenda in Vancouver > >we are currently scheduled for wed afternoon from 1300-1610 > >we expect that a good chunk of the time will be spent on discussions of >the Architecture draft but if others have other (in-charter) topics >please let us know > >Scott & Steve > > >_______________________________________________ >PCN mailing list >PCN@ietf.org >https://www1.ietf.org/mailman/listinfo/pcn _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Mon Nov 19 10:20:06 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iu8Pu-0005gx-EO; Mon, 19 Nov 2007 10:20:06 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Iu8Pr-0005ad-Ak for pcn-confirm+ok@megatron.ietf.org; Mon, 19 Nov 2007 10:20:03 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iu8Pq-0005Yr-So; Mon, 19 Nov 2007 10:20:02 -0500 Received: from ns4.neustar.com ([156.154.24.139]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1Iu8Pq-0007Bc-HQ; Mon, 19 Nov 2007 10:20:02 -0500 Received: from stiedprstage1.ietf.org (stiedprstage1.va.neustar.com [10.31.47.10]) by ns4.neustar.com (Postfix) with ESMTP id 6910F2AC5D; Mon, 19 Nov 2007 15:20:02 +0000 (GMT) Received: from ietf by stiedprstage1.ietf.org with local (Exim 4.43) id 1Iu8Pq-0002pR-4n; Mon, 19 Nov 2007 10:20:02 -0500 Content-Type: Multipart/Mixed; Boundary="NextPart" Mime-Version: 1.0 To: i-d-announce@ietf.org From: Internet-Drafts@ietf.org Message-Id: Date: Mon, 19 Nov 2007 10:20:02 -0500 X-Spam-Score: 0.0 (/) X-Scan-Signature: 31247fb3be228bb596db9127becad0bc Cc: pcn@ietf.org Subject: [PCN] I-D Action:draft-ietf-pcn-architecture-02.txt X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org --NextPart A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Congestion and Pre-Congestion Notification Working Group of the IETF. Title : Pre-Congestion Notification Architecture Author(s) : P. Eardley Filename : draft-ietf-pcn-architecture-02.txt Pages : 45 Date : 2007-11-19 The purpose of this document is to describe a general architecture for flow admission and termination based on aggregated pre-congestion information in order to protect the quality of service of established inelastic flows within a single DiffServ domain.Status A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-pcn-architecture-02.txt To remove yourself from the I-D Announcement list, send a message to i-d-announce-request@ietf.org with the word unsubscribe in the body of the message. You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce to change your subscription settings. Internet-Drafts are also available by anonymous FTP. Login with the username "anonymous" and a password of your e-mail address. After logging in, type "cd internet-drafts" and then "get draft-ietf-pcn-architecture-02.txt". A list of Internet-Drafts directories can be found in http://www.ietf.org/shadow.html or ftp://ftp.ietf.org/ietf/1shadow-sites.txt Internet-Drafts can also be obtained by e-mail. Send a message to: mailserv@ietf.org. In the body type: "FILE /internet-drafts/draft-ietf-pcn-architecture-02.txt". NOTE: The mail server at ietf.org can return the document in MIME-encoded form by using the "mpack" utility. To use this feature, insert the command "ENCODING mime" before the "FILE" command. To decode the response(s), you will need "munpack" or a MIME-compliant mail reader. Different MIME-compliant mail readers exhibit different behavior, especially when dealing with "multipart" MIME messages (i.e. documents which have been split up into multiple messages), so check your local documentation on how to manipulate these messages. Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. --NextPart Content-Type: Multipart/Alternative; Boundary="OtherAccess" --OtherAccess Content-Type: Message/External-body; access-type="mail-server"; server="mailserv@ietf.org" Content-Type: text/plain Content-ID: <2007-11-19101818.I-D@ietf.org> ENCODING mime FILE /internet-drafts/draft-ietf-pcn-architecture-02.txt --OtherAccess Content-Type: Message/External-body; name="draft-ietf-pcn-architecture-02.txt"; site="ftp.ietf.org"; access-type="anon-ftp"; directory="internet-drafts" Content-Type: text/plain Content-ID: <2007-11-19101818.I-D\@ietf.org> --OtherAccess-- --NextPart Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn --NextPart-- From pcn-bounces@ietf.org Mon Nov 19 11:39:15 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iu9eT-0002OW-N0; Mon, 19 Nov 2007 11:39:13 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Iu9eS-0002Ns-Qx for pcn-confirm+ok@megatron.ietf.org; Mon, 19 Nov 2007 11:39:12 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iu9eS-0002NP-FY for pcn@ietf.org; Mon, 19 Nov 2007 11:39:12 -0500 Received: from smtp2.smtp.bt.com ([217.32.164.150]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1Iu9eR-0006Bk-Sf for pcn@ietf.org; Mon, 19 Nov 2007 11:39:12 -0500 Received: from E03MVZ1-UKDY.domain1.systemhost.net ([193.113.30.65]) by smtp2.smtp.bt.com with Microsoft SMTPSVC(6.0.3790.1830); Mon, 19 Nov 2007 16:39:10 +0000 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [PCN] I-D Action:draft-ietf-pcn-architecture-02.txt Date: Mon, 19 Nov 2007 16:38:43 -0000 Message-ID: <75A199C5D243C741BF3D3F1EBCEF9BA503B34377@E03MVZ1-UKDY.domain1.systemhost.net> In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] I-D Action:draft-ietf-pcn-architecture-02.txt Thread-Index: AcgqxqGl81rUg3oXSxKQPQt45B53dQAATG8g From: To: X-OriginalArrivalTime: 19 Nov 2007 16:39:10.0411 (UTC) FILETIME=[B5EE41B0:01C82ACA] X-Spam-Score: 1.1 (+) X-Scan-Signature: a2c12dacc0736f14d6b540e805505a86 X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org I updated the architecture draft with (I think) all the comments over the last month.=20 The main changes are:- (1) the OAM section is extensively revised, thanks to Bob Briscoe. Could another OAM person give this section a review please? (Tom T??) (2) the probing section is extended.=20 And a change to:- (3) discussion & rule for 'partially PCN-capable tunnels' (S5.7) On (2) I tried to capture all the recent discussion, hopefully fairly successfully & not too controversially? Also, because it was getting quite long I created a new section for all Probing text (S7). Personally I wonder if this is now getting too long & with too much discussion (of options and the circumstances when it might be a good idea or not) - so I wonder if it should be removed to another draft. Joe, I remember you saying you were (thinking of/) writing a new draft on probing, not sure if it would fit with that [sorry not to ask you before, but I only decided to write it at the weekend] Other minor changes=20 http://tools.ietf.org/rfcdiff?url1=3Dhttp://tools.ietf.org/id/draft-ietf-= p cn-architecture-02.txt&url2=3Dhttp://tools.ietf.org/id/draft-ietf-pcn-arc= h itecture-01.txt=20 Best wishes, Phil/ > -----Original Message----- > From: Internet-Drafts@ietf.org [mailto:Internet-Drafts@ietf.org] > Sent: 19 November 2007 15:20 > To: i-d-announce@ietf.org > Cc: pcn@ietf.org > Subject: [PCN] I-D Action:draft-ietf-pcn-architecture-02.txt >=20 > A New Internet-Draft is available from the on-line Internet-Drafts > directories. > This draft is a work item of the Congestion and Pre-Congestion > Notification Working Group of the IETF. >=20 >=20 > Title : Pre-Congestion Notification Architecture > Author(s) : P. Eardley > Filename : draft-ietf-pcn-architecture-02.txt > Pages : 45 > Date : 2007-11-19 >=20 > The purpose of this document is to describe a general architecture > for flow admission and termination based on aggregated pre-congestion > information in order to protect the quality of service of established > inelastic flows within a single DiffServ domain.Status >=20 > A URL for this Internet-Draft is: > http://www.ietf.org/internet-drafts/draft-ietf-pcn-architecture-02.txt >=20 > To remove yourself from the I-D Announcement list, send a message to > i-d-announce-request@ietf.org with the word unsubscribe in the body of > the message. > You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce > to change your subscription settings. >=20 > Internet-Drafts are also available by anonymous FTP. Login with the > username "anonymous" and a password of your e-mail address. After > logging in, type "cd internet-drafts" and then > "get draft-ietf-pcn-architecture-02.txt". >=20 > A list of Internet-Drafts directories can be found in > http://www.ietf.org/shadow.html > or ftp://ftp.ietf.org/ietf/1shadow-sites.txt >=20 > Internet-Drafts can also be obtained by e-mail. >=20 > Send a message to: > mailserv@ietf.org. > In the body type: > "FILE /internet-drafts/draft-ietf-pcn-architecture-02.txt". >=20 > NOTE: The mail server at ietf.org can return the document in > MIME-encoded form by using the "mpack" utility. To use this > feature, insert the command "ENCODING mime" before the "FILE" > command. To decode the response(s), you will need "munpack" or > a MIME-compliant mail reader. Different MIME-compliant mail readers > exhibit different behavior, especially when dealing with > "multipart" MIME messages (i.e. documents which have been split > up into multiple messages), so check your local documentation on > how to manipulate these messages. >=20 > Below is the data which will enable a MIME compliant mail reader > implementation to automatically retrieve the ASCII version of the > Internet-Draft. _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Mon Nov 19 12:05:53 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IuA4H-0007wP-LK; Mon, 19 Nov 2007 12:05:53 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IuA4G-0007vw-8e for pcn-confirm+ok@megatron.ietf.org; Mon, 19 Nov 2007 12:05:52 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IuA4F-0007vl-TI for pcn@ietf.org; Mon, 19 Nov 2007 12:05:51 -0500 Received: from sj-iport-2-in.cisco.com ([171.71.176.71] helo=sj-iport-2.cisco.com) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IuA4F-0000QQ-JN for pcn@ietf.org; Mon, 19 Nov 2007 12:05:51 -0500 Received: from sj-dkim-4.cisco.com ([171.71.179.196]) by sj-iport-2.cisco.com with ESMTP; 19 Nov 2007 09:05:51 -0800 Received: from sj-core-2.cisco.com (sj-core-2.cisco.com [171.71.177.254]) by sj-dkim-4.cisco.com (8.12.11/8.12.11) with ESMTP id lAJH5oiw027169 for ; Mon, 19 Nov 2007 09:05:51 -0800 Received: from xbh-rtp-211.amer.cisco.com (xbh-rtp-211.cisco.com [64.102.31.102]) by sj-core-2.cisco.com (8.12.10/8.12.6) with ESMTP id lAJH5d46007182 for ; Mon, 19 Nov 2007 17:05:50 GMT Received: from xmb-rtp-203.amer.cisco.com ([64.102.31.20]) by xbh-rtp-211.amer.cisco.com with Microsoft SMTPSVC(6.0.3790.1830); Mon, 19 Nov 2007 12:05:39 -0500 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Mon, 19 Nov 2007 12:05:38 -0500 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: draft-charny-pcn-single-marking-03 submitted Thread-Index: Acgqzmg5sIIPa99yTOCgeLLnxai1NQ== From: "Anna Charny (acharny)" To: X-OriginalArrivalTime: 19 Nov 2007 17:05:39.0900 (UTC) FILETIME=[695707C0:01C82ACE] X-TM-AS-Product-Ver: SMEX-8.0.0.1181-5.000.1023-15554.002 X-TM-AS-Result: No--0.638400-8.000000-31 X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No DKIM-Signature: v=0.5; a=rsa-sha256; q=dns/txt; l=398; t=1195491951; x=1196355951; c=relaxed/simple; s=sjdkim4002; h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version; d=cisco.com; i=acharny@cisco.com; z=From:=20=22Anna=20Charny=20(acharny)=22=20 |Subject:=20draft-charny-pcn-single-marking-03=20submitted |Sender:=20; bh=PcGz8zdQzRzz8t2qNqdpYQNGl07CcXq6p25K7aL4cG8=; b=meb3MuBXY1yu5mDAezp2LiJh4A+BS35EFxo2pMRWUbbNa1Xk8zuTIwQkVWhLr5qKnF04JHAF VYeLBIj3Q7JOc76uRIl4LgXmstW4Lin2h8a569YN9LX9XQ6TzqlZYFa+; Authentication-Results: sj-dkim-4; header.From=acharny@cisco.com; dkim=pass ( sig from cisco.com/sjdkim4002 verified; ); X-Spam-Score: 0.0 (/) X-Scan-Signature: cf4fa59384e76e63313391b70cd0dd25 Subject: [PCN] draft-charny-pcn-single-marking-03 submitted X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi all, We submitted an update to the Single Marking draft. The changes from the previous versions are: 1) quantification of performance tradeoffs for termination (sections 7.3 and 8.3 added) 2) alignment with draft-charny-pcn-comparison-00 3) misc minor edits The draft is available at http://www.ietf.org/internet-drafts/draft-charny-pcn-single-marking-03.t xt Best, Anna=20 _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Mon Nov 19 17:32:18 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IuFA9-0006X9-Tz; Mon, 19 Nov 2007 17:32:17 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IuFA8-0006X3-Dh for pcn-confirm+ok@megatron.ietf.org; Mon, 19 Nov 2007 17:32:16 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IuFA7-0006Wq-DZ for pcn@ietf.org; Mon, 19 Nov 2007 17:32:15 -0500 Received: from smtp101.rog.mail.re2.yahoo.com ([206.190.36.79]) by chiedprmail1.ietf.org with smtp (Exim 4.43) id 1IuFA7-0005qw-2H for pcn@ietf.org; Mon, 19 Nov 2007 17:32:15 -0500 Received: (qmail 60626 invoked from network); 19 Nov 2007 22:32:14 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=rogers.com; h=Received:X-YMail-OSG:Message-ID:Date:From:User-Agent:MIME-Version:To:CC:Subject:Content-Type:Content-Transfer-Encoding; b=x+lnxX5qLdm4nAK4a1sEjMjsdrt0yboq52P2aKsqqF3PM8O6lVeogz3EnGumtZ1S/8ib7nYP7iQWj8jLkIgHVNaLBBMXy+576I1tZbkW/+4QKUQBbTKpp/0Ev1IGQ1Zc8eOe6vnMpmrvrax5WZNS/gKxX8HDWpqfDwaXaxup2Ng= ; Received: from unknown (HELO ?218.18.236.230?) (tom.taylor@rogers.com@218.18.236.230 with plain) by smtp101.rog.mail.re2.yahoo.com with SMTP; 19 Nov 2007 22:32:14 -0000 X-YMail-OSG: 8qKe0X4VM1m11w42rgyHVz8_3Dvjfjxajl3gbHzYattbQih4deh4tFhoAQG08_z.qA-- Message-ID: <47420EEC.10807@rogers.com> Date: Mon, 19 Nov 2007 17:32:12 -0500 From: Tom Taylor User-Agent: Thunderbird 2.0.0.9 (Windows/20071031) MIME-Version: 1.0 To: pcn Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Scan-Signature: 93238566e09e6e262849b4f805833007 Cc: bob.briscoe@bt.com Subject: [PCN] [Fwd: Replacement of draft-briscoe-pcn-boundary-behav-00.txt with draft-tsou-pcn-boundary-behav-00.txt] X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org We were a bit previous in adding Bob's name to the draft without getting an explicit go-ahead. For that, we apologize to Bob. The draft will see a major rewrite to achieve its intended objectives. In the meantime, some of the questions it raises are receiving a fair amount of discussion behind the scenes. Tom Taylor -------- Original Message -------- Subject: Replacement of draft-briscoe-pcn-boundary-behav-00.txt with draft-tsou-pcn-boundary-behav-00.txt Date: Mon, 19 Nov 2007 12:15:02 -0500 From: IETF Secretariat To: Tom Taylor CC: Bob Briscoe , Tina Tsou , As you requested, draft-briscoe-pcn-boundary-behav-00.txt has been marked as replaced by draft-tsou-pcn-boundary-behav-00.txt in the IETF Internet-Drafts database. The IETF Secretariat. _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Tue Nov 20 03:19:07 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IuOK2-0002cP-R4; Tue, 20 Nov 2007 03:19:06 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IuOK2-0002cI-1T for pcn-confirm+ok@megatron.ietf.org; Tue, 20 Nov 2007 03:19:06 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IuOJv-00029o-Rg for pcn@ietf.org; Tue, 20 Nov 2007 03:19:00 -0500 Received: from smtp3.smtp.bt.com ([217.32.164.138]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IuOJk-0002F9-L0 for pcn@ietf.org; Tue, 20 Nov 2007 03:18:57 -0500 Received: from E03MVZ1-UKDY.domain1.systemhost.net ([193.113.30.65]) by smtp3.smtp.bt.com with Microsoft SMTPSVC(6.0.3790.1830); Tue, 20 Nov 2007 08:18:47 +0000 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----_=_NextPart_001_01C82B4D.F8E6A7B0" Subject: RE: [PCN] I-D Action:draft-ietf-pcn-architecture-02.txt Date: Tue, 20 Nov 2007 08:18:46 -0000 Message-ID: <75A199C5D243C741BF3D3F1EBCEF9BA503B34379@E03MVZ1-UKDY.domain1.systemhost.net> In-Reply-To: <75A199C5D243C741BF3D3F1EBCEF9BA503B34377@E03MVZ1-UKDY.domain1.systemhost.net> X-MS-Has-Attach: yes X-MS-TNEF-Correlator: Thread-Topic: [PCN] I-D Action:draft-ietf-pcn-architecture-02.txt Thread-Index: AcgqxqGl81rUg3oXSxKQPQt45B53dQAATG8gACGAyaA= From: To: X-OriginalArrivalTime: 20 Nov 2007 08:18:47.0568 (UTC) FILETIME=[F955B900:01C82B4D] X-Spam-Score: 1.1 (+) X-Scan-Signature: d6985da47950eee4daf1f525dc71d5d2 X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org This is a multi-part message in MIME format. ------_=_NextPart_001_01C82B4D.F8E6A7B0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable This hasn't yet appeared on the ietf server, so here it is > -----Original Message----- > From: philip.eardley@bt.com [mailto:philip.eardley@bt.com] > Sent: 19 November 2007 16:39 > To: pcn@ietf.org > Subject: RE: [PCN] I-D Action:draft-ietf-pcn-architecture-02.txt >=20 > I updated the architecture draft with (I think) all the comments over > the last month. >=20 > The main changes are:- > (1) the OAM section is extensively revised, thanks to Bob Briscoe. Could > another OAM person give this section a review please? (Tom T??) > (2) the probing section is extended. >=20 > And a change to:- > (3) discussion & rule for 'partially PCN-capable tunnels' (S5.7) >=20 > On (2) I tried to capture all the recent discussion, hopefully fairly > successfully & not too controversially? Also, because it was getting > quite long I created a new section for all Probing text (S7). Personally > I wonder if this is now getting too long & with too much discussion (of > options and the circumstances when it might be a good idea or not) - so > I wonder if it should be removed to another draft. Joe, I remember you > saying you were (thinking of/) writing a new draft on probing, not sure > if it would fit with that [sorry not to ask you before, but I only > decided to write it at the weekend] >=20 > Other minor changes > http://tools.ietf.org/rfcdiff?url1=3Dhttp://tools.ietf.org/id/draft-ietf-= p > cn-architecture-02.txt&url2=3Dhttp://tools.ietf.org/id/draft-ietf-pcn-arc= h > itecture-01.txt >=20 > Best wishes, > Phil/ >=20 > > -----Original Message----- > > From: Internet-Drafts@ietf.org [mailto:Internet-Drafts@ietf.org] > > Sent: 19 November 2007 15:20 > > To: i-d-announce@ietf.org > > Cc: pcn@ietf.org > > Subject: [PCN] I-D Action:draft-ietf-pcn-architecture-02.txt > > > > A New Internet-Draft is available from the on-line Internet-Drafts > > directories. > > This draft is a work item of the Congestion and Pre-Congestion > > Notification Working Group of the IETF. > > > > > > Title : Pre-Congestion Notification Architecture > > Author(s) : P. Eardley > > Filename : draft-ietf-pcn-architecture-02.txt > > Pages : 45 > > Date : 2007-11-19 > > > > The purpose of this document is to describe a general architecture > > for flow admission and termination based on aggregated pre-congestion > > information in order to protect the quality of service of established > > inelastic flows within a single DiffServ domain.Status > > > > A URL for this Internet-Draft is: > > http://www.ietf.org/internet-drafts/draft-ietf-pcn-architecture-02.txt > > > > To remove yourself from the I-D Announcement list, send a message to > > i-d-announce-request@ietf.org with the word unsubscribe in the body of > > the message. > > You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce > > to change your subscription settings. > > > > Internet-Drafts are also available by anonymous FTP. Login with the > > username "anonymous" and a password of your e-mail address. After > > logging in, type "cd internet-drafts" and then > > "get draft-ietf-pcn-architecture-02.txt". > > > > A list of Internet-Drafts directories can be found in > > http://www.ietf.org/shadow.html > > or ftp://ftp.ietf.org/ietf/1shadow-sites.txt > > > > Internet-Drafts can also be obtained by e-mail. > > > > Send a message to: > > mailserv@ietf.org. > > In the body type: > > "FILE /internet-drafts/draft-ietf-pcn-architecture-02.txt". > > > > NOTE: The mail server at ietf.org can return the document in > > MIME-encoded form by using the "mpack" utility. To use this > > feature, insert the command "ENCODING mime" before the "FILE" > > command. To decode the response(s), you will need "munpack" or > > a MIME-compliant mail reader. Different MIME-compliant mail > readers > > exhibit different behavior, especially when dealing with > > "multipart" MIME messages (i.e. documents which have been split > > up into multiple messages), so check your local documentation on > > how to manipulate these messages. > > > > Below is the data which will enable a MIME compliant mail reader > > implementation to automatically retrieve the ASCII version of the > > Internet-Draft. >=20 >=20 > _______________________________________________ > PCN mailing list > PCN@ietf.org > https://www1.ietf.org/mailman/listinfo/pcn ------_=_NextPart_001_01C82B4D.F8E6A7B0 Content-Type: text/plain; name="draft-ietf-pcn-architecture-02.txt" Content-Transfer-Encoding: base64 Content-Description: draft-ietf-pcn-architecture-02.txt Content-Disposition: attachment; filename="draft-ietf-pcn-architecture-02.txt" DQoNCg0KQ29uZ2VzdGlvbiBhbmQgUHJlLUNvbmdlc3Rpb24gICAgICAgICAgICAgICAgICAgUGhp bGlwLiBFYXJkbGV5IChFZGl0b3IpDQpOb3RpZmljYXRpb24gV29ya2luZyBHcm91cCAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgQlQNCkludGVybmV0LURyYWZ0ICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBOb3ZlbWJlciAxOSwgMjAwNw0K SW50ZW5kZWQgc3RhdHVzOiBJbmZvcm1hdGlvbmFsDQpFeHBpcmVzOiBNYXkgMjIsIDIwMDgNCg0K DQogICAgICAgICAgICAgICAgUHJlLUNvbmdlc3Rpb24gTm90aWZpY2F0aW9uIEFyY2hpdGVjdHVy ZQ0KICAgICAgICAgICAgICAgICAgICAgZHJhZnQtaWV0Zi1wY24tYXJjaGl0ZWN0dXJlLTAyDQoN ClN0YXR1cyBvZiB0aGlzIE1lbW8NCg0KICAgQnkgc3VibWl0dGluZyB0aGlzIEludGVybmV0LURy YWZ0LCBlYWNoIGF1dGhvciByZXByZXNlbnRzIHRoYXQgYW55DQogICBhcHBsaWNhYmxlIHBhdGVu dCBvciBvdGhlciBJUFIgY2xhaW1zIG9mIHdoaWNoIGhlIG9yIHNoZSBpcyBhd2FyZQ0KICAgaGF2 ZSBiZWVuIG9yIHdpbGwgYmUgZGlzY2xvc2VkLCBhbmQgYW55IG9mIHdoaWNoIGhlIG9yIHNoZSBi ZWNvbWVzDQogICBhd2FyZSB3aWxsIGJlIGRpc2Nsb3NlZCwgaW4gYWNjb3JkYW5jZSB3aXRoIFNl Y3Rpb24gNiBvZiBCQ1AgNzkuDQoNCiAgIEludGVybmV0LURyYWZ0cyBhcmUgd29ya2luZyBkb2N1 bWVudHMgb2YgdGhlIEludGVybmV0IEVuZ2luZWVyaW5nDQogICBUYXNrIEZvcmNlIChJRVRGKSwg aXRzIGFyZWFzLCBhbmQgaXRzIHdvcmtpbmcgZ3JvdXBzLiAgTm90ZSB0aGF0DQogICBvdGhlciBn cm91cHMgbWF5IGFsc28gZGlzdHJpYnV0ZSB3b3JraW5nIGRvY3VtZW50cyBhcyBJbnRlcm5ldC0N CiAgIERyYWZ0cy4NCg0KICAgSW50ZXJuZXQtRHJhZnRzIGFyZSBkcmFmdCBkb2N1bWVudHMgdmFs aWQgZm9yIGEgbWF4aW11bSBvZiBzaXggbW9udGhzDQogICBhbmQgbWF5IGJlIHVwZGF0ZWQsIHJl cGxhY2VkLCBvciBvYnNvbGV0ZWQgYnkgb3RoZXIgZG9jdW1lbnRzIGF0IGFueQ0KICAgdGltZS4g IEl0IGlzIGluYXBwcm9wcmlhdGUgdG8gdXNlIEludGVybmV0LURyYWZ0cyBhcyByZWZlcmVuY2UN CiAgIG1hdGVyaWFsIG9yIHRvIGNpdGUgdGhlbSBvdGhlciB0aGFuIGFzICJ3b3JrIGluIHByb2dy ZXNzLiINCg0KICAgVGhlIGxpc3Qgb2YgY3VycmVudCBJbnRlcm5ldC1EcmFmdHMgY2FuIGJlIGFj Y2Vzc2VkIGF0DQogICBodHRwOi8vd3d3LmlldGYub3JnL2lldGYvMWlkLWFic3RyYWN0cy50eHQu DQoNCiAgIFRoZSBsaXN0IG9mIEludGVybmV0LURyYWZ0IFNoYWRvdyBEaXJlY3RvcmllcyBjYW4g YmUgYWNjZXNzZWQgYXQNCiAgIGh0dHA6Ly93d3cuaWV0Zi5vcmcvc2hhZG93Lmh0bWwuDQoNCiAg IFRoaXMgSW50ZXJuZXQtRHJhZnQgd2lsbCBleHBpcmUgb24gTWF5IDIyLCAyMDA4Lg0KDQpDb3B5 cmlnaHQgTm90aWNlDQoNCiAgIENvcHlyaWdodCAoQykgVGhlIElFVEYgVHJ1c3QgKDIwMDcpLg0K DQpBYnN0cmFjdA0KDQogICBUaGUgcHVycG9zZSBvZiB0aGlzIGRvY3VtZW50IGlzIHRvIGRlc2Ny aWJlIGEgZ2VuZXJhbCBhcmNoaXRlY3R1cmUNCiAgIGZvciBmbG93IGFkbWlzc2lvbiBhbmQgdGVy bWluYXRpb24gYmFzZWQgb24gYWdncmVnYXRlZCBwcmUtY29uZ2VzdGlvbg0KICAgaW5mb3JtYXRp b24gaW4gb3JkZXIgdG8gcHJvdGVjdCB0aGUgcXVhbGl0eSBvZiBzZXJ2aWNlIG9mIGVzdGFibGlz aGVkDQogICBpbmVsYXN0aWMgZmxvd3Mgd2l0aGluIGEgc2luZ2xlIERpZmZTZXJ2IGRvbWFpbi4N Cg0KDQoNCg0KDQoNCkVhcmRsZXkgKEVkaXRvcikgICAgICAgICAgRXhwaXJlcyBNYXkgMjIsIDIw MDggICAgICAgICAgICAgICAgICBbUGFnZSAxXQ0KDA0KSW50ZXJuZXQtRHJhZnQgICAgICAgICAg ICAgICAgICBEb2N1bWVudCAgICAgICAgICAgICAgICAgICBOb3ZlbWJlciAyMDA3DQoNCg0KU3Rh dHVzDQoNCg0KDQoNCg0KDQoNCg0KDQoNCg0KDQoNCg0KDQoNCg0KDQoNCg0KDQoNCg0KDQoNCg0K DQoNCg0KDQoNCg0KDQoNCg0KDQoNCg0KDQoNCg0KDQoNCg0KDQoNCg0KDQoNCg0KRWFyZGxleSAo RWRpdG9yKSAgICAgICAgICBFeHBpcmVzIE1heSAyMiwgMjAwOCAgICAgICAgICAgICAgICAgIFtQ YWdlIDJdDQoMDQpJbnRlcm5ldC1EcmFmdCAgICAgICAgICAgICAgICAgIERvY3VtZW50ICAgICAg ICAgICAgICAgICAgIE5vdmVtYmVyIDIwMDcNCg0KDQpUYWJsZSBvZiBDb250ZW50cw0KDQogICAx LiAgSW50cm9kdWN0aW9uIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAu IC4gLiAuIC4gIDQNCiAgIDIuICBUZXJtaW5vbG9neSAgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAgNw0KICAgMy4gIEFzc3VtcHRpb25zIGFuZCBjb25z dHJhaW50cyBvbiBzY29wZSAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuICA5DQogICAgIDMuMS4g IEFzc3VtcHRpb24gMTogVHJ1c3QgLSBjb250cm9sbGVkIGVudmlyb25tZW50IC4gLiAuIC4gLiAu IC4gMTANCiAgICAgMy4yLiAgQXNzdW1wdGlvbiAyOiBSZWFsLXRpbWUgYXBwbGljYXRpb25zIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAxMA0KICAgICAzLjMuICBBc3N1bXB0aW9uIDM6IE1hbnkgZmxv d3MgYW5kIGFkZGl0aW9uYWwgbG9hZCAuIC4gLiAuIC4gLiAuIDExDQogICAgIDMuNC4gIEFzc3Vt cHRpb24gNDogRW1lcmdlbmN5IHVzZSBvdXQgb2Ygc2NvcGUgLiAuIC4gLiAuIC4gLiAuIC4gMTEN CiAgICAgMy41LiAgT3RoZXIgYXNzdW1wdGlvbnMgIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAxMQ0KICAgNC4gIEhpZ2gtbGV2ZWwgZnVuY3Rpb25hbCBhcmNoaXRlY3R1 cmUgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIDEyDQogICAgIDQuMS4gIEZsb3cgYWRtaXNz aW9uIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gMTINCiAgICAg NC4yLiAgRmxvdyB0ZXJtaW5hdGlvbiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAxMw0KICAgICA0LjMuICBGbG93IGFkbWlzc2lvbiBhbmQgZmxvdyB0ZXJtaW5hdGlv biAgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIDE0DQogICAgIDQuNC4gIEluZm9ybWF0aW9uIHRyYW5z cG9ydCAgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gMTUNCiAgICAgNC41LiAg UENOLXRyYWZmaWMgIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4g LiAxNQ0KICAgNS4gIERldGFpbGVkIEZ1bmN0aW9uYWwgYXJjaGl0ZWN0dXJlIC4gLiAuIC4gLiAu IC4gLiAuIC4gLiAuIC4gLiAuIDE2DQogICAgIDUuMS4gIFBDTi1pbnRlcmlvci1ub2RlIGZ1bmN0 aW9ucyAgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gMTcNCiAgICAgNS4yLiAgUENOLWlu Z3Jlc3Mtbm9kZSBmdW5jdGlvbnMgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAxNw0K ICAgICA1LjMuICBQQ04tZWdyZXNzLW5vZGUgZnVuY3Rpb25zICAuIC4gLiAuIC4gLiAuIC4gLiAu IC4gLiAuIC4gLiAuIDE4DQogICAgIDUuNC4gIEFkbWlzc2lvbiBjb250cm9sIGZ1bmN0aW9ucyAg LiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gMTgNCiAgICAgNS41LiAgRmxvdyB0ZXJtaW5h dGlvbiBmdW5jdGlvbnMgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAxOQ0KICAgICA1 LjYuICBBZGRyZXNzaW5nIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAu IC4gLiAuIDIwDQogICAgIDUuNy4gIFR1bm5lbGxpbmcgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gMjENCiAgICAgNS44LiAgRmF1bHQgaGFuZGxpbmcgLiAu IC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAyMg0KICAgNi4gIERlc2ln biBnb2FscyBhbmQgY2hhbGxlbmdlcyAgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAu IDIzDQogICA3LiAgUHJvYmluZyAgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gMjUNCiAgICAgNy4xLiAgSW50cm9kdWN0aW9uIC4gLiAuIC4gLiAu IC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAyNQ0KICAgICA3LjIuICBQcm9iaW5n IGZ1bmN0aW9ucyAgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIDI2DQog ICAgIDcuMy4gIERpc2N1c3Npb24gb2YgcmF0aW9uYWxlIGZvciBwcm9iaW5nLCBpdHMgZG93bnNp ZGVzIGFuZA0KICAgICAgICAgICBvcGVuIGlzc3VlcyAgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAuIDI3DQogICA4LiAgT3BlcmF0aW9ucyBhbmQgTWFuYWdlbWVu dCAgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gMzANCiAgICAgOC4xLiAgQ29u ZmlndXJhdGlvbiBPQU0gIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAz MA0KICAgICAgIDguMS4xLiAgU3lzdGVtIG9wdGlvbnMgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAuIDMwDQogICAgICAgOC4xLjIuICBQYXJhbWV0ZXJzIC4gLiAuIC4gLiAu IC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gMzENCiAgICAgOC4yLiAgUGVyZm9ybWFu Y2UgJiBQcm92aXNpb25pbmcgT0FNIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAzMw0KICAg ICA4LjMuICBBY2NvdW50aW5nIE9BTSAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIDM0DQogICAgIDguNC4gIEZhdWx0IE9BTSAgLiAuIC4gLiAuIC4gLiAuIC4gLiAu IC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gMzQNCiAgICAgOC41LiAgU2VjdXJpdHkgT0FNIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAzNQ0KICAgOS4gIElB TkEgQ29uc2lkZXJhdGlvbnMgIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4g LiAuIDM2DQogICAxMC4gU2VjdXJpdHkgY29uc2lkZXJhdGlvbnMgIC4gLiAuIC4gLiAuIC4gLiAu IC4gLiAuIC4gLiAuIC4gLiAuIC4gMzYNCiAgIDExLiBDb25jbHVzaW9ucyAgLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAzNw0KICAgMTIuIEFja25vd2xl ZGdlbWVudHMgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIDM3 DQogICAxMy4gQ29tbWVudHMgU29saWNpdGVkIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAu IC4gLiAuIC4gLiAuIC4gMzgNCiAgIDE0LiBDaGFuZ2VzICAuIC4gLiAuIC4gLiAuIC4gLiAuIC4g LiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAzOA0KICAgMTUuIEluZm9ybWF0aXZlIFJl ZmVyZW5jZXMgLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIDQwDQogICBB dXRob3IncyBBZGRyZXNzIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiAu IC4gLiAuIC4gNDQNCiAgIEludGVsbGVjdHVhbCBQcm9wZXJ0eSBhbmQgQ29weXJpZ2h0IFN0YXRl bWVudHMgLiAuIC4gLiAuIC4gLiAuIC4gLiA0NQ0KDQoNCg0KRWFyZGxleSAoRWRpdG9yKSAgICAg ICAgICBFeHBpcmVzIE1heSAyMiwgMjAwOCAgICAgICAgICAgICAgICAgIFtQYWdlIDNdDQoMDQpJ bnRlcm5ldC1EcmFmdCAgICAgICAgICAgICAgICAgIERvY3VtZW50ICAgICAgICAgICAgICAgICAg IE5vdmVtYmVyIDIwMDcNCg0KDQoxLiAgSW50cm9kdWN0aW9uDQoNCiAgIFRoZSBwdXJwb3NlIG9m IHRoaXMgZG9jdW1lbnQgaXMgdG8gZGVzY3JpYmUgYSBnZW5lcmFsIGFyY2hpdGVjdHVyZQ0KICAg Zm9yIGZsb3cgYWRtaXNzaW9uIGFuZCB0ZXJtaW5hdGlvbiBiYXNlZCBvbiBhZ2dyZWdhdGVkIChw cmUtKQ0KICAgY29uZ2VzdGlvbiBpbmZvcm1hdGlvbiBpbiBvcmRlciB0byBwcm90ZWN0IHRoZSBx dWFsaXR5IG9mIHNlcnZpY2Ugb2YNCiAgIGZsb3dzIHdpdGhpbiBhIERpZmZTZXJ2IGRvbWFpbiBb UkZDMjQ3NV0uICBUaGlzIGRvY3VtZW50IGRlZmluZXMgYW4NCiAgIGFyY2hpdGVjdHVyZSBmb3Ig aW1wbGVtZW50aW5nIHR3byBtZWNoYW5pc21zIHRvIHByb3RlY3QgdGhlIHF1YWxpdHkNCiAgIG9m IHNlcnZpY2Ugb2YgZXN0YWJsaXNoZWQgaW5lbGFzdGljIGZsb3dzIHdpdGhpbiBhIHNpbmdsZSBE aWZmU2Vydg0KICAgZG9tYWluLCB3aGVyZSBhbGwgYm91bmRhcnkgYW5kIGludGVyaW9yIG5vZGVz IGFyZSBQQ04tZW5hYmxlZCBhbmQNCiAgIHRydXN0IGVhY2ggb3RoZXIgZm9yIGNvcnJlY3QgUENO IG9wZXJhdGlvbi4gIEZsb3cgYWRtaXNzaW9uIGNvbnRyb2wNCiAgIGRldGVybWluZXMgd2hldGhl ciBhIG5ldyBmbG93IHNob3VsZCBiZSBhZG1pdHRlZCBhbmQgcHJvdGVjdHMgdGhlIFFvUw0KICAg b2YgZXhpc3RpbmcgUENOLWZsb3dzIGluIG5vcm1hbCBjaXJjdW1zdGFuY2VzLCBieSBhdm9pZGlu ZyBjb25nZXN0aW9uDQogICBvY2N1cnJpbmcuICBIb3dldmVyLCBpbiBhYm5vcm1hbCBjaXJjdW1z dGFuY2VzLCBmb3IgaW5zdGFuY2UgYQ0KICAgZGlzYXN0ZXIgYWZmZWN0aW5nIG11bHRpcGxlIG5v ZGVzIGFuZCBjYXVzaW5nIHRyYWZmaWMgcmUtcm91dGVzLCB0aGVuDQogICB0aGUgUW9TIG9uIGV4 aXN0aW5nIFBDTi1mbG93cyBtYXkgZGVncmFkZSBldmVuIHRob3VnaCBjYXJlIHdhcw0KICAgZXhl cmNpc2VkIHdoZW4gYWRtaXR0aW5nIHRob3NlIGZsb3dzIGJlZm9yZSB0aG9zZSBjaXJjdW1zdGFu Y2VzLg0KICAgVGhlcmVmb3JlIHdlIGFsc28gcHJvcG9zZSBhIG1lY2hhbmlzbSBmb3IgZmxvdyB0 ZXJtaW5hdGlvbiwgd2hpY2gNCiAgIHJlbW92ZXMgZW5vdWdoIHRyYWZmaWMgaW4gb3JkZXIgdG8g cHJvdGVjdCB0aGUgUW9TIG9mIHRoZSByZW1haW5pbmcNCiAgIFBDTi1mbG93cy4NCg0KICAgQXMg YSBmdW5kYW1lbnRhbCBidWlsZGluZyBibG9jayB0byBlbmFibGUgdGhlc2UgdHdvIG1lY2hhbmlz bXMsIFBDTi0NCiAgIGludGVyaW9yLW5vZGVzIGdlbmVyYXRlLCBlbmNvZGUgYW5kIHRyYW5zcG9y dCBwcmUtY29uZ2VzdGlvbg0KICAgaW5mb3JtYXRpb24gdG93YXJkcyB0aGUgUENOLWVncmVzcy1u b2Rlcy4gIFR3byByYXRlcywgYSBQQ04tbG93ZXItDQogICByYXRlIGFuZCBhIFBDTi11cHBlci1y YXRlLCBjYW4gYmUgYXNzb2NpYXRlZCB3aXRoIGVhY2ggbGluayBvZiB0aGUNCiAgIFBDTi1kb21h aW4uICBFYWNoIHJhdGUgaXMgdXNlZCBieSBhIG1hcmtpbmcgYmVoYXZpb3VyIChzcGVjaWZpZWQg aW4NCiAgIGFub3RoZXIgZG9jdW1lbnQpIHRoYXQgZGV0ZXJtaW5lcyBob3cgYW5kIHdoZW4gYSBu dW1iZXIgb2YgUENOLQ0KICAgcGFja2V0cyBhcmUgbWFya2VkLCBhbmQgaG93IHRoZSBtYXJraW5n cyBhcmUgZW5jb2RlZCBpbiBwYWNrZXQNCiAgIGhlYWRlcnMuICBQQ04tZWdyZXNzLW5vZGVzIG1h a2UgbWVhc3VyZW1lbnRzIG9mIHRoZSBwYWNrZXQgbWFya2luZ3MNCiAgIGFuZCBzZW5kIGluZm9y bWF0aW9uIGFzIG5lY2Vzc2FyeSB0byB0aGUgbm9kZXMgdGhhdCBtYWtlIHRoZSBkZWNpc2lvbg0K ICAgYWJvdXQgd2hpY2ggUENOLWZsb3dzIHRvIGFjY2VwdC9yZWplY3Qgb3IgdGVybWluYXRlLCBi YXNlZCBvbiB0aGlzDQogICBpbmZvcm1hdGlvbi4gIEFub3RoZXIgZG9jdW1lbnQgd2lsbCBkZXNj cmliZSB0aGUgZGVjaXNpb24tbWFraW5nDQogICBiZWhhdmlvdXJzLiAgT3ZlcmFsbCB0aGUgYWlt IGlzIHRvIGVuYWJsZSBQQ04tbm9kZXMgdG8gZ2l2ZSBhbiAiZWFybHkNCiAgIHdhcm5pbmciIG9m IHBvdGVudGlhbCBjb25nZXN0aW9uIGJlZm9yZSB0aGVyZSBpcyBhbnkgc2lnbmlmaWNhbnQNCiAg IGJ1aWxkLXVwIG9mIFBDTi1wYWNrZXRzIGluIHRoZSBxdWV1ZTsgdGhlIGFkbWlzc2lvbiBjb250 cm9sIG1lY2hhbmlzbQ0KICAgbGltaXRzIHRoZSBQQ04tdHJhZmZpYyBvbiBlYWNoIGxpbmsgdG8g KnJvdWdobHkqIGl0cyBQQ04tbG93ZXItcmF0ZQ0KICAgYW5kIHRoZSBmbG93IHRlcm1pbmF0aW9u IG1lY2hhbmlzbSBsaW1pdHMgdGhlIFBDTi10cmFmZmljIG9uIGVhY2gNCiAgIGxpbmsgdG8gKnJv dWdobHkqIGl0cyBQQ04tdXBwZXItcmF0ZS4NCg0KICAgV2UgYmVsaWV2ZSB0aGF0IHRoZSBrZXkg YmVuZWZpdHMgb2YgdGhlIFBDTiBtZWNoYW5pc21zIGRlc2NyaWJlZCBpbg0KICAgdGhpcyBkb2N1 bWVudCBhcmUgdGhhdCB0aGV5IGFyZSBzaW1wbGUsIHNjYWxhYmxlLCBhbmQgcm9idXN0IGJlY2F1 c2U6DQoNCiAgIG8gIFBlciBmbG93IHN0YXRlIGlzIG9ubHkgcmVxdWlyZWQgYXQgdGhlIFBDTi1p bmdyZXNzLW5vZGVzDQogICAgICAoInN0YXRlbGVzcyBjb3JlIikuICBUaGlzIGlzIHJlcXVpcmVk IGZvciBwb2xpY2luZyBwdXJwb3NlcyAodG8NCiAgICAgIHByZXZlbnQgbm9uLWFkbWl0dGVkIFBD TiB0cmFmZmljIGZyb20gZW50ZXJpbmcgdGhlIFBDTi1kb21haW4pIGFuZA0KICAgICAgc28gb24u ICBJdCBpcyBub3QgZ2VuZXJhbGx5IHJlcXVpcmVkIHRoYXQgb3RoZXIgbmV0d29yayBlbnRpdGll cw0KICAgICAgYXJlIGF3YXJlIG9mIGluZGl2aWR1YWwgZmxvd3MgKGFsdGhvdWdoIHRoZXkgbWF5 IGJlIGluIHBhcnRpY3VsYXINCiAgICAgIGRlcGxveW1lbnQgc2NlbmFyaW9zKS4NCg0KDQoNCg0K RWFyZGxleSAoRWRpdG9yKSAgICAgICAgICBFeHBpcmVzIE1heSAyMiwgMjAwOCAgICAgICAgICAg ICAgICAgIFtQYWdlIDRdDQoMDQpJbnRlcm5ldC1EcmFmdCAgICAgICAgICAgICAgICAgIERvY3Vt ZW50ICAgICAgICAgICAgICAgICAgIE5vdmVtYmVyIDIwMDcNCg0KDQogICBvICBBZG1pc3Npb24g Y29udHJvbCBpcyByZXNpbGllbnQ6IFBDTidzIFFvUyBpcyBkZWNvdXBsZWQgZnJvbSB0aGUNCiAg ICAgIHJvdXRpbmcgc3lzdGVtOyBoZW5jZSBpbiBnZW5lcmFsIGFkbWl0dGVkIGZsb3dzIGNhbiBz dXJ2aXZlDQogICAgICBjYXBhY2l0eSwgcm91dGluZyBvciB0b3BvbG9neSBjaGFuZ2VzIHdpdGhv dXQgYWRkaXRpb25hbA0KICAgICAgc2lnbmFsbGluZywgYW5kIHRoZXkgZG9uJ3QgaGF2ZSB0byBi ZSB0b2xkIChvciBsZWFybikgYWJvdXQgc3VjaA0KICAgICAgY2hhbmdlcy4gIFRoZSBQQ04tbG93 ZXItcmF0ZXMgY2FuIGJlIGNob3NlbiBzbWFsbCBlbm91Z2ggdGhhdA0KICAgICAgYWRtaXR0ZWQg dHJhZmZpYyBjYW4gc3RpbGwgYmUgY2FycmllZCBhZnRlciBhIHJlcm91dGluZyBpbiBtb3N0DQog ICAgICBmYWlsdXJlIGNhc2VzLiAgVGhpcyBpcyBhbiBpbXBvcnRhbnQgZmVhdHVyZSBhcyBRb1Mg dmlvbGF0aW9ucyBpbg0KICAgICAgY29yZSBuZXR3b3JrcyBkdWUgdG8gbGluayBmYWlsdXJlcyBh cmUgbW9yZSBsaWtlbHkgdGhhbiBRb1MNCiAgICAgIHZpb2xhdGlvbnMgZHVlIHRvIGluY3JlYXNl ZCB0cmFmZmljIHZvbHVtZSBbSXllcl0uDQoNCiAgIG8gIFRoZSBQQ04tbWFya2luZyBiZWhhdmlv dXJzIG9ubHkgb3BlcmF0ZSBvbiB0aGUgb3ZlcmFsbCBQQ04tdHJhZmZpYw0KICAgICAgb24gdGhl IGxpbmssIG5vdCBwZXIgZmxvdy4NCg0KICAgbyAgVGhlIGluZm9ybWF0aW9uIG9mIHRoZXNlIG1l YXN1cmVtZW50cyBpcyBzaWduYWxsZWQgdG8gdGhlIFBDTi0NCiAgICAgIGVncmVzcy1ub2RlcyBi eSB0aGUgUENOLW1hcmtzIGluIHRoZSBwYWNrZXQgaGVhZGVycy4gIE5vDQogICAgICBhZGRpdGlv bmFsIHNpZ25hbGxpbmcgcHJvdG9jb2wgaXMgcmVxdWlyZWQgZm9yIHRyYW5zcG9ydGluZyB0aGUN CiAgICAgIFBDTi1tYXJrcy4gIFRoZXJlZm9yZSBubyBzZWN1cmUgYmluZGluZyBpcyByZXF1aXJl ZCBiZXR3ZWVuIGRhdGENCiAgICAgIHBhY2tldHMgYW5kIHNlcGFyYXRlIGNvbmdlc3Rpb24gbWVz c2FnZXMuDQoNCiAgIG8gIFRoZSBQQ04tZWdyZXNzLW5vZGVzIG1ha2Ugc2VwYXJhdGUgbWVhc3Vy ZW1lbnRzLCBvcGVyYXRpbmcgb24gdGhlDQogICAgICBvdmVyYWxsIFBDTi10cmFmZmljLCBmb3Ig ZWFjaCBQQ04taW5ncmVzcy1ub2RlLCBpZSBub3QgcGVyIGZsb3cuDQogICAgICBTaW1pbGFybHks IHNpZ25hbGxpbmcgYnkgdGhlIFBDTi1lZ3Jlc3Mtbm9kZSBvZiBQQ04tZmVlZGJhY2stDQogICAg ICBpbmZvcm1hdGlvbiAod2hpY2ggaXMgdXNlZCBmb3IgZmxvdyBhZG1pc3Npb24gYW5kIHRlcm1p bmF0aW9uDQogICAgICBkZWNpc2lvbnMpIGlzIGF0IHRoZSBncmFudWxhcml0eSBvZiB0aGUgaW5n cmVzcy1lZ3Jlc3MtYWdncmVnYXRlLg0KDQogICBvICBUaGUgYWRtaXR0ZWQgUENOLWxvYWQgaXMg Y29udHJvbGxlZCBkeW5hbWljYWxseS4gIFRoZXJlZm9yZSBpdA0KICAgICAgYWRhcHRzIGFzIHRo ZSB0cmFmZmljIG1hdHJpeCBjaGFuZ2VzLCBhbmQgYWxzbyBpZiB0aGUgbmV0d29yaw0KICAgICAg dG9wb2xvZ3kgY2hhbmdlcyAoZWcgYWZ0ZXIgYSBsaW5rIGZhaWx1cmUpLiAgSGVuY2UgYW4gb3Bl cmF0b3IgY2FuDQogICAgICBiZSBsZXNzIGNvbnNlcnZhdGl2ZSB3aGVuIGRlcGxveWluZyBuZXR3 b3JrIGNhcGFjaXR5LCBhbmQgbGVzcw0KICAgICAgYWNjdXJhdGUgaW4gdGhlaXIgcHJlZGljdGlv biBvZiB0aGUgUENOLXRyYWZmaWMgbWF0cml4Lg0KDQogICBvICBUaGUgdGVybWluYXRpb24gbWVj aGFuaXNtIGNvbXBsZW1lbnRzIGFkbWlzc2lvbiBjb250cm9sLiAgSXQNCiAgICAgIGFsbG93cyB0 aGUgbmV0d29yayB0byByZWNvdmVyIGZyb20gc3VkZGVuIHVuZXhwZWN0ZWQgc3VyZ2VzIG9mDQog ICAgICBQQ04tdHJhZmZpYyBvbiBzb21lIGxpbmtzLCB0aHVzIHJlc3RvcmluZyBRb1MgdG8gdGhl IHJlbWFpbmluZw0KICAgICAgZmxvd3MuICBTdWNoIHNjZW5hcmlvcyBhcmUgZXhwZWN0ZWQgdG8g YmUgcmFyZSBidXQgbm90IGltcG9zc2libGUuDQogICAgICBUaGV5IGNhbiBiZSBjYXVzZWQgYnkg bGFyZ2UgbmV0d29yayBmYWlsdXJlcyB0aGF0IHJlZGlyZWN0IGxvdHMgb2YNCiAgICAgIGFkbWl0 dGVkIFBDTi10cmFmZmljIHRvIG90aGVyIGxpbmtzLCBvciBieSBtYWxmdW5jdGlvbiBvZiB0aGUN CiAgICAgIG1lYXN1cmVtZW50LWJhc2VkIGFkbWlzc2lvbiBjb250cm9sIGluIHRoZSBwcmVzZW5j ZSBvZiBhZG1pdHRlZA0KICAgICAgZmxvd3MgdGhhdCBzZW5kIGZvciBhIHdoaWxlIHdpdGggYW4g YXR5cGljYWxseSBsb3cgcmF0ZSBhbmQgdGhlbg0KICAgICAgaW5jcmVhc2UgdGhlaXIgcmF0ZXMg aW4gYSBjb3JyZWxhdGVkIHdheS4NCg0KICAgbyAgVGhlIFBDTi11cHBlci1yYXRlIG1heSBiZSBz ZXQgYmVsb3cgdGhlIG1heGltdW0gcmF0ZSB0aGF0IFBDTi0NCiAgICAgIHRyYWZmaWMgY2FuIGJl IHRyYW5zbWl0dGVkIG9uIGEgbGluaywgaW4gb3JkZXIgdG8gdHJpZ2dlcg0KICAgICAgdGVybWlu YXRpb24gb2Ygc29tZSBQQ04tZmxvd3MgYmVmb3JlIGxvc3MgKG9yIGV4Y2Vzc2l2ZSBkZWxheSkg b2YNCiAgICAgIFBDTi1wYWNrZXRzIG9jY3Vycywgb3IgdG8ga2VlcCB0aGUgbWF4aW11bSBQQ04t bG9hZCBvbiBhIGxpbmsNCiAgICAgIGJlbG93IGEgbGV2ZWwgY29uZmlndXJlZCBieSB0aGUgb3Bl cmF0b3IuDQoNCg0KDQoNCg0KRWFyZGxleSAoRWRpdG9yKSAgICAgICAgICBFeHBpcmVzIE1heSAy MiwgMjAwOCAgICAgICAgICAgICAgICAgIFtQYWdlIDVdDQoMDQpJbnRlcm5ldC1EcmFmdCAgICAg ICAgICAgICAgICAgIERvY3VtZW50ICAgICAgICAgICAgICAgICAgIE5vdmVtYmVyIDIwMDcNCg0K DQogICBvICBQcm92aXNpb25pbmcgb2YgdGhlIG5ldHdvcmsgaXMgZGVjb3VwbGVkIGZyb20gdGhl IHByb2Nlc3Mgb2YNCiAgICAgIGFkZGluZyBuZXcgY3VzdG9tZXJzLiAgQnkgY29udHJhc3QsIHdp dGggdGhlIERpZmZTZXJ2IGFyY2hpdGVjdHVyZQ0KICAgICAgW1JGQzI0NzVdIG9wZXJhdG9ycyBy ZWx5IG9uIHN1YnNjcmlwdGlvbi10aW1lIFNlcnZpY2UgTGV2ZWwNCiAgICAgIEFncmVlbWVudHMg dGhhdCBzdGF0aWNhbGx5IGRlZmluZSB0aGUgcGFyYW1ldGVycyBvZiB0aGUgdHJhZmZpYw0KICAg ICAgdGhhdCB3aWxsIGJlIGFjY2VwdGVkIGZyb20gYSBjdXN0b21lciwgYW5kIHNvIHRoZSBvcGVy YXRvciBoYXMgdG8NCiAgICAgIHJ1biB0aGUgcHJvdmlzaW9uaW5nIHByb2Nlc3MgZWFjaCB0aW1l IGEgbmV3IGN1c3RvbWVyIGlzIGFkZGVkIHRvDQogICAgICBjaGVjayB0aGF0IHRoZSBTZXJ2aWNl IExldmVsIEFncmVlbWVudCBjYW4gYmUgZnVsZmlsbGVkLiAgUENOIGRvZXMNCiAgICAgIG5vdCB1 c2UgUkZDMjQ3NS1zdHlsZSB0cmFmZmljIGNvbmRpdGlvbmluZy4NCg0KICAgT3BlcmF0b3JzIG9m IG5ldHdvcmtzIHdpbGwgd2FudCB0byB1c2UgdGhlIFBDTiBtZWNoYW5pc21zIGluIHZhcmlvdXMN CiAgIGFycmFuZ2VtZW50cywgZm9yIGluc3RhbmNlIGRlcGVuZGluZyBvbiBob3cgdGhleSBhcmUg cGVyZm9ybWluZw0KICAgYWRtaXNzaW9uIGNvbnRyb2wgb3V0c2lkZSB0aGUgUENOLWRvbWFpbiAo dXNlcnMgYWZ0ZXIgYWxsIGFyZQ0KICAgY29uY2VybmVkIGFib3V0IFFvUyBlbmQtdG8tZW5kKSwg d2hhdCB0aGVpciBwYXJ0aWN1bGFyIGdvYWxzIGFuZA0KICAgYXNzdW1wdGlvbnMgYXJlLCBhbmQg c28gb24uICBTZXZlcmFsIGRlcGxveW1lbnQgbW9kZWxzIGFyZSBwb3NzaWJsZToNCg0KICAgbyAg QW4gb3BlcmF0b3IgbWF5IGNob29zZSB0byBkZXBsb3kgZWl0aGVyIGFkbWlzc2lvbiBjb250cm9s IG9yIGZsb3cNCiAgICAgIHRlcm1pbmF0aW9uIG9yIGJvdGggKHNlZSBTZWN0aW9uIDQuMykuDQoN CiAgIG8gIEludFNlcnYgb3ZlciBEaWZmU2VydiBbUkZDMjk5OF0uICBUaGUgRGlmZlNlcnYgcmVn aW9uIGlzIFBDTi0NCiAgICAgIGVuYWJsZWQsIFJTVlAgc2lnbmFsbGluZyBpcyB1c2VkIGVuZC10 by1lbmQgYW5kIHRoZSBQQ04tZG9tYWluIGlzDQogICAgICBhIHNpbmdsZSBSU1ZQIGhvcCwgaWUg b25seSB0aGUgUENOLWJvdW5kYXJ5LW5vZGVzIHByb2Nlc3MgUlNWUA0KICAgICAgbWVzc2FnZXMu ICBPdXRzaWRlIHRoZSBQQ04tZG9tYWluIFJTVlAgbWVzc2FnZXMgYXJlIHByb2Nlc3NlZCBvbg0K ICAgICAgZWFjaCBob3AuICBUaGlzIGlzIGRlc2NyaWJlZCBpbg0KICAgICAgW0ktRC5icmlzY29l LXRzdndnLWNsLWFyY2hpdGVjdHVyZV0NCg0KICAgbyAgUlNWUCBzaWduYWxsaW5nIGlzIG9yaWdp bmF0ZWQgYW5kL29yIHRlcm1pbmF0ZWQgYnkgcHJveGllcywgd2l0aA0KICAgICAgYXBwbGljYXRp b24tbGF5ZXIgc2lnbmFsbGluZyBiZXR3ZWVuIHRoZSBlbmQgdXNlciBhbmQgdGhlIHByb3h5Lg0K ICAgICAgRm9yIGluc3RhbmNlIFNJUCBzaWduYWxsaW5nIHdpdGggYSBob21lIGh1Yi4NCg0KICAg byAgU2ltaWxhciB0byBwcmV2aW91cyBidWxsZXRzIGJ1dCBOU0lTIHNpZ25hbGxpbmcgaXMgdXNl ZCBpbnN0ZWFkIG9mDQogICAgICBSU1ZQLg0KDQogICBvICBOT1RFOiBDb25zaWRlcmF0aW9uIG9m IHNpZ25hbGxpbmcgZXh0ZW5zaW9ucyBmb3Igc3BlY2lmaWMNCiAgICAgIHByb3RvY29scyBpcyBv dXRzaWRlIHRoZSBzY29wZSBvZiB0aGUgUENOIFdHLCBob3dldmVyIGl0IHdpbGwNCiAgICAgIHBy b2R1Y2UgYSAiUmVxdWlyZW1lbnRzIGZvciBzaWduYWxsaW5nIiBkb2N1bWVudCBhcyBwb3RlbnRp YWwNCiAgICAgIGlucHV0IGZvciB0aGUgYXBwcm9wcmlhdGUgV0dzLg0KDQogICBvICBEZXBlbmRp bmcgb24gdGhlIGRlcGxveW1lbnQgc2NlbmFyaW8sIHRoZSBkZWNpc2lvbi1tYWtpbmcNCiAgICAg IGZ1bmN0aW9uYWxpdHkgKGFib3V0IGZsb3cgYWRtaXNzaW9uIGFuZCB0ZXJtaW5hdGlvbikgY291 bGQgcmVzaWRlDQogICAgICBhdCB0aGUgUENOLWluZ3Jlc3Mtbm9kZXMgb3IgUENOLWVncmVzcy1u b2RlcyBvciBhdCBzb21lIGNlbnRyYWwNCiAgICAgIGNvbnRyb2wgbm9kZSBpbiB0aGUgUENOLWRv bWFpbi4gIE5PVEU6IFRoZSBDaGFydGVyIHJlc3RyaWN0cyB1czoNCiAgICAgIHRoZSBkZWNpc2lv bi1tYWtpbmcgZnVuY3Rpb25hbGl0eSBpcyBhdCB0aGUgUENOLWJvdW5kYXJ5LW5vZGVzLg0KDQog ICBvICBJZiB0aGUgb3BlcmF0b3IgcnVucyBib3RoIHRoZSBhY2Nlc3MgbmV0d29yayBhbmQgdGhl IGNvcmUgbmV0d29yaywNCiAgICAgIG9uZSBkZXBsb3ltZW50IHNjZW5hcmlvIGlzIHRoYXQgb25s eSB0aGUgY29yZSBuZXR3b3JrIHVzZXMgUENODQogICAgICBhZG1pc3Npb24gY29udHJvbCBidXQg cGVyIG1pY3JvZmxvdyBwb2xpY2luZyBpcyBkb25lIGF0IHRoZQ0KICAgICAgaW5ncmVzcyB0byB0 aGUgYWNjZXNzIG5ldHdvcmsgYW5kIG5vdCBhdCB0aGUgUENOLWluZ3Jlc3Mtbm9kZS4NCiAgICAg IE5vdGU6IHRvIGFpZCByZWFkYWJpbGl0eSwgdGhlIHJlc3Qgb2YgdGhpcyBkcmFmdCBhc3N1bWVz IHRoYXQNCg0KDQoNCkVhcmRsZXkgKEVkaXRvcikgICAgICAgICAgRXhwaXJlcyBNYXkgMjIsIDIw MDggICAgICAgICAgICAgICAgICBbUGFnZSA2XQ0KDA0KSW50ZXJuZXQtRHJhZnQgICAgICAgICAg ICAgICAgICBEb2N1bWVudCAgICAgICAgICAgICAgICAgICBOb3ZlbWJlciAyMDA3DQoNCg0KICAg ICAgcG9saWNpbmcgaXMgZG9uZSBieSB0aGUgUENOLWluZ3Jlc3Mtbm9kZXMuDQoNCiAgIG8gIFRo ZXJlIGFyZSBzZXZlcmFsIFBDTi1kb21haW5zIG9uIHRoZSBlbmQtdG8tZW5kIHBhdGgsIGVhY2gN CiAgICAgIG9wZXJhdGluZyBQQ04gbWVjaGFuaXNtcyBpbmRlcGVuZGVudGx5LiAgTk9URTogVGhl IENoYXJ0ZXINCiAgICAgIHJlc3RyaWN0cyB1cyB0byBjb25zaWRlcmluZyBhIHNpbmdsZSBQQ04t ZG9tYWluLiAgQSBwb3NzaWJpbGl0eQ0KICAgICAgYWZ0ZXIgcmUtY2hhcnRlcmluZyBpcyB0byBj b25zaWRlciB0aGF0IHRoZSBQQ04tZG9tYWluIGVuY29tcGFzc2VzDQogICAgICBzZXZlcmFsIGF1 dG9ub21vdXMgc3lzdGVtcyB0aGF0IGRvbid0IHRydXN0IGVhY2ggb3RoZXIgKGllIHdlYWtlbnMN CiAgICAgIEFzc3VtcHRpb24gMSBhYm91dCB0cnVzdCwgc2VlIFNlY3Rpb24gMy4xKQ0KDQogICBv ICBUaGUgUENOLWRvbWFpbiBleHRlbmRzIHRvIHRoZSBlbmQgdXNlcnMuICBOT1RFOiBUaGlzIGlz bid0DQogICAgICBuZWNlc3NhcmlseSBvdXRzaWRlIHRoZSBDaGFydGVyIGJlY2F1c2UgaXQgbWF5 IG5vdCBicmVhaw0KICAgICAgQXNzdW1wdGlvbiAzIChhZ2dyZWdhdGlvbiBzZWUgbGF0ZXIpIGlm IGl0J3Mga25vd24gdGhlcmUncw0KICAgICAgc3VmZmljaWVudCBhZ2dyZWdhdGlvbiBhdCBhbnkg Ym90dGxlbmVjaywgYW5kIGl0IGRvZXNuJ3QNCiAgICAgIG5lY2Vzc2FyaWx5IGJyZWFrIEFzc3Vt cHRpb24gMSAodHJ1c3QpLCBiZWNhdXNlIGluIHNvbWUNCiAgICAgIGVudmlyb25tZW50cywgZWcg Y29ycG9yYXRlLCB0aGUgZW5kIHVzZXIgbWF5IGhhdmUgYSBjb250cm9sbGVkDQogICAgICBjb25m aWd1cmF0aW9uIGFuZCBzbyBiZSB0cnVzdGVkLiAgVGhlIHNjZW5hcmlvIGlzIGRlc2NyaWJlZCBp bg0KICAgICAgW0ktRC5iYWJpYXJ6LXBjbi1zaXAtY2FwXS4gIEEgdmFyaWFudCBpcyB0aGF0IHRo ZSBQQ04tZG9tYWluDQogICAgICBleHRlbmRzIG91dCBhcyBmYXIgYXMgdGhlIExBTiBlZGdlIHN3 aXRjaC4NCg0KICAgbyAgUHNldWRvd2lyZTogUENOIG1heSBiZSB1c2VkIGFzIGEgY29uZ2VzdGlv biBhdm9pZGFuY2UgbWVjaGFuaXNtDQogICAgICBmb3IgZWRnZSB0byBlZGdlIHBzZXVkb3dpcmUg ZW11bGF0aW9ucw0KICAgICAgW0ktRC5pZXRmLXB3ZTMtY29uZ2VzdGlvbi1mcm13a10uICBOT1RF OiBTcGVjaWZpYyBjb25zaWRlcmF0aW9uIG9mDQogICAgICBwc2V1ZG93aXJlcyBpcyBub3QgaW4g dGhlIFBDTiBXRyBDaGFydGVyLg0KDQogICBvICBNUExTOiBbUkZDMzI3MF0gZGVmaW5lcyBob3cg dG8gc3VwcG9ydCB0aGUgRGlmZlNlcnYgYXJjaGl0ZWN0dXJlDQogICAgICBpbiBNUExTIG5ldHdv cmtzLiAgW0ktRC5pZXRmLXRzdndnLWVjbi1tcGxzXSBkZXNjcmliZXMgaG93IHRvIGFkZA0KICAg ICAgUENOIGZvciBhZG1pc3Npb24gY29udHJvbCBvZiBtaWNyb2Zsb3dzIGludG8gYSBzZXQgb2Yg TVBMUw0KICAgICAgYWdncmVnYXRlcyAoTXVsdGktcHJvdG9jb2wgbGFiZWwgc3dpdGNoaW5nKS4g IFBDTi1tYXJraW5nIGlzIGRvbmUNCiAgICAgIGluIE1QTFMncyBFWFAgZmllbGQuDQoNCiAgIG8g IFNpbWlsYXJseSwgaXQgbWF5IGJlIHBvc3NpYmxlIHRvIGV4dGVuZCBQQ04gaW50byBFdGhlcm5l dA0KICAgICAgbmV0d29ya3MsIHdoZXJlIFBDTi1tYXJraW5nIGlzIGRvbmUgaW4gdGhlIEV0aGVy bmV0IGhlYWRlci4gIE5PVEU6DQogICAgICBTcGVjaWZpYyBjb25zaWRlcmF0aW9uIG9mIHRoaXMg ZXh0ZW5zaW9uIGlzIG91dHNpZGUgdGhlIElFVEYncw0KICAgICAgcmVtaXQuDQoNCg0KMi4gIFRl cm1pbm9sb2d5DQoNCiAgIG8gIFBDTi1kb21haW46IGEgUENOLWNhcGFibGUgZG9tYWluOyBhIGNv bnRpZ3VvdXMgc2V0IG9mIFBDTi1lbmFibGVkDQogICAgICBub2RlcyB0aGF0IHBlcmZvcm0gRGlm ZlNlcnYgc2NoZWR1bGluZzsgdGhlIGNvbXBldGUgc2V0IG9mIFBDTi0NCiAgICAgIG5vZGVzIHdo b3NlIFBDTi1tYXJraW5nIGNhbiBpbiBwcmluY2lwbGUgaW5mbHVlbmNlIGRlY2lzaW9ucyBhYm91 dA0KICAgICAgZmxvdyBhZG1pc3Npb24gYW5kIHRlcm1pbmF0aW9uIGZvciB0aGUgUENOLWRvbWFp biwgaW5jbHVkaW5nIHRoZQ0KICAgICAgUENOLWVncmVzcy1ub2RlcyB3aGljaCBtZWFzdXJlIHRo ZXNlIFBDTi1tYXJrcy4NCg0KICAgbyAgUENOLWJvdW5kYXJ5LW5vZGU6IGEgUENOLW5vZGUgdGhh dCBjb25uZWN0cyBvbmUgUENOLWRvbWFpbiB0byBhDQogICAgICBub2RlIGVpdGhlciBpbiBhbm90 aGVyIFBDTi1kb21haW4gb3IgaW4gYSBub24gUENOLWRvbWFpbi4NCg0KDQoNCg0KDQpFYXJkbGV5 IChFZGl0b3IpICAgICAgICAgIEV4cGlyZXMgTWF5IDIyLCAyMDA4ICAgICAgICAgICAgICAgICAg W1BhZ2UgN10NCgwNCkludGVybmV0LURyYWZ0ICAgICAgICAgICAgICAgICAgRG9jdW1lbnQgICAg ICAgICAgICAgICAgICAgTm92ZW1iZXIgMjAwNw0KDQoNCiAgIG8gIFBDTi1pbnRlcmlvci1ub2Rl OiBhIG5vZGUgaW4gYSBQQ04tZG9tYWluIHRoYXQgaXMgbm90IGEgUENOLQ0KICAgICAgYm91bmRh cnktbm9kZS4NCg0KICAgbyAgUENOLW5vZGU6IGEgUENOLWJvdW5kYXJ5LW5vZGUgb3IgYSBQQ04t aW50ZXJpb3Itbm9kZQ0KDQogICBvICBQQ04tZWdyZXNzLW5vZGU6IGEgUENOLWJvdW5kYXJ5LW5v ZGUgaW4gaXRzIHJvbGUgaW4gaGFuZGxpbmcNCiAgICAgIHRyYWZmaWMgYXMgaXQgbGVhdmVzIGEg UENOLWRvbWFpbi4NCg0KICAgbyAgUENOLWluZ3Jlc3Mtbm9kZTogYSBQQ04tYm91bmRhcnktbm9k ZSBpbiBpdHMgcm9sZSBpbiBoYW5kbGluZw0KICAgICAgdHJhZmZpYyBhcyBpdCBlbnRlcnMgYSBQ Q04tZG9tYWluLg0KDQogICBvICBQQ04tdHJhZmZpYzogQSBQQ04tZG9tYWluIGNhcnJpZXMgdHJh ZmZpYyBvZiBkaWZmZXJlbnQgRGlmZlNlcnYNCiAgICAgIGNsYXNzZXMgW1JGQzQ1OTRdLiAgVGhv c2UgdXNpbmcgdGhlIFBDTiBtZWNoYW5pc21zIGFyZSBjYWxsZWQgUENOLQ0KICAgICAgY2xhc3Nl cyAoY29sbGVjdGl2ZWx5IGNhbGxlZCBQQ04tdHJhZmZpYykgYW5kIHRoZSBjb3JyZXNwb25kaW5n DQogICAgICBwYWNrZXRzIGFyZSBQQ04tcGFja2V0cy4gIFRoZSBzYW1lIG5ldHdvcmsgbWF5IGNh cnJ5IHRyYWZmaWMgdXNpbmcNCiAgICAgIG90aGVyIERpZmZTZXJ2IGNsYXNzZXMuDQoNCiAgIG8g IEluZ3Jlc3MtZWdyZXNzLWFnZ3JlZ2F0ZTogVGhlIGNvbGxlY3Rpb24gb2YgUENOLXBhY2tldHMg ZnJvbSBhbGwNCiAgICAgIFBDTi1mbG93cyB0aGF0IHRyYXZlbCBpbiBvbmUgZGlyZWN0aW9uIGJl dHdlZW4gYSBzcGVjaWZpYyBwYWlyIG9mDQogICAgICBQQ04tYm91bmRhcnktbm9kZXMuDQoNCiAg IG8gIFBDTi1sb3dlci1yYXRlOiBhIHJlZmVyZW5jZSByYXRlIGNvbmZpZ3VyZWQgZm9yIGVhY2gg bGluayBpbiB0aGUNCiAgICAgIFBDTi1kb21haW4sIHdoaWNoIGlzIGxvd2VyIHRoYW4gdGhlIFBD Ti11cHBlci1yYXRlLiAgSXQgaXMgdXNlZCBieQ0KICAgICAgYSBtYXJraW5nIGJlaGF2aW91ciB0 aGF0IGRldGVybWluZXMgd2hldGhlciBhIHBhY2tldCBzaG91bGQgYmUNCiAgICAgIFBDTi1tYXJr ZWQgd2l0aCBhIGZpcnN0IGVuY29kaW5nLg0KDQogICBvICBQQ04tdXBwZXItcmF0ZTogYSByZWZl cmVuY2UgcmF0ZSBjb25maWd1cmVkIGZvciBlYWNoIGxpbmsgaW4gdGhlDQogICAgICBQQ04tZG9t YWluLCB3aGljaCBpcyBoaWdoZXIgdGhhbiB0aGUgUENOLWxvd2VyLXJhdGUuICBJdCBpcyB1c2Vk DQogICAgICBieSBhIG1hcmtpbmcgYmVoYXZpb3VyIHRoYXQgZGV0ZXJtaW5lcyB3aGV0aGVyIGEg cGFja2V0IHNob3VsZCBiZQ0KICAgICAgUENOLW1hcmtlZCB3aXRoIGEgc2Vjb25kIGVuY29kaW5n Lg0KDQogICBvICBUaHJlc2hvbGQtbWFya2luZzogYSBQQ04tbWFya2luZyBiZWhhdmlvdXIgc3Vj aCB0aGF0IGFsbCBQQ04tDQogICAgICB0cmFmZmljIGlzIG1hcmtlZCBpZiB0aGUgUENOLXRyYWZm aWMgZXhjZWVkcyBhIHBhcnRpY3VsYXIgcmF0ZQ0KICAgICAgKGVpdGhlciB0aGUgUENOLWxvd2Vy LXJhdGUgb3IgUENOLXVwcGVyLXJhdGUpLiAgTk9URTogVGhlDQogICAgICBkZWZpbml0aW9uIHJl ZmxlY3RzIHRoZSBvdmVyYWxsIGludGVudCByYXRoZXIgdGhhbiBpdHMNCiAgICAgIGluc3RhbnRh bmVvdXMgYmVoYXZpb3VyLCBzaW5jZSB0aGUgcmF0ZSBtZWFzdXJlZCBhdCBhIHBhcnRpY3VsYXIN CiAgICAgIG1vbWVudCBkZXBlbmRzIG9uIHRoZSBiZWhhdmlvdXIsIGl0cyBpbXBsZW1lbnRhdGlv biBhbmQgdGhlDQogICAgICB0cmFmZmljJ3MgdmFyaWFuY2UgYXMgd2VsbCBhcyBpdHMgcmF0ZS4N Cg0KICAgbyAgRXhjZXNzLXJhdGUtbWFya2luZzogYSBQQ04tbWFya2luZyBiZWhhdmlvdXIgc3Vj aCB0aGF0IHRoZSBhbW91bnQNCiAgICAgIG9mIFBDTi10cmFmZmljIHRoYXQgaXMgUENOLW1hcmtl ZCBpcyBlcXVhbCB0byB0aGUgYW1vdW50IHRoYXQNCiAgICAgIGV4Y2VlZHMgYSBwYXJ0aWN1bGFy IHJhdGUgKGVpdGhlciB0aGUgUENOLWxvd2VyLXJhdGUgb3IgUENOLXVwcGVyLQ0KICAgICAgcmF0 ZSkuICBOT1RFOiBUaGUgZGVmaW5pdGlvbiByZWZsZWN0cyB0aGUgb3ZlcmFsbCBpbnRlbnQgcmF0 aGVyDQogICAgICB0aGFuIGl0cyBpbnN0YW50YW5lb3VzIGJlaGF2aW91ciwgc2luY2UgdGhlIHJh dGUgbWVhc3VyZWQgYXQgYQ0KICAgICAgcGFydGljdWxhciBtb21lbnQgZGVwZW5kcyBvbiB0aGUg YmVoYXZpb3VyLCBpdHMgaW1wbGVtZW50YXRpb24gYW5kDQogICAgICB0aGUgdHJhZmZpYydzIHZh cmlhbmNlIGFzIHdlbGwgYXMgaXRzIHJhdGUuDQoNCg0KDQoNCg0KRWFyZGxleSAoRWRpdG9yKSAg ICAgICAgICBFeHBpcmVzIE1heSAyMiwgMjAwOCAgICAgICAgICAgICAgICAgIFtQYWdlIDhdDQoM DQpJbnRlcm5ldC1EcmFmdCAgICAgICAgICAgICAgICAgIERvY3VtZW50ICAgICAgICAgICAgICAg ICAgIE5vdmVtYmVyIDIwMDcNCg0KDQogICBvICBQcmUtY29uZ2VzdGlvbjogYSBjb25kaXRpb24g b2YgYSBsaW5rIHdpdGhpbiBhIFBDTi1kb21haW4gaW4gd2hpY2gNCiAgICAgIHRoZSBQQ04tbm9k ZSBwZXJmb3JtcyBQQ04tbWFya2luZywgaW4gb3JkZXIgdG8gcHJvdmlkZSBhbiAiZWFybHkNCiAg ICAgIHdhcm5pbmciIG9mIHBvdGVudGlhbCBjb25nZXN0aW9uIGJlZm9yZSB0aGVyZSBpcyBhbnkg c2lnbmlmaWNhbnQNCiAgICAgIGJ1aWxkLXVwIG9mIFBDTi1wYWNrZXRzIGluIHRoZSByZWFsIHF1 ZXVlLg0KDQogICBvICBQQ04tbWFya2luZzogdGhlIHByb2Nlc3Mgb2Ygc2V0dGluZyB0aGUgaGVh ZGVyIGluIGEgUENOLXBhY2tldA0KICAgICAgYmFzZWQgb24gZGVmaW5lZCBydWxlcywgaW4gcmVh Y3Rpb24gdG8gcHJlLWNvbmdlc3Rpb24uDQoNCiAgIG8gIFBDTi1mZWVkYmFjay1pbmZvcm1hdGlv bjogaW5mb3JtYXRpb24gc2lnbmFsbGVkIGJ5IGEgUENOLWVncmVzcy0NCiAgICAgIG5vZGUgdG8g YSBQQ04taW5ncmVzcy1ub2RlIG9yIGNlbnRyYWwgY29udHJvbCBub2RlLCB3aGljaCBpcw0KICAg ICAgbmVlZGVkIGZvciB0aGUgZmxvdyBhZG1pc3Npb24gYW5kIGZsb3cgdGVybWluYXRpb24gbWVj aGFuaXNtcy4NCg0KDQozLiAgQXNzdW1wdGlvbnMgYW5kIGNvbnN0cmFpbnRzIG9uIHNjb3BlDQoN CiAgIFRoZSBQQ04gV0cncyBjaGFydGVyIHJlc3RyaWN0cyB0aGUgaW5pdGlhbCBzY29wZSBieSBh IHNldCBvZg0KICAgYXNzdW1wdGlvbnMuICBIZXJlIHdlIGxpc3QgdGhvc2UgYXNzdW1wdGlvbnMg YW5kIGV4cGxhaW4gdGhlbS4NCg0KICAgMS4gIHRoZXNlIGNvbXBvbmVudHMgYXJlIGRlcGxveWVk IGluIGEgc2luZ2xlIERpZmZTZXJ2IGRvbWFpbiwgd2l0aGluDQogICAgICAgd2hpY2ggYWxsIFBD Ti1ub2RlcyBhcmUgUENOLWVuYWJsZWQgYW5kIHRydXN0IGVhY2ggb3RoZXIgZm9yDQogICAgICAg dHJ1dGhmdWwgUENOLW1hcmtpbmcgYW5kIHRyYW5zcG9ydA0KDQogICAyLiAgYWxsIGZsb3dzIGhh bmRsZWQgYnkgdGhlc2UgbWVjaGFuaXNtcyBhcmUgaW5lbGFzdGljIGFuZA0KICAgICAgIGNvbnN0 cmFpbmVkIHRvIGEga25vd24gcGVhayByYXRlIHRocm91Z2ggcG9saWNpbmcgb3Igc2hhcGluZw0K DQogICAzLiAgdGhlIG51bWJlciBvZiBQQ04tZmxvd3MgYWNyb3NzIGFueSBwb3RlbnRpYWwgYm90 dGxlbmVjayBsaW5rIGlzDQogICAgICAgc3VmZmljaWVudGx5IGxhcmdlIHRoYXQgc3RhdGVsZXNz LCBzdGF0aXN0aWNhbCBtZWNoYW5pc21zIGNhbiBiZQ0KICAgICAgIGVmZmVjdGl2ZS4gIFRvIHB1 dCBpdCBhbm90aGVyIHdheSwgdGhlIGFnZ3JlZ2F0ZSBiaXQgcmF0ZSBvZiBQQ04tDQogICAgICAg dHJhZmZpYyBhY3Jvc3MgYW55IHBvdGVudGlhbCBib3R0bGVuZWNrIGxpbmsgbmVlZHMgdG8gYmUN CiAgICAgICBzdWZmaWNpZW50bHkgbGFyZ2UgcmVsYXRpdmUgdG8gdGhlIG1heGltdW0gYWRkaXRp b25hbCBiaXQgcmF0ZQ0KICAgICAgIGFkZGVkIGJ5IG9uZSBmbG93DQoNCiAgIDQuICBQQ04tZmxv d3MgbWF5IGhhdmUgZGlmZmVyZW50IHByZWNlZGVuY2UsIGJ1dCB0aGUgYXBwbGljYWJpbGl0eSBv Zg0KICAgICAgIHRoZSBQQ04gbWVjaGFuaXNtcyBmb3IgZW1lcmdlbmN5IHVzZSAoOTExLCBHRVRT LCBXUFMsIE1MUFAsIGV0Yy4pDQogICAgICAgaXMgb3V0IG9mIHNjb3BlDQoNCiAgIEFmdGVyIGNv bXBsZXRpb24gb2YgdGhlIGluaXRpYWwgcGhhc2UsIHRoZSBQQ04gV0cgbWF5IHJlLWNoYXJ0ZXIg dG8NCiAgIGRldmVsb3Agc29sdXRpb25zIGZvciBzcGVjaWZpYyBzY2VuYXJpb3Mgd2hlcmUgc29t ZSBvZiB0aGVzZQ0KICAgcmVzdHJpY3Rpb25zIGFyZSBub3QgaW4gcGxhY2UuICBJdCBtYXkgYWxz byByZS1jaGFydGVyIHRvIGNvbnNpZGVyDQogICBhcHBseWluZyB0aGUgUENOIG1lY2hhbmlzbXMg dG8gYWRkaXRpb25hbCBkZXBsb3ltZW50IHNjZW5hcmlvcy4gIE9uZQ0KICAgcG9zc2libGUgZXhh bXBsZSBpcyB3aGVyZSBhIHNpbmdsZSBQQ04tZG9tYWluIGVuY29tcGFzc2VzIHNldmVyYWwNCiAg IERpZmZTZXJ2IGRvbWFpbnMgdGhhdCBkb24ndCB0cnVzdCBlYWNoIG90aGVyIChwZXJoYXBzIGJ5 IHVzaW5nIGENCiAgIG1lY2hhbmlzbSBsaWtlIHJlLUVDTiwgW0ktRC5icmlzY29lLXJlLXBjbi1i b3JkZXItY2hlYXRdLiAgVGhlIFdHIG1heQ0KICAgYWxzbyByZS1jaGFydGVyIHRvIGludmVzdGln YXRlIGFkZGl0aW9uYWwgcmVzcG9uc2UgbWVjaGFuaXNtcyB0aGF0DQogICBhY3Qgb24gKHByZS0p Y29uZ2VzdGlvbiBpbmZvcm1hdGlvbi4gIE9uZSBleGFtcGxlIGNvdWxkIGJlIGZsb3ctcmF0ZQ0K ICAgYWRhcHRhdGlvbiBieSBlbGFzdGljIGFwcGxpY2F0aW9ucyAocmF0aGVyIHRoYW4gZmxvdyBh ZG1pc3Npb24gb3INCiAgIHRlcm1pbmF0aW9uKS4gIFRoZSBkZXRhaWxzIG9mIHRoZXNlIHdvcmsg aXRlbXMgYXJlIG91dHNpZGUgdGhlIHNjb3BlDQogICBvZiB0aGUgaW5pdGlhbCBwaGFzZSwgYnV0 IHRoZSBXRyBtYXkgY29uc2lkZXIgdGhlaXIgcmVxdWlyZW1lbnRzIGluDQoNCg0KDQpFYXJkbGV5 IChFZGl0b3IpICAgICAgICAgIEV4cGlyZXMgTWF5IDIyLCAyMDA4ICAgICAgICAgICAgICAgICAg W1BhZ2UgOV0NCgwNCkludGVybmV0LURyYWZ0ICAgICAgICAgICAgICAgICAgRG9jdW1lbnQgICAg ICAgICAgICAgICAgICAgTm92ZW1iZXIgMjAwNw0KDQoNCiAgIG9yZGVyIHRvIGRlc2lnbiBjb21w b25lbnRzIHRoYXQgYXJlIHN1ZmZpY2llbnRseSBnZW5lcmFsIHRvIHN1cHBvcnQNCiAgIHN1Y2gg ZXh0ZW5zaW9ucyBpbiB0aGUgZnV0dXJlLiAgVGhlIHdvcmtpbmcgYXNzdW1wdGlvbiBpcyB0aGF0 IHRoZQ0KICAgc3RhbmRhcmRzIGRldmVsb3BlZCBpbiB0aGUgaW5pdGlhbCBwaGFzZSBzaG91bGQg bm90IG5lZWQgdG8gYmUNCiAgIG1vZGlmaWVkIHRvIHNhdGlzZnkgdGhlIHNvbHV0aW9ucyBmb3Ig d2hlbiB0aGVzZSByZXN0cmljdGlvbnMgYXJlDQogICByZW1vdmVkLg0KDQozLjEuICBBc3N1bXB0 aW9uIDE6IFRydXN0IC0gY29udHJvbGxlZCBlbnZpcm9ubWVudA0KDQogICBXZSBhc3N1bWUgdGhh dCB0aGUgUENOLWRvbWFpbiBpcyBhIGNvbnRyb2xsZWQgZW52aXJvbm1lbnQsIGkuZS4gYWxsDQog ICB0aGUgbm9kZXMgaW4gYSBQQ04tZG9tYWluIHJ1biBQQ04gYW5kIHRydXN0IGVhY2ggb3RoZXIu ICBUaGVyZSBhcmUNCiAgIHNldmVyYWwgcmVhc29ucyBmb3IgcHJvcG9zaW5nIHRoaXMgYXNzdW1w dGlvbjoNCg0KICAgbyAgVGhlIFBDTi1kb21haW4gaGFzIHRvIGJlIGVuY2lyY2xlZCBieSBhIHJp bmcgb2YgUENOLWJvdW5kYXJ5LQ0KICAgICAgbm9kZXMsIG90aGVyd2lzZSBQQ04tcGFja2V0cyBj b3VsZCBlbnRlciB0aGUgUENOLWRvbWFpbiB3aXRob3V0DQogICAgICBiZWluZyBzdWJqZWN0IHRv IGFkbWlzc2lvbiBjb250cm9sLCB3aGljaCB3b3VsZCBwb3RlbnRpYWxseQ0KICAgICAgZGVzdHJv eSB0aGUgUW9TIG9mIGV4aXN0aW5nIGZsb3dzLg0KDQogICBvICBTaW1pbGFybHksIGEgUENOLWJv dW5kYXJ5LW5vZGUgaGFzIHRvIHRydXN0IHRoYXQgYWxsIHRoZSBQQ04tbm9kZXMNCiAgICAgIGFy ZSBkb2luZyBQQ04tbWFya2luZy4gIEEgbm9uIFBDTi1ub2RlIHdvdWxkbid0IGJlIGFibGUgdG8g YWxlcnQNCiAgICAgIHRoYXQgaXQgaXMgc3VmZmVyaW5nIHByZS1jb25nZXN0aW9uLCB3aGljaCBw b3RlbnRpYWxseSB3b3VsZCBsZWFkDQogICAgICB0byB0b28gbWFueSBQQ04tZmxvd3MgYmVpbmcg YWRtaXR0ZWQgKG9yIHRvbyBmZXcgYmVpbmcNCiAgICAgIHRlcm1pbmF0ZWQpLiAgV29yc2UsIGEg cm9ndWUgbm9kZSBjb3VsZCBwZXJmb3JtIHZhcmlvdXMgYXR0YWNrcywNCiAgICAgIGFzIGRpc2N1 c3NlZCBpbiB0aGUgU2VjdXJpdHkgQ29uc2lkZXJhdGlvbnMgc2VjdGlvbi4NCg0KICAgT25lIHdh eSBvZiBhc3N1cmluZyB0aGUgYWJvdmUgdHdvIHBvaW50cyBpcyB0aGF0IHRoZSBlbnRpcmUgUENO LQ0KICAgZG9tYWluIGlzIHJ1biBieSBhIHNpbmdsZSBvcGVyYXRvci4gIEFub3RoZXIgcG9zc2li aWxpdHkgaXMgdGhhdA0KICAgdGhlcmUgYXJlIHNldmVyYWwgb3BlcmF0b3JzIGJ1dCB0aGV5IHRy dXN0IGVhY2ggb3RoZXIgdG8gYSBzdWZmaWNpZW50DQogICBsZXZlbCwgaW4gdGhlaXIgaGFuZGxp bmcgb2YgUENOLXRyYWZmaWMuDQoNCiAgIE5vdGU6IEFsbCBQQ04tbm9kZXMgbmVlZCB0byBiZSB0 cnVzdHdvcnRoeS4gIEhvd2V2ZXIgaWYgaXQncyBrbm93bg0KICAgdGhhdCBhbiBpbnRlcmZhY2Ug Y2Fubm90IGJlY29tZSBwcmUtY29uZ2VzdGVkIHRoZW4gaXQncyBub3Qgc3RyaWN0bHkNCiAgIG5l Y2Vzc2FyeSBmb3IgaXQgdG8gYmUgY2FwYWJsZSBvZiBQQ04tbWFya2luZy4gIEJ1dCB0aGlzIG11 c3QgYmUNCiAgIGtub3duIGV2ZW4gaW4gdW51c3VhbCBjaXJjdW1zdGFuY2VzLCBlZyBhZnRlciB0 aGUgZmFpbHVyZSBvZiBzb21lDQogICBsaW5rcy4NCg0KMy4yLiAgQXNzdW1wdGlvbiAyOiBSZWFs LXRpbWUgYXBwbGljYXRpb25zDQoNCiAgIFdlIGFzc3VtZSB0aGF0IGFueSB2YXJpYXRpb24gb2Yg c291cmNlIGJpdCByYXRlIGlzIGluZGVwZW5kZW50IG9mIHRoZQ0KICAgbGV2ZWwgb2YgcHJlLWNv bmdlc3Rpb24uICBXZSBhc3N1bWUgdGhhdCBQQ04tcGFja2V0cyBjb21lIGZyb20gcmVhbA0KICAg dGltZSBhcHBsaWNhdGlvbnMgZ2VuZXJhdGluZyBpbmVsYXN0aWMgdHJhZmZpYyBbU2hlbmtlcl0g bGlrZSB2b2ljZQ0KICAgYW5kIHZpZGVvIHJlcXVpcmluZyBsb3cgZGVsYXksIGppdHRlciBhbmQg cGFja2V0IGxvc3MsIGZvciBleGFtcGxlDQogICB0aGUgQ29udHJvbGxlZCBMb2FkIFNlcnZpY2Us IFtSRkMyMjExXSwgYW5kIHRoZSBUZWxlcGhvbnkgc2VydmljZQ0KICAgY2xhc3MsIFtSRkM0NTk0 XS4gIFRoaXMgYXNzdW1wdGlvbiBpcyB0byBoZWxwIGZvY3VzIHRoZSBlZmZvcnQgd2hlcmUNCiAg IGl0IGxvb2tzIGxpa2UgUENOIHdvdWxkIGJlIG1vc3QgdXNlZnVsLCBpZSB0aGUgc29ydHMgb2Yg YXBwbGljYXRpb25zDQogICB3aGVyZSBwZXIgZmxvdyBRb1MgaXMgYSBrbm93biByZXF1aXJlbWVu dC4gIEZvciBpbnN0YW5jZSwgdGhlIGltcGFjdA0KICAgb2YgdGhpcyBhc3N1bXB0aW9uIHdvdWxk IGJlIHRvIGd1aWRlIHNpbXVsYXRpb25zIHdvcmsuDQoNCg0KDQoNCg0KRWFyZGxleSAoRWRpdG9y KSAgICAgICAgICBFeHBpcmVzIE1heSAyMiwgMjAwOCAgICAgICAgICAgICAgICAgW1BhZ2UgMTBd DQoMDQpJbnRlcm5ldC1EcmFmdCAgICAgICAgICAgICAgICAgIERvY3VtZW50ICAgICAgICAgICAg ICAgICAgIE5vdmVtYmVyIDIwMDcNCg0KDQozLjMuICBBc3N1bXB0aW9uIDM6IE1hbnkgZmxvd3Mg YW5kIGFkZGl0aW9uYWwgbG9hZA0KDQogICBXZSBhc3N1bWUgdGhhdCB0aGVyZSBhcmUgbWFueSBm bG93cyBvbiBhbnkgYm90dGxlbmVjayBsaW5rIGluIHRoZQ0KICAgUENOLWRvbWFpbiAob3IsIHRv IHB1dCBpdCBhbm90aGVyIHdheSwgdGhlIGFnZ3JlZ2F0ZSBiaXQgcmF0ZSBvZiBQQ04tDQogICB0 cmFmZmljIGFjcm9zcyBhbnkgcG90ZW50aWFsIGJvdHRsZW5lY2sgbGluayBpcyBzdWZmaWNpZW50 bHkgbGFyZ2UNCiAgIHJlbGF0aXZlIHRvIHRoZSBtYXhpbXVtIGFkZGl0aW9uYWwgYml0IHJhdGUg YWRkZWQgYnkgb25lIGZsb3cpLg0KICAgTWVhc3VyZW1lbnQtYmFzZWQgYWRtaXNzaW9uIGNvbnRy b2wgYXNzdW1lcyB0aGF0IHRoZSBwcmVzZW50IGlzIGENCiAgIHJlYXNvbmFibGUgcHJlZGljdGlv biBvZiB0aGUgZnV0dXJlOiB0aGUgbmV0d29yayBjb25kaXRpb25zIGFyZQ0KICAgbWVhc3VyZWQg YXQgdGhlIHRpbWUgb2YgYSBuZXcgZmxvdyByZXF1ZXN0LCBob3dldmVyIHRoZSBhY3R1YWwNCiAg IG5ldHdvcmsgcGVyZm9ybWFuY2UgbXVzdCBiZSBPSyBkdXJpbmcgdGhlIGNhbGwgc29tZSB0aW1l IGxhdGVyLiAgT25lDQogICBpc3N1ZSBpcyB0aGF0IGlmIHRoZXJlIGFyZSBvbmx5IGEgZmV3IHZh cmlhYmxlIHJhdGUgZmxvd3MsIHRoZW4gdGhlDQogICBhZ2dyZWdhdGUgdHJhZmZpYyBsZXZlbCBt YXkgdmFyeSBhIGxvdCwgcGVyaGFwcyBlbm91Z2ggdG8gY2F1c2Ugc29tZQ0KICAgcGFja2V0cyB0 byBnZXQgZHJvcHBlZC4gIElmIHRoZXJlIGFyZSBtYW55IGZsb3dzIHRoZW4gdGhlIGFnZ3JlZ2F0 ZQ0KICAgdHJhZmZpYyBsZXZlbCBzaG91bGQgYmUgc3RhdGlzdGljYWxseSBzbW9vdGhlZC4gIEhv dyBtYW55IGZsb3dzIGlzDQogICBlbm91Z2ggZGVwZW5kcyBvbiBhIG51bWJlciBvZiB0aGluZ3Mg c3VjaCBhcyB0aGUgdmFyaWF0aW9uIGluIGVhY2gNCiAgIGZsb3cncyByYXRlLCB0aGUgdG90YWwg cmF0ZSBvZiBQQ04tdHJhZmZpYywgYW5kIHRoZSBzaXplIG9mIHRoZQ0KICAgInNhZmV0eSBtYXJn aW4iIGJldHdlZW4gdGhlIHRyYWZmaWMgbGV2ZWwgYXQgd2hpY2ggd2Ugc3RhcnQNCiAgIGFkbWlz c2lvbi1tYXJraW5nIGFuZCBhdCB3aGljaCBwYWNrZXRzIGFyZSBkcm9wcGVkIG9yIHNpZ25pZmlj YW50bHkNCiAgIGRlbGF5ZWQuDQoNCiAgIFdlIGRvIG5vdCBtYWtlIGV4cGxpY2l0IGFzc3VtcHRp b25zIG9uIGhvdyBtYW55IFBDTi1mbG93cyBhcmUgaW4gZWFjaA0KICAgaW5ncmVzcy1lZ3Jlc3Mt YWdncmVnYXRlLiAgUGVyZm9ybWFuY2UgZXZhbHVhdGlvbiB3b3JrIG1heSBjbGFyaWZ5DQogICB3 aGV0aGVyIGl0IGlzIG5lY2Vzc2FyeSB0byBtYWtlIGFueSBhZGRpdGlvbmFsIGFzc3VtcHRpb24g b24NCiAgIGFnZ3JlZ2F0aW9uIGF0IHRoZSBpbmdyZXNzLWVncmVzcy1hZ2dyZWdhdGUgbGV2ZWwu DQoNCjMuNC4gIEFzc3VtcHRpb24gNDogRW1lcmdlbmN5IHVzZSBvdXQgb2Ygc2NvcGUNCg0KICAg UENOLWZsb3dzIG1heSBoYXZlIGRpZmZlcmVudCBwcmVjZWRlbmNlLCBidXQgdGhlIGFwcGxpY2Fi aWxpdHkgb2YgdGhlDQogICBQQ04gbWVjaGFuaXNtcyBmb3IgZW1lcmdlbmN5IHVzZSAoOTExLCBH RVRTLCBXUFMsIE1MUFAsIGV0YykgaXMgb3V0DQogICBvZiBzY29wZSBmb3IgY29uc2lkZXJhdGlv biBieSB0aGUgUENOIFdHLg0KDQozLjUuICBPdGhlciBhc3N1bXB0aW9ucw0KDQogICBBcyBhIGNv bnNlcXVlbmNlIG9mIEFzc3VtcHRpb24gMiBhYm92ZSwgaXQgaXMgYXNzdW1lZCB0aGF0IFBDTi0N CiAgIG1hcmtpbmcgaXMgYmVpbmcgYXBwbGllZCB0byB0cmFmZmljIHNjaGVkdWxlZCB3aXRoIHRo ZSBleHBlZGl0ZWQNCiAgIGZvcndhcmRpbmcgcGVyLWhvcCBiZWhhdmlvdXIsIFtSRkMzMjQ2XSwg b3IgdHJhZmZpYyB3aXRoIHNpbWlsYXINCiAgIGNoYXJhY3RlcmlzdGljcy4NCg0KICAgVGhlIGZv bGxvd2luZyB0d28gYXNzdW1wdGlvbnMgYXBwbHkgaWYgdGhlIFBDTiBXRyBkZWNpZGVzIHRvIGVu Y29kZQ0KICAgUENOLW1hcmtpbmcgaW4gdGhlIEVDTi1maWVsZC4NCg0KICAgbyAgSXQgaXMgYXNz dW1lZCB0aGF0IFBDTi1ub2RlcyBkbyBub3QgcGVyZm9ybSBFQ04sIFtSRkMzMTY4XSwgb24NCiAg ICAgIFBDTi1wYWNrZXRzLg0KDQogICBvICBJZiBhIHBhY2tldCB0aGF0IGlzIHBhcnQgb2YgYSBQ Q04tZmxvdyBhcnJpdmVzIGF0IGEgUENOLWluZ3Jlc3MtDQogICAgICBub2RlIHdpdGggaXRzIENF IChDb25nZXN0aW9uIGV4cGVyaWVuY2VkKSBjb2RlcG9pbnQgc2V0LCB0aGVuIHdlDQogICAgICBh c3N1bWUgdGhhdCB0aGUgUENOLWluZ3Jlc3Mtbm9kZSBkcm9wcyB0aGUgcGFja2V0LiAgQWZ0ZXIg aXRzDQogICAgICBpbml0aWFsIENoYXJ0ZXIgaXMgY29tcGxldGUsIHRoZSBXRyBtYXkgZGVjaWRl IHRvIHdvcmsgb24gYQ0KDQoNCg0KRWFyZGxleSAoRWRpdG9yKSAgICAgICAgICBFeHBpcmVzIE1h eSAyMiwgMjAwOCAgICAgICAgICAgICAgICAgW1BhZ2UgMTFdDQoMDQpJbnRlcm5ldC1EcmFmdCAg ICAgICAgICAgICAgICAgIERvY3VtZW50ICAgICAgICAgICAgICAgICAgIE5vdmVtYmVyIDIwMDcN Cg0KDQogICAgICBtZWNoYW5pc20gKHN1Y2ggYXMgdGhyb3VnaCBhIHNpZ25hbGxpbmcgZXh0ZW5z aW9uKSB0aGF0IGVuYWJsZXMNCiAgICAgIEVDTi1tYXJraW5nIHRvIGJlIGNhcnJpZWQgdHJhbnNw YXJlbnRseSBhY3Jvc3MgdGhlIFBDTi1kb21haW4uDQoNCg0KNC4gIEhpZ2gtbGV2ZWwgZnVuY3Rp b25hbCBhcmNoaXRlY3R1cmUNCg0KICAgVGhlIGhpZ2gtbGV2ZWwgYXBwcm9hY2ggaXMgdG8gc3Bs aXQgZnVuY3Rpb25hbGl0eSBiZXR3ZWVuOg0KDQogICBvICBQQ04taW50ZXJpb3Itbm9kZXMgJ2lu c2lkZScgdGhlIFBDTi1kb21haW4sIHdoaWNoIG1vbml0b3IgdGhlaXINCiAgICAgIG93biBzdGF0 ZSBvZiBwcmUtY29uZ2VzdGlvbiBvbiBlYWNoIG91dGdvaW5nIGludGVyZmFjZSBhbmQgbWFyaw0K ICAgICAgUENOLXBhY2tldHMgaWYgYXBwcm9wcmlhdGUuICBUaGV5IGFyZSBub3QgZmxvdy1hd2Fy ZSwgbm9yIGF3YXJlIG9mDQogICAgICBpbmdyZXNzLWVncmVzcy1hZ2dyZWdhdGVzLiAgVGhlIGZ1 bmN0aW9uYWxpdHkgaXMgYWxzbyBkb25lIGJ5IFBDTi0NCiAgICAgIGluZ3Jlc3Mtbm9kZXMgZm9y IHRoZWlyIG91dGdvaW5nIGludGVyZmFjZXMgKGllIHRob3NlICdpbnNpZGUnIHRoZQ0KICAgICAg UENOLWRvbWFpbikuDQoNCiAgIG8gIFBDTi1ib3VuZGFyeS1ub2RlcyBhdCB0aGUgZWRnZSBvZiB0 aGUgUENOLWRvbWFpbiwgd2hpY2ggY29udHJvbA0KICAgICAgYWRtaXNzaW9uIG9mIG5ldyBQQ04t Zmxvd3MgYW5kIHRlcm1pbmF0aW9uIG9mIGV4aXN0aW5nIFBDTi1mbG93cywNCiAgICAgIGJhc2Vk IG9uIGluZm9ybWF0aW9uIGZyb20gUENOLWludGVyaW9yLW5vZGVzLiAgVGhpcyBpbmZvcm1hdGlv biBpcw0KICAgICAgaW4gdGhlIGZvcm0gb2YgdGhlIFBDTi1tYXJrZWQgZGF0YSBwYWNrZXRzICh3 aGljaCBhcmUgaW50ZXJjZXB0ZWQNCiAgICAgIGJ5IHRoZSBQQ04tZWdyZXNzLW5vZGVzKSBhbmQg bm90IHNpZ25hbGxpbmcgbWVzc2FnZXMuICBHZW5lcmFsbHkNCiAgICAgIFBDTi1pbmdyZXNzLW5v ZGVzIGFyZSBmbG93LWF3YXJlIGFuZCBpbiBzZXZlcmFsIGRlcGxveW1lbnQNCiAgICAgIHNjZW5h cmlvcyBQQ04tZWdyZXNzLW5vZGVzIHdpbGwgYWxzbyBiZSBmbG93IGF3YXJlLg0KDQogICBUaGUg YWltIG9mIHRoaXMgc3BsaXQgaXMgdG8ga2VlcCB0aGUgYnVsayBvZiB0aGUgbmV0d29yayBzaW1w bGUsDQogICBzY2FsYWJsZSBhbmQgcm9idXN0LCB3aGlsc3QgY29uZmluaW5nIHBvbGljeSwgYXBw bGljYXRpb24tbGV2ZWwgYW5kDQogICBzZWN1cml0eSBpbnRlcmFjdGlvbnMgdG8gdGhlIGVkZ2Ug b2YgdGhlIFBDTi1kb21haW4uICBGb3IgZXhhbXBsZSB0aGUNCiAgIGxhY2sgb2YgZmxvdyBhd2Fy ZW5lc3MgbWVhbnMgdGhhdCB0aGUgUENOLWludGVyaW9yLW5vZGVzIGRvbid0IGNhcmUNCiAgIGFi b3V0IHRoZSBmbG93IGluZm9ybWF0aW9uIGFzc29jaWF0ZWQgd2l0aCB0aGUgUENOLXBhY2tldHMg dGhhdCB0aGV5DQogICBjYXJyeSwgbm9yIGRvIHRoZSBQQ04tYm91bmRhcnktbm9kZXMgY2FyZSBh Ym91dCB3aGljaCBQQ04taW50ZXJpb3ItDQogICBub2RlcyBpdHMgZmxvd3MgdHJhdmVyc2UuDQoN CjQuMS4gIEZsb3cgYWRtaXNzaW9uDQoNCiAgIEF0IGEgaGlnaCBsZXZlbCwgZmxvdyBhZG1pc3Np b24gY29udHJvbCB3b3JrcyBhcyBmb2xsb3dzLiAgSW4gb3JkZXINCiAgIHRvIGdlbmVyYXRlIGlu Zm9ybWF0aW9uIGFib3V0IHRoZSBjdXJyZW50IHN0YXRlIG9mIHRoZSBQQ04tZG9tYWluLA0KICAg ZWFjaCBQQ04tbm9kZSBQQ04tbWFya3MgcGFja2V0cyBpZiBpdCBpcyAicHJlLWNvbmdlc3RlZCIu ICBFeGFjdGx5DQogICBob3cgYSBQQ04tbm9kZSBkZWNpZGVzIGlmIGl0IGlzICJwcmUtY29uZ2Vz dGVkIiAodGhlIGFsZ29yaXRobSkgYW5kDQogICBleGFjdGx5IGhvdyBwYWNrZXRzIGFyZSAiUENO LW1hcmtlZCIgKHRoZSBlbmNvZGluZykgd2lsbCBiZSBkZWZpbmVkDQogICBpbiBhIHNlcGFyYXRl IHN0YW5kYXJkcy10cmFjayBkb2N1bWVudCwgYnV0IGF0IGEgaGlnaCBsZXZlbCBpdCBpcw0KICAg ZXhwZWN0ZWQgdG8gYmUgYXMgZm9sbG93czoNCg0KICAgbyAgdGhlIGFsZ29yaXRobTogYSBQQ04t bm9kZSBtZXRlcnMgdGhlIGFtb3VudCBvZiBQQ04tdHJhZmZpYyBvbiBlYWNoDQogICAgICBvbmUg b2YgaXRzIG91dGdvaW5nIGxpbmtzLiAgVGhlIG1lYXN1cmVtZW50IGlzIG1hZGUgYXMgYW4NCiAg ICAgIGFnZ3JlZ2F0ZSBvZiBhbGwgUENOLXBhY2tldHMsIGFuZCBub3QgcGVyIGZsb3cuICBUaGUg YWxnb3JpdGhtIGhhcw0KICAgICAgYSBjb25maWd1cmVkIHBhcmFtZXRlciwgUENOLWxvd2VyLXJh dGUuICBBcyB0aGUgYW1vdW50IG9mIFBDTi0NCiAgICAgIHRyYWZmaWMgZXhjZWVkcyB0aGUgUENO LWxvd2VyLXJhdGUsIHRoZW4gUENOLXBhY2tldHMgYXJlIFBDTi0NCiAgICAgIG1hcmtlZC4gIFNl ZSBOT1RFIGJlbG93IGZvciBtb3JlIGV4cGxhbmF0aW9uLg0KDQoNCg0KDQpFYXJkbGV5IChFZGl0 b3IpICAgICAgICAgIEV4cGlyZXMgTWF5IDIyLCAyMDA4ICAgICAgICAgICAgICAgICBbUGFnZSAx Ml0NCgwNCkludGVybmV0LURyYWZ0ICAgICAgICAgICAgICAgICAgRG9jdW1lbnQgICAgICAgICAg ICAgICAgICAgTm92ZW1iZXIgMjAwNw0KDQoNCiAgIG8gIHRoZSBlbmNvZGluZzogYSBQQ04tbm9k ZSBQQ04tbWFya3MgYSBQQ04tcGFja2V0ICh3aXRoIGEgZmlyc3QNCiAgICAgIGVuY29kaW5nKSBi eSBzZXR0aW5nIGZpZWxkcyBpbiB0aGUgaGVhZGVyIHRvIHNwZWNpZmljIHZhbHVlcy4gIEl0DQog ICAgICBpcyBleHBlY3RlZCB0aGF0IHRoZSBFQ04gYW5kL29yIERTQ1AgZmllbGRzIHdpbGwgYmUg dXNlZC4NCg0KICAgTk9URTogVHdvIG1haW4gY2F0ZWdvcmllcyBvZiBhbGdvcml0aG0gaGF2ZSBi ZWVuIHByb3Bvc2VkOiBpZiB0aGUNCiAgIGFsZ29yaXRobSB1c2VzIHRocmVzaG9sZC1tYXJraW5n IHRoZW4gYWxsIFBDTi1wYWNrZXRzIGFyZSBtYXJrZWQgaWYNCiAgIHRoZSBjdXJyZW50IHJhdGUg ZXhjZWVkcyB0aGUgUENOLWxvd2VyLXJhdGUsIHdoZXJlYXMgaWYgdGhlIGFsZ29yaXRobQ0KICAg dXNlcyBleGNlc3MtcmF0ZS1tYXJraW5nIHRoZSBhbW91bnQgbWFya2VkIGlzIGVxdWFsIHRvIHRo ZSBhbW91bnQgaW4NCiAgIGV4Y2VzcyBvZiB0aGUgUENOLWxvd2VyLXJhdGUuICBIb3dldmVyLCBu b3RlIHRoYXQgdGhpcyBkZXNjcmlwdGlvbg0KICAgcmVmbGVjdHMgdGhlIG92ZXJhbGwgaW50ZW50 IG9mIHRoZSBhbGdvcml0aG0gcmF0aGVyIHRoYW4gaXRzDQogICBpbnN0YW50YW5lb3VzIGJlaGF2 aW91ciwgc2luY2UgdGhlIHJhdGUgbWVhc3VyZWQgYXQgYSBwYXJ0aWN1bGFyDQogICBtb21lbnQg ZGVwZW5kcyBvbiB0aGUgZGV0YWlsZWQgYWxnb3JpdGhtLCBpdHMgaW1wbGVtZW50YXRpb24gKGVn DQogICB2aXJ0dWFsIHF1ZXVlLCB0b2tlbiBidWNrZXQuLi4pIGFuZCB0aGUgdHJhZmZpYydzIHZh cmlhbmNlIGFzIHdlbGwgYXMNCiAgIGl0cyByYXRlIChlZyBtYXJraW5nIG1heSB3ZWxsIGNvbnRp bnVlIGFmdGVyIGEgcmVjZW50IG92ZXJsb2FkIGV2ZW4NCiAgIGFmdGVyIHRoZSBpbnN0YW50YW5l b3VzIHJhdGUgaGFzIGRyb3BwZWQpLg0KDQogICBUaGUgUENOLWJvdW5kYXJ5LW5vZGVzIG1vbml0 b3IgdGhlIFBDTi1tYXJrZWQgcGFja2V0cyBpbiBvcmRlciB0bw0KICAgZXh0cmFjdCBpbmZvcm1h dGlvbiBhYm91dCB0aGUgY3VycmVudCBzdGF0ZSBvZiB0aGUgUENOLWRvbWFpbi4gIEJhc2VkDQog ICBvbiB0aGlzIG1vbml0b3JpbmcsIGEgZGVjaXNpb24gaXMgbWFkZSBhYm91dCB3aGV0aGVyIHRv IGFkbWl0IGENCiAgIHByb3NwZWN0aXZlIG5ldyBmbG93LiAgRXhhY3RseSBob3cgdGhlIGFkbWlz c2lvbiBjb250cm9sIGRlY2lzaW9uIGlzDQogICBtYWRlIHdpbGwgYmUgZGVmaW5lZCBzZXBhcmF0 ZWx5IChhdCB0aGUgbW9tZW50IHRoZSBpbnRlbnRpb24gaXMgdGhhdA0KICAgdGhlcmUgd2lsbCBi ZSBvbmUgb3IgbW9yZSBpbmZvcm1hdGlvbmFsLXRyYWNrIFJGQ3MpLCBidXQgYXQgYSBoaWdoDQog ICBsZXZlbCB0d28gYXBwcm9hY2hlcyBoYXZlIGJlZW4gcHJvcG9zZWQgdG8gZGF0ZToNCg0KICAg byAgdGhlIFBDTi1lZ3Jlc3Mtbm9kZSBtZWFzdXJlcyAocG9zc2libHkgYXMgYSBtb3ZpbmcgYXZl cmFnZSkgdGhlDQogICAgICBmcmFjdGlvbiBvZiB0aGUgUENOLXRyYWZmaWMgdGhhdCBpcyBQQ04t bWFya2VkLiAgVGhlIGZyYWN0aW9uIGlzDQogICAgICBtZWFzdXJlZCBmb3IgYSBzcGVjaWZpYyBp bmdyZXNzLWVncmVzcy1hZ2dyZWdhdGUuICBJZiB0aGUgZnJhY3Rpb24NCiAgICAgIGlzIGJlbG93 IGEgdGhyZXNob2xkIHZhbHVlIHRoZW4gdGhlIG5ldyBmbG93IGlzIGFkbWl0dGVkLg0KDQogICBv ICBpZiB0aGUgUENOLWVncmVzcy1ub2RlIHJlY2VpdmVzIG9uZSAob3Igc2V2ZXJhbCkgUENOLW1h cmtlZA0KICAgICAgcGFja2V0cywgdGhlbiBhIG5ldyBmbG93IGlzIGJsb2NrZWQuDQoNCiAgIE5v dGUgdGhhdCB0aGUgUENOLWxvd2VyLXJhdGUgaXMgYSBwYXJhbWV0ZXIgdGhhdCBjYW4gYmUgY29u ZmlndXJlZCBieQ0KICAgdGhlIG9wZXJhdG9yLiAgSXQgd2lsbCBiZSBzZXQgbG93ZXIgdGhhbiB0 aGUgdHJhZmZpYyByYXRlIGF0IHdoaWNoDQogICB0aGUgbGluayBiZWNvbWVzIGNvbmdlc3RlZCBh bmQgdGhlIG5vZGUgZHJvcHMgcGFja2V0cy4gIChIZW5jZSwgYnkNCiAgIGFuYWxvZ3kgd2l0aCBF Q04gd2UgY2FsbCBvdXIgbWVjaGFuaXNtIFByZS1Db25nZXN0aW9uIE5vdGlmaWNhdGlvbi4pDQoN CiAgIE5vdGUgYWxzbyB0aGF0IHRoZSBhZG1pc3Npb24gY29udHJvbCBkZWNpc2lvbiBpcyBtYWRl IGZvciBhDQogICBwYXJ0aWN1bGFyIGluZ3Jlc3MtZWdyZXNzLWFnZ3JlZ2F0ZS4gIFNvIGl0IGlz IHF1aXRlIHBvc3NpYmxlIGZvciBhDQogICBuZXcgZmxvdyB0byBiZSBhZG1pdHRlZCBiZXR3ZWVu IG9uZSBwYWlyIG9mIFBDTi1ib3VuZGFyeS1ub2RlcywNCiAgIHdoaWxzdCBhdCB0aGUgc2FtZSB0 aW1lIGFub3RoZXIgYWRtaXNzaW9uIHJlcXVlc3QgaXMgYmxvY2tlZCBiZXR3ZWVuDQogICBhIGRp ZmZlcmVudCBwYWlyIG9mIFBDTi1ib3VuZGFyeS1ub2Rlcy4NCg0KNC4yLiAgRmxvdyB0ZXJtaW5h dGlvbg0KDQogICBBdCBhIGhpZ2ggbGV2ZWwsIGZsb3cgdGVybWluYXRpb24gY29udHJvbCB3b3Jr cyBhcyBmb2xsb3dzLiAgRWFjaA0KICAgUENOLW5vZGUgUENOLW1hcmtzIHBhY2tldHMgaW4gYSBz aW1pbGFyIGZhc2hpb24gdG8gYWJvdmUuICBBbiBvYnZpb3VzDQogICBhcHByb2FjaCBpcyBmb3Ig dGhlIGFsZ29yaXRobSB0byB1c2UgYSBzZWNvbmQgY29uZmlndXJlZCBwYXJhbWV0ZXIsDQoNCg0K DQpFYXJkbGV5IChFZGl0b3IpICAgICAgICAgIEV4cGlyZXMgTWF5IDIyLCAyMDA4ICAgICAgICAg ICAgICAgICBbUGFnZSAxM10NCgwNCkludGVybmV0LURyYWZ0ICAgICAgICAgICAgICAgICAgRG9j dW1lbnQgICAgICAgICAgICAgICAgICAgTm92ZW1iZXIgMjAwNw0KDQoNCiAgIFBDTi11cHBlci1y YXRlLCBhbmQgYSBzZWNvbmQgaGVhZGVyIGVuY29kaW5nLiAgSG93ZXZlciB0aGVyZSBpcyBhbHNv DQogICBhIHByb3Bvc2FsIHRvIHVzZSB0aGUgc2FtZSByYXRlIGFuZCB0aGUgc2FtZSBlbmNvZGlu Zy4gIFNldmVyYWwNCiAgIGFwcHJvYWNoZXMgaGF2ZSBiZWVuIHByb3Bvc2VkIHRvIGRhdGUgYWJv dXQgaG93IHRvIGNvbnZlcnQgdGhpcw0KICAgaW5mb3JtYXRpb24gaW50byBhIGZsb3cgdGVybWlu YXRpb24gZGVjaXNpb247IGF0IGEgaGlnaCBsZXZlbCB0aGVzZQ0KICAgYXJlIGFzIGZvbGxvd3M6 DQoNCiAgIG8gIE9uZSBhcHByb2FjaCBtZWFzdXJlcyB0aGUgcmF0ZSBvZiB1bm1hcmtlZCBQQ04t dHJhZmZpYyAoaWUgbm90DQogICAgICBQQ04tdXBwZXItcmF0ZS1tYXJrZWQpIGF0IHRoZSBQQ04t ZWdyZXNzLW5vZGUsIHdoaWNoIGlzIHRoZSBhbW91bnQNCiAgICAgIG9mIFBDTi10cmFmZmljIHRo YXQgY2FuIGFjdHVhbGx5IGJlIHN1cHBvcnRlZDsgdGhlIFBDTi1pbmdyZXNzLQ0KICAgICAgbm9k ZSBtZWFzdXJlcyB0aGUgcmF0ZSBvZiBQQ04tdHJhZmZpYyB0aGF0IGlzIGRlc3RpbmVkIGZvciB0 aGlzDQogICAgICBzcGVjaWZpYyBQQ04tZWdyZXNzLW5vZGUsIGFuZCBoZW5jZSBjYW4gY2FsY3Vs YXRlIHRoZSBleGNlc3MNCiAgICAgIGFtb3VudCB0aGF0IHNob3VsZCBiZSB0ZXJtaW5hdGVkLg0K DQogICBvICBBbm90aGVyIGFwcHJvYWNoIGluc3RlYWQgbWVhc3VyZXMgdGhlIHJhdGUgb2YgUENO LXVwcGVyLXJhdGUtDQogICAgICBtYXJrZWQgdHJhZmZpYyBhbmQgY2FsY3VsYXRlcyBhbmQgc2Vs ZWN0cyB0aGUgZmxvd3MgdGhhdCBzaG91bGQgYmUNCiAgICAgIHRlcm1pbmF0ZWQuDQoNCiAgIG8g IEFub3RoZXIgYXBwcm9hY2ggdGVybWluYXRlcyBhbnkgUENOLWZsb3cgd2l0aCBhIFBDTi11cHBl ci1yYXRlLQ0KICAgICAgbWFya2VkIHBhY2tldC4gIENvbXBhcmVkIHdpdGggdGhlIGFwcHJvYWNo ZXMgYWJvdmUsIFBDTi1tYXJraW5nDQogICAgICBuZWVkcyB0byBiZSBkb25lIGF0IGEgcmVkdWNl ZCByYXRlIG90aGVyd2lzZSBmYXIgdG9vIG11Y2ggdHJhZmZpYw0KICAgICAgd291bGQgYmUgdGVy bWluYXRlZC4NCg0KICAgbyAgQW5vdGhlciBhcHByb2FjaCB1c2VzIG9ubHkgb25lIHNvcnQgb2Yg bWFya2luZywgd2hpY2ggaXMgYmFzZWQgb24NCiAgICAgIHRoZSBQQ04tbG93ZXItcmF0ZSwgdG8g ZGVjaWRlIG5vdCBvbmx5IHdoZXRoZXIgdG8gYWRtaXQgbW9yZSBQQ04tDQogICAgICBmbG93cyBi dXQgYWxzbyB3aGV0aGVyIGFueSBQQ04tZmxvd3MgbmVlZCB0byBiZSB0ZXJtaW5hdGVkLiAgSXQN CiAgICAgIGFzc3VtZXMgdGhhdCB0aGUgcmF0aW8gb2YgdGhlIChpbXBsaWNpdCkgUENOLXVwcGVy LXJhdGUgYW5kIHRoZQ0KICAgICAgUENOLWxvd2VyLXJhdGUgaXMgdGhlIHNhbWUgb24gYWxsIGxp bmtzLiAgVGhpcyBhcHByb2FjaCBtZWFzdXJlcw0KICAgICAgdGhlIHJhdGUgb2YgdW5tYXJrZWQg UENOLXRyYWZmaWMgYXQgYSBQQ04tZWdyZXNzLW5vZGUuICBUaGUgUENOLQ0KICAgICAgaW5ncmVz cy1ub2RlIHVzZXMgdGhpcyBtZWFzdXJlbWVudCB0byBjb21wdXRlIHRoZSBpbXBsaWNpdCBQQ04t DQogICAgICB1cHBlci1yYXRlIG9mIHRoZSBib3R0bGVuZWNrIGxpbmsuICBJdCB0aGVuIG1lYXN1 cmVzIHRoZSByYXRlIG9mDQogICAgICBQQ04tdHJhZmZpYyB0aGF0IGlzIGRlc3RpbmVkIGZvciB0 aGlzIHNwZWNpZmljIFBDTi1lZ3Jlc3Mtbm9kZSBhbmQNCiAgICAgIGhlbmNlIGNhbiBjYWxjdWxh dGUgdGhlIGFtb3VudCB0aGF0IHNob3VsZCBiZSB0ZXJtaW5hdGVkLg0KDQogICBTaW5jZSBmbG93 IHRlcm1pbmF0aW9uIGlzIGRlc2lnbmVkIGZvciAiYWJub3JtYWwiIGNpcmN1bXN0YW5jZXMsIGl0 DQogICBpcyBxdWl0ZSBsaWtlbHkgdGhhdCBzb21lIFBDTi1ub2RlcyBhcmUgY29uZ2VzdGVkIGFu ZCBoZW5jZSBwYWNrZXRzDQogICBhcmUgYmVpbmcgZHJvcHBlZCBhbmQvb3Igc2lnbmlmaWNhbnRs eSBxdWV1ZWQuICBUaGUgZmxvdyB0ZXJtaW5hdGlvbg0KICAgbWVjaGFuaXNtIG11c3QgYmVhciB0 aGlzIGluIG1pbmQuDQoNCiAgIE5vdGUgYWxzbyB0aGF0IHRoZSB0ZXJtaW5hdGlvbiBjb250cm9s IGRlY2lzaW9uIGlzIG1hZGUgZm9yIGENCiAgIHBhcnRpY3VsYXIgaW5ncmVzcy1lZ3Jlc3MtYWdn cmVnYXRlLiAgU28gaXQgaXMgcXVpdGUgcG9zc2libGUgZm9yDQogICBQQ04tZmxvd3MgdG8gYmUg dGVybWluYXRlZCBiZXR3ZWVuIG9uZSBwYWlyIG9mIFBDTi1ib3VuZGFyeS1ub2RlcywNCiAgIHdo aWxzdCBhdCB0aGUgc2FtZSB0aW1lIG5vbmUgYXJlIHRlcm1pbmF0ZWQgYmV0d2VlbiBhIGRpZmZl cmVudCBwYWlyDQogICBvZiBQQ04tYm91bmRhcnktbm9kZXMuDQoNCjQuMy4gIEZsb3cgYWRtaXNz aW9uIGFuZCBmbG93IHRlcm1pbmF0aW9uDQoNCiAgIEFsdGhvdWdoIGRlc2lnbmVkIHRvIHdvcmsg dG9nZXRoZXIsIGZsb3cgYWRtaXNzaW9uIGFuZCBmbG93DQogICB0ZXJtaW5hdGlvbiBhcmUgaW5k ZXBlbmRlbnQgbWVjaGFuaXNtcywgYW5kIHRoZSB1c2Ugb2Ygb25lIGRvZXMgbm90DQoNCg0KDQpF YXJkbGV5IChFZGl0b3IpICAgICAgICAgIEV4cGlyZXMgTWF5IDIyLCAyMDA4ICAgICAgICAgICAg ICAgICBbUGFnZSAxNF0NCgwNCkludGVybmV0LURyYWZ0ICAgICAgICAgICAgICAgICAgRG9jdW1l bnQgICAgICAgICAgICAgICAgICAgTm92ZW1iZXIgMjAwNw0KDQoNCiAgIHJlcXVpcmUgb3IgcHJl dmVudCB0aGUgdXNlIG9mIHRoZSBvdGhlci4NCg0KICAgRm9yIGV4YW1wbGUsIGFuIG9wZXJhdG9y IGNvdWxkIHVzZSBqdXN0IGFkbWlzc2lvbiBjb250cm9sLCBzb2x2aW5nDQogICBoZWF2eSBjb25n ZXN0aW9uIChjYXVzZWQgYnkgcmUtcm91dGluZykgYnkgJ2p1c3Qgd2FpdGluZycgLSBhcw0KICAg c2Vzc2lvbnMgZW5kLCBleGlzdGluZyBtaWNyb2Zsb3dzIG5hdHVyYWxseSBkZXBhcnQgZnJvbSB0 aGUgc3lzdGVtDQogICBvdmVyIHRpbWUsIGFuZCB0aGUgYWRtaXNzaW9uIGNvbnRyb2wgbWVjaGFu aXNtIHdpbGwgcHJldmVudCBhZG1pc3Npb24NCiAgIG9mIG5ldyBtaWNyb2Zsb3dzIHRoYXQgdXNl IHRoZSBhZmZlY3RlZCBsaW5rcy4gIFNvIHRoZSBQQ04tZG9tYWluDQogICB3aWxsIG5hdHVyYWxs eSByZXR1cm4gdG8gbm9ybWFsIG9wZXJhdGlvbiwgYnV0IHdpdGggcmVkdWNlZCBjYXBhY2l0eS4N CiAgIFRoZSBkcmF3YmFjayBvZiB0aGlzIGFwcHJvYWNoIHdvdWxkIGJlIHRoYXQgdW50aWwgUENO LWZsb3dzIG5hdHVyYWxseQ0KICAgZGVwYXJ0IHRvIHJlbGlldmUgdGhlIGNvbmdlc3Rpb24sIGFs bCBQQ04tZmxvd3MgYXMgd2VsbCBhcyBsb3dlcg0KICAgcHJpb3JpdHkgc2VydmljZXMgd2lsbCBi ZSBhZHZlcnNlbHkgYWZmZWN0ZWQuICBPbiB0aGUgb3RoZXIgaGFuZCwgYW4NCiAgIG9wZXJhdG9y IGNvdWxkIGp1c3QgcmVseSBmb3IgYWRtaXNzaW9uIGNvbnRyb2wgb24gc3RhdGljYWxseQ0KICAg cHJvdmlzaW9uZWQgY2FwYWNpdHkgcGVyIFBDTi1pbmdyZXNzLW5vZGUgKHJlZ2FyZGxlc3Mgb2Yg dGhlIFBDTi0NCiAgIGVncmVzcy1ub2RlIG9mIGEgZmxvdyksIGFzIGlzIHR5cGljYWwgaW4gdGhl IGhvc2UgbW9kZWwgb2YgdGhlDQogICBEaWZmU2VydiBhcmNoaXRlY3R1cmUgW1JGQzI0NzVdLiAg U3VjaCB0cmFmZmljIGNvbmRpdGlvbmluZw0KICAgYWdyZWVtZW50cyBjYW4gbGVhZCB0byBmb2N1 c2VkIG92ZXJsb2FkOiBtYW55IGZsb3dzIGhhcHBlbiB0byBmb2N1cw0KICAgb24gYSBwYXJ0aWN1 bGFyIGxpbmsgYW5kIHRoZW4gYWxsIGZsb3dzIHRocm91Z2ggdGhlIGNvbmdlc3RlZCBsaW5rDQog ICBmYWlsIGNhdGFzdHJvcGhpY2FsbHkuICBUaGUgZmxvdyB0ZXJtaW5hdGlvbiBtZWNoYW5pc20g Y291bGQgdGhlbiBiZQ0KICAgdXNlZCB0byBjb3VudGVyYWN0IHN1Y2ggYSBwcm9ibGVtLg0KDQog ICBBIGRpZmZlcmVudCBwb3NzaWJpbGl0eSBpcyB0byBjb25maWd1cmUgb25seSB0aGUgUENOLWxv d2VyLXJhdGUgYW5kDQogICBoZW5jZSBvbmx5IGRvIG9uZSB0eXBlIG9mIFBDTi1tYXJraW5nLCBi dXQgZ2VuZXJhdGUgYWRtaXNzaW9uIGFuZA0KICAgZmxvdyB0ZXJtaW5hdGlvbiByZXNwb25zZXMg ZnJvbSBkaWZmZXJlbnQgbGV2ZWxzIG9mIG1hcmtpbmcuICBUaGlzIGlzDQogICBzdWdnZXN0ZWQg aW4gW0ktRC5jaGFybnktcGNuLXNpbmdsZS1tYXJraW5nXSB3aGljaCBnaXZlcyBzb21lIG9mIHRo ZQ0KICAgcHJvcyBhbmQgY29ucyBvZiB0aGlzIGFwcHJvYWNoLg0KDQo0LjQuICBJbmZvcm1hdGlv biB0cmFuc3BvcnQNCg0KICAgVGhlIHRyYW5zcG9ydCBvZiBwcmUtY29uZ2VzdGlvbiBpbmZvcm1h dGlvbiBmcm9tIGEgUENOLW5vZGUgdG8gYSBQQ04tDQogICBlZ3Jlc3Mtbm9kZSBpcyB0aHJvdWdo IFBDTi1tYXJraW5ncyBpbiBkYXRhIHBhY2tldCBoZWFkZXJzLCBubw0KICAgc2lnbmFsbGluZyBw cm90b2NvbCBtZXNzYWdpbmcgaXMgbmVlZGVkLiAgSG93ZXZlciwgc2lnbmFsbGluZyBpcw0KICAg bmVlZGVkIHRvIHRyYW5zcG9ydCBQQ04tZmVlZGJhY2staW5mb3JtYXRpb24gYmV0d2VlbiB0aGUg UENOLQ0KICAgYm91bmRhcnktbm9kZXMsIGZvciBleGFtcGxlIHRvIGNvbnZleSB0aGUgZnJhY3Rp b24gb2YgUENOLW1hcmtlZA0KICAgdHJhZmZpYyBmcm9tIGEgUENOLWVncmVzcy1ub2RlIHRvIHRo ZSByZWxldmFudCBQQ04taW5ncmVzcy1ub2RlLg0KICAgRXhhY3RseSB3aGF0IGluZm9ybWF0aW9u IG5lZWRzIHRvIGJlIHRyYW5zcG9ydGVkIHdpbGwgYmUgZGVzY3JpYmVkIGluDQogICB0aGUgZnV0 dXJlIFBDTiBXRyBkb2N1bWVudChzKSBhYm91dCB0aGUgYm91bmRhcnkgbWVjaGFuaXNtcy4gIFRo ZQ0KICAgc2lnbmFsbGluZyBjb3VsZCBiZSBkb25lIGJ5IGFuIGV4dGVuc2lvbiBvZiBSU1ZQIG9y IE5TSVMsIGZvcg0KICAgaW5zdGFuY2U7IHByb3RvY29sIHdvcmsgd2lsbCBiZSBkb25lIGJ5IHRo ZSByZWxldmFudCBXRywgYnV0IGZvcg0KICAgZXhhbXBsZSBbSS1ELmxlZmF1Y2hldXItcnN2cC1l Y25dIGRlc2NyaWJlcyB0aGUgZXh0ZW5zaW9ucyBuZWVkZWQgZm9yDQogICBSU1ZQLg0KDQo0LjUu ICBQQ04tdHJhZmZpYw0KDQogICBUaGUgZm9sbG93aW5nIGFyZSBzb21lIGhpZ2gtbGV2ZWwgcG9p bnRzIGFib3V0IGhvdyBQQ04gd29ya3M6DQoNCiAgIG8gIFRoZXJlIG5lZWRzIHRvIGJlIGEgd2F5 IGZvciBhIFBDTi1ub2RlIHRvIGRpc3Rpbmd1aXNoIFBDTi10cmFmZmljDQogICAgICBmcm9tIG5v biBQQ04tdHJhZmZpYy4gIFRoZXkgbWF5IGJlIGRpc3Rpbmd1aXNoZWQgdXNpbmcgdGhlIERTQ1AN CiAgICAgIGZpZWxkIGFuZC9vciBFQ04gZmllbGQuDQoNCg0KDQpFYXJkbGV5IChFZGl0b3IpICAg ICAgICAgIEV4cGlyZXMgTWF5IDIyLCAyMDA4ICAgICAgICAgICAgICAgICBbUGFnZSAxNV0NCgwN CkludGVybmV0LURyYWZ0ICAgICAgICAgICAgICAgICAgRG9jdW1lbnQgICAgICAgICAgICAgICAg ICAgTm92ZW1iZXIgMjAwNw0KDQoNCiAgIG8gIFRoZSBQQ04gbWVjaGFuaXNtcyBtYXkgYmUgYXBw bGllZCB0byBtb3JlIHRoYW4gb25lIHRyYWZmaWMgY2xhc3MNCiAgICAgICh3aGljaCBhcmUgZGlz dGluZ3Vpc2hlZCBieSBEU0NQKS4NCg0KICAgbyAgVGhlcmUgbWF5IGJlIHRyYWZmaWMgdGhhdCBp cyBtb3JlIGltcG9ydGFudCB0aGFuIFBDTiwgcGVyaGFwcyBhDQogICAgICBwYXJ0aWN1bGFyIGFw cGxpY2F0aW9uIG9yIGFuIG9wZXJhdG9yJ3MgY29udHJvbCBtZXNzYWdlcy4gIEEgUENOLQ0KICAg ICAgbm9kZSBtYXkgZGVkaWNhdGUgY2FwYWNpdHkgdG8gc3VjaCB0cmFmZmljIG9yIHByaW9yaXR5 IHNjaGVkdWxlIGl0DQogICAgICBvdmVyIFBDTi4gIEluIHRoZSBsYXR0ZXIgY2FzZSBpdHMgdHJh ZmZpYyBuZWVkcyB0byBjb250cmlidXRlIHRvDQogICAgICB0aGUgUENOIG1ldGVycy4NCg0KICAg byAgVGhlcmUgd2lsbCBiZSB0cmFmZmljIGxlc3MgaW1wb3J0YW50IHRoYW4gUENOLiAgRm9yIGlu c3RhbmNlIGJlc3QNCiAgICAgIGVmZm9ydCBvciBhc3N1cmVkIGZvcndhcmRpbmcgdHJhZmZpYy4g IEl0IHdpbGwgYmUgc2NoZWR1bGVkIGF0DQogICAgICBsb3dlciBwcmlvcml0eSB0aGFuIFBDTiwg YW5kIHVzZSBhIHNlcGFyYXRlIHF1ZXVlIG9yIHF1ZXVlcy4NCiAgICAgIEhvd2V2ZXIsIGEgUENO LW5vZGUgc2hvdWxkIGRlZGljYXRlIHNvbWUgY2FwYWNpdHkgdG8gbG93ZXINCiAgICAgIHByaW9y aXR5IHRyYWZmaWMgc28gdGhhdCBpdCBpc24ndCBzdGFydmVkLg0KDQogICBvICBUaGVyZSBtYXkg YmUgb3RoZXIgdHJhZmZpYyB3aXRoIHRoZSBzYW1lIHByaW9yaXR5IGFzIFBDTi10cmFmZmljLg0K ICAgICAgRm9yIGluc3RhbmNlLCBFeHBlZGl0ZWQgRm9yd2FyZGluZyBzZXNzaW9ucyB0aGF0IGFy ZSBvcmlnaW5hdGVkDQogICAgICBlaXRoZXIgd2l0aG91dCBjYXBhY2l0eSBhZG1pc3Npb24gb3Ig d2l0aCB0cmFmZmljIGVuZ2luZWVyaW5nLiAgSW4NCiAgICAgIFtJLUQuaWV0Zi10c3Z3Zy1hZG1p dHRlZC1yZWFsdGltZS1kc2NwXSB0aGUgdHdvIHRyYWZmaWMgY2xhc3Nlcw0KICAgICAgYXJlIGNh bGxlZCBFRiBhbmQgRUYtQURNSVQuICBBIFBDTi1ub2RlIGNvdWxkIGVpdGhlciB1c2Ugc2VwYXJh dGUNCiAgICAgIHF1ZXVlcywgb3Igc2VwYXJhdGUgcG9saWNlcnMgYW5kIGEgY29tbW9uIHF1ZXVl OyB0aGUgZHJhZnQNCiAgICAgIHByb3ZpZGVzIHNvbWUgZ3VpZGFuY2Ugd2hlbiBlYWNoIGlzIGJl dHRlciwgYnV0IGZvciBpbnN0YW5jZSB0aGUNCiAgICAgIGxhdHRlciBpcyBwcmVmZXJyZWQgd2hl biB0aGUgdHdvIHRyYWZmaWMgY2xhc3NlcyBhcmUgY2FycnlpbmcgdGhlDQogICAgICBzYW1lIHR5 cGUgb2YgYXBwbGljYXRpb24gd2l0aCB0aGUgc2FtZSBqaXR0ZXIgcmVxdWlyZW1lbnRzLg0KDQoN CjUuICBEZXRhaWxlZCBGdW5jdGlvbmFsIGFyY2hpdGVjdHVyZQ0KDQogICBUaGlzIHNlY3Rpb24g aXMgaW50ZW5kZWQgdG8gcHJvdmlkZSBhIHN5c3RlbWF0aWMgc3VtbWFyeSBvZiB0aGUgbmV3DQog ICBmdW5jdGlvbmFsIGFyY2hpdGVjdHVyZSBpbiB0aGUgUENOLWRvbWFpbi4gIEZpcnN0IGl0IGRl c2NyaWJlcw0KICAgZnVuY3Rpb25zIG5lZWRlZCBhdCB0aGUgdGhyZWUgc3BlY2lmaWMgdHlwZXMg b2YgUENOLW5vZGU7IHRoZXNlIGFyZQ0KICAgZGF0YSBwbGFuZSBmdW5jdGlvbnMgYW5kIGFyZSBp biBhZGRpdGlvbiB0byB0aGVpciBub3JtYWwgcm91dGVyDQogICBmdW5jdGlvbnMuICBUaGVuIGl0 IGRlc2NyaWJlcyBmdXJ0aGVyIGZ1bmN0aW9uYWxpdHkgbmVlZGVkIGZvciBib3RoDQogICBmbG93 IGFkbWlzc2lvbiBjb250cm9sIGFuZCBmbG93IHRlcm1pbmF0aW9uOyB0aGVzZSBhcmUgc2lnbmFs bGluZyBhbmQNCiAgIGRlY2lzaW9uLW1ha2luZyBmdW5jdGlvbnMsIGFuZCB0aGVyZSBhcmUgdmFy aW91cyBwb3NzaWJpbGl0aWVzIGZvcg0KICAgd2hlcmUgdGhlIGZ1bmN0aW9ucyBhcmUgcGh5c2lj YWxseSBsb2NhdGVkLiAgVGhlIHNlY3Rpb24gaXMgc3BsaXQNCiAgIGludG86DQoNCiAgIDEuICBm dW5jdGlvbnMgbmVlZGVkIGF0IFBDTi1pbnRlcmlvci1ub2Rlcw0KDQogICAyLiAgZnVuY3Rpb25z IG5lZWRlZCBhdCBQQ04taW5ncmVzcy1ub2Rlcw0KDQogICAzLiAgZnVuY3Rpb25zIG5lZWRlZCBh dCBQQ04tZWdyZXNzLW5vZGVzDQoNCiAgIDQuICBvdGhlciBmdW5jdGlvbnMgbmVlZGVkIGZvciBm bG93IGFkbWlzc2lvbiBjb250cm9sDQoNCiAgIDUuICBvdGhlciBmdW5jdGlvbnMgbmVlZGVkIGZv ciBmbG93IHRlcm1pbmF0aW9uIGNvbnRyb2wNCg0KDQoNCg0KRWFyZGxleSAoRWRpdG9yKSAgICAg ICAgICBFeHBpcmVzIE1heSAyMiwgMjAwOCAgICAgICAgICAgICAgICAgW1BhZ2UgMTZdDQoMDQpJ bnRlcm5ldC1EcmFmdCAgICAgICAgICAgICAgICAgIERvY3VtZW50ICAgICAgICAgICAgICAgICAg IE5vdmVtYmVyIDIwMDcNCg0KDQogICBOb3RlOiBQcm9iaW5nIGlzIGNvdmVyZWQgaW4gU2VjdGlv biA3Lg0KDQogICBUaGUgc2VjdGlvbiB0aGVuIGRpc2N1c3NlcyBzb21lIG90aGVyIGRldGFpbGVk IHRvcGljczoNCg0KICAgMS4gIGFkZHJlc3NpbmcNCg0KICAgMi4gIHR1bm5lbGxpbmcNCg0KICAg My4gIGZhdWx0IGhhbmRsaW5nDQoNCjUuMS4gIFBDTi1pbnRlcmlvci1ub2RlIGZ1bmN0aW9ucw0K DQogICBFYWNoIGludGVyZmFjZSBvZiB0aGUgUENOLWRvbWFpbiBpcyB1cGdyYWRlZCB3aXRoIHRo ZSBmb2xsb3dpbmcNCiAgIGZ1bmN0aW9uYWxpdHk6DQoNCiAgIG8gIFBhY2tldCBjbGFzc2lmeSAt IGRlY2lkZSB3aGV0aGVyIGFuIGluY29taW5nIHBhY2tldCBpcyBhIFBDTi0NCiAgICAgIHBhY2tl dCBvciBub3QuICBBbm90aGVyIFBDTiBXRyBkb2N1bWVudCB3aWxsIHNwZWNpZnkgZW5jb2Rpbmcs DQogICAgICB1c2luZyB0aGUgRFNDUCBhbmQvb3IgRUNOIGZpZWxkcy4NCg0KICAgbyAgUENOLW1l dGVyIC0gbWVhc3VyZSB0aGUgJ2Ftb3VudCBvZiBQQ04tdHJhZmZpYycuICBUaGUgbWVhc3VyZW1l bnQNCiAgICAgIGlzIG1hZGUgYXMgYW4gYWdncmVnYXRlIG9mIGFsbCBQQ04tcGFja2V0cywgYW5k IG5vdCBwZXIgZmxvdy4NCg0KICAgbyAgUENOLW1hcmsgLSBhbGdvcml0aG1zIGRldGVybWluZSB3 aGV0aGVyIHRvIFBDTi1tYXJrIFBDTi1wYWNrZXRzDQogICAgICBhbmQgd2hhdCBwYWNrZXQgZW5j b2RpbmcgaXMgdXNlZCAoYXMgc3BlY2lmaWVkIGluIGFub3RoZXIgUENOIFdHDQogICAgICBkb2N1 bWVudCkuDQoNCiAgIFRoZSBzYW1lIGdlbmVyYWwgYXBwcm9hY2ggb2YgbWV0ZXJpbmcgYW5kIFBD Ti1tYXJraW5nIGlzIHBlcmZvcm1lZA0KICAgZm9yIGJvdGggZmxvdyBhZG1pc3Npb24gY29udHJv bCBhbmQgZmxvdyB0ZXJtaW5hdGlvbiwgaG93ZXZlciB0aGUNCiAgIGFsZ29yaXRobXMgYW5kIGVu Y29kaW5nIG1heSBiZSBkaWZmZXJlbnQuDQoNCiAgIFRoZXNlIGZ1bmN0aW9ucyBhcmUgbmVlZGVk IGZvciBlYWNoIGludGVyZmFjZSBvZiB0aGUgUENOLWRvbWFpbi4NCiAgIFRoZXkgYXJlIHRoZXJl Zm9yZSBuZWVkZWQgb24gYWxsIGludGVyZmFjZXMgb2YgUENOLWludGVyaW9yLW5vZGVzLA0KICAg YW5kIG9uIHRoZSBpbnRlcmZhY2VzIG9mIFBDTi1ib3VuZGFyeS1ub2RlcyB0aGF0IGFyZSBpbnRl cm5hbCB0byB0aGUNCiAgIFBDTi1kb21haW4uICBUaGVyZSBtYXkgYmUgbW9yZSB0aGFuIG9uZSBQ Q04tbWV0ZXIgYW5kIG1hcmtlcg0KICAgaW5zdGFsbGVkIGF0IGEgZ2l2ZW4gaW50ZXJmYWNlLCBl ZyBvbmUgZm9yIGFkbWlzc2lvbiBhbmQgb25lIGZvcg0KICAgdGVybWluYXRpb24uDQoNCjUuMi4g IFBDTi1pbmdyZXNzLW5vZGUgZnVuY3Rpb25zDQoNCiAgIEVhY2ggaW5ncmVzcyBpbnRlcmZhY2Ug b2YgdGhlIFBDTi1kb21haW4gaXMgdXBncmFkZWQgd2l0aCB0aGUNCiAgIGZvbGxvd2luZyBmdW5j dGlvbmFsaXR5Og0KDQogICBvICBQYWNrZXQgY2xhc3NpZnkgLSBkZWNpZGUgd2hldGhlciBhbiBp bmNvbWluZyBwYWNrZXQgaXMgcGFydCBvZiBhDQogICAgICBwcmV2aW91c2x5IGFkbWl0dGVkIG1p Y3JvZmxvdywgYnkgdXNpbmcgYSBmaWx0ZXIgc3BlYyAoZWcgRFNDUCwNCiAgICAgIHNvdXJjZSBh bmQgZGVzdGluYXRpb24gYWRkcmVzc2VzIGFuZCBwb3J0IG51bWJlcnMpDQoNCiAgIG8gIFBvbGlj ZSAtIHBvbGljZSwgYnkgZHJvcHBpbmcgb3IgcmUtbWFya2luZyB3aXRoIGEgbm9uLVBDTiBEU0NQ LA0KICAgICAgYW55IHBhY2tldHMgcmVjZWl2ZWQgd2l0aCBhIERTQ1AgZGVtYW5kaW5nIFBDTiB0 cmFuc3BvcnQgdGhhdCBkbw0KDQoNCg0KRWFyZGxleSAoRWRpdG9yKSAgICAgICAgICBFeHBpcmVz IE1heSAyMiwgMjAwOCAgICAgICAgICAgICAgICAgW1BhZ2UgMTddDQoMDQpJbnRlcm5ldC1EcmFm dCAgICAgICAgICAgICAgICAgIERvY3VtZW50ICAgICAgICAgICAgICAgICAgIE5vdmVtYmVyIDIw MDcNCg0KDQogICAgICBub3QgYmVsb25nIHRvIGFuIGFkbWl0dGVkIGZsb3cuICBTaW1pbGFybHks IHBvbGljZSBwYWNrZXRzIHRoYXQNCiAgICAgIGFyZSBwYXJ0IG9mIGEgcHJldmlvdXNseSBhZG1p dHRlZCBtaWNyb2Zsb3csIHRvIGNoZWNrIHRoYXQgdGhlDQogICAgICBtaWNyb2Zsb3cga2VlcHMg dG8gdGhlIGFncmVlZCByYXRlIG9yIGZsb3dzcGVjIChlZyBSRkMxNjMzDQogICAgICBbUkZDMTYz M10gYW5kIE5TSVMgZXF1aXZhbGVudCkuDQoNCiAgIG8gIFBDTi1jb2xvdXIgLSBzZXQgdGhlIERT Q1AgZmllbGQgb3IgRFNDUCBhbmQgRUNOIGZpZWxkcyB0byB0aGUNCiAgICAgIGFwcHJvcHJpYXRl IHZhbHVlKHMpIGZvciBhIFBDTi1wYWNrZXQuICBUaGUgZHJhZnQgYWJvdXQgUENOLQ0KICAgICAg ZW5jb2Rpbmcgd2lsbCBkaXNjdXNzIGZ1cnRoZXIuDQoNCiAgIG8gIFBDTi1tZXRlciAtIG1ha2Ug Im1lYXN1cmVtZW50cyBvZiBQQ04tdHJhZmZpYyIuICBTb21lIGFwcHJvYWNoZXMNCiAgICAgIHRv IGZsb3cgdGVybWluYXRpb24gcmVxdWlyZSB0aGUgUENOLWluZ3Jlc3Mtbm9kZSB0byBtZWFzdXJl IHRoZQ0KICAgICAgKGFnZ3JlZ2F0ZSkgcmF0ZSBvZiBQQ04tdHJhZmZpYyB0b3dhcmRzIGEgcGFy dGljdWxhciBQQ04tZWdyZXNzLQ0KICAgICAgbm9kZS4NCg0KICAgVGhlIGZpcnN0IHR3byBhcmUg cG9saWNpbmcgZnVuY3Rpb25zLCBuZWVkZWQgdG8gbWFrZSBzdXJlIHRoYXQgUENOLQ0KICAgcGFj a2V0cyBsZXQgaW50byB0aGUgUENOLWRvbWFpbiBiZWxvbmcgdG8gYSBmbG93IHRoYXQncyBiZWVu IGFkbWl0dGVkDQogICBhbmQgdG8gZW5zdXJlIHRoYXQgdGhlIGZsb3cgZG9lc24ndCBnbyBhdCBh IGZhc3RlciByYXRlIHRoYW4gYWdyZWVkLg0KICAgVGhlIGZpbHRlciBzcGVjIHdpbGwgZm9yIGV4 YW1wbGUgY29tZSBmcm9tIHRoZSBmbG93IHJlcXVlc3QgbWVzc2FnZQ0KICAgKG91dHNpZGUgc2Nv cGUgb2YgUENOIFdHLCBzZWUgW0ktRC5icmlzY29lLXRzdndnLWNsLWFyY2hpdGVjdHVyZV0gZm9y DQogICBhbiBleGFtcGxlIHVzaW5nIFJTVlApLiAgUENOLWNvbG91cmluZyBhbGxvd3MgdGhlIHJl c3Qgb2YgdGhlIFBDTi0NCiAgIGRvbWFpbiB0byByZWNvZ25pc2UgUENOLXBhY2tldHMuDQoNCjUu My4gIFBDTi1lZ3Jlc3Mtbm9kZSBmdW5jdGlvbnMNCg0KICAgRWFjaCBlZ3Jlc3MgaW50ZXJmYWNl IG9mIHRoZSBQQ04tZG9tYWluIGlzIHVwZ3JhZGVkIHdpdGggdGhlDQogICBmb2xsb3dpbmcgZnVu Y3Rpb25hbGl0eToNCg0KICAgbyAgUGFja2V0IGNsYXNzaWZ5IC0gZGV0ZXJtaW5lIHdoaWNoIFBD Ti1pbmdyZXNzLW5vZGUgYSBQQ04tcGFja2V0DQogICAgICBoYXMgY29tZSBmcm9tLg0KDQogICBv ICBQQ04tbWV0ZXIgLSBtYWtlIG1lYXN1cmVtZW50cyBvZiBQQ04tdHJhZmZpYy4gIFRoZSBtZWFz dXJlbWVudChzKQ0KICAgICAgaXMgbWFkZSBhcyBhbiBhZ2dyZWdhdGUgKGllIG5vdCBwZXIgZmxv dykgb2YgYWxsIFBDTi1wYWNrZXRzIGZyb20NCiAgICAgIGEgcGFydGljdWxhciBQQ04taW5ncmVz cy1ub2RlLg0KDQogICBvICBQQ04tY29sb3VyIC0gZm9yIFBDTi1wYWNrZXRzLCBzZXQgdGhlIERT Q1AgYW5kIEVDTiBmaWVsZHMgdG8gdGhlDQogICAgICBhcHByb3ByaWF0ZSB2YWx1ZXMgZm9yIHVz ZSBvdXRzaWRlIHRoZSBQQ04tZG9tYWluLg0KDQogICBBbm90aGVyIFBDTiBXRyBkb2N1bWVudCwg YWJvdXQgYm91bmRhcnkgbWVjaGFuaXNtcywgd2lsbCBkZXNjcmliZQ0KICAgd2hhdCB0aGUgIm1l YXN1cmVtZW50cyBvZiBQQ04tdHJhZmZpYyIgYXJlLiAgVGhpcyBkZXBlbmRzIG9uIHdoZXRoZXIN CiAgIHRoZSBtZWFzdXJlbWVudCBpcyB0YXJnZXRlZCBhdCBhZG1pc3Npb24gY29udHJvbCBvciBm bG93IHRlcm1pbmF0aW9uLg0KICAgSXQgYWxzbyBkZXBlbmRzIG9uIHdoYXQgZW5jb2RpbmcgYW5k IFBDTi1tYXJraW5nIGFsZ29yaXRobXMgYXJlDQogICBzcGVjaWZpZWQgYnkgdGhlIFBDTiBXRy4N Cg0KNS40LiAgQWRtaXNzaW9uIGNvbnRyb2wgZnVuY3Rpb25zDQoNCiAgIFNwZWNpZmljIGFkbWlz c2lvbiBjb250cm9sIGZ1bmN0aW9ucyBjYW4gYmUgcGVyZm9ybWVkIGF0IGEgUENOLQ0KICAgYm91 bmRhcnktbm9kZSAoUENOLWluZ3Jlc3Mtbm9kZSBvciBQQ04tZWdyZXNzLW5vZGUpIG9yIGF0IGEN CiAgIGNlbnRyYWxpc2VkIG5vZGUsIGJ1dCBub3QgYXQgbm9ybWFsIFBDTi1pbnRlcmlvci1ub2Rl cy4gIFRoZQ0KDQoNCg0KRWFyZGxleSAoRWRpdG9yKSAgICAgICAgICBFeHBpcmVzIE1heSAyMiwg MjAwOCAgICAgICAgICAgICAgICAgW1BhZ2UgMThdDQoMDQpJbnRlcm5ldC1EcmFmdCAgICAgICAg ICAgICAgICAgIERvY3VtZW50ICAgICAgICAgICAgICAgICAgIE5vdmVtYmVyIDIwMDcNCg0KDQog ICBmdW5jdGlvbnMgYXJlOg0KDQogICBvICBNYWtlIGRlY2lzaW9uIGFib3V0IGFkbWlzc2lvbiAt IGNvbXBhcmUgdGhlIHJlcXVpcmVkICJtZWFzdXJlbWVudHMNCiAgICAgIG9mIFBDTi10cmFmZmlj IiAob3V0cHV0IG9mIHRoZSBQQ04tZWdyZXNzLW5vZGUncyBQQ04tbWV0ZXINCiAgICAgIGZ1bmN0 aW9uKSB3aXRoIHNvbWUgcmVmZXJlbmNlIGxldmVsLCBhbmQgaGVuY2UgZGVjaWRlIHdoZXRoZXIg dG8NCiAgICAgIGFkbWl0IHRoZSBwb3RlbnRpYWwgbmV3IFBDTi1mbG93LiAgQXMgd2VsbCBhcyB0 aGUgUENODQogICAgICBtZWFzdXJlbWVudHMsIHRoZSBkZWNpc2lvbiB0YWtlcyBhY2NvdW50IG9m IHBvbGljeSBhbmQgYXBwbGljYXRpb24NCiAgICAgIGxheWVyIHJlcXVpcmVtZW50cy4NCg0KICAg byAgQ29tbXVuaWNhdGUgZGVjaXNpb24gYWJvdXQgYWRtaXNzaW9uIC0gc2lnbmFsIHRoZSBkZWNp c2lvbiB0byB0aGUNCiAgICAgIG5vZGUgbWFraW5nIHRoZSBhZG1pc3Npb24gY29udHJvbCByZXF1 ZXN0ICh3aGljaCBtYXkgYmUgb3V0c2lkZQ0KICAgICAgdGhlIFBDTi1kb21haW4pLCBhbmQgdG8g dGhlIHBvbGljZXIgKFBDTi1pbmdyZXNzLW5vZGUgZnVuY3Rpb24pDQoNCiAgIFRoZXJlIGFyZSB2 YXJpb3VzIHBvc3NpYmlsaXRpZXMgZm9yIGhvdyB0aGUgZnVuY3Rpb25hbGl0eSBjYW4gYmUNCiAg IGRpc3RyaWJ1dGVkICh3ZSBhc3N1bWUgdGhlIG9wZXJhdG9yIHdvdWxkIGNvbmZpZ3VyZSB3aGlj aCBpcyB1c2VkKToNCg0KICAgbyAgVGhlIGRlY2lzaW9uIGlzIG1hZGUgYXQgdGhlIFBDTi1lZ3Jl c3Mtbm9kZSBhbmQgc2lnbmFsbGVkIHRvIHRoZQ0KICAgICAgUENOLWluZ3Jlc3Mtbm9kZQ0KDQog ICBvICBUaGUgZGVjaXNpb24gaXMgbWFkZSBhdCB0aGUgUENOLWluZ3Jlc3Mtbm9kZSwgd2hpY2gg cmVxdWlyZXMgdGhhdA0KICAgICAgdGhlIFBDTi1lZ3Jlc3Mtbm9kZSBzaWduYWxzIHRvIHRoZSBQ Q04taW5ncmVzcy1ub2RlIHRoZSBmcmFjdGlvbg0KICAgICAgb2YgUENOLXRyYWZmaWMgdGhhdCBp cyBQQ04tbWFya2VkIChvciB3aGF0ZXZlciB0aGUgUENOIFdHIGFncmVlcw0KICAgICAgYXMgdGhl IHJlcXVpcmVkICJtZWFzdXJlbWVudHMgb2YgUENOLXRyYWZmaWMiKS4NCg0KICAgbyAgVGhlIGRl Y2lzaW9uIGlzIG1hZGUgYXQgYSBjZW50cmFsaXNlZCBub2RlLCB3aGljaCByZXF1aXJlcyB0aGF0 DQogICAgICB0aGUgUENOLWVncmVzcy1ub2RlIHNpZ25hbHMgaXRzIG1lYXN1cmVtZW50cyB0byB0 aGUgY2VudHJhbGlzZWQNCiAgICAgIG5vZGUsIGFuZCB0aGF0IHRoZSBjZW50cmFsaXNlZCBub2Rl IHNpZ25hbHMgdG8gdGhlIFBDTi1pbmdyZXNzLQ0KICAgICAgbm9kZSBhYm91dCB0aGUgZGVjaXNp b24gYWJvdXQgYWRtaXNzaW9uIGNvbnRyb2wuICBJdCB3b3VsZCBiZQ0KICAgICAgcG9zc2libGUg Zm9yIHRoZSBjZW50cmFsaXNlZCBub2RlIHRvIGJlIG9uZSBvZiB0aGUgUENOLWJvdW5kYXJ5LQ0K ICAgICAgbm9kZXMsIHdoZW4gY2xlYXJseSB0aGUgc2lnbmFsbGluZyB3b3VsZCBzb21ldGltZXMg YmUgcmVwbGFjZWQgYnkNCiAgICAgIGEgbWVzc2FnZSBpbnRlcm5hbCB0byB0aGUgbm9kZS4NCg0K NS41LiAgRmxvdyB0ZXJtaW5hdGlvbiBmdW5jdGlvbnMNCg0KICAgU3BlY2lmaWMgdGVybWluYXRp b24gY29udHJvbCBmdW5jdGlvbnMgY2FuIGJlIHBlcmZvcm1lZCBhdCBhIFBDTi0NCiAgIGJvdW5k YXJ5LW5vZGUgKFBDTi1pbmdyZXNzLW5vZGUgb3IgUENOLWVncmVzcy1ub2RlKSBvciBhdCBhDQog ICBjZW50cmFsaXNlZCBub2RlLCBidXQgbm90IGF0IG5vcm1hbCBQQ04taW50ZXJpb3Itbm9kZXMu ICBUaGVyZSBhcmUNCiAgIHZhcmlvdXMgcG9zc2liaWxpdGllcyBmb3IgaG93IHRoZSBmdW5jdGlv bmFsaXR5IGNhbiBiZSBkaXN0cmlidXRlZCwNCiAgIHNpbWlsYXIgdG8gdGhvc2UgZGlzY3Vzc2Vk IGFib3ZlIGluIHRoZSBBZG1pc3Npb24gY29udHJvbCBzZWN0aW9uOw0KICAgdGhlIGZsb3cgdGVy bWluYXRpb24gZGVjaXNpb24gY291bGQgYmUgbWFkZSBhdCB0aGUgUENOLWluZ3Jlc3Mtbm9kZSwN CiAgIHRoZSBQQ04tZWdyZXNzLW5vZGUgb3IgYXQgc29tZSBjZW50cmFsaXNlZCBub2RlLiAgVGhl IGZ1bmN0aW9ucyBhcmU6DQoNCiAgIG8gIFBDTi1tZXRlciBhdCBQQ04tZWdyZXNzLW5vZGUgLSBt YWtlICJtZWFzdXJlbWVudHMgb2YgUENOLXRyYWZmaWMiDQogICAgICBmcm9tIGEgcGFydGljdWxh ciBQQ04taW5ncmVzcy1ub2RlLg0KDQogICBvICAoaWYgcmVxdWlyZWQpIFBDTi1tZXRlciBhdCBQ Q04taW5ncmVzcy1ub2RlIC0gbWFrZSAibWVhc3VyZW1lbnRzDQogICAgICBvZiBQQ04tdHJhZmZp YyIgYmVpbmcgc2VudCB0b3dhcmRzIGEgcGFydGljdWxhciBQQ04tZWdyZXNzLW5vZGU7DQogICAg ICBhZ2FpbiwgdGhpcyBpcyBkb25lIGZvciB0aGUgaW5ncmVzcy1lZ3Jlc3MtYWdncmVnYXRlIGFu ZCBub3QgcGVyDQoNCg0KDQpFYXJkbGV5IChFZGl0b3IpICAgICAgICAgIEV4cGlyZXMgTWF5IDIy LCAyMDA4ICAgICAgICAgICAgICAgICBbUGFnZSAxOV0NCgwNCkludGVybmV0LURyYWZ0ICAgICAg ICAgICAgICAgICAgRG9jdW1lbnQgICAgICAgICAgICAgICAgICAgTm92ZW1iZXIgMjAwNw0KDQoN CiAgICAgIGZsb3cuDQoNCiAgIG8gIChpZiByZXF1aXJlZCkgQ29tbXVuaWNhdGUgIm1lYXN1cmVt ZW50cyBvZiBQQ04tdHJhZmZpYyIgdG8gdGhlDQogICAgICBub2RlIHRoYXQgbWFrZXMgdGhlIGZs b3cgdGVybWluYXRpb24gZGVjaXNpb24uICBGb3IgZXhhbXBsZSwgaWYNCiAgICAgIHRoZSBQQ04t aW5ncmVzcy1ub2RlIG1ha2VzIHRoZSBkZWNpc2lvbiB0aGVuIGNvbW11bmljYXRlIHRoZSBQQ04t DQogICAgICBlZ3Jlc3Mtbm9kZSdzIG1lYXN1cmVtZW50cyB0byBpdCAoYXMgaW4NCiAgICAgIFtJ LUQuYnJpc2NvZS10c3Z3Zy1jbC1hcmNoaXRlY3R1cmVdKS4NCg0KICAgbyAgTWFrZSBkZWNpc2lv biBhYm91dCBmbG93IHRlcm1pbmF0aW9uIC0gdXNlIHRoZSAibWVhc3VyZW1lbnRzIG9mDQogICAg ICBQQ04tdHJhZmZpYyIgdG8gZGVjaWRlIHdoaWNoIFBDTi1mbG93IG9yIFBDTi1mbG93cyB0byB0 ZXJtaW5hdGUuDQogICAgICBUaGUgZGVjaXNpb24gdGFrZXMgYWNjb3VudCBvZiBwb2xpY3kgYW5k IGFwcGxpY2F0aW9uIGxheWVyDQogICAgICByZXF1aXJlbWVudHMuDQoNCiAgIG8gIENvbW11bmlj YXRlIGRlY2lzaW9uIGFib3V0IGZsb3cgdGVybWluYXRpb24gLSBzaWduYWwgdGhlIGRlY2lzaW9u DQogICAgICB0byB0aGUgbm9kZSB0aGF0IGlzIGFibGUgdG8gdGVybWluYXRlIHRoZSBmbG93ICh3 aGljaCBtYXkgYmUNCiAgICAgIG91dHNpZGUgdGhlIFBDTi1kb21haW4pLCBhbmQgdG8gdGhlIHBv bGljZXIgKFBDTi1pbmdyZXNzLW5vZGUNCiAgICAgIGZ1bmN0aW9uKS4NCg0KNS42LiAgQWRkcmVz c2luZw0KDQogICBQQ04tbm9kZXMgbWF5IG5lZWQgdG8ga25vdyB0aGUgYWRkcmVzcyBvZiBvdGhl ciBQQ04tbm9kZXM6DQoNCiAgIG8gIE5vdGU6IGluIGFsbCBjYXNlcyBQQ04taW50ZXJpb3Itbm9k ZXMgZG9uJ3QgbmVlZCB0byBrbm93IHRoZQ0KICAgICAgYWRkcmVzcyBvZiBhbnkgb3RoZXIgUENO LW5vZGVzIChleGNlcHQgYXMgbm9ybWFsIHRoZWlyIG5leHQgaG9wDQogICAgICBuZWlnaGJvdXJz LCBmb3Igcm91dGluZyBwdXJwb3NlcykNCg0KICAgbyAgaW4gdGhlIGNhc2VzIG9mIGFkbWlzc2lv biBvciB0ZXJtaW5hdGlvbiBkZWNpc2lvbiBieSBhIFBDTi0NCiAgICAgIGJvdW5kYXJ5LW5vZGUs IHRoZSBQQ04tZWdyZXNzLW5vZGUgbmVlZHMgdG8ga25vdyB0aGUgYWRkcmVzcyBvZg0KICAgICAg dGhlIFBDTi1pbmdyZXNzLW5vZGUgYXNzb2NpYXRlZCB3aXRoIGEgZmxvdywgYXQgYSBtaW5pbXVt IHNvIHRoYXQNCiAgICAgIHRoZSBQQ04taW5ncmVzcy1ub2RlIGNhbiBiZSBpbmZvcm1lZCB0byBl bmZvcmNlIHRoZSBhZG1pc3Npb24NCiAgICAgIGRlY2lzaW9uIChhbmQgYW55IGZsb3cgdGVybWlu YXRpb24gZGVjaXNpb24pIHRocm91Z2ggcG9saWNpbmcuDQogICAgICBUaGUgYWRkcmVzc2luZyBp bmZvcm1hdGlvbiBjYW4gYmUgZ2F0aGVyZWQgZnJvbSBzaWduYWxsaW5nLCBmb3INCiAgICAgIGV4 YW1wbGUgYXMgZGVzY3JpYmVkIGZvciBSU1ZQIGluIFtJLUQubGVmYXVjaGV1ci1yc3ZwLWVjbl0u DQogICAgICBBbm90aGVyIGFsdGVybmF0aXZlIGlzIHRvIHVzZSBhIHByb2JlIHBhY2tldCB0aGF0 IGluY2x1ZGVzIGFzDQogICAgICBwYXlsb2FkIHRoZSBhZGRyZXNzIG9mIHRoZSBQQ04taW5ncmVz cy1ub2RlLiAgQWx0ZXJuYXRpdmVseSwgaWYNCiAgICAgIFBDTi10cmFmZmljIGlzIGFsd2F5cyB0 dW5uZWxsZWQgYWNyb3NzIHRoZSBQQ04tZG9tYWluLCB0aGVuIHRoZQ0KICAgICAgUENOLWluZ3Jl c3Mtbm9kZSdzIGFkZHJlc3MgaXMgc2ltcGx5IHRoZSBzb3VyY2UgYWRkcmVzcyBvZiB0aGUNCiAg ICAgIG91dGVyIHBhY2tldCBoZWFkZXI7IHRoZW4gdGhlIFBDTi1pbmdyZXNzLW5vZGUgbmVlZHMg dG8gbGVhcm4gdGhlDQogICAgICBhZGRyZXNzIG9mIHRoZSBQQ04tZWdyZXNzLW5vZGUsIGVpdGhl ciBieSBtYW51YWwgY29uZmlndXJhdGlvbiBvcg0KICAgICAgYnkgb25lIG9mIHRoZSBhdXRvbWF0 ZWQgdHVubmVsIGVuZHBvaW50IGRpc2NvdmVyeSBtZWNoYW5pc21zIChzdWNoDQogICAgICBhcyBz aWduYWxsaW5nIG9yIHByb2Jpbmcgb3ZlciB0aGUgZGF0YSByb3V0ZSwgaW50ZXJyb2dhdGluZw0K ICAgICAgcm91dGluZyBvciB1c2luZyBhIGNlbnRyYWxpc2VkIGJyb2tlcikuDQoNCiAgIG8gIGlu IHRoZSBjYXNlcyBvZiBhZG1pc3Npb24gb3IgdGVybWluYXRpb24gZGVjaXNpb24gYnkgYSBjZW50 cmFsDQogICAgICBjb250cm9sIG5vZGUsIHRoZSBQQ04tZWdyZXNzLW5vZGUgbmVlZHMgdG8gYmUg Y29uZmlndXJlZCB3aXRoIHRoZQ0KICAgICAgYWRkcmVzcyBvZiB0aGUgY2VudHJhbGlzZWQgbm9k ZS4gIEluIGFkZGl0aW9uLCBkZXBlbmRpbmcgb24gdGhlDQogICAgICBleGFjdCBkZXBsb3ltZW50 IHNjZW5hcmlvIGFuZCBpdHMgc2lnbmFsbGluZywgdGhlIGNlbnRyYWxpc2VkIG5vZGUNCiAgICAg IG1heSBuZWVkIHRvIGtub3cgdGhlIGFkZHJlc3NlcyBvZiB0aGUgUENOLWluZ3Jlc3Mtbm9kZSBh bmQgUENOLQ0KDQoNCg0KRWFyZGxleSAoRWRpdG9yKSAgICAgICAgICBFeHBpcmVzIE1heSAyMiwg MjAwOCAgICAgICAgICAgICAgICAgW1BhZ2UgMjBdDQoMDQpJbnRlcm5ldC1EcmFmdCAgICAgICAg ICAgICAgICAgIERvY3VtZW50ICAgICAgICAgICAgICAgICAgIE5vdmVtYmVyIDIwMDcNCg0KDQog ICAgICBlZ3Jlc3Mtbm9kZSwgdGhlIFBDTi1lZ3Jlc3Mtbm9kZSBtYXkgbmVlZCB0byBrbm93IHRo ZSBhZGRyZXNzIG9mDQogICAgICB0aGUgUENOLWluZ3Jlc3Mtbm9kZSwgYW5kIHRoZSBQQ04taW5n cmVzcy1ub2RlIG1heSBuZWVkIHRvIGtub3cNCiAgICAgIHRoZSBhZGRyZXNzIG9mIHRoZSBjZW50 cmFsaXNlZCBub2RlIGFuZCB0aGUgUENOLWVncmVzcy1ub2RlLg0KICAgICAgTk9URTogQ29uc2lk ZXJhdGlvbiBvZiB0aGUgY2VudHJhbGlzZWQgY2FzZSBpcyBvdXQgb2Ygc2NvcGUgb2YgdGhlDQog ICAgICBpbml0aWFsIFBDTiBXRyBDaGFydGVyLg0KDQo1LjcuICBUdW5uZWxsaW5nDQoNCiAgIFR1 bm5lbHMgbWF5IG9yaWdpbmF0ZSBhbmQvb3IgdGVybWluYXRlIHdpdGhpbiBhIFBDTi1kb21haW4u ICBJdCBpcw0KICAgaW1wb3J0YW50IHRoYXQgdGhlIFBDTi1tYXJraW5nIG9mIGFueSBwYWNrZXQg Y2FuIHBvdGVudGlhbGx5DQogICBpbmZsdWVuY2UgUENOJ3MgZmxvdyBhZG1pc3Npb24gY29udHJv bCBhbmQgdGVybWluYXRpb24gLSBpdCBzaG91bGRuJ3QNCiAgIG1hdHRlciB3aGV0aGVyIHRoZSBw YWNrZXQgaGFwcGVucyB0byBiZSB0dW5uZWxsZWQgYXQgdGhlIFBDTi1ub2RlDQogICB0aGF0IFBD Ti1tYXJrcyB0aGUgcGFja2V0LCBvciBpbmRlZWQgd2hldGhlciBpdCdzIGRlY2Fwc3VsYXRlZCBv cg0KICAgZW5jYXBzdWxhdGVkIGJ5IGEgc3Vic2VxdWVudCBQQ04tbm9kZS4gIFRoaXMgc3VnZ2Vz dHMgdGhhdCB0aGUNCiAgICJ1bmlmb3JtIGNvbmNlcHR1YWwgbW9kZWwiIGRlc2NyaWJlZCBpbiBb UkZDMjk4M10gc2hvdWxkIGJlIHJlLQ0KICAgYXBwbGllZCBpbiB0aGUgUENOIGNvbnRleHQuICBJ biBsaW5lIHdpdGggdGhpcyBhbmQgdGhlIGFwcHJvYWNoIG9mDQogICBbUkZDNDMwM10gYW5kIFtJ LUQuYnJpc2NvZS10c3Z3Zy1lY24tdHVubmVsXSwgdGhlIGZvbGxvd2luZyBydWxlIGlzDQogICBh cHBsaWVkIGlmIGVuY2Fwc3VsYXRpb24gaXMgZG9uZSB3aXRoaW4gdGhlIFBDTi1kb21haW46DQoN CiAgIG8gIGFueSBQQ04tbWFya2luZyBpcyBjb3BpZWQgaW50byB0aGUgb3V0ZXIgaGVhZGVyDQoN CiAgIFNpbWlsYXJseSwgaW4gbGluZSB3aXRoIHRoZSAidW5pZm9ybSBjb25jZXB0dWFsIG1vZGVs IiBvZiBbUkZDMjk4M10NCiAgIGFuZCB0aGUgImZ1bGwtZnVuY3Rpb25hbGl0eSBvcHRpb24iIG9m IFtSRkMzMTY4XSwgdGhlIGZvbGxvd2luZyBydWxlDQogICBpcyBhcHBsaWVkIGlmIGRlY2Fwc3Vs YXRpb24gaXMgZG9uZSB3aXRoaW4gdGhlIFBDTi1kb21haW46DQoNCiAgIG8gIGlmIHRoZSBvdXRl ciBoZWFkZXIncyBtYXJraW5nIHN0YXRlIGlzIG1vcmUgc2V2ZXJlIHRoZW4gaXQgaXMNCiAgICAg IGNvcGllZCBvbnRvIHRoZSBpbm5lciBoZWFkZXINCg0KICAgbyAgTkIgdGhlIG9yZGVyIG9mIGlu Y3JlYXNpbmcgc2V2ZXJpdHkgaXM6IHVubWFya2VkOyBQQ04tbWFya2luZyB3aXRoDQogICAgICBm aXJzdCBlbmNvZGluZyAoaWUgYXNzb2NpYXRlZCB3aXRoIHRoZSBQQ04tbG93ZXItcmF0ZSk7IFBD Ti0NCiAgICAgIG1hcmtpbmcgd2l0aCBzZWNvbmQgZW5jb2RpbmcgKGllIGFzc29jaWF0ZWQgd2l0 aCB0aGUgUENOLXVwcGVyLQ0KICAgICAgcmF0ZSkNCg0KICAgQW4gb3BlcmF0b3IgbWF5IHdpc2gg dG8gdHVubmVsIFBDTi10cmFmZmljIGZyb20gUENOLWluZ3Jlc3Mtbm9kZXMgdG8NCiAgIFBDTi1l Z3Jlc3Mtbm9kZXMuICBUaGUgUENOLW1hcmtzIHNob3VsZG4ndCBiZSB2aXNpYmxlIG91dHNpZGUg dGhlDQogICBQQ04tZG9tYWluLCB3aGljaCBjYW4gYmUgYWNoaWV2ZWQgYnkgZG9pbmcgdGhlIFBD Ti1jb2xvdXIgZnVuY3Rpb24NCiAgIChTZWN0aW9uIDUuMykgYWZ0ZXIgYWxsIHRoZSBvdGhlciAo UENOIGFuZCB0dW5uZWxsaW5nKSBmdW5jdGlvbnMuDQogICBUaGUgcG90ZW50aWFsIHJlYXNvbnMg Zm9yIGRvaW5nIHN1Y2ggdHVubmVsbGluZyBhcmU6IHRoZSBQQ04tZWdyZXNzLQ0KICAgbm9kZSB0 aGVuIGF1dG9tYXRpY2FsbHkga25vd3MgdGhlIGFkZHJlc3Mgb2YgdGhlIHJlbGV2YW50IFBDTi0N CiAgIGluZ3Jlc3Mtbm9kZSBmb3IgYSBmbG93OyBldmVuIGlmIEVDTVAgaXMgcnVubmluZywgYWxs IFBDTi1wYWNrZXRzIG9uDQogICBhIHBhcnRpY3VsYXIgaW5ncmVzcy1lZ3Jlc3MtYWdncmVnYXRl IGZvbGxvdyB0aGUgc2FtZSBwYXRoLiAgQnV0IGl0DQogICBhbHNvIGhhcyBkcmF3YmFja3MsIGZv ciBleGFtcGxlIHRoZSBhZGRpdGlvbmFsIG92ZXJoZWFkIGluIHRlcm1zIG9mDQogICBiYW5kd2lk dGggYW5kIHByb2Nlc3NpbmcuDQoNCiAgIFBvdGVudGlhbCBpc3N1ZXMgYXJpc2UgZm9yIGEgInBh cnRpYWxseSBQQ04tY2FwYWJsZSB0dW5uZWwiLCBpZSB3aGVyZQ0KICAgb25seSBvbmUgdHVubmVs IGVuZHBvaW50IGlzIGluIHRoZSBQQ04gZG9tYWluOg0KDQoNCg0KDQoNCkVhcmRsZXkgKEVkaXRv cikgICAgICAgICAgRXhwaXJlcyBNYXkgMjIsIDIwMDggICAgICAgICAgICAgICAgIFtQYWdlIDIx XQ0KDA0KSW50ZXJuZXQtRHJhZnQgICAgICAgICAgICAgICAgICBEb2N1bWVudCAgICAgICAgICAg ICAgICAgICBOb3ZlbWJlciAyMDA3DQoNCg0KICAgMS4gIFRoZSB0dW5uZWwgc3RhcnRzIG91dHNp ZGUgYSBQQ04tZG9tYWluIGFuZCBmaW5pc2hlcyBpbnNpZGUgaXQuDQogICAgICAgSWYgdGhlIHBh Y2tldCBhcnJpdmVzIGF0IHRoZSB0dW5uZWwgaW5ncmVzcyB3aXRoIHRoZSBzYW1lDQogICAgICAg ZW5jb2RpbmcgYXMgdXNlZCB3aXRoaW4gdGhlIFBDTi1kb21haW4gdG8gaW5kaWNhdGUgUENOLW1h cmtpbmcsDQogICAgICAgdGhlbiB0aGlzIGNvdWxkIGxlYWQgdGhlIFBDTi1lZ3Jlc3Mtbm9kZSB0 byBmYWxzZWx5IG1lYXN1cmUgcHJlLQ0KICAgICAgIGNvbmdlc3Rpb24uDQoNCiAgIDIuICBUaGUg dHVubmVsIHN0YXJ0cyBpbnNpZGUgYSBQQ04tZG9tYWluIGFuZCBmaW5pc2hlcyBvdXRzaWRlIGl0 Lg0KICAgICAgIElmIHRoZSBwYWNrZXQgYXJyaXZlcyBhdCB0aGUgdHVubmVsIGluZ3Jlc3MgYWxy ZWFkeSBQQ04tbWFya2VkLA0KICAgICAgIHRoZW4gaXQgd2lsbCBzdGlsbCBoYXZlIHRoZSBzYW1l IGVuY29kaW5nIHdoZW4gaXQncyBkZWNhcHN1bGF0ZWQNCiAgICAgICB3aGljaCBjb3VsZCBwb3Rl bnRpYWxseSBjb25mdXNlIG5vZGVzIGJleW9uZCB0aGUgdHVubmVsIGVncmVzcy4NCg0KICAgSW4g bGluZSB3aXRoIHRoZSBzb2x1dGlvbiBmb3IgcGFydGlhbGx5IGNhcGFibGUgRGlmZlNlcnYgdHVu bmVscyBpbg0KICAgWzI5ODNdLCB0aGUgZm9sbG93aW5nIHJ1bGVzIGFyZSBhcHBsaWVkOg0KDQog ICBvICBGb3IgY2FzZSAoMSksIHRoZSB0dW5uZWwgZWdyZXNzIG5vZGUgY2xlYXJzIGFueSBQQ04t bWFya2luZyBvbiB0aGUNCiAgICAgIGlubmVyIGhlYWRlci4gIFRoaXMgcnVsZSBpcyBhcHBsaWVk IGJlZm9yZSB0aGUgJ2NvcHkgb24NCiAgICAgIGRlY2Fwc3VsYXRpb24nIHJ1bGUgYWJvdmUuDQoN CiAgIG8gIEZvciBjYXNlICgyKSwgdGhlIHR1bm5lbCBpbmdyZXNzIG5vZGUgY2xlYXJzIGFueSBQ Q04tbWFya2luZyBvbg0KICAgICAgdGhlIGlubmVyIGhlYWRlci4gIFRoaXMgcnVsZSBpcyBhcHBs aWVkIGFmdGVyIHRoZSAnY29weSBvbg0KICAgICAgZW5jYXBzdWxhdGlvbicgcnVsZSBhYm92ZS4N Cg0KICAgTm90ZSB0aGF0IHRoZSBhYm92ZSBpbXBsaWVzIHRoYXQgb25lIGhhcyB0byBrbm93LCBv ciBmaWd1cmUgb3V0LCB0aGUNCiAgIGNoYXJhY3RlcmlzdGljcyBvZiB0aGUgb3RoZXIgZW5kIG9m IHRoZSB0dW5uZWwgYXMgcGFydCBvZiBzZXR0aW5nIGl0DQogICB1cC4NCg0KNS44LiAgRmF1bHQg aGFuZGxpbmcNCg0KICAgSWYgYSBQQ04taW50ZXJpb3Itbm9kZSBmYWlscyAob3Igb25lIG9mIGl0 cyBsaW5rcyksIHRoZW4gbG93ZXIgbGF5ZXINCiAgIHByb3RlY3Rpb24gbWVjaGFuaXNtcyBvciB0 aGUgcmVndWxhciBJUCByb3V0aW5nIHByb3RvY29sIHdpbGwNCiAgIGV2ZW50dWFsbHkgcmUtcm91 dGUgcm91bmQgaXQuICBJZiB0aGUgbmV3IHJvdXRlIGNhbiBjYXJyeSBhbGwgdGhlDQogICBhZG1p dHRlZCB0cmFmZmljLCBmbG93cyB3aWxsIGdyYWNlZnVsbHkgY29udGludWUuICBJZiBpbnN0ZWFk IHRoaXMNCiAgIGNhdXNlcyBlYXJseSB3YXJuaW5nIG9mIHByZS1jb25nZXN0aW9uIG9uIHRoZSBu ZXcgcm91dGUsIHRoZW4NCiAgIGFkbWlzc2lvbiBjb250cm9sIGJhc2VkIG9uIHByZS1jb25nZXN0 aW9uIG5vdGlmaWNhdGlvbiB3aWxsIGVuc3VyZQ0KICAgbmV3IGZsb3dzIHdpbGwgbm90IGJlIGFk bWl0dGVkIHVudGlsIGVub3VnaCBleGlzdGluZyBmbG93cyBoYXZlDQogICBkZXBhcnRlZC4gIFJl LXJvdXRpbmcgbWF5IHJlc3VsdCBpbiBoZWF2eSAocHJlLSljb25nZXN0aW9uLCB3aGVuIHRoZQ0K ICAgZmxvdyB0ZXJtaW5hdGlvbiBtZWNoYW5pc20gd2lsbCBraWNrIGluLg0KDQogICBJZiBhIFBD Ti1ib3VuZGFyeS1ub2RlIGZhaWxzIHRoZW4gd2Ugd291bGQgbGlrZSB0aGUgcmVndWxhciBRb1MN CiAgIHNpZ25hbGxpbmcgcHJvdG9jb2wgdG8gdGFrZSBjYXJlIG9mIHRoaW5ncy4gIEFzIGFuIGV4 YW1wbGUNCiAgIFtJLUQuYnJpc2NvZS10c3Z3Zy1jbC1hcmNoaXRlY3R1cmVdIGNvbnNpZGVycyB3 aGF0IGhhcHBlbnMgaWYgUlNWUCBpcw0KICAgdGhlIFFvUyBzaWduYWxsaW5nIHByb3RvY29sLiAg VGhlIGRldGFpbHMgZm9yIGEgc3BlY2lmaWMgc2lnbmFsbGluZw0KICAgcHJvdG9jb2wgYXJlIG91 dCBvZiBzY29wZSBvZiB0aGUgUENOIFdHLCBob3dldmVyIHRoZXJlIGlzIGEgV0cNCiAgIE1pbGVz dG9uZSBvbiBnZW5lcmljICJSZXF1aXJlbWVudHMgZm9yIHNpZ25hbGxpbmciLg0KDQoNCg0KDQoN Cg0KDQpFYXJkbGV5IChFZGl0b3IpICAgICAgICAgIEV4cGlyZXMgTWF5IDIyLCAyMDA4ICAgICAg ICAgICAgICAgICBbUGFnZSAyMl0NCgwNCkludGVybmV0LURyYWZ0ICAgICAgICAgICAgICAgICAg RG9jdW1lbnQgICAgICAgICAgICAgICAgICAgTm92ZW1iZXIgMjAwNw0KDQoNCjYuICBEZXNpZ24g Z29hbHMgYW5kIGNoYWxsZW5nZXMNCg0KICAgUHJpb3Igd29yayBvbiBQQ04gYW5kIHNpbWlsYXIg bWVjaGFuaXNtcyBoYXMgdGhyb3duIHVwIGEgbnVtYmVyIG9mDQogICBjb25zaWRlcmF0aW9ucyBh Ym91dCBQQ04ncyBkZXNpZ24gZ29hbHMgKHRoaW5ncyBQQ04gc2hvdWxkIGJlIGdvb2QNCiAgIGF0 KSBhbmQgc29tZSBpc3N1ZXMgdGhhdCBoYXZlIGJlZW4gaGFyZCB0byBzb2x2ZSBpbiBhIGZ1bGx5 DQogICBzYXRpc2ZhY3RvcnkgbWFubmVyLiAgVGFrZW4gYXMgYSB3aG9sZSBpdCByZXByZXNlbnRz IGEgbGlzdCBvZiB0cmFkZS0NCiAgIG9mZnMgKGl0J3MgdW5saWtlbHkgdGhhdCB0aGV5IGNhbiBh bGwgYmUgMTAwJSBhY2hpZXZlZCkgYW5kIHBlcmhhcHMNCiAgIGFzIGV2YWx1YXRpb24gY3JpdGVy aWEgdG8gaGVscCBhbiBvcGVyYXRvciAob3IgdGhlIElFVEYpIGRlY2lkZQ0KICAgYmV0d2VlbiBv cHRpb25zLg0KDQogICBUaGUgZm9sbG93aW5nIGFyZSBrZXkgZGVzaWduIGdvYWxzIGZvciBQQ04g KGJhc2VkIG9uDQogICBbSS1ELmNoYW4tcGNuLXByb2JsZW0tc3RhdGVtZW50XSk6DQoNCiAgIG8g IFRoZSBQQ04tZW5hYmxlZCBwYWNrZXQgZm9yd2FyZGluZyBuZXR3b3JrIHNob3VsZCBiZSBzaW1w bGUsDQogICAgICBzY2FsYWJsZSBhbmQgcm9idXN0DQoNCiAgIG8gIENvbXBhdGliaWxpdHkgd2l0 aCBvdGhlciB0cmFmZmljIChpZSBhIHByb3Bvc2VkIHNvbHV0aW9uIHNob3VsZA0KICAgICAgd29y ayB3ZWxsIHdoZW4gbm9uLVBDTiB0cmFmZmljIGlzIGFsc28gcHJlc2VudCBpbiB0aGUgbmV0d29y aykNCg0KICAgbyAgU3VwcG9ydCBvZiBkaWZmZXJlbnQgdHlwZXMgb2YgcmVhbC10aW1lIHRyYWZm aWMgKGVnIHNob3VsZCB3b3JrDQogICAgICB3ZWxsIHdpdGggQ0JSIGFuZCBWQlIgdm9pY2UgYW5k IHZpZGVvIHNvdXJjZXMgdHJlYXRlZCB0b2dldGhlcikNCg0KICAgbyAgUmVhY3Rpb24gdGltZSBv ZiB0aGUgbWVjaGFuaXNtcyBzaG91bGQgYmUgY29tbWVuc3VyYXRlIHdpdGggdGhlDQogICAgICBk ZXNpcmVkIGFwcGxpY2F0aW9uLWxldmVsIHJlcXVpcmVtZW50cyAoZWcgYSB0ZXJtaW5hdGlvbiBt ZWNoYW5pc20NCiAgICAgIG5lZWRzIHRvIHRlcm1pbmF0ZSBmbG93cyBiZWZvcmUgc2lnbmlmaWNh bnQgUW9TIGlzc3VlcyBhcmUNCiAgICAgIGV4cGVyaWVuY2VkIGJ5IHJlYWwtdGltZSB0cmFmZmlj LCBhbmQgYmVmb3JlIG1vc3QgdXNlcnMgaGFuZyB1cCkuDQoNCiAgIG8gIENvbXBhdGliaWxpdHkg d2l0aCBkaWZmZXJlbnQgcHJlY2VkZW5jZSBsZXZlbHMgb2YgcmVhbC10aW1lDQogICAgICBhcHBs aWNhdGlvbnMgKGUuZy4gcHJlZmVyZW50aWFsIHRyZWF0bWVudCBvZiBoaWdoZXIgcHJlY2VkZW5j ZQ0KICAgICAgY2FsbHMgb3ZlciBsb3dlciBwcmVjZWRlbmNlIGNhbGxzLCBbSVRVLU1MUFBdLg0K DQogICBUaGUgZm9sbG93aW5nIGFyZSBvcGVuIGlzc3Vlcy4gIFRoZXkgYXJlIG1haW5seSB0YWtl biBmcm9tDQogICBbSS1ELmJyaXNjb2UtdHN2d2ctY2wtYXJjaGl0ZWN0dXJlXSB3aGljaCBhbHNv IGRlc2NyaWJlcyBzb21lDQogICBwb3NzaWJsZSBzb2x1dGlvbnMuICBOb3RlIHRoYXQgc29tZSBt YXkgYmUgY29uc2lkZXJlZCB1bmltcG9ydGFudCBpbg0KICAgZ2VuZXJhbCBvciBpbiBzcGVjaWZp YyBkZXBsb3ltZW50IHNjZW5hcmlvcyBvciBieSBzb21lIG9wZXJhdG9ycy4NCg0KICAgTk9URTog UG90ZW50aWFsIHNvbHV0aW9ucyBhcmUgb3V0IG9mIHNjb3BlIGZvciB0aGlzIGRvY3VtZW50Lg0K DQogICBvICBFQ01QIChFcXVhbCBDb3N0IE11bHRpLVBhdGgpIFJvdXRpbmc6IFRoZSBsZXZlbCBv ZiBwcmUtY29uZ2VzdGlvbg0KICAgICAgaXMgbWVhc3VyZWQgb24gYSBzcGVjaWZpYyBpbmdyZXNz LWVncmVzcy1hZ2dyZWdhdGUuICBIb3dldmVyLCBpZg0KICAgICAgdGhlIFBDTi1kb21haW4gcnVu cyBFQ01QLCB0aGVuIHRyYWZmaWMgb24gdGhpcyBpbmdyZXNzLWVncmVzcy0NCiAgICAgIGFnZ3Jl Z2F0ZSBtYXkgZm9sbG93IHNldmVyYWwgZGlmZmVyZW50IHBhdGhzIC0gc29tZSBvZiB0aGUgcGF0 aHMNCiAgICAgIGNvdWxkIGJlIHByZS1jb25nZXN0ZWQgd2hpbHN0IG90aGVycyBhcmUgbm90LiAg VGhlcmUgYXJlIHRocmVlDQogICAgICBwb3RlbnRpYWwgcHJvYmxlbXM6DQoNCiAgICAgIDEuICBv dmVyLWFkbWlzc2lvbjogYSBuZXcgZmxvdyBpcyBhZG1pdHRlZCAoYmVjYXVzZSB0aGUgcHJlLQ0K ICAgICAgICAgIGNvbmdlc3Rpb24gbGV2ZWwgbWVhc3VyZWQgYnkgdGhlIFBDTi1lZ3Jlc3Mtbm9k ZSBpcw0KICAgICAgICAgIHN1ZmZpY2llbnRseSBkaWx1dGVkIGJ5IHVubWFya2VkIHBhY2tldHMg ZnJvbSBub24tY29uZ2VzdGVkDQoNCg0KDQpFYXJkbGV5IChFZGl0b3IpICAgICAgICAgIEV4cGly ZXMgTWF5IDIyLCAyMDA4ICAgICAgICAgICAgICAgICBbUGFnZSAyM10NCgwNCkludGVybmV0LURy YWZ0ICAgICAgICAgICAgICAgICAgRG9jdW1lbnQgICAgICAgICAgICAgICAgICAgTm92ZW1iZXIg MjAwNw0KDQoNCiAgICAgICAgICBwYXRocyB0aGF0IGEgbmV3IGZsb3cgaXMgYWRtaXR0ZWQpLCBi dXQgaXRzIHBhY2tldHMgdHJhdmVsDQogICAgICAgICAgdGhyb3VnaCBhIHByZS1jb25nZXN0ZWQg UENOLW5vZGUNCg0KICAgICAgMi4gIHVuZGVyLWFkbWlzc2lvbjogYSBuZXcgZmxvdyBpcyBibG9j a2VkIChiZWNhdXNlIHRoZSBwcmUtDQogICAgICAgICAgY29uZ2VzdGlvbiBsZXZlbCBtZWFzdXJl ZCBieSB0aGUgUENOLWVncmVzcy1ub2RlIGlzDQogICAgICAgICAgc3VmZmljaWVudGx5IGluY3Jl YXNlZCBieSBQQ04tbWFya2VkIHBhY2tldHMgZnJvbSBwcmUtDQogICAgICAgICAgY29uZ2VzdGVk IHBhdGhzIHRoYXQgYSBuZXcgZmxvdyBpcyBibG9ja2VkKSwgYnV0IGl0cyBwYWNrZXRzDQogICAg ICAgICAgdHJhdmVsIGFsb25nIGFuIHVuY29uZ2VzdGVkIHBhdGgNCg0KICAgICAgMy4gIGluZWZm ZWN0aXZlIHRlcm1pbmF0aW9uOiBmbG93cyBhcmUgdGVybWluYXRlZCwgaG93ZXZlciB0aGVpcg0K ICAgICAgICAgIHBhdGggZG9lc24ndCB0cmF2ZWwgdGhyb3VnaCB0aGUgKHByZS0pY29uZ2VzdGVk IHJvdXRlcihzKS4NCiAgICAgICAgICBTaW5jZSBmbG93IHRlcm1pbmF0aW9uIGlzIGEgJ2xhc3Qg cmVzb3J0JyB0aGF0IHByb3RlY3RzIHRoZQ0KICAgICAgICAgIG5ldHdvcmsgc2hvdWxkIG92ZXIt YWRtaXNzaW9uIG9jY3VyLCB0aGlzIHByb2JsZW0gaXMgcHJvYmFibHkNCiAgICAgICAgICBtb3Jl IGltcG9ydGFudCB0byBzb2x2ZSB0aGFuIHRoZSBvdGhlciB0d28uDQoNCiAgIG8gIEVDTVAgYW5k IHNpZ25hbGxpbmc6IEl0IGlzIHBvc3NpYmxlIHRoYXQsIGluIGEgUENOLWRvbWFpbiBydW5uaW5n DQogICAgICBFQ01QLCB0aGUgc2lnbmFsbGluZyBwYWNrZXRzIChlZyBSU1ZQLCBOU0lTKSBmb2xs b3cgYSBkaWZmZXJlbnQNCiAgICAgIHBhdGggdGhhbiB0aGUgZGF0YSBwYWNrZXRzIC0gaXQgZGVw ZW5kcyBvbiB3aGljaCBmaWVsZHMgdGhlIEVDTVANCiAgICAgIGFsZ29yaXRobSB1c2VzLiAgVGhp cyBjb3VsZCBtYXR0ZXIgaWYgdGhlIHNpZ25hbGxpbmcgcGFja2V0cyBhcmUNCiAgICAgIHVzZWQg YXMgcHJvYmVzLg0KDQogICBvICBUdW5uZWxsaW5nOiBUaGVyZSBhcmUgc2NlbmFyaW9zIHdoZXJl IHR1bm5lbGxpbmcgbWFrZXMgaXQgaGFyZCB0bw0KICAgICAgZGV0ZXJtaW5lIHRoZSBwYXRoIGlu IHRoZSBQQ04tZG9tYWluLiAgVGhlIHByb2JsZW0sIGl0cyBpbXBhY3QgYW5kDQogICAgICB0aGUg cG90ZW50aWFsIHNvbHV0aW9ucyBhcmUgc2ltaWxhciB0byB0aG9zZSBmb3IgRUNNUC4NCg0KICAg byAgU2NlbmFyaW9zIHdpdGggb25seSBvbmUgdHVubmVsIGVuZHBvaW50IGluIHRoZSBQQ04gZG9t YWluIG1heSBtYWtlDQogICAgICBpdCBoYXJkZXIgZm9yIHRoZSBQQ04tZWdyZXNzLW5vZGUgdG8g Z2F0aGVyIGZyb20gdGhlIHNpZ25hbGxpbmcNCiAgICAgIG1lc3NhZ2VzIChlZyBSU1ZQLCBOU0lT KSB0aGUgaWRlbnRpdHkgb2YgdGhlIFBDTi1pbmdyZXNzLW5vZGUuDQoNCiAgIG8gIEJpLURpcmVj dGlvbmFsIFNlc3Npb25zOiBNYW55IGFwcGxpY2F0aW9ucyBoYXZlIGJpLWRpcmVjdGlvbmFsDQog ICAgICBzZXNzaW9ucyAtIGhlbmNlIHRoZXJlIGFyZSB0d28gZmxvd3MgdGhhdCBzaG91bGQgYmUg YWRtaXR0ZWQgKG9yDQogICAgICB0ZXJtaW5hdGVkKSBhcyBhIHBhaXIgLSBmb3IgaW5zdGFuY2Ug YSBiaS1kaXJlY3Rpb25hbCB2b2ljZSBjYWxsDQogICAgICBvbmx5IG1ha2VzIHNlbnNlIGlmIGZs b3dzIGluIGJvdGggZGlyZWN0aW9ucyBhcmUgYWRtaXR0ZWQuDQogICAgICBIb3dldmVyLCBQQ04n cyBtZWNoYW5pc21zIGNvbmNlcm4gYWRtaXNzaW9uIGFuZCB0ZXJtaW5hdGlvbiBvZiBhDQogICAg ICBzaW5nbGUgZmxvdywgYW5kIGNvb3JkaW5hdGlvbiBvZiB0aGUgZGVjaXNpb24gZm9yIGJvdGgg Zmxvd3MgaXMgYQ0KICAgICAgbWF0dGVyIGZvciB0aGUgc2lnbmFsbGluZyBwcm90b2NvbCBhbmQg b3V0IG9mIHNjb3BlIG9mIFBDTi4gIE9uZQ0KICAgICAgcG9zc2libGUgZXhhbXBsZSB3b3VsZCB1 c2UgU0lQIHByZS1jb25kaXRpb25zOyB0aGVyZSBhcmUgb3RoZXJzLg0KDQogICBvICBHbG9iYWwg Q29vcmRpbmF0aW9uOiBQQ04gbWFrZXMgaXRzIGFkbWlzc2lvbiBkZWNpc2lvbiBiYXNlZCBvbg0K ICAgICAgUENOLW1hcmtpbmdzIG9uIGEgcGFydGljdWxhciBpbmdyZXNzLWVncmVzcy1hZ2dyZWdh dGUuICBEZWNpc2lvbnMNCiAgICAgIGFib3V0IGZsb3dzIHRocm91Z2ggYSBkaWZmZXJlbnQgaW5n cmVzcy1lZ3Jlc3MtYWdncmVnYXRlIGFyZSBtYWRlDQogICAgICBpbmRlcGVuZGVudGx5LiAgSG93 ZXZlciwgb25lIGNhbiBpbWFnaW5lIG5ldHdvcmsgdG9wb2xvZ2llcyBhbmQNCiAgICAgIHRyYWZm aWMgbWF0cmljZXMgd2hlcmUsIGZyb20gYSBnbG9iYWwgcGVyc3BlY3RpdmUsIGl0IHdvdWxkIGJl DQogICAgICBiZXR0ZXIgdG8gbWFrZSBhIGNvb3JkaW5hdGVkIGRlY2lzaW9uIGFjcm9zcyBhbGwg dGhlIGluZ3Jlc3MtDQogICAgICBlZ3Jlc3MtYWdncmVnYXRlcyBmb3IgdGhlIHdob2xlIFBDTi1k b21haW4uICBGb3IgZXhhbXBsZSwgdG8gYmxvY2sNCiAgICAgIChvciBldmVuIHRlcm1pbmF0ZSkg Zmxvd3Mgb24gb25lIGluZ3Jlc3MtZWdyZXNzLWFnZ3JlZ2F0ZSBzbyB0aGF0DQogICAgICBtb3Jl IGltcG9ydGFudCBmbG93cyB0aHJvdWdoIGEgZGlmZmVyZW50IGluZ3Jlc3MtZWdyZXNzLWFnZ3Jl Z2F0ZQ0KICAgICAgY291bGQgYmUgYWRtaXR0ZWQuICBUaGUgcHJvYmxlbSBtYXkgd2VsbCBiZSBz ZWNvbmQgb3JkZXIuDQoNCg0KDQpFYXJkbGV5IChFZGl0b3IpICAgICAgICAgIEV4cGlyZXMgTWF5 IDIyLCAyMDA4ICAgICAgICAgICAgICAgICBbUGFnZSAyNF0NCgwNCkludGVybmV0LURyYWZ0ICAg ICAgICAgICAgICAgICAgRG9jdW1lbnQgICAgICAgICAgICAgICAgICAgTm92ZW1iZXIgMjAwNw0K DQoNCiAgIG8gIEFnZ3JlZ2F0ZSBUcmFmZmljIENoYXJhY3RlcmlzdGljczogRXZlbiB3aGVuIHRo ZSBudW1iZXIgb2YgZmxvd3MNCiAgICAgIGlzIHN0YWJsZSwgdGhlIHRyYWZmaWMgbGV2ZWwgdGhy b3VnaCB0aGUgUENOLWRvbWFpbiB3aWxsIHZhcnkNCiAgICAgIGJlY2F1c2UgdGhlIHNvdXJjZXMg dmFyeSB0aGVpciB0cmFmZmljIHJhdGVzLiAgUENOIHdvcmtzIGJlc3Qgd2hlbg0KICAgICAgdGhl cmUncyBub3QgdG9vIG11Y2ggdmFyaWFiaWxpdHkgaW4gdGhlIHRvdGFsIHRyYWZmaWMgbGV2ZWwg YXQgYQ0KICAgICAgUENOLW5vZGUncyBpbnRlcmZhY2UgKGllIGluIHRoZSBhZ2dyZWdhdGUgdHJh ZmZpYyBmcm9tIGFsbA0KICAgICAgc291cmNlcykuICBUb28gbXVjaCB2YXJpYXRpb24gbWVhbnMg dGhhdCBhIG5vZGUgbWF5IChhdCBvbmUNCiAgICAgIG1vbWVudCkgbm90IGJlIGRvaW5nIGFueSBQ Q04tbWFya2luZyBhbmQgdGhlbiAoYXQgYW5vdGhlciBtb21lbnQpDQogICAgICBkcm9wIHBhY2tl dHMgYmVjYXVzZSBpdCdzIG92ZXJsb2FkZWQuICBUaGlzIG1ha2VzIGl0IGhhcmQgdG8gdHVuZQ0K ICAgICAgdGhlIGFkbWlzc2lvbiBjb250cm9sIHNjaGVtZSB0byBzdG9wIGFkbWl0dGluZyBuZXcg Zmxvd3MgYXQgdGhlDQogICAgICByaWdodCB0aW1lLiAgVGhlcmVmb3JlIHRoZSBwcm9ibGVtIGlz IG1vcmUgbGlrZWx5IHdpdGggZmV3ZXIsDQogICAgICBidXJzdGllciBmbG93cy4NCg0KICAgbyAg Rmxhc2ggY3Jvd2RzIGFuZCBTcGVlZCBvZiBSZWFjdGlvbjogUENOIGlzIGEgbWVhc3VyZW1lbnQt YmFzZWQNCiAgICAgIG1lY2hhbmlzbSBhbmQgc28gdGhlcmUgaXMgYW4gaW5oZXJlbnQgZGVsYXkg YmV0d2VlbiBwYWNrZXQgbWFya2luZw0KICAgICAgYnkgUENOLWludGVyaW9yLW5vZGVzIGFuZCBh bnkgYWRtaXNzaW9uIGNvbnRyb2wgcmVhY3Rpb24gYXQgUENOLQ0KICAgICAgYm91bmRhcnktbm9k ZXMuICBGb3IgZXhhbXBsZSwgcG90ZW50aWFsbHkgaWYgYSBiaWcgYnVyc3Qgb2YNCiAgICAgIGFk bWlzc2lvbiByZXF1ZXN0cyBvY2N1cnMgaW4gYSB2ZXJ5IHNob3J0IHNwYWNlIG9mIHRpbWUgKGVn DQogICAgICBwcm9tcHRlZCBieSBhIHRlbGV2b3RlKSwgdGhleSBjb3VsZCBhbGwgZ2V0IGFkbWl0 dGVkIGJlZm9yZSBlbm91Z2gNCiAgICAgIFBDTi1tYXJrcyBhcmUgc2VlbiB0byBibG9jayBuZXcg Zmxvd3MuICBJbiBvdGhlciB3b3JkcywgYW55DQogICAgICBhZGRpdGlvbmFsIGxvYWQgb2ZmZXJl ZCB3aXRoaW4gdGhlIHJlYWN0aW9uIHRpbWUgb2YgdGhlIG1lY2hhbmlzbQ0KICAgICAgbXVzdG4n dCBtb3ZlIHRoZSBQQ04tZG9tYWluIGRpcmVjdGx5IGZyb20gbm8gY29uZ2VzdGlvbiB0bw0KICAg ICAgb3ZlcmxvYWQuICBUaGlzICd2dWxuZXJhYmlsaXR5IHBlcmlvZCcgbWF5IGltcGFjdCBhdCB0 aGUNCiAgICAgIHNpZ25hbGxpbmcgbGV2ZWwsIGZvciBpbnN0YW5jZSBRb1MgcmVxdWVzdHMgc2hv dWxkIGJlIHJhdGUgbGltaXRlZA0KICAgICAgdG8gYm91bmQgdGhlIG51bWJlciBvZiByZXF1ZXN0 cyBhYmxlIHRvIGFycml2ZSB3aXRoaW4gdGhlDQogICAgICB2dWxuZXJhYmlsaXR5IHBlcmlvZC4N Cg0KICAgbyAgU2lsZW50IGF0IHN0YXJ0OiBhZnRlciBhIHN1Y2Nlc3NmdWwgYWRtaXNzaW9uIHJl cXVlc3QgdGhlIHNvdXJjZQ0KICAgICAgbWF5IHdhaXQgc29tZSB0aW1lIGJlZm9yZSBzZW5kaW5n IGRhdGEgKGVnIHdhaXRpbmcgZm9yIHRoZSBjYWxsZWQNCiAgICAgIHBhcnR5IHRvIGFuc3dlciku ICBUaGVuIHRoZSByaXNrIGlzIHRoYXQsIGluIHNvbWUgY2lyY3Vtc3RhbmNlcywNCiAgICAgIFBD TidzIG1lYXN1cmVtZW50cyB1bmRlcmVzdGltYXRlIHdoYXQgdGhlIHByZS1jb25nZXN0aW9uIGxl dmVsDQogICAgICB3aWxsIGJlIHdoZW4gdGhlIHNvdXJjZSBkb2VzIHN0YXJ0IHNlbmRpbmcgZGF0 YS4NCg0KICAgbyAgQ29tcGF0aWJpbGl0eSBvZiBQQ04tZW5jb2Rpbmcgd2l0aCBFQ04tZW5jb2Rp bmcuICBUaGlzIGlzc3VlIHdpbGwNCiAgICAgIGJlIGNvbnNpZGVyZWQgZnVydGhlciBpbiB0aGUg UENOIFdHIE1pbGVzdG9uZSAnU3VydmV5IG9mIGVuY29kaW5nDQogICAgICBjaG9pY2VzJy4NCg0K DQo3LiAgUHJvYmluZw0KDQo3LjEuICBJbnRyb2R1Y3Rpb24NCg0KICAgUHJvYmluZyBpcyBhbiBv cHRpb25hbCBtZWNoYW5pc20gdG8gYXNzaXN0IGFkbWlzc2lvbiBjb250cm9sLg0KDQogICBQQ04n cyBhZG1pc3Npb24gY29udHJvbCwgYXMgZGVzY3JpYmVkIHNvIGZhciwgaXMgZXNzZW50aWFsbHkg YQ0KICAgcmVhY3RpdmUgbWVjaGFuaXNtIHdoZXJlIHRoZSBQQ04tZWdyZXNzLW5vZGUgbW9uaXRv cnMgdGhlIHByZS0NCiAgIGNvbmdlc3Rpb24gbGV2ZWwgZm9yIHRyYWZmaWMgZnJvbSBlYWNoIFBD Ti1pbmdyZXNzLW5vZGU7IGlmIHRoZSBsZXZlbA0KICAgcmlzZXMgdGhlbiBpdCBibG9ja3MgbmV3 IGZsb3dzIG9uIHRoYXQgaW5ncmVzcy1lZ3Jlc3MtYWdncmVnYXRlLg0KICAgSG93ZXZlciwgaXQn cyBwb3NzaWJsZSB0aGF0IGFuIGluZ3Jlc3MtZWdyZXNzLWFnZ3JlZ2F0ZSBjYXJyaWVzIG5vDQoN Cg0KDQpFYXJkbGV5IChFZGl0b3IpICAgICAgICAgIEV4cGlyZXMgTWF5IDIyLCAyMDA4ICAgICAg ICAgICAgICAgICBbUGFnZSAyNV0NCgwNCkludGVybmV0LURyYWZ0ICAgICAgICAgICAgICAgICAg RG9jdW1lbnQgICAgICAgICAgICAgICAgICAgTm92ZW1iZXIgMjAwNw0KDQoNCiAgIHRyYWZmaWMs IGFuZCBzbyB0aGUgUENOLWVncmVzcy1ub2RlIGNhbid0IG1ha2UgYW4gYWRtaXNzaW9uIGRlY2lz aW9uDQogICB1c2luZyB0aGUgdXN1YWwgbWV0aG9kIGRlc2NyaWJlZCBlYXJsaWVyLg0KDQogICBP bmUgYXBwcm9hY2ggaXMgdG8gYmUgIm9wdGltaXN0aWMiIGFuZCBzaW1wbHkgYWRtaXQgdGhlIG5l dyBmbG93Lg0KICAgSG93ZXZlciBpdCdzIHBvc3NpYmxlIHRvIGVudmlzYWdlIGEgc2NlbmFyaW8g d2hlcmUgdGhlIHRyYWZmaWMgbGV2ZWxzDQogICBvbiBvdGhlciBpbmdyZXNzLWVncmVzcy1hZ2dy ZWdhdGVzIGFyZSBhbHJlYWR5IHNvIGhpZ2ggdGhhdCB0aGV5J3JlDQogICBibG9ja2luZyBuZXcg UENOLWZsb3dzLCBhbmQgYWRtaXR0aW5nIGEgbmV3IGZsb3cgb250byB0aGlzICdlbXB0eScNCiAg IGluZ3Jlc3MtZWdyZXNzLWFnZ3JlZ2F0ZSBhZGRzIGV4dHJhIHRyYWZmaWMgb250byB0aGUgbGlu ayB0aGF0J3MNCiAgIGFscmVhZHkgcHJlLWNvbmdlc3RlZCAtIHdoaWNoIG1heSAndGlwIHRoZSBi YWxhbmNlJyBzbyB0aGF0IFBDTidzDQogICBmbG93IHRlcm1pbmF0aW9uIG1lY2hhbmlzbSBpcyBh Y3RpdmF0ZWQgb3Igc29tZSBwYWNrZXRzIGFyZSBkcm9wcGVkLg0KICAgVGhpcyByaXNrIGNvdWxk IGJlIGxlc3NlbmVkIGJ5IGNvbmZpZ3VyaW5nIG9uIGVhY2ggbGluayBzdWZmaWNpZW50DQogICAn c2FmZXR5IG1hcmdpbicgYWJvdmUgdGhlIFBDTi1sb3dlci1yYXRlLg0KDQogICBBbiBhbHRlcm5h dGl2ZSBhcHByb2FjaCBpcyB0byBtYWtlIFBDTiBhIG1vcmUgcHJvYWN0aXZlIG1lY2hhbmlzbS4N CiAgIFRoZSBQQ04taW5ncmVzcy1ub2RlIGV4cGxpY2l0bHkgZGV0ZXJtaW5lcywgYmVmb3JlIGFk bWl0dGluZyB0aGUNCiAgIHByb3NwZWN0aXZlIG5ldyBmbG93LCB3aGV0aGVyIHRoZSBpbmdyZXNz LWVncmVzcy1hZ2dyZWdhdGUgY2FuDQogICBzdXBwb3J0IGl0LiAgVGhpcyBjYW4gYmUgc2VlbiBh cyBhICJwZXNzaW1pc3RpYyIgYXBwcm9hY2gsIGluDQogICBjb250cmFzdCB0byB0aGUgIm9wdGlt aXNtIiBvZiB0aGUgYXBwcm9hY2ggYWJvdmUuICBJdCBpbnZvbHZlcw0KICAgcHJvYmluZzogYSBQ Q04taW5ncmVzcy1ub2RlIGdlbmVyYXRlcyBhbmQgc2VuZHMgcHJvYmUgcGFja2V0cyBpbg0KICAg b3JkZXIgdG8gdGVzdCB0aGUgcHJlLWNvbmdlc3Rpb24gbGV2ZWwgdGhhdCB0aGUgZmxvdyB3b3Vs ZA0KICAgZXhwZXJpZW5jZS4NCg0KICAgT25lIHBvc3NpYmlsaXR5IGlzIHRoYXQgYSBwcm9iZSBw YWNrZXQgaXMganVzdCBhIGR1bW15IGRhdGEgcGFja2V0LA0KICAgZ2VuZXJhdGVkIGJ5IHRoZSBQ Q04taW5ncmVzcy1ub2RlIGFuZCBhZGRyZXNzZWQgdG8gdGhlIFBDTi1lZ3Jlc3MtDQogICBub2Rl LiAgQW5vdGhlciBwb3NzaWJpbGl0eSBpcyB0aGF0IGEgcHJvYmUgcGFja2V0IGlzIGEgc2lnbmFs bGluZw0KICAgcGFja2V0IHRoYXQgaXMgYW55d2F5IHRyYXZlbGxpbmcgZnJvbSB0aGUgUENOLWlu Z3Jlc3Mtbm9kZSB0byB0aGUNCiAgIFBDTi1lZ3Jlc3Mtbm9kZSAoZWcgYW4gUlNWUCBQQVRIIG1l c3NhZ2UgdHJhdmVsbGluZyBmcm9tIHNvdXJjZSB0bw0KICAgZGVzdGluYXRpb24pLg0KDQo3LjIu ICBQcm9iaW5nIGZ1bmN0aW9ucw0KDQogICBUaGUgcHJvYmluZyBmdW5jdGlvbnMgYXJlOg0KDQog ICBvICBNYWtlIGRlY2lzaW9uIHRoYXQgcHJvYmluZyBpcyBuZWVkZWQuICBBcyBkZXNjcmliZWQg YWJvdmUsIHRoaXMgaXMNCiAgICAgIHdoZW4gdGhlIGluZ3Jlc3MtZWdyZXNzLWFnZ3JlZ2F0ZSBv ciB0aGUgRUNNUCBwYXRoIGNhcnJpZXMgbm8gUENOLQ0KICAgICAgdHJhZmZpYy4gIEFuIGFsdGVy bmF0aXZlIGlzIGFsd2F5cyB0byBwcm9iZSwgaWUgcHJvYmUgYmVmb3JlDQogICAgICBhZG1pdHRp bmcgZXZlcnkgUENOLWZsb3cuDQoNCiAgIG8gIChpZiByZXF1aXJlZCkgQ29tbXVuaWNhdGUgdGhl IHJlcXVlc3QgdGhhdCBwcm9iaW5nIGlzIG5lZWRlZCAtIHRoZQ0KICAgICAgUENOLWVncmVzcy1u b2RlIHNpZ25hbHMgdG8gdGhlIFBDTi1pbmdyZXNzLW5vZGUgdGhhdCBwcm9iaW5nIGlzDQogICAg ICBuZWVkZWQNCg0KICAgbyAgKGlmIHJlcXVpcmVkKSBHZW5lcmF0ZSBwcm9iZSB0cmFmZmljIC0g dGhlIFBDTi1pbmdyZXNzLW5vZGUNCiAgICAgIGdlbmVyYXRlcyB0aGUgcHJvYmUgdHJhZmZpYy4g IFRoZSBhcHByb3ByaWF0ZSBudW1iZXIgKG9yIHJhdGUpIG9mDQogICAgICBwcm9iZSBwYWNrZXRz IHdpbGwgZGVwZW5kIG9uIHRoZSBQQ04tbWFya2luZyBhbGdvcml0aG07IGZvcg0KICAgICAgZXhh bXBsZSBhbiBleGNlc3MtcmF0ZS1tYXJraW5nIGFsZ29yaXRobSBnZW5lcmF0ZXMgZmV3ZXIgUENO LW1hcmtzDQogICAgICB0aGFuIGEgdGhyZXNob2xkLW1hcmtpbmcgYWxnb3JpdGhtLCBhbmQgc28g d2lsbCBuZWVkIG1vcmUgcHJvYmUNCiAgICAgIHBhY2tldHMuDQoNCg0KDQpFYXJkbGV5IChFZGl0 b3IpICAgICAgICAgIEV4cGlyZXMgTWF5IDIyLCAyMDA4ICAgICAgICAgICAgICAgICBbUGFnZSAy Nl0NCgwNCkludGVybmV0LURyYWZ0ICAgICAgICAgICAgICAgICAgRG9jdW1lbnQgICAgICAgICAg ICAgICAgICAgTm92ZW1iZXIgMjAwNw0KDQoNCiAgIG8gIEZvcndhcmQgcHJvYmUgcGFja2V0cyAt IGFzIGZhciBhcyBQQ04taW50ZXJpb3Itbm9kZXMgYXJlDQogICAgICBjb25jZXJuZWQsIHByb2Jl IHBhY2tldHMgbXVzdCBiZSBoYW5kbGVkIHRoZSBzYW1lIGFzIChvcmRpbmFyeQ0KICAgICAgZGF0 YSkgUENOLXBhY2tldHMsIGluIHRlcm1zIG9mIHJvdXRpbmcsIHNjaGVkdWxpbmcgYW5kIFBDTi0N CiAgICAgIG1hcmtpbmcuDQoNCiAgIG8gIENvbnN1bWUgcHJvYmUgcGFja2V0cyAtIHRoZSBQQ04t ZWdyZXNzLW5vZGUgY29uc3VtZXMgcHJvYmUgcGFja2V0cw0KICAgICAgdG8gZW5zdXJlIHRoYXQg dGhleSBkb24ndCB0cmF2ZWwgYmV5b25kIHRoZSBQQ04tZG9tYWluLg0KDQo3LjMuICBEaXNjdXNz aW9uIG9mIHJhdGlvbmFsZSBmb3IgcHJvYmluZywgaXRzIGRvd25zaWRlcyBhbmQgb3BlbiBpc3N1 ZXMNCg0KICAgSXQgaXMgYW4gdW5yZXNvbHZlZCBxdWVzdGlvbiB3aGV0aGVyIHByb2JpbmcgaXMg cmVhbGx5IG5lZWRlZCwgYnV0DQogICB0aHJlZSB2aWV3cG9pbnRzIGhhdmUgYmVlbiBwdXQgZm9y d2FyZCBhcyB0byB3aHkgaXQgaXMgdXNlZnVsLiAgVGhlDQogICBmaXJzdCBpcyBwZXJoYXBzIHRo ZSBtb3N0IG9idmlvdXM6IHRoZXJlIGlzIG5vIFBDTi10cmFmZmljIG9uIHRoZQ0KICAgaW5ncmVz cy1lZ3Jlc3MtYWdncmVnYXRlLiAgVGhlIHNlY29uZCBhc3N1bWVzIHRoYXQgbXVsdGlwYXRoIHJv dXRpbmcNCiAgIEVDTVAgaXMgcnVubmluZyBpbiB0aGUgUENOLWRvbWFpbi4gIFRoZSB0aGlyZCB2 aWV3cG9pbnQgaXMgdGhhdA0KICAgYWRtaXNzaW9uIGNvbnRyb2wgaXMgYWx3YXlzIGRvbmUgYnkg cHJvYmluZy4gIFdlIG5vdyBjb25zaWRlciBlYWNoIGluDQogICB0dXJuLg0KDQogICBUaGUgZmly c3Qgdmlld3BvaW50IGFzc3VtZXMgdGhlIGZvbGxvd2luZzoNCg0KICAgbyAgVGhlcmUgaXMgbm8g UENOLXRyYWZmaWMgb24gdGhlIGluZ3Jlc3MtZWdyZXNzLWFnZ3JlZ2F0ZSAoc28gYQ0KICAgICAg bm9ybWFsIGFkbWlzc2lvbiBkZWNpc2lvbiBjYW5ub3QgYmUgbWFkZSkuDQoNCiAgIG8gIFNpbXBs eSBhZG1pdHRpbmcgdGhlIG5ldyBmbG93IGhhcyBhIHNpZ25pZmljYW50IHJpc2sgb2YgbGVhZGlu ZyB0bw0KICAgICAgb3ZlcmxvYWQ6IHBhY2tldHMgZHJvcHBlZCBvciBmbG93cyB0ZXJtaW5hdGVk Lg0KDQogICBPbiB0aGUgZm9ybWVyIGJ1bGxldCwgW1BDTi1lbWFpbC10cmFmZmljLWVtcHR5LWFn Z3JlZ2F0ZXNdIHN1Z2dlc3RzDQogICB0aGF0LCBkdXJpbmcgdGhlIGZ1dHVyZSBidXN5IGhvdXIg b2YgYSBuYXRpb25hbCBuZXR3b3JrIHdpdGggYWJvdXQNCiAgIDEwMCBQQ04tYm91bmRhcnktbm9k ZXMsIHRoZXJlIGFyZSBsaWtlbHkgdG8gYmUgc2lnbmlmaWNhbnQgbnVtYmVycyBvZg0KICAgYWdn cmVnYXRlcyB3aXRoIHZlcnkgZmV3IGZsb3dzIHVuZGVyIG5lYXJseSBhbGwgY2lyY3Vtc3RhbmNl cy4NCg0KICAgVGhlIGxhdHRlciBidWxsZXQgY291bGQgb2NjdXIgaWYgYSBuZXcgZmxvdyBzdGFy dHMgb24gbWFueSBvZiB0aGUNCiAgIGVtcHR5IGluZ3Jlc3MtZWdyZXNzLWFnZ3JlZ2F0ZXMgYW5k IGNhdXNlcyBvdmVybG9hZCBvbiBhIGxpbmsgaW4gdGhlDQogICBQQ04tZG9tYWluLiAgVG8gYmUg YSBwcm9ibGVtIHRoaXMgd291bGQgcHJvYmFibHkgaGF2ZSB0byBoYXBwZW4gaW4gYQ0KICAgc2hv cnQgdGltZSBwZXJpb2QgKGZsYXNoIGNyb3dkKSBiZWNhdXNlLCBhZnRlciB0aGUgcmVhY3Rpb24g dGltZSBvZg0KICAgdGhlIHN5c3RlbSwgb3RoZXIgKG5vbi1lbXB0eSkgaW5ncmVzcy1lZ3Jlc3Mt YWdncmVnYXRlcyB0aGF0IHBhc3MNCiAgIHRocm91Z2ggdGhlIGxpbmsgd2lsbCBtZWFzdXJlIHBy ZS1jb25nZXN0aW9uIGFuZCBzbyBibG9jayBuZXcgZmxvd3MsDQogICBhbmQgYWxzbyBmbG93cyBu YXR1cmFsbHkgZW5kIGFueXdheS4NCg0KICAgVGhlIGRvd25zaWRlcyBvZiBwcm9iaW5nIGZvciB0 aGlzIHZpZXdwb2ludCBhcmU6DQoNCiAgIG8gIFByb2JpbmcgYWRkcyBkZWxheSB0byB0aGUgYWRt aXNzaW9uIGNvbnRyb2wgcHJvY2Vzcy4NCg0KICAgbyAgU3VmZmljaWVudCBwcm9iaW5nIHRyYWZm aWMgaGFzIHRvIGJlIGdlbmVyYXRlZCB0byB0ZXN0IHRoZSBwcmUtDQogICAgICBjb25nZXN0aW9u IGxldmVsIG9mIHRoZSBpbmdyZXNzLWVncmVzcy1hZ2dyZWdhdGUuICBCdXQgdGhlIHByb2JpbmcN CiAgICAgIHRyYWZmaWMgaXRzZWxmIG1heSBjYXVzZSBwcmUtY29uZ2VzdGlvbiwgY2F1c2luZyBv dGhlciBQQ04tZmxvd3MNCiAgICAgIHRvIGJlIGJsb2NrZWQgb3IgZXZlbiB0ZXJtaW5hdGVkIC0g YW5kIGluIHRoZSBmbGFzaCBjcm93ZCBzY2VuYXJpbw0KICAgICAgdGhlcmUgd2lsbCBiZSBwcm9i aW5nIG9uIG1hbnkgaW5ncmVzcy1lZ3Jlc3MtYWdncmVnYXRlcy4NCg0KDQoNCkVhcmRsZXkgKEVk aXRvcikgICAgICAgICAgRXhwaXJlcyBNYXkgMjIsIDIwMDggICAgICAgICAgICAgICAgIFtQYWdl IDI3XQ0KDA0KSW50ZXJuZXQtRHJhZnQgICAgICAgICAgICAgICAgICBEb2N1bWVudCAgICAgICAg ICAgICAgICAgICBOb3ZlbWJlciAyMDA3DQoNCg0KICAgVGhlIG9wZW4gaXNzdWVzIGFzc29jaWF0 ZWQgd2l0aCB0aGlzIHZpZXdwb2ludCBpbmNsdWRlOg0KDQogICBvICBXaGF0IHJhdGUgYW5kIHBh dHRlcm4gb2YgcHJvYmUgcGFja2V0cyBkb2VzIHRoZSBQQ04taW5ncmVzcy1ub2RlDQogICAgICBu ZWVkIHRvIGdlbmVyYXRlLCBzbyB0aGF0IHRoZXJlJ3MgZW5vdWdoIHRyYWZmaWMgdG8gbWFrZSB0 aGUNCiAgICAgIGFkbWlzc2lvbiBkZWNpc2lvbj8NCg0KICAgbyAgV2hhdCBkaWZmaWN1bHR5IGRv ZXMgdGhlIGRlbGF5ICh3aGlsc3QgcHJvYmluZyBpcyBkb25lKSBjYXVzZQ0KICAgICAgYXBwbGlj YXRpb25zLCBlZyBwYWNrZXRzIG1pZ2h0IGJlIGRyb3BwZWQ/DQoNCiAgIG8gIEFyZSB0aGVyZSBv dGhlciB3YXlzIG9mIGRlYWxpbmcgd2l0aCB0aGUgZmxhc2ggY3Jvd2Qgc2NlbmFyaW8/DQogICAg ICBGb3IgaW5zdGFuY2UgbGltaXQgdGhlIHJhdGUgYXQgd2hpY2ggbmV3IGZsb3dzIGFyZSBhZG1p dHRlZDsgb3INCiAgICAgIHBlcmhhcHMgZm9yIGEgUENOLWVncmVzcy1ub2RlIHRvIGJsb2NrIG5l dyBmbG93cyBvbiBpdHMgZW1wdHkNCiAgICAgIGluZ3Jlc3MtZWdyZXNzLWFnZ3JlZ2F0ZXMgd2hl biBpdHMgbm9uLWVtcHR5IG9uZXMgYXJlIHByZS0NCiAgICAgIGNvbmdlc3RlZC4NCg0KICAgVGhl IHNlY29uZCB2aWV3cG9pbnQgYXBwbGllcyBpbiB0aGUgY2FzZSB3aGVyZSB0aGVyZSBpcyBtdWx0 aXBhdGgNCiAgIHJvdXRpbmcgKEVDTVApIGluIHRoZSBQQ04tZG9tYWluLiAgTm90ZSB0aGF0IEVD TVAgaXMgb2Z0ZW4gdXNlZCBvbg0KICAgY29yZSBuZXR3b3Jrcy4gIFRoZXJlIGFyZSB0d28gcG9z c2liaWxpdGllczoNCg0KICAgKDEpIElmIGFkbWlzc2lvbiBjb250cm9sIGlzIGJhc2VkIG9uIG1l YXN1cmVtZW50cyBvZiB0aGUgaW5ncmVzcy0NCiAgIGVncmVzcy1hZ2dyZWdhdGUsIHRoZW4gdGhl IHZpZXdwb2ludCB0aGF0IHByb2JpbmcgaXMgdXNlZnVsIGFzc3VtZXM6DQoNCiAgIG8gIHRoZXJl J3MgYSBzaWduaWZpY2FudCBjaGFuY2UgdGhhdCB0aGUgdHJhZmZpYyBpcyB1bmV2ZW5seSBiYWxh bmNlZA0KICAgICAgYWNyb3NzIHRoZSBFQ01QIHBhdGhzLCBhbmQgaGVuY2UgdGhlcmUncyBhIHNp Z25pZmljYW50IHJpc2sgb2YNCiAgICAgIGFkbWl0dGluZyBhIGZsb3cgdGhhdCBzaG91bGQgYmUg YmxvY2tlZCAoYmVjYXVzZSBpdCBmb2xsb3dzIGFuDQogICAgICBFQ01QIHBhdGggdGhhdCBpcyBw cmUtY29uZ2VzdGVkKSBvciBibG9ja2luZyBhIGZsb3cgdGhhdCBzaG91bGQgYmUNCiAgICAgIGFk bWl0dGVkLg0KDQogICBvICBOb3RlOiBbUENOLWVtYWlsLUVDTVBdIHN1Z2dlc3RzIHVuYmFsYW5j ZWQgdHJhZmZpYyBpcyBxdWl0ZQ0KICAgICAgcG9zc2libGUsIGV2ZW4gd2l0aCBxdWl0ZSBhIGxh cmdlIG51bWJlciBvZiBmbG93cyBvbiBhIFBDTi1saW5rDQogICAgICAoZWcgMTAwMCkgd2hlbiBB c3N1bXB0aW9uIDMgKGFnZ3JlZ2F0aW9uKSBpcyBsaWtlbHkgdG8gYmUNCiAgICAgIHNhdGlzZmll ZC4NCg0KICAgKDIpIElmIGFkbWlzc2lvbiBjb250cm9sIGlzIGJhc2VkIG9uIG1lYXN1cmVtZW50 cyBvZiBwcmUtY29uZ2VzdGlvbg0KICAgb24gc3BlY2lmaWMgRUNNUCBwYXRocywgdGhlbiB0aGUg dmlld3BvaW50IHRoYXQgcHJvYmluZyBpcyB1c2VmdWwNCiAgIGFzc3VtZXM6DQoNCiAgIG8gIFRo ZXJlIGlzIG5vIFBDTi10cmFmZmljIG9uIHRoZSBFQ01QIHBhdGggb24gd2hpY2ggdG8gYmFzZSBh bg0KICAgICAgYWRtaXNzaW9uIGRlY2lzaW9uLg0KDQogICBvICBTaW1wbHkgYWRtaXR0aW5nIHRo ZSBuZXcgZmxvdyBoYXMgYSBzaWduaWZpY2FudCByaXNrIG9mIGxlYWRpbmcgdG8NCiAgICAgIG92 ZXJsb2FkLg0KDQogICBvICBUaGUgUENOLWVncmVzcy1ub2RlIGNhbiBtYXRjaCBhIHBhY2tldCB0 byBhbiBFQ01QIHBhdGguDQoNCiAgIG8gIE5vdGU6IFRoaXMgaXMgc2ltaWxhciB0byB0aGUgZmly c3Qgdmlld3BvaW50IGFuZCBzbyBzaW1pbGFybHkNCiAgICAgIGNvdWxkIG9jY3VyIGluIGEgZmxh c2ggY3Jvd2QgaWYgYSBuZXcgZmxvdyBzdGFydHMgbW9yZS1vci1sZXNzDQogICAgICBzaW11bHRh bmVvdXNseSBvbiBtYW55IG9mIHRoZSBlbXB0eSBFQ01QIHBhdGhzLiAgQmVjYXVzZSB0aGVyZSBh cmUNCg0KDQoNCkVhcmRsZXkgKEVkaXRvcikgICAgICAgICAgRXhwaXJlcyBNYXkgMjIsIDIwMDgg ICAgICAgICAgICAgICAgIFtQYWdlIDI4XQ0KDA0KSW50ZXJuZXQtRHJhZnQgICAgICAgICAgICAg ICAgICBEb2N1bWVudCAgICAgICAgICAgICAgICAgICBOb3ZlbWJlciAyMDA3DQoNCg0KICAgICAg c2V2ZXJhbCAoc29tZXRpbWVzIG1hbnkpIEVDTVAgcGF0aHMgYmV0d2VlbiBlYWNoIHBhaXIgb2Yg UENOLQ0KICAgICAgYm91bmRhcnktbm9kZXMsIGl0J3MgcHJlc3VtYWJseSBtb3JlIGxpa2VseSB0 aGF0IGFuIEVDTVAgcGF0aCBpcw0KICAgICAgJ2VtcHR5JyB0aGFuIGFuIGluZ3Jlc3MtZWdyZXNz LWFnZ3JlZ2F0ZS4gIFRvIGNvbnN0cmFpbiB0aGUgbnVtYmVyDQogICAgICBvZiBFQ01QIHBhdGhz LCBhIGZldyB0dW5uZWxzIGNvdWxkIGJlIHNldC11cCBiZXR3ZWVuIGVhY2ggcGFpciBvZg0KICAg ICAgUENOLWJvdW5kYXJ5LW5vZGVzLiAgVHVubmVsbGluZyBhbHNvIHNvbHZlcyB0aGUgdGhpcmQg YnVsbGV0DQogICAgICAod2hpY2ggaXMgb3RoZXJ3aXNlIGhhcmQgYmVjYXVzZSBhbiBFQ01QIHJv dXRpbmcgZGVjaXNpb24gaXMgbWFkZQ0KICAgICAgaW5kZXBlbmRlbnRseSBvbiBlYWNoIG5vZGUp Lg0KDQogICBUaGUgZG93bnNpZGVzIG9mIHByb2JpbmcgZm9yIHRoaXMgdmlld3BvaW50IGFyZToN Cg0KICAgbyAgUHJvYmluZyBhZGRzIGRlbGF5IHRvIHRoZSBhZG1pc3Npb24gY29udHJvbCBwcm9j ZXNzLg0KDQogICBvICBTdWZmaWNpZW50IHByb2JpbmcgdHJhZmZpYyBoYXMgdG8gYmUgZ2VuZXJh dGVkIHRvIHRlc3QgdGhlIHByZS0NCiAgICAgIGNvbmdlc3Rpb24gbGV2ZWwgb2YgdGhlIEVDTVAg cGF0aC4gIEJ1dCB0aGVyZSdzIHRoZSByaXNrIHRoYXQgdGhlDQogICAgICBwcm9iaW5nIHRyYWZm aWMgaXRzZWxmIG1heSBjYXVzZSBwcmUtY29uZ2VzdGlvbiwgY2F1c2luZyBvdGhlcg0KICAgICAg UENOLWZsb3dzIHRvIGJlIGJsb2NrZWQgb3IgZXZlbiB0ZXJtaW5hdGVkLg0KDQogICBvICBUaGUg UENOLWVncmVzcy1ub2RlIG5lZWRzIHRvIGNvbnN1bWUgdGhlIHByb2JlIHBhY2tldHMgdG8gZW5z dXJlDQogICAgICB0aGV5IGRvbid0IHRyYXZlbCBiZXlvbmQgdGhlIFBDTi1kb21haW4gKGVnIHRo ZXkgbWlnaHQgY29uZnVzZSB0aGUNCiAgICAgIGRlc3RpbmF0aW9uIGVuZCBub2RlKS4gIEhlbmNl IHNvbWVob3cgdGhlIFBDTi1lZ3Jlc3Mtbm9kZSBoYXMgdG8NCiAgICAgIGJlIGFibGUgdG8gZGlz YW1iaWd1YXRlIGEgcHJvYmUgcGFja2V0IGZyb20gYSBkYXRhIHBhY2tldCwgdmlhIHRoZQ0KICAg ICAgY2hhcmFjdGVyaXN0aWMgc2V0dGluZyBvZiBwYXJ0aWN1bGFyIGJpdChzKSBpbiB0aGUgcGFj a2V0J3MgaGVhZGVyDQogICAgICBvciBib2R5IC0gYnV0IHRoZXNlIGJpdChzKSBtdXN0bid0IGJl IHVzZWQgYnkgYW55IFBDTi1pbnRlcmlvci0NCiAgICAgIG5vZGUncyBFQ01QIGFsZ29yaXRobS4g IEluIHRoZSBnZW5lcmFsIGNhc2UgdGhpcyBpc24ndCBwb3NzaWJsZSwNCiAgICAgIGJ1dCBpdCBz aG91bGQgYmUgT0sgZm9yIGEgdHlwaWNhbCBFQ01QIGFsZ29yaXRobSB3aGljaCBleGFtaW5lczoN CiAgICAgIHRoZSBzb3VyY2UgYW5kIGRlc3RpbmF0aW9uIElQIGFkZHJlc3NlcyBhbmQgcG9ydCBu dW1iZXJzLCB0aGUNCiAgICAgIHByb3RvY29sIElEIGFuZCB0aGUgRFNDUC4NCg0KICAgVGhlIHRo aXJkIHZpZXdwb2ludCBhc3N1bWVzIHRoZSBmb2xsb3dpbmc6DQoNCiAgIG8gIFNpbXBseSBhZG1p dHRpbmcgdGhlIG5ldyBmbG93IGhhcyBhIHNpZ25pZmljYW50IHJpc2sgb2YgbGVhZGluZyB0bw0K ICAgICAgb3ZlcmxvYWQsIGJlY2F1c2UgdGhlIFBDTi1kb21haW4gcmVhY2hlcyBvdXQgdG93YXJk cyB0aGUgZW5kDQogICAgICB0ZXJtaW5hbHMgd2hlcmUgbGluayBjYXBhY2l0eSBpcyBsb3cuDQoN CiAgIG8gIEV2ZXJ5IGFkbWlzc2lvbiBjb250cm9sIGRlY2lzaW9uIGludm9sdmVzIHByb2Jpbmcs IHVzaW5nIHRoZQ0KICAgICAgc2lnbmFsbGluZyBzZXQtdXAgbWVzc2FnZSBhcyB0aGUgcHJvYmUg cGFja2V0IChlZyBSU1ZQIFBBVEgpLg0KDQogICBvICBUaGUgUENOLW1hcmtpbmcgYmVoYXZpb3Vy IGlzIHN1Y2ggdGhhdCBldmVyeSBwYWNrZXQgaXMgUENOLW1hcmtlZA0KICAgICAgaWYgdGhlIGZs b3cgc2hvdWxkIGJlIGJsb2NrZWQsIGhlbmNlIG9ubHkgYSBzaW5nbGUgcHJvYmluZyBwYWNrZXQN CiAgICAgIGlzIG5lZWRlZC4NCg0KICAgVGhlIGZpcnN0IHBvaW50IGJyZWFrcyBBc3N1bXB0aW9u IDMgKGFnZ3JlZ2F0aW9uKSBhbmQgaGVuY2UgbWVhbnMNCiAgIHRoYXQgdGhpcyB2aWV3cG9pbnQg aXMgb3V0IG9mIHNjb3BlIG9mIHRoZSBpbml0aWFsIENoYXJ0ZXIgb2YgdGhlIFBDTg0KICAgV0cu DQoNCg0KDQoNCg0KDQoNCkVhcmRsZXkgKEVkaXRvcikgICAgICAgICAgRXhwaXJlcyBNYXkgMjIs IDIwMDggICAgICAgICAgICAgICAgIFtQYWdlIDI5XQ0KDA0KSW50ZXJuZXQtRHJhZnQgICAgICAg ICAgICAgICAgICBEb2N1bWVudCAgICAgICAgICAgICAgICAgICBOb3ZlbWJlciAyMDA3DQoNCg0K OC4gIE9wZXJhdGlvbnMgYW5kIE1hbmFnZW1lbnQNCg0KICAgVGhpcyBTZWN0aW9uIGNvbnNpZGVy cyBvcGVyYXRpb25zIGFuZCBtYW5hZ2VtZW50IGlzc3VlcywgdW5kZXIgdGhlDQogICBGQ0FQUyBo ZWFkaW5nczogT0FNIG9mIEZhdWx0cywgQ29uZmlndXJhdGlvbiwgQWNjb3VudGluZywgUGVyZm9y bWFuY2UNCiAgIGFuZCBTZWN1cml0eS4gIFByb3Zpc2lvbmluZyBpcyBkaXNjdXNzZWQgd2l0aCBw ZXJmb3JtYW5jZS4NCg0KOC4xLiAgQ29uZmlndXJhdGlvbiBPQU0NCg0KICAgVGhpcyBhcmNoaXRl Y3R1cmUgZG9jdW1lbnQgcHJlZGF0ZXMgdGhlIGRldGFpbGVkIHN0YW5kYXJkcyBhY3Rpb25zIG9m DQogICB0aGUgUENOIFdHLiAgSGVyZSB3ZSBhc3N1bWUgdGhhdCBvbmx5IGludGVyb3BlcmFibGUg UENOLW1hcmtpbmcNCiAgIGJlaGF2aW91cnMgd2lsbCBiZSBzdGFuZGFyZGlzZWQsIG90aGVyd2lz ZSB3ZSB3b3VsZCBoYXZlIHRvIGNvbnNpZGVyDQogICBob3cgdG8gYXZvaWQgaW50ZXJhY3Rpb25z IGJldHdlZW4gbm9uLWludGVyb3BlcmFibGUgbWFya2luZw0KICAgYmVoYXZpb3Vycy4gIEhvd2V2 ZXIsIG1vcmUgZGl2ZXJzaXR5IGluIGVkZ2Utbm9kZSBiZWhhdmlvdXJzIGlzDQogICBleHBlY3Rl ZCwgaW4gb3JkZXIgdG8gaW50ZXJmYWNlIHdpdGggZGl2ZXJzZSBpbmR1c3RyeSBhcmNoaXRlY3R1 cmVzLg0KDQogICBQQ04gY29uZmlndXJhdGlvbiBjb250cm9sIHZhcmlhYmxlcyBmYWxsIGludG8g dGhlIGZvbGxvd2luZw0KICAgY2F0ZWdvcmllczoNCg0KICAgbyAgc3lzdGVtIG9wdGlvbnMgKGVu YWJsaW5nIG9yIGRpc2FibGluZyBiZWhhdmlvdXJzKQ0KDQogICBvICBwYXJhbWV0ZXJzIChzZXR0 aW5nIGxldmVscywgYWRkcmVzc2VzIGV0YykNCg0KICAgQWxsIGNvbmZpZ3VyYWJsZSB2YXJpYWJs ZXMgd2lsbCBuZWVkIHRvIHNpdCB3aXRoaW4gYW4gU05NUCBtYW5hZ2VtZW50DQogICBmcmFtZXdv cmsgW1JGQzM0MTFdLCBiZWluZyBzdHJ1Y3R1cmVkIHdpdGhpbiBhIGRlZmluZWQgbWFuYWdlbWVu dA0KICAgaW5mb3JtYXRpb24gYmFzZSAoTUlCKSBvbiBlYWNoIG5vZGUsIGFuZCBiZWluZyByZW1v dGVseSByZWFkYWJsZSBhbmQNCiAgIHNldHRhYmxlIHZpYSBhIHN1aXRhYmx5IHNlY3VyZSBtYW5h Z2VtZW50IHByb3RvY29sIChTTk1QdjMpLg0KDQogICBTb21lIGNvbmZpZ3VyYXRpb24gb3B0aW9u cyBhbmQgcGFyYW1ldGVycyBoYXZlIHRvIGJlIHNldCBvbmNlIHRvDQogICAnZ2xvYmFsbHknIGNv bnRyb2wgdGhlIHdob2xlIFBDTi1kb21haW4uICBXaGVyZSBwb3NzaWJsZSwgdGhlc2UgYXJlDQog ICBpZGVudGlmaWVkIGJlbG93LiAgVGhpcyBtYXkgYWZmZWN0IG9wZXJhdGlvbmFsIGNvbXBsZXhp dHkgYW5kIHRoZQ0KICAgY2hhbmNlcyBvZiBpbnRlcm9wZXJhYmlsaXR5IHByb2JsZW1zIGJldHdl ZW4ga2l0IGZyb20gZGlmZmVyZW50DQogICB2ZW5kb3JzLg0KDQo4LjEuMS4gIFN5c3RlbSBvcHRp b25zDQoNCiAgIE9uIFBDTi1pbnRlcmlvci1ub2RlcyB0aGVyZSB3aWxsIGJlIHZlcnkgZmV3IHN5 c3RlbSBvcHRpb25zOg0KDQogICBvICBXaGV0aGVyIHR3byBQQ04tbWFya2luZ3MgKGJhc2VkIG9u IHRoZSBQQ04tbG93ZXItcmF0ZSBhbmQgUENOLQ0KICAgICAgdXBwZXItcmF0ZSkgYXJlIGVuYWJs ZWQgb3Igb25seSBvbmUgKHNlZSBTZWN0aW9uIDQuMykuICBUeXBpY2FsbHkNCiAgICAgIGFsbCBu b2RlcyB0aHJvdWdob3V0IGEgUENOLWRvbWFpbiB3aWxsIGJlIGNvbmZpZ3VyZWQgdGhlIHNhbWUg aW4NCiAgICAgIHRoaXMgcmVzcGVjdC4gIEhvd2V2ZXIsIGV4Y2VwdGlvbnMgY291bGQgYmUgbWFk ZS4gIEZvciBleGFtcGxlLCBpZg0KICAgICAgbW9zdCBQQ04tbm9kZXMgdXNlZCBib3RoIG1hcmtp bmdzLCBidXQgc29tZSBsZWdhY3kgaGFyZHdhcmUgd2FzDQogICAgICBpbmNhcGFibGUgb2YgcnVu bmluZyB0d28gYWxnb3JpdGhtcywgYW4gb3BlcmF0b3IgbWlnaHQgYmUgd2lsbGluZw0KICAgICAg dG8gY29uZmlndXJlIHRoZXNlIGxlZ2FjeSBub2RlcyBzb2xlbHkgZm9yIFBDTi1tYXJraW5nIGJh c2VkIG9uDQogICAgICB0aGUgUENOLXVwcGVyLXJhdGUgdG8gZW5hYmxlIGZsb3cgdGVybWluYXRp b24gYXMgYSBiYWNrLXN0b3AuICBJdA0KICAgICAgd291bGQgYmUgc2Vuc2libGUgdG8gcGxhY2Ug c3VjaCBub2RlcyB3aGVyZSB0aGV5IGNvdWxkIGJlDQogICAgICBwcm92aXNpb25lZCB3aXRoIGEg Z3JlYXRlciBsZWV3YXkgb3ZlciBleHBlY3RlZCB0cmFmZmljIGxldmVscy4NCg0KDQoNCg0KRWFy ZGxleSAoRWRpdG9yKSAgICAgICAgICBFeHBpcmVzIE1heSAyMiwgMjAwOCAgICAgICAgICAgICAg ICAgW1BhZ2UgMzBdDQoMDQpJbnRlcm5ldC1EcmFmdCAgICAgICAgICAgICAgICAgIERvY3VtZW50 ICAgICAgICAgICAgICAgICAgIE5vdmVtYmVyIDIwMDcNCg0KDQogICBvICB3aGljaCBtYXJraW5n IGFsZ29yaXRobSB0byB1c2UsIGlmIGFuIGVxdWlwbWVudCB2ZW5kb3IgcHJvdmlkZXMgYQ0KICAg ICAgY2hvaWNlDQoNCiAgIFBDTi1ib3VuZGFyeS1ub2RlcyAoaW5ncmVzcyBhbmQgZWdyZXNzKSB3 aWxsIGhhdmUgbW9yZSBzeXN0ZW0NCiAgIG9wdGlvbnM6DQoNCiAgIG8gIFdoaWNoIG9mIGFkbWlz c2lvbiBhbmQgZmxvdyB0ZXJtaW5hdGlvbiBhcmUgZW5hYmxlZC4gIElmIGFueSBQQ04tDQogICAg ICBpbnRlcmlvci1ub2RlIGlzIGNvbmZpZ3VyZWQgdG8gZ2VuZXJhdGUgYSBtYXJraW5nLCBhbGwg UENOLQ0KICAgICAgYm91bmRhcnktbm9kZXMgbXVzdCBiZSBhYmxlIHRvIGhhbmRsZSB0aGF0IG1h cmtpbmcuICBUaGVyZWZvcmUgYWxsDQogICAgICBQQ04tYm91bmRhcnktbm9kZXMgbXVzdCBiZSBj b25maWd1cmVkIHRoZSBzYW1lIGluIHRoaXMgcmVzcGVjdC4NCg0KICAgbyAgV2hlcmUgZmxvdyBh ZG1pc3Npb24gYW5kIHRlcm1pbmF0aW9uIGRlY2lzaW9ucyBhcmUgbWFkZTogYXQgdGhlDQogICAg ICBQQ04taW5ncmVzcy1ub2RlLCBQQ04tZWdyZXNzLW5vZGUgb3IgYXQgYSBjZW50cmFsaXNlZCBu b2RlIChzZWUNCiAgICAgIFNlY3Rpb25zIDUuNCBhbmQgNS41KS4gIFRoZW9yZXRpY2FsbHksIHRo aXMgY29uZmlndXJhdGlvbiBjaG9pY2UNCiAgICAgIGNvdWxkIGJlIG5lZ290aWF0ZWQgZm9yIGVh Y2ggcGFpciBvZiBQQ04tYm91bmRhcnktbm9kZXMsIGJ1dCB3ZQ0KICAgICAgY2Fubm90IGltYWdp bmUgd2h5IHN1Y2ggY29tcGxleGl0eSB3b3VsZCBiZSByZXF1aXJlZCwgZXhjZXB0DQogICAgICBw ZXJoYXBzIGluIGZ1dHVyZSBpbnRlci1kb21haW4gc2NlbmFyaW9zLg0KDQogICBQQ04tZWdyZXNz LW5vZGVzIHdpbGwgaGF2ZSBmdXJ0aGVyIHN5c3RlbSBvcHRpb25zOg0KDQogICBvICBIb3cgdGhl IG1hcHBpbmcgc2hvdWxkIGJlIGVzdGFibGlzaGVkIGJldHdlZW4gZWFjaCBwYWNrZXQgYW5kIGl0 cw0KICAgICAgYWdncmVnYXRlLCBlZyBieSBNUExTIGxhYmVsLCBieSBJUCBwYWNrZXQgZmlsdGVy c3BlYzsgYW5kIGhvdyB0bw0KICAgICAgdGFrZSBhY2NvdW50IG9mIEVDTVAuDQoNCiAgIG8gIElm IGFuIGVxdWlwbWVudCB2ZW5kb3IgcHJvdmlkZXMgYSBjaG9pY2UsIHRoZXJlIG1heSBiZSBvcHRp b25zIHRvDQogICAgICBzZWxlY3Qgd2hpY2ggc21vb3RoaW5nIGFsZ29yaXRobSB0byB1c2UgZm9y IG1lYXN1cmVtZW50cy4NCg0KOC4xLjIuICBQYXJhbWV0ZXJzDQoNCiAgIExpa2UgYW55IERpZmZT ZXJ2IGRvbWFpbiwgZXZlcnkgbm9kZSB3aXRoaW4gYSBQQ04tZG9tYWluIHdpbGwgbmVlZCB0bw0K ICAgYmUgY29uZmlndXJlZCB3aXRoIHRoZSBEU0NQKHMpIHVzZWQgdG8gaWRlbnRpZnkgUENOLXBh Y2tldHMuICBPbiBlYWNoDQogICBpbnRlcmlvciBsaW5rIHRoZSBtYWluIGNvbmZpZ3VyYXRpb24g cGFyYW1ldGVycyBhcmUgdGhlIFBDTi1sb3dlci0NCiAgIHJhdGUgYW5kIFBDTi11cHBlci1yYXRl LiAgQSBsYXJnZXIgUENOLWxvd2VyLXJhdGUgZW5hYmxlcyBtb3JlIFBDTi0NCiAgIHRyYWZmaWMg dG8gYmUgYWRtaXR0ZWQgb24gYSBsaW5rLCBoZW5jZSBpbXByb3ZpbmcgY2FwYWNpdHkNCiAgIHV0 aWxpc2F0aW9uLiAgQSBQQ04tdXBwZXItcmF0ZSBzZXQgZnVydGhlciBhYm92ZSB0aGUgUENOLWxv d2VyLXJhdGUNCiAgIGFsbG93cyBncmVhdGVyIGluY3JlYXNlcyBpbiB0cmFmZmljICh3aGV0aGVy IGR1ZSB0byBuYXR1cmFsDQogICBmbHVjdHVhdGlvbnMgb3Igc29tZSB1bmV4cGVjdGVkIGV2ZW50 KSBiZWZvcmUgYW55IGZsb3dzIGFyZQ0KICAgdGVybWluYXRlZCwgaWUgbWluaW1pc2VzIHRoZSBj aGFuY2VzIG9mIHVubmVjZXNzYXJpbHkgdHJpZ2dlcmluZyB0aGUNCiAgIHRlcm1pbmF0aW9uIG1l Y2hhbmlzbS4gIEZvciBpbnN0YW5jZSBhbiBvcGVyYXRvciBtYXkgd2FudCB0byBkZXNpZ24NCiAg IHRoZWlyIG5ldHdvcmsgc28gdGhhdCBpdCBjYW4gY29wZSB3aXRoIGEgZmFpbHVyZSBvZiBhbnkg c2luZ2xlIFBDTi0NCiAgIG5vZGUgd2l0aG91dCB0ZXJtaW5hdGluZyBhbnkgZmxvd3MuDQoNCiAg IFNldHRpbmcgdGhlc2UgcmF0ZXMgb24gZmlyc3QgZGVwbG95bWVudCBvZiBQQ04gd2lsbCBiZSB2 ZXJ5IHNpbWlsYXINCiAgIHRvIHRoZSB0cmFkaXRpb25hbCBwcm9jZXNzIGZvciBzaXppbmcgYW4g YWRtaXNzaW9uIGNvbnRyb2xsZWQNCiAgIG5ldHdvcmssIGRlcGVuZGluZyBvbjogdGhlIG9wZXJh dG9yJ3MgcmVxdWlyZW1lbnRzIGZvciBtaW5pbWlzaW5nDQogICBmbG93IGJsb2NraW5nIChncmFk ZSBvZiBzZXJ2aWNlKSwgdGhlIGV4cGVjdGVkIFBDTiB0cmFmZmljIGxvYWQgb24NCiAgIGVhY2gg bGluayBhbmQgaXRzIHN0YXRpc3RpY2FsIGNoYXJhY3RlcmlzdGljcyAodGhlIHRyYWZmaWMgbWF0 cml4KSwNCiAgIGNvbnRpbmdlbmN5IGZvciByZS1yb3V0aW5nIHRoZSBQQ04gdHJhZmZpYyBtYXRy aXggaW4gdGhlIGV2ZW50IG9mDQoNCg0KDQpFYXJkbGV5IChFZGl0b3IpICAgICAgICAgIEV4cGly ZXMgTWF5IDIyLCAyMDA4ICAgICAgICAgICAgICAgICBbUGFnZSAzMV0NCgwNCkludGVybmV0LURy YWZ0ICAgICAgICAgICAgICAgICAgRG9jdW1lbnQgICAgICAgICAgICAgICAgICAgTm92ZW1iZXIg MjAwNw0KDQoNCiAgIHNpbmdsZSBvciBtdWx0aXBsZSBmYWlsdXJlcyBhbmQgdGhlIGV4cGVjdGVk IGxvYWQgZnJvbSBvdGhlciBjbGFzc2VzDQogICByZWxhdGl2ZSB0byBsaW5rIGNhcGFjaXRpZXMu ICBCdXQgb25jZSBhIGRvbWFpbiBpcyB1cCBhbmQgcnVubmluZywgYQ0KICAgUENOIGRlc2lnbiBn b2FsIGlzIHRvIGJlIGFibGUgdG8gZGV0ZXJtaW5lIGdyb3d0aCBpbiB0aGVzZSBjb25maWd1cmVk DQogICByYXRlcyBtdWNoIG1vcmUgc2ltcGx5LCBieSBtb25pdG9yaW5nIFBDTi1tYXJraW5nIHJh dGVzIGZyb20gYWN0dWFsDQogICByYXRoZXIgdGhhbiBleHBlY3RlZCB0cmFmZmljIChzZWUgU2Vj dGlvbiA4LjIgb24gUGVyZm9ybWFuY2UgJg0KICAgUHJvdmlzaW9uaW5nKS4NCg0KICAgT3BlcmF0 b3JzIG1heSBhbHNvIHdpc2ggdG8gY29uZmlndXJlIGEgcmF0ZSBncmVhdGVyIHRoYW4gdGhlIFBD Ti0NCiAgIHVwcGVyLXJhdGUgdGhhdCBpcyB0aGUgYWJzb2x1dGUgbWF4aW11bSByYXRlIHRoYXQg YSBsaW5rIGFsbG93cyBmb3INCiAgIFBDTi10cmFmZmljLiAgVGhpcyBtYXkgc2ltcGx5IGJlIHRo ZSBwaHlzaWNhbCBsaW5rIHJhdGUsIGJ1dCBzb21lDQogICBvcGVyYXRvcnMgbWF5IHdpc2ggdG8g Y29uZmlndXJlIGEgbG9naWNhbCBsaW1pdCB0byBwcmV2ZW50IHN0YXJ2YXRpb24NCiAgIG9mIG90 aGVyIHRyYWZmaWMgY2xhc3NlcyBkdXJpbmcgYW55IGJyaWVmIHBlcmlvZCBhZnRlciBQQ04tdHJh ZmZpYw0KICAgZXhjZWVkcyB0aGUgUENOLXVwcGVyLXJhdGUgYnV0IGJlZm9yZSBmbG93IHRlcm1p bmF0aW9uIGJyaW5ncyBpdCBiYWNrDQogICBiZWxvdyB0aGlzIHJhdGUuDQoNCiAgIFNwZWNpZmlj IG1hcmtpbmcgYWxnb3JpdGhtcyB3aWxsIGFsc28gZGVwZW5kIG9uIGZ1cnRoZXIgY29uZmlndXJh dGlvbg0KICAgcGFyYW1ldGVycy4gIEZvciBpbnN0YW5jZSwgdGhyZXNob2xkLW1hcmtpbmcgd2ls bCByZXF1aXJlIGEgdGhyZXNob2xkDQogICBxdWV1ZSBkZXB0aCBhbmQgZXhjZXNzLXJhdGUtbWFy a2luZyBtYXkgcmVxdWlyZSBhIHNjYWxpbmcgcGFyYW1ldGVyLg0KICAgSXQgd2lsbCBiZSBwcmVm ZXJhYmxlIGZvciBlYWNoIG1hcmtpbmcgYWxnb3JpdGhtIHRvIGhhdmUgcnVsZXMgdG8gc2V0DQog ICBkZWZhdWx0cyBmb3IgdGhlc2UgcGFyYW1ldGVycyByZWxhdGl2ZSB0byB0aGUgcmVmZXJlbmNl IG1hcmtpbmcgcmF0ZSwNCiAgIGJ1dCB0aGVuIGFsbG93IG9wZXJhdG9ycyB0byBjaGFuZ2UgdGhl bSwgZm9yIGluc3RhbmNlIGlmIGF2ZXJhZ2UNCiAgIHRyYWZmaWMgY2hhcmFjdGVyaXN0aWNzIGNo YW5nZSBvdmVyIHRpbWUuICBUaGUgUENOLWVncmVzcy1ub2RlIG1heQ0KICAgYWxsb3cgY29uZmln dXJhdGlvbiBvZiB0aGUgZm9sbG93aW5nOg0KDQogICBvICBob3cgaXQgc21vb3RoZXMgbWV0ZXJp bmcgb2YgUENOLW1hcmtpbmdzIChlZyBFV01BIHBhcmFtZXRlcnMpDQoNCiAgIFdoaWNoZXZlciBu b2RlIG1ha2VzIGFkbWlzc2lvbiBhbmQgZmxvdyB0ZXJtaW5hdGlvbiBkZWNpc2lvbnMgd2lsbA0K ICAgY29udGFpbiBhbGdvcml0aG1zIGZvciBjb252ZXJ0aW5nIFBDTi1tYXJraW5nIGxldmVscyBp bnRvIGFkbWlzc2lvbg0KICAgb3IgZmxvdyB0ZXJtaW5hdGlvbiBkZWNpc2lvbnMuICBUaGVzZSB3 aWxsIGFsc28gcmVxdWlyZSBjb25maWd1cmFibGUNCiAgIHBhcmFtZXRlcnMsIGZvciBpbnN0YW5j ZToNCg0KICAgbyAgQW55IGFkbWlzc2lvbiBjb250cm9sIGFsZ29yaXRobSB3aWxsIGF0IGxlYXN0 IHJlcXVpcmUgYSBtYXJraW5nDQogICAgICB0aHJlc2hvbGQgc2V0dGluZyBhYm92ZSB3aGljaCBp dCBkZW5pZXMgYWRtaXNzaW9uIHRvIG5ldyBmbG93czsNCg0KICAgbyAgZmxvdyB0ZXJtaW5hdGlv biBhbGdvcml0aG1zIHdpbGwgcHJvYmFibHkgcmVxdWlyZSBhIHBhcmFtZXRlciB0bw0KICAgICAg ZGVsYXkgdGVybWluYXRpb24gb2YgYW55IGZsb3dzIHVudGlsIGl0IGlzIG1vcmUgY2VydGFpbiB0 aGF0IGFuDQogICAgICBhbm9tYWxvdXMgZXZlbnQgaXMgbm90IHRyYW5zaWVudDsNCg0KICAgbyAg YSBwYXJhbWV0ZXIgdG8gY29udHJvbCB0aGUgdHJhZGUtb2ZmIGJldHdlZW4gaG93IHF1aWNrbHkg ZXhjZXNzDQogICAgICBmbG93cyBhcmUgdGVybWluYXRlZCBhbmQgb3Zlci10ZXJtaW5hdGlvbi4N Cg0KICAgT25lIHBhcnRpY3VsYXIgcHJvcG9zYWwsIFtJLUQuY2hhcm55LXBjbi1zaW5nbGUtbWFy a2luZ10gd291bGQNCiAgIHJlcXVpcmUgYSBnbG9iYWwgcGFyYW1ldGVyIHRvIGJlIGRlZmluZWQg b24gYWxsIFBDTi1ub2RlcywgYnV0IG9ubHkNCiAgIG5lZWRzIHRoZSBQQ04tbG93ZXItcmF0ZSB0 byBiZSBjb25maWd1cmVkIG9uIGVhY2ggbGluay4gIFRoZSBnbG9iYWwNCiAgIHBhcmFtZXRlciBp cyBhIHNjYWxpbmcgZmFjdG9yIGJldHdlZW4gYWRtaXNzaW9uIGFuZCB0ZXJtaW5hdGlvbiwgZm9y DQogICBleGFtcGxlIHRoZSBhbW91bnQgYnkgd2hpY2ggdGhlIFBDTi11cHBlci1yYXRlIGlzIGlt cGxpY2l0bHkgYXNzdW1lZA0KICAgdG8gYmUgYWJvdmUgdGhlIFBDTi1sb3dlci1yYXRlLiAgW0kt RC5jaGFybnktcGNuLXNpbmdsZS1tYXJraW5nXQ0KICAgZGlzY3Vzc2VzIGluIGZ1bGwgdGhlIGlt cGFjdCBvZiB0aGlzIHBhcnRpY3VsYXIgcHJvcG9zYWwgb24gdGhlDQoNCg0KDQpFYXJkbGV5IChF ZGl0b3IpICAgICAgICAgIEV4cGlyZXMgTWF5IDIyLCAyMDA4ICAgICAgICAgICAgICAgICBbUGFn ZSAzMl0NCgwNCkludGVybmV0LURyYWZ0ICAgICAgICAgICAgICAgICAgRG9jdW1lbnQgICAgICAg ICAgICAgICAgICAgTm92ZW1iZXIgMjAwNw0KDQoNCiAgIG9wZXJhdGlvbiBvZiBQQ04uDQoNCjgu Mi4gIFBlcmZvcm1hbmNlICYgUHJvdmlzaW9uaW5nIE9BTQ0KDQogICBNb25pdG9yaW5nIG9mIHBl cmZvcm1hbmNlIGZhY3RvcnMgbWVhc3VyYWJsZSBmcm9tICpvdXRzaWRlKiB0aGUgUENODQogICBk b21haW4gd2lsbCBiZSBubyBkaWZmZXJlbnQgd2l0aCBQQ04gdGhhbiB3aXRoIGFueSBvdGhlciBw YWNrZXQtYmFzZWQNCiAgIGZsb3cgYWRtaXNzaW9uIGNvbnRyb2wgc3lzdGVtLCBib3RoIGF0IHRo ZSBmbG93IGxldmVsIChibG9ja2luZw0KICAgcHJvYmFiaWxpdHkgZXRjKSBhbmQgdGhlIHBhY2tl dCBsZXZlbCAoaml0dGVyIFtSRkMzMzkzXSwgW1kuMTU0MV0sDQogICBsb3NzIHJhdGUgW1JGQzQ2 NTZdLCBtZWFuIG9waW5pb24gc2NvcmUgW1AuODAwXSwgZXRjKS4gIFRoZQ0KICAgZGlmZmVyZW5j ZSBpcyB0aGF0IFBDTiBpcyBpbnRlbnRpb25hbGx5IGRlc2lnbmVkIHRvIGluZGljYXRlDQogICAq aW50ZXJuYWxseSogd2hpY2ggZXhhY3QgcmVzb3VyY2UocykgYXJlIHRoZSBjYXVzZSBvZiBwZXJm b3JtYW5jZQ0KICAgcHJvYmxlbXMgYW5kIGJ5IGhvdyBtdWNoLg0KDQogICBFdmVuIGJldHRlciwg UENOIGluZGljYXRlcyB3aGljaCByZXNvdXJjZXMgd2lsbCBwcm9iYWJseSBjYXVzZQ0KICAgcHJv YmxlbXMgaWYgdGhleSBhcmUgbm90IHVwZ3JhZGVkIHNvb24uICBUaGlzIGNhbiBiZSBhY2hpZXZl ZCBieSB0aGUNCiAgIG1hbmFnZW1lbnQgc3lzdGVtIG1vbml0b3JpbmcgdGhlIHRvdGFsIGFtb3Vu dCAoaW4gYnl0ZXMpIG9mIFBDTi0NCiAgIG1hcmtpbmcgZ2VuZXJhdGVkIGJ5IGVhY2ggcXVldWUg b3ZlciBhIHBlcmlvZC4gIEdpdmVuIHBvc3NpYmxlIGxvbmcNCiAgIHByb3Zpc2lvbmluZyBsZWFk IHRpbWVzLCBwcmUtY29uZ2VzdGlvbiB2b2x1bWUgaXMgdGhlIGJlc3QgbWV0cmljIHRvDQogICBy ZXZlYWwgd2hldGhlciBzdWZmaWNpZW50IHBlcnNpc3RlbnQgZGVtYW5kIGhhcyBtb3VudGVkIHVw IHRvIHdhcnJhbnQNCiAgIGFuIHVwZ3JhZGUuICBCZWNhdXNlLCBldmVuIGJlZm9yZSB1dGlsaXNh dGlvbiBiZWNvbWVzIHByb2JsZW1hdGljLA0KICAgdGhlIHN0YXRpc3RpY2FsIHZhcmlhYmlsaXR5 IG9mIHRyYWZmaWMgd2lsbCBjYXVzZSBvY2Nhc2lvbmFsIGJ1cnN0cw0KICAgb2YgcHJlLWNvbmdl c3Rpb24uICBUaGlzICdlYXJseSB3YXJuaW5nIHN5c3RlbScgZGVjb3VwbGVzIHRoZSBwcm9jZXNz DQogICBvZiBhZGRpbmcgY3VzdG9tZXJzIGZyb20gdGhlIHByb3Zpc2lvbmluZyBwcm9jZXNzLiAg VGhpcyBzaG91bGQgY3V0DQogICB0aGUgdGltZSB0byBhZGQgYSBjdXN0b21lciB3aGVuIGNvbXBh cmVkIGFnYWluc3QgYWRtaXNzaW9uIGNvbnRyb2wNCiAgIHByb3ZpZGVkIG92ZXIgbmF0aXZlIERp ZmZTZXJ2IFtSRkMyOTk4XSwgYmVjYXVzZSBpdCBzYXZlcyBoYXZpbmcgdG8NCiAgIHJlLXJ1biB0 aGUgY2FwYWNpdHkgcGxhbm5pbmcgcHJvY2VzcyBiZWZvcmUgYWRkaW5nIGVhY2ggY3VzdG9tZXIu DQoNCiAgIEFsdGVybmF0aXZlbHksIGJlZm9yZSB0cmlnZ2VyaW5nIGFuIHVwZ3JhZGUsIHRoZSBs b25nIHRlcm0gcHJlLQ0KICAgY29uZ2VzdGlvbiB2b2x1bWUgb24gZWFjaCBsaW5rIGNhbiBiZSB1 c2VkIHRvIGJhbGFuY2UgdHJhZmZpYyBsb2FkDQogICBhY3Jvc3MgdGhlIFBDTi1kb21haW4gYnkg YWRqdXN0aW5nIHRoZSBsaW5rIHdlaWdodHMgb2YgdGhlIHJvdXRpbmcNCiAgIHN5c3RlbS4gIFdo ZW4gYW4gdXBncmFkZSB0byBhIGxpbmsncyBjb25maWd1cmVkIFBDTi1yYXRlcyBpcw0KICAgcmVx dWlyZWQsIGl0IG1heSBhbHNvIGJlIG5lY2Vzc2FyeSB0byB1cGdyYWRlIHRoZSBwaHlzaWNhbCBj YXBhY2l0eQ0KICAgYXZhaWxhYmxlIHRvIG90aGVyIGNsYXNzZXMuICBCdXQgdXN1YWxseSB0aGVy ZSB3aWxsIGJlIHN1ZmZpY2llbnQNCiAgIHBoeXNpY2FsIGNhcGFjaXR5IGZvciB0aGUgdXBncmFk ZSB0byBnbyBhaGVhZCBhcyBhIHNpbXBsZQ0KICAgY29uZmlndXJhdGlvbiBjaGFuZ2UuICBBbHRl cm5hdGl2ZWx5LCBbU29uZ2h1cnN0XSBoYXMgcHJvcG9zZWQgYW4NCiAgIGFkYXB0aXZlIHJhdGhl ciB0aGFuIHByZWNvbmZpZ3VyZWQgc3lzdGVtLCB3aGVyZSB0aGUgY29uZmlndXJlZCBQQ04tDQog ICBsb3dlci1yYXRlIGlzIHJlcGxhY2VkIHdpdGggYSBoaWdoIGFuZCBsb3cgd2F0ZXIgbWFyayBh bmQgdGhlIG1hcmtpbmcNCiAgIGFsZ29yaXRobSBhdXRvbWF0aWNhbGx5IG9wdGltaXNlcyBob3cg cGh5c2ljYWwgY2FwYWNpdHkgaXMgc2hhcmVkDQogICB1c2luZyB0aGUgcmVsYXRpdmUgbG9hZHMg ZnJvbSBQQ04gYW5kIG90aGVyIHRyYWZmaWMgY2xhc3Nlcy4NCg0KICAgQWxsIHRoZSBhYm92ZSBw cm9jZXNzZXMgcmVxdWlyZSBqdXN0IHRocmVlIGV4dHJhIGNvdW50ZXJzIGFzc29jaWF0ZWQNCiAg IHdpdGggZWFjaCBQQ04gcXVldWU6IFBDTi1tYXJraW5ncyBhc3NvY2lhdGVkIHdpdGggdGhlIFBD Ti1sb3dlci1yYXRlDQogICBhbmQgUENOLXVwcGVyLXJhdGUsIGFuZCBkcm9wLiAgRXZlcnkgdGlt ZSBhIFBDTiBwYWNrZXQgaXMgbWFya2VkIG9yDQogICBkcm9wcGVkIGl0cyBzaXplIGluIGJ5dGVz IHNob3VsZCBiZSBhZGRlZCB0byB0aGUgYXBwcm9wcmlhdGUgY291bnRlci4NCiAgIFRoZW4gdGhl IG1hbmFnZW1lbnQgc3lzdGVtIGNhbiByZWFkIHRoZSBjb3VudGVycyBhdCBhbnkgdGltZSBhbmQN CiAgIHN1YnRyYWN0IGEgcHJldmlvdXMgcmVhZGluZyB0byBlc3RhYmxpc2ggdGhlIGluY3JlbWVu dGFsIHZvbHVtZSBvZg0KICAgZWFjaCB0eXBlIG9mIChwcmUtKWNvbmdlc3Rpb24uICBSZWFkaW5n cyBzaG91bGQgYmUgdGFrZW4gZnJlcXVlbnRseSwNCiAgIHNvIHRoYXQgYW5vbWFsb3VzIGV2ZW50 cyAoZWcgcmUtcm91dGVzKSBjYW4gYmUgc2VwYXJhdGVkIGZyb20gcmVndWxhcg0KDQoNCg0KRWFy ZGxleSAoRWRpdG9yKSAgICAgICAgICBFeHBpcmVzIE1heSAyMiwgMjAwOCAgICAgICAgICAgICAg ICAgW1BhZ2UgMzNdDQoMDQpJbnRlcm5ldC1EcmFmdCAgICAgICAgICAgICAgICAgIERvY3VtZW50 ICAgICAgICAgICAgICAgICAgIE5vdmVtYmVyIDIwMDcNCg0KDQogICBmbHVjdHVhdGluZyBkZW1h bmQgaWYgcmVxdWlyZWQuDQoNCjguMy4gIEFjY291bnRpbmcgT0FNDQoNCiAgIEFjY291bnRpbmcg aXMgb25seSBkb25lIGF0IHRydXN0IGJvdW5kYXJpZXMgc28gaXQgaXMgb3V0IG9mIHNjb3BlIG9m DQogICB0aGUgaW5pdGlhbCBDaGFydGVyIG9mIHRoZSBQQ04gV0cgd2hpY2ggaXMgY29uZmluZWQg dG8gaW50cmEtZG9tYWluDQogICBpc3N1ZXMuICBVc2Ugb2YgUENOIGludGVybmFsIHRvIGEgZG9t YWluIG1ha2VzIG5vIGRpZmZlcmVuY2UgdG8gdGhlDQogICBmbG93IHNpZ25hbGxpbmcgZXZlbnRz IGNyb3NzaW5nIHRydXN0IGJvdW5kYXJpZXMgb3V0c2lkZSB0aGUgUENOLQ0KICAgZG9tYWluLCB3 aGljaCBhcmUgdHlwaWNhbGx5IHVzZWQgZm9yIGFjY291bnRpbmcuDQoNCjguNC4gIEZhdWx0IE9B TQ0KDQogICBGYXVsdCBPQU0gaXMgYWJvdXQgcHJldmVudGluZyBmYXVsdHMsIHRlbGxpbmcgdGhl IG1hbmFnZW1lbnQgc3lzdGVtDQogICAob3IgbWFudWFsIG9wZXJhdG9yKSB0aGF0IHRoZSBzeXN0 ZW0gaGFzIHJlY292ZXJlZCAob3Igbm90KSBmcm9tIGENCiAgIGZhaWx1cmUsIGFuZCBhYm91dCBt YWludGFpbmluZyBpbmZvcm1hdGlvbiB0byBhaWQgZmF1bHQgZGlhZ25vc2lzLg0KDQogICBBZG1p c3Npb24gYmxvY2tpbmcgYW5kIHBhcnRpY3VsYXJseSBmbG93IHRlcm1pbmF0aW9uIG1lY2hhbmlz bXMNCiAgIHNob3VsZCByYXJlbHkgYmUgbmVlZGVkIGluIHByYWN0aWNlLiAgSXQgd291bGQgYmUg dW5mb3J0dW5hdGUgaWYgdGhleQ0KICAgZGlkbid0IHdvcmsgYWZ0ZXIgYW4gb3B0aW9uIGhhZCBi ZWVuIGFjY2lkZW50YWxseSBkaXNhYmxlZC4NCiAgIFRoZXJlZm9yZSBpdCB3aWxsIGJlIG5lY2Vz c2FyeSB0byByZWd1bGFybHkgdGVzdCB0aGF0IHRoZSBsaXZlIHN5c3RlbQ0KICAgd29ya3MgYXMg aW50ZW5kZWQgKGRldmlzaW5nIGEgbWVhbmluZ2Z1bCB0ZXN0IGlzIGxlZnQgYXMgYW4gZXhlcmNp c2UNCiAgIGZvciB0aGUgb3BlcmF0b3IpLg0KDQogICBTZWN0aW9uIDUuOSBkZXNjcmliZXMgaG93 IHRoZSBQQ04gYXJjaGl0ZWN0dXJlIGhhcyBiZWVuIGRlc2lnbmVkIHRvDQogICBlbnN1cmUgYWRt aXR0ZWQgZmxvd3MgY29udGludWUgZ3JhY2VmdWxseSBhZnRlciByZWNvdmVyaW5nDQogICBhdXRv bWF0aWNhbGx5IGZyb20gbGluayBvciBub2RlIGZhaWx1cmVzLiAgVGhlIG5lZWQgdG8gcmVjb3Jk IGFuZA0KICAgbW9uaXRvciByZS1yb3V0aW5nIGV2ZW50cyBhZmZlY3Rpbmcgc2lnbmFsbGluZyBp cyB1bmNoYW5nZWQgYnkgdGhlDQogICBhZGRpdGlvbiBvZiBQQ04gdG8gYSBEaWZmU2VydiBkb21h aW4uICBTaW1pbGFybHksIHJlLXJvdXRpbmcgZXZlbnRzDQogICB3aXRoaW4gdGhlIFBDTi1kb21h aW4gd2lsbCBiZSByZWNvcmRlZCBhbmQgbW9uaXRvcmVkIGp1c3QgYXMgdGhleQ0KICAgd291bGQg YmUgd2l0aG91dCBQQ04uDQoNCiAgIFBDTi1tYXJraW5nIGRvZXMgbWFrZSBpdCBwb3NzaWJsZSB0 byByZWNvcmQgJ25lYXItbWlzc2VzJy4gIEZvcg0KICAgaW5zdGFuY2UsIGF0IHRoZSBQQ04tZWdy ZXNzLW5vZGUgYSAncmVwb3J0aW5nIHRocmVzaG9sZCcgY291bGQgYmUgc2V0DQogICB0byBtb25p dG9yIGhvdyBvZnRlbiB0aGUgc3lzdGVtIGNvbWVzIGNsb3NlIHRvIHRyaWdnZXJpbmcgZmxvdw0K ICAgYmxvY2tpbmcgd2l0aG91dCBhY3R1YWxseSBkb2luZyBzby4gIFNpbWlsYXJseSwgYnVyc3Rz IG9mIGZsb3cNCiAgIHRlcm1pbmF0aW9uIG1hcmtpbmcgY291bGQgYmUgcmVjb3JkZWQgZXZlbiBp ZiB0aGV5IGFyZSBub3QNCiAgIHN1ZmZpY2llbnRseSBzdXN0YWluZWQgdG8gdHJpZ2dlciBmbG93 IHRlcm1pbmF0aW9uLiAgU3VjaCBzdGF0aXN0aWNzDQogICBjb3VsZCBiZSBjb3JyZWxhdGVkIHdp dGggcGVyLXF1ZXVlIGNvdW50cyBvZiBtYXJraW5nIHZvbHVtZSAoU2VjdGlvbg0KICAgOC4yKSB0 byB1cGdyYWRlIHJlc291cmNlcyBpbiBkYW5nZXIgb2YgY2F1c2luZyBzZXJ2aWNlIGRlZ3JhZGF0 aW9uLA0KICAgb3IgdG8gdHJpZ2dlciBtYW51YWwgdHJhY2luZyBvZiBpbnRlcm1pdHRlbnQgaW5j aXBpZW50IGVycm9ycyB0aGF0DQogICB3b3VsZCBvdGhlcndpc2UgaGF2ZSBnb25lIHVubm90aWNl ZC4NCg0KICAgRmluYWxseSwgb2YgY291cnNlLCBtYW55IGZhdWx0cyBhcmUgY2F1c2VkIGJ5IGZh aWxpbmdzIGluIHRoZQ0KICAgbWFuYWdlbWVudCBwcm9jZXNzICgnaHVtYW4gZXJyb3InKTogYSB3 cm9uZ2x5IGNvbmZpZ3VyZWQgYWRkcmVzcyBpbiBhDQogICBub2RlLCBhIHdyb25nIGFkZHJlc3Mg Z2l2ZW4gaW4gYSBzaWduYWxsaW5nIHByb3RvY29sLCBhIHdyb25nbHkNCiAgIGNvbmZpZ3VyZWQg cGFyYW1ldGVyIGluIGEgcXVldWVpbmcgYWxnb3JpdGhtLCBhIG5vZGUgc2V0IGludG8gYQ0KICAg ZGlmZmVyZW50IG1vZGUgZnJvbSBvdGhlciBub2RlcywgYW5kIHNvIG9uLiAgR2VuZXJhbGx5LCBh IGNsZWFuDQogICBkZXNpZ24gd2l0aCBmZXcgY29uZmlndXJhYmxlIG9wdGlvbnMgZW5zdXJlcyB0 aGlzIGNsYXNzIG9mIGZhdWx0cyBjYW4NCg0KDQoNCkVhcmRsZXkgKEVkaXRvcikgICAgICAgICAg RXhwaXJlcyBNYXkgMjIsIDIwMDggICAgICAgICAgICAgICAgIFtQYWdlIDM0XQ0KDA0KSW50ZXJu ZXQtRHJhZnQgICAgICAgICAgICAgICAgICBEb2N1bWVudCAgICAgICAgICAgICAgICAgICBOb3Zl bWJlciAyMDA3DQoNCg0KICAgYmUgdHJhY2VkIG1vcmUgZWFzaWx5IGFuZCBwcmV2ZW50ZWQgbW9y ZSBvZnRlbi4gIFNvdW5kIG1hbmFnZW1lbnQNCiAgIHByYWN0aWNlIGF0IHJ1bi10aW1lIGFsc28g aGVscHMuICBGb3IgaW5zdGFuY2U6IGEgbWFuYWdlbWVudCBzeXN0ZW0NCiAgIHNob3VsZCBiZSB1 c2VkIHRoYXQgY29uc3RyYWlucyBjb25maWd1cmF0aW9uIGNoYW5nZXMgd2l0aGluIHN5c3RlbQ0K ICAgcnVsZXMgKGVnIHByZXZlbnRpbmcgYW4gb3B0aW9uIHNldHRpbmcgaW5jb25zaXN0ZW50IHdp dGggb3RoZXINCiAgIG5vZGVzKTsgY29uZmlndXJhdGlvbiBvcHRpb25zIHNob3VsZCBhbHNvIGJl IHJlY29yZGVkIGluIGFuIG9mZmxpbmUNCiAgIGRhdGFiYXNlOyBhbmQgcmVndWxhciBhdXRvbWF0 aWMgY29uc2lzdGVuY3kgY2hlY2tzIGJldHdlZW4gbGl2ZQ0KICAgc3lzdGVtcyBhbmQgdGhlIGRh dGFiYXNlLiAgUENOIGFkZHMgbm90aGluZyBzcGVjaWZpYyB0byB0aGlzIGNsYXNzIG9mDQogICBw cm9ibGVtcy4gIEJ5IHRoZSB0aW1lIHN0YW5kYXJkcyBhcmUgaW4gcGxhY2UsIHdlIGV4cGVjdCB0 aGF0IHRoZSBQQ04NCiAgIFdHIHdpbGwgaGF2ZSBydXRobGVzc2x5IHJlbW92ZWQgZ3JhdHVpdG91 cyBjb25maWd1cmF0aW9uIGNob2ljZXMuDQogICBIb3dldmVyLCBhdCB0aGUgdGltZSBvZiB3cml0 aW5nLCB0aGUgV0cgaXMgeWV0IHRvIGNob29zZSBiZXR3ZWVuDQogICBtdWx0aXBsZSBjb21wZXRp bmcgcHJvcG9zYWxzLCBzbyB0aGUgcmFuZ2Ugb2YgcG9zc2libGUgb3B0aW9ucyBpbg0KICAgU2Vj dGlvbiA4LjEgZG9lcyBzZWVtIHJhdGhlciB3aWRlIGNvbXBhcmVkIHRvIHRoZSBvcmlnaW5hbCBu ZWFyLXplcm8NCiAgIGNvbmZpZ3VyYXRpb24gaW50ZW50IG9mIHRoZSBhcmNoaXRlY3R1cmUuDQoN CjguNS4gIFNlY3VyaXR5IE9BTQ0KDQogICBTZWN1cml0eSBPQU0gaXMgYWJvdXQgdXNpbmcgc2Vj dXJlIG9wZXJhdGlvbmFsIHByYWN0aWNlcyBhcyB3ZWxsIGFzDQogICBiZWluZyBhYmxlIHRvIHRy YWNrIHNlY3VyaXR5IGJyZWFjaGVzIG9yIG5lYXItbWlzc2VzIGF0IHJ1bi10aW1lLg0KICAgUENO IGFkZHMgZmV3IHNwZWNpZmljcyB0byB0aGUgZ2VuZXJhbCBnb29kIHByYWN0aWNlIHJlcXVpcmVk IGluIHRoaXMNCiAgIGZpZWxkIFtSRkM0Nzc4XSwgb3RoZXIgdGhhbiB0aG9zZSBiZWxvdy4gIFRo ZSBjb3JyZWN0IGZ1bmN0aW9ucyBvZg0KICAgdGhlIHN5c3RlbSBzaG91bGQgYmUgbW9uaXRvcmVk IChTZWN0aW9uIDguMikgaW4gbXVsdGlwbGUgaW5kZXBlbmRlbnQNCiAgIHdheXMgYW5kIGNvcnJl bGF0ZWQgdG8gZGV0ZWN0IHBvc3NpYmxlIHNlY3VyaXR5IGJyZWFjaGVzLiAgUGVyc2lzdGVudA0K ICAgKHByZS0pY29uZ2VzdGlvbiBtYXJraW5nIHNob3VsZCByYWlzZSBhbiBhbGFybSAoYm90aCBv biB0aGUgbm9kZQ0KICAgZG9pbmcgdGhlIG1hcmtpbmcgYW5kIG9uIHRoZSBQQ04tZWdyZXNzLW5v ZGUgbWV0ZXJpbmcgaXQpLg0KICAgU2ltaWxhcmx5LCBwZXJzaXN0ZW50bHkgcG9vciBleHRlcm5h bCBRb1MgbWV0cmljcyBzdWNoIGFzIGppdHRlciBvcg0KICAgTU9TIHNob3VsZCByYWlzZSBhbiBh bGFybS4gIFRoZSBmb2xsb3dpbmcgYXJlIGV4YW1wbGVzIG9mIHN5bXB0b21zDQogICB0aGF0IG1h eSBiZSB0aGUgcmVzdWx0IG9mIGlubm9jZW50IGZhdWx0cywgcmF0aGVyIHRoYW4gYXR0YWNrcywg YnV0DQogICB1bnRpbCBkaWFnbm9zZWQgdGhleSBzaG91bGQgYmUgbG9nZ2VkIGFuZCB0cmlnZ2Vy IGEgc2VjdXJpdHkgYWxhcm06DQoNCiAgIG8gIEFub21hbG91cyBwYXR0ZXJucyBvZiBub24tY29u Zm9ybWluZyBpbmNvbWluZyBzaWduYWxzIGFuZCBwYWNrZXRzDQogICAgICByZWplY3RlZCBhdCB0 aGUgUENOLWluZ3Jlc3Mtbm9kZXMgKGVnIHBhY2tldHMgYWxyZWFkeSBtYXJrZWQgUENOLQ0KICAg ICAgY2FwYWJsZSwgb3IgdHJhZmZpYyBwZXJzaXN0ZW50bHkgc3RhcnZpbmcgdG9rZW4gYnVja2V0 IHBvbGljZXJzKS4NCg0KICAgbyAgUENOLWNhcGFibGUgcGFja2V0cyBhcnJpdmluZyBhdCBhIFBD Ti1lZ3Jlc3Mtbm9kZSB3aXRoIG5vDQogICAgICBhc3NvY2lhdGVkIHN0YXRlIGZvciBtYXBwaW5n IHRoZW0gdG8gYSB2YWxpZCBpbmdyZXNzLWVncmVzcy0NCiAgICAgIGFnZ3JlZ2F0ZS4NCg0KICAg byAgQSBQQ04taW5ncmVzcy1ub2RlIHJlY2VpdmluZyBmZWVkYmFjayBzaWduYWxzIGFib3V0IHRo ZSBwcmUtDQogICAgICBjb25nZXN0aW9uIGxldmVsIG9uIGEgbm9uLWV4aXN0ZW50IGFnZ3JlZ2F0 ZSwgb3IgdGhhdCBhcmUNCiAgICAgIGluY29uc2lzdGVudCB3aXRoIG90aGVyIHNpZ25hbHMgKGVn IHVuZXhwZWN0ZWQgc2VxdWVuY2UgbnVtYmVycywNCiAgICAgIGluY29uc2lzdGVudCBhZGRyZXNz aW5nLCBjb25mbGljdGluZyByZXBvcnRzIG9mIHRoZSBwcmUtY29uZ2VzdGlvbg0KICAgICAgbGV2 ZWwsIGV0YykuDQoNCiAgIG8gIFByZS1jb25nZXN0aW9uIG1hcmtpbmcgYXJyaXZpbmcgYXQgYW4g UENOLWVncmVzcy1ub2RlIHdpdGgNCiAgICAgIChwcmUtKWNvbmdlc3Rpb24gbWFya2luZ3MgZm9j dXNlZCBvbiBwYXJ0aWN1bGFyIGZsb3dzLCByYXRoZXIgdGhhbg0KICAgICAgcmFuZG9tbHkgZGlz dHJpYnV0ZWQgdGhyb3VnaG91dCB0aGUgYWdncmVnYXRlLg0KDQoNCg0KDQoNCkVhcmRsZXkgKEVk aXRvcikgICAgICAgICAgRXhwaXJlcyBNYXkgMjIsIDIwMDggICAgICAgICAgICAgICAgIFtQYWdl IDM1XQ0KDA0KSW50ZXJuZXQtRHJhZnQgICAgICAgICAgICAgICAgICBEb2N1bWVudCAgICAgICAg ICAgICAgICAgICBOb3ZlbWJlciAyMDA3DQoNCg0KOS4gIElBTkEgQ29uc2lkZXJhdGlvbnMNCg0K ICAgVGhpcyBtZW1vIGluY2x1ZGVzIG5vIHJlcXVlc3QgdG8gSUFOQS4NCg0KDQoxMC4gIFNlY3Vy aXR5IGNvbnNpZGVyYXRpb25zDQoNCiAgIFNlY3VyaXR5IGNvbnNpZGVyYXRpb25zIGVzc2VudGlh bGx5IGNvbWUgZnJvbSB0aGUgVHJ1c3QgQXNzdW1wdGlvbg0KICAgKFNlY3Rpb24gMy4xKSwgaWUg dGhhdCBhbGwgUENOLW5vZGVzIGFyZSBQQ04tZW5hYmxlZCBhbmQgdHJ1c3QgZWFjaA0KICAgb3Ro ZXIgZm9yIHRydXRoZnVsIFBDTi1tYXJraW5nIGFuZCB0cmFuc3BvcnQuICBQQ04gc3BsaXRzDQog ICBmdW5jdGlvbmFsaXR5IGJldHdlZW4gUENOLWludGVyaW9yLW5vZGVzIGFuZCBQQ04tYm91bmRh cnktbm9kZXMsIGFuZA0KICAgdGhlIHNlY3VyaXR5IGNvbnNpZGVyYXRpb25zIGFyZSBzb21ld2hh dCBkaWZmZXJlbnQgZm9yIGVhY2gsIG1haW5seQ0KICAgYmVjYXVzZSBQQ04tYm91bmRhcnktbm9k ZXMgYXJlIGZsb3ctYXdhcmUgYW5kIFBDTi1pbnRlcmlvci1ub2RlcyBhcmUNCiAgIG5vdC4NCg0K ICAgbyAgYmVjYXVzZSB0aGUgUENOLWJvdW5kYXJ5LW5vZGVzIGFyZSBmbG93LWF3YXJlLCB0aGV5 IGFyZSB0cnVzdGVkIHRvDQogICAgICB1c2UgdGhhdCBhd2FyZW5lc3MgY29ycmVjdGx5LiAgVGhl IGRlZ3JlZSBvZiB0cnVzdCByZXF1aXJlZA0KICAgICAgZGVwZW5kcyBvbiB0aGUga2luZHMgb2Yg ZGVjaXNpb25zIHRoZXkgaGF2ZSB0byBtYWtlIGFuZCB0aGUga2luZHMNCiAgICAgIG9mIGluZm9y bWF0aW9uIHRoZXkgbmVlZCB0byBtYWtlIHRoZW0uICBGb3IgZXhhbXBsZSB3aGVuIHRoZSBQQ04t DQogICAgICBib3VuZGFyeS1ub2RlIG5lZWRzIHRvIGtub3cgdGhlIGNvbnRlbnRzIG9mIHRoZSBz ZXNzaW9ucyBmb3INCiAgICAgIG1ha2luZyB0aGUgYWRtaXNzaW9uIGFuZCB0ZXJtaW5hdGlvbiBk ZWNpc2lvbnMgKHBlcmhhcHMgYmFzZWQgb24NCiAgICAgIHRoZSBNTFBQIHByZWNlZGVuY2UpLCBv ciB3aGVuIHRoZSBjb250ZW50cyBhcmUgaGlnaGx5IGNsYXNzaWZpZWQsDQogICAgICB0aGVuIHRo ZSBzZWN1cml0eSByZXF1aXJlbWVudHMgZm9yIHRoZSBQQ04tYm91bmRhcnktbm9kZXMgaW52b2x2 ZWQNCiAgICAgIHdpbGwgYWxzbyBuZWVkIHRvIGJlIGhpZ2guDQoNCiAgIG8gIHRoZSBQQ04taW5n cmVzcy1ub2RlcyBwb2xpY2UgcGFja2V0cyB0byBlbnN1cmUgYSBmbG93IHN0aWNrcw0KICAgICAg d2l0aGluIGl0cyBhZ3JlZWQgbGltaXQsIGFuZCB0byBlbnN1cmUgdGhhdCBvbmx5IGZsb3dzIHdo aWNoIGhhdmUNCiAgICAgIGJlZW4gYWRtaXR0ZWQgY29udHJpYnV0ZSBQQ04tdHJhZmZpYyBpbnRv IHRoZSBQQ04tZG9tYWluLiAgVGhlDQogICAgICBwb2xpY2VyIG11c3QgZHJvcCAob3IgcGVyaGFw cyByZS1tYXJrIHRvIGEgZGlmZmVyZW50IERTQ1ApIGFueQ0KICAgICAgUENOLXBhY2tldHMgcmVj ZWl2ZWQgdGhhdCBhcmUgb3V0c2lkZSB0aGlzIHJlbWl0LiAgVGhpcyBpcyBzaW1pbGFyDQogICAg ICB0byB0aGUgZXhpc3RpbmcgSW50U2VydiBiZWhhdmlvdXIuICBCZXR3ZWVuIHRoZW0gdGhlIFBD Ti1ib3VuZGFyeS0NCiAgICAgIG5vZGVzIG11c3QgZW5jaXJjbGUgdGhlIFBDTi1kb21haW4sIG90 aGVyd2lzZSBQQ04tcGFja2V0cyBjb3VsZA0KICAgICAgZW50ZXIgdGhlIFBDTi1kb21haW4gd2l0 aG91dCBiZWluZyBzdWJqZWN0IHRvIGFkbWlzc2lvbiBjb250cm9sLA0KICAgICAgd2hpY2ggd291 bGQgcG90ZW50aWFsbHkgZGVzdHJveSB0aGUgUW9TIG9mIGV4aXN0aW5nIGZsb3dzLg0KDQogICBv ICBQQ04taW50ZXJpb3Itbm9kZXMgYXJlbid0IGZsb3ctYXdhcmUuICBUaGlzIHByZXZlbnRzIHNv bWUgc2VjdXJpdHkNCiAgICAgIGF0dGFja3Mgd2hlcmUgYW4gYXR0YWNrZXIgdGFyZ2V0cyBzcGVj aWZpYyBmbG93cyBpbiB0aGUgZGF0YSBwbGFuZQ0KICAgICAgLSBmb3IgaW5zdGFuY2UgZm9yIERv UyBvciBlYXZlc2Ryb3BwaW5nLg0KDQogICBvICBQQ04tbWFya2luZyBieSB0aGUgUENOLWludGVy aW9yLW5vZGVzIGFsb25nIHRoZSBwYWNrZXQgZm9yd2FyZGluZw0KICAgICAgcGF0aCBuZWVkcyB0 byBiZSB0cnVzdGVkLCBiZWNhdXNlIHRoZSBQQ04tYm91bmRhcnktbm9kZXMgcmVseSBvbg0KICAg ICAgdGhpcyBpbmZvcm1hdGlvbi4gIEZvciBpbnN0YW5jZSBhIHJvZ3VlIFBDTi1pbnRlcmlvci1u b2RlIGNvdWxkDQogICAgICBQQ04tbWFyayBhbGwgcGFja2V0cyBzbyB0aGF0IG5vIGZsb3dzIHdl cmUgYWRtaXR0ZWQuICBBbm90aGVyDQogICAgICBwb3NzaWJpbGl0eSBpcyB0aGF0IGl0IGRvZXNu J3QgUENOLW1hcmsgYW55IHBhY2tldHMsIGV2ZW4gd2hlbg0KICAgICAgaXQncyBwcmUtY29uZ2Vz dGVkLiAgTW9yZSBzdWJ0bHksIHRoZSByb2d1ZSBQQ04taW50ZXJpb3Itbm9kZQ0KICAgICAgY291 bGQgcGVyZm9ybSB0aGVzZSBhdHRhY2tzIHNlbGVjdGl2ZWx5IG9uIHBhcnRpY3VsYXIgZmxvd3Ms IG9yIGl0DQogICAgICBjb3VsZCBQQ04tbWFyayB0aGUgY29ycmVjdCBmcmFjdGlvbiBvdmVyYWxs LCBidXQgY2FyZWZ1bGx5IGNob29zZQ0KICAgICAgd2hpY2ggZmxvd3MgaXQgbWFya2VkLg0KDQoN Cg0KRWFyZGxleSAoRWRpdG9yKSAgICAgICAgICBFeHBpcmVzIE1heSAyMiwgMjAwOCAgICAgICAg ICAgICAgICAgW1BhZ2UgMzZdDQoMDQpJbnRlcm5ldC1EcmFmdCAgICAgICAgICAgICAgICAgIERv Y3VtZW50ICAgICAgICAgICAgICAgICAgIE5vdmVtYmVyIDIwMDcNCg0KDQogICBvICB0aGUgUENO LWJvdW5kYXJ5LW5vZGVzIHNob3VsZCBiZSBhYmxlIHRvIGRlYWwgd2l0aCBEb1MgYXR0YWNrcyBh bmQNCiAgICAgIHN0YXRlIGV4aGF1c3Rpb24gYXR0YWNrcyBiYXNlZCBvbiBmYXN0IGNoYW5nZXMg aW4gcGVyIGZsb3cNCiAgICAgIHNpZ25hbGxpbmcuDQoNCiAgIG8gIHRoZSBzaWduYWxsaW5nIGJl dHdlZW4gdGhlIFBDTi1ib3VuZGFyeS1ub2RlcyAoYW5kIHBvc3NpYmx5IGENCiAgICAgIGNlbnRy YWwgY29udHJvbCBub2RlKSBtdXN0IGJlIHByb3RlY3RlZCBmcm9tIGF0dGFja3MuICBGb3IgZXhh bXBsZQ0KICAgICAgdGhlIHJlY2lwaWVudCBuZWVkcyB0byB2YWxpZGF0ZSB0aGF0IHRoZSBtZXNz YWdlIGlzIGluZGVlZCBmcm9tDQogICAgICB0aGUgbm9kZSB0aGF0IGNsYWltcyB0byBoYXZlIHNl bnQgaXQuICBQb3NzaWJsZSBtZWFzdXJlcyBpbmNsdWRlDQogICAgICBkaWdlc3QgYXV0aGVudGlj YXRpb24gYW5kIHByb3RlY3Rpb24gYWdhaW5zdCByZXBsYXkgYW5kIG1hbi1pbi0NCiAgICAgIHRo ZS1taWRkbGUgYXR0YWNrcy4gIEZvciB0aGUgc3BlY2lmaWMgcHJvdG9jb2wgUlNWUCwgaG9wLWJ5 LWhvcA0KICAgICAgYXV0aGVudGljYXRpb24gaXMgaW4gW1JGQzI3NDddLCBhbmQNCiAgICAgIFtJ LUQuYmVocmluZ2VyLXRzdndnLXJzdnAtc2VjdXJpdHktZ3JvdXBrZXlpbmddIG1heSBhbHNvIGJl DQogICAgICB1c2VmdWw7IGZvciBhIGdlbmVyaWMgc2lnbmFsbGluZyBwcm90b2NvbCB0aGUgUENO IFdHIGRvY3VtZW50IG9uDQogICAgICAiUmVxdWlyZW1lbnRzIGZvciBzaWduYWxsaW5nIiB3aWxs IGRlc2NyaWJlIHRoZSByZXF1aXJlbWVudHMgaW4NCiAgICAgIG1vcmUgZGV0YWlsLg0KDQogICBP cGVyYXRpb25hbCBzZWN1cml0eSBhZHZpY2UgaXMgZ2l2ZW4gaW4gU2VjdGlvbiA4LjUuDQoNCg0K MTEuICBDb25jbHVzaW9ucw0KDQogICBUaGUgZG9jdW1lbnQgZGVzY3JpYmVzIGEgZ2VuZXJhbCBh cmNoaXRlY3R1cmUgZm9yIGZsb3cgYWRtaXNzaW9uIGFuZA0KICAgdGVybWluYXRpb24gYmFzZWQg b24gYWdncmVnYXRlZCBwcmUtY29uZ2VzdGlvbiBpbmZvcm1hdGlvbiBpbiBvcmRlcg0KICAgdG8g cHJvdGVjdCB0aGUgcXVhbGl0eSBvZiBzZXJ2aWNlIG9mIGVzdGFibGlzaGVkIGluZWxhc3RpYyBm bG93cw0KICAgd2l0aGluIGEgc2luZ2xlIERpZmZTZXJ2IGRvbWFpbi4gIFRoZSBtYWluIHRvcGlj IGlzIHRoZSBmdW5jdGlvbmFsDQogICBhcmNoaXRlY3R1cmUgKGZpcnN0IGNvdmVyZWQgYXQgYSBo aWdoIGxldmVsIGFuZCB0aGVuIGF0IGEgZ3JlYXRlcg0KICAgbGV2ZWwgb2YgZGV0YWlsKS4gIEl0 IGFsc28gbWVudGlvbnMgb3RoZXIgdG9waWNzIGxpa2UgdGhlIGFzc3VtcHRpb25zDQogICBhbmQg b3BlbiBpc3N1ZXMuDQoNCg0KMTIuICBBY2tub3dsZWRnZW1lbnRzDQoNCiAgIFRoaXMgZG9jdW1l bnQgaXMgYSByZXZpc2VkIHZlcnNpb24gb2YgW0ktRC5lYXJkbGV5LXBjbi1hcmNoaXRlY3R1cmVd Lg0KICAgSXRzIGF1dGhvcnMgd2VyZTogUC4gRWFyZGxleSwgSi4gQmFiaWFyeiwgSy4gQ2hhbiwg QS4gQ2hhcm55LCBSLg0KICAgR2VpYiwgRy4gS2FyYWdpYW5uaXMsIE0uIE1lbnRoLCBULiBUc291 LiAgVGhleSBhcmUgdGhlcmVmb3JlDQogICBjb250cmlidXRvcnMgdG8gdGhpcyBkb2N1bWVudC4N Cg0KICAgVGhhbmtzIHRvIHRob3NlIHdobyd2ZSBtYWRlIGNvbW1lbnRzIG9uDQogICBbSS1ELmVh cmRsZXktcGNuLWFyY2hpdGVjdHVyZV0gYW5kIG9uIGVhcmxpZXIgdmVyc2lvbnMgb2YgdGhpcyBk cmFmdDoNCiAgIExhY2hsYW4gQW5kcmV3LCBKb2UgQmFiaWFyeiwgRnJlZCBCYWtlciwgRGF2aWQg QmxhY2ssIFN0ZXZlbiBCbGFrZSwNCiAgIEJvYiBCcmlzY29lLCBLZW4gQ2FybGJlcmcsIEFubmEg Q2hhcm55LCBKb2FjaGltIENoYXJ6aW5za2ksIEFuZHJhcw0KICAgQ3Nhc3phciwgTGFycyBFZ2dl cnQsIFJ1ZWRpZ2VyIEdlaWIsIFJvYmVydCBIYW5jb2NrLCBHZW9yZ2lvcw0KICAgS2FyYWdpYW5u aXMsIE1pY2hhZWwgTWVudGgsIFRvbSBUYXlsb3IsIFRpbmEgVHNvdSwgRGVsZWkgWXUuICBUaGFu a3MNCiAgIHRvIEJvYiBCcmlzY29lIHdobyBleHRlbnNpdmVseSByZXZpc2VkIHRoZSBPcGVyYXRp b25zIGFuZCBNYW5hZ2VtZW50DQogICBzZWN0aW9uLg0KDQogICBUaGlzIGRvY3VtZW50IGlzIHRo ZSByZXN1bHQgb2YgZGlzY3Vzc2lvbnMgaW4gdGhlIFBDTiBXRyBhbmQNCiAgIGZvcmVydW5uZXIg YWN0aXZpdHkgaW4gdGhlIFRTVldHLiAgQSBudW1iZXIgb2YgcHJldmlvdXMgZHJhZnRzIHdlcmUN Cg0KDQoNCkVhcmRsZXkgKEVkaXRvcikgICAgICAgICAgRXhwaXJlcyBNYXkgMjIsIDIwMDggICAg ICAgICAgICAgICAgIFtQYWdlIDM3XQ0KDA0KSW50ZXJuZXQtRHJhZnQgICAgICAgICAgICAgICAg ICBEb2N1bWVudCAgICAgICAgICAgICAgICAgICBOb3ZlbWJlciAyMDA3DQoNCg0KICAgcHJlc2Vu dGVkIHRvIFRTVldHOiBbSS1ELmNoYW4tcGNuLXByb2JsZW0tc3RhdGVtZW50XSwNCiAgIFtJLUQu YnJpc2NvZS10c3Z3Zy1jbC1hcmNoaXRlY3R1cmVdLCBbSS1ELmJyaXNjb2UtdHN2d2ctY2wtcGhi XSwNCiAgIFtJLUQuY2hhcm55LXBjbi1zaW5nbGUtbWFya2luZ10sIFtJLUQuYmFiaWFyei1wY24t c2lwLWNhcF0sDQogICBbSS1ELmxlZmF1Y2hldXItcnN2cC1lY25dLCBbSS1ELndlc3RiZXJnLXBj bi1sb2FkLWNvbnRyb2xdLiAgVGhlDQogICBhdXRob3JzIG9mIHRoZW0gd2VyZTogQiwgQnJpc2Nv ZSwgUC4gRWFyZGxleSwgRC4gU29uZ2h1cnN0LCBGLiBMZQ0KICAgRmF1Y2hldXIsIEEuIENoYXJu eSwgSi4gQmFiaWFyeiwgSy4gQ2hhbiwgUy4gRHVkbGV5LCBHLiBLYXJhZ2lhbm5pcywNCiAgIEEu IEJhZGVyLCBMLiBXZXN0YmVyZywgSi4gWmhhbmcsIFYuIExpYXRzb3MsIFgtRy4gIExpdSwgQS4g QmhhcmdhdmEuDQoNCg0KMTMuICBDb21tZW50cyBTb2xpY2l0ZWQNCg0KICAgQ29tbWVudHMgYW5k IHF1ZXN0aW9ucyBhcmUgZW5jb3VyYWdlZCBhbmQgdmVyeSB3ZWxjb21lLiAgVGhleSBjYW4gYmUN CiAgIGFkZHJlc3NlZCB0byB0aGUgSUVURiBQQ04gd29ya2luZyBncm91cCBtYWlsaW5nIGxpc3Qg PHBjbkBpZXRmLm9yZz4uDQoNCg0KMTQuICBDaGFuZ2VzDQoNCiAgIENoYW5nZXMgZnJvbSAtMDEg dG8gLTAyOg0KDQogICBvICBTMTogQmVuZWZpdHM6IHByb3Zpc2lvbmluZyBidWxsZXQgZXh0ZW5k ZWQgdG8gc3RyZXNzIHRoYXQgUENOIGRvZXMNCiAgICAgIG5vdCB1c2UgUkZDMjQ3NS1zdHlsZSB0 cmFmZmljIGNvbmRpdGlvbmluZy4NCg0KICAgbyAgUzE6IERlcGxveW1lbnQgbW9kZWxzOiBtZW50 aW9uZWQsIGFzIHZhcmlhbnQgb2YgUENOLWRvbWFpbg0KICAgICAgZXh0ZW5kaW5nIHRvIGVuZCBu b2RlcywgdGhhdCBtYXkgZXh0ZW5kIHRvIExBTiBlZGdlIHN3aXRjaC4NCg0KICAgbyAgUzMuMTog VHJ1c3QgQXNzdW1wdGlvbjogYWRkZWQgbm90ZSBhYm91dCBub3QgbmVlZGluZyBQQ04tbWFya2lu Zw0KICAgICAgY2FwYWJpbGl0eSBpZiBrbm93biB0aGF0IGFuIGludGVyZmFjZSBjYW5ub3QgYmVj b21lIHByZS1jb25nZXN0ZWQuDQoNCiAgIG8gIFM0OiBub3cgZGl2aWRlZCBpbnRvIHN1Yi1zZWN0 aW9ucw0KDQogICBvICBTNC4xOiBBZG1pc3Npb24gY29udHJvbDogYWRkZWQgc2Vjb25kIHByb3Bv c2VkIG1ldGhvZCBmb3IgaG93IHRvDQogICAgICBkZWNpZGUgdG8gYmxvY2sgbmV3IGZsb3dzIChQ Q04tZWdyZXNzLW5vZGUgcmVjZWl2ZXMgb25lIChvcg0KICAgICAgc2V2ZXJhbCkgUENOLW1hcmtl ZCBwYWNrZXRzKS4NCg0KICAgbyAgUzU6IFByb2Jpbmcgc3ViLXNlY3Rpb24gcmVtb3ZlZC4gIE1h dGVyaWFsIG5vdyBpbiBuZXcgUzcuDQoNCiAgIG8gIFM1LjY6IEFkZHJlc3Npbmc6IGNsYXJpZmll ZCBob3cgUENOLWluZ3Jlc3Mtbm9kZSBjYW4gZGlzY292ZXINCiAgICAgIGFkZHJlc3Mgb2YgUENO LWVncmVzcy1ub2RlDQoNCiAgIG8gIFM1LjY6IEFkZHJlc3Npbmc6IGNlbnRyYWxpc2VkIG5vZGUg Y2FzZSwgYWRkZWQgdGhhdCBQQ04taW5ncmVzcy0NCiAgICAgIG5vZGUgbWF5IG5lZWQgdG8ga25v dyBhZGRyZXNzIG9mIFBDTi1lZ3Jlc3Mtbm9kZQ0KDQogICBvICBTNS44OiBUdW5uZWxsaW5nOiBh ZGRlZCBjYXNlIG9mICJwYXJ0aWFsbHkgUENOLWNhcGFibGUgdHVubmVsIiBhbmQNCiAgICAgIGRl Z3JhZGVkIGJ1bGxldCBvbiB0aGlzIGluIFM2IChPcGVuIElzc3VlcykNCg0KICAgbyAgUzc6IFBy b2Jpbmc6IG5ldyBzZWN0aW9uLiAgTXVjaCBtb3JlIGNvbXByZWhlbnNpdmUgdGhhbiBvbGQgUzUu NS4NCg0KDQoNCg0KDQpFYXJkbGV5IChFZGl0b3IpICAgICAgICAgIEV4cGlyZXMgTWF5IDIyLCAy MDA4ICAgICAgICAgICAgICAgICBbUGFnZSAzOF0NCgwNCkludGVybmV0LURyYWZ0ICAgICAgICAg ICAgICAgICAgRG9jdW1lbnQgICAgICAgICAgICAgICAgICAgTm92ZW1iZXIgMjAwNw0KDQoNCiAg IG8gIFM4OiBPcGVyYXRpb25zIGFuZCBNYW5hZ2VtZW50OiBzdWJzdGFudGlhbGx5IHJldmlzZWQu DQoNCiAgIG8gIG90aGVyIG1pbm9yIGNoYW5nZXMgbm90IGFmZmVjdGluZyBzZW1hbnRpY3MNCg0K ICAgQ2hhbmdlcyBmcm9tIC0wMCB0byAtMDE6DQoNCiAgIEluIGFkZGl0aW9uIHRvIGNsYXJpZmlj YXRpb25zIGFuZCBuaXQgc3F1YXNoaW5nLCB0aGUgbWFpbiBjaGFuZ2VzDQogICBhcmU6DQoNCiAg IG8gIFMxOiBCZW5lZml0czogYWRkZWQgb25lIGFib3V0IHByb3Zpc2lvbmluZyAoYW5kIGNvbnRy YXN0IHdpdGgNCiAgICAgIERpZmZTZXJ2IFNMQXMpDQoNCiAgIG8gIFMxOiBCZW5lZml0czogY2xh cmlmaWVkIHRoYXQgdGhlIG9iamVjdGl2ZSBpcyBhbHNvIHRvIHN0b3AgUENOLQ0KICAgICAgcGFj a2V0cyBiZWluZyBzaWduaWZpY2FudGx5IGRlbGF5ZWQgKHByZXZpb3VzbHkgb25seSBtZW50aW9u ZWQgbm90DQogICAgICBkcm9wcGluZyBwYWNrZXRzKQ0KDQogICBvICBTMTogRGVwbG95bWVudCBt b2RlbHM6IGFkZGVkIG9uZSB3aGVyZSBwb2xpY2luZyBpcyBkb25lIGF0IGluZ3Jlc3MNCiAgICAg IG9mIGFjY2VzcyBuZXR3b3JrIGFuZCBub3QgYXQgaW5ncmVzcyBvZiBQQ04tZG9tYWluIChhc3N1 bWUgdHJ1c3QNCiAgICAgIGJldHdlZW4gbmV0d29ya3MpDQoNCiAgIG8gIFMxOiBEZXBsb3ltZW50 IG1vZGVsczogY29ycmVjdGVkIE1QTFMtVEUgdG8gTVBMUw0KDQogICBvICBTMjogVGVybWlub2xv Z3k6IGFkanVzdGVkIGRlZmluaXRpb24gb2YgUENOLWRvbWFpbg0KDQogICBvICBTMy41OiBPdGhl ciBhc3N1bXB0aW9uczogY29ycmVjdGVkLCBzbyB0aGF0IHR3byBhc3N1bXB0aW9ucyAoUENOLQ0K ICAgICAgbm9kZXMgbm90IHBlcmZvcm1pbmcgRUNOIGFuZCBQQ04taW5ncmVzcy1ub2RlIGRpc2Nh cmRpbmcgYXJyaXZpbmcNCiAgICAgIENFIHBhY2tldCkgb25seSBhcHBseSBpZiB0aGUgUENOIFdH IGRlY2lkZXMgdG8gZW5jb2RlIFBDTi1tYXJraW5nDQogICAgICBpbiB0aGUgRUNOLWZpZWxkLg0K DQogICBvICBTNCAmIFM1OiBjaGFuZ2VkIFBDTi1tYXJraW5nIGFsZ29yaXRobSB0byBtYXJraW5n IGJlaGF2aW91cg0KDQogICBvICBTNDogY2xhcmlmaWVkIHRoYXQgUENOLWludGVyaW9yLW5vZGUg ZnVuY3Rpb25hbGl0eSBhcHBsaWVzIGZvcg0KICAgICAgZWFjaCBvdXRnb2luZyBpbnRlcmZhY2Us IGFuZCBhZGRlZCBjbGFyaWZpY2F0aW9uOiAiVGhlDQogICAgICBmdW5jdGlvbmFsaXR5IGlzIGFs c28gZG9uZSBieSBQQ04taW5ncmVzcy1ub2RlcyBmb3IgdGhlaXIgb3V0Z29pbmcNCiAgICAgIGlu dGVyZmFjZXMgKGllIHRob3NlICdpbnNpZGUnIHRoZSBQQ04tZG9tYWluKS4iDQoNCiAgIG8gIFM0 IChuZWFyIGVuZCk6IGFsdGVyZWQgdG8gc2F5IHRoYXQgYSBQQ04tbm9kZSAic2hvdWxkIiBkZWRp Y2F0ZQ0KICAgICAgc29tZSBjYXBhY2l0eSB0byBsb3dlciBwcmlvcml0eSB0cmFmZmljIHNvIHRo YXQgaXQgaXNuJ3Qgc3RhcnZlZA0KICAgICAgKHdhcyAibWF5IikNCg0KICAgbyAgUzU6IGNsYXJp ZmllZCB0byBzYXkgdGhhdCBQQ04gZnVuY3Rpb25hbGl0eSBpcyBkb25lIG9uIGFuDQogICAgICAn aW50ZXJmYWNlJyAocmF0aGVyIHRoYW4gb24gYSAnbGluaycpDQoNCiAgIG8gIFM1LjI6IGRlbGV0 ZWQgZXJyb25lb3VzIG1lbnRpb24gb2Ygc2VydmljZSBsZXZlbCBhZ3JlZW1lbnQNCg0KICAgbyAg UzUuNTogUHJvYmluZzogcmUtd3JpdHRlbiwgZXNwZWNpYWxseSB0byBkaXN0aW5ndWlzaCBwcm9i aW5nIHRvDQogICAgICB0ZXN0IHRoZSBpbmdyZXNzLWVncmVzcy1hZ2dyZWdhdGUgZnJvbSBwcm9i aW5nIHRvIHRlc3QgYQ0KICAgICAgcGFydGljdWxhciBFQ01QIHBhdGguDQoNCg0KDQpFYXJkbGV5 IChFZGl0b3IpICAgICAgICAgIEV4cGlyZXMgTWF5IDIyLCAyMDA4ICAgICAgICAgICAgICAgICBb UGFnZSAzOV0NCgwNCkludGVybmV0LURyYWZ0ICAgICAgICAgICAgICAgICAgRG9jdW1lbnQgICAg ICAgICAgICAgICAgICAgTm92ZW1iZXIgMjAwNw0KDQoNCiAgIG8gIFM1Ljc6IEFkZHJlc3Npbmc6 IGFkZGVkIG1lbnRpb24gb2YgcHJvYmluZzsgYWRkZWQgdGhhdCBpbiB0aGUgY2FzZQ0KICAgICAg d2hlcmUgdHJhZmZpYyBpcyBhbHdheXMgdHVubmVsbGVkIGFjcm9zcyB0aGUgUENOLWRvbWFpbiwg YWRkIGENCiAgICAgIG5vdGUgdGhhdCBoZSBQQ04taW5ncmVzcy1ub2RlIG5lZWRzIHRvIGtub3cg dGhlIGFkZHJlc3Mgb2YgdGhlDQogICAgICBQQ04tZWdyZXNzLW5vZGUuDQoNCiAgIG8gIFM1Ljg6 IFR1bm5lbGxpbmc6IHJlLXdyaXR0ZW4sIGVzcGVjaWFsbHkgdG8gcHJvdmlkZSBhIGNsZWFyZXIN CiAgICAgIGRlc2NyaXB0aW9uIG9mIGNvcHlpbmcgb24gdHVubmVsIGVudHJ5L2V4aXQsIGJ5IGFk ZGluZyBleHBsYW5hdGlvbg0KICAgICAgKGtlZXBpbmcgdHVubmVsIGVuY2Fwcy9kZWNhcHMgYW5k IFBDTi1tYXJraW5nIG9ydGhvZ29uYWwpLA0KICAgICAgZGVsZXRpbmcgb25lIGJ1bGxldCAoImlm IHRoZSBpbm5lciBoZWFkZXIncyBtYXJraW5nIHN0YXRlIGlzIG1vcmUNCiAgICAgIHNldmVyIHRo ZW4gaXQgaXMgcHJlc2VydmVkIiAtIHNob3VsZG4ndCBoYXBwZW4pLCBhbmQgYmV0dGVyDQogICAg ICByZWZlcmVuY2luZyBvZiBvdGhlciBJRVRGIGRvY3VtZW50cy4NCg0KICAgbyAgUzY6IE9wZW4g aXNzdWVzOiBzdHJlc3NlZCB0aGF0ICJOT1RFOiBQb3RlbnRpYWwgc29sdXRpb25zIGFyZSBvdXQN CiAgICAgIG9mIHNjb3BlIGZvciB0aGlzIGRvY3VtZW50IiBhbmQgZWRpdGVkIGEgY291cGxlIG9m IHNlbnRlbmNlcyB0aGF0DQogICAgICB3ZXJlIGNsb3NlIHRvIHNvbHV0aW9uIHNwYWNlLg0KDQog ICBvICBTNjogT3BlbiBpc3N1ZXM6IGFkZGVkIG9uZSBhYm91dCBzY2VuYXJpb3Mgd2l0aCBvbmx5 IG9uZSB0dW5uZWwNCiAgICAgIGVuZHBvaW50IGluIHRoZSBQQ04gZG9tYWluIC4NCg0KICAgbyAg UzY6IE9wZW4gaXNzdWVzOiBFQ01QOiBhZGRlZCB1bmRlci1hZG1pc3Npb24gYXMgYW5vdGhlciBw b3RlbnRpYWwNCiAgICAgIHJpc2sNCg0KICAgbyAgUzY6IE9wZW4gaXNzdWVzOiBhZGRlZCBvbmUg YWJvdXQgIlNpbGVudCBhdCBzdGFydCINCg0KICAgbyAgUzEwOiBDb25jbHVzaW9uczogYSBzbWFs bCBjb25jbHVzaW9ucyBzZWN0aW9uIGFkZGVkLg0KDQoNCjE1LiAgSW5mb3JtYXRpdmUgUmVmZXJl bmNlcw0KDQogICBbSS1ELmJyaXNjb2UtdHN2d2ctY2wtYXJjaGl0ZWN0dXJlXQ0KICAgICAgICAg ICAgICBCcmlzY29lLCBCLiwgIkFuIGVkZ2UtdG8tZWRnZSBEZXBsb3ltZW50IE1vZGVsIGZvciBQ cmUtDQogICAgICAgICAgICAgIENvbmdlc3Rpb24gTm90aWZpY2F0aW9uOiBBZG1pc3Npb24gIENv bnRyb2wgb3ZlciBhDQogICAgICAgICAgICAgIERpZmZTZXJ2IFJlZ2lvbiIsIGRyYWZ0LWJyaXNj b2UtdHN2d2ctY2wtYXJjaGl0ZWN0dXJlLTA0DQogICAgICAgICAgICAgICh3b3JrIGluIHByb2dy ZXNzKSwgT2N0b2JlciAyMDA2Lg0KDQogICBbSS1ELmJyaXNjb2UtdHN2d2ctY2wtcGhiXQ0KICAg ICAgICAgICAgICBCcmlzY29lLCBCLiwgIlByZS1Db25nZXN0aW9uIE5vdGlmaWNhdGlvbiBtYXJr aW5nIiwNCiAgICAgICAgICAgICAgZHJhZnQtYnJpc2NvZS10c3Z3Zy1jbC1waGItMDMgKHdvcmsg aW4gcHJvZ3Jlc3MpLA0KICAgICAgICAgICAgICBPY3RvYmVyIDIwMDYuDQoNCiAgIFtJLUQuYmFi aWFyei1wY24tc2lwLWNhcF0NCiAgICAgICAgICAgICAgQmFiaWFyeiwgSi4sICJTSVAgQ29udHJv bGxlZCBBZG1pc3Npb24gYW5kIFByZWVtcHRpb24iLA0KICAgICAgICAgICAgICBkcmFmdC1iYWJp YXJ6LXBjbi1zaXAtY2FwLTAwICh3b3JrIGluIHByb2dyZXNzKSwNCiAgICAgICAgICAgICAgT2N0 b2JlciAyMDA2Lg0KDQogICBbSS1ELmxlZmF1Y2hldXItcnN2cC1lY25dDQogICAgICAgICAgICAg IEZhdWNoZXVyLCBGLiwgIlJTVlAgRXh0ZW5zaW9ucyBmb3IgQWRtaXNzaW9uIENvbnRyb2wgb3Zl cg0KICAgICAgICAgICAgICBEaWZmc2VydiB1c2luZyBQcmUtY29uZ2VzdGlvbiAgTm90aWZpY2F0 aW9uIChQQ04pIiwNCg0KDQoNCkVhcmRsZXkgKEVkaXRvcikgICAgICAgICAgRXhwaXJlcyBNYXkg MjIsIDIwMDggICAgICAgICAgICAgICAgIFtQYWdlIDQwXQ0KDA0KSW50ZXJuZXQtRHJhZnQgICAg ICAgICAgICAgICAgICBEb2N1bWVudCAgICAgICAgICAgICAgICAgICBOb3ZlbWJlciAyMDA3DQoN Cg0KICAgICAgICAgICAgICBkcmFmdC1sZWZhdWNoZXVyLXJzdnAtZWNuLTAxICh3b3JrIGluIHBy b2dyZXNzKSwNCiAgICAgICAgICAgICAgSnVuZSAyMDA2Lg0KDQogICBbSS1ELmNoYW4tcGNuLXBy b2JsZW0tc3RhdGVtZW50XQ0KICAgICAgICAgICAgICBDaGFuLCBLLiwgIlByZS1Db25nZXN0aW9u IE5vdGlmaWNhdGlvbiBQcm9ibGVtIFN0YXRlbWVudCIsDQogICAgICAgICAgICAgIGRyYWZ0LWNo YW4tcGNuLXByb2JsZW0tc3RhdGVtZW50LTAxICh3b3JrIGluIHByb2dyZXNzKSwNCiAgICAgICAg ICAgICAgT2N0b2JlciAyMDA2Lg0KDQogICBbSS1ELmlldGYtcHdlMy1jb25nZXN0aW9uLWZybXdr XQ0KICAgICAgICAgICAgICBCcnlhbnQsIFMuLCAiUHNldWRvd2lyZSBDb25nZXN0aW9uIENvbnRy b2wgRnJhbWV3b3JrIiwNCiAgICAgICAgICAgICAgZHJhZnQtaWV0Zi1wd2UzLWNvbmdlc3Rpb24t ZnJtd2stMDAgKHdvcmsgaW4gcHJvZ3Jlc3MpLA0KICAgICAgICAgICAgICBGZWJydWFyeSAyMDA3 Lg0KDQogICBbSS1ELmlldGYtdHN2d2ctYWRtaXR0ZWQtcmVhbHRpbWUtZHNjcF0NCiAgICAgICAg ICAgICAgIkRTQ1BzIGZvciBDYXBhY2l0eS1BZG1pdHRlZCBUcmFmZmljIiwgTm92ZW1iZXIgMjAw NiwgPGh0dA0KICAgICAgICAgICAgICBwOi8vd3d3LmlldGYub3JnL2ludGVybmV0LWRyYWZ0cy8N CiAgICAgICAgICAgICAgaWV0Zi10c3Z3Zy1hZG1pdHRlZC1yZWFsdGltZS1kc2NwLTAyLnR4dD4u DQoNCiAgIFtJLUQuYnJpc2NvZS10c3Z3Zy1lY24tdHVubmVsXQ0KICAgICAgICAgICAgICAiTGF5 ZXJlZCBFbmNhcHN1bGF0aW9uIG9mIENvbmdlc3Rpb24gTm90aWZpY2F0aW9uIiwNCiAgICAgICAg ICAgICAgSnVuZSAyMDA3LCA8aHR0cDovL3d3dy5pZXRmLm9yZy9pbnRlcm5ldC1kcmFmdHMvDQog ICAgICAgICAgICAgIGJyaXNjb2UtdHN2d2ctZWNuLXR1bm5lbC0wMC50eHQ+Lg0KDQogICBbSS1E LmlldGYtdHN2d2ctZWNuLW1wbHNdDQogICAgICAgICAgICAgICJFeHBsaWNpdCBDb25nZXN0aW9u IE1hcmtpbmcgaW4gTVBMUyIsIE9jdG9iZXIgMjAwNywgPGh0dHANCiAgICAgICAgICAgICAgOi8v d3d3LmlldGYub3JnL2ludGVybmV0LWRyYWZ0cy8NCiAgICAgICAgICAgICAgZHJhZnQtaWV0Zi10 c3Z3Zy1lY24tbXBscy0wMi50eHQ+Lg0KDQogICBbSS1ELmNoYXJueS1wY24tc2luZ2xlLW1hcmtp bmddDQogICAgICAgICAgICAgICJQcmUtQ29uZ2VzdGlvbiBOb3RpZmljYXRpb24gVXNpbmcgU2lu Z2xlIE1hcmtpbmcgZm9yDQogICAgICAgICAgICAgIEFkbWlzc2lvbiBhbmQgVGVybWluYXRpb24i LCBOb3ZlbWJlciAyMDA3LCA8aHR0cDovLw0KICAgICAgICAgICAgICB3d3cuaWV0Zi5vcmcvaW50 ZXJuZXQtZHJhZnRzLw0KICAgICAgICAgICAgICBkcmFmdC1jaGFybnktcGNuLXNpbmdsZS1tYXJr aW5nLTAzLnR4dD4uDQoNCiAgIFtJLUQuZWFyZGxleS1wY24tYXJjaGl0ZWN0dXJlXQ0KICAgICAg ICAgICAgICAiUHJlLUNvbmdlc3Rpb24gTm90aWZpY2F0aW9uIEFyY2hpdGVjdHVyZSIsIEp1bmUg MjAwNywgPGh0DQogICAgICAgICAgICAgIHRwOi8vd3d3LmlldGYub3JnL2ludGVybmV0LWRyYWZ0 cy8NCiAgICAgICAgICAgICAgZHJhZnQtZWFyZGxleS1wY24tYXJjaGl0ZWN0dXJlLTAwLnR4dD4u DQoNCiAgIFtJLUQud2VzdGJlcmctcGNuLWxvYWQtY29udHJvbF0NCiAgICAgICAgICAgICAgIkxD LVBDTjogVGhlIExvYWQgQ29udHJvbCBQQ04gU29sdXRpb24iLCBOb3ZlbWJlciAyMDA3LCA8aA0K ICAgICAgICAgICAgICB0dHA6Ly93d3cuaWV0Zi5vcmcvaW50ZXJuZXQtZHJhZnRzLw0KICAgICAg ICAgICAgICBkcmFmdC13ZXN0YmVyZy1wY24tbG9hZC1jb250cm9sLTAyLnR4dD4uDQoNCiAgIFtJ LUQuYmVocmluZ2VyLXRzdndnLXJzdnAtc2VjdXJpdHktZ3JvdXBrZXlpbmddDQogICAgICAgICAg ICAgICJBIEZyYW1ld29yayBmb3IgUlNWUCBTZWN1cml0eSBVc2luZyBEeW5hbWljIEdyb3VwDQog ICAgICAgICAgICAgIEtleWluZyIsIEp1bmUgMjAwNywgPGh0dHA6Ly93d3cud2F0ZXJzcHJpbmdz Lm9yZy9wdWIvaWQvDQogICAgICAgICAgICAgIGRyYWZ0LWJlaHJpbmdlci10c3Z3Zy1yc3ZwLXNl Y3VyaXR5LWdyb3Vwa2V5aW5nLTAwLnR4dD4uDQoNCg0KDQpFYXJkbGV5IChFZGl0b3IpICAgICAg ICAgIEV4cGlyZXMgTWF5IDIyLCAyMDA4ICAgICAgICAgICAgICAgICBbUGFnZSA0MV0NCgwNCklu dGVybmV0LURyYWZ0ICAgICAgICAgICAgICAgICAgRG9jdW1lbnQgICAgICAgICAgICAgICAgICAg Tm92ZW1iZXIgMjAwNw0KDQoNCiAgIFtJLUQuYnJpc2NvZS1yZS1wY24tYm9yZGVyLWNoZWF0XQ0K ICAgICAgICAgICAgICAiRW11bGF0aW5nIEJvcmRlciBGbG93IFBvbGljaW5nIHVzaW5nIFJlLUVD TiBvbiBCdWxrDQogICAgICAgICAgICAgIERhdGEiLCBKdW5lIDIwMDYsIDxodHRwOi8vd3d3Lndh dGVyc3ByaW5ncy5vcmcvcHViL2lkLw0KICAgICAgICAgICAgICBicmlzY29lLXJlLXBjbi1ib3Jk ZXItY2hlYXQtMDEudHh0Pi4NCg0KICAgW1JGQzQzMDNdICBLZW50LCBTLiwgIklQIEVuY2Fwc3Vs YXRpbmcgU2VjdXJpdHkgUGF5bG9hZCAoRVNQKSIsDQogICAgICAgICAgICAgIFJGQyA0MzAzLCBE ZWNlbWJlciAyMDA1Lg0KDQogICBbUkZDMjQ3NV0gIEJsYWtlLCBTLiwgQmxhY2ssIEQuLCBDYXJs c29uLCBNLiwgRGF2aWVzLCBFLiwgV2FuZywgWi4sDQogICAgICAgICAgICAgIGFuZCBXLiBXZWlz cywgIkFuIEFyY2hpdGVjdHVyZSBmb3IgRGlmZmVyZW50aWF0ZWQNCiAgICAgICAgICAgICAgU2Vy dmljZXMiLCBSRkMgMjQ3NSwgRGVjZW1iZXIgMTk5OC4NCg0KICAgW1JGQzMyNDZdICBEYXZpZSwg Qi4sIENoYXJueSwgQS4sIEJlbm5ldCwgSi4sIEJlbnNvbiwgSy4sIExlIEJvdWRlYywNCiAgICAg ICAgICAgICAgSi4sIENvdXJ0bmV5LCBXLiwgRGF2YXJpLCBTLiwgRmlyb2l1LCBWLiwgYW5kIEQu DQogICAgICAgICAgICAgIFN0aWxpYWRpcywgIkFuIEV4cGVkaXRlZCBGb3J3YXJkaW5nIFBIQiAo UGVyLUhvcA0KICAgICAgICAgICAgICBCZWhhdmlvcikiLCBSRkMgMzI0NiwgTWFyY2ggMjAwMi4N Cg0KICAgW1JGQzQ1OTRdICBCYWJpYXJ6LCBKLiwgQ2hhbiwgSy4sIGFuZCBGLiBCYWtlciwgIkNv bmZpZ3VyYXRpb24NCiAgICAgICAgICAgICAgR3VpZGVsaW5lcyBmb3IgRGlmZlNlcnYgU2Vydmlj ZSBDbGFzc2VzIiwgUkZDIDQ1OTQsDQogICAgICAgICAgICAgIEF1Z3VzdCAyMDA2Lg0KDQogICBb UkZDMzE2OF0gIFJhbWFrcmlzaG5hbiwgSy4sIEZsb3lkLCBTLiwgYW5kIEQuIEJsYWNrLCAiVGhl IEFkZGl0aW9uDQogICAgICAgICAgICAgIG9mIEV4cGxpY2l0IENvbmdlc3Rpb24gTm90aWZpY2F0 aW9uIChFQ04pIHRvIElQIiwNCiAgICAgICAgICAgICAgUkZDIDMxNjgsIFNlcHRlbWJlciAyMDAx Lg0KDQogICBbUkZDMjIxMV0gIFdyb2NsYXdza2ksIEouLCAiU3BlY2lmaWNhdGlvbiBvZiB0aGUg Q29udHJvbGxlZC1Mb2FkDQogICAgICAgICAgICAgIE5ldHdvcmsgRWxlbWVudCBTZXJ2aWNlIiwg UkZDIDIyMTEsIFNlcHRlbWJlciAxOTk3Lg0KDQogICBbUkZDMjk5OF0gIEJlcm5ldCwgWS4sIEZv cmQsIFAuLCBZYXZhdGthciwgUi4sIEJha2VyLCBGLiwgWmhhbmcsIEwuLA0KICAgICAgICAgICAg ICBTcGVlciwgTS4sIEJyYWRlbiwgUi4sIERhdmllLCBCLiwgV3JvY2xhd3NraSwgSi4sIGFuZCBF Lg0KICAgICAgICAgICAgICBGZWxzdGFpbmUsICJBIEZyYW1ld29yayBmb3IgSW50ZWdyYXRlZCBT ZXJ2aWNlcyBPcGVyYXRpb24NCiAgICAgICAgICAgICAgb3ZlciBEaWZmc2VydiBOZXR3b3JrcyIs IFJGQyAyOTk4LCBOb3ZlbWJlciAyMDAwLg0KDQogICBbUkZDMzI3MF0gIExlIEZhdWNoZXVyLCBG LiwgV3UsIEwuLCBEYXZpZSwgQi4sIERhdmFyaSwgUy4sIFZhYW5hbmVuLA0KICAgICAgICAgICAg ICBQLiwgS3Jpc2huYW4sIFIuLCBDaGV2YWwsIFAuLCBhbmQgSi4gSGVpbmFuZW4sICJNdWx0aS0N CiAgICAgICAgICAgICAgUHJvdG9jb2wgTGFiZWwgU3dpdGNoaW5nIChNUExTKSBTdXBwb3J0IG9m IERpZmZlcmVudGlhdGVkDQogICAgICAgICAgICAgIFNlcnZpY2VzIiwgUkZDIDMyNzAsIE1heSAy MDAyLg0KDQogICBbUkZDMTYzM10gIEJyYWRlbiwgQi4sIENsYXJrLCBELiwgYW5kIFMuIFNoZW5r ZXIsICJJbnRlZ3JhdGVkDQogICAgICAgICAgICAgIFNlcnZpY2VzIGluIHRoZSBJbnRlcm5ldCBB cmNoaXRlY3R1cmU6IGFuIE92ZXJ2aWV3IiwNCiAgICAgICAgICAgICAgUkZDIDE2MzMsIEp1bmUg MTk5NC4NCg0KICAgW1JGQzIxMTldICBCcmFkbmVyLCBTLiwgIktleSB3b3JkcyBmb3IgdXNlIGlu IFJGQ3MgdG8gSW5kaWNhdGUNCiAgICAgICAgICAgICAgUmVxdWlyZW1lbnQgTGV2ZWxzIiwgQkNQ IDE0LCBSRkMgMjExOSwgTWFyY2ggMTk5Ny4NCg0KICAgW1JGQzI5ODNdICBCbGFjaywgRC4sICJE aWZmZXJlbnRpYXRlZCBTZXJ2aWNlcyBhbmQgVHVubmVscyIsDQogICAgICAgICAgICAgIFJGQyAy OTgzLCBPY3RvYmVyIDIwMDAuDQoNCg0KDQoNCkVhcmRsZXkgKEVkaXRvcikgICAgICAgICAgRXhw aXJlcyBNYXkgMjIsIDIwMDggICAgICAgICAgICAgICAgIFtQYWdlIDQyXQ0KDA0KSW50ZXJuZXQt RHJhZnQgICAgICAgICAgICAgICAgICBEb2N1bWVudCAgICAgICAgICAgICAgICAgICBOb3ZlbWJl ciAyMDA3DQoNCg0KICAgW1JGQzI3NDddICBCYWtlciwgRi4sIExpbmRlbGwsIEIuLCBhbmQgTS4g VGFsd2FyLCAiUlNWUCBDcnlwdG9ncmFwaGljDQogICAgICAgICAgICAgIEF1dGhlbnRpY2F0aW9u IiwgUkZDIDI3NDcsIEphbnVhcnkgMjAwMC4NCg0KICAgW1JGQzM0MTFdICBIYXJyaW5ndG9uLCBE LiwgUHJlc3VobiwgUi4sIGFuZCBCLiBXaWpuZW4sICJBbg0KICAgICAgICAgICAgICBBcmNoaXRl Y3R1cmUgZm9yIERlc2NyaWJpbmcgU2ltcGxlIE5ldHdvcmsgTWFuYWdlbWVudA0KICAgICAgICAg ICAgICBQcm90b2NvbCAoU05NUCkgTWFuYWdlbWVudCBGcmFtZXdvcmtzIiwgU1REIDYyLCBSRkMg MzQxMSwNCiAgICAgICAgICAgICAgRGVjZW1iZXIgMjAwMi4NCg0KICAgW1JGQzMzOTNdICBEZW1p Y2hlbGlzLCBDLiBhbmQgUC4gQ2hpbWVudG8sICJJUCBQYWNrZXQgRGVsYXkgVmFyaWF0aW9uDQog ICAgICAgICAgICAgIE1ldHJpYyBmb3IgSVAgUGVyZm9ybWFuY2UgTWV0cmljcyAoSVBQTSkiLCBS RkMgMzM5MywNCiAgICAgICAgICAgICAgTm92ZW1iZXIgMjAwMi4NCg0KICAgW1JGQzQ2NTZdICBT aGFsdW5vdiwgUy4sIFRlaXRlbGJhdW0sIEIuLCBLYXJwLCBBLiwgQm9vdGUsIEouLCBhbmQgTS4N CiAgICAgICAgICAgICAgWmVrYXVza2FzLCAiQSBPbmUtd2F5IEFjdGl2ZSBNZWFzdXJlbWVudCBQ cm90b2NvbA0KICAgICAgICAgICAgICAoT1dBTVApIiwgUkZDIDQ2NTYsIFNlcHRlbWJlciAyMDA2 Lg0KDQogICBbUkZDNDc3OF0gIEthZW8sIE0uLCAiT3BlcmF0aW9uYWwgU2VjdXJpdHkgQ3VycmVu dCBQcmFjdGljZXMgaW4NCiAgICAgICAgICAgICAgSW50ZXJuZXQgU2VydmljZSBQcm92aWRlciBF bnZpcm9ubWVudHMiLCBSRkMgNDc3OCwNCiAgICAgICAgICAgICAgSmFudWFyeSAyMDA3Lg0KDQog ICBbSVRVLU1MUFBdDQogICAgICAgICAgICAgICJNdWx0aWxldmVsIFByZWNlZGVuY2UgYW5kIFBy ZS1lbXB0aW9uIFNlcnZpY2UgKE1MUFApIiwNCiAgICAgICAgICAgICAgSVRVLVQgUmVjb21tZW5k YXRpb24gSS4yNTUuMywgMTk5MC4NCg0KICAgW0l5ZXJdICAgICAiQW4gYXBwcm9hY2ggdG8gYWxs ZXZpYXRlIGxpbmsgb3ZlcmxvYWQgYXMgb2JzZXJ2ZWQgb24gYW4NCiAgICAgICAgICAgICAgSVAg YmFja2JvbmUiLCBJRUVFIElORk9DT00gLCAyMDAzLA0KICAgICAgICAgICAgICA8aHR0cDovL3d3 dy5pZWVlLWluZm9jb20ub3JnLzIwMDMvcGFwZXJzLzEwXzA0LnBkZj4uDQoNCiAgIFtTaGVua2Vy XSAgIkZ1bmRhbWVudGFsIGRlc2lnbiBpc3N1ZXMgZm9yIHRoZSBmdXR1cmUgSW50ZXJuZXQiLCBJ RUVFDQogICAgICAgICAgICAgIEpvdXJuYWwgb24gc2VsZWN0ZWQgYXJlYXMgaW4gY29tbXVuaWNh dGlvbnMgcHAgMTE3NiAtDQogICAgICAgICAgICAgIDExODgsIFZvbCAxMyAoNyksIDE5OTUuDQoN CiAgIFtZLjE1NDFdICAgIk5ldHdvcmsgUGVyZm9ybWFuY2UgT2JqZWN0aXZlcyBmb3IgSVAtYmFz ZWQgU2VydmljZXMiLA0KICAgICAgICAgICAgICBJVFUtVCBSZWNvbW1lbmRhdGlvbiBZLjE1NDEs IEZlYnJ1YXJ5IDIwMDYuDQoNCiAgIFtQLjgwMF0gICAgIk1ldGhvZHMgZm9yIHN1YmplY3RpdmUg ZGV0ZXJtaW5hdGlvbiBvZiB0cmFuc21pc3Npb24NCiAgICAgICAgICAgICAgcXVhbGl0eSIsIElU VS1UIFJlY29tbWVuZGF0aW9uIFAuODAwLCBBdWd1c3QgMTk5Ni4NCg0KICAgW1NvbmdodXJzdF0N CiAgICAgICAgICAgICAgIkd1YXJhbnRlZWQgUW9TIFN5bnRoZXNpcyBmb3IgQWRtaXNzaW9uIENv bnRyb2wgd2l0aA0KICAgICAgICAgICAgICBTaGFyZWQgQ2FwYWNpdHkiLCBCVCBUZWNobmljYWwg UmVwb3J0IFRSLUNYUjktMjAwNi0wMDEsDQogICAgICAgICAgICAgIEZlYnVyYXJ5IDIwMDYsIDxo dHRwOi8vd3d3LmNzLnVjbC5hYy51ay9zdGFmZi9CLkJyaXNjb2UvDQogICAgICAgICAgICAgIHBy b2plY3RzL2lwZTJlcW9zL2dxcy9wYXBlcnMvR1FTX3NoYXJlZF90ci5wZGY+Lg0KDQogICBbUENO LWVtYWlsLUVDTVBdDQogICAgICAgICAgICAgICJFbWFpbCB0byBQQ04gV0cgbWFpbGluZyBsaXN0 IiwgTm92ZW1iZXIgMjAwNywgPGh0dHA6Ly8NCiAgICAgICAgICAgICAgd3d3MS5pZXRmLm9yZy9t YWlsLWFyY2hpdmUvd2ViL3Bjbi9jdXJyZW50L21zZzAwODcxLmh0bWw+Lg0KDQoNCg0KDQpFYXJk bGV5IChFZGl0b3IpICAgICAgICAgIEV4cGlyZXMgTWF5IDIyLCAyMDA4ICAgICAgICAgICAgICAg ICBbUGFnZSA0M10NCgwNCkludGVybmV0LURyYWZ0ICAgICAgICAgICAgICAgICAgRG9jdW1lbnQg ICAgICAgICAgICAgICAgICAgTm92ZW1iZXIgMjAwNw0KDQoNCiAgIFtQQ04tZW1haWwtdHJhZmZp Yy1lbXB0eS1hZ2dyZWdhdGVzXQ0KICAgICAgICAgICAgICAiRW1haWwgdG8gUENOIFdHIG1haWxp bmcgbGlzdCIsIE9jdG9iZXIgMjAwNywgPGh0dHA6Ly8NCiAgICAgICAgICAgICAgd3d3MS5pZXRm Lm9yZy9tYWlsLWFyY2hpdmUvd2ViL3Bjbi9jdXJyZW50L21zZzAwODMxLmh0bWw+Lg0KDQoNCkF1 dGhvcidzIEFkZHJlc3MNCg0KICAgUGhpbGlwIEVhcmRsZXkNCiAgIEJUDQogICBCNTQvNzcsIFNp cml1cyBIb3VzZSBBZGFzdHJhbCBQYXJrIE1hcnRsZXNoYW0gSGVhdGgNCiAgIElwc3dpY2gsIFN1 ZmZvbGsgIElQNSAzUkUNCiAgIFVuaXRlZCBLaW5nZG9tDQoNCiAgIEVtYWlsOiBwaGlsaXAuZWFy ZGxleUBidC5jb20NCg0KDQoNCg0KDQoNCg0KDQoNCg0KDQoNCg0KDQoNCg0KDQoNCg0KDQoNCg0K DQoNCg0KDQoNCg0KDQoNCg0KDQoNCg0KDQoNCg0KRWFyZGxleSAoRWRpdG9yKSAgICAgICAgICBF eHBpcmVzIE1heSAyMiwgMjAwOCAgICAgICAgICAgICAgICAgW1BhZ2UgNDRdDQoMDQpJbnRlcm5l dC1EcmFmdCAgICAgICAgICAgICAgICAgIERvY3VtZW50ICAgICAgICAgICAgICAgICAgIE5vdmVt YmVyIDIwMDcNCg0KDQpGdWxsIENvcHlyaWdodCBTdGF0ZW1lbnQNCg0KICAgQ29weXJpZ2h0IChD KSBUaGUgSUVURiBUcnVzdCAoMjAwNykuDQoNCiAgIFRoaXMgZG9jdW1lbnQgaXMgc3ViamVjdCB0 byB0aGUgcmlnaHRzLCBsaWNlbnNlcyBhbmQgcmVzdHJpY3Rpb25zDQogICBjb250YWluZWQgaW4g QkNQIDc4LCBhbmQgZXhjZXB0IGFzIHNldCBmb3J0aCB0aGVyZWluLCB0aGUgYXV0aG9ycw0KICAg cmV0YWluIGFsbCB0aGVpciByaWdodHMuDQoNCiAgIFRoaXMgZG9jdW1lbnQgYW5kIHRoZSBpbmZv cm1hdGlvbiBjb250YWluZWQgaGVyZWluIGFyZSBwcm92aWRlZCBvbiBhbg0KICAgIkFTIElTIiBi YXNpcyBhbmQgVEhFIENPTlRSSUJVVE9SLCBUSEUgT1JHQU5JWkFUSU9OIEhFL1NIRSBSRVBSRVNF TlRTDQogICBPUiBJUyBTUE9OU09SRUQgQlkgKElGIEFOWSksIFRIRSBJTlRFUk5FVCBTT0NJRVRZ LCBUSEUgSUVURiBUUlVTVCBBTkQNCiAgIFRIRSBJTlRFUk5FVCBFTkdJTkVFUklORyBUQVNLIEZP UkNFIERJU0NMQUlNIEFMTCBXQVJSQU5USUVTLCBFWFBSRVNTDQogICBPUiBJTVBMSUVELCBJTkNM VURJTkcgQlVUIE5PVCBMSU1JVEVEIFRPIEFOWSBXQVJSQU5UWSBUSEFUIFRIRSBVU0UgT0YNCiAg IFRIRSBJTkZPUk1BVElPTiBIRVJFSU4gV0lMTCBOT1QgSU5GUklOR0UgQU5ZIFJJR0hUUyBPUiBB TlkgSU1QTElFRA0KICAgV0FSUkFOVElFUyBPRiBNRVJDSEFOVEFCSUxJVFkgT1IgRklUTkVTUyBG T1IgQSBQQVJUSUNVTEFSIFBVUlBPU0UuDQoNCg0KSW50ZWxsZWN0dWFsIFByb3BlcnR5DQoNCiAg IFRoZSBJRVRGIHRha2VzIG5vIHBvc2l0aW9uIHJlZ2FyZGluZyB0aGUgdmFsaWRpdHkgb3Igc2Nv cGUgb2YgYW55DQogICBJbnRlbGxlY3R1YWwgUHJvcGVydHkgUmlnaHRzIG9yIG90aGVyIHJpZ2h0 cyB0aGF0IG1pZ2h0IGJlIGNsYWltZWQgdG8NCiAgIHBlcnRhaW4gdG8gdGhlIGltcGxlbWVudGF0 aW9uIG9yIHVzZSBvZiB0aGUgdGVjaG5vbG9neSBkZXNjcmliZWQgaW4NCiAgIHRoaXMgZG9jdW1l bnQgb3IgdGhlIGV4dGVudCB0byB3aGljaCBhbnkgbGljZW5zZSB1bmRlciBzdWNoIHJpZ2h0cw0K ICAgbWlnaHQgb3IgbWlnaHQgbm90IGJlIGF2YWlsYWJsZTsgbm9yIGRvZXMgaXQgcmVwcmVzZW50 IHRoYXQgaXQgaGFzDQogICBtYWRlIGFueSBpbmRlcGVuZGVudCBlZmZvcnQgdG8gaWRlbnRpZnkg YW55IHN1Y2ggcmlnaHRzLiAgSW5mb3JtYXRpb24NCiAgIG9uIHRoZSBwcm9jZWR1cmVzIHdpdGgg cmVzcGVjdCB0byByaWdodHMgaW4gUkZDIGRvY3VtZW50cyBjYW4gYmUNCiAgIGZvdW5kIGluIEJD UCA3OCBhbmQgQkNQIDc5Lg0KDQogICBDb3BpZXMgb2YgSVBSIGRpc2Nsb3N1cmVzIG1hZGUgdG8g dGhlIElFVEYgU2VjcmV0YXJpYXQgYW5kIGFueQ0KICAgYXNzdXJhbmNlcyBvZiBsaWNlbnNlcyB0 byBiZSBtYWRlIGF2YWlsYWJsZSwgb3IgdGhlIHJlc3VsdCBvZiBhbg0KICAgYXR0ZW1wdCBtYWRl IHRvIG9idGFpbiBhIGdlbmVyYWwgbGljZW5zZSBvciBwZXJtaXNzaW9uIGZvciB0aGUgdXNlIG9m DQogICBzdWNoIHByb3ByaWV0YXJ5IHJpZ2h0cyBieSBpbXBsZW1lbnRlcnMgb3IgdXNlcnMgb2Yg dGhpcw0KICAgc3BlY2lmaWNhdGlvbiBjYW4gYmUgb2J0YWluZWQgZnJvbSB0aGUgSUVURiBvbi1s aW5lIElQUiByZXBvc2l0b3J5IGF0DQogICBodHRwOi8vd3d3LmlldGYub3JnL2lwci4NCg0KICAg VGhlIElFVEYgaW52aXRlcyBhbnkgaW50ZXJlc3RlZCBwYXJ0eSB0byBicmluZyB0byBpdHMgYXR0 ZW50aW9uIGFueQ0KICAgY29weXJpZ2h0cywgcGF0ZW50cyBvciBwYXRlbnQgYXBwbGljYXRpb25z LCBvciBvdGhlciBwcm9wcmlldGFyeQ0KICAgcmlnaHRzIHRoYXQgbWF5IGNvdmVyIHRlY2hub2xv Z3kgdGhhdCBtYXkgYmUgcmVxdWlyZWQgdG8gaW1wbGVtZW50DQogICB0aGlzIHN0YW5kYXJkLiAg UGxlYXNlIGFkZHJlc3MgdGhlIGluZm9ybWF0aW9uIHRvIHRoZSBJRVRGIGF0DQogICBpZXRmLWlw ckBpZXRmLm9yZy4NCg0KDQpBY2tub3dsZWRnbWVudA0KDQogICBGdW5kaW5nIGZvciB0aGUgUkZD IEVkaXRvciBmdW5jdGlvbiBpcyBwcm92aWRlZCBieSB0aGUgSUVURg0KICAgQWRtaW5pc3RyYXRp dmUgU3VwcG9ydCBBY3Rpdml0eSAoSUFTQSkuDQoNCg0KDQoNCg0KRWFyZGxleSAoRWRpdG9yKSAg ICAgICAgICBFeHBpcmVzIE1heSAyMiwgMjAwOCAgICAgICAgICAgICAgICAgW1BhZ2UgNDVdDQoM DQo= ------_=_NextPart_001_01C82B4D.F8E6A7B0 Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn ------_=_NextPart_001_01C82B4D.F8E6A7B0-- From pcn-bounces@ietf.org Tue Nov 20 06:29:47 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IuRIX-0003mE-BH; Tue, 20 Nov 2007 06:29:45 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IuRIV-0003jm-6t for pcn-confirm+ok@megatron.ietf.org; Tue, 20 Nov 2007 06:29:43 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IuRIU-0003jN-T2 for pcn@ietf.org; Tue, 20 Nov 2007 06:29:42 -0500 Received: from smtp1.smtp.bt.com ([217.32.164.137]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IuRIP-00079H-7Z for pcn@ietf.org; Tue, 20 Nov 2007 06:29:42 -0500 Received: from E03MVZ1-UKDY.domain1.systemhost.net ([193.113.30.65]) by smtp1.smtp.bt.com with Microsoft SMTPSVC(6.0.3790.1830); Tue, 20 Nov 2007 11:29:36 +0000 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [PCN] Fwd: I-D Action:draft-chan-pcn-encoding-comparison-01.txt Date: Tue, 20 Nov 2007 11:29:35 -0000 Message-ID: <75A199C5D243C741BF3D3F1EBCEF9BA503B3437C@E03MVZ1-UKDY.domain1.systemhost.net> In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] Fwd: I-D Action:draft-chan-pcn-encoding-comparison-01.txt Thread-Index: AcgquPGUzlNxdsseRSa+YUAOW0LCxAAlb2RA From: To: , X-OriginalArrivalTime: 20 Nov 2007 11:29:36.0159 (UTC) FILETIME=[A13A12F0:01C82B68] X-Spam-Score: -1.0 (-) X-Scan-Signature: 515708a075ffdf0a79d1c83b601e2afd Cc: X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Kwok, Georgios, all, Here are some comments on this draft http://www.ietf.org/internet-drafts/draft-chan-pcn-encoding-comparison-0 1.txt - it's much better than 00! Sorry for the lengthy email Thanks, Phil/ 1. Introduction Good.=20 2. Encoding requirements First 2 paras good.=20 last para /bullets can be improved & I do not agree with "A total of six required encoding states". Suggested replacement: << Different PCN-marking behaviour documents have suggested different encoding states are needed (2, 3 or 4 of the ones below). In total, the following possibilities have been suggested: o Not pre-congested Marking, for indication of No pre-Congestion=20 o Admission Marking, for indication of Flow Admission Information o Termination Marking, for indication of Flow Termination Information o Single Marking, for indication of pre-congestion Information o Affected Marking, also for indication of Flow Termination Information In addition there must be a way of distinguishing PCN-packets (whether PCN-marked or not) from non PCN-pkts. NOTE: o Single Marking: this has been suggested by [ref]. a single marking is used to indicate both admission and termination marking=20 o Affected Marking: this has been suggested by [ref]. Its suggestion is that a PCN-node which is termin-marking some pkts (ie is in the termin-marking state) encodes the rest of the PCN-pkts with the affected marking. o the WG will select which of the 5 possible encoding states listed will actually be supported, or if the choice is a deployment choice for the operator or under the control of an OAM Configuration system option (eg where an operator employs either adm-marking or termin-marking but not both in the PCN-domain).=20 >> a few words about the above.=20 I think have to talk about single marking - calling it AM/TM in the tables is confusing (for me).=20 I think it's better to say there has to be a way of distinguishing PCN-packets & non PCN-pkts, rather than saying there has to be an extra encoding state.=20 I don't think nonce cheater detection is relevant [I think you're talking about ECN nonce. It's fair enough to talk at the appropriate moments about loss of ECN info if the ECN field is used to encode PCN. but I think it's confusing (and on occasions, in some of your encoding tables, wrong] to list ECN nonce as a PCN encoding state. Or are you suggesting a *PCN* nonce? (would seem this isn't needed, as PCN-domain is trusted. However, maybe thinking about future extensions?) New sub-section 2.2=20 I wonder if it's worth introducing what will be the evaluation criteria, eg: How easy is it to support multiple PCN classes ? Are there any interactions with ecmp? Are there any interactions with ecn? Codepoint shortage for mpls.=20 etc Section 3 Encoding options The first few lines (before bullets) could be shortened to: "There are various encoding options, which can be grouped by which fields they use:" You can add a note here that "using a different channel (e.g., IPFIX .." is out of scope of WG. I believe this is agreed. Section 3.4 can therefore be safely deleted.=20 The list of abbreviations can be tweaked: Add:=20 SM single marking ;=20 Not-pc [or something meaning 'not pre-congested'] Delete: Not-ce, NDS-CE=20 Section 3.1+ Think the "ecn & dscp" [current 3.1] needs the most explanation, so I don't think it should be done first. suggest swapping with 3.3. 3.1 ECN & DSCP: I found the first part of this [before 3.1.1] too dense. I think more options need to be listed. I think the terminology in the table is confusing, replace: ----------------------------------------------------------------------- | ECN Bits || 00 | 01 | 10 | 11 || DSCP | |=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D | | RFC 3168 || Not-ECT | ECT(1) | ECT(0) | CE || NA | |=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D | | Option 1 || AM | NA | NA | TM || PCN | |--------------++----------+----------+----------+----------++---------- | | Option 2 || Not-PC | NA | NA | SM || PCN | |--------------++----------+----------+----------+----------++---------- | | Option 3 || NA |Not-PC(A) | Not-PC(T)| SM || PCN | |--------------++----------+----------+----------+----------++---------- | | Option 4 ...... ----------------------------------------------------------------------- In Option 3 I tried to work out what you meant by ECT(A) & ECT(T). I think you're saying that the PCN-ingress-node sets the pkt with a mix of 2 codepoints; and only Single Marking state is used to indicate pre-congestion; a pcn-node that wants to adm mark a pkt then selects a Not-PC(A) pkt and re-marks it to SM. Is that right? need to explain why this might be useful.=20 There was an option once mentioned [in a draft-briscoe-architecture I think] of using a 2nd DSCP to indicate TM (or perhaps to indicate Affected marking). I also think you can talk separately about the Affected marking. In my table, AFM can obviously be added to all 3 Options I also think there's a separate series of options where you're trying to preserve ECN information on an end-to-end basis whilst having PCN running in some domain in the middle. And you don't want to solve this by doing something else like tunnelling over the PCN-domain. I think this needs separate discussion as the issues are different and quite hard to think about. For instance, there are two separate cases: just preserving the Not-CE & CE info, and also preserving the ECT-nonce info [there are different levels of interest/deployment/standardisation]. So I think it would be better to have a separate section discussing encoding options when ECN info needs to be preserved. 3.1.1 could expand with something like:=20 For example, this means that PCN can easily be applied to multiple classes.=20 3.1.1.1, [RFC4774] interactions: as well as the things you talk about, I thought [not sure though] that 4774 was concerned about "what happens if things go wrong", eg: - a RFC4774 pkt gets accidentally dscp-re-marked to the pcn-dscp; a PCN-ingress-node gets misconfigured so it allows a RFC4774 pkt in on the pcn-dscp. Can anything nasty happen? - a router in the PCN-domain misconfigures itself so it's RFC3246-enabled (ie no longer PCN-enabled). Can anything nasty happen, eg when PCN-marked pkts encounter this router? - a PCN-egress-node gets misconfigured so it allows a pcn-marked pkt 'out' of the PCN-domain. eg if the first case happens, may want to try to minimise the impact of any ECN-CE pkts that therefore sneak in on the PCN class 3.1.2 "possibly adding complexity when tunneling of the PCN encoding is required" - explain. 3.2 using ECN field if I understand this case right, you're saying that you're NOT using a different DSCP to distinguish PCN traffic from non-PCN traffic. Think you need to make this really clear, as it's quite bizarre. However, as draft-pcn-architecture says "There needs to be a way for a PCN-node to distinguish PCN-traffic from non PCN-traffic" - I don't understand how you propose to do this, although I could imagine an Option like this: |=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D | | Option 1 || Not-CE | Not-PC | SM | CE || NA | |--------------++----------+----------+----------+----------++---------- | One downside of this is that there can only be 1 PCN class [PHB] basically I don't understand the table.=20 3.3 DSCP in table, I think you should change things like "PCN/AM/TM" to SM, eg: ----------------------------------------------------------------------- | DSCP Bits || Original |Experimental 1 |Experimental 2 |Experimental 3 | |=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D | | Option 7 || Not-PC | SM | NA | NA | |-----------++----------+---------------+---------------+--------------- | 3.3.1 Benefits " concerns raised in RFC 4774 [20] are not applicable" - pedantic point: I think it's really that they should be easier to deal with [whether or not they're applicable depends on whether a PCN-node knows how to process a pkt with the ECN field set] " all 4 DSCP encoding options depicted in Figure 3 can support the not congested indication, PCN capable transport marking, the admission control and flow termination encoding states." Don't understand this, only true for options 9 & 10. "In addition Option 8 and 10 can in addition support the ECMP solution." Replace: "They support Affected marking, which [ref] says has benefits if the PCN-domain operates ECMP multi-path routing" additional benefit (and in my view the most significant): exptal dscps are lightly standardised - limited rules on how you use them, largely up to the operator. 3.3.2 Drawbacks clarify by replacing=20 "This type of encoding needs to use per PHB, in addition to the original DSCP and depending on the encoding option used, one, two or three DSCP values, respectivelly."=20 With "This type of encoding needs to use, for each DS class (PHB) that is PCN-enabled, 2 3 or 4 DSCP values, depending on the encoding option." Clarify that you need this set of 2/3/4 dscp values for each class that's PCN-enabled. Clarify that particular issue for mpls [codepoint shortage] "Furthermore, if the separation between the PCN traffic and non- PCN traffic is required". According to draft-pcn-architecture S4.5, there is no "if" about it. last para repeats what already said.=20 Major disadvantage: interactions with ECMP. ECMP algos can use a packet's DSCP when deciding what interface to fwd it on. In a PCN-domain running ECMP, for a flow which has some packets PCN-marked and some unmarked. its packets will travel over more than one path. This may well lead to mis-ordering. To me, this means that the S3.3 DSCP option is a non-starter in any PCN-domain that uses an ECMP algorithm that uses the DSCP. NB ECMP algos are proprietary to router vendors, ie operator has poor control over them. S3.4 out of band channel - section can be safely deleted. S5 Security implications All these are I think covered by the draft-pcn-architecture, so can safely delete everything.=20 Frequent Typo: None PCN capable =3D> non PCN capable=20 _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Tue Nov 20 09:12:21 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IuTps-00067P-HW; Tue, 20 Nov 2007 09:12:20 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IuTpr-00067K-Iu for pcn-confirm+ok@megatron.ietf.org; Tue, 20 Nov 2007 09:12:19 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IuTpr-00067C-1k for pcn@ietf.org; Tue, 20 Nov 2007 09:12:19 -0500 Received: from zcars04f.nortel.com ([47.129.242.57]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IuTpn-0003CS-Rh for pcn@ietf.org; Tue, 20 Nov 2007 09:12:19 -0500 Received: from zrtphxm1.corp.nortel.com (zrtphxm1.corp.nortel.com [47.140.202.50]) by zcars04f.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id lAKECB718796; Tue, 20 Nov 2007 14:12:11 GMT Received: from KCHAN-2K3.nortel.com ([47.16.54.161] RDNS failed) by zrtphxm1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.1830); Tue, 20 Nov 2007 09:11:27 -0500 X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Tue, 20 Nov 2007 09:11:52 -0500 To: From: "Kwok-Ho Chan" Subject: RE: [PCN] Fwd: I-D Action:draft-chan-pcn-encoding-comparison-01.txt In-Reply-To: <75A199C5D243C741BF3D3F1EBCEF9BA503B3437C@E03MVZ1-UKDY.doma in1.systemhost.net> References: <75A199C5D243C741BF3D3F1EBCEF9BA503B3437C@E03MVZ1-UKDY.domain1.systemhost.net> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Message-ID: X-OriginalArrivalTime: 20 Nov 2007 14:11:27.0901 (UTC) FILETIME=[3DE04CD0:01C82B7F] X-Spam-Score: 0.0 (/) X-Scan-Signature: 71f780ffdd80c541d3e75aa5f2710d3d Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Phil: Thank you very much for your comments. You know I am not shy and will receive and work on the comments positively. With Hope to get them shorter as we improve this draft. :) Will work on each issue you indicated. Thanks! -- Kwok -- At 06:29 AM 11/20/2007, philip.eardley@bt.com wrote: >Kwok, Georgios, all, > >Here are some comments on this draft >http://www.ietf.org/internet-drafts/draft-chan-pcn-encoding-comparison-0 >1.txt - it's much better than 00! >Sorry for the lengthy email >Thanks, >Phil/ > >1. Introduction >Good. > >2. Encoding requirements >First 2 paras good. >last para /bullets can be improved & I do not agree with "A total of six >required encoding states". Suggested replacement: ><< >Different PCN-marking behaviour documents have suggested different >encoding states are needed (2, 3 or 4 of the ones below). In total, the >following possibilities have been suggested: > >o Not pre-congested Marking, for indication of No pre-Congestion >o Admission Marking, for indication of Flow Admission Information >o Termination Marking, for indication of Flow Termination Information >o Single Marking, for indication of pre-congestion Information >o Affected Marking, also for indication of Flow Termination Information > >In addition there must be a way of distinguishing PCN-packets (whether >PCN-marked or not) from non PCN-pkts. > >NOTE: >o Single Marking: this has been suggested by [ref]. a single marking is >used to indicate both admission and termination marking >o Affected Marking: this has been suggested by [ref]. Its suggestion is >that a PCN-node which is termin-marking some pkts (ie is in the >termin-marking state) encodes the rest of the PCN-pkts with the affected >marking. >o the WG will select which of the 5 possible encoding states listed >will actually be supported, or if the choice is a deployment choice for >the operator or under the control of an OAM Configuration system option >(eg where an operator employs either adm-marking or termin-marking but >not both in the PCN-domain). > >> > >a few words about the above. >I think have to talk about single marking - calling it AM/TM in the >tables is confusing (for me). >I think it's better to say there has to be a way of distinguishing >PCN-packets & non PCN-pkts, rather than saying there has to be an extra >encoding state. >I don't think nonce cheater detection is relevant [I think you're >talking about ECN nonce. It's fair enough to talk at the appropriate >moments about loss of ECN info if the ECN field is used to encode PCN. >but I think it's confusing (and on occasions, in some of your encoding >tables, wrong] to list ECN nonce as a PCN encoding state. Or are you >suggesting a *PCN* nonce? (would seem this isn't needed, as PCN-domain >is trusted. However, maybe thinking about future extensions?) > >New sub-section 2.2 >I wonder if it's worth introducing what will be the evaluation criteria, >eg: >How easy is it to support multiple PCN classes ? >Are there any interactions with ecmp? >Are there any interactions with ecn? >Codepoint shortage for mpls. >etc > > >Section 3 Encoding options >The first few lines (before bullets) could be shortened to: >"There are various encoding options, which can be grouped by which >fields they use:" >You can add a note here that "using a different channel (e.g., IPFIX .." >is out of scope of WG. I believe this is agreed. Section 3.4 can >therefore be safely deleted. > >The list of abbreviations can be tweaked: >Add: >SM single marking ; >Not-pc [or something meaning 'not pre-congested'] > >Delete: Not-ce, NDS-CE > >Section 3.1+ >Think the "ecn & dscp" [current 3.1] needs the most explanation, so I >don't think it should be done first. suggest swapping with 3.3. > >3.1 ECN & DSCP: >I found the first part of this [before 3.1.1] too dense. I think more >options need to be listed. I think the terminology in the table is >confusing, replace: >----------------------------------------------------------------------- >| ECN Bits || 00 | 01 | 10 | 11 || DSCP >| >|==============++==========+==========+==========+==========++========== >| >| RFC 3168 || Not-ECT | ECT(1) | ECT(0) | CE || NA >| >|==============++==========+==========+==========+==========++========== >| >| Option 1 || AM | NA | NA | TM || PCN >| >|--------------++----------+----------+----------+----------++---------- >| >| Option 2 || Not-PC | NA | NA | SM || PCN >| >|--------------++----------+----------+----------+----------++---------- >| >| Option 3 || NA |Not-PC(A) | Not-PC(T)| SM || PCN >| >|--------------++----------+----------+----------+----------++---------- >| >| Option 4 ...... > ----------------------------------------------------------------------- > >In Option 3 I tried to work out what you meant by ECT(A) & ECT(T). I >think you're saying that the PCN-ingress-node sets the pkt with a mix of >2 codepoints; and only Single Marking state is used to indicate >pre-congestion; a pcn-node that wants to adm mark a pkt then selects a >Not-PC(A) pkt and re-marks it to SM. Is that right? need to explain why >this might be useful. > >There was an option once mentioned [in a draft-briscoe-architecture I >think] of using a 2nd DSCP to indicate TM (or perhaps to indicate >Affected marking). > >I also think you can talk separately about the Affected marking. In my >table, AFM can obviously be added to all 3 Options > >I also think there's a separate series of options where you're trying to >preserve ECN information on an end-to-end basis whilst having PCN >running in some domain in the middle. And you don't want to solve this >by doing something else like tunnelling over the PCN-domain. I think >this needs separate discussion as the issues are different and quite >hard to think about. For instance, there are two separate cases: just >preserving the Not-CE & CE info, and also preserving the ECT-nonce info >[there are different levels of interest/deployment/standardisation]. >So I think it would be better to have a separate section discussing >encoding options when ECN info needs to be preserved. > > >3.1.1 >could expand with something like: >For example, this means that PCN can easily be applied to multiple >classes. > >3.1.1.1, [RFC4774] interactions: >as well as the things you talk about, I thought [not sure though] that >4774 was concerned about "what happens if things go wrong", eg: >- a RFC4774 pkt gets accidentally dscp-re-marked to the pcn-dscp; a >PCN-ingress-node gets misconfigured so it allows a RFC4774 pkt in on the >pcn-dscp. Can anything nasty happen? >- a router in the PCN-domain misconfigures itself so it's >RFC3246-enabled (ie no longer PCN-enabled). Can anything nasty happen, >eg when PCN-marked pkts encounter this router? >- a PCN-egress-node gets misconfigured so it allows a pcn-marked pkt >'out' of the PCN-domain. > >eg if the first case happens, may want to try to minimise the impact of >any ECN-CE pkts that therefore sneak in on the PCN class > >3.1.2 >"possibly adding complexity when tunneling of the PCN encoding is >required" - explain. > >3.2 using ECN field >if I understand this case right, you're saying that you're NOT using a >different DSCP to distinguish PCN traffic from non-PCN traffic. Think >you need to make this really clear, as it's quite bizarre. However, as >draft-pcn-architecture says "There needs to be a way for a PCN-node to >distinguish PCN-traffic from non PCN-traffic" - I don't understand how >you propose to do this, although I could imagine an Option like this: >|==============++==========+==========+==========+==========++========== >| >| Option 1 || Not-CE | Not-PC | SM | CE || NA >| >|--------------++----------+----------+----------+----------++---------- >| >One downside of this is that there can only be 1 PCN class [PHB] > >basically I don't understand the table. > > >3.3 DSCP >in table, I think you should change things like "PCN/AM/TM" to SM, eg: >----------------------------------------------------------------------- >| DSCP Bits || Original |Experimental 1 |Experimental 2 |Experimental 3 >| >|===========++==========+===============+===============+=============== >| >| Option 7 || Not-PC | SM | NA | NA >| >|-----------++----------+---------------+---------------+--------------- >| > >3.3.1 Benefits >" concerns raised in RFC 4774 [20] are not applicable" - pedantic point: >I think it's really that they should be easier to deal with [whether or >not they're applicable depends on whether a PCN-node knows how to >process a pkt with the ECN field set] > >" all 4 DSCP encoding options depicted in Figure 3 can support the not >congested indication, PCN capable transport marking, the admission >control and flow termination encoding states." Don't understand this, >only true for options 9 & 10. > >"In addition Option 8 and 10 can in addition support the ECMP solution." >Replace: "They support Affected marking, which [ref] says has benefits >if the PCN-domain operates ECMP multi-path routing" > >additional benefit (and in my view the most significant): exptal dscps >are lightly standardised - limited rules on how you use them, largely up >to the operator. > >3.3.2 Drawbacks >clarify by replacing >"This type of encoding needs to use per PHB, in addition to the original >DSCP and depending on the encoding option used, one, two or three DSCP >values, respectivelly." >With "This type of encoding needs to use, for each DS class (PHB) that >is PCN-enabled, 2 3 or 4 DSCP values, depending on the encoding option." > >Clarify that you need this set of 2/3/4 dscp values for each class >that's PCN-enabled. > >Clarify that particular issue for mpls [codepoint shortage] > >"Furthermore, if the separation between the PCN traffic and non- PCN >traffic is required". According to draft-pcn-architecture S4.5, there is >no "if" about it. > >last para repeats what already said. > >Major disadvantage: interactions with ECMP. ECMP algos can use a >packet's DSCP when deciding what interface to fwd it on. In a PCN-domain >running ECMP, for a flow which has some packets PCN-marked and some >unmarked. its packets will travel over more than one path. This may well >lead to mis-ordering. >To me, this means that the S3.3 DSCP option is a non-starter in any >PCN-domain that uses an ECMP algorithm that uses the DSCP. NB ECMP algos >are proprietary to router vendors, ie operator has poor control over >them. > >S3.4 out of band channel - section can be safely deleted. > >S5 Security implications >All these are I think covered by the draft-pcn-architecture, so can >safely delete everything. > >Frequent Typo: >None PCN capable => non PCN capable _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Tue Nov 20 13:09:21 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IuXXF-00083T-1h; Tue, 20 Nov 2007 13:09:21 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IuXXD-000837-LQ for pcn-confirm+ok@megatron.ietf.org; Tue, 20 Nov 2007 13:09:19 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IuXXB-0007j5-SL for pcn@ietf.org; Tue, 20 Nov 2007 13:09:17 -0500 Received: from smtp4.smtp.bt.com ([217.32.164.151]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IuXXB-0003QT-86 for pcn@ietf.org; Tue, 20 Nov 2007 13:09:17 -0500 Received: from i2kc08-ukbr.domain1.systemhost.net ([193.113.197.71]) by smtp4.smtp.bt.com with Microsoft SMTPSVC(6.0.3790.1830); Tue, 20 Nov 2007 18:09:16 +0000 Received: from cbibipnt08.iuser.iroot.adidom.com ([147.149.100.81]) by i2kc08-ukbr.domain1.systemhost.net with Microsoft SMTPSVC(6.0.3790.1830); Tue, 20 Nov 2007 18:09:15 +0000 Received: From bagheera.jungle.bt.co.uk ([132.146.168.158]) by cbibipnt08.iuser.iroot.adidom.com (WebShield SMTP v4.5 MR1a P0803.399); id 1195582154611; Tue, 20 Nov 2007 18:09:14 +0000 Received: from mut.jungle.bt.co.uk ([10.215.130.87]) by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id lAKI99J3019778; Tue, 20 Nov 2007 18:09:12 GMT Message-Id: <5.2.1.1.2.20071120171112.02744e10@pop3.jungle.bt.co.uk> X-Sender: rbriscoe@pop3.jungle.bt.co.uk X-Mailer: QUALCOMM Windows Eudora Version 5.2.1 Date: Tue, 20 Nov 2007 18:08:55 +0000 To: Tina TSOU From: Bob Briscoe Subject: Re: [PCN] O+M section In-Reply-To: <01b901c7ced0$91743520$4fa81cac@jys3105121962> Mime-Version: 1.0 X-Spam-Score: -0.034 () ALL_TRUSTED,HTML_10_20,HTML_MESSAGE,MIME_HTML_ONLY X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158 X-OriginalArrivalTime: 20 Nov 2007 18:09:15.0580 (UTC) FILETIME=[761357C0:01C82BA0] X-Spam-Score: 1.7 (+) X-Scan-Signature: 73734d43604d52d23b3eba644a169745 Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0547675383==" Errors-To: pcn-bounces@ietf.org --===============0547675383== Content-Type: text/html; charset="us-ascii" Tina,

I just helped Phil re-write the OAM section of draft-ietf-pcn-architecture-02.txt just posted.

We included most of the suggestions in your text sent back in Jul below, but in a more discursive and less normative form, given this is an architecture doc, rather than an OAM stds track doc (taking general advice from a w-g co-chairs on what the charter wanted on OAM). We also added a fair bit of new material as you will see.

We'd appreciate a review of the new text (if not the whole draft), given people interested in OAM are hard to come by.

-----------
I also wanted to explain the reason for omitting two whole chunks of ideas:
1) Address configuration details
This needs adding to 5.7 on Addressing as we've been discussing offlist. It would be too complex to record all the configuration permutations in the OAM section of this overall architecture draft - that will need doing separately for each different scenario.

2) Per-aggregate stats collected by boundary nodes
These fell into two categories:
- flow-level
- packet-level
Instead, we covered these all under the catch-all paragraph below:

"Monitoring of performance factors measurable from /outside/ the PCN domain will be no different with PCN than with any other packet-based flow admission control system, both at the flow level (blocking probability etc) and the  packet level (jitter [RFC3393][Y.1541], loss rate [RFC4656], mean opinion score [P.800], etc)."

Essentially, we figured that if a domain wasn't collecting any specific stats across the domain without PCN, there would be no need to start collecting them with PCN.

Thus we omitted mention of the specific flow-level per-aggregate and packet-level per-aggregate stats you had in there (flows offered, admitted, terminated, no of packets sent, no of packets received at each marking level, number of reports sent etc).

Essentially the reasoning was that an operator /might/ want to collect any combination of stats, but we couldn't think what they would use these particular ones for. So we confined the discussion to stuff:
- that will be different with PCN
- and that we could describe a reasonable motivation for collecting.
We figured an operator could specify if they wanted all these other possible stats in an invitation to tender, rather than an IETF doc.

------------
I have just noticed that I omitted the useful ideas on recording cumulative durations of faults. These would naturally fit where we talk of recording near-misses and faults in the Fault OAM section. We should add this to the next draft. Sorry about that.

If you feel strongly that anything else we omitted should have been in there, just shout.


Bob

At 15:29 25/07/2007, Tina TSOU wrote:
Hi all,
As promised, attached please find the O+M section which was discussed in the arch designing team.
Hope it can be a basis or starting point as the O+M part in the ML.
 
B. R.
Tina
Messengers:
MSN: tinatsou6@hotmail.com   Yahoo: tina_tsou    Skype: tinaTSOU    Jabber: tina@jabber.org    Google talk: tinatsou6@gmail.com

_______________________________________________
PCN mailing list
PCN@ietf.org
https://www1.ietf.org/mailman/listinfo/pcn

____________________________________________________________________________
Bob Briscoe, <bob.briscoe@bt.com>      Networks Research Centre, BT Research
B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.    +44 1473 645196 --===============0547675383== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn --===============0547675383==-- From pcn-bounces@ietf.org Tue Nov 20 13:44:16 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IuY52-0007Yz-O0; Tue, 20 Nov 2007 13:44:16 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IuY51-0007Yo-Ki for pcn-confirm+ok@megatron.ietf.org; Tue, 20 Nov 2007 13:44:15 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IuY51-0007Yg-B8 for pcn@ietf.org; Tue, 20 Nov 2007 13:44:15 -0500 Received: from smtp3.smtp.bt.com ([217.32.164.138]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IuY4x-0004EI-Ob for pcn@ietf.org; Tue, 20 Nov 2007 13:44:15 -0500 Received: from i2kc06-ukbr.domain1.systemhost.net ([193.113.197.70]) by smtp3.smtp.bt.com with Microsoft SMTPSVC(6.0.3790.1830); Tue, 20 Nov 2007 18:44:11 +0000 Received: from cbibipnt05.iuser.iroot.adidom.com ([147.149.196.177]) by i2kc06-ukbr.domain1.systemhost.net with Microsoft SMTPSVC(6.0.3790.1830); Tue, 20 Nov 2007 18:44:01 +0000 Received: From bagheera.jungle.bt.co.uk ([132.146.168.158]) by cbibipnt05.iuser.iroot.adidom.com (WebShield SMTP v4.5 MR1a P0803.399); id 1195584240212; Tue, 20 Nov 2007 18:44:00 +0000 Received: from mut.jungle.bt.co.uk ([10.215.130.87]) by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id lAKIhnag020220; Tue, 20 Nov 2007 18:43:57 GMT Message-Id: <5.2.1.1.2.20071120181307.03b995f0@pop3.jungle.bt.co.uk> X-Sender: rbriscoe@pop3.jungle.bt.co.uk X-Mailer: QUALCOMM Windows Eudora Version 5.2.1 Date: Tue, 20 Nov 2007 18:44:06 +0000 To: "Tom Taylor" From: Bob Briscoe In-Reply-To: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Spam-Score: -1.36 () ALL_TRUSTED X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158 X-OriginalArrivalTime: 20 Nov 2007 18:44:01.0287 (UTC) FILETIME=[5140F970:01C82BA5] X-Spam-Score: -1.0 (-) X-Scan-Signature: 21c69d3cfc2dd19218717dbe1d974352 Cc: PCN IETF list Subject: [PCN] Re: Replacement of draft-briscoe-pcn-boundary-behav-00.txt with draft-tsou-pcn-boundary-behav-00.txt X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Tom, I just want the PCN list to know what I said offlist - that I don't disagree with the content of this draft /in the right context/, I just needed the intended context to be added around the content. It was intended to be about how PCN could work in a TISPAN RACS environment. In that context it's a valuable doc. But as it stands it reads like it's about PCN boundary node behaviour generically. I didn't want people to think I'd become a convert to the way of thinking described in this draft, for /all/ possible PCN scenarios. Thanks for reversing the previous author list out of internet-drafts so gallantly. Bob At 17:32:12 -0500 Mon, 19 Nov 2007 Tom Taylor wrote: >We were a bit previous in adding Bob's name to the draft without getting >an explicit go-ahead. For that, we apologize to Bob. > >The draft will see a major rewrite to achieve its intended objectives. In >the meantime, some of the questions it raises are receiving a fair amount >of discussion behind the scenes. > >Tom Taylor >>-------- Original Message -------- >> >>At 17:15 19/11/2007, IETF Secretariat wrote: >>As you requested, draft-briscoe-pcn-boundary-behav-00.txt has been marked as >>replaced by draft-tsou-pcn-boundary-behav-00.txt in the IETF Internet-Drafts >>database. >> >>The IETF Secretariat. ____________________________________________________________________________ Bob Briscoe, Networks Research Centre, BT Research B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK. +44 1473 645196 _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Tue Nov 20 15:57:53 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IuaAL-0001VC-5M; Tue, 20 Nov 2007 15:57:53 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IuaAJ-0001Uo-4f for pcn-confirm+ok@megatron.ietf.org; Tue, 20 Nov 2007 15:57:51 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IuaAI-0001Ug-PQ for pcn@ietf.org; Tue, 20 Nov 2007 15:57:50 -0500 Received: from zrtps0kn.nortel.com ([47.140.192.55]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IuaAG-0001yB-EE for pcn@ietf.org; Tue, 20 Nov 2007 15:57:50 -0500 Received: from zcarhxm1.corp.nortel.com (zcarhxm1.corp.nortel.com [47.129.230.97]) by zrtps0kn.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id lAKKvka13712 for ; Tue, 20 Nov 2007 20:57:46 GMT X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Tue, 20 Nov 2007 15:57:30 -0500 Message-ID: <183DD1B052A11A40B76125E42F1CBAAB0FD6E81B@zcarhxm1.corp.nortel.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: I-D Action:draft-babiarz-pcn-explicit-marking-02.txt Thread-Index: AcgrmL3wz6cx45TdQS+ZcxT+bSMMUAAFyClQAAG0elA= From: "Xiao-Gao Liu" To: X-Spam-Score: 0.0 (/) X-Scan-Signature: 8abaac9e10c826e8252866cbe6766464 Subject: [PCN] FW: I-D Action:draft-babiarz-pcn-explicit-marking-02.txt X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org FYI, the updated 3sm simulation draft.=20 Regards, Xiao-Gao Liu -----Original Message----- From: Internet-Drafts@ietf.org [mailto:Internet-Drafts@ietf.org]=20 Sent: November 20, 2007 12:10 PM To: i-d-announce@ietf.org Subject: I-D Action:draft-babiarz-pcn-explicit-marking-02.txt A New Internet-Draft is available from the on-line Internet-Drafts directories. Title : Simulations Results for 3sm Author(s) : J. Babiarz, et al. Filename : draft-babiarz-pcn-explicit-marking-02.txt Pages : 41 Date : 2007-11-19 This document describes the simulation setups and results for testing=20 the Three State PCN Marking approach. Simulations done to date,=20 demonstrate that the three state PCN marking approach has certain=20 ability to support admission control and flow termination of real- time application flows at the congestion point(s) of the PCN-enabled=20 network. The real-time traffic used in the simulation covers voice=20 and video traffic with large and small numbers of flows. =20 A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-babiarz-pcn-explicit-marking-0 2.txt _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Tue Nov 20 21:24:38 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IufGY-0005yK-Al; Tue, 20 Nov 2007 21:24:38 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IufGW-0005yE-Po for pcn-confirm+ok@megatron.ietf.org; Tue, 20 Nov 2007 21:24:36 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IufGW-0005y6-CP for pcn@ietf.org; Tue, 20 Nov 2007 21:24:36 -0500 Received: from imr2.ericy.com ([198.24.6.3]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IufGR-0004SD-Tj for pcn@ietf.org; Tue, 20 Nov 2007 21:24:36 -0500 Received: from eusrcmw750.eamcs.ericsson.se (eusrcmw750.exu.ericsson.se [138.85.77.50]) by imr2.ericy.com (8.13.1/8.13.1) with ESMTP id lAL2OVOG007199 for ; Tue, 20 Nov 2007 20:24:31 -0600 Received: from eusrcmw751.eamcs.ericsson.se ([138.85.77.51]) by eusrcmw750.eamcs.ericsson.se with Microsoft SMTPSVC(6.0.3790.1830); Tue, 20 Nov 2007 20:24:31 -0600 Received: from [147.117.169.183] ([147.117.169.183]) by eusrcmw751.eamcs.ericsson.se with Microsoft SMTPSVC(6.0.3790.1830); Tue, 20 Nov 2007 20:24:30 -0600 From: Steven Blake To: pcn Content-Type: multipart/mixed; boundary="=-GSWpD7hNkcR9MRfb+kBa" Organization: Ericsson IP Infrastructure Date: Tue, 20 Nov 2007 21:24:30 -0500 Message-Id: <1195611870.28291.72.camel@neutrino> Mime-Version: 1.0 X-Mailer: Evolution 2.8.3 (2.8.3-2.fc6) X-OriginalArrivalTime: 21 Nov 2007 02:24:31.0017 (UTC) FILETIME=[A5DB3190:01C82BE5] X-Spam-Score: 0.0 (/) X-Scan-Signature: cf3becbbd6d1a45acbe2ffd4ab88bdc2 Subject: [PCN] IETF 70 *DRAFT* PCN agenda X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org --=-GSWpD7hNkcR9MRfb+kBa Content-Type: text/plain Content-Transfer-Encoding: 7bit Attached is the draft agenda for the Vancouver meeting, also available at: http://www3.ietf.org/proceedings/07dec/agenda/pcn.txt Note that Scott and I are proposing to do something different from the previous meetings: we are only planning to discuss the PCN architecture draft and Kwok's and Georgios' encoding draft. There are several reasons for this, but to summarize: - These drafts address the first two milestones for PCN. According to the working group charter, we are already late on these milestones (and when working groups are late, the chairs don't get their bonus checks). So it is critical that we make progress on resolving the open issues with these documents so that we can move them to working group last call (1). - Some design decisions for future milestones are depending on the resolution of some open issues with these documents, so by making progress now we should be able to accelerate subsequent work. - We had requests to discuss nine drafts in this meeting, some of which just recently popped into existence with no prior discussion on the mailing list. With only 120 minutes of meeting time, there is not enough time to facilitate good discussions for all of these drafts. So we plan to spend our precious face time on discussion of the open issues that we know we need to resolve as soon as possible. Philip and Kwok have volunteered to each post a list of open issues with their drafts. We should review and refine these lists prior to the meeting, and of course continue to discuss the open issues. In the meeting we will let Philip and Kwok drive the discussion, but anyone with anything to say on these issues will be welcome to queue up for mic and screen time (if you do produce slides (3 max, please), then please send them to the list before the meeting). As always, all working group decisions are finalized on the mailing list. Please send comments on this draft agenda to the list ASAP. (1) draft-chan-pcn-encoding-comparison-01 is not a working group document, so it has no official status. In the absence of any alternative drafts, we need to make progress on this one. (2) If any author feels that comprehension of his or her draft would benefit from slides and/or oral commentary, feel free to post those to the list (and we can arrange hosting for these if necessary). See you in Vancouver! Scott & Steve =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Steven Blake Ericsson/Redback Networks +1 919-472-9913 --=-GSWpD7hNkcR9MRfb+kBa Content-Disposition: attachment; filename=pcn-ietf70-agenda.txt Content-Type: text/plain; name=pcn-ietf70-agenda.txt; charset=UTF-8 Content-Transfer-Encoding: 7bit **DRAFT** Congestion and Pre-Congestion Notification WG (pcn) Monday, December 3, 2007 1740-1950 Afternoon Session III Salon 2 ==================================== CHAIRs: Scott Bradner Steven Blake AGENDA: o Administrivia chairs 10 min - Blue sheets - Scribe - Agenda bash - Milestones status o Discuss open issues in draft-ietf-pcn-architecture-02 all 75 min o Discuss open issues in draft-chan-pcn-encoding-comparison-01 all 45 min --=-GSWpD7hNkcR9MRfb+kBa Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn --=-GSWpD7hNkcR9MRfb+kBa-- From pcn-bounces@ietf.org Tue Nov 20 22:46:49 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IugY3-0000Hg-A7; Tue, 20 Nov 2007 22:46:47 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IugY2-0000Hb-Sh for pcn-confirm+ok@megatron.ietf.org; Tue, 20 Nov 2007 22:46:46 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IugY2-0000HT-Ig for pcn@ietf.org; Tue, 20 Nov 2007 22:46:46 -0500 Received: from zrtps0kn.nortel.com ([47.140.192.55]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IugXz-0006Sv-SK for pcn@ietf.org; Tue, 20 Nov 2007 22:46:46 -0500 Received: from zcarhxm1.corp.nortel.com (zcarhxm1.corp.nortel.com [47.129.230.97]) by zrtps0kn.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id lAL3kfr12734; Wed, 21 Nov 2007 03:46:41 GMT X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [PCN] I-D Action:draft-ietf-pcn-architecture-02.txt Date: Tue, 20 Nov 2007 22:46:28 -0500 Message-ID: <9671A92C3C8B5744BC97F855F7CB646513422A62@zcarhxm1.corp.nortel.com> In-Reply-To: <75A199C5D243C741BF3D3F1EBCEF9BA503B34377@E03MVZ1-UKDY.domain1.systemhost.net> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] I-D Action:draft-ietf-pcn-architecture-02.txt Thread-Index: AcgqxqGl81rUg3oXSxKQPQt45B53dQAATG8gAEoL73A= References: <75A199C5D243C741BF3D3F1EBCEF9BA503B34377@E03MVZ1-UKDY.domain1.systemhost.net> From: "Jozef Babiarz" To: , X-Spam-Score: 1.1 (+) X-Scan-Signature: 6ffdee8af20de249c24731d8414917d3 Cc: X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Phil wrote: >On (2) I tried to capture all the recent discussion, hopefully fairly >successfully & not too controversially? Also, because it was getting >quite long I created a new section for all Probing text (S7). Personally >I wonder if this is now getting too long & with too much discussion (of >options and the circumstances when it might be a good idea or not) - so >I wonder if it should be removed to another draft. Joe, I remember you >saying you were (thinking of/) writing a new draft on probing, not sure >if it would fit with that [sorry not to ask you before, but I only >decided to write it at the weekend] [Joe]No Phil I did not write a draft of probing. However we added a section into 3sm draft the uses on-path signalling (such as RSVP) as a probe for admission control. I will review the new section in the architecture draft on probing and send any comments to the list.=20 Regards, Joe email:babiarz@nortel.com Telephone:613-763-6098 -----Original Message----- From: philip.eardley@bt.com [mailto:philip.eardley@bt.com]=20 Sent: November 19, 2007 11:39 AM To: pcn@ietf.org Subject: RE: [PCN] I-D Action:draft-ietf-pcn-architecture-02.txt I updated the architecture draft with (I think) all the comments over the last month.=20 The main changes are:- (1) the OAM section is extensively revised, thanks to Bob Briscoe. Could another OAM person give this section a review please? (Tom T??) (2) the probing section is extended.=20 And a change to:- (3) discussion & rule for 'partially PCN-capable tunnels' (S5.7) On (2) I tried to capture all the recent discussion, hopefully fairly successfully & not too controversially? Also, because it was getting quite long I created a new section for all Probing text (S7). Personally I wonder if this is now getting too long & with too much discussion (of options and the circumstances when it might be a good idea or not) - so I wonder if it should be removed to another draft. Joe, I remember you saying you were (thinking of/) writing a new draft on probing, not sure if it would fit with that [sorry not to ask you before, but I only decided to write it at the weekend] Other minor changes=20 http://tools.ietf.org/rfcdiff?url1=3Dhttp://tools.ietf.org/id/draft-ietf-= p cn-architecture-02.txt&url2=3Dhttp://tools.ietf.org/id/draft-ietf-pcn-arc= h itecture-01.txt=20 Best wishes, Phil/ > -----Original Message----- > From: Internet-Drafts@ietf.org [mailto:Internet-Drafts@ietf.org] > Sent: 19 November 2007 15:20 > To: i-d-announce@ietf.org > Cc: pcn@ietf.org > Subject: [PCN] I-D Action:draft-ietf-pcn-architecture-02.txt >=20 > A New Internet-Draft is available from the on-line Internet-Drafts > directories. > This draft is a work item of the Congestion and Pre-Congestion > Notification Working Group of the IETF. >=20 >=20 > Title : Pre-Congestion Notification Architecture > Author(s) : P. Eardley > Filename : draft-ietf-pcn-architecture-02.txt > Pages : 45 > Date : 2007-11-19 >=20 > The purpose of this document is to describe a general architecture > for flow admission and termination based on aggregated pre-congestion > information in order to protect the quality of service of established > inelastic flows within a single DiffServ domain.Status >=20 > A URL for this Internet-Draft is: > http://www.ietf.org/internet-drafts/draft-ietf-pcn-architecture-02.txt >=20 > To remove yourself from the I-D Announcement list, send a message to > i-d-announce-request@ietf.org with the word unsubscribe in the body of > the message. > You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce > to change your subscription settings. >=20 > Internet-Drafts are also available by anonymous FTP. Login with the > username "anonymous" and a password of your e-mail address. After > logging in, type "cd internet-drafts" and then > "get draft-ietf-pcn-architecture-02.txt". >=20 > A list of Internet-Drafts directories can be found in > http://www.ietf.org/shadow.html > or ftp://ftp.ietf.org/ietf/1shadow-sites.txt >=20 > Internet-Drafts can also be obtained by e-mail. >=20 > Send a message to: > mailserv@ietf.org. > In the body type: > "FILE /internet-drafts/draft-ietf-pcn-architecture-02.txt". >=20 > NOTE: The mail server at ietf.org can return the document in > MIME-encoded form by using the "mpack" utility. To use this > feature, insert the command "ENCODING mime" before the "FILE" > command. To decode the response(s), you will need "munpack" or > a MIME-compliant mail reader. Different MIME-compliant mail readers > exhibit different behavior, especially when dealing with > "multipart" MIME messages (i.e. documents which have been split > up into multiple messages), so check your local documentation on > how to manipulate these messages. >=20 > Below is the data which will enable a MIME compliant mail reader > implementation to automatically retrieve the ASCII version of the > Internet-Draft. _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 22 01:37:21 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iv5ge-0007HR-DE; Thu, 22 Nov 2007 01:37:20 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Iv5gc-0007HG-Lb for pcn-confirm+ok@megatron.ietf.org; Thu, 22 Nov 2007 01:37:18 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iv5gc-0007H7-6P for pcn@ietf.org; Thu, 22 Nov 2007 01:37:18 -0500 Received: from rotterdam.ewi.utwente.nl ([130.89.10.5]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1Iv5gb-00083n-Cp for pcn@ietf.org; Thu, 22 Nov 2007 01:37:17 -0500 Received: from webmail.cs.utwente.nl (janus.ewi.utwente.nl [130.89.10.26]) by rotterdam.ewi.utwente.nl (8.13.6/8.13.6) with SMTP id lAM6bB6U012618 for ; Thu, 22 Nov 2007 07:37:13 +0100 (MET) Received: from 84.82.109.231 (auth. user karagian@imap1.ewi.utwente.nl) by webmail.cs.utwente.nl with HTTP; Thu, 22 Nov 2007 06:37:10 +0000 To: pcn Date: Thu, 22 Nov 2007 06:37:10 +0000 X-Mailer: IlohaMail/0.8.13 (On: webmail.cs.utwente.nl) Message-ID: From: "Georgios Karagiannis" Bounce-To: "Georgios Karagiannis" MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.032 () AWL X-Scanned-By: MIMEDefang 2.52 on 130.89.10.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0rc3 (rotterdam.ewi.utwente.nl [130.89.10.5]); Thu, 22 Nov 2007 07:37:14 +0100 (MET) X-Spam-Score: 0.0 (/) X-Scan-Signature: d8ae4fd88fcaf47c1a71c804d04f413d Subject: [PCN] comments on draft-ietf-pcn-architecture-02 X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Phil, Hi all I think that version 02 of the the PCN architecture draft is good! However, I have three comments: Comment_1: I could not find information in the draft that mentions the below issue given in the PCN charter: "The WG may also consider to investigate additional response mechanisms that act on (pre-)congestion information. One example could be flow-rate adaptation (rather than flow admission/ termination) during times of congestion." Please include a paragraph that discusses this! ------------------------------------------- Comment_2: Section 3.5: You mention: "The following two assumptions apply if the PCN WG decides to encode PCN-marking in the ECN-field. o It is assumed that PCN-nodes do not perform ECN, [RFC3168], on PCN-packets. o If a packet that is part of a PCN-flow arrives at a PCN-ingress- node with its CE (Congestion experienced) codepoint set, then we assume that the PCN-ingress-node drops the packet. After its initial Charter is complete, the WG may decide to work on a mechanism (such as through a signalling extension) that enables ECN-marking to be carried transparently across the PCN-domain." If I understand the two above assumptions well, they say that traffic that is arriving into the PCN domain and is end-to-end ECN aware can only keep its end-to-end ECN semantics if it is tunnelled. In my opinion this is one possible implementation. There might also be other implementation. I think that this assumption should refer to the requirements described in RFC 4774: http://www.ietf.org/rfc/rfc4774.txt?number=3D4774 Please use the below paragraph instead the one that is given above: "The following two assumptions apply if the PCN WG decides to encode PCN-marking in the ECN-field. o It is assumed that the requirements imposed by RFC 4774 are fulfilled." ------------------- Comment_3: In Section 3, page 29, you provide the below conclusion on the viewpoint that admission control is always done by probing: "The first point breaks Assumption 3 (aggregation) and hence means that this viewpoint is out of scope of initial Charter of the PCN WG." I do not agree with this conclusion, since I do not agree with the argumentation that the first point breaks Assumption 3 more than the situation that no probing is used during admission control. Therefore, please remove the sentence: "The first point breaks Assumption 3 (aggregation) and hence means that this viewpoint is out of scope of initial Charter of the PCN WG." Best regards, Georgios it will be better to have the follwoing assumption instead of _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 22 02:08:17 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iv6AZ-0003uN-CW; Thu, 22 Nov 2007 02:08:15 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Iv6AX-0003sv-U5 for pcn-confirm+ok@megatron.ietf.org; Thu, 22 Nov 2007 02:08:13 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iv6AW-0003pU-M8 for pcn@ietf.org; Thu, 22 Nov 2007 02:08:12 -0500 Received: from rotterdam.ewi.utwente.nl ([130.89.10.5]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1Iv6AV-0000Wd-6d for pcn@ietf.org; Thu, 22 Nov 2007 02:08:12 -0500 Received: from webmail.cs.utwente.nl (janus.ewi.utwente.nl [130.89.10.26]) by rotterdam.ewi.utwente.nl (8.13.6/8.13.6) with SMTP id lAM784oD016084; Thu, 22 Nov 2007 08:08:04 +0100 (MET) Received: from 84.82.109.231 (auth. user karagian@imap1.ewi.utwente.nl) by webmail.cs.utwente.nl with HTTP; Thu, 22 Nov 2007 07:08:03 +0000 To: "philip.eardley@bt.com" , "khchan@nortel.com" , "pcn@ietf.org" Subject: RE: [PCN] Fwd: I-D Action:draft-chan-pcn-encoding-comparison-01.txt Date: Thu, 22 Nov 2007 07:08:03 +0000 X-Mailer: IlohaMail/0.8.13 (On: webmail.cs.utwente.nl) Message-ID: In-Reply-To: <75A199C5D243C741BF3D3F1EBCEF9BA503B3437C@E03MVZ1-UKDY.domain1.systemhost.net> From: "Georgios Karagiannis" Bounce-To: "Georgios Karagiannis" MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.082 () AWL,FVGT_u_HAS_2LETTERFLDR X-Scanned-By: MIMEDefang 2.52 on 130.89.10.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0rc3 (rotterdam.ewi.utwente.nl [130.89.10.5]); Thu, 22 Nov 2007 08:08:10 +0100 (MET) X-Spam-Score: 1.8 (+) X-Scan-Signature: 58b614506802734014829a093beb6879 Cc: X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Phil Thanks for your comments! Please see in line! Best regards, Georgios On 11/20/2007, "philip.eardley@bt.com" wrote: >Kwok, Georgios, all, > >Here are some comments on this draft >http://www.ietf.org/internet-drafts/draft-chan-pcn-encoding-comparison-0 >1.txt - it's much better than 00! >Sorry for the lengthy email >Thanks, >Phil/ > >1. Introduction >Good. > >2. Encoding requirements >First 2 paras good. >last para /bullets can be improved & I do not agree with "A total of six >required encoding states". Suggested replacement: ><< >Different PCN-marking behaviour documents have suggested different >encoding states are needed (2, 3 or 4 of the ones below). In total, the >following possibilities have been suggested: > >o Not pre-congested Marking, for indication of No pre-Congestion >o Admission Marking, for indication of Flow Admission Information >o Termination Marking, for indication of Flow Termination Information >o Single Marking, for indication of pre-congestion Information >o Affected Marking, also for indication of Flow Termination Information > >In addition there must be a way of distinguishing PCN-packets (whether >PCN-marked or not) from non PCN-pkts. > >NOTE: >o Single Marking: this has been suggested by [ref]. a single marking is >used to indicate both admission and termination marking >o Affected Marking: this has been suggested by [ref]. Its suggestion is >that a PCN-node which is termin-marking some pkts (ie is in the >termin-marking state) encodes the rest of the PCN-pkts with the affected >marking. >o the WG will select which of the 5 possible encoding states listed >will actually be supported, or if the choice is a deployment choice for >the operator or under the control of an OAM Configuration system option >(eg where an operator employs either adm-marking or termin-marking but >not both in the PCN-domain). >>> Georgios: Agree with the above, except that Single Marking is vague, since it could have different meaning for different situations. Insome situations is used to identify AM and/or TM and in others is used to identify AM and/or TM and a way to distinguish PCn packets from non PCN-pkts. But if the PCN WG is willing to include SM is okay for me! > >a few words about the above. >I think have to talk about single marking - calling it AM/TM in the >tables is confusing (for me). >I think it's better to say there has to be a way of distinguishing >PCN-packets & non PCN-pkts, rather than saying there has to be an extra >encoding state. >I don't think nonce cheater detection is relevant [I think you're >talking about ECN nonce. It's fair enough to talk at the appropriate >moments about loss of ECN info if the ECN field is used to encode PCN. >but I think it's confusing (and on occasions, in some of your encoding >tables, wrong] to list ECN nonce as a PCN encoding state. Or are you >suggesting a *PCN* nonce? (would seem this isn't needed, as PCN-domain >is trusted. However, maybe thinking about future extensions?) Georgios: The ECN nonce discussion is associated with Section 4.3 in: http://www.ietf.org/rfc/rfc4774.txt?number=3D4774 We have to clarify this in the text. > >New sub-section 2.2 >I wonder if it's worth introducing what will be the evaluation criteria, >eg: >How easy is it to support multiple PCN classes ? >Are there any interactions with ecmp? >Are there any interactions with ecn? >Codepoint shortage for mpls. >etc Georgios: You are right, we had such criteria in the previous version, but they were skipped during this version. I think is important to include criteria! Are these 4 enough? > > >Section 3 Encoding options >The first few lines (before bullets) could be shortened to: >"There are various encoding options, which can be grouped by which >fields they use:" >You can add a note here that "using a different channel (e.g., IPFIX .." >is out of scope of WG. I believe this is agreed. Section 3.4 can >therefore be safely deleted. Georgios: Agree! > >The list of abbreviations can be tweaked: >Add: >SM single marking ; >Not-pc [or something meaning 'not pre-congested'] > >Delete: Not-ce, NDS-CE Georgios: The above can be applied if the PCN WG decides to use SM as additional encoding state, see above. > >Section 3.1+ >Think the "ecn & dscp" [current 3.1] needs the most explanation, so I >don't think it should be done first. suggest swapping with 3.3. Georgios: Agree! > >3.1 ECN & DSCP: >I found the first part of this [before 3.1.1] too dense. I think more >options need to be listed. I think the terminology in the table is >confusing, replace: >----------------------------------------------------------------------- >| ECN Bits || 00 | 01 | 10 | 11 || DSCP >| >|=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >| >| RFC 3168 || Not-ECT | ECT(1) | ECT(0) | CE || NA >| >|=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >| >| Option 1 || AM | NA | NA | TM || PCN >| >|--------------++----------+----------+----------+----------++---------- >| >| Option 2 || Not-PC | NA | NA | SM || PCN >| >|--------------++----------+----------+----------+----------++---------- >| >| Option 3 || NA |Not-PC(A) | Not-PC(T)| SM || PCN >| >|--------------++----------+----------+----------+----------++---------- >| >| Option 4 ...... > ----------------------------------------------------------------------- > Georgios: I think that this can be replaced if the PCN WG decides to use SM as an additional state. >In Option 3 I tried to work out what you meant by ECT(A) & ECT(T). I >think you're saying that the PCN-ingress-node sets the pkt with a mix of >2 codepoints; and only Single Marking state is used to indicate >pre-congestion; a pcn-node that wants to adm mark a pkt then selects a >Not-PC(A) pkt and re-marks it to SM. Is that right? need to explain why >this might be useful. Georgios: We will clarify this, but please note that the option is taken from: http://www.watersprings.org/pub/id/draft-briscoe-tsvwg-cl-phb-03.txt In particular see Alternative 2 from the above given draft. > >There was an option once mentioned [in a draft-briscoe-architecture I >think] of using a 2nd DSCP to indicate TM (or perhaps to indicate >Affected marking). Georgios: Agree! > >I also think you can talk separately about the Affected marking. In my >table, AFM can obviously be added to all 3 Options Georgios: Agree! > >I also think there's a separate series of options where you're trying to >preserve ECN information on an end-to-end basis whilst having PCN >running in some domain in the middle. And you don't want to solve this >by doing something else like tunnelling over the PCN-domain. I think >this needs separate discussion as the issues are different and quite >hard to think about. For instance, there are two separate cases: just >preserving the Not-CE & CE info, and also preserving the ECT-nonce info >[there are different levels of interest/deployment/standardisation]. >So I think it would be better to have a separate section discussing >encoding options when ECN info needs to be preserved. Georgios: Agree, but this is what we wanted to emphasize in Section 3.1.1.1! We could provide a disussion on ECN tunelling though in an additional section. > > >3.1.1 >could expand with something like: >For example, this means that PCN can easily be applied to multiple >classes. Georgios: Okay! > >3.1.1.1, [RFC4774] interactions: >as well as the things you talk about, I thought [not sure though] that >4774 was concerned about "what happens if things go wrong", eg: >- a RFC4774 pkt gets accidentally dscp-re-marked to the pcn-dscp; a >PCN-ingress-node gets misconfigured so it allows a RFC4774 pkt in on the >pcn-dscp. Can anything nasty happen? >- a router in the PCN-domain misconfigures itself so it's >RFC3246-enabled (ie no longer PCN-enabled). Can anything nasty happen, >eg when PCN-marked pkts encounter this router? >- a PCN-egress-node gets misconfigured so it allows a pcn-marked pkt >'out' of the PCN-domain. > >eg if the first case happens, may want to try to minimise the impact of >any ECN-CE pkts that therefore sneak in on the PCN class Georgios: I think that the issues that you mentioned are related to Issue 4 given in Section 3.1.1.1. This means that it will be good if we will work out Issue 4. At the moment we avoid the discussion of this issue in the draft. >3.1.2 >"possibly adding complexity when tunneling of the PCN encoding is >required" - explain. Georgios: Agree! > >3.2 using ECN field >if I understand this case right, you're saying that you're NOT using a >different DSCP to distinguish PCN traffic from non-PCN traffic. Think >you need to make this really clear, as it's quite bizarre. However, as >draft-pcn-architecture says "There needs to be a way for a PCN-node to >distinguish PCN-traffic from non PCN-traffic" - I don't understand how >you propose to do this, although I could imagine an Option like this: >|=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >| >| Option 1 || Not-CE | Not-PC | SM | CE || NA >| >|--------------++----------+----------+----------+----------++---------- >| >One downside of this is that there can only be 1 PCN class [PHB] > >basically I don't understand the table. Georgios: We will clarify this, but please note that the option is taken from: http://www.watersprings.org/pub/id/draft-briscoe-tsvwg-cl-phb-03.txt In particular see Alternative 2 and 5 from the above given draft. > > >3.3 DSCP >in table, I think you should change things like "PCN/AM/TM" to SM, eg: Georgios: If the PCN WG decides to use SM as additional state then OKAY! >----------------------------------------------------------------------- >| DSCP Bits || Original |Experimental 1 |Experimental 2 |Experimental 3 >| >|=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >| >| Option 7 || Not-PC | SM | NA | NA >| >|-----------++----------+---------------+---------------+--------------- >| > >3.3.1 Benefits >" concerns raised in RFC 4774 [20] are not applicable" - pedantic point: >I think it's really that they should be easier to deal with [whether or >not they're applicable depends on whether a PCN-node knows how to >process a pkt with the ECN field set] Georgios: I do not understand your argumentation: IF ECN is not used within the PCN domain then the requirements imposed by RFC 4774 do not apply! > >" all 4 DSCP encoding options depicted in Figure 3 can support the not >congested indication, PCN capable transport marking, the admission >control and flow termination encoding states." Don't understand this, >only true for options 9 & 10. Georgios: Agree, needs editing! > >"In addition Option 8 and 10 can in addition support the ECMP solution." >Replace: "They support Affected marking, which [ref] says has benefits >if the PCN-domain operates ECMP multi-path routing" > >additional benefit (and in my view the most significant): exptal dscps >are lightly standardised - limited rules on how you use them, largely up >to the operator. Georgios: Agree! > >3.3.2 Drawbacks >clarify by replacing >"This type of encoding needs to use per PHB, in addition to the original >DSCP and depending on the encoding option used, one, two or three DSCP >values, respectivelly." >With "This type of encoding needs to use, for each DS class (PHB) that >is PCN-enabled, 2 3 or 4 DSCP values, depending on the encoding option." > >Clarify that you need this set of 2/3/4 dscp values for each class >that's PCN-enabled. Georgios: Agree! > >Clarify that particular issue for mpls [codepoint shortage] Georgios: Agree! > >"Furthermore, if the separation between the PCN traffic and non- PCN >traffic is required". According to draft-pcn-architecture S4.5, there is >no "if" about it. Georgios: Agree! > >last para repeats what already said. Georgios: Okay! > >Major disadvantage: interactions with ECMP. ECMP algos can use a >packet's DSCP when deciding what interface to fwd it on. In a PCN-domain >running ECMP, for a flow which has some packets PCN-marked and some >unmarked. its packets will travel over more than one path. This may well >lead to mis-ordering. >To me, this means that the S3.3 DSCP option is a non-starter in any >PCN-domain that uses an ECMP algorithm that uses the DSCP.=20 Georgios: well I do not see this as a disadvantage. However, what you mention about having an ECMP algorithm that uses DSCP is relevant. Thus what we can do is to include a sentence in that emphasizes that the ECMP situation is solved only if the ECMP algorithm does not use DSCP during the implementation of the equal cost multi path algorithm. > NB ECMP algos >are proprietary to router vendors, ie operator has poor control over >them. Georgios: I am not sure about the above! An operator could at least disallow the ECN option. > >S3.4 out of band channel - section can be safely deleted. georgios: Agree! > >S5 Security implications >All these are I think covered by the draft-pcn-architecture, so can >safely delete everything. Georgios: Agree! > >Frequent Typo: >None PCN capable =3D> non PCN capable Georgios: Okay! > > _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 22 02:58:58 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iv6xd-0004im-T1; Thu, 22 Nov 2007 02:58:57 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Iv6xb-0004aJ-KU for pcn-confirm+ok@megatron.ietf.org; Thu, 22 Nov 2007 02:58:55 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iv6xU-0004X5-HB for pcn@ietf.org; Thu, 22 Nov 2007 02:58:48 -0500 Received: from smtp1.smtp.bt.com ([217.32.164.137]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1Iv6xT-00028e-TW for pcn@ietf.org; Thu, 22 Nov 2007 02:58:48 -0500 Received: from E03MVZ1-UKDY.domain1.systemhost.net ([193.113.30.65]) by smtp1.smtp.bt.com with Microsoft SMTPSVC(6.0.3790.1830); Thu, 22 Nov 2007 07:58:46 +0000 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Thu, 22 Nov 2007 07:58:46 -0000 Message-ID: <75A199C5D243C741BF3D3F1EBCEF9BA503B34391@E03MVZ1-UKDY.domain1.systemhost.net> In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Encoding PCN-markings into DSCPs - are there any interactions with ECMP? Thread-Index: Acgs1noHJ7z7sHXCTi+2W8eLihIpwAABlPhQ From: To: X-OriginalArrivalTime: 22 Nov 2007 07:58:46.0461 (UTC) FILETIME=[823ED6D0:01C82CDD] X-Spam-Score: 0.0 (/) X-Scan-Signature: bb8f917bb6b8da28fc948aeffb74aa17 Subject: [PCN] Encoding PCN-markings into DSCPs - are there any interactions with ECMP? X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi, Have extracted this issue as a separate thread. The context of the discussion below is encoding PCN-markings into different DSCPs (ie not touching the ECN field). Question is, if you do this are there any nasty interactions with ECMP. Any comments? phil > > > >Major disadvantage: interactions with ECMP. ECMP algos can use a > >packet's DSCP when deciding what interface to fwd it on. In a PCN-domain > >running ECMP, for a flow which has some packets PCN-marked and some > >unmarked. its packets will travel over more than one path. This may well > >lead to mis-ordering. > >To me, this means that the S3.3 DSCP option is a non-starter in any > >PCN-domain that uses an ECMP algorithm that uses the DSCP. >=20 > Georgios: well I do not see this as a disadvantage. However, what you > mention about having an ECMP algorithm that uses DSCP is relevant. > Thus what we can do is to include a sentence in that emphasizes that the > ECMP situation is solved only if the ECMP algorithm does not use DSCP > during the implementation of the equal cost multi path algorithm. >=20 > > NB ECMP algos > >are proprietary to router vendors, ie operator has poor control over > >them. >=20 > Georgios: I am not sure about the above! An operator could at least > disallow the ECN option. >=20 >=20 > _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 22 03:26:13 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iv7O0-0000bz-SE; Thu, 22 Nov 2007 03:26:12 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Iv7Nz-0000ZA-W2 for pcn-confirm+ok@megatron.ietf.org; Thu, 22 Nov 2007 03:26:11 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iv7Nz-0000Yr-Ht for pcn@ietf.org; Thu, 22 Nov 2007 03:26:11 -0500 Received: from rotterdam.ewi.utwente.nl ([130.89.10.5]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1Iv7Ny-0003Hh-Va for pcn@ietf.org; Thu, 22 Nov 2007 03:26:11 -0500 Received: from webmail.cs.utwente.nl (janus.ewi.utwente.nl [130.89.10.26]) by rotterdam.ewi.utwente.nl (8.13.6/8.13.6) with SMTP id lAM8Q06L025550; Thu, 22 Nov 2007 09:26:04 +0100 (MET) Received: from 84.82.109.231 (auth. user karagian@imap1.ewi.utwente.nl) by webmail.cs.utwente.nl with HTTP; Thu, 22 Nov 2007 08:25:59 +0000 To: "philip.eardley@bt.com" , "pcn@ietf.org" Subject: Re: [PCN] Encoding PCN-markings into DSCPs - are there any interactions with ECMP? Date: Thu, 22 Nov 2007 08:25:59 +0000 X-Mailer: IlohaMail/0.8.13 (On: webmail.cs.utwente.nl) Message-ID: In-Reply-To: <75A199C5D243C741BF3D3F1EBCEF9BA503B34391@E03MVZ1-UKDY.domain1.systemhost.net> From: "Georgios Karagiannis" Bounce-To: "Georgios Karagiannis" MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.032 () AWL X-Scanned-By: MIMEDefang 2.52 on 130.89.10.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0rc3 (rotterdam.ewi.utwente.nl [130.89.10.5]); Thu, 22 Nov 2007 09:26:04 +0100 (MET) X-Spam-Score: 0.0 (/) X-Scan-Signature: 5a9a1bd6c2d06a21d748b7d0070ddcb8 Cc: X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi all I have seen that I made a typing error on the last sentence! It should be: Georgios: I am not sure about the above! An operator could at least disallow the ECMP option. Best regards, Georgios On 11/22/2007, "philip.eardley@bt.com" wrote: >Hi, >Have extracted this issue as a separate thread. >The context of the discussion below is encoding PCN-markings into >different DSCPs (ie not touching the ECN field). Question is, if you do >this are there any nasty interactions with ECMP. >Any comments? >phil > >> > >> >Major disadvantage: interactions with ECMP. ECMP algos can use a >> >packet's DSCP when deciding what interface to fwd it on. In a >PCN-domain >> >running ECMP, for a flow which has some packets PCN-marked and some >> >unmarked. its packets will travel over more than one path. This may >well >> >lead to mis-ordering. >> >To me, this means that the S3.3 DSCP option is a non-starter in any >> >PCN-domain that uses an ECMP algorithm that uses the DSCP. >>=20 >> Georgios: well I do not see this as a disadvantage. However, what you >> mention about having an ECMP algorithm that uses DSCP is relevant. >> Thus what we can do is to include a sentence in that emphasizes that >the >> ECMP situation is solved only if the ECMP algorithm does not use DSCP >> during the implementation of the equal cost multi path algorithm. >>=20 >> > NB ECMP algos >> >are proprietary to router vendors, ie operator has poor control over >> >them. >>=20 >> Georgios: I am not sure about the above! An operator could at least >> disallow the ECN option. >>=20 >>=20 >> > > >_______________________________________________ >PCN mailing list >PCN@ietf.org >https://www1.ietf.org/mailman/listinfo/pcn _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 22 03:39:58 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iv7bH-0007GY-IR; Thu, 22 Nov 2007 03:39:55 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Iv7bG-0007GO-3f for pcn-confirm+ok@megatron.ietf.org; Thu, 22 Nov 2007 03:39:54 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iv7bF-0007F7-Ku for pcn@ietf.org; Thu, 22 Nov 2007 03:39:53 -0500 Received: from tcmail31.telekom.de ([217.6.95.238]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1Iv7bF-0003iT-0s for pcn@ietf.org; Thu, 22 Nov 2007 03:39:53 -0500 Received: from s4de8psaans.mitte.t-com.de by tcmail31.telekom.de with ESMTP; Thu, 22 Nov 2007 09:39:50 +0100 Received: from S4DE8PSAANK.mitte.t-com.de ([10.151.229.10]) by s4de8psaans.mitte.t-com.de with Microsoft SMTPSVC(6.0.3790.3959); Thu, 22 Nov 2007 09:39:49 +0100 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Subject: RE: [PCN] Encoding PCN-markings into DSCPs - are there anyinteractions with ECMP? Date: Thu, 22 Nov 2007 09:39:49 +0100 Message-Id: <1B6169C658325341A3B8066E23919E1C4C14D5@S4DE8PSAANK.mitte.t-com.de> In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] Encoding PCN-markings into DSCPs - are there anyinteractions with ECMP? Thread-Index: Acgs4WGvKbHSmuQVRPu3zgIC+pF8DQAAEibw From: "Geib, Ruediger" To: X-OriginalArrivalTime: 22 Nov 2007 08:39:49.0881 (UTC) FILETIME=[3E8EE690:01C82CE3] X-Spam-Score: 0.0 (/) X-Scan-Signature: 25620135586de10c627e3628c432b04a Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Georgios, if you want to convince Deutsche Telekom's backbone engineering team to=20 turn off ECMP, you must offer them very convincing arguments. I'd=20 suggest the PCN WG to expect or allow ECMP to be active within a=20 PCN network, at least if is a carrier network. Regards, Rudiger |Georgios: I am not sure about the above! An operator could at least |disallow the ECMP option. | |Best regards, |Georgios | | | | |On 11/22/2007, "philip.eardley@bt.com" wrote: | |>Hi, |>Have extracted this issue as a separate thread. |>The context of the discussion below is encoding PCN-markings into |>different DSCPs (ie not touching the ECN field). Question is,=20 |if you do |>this are there any nasty interactions with ECMP. |>Any comments? |>phil |> |>> > |>> >Major disadvantage: interactions with ECMP. ECMP algos can use a |>> >packet's DSCP when deciding what interface to fwd it on. In a |>PCN-domain |>> >running ECMP, for a flow which has some packets PCN-marked and some |>> >unmarked. its packets will travel over more than one path. This may |>well |>> >lead to mis-ordering. |>> >To me, this means that the S3.3 DSCP option is a non-starter in any |>> >PCN-domain that uses an ECMP algorithm that uses the DSCP. |>>=20 |>> Georgios: well I do not see this as a disadvantage.=20 |However, what you |>> mention about having an ECMP algorithm that uses DSCP is relevant. |>> Thus what we can do is to include a sentence in that emphasizes that |>the |>> ECMP situation is solved only if the ECMP algorithm does=20 |not use DSCP |>> during the implementation of the equal cost multi path algorithm. |>>=20 |>> > NB ECMP algos |>> >are proprietary to router vendors, ie operator has poor=20 |control over |>> >them. |>>=20 |>> Georgios: I am not sure about the above! An operator could at least |>> disallow the ECN option. |>>=20 |>>=20 |>> |> |> |>_______________________________________________ |>PCN mailing list |>PCN@ietf.org |>https://www1.ietf.org/mailman/listinfo/pcn | | |_______________________________________________ |PCN mailing list |PCN@ietf.org |https://www1.ietf.org/mailman/listinfo/pcn | _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 22 04:27:36 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iv8LP-00014e-8n; Thu, 22 Nov 2007 04:27:35 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Iv8LM-000145-QK for pcn-confirm+ok@megatron.ietf.org; Thu, 22 Nov 2007 04:27:32 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iv8LM-00011v-4S for pcn@ietf.org; Thu, 22 Nov 2007 04:27:32 -0500 Received: from smtp.nokia.com ([131.228.20.172] helo=mgw-ext13.nokia.com) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1Iv8LJ-0000WU-HN for pcn@ietf.org; Thu, 22 Nov 2007 04:27:32 -0500 Received: from esebh105.NOE.Nokia.com (esebh105.ntc.nokia.com [172.21.138.211]) by mgw-ext13.nokia.com (Switch-3.2.5/Switch-3.2.5) with ESMTP id lAM9R4uu014663; Thu, 22 Nov 2007 11:27:23 +0200 Received: from esebh103.NOE.Nokia.com ([172.21.143.33]) by esebh105.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.1830); Thu, 22 Nov 2007 11:26:52 +0200 Received: from esebh101.NOE.Nokia.com ([172.21.138.177]) by esebh103.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.1830); Thu, 22 Nov 2007 11:26:52 +0200 Received: from mgw-int01.ntc.nokia.com ([172.21.143.96]) by esebh101.NOE.Nokia.com over TLS secured channel with Microsoft SMTPSVC(6.0.3790.1830); Thu, 22 Nov 2007 11:26:51 +0200 Received: from esdhcp036138.research.nokia.com (esdhcp036138.research.nokia.com [172.21.36.138]) by mgw-int01.ntc.nokia.com (Switch-3.2.5/Switch-3.2.5) with ESMTP id lAM9QoKR021487; Thu, 22 Nov 2007 11:26:50 +0200 Message-Id: From: Lars Eggert To: ext Georgios Karagiannis In-Reply-To: Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v915) Subject: Re: [PCN] comments on draft-ietf-pcn-architecture-02 Date: Thu, 22 Nov 2007 11:26:49 +0200 References: X-Mailer: Apple Mail (2.915) X-OriginalArrivalTime: 22 Nov 2007 09:26:52.0046 (UTC) FILETIME=[D0B31AE0:01C82CE9] X-Nokia-AV: Clean X-Spam-Score: 0.0 (/) X-Scan-Signature: 856eb5f76e7a34990d1d457d8e8e5b7f Cc: pcn X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org On 2007-11-22, at 8:37, ext Georgios Karagiannis wrote: > I could not find information in the draft that mentions the below > issue > given in the PCN charter: > > "The WG may also consider to investigate additional response > mechanisms that act on (pre-)congestion information. One example > could be flow-rate adaptation (rather than flow admission/ > termination) during times of congestion." > > Please include a paragraph that discusses this! The sentence you quote is from a paragraph that starts with "After completion of the initial phase..." Many of the examples given in that paragraph will significantly revamp the PCN architecture, and duscissug them in the initial phase is likely to be unproductive. Lars _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 22 05:29:48 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iv9JZ-0001nH-61; Thu, 22 Nov 2007 05:29:45 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Iv9JY-0001n6-DY for pcn-confirm+ok@megatron.ietf.org; Thu, 22 Nov 2007 05:29:44 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iv9JY-0001mt-0i for pcn@ietf.org; Thu, 22 Nov 2007 05:29:44 -0500 Received: from smtp3.smtp.bt.com ([217.32.164.138]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1Iv9JU-0002sD-8i for pcn@ietf.org; Thu, 22 Nov 2007 05:29:43 -0500 Received: from E03MVZ1-UKDY.domain1.systemhost.net ([193.113.30.65]) by smtp3.smtp.bt.com with Microsoft SMTPSVC(6.0.3790.1830); Thu, 22 Nov 2007 10:29:39 +0000 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [PCN] comments on draft-ietf-pcn-architecture-02 Date: Thu, 22 Nov 2007 10:29:38 -0000 Message-ID: <75A199C5D243C741BF3D3F1EBCEF9BA503B34392@E03MVZ1-UKDY.domain1.systemhost.net> In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] comments on draft-ietf-pcn-architecture-02 Thread-Index: Acgs0ibapJhVEKTeSKaERK5iVAsa7AAHKj+A From: To: , X-OriginalArrivalTime: 22 Nov 2007 10:29:39.0356 (UTC) FILETIME=[9630FDC0:01C82CF2] X-Spam-Score: -1.0 (-) X-Scan-Signature: 00e94c813bef7832af255170dca19e36 Cc: X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Georgios, Thanks for the comments! > Comment_1: Lars has answered. =20 > Comment_2: > Section 3.5: > You mention: > "The following two assumptions apply if the PCN WG decides to encode > PCN-marking in the ECN-field. >=20 > o It is assumed that PCN-nodes do not perform ECN, [RFC3168], on > PCN-packets. >=20 > o If a packet that is part of a PCN-flow arrives at a PCN-ingress- > node with its CE (Congestion experienced) codepoint set, then we > assume that the PCN-ingress-node drops the packet. After its > initial Charter is complete, the WG may decide to work on a > mechanism (such as through a signalling extension) that enables > ECN-marking to be carried transparently across the PCN-domain." >=20 > If I understand the two above assumptions well, they say that traffic > that is arriving into the PCN domain and is end-to-end ECN aware can > only keep its end-to-end ECN semantics if it is tunnelled. > In my opinion this is one possible implementation. There might also be > other implementation.=20 Yes, I agree tunnelling would be one option, and that there would be other implementations. Note the issue of how to carry ECN-marking transparently across the PCN-domain is one that the WG may look at after the initial Charter is done. Note also, the if: "if the PCN WG decides to encode PCN-marking in the ECN-field" [in the terms of your encoding draft, this would include ECN+DSCP option] > I think that this assumption should refer to the > requirements described in RFC 4774: > http://www.ietf.org/rfc/rfc4774.txt?number=3D4774 >=20 > Please use the below paragraph instead the one that is given above: > "The following two assumptions apply if the PCN WG decides to encode > PCN-marking in the ECN-field. >=20 > o It is assumed that the requirements imposed by RFC 4774 are > fulfilled." Yes, I think we could add a ref to 4774 somewhere in the draft, and this seems a good place to do so. Will add as a 3rd bullet something like "the Milestone on 'Survey of Encoding Choices' will consider the implications of [RFC4774]" >=20 > ------------------- >=20 > Comment_3: > In Section 3, page 29, you provide the below conclusion on the viewpoint > that admission control is always done by probing: >=20 > "The first point breaks Assumption 3 (aggregation) and hence means that > this viewpoint is out of scope of initial Charter of the PCN WG." >=20 > I do not agree with this conclusion, since I do not agree with the > argumentation that the first point breaks Assumption 3=20 the "first point" is this: o Simply admitting the new flow has a significant risk of leading to overload, because the PCN-domain reaches out towards the end terminals where link capacity is low. The link capacity is low means that adding a single flow to the link can move it directly from no congestion to overloaded [real queue grows significantly or overflows]. This breaks the aggregation assumption.=20 Best wishes Phil/ _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 22 05:33:03 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iv9Mk-0000no-VV; Thu, 22 Nov 2007 05:33:02 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Iv9Mk-0000ni-2K for pcn-confirm+ok@megatron.ietf.org; Thu, 22 Nov 2007 05:33:02 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iv9Mj-0000na-Of for pcn@ietf.org; Thu, 22 Nov 2007 05:33:01 -0500 Received: from mailgw4.ericsson.se ([193.180.251.62]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1Iv9Me-00030P-CF for pcn@ietf.org; Thu, 22 Nov 2007 05:33:01 -0500 Received: from mailgw4.ericsson.se (unknown [127.0.0.1]) by mailgw4.ericsson.se (Symantec Mail Security) with ESMTP id 98ED621027; Thu, 22 Nov 2007 11:32:55 +0100 (CET) X-AuditID: c1b4fb3e-af6a0bb00000459d-a0-47455ad7712b Received: from esealmw127.eemea.ericsson.se (unknown [153.88.254.122]) by mailgw4.ericsson.se (Symantec Mail Security) with ESMTP id 76C56205C0; Thu, 22 Nov 2007 11:32:55 +0100 (CET) Received: from esealmw129.eemea.ericsson.se ([153.88.254.177]) by esealmw127.eemea.ericsson.se with Microsoft SMTPSVC(6.0.3790.1830); Thu, 22 Nov 2007 11:32:55 +0100 Received: from [147.214.30.247] ([147.214.30.247]) by esealmw129.eemea.ericsson.se with Microsoft SMTPSVC(6.0.3790.1830); Thu, 22 Nov 2007 11:32:54 +0100 Message-ID: <47455AD6.2030404@ericsson.com> Date: Thu, 22 Nov 2007 11:32:54 +0100 From: Magnus Westerlund User-Agent: Thunderbird 2.0.0.9 (Windows/20071031) MIME-Version: 1.0 To: Georgios Karagiannis Subject: ECN support in a PCN domain (was: Re: [PCN] comments on draft-ietf-pcn-architecture-02) References: In-Reply-To: X-Enigmail-Version: 0.95.5 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 22 Nov 2007 10:32:54.0752 (UTC) FILETIME=[0AA80E00:01C82CF3] X-Brightmail-Tracker: AAAAAA== X-Spam-Score: -1.0 (-) X-Scan-Signature: 5a9a1bd6c2d06a21d748b7d0070ddcb8 Cc: pcn X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi, See inline > Section 3.5: > > "The following two assumptions apply if the PCN WG decides to encode > PCN-marking in the ECN-field. > > o It is assumed that PCN-nodes do not perform ECN, [RFC3168], on > PCN-packets. I would like to have a clarification on what is meant with PCN not doing ECN. To me it seems that in situation where a ECN enabled flow goes through a PCN domain and the flow are in a situation where a router actually is congested, it should be able to have that those packets being marked with CE when leaving the PCN domain. I know this is phase two of the work if we ever gets there as it discusses another response mechanism. > > o If a packet that is part of a PCN-flow arrives at a PCN-ingress- > node with its CE (Congestion experienced) codepoint set, then we > assume that the PCN-ingress-node drops the packet. After its > initial Charter is complete, the WG may decide to work on a > mechanism (such as through a signalling extension) that enables > ECN-marking to be carried transparently across the PCN-domain." This one really surprise me. How can you make this assumption? In case a ECN enabled flow with some packets CE marked arrive to the ingress of the PCN domain and are within the limits for which the flow has been admitted into the PCN domain, then the PCN domain has no right to summarily drop that packet as being less important than any of the unmarked packets. Only in cases if congestion actually is experience within the PCN domain would a differential treatment of CE marked packets being warranted. Can someone please explain how the above assumptions still satisfy the following sentence in the charter? "All PCN mechanisms, including transport and encoding of (pre-congestion) information, are required to cleanly integrate with existing architectures and protocols such as DiffServ and ECN." Cheers Magnus Westerlund IETF Transport Area Director & TSVWG Chair ---------------------------------------------------------------------- Multimedia Technologies, Ericsson Research EAB/TVM/M ---------------------------------------------------------------------- Ericsson AB | Phone +46 8 4048287 Torshamsgatan 23 | Fax +46 8 7575550 S-164 80 Stockholm, Sweden | mailto: magnus.westerlund@ericsson.com ---------------------------------------------------------------------- _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 22 05:52:32 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iv9fY-00031f-Fw; Thu, 22 Nov 2007 05:52:28 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Iv9fW-00031Y-Vg for pcn-confirm+ok@megatron.ietf.org; Thu, 22 Nov 2007 05:52:26 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iv9fW-00031Q-Hl for pcn@ietf.org; Thu, 22 Nov 2007 05:52:26 -0500 Received: from tcmail31.telekom.de ([217.6.95.238]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1Iv9fV-0000lB-FW for pcn@ietf.org; Thu, 22 Nov 2007 05:52:26 -0500 Received: from s4de8psaans.mitte.t-com.de by tcmail31.telekom.de with ESMTP; Thu, 22 Nov 2007 11:52:22 +0100 Received: from S4DE8PSAANK.mitte.t-com.de ([10.151.229.10]) by s4de8psaans.mitte.t-com.de with Microsoft SMTPSVC(6.0.3790.3959); Thu, 22 Nov 2007 11:52:22 +0100 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [PCN] I-D Action:draft-ietf-pcn-architecture-01.txt Date: Thu, 22 Nov 2007 11:52:21 +0100 Message-Id: <1B6169C658325341A3B8066E23919E1C4C14EA@S4DE8PSAANK.mitte.t-com.de> In-Reply-To: <75A199C5D243C741BF3D3F1EBCEF9BA503B34377@E03MVZ1-UKDY.domain1.systemhost.net> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] I-D Action:draft-ietf-pcn-architecture-02.txt Thread-Index: AcgqxqGl81rUg3oXSxKQPQt45B53dQAATG8gAIiSlTA= From: "Geib, Ruediger" To: X-OriginalArrivalTime: 22 Nov 2007 10:52:22.0040 (UTC) FILETIME=[C26A1180:01C82CF5] X-Spam-Score: 0.0 (/) X-Scan-Signature: 2ed806e2f53ff1a061ad4f97e00345ac Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Phil, the latest two versions of the architecture are=20 indeed well written. Most of my comments on the=20 chartered topics are editorial (with the=20 exception of ingress policing). And that more=20 discussion is useful on chartered topics is=20 not surprising. I've read the -01 version and checked that my=20 comments still apply with -02. This is to say,=20 that missing comments to the new -02 parts=20 don't mean consent. Regards, Rudiger -----comments------------------------------- 1. Introduction Middle of second paragraph refers to "measurements" "...PCN-egress-nodes make measurements of the=20 packet markings and send information..." Next reference to "measurements" in 4th bullet point: "The information of these measurements is signalled=20 to the PCN-egress-nodes by the PCN-marks in the=20 packet headers." A first time reader my be puzzled by "these", as the=20 last time measurements were explicity mentioned it=20 were those of the egress and then "the egress sends" while now something "is sent to the egress". Proposal:=20 "The congestion status of an interface marking=20 PCN packets is signalled to the PCN-egress..." ------------------------------------------------------ First bullet point, last sentence "It is not generally required that other network entities are aware of individual flows (although they may be in=20 particular deployment scenarios)." A reference would be helpful to give the reader guidance=20 on the particular deployment scenarios. If the above=20 refers to signaling supported by all boundary nodes,=20 "e.g. [I-D.briscoe-tsvwg-cl-architecture]" may be a good=20 reference, if something else is meant, please=20 refer to that something. ------------------------------------------------------ 4th last bullet point "The PCN-domain extends to the end users. NOTE: This=20 isn't necessarily outside the Charter because it may=20 not break Assumption 3 (aggregation see later) if=20 it's known there's sufficient aggregation at any=20 bottleneck,......" Could we add a bullet point on=20 "Extending PCN-pre congestion feedback to the user=20 application. While this currently outside of the charter,=20 providing feedback of the PCN pre-congestion status=20 to user applications is a viable alternative to=20 termination, from a provider point of view as well=20 as from an end user point of view. Rate reduction=20 resulting from codec renegotiation is a typical=20 feature of real time applications optimised for a=20 connectionless Internet. Once load levels approach=20 the PCN-upper-rate, user applications may either be=20 informed by PCN packet marks carried with application=20 traffic (ECN or DSCP) or by signaling, if the=20 boundary node supports a per flow signaling protocol." A sentence may be added to tell the reader, where this=20 would be done (which is not clear to me, help is welcome). ---------------------------------------------------------- 5.2. PCN-ingress-node functions "Packet classify" and "Policing" are only required at the=20 PCN ingress, if no other trusted edge system is doing that. If there's another edge system doing policing and=20 classification, the PCN ingress node doesn't need=20 to do this. The two bullet points may add: "Note that packet classify/policing is only required by the=20 PCN ingress node, if no other trusted upstream node operates=20 these features." Further, the section misses a statement on what the ingress=20 node is supposed to do, if a PCN coloured packet is=20 received on an interface where it is not supposed=20 to be received (given the flow is admitted): Ignore and remark or drop? ----------------------------------------------------------=20 7.3. Discussion of rationale for probing, its downsides=20 and open issues 7th bullet point "Are there other ways of dealing with the flash crowd=20 scenario? .... or perhaps for a PCN-egress-node to block=20 new flows on its empty ingress-egress-aggregates when its=20 non-empty ones are pre-congested." Should read "...or for a PCN-boundary node to limit admission of new=20 flows on its empty ingress-egress-aggregates..." It should read boundary node, as the ingress receives=20 pre-congestion feedback too and thus is aware of=20 pre-congestion in the network. Blocking is an extreme case of limiting admission and I=20 don't think blocking is required in all cases. This topic=20 clearly is for further study and we shouldn't limit=20 solution space unless we have clear indications on=20 these limits. -------------------------------------------------------- 10. Security considerations second bullet point "the PCN-ingress-nodes police packets to ensure a flow=20 sticks within its agreed limit, and to ensure that only=20 flows which have been admitted contribute PCN-traffic=20 into the PCN-domain." Respecting one of my earlier comments, this should read=20 "the PCN-ingress-node or another trusted node controlling=20 access to PCN based transport" ----------------------------------end----------------- _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 22 05:57:38 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iv9kX-0003lS-U0; Thu, 22 Nov 2007 05:57:37 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Iv9kW-0003lL-JF for pcn-confirm+ok@megatron.ietf.org; Thu, 22 Nov 2007 05:57:36 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iv9kV-0003lC-TH for pcn@ietf.org; Thu, 22 Nov 2007 05:57:35 -0500 Received: from smtp4.smtp.bt.com ([217.32.164.151]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1Iv9kR-0003tF-JB for pcn@ietf.org; Thu, 22 Nov 2007 05:57:35 -0500 Received: from E03MVZ1-UKDY.domain1.systemhost.net ([193.113.30.65]) by smtp4.smtp.bt.com with Microsoft SMTPSVC(6.0.3790.1830); Thu, 22 Nov 2007 10:57:26 +0000 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [PCN] Fwd: I-D Action:draft-chan-pcn-encoding-comparison-01.txt Date: Thu, 22 Nov 2007 10:57:25 -0000 Message-ID: <75A199C5D243C741BF3D3F1EBCEF9BA503B34393@E03MVZ1-UKDY.domain1.systemhost.net> In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] Fwd: I-D Action:draft-chan-pcn-encoding-comparison-01.txt Thread-Index: Acgs1noHJ7z7sHXCTi+2W8eLihIpwAAA6cog From: To: , , X-OriginalArrivalTime: 22 Nov 2007 10:57:26.0759 (UTC) FILETIME=[780A7F70:01C82CF6] X-Spam-Score: -1.0 (-) X-Scan-Signature: 057ebe9b96adec30a7efb2aeda4c26a4 Cc: X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Thanks Georgios - inline phil > -----Original Message----- > >I don't think nonce cheater detection is relevant [I think you're > >talking about ECN nonce. It's fair enough to talk at the appropriate > >moments about loss of ECN info if the ECN field is used to encode PCN. > >but I think it's confusing (and on occasions, in some of your encoding > >tables, wrong] to list ECN nonce as a PCN encoding state. Or are you > >suggesting a *PCN* nonce? (would seem this isn't needed, as PCN-domain > >is trusted. However, maybe thinking about future extensions?) >=20 > Georgios: The ECN nonce discussion is associated with Section 4.3 in: > http://www.ietf.org/rfc/rfc4774.txt?number=3D4774 > We have to clarify this in the text. I don't think S4.3 is relevant (other sections are) as the scope of Section 4.3 of 4774 is "The third option specified above is for the alternate ECN semantics to be defined so that traffic using the alternate semantics would coexist safely in the Internet on a path with one or more old routers that use only the default ECN semantics." Which seems not applicable as Assumption 1 of archit says "We assume that the PCN-domain is a controlled environment, i.e. all the nodes in a PCN-domain run PCN and trust each other."=20 >=20 >=20 > Georgios: We will clarify this, but please note that the option is taken > from: > http://www.watersprings.org/pub/id/draft-briscoe-tsvwg-cl-phb-03.txt > In particular see Alternative 2 and 5 from the above given draft. I had another read of phb-03 & found it a bit confusing. Basically these options can be done either with a PCN-DSCP, or try & do in existing PHB (which may already be being used for service that's using ECN e2e). there needs to be some very careful explanation of the issues. Case 1: ECT nonce not being used e2e, ie concerned about impact on 00 & CE codepoints. Need to think about what happens if pkt [1] arrives 00 & gets PCN-marked, impact on ECN; [2] arrives CE, what's impact on PCN (& on ECN if it can get re-marked) Case 2: ECT nonce is being used e2e, ie concerned about impact on all 4 codepoints. >=20 > > > > > >3.3 DSCP > >in table, I think you should change things like "PCN/AM/TM" to SM, eg: >=20 > Georgios: If the PCN WG decides to use SM as additional state then OKAY! On 2nd thoughts, I'm not sure if introducing an SM encoding state is the right ways of handling this.=20 Really we have markings associated with the PCN-lower-rate & PCN-upper-rate, so we should describe them as: PCN-lower-rate-marking PCN-upper-rate-marking Single Marking is then an encoding scheme that only uses the first of these. This would be more accurate, although it would probably make it harder for the reader to understand the draft. >=20 > >----------------------------------------------------------------------- > >| DSCP Bits || Original |Experimental 1 |Experimental 2 |Experimental 3 > >| > >|=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D++=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D > >| > >| Option 7 || Not-PC | SM | NA | NA > >| > >|-----------++----------+---------------+---------------+-------------- - > >| > > > >3.3.1 Benefits > >" concerns raised in RFC 4774 [20] are not applicable" - pedantic point: > >I think it's really that they should be easier to deal with [whether or > >not they're applicable depends on whether a PCN-node knows how to > >process a pkt with the ECN field set] >=20 > Georgios: I do not understand your argumentation: > IF ECN is not used within the PCN domain then the requirements imposed by > RFC 4774 do not apply! On 2nd reading, what you said is clear.=20 _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 22 06:13:56 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvA0K-0004sH-1C; Thu, 22 Nov 2007 06:13:56 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IvA0I-0004rz-OU for pcn-confirm+ok@megatron.ietf.org; Thu, 22 Nov 2007 06:13:54 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvA0H-0004pQ-OA for pcn@ietf.org; Thu, 22 Nov 2007 06:13:53 -0500 Received: from tcmail31.telekom.de ([217.6.95.238]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IvA0H-0001ie-2r for pcn@ietf.org; Thu, 22 Nov 2007 06:13:53 -0500 Received: from s4de8psaans.mitte.t-com.de by tcmail31.telekom.de with ESMTP; Thu, 22 Nov 2007 12:13:51 +0100 Received: from S4DE8PSAANK.mitte.t-com.de ([10.151.229.10]) by s4de8psaans.mitte.t-com.de with Microsoft SMTPSVC(6.0.3790.3959); Thu, 22 Nov 2007 12:13:51 +0100 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Subject: RE: ECN support in a PCN domain (was: Re: [PCN] comments ondraft-ietf-pcn-architecture-02) Date: Thu, 22 Nov 2007 12:13:50 +0100 Message-Id: <1B6169C658325341A3B8066E23919E1C4C14EB@S4DE8PSAANK.mitte.t-com.de> In-Reply-To: <47455AD6.2030404@ericsson.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: ECN support in a PCN domain (was: Re: [PCN] comments ondraft-ietf-pcn-architecture-02) Thread-Index: Acgs8zzt10S53v6qR8eaNn6LHzZkfQABDhEw From: "Geib, Ruediger" To: X-OriginalArrivalTime: 22 Nov 2007 11:13:51.0149 (UTC) FILETIME=[C2C87DD0:01C82CF8] X-Spam-Score: 0.0 (/) X-Scan-Signature: b4a0a5f5992e2a4954405484e7717d8c Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Magnus, I agree to your request, I've asked Phil for a statement on the = behaviour=20 of an ingress node receiving a PCN coloured packet in the architecture=20 draft in my latest mail. I thougth about ignore and remark DSCP and ECN. Drop is a tough option.=20 Forward with DSCP Best Effort and unchanged ECM may be another one (if=20 the DSCP is unknown on the ingress interface). There are more possibilities, depending on the PCN marking mechanism and = whether the ECN codepoints are used. Regards, Rudiger |>=20 |> o If a packet that is part of a PCN-flow arrives at a = PCN-ingress- |> node with its CE (Congestion experienced) codepoint set, then = we |> assume that the PCN-ingress-node drops the packet. After its |> initial Charter is complete, the WG may decide to work on a |> mechanism (such as through a signalling extension) that enables |> ECN-marking to be carried transparently across the PCN-domain." | |This one really surprise me. How can you make this assumption? In case = a |ECN enabled flow with some packets CE marked arrive to the ingress of |the PCN domain and are within the limits for which the flow has been |admitted into the PCN domain, then the PCN domain has no right to |summarily drop that packet as being less important than any of the |unmarked packets. Only in cases if congestion actually is experience |within the PCN domain would a differential treatment of CE marked |packets being warranted. | |Can someone please explain how the above assumptions still satisfy the |following sentence in the charter? | |"All PCN mechanisms, including transport and encoding |of (pre-congestion) information, are required to cleanly integrate |with existing architectures and protocols such as DiffServ and ECN." | | |Cheers | |Magnus Westerlund | |IETF Transport Area Director & TSVWG Chair |---------------------------------------------------------------------- |Multimedia Technologies, Ericsson Research EAB/TVM/M |---------------------------------------------------------------------- |Ericsson AB | Phone +46 8 4048287 |Torshamsgatan 23 | Fax +46 8 7575550 |S-164 80 Stockholm, Sweden | mailto: magnus.westerlund@ericsson.com |---------------------------------------------------------------------- | | |_______________________________________________ |PCN mailing list |PCN@ietf.org |https://www1.ietf.org/mailman/listinfo/pcn | _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 22 06:19:41 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvA5t-0001th-Lb; Thu, 22 Nov 2007 06:19:41 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IvA5s-0001tY-76 for pcn-confirm+ok@megatron.ietf.org; Thu, 22 Nov 2007 06:19:40 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvA5r-0001tQ-TY for pcn@ietf.org; Thu, 22 Nov 2007 06:19:39 -0500 Received: from smtp2.smtp.bt.com ([217.32.164.150]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IvA5n-0004kx-Et for pcn@ietf.org; Thu, 22 Nov 2007 06:19:39 -0500 Received: from E03MVZ1-UKDY.domain1.systemhost.net ([193.113.30.65]) by smtp2.smtp.bt.com with Microsoft SMTPSVC(6.0.3790.1830); Thu, 22 Nov 2007 11:19:34 +0000 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [PCN] I-D Action:draft-ietf-pcn-architecture-01.txt Date: Thu, 22 Nov 2007 11:19:30 -0000 Message-ID: <75A199C5D243C741BF3D3F1EBCEF9BA503B34396@E03MVZ1-UKDY.domain1.systemhost.net> In-Reply-To: <1B6169C658325341A3B8066E23919E1C4C14EA@S4DE8PSAANK.mitte.t-com.de> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] I-D Action:draft-ietf-pcn-architecture-01.txt Thread-Index: AcgqxqGl81rUg3oXSxKQPQt45B53dQAATG8gAIiSlTAAA42YQA== From: To: X-OriginalArrivalTime: 22 Nov 2007 11:19:34.0663 (UTC) FILETIME=[8F889170:01C82CF9] X-Spam-Score: -1.0 (-) X-Scan-Signature: 3e15cc4fdc61d7bce84032741d11c8e5 Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi rudiger - in-line, thanks phil > -----Original Message----- > From: Geib, Ruediger [mailto:Ruediger.Geib@t-systems.com] > Sent: 22 November 2007 10:52 > To: Eardley,PL,Philip,CXR9 R > Cc: pcn@ietf.org > Subject: RE: [PCN] I-D Action:draft-ietf-pcn-architecture-01.txt >=20 > Phil, >=20 > the latest two versions of the architecture are > indeed well written. Most of my comments on the > chartered topics are editorial (with the > exception of ingress policing).=20 All Editorials - thanks will try to put into next version.=20 >=20 > 5.2. PCN-ingress-node functions >=20 > "Packet classify" and "Policing" are only required at the > PCN ingress, if no other trusted edge system is doing that. >=20 > If there's another edge system doing policing and > classification, the PCN ingress node doesn't need > to do this.=20 true. In the scenarios it says: o If the operator runs both the access network and the core network, one deployment scenario is that only the core network uses PCN admission control but per microflow policing is done at the ingress to the access network and not at the PCN-ingress-node. Note: to aid readability, the rest of this draft assumes that policing is done by the PCN-ingress-nodes. However, I can add a small comment here & in section 10, as not mentioning it is hindering rather than aiding readability! >=20 > "Note that packet classify/policing is only required by the > PCN ingress node, if no other trusted upstream node operates > these features." >=20 > Further, the section misses a statement on what the ingress > node is supposed to do, if a PCN coloured packet is > received on an interface where it is not supposed > to be received (given the flow is admitted): > Ignore and remark or drop? Good point. Can we just refer to some Diffserv text? phil _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 22 07:09:26 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvArv-0000SE-MK; Thu, 22 Nov 2007 07:09:19 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IvAru-0000NS-2u for pcn-confirm+ok@megatron.ietf.org; Thu, 22 Nov 2007 07:09:18 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvArt-0000Hl-50 for pcn@ietf.org; Thu, 22 Nov 2007 07:09:17 -0500 Received: from tcmail31.telekom.de ([217.6.95.238]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IvArq-0006MV-Ef for pcn@ietf.org; Thu, 22 Nov 2007 07:09:17 -0500 Received: from s4de8psaans.mitte.t-com.de by tcmail31.telekom.de with ESMTP; Thu, 22 Nov 2007 13:09:12 +0100 Received: from S4DE8PSAANK.mitte.t-com.de ([10.151.229.10]) by s4de8psaans.mitte.t-com.de with Microsoft SMTPSVC(6.0.3790.3959); Thu, 22 Nov 2007 13:09:12 +0100 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [PCN] I-D Action:draft-ietf-pcn-architecture-01.txt Date: Thu, 22 Nov 2007 13:09:12 +0100 Message-Id: <1B6169C658325341A3B8066E23919E1C4C14EE@S4DE8PSAANK.mitte.t-com.de> In-Reply-To: <75A199C5D243C741BF3D3F1EBCEF9BA503B34396@E03MVZ1-UKDY.domain1.systemhost.net> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] I-D Action:draft-ietf-pcn-architecture-01.txt Thread-Index: AcgqxqGl81rUg3oXSxKQPQt45B53dQAATG8gAIiSlTAAA42YQAAB5JyQ From: "Geib, Ruediger" To: X-OriginalArrivalTime: 22 Nov 2007 12:09:12.0430 (UTC) FILETIME=[7E6BC0E0:01C82D00] X-Spam-Score: -1.0 (-) X-Scan-Signature: 02ec665d00de228c50c93ed6b5e4fc1a Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Phil, your suggestion is allright with me. I found this sentence in RFC2475: Ingress nodes must condition all other inbound traffic to ensure that the DS codepoints are acceptable; packets found to have unacceptable codepoints must either be discarded or must have their DS codepoints modified to acceptable values before being forwarded. As PCN may also modify ECN codepoints, the staement must be extended to cover these too. If only DSCPs are used, no traffic with "congestion=20 experienced" marking must pass the ingress node unchanged. Regards, Rudiger |All Editorials - thanks will try to put into next version.=20 | |>=20 |> 5.2. PCN-ingress-node functions |>=20 |> "Packet classify" and "Policing" are only required at the |> PCN ingress, if no other trusted edge system is doing that. |>=20 |> If there's another edge system doing policing and |> classification, the PCN ingress node doesn't need |> to do this.=20 | |true. In the scenarios it says: |o If the operator runs both the access network and the core network, | one deployment scenario is that only the core network uses PCN | admission control but per microflow policing is done at the | ingress to the access network and not at the PCN-ingress-node. | Note: to aid readability, the rest of this draft assumes that | policing is done by the PCN-ingress-nodes. | |However, I can add a small comment here & in section 10, as not |mentioning it is hindering rather than aiding readability! | |>=20 |> "Note that packet classify/policing is only required by the |> PCN ingress node, if no other trusted upstream node operates |> these features." |>=20 |> Further, the section misses a statement on what the ingress |> node is supposed to do, if a PCN coloured packet is |> received on an interface where it is not supposed |> to be received (given the flow is admitted): |> Ignore and remark or drop? | |Good point. Can we just refer to some Diffserv text? | |phil | | _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 22 07:13:20 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvAvo-0001OZ-BZ; Thu, 22 Nov 2007 07:13:20 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IvAvn-0001Ly-5J for pcn-confirm+ok@megatron.ietf.org; Thu, 22 Nov 2007 07:13:19 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvAvm-0001Lq-RV for pcn@ietf.org; Thu, 22 Nov 2007 07:13:18 -0500 Received: from smtp1.smtp.bt.com ([217.32.164.137]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IvAvd-0006So-6x for pcn@ietf.org; Thu, 22 Nov 2007 07:13:18 -0500 Received: from E03MVZ1-UKDY.domain1.systemhost.net ([193.113.30.65]) by smtp1.smtp.bt.com with Microsoft SMTPSVC(6.0.3790.1830); Thu, 22 Nov 2007 12:13:06 +0000 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Subject: RE: [PCN] PCN & tunnelling Date: Thu, 22 Nov 2007 12:13:05 -0000 Message-ID: <75A199C5D243C741BF3D3F1EBCEF9BA503B3439A@E03MVZ1-UKDY.domain1.systemhost.net> In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] PCN & tunnelling Thread-Index: AcgWJ+54M2I8eYc4QJu/kWUfuySz/QAGbMXgAAt1T+AA8fzNIASySaVQ From: To: , X-OriginalArrivalTime: 22 Nov 2007 12:13:06.0590 (UTC) FILETIME=[09FDBBE0:01C82D01] X-Spam-Score: -1.0 (-) X-Scan-Signature: 946b615681bce2b72aace5af8d0d8a1d Cc: X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1754162801==" Errors-To: pcn-bounces@ietf.org This is a multi-part message in MIME format. --===============1754162801== Content-class: urn:content-classes:message Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C82D01.093E1BFB" This is a multi-part message in MIME format. ------_=_NextPart_001_01C82D01.093E1BFB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable David,=20 I tried to improve the draft to reflect your comments, is it ok? =20 [from draft-pcn-architecture-02 S5.7] =20 Potential issues arise for a "partially PCN-capable tunnel", ie where only one tunnel endpoint is in the PCN domain: =20 1. The tunnel starts outside a PCN-domain and finishes inside it. If the packet arrives at the tunnel ingress with the same encoding as used within the PCN-domain to indicate PCN-marking, then this could lead the PCN-egress-node to falsely measure pre- congestion. =20 2. The tunnel starts inside a PCN-domain and finishes outside it. If the packet arrives at the tunnel ingress already PCN-marked, then it will still have the same encoding when it's decapsulated which could potentially confuse nodes beyond the tunnel egress. =20 In line with the solution for partially capable DiffServ tunnels in [2983], the following rules are applied: =20 o For case (1), the tunnel egress node clears any PCN-marking on the inner header. This rule is applied before the 'copy on decapsulation' rule above. =20 o For case (2), the tunnel ingress node clears any PCN-marking on the inner header. This rule is applied after the 'copy on encapsulation' rule above. =20 Note that the above implies that one has to know, or figure out, the characteristics of the other end of the tunnel as part of setting it up. =20 Ok? =20 Also, in S6 there's listed an Open issue: o Scenarios with only one tunnel endpoint in the PCN domain may make it harder for the PCN-egress-node to gather from the signalling messages (eg RSVP, NSIS) the identity of the PCN-ingress-node. =20 Any comments? =20 Thanks phil =20 =20 -----Original Message----- From: Black_David@emc.com [mailto:Black_David@emc.com]=20 Sent: 29 October 2007 19:58 To: Eardley,PL,Philip,CXR9 R; pcn@ietf.org Cc: Black_David@emc.com Subject: RE: [PCN] PCN & tunnelling =20 Phil, =20 As the author of RFC 2983, I was going to point you to this text. =20 Regarding the characteristics of the other end of the tunnel, there isn't any magic; one has to know this (or figure it out) as part of setting up the tunnel. RFC 2983 was written from a core networks point of view so the expected sort of tunnel is one being used for traffic management, so its reasonable to expect that the nature of the other endpoint is known when the tunnel is set up. =20 If in doubt, clearing out everything (e.g., DSCP of 0) is generally a good course of action for a tunnel headed elsewhere, and if the tunnel winds up back in the same domain, unbeknownst to the admins, shame on those who weren't paying attention. =20 Thanks, --David ---------------------------------------------------- David L. Black, Distinguished Engineer EMC Corporation, 176 South St., Hopkinton, MA 01748 +1 (508) 293-7953 FAX: +1 (508) 293-7786 black_david@emc.com Mobile: +1 (978) 394-7754 ---------------------------------------------------- =20 =09 ________________________________ From: philip.eardley@bt.com [mailto:philip.eardley@bt.com]=20 Sent: Wednesday, October 24, 2007 3:06 PM To: acharny@cisco.com; pcn@ietf.org Subject: RE: [PCN] PCN & tunnelling Anna, =20 I don't know. =20 For inspiration, I looked at rfc2983, diffserv & tunnels. S3.2 of this talks about partially capable DS configs - only tunnel ingress is DS-capable but not the egress. It says that=20 If tunnel decapsulation processing discards the outer header's DSCP value without changing the inner header's DSCP value, the DS-capable tunnel ingress node is obligated to set the inner header's DSCP to a value compatible with the network at the tunnel egress. The value 0 [is a good suggestion] =20 this approach is along the same lines to the one in the original email below. Unfortunately 2983 doesn't say how the tunnel ingress node knows about the characteristics of the network at the tunnel egress. Did the DS people have anything in mind that might apply in the PCN case? =20 phil =20 -----Original Message----- From: Anna Charny (acharny) [mailto:acharny@cisco.com]=20 Sent: 24 October 2007 14:30 To: Eardley,PL,Philip,CXR9 R; pcn@ietf.org Subject: RE: [PCN] PCN & tunnelling =20 Hi Phil, =20 Question: what would be the mechanism for knowing whether the egress of the tunnel is in the PCN domain or not? Are we assuming that a node somehow advertises its PCN capability? I do not think we ever explicitly assumed that? =20 Anna=20 =20 =09 ________________________________ From: philip.eardley@bt.com [mailto:philip.eardley@bt.com]=20 Sent: Wednesday, October 24, 2007 6:24 AM To: pcn@ietf.org Subject: [PCN] PCN & tunnelling Hi, =20 I was chatting with Bob about section 5.8 tunnelling. We've realised there's the following nasty case which we hadn't thought about. The scenario is when a tunnel starts inside a PCN-domain and finishes outside it.=20 =20 First we follow what happens when the current text is followed. Imagine that the pkt is PCN-marked by some PCN-node before the tunnel start node. On encapsulation the PCN-mark is copied onto the outer header. Hence the PCN-egress-node 'sees' the PCN-mark as normal - this is ok. The PCN-egress-node clears the marking in the (outer) header and forwards the pkt into the next domain. The pkt is decapsulated at the tunnel end point. But the decapsulated pkt is already PCN-marked. Potential problems: [1] if it's decapsulated in a non-PCN-domain, then the pkt may confuse nodes in this domain [depending on what encoding is used for a PCN-mark] ; [2] if it's decapsulated in a PCN-domain, then the pkt is PCN-marked [which might lead this PCN-domain to terminate or block a flow unnecessarily]. =20 The problem arises because the PCN-egress-node clears PCN-marking on the outer header but not on the inner header. (This is a new problem compared with ECN & tunnelling.)=20 =20 Possible solution: if the pkt is PCN-marked, then the tunnel start node checks whether the tunnel egress is inside or outside the PCN-domain - if it's outside, then it clears the PCN-marking on the inner header (effectively it does this on behalf of the PCN-egress-node). (Also, the PCN-mark is copied onto the outer header, as the current text says.) =20 Thoughts? =20 Thanks, Phil/ =20 ------_=_NextPart_001_01C82D01.093E1BFB Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

David,

I tried to improve the draft to = reflect your comments, is it ok?

 

[from draft-pcn-architecture-02 = S5.7]

 

Potential issues arise for a "partially =
PCN-capable tunnel", ie where
   only one tunnel endpoint is in =
the PCN domain:
 
   1.  The tunnel starts =
outside a PCN-domain and finishes inside =
it.
       If the =
packet arrives at the tunnel ingress with the =
same
       encoding =
as used within the PCN-domain to indicate =
PCN-marking,
       then =
this could lead the PCN-egress-node to falsely measure =
pre-
       =
congestion.
 
   2.  The tunnel starts =
inside a PCN-domain and finishes outside =
it.
       If the =
packet arrives at the tunnel ingress already =
PCN-marked,
       then it =
will still have the same encoding when it's =
decapsulated
       which =
could potentially confuse nodes beyond the tunnel =
egress.
 
   In line with the solution for =
partially capable DiffServ tunnels in
   [2983], the following rules are =
applied:
 
   o  For case (1), the tunnel =
egress node clears any PCN-marking on the
      inner =
header.  This rule is applied before the 'copy =
on
      decapsulation' =
rule above.
 
   o  For case (2), the tunnel =
ingress node clears any PCN-marking on
      the inner =
header.  This rule is applied after the 'copy =
on
      encapsulation' =
rule above.
 
   Note that the above implies that =
one has to know, or figure out, the
   characteristics of the other end =
of the tunnel as part of setting it
   up.

 

Ok?

 

Also, in S6 there’s listed an = Open issue:

o  Scenarios with only one tunnel =
endpoint in the PCN domain may make
      it harder for =
the PCN-egress-node to gather from the =
signalling
      messages (eg =
RSVP, NSIS) the identity of the PCN-ingress-node.

 

Any comments?

 

Thanks

phil

 

 

-----Original Message-----
From: Black_David@emc.com [mailto:Black_David@emc.com]
Sent: 29 October 2007 = 19:58
To: = Eardley,PL,Philip,CXR9 R; pcn@ietf.org
Cc: = Black_David@emc.com
Subject: RE: [PCN] PCN = & tunnelling

 

Phil,

 

As the author of RFC 2983, I was going to = point you to this text.

 

Regarding the characteristics of the other = end of the tunnel, there

isn't any magic; one has to know this (or = figure it out) as part of

setting up the tunnel.  RFC 2983 was = written from a core networks

point of view so the expected sort of tunnel = is one being used for

traffic management, so its reasonable to = expect that the nature of

the other endpoint is known when the tunnel = is set up.

 

If in doubt, clearing out everything (e.g., = DSCP of 0) is generally

a good course of action for a tunnel headed elsewhere, and if the

tunnel winds up back in the same domain, unbeknownst to the admins,

shame on those who weren't paying = attention.

 

Thanks,

--David

----------------------------------------------------
David L. Black, Distinguished Engineer
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953           = ;  FAX: +1 (508) 293-7786
black_david@emc.com        Mobile: +1 = (978) 394-7754
----------------------------------------------------

 

From: philip.eardley@bt.com [mailto:philip.eardley@bt.com]
Sent: Wednesday, October = 24, 2007 3:06 PM
To: acharny@cisco.com; pcn@ietf.org
Subject: RE: [PCN] PCN = & tunnelling

Anna,

 

I don’t = know.

 

For inspiration, I looked at = rfc2983, diffserv & tunnels. S3.2 of this talks about partially capable DS = configs – only tunnel ingress is DS-capable but not the egress. It says = that 

If tunnel decapsulation processing =
discards
   the outer header's DSCP value =
without changing the inner header's
   DSCP value, the DS-capable =
tunnel ingress node is obligated to set
   the inner header's DSCP to a =
value compatible with the network at the
   tunnel egress.  The value 0 =
[is a good suggestion]

 

this approach is along the same = lines to the one in the original email below. Unfortunately 2983 doesn’t = say how the tunnel ingress node knows about the characteristics of the network = at the tunnel egress. Did the DS people have anything in mind that might apply = in the PCN case?

 

phil

 

-----Original Message-----
From: Anna Charny = (acharny) [mailto:acharny@cisco.com]
Sent: 24 October 2007 = 14:30
To: = Eardley,PL,Philip,CXR9 R; pcn@ietf.org
Subject: RE: [PCN] PCN = & tunnelling

 

Hi Phil,

 

Question: what would be the = mechanism for knowing whether the egress of the tunnel is in the PCN domain or = not?  Are we assuming that a node somehow advertises its PCN capability?  I = do not think we ever explicitly assumed that?

 

Anna

 

From: philip.eardley@bt.com [mailto:philip.eardley@bt.com]
Sent: Wednesday, October = 24, 2007 6:24 AM
To: pcn@ietf.org
Subject: [PCN] PCN & tunnelling

Hi,

 

I was chatting with Bob about section 5.8 tunnelling. = We’ve realised there’s the following nasty case which we hadn’t = thought about. The scenario is when a tunnel starts inside a PCN-domain and = finishes outside it.

 

First we follow what happens when the current text is followed. = Imagine that the pkt is PCN-marked by some PCN-node before the tunnel start = node. On encapsulation the PCN-mark is copied onto the outer header. Hence the PCN-egress-node ‘sees’ the PCN-mark as normal – this = is ok. The PCN-egress-node clears the marking in the (outer) header and = forwards the pkt into the next domain. The pkt is decapsulated at the tunnel end = point. But the decapsulated pkt is already PCN-marked. Potential problems: [1] if it’s decapsulated in a non-PCN-domain, then the pkt may confuse = nodes in this domain [depending on what encoding is used for a PCN-mark] ; [2] if it’s decapsulated in a PCN-domain, then the pkt is PCN-marked = [which might lead this PCN-domain to terminate or block a flow = unnecessarily].

 

The problem arises because the PCN-egress-node clears = PCN-marking on the outer header but not on the inner header. (This is a new problem = compared with ECN & tunnelling.)

 

Possible solution: if the pkt is PCN-marked, then the tunnel = start node checks whether the tunnel egress is inside or outside the PCN-domain - = if it’s outside, then it clears the PCN-marking on the inner header (effectively it does this on behalf of the PCN-egress-node). (Also, the PCN-mark is copied onto the outer header, as the current text = says.)

 

Thoughts?

 

Thanks,

Phil/  

=00 ------_=_NextPart_001_01C82D01.093E1BFB-- --===============1754162801== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn --===============1754162801==-- From pcn-bounces@ietf.org Thu Nov 22 07:30:12 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvBC8-0001MK-Uq; Thu, 22 Nov 2007 07:30:12 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IvBC7-0001MD-SW for pcn-confirm+ok@megatron.ietf.org; Thu, 22 Nov 2007 07:30:11 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvBC7-0001M5-I1 for pcn@ietf.org; Thu, 22 Nov 2007 07:30:11 -0500 Received: from rotterdam.ewi.utwente.nl ([130.89.10.5]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IvBC4-0006vZ-Lo for pcn@ietf.org; Thu, 22 Nov 2007 07:30:11 -0500 Received: from utip105 (utip105.ewi.utwente.nl [130.89.13.76]) by rotterdam.ewi.utwente.nl (8.13.6/8.13.6) with ESMTP id lAMCTZwX004090; Thu, 22 Nov 2007 13:30:05 +0100 (MET) From: "Georgios Karagiannis" To: , References: <75A199C5D243C741BF3D3F1EBCEF9BA503B34392@E03MVZ1-UKDY.domain1.systemhost.net> Subject: RE: [PCN] comments on draft-ietf-pcn-architecture-02 Date: Thu, 22 Nov 2007 13:29:24 +0100 Message-ID: <000001c82d03$5a4a4df0$4c0d5982@dynamic.ewi.utwente.nl> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 11 Thread-Index: Acgs0ibapJhVEKTeSKaERK5iVAsa7AAHKj+AAAT0iCA= In-Reply-To: <75A199C5D243C741BF3D3F1EBCEF9BA503B34392@E03MVZ1-UKDY.domain1.systemhost.net> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3198 X-Spam-Score: 0.388 () AWL,J_CHICKENPOX_34 X-Scanned-By: MIMEDefang 2.52 on 130.89.10.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0rc3 (rotterdam.ewi.utwente.nl [130.89.10.5]); Thu, 22 Nov 2007 13:30:05 +0100 (MET) X-Spam-Score: 0.0 (/) X-Scan-Signature: 68ba2b07ef271dba6ee42a93832cfa4c Cc: X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Phil Thank you! Please see comments regarding comment_3: > > Comment_3: > > In Section 3, page 29, you provide the below conclusion on the > viewpoint > > that admission control is always done by probing: > > > > "The first point breaks Assumption 3 (aggregation) and hence means > that > > this viewpoint is out of scope of initial Charter of the PCN WG." > > > > I do not agree with this conclusion, since I do not agree with the > > argumentation that the first point breaks Assumption 3 > > the "first point" is this: > o Simply admitting the new flow has a significant risk of leading to > overload, because the PCN-domain reaches out towards the end > terminals where link capacity is low. > The link capacity is low means that adding a single flow to > the link can move it directly from no congestion to > overloaded [real queue grows significantly or overflows]. > This breaks the aggregation assumption. Georgios: yes, But if the link capacity is low and the admission threshold/rate in an PCN-interior node is high (by configuration), how then a single flow can move the state in the PCN_interior_node from no congestion to congestion? This means that the admission threshold/rate should be doen properly such that such a situation will not occur! Best regards, Georgios > -----Original Message----- > From: philip.eardley@bt.com [mailto:philip.eardley@bt.com] > Sent: donderdag 22 november 2007 11:30 > To: karagian@cs.utwente.nl; pcn@ietf.org > Subject: RE: [PCN] comments on draft-ietf-pcn-architecture-02 > > Georgios, > Thanks for the comments! > > > Comment_1: > Lars has answered. > > > Comment_2: > > Section 3.5: > > You mention: > > "The following two assumptions apply if the PCN WG decides to > encode > > PCN-marking in the ECN-field. > > > > o It is assumed that PCN-nodes do not perform ECN, [RFC3168], on > > PCN-packets. > > > > o If a packet that is part of a PCN-flow arrives at a > PCN-ingress- > > node with its CE (Congestion experienced) codepoint > set, then we > > assume that the PCN-ingress-node drops the packet. After its > > initial Charter is complete, the WG may decide to work on a > > mechanism (such as through a signalling extension) > that enables > > ECN-marking to be carried transparently across the > PCN-domain." > > > > If I understand the two above assumptions well, they say > that traffic > > that is arriving into the PCN domain and is end-to-end ECN > aware can > > only keep its end-to-end ECN semantics if it is tunnelled. > > In my opinion this is one possible implementation. There > might also be > > other implementation. > > Yes, I agree tunnelling would be one option, and that there > would be other implementations. Note the issue of how to > carry ECN-marking transparently across the PCN-domain is one > that the WG may look at after the initial Charter is done. > Note also, the if: "if the PCN WG decides to encode > PCN-marking in the ECN-field" [in the terms of your encoding > draft, this would include ECN+DSCP option] > > I think that this assumption should refer to the requirements > > described in RFC 4774: > > http://www.ietf.org/rfc/rfc4774.txt?number=4774 > > > > Please use the below paragraph instead the one that is given above: > > "The following two assumptions apply if the PCN WG > decides to encode > > PCN-marking in the ECN-field. > > > > o It is assumed that the requirements imposed by RFC 4774 are > > fulfilled." > > Yes, I think we could add a ref to 4774 somewhere in the > draft, and this seems a good place to do so. Will add as a > 3rd bullet something like "the Milestone on 'Survey of > Encoding Choices' will consider the implications of [RFC4774]" > > > > > ------------------- > > > > Comment_3: > > In Section 3, page 29, you provide the below conclusion on the > viewpoint > > that admission control is always done by probing: > > > > "The first point breaks Assumption 3 (aggregation) and hence means > that > > this viewpoint is out of scope of initial Charter of the PCN WG." > > > > I do not agree with this conclusion, since I do not agree with the > > argumentation that the first point breaks Assumption 3 > > the "first point" is this: > o Simply admitting the new flow has a significant risk of leading to > overload, because the PCN-domain reaches out towards the end > terminals where link capacity is low. > The link capacity is low means that adding a single flow to > the link can move it directly from no congestion to > overloaded [real queue grows significantly or overflows]. > This breaks the aggregation assumption. > Best wishes > Phil/ > _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 22 07:35:09 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvBGv-0000rc-89; Thu, 22 Nov 2007 07:35:09 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IvBGt-0000rS-K9 for pcn-confirm+ok@megatron.ietf.org; Thu, 22 Nov 2007 07:35:07 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvBGs-0000rK-Tp for pcn@ietf.org; Thu, 22 Nov 2007 07:35:06 -0500 Received: from rotterdam.ewi.utwente.nl ([130.89.10.5]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IvBGp-00075Z-1b for pcn@ietf.org; Thu, 22 Nov 2007 07:35:06 -0500 Received: from utip105 (utip105.ewi.utwente.nl [130.89.13.76]) by rotterdam.ewi.utwente.nl (8.13.6/8.13.6) with ESMTP id lAMCYvpT005042; Thu, 22 Nov 2007 13:35:02 +0100 (MET) From: "Georgios Karagiannis" To: "'Lars Eggert'" References: Subject: RE: [PCN] comments on draft-ietf-pcn-architecture-02 Date: Thu, 22 Nov 2007 13:34:52 +0100 Message-ID: <000101c82d04$17462050$4c0d5982@dynamic.ewi.utwente.nl> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 11 Thread-Index: Acgs7iVRKbAjXiVKTke+Ih++AptTIwAFdgOw In-Reply-To: X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3198 X-Spam-Score: 0.088 () AWL X-Scanned-By: MIMEDefang 2.52 on 130.89.10.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0rc3 (rotterdam.ewi.utwente.nl [130.89.10.5]); Thu, 22 Nov 2007 13:35:02 +0100 (MET) X-Spam-Score: 0.0 (/) X-Scan-Signature: 39bd8f8cbb76cae18b7e23f7cf6b2b9f Cc: 'pcn' X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Lars Thank you ver much! Best regards, Georgios > -----Original Message----- > From: Lars Eggert [mailto:lars.eggert@nokia.com] > Sent: donderdag 22 november 2007 10:27 > To: ext Georgios Karagiannis > Cc: pcn > Subject: Re: [PCN] comments on draft-ietf-pcn-architecture-02 > > On 2007-11-22, at 8:37, ext Georgios Karagiannis wrote: > > I could not find information in the draft that mentions the below > > issue given in the PCN charter: > > > > "The WG may also consider to investigate additional response > > mechanisms that act on (pre-)congestion information. One > example could > > be flow-rate adaptation (rather than flow admission/ > > termination) during times of congestion." > > > > Please include a paragraph that discusses this! > > The sentence you quote is from a paragraph that starts with > "After completion of the initial phase..." Many of the > examples given in that paragraph will significantly revamp > the PCN architecture, and duscissug them in the initial phase > is likely to be unproductive. > > Lars > _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 22 07:38:45 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvBKP-0006BF-1x; Thu, 22 Nov 2007 07:38:45 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IvBKN-00060Y-DM for pcn-confirm+ok@megatron.ietf.org; Thu, 22 Nov 2007 07:38:43 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvBKM-0005xy-W0 for pcn@ietf.org; Thu, 22 Nov 2007 07:38:43 -0500 Received: from smtp1.smtp.bt.com ([217.32.164.137]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IvBKM-0004z6-Gt for pcn@ietf.org; Thu, 22 Nov 2007 07:38:42 -0500 Received: from E03MVZ1-UKDY.domain1.systemhost.net ([193.113.30.65]) by smtp1.smtp.bt.com with Microsoft SMTPSVC(6.0.3790.1830); Thu, 22 Nov 2007 12:38:41 +0000 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [PCN] comments on draft-ietf-pcn-architecture-02 Date: Thu, 22 Nov 2007 12:38:40 -0000 Message-ID: <75A199C5D243C741BF3D3F1EBCEF9BA503B3439B@E03MVZ1-UKDY.domain1.systemhost.net> In-Reply-To: <000001c82d03$5a4a4df0$4c0d5982@dynamic.ewi.utwente.nl> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] comments on draft-ietf-pcn-architecture-02 Thread-Index: Acgs0ibapJhVEKTeSKaERK5iVAsa7AAHKj+AAAT0iCAAAEJfEA== From: To: , X-OriginalArrivalTime: 22 Nov 2007 12:38:41.0294 (UTC) FILETIME=[9CBEFAE0:01C82D04] X-Spam-Score: 0.0 (/) X-Scan-Signature: f4c2cf0bccc868e4cc88dace71fb3f44 Cc: X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org > -----Original Message----- > From: Georgios Karagiannis [mailto:karagian@cs.utwente.nl] > Sent: 22 November 2007 12:29 > To: Eardley,PL,Philip,CXR9 R; pcn@ietf.org > Subject: RE: [PCN] comments on draft-ietf-pcn-architecture-02 >=20 > Hi Phil >=20 > Thank you! > Please see comments regarding comment_3: >=20 > > > Comment_3: > > > In Section 3, page 29, you provide the below conclusion on the > > viewpoint > > > that admission control is always done by probing: > > > > > > "The first point breaks Assumption 3 (aggregation) and hence means > > that > > > this viewpoint is out of scope of initial Charter of the PCN WG." > > > > > > I do not agree with this conclusion, since I do not agree with the > > > argumentation that the first point breaks Assumption 3 > > > > the "first point" is this: > > o Simply admitting the new flow has a significant risk of leading to > > overload, because the PCN-domain reaches out towards the end > > terminals where link capacity is low. > > The link capacity is low means that adding a single flow to > > the link can move it directly from no congestion to > > overloaded [real queue grows significantly or overflows]. > > This breaks the aggregation assumption. >=20 > Georgios: yes, But if the link capacity is low and the admission > threshold/rate in an PCN-interior node > is high (by configuration), how then a single flow can move the state in > the > PCN_interior_node from > no congestion to congestion? This means that the admission threshold/rate > should be doen properly such > that such a situation will not occur! >=20 after a bit of puzzling... I was talking about the situation where the low capacity link is inside the PCN-domain. I think your situation is different, where there's an access network not running PCN, and the low capacity link is in the access nw. Is this right? If yes, I'd say that how you do admission control in the access nw (which isn't running PCN) is not a concern of PCN. if you want to always do 'adm ctrl by probing' in the access nw, that's fine. presumably when the PCN-boundary-nodes sees one of these probes then it's triggered to do some PCN adm ctrl activity. _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 22 07:48:20 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvBTf-000452-Sr; Thu, 22 Nov 2007 07:48:19 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IvBTe-00044s-KF for pcn-confirm+ok@megatron.ietf.org; Thu, 22 Nov 2007 07:48:18 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvBTe-00044k-78 for pcn@ietf.org; Thu, 22 Nov 2007 07:48:18 -0500 Received: from rotterdam.ewi.utwente.nl ([130.89.10.5]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IvBTb-0007Sg-IA for pcn@ietf.org; Thu, 22 Nov 2007 07:48:18 -0500 Received: from utip105 (utip105.ewi.utwente.nl [130.89.13.76]) by rotterdam.ewi.utwente.nl (8.13.6/8.13.6) with ESMTP id lAMClpZG007856; Thu, 22 Nov 2007 13:48:14 +0100 (MET) From: "Georgios Karagiannis" To: "'Geib, Ruediger'" References: <1B6169C658325341A3B8066E23919E1C4C14D5@S4DE8PSAANK.mitte.t-com.de> Subject: RE: [PCN] Encoding PCN-markings into DSCPs - are there anyinteractions with ECMP? Date: Thu, 22 Nov 2007 13:47:46 +0100 Message-ID: <000201c82d05$e7d15040$4c0d5982@dynamic.ewi.utwente.nl> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 11 Thread-Index: Acgs4WGvKbHSmuQVRPu3zgIC+pF8DQAAEibwAAhw/CA= In-Reply-To: <1B6169C658325341A3B8066E23919E1C4C14D5@S4DE8PSAANK.mitte.t-com.de> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3198 X-Spam-Score: 0.088 () AWL X-Scanned-By: MIMEDefang 2.52 on 130.89.10.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0rc3 (rotterdam.ewi.utwente.nl [130.89.10.5]); Thu, 22 Nov 2007 13:48:15 +0100 (MET) X-Spam-Score: 0.0 (/) X-Scan-Signature: c83ccb5cc10e751496398f1233ca9c3a Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Ruediger An operator can however choose which type of ECMP algorithm should be used in its network, or? Which types of ECMP algorithm is Deutsche Telecom using? We have to clarify which types of ECMP algorithms will be affected when the DSCP is used for PCN purposes. Not all the ECMP algorithms are affected. I am not sure about it but if the ECMP algorithms that are using DSCP to select packets for load sharing might not be affected (when DSCP is used for PCN purposes) if the selection process in addition to DSCP also uses the 5 tuple (for flow identification). Best regards, Georgios > -----Original Message----- > From: Geib, Ruediger [mailto:Ruediger.Geib@t-systems.com] > Sent: donderdag 22 november 2007 9:40 > To: karagian@cs.utwente.nl > Cc: pcn@ietf.org > Subject: RE: [PCN] Encoding PCN-markings into DSCPs - are > there anyinteractions with ECMP? > > Georgios, > > if you want to convince Deutsche Telekom's backbone > engineering team to turn off ECMP, you must offer them very > convincing arguments. I'd suggest the PCN WG to expect or > allow ECMP to be active within a PCN network, at least if is > a carrier network. > > Regards, > > Rudiger > > |Georgios: I am not sure about the above! An operator could at least > |disallow the ECMP option. > | > |Best regards, > |Georgios > | > | > | > | > |On 11/22/2007, "philip.eardley@bt.com" wrote: > | > |>Hi, > |>Have extracted this issue as a separate thread. > |>The context of the discussion below is encoding PCN-markings into > |>different DSCPs (ie not touching the ECN field). Question is, > |if you do > |>this are there any nasty interactions with ECMP. > |>Any comments? > |>phil > |> > |>> > > |>> >Major disadvantage: interactions with ECMP. ECMP algos can use a > |>> >packet's DSCP when deciding what interface to fwd it on. In a > |>PCN-domain > |>> >running ECMP, for a flow which has some packets > PCN-marked and some > |>> >unmarked. its packets will travel over more than one > path. This may > |>well > |>> >lead to mis-ordering. > |>> >To me, this means that the S3.3 DSCP option is a > non-starter in any > |>> >PCN-domain that uses an ECMP algorithm that uses the DSCP. > |>> > |>> Georgios: well I do not see this as a disadvantage. > |However, what you > |>> mention about having an ECMP algorithm that uses DSCP is relevant. > |>> Thus what we can do is to include a sentence in that > emphasizes that > |>the > |>> ECMP situation is solved only if the ECMP algorithm does > |not use DSCP > |>> during the implementation of the equal cost multi path algorithm. > |>> > |>> > NB ECMP algos > |>> >are proprietary to router vendors, ie operator has poor > |control over > |>> >them. > |>> > |>> Georgios: I am not sure about the above! An operator > could at least > |>> disallow the ECN option. > |>> > |>> > |>> > |> > |> > |>_______________________________________________ > |>PCN mailing list > |>PCN@ietf.org > |>https://www1.ietf.org/mailman/listinfo/pcn > | > | > |_______________________________________________ > |PCN mailing list > |PCN@ietf.org > |https://www1.ietf.org/mailman/listinfo/pcn > | > _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 22 08:09:39 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvBoG-0001jX-Vr; Thu, 22 Nov 2007 08:09:36 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IvBoF-0001jR-Ik for pcn-confirm+ok@megatron.ietf.org; Thu, 22 Nov 2007 08:09:35 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvBoF-0001jJ-6s for pcn@ietf.org; Thu, 22 Nov 2007 08:09:35 -0500 Received: from tcmail31.telekom.de ([217.6.95.238]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IvBoB-000880-Qn for pcn@ietf.org; Thu, 22 Nov 2007 08:09:35 -0500 Received: from s4de8psaans.mitte.t-com.de by tcmail31.telekom.de with ESMTP; Thu, 22 Nov 2007 14:09:00 +0100 Received: from S4DE8PSAANK.mitte.t-com.de ([10.151.229.10]) by s4de8psaans.mitte.t-com.de with Microsoft SMTPSVC(6.0.3790.3959); Thu, 22 Nov 2007 14:08:58 +0100 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [PCN] Encoding PCN-markings into DSCPs - are thereanyinteractions with ECMP? Date: Thu, 22 Nov 2007 14:08:04 +0100 Message-Id: <1B6169C658325341A3B8066E23919E1C4C14EF@S4DE8PSAANK.mitte.t-com.de> In-Reply-To: <000201c82d05$e7d15040$4c0d5982@dynamic.ewi.utwente.nl> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] Encoding PCN-markings into DSCPs - are thereanyinteractions with ECMP? Thread-Index: Acgs4WGvKbHSmuQVRPu3zgIC+pF8DQAAEibwAAhw/CAAATWFoA== From: "Geib, Ruediger" To: X-OriginalArrivalTime: 22 Nov 2007 13:08:58.0946 (UTC) FILETIME=[D826AA20:01C82D08] X-Spam-Score: -1.0 (-) X-Scan-Signature: 0ff9c467ad7f19c2a6d058acd7faaec8 Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Georgios, true, the DSCP shouldn't be used to influence ECMP decissions. IP adresses and may be ports. I'd have to ask my staff to get=20 details. I've no problem to recommend not to base ECMP=20 decisions on DSCPs. Regards, Rudiger |-----Original Message----- |From: Georgios Karagiannis [mailto:karagian@cs.utwente.nl] |Sent: Thursday, November 22, 2007 1:48 PM |To: Geib, Rudiger |Cc: pcn@ietf.org |Subject: RE: [PCN] Encoding PCN-markings into DSCPs - are |thereanyinteractions with ECMP? | | |Hi Ruediger | |An operator can however choose which type of ECMP algorithm=20 |should be used in its network, or? |Which types of ECMP algorithm is Deutsche Telecom using? | |We have to clarify which types of ECMP algorithms will=20 |be affected when the DSCP is used for PCN purposes. | |Not all the ECMP algorithms are affected. I am not sure about it but=20 |if the ECMP algorithms=20 |that are using DSCP to select packets for load sharing might=20 |not be affected |(when DSCP is used for=20 |PCN purposes) if the selection process in addition to DSCP also uses=20 |the 5 tuple (for flow identification). | | |Best regards, |Georgios | | |> -----Original Message----- |> From: Geib, Ruediger [mailto:Ruediger.Geib@t-systems.com]=20 |> Sent: donderdag 22 november 2007 9:40 |> To: karagian@cs.utwente.nl |> Cc: pcn@ietf.org |> Subject: RE: [PCN] Encoding PCN-markings into DSCPs - are=20 |> there anyinteractions with ECMP? |>=20 |> Georgios, |>=20 |> if you want to convince Deutsche Telekom's backbone=20 |> engineering team to turn off ECMP, you must offer them very=20 |> convincing arguments. I'd suggest the PCN WG to expect or=20 |> allow ECMP to be active within a PCN network, at least if is=20 |> a carrier network. |>=20 |> Regards, |>=20 |> Rudiger |>=20 |> |Georgios: I am not sure about the above! An operator could at least=20 |> |disallow the ECMP option. |> | |> |Best regards, |> |Georgios |> | |> | |> | |> | |> |On 11/22/2007, "philip.eardley@bt.com"=20 | wrote: |> | |> |>Hi, |> |>Have extracted this issue as a separate thread. |> |>The context of the discussion below is encoding PCN-markings into=20 |> |>different DSCPs (ie not touching the ECN field). Question is, |> |if you do |> |>this are there any nasty interactions with ECMP. |> |>Any comments? |> |>phil |> |> |> |>> > |> |>> >Major disadvantage: interactions with ECMP. ECMP algos=20 |can use a=20 |> |>> >packet's DSCP when deciding what interface to fwd it on. In a |> |>PCN-domain |> |>> >running ECMP, for a flow which has some packets=20 |> PCN-marked and some=20 |> |>> >unmarked. its packets will travel over more than one=20 |> path. This may |> |>well |> |>> >lead to mis-ordering. |> |>> >To me, this means that the S3.3 DSCP option is a=20 |> non-starter in any=20 |> |>> >PCN-domain that uses an ECMP algorithm that uses the DSCP. |> |>>=20 |> |>> Georgios: well I do not see this as a disadvantage.=20 |> |However, what you |> |>> mention about having an ECMP algorithm that uses DSCP is=20 |relevant. |> |>> Thus what we can do is to include a sentence in that=20 |> emphasizes that |> |>the |> |>> ECMP situation is solved only if the ECMP algorithm does |> |not use DSCP |> |>> during the implementation of the equal cost multi path algorithm. |> |>>=20 |> |>> > NB ECMP algos |> |>> >are proprietary to router vendors, ie operator has poor |> |control over |> |>> >them. |> |>>=20 |> |>> Georgios: I am not sure about the above! An operator=20 |> could at least=20 |> |>> disallow the ECN option. |> |>>=20 |> |>>=20 |> |>> |> |> |> |> |> |>_______________________________________________ |> |>PCN mailing list |> |>PCN@ietf.org |> |>https://www1.ietf.org/mailman/listinfo/pcn |> | |> | |> |_______________________________________________ |> |PCN mailing list |> |PCN@ietf.org |> |https://www1.ietf.org/mailman/listinfo/pcn |> | |>=20 | | | | |_______________________________________________ |PCN mailing list |PCN@ietf.org |https://www1.ietf.org/mailman/listinfo/pcn | _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 22 08:32:31 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvCAR-0002q5-9h; Thu, 22 Nov 2007 08:32:31 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IvCAQ-0002q0-KY for pcn-confirm+ok@megatron.ietf.org; Thu, 22 Nov 2007 08:32:30 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvCAQ-0002ps-0u for pcn@ietf.org; Thu, 22 Nov 2007 08:32:30 -0500 Received: from smtp2.smtp.bt.com ([217.32.164.150]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IvCAL-0000Np-VM for pcn@ietf.org; Thu, 22 Nov 2007 08:32:30 -0500 Received: from E03MVZ1-UKDY.domain1.systemhost.net ([193.113.30.65]) by smtp2.smtp.bt.com with Microsoft SMTPSVC(6.0.3790.1830); Thu, 22 Nov 2007 13:32:20 +0000 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Thu, 22 Nov 2007 13:32:04 -0000 Message-ID: <75A199C5D243C741BF3D3F1EBCEF9BA503B3439E@E03MVZ1-UKDY.domain1.systemhost.net> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Open issues on architecture draft. Thread-Index: AcgrfK0U7r6BeHC2SK6olzcaLnkvGwAEHrbAAFyLSkA= From: To: X-OriginalArrivalTime: 22 Nov 2007 13:32:20.0976 (UTC) FILETIME=[1BD37700:01C82D0C] X-Spam-Score: -1.0 (-) X-Scan-Signature: 67c1ea29f88502ef6a32ccec927970f0 Subject: [PCN] Open issues on architecture draft. X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi, Hi, Scott & Steven asked me to prepare a list of open/problematic/controversial issues on the draft-pcn-architecture-02.=20 Please feel free to add things that are missing. We should try & close these issues, or at least advance them, before Vancouver. To make the meeting itself as useful as poss. I've divided them into issues that I think need some discussion, and issues that I think are closed but I want to make sure. Known issues needing some discussion =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 1. OAM (Section 8) The text needs reviewing. Personally I think it needs only minor refinements, but it needs an OAM expert to read it. See bob's request http://www1.ietf.org/mail-archive/web/pcn/current/msg00911.html=20 2. Partially PCN-capable tunnel (ie one end of the tunnel is in the PCN-domain, the other end isn't) the new text in S5.7 needs checking.=20 See http://www1.ietf.org/mail-archive/web/pcn/current/msg00929.html=20 3. Probing I believe the current S7 (new in version -02) is too much of a discussion to be appropriate for this architecture doc. There's no consensus on: A) in what circumstance(s) probing is actually useful or not B) whether the decision on A can be punted until later (although probing isn't, I think, in the scope of the initial Charter, the answer to A seems to affect significantly what kind of marking behaviour is good). Need to decide on this for all 3 (or 4) reasons that people have discussed for probing (see S7.3): (1) no PCN-traffic on the ingress-egress-aggregate, so can't estimate level of pre-congestion on it (2) ECMP (a) traffic unbalanced across ECMP paths, so decision based on level of pre-congestion on ingress-egress-aggregate is likely to be inaccurate (b) base decision on level of pre-congestion on ECMP path, but no PCN-traffic on the path (3) PCN-domain extends towards end terminals, probe for every admission decision for what it's worth, my personal opinion is: (1) & (2b) it isn't worth probing for (Lars's solution of "just admitting" has equal or fewer drawbacks) (3) is out of scope=20 (2a) I'm undecided about 4. ECN support in a PCN domain just raised by Magnus http://www1.ietf.org/mail-archive/web/pcn/current/msg00923.html=20 Issues that I think are closed=20 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D no need to say anything if you agree they're closed, but shout now if you want to discuss them! 5. Open issues in S6 S6 lists some open issues and says " NOTE: Potential solutions are out of scope for this document." Is this ok? Personally I think it's ok. If not it would probably be most problematic for the topics where we know of no ideal solutions (ECMP, flash crowds & maybe tunnelling) NB Just to be clear: "Open issues" in S6 means: issues that need careful consideration by the operator in their deployment situation, and sometimes during subsequent PCN WG work eg the Milestone on PCN-boundary-node behaviour. It doesn't mean: "issues that will be solved in a later version of this architecture draft".=20 6. Generic signalling model archit draft says 'you could make decision at ingress, egress or centralised node & this will involve signalling of (1) measurements from metering node to decision node (2) decision from decision node to node(s) doing admission, termination & policing of flows.' It was deliberately left vague. Assumed that the later Milestones would cover what the options were in detail, their reqts etc for boundary-node behaviour & signalling protocols.=20 7. Vagueness on marking behaviours, encoding schemes, edge measurements Archit draft basically says these things will be decided by other Milestone docs. Assume this is ok.=20 Qus: when the decisions are made, should we revise the archit doc to reflect this (eg it would simplify text in S4.1 & 4.2)? if decision allows more than one option, presumably this would be under operator control either at deployment time or through Config OAM - should the archit doc be revised later eg to capture how to choose which option? My view: we shouldn't hold up the archi doc for this. 8. Introduction is this understandable? Should it offer a gentler intro, eg as an evolutionary step on the ietf qos activities? My view: it's ok [but then I've read it several 100s times!] 9. Charter scope there's text in S3 Assumptions like "The PCN charter restricts the initial scope...". Someone might say 'an RFC should be written independently of what a charter says' Personally I think it's ok, it reflects the situation at the time it's written. Also I found it too hard to write without referring to the Charter's scope. 10. Terminology there is always something to discuss 11. Refs There are mixed formats, rfcs aren't in order, etc Help: Is there xml in XMLmindXML editor format with best practice? _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 22 08:49:42 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvCR3-00023n-Ul; Thu, 22 Nov 2007 08:49:41 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IvCR1-0001uw-UV for pcn-confirm+ok@megatron.ietf.org; Thu, 22 Nov 2007 08:49:39 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvCR1-0001s0-I6 for pcn@ietf.org; Thu, 22 Nov 2007 08:49:39 -0500 Received: from rotterdam.ewi.utwente.nl ([130.89.10.5]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IvCR0-0007Jc-QM for pcn@ietf.org; Thu, 22 Nov 2007 08:49:39 -0500 Received: from utip105 (utip105.ewi.utwente.nl [130.89.13.76]) by rotterdam.ewi.utwente.nl (8.13.6/8.13.6) with ESMTP id lAMDnPJc016490; Thu, 22 Nov 2007 14:49:37 +0100 (MET) From: "Georgios Karagiannis" To: "'Geib, Ruediger'" References: <000201c82d05$e7d15040$4c0d5982@dynamic.ewi.utwente.nl> <1B6169C658325341A3B8066E23919E1C4C14EF@S4DE8PSAANK.mitte.t-com.de> Subject: RE: [PCN] Encoding PCN-markings into DSCPs - are thereanyinteractions with ECMP? Date: Thu, 22 Nov 2007 14:49:20 +0100 Message-ID: <000501c82d0e$7eb78e90$4c0d5982@dynamic.ewi.utwente.nl> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 11 Thread-Index: Acgs4WGvKbHSmuQVRPu3zgIC+pF8DQAAEibwAAhw/CAAATWFoAABiF7g In-Reply-To: <1B6169C658325341A3B8066E23919E1C4C14EF@S4DE8PSAANK.mitte.t-com.de> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3198 X-Spam-Score: 0.088 () AWL X-Scanned-By: MIMEDefang 2.52 on 130.89.10.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0rc3 (rotterdam.ewi.utwente.nl [130.89.10.5]); Thu, 22 Nov 2007 14:49:38 +0100 (MET) X-Spam-Score: 0.0 (/) X-Scan-Signature: 25eb6223a37c19d53ede858176b14339 Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Rudiger Thank you very much! Best regards, Georgios > -----Original Message----- > From: Geib, Ruediger [mailto:Ruediger.Geib@t-systems.com] > Sent: donderdag 22 november 2007 14:08 > To: karagian@cs.utwente.nl > Cc: pcn@ietf.org > Subject: RE: [PCN] Encoding PCN-markings into DSCPs - are > thereanyinteractions with ECMP? > > Hi Georgios, > > true, the DSCP shouldn't be used to influence ECMP decissions. > IP adresses and may be ports. I'd have to ask my staff to get > details. I've no problem to recommend not to base ECMP > decisions on DSCPs. > > Regards, > > Rudiger > > |-----Original Message----- > |From: Georgios Karagiannis [mailto:karagian@cs.utwente.nl] > |Sent: Thursday, November 22, 2007 1:48 PM > |To: Geib, Rudiger > |Cc: pcn@ietf.org > |Subject: RE: [PCN] Encoding PCN-markings into DSCPs - are > |thereanyinteractions with ECMP? > | > | > |Hi Ruediger > | > |An operator can however choose which type of ECMP algorithm > should be > |used in its network, or? > |Which types of ECMP algorithm is Deutsche Telecom using? > | > |We have to clarify which types of ECMP algorithms will be > affected when > |the DSCP is used for PCN purposes. > | > |Not all the ECMP algorithms are affected. I am not sure > about it but if > |the ECMP algorithms that are using DSCP to select packets for load > |sharing might not be affected (when DSCP is used for PCN > purposes) if > |the selection process in addition to DSCP also uses the 5 tuple (for > |flow identification). > | > | > |Best regards, > |Georgios > | > | > |> -----Original Message----- > |> From: Geib, Ruediger [mailto:Ruediger.Geib@t-systems.com] > |> Sent: donderdag 22 november 2007 9:40 > |> To: karagian@cs.utwente.nl > |> Cc: pcn@ietf.org > |> Subject: RE: [PCN] Encoding PCN-markings into DSCPs - are there > |> anyinteractions with ECMP? > |> > |> Georgios, > |> > |> if you want to convince Deutsche Telekom's backbone > engineering team > |> to turn off ECMP, you must offer them very convincing > arguments. I'd > |> suggest the PCN WG to expect or allow ECMP to be active > within a PCN > |> network, at least if is a carrier network. > |> > |> Regards, > |> > |> Rudiger > |> > |> |Georgios: I am not sure about the above! An operator > could at least > |> |disallow the ECMP option. > |> | > |> |Best regards, > |> |Georgios > |> | > |> | > |> | > |> | > |> |On 11/22/2007, "philip.eardley@bt.com" > | wrote: > |> | > |> |>Hi, > |> |>Have extracted this issue as a separate thread. > |> |>The context of the discussion below is encoding > PCN-markings into > |> |>different DSCPs (ie not touching the ECN field). Question is, > |> |if you do > |> |>this are there any nasty interactions with ECMP. > |> |>Any comments? > |> |>phil > |> |> > |> |>> > > |> |>> >Major disadvantage: interactions with ECMP. ECMP algos > |can use a > |> |>> >packet's DSCP when deciding what interface to fwd it on. In a > |> |>PCN-domain > |> |>> >running ECMP, for a flow which has some packets > |> PCN-marked and some > |> |>> >unmarked. its packets will travel over more than one > |> path. This may > |> |>well > |> |>> >lead to mis-ordering. > |> |>> >To me, this means that the S3.3 DSCP option is a > |> non-starter in any > |> |>> >PCN-domain that uses an ECMP algorithm that uses the DSCP. > |> |>> > |> |>> Georgios: well I do not see this as a disadvantage. > |> |However, what you > |> |>> mention about having an ECMP algorithm that uses DSCP is > |relevant. > |> |>> Thus what we can do is to include a sentence in that > |> emphasizes that > |> |>the > |> |>> ECMP situation is solved only if the ECMP algorithm does > |> |not use DSCP > |> |>> during the implementation of the equal cost multi path > algorithm. > |> |>> > |> |>> > NB ECMP algos > |> |>> >are proprietary to router vendors, ie operator has poor > |> |control over > |> |>> >them. > |> |>> > |> |>> Georgios: I am not sure about the above! An operator > |> could at least > |> |>> disallow the ECN option. > |> |>> > |> |>> > |> |>> > |> |> > |> |> > |> |>_______________________________________________ > |> |>PCN mailing list > |> |>PCN@ietf.org > |> |>https://www1.ietf.org/mailman/listinfo/pcn > |> | > |> | > |> |_______________________________________________ > |> |PCN mailing list > |> |PCN@ietf.org > |> |https://www1.ietf.org/mailman/listinfo/pcn > |> | > |> > | > | > | > | > |_______________________________________________ > |PCN mailing list > |PCN@ietf.org > |https://www1.ietf.org/mailman/listinfo/pcn > | > _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 22 12:04:04 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvFTA-00085w-P7; Thu, 22 Nov 2007 12:04:04 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IvFT9-00084c-PI for pcn-confirm+ok@megatron.ietf.org; Thu, 22 Nov 2007 12:04:03 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvFT9-00084H-5p; Thu, 22 Nov 2007 12:04:03 -0500 Received: from rotterdam.ewi.utwente.nl ([130.89.10.5]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IvFT3-0007Nh-7j; Thu, 22 Nov 2007 12:04:03 -0500 Received: from utip105 (utip105.ewi.utwente.nl [130.89.13.76]) by rotterdam.ewi.utwente.nl (8.13.6/8.13.6) with ESMTP id lAMH3q2P021660; Thu, 22 Nov 2007 18:03:56 +0100 (MET) From: "Georgios Karagiannis" To: , , "'tsvwg'" Date: Thu, 22 Nov 2007 18:03:47 +0100 Message-ID: <002101c82d29$a8779f20$4c0d5982@dynamic.ewi.utwente.nl> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 11 Thread-Index: AcgtKaUM6aO1LskDT3KzpbkhSeWGXQ== X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3198 X-Spam-Score: 0.238 () AWL,FVGT_s_OBFU_Q0 X-Scanned-By: MIMEDefang 2.52 on 130.89.10.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0rc3 (rotterdam.ewi.utwente.nl [130.89.10.5]); Thu, 22 Nov 2007 18:03:56 +0100 (MET) X-Spam-Score: 0.3 (/) X-Scan-Signature: 944ecb6e61f753561f559a497458fb4f Cc: Subject: [PCN] CFP - IWQoS 2008 X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org ------------------------------------------------------------ 16th INTERNATIONAL WORKSHOP ON QUALITY OF SERVICE (IWQoS 2008) June 2-4, 2008 University of Twente, Enschede, The Netherlands http://iwqos08.ewi.utwente.nl/ ------------------------------------------------------------ *** Abstract deadline: January 21, 2008 *** *** Paper submission deadline: January 28, 2008 *** CALL FOR PAPERS Since 1994, IWQoS has served as the prime annual workshop on Quality of Service, providing an international forum for the presentation and discussion of cutting edge research in the field. Building on the success of previous workshops, the objective of the workshop is to bring together researchers, developers, and practitioners working in this area to discuss recent and innovative results, and to identify future directions and challenges in developing practical computer/communication systems where predictable, controlled, and robust performance is a central requirement. IWQoS has a long standing tradition of being highly interactive, while maintaining highest standards of competitiveness and excellence. The scope of the workshop covers the main and currently important aspects of QoS research, including related issues such as availability, reliability, security, pricing, resource management, and performance guarantees. The program will include the following highlights: - Key-note by industrial speaker - A session with short position papers - Key-note and invited paper The session with short position papers will be highly interactive and leave much room for the audience to get involved. The short papers will go through the general reviewing process. TECHNICAL PROGRAM As in previous years, the workshop covers a broad spectrum of QoS issues in communications networking. IWQoS 2008 will particularly emphasize the increasing importance of application level QoS and perceived QoS, i.e., the quality of applications as experienced by the end users, which is usually expressed in terms of a 'mean opinion score' (MOS). Relevant topics for the workshop include the following: - End-to-end QoS provisioning in heterogeneous network environments - Application protocols and QoS - QoS in overlay and peer-to-peer networks - QoS adaptation, adaptive applications - Perceived QoS for VoIP and multimedia applications - Mapping network performance to perceived QoS - Service-level agreements (SLAs) and QoS - QoS modelling for gaming applications - Measurement based QoS estimation and verification - QoS in wireless, mobile, ad hoc and sensor networks - QoS in context aware systems (also related to mobility) - End-to-end QoS signalling and support - QoS in home networks, intranets and Metro Ethernets - Operating systems support and end-system design for QoS COMMITTEE General chair: Boudewijn Haverkort, University of Twente TPC co-chairs: Hans van den Berg, TNO Information and Communication Technology Gunnar Karlsson, KTH, Royal Institute of Technology Local chair: Georgios Karagiannis, University of Twente BEST STUDENT PAPER AWARD Award will be given at the conference to the best student paper, whose first author is/was a student at the time of the paper submission. PROCEEDINGS IWQoS invites submission of manuscripts that present original research results and that neither have been published nor are currently under review for another conference or journal. Any previous or simultaneous publication of related material should be explicitly noted in the submission. The proceedings will be published by IEEE ComSoc. We also solicit submissions for the short position papers session. The short papers will be included in the conference proceedings. IMPORTANT DATES - Abstract deadline: January 21, 2008, 11:59pm CET - Paper submission deadline: January 28, 2008, 11:59pm CET - Notification of acceptance: March 18, 2008 - Camera-ready papers due: April 8, 2008 - Early registration deadline and hotel reservation cut-off date: April 15th, 2008 - Workshop dates: June 2-4, 2008 - Workshop reception and welcome party: June 1, 2008 _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 22 19:12:37 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvM9s-0001p3-6e; Thu, 22 Nov 2007 19:12:36 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IvM9r-0001or-Fq for pcn-confirm+ok@megatron.ietf.org; Thu, 22 Nov 2007 19:12:35 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvM9r-0001oe-5a for pcn@ietf.org; Thu, 22 Nov 2007 19:12:35 -0500 Received: from smtp105.rog.mail.re2.yahoo.com ([206.190.36.83]) by ietf-mx.ietf.org with smtp (Exim 4.43) id 1IvM9o-00051d-RJ for pcn@ietf.org; Thu, 22 Nov 2007 19:12:35 -0500 Received: (qmail 57553 invoked from network); 23 Nov 2007 00:12:31 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=rogers.com; h=Received:X-YMail-OSG:Message-ID:Date:From:User-Agent:MIME-Version:To:CC:Subject:References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=Fj2+d5NZfcDD9g4FMD8rea4q5eOTnLPP3mrSYNKbLpJtb1yX9iXLzFEIY/PuPHiQquR/xkiHodS0sCP9H/NId1/12/QSNYn5BqpmgP7F/OKlzebwoI3i6S2gYxi5Q8jfhxGLj1eqI3oFPGgTq0MzCBrh0TFJ3SYPEThW6kGJZNA= ; Received: from unknown (HELO ?192.168.9.189?) (tom.taylor@rogers.com@210.21.226.44 with plain) by smtp105.rog.mail.re2.yahoo.com with SMTP; 23 Nov 2007 00:12:31 -0000 X-YMail-OSG: GHiBAL0VM1laAEJlhCndtb6gxwtBcdrqvELmegqzqNafwyXQLZMg0uuQPuyDNDQ9Aw-- Message-ID: <47461AED.6090004@rogers.com> Date: Thu, 22 Nov 2007 19:12:29 -0500 From: Tom Taylor User-Agent: Thunderbird 2.0.0.9 (Windows/20071031) MIME-Version: 1.0 To: philip.eardley@bt.com Subject: Re: [PCN] Open issues on architecture draft. References: <75A199C5D243C741BF3D3F1EBCEF9BA503B3439E@E03MVZ1-UKDY.domain1.systemhost.net> In-Reply-To: <75A199C5D243C741BF3D3F1EBCEF9BA503B3439E@E03MVZ1-UKDY.domain1.systemhost.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Scan-Signature: 30ac594df0e66ffa5a93eb4c48bcb014 Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Looks like I'm too late to comment :) -- I've tried and failed to find a copy of arch-02 on the web. The PCN charter page link and Datatracker give me "document not found", and it isn't in ISI's archive. So far I haven't found it on Google either. philip.eardley@bt.com wrote: > Hi, > ... _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Fri Nov 23 02:40:44 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvT9Y-0008Va-44; Fri, 23 Nov 2007 02:40:44 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IvT9X-0008VV-46 for pcn-confirm+ok@megatron.ietf.org; Fri, 23 Nov 2007 02:40:43 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvT9W-0008VI-GS for pcn@ietf.org; Fri, 23 Nov 2007 02:40:42 -0500 Received: from smtp104.rog.mail.re2.yahoo.com ([206.190.36.82]) by chiedprmail1.ietf.org with smtp (Exim 4.43) id 1IvT9W-0006ii-5l for pcn@ietf.org; Fri, 23 Nov 2007 02:40:42 -0500 Received: (qmail 90705 invoked from network); 23 Nov 2007 07:40:41 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=rogers.com; h=Received:X-YMail-OSG:Message-ID:Date:From:User-Agent:MIME-Version:To:CC:Subject:References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=fqurmQsPwHrsQ3Mz0QotYmhbc6wPEY6q9R5y4IFkJGVd1AHBGQtyOUVkNhG+pJlhbFuwSpCcPL5kFvbLYNL1pgfPg3xgKSCajJV+atpNA+ooYEVQbu90jrr1CLMiyGjlFgDOPue9zbV1ym48pn2bYvoss4MtWoXn1PzyyOvlHpo= ; Received: from unknown (HELO ?192.168.9.189?) (tom.taylor@rogers.com@210.21.226.44 with plain) by smtp104.rog.mail.re2.yahoo.com with SMTP; 23 Nov 2007 07:40:41 -0000 X-YMail-OSG: l8S7aAAVM1n0C5sq8qgBX5qHqmmpjPMKgL3K.OJ5B91knB462TqG4T54cWzBDPkTAg-- Message-ID: <474683F7.5050300@rogers.com> Date: Fri, 23 Nov 2007 02:40:39 -0500 From: Tom Taylor User-Agent: Thunderbird 2.0.0.9 (Windows/20071031) MIME-Version: 1.0 To: philip.eardley@bt.com Subject: Re: [PCN] Open issues on architecture draft. References: <75A199C5D243C741BF3D3F1EBCEF9BA503B3439E@E03MVZ1-UKDY.domain1.systemhost.net> <47461AED.6090004@rogers.com> In-Reply-To: <47461AED.6090004@rogers.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Scan-Signature: 798b2e660f1819ae38035ac1d8d5e3ab Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org I found the message in which you distributed the document. The OAM section looks great, though I may think of something later. I'll probably have some editorial suggestions for the earlier sections, but I'll send those privately. Tom Taylor wrote: > Looks like I'm too late to comment :) -- I've tried and failed to find a > copy of arch-02 on the web. The PCN charter page link and Datatracker > give me "document not found", and it isn't in ISI's archive. So far I > haven't found it on Google either. > > philip.eardley@bt.com wrote: >> Hi, >> > ... > > > _______________________________________________ > PCN mailing list > PCN@ietf.org > https://www1.ietf.org/mailman/listinfo/pcn > > _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Fri Nov 23 04:59:07 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvVJO-0007SJ-UK; Fri, 23 Nov 2007 04:59:02 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IvVJN-0007S4-Dg for pcn-confirm+ok@megatron.ietf.org; Fri, 23 Nov 2007 04:59:01 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvVJM-0007Rp-Gr for pcn@ietf.org; Fri, 23 Nov 2007 04:59:00 -0500 Received: from tcmail31.telekom.de ([217.6.95.238]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IvVJJ-0002Qb-KC for pcn@ietf.org; Fri, 23 Nov 2007 04:59:00 -0500 Received: from S4DE8PSAANQ.mitte.t-com.de by tcmail31.telekom.de with ESMTP; Fri, 23 Nov 2007 10:58:53 +0100 Received: from S4DE8PSAANK.mitte.t-com.de ([10.151.229.10]) by S4DE8PSAANQ.mitte.t-com.de with Microsoft SMTPSVC(6.0.3790.3959); Fri, 23 Nov 2007 10:58:52 +0100 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [PCN] I-D Action:draft-ietf-pcn-architecture-02.txt Date: Fri, 23 Nov 2007 10:58:52 +0100 Message-Id: <1B6169C658325341A3B8066E23919E1C4C1500@S4DE8PSAANK.mitte.t-com.de> In-Reply-To: <75A199C5D243C741BF3D3F1EBCEF9BA503B34396@E03MVZ1-UKDY.domain1.systemhost.net> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] I-D Action:draft-ietf-pcn-architecture-01.txt Thread-Index: AcgqxqGl81rUg3oXSxKQPQt45B53dQAATG8gAIiSlTAAA42YQAAsEGcQ From: "Geib, Ruediger" To: X-OriginalArrivalTime: 23 Nov 2007 09:58:52.0769 (UTC) FILETIME=[73F3ED10:01C82DB7] X-Spam-Score: -1.0 (-) X-Scan-Signature: 6ffdee8af20de249c24731d8414917d3 Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Phil, I've read the chapters on ECMP, probing and OAM. I'm=20 not an expert on the latter however. Your text captures=20 most of the discussion. I however think, it is to=20 coarse in leading to a decision whether to probe or=20 not to. My comments center around this issue. Regards, Rudiger ----------hereWeGo------------------------------------ 6. Design goals and challenges 7th bullet point "ECMP and signalling: It is possible that, in a PCN-domain running ECMP, the signalling packets (eg RSVP, NSIS)=20 follow a different path than the data packets - it depends on which fields the ECMP algorithm uses. This could matter if the signalling packets are used as probes." We can be more specific on this one, I think: "ECMP and signalling: If the ECMP hashing algorithm applied=20 by the routers of a PCN domain is based on more than just=20 the IP addresses of a packet, is possible that the=20 signalling packets (eg RSVP, NSIS) follow a different path=20 than the data packets. If the ECMP hashing algorithm can be=20 restricted to a packets IP adresses, per flow RSVP and NSIS=20 signaling messages may be transmitted coloured as PCN=20 traffic. They will then pass the same links as the data=20 packets and hence allow identification of flows requesting=20 admission on links approaching the upper PCN rate=20 (independent of the marking algorithm)." The statement expresses the minimum solution we can offer=20 to deal with the problem. I prefer this to leaving the reader with "this could matter". ----------------------------------------------------------- 7.1. Introduction (to probing) 2nd paragraph, last sentence "This risk could be lessened by configuring on each link=20 sufficient 'safety margin' above the PCN-lower-rate." Please add: "Further, the decision could be based on whether any of=20 the boundary nodes involved in admission of the new flow on the unused aggregate is aware of pre-congestion for any of his already admitted aggregates. If there is no=20 pre-congestion, the flow can be admitted with little=20 risk. " I prefer to be explicit here, as I don't think the options=20 are just "being optimistic" or "probing". ------------------------------------------------------------ 3rd paragraph "The PCN-ingress-node explicitly determines, before admitting the prospective new flow, whether the ingress-egress- aggregate can support it." I'd suggest to add: "If necessary, the PCN-ingress-node...." I don't completely object to probing as long as I'm not sure=20 that all options avoiding it have been explored. I'd like to make sure that it is a "last resort". That's why I prefer to have the statements on probing to be less "binary", just=20 leaving you with an on/off decision.=20 ------------------------------------------------------------- 7.2. Probing functions Please change first bullet point to: " Make decision that probing is needed. As described above,=20 this is when the ingress-egress-aggregate or the ECMP path=20 is likely to carry no PCN-traffic." ^^^^^^^^^^^^^^^^^^ -------------------------------------------------------------- 2nd bullet point Please change to: " (if required) Communicate the request that probing is needed - the PCN-egress-node signals to the PCN-ingress-node that=20 probing is needed (or the other way around). ^^^^^^^^^^^^^^^^^^^^^^^^^^ The ingress could also indicate that probing is required and=20 will start to the egress. -------------------------------------------------------------- 7.3. Discussion of rationale for probing, its downsides=20 and open issues I don't agree to the negative statement on the PCN coloured=20 signaling messages (mentioned in third viewpoint). Given that an ECMP algorithm is limited to evaluate addresses, PCN=20 coloured per flow signaling messages are good means to avoid=20 unnecessary termination of admitted flows. The signalling=20 packet is received with a congestion mark, and the new flow=20 is blocked. This method solves downside issues of the other=20 two viewpoints - it is just a single packet transmitted via=20 a link approaching the upper rate. So it is unlikly that it=20 causes admitted flows to be terminated. Note that I don't regard this as probing, as the signaling=20 message is transmitted anyway. --------------------------------------------------------------- My general take on the probing disussion: Leave the=20 introduction in the architecture and refer to another document=20 for a detailed discussion. Alternatively, move 7.2 and 7.3 to=20 the annex of the architecture doc. ----------------------------------------end------------------- _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Fri Nov 23 05:39:18 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvVwM-0006Qo-2M; Fri, 23 Nov 2007 05:39:18 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IvVwK-0006Qg-SD for pcn-confirm+ok@megatron.ietf.org; Fri, 23 Nov 2007 05:39:16 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvVwK-0006QY-Ie for pcn@ietf.org; Fri, 23 Nov 2007 05:39:16 -0500 Received: from wrzx28.rz.uni-wuerzburg.de ([132.187.3.28] helo=mailrelay.rz.uni-wuerzburg.de) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IvVwG-0003eo-0y for pcn@ietf.org; Fri, 23 Nov 2007 05:39:16 -0500 Received: from virusscan.mail (localhost [127.0.0.1]) by mailrelay.mail (Postfix) with ESMTP id 30893C433; Fri, 23 Nov 2007 11:39:07 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by virusscan.mail (Postfix) with ESMTP id 23338C563; Fri, 23 Nov 2007 11:39:07 +0100 (CET) Received: from europa.informatik.uni-wuerzburg.de (wicx01.informatik.uni-wuerzburg.de [132.187.11.1]) by mailmaster.uni-wuerzburg.de (Postfix) with ESMTP id DBA4AC510; Fri, 23 Nov 2007 11:39:05 +0100 (CET) Received: from nero.informatik.uni-wuerzburg.de (win3005.informatik.uni-wuerzburg.de [132.187.106.5]) by europa.informatik.uni-wuerzburg.de (8.11.3/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id lANAd5I31282; Fri, 23 Nov 2007 11:39:05 +0100 Received: from [127.0.0.1] (nero.informatik.uni-wuerzburg.de [132.187.106.5]) by nero.informatik.uni-wuerzburg.de (Postfix) with ESMTP id 60C2F6F591; Fri, 23 Nov 2007 11:31:22 +0100 (CET) Message-ID: <4746AE39.6000409@informatik.uni-wuerzburg.de> Date: Fri, 23 Nov 2007 11:40:57 +0100 From: Michael Menth Organization: University of Wuerzburg User-Agent: Thunderbird 2.0.0.9 (Windows/20071031) MIME-Version: 1.0 To: Philip Eardley Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at uni-wuerzburg.de X-Spam-Score: 0.0 (/) X-Scan-Signature: ce732c7d36989a1bd55104ba259c40a1 Cc: pcn@ietf.org Subject: [PCN] Comments on draft-ietf-pcn-architecture-02.txt X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: menth@informatik.uni-wuerzburg.de List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Phil, here are my comments to the -02 version of the architecture draft. Regards, Michael ===================== Introduction "PCN-egress-nodes make measurements of the packet markings and send information as necessary to the nodes that make the decision about which PCN-flows to accept/reject or terminate, based on this information." [Michael] "PCN-egress nodes monitor the packet markings and ..." (measurement is not necessarily required, both for admission and termination.) ===================== "the admission control mechanism limits the PCN-traffic on each link to *roughly* its PCN-lower-rate and the flow termination mechanism limits the PCN-traffic on each link to *roughly* its PCN-upper-rate." [Michael] Assuming this sentence is based on agreement, I cannot refrain from pointing out that texts on PCN are better readable with terms "admissible rate" and "supportable rate" instead of PCN-lower-rate and PCN-upper-rate. ===================== "The PCN-lower-rates can be chosen small enough that admitted traffic can still be carried after a rerouting in most failure cases." [Michael] The general concept of resilient admission control has been proposed in [Menth04g] and applied to PCN in [PCN-Config] including algorithms how to set PCN rate thresholds appropriately. [Menth04g] M. Menth, S. Kopf, and J. Charzinski: "Network Admission Control for Fault-Tolerant QoS Provisioning" in IEEE High-Speed Networks for Multimedia Communication (HSNMC), June 2004 [PCN-Config] M. Menth, and M. Hartmann: "PCN-Based Resilient Network Admission Control: The Impact of a Single Bit", ", 2007. ===================== Terminology "o Excess-rate-marking: a PCN-marking behaviour such that the amount of PCN-traffic that is PCN-marked is equal to the amount that exceeds a particular rate (either the PCN-lower-rate or PCN-upper-rate). NOTE: The definition reflects the overall intent rather than its instantaneous behaviour, since the rate measured at a particular moment depends on the behaviour, its implementation and the traffic's variance as well as its rate." [Michael] The term "Excess-rate-marking" implies that exactly the excess rate is marked. Modifications of this algorithm using e.g. marking frequency reduction don't deserve this name anymore. This is different with "Excess-traffic-marking": if all packets exceeding a particular rate are marked, it's exactly the excess traffic rate, but the algorithm may also mark fewer packets. "o Excess-traffic-marking: a PCN-marking behaviour marking packets exceeding a particular rate (either the PCN-lower-rate or PCN-upper-rate). ..." ===================== "o Pre-congestion: a condition of a link within a PCN-domain in which the PCN-node performs PCN-marking, in order to provide an "early warning" of potential congestion before there is any significant build-up of PCN-packets in the real queue." [Michael] Pre-congestion is always connected to a PCN rate threshold. "o X-Pre-congestion: a condition of a link within a PCN-domain in which the PCN-node performs PCN-marking since the PCN-traffic of the link has *roughly* exceeded the PCN rate threshold "X" (e.g. PCN-lower-rate or PCN-upper-rate); it is done to provide an "early warning" of potential congestion before there is any significant build-up of PCN-packets in the real queue. When the context is clear, "X" can be omitted." ===================== 4.1 "However, note that this description reflects the overall intent of the algorithm rather than its instantaneous behaviour, since the rate measured at a particular moment depends on the detailed algorithm, its implementation (eg virtual queue, token bucket...) and the traffic's variance as well as its rate (eg marking may well continue after a recent overload even after the instantaneous rate has dropped)." [Michael]: virtual queue and token bucket are absolutely equivalent paradigms on which metering and marking may be based. see also: http://www.ietf.org/internet-drafts/draft-charny-pcn-comparison-00.txt Just omit the example! ===================== (Hence, by analogy with ECN we call our mechanism Pre-Congestion Notification.) [Michael] This should come earlier in the document. E.g. in the terminology section. ===================== 5.5 o PCN-meter at PCN-egress-node - make "measurements of PCN-traffic" from a particular PCN-ingress-node. [Michael] Could you add "(if required)" to that point? 3sm http://www.ietf.org/internet-drafts/draft-babiarz-pcn-3sm-01.txt does not need metering or measurement of marked packets for flow termination. ===================== o Make decision about flow termination - use the "measurements of PCN-traffic" to decide which PCN-flow or PCN-flows to terminate. The decision takes account of policy and application layer requirements. [Michael] Similar issue: could you add "possibly" somewhere? ===================== 5.7 o NB the order of increasing severity is: unmarked; PCN-marking with first encoding (ie associated with the PCN-lower-rate); PCN- marking with second encoding (ie associated with the PCN-upper- rate) [Michael] What does "NB" stand for? ===================== 7.2 o (if required) Generate probe traffic - the PCN-ingress-node generates the probe traffic. The appropriate number (or rate) of probe packets will depend on the PCN-marking algorithm; for example an excess-rate-marking algorithm generates fewer PCN-marks than a threshold-marking algorithm, and so will need more probe packets. and 7.3 The open issues associated with this viewpoint include: o What rate and pattern of probe packets does the PCN-ingress-node need to generate, so that there's enough traffic to make the admission decision? [Michael] We've done some work on this. See Table 1 in 4.1 of http://www.ietf.org/internet-drafts/draft-menth-pcn-performance-01.txt ===================== 7.3 o Probing adds delay to the admission control process. o Sufficient probing traffic has to be generated to test the pre- congestion level of the ingress-egress-aggregate. But the probing traffic itself may cause pre-congestion, causing other PCN-flows to be blocked or even terminated - and in the flash crowd scenario there will be probing on many ingress-egress-aggregates. [Michael] Could you add "possibly" please? These statements are not true for the probing mechanisms in 2.3.2 of http://www.ietf.org/internet-drafts/draft-babiarz-pcn-3sm-01.txt ===================== The downsides of probing for this viewpoint are: o Probing adds delay to the admission control process. o Sufficient probing traffic has to be generated to test the pre- congestion level of the ECMP path. But there's the risk that the probing traffic itself may cause pre-congestion, causing other PCN-flows to be blocked or even terminated. [Michael] Could you please add "possibly" to these points? ===================== Viewpoint 3 or probably extra viewpoint: [Michael] Probing might be used because it is easier than measuring and signalling aggregate packet markings. This is the case when a signalling protocol already exists, e.g., to find out the correct ingress-egress-aggregate of a flow. Then, this signalling protocol might be reused for probing purposes saving complex operations in the egress node and signalling. An example is given in 2.4 of http://www.ietf.org/internet-drafts/draft-babiarz-pcn-3sm-01.txt Also 4.3 of http://tools.ietf.org/wg/pcn/draft-briscoe-pcn-boundary-behav-00.txt suggests to use a protocol to find out the right egress node such that probing would in fact lead to simplifications. ===================== 8.1.2 Parameters [Michael] A citation to our work on parameter settings would be appropriate. [PCN-Config] M. Menth, and M. Hartmann: "PCN-Based Resilient Network Admission Control: The Impact of a Single Bit", ", 2007. -- Dr. Michael Menth, Assistant Professor University of Wuerzburg, Institute of Computer Science Am Hubland, D-97074 Wuerzburg, Germany, room B206 phone: (+49)-931/888-6644, fax: (+49)-931/888-6632 mailto:menth@informatik.uni-wuerzburg.de http://www3.informatik.uni-wuerzburg.de/research/ngn _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Fri Nov 23 09:43:11 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvZkM-0005m1-JZ; Fri, 23 Nov 2007 09:43:10 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IvZkK-0005lv-SV for pcn-confirm+ok@megatron.ietf.org; Fri, 23 Nov 2007 09:43:08 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvZkK-0005ln-HX for pcn@ietf.org; Fri, 23 Nov 2007 09:43:08 -0500 Received: from zrtps0kn.nortel.com ([47.140.192.55]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IvZkK-0002wn-1V for pcn@ietf.org; Fri, 23 Nov 2007 09:43:08 -0500 Received: from zcarhxm1.corp.nortel.com (zcarhxm1.corp.nortel.com [47.129.230.97]) by zrtps0kn.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id lANEh1019476; Fri, 23 Nov 2007 14:43:01 GMT X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [PCN] IETF 70 *DRAFT* PCN agenda Date: Fri, 23 Nov 2007 09:42:45 -0500 Message-ID: <9671A92C3C8B5744BC97F855F7CB6465134D3186@zcarhxm1.corp.nortel.com> In-Reply-To: <1195611870.28291.72.camel@neutrino> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] IETF 70 *DRAFT* PCN agenda Thread-Index: Acgr5a+z2bpqFP1bSIuOARifcqIxHAB90Mig References: <1195611870.28291.72.camel@neutrino> From: "Jozef Babiarz" To: "Steven Blake" , "pcn" X-Spam-Score: 0.0 (/) X-Scan-Signature: c3a18ef96977fc9bcc21a621cbf1174b Cc: X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Steve, I agree that the wg needs to focus and reach closure on the architectural draft and survey of encoding and transport choices as they are Nov. 2007 deliverables. I would like to request that the PCN wg get a second two hour meet slot later in the week to discuss Mar. 2008 deliverables, mostly what marking behavior will be standardized for PCN interior nodes and edge-to-edge PCN signalling requirements.=20 Also, I believe that the wg needs to review the posted schedule of deliverables and update it based on the agreements we reach in Vancouver.=20 Regards, Joe email:babiarz@nortel.com Telephone:613-763-6098 -----Original Message----- From: Steven Blake [mailto:steven.blake@ericsson.com]=20 Sent: November 20, 2007 9:25 PM To: pcn Subject: [PCN] IETF 70 *DRAFT* PCN agenda Attached is the draft agenda for the Vancouver meeting, also available at: http://www3.ietf.org/proceedings/07dec/agenda/pcn.txt Note that Scott and I are proposing to do something different from the previous meetings: we are only planning to discuss the PCN architecture draft and Kwok's and Georgios' encoding draft. There are several reasons for this, but to summarize: - These drafts address the first two milestones for PCN. According to the working group charter, we are already late on these milestones (and when working groups are late, the chairs don't get their bonus checks). So it is critical that we make progress on resolving the open issues with these documents so that we can move them to working group last call (1). - Some design decisions for future milestones are depending on the resolution of some open issues with these documents, so by making progress now we should be able to accelerate subsequent work. - We had requests to discuss nine drafts in this meeting, some of which just recently popped into existence with no prior discussion on the mailing list. With only 120 minutes of meeting time, there is not enough time to facilitate good discussions for all of these drafts.=20 So we plan to spend our precious face time on discussion of the open issues that we know we need to resolve as soon as possible. Philip and Kwok have volunteered to each post a list of open issues with their drafts. We should review and refine these lists prior to the meeting, and of course continue to discuss the open issues. In the meeting we will let Philip and Kwok drive the discussion, but anyone with anything to say on these issues will be welcome to queue up for mic and screen time (if you do produce slides (3 max, please), then please send them to the list before the meeting). As always, all working group decisions are finalized on the mailing list. Please send comments on this draft agenda to the list ASAP. (1) draft-chan-pcn-encoding-comparison-01 is not a working group=20 document, so it has no official status. In the absence of any=20 alternative drafts, we need to make progress on this one. (2) If any author feels that comprehension of his or her draft would benefit from slides and/or oral commentary, feel free to post those to the list (and we can arrange hosting for these if necessary). See you in Vancouver! Scott & Steve =3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D= -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D Steven Blake Ericsson/Redback Networks +1 919-472-9913 _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Fri Nov 23 11:13:33 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Ivb9p-0004N3-3Y; Fri, 23 Nov 2007 11:13:33 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Ivb9n-0004Hc-Lz for pcn-confirm+ok@megatron.ietf.org; Fri, 23 Nov 2007 11:13:31 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Ivb9m-0004FI-Iw for pcn@ietf.org; Fri, 23 Nov 2007 11:13:31 -0500 Received: from wrzx28.rz.uni-wuerzburg.de ([132.187.3.28] helo=mailrelay.rz.uni-wuerzburg.de) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1Ivb9i-000523-7r for pcn@ietf.org; Fri, 23 Nov 2007 11:13:30 -0500 Received: from virusscan.mail (localhost [127.0.0.1]) by mailrelay.mail (Postfix) with ESMTP id 9F5A4C09E; Fri, 23 Nov 2007 17:13:25 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by virusscan.mail (Postfix) with ESMTP id 91960C158; Fri, 23 Nov 2007 17:13:25 +0100 (CET) Received: from europa.informatik.uni-wuerzburg.de (wicx01.informatik.uni-wuerzburg.de [132.187.11.1]) by mailmaster.uni-wuerzburg.de (Postfix) with ESMTP id 435AEC09E; Fri, 23 Nov 2007 17:13:23 +0100 (CET) Received: from nero.informatik.uni-wuerzburg.de (win3005.informatik.uni-wuerzburg.de [132.187.106.5]) by europa.informatik.uni-wuerzburg.de (8.11.3/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id lANGDNI01689; Fri, 23 Nov 2007 17:13:23 +0100 Received: from [127.0.0.1] (nero.informatik.uni-wuerzburg.de [132.187.106.5]) by nero.informatik.uni-wuerzburg.de (Postfix) with ESMTP id 3BA786F591; Fri, 23 Nov 2007 17:05:39 +0100 (CET) Message-ID: <4746FC94.6070804@informatik.uni-wuerzburg.de> Date: Fri, 23 Nov 2007 17:15:16 +0100 From: Michael Menth Organization: University of Wuerzburg User-Agent: Thunderbird 2.0.0.9 (Windows/20071031) MIME-Version: 1.0 To: Steven Blake Subject: Re: [PCN] IETF 70 *DRAFT* PCN agenda References: <1195611870.28291.72.camel@neutrino> <9671A92C3C8B5744BC97F855F7CB6465134D3186@zcarhxm1.corp.nortel.com> In-Reply-To: <9671A92C3C8B5744BC97F855F7CB6465134D3186@zcarhxm1.corp.nortel.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at uni-wuerzburg.de X-Spam-Score: 0.0 (/) X-Scan-Signature: 5ebbf074524e58e662bc8209a6235027 Cc: pcn X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: menth@informatik.uni-wuerzburg.de List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Steven, yes, this would certainly be a good idea to advance the PCN work. I have the impression that a lot of work was done since the last meeting and it would be good to give room for official discussions on the achieved results and further steps. Regards, Michael Jozef Babiarz wrote: > Steve, > I agree that the wg needs to focus and reach closure on the > architectural draft and survey of encoding and transport choices as they > are Nov. 2007 deliverables. > > I would like to request that the PCN wg get a second two hour meet slot > later in the week to discuss Mar. 2008 deliverables, mostly what marking > behavior will be standardized for PCN interior nodes and edge-to-edge > PCN signalling requirements. > > Also, I believe that the wg needs to review the posted schedule of > deliverables and update it based on the agreements we reach in > Vancouver. > > > Regards, Joe > email:babiarz@nortel.com > Telephone:613-763-6098 > -----Original Message----- > From: Steven Blake [mailto:steven.blake@ericsson.com] > Sent: November 20, 2007 9:25 PM > To: pcn > Subject: [PCN] IETF 70 *DRAFT* PCN agenda > > Attached is the draft agenda for the Vancouver meeting, > also available at: > > http://www3.ietf.org/proceedings/07dec/agenda/pcn.txt > > Note that Scott and I are proposing to do something different from the > previous meetings: we are only planning to discuss the PCN architecture > draft and Kwok's and Georgios' encoding draft. > > There are several reasons for this, but to summarize: > > - These drafts address the first two milestones for PCN. According to > the working group charter, we are already late on these milestones > (and when working groups are late, the chairs don't get their bonus > checks). So it is critical that we make progress on resolving the > open issues with these documents so that we can move them to working > group last call (1). > > - Some design decisions for future milestones are depending on the > resolution of some open issues with these documents, so by making > progress now we should be able to accelerate subsequent work. > > - We had requests to discuss nine drafts in this meeting, some of which > just recently popped into existence with no prior discussion on the > mailing list. With only 120 minutes of meeting time, there is not > enough time to facilitate good discussions for all of these drafts. > So we plan to spend our precious face time on discussion of the open > issues that we know we need to resolve as soon as possible. > > Philip and Kwok have volunteered to each post a list of open issues with > their drafts. We should review and refine these lists prior to the > meeting, and of course continue to discuss the open issues. In the > meeting we will let Philip and Kwok drive the discussion, but anyone > with anything to say on these issues will be welcome to queue up for mic > and screen time (if you do produce slides (3 max, please), then please > send them to the list before the meeting). > > As always, all working group decisions are finalized on the mailing > list. > > Please send comments on this draft agenda to the list ASAP. > > > (1) draft-chan-pcn-encoding-comparison-01 is not a working group > document, so it has no official status. In the absence of any > alternative drafts, we need to make progress on this one. > > (2) If any author feels that comprehension of his or her > draft would benefit from slides and/or oral commentary, feel free > to post those to the list (and we can arrange hosting for these if > necessary). > > > See you in Vancouver! > > > Scott & Steve > > =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= > Steven Blake > Ericsson/Redback Networks +1 919-472-9913 > > > _______________________________________________ > PCN mailing list > PCN@ietf.org > https://www1.ietf.org/mailman/listinfo/pcn > -- Dr. Michael Menth, Assistant Professor University of Wuerzburg, Institute of Computer Science Am Hubland, D-97074 Wuerzburg, Germany, room B206 phone: (+49)-931/888-6644, fax: (+49)-931/888-6632 mailto:menth@informatik.uni-wuerzburg.de http://www3.informatik.uni-wuerzburg.de/research/ngn _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Fri Nov 23 13:14:55 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Ivd3E-0000iO-NZ; Fri, 23 Nov 2007 13:14:52 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Ivd3D-0000iE-Se for pcn-confirm+ok@megatron.ietf.org; Fri, 23 Nov 2007 13:14:51 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Ivd3D-0000i6-Co for pcn@ietf.org; Fri, 23 Nov 2007 13:14:51 -0500 Received: from imr2.ericy.com ([198.24.6.3]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1Ivd3C-0002Uu-R6 for pcn@ietf.org; Fri, 23 Nov 2007 13:14:51 -0500 Received: from eusrcmw751.eamcs.ericsson.se (eusrcmw751.exu.ericsson.se [138.85.77.51]) by imr2.ericy.com (8.13.1/8.13.1) with ESMTP id lANIEoGD028867 for ; Fri, 23 Nov 2007 12:14:50 -0600 Received: from eusrcmw751.eamcs.ericsson.se ([138.85.77.56]) by eusrcmw751.eamcs.ericsson.se with Microsoft SMTPSVC(6.0.3790.1830); Fri, 23 Nov 2007 12:14:50 -0600 Received: from [147.117.169.183] ([147.117.169.183]) by eusrcmw751.eamcs.ericsson.se with Microsoft SMTPSVC(6.0.3790.1830); Fri, 23 Nov 2007 12:14:49 -0600 Subject: Re: ECN support in a PCN domain (was: Re: [PCN] comments on draft-ietf-pcn-architecture-02) From: Steven Blake To: Magnus Westerlund In-Reply-To: <47455AD6.2030404@ericsson.com> References: <47455AD6.2030404@ericsson.com> Content-Type: text/plain Organization: Ericsson IP Infrastructure Date: Fri, 23 Nov 2007 13:14:49 -0500 Message-Id: <1195841689.4309.9.camel@neutrino> Mime-Version: 1.0 X-Mailer: Evolution 2.8.3 (2.8.3-2.fc6) Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 23 Nov 2007 18:14:49.0983 (UTC) FILETIME=[BCA270F0:01C82DFC] X-Spam-Score: 0.0 (/) X-Scan-Signature: 92df29fa99cf13e554b84c8374345c17 Cc: pcn X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org On Thu, 2007-11-22 at 11:32 +0100, Magnus Westerlund wrote: > Hi, > > See inline > > > Section 3.5: > > > > "The following two assumptions apply if the PCN WG decides to encode > > PCN-marking in the ECN-field. > > > > o It is assumed that PCN-nodes do not perform ECN, [RFC3168], on > > PCN-packets. > > I would like to have a clarification on what is meant with PCN not doing > ECN. To me it seems that in situation where a ECN enabled flow goes > through a PCN domain and the flow are in a situation where a router > actually is congested, it should be able to have that those packets > being marked with CE when leaving the PCN domain. I know this is phase > two of the work if we ever gets there as it discusses another response > mechanism. Sentence one of the PCN charter says: "The Congestion and Pre-Congestion Notification (PCN) working group develops mechanisms to protect the quality-of-service of established inelastic flows within a DiffServ domain when congestion is imminent or existing." Should "inelastic" flows ever have one of the ECT nonces set? The reason this might be an issue, is that PCN is considering reusing the ECN bits. For clarification, the draft text does not apply to elastic traffic that is not being handled by PCN mechanisms (i.e., not part of a PCN-managed behavior aggregate). > > o If a packet that is part of a PCN-flow arrives at a PCN-ingress- > > node with its CE (Congestion experienced) codepoint set, then we > > assume that the PCN-ingress-node drops the packet. After its > > initial Charter is complete, the WG may decide to work on a > > mechanism (such as through a signalling extension) that enables > > ECN-marking to be carried transparently across the PCN-domain." > > This one really surprise me. How can you make this assumption? In case a > ECN enabled flow with some packets CE marked arrive to the ingress of > the PCN domain and are within the limits for which the flow has been > admitted into the PCN domain, then the PCN domain has no right to > summarily drop that packet as being less important than any of the > unmarked packets. Only in cases if congestion actually is experience > within the PCN domain would a differential treatment of CE marked > packets being warranted. > > Can someone please explain how the above assumptions still satisfy the > following sentence in the charter? > > "All PCN mechanisms, including transport and encoding > of (pre-congestion) information, are required to cleanly integrate > with existing architectures and protocols such as DiffServ and ECN." I think that PCN has been defined in the charter to not apply for traffic which might use an ECN-capable transport protocol. Regards, =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Steven Blake Ericsson/Redback Networks +1 919-472-9913 _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Fri Nov 23 16:48:10 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvgNe-0006Tf-Dz; Fri, 23 Nov 2007 16:48:10 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IvgNd-0006T4-0t for pcn-confirm+ok@megatron.ietf.org; Fri, 23 Nov 2007 16:48:09 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IvgNb-0006QJ-Hr for pcn@ietf.org; Fri, 23 Nov 2007 16:48:08 -0500 Received: from zrtps0kn.nortel.com ([47.140.192.55]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IvgNX-0000Li-T8 for pcn@ietf.org; Fri, 23 Nov 2007 16:48:07 -0500 Received: from zcarhxm1.corp.nortel.com (zcarhxm1.corp.nortel.com [47.129.230.97]) by zrtps0kn.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id lANLlw001127; Fri, 23 Nov 2007 21:47:58 GMT X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: ECN support in a PCN domain (was: Re: [PCN] commentsondraft-ietf-pcn-architecture-02) Date: Fri, 23 Nov 2007 16:47:44 -0500 Message-ID: <9671A92C3C8B5744BC97F855F7CB6465134D396E@zcarhxm1.corp.nortel.com> In-Reply-To: <75A199C5D243C741BF3D3F1EBCEF9BA503B34398@E03MVZ1-UKDY.domain1.systemhost.net> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: ECN support in a PCN domain (was: Re: [PCN] commentsondraft-ietf-pcn-architecture-02) Thread-Index: Acgs8zzt10S53v6qR8eaNn6LHzZkfQABDhEwAAC2KKAAR2ew0A== References: <1B6169C658325341A3B8066E23919E1C4C14EB@S4DE8PSAANK.mitte.t-com.de> <75A199C5D243C741BF3D3F1EBCEF9BA503B34398@E03MVZ1-UKDY.domain1.systemhost.net> From: "Jozef Babiarz" To: , , , X-Spam-Score: -4.0 (----) X-Scan-Signature: b132cb3ed2d4be2017585bf6859e1ede Cc: X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Phil, Ruediger and Magnus,=20 For Section 3.5, I would like to propose that it be replaced with the statement below. If the WG choose to use ECN flied for convey PCN information through the network. The PCN WG will request that IANA assign a DSCP value for PCN traffic class consistent with RFC3246 and with PCN marking behavior defined by this workgroup. The Diffserv code point should be referred to as PCN. Explanation: The PCN mechanism will use a new standardized DSCP and will reuse bits 14 & 15 (the ECN-field) of the IPv4 header for PCN therefore no impact to RFC3168, meaning that PCN and ECN can co-exist in the same network. Could discuss other options but they get really messy.=20 Regards, Joe email:babiarz@nortel.com Telephone:613-763-6098 -----Original Message----- From: philip.eardley@bt.com [mailto:philip.eardley@bt.com]=20 Sent: November 22, 2007 6:34 AM To: Ruediger.Geib@t-systems.com; magnus.westerlund@ericsson.com; Babiarz, Jozef (CAR:0S03) Subject: RE: ECN support in a PCN domain (was: Re: [PCN] commentsondraft-ietf-pcn-architecture-02) Joe, will leave this one to you as you thought about it more than I did! Thanks phil > -----Original Message----- > From: Geib, Ruediger [mailto:Ruediger.Geib@t-systems.com] > Sent: 22 November 2007 11:14 > To: magnus.westerlund@ericsson.com > Cc: pcn@ietf.org > Subject: RE: ECN support in a PCN domain (was: Re: [PCN] commentsondraft- > ietf-pcn-architecture-02) >=20 > Magnus, >=20 > I agree to your request, I've asked Phil for a statement on the behaviour > of an ingress node receiving a PCN coloured packet in the architecture > draft in my latest mail. >=20 > I thougth about ignore and remark DSCP and ECN. Drop is a tough option. > Forward with DSCP Best Effort and unchanged ECM may be another one (if > the DSCP is unknown on the ingress interface). >=20 > There are more possibilities, depending on the PCN marking mechanism and > whether the ECN codepoints are used. >=20 > Regards, >=20 > Rudiger >=20 > |> > |> o If a packet that is part of a PCN-flow arrives at a PCN-ingress- > |> node with its CE (Congestion experienced) codepoint set, then we > |> assume that the PCN-ingress-node drops the packet. After its > |> initial Charter is complete, the WG may decide to work on a > |> mechanism (such as through a signalling extension) that enables > |> ECN-marking to be carried transparently across the PCN-domain." > | > |This one really surprise me. How can you make this assumption? In case a > |ECN enabled flow with some packets CE marked arrive to the ingress of > |the PCN domain and are within the limits for which the flow has been > |admitted into the PCN domain, then the PCN domain has no right to > |summarily drop that packet as being less important than any of the > |unmarked packets. Only in cases if congestion actually is experience > |within the PCN domain would a differential treatment of CE marked > |packets being warranted. > | > |Can someone please explain how the above assumptions still satisfy the > |following sentence in the charter? > | > |"All PCN mechanisms, including transport and encoding > |of (pre-congestion) information, are required to cleanly integrate > |with existing architectures and protocols such as DiffServ and ECN." > | > | > |Cheers > | > |Magnus Westerlund > | > |IETF Transport Area Director & TSVWG Chair > |---------------------------------------------------------------------- > |Multimedia Technologies, Ericsson Research EAB/TVM/M > |---------------------------------------------------------------------- > |Ericsson AB | Phone +46 8 4048287 > |Torshamsgatan 23 | Fax +46 8 7575550 > |S-164 80 Stockholm, Sweden | mailto: magnus.westerlund@ericsson.com > |---------------------------------------------------------------------- > | > | > |_______________________________________________ > |PCN mailing list > |PCN@ietf.org > |https://www1.ietf.org/mailman/listinfo/pcn > | >=20 >=20 > _______________________________________________ > PCN mailing list > PCN@ietf.org > https://www1.ietf.org/mailman/listinfo/pcn _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Fri Nov 23 23:31:21 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Ivmfl-0001RK-Dp; Fri, 23 Nov 2007 23:31:17 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Ivmfj-0001R2-Je for pcn-confirm+ok@megatron.ietf.org; Fri, 23 Nov 2007 23:31:15 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Ivmfj-0001Pl-7V for pcn@ietf.org; Fri, 23 Nov 2007 23:31:15 -0500 Received: from ihemail2.lucent.com ([135.245.0.35]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1Ivmfg-0005CG-8d for pcn@ietf.org; Fri, 23 Nov 2007 23:31:12 -0500 Received: from ilexp02.ndc.lucent.com (h135-3-39-2.lucent.com [135.3.39.2]) by ihemail2.lucent.com (8.13.8/IER-o) with ESMTP id lAO4Uq6e011229; Fri, 23 Nov 2007 22:30:54 -0600 (CST) Received: from ILEXC1U02.ndc.lucent.com ([135.3.39.4]) by ilexp02.ndc.lucent.com with Microsoft SMTPSVC(6.0.3790.1830); Fri, 23 Nov 2007 22:30:52 -0600 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Subject: RE: [PCN] IETF 70 *DRAFT* PCN agenda Date: Fri, 23 Nov 2007 22:30:53 -0600 Message-ID: <7D096A439A3D0D48A3D10872022CFD0901D572C5@ILEXC1U02.ndc.lucent.com> In-Reply-To: <1195611870.28291.72.camel@neutrino> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] IETF 70 *DRAFT* PCN agenda Thread-Index: Acgr5a5yBZPfBeqVR2OBtvmbfvk/5gCawYMA References: <1195611870.28291.72.camel@neutrino> From: "GOLDMAN, STUART O \(STUART\)" To: "Steven Blake" , "pcn" X-OriginalArrivalTime: 24 Nov 2007 04:30:52.0100 (UTC) FILETIME=[CBC5E840:01C82E52] X-Scanned-By: MIMEDefang 2.57 on 135.245.2.35 X-Spam-Score: 0.0 (/) X-Scan-Signature: f4c2cf0bccc868e4cc88dace71fb3f44 Cc: "Suraci, Frank J" , Robert.schafer@verizon.com, "Schaefer, Robert" X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1896600734==" Errors-To: pcn-bounces@ietf.org --===============1896600734== Content-class: urn:content-classes:message Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 U2luY2UgdGhlIGZvbGxvd2luZyB0b3BpYyBpcyBub3Qgb24gdGhlIFBDTiBhZ2VuZGEgYXMgZGlz Y3Vzc2VkIGJlbG93LCBwZW9wbGUgd2l0aCBhbiBpbnRlcmVzdCBpbiBQQ04gd29ya2luZyBhY3Jv c3MgYW4gaW50ZXItZG9tYWluIGJvdW5kYXJ5IHdpdGggdGhlIFBUU0MgbWlnaHQgd2lzaCB0byBj b25zaWRlcjoNCg0KZHJhZnQtZ29sZG1hbi1wY24tcHN0bi1zY29wZS0wMSAyMDA3LTExLTA1IEFj dGl2ZSAgICBJLUQgRXhpc3RzICANCg0KaHR0cHM6Ly9kYXRhdHJhY2tlci5pZXRmLm9yZy9kcmFm dHMvZHJhZnQtZ29sZG1hbi1wY24tcHN0bi1zY29wZS8NCg0KU3R1YXJ0IEdvbGRtYW4NCkFsY2F0 ZWwtTHVjZW50IA0Kc2dvbGRtYW5AYWxjYXRlbC1sdWNlbnQuY29tDQorMSA2MDIgNDkzIDg0MzgN Cu+BkCBwbGVhc2Ugc2F2ZSBhIHRyZWUgYnkgbm90IHByaW50aW5nIHRoaXMgZS1tYWlsLg0KDQoN Cg0KDQotLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KRnJvbTogU3RldmVuIEJsYWtlIFttYWls dG86c3RldmVuLmJsYWtlQGVyaWNzc29uLmNvbV0gDQpTZW50OiBUdWVzZGF5LCBOb3ZlbWJlciAy MCwgMjAwNyA3OjI1IFBNDQpUbzogcGNuDQpTdWJqZWN0OiBbUENOXSBJRVRGIDcwICpEUkFGVCog UENOIGFnZW5kYQ0KDQpBdHRhY2hlZCBpcyB0aGUgZHJhZnQgYWdlbmRhIGZvciB0aGUgVmFuY291 dmVyIG1lZXRpbmcsDQphbHNvIGF2YWlsYWJsZSBhdDoNCg0KaHR0cDovL3d3dzMuaWV0Zi5vcmcv cHJvY2VlZGluZ3MvMDdkZWMvYWdlbmRhL3Bjbi50eHQNCg0KTm90ZSB0aGF0IFNjb3R0IGFuZCBJ IGFyZSBwcm9wb3NpbmcgdG8gZG8gc29tZXRoaW5nIGRpZmZlcmVudCBmcm9tIHRoZQ0KcHJldmlv dXMgbWVldGluZ3M6IHdlIGFyZSBvbmx5IHBsYW5uaW5nIHRvIGRpc2N1c3MgdGhlIFBDTiBhcmNo aXRlY3R1cmUNCmRyYWZ0IGFuZCBLd29rJ3MgYW5kIEdlb3JnaW9zJyBlbmNvZGluZyBkcmFmdC4N Cg0KVGhlcmUgYXJlIHNldmVyYWwgcmVhc29ucyBmb3IgdGhpcywgYnV0IHRvIHN1bW1hcml6ZToN Cg0KLSBUaGVzZSBkcmFmdHMgYWRkcmVzcyB0aGUgZmlyc3QgdHdvIG1pbGVzdG9uZXMgZm9yIFBD Ti4gIEFjY29yZGluZyB0bw0KICB0aGUgd29ya2luZyBncm91cCBjaGFydGVyLCB3ZSBhcmUgYWxy ZWFkeSBsYXRlIG9uIHRoZXNlIG1pbGVzdG9uZXMNCiAgKGFuZCB3aGVuIHdvcmtpbmcgZ3JvdXBz IGFyZSBsYXRlLCB0aGUgY2hhaXJzIGRvbid0IGdldCB0aGVpciBib251cw0KICBjaGVja3MpLiAg U28gaXQgaXMgY3JpdGljYWwgdGhhdCB3ZSBtYWtlIHByb2dyZXNzIG9uIHJlc29sdmluZyB0aGUN CiAgb3BlbiBpc3N1ZXMgd2l0aCB0aGVzZSBkb2N1bWVudHMgc28gdGhhdCB3ZSBjYW4gbW92ZSB0 aGVtIHRvIHdvcmtpbmcNCiAgZ3JvdXAgbGFzdCBjYWxsICgxKS4NCg0KLSBTb21lIGRlc2lnbiBk ZWNpc2lvbnMgZm9yIGZ1dHVyZSBtaWxlc3RvbmVzIGFyZSBkZXBlbmRpbmcgb24gdGhlDQogIHJl c29sdXRpb24gb2Ygc29tZSBvcGVuIGlzc3VlcyB3aXRoIHRoZXNlIGRvY3VtZW50cywgc28gYnkg bWFraW5nDQogIHByb2dyZXNzIG5vdyB3ZSBzaG91bGQgYmUgYWJsZSB0byBhY2NlbGVyYXRlIHN1 YnNlcXVlbnQgd29yay4NCg0KLSBXZSBoYWQgcmVxdWVzdHMgdG8gZGlzY3VzcyBuaW5lIGRyYWZ0 cyBpbiB0aGlzIG1lZXRpbmcsIHNvbWUgb2Ygd2hpY2gNCiAganVzdCByZWNlbnRseSBwb3BwZWQg aW50byBleGlzdGVuY2Ugd2l0aCBubyBwcmlvciBkaXNjdXNzaW9uIG9uIHRoZQ0KICBtYWlsaW5n IGxpc3QuICBXaXRoIG9ubHkgMTIwIG1pbnV0ZXMgb2YgbWVldGluZyB0aW1lLCB0aGVyZSBpcyBu b3QNCiAgZW5vdWdoIHRpbWUgdG8gZmFjaWxpdGF0ZSBnb29kIGRpc2N1c3Npb25zIGZvciBhbGwg b2YgdGhlc2UgZHJhZnRzLiANCiAgU28gd2UgcGxhbiB0byBzcGVuZCBvdXIgcHJlY2lvdXMgZmFj ZSB0aW1lIG9uIGRpc2N1c3Npb24gb2YgdGhlIG9wZW4NCiAgaXNzdWVzIHRoYXQgd2Uga25vdyB3 ZSBuZWVkIHRvIHJlc29sdmUgYXMgc29vbiBhcyBwb3NzaWJsZS4NCg0KUGhpbGlwIGFuZCBLd29r IGhhdmUgdm9sdW50ZWVyZWQgdG8gZWFjaCBwb3N0IGEgbGlzdCBvZiBvcGVuIGlzc3VlcyB3aXRo DQp0aGVpciBkcmFmdHMuICBXZSBzaG91bGQgcmV2aWV3IGFuZCByZWZpbmUgdGhlc2UgbGlzdHMg cHJpb3IgdG8gdGhlDQptZWV0aW5nLCBhbmQgb2YgY291cnNlIGNvbnRpbnVlIHRvIGRpc2N1c3Mg dGhlIG9wZW4gaXNzdWVzLiAgSW4gdGhlDQptZWV0aW5nIHdlIHdpbGwgbGV0IFBoaWxpcCBhbmQg S3dvayBkcml2ZSB0aGUgZGlzY3Vzc2lvbiwgYnV0IGFueW9uZQ0Kd2l0aCBhbnl0aGluZyB0byBz YXkgb24gdGhlc2UgaXNzdWVzIHdpbGwgYmUgd2VsY29tZSB0byBxdWV1ZSB1cCBmb3IgbWljDQph bmQgc2NyZWVuIHRpbWUgKGlmIHlvdSBkbyBwcm9kdWNlIHNsaWRlcyAoMyBtYXgsIHBsZWFzZSks IHRoZW4gcGxlYXNlDQpzZW5kIHRoZW0gdG8gdGhlIGxpc3QgYmVmb3JlIHRoZSBtZWV0aW5nKS4N Cg0KQXMgYWx3YXlzLCBhbGwgd29ya2luZyBncm91cCBkZWNpc2lvbnMgYXJlIGZpbmFsaXplZCBv biB0aGUgbWFpbGluZw0KbGlzdC4NCg0KUGxlYXNlIHNlbmQgY29tbWVudHMgb24gdGhpcyBkcmFm dCBhZ2VuZGEgdG8gdGhlIGxpc3QgQVNBUC4NCg0KDQooMSkgZHJhZnQtY2hhbi1wY24tZW5jb2Rp bmctY29tcGFyaXNvbi0wMSBpcyBub3QgYSB3b3JraW5nIGdyb3VwIA0KICAgIGRvY3VtZW50LCBz byBpdCBoYXMgbm8gb2ZmaWNpYWwgc3RhdHVzLiAgSW4gdGhlIGFic2VuY2Ugb2YgYW55IA0KICAg IGFsdGVybmF0aXZlIGRyYWZ0cywgd2UgbmVlZCB0byBtYWtlIHByb2dyZXNzIG9uIHRoaXMgb25l Lg0KDQooMikgSWYgYW55IGF1dGhvciBmZWVscyB0aGF0IGNvbXByZWhlbnNpb24gb2YgaGlzIG9y IGhlcg0KICAgIGRyYWZ0IHdvdWxkIGJlbmVmaXQgZnJvbSBzbGlkZXMgYW5kL29yIG9yYWwgY29t bWVudGFyeSwgZmVlbCBmcmVlDQogICAgdG8gcG9zdCB0aG9zZSB0byB0aGUgbGlzdCAoYW5kIHdl IGNhbiBhcnJhbmdlIGhvc3RpbmcgZm9yIHRoZXNlIGlmDQogICAgbmVjZXNzYXJ5KS4NCg0KDQpT ZWUgeW91IGluIFZhbmNvdXZlciENCg0KDQpTY290dCAmIFN0ZXZlDQoNCj0tPS09LT0tPS09LT0t PS09LT0tPS09LT0tPS09LT0tPS09LT0tPS09LT0tPS09LT0tPS09LT0NClN0ZXZlbiBCbGFrZSAg ICAgICAgICAgICAgICA8c3RldmVuLmJsYWtlQGVyaWNzc29uLmNvbT4NCkVyaWNzc29uL1JlZGJh Y2sgTmV0d29ya3MgICAgICAgICAgICAgICArMSA5MTktNDcyLTk5MTMNCg== --===============1896600734== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn --===============1896600734==-- From pcn-bounces@ietf.org Sat Nov 24 06:19:11 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Ivt2T-0006W0-0G; Sat, 24 Nov 2007 06:19:09 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Ivt2R-0006V6-SS for pcn-confirm+ok@megatron.ietf.org; Sat, 24 Nov 2007 06:19:07 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Ivt2R-0006Us-Ib; Sat, 24 Nov 2007 06:19:07 -0500 Received: from [202.99.23.227] (helo=people.com.cn) by ietf-mx.ietf.org with smtp (Exim 4.43) id 1Ivt2N-0006GZ-I4; Sat, 24 Nov 2007 06:19:07 -0500 Received: from people.com.cn([127.0.0.1]) by people.com.cn(AIMC 2.9.5.8) with SMTP id jm13b474813bc; Sat, 24 Nov 2007 19:31:59 +0800 Received: from megatron.ietf.org([156.154.16.145]) by people.com.cn(AIMC 2.9.5.8) with SMTP id jm684741f08e; Mon, 19 Nov 2007 23:55:23 +0800 Received: from megatron.ietf.org([156.154.16.145]) by people.com.cn(AIMC 2.9.5.8) with SMTP id AISP action; Mon, 19 Nov 2007 23:55:23 +0800 Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iu8Pu-0005dp-2R; Mon, 19 Nov 2007 10:20:06 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iu8Pq-0005Yr-So; Mon, 19 Nov 2007 10:20:02 -0500 Received: from ns4.neustar.com ([156.154.24.139]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1Iu8Pq-0007Bc-HQ; Mon, 19 Nov 2007 10:20:02 -0500 Received: from stiedprstage1.ietf.org (stiedprstage1.va.neustar.com [10.31.47.10]) by ns4.neustar.com (Postfix) with ESMTP id 6910F2AC5D; Mon, 19 Nov 2007 15:20:02 +0000 (GMT) Received: from ietf by stiedprstage1.ietf.org with local (Exim 4.43) id 1Iu8Pq-0002pR-4n; Mon, 19 Nov 2007 10:20:02 -0500 Content-Type: Multipart/Mixed; Boundary="NextPart" Mime-Version: 1.0 To: i-d-announce@ietf.org From: Internet-Drafts@ietf.org Message-Id: Date: Mon, 19 Nov 2007 10:20:02 -0500 X-Spam-Score: 0.0 (/) X-Scan-Signature: 31247fb3be228bb596db9127becad0bc X-BeenThere: i-d-announce@ietf.org X-Mailman-Version: 2.1.5 Precedence: list X-AIMC-AUTH: (null) X-AIMC-MAILFROM: i-d-announce-bounces@ietf.org X-AIMC-AUTH: (null) X-AIMC-MAILFROM: Internet-Drafts@ietf.org X-Auto-Forward: jaglee@people.com.cn jag@kw.com.cn X-Spam-Score: 2.8 (++) X-Scan-Signature: b5d20af10c334b36874c0264b10f59f1 Cc: pcn@ietf.org Subject: [PCN] I-D Action:draft-ietf-pcn-architecture-02.txt X-BeenThere: pcn@ietf.org Reply-To: internet-drafts@ietf.org List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org --NextPart A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Congestion and Pre-Congestion Notification Working Group of the IETF. Title : Pre-Congestion Notification Architecture Author(s) : P. Eardley Filename : draft-ietf-pcn-architecture-02.txt Pages : 45 Date : 2007-11-19 The purpose of this document is to describe a general architecture for flow admission and termination based on aggregated pre-congestion information in order to protect the quality of service of established inelastic flows within a single DiffServ domain.Status A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-pcn-architecture-02.txt To remove yourself from the I-D Announcement list, send a message to i-d-announce-request@ietf.org with the word unsubscribe in the body of the message. You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce to change your subscription settings. Internet-Drafts are also available by anonymous FTP. Login with the username "anonymous" and a password of your e-mail address. After logging in, type "cd internet-drafts" and then "get draft-ietf-pcn-architecture-02.txt". A list of Internet-Drafts directories can be found in http://www.ietf.org/shadow.html or ftp://ftp.ietf.org/ietf/1shadow-sites.txt Internet-Drafts can also be obtained by e-mail. Send a message to: mailserv@ietf.org. In the body type: "FILE /internet-drafts/draft-ietf-pcn-architecture-02.txt". NOTE: The mail server at ietf.org can return the document in MIME-encoded form by using the "mpack" utility. To use this feature, insert the command "ENCODING mime" before the "FILE" command. To decode the response(s), you will need "munpack" or a MIME-compliant mail reader. Different MIME-compliant mail readers exhibit different behavior, especially when dealing with "multipart" MIME messages (i.e. documents which have been split up into multiple messages), so check your local documentation on how to manipulate these messages. Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. --NextPart Content-Type: Multipart/Alternative; Boundary="OtherAccess" --OtherAccess Content-Type: Message/External-body; access-type="mail-server"; server="mailserv@ietf.org" Content-Type: text/plain Content-ID: <2007-11-19101818.I-D@ietf.org> ENCODING mime FILE /internet-drafts/draft-ietf-pcn-architecture-02.txt --OtherAccess Content-Type: Message/External-body; name="draft-ietf-pcn-architecture-02.txt"; site="ftp.ietf.org"; access-type="anon-ftp"; directory="internet-drafts" Content-Type: text/plain Content-ID: <2007-11-19101818.I-D\@ietf.org> --OtherAccess-- --NextPart Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ I-D-Announce mailing list I-D-Announce@ietf.org https://www1.ietf.org/mailman/listinfo/i-d-announce --NextPart Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn --NextPart-- From pcn-bounces@ietf.org Mon Nov 26 03:04:32 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IwYxC-0005qh-A1; Mon, 26 Nov 2007 03:04:30 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IwYxA-0005qV-PE for pcn-confirm+ok@megatron.ietf.org; Mon, 26 Nov 2007 03:04:29 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IwYx5-0005oA-Dc for pcn@ietf.org; Mon, 26 Nov 2007 03:04:23 -0500 Received: from tcmail31.telekom.de ([217.6.95.238]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IwYx4-000168-Hx for pcn@ietf.org; Mon, 26 Nov 2007 03:04:23 -0500 Received: from S4DE8PSAANQ.mitte.t-com.de by tcmail31.telekom.de with ESMTP; Mon, 26 Nov 2007 09:04:21 +0100 Received: from S4DE8PSAANK.mitte.t-com.de ([10.151.229.10]) by S4DE8PSAANQ.mitte.t-com.de with Microsoft SMTPSVC(6.0.3790.3959); Mon, 26 Nov 2007 09:04:20 +0100 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: quoted-printable Subject: RE: ECN support in a PCN domain (was: Re: [PCN] commentsondraft-ietf-pcn-architecture-02) Date: Mon, 26 Nov 2007 09:04:20 +0100 Message-Id: <1B6169C658325341A3B8066E23919E1C4C1511@S4DE8PSAANK.mitte.t-com.de> In-Reply-To: <9671A92C3C8B5744BC97F855F7CB6465134D396E@zcarhxm1.corp.nortel.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: ECN support in a PCN domain (was: Re: [PCN] commentsondraft-ietf-pcn-architecture-02) Thread-Index: Acgs8zzt10S53v6qR8eaNn6LHzZkfQABDhEwAAC2KKAAR2ew0AB6PO5w From: "Geib, Ruediger" To: X-OriginalArrivalTime: 26 Nov 2007 08:04:20.0994 (UTC) FILETIME=[F34B8620:01C83002] X-Spam-Score: 0.0 (/) X-Scan-Signature: 20f22c03b5c66958bff5ef54fcda6e48 Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Joe, you and Magnus consider fundamental issues. I'm more interested in daily business and there specifically in "unexpected events": the PCN ingress node receives a packet on a non PCN interface and this packet carries a DSCP which the ingress node will map to the PCN DSCP. Further, this packet has some ECN bits set (which is the unexpected event). The architecture document (or any other document indicated by the architecture doc) should recommend an action. And this action should allow for later extensions - congestion feedback given to the end system, especially. Assume the flow to be admitted. My personal preference is to ignore the ECN bits and overwrite them. Dropping seems to tough. And keeping the markings may result in wrong congestion status (may be only depending on the specific ECN codepoint). The same holds, if DSCPs are used for pre-congestion marking and a packet arrives the ingress with a pre-congestion DSCP. We need to define an action of the ingress node. Regards, Rudiger |-----Original Message----- |From: Jozef Babiarz [mailto:babiarz@nortel.com] |Sent: Friday, November 23, 2007 10:48 PM |To: philip.eardley@bt.com; Geib, Rudiger; |magnus.westerlund@ericsson.com; pcn@ietf.org |Subject: RE: ECN support in a PCN domain (was: Re: [PCN] |commentsondraft-ietf-pcn-architecture-02) | | |Phil, Ruediger and Magnus,=20 | |For Section 3.5, I would like to propose that it be replaced with the |statement below. | |If the WG choose to use ECN flied for convey PCN information=20 |through the |network. The PCN WG will request that IANA assign a DSCP value for PCN |traffic class consistent with RFC3246 and with PCN marking behavior |defined by this workgroup. The Diffserv code point should be=20 |referred to |as PCN. | |Explanation: |The PCN mechanism will use a new standardized DSCP and will reuse bits |14 & 15 (the ECN-field) of the IPv4 header for PCN therefore no impact |to RFC3168, meaning that PCN and ECN can co-exist in the same network. | |Could discuss other options but they get really messy.=20 | |Regards, Joe |email:babiarz@nortel.com |Telephone:613-763-6098 |-----Original Message----- |From: philip.eardley@bt.com [mailto:philip.eardley@bt.com]=20 |Sent: November 22, 2007 6:34 AM |To: Ruediger.Geib@t-systems.com; magnus.westerlund@ericsson.com; |Babiarz, Jozef (CAR:0S03) |Subject: RE: ECN support in a PCN domain (was: Re: [PCN] |commentsondraft-ietf-pcn-architecture-02) | |Joe, will leave this one to you as you thought about it more=20 |than I did! |Thanks |phil | |> -----Original Message----- |> From: Geib, Ruediger [mailto:Ruediger.Geib@t-systems.com] |> Sent: 22 November 2007 11:14 |> To: magnus.westerlund@ericsson.com |> Cc: pcn@ietf.org |> Subject: RE: ECN support in a PCN domain (was: Re: [PCN] |commentsondraft- |> ietf-pcn-architecture-02) |>=20 |> Magnus, |>=20 |> I agree to your request, I've asked Phil for a statement on the |behaviour |> of an ingress node receiving a PCN coloured packet in the=20 |architecture |> draft in my latest mail. |>=20 |> I thougth about ignore and remark DSCP and ECN. Drop is a tough |option. |> Forward with DSCP Best Effort and unchanged ECM may be=20 |another one (if |> the DSCP is unknown on the ingress interface). |>=20 |> There are more possibilities, depending on the PCN marking mechanism |and |> whether the ECN codepoints are used. |>=20 |> Regards, |>=20 |> Rudiger |>=20 |> |> |> |> o If a packet that is part of a PCN-flow arrives at a |PCN-ingress- |> |> node with its CE (Congestion experienced) codepoint=20 |set, then |we |> |> assume that the PCN-ingress-node drops the packet. =20 |After its |> |> initial Charter is complete, the WG may decide to work on a |> |> mechanism (such as through a signalling extension) that |enables |> |> ECN-marking to be carried transparently across the |PCN-domain." |> | |> |This one really surprise me. How can you make this assumption? In |case a |> |ECN enabled flow with some packets CE marked arrive to the=20 |ingress of |> |the PCN domain and are within the limits for which the flow has been |> |admitted into the PCN domain, then the PCN domain has no right to |> |summarily drop that packet as being less important than any of the |> |unmarked packets. Only in cases if congestion actually is experience |> |within the PCN domain would a differential treatment of CE marked |> |packets being warranted. |> | |> |Can someone please explain how the above assumptions still satisfy |the |> |following sentence in the charter? |> | |> |"All PCN mechanisms, including transport and encoding |> |of (pre-congestion) information, are required to cleanly integrate |> |with existing architectures and protocols such as DiffServ and ECN." |> | |> | |> |Cheers |> | |> |Magnus Westerlund |> | |> |IETF Transport Area Director & TSVWG Chair |> ||---------------------------------------------------------------------- |> |Multimedia Technologies, Ericsson Research EAB/TVM/M |> ||---------------------------------------------------------------------- |> |Ericsson AB | Phone +46 8 4048287 |> |Torshamsgatan 23 | Fax +46 8 7575550 |> |S-164 80 Stockholm, Sweden | mailto: magnus.westerlund@ericsson.com |> ||---------------------------------------------------------------------- |> | |> | |> |_______________________________________________ |> |PCN mailing list |> |PCN@ietf.org |> |https://www1.ietf.org/mailman/listinfo/pcn |> | |>=20 |>=20 |> _______________________________________________ |> PCN mailing list |> PCN@ietf.org |> https://www1.ietf.org/mailman/listinfo/pcn | _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Mon Nov 26 08:18:54 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IwdrR-0003jV-PO; Mon, 26 Nov 2007 08:18:53 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IwdrQ-0003jP-Fd for pcn-confirm+ok@megatron.ietf.org; Mon, 26 Nov 2007 08:18:52 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IwdrQ-0003jH-3t for pcn@ietf.org; Mon, 26 Nov 2007 08:18:52 -0500 Received: from smtp2.smtp.bt.com ([217.32.164.150]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IwdrJ-0005Zp-B4 for pcn@ietf.org; Mon, 26 Nov 2007 08:18:52 -0500 Received: from E03MVZ1-UKDY.domain1.systemhost.net ([193.113.30.65]) by smtp2.smtp.bt.com with Microsoft SMTPSVC(6.0.3790.1830); Mon, 26 Nov 2007 13:18:44 +0000 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: ECN support in a PCN domain (was: Re: [PCN] commentsondraft-ietf-pcn-architecture-02) Date: Mon, 26 Nov 2007 13:18:43 -0000 Message-ID: <75A199C5D243C741BF3D3F1EBCEF9BA503B343B5@E03MVZ1-UKDY.domain1.systemhost.net> In-Reply-To: <9671A92C3C8B5744BC97F855F7CB6465134D396E@zcarhxm1.corp.nortel.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: ECN support in a PCN domain (was: Re: [PCN] commentsondraft-ietf-pcn-architecture-02) Thread-Index: Acgs8zzt10S53v6qR8eaNn6LHzZkfQABDhEwAAC2KKAAR2ew0ACFlHXg From: To: , , , X-OriginalArrivalTime: 26 Nov 2007 13:18:44.0513 (UTC) FILETIME=[DED41910:01C8302E] X-Spam-Score: -1.0 (-) X-Scan-Signature: 3971661e40967acfc35f708dd5f33760 Cc: X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Joe, whilst I personally agree that your suggestion is sensible, was trying to keep the architecture draft independent of decisions about encoding which are taken later in the WGs life.=20 Rudiger Yes, there are several possibilities. The one in the draft seemed the safest (given that end point reaction is same to dropped pkt as CE). phil > -----Original Message----- > From: Jozef Babiarz [mailto:babiarz@nortel.com] > Sent: 23 November 2007 21:48 > To: Eardley,PL,Philip,CXR9 R; Ruediger.Geib@t-systems.com; > magnus.westerlund@ericsson.com; pcn@ietf.org > Subject: RE: ECN support in a PCN domain (was: Re: [PCN] commentsondraft- > ietf-pcn-architecture-02) >=20 > Phil, Ruediger and Magnus, >=20 > For Section 3.5, I would like to propose that it be replaced with the > statement below. >=20 > If the WG choose to use ECN flied for convey PCN information through the > network. The PCN WG will request that IANA assign a DSCP value for PCN > traffic class consistent with RFC3246 and with PCN marking behavior > defined by this workgroup. The Diffserv code point should be referred to > as PCN. >=20 > Explanation: > The PCN mechanism will use a new standardized DSCP and will reuse bits > 14 & 15 (the ECN-field) of the IPv4 header for PCN therefore no impact > to RFC3168, meaning that PCN and ECN can co-exist in the same network. >=20 > Could discuss other options but they get really messy. >=20 > Regards, Joe > email:babiarz@nortel.com > Telephone:613-763-6098 > -----Original Message----- > From: philip.eardley@bt.com [mailto:philip.eardley@bt.com] > Sent: November 22, 2007 6:34 AM > To: Ruediger.Geib@t-systems.com; magnus.westerlund@ericsson.com; > Babiarz, Jozef (CAR:0S03) > Subject: RE: ECN support in a PCN domain (was: Re: [PCN] > commentsondraft-ietf-pcn-architecture-02) >=20 > Joe, will leave this one to you as you thought about it more than I did! > Thanks > phil >=20 > > -----Original Message----- > > From: Geib, Ruediger [mailto:Ruediger.Geib@t-systems.com] > > Sent: 22 November 2007 11:14 > > To: magnus.westerlund@ericsson.com > > Cc: pcn@ietf.org > > Subject: RE: ECN support in a PCN domain (was: Re: [PCN] > commentsondraft- > > ietf-pcn-architecture-02) > > > > Magnus, > > > > I agree to your request, I've asked Phil for a statement on the > behaviour > > of an ingress node receiving a PCN coloured packet in the architecture > > draft in my latest mail. > > > > I thougth about ignore and remark DSCP and ECN. Drop is a tough > option. > > Forward with DSCP Best Effort and unchanged ECM may be another one (if > > the DSCP is unknown on the ingress interface). > > > > There are more possibilities, depending on the PCN marking mechanism > and > > whether the ECN codepoints are used. > > > > Regards, > > > > Rudiger > > > > |> > > |> o If a packet that is part of a PCN-flow arrives at a > PCN-ingress- > > |> node with its CE (Congestion experienced) codepoint set, then > we > > |> assume that the PCN-ingress-node drops the packet. After its > > |> initial Charter is complete, the WG may decide to work on a > > |> mechanism (such as through a signalling extension) that > enables > > |> ECN-marking to be carried transparently across the > PCN-domain." > > | > > |This one really surprise me. How can you make this assumption? In > case a > > |ECN enabled flow with some packets CE marked arrive to the ingress of > > |the PCN domain and are within the limits for which the flow has been > > |admitted into the PCN domain, then the PCN domain has no right to > > |summarily drop that packet as being less important than any of the > > |unmarked packets. Only in cases if congestion actually is experience > > |within the PCN domain would a differential treatment of CE marked > > |packets being warranted. > > | > > |Can someone please explain how the above assumptions still satisfy > the > > |following sentence in the charter? > > | > > |"All PCN mechanisms, including transport and encoding > > |of (pre-congestion) information, are required to cleanly integrate > > |with existing architectures and protocols such as DiffServ and ECN." > > | > > | > > |Cheers > > | > > |Magnus Westerlund > > | > > |IETF Transport Area Director & TSVWG Chair > > > |---------------------------------------------------------------------- > > |Multimedia Technologies, Ericsson Research EAB/TVM/M > > > |---------------------------------------------------------------------- > > |Ericsson AB | Phone +46 8 4048287 > > |Torshamsgatan 23 | Fax +46 8 7575550 > > |S-164 80 Stockholm, Sweden | mailto: magnus.westerlund@ericsson.com > > > |---------------------------------------------------------------------- > > | > > | > > |_______________________________________________ > > |PCN mailing list > > |PCN@ietf.org > > |https://www1.ietf.org/mailman/listinfo/pcn > > | > > > > > > _______________________________________________ > > PCN mailing list > > PCN@ietf.org > > https://www1.ietf.org/mailman/listinfo/pcn _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Mon Nov 26 13:28:02 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iwigc-0004en-Dr; Mon, 26 Nov 2007 13:28:02 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Iwigb-0004ei-9q for pcn-confirm+ok@megatron.ietf.org; Mon, 26 Nov 2007 13:28:01 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iwiga-0004ea-TY for pcn@ietf.org; Mon, 26 Nov 2007 13:28:00 -0500 Received: from wrzx28.rz.uni-wuerzburg.de ([132.187.3.28] helo=mailrelay.rz.uni-wuerzburg.de) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1Iwiga-0001RL-CA for pcn@ietf.org; Mon, 26 Nov 2007 13:28:00 -0500 Received: from virusscan.mail (localhost [127.0.0.1]) by mailrelay.mail (Postfix) with ESMTP id 664C4B6BD for ; Mon, 26 Nov 2007 19:27:57 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by virusscan.mail (Postfix) with ESMTP id 59933C630 for ; Mon, 26 Nov 2007 19:27:57 +0100 (CET) Received: from europa.informatik.uni-wuerzburg.de (wicx01.informatik.uni-wuerzburg.de [132.187.11.1]) by mailmaster.uni-wuerzburg.de (Postfix) with ESMTP id 46C34C614 for ; Mon, 26 Nov 2007 19:27:57 +0100 (CET) Received: from nero.informatik.uni-wuerzburg.de (win3005.informatik.uni-wuerzburg.de [132.187.106.5]) by europa.informatik.uni-wuerzburg.de (8.11.3/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id lAQIRvI24063 for ; Mon, 26 Nov 2007 19:27:57 +0100 Received: from [127.0.0.1] (nero.informatik.uni-wuerzburg.de [132.187.106.5]) by nero.informatik.uni-wuerzburg.de (Postfix) with ESMTP id A6B666F591 for ; Mon, 26 Nov 2007 19:20:10 +0100 (CET) Message-ID: <474B109A.8020005@informatik.uni-wuerzburg.de> Date: Mon, 26 Nov 2007 19:29:46 +0100 From: Michael Menth Organization: University of Wuerzburg User-Agent: Thunderbird 2.0.0.9 (Windows/20071031) MIME-Version: 1.0 To: pcn Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at uni-wuerzburg.de X-Spam-Score: 0.0 (/) X-Scan-Signature: 769a46790fb42fbb0b0cc700c82f7081 Subject: [PCN] [Fwd: New Version Notification for draft-menth-pcn-performance-01] X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: menth@informatik.uni-wuerzburg.de List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi, we've got an updated version of our performance draft. A more detailed and better readable version of the results can be found in http://www3.informatik.uni-wuerzburg.de/TR/tr437.pdf and http://www3.informatik.uni-wuerzburg.de/staff/menth/Publications/Menth07-PCN-Eval.pdf Regards, Michael -------- Original Message -------- Subject: New Version Notification for draft-menth-pcn-performance-01 Date: Mon, 19 Nov 2007 07:21:46 -0500 From: IETF I-D Submission Tool To: menth@informatik.uni-wuerzburg.de CC: lehrieder@informatik.uni-wuerzburg.de A new version of I-D, draft-menth-pcn-performance-01.txt has been successfuly submitted by Michael Menth and posted to the IETF repository. Filename: draft-menth-pcn-performance Revision: 01 Title: Performance Evaluation of PCN-Based Algorithms Creation_date: 2007-11-19 WG ID: Independent Submission Number_of_pages: 29 Abstract: This document presents a summary of performance studies for PCN-based admission control and flow termination. The numerical results were obtained by simulation or mathematical analysis. The IETF Secretariat. -- Dr. Michael Menth, Assistant Professor University of Wuerzburg, Institute of Computer Science Am Hubland, D-97074 Wuerzburg, Germany, room B206 phone: (+49)-931/888-6644, fax: (+49)-931/888-6632 mailto:menth@informatik.uni-wuerzburg.de http://www3.informatik.uni-wuerzburg.de/research/ngn _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Tue Nov 27 17:19:58 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Ix8mb-0004wG-Sa; Tue, 27 Nov 2007 17:19:57 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Ix8mb-0004w6-0O for pcn-confirm+ok@megatron.ietf.org; Tue, 27 Nov 2007 17:19:57 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Ix8mV-0004pp-Js for pcn@ietf.org; Tue, 27 Nov 2007 17:19:51 -0500 Received: from zcars04f.nortel.com ([47.129.242.57]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1Ix8mU-0001Bv-P2 for pcn@ietf.org; Tue, 27 Nov 2007 17:19:51 -0500 Received: from zcarhxm1.corp.nortel.com (zcarhxm1.corp.nortel.com [47.129.230.97]) by zcars04f.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id lARMJfR26741; Tue, 27 Nov 2007 22:19:41 GMT X-MIMEOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: ECN support in a PCN domain (was: Re: [PCN] commentsondraft-ietf-pcn-architecture-02) Date: Tue, 27 Nov 2007 17:19:40 -0500 Message-ID: <9671A92C3C8B5744BC97F855F7CB6465135B328F@zcarhxm1.corp.nortel.com> In-Reply-To: <1B6169C658325341A3B8066E23919E1C4C1511@S4DE8PSAANK.mitte.t-com.de> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: ECN support in a PCN domain (was: Re: [PCN] commentsondraft-ietf-pcn-architecture-02) Thread-Index: Acgs8zzt10S53v6qR8eaNn6LHzZkfQABDhEwAAC2KKAAR2ew0AB6PO5wABg/ftA= References: <9671A92C3C8B5744BC97F855F7CB6465134D396E@zcarhxm1.corp.nortel.com> <1B6169C658325341A3B8066E23919E1C4C1511@S4DE8PSAANK.mitte.t-com.de> From: "Jozef Babiarz" To: "Geib, Ruediger" X-Spam-Score: 0.0 (/) X-Scan-Signature: a492040269d440726bfd84680622cee7 Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Ruediger, I'm not sure if I understand your scenario completely but will state some assumption so hopefully it will help. Assumptions:=20 PCN function is added to a Diffserv enabled network. That means that this network has at least two service classes (or traffic classes) but most likely more than two.=20 In the Diffserv enabled network there is at least on service class that does not support the PCN function. So if a flow arrives that is not PCN enabled, it should be mapped at ingress to a non-PCN capable service class. The ECN field must be unchanged.=20 Only PCN-capable flows/packets should be mapped into the PCN-enabled service class. If the wg decides to define a new codepoint and PHB for PCN that reuses ECN field for PCN function, than we need to state what happens if a PCN packet hits a router that is not PCN-capable or if it is ECN-capable. If a PCN marked packet hits a router that is not PCN-capable, the router provides the default treatment that any packet would receive. If the ECN field is used for PCN marking and a PCN marked packet hits an ECN-capable router, than the router should threat this packet like any other ECN-cable flow (assuming the ECN-field is other then zero) and set the CE bits if congestion is experienced. I'm not sure if this should be discussed in the architecture draft as it depends on what marking (bits in IP header) approach is agreed to. Regards, Joe email:babiarz@nortel.com Telephone:613-763-6098 -----Original Message----- From: Geib, Ruediger [mailto:Ruediger.Geib@t-systems.com]=20 Sent: November 26, 2007 3:04 AM To: Babiarz, Jozef (CAR:0S03) Cc: philip.eardley@bt.com; magnus.westerlund@ericsson.com; pcn@ietf.org Subject: RE: ECN support in a PCN domain (was: Re: [PCN] commentsondraft-ietf-pcn-architecture-02) Joe, you and Magnus consider fundamental issues. I'm more interested in daily business and there specifically in "unexpected events": the PCN ingress node receives a packet on a non PCN interface and this packet carries a DSCP which the ingress node will map to the PCN DSCP. Further, this packet has some ECN bits set (which is the unexpected event). The architecture document (or any other document indicated by the architecture doc) should recommend an action. And this action should allow for later extensions - congestion feedback given to the end system, especially. Assume the flow to be admitted. My personal preference is to ignore the ECN bits and overwrite them. Dropping seems to tough. And keeping the markings may result in wrong congestion status (may be only depending on the specific ECN codepoint). The same holds, if DSCPs are used for pre-congestion marking and a packet arrives the ingress with a pre-congestion DSCP. We need to define an action of the ingress node. Regards, Rudiger |-----Original Message----- |From: Jozef Babiarz [mailto:babiarz@nortel.com] |Sent: Friday, November 23, 2007 10:48 PM |To: philip.eardley@bt.com; Geib, Rudiger; |magnus.westerlund@ericsson.com; pcn@ietf.org |Subject: RE: ECN support in a PCN domain (was: Re: [PCN] |commentsondraft-ietf-pcn-architecture-02) | | |Phil, Ruediger and Magnus,=20 | |For Section 3.5, I would like to propose that it be replaced with the |statement below. | |If the WG choose to use ECN flied for convey PCN information=20 |through the |network. The PCN WG will request that IANA assign a DSCP value for PCN |traffic class consistent with RFC3246 and with PCN marking behavior |defined by this workgroup. The Diffserv code point should be=20 |referred to |as PCN. | |Explanation: |The PCN mechanism will use a new standardized DSCP and will reuse bits |14 & 15 (the ECN-field) of the IPv4 header for PCN therefore no impact |to RFC3168, meaning that PCN and ECN can co-exist in the same network. | |Could discuss other options but they get really messy.=20 | |Regards, Joe |email:babiarz@nortel.com |Telephone:613-763-6098 |-----Original Message----- |From: philip.eardley@bt.com [mailto:philip.eardley@bt.com]=20 |Sent: November 22, 2007 6:34 AM |To: Ruediger.Geib@t-systems.com; magnus.westerlund@ericsson.com; |Babiarz, Jozef (CAR:0S03) |Subject: RE: ECN support in a PCN domain (was: Re: [PCN] |commentsondraft-ietf-pcn-architecture-02) | |Joe, will leave this one to you as you thought about it more=20 |than I did! |Thanks |phil | |> -----Original Message----- |> From: Geib, Ruediger [mailto:Ruediger.Geib@t-systems.com] |> Sent: 22 November 2007 11:14 |> To: magnus.westerlund@ericsson.com |> Cc: pcn@ietf.org |> Subject: RE: ECN support in a PCN domain (was: Re: [PCN] |commentsondraft- |> ietf-pcn-architecture-02) |>=20 |> Magnus, |>=20 |> I agree to your request, I've asked Phil for a statement on the |behaviour |> of an ingress node receiving a PCN coloured packet in the=20 |architecture |> draft in my latest mail. |>=20 |> I thougth about ignore and remark DSCP and ECN. Drop is a tough |option. |> Forward with DSCP Best Effort and unchanged ECM may be=20 |another one (if |> the DSCP is unknown on the ingress interface). |>=20 |> There are more possibilities, depending on the PCN marking mechanism |and |> whether the ECN codepoints are used. |>=20 |> Regards, |>=20 |> Rudiger |>=20 |> |> |> |> o If a packet that is part of a PCN-flow arrives at a |PCN-ingress- |> |> node with its CE (Congestion experienced) codepoint=20 |set, then |we |> |> assume that the PCN-ingress-node drops the packet. =20 |After its |> |> initial Charter is complete, the WG may decide to work on a |> |> mechanism (such as through a signalling extension) that |enables |> |> ECN-marking to be carried transparently across the |PCN-domain." |> | |> |This one really surprise me. How can you make this assumption? In |case a |> |ECN enabled flow with some packets CE marked arrive to the=20 |ingress of |> |the PCN domain and are within the limits for which the flow has been |> |admitted into the PCN domain, then the PCN domain has no right to |> |summarily drop that packet as being less important than any of the |> |unmarked packets. Only in cases if congestion actually is experience |> |within the PCN domain would a differential treatment of CE marked |> |packets being warranted. |> | |> |Can someone please explain how the above assumptions still satisfy |the |> |following sentence in the charter? |> | |> |"All PCN mechanisms, including transport and encoding |> |of (pre-congestion) information, are required to cleanly integrate |> |with existing architectures and protocols such as DiffServ and ECN." |> | |> | |> |Cheers |> | |> |Magnus Westerlund |> | |> |IETF Transport Area Director & TSVWG Chair |> ||---------------------------------------------------------------------- |> |Multimedia Technologies, Ericsson Research EAB/TVM/M |> ||---------------------------------------------------------------------- |> |Ericsson AB | Phone +46 8 4048287 |> |Torshamsgatan 23 | Fax +46 8 7575550 |> |S-164 80 Stockholm, Sweden | mailto: magnus.westerlund@ericsson.com |> ||---------------------------------------------------------------------- |> | |> | |> |_______________________________________________ |> |PCN mailing list |> |PCN@ietf.org |> |https://www1.ietf.org/mailman/listinfo/pcn |> | |>=20 |>=20 |> _______________________________________________ |> PCN mailing list |> PCN@ietf.org |> https://www1.ietf.org/mailman/listinfo/pcn | _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Tue Nov 27 23:29:16 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IxEXz-0005Bz-Pj; Tue, 27 Nov 2007 23:29:15 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IxEXy-0005Bg-Ho for pcn-confirm+ok@megatron.ietf.org; Tue, 27 Nov 2007 23:29:14 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IxEXx-0005BQ-PB for pcn@ietf.org; Tue, 27 Nov 2007 23:29:14 -0500 Received: from zcars04e.nortel.com ([47.129.242.56]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IxEXv-0001B6-KP for pcn@ietf.org; Tue, 27 Nov 2007 23:29:13 -0500 Received: from zcarhxm1.corp.nortel.com (zcarhxm1.corp.nortel.com [47.129.230.97]) by zcars04e.nortel.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id lAS4Px102697; Wed, 28 Nov 2007 04:25:59 GMT X-MIMEOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [PCN] Architecture draft - probing section & general updates. Date: Tue, 27 Nov 2007 23:28:48 -0500 Message-ID: <9671A92C3C8B5744BC97F855F7CB6465135B3428@zcarhxm1.corp.nortel.com> In-Reply-To: <75A199C5D243C741BF3D3F1EBCEF9BA503B342DC@E03MVZ1-UKDY.domain1.systemhost.net> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] Architecture draft - probing section & general updates. Thread-Index: AcgW7SC2PEVcOi1+Qg+vZ1UxgSPGoQDfIixAAVzgg4AEZKGNsA== References: ben.strulo@bt.com, marc.wennink@bt.com, arnaud.jacquet@bt.com, peter.hovell@bt.com, philip.eardley@bt.com <75A199C5D243C741BF3D3F1EBCEF9BA503B342DC@E03MVZ1-UKDY.domain1.systemhost.net> From: "Jozef Babiarz" To: , X-Spam-Score: -4.0 (----) X-Scan-Signature: a1852b4f554b02e7e4548cc7928acc1f Cc: pcn@ietf.org, Ruediger.Geib@t-systems.com X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Phil, PCN-domain is enterprise network that is within one building or campus. Large enterprises have several locations that are distributed throughout the city, counter or even the world. Between different locations, Layer 2 or Layer 3 VPNs are used to interconnect usually using multiple VPNs to each location to provide resilience should one VPN fail for a period of time. Different approaches (ECMP, policy routing, etc) are used to distribute traffic on two or more VPNs between each location. Each or most of the enterprise locations are PCN-capable and the VPNs (tunnels) are used to interconnect the different locations. The ISPs that provide the tunnels are not PCN-capable. PCN function needs only to be supported within the global enterprise network. Edge nodes that provide admission control and flow termination need to support PCN-edge-node function. Enterprise core nodes and WAN nodes need to provide PCN-interior-node function. Terminally: In enterprise networks WAN (Wide Area Network) edge node is the node that provides connects to a foreign network and where normally Layer 2 or 3 VPN start and end. There may be other tunnels running inside the enterprise network but that is outside the scope of this example. Edge node is a switch or router where end terminals, hosts, servers connect to. Those the above help? Regards, Joe email:babiarz@nortel.com Telephone:613-763-6098 -----Original Message----- From: philip.eardley@bt.com [mailto:philip.eardley@bt.com]=20 Sent: November 5, 2007 1:57 PM To: Babiarz, Jozef (CAR:0S03); lars.eggert@nokia.com Cc: pcn@ietf.org; Ruediger.Geib@t-systems.com Subject: RE: [PCN] Architecture draft - probing section & general updates. Joe - trying to clarify... what's the PCN-domain? Does it run from one *enterprise* LAN edge switch/router to another (there's a tunnelled link between them), or from the end terminals. Ie what are the pcn-boundary-nodes {&pcn-interior-nodes)? The scenario isnt captured I think in the draft-ietf-pcn-architecture, so something should probably be added... Thanks Phil.=20 > -----Original Message----- > From: Jozef Babiarz [mailto:babiarz@nortel.com] > Sent: 29 October 2007 21:06 > To: Lars Eggert > Cc: pcn@ietf.org; Geib, Ruediger > Subject: RE: [PCN] Architecture draft - probing section & general updates. >=20 > Hi Lars, > I did not explain my scenario that clearly, I confused people the word > "access node". So here goes a second try. >=20 > I'm thinking of a scenario that many enterprises face today. Large multi > location enterprises use multiple WAN links or VPS to interconnect their > locations (branch offices). PCN is run inside the enterprise network > including WAN links which may be tunneled across the carrier network. > Carrier network is not required to support PCN as the enterprise traffic > including PCN marking is tunneled, however it could. Network Access > Control with PCN admission control is done at the *enterprise* LAN edge > switch/router. New flows can be routed to any egress LAN edge > switch/router and some flows will (between different locations) be > routed over one of the WAN links which are normally bandwidth > constrained. There is a high probability that many of the edge LAN > switches/routers will have no flows setup between each other (no > ingress-egress aggregate) as there are a large number of edge LAN > switches/routers in the enterprise network and people calling patterns > change. They normal expect that they should be able to call anyone. >=20 >=20 > Regards, Joe > email:babiarz@nortel.com > Telephone:613-763-6098 >=20 > -----Original Message----- > From: Lars Eggert [mailto:lars.eggert@nokia.com] > Sent: October 25, 2007 5:54 AM > To: Babiarz, Jozef (CAR:0S03) > Cc: Geib, Ruediger; pcn@ietf.org > Subject: Re: [PCN] Architecture draft - probing section & general > updates. >=20 > On 2007-10-24, at 19:14, ext Jozef Babiarz wrote: > > I'm thinking of a scenario that many enterprises face today. Large > > multi > > location enterprises use multiple WAN links or VPS to interconnect > > their > > locations. PCN is run inside the enterprise network including WAN > > links > > which maybe tunneled across the carrier network. Network Access > > Control > > with PCN admission control is done at the enterprise access edge > > nodes. > > New flows can be routed to any egress access edge node and some flows > > will (between different locations) be routed over one of the WAN links > > which are normally bandwidth constrained. >=20 > I understand your scenario so far. >=20 > > There is a high probability > > that many access nodes will only have one flow between each other as > > there are a large number of them. >=20 > I don't see how this follows, however. It seems that if the sites > that are being interconnected aren't tiny, there should very likely > be multiple flows per ingress/egress pair, no? >=20 > Lars >=20 >=20 > _______________________________________________ > PCN mailing list > PCN@ietf.org > https://www1.ietf.org/mailman/listinfo/pcn _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Wed Nov 28 03:13:14 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IxI2k-0004Os-HS; Wed, 28 Nov 2007 03:13:14 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IxI2j-0004Oe-UX for pcn-confirm+ok@megatron.ietf.org; Wed, 28 Nov 2007 03:13:13 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IxI2e-0004MH-64 for pcn@ietf.org; Wed, 28 Nov 2007 03:13:08 -0500 Received: from tcmail31.telekom.de ([217.6.95.238]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IxI2b-0005aj-O2 for pcn@ietf.org; Wed, 28 Nov 2007 03:13:08 -0500 Received: from s4de8psaans.mitte.t-com.de by tcmail31.telekom.de with ESMTP; Wed, 28 Nov 2007 09:13:02 +0100 Received: from S4DE8PSAANK.mitte.t-com.de ([10.151.229.10]) by s4de8psaans.mitte.t-com.de with Microsoft SMTPSVC(6.0.3790.3959); Wed, 28 Nov 2007 09:13:02 +0100 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: quoted-printable Subject: RE: ECN support in a PCN domain (was: Re: [PCN] commentsondraft-ietf-pcn-architecture-02) Date: Wed, 28 Nov 2007 09:13:02 +0100 Message-Id: <1B6169C658325341A3B8066E23919E1C4C1557@S4DE8PSAANK.mitte.t-com.de> In-Reply-To: <9671A92C3C8B5744BC97F855F7CB6465135B328F@zcarhxm1.corp.nortel.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: ECN support in a PCN domain (was: Re: [PCN] commentsondraft-ietf-pcn-architecture-02) Thread-Index: Acgs8zzt10S53v6qR8eaNn6LHzZkfQABDhEwAAC2KKAAR2ew0AB6PO5wABg/ftAATCehcA== From: "Geib, Ruediger" To: X-OriginalArrivalTime: 28 Nov 2007 08:13:02.0410 (UTC) FILETIME=[7EE8CAA0:01C83196] X-Spam-Score: -1.0 (-) X-Scan-Signature: 5fb88b8381f3896aeacc5a021513237b Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Joe, the issue on which I suggest to have text on is simpler than the ones=20 you suggest. To clarify my point, I add a short sketch: +--------------+ ---non PCN domain-->-| Ingress Node |->-PCN domain------ +--------------+ My point is that I'd like to have an explicit statement in the=20 architecture, how the ingress node should behave if it receives a=20 "PCN coloured" packet on the non PCN domain interface. The=20 behaviour should cover all cases, i.e. also those=20 which are not planned like a packet of an admitted flow with=20 an ECN codepoint or DSCP indicating pre-congestion received=20 on the non PCN domain interface. Phil mentioned that there's text in the draft architecture. I don't find a clear statement or a reference on the text=20 defining the desired behaviour for the above case in: 5.2. PCN-ingress-node functions 10. Security considerations Regards, Rudiger |-----Original Message----- |From: Jozef Babiarz [mailto:babiarz@nortel.com] |Sent: Tuesday, November 27, 2007 11:20 PM |To: Geib, Rudiger |Cc: philip.eardley@bt.com; magnus.westerlund@ericsson.com; pcn@ietf.org |Subject: RE: ECN support in a PCN domain (was: Re: [PCN] |commentsondraft-ietf-pcn-architecture-02) | | |Ruediger, I'm not sure if I understand your scenario=20 |completely but will |state some assumption so hopefully it will help. | |Assumptions:=20 |PCN function is added to a Diffserv enabled network. That means that |this network has at least two service classes (or traffic classes) but |most likely more than two.=20 | |In the Diffserv enabled network there is at least on service class that |does not support the PCN function. | |So if a flow arrives that is not PCN enabled, it should be mapped at |ingress to a non-PCN capable service class. The ECN field must be |unchanged.=20 | |Only PCN-capable flows/packets should be mapped into the PCN-enabled |service class. | |If the wg decides to define a new codepoint and PHB for PCN that reuses |ECN field for PCN function, than we need to state what happens if a PCN |packet hits a router that is not PCN-capable or if it is ECN-capable. | |If a PCN marked packet hits a router that is not PCN-capable,=20 |the router |provides the default treatment that any packet would receive. | |If the ECN field is used for PCN marking and a PCN marked=20 |packet hits an |ECN-capable router, than the router should threat this packet like any |other ECN-cable flow (assuming the ECN-field is other then=20 |zero) and set |the CE bits if congestion is experienced. | |I'm not sure if this should be discussed in the architecture=20 |draft as it |depends on what marking (bits in IP header) approach is agreed to. | | |Regards, Joe |email:babiarz@nortel.com |Telephone:613-763-6098 | |-----Original Message----- |From: Geib, Ruediger [mailto:Ruediger.Geib@t-systems.com]=20 |Sent: November 26, 2007 3:04 AM |To: Babiarz, Jozef (CAR:0S03) |Cc: philip.eardley@bt.com; magnus.westerlund@ericsson.com; pcn@ietf.org |Subject: RE: ECN support in a PCN domain (was: Re: [PCN] |commentsondraft-ietf-pcn-architecture-02) | |Joe, | |you and Magnus consider fundamental issues. I'm more=20 |interested in daily |business and there specifically in "unexpected events": the PCN ingress |node receives a packet on a non PCN interface and this packet carries a |DSCP which the ingress node will map to the PCN DSCP. Further, this |packet has some ECN bits set (which is the unexpected event). The |architecture document (or any other document indicated by the |architecture doc) should recommend an action. And this action should |allow for later extensions - congestion feedback given to the end |system, especially. | |Assume the flow to be admitted. My personal preference is to ignore the |ECN bits and overwrite them. Dropping seems to tough. And keeping the |markings may result in wrong congestion status (may be only=20 |depending on |the specific ECN codepoint). | |The same holds, if DSCPs are used for pre-congestion marking and a |packet arrives the ingress with a pre-congestion DSCP. | |We need to define an action of the ingress node. | |Regards, | |Rudiger | ||-----Original Message----- ||From: Jozef Babiarz [mailto:babiarz@nortel.com] ||Sent: Friday, November 23, 2007 10:48 PM ||To: philip.eardley@bt.com; Geib, Rudiger; ||magnus.westerlund@ericsson.com; pcn@ietf.org ||Subject: RE: ECN support in a PCN domain (was: Re: [PCN] ||commentsondraft-ietf-pcn-architecture-02) || || ||Phil, Ruediger and Magnus,=20 || ||For Section 3.5, I would like to propose that it be replaced with the ||statement below. || ||If the WG choose to use ECN flied for convey PCN information=20 ||through the ||network. The PCN WG will request that IANA assign a DSCP value for PCN ||traffic class consistent with RFC3246 and with PCN marking behavior ||defined by this workgroup. The Diffserv code point should be=20 ||referred to ||as PCN. || ||Explanation: ||The PCN mechanism will use a new standardized DSCP and will reuse bits ||14 & 15 (the ECN-field) of the IPv4 header for PCN therefore no impact ||to RFC3168, meaning that PCN and ECN can co-exist in the same network. || ||Could discuss other options but they get really messy.=20 || ||Regards, Joe ||email:babiarz@nortel.com ||Telephone:613-763-6098 ||-----Original Message----- ||From: philip.eardley@bt.com [mailto:philip.eardley@bt.com]=20 ||Sent: November 22, 2007 6:34 AM ||To: Ruediger.Geib@t-systems.com; magnus.westerlund@ericsson.com; ||Babiarz, Jozef (CAR:0S03) ||Subject: RE: ECN support in a PCN domain (was: Re: [PCN] ||commentsondraft-ietf-pcn-architecture-02) || ||Joe, will leave this one to you as you thought about it more=20 ||than I did! ||Thanks ||phil || ||> -----Original Message----- ||> From: Geib, Ruediger [mailto:Ruediger.Geib@t-systems.com] ||> Sent: 22 November 2007 11:14 ||> To: magnus.westerlund@ericsson.com ||> Cc: pcn@ietf.org ||> Subject: RE: ECN support in a PCN domain (was: Re: [PCN] ||commentsondraft- ||> ietf-pcn-architecture-02) ||>=20 ||> Magnus, ||>=20 ||> I agree to your request, I've asked Phil for a statement on the ||behaviour ||> of an ingress node receiving a PCN coloured packet in the=20 ||architecture ||> draft in my latest mail. ||>=20 ||> I thougth about ignore and remark DSCP and ECN. Drop is a tough ||option. ||> Forward with DSCP Best Effort and unchanged ECM may be=20 ||another one (if ||> the DSCP is unknown on the ingress interface). ||>=20 ||> There are more possibilities, depending on the PCN marking mechanism ||and ||> whether the ECN codepoints are used. ||>=20 ||> Regards, ||>=20 ||> Rudiger ||>=20 ||> |> ||> |> o If a packet that is part of a PCN-flow arrives at a ||PCN-ingress- ||> |> node with its CE (Congestion experienced) codepoint=20 ||set, then ||we ||> |> assume that the PCN-ingress-node drops the packet. =20 ||After its ||> |> initial Charter is complete, the WG may decide to work on a ||> |> mechanism (such as through a signalling extension) that ||enables ||> |> ECN-marking to be carried transparently across the ||PCN-domain." ||> | ||> |This one really surprise me. How can you make this assumption? In ||case a ||> |ECN enabled flow with some packets CE marked arrive to the=20 ||ingress of ||> |the PCN domain and are within the limits for which the=20 |flow has been ||> |admitted into the PCN domain, then the PCN domain has no right to ||> |summarily drop that packet as being less important than any of the ||> |unmarked packets. Only in cases if congestion actually is=20 |experience ||> |within the PCN domain would a differential treatment of CE marked ||> |packets being warranted. ||> | ||> |Can someone please explain how the above assumptions still satisfy ||the ||> |following sentence in the charter? ||> | ||> |"All PCN mechanisms, including transport and encoding ||> |of (pre-congestion) information, are required to cleanly integrate ||> |with existing architectures and protocols such as DiffServ=20 |and ECN." ||> | ||> | ||> |Cheers ||> | ||> |Magnus Westerlund ||> | ||> |IETF Transport Area Director & TSVWG Chair ||> |||------------------------------------------------------------- |--------- ||> |Multimedia Technologies, Ericsson Research EAB/TVM/M ||> |||------------------------------------------------------------- |--------- ||> |Ericsson AB | Phone +46 8 4048287 ||> |Torshamsgatan 23 | Fax +46 8 7575550 ||> |S-164 80 Stockholm, Sweden | mailto: magnus.westerlund@ericsson.com ||> |||------------------------------------------------------------- |--------- ||> | ||> | ||> |_______________________________________________ ||> |PCN mailing list ||> |PCN@ietf.org ||> |https://www1.ietf.org/mailman/listinfo/pcn ||> | ||>=20 ||>=20 ||> _______________________________________________ ||> PCN mailing list ||> PCN@ietf.org ||> https://www1.ietf.org/mailman/listinfo/pcn || | _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Wed Nov 28 05:44:44 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IxKPL-0006SF-FF; Wed, 28 Nov 2007 05:44:43 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IxKPJ-0006S1-MJ for pcn-confirm+ok@megatron.ietf.org; Wed, 28 Nov 2007 05:44:41 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IxKPJ-0006Rs-7H for pcn@ietf.org; Wed, 28 Nov 2007 05:44:41 -0500 Received: from mailgw3.ericsson.se ([193.180.251.60]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IxKPI-0005q6-2j for pcn@ietf.org; Wed, 28 Nov 2007 05:44:41 -0500 Received: from mailgw3.ericsson.se (unknown [127.0.0.1]) by mailgw3.ericsson.se (Symantec Mail Security) with ESMTP id B72E420AD9 for ; Wed, 28 Nov 2007 11:44:20 +0100 (CET) X-AuditID: c1b4fb3c-aef95bb0000030cf-7b-474d4684912b Received: from esealmw126.eemea.ericsson.se (unknown [153.88.254.123]) by mailgw3.ericsson.se (Symantec Mail Security) with ESMTP id A55F0208A3 for ; Wed, 28 Nov 2007 11:44:20 +0100 (CET) Received: from esealmw128.eemea.ericsson.se ([153.88.254.172]) by esealmw126.eemea.ericsson.se with Microsoft SMTPSVC(6.0.3790.1830); Wed, 28 Nov 2007 11:44:20 +0100 Received: from [147.214.30.247] ([147.214.30.247]) by esealmw128.eemea.ericsson.se with Microsoft SMTPSVC(6.0.3790.1830); Wed, 28 Nov 2007 11:44:20 +0100 Message-ID: <474D4683.4080209@ericsson.com> Date: Wed, 28 Nov 2007 11:44:19 +0100 From: Magnus Westerlund User-Agent: Thunderbird 2.0.0.9 (Windows/20071031) MIME-Version: 1.0 To: Steven Blake References: <47455AD6.2030404@ericsson.com> <1195841689.4309.9.camel@neutrino> In-Reply-To: <1195841689.4309.9.camel@neutrino> X-Enigmail-Version: 0.95.5 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 28 Nov 2007 10:44:20.0301 (UTC) FILETIME=[A1C107D0:01C831AB] X-Brightmail-Tracker: AAAAAA== X-Spam-Score: -1.0 (-) X-Scan-Signature: 3a4bc66230659131057bb68ed51598f8 Cc: pcn Subject: [PCN] Re: ECN support in a PCN domain X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Steven Blake skrev: > On Thu, 2007-11-22 at 11:32 +0100, Magnus Westerlund wrote: > >> Hi, >> >> See inline >> >>> Section 3.5: >>> >>> "The following two assumptions apply if the PCN WG decides to encode >>> PCN-marking in the ECN-field. >>> >>> o It is assumed that PCN-nodes do not perform ECN, [RFC3168], on >>> PCN-packets. >> I would like to have a clarification on what is meant with PCN not doing >> ECN. To me it seems that in situation where a ECN enabled flow goes >> through a PCN domain and the flow are in a situation where a router >> actually is congested, it should be able to have that those packets >> being marked with CE when leaving the PCN domain. I know this is phase >> two of the work if we ever gets there as it discusses another response >> mechanism. > > Sentence one of the PCN charter says: > > "The Congestion and Pre-Congestion Notification (PCN) working group > develops mechanisms to protect the quality-of-service of established > inelastic flows within a DiffServ domain when congestion is imminent > or existing." > > Should "inelastic" flows ever have one of the ECT nonces set? It could have, and also inelastic is not far from being rate change capable which in my mind is a future step for PCN. In other words indicate rate change needed rather than flow termination when being congested. And from that point of view, is that what PCN treats as inelastic traffic may actually use ECN for rate adaptation end to end. Thus ECN enabled traffic may show up on a PCN enabled ingress node with a DSCP mapping it into a PCN enabled flow. > > The reason this might be an issue, is that PCN is considering reusing > the ECN bits. Yes, and that is why PCN needs to consider on how to maintain ECN capability. I would like to point to the following sentence in the PCN charter: "All PCN mechanisms, including transport and encoding of (pre-congestion) information, are required to cleanly integrate with existing architectures and protocols such as DiffServ and ECN." > > For clarification, the draft text does not apply to elastic traffic that > is not being handled by PCN mechanisms (i.e., not part of a PCN-managed > behavior aggregate). Agreed. > >>> o If a packet that is part of a PCN-flow arrives at a PCN-ingress- >>> node with its CE (Congestion experienced) codepoint set, then we >>> assume that the PCN-ingress-node drops the packet. After its >>> initial Charter is complete, the WG may decide to work on a >>> mechanism (such as through a signalling extension) that enables >>> ECN-marking to be carried transparently across the PCN-domain." >> This one really surprise me. How can you make this assumption? In case a >> ECN enabled flow with some packets CE marked arrive to the ingress of >> the PCN domain and are within the limits for which the flow has been >> admitted into the PCN domain, then the PCN domain has no right to >> summarily drop that packet as being less important than any of the >> unmarked packets. Only in cases if congestion actually is experience >> within the PCN domain would a differential treatment of CE marked >> packets being warranted. >> >> Can someone please explain how the above assumptions still satisfy the >> following sentence in the charter? >> >> "All PCN mechanisms, including transport and encoding >> of (pre-congestion) information, are required to cleanly integrate >> with existing architectures and protocols such as DiffServ and ECN." > > I think that PCN has been defined in the charter to not apply for > traffic which might use an ECN-capable transport protocol. > No, I don't agree with this characterization. I can't remember that being the notion. If I remember Lars and my discussion one of the reason we did charter PCN like we did was to ensure that you actually could find a encoding that works and maintains ECN across the PCN domain. And that is according to my memory what the following sentence is intended for: All PCN mechanisms, including transport and encoding of (pre-congestion) information, are required to cleanly integrate with existing architectures and protocols such as DiffServ and ECN. And if there exist no reasonable solution to encode PCN while maintaining ECN across a PCN domain. Well that would be tough luck and we would have to revisit the fundamentals. Which could require a new BOF and rechartering to remove those. My main message is: As I see it you are not allowed to take the easy way out and ignore that incomming traffic may be ECN enabled. You will have to do something reasonable with that. cheers Magnus Westerlund IETF Transport Area Director & TSVWG Chair ---------------------------------------------------------------------- Multimedia Technologies, Ericsson Research EAB/TVM ---------------------------------------------------------------------- Ericsson AB | Phone +46 8 4048287 Torshamsgatan 23 | Fax +46 8 7575550 S-164 80 Stockholm, Sweden | mailto: magnus.westerlund@ericsson.com ---------------------------------------------------------------------- _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Wed Nov 28 11:09:30 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IxPTe-0004B2-HE; Wed, 28 Nov 2007 11:09:30 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IxPTe-0004Aw-1t for pcn-confirm+ok@megatron.ietf.org; Wed, 28 Nov 2007 11:09:30 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IxPTd-0004Ao-LQ for pcn@ietf.org; Wed, 28 Nov 2007 11:09:29 -0500 Received: from imr2.ericy.com ([198.24.6.3]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IxPTd-0001G0-4f for pcn@ietf.org; Wed, 28 Nov 2007 11:09:29 -0500 Received: from eusrcmw751.eamcs.ericsson.se (eusrcmw751.exu.ericsson.se [138.85.77.51]) by imr2.ericy.com (8.13.1/8.13.1) with ESMTP id lASG9SuB021133 for ; Wed, 28 Nov 2007 10:09:28 -0600 Received: from eusrcmw751.eamcs.ericsson.se ([138.85.77.56]) by eusrcmw751.eamcs.ericsson.se with Microsoft SMTPSVC(6.0.3790.1830); Wed, 28 Nov 2007 10:09:28 -0600 Received: from [147.117.169.183] ([147.117.169.183]) by eusrcmw751.eamcs.ericsson.se with Microsoft SMTPSVC(6.0.3790.1830); Wed, 28 Nov 2007 10:09:28 -0600 From: Steven Blake To: Magnus Westerlund In-Reply-To: <474D4683.4080209@ericsson.com> References: <47455AD6.2030404@ericsson.com> <1195841689.4309.9.camel@neutrino> <474D4683.4080209@ericsson.com> Content-Type: text/plain Organization: Ericsson IP Infrastructure Date: Wed, 28 Nov 2007 11:09:28 -0500 Message-Id: <1196266168.5664.14.camel@neutrino> Mime-Version: 1.0 X-Mailer: Evolution 2.12.1 (2.12.1-3.fc8) Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 28 Nov 2007 16:09:28.0337 (UTC) FILETIME=[0D731810:01C831D9] X-Spam-Score: 0.0 (/) X-Scan-Signature: 244a2fd369eaf00ce6820a760a3de2e8 Cc: pcn Subject: [PCN] Re: ECN support in a PCN domain X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org On Wed, 2007-11-28 at 11:44 +0100, Magnus Westerlund wrote: > Steven Blake skrev: > > On Thu, 2007-11-22 at 11:32 +0100, Magnus Westerlund wrote: > > > >> Hi, > >> > >> See inline > >> > >>> Section 3.5: > >>> > >>> "The following two assumptions apply if the PCN WG decides to encode > >>> PCN-marking in the ECN-field. > >>> > >>> o It is assumed that PCN-nodes do not perform ECN, [RFC3168], on > >>> PCN-packets. > >> I would like to have a clarification on what is meant with PCN not doing > >> ECN. To me it seems that in situation where a ECN enabled flow goes > >> through a PCN domain and the flow are in a situation where a router > >> actually is congested, it should be able to have that those packets > >> being marked with CE when leaving the PCN domain. I know this is phase > >> two of the work if we ever gets there as it discusses another response > >> mechanism. > > > > Sentence one of the PCN charter says: > > > > "The Congestion and Pre-Congestion Notification (PCN) working group > > develops mechanisms to protect the quality-of-service of established > > inelastic flows within a DiffServ domain when congestion is imminent > > or existing." > > > > Should "inelastic" flows ever have one of the ECT nonces set? > > It could have, and also inelastic is not far from being rate change > capable which in my mind is a future step for PCN. In other words > indicate rate change needed rather than flow termination when being > congested. And from that point of view, is that what PCN treats as > inelastic traffic may actually use ECN for rate adaptation end to end. > > Thus ECN enabled traffic may show up on a PCN enabled ingress node with > a DSCP mapping it into a PCN enabled flow. We obviously have a different understanding of ECN (which means that mine is probably wrong). [snip] > My main message is: As I see it you are not allowed to take the easy way > out and ignore that incomming traffic may be ECN enabled. You will have > to do something reasonable with that. Understood. Note to wg: let us keep this point in mind when discussing draft-chan-pcn-encoding-comparison-01. Regards, =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Steven Blake Ericsson/Redback Networks +1 919-472-9913 _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Wed Nov 28 11:21:10 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IxPew-0003ki-A7; Wed, 28 Nov 2007 11:21:10 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IxPeu-0003jM-8y for pcn-confirm+ok@megatron.ietf.org; Wed, 28 Nov 2007 11:21:08 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IxPet-0003iz-Ur for pcn@ietf.org; Wed, 28 Nov 2007 11:21:07 -0500 Received: from imr2.ericy.com ([198.24.6.3]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IxPet-0002wj-Hl for pcn@ietf.org; Wed, 28 Nov 2007 11:21:07 -0500 Received: from eusrcmw751.eamcs.ericsson.se (eusrcmw751.exu.ericsson.se [138.85.77.51]) by imr2.ericy.com (8.13.1/8.13.1) with ESMTP id lASGL7fK024483 for ; Wed, 28 Nov 2007 10:21:07 -0600 Received: from eusrcmw750.eamcs.ericsson.se ([138.85.77.50]) by eusrcmw751.eamcs.ericsson.se with Microsoft SMTPSVC(6.0.3790.1830); Wed, 28 Nov 2007 10:21:06 -0600 Received: from [147.117.169.183] ([147.117.169.183]) by eusrcmw750.eamcs.ericsson.se with Microsoft SMTPSVC(6.0.3790.1830); Wed, 28 Nov 2007 10:21:06 -0600 From: Steven Blake To: pcn Content-Type: text/plain Organization: Ericsson IP Infrastructure Date: Wed, 28 Nov 2007 11:21:06 -0500 Message-Id: <1196266866.5664.25.camel@neutrino> Mime-Version: 1.0 X-Mailer: Evolution 2.12.1 (2.12.1-3.fc8) Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 28 Nov 2007 16:21:06.0752 (UTC) FILETIME=[ADBCC400:01C831DA] X-Spam-Score: 0.0 (/) X-Scan-Signature: e5ba305d0e64821bf3d8bc5d3bb07228 Subject: [PCN] IETF 70 PCN agenda X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org We will stick with the meeting agenda I posted last week; see: http://www3.ietf.org/proceedings/07dec/agenda/pcn.txt The chairs received the feedback about getting another timeslot. This is problematic logistically and we decided to stick with the proposed agenda for the reasons listed previously. Thanks to Philip for posting a list of open issues on draft-ietf-pcn-architecture-02. Let's keep the discussion going on the list. Kwok, could you please post a similar list of issues for draft-chan-pcn-encoding-comparison-01? We should all heed our AD's input regarding ECN compatibility and the constraints this may place on the encoding options. Everyone is encouraged and expected to review these drafts prior to the meeting. If you are preparing slides to discuss one or more of the open issues, please notify the chairs and please submit your slides prior to the meeting. Have a safe trip to Vancouver. Regards, =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Steven Blake Ericsson/Redback Networks +1 919-472-9913 _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Wed Nov 28 12:49:46 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IxR2g-0002KC-84; Wed, 28 Nov 2007 12:49:46 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IxR2f-0002GP-8I for pcn-confirm+ok@megatron.ietf.org; Wed, 28 Nov 2007 12:49:45 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IxR2e-0002GF-UX for pcn@ietf.org; Wed, 28 Nov 2007 12:49:44 -0500 Received: from smtp2.smtp.bt.com ([217.32.164.150]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IxR2d-0003GU-3F for pcn@ietf.org; Wed, 28 Nov 2007 12:49:44 -0500 Received: from E03MVZ1-UKDY.domain1.systemhost.net ([193.113.30.65]) by smtp2.smtp.bt.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 28 Nov 2007 17:49:41 +0000 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [PCN] Re: ECN support in a PCN domain Date: Wed, 28 Nov 2007 17:49:41 -0000 Message-ID: <75A199C5D243C741BF3D3F1EBCEF9BA503B343D6@E03MVZ1-UKDY.domain1.systemhost.net> In-Reply-To: <474D4683.4080209@ericsson.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] Re: ECN support in a PCN domain Thread-Index: Acgxq7MTUQ7YyIB6S/Sr7L4i/hUjJQANz74A From: To: , X-OriginalArrivalTime: 28 Nov 2007 17:49:41.0935 (UTC) FILETIME=[0DD553F0:01C831E7] X-Spam-Score: -1.0 (-) X-Scan-Signature: b5d20af10c334b36874c0264b10f59f1 Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org > > > >>> o If a packet that is part of a PCN-flow arrives at a PCN-ingress- > >>> node with its CE (Congestion experienced) codepoint set, then we > >>> assume that the PCN-ingress-node drops the packet. After its > >>> initial Charter is complete, the WG may decide to work on a > >>> mechanism (such as through a signalling extension) that enables > >>> ECN-marking to be carried transparently across the PCN-domain." > >> This one really surprise me. How can you make this assumption? In case > a > >> ECN enabled flow with some packets CE marked arrive to the ingress of > >> the PCN domain and are within the limits for which the flow has been > >> admitted into the PCN domain, then the PCN domain has no right to > >> summarily drop that packet as being less important than any of the > >> unmarked packets. Only in cases if congestion actually is experience > >> within the PCN domain would a differential treatment of CE marked > >> packets being warranted. > >> > >> Can someone please explain how the above assumptions still satisfy the > >> following sentence in the charter? > >> > >> "All PCN mechanisms, including transport and encoding > >> of (pre-congestion) information, are required to cleanly integrate > >> with existing architectures and protocols such as DiffServ and ECN." > > > > I think that PCN has been defined in the charter to not apply for > > traffic which might use an ECN-capable transport protocol. > > >=20 > No, I don't agree with this characterization. I can't remember that > being the notion. If I remember Lars and my discussion one of the > reason we did charter PCN like we did was to ensure that you actually > could find a encoding that works and maintains ECN across the PCN > domain. And that is according to my memory what the following sentence > is intended for: >=20 > All PCN mechanisms, including transport and encoding > of (pre-congestion) information, are required to cleanly integrate > with existing architectures and protocols such as DiffServ and ECN. >=20 > And if there exist no reasonable solution to encode PCN while > maintaining ECN across a PCN domain. Well that would be tough luck and > we would have to revisit the fundamentals. Which could require a new BOF > and rechartering to remove those. >=20 > My main message is: As I see it you are not allowed to take the easy way > out and ignore that incomming traffic may be ECN enabled. You will have > to do something reasonable with that. >=20 magnus, here's the background to the bulleted text [at the start of above - in current I-D]. or rather, my perspective on the background. there was quite a lot of discussion (without resolution) whether it's reasonable for a flow that uses PCN adm ctrl to also want e2e ECN for rate adaptation (discussion both in terms of whether a flow would actually ever want this, and how well the PCN mechanism would work if flows were adapting like this).=20 this was resolved by saying that the support of e2e ecn is out of scope of initial charter of pcn.=20 this leaves the question: if/when after re-chartering the scope does include e2e ecn, how would this impact pcn encoding choice (which we have to make now)?=20 The solution:=20 - we know that if use (only) DSCPs for encoding PCN-marks, then e2e ECN would be ok - we know that if use ECN field for encoding PCN-marks, then e2e ECN could be supported; the obvious way is to tunnel the flow over the PCN-domain & to establish the tunnel as part of the signalling set-up for this flow.=20 - [rationale for the text at top of this email quoted from the I-D]: but we haven't worked out the details for the tunnelling /signalling, so let's give some very simple advice if the situation does occur. The best thing seems to drop the pkt, as at least this means the receiver behaves in the same way as if it got the ecn-marked pkt. If this is felt to be unhelpful or misleading, we could instead say something about signalling/tunnelling approach ['and details would be worked out after re-chartering as appropriate..'] phil _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Wed Nov 28 13:12:34 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IxROk-0008Rf-5G; Wed, 28 Nov 2007 13:12:34 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IxROj-0008RS-6A for pcn-confirm+ok@megatron.ietf.org; Wed, 28 Nov 2007 13:12:33 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IxROi-0008RJ-ST; Wed, 28 Nov 2007 13:12:32 -0500 Received: from smtp.nokia.com ([192.100.122.230] helo=mgw-mx03.nokia.com) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IxROi-00067n-Au; Wed, 28 Nov 2007 13:12:32 -0500 Received: from esebh106.NOE.Nokia.com (esebh106.ntc.nokia.com [172.21.138.213]) by mgw-mx03.nokia.com (Switch-3.2.6/Switch-3.2.6) with ESMTP id lASIBlEj008159; Wed, 28 Nov 2007 20:12:29 +0200 Received: from esebh102.NOE.Nokia.com ([172.21.138.183]) by esebh106.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 28 Nov 2007 20:12:15 +0200 Received: from mgw-int01.ntc.nokia.com ([172.21.143.96]) by esebh102.NOE.Nokia.com over TLS secured channel with Microsoft SMTPSVC(6.0.3790.1830); Wed, 28 Nov 2007 20:12:14 +0200 Received: from [192.168.255.2] (essapo-nirac252204.europe.nokia.com [10.162.252.204]) by mgw-int01.ntc.nokia.com (Switch-3.2.5/Switch-3.2.5) with ESMTP id lASICD09027474; Wed, 28 Nov 2007 20:12:13 +0200 Message-Id: From: Lars Eggert To: "ext pcn-bounces@ietf.org" In-Reply-To: <75A199C5D243C741BF3D3F1EBCEF9BA503B343D6@E03MVZ1-UKDY.domain1.systemhost.net> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v915) Subject: Re: [PCN] Re: ECN support in a PCN domain Date: Wed, 28 Nov 2007 20:12:12 +0200 References: <75A199C5D243C741BF3D3F1EBCEF9BA503B343D6@E03MVZ1-UKDY.domain1.systemhost.net> X-Mailer: Apple Mail (2.915) X-OriginalArrivalTime: 28 Nov 2007 18:12:14.0785 (UTC) FILETIME=[3431DB10:01C831EA] X-Nokia-AV: Clean X-Spam-Score: 0.0 (/) X-Scan-Signature: 9182cfff02fae4f1b6e9349e01d62f32 Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org On 2007-11-28, at 19:49, ext pcn-bounces@ietf.org wrote: > there was quite a lot of discussion (without resolution) whether it's > reasonable for a flow that uses PCN adm ctrl to also want e2e ECN for > rate adaptation (discussion both in terms of whether a flow would > actually ever want this, and how well the PCN mechanism would work if > flows were adapting like this). > this was resolved by saying that the support of e2e ecn is out of > scope > of initial charter of pcn. Huh? The charter says: All PCN mechanisms, including transport and encoding of (pre-congestion) information, are required to cleanly integrate with existing architectures and protocols such as DiffServ and ECN. "Cleanly integrate" doesn't mean "out of scope", it means "must work together with". TSVWG published RFC4774 to describe which alternate uses of the ECN bits are OK; the PCN mechanisms are supposed to instantiate such an accepted alternate use. Lars _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Wed Nov 28 13:47:29 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IxRwW-0007Xk-TA; Wed, 28 Nov 2007 13:47:28 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IxRwV-0007Wf-Sp for pcn-confirm+ok@megatron.ietf.org; Wed, 28 Nov 2007 13:47:27 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IxRwV-0007WT-HT for pcn@ietf.org; Wed, 28 Nov 2007 13:47:27 -0500 Received: from imr2.ericy.com ([198.24.6.3]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IxRwV-0005ch-25 for pcn@ietf.org; Wed, 28 Nov 2007 13:47:27 -0500 Received: from eusrcmw750.eamcs.ericsson.se (eusrcmw750.exu.ericsson.se [138.85.77.50]) by imr2.ericy.com (8.13.1/8.13.1) with ESMTP id lASIlOWX017588; Wed, 28 Nov 2007 12:47:24 -0600 Received: from eusrcmw751.eamcs.ericsson.se ([138.85.77.51]) by eusrcmw750.eamcs.ericsson.se with Microsoft SMTPSVC(6.0.3790.1830); Wed, 28 Nov 2007 12:47:24 -0600 Received: from [147.117.169.183] ([147.117.169.183]) by eusrcmw751.eamcs.ericsson.se with Microsoft SMTPSVC(6.0.3790.1830); Wed, 28 Nov 2007 12:47:24 -0600 Subject: Re: [PCN] Open issues on architecture draft. From: Steven Blake To: philip.eardley@bt.com In-Reply-To: <75A199C5D243C741BF3D3F1EBCEF9BA503B3439E@E03MVZ1-UKDY.domain1.systemhost.net> References: <75A199C5D243C741BF3D3F1EBCEF9BA503B3439E@E03MVZ1-UKDY.domain1.systemhost.net> Content-Type: text/plain Organization: Ericsson IP Infrastructure Date: Wed, 28 Nov 2007 13:47:23 -0500 Message-Id: <1196275643.5664.45.camel@neutrino> Mime-Version: 1.0 X-Mailer: Evolution 2.12.1 (2.12.1-3.fc8) Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 28 Nov 2007 18:47:24.0216 (UTC) FILETIME=[1D837B80:01C831EF] X-Spam-Score: 0.0 (/) X-Scan-Signature: 2409bba43e9c8d580670fda8b695204a Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Philip et. al., I think there are open issues surrounding Sec. 5.6, namely: - how an egress node associates a received packet with an ingress-egress aggregate - whether centralized nodes are in scope for the architecture Regarding the former: I don't believe that it is the responsibility of PCN to mandate a mechanism for making the packet->aggregate association, nor is it our responsibility to define protocol mechanisms to facilitate this (although I would appreciate input from the ADs). It would be good to discuss the issue in more detail in the document; e.g., list alternatives, discuss trade-offs, etc. Regarding the latter issue: I have a strong opinion. I think this needs more discussion by the working group. Regards, =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Steven Blake Ericsson/Redback Networks +1 919-472-9913 _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Wed Nov 28 16:20:28 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IxUKa-0001oB-L2; Wed, 28 Nov 2007 16:20:28 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IxUKZ-0001kA-GT for pcn-confirm+ok@megatron.ietf.org; Wed, 28 Nov 2007 16:20:27 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IxUKY-0001ex-NN for pcn@ietf.org; Wed, 28 Nov 2007 16:20:26 -0500 Received: from wrzx28.rz.uni-wuerzburg.de ([132.187.3.28] helo=mailrelay.rz.uni-wuerzburg.de) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IxUKY-0007QQ-3t for pcn@ietf.org; Wed, 28 Nov 2007 16:20:26 -0500 Received: from virusscan.mail (localhost [127.0.0.1]) by mailrelay.mail (Postfix) with ESMTP id 1E995B606; Wed, 28 Nov 2007 22:20:25 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by virusscan.mail (Postfix) with ESMTP id 11178C4D3; Wed, 28 Nov 2007 22:20:25 +0100 (CET) Received: from europa.informatik.uni-wuerzburg.de (wicx01.informatik.uni-wuerzburg.de [132.187.11.1]) by mailmaster.uni-wuerzburg.de (Postfix) with ESMTP id BE90EB606; Wed, 28 Nov 2007 22:20:22 +0100 (CET) Received: from nero.informatik.uni-wuerzburg.de (win3005.informatik.uni-wuerzburg.de [132.187.106.5]) by europa.informatik.uni-wuerzburg.de (8.11.3/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id lASLKMI12881; Wed, 28 Nov 2007 22:20:22 +0100 Received: from [127.0.0.1] (nero.informatik.uni-wuerzburg.de [132.187.106.5]) by nero.informatik.uni-wuerzburg.de (Postfix) with ESMTP id 06B426F591; Wed, 28 Nov 2007 22:12:34 +0100 (CET) Message-ID: <474DDC09.4040504@informatik.uni-wuerzburg.de> Date: Wed, 28 Nov 2007 22:22:17 +0100 From: Michael Menth Organization: University of Wuerzburg User-Agent: Thunderbird 2.0.0.9 (Windows/20071031) MIME-Version: 1.0 To: Steven Blake Subject: Re: [PCN] Open issues on architecture draft. References: <75A199C5D243C741BF3D3F1EBCEF9BA503B3439E@E03MVZ1-UKDY.domain1.systemhost.net> <1196275643.5664.45.camel@neutrino> In-Reply-To: <1196275643.5664.45.camel@neutrino> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at uni-wuerzburg.de X-Spam-Score: 0.0 (/) X-Scan-Signature: 02ec665d00de228c50c93ed6b5e4fc1a Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: menth@informatik.uni-wuerzburg.de List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi Steven, Steven Blake wrote: > Philip et. al., > > I think there are open issues surrounding Sec. 5.6, > namely: > > - how an egress node associates a received packet with an > ingress-egress aggregate > > - whether centralized nodes are in scope for the architecture > > Regarding the former: I don't believe that it is the responsibility of > PCN to mandate a mechanism for making the packet->aggregate association, > nor is it our responsibility to define protocol mechanisms to facilitate > this (although I would appreciate input from the ADs). It would be good > to discuss the issue in more detail in the document; e.g., list > alternatives, discuss trade-offs, etc. > I think the association of packet - ingress-egress-aggregate could depend on the scenario in which PCN is applied. MPLS tunnels might help or also per-flow classifiers at the PCN ingress and egress when some resource signalling protocol is used. We've called this "deployment models" in Section 2.2 of http://www.ietf.org/internet-drafts/draft-babiarz-pcn-3sm-01.txt Regards, Michael > Regarding the latter issue: I have a strong opinion. I think this needs > more discussion by the working group. > > > Regards, > > =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= > Steven Blake > Ericsson/Redback Networks +1 919-472-9913 > > > > _______________________________________________ > PCN mailing list > PCN@ietf.org > https://www1.ietf.org/mailman/listinfo/pcn > -- Dr. Michael Menth, Assistant Professor University of Wuerzburg, Institute of Computer Science Am Hubland, D-97074 Wuerzburg, Germany, room B206 phone: (+49)-931/888-6644, fax: (+49)-931/888-6632 mailto:menth@informatik.uni-wuerzburg.de http://www3.informatik.uni-wuerzburg.de/research/ngn _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Thu Nov 29 03:02:41 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IxeM4-0000Gw-Q3; Thu, 29 Nov 2007 03:02:40 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IxeM3-0000Gg-IP for pcn-confirm+ok@megatron.ietf.org; Thu, 29 Nov 2007 03:02:39 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IxeLx-0008Rw-1I for pcn@ietf.org; Thu, 29 Nov 2007 03:02:33 -0500 Received: from tcmail31.telekom.de ([217.6.95.238]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IxeLw-00037s-Es for pcn@ietf.org; Thu, 29 Nov 2007 03:02:32 -0500 Received: from s4de8psaans.mitte.t-com.de by tcmail31.telekom.de with ESMTP; Thu, 29 Nov 2007 09:02:28 +0100 Received: from S4DE8PSAANK.mitte.t-com.de ([10.151.229.10]) by s4de8psaans.mitte.t-com.de with Microsoft SMTPSVC(6.0.3790.3959); Thu, 29 Nov 2007 09:02:27 +0100 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: quoted-printable Subject: Re: [PCN] Re: ECN support in a PCN domain Date: Thu, 29 Nov 2007 09:02:27 +0100 Message-Id: <1B6169C658325341A3B8066E23919E1C4C156E@S4DE8PSAANK.mitte.t-com.de> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PCN] Re: ECN support in a PCN domain Thread-Index: Acgx6kWteh75RR01S9av++qixthhIwAcZ2Sw From: "Geib, Ruediger" To: X-OriginalArrivalTime: 29 Nov 2007 08:02:27.0824 (UTC) FILETIME=[2F146700:01C8325E] X-Spam-Score: 0.0 (/) X-Scan-Signature: 0a7aa2e6e558383d84476dc338324fab Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Lars, my own concern about the issue was weaker (behaviour of an ingress node=20 in the case of a possible misuse of PCN colouring from an external=20 interface).=20 But I agree, the congestion feedback received by an end node is an=20 interesting feature and if the WG chairs and the AD feel it to be=20 important, we should work on it. The charter says, PCN aware=20 application mechanism and flow adaptation are out of scope. Could you point out the use case for the legal ECN information received=20 by the ingress node, if it is not the latter two? Regards, Rudiger -----Original Message----- From: Lars Eggert [mailto:lars.eggert@nokia.com] Sent: Wednesday, November 28, 2007 7:12 PM To: ext pcn-bounces@ietf.org Cc: pcn@ietf.org Subject: Re: [PCN] Re: ECN support in a PCN domain On 2007-11-28, at 19:49, ext pcn-bounces@ietf.org wrote: > there was quite a lot of discussion (without resolution) whether it's > reasonable for a flow that uses PCN adm ctrl to also want e2e ECN for > rate adaptation (discussion both in terms of whether a flow would > actually ever want this, and how well the PCN mechanism would work if > flows were adapting like this). > this was resolved by saying that the support of e2e ecn is out of =20 > scope > of initial charter of pcn. Lars wrote: Huh? The charter says: All PCN mechanisms, including transport and encoding of (pre-congestion) information, are required to cleanly integrate with existing architectures and protocols such as DiffServ and ECN. "Cleanly integrate" doesn't mean "out of scope", it means "must work =20 together with". TSVWG published RFC4774 to describe which alternate uses of the ECN =20 bits are OK; the PCN mechanisms are supposed to instantiate such an =20 accepted alternate use. Lars _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Fri Nov 30 05:53:42 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iy3V7-0000TL-Gx; Fri, 30 Nov 2007 05:53:41 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1Iy3V5-0000Sh-Ql for pcn-confirm+ok@megatron.ietf.org; Fri, 30 Nov 2007 05:53:40 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1Iy3V0-0000HA-95 for pcn@ietf.org; Fri, 30 Nov 2007 05:53:34 -0500 Received: from rsys002x.roke.co.uk ([193.118.201.109]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1Iy3Uy-0000PT-87 for pcn@ietf.org; Fri, 30 Nov 2007 05:53:34 -0500 Received: from rsys005a.comm.ad.roke.co.uk (rsys005a [193.118.193.85]) by rsys002x.roke.co.uk (8.13.1/8.13.1) with ESMTP id lAUArIbi018362; Fri, 30 Nov 2007 10:53:18 GMT MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Office Outlook 11 X-MimeOLE: Produced By Microsoft Exchange V6.5 x-olkeid: 1744AA31FCBCCB6564C09741A47BAB349649973C Content-class: urn:content-classes:message Date: Fri, 30 Nov 2007 10:53:18 -0000 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: some comments on pcn-architecture-02 Thread-Index: AcgzPyyK4uKTmDDHS5usTBXBuUnxhw== From: "Hancock, Robert" To: X-MailScanner-roke-co-uk: Found to be clean X-MailScanner-roke-co-uk-SpamCheck: X-MailScanner-From: robert.hancock@roke.co.uk X-Spam-Score: -1.0 (-) X-Scan-Signature: 6e922792024732fb1bb6f346e63517e4 Cc: pcn@ietf.org Subject: [PCN] some comments on pcn-architecture-02 X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org Hi all,=20 Some comments on the architecture draft, mainly (but not totally) stimulated by the ECN discussion.=20 1. The draft is mainly about what building blocks are needed to construct a PCN domain. That's fair enough; however, it would also be nice to have some overview of how a PCN domain looks (as a "black network") to the outside world. If I'm primarily interested in terminals or edge networks then I don't really care about how ingress and egress nodes communicate, or OAM, or internal encoding issues, but I do care about how the introduction of PCN will affect my traffic. This discussion would be short if the PCN domain was transparent in which case that's all that needs to be said. However, if for example the domain blocks packets with particular bit patterns in particular circumstances, I'd like to know what those circumstances are, and what - if anything - I can do about them (and to have that information conveniently centralised). 2. The draft uses the term 'PCN-flow' several times without defining it. I imagine that a PCN-flow could be defined as just a subset of PCN-traffic, but I'm not clear how arbitrary those subsets could be. (Does a subset have to have a single ingress/egress? Can subsets nest/overlap? Do all the ingress/egress nodes have to agree on what the subsets are?) It may be that a PCN-flow is just one of - whatever a boundary node decides to admit=20 - whatever a boundary node decides to terminate=20 and that the granularity of those decisions can be absolutely whatever one wants (but again it would be nice to say so). 3. It's clear that the ingress nodes are responsible for deciding whether or not an incoming packet is a PCN packet, and that the criteria for the decision are a header classifier. However, it's rather unclear what needs to be assumed about how these classifiers get installed. It may be that for most of the details of PCN operation, how the classifiers get installed just doesn't matter, but from the point of view of the external user or peer network it could be critical. (How does it work if a domain is working without admission control, as in section 4.3? Will an operator provision the classifiers statically? Or, if there is rerouting which causes the ingress for a flow to change, whose job is it to move the classifiers around? And so on.) 4. There are a few comments in the draft (and elsewhere) about PCN being intended for inelastic traffic. I think it needs to be really clear what the significance of this assumption is. If the claim is "we know PCN will only be used for inelastic traffic therefore we can make a simplifying assumption about how to implement it" I think that's very dangerous. If the claim is "we only have to provide a benefit for inelastic traffic and nothing else sees any difference" that's quite safe. (To some extent this is tied up with point 3: if you stated explicitly how the PCN domain decided whether traffic was PCN traffic, then it would be clear to what extent the (in-)elasticity of the admitted traffic was known and controlled. Without specifying the answer to (3), it isn't clear how the PCN domain can enforce the assumption or monitor whether the assumption is being violated.) 5. It's clear that the assumption in 3.5 is driven by encoding considerations. However, what is really unclear (to the naive reader such as myself) is why CE is problematic, but ECT(0/1) are not, since they all use the same IP header bits. If a packet in a PCN flow arrives with ECT set, does that pass through the domain transparently? (It would also be reassuring to state explicitly that ECN is transparent for non-PCN-flows.) 6. On the specific point about what to do with PCN flows using ECN, I can just about follow the logic - if CE was set, then it is possible that it was set by a router using the default ECN semantics - a router using the default ECN semantics should be able to assume that the receiver would react in a similar way as to a packet drop - therefore the PCN domain can drop the packet However, it's quite a long journey from the beginning of the argument to the end. In particular, for an endpoint, receiving a packet with CE is quite different from not receiving a packet at all. For one thing, it means the receiver gets data. It also gets a positive signal of congestion (rather than, for example, wondering whether it was a congestive or corruptive packet loss). In general, I think it's really important not to do things which act as deployment disincentives for ECN, or any additional barriers to alternative semantics, since e2e packet marking is so useful for congestion control in some environments (multi-hop wireless in particular).=20 If the desire is to have a simple solution for PCN which uses the ECN bits for encoding, there might be an alternative: if you detect that a PCN-flow is using ECN (because ECT is set on any of its packets), simply reclassify that flow as non-PCN and let it pass transparently through the domain as non-PCN (e.g. best efforts) traffic. In performance terms the flow is getting no worse service than in a pre-PCN world. If you really think that there are no use cases for combined ECN/PCN this situation will never arise. If applications are developed in the future which can exploit ECN for semi-elastic flows, then it becomes a question of writing guidelines for application developers to help them either interpret signalling protocols (to find out whether their success in admission control is being harmed by their use of ECN) or split their flows into elastic and inelastic sub-flows, or indeed work out a PCN solution which didn't use the ECN bits internally. All of these would obviously be a later stage of the work. cheers & have fun in vancouver, robert h. _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Fri Nov 30 14:26:47 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IyBVf-0000k9-EF; Fri, 30 Nov 2007 14:26:47 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IyBVe-0000gj-Id for pcn-confirm+ok@megatron.ietf.org; Fri, 30 Nov 2007 14:26:46 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IyBVe-0000g9-7R for pcn@ietf.org; Fri, 30 Nov 2007 14:26:46 -0500 Received: from zcars04f.nortel.com ([47.129.242.57]) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IyBVd-0004vu-K9 for pcn@ietf.org; Fri, 30 Nov 2007 14:26:46 -0500 Received: from zrtphxm1.corp.nortel.com (zrtphxm1.corp.nortel.com [47.140.202.50]) by zcars04f.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id lAUJQhU08169 for ; Fri, 30 Nov 2007 19:26:43 GMT Received: from KCHAN-2K3.nortel.com ([47.16.54.161] RDNS failed) by zrtphxm1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.1830); Fri, 30 Nov 2007 14:26:08 -0500 X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Fri, 30 Nov 2007 14:26:41 -0500 To: pcn@ietf.org From: "Kwok-Ho Chan" Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Message-ID: X-OriginalArrivalTime: 30 Nov 2007 19:26:08.0632 (UTC) FILETIME=[DBCCB780:01C83386] X-Spam-Score: 0.0 (/) X-Scan-Signature: 5ebbf074524e58e662bc8209a6235027 Subject: [PCN] PCN Encoding Comparison Draft Open Issues X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org The current open issues for: http://www.ietf.org/internet-drafts/draft-chan-pcn-encoding-comparison-01.txt are documented at: http://standards.nortelnetworks.com/pcn/PCN_Encoding_Comparison_Draft_OpenIssues.txt This file will be updated based on discussions and resolutions of these and additional open issues for the draft. Please review the draft and the issues list. E-Mail list discussions on the open issues should have: "EC Issue 03.c" as the start of the E-Mail subject field for "PCN Encoding Comparison Draft Issue 3 Sub-Issue a" to help multiple drafts, multiple issues, discussion management. Please let me know if you want any changes/corrections/additions to this file. For example: the creation of a new issue, or sub-issue of an existing issue. I have the content of the Issues List cut-and-pasted below in this E-Mail. But will be using the file as the master copy going forward. Thank you very much for your time and attention to this work. -- Kwok -- Content of the PCN Encoding Comparison Draft Issues List file: Issues List for PCN Encoding Comparison Draft ============================================= Updated: Nov 28, 2007 Editor: Kwok Ho Chan Draft: http://www.ietf.org/internet-drafts/draft-chan-pcn-encoding-comparison-01.txt For further discussions, please use notations of: - EComp03.c is used to reference Encoding Comparison draft Issue "03.c". - "chan-01" is used to reference version -01 of the personal draft. Issue #1: ECN Support in a PCN Domain ===================================== The resolution of this issue will clarify how a PCN Domain needs to treat ECN (RFC3168) packets, in terms of Interior Nodes and Edge Nodes. This issue arise with discussion on the Architecture draft, but this will definitely impact the PCN Encoding Comparison draft, with possible impact on the algorithm discussions. The resolution of this issue will also solidify the definition of a PCN Domain. The starting point will be: Does the use of DiffServ sufficiently define the logical boundaries of a PCN Domain? Logical because the same physical network device (router,switch,etc) may support a PCN Domain as well as some other non-PCN domains. Issue #2: Deletion of the Out of Band Channel Section ===================================================== It have been proposed to remove chan-01 section 3.4 Out-of-Band Channel as Encoding Transport. This is considered to be out of scope for the WG. Would like some agreements on the list before removal. Issue #3: Encoding States ========================= The Encoding States indicated in chan-01 section 2 Encoding Requirements needs to be better defined and clarified. More specifically: a. Removing PCN Capable Transport Marking (Non-PCN Capable Transport Marking) as a notion of Encoding State, just indicate the need for this separation as a requirment (possibly in the Architecture). b. Better explanation/indication of Single Marking for both Admission Marking and Termination Marking c. Possible additional encoding choices when DSCP+ECN fields are used. d. Which Encoding States should the WG require? Should there be optional Encoding States? And their deployment options. e. Is(are) "nounce" encoding state(s) needed? Issue #4: Encoding Option Evaluation Criteria ============================================= Need to add a section that indicates and describes the encoding option evaluation criteria. May want to add OAM impacts/considerations here or in some new section. Issue #5: Co-existence of PCN and ECN without use of tunnel =========================================================== Need to determine if this should be addressed by this draft. Issue #6: Use of PCN by multiple DiffServ Classes ================================================= This can be added to the Intro section of this draft. But this discussion may be more appropriate in the Arch draft. Issue #7: ECMP considerations for PCN Encoding ============================================== This should be one of the Encoding Option Evaluation Criteria. But may also want to add new subsections under each of the encoding choices on how they address this (and other) encoding option evaluation criterion. Issue #8: Document organization and text cleanup/addition ========================================================= There are number of open issues relating to possible document organization changes and addition of new text (or clean up of existing text). One of them is expansion on tunneling's impact on encoding options. Fixes to typo should also be part of this. _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Fri Nov 30 14:46:48 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IyBp0-0001Ks-HN; Fri, 30 Nov 2007 14:46:46 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IyBoz-0001JV-2L for pcn-confirm+ok@megatron.ietf.org; Fri, 30 Nov 2007 14:46:45 -0500 Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IyBoy-0001JH-OZ for pcn@ietf.org; Fri, 30 Nov 2007 14:46:44 -0500 Received: from zcars04f.nortel.com ([47.129.242.57]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1IyBox-0002uG-Rh for pcn@ietf.org; Fri, 30 Nov 2007 14:46:44 -0500 Received: from zrtphxm1.corp.nortel.com (zrtphxm1.corp.nortel.com [47.140.202.50]) by zcars04f.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id lAUJkfU11256 for ; Fri, 30 Nov 2007 19:46:42 GMT Received: from KCHAN-2K3.nortel.com ([47.16.54.161] RDNS failed) by zrtphxm1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.1830); Fri, 30 Nov 2007 14:45:35 -0500 X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Fri, 30 Nov 2007 14:46:09 -0500 To: pcn@ietf.org From: "Kwok-Ho Chan" Subject: Re: [PCN] PCN Encoding Comparison Draft Open Issues In-Reply-To: References: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Message-ID: X-OriginalArrivalTime: 30 Nov 2007 19:45:35.0320 (UTC) FILETIME=[93332D80:01C83389] X-Spam-Score: -4.0 (----) X-Scan-Signature: 2beba50d0fcdeee5f091c59f204d4365 X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org If you have problem with the link to the Issues List, You can use the link: http://standards.nortel.com/pcn/PCN_Encoding_Comparison_Draft_OpenIssues.txt Sorry for any inconvenience. -- Kwok -- At 02:26 PM 11/30/2007, you wrote: >The current open issues for: > >http://www.ietf.org/internet-drafts/draft-chan-pcn-encoding-comparison-01.txt >are documented at: > >http://standards.nortelnetworks.com/pcn/PCN_Encoding_Comparison_Draft_OpenIssues.txt > >This file will be updated based on discussions and resolutions of >these and additional >open issues for the draft. > >Please review the draft and the issues list. > >E-Mail list discussions on the open issues should have: >"EC Issue 03.c" as the start of the E-Mail subject field for >"PCN Encoding Comparison Draft Issue 3 Sub-Issue a" >to help multiple drafts, multiple issues, discussion management. > >Please let me know if you want any changes/corrections/additions to this file. >For example: the creation of a new issue, or sub-issue of an existing issue. > >I have the content of the Issues List cut-and-pasted below in this E-Mail. >But will be using the file as the master copy going forward. > >Thank you very much for your time and attention to this work. >-- Kwok -- > > >Content of the PCN Encoding Comparison Draft Issues List file: > >Issues List for PCN Encoding Comparison Draft >============================================= >Updated: Nov 28, 2007 >Editor: Kwok Ho Chan >Draft: >http://www.ietf.org/internet-drafts/draft-chan-pcn-encoding-comparison-01.txt > >For further discussions, please use notations of: >- EComp03.c is used to reference Encoding Comparison draft Issue "03.c". >- "chan-01" is used to reference version -01 of the personal draft. > > >Issue #1: ECN Support in a PCN Domain >===================================== >The resolution of this issue will clarify how a PCN Domain needs >to treat ECN (RFC3168) packets, in terms of Interior Nodes and >Edge Nodes. This issue arise with discussion on the Architecture >draft, but this will definitely impact the PCN Encoding Comparison >draft, with possible impact on the algorithm discussions. >The resolution of this issue will also solidify the definition of >a PCN Domain. >The starting point will be: Does the use of DiffServ sufficiently >define the logical boundaries of a PCN Domain? Logical because >the same physical network device (router,switch,etc) may support >a PCN Domain as well as some other non-PCN domains. > > >Issue #2: Deletion of the Out of Band Channel Section >===================================================== >It have been proposed to remove chan-01 section 3.4 Out-of-Band >Channel as Encoding Transport. This is considered to be out of scope >for the WG. Would like some agreements on the list before removal. > > >Issue #3: Encoding States >========================= >The Encoding States indicated in chan-01 section 2 Encoding Requirements >needs to be better defined and clarified. >More specifically: >a. Removing PCN Capable Transport Marking (Non-PCN Capable Transport > Marking) as a notion of Encoding State, just indicate the need > for this separation as a requirment (possibly in the Architecture). >b. Better explanation/indication of Single Marking for both Admission > Marking and Termination Marking >c. Possible additional encoding choices when DSCP+ECN fields are used. >d. Which Encoding States should the WG require? > Should there be optional Encoding States? > And their deployment options. >e. Is(are) "nounce" encoding state(s) needed? > > >Issue #4: Encoding Option Evaluation Criteria >============================================= >Need to add a section that indicates and describes the encoding option >evaluation criteria. May want to add OAM impacts/considerations here or in >some new section. > > >Issue #5: Co-existence of PCN and ECN without use of tunnel >=========================================================== >Need to determine if this should be addressed by this draft. > > >Issue #6: Use of PCN by multiple DiffServ Classes >================================================= >This can be added to the Intro section of this draft. >But this discussion may be more appropriate in the Arch draft. > > >Issue #7: ECMP considerations for PCN Encoding >============================================== >This should be one of the Encoding Option Evaluation Criteria. >But may also want to add new subsections under each of the >encoding choices on how they address this (and other) encoding >option evaluation criterion. > > >Issue #8: Document organization and text cleanup/addition >========================================================= >There are number of open issues relating to possible document >organization changes and addition of new text (or clean up of >existing text). One of them is expansion on tunneling's impact >on encoding options. Fixes to typo should also be part of this. > > > > >_______________________________________________ >PCN mailing list >PCN@ietf.org >https://www1.ietf.org/mailman/listinfo/pcn _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn From pcn-bounces@ietf.org Fri Nov 30 18:33:36 2007 Return-path: Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1IyFMV-0004wE-VQ; Fri, 30 Nov 2007 18:33:35 -0500 Received: from pcn by megatron.ietf.org with local (Exim 4.43) id 1IyFMU-0004vu-Qf for pcn-confirm+ok@megatron.ietf.org; Fri, 30 Nov 2007 18:33:34 -0500 Received: from [10.90.34.44] (helo=chiedprmail1.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1IyFMU-0004vj-EW for pcn@ietf.org; Fri, 30 Nov 2007 18:33:34 -0500 Received: from smtp.nokia.com ([131.228.20.172] helo=mgw-ext13.nokia.com) by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1IyFMT-0007qy-T7 for pcn@ietf.org; Fri, 30 Nov 2007 18:33:34 -0500 Received: from esebh108.NOE.Nokia.com (esebh108.ntc.nokia.com [172.21.143.145]) by mgw-ext13.nokia.com (Switch-3.2.5/Switch-3.2.5) with ESMTP id lAUNXVa5016065; Sat, 1 Dec 2007 01:33:31 +0200 Received: from esebh103.NOE.Nokia.com ([172.21.143.33]) by esebh108.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.1830); Sat, 1 Dec 2007 01:33:31 +0200 Received: from mgw-int01.ntc.nokia.com ([172.21.143.96]) by esebh103.NOE.Nokia.com over TLS secured channel with Microsoft SMTPSVC(6.0.3790.1830); Sat, 1 Dec 2007 01:33:30 +0200 Received: from [192.168.101.247] (daec-linuxvpn05858.americas.nokia.com [10.241.58.58]) by mgw-int01.ntc.nokia.com (Switch-3.2.5/Switch-3.2.5) with ESMTP id lAUNXR8Q008354; Sat, 1 Dec 2007 01:33:28 +0200 Message-Id: From: Lars Eggert To: "ext Geib, Ruediger" In-Reply-To: <1B6169C658325341A3B8066E23919E1C4C156E@S4DE8PSAANK.mitte.t-com.de> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v915) Subject: Re: [PCN] Re: ECN support in a PCN domain Date: Fri, 30 Nov 2007 15:33:26 -0800 References: <1B6169C658325341A3B8066E23919E1C4C156E@S4DE8PSAANK.mitte.t-com.de> X-Mailer: Apple Mail (2.915) X-OriginalArrivalTime: 30 Nov 2007 23:33:30.0736 (UTC) FILETIME=[6A620F00:01C833A9] X-Nokia-AV: Clean X-Spam-Score: 0.0 (/) X-Scan-Signature: 93238566e09e6e262849b4f805833007 Cc: pcn@ietf.org X-BeenThere: pcn@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: PCN WG list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pcn-bounces@ietf.org On 2007-11-29, at 0:02, ext Geib, Ruediger wrote: > But I agree, the congestion feedback received by an end node is an > interesting feature and if the WG chairs and the AD feel it to be > important, we should work on it. The charter says, PCN aware > application mechanism and flow adaptation are out of scope. Could > you point out the use case for the legal ECN information received > by the ingress node, if it is not the latter two? Most people think of ECN only in conjunction with TCP, because that's the most common use. But ECN can be used with any transmission scheme that reacts to loss as a signal. For example, a CBR sender might react to loss by suspending transmission. This scheme can be improved through ECN, so that the sender would suspend transmission when ECN marks are received, i.e., before loss occurs. This is why I think that allowing end-to-end ECN to happen in conjunction with PCN is important. (I saw the idea in Robert's recent email of "downgrading" ECN-enabled flows into the best-effort class of PCN. I need to think more about this, but on first glance this looks like a pretty elegant resolution. A flow that is ECN-enabled has basically announced to the network that it will react to loss in some way, i.e., probably has some form of congestion control. But I need to think about this more.) Lars _______________________________________________ PCN mailing list PCN@ietf.org https://www1.ietf.org/mailman/listinfo/pcn