Subject: Re: [spring] draft-ietf-spring-conflict-resolution - Preference Rule

From: Marc Binderberger <[email-masked]>
Date: Mon, 26 Sep 2016 10:17:17 -0700

Hello Bruno,

thanks for the detailed reply and apology for my late response.

First, let me re-phrase what I propose. Realized that I never explained 
exactly what I had in mind and using the "ignore" keyword was just a bad idea 
too. Will avoid "ignore" as this keyword has been used in different ways 
already.

So what I propose is: if a prefix has conflicting mapping information - 
prefix or SID conflict - then keep this prefix unlabeled. [1]

I don't see that draft-ietf-spring-conflict-resolution is exactly covering 
this idea.

Then:

> But I'm not sure that you are really calling for the network operator to 
> have this flexibility.

I actually do expect requests from network operators to implement their 
specific view on how to operate the network into the conflict algorithm. The 
protocol design should allow for it.

But there are also network operators who prefer to keep the network 
relatively "dumb" and implement their operational logic into their OSS 
systems. I don't see why vendors would want to impose the more complex logic 
on customers not asking or not wanting it. Which is why I argue against the 
idea to make the preference algorithm "the" standard.

(yes, "simple" or "dumb" allows for interpretation. I'm sure you get the 
point though)

Saying all this, of course we should support the request from Orange and 
others for the preference algorithm. We do have the "algorithm" field, e.g. 
draft-ietf-isis-segment-routing-extensions, that could carry the 
SPF+conflict+... algorithm required.

There was also a misunderstanding about my "solving a problem that should be 
rare" statement. I said it with a qualifier "with the quoted rule above" and 
the rule was

   "Local configuration conflicts can be prevented before they are advertised"

If implementations check for a conflict between what they want to advertise 
and what is already advertised by others in the network then we filter out 
conflicts from config change that is happening after a time window of e.g. 
several seconds since the last SR config change in the network. Not saying 
this is a fool-proof algorithm but it puts a perspective on the problem.

> Basically, what I propose is to keep it safe, and not introduce new risks

sorry but I disagree. I cannot name customers but there are network operators 
who consider code complexity that they don't need a risk.

To summarize: I see your point for the preference algorithm but I disagree 
that this should be "the" standard behaviour of Segment Routing 
implementations. Rather we should allow for different algorithm. To have 
basic interoperability I propose the algorithm [1] above as a starting point. 
Other algorithm can be defined within IETF if network operators are 
interested in - we obviously have one algorithm with interest already :-)

Best regards,
Marc

On Tue, 30 Aug 2016 17:08:24 +0000, [email-masked] wrote:
> Hello Marc,
> 
> Speaking as an individual contributor, and network operator.
> 
> Please see inline.
> 
>> From: Marc Binderberger [mailto:[email-masked]] > Sent: Tuesday, August 09, 
>> 2016 9:56 AM
>> 
>> Hello Bruno and spring list experts,
>> 
>> jumping late into the discussion (sorry).
> 
> Thanks for the feedback.
>  
>> 
>> I would see any kind of preference list to handle a conflict to be an
>> somewhat arbitrary decision.
> 
> Agreed.
> Stéphane has proposed the addition of a "preference" field attached to 
> mapping server (advertisements) to allow a network operator to control this 
> preference decision, depending on SP rules or current network status (e.g. 
> migrations)
> To increase this capacity for customization, I guess that an option would 
> be to extend this preference to other SID advertisements.
> But I'm not sure that you are really calling for the network operator to 
> have this flexibility. So I don't think that this comment is really the 
> point that you want to make.
> 
>> That is okay for a particular network (assuming
>> the network operator had a word or two about the rules) but I don't 
>> consider
>> this a good idea for a standard to reflect specific designs or
>> operational/troubleshooting procedures.
>> 
>> E.g. who is saying that IPv6 should be preferred over IPv4?
> 
> The working group.
> You are welcome to contribute. IIRC, this specific point has already been 
> changed based on WG feedback.
> 
>> What when the
>> operator's workhorse is still IPv4 traffic and s/he would prefer to 
>> sacrifice
>> IPv6 to keep IPv4 running?
>> 
>> 
>> In reality, don't we expect the rule ...
>> 
>>    "Local configuration conflicts can be prevented before they are 
>> advertised"
>> 
>> ... to cover the large amount of mistakes? 
> 
> I'm not sure to see why.
> A priori, I would expect that the more pre-existing configuration/states, 
> the more chance to conflict. Given that the network wide 
> states/configurations is greater (by at least 1 or 2 order of magnitude) 
> compared to the state/configuration of one device, I'd would tend to expect 
> the opposite. (i.e. there is more chance to conflict with 100s nodes than 
> with itself)
> 
> 
>> Then the basic "ignore" policy
>> would cover the few cases where things went really bad. 
> 
> The basic "ignore policy" would disable the traffic to 100s of PE in case 
> of a single SID conflict. I would not call this "cover the few cases where 
> things went really bad". I would call this create a very bad situation, 
> from one single misconfiguration.
> 
>> We seem to spend a
>> lot of time and energy on solving a problem that should be rare (with the
>> quoted rule above).
> 
> In addition, to the above discussion regarding "large amount of mistake 
> covered" / "few remaining cases" / "rare" which I don't see the basis for, 
> could you state how often, in your personal opinion, is it ok to drop the 
> traffic to 100s of PE, in a multi-service network carrying Internet, TV, 
> Mobile, Enterprise VPN services?
> Just as a hint, how do you propose to call the people needed to resolve the 
> issue, whether internal to the company, or outsourced or at the vendor? 
> What about emergency calls which won't be honored?
> Then, how often do you expect misconfiguration? (or bug, if you are on the 
> implementation side, even though I would expect that worldwide, across all 
> vendors and networks, configurations change are more frequent than code 
> chance, hence more test effort is given to code change rather than 
> configuration change)
> 
> Most network operator, pay twice to get link and node redundancy, in order 
> to protect important communications from a single link or node failure. 
> While with the "Ignore policy" one single error have the ability to shut 
> down the whole network or a significant part of it.
> 
> Indeed, in risk analysis, probability of occurrence is one important point 
> to consider. But consequences/cost is another important point to consider. 
> And in the old telco world, some level of consequences may not be 
> considered acceptable. e.g. there is usually a limit defined for the number 
> of people which may be affected by a single failure. When this limit is 
> reached, another equipment is deployed to service those additional people, 
> even though the first equipment had enough capacity. 
> 
>> Basically what I propose is to keep it simple.
> 
> Hum... "Everything should be made as simple as possible, but no simpler." 
> Now the problem is to draw the line. (and that's not simple)
> And there are multiple metrics to consider. Implementation (simplicity) is 
> one, network operation (simplicity) is another one. And I would note that 
> the number of networks is significantly higher than the number of 
> implementations so it may be advantageous to prefer the network operation 
> metric over the implementation metric.
> 
> Basically, what I propose is to keep it safe, and not introduce new risks, 
> especially with massive consequences, compared to existing signaling 
> solutions (e.g. LDP or RSVP-TE) which may be seen as a show stopper for 
> some person (before or after the failure, depending on their anticipations).
> 
>> At least start simple. _If_
>> network operators find during operation that the basic 
>> don't-advertise/ignore
>> rule is insufficient then we can still increase the complexity of the
>> conflict handling procedure (and add a config knob to select the new
>> procedure).
> 
> I believe that we are in the process of getting network operator feedback.
> 
> Regards,
> Bruno
>  
>> 
>> Regards, Marc
>> 
>> 
>> 
>> 
>> On Thu, 28 Jul 2016 12:36:01 +0000, [email-masked] wrote:
>>> Hi Les, all
>>> 
>>> As an individual contributor.
>>> 
>>> As a network operator, I have a slight preference for the older preference
>>> rule, and more specifically for the following preference rule:
>>> 1) PFX source wins over SRMS source
>>> 2) Between redundant SRMS, operator defined preference (aka weight)
>>> 
>>> Note however, that for me, this is a lighter preference compared to the
>>> choice of the policy. Besides, my above preference assumes that the policy
>>> "Per FEC/Ignore overlap only" be selected. If "Quarantine" were selected, 
>>> I
>>> would have a strong preference for the revised preference rule (Larger
>>> range wins) in order to limit the consequences on the network 
>>> availability.
>>> 
>>> Regarding 1,
>>> I would assume that before the conflicting advertisement, the network was
>>> running fine. i.e. conflict entries is not the nominal behavior in the
>>> network, and conflict are detected and reported to the network operator 
>>> for
>>> correction. (e.g. via the yang model, syslog, error message on the 
>>> terminal
>>> (hence in particular the one configuring the conflicting entry...).
>>> With such assumption, the conflict is likely the result of a
>>> misconfiguration on one node. Preferring PFX source over SRMS give
>>> preference to diversity/the majority of the nodes rather than the
>>> individual (mapping server). In this assumption where a single node is
>>> misconfigured, preference many advertisement over a single one, maximize
>>> the number of valid advertisement kept. I agree that this is dependent on
>>> the assumption, and another scenario could be that one had mis-program the
>>> script configuring the prefix SID on N routers, which would results in N
>>> simultaneous misconfigurations.
>>> Additionally, following the principle that the one speaking for himself is
>>> probably the best source, I'd be inclined to trust the originator of the 
>>> IP
>>> prefix, as the reference for the SID to be used.
>>> 
>>> Regarding 2,
>>> Some network operation people have expressed a need to control which
>>> advertisement is preferred, especially to control SID renumbering  (e.g. 
>>> in
>>> case of network merge). cf Stéphane email. Putting this preference lower
>>> (e.g. after preferring the larger range) would somewhat defeat the goal or
>>> make it less predictable for people.
>>> 
>>> Regards,
>>> --Bruno
>>> 
>>>> -----Original Message-----
>>>> From: spring [mailto:[email-masked]] On Behalf Of Martin 
>>>> Horneffer
>>>> Sent: Friday, July 22, 2016 10:59 AM
>>>> To: Les Ginsberg (ginsberg); [email-masked]
>>>> Cc: Horneffer, Martin
>>>> Subject: [spring] draft-ietf-spring-conflict-resolution - Preference Rule
>>>> 
>>>> Hi Les,
>>>> 
>>>> this topic, and this document is in my eyes a very important one. Thanks
>>>> a lot for writing and promoting it!
>>>> 
>>>> During the Berlin WG session you proposed a new preference rule which
>>>> would make the policy choice easier. You asked for a discussion on the
>>>> list - more on your slides rather than the existing draft document.
>>>> 
>>>> As an operator, and as an individual that has insight in more than just
>>>> one or two IP/MPLS carrier networks, that has the main engineering
>>>> responsibility for a rather large backbone, and that stays in actual
>>>> contact with the operational staff and security authorities, I strongly
>>>> ask you: PLEASE DO NOT CHANGE THE PREFERENCE RULE!
>>>> 
>>>> The first two elements of the preference rule are, in my eyes, the most
>>>> important ones of the whole document and must not be changed or dropped!
>>>>   1) PFX source wins over SRMS soucre
>>>>   2) Smaller range wins
>>>> 
>>>> Why is this so important?
>>>> 
>>>> I don't care so much about the _amount_ of traffic that would be
>>>> affected by a conflict. No amount of traffic lost due to a network
>>>> design or configuration error is permissible. But I do care about the
>>>> overall _robustness_ and _security_ of the network.
>>>> 
>>>> Of course - in terms of security a first approximation would say that
>>>> segment routing plays within the IGP only, and that the IGP needs to be
>>>> trusted anyways. It must be secured against the outside. While this is
>>>> true, I nevertheless would like to differentiate a bit more.
>>>> 
>>>> For the sake of robustness, and possibly also for security, I would like
>>>> to apply the following guidelines:
>>>>   a) Effects of local misconfiguration should be as local as possible.
>>>>   b) The more reliable and controllable source should win over a less
>>>> reliable or controllable one.
>>>> 
>>>> As I see it, both guidelines lead to a clear preference of PFX sources
>>>> over SRMS sources. Also the preference for smaller ranges seems to fit.
>>>> 
>>>> Please do consider environments where more and more formely separate
>>>> IP/MPLS networks get merged into a single IGP domain. I am seeing this a
>>>> lot since a couple of years - several times within DT, but also at other
>>>> carriers. Sometimes this is done as a complete merge e.g. into a single
>>>> IS-IS area, sometimes different areas are used, and sometimes seperate
>>>> IGP instances are maintained but connected. While redistributing from
>>>> one IGP area or instance to the other you can do more or less filtering,
>>>> but it definitely is being done. Thus, even within the IGP filters and
>>>> policies are being applied - be it for the sake of security or
>>>> scalability. While there are well-known mechanisms and tools to filter
>>>> and control prefix redistribution, I am not so sure about SRMS.
>>>> 
>>>> 
>>>> I'm going to also write my opinion about the policy selection, but
>>>> keeping the preference rule really is my main concern.
>>>> 
>>>> 
>>>> BR,
>>>> Martin
>>>> 
>>>> _______________________________________________
>>>> spring mailing list
>>>> [email-masked]
>>>> https://www.ietf.org/mailman/listinfo/spring
>>> 
>>> 
>> __________________________________________________________________________
>> _______________________________________________
>>> 
>>> Ce message et ses pieces jointes peuvent contenir des informations
>>> confidentielles ou privilegiees et ne doivent donc
>>> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez 
>>> recu
>>> ce message par erreur, veuillez le signaler
>>> a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
>>> electroniques etant susceptibles d'alteration,
>>> Orange decline toute responsabilite si ce message a ete altere, deforme ou
>>> falsifie. Merci.
>>> 
>>> This message and its attachments may contain confidential or privileged
>>> information that may be protected by law;
>>> they should not be distributed, used or copied without authorisation.
>>> If you have received this email in error, please notify the sender and
>>> delete this message and its attachments.
>>> As emails may be altered, Orange is not liable for messages that have been
>>> modified, changed or falsified.
>>> Thank you.
>>> 
>>> _______________________________________________
>>> spring mailing list
>>> [email-masked]
>>> https://www.ietf.org/mailman/listinfo/spring
>>> 
> 
> 
_________________________________________________________________________________________________________________________
> 
> Ce message et ses pieces jointes peuvent contenir des informations 
> confidentielles ou privilegiees et ne doivent donc
> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu 
> ce message par erreur, veuillez le signaler
> a l'expediteur et le detruire ainsi que les pieces jointes. Les messages 
> electroniques etant susceptibles d'alteration,
> Orange decline toute responsabilite si ce message a ete altere, deforme ou 
> falsifie. Merci.
> 
> This message and its attachments may contain confidential or privileged 
> information that may be protected by law;
> they should not be distributed, used or copied without authorisation.
> If you have received this email in error, please notify the sender and 
> delete this message and its attachments.
> As emails may be altered, Orange is not liable for messages that have been 
> modified, changed or falsified.
> Thank you.
> 
>