Subject: [spring] SPRING MPLS and Entropy Label

From: Rob Shakir <[email-masked]>
Date: Mon, 21 Jul 2014 10:38:20 -0400

Hi SPRING, draft-kini… authors,

I have a few comments on the discussion within this draft — and a quick question
on the intention around the document.

- I feel that it would be useful to provide some clarification as to where we
  expect entropy to be required for load-sharing in the draft. The current
  wording (to me) could be read to say that where Adj-SID is used, then there
  is no requirement to consider placement for EL since there will be no
  load-balancing. However, since Adj-SID can apply to a bundle (3.5.1 in
  draft-filsfils-spring-segment-routing-04), or the underlying transport for an
  IGP adjacency can be multi-path (e.g., an aggregated Ethernet link)
  this is not true unless a head-end PE can guarantee that a particular
  adjacency is made up of a transport that has only 1 path (i.e., explicitly
  requires no load-balancing). I'm not sure how the iLER would actually determine
  this, short of the definition of new per-adjacency advertisement capabilities.
  Even if it were able to, then it seems the result is very likely to be that an
  approach that requires per-hop entropy labels to be injected is going to
  guarantee label stack depth >= 3*[number of path SIDs].

  I think that the above means that the current draft under-estimates the depth
  of the stack when considering an EL per tunnel (§3.2), which might amplify
  some of the concerns raised for this option. 

- §3.4 - I am not clear how we would really implement this option. If we learn
  midpoint capabilities, and then tune our placement of the EL based on these,
  then it seems like the only viable approach is really to consider _all_
  midpoints, and tune the placement according to the "shallowest" midpoint in
  the network. This is because, whilst we might expect that there is routing via
  some particular path, this might change if there is ECMP between mid-points
  nodes where some paths have different capabilities than others, and equally
  might change whilst re-routing occurs.

  In terms of discovery of the capabilities of a mid-point -- do we need to
  consider how, in a multi-area network, we discover the capabilities of a
  mid-point which might not have information about it leaked between areas?
    
- Finally, it'd be good to determine what the intention of the authors for this
  document is. Do we expect that we make a recommendation as to what equipment
  vendors who are going to support SR-MPLS should optimise for? At the moment,
  the document is good at classifying the different approaches that _could_ be 
  taken, but potentially requires an operator/vendor to consider optimisation
  for both a) reading very deep into the stack when acting as an LSR, b) pushing
  very deep stacks when acting as iLER, c) potentially implementing more complex
  swap() operations, whereby some reshuffling of the EL is required. 

  My feeling is that it would be very useful for this document to be able to
  make a clear recommendation as to which of these approaches should be
  optimised for, and hence we should debate their technical efficacy. It'd be
  good to understand whether the authors are intending to do this, or rather
  classify the approaches.

Cheers,
r.

---

From: Rob Shakir <[email-masked]>
Date: Mon, 21 Jul 2014 10:46:45 -0400

Adding the MPLS wg to this discussion.

[Apologies for the spam.]

r.

On 21 Jul 2014, at 10:38, Rob Shakir <[email-masked]> wrote:

> _______________________________________________
> spring mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/spring

---

From: Xuxiaohu <[email-masked]>
Date: Mon, 21 Jul 2014 14:56:54 +0000

Hi Rob,

>  mid-point which might not have information about it leaked between areas?

Did you mean that when propagating the capabilities TLV/sub-TLV of a given node across areas, the information (e.g., IP address) of the originator of that capability TLV/sub-TLV is lost?

Best regards,
Xiaohu

---

From: Rob Shakir <[email-masked]>
Date: Mon, 21 Jul 2014 11:04:13 -0400

Hi Xiaohu,

> Did you mean that when propagating the capabilities TLV/sub-TLV of a given node across areas, the information (e.g., IP address) of the originator of that capability TLV/sub-TLV is lost?

I was more considering the case that the information is simply not propagated into the “downstream” area. There is no clear requirement to me that we need propagate capabilities and/or prefix SIDs for non-endpoint nodes across area boundaries, and hence we may not have visibility of each mid-point on the path.

r.

---

From: Xuxiaohu <[email-masked]>
Date: Mon, 21 Jul 2014 15:13:28 +0000

Hi Rob,

________________________________________
发件人: spring [[email-masked]] 代表 Rob Shakir [[email-masked]]
发送时间: 2014年7月21日 23:04
收件人: Xuxiaohu
抄送: [email-masked]; [email-masked]; [email-masked]
主题: Re: [spring] SPRING MPLS and Entropy Label

Hi Xiaohu,

[Xiaohu] If we need visibility of each mid-point on the path indeed, does it mean that the capability of each mid-point should be propagated across area boundaries?

Best regards
Xiaohu

r.

---

From: Rob Shakir <[email-masked]>
Date: Mon, 21 Jul 2014 11:21:37 -0400

> [Xiaohu] If we need visibility of each mid-point on the path indeed, does it mean that the capability of each mid-point should be propagated across area boundaries?

This could be one solution. One could observe that if we need to propagate mid-point information across area boundaries, then we are beginning to reduce any (perceived) benefit of running multiple areas. 

Please note, I am not trying to state the need for any solution to this issue, just note that there are challenges with such an approach to the authors of the draft. If the intention of the draft is to recommend an approach to the SPRING+EL issue, then potentially such considerations should be taken into account when considering the technical efficacy of each of the options described in the document.

Cheers,
r.

---

From: <[email-masked]>
Date: Tue, 22 Jul 2014 02:52:36 +0000

Hi Rob,

Many thanks to raise this important subject back on the table.
Entropy label in SPRING is mainly a matter of what are the current capabilities of our favorite vendor's current HW ... Unfortunately, I think we are touching something quite "secret"  and vendors would not share their issues publicly :)

I think some options may also introduce some "HW capability" issues :
- reusable EL and EL top of the stack  : need multiple MPLS operation , so it may require forwarding feedback loops on some HWs reducing the forwarding capacity

Personally, I don't like (hate :) ), the multiple EL/ELI (at each tunnel level or at specific point of insertion) because it would again reduce the available MTU for the customer jumbo frames.

IMHO, I would be in favor of doing nothing if all vendors can publicly confirm that their current generation of HW are able to inspect between 10/15 labels :) Otherwise reusable EL and even EL at top of the stack sounds good to me. EL at top of the stack may make some MPLS people crazy, but I personally found this kind of forwarding straightforward compared to reusable EL (I like it also).  EL at top of the stack may be compared to a MPLS router alert label at top of the stack ... 

We may also need to deal with cases where some SPRING nodes are not ELC on the transit path, do we want to loose the EL or just try to avoid the non ELC node(s) by reinserting ELI/EL at a position it could be reused further in the path on the next ELC capable SPRING node.

Anyway as I said at the beginning, unfortunately for us (Service Providers), the technical solution is at more than 80% a vendor discussion ...

Thanks again for bringing this back on the table :)

Best regards,

Stephane

-----Original Message-----
From: spring [mailto:[email-masked]] On Behalf Of Rob Shakir
Sent: Monday, July 21, 2014 10:38
To: [email-masked]
Cc: [email-masked]
Subject: [spring] SPRING MPLS and Entropy Label

Hi SPRING, draft-kini. authors,

I have a few comments on the discussion within this draft - and a quick question on the intention around the document.

    

Cheers,
r.

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: Rob Shakir <[email-masked]>
Date: Tue, 22 Jul 2014 10:22:44 -0400

Hi Stephane,

> Entropy label in SPRING is mainly a matter of what are the current capabilities of our favorite vendor's current HW ... Unfortunately, I think we are touching something quite "secret"  and vendors…

There are two things that we need to achieve w.r.t SR/MPLS and EL from my perspective:

1) How do we exploit EL with *today’s* hardware such that SR LSPs can be efficiently load shared?
2) Going forward, which of the approaches that this draft identifies for dealing with deep label stacks produced by SR/MPLS should we optimise for?

So, yes, I agree that there’s something around current capabilities that we need to figure out — but equally, I’d like to visit question (2), such that we can see whether the WG thinks that it is possible that we can reach a recommendation and hence future hardware revisions can potentially focus on optimising for one particular SR/MPLS + EL approach. 

> Personally, I don't like (hate :) ), the multiple EL/ELI (at each tunnel level or at specific point of insertion) because it would again reduce the available MTU for the customer jumbo frames.

ACK — this is one of the reasons that I would like to understand the author’s intentions on this draft. At the moment, the draft is a taxonomy of the approaches, but rather stops short of making any recommendations. It sounds like you’d be supportive of asking the authors to extend this draft to making a recommendation? :-)

Cheers,
r.

---

From: <[email-masked]>
Date: Tue, 22 Jul 2014 14:27:27 +0000

> It sounds like you'd be supportive of asking the authors to extend this draft to making a recommendation? :-)

Completely right :)

-----Original Message-----
From: Rob Shakir [mailto:[email-masked]] 
Sent: Tuesday, July 22, 2014 10:23
To: LITKOWSKI Stephane SCE/IBNF
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: [spring] SPRING MPLS and Entropy Label

Hi Stephane,

1) How do we exploit EL with *today's* hardware such that SR LSPs can be efficiently load shared?
2) Going forward, which of the approaches that this draft identifies for dealing with deep label stacks produced by SR/MPLS should we optimise for?

So, yes, I agree that there's something around current capabilities that we need to figure out - but equally, I'd like to visit question (2), such that we can see whether the WG thinks that it is possible that we can reach a recommendation and hence future hardware revisions can potentially focus on optimising for one particular SR/MPLS + EL approach. 

> Personally, I don't like (hate :) ), the multiple EL/ELI (at each tunnel level or at specific point of insertion) because it would again reduce the available MTU for the customer jumbo frames.

ACK - this is one of the reasons that I would like to understand the author's intentions on this draft. At the moment, the draft is a taxonomy of the approaches, but rather stops short of making any recommendations. It sounds like you'd be supportive of asking the authors to extend this draft to making a recommendation? :-)

Cheers,
r.

---

From: Xuxiaohu <[email-masked]>
Date: Tue, 22 Jul 2014 14:39:04 +0000

I personally don't believe there is any possibility of making that recommendation at present since each option is only applicalbe in a certain condition. 

Best regards,
Xiaohu
________________________________________
发件人: spring [[email-masked]] 代表 [email-masked] [[email-masked]]
发送时间: 2014年7月22日 22:27
收件人: Rob Shakir
抄送: [email-masked]; [email-masked]; [email-masked]
主题: Re: [spring] SPRING MPLS and Entropy Label

Completely right :)

-----Original Message-----
From: Rob Shakir [mailto:[email-masked]]
Sent: Tuesday, July 22, 2014 10:23
To: LITKOWSKI Stephane SCE/IBNF
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: [spring] SPRING MPLS and Entropy Label

Hi Stephane,

Cheers,
r.

---

From: <[email-masked]>
Date: Tue, 22 Jul 2014 14:51:39 +0000

So let's find the best compromise ... but we can't stay stuck ...

-----Original Message-----
From: Xuxiaohu [mailto:[email-masked]] 
Sent: Tuesday, July 22, 2014 10:39
To: LITKOWSKI Stephane SCE/IBNF; Rob Shakir
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: [spring] SPRING MPLS and Entropy Label

Best regards,
Xiaohu

> It sounds like you'd be supportive of asking the authors to extend 
> this draft to making a recommendation? :-)

Completely right :)

-----Original Message-----
From: Rob Shakir [mailto:[email-masked]]
Sent: Tuesday, July 22, 2014 10:23
To: LITKOWSKI Stephane SCE/IBNF
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: [spring] SPRING MPLS and Entropy Label

Hi Stephane,

On 21 Jul 2014, at 22:52, <[email-masked]> <[email-masked]> wrote:

> Entropy label in SPRING is mainly a matter of what are the current 
> capabilities of our favorite vendor's current HW ... Unfortunately, I 
> think we are touching something quite "secret"  and vendors would not 
> share their issues publicly :)

> - reusable EL and EL top of the stack  : need multiple MPLS operation 
> , so it may require forwarding feedback loops on some HWs reducing the 
> forwarding capacity
>

Cheers,
r.

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: Xuxiaohu <[email-masked]>
Date: Tue, 22 Jul 2014 15:05:08 +0000

It makes no sense of making that compromise, IMHO. It seems that the most feasible way is to allow operators themselves to make the choice among these options according to their network conditions.

Best regards,
Xiaohu

________________________________________
发件人: [email-masked] [[email-masked]]
发送时间: 2014年7月22日 22:51
收件人: Xuxiaohu; Rob Shakir
抄送: [email-masked]; [email-masked]; [email-masked]
主题: RE: [spring] SPRING MPLS and Entropy Label

-----Original Message-----
From: Xuxiaohu [mailto:[email-masked]]
Sent: Tuesday, July 22, 2014 10:39
To: LITKOWSKI Stephane SCE/IBNF; Rob Shakir
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: [spring] SPRING MPLS and Entropy Label

Best regards,
Xiaohu

Completely right :)

-----Original Message-----
From: Rob Shakir [mailto:[email-masked]]
Sent: Tuesday, July 22, 2014 10:23
To: LITKOWSKI Stephane SCE/IBNF
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: [spring] SPRING MPLS and Entropy Label

Hi Stephane,

Cheers,
r.

---

From: Rob Shakir <[email-masked]>
Date: Tue, 22 Jul 2014 11:20:02 -0400

> It makes no sense of making that compromise, IMHO. It seems that the most feasible way is to allow operators themselves to make the choice among these options according to their network conditions.

Such a recommendation does not have to say that this is the ONLY solution that can be used. However, it is very likely that a number of operators have a common set of limitations (given that we know there are commonalities in deployed h/w). IMVHO, the thing that we should avoid is that e.g., Stephane asks to support solution 1, and I ask for support for solution 2 — such that (as a community) we end up needing to support 1 && 2, when maybe we could have reached some comment recommendation such that there was only need for support for 1 || 2.

Let’s wait and hear from the authors of the draft whether they intend to put forward some recommendation. If they do not, and rather it should be an operator’s choice, perhaps then the numerous operators that are involved in the SR/MPLS can instead try and make such a recommendation if we see that there is value in it. However, as Stephane pointed out, this is not a decision that can be made in isolation from those that build h/w.

r.

---

From: <[email-masked]>
Date: Tue, 22 Jul 2014 15:27:59 +0000

Having multiple implementations makes no sense to me :
- complex to understand in operations
-  interop to handle
- vendors will not implement all the options, as some have HW development needs.

-----Original Message-----
From: Xuxiaohu [mailto:[email-masked]] 
Sent: Tuesday, July 22, 2014 11:05
To: LITKOWSKI Stephane SCE/IBNF; Rob Shakir
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: [spring] SPRING MPLS and Entropy Label

Best regards,
Xiaohu

-----Original Message-----
From: Xuxiaohu [mailto:[email-masked]]
Sent: Tuesday, July 22, 2014 10:39
To: LITKOWSKI Stephane SCE/IBNF; Rob Shakir
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: [spring] SPRING MPLS and Entropy Label

Best regards,
Xiaohu

Completely right :)

-----Original Message-----
From: Rob Shakir [mailto:[email-masked]]
Sent: Tuesday, July 22, 2014 10:23
To: LITKOWSKI Stephane SCE/IBNF
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: [spring] SPRING MPLS and Entropy Label

Hi Stephane,

Cheers,
r.

---

From: Sri <[email-masked]>
Date: Tue, 22 Jul 2014 23:40:08 -0700

Hi Rob, see inline.

[MPLS WG added to discussion]

>   guarantee label stack depth >= 3*[number of path SIDs].

Sri> You are correct. If a SID identifies multiple physical paths as in the
case of Adj-SID identifying a bundle, then load balancing on that segment
would be useful. When there is no information that every SID of a LSP maps
to a single physical path, a load-balancing mechanism should be used for
that LSP. I agree that today there is no mechanism to give such information
about the mapping, so load balancing on every segment is useful. Whether
that is achieved by using an EL per segment as described in sec 3.2 thereby
resulting in a label stack of 3*#SIDs is something to be discussed. The
draft lists the downsides of choosing that option.

>   some of the concerns raised for this option.

Sri> The draft does not estimate stack depth (without EL) since the depth
is dependent on the degree of strict-hop traffic-engineering desired by the
application. If you meant to say that the label stack is deep due to the
application trying to make it as close to the desired physical path but you
end up using the EL per segment anyways since there isn't information to
determine for sure whether the mapping is to a single physical path, then
yes the label stack will be very deep.

>   might change whilst re-routing occurs.

Sri> This is correct. In case of re-routing from ingress, the algorithm to
compute EL depths have to be re-computed. If the re-route is like FRR via a
bypass LSP, the PLR may want to insert ELs if load-balancing is desired
along the bypass.

>   mid-point which might not have information about it leaked between areas?

Sri> Option in 3.4 runs into such complications in case of multi-area. All
such cases have not been addressed yet for the option in 3.4

>   good to understand whether the authors are intending to do this, or

Sri> The first goal of the draft was to list the various approaches. But
the final goal is to come up with recommendations through the discussions
on the various approaches.

> r.

Thanks
Sri

---

From: Rob Shakir <[email-masked]>
Date: Wed, 23 Jul 2014 11:25:54 -0400

Hi Sri!

Thanks for the feedback.

On 23 Jul 2014, at 02:40, Sri <[email-masked]> wrote:

> Sri> You are correct. If a SID identifies multiple physical paths as in the case of Adj-SID identifying a bundle, then load balancing on that segment would be useful. When there is no information that every SID of a LSP maps to a single physical path, a load-balancing mechanism should be used for that LSP. I agree that today there is no mechanism to give such information about the mapping, so load balancing on every segment is useful. Whether that is achieved by using an EL per segment as described in sec 3.2 thereby resulting in a label stack of 3*#SIDs is something to be discussed. The draft lists the downsides of choosing that option. 
>  
> 
> Sri> The draft does not estimate stack depth (without EL) since the depth is dependent on the degree of strict-hop traffic-engineering desired by the application. If you meant to say that the label stack is deep due to the application trying to make it as close to the desired physical path but you end up using the EL per segment anyways since there isn't information to determine for sure whether the mapping is to a single physical path, then yes the label stack will be very deep. 

rjs> Absolutely. The point that I think is important here is that in the case that we have:

src->[ A ] -- [ B ] -- [ C ] -- [ D ] -- [ E ]->sink
        \                |        /
         --------------[ F ] -----

And we have a strict requirement for A-B-C-D-E, then whilst it might immediately look like we could specify:

<SA{B},SB{C},SC{D},SD{E},SE{sink}> [5 labels]

then actually, the requirement for EL per tunnel is now:

<SA{B},ELI,EL,SB{C},ELI,EL,SC{D},ELI,EL,SD{E},ELI,EL,SE{sink},ELI,EL> [15 labels]

The example that you have in the document makes the case look somewhat better than this. As you say, this is because the application that is chosen in the example is somewhat less strict with its path placement requirements, and a number of the labels in the stack are associated with services. A reader could easily conclude that since the strict src->A->B->C->D->E->sink path that is chosen has affinity to individual adjacencies, this concern does not apply, but to deal with the fact that things are bundles/LAGs, then I think it does. 

It’d be great if we could patch the wording (happy to help contribute) to provide some wording that says that applications that already produce deep label stacks would produce even deeper label stacks with this option AND we know that there are limitations with the number of labels that can be pushed by real h/w today.

> Sri> This is correct. In case of re-routing from ingress, the algorithm to compute EL depths have to be re-computed. If the re-route is like FRR via a bypass LSP, the PLR may want to insert ELs if load-balancing is desired along the bypass. 
> 
> 
> Sri> Option in 3.4 runs into such complications in case of multi-area. All such cases have not been addressed yet for the option in 3.4

rjs> Sure — it would potentially be good to note this as a downside. Especially if there’s no clear solution other than leaking information between areas, since this has impact on operator’s network designs today.

> Sri> The first goal of the draft was to list the various approaches. But the final goal is to come up with recommendations through the discussions on the various approaches.

rjs> Great — aligned with this, I think there would be some utility to adding another option, which is that where there are ECMPs that can be differentiated in terms of use of different SIDs (e.g., load-balancing across two core planes), then a head-end based option (as per the approach that we would take today with RSVP-TE LSPs) is possible.

rjs> Addressing the final goal - as per Stephane’s comments - I would like to recommend that we specify §3.1 as the target option (such that we have a single ELI/EL deeper in the stack) as the “ideal” option. It then seems to me that a debate as to whether the forwarding behaviour in §3.3 is possible with most h/w today (both mean that either we change the number of operations that are required on a forwarding device when popping/swapping labels as far as I can see), or whether the approach in §3.4 (with the caveats noted above) is the best way forward.

Thanks for your consideration.
r.

---

From: Rob Shakir <[email-masked]>
Date: Wed, 23 Jul 2014 17:56:09 -0400

Hi Kireeti,

(Responding on-list for completeness - and a reminder to myself.)

On 23 Jul 2014, at 12:54, Kireeti Kompella <[email-masked]> wrote:

>> expect entropy to be required for load-sharing in the draft.
> 
> There are three questions hiding here:
> 1) is ECMP needed with MPLS SPRING (in particular, Adj-SID)?
> Valid question, as ECMP wasn't seen at first as necessary with RSVP-TE, which shares intent with Adj-SID. I'd say the answer is Yes.  You give an example of why; analogies from the multi path RSVP draft can also be used. 
> 
> 2) is EL the way to achieve ECMP with MPLS SPRING?
> I sincerely hope so; having two mechanisms in MPLS to enable ECMP wouldn't be ideal. But I won't rule out other ways. 
> 
> [Your point is taken: laying out (1) and (2) more clearly would help the draft.]

rjs> I agree with both points. Let me work to contribute some material to expand on the couple of points I raised following the meeting this week.

> 
> 3) Given Yeses to (1) and (2), how?
> That's the current focus of the draft. This part is for now an exploration. I agree that at some point, we have to narrow the options down to a few (ideally one). This would be an action for vendors, knowing their chips, and providers, knowing their requirements to sit together and work it out. 
> 
> There is an aspect of the draft that hasn't come out, which is along the lines of the MPLS forwarding draft, soon to be an RFC: what is the best way in the future, given the requirements of SPRING and EL? I.e., if you built new chips today, knowing what we know, which option would you choose?  Getting a sense of that would help the community to come to a common choice for the future -- as someone mentioned, today's choices/compromises may end up varying widely vendor-to-vendor. 

rjs> Agreed — if we can determine whether we think that the list of options is complete following the exercise of solving (1) and (2), then perhaps we can start to address these questions and try to conclude the discussions at the next meeting.

Cheers,
r.

---

From: <[email-masked]>
Date: Thu, 24 Jul 2014 08:31:49 +0200

Hi Rob,

I agree, a single ELI/EL deeper in the stack as proposed by §3.1 is the most useful target solution also from my point of view.

Regards,

Ruediger

-----Message-----
From: spring [mailto:[email-masked]] Rob Shakir

Hi Sri!

[snip]

rjs> Addressing the final goal - as per Stephane's comments - I would like to recommend that we specify §3.1 as the target option (such that we have a single ELI/EL deeper in the stack) as the "ideal" option. It then seems to me that a debate as to whether the forwarding behaviour in §3.3 is possible with most h/w today (both mean that either we change the number of operations that are required on a forwarding device when popping/swapping labels as far as I can see), or whether the approach in §3.4 (with the caveats noted above) is the best way forward.

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 24 Jul 2014 09:21:32 +0200

All,

Observing this discussion I have a question why the draft as well as this
discussion is only limited to one solution for efficient load balancing
across parallel links in the network - namely entropy labels ?

Let's observe that unlike in general MPLS label allocation per FEC in
SPRING labels are allocated per much more contained number of segment node
properties.

Therefor an alternative of using any form of entropy labels in data plane
all together would be to allocate a SID blocks (say 64 or 128 wide) and
allow SPRING header imposition to use such pools of SIDs per group of flows
to effectively allow for efficient load balancing in the network.

Moreover this solution does not put any additional requirements on MPLS
hardware and can be easily supported today by most of shipping routers.

Best,
r.

On Thu, Jul 24, 2014 at 8:31 AM, <[email-masked]> wrote:

> Hi Rob,
>
> I agree, a single ELI/EL deeper in the stack as proposed by §3.1 is the
> most useful target solution also from my point of view.
>
> Regards,
>
> Ruediger
>
> -----Message-----
> rjs> Addressing the final goal - as per Stephane's comments - I would like
> to recommend that we specify §3.1 as the target option (such that we have a
> single ELI/EL deeper in the stack) as the "ideal" option. It then seems to
> me that a debate as to whether the forwarding behaviour in §3.3 is possible
> with most h/w today (both mean that either we change the number of
> operations that are required on a forwarding device when popping/swapping
> labels as far as I can see), or whether the approach in §3.4 (with the
> caveats noted above) is the best way forward.
>

---

From: <[email-masked]>
Date: Thu, 24 Jul 2014 11:48:12 +0000

IMHO,
> 1) is ECMP needed with MPLS SPRING (in particular, Adj-SID)?
Yes it is ... SPRING brings more flexibility in ECMP for TE tunnels than RSVP-TE does. And ECMP with Bundle-Adj-SID is something really useful when a Service Provider is using parallel ECMP links rather than LAGs.

>2) is EL the way to achieve ECMP with MPLS SPRING?
I would also agree that having two mechanism would be bad from an operational point of view.
Entropy label is not new at IETF, but new in deployment (feature availability is something really recent). If SP that have deployed Entropy for loadbalancing flows needs to move to something else for their SR-TE tunnels, well ... sounds a bad thing ...
Moreover entropy label has the magic of keeping loadbalancing stateless which is a pretty good advantage compared to adding more states at Ingress just for loadbalancing.

Stephane

-----Original Message-----
From: spring [mailto:[email-masked]] On Behalf Of Rob Shakir
Sent: Wednesday, July 23, 2014 17:56
To: Kireeti Kompella
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: [spring] SPRING MPLS and Entropy Label

Hi Kireeti,

On 23 Jul 2014, at 12:54, Kireeti Kompella <[email-masked]> wrote:

>> - I feel that it would be useful to provide some clarification as to 
>> where we expect entropy to be required for load-sharing in the draft.
> 

> There is an aspect of the draft that hasn't come out, which is along the lines of the MPLS forwarding draft, soon to be an RFC: what is the best way in the future, given the requirements of SPRING…

rjs> Agreed - if we can determine whether we think that the list of options is complete following the exercise of solving (1) and (2), then perhaps we can start to address these questions and try to conclude the discussions at the next meeting.

Cheers,
r.

---

From: Kireeti Kompella <[email-masked]>
Date: Thu, 24 Jul 2014 08:16:18 -0400

(resending; sorry for the spam)

Hi Robert,

On Jul 24, 2014, at 03:21 , Robert Raszuk <[email-masked]> wrote:

> Therefor an alternative of using any form of entropy labels in data plane all together would be to allocate a SID blocks (say 64 or 128 wide) and allow SPRING header imposition to use such pools of SIDs per group of flows to effectively allow for efficient load balancing in the network.

We’re rehashing (pun unintended) the entropy label discussion.

Before we settled on entropy labels the way we did, we discussed advertising label blocks in LDP.  There are two problems:
a) as Stephane mentions, ELs are stateless, but pools of SIDs impose more state in the forwarding plane (and to some extent, in the control plane).
b) if you use small blocks, you get uneven load balancing; if you use large blocks, you blow up the state more.

Kireeti

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 24 Jul 2014 14:48:20 +0200

Hi Kireeti,

I quite disagree about the more state argument.

If you treat mpls label as prefix with mask which is all this boils down to
there is no more state either in data or control plane.

As to the point of not perfect load balancing it all depends on your hash
function. If you always advertise at least as big block as you have max
parallel links on any interface you should be fine.

Best ,
R.
On Jul 24, 2014 2:12 PM, "Kireeti Kompella" <[email-masked]> wrote:

>  Therefor an alternative of using any form of entropy labels in data
> plane all together would be to allocate a SID blocks (say 64 or 128 wide)
> and allow SPRING header imposition to use such pools of SIDs per group of
> flows to effectively allow for efficient load balancing in the network.
>
>
>  Before we settled on entropy labels the way we did, we discussed
> advertising label blocks in LDP.  There are two problems:
> a) as Stephane mentions, ELs are stateless, but pools of SIDs impose more
> state in the forwarding plane (and to some extent, in the control plane).
> b) if you use small blocks, you get uneven load balancing; if you use
> large blocks, you blow up the state more.
>
>  Kireeti
>
>

---

From: Xuxiaohu <[email-masked]>
Date: Thu, 24 Jul 2014 13:27:04 +0000

Concur.

Best regards,

Xiaohu

________________________________
发件人: mpls [[email-masked]] 代表 Kireeti Kompella [[email-masked]]
发送时间: 2014年7月24日 20:12
收件人: Robert Raszuk
抄送: Kireeti Kompella; [email-masked]; [email-masked]; [email-masked]
主题: Re: [mpls] [spring] SPRING MPLS and Entropy Label

Hi Robert,

On Jul 24, 2014, at 03:21 , Robert Raszuk <[email-masked]<mailto:[email-masked]>> wrote:

Kireeti

---

From: <[email-masked]>
Date: Thu, 24 Jul 2014 16:46:14 +0000

Robert,

Talking MPLS, I don’t see a way to use MPLS label “prefix” … this is a discussion we already had together. I think you are trying to sell IPv6 SR that I will unfortunately not buy again ☺

From: spring [mailto:[email-masked]] On Behalf Of Robert Raszuk
Sent: Thursday, July 24, 2014 08:48
To: Kireeti Kompella
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: [spring] SPRING MPLS and Entropy Label

Hi Kireeti,

If you treat mpls label as prefix with mask which is all this boils down to there is no more state either in data or control plane.

As to the point of not perfect load balancing it all depends on your hash function. If you always advertise at least as big block as you have max parallel links on any interface you should be fine.

Best ,
R.
On Jul 24, 2014 2:12 PM, "Kireeti Kompella" <[email-masked]<mailto:[email-masked]>> wrote:
Hi Robert,

Kireeti

---

From: Kireeti Kompella <[email-masked]>
Date: Wed, 23 Jul 2014 16:54:44 +0000

Hi Rob,

> on the intention around the document.

Thanks for bringing this up. Good questions; I'll answer them from my perspective. 

HTH,
Kireeti

---

From: Kireeti Kompella <[email-masked]>
Date: Thu, 24 Jul 2014 12:12:52 +0000

Hi Robert,

Kireeti

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 24 Jul 2014 18:54:14 +0200

Stephane,

This has NOTHING TO DO with IPv6. I am not even sure based on what you draw
such strange conclusion ....

I am talking pure about MPLS data plane (even without any IP above) and how
we can solve the load balancing problems.

At min I expect for any discussion to be valid to discuss and compare all
options.

Note also that SPRING does not use LDP so I am much less interested in
solving the LDP load balancing issues.

Rgs,
R.

On Thu, Jul 24, 2014 at 6:46 PM, <[email-masked]> wrote:

>  Robert,
>
>
>
> Talking MPLS, I don’t see a way to use MPLS label “prefix” … this is a
> discussion we already had together. I think you are trying to sell IPv6 SR
> that I will unfortunately not buy again J
>
>
>
> *From:* spring [mailto:[email-masked]] *On Behalf Of *Robert
> Raszuk
> *Sent:* Thursday, July 24, 2014 08:48
> *To:* Kireeti Kompella
> *Cc:* [email-masked]; [email-masked];
> [email-masked]
> *Subject:* Re: [spring] SPRING MPLS and Entropy Label
>
>
>
> Hi Kireeti,
>
> If you treat mpls label as prefix with mask which is all this boils down
> to there is no more state either in data or control plane.
>

---

From: <[email-masked]>
Date: Thu, 24 Jul 2014 17:05:51 +0000

Robert,

From a practical point of view , how do  you represent such MPLS label range with a prefix ?
[70, 170]
[17, 69]

MPLS was not designed for “label aggregation” scheme …
Or we need to change of labels are allocated from the label space for all label consumers … to permit this aggregation.

From: [email-masked] [mailto:[email-masked]] On Behalf Of Robert Raszuk
Sent: Thursday, July 24, 2014 12:54
To: LITKOWSKI Stephane SCE/IBNF
Cc: Kireeti Kompella; [email-masked]; [email-masked]; [email-masked]
Subject: Re: [spring] SPRING MPLS and Entropy Label

Stephane,

This has NOTHING TO DO with IPv6. I am not even sure based on what you draw such strange conclusion ....

I am talking pure about MPLS data plane (even without any IP above) and how we can solve the load balancing problems.

At min I expect for any discussion to be valid to discuss and compare all options.

Note also that SPRING does not use LDP so I am much less interested in solving the LDP load balancing issues.

Rgs,
R.

On Thu, Jul 24, 2014 at 6:46 PM, <[email-masked]<mailto:[email-masked]>> wrote:
Robert,

From: spring [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Robert Raszuk
Sent: Thursday, July 24, 2014 08:48
To: Kireeti Kompella
Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Re: [spring] SPRING MPLS and Entropy Label

Hi Kireeti,

Best ,
R.

Kireeti

Thank you.

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 24 Jul 2014 19:23:46 +0200

Hi,

That is also solved problem ;)

We already know how to encode label offset. Using the same encoding you can
treat offset value as number of significant bits within the 20 bit label.

We do not need to extend this to current label use cases. Hence there is no
issue with backwards compatibility.

Very basic and simple ;)

Cheers,
R.
On Jul 24, 2014 7:05 PM, <[email-masked]> wrote:

> Or we need to change of labels are allocated from the label space for all
> label consumers … to permit this aggregation.
>
>
>
>
>
> *From:* [email-masked] [mailto:[email-masked]] *On Behalf Of *Robert
> Raszuk
> *Sent:* Thursday, July 24, 2014 12:54
> *To:* LITKOWSKI Stephane SCE/IBNF
> *Cc:* Kireeti Kompella; [email-masked]; [email-masked];
> I am talking pure about MPLS data plane (even without any IP above) and
> how we can solve the load balancing problems.
>
>
>

---

From: <[email-masked]>
Date: Thu, 24 Jul 2014 17:39:14 +0000

> We already know how to encode label offset.
Are you referring to Segment Index as offset  of label base ?

> Using the same encoding you can treat offset value as number of significant bits within the 20 bit label.
Could you give an example  of lookup and forwarding structure for your label range/label prefix ? I see something possible but I still have an issue with the lookup structure.

From: [email-masked] [mailto:[email-masked]] On Behalf Of Robert Raszuk
Sent: Thursday, July 24, 2014 13:24
To: LITKOWSKI Stephane SCE/IBNF
Cc: Kireeti Kompella; [email-masked]; [email-masked]; [email-masked]
Subject: RE: [spring] SPRING MPLS and Entropy Label

Hi,

We already know how to encode label offset. Using the same encoding you can treat offset value as number of significant bits within the 20 bit label.

We do not need to extend this to current label use cases. Hence there is no issue with backwards compatibility.

Very basic and simple ;)

Cheers,
R.
On Jul 24, 2014 7:05 PM, <[email-masked]<mailto:[email-masked]>> wrote:
Robert,

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Robert Raszuk
Sent: Thursday, July 24, 2014 12:54
To: LITKOWSKI Stephane SCE/IBNF
Cc: Kireeti Kompella; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Re: [spring] SPRING MPLS and Entropy Label

Stephane,

Rgs,
R.

Hi Kireeti,

Best ,
R.

Kireeti

Thank you.

Thank you.

---

From: Rob Shakir <[email-masked]>
Date: Thu, 24 Jul 2014 14:31:34 -0400

Hi Robert,

> Therefor an alternative of using any form of entropy labels in data plane all together would be to allocate a SID blocks (say 64 or 128 wide) and allow SPRING header imposition to use such pools of…

Sorry, maybe I’m missing something here, but I found your message a little cryptic.

From what I comprehend, you are saying that we define that a single SID maps to N labels. These labels are split somehow into a “forwarding” and “hash” value. Simplistically, this could say that we have the first 12 bits as the “forwarding” and the bottom 8 as the “hash” value. 

e.g. Node A advertises an Adj-SID with value 0b00000101000 (40), anything that is in the range 0b0000010100000000000 (10240) - 0b0000010100011111111 (10495) has the same forwarding behaviour, but the last 8 bits are used to hash (allowing 256 different hash-able objects).

If I understood right - then this seems to have a number of challenges to me:

- For Node-SIDs, we now must sterilise the SRGB to be as wide as (number of required entries)*(number of hash-able objects required) - which means that we must be able to determine a label block that can be used for this.
- We reduce the size of the label space from 20-bits to a smaller value - this might be a material concern in cases where we have many other labels that are allocated on a device (per-prefix label allocation within an L3VPN environment that has many prefixes might be a concern?).
- We need every device in the network to comprehend this new format of label allocation and imposition - such that we do not increase the number of forwarding entries that are required (as Stephane and Kireeti have observed). This is problematic where we have limited LFIB already since instead of programming a number of LFIB entries O(the number of labelled endpoints) then we instead need to have O(number of labelled endpoints * number of hashable objects).

Equally, I think the assertion that you have made is that deeper EL is not “easily supported by most of [today’s] shipping routers”. I think that we need to determine whether this is really the case — given that MPLS WG has standardised EL we need to figure *why* a change from this is required — which is (I guess) the intention of the draft we’re discussing here. *If* we conclude that none of the options that are presented using existing technology are suitable, then potentially then we can look at new solutions.

If my reading of your mail is completely whacky, then please ignore the above - and a clarifying example would be greatly appreciated.

thx,
r.

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 24 Jul 2014 22:21:33 +0200

Hi Rob,

I see your interpretation and where it could be confusing. You have clearly
went beyond my boldest intentions :)

> From what I comprehend, you are saying that we define
> that a single SID maps to N labels.

Nope. That is perhaps key difference.

My proposal is that node advertises block of Node-SIDs which in turn map
1:1 to MPLS labels. The block of Node-SIDs as as big as largest expected
number of parallel links.

/* There is no harm in wrong estimation as simply you will just readvertise
larger or smaller block */

While this could be accomplished today and may work fine in number of
deployments as is there is clear opportunity for optimization. That say
that SID block can be represented as prefix so naturally corresponding to
it MPLS labels.

That said there are two things missing ...

* fundamental the SR architecture does not accommodate that
* MPLS data plane can be optimized to reduce number of LFIB entries.

And that's it. How you use it seems just natural both on ingress as well as
on the transit nodes.

Best,
r.

On Thu, Jul 24, 2014 at 8:31 PM, Rob Shakir <[email-masked]> wrote:

> From what I comprehend, you are saying that we define that a single SID
> maps to N labels. These labels are split somehow into a “forwarding” and
> “hash” value. Simplistically, this could say that we have the first 12 bits
> as the “forwarding” and the bottom 8 as the “hash” value.
>
> e.g. Node A advertises an Adj-SID with value 0b00000101000 (40), anything
> that is in the range 0b0000010100000000000 (10240) - 0b0000010100011111111
> (10495) has the same forwarding behaviour, but the last 8 bits are used to
> hash (allowing 256 different hash-able objects).
>
> - For Node-SIDs, we now must sterilise the SRGB to be as wide as (number
> of required entries)*(number of hash-able objects required) - which means
> that we must be able to determine a label block that can be used for this.
> - We reduce the size of the label space from 20-bits to a smaller value -
> this might be a material concern in cases where we have many other labels
> that are allocated on a device (per-prefix label allocation within an L3VPN
> environment that has many prefixes might be a concern?).
> - We need every device in the network to comprehend this new format of
> label allocation and imposition - such that we do not increase the number
> of forwarding entries that are required (as Stephane and Kireeti have
> observed). This is problematic where we have limited LFIB already since
> instead of programming a number of LFIB entries O(the number of labelled
> endpoints) then we instead need to have O(number of labelled endpoints *
> number of hashable objects).
>
> Equally, I think the assertion that you have made is that deeper EL is not
> “easily supported by most of [today’s] shipping routers”. I think that we
> need to determine whether this is really the case — given that MPLS WG has
> standardised EL we need to figure *why* a change from this is required —
> which is (I guess) the intention of the draft we’re discussing here. *If*
> we conclude that none of the options that are presented using existing
> technology are suitable, then potentially then we can look at new solutions.
>
> If my reading of your mail is completely whacky, then please ignore the
> above - and a clarifying example would be greatly appreciated.
>
> thx,
> r.
>

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 24 Jul 2014 22:34:24 +0200

Hi,

> Are you referring to Segment Index as offset  of label base ?

​No.​ SI is already defined for different use. I am describing general
concept how we could indicate in 5 bits the number of significant (for
forwarding) bits in MPLS label.

 > Using the same encoding you can treat offset value as number of
> significant bits within the 20 bit label.
>
>  Could you give an example  of lookup and forwarding structure for your
> label range/label prefix ? I see something possible but I still have an
> issue with the lookup structure.
>

​The lookup is trivial as there is no LPM needed. You simply do partial
match on your mpls labels in the LFIB.

thx,
R.

> *Sent:* Thursday, July 24, 2014 13:24
> *Cc:* Kireeti Kompella; [email-masked];
> [email-masked]; [email-masked]
> *Subject:* RE: [spring] SPRING MPLS and Entropy Label
>
>
>
> Hi,
>
> We already know how to encode label offset. Using the same encoding you
> can treat offset value as number of significant bits within the 20 bit
> label.
>

---

From: Rob Shakir <[email-masked]>
Date: Fri, 25 Jul 2014 11:44:26 -0400

Hi Robert,

On 24 Jul 2014, at 16:21, Robert Raszuk <[email-masked]> wrote:

> I see your interpretation and where it could be confusing. You have clearly went beyond my boldest intentions :) 

Thanks for the clarification.

> My proposal is that node advertises block of Node-SIDs which in turn map 1:1 to MPLS labels. The block of Node-SIDs as as big as largest expected number of parallel links. 

Operationally — this doesn’t seem ideal to me. AIUI, If we have some forwarding issue between two LSRs, then we might end up with partial reachability if a subset of the paths do not have correct LFIB entries throughout the network. It introduces a requirement, where failure detection is required, to run N different OAM probes to be able to monitor reachability (one for each Node-SID assigned to the remote device - even in cases where we have 1 available path).

I think the question for the MPLS WG here is whether there is any appetite to introduce a new means by which entropy in MPLS networks is introduced. We already have Entropy Label which has been published and implemented. The question to me seems to be why we need anything else. Fewer solutions seems optimal :-)

Cheers,
r.

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 25 Jul 2014 18:16:39 +0200

​Hi Rob,​

> ​
> It introduces a requirement, where failure detection is required, to run N
> different OAM probes to be able to monitor reachability (one for each
> Node-SID assigned to the remote device - even in cases where we have 1
> available path).
>

​When you get today /24 IP prefix ​do you monitor reachability to all 255
host addresses within this prefix or to the next hop which advertised it ?

Likewise when you get a block of SID monitoring OEM to the node which
advertised it seems sufficient to me. I assume there is still IGP there and
router_id or s-bfd.

Comparing two options it seems that mpls hardware is to much faster support
match on subset of MPLS label bits vs ability to look over long chain of
labels for ELI/EL.

I think the question for the MPLS WG here is whether there is any appetite
> to introduce a new means by which entropy in MPLS networks is introduced.
> We already have Entropy Label which has been published and implemented. The
> question to me seems to be why we need anything else. Fewer solutions seems
> optimal :-)
>

​Of course fewer is better provided that:

a) fewer works
b) is architecturally sound

Moreover SID/label block vs entropy labels is not apple to apple
comparison.

Former is just control plane solution, does not insert anything extra into
each packet while latter is based on data plane modifications and new
hardware support to either handle Eiffel Tower of label stack or tricks
with search along the stack for ELI/EL tupel(s).

Cheers,
R.
​

---

From: Rob Shakir <[email-masked]>
Date: Sat, 26 Jul 2014 18:04:37 -0400

Hi Robert,

On 25 Jul 2014, at 12:16, Robert Raszuk <[email-masked]> wrote:

> ​Hi Rob,​
>  
> ​ It introduces a requirement, where failure detection is required, to run N different OAM probes to be able to monitor reachability (one for each Node-SID assigned to the remote device - even in cases where we have 1 available path).
> 
> ​When you get today /24 IP prefix ​do you monitor reachability to all 255 host addresses within this prefix or to the next hop which advertised it ? 
> 
> Likewise when you get a block of SID monitoring OEM to the node which advertised it seems sufficient to me. I assume there is still IGP there and router_id or s-bcd.

I don’t think this is analogous — there is only one forwarding entry for a /24 if it has a single next-hop. With the multiple SIDs (and 1:1 SID to label mapping) then there will be multiple LFIB entries.

BR,
r.

---

From: Robert Raszuk <[email-masked]>
Date: Sun, 27 Jul 2014 00:07:29 +0200

>
> I don’t think this is analogous — there is only one forwarding entry for a
> /24 if it has a single next-hop. With the multiple SIDs (and 1:1 SID to
> label mapping) then there will be multiple LFIB entries.
>

​Nope.

The entire idea is to have a single LFIB entry as well.

Best,
r.

​

---

From: Rob Shakir <[email-masked]>
Date: Sat, 26 Jul 2014 18:14:14 -0400

> The entire idea is to have a single LFIB entry as well. 

Again, I’m lost. :-(

Perhaps you can expand on how an LSR programs its LFIB such that it has multiple labels matched with a single entry please? I’m struggling to comprehend how this fits into the existing implementations, or can be realised without requiring some forwarding/entropy split of the 20-bit label as I described before.

Kind regards,
r.

---

From: Robert Raszuk <[email-masked]>
Date: Sun, 27 Jul 2014 00:26:01 +0200

>
> Perhaps you can expand on how an LSR programs its LFIB such that it has
> multiple labels matched with a single entry please?
>

​When you install a label to LFIB you also pass its depth ie number of
significant bits to match on.

Hardware will treat the remaining bits as "do not care". That is pretty
basic logical function for any match operation.

At least I am quite sure it is much simpler then any other form of search
for ELI/EL within the SR deep stack :)

I’m struggling to comprehend how this fits into the existing
> implementations, or can be realised without requiring some
> forwarding/entropy split of the 20-bit label as I described before.
>

​The split I have in mind is passed explicitiely in control plane. So there
is no additional logic required for it. ​

​Now you may rightfully state that this requires an upgrade. Well true but
any alternative discussed here requires an upgrade as well. Moreover even
support of SR requires an upgrade :)

Control plane vs data plane .. EL based solution requires both day one.
Label mask does not strictly require any data plane change during the
transition period as it could install all atomic labels till the mask
support in LFIB is available later.

Cheers,
R.​

---

From: Rob Shakir <[email-masked]>
Date: Mon, 28 Jul 2014 11:58:03 +0100

Hi Robert,

On 26 Jul 2014, at 23:26, Robert Raszuk <[email-masked]> wrote:

> Perhaps you can expand on how an LSR programs its LFIB such that it has multiple labels matched with a single entry please?
> 
> ​When you install a label to LFIB you also pass its depth ie number of significant bits to match on. 
> 
> Hardware will treat the remaining bits as "do not care". That is pretty basic logical function for any match operation. 

rjs> So, this is exactly akin to the description I gave of partitioning the label space - since it requires one to sterilise some of the label space (N-bits are now significant for forwarding, with the remaining (20-M) being used for entropy).

> At least I am quite sure it is much simpler then any other form of search for ELI/EL within the SR deep stack :)

rjs> The look-up logic has to change somewhat for the data-plane. Particularly as:

- We need to check whether the top-most label is within the block of labels that we are using for this function.
- If it is, we do a lookup based on the first N-bits to determine the next-hop.
	- if it’s not, we need to do a lookup based on the entire label.
- If this was in the entropy block we determine the OIF by matching this next-hop along with the hash of the remaining (20-N) bits.
	- else we fallback to the existing hashing procedures.

Alternatively, we need to:

- Look-up the next-hop based on the top-most label.
- If the ELI is in the label stack, take the following label as the EL.
- Determine the OIF based on the next-hop and the EL value received.
	- if the EL is not present, then hash based on existing procedures.

Is the former “much simpler”? I am no hardware expert, but I would say they seem roughly analogous in terms of decision process that needs to be made - other than the latter needs visibility into more of the packet (such that ELI/EL is included in the header that is extracted). 

It’s notable that the latter logic must be implemented anyway at an LSR, because we have now standardised ELI/EL and it is implemented.

> I’m struggling to comprehend how this fits into the existing implementations, or can be realised without requiring some forwarding/entropy split of the 20-bit label as I described before.
> 
> ​The split I have in mind is passed explicitiely in control plane. So there is no additional logic required for it. ​
> 
> ​Now you may rightfully state that this requires an upgrade. Well true but any alternative discussed here requires an upgrade as well. Moreover even support of SR requires an upgrade :)
> 
> Control plane vs data plane .. EL based solution requires both day one. Label mask does not strictly require any data plane change during the transition period as it could install all atomic labels till the mask support in LFIB is available later.
> 

rjs> In smaller nodes (aggregation/access nodes) then this might have significant impact, given LFIB limitations. If we need to have M*(number of destination FECs) installed in the LFIB then this might bust some limits.

Whilst label mask is a potential solution, my view is that we should continue to leverage the EL work already done by this WG.

Kind regards,

---

From: Robert Raszuk <[email-masked]>
Date: Mon, 28 Jul 2014 23:03:26 +0200

> - We need to check whether the top-most label is within the block of
> labels that we are using for this function.

​Nope. Sorry.

We always do identical single lookup.

It is that some entries in the LFIB have mask (read: do not care bits).
There are no two lookups nor two tables etc ... That's it. ​Trivial for
anyone who has some min hardware design background :).

Whilst label mask is a potential solution, my view is that we should
> continue to leverage the EL work already done by this WG.
>

​Great. Go for it !

/* After all if we would have spend little cycles now and innovate tiny bit
to improve MPLS data plane it would result in better operation and lasting
longer .... ;-) */

Best,
r.​