Subject: [lisp] Comments to draft-kjsun-lisp-dyncast-00

From: Dino Farinacci <[email-masked]>
Date: Tue, 10 Aug 2021 15:26:20 -0700

Kyoungjae/Younghan, thanks for the draft. See my comments below following your text, that I indented.

Note that the architecture can support anycast EIDs and/or anycast RLOCs with no special protocol considerations. It just depends on how you attach/assign an EID to a service, how the mapping system returns a partial RLOC-set, and how RLOCs are advertised in the underlay (like it is today with existing anycast services).

And in the abstract, it would be good to describe the high-level problem you want to solve. Because in the Introduction section you start out writing how it is solved.

> Having said that dynamic-anycast is an interesting concept so I welcome your contribution."

It would be useful to define the references to nodes in Figure 1 so it is understood when you refer to the diagram what is the relationship of LISP terms with dynamic-anycast terms. Just by looking at the figure, it iw not clear what a D-MA is and why in one case its co-located with an xTR and in another case it is not.

It is also not clear in the diagram where the LISP site boundaries are.  Please add circules that better define the boundary. Thanks.

It is not clear to me at this point in the draft, what a DBID is. The binding between an EID and an RLOC uses a standard term EID-to-RLOC mapping where I think you are proposing a DSID-to-RLOC-set binding. Please confirm.

I'd like to make a suggestion to call this type of EID an "DSEID" so it is clear that you are assigning the anycast attribute to an EID. Or simply make it "AEID" for "Anycast EID" so these concepts can be used more generally.

It sounds like you are giving the mapping system some policy control on what DBID to return and then having the ITR choose an RLOC within the DBID. Is that true? Is that the reason for a indirect mapping with a single lookup?

From this diagram, it appears there is a one-to-one relationship between a given DBID and an RLOC. So why is there a need for another level of indirection?

Today the way LISP supports an EID to RLOC-set mapping is that the EID-to-RLOC-set is registered to the mapping system and the mapping system returns the *entire RLOC-set*, but implementations support the map-server returning a partial set from the registered RLOC-set. But whatever is returned to the ITR allows it to choose which RLOCs to use. Usually the priority value that is created and sent by the ETR in a Map-Register allows the ETR to have control over what is chosen. We have many multi-homing examples of this.

So for your application, you could have a DSEID map to this RLOC-set:

RLOC-A, priority 1
RLOC-B, priority 1
RLOC-C, priority 2
RLOC-D, priority 2

Which indicates that ETR only wants packets sent to C and D when A and B are not reachable from the ITR (by using multiple mechanisms but the most robust is RLOC-probing).

Note the use of an RLOC-set comes in multiple forms:

(1) A list like above (default)
(2) An ELP, where the RLOC-record indicates a encap-path from ITR to ETR.
(3) An RLE, where the RLOC-record indicates that all RLOCs listed are used (multicast replication).

You could use 1 or all combinations of these with filtering/policy support on the map-server.

It is important to note that if you use equal priorities in an RLOC-set, that the ITR can load-split traffic across RLOCs and that will break connnection oriented applications (or applications that require remote state for a session).

So an ITR that is configured that a particular EID in its map-cache is an anycast-EID, it can use an RLOC-set above with each RLOC priority=1 and then choose based on the implementation's idea of what is a better RLOC to use.

IMO opinion, the better way to design/implement this is using an anycast-EID-to-anycast-RLOC mapping. Which means you only need a register from one place (an SDN controller) or each ETR registering the same RLOC (without merge semantics). So the serivce is chosen by destination address in a packet (the anycast-EID) which maps to an anycast-RLOC where the underlay takes you to the "closest" LISP site. But closest is just one way of getting to a service. You may have other ways and hence why you introduced a metric.

This is already supported if you use the multi-tuple EID draft. But I'd use it as a multi-stage lookup, where the multi-tuple EID mappings that point to an anycast-EID and then there is a anycast-EID mapping that points to RLOCs. It does require a 2-stage lookup but allows the anycast-EID to change for a given multi-tuple EID.

As long as packets are sent to a map-cache entry, the entry stays in the map-cache. And just before it is removed, a refresh Map-Request is sent. If you need a shorter TTL to update the entry, you can (1) support SMRs per RFC6833bis or (2) PubSub per draft-ietf-lisp-pubsub draft.

Or draft-kowal-lisp-policy-distribution.

There is one problem with the mapping-system making the decision. It doesn't have data relative to the source LISP site to make service access optimal for it. So we typically don't have map-servers make decisions based a dependence of a (source, dest) path or (5-tuple) EID. The ITR should make this decision given a set of possibilties that come from the mapping system (but registered somewhere else, note you can co-located an ETR and a map-server so the ETR component registers to "itself", per SDN-controller mentality).

Yes, it is better to gather the metrics data outside of LISP. But yes, the control-plane could store it. I would suggest just mapping your collection metrics to priorities so no protocol messages would need to be modified. I believe LISP already has designed in everything you need. Do you agree?

Yes, it can, see draft-rodrigueznatal-lisp-multi-tuple-eids.

Thanks again for the draft,
Dino

---

From: 선경재 <[email-masked]>
Date: Wed, 11 Aug 2021 21:24:27 +0900

Dino,

First of all, thank you very much to give detailed comments on our draft. Please check answers for your comments inline.

Additional comments and discussions are always welcome.

Regards,

KJ.

Kyoungjae Sun / K. J. SUN
————————————————————————————————————
Researcher
Distributed Cloud and Network Laboratory (DCN Lab)
Internet Infrastructure System Technology Research Center(IISTRC)
369 Sangdo-ro, Dongjak-gu, Seoul.
————————————————————————————————————
Tel. +82-2-814-0151
Mobile +82-10-3643-5627
Email [email-masked]

> 2021. 8. 11. 오전 7:26, Dino Farinacci <[email-masked]> 작성:
> 

KJ: In this draft, the concept and some architectural components of Dynamic anycasting come from the draft already proposed <draft-li-dyncast-architecture>. The purpose of this draft is to analyze the applicability of that concept based on the LISP.  

KJ: Yes, as your comments, LISP can support anycast EIDs and/or RLOCs with modification but in the dyncast concept, there are additional requirements to share service-related metrics and select in a more dynamic manner. Following your comment, I will update the abstract to clearly describe the purpose of our draft.

> It would be useful to define the references to nodes in Figure 1 so it is understood when you refer to the diagram what is the relationship of LISP terms with dynamic-anycast terms. Just by looking…

KJ: I agree with your comment and I felt also a little confused to be tried using different terminologies in the Dyncast draft together with LISP terminologies. In dyncast, they use Binding ID (BID) as locator where service instance is running so at the first time I just mapped that to RLOC but was still confused. I think it may be better to use RLOC instead of DBID to understand clearly. I will modify the figure and contents to make them clear.

> It is also not clear in the diagram where the LISP site boundaries are.  Please add circules that better define the boundary. Thanks.

KJ:  I will draw the borders in the next version.    
> 
> It is not clear to me at this point in the draft, what a DBID is. The binding between an EID and an RLOC uses a standard term EID-to-RLOC mapping where I think you are proposing a DSID-to-RLOC-set binding. Please confirm

KJ: Yes, I also think that DSID-to-RLOC-set is the more clear term. Thanks for your suggestion.

> .

KJ:  Thank you for your suggestions. The first time I tried to use "Anycast EID", but I want to separate it from normal anycast routing so I selected DSID. Since there is a "SID" term in other WG, SRv6, so I was concerned to make confusion for using un-clear terminology. I think DSEID is good for our draft which contains both LISP and Dyncast concepts. I will change that terms in the next version.    

> It sounds like you are giving the mapping system some policy control on what DBID to return and then having the ITR choose an RLOC within the DBID. Is that true? Is that the reason for a indirect m…

KJ:  In that sentence, what I want to say was there are two options to make a decision of selecting the proper DBID for the request. In my opinion, indirect mapping for dividing each mapping depending on their metrics is an important feature to support dynamic anycasting in LISP, which selects proper service instances with both network-related and service-instance-related metrics. 
   
> You could use 1 or all combinations of these with filtering/policy support on the map-server.

KJ: The priority you mentioned just represents the network-related capability in the dynamic anycast viewpoint. However, if parameters to be required for selecting routing path are not only network-related metrics but also service-related metrics (i.e., computing loads, number of flows, etc.), we need other indirection for that.

> IMO opinion, the better way to design/implement this is using an anycast-EID-to-anycast-RLOC mapping. Which means you only need a register from one place (an SDN controller) or each ETR registering…

KJ:  In my understanding, in your suggestion, anycast RLOC is allocated to the ETRs which can deliver packet destined to anycast-EID by internal routing and routing between those ETRs are chosen in the underlay. Is it right? 

KJ: In this draft, we consider that service instances can be dynamically instantiated/released across multiple edges at the same time. For that, anycast-RLOC should be registered/deregistered at the ETR depending on the absence of specific anycast-EID. I wonder that is more efficient rather than using indirection mapping and update it with unicast-RLOC with metric information.
> This is already supported if you use the multi-tuple EID draft. But I'd use it as a multi-stage lookup, where the multi-tuple EID mappings that point to an anycast-EID and then there is a anycast-E…

Thank you for noticing me multi-tuple EID draft. I will check that draft and update contents. 
   
> As long as packets are sent to a map-cache entry, the entry stays in the map-cache. And just before it is removed, a refresh Map-Request is sent. If you need a shorter TTL to update the entry, you…

KJ: Yes. the point I indicated that for dynamically changing map-cache entry, TTL can be shorter but I missed the existing solutions. According to your comment, I will update those solutions. As well as SMRs support, I think PubSub is also a good solution to support dynamic map-cache updates. Thank you for your point.

> Or draft-kowal-lisp-policy-distribution.

KJ : Thanks. I will check and add that too.

> There is one problem with the mapping-system making the decision. It doesn't have data relative to the source LISP site to make service access optimal for it. So we typically don't have map-servers…

KJ: I understand that making the decision at the mapping system has limitations in the current LISP. ITR is a better point to select the destination ETR. I will update draft with considerations of your comments.    

> Yes, it is better to gather the metrics data outside of LISP. But yes, the control-plane could store it. I would suggest just mapping your collection metrics to priorities so no protocol messages w…

KJ: yes. I agree with your point that LISP already has the solution to mapping metrics to priorities. That is the one of reasons that I think that it is possible to provide dynamic anycast routing using LISP without huge modifications. So, I think that we should not give fixed solutions on how to collect and store metrics but we can mention possible solutions for that. Because it is more implemental issue. Is it right approach?

> Yes, it can, see draft-rodrigueznatal-lisp-multi-tuple-eids.

KJ:  Thank you for giving that information. I will check and update

---

From: Dino Farinacci <[email-masked]>
Date: Wed, 11 Aug 2021 13:29:17 -0700

> First of all, thank you very much to give detailed comments on our draft. Please check answers for your comments inline.

KJ, thanks for the quick response. I have cut out the text where I agree/accept with your responses. See inline for additional commentary and answers to your questions.

Once you update the spec, I'll give another thorough review. Thanks!

> KJ: The priority you mentioned just represents the network-related capability in the dynamic anycast viewpoint. However, if parameters to be required for selecting routing path are not only network…

If you want to control where packets go on the underay, you can also consider draft-ietf-lisp-te. That is where ELPs are introduced and designed into the LISP architecture.

> KJ:  In my understanding, in your suggestion, anycast RLOC is allocated to the ETRs which can deliver packet destined to anycast-EID by internal routing and routing between those ETRs are chosen in…

Yes. You assign the same RLOC address to multiple ETRs and have underlay routing take a source to the closest LISP site.

> KJ: yes. I agree with your point that LISP already has the solution to mapping metrics to priorities. That is the one of reasons that I think that it is possible to provide dynamic anycast routing using LISP without huge modifications.

Yes agree. Then the ITR came make an RLOC selection based on existing documented approaches. But if the metrics change too much then the contents of the RLOC-set changes which requires more frequent map-cache updates. So we have to analyze this tradeoff more in depth.

> So, I think that we should not give fixed solutions on how to collect and store metrics but we can mention possible solutions for that. Because it is more implemental issue. Is it right approach?

Yes, I agree.

> KJ:  Thank you for giving that information. I will check and update    

Thanks again,
Dino

---

From: 선경재 <[email-masked]>
Date: Thu, 12 Aug 2021 10:14:09 +0900

Dino,

Thank you for discussing this topic. It was very helpful for making clear
to our proposal.

I will update draft depending on these discussions.

Additional comments and discussion by LISP folks are always welcome.

Sincerely

KJ.
ᐧ

2021년 8월 12일 (목) 오전 5:29, Dino Farinacci <[email-masked]>님이 작성:

> > First of all, thank you very much to give detailed comments on our
> draft. Please check answers for your comments inline.
>
> KJ, thanks for the quick response. I have cut out the text where I
> agree/accept with your responses. See inline for additional commentary and
> answers to your questions.
>
> >> Which indicates that ETR only wants packets sent to C and D when A and
> B are not reachable from the ITR (by using multiple mechanisms but the most
> robust is RLOC-probing).
> >>
> > KJ: The priority you mentioned just represents the network-related
> capability in the dynamic anycast viewpoint. However, if parameters to be
> required for selecting routing path are not only network-related metrics
> but also service-related metrics (i.e., computing loads, number of flows,
> etc.), we need other indirection for that.
>
> If you want to control where packets go on the underay, you can also
> consider draft-ietf-lisp-te. That is where ELPs are introduced and designed
> into the LISP architecture.
>
> >>
> >>
> >> It is important to note that if you use equal priorities in an
> RLOC-set, that the ITR can load-split traffic across RLOCs and that will
> break connnection oriented applications (or applications that require
> remote state for a session).
> >>
> >> So an ITR that is configured that a particular EID in its map-cache is
> an anycast-EID, it can use an RLOC-set above with each RLOC priority=1 and
> then choose based on the implementation's idea of what is a better RLOC to
> use.
> >>
> >> IMO opinion, the better way to design/implement this is using an
> anycast-EID-to-anycast-RLOC mapping. Which means you only need a register
> from one place (an SDN controller) or each ETR registering the same RLOC
> (without merge semantics). So the serivce is chosen by destination address
> in a packet (the anycast-EID) which maps to an anycast-RLOC where the
> underlay takes you to the "closest" LISP site. But closest is just one way
> of getting to a service. You may have other ways and hence why you
> introduced a metric.
> >
> > KJ:  In my understanding, in your suggestion, anycast RLOC is allocated
> to the ETRs which can deliver packet destined to anycast-EID by internal
> routing and routing between those ETRs are chosen in the underlay. Is it
> right?
>
> Yes. You assign the same RLOC address to multiple ETRs and have underlay
> routing take a source to the closest LISP site.
>
> >>
> >> Yes, it is better to gather the metrics data outside of LISP. But yes,
> the control-plane could store it. I would suggest just mapping your
> collection metrics to priorities so no protocol messages would need to be
> modified. I believe LISP already has designed in everything you need. Do
> you agree?
> >
> > KJ: yes. I agree with your point that LISP already has the solution to
> mapping metrics to priorities. That is the one of reasons that I think that
> it is possible to provide dynamic anycast routing using LISP without huge
> modifications.
>
> Yes agree. Then the ITR came make an RLOC selection based on existing
> documented approaches. But if the metrics change too much then the contents
> of the RLOC-set changes which requires more frequent map-cache updates. So
> we have to analyze this tradeoff more in depth.
>
> > So, I think that we should not give fixed solutions on how to collect
> and store metrics but we can mention possible solutions for that. Because
> it is more implemental issue. Is it right approach?
>
> Yes, I agree.
>
> >>

-- 
====================================================================

선경재 / Kyoungjae Sun (KJ Sun)

연구원 / 클라우드 및 네트워크 연구실

Researcher / Distributed Cloud and Network lab

Office : +82-2-814-0151    Phone : +82-10-3643-5627

Email : [email-masked]

====================================================================

---

From: Dino Farinacci <[email-masked]>
Date: Wed, 11 Aug 2021 19:03:25 -0700

Thanks again KJ. 

Dino

> On Aug 11, 2021, at 6:14 PM, 선경재 <[email-masked]> wrote:
> 
> 
> Dino,
> 
> Thank you for discussing this topic. It was very helpful for making clear to our proposal.
>