Subject: [spring] IPv6-compressed-routing-header-crh

From: "Rajiv Asati (rajiva)" <[email-masked]>
Date: Thu, 28 Mar 2019 15:36:55 +0000

Hi Ron,

Very Interesting idea that you presented during SPRING session today. Seems useful. 

Two comments/clarification - 

1. One of the slides indicated that small packet size on the Internet was ~500B and calculated ~10% due to Routing EH overhead accordingly. Of course, if we look at mid packet size (800-1000B) or large packet size (1000~1400B), then the overhead would be a lot less. 

We should also look at the % mix of small packets vs mid vs large size to calculate the impact. If mid to large packets were dominant (say, as much as >70% given >80% of traffic is video (ABR etc) per the latest VNI studies ), then the overhead impact due to non-compressed SID usage on the traffic would be even less over all. 

2. Also, what % of savings do we get by using compressed RH vs non-compressed RH ? 24B vs 64B per packet !! 

Cheers,
Rajiv

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 28 Mar 2019 16:44:24 +0100

One needs to be very careful about packet size reasoning.
For TCP, something like 1/3 of all packets are tiny (acks).  A lot less 
than 1/3 of the bytes are in tiny packets :-)
For voice traffic, almost all packets are small.
Yours,
Joel

On 3/28/19 4:36 PM, Rajiv Asati (rajiva) wrote:
> _______________________________________________
> spring mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/spring
>

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 28 Mar 2019 16:51:19 +0100

Hi Joel,

Is your hidden message to express that neither TCP nor UDP with voice app
should use IPv6 at all as overhead is just too big even without additional
extension headers ?

Best,
R.

On Thu, Mar 28, 2019, 16:44 Joel M. Halpern <[email-masked]> wrote:

> > 1. One of the slides indicated that small packet size on the Internet
> was ~500B and calculated ~10% due to Routing EH overhead accordingly. Of
> course, if we look at mid packet size (800-1000B) or large packet size
> (1000~1400B), then the overhead would be a lot less.
> >
> > We should also look at the % mix of small packets vs mid vs large size
> to calculate the impact. If mid to large packets were dominant (say, as
> much as >70% given >80% of traffic is video (ABR etc) per the latest VNI
> studies ), then the overhead impact due to non-compressed SID usage on the
> traffic would be even less over all.
> >
> > 2. Also, what % of savings do we get by using compressed RH vs
> non-compressed RH ? 24B vs 64B per packet !!
> >
> >
> > Cheers,
> > Rajiv
> >
> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> [email-masked]
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------
>

---

From: "john leddy.net" <[email-masked]>
Date: Thu, 28 Mar 2019 12:18:29 -0400 (EDT)

Joel,

It would be good to factor in the ever growing amount of video on the Internet (and other large data transfer applications vs voice traffic).

If larger MTU's could reliably be used, I think you would see a large amount of traffic starting to use something larger than 1500 byte Cells.

Getting to a reliable PMTU mechanism would be a great way to drive efficiency.
Use of larger packets should generate less TCP Ack traffic as well in general.

I'm assuming the voice concern was around efficiency not about whether a voice application would work with the increased overhead of an extension header?

John

> On March 28, 2019 at 11:51 AM Robert Raszuk <[email-masked]> wrote:
> 
>     Hi Joel,
> 
>     Is your hidden message to express that neither TCP nor UDP with voice app should use IPv6 at all as overhead is just too big even without additional extension headers ?
> 
>     Best,
>     R.
> 
> 
>     On Thu, Mar 28, 2019, 16:44 Joel M. Halpern < [email-masked] mailto:[email-masked] > wrote:
> 
> >         > [email-masked] mailto:[email-masked]
> >         [email-masked] mailto:[email-masked]

---

From: "Rajiv Asati (rajiva)" <[email-masked]>
Date: Thu, 28 Mar 2019 17:43:00 +0000

That was one of my points. We would want to substantiate both pps and bps sides for the overhead argument to work.

Now, I would reserve the viewpoint whether it is useful or not. :-)

Cheers,
Rajiv

On Mar 28, 2019, at 5:18 PM, john leddy.net<http://leddy.net> <[email-masked]<mailto:[email-masked]>> wrote:

Joel,

John

On March 28, 2019 at 11:51 AM Robert Raszuk <[email-masked]<mailto:[email-masked]>> wrote:

Hi Joel,

Best,
R.

On Thu, Mar 28, 2019, 16:44 Joel M. Halpern < [email-masked]<mailto:[email-masked]>> wrote:

On 3/28/19 4:36 PM, Rajiv Asati (rajiva) wrote:
> Hi Ron,
>
> Very Interesting idea that you presented during SPRING session today.
>
> [email-masked]<mailto:[email-masked]>

--------------------------------------------------------------------
IETF IPv6 working group mailing list
[email-masked]<mailto:[email-masked]>
Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
--------------------------------------------------------------------
_______________________________________________
spring mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/spring

---

From: Joel Halpern Direct <[email-masked]>
Date: Thu, 28 Mar 2019 21:51:38 +0100

I would like to see working path MTU of some form for many different 
reasons.
I would also like to see larger practical MTUs.
However, history suggests that both goals may be more aspirational than 
practical.

I tend to be very skeptical of any anlysis that says that user traffic 
is changing to X, and that because X has property Y, we should plan our 
networks for property Y.  I am not at all sure I have ever seen this 
proven correct.

All I was saying is that thinking about overhead in terms of a single 
average packet size is mathematically correct but practically misleading.

Yours,
Joel

On 3/28/19 5:18 PM, john leddy.net wrote:
> Joel,
> 
> It would be good to factor in the ever growing amount of video on the 
> Internet (and other large data transfer applications vs voice traffic).
> 
> If larger MTU's could reliably be used, I think you would see a large 
> amount of traffic starting to use something larger than 1500 byte Cells.
> 
> I'm assuming the voice concern was around efficiency not about whether a 
> voice application would work with the increased overhead of an extension 
> header?
> 
> John
> 
> 
> 
> 
>> Is your hidden message to express that neither TCP nor UDP with voice 
>> app should use IPv6 at all as overhead is just too big even without 
>> additional extension headers ?
>>
>> Best,
>> R.
>>
>>
>>     > 1. One of the slides indicated that small packet size on the
>>     Internet was ~500B and calculated ~10% due to Routing EH overhead
>>     accordingly. Of course, if we look at mid packet size (800-1000B)
>>     or large packet size (1000~1400B), then the overhead would be a
>>     lot less.
>>     >
>>     > We should also look at the % mix of small packets vs mid vs
>>     large size to calculate the impact. If mid to large packets were
>>     dominant (say, as much as >70% given >80% of traffic is video (ABR
>>     etc) per the latest VNI studies ), then the overhead impact due to
>>     non-compressed SID usage on the traffic would be even less over all.
>>     >
>>     > [email-masked] <mailto:[email-masked]>
>>     [email-masked] <mailto:[email-masked]>

---

From: Ron Bonica <[email-masked]>
Date: Fri, 29 Mar 2019 16:01:26 +0000

Hi Rajiv,

Thanks for these kind words. 

Generally speaking, we should design for efficiency.  Furthermore, we should engineer for future requirements. For example, in the future, service providers may require longer SID lists (5 or more!). We should plan for that today.

Likewise, in the future, people may deploy new applications that emit short packets. We should also plan for that today.

If we don't plan for future requirements, obsolescence will be much closer than we expect.

                                                    Ron

Juniper Internal

> -----Original Message-----
> From: spring <[email-masked]> On Behalf Of Rajiv Asati (rajiva)
> Sent: Thursday, March 28, 2019 11:37 AM
> To: [email-masked]
> Cc: [email-masked]
> Subject: [spring] IPv6-compressed-routing-header-crh
> 
> Hi Ron,
> 
> 1. One of the slides indicated that small packet size on the Internet was ~500B
> and calculated ~10% due to Routing EH overhead accordingly. Of course, if we
> look at mid packet size (800-1000B) or large packet size (1000~1400B), then
> the overhead would be a lot less.
> 
> We should also look at the % mix of small packets vs mid vs large size to
> calculate the impact. If mid to large packets were dominant (say, as much as
> >70% given >80% of traffic is video (ABR etc) per the latest VNI studies ), then
> the overhead impact due to non-compressed SID usage on the traffic would
> be even less over all.
> 
> https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__www.ietf.org_mailman_listinfo_spring&d=DwICAg&c=HAkYuh63rsuhr6
> Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=Fch9FQ82sir-BoLx84hKuKwl-
> AWF2EfpHcAwrDThKP8&m=jmZxWm2Aql61pHCtvKteTsWzTqqt883W1s5m4
> Hg4Fxg&s=Jd6XyAybfJQVqups1svIV2qgpN8t39yQZOjL0HT5UZw&e=

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 30 Mar 2019 14:22:40 +0100

Dear Ron,

*> Furthermore, we should engineer for future requirements.  *

Well let's see if we can meet current requirements first.

Imagine that I have a trusted overlay domain over non trusted and non SR
aware IPv6 underlay (likely operated by completely different team of
engineers).

My POPs are addressed from PA space I am attached to (for example my POPs
are virtual computes in various cloud providers) and I have no freedom to
choose IPv6 prefix to be the same across all such POPs.

That effectively means that unless I insert full 128 bits to the packet
header I will not be able to perform any segment routing in my existing
network as clearly the base of your proposal relies on the fact that you
have common IPv6 prefix (in your appendix 2001:db8::/64).

Bottom line CRH does not work for this deployment model.

*> Generally speaking, we should design for efficiency.  *

True and there is always the question of the right balance. Let's zoom into
control plane aspect of your proposal.

In your proposal your packet destination addresses MUST not be aggregated
nor summarized for the apparatus to work. That is the same mistake as was
made with MPLS exact match where today many networks is suffering with
1000s of /32s all over the place. Now we are talking about the same or even
greater amount of /128s.

No transit L3 network will allow me to inject those /128s for a very valid
reasons. Consider that my network grows I would see value of ISIS area
abstraction as presented this week in LSR. Well that means that again I
would need to pollute my entire domain with host routes to make use of your
idea.

*Conclusion:*

Yes I would like to significantly reduce packet size too that's why I
proposed solution which can accomplish the goal all in control plane by
proper handing of BGP indirection and recursion: REF:
https://slideplayer.com/slide/12853563/

CRH assuming that the described above properties are not showstoppers for
given deployment saves with say stack of five 32 bit SIDs for most common
576 bytes imix packet size 10% of saving when SID size goes down to 8 bits
the saving are 13%.

IMO we should have proper data plane solution as well as control plane
solution not something in between which works only in specific scenarios
are which again breaks basic IP address aggregation.

Kind regards,
Robert.

PS. But if you choose to go ahead with CRH I would highly advise to make
your CRH SID a variable length.

On Fri, Mar 29, 2019 at 7:02 PM Ron Bonica <rbonica=
[email-masked]> wrote:

> Generally speaking, we should design for efficiency.  Furthermore, we
> should engineer for future requirements. For example, in the future,
> service providers may require longer SID lists (5 or more!). We should plan
> for that today.
>
> Likewise, in the future, people may deploy new applications that emit
> short packets. We should also plan for that today.
>
> Seems
> > useful.
> was ~500B

---

From: Mark Smith <[email-masked]>
Date: Sun, 31 Mar 2019 02:07:30 +1100

On Sun, 31 Mar 2019 at 00:23, Robert Raszuk <[email-masked]> wrote:
>
> Dear Ron,
>
> > Furthermore, we should engineer for future requirements.
>
> Imagine that I have a trusted overlay domain over non trusted and non SR aware IPv6 underlay (likely operated by completely different team of engineers).
>
> My POPs are addressed from PA space I am attached to (for example my POPs are virtual computes in various cloud providers) and I have no freedom to choose IPv6 prefix to be the same across all such POPs.
>
> That effectively means that unless I insert full 128 bits to the packet header I will not be able to perform any segment routing in my existing network as clearly the base of your proposal relies on the fact that you have common IPv6 prefix (in your appendix 2001:db8::/64).
>

I can't see that requirement.

These are the steps where the IPv6 DA is determined from the SID.
There is no requirement stated that the IPv6 DAs in the mapping table
must all come from the same /64.

* Search for SID[SL] in the table that maps SID's to IPv6 addresses
and interfaces. If SID[SL] cannot be found in that table, send an ICMP
Parameter Problem, Code 0, message to the Source Address, pointing to
SID[SL], and discard the packet.

* Copy the address associated with SID[SL] to the IPv6 Destination Address.

In the Strict Source Routing example (A.3), the SA and 3 DAs are all
from different /64s.

> > Generally speaking, we should design for efficiency.
>
> True and there is always the question of the right balance. Let's zoom into control plane aspect of your proposal.
>
> In your proposal your packet destination addresses MUST not be aggregated nor summarized for the apparatus to work. That is the same mistake as was made with MPLS exact match where today many networks is suffering with 1000s of /32s all over the place. Now we are talking about the same or even greater amount of /128s.
>
> No transit L3 network will allow me to inject those /128s for a very valid reasons. Consider that my network grows I would see value of ISIS area abstraction as presented this week in LSR. Well that means that again I would need to pollute my entire domain with host routes to make use of your idea.
>
> Conclusion:
>
> Yes I would like to significantly reduce packet size too that's why I proposed solution which can accomplish the goal all in control plane by proper handing of BGP indirection and recursion: REF: https://slideplayer.com/slide/12853563/
>
> CRH assuming that the described above properties are not showstoppers for given deployment saves with say stack of five 32 bit SIDs for most common 576 bytes imix packet size 10% of saving when SID size goes down to 8 bits the saving are 13%.
>
> IMO we should have proper data plane solution as well as control plane solution not something in between which works only in specific scenarios are which again breaks basic IP address aggregation.
>
> Kind regards,
> Robert.
>
> PS. But if you choose to go ahead with CRH I would highly advise to make your CRH SID a variable length.
>
>
> On Fri, Mar 29, 2019 at 7:02 PM Ron Bonica <rbonica=[email-masked]> wrote:
>>
>> Hi Rajiv,
>>

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 30 Mar 2019 16:33:25 +0100

Hi Mark,

1) Can you point me to IGP and BGP extensions which would allow to
propagate such "mapping tables" around ?

2) Is there an agreement that solutions which require additional per SR
path state in both control plane and now in data plane are really something
we should be endorsing here ?

Kind regards,
R.

On Sat, Mar 30, 2019 at 4:07 PM Mark Smith <[email-masked]> wrote:

> > My POPs are addressed from PA space I am attached to (for example my
> POPs are virtual computes in various cloud providers) and I have no freedom
> to choose IPv6 prefix to be the same across all such POPs.
> >
> > True and there is always the question of the right balance. Let's zoom
> into control plane aspect of your proposal.
> >
> > In your proposal your packet destination addresses MUST not be
> aggregated nor summarized for the apparatus to work. That is the same
> mistake as was made with MPLS exact match where today many networks is
> suffering with 1000s of /32s all over the place. Now we are talking about
> the same or even greater amount of /128s.
> >
> > No transit L3 network will allow me to inject those /128s for a very
> valid reasons. Consider that my network grows I would see value of ISIS
> area abstraction as presented this week in LSR. Well that means that again
> I would need to pollute my entire domain with host routes to make use of
> your idea.
> >
> > Conclusion:
> >
> > CRH assuming that the described above properties are not showstoppers
> for given deployment saves with say stack of five 32 bit SIDs for most
> common 576 bytes imix packet size 10% of saving when SID size goes down to
> 8 bits the saving are 13%.
> >

---

From: Ron Bonica <[email-masked]>
Date: Sat, 30 Mar 2019 18:50:46 +0000

Hi Robert,

Thanks for the review.

There is no requirement that all SIDs come from the same prefix. However, I can see how you might get that impression. In two of the examples, all SIDs do come from the same prefix. In the next draft version, I will update the examples so that prefixes come from different prefixes.

I have not yet submitted the control plane documents. However, I can assure you that the will be remarkably short and simple. All you have to do is advertise a capability and a SID to address mapping.

                                                              Ron

Juniper Internal
From: Robert Raszuk <[email-masked]>
Sent: Saturday, March 30, 2019 9:23 AM
To: Ron Bonica <[email-masked]>
Cc: Rajiv Asati (rajiva) <[email-masked]>; [email-masked]; [email-masked]
Subject: Re: [spring] IPv6-compressed-routing-header-crh

Dear Ron,

Conclusion:

Yes I would like to significantly reduce packet size too that's why I proposed solution which can accomplish the goal all in control plane by proper handing of BGP indirection and recursion: REF: https://slideplayer.com/slide/12853563/<https://urldefense.proofpoint.com/v2/url?u=https-3A__slideplayer.com_slide_12853563_&d=DwMFaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=Fch9FQ82sir-BoLx84hKuKwl-AWF2EfpHcAwrDThKP8&m=ewuLblkfLjwoO8YK40dZe30-7hMWm2NinGDf4UsUpJA&s=GOt3LiSwdCnkoijg62t5ij7ZxyrhbAHQ7RQW9E84jhk&e=>

Kind regards,
Robert.

On Fri, Mar 29, 2019 at 7:02 PM Ron Bonica <rbonica=[email-masked]<mailto:[email-masked]>> wrote:
Hi Rajiv,

                                                    Ron

Juniper Internal

> From: spring <[email-masked]<mailto:[email-masked]>> On Behalf Of Rajiv Asati (rajiva)
> To: [email-masked]<mailto:[email-masked]>
> Cc: [email-masked]<mailto:[email-masked]>

--------------------------------------------------------------------
IETF IPv6 working group mailing list
[email-masked]<mailto:[email-masked]>
Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_ipv6&d=DwMFaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=Fch9FQ82sir-BoLx84hKuKwl-AWF2EfpHcAwrDThKP8&m=ewuLblkfLjwoO8YK40dZe30-7hMWm2NinGDf4UsUpJA&s=meQ-iz0QWyQ093HWHIwJyq5tb5j9wctL5eeH3TpRKXc&e=>
--------------------------------------------------------------------

---

From: Mark Smith <[email-masked]>
Date: Sun, 31 Mar 2019 13:38:50 +1100

Hi Robert,

> propagate such "mapping tables" around ?

No, as don't think one specifically exists yet. That can be developed.

There are already similar ones of course, such as those that distribute IP
prefix to MPLS label mapping information.

> we should be endorsing here ?

I think so.

My understanding of what SR is fundamentally about is to reduce control
plane state and processing. The trade-off for reduced control plane state
and processing is to instead carry and encode most or all of that
information or its semantics as per-packet overhead.

If the per-packet overhead becomes too large and expensive, then pushing
some of that information and processing back into the control plane should
be ok, as long as there is still a beneficial overall reduction in control
plane state and processing.

As MPLS SR SIDs are 20 bits, then rounding up to an octet boundary and a 32
bit alignment, I'd think 32 bit SIDs would be adequate to perform SR in an
IPv6 network.

As 32 bit SIDs are also the same size as IPv4 addresses, that may also
create some opportunities to leverage IPv4 support in existing protocols to
suite carrying and processing 32 bit SIDs with some, possibly slight,
modification. For example, perhaps IPv4 Address Family support in OSPFv3
(RFC 5838) could be somehow leveraged to suit SR.

Regards,
Mark.

>> > Imagine that I have a trusted overlay domain over non trusted and non
>> SR aware IPv6 underlay (likely operated by completely different team of
>> engineers).
>> >
>> >> > Very Interesting idea that you presented during SPRING session
>> today. Seems
>> >> > useful.
>> >> >
>> Internet was ~500B

---

From: Robert Raszuk <[email-masked]>
Date: Sun, 31 Mar 2019 16:39:49 +0200

Hi Mark,

> As MPLS SR SIDs are 20 bits, then rounding up to an octet boundary and a
32 bit alignment,
> I'd think 32 bit SIDs would be adequate to perform SR in an IPv6 network.
>
create some opportunities to
> leverage IPv4 support in existing protocols to suite carrying and
processing 32 bit SIDs with some, possibly
> slight, modification. For example, perhaps IPv4 Address Family support in
OSPFv3 (RFC 5838) could be
> somehow leveraged to suit SR.

Thank you for describing your understanding of fundamentals of SR.

I think SR while indeed started with the story of "less control plane is
good for you" now clearly has evolved into not only reduction of control
plane but what can be even more important to some users ability to request
specific behavior via programmed functions of network elements on a per
flow basis without actually per flow or per path signalling or state.

Yes for some it may be very useful feature and I am sure some will call it
overload of data plane or . There is no one size fits all.

With that let's observe that till today SR did not require any new mapping
plane to be distributed in control plane and to be inserted into data
plane. This is clearly a precedent.

Furthermore as we see in companion documents all additional network
functionality is being taken away from SRH and is being shifted to
Destination Options .

As far as mapping plane I already pointed out in my Vector Routing proposal
that we have one already it is called BGP. One needs to also observe that
we as industry worked number of years of protocol suite called LISP
allowing not only very good mapping plane, but also data plane integration.
CC-ing lisp authors for their comments. Note also work for integrating SRv6
with LISP which is already is published.

Since you correctly observed that now SID can be 32 bit and that is similar
to the size of IPv4 my fundamental question is why not use something which
already exists instead of defining some sort of new  from scratch ?

It will be perfectly fine to have full proper SRv6 with SRH and LISP or
Vector Routing as an alternative options. I really do not see a room or
need for yet one more mapping plane. What problem does it solve which would
not be already solved elsewhere ?

Kind regards,
Robert

> As MPLS SR SIDs are 20 bits, then rounding up to an octet boundary and a
> 32 bit alignment, I'd think 32 bit SIDs would be adequate to perform SR in
> an IPv6 network.
>

---

From: Dino Farinacci <[email-masked]>
Date: Sun, 31 Mar 2019 14:38:49 -0700

> As far as mapping plane I already pointed out in my Vector Routing proposal that we have one already it is called BGP. One needs to also observe that we as industry worked number of years of protocol suite called LISP allowing not only very good mapping plane, but also data plane integration. CC-ing lisp authors for their comments. Note also work for integrating SRv6 with LISP which is already is published. 

The main motivationn to have EID-to-SRH mappings in the LISP mapping system is so you can run a LISP overlay on top of an underlay that supports SR. If you want the paths between LISP RLOCs to be “underlay engineered”, then the elements of the SRH header can be part of the RLOCs LISP selects. 

And when you do underlay path optimization, the new SRHs can be registered dynamically to the mapping system.

> Since you correctly observed that now SID can be 32 bit and that is similar to the size of IPv4 my fundamental question is why not use something which already exists instead of defining some sort of new  from scratch ? 
> 
> It will be perfectly fine to have full proper SRv6 with SRH and LISP or Vector Routing as an alternative options. I really do not see a room or need for yet one more mapping plane. What problem does it solve which would not be already solved elsewhere ? 

In the LISP mapping system I run, I was able to simulate (did not implement referenced draft above), an EID-to-SRH mapping by simply having an IPv6 EID map to an RLOC-set that used ELPs (Explicit Locator Paths - where if the ELP is (A,B,C) it means A encaps to B, B encaps to C), but rather than doing the encapsulation we take the (A,B,C) inserted into one packet as an SRH.
 
The point of the exercise was to show the SR authors that the LISP mapping system was a control-plane they could use and depend on (and ready to go).

Dino