Subject: [babel] source and tos routing thoughts

From: Dave Taht <[email-masked]>
Date: Thu, 13 Apr 2017 21:38:23 -0700

>The bit that worries me is what to do if we want to use 
>Source-and-ToS-specific routing (or the intersection with any 
>future extension that also uses a new AE)?

>This choice suggests that we would need a new AE for any combination
>of existing extensions (and any future extension must define those
>new AE values, deciding which other extensions are compatible).
>However, a whole octet looks enough to code all these possibilities.

>Could you please tell your thoughts on this?

A) TOS is an obsolete term, dscp or diffserv are more current, and have
a specific meaning of 64 values, of which, something like 23 are
defined. Two bits of the former tos field are reserved for ECN.

This asks the question - do you intend "tos routing" to be on those
64 values or the whole shmere?

B) What are your promotion/demotion rules for the TOS routed links?

If no AF42 route is available, do you go to AF41? to best effort?
Drop the packets on the floor? Do you have to cover the entire
256 values in the protocol? This explosion of what is already
34 bytes of data (src/dst plen/plen) into (potentially) a set of
routing tables 256 times bigger that might prove troublesome.

C) Presently, in linux, you can set ecn capability on and off on a per
route basis, which is used, for example, along with congctl, to make
DCTCP work in places where it is safe to run it on a portion of the
network, and unsafe elsewhere.

ip route is showing the following options these days:

OPTIONS := FLAGS [ mtu NUMBER ] [ advmss NUMBER ] [ as [ to ] ADDRESS ]
           [ rtt TIME ] [ rttvar TIME ] [ reordering NUMBER ]
           [ window NUMBER] [ cwnd NUMBER ] [ initcwnd NUMBER ]
           [ ssthresh NUMBER ] [ realms REALM ] [ src ADDRESS ]
           [ rto_min TIME ] [ hoplimit NUMBER ] [ initrwnd NUMBER ]
           [ features FEATURES ] [ quickack BOOL ] [ congctl NAME ]
           [ pref PREF ]

FEATURES: ecn

Somewhat offtopic:

D) There are a lot of other characteristics of routes that are
"interesting". In my case, for example, I'd rather dearly love it if
initcwd (& ssthresh) could be massively reduced for routes exiting the
default gw on slow networks. (initcwnd of 2, etc)

Similarly, mtu could be advertised instead of probing.

Back on topic

So while "tos" can be represented, what are the use cases
for it?

---

From: Juliusz Chroboczek <[email-masked]>
Date: Fri, 14 Apr 2017 19:00:35 +0200

> B) What are your promotion/demotion rules for the TOS routed links?

As far as I can tell at that early stage, any ordering would work.
Gwendoline and I have slightly different opinions here -- I'd like her to
do the simple thing first (TOS exact match, or use non-specific route),
while she'd like to do something more exciting.

> a set of routing tables 256 times bigger that might prove troublesome.

The idea is that just a small set of TOS are announced, the remainder
follow the non-specific route.  See the use-case below.

> route basis

I'm not sure how that's related to routing.  You are not seriously
suggesting that packets should be routed differently depending on the
value of the ECN bits?

> So while "tos" can be represented, what are the use cases for it?

The main use case is due to Nexedi.  They're considering a network with
multiple edge links, some of which have low latency and jitter but are
expensive (MPLS leased lines, most probably), some of which are cheap but
jittery (tunnels over the public Internet).  They want voice traffic to
flow over the expensive links, and data traffic to use the cheap ones.
They're not using source-specific routing in their network, and they only
really care about IPv6, which makes the problem somewhat more tractable.

-- Juliusz

---

From: Dave Taht <[email-masked]>
Date: Fri, 14 Apr 2017 11:58:43 -0700

> follow the non-specific route.  See the use-case below.

I do like the idea of solving for each best route for a different goal(s).

( https://github.com/dtaht/libv6/tree/master/clients/bellman-ford )

> suggesting that packets should be routed differently depending on the

Me? Serious? *never*. :)

The properties of a route announcement via babel are potentially
different from the actual encoding of the packets on the wire.

Here what you are proposing is a 1x2 (if you have a more specific tos,
use that, otherwise, best effort)... in which case you don't need 256
values for it, 64, or 23.

And that tos alone as a determination of what to do with a route doesn't
seem like enough.

actual example, from my case. BBR doesn't support ECN, and stuff going
out my default gateway gets initcwnd cut, and another portion of the
network uses dctcp and ecn.

root@nemesis:~# ip route
default via 172.22.232.1 dev enp2s0  proto babel  initcwnd 2 features congctl bbr
...
172.22.148.0/22 via 172.22.232.154 dev enp2s0  proto babel rtt 1ms features ecn congctl dctcp
...

> really care about IPv6, which makes the problem somewhat more tractable.

Recently (4.11) much progress was made towards disambiguating tunneled
voice and bulk traffic through fq_codel'd bottleneck backhaul links. The
behavior before, included 100+ms of induced latency and jitter:

http://www.taht.net/~d/ipsec_fq_codel/oldqos.png

The behavior afterwards, was 2ms:

http://www.taht.net/~d/ipsec_fq_codel/newqos.png

We haven't added this feature to sch_cake, per se, as that does also
already respect the dscp bits with similar results. Still, this strikes
me as a good addition to the network and lessens the need to do much
with "tos as tos" in the routing protocol.

I know, of course, that convincing the world to just run fq_codel on
everything is something of an uphill battle, my meta-point is merely
that if you wish to solve for two different goals (high bandwidth vs low
latency), and then express that in the routing protocol, that it doesn't
need to be a direct representation of the tos bits. You'd express "this
bit represents a low latency, expensive, low bandwidth route", and then
map a selection of tos bits on top of it to go out that way.

---

From: Juliusz Chroboczek <[email-masked]>
Date: Sat, 15 Apr 2017 01:12:18 +0200

> voice and bulk traffic through fq_codel'd bottleneck backhaul links.

Dave, you must distinguish queueing/forwarding from routing.  Routing is
about deciding where a packet is sent; forwarding is about actually
sending it that way, which might involve queueing.  Gwendoline's work is
about TOS-sensitive *routing*, and is completely orthogonal to the
queueing policy.

Let me be even more specific.  Gwendoline's work is about using the ToS
octet as part of the decision where to send a given packet.  Once
Gwendoline has decided where the packet goes, then your favourite AQM
kicks in to forward the packet -- but that's none of Gwendoline's business.

-- Juliusz

---

From: Dave Taht <[email-masked]>
Date: Fri, 14 Apr 2017 16:37:51 -0700

Juliusz Chroboczek <[email-masked]> writes:

> sending it that way, which might involve queueing.  Gwendoline's work

I certainly do distinguish between the two, I was pointing out the
demand for this use case sprung from a choice of route types caused
(in part) by bad queuing.

To me this factors into a different way to calculate the metric
(optimizing for jitter and latency for a given route) and to come up
with a "best route" for that.

rather than into any direct tos representation.

> is

> kicks in to forward the packet -- but that's none of Gwendoline's

Well, for the stated use case of optimizing for voice traffic, "VA" -
voice admit, and EF - are two of the most common markings. I've also
seen CS2, CS3 (sip), CS4, CS5 (originally intended for video), CS6, and
desparate folk, even trying CS7.

Google has put "goog" congestion control into their webrtc, which uses
AF41 for both voice and video on the same tuple. There is a working
group, stalled out on something even more complex:

https://tools.ietf.org/html/draft-ietf-tsvwg-rtcweb-qos-18

So a single dcsp value, carried by the routing protocol, cannot cover
the stated use case, except in limited, highly controlled circumstances.

Perhaps shipping a list of tos values per that route? Or, as I was
suggesting, using up a bit (somewhere) as a tag saying, please use this
for voice traffic. 

and then the daemon would insert appropriate routes based on all the
needed dscp values.

see also

https://datatracker.ietf.org/doc/rfc7657/?include_text=1

https://en.wikipedia.org/wiki/Differentiated_services

About the only dscp value I like is CS1, which means background, except
where it doesn't.

>
> -- Juliusz
>
> _______________________________________________
> babel mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/babel