Subject: [babel] Mandatory bits and compression

From: Juliusz Chroboczek <[email-masked]>
Date: Tue, 18 Apr 2017 18:18:37 +0200

An Update TLV with an unknown mandatory sub-TLV cannot be simply
ignored -- it must be parsed and the compression state updated
accordingly, only then can it be thrown away.  Otherwise implementations
end up with different compression states depending on the set of
extensions they understand.

Matthieu suggests dropping support for compression, but compression
reduces the number of packets by close to 40% in large double-stack
networks, so I'm a little wary of letting it go.

-- Juliusz

---

From: David Schinazi <[email-masked]>
Date: Tue, 18 Apr 2017 10:13:26 -0700

I think there's value in compression, and updating the decompressor
state before checking for unknown mandatory sub-TLVs shouldn't add
complexity to the code. As long as we clearly define this in the spec,
I'm fine with your proposal.

David

> On Apr 18, 2017, at 09:18, Juliusz Chroboczek <[email-masked]> wrote:
> 
> _______________________________________________
> babel mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/babel

---

From: Toke Høiland-Jørgensen <[email-masked]>
Date: Tue, 18 Apr 2017 19:33:23 +0200

Juliusz Chroboczek <[email-masked]> writes:

> accordingly, only then can it be thrown away. Otherwise
> implementations end up with different compression states depending on
> the set of extensions they understand.

Hmm, while in principle that's not too onerous, I wonder if it is
something that is likely to be overlooked by implementers. Which would
lead to subtle and hard to debug bugs...

>From what to what? Absolute numbers matter.

I didn't bother to implement compression on the sender side for Bird
(but it will parse it, of course). Have not looked into how much of a
difference it makes for my use cases, just put it on the "later" list to
avoid the complexity.

How does the prefix compression compare to just gzip'ing the whole
packet? ;)

-Toke

---

From: Markus Stenberg <[email-masked]>
Date: Tue, 18 Apr 2017 21:13:35 +0300

> How does the prefix compression compare to just gzip'ing the whole

Ironically enough this is exactly what I was thinking of (although something with less computationally heavy compression would be preferable on my opinion). 

Just for amusement value, I did quick study: pybabel unittests generate 287 (roughly) packets. Of them, only 36 compress better with gzip than with the default prefix encoding, but actually _every_ update I can briefly see compresses better with gzip. For them, typically size of packet roughly halves. 

(gzip doesn’t help much with small packets, which the rest of the packets are.)

At a guess, every payload > 80 bytes, gzip wins hands down based on my very limited dataset.

e.g. former: raw Babel encoding. later: Babel encoding gzipped.

 52 / 56
 58 / 70
 67 / 66
 67 / 67
 92 / 73
 92 / 75 
 117 / 80
 117 / 83
 142 / 89
 158 / 99
 167 / 93
 192 / 94
 208 / 106
 208 / 107
 208 / 108
 224 / 109
 224 / 111

If one really cared about size of stuff on the link, I’d specify something like optional LZ4 with first bit on the wire indicating compression or not. Use compressed version if it is smaller, and non-compressed otherwise.

Cheers,

-Markus

---

From: Gabriel Kerneis <[email-masked]>
Date: Tue, 18 Apr 2017 22:46:30 +0200

On Tue, Apr 18, 2017 at 8:13 PM, Markus Stenberg <[email-masked]>
wrote:

> If one really cared about size of stuff on the link, I’d specify something
> like optional LZ4 with first bit on the wire indicating compression or not.
> Use compressed version if it is smaller, and non-compressed otherwise.
>

Meaning every babeld implementation needs to embed a gzip decoder (or you
need some feature negotiation mechanism, ahem). I'm not saying it is
necessarily a bad idea, but several people here seem to care about minimal
and simple implementations.

Gabriel

---

From: Juliusz Chroboczek <[email-masked]>
Date: Wed, 19 Apr 2017 03:50:12 +0200

Is there a proposal for rfc6126bis here, or are you guys just enjoying
slagging off my poor taste ;-)

FWIW, if I were to redo Babel today in a completely incompatible manner,
I'd probably keep the stateful parser.  Sharing router-ids and next hops
between Update TLVs is a simple and effective optimisation, and once you
have a stateful parser anyway, adding prefix compression doesn't add a lot
of complexity.  I'd probably encode it differently, with the Router-ID and
Next-Hop TLVs replaced with sub-TLVs of Update, but these are mere
details.

(Matthieu has suggested that the stateful parser should be replaced with
a dictionary of router-ids at the beginning of a packet.  While simple to
parse, that's a little more complicated to generate, so I'm not sure where
the tradeoff lies.)

I think the problem with compression is not the implementation complexity,
but the fragility of the state -- compression tends to make extensions
more difficult to design and specify.  I'm not sure that justifies
avoiding compression, it just means that extensions should not be
over-optimised, they should simply avoid using compression.

-- Juliusz

---

From: Juliusz Chroboczek <[email-masked]>
Date: Wed, 19 Apr 2017 04:04:02 +0200

>> like optional LZ4 with first bit on the wire indicating compression or
>> not. Use compressed version if it is smaller, and non-compressed
>> otherwise.

Point is, the main purpose of compression is to reduce the number of
link-layer frames being used for a full table dump (think WiFi, with its
outrageous per-frame overhead; yeah, I know about frame aggregation).  So
in order to compress effectively, you need to keep track of the number of
bytes accumulated in the compressed buffer, so that you flush your buffer
just before you reach the MTU.

Not saying this couldn't be done with LZ4 or something, but it's not
a simple matter of accumulating a packet and sending it off to the
compressor.

> Meaning every babeld implementation needs to embed a gzip decoder (or
> you need some feature negotiation mechanism, ahem).

Agreed.

-- Juliusz