Subject: [Sidrops] Talk: RPKI Deployment: Status, Challenges and the Learning-Validator

From: Matthias Waehlisch <[email-masked]>
Date: Mon, 17 Jul 2017 16:36:55 +0200

two comments via the list because we run out of time:

(1) I'm wondering about the statement that the quality of ROAs decreases 
over time. My impression is that the quality improved because of 
excellent training by RIRs and others.

Slide 4 shows absolute values, which is not helpful in this context.

(2) Regarding ROV measurements: "Similar results apparently from 
measurements by Randy Bush and others (didn't yet see details)"

Details are available, easy to find using Google:

* "Towards a Rigorous Methodology for Measuring Adoption of RPKI Route 
Validation and Filtering", https://arxiv.org/abs/1706.04263. Some of 
this work was also presented at the last RIPE meeting: 
https://ripe74.ripe.net/archives/video/46/

Cheers
  matthias

-- 
Matthias Waehlisch
.  Freie Universitaet Berlin, Computer Science
.. http://www.cs.fu-berlin.de/~waehl

---

From: "Carlos M. Martinez" <[email-masked]>
Date: Thu, 20 Jul 2017 17:41:24 +0200

I was about to ask a question on the mic but was late to the line. Maybe 
it’s worth sharing it here:

- How we define a ROA to be “wrong” ? One that invalidates routes or 
one that causes legitimate traffic to be dropped ?

Why? because we’ve seen in many cases ROAs that create lots of 
invalids but validate a less-specific route that covers those invalids

In a way, this even a positive side-effect, sort of vacuum-cleaning your 
routing tables.

I believe analyzing what ROAs are wrong is quite important, but i’d 
believe this particular case should not count as wrong.

thanks!

/Carlos

On 17 Jul 2017, at 16:36, Matthias Waehlisch wrote:

> _______________________________________________
> Sidrops mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sidrops

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Thu, 20 Jul 2017 17:01:32 +0000

If it causes legitimate traffic to fail, then it's wrong.

Thanks,
Jakob.

From: Sidrops [mailto:[email-masked]] On Behalf Of Carlos M. Martinez
Sent: Thursday, July 20, 2017 8:41 AM
To: Matthias Waehlisch <[email-masked]>
Cc: [email-masked]
Subject: Re: [Sidrops] Talk: RPKI Deployment: Status, Challenges and the Learning-Validator

I was about to ask a question on the mic but was late to the line. Maybe it’s worth sharing it here:

  *   How we define a ROA to be “wrong” ? One that invalidates routes or one that causes legitimate traffic to be dropped ?

Why? because we’ve seen in many cases ROAs that create lots of invalids but validate a less-specific route that covers those invalids

In a way, this even a positive side-effect, sort of vacuum-cleaning your routing tables.

I believe analyzing what ROAs are wrong is quite important, but i’d believe this particular case should not count as wrong.

thanks!

/Carlos

Cheers
matthias

_______________________________________________
Sidrops mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/sidrops

---

From: Randy Bush <[email-masked]>
Date: Fri, 21 Jul 2017 09:42:49 +0200

< pedantry >

>   one that causes legitimate traffic to be dropped ?

neither.  a roa is 'wrong' when it does not validate up to the iana TA
or the political TA kludge of the season.

a roa may cover but not validate a particular bgp announcement; though
that announcement may be validated by a different roa.

a roa may not validate a particular announcement which it covers for
which there are no other roas.  the roa is neither right nor wrong; it
merely disagrees with the current bgp announcements.  which needs to be
changed is not obviously determinable.

of course there was the work in new mexico a dozen years ago to create
roas to match the bgp table with a bit of delay.  we all laughed.

randy

---

From: Randy Bush <[email-masked]>
Date: Fri, 21 Jul 2017 09:43:28 +0200

> If it causes legitimate traffic to fail, then it's wrong.

that's the traffic with the evil bit off, right?

randy

---

From: Christopher Morrow <[email-masked]>
Date: Sun, 23 Jul 2017 20:06:49 +0200

> roas to match the bgp table with a bit of delay.  we all laughed.

I took from the preso here that there was work going on to make the NM
tooling (or similar) more operationally useful.. that along with some
reporting / alerting anyway.

---

From: Randy Bush <[email-masked]>
Date: Mon, 24 Jul 2017 09:22:27 +0900

> reporting / alerting anyway.

in which it was shown that the length of time spammers/attackers
mis-announce space turns out to have a long and fat tail.

randy

---

From: Tim Bruijnzeels <[email-masked]>
Date: Tue, 25 Jul 2017 15:49:28 +0200

Hi,

Jakob, not picking on you - but your statement gave me a good angle for what I wanted to say on this.

> On 20 Jul 2017, at 19:01, Jakob Heitz (jheitz) <[email-masked]> wrote:
> 

Sure, but knowing what constitutes legitimate is the trick then. If an announcement has status “invalid”, then this means that the holder of the prefix made one or more authorisation(s) for other announcement(s) for this space, but not for *this* announcement.

Whether this is:
 a) a bug, because the holder wants the announcement but neglected to authorise; or
 b) a feature, because the holder does not want this announcement

That is impossible to tell based only on the observation that there is a disagreement. And I fear that any smart heuristics, e.g. based on announcement history, are in the end only estimated guesswork. It may be smart and it may be right more often than wrong, but it’s still guesswork.

Now, I see value in exposing such differences and helping operators anywhere to understand and debug them and take action, but I have to say that I disagree quite fundamentally on automating decisions on the validator side. I believe that it will inevitably result in announcements that are supposed to be “invalid” (by choice of the holder) to become authorised. This will degrade the benefit of using RPKI Origin Validation for networks that do maintain their ROAs properly. Secondly I believe that ‘fixing’ this on the RP side will remove the incentive for networks to fix their ROAs so that they match their intended announcements. If it hurts, people will fix it. This has been happening in Europe for decades w.r.t. route objects.

Let me provide a further datapoint. We have 4880 organisations that activated their hosted RPKI service. Some organisations only switched on the system, but a total of 2874 organisations actively maintain ROAs. We provide an opt-in service whereby organisations can receive alerts in case we see announcements that are inconsistent with their ROAs. Although this is not mandatory, the majority of organisations (2020) have opted in to this service. The alerts will inform the organisation of any “invalid” or “unknown” announcements. On receiving this organisations can choose to authorise these announcements - but ONLY if they are supposed to be correct (their choice). If they do not authorise the announcement, and it keeps happening, they will receive an email every day. For this reason users can also choose to suppress the warnings on announcements they do not wish to authorise. We have 219 organisations that have at lease one announcement suppressed.

So, in short: I believe that this in itself proves that a lot of our users are actively managing their ROAs, and that quite a few of the “invalids” out there are really supposed to be considered just that: “invalid”

Thanks
Tim

> 	• How we define a ROA to be “wrong” ? One that invalidates routes or one that causes legitimate traffic to be dropped ?

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Tue, 25 Jul 2017 17:55:30 +0000

I could understand if there are providers that would rather accept
the "maybe invalid" than drop them. Dropping prefixes breaks connectivity.
If they can't tell the difference between "matched with wrong AS"
and covered, then they'll just accept them all.

Thanks,
Jakob.

> -----Original Message-----
> From: Tim Bruijnzeels [mailto:[email-masked]]
> Sent: Tuesday, July 25, 2017 6:49 AM
> To: Jakob Heitz (jheitz) <[email-masked]>
> Cc: Carlos M. Martinez <[email-masked]>; Matthias Waehlisch <[email-masked]>; [email-masked]
> Sure, but knowing what constitutes legitimate is the trick then. If an announcement has status “invalid”, then this
> means that the holder of the prefix made one or more authorisation(s) for other announcement(s) for this space, but
> not for *this* announcement.
> 
> Whether this is:
> That is impossible to tell based only on the observation that there is a disagreement. And I fear that any smart
> heuristics, e.g. based on announcement history, are in the end only estimated guesswork. It may be smart and it may
> be right more often than wrong, but it’s still guesswork.
> 
> Now, I see value in exposing such differences and helping operators anywhere to understand and debug them and take
> action, but I have to say that I disagree quite fundamentally on automating decisions on the validator side. I
> believe that it will inevitably result in announcements that are supposed to be “invalid” (by choice of the holder)
> to become authorised. This will degrade the benefit of using RPKI Origin Validation for networks that do maintain
> their ROAs properly. Secondly I believe that ‘fixing’ this on the RP side will remove the incentive for networks to
> fix their ROAs so that they match their intended announcements. If it hurts, people will fix it. This has been
> happening in Europe for decades w.r.t. route objects.
> 
> Let me provide a further datapoint. We have 4880 organisations that activated their hosted RPKI service. Some
> organisations only switched on the system, but a total of 2874 organisations actively maintain ROAs. We provide an
> opt-in service whereby organisations can receive alerts in case we see announcements that are inconsistent with
> their ROAs. Although this is not mandatory, the majority of organisations (2020) have opted in to this service. The
> alerts will inform the organisation of any “invalid” or “unknown” announcements. On receiving this organisations can
> choose to authorise these announcements - but ONLY if they are supposed to be correct (their choice). If they do not
> authorise the announcement, and it keeps happening, they will receive an email every day. For this reason users can
> also choose to suppress the warnings on announcements they do not wish to authorise. We have 219 organisations that
> have at lease one announcement suppressed.
> 
> So, in short: I believe that this in itself proves that a lot of our users are actively managing their ROAs, and
> that quite a few of the “invalids” out there are really supposed to be considered just that: “invalid”
> 
> Thanks
> Tim
> 
> 
> 
> 
> 
> >
> > Thanks,
> > Jakob.
> >

---

From: Tim Bruijnzeels <[email-masked]>
Date: Wed, 26 Jul 2017 10:25:14 +0200

> On 25 Jul 2017, at 19:55, Jakob Heitz (jheitz) <[email-masked]> wrote:
> 

True, but they are really “invalid” in the validation sense. You can choose to ignore this, or the holder can authorise them so they will be valid - which would be the better solution.

I believe a lot of this comes down to a burden of proof kind of concept - if the connectivity breaks because the holder only partially authorised their announcements - then they are to ‘blame’ and they have the power to fix.

> and covered, then they'll just accept them all.

I believe it would be better then to make this difference available to operators, rather than artificially marking things as valid.

Accepting more specific announcements leaves one vulnerable though to more specific announcements done with a spoofed AS. Therefore, if the path cannot be trusted, e.g. by use of BGPSec, doing blanket acceptance policy to accept “invalid length” introduces a major vulnerability.

That said, if you as an operator have other ways of determining whether you can trust the path, e.g. because it’s a leaf customer of yours and its path length is 1, or because the path is short and you know and trust all the ASNs on it, then I think there is something to be said for a pragmatic policy of accepting more specifics here. It doesn’t give the full RPKI+Origin Validation+BGPSec certainty, but it’s probably a big step up from today without causing too many issues.

I am quite sure that talking about this will also generate quite some discussion between more purist and more pragmatic world views on security, but I think it’s probably a good discussion to have.

---

From: Randy Bush <[email-masked]>
Date: Thu, 27 Jul 2017 05:18:15 +0900

> Accepting more specific announcements leaves one vulnerable though to
> more specific announcements done with a spoofed AS.

one of the principle goals of the whole effort

---

From: George Michaelson <[email-masked]>
Date: Thu, 27 Jul 2017 08:48:58 +1000

I am probably being simplistic, but my memory of this conversation
during happier more innocent days was that a covering prefix's use of
a ROA signalled that RPKI applied to all origin-as assertions made
below it, as more specific. Thats how I understood us to be discussing
it. For this mail, I discount a ROA which fails cryptographic checks:
its being spoofed or is otherwise mechanistically in error and cannot
apply.

The address holder is the sole signatory of a ROA. The address holder
of the covering prefix has a delegation authority over the more
specific. If they had transferred the resources, there would be an
overt 'hole' in their addresses and it would have a different RFC3779
representation, so there is no 'prefix' as such to announce: if they
continue to announce the aggregate covering prefix when there is an
explicit transfer out of the more specific, that wasn't engineered for
in the model. There can be no covering ROA which refers to it since
the 3779 state is not legal under the signing (even under modified
validation, the ROA which overclaims is going to be invalid)

So excluding explicit transfer, I believe the 'reductionist' sense of
a more specific is: either its authorized by an explicit ROA referring
to itself, or its implied as legal in the values specified in a
covering aggregate ROA, or its not actually permissable. I can't see
any other state implied. I do not see this as a tri-state situation:
if the aggregate over it is legal, then all more specifics of that
aggregate require a ROA or implied consent from the aggregate to be
true. (basically, the same AS and a length-permitting setting)

Now, as to intent: if somebody *wants* the more specific to exist,
there is a path out: make a ROA. If this is a timeliness situation,
they haven't published one, thats a transitional fault. if it
persists, its an operational failure. But its not in system to say
"oh, they probably meant this" -Thats not how validation is meant to
work.  Do we really want to move to a world where crypto intent and
declared validation semantics can be _interpreted_ ?

Why do a ROA, for the covering aggregate, if you didn't intend
applying ROA semantics to all addresses in the covering ROA? The
semantics should be concrete. Not inferential.

What am I missing here? What did I not understand? 'inferring' intent,
belief, which is not backed by the literal meaning of the ROAs which
exist, and their cryptographic checks, feels to me like a losing game.