Subject: [Bier] AD review of draft-ietf-bier-architecture-06

From: Alia Atlas <[email-masked]>
Date: Thu, 15 Jun 2017 18:22:48 -0400

First, I would like to thank the authors - Ice, Eric, Andrew, Tony, and Sam
- for an excellently written and very clear document which forms the
foundation for this technology. I wish I saw many more drafts with this
level of clarity, thought, and details.  I would also like to thank the
BIER WG for the reviews and discussions that have led to this quality.

As is customary, I have done my AD review.  I do have a few minor issues
that need to be resolved, but I am comfortable requesting that the IETF
Last Call start immediately, with the assumption that the excellent &
responsive authors will resolve these issues and issue an updated draft
quite rapidly.

My goal is to have this document on the IESG telechat of July 7 so that it
will either be in the RFC Editor queue or the WG will be aware of any
issues before IETF 99.

Minor:

1) Sec 4.1: "its BFR prefix"  Is it possible to have multiple BFR prefixes
for the same BFR?  I am thinking about renumbering cases (for acquisitions)
as well as migrations from IPv4 to IPv4 + IPv6 and then to IPv6.
Conceptually, I see no reason that there couldn't be multiple BFR prefixes
for the same BFR as long as they really are that BFR's prefixes.

2) Sec 4.2: This is missing the procedures for removing the BIER header for
the payload - which would presumably include potential details such as TTL
inheritance and DSCP/EXP handling....

3) Sec 6.5:  This has a block of "pseudocode", but it isn't surrounded by
the <CODE_BEGINS> and <CODE_ENDS> indications.  Those are useful because
there are different copyright terms for code in RFCs versus the text of
RFCs.  It would be worth taking a look at the difference and deciding
whether using <CODE_BEGINS> and <CODE_ENDS> would be useful.

4) Sec 6.6.1:"At BFER-D, the BIFT entry (not pictured) for BFR-id 1 will
specify an
   F-BM of 0000 and a BFR-NBR of BFR-D itself."
Sec 6.6.2: "At BFER-D, the BIFT entry (not pictured) for BFR-id 1 will
specify an
   F-BM of 0000 and a BFR-NBR of BFR-D itself."  and "At BFER-E, the BIFT
entry (not pictured) for BFR-id 3 will specify an F-BM of 0000 and a
BFR-NBR of BFR-E itself."
I don't understand why the F-BM would be 0000 and not 0001 nor how the
packet's BitString would end up cleared if the F-BM were 0000.   I suspect
a typo, but it is repeated thrice...  I will note that in Sec 6.5 after the
pseudocode, it says " It also assumes that the corresponding F-BM has only
one bit set, the bit representing the BFER itself."  which appears to
contradict the two examples.

5) Sec 6.9:"This may be as simple as finding the IGP unicast next hop to
the child node, and pushing on (above the BIER-MPLS label advertised by the
child) the MPLS label that the IGP next hop has bound to an address of the
child node.  (If
   for some reason the unicast tunnel cannot be an MPLS tunnel, any
   other kind of tunnel can be used, as long as it is possible to
   encapsulate MPLS within that kind of tunnel.)"
Given the generalization of the BIER encapsulation & possibility of IPv6
encapsulations & that a BIER-MPLS label hasn't been introduced,  could this
be rephrased a bit more generally?  The concept, obviously, is fine.
The same applies to "That way, the TTL of the BIER-MPLS label constrains
only the number of BFRs that the packet may traverse, not the total number
of hops."
which could easily be rewritten as "...the TTL of the BIER header
constrains..."

6) Sec 10.2.1: "This fallback procedure will work best if the value of F is
   established by the architecture, rather than by provisioning."  Since
this is the
architecture and it does specify that all BFRs must support a
BitStringLength of 256 - surely this is exactly the point to establish the
value of F and there isn't a better time. :-)

Nit:

a) End of Section 3:"An encapsulation for use in MPLS networks is described
in
   [MPLS_BIER_ENCAPS]"
Given that encap is also now a generic encapsulation that will work for
Ethernet, this statement could be expanded to be more complete.

b) Sec 4.3: "Since BIER is, in effect, a new type of "tunneling
   technology", some extensions to the MVPN signaling are needed in
   order to properly interface the multicast flow overlay with the BIER
   layer.  These will be specified in a companion document."
I think that this is draft-ietf-bier-mvpn, so it'd be helpful to have an
informative reference to it.

Regards,
Alia

---

From: IJsbrand Wijnands <[email-masked]>
Date: Fri, 16 Jun 2017 09:54:05 +0200

Hi Alia,

> On 16 Jun 2017, at 00:22, Alia Atlas <[email-masked]> wrote:
> 
> First, I would like to thank the authors - Ice, Eric, Andrew, Tony, and Sam - for an excellently written and very clear document which forms the foundation for this technology. I wish I saw many more drafts with this level of clarity, thought, and details.  I would also like to thank the BIER WG for the reviews and discussions that have led to this quality.

Many thanks for the nice words, the BIER WG has been the most pleasant WG ever. It is the Woodstock of WG’s, where the attendees in a few years from now will say “Yes, I was there” :-) Special thanks to Eric, without his writing (and consensus capturing) skills this document would not have reach this level of quality!

> 
> As is customary, I have done my AD review.  I do have a few minor issues that need to be resolved, but I am comfortable requesting that the IETF Last Call start immediately, with the assumption that the excellent & responsive authors will resolve these issues and issue an updated draft quite rapidly.
> 
> My goal is to have this document on the IESG telechat of July 7 so that it will either be in the RFC Editor queue or the WG will be aware of any issues before IETF 99.
> 
> Minor:
> 
> 1) Sec 4.1: "its BFR prefix"  Is it possible to have multiple BFR prefixes for the same BFR?  I am thinking about renumbering cases (for acquisitions) as well as migrations from IPv4 to IPv4 + IPv6 and then to IPv6.  Conceptually, I see no reason that there couldn't be multiple BFR prefixes for the same BFR as long as they really are that BFR's prefixes. 

Ice: I don’t see "its BFR prefix” in 4.1, but 4.2. Are you asking to allow a router to have say two different BFR-Prefixes for the same BFR-id? I agree if the BFR-Prefixes are on the same router it would not hurt, but it would mess up the duplicate BFR-id detection in the rest of the network. A BFR-id that is owned by multiple BFR-Prefixes indicates an inconsistency and will make the BFR-id to be removed from the BIFT.

>  
> 
> 2) Sec 4.2: This is missing the procedures for removing the BIER header for the payload - which would presumably include potential details such as TTL inheritance and DSCP/EXP handling….

Ice: Yes, part of the procedures should also be “Disposition” of the BIER header.

> 
> 3) Sec 6.5:  This has a block of "pseudocode", but it isn't surrounded by the <CODE_BEGINS> and <CODE_ENDS> indications.  Those are useful because there are different copyright terms for code in RFCs versus the text of RFCs.  It would be worth taking a look at the difference and deciding whether using <CODE_BEGINS> and <CODE_ENDS> would be useful.

Ice: I’ve checkout some examples of the use of <CODE_BEGINS>. Its mostly used today for YANG models that actual document code that compiles. In our example (pseudo) its code that does not really compile, so I don’t think its necessary to add this.

> 
> 4) Sec 6.6.1:"At BFER-D, the BIFT entry (not pictured) for BFR-id 1 will specify an
> Sec 6.6.2: "At BFER-D, the BIFT entry (not pictured) for BFR-id 1 will specify an
>    F-BM of 0000 and a BFR-NBR of BFR-D itself."  and "At BFER-E, the BIFT entry (not pictured) for BFR-id 3 will specify an F-BM of 0000 and a BFR-NBR of BFR-E itself."
> I don't understand why the F-BM would be 0000 and not 0001 nor how the packet's BitString would end up cleared if the F-BM were 0000.   I suspect a typo, but it is repeated thrice...  I will note that in Sec 6.5 after the pseudocode, it says " It also assumes that the corresponding F-BM has only one bit set, the bit representing the BFER itself."  which appears to contradict the two examples.

Ice: I think you’re right. The F-BM should be 0001, otherwise the procedure to clear the bit would also not work. We’ll fix it, good catch!

> 
> 5) Sec 6.9:"This may be as simple as finding the IGP unicast next hop to the child node, and pushing on (above the BIER-MPLS label advertised by the child) the MPLS label that the IGP next hop has bound to an address of the child node.  (If
> Given the generalization of the BIER encapsulation & possibility of IPv6 encapsulations & that a BIER-MPLS label hasn't been introduced,  could this be rephrased a bit more generally?  The concept, obviously, is fine.
> The same applies to "That way, the TTL of the BIER-MPLS label constrains only the number of BFRs that the packet may traverse, not the total number of hops."
> which could easily be rewritten as "...the TTL of the BIER header constrains…"

Ice: Ok.

>    established by the architecture, rather than by provisioning."  Since this is the 
> architecture and it does specify that all BFRs must support a BitStringLength of 256 - surely this is exactly the point to establish the value of F and there isn't a better time. :-)

Ice: Sec 6.10.1? Ok, maybe we can add that it is RECOMMENDED that the fallback disposition bit-string length is 256...

> 
> Nit:
> 
> a) End of Section 3:"An encapsulation for use in MPLS networks is described in
>    [MPLS_BIER_ENCAPS]"
> Given that encap is also now a generic encapsulation that will work for Ethernet, this statement could be expanded to be more complete.

Ice: Ok.

> I think that this is draft-ietf-bier-mvpn, so it'd be helpful to have an informative reference to it.

Ice: Ack!

Many thanks Alia, great comments,

Ice.

---

From: Alia Atlas <[email-masked]>
Date: Fri, 16 Jun 2017 10:54:46 -0400

Hi Ice,

On Fri, Jun 16, 2017 at 3:54 AM, IJsbrand Wijnands <[email-masked]> wrote:

> > First, I would like to thank the authors - Ice, Eric, Andrew, Tony, and
> Sam - for an excellently written and very clear document which forms the
> Many thanks for the nice words, the BIER WG has been the most pleasant WG
> ever. It is the Woodstock of WG’s, where the attendees in a few years from
> now will say “Yes, I was there” :-) Special thanks to Eric, without his
> writing (and consensus capturing) skills this document would not have reach
> this level of quality!
>
> >
> > My goal is to have this document on the IESG telechat of July 7 so that
> it will either be in the RFC Editor queue or the WG will be aware of any
> issues before IETF 99.
> >
> > Minor:
> >
> > 1) Sec 4.1: "its BFR prefix"  Is it possible to have multiple BFR
> prefixes for the same BFR?  I am thinking about renumbering cases (for
> acquisitions) as well as migrations from IPv4 to IPv4 + IPv6 and then to
> IPv6.  Conceptually, I see no reason that there couldn't be multiple BFR
> prefixes for the same BFR as long as they really are that BFR's prefixes.
>
> Ice: I don’t see "its BFR prefix” in 4.1, but 4.2. Are you asking to allow
> a router to have say two different BFR-Prefixes for the same BFR-id? I
> agree if the BFR-Prefixes are on the same router it would not hurt, but it
> would mess up the duplicate BFR-id detection in the rest of the network. A
> BFR-id that is owned by multiple BFR-Prefixes indicates an inconsistency
> and will make the BFR-id to be removed from the BIFT.
>

Sorry - yes - Sec 4.2.
I was assuming that one would use the Router ID to determine ownership,
rather than the BFR-Prefix. Regardless, clarifying it would be useful to
avoid issues...

The real ask is to think through address migration & network merge cases.
For instance, having multiple BFR-Prefixes might not help, if the Router ID
has to change as well.  RFC 7705 talks about AS migration, which is what
increased my awareness of  the frequency of network changes due to business
changes (mergers, acquisitions, selling, etc.).

I'm not convinced that merely having multiple BFR-Prefixes would help with
the issue; I'd have to think it through in significantly more detail.

> >
> >
> > 2) Sec 4.2: This is missing the procedures for removing the BIER header
> for the payload - which would presumably include potential details such as
> TTL inheritance and DSCP/EXP handling….
>

Of course, the drafts are written assuming that - this is just a
documentation oversight.

> >
> > 3) Sec 6.5:  This has a block of "pseudocode", but it isn't surrounded
> by the <CODE_BEGINS> and <CODE_ENDS> indications.  Those are useful because
> Ice: I’ve checkout some examples of the use of <CODE_BEGINS>. Its mostly
> used today for YANG models that actual document code that compiles. In our
> example (pseudo) its code that does not really compile, so I don’t think
> its necessary to add this.
>

Ok - it's very C-like pseudo-code :-)

> >    F-BM of 0000 and a BFR-NBR of BFR-D itself."  and "At BFER-E, the
> BIFT entry (not pictured) for BFR-id 3 will specify an F-BM of 0000 and a
> Ice: I think you’re right. The F-BM should be 0001, otherwise the
> procedure to clear the bit would also not work. We’ll fix it, good catch!
>

The same error is repeated 3 times - so I wasn't certain if there was a
previous idea behind it.

> constrains…"
> Ice: Sec 6.10.1? Ok, maybe we can add that it is RECOMMENDED that the
> fallback disposition bit-string length is 256...
>

K - up to you and WG as to whether it is RECOMMENDED or SHOULD.  This just
looked like an aspect
that could be updated now.

> Many thanks Alia, great comments,

Thanks :-)   If you could get an updated draft out ASAP, it will help
others doing the reviews.

Regards,
Alia

---

From: Antoni Przygienda <[email-masked]>
Date: Fri, 16 Jun 2017 18:33:38 +0000

Inline …

--- tony

      Prz: All procedures are written with a tactic assumption that we have a single BFR-prefix, the implied conceptual model has been spread in language but bit below it is as rough UML (it is probably easiest derived from the ISIS encoding document & the restrictions put on TLV). Having said that, BIER does NOT govern how many such prefixes can show up but reading the ISIS spec we clearly govern for subdomains in BIER Info sub-TLV that

“This sub-TLV MAY appear multiple

   times in a given prefix-reachability TLV - once for each sub-domain

   supported in the associated topology”

We could say more explicitly that “if the subdomain shows up twice ignore it” or “ignore the whole prefix” or “ignore subdomain” or “behavior unspecified”. Thinking through it however whether the domains show up in a single prefix or multiple does not even matter except for possibly BIFT computation where you may need an IP address for a next-hop but then again (and practically; when flooding you really prefer all in a single flooded packet rather than stitching the bits). BIFT construction/computation is implementation dependent so maybe the spec does not have to govern ANYTHING about BFR prefix. With that, the <: BFR-->(1)BFR prefix :> multiplicity may be left out without making the model=BIER architecture defective.

BTW, if anybody sees deviation in this picture of what BIER architecture consists off, please speak up now or hold your piece forever ;-)

Prz: Correct, we need clear procedures for the bits in the MPLS label we’re silent about.

Prz: Looked @ it as well and yes, it needs the fix. Alia, a most impressive diving catch ;-)

Prz: I suggest to say that F=256 simply and be done with it …

    Ice: Ok.

    Ice: Ack!

Prz: I suggest to refer to more generic “overlay signaling” [we just published EVPN] and refer to MPVN draft as a specific incarnation of such an “overlay signaling”

--- tony

---

From: Eric C Rosen <[email-masked]>
Date: Mon, 19 Jun 2017 10:41:05 -0400

So here is my take on this discussion:

1. The purpose of the BFR-prefix is to have an IP address that is 
associated with a BFR-id.  I don't think we want to allow arbitrary 
numbers of BFR-prefixes to have the same BFR-id.  That creates the 
possibility for lots of error cases.  For one thing, we'd have a problem 
if the BFR-prefixes corresponding to a given BFR-id aren't all reachable 
via the same path.  For another, when we pass a BFR-prefix in the 
overlay signaling, we'd have a whole bunch of issues in figuring out 
which one to pass when.

Having said that, I don't think there is actually anything in the 
architecture draft that would prohibit a BFR-id from corresponding to 
both an IPv4 address and an IPv6 address.

BTW, the "router id" is not a meaningful concept in some of the BIER 
signaling planes, e.g., the BGP extensions for BIER signaling.

2. The issue of how to combine two BIER domains into one is something we 
haven't thought about a lot, but I don't think having multiple 
BFR-prefixes is much of a help.  The problem is that the BFR-ids have to 
change.

3. Renumbering the routers is always an interesting problem, but I don't 
really see that the problem is BIER-specific.

4. With regard to the disposition of the BIER header, I'm not sure that 
there is much to say.  As in MPLS generally, whether the TTL is 
inherited by the payload or whether the TC bits have any impact is a 
matter of local policy at the BFER.  We can add a sentence saying that.

5. Yes, the F-BM at a BFER's BIFT entry for itself should have that that 
BFER's own bit set and the rest 0.  However, I will point out that if 
the F-BM is 0, the textual description of the forwarding procedure in 
section 6.5 still works, even if the pseudocode does not ;-)  Will fix.

6. I'll look at the places we mention MPLS to see if there's a 
straightforward way to say that the same applies to non-MPLS 
encapsulations.

7. I like the idea of modifying section 6.10.1 to require support for a 
BML of 256.  However, RECOMMENDED is probably a better match to the WG 
consensus.

---

From: Alia Atlas <[email-masked]>
Date: Mon, 19 Jun 2017 12:09:53 -0400

Hi Eric,

On Mon, Jun 19, 2017 at 10:41 AM, Eric C Rosen <[email-masked]> wrote:

> associated with a BFR-id.  I don't think we want to allow arbitrary numbers
> of BFR-prefixes to have the same BFR-id.  That creates the possibility for
> lots of error cases.  For one thing, we'd have a problem if the
> BFR-prefixes corresponding to a given BFR-id aren't all reachable via the
> same path.  For another, when we pass a BFR-prefix in the overlay
> signaling, we'd have a whole bunch of issues in figuring out which one to
> pass when.
>
> architecture draft that would prohibit a BFR-id from corresponding to both
> an IPv4 address and an IPv6 address.
>

We definitely need to be able so support both an IPv6 and IPv4 address.
If "Router ID" isn't a meaningful concept, it would be useful to state
that.  At the moment, the detection of BFR-id duplication is important in
the architecture, but the details of how to do so aren't nailed down and
the "obvious" answers seems like they aren't quite right/clear.

> haven't thought about a lot, but I don't think having multiple BFR-prefixes
> is much of a help.  The problem is that the BFR-ids have to change.
>

Right - which is why I was asking to think about the problem, rather than
having a specific ask as far BFR-prefix count - though it is important to
support dual stack IPv4 and IPv6 as well as transitioning.  That is a case
where the BFR-id doesn't need to change.

Another question is whether a BFR can claim multiple BFR-ids so as to
handle renumbering/reassignment of BFR-ids?

> really see that the problem is BIER-specific.

Sure - the question is whether there is something that can be done now to
simplify that process later on?  If the answer is no, that is ok - but I
think raising the question has triggered some useful discussion.

> there is much to say.  As in MPLS generally, whether the TTL is inherited
> by the payload or whether the TC bits have any impact is a matter of local
> policy at the BFER.  We can add a sentence saying that.
>

Sounds good - the point was merely that it isn't listed at all in the
architecture and is clearly part of the BIER layer.

> BFER's own bit set and the rest 0.  However, I will point out that if the
> F-BM is 0, the textual description of the forwarding procedure in section
> 6.5 still works, even if the pseudocode does not ;-)  Will fix.
>

Sure - step 4 makes it work - but inconsistencies are unfortunate :-(

> straightforward way to say that the same applies to non-MPLS encapsulations.
>

Sounds good. This is just a case of tone and broadening the scope.

> BML of 256.  However, RECOMMENDED is probably a better match to the WG

WG call here.

Regards,
Alia

---

From: Eric C Rosen <[email-masked]>
Date: Mon, 19 Jun 2017 13:54:29 -0400

> We definitely need to be able to support both an IPv6 and IPv4 address.

Perhaps we just have to state the following caveat:

- Within a given BIER domain, each BFR-prefix is assumed to identify a 
unique BFR.  If a given BFR announces multiple BFR-prefixes for itself 
(within the same BIER domain), other BFRs will  treat the given BFR as 
if it were multiple BFRs, one per BFR-prefix.

Then it would be up to the signaling protocols to allow or disallow 
muliple BFR-prefix announcements per node.

> it is important to support dual stack IPv4 and IPv6 as well as 
> transitioning.

I think supporting dual stack means supporting both v4 routing and v6 
routing.  That would require two sub-domains, one bound to the v4 
routing underlay and one bound to the v6 routing underlay.

>   That is a case where the BFR-id doesn't need to change.

If one tried to do dual stack or transition using a single sub-domain, I 
would certainly not recommend that the same BFR-id be used for both an 
IPv4 BFR-prefix and an IPv6 BFR-prefix.  But I'm not sure how one does 
it in one sub-domain anyway, given that v4 routing is different than v6 
routing.

If it's done with two sub-domains, then using the same BFR-id for a v4 
BFR-prefix in one sub-domain and a v6 BFR-prefix in another should work, 
given the caveat above.  There would be no need for BIER to know that a 
particular v4 BFR-prefix denote the same BFR as a particular v6 BFR-prefix.

I don't think we actually want to work out a v4-to-v6 transition plan 
for this document, but I think there are enough hooks to allow it.

> the detection of BFR-id duplication is important in the architecture, 
> but the details of how to do so aren't nailed down and the "obvious" 
> answers seems like they aren't quite right/clear.

What the architecture says is that two different BFRs in the same BIER 
domain must not have the same BFR-ids.  If we change that to "in a given 
sub-domain of a given domain, two different BFR-ids MUST NOT be 
associated with the same BFR-prefix", I think we're covered.

I'd like to avoid specification of conflict resolution rules, that would 
be a real rathole.  It might make a good follow-on draft for someone, 
however ;-)

---

From: Tony Przygienda <[email-masked]>
Date: Mon, 19 Jun 2017 11:21:08 -0700

> (within the same BIER domain), other BFRs will  treat the given BFR as if
> it were multiple BFRs, one per BFR-prefix.
>

Surprisingly, I completely disagree here. Supporting multiple prefixes per
BFR and trying to differentiate now BIER underlays based on prefixes is a
deep rathole. I suspect we are trying to fix an SR architecture defect with
that (read below).  I would specify that if multiple prefixes (in same AF)
are present, the behavior is undefined and that's it.

The fact that we have a BFR prefix not being AF independent (i.e. a BFR
prefix [inherited from the SR concepts?] should be really IPv4 _and_ IPv6
as a single entity but alas, we can't fix that in BIER, can we ?).

Footnote: that's why conceptual models are so important (yes, albeit
frowned upon in IETF since they are so "academic". Conceptual models are
best protection against massive architectural defects in a technology down
the road). It's all simple until one crams enough ad-hoc unverified
assumptions together and ends up with a
tower-of-babel-sized-box-of-bandaids.

> routing.  That would require two sub-domains, one bound to the v4 routing
> underlay and one bound to the v6 routing underlay.

This is in fact trivial. MPLS/Ether encaps can carry both v4 and v6 frames
in same underlay as it should so it's irrelevant what the BFR prefix is
(and in fact BIER is 2.5 so it doesn't even use the IP forwarding step
strictly speaking). Unless I'm missing something here around the tunneling
issues, Eric ...  So generally, nothing prevents us from having a v4 BFR
prefix and shuffle v6 frames in BIER over it ...

But yes, allowing both v4 and v6 BFR prefix in the same BIER _sub_-domain
as prefix can lead to routing inconsistencies as Eric points out and that
we'll never be able to fix except with obscure per-case ad-hoc hacks ...

Alternately, to fix SR concepts here, we can specify BFR allowing 0..1 ipv4
BFR prefix and 0..1 ipv6 BFR prefix with the restriction that the
sub-domains MUST NOT overlap. That would clearly fix a sub-domain being
either v6 or v4 (in terms of the BFR prefixes) while being able to carry in
frames anything as it should ...

I attach corrected conceptual model below, describing it in language will
be quite some twist.

--- tony

---

From: Greg Shepherd <[email-masked]>
Date: Mon, 19 Jun 2017 11:41:20 -0700

Inline:

On Mon, Jun 19, 2017 at 11:21 AM, Tony Przygienda <[email-masked]>
wrote:

> assumptions together and ends up with a tower-of-babel-sized-box-of-bandaids.
>
>
>

I agree with Tony here. The BFR prefix is only used as a unique identifier
to build the intermediate BFIBs. There is no IPv4 or v6 routing associated
with the BFR prefix. We just need to ensure that for any given BFR-ID there
is a unique BFR prefix advertised into the IGP. It's an endpoint only. And
to his last point the payload of the BIER encapped packet is not associated
with the BFR-prefix af or subaf.

> we'll never be able to fix except with obscure per-case ad-hoc hacks ...

Right, that would get ugly, I would expect.

> Alternately, to fix SR concepts here, we can specify BFR allowing 0..1
> ipv4 BFR prefix and 0..1 ipv6 BFR prefix with the restriction that the

Agree.

> I attach corrected conceptual model below, describing it in language will

Nice pic.

Greg

> --- tony
>
>
>
> _______________________________________________
> BIER mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/bier
>
>

---

From: IJsbrand Wijnands <[email-masked]>
Date: Mon, 19 Jun 2017 21:30:16 +0200

Folks,

> But yes, allowing both v4 and v6 BFR prefix in the same BIER _sub_-domain as prefix can lead to routing inconsistencies as Eric points out and that we'll never be able to fix except with obscure per-case ad-hoc hacks … 

I have a different perspective on this…

I think its no problem at all to have the same BFR-id to be shared between IPv4 and IPv6 by a given node in the same sub-domain, as long as it allocates and advertises a unique BIER Label for both IPv4 and IPv6. This will then result into creating two different BIFT’s on all the routers. It is up to the overlay to choose the BIFT to forward a Multicast packets (the BIFT build through the IPv4 or IPv6 BIER topology). The BIFT key is {AF, SD, BSL}.

> 
> Alternately, to fix SR concepts here, we can specify BFR allowing 0..1 ipv4 BFR prefix and 0..1 ipv6 BFR prefix with the restriction that the sub-domains MUST NOT overlap. That would clearly fix a sub-domain being either v6 or v4 (in terms of the BFR prefixes) while being able to carry in frames anything as it should … 

We can fix a sub-domain to a particular AFI, but it looks more cumbersome to me from an operational perspective.

Thx,

Ice.

---

From: Eric C Rosen <[email-masked]>
Date: Mon, 19 Jun 2017 15:52:10 -0400

On 6/19/2017 2:21 PM, Tony Przygienda wrote:
>     - Within a given BIER domain, each BFR-prefix is assumed to
>     identify a unique BFR.  If a given BFR announces multiple
>     BFR-prefixes for itself (within the same BIER domain), other BFRs
>     will  treat the given BFR as if it were multiple BFRs, one per
>     BFR-prefix.
>
>     Then it would be up to the signaling protocols to allow or
>     disallow muliple BFR-prefix announcements per node.
>
>
>
> Surprisingly, I completely disagree here. Supporting multiple prefixes 
> per BFR and trying to differentiate now BIER underlays based on 
> prefixes is a deep rathole. I suspect we are trying to fix an SR 
> architecture defect with that (read below).  I would specify that if 
> multiple prefixes (in same AF) are present, the behavior is undefined 
> and that's it.

How do you know that two BFR-prefixes denote the same node?  Why would 
you even want to know that?

>
> The fact that we have a BFR prefix not being AF independent (i.e. a 
> BFR prefix [inherited from the SR concepts?] should be really IPv4 
> _and_ IPv6 as a single entity but alas, we can't fix that in BIER, can 
> we ?).

I don't see why we'd want a BFR prefix to be AF independent.

>     I think supporting dual stack means supporting both v4 routing and
>     v6 routing.  That would require two sub-domains, one bound to the
>     v4 routing underlay and one bound to the v6 routing underlay.
>
>
> This is in fact trivial. MPLS/Ether encaps can carry both v4 and v6 
> frames in same underlay as it should so it's irrelevant what the BFR 
> prefix is (and in fact BIER is 2.5 so it doesn't even use the IP 
> forwarding step strictly speaking).

Actually, I don't think there's any such thing as layer 2.5.

But the issue is not about the payload, it's about the routing 
underlay.  If a BFR-prefix is v4, the next hop would be determined by 
the v4 unicast routing.  If a BFR-prefix is v6, the next hop would be 
determined by the v6 unicast routing.  Whether the payload is v4 or v6 
is irrelevant.

> nothing prevents us from having a v4 BFR prefix and shuffle v6 frames 
> in BIER over it ...

This doesn't seem relevant to the point under discussion.  The issue is 
not which kind of frame is being encapsulated, it's how to choose the 
next hop for a given bit position.

>
> But yes, allowing both v4 and v6 BFR prefix in the same BIER 
> _sub_-domain as prefix can lead to routing inconsistencies as Eric 
> points out and that we'll never be able to fix except with obscure 
> per-case ad-hoc hacks ...
>
> sub-domains MUST NOT overlap. That would clearly fix a sub-domain 
> being either v6 or v4 (in terms of the BFR prefixes) while being able 
> to carry in frames anything as it should ...

Isn't that what I suggested?

I don't think we need to say a MUST NOT here.  If one is using v6 
BFR-prefixes on top of v4 unicast routing, presumably one has some way 
of making it work ;-)

---

From: Eric C Rosen <[email-masked]>
Date: Mon, 19 Jun 2017 16:03:03 -0400

On 6/19/2017 2:41 PM, Greg Shepherd wrote:
> The BFR prefix is only used as a unique identifier to build the 
> intermediate BFIBs. There is no IPv4 or v6 routing associated with the 
> BFR prefix. 

To build the BIFT, you need to determine the next hop(s) for each 
BFR-prefix.  I don't know how you determine this if the BFR-prefix is a 
v6 address and the unicast routing algorithm is v4-only.

I suppose one could define the routing underlay to be the combination of 
the v4 routing and the v6 routing, so that determining the next hop for 
a given BFR-id is determined by looking up the corresponding BFR-prefix 
in the proper unicast routing table for its address family.  That would 
be another way of dealing with the dual stack and/or transition problems.

> We just need to ensure that for any given BFR-ID there is a unique BFR 
> prefix advertised into the IGP. 

Did somebody say otherwise?

That would make it okay for a node to advertise multiple BFR-prefixes, 
as long as it advertised a different BFR-id for each one.

>  And to his last point the payload of the BIER encapped packet is not 
> associated with the BFR-prefix af or subaf.

Yes, the type of the payload is not relevant to this discussion.

---

From: Tony Przygienda <[email-masked]>
Date: Mon, 19 Jun 2017 13:04:00 -0700

On Mon, Jun 19, 2017 at 12:30 PM, IJsbrand Wijnands <[email-masked]> wrote:

> _sub_-domain as prefix can lead to routing inconsistencies as Eric points
> out and that we'll never be able to fix except with obscure per-case ad-hoc
> hacks …
>
> I think its no problem at all to have the same BFR-id to be shared between
> IPv4 and IPv6 by a given node in the same sub-domain, as long as it
> allocates and advertises a unique BIER Label for both IPv4 and IPv6. This
> will then result into creating two different BIFT’s on all the routers. It
> is up to the overlay to choose the BIFT to forward a Multicast packets (the
> BIFT build through the IPv4 or IPv6 BIER topology). The BIFT key is {AF,
> SD, BSL}.
>

Ice, you won't be able to deal consistently with the fact that you get from
a BFR two advertisements for same subdomain

BFR-prefix v6 A/128
  SD 0 SI 0 BML 256 : label 10-20

BFR-prefix v4 B/32
  SD 0 SI 0 BML 256 : label 30-40

yes, you can differentiate when you receive as BFR which is which (albeit
it's the same really) but in case of v4/v6 routing being incongruent, now,
which one do you use on BFIR to impose and how do you make sure all routers
use v4 or v6 accordinly and how do you deal with routers that advertise v4
or v6 only and you end up with "islands" ? To imitate Yakov: "it's all in
the model, look at the model" ;-) (and yes, everything works for some
_special_ case with some special band-aid of course but let's not have
THOSE discussions)

Otherwise, I agree with Greg ...

---

From: Eric C Rosen <[email-masked]>
Date: Mon, 19 Jun 2017 16:19:31 -0400

On 6/19/2017 4:04 PM, Tony Przygienda wrote:
>
>
> On Mon, Jun 19, 2017 at 12:30 PM, IJsbrand Wijnands <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     Folks,
>
>     per-case ad-hoc hacks …
>     I think its no problem at all to have the same BFR-id to be shared
>     between IPv4 and IPv6 by a given node in the same sub-domain, as
>     long as it allocates and advertises a unique BIER Label for both
>     IPv4 and IPv6. This will then result into creating two different
>     BIFT’s on all the routers. It is up to the overlay to choose the
>     BIFT to forward a Multicast packets (the BIFT build through the
>     IPv4 or IPv6 BIER topology). The BIFT key is {AF, SD, BSL}.
>
>
> Ice, you won't be able to deal consistently with the fact that you get 
> from a BFR two advertisements for same subdomain
>
> BFR-prefix v6 A/128
> yes, you can differentiate when you receive as BFR which is which 
> (albeit it's the same really) but in case of v4/v6 routing being 
> incongruent, now, which one do you use on BFIR to impose and how do 
> you make sure all routers use v4 or v6 accordinly and how do you deal 
> with routers that advertise v4 or v6 only and you end up with "islands" ?

I think Ice is saying that the BFIR would determine by policy whether to 
use the v4 or v6 infrastructure, and would apply the properly BIER-MPLS 
label.  This of course would only work if the all the BFERs can be 
reached by either infrastructure.

My thinking here is that we already have a mechanism allowing a BFIR to 
choose the transportation infrastructure, and it's called "sub-domain". 
   I think we should leverage that concept rather than introduce a new 
value into the lookup key.

> Otherwise, I agree with Greg ...

I'm having trouble figuring out just what it is that you and Greg are 
agreeing ;-)

---

From: Eric C Rosen <[email-masked]>
Date: Mon, 19 Jun 2017 16:17:28 -0400

---

From: Eric C Rosen <[email-masked]>
Date: Mon, 19 Jun 2017 16:09:25 -0400

> I think its no problem at all to have the same BFR-id to be shared between IPv4 and IPv6 by a given node in the same sub-domain, as long as it allocates and advertises a unique BIER Label for both…

This is very similar to having two sub-domains, one for each AF. Since 
we already have the sub-domain mechanisms specified, and we've never 
talked about assigning different label ranges per AF, I'm not sure I see 
the value of introducing a change to the way the BIFT key is constructed.

---

From: Greg Shepherd <[email-masked]>
Date: Mon, 19 Jun 2017 13:33:14 -0700

Inline:

On Mon, Jun 19, 2017 at 1:03 PM, Eric C Rosen <[email-masked]> wrote:

>> intermediate BFIBs. There is no IPv4 or v6 routing associated with the BFR
>> prefix.
>>
>
> BFR-prefix.  I don't know how you determine this if the BFR-prefix is a v6
> address and the unicast routing algorithm is v4-only.
>

Ah, ok, I get your point.

> the v4 routing and the v6 routing, so that determining the next hop for a
> given BFR-id is determined by looking up the corresponding BFR-prefix in
> the proper unicast routing table for its address family.  That would be
> another way of dealing with the dual stack and/or transition problems.
>

Sorry, my reading of the thread seemed to miss this point.

> That would make it okay for a node to advertise multiple BFR-prefixes, as
> long as it advertised a different BFR-id for each one.
>

Agree. Sorry for the distraction.

Greg

---

From: "IJsbrand Wijnands (iwijnand)" <[email-masked]>
Date: Mon, 19 Jun 2017 20:36:43 +0000

Eric,

Correct, under the hood it's like having an other sub-domain. The value add is that you can very easily introduce IPv6 without having to configure additional BFR-id's and sub-domain. So it's easier for the operator.

As for choosing the right BIFT at the ingress, by default for IPv6 mcast, look for the IPv6 BIFT, otherwise IPv4. 

Sent from my iPhone

> On 19 Jun 2017, at 22:23, Eric C Rosen <[email-masked]> wrote:
> 
> This is very similar to having two sub-domains, one for each AF. Since we already have the sub-domain mechanisms specified, and we've never talked about assigning different label ranges per AF, I'm not sure I see the value of introducing a change to the way the BIFT key is constructed.
> 
>

---

From: "IJsbrand Wijnands (iwijnand)" <[email-masked]>
Date: Mon, 19 Jun 2017 20:49:13 +0000

Tony,

> Ice, you won't be able to deal consistently with the fact that you get from a BFR two advertisements for same subdomain 
> 
> BFR-prefix v6 A/128   

Maybe it's an implementation issue, but I expect the IPv4 and IPv6 prefixes to be in different tables, so I don't see a problem..

> 
> yes, you can differentiate when you receive as BFR which is which (albeit it's the same really) but in case of v4/v6 routing being incongruent, now, which one do you use on BFIR to impose and how do you make sure all routers use v4 or v6 accordinly and how do you deal with routers that advertise v4 or v6 only and you end up with "islands" ? To imitate Yakov: "it's all in the model, look at the model" ;-) (and yes, everything works for some _special_ case with some special band-aid of course but let's not have THOSE discussions)

I don't get your point, choosing the the BIFT at ingress is easy. How do you select a SD? With IPv6 it's even simpler as you could make IPv6 multicast traffic prefer a IPv6 BIER underlay if available, other IPv4...

No band-aid... no special case...IMO :-)

Thx,

Ice.

---

From: Tony Przygienda <[email-masked]>
Date: Mon, 19 Jun 2017 13:52:21 -0700

On Mon, Jun 19, 2017 at 12:52 PM, Eric C Rosen <[email-masked]> wrote:

> How do you know that two BFR-prefixes denote the same node?  Why would you
> even want to know that?
>

All the BFR prefixes are advertised by the same router ? Which is kind of
easily traceable, at least in IGP ... ?  Or do you like to go to a model
where a single BFR can carry multiple prefixes, each being a "virtual BIER
underlay in v4 or v6" ? I read your further emails saying that this is
overkill as far I understand ...

And a suggestion that "all BFERs must have same magic policies to agree
which BFR-prefix they use to encapsulate" sounds to me like a "global magic
bandaid" ...

--- tony

---

From: Eric C Rosen <[email-masked]>
Date: Mon, 19 Jun 2017 17:01:06 -0400

> As for choosing the right BIFT at the ingress, by default for IPv6 mcast, look for the IPv6 BIFT, otherwise IPv4.

I think this would always be a matter of policy, and I don't think there 
should be a default.  If you've been doing v6 multicast over a v4 
infrastructure,  I don't think you should automatically switch to the v6 
infrastructure just because your neighbor has advertised a label for 
v6.  You also start to get into issues like "what if one of my neighbors 
advertises a v6 BIFT and another doesn't?", "what if some BFERs don't 
have a v6 BFR-prefix?", "what if every known BFER has a v6 BFR-prefix, 
but suddently a new BFER appears that does not?", "what if an MVPN 
egress PE does have a v6 BFR-prefix but advertises its v4 BFR-prefix in 
the PMSI Tunnel attribute?"

---

From: Tony Przygienda <[email-masked]>
Date: Mon, 19 Jun 2017 14:01:07 -0700

> I don't get your point, choosing the the BIFT at ingress is easy. How do
> you select a SD? With IPv6 it's even simpler as you could make IPv6
> multicast traffic prefer a IPv6 BIER underlay if available, other IPv4...
>

So what will you do if you get a layer-2 frame to carry over BIER then?
Look into the packet to decide whether it carries v6 or v4 multicast (if
even) ... ?

Implying that the transport has to semantically parse what is the payload
it carries to work correctly is an architectural layer violation of first
degree ...

The mechanism of mapping of the transported content/overlay to BIER
underlay must be completely left out of the BIER architecture ... We just
provide a de-muxing codepoint (frame type) for the upper layer but that's
it. It can be even abused by the BIER overlay if they know what they doing,
none of that should prevent BIER underlay from working ...

--- tony

---

From: Eric C Rosen <[email-masked]>
Date: Mon, 19 Jun 2017 17:16:42 -0400

On 6/19/2017 4:52 PM, Tony Przygienda wrote:
>
>     How do you know that two BFR-prefixes denote the same node?  Why
>     would you even want to know that?
>
>
> All the BFR prefixes are advertised by the same router ? Which is kind 
> of easily traceable, at least in IGP ... ?

And in BGP?

Or if you are running OSPF for v4 but ISIS for v6 as ships in the night?

Or if the all the BFER-ids are configured in each BFIR by a controller?

All that's really necessary is that the BFR-id maps to an address that 
can in turn be mapped to a set of next hops.

> And a suggestion that "all BFERs must have same magic policies to 
> agree which BFR-prefix they use to encapsulate" sounds to me like a 
> "global magic bandaid" ... 

I don't recall making that suggestion.

I think you will agree that choosing a sub-domain when encapsulating is 
either a matter of policy or is determined by the overlay signaling.  I 
don't see how the scenario we are discussing is any different.

---

From: "IJsbrand Wijnands (iwijnand)" <[email-masked]>
Date: Mon, 19 Jun 2017 21:20:06 +0000

Eric,

What I've suggested is just like running an other sub-domain, it's just that it's automagically created based on advertising IPv6 prefixes and easily shares the already configured BFR-Id. All the same sub-domain pro/conn's apply, how do you map traffic to it, inconsistency, etc....

Sent from my iPhone

> On 19 Jun 2017, at 23:01, Eric C Rosen <[email-masked]> wrote:
> 
> I think this would always be a matter of policy, and I don't think there should be a default.  If you've been doing v6 multicast over a v4 infrastructure,  I don't think you should automatically switch to the v6 infrastructure just because your neighbor has advertised a label for v6.  You also start to get into issues like "what if one of my neighbors advertises a v6 BIFT and another doesn't?", "what if some BFERs don't have a v6 BFR-prefix?", "what if every known BFER has a v6 BFR-prefix, but suddently a new BFER appears that does not?", "what if an MVPN egress PE does have a v6 BFR-prefix but advertises its v4 BFR-prefix in the PMSI Tunnel attribute?"
> 
> 
> 
>

---

From: Tony Przygienda <[email-masked]>
Date: Mon, 19 Jun 2017 16:38:59 -0700

On Mon, Jun 19, 2017 at 2:16 PM, Eric C Rosen <[email-masked]> wrote:

> easily traceable, at least in IGP ... ?
>
>
> And in BGP?
>
> All that's really necessary is that the BFR-id maps to an address that can
> in turn be mapped to a set of next hops.
>
Sigh, fair points ... Anyone trying to run multiple signaling protocols
over same BIER underlay is probably into a world of pain anyway ...

> I don't recall making that suggestion.

OK, then I misread your emails. Mea culpa ...

---

From: IJsbrand Wijnands <[email-masked]>
Date: Tue, 20 Jun 2017 09:58:07 +0200

Tony,

> So what will you do if you get a layer-2 frame to carry over BIER then? Look into the packet to decide whether it carries v6 or v4 multicast (if even) ... ? 
> 
> Implying that the transport has to semantically parse what is the payload it carries to work correctly is an architectural layer violation of first degree … 

That is not what I’m applying at all. I said “.. you could make IPv6 multicast traffic prefer ..”, as that does not require any parsing of the payload. Its the IPv6 L3 MFIB/MRIB where you set that POLICY.

> 
> The mechanism of mapping of the transported content/overlay to BIER underlay must be completely left out of the BIER architecture …

Agreed, its a customer policy, like how SD’s are used. I was not implying this becomes part of the architecture, you are over extrapolating.

>  We just provide a de-muxing codepoint (frame type) for the upper layer but that's it. It can be even abused by the BIER overlay if they know what they doing, none of that should prevent BIER underlay from working … 

So there are two choices;

1. For an IPv6 underlay we introduce a new SD. 

2. Modify the BIFT Key such that is becomes AF specific.

Both work for me!

Thx,

Ice.

---

From: Tony Przygienda <[email-masked]>
Date: Tue, 20 Jun 2017 08:34:58 -0700

> underlay must be completely left out of the BIER architecture …
>
> Agreed, its a customer policy, like how SD’s are used. I was not implying
> this becomes part of the architecture, you are over extrapolating.
>
>
Ice, great, my bad then. One cannot be firm enough on first principles
though ;-) and I'm glad we agree ...

> >  We just provide a de-muxing codepoint (frame type) for the upper layer
> but that's it. It can be even abused by the BIER overlay if they know what
> they doing, none of that should prevent BIER underlay from working …
>

I would not make BIFT key AF specific. Observe that strictly speaking BIER
will work without IP @ all! If we  have e.g. some magic out-of-band
signalling (controller) we can populate all the BIFTs everywhere and in
case e.g. of ethernet encapsulation without any tunnels we don't even need
IP addresses for anything, ether carries BIER directly and that's it. That
sounds theoretical until one meets the "precomputed Steiner tree" crowd
which tends to be small but have sometimes big budgets ;-)

My suggestion is still to KISS it, i.e. only one of ipv4/ipv6 BFR prefix
allowed in IGP encoding and both carry a disjoint set of SDs.  Yes, SD in
IPv6 BFR prefix should imply "please use IPv6 forwarding when carrying it
over tunnels, using IGP neighbor resolution". That's perfect with me ...

--- tony

---

From: IJsbrand Wijnands <[email-masked]>
Date: Tue, 20 Jun 2017 17:42:23 +0200

Tony,

> My suggestion is still to KISS it, i.e. only one of ipv4/ipv6 BFR prefix allowed in IGP encoding and both carry a disjoint set of SDs.  Yes, SD in IPv6 BFR prefix should imply "please use IPv6 forwarding when carrying it over tunnels, using IGP neighbor resolution". That's perfect with me … 

Ok, its the simpler approach, agree,

Thx,

Ice.

---

From: Tony Przygienda <[email-masked]>
Date: Tue, 20 Jun 2017 08:48:40 -0700

Question being, is that in the section of  "Special Considerations for IP
based IGP signalling" in the architecture, is that "Special Consideration"
in IGP drafts or is that a clarifying, separate document ...

Given architecture is already "polluted" with the concept of an BFR Prefix
implying mandatory IP prefix to SD binding, first maybe the best choice
currently but I'm kind of OK with all of the above ...

thanks

--- tony

On Tue, Jun 20, 2017 at 8:42 AM, IJsbrand Wijnands <[email-masked]> wrote:

> over tunnels, using IGP neighbor resolution". That's perfect with me …
>

-- 
*We’ve heard that a million monkeys at a million keyboards could produce
the complete works of Shakespeare; now, thanks to the Internet, we know
that is not true.*
—Robert Wilensky