Subject: [Bier] Comment on BIER-TE

From: <[email-masked]>
Date: Wed, 2 Aug 2017 11:39:07 +0800 (CST)

Hi,athors of BIER-TE

I have a comment on BIRE-TE.

BitPositions are assigned to adjacencies in the BIER-TE architecture instead of BFER in BIER, so it is necessary to have SI in the BIER-TE domain to reduce the Bitstring length.  The issues is that if the transmit adjacency for a  BIER-node doesn't have the given SI value for a BIER-TE packet(the BitString in BIER encapsulation is related to SubDomain,BSL and SI combination), the BIER-TE packet may be dropped in that BIER-node.

The BIER-TE packet(SI =1 ) will be dropped in the BFR2 in the following figure, for there is no SI=1 entry in the BFR2 BIFT table.  How to solve it? Thanks.

Regards.

Fangwei.

---

From: Toerless Eckert <[email-masked]>
Date: Tue, 8 Aug 2017 19:08:59 +0200

sorry, clearing up backlog...

Good question,

I am not sure if or how much i elaborated on BIER-TE controller strategies for
bit assignments on the list. And that this could even be a question in BIER.

I didn't want to put this all into the BIER-TE
draft for matter of size, and consistency: For most parts i think in unicast
TE a lot of strategies on PCE controllers are equally not documented because
they are not needed for standardization - for better or worse. I am open
to any suggestions of how to document this if at all.

A simple strategy is like this: consider that you have a large network with
a core and a bunch of regions on the edge. You could assign to each region only
BFR-IDs with the same SI (or subdomain for that matter). When you then need to
replicate  from some region, you would need to send one copy per SI or subdomain.
If you do not want to have all those SI/subdomains on all possible BFIRs, you
could look into using another unicast MPLS label (eg: SR based) that would get you
to the edge-router into a particular area. Otherwise you'd have one remote adjacency
bit for each SI/subdomain that would do the same thing. And if you want to get
more fancy about TE, you would have bits for multiple redundant ingres area
exit routers and exit-area-ingres-routers to steer the traffic with load-splitting.

I mentioned how this could btw. equally be a question for BIER: If you had such
a large multi-are network and you would randomnly assign BFR-IDs to BFERs and end
up having in one region BFER with different SI's, then you would also need to 
send per-SI copies into that region, which would not be traffic-optimal. Aka:
You would also like to assign BFR-ID in SI groups in topological vicnity to maximize
the replication efficiency in BIER if you have such large networks.

Cheers
    Toerless

> _______________________________________________
> BIER mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/bier

-- 
---
[email-masked]

---

From: IJsbrand Wijnands <[email-masked]>
Date: Tue, 8 Aug 2017 20:49:44 +0200

Hi,

> the replication efficiency in BIER if you have such large networks.

The big difference with BIER is that when using multiple SI’s, the forwarding tables for each SI are build throughout the whole network. So there is never a problem that BIER packets get stuck because there is no BIFT for a particular SI. I agree it might be more efficient if BFER’s within an SI are co-located to have a more efficient forwarding, but its not a requirement to avoid packet loss.

IMO, the concept of SI does not apply to BIER-TE in order to scale up. One can create multiple “domains” of BFR’s that use bits out of the same BitString (like sub-domains), but these domains act completely independently (like Segmented Inter-AS scenario). 

The power of BIER-TE is that one can traffic engineer on a per packet basis without creating state in the network. Unfortunately, the scope is small. 

If traffic engineering on a bigger scale is needed, it may not be required to do it on a per flow basis but rather on a topology. This can be done with normal BIER and different topologies (sub-domains) created BIFT’s. Maybe this is similar to what Greg Mirsky was saying on the other thread, but I don’t think we would run BIER over RSVP-TE. We run it over different topologies.

Thx,

Ice.

---

From: Toerless Eckert <[email-masked]>
Date: Wed, 9 Aug 2017 19:03:40 +0200

On Tue, Aug 08, 2017 at 08:49:44PM +0200, IJsbrand Wijnands wrote:
> If traffic engineering on a bigger scale is needed, it may not be required to do it on a per flow basis but rather on a topology. This can be done with normal BIER and different topologies (sub-domains) created BIFT???s. Maybe this is similar to what Greg Mirsky was saying on the other thread, but I don???t think we would run BIER over RSVP-TE. We run it over different topologies.

Right. I left that out of my reply to GregM because he did not
list that option, but i agee that this is a good option in the playbook.

The problem when you do create alternatives paths by using
multiple topologies of an IGP (like ISIS or OSPF) is that
it very much depends on your goals how many topologies you need:

My favourite topic is live-live redundancy, and depending on
topology and where sources and receivers are, this can be
very simple (2 topologies) or very difficult (2 topologies per
set of sources). And of course the latter is a really unnecessary
overhead and in the past operators where alrady weary having
to deal with just two topologies due to the need to consider
two set of per-link metrics. Effectively you're using a very
flexible tool (topologies) to solve a specialized problem.

In general i recommend to instead use MRT (RFC7811) for live-live.
It would be similarily useable with BIER (or any other multicast,
mLDP, PIM for that matter). And it would forego the need to
come up and provision multiple set of metrics. Note that
protocols like ISIS and OSPF may also start to include MTR,
eg: RFC7813, so it becomes more and more confusing to sift through
the options ;-)

Now if you want to use traffic-engineering ("path engineering")
to do steiner trees, IGP topologies will not help. If you want
to do N-path load-splitting, you need N topologies, etc. pp.
"Your mileage will vary" ;-)

Cheers
    Teorless

---

From: IJsbrand Wijnands <[email-masked]>
Date: Wed, 9 Aug 2017 20:51:18 +0200

Toerless,

> "Your mileage will vary" ;-)

https://en.wikipedia.org/wiki/Wikipedia:Solutions_looking_for_a_problem

:-)

Thx,

Ice.

---

From: Greg Mirsky <[email-masked]>
Date: Wed, 9 Aug 2017 12:29:15 -0700

Hi Toerless, Ice, et. al,
I think that we arrived to realization that it may be helpful first to
formulate the problem, requirements towards TE at BIER layer, if any.

Ice,
we may have slightly different terminology in using 'topology', but you've
captured my message - traffic engineering, as well as protection, may be
better achieved in BIER's underlay, i.e. transport network. And segment
routing with MPLS dataplane may fit the bill. Whether one uses distributed
control plane or central controller, e.g. PCE - secondary issue.

Regards,
Greg

---

From: Toerless Eckert <[email-masked]>
Date: Wed, 9 Aug 2017 23:03:47 +0200

;-) exactly.

I am mentioning steiner trees every year once since 20 year
in the hope someone will stand up and say he needs it in reality.
No luck yet.

Load splitting obviously is very important but was maybe not
a good example for using topologies given how BIER already has
a pretty good native ECMP scheme.

Cheers
    Toerless

-- 
---
[email-masked]

---

From: Toerless Eckert <[email-masked]>
Date: Wed, 9 Aug 2017 23:18:53 +0200

Greg:

I think that the use-cases for BIER-TE are pretty clear:
It is at least the same ones as the ones SR has for steering traffic,
just for multicast traffic. On top of it, it also allows easy
controller define live-live multicast without having to bring in
complex new routing protocol like MRT into the network (live-live
is rarely a unicast requirement, so this is where multicast
requirements will be more than typical unicast).

The goals of the designs are also clear: It is designed to provide an 
evolution from RSVP-TE/P2MP for multicast with the same goals as the
evolution from RSVP-TE/P2P to SR. Aka: like SR for unicast, it provides
for multicast traffic engineering in a comparable lightweight 
signaling and state fashion as SR using the principle concepts of
BIER and expanding its architecture.

Of coure, SR is now a more inclusive framework also supporting other
functions, but i am not relating to those.

If any of this is not clear enough to you from the current draft
text, i am happy to improve this according to above text or more
detailed. Pls. let me know.

Cheers
    Toerless

-- 
---
[email-masked]

---

From: Greg Mirsky <[email-masked]>
Date: Wed, 9 Aug 2017 19:43:05 -0700

Hi Toerless,
though the goal perhaps was to introduce segment routing paradigm in BIER
architecture, the BIER-TE, in my opinion, does have serious issues when
comparing to a solution that overlays BIER on traffic engineered transport
network constructed using the segment routing with, for example, MPLS data
plane. I'll refer to the latter as multi-layer TEed BIER.
In order to achieve full control over flows, as I understand the proposed
BIER-TE architecture, the BIER-TE domain must be homogeneous, i.e. all
nodes must be BFRs. That would highly likely not allow easy brownfield
deployment as BIER-TE may require new HW. The multi-layer approach will
only require limited upgrade of HW in the domain to add BIER functionality
(assuming that the domain is already using MPLS in its data plane).

If you consider relaxing requirement to use BIER-TE in only homogeneous
domains, then, I believe, you lose control over path selection. Would you
agree?

Regards,
Greg

---

From: Toerless Eckert <[email-masked]>
Date: Thu, 10 Aug 2017 16:43:11 +0200

Inline.

> plane. I'll refer to the latter as multi-layer TEed BIER.

Do you have pointers to any document describing that solution approach
or would you mind to elaborate on it ? I asked the first time around and
provided explanations about the insufficiency of various approaches
and you did not answer to that first request for references or commonted
on my proposed interpretation of your claims.

> (assuming that the domain is already using MPLS in its data plane).

| 4.8.2.  Supporting nodes without BIER-TE
| 
|    Routed adjacencies also enable incremental deployment of BIER-TE.
|    Only the nodes through which BIER-TE traffic needs to be steered -
|    with or without replication - need to support BIER-TE.  Where they
|    are not directly connected to each other, forward_routed adjacencies
|    are used to pass over non BIER-TE enabled nodes.
| 
| 8.  BIER-TE and Segment Routing
| 
|    Segment Routing aims to achieve lightweight path engineering via
|    loose source routing.  Compared for example to RSVP-TE, it does not
|    require per-path signaling to each of these hops.
| 
|    BIER-TE is supports the same design philosophy for multicast.  Like
|    in SR, it relies on source-routing - via the definition of a
|    BitString.  Like SR, it only requires to consider the "hops" on which
|    either replication has to happen, or across which the traffic should
|    be steered (even without replication).  Any other hops can be skipped
|    via the use of routed adjacencies.
| 
|    Instead of defining BitPositions for non-replicating hops, it is
|    equally possible to use segment routing encapsulations (eg: MPLS
|    label stacks) for "forward_routed" adjacencies.

Aka: the maximum bit-saving and minimum HW-deployment option is to
only deploy it for nodes where replication is needed.

What the existing text i think misses to discuss yet is to discuss is a bit whether or
how much BIER-TE BFIR and BFER need to supprt, eg: for imposition/disposition,
so that needs to be more detailed (in line with my promise to Alia):

A) On a BFIR, imposition does not mean the need to support full BIER-TE forwarding
  plane semantic. Just the imposition of some opaque header (which later BFRs will see
  to be BIER) and (eg: SR) label to the first replicating BFR. Only that first replicating
  BFR would need to understand the BIER-TE forwarding semantics., the BFIR just does this 
  imposition and sends an MPLS packet.
B) If the BFIR needs to send multiple packets, eg: for different SI or even for the same
  SI, but multiple remote next-hop BFR: IN this case  the forwarding plane
  would just need to be programmed with multiple adjacencies as in A). Aka:
  These adjacencies could simply be part of the IP multicast OIF list.
C) On a BFER, support for BIER-TE forwarding could be avoided if the encap
  would provide some form of BIER-header disposition label. The remote-adjacency
  on the preultimate hop would then simply have the eg: SR label stack that
  not only steers the packet to the BFER, but also includes that disposition label.
  Aka: The BFERs functionaly would just have to be "see this label -> dispose
  of some magic N-byte junk at the start of the packet (BIER header) and pass the rest to
  IP (multicast) routing in VRF xyz"

> domains, then, I believe, you lose control over path selection. Would you

See above. Its really like SR. Only need to support where you need to use it,
and use really means in BIER-TE where you want to replicate.

In the picture i pained for Hu in the SI thread, i was suggesting the example of
a large network with edge areas and a core, interconnected with a few edge routers
each, and yu want to steer traffic maybe across specific combinations of
(ingres-edge-2-core-routerX, core-2-egres-edge-routerY) and replicate in the
receiver edge area also with explicit path selection because it may have
eg: rings (like in MSO networks) where you want to control traffics ring direction).

Hard to suggest the best setups without a specific topology/set of requirements,
but of course i am happy to write down an example.  I was just hoping that examples
of such controller strategies how to allocate bits etc. would be rather a nice
follow up document instead of bloating the architecture doc.

The model of BIER-TE is really very much in spirit also with the fully programmable forwarding
plane directions happening in unicast (SRv6 etc). I hope they're defining
"opaque header imposition/disposition" actions, if not, then i would certainly like to suggest
that if/once BIER-TE was a WG item.

Cheers
    Toerless

-- 
---
[email-masked]

---

From: Toerless Eckert <[email-masked]>
Date: Thu, 10 Aug 2017 17:05:13 +0200

Btw: one quick amendment:

It would IMHO be perfectly valid to also define one additional level of
reduced support, and that is to have nodes that can just replicate based on
a label but not support BIER. You would use that if you knew that you always
would want to replicate to specific next-hops, and that there is no need to
let this be flexibly selected by bits. The example would be replication in
this big core where we know that there will always be _some_ receiver behind
the nexthop because of statistics (each region aggregates a few million viewers
for example). So if you amend the example: the ingres-edge-2-core router
was selected by bits, and the bits it is looking at are adjacencies to
lets say 4 alternative "always-replicate" routers with no BIER knowledge in 
the core. One of hose 4 bits is set, to select one. When it arrives at that
core router its just label lookup and replicate, no knowledge of BIER-TE,
those copies go to multiple core-2-egres-edge routers, and lets assume those
would do BIER-TE again.

Cheers
    Toerless

-- 
---
[email-masked]

---

From: Greg Mirsky <[email-masked]>
Date: Sun, 13 Aug 2017 11:40:27 -0700

Hi Toerless,
got busy and couldn't respond in time. Please find my answers, notes
in-lined tagged GIM>>.

Regards,
Greg

> Do you have pointers to any document describing that solution approach
GIM>> I don't think that there's need to describe how to construct per BIER
sub-domain BIRTs. And with that one can have traffic engineered sub-domains
of his/her liking. And choosing among them is as simple as choosing which
sub-domain to use for forwarding.

> only deploy it for nodes where replication is needed.
GIM>> Not with BIER-TE model.

>   IP (multicast) routing in VRF xyz"
GIM>> If, as you say, "BFER-TE support could be avoided" by using segment
routing in the underlay, then why do we need BIER-TE? You just re-stated my
point, thank you.

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 14 Aug 2017 06:14:58 +0200

In-line

> sub-domain to use for forwarding.

I think there is. I am still very much guessing what you are suggesting, and you
did not comment the extensive explanations i provided you in my prior 
answers. If you want to establish static, non-IGP driven BIRT, that should be
written down as a deployment model. Especially if you want to support N
alterntive paths/"trees" through the network. This requires N * BIRT and this
can easily become a lot more aggregate BIRT state in routers than what you
would get with BIER-TE. And how do you udate the state when topology changes
happen. This is all quite non-obvious IMHO.

> GIM>> Not with BIER-TE model.

Yes, with BIER-TE model.

> routing in the underlay, then why do we need BIER-TE? You just re-stated my

Please read what i have written. I have been adressing your prior email
concerns about incremental deployability. The sentence you referred to out
of context said "On a BFER, support for BIER-TE forwarding could be avoided if..."
                 ^^^^^^^^^                                                  ^^^^^

Woth SR, you can avoid supporting/enabling BIER-TE on nodes that only need to steer,
but not replicate. With both SR and BIER-TE, you do not need to change state
in midpoints when you want to change steering/replication. With your proposal
"provisioned BIRT", that would much more likely be necessary.

Does my explanation help to understand how BIER-TE is only required on
the nodes where replication is desired ? If not, then maybe we could
have a higher bandwidth channel to solve the misundertanding. 

Cheers
    Toerless

-- 
---
[email-masked]

---

From: Greg Mirsky <[email-masked]>
Date: Mon, 14 Aug 2017 11:59:12 -0700

Hi Toerless,
thank you for the continued discussion. I hope it is interesting to others
as well even though we may moved from the original question - Adopt or not
to adopt?
Please find my notes in-lined and tagged GIM2>>.

Regards,
Greg

> happen. This is all quite non-obvious IMHO.
GIM2>> I haven't stated that IGP would not play role. And if by "static"
you mean "instantiated by an SDN controller", then yes, that is plausible.
But we know how to build VPNs, L3VPN in particular. Why the same approach,
BGP or EVPN, not to use for BIER sub-domains? And support of N VPNs doesn't
seem as major problem these days either. As for updating the state when
underlay network changes, then we should consider this as multi-layer
network. Firstly, the underlay may be protected thus minimizing packet loss
in the overlay. And since the underlay is TE, we should use restoration in
the underlay as well which will preserve required characteristics for the
overlay network.

> > >   plane semantic. Just the imposition of some opaque header (which
> later
> > > BFRs will see
> > >   to be BIER) and (eg: SR) label to the first replicating BFR. Only
> that
> > > first replicating
> > >   BFR would need to understand the BIER-TE forwarding semantics., the
> BFIR
> > > just does this
> > >   of some magic N-byte junk at the start of the packet (BIER header)
> and
> > > pass the rest to
>                  ^^^^^^^^^
>   ^^^^^
GIM2>> I think I understand BIER-TE model and I believe that BIER+SR can
achieve the same results though by building underlay with more constrains.
Thus I don't see the value in BIER-TE work.

> > > See above. Its really like SR. Only need to support where you need to
> use
> > > it,
> > > a large network with edge areas and a core, interconnected with a few
> edge
> > > routers
> > > (ingres-edge-2-core-routerX, core-2-egres-edge-routerY) and replicate
> in
> > > the
> > > eg: rings (like in MSO networks) where you want to control traffics
> ring
> > > direction).
> > >
> > > but of course i am happy to write down an example.  I was just hoping
> that
> > > examples
> > > of such controller strategies how to allocate bits etc. would be
> rather a
> > > nice
> > > > > > I think that we arrived to realization that it may be helpful
> first
> > > to
> > > > > > we may have slightly different terminology in using 'topology',
> but
> > > > > you've
> > > > > > On Wed, Aug 9, 2017 at 11:51 AM, IJsbrand Wijnands <
> [email-masked]>
> > > > > wrote:
> > > > > >
> > > > > > > Toerless,
> > > > > > >