Subject: [Roll] using the flow label instead of hop by hop

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Thu, 18 Oct 2012 16:43:06 +0000

Hi

When I started this draft, we had a series of chats with Brian Carpenter and apparently reached a gentleman agreement that it was doable within the RPL domain to use the flow label and avoid the Hop by Hop option.

I published http://tools.ietf.org/html/draft-thubert-roll-flow-label-01 based on that discussion but did not get news from the group since then.

This technique has a number of advantages, in particular
-it saves extra bytes for the RPL option and the HbH header.
-it also avoids the prescribed tunneling within the RPL domain for packets from the outside.
- it has an optimized compression with 6LoWPAN.

Is there interest in the group to continue? If so I'd be glad to have some discussion time at the next meeting.

Cheers,

Pascal Thubert
IPv6 Engineering

[email-masked]<mailto:[email-masked]>
Phone :+33 497 23 26 34
Mobile :+33 619 98 29 85

Cisco Systems
Village d'Entreprises Green Side bat. T3
400, Avenue Roumanille
06410 Biot - Sophia Antipolis
France
Cisco.com<http://www.cisco.com/global/FR/>

[Description: Description: http://www.cisco.com/web/europe/images/email/signature/vertical04.jpg]

For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/index.html
This e-mail may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply e-mail and delete all copies of this message.

---

From: Abdussalam Baryun <[email-masked]>
Date: Fri, 19 Oct 2012 09:30:54 +0200

Hi Pascal,

Yes I have totally interest to continue this work, the concept/ideas are greate.
I will review and comment when available, I hope we get discussion on
list or in f2f meeting, I recommend you to present draft and list it
in the f2f ROLL-Agenda, thanking you,

AB

On 10/18/12, Pascal Thubert (pthubert) <[email-masked]> wrote:
> Hi
>
> When I started this draft, we had a series of chats with Brian Carpenter and
> apparently reached a gentleman agreement that it was doable within the RPL
> domain to use the flow label and avoid the Hop by Hop option.
>
> I published http://tools.ietf.org/html/draft-thubert-roll-flow-label-01
> based on that discussion but did not get news from the group since then.
>
> [Description: Description:
> http://www.cisco.com/web/europe/images/email/signature/vertical04.jpg]
>
> For corporate legal information go to:
> http://www.cisco.com/web/about/doing_business/legal/cri/index.html
> This e-mail may contain confidential and privileged material for the sole
> use of the intended recipient. Any review, use, distribution or disclosure
> by others is strictly prohibited. If you are not the intended recipient (or
> authorized to receive for the recipient), please contact the sender by reply
> e-mail and delete all copies of this message.
>
>
>
>

---

From: Owen Kirby <[email-masked]>
Date: Fri, 19 Oct 2012 11:59:51 -0700

Pascal,

I'm not entirely convinced that using the flow label like this is a good 
idea. Using the flow labels to carry the RPL HbH information works on 
the premise that packets being routed through the RPL domain wouldn't 
otherwise use the flow label. This assumption might be correct in a 
network of homogenous nodes with no external prefixes (eg: appendix 
A.5), but there may still be cases where packets being routed through 
the RPL domain might want to set the flow label. If this draft were to 
go forward, I would have some questions:

When a  forwarding node receives a packet with the flow label set, how 
does it determine whether the flow label contains an identifier of the 
5-tuple, or it contains the RPL HbH information? To get it wrong would 
interfere with the forwarding behavior and lead to interoperability issues.

When packets are received from an external prefix, how does the ingress 
router handle the flow label? Would it destroy the original label, leave 
the original label in tact, or use IPv6-in-IPv6 encapsulation to 
preserve the label (ie: the inner header contains the original flow 
label, and the outer header contains the HbH information)?

How would the DODAG root rebuild the flow label from the 5-tuple if it 
encounters packets that have been fragmented at the IPv6 layer? The 
primary use of the flow label is so that routers don't have to 
reassemble IPv6 fragments when forwarding to determine the 5-tuple 
(which is a challenging thing for a router to do).

Cheers,
Owen

On 18/10/2012 9:43 AM, Pascal Thubert (pthubert) wrote:
>
> Hi
>
> When I started this draft, we had a series of chats with Brian 
> Carpenter and apparently reached a gentleman agreement that it was 
> doable within the RPL domain to use the flow label and avoid the Hop 
> by Hop option.
>
> I published 
> http://tools.ietf.org/html/draft-thubert-roll-flow-label-01 
> <http://tools.ietf.org/html/draft-thubert-roll-flow-label-01> based on 
> that discussion but did not get news from the group since then.
>
> Is there interest in the group to continue? If so I'd be glad to have 
> some discussion time at the next meeting.
>
> Cheers,
>
>
> Pascal Thubert
> IPv6 Engineering
>
> [email-masked] <mailto:[email-masked]>
> Phone :+33 497 23 26 34
> Mobile :+33 619 98 29 85
>
> 	
>
>
> Cisco Systems
> Cisco.com <http://www.cisco.com/global/FR/>
>
> 	
>
> Description: Description: 
> http://www.cisco.com/web/europe/images/email/signature/vertical04.jpg
>
> This e-mail may contain confidential and privileged material for the 
> sole use of the intended recipient. Any review, use, distribution or 
> disclosure by others is strictly prohibited. If you are not the 
> intended recipient (or authorized to receive for the recipient), 
> please contact the sender by reply e-mail and delete all copies of 
> this message.
>
>
>
> _______________________________________________
> Roll mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/roll

---

From: Philip Levis <[email-masked]>
Date: Fri, 19 Oct 2012 12:46:27 -0700

I realize there's a lot of pressure for this in terms of header sizes, but this seems like exactly the kind of engineering hack/optimization that has painful long-term implications. Such as the Internet of Things prevents the flow label from ever being used effectively.

Phil

On Oct 19, 2012, at 11:59 AM, Owen Kirby wrote:

> Pascal,
> 
> I'm not entirely convinced that using the flow label like this is a good idea. Using the flow labels to carry the RPL HbH information works on the premise that packets being routed through the RPL domain wouldn't otherwise use the flow label. This assumption might be correct in a network of homogenous nodes with no external prefixes (eg: appendix A.5), but there may still be cases where packets being routed through the RPL domain might want to set the flow label. If this draft were to go forward, I would have some questions:
> 
> When a  forwarding node receives a packet with the flow label set, how does it determine whether the flow label contains an identifier of the 5-tuple, or it contains the RPL HbH information? To get it wrong would interfere with the forwarding behavior and lead to interoperability issues.
> 
> When packets are received from an external prefix, how does the ingress router handle the flow label? Would it destroy the original label, leave the original label in tact, or use IPv6-in-IPv6 encapsulation to preserve the label (ie: the inner header contains the original flow label, and the outer header contains the HbH information)?
> 
> How would the DODAG root rebuild the flow label from the 5-tuple if it encounters packets that have been fragmented at the IPv6 layer? The primary use of the flow label is so that routers don't have to reassemble IPv6 fragments when forwarding to determine the 5-tuple (which is a challenging thing for a router to do).
> 
> Cheers,
> Owen
> 
>> Is there interest in the group to continue? If so I’d be glad to have some discussion time at the next meeting.
>>  
>> Cheers,
>>  
>>  
>> 
>> Pascal Thubert
>> IPv6 Engineering
>> 
>> [email-masked]
>> Phone :+33 497 23 26 34
>> Mobile :+33 619 98 29 85
>> 
>> 
>> Cisco Systems
>> Cisco.com
>> <Mail Attachment.jpeg>

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Sat, 20 Oct 2012 08:09:20 +0000

Hi Owen:

I think we are back to the discussion with Brian (cc'ing Brian to make sure my account is faithful).
I think that we converged that the draft is actually doable, considering the LLN as one bug host from the perspective of the wider Internet.

With the proposal, the LLN gateway is in charge of setting the flow label towards the Internet and the flow label does not belong to the packet originator in the LLN, which, like any host in the Internet for the last 20 years, are wasting the precious bits.
So if an LLN originator sets the flow label it will be erased and recalculated if the LLN uses that draft. Can we justify that?

1) First thing first, the flow label is part of the IP header. Which is where you place stuff that you want examined by the router.
Anything farther down the packet will be a lot more trouble for the router. And app level communication does not belong here, whether it is a LLN app or any other app.

2) After 15 years of IPv6, the internet community finally found and standardized a usage for the label that basically adds burden to the hosts to serve the core but neiter sender nor receiver can use the value of the field for any practical usage.
For LLN devices that would mean ask a battery operated constrained node to do additional computation in case the packet goes in the powered Internet and needs load balancing. Do you know of any LLN device that would spend energy doing that?
It makes a lot more sense to push the burden to the power border router at the edge of the LLN in case the packet goes out the LLN towards the Internet.

3) The FL can only be compressed if it is NULL. This is the natural value for any constrained device, whether it aims at forwarding over RPL or not.
The burden of computing the tuple in the originator is not only on that node, but on all nodes in the LLN, and that would be whether the packet goes out to the Internet where the tuple can be used, or not.
HbH adds up to the waste, additional encap then adds up too. In a LLN that means wasting energy on every packet, with practical and measurable consequences in terms of battery life. In classical 15.4 that also means more fragmentation, and the consequences cascade.

The draft does not prevent an instance that needs the flow label for other usages to use the HbH. But is allows to control the damage when flow label is not used otherwise. Please see inline

[] Does not. If the instance is configured for Flow Label, then the original value is lost. In the future, we could store it in a new HC option but so far the need as not emerged.

[] The only usage is in the core. When the FL comes in the LLN, the role is done, the FL would be ignored anyway.
I hoped we could convey something end to end here, like a hint for the instanceId, but failed so far.
So we are back to the BR selecting the instance based on the same heuristics as if the HbH is used.

[] Since the BR compute the FL, it uses the same heuristic for the first and all fragments. Mixing instanceID and IP addresses mostly. Or keeping only the instanceID when we want the latency+jitter to be homogeneous within an instance over the net. For LLN applications, the distribution of 5 tuples is a lot wider than necessary or even beneficial for the core,  and instanceID + IP provides a better granularity and more meaningful volumes.

Cheers,

Pascal

Cheers,
Owen

Cheers,

Pascal Thubert
IPv6 Engineering

Cisco Systems

Roll mailing list

[email-masked]<mailto:[email-masked]>

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Sat, 20 Oct 2012 08:56:39 +0000

Hello Michael:

This is actually what the early RPL drafts would do. 

But at the time that collisioned with work at 6MAN, so we opted for the HbH/encaps alternate.

Now that the 6MAN work has stabilized, we have the opportunity to reconsider.
The bright side is that Brian helped and the current draft incorporates the resolution of his comments.

Cheers,

Pascal

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Michael Richardson
Sent: vendredi 19 octobre 2012 15:26
To: Pascal Thubert (pthubert)
Cc: [email-masked]
Subject: Re: [Roll] using the flow label instead of hop by hop

>>>>> "pthubert" == pthubert  <Pascal> writes:
    pthubert> I published
    pthubert>
    pthubert> http://tools.ietf.org/html/draft-thubert-roll-flow-label-01 based on that discussion but did not get news from the group since then. 

    pthubert> This technique has a number of advantages, in particular
    pthubert> -it saves extra bytes for the RPL option and the HbH
    pthubert> header.

Has it been field tested? i.e. has it got code?
Is there some other interest it it?  It seems like a big win to me.

-- 
Michael Richardson <[email-masked]>, Sandelman Software Works 
IETF ROLL WG co-chair.    http://datatracker.ietf.org/wg/roll/charter/

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 24 Oct 2012 14:59:51 +0100

The worst case I can see from this approach is that all packets with the same
source/dest address pair would get the same flow label and hence be treated
the same for ECMP, LAG or any other kind of load balancing. That will arise
if the flow label has to be calculated based only on the address 2-tuple due
to fragmentation. I don't see that as a disaster - a lot of load balancing
is already based on the 2-tuple anyway.

The revised flow label spec explicitly allows the case where it's a router
(often but not necessarily the first hop router) that sets the flow label,
iff the source doesn't do so. As far as I understand the proposal, it's
compatible with that.

Regards
   Brian Carpenter

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Wed, 24 Oct 2012 14:14:06 +0000

Hi Brian,

>  The worst case I can see from this approach is that all packets with the same source/dest address pair would get the same flow label and hence be treated the same for ECMP, LAG or any other kind of load balancing

This is actually not the case. 

A same node may participate to multiple instances, each instance designed to serve a certain class of traffic with a certain optimization. 
With the proposal, the instance-ID is stamped in the flow label.
The root will probably use the instance-ID for the computation of the final flow label, so that packets received over a same instance are treated the same over the core. 

This is true whether the packets are from a same or from multiple sources. So the root may or may not aggregate multiple sensors (field devices) in a same flow - or not.

For some applications such as industrial control flows, jitter and latency are critical, and it is good that different sources are treated the same, thus considered a same flow.
This would be difficult to coordinate ifs the sensors themselves were in charge of computing the label but is an obvious thing to do for the root.

Cheers,

Pascal

-----Original Message-----
From: Brian E Carpenter [mailto:[email-masked]] 
Sent: mercredi 24 octobre 2012 16:00
To: Pascal Thubert (pthubert)
Cc: Owen Kirby; [email-masked]
Subject: Re: [Roll] using the flow label instead of hop by hop

The worst case I can see from this approach is that all packets with the same source/dest address pair would get the same flow label and hence be treated the same for ECMP, LAG or any other kind of load balancing. That will arise if the flow label has to be calculated based only on the address 2-tuple due to fragmentation. I don't see that as a disaster - a lot of load balancing is already based on the 2-tuple anyway.

The revised flow label spec explicitly allows the case where it's a router (often but not necessarily the first hop router) that sets the flow label, iff the source doesn't do so. As far as I understand the proposal, it's compatible with that.

Regards
   Brian Carpenter

On 20/10/2012 09:09, Pascal Thubert (pthubert) wrote:
> The draft does not prevent an instance that needs the flow label for 
> other usages to use the HbH. But is allows to control the damage when 
> flow label is not used otherwise. Please see inline
> 
> 
> This technique has a number of advantages, in particular -it saves 
> extra bytes for the RPL option and the HbH header.
> Village d'Entreprises Green Side bat. T3 400, Avenue Roumanille

---

From: Ralph Droms <[email-masked]>
Date: Wed, 24 Oct 2012 16:40:05 +0200

Seems to me this conversation ought to include 6man fairly soon.

- Ralph

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 24 Oct 2012 17:08:55 +0100

> Seems to me this conversation ought to include 6man fairly soon.

If ROLL wants to go this way, I agree. I *think* the proposal does
no violence to the 6man RFCs, but that's only my opinion.

    Brian

---

From: Ralph Droms <[email-masked]>
Date: Wed, 24 Oct 2012 18:28:53 +0200

On Oct 24, 2012, at 6:08 PM, Brian E Carpenter <[email-masked]> wrote:

> If ROLL wants to go this way, I agree.

Right.  I've been waiting to see how the conversation here plays out before bringing up 6man.

> I *think* the proposal does

Well, of course yours is a carefully considered and respected opinion...

- Ralph

---

From: Philip Levis <[email-masked]>
Date: Thu, 25 Oct 2012 11:00:20 -0700

> no violence to the 6man RFCs, but that's only my opinion.

Brian,

My thought process is simple. From my reading, the proposal violates RFC6437:

 "A forwarding node MUST either leave a non-zero flow label value unchanged or change it only for compelling operational security reasons as described in Section 6.1."

Its seems to me that using the flow label as more efficient representation of a hop-by-hop option isn't a "compelling operational security reason." It's an optimization. There hasn't yet been any evidence or data presented on the potential efficiency benefits this will have (hand-waving "energy" is not evidence). Adding *more* complexity to RPL though an alternative representation of the hop-by-hop header seems to me to add complexity without much benefit. If there were evidence indicating that in a large number of operational scenarios this optimization would lead to a significant savings, then that might be compelling. But my experience is that the overall benefit of this compression is going to be minimal. Using the flow label in a way which someone reading RFC6437 would not guess or assume is problematic from an interoperability standpoint. So, significant possible cost in terms of interoperability, significant cost in terms of increased complexity, minimal gain in terms of performance.

If 6man obsoletes RFC6437 such that this proposed use doesn't violate a MUST, then I'd be all ears. RFC6437 does say that you MAY modify a flow label of zero (so routers may fill it in for end hosts that not do so), says that the end host SHOULD set it to a non-zero value, and that nodes MUST keep non-zero labels unchanged. I don't want to repeat the headache that NATs introduced by violating end-to-end in an unanticipated way.

Phil

------

Philip Levis
President, Kumu Networks
Associate Professor, Stanford University
http://csl.stanford.edu/~pal

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 25 Oct 2012 19:17:13 +0100

>  "A forwarding node MUST either leave a non-zero flow label value unchanged or change it only for compelling operational security reasons as described in Section 6.1."

I have to say it's a while since I read the draft all through, but I seemed
to remember that the packet would always be encapsulated in an outer packet,
in which case there is no problem if the outer packet's flow label is
set locally and vanishes at the ROLL boundary. If not, then yes, there's
a violation.

> If 6man obsoletes RFC6437 such that this proposed use doesn't violate a MUST, then I'd be all ears. RFC6437 does say that you MAY modify a flow label of zero (so routers may fill it in for end host…

I think it would be entirely possible (I am not particularly advocating this)
for RFC-roll to formally update RFC 6437 for this particular usage. Speaking
purely personally, I wouldn't oppose this, although others might object.

The reason I wouldn't oppose it is that the idea is that on the open
Internet, any forwarding device should be able to use the flow label
for ECMP, LAG or some other kind of load balancing. If the label is
set according to RFC 6437 at a well-defined point (such as the ROLL/Internet
boundary), this would be satisfied. Yes, I am being a bit heretical
about my own RFC ;-).

   Brian

---

From: Philip Levis <[email-masked]>
Date: Thu, 25 Oct 2012 15:54:25 -0700

> boundary), this would be satisfied. Yes, I am being a bit heretical

Again, my concern is being in accordance with RFC6437. If 6man obsoletes it for a more flexible use of the flow label, then I have zero objections to the proposal (and even think it is good).

Phil