Subject: [sfc] Congruent metadata signaling considerations

From: Nicolas BOUTHORS <[email-masked]>
Date: Fri, 11 Apr 2014 17:44:54 +0000

I am interested in exploring the notion of congruent out-of-band metadata signaling 
as defined in section 3.3 of draft-rijsman-sfc-metadata-considerations.  

A simple view is to allow SFs in a chain to send Metadata along the chain, potentially thanks to the
routing capabilities introduced by the SFC header.  

A simple flag in the SFC header could point out that the frame is a Metadata signaling frame (no 
subscriber traffic payload included), so it should be simple for a Service Node SFF receiving a frame 
to trigger some special processing, for example to pass the information to an application.   

Still as Joel pointed out in previous discussions, such a service incur some limitations. In  particular:

- Packet ordering is not insured: so a packet triggering some metadata creation could arrive after 
the corresponding Congruent Metadata signaling message.
- Metadata messages can be lost

This said, there are some well known solutions to send reliably data or frames over an unreliable 
network. I can see using SCTP are one option, to send Metadata hop to hop while in a chain.

This can probably done in different ways, and deserves to be studied and written down.

I am interested to know if members thinks of other candidates on building a reliable congruent
metadata signaling service for our chains.

SCTP seems interesting in that it provide a reliable service and has some support for security. It might
be however difficult to deal with packet ordering, unless we can call for some support from the SFC header for (normal) subscriber traffic.

Any thoughts?

Nicolas

________________________________________
From: Nicolas BOUTHORS
Sent: Thursday, April 10, 2014 3:16 PM
To: Joel M. Halpern; Linda Dunbar; [email-masked]; [email-masked]; Ron Parker
Cc: [email-masked]
Subject: RE: [sfc] Comments and suggestions to draft-rijsman-sfc-metadata-considerations

Hello

I can see the benefit, as shown by Linda, to distinguish how to convey metadata according to which function
will benefit from its use.

The metadata draft from Bruno shows many possibilities. At the same time, there are discussions on
how intelligent/flexible a SFF should/could be. I think an Openflow based SFF for example could
provide added value, provided it finds in the SFC header enough information.

A fixed size header can carry a number of information allowing Service Switching Functions to take local
decision on how to handle incoming or outgoing traffic. I share with Linda the feeling that an application Id
field and a fine grain policy field,  will enable innovation compared to a header opaque or without any
metadata.

Congruent metadata messages are a good complement to send information from one SF or another one,
There are ways to allow SF to retrieve this information, just as there are ways for SF to retreive out of band
metadata. In both cases, the metadata messages are not constrained in size, and some standard format
exists already to carry similar information.

Nicolas
________________________________________
From: Joel M. Halpern [[email-masked]]
Sent: Wednesday, April 09, 2014 2:17 PM
To: Linda Dunbar; [email-masked]; [email-masked]; Ron Parker
Cc: [email-masked]
Subject: Re: [sfc] Comments and suggestions to draft-rijsman-sfc-metadata-considerations

Maybe I am misreading your notes.

The first part of your reply seems to say that there are existing
techniques by which some service functions can pass data on to some
other service functions.  That is true.  That does not mean that the
range of needs is addressed.  You even seem to acknowledge that.

You then go on to say that we should only address the other problem
(metadata between service functions and network nodes).

Given that a large part of the problem of carrying metadata between
service functions is not addressed currently, I do not see why we should
neglect addressing that major problem.

Yours,
Joel

On 4/8/14, 9:12 PM, Linda Dunbar wrote:
>
> Joel,
>
> Today's proxy based service functions terminate TCP session and
> restart the TCP session to the next service function or the
> destination of the packet. Some of them attach data/cookies to the
> packet.
>
> All those are strictly between service functions.
>
> I didn't say anything about "DPI service function passing to a
> content manipulation service function the media type and offset to
> the content".
>
> There have been proposals (not at IETF yet) for having a fixed header
> field in data packets for a service function to insert some marker
> (or identifier) for network nodes to steer the packet without looking
> deeper into the packets, or vise versa. Those metadata between
> "service function" and "network node" should be the one that requires
> Metadata standardization.
>
> Yours,
>
> Linda
>
>
> -----Original Message----- From: Joel M. Halpern
> [mailto:[email-masked]] Sent: Tuesday, April 08, 2014 5:37 PM
> To: Linda Dunbar; [email-masked]; [email-masked]; Ron
> Parker Cc: [email-masked] Subject: Re: Comments and suggestions to
> draft-rijsman-sfc-metadata-considerations
>
> Your comment about metadata between service functions being handled
> today seems to be based on some set of less than obvious assumptions.
> I do not know of any applicable mechanism today whereby a DPI service
> function can pass to a content manipulation service function the
> media type and offset to the content.  And that is only one example
> of many.
>
> Even the few cases that are used today are typically done by
> manipulation of the message content in a fragile fashion.
>
> Yours, Joel
>
> On 4/8/14, 6:27 PM, Linda Dunbar wrote:
>> Bruno and Jerome,
>>
>> I like your analysis of various methods of carrying metadata and
>> their associated challenges.
>>
>> But some "metadata" mentioned in your draft have been traditionally
>> considered as part of policy, as the "20Mbps/30Mbps" used in your
>> Figure 2.
>>
> ...
>> For the "metadata" strictly between applications or between service
>> functions, like the cookies attached to HTTP, today's Proxy Based
>> service functions can already handle them well. It is debatable if
>> any network header extension is needed for those metadata.
>>
>> For the "metadata" exchanged between Service functions and Network
>> Nodes, there is definitely the need for standardized network layer
>> header to carry those extra information.
>>
> ...
>

---

From: Ron Parker <[email-masked]>
Date: Sun, 13 Apr 2014 20:11:34 +0000

Hi, Nicolas.

I think that out-of-band metadata is a useful complement to inband metadata, especially for long lived flows.   In the same way that we haven't forced a single transport encapsulation for the actual data packets, we don't need to specify a transport for the out-of-band variant either.   It could be over TCP, SCTP, or even UDP or MAC if reliability is not required.

It would be most beneficial if the same SFC header carried both inband and out-of-band metadata.   Have you evaluated http://datatracker.ietf.org/doc/draft-zhang-sfc-sch/ for use as that common header?   In the I-D, we propose a flexible TLV oriented approach to carriage of metadata.   Since the proposed SCH header has, as one of its fields, the ethertype of the frame or packet that follows, we could reserve value 0 to mean "metadata only".

We would also want to consider correlation.   The easiest approach would be to include the full MAC header or IP header of the flow that the metadata belongs to.   To accomplish this, we could define 2 metadata TLV types -- one for a correlation MAC header and one for a correlation IP header.   In the more complex case of wanting to correlate the out-of-band metadata to an individual transaction (i.e., HTTP GET), then we could define a flow or transaction correlation TLV type that would be added to both the data stream and the out-of-band data.

Thanks.

    Ron

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Nicolas BOUTHORS
Sent: Friday, April 11, 2014 1:45 PM
To: [email-masked]
Subject: [sfc] Congruent metadata signaling considerations

I am interested in exploring the notion of congruent out-of-band metadata signaling as defined in section 3.3 of draft-rijsman-sfc-metadata-considerations.  

A simple view is to allow SFs in a chain to send Metadata along the chain, potentially thanks to the routing capabilities introduced by the SFC header.  

A simple flag in the SFC header could point out that the frame is a Metadata signaling frame (no subscriber traffic payload included), so it should be simple for a Service Node SFF receiving a frame 
to trigger some special processing, for example to pass the information to an application.   

- Packet ordering is not insured: so a packet triggering some metadata creation could arrive after the corresponding Congruent Metadata signaling message.
- Metadata messages can be lost

This said, there are some well known solutions to send reliably data or frames over an unreliable network. I can see using SCTP are one option, to send Metadata hop to hop while in a chain.

I am interested to know if members thinks of other candidates on building a reliable congruent metadata signaling service for our chains.

SCTP seems interesting in that it provide a reliable service and has some support for security. It might be however difficult to deal with packet ordering, unless we can call for some support from the SFC header for (normal) subscriber traffic.

Any thoughts?

Nicolas

Hello

I can see the benefit, as shown by Linda, to distinguish how to convey metadata according to which function will benefit from its use.

The metadata draft from Bruno shows many possibilities. At the same time, there are discussions on how intelligent/flexible a SFF should/could be. I think an Openflow based SFF for example could provide added value, provided it finds in the SFC header enough information.

A fixed size header can carry a number of information allowing Service Switching Functions to take local decision on how to handle incoming or outgoing traffic. I share with Linda the feeling that an application Id field and a fine grain policy field,  will enable innovation compared to a header opaque or without any metadata.

Congruent metadata messages are a good complement to send information from one SF or another one, There are ways to allow SF to retrieve this information, just as there are ways for SF to retreive out of band metadata. In both cases, the metadata messages are not constrained in size, and some standard format exists already to carry similar information.

Nicolas

The first part of your reply seems to say that there are existing techniques by which some service functions can pass data on to some other service functions.  That is true.  That does not mean that the range of needs is addressed.  You even seem to acknowledge that.

You then go on to say that we should only address the other problem (metadata between service functions and network nodes).

Given that a large part of the problem of carrying metadata between service functions is not addressed currently, I do not see why we should neglect addressing that major problem.

Yours,
Joel

On 4/8/14, 9:12 PM, Linda Dunbar wrote:
> I didn't say anything about "DPI service function passing to a content 
> manipulation service function the media type and offset to the 
> content".
>
> field in data packets for a service function to insert some marker (or 
> identifier) for network nodes to steer the packet without looking 
> function can pass to a content manipulation service function the media 
> type and offset to the content.  And that is only one example of many.
>

_______________________________________________
sfc mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/sfc

---

From: "Joel M. Halpern" <[email-masked]>
Date: Sun, 13 Apr 2014 16:29:59 -0400

I am a little concerned by the suggestion of SCTP or TCP for propagating 
out-of-band metadata.
It implies that the application providing the metadata is aware of the 
chain, aware of which application needs the metadata, and has already 
setup connections to all the apps which follow it in the chain.  It also 
seems to imply that even applications which are not handling metadata 
need to have such information and connectivity, since they need to 
propagate the metadata along the chain.
This leads to having to create two fully aligned sets of information 
flows, the second one of which requires connection maintenance.

Having information which is maintained by other systems, and which is 
fetched and cached by applications seems quite tractable.  Trying to 
pass the information reliably along the chain, but separate from the 
actual data, seems quite difficult and complex.

Yours,
Joel

On 4/13/14, 4:11 PM, Ron Parker wrote:
> Hi, Nicolas.
>
> I think that out-of-band metadata is a useful complement to inband
> metadata, especially for long lived flows.   In the same way that we
> haven't forced a single transport encapsulation for the actual data
> packets, we don't need to specify a transport for the out-of-band
> variant either.   It could be over TCP, SCTP, or even UDP or MAC if
> reliability is not required.
>
> It would be most beneficial if the same SFC header carried both
> inband and out-of-band metadata.   Have you evaluated
> http://datatracker.ietf.org/doc/draft-zhang-sfc-sch/ for use as that
> common header?   In the I-D, we propose a flexible TLV oriented
> approach to carriage of metadata.   Since the proposed SCH header
> has, as one of its fields, the ethertype of the frame or packet that
> follows, we could reserve value 0 to mean "metadata only".
>
> We would also want to consider correlation.   The easiest approach
> would be to include the full MAC header or IP header of the flow that
> the metadata belongs to.   To accomplish this, we could define 2
> metadata TLV types -- one for a correlation MAC header and one for a
> correlation IP header.   In the more complex case of wanting to
> correlate the out-of-band metadata to an individual transaction
> (i.e., HTTP GET), then we could define a flow or transaction
> correlation TLV type that would be added to both the data stream and
> the out-of-band data.
>
> Thanks.
>
> Ron
>
>
> -----Original Message----- From: sfc [mailto:[email-masked]] On
> Behalf Of Nicolas BOUTHORS Sent: Friday, April 11, 2014 1:45 PM To:
> [email-masked] Subject: [sfc] Congruent metadata signaling
> considerations
>
> I am interested in exploring the notion of congruent out-of-band
> metadata signaling as defined in section 3.3 of
> draft-rijsman-sfc-metadata-considerations.
>
> A simple view is to allow SFs in a chain to send Metadata along the
> chain, potentially thanks to the routing capabilities introduced by
> the SFC header.
>
> A simple flag in the SFC header could point out that the frame is a
> Metadata signaling frame (no subscriber traffic payload included), so
> it should be simple for a Service Node SFF receiving a frame to
> trigger some special processing, for example to pass the information
> to an application.
>
> - Packet ordering is not insured: so a packet triggering some
> metadata creation could arrive after the corresponding Congruent
> Metadata signaling message. - Metadata messages can be lost
>
> This said, there are some well known solutions to send reliably data
> or frames over an unreliable network. I can see using SCTP are one
> option, to send Metadata hop to hop while in a chain.
>
> I am interested to know if members thinks of other candidates on
> building a reliable congruent metadata signaling service for our
> chains.
>
> SCTP seems interesting in that it provide a reliable service and has
> some support for security. It might be however difficult to deal with
> packet ordering, unless we can call for some support from the SFC
> header for (normal) subscriber traffic.
>
> Any thoughts?
>
> Nicolas
>
>
> ________________________________________ From: Nicolas BOUTHORS Sent:
> Thursday, April 10, 2014 3:16 PM To: Joel M. Halpern; Linda Dunbar;
> [email-masked]; [email-masked]; Ron Parker Cc:
> [email-masked] Subject: RE: [sfc] Comments and suggestions to
> I can see the benefit, as shown by Linda, to distinguish how to
> convey metadata according to which function will benefit from its
> use.
>
> The metadata draft from Bruno shows many possibilities. At the same
> time, there are discussions on how intelligent/flexible a SFF
> should/could be. I think an Openflow based SFF for example could
> A fixed size header can carry a number of information allowing
> Service Switching Functions to take local decision on how to handle
> incoming or outgoing traffic. I share with Linda the feeling that an
> application Id field and a fine grain policy field,  will enable
> innovation compared to a header opaque or without any metadata.
>
> Congruent metadata messages are a good complement to send information
> from one SF or another one, There are ways to allow SF to retrieve
> this information, just as there are ways for SF to retreive out of
> band metadata. In both cases, the metadata messages are not
> constrained in size, and some standard format exists already to carry
> similar information.
>
>
> Nicolas ________________________________________ From: Joel M.
> Halpern [[email-masked]] Sent: Wednesday, April 09, 2014 2:17
> PM To: Linda Dunbar; [email-masked]; [email-masked]; Ron
> Parker Cc: [email-masked] Subject: Re: [sfc] Comments and suggestions
> to draft-rijsman-sfc-metadata-considerations
>
> service functions is not addressed currently, I do not see why we
> should neglect addressing that major problem.
>
> Yours, Joel
>
>> There have been proposals (not at IETF yet) for having a fixed
>> header field in data packets for a service function to insert some
>> marker (or identifier) for network nodes to steer the packet
>> without looking deeper into the packets, or vise versa. Those
>> metadata between "service function" and "network node" should be
>> the one that requires Metadata standardization.
>>
>> Yours,
>>
>> Linda
>>
>>
>> Your comment about metadata between service functions being
>> handled today seems to be based on some set of less than obvious
>> assumptions. I do not know of any applicable mechanism today
>> whereby a DPI service function can pass to a content manipulation
>> service function the media type and offset to the content.  And
>> that is only one example of many.
>>
>>> But some "metadata" mentioned in your draft have been
>>> traditionally considered as part of policy, as the
>>> "20Mbps/30Mbps" used in your Figure 2.
>>>
>> ...
>>> For the "metadata" strictly between applications or between
>>> service functions, like the cookies attached to HTTP, today's
>>> Proxy Based service functions can already handle them well. It is
>>> debatable if any network header extension is needed for those
>>> metadata.
>>>
>>> For the "metadata" exchanged between Service functions and
>>> Network Nodes, there is definitely the need for standardized
>>> network layer header to carry those extra information.
>>>
>> ...
>>
>
>
>
> _______________________________________________ sfc mailing list
> [email-masked] https://www.ietf.org/mailman/listinfo/sfc
>
> _______________________________________________ sfc mailing list
> [email-masked] https://www.ietf.org/mailman/listinfo/sfc
>

---

From: Ron Parker <[email-masked]>
Date: Sun, 13 Apr 2014 20:36:10 +0000

Joel,

I don't think out-of-band will be the predominant mode due to the complexities that have been discussed on this thread and those that you raise here.   However, for certain special case scenarios, I could see benefit.   Regarding the need for the application to understand the other located service function instances in the chain, I think this can be handled in a control plane. 

   Ron

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Joel M. Halpern
Sent: Sunday, April 13, 2014 4:30 PM
To: Ron Parker; Nicolas BOUTHORS; [email-masked]
Subject: Re: [sfc] Congruent metadata signaling considerations

I am a little concerned by the suggestion of SCTP or TCP for propagating out-of-band metadata.
It implies that the application providing the metadata is aware of the chain, aware of which application needs the metadata, and has already setup connections to all the apps which follow it in the chain.  It also seems to imply that even applications which are not handling metadata need to have such information and connectivity, since they need to propagate the metadata along the chain.
This leads to having to create two fully aligned sets of information flows, the second one of which requires connection maintenance.

Having information which is maintained by other systems, and which is fetched and cached by applications seems quite tractable.  Trying to pass the information reliably along the chain, but separate from the actual data, seems quite difficult and complex.

Yours,
Joel

On 4/13/14, 4:11 PM, Ron Parker wrote:
> correlate the out-of-band metadata to an individual transaction (i.e., 
> HTTP GET), then we could define a flow or transaction correlation TLV 
> type that would be added to both the data stream and the out-of-band 
> data.
>
> Thanks.
>
> Ron
>
>
> - Packet ordering is not insured: so a packet triggering some metadata 
> creation could arrive after the corresponding Congruent Metadata 
> signaling message. - Metadata messages can be lost
>
> I can see the benefit, as shown by Linda, to distinguish how to convey 
> metadata according to which function will benefit from its use.
>
> A fixed size header can carry a number of information allowing Service 
> Switching Functions to take local decision on how to handle incoming 
> or outgoing traffic. I share with Linda the feeling that an 
> Halpern [[email-masked]] Sent: Wednesday, April 09, 2014 2:17 PM 
> Parker Cc: [email-masked] Subject: Re: [sfc] Comments and suggestions to

---

From: Linda Dunbar <[email-masked]>
Date: Wed, 16 Apr 2014 16:15:03 +0000

There are two aspects of "congruency": 
- Out-of-band metadata traverse same path as actual user data flows.
- forward and reverse paths traversing same service functions. 

To enforce the Out-of-band Metadata traversing the same path as actual user data flows, it is important that they traverse the same SSF nodes in the same order, but don't really matter which underlay nodes being traversed through, is it correct?

To enforce the Metadata and user flow traversing the same SFF nodes in the same order, can't we use the Tunneling methods (VxLAN, IP in IP, or others) to achieve the goal?

But how to enforce the forward & reverse paths traversing the same service functions? 

Linda  
-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Joel M. Halpern
Sent: Sunday, April 13, 2014 3:30 PM
To: Ron Parker; Nicolas BOUTHORS; [email-masked]
Subject: Re: [sfc] Congruent metadata signaling considerations

Yours,
Joel