Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

From: <[email-masked]>
Date: Mon, 11 Apr 2016 11:14:02 +0000

Dear Thomas, all, 

As I mentioned during the meeting, there are several issues that are not covered in the last version of the draft. I already provided examples of the issues offline as requested by Martin. 

Cheers,
Med

> -----Message d'origine-----
> De : sfc [mailto:[email-masked]] De la part de Thomas Narten
> Envoyé : jeudi 31 mars 2016 04:48
> À : [email-masked]
> Objet : [sfc] WG last call for draft-ietf-sfc-nsh-04.txt
> 
> Dear WG:
> 
> This note begins a WG last call on draft-ietf-sfc-nsh-04.txt
> (https://datatracker.ietf.org/doc/draft-ietf-sfc-nsh/).
> 
> The editors of the NSH document have indicated that they have
> addressed all known comments and that there are no open issues with
> the current version of the document.
> 
> Substantive comments to the list please, editorial comments can go
> directly to the document editors.
> 
> We'll also get a brief update from the editors at next week's
> meeting. If there are any remaining issues with the document, raising
> them before the meeting would be especially helpful.
> 
> For the chairs,
> Thomas
> 
> _______________________________________________
> sfc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sfc

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Tue, 12 Apr 2016 19:16:52 +0000

Thomas, SFC,

I spent some time in the airplane back from beautiful Buenos Aires, re-reading draft-ietf-sfc-nsh-04.txt from top to bottom.

I fully support this document advancing now and being sent to the IESG. It realizes the SFC Architecture and squarely fulfills the WG’s milestone of:
  Aug 2015 - Submit to IESG Standards Track document specifying the generic service function chaining header encapsulation

Plus, frankly, it was quite refreshing to see the presentation at IETF95 and the four slides of implementation report and running code.

I do have quite a number of comments and suggestions. None of these are totally blocking, but I strongly recommend these are fixed sooner rather than later:

Abstract

   This draft describes a Network Service Header (NSH) inserted onto
   encapsulated packets or frames to realize service function paths.
   NSH also provides a mechanism for metadata exchange along the
   instantiated service path.  NSH is the SFC encapsulation as per SFC
   Architecture [SFC-arch]

:s/draft/document/g

Please, no citations in the Abstract. You can write down “RFC 7665” but not “[RFC 7665]”.

1.  Requirements Language

Please add that the Terminology from RFC 7665 is used.

2.1.  Definition of Terms

There is a bit of repetition with the terminology in RFC 7665. It might be cleaner to just refer to the architecture and only expand definition of new terms.

Perhaps define also “Service Plane”, as it is thoroughly used?

3.  Network Service Header

[…]
   NSH is added by a Service Classifier.  The NSH header is removed by
   the last SFF in the chain or by a SF that consumes the packet.

This sentence should start as “The NSH” (missing article).

3.1.  Network Service Header Format

   A NSH is composed of a 4-byte Base Header, a 4-byte Service Path
   Header and Context Headers, as shown in Figure 1 below.

:s/A NSH/An NSH/g

   MD- Type = 0x2

Extraneous space.

   2.  Path MTU Discovery [RFC1191]"describes a technique for
       dynamically discovering the maximum transmission unit (MTU) of an
       arbitrary internet path" and can be utilized to ensure the the
       required packet size is used.

Please also add a reference to IPv6 PMTUD.

   +-------------------------------------------------------+
   |  SPI |  SI |  NH                 |   Transport        |
   +-------------------------------------------------------+
   |  10  | 255 |  1.1.1.1            |   VXLAN-gpe        |
   |  10  | 254 |  2.2.2.2            |   nvGRE            |
   |  10  | 251 |  10.1.2.3           |   GRE              |
   |  40  | 251 |  10.1.2.3           |   GRE              |
   |  50  | 200 |  01:23:45:67:89:ab  |   Ethernet         |
   |  15  | 212 |  Null (end of path) |   None             |
   +———————————————————————————+
[…]
    +-----------------------------------+
    |  SF  |  NH          |  Transport  |
    +-----------------------------------|
    |  SF2 |  10.1.1.1    |  VXLAN-gpe  |
    |  SF34|  192.168.1.1 |  UDP        |
    |  SF9 |  1.1.1.1     |  GRE        |
    +-----------------------------------+
[…]
   Examples of mapping for a topology:

   1.  Next SF is located at SFFb with locator 10.1.1.1
       SFFa mapping: SPI=10 --> VXLAN-gpe, dst-ip: 10.1.1.1

This comment applies to these tables, as well as to a few other tables, and paragraphs throughout:

Please use IP addresses from the TEST-NET / documentation block.

Further, please use IPv6 documentation prefixes.

9.  NSH Encapsulation Examples

The word “Examples” speaks to this already, but it might be useful to add a sentence about the non-normativeness of this section.

10.  Security Considerations

   Further security considerations are discussed in [nsh-sec].

Perhaps some of these, high-level, could be ported here to strengthen the NSH security considerations section?

14.  IANA Considerations

There’s a registry missing for the 3 "Reserved bits” of the MD-Type 2 TLV header:

   Reserved bits: three reserved bit are present for future use.  The
   reserved bits MUST be set to 0x0.

References:

ID-Nits complains about a number of reference and citation errors, which should be fixed:

  Checking references for intended status: Proposed Standard

(Missing, Unused).

Net,net, this draft should progress to the IESG now.

Thanks,

— Carlos.

> On Mar 30, 2016, at 10:48 PM, Thomas Narten <[email-masked]> wrote:
> 
> Dear WG:
>

---

From: Dave Dolson <[email-masked]>
Date: Tue, 12 Apr 2016 19:29:23 +0000

The NSH draft indicates that MD-Type 1 MUST be supported, yet the Mandatory Context Header fields are not defined in the document.

Issues with metadata semantics have been raised various times on this list and in drafts, and never satisfactorily addressed.
Examples:
https://tools.ietf.org/html/draft-vallamkonda-sfc-metadata-model-00.txt
https://www.ietf.org/archive/id/draft-wang-sfc-md-type-issues-00.txt
https://www.ietf.org/archive/id/draft-penno-sfc-packet-02.txt
https://mailarchive.ietf.org/arch/msg/sfc/2YvBakDgJ_hjY-BGvgWdcTTUDIs

So I don't feel this draft sufficiently explains the MD-Type 1 metadata well enough to explain how one can make an interoperable implementation.

The draft references metadata allocation drafts, but they have not been adopted yet, so (as I understand it) these "works in progress" cannot support draft-ietf-sfc-nsh.

I think it might be acceptable to say that the metadata must be undirectionally clonable (or better phrase).
In other words,

   Metadata used in NSH headers must been the condition that
   a transport-layer-stateful Service Function can save away
   metadata values that it has witnessed.  An injected packet can
   therefore be assigned a clone of metadata taken from an earlier
   packet going in the same direction.  For example, a Service Function
   can send a TCP packet using metadata cloned from another TCP packet
   of the same connection and direction.

   Note that the Service Function need not know the meaning of the
   metadata; it just needs to know it is safe to clone in this manner.

Another alternative would be to import one of the metadata allocation schemes into this draft as MD-Type 1, and allow IANA to allocate MD-types for other schemes.

-Dave

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Thomas Narten
Sent: Wednesday, March 30, 2016 10:48 PM
To: [email-masked]
Subject: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Dear WG:

For the chairs,
Thomas

---

From: Ron Parker <[email-masked]>
Date: Tue, 12 Apr 2016 19:41:21 +0000

Dave,

I, too, have raised this issue on the thread.   I think you hit the nail on the head regarding interoperability, which is the primary reason for having a specification, at all.    My mental model for the type-1 mandatory context headers is that of a composite 16-byte locally defined scratchpad.   I struggle with having its definition completely undefined in the document and still making it mandatory.    

Much of the justification offered has been to be friendly to hardware.   Surely hardware needs to know the exact format and meaning of these 16 bytes.

   Ron

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Dave Dolson
Sent: Tuesday, April 12, 2016 3:29 PM
To: Thomas Narten <[email-masked]>; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

-Dave

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Thomas Narten
Sent: Wednesday, March 30, 2016 10:48 PM
To: [email-masked]
Subject: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Dear WG:

This note begins a WG last call on draft-ietf-sfc-nsh-04.txt (https://datatracker.ietf.org/doc/draft-ietf-sfc-nsh/).

The editors of the NSH document have indicated that they have addressed all known comments and that there are no open issues with the current version of the document.

Substantive comments to the list please, editorial comments can go directly to the document editors.

We'll also get a brief update from the editors at next week's meeting. If there are any remaining issues with the document, raising them before the meeting would be especially helpful.

For the chairs,
Thomas

---

From: "Andrew G. Malis" <[email-masked]>
Date: Tue, 12 Apr 2016 17:18:19 -0400

Thomas,

I do NOT support this WG last call.

Back in November, I reported a bug in section 14.2.3 that was acknowledged
on the WG list, but was never corrected in the text. To repeat my report:

There’s a bug with section 14.2.3. The text should be corrected as follows:

OLD:

14.2.3.  TLV Class Registry

   IANA is requested to set up a registry of "TLV Types".  These are 16-
   bit values.  Registry entries are assigned by using the "IETF Review"
   policy defined in RFC 5226 [RFC5226].

NEW:

14.2.3.  TLV Class Registry

   IANA is requested to set up a registry of "TLV Classes".  These are 16-
   bit values.  Registry entries are assigned by using the "IETF Review"
   policy defined in RFC 5226 [RFC5226].

This leaves me curious as to what other acknowledged comments on the draft
may have been also left out of the editing process.

I also note that although the TLV Class registry is being created, there
are no values to be allocated within the registry. I believe that this is
contrary to IANA policy. Or if not, it’s at least questionable.

Section 11 still lists open items for WG resolution.

I also agree with the interoperability concerns raised by others on the
list.

Thanks,
Andy

---

From: "Paul Quinn (paulq)" <[email-masked]>
Date: Wed, 13 Apr 2016 01:03:11 +0000

Ron, Dave,

The draft describes the metadata as opaque, and really NSH is just the envelop for it.  The same can be said for the TLV as well -- there's no spec in NSH, not should there be.  In both cases, the metadata can/should/(must?) be define in auxiliary drafts.  There are several drafts re: metadata that can be discussed and eventually standardized.  It also bears mentioning that the control plane, today in opensource, defines the type 1 semantics.  That schemes works well, and mixed implementations can play together.

So, perhaps the best way forward:

1.  Agree on the envelop and the use of that envelop
2.  As a WG, adopt --> standardize the opaque data, and the TLVs
3.  As a WG, work on more detailed of use of the control plane to, when needed, define semantics

Thoughts?

Thanks,
Paul

> On Apr 12, 2016, at 3:41 PM, Ron Parker <[email-masked]> wrote:
> 
> Dave,
>

---

From: "Paul Quinn (paulq)" <[email-masked]>
Date: Wed, 13 Apr 2016 01:05:56 +0000

Andy,

On Apr 12, 2016, at 5:18 PM, Andrew G. Malis <[email-masked]<mailto:[email-masked]>> wrote:

Thomas,

Back in November, I reported a bug in section 14.2.3 that was acknowledged on the WG list, but was never corrected in the text. To repeat my report:

First off, my apologies if we missed including your editorial update.  We certainly will.  However, with all due respect, I'm not sure that warrants objecting to last call.  Carlos, in his email of support, also raised several editorial changes, which will be reflected in the new version.

OLD:

NEW:

This leaves me curious as to what other acknowledged comments on the draft may have been also left out of the editing process.

I also note that although the TLV Class registry is being created, there are no values to be allocated within the registry. I believe that this is contrary to IANA policy. Or if not, it’s at least questionable.

I also agree with the interoperability concerns raised by others on the list.

Thanks,
Andy

On Wed, Mar 30, 2016 at 10:48 PM, Thomas Narten <[email-masked]<mailto:[email-masked]>> wrote:
Dear WG:

For the chairs,
Thomas

_______________________________________________
sfc mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/sfc

_______________________________________________
sfc mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/sfc

---

From: Xuxiaohu <[email-masked]>
Date: Wed, 13 Apr 2016 01:41:32 +0000

Hi Thomas,

It said in the NSH draft:
 
  "6.  Transport Agnostic: NSH is transport independent and is carried
       in an overlay, over existing underlays.  If an existing overlay
       topology provides the required service path connectivity, that
       existing overlay may be used."

That means the NSH should be able to be transported over MPLS. However, according to the current NSH format definition, it's not safe to carry the NSH over MPLS due to the first nibble issue. Therefore, I believe this issue needs to be addressed before publication.

Best regards,
Xiaohu

> Sent: Thursday, March 31, 2016 10:48 AM
> To: [email-masked]
> The editors of the NSH document have indicated that they have addressed all
> known comments and that there are no open issues with the current version of
> the document.
> 
> We'll also get a brief update from the editors at next week's meeting. If there are
> any remaining issues with the document, raising them before the meeting would
> be especially helpful.
> 
> For the chairs,
> Thomas
>

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 12 Apr 2016 21:44:42 -0400

Xu,
      I do not believe that there is an MPLS specification that requires 
that all content other than IP must have a first nibble of 0 or 1. 
There are specifications where it is discussed as desirable.

It is in fact pretty trivial for us to change the format so that the 
first three bits are 0, but it burns several valuable flag bits.  It is 
an SFC working group decision, not, as far as I can tell, a violation of 
the MPLS specification.

Yours,
Joel

---

From: Xuxiaohu <[email-masked]>
Date: Wed, 13 Apr 2016 01:53:57 +0000

Joel,

> From: Joel M. Halpern [mailto:[email-masked]]
> Sent: Wednesday, April 13, 2016 9:45 AM
> To: Xuxiaohu; Thomas Narten; [email-masked]
>       I do not believe that there is an MPLS specification that requires that all
> content other than IP must have a first nibble of 0 or 1.

When encapsulating NSH over MPLS directly, the first nibble of the NSH must not be 4 or 6.

> It is in fact pretty trivial for us to change the format so that the first three bits are
> 0, but it burns several valuable flag bits.  It is an SFC working group decision,

That's the reason why I raised the first nibble question.

Best regards,
Xiaohu

> not, as far as I can tell, a violation of the MPLS specification.
> 
> Yours,
> Joel
> 
> > That means the NSH should be able to be transported over MPLS. However,
> according to the current NSH format definition, it's not safe to carry the NSH
> over MPLS due to the first nibble issue. Therefore, I believe this issue needs to be
> addressed before publication.
> >
> > Best regards,
> > Xiaohu
> >

---

From: Xuxiaohu <[email-masked]>
Date: Wed, 13 Apr 2016 02:26:56 +0000

Hi Joel,

Please see https://tools.ietf.org/html/rfc4928#page-4. Although it's a BCP RFC, it'd better to obey it, IMO. No?

Best regards,
Xiaohu

> From: Joel Halpern Direct [mailto:[email-masked]]
> Sent: Wednesday, April 13, 2016 10:09 AM
> To: Xuxiaohu
> Is there a standards track RFC that requires this?
> 
> I know it is a good practice.  It may even be a good idea for us as a design point,
> as I do want MPLS to be a usable transport for NSH.
> 
> But as far as I can tell, there is no standards track RFC that requires this.
> 
> Yours,
> Joel
> 
> On 4/12/16 9:53 PM, Xuxiaohu wrote:
> > Joel,
> >
> >>        I do not believe that there is an MPLS specification that
> >> requires that all content other than IP must have a first nibble of 0 or 1.
> >
> >> first three bits are 0, but it burns several valuable flag bits.  It
> >> is an SFC working group decision,
> >
> >> according to the current NSH format definition, it's not safe to
> >> carry the NSH over MPLS due to the first nibble issue. Therefore, I
> >> believe this issue needs to be addressed before publication.
> >>>
> >>> Best regards,
> >>> Xiaohu
> >>>
> >>>> meeting. If there are any remaining issues with the document,
> >>>> raising them before the meeting would be especially helpful.
> >>>>
> >>>> For the chairs,
> >>>> Thomas
> >>>>

---

From: Alia Atlas <[email-masked]>
Date: Tue, 12 Apr 2016 22:29:04 -0400

Xiaohu, Joel, and SFC WG group,

The first nibble question is quite relevant if it is expected that the NSH
header might be directly under
an MPLS label stack.  This issue arose rather unpleasantly for pseudo-wires
a long time ago and the
reasoning for it is much better documented, as Carlos pointed out in a
different thread, in RFC 4928 Section 3.

At the time that PWE3 was working on the control word and whether it was to
be mandatory (RFC 4385), it was clear that
there were devices that looked at the first nibble of a packet under the
MPLS header as a way to determine
if the packet was IPv4 or IPv6 and obtain flow-diversity from it.  The cost
of also verifying the associated checksum
if available wasn't deemed acceptable for several implementations.  Given
that determining as much entropy as
possible is still really important and that it is non-trivial to
negotiate/indicate the capability for including
an Entropy Label in the MPLS stack, I have no reason to believe that this
shortcut of looking at the first nibble
is no longer used or deployed.   Feel free to contact me off-list if you
believe this to be the case.

It is clear from the NSH base header:

  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |Ver|O|C|R|R|R|R|R|R|   Length  |    MD Type    | Next Protocol |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

that this consideration has been completely ignored.  In fact, an NSH base
header might have any value
of 0b0000, 0b0001, 0b0010, 0b0010 and if a version 01 for NSH were ever
defined, suddenly the traffic might
be subject to startling reordering if an MPLS transport were to be
considered.

Given a claim of NSH being transport-agnostic and repeated emphasis that
MPLS, as well as UDP,
is a valid transport for NSH, I do think this is a significant oversight
and needs, at a minimum, discussion and
explanation, and  - quite likely -  correction.

While I am on the topic of details of the NSH encapsulation, I would
request that Section 8 of draft-ietf-rtgwg-dt-encap-01
be read and considered!   I am not excited by having many different and
unique Next Protocol fields defined.
For instance, I note the definition of a nearly identical Next Protocol
field in https://datatracker.ietf.org/doc/draft-ietf-nvo3-vxlan-gpe .

While I am, of course, willing to reason to detailed and well-thought out
explanations for why each and every new
encapsulation needs its very own IANA-defined Next Protocol field, this
seems to me to be highly likely to require
consolidation so that the same Next Protocol field can be used between the
various encapsulations.

I will work on giving a through review of NSH as soon as practical.  I do
realize that there are multiple implementations
and while details of how the "Next Protocol" field are documented shouldn't
have a significant impact there, the
discussion about the first nibble is likely to.

Regards,
Alia

On Tue, Apr 12, 2016 at 9:53 PM, Xuxiaohu <[email-masked]> wrote:

> that all
> first three bits are

---

From: "Andrew G. Malis" <[email-masked]>
Date: Tue, 12 Apr 2016 23:08:14 -0400

Paul,

The editorial correction was only one of my objections, there are others in
my email as well.

Cheers,
Andy

On Tue, Apr 12, 2016 at 9:05 PM, Paul Quinn (paulq) <[email-masked]> wrote:

> Andy,
>
>
>
> On Apr 12, 2016, at 5:18 PM, Andrew G. Malis <[email-masked]> wrote:
>
> Thomas,
>
> First off, my apologies if we missed including your editorial update.  We
> certainly will.  However, with all due respect, I'm not sure that warrants
> objecting to last call.  Carlos, in his email of support, also raised
> several editorial changes, which will be reflected in the new version.
>
>

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 12 Apr 2016 23:33:19 -0400

With regard to MPLS encapsulation, I am not objecting to considering if 
we want to rearrange things to make it simpler to carry NSH in MPLS.

Given that the IETF has not ratified a standards track RFC stating that 
it is required for MPLS carriage to avoid the values, I was objecting 
the Xu overstating the situation.

It is not actually hard to avoid the values.  If we feel it is 
imnportant (and I am tempted to agree that it is worth considering) we 
can rearrange some reserved bits so that the first three bits are 0, the 
next two are the version, and the remaining bits are flags.  Makes us a 
little starved for flags, but if we agree that MPLS has stolen three 
bits of header, so be it.

There are other answers.  We could, for example, agree that when NSH is 
carried over MPLS, a dummy 0 label is inserted after the bottom of 
stack.  No worse than many things MPLS has done.  (I could argue both 
sides of which answer is right, and not persuade myself of either one.)

On the next protocol IDs, the last time we looked we could not find an 
ID registration that was reasonably small and covered the needed 
meanings.  I note that even the section you point to concludes that 
there may well be good reasons for using a separate ID space.

Yours,
Joel

On 4/12/16 10:29 PM, Alia Atlas wrote:
> The first nibble question is quite relevant if it is expected that the
> NSH header might be directly under
> an MPLS label stack.  This issue arose rather unpleasantly for
> pseudo-wires a long time ago and the
> At the time that PWE3 was working on the control word and whether it was
> to be mandatory (RFC 4385), it was clear that
> if the packet was IPv4 or IPv6 and obtain flow-diversity from it.  The
> cost of also verifying the associated checksum
> if available wasn't deemed acceptable for several implementations.
> Given that determining as much entropy as
> an Entropy Label in the MPLS stack, I have no reason to believe that
> this shortcut of looking at the first nibble
> that this consideration has been completely ignored.  In fact, an NSH
> base header might have any value
> While I am, of course, willing to reason to detailed and well-thought
> out explanations for why each and every new
> consolidation so that the same Next Protocol field can be used between
> the various encapsulations.
>
> I will work on giving a through review of NSH as soon as practical.  I
> do realize that there are multiple implementations
> and while details of how the "Next Protocol" field are documented
> shouldn't have a significant impact there, the
> On Tue, Apr 12, 2016 at 9:53 PM, Xuxiaohu <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     Joel,
>
>     > From: Joel M. Halpern [mailto:[email-masked] <mailto:[email-masked]>]
>     > To: Xuxiaohu; Thomas Narten;[email-masked] <mailto:[email-masked]>
>     carry the NSH
>      > >> From: sfc [mailto:[email-masked]
>     <mailto:[email-masked]>] On Behalf Of Thomas Narten
>      > >> To: [email-masked] <mailto:[email-masked]>
>     raising
>      > >> [email-masked] <mailto:[email-masked]>
>      > > [email-masked] <mailto:[email-masked]>
>     [email-masked] <mailto:[email-masked]>

---

From: Alia Atlas <[email-masked]>
Date: Tue, 12 Apr 2016 23:43:41 -0400

Joel,

Agreed on the first nibble question that there are multiple ways of
handling.  There is a BCP about it, which isn't quite standards track but
pretty darn close.   At the very least, the issue needs to be discussed and
addressed.

For the "Next Protocol" field, it looks like that may need to be a
discussion between NVO3, SFC and possibly others.  I think that'll just be a
documentation issue, as far as implementations are concerned - but I do
think that having a common "Next Protocol" field will be important for ease
in the future.

All things subject to debate of course - particularly at 11:38pm  local
time.

Regards,
Alia

On Tue, Apr 12, 2016 at 11:33 PM, Joel M. Halpern <[email-masked]>
wrote:

> With regard to MPLS encapsulation, I am not objecting to considering if we
> want to rearrange things to make it simpler to carry NSH in MPLS.
>
> Given that the IETF has not ratified a standards track RFC stating that it
> is required for MPLS carriage to avoid the values, I was objecting the Xu
> overstating the situation.
>
> It is not actually hard to avoid the values.  If we feel it is imnportant
> (and I am tempted to agree that it is worth considering) we can rearrange
> some reserved bits so that the first three bits are 0, the next two are the
> version, and the remaining bits are flags.  Makes us a little starved for
> flags, but if we agree that MPLS has stolen three bits of header, so be it.
>
> carried over MPLS, a dummy 0 label is inserted after the bottom of stack.
> No worse than many things MPLS has done.  (I could argue both sides of
> which answer is right, and not persuade myself of either one.)
>
> On the next protocol IDs, the last time we looked we could not find an ID
> registration that was reasonably small and covered the needed meanings.  I
> note that even the section you point to concludes that there may well be
> good reasons for using a separate ID space.
>
> Yours,
> Joel
>
>
>> requires that all

---

From: Loa Andersson <[email-masked]>
Date: Wed, 13 Apr 2016 12:31:35 +0800

Joel,

> the Xu overstating the situation.

RFC 2026 says:

    Some RFCs standardize the results of community deliberations about
    statements of principle or conclusions about what is the best way to
    perform some operations or IETF process function.  These RFCs form
    the specification has been adopted as a BCP,

I guess that the operative word is the "standardize" in the first
sentence. RFC 4928 and standardize treatment of the first nibble after
the label stack in MPLS networks.

/Loa

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Wed, 13 Apr 2016 12:17:19 +0000

Hi, Andy,

> I do NOT support this WG last call.

I’d like to better understand the basis for your note.

The NSH draft is the only WG document with a solution to the SFC encapsulation, satisfying a WG important milestone and deliverable.

>    policy defined in RFC 5226 [RFC5226].

Thank you for checking! I have the same routine of checking whether my review comments and suggestions had been incorporated, and that improves document quality.

This one is clearly a small editorial, which ought to be fixed. While it may be disconcerting or disappointing, the good thing is that it has an easy fix, and does not look like a publication showstopper... This is why we do WG LCs, and have several levels of review in the IETF.

> This leaves me curious as to what other acknowledged comments on the draft may have been also left out of the editing process.

I’m not sure how useful this comment is. Are you aware of other acked-but-unfixed comments? If so, please do raise them. Otherwise, this is not an actionable comment, it only creates FUD.

> I also note that although the TLV Class registry is being created, there are no values to be allocated within the registry. I believe that this is contrary to IANA policy. Or if not, it’s at least…

Questioning is good. I also have the same tendency.

In this case, there’s no IANA policy against creating registries with unallocated values. I just checked draft-leiba-cotton-iana-5226bis-11. Do you have a different pointer?

Regarding questioning it, I’ve seen it many times. I’ve done it in some of my RFCs as well, e.g., https://tools.ietf.org/html/rfc4884#section-10

> Section 11 still lists open items for WG resolution.

Tracking this back, this is from draft-quinn-…-old. This section should be removed.

Another good catch.

Thus far, I counted two editorials which should be fixed.

> I also agree with the interoperability concerns raised by others on the list.

This seems to be too generic of a comment. The open source and implementation report presented in Buenos Aires seems to speak louder than this one-liner.

I believe we should take interoperability most seriously. That’s why we are here.

Consequently, do you have specific interop concerns that have not been mentioned as part of the WGLC?

Thanks!

— Carlos.

> Thanks,
> Andy
> 
> 
> On Wed, Mar 30, 2016 at 10:48 PM, Thomas Narten <[email-masked] <mailto:[email-masked]>> wrote:
> Dear WG:
> 
> (https://datatracker.ietf.org/doc/draft-ietf-sfc-nsh/ <https://datatracker.ietf.org/doc/draft-ietf-sfc-nsh/>).
> 
> https://www.ietf.org/mailman/listinfo/sfc <https://www.ietf.org/mailman/listinfo/sfc>
>

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Wed, 13 Apr 2016 12:34:13 +0000

Xiaohu,

I think you are missing the point with this comment.

The fact that NSH should be able to be transported over MPLS does not mean that the NSH document (or the SFC WG) need to specify exactly how this happens, given each transport-specific constrains.

Transporting NSH over ethernet needs to include a trailing FCS. Transporting NSH over VxLAN-GPE needs a VXLAN-GPE NP assigned in that document. If someone wants to transport NSH over Frame Relay, they need a document specifying the actual encap format, an NLPID, etc.

All of these transports for NSH are possible. All of these need encap-specific specifications.

To carry NSH over MPLS, then specifics like the TTL value, ECMP-prevention, etc. needs to be specified.  It is possible. It should not be defined here.

As I understand, this issue should NOT be addressed in any SFC document. Certainly not before NSH publication (otherwise, there’s a potentially never-ending list of transports to specify).

Thanks,

— Carlos.

> On Apr 12, 2016, at 9:41 PM, Xuxiaohu <[email-masked]> wrote:
> 
> Hi Thomas,
>

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Wed, 13 Apr 2016 12:34:36 +0000

+1

This is the type of approach I have been suggesting in my various threads. Before re-inventing the wheel maybe we could add some spokes to an existing one. 

Jim Uttaro

"This email and any files transmitted with it are AT&T property, are confidential, and are intended solely for the use of the individual or entity to whom this email is addressed. If you are not one of the named recipient(s) or otherwise have reason to believe that you have received this message in error, please notify the sender and delete this message immediately from your computer. Any other use, retention, dissemination, forwarding, printing, or copying of this email is strictly prohibited."

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Joel M. Halpern
Sent: Tuesday, April 12, 2016 11:33 PM
To: Alia Atlas <[email-masked]>; Xuxiaohu <[email-masked]>; Martin Stiemerling <[email-masked]>; [email-masked]
Cc: Thomas Narten <[email-masked]>; Joel M. Halpern <[email-masked]>; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Yours,
Joel

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Wed, 13 Apr 2016 12:53:30 +0000

Alia,

Thanks for looking into this.

It is relevant. I believe ECMP-prevention (anti-aliasing with 0x4 and 0x6) is necessary for traffic over MPLS expecting in-order delivery.

Transporting NSH over MPLS, however, does not necessarily imply transporting NSH *directly* following the bottom LSE. There are other ways, as it was done with PWs.

I think showing that it is possible to appropriately (i.e., respecting BCPs) transport NSH over MPLS is very important. This may or may not have implications on the NSH base header. I believe however that such actual formal specification (beyond showing its possible) should not gate NSH.

Thanks!

— Carlos.

> On Apr 12, 2016, at 10:29 PM, Alia Atlas <[email-masked]> wrote:
> 
> The first nibble question is quite relevant if it is expected that the NSH header might be directly under
> an MPLS label stack.  This issue arose rather unpleasantly for pseudo-wires a long time ago and the
> reasoning for it is much better documented, as Carlos pointed out in a different thread, in RFC 4928 Section 3.
> 
> At the time that PWE3 was working on the control word and whether it was to be mandatory (RFC 4385), it was clear that
> there were devices that looked at the first nibble of a packet under the MPLS header as a way to determine
> if the packet was IPv4 or IPv6 and obtain flow-diversity from it.  The cost of also verifying the associated checksum
> if available wasn't deemed acceptable for several implementations.  Given that determining as much entropy as
> possible is still really important and that it is non-trivial to negotiate/indicate the capability for including
> an Entropy Label in the MPLS stack, I have no reason to believe that this shortcut of looking at the first nibble
> is no longer used or deployed.   Feel free to contact me off-list if you believe this to be the case.
> 
> that this consideration has been completely ignored.  In fact, an NSH base header might have any value
> of 0b0000, 0b0001, 0b0010, 0b0010 and if a version 01 for NSH were ever defined, suddenly the traffic might
> be subject to startling reordering if an MPLS transport were to be considered.
> 
> Given a claim of NSH being transport-agnostic and repeated emphasis that MPLS, as well as UDP,
> is a valid transport for NSH, I do think this is a significant oversight and needs, at a minimum, discussion and
> While I am on the topic of details of the NSH encapsulation, I would request that Section 8 of draft-ietf-rtgwg-dt-encap-01
> be read and considered!   I am not excited by having many different and unique Next Protocol fields defined.
> For instance, I note the definition of a nearly identical Next Protocol field in https://datatracker.ietf.org/doc/draft-ietf-nvo3-vxlan-gpe <https://datatracker.ietf.org/doc/draft-ietf-nvo3-vxlan-gpe> .
> 
> While I am, of course, willing to reason to detailed and well-thought out explanations for why each and every new
> encapsulation needs its very own IANA-defined Next Protocol field, this seems to me to be highly likely to require
> consolidation so that the same Next Protocol field can be used between the various encapsulations.
> 
> I will work on giving a through review of NSH as soon as practical.  I do realize that there are multiple implementations
> and while details of how the "Next Protocol" field are documented shouldn't have a significant impact there, the
> On Tue, Apr 12, 2016 at 9:53 PM, Xuxiaohu <[email-masked] <mailto:[email-masked]>> wrote:
> Joel,
> 
> > To: Xuxiaohu; Thomas Narten; [email-masked] <mailto:[email-masked]>
> > >> From: sfc [mailto:[email-masked] <mailto:[email-masked]>] On Behalf Of Thomas Narten

---

From: Nadeau Thomas <[email-masked]>
Date: Wed, 13 Apr 2016 09:09:05 -0400

I agree with Paul. WG LC is exactly the time to address any loose ends such as this. There was no concerted effort to avoid/ignore your suggested updates as far as I know.

	—Tom

> On Apr 12, 2016:9:05 PM, at 9:05 PM, Paul Quinn (paulq) <[email-masked]> wrote:
> 
> Andy,
> 
> 
> 
>> On Apr 12, 2016, at 5:18 PM, Andrew G. Malis <[email-masked] <mailto:[email-masked]>> wrote:
>> 
>> Thomas,
>>

---

From: Dave Dolson <[email-masked]>
Date: Wed, 13 Apr 2016 13:13:06 +0000

Paul,
Are you unhappy with the metadata constraints I suggested below?
Perhaps qualified by "In the absence of out-of-band configuration..." ?

I believe that the schemes proposed in these drafts satisfy the constraints:
- draft-sfc-nsh-dc-allocation-4, 
- draft-napper-sfc-nsh-broadband-allocation-00 

(I thought there were more allocation schemes, but I guess they expired.)

I'll update it here:

   In the absence of out-of-band configuration to indicate otherwise,
   metadata used in NSH headers must meet the condition that
   a transport-layer-stateful Service Function can save away
   metadata values that it has witnessed.  An injected packet can
   therefore be assigned a clone of metadata taken from an earlier
   packet going in the same direction.  For example, a Service Function
   can send a TCP packet using metadata cloned from another TCP packet
   of the same connection and direction.

-Dave

-----Original Message-----
From: Paul Quinn (paulq) [mailto:[email-masked]] 
Sent: Tuesday, April 12, 2016 9:03 PM
To: Ron Parker
Cc: Dave Dolson; Thomas Narten; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Ron, Dave,

Thoughts?

Thanks,
Paul

---

From: Alia Atlas <[email-masked]>
Date: Wed, 13 Apr 2016 09:13:22 -0400

Carlos,

As we all know, it is possible to do many many things with technology -
once we figure out how.
Saying that there are approaches other than reorganizing the first nibble
is fine.  Claiming that we'll
be anywhere near interoperability or standardization without it being
clarified does not seem plausible.

For instance, I will note that the description of 0 (Explicit NULL) is
still defined in RFC3032 as implying
the packet underneath is IPv4.  I really did think that we'd talked in MPLS
about fixing this - and maybe
Loa or others can give us the reference - but that's what the IANA pointer
has right now.

I understand the desire to get NSH done - deeply!

I also hate late surprises and am trying to clear time to do a more
thorough review of NSH as well as
requesting other focused reviews.

This issue is not new - and the issues I saw in a quick look are those that
are thoroughly documented
in draft-ietf-rtgwg-dt-encap-01
<https://datatracker.ietf.org/doc/draft-ietf-rtgwg-dt-encap/> which
considered NSH as one of the encapsulations.

It seems clear that there *are* transport-specific issues and we need a
place and a way to capture them.

Regards,
Alia

On Wed, Apr 13, 2016 at 8:53 AM, Carlos Pignataro (cpignata) <
[email-masked]> wrote:

> It is relevant. I believe ECMP-prevention (anti-aliasing with 0x4 and 0x6)
> is necessary for traffic over MPLS expecting in-order delivery.
>
> Transporting NSH over MPLS, however, does not necessarily imply
> transporting NSH *directly* following the bottom LSE. There are other ways,
> as it was done with PWs.
>
> I think showing that it is possible to appropriately (i.e., respecting
> BCPs) transport NSH over MPLS is very important. This may or may not have
> implications on the NSH base header. I believe however that such actual
> formal specification (beyond showing its possible) should not gate NSH.
>
> Thanks!
>
> — Carlos.
>
>

---

From: Loa Andersson <[email-masked]>
Date: Wed, 13 Apr 2016 21:43:35 +0800

Alia,

I think you are referring to RFC 4182, it is more than a decade
since it was published.

/Loa

On 2016-04-13 21:13, Alia Atlas wrote:
> Saying that there are approaches other than reorganizing the first
> nibble is fine.Â  Claiming that we'll
> the packet underneath is IPv4.Â  I really did think that we'd talked in
> MPLS about fixing this - and maybe
> Loa or others can give us the reference - but that's what the IANA
> pointer has right now.
>
> I understand the desire to get NSH done - deeply! Â
>
> This issue is not new - and the issues I saw in a quick look are those
> that are thoroughly documented
> inÂ draft-ietf-rtgwg-dt-encap-01
> <https://datatracker.ietf.org/doc/draft-ietf-rtgwg-dt-encap/>Â which
> It seems clear that there *are*Â transport-specific issues and we need a
> On Wed, Apr 13, 2016 at 8:53 AM, Carlos Pignataro (cpignata)
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>     Alia,
>
>     It is relevant. I believe ECMP-prevention (anti-aliasing with 0x4
>     and 0x6) is necessary for traffic over MPLS expecting in-order
>     delivery.Â
>
>     transporting NSH *directly* following the bottom LSE. There are
>     other ways, as it was done with PWs.Â
>
>     I think showing that it is possible to appropriately (i.e.,
>     respecting BCPs) transport NSH over MPLS is very important. This may
>     or may not have implications on the NSH base header. I believe
>     however that such actual formal specification (beyond showing its
>     possible) should not gate NSH.
>
>     Thanks!
>
>     â€” Carlos.
>
>
>>     On Apr 12, 2016, at 10:29 PM, Alia Atlas <[email-masked]
>>     <mailto:[email-masked]>> wrote:
>>
>>     an MPLS label stack.Â  This issue arose rather unpleasantly for
>>     reasoning for it is much better documented, as Carlos pointed out
>>     in a different thread, in RFC 4928 Section 3.
>>
>>     At the time that PWE3 was working on the control word and whether
>>     it was to be mandatory (RFC 4385), it was clear that
>>     there were devices that looked at the first nibble of a packet
>>     under the MPLS header as a way to determine
>>     if the packet was IPv4 or IPv6 and obtain flow-diversity from
>>     it.Â  The cost of also verifying the associated checksum
>>     if available wasn't deemed acceptable for several
>>     implementations.Â  Given that determining as much entropy as
>>     an Entropy Label in the MPLS stack, I have no reason to believe
>>     that this shortcut of looking at the first nibble
>>     is no longer used or deployed. Â  Feel free to contact me off-list
>>     if you believe this to be the case.
>>
>>     that this consideration has been completely ignored.Â  In fact, an
>>     NSH base header might have any value
>>     is a valid transport for NSH, I do think this is a significant
>>     oversight and needs, at a minimum, discussion and
>>     explanation, and Â - quite likely - Â correction.
>>
>>     While I am on the topic of details of the NSH encapsulation, I
>>     would request that Section 8 of draft-ietf-rtgwg-dt-encap-01
>>     be read and considered! Â  I am not excited by having many
>>     different and unique Next Protocol fields defined.
>>     For instance, I note the definition of a nearly identical Next
>>     Protocol field in
>>     https://datatracker.ietf.org/doc/draft-ietf-nvo3-vxlan-gpe .
>>
>>     While I am, of course, willing to reason to detailed and
>>     well-thought out explanations for why each and every new
>>     encapsulation needs its very own IANA-defined Next Protocol field,
>>     this seems to me to be highly likely to require
>>     I will work on giving a through review of NSH as soon as
>>     practical.Â  I do realize that there are multiple implementations
>>         >Â  Â  Â  Â I do not believe that there is an MPLS specification that requires that all
>>         > 0, but it burns several valuable flag bits.Â  It is an SFC working group decision,
>>
>>         > >Â  Â  "6.Â  Transport Agnostic: NSH is transport
>>         independent and is carried
>>         > >Â  Â  Â  Â  Â in an overlay, over existing underlays.Â  If
>>         an existing overlay
>>         > >Â  Â  Â  Â  Â topology provides the required service path
>>         connectivity, that
>>         > >Â  Â  Â  Â  Â existing overlay may be used."
>>         > >

---

From: Alia Atlas <[email-masked]>
Date: Wed, 13 Apr 2016 09:55:57 -0400

Loa,

Thanks!   I knew it was out there.

Regardless,  the first nibble question can't be resolved by adding more
labels to the stack!    The whole issue is looking at the S bit to guess
what's under the MPLS label stack.  That's why PWE3 uses code-words.

There needs to be an answer beyond handwaving that something can be
invented.

Of all the implementations, what transports is NSH carried over?  Do we
have any diversity?

Regards,
Alia
On Apr 13, 2016 9:43 AM, "Loa Andersson" <[email-masked]> wrote:

Alia,

/Loa

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Wed, 13 Apr 2016 13:59:19 +0000

Alia,

It seems I was not totally clear before. Let me try again.

My point is that if there are approaches to transport the SFC Encapsulation over MPLS, such as defining a CW-like thing starting with 0x0 (I’m not talking about Explicit Null) or defining an “Explicit NSH” Label value, or others, we do not need to wait for those specifications to become RFCs before advancing the SFC encapsulation. Further, that that’s probably work for MPLS and not for SFC (to find the best way to transport NSH over MPLS) — although clearly we take your guidance there :-) Yes, clarifications are good. But to exemplify, we cannot say IPv6 does not interop because IPv6 over Bluetooth Low-Energy or IPv6 over 802.XYZ.alpha.pi is still being defined.

Thorough deep reviews as well as focused reviews are good! Thanks. Not to go down a tangent, but I had mentioned also that perhaps there was not deep SFC representation in the Encap-DT.

Thanks,

— Carlos.

> On Apr 13, 2016, at 9:13 AM, Alia Atlas <[email-masked]> wrote:
> 
> Carlos,
> 
> As we all know, it is possible to do many many things with technology - once we figure out how.
> Saying that there are approaches other than reorganizing the first nibble is fine.  Claiming that we'll
> be anywhere near interoperability or standardization without it being clarified does not seem plausible.
> 
> For instance, I will note that the description of 0 (Explicit NULL) is still defined in RFC3032 as implying
> the packet underneath is IPv4.  I really did think that we'd talked in MPLS about fixing this - and maybe
> Loa or others can give us the reference - but that's what the IANA pointer has right now.
> 
> I also hate late surprises and am trying to clear time to do a more thorough review of NSH as well as
> This issue is not new - and the issues I saw in a quick look are those that are thoroughly documented
> in draft-ietf-rtgwg-dt-encap-01 <https://datatracker.ietf.org/doc/draft-ietf-rtgwg-dt-encap/> which considered NSH as one of the encapsulations.
> 
> It seems clear that there are transport-specific issues and we need a place and a way to capture them.
> 
> Regards,
> Alia
> 
> On Wed, Apr 13, 2016 at 8:53 AM, Carlos Pignataro (cpignata) <[email-masked] <mailto:[email-masked]>> wrote:
> Alia,
> 
>> On Apr 12, 2016, at 10:29 PM, Alia Atlas <[email-masked] <mailto:[email-masked]>> wrote:
>>

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 13 Apr 2016 10:17:00 -0400

Yes, Alia, there are many transports out there being used for service 
chaining.  Most of the folks using these farious transports expect to 
support NSH.  Several of these have been brought forward  as 
Internet-Drafts.  Not all.  Which makes sense as standardizing transport 
behavior for NSH is out of scope for the working group.

It does seem that the WG has to decide if they want to adjust the header 
format to slightly simplify carrying one transport.  The difficulty is 
that the WG is not scoped to evaluate the various ways NSH might be 
carried on MPLS, much less to standardize and pick one.

Yours,
Joel

On 4/13/16 9:55 AM, Alia Atlas wrote:
> On Apr 13, 2016 9:43 AM, "Loa Andersson" <[email-masked] <mailto:[email-masked]>>
> wrote:
>
>     Alia,
>
>         <[email-masked] <mailto:[email-masked]>
>         <mailto:[email-masked] <mailto:[email-masked]>>> wrote:
>
>              Alia,
>
>                  On Apr 12, 2016, at 10:29 PM, Alia Atlas
>             <[email-masked] <mailto:[email-masked]>
>                  <mailto:[email-masked] <mailto:[email-masked]>>>
>             wrote:
>
>                  The first nibble question is quite relevant if it is
>             expected that
>                  the NSH header might be directly under
>                  of 0b0000, 0b0001, 0b0010, 0b0010 and if a version 01
>             for NSH were
>                  ever defined, suddenly the traffic might
>                  consolidation so that the same Next Protocol field can
>             be used
>                  between the various encapsulations.
>
>                  On Tue, Apr 12, 2016 at 9:53 PM, Xuxiaohu
>             <[email-masked] <mailto:[email-masked]>
>                  <mailto:[email-masked]
>             <mailto:[email-masked]>>> wrote:
>
>                      Joel,
>
>                      > From: Joel M. Halpern [mailto:[email-masked]
>             <mailto:[email-masked]> <mailto:[email-masked]
>             <mailto:[email-masked]>>]
>                      > To: Xuxiaohu; Thomas Narten;[email-masked]
>             <mailto:[email-masked]> <mailto:[email-masked]
>             <mailto:[email-masked]>>
>                      When encapsulating NSH over MPLS directly, the
>             first nibble of
>                      the NSH must not be 4 or 6.
>
>                      > > That means the NSH should be able to be
>             transported over
>                      MPLS. However,
>                      > according to the current NSH format definition,
>             it's not
>                      safe to carry the NSH
>             <mailto:[email-masked]>
>                      <mailto:[email-masked]
>             <mailto:[email-masked]>>] On Behalf Of Thomas Narten
>             <mailto:[email-masked] <mailto:[email-masked]>>
>                      > >> addressed all known comments and that there
>             are no open
>                      issues with
>                      > >> meeting. If there are any remaining issues
>             with the
>                      document, raising
>             <mailto:[email-masked] <mailto:[email-masked]>>
>
>             <mailto:[email-masked] <mailto:[email-masked]>>
>
>             [email-masked] <mailto:[email-masked]> <mailto:[email-masked]
>             <mailto:[email-masked]>>
>
>             [email-masked] <mailto:[email-masked]> <mailto:[email-masked]
>             <mailto:[email-masked]>>

---

From: Ron Parker <[email-masked]>
Date: Wed, 13 Apr 2016 14:18:55 +0000

I'll place my vote to make the minor modifications to the NSH header in order to be MPLS friendly.   I think this is a good return on investment.

Thanks.

   Ron

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Joel M. Halpern
Sent: Wednesday, April 13, 2016 10:17 AM
To: Alia Atlas <[email-masked]>; Loa Andersson <[email-masked]>
Cc: Thomas Narten <[email-masked]>; Carlos Pignataro (cpignata) <[email-masked]>; [email-masked]; Jim Guichard (jguichar) <[email-masked]>; Martin Stiemerling <[email-masked]>; Xuxiaohu <[email-masked]>
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Yes, Alia, there are many transports out there being used for service chaining.  Most of the folks using these farious transports expect to support NSH.  Several of these have been brought forward  as Internet-Drafts.  Not all.  Which makes sense as standardizing transport behavior for NSH is out of scope for the working group.

It does seem that the WG has to decide if they want to adjust the header format to slightly simplify carrying one transport.  The difficulty is that the WG is not scoped to evaluate the various ways NSH might be carried on MPLS, much less to standardize and pick one.

Yours,
Joel

On 4/13/16 9:55 AM, Alia Atlas wrote:
> Of all the implementations, what transports is NSH carried over?  Do 
> we have any diversity?
>
> Regards,
> Alia
>

---

From: Azhar Sayeed <[email-masked]>
Date: Wed, 13 Apr 2016 10:18:55 -0400

Watching this discussion with amusement 

How many times have had to patch things later after documents were published and rev them..since some one has raised the issue of first nibble there are two ways of tackling this

1. Address it in this document for most generic transport types - IPv4, IPv6, MPLS and Ethernet - 
2. Write a separate draft that addresses all transport types and how to make it happen for various use cases 

Because no document by itself completely guarantees interoperability - we can take any specification; For example MPLS -TE requires RSVP protocol specification to guarantee interoperability and on an on..There are always dependencies for a complete implementation and that minimal set is necessary for Interop

Thanks
Azhar

> On Apr 13, 2016, at 9:59 AM, Carlos Pignataro (cpignata) <[email-masked]> wrote:
> 
> Alia,
> 
>> On Apr 13, 2016, at 9:13 AM, Alia Atlas <[email-masked] <mailto:[email-masked]>> wrote:
>> 
>> Carlos,
>>

---

From: Dave Dolson <[email-masked]>
Date: Wed, 13 Apr 2016 17:34:04 +0000

I’m going to show my ignorance and ask how Ethernet is carried over MPLS? The first nybble is part of the Destination MAC address.
OUIs have been allocated beginning with 4 and also with 6.

From: sfc [mailto:[email-masked]] On Behalf Of Alia Atlas
Sent: Wednesday, April 13, 2016 9:56 AM
To: Loa Andersson
Cc: Thomas Narten; Carlos Pignataro (cpignata); [email-masked]; Jim Guichard (jguichar); Martin Stiemerling; Xuxiaohu; Joel M. Halpern
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Loa,

Regardless,  the first nibble question can't be resolved by adding more labels to the stack!    The whole issue is looking at the S bit to guess what's under the MPLS label stack.  That's why PWE3 uses code-words.

There needs to be an answer beyond handwaving that something can be invented.

Of all the implementations, what transports is NSH carried over?  Do we have any diversity?

Regards,
Alia
On Apr 13, 2016 9:43 AM, "Loa Andersson" <[email-masked]<mailto:[email-masked]>> wrote:
Alia,

/Loa

Regards,
Alia

On Wed, Apr 13, 2016 at 8:53 AM, Carlos Pignataro (cpignata)
<[email-masked]<mailto:[email-masked]> <mailto:[email-masked]<mailto:[email-masked]>>> wrote:

    Alia,

    Thanks!
    â€” Carlos.

    On Apr 12, 2016, at 10:29 PM, Alia Atlas <[email-masked]<mailto:[email-masked]>
    <mailto:[email-masked]<mailto:[email-masked]>>> wrote:

    The first nibble question is quite relevant if it is expected that
    the NSH header might be directly under
    an MPLS label stack.Â  This issue arose rather unpleasantly for

    NSH base header might have any value
    of 0b0000, 0b0001, 0b0010, 0b0010 and if a version 01 for NSH were
    ever defined, suddenly the traffic might
    be subject to startling reordering if an MPLS transport were to be
    considered.

    Given a claim of NSH being transport-agnostic and repeated
    emphasis that MPLS, as well as UDP,
    is a valid transport for NSH, I do think this is a significant
    oversight and needs, at a minimum, discussion and
    explanation, and Â - quite likely - Â correction.

    consolidation so that the same Next Protocol field can be used

    Regards,
    Alia

    On Tue, Apr 12, 2016 at 9:53 PM, Xuxiaohu <[email-masked]<mailto:[email-masked]>
    <mailto:[email-masked]<mailto:[email-masked]>>> wrote:

        Joel,

        > From: Joel M. Halpern [mailto:[email-masked]<mailto:[email-masked]> <mailto:[email-masked]<mailto:[email-masked]>>]
        > To: Xuxiaohu; Thomas Narten;[email-masked]<mailto:[email-masked]> <mailto:[email-masked]<mailto:[email-masked]>>

        > content other than IP must have a first nibble of 0 or 1.

        When encapsulating NSH over MPLS directly, the first nibble of
        the NSH must not be 4 or 6.

        Best regards,
        Xiaohu

        an existing overlay
        connectivity, that

        > >> From: sfc [mailto:[email-masked]<mailto:[email-masked]>
        <mailto:[email-masked]<mailto:[email-masked]>>] On Behalf Of Thomas Narten
        > >> To: [email-masked]<mailto:[email-masked]> <mailto:[email-masked]<mailto:[email-masked]>>
        > >> [email-masked]<mailto:[email-masked]> <mailto:[email-masked]<mailto:[email-masked]>>

        > > [email-masked]<mailto:[email-masked]> <mailto:[email-masked]<mailto:[email-masked]>>

        _______________________________________________
        sfc mailing list
        [email-masked]<mailto:[email-masked]> <mailto:[email-masked]<mailto:[email-masked]>>

    _______________________________________________
    sfc mailing list
    [email-masked]<mailto:[email-masked]> <mailto:[email-masked]<mailto:[email-masked]>>
    https://www.ietf.org/mailman/listinfo/sfc

---

From: Alia Atlas <[email-masked]>
Date: Wed, 13 Apr 2016 14:12:15 -0400

Hi Dave,

On Wed, Apr 13, 2016 at 1:34 PM, Dave Dolson <[email-masked]> wrote:

> I’m going to show my ignorance and ask how Ethernet is carried over MPLS?
> The first nybble is part of the Destination MAC address.
>

Thanks for asking and don't be sorry for not having to live through the fun
and joy of standardizing pseudo-wires.   The very very short form is that
there is a 32-bit code-word defined that sits above the Ethernet frame and
right below the MPLS stack.  This is to deal with implementations that look
beneath the MPLS stack to hunt down more fields to hash to get good entropy
for ECMP or LAG.

As I recall, when I looked a long long (er, let's not get into how long)
time ago, the likeliness of actual collision with allocated OUIs was quite
small.

Alia

>
>
>
>
>
>
> *From:* sfc [mailto:[email-masked]] *On Behalf Of *Alia Atlas
> *Sent:* Wednesday, April 13, 2016 9:56 AM
> *To:* Loa Andersson
> *Cc:* Thomas Narten; Carlos Pignataro (cpignata); [email-masked]; Jim
> Guichard (jguichar); Martin Stiemerling; Xuxiaohu; Joel M. Halpern
>
> *Subject:* Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt
>
>
>
> Loa,
>

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Wed, 13 Apr 2016 18:27:48 +0000

> On Apr 13, 2016, at 10:17 AM, Joel M. Halpern <[email-masked]> wrote:
> 

I’ll add to this, without "taking sides", that it is a decision tradeoff between a detriment to all-but-one transports (an unused nibble) to cater to the specifics of one, with modulo to implementation implications, versus optimizing for one transport. But as Joel said, not SFCs scope, charter; perhaps expertise ought to be drawn from MPLS.

Alia,

If the requirement is to encapsulate NSH to transport it over MPLS networks, in a point-to-point fashion, in a way that prevents ECMP load-balancing, the IETF knows how to do that: It’s called a Pseudowire.

That’s not "handwaving that something can be invented”. The current NSH can be transported over MPLS.

There are other alternatives such as specifying an “NSH Explicit Label” (or reusing GAL to indicate a header follows), followed by a CW/ACH with the first nibble as 0x0.

That one liner is clearly not a specification, but it’s also not handwaving.

These ought to be weighted against the alternative of modifying the NSH encap (plus, really, if we change NSH to start with 0x0, it would not be transport independent anymore :-)

BTW, as an aside, as you know not all Encapsulations defined in the IETF begin with a fixed 0x0.

Thanks,

— Carlos.

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Wed, 13 Apr 2016 18:29:05 +0000

Hi, Dave,

> On Apr 13, 2016, at 1:34 PM, Dave Dolson <[email-masked]> wrote:
> 

https://tools.ietf.org/html/rfc4448

> OUIs have been allocated beginning with 4 and also with 6.

https://tools.ietf.org/html/rfc4448#section-4.6

Thanks!

— Carlos.

> 
> 
> From: sfc [mailto:[email-masked] <mailto:[email-masked]>] On Behalf Of Alia Atlas
> Cc: Thomas Narten; Carlos Pignataro (cpignata); [email-masked] <mailto:[email-masked]>; Jim Guichard (jguichar); Martin Stiemerling; Xuxiaohu; Joel M. Halpern
> On Apr 13, 2016 9:43 AM, "Loa Andersson" <[email-masked] <mailto:[email-masked]>> wrote:
> Alia,
> 
> <https://datatracker.ietf.org/doc/draft-ietf-rtgwg-dt-encap/ <https://datatracker.ietf.org/doc/draft-ietf-rtgwg-dt-encap/>>Â which
> 
> <[email-masked] <mailto:[email-masked]> <mailto:[email-masked] <mailto:[email-masked]>>> wrote:
> 
>     Alia,
> 
>     On Apr 12, 2016, at 10:29 PM, Alia Atlas <[email-masked] <mailto:[email-masked]>
>     <mailto:[email-masked] <mailto:[email-masked]>>> wrote:
> 
>     https://datatracker.ietf.org/doc/draft-ietf-nvo3-vxlan-gpe <https://datatracker.ietf.org/doc/draft-ietf-nvo3-vxlan-gpe> .
> 
>     On Tue, Apr 12, 2016 at 9:53 PM, Xuxiaohu <[email-masked] <mailto:[email-masked]>
>     <mailto:[email-masked] <mailto:[email-masked]>>> wrote:
> 
>         Joel,
> 
>         > From: Joel M. Halpern [mailto:[email-masked] <mailto:[email-masked]> <mailto:[email-masked] <mailto:[email-masked]>>]
>         > To: Xuxiaohu; Thomas Narten;[email-masked] <mailto:[email-masked]> <mailto:[email-masked] <mailto:[email-masked]>>
>         > >> From: sfc [mailto:[email-masked] <mailto:[email-masked]>
>         <mailto:[email-masked] <mailto:[email-masked]>>] On Behalf Of Thomas Narten
>         > >> To: [email-masked] <mailto:[email-masked]> <mailto:[email-masked] <mailto:[email-masked]>>
>         > >> [email-masked] <mailto:[email-masked]> <mailto:[email-masked] <mailto:[email-masked]>>
> 
>         > > [email-masked] <mailto:[email-masked]> <mailto:[email-masked] <mailto:[email-masked]>>
> 
>         [email-masked] <mailto:[email-masked]> <mailto:[email-masked] <mailto:[email-masked]>>
> 
>     [email-masked] <mailto:[email-masked]> <mailto:[email-masked] <mailto:[email-masked]>>

---

From: Alia Atlas <[email-masked]>
Date: Wed, 13 Apr 2016 14:58:46 -0400

Carlos,

On Wed, Apr 13, 2016 at 9:59 AM, Carlos Pignataro (cpignata) <
[email-masked]> wrote:

> My point is that if there are approaches to transport the SFC
> Encapsulation over MPLS, such as defining a CW-like thing starting with 0x0
> (I’m not talking about Explicit Null) or defining an “Explicit NSH” Label
> value, or others, we do not need to wait for those specifications to become
> RFCs before advancing the SFC encapsulation. Further, that that’s probably
> work for MPLS and not for SFC (to find the best way to transport NSH over
> MPLS) — although clearly we take your guidance there :-) Yes,
> clarifications are good. But to exemplify, we cannot say IPv6 does not
> interop because IPv6 over Bluetooth Low-Energy or IPv6 over
> 802.XYZ.alpha.pi is still being defined.
>

Sure - but there are link layers that IPv6 does work over already ;-)
I do feel that there is a need for at least a WG-agreed  approach to the
issue.
I am thinking in general about how to handle the transport-specific aspects
when SFC is the WG with full context.  I'm fairly certain that explaining
issues around congestion control for UDP, for instance, are quite
interesting.

> Thorough deep reviews as well as focused reviews are good! Thanks. Not to
> go down a tangent, but I had mentioned also that perhaps there was not deep
> SFC representation in the Encap-DT.
>

Yes and I had hoped it would facilitate even better coordination on this
aspect of things.  Lessons learned.

Regards,
alia

>> It is relevant. I believe ECMP-prevention (anti-aliasing with 0x4 and
>> 0x6) is necessary for traffic over MPLS expecting in-order delivery.
>>

---

From: "Fedyk, Don" <[email-masked]>
Date: Wed, 13 Apr 2016 19:05:17 +0000

Hi Thomas 

At this point I do not support a last call on the NSH header. There is a lot of comments and I don't think it is at a point where it is last call ready.

I have be banging my head against the wall on this one because I would like more concrete around the forwarding aspects.  I do accept that there is a draft (draft-kumar-sfc-nsh-forwarding-00 ) now that addresses some the confusion in NSH on forwarding and it helps.  Although it seems odd to let a NSH draft progress as confusing and to clarify it later in another draft. 

My main objection is related to the fact that for a draft that claims it is independent of forwarding it simply has not been proven.  That does not mean we need to specify forwarding but I'd still like to see the data formats of what are being "prototyped" today so I can compare with the other data formats that I know today.  For example the MAC chaining draft we authored would like to use NSH for metadata but the MAC chaining local MACs bear a relationship with the NSH SDI & SI.  In fact without these fields as mandatory it would be more independent in my case.  I'm sure there are similar aspects with other transports types but until I see them and the operations I can't assess transport independence claims.

There are other inconsistencies that I could live with in the draft relating to the logic of the MD types etc. (A length field seems to be the only thing that is really needed and local context will determine the contents).  But we have made these comments more than once and I've seen little acknowledgment that if the NSH data is basically context driven fixed fields and TLVs could be mixed. 

So in short I support of the drafts goals as a way to carry metadata but it is not ready for last call yet in my opinion. 

Thanks
Don   

> The editors of the NSH document have indicated that they have addressed
> all known comments and that there are no open issues with the current
> version of the document.
> 
> We'll also get a brief update from the editors at next week's meeting. If
> there are any remaining issues with the document, raising them before the
> meeting would be especially helpful.
> 
> For the chairs,
> Thomas
>

---

From: "Surendra Kumar (smkumar)" <[email-masked]>
Date: Wed, 13 Apr 2016 19:11:11 +0000

From: sfc [mailto:[email-masked]] On Behalf Of Alia Atlas
Sent: Wednesday, April 13, 2016 11:59 AM
To: Carlos Pignataro (cpignata) <[email-masked]>
Cc: Thomas Narten <[email-masked]>; [email-masked]; Jim Guichard (jguichar) <[email-masked]>; Martin Stiemerling <[email-masked]>; Xuxiaohu <[email-masked]>; Joel M. Halpern <[email-masked]>
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Carlos,

On Wed, Apr 13, 2016 at 9:59 AM, Carlos Pignataro (cpignata) <[email-masked]<mailto:[email-masked]>> wrote:
Alia,

Sure - but there are link layers that IPv6 does work over already ;-)
I do feel that there is a need for at least a WG-agreed  approach to the issue.
I am thinking in general about how to handle the transport-specific aspects when SFC is the WG with full context.  I'm fairly certain that explaining issues around congestion control for UDP, for instance, are quite interesting.

SK> We have tried addressing it with the NSH UDP draft(https://datatracker.ietf.org/doc/draft-kumar-sfc-nsh-udp-transport/). It is pending an update to reflect additional comments from TSV.
Although SFC is not chartered to talk about transports, ability to carry NSH over the most common transports of today and any issues thereof need to be addressed, somewhere.

Yes and I had hoped it would facilitate even better coordination on this aspect of things.  Lessons learned.

Regards,
alia

Thanks,

— Carlos.

On Apr 13, 2016, at 9:13 AM, Alia Atlas <[email-masked]<mailto:[email-masked]>> wrote:

Carlos,

This issue is not new - and the issues I saw in a quick look are those that are thoroughly documented
in draft-ietf-rtgwg-dt-encap-01<https://datatracker.ietf.org/doc/draft-ietf-rtgwg-dt-encap/> which considered NSH as one of the encapsulations.

Regards,
Alia

On Wed, Apr 13, 2016 at 8:53 AM, Carlos Pignataro (cpignata) <[email-masked]<mailto:[email-masked]>> wrote:
Alia,

Thanks!

— Carlos.

On Apr 12, 2016, at 10:29 PM, Alia Atlas <[email-masked]<mailto:[email-masked]>> wrote:

While I am on the topic of details of the NSH encapsulation, I would request that Section 8 of draft-ietf-rtgwg-dt-encap-01
be read and considered!   I am not excited by having many different and unique Next Protocol fields defined.
For instance, I note the definition of a nearly identical Next Protocol field in https://datatracker.ietf.org/doc/draft-ietf-nvo3-vxlan-gpe .

Regards,
Alia

On Tue, Apr 12, 2016 at 9:53 PM, Xuxiaohu <[email-masked]<mailto:[email-masked]>> wrote:
Joel,

> From: Joel M. Halpern [mailto:[email-masked]<mailto:[email-masked]>]
> To: Xuxiaohu; Thomas Narten; [email-masked]<mailto:[email-masked]>

Best regards,
Xiaohu

> >> From: sfc [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Thomas Narten
> >> To: [email-masked]<mailto:[email-masked]>

---

From: Sumandra Majee <[email-masked]>
Date: Wed, 13 Apr 2016 19:11:16 +0000

Thomas,

I do support the adoption of this draft.

The overriding concern seems to be that for MD type 1 the semantics of
context header is not standardized, and my view is that it is GOOD that
this draft considers context info area to be a scratchpad.

A) I don¹t think neither DC allocation and mobility allocation of context
headers will satisfy all cases. Our implementation uses context header
scratchpad to put some different information.

B) Standardizing context header too early also has a higher chance of not
meeting the actual implementation need. This is a new technology that
doesn¹t have much milage.

C) YES interoperability is an issue. And I would like to see the
allocation draft(s) reserve 3/4 bits to specify the allocation type/schema
itself. I would like to see a context header registry for MD type 1
allocation itself. However that can be addressed separately in the
allocation draft(s).

Sumandra

---

From: Ron Parker <[email-masked]>
Date: Wed, 13 Apr 2016 19:43:56 +0000

Sumandra,

I don't object to the scratchpad nature of the MD-type-1 metadata.   But I have a small reservation and a large reservation around other aspects of it:

* small reservation -- why call this 4 context headers of 4 bytes each?   It would be much more intuitive to call it a 16 byte scratch pad.   The former implies structure that is present, at best, in an implementation-specific manner.

* large reservation -- since this is a scratch pad, it does not seem justifiable to make it mandatory.

My objections would be removed if it were a 16-byte scratchpad that was optional. 

Thanks.

   Ron

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Sumandra Majee
Sent: Wednesday, April 13, 2016 3:11 PM
To: Thomas Narten <[email-masked]>; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Thomas,

The overriding concern seems to be that for MD type 1 the semantics of context header is not standardized, and my view is that it is GOOD that this draft considers context info area to be a scratchpad.

A) I don¹t think neither DC allocation and mobility allocation of context headers will satisfy all cases. Our implementation uses context header scratchpad to put some different information.

B) Standardizing context header too early also has a higher chance of not meeting the actual implementation need. This is a new technology that doesn¹t have much milage.

C) YES interoperability is an issue. And I would like to see the allocation draft(s) reserve 3/4 bits to specify the allocation type/schema itself. I would like to see a context header registry for MD type 1 allocation itself. However that can be addressed separately in the allocation draft(s).

Sumandra

On 3/30/16, 7:48 PM, "sfc on behalf of Thomas Narten"
<[email-masked] on behalf of [email-masked]> wrote:

>We'll also get a brief update from the editors at next week's meeting. 
>If there are any remaining issues with the document, raising them 
>before the meeting would be especially helpful.
>
>For the chairs,
>Thomas
>

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 13 Apr 2016 15:58:38 -0400

I am having trouble following your argument about forwarding in this email.

Yes, transports can carry forwarding information.  The SFC archtiecture 
allows it.  that does not make including service paht identification in 
ht NSH header either redundant or incorrect.

I do not know what you mean by needing more examples of the forwarding. 
  The draft itself describes how the NSH header SFP ID and index can be 
used for the SFF forwarding decision.  That description seems quite 
clear to me.  If there is ambiguity or unclarity, please let the WG 
know, so we can improve it.

Yours,
Joel

---

From: "Andrew G. Malis" <[email-masked]>
Date: Wed, 13 Apr 2016 16:02:39 -0400

Carlos,

We certainly could define a new NSH PW type, and require that when NSH is
transported over MPLS, then it’s in a PW. That would just require the
consensus of the SFC WG on that approach, a draft for the PALS WG to define
the PW, and consensus there, of course. Of course, this approach would
require the use of LDP-based signaling, as always, to dynamically establish
the PW, or static provisioning of the PW label.

However, the SFC group is also free to rearrange the bits in the header
such that the first niblble is always 0x0 or 0x1, to prevent current HW
from incorrectly reordering SFC packets when carried over MPLS. I disagree
with you and Joel that the WG isn’t chartered to discuss how to carry NSH
over various encapsulations, otherwise we wouldn’t have drafts (already
existing) for how to carry the NSH header over Ethernet and UDP. Also,
optimizing the header to work well over a particular transport to account
for current implementations in the field doesn’t make the header
non-transport-dependent, since it wouldn’t prevent the header from being
carried over any other transport as well.

Cheers,
Andy

On Wed, Apr 13, 2016 at 2:27 PM, Carlos Pignataro (cpignata) <
[email-masked]> wrote:

> format to slightly simplify carrying one transport. The difficulty is that
> the WG is not scoped to evaluate the various ways NSH might be carried on
> MPLS, much less to standardize and pick one.
>
>
> I’ll add to this, without "taking sides", that it is a decision tradeoff
> between a detriment to all-but-one transports (an unused nibble) to cater
> to the specifics of one, with modulo to implementation implications, versus
> optimizing for one transport. But as Joel said, not SFCs scope, charter;
> perhaps expertise ought to be drawn from MPLS.
>
> Yours,
> Joel
>
> If the requirement is to encapsulate NSH to transport it over MPLS
> networks, in a point-to-point fashion, in a way that prevents ECMP
> load-balancing, the IETF knows how to do that: It’s called a Pseudowire.
>
> There are other alternatives such as specifying an “NSH Explicit Label”
> (or reusing GAL to indicate a header follows), followed by a CW/ACH with
> the first nibble as 0x0.
>
> These ought to be weighted against the alternative of modifying the NSH
> encap (plus, really, if we change NSH to start with 0x0, it would not be
> transport independent anymore :-)
>

---

From: "Fedyk, Don" <[email-masked]>
Date: Wed, 13 Apr 2016 20:05:26 +0000

Hi Joel

The argument has been put forward that NSH is only carrying identifiers.  When I look at NSH service forwarding that I know I disagree with that statement.  I think that NSH SID + SI is an address space and hence is not completely independent of forwarding.  The only way I can reason it out is to see other examples of complete packet headers and operations per hop.   There has been a lot of hand waving.

Cheers
Don 

> Sent: Wednesday, April 13, 2016 3:59 PM
> To: Fedyk, Don <[email-masked]>; Thomas Narten
> <[email-masked]>; [email-masked]
> Yes, transports can carry forwarding information.  The SFC archtiecture allows
> it.  that does not make including service paht identification in ht NSH header
> either redundant or incorrect.
> 
>   The draft itself describes how the NSH header SFP ID and index can be used
> for the SFF forwarding decision.  That description seems quite clear to me.  If
> there is ambiguity or unclarity, please let the WG know, so we can improve it.
> 
> Yours,
> Joel
> 
> > At this point I do not support a last call on the NSH header. There is a lot of
> comments and I don't think it is at a point where it is last call ready.
> >
> > I have be banging my head against the wall on this one because I would like
> more concrete around the forwarding aspects.  I do accept that there is a
> draft (draft-kumar-sfc-nsh-forwarding-00 ) now that addresses some the
> confusion in NSH on forwarding and it helps.  Although it seems odd to let a
> NSH draft progress as confusing and to clarify it later in another draft.
> >
> > My main objection is related to the fact that for a draft that claims it is
> independent of forwarding it simply has not been proven.  That does not
> mean we need to specify forwarding but I'd still like to see the data formats
> of what are being "prototyped" today so I can compare with the other data
> formats that I know today.  For example the MAC chaining draft we authored
> would like to use NSH for metadata but the MAC chaining local MACs bear a
> relationship with the NSH SDI & SI.  In fact without these fields as mandatory
> it would be more independent in my case.  I'm sure there are similar aspects
> with other transports types but until I see them and the operations I can't
> assess transport independence claims.
> >
> > There are other inconsistencies that I could live with in the draft relating to
> the logic of the MD types etc. (A length field seems to be the only thing that
> is really needed and local context will determine the contents).  But we have
> made these comments more than once and I've seen little acknowledgment
> that if the NSH data is basically context driven fixed fields and TLVs could be
> mixed.
> >
> > So in short I support of the drafts goals as a way to carry metadata but it is
> not ready for last call yet in my opinion.
> >
> > Thanks
> > Don
> >

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 13 Apr 2016 16:09:17 -0400

Given that the draft describes how to use the SFP ID and Index for
forwarding, it would seem very strange to argue that it does not
describe a forwarding behavior, or that those fields are not for use by
forwarding.  It is not REQUIRED that they be used for forwarding at an
SFF, but it is certainly permitted.

Is there wording in the draft that says that these are non-forwarding 
identifiers?  If so, we really should fix that.

Yours,
Joel

On 4/13/16 4:05 PM, Fedyk, Don wrote:
> Hi Joel
>
> The argument has been put forward that NSH is only carrying
> identifiers.  When I look at NSH service forwarding that I know I
> disagree with that statement.  I think that NSH SID + SI is an
> address space and hence is not completely independent of forwarding.
> The only way I can reason it out is to see other examples of complete
> packet headers and operations per hop.   There has been a lot of hand
> waving.
>
> Cheers Don
>
>> -----Original Message----- From: Joel M. Halpern
>> [mailto:[email-masked]] Sent: Wednesday, April 13, 2016 3:59
>> PM To: Fedyk, Don <[email-masked]>; Thomas Narten
>> <[email-masked]>; [email-masked] Subject: Re: [sfc] WG last call
>> for draft-ietf-sfc-nsh-04.txt
>>
>> Yes, transports can carry forwarding information.  The SFC
>> archtiecture allows it.  that does not make including service paht
>> identification in ht NSH header either redundant or incorrect.
>>
>> I do not know what you mean by needing more examples of the
>> forwarding. The draft itself describes how the NSH header SFP ID
>> and index can be used for the SFF forwarding decision.  That
>> description seems quite clear to me.  If there is ambiguity or
>> unclarity, please let the WG know, so we can improve it.
>>
>> Yours, Joel
>>
>> more concrete around the forwarding aspects.  I do accept that
>> there is a draft (draft-kumar-sfc-nsh-forwarding-00 ) now that
>> addresses some the confusion in NSH on forwarding and it helps.
>> Although it seems odd to let a NSH draft progress as confusing and
>> to clarify it later in another draft.
>>>
>> independent of forwarding it simply has not been proven.  That does
>> not mean we need to specify forwarding but I'd still like to see
>> the data formats of what are being "prototyped" today so I can
>> compare with the other data formats that I know today.  For example
>> the MAC chaining draft we authored would like to use NSH for
>> metadata but the MAC chaining local MACs bear a relationship with
>> the NSH SDI & SI.  In fact without these fields as mandatory it
>> would be more independent in my case.  I'm sure there are similar
>> aspects with other transports types but until I see them and the
>> operations I can't assess transport independence claims.
>>>
>> the logic of the MD types etc. (A length field seems to be the only
>> thing that is really needed and local context will determine the
>> contents).  But we have made these comments more than once and I've
>> seen little acknowledgment that if the NSH data is basically
>> context driven fixed fields and TLVs could be mixed.
>>>
>>> Thanks Don
>>>> -----Original Message----- From: sfc
>>>> [mailto:[email-masked]] On Behalf Of Thomas Narten Sent:
>>>> Wednesday, March 30, 2016 10:48 PM To: [email-masked] Subject:
>>>> [sfc] WG last call for draft-ietf-sfc-nsh-04.txt
>>>>
>>>> Dear WG:
>>>>
>>>> addressed all known comments and that there are no open issues
>>>> with the current version of the document.
>>>>
>>>> For the chairs, Thomas
>>>> _______________________________________________ sfc mailing
>>>> list [email-masked] https://www.ietf.org/mailman/listinfo/sfc
>>>
>>> _______________________________________________ sfc mailing list
>>> [email-masked] https://www.ietf.org/mailman/listinfo/sfc
>>>

---

From: Joel Halpern <[email-masked]>
Date: Wed, 13 Apr 2016 16:10:44 -0400

[Resend, reducing CCs]

Andy, I have never seen a working group charter prevent the submission 
of individual drafts on a topic.  So the presence of drafts on UDP, 
Ethernet MAC Chaining, or any other transports says nothing about what 
is in our out of scope for the WG.

I have also seen WGs adopt drafts that are out of scope, but that is a 
more complicated situation, and usually indicates an expectation to 
expand the scope.

I have actually expressed concerns on the list about both the UDP draft 
and the Ethernet draft because I can not see a goo dpath forward for 
handling the material.

Changing the NSH layout is clearly within scope for the working group.

Yours,
Joel

On 4/13/16 4:02 PM, Andrew G. Malis wrote:
> Carlos,
>
> We certainly could define a new NSH PW type, and require that when NSH
> is transported over MPLS, then it’s in a PW. That would just require the
> consensus of the SFC WG on that approach, a draft for the PALS WG to
> define the PW, and consensus there, of course. Of course, this approach
> would require the use of LDP-based signaling, as always, to dynamically
> establish the PW, or static provisioning of the PW label.
>
> from incorrectly reordering SFC packets when carried over MPLS. I
> disagree with you and Joel that the WG isn’t chartered to discuss how to
> carry NSH over various encapsulations, otherwise we wouldn’t have drafts
> (already existing) for how to carry the NSH header over Ethernet and
> UDP. Also, optimizing the header to work well over a particular
> transport to account for current implementations in the field doesn’t
> make the header non-transport-dependent, since it wouldn’t prevent the
> header from being carried over any other transport as well.
>
> Cheers,
> Andy
>
> On Wed, Apr 13, 2016 at 2:27 PM, Carlos Pignataro (cpignata)
>>     On Apr 13, 2016, at 10:17 AM, Joel M. Halpern <[email-masked]
>>     <mailto:[email-masked]>> wrote:
>>
>>     Yes, Alia, there are many transports out there being used for
>>     service chaining.  Most of the folks using these farious
>>     transports expect to support NSH.  Several of these have been
>>     brought forward  as Internet-Drafts.  Not all.  Which makes sense
>>     as standardizing transport behavior for NSH is out of scope for
>>     the working group.
>>
>>     It does seem that the WG has to decide if they want to adjust the
>>     header format to slightly simplify carrying one transport. The
>>     difficulty is that the WG is not scoped to evaluate the various
>>     ways NSH might be carried on MPLS, much less to standardize and
>>     pick one.
>>
>
>     I’ll add to this, without "taking sides", that it is a decision
>     tradeoff between a detriment to all-but-one transports (an unused
>     nibble) to cater to the specifics of one, with modulo to
>     implementation implications, versus optimizing for one transport.
>     But as Joel said, not SFCs scope, charter; perhaps expertise ought
>     to be drawn from MPLS.
>
>>     Yours,
>>     Joel
>>
>     There are other alternatives such as specifying an “NSH Explicit
>     Label” (or reusing GAL to indicate a header follows), followed by a
>     CW/ACH with the first nibble as 0x0.
>
>     These ought to be weighted against the alternative of modifying the
>     NSH encap (plus, really, if we change NSH to start with 0x0, it
>     would not be transport independent anymore :-)
>

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Wed, 13 Apr 2016 20:35:43 +0000

Hi, Andy,

Clearly, rearranging the bits of the NSH, such that the first nibble is always 0x0 (not 0x1, remember that’s PW-ACH) does not define an MPLS encapsulation for NSH.

The whole set of things you describe that we could do for an NSH PW, including encapsulation definition, signaling (LDP, static, or controller-based config, BGP, other), Internet-Drafts, consensus seeking, and others that you did not include like TTL settings, congestion considerations, security considerations, etc, would need to be specified in either case, of course. It’s not like SFC can say “first nibble is 0x0” and the MPLS encap is done — the ToDo list is even on either side.

But all this discussion seems tangential and a red herring. Since NSH can run over MPLS, but that’s not to be defined by SFC.

To clarify one additional point, the WG does not have drafts on various encapsulations, contrary to what you state or imply below. There are individual submissions.

Sorry I was not clear when I wrote that "setting the first nibble to 0x0 would make it transport-dependent". I did not intend to imply that the header would not run over various lower layers. That’s of course still possible. What I meant is that the header would have 4 bits dictated specifically in a dependent way based on one encapsulation, and all other encapsulations (that do not care about the first nibble) would inherit that dependency, dependent from an MPLS transport.

Thank you,

— Carlos.

> On Apr 13, 2016, at 4:02 PM, Andrew G. Malis <[email-masked]> wrote:
> 
> Carlos,
> 
> We certainly could define a new NSH PW type, and require that when NSH is transported over MPLS, then it’s in a PW. That would just require the consensus of the SFC WG on that approach, a draft for the PALS WG to define the PW, and consensus there, of course. Of course, this approach would require the use of LDP-based signaling, as always, to dynamically establish the PW, or static provisioning of the PW label.
> 
> However, the SFC group is also free to rearrange the bits in the header such that the first niblble is always 0x0 or 0x1, to prevent current HW from incorrectly reordering SFC packets when carried over MPLS. I disagree with you and Joel that the WG isn’t chartered to discuss how to carry NSH over various encapsulations, otherwise we wouldn’t have drafts (already existing) for how to carry the NSH header over Ethernet and UDP. Also, optimizing the header to work well over a particular transport to account for current implementations in the field doesn’t make the header non-transport-dependent, since it wouldn’t prevent the header from being carried over any other transport as well.
> 
> Cheers,
> Andy
> 
> On Wed, Apr 13, 2016 at 2:27 PM, Carlos Pignataro (cpignata) <[email-masked] <mailto:[email-masked]>> wrote:
> 
>> On Apr 13, 2016, at 10:17 AM, Joel M. Halpern <[email-masked] <mailto:[email-masked]>> wrote:
>>

---

From: "Fedyk, Don" <[email-masked]>
Date: Wed, 13 Apr 2016 20:39:45 +0000

Joel 

I think that https://tools.ietf.org/html/draft-kumar-sfc-nsh-forwarding-00 clarifies Service Forwarding (along the chain) versus Packet forwarding (elsewhere).    But this draft points out that the NSH draft is "confusing" with respect to some parts of service forwarding.   And when I and others have asked on the list about subtleties of Service forwarding  behaviors and the relationship of chain ID with various packet formats;  the answers were NSH SID and SI were just identifiers.  So while there is not a statement they are non-forwarding identifiers the confusion still exists because the behavior is not clear.  
I still think NSH SID and SI  behave as an address space - receive a packet; lookup the address; if recognized process; then place the next hop address (includes not modifying the address in some cases) on the packet and forward on.  If the fields don't match expected values;  drop or exception process. 

Cheers
Don 

> Sent: Wednesday, April 13, 2016 4:09 PM
> forwarding, it would seem very strange to argue that it does not describe a
> forwarding behavior, or that those fields are not for use by forwarding.  It is
> not REQUIRED that they be used for forwarding at an SFF, but it is certainly
> permitted.
> 
> > disagree with that statement.  I think that NSH SID + SI is an address
> > space and hence is not completely independent of forwarding.
> >> [mailto:[email-masked]] Sent: Wednesday, April 13, 2016 3:59 PM
> >> <[email-masked]>; [email-masked] Subject: Re: [sfc] WG last call for
> >> draft-ietf-sfc-nsh-04.txt
> >>
> >> forwarding. The draft itself describes how the NSH header SFP ID and
> >> index can be used for the SFF forwarding decision.  That description
> >> seems quite clear to me.  If there is ambiguity or unclarity, please
> >> let the WG know, so we can improve it.
> >>
> >> Yours, Joel
> >>
> >> more concrete around the forwarding aspects.  I do accept that there
> >> is a draft (draft-kumar-sfc-nsh-forwarding-00 ) now that addresses
> >> some the confusion in NSH on forwarding and it helps.
> >> Although it seems odd to let a NSH draft progress as confusing and to
> >> clarify it later in another draft.
> >>>
> >> not mean we need to specify forwarding but I'd still like to see the
> >> data formats of what are being "prototyped" today so I can compare
> >> with the other data formats that I know today.  For example the MAC
> >> chaining draft we authored would like to use NSH for metadata but the
> >> MAC chaining local MACs bear a relationship with the NSH SDI & SI.
> >> In fact without these fields as mandatory it would be more
> >> independent in my case.  I'm sure there are similar aspects with
> >> other transports types but until I see them and the operations I
> >> can't assess transport independence claims.
> >>>
> >> seen little acknowledgment that if the NSH data is basically context
> >> driven fixed fields and TLVs could be mixed.
> >>>
> >>>> -----Original Message----- From: sfc [mailto:[email-masked]]
> >>>> On Behalf Of Thomas Narten Sent:
> list

---

From: "Paul Quinn (paulq)" <[email-masked]>
Date: Wed, 13 Apr 2016 20:41:09 +0000

Don,

Considered through the lens of a mac forwarding person I can see the disconnect.  NSH forwarding occurs at the service plane, not the network plane and as such the SPI/SI acts an identifier for transport selection.  This de-couples NSH service path from the network.

So, in the service plane NSH SPI/SI indicate information about the path and location within that path, but no actual movement (i.e. forwarding) can occur without the actual transport imposition.  This transport provides all the normal transport characteristics you seek (dependent on the transport selected).

Paul

> On Apr 13, 2016, at 4:05 PM, Fedyk, Don <[email-masked]> wrote:
> 
> Hi Joel
>

---

From: "Fedyk, Don" <[email-masked]>
Date: Wed, 13 Apr 2016 21:00:35 +0000

Hi Paul

The same statements are true when you put the ChainID in local MACs and use them in the Service Plane.  No actual forwarding has to happen you can just change the IDs and message pass within a compute element. I'd say that is an API call to me.  Packet formats are what is on the wire or the virtual wire. It so happens the MAC format can work for both.  But I believe that MPLS is similar in this regard. 

Cheers
Don  

> Sent: Wednesday, April 13, 2016 4:41 PM
> To: Fedyk, Don <[email-masked]>
> Cc: Joel M. Halpern <[email-masked]>; Thomas Narten
> Considered through the lens of a mac forwarding person I can see the
> disconnect.  NSH forwarding occurs at the service plane, not the network
> plane and as such the SPI/SI acts an identifier for transport selection.  This
> de-couples NSH service path from the network.
> 
> So, in the service plane NSH SPI/SI indicate information about the path and
> location within that path, but no actual movement (i.e. forwarding) can occur
> without the actual transport imposition.  This transport provides all the
> normal transport characteristics you seek (dependent on the transport
> selected).
> 
> Paul
> 
> > The argument has been put forward that NSH is only carrying identifiers.
> When I look at NSH service forwarding that I know I disagree with that
> statement.  I think that NSH SID + SI is an address space and hence is not
> completely independent of forwarding.  The only way I can reason it out is to
> see other examples of complete packet headers and operations per hop.
> There has been a lot of hand waving.
> >
> > Cheers
> > Don
> >
> forwarding.
> >>  The draft itself describes how the NSH header SFP ID and index can
> >> be used for the SFF forwarding decision.  That description seems
> >> quite clear to me.  If there is ambiguity or unclarity, please let the WG
> >> some the confusion in NSH on forwarding and it helps.  Although it
> >> seems odd to let a NSH draft progress as confusing and to clarify it later in
> another draft.
> >>>
> >> other transports types but until I see them and the operations I can't

---

From: "Bottorff, Paul" <[email-masked]>
Date: Wed, 13 Apr 2016 22:05:33 +0000

Hi Thomas:

I do not support last call on draft-eitf-sfc-nsh-04. The risk of rushing to close a format before we have clear agreement on the forwarding principles and features is just too high.

The major problems with the current draft are: 
1)the service address formed by the combined SPI+SI appears to have a number of deficiencies which lead to transport dependencies and to layer violations,
2)there is no resolution to forward/reverse addressing necessary for support of many L4-high service functions,
3)there is no resolution to encoding hierarchies which are of importance to SFC applications in NFV environments,
4)the draft needs a transport service interface and a description of the mappings used for the service layer onto the transport layer,
5)the draft specifies two independent formats for encoding the NSH header which is likely to divide the market slowing the deployment of NSH.

>From my point of view the SFC formed service plane layer network with a peer dialog between the SCF, SFF and SF.  The Service Path Header (SPI+SI) forms the address for this service plane. As it stands there is an attempt to overload the function of the SI as both the SF identifier with the Service Path and as a form of loop detection, however the coupling has resulted in two problems: 
1)Since every SF in a path must have a sequential SI, in the event we need to insert or delete a SF it is necessary to update the SFF forwarding entries for the entire chain. To fix this either the SI should be allowed to take non-sequential values along the chain or should not be used for SF addressing.
2)Since the SI is only decremented on SF hops it can't guard against a loop between SFFs or SCFs, hence SI is not a reliable method for loop squelching. If a TTL is needed it needs to be separated from SI. Given that there is no clear alternative to TTL for loop squelching at the service layer I believe we do need to include an effective TTL to go forward.
3)Every other successful address space has needed to add a few bits to support QoS options. I would suggest taking 3-4 bits from the MD-type and to reserve for QoS handling.
4)In may data centers we have multi-vendor deployments. In these environments we may have different managers assigning chains. There should to be some convention for splitting the address space between these vendors. The most desirable solution to this is to add an assigned organization ID to the address space. An alternative would be to separate the SPI into a 4 and 20 bit field designating the high 4 bits as a local manager ID.
  
There has been periodic questions about the use of a bit to designate the forward/reverse direction. This is an important issue for support of L4-higher SF which typically generate frames in addition to forwarding frames. I have proposed using the transport source address as part of the forwarding to eliminate the need for a bit, however if the committee wants to designate a specific indication for forward/reverse then it should be fully documented in the draft.

The discussion on hierarchies has proposed stacking NSH headers. Since a hierarchy probably will require some type of indication in the NSH header we should have settled a solution before closing on the NSH header.

The way to make the draft transport independent is to specify an abstract service interface between the service layer and the transport layer. The current example transport mappings do not make clear what service the transport must provide to support the service plane. Looking at the current examples I think a transport service interface should include: the transport type, the source transport address, the destination transport address, the transport QoS.

Having two formats which accomplish essentially the same thing will divide the market. It also will slow implementations since vendors will either have to take a chance and pick one or implement both. Though all the technical problems of two formats can be solved failing to choose a format is not a service to the industry.

Sincerely,

Paul

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Thomas Narten
Sent: Wednesday, March 30, 2016 7:48 PM
To: [email-masked]
Subject: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Dear WG:

For the chairs,
Thomas

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 13 Apr 2016 18:09:34 -0400

I am having trouble following you.  Sorry.  the kumar forwarding draft 
describes some ways one can implement things.  It is not define 
anything.  When I last looked, they appeared to be reasonable ways to 
implement things, but not the only ones.

So the question I have is what wording in the NSH draft causes you 
concern.  Is there wording in the NSH draft that gives the impression 
that the SPI / SI is only an "identifier"?  Is there wording that we 
could add that would help?

Thanks,
Joel

On 4/13/16 4:39 PM, Fedyk, Don wrote:
> Joel
>
> I think that
> https://tools.ietf.org/html/draft-kumar-sfc-nsh-forwarding-00
> clarifies Service Forwarding (along the chain) versus Packet
> forwarding (elsewhere).    But this draft points out that the NSH
> draft is "confusing" with respect to some parts of service
> forwarding.   And when I and others have asked on the list about
> subtleties of Service forwarding  behaviors and the relationship of
> chain ID with various packet formats;  the answers were NSH SID and
> SI were just identifiers.  So while there is not a statement they are
> non-forwarding identifiers the confusion still exists because the
> behavior is not clear. I still think NSH SID and SI  behave as an
> address space - receive a packet; lookup the address; if recognized
> process; then place the next hop address (includes not modifying the
> address in some cases) on the packet and forward on.  If the fields
> don't match expected values;  drop or exception process.
>
> Cheers Don
>
>
>> [mailto:[email-masked]] Sent: Wednesday, April 13, 2016 4:09
>> describe a forwarding behavior, or that those fields are not for
>> use by forwarding.  It is not REQUIRED that they be used for
>> forwarding at an SFF, but it is certainly permitted.
>>
>> Is there wording in the draft that says that these are
>> non-forwarding identifiers?  If so, we really should fix that.
>>
>> Yours, Joel
>>
>>> identifiers.  When I look at NSH service forwarding that I know
>>> I disagree with that statement.  I think that NSH SID + SI is an
>>> address space and hence is not completely independent of
>>> forwarding. The only way I can reason it out is to see other
>>> examples of complete packet headers and operations per hop.
>>>> [mailto:[email-masked]] Sent: Wednesday, April 13, 2016
>>>> 3:59 PM To: Fedyk, Don <[email-masked]>; Thomas Narten
>>>> <[email-masked]>; [email-masked] Subject: Re: [sfc] WG last
>>>> call for draft-ietf-sfc-nsh-04.txt
>>>>
>>>> archtiecture allows it.  that does not make including service
>>>> paht identification in ht NSH header either redundant or
>>>> incorrect.
>>>>
>>>> forwarding. The draft itself describes how the NSH header SFP
>>>> ID and index can be used for the SFF forwarding decision.  That
>>>> addresses some the confusion in NSH on forwarding and it
>>>> helps. Although it seems odd to let a NSH draft progress as
>>>> confusing and to clarify it later in another draft.
>>>>>
>>>> independent of forwarding it simply has not been proven.  That
>>>> does not mean we need to specify forwarding but I'd still like
>>>> to see the data formats of what are being "prototyped" today so
>>>> I can compare with the other data formats that I know today.
>>>> For example the MAC chaining draft we authored would like to
>>>> use NSH for metadata but the MAC chaining local MACs bear a
>>>> relationship with the NSH SDI & SI. In fact without these
>>>> fields as mandatory it would be more independent in my case.
>>>> I'm sure there are similar aspects with other transports types
>>>> but until I see them and the operations I can't assess
>>>> transport independence claims.
>>>>>
>>>> the logic of the MD types etc. (A length field seems to be the
>>>> only thing that is really needed and local context will
>>>> determine the contents).  But we have made these comments more
>>>> than once and I've seen little acknowledgment that if the NSH
>>>> data is basically context driven fixed fields and TLVs could be
>>>> mixed.
>>>>>
>>>>>> [mailto:[email-masked]] On Behalf Of Thomas Narten
>>>>>> Sent: Wednesday, March 30, 2016 10:48 PM To: [email-masked]
>>>>>> The editors of the NSH document have indicated that they
>>>>>> have addressed all known comments and that there are no
>>>>>> open issues with the current version of the document.
>>>>>>
>>>>>> We'll also get a brief update from the editors at next
>>>>>> week's meeting. If there are any remaining issues with the
>>>>>> document, raising them before the meeting would be
>>>>>> especially helpful.
>>>>>>
>>>>>> For the chairs, Thomas
>>>>>>

---

From: Sumandra Majee <[email-masked]>
Date: Wed, 13 Apr 2016 22:52:56 +0000

Inline

On 4/13/16, 12:43 PM, "Ron Parker" <[email-masked]> wrote:

>Sumandra,
>
>I don't object to the scratchpad nature of the MD-type-1 metadata.   But
>I have a small reservation and a large reservation around other aspects
>of it:
>
>* small reservation -- why call this 4 context headers of 4 bytes each?
>It would be much more intuitive to call it a 16 byte scratch pad.   The
>former implies structure that is present, at best, in an
>implementation-specific manner.
[SM] If you are suggesting that it be called as 16 byte context
information, yes sure I don’t have any objection.
>justifiable to make it mandatory.
[SM]  To me MD=1 means LEN = 24B and there is 16B context header that has
some relevant information for one or more SF in the chain. Many flow based
engines like LB, Firewall, various proxy based optimizers etc. do save
states. In that case if we want to carry only the forwarding part of NSH
then why not use MD type=2 with LEN=0.
>allocation draft(s) reserve 3/4 bits to specify the allocation
>type/schema itself. I would like to see a context header registry for MD
>type 1 allocation itself. However that can be addressed separately in the
>allocation draft(s).
>
>Sumandra
>
>

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 13 Apr 2016 16:07:10 -0400

Yours,
Joel

---

From: "Ken Gray (kegray)" <[email-masked]>
Date: Wed, 13 Apr 2016 23:22:16 +0000

At this point, I'll weigh in with two things:

Yes, I support the draft.

Aside from the editorial suggestions (welcome, fix that stuff) and the
constant returning to wanting to define metadata - which is a deep abyss
we seem fascinated with but doesn't HAVE to be done here and now, IMO - I
have to express both my appreciation for the thoroughness that brings the
MPLS nibble issue to the surface and my absolute awe that we would
perpetuate that design choice and cripple the NSH header rather than
attempt to fix it at it's root.

>Thanks for asking and don't be sorry for not having to live through the
>fun and joy of standardizing pseudo-wires.   The very very short form is
>that there is a 32-bit code-word defined that sits above the Ethernet
>frame and right below the MPLS stack.
> This is to deal with implementations that look beneath the MPLS stack to
>hunt down more fields to hash to get good entropy for ECMP or LAG.
>
>
>
>time ago, the likeliness of actual collision with allocated OUIs was
>quite small.
>
>
>Alia
> 
>
> 
> 
> 
>Cc: Thomas Narten; Carlos Pignataro (cpignata);
>[email-masked] <mailto:[email-masked]>; Jim Guichard (jguichar); Martin
>Stiemerling; Xuxiaohu; Joel M. Halpern
>
>https://datatracker.ietf.org/doc/draft-ietf-nvo3-vxlan-gpe
><https://datatracker.ietf.org/doc/draft-ietf-nvo3-vxlan-gpe> .
>
><mailto:[email-masked]>]
>        > To: Xuxiaohu; Thomas
>Narten;[email-masked] <mailto:[email-masked]> <mailto:[email-masked]>
>        > >> To: 
>[email-masked] <mailto:[email-masked]> <mailto:[email-masked]>
><https://www.ietf.org/mailman/listinfo/sfc>
>        > >

---

From: "Nagendra Kumar Nainar (naikumar)" <[email-masked]>
Date: Wed, 13 Apr 2016 18:25:32 +0000

Just to add to what Alia mentioned.

My understanding is  - the control word starts with 0000b. So even if it collides with existing OUI, it does not create any issue, as the intention is to avoid treating MAC address starting with 4 or 6 as first nibble as IPv4 or IPv6 packet.

Thanks,
Nagendra

From: sfc <[email-masked]<mailto:[email-masked]>> on behalf of Alia Atlas <[email-masked]<mailto:[email-masked]>>
Date: Wednesday, April 13, 2016 at 2:12 PM
To: Dave Dolson <[email-masked]<mailto:[email-masked]>>
Cc: Thomas Narten <[email-masked]<mailto:[email-masked]>>, "Carlos Pignataro (cpignata)" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "Jim Guichard (jguichar)" <[email-masked]<mailto:[email-masked]>>, Martin Stiemerling <[email-masked]<mailto:[email-masked]>>, Xuxiaohu <[email-masked]<mailto:[email-masked]>>, "Joel M. Halpern" <[email-masked]<mailto:[email-masked]>>, Loa Andersson <[email-masked]<mailto:[email-masked]>>
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Hi Dave,

On Wed, Apr 13, 2016 at 1:34 PM, Dave Dolson <[email-masked]<mailto:[email-masked]>> wrote:
I’m going to show my ignorance and ask how Ethernet is carried over MPLS? The first nybble is part of the Destination MAC address.
OUIs have been allocated beginning with 4 and also with 6.

Thanks for asking and don't be sorry for not having to live through the fun and joy of standardizing pseudo-wires.   The very very short form is that there is a 32-bit code-word defined that sits above the Ethernet frame and right below the MPLS stack.  This is to deal with implementations that look beneath the MPLS stack to hunt down more fields to hash to get good entropy for ECMP or LAG.

As I recall, when I looked a long long (er, let's not get into how long) time ago, the likeliness of actual collision with allocated OUIs was quite small.

Alia

From: sfc [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Alia Atlas
Sent: Wednesday, April 13, 2016 9:56 AM
To: Loa Andersson
Cc: Thomas Narten; Carlos Pignataro (cpignata); [email-masked]<mailto:[email-masked]>; Jim Guichard (jguichar); Martin Stiemerling; Xuxiaohu; Joel M. Halpern

Loa,

Regards,
Alia

/Loa

Regards,
Alia

    Alia,

    Thanks!
    â€” Carlos.

    Regards,
    Alia

        Joel,

        Best regards,
        Xiaohu

        an existing overlay
        connectivity, that

---

From: Alia Atlas <[email-masked]>
Date: Wed, 13 Apr 2016 21:29:03 -0400

Resending with fewer recipients.

On Wed, Apr 13, 2016 at 9:27 PM, Alia Atlas <[email-masked]> wrote:

> Hi Ken,
>
> On Wed, Apr 13, 2016 at 7:22 PM, Ken Gray (kegray) <[email-masked]>
> wrote:
>
> Deployed base, sadly - and a strong need for small microflows to support
> splitting
> traffic among different paths.
>
> Despite it being a good 10 years since the problem was admitted around
> pseudo-wires
> and then admitted again in conversations around MPLS-TP, I've heard no one
> claim
> that this isn't an issue for the MPLS transport.  Of course, my email is
> always open :-)
>
> I will have more comments.
>
> Cheers,
> Alia
>
>
>> >Hi Dave,
>> >
>> >
>> have
>> week's

---

From: Xuxiaohu <[email-masked]>
Date: Thu, 14 Apr 2016 03:41:40 +0000

> Sent: Wednesday, April 13, 2016 11:33 AM
> To: Alia Atlas; Xuxiaohu; Martin Stiemerling; [email-masked]
> Cc: Joel M. Halpern; Thomas Narten; [email-masked]
> With regard to MPLS encapsulation, I am not objecting to considering if we want
> to rearrange things to make it simpler to carry NSH in MPLS.
> 
> Given that the IETF has not ratified a standards track RFC stating that it is
> required for MPLS carriage to avoid the values, I was objecting the Xu
> It is not actually hard to avoid the values.  If we feel it is imnportant (and I am
> tempted to agree that it is worth considering) we can rearrange some reserved
> bits so that the first three bits are 0, the next two are the version, and the
> remaining bits are flags.  Makes us a little starved for flags, but if we agree that
> MPLS has stolen three bits of header, so be it.

It'd better to be determined by the IETF whether or not MPLS should continuously steal the first nibble of any emerging encapsulation header, IMHO.

Best regards,
Xiaohu

> carried over MPLS, a dummy 0 label is inserted after the bottom of stack.  No
> worse than many things MPLS has done.  (I could argue both sides of which
> answer is right, and not persuade myself of either one.)
> 
> note that even the section you point to concludes that there may well be good
> reasons for using a separate ID space.
> 
> Yours,
> Joel
> 
> > NSH header might be directly under an MPLS label stack.  This issue
> > arose rather unpleasantly for pseudo-wires a long time ago and the
> > At the time that PWE3 was working on the control word and whether it
> > was to be mandatory (RFC 4385), it was clear that there were devices
> > that looked at the first nibble of a packet under the MPLS header as a
> > way to determine if the packet was IPv4 or IPv6 and obtain
> > flow-diversity from it.  The cost of also verifying the associated
> > checksum if available wasn't deemed acceptable for several
> > implementations.
> > Given that determining as much entropy as possible is still really
> > important and that it is non-trivial to negotiate/indicate the
> > capability for including an Entropy Label in the MPLS stack, I have no
> > reason to believe that this shortcut of looking at the first nibble
> > base header might have any value of 0b0000, 0b0001, 0b0010, 0b0010 and
> > if a version 01 for NSH were ever defined, suddenly the traffic might
> > Given a claim of NSH being transport-agnostic and repeated emphasis
> > that MPLS, as well as UDP, is a valid transport for NSH, I do think
> > this is a significant oversight and needs, at a minimum, discussion
> > and explanation, and  - quite likely -  correction.
> >
> > Protocol field in https://datatracker.ietf.org/doc/draft-ietf-nvo3-vxlan-gpe .
> >
> > out explanations for why each and every new encapsulation needs its
> > very own IANA-defined Next Protocol field, this seems to me to be
> > highly likely to require consolidation so that the same Next Protocol
> > field can be used between the various encapsulations.
> >
> > do realize that there are multiple implementations and while details
> > of how the "Next Protocol" field are documented shouldn't have a
> > significant impact there, the discussion about the first nibble is
> > likely to.
> >
> > Regards,
> > Alia
> >
> >

---

From: <[email-masked]>
Date: Thu, 14 Apr 2016 05:41:41 +0000

Hi Ron, all,

Same position here.

Note, that making the 16-bute optional is just MD#2.

Cheers,
Med

> De : sfc [mailto:[email-masked]] De la part de Ron Parker
> Envoyé : mercredi 13 avril 2016 21:44
> À : Sumandra Majee; Thomas Narten; [email-masked]
> Objet : Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt
> 
> Sumandra,
> 
> I don't object to the scratchpad nature of the MD-type-1 metadata.   But I
> have a small reservation and a large reservation around other aspects of
> it:
> 
> former implies structure that is present, at best, in an implementation-
> specific manner.
>

---

From: <[email-masked]>
Date: Thu, 14 Apr 2016 07:42:55 +0000

R-,

In addition to other pending issues raised for this draft, I would like to raise this additional one about Section 6. 

==
6.  Fragmentation Considerations

   NSH and the associated transport header are "added" to the
   encapsulated packet/frame.  This additional information increases the
   size of the packet.  In order the ensure proper forwarding of NSH
   data, several options for handling fragmentation and re-assembly
   exist:

   1.  Jumbo Frames, when supported, enable the transport of NSH and
       associated transport packets without requiring fragmentation.

   3.  [RFC6830] describes two schemes for fragmentation and re-assembly
       in section 5.4.
==

* The text is weak for a Standard track document that is intended to solve the problem in https://tools.ietf.org/html/rfc7498#section-2.12. There should be a clear behavior to be followed by an implementation. Further, I would avoid the use of words such as "can".

* The text covers only fragmentation when it is induced by SFC operations, it does not discuss the treatment of a fragment when received in an SFC domain. IMHO, the draft should also specify the behavior of the Classifier with regards to fragments for the sake of proper SFC operation. Applying classification policies may require the full packet, not only a fragment. In particular, dedicated resources should dedicated for handling out of order fragments. Of course, it is out of scope of this document to describe how SFs handle fragments. 

* If an SFC header is prepended for all fragments, I'm not sure there is any particular issue at the SFF level...except if stripping/adding context TLVs depends on the full packet (not just fragment). It is warranted to consider a little bit this point before declaring there is no issue. 

* The text states "several options". This may be interpreted as if implementing one of them is sufficient...which is not true. The first two points contribute to minimize the fragmentation risk, but fragmentation may still be experienced (e.g., other shims are prepended by other nodes for some other purposes, nested nsh, etc.)

* The first two points have nothing to do with reassembly.

* The support of jumbo frames by a router/device does not mean that it can make use of it. Appropriate MTU configuration should be undertaken in a consistent manner within an SFC domain. The text should be updated to make it is about (consistent) MTU configuration. 

* BTW, shouldn't the text be reworded to recommended to increase the MTU of **all nodes** of an SFC-enabled domain by at least the length of SFC header + transport header?  

* Bullet 2, how PMTU discovery is actually used in this context? Do you assume that all SFC-aware nodes will issue such messages towards other SFC-aware node, arbitrary destination, else? 

* Bullet 2, I would drop "describes a technique for dynamically discovering the maximum transmission unit (MTU) of an arbitrary internet path".  

* Bullet 2, s/ the the/the.

* The reference to the LISP specification raises two concerns and one comment:

(1) I don't think it is a good approach that fragments induced by the network are passed to their ultimate destinations as such (stateless). IMO, reassembly should be done within the SFC domain (responsible for fragmentation) not passed away.
(2) Does the stateful mode require all SFC data plane elements maintain a full list of MTU for the any SFF/SF instance of the SFC domain? 
 
The current text suggests that [RFC6830] should be listed as normative reference (not an informative one). I would personally favor removing the reference to LISP (as it is an Experimental RFC). 

* The security section of the draft may be augmented with informational fragmentation-related pointers to: e.g., RFC1858 (Security Considerations for IP Fragment Filtering), RFC3128 (Protection Against a Variant of the Tiny Fragment Attack), and RFC 4963 (IPv4 Reassembly Errors at High Data Rates). 

Cheers,
Med

> De : sfc [mailto:[email-masked]] De la part de
> [email-masked]
> Envoyé : lundi 11 avril 2016 13:14
> À : Thomas Narten; [email-masked]
> As I mentioned during the meeting, there are several issues that are not
> covered in the last version of the draft. I already provided examples of
> the issues offline as requested by Martin.
> 
> Cheers,
> Med
>

---

From: Dave Dolson <[email-masked]>
Date: Thu, 14 Apr 2016 13:33:56 +0000

Paul and Uri, can you please comment as to whether you agree or disagree with this wording?

I feel that it allows the actual content to be opaque while giving guidance for
service functions that create or replay packets in the chain.

>From my point of view, this is how such devices will work anyhow.

-Dave

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Dave Dolson
Sent: Wednesday, April 13, 2016 9:13 AM
To: Paul Quinn (paulq); Ron Parker
Cc: Thomas Narten; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Paul,

I'll update it here:

-Dave

-----Original Message-----
From: Paul Quinn (paulq) [mailto:[email-masked]] 
Sent: Tuesday, April 12, 2016 9:03 PM
To: Ron Parker
Cc: Dave Dolson; Thomas Narten; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Ron, Dave,

Thoughts?

Thanks,
Paul

---

From: Xuxiaohu <[email-masked]>
Date: Thu, 14 Apr 2016 02:51:31 +0000

> From: Ken Gray (kegray) [mailto:[email-masked]]
> Sent: Thursday, April 14, 2016 7:22 AM
> To: Alia Atlas; Dave Dolson
> Cc: Thomas Narten; Carlos Pignataro (cpignata); [email-masked]; Jim Guichard
> (jguichar); Martin Stiemerling; Xuxiaohu; Joel M. Halpern; Loa Andersson
> Aside from the editorial suggestions (welcome, fix that stuff) and the constant
> returning to wanting to define metadata - which is a deep abyss we seem
> fascinated with but doesn't HAVE to be done here and now, IMO - I have to
> express both my appreciation for the thoroughness that brings the MPLS nibble
> issue to the surface and my absolute awe that we would perpetuate that design
> choice and cripple the NSH header rather than attempt to fix it at it's root.

Agree that the root of this issue is the lack of a protocol identifier field in the MPLS encapsulation header. That's the reason why I raised the first nibble issue associated with MPLS encapsulation (https://mailarchive.ietf.org/arch/msg/bier/QmBmGKn8A_Q5lZSFNcFNVHKvEjc) to BIER, SFC and MPLS mailing-lists. Should this issue be addressed by each new encapsulation header or fixed by the MPLS itself is worthwhile to be discussed widely.

Best regards,
Xiaohu

> > This is to deal with implementations that look beneath the MPLS stack
> >to hunt down more fields to hash to get good entropy for ECMP or LAG.
> >
> >
> >
> >As I recall, when I looked a long long (er, let's not get into how
> >long) time ago, the likeliness of actual collision with allocated OUIs
> >was quite small.
> >
> >
> >Alia
> >
> >
> >
> >
> >
> >Cc: Thomas Narten; Carlos Pignataro (cpignata); [email-masked]
> ><mailto:[email-masked]>; Jim Guichard (jguichar); Martin Stiemerling;
> >Xuxiaohu; Joel M. Halpern
> >
> >I think you are referring to RFC 4182, it is more than a decade since
> >it was published.
> >
> >/Loa
> >
> >
> >thorough review of NSH as well as requesting other focused reviews.
> >
> >          |Ver|O|C|R|R|R|R|R|R|   Length  |    MD Type    | Next
> Protocol
> >|
> >

---

From: "Paul Quinn (paulq)" <[email-masked]>
Date: Thu, 14 Apr 2016 17:03:38 +0000

Hi Dave,

Sorry, I was traveling yesterday.  In principal I don't have any issues with the text since it doesn't impose requirements but rather clarifies a possible behavior.

I do wonder about how common this will be: a control of sorts is needed for semantic meaning -- in type 1 or type 2 cases.

Paul

> On Apr 14, 2016, at 9:33 AM, Dave Dolson <[email-masked]> wrote:
>

---

From: "Paul Quinn (paulq)" <[email-masked]>
Date: Thu, 14 Apr 2016 17:20:32 +0000

Paul B.

Some comments inline.

> On Apr 13, 2016, at 6:05 PM, Bottorff, Paul <[email-masked]> wrote:
> 
> Hi Thomas:
> 

PQ>  You draw this broad stoke, however, it is relatively unsubstantiated with technical facts.  The WG discussed the representation of the SPI/SI prior to NSH adoption.  The SPI/SI are not transport values -- and please don't conflate transport (and the associated requirements) with address family.  

> 2)there is no resolution to forward/reverse addressing necessary for support of many L4-high service functions,

PQ> That is factually incorrect and I think derives from your opinion that SPI/SI == address family: in general, you can create two unidirectional chains and use them for forward and reverse traffic.  This has been reviewed with /by  L4+ vendors/experts.

> 3)there is no resolution to encoding hierarchies which are of importance to SFC applications in NFV environments,

PQ> Encoding hierarchies is an optimization of the base protocol to support scaling in a large deployment; in any case such optimization does not require changes to the base SFC encapsulation.

> 4)the draft needs a transport service interface and a description of the mappings used for the service layer onto the transport layer,

PQ> I am not sure what you mean by "transport service interface" - it is true that the SPI/SI need to map to a transport layer forwarding entry but I don't know what you are implying.

> 5)the draft specifies two independent formats for encoding the NSH header which is likely to divide the market slowing the deployment of NSH.

PQ>  The draft actually specifies two metadata formats for flexibility and the market will decide which is most appropriate based on specific use cases.  The evidence is clear: a wide-range of _implementations_ already exist. In fact, the real impediment to deployment is esoteric opinion.

> 1)Since every SF in a path must have a sequential SI, in the event we need to insert or delete a SF it is necessary to update the SFF forwarding entries for the entire chain. To fix this either the…

PQ> This is not a true statement; if an SF needs to be added or deleted then it is necessary to update the SFF forwarding entries for all SFFs downstream of where the change occurs. 

> 2)Since the SI is only decremented on SF hops it can't guard against a loop between SFFs or SCFs, hence SI is not a reliable method for loop squelching. If a TTL is needed it needs to be separated…

PQ> The SI can provide a simple loop check within the serivce plane but it's primary semantic value is one of location.  The transport itself can provide loop detection as well as SFF verification of SPI/SI + source.  

> 3)Every other successful address space has needed to add a few bits to support QoS options. I would suggest taking 3-4 bits from the MD-type and to reserve for QoS handling.

PQ>  Again, this is blurring the lines: QoS is handled, as expected by the selection transport.  This is a feature, not a bug, architecturally speaking.  

> 4)In may data centers we have multi-vendor deployments. In these environments we may have different managers assigning chains. There should to be some convention for splitting the address space bet…

PQ>  You are conflating federation/configuration and frame format.  The multi-manager scenario is a control-plane/management-plane function.

---

From: Henry Fourie <[email-masked]>
Date: Thu, 14 Apr 2016 18:27:40 +0000

Paul,
   Updating SFF forwarding entries on all downstream SFFs can introduce disruption
of traffic on the service path. It would help if there is a mechanism where a 
chain update to add/remove an SF can be localized to its subtending SFF only.

 - Louis 

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Paul Quinn (paulq)
Sent: Thursday, April 14, 2016 10:21 AM
To: Bottorff, Paul
Cc: Thomas Narten; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Paul B.

Some comments inline.

---

From: "Bottorff, Paul" <[email-masked]>
Date: Fri, 15 Apr 2016 21:36:20 +0000

Hi Paul Q.

Some responses to comments.

Cheers,

Paul

-----Original Message-----
From: Paul Quinn (paulq) [mailto:[email-masked]] 
Sent: Thursday, April 14, 2016 10:21 AM
To: Bottorff, Paul <[email-masked]>
Cc: Thomas Narten <[email-masked]>; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Paul B.

Some comments inline.

PQ>  You draw this broad stoke, however, it is relatively unsubstantiated with technical facts.  The WG discussed the representation of the SPI/SI prior to NSH adoption.  The SPI/SI are not transport values -- and please don't conflate transport (and the associated requirements) with address family.  
PB> I have never asserted that SPI+SI for a "transport" address. Please don't confuse what I've said. The modeling of SPI+SI as a service layer address is very soundly rooted in classic network layer modeling. It is the assertion that that the service layer is independent from the "transport" layer below that indicates the SPI+SI form the address used by the service layer dialog. This address is mapped into the "transport" by mapping the SPI+SI into a transport tunnel and destination address.

PQ> That is factually incorrect and I think derives from your opinion that SPI/SI == address family: in general, you can create two unidirectional chains and use them for forward and reverse traffic.  This has been reviewed with /by  L4+ vendors/experts.
PB>We have a lot of work on both L4-higher SFs and proxy forwarders for L4-higher SFs for MAC Chaining and understand this problem in a lot of detail. Having two unidirectional chains don't solve the forward/reverse traffic problem because the reverse path SPI/SI are not included in the NSH header and therefore not available at the time we need to reverse the forwarding direction. Are you mandating every SF and Proxy be provisioned with the forward and reverse SPI/SI? If so the solution is unacceptable to me.  

PQ> Encoding hierarchies is an optimization of the base protocol to support scaling in a large deployment; in any case such optimization does not require changes to the base SFC encapsulation.
PB> A general stacking solution will require some type of support which I believe should be in the base header so the context words are kept clear for pure meta-data.

PQ> I am not sure what you mean by "transport service interface" - it is true that the SPI/SI need to map to a transport layer forwarding entry but I don't know what you are implying.
PB>You are trying to specify a "transport" independent layer, however have not specified what services the transport must provide to support the SFC layer. A service interface is one of the formalized ways to specify the feature requirements of the underlying layer.

PQ>  The draft actually specifies two metadata formats for flexibility and the market will decide which is most appropriate based on specific use cases.  The evidence is clear: a wide-range of _implementations_ already exist. In fact, the real impediment to deployment is esoteric opinion.
PB> This is always the last resort dodge we use in standards, however it always comes at the price of some market confusion and delay. Even though I can accept this if no middle ground is possible, I don't think we have pushed the parties hard enough toward a converged solution to give it up on it yet.

PQ> This is not a true statement; if an SF needs to be added or deleted then it is necessary to update the SFF forwarding entries for all SFFs downstream of where the change occurs. 
PB> Agreed, I stand corrected.

PQ> The SI can provide a simple loop check within the serivce plane but it's primary semantic value is one of location.  The transport itself can provide loop detection as well as SFF verification of SPI/SI + source.  
PB> I both disagree that SI is useful for loop squeltching or that "transport" TTL can stop service layer looping. To use the "transport" TTL  the field would need to be copied from one "transport" to the next "transport" over the SFC service layer. Disregarding the fact that this operation is building a dependency between the "transport" and service layers, which we expressly are trying to avoid, the TTL fields of the transports are not sized to allow for the expanded path lengths which could be created in chains. In addition, some of the transports like Ethernet only support TTL for in special cases (Ethernet only supports TTL when using F-Tag forwarding).

PQ>  Again, this is blurring the lines: QoS is handled, as expected by the selection transport.  This is a feature, not a bug, architecturally speaking.  
PB> Just the opposite, failing to add QoS to the NSH header is blurring the lines between the service layer and the "transport" layer. Consider an a chain which uses both a best effort and an expedited service. Next consider building the chain where each hop uses a different type of "transport". As we map from one "transport" to the next we may over some hops have less QoS information than others. As we continue to the next "transport" we retain the integrity of the end-to-end communication by regenerating QoS from the original QoS contained at the service layer a driven down into the "transport". 

PQ>  You are conflating federation/configuration and frame format.  The multi-manager scenario is a control-plane/management-plane function.
PB> In a multi-manager situation I need a way to avoid collision in the allocation of the SPI/SI space which is being independently allocated by different managers.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 15 Apr 2016 17:54:39 -0400

Paul B., just to make sure I am reading you correctly, on the issue of 
reverse direction SFPs, I believe the issue you are referring to is 
specifically the case of SFs which need to originate a packet back to 
the source of a packet the SF has processed?
While I think SFs can do better, it would seem that a properly 
configured classifier at the SFF service that SF can take care of that 
case.  Requiring that it be encoded in the SPI seems overkill.

Yours,
Joel

---

From: "Elzur, Uri" <[email-masked]>
Date: Sat, 16 Apr 2016 00:00:50 +0000

Catching up (was traveling myself, sigh)

I'm ok with the added text too.  I think it may solve your need/case, not sure how broad it is, but see no harm. It also requires some control plane interaction as the metadata is not guaranteed to stay constant for the duration, so the disclaimer you propose is necessary

Thx

Uri ("Oo-Ree")
C: 949-378-7568

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Paul Quinn (paulq)
Sent: Thursday, April 14, 2016 10:04 AM
To: Dave Dolson <[email-masked]>
Cc: Elzur, Uri <[email-masked]>; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Hi Dave,

Paul

> I feel that it allows the actual content to be opaque while giving 
> guidance for service functions that create or replay packets in the chain.
>

---

From: "Elzur, Uri" <[email-masked]>
Date: Sat, 16 Apr 2016 01:35:23 +0000

And what is a symmetric return path is desired? Now that poor SFF need to be the traffic cop for the benefit of the downstream SFFs

If we assume that the LB/HA are accomplished by control Plane or by means other than direct insert/remove, this is not a common case

Thx

Uri ("Oo-Ree")
C: 949-378-7568

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Henry Fourie
Sent: Thursday, April 14, 2016 11:28 AM
To: Paul Quinn (paulq) <[email-masked]>; Bottorff, Paul <[email-masked]>
Cc: Thomas Narten <[email-masked]>; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Paul,
   Updating SFF forwarding entries on all downstream SFFs can introduce disruption of traffic on the service path. It would help if there is a mechanism where a chain update to add/remove an SF can be localized to its subtending SFF only.

 - Louis 

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Paul Quinn (paulq)
Sent: Thursday, April 14, 2016 10:21 AM
To: Bottorff, Paul
Cc: Thomas Narten; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Paul B.

Some comments inline.

> 1)the service address formed by the combined SPI+SI appears to have a 
> number of deficiencies which lead to transport dependencies and to 
> layer violations,

> 2)there is no resolution to forward/reverse addressing necessary for 
> support of many L4-high service functions,

> 3)there is no resolution to encoding hierarchies which are of 
> importance to SFC applications in NFV environments,

> 4)the draft needs a transport service interface and a description of 
> the mappings used for the service layer onto the transport layer,

---

From: Gregory Mirsky <[email-masked]>
Date: Sat, 16 Apr 2016 04:49:07 +0000

Hi Uri,
bi-directional path presumes that the service layer is terminated at egress and ingress, not that client data circle, that may be characteristic of a ring or routing loop. And if there's classifier at ingress, then, wouldn't there be a classifier at egress? If there's no classifier at the turning SF, then isn't that rather case of unidirectional path that is terminated at its very ingress?

	Regards,
		Greg

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Elzur, Uri
Sent: Friday, April 15, 2016 6:35 PM
To: Henry Fourie; Paul Quinn (paulq); Bottorff, Paul
Cc: Thomas Narten; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Thx

Uri ("Oo-Ree")
C: 949-378-7568

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Henry Fourie
Sent: Thursday, April 14, 2016 11:28 AM
To: Paul Quinn (paulq) <[email-masked]>; Bottorff, Paul <[email-masked]>
Cc: Thomas Narten <[email-masked]>; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Paul,

 - Louis 

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Paul Quinn (paulq)
Sent: Thursday, April 14, 2016 10:21 AM
To: Bottorff, Paul
Cc: Thomas Narten; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Paul B.

Some comments inline.

---

From: David Melman <[email-masked]>
Date: Sun, 17 Apr 2016 07:48:16 +0000

Hi Thomas:

I support last call on draft-ietf-sfc-nsh-04.

Regarding MPLS transport, I agree this is not an issue for this draft, which defines NSH as transport independent.   NSH-over-MPLS should be addressed by the MPLS WG, where a solution needs to be found to resolve the nibble issue, e.g. CW-like solution.

Regarding MD type 1, 16B opaque context data is a good generic approach.  Metadata will evolve over time and be standardized.

David

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Thomas Narten
Sent: Wednesday, March 30, 2016 7:48 PM
To: [email-masked]
Subject: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Dear WG:

For the chairs,
Thomas

---

From: "Elzur, Uri" <[email-masked]>
Date: Mon, 18 Apr 2016 00:05:16 +0000

Hi Greg

My comment was towards the "inefficiency" stated with the need to update all downstream SFF in case of insertion/removal
It was suggested that a mechanism where "a localized" mechanism may be better

My comment was that in case of any returning traffic (and the graph may be complex...) the affected SFF will know what to do with any returning traffic on any branch of the graph... so it may not be a bad choice to notify all downstream SFFs

The second comment is that in case LB/HA are accomplished by/with control Plane, the SFF involved in the LB, for example, may hide these actions (or localize them) rendering this to a more rare event 

Thx

Uri ("Oo-Ree")
C: 949-378-7568

-----Original Message-----
From: Gregory Mirsky [mailto:[email-masked]] 
Sent: Friday, April 15, 2016 9:49 PM
To: Elzur, Uri <[email-masked]>; Henry Fourie <[email-masked]>; Paul Quinn (paulq) <[email-masked]>; Bottorff, Paul <[email-masked]>
Cc: Thomas Narten <[email-masked]>; [email-masked]
Subject: RE: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Hi Uri,

	Regards,
		Greg

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Elzur, Uri
Sent: Friday, April 15, 2016 6:35 PM
To: Henry Fourie; Paul Quinn (paulq); Bottorff, Paul
Cc: Thomas Narten; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Thx

Uri ("Oo-Ree")
C: 949-378-7568

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Henry Fourie
Sent: Thursday, April 14, 2016 11:28 AM
To: Paul Quinn (paulq) <[email-masked]>; Bottorff, Paul <[email-masked]>
Cc: Thomas Narten <[email-masked]>; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Paul,

 - Louis 

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Paul Quinn (paulq)
Sent: Thursday, April 14, 2016 10:21 AM
To: Bottorff, Paul
Cc: Thomas Narten; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Paul B.

Some comments inline.

---

From: "Fedyk, Don" <[email-masked]>
Date: Mon, 18 Apr 2016 18:04:21 +0000

Hi Joel

> Sent: Wednesday, April 13, 2016 6:10 PM
> To: Fedyk, Don <[email-masked]>; [email-masked]
<Snip>  
> So the question I have is what wording in the NSH draft causes you concern.
> Is there wording in the NSH draft that gives the impression that the SPI / SI is
> only an "identifier"?  Is there wording that we could add that would help?
[Don]  - Since you bring it up: Short answer yes when you get to section 7.4 Figure 14. 

The wording around service forwarding is a bit disconnected spread over a few sections as I understand it there are 3 modes of forwarding operations involving SFFs and SFs that I can see. It was initially discussions on the list about identifiers that made me look more deeply at the forwarding.   I'll keep my comments to service forwarding even though these sections also discuss the mapping to transport I'll focus on SPI and SI handling text. 

Also note there are two occurrences of SPI/ID and 4 occurrences of SPI/SI which I assume are both Service Path Identifier and Service Index.  ID is never defined but SI is equated to node ID or location.  I think SPI/ID is a typo.

With respect to SF and SFF service forwarding 
Section 4 and Sections 7.1 - 7.3 
1) SF NSH Aware using SI decrement.   The SFF sends to an SF that does its thing, and has access to any metadata and passes the packet back (to the same SFF) with a decremented SI.   The SFF then looks up the next hop based on the SPI/SI that maps to another SF optionally via a Network transport mapping.  
2) SF NSH Unaware - A proxy does the decrementing of the SI. 

Observation:  The decrementing of SI keeps the forwarding through an SF to a simplistic form. The text describing this mode implies SF tracking using SI and uses a programmed lookup of the next SPI/SI. 

Note: The text about decrementing to zero seems unnecessary given that any SFF that receives an SPI with any SI value that is unexpected is a forwarding mistake.   Zero is just a specific example of the general case of chain programming where SPI/SI must make sense at each hop. SI of zero does not make sense but neither do any other values of SI that are not programmed for that SPI/SI combination.   
We have always assumed that a control plane/management plane are used to setup/verify chains and some form of OAM should be used to verify chains. 

Since the simplistic form does not allow the more complex chains, in Section 7.4 you have the third mode. 
3) SF NSH Aware with reclassification - This mode allows an SF to do anything to SPI/SI - not specified exactly what, just that reclassification is allowed and presumably SPI and or SI can be changed/reset.   One might imply from reading and figure 14 that this is based on "identifier" (SPI ) lookup and context (or Class) in this example.  It seems to follow that SFs are less concerned with SI actual values.  This sections implies SPI can be changed and does not describe what happens to SI. Presumably the SI is set to some "new' value since an initial classifier would not know what SI should be for all combinations of chains so SI is  location only with respect to a specific SPI. 

Is it true that SI is not consulted when reclassification takes place?   

Cheers
Don 

> > chain ID with various packet formats;  the answers were NSH SID and SI
> > were just identifiers.  So while there is not a statement they are
> >> [mailto:[email-masked]] Sent: Wednesday, April 13, 2016 4:09 PM
> >> describe a forwarding behavior, or that those fields are not for use
> >> by forwarding.  It is not REQUIRED that they be used for forwarding
> >> at an SFF, but it is certainly permitted.
> >>
> >>> The only way I can reason it out is to see other examples of
> >>> complete packet headers and operations per hop.
> >>>> compare with the other data formats that I know today.
> >>>> For example the MAC chaining draft we authored would like to use
> >>>> NSH for metadata but the MAC chaining local MACs bear a
> >>>> relationship with the NSH SDI & SI. In fact without these fields as
> >>>> mandatory it would be more independent in my case.
> >>>> I'm sure there are similar aspects with other transports types but
> >>>> until I see them and the operations I can't assess transport
> >>>> independence claims.
> >>>>>

---

From: "Bottorff, Paul" <[email-masked]>
Date: Mon, 18 Apr 2016 18:38:41 +0000

Hi Joel:

Some comments below,

Cheers,

Paul

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Joel M. Halpern
Sent: Friday, April 15, 2016 2:55 PM
To: Bottorff, Paul <[email-masked]>; Paul Quinn (paulq) <[email-masked]>; Elzur, Uri <[email-masked]>
Cc: Thomas Narten <[email-masked]>; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Paul B., just to make sure I am reading you correctly, on the issue of reverse direction SFPs, I believe the issue you are referring to is specifically the case of SFs which need to originate a packet back to the source of a packet the SF has processed?
PB>Yes, this is the original reference. This is also something which OAM sometimes depends on since an OAM may generate a reply.

While I think SFs can do better, it would seem that a properly configured classifier at the SFF service that SF can take care of that case.  Requiring that it be encoded in the SPI seems overkill.
PB>Fixing this at the SFF requires some indication for flag from the SF to indicate the frame is traveling (or is to travel) in the reverse direction. I have proposed that using the transport source address could be used at the SFF to make this determination, however have received pushback on this solution since it does not work for single armed SFs. Since the SA solution may not be acceptable, I believe we need to provide a forward/reverse indicator.

Yours,
Joel

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 19 Apr 2016 01:05:58 -0400

Paul B., if we are using a classifier to generate the NSH header, then 
all we need to do is send the packet back either without an NSH, or with 
an NSH with a bit indicating that reclassification is needed.  The 
source and destination addresses in the packet will tell the classifier 
what it needs to know in order to generate the right NSH header.

Other options include configuring the SF with the reverse direction SPI 
/ SI.

None of these require a direction bit in the neader.  Which is good, as 
trying to define forward and backwards is pretty confusing in a network.

Yours,
Joel

On 4/18/16 2:38 PM, Bottorff, Paul wrote:
> Hi Joel:
>
> Some comments below,
>
> Cheers,
>
> Paul
>
> -----Original Message----- From: sfc [mailto:[email-masked]] On
> Behalf Of Joel M. Halpern Sent: Friday, April 15, 2016 2:55 PM To:
> Bottorff, Paul <[email-masked]>; Paul Quinn (paulq)
> <[email-masked]>; Elzur, Uri <[email-masked]> Cc: Thomas Narten
> Paul B., just to make sure I am reading you correctly, on the issue
> of reverse direction SFPs, I believe the issue you are referring to
> is specifically the case of SFs which need to originate a packet back
> to the source of a packet the SF has processed? PB>Yes, this is the
> original reference. This is also something which OAM sometimes
> depends on since an OAM may generate a reply.
>
> configured classifier at the SFF service that SF can take care of
> that case.  Requiring that it be encoded in the SPI seems overkill.
> PB>Fixing this at the SFF requires some indication for flag from the
> SF to indicate the frame is traveling (or is to travel) in the
> reverse direction. I have proposed that using the transport source
> address could be used at the SFF to make this determination, however
> have received pushback on this solution since it does not work for
> single armed SFs. Since the SA solution may not be acceptable, I
> believe we need to provide a forward/reverse indicator.
>
> Yours, Joel
>
>> -----Original Message----- From: Paul Quinn (paulq)
>> [mailto:[email-masked]] Sent: Thursday, April 14, 2016 10:21 AM
>> To: Bottorff, Paul <[email-masked]> Cc: Thomas Narten
>>> I do not support last call on draft-eitf-sfc-nsh-04. The risk of
>>> rushing to close a format before we have clear agreement on the
>>> forwarding principles and features is just too high.
>>>
>>> The major problems with the current draft are: 1)the service
>>> address formed by the combined SPI+SI appears to have a number of
>>> deficiencies which lead to transport dependencies and to layer
>>> violations,
>>
>> PQ>  You draw this broad stoke, however, it is relatively
>> unsubstantiated with technical facts.  The WG discussed the
>> representation of the SPI/SI prior to NSH adoption.  The SPI/SI are
>> not transport values -- and please don't conflate transport (and
>> the associated requirements) with address family. PB> I have never
>> asserted that SPI+SI for a "transport" address. Please don't
>> confuse what I've said. The modeling of SPI+SI as a service layer
>> address is very soundly rooted in classic network layer modeling.
>> It is the assertion that that the service layer is independent from
>> the "transport" layer below that indicates the SPI+SI form the
>> address used by the service layer dialog. This address is mapped
>> into the "transport" by mapping the SPI+SI into a transport tunnel
>> and destination address.
>>
>>> 2)there is no resolution to forward/reverse addressing necessary
>>> for support of many L4-high service functions,
>>
>> PQ> That is factually incorrect and I think derives from your
>> opinion that SPI/SI == address family: in general, you can create
>> two unidirectional chains and use them for forward and reverse
>> traffic.  This has been reviewed with /by  L4+ vendors/experts.
>> PB>We have a lot of work on both L4-higher SFs and proxy forwarders
>> for L4-higher SFs for MAC Chaining and understand this problem in a
>> lot of detail. Having two unidirectional chains don't solve the
>> forward/reverse traffic problem because the reverse path SPI/SI are
>> not included in the NSH header and therefore not available at the
>> time we need to reverse the forwarding direction. Are you mandating
>> every SF and Proxy be provisioned with the forward and reverse
>> SPI/SI? If so the solution is unacceptable to me.
>>
>>
>> PQ> Encoding hierarchies is an optimization of the base protocol to
>> support scaling in a large deployment; in any case such
>> optimization does not require changes to the base SFC
>> encapsulation. PB> A general stacking solution will require some
>> type of support which I believe should be in the base header so the
>> context words are kept clear for pure meta-data.
>>
>>> 4)the draft needs a transport service interface and a description
>>> of the mappings used for the service layer onto the transport
>>> layer,
>>
>> PQ> I am not sure what you mean by "transport service interface" -
>> it is true that the SPI/SI need to map to a transport layer
>> forwarding entry but I don't know what you are implying. PB>You are
>> trying to specify a "transport" independent layer, however have not
>> specified what services the transport must provide to support the
>> SFC layer. A service interface is one of the formalized ways to
>> specify the feature requirements of the underlying layer.
>>
>>> 5)the draft specifies two independent formats for encoding the
>>> NSH header which is likely to divide the market slowing the
>>> deployment of NSH.
>>
>> PQ>  The draft actually specifies two metadata formats for
>> flexibility and the market will decide which is most appropriate
>> based on specific use cases.  The evidence is clear: a wide-range
>> of _implementations_ already exist. In fact, the real impediment to
>> deployment is esoteric opinion. PB> This is always the last resort
>> dodge we use in standards, however it always comes at the price of
>> some market confusion and delay. Even though I can accept this if
>> no middle ground is possible, I don't think we have pushed the
>> parties hard enough toward a converged solution to give it up on it
>> yet.
>>
>>>
>>>> From my point of view the SFC formed service plane layer
>>>> network with a peer dialog between the SCF, SFF and SF.  The
>>>> Service Path Header (SPI+SI) forms the address for this service
>>>> plane. As it stands there is an attempt to overload the
>>>> function of the SI as both the SF identifier with the Service
>>>> Path and as a form of loop detection, however the coupling has
>>>> resulted in two problems:
>>> 1)Since every SF in a path must have a sequential SI, in the
>>> event we need to insert or delete a SF it is necessary to update
>>> the SFF forwarding entries for the entire chain. To fix this
>>> either the SI should be allowed to take non-sequential values
>>> along the chain or should not be used for SF addressing.
>>
>> PQ> This is not a true statement; if an SF needs to be added or
>> deleted then it is necessary to update the SFF forwarding entries
>> for all SFFs downstream of where the change occurs. PB> Agreed, I
>> stand corrected.
>>
>>
>>> 2)Since the SI is only decremented on SF hops it can't guard
>>> against a loop between SFFs or SCFs, hence SI is not a reliable
>>> method for loop squelching. If a TTL is needed it needs to be
>>> separated from SI. Given that there is no clear alternative to
>>> TTL for loop squelching at the service layer I believe we do need
>>> to include an effective TTL to go forward.
>>
>> PQ> The SI can provide a simple loop check within the serivce plane
>> but it's primary semantic value is one of location.  The transport
>> itself can provide loop detection as well as SFF verification of
>> SPI/SI + source. PB> I both disagree that SI is useful for loop
>> squeltching or that "transport" TTL can stop service layer looping.
>> To use the "transport" TTL  the field would need to be copied from
>> one "transport" to the next "transport" over the SFC service layer.
>> Disregarding the fact that this operation is building a dependency
>> between the "transport" and service layers, which we expressly are
>> trying to avoid, the TTL fields of the transports are not sized to
>> allow for the expanded path lengths which could be created in
>> chains. In addition, some of the transports like Ethernet only
>> support TTL for in special cases (Ethernet only supports TTL when
>> using F-Tag forwarding).
>>
>>> 3)Every other successful address space has needed to add a few
>>> bits to support QoS options. I would suggest taking 3-4 bits from
>>> the MD-type and to reserve for QoS handling.
>>
>> PQ>  Again, this is blurring the lines: QoS is handled, as expected
>> by the selection transport.  This is a feature, not a bug,
>> architecturally speaking. PB> Just the opposite, failing to add QoS
>> to the NSH header is blurring the lines between the service layer
>> and the "transport" layer. Consider an a chain which uses both a
>> best effort and an expedited service. Next consider building the
>> chain where each hop uses a different type of "transport". As we
>> map from one "transport" to the next we may over some hops have
>> less QoS information than others. As we continue to the next
>> "transport" we retain the integrity of the end-to-end communication
>> by regenerating QoS from the original QoS contained at the service
>> layer a driven down into the "transport".
>>
>>> 4)In may data centers we have multi-vendor deployments. In these
>>> environments we may have different managers assigning chains.
>>> There should to be some convention for splitting the address
>>> space between these vendors. The most desirable solution to this
>>> is to add an assigned organization ID to the address space. An
>>> alternative would be to separate the SPI into a 4 and 20 bit
>>> field designating the high 4 bits as a local manager ID.
>>>
>>
>> PQ>  You are conflating federation/configuration and frame format.
>> The multi-manager scenario is a control-plane/management-plane
>> function. PB> In a multi-manager situation I need a way to avoid
>> collision in the allocation of the SPI/SI space which is being
>> independently allocated by different managers.
>>

---

From: "Jim Guichard (jguichar)" <[email-masked]>
Date: Thu, 21 Apr 2016 01:34:47 +0000

[individual contributor hat on ...]

>From a purely technical standpoint, the MPLS community has already addressed the ECMP issue through RFC6790 Entropy Labels.

Jim

Sent from my iPhone

> On Apr 13, 2016, at 11:42 PM, Xuxiaohu <[email-masked]> wrote:
> 
> 
> 
>> Protocol |

---

From: Sunil Vallamkonda <[email-masked]>
Date: Thu, 21 Apr 2016 18:44:59 +0000

Hi Paul,

Does MDType 1 need to be a MUST ?
If so, can Context header be customizable per implementation ?
Futher re: Sec.3.4, is there way for "Context Headers to carry no metadata MUST be set to zero", is it possible have a bit in header to indicate to ignore Context header for recipient to skip reading them ? 

Thanks,
Sunil.

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Paul Quinn (paulq)
Sent: Thursday, April 14, 2016 10:04 AM
To: Dave Dolson <[email-masked]>
Cc: [email-masked]; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Hi Dave,

Paul

---

From: "Elzur, Uri" <[email-masked]>
Date: Sun, 24 Apr 2016 15:36:20 +0000

Hi Med

I see no need to specify the exact behavior as NSH is transport independent i.e. like NSH interaction with any other Transport eh issue of Fragmentation is to be dealt in a way that matches the mechanisms supported by the Transport used and do not belong in the NSH draft

Thx

Uri ("Oo-Ree")
C: 949-378-7568

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Thursday, April 14, 2016 12:43 AM
To: Thomas Narten <[email-masked]>; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

R-,

==

 

Cheers,
Med

> [email-masked] Envoyé : lundi 11 avril 2016 13:14 À : 
> Thomas Narten; [email-masked] Objet : Re: [sfc] WG last call for 
> As I mentioned during the meeting, there are several issues that are 
> not covered in the last version of the draft. I already provided 
> examples of the issues offline as requested by Martin.
> 
> Cheers,
> Med
> 
> > Envoyé : jeudi 31 mars 2016 04:48 À : [email-masked] Objet : [sfc] WG 
> > last call for draft-ietf-sfc-nsh-04.txt
> >
> > Dear WG:
> >

---

From: <[email-masked]>
Date: Mon, 25 Apr 2016 05:31:42 +0000

Hi Uri,

That's another option that needs to be discussed/investigated. I'm afraid this is not the rationale adopted in -04 since it includes some text that is far to be sufficient to ensure interoperable implementations.

BTW, saying that nsh does not need to deal with fragmentation because it is transport-independent is not IMHO a good strategy to adopt here because it opens the door for interoperable issues, it may lead to sub-optimal implementations if the sfc information is present only in one fragments, etc.   

My comments are related to the current text in the -04. This text needs to be fixed somehow.

Cheers,
Med

> De : Elzur, Uri [mailto:[email-masked]]
> Envoyé : dimanche 24 avril 2016 17:36
> À : BOUCADAIR Mohamed IMT/OLN; Thomas Narten; [email-masked]
> Objet : RE: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt
> 
> Hi Med
> 
> I see no need to specify the exact behavior as NSH is transport
> independent i.e. like NSH interaction with any other Transport eh issue of
> Fragmentation is to be dealt in a way that matches the mechanisms
> supported by the Transport used and do not belong in the NSH draft
> 
> Thx
> 
> Uri ("Oo-Ree")
> C: 949-378-7568
> 
> 
> In addition to other pending issues raised for this draft, I would like to
> raise this additional one about Section 6.
> 
> ==
> * The text is weak for a Standard track document that is intended to solve
> the problem in https://tools.ietf.org/html/rfc7498#section-2.12. There
> should be a clear behavior to be followed by an implementation. Further, I
> would avoid the use of words such as "can".
> 
> * The text covers only fragmentation when it is induced by SFC operations,
> it does not discuss the treatment of a fragment when received in an SFC
> domain. IMHO, the draft should also specify the behavior of the Classifier
> with regards to fragments for the sake of proper SFC operation. Applying
> classification policies may require the full packet, not only a fragment.
> In particular, dedicated resources should dedicated for handling out of
> order fragments. Of course, it is out of scope of this document to
> describe how SFs handle fragments.
> 
> * If an SFC header is prepended for all fragments, I'm not sure there is
> any particular issue at the SFF level...except if stripping/adding context
> TLVs depends on the full packet (not just fragment). It is warranted to
> consider a little bit this point before declaring there is no issue.
> 
> * The text states "several options". This may be interpreted as if
> implementing one of them is sufficient...which is not true. The first two
> points contribute to minimize the fragmentation risk, but fragmentation
> may still be experienced (e.g., other shims are prepended by other nodes
> for some other purposes, nested nsh, etc.)
> 
> * The support of jumbo frames by a router/device does not mean that it can
> make use of it. Appropriate MTU configuration should be undertaken in a
> consistent manner within an SFC domain. The text should be updated to make
> it is about (consistent) MTU configuration.
> 
> * BTW, shouldn't the text be reworded to recommended to increase the MTU
> of **all nodes** of an SFC-enabled domain by at least the length of SFC
> header + transport header?
> 
> * Bullet 2, how PMTU discovery is actually used in this context? Do you
> assume that all SFC-aware nodes will issue such messages towards other
> SFC-aware node, arbitrary destination, else?
> 
> * Bullet 2, I would drop "describes a technique for dynamically
> discovering the maximum transmission unit (MTU) of an arbitrary internet
> path".
> 
> (1) I don't think it is a good approach that fragments induced by the
> network are passed to their ultimate destinations as such (stateless).
> IMO, reassembly should be done within the SFC domain (responsible for
> fragmentation) not passed away.
> (2) Does the stateful mode require all SFC data plane elements maintain a
> full list of MTU for the any SFF/SF instance of the SFC domain?
> 
> The current text suggests that [RFC6830] should be listed as normative
> reference (not an informative one). I would personally favor removing the
> reference to LISP (as it is an Experimental RFC).
> 
> * The security section of the draft may be augmented with informational
> fragmentation-related pointers to: e.g., RFC1858 (Security Considerations
> for IP Fragment Filtering), RFC3128 (Protection Against a Variant of the
> Tiny Fragment Attack), and RFC 4963 (IPv4 Reassembly Errors at High Data
> Rates).
> 
> Cheers,
> Med
>

---

From: "Elzur, Uri" <[email-masked]>
Date: Mon, 25 Apr 2016 06:29:46 +0000

Hi Med

My point is that Fragmentation is yet another transport related issue that is beyond the scope of NSH and beyond the charter of the WG, so it doesn't really belong in the draft. We are providing an advice as extending the size of the packet may lead to fragmentation, but as you check RFC 7348 VxLAN, which my create the same issue, you'll find it doesn't even relate to it.

Thx

Uri ("Oo-Ree")
C: 949-378-7568

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Sunday, April 24, 2016 10:32 PM
To: Elzur, Uri <[email-masked]>; Thomas Narten <[email-masked]>; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Hi Uri,

Cheers,
Med

> De : Elzur, Uri [mailto:[email-masked]] Envoyé : dimanche 24 
> avril 2016 17:36 À : BOUCADAIR Mohamed IMT/OLN; Thomas Narten; 
> [email-masked] Objet : RE: [sfc] WG last call for 
> independent i.e. like NSH interaction with any other Transport eh 
> issue of Fragmentation is to be dealt in a way that matches the 
> mechanisms supported by the Transport used and do not belong in the 
> NSH draft
> 
> Thx
> 
> Uri ("Oo-Ree")
> C: 949-378-7568
> 
> 
> In addition to other pending issues raised for this draft, I would 
> like to raise this additional one about Section 6.
> 
> ==
> * The text is weak for a Standard track document that is intended to 
> solve the problem in https://tools.ietf.org/html/rfc7498#section-2.12. 
> There should be a clear behavior to be followed by an implementation. 
> Further, I would avoid the use of words such as "can".
> 
> * The text covers only fragmentation when it is induced by SFC 
> operations, it does not discuss the treatment of a fragment when 
> received in an SFC domain. IMHO, the draft should also specify the 
> behavior of the Classifier with regards to fragments for the sake of 
> proper SFC operation. Applying classification policies may require the full packet, not only a fragment.
> In particular, dedicated resources should dedicated for handling out 
> of order fragments. Of course, it is out of scope of this document to 
> * If an SFC header is prepended for all fragments, I'm not sure there 
> is any particular issue at the SFF level...except if stripping/adding 
> context TLVs depends on the full packet (not just fragment). It is 
> warranted to consider a little bit this point before declaring there is no issue.
> 
> implementing one of them is sufficient...which is not true. The first 
> two points contribute to minimize the fragmentation risk, but 
> fragmentation may still be experienced (e.g., other shims are 
> prepended by other nodes for some other purposes, nested nsh, etc.)
> 
> * The support of jumbo frames by a router/device does not mean that it 
> can make use of it. Appropriate MTU configuration should be undertaken 
> in a consistent manner within an SFC domain. The text should be 
> updated to make it is about (consistent) MTU configuration.
> 
> * BTW, shouldn't the text be reworded to recommended to increase the 
> MTU of **all nodes** of an SFC-enabled domain by at least the length 
> of SFC header + transport header?
> 
> * Bullet 2, how PMTU discovery is actually used in this context? Do 
> you assume that all SFC-aware nodes will issue such messages towards 
> other SFC-aware node, arbitrary destination, else?
> 
> discovering the maximum transmission unit (MTU) of an arbitrary 
> internet path".
> 
> (2) Does the stateful mode require all SFC data plane elements 
> maintain a full list of MTU for the any SFF/SF instance of the SFC domain?
> 
> reference (not an informative one). I would personally favor removing 
> the reference to LISP (as it is an Experimental RFC).
> 
> * The security section of the draft may be augmented with 
> informational fragmentation-related pointers to: e.g., RFC1858 
> (Security Considerations for IP Fragment Filtering), RFC3128 
> (Protection Against a Variant of the Tiny Fragment Attack), and RFC 
> 4963 (IPv4 Reassembly Errors at High Data Rates).
> 
> Cheers,
> Med
> 
> > > Envoyé : jeudi 31 mars 2016 04:48 À : [email-masked] Objet : [sfc] 
> > > WG last call for draft-ietf-sfc-nsh-04.txt
> > >
> > > Dear WG:
> > >

---

From: <[email-masked]>
Date: Mon, 25 Apr 2016 07:00:18 +0000

Re-,

I hear you, but my comment is that we need, as a WG, to decide what to put in the draft. FWIW, I'm discussing two fragmentation issues: 

(1) Fragmentation that is caused by prepending an SFC header.
(2) Handling fragments at the ingress of an SFC-enabled domain. 

Increasing the MTU is for sure a recommendation is to be explicitly called out in the text (see my first message).

There are other issues that need to be discussed, e.g., how to deal with fragments in SFFs/Classifiers? 

It is also "prudent" to check that no issues will be experienced in SFF to handle fragments. If an SFC header is prepended for all fragments, I'm not sure there
is any particular issue at the SFF level, except if stripping/adding context TLVs depends on the full packet (not just fragment). It is warranted to consider a little bit this point before declaring there is no issue.

For point (1), declaring fragmentation out of scope would be meant that an implementation must be prepared to receive fragments with or without NSH header as this is a decision that is left to the transport encapsulation. This is a requirement per se!

I won't reiterate all the comments I have about the current wording of this section. 

Cheers,
Med

> Envoyé : lundi 25 avril 2016 08:30
> My point is that Fragmentation is yet another transport related issue that
> is beyond the scope of NSH and beyond the charter of the WG, so it doesn't
> really belong in the draft. We are providing an advice as extending the
> size of the packet may lead to fragmentation, but as you check RFC 7348
> VxLAN, which my create the same issue, you'll find it doesn't even relate
> to it.
> 
> Thx
> 
> Uri ("Oo-Ree")
> C: 949-378-7568
> 
> 
> That's another option that needs to be discussed/investigated. I'm afraid
> this is not the rationale adopted in -04 since it includes some text that
> is far to be sufficient to ensure interoperable implementations.
> 
> BTW, saying that nsh does not need to deal with fragmentation because it
> is transport-independent is not IMHO a good strategy to adopt here because
> it opens the door for interoperable issues, it may lead to sub-optimal
> implementations if the sfc information is present only in one fragments,
> etc.
>

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 25 Apr 2016 09:47:43 -0400

If I am understanding you correctly Med, you are asking that the NSH 
draft specify how service chaining should cope with packets that have 
been fragmented?

NSH, and the SFF functionality, does not care.  It just does its job.
Ingress and intermediate classifiers may cope with fragments in any 
number of ways.  Specifying such is clearly out of scope for this draft. 
  I suppose one could write an informational draft on possible ways of 
coping.  The IETF has not usually published such.
Service functions have to cope with fragmented packets just as they had 
to before the advent of NSH, so describing that is clearly not needed here.

Yours,
Joel

---

From: <[email-masked]>
Date: Mon, 25 Apr 2016 14:18:12 +0000

Hi Joel, 

Please see inline.

Cheers,
Med

> De : Joel M. Halpern [mailto:[email-masked]]
> Envoyé : lundi 25 avril 2016 15:48
> À : BOUCADAIR Mohamed IMT/OLN; [email-masked]

[Med] To be accurate, I'm asking to assess whether there are implications. If there are, then the draft should be updated accordingly.

> NSH, and the SFF functionality, does not care.

[Med] I'm not that sure. Some typical implications are listed below:

* SFF: If the NSH header is present only in the a fragment but SFF didn't maintained a state, subsequent fragments won't be appropriately processed. 
* SFC-aware function: if prepending a context information depends on the full packet, not only a fragment. 

  It just does its job.

[Med] which may be disturbed in some situations as listed in the examples above.

> number of ways.
  Specifying such is clearly out of scope for this draft.

[Med] The purpose is not to specify the internal implementation details but the external behavior of the classifier function when it comes to handle fragments. That behavior may have an incidence on SFF, in particular. The purpose is not to recommend the maximum resources to be dedicated to out of order fragments nor the timeout to cache those; these considerations are of course out of scope. Nevertheless, an implementation should offer a configurable parameter so that an operator tweak those according to its context.  

> to before the advent of NSH, so describing that is clearly not needed

[Med] The advent of NSH has the following implications:
* it exacerbates fragmentation. Handing over this issue to the transport layer may lead to interoperability issues. 
* the chaining may not be efficient if fragments are inappropriately handled. 

Introducing NSH should not degrade the overall service compared to legacy service deployment schemes.

> > I hear you, but my comment is that we need, as a WG, to decide what to
> put in the draft. FWIW, I'm discussing two fragmentation issues:
> >
> > Increasing the MTU is for sure a recommendation is to be explicitly
> called out in the text (see my first message).
> >
> > It is also "prudent" to check that no issues will be experienced in SFF
> to handle fragments. If an SFC header is prepended for all fragments, I'm
> not sure there
> > is any particular issue at the SFF level, except if stripping/adding
> > For point (1), declaring fragmentation out of scope would be meant that
> an implementation must be prepared to receive fragments with or without
> NSH header as this is a decision that is left to the transport
> encapsulation. This is a requirement per se!
> >
> >> To: Elzur, Uri <[email-masked]>; Thomas Narten
> <[email-masked]>;
> >> [email-masked]
> >> My comments are related to the current text in the -04. This text needs
> to
> >> be fixed somehow.
> >>
> >> Cheers,
> >> Med
> >>
> >>> warranted to consider a little bit this point before declaring there
> is
> >> no issue.
> >>>

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 25 Apr 2016 10:25:18 -0400

There are two different issues.
There is the question of what happens if adding the NSH header causes 
fragmentation.  I believe that is dealt with sufficiently.  It is in 
scope for the document.

The other issue is what happens if a packet is fragmented before it 
arrives at SFC.  the classifier can play any games it likes, but all of 
the packets it emits will end up getting NSH encapsulated.  The 
mechanism for doing that depends upon how the classifier works, and how 
the implementation couples the classifier to the encapsulator (which is 
internal and not subject ot standardization.)

So I do not see the disturbances you refer to.

Yours,
Joel

---

From: "Elzur, Uri" <[email-masked]>
Date: Mon, 25 Apr 2016 14:26:15 +0000

Hi Med

Not to repeat my position, but I'll do it anyhow :-)
As NSH is *NOT* a transport, dealing with fragmentation is left to the Transport used.

The model I use for NSH, is basically similar to VXLAN . It is an overly. 

Thx

Uri ("Oo-Ree")
C: 949-378-7568

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Monday, April 25, 2016 7:18 AM
To: Joel M. Halpern <[email-masked]>; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Hi Joel, 

Please see inline.

Cheers,
Med

> De : Joel M. Halpern [mailto:[email-masked]] Envoyé : lundi 25 
> avril 2016 15:48 À : BOUCADAIR Mohamed IMT/OLN; [email-masked] Objet : 
> Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt
> 

  It just does its job.

> Service functions have to cope with fragmented packets just as they 
> had to before the advent of NSH, so describing that is clearly not 
> needed here.

> > There are other issues that need to be discussed, e.g., how to deal 
> > with
> fragments in SFFs/Classifiers?
> >
> to handle fragments. If an SFC header is prepended for all fragments, 
> I'm not sure there
> is no issue.
> an implementation must be prepared to receive fragments with or 
> without NSH header as this is a decision that is left to the transport 
> > I won't reiterate all the comments I have about the current wording 
> > of
> this section.
> >
> > Cheers,
> > Med
> >
> >> De : Elzur, Uri [mailto:[email-masked]] Envoyé : lundi 25 
> >> avril 2016 08:30 À : BOUCADAIR Mohamed IMT/OLN; Thomas Narten; 
> >> My point is that Fragmentation is yet another transport related 
> >> issue
> that
> >> really belong in the draft. We are providing an advice as extending 
> >> the size of the packet may lead to fragmentation, but as you check 
> >> RFC 7348 VxLAN, which my create the same issue, you'll find it 
> >> doesn't even
> relate
> >> to it.
> >>
> >> Thx
> >>
> >> Uri ("Oo-Ree")
> >> C: 949-378-7568
> >>
> >>
> >> this is not the rationale adopted in -04 since it includes some 
> >> text
> that
> >> BTW, saying that nsh does not need to deal with fragmentation 
> >> because
> it
> >> it opens the door for interoperable issues, it may lead to 
> >> sub-optimal implementations if the sfc information is present only 
> >> in one
> fragments,
> >> etc.
> >>
> >>> mechanisms supported by the Transport used and do not belong in 
> >>> the NSH draft
> >>>
> >>> Thx
> >>>
> >>> Uri ("Oo-Ree")
> >>> C: 949-378-7568
> >>>
> >>>
> >>>    encapsulated packet/frame.  This additional information 
> >>> increases
> the
> >>>        dynamically discovering the maximum transmission unit (MTU) 
> >>> of
> an
> >>> * The text is weak for a Standard track document that is intended 
> >>> to solve the problem in https://tools.ietf.org/html/rfc7498#section-2.12.
> >>> behavior of the Classifier with regards to fragments for the sake 
> >>> of proper SFC operation. Applying classification policies may 
> >>> require the
> >> full packet, not only a fragment.
> >>> In particular, dedicated resources should dedicated for handling 
> >>> out of order fragments. Of course, it is out of scope of this 
> >>> document to describe how SFs handle fragments.
> >>>
> >>> * If an SFC header is prepended for all fragments, I'm not sure 
> >>> there is any particular issue at the SFF level...except if 
> >>> stripping/adding context TLVs depends on the full packet (not just 
> >>> fragment). It is warranted to consider a little bit this point 
> >>> before declaring there
> is
> >> no issue.
> >>>
> >>> implementing one of them is sufficient...which is not true. The 
> >>> first two points contribute to minimize the fragmentation risk, 
> >>> but fragmentation may still be experienced (e.g., other shims are 
> >>> * The support of jumbo frames by a router/device does not mean 
> >>> that it can make use of it. Appropriate MTU configuration should 
> >>> be undertaken in a consistent manner within an SFC domain. The 
> >>> text should be updated to make it is about (consistent) MTU configuration.
> >>>
> >>> * BTW, shouldn't the text be reworded to recommended to increase 
> >>> the MTU of **all nodes** of an SFC-enabled domain by at least the 
> >>> length of SFC header + transport header?
> >>>
> >>> * Bullet 2, how PMTU discovery is actually used in this context? 
> >>> Do you assume that all SFC-aware nodes will issue such messages 
> >>> towards other SFC-aware node, arbitrary destination, else?
> >>>
> >>> * The reference to the LISP specification raises two concerns and 
> >>> one
> >>> comment:
> >>>
> >>> (1) I don't think it is a good approach that fragments induced by 
> >>> the network are passed to their ultimate destinations as such (stateless).
> >>> The current text suggests that [RFC6830] should be listed as 
> >>> normative reference (not an informative one). I would personally 
> >>> favor removing the reference to LISP (as it is an Experimental RFC).
> >>>
> >>>> As I mentioned during the meeting, there are several issues that 
> >>>> are not covered in the last version of the draft. I already 
> >>>> provided examples of the issues offline as requested by Martin.
> >>>>
> >>>> Cheers,
> >>>> Med
> >>>>
> >>>>> De : sfc [mailto:[email-masked]] De la part de Thomas 
> >>>>> Narten Envoyé : jeudi 31 mars 2016 04:48 À : [email-masked] Objet 
> >>>>> : [sfc] WG last call for draft-ietf-sfc-nsh-04.txt
> >>>>>
> >>>>> Dear WG:
> >>>>>

---

From: <[email-masked]>
Date: Mon, 25 Apr 2016 14:40:26 +0000

Re-,

Please see inline.

Cheers,
Med

> Envoyé : lundi 25 avril 2016 16:25

[Med] Glad to see that we both agree that this is in scope. Unlike you, the current text needs more work IMHO (please refer to may detailed comments on this part).

> the packets it emits will end up getting NSH encapsulated.

[Med] We fully agree here. This is a what I called "the external behavior of the classifier function when it comes to handle fragments" in my reply to you. This should be cited as such in the specification. 

  The
> internal and not subject ot standardization.)

[Med] We are fully inline. That's why I said in my answer "The purpose is not to specify the internal implementation details".  

> So I do not see the disturbances you refer to.

[Med] If you don't have NSH encapsulation for all fragments, then disturbances will be experienced.

> > [Med] To be accurate, I'm asking to assess whether there are
> implications. If there are, then the draft should be updated accordingly.
> >
> > * SFF: If the NSH header is present only in the a fragment but SFF
> didn't maintained a state, subsequent fragments won't be appropriately
> processed.
> > [Med] The purpose is not to specify the internal implementation details
> but the external behavior of the classifier function when it comes to
> handle fragments. That behavior may have an incidence on SFF, in
> particular. The purpose is not to recommend the maximum resources to be
> dedicated to out of order fragments nor the timeout to cache those; these
> considerations are of course out of scope. Nevertheless, an implementation
> should offer a configurable parameter so that an operator tweak those
> according to its context.
> >
> > * it exacerbates fragmentation. Handing over this issue to the transport
> layer may lead to interoperability issues.
> I'm
> >> warranted to consider a little bit this point before declaring there is
> no
> >> issue.
> >>>
> >>>>> proper SFC operation. Applying classification policies may require
> the
> there
> first
> normative
> are

---

From: <[email-masked]>
Date: Mon, 25 Apr 2016 14:41:54 +0000

Re-,

How do you instruct the transport layer to ALWAYS prepend an NSH header even for fragments? Or you don't care about that?

Thank you.

Cheers,
Med

> Envoyé : lundi 25 avril 2016 16:26
> À : BOUCADAIR Mohamed IMT/OLN; Joel M. Halpern; [email-masked]
> [Med] To be accurate, I'm asking to assess whether there are implications.
> If there are, then the draft should be updated accordingly.
> 
> * SFF: If the NSH header is present only in the a fragment but SFF didn't
> maintained a state, subsequent fragments won't be appropriately processed.

---

From: Linda Dunbar <[email-masked]>
Date: Tue, 26 Apr 2016 15:10:02 +0000

Agree with Med.  

Even if each fragment piece of a packet with NSH header carries the NSH header, the intermediate SFF nodes still need to put together all the fragments together before passing the whole data frame to the SF.

Linda

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Monday, April 25, 2016 9:42 AM
To: Elzur, Uri; Joel M. Halpern; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Re-,

Thank you.

Cheers,
Med

> De : Elzur, Uri [mailto:[email-masked]] Envoyé : lundi 25 avril 
> 2016 16:26 À : BOUCADAIR Mohamed IMT/OLN; Joel M. Halpern; 
> Not to repeat my position, but I'll do it anyhow :-) As NSH is *NOT* a 
> transport, dealing with fragmentation is left to the Transport used.
> 
> > draft specify how service chaining should cope with packets that 
> > have been fragmented?
> >
> 
> didn't maintained a state, subsequent fragments won't be appropriately processed.
> [Med] The purpose is not to specify the internal implementation 
> details but the external behavior of the classifier function when it 
> comes to handle fragments. That behavior may have an incidence on SFF, 
> in particular. The purpose is not to recommend the maximum resources 
> to be dedicated to out of order fragments nor the timeout to cache 
> those; these considerations are of course out of scope. Nevertheless, 
> an implementation should offer a configurable parameter so that an 
> operator tweak those according to its context.
> 
> >   I suppose one could write an informational draft on possible ways 
> > of coping.  The IETF has not usually published such.
> * it exacerbates fragmentation. Handing over this issue to the 
> transport layer may lead to interoperability issues.
> > > There are other issues that need to be discussed, e.g., how to 
> > > deal with
> > to handle fragments. If an SFC header is prepended for all 
> > fragments, I'm not sure there
> > without NSH header as this is a decision that is left to the 
> > transport encapsulation. This is a requirement per se!
> > >
> > > I won't reiterate all the comments I have about the current 
> > > wording of
> > this section.
> > >
> > > Cheers,
> > > Med
> > >
> > >> is beyond the scope of NSH and beyond the charter of the WG, so 
> > >> it
> > doesn't
> > >> really belong in the draft. We are providing an advice as 
> > >> extending the size of the packet may lead to fragmentation, but 
> > >> as you check RFC 7348 VxLAN, which my create the same issue, 
> > >> you'll find it doesn't even
> > relate
> > >> to it.
> > >>
> > >> Thx
> > >>
> > >> Uri ("Oo-Ree")
> > >> C: 949-378-7568
> > >>
> > >>
> > >> That's another option that needs to be discussed/investigated. 
> > >> I'm
> > afraid
> > >> is transport-independent is not IMHO a good strategy to adopt 
> > >> here
> > because
> > >> sub-optimal implementations if the sfc information is present 
> > >> only in one
> > fragments,
> > >> etc.
> > >>
> > >>> De : Elzur, Uri [mailto:[email-masked]] Envoyé : dimanche 
> > >>> 24 avril 2016 17:36 À : BOUCADAIR Mohamed IMT/OLN; Thomas 
> > >>> Narten; [email-masked] Objet : RE: [sfc] WG last call for 
> > >>> independent i.e. like NSH interaction with any other Transport 
> > >>> eh issue of Fragmentation is to be dealt in a way that matches 
> > >>> the mechanisms supported by the Transport used and do not belong 
> > >>> in the NSH draft
> > >>>
> > >>> Thx
> > >>>
> > >>> Uri ("Oo-Ree")
> > >>> C: 949-378-7568
> > >>>
> > >>>
> > >>> In addition to other pending issues raised for this draft, I 
> > >>> would like to raise this additional one about Section 6.
> > >>>
> > >>> ==
> > >>>        dynamically discovering the maximum transmission unit 
> > >>> (MTU) of
> > an
> > >>>        arbitrary internet path" and can be utilized to ensure 
> > >>> the
> the
> > >>> * The text is weak for a Standard track document that is 
> > >>> intended to solve the problem in 
> > >>> https://tools.ietf.org/html/rfc7498#section-
> 2.12.
> > >>> received in an SFC domain. IMHO, the draft should also specify 
> > >>> the behavior of the Classifier with regards to fragments for the 
> > >>> sake of proper SFC operation. Applying classification policies 
> > >>> may require the
> > >>> stripping/adding context TLVs depends on the full packet (not 
> > >>> just fragment). It is warranted to consider a little bit this 
> > >>> point before declaring there
> > is
> > >> no issue.
> > >>>
> > >>> * The text states "several options". This may be interpreted as 
> > >>> if implementing one of them is sufficient...which is not true. 
> > >>> The first two points contribute to minimize the fragmentation 
> > >>> risk, but fragmentation may still be experienced (e.g., other 
> > >>> shims are prepended by other nodes for some other purposes, 
> > >>> nested nsh,
> > >>> etc.)
> > >>>
> > >>> the MTU of **all nodes** of an SFC-enabled domain by at least 
> > >>> the length of SFC header + transport header?
> > >>>
> > >>> * The reference to the LISP specification raises two concerns 
> > >>> and one
> > >>> comment:
> > >>>
> > >>> (1) I don't think it is a good approach that fragments induced 
> > >>> by the network are passed to their ultimate destinations as such
> (stateless).
> > >>> maintain a full list of MTU for the any SFF/SF instance of the 
> > >>> SFC
> > >> domain?
> > >>>
> > >>>> As I mentioned during the meeting, there are several issues 
> > >>>> that are not covered in the last version of the draft. I 
> > >>>> already provided examples of the issues offline as requested by Martin.
> > >>>>
> > >>>> Cheers,
> > >>>> Med
> > >>>>

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 26 Apr 2016 11:19:10 -0400

Linda, on what basis do you reach that conclusion?  Service functions 
have to be able to deal with fragmented IP packets.  I don't see any 
reason why we would expect SFF to reassemble packets, even in the rare 
case where service chaining forces the fragmentation.  (Note that such a 
case is generally actually indistinguishable from the origin fragmenting 
the packet, and it would be very odd for SFF to reassemble packets that 
the origin had fragmented.)

Yours,
Joel

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 26 Apr 2016 11:32:53 -0400

Re-reading your note, it is possible that you are referring only to the 
case of transport generated fragmentation of the outer packet.  I had 
assumed you were not talking about that, since the resulting fragments 
will not all have NSH headers.  As with any tunnel technology, if the 
tunnel chooses to fragment at its layer, then the tunnel is responsible 
for reassembly.  That would be invisible to the SFF.

Yours,
Joel

---

From: Linda Dunbar <[email-masked]>
Date: Tue, 26 Apr 2016 15:47:56 +0000

Joel, 

I think the document should add the description on the following two fragmentation scenarios:

- Transport tunnel generated fragmentation: 
  When a packet fragmentation is caused by transport tunnel (i.e. various encapsulations), the termination point of the transport tunnel is responsible for re-assembling the fragmented pieces of the packet. Since there won't be any SFF nodes in between the Transport Tunnel, the tunnel generated fragmentation does not require any actions by SFF nodes or SF nodes. 

- Source node generated fragmentation (after adding on the NSH header):
   When fragmentation has to be performed for a packet being encapsulated with the NSH header, either the intermediate SFF nodes or the SF nodes have to be able to reassemble the fragmented pieces. 

Cheers, 

Linda

-----Original Message-----
From: Joel M. Halpern [mailto:[email-masked]] 
Sent: Tuesday, April 26, 2016 10:33 AM
To: Linda Dunbar; [email-masked]; Elzur, Uri; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Re-reading your note, it is possible that you are referring only to the case of transport generated fragmentation of the outer packet.  I had assumed you were not talking about that, since the resulting fragments will not all have NSH headers.  As with any tunnel technology, if the tunnel chooses to fragment at its layer, then the tunnel is responsible for reassembly.  That would be invisible to the SFF.

Yours,
Joel

On 4/26/16 11:10 AM, Linda Dunbar wrote:
>> Not to repeat my position, but I'll do it anyhow :-) As NSH is *NOT* 
>> a transport, dealing with fragmentation is left to the Transport used.
>>
>> comes to handle fragments. That behavior may have an incidence on 
>> SFF, in particular. The purpose is not to recommend the maximum 
>> resources to be dedicated to out of order fragments nor the timeout 
>> to cache those; these considerations are of course out of scope. 
>> Nevertheless, an implementation should offer a configurable parameter 
>> so that an operator tweak those according to its context.
>>

---

From: Ron Parker <[email-masked]>
Date: Tue, 26 Apr 2016 15:58:09 +0000

Linda,

I like the first clause (transport tunnel based fragmentation).

Regarding the second (source based), I think there are more subcases to consider:

1.  Propagation of NSH encapsulated packet from classifier to SFF
2.  Propagation of NSH encapsulated packet from SFF to SF
3.  Propagation of NSH encapsulated packet from SF to SFF (including possibility that NSH size increased at SF)
4.  Propagation of NSH encapsulated packet from SFF to SFF

In each of the 4 cases above,  there is the possibility that MTU will be exceeded and therefore there are procedures to be defined in that eventuality.

Thanks.

    Ron

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Linda Dunbar
Sent: Tuesday, April 26, 2016 11:48 AM
To: Joel M. Halpern <[email-masked]>; [email-masked]; Elzur, Uri <[email-masked]>; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Joel, 

Cheers, 

Linda

-----Original Message-----
From: Joel M. Halpern [mailto:[email-masked]]
Sent: Tuesday, April 26, 2016 10:33 AM
To: Linda Dunbar; [email-masked]; Elzur, Uri; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Yours,
Joel

On 4/26/16 11:10 AM, Linda Dunbar wrote:
>>>>> extending the size of the packet may lead to fragmentation, but as 
>>>>> you check RFC 7348 VxLAN, which my create the same issue, you'll 
>>>>> find it doesn't even
>>> relate
>>>>> to it.
>>>>>
>>>>> Thx
>>>>>
>>>>> Uri ("Oo-Ree")
>>>>> C: 949-378-7568
>>>>>
>>>>>
>>>>>> 24 avril 2016 17:36 À : BOUCADAIR Mohamed IMT/OLN; Thomas Narten; 
>>>>>> to solve the problem in

---

From: Linda Dunbar <[email-masked]>
Date: Tue, 26 Apr 2016 16:13:13 +0000

Ron, 

That is very good addition. For the Source Based fragmentation, also need to add the description if the NSH header is added to each fragmented piece. 

Linda

-----Original Message-----
From: Ron Parker [mailto:[email-masked]] 
Sent: Tuesday, April 26, 2016 10:58 AM
To: Linda Dunbar; Joel M. Halpern; [email-masked]; Elzur, Uri; [email-masked]
Subject: RE: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Linda,

1.  Propagation of NSH encapsulated packet from classifier to SFF 2.  Propagation of NSH encapsulated packet from SFF to SF 3.  Propagation of NSH encapsulated packet from SF to SFF (including possibility that NSH size increased at SF) 4.  Propagation of NSH encapsulated packet from SFF to SFF

Thanks.

    Ron

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Linda Dunbar
Sent: Tuesday, April 26, 2016 11:48 AM
To: Joel M. Halpern <[email-masked]>; [email-masked]; Elzur, Uri <[email-masked]>; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Joel, 

Cheers, 

Linda

-----Original Message-----
From: Joel M. Halpern [mailto:[email-masked]]
Sent: Tuesday, April 26, 2016 10:33 AM
To: Linda Dunbar; [email-masked]; Elzur, Uri; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Yours,
Joel

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 26 Apr 2016 13:17:35 -0400

With regard to Transport tunnel fragementation, that seems an issue for
the transport protocol.  I don't actually object to a sentence, but it
does not seem that it will accomplish anything.

With regard to packets fragmented by the source, I completely disagree
with your assertion.  If an SFF were to reassemble the packets, that 
would be a violation of its job.  There is no reason for an SFF to 
reassemble a packet fragmented by the source.  The classifier may have 
to do some interesting things in order to properly classify succeeding 
fragments, but that is an implementation issue (most commonly addressed 
with virtual reassembly, which doe snot delay the fragments.)  We don't 
specity that.

If an SF needs to reassemble fragments to do its job, that is up to the 
SF.  Some will need to actually reassemble.  Some will need to perform 
virtual reassembly, and some will happily process the fragments.  I can 
not see what the NSH document could possibly mandate.

Yours,
Joel

On 4/26/16 11:47 AM, Linda Dunbar wrote:
> - Transport tunnel generated fragmentation: When a packet
> fragmentation is caused by transport tunnel (i.e. various
> encapsulations), the termination point of the transport tunnel is
> responsible for re-assembling the fragmented pieces of the packet.
> Since there won't be any SFF nodes in between the Transport Tunnel,
> the tunnel generated fragmentation does not require any actions by
> SFF nodes or SF nodes.
>
>
> - Source node generated fragmentation (after adding on the NSH
> header): When fragmentation has to be performed for a packet being
> encapsulated with the NSH header, either the intermediate SFF nodes
> or the SF nodes have to be able to reassemble the fragmented pieces.
>
>
>
>
> Cheers,
>
>
> Linda
>
> [mailto:[email-masked]] Sent: Tuesday, April 26, 2016 10:33 AM
> To: Linda Dunbar; [email-masked]; Elzur, Uri;
> [email-masked] Subject: Re: [sfc] WG last call for
> Re-reading your note, it is possible that you are referring only to
> the case of transport generated fragmentation of the outer packet.  I
> had assumed you were not talking about that, since the resulting
> fragments will not all have NSH headers.  As with any tunnel
> technology, if the tunnel chooses to fragment at its layer, then the
> tunnel is responsible for reassembly.  That would be invisible to the
> SFF.
>
> Yours, Joel
>
>> Even if each fragment piece of a packet with NSH header carries the
>> NSH header, the intermediate SFF nodes still need to put together
>> all the fragments together before passing the whole data frame to
>> the SF.
>>
>> Linda
>>
>> On Behalf Of [email-masked] Sent: Monday, April 25,
>> 2016 9:42 AM To: Elzur, Uri; Joel M. Halpern; [email-masked] Subject:
>> How do you instruct the transport layer to ALWAYS prepend an NSH
>> header even for fragments? Or you don't care about that?
>>
>> Thank you.
>>
>> Cheers, Med
>>
>>> -----Message d'origine----- De : Elzur, Uri
>>> [mailto:[email-masked]] Envoyé : lundi 25 avril 2016 16:26 À
>>> : BOUCADAIR Mohamed IMT/OLN; Joel M. Halpern; [email-masked] Objet
>>> : RE: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt
>>>
>>> Hi Med
>>>
>>> Not to repeat my position, but I'll do it anyhow :-) As NSH is
>>> *NOT* a transport, dealing with fragmentation is left to the
>>> Transport used.
>>>
>>> Uri ("Oo-Ree") C: 949-378-7568
>>>
>>>
>>> [mailto:[email-masked]] On Behalf Of
>>> [email-masked] Sent: Monday, April 25, 2016 7:18
>>> AM To: Joel M. Halpern <[email-masked]>; [email-masked]
>>> Cheers, Med
>>>> -----Message d'origine----- De : Joel M. Halpern
>>>> [mailto:[email-masked]] Envoyé : lundi 25 avril 2016 15:48
>>>> À : BOUCADAIR Mohamed IMT/OLN; [email-masked] Objet : Re: [sfc]
>>>> If I am understanding you correctly Med, you are asking that
>>>> the NSH draft specify how service chaining should cope with
>>>> packets that have been fragmented?
>>>>
>>>
>>> * SFF: If the NSH header is present only in the a fragment but
>>> SFF didn't maintained a state, subsequent fragments won't be
>>> appropriately processed. * SFC-aware function: if prepending a
>>> context information depends on the full packet, not only a
>>> fragment.
>>>
>>> It just does its job.
>>>
>>>> Ingress and intermediate classifiers may cope with fragments in
>>>> any number of ways.
>>> details but the external behavior of the classifier function when
>>> it comes to handle fragments. That behavior may have an incidence
>>> on SFF, in particular. The purpose is not to recommend the
>>> maximum resources to be dedicated to out of order fragments nor
>>> the timeout to cache those; these considerations are of course
>>> out of scope. Nevertheless, an implementation should offer a
>>> configurable parameter so that an operator tweak those according
>>> to its context.
>>>
>>>> I suppose one could write an informational draft on possible
>>>> ways of coping.  The IETF has not usually published such.
>>>> Service functions have to cope with fragmented packets just as
>>>> they had to before the advent of NSH, so describing that is
>>>> clearly not needed here.
>>>
>>> [Med] The advent of NSH has the following implications: * it
>>> exacerbates fragmentation. Handing over this issue to the
>>> transport layer may lead to interoperability issues. * the
>>> chaining may not be efficient if fragments are inappropriately
>>> handled.
>>>
>>>>> (1) Fragmentation that is caused by prepending an SFC
>>>>> header. (2) Handling fragments at the ingress of an
>>>>> SFC-enabled domain.
>>>>>
>>>>> There are other issues that need to be discussed, e.g., how
>>>>> to deal with
>>>> context TLVs depends on the full packet (not just fragment). It
>>>> is warranted to consider a little bit this point before
>>>> declaring there is no issue.
>>>>>
>>>> an implementation must be prepared to receive fragments with
>>>> or without NSH header as this is a decision that is left to
>>>> the transport encapsulation. This is a requirement per se!
>>>>>
>>>>> Cheers, Med
>>>>>> -----Message d'origine----- De : Elzur, Uri
>>>>>> [mailto:[email-masked]] Envoyé : lundi 25 avril 2016
>>>>>> 08:30 À : BOUCADAIR Mohamed IMT/OLN; Thomas Narten;
>>>>>> My point is that Fragmentation is yet another transport
>>>>>> related issue
>>>> that
>>>>>> is beyond the scope of NSH and beyond the charter of the
>>>>>> WG, so it
>>>> doesn't
>>>>>> extending the size of the packet may lead to fragmentation,
>>>>>> but as you check RFC 7348 VxLAN, which my create the same
>>>>>> issue, you'll find it doesn't even
>>>> relate
>>>>>> to it.
>>>>>>
>>>>>> Thx
>>>>>>
>>>>>> Uri ("Oo-Ree") C: 949-378-7568
>>>>>>
>>>>>>
>>>>>> [mailto:[email-masked]] On Behalf Of
>>>>>> [email-masked] Sent: Sunday, April 24, 2016
>>>>>> 10:32 PM To: Elzur, Uri <[email-masked]>; Thomas
>>>>>> Narten
>>>> <[email-masked]>;
>>>>>> [email-masked] Subject: Re: [sfc] WG last call for
>>>>>> That's another option that needs to be
>>>>>> discussed/investigated. I'm
>>>> afraid
>>>>>> this is not the rationale adopted in -04 since it includes
>>>>>> some text
>>>> that
>>>>>> BTW, saying that nsh does not need to deal with
>>>>>> fragmentation because
>>>> it
>>>>>> is transport-independent is not IMHO a good strategy to
>>>>>> adopt here
>>>> because
>>>>>> sub-optimal implementations if the sfc information is
>>>>>> present only in one
>>>> fragments,
>>>>>> etc.
>>>>>>
>>>>>> Cheers, Med
>>>>>>> -----Message d'origine----- De : Elzur, Uri
>>>>>>> [mailto:[email-masked]] Envoyé : dimanche 24 avril
>>>>>>> 2016 17:36 À : BOUCADAIR Mohamed IMT/OLN; Thomas Narten;
>>>>>>> I see no need to specify the exact behavior as NSH is
>>>>>>> transport independent i.e. like NSH interaction with any
>>>>>>> other Transport eh issue of Fragmentation is to be dealt
>>>>>>> in a way that matches the mechanisms supported by the
>>>>>>> Transport used and do not belong in the NSH draft
>>>>>>>
>>>>>>> Thx
>>>>>>>
>>>>>>> Uri ("Oo-Ree") C: 949-378-7568
>>>>>>>
>>>>>>>
>>>>>>> [mailto:[email-masked]] On Behalf Of
>>>>>>> [email-masked] Sent: Thursday, April 14,
>>>>>>> 2016 12:43 AM To: Thomas Narten <[email-masked]>;
>>>>>>> [email-masked] Subject: Re: [sfc] WG last call for
>>>>>>> In addition to other pending issues raised for this
>>>>>>> draft, I would like to raise this additional one about
>>>>>>> Section 6.
>>>>>>>
>>>>>>> == 6.  Fragmentation Considerations
>>>>>>>
>>>>>>> NSH and the associated transport header are "added" to
>>>>>>> the encapsulated packet/frame.  This additional
>>>>>>> information increases
>>>> the
>>>>>>> size of the packet.  In order the ensure proper
>>>>>>> forwarding of NSH data, several options for handling
>>>>>>> fragmentation and re-assembly exist:
>>>>>>>
>>>>>>> 1.  Jumbo Frames, when supported, enable the transport of
>>>>>>> NSH and associated transport packets without requiring
>>>>>>> fragmentation.
>>>>>>>
>>>>>>> 2.  Path MTU Discovery [RFC1191]"describes a technique
>>>>>>> for dynamically discovering the maximum transmission
>>>>>>> unit (MTU) of
>>>> an
>>>>>>> 3.  [RFC6830] describes two schemes for fragmentation and
>>>>>>> re-
>>>> assembly
>>>>>>> in section 5.4. ==
>>>>>>>
>>>>>>> * The text covers only fragmentation when it is induced
>>>>>>> by SFC operations, it does not discuss the treatment of a
>>>>>>> fragment when received in an SFC domain. IMHO, the draft
>>>>>>> should also specify the behavior of the Classifier with
>>>>>>> regards to fragments for the sake of proper SFC
>>>>>>> operation. Applying classification policies may require
>>>>>>> the
>>>>>>> In particular, dedicated resources should dedicated for
>>>>>>> handling out of order fragments. Of course, it is out of
>>>>>>> scope of this document to describe how SFs handle
>>>>>>> fragments.
>>>>>>>
>>>>>>> * If an SFC header is prepended for all fragments, I'm
>>>>>>> not sure there is any particular issue at the SFF
>>>>>>> level...except if stripping/adding context TLVs depends
>>>>>>> on the full packet (not just fragment). It is warranted
>>>>>>> to consider a little bit this point before declaring
>>>>>>> there
>>>> is
>>>>>> no issue.
>>>>>>>
>>>>>>> * The text states "several options". This may be
>>>>>>> interpreted as if implementing one of them is
>>>>>>> sufficient...which is not true. The first two points
>>>>>>> contribute to minimize the fragmentation risk, but
>>>>>>> fragmentation may still be experienced (e.g., other shims
>>>>>>> are prepended by other nodes for some other purposes,
>>>>>>> nested nsh, etc.)
>>>>>>>
>>>>>>> * The support of jumbo frames by a router/device does not
>>>>>>> mean that it can make use of it. Appropriate MTU
>>>>>>> configuration should be undertaken in a consistent manner
>>>>>>> within an SFC domain. The text should be updated to make
>>>>>>> * BTW, shouldn't the text be reworded to recommended to
>>>>>>> increase the MTU of **all nodes** of an SFC-enabled
>>>>>>> domain by at least the length of SFC header + transport
>>>>>>> header?
>>>>>>>
>>>>>>> * Bullet 2, how PMTU discovery is actually used in this
>>>>>>> context? Do you assume that all SFC-aware nodes will
>>>>>>> issue such messages towards other SFC-aware node,
>>>>>>> arbitrary destination, else?
>>>>>>>
>>>>>>> * Bullet 2, I would drop "describes a technique for
>>>>>>> (MTU) of an arbitrary internet path".
>>>>>>>
>>>>>>> (1) I don't think it is a good approach that fragments
>>>>>>> induced by the network are passed to their ultimate
>>>>>>> destinations as such
>>> (stateless).
>>>>>>> IMO, reassembly should be done within the SFC domain
>>>>>>> (responsible for fragmentation) not passed away. (2) Does
>>>>>>> the stateful mode require all SFC data plane elements
>>>>>>> maintain a full list of MTU for the any SFF/SF instance
>>>>>>> of the SFC
>>>>>> domain?
>>>>>>>
>>>>>>> The current text suggests that [RFC6830] should be listed
>>>>>>> as normative reference (not an informative one). I would
>>>>>>> personally favor removing the reference to LISP (as it is
>>>>>>> an Experimental RFC).
>>>>>>>
>>>>>>> * The security section of the draft may be augmented
>>>>>>> with informational fragmentation-related pointers to:
>>>>>>> e.g., RFC1858 (Security Considerations for IP Fragment
>>>>>>> Filtering), RFC3128 (Protection Against a Variant of the
>>>>>>> Tiny Fragment Attack), and RFC 4963 (IPv4 Reassembly
>>>>>>> Errors at High Data Rates).
>>>>>>>
>>>>>>> Cheers, Med
>>>>>>>
>>>>>>>> -----Message d'origine----- De : sfc
>>>>>>>> [mailto:[email-masked]] De la part de
>>>>>>>> [email-masked] Envoyé : lundi 11 avril
>>>>>>>> 2016 13:14 À : Thomas Narten; [email-masked] Objet : Re:
>>>>>>>> As I mentioned during the meeting, there are several
>>>>>>>> issues that are not covered in the last version of the
>>>>>>>> draft. I already provided examples of the issues
>>>>>>>> offline as requested by Martin.
>>>>>>>>
>>>>>>>> Cheers, Med
>>>>>>>>
>>>>>>>>> -----Message d'origine----- De : sfc
>>>>>>>>> [mailto:[email-masked]] De la part de Thomas
>>>>>>>>> Narten Envoyé : jeudi 31 mars 2016 04:48 À :
>>>>>>>>> [email-masked] Objet : [sfc] WG last call for
>>>>>>>>> addressed all known comments and that there are no
>>>>>>>>> Substantive comments to the list please, editorial
>>>>>>>>> comments can go directly to the document editors.
>>>>>>>>>
>>>>>>>>> We'll also get a brief update from the editors at
>>>>>>>>> next week's meeting. If there are any remaining
>>>>>>>>> issues with the document, raising them before the
>>>>>>>>> _______________________________________________ sfc
>>>>>>>>> mailing list [email-masked]
>>>>>>>> _______________________________________________ sfc
>>>>>>>> mailing list [email-masked]
>>>>>>> _______________________________________________ sfc
>>>>>>> mailing list [email-masked]
>>>>>> list [email-masked]

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 26 Apr 2016 13:21:40 -0400

At least so far, we have treated the communication between classifier 
and SFF (ingress or intermediate) as outside of the scope of our work. 
For SF <-> SFF communication we have ntoed the need for a proxy to 
handle the case when the SF can not accept and return the NSH header, 
since we assume that is preserved.  Other than that preservation, 
similar to the above, we have treated the details as outside of our 
scope. That is because they depend so heavily on the implementation 
technique used.  (Hypervisor ,_> application communication vs 
communication inside a user space or communication over an actual 
Ethernet are all very different.)

Trying to extend the NSH document to address these would be a major 
change, and would step into the issues related to transport selection, 
which are explicitly out of scope.

Yours,
Joel

---

From: Ron Parker <[email-masked]>
Date: Tue, 26 Apr 2016 18:38:55 +0000

Joel,

I can't say that I disagree with your analysis on the scope.   I think the question raised by some on the thread is if an SFC-node (i.e., classifier, SF, SFF) creates fragments, is the NSH header repeated in each fragment ?    I think the answer is "no".   If so, is it worth stating this explicitly in the document, or is it just obvious that properly layered systems would behave that way.

   Ron

-----Original Message-----
From: Joel M. Halpern [mailto:[email-masked]] 
Sent: Tuesday, April 26, 2016 1:22 PM
To: Ron Parker <[email-masked]>; Joel M. Halpern <[email-masked]>; [email-masked]; Elzur, Uri <[email-masked]>; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

At least so far, we have treated the communication between classifier and SFF (ingress or intermediate) as outside of the scope of our work. 
For SF <-> SFF communication we have ntoed the need for a proxy to handle the case when the SF can not accept and return the NSH header, since we assume that is preserved.  Other than that preservation, similar to the above, we have treated the details as outside of our scope. That is because they depend so heavily on the implementation technique used.  (Hypervisor ,_> application communication vs communication inside a user space or communication over an actual Ethernet are all very different.)

Trying to extend the NSH document to address these would be a major change, and would step into the issues related to transport selection, which are explicitly out of scope.

Yours,
Joel

On 4/26/16 11:58 AM, Ron Parker wrote:
> 1.  Propagation of NSH encapsulated packet from classifier to SFF 2.  
> Propagation of NSH encapsulated packet from SFF to SF 3.  Propagation 
> of NSH encapsulated packet from SF to SFF (including possibility that 
> NSH size increased at SF) 4.  Propagation of NSH encapsulated packet 
> from SFF to SFF
>
> To: Joel M. Halpern <[email-masked]>; 
> [email-masked]; Elzur, Uri <[email-masked]>; 
> [email-masked]
>>>> If I am understanding you correctly Med, you are asking that the 
>>>> NSH draft specify how service chaining should cope with packets 
>>>> that have been fragmented?
>>>>
>>>
>>> Nevertheless, an implementation should offer a configurable 
>>> parameter so that an operator tweak those according to its context.
>>>
>>>>>>> (responsible for

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 26 Apr 2016 15:12:11 -0400

In most tunnel anlysis, a tunnel entry point (in this case, the 
transport interface from the SFC component) can either fragment outer 
or, if available fragment inner.  This is defined by the transport 
technology, not the user.

In the case of SFC NSH, I would usually expect outer fragmentation, 
which is a pure transport issue.  If the innermost packet is IP, and the 
tunnel entry wants to be complicated, it is permitted to fragment the 
inner packet and produce the proper NSH headers for the fragments. 
Inherently, an implementation can only choose to do this if it can 
construct the right metdata to put on each fragment.  I don't actually 
know how to reliably do that, so I don't recommend it.

In either case, the NSH document can't tell the implementation what to 
do or how to do it.

Yours,
Joel

---

From: Linda Dunbar <[email-masked]>
Date: Tue, 26 Apr 2016 19:20:32 +0000

Joel,

Replies are inserted below:

-----Original Message-----
From: Joel M. Halpern [mailto:[email-masked]]
Sent: Tuesday, April 26, 2016 12:18 PM
To: Linda Dunbar; [email-masked]; Elzur, Uri; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

With regard to Transport tunnel fragementation, that seems an issue for the transport protocol.  I don't actually object to a sentence, but it does not seem that it will accomplish anything.

[Linda] By having this description, you avoid a lot of future unnecessary debate about how fragmentation should be processed because some people are only thinking about Tunnel induced fragmentation where as some other people are thinking of Source induced fragmentation.

With regard to packets fragmented by the source, I completely disagree with your assertion.  If an SFF were to reassemble the packets, that would be a violation of its job.  There is no reason for an SFF to reassemble a packet fragmented by the source.  The classifier may have to do some interesting things in order to properly classify succeeding fragments, but that is an implementation issue (most commonly addressed with virtual reassembly, which doe snot delay the fragments.)  We don't specity that.

If an SF needs to reassemble fragments to do its job, that is up to the SF.  Some will need to actually reassemble.  Some will need to perform virtual reassembly, and some will happily process the fragments.  I can not see what the NSH document could possibly mandate.

[Linda] I said "either SFF or SF" has to re-assemble the fragmented packets. I don't have strong opinion on who perform the re-assembling job (as long as one of them do). If a set of SFs attached to a SFF are not capable of performing the re-assembling and they need the whole packet to do inspection, then the SFF don't have any choice.  I just think that it is necessary to add the description because it can be very messy (if not impossible) for SFs to process partial packets.

Linda

Yours,
Joel

On 4/26/16 11:47 AM, Linda Dunbar wrote:
> the tunnel generated fragmentation does not require any actions by SFF
> nodes or SF nodes.
>
>
> encapsulated with the NSH header, either the intermediate SFF nodes or
> the SF nodes have to be able to reassemble the fragmented pieces.
>
>
>
>
> Cheers,
>
>
> Linda
>
> To: Linda Dunbar; [email-masked]<mailto:[email-masked]>; Elzur, Uri;
> [email-masked]<mailto:[email-masked]> Subject: Re: [sfc] WG last call for
>> NSH header, the intermediate SFF nodes still need to put together all
>> the fragments together before passing the whole data frame to the SF.
>>
>> Linda
>>
>> Behalf Of [email-masked]<mailto:[email-masked]> Sent: Monday, April 25,
>> 2016 9:42 AM To: Elzur, Uri; Joel M. Halpern; [email-masked]<mailto:[email-masked]> Subject:
>>> : BOUCADAIR Mohamed IMT/OLN; Joel M. Halpern; [email-masked]<mailto:[email-masked]> Objet
>>> On Behalf Of [email-masked]<mailto:[email-masked]> Sent: Monday, April 25,
>>> 2016 7:18 AM To: Joel M. Halpern <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
>>>> [mailto:[email-masked]] Envoyé : lundi 25 avril 2016 15:48 À :
>>>> BOUCADAIR Mohamed IMT/OLN; [email-masked]<mailto:[email-masked]> Objet : Re: [sfc] WG last
>>> didn't maintained a state, subsequent fragments won't be
>>> context information depends on the full packet, not only a fragment.
>>>
>>> It just does its job.
>>>
>>> exacerbates fragmentation. Handing over this issue to the transport
>>> layer may lead to interoperability issues. * the chaining may not be
>>> efficient if fragments are inappropriately handled.
>>>
>>>> On 4/25/16 3:00 AM, [email-masked]<mailto:[email-masked]> wrote:
>>>>> Re-,
>>>>>
>>>>> (1) Fragmentation that is caused by prepending an SFC header. (2)
>>>>> Handling fragments at the ingress of an SFC-enabled domain.
>>>>>
>>>>>> 08:30 À : BOUCADAIR Mohamed IMT/OLN; Thomas Narten; [email-masked]<mailto:[email-masked]>
>>>>>> [email-masked]<mailto:[email-masked]> Sent: Sunday, April 24, 2016
>>>>>> 10:32 PM To: Elzur, Uri <[email-masked]<mailto:[email-masked]>>; Thomas Narten
>>>> <[email-masked]<mailto:[email-masked]>>;
>>>>>> [email-masked]<mailto:[email-masked]> Subject: Re: [sfc] WG last call for
>>>>>>> [email-masked]<mailto:[email-masked]> Objet : RE: [sfc] WG last call for
>>>>>>> [email-masked]<mailto:[email-masked]> Sent: Thursday, April 14,
>>>>>>> 2016 12:43 AM To: Thomas Narten <[email-masked]<mailto:[email-masked]>>;
>>>>>>> [email-masked]<mailto:[email-masked]> Subject: Re: [sfc] WG last call for
>>>>>>> size of the packet.  In order the ensure proper forwarding of
>>>>>>> NSH data, several options for handling fragmentation and
>>>>>>> re-assembly exist:
>>>>>>>
>>>>>>> 1.  Jumbo Frames, when supported, enable the transport of NSH
>>>>>>> and associated transport packets without requiring
>>>>>>> fragmentation.
>>>>>>>
>>>>>>> discovering the maximum transmission unit
>>>>>>> and one comment:
>>>>>>> (responsible for fragmentation) not passed away. (2) Does the
>>>>>>> stateful mode require all SFC data plane elements maintain a
>>>>>>> informational fragmentation-related pointers to:
>>>>>>> Filtering), RFC3128 (Protection Against a Variant of the Tiny
>>>>>>> Fragment Attack), and RFC 4963 (IPv4 Reassembly Errors at High
>>>>>>> Data Rates).
>>>>>>>
>>>>>>> Cheers, Med
>>>>>>>
>>>>>>>> [email-masked]<mailto:[email-masked]> Envoyé : lundi 11 avril
>>>>>>>> 2016 13:14 À : Thomas Narten; [email-masked]<mailto:[email-masked]> Objet : Re:
>>>>>>>>> [mailto:[email-masked]] De la part de Thomas Narten
>>>>>>>>> Envoyé : jeudi 31 mars 2016 04:48 À :
>>>>>>>>> [email-masked]<mailto:[email-masked]> Objet : [sfc] WG last call for
>>>>>>>>> list [email-masked]<mailto:[email-masked]> https://www.ietf.org/mailman/listinfo/sfc
>>>>>>>>
>>>>>>>> list [email-masked]<mailto:[email-masked]> https://www.ietf.org/mailman/listinfo/sfc
>>>>>>>
>>>>>>> [email-masked]<mailto:[email-masked]> https://www.ietf.org/mailman/listinfo/sfc
>>>>>>
>>>>>> [email-masked]<mailto:[email-masked]> https://www.ietf.org/mailman/listinfo/sfc
>>>>>
>>>>> [email-masked]<mailto:[email-masked]> https://www.ietf.org/mailman/listinfo/sfc
>>>>>
>>>
>>> [email-masked]<mailto:[email-masked]> https://www.ietf.org/mailman/listinfo/sfc
>>
>> [email-masked]<mailto:[email-masked]> https://www.ietf.org/mailman/listinfo/sfc
>>

---

From: Dave Dolson <[email-masked]>
Date: Tue, 26 Apr 2016 19:55:22 +0000

Fragment reassembly is a computationally expensive process
and tunnel fragmentation can substantially inflate the number of 
packets, so I prefer fragmenting the inner packet.
This allows SFFs and SFs to forward NSH packets without first reassembling
them.

If the inner packet is IPv6 or IPv4 with DF=1, Path-MTU discovery
can be facilitated by sending an ICMP "Packet too big" message to 
the sender. (RFC 1191, RFC 1981)

There is an additional benefit that inner fragmentation/PMTU discovery
allow service chaining to work even if the transport layer does not 
facilitate fragmentation (e.g., when NSH is transported directly over 
Ethernet).

The fragmentation and Path MTU discovery are standard mechanisms,
and permitted to be used for any link, so nothing needs to be
said in the NSH draft to permit this. However, I think it would
be useful to recommend these approaches.

-Dave

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Joel M. Halpern
Sent: Tuesday, April 26, 2016 3:12 PM
To: Ron Parker; [email-masked]; Elzur, Uri; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Yours,
Joel

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 26 Apr 2016 16:52:54 -0400

You second insertion below seems to assume that service functions 
require reassembled packets.  That is not generally true.  I am sure 
that there are some service functions that require that.  But most of 
them either:
1) do not require reassembly at all;
2) can get by with virtual reassembly where they preserve needed state 
between fragments;
3) or, even more rarely, can get by with reassembling the first and 
second packet, and then applying stateful processing thereafter.

Yours,
Joel

On 4/26/16 3:20 PM, Linda Dunbar wrote:
> [Linda] By having this description, you avoid a lot of future
> unnecessary debate about how fragmentation should be processed because
> some people are only thinking about Tunnel induced fragmentation where
> as some other people are thinking of Source induced fragmentation.
>
>
> [Linda] I said “either SFF or SF” has to re-assemble the fragmented
> packets. I don’t have strong opinion on who perform the re-assembling
> job (as long as one of them do). If a set of SFs attached to a SFF are
> not capable of performing the re-assembling and they need the whole
> packet to do inspection, then the SFF don’t have any choice. I just
> think that it is necessary to add the description because it can be very
> messy (if not impossible) for SFs to process partial packets.
>
>
> Linda
>
>
>
>
> Yours,
> Joel
>
>> To: Linda Dunbar; [email-masked] <mailto:[email-masked]>;
> Elzur, Uri;
>> [email-masked] <mailto:[email-masked]> Subject: Re: [sfc] WG last call for
>>> Behalf Of [email-masked] <mailto:[email-masked]> Sent:
> Monday, April 25,
>>> 2016 9:42 AM To: Elzur, Uri; Joel M. Halpern; [email-masked] <mailto:[email-masked]> Subject:
>>>> : BOUCADAIR Mohamed IMT/OLN; Joel M. Halpern; [email-masked] <mailto:[email-masked]> Objet
>>>> On Behalf Of [email-masked] <mailto:[email-masked]> Sent:
> Monday, April 25,
>>>> 2016 7:18 AM To: Joel M. Halpern <[email-masked] <mailto:[email-masked]>>; [email-masked]
> <mailto:[email-masked]>
>>>>> BOUCADAIR Mohamed IMT/OLN; [email-masked] <mailto:[email-masked]> Objet : Re: [sfc] WG last
>>>>> On 4/25/16 3:00 AM, [email-masked] <mailto:[email-masked]> wrote:
>>>>>> Re-,
>>>>>>
>>>>>>> 08:30 À : BOUCADAIR Mohamed IMT/OLN; Thomas Narten; [email-masked] <mailto:[email-masked]>
>>>>>>> [email-masked] <mailto:[email-masked]> Sent:
> Sunday, April 24, 2016
>>>>>>> 10:32 PM To: Elzur, Uri <[email-masked] <mailto:[email-masked]>>; Thomas Narten
>>>>> <[email-masked] <mailto:[email-masked]>>;
>>>>>>> [email-masked] <mailto:[email-masked]> Subject: Re: [sfc] WG last call for
>>>>>>>> [email-masked] <mailto:[email-masked]> Objet : RE: [sfc] WG last call for
>>>>>>>> [email-masked] <mailto:[email-masked]> Sent:
> Thursday, April 14,
>>>>>>>> 2016 12:43 AM To: Thomas Narten <[email-masked] <mailto:[email-masked]>>;
>>>>>>>> [email-masked] <mailto:[email-masked]> Subject: Re: [sfc] WG last call for
>>>>>>>>> [email-masked] <mailto:[email-masked]>
> Envoyé : lundi 11 avril
>>>>>>>>> 2016 13:14 À : Thomas Narten; [email-masked] <mailto:[email-masked]> Objet : Re:
>>>>>>>>>> [email-masked] <mailto:[email-masked]> Objet : [sfc] WG last call for
>>>>>>>>>> list [email-masked] <mailto:[email-masked]> https://www.ietf.org/mailman/listinfo/sfc
>>>>>>>>>
>>>>>>>>> list [email-masked] <mailto:[email-masked]> https://www.ietf.org/mailman/listinfo/sfc
>>>>>>>>
>>>>>>>> [email-masked] <mailto:[email-masked]> https://www.ietf.org/mailman/listinfo/sfc
>>>>>>>
>>>>>>> [email-masked] <mailto:[email-masked]> https://www.ietf.org/mailman/listinfo/sfc
>>>>>>
>>>>>> [email-masked] <mailto:[email-masked]> https://www.ietf.org/mailman/listinfo/sfc
>>>>>>
>>>>
>>>> [email-masked] <mailto:[email-masked]> https://www.ietf.org/mailman/listinfo/sfc
>>>
>>> [email-masked] <mailto:[email-masked]> https://www.ietf.org/mailman/listinfo/sfc
>>>
>

---

From: Linda Dunbar <[email-masked]>
Date: Tue, 26 Apr 2016 21:32:15 +0000

Joel, 

I agree that some SFs don't need to re-assemble fragmented packets. But I am also aware of SFs that need to process the whole packets. 
It would be helpful to describe your assumptions in the draft

Linda
-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Joel M. Halpern
Sent: Tuesday, April 26, 2016 3:53 PM
To: Linda Dunbar; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

You second insertion below seems to assume that service functions require reassembled packets.  That is not generally true.  I am sure that there are some service functions that require that.  But most of them either:
1) do not require reassembly at all;
2) can get by with virtual reassembly where they preserve needed state between fragments;
3) or, even more rarely, can get by with reassembling the first and second packet, and then applying stateful processing thereafter.

Yours,
Joel

On 4/26/16 3:20 PM, Linda Dunbar wrote:
> With regard to Transport tunnel fragementation, that seems an issue 
> for the transport protocol.  I don't actually object to a sentence, 
> but it does not seem that it will accomplish anything.
>
> fragments, but that is an implementation issue (most commonly 
> addressed with virtual reassembly, which doe snot delay the 
> fragments.)  We don't specity that.
>
> If an SF needs to reassemble fragments to do its job, that is up to 
> the SF.  Some will need to actually reassemble.  Some will need to 
> perform virtual reassembly, and some will happily process the 
> fragments.  I can not see what the NSH document could possibly mandate.
>
> [Linda] I said "either SFF or SF" has to re-assemble the fragmented 
> packets. I don't have strong opinion on who perform the re-assembling 
> packet to do inspection, then the SFF don't have any choice. I just 
> think that it is necessary to add the description because it can be 
> very messy (if not impossible) for SFs to process partial packets.
>
>
> Linda
>
>
>
>
> Yours,
> Joel
>
>> [email-masked] <mailto:[email-masked]> Subject: Re: [sfc] WG last call 
>>> all the fragments together before passing the whole data frame to the SF.
>>>
>>> Linda
>>>
>>>> 2016 7:18 AM To: Joel M. Halpern <[email-masked] 
>>>> <mailto:[email-masked]>>; [email-masked]
> <mailto:[email-masked]>
>>>>> BOUCADAIR Mohamed IMT/OLN; [email-masked] <mailto:[email-masked]> 
>>>> on SFF, in particular. The purpose is not to recommend the maximum 
>>>> layer may lead to interoperability issues. * the chaining may not 
>>>> be efficient if fragments are inappropriately handled.
>>>>
>>>>>>> 08:30 À : BOUCADAIR Mohamed IMT/OLN; Thomas Narten; [email-masked] 
>>>>>>> <mailto:[email-masked]> Objet : RE: [sfc] WG last call for 
>>>>>>> 10:32 PM To: Elzur, Uri <[email-masked] 
>>>>>>> <mailto:[email-masked]>>; Thomas Narten
>>>>>>> [email-masked] <mailto:[email-masked]> Subject: Re: [sfc] WG last 
>>>>>>>> [email-masked] <mailto:[email-masked]> Objet : RE: [sfc] WG last 
>>>>>>>> the mechanisms supported by the Transport used and do not 
>>>>>>>> belong in the NSH draft
>>>>>>>>
>>>>>>>> Thx
>>>>>>>>
>>>>>>>> 2016 12:43 AM To: Thomas Narten <[email-masked] 
>>>>>>>> <mailto:[email-masked]>>; [email-masked] <mailto:[email-masked]> 
>>>>>>>> operations, it does not discuss the treatment of a fragment 
>>>>>>>> when received in an SFC domain. IMHO, the draft should also 
>>>>>>>> specify the behavior of the Classifier with regards to 
>>>>>>>> fragments for the sake of proper SFC operation. Applying 
>>>>>>>> handling out of order fragments. Of course, it is out of scope 
>>>>>>>> of this document to describe how SFs handle fragments.
>>>>>>>>
>>>>>>>> that it can make use of it. Appropriate MTU configuration 
>>>>>>>> should be undertaken in a consistent manner within an SFC 
>>>>>>>> domain. The text should be updated to make it is about 
>>>>>>>> (consistent) MTU
>>>> configuration.
>>>>>>>>
>>>>>>>> increase the MTU of **all nodes** of an SFC-enabled domain by 
>>>>>>>> at least the length of SFC header + transport header?
>>>>>>>>
>>>>>>>> normative reference (not an informative one). I would 
>>>>>>>> personally favor removing the reference to LISP (as it is an 
>>>>>>>> Experimental RFC).
>>>>>>>>
>>>>>>>>> <mailto:[email-masked]>
> Envoyé : lundi 11 avril
>>>>>>>>>> [email-masked] <mailto:[email-masked]> Objet : [sfc] WG last call 
>>>>>>>>>> addressed all known comments and that there are no open 
>>>>>>>>>> issues with the current version of the document.
>>>>>>>>>>
>>>>>>>>>> list [email-masked] <mailto:[email-masked]> 
>>>>>>>>> list [email-masked] <mailto:[email-masked]> 
>>>>>>>> list [email-masked] <mailto:[email-masked]>

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 26 Apr 2016 17:42:54 -0400

The draft assumes that SFF are network elements.  Network elements do 
not normally reassemble packets not addressed to themselves.  As such, 
the assumption you refer to is the norm.

Yes, some SF need to reassemble packets.  We presume that functions 
which need to do somethign will do so.  This is not something that 
changes by adding SFC or NSH to service chaining or even to non-chaining 
transparent in-line service delivery.

I have trouble seeing how a sentence such as "Service Functions which 
require fully reassembled transport packets are responsible for such 
reassembly" would add value.  I also don't see how it could go in the 
NSH document, as it has nothing to do with NSH.

Yours,
Joel

---

From: Linda Dunbar <[email-masked]>
Date: Tue, 26 Apr 2016 22:22:25 +0000

Joel, 

Where is a better place to describe how to deal with the Fragmentation induced by NSH header than the NSF document? 

The NSH header induced fragmentation is very different from Tunnel encapsulation induced fragmentation in the following aspects:

- Tunnel induced fragmented pieces of a packet will not be processed by any intermediate nodes other than forwarding. As all fragmented pieces have the Dest/Source, all the intermediate nodes can forward based on the Dst/Src. 

- NSH induced fragmented pieces of a packet will be processed by SF in the middle. Some of the SFs do need to inspect the whole packet or multiple fragmented pieces --> warrant partial or whole re-assembling. 

Therefore, I disagree with your statement of "NSH header induced fragmentation has nothing to do with NSH". 

Linda 

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Joel M. Halpern
Sent: Tuesday, April 26, 2016 4:43 PM
To: Linda Dunbar; [email-masked]
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

The draft assumes that SFF are network elements.  Network elements do not normally reassemble packets not addressed to themselves.  As such, the assumption you refer to is the norm.

Yes, some SF need to reassemble packets.  We presume that functions which need to do somethign will do so.  This is not something that changes by adding SFC or NSH to service chaining or even to non-chaining transparent in-line service delivery.

I have trouble seeing how a sentence such as "Service Functions which require fully reassembled transport packets are responsible for such reassembly" would add value.  I also don't see how it could go in the NSH document, as it has nothing to do with NSH.

Yours,
Joel

On 4/26/16 5:32 PM, Linda Dunbar wrote:
>> unnecessary debate about how fragmentation should be processed 
>> because some people are only thinking about Tunnel induced 
>> fragmentation where as some other people are thinking of Source induced fragmentation.
>>
>>
>> With regard to packets fragmented by the source, I completely 
>> disagree with your assertion.  If an SFF were to reassemble the 
>> packets, that would be a violation of its job.  There is no reason 
>> for an SFF to reassemble a packet fragmented by the source.  The 
>> classifier may have to do some interesting things in order to 
>> properly classify succeeding fragments, but that is an implementation 
>> issue (most commonly addressed with virtual reassembly, which doe 
>> snot delay the
>> job (as long as one of them do). If a set of SFs attached to a SFF 
>> are not capable of performing the re-assembling and they need the 
>> whole packet to do inspection, then the SFF don't have any choice. I 
>> just think that it is necessary to add the description because it can 
>> be very messy (if not impossible) for SFs to process partial packets.
>>
>>
>> Linda
>>
>>
>>
>>
>> Yours,
>> Joel
>>
>>> the case of transport generated fragmentation of the outer packet.  
>>> I had assumed you were not talking about that, since the resulting 
>>> tunnel is responsible for reassembly.  That would be invisible to 
>>> the SFF.
>>>
>>> Yours, Joel
>>>
>>>>> resources to be dedicated to out of order fragments nor the 
>>>>> timeout to cache those; these considerations are of course out of scope.
>>>>> chaining may not be efficient if fragments are inappropriately handled.
>>>>>
>>>>>> is warranted to consider a little bit this point before declaring 
>>>>>> there is no issue.
>>>>>>>
>>>>>>>>> transport independent i.e. like NSH interaction with any other 
>>>>>>>>> Transport eh issue of Fragmentation is to be dealt in a way 
>>>>>>>>> that matches the mechanisms supported by the Transport used 
>>>>>>>>> and do not belong in the NSH draft
>>>>>>>>>
>>>>>>>>> Thx
>>>>>>>>>
>>>>>>>>> * If an SFC header is prepended for all fragments, I'm not 
>>>>>>>>> sure there is any particular issue at the SFF level...except 
>>>>>>>>> if stripping/adding context TLVs depends on the full packet 
>>>>>>>>> (not just fragment). It is warranted to consider a little bit 
>>>>>>>>> this point before declaring there
>>>>>> is
>>>>>>>> no issue.
>>>>>>>>>
>>>>>>>>> * The text states "several options". This may be interpreted 
>>>>>>>>> as if implementing one of them is sufficient...which is not true.
>>>>>>>>> Do you assume that all SFC-aware nodes will issue such 
>>>>>>>>> messages towards other SFC-aware node, arbitrary destination, else?
>>>>>>>>>
>>>>>>>>>>> [email-masked] <mailto:[email-masked]> Objet : [sfc] WG last 
>>>>>>>>>>> document, raising them before the meeting would be especially helpful.
>>>>>>>>>>>
>>>>>>>>>>> For the chairs, Thomas
>>>>>>>>>>>

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 26 Apr 2016 18:29:07 -0400

That is not what I said.
There are two different cases.  One is Transport fragmentation.
And one is source fragmentation.

You argued that source fragmentation needs to be discussed in the NSH 
document.  I pointed out that the requirement you claim is not 
wide-spread, and that for the cases where it exists, it exists before 
SFC.  So that does not seem to be an NSH problem at all.

With regard to Tunnel fragmentation, the question of where to describe 
all the possible issues is more complex.  There is text in section 6 of 
the document.  Beyond that, the problem is that the mechanism needed 
depends upon the underlying packet and the transport technology.  It 
also depends upon whether the service chaining environment can enable a 
larger MTU, and thereby avoid all of the problems.  I will grant that 
this problem is not unrelated to NSH.  However, given our requirement to 
be transport agnostic, I am confused as to why anything more than what 
is stated in section 6 is needed.

Yours,
Joel

---

From: <[email-masked]>
Date: Wed, 27 Apr 2016 05:39:50 +0000

Hi Joel, all,

Please see inline. 

Cheers,
Med

> Envoyé : mardi 26 avril 2016 19:18
> À : Linda Dunbar; BOUCADAIR Mohamed IMT/OLN; Elzur, Uri; [email-masked]

[Med] I would like to see some text added to the draft.

> would be a violation of its job.

[Med] I agree with you.

  There is no reason for an SFF to
> with virtual reassembly, which doe snot delay the fragments.)  We don't

[Med] Still, the external behavior of the classifier needs to be clear in the document. I don't find any text in the draft saying for instance that an NSH header must be present in all fragments. (There some processing that might be needed at the SFF to "do its job" with regards to fragments (receive out of order fragments + forwarding policy on the full packet).)

> not see what the NSH document could possibly mandate.

[Med] Fully agree.

---

From: <[email-masked]>
Date: Wed, 27 Apr 2016 05:46:05 +0000

Re-,

Please see inline.

Cheers,
Med

> Envoyé : mardi 26 avril 2016 19:22
> À : Ron Parker; Joel M. Halpern; BOUCADAIR Mohamed IMT/OLN; Elzur, Uri;
> [email-masked]

[Med] I'm afraid I don't get this one. Isn't this what is depicted in https://tools.ietf.org/html/rfc7665#section-4? 

> Ethernet are all very different.)

[Med] Specifying the external behavior of the interface between an SFF and an SFC-aware SF is completely in scope, imho. 

> > 3.  Propagation of NSH encapsulated packet from SF to SFF (including
> possibility that NSH size increased at SF)
> > In each of the 4 cases above,  there is the possibility that MTU will be
> exceeded and therefore there are procedures to be defined in that
> eventuality.
> >
> > Thanks.
> >
> >     Ron
> >
> >
> > To: Joel M. Halpern <[email-masked]>; [email-masked];
> Elzur, Uri <[email-masked]>; [email-masked]
> >   When a packet fragmentation is caused by transport tunnel (i.e.
> various encapsulations), the termination point of the transport tunnel is
> responsible for re-assembling the fragmented pieces of the packet. Since
> there won't be any SFF nodes in between the Transport Tunnel, the tunnel
> generated fragmentation does not require any actions by SFF nodes or SF
> nodes.
> >
> >
> >    When fragmentation has to be performed for a packet being
> encapsulated with the NSH header, either the intermediate SFF nodes or the
> SF nodes have to be able to reassemble the fragmented pieces.
> >
> >
> >
> > Cheers,
> >
> >
> > Linda
> >
> >> Even if each fragment piece of a packet with NSH header carries the NSH
> header, the intermediate SFF nodes still need to put together all the
> fragments together before passing the whole data frame to the SF.
> >>
> >> Linda
> >>
> >> How do you instruct the transport layer to ALWAYS prepend an NSH header
> even for fragments? Or you don't care about that?
> >>
> >> Thank you.
> >>
> >> Cheers,
> >> Med
> >>
> implications.
> NSH
> and

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 27 Apr 2016 10:03:57 -0400

On the Classifier<->SFF communication, you have a fiar point.  My
personal model is off in that regard.  My apologies.

I believe that the assumption there is that just as with an SFF, it will
use a local means to couple to a transport encapsulator.  I tend to
think of this as putting an SFF co-located with the classifier, and then
using SFF functionality, but that is not required by the architecture.

The external behavior of the SFF,_>SF interface, where the SF speaks 
NSH, is that an NSH encapsulated frame is communicated between the two. 
The transport mechanism (which may need to fragment) is out of scope.

Yours,
Joel

On 4/27/16 1:46 AM, [email-masked] wrote:
>> [mailto:[email-masked]] Envoyé : mardi 26 avril 2016 19:22 À :
>> Ron Parker; Joel M. Halpern; BOUCADAIR Mohamed IMT/OLN; Elzur,
>> Uri; [email-masked] Objet : Re: [sfc] WG last call for
>> At least so far, we have treated the communication between
>> classifier and SFF (ingress or intermediate) as outside of the
>> scope of our work.
>
> [Med] I'm afraid I don't get this one. Isn't this what is depicted in
> https://tools.ietf.org/html/rfc7665#section-4?
>
>> handle the case when the SF can not accept and return the NSH
>> header, since we assume that is preserved.  Other than that
>> preservation, similar to the above, we have treated the details as
>> outside of our scope. That is because they depend so heavily on the
>> implementation technique used.  (Hypervisor ,_> application
>> communication vs communication inside a user space or communication
>> over an actual Ethernet are all very different.)
>
> [Med] Specifying the external behavior of the interface between an
> SFF and an SFC-aware SF is completely in scope, imho.
>
>>
>> Trying to extend the NSH document to address these would be a
>> major change, and would step into the issues related to transport
>> selection, which are explicitly out of scope.
>>
>> Yours, Joel
>>
>>> Regarding the second (source based), I think there are more
>>> subcases to
>> consider:
>>>
>>> 1.  Propagation of NSH encapsulated packet from classifier to
>>> SFF 2.  Propagation of NSH encapsulated packet from SFF to SF 3.
>>> Propagation of NSH encapsulated packet from SF to SFF (including
>>> [mailto:[email-masked]] On Behalf Of Linda Dunbar Sent:
>>> Tuesday, April 26, 2016 11:48 AM To: Joel M. Halpern
>>> <[email-masked]>; [email-masked];
>>> I think the document should add the description on the following
>>> two
>> fragmentation scenarios:
>>>
>>> fragmentation is caused by transport tunnel (i.e.
>> various encapsulations), the termination point of the transport
>> tunnel is responsible for re-assembling the fragmented pieces of
>> the packet. Since there won't be any SFF nodes in between the
>> Transport Tunnel, the tunnel generated fragmentation does not
>> require any actions by SFF nodes or SF nodes.
>>>
>>>
>>> [mailto:[email-masked]] Sent: Tuesday, April 26, 2016 10:33
>>> AM To: Linda Dunbar; [email-masked]; Elzur, Uri;
>> case of transport generated fragmentation of the outer packet.  I
>> technology, if the tunnel chooses to fragment at its layer, then
>> the tunnel is responsible for reassembly.  That would be invisible
>> to the SFF.
>>>
>>> Yours, Joel
>>>
>> header, the intermediate SFF nodes still need to put together all
>>>> [email-masked] Sent: Monday, April 25, 2016 9:42
>>>> AM To: Elzur, Uri; Joel M. Halpern; [email-masked] Subject: Re:
>>>>> 16:26 À : BOUCADAIR Mohamed IMT/OLN; Joel M. Halpern;
>>>>> Not to repeat my position, but I'll do it anyhow :-) As NSH
>>>>> is *NOT* a transport, dealing with fragmentation is left to
>>>>> the Transport used.
>>>>>
>>>>> The model I use for NSH, is basically similar to VXLAN . It
>>>>> is an
>> overly.
>>>>>
>>>>> Thx
>>>>>
>>>>> [email-masked] Sent: Monday, April 25, 2016
>>>>> 7:18 AM To: Joel M. Halpern <[email-masked]>;
>>>>>> [mailto:[email-masked]] Envoyé : lundi 25 avril 2016
>>>>>> 15:48 À : BOUCADAIR Mohamed IMT/OLN; [email-masked] Objet :
>>>>>> If I am understanding you correctly Med, you are asking
>>>>>> that the NSH draft specify how service chaining should cope
>>>>>> with packets that have been fragmented?
>>>>>>
>>>>>
>>>>> * SFF: If the NSH header is present only in the a fragment
>>>>> but SFF didn't maintained a state, subsequent fragments won't
>>>>> be appropriately
>> processed.
>>>>> * SFC-aware function: if prepending a context information
>>>>> depends on the full packet, not only a fragment.
>>>>>
>>>>> It just does its job.
>>>>>
>>>>>> Ingress and intermediate classifiers may cope with
>>>>>> fragments in any number of ways.
>>>>> [Med] The purpose is not to specify the internal
>>>>> implementation details but the external behavior of the
>>>>> classifier function when it comes to handle fragments. That
>>>>> behavior may have an incidence on SFF, in particular. The
>>>>> purpose is not to recommend the maximum resources to be
>>>>> dedicated to out of order fragments nor the timeout to cache
>>>>> those; these considerations are of course out of scope.
>>>>>> I suppose one could write an informational draft on
>>>>>> possible ways of coping.  The IETF has not usually
>>>>>> published such. Service functions have to cope with
>>>>>> fragmented packets just as they had to before the advent of
>>>>>> NSH, so describing that is clearly not needed here.
>>>>>
>>>>> Introducing NSH should not degrade the overall service
>>>>> compared to legacy service deployment schemes.
>>>>>
>>>>>>
>>>>>> Yours, Joel
>>>>>>
>>>>>>> There are other issues that need to be discussed, e.g.,
>>>>>>> how to deal with
>>>>>> context TLVs depends on the full packet (not just
>>>>>> fragment). It is warranted to consider a little bit this
>>>>>> point before declaring there is no issue.
>>>>>>>
>>>>>> an implementation must be prepared to receive fragments
>>>>>> with or without NSH header as this is a decision that is
>>>>>> left to the transport encapsulation. This is a requirement
>>>>>> per se!
>>>>>>>
>>>>>>> I won't reiterate all the comments I have about the
>>>>>>> current wording of
>>>>>> this section.
>>>>>>>
>>>>>>> Cheers, Med
>>>>>>>
>>>>>>>> [mailto:[email-masked]] Envoyé : lundi 25 avril
>>>>>>>> 2016 08:30 À : BOUCADAIR Mohamed IMT/OLN; Thomas
>>>>>>>> Narten; [email-masked] Objet : RE: [sfc] WG last call
>>>>>>>> is beyond the scope of NSH and beyond the charter of
>>>>>>>> the WG, so it
>>>>>> doesn't
>>>>>>>> really belong in the draft. We are providing an advice
>>>>>>>> as extending the size of the packet may lead to
>>>>>>>> fragmentation, but as you check RFC 7348 VxLAN, which
>>>>>>>> my create the same issue, you'll find it doesn't even
>>>>>> relate
>>>>>>>> to it.
>>>>>>>>
>>>>>>>> Thx
>>>>>>>>
>>>>>>>> [email-masked] Sent: Sunday, April 24,
>>>>>>>> 2016 10:32 PM To: Elzur, Uri <[email-masked]>;
>>>>>>>> Thomas Narten
>>>>>> <[email-masked]>;
>>>>>>>> this is not the rationale adopted in -04 since it
>>>>>>>> includes some text
>>>>>> that
>>>>>>>> it opens the door for interoperable issues, it may lead
>>>>>>>> to sub-optimal implementations if the sfc information
>>>>>>>> is present only in one
>>>>>> fragments,
>>>>>>>> etc.
>>>>>>>>
>>>>>>>>> [mailto:[email-masked]] Envoyé : dimanche 24
>>>>>>>>> avril 2016 17:36 À : BOUCADAIR Mohamed IMT/OLN;
>>>>>>>>> Thomas Narten; [email-masked] Objet : RE: [sfc] WG last
>>>>>>>>> transport independent i.e. like NSH interaction with
>>>>>>>>> any other Transport eh issue of Fragmentation is to
>>>>>>>>> be dealt in a way that matches the mechanisms
>>>>>>>>> [email-masked] Sent: Thursday, April
>>>>>>>>> 14, 2016 12:43 AM To: Thomas Narten
>>>>>>>>> <[email-masked]>; [email-masked] Subject: Re: [sfc]
>>>>>>>>> draft, I would like to raise this additional one
>>>>>>>>> about Section 6.
>>>>>>>>>
>>>>>>>>> NSH and the associated transport header are "added"
>>>>>>>>> to the encapsulated packet/frame.  This additional
>>>>>>>>> information increases
>>>>>> the
>>>>>>>>> forwarding of
>>>>>>>>> data, several options for handling fragmentation and
>>>>>>>>> re-
>> assembly
>>>>>>>>> exist:
>>>>>>>>>
>>>>>>>>> 2.  Path MTU Discovery [RFC1191]"describes a
>>>>>>>>> technique for dynamically discovering the maximum
>>>>>>>>> transmission unit (MTU) of
>>>>>> an
>>>>>>>>> arbitrary internet path" and can be utilized to
>>>>>>>>> ensure the
>>>>> the
>>>>>>>>> 3.  [RFC6830] describes two schemes for fragmentation
>>>>>>>>> and re-
>>>>>> assembly
>>>>>>>>> in section 5.4. ==
>>>>>>>>>
>>>>>>>>> * The text is weak for a Standard track document that
>>>>>>>>> is intended to solve the problem in
>>>>>>>>> There should be a clear behavior to be followed by
>>>>>>>>> an
>>>>> implementation.
>>>>>>>>> * The text covers only fragmentation when it is
>>>>>>>>> induced by SFC operations, it does not discuss the
>>>>>>>>> treatment of a fragment when received in an SFC
>>>>>>>>> domain. IMHO, the draft should also specify the
>>>>>>>>> behavior of the Classifier with regards to fragments
>>>>>>>>> for the sake of proper SFC operation. Applying
>>>>>>>>> In particular, dedicated resources should dedicated
>>>>>>>>> for handling out of order fragments. Of course, it is
>>>>>>>>> out of scope of this document to describe how SFs
>>>>>>>>> handle fragments.
>>>>>>>>>
>>>>>>>>> * If an SFC header is prepended for all fragments,
>>>>>>>>> I'm not sure there is any particular issue at the SFF
>>>>>>>>> level...except if stripping/adding context TLVs
>>>>>>>>> depends on the full packet (not just fragment). It is
>>>>>>>>> fragmentation may still be experienced (e.g., other
>>>>>>>>> shims are prepended by other nodes for some other
>>>>>>>>> purposes, nested nsh, etc.)
>>>>>>>>>
>>>>>>>>> * The support of jumbo frames by a router/device does
>>>>>>>>> not mean that it can make use of it. Appropriate MTU
>>>>>>>>> configuration should be undertaken in a consistent
>>>>>>>>> manner within an SFC domain. The text should be
>>>>>>>>> * BTW, shouldn't the text be reworded to recommended
>>>>>>>>> to increase the MTU of **all nodes** of an
>>>>>>>>> SFC-enabled domain by at least the length of SFC
>>>>>>>>> * Bullet 2, how PMTU discovery is actually used in
>>>>>>>>> this context? Do you assume that all SFC-aware nodes
>>>>>>>>> will issue such messages towards other SFC-aware
>>>>>>>>> node, arbitrary destination, else?
>>>>>>>>>
>>>>>>>>> (1) I don't think it is a good approach that
>>>>>>>>> fragments induced by the network are passed to their
>>>>>>>>> ultimate destinations as such
>>>>> (stateless).
>>>>>>>>> (responsible for fragmentation) not passed away. (2)
>>>>>>>>> Does the stateful mode require all SFC data plane
>>>>>>>>> elements maintain a full list of MTU for the any
>>>>>>>>> SFF/SF instance of the SFC
>>>>>>>> domain?
>>>>>>>>>
>>>>>>>>> The current text suggests that [RFC6830] should be
>>>>>>>>> listed as normative reference (not an informative
>>>>>>>>> one). I would personally favor removing the reference
>>>>>>>>> to LISP (as it is an Experimental
>> RFC).
>>>>>>>>>
>>>>>>>>> e.g., RFC1858 (Security Considerations for IP
>>>>>>>>> Fragment Filtering), RFC3128 (Protection Against a
>>>>>>>>> Variant of the Tiny Fragment Attack), and RFC 4963
>>>>>>>>> (IPv4 Reassembly Errors at High Data Rates).
>>>>>>>>>
>>>>>>>>> Cheers, Med
>>>>>>>>>
>>>>>>>>>> [email-masked] Envoyé : lundi 11
>>>>>>>>>> avril 2016 13:14 À
>> :
>>>>>>>>>> Thomas Narten; [email-masked] Objet : Re: [sfc] WG
>>>>>>>>>> As I mentioned during the meeting, there are
>>>>>>>>>> several issues that are not covered in the last
>>>>>>>>>> version of the draft. I already provided examples
>>>>>>>>>> of the issues offline as requested by Martin.
>>>>>>>>>>
>>>>>>>>>> Cheers, Med
>>>>>>>>>>
>>>>>>>>>>> Thomas Narten Envoyé : jeudi 31 mars 2016 04:48 À
>>>>>>>>>>> : [email-masked] Objet : [sfc] WG last call for
>>>>>>>>>>> addressed all known comments and that there are
>>>>>>>>>>> no open issues with the current version of the
>>>>>>>>>>> document.
>>>>>>>>>>>
>>>>>>>>>>> Substantive comments to the list please,
>>>>>>>>>>> editorial comments can go directly to the
>>>>>>>>>>> document editors.
>>>>>>>>>>>
>>>>>>>>>>> sfc mailing list [email-masked]

---

From: Linda Dunbar <[email-masked]>
Date: Wed, 27 Apr 2016 19:40:35 +0000

I suggest adding the following paragraphs after the Bullet 3 of the Section 6 Fragmentation Consideration to make the process more clear and less controversial:

--------------------------------

RFC6830 describes the fragmentation method of breaking the original packet into two equal sub-frames and encapsulating [LISP Header + Transport header] to each sub-frame. 

If LISP fragmentation [RFC6830 Section 5.4] is used, the [SFC Header + Transport Header] will be added to each half frame (or the original data frame). As the Transport Header is terminated by the next SFF node's tunnel transport layer, the combined two sub-frames will have two SFC Headers. 

Therefore, the fragmentation for NSH encapsulated data frame should be done completely by the node tunnel transport layer, which should break the [SFC + original packet] into two equal sub-frames and each sub-frame being encapsulated with its respective tunnel header.  The next SFF node's tunnel transport layer should combine the two sub-frames before sending to the next node.   

------------------------------------------------------

By the way, there are to typo in the Section 6: 
3rd line: "In order the" ==> "In order to"
Last line of Bullet 2: extra "the" in the "utilized to ensure the the required packet". 

Hope it helps. 

Linda

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Wednesday, April 27, 2016 12:40 AM
To: Joel M. Halpern; Linda Dunbar; Elzur, Uri; [email-masked]
Subject: RE: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Hi Joel, all,

Please see inline. 

Cheers,
Med

> De : Joel M. Halpern [mailto:[email-masked]] Envoyé : mardi 26 
> avril 2016 19:18 À : Linda Dunbar; BOUCADAIR Mohamed IMT/OLN; Elzur, 

[Med] I agree with you.

[Med] Fully agree.

> >>> 2016 7:18 AM To: Joel M. Halpern <[email-masked]>; 
> >>> [email-masked]
> >>>> [mailto:[email-masked]] Envoyé : lundi 25 avril 2016 15:48 À 
> >>>> : BOUCADAIR Mohamed IMT/OLN; [email-masked] Objet : Re: [sfc] WG 
> >>> timeout to cache those; these considerations are of course out of 
> >>> scope. Nevertheless, an implementation should offer a configurable 
> >>>>>> 10:32 PM To: Elzur, Uri <[email-masked]>; Thomas Narten
> >>>> <[email-masked]>;
> >>>>>>> as if implementing one of them is sufficient...which is not 
> >>>>>>> true. The first two points contribute to minimize the 
> >>>>>>> fragmentation risk, but fragmentation may still be experienced 
> >>>>>>> (e.g., other shims are prepended by other nodes for some other 
> >>>>>>> context? Do you assume that all SFC-aware nodes will issue 
> >>>>>>> such messages towards other SFC-aware node, arbitrary 
> >>>>>>> destination, else?
> >>>>>>>

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 27 Apr 2016 16:19:16 -0400

Both methods are valid, and both depend upon details of the underlying 
packet and the transport.  As such, I don't see why this document 
benefits from describing them, much less why it should mark one method 
as a "should".  Implementation details are likely to be more significant 
than any bit consumption difference between the two alternatives.

Yours,
Joel

---

From: Linda Dunbar <[email-masked]>
Date: Wed, 27 Apr 2016 20:33:25 +0000

Joel, 

If you think "both methods are valid", then the implication of both methods should be included in the document.

I don't believe the current description in the Section 6 is adequate enough. Otherwise, there won't be so much debate on the list. 

Linda 

-----Original Message-----
From: Joel M. Halpern [mailto:[email-masked]] 
Sent: Wednesday, April 27, 2016 3:19 PM
To: Linda Dunbar; [email-masked]; Elzur, Uri; [email-masked]
Subject: Re: Suggested wording addtion to the Section 6 (Fragmentation Consideration) of the draft-ietf-sfc-nsh-04.txt

Both methods are valid, and both depend upon details of the underlying packet and the transport.  As such, I don't see why this document benefits from describing them, much less why it should mark one method as a "should".  Implementation details are likely to be more significant than any bit consumption difference between the two alternatives.

Yours,
Joel

On 4/27/16 3:40 PM, Linda Dunbar wrote:
>> packets, that would be a violation of its job.
>
> [Med] I agree with you.
>
>> reassemble a packet fragmented by the source.  The classifier may 
>> have to do some interesting things in order to properly classify 
>> succeeding fragments, but that is an implementation issue (most 
>> commonly addressed with virtual reassembly, which doe snot delay the
> [Med] Still, the external behavior of the classifier needs to be clear 
> in the document. I don't find any text in the draft saying for 
> instance that an NSH header must be present in all fragments. (There 
> some processing that might be needed at the SFF to "do its job" with 
> regards to fragments (receive out of order fragments + forwarding 
> policy on the full packet).)
>
>>

---

From: "Andrew G. Malis" <[email-masked]>
Date: Wed, 27 Apr 2016 19:02:44 -0400

Joel et al,

All this talk about fragmentation prompted me to re-read section 6 of the
draft, which recommends (as option 3) using the procedures in section 5.4
of RFC 6830 (presumably with the “NSH header” replacing the “LISP header”
in the description of the procedures in that section).

So that led me to read that section (and the LISP header definition in
section 5.1), and I see that LISP does fragmentation and reassembly
identically to IPv4, using the Fragment Offset field so that fragments can
be correctly reassembled in the proper order.

However, the NSH Header doesn’t have a Fragment Offset field or any other
way to order the fragments.

So how can the procedures in Section 5.4 of 6830 be used?

Thanks,
Andy

On Wed, Apr 27, 2016 at 4:19 PM, Joel M. Halpern <[email-masked]>
wrote:

> packet and the transport.  As such, I don't see why this document benefits
> from describing them, much less why it should mark one method as a
> "should".  Implementation details are likely to be more significant than
> any bit consumption difference between the two alternatives.
>
> Yours,
> Joel
>
>
>> I suggest adding the following paragraphs after the Bullet 3 of the
>> Section 6 Fragmentation Consideration to make the process more clear and
>> less controversial:
>>
>>
>> RFC6830 describes the fragmentation method of breaking the original
>> packet into two equal sub-frames and encapsulating [LISP Header + Transport
>> header] to each sub-frame.
>>
>> If LISP fragmentation [RFC6830 Section 5.4] is used, the [SFC Header +
>> Transport Header] will be added to each half frame (or the original data
>> frame). As the Transport Header is terminated by the next SFF node's tunnel
>> transport layer, the combined two sub-frames will have two SFC Headers.
>>
>> Therefore, the fragmentation for NSH encapsulated data frame should be
>> done completely by the node tunnel transport layer, which should break the
>> [SFC + original packet] into two equal sub-frames and each sub-frame being
>> encapsulated with its respective tunnel header.  The next SFF node's tunnel
>> transport layer should combine the two sub-frames before sending to the
>> next node.
>>
>> [Med] Still, the external behavior of the classifier needs to be clear in
>> the document. I don't find any text in the draft saying for instance that
>> an NSH header must be present in all fragments. (There some processing that
>> might be needed at the SFF to "do its job" with regards to fragments
>> (receive out of order fragments + forwarding policy on the full packet).)
>>
>>
>>>>>>>>>> dynamically discovering the maximum transmission unit (MTU) of
>>>>>>>>>>
>>>>>>>>> an
>>>>>>>
>>>>>>>> arbitrary internet path" and can be utilized to ensure the
>>>>>>>>>>
>>>>>>>>> the
>>>>>>
>>>>>>> There should be a clear behavior to be followed by an
>>>>>>>>>>
>>>>>>>>> implementation.
>>>>>>
>>>>>>>>>> classification policies may require the
>>>>>>>>>>
>>>>>>>>>> full list of MTU for the any SFF/SF instance of the SFC
>>>>>>>>>>
>>>>>>>>> domain?
>>>>>>>>>
>>>>>>>>>>

---

From: Dino Farinacci <[email-masked]>
Date: Wed, 27 Apr 2016 16:10:37 -0700

Andy, there is a difference though. RFC6830 says an ITR can fragment but an ETR does not reassemble. That is, an arrived packet at the ITR causes the “user’s header” to be fragmented and then each fragment is encapsulated. So the fragment reassembly is done at the destination host.

For NSH, I think you want the reassembly to happen at the end of the SFF. But I’m not sure, so for you all to comment on.

Dino

> On Apr 27, 2016, at 4:02 PM, Andrew G. Malis <[email-masked]> wrote:
> 
> Joel et al,
> 
> All this talk about fragmentation prompted me to re-read section 6 of the draft, which recommends (as option 3) using the procedures in section 5.4 of RFC 6830 (presumably with the “NSH header” replacing the “LISP header” in the description of the procedures in that section).
> 
> So that led me to read that section (and the LISP header definition in section 5.1), and I see that LISP does fragmentation and reassembly identically to IPv4, using the Fragment Offset field so that fragments can be correctly reassembled in the proper order.
> 
> However, the NSH Header doesn’t have a Fragment Offset field or any other way to order the fragments.
> 
> On Wed, Apr 27, 2016 at 4:19 PM, Joel M. Halpern <[email-masked]> wrote:

---

From: "Andrew G. Malis" <[email-masked]>
Date: Wed, 27 Apr 2016 19:16:31 -0400

Hey, Dino!

Regardless of where the reassembly is done, it needs to be done correctly.
I’m just wondering how that can happen without the necessary header fields.

Cheers,
Andy

On Wed, Apr 27, 2016 at 7:10 PM, Dino Farinacci <[email-masked]> wrote:

> Andy, there is a difference though. RFC6830 says an ITR can fragment but
> an ETR does not reassemble. That is, an arrived packet at the ITR causes
> the “user’s header” to be fragmented and then each fragment is
> encapsulated. So the fragment reassembly is done at the destination host.
>
> For NSH, I think you want the reassembly to happen at the end of the SFF.
> But I’m not sure, so for you all to comment on.
>
> Dino
>
> > All this talk about fragmentation prompted me to re-read section 6 of
> the draft, which recommends (as option 3) using the procedures in section
> 5.4 of RFC 6830 (presumably with the “NSH header” replacing the “LISP
> header” in the description of the procedures in that section).
> >
> in the document. I don't find any text in the draft saying for instance
> that an NSH header must be present in all fragments. (There some processing
> that might be needed at the SFF to "do its job" with regards to fragments

---

From: Joel Halpern Direct <[email-masked]>
Date: Wed, 27 Apr 2016 19:46:09 -0400

The LISP header does not have a fragment flag or fragment offset.  The 
diagrams in section 6 include the outer encapsulating header (the 
equivalent of the transport header in SFC.)  Yes, the text is a little 
hard to follow in this regard.  The LISP working group is going to be 
rewriting RFC 6830 as part of moving to standards track.

So there is no difference in this regard between NSH and LISP.

Yours,
Joel

On 4/27/16 7:02 PM, Andrew G. Malis wrote:
> the draft, which recommends (as option 3) using the procedures in
> section 5.4 of RFC 6830 (presumably with the “NSH header” replacing the
> “LISP header” in the description of the procedures in that section).
>
> identically to IPv4, using the Fragment Offset field so that fragments
> can be correctly reassembled in the proper order.
>
> On Wed, Apr 27, 2016 at 4:19 PM, Joel M. Halpern <[email-masked]
>     Both methods are valid, and both depend upon details of the
>     underlying packet and the transport.  As such, I don't see why this
>     document benefits from describing them, much less why it should mark
>     one method as a "should".  Implementation details are likely to be
>     more significant than any bit consumption difference between the two
>     alternatives.
>
>     Yours,
>     Joel
>
>
>         I suggest adding the following paragraphs after the Bullet 3 of
>         the Section 6 Fragmentation Consideration to make the process
>         more clear and less controversial:
>
>
>         RFC6830 describes the fragmentation method of breaking the
>         original packet into two equal sub-frames and encapsulating
>         [LISP Header + Transport header] to each sub-frame.
>
>         If LISP fragmentation [RFC6830 Section 5.4] is used, the [SFC
>         Header + Transport Header] will be added to each half frame (or
>         the original data frame). As the Transport Header is terminated
>         by the next SFF node's tunnel transport layer, the combined two
>         sub-frames will have two SFC Headers.
>
>         Therefore, the fragmentation for NSH encapsulated data frame
>         should be done completely by the node tunnel transport layer,
>         which should break the [SFC + original packet] into two equal
>         sub-frames and each sub-frame being encapsulated with its
>         respective tunnel header.  The next SFF node's tunnel transport
>         layer should combine the two sub-frames before sending to the
>         next node.
>
>         From: [email-masked]
>         [mailto:[email-masked]
>         <mailto:[email-masked]>]
>             De : Joel M. Halpern [mailto:[email-masked]
>             <mailto:[email-masked]>] Envoyé : mardi 26
>             Uri; [email-masked] <mailto:[email-masked]> Objet : Re: [sfc] WG
>             last call for
>         [Med] Still, the external behavior of the classifier needs to be
>         clear in the document. I don't find any text in the draft saying
>         for instance that an NSH header must be present in all
>         fragments. (There some processing that might be needed at the
>         SFF to "do its job" with regards to fragments (receive out of
>         order fragments + forwarding policy on the full packet).)
>
>
>                 I think the document should add the description on the
>                 following two
>                 fragmentation scenarios:
>
>                 [mailto:[email-masked]
>                 <mailto:[email-masked]>] Sent: Tuesday, April 26,
>                 2016 10:33 AM
>                 To: Linda Dunbar; [email-masked]
>                 <mailto:[email-masked]>; Elzur, Uri;
>                     [mailto:[email-masked]
>                     <mailto:[email-masked]>]
>                     On Behalf Of [email-masked]
>                     <mailto:[email-masked]> Sent: Monday,
>                     April 25,
>                     2016 9:42 AM To: Elzur, Uri; Joel M. Halpern;
>                     [email-masked] <mailto:[email-masked]> Subject:
>                         [mailto:[email-masked]
>                         <mailto:[email-masked]>] Envoyé : lundi 25
>                         avril 2016 16:26 À
>                         The model I use for NSH, is basically similar to
>                         VXLAN . It is an
>                         overly.
>
>                         Thx
>
>                         Uri ("Oo-Ree") C: 949-378-7568 <tel:949-378-7568>
>
>
>                         [mailto:[email-masked]
>                         <mailto:[email-masked]>]
>                         On Behalf Of [email-masked]
>                         <mailto:[email-masked]> Sent:
>                         Monday, April 25,
>                         2016 7:18 AM To: Joel M. Halpern
>                         <[email-masked] <mailto:[email-masked]>>;
>                             [mailto:[email-masked]
>                             <mailto:[email-masked]>] Envoyé : lundi
>                             25 avril 2016 15:48 À
>                             : BOUCADAIR Mohamed IMT/OLN; [email-masked]
>                             <mailto:[email-masked]> Objet : Re: [sfc] WG
>                         implications. If there are, then the draft
>                         should be updated
>                         accordingly.
>
>                         [Med] I'm not that sure. Some typical
>                         implications are listed
>                         below:
>
>                         [Med] which may be disturbed in some situations
>                         as listed in the
>                         examples above.
>
>                         classifier function when
>                         parameter so that an operator tweak those
>                         according to its
>                         context.
>
>                         Introducing NSH should not degrade the overall
>                         service compared to
>                         legacy service deployment schemes.
>
>
>                             Yours, Joel
>
>                             On 4/25/16 3:00 AM,
>                             <mailto:[email-masked]> wrote:
>
>                                 Re-,
>
>                                 I hear you, but my comment is that we
>                                 need, as a WG, to decide
>                                 what to
>
>                             put in the draft. FWIW, I'm discussing two
>                             fragmentation
>                             issues:
>
>
>                                 Increasing the MTU is for sure a
>                                 recommendation is to be
>                                 explicitly
>
>                                 It is also "prudent" to check that no
>                                 issues will be experienced
>                                 in SFF
>
>                                 is any particular issue at the SFF
>                                 level, except if
>                                 stripping/adding
>
>                                 For point (1), declaring fragmentation
>                                 out of scope would be
>                                 meant that
>
>                                     [mailto:[email-masked]
>                                     <mailto:[email-masked]>] Envoyé
>                                     : lundi 25 avril 2016
>                                     <tel:949-378-7568>
>                                     [mailto:[email-masked]
>                                     <mailto:[email-masked]>] On
>                                     Behalf Of
>                                     <mailto:[email-masked]> Sent:
>                                     Sunday, April 24, 2016
>                                     10:32 PM To: Elzur, Uri
>                                     <[email-masked]
>                                     is far to be sufficient to ensure
>                                     interoperable
>                                     implementations.
>
>                                     My comments are related to the
>                                     current text in the -04.
>                                     This text needs
>
>                             to
>
>                                     be fixed somehow.
>
>                                     Cheers, Med
>
>                                         [mailto:[email-masked]
>                                         <mailto:[email-masked]>]
>                                         Envoyé : dimanche 24 avril
>                                         Transport eh issue of
>                                         Fragmentation is to be dealt in
>                                         a way
>                                         <tel:949-378-7568>
>                                         [mailto:[email-masked]
>                                         <mailto:[email-masked]>]
>                                         On Behalf Of
>                                         Sent: Thursday, April 14,
>                                         <mailto:[email-masked]>>;
>                                         [email-masked]
>                                         <mailto:[email-masked]> Subject:
>                                         1.  Jumbo Frames, when
>                                         supported, enable the transport
>                                         of NSH
>                                         by SFC
>                                         fragmentation risk, but
>                                         fragmentation may still be
>                                         experienced
>                                         by the network are passed to
>                                         their ultimate destinations as
>                                         such
>
>                         (stateless).
>
>                                             De : sfc
>                                             [mailto:[email-masked]
>                                             <mailto:[email-masked]>]
>                                             De la part de
>                                             2016 13:14 À : Thomas
>                                             Narten; [email-masked]
>                                             <mailto:[email-masked]> Objet
>                                             : Re:
>                                             [sfc] WG last call for
>                                             issues
>                                             already provided examples of
>                                                 [mailto:[email-masked]
>                                                 <mailto:[email-masked]>]
>                                                 De la part de Thomas Narten
>                                                 Envoyé : jeudi 31 mars
>                                                 2016 04:48 À :
>                                                 [email-masked]
>                                                 <mailto:[email-masked]>
>                                                 Objet : [sfc] WG last
>                                                 call for
>                                                 This note begins a WG
>                                                 last call on
>                                                 We'll also get a brief
>                                                 update from the editors
>                                                 at next
>                                                 week's meeting. If there
>                                                 are any remaining issues
>                     mailing list

---

From: <[email-masked]>
Date: Thu, 28 Apr 2016 08:06:51 +0000

Hi Joel,

Now that we agree on the scope, shouldn't the text be /updated to reflect and clarify what was discussed in this thread?

>From where I stand, I would like to see added in particular text saying that nsh header is present in every fragment.  

Cheers,
Med

> Envoyé : mercredi 27 avril 2016 16:04
> À : BOUCADAIR Mohamed IMT/OLN; Ron Parker; Elzur, Uri; [email-masked]

---

From: "Andrew G. Malis" <[email-masked]>
Date: Thu, 28 Apr 2016 08:17:24 -0400

Joel,

The diagrams in section 6 of RFC 6830 are control plane, not data plane.
The data plane diagrams are in section 5.

But the overriding question is - without any fields in the NSH header to
sequence fragments, how can the fragments be correctly reassembled?

Cheers,
Andy

On Wed, Apr 27, 2016 at 7:46 PM, Joel Halpern Direct <
[email-masked]> wrote:

> equivalent of the transport header in SFC.)  Yes, the text is a little hard
> to follow in this regard.  The LISP working group is going to be rewriting
> RFC 6830 as part of moving to standards track.
>
>>                         <[email-masked] <mailto:[email-masked]
>> >>;
>> Sent:
>>                             <[email-masked] <mailto:[email-masked]
>> >>;
>>
>> for
>>                                         increase the MTU of **all
>>                                         nodes** of an SFC-enabled domain
>> by
>> >]
>>                                             the issues offline as
>> requested
>>                                             by Martin.
>>                                                 (
>> https://datatracker.ietf.org/doc/draft-ietf-sfc-nsh/).
>>
>>
>>
>>                                                 document, raising them
>>                                                 before the meeting would
>> be
>>                                     sfc mailing

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 28 Apr 2016 08:45:28 -0400

The fragmentation fields shown in all the diagrams in the LISP spec are 
the IP fragmentation fields.  There is no LISP fragmentation information.

Thus, in parallel, I don't see nay need for NSH to have fragmentation 
information.

Yes, there is an implication.  Inner packet fragmentation can only be 
used as a response to an MTU issue when the inner packet format supports 
fragmentation.  As far as I know, that is only IPv4.

Outer (transport) packet fragmentation can only be used to address the 
MTU issue when the outer (transport) packet supports fragmentation, that 
being IPv4 and IPv6.

Note that a corollary of this is that SFC simply puts NSH headers on the 
packets it (the classifier or SFF) sends.

Yours,
Joel

On 4/28/16 8:17 AM, Andrew G. Malis wrote:
> On Wed, Apr 27, 2016 at 7:46 PM, Joel Halpern Direct
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>     The LISP header does not have a fragment flag or fragment offset.
>     The diagrams in section 6 include the outer encapsulating header
>     (the equivalent of the transport header in SFC.)  Yes, the text is a
>     little hard to follow in this regard.  The LISP working group is
>     going to be rewriting RFC 6830 as part of moving to standards track.
>
>         On Wed, Apr 27, 2016 at 4:19 PM, Joel M. Halpern
>         <[email-masked] <mailto:[email-masked]>
>         <mailto:[email-masked] <mailto:[email-masked]>>> wrote:
>
>                 Last line of Bullet 2: extra "the" in the "utilized to
>         ensure
>                 the the required packet".
>
>
>                 Hope it helps.
>
>                 Linda
>
>
>
>                 <mailto:[email-masked]
>         <mailto:[email-masked]>>
>                 <mailto:[email-masked]
>         <mailto:[email-masked]>>]
>                 To: Joel M. Halpern; Linda Dunbar; Elzur, Uri;
>         <mailto:[email-masked]>>] Envoyé : mardi 26
>                     Uri; [email-masked] <mailto:[email-masked]>
>         <mailto:[email-masked] <mailto:[email-masked]>> Objet : Re: [sfc] WG
>                     last call for
>                     With regard to Transport tunnel fragementation, that
>         seems
>                     an issue
>                     for the transport protocol.  I don't actually object
>         to a
>                     sentence,
>                     to do some interesting things in order to properly
>         classify
>                     succeeding
>                     If an SF needs to reassemble fragments to do its
>         job, that
>                     is up to
>                     the SF.  Some will need to actually reassemble.
>         Some will
>                     need to
>                         encapsulations), the termination point of the
>         transport
>                         tunnel is
>                         responsible for re-assembling the fragmented
>         pieces of
>                         the packet.
>                         the tunnel generated fragmentation does not
>         require any
>                         actions by
>                         SFF nodes or SF nodes.
>
>
>                         - Source node generated fragmentation (after
>         adding on
>                         the NSH
>                         header): When fragmentation has to be performed
>         for a
>                         packet being
>         <mailto:[email-masked]>>] Sent: Tuesday, April 26,
>                         2016 10:33 AM
>                         <mailto:[email-masked]
>         <mailto:[email-masked]>>; Elzur, Uri;
>         <mailto:[email-masked] <mailto:[email-masked]>> Subject: Re: [sfc] WG
>                         last call for
>                         I had assumed you were not talking about that,
>         since the
>                         resulting
>                             NSH header, the intermediate SFF nodes still
>         need to
>                             put together
>                             all the fragments together before passing
>         the whole
>                             data frame to
>                             the SF.
>
>                             Linda
>
>         <mailto:[email-masked]>>]
>                             <mailto:[email-masked]
>         <mailto:[email-masked]>> Sent: Monday,
>                             April 25,
>         <mailto:[email-masked] <mailto:[email-masked]>> Subject:
>                             How do you instruct the transport layer to
>         ALWAYS
>                             prepend an NSH
>         <mailto:[email-masked]>
>                                 <mailto:[email-masked]
>         <mailto:[email-masked]>>] Envoyé : lundi 25
>                                 avril 2016 16:26 À
>                                 : BOUCADAIR Mohamed IMT/OLN; Joel M.
>         Halpern;
>         <mailto:[email-masked] <mailto:[email-masked]>> Objet
>                                 Not to repeat my position, but I'll do
>         it anyhow
>                                 :-) As NSH is
>                                 *NOT* a transport, dealing with
>         fragmentation is
>                                 left to the
>                                 Transport used.
>
>         <tel:949-378-7568> <tel:949-378-7568 <tel:949-378-7568>>
>
>
>         <mailto:[email-masked]>>]
>                                 On Behalf Of
>                                 <mailto:[email-masked]
>         <mailto:[email-masked]>> Sent:
>                                 Monday, April 25,
>                                 <[email-masked]
>         <mailto:[email-masked]>>>;
>         <mailto:[email-masked]>>] Envoyé : lundi
>                                     25 avril 2016 15:48 À
>                                     : BOUCADAIR Mohamed IMT/OLN;
>                                     <mailto:[email-masked]
>         <mailto:[email-masked]>> Objet : Re: [sfc] WG
>                                     If I am understanding you correctly
>         Med, you
>                                     are asking that the
>                                 * SFF: If the NSH header is present only
>         in the
>                                 a fragment but SFF
>                                 didn't maintained a state, subsequent
>         fragments
>                                 won't be
>                                 appropriately processed. * SFC-aware
>         function:
>                                 if prepending a
>                                 context information depends on the full
>         packet,
>                                 not only a
>                                 fragment.
>
>                                 It just does its job.
>
>                                 Specifying such is clearly out of scope
>         for this
>                                 draft.
>
>                                 it comes to handle fragments. That
>         behavior may
>                                 have an incidence
>                                 timeout to cache those; these
>         considerations are
>                                 of course out of
>                                 scope. Nevertheless, an implementation
>         should
>                                 offer a configurable
>                                     I suppose one could write an
>         informational
>                                     draft on possible ways
>                                 chaining may not be efficient if
>         fragments are
>                                 inappropriately
>                                 handled.
>
>                                     <mailto:[email-masked]
>         <mailto:[email-masked]>> wrote:
>
>                                         Re-,
>
>                                         (2) Handling fragments at the
>         ingress of
>                                         an SFC-enabled domain.
>
>                                         There are other issues that need
>         to be
>                                         discussed, e.g., how to
>                                         deal with
>
>                                     context TLVs depends on the full
>         packet (not
>                                     just fragment). It
>                                     is warranted to consider a little
>         bit this
>                                     point before declaring
>                                     there is no issue.
>
>
>                                         For point (1), declaring
>         fragmentation
>                                         out of scope would be
>                                         meant that
>
>         <mailto:[email-masked]>
>                                             <mailto:[email-masked]
>         <mailto:[email-masked]>>] Envoyé
>                                             : lundi 25 avril 2016
>                                             08:30 À : BOUCADAIR Mohamed
>         IMT/OLN;
>                                             Thomas Narten;
>                                             [email-masked]
>         <mailto:[email-masked]> <mailto:[email-masked] <mailto:[email-masked]>>
>                                             Objet : RE: [sfc] WG last
>         call for
>                                             My point is that
>         Fragmentation is
>                                             yet another transport related
>                                             issue
>
>                                     that
>
>                                             really belong in the draft.
>         We are
>                                             providing an advice as
>                                             extending the size of the
>         packet may
>                                             lead to fragmentation, but
>                                             as you check RFC 7348 VxLAN,
>         which
>                                             <tel:949-378-7568
>         <tel:949-378-7568>>
>         <mailto:[email-masked]>>] On
>                                             Behalf Of
>         <mailto:[email-masked]
>         <mailto:[email-masked]>> Sent:
>                                             Sunday, April 24, 2016
>                                             10:32 PM To: Elzur, Uri
>                                             <[email-masked]
>         <mailto:[email-masked]>
>                                             <mailto:[email-masked]
>         <mailto:[email-masked]>>>;
>                                             Thomas Narten
>
>                                     <[email-masked]
>         <mailto:[email-masked]> <mailto:[email-masked]
>         <mailto:[email-masked]>>>;
>
>                                             [email-masked]
>         <mailto:[email-masked]> <mailto:[email-masked] <mailto:[email-masked]>>
>                                             Subject: Re: [sfc] WG last
>         call for
>                                             That's another option that
>         needs to
>                                             be discussed/investigated.
>                                             I'm
>
>                                     afraid
>
>                                             this is not the rationale
>         adopted in
>                                             -04 since it includes some
>                                             text
>
>                                     that
>
>                                             BTW, saying that nsh does
>         not need
>                                             to deal with fragmentation
>                                             because
>
>                                     it
>
>                                             is transport-independent is
>         not IMHO
>                                             a good strategy to adopt
>                                             here
>
>                                     because
>
>                                             it opens the door for
>         interoperable
>                                             issues, it may lead to
>                                             sub-optimal implementations
>         if the
>                                             sfc information is present
>                                             only in one
>
>                                     fragments,
>
>                                             etc.
>
>                                                 -----Message
>         d'origine----- De :
>                                                 Elzur, Uri
>         [mailto:[email-masked] <mailto:[email-masked]>
>
>         <mailto:[email-masked] <mailto:[email-masked]>>]
>                                                 2016 17:36 À : BOUCADAIR
>         Mohamed
>                                                 IMT/OLN; Thomas Narten;
>                                                 <mailto:[email-masked]
>         <mailto:[email-masked]>> Objet :
>                                                 RE: [sfc] WG last call for
>                                                 transport independent
>         i.e. like
>                                                 NSH interaction with any
>         other
>                                                 that matches the mechanisms
>                                                 supported by the
>         Transport used
>                                                 and do not belong in the
>         NSH draft
>
>                                                 Thx
>
>                                                 Uri ("Oo-Ree") C:
>                                                 <tel:949-378-7568
>         <tel:949-378-7568>>
>         [mailto:[email-masked] <mailto:[email-masked]>
>
>         <mailto:[email-masked] <mailto:[email-masked]>>]
>                                                 On Behalf Of
>
>         <mailto:[email-masked]
>         <mailto:[email-masked]>>
>                                                 2016 12:43 AM To: Thomas
>                                                 <[email-masked]
>         <mailto:[email-masked]>
>
>         <mailto:[email-masked] <mailto:[email-masked]>>>;
>                                                 [email-masked]
>         <mailto:[email-masked]>
>                                                 <mailto:[email-masked]
>         <mailto:[email-masked]>> Subject:
>         draft, I
>                                                 would like to raise this
>                                                 additional one about
>                                                 NSH and the associated
>         transport
>                                                 header are "added" to the
>                                                 encapsulated
>         packet/frame.  This
>                                                 additional information
>                                                 increases
>
>                                     the
>
>                                                 size of the packet.  In
>         order
>                                                 the ensure proper
>         forwarding of
>                                                 NSH data, several
>         options for
>                                                 handling fragmentation and
>                                                 re-assembly exist:
>
>                                                 1.  Jumbo Frames, when
>                                                 supported, enable the
>         transport
>                                                 of NSH
>                                                 and associated transport
>         packets
>                                                 without requiring
>                                                 fragmentation.
>
>                                                 2.  Path MTU Discovery
>                                                 [RFC1191]"describes a
>         technique for
>                                                 dynamically discovering the
>                                                 maximum transmission
>         unit (MTU) of
>
>                                     an
>
>                                                 arbitrary internet path"
>         and can
>                                                 be utilized to ensure the
>
>                                 the
>
>                                                 required packet size is
>         used.
>
>                                                 3.  [RFC6830] describes two
>                                                 schemes for
>         fragmentation and
>                                                 re-
>
>                                     assembly
>
>                                                 in section 5.4. ==
>
>                                                 * The text is weak for a
>                                                 Standard track document
>         that is
>                                                 intended to solve the
>         problem in
>
>                                                 There should be a clear
>         behavior
>                                                 to be followed by an
>                                                 Further, I would avoid
>         the use
>                                                 of words such as "can".
>         induced
>                                                 operations, it does not
>         discuss
>                                                 the treatment of a fragment
>                                                 when received in an SFC
>         domain.
>                                                 IMHO, the draft should also
>                                                 fragments for the sake
>         of proper
>                                                 SFC operation. Applying
>                                                 In particular, dedicated
>                                                 resources should
>         dedicated for
>                                                 handling out of order
>                                                 Of course, it is out of
>         scope
>                                                 of this document to
>                                                 * If an SFC header is
>         prepended
>                                                 for all fragments, I'm not
>                                                 sure there is any particular
>                                                 issue at the SFF
>         level...except
>                                                 if stripping/adding
>         context TLVs
>                                                 warranted to consider a
>         little bit
>         interpreted
>                                                 as if implementing one
>         of them
>                                                 is sufficient...which is not
>                                                 (e.g., other shims are
>         prepended
>                                                 by other nodes for some
>         other
>                                                 * The first two points have
>                                                 nothing to do with
>         reassembly.
>
>                                                 * The support of jumbo
>         frames by
>                                                 a router/device does not
>         mean
>                                                 that it can make use of it.
>                                                 Appropriate MTU
>         configuration
>                                                 should be undertaken in a
>                                                 consistent manner within
>         an SFC
>                                                 nodes** of an
>                                                 * Bullet 2, how PMTU
>         discovery
>                                                 is actually used in this
>                                                 context? Do you assume
>                                                 SFC-aware nodes will issue
>                                                 (MTU) of an arbitrary
>         internet
>                                                 * The reference to the LISP
>                                                 specification raises two
>         concerns
>                                                 and one comment:
>
>                                                 (1) I don't think it is
>         a good
>                                                 approach that fragments
>         induced
>                                                 IMO, reassembly should
>         be done
>                                                 within the SFC domain
>         fragmentation)
>                                                 not passed away. (2)
>         Does the
>                                                 stateful mode require
>         all SFC
>                                                 data plane elements
>         maintain a
>                                                 The current text
>         suggests that
>                                                 [RFC6830] should be
>         listed as
>                                                 personally favor
>         removing the
>                                                 reference to LISP (as it
>                                                 * The security section
>         of the
>                                                 draft may be augmented with
>                                                 informational
>                                                 fragmentation-related
>         pointers to:
>                                                 e.g., RFC1858 (Security
>                                                 Considerations for IP
>         Fragment
>                                                 Filtering), RFC3128
>         (Protection
>                                                 Against a Variant of the
>         Tiny
>                                                 Fragment Attack), and
>         RFC 4963
>                                                 (IPv4 Reassembly Errors
>         at High
>                                                 Data Rates).
>
>                                                 Cheers, Med
>
>                                                     -----Message
>         d'origine-----
>                                                     De : sfc
>
>         [mailto:[email-masked] <mailto:[email-masked]>
>
>         <mailto:[email-masked] <mailto:[email-masked]>>]
>                                                     De la part de
>
>         <mailto:[email-masked]
>         <mailto:[email-masked]>>
>                                                     Envoyé : lundi 11 avril
>                                                     2016 13:14 À : Thomas
>                                                     Narten; [email-masked]
>         <mailto:[email-masked]>
>                                                     <mailto:[email-masked]
>         <mailto:[email-masked]>> Objet
>                                                     : Re:
>                                                     [sfc] WG last call for
>
>                                                     As I mentioned
>         during the
>                                                     meeting, there are
>         several
>                                                     that are not covered
>         in the
>                                                     last version of the
>         draft. I
>                                                     the issues offline
>         as requested
>         [mailto:[email-masked] <mailto:[email-masked]>
>
>         <mailto:[email-masked] <mailto:[email-masked]>>]
>                                                         De la part de
>         Thomas Narten
>                                                         Envoyé : jeudi
>         31 mars
>                                                         2016 04:48 À :
>                                                         [email-masked]
>         <mailto:[email-masked]>
>
>                                                         Objet : [sfc] WG
>         last
>                                                         call for
>                                                         This note begins
>         a WG
>                                                         addressed all known
>                                                         comments and
>         that there
>                                                         issues with the
>         current
>                                                         version of the
>                                                         Substantive
>         comments to
>                                                         the list please,
>                                                         editorial comments
>                                                         can go directly
>         to the
>                                                         We'll also get a
>         brief
>                                                         update from the
>         editors
>                                                         week's meeting.
>         If there
>                                                         are any
>         remaining issues
>                                                         document,
>         raising them
>                                                         before the

---

From: "Andrew G. Malis" <[email-masked]>
Date: Thu, 28 Apr 2016 09:22:11 -0400

Joel,

If this is actually how section 6 should work, there should be text to make
it clear.

I’m curious if any of the existing implementations of this draft have
implemented fragmentation, and if so, if they’re interoperable.

Thanks,
Andy

On Thu, Apr 28, 2016 at 8:45 AM, Joel M. Halpern <[email-masked]>
wrote:

> Yes, there is an implication.  Inner packet fragmentation can only be used
> as a response to an MTU issue when the inner packet format supports
> Outer (transport) packet fragmentation can only be used to address the MTU
> issue when the outer (transport) packet supports fragmentation, that being
> IPv4 and IPv6.
>
>>         However, the NSH Header doesn’t have a Fragment Offset field or
>> any
>>         other way to order the fragments.
>>
>>                         the case of transport generated fragmentation of
>> the
>>                         outer packet.
>>                         technology, if the tunnel chooses to fragment at
>> its
>>                         layer, then the
>>                         tunnel is responsible for reassembly.  That would
>> be
>>                         invisible to
>>                         the SFF.
>>
>>                         Yours, Joel
>>
>>                             Even if each fragment piece of a packet with
>> NSH
>>                             header carries the
>>         [email-masked] <mailto:[email-masked]
>> >
>>                                     Ingress and intermediate classifiers
>> may
>>                                     cope with fragments in
>>                                     any number of ways.
>>
>>                                 on SFF, in particular. The purpose is not
>> to
>>                                 recommend the maximum
>>                                     they had to before the advent of NSH,
>> so
>>                                     describing that is
>>                                     clearly not needed here.
>>
>>
>>                                 exacerbates fragmentation. Handing over
>> this
>>                                 issue to the
>>                                 transport layer may lead to
>> interoperability
>>                                 issues. * the
>>                                         (1) Fragmentation that is caused
>> by
>>                                         prepending an SFC header.
>>                                     to handle fragments. If an SFC header
>> is
>>                                     prepended for all
>>                                     without NSH header as this is a
>> decision
>>                                     that is left to the
>>                                         I won't reiterate all the
>> comments I
>>                                         have about the current
>>                                         wording of
>>
>>                                     this section.
>>
>>
>>                                         Cheers, Med
>>
>> De :
>>                                             is beyond the scope of NSH and
>>                                             beyond the charter of the WG,
>> so
>>                                             it
>>
>>                                     doesn't
>>
>>                                                 I see no need to specify
>>                                                 exact behavior as NSH is
>>         [email-masked] <mailto:[email-masked]
>> >
>>
>>                                                 In addition to other
>> pending
>>                                                 issues raised for this
>>                                                 specify the behavior of
>>                                                 Classifier with regards to
>>                                                 fragments for the sake
>>         of proper
>>                                                 SFC operation. Applying
>>                                                 classification policies
>> may
>>                                                 require the
>>
>>                                             full packet, not only a
>>         fragment.
>>
>>                                                 In particular, dedicated
>>                                                 resources should
>>         dedicated for
>>                                                 handling out of order
>>         fragments.
>>                                                 Of course, it is out of
>>         scope
>>                                                 of this document to
>>         describe how
>>                                                 SFs handle fragments.
>>
>>                                                 * If an SFC header is
>>         prepended

---

From: "Andrew G. Malis" <[email-masked]>
Date: Thu, 28 Apr 2016 09:23:56 -0400

Jim,

Thanks. To paraphrase what I just said to Joel, if that’s the intent, then
the text should make it clear.

Thanks,
Andy

On Thu, Apr 28, 2016 at 9:22 AM, Jim Guichard (jguichar) <[email-masked]

> Hi Andy,
>
> I think the key point is that NSH != network overlay but rather utilizes a
>  network overlay for packet transport between SFC network elements. The
> reference is just an example of how a network overlay might deal with
> fragmentation and is not a suggestion that NSH adopt that mechanism but
> rather makes the point that it can utilize the existing network overlay
> mechanics.
>
> Jim
>
> From: sfc <[email-masked]> on behalf of "Andrew G. Malis" <
> [email-masked]>
> Date: Thursday, April 28, 2016 at 8:17 AM
> To: Joel Halpern Direct <[email-masked]>
> Cc: "Elzur, Uri" <[email-masked]>, "[email-masked]" <
> [email-masked]>, "[email-masked]" <[email-masked]>, Linda
> Dunbar <[email-masked]>
> Subject: Re: [sfc] Suggested wording addtion to the Section 6
> (Fragmentation Consideration) of the draft-ietf-sfc-nsh-04.txt
>
> Joel,
>
>>>                         Uri ("Oo-Ree") C: 949-378-7568 <tel:949-378-7568
>>> >
>>>
>>>
>>>                                     beyond the charter of the WG, so
>>>                                     it
>>>
>>>                             doesn't
>>>
>>>                                         supported by the Transport used
>>>                                         additional one about Section 6.
>>>
>>>                                         == 6.  Fragmentation
>>> Considerations
>>>
>>>                                         maximum transmission unit (MTU)
>>> of
>>>
>>>                             an
>>>
>>>                                         schemes for fragmentation and
>>>                                         re-
>>>
>>>                             assembly
>>>
>>>                                         in section 5.4. ==
>>>
>>>                                         * The text is weak for a
>>>                                         Standard track document that is
>>>                                         resources should dedicated for
>>>                                         issue at the SFF level...except
>>>                                         (not just fragment). It is
>>>                                         warranted to consider a little
>>> bit
>>>                                         nothing to do with reassembly.
>>>
>>>                                         Appropriate MTU configuration
>>>                                         consistent manner within an SFC
>>>                                         maximum transmission unit
>>>                                         specification raises two concerns
>>>                                         and one comment:
>>>
>>>                                         fragmentation-related pointers
>>> to:
>>>                                         e.g., RFC1858 (Security
>>>                                                 d'origine----- De : sfc
>>>                                                 comments and that there
>>>                                                 can go directly to the

---

From: "Jim Guichard (jguichar)" <[email-masked]>
Date: Thu, 28 Apr 2016 13:22:31 +0000

Hi Andy,

I think the key point is that NSH != network overlay but rather utilizes a network overlay for packet transport between SFC network elements. The reference is just an example of how a network overlay might deal with fragmentation and is not a suggestion that NSH adopt that mechanism but rather makes the point that it can utilize the existing network overlay mechanics.

Jim

From: sfc <[email-masked]<mailto:[email-masked]>> on behalf of "Andrew G. Malis" <[email-masked]<mailto:[email-masked]>>
Date: Thursday, April 28, 2016 at 8:17 AM
To: Joel Halpern Direct <[email-masked]<mailto:[email-masked]>>
Cc: "Elzur, Uri" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, Linda Dunbar <[email-masked]<mailto:[email-masked]>>
Subject: Re: [sfc] Suggested wording addtion to the Section 6 (Fragmentation Consideration) of the draft-ietf-sfc-nsh-04.txt

Joel,

The diagrams in section 6 of RFC 6830 are control plane, not data plane. The data plane diagrams are in section 5.

But the overriding question is - without any fields in the NSH header to sequence fragments, how can the fragments be correctly reassembled?

Cheers,
Andy

On Wed, Apr 27, 2016 at 7:46 PM, Joel Halpern Direct <[email-masked]<mailto:[email-masked]>> wrote:
The LISP header does not have a fragment flag or fragment offset.  The diagrams in section 6 include the outer encapsulating header (the equivalent of the transport header in SFC.)  Yes, the text is a little hard to follow in this regard.  The LISP working group is going to be rewriting RFC 6830 as part of moving to standards track.

Yours,
Joel

However, the NSH Header doesn’t have a Fragment Offset field or any
other way to order the fragments.

Thanks,
Andy

On Wed, Apr 27, 2016 at 4:19 PM, Joel M. Halpern <[email-masked]<mailto:[email-masked]>
<mailto:[email-masked]<mailto:[email-masked]>>> wrote:

    Yours,
    Joel

        By the way, there are to typo in the Section 6:
        3rd line: "In order the" ==> "In order to"
        Last line of Bullet 2: extra "the" in the "utilized to ensure
        the the required packet".

        Hope it helps.

        Linda

        -----Original Message-----
        From: [email-masked]<mailto:[email-masked]>
        <mailto:[email-masked]<mailto:[email-masked]>>
        [mailto:[email-masked]<mailto:[email-masked]>
        <mailto:[email-masked]<mailto:[email-masked]>>]
        Sent: Wednesday, April 27, 2016 12:40 AM
        To: Joel M. Halpern; Linda Dunbar; Elzur, Uri; [email-masked]<mailto:[email-masked]>
        <mailto:[email-masked]<mailto:[email-masked]>>
        Subject: RE: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

        Hi Joel, all,

        Please see inline.

        Cheers,
        Med

            -----Message d'origine-----
            De : Joel M. Halpern [mailto:[email-masked]<mailto:[email-masked]>
            <mailto:[email-masked]<mailto:[email-masked]>>] Envoyé : mardi 26
            avril 2016 19:18 À : Linda Dunbar; BOUCADAIR Mohamed
            IMT/OLN; Elzur,
            Uri; [email-masked]<mailto:[email-masked]> <mailto:[email-masked]<mailto:[email-masked]>> Objet : Re: [sfc] WG
            last call for
            draft-ietf-sfc-nsh-04.txt

            With regard to Transport tunnel fragementation, that seems
            an issue
            for the transport protocol.  I don't actually object to a
            sentence,
            but it does not seem that it will accomplish anything.

            With regard to packets fragmented by the source, I
            completely disagree
            with your assertion.  If an SFF were to reassemble the
            packets, that
            would be a violation of its job.

        [Med] I agree with you.

            reassemble a packet fragmented by the source.  The
            classifier may have
            to do some interesting things in order to properly classify
            succeeding
            fragments, but that is an implementation issue (most commonly
            addressed with virtual reassembly, which doe snot delay the
            fragments.)  We don't specity that.

            If an SF needs to reassemble fragments to do its job, that
            is up to
            the SF.  Some will need to actually reassemble.  Some will
            need to
            perform virtual reassembly, and some will happily process the
            fragments.  I can not see what the NSH document could
            possibly mandate.

        [Med] Fully agree.

            Yours,
            Joel

                Joel,

                - Transport tunnel generated fragmentation: When a packet
                fragmentation is caused by transport tunnel (i.e. various
                encapsulations), the termination point of the transport
                tunnel is
                responsible for re-assembling the fragmented pieces of
                the packet.
                Since there won't be any SFF nodes in between the
                Transport Tunnel,
                the tunnel generated fragmentation does not require any
                actions by
                SFF nodes or SF nodes.

                - Source node generated fragmentation (after adding on
                the NSH
                header): When fragmentation has to be performed for a
                packet being
                encapsulated with the NSH header, either the
                intermediate SFF nodes
                or the SF nodes have to be able to reassemble the
                fragmented pieces.

                Cheers,

                Linda

                -----Original Message----- From: Joel M. Halpern
                [mailto:[email-masked]<mailto:[email-masked]>
                <mailto:[email-masked]<mailto:[email-masked]>>] Sent: Tuesday, April 26,
                2016 10:33 AM
                To: Linda Dunbar; [email-masked]<mailto:[email-masked]>
                <mailto:[email-masked]<mailto:[email-masked]>>; Elzur, Uri;
                [email-masked]<mailto:[email-masked]> <mailto:[email-masked]<mailto:[email-masked]>> Subject: Re: [sfc] WG
                last call for
                draft-ietf-sfc-nsh-04.txt

                Re-reading your note, it is possible that you are
                referring only to
                the case of transport generated fragmentation of the
                outer packet.
                I had assumed you were not talking about that, since the
                resulting
                fragments will not all have NSH headers.  As with any tunnel
                technology, if the tunnel chooses to fragment at its
                layer, then the
                tunnel is responsible for reassembly.  That would be
                invisible to
                the SFF.

                Yours, Joel

                    Agree with Med.

                    Even if each fragment piece of a packet with NSH
                    header carries the
                    NSH header, the intermediate SFF nodes still need to
                    put together
                    all the fragments together before passing the whole
                    data frame to
                    the SF.

                    Linda

                    -----Original Message----- From: sfc
                    [mailto:[email-masked]<mailto:[email-masked]>
                    <mailto:[email-masked]<mailto:[email-masked]>>]
                    On Behalf Of [email-masked]<mailto:[email-masked]>
                    <mailto:[email-masked]<mailto:[email-masked]>> Sent: Monday,
                    April 25,
                    2016 9:42 AM To: Elzur, Uri; Joel M. Halpern;
                    [email-masked]<mailto:[email-masked]> <mailto:[email-masked]<mailto:[email-masked]>> Subject:
                    Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

                    Re-,

                    How do you instruct the transport layer to ALWAYS
                    prepend an NSH
                    header even for fragments? Or you don't care about that?

                    Thank you.

                    Cheers, Med

                        -----Message d'origine----- De : Elzur, Uri
                        [mailto:[email-masked]<mailto:[email-masked]>
                        <mailto:[email-masked]<mailto:[email-masked]>>] Envoyé : lundi 25
                        avril 2016 16:26 À
                        : BOUCADAIR Mohamed IMT/OLN; Joel M. Halpern;
                        [email-masked]<mailto:[email-masked]> <mailto:[email-masked]<mailto:[email-masked]>> Objet
                        : RE: [sfc] WG last call for
                        draft-ietf-sfc-nsh-04.txt

                        Hi Med

                        Not to repeat my position, but I'll do it anyhow
                        :-) As NSH is
                        *NOT* a transport, dealing with fragmentation is
                        left to the
                        Transport used.

                        Thx

                        Uri ("Oo-Ree") C: 949-378-7568<tel:949-378-7568> <tel:949-378-7568<tel:949-378-7568>>

                        -----Original Message----- From: sfc
                        [mailto:[email-masked]<mailto:[email-masked]>
                        <mailto:[email-masked]<mailto:[email-masked]>>]
                        On Behalf Of [email-masked]<mailto:[email-masked]>
                        <mailto:[email-masked]<mailto:[email-masked]>> Sent:
                        Monday, April 25,
                        2016 7:18 AM To: Joel M. Halpern
                        <[email-masked]<mailto:[email-masked]> <mailto:[email-masked]<mailto:[email-masked]>>>;
                        [email-masked]<mailto:[email-masked]> <mailto:[email-masked]<mailto:[email-masked]>>
                        Subject: Re: [sfc] WG last call for
                        draft-ietf-sfc-nsh-04.txt

                        Hi Joel,

                        Please see inline.

                        Cheers, Med

                            -----Message d'origine----- De : Joel M. Halpern
                            [mailto:[email-masked]<mailto:[email-masked]>
                            <mailto:[email-masked]<mailto:[email-masked]>>] Envoyé : lundi
                            25 avril 2016 15:48 À
                            : BOUCADAIR Mohamed IMT/OLN; [email-masked]<mailto:[email-masked]>
                            <mailto:[email-masked]<mailto:[email-masked]>> Objet : Re: [sfc] WG
                            last call for draft-ietf-sfc-nsh-04.txt

                            If I am understanding you correctly Med, you
                            are asking that the
                            NSH draft specify how service chaining
                            should cope with packets
                            that have been fragmented?

                        [Med] To be accurate, I'm asking to assess
                        whether there are
                        implications. If there are, then the draft
                        should be updated
                        accordingly.

                        * SFF: If the NSH header is present only in the
                        a fragment but SFF
                        didn't maintained a state, subsequent fragments
                        won't be
                        appropriately processed. * SFC-aware function:
                        if prepending a
                        context information depends on the full packet,
                        not only a
                        fragment.

                        It just does its job.

                            Ingress and intermediate classifiers may
                            cope with fragments in
                            any number of ways.

                        Specifying such is clearly out of scope for this
                        draft.

                        [Med] The purpose is not to specify the internal
                        implementation
                        details but the external behavior of the
                        classifier function when
                        it comes to handle fragments. That behavior may
                        have an incidence
                        on SFF, in particular. The purpose is not to
                        recommend the maximum
                        resources to be dedicated to out of order
                        fragments nor the
                        timeout to cache those; these considerations are
                        of course out of
                        scope. Nevertheless, an implementation should
                        offer a configurable
                        parameter so that an operator tweak those
                        according to its
                        context.

                            I suppose one could write an informational
                            draft on possible ways
                            of coping.  The IETF has not usually
                            published such.
                            Service functions have to cope with
                            fragmented packets just as
                            they had to before the advent of NSH, so
                            describing that is
                            clearly not needed here.

                        [Med] The advent of NSH has the following
                        implications: * it
                        exacerbates fragmentation. Handing over this
                        issue to the
                        transport layer may lead to interoperability
                        issues. * the
                        chaining may not be efficient if fragments are
                        inappropriately
                        handled.

                            Yours, Joel

                            On 4/25/16 3:00 AM,
                            [email-masked]<mailto:[email-masked]>
                            <mailto:[email-masked]<mailto:[email-masked]>> wrote:

                                Re-,

                                (1) Fragmentation that is caused by
                                prepending an SFC header.
                                (2) Handling fragments at the ingress of
                                an SFC-enabled domain.

                                There are other issues that need to be
                                discussed, e.g., how to
                                deal with

                            to handle fragments. If an SFC header is
                            prepended for all
                            fragments, I'm not sure there

                            context TLVs depends on the full packet (not
                            just fragment). It
                            is warranted to consider a little bit this
                            point before declaring
                            there is no issue.

                            an implementation must be prepared to
                            receive fragments with or
                            without NSH header as this is a decision
                            that is left to the
                            transport encapsulation. This is a
                            requirement per se!

                                I won't reiterate all the comments I
                                have about the current
                                wording of

                            this section.

                                Cheers, Med

                                    -----Message d'origine----- De :
                                    Elzur, Uri
                                    [mailto:[email-masked]<mailto:[email-masked]>
                                    <mailto:[email-masked]<mailto:[email-masked]>>] Envoyé
                                    : lundi 25 avril 2016
                                    08:30 À : BOUCADAIR Mohamed IMT/OLN;
                                    Thomas Narten;
                                    [email-masked]<mailto:[email-masked]> <mailto:[email-masked]<mailto:[email-masked]>>
                                    Objet : RE: [sfc] WG last call for
                                    draft-ietf-sfc-nsh-04.txt

                                    Hi Med

                                    My point is that Fragmentation is
                                    yet another transport related
                                    issue

                            that

                            doesn't

                                    really belong in the draft. We are
                                    providing an advice as
                                    extending the size of the packet may
                                    lead to fragmentation, but
                                    as you check RFC 7348 VxLAN, which
                                    my create the same issue,
                                    you'll find it doesn't even

                            relate

                                    to it.

                                    Thx

                                    Uri ("Oo-Ree") C: 949-378-7568<tel:949-378-7568>
                                    <tel:949-378-7568<tel:949-378-7568>>

                                    -----Original Message----- From: sfc
                                    [mailto:[email-masked]<mailto:[email-masked]>
                                    <mailto:[email-masked]<mailto:[email-masked]>>] On
                                    Behalf Of
                                    [email-masked]<mailto:[email-masked]>
                                    <mailto:[email-masked]<mailto:[email-masked]>> Sent:
                                    Sunday, April 24, 2016
                                    10:32 PM To: Elzur, Uri
                                    <[email-masked]<mailto:[email-masked]>
                                    <mailto:[email-masked]<mailto:[email-masked]>>>;
                                    Thomas Narten

                            <[email-masked]<mailto:[email-masked]> <mailto:[email-masked]<mailto:[email-masked]>>>;

                                    [email-masked]<mailto:[email-masked]> <mailto:[email-masked]<mailto:[email-masked]>>
                                    Subject: Re: [sfc] WG last call for
                                    draft-ietf-sfc-nsh-04.txt

                                    Hi Uri,

                                    That's another option that needs to
                                    be discussed/investigated.
                                    I'm

                            afraid

                                    this is not the rationale adopted in
                                    -04 since it includes some
                                    text

                            that

                                    BTW, saying that nsh does not need
                                    to deal with fragmentation
                                    because

                            it

                                    is transport-independent is not IMHO
                                    a good strategy to adopt
                                    here

                            because

                                    it opens the door for interoperable
                                    issues, it may lead to
                                    sub-optimal implementations if the
                                    sfc information is present
                                    only in one

                            fragments,

                                    etc.

                            to

                                    be fixed somehow.

                                    Cheers, Med

                                        -----Message d'origine----- De :
                                        Elzur, Uri
                                        [mailto:[email-masked]<mailto:[email-masked]>
                                        <mailto:[email-masked]<mailto:[email-masked]>>]
                                        Envoyé : dimanche 24 avril
                                        2016 17:36 À : BOUCADAIR Mohamed
                                        IMT/OLN; Thomas Narten;
                                        [email-masked]<mailto:[email-masked]>
                                        <mailto:[email-masked]<mailto:[email-masked]>> Objet :
                                        RE: [sfc] WG last call for
                                        draft-ietf-sfc-nsh-04.txt

                                        Hi Med

                                        I see no need to specify the
                                        exact behavior as NSH is
                                        transport independent i.e. like
                                        NSH interaction with any other
                                        Transport eh issue of
                                        Fragmentation is to be dealt in
                                        a way
                                        that matches the mechanisms
                                        supported by the Transport used
                                        and do not belong in the NSH draft

                                        Thx

                                        Uri ("Oo-Ree") C: 949-378-7568<tel:949-378-7568>
                                        <tel:949-378-7568<tel:949-378-7568>>

                                        -----Original Message----- From: sfc
                                        [mailto:[email-masked]<mailto:[email-masked]>
                                        <mailto:[email-masked]<mailto:[email-masked]>>]
                                        On Behalf Of
                                        [email-masked]<mailto:[email-masked]>
                                        <mailto:[email-masked]<mailto:[email-masked]>>
                                        Sent: Thursday, April 14,
                                        2016 12:43 AM To: Thomas Narten
                                        <[email-masked]<mailto:[email-masked]>
                                        <mailto:[email-masked]<mailto:[email-masked]>>>;
                                        [email-masked]<mailto:[email-masked]>
                                        <mailto:[email-masked]<mailto:[email-masked]>> Subject:
                                        Re: [sfc] WG last call for
                                        draft-ietf-sfc-nsh-04.txt

                                        R-,

                                        In addition to other pending
                                        issues raised for this draft, I
                                        would like to raise this
                                        additional one about Section 6.

                                        NSH and the associated transport
                                        header are "added" to the
                                        encapsulated packet/frame.  This
                                        additional information
                                        increases

                            the

                                        size of the packet.  In order
                                        the ensure proper forwarding of
                                        NSH data, several options for
                                        handling fragmentation and
                                        re-assembly exist:

                                        1.  Jumbo Frames, when
                                        supported, enable the transport
                                        of NSH
                                        and associated transport packets
                                        without requiring
                                        fragmentation.

                                        2.  Path MTU Discovery
                                        [RFC1191]"describes a technique for
                                        dynamically discovering the
                                        maximum transmission unit (MTU) of

                            an

                                        arbitrary internet path" and can
                                        be utilized to ensure the

                        the

                            assembly

                                        in section 5.4. ==

                                        * The text is weak for a

                        2.12.

                                        There should be a clear behavior
                                        to be followed by an

                        implementation.

                                        Further, I would avoid the use
                                        of words such as "can".

                                        * The text covers only
                                        fragmentation when it is induced
                                        by SFC
                                        operations, it does not discuss
                                        the treatment of a fragment
                                        when received in an SFC domain.
                                        IMHO, the draft should also
                                        specify the behavior of the
                                        Classifier with regards to
                                        fragments for the sake of proper
                                        SFC operation. Applying
                                        classification policies may
                                        require the

                                        In particular, dedicated
                                        resources should dedicated for
                                        handling out of order fragments.
                                        Of course, it is out of scope
                                        of this document to describe how
                                        SFs handle fragments.

                                        * If an SFC header is prepended
                                        for all fragments, I'm not
                                        sure there is any particular
                                        issue at the SFF level...except
                                        if stripping/adding context TLVs
                                        depends on the full packet
                                        (not just fragment). It is
                                        warranted to consider a little bit
                                        this point before declaring there

                            is

                                    no issue.

                                        * The text states "several
                                        options". This may be interpreted
                                        as if implementing one of them
                                        is sufficient...which is not
                                        true. The first two points
                                        contribute to minimize the
                                        fragmentation risk, but
                                        fragmentation may still be
                                        experienced
                                        (e.g., other shims are prepended
                                        by other nodes for some other
                                        purposes, nested nsh, etc.)

                                        * The support of jumbo frames by
                                        a router/device does not mean
                                        that it can make use of it.
                                        Appropriate MTU configuration
                                        should be undertaken in a
                                        consistent manner within an SFC
                                        domain. The text should be
                                        updated to make it is about
                                        (consistent) MTU

                        configuration.

                                        * BTW, shouldn't the text be
                                        reworded to recommended to
                                        increase the MTU of **all
                                        nodes** of an SFC-enabled domain by
                                        at least the length of SFC
                                        header + transport header?

                                        * Bullet 2, how PMTU discovery
                                        is actually used in this
                                        context? Do you assume that all
                                        SFC-aware nodes will issue
                                        such messages towards other
                                        SFC-aware node, arbitrary
                                        destination, else?

                                        * Bullet 2, I would drop
                                        "describes a technique for
                                        dynamically discovering the
                                        maximum transmission unit
                                        (MTU) of an arbitrary internet
                                        path".

                                        (1) I don't think it is a good
                                        approach that fragments induced
                                        by the network are passed to
                                        their ultimate destinations as
                                        such

                        (stateless).

                                        IMO, reassembly should be done
                                        within the SFC domain
                                        (responsible for fragmentation)
                                        not passed away. (2) Does the
                                        stateful mode require all SFC
                                        data plane elements maintain a
                                        full list of MTU for the any
                                        SFF/SF instance of the SFC

                                    domain?

                                        The current text suggests that
                                        [RFC6830] should be listed as
                                        normative reference (not an
                                        informative one). I would
                                        personally favor removing the
                                        reference to LISP (as it is an
                                        Experimental RFC).

                                        * The security section of the
                                        draft may be augmented with
                                        informational
                                        fragmentation-related pointers to:
                                        e.g., RFC1858 (Security
                                        Considerations for IP Fragment
                                        Filtering), RFC3128 (Protection
                                        Against a Variant of the Tiny
                                        Fragment Attack), and RFC 4963
                                        (IPv4 Reassembly Errors at High
                                        Data Rates).

                                        Cheers, Med

                                            -----Message d'origine-----
                                            De : sfc
                                            [mailto:[email-masked]<mailto:[email-masked]>
                                            <mailto:[email-masked]<mailto:[email-masked]>>]
                                            De la part de
                                            [email-masked]<mailto:[email-masked]>
                                            <mailto:[email-masked]<mailto:[email-masked]>>
                                            Envoyé : lundi 11 avril
                                            2016 13:14 À : Thomas
                                            Narten; [email-masked]<mailto:[email-masked]>
                                            <mailto:[email-masked]<mailto:[email-masked]>> Objet
                                            : Re:
                                            [sfc] WG last call for
                                            draft-ietf-sfc-nsh-04.txt

                                            Dear Thomas, all,

                                            As I mentioned during the
                                            meeting, there are several
                                            issues
                                            that are not covered in the
                                            last version of the draft. I
                                            already provided examples of
                                            the issues offline as requested
                                            by Martin.

                                            Cheers, Med

                                                -----Message
                                                d'origine----- De : sfc
                                                [mailto:[email-masked]<mailto:[email-masked]>
                                                <mailto:[email-masked]<mailto:[email-masked]>>]
                                                De la part de Thomas Narten
                                                Envoyé : jeudi 31 mars
                                                2016 04:48 À :
                                                [email-masked]<mailto:[email-masked]>
                                                <mailto:[email-masked]<mailto:[email-masked]>>
                                                Objet : [sfc] WG last
                                                call for
                                                draft-ietf-sfc-nsh-04.txt

                                                Dear WG:

                                                This note begins a WG
                                                last call on

                                                addressed all known
                                                comments and that there
                                                are no open
                                                issues with the current
                                                version of the document.

                                                Substantive comments to
                                                the list please,
                                                editorial comments
                                                can go directly to the
                                                document editors.

                                                We'll also get a brief
                                                update from the editors
                                                at next
                                                week's meeting. If there
                                                are any remaining issues
                                                with the
                                                document, raising them
                                                before the meeting would be
                                                especially helpful.

                                                For the chairs, Thomas

                                                _______________________________________________
                                                sfc mailing
                                                list [email-masked]<mailto:[email-masked]>
                                                <mailto:[email-masked]<mailto:[email-masked]>>
                                                https://www.ietf.org/mailman/listinfo/sfc

                                            _______________________________________________
                                            sfc mailing
                                            list [email-masked]<mailto:[email-masked]>
                                            <mailto:[email-masked]<mailto:[email-masked]>>
                                            https://www.ietf.org/mailman/listinfo/sfc

                                        _______________________________________________
                                        sfc mailing
                                        list [email-masked]<mailto:[email-masked]>
                                        <mailto:[email-masked]<mailto:[email-masked]>>
                                        https://www.ietf.org/mailman/listinfo/sfc

---

From: "Andrew G. Malis" <[email-masked]>
Date: Thu, 28 Apr 2016 11:02:10 -0400

Jim,

The way option 3 is currently worded, there’s no indication that this is
just an example, and not a direction to use RFC 6830, section 5.4. I just
had an offline discussion with Joel, and he and I agree that a better
wording for option 3 would be "Use the fragmentation provided by the
network overlay mechanics. One example can be found in RFC 6830, section
5.4.”

Thanks,
Andy

>>>                                     <[email-masked]

---

From: Joel Halpern Direct <[email-masked]>
Date: Thu, 28 Apr 2016 11:05:35 -0400

Thanks Andy.  That wording would indeed work for me.
Yours,
Joel

On 4/28/16 11:02 AM, Andrew G. Malis wrote:
> just an example, and not a direction to use RFC 6830, section 5.4. I
> just had an offline discussion with Joel, and he and I agree that a
> better wording for option 3 would be "Use the fragmentation provided by
> the network overlay mechanics. One example can be found in RFC 6830,
> section 5.4.”
>
> Thanks,
> Andy
>
> On Thu, Apr 28, 2016 at 9:22 AM, Jim Guichard (jguichar)
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>     Hi Andy,
>
>     I think the key point is that NSH != network overlay but rather
>     utilizes a network overlay for packet transport between SFC network
>     elements. The reference is just an example of how a network overlay
>     might deal with fragmentation and is not a suggestion that NSH adopt
>     that mechanism but rather makes the point that it can utilize the
>     existing network overlay mechanics.
>
>     Jim
>
>     From: sfc <[email-masked] <mailto:[email-masked]>> on
>     behalf of "Andrew G. Malis" <[email-masked]
>     <mailto:[email-masked]>>
>     Cc: "Elzur, Uri" <[email-masked] <mailto:[email-masked]>>,
>     "[email-masked] <mailto:[email-masked]>"
>     <[email-masked]
>     <mailto:[email-masked]>>, "[email-masked]
>     <mailto:[email-masked]>" <[email-masked] <mailto:[email-masked]>>, Linda
>     Dunbar <[email-masked] <mailto:[email-masked]>>
>     The diagrams in section 6 of RFC 6830 are control plane, not data
>     plane. The data plane diagrams are in section 5.
>
>     But the overriding question is - without any fields in the NSH
>     header to sequence fragments, how can the fragments be correctly
>     reassembled?
>
>     Cheers,
>     Andy
>
>         The LISP header does not have a fragment flag or fragment
>         offset.  The diagrams in section 6 include the outer
>         encapsulating header (the equivalent of the transport header in
>         SFC.)  Yes, the text is a little hard to follow in this regard.
>         The LISP working group is going to be rewriting RFC 6830 as part
>         of moving to standards track.
>
>             When a packet
>             Joel M. Halpern
>             Envoyé
>             <mailto:[email-masked] <mailto:[email-masked]>>] On
>                                                 Behalf Of
>
>             <mailto:[email-masked] <mailto:[email-masked]>>>;
>                                                 Thomas Narten
>
>                                         <[email-masked]
>             independent i.e. like
>                                                     Sent: Thursday,
>             April 14,
>                                                     header are "added"
>                                                     be utilized to
>             sufficient...which is not
>                                                     true. The first two
>             points
>                                                     contribute to
>             minimize the
>                                                     fragmentation risk, but
>                                                     fragmentation may
>             still be
>                                                     experienced
>                                                     (e.g., other shims
>             are prepended
>                                                     by other nodes for
>             some other
>                                                     purposes, nested
>             nsh, etc.)
>
>                                                     * The first two
>             points have
>                                                     nothing to do with
>             reassembly.
>
>                                                     * The support of
>             jumbo frames by
>                                                     a router/device does
>             not mean
>                                                     that it can make use
>             of it.
>                                                     Appropriate MTU
>             configuration
>                                                     should be undertaken
>             in a
>                                                     consistent manner
>             within an SFC
>                                                     domain. The text
>             should be
>                                                     updated to make it
>             is about
>                                                     (consistent) MTU
>
>                                     configuration.
>
>
>                                                     * BTW, shouldn't the
>             text be
>                                                     reworded to
>             recommended to
>                                                     increase the MTU of
>             **all
>                                                     nodes** of an
>             SFC-enabled domain by
>                                                     at least the length
>             of SFC
>                                                     header + transport
>             header?
>
>                                                     * Bullet 2, how PMTU
>             discovery
>                                                     is actually used in this
>                                                     context? Do you
>             assume that all
>                                                     SFC-aware nodes will
>             issue
>                                                     such messages
>             towards other
>                                                     SFC-aware node,
>             arbitrary
>                                                     destination, else?
>
>                                                     * Bullet 2, I would drop
>                                                     "describes a
>             technique for
>                                                     dynamically
>             discovering the
>                                                     maximum transmission
>             unit
>                                                     (MTU) of an
>             arbitrary internet
>                                                     path".
>
>                                                     * Bullet 2, s/ the
>             the/the.
>
>                                                     * The reference to
>             the LISP
>                                                     specification raises
>             two concerns
>                                                     and one comment:
>
>                                                     (1) I don't think it
>             is a good
>                                                     approach that
>             fragments induced
>                                                     by the network are
>             passed to
>                                                     their ultimate
>             destinations as
>                                                     such
>
>                                     (stateless).
>
>                                                     IMO, reassembly
>             should be done
>                                                     within the SFC domain
>                                                     (responsible for
>             fragmentation)
>                                                     not passed away. (2)
>             Does the
>                                                     stateful mode
>             require all SFC
>                                                     data plane elements
>             maintain a
>                                                     full list of MTU for
>             the any
>                                                     SFF/SF instance of
>             the SFC
>
>                                                 domain?
>
>
>                                                     The current text
>             suggests that
>                                                     [RFC6830] should be
>             listed as
>                                                     normative reference
>             (not an
>                                                     informative one). I
>             would
>                                                     personally favor
>             removing the
>                                                     reference to LISP
>             (as it is an
>                                                     Experimental RFC).
>
>                                                     * The security
>             section of the
>                                                     draft may be
>             augmented with
>                                                     informational
>
>             d'origine-----
>                                                         Envoyé : lundi
>             11 avril
>                                                         2016 13:14 À :
>                                                         Narten;
>                                                         [sfc] WG last
>                                                             Envoyé :
>             jeudi 31 mars
>                                                             Objet :
>             [sfc] WG last
>                                                             This note
>             begins a WG
>                                                             addressed
>             all known
>             the current
>                                                             version of
>                                                             editorial
>             comments
>                                                             can go
>             directly to the
>                                                             document
>             editors.
>                                                             We'll also
>             get a brief
>                                                             update from
>             the editors
>             meeting. If there
>             meeting would be
>                                                             especially
>             helpful.
>                                                             For the
>             chairs, Thomas

---

From: "Jim Guichard (jguichar)" <[email-masked]>
Date: Thu, 28 Apr 2016 17:11:15 +0000

That wording works for me.

Paul/Uri, as editors of draft-ietf-sfc-nsh-04, unless there are any
objections from the WG or yourselves, can you please make the suggested
change to the text?

Jim

On 4/28/16, 11:05 AM, "Joel Halpern Direct" <[email-masked]>
wrote:

>> The way option 3 is currently worded, there¹s no indication that this is
>> section 5.4.²
>>             section 5.4 of RFC 6830 (presumably with the ³NSH header²
>>             replacing the
>>             ³LISP header² in the description of the procedures in that
>>             section).
>>
>>             However, the NSH Header doesn¹t have a Fragment Offset field
>>             or any
>>                                                 my create the same
>>issue,
>>                                                     IMT/OLN; Thomas 
>>                                                     exact behavior as 
>>NSH is
>>                                                     Transport eh issue 
>>                                                     would like to raise 
>>                                                     additional 
>>information
>>                                                     1.  Jumbo Frames, 
>>when
>>                                                     2.  Path MTU 
>>Discovery
>>                                                     * The text is weak 
>>                                                     of words such as 
>>"can".
>>                                                     * The text covers 
>>only
>>                                                     SFC operation. 
>>Applying
>>                                                     In particular, 
>>dedicated
>>             describe how
>>                                                     SFs handle 
>>                                                     fragmentation risk, 
>>but
>>                                                     is actually used in 
>>                                                     * Bullet 2, I would 
>>drop
>>                                                     within the SFC 
>>domain
>>                                                     e.g., RFC1858 
>>(Security
>>                                                         Dear Thomas, 
>>all,
>>                                                             2016 04:48 
>>À :
>>                                                             the list 
>>please,

---

From: Sunil Vallamkonda <[email-masked]>
Date: Thu, 28 Apr 2016 18:47:46 +0000

Paul, Ron,

As we see the below exposes the requirement of a specification framework to address interoperability and scale across domains both for MDType 1 and 2 metadata.
The use case scenarios can be challenging without this making it hard for massive deployments and NSH as a pure traffic steering solution while other solutions used in production today.
The semantics document proposal: draft-vallamkonda-sfc-metadata-model-00 precisely addresses this requirement and eliminates ambiguity with IANA registry.
It also aligns well with draft-ietf-sfc-control-plane-04.txt, with draft-ietf-sfc-nsh-04 and RFC7665.

Thank you,
Sunil.
-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Paul Quinn (paulq)
Sent: Tuesday, April 12, 2016 6:03 PM
To: Ron Parker <[email-masked]>
Cc: Thomas Narten <[email-masked]>; [email-masked]; Dave Dolson <[email-masked]>
Subject: Re: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

Ron, Dave,

1.  Agree on the envelop and the use of that envelop 2.  As a WG, adopt --> standardize the opaque data, and the TLVs 3.  As a WG, work on more detailed of use of the control plane to, when needed, define semantics

Thoughts?

Thanks,
Paul

> https://tools.ietf.org/html/draft-vallamkonda-sfc-metadata-model-00.tx
> t https://www.ietf.org/archive/id/draft-wang-sfc-md-type-issues-00.txt

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 28 Apr 2016 14:50:53 -0400

As has been discussed in several contexts, we need (and will have) a 
registry for MD-2 Type values (as called for in the draft.)

I am still unable to see what this framework adds beyond that registry, 
or where the YANG model would be used.

Yours,
Joel

---

From: Sunil Vallamkonda <[email-masked]>
Date: Thu, 28 Apr 2016 18:54:32 +0000

Jim,

I agree, the confusion being NSH treated as a network overlay.
Would it be possible for the document to have wording on this ?
This could also clarify interpretations how NSH to be used with ARP, DHCP and encapsulations.

Thank you,
Sunil

From: sfc [mailto:[email-masked]] On Behalf Of Jim Guichard (jguichar)
Sent: Thursday, April 28, 2016 6:23 AM
To: Andrew G. Malis <[email-masked]>; Joel Halpern Direct <[email-masked]>
Cc: Elzur, Uri <[email-masked]>; [email-masked]; [email-masked]; Linda Dunbar <[email-masked]>
Subject: Re: [sfc] Suggested wording addtion to the Section 6 (Fragmentation Consideration) of the draft-ietf-sfc-nsh-04.txt

Hi Andy,

Jim

Joel,

Cheers,
Andy

Yours,
Joel

All this talk about fragmentation prompted me to re-read section 6 of
the draft, which recommends (as option 3) using the procedures in
section 5.4 of RFC 6830 (presumably with the "NSH header" replacing the
"LISP header" in the description of the procedures in that section).

However, the NSH Header doesn't have a Fragment Offset field or any
other way to order the fragments.

Thanks,
Andy

    Yours,
    Joel

        Hope it helps.

        Linda

        -----Original Message-----
        From: [email-masked]<mailto:[email-masked]>
        <mailto:[email-masked]<mailto:[email-masked]>>
        [mailto:[email-masked]<mailto:[email-masked]>
        <mailto:[email-masked]<mailto:[email-masked]>>]
        Sent: Wednesday, April 27, 2016 12:40 AM
        To: Joel M. Halpern; Linda Dunbar; Elzur, Uri; [email-masked]<mailto:[email-masked]>
        <mailto:[email-masked]<mailto:[email-masked]>>
        Subject: RE: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

        Hi Joel, all,

        Please see inline.

        Cheers,
        Med

            -----Message d'origine-----

        [Med] I agree with you.

        [Med] Fully agree.

            Yours,
            Joel

                Joel,

                Cheers,

                Linda

                Yours, Joel

                    Agree with Med.

                    Linda

                    Re-,

                    Thank you.

                    Cheers, Med

                        Hi Med

                        Thx

                        Hi Joel,

                        Please see inline.

                        Cheers, Med

                        It just does its job.

                            Yours, Joel

                            On 4/25/16 3:00 AM,

                                Re-,

                            this section.

                                Cheers, Med

                                    Hi Med

                            that

                            doesn't

                            relate

                                    to it.

                                    Thx

                                    Hi Uri,

                            afraid

                            that

                            it

                            because

                            fragments,

                                    etc.

                            to

                                    be fixed somehow.

                                    Cheers, Med

                                        Hi Med

                                        Thx

                                        R-,

                            the

                                        1.  Jumbo Frames, when

                                        2.  Path MTU Discovery

                            an

                        the

                            assembly

                                        in section 5.4. ==

                                        * The text is weak for a

                        2.12.

                        implementation.

                                        * The text covers only

                                        In particular, dedicated

                            is

                                    no issue.

                        configuration.

                                        * Bullet 2, I would drop

                        (stateless).

                                    domain?

                                        Cheers, Med

                                            -----Message d'origine-----
                                            De : sfc

                                            Dear Thomas, all,

                                            Cheers, Med

                                                -----Message
                                                d'origine----- De : sfc

                                                Dear WG:

                                                This note begins a WG
                                                last call on

                                                addressed all known
                                                comments and that there
                                                are no open
                                                issues with the current
                                                version of the document.

                                                Substantive comments to
                                                the list please,
                                                editorial comments
                                                can go directly to the
                                                document editors.

                                                We'll also get a brief
                                                update from the editors
                                                at next
                                                week's meeting. If there
                                                are any remaining issues
                                                with the
                                                document, raising them

                                                For the chairs, Thomas

---

From: "Paul Quinn (paulq)" <[email-masked]>
Date: Thu, 28 Apr 2016 20:28:56 +0000

Jim, Andy,

That works for me.  We'll mark it for inclusion in the next revision.

Paul

> On Apr 28, 2016, at 10:11 AM, Jim Guichard (jguichar) <[email-masked]> wrote:
>

---

From: "Jim Guichard (jguichar)" <[email-masked]>
Date: Fri, 29 Apr 2016 15:47:34 +0000

Hi Sunil,

I think the suggested text to add is "Use the fragmentation provided by the network overlay mechanics. One example can be found in RFC 6830, section 5.4.” - hope this clarifies.

Jim

From: Sunil Vallamkonda <[email-masked]<mailto:[email-masked]>>
Date: Thursday, April 28, 2016 at 2:54 PM
To: Jim Guichard <[email-masked]<mailto:[email-masked]>>, "Andrew G. Malis" <[email-masked]<mailto:[email-masked]>>, Joel Halpern Direct <[email-masked]<mailto:[email-masked]>>
Cc: "Elzur, Uri" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, Linda Dunbar <[email-masked]<mailto:[email-masked]>>
Subject: RE: [sfc] Suggested wording addtion to the Section 6 (Fragmentation Consideration) of the draft-ietf-sfc-nsh-04.txt

Jim,

Thank you,
Sunil

From: sfc [mailto:[email-masked]] On Behalf Of Jim Guichard (jguichar)
Sent: Thursday, April 28, 2016 6:23 AM
To: Andrew G. Malis <[email-masked]<mailto:[email-masked]>>; Joel Halpern Direct <[email-masked]<mailto:[email-masked]>>
Cc: Elzur, Uri <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; Linda Dunbar <[email-masked]<mailto:[email-masked]>>
Subject: Re: [sfc] Suggested wording addtion to the Section 6 (Fragmentation Consideration) of the draft-ietf-sfc-nsh-04.txt

Hi Andy,

Jim

Joel,

Cheers,
Andy

Yours,
Joel

Thanks,
Andy

    Yours,
    Joel

        Hope it helps.

        Linda

        -----Original Message-----
        From: [email-masked]<mailto:[email-masked]>
        <mailto:[email-masked]<mailto:[email-masked]>>
        [mailto:[email-masked]<mailto:[email-masked]>
        <mailto:[email-masked]<mailto:[email-masked]>>]
        Sent: Wednesday, April 27, 2016 12:40 AM
        To: Joel M. Halpern; Linda Dunbar; Elzur, Uri; [email-masked]<mailto:[email-masked]>
        <mailto:[email-masked]<mailto:[email-masked]>>
        Subject: RE: [sfc] WG last call for draft-ietf-sfc-nsh-04.txt

        Hi Joel, all,

        Please see inline.

        Cheers,
        Med

            -----Message d'origine-----

        [Med] I agree with you.

        [Med] Fully agree.

            Yours,
            Joel

                Joel,

                Cheers,

                Linda

                Yours, Joel

                    Agree with Med.

                    Linda

                    Re-,

                    Thank you.

                    Cheers, Med

                        Hi Med

                        Thx

                        Hi Joel,

                        Please see inline.

                        Cheers, Med

                        It just does its job.

                            Yours, Joel

                            On 4/25/16 3:00 AM,

                                Re-,

                            this section.

                                Cheers, Med

                                    Hi Med

                            that

                            doesn't

                            relate

                                    to it.

                                    Thx

                                    Hi Uri,

                            afraid

                            that

                            it

                            because

                            fragments,

                                    etc.

                            to

                                    be fixed somehow.

                                    Cheers, Med

                                        Hi Med

                                        Thx

                                        R-,

                            the

                                        1.  Jumbo Frames, when

                                        2.  Path MTU Discovery

                            an

                        the

                            assembly

                                        in section 5.4. ==

                                        * The text is weak for a

                        2.12.

                        implementation.

                                        * The text covers only

                                        In particular, dedicated

                            is

                                    no issue.

                        configuration.

                                        * Bullet 2, I would drop

                        (stateless).

                                    domain?

                                        Cheers, Med

                                            -----Message d'origine-----
                                            De : sfc

                                            Dear Thomas, all,

                                            Cheers, Med

                                                -----Message
                                                d'origine----- De : sfc

                                                Dear WG:

                                                This note begins a WG
                                                last call on

                                                addressed all known
                                                comments and that there
                                                are no open
                                                issues with the current
                                                version of the document.

                                                Substantive comments to
                                                the list please,
                                                editorial comments
                                                can go directly to the
                                                document editors.

                                                We'll also get a brief
                                                update from the editors
                                                at next
                                                week's meeting. If there
                                                are any remaining issues
                                                with the
                                                document, raising them

                                                For the chairs, Thomas